2024-04-25T10:56:20.925 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T10:56:20.926 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/client-limits} --owner scheduled_pdonnell@teuthology --verbose --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113 --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113/orig.config.yaml 2024-04-25T10:56:20.974 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:56:21.055 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/client-limits} email: pdonnell@redhat.com first_in_suite: false job_id: '7673113' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - responding to mclientcaps\(revoke\) - not advance its oldest_client_tid - failing to advance its oldest client/flush tid - Too many inodes in cache - failing to respond to cache pressure - slow requests are blocked - failing to respond to capability release - MDS cache is too large - \(MDS_CLIENT_OLDEST_TID\) - \(MDS_CACHE_OVERSIZED\) sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBF9DBbDueiRm4eDdN0mrzKTJprD7x+i5FZv0vs3HthC+LUltS7s7AUgunI7ikqP83+TEI+YWv3AyIG6ogCgVHQ= smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNeDhjrKhWH3UftX4hlFg2IblmuJy9LaZzw/IPWE2EoQR6d7VNAsXTvAtNKFmQsC1IYgilb3kTOBH94c524Blrg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_client_limits teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T10:56:21.056 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T10:56:21.056 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T10:56:21.056 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T10:56:21.058 INFO:teuthology.task.internal:Checking packages... 2024-04-25T10:56:21.082 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T10:56:21.082 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T10:56:21.083 INFO:teuthology.packaging:ref: None 2024-04-25T10:56:21.083 INFO:teuthology.packaging:tag: None 2024-04-25T10:56:21.083 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:56:21.083 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T10:56:21.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T10:56:21.305 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T10:56:21.307 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T10:56:21.312 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T10:56:21.313 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T10:56:21.355 INFO:teuthology.task.internal:Saving configuration 2024-04-25T10:56:21.372 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T10:56:21.446 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T10:56:21.469 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:51:17.853111', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBF9DBbDueiRm4eDdN0mrzKTJprD7x+i5FZv0vs3HthC+LUltS7s7AUgunI7ikqP83+TEI+YWv3AyIG6ogCgVHQ='} 2024-04-25T10:56:21.489 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 10:51:17.854301', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNeDhjrKhWH3UftX4hlFg2IblmuJy9LaZzw/IPWE2EoQR6d7VNAsXTvAtNKFmQsC1IYgilb3kTOBH94c524Blrg='} 2024-04-25T10:56:21.489 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T10:56:21.501 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T10:56:21.501 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T10:56:21.502 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T10:56:21.607 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcd3f628f70>, signals=[15]) 2024-04-25T10:56:21.608 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T10:56:21.613 INFO:teuthology.task.internal:Opening connections... 2024-04-25T10:56:21.613 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T10:56:21.616 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:56:21.701 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T10:56:21.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:56:21.776 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T10:56:21.782 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-04-25T10:56:21.800 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-04-25T10:56:21.800 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-04-25T10:56:21.858 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-04-25T10:56:21.858 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-04-25T10:56:21.858 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-04-25T10:56:21.858 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:56:21.858 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-04-25T10:56:21.858 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:56:21.859 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:56:21.859 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-04-25T10:56:21.859 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:56:21.859 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:56:21.859 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:56:21.859 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:56:21.859 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:56:21.859 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:56:21.860 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-04-25T10:56:21.886 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2024-04-25T10:56:21.903 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2024-04-25T10:56:21.903 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="9" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="9" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el9" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:LOGO="fedora-logo-icon" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T10:56:21.960 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T10:56:21.961 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T10:56:21.961 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2024-04-25T10:56:21.985 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T10:56:21.994 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T10:56:22.035 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T10:56:22.035 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:56:22.038 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2024-04-25T10:56:22.054 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T10:56:22.106 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T10:56:22.107 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:56:22.109 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2024-04-25T10:56:22.193 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T10:56:22.245 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T10:56:22.260 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T10:56:22.261 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-25T10:56:22.261 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-25T10:56:22.261 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:56:22.262 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T10:56:22.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:56:22.279 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T10:56:22.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:56:22.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T10:56:22.464 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:56:22.464 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T10:56:22.464 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-04-25T10:56:22.482 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-437.el9.x86_64 2024-04-25T10:56:22.482 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:56:22.482 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T10:56:22.483 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum upgrade -y linux-firmware 2024-04-25T10:56:22.532 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:56:22.533 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T10:56:22.533 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-04-25T10:56:22.550 INFO:teuthology.orchestra.run.smithi087.stdout:5.14.0-437.el9.x86_64 2024-04-25T10:56:22.550 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:56:22.551 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T10:56:22.551 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum upgrade -y linux-firmware 2024-04-25T10:56:24.833 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T10:56:24.934 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.0 MB 00:00 2024-04-25T10:56:26.701 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-25T10:56:26.900 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-04-25T10:56:31.068 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T10:56:31.226 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.2 MB 00:00 2024-04-25T10:56:32.677 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-25T10:56:32.877 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-04-25T10:56:34.195 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T10:56:34.287 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T10:56:39.320 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-25T10:56:39.412 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T10:56:41.401 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-04-25T10:56:41.402 INFO:teuthology.orchestra.run.smithi087.stdout:Nothing to do. 2024-04-25T10:56:41.402 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-04-25T10:56:41.477 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-04-25T10:56:41.478 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-04-25T10:56:41.479 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-04-25T10:56:41.520 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T10:56:41.521 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:56:41.598 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T10:56:41.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T10:56:41.674 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T10:56:41.674 DEBUG:teuthology.orchestra.run.smithi087:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T10:56:41.676 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi087.front.sepia.ceph.com to complete... 2024-04-25T10:56:41.737 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T10:56:41.737 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T10:56:41.740 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi007.front.sepia.ceph.com to complete... 2024-04-25T10:56:57.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25 10:56:57 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T10:56:57.681 INFO:teuthology.orchestra.run.smithi087.stderr:FINISHED --2024-04-25 10:56:57-- 2024-04-25T10:56:57.681 INFO:teuthology.orchestra.run.smithi087.stderr:Total wall clock time: 16s 2024-04-25T10:56:57.681 INFO:teuthology.orchestra.run.smithi087.stderr:Downloaded: 1 files, 341M in 16s (21.8 MB/s) 2024-04-25T10:56:57.684 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T10:56:57.684 DEBUG:teuthology.orchestra.run.smithi087:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T10:56:59.778 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying... ######################################## 2024-04-25T10:56:59.922 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing... ######################################## 2024-04-25T10:57:00.053 INFO:teuthology.orchestra.run.smithi087.stdout:Updating / installing... 2024-04-25T10:57:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25 10:57:06 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T10:57:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:FINISHED --2024-04-25 10:57:06-- 2024-04-25T10:57:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:Total wall clock time: 25s 2024-04-25T10:57:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:Downloaded: 1 files, 341M in 24s (14.0 MB/s) 2024-04-25T10:57:06.429 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T10:57:06.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T10:57:08.091 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T10:57:08.623 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying... ######################################## 2024-04-25T10:57:08.875 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing... ######################################## 2024-04-25T10:57:08.993 INFO:teuthology.orchestra.run.smithi007.stdout:Updating / installing... 2024-04-25T10:57:17.339 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T10:57:47.027 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi087.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T10:57:47.027 DEBUG:teuthology.orchestra.run.smithi087:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T10:57:47.102 INFO:teuthology.orchestra.run.smithi087.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:47.102 INFO:teuthology.orchestra.run.smithi087.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:47.102 INFO:teuthology.orchestra.run.smithi087.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:47.102 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T10:57:47.102 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T10:57:47.102 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T10:57:47.102 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T10:57:47.102 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T10:57:47.103 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T10:57:47.104 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T10:57:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T10:57:47.106 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T10:57:47.107 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T10:57:47.108 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T10:57:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T10:57:47.110 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T10:57:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T10:57:47.112 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T10:57:47.113 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T10:57:47.114 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T10:57:47.115 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T10:57:47.116 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T10:57:47.117 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T10:57:47.118 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T10:57:47.119 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T10:57:47.120 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T10:57:47.121 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T10:57:47.122 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T10:57:47.123 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T10:57:47.124 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T10:57:47.125 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T10:57:47.126 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T10:57:47.127 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T10:57:47.128 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T10:57:47.129 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T10:57:47.130 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T10:57:47.131 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T10:57:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T10:57:47.133 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T10:57:47.134 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T10:57:47.135 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T10:57:47.136 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T10:57:47.137 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T10:57:47.138 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T10:57:47.139 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T10:57:47.140 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T10:57:47.141 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T10:57:47.142 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T10:57:47.143 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T10:57:47.144 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T10:57:47.145 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T10:57:47.146 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T10:57:47.147 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T10:57:47.148 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T10:57:47.149 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T10:57:47.150 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T10:57:47.151 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T10:57:47.152 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T10:57:47.153 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T10:57:47.154 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T10:57:47.155 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T10:57:47.156 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T10:57:47.157 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T10:57:47.158 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T10:57:47.159 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T10:57:47.160 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T10:57:47.161 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T10:57:47.162 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T10:57:47.163 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T10:57:47.164 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T10:57:47.165 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T10:57:47.166 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T10:57:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T10:57:47.168 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T10:57:47.169 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T10:57:47.170 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T10:57:47.171 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T10:57:47.172 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T10:57:47.173 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T10:57:47.174 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T10:57:47.175 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T10:57:47.176 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T10:57:47.177 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T10:57:47.178 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T10:57:47.179 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T10:57:47.180 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T10:57:47.181 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T10:57:47.182 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T10:57:47.183 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T10:57:47.184 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T10:57:47.185 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T10:57:47.186 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T10:57:47.187 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T10:57:47.188 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T10:57:47.188 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T10:57:47.188 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T10:57:47.188 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T10:57:47.188 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T10:57:47.188 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T10:57:47.188 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T10:57:47.188 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T10:57:47.188 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T10:57:47.189 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T10:57:47.189 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T10:57:47.189 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T10:57:47.189 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T10:57:47.189 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T10:57:47.189 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T10:57:47.189 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T10:57:47.189 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T10:57:47.190 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T10:57:47.191 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T10:57:47.192 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T10:57:47.193 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T10:57:47.194 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T10:57:47.195 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T10:57:47.196 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T10:57:47.197 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T10:57:47.198 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T10:57:47.199 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T10:57:47.200 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T10:57:47.201 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T10:57:47.202 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T10:57:47.203 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T10:57:47.204 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T10:57:47.204 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T10:57:47.204 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T10:57:47.204 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T10:57:47.204 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T10:57:47.204 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T10:57:47.204 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T10:57:47.204 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T10:57:47.204 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T10:57:47.204 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T10:57:47.204 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T10:57:47.205 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T10:57:47.206 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T10:57:47.207 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T10:57:47.208 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T10:57:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T10:57:47.210 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T10:57:47.211 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T10:57:47.212 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T10:57:47.213 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T10:57:47.214 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T10:57:47.215 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T10:57:47.216 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T10:57:47.217 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T10:57:47.218 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T10:57:47.219 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T10:57:47.220 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T10:57:47.221 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T10:57:47.222 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T10:57:47.223 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T10:57:47.224 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T10:57:47.225 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T10:57:47.226 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T10:57:47.226 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T10:57:47.226 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T10:57:47.226 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T10:57:47.226 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T10:57:47.226 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T10:57:47.226 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T10:57:47.226 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T10:57:47.226 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T10:57:47.226 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T10:57:47.226 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T10:57:47.227 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T10:57:47.228 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T10:57:47.229 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T10:57:47.230 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T10:57:47.231 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T10:57:47.232 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T10:57:47.233 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T10:57:47.234 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T10:57:47.235 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T10:57:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T10:57:47.237 INFO:teuthology.orchestra.run.smithi087.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T10:57:47.242 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:47.242 DEBUG:teuthology.orchestra.run.smithi087:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T10:57:47.265 INFO:teuthology.orchestra.run.smithi087.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T10:57:47.265 INFO:teuthology.orchestra.run.smithi087.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout:fi 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout:for file in vmlinuz System.map config; do 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout: fi 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout:done 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout:fi 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T10:57:47.266 INFO:teuthology.orchestra.run.smithi087.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:47.267 INFO:teuthology.orchestra.run.smithi087.stdout:fi 2024-04-25T10:57:47.267 INFO:teuthology.orchestra.run.smithi087.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T10:57:47.267 INFO:teuthology.orchestra.run.smithi087.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T10:57:47.267 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:47.267 INFO:teuthology.orchestra.run.smithi087.stdout:fi 2024-04-25T10:57:47.267 DEBUG:teuthology.orchestra.run.smithi087:> sudo rpm -qi grub2-tools 2024-04-25T10:57:47.337 INFO:teuthology.orchestra.run.smithi087.stdout:Name : grub2-tools 2024-04-25T10:57:47.337 INFO:teuthology.orchestra.run.smithi087.stdout:Epoch : 1 2024-04-25T10:57:47.337 INFO:teuthology.orchestra.run.smithi087.stdout:Version : 2.06 2024-04-25T10:57:47.337 INFO:teuthology.orchestra.run.smithi087.stdout:Release : 68.el9 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: x86_64 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Group : Unspecified 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Size : 8264254 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:License : GPLv3+ 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Packager : builder@centos.org 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Vendor : CentOS 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Summary : Support tools for GRUB. 2024-04-25T10:57:47.338 INFO:teuthology.orchestra.run.smithi087.stdout:Description : 2024-04-25T10:57:47.339 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:57:47.339 INFO:teuthology.orchestra.run.smithi087.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T10:57:47.339 INFO:teuthology.orchestra.run.smithi087.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T10:57:47.339 INFO:teuthology.orchestra.run.smithi087.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T10:57:47.339 INFO:teuthology.orchestra.run.smithi087.stdout:hardware devices. 2024-04-25T10:57:47.339 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T10:57:47.339 INFO:teuthology.orchestra.run.smithi087.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T10:57:47.340 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T10:57:47.340 INFO:teuthology.task.kernel:Updating grub on smithi087 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:47.340 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T10:57:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2024-04-25T10:57:49.264 INFO:teuthology.orchestra.run.smithi087.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T10:57:49.294 INFO:teuthology.orchestra.run.smithi087.stderr:done 2024-04-25T10:57:49.297 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-04-25T10:57:49.314 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.oX1mgQyj1n 2024-04-25T10:57:49.314 DEBUG:teuthology.orchestra.run.smithi087:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.oX1mgQyj1n 2024-04-25T10:57:49.380 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.oX1mgQyj1n 2024-04-25T10:57:49.523 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.oX1mgQyj1n is 6KB 2024-04-25T10:57:49.576 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.oX1mgQyj1n 2024-04-25T10:57:49.590 DEBUG:teuthology.orchestra.run.smithi087:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T10:57:49.658 INFO:teuthology.orchestra.run.smithi087.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T10:57:49.658 INFO:teuthology.orchestra.run.smithi087.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T10:57:49.658 INFO:teuthology.orchestra.run.smithi087.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T10:57:49.659 INFO:teuthology.orchestra.run.smithi087.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T10:57:49.660 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:49.810 DEBUG:teuthology.orchestra.run.smithi087:> sudo shutdown -r now 2024-04-25T10:57:55.597 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T10:57:55.597 DEBUG:teuthology.orchestra.run.smithi007:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T10:57:55.684 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:55.684 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:55.684 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:55.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T10:57:55.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T10:57:55.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T10:57:55.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T10:57:55.684 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T10:57:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T10:57:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T10:57:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T10:57:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T10:57:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T10:57:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T10:57:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T10:57:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T10:57:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T10:57:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T10:57:55.685 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T10:57:55.686 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T10:57:55.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T10:57:55.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T10:57:55.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T10:57:55.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T10:57:55.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T10:57:55.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T10:57:55.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T10:57:55.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T10:57:55.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T10:57:55.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T10:57:55.687 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T10:57:55.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T10:57:55.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T10:57:55.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T10:57:55.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T10:57:55.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T10:57:55.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T10:57:55.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T10:57:55.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T10:57:55.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T10:57:55.688 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T10:57:55.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T10:57:55.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T10:57:55.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T10:57:55.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T10:57:55.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T10:57:55.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T10:57:55.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T10:57:55.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T10:57:55.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T10:57:55.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T10:57:55.689 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T10:57:55.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T10:57:55.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T10:57:55.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T10:57:55.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T10:57:55.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T10:57:55.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T10:57:55.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T10:57:55.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T10:57:55.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T10:57:55.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T10:57:55.690 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T10:57:55.691 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T10:57:55.692 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T10:57:55.693 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T10:57:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T10:57:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T10:57:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T10:57:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T10:57:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T10:57:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T10:57:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T10:57:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T10:57:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T10:57:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T10:57:55.694 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T10:57:55.695 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T10:57:55.696 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T10:57:55.697 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T10:57:55.698 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T10:57:55.699 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T10:57:55.700 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T10:57:55.701 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T10:57:55.702 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T10:57:55.703 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T10:57:55.704 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T10:57:55.705 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T10:57:55.706 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T10:57:55.707 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T10:57:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T10:57:55.709 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T10:57:55.710 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T10:57:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T10:57:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T10:57:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T10:57:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T10:57:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T10:57:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T10:57:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T10:57:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T10:57:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T10:57:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T10:57:55.711 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T10:57:55.712 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T10:57:55.713 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T10:57:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T10:57:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T10:57:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T10:57:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T10:57:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T10:57:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T10:57:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T10:57:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T10:57:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T10:57:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T10:57:55.714 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T10:57:55.715 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T10:57:55.716 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T10:57:55.717 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T10:57:55.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T10:57:55.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T10:57:55.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T10:57:55.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T10:57:55.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T10:57:55.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T10:57:55.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T10:57:55.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T10:57:55.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T10:57:55.718 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T10:57:55.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T10:57:55.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T10:57:55.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T10:57:55.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T10:57:55.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T10:57:55.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T10:57:55.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T10:57:55.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T10:57:55.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T10:57:55.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T10:57:55.719 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T10:57:55.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T10:57:55.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T10:57:55.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T10:57:55.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T10:57:55.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T10:57:55.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T10:57:55.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T10:57:55.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T10:57:55.720 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T10:57:55.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T10:57:55.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T10:57:55.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T10:57:55.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T10:57:55.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T10:57:55.721 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T10:57:55.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T10:57:55.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T10:57:55.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T10:57:55.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T10:57:55.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T10:57:55.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T10:57:55.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T10:57:55.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T10:57:55.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T10:57:55.722 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T10:57:55.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T10:57:55.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T10:57:55.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T10:57:55.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T10:57:55.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T10:57:55.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T10:57:55.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T10:57:55.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T10:57:55.723 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T10:57:55.724 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T10:57:55.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T10:57:55.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T10:57:55.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T10:57:55.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T10:57:55.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T10:57:55.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T10:57:55.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T10:57:55.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T10:57:55.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T10:57:55.725 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T10:57:55.726 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T10:57:55.727 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T10:57:55.728 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T10:57:55.729 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T10:57:55.730 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T10:57:55.731 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T10:57:55.732 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T10:57:55.733 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T10:57:55.734 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T10:57:55.735 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T10:57:55.736 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T10:57:55.737 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T10:57:55.738 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T10:57:55.739 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T10:57:55.740 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T10:57:55.741 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T10:57:55.742 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T10:57:55.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T10:57:55.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T10:57:55.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T10:57:55.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T10:57:55.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T10:57:55.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T10:57:55.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T10:57:55.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T10:57:55.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T10:57:55.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T10:57:55.743 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T10:57:55.744 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T10:57:55.745 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T10:57:55.746 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T10:57:55.747 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T10:57:55.748 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T10:57:55.749 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T10:57:55.750 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T10:57:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T10:57:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T10:57:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T10:57:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T10:57:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T10:57:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T10:57:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T10:57:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T10:57:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T10:57:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T10:57:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T10:57:55.752 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T10:57:55.753 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T10:57:55.754 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T10:57:55.755 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T10:57:55.756 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T10:57:55.757 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T10:57:55.758 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T10:57:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T10:57:55.760 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T10:57:55.761 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T10:57:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T10:57:55.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T10:57:55.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T10:57:55.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T10:57:55.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T10:57:55.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T10:57:55.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T10:57:55.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T10:57:55.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T10:57:55.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T10:57:55.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T10:57:55.763 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T10:57:55.764 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T10:57:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T10:57:55.766 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T10:57:55.767 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T10:57:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T10:57:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T10:57:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T10:57:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T10:57:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T10:57:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T10:57:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T10:57:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T10:57:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T10:57:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T10:57:55.768 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T10:57:55.769 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T10:57:55.770 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T10:57:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T10:57:55.772 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T10:57:55.773 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T10:57:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T10:57:55.775 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T10:57:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T10:57:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T10:57:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T10:57:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T10:57:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T10:57:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T10:57:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T10:57:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T10:57:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T10:57:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T10:57:55.776 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T10:57:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T10:57:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T10:57:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T10:57:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T10:57:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T10:57:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T10:57:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T10:57:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T10:57:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T10:57:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T10:57:55.777 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T10:57:55.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T10:57:55.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T10:57:55.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T10:57:55.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T10:57:55.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T10:57:55.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T10:57:55.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T10:57:55.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T10:57:55.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T10:57:55.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T10:57:55.778 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T10:57:55.779 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T10:57:55.780 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T10:57:55.781 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T10:57:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T10:57:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T10:57:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T10:57:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T10:57:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T10:57:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T10:57:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T10:57:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T10:57:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T10:57:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T10:57:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T10:57:55.783 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T10:57:55.784 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T10:57:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T10:57:55.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T10:57:55.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T10:57:55.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T10:57:55.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T10:57:55.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T10:57:55.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T10:57:55.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T10:57:55.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T10:57:55.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T10:57:55.786 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T10:57:55.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T10:57:55.789 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T10:57:55.790 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T10:57:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T10:57:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T10:57:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T10:57:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T10:57:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T10:57:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T10:57:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T10:57:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T10:57:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T10:57:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T10:57:55.791 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T10:57:55.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T10:57:55.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T10:57:55.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T10:57:55.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T10:57:55.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T10:57:55.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T10:57:55.792 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T10:57:55.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T10:57:55.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T10:57:55.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T10:57:55.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T10:57:55.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T10:57:55.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T10:57:55.793 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T10:57:55.794 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T10:57:55.795 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T10:57:55.796 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T10:57:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T10:57:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T10:57:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T10:57:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T10:57:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T10:57:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T10:57:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T10:57:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T10:57:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T10:57:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T10:57:55.798 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T10:57:55.799 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T10:57:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T10:57:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T10:57:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T10:57:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T10:57:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T10:57:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T10:57:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T10:57:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T10:57:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T10:57:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T10:57:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T10:57:55.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T10:57:55.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T10:57:55.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T10:57:55.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T10:57:55.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T10:57:55.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T10:57:55.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T10:57:55.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T10:57:55.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T10:57:55.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T10:57:55.801 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T10:57:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T10:57:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T10:57:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T10:57:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T10:57:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T10:57:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T10:57:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T10:57:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T10:57:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T10:57:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T10:57:55.802 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T10:57:55.803 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T10:57:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T10:57:55.805 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T10:57:55.806 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T10:57:55.807 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T10:57:55.808 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T10:57:55.809 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T10:57:55.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T10:57:55.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T10:57:55.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T10:57:55.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T10:57:55.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T10:57:55.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T10:57:55.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T10:57:55.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T10:57:55.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T10:57:55.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T10:57:55.810 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T10:57:55.811 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T10:57:55.812 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T10:57:55.813 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T10:57:55.814 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T10:57:55.815 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T10:57:55.816 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T10:57:55.817 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T10:57:55.818 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T10:57:55.819 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T10:57:55.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T10:57:55.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T10:57:55.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T10:57:55.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T10:57:55.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T10:57:55.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T10:57:55.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T10:57:55.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T10:57:55.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T10:57:55.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T10:57:55.820 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T10:57:55.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T10:57:55.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T10:57:55.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T10:57:55.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T10:57:55.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T10:57:55.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T10:57:55.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T10:57:55.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T10:57:55.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T10:57:55.821 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T10:57:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T10:57:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T10:57:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T10:57:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T10:57:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T10:57:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T10:57:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T10:57:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T10:57:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T10:57:55.822 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T10:57:55.823 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T10:57:55.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T10:57:55.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T10:57:55.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T10:57:55.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T10:57:55.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T10:57:55.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T10:57:55.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T10:57:55.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T10:57:55.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T10:57:55.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T10:57:55.824 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T10:57:55.825 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T10:57:55.826 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T10:57:55.827 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T10:57:55.828 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T10:57:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T10:57:55.830 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T10:57:55.831 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T10:57:55.832 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T10:57:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T10:57:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T10:57:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T10:57:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T10:57:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T10:57:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T10:57:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T10:57:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T10:57:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T10:57:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T10:57:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T10:57:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T10:57:55.835 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T10:57:55.836 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T10:57:55.837 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T10:57:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T10:57:55.839 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T10:57:55.840 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T10:57:55.841 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T10:57:55.842 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T10:57:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T10:57:55.844 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T10:57:55.845 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T10:57:55.846 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T10:57:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T10:57:55.848 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T10:57:55.848 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T10:57:55.852 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:55.852 DEBUG:teuthology.orchestra.run.smithi007:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T10:57:55.876 INFO:teuthology.orchestra.run.smithi007.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T10:57:55.876 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T10:57:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T10:57:55.876 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-04-25T10:57:55.876 INFO:teuthology.orchestra.run.smithi007.stdout:for file in vmlinuz System.map config; do 2024-04-25T10:57:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T10:57:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T10:57:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: fi 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout:done 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T10:57:55.877 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:55.878 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-04-25T10:57:55.878 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-04-25T10:57:55.954 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-04-25T10:57:55.954 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-04-25T10:57:55.954 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-04-25T10:57:55.954 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 68.el9 2024-04-25T10:57:55.954 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-04-25T10:57:55.954 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T10:57:55.954 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-04-25T10:57:55.954 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8264254 2024-04-25T10:57:55.954 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-04-25T10:57:55.954 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T10:57:55.955 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T10:57:55.955 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T10:57:55.955 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T10:57:55.955 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-04-25T10:57:55.955 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-04-25T10:57:55.955 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T10:57:55.955 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-04-25T10:57:55.955 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-04-25T10:57:55.955 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T10:57:55.955 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T10:57:55.955 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T10:57:55.956 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T10:57:55.956 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-04-25T10:57:55.956 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T10:57:55.956 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T10:57:55.957 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T10:57:55.957 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:55.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T10:57:56.534 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-04-25T10:57:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T10:57:57.901 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-04-25T10:57:57.903 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-04-25T10:57:57.959 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.8T66K0vATc 2024-04-25T10:57:57.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8T66K0vATc 2024-04-25T10:57:57.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.8T66K0vATc 2024-04-25T10:57:58.124 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.8T66K0vATc is 6KB 2024-04-25T10:57:58.173 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.8T66K0vATc 2024-04-25T10:57:58.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T10:57:58.250 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T10:57:58.251 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T10:57:58.251 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T10:57:58.251 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T10:57:58.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T10:57:58.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-04-25T10:58:19.832 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-25T10:58:19.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T10:58:19.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:58:28.407 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T10:58:28.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-25T10:58:28.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:58:38.238 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-04-25T10:58:46.946 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-04-25T10:58:47.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T10:58:47.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:58:50.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-04-25T10:58:55.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-25T10:58:55.951 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:59:02.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T10:59:02.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T10:59:56.003 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T11:00:02.314 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T11:00:08.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-25T11:00:08.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:00:08.404 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-04-25T11:00:09.039 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-25T11:00:09.039 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T11:00:09.039 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-04-25T11:00:09.093 INFO:teuthology.orchestra.run.smithi007.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T11:00:09.093 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T11:00:09.094 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T11:00:09.094 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T11:00:10.095 DEBUG:teuthology.parallel:result is None 2024-04-25T11:00:17.315 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T11:00:17.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:00:17.743 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-04-25T11:00:18.443 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T11:00:18.443 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T11:00:18.443 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-04-25T11:00:18.492 INFO:teuthology.orchestra.run.smithi087.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T11:00:18.492 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T11:00:18.492 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T11:00:18.492 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T11:00:19.493 DEBUG:teuthology.parallel:result is None 2024-04-25T11:00:19.494 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T11:00:19.537 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T11:00:19.537 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:00:19.546 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T11:00:19.602 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T11:00:19.613 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T11:00:19.690 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T11:00:19.690 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:00:19.698 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T11:00:19.782 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T11:00:19.788 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T11:00:19.789 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:00:19.796 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T11:00:19.920 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:00:19.938 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:00:19.966 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:00:19.998 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T11:00:20.003 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T11:00:20.011 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T11:00:20.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:00:20.019 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T11:00:20.105 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T11:00:20.115 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T11:00:20.115 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:00:20.122 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T11:00:20.175 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:00:20.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:00:20.369 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:00:20.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:00:20.543 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:00:20.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:00:20.633 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:00:20.719 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T11:00:20.829 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:00:20.912 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T11:00:20.999 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:00:20.999 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T11:00:21.083 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-04-25T11:00:21.091 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2024-04-25T11:00:21.201 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:00:21.238 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:00:21.575 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T11:00:21.584 INFO:teuthology.task.internal:Starting timer... 2024-04-25T11:00:21.584 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T11:00:21.627 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T11:00:21.681 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-25T11:00:21.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-04-25T11:00:21.783 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-04-25T11:00:21.788 DEBUG:teuthology.orchestra.run.smithi087:> sudo service auditd rotate 2024-04-25T11:00:21.913 INFO:teuthology.orchestra.run.smithi087.stdout:Rotating logs: 2024-04-25T11:00:21.918 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T11:00:21.918 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-04-25T11:00:21.979 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-04-25T11:00:21.979 DEBUG:teuthology.orchestra.run.smithi087:> /usr/sbin/getenforce 2024-04-25T11:00:22.040 INFO:teuthology.orchestra.run.smithi087.stdout:Permissive 2024-04-25T11:00:22.040 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi087.front.sepia.ceph.com': 'permissive'} 2024-04-25T11:00:22.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:00:22.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:00:22.116 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:00:22.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:00:22.199 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T11:00:22.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:00:22.285 DEBUG:teuthology.orchestra.run.smithi087:> sudo /usr/sbin/setenforce permissive 2024-04-25T11:00:22.361 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T11:00:22.371 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T11:00:22.376 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T11:00:22.546 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T11:00:22.560 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T11:00:22.562 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi087.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T11:05:36.657 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi087.front.sepia.ceph.com')] 2024-04-25T11:05:36.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-25T11:05:36.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:05:36.771 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-04-25T11:05:36.922 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-04-25T11:05:36.923 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T11:05:36.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T11:05:37.037 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-04-25T11:05:37.183 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T11:05:37.183 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T11:05:37.193 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T11:05:37.193 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:05:37.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:05:37.201 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T11:05:37.201 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:05:37.292 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:05:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T11:05:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:05:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T11:05:37.406 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: ntpd: command not found 2024-04-25T11:05:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-04-25T11:05:37.437 INFO:teuthology.orchestra.run.smithi087.stdout:506 Cannot talk to daemon 2024-04-25T11:05:37.452 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-04-25T11:05:37.478 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:05:37.495 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T11:05:37.522 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:05:37.540 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T11:05:37.667 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:05:37.676 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:05:37.676 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-04-25T11:05:37.676 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:05:37.676 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:05:37.676 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:05:37.676 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:05:37.679 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:05:37.689 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:05:37.689 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-04-25T11:05:37.689 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:05:37.690 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:05:37.690 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:05:37.690 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:05:37.691 INFO:teuthology.run_tasks:Running task install... 2024-04-25T11:05:37.699 DEBUG:teuthology.task.install:project ceph 2024-04-25T11:05:37.699 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T11:05:37.699 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T11:05:37.699 INFO:teuthology.task.install:Using flavor: default 2024-04-25T11:05:37.705 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T11:05:37.705 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T11:05:37.705 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T11:05:37.706 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:05:37.708 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T11:05:37.708 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:05:37.939 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T11:05:37.939 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T11:05:37.942 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T11:05:37.942 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T11:05:38.049 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:05:38.049 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:05:38.049 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:05:38.113 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T11:05:38.114 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:05:38.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T11:05:38.137 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:05:38.138 DEBUG:teuthology.orchestra.run.smithi087:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:05:38.207 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T11:05:38.207 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T11:05:38.267 DEBUG:teuthology.orchestra.run.smithi087:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:05:38.339 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T11:05:38.369 DEBUG:teuthology.orchestra.run.smithi087:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:05:38.445 INFO:teuthology.orchestra.run.smithi087.stdout:check_obsoletes = 1 2024-04-25T11:05:38.449 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum clean all 2024-04-25T11:05:38.461 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T11:05:38.542 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-04-25T11:05:38.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-04-25T11:05:38.762 INFO:teuthology.orchestra.run.smithi087.stdout:48 files removed 2024-04-25T11:05:38.800 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:05:38.879 INFO:teuthology.orchestra.run.smithi007.stdout:48 files removed 2024-04-25T11:05:38.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T11:05:39.662 INFO:teuthology.orchestra.run.smithi087.stdout:ceph packages for x86_64 196 kB/s | 84 kB 00:00 2024-04-25T11:05:39.759 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 208 kB/s | 84 kB 00:00 2024-04-25T11:05:40.285 INFO:teuthology.orchestra.run.smithi087.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-04-25T11:05:40.509 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-04-25T11:05:41.160 INFO:teuthology.orchestra.run.smithi087.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2024-04-25T11:05:41.260 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-04-25T11:05:41.850 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T11:05:42.125 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T11:05:44.143 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-25T11:05:49.044 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.2 MB 00:00 2024-04-25T11:05:50.778 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-04-25T11:05:52.821 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T11:05:58.954 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-04-25T11:06:01.447 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:06:01.449 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:06:01.470 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:06:01.471 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:06:01.473 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:06:01.474 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:06:01.477 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:06:01.480 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:06:01.481 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:06:01.555 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-04-25T11:06:01.576 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-04-25T11:06:01.576 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-04-25T11:06:01.576 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-04-25T11:06:01.576 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-04-25T11:06:01.576 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:06:01.576 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T11:06:01.576 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T11:06:01.576 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T11:06:01.576 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T11:06:01.576 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T11:06:01.577 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T11:06:01.578 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:06:01.579 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:06:01.580 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:06:01.581 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:06:01.582 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:06:01.583 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:06:01.584 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:06:01.585 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:06:01.586 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:06:01.587 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-04-25T11:06:01.588 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-04-25T11:06:01.588 INFO:teuthology.orchestra.run.smithi007.stdout:Install 148 Packages 2024-04-25T11:06:01.588 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 4 Packages 2024-04-25T11:06:01.588 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:06:01.588 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 214 M 2024-04-25T11:06:01.588 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-04-25T11:06:03.056 INFO:teuthology.orchestra.run.smithi007.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-04-25T11:06:03.264 INFO:teuthology.orchestra.run.smithi007.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 4.1 MB/s | 863 kB 00:00 2024-04-25T11:06:03.323 INFO:teuthology.orchestra.run.smithi007.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 2.5 MB/s | 147 kB 00:00 2024-04-25T11:06:03.615 INFO:teuthology.orchestra.run.smithi007.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 7.8 MB/s | 2.3 MB 00:00 2024-04-25T11:06:03.823 INFO:teuthology.orchestra.run.smithi007.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 7.3 MB/s | 1.5 MB 00:00 2024-04-25T11:06:04.098 INFO:teuthology.orchestra.run.smithi007.stdout:(6/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 4.1 MB/s | 5.4 MB 00:01 2024-04-25T11:06:04.515 INFO:teuthology.orchestra.run.smithi007.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 7.1 MB/s | 4.9 MB 00:00 2024-04-25T11:06:04.783 INFO:teuthology.orchestra.run.smithi007.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 9.8 MB/s | 20 MB 00:02 2024-04-25T11:06:04.833 INFO:teuthology.orchestra.run.smithi007.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 518 kB/s | 25 kB 00:00 2024-04-25T11:06:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 11 MB/s | 10 MB 00:00 2024-04-25T11:06:05.516 INFO:teuthology.orchestra.run.smithi007.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 3.0 MB/s | 224 kB 00:00 2024-04-25T11:06:05.575 INFO:teuthology.orchestra.run.smithi007.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 469 kB/s | 27 kB 00:00 2024-04-25T11:06:05.634 INFO:teuthology.orchestra.run.smithi007.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 555 kB/s | 32 kB 00:00 2024-04-25T11:06:05.776 INFO:teuthology.orchestra.run.smithi007.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 5.3 MB/s | 756 kB 00:00 2024-04-25T11:06:05.851 INFO:teuthology.orchestra.run.smithi007.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.1 MB/s | 158 kB 00:00 2024-04-25T11:06:05.926 INFO:teuthology.orchestra.run.smithi007.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.7 MB/s | 127 kB 00:00 2024-04-25T11:06:06.010 INFO:teuthology.orchestra.run.smithi007.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 5.9 MB/s | 497 kB 00:00 2024-04-25T11:06:06.241 INFO:teuthology.orchestra.run.smithi007.stdout:(18/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 8.3 MB/s | 18 MB 00:02 2024-04-25T11:06:06.327 INFO:teuthology.orchestra.run.smithi007.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 531 kB/s | 45 kB 00:00 2024-04-25T11:06:06.419 INFO:teuthology.orchestra.run.smithi007.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.5 MB/s | 135 kB 00:00 2024-04-25T11:06:06.528 INFO:teuthology.orchestra.run.smithi007.stdout:(21/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 10 MB/s | 5.2 MB 00:00 2024-04-25T11:06:06.561 INFO:teuthology.orchestra.run.smithi007.stdout:(22/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.1 MB/s | 161 kB 00:00 2024-04-25T11:06:06.611 INFO:teuthology.orchestra.run.smithi007.stdout:(23/152): python3-rados-19.0.0-3291.g994e2f20.e 3.8 MB/s | 321 kB 00:00 2024-04-25T11:06:06.653 INFO:teuthology.orchestra.run.smithi007.stdout:(24/152): python3-rbd-19.0.0-3291.g994e2f20.el9 3.2 MB/s | 298 kB 00:00 2024-04-25T11:06:06.687 INFO:teuthology.orchestra.run.smithi007.stdout:(25/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.3 MB/s | 100 kB 00:00 2024-04-25T11:06:06.729 INFO:teuthology.orchestra.run.smithi007.stdout:(26/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 1.1 MB/s | 86 kB 00:00 2024-04-25T11:06:06.821 INFO:teuthology.orchestra.run.smithi007.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.8 MB/s | 169 kB 00:00 2024-04-25T11:06:06.913 INFO:teuthology.orchestra.run.smithi007.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 376 kB/s | 34 kB 00:00 2024-04-25T11:06:06.972 INFO:teuthology.orchestra.run.smithi007.stdout:(29/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 11 MB/s | 3.0 MB 00:00 2024-04-25T11:06:06.997 INFO:teuthology.orchestra.run.smithi007.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.7 MB/s | 142 kB 00:00 2024-04-25T11:06:07.381 INFO:teuthology.orchestra.run.smithi007.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 8.7 MB/s | 3.5 MB 00:00 2024-04-25T11:06:07.490 INFO:teuthology.orchestra.run.smithi007.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 3.0 MB/s | 249 kB 00:00 2024-04-25T11:06:07.507 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 797 kB/s | 19 MB 00:24 2024-04-25T11:06:07.531 INFO:teuthology.orchestra.run.smithi007.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 752 kB/s | 49 kB 00:00 2024-04-25T11:06:07.607 INFO:teuthology.orchestra.run.smithi007.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 222 kB/s | 16 kB 00:00 2024-04-25T11:06:07.707 INFO:teuthology.orchestra.run.smithi007.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 2.7 MB/s | 273 kB 00:00 2024-04-25T11:06:07.807 INFO:teuthology.orchestra.run.smithi007.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 7.5 MB/s | 762 kB 00:00 2024-04-25T11:06:07.941 INFO:teuthology.orchestra.run.smithi007.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 7.8 MB/s | 7.4 MB 00:00 2024-04-25T11:06:07.958 INFO:teuthology.orchestra.run.smithi007.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 122 kB/s | 18 kB 00:00 2024-04-25T11:06:08.066 INFO:teuthology.orchestra.run.smithi007.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 648 kB/s | 80 kB 00:00 2024-04-25T11:06:08.567 INFO:teuthology.orchestra.run.smithi007.stdout:(40/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 12 MB/s | 43 MB 00:03 2024-04-25T11:06:08.600 INFO:teuthology.orchestra.run.smithi007.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 367 kB/s | 195 kB 00:00 2024-04-25T11:06:08.634 INFO:teuthology.orchestra.run.smithi007.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-04-25T11:06:08.684 INFO:teuthology.orchestra.run.smithi007.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-25T11:06:08.734 INFO:teuthology.orchestra.run.smithi007.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-25T11:06:08.768 INFO:teuthology.orchestra.run.smithi007.stdout:(45/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-04-25T11:06:08.793 INFO:teuthology.orchestra.run.smithi007.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-25T11:06:08.835 INFO:teuthology.orchestra.run.smithi007.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-04-25T11:06:08.860 INFO:teuthology.orchestra.run.smithi007.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-25T11:06:09.294 INFO:teuthology.orchestra.run.smithi007.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 247 kB/s | 107 kB 00:00 2024-04-25T11:06:09.553 INFO:teuthology.orchestra.run.smithi007.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 83 kB/s | 21 kB 00:00 2024-04-25T11:06:09.903 INFO:teuthology.orchestra.run.smithi007.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 621 kB/s | 689 kB 00:01 2024-04-25T11:06:10.153 INFO:teuthology.orchestra.run.smithi007.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 523 kB/s | 313 kB 00:00 2024-04-25T11:06:10.178 INFO:teuthology.orchestra.run.smithi007.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 119 kB/s | 33 kB 00:00 2024-04-25T11:06:10.223 INFO:teuthology.orchestra.run.smithi007.stdout:(54/152): bison-3.7.4-5.el9.x86_64.rpm 680 kB/s | 943 kB 00:01 2024-04-25T11:06:10.415 INFO:teuthology.orchestra.run.smithi007.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 58 kB/s | 11 kB 00:00 2024-04-25T11:06:10.440 INFO:teuthology.orchestra.run.smithi007.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 70 kB/s | 18 kB 00:00 2024-04-25T11:06:10.782 INFO:teuthology.orchestra.run.smithi007.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 133 kB/s | 45 kB 00:00 2024-04-25T11:06:10.932 INFO:teuthology.orchestra.run.smithi007.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 310 kB/s | 160 kB 00:00 2024-04-25T11:06:11.516 INFO:teuthology.orchestra.run.smithi007.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 1.0 MB/s | 582 kB 00:00 2024-04-25T11:06:11.757 INFO:teuthology.orchestra.run.smithi007.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 680 kB/s | 662 kB 00:00 2024-04-25T11:06:11.774 INFO:teuthology.orchestra.run.smithi007.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 99 kB/s | 25 kB 00:00 2024-04-25T11:06:12.091 INFO:teuthology.orchestra.run.smithi007.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 143 kB/s | 47 kB 00:00 2024-04-25T11:06:12.499 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.2 MB 00:01 2024-04-25T11:06:12.658 INFO:teuthology.orchestra.run.smithi007.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 333 kB/s | 188 kB 00:00 2024-04-25T11:06:12.692 INFO:teuthology.orchestra.run.smithi007.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 319 kB/s | 292 kB 00:00 2024-04-25T11:06:12.917 INFO:teuthology.orchestra.run.smithi007.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 136 kB/s | 35 kB 00:00 2024-04-25T11:06:13.534 INFO:teuthology.orchestra.run.smithi007.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 906 kB/s | 3.0 MB 00:03 2024-04-25T11:06:14.984 INFO:teuthology.orchestra.run.smithi007.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 1.1 MB/s | 1.6 MB 00:01 2024-04-25T11:06:15.710 INFO:teuthology.orchestra.run.smithi007.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 1.6 MB/s | 4.7 MB 00:03 2024-04-25T11:06:15.760 INFO:teuthology.orchestra.run.smithi007.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 1.1 MB/s | 911 kB 00:00 2024-04-25T11:06:16.135 INFO:teuthology.orchestra.run.smithi007.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 314 kB/s | 133 kB 00:00 2024-04-25T11:06:16.877 INFO:teuthology.orchestra.run.smithi007.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.1 MB/s | 4.4 MB 00:03 2024-04-25T11:06:17.102 INFO:teuthology.orchestra.run.smithi007.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-04-25T11:06:17.344 INFO:teuthology.orchestra.run.smithi007.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 188 kB/s | 87 kB 00:00 2024-04-25T11:06:17.853 INFO:teuthology.orchestra.run.smithi007.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 481 kB/s | 244 kB 00:00 2024-04-25T11:06:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 420 kB/s | 248 kB 00:00 2024-04-25T11:06:18.753 INFO:teuthology.orchestra.run.smithi007.stdout:(76/152): postgresql-server-13.14-1.el9.x86_64. 1.9 MB/s | 5.8 MB 00:02 2024-04-25T11:06:18.778 INFO:teuthology.orchestra.run.smithi007.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 77 kB/s | 26 kB 00:00 2024-04-25T11:06:19.020 INFO:teuthology.orchestra.run.smithi007.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 72 kB/s | 19 kB 00:00 2024-04-25T11:06:19.145 INFO:teuthology.orchestra.run.smithi007.stdout:(79/152): python3-babel-2.9.1-2.el9.noarch.rpm 2.9 MB/s | 6.0 MB 00:02 2024-04-25T11:06:19.262 INFO:teuthology.orchestra.run.smithi007.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 356 kB/s | 172 kB 00:00 2024-04-25T11:06:19.329 INFO:teuthology.orchestra.run.smithi007.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 113 kB/s | 35 kB 00:00 2024-04-25T11:06:19.788 INFO:teuthology.orchestra.run.smithi007.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 477 kB/s | 250 kB 00:00 2024-04-25T11:06:20.055 INFO:teuthology.orchestra.run.smithi007.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 306 kB/s | 222 kB 00:00 2024-04-25T11:06:20.214 INFO:teuthology.orchestra.run.smithi007.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 182 kB/s | 77 kB 00:00 2024-04-25T11:06:20.397 INFO:teuthology.orchestra.run.smithi007.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 124 kB/s | 42 kB 00:00 2024-04-25T11:06:20.722 INFO:teuthology.orchestra.run.smithi007.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 530 kB/s | 269 kB 00:00 2024-04-25T11:06:20.906 INFO:teuthology.orchestra.run.smithi007.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 313 kB/s | 159 kB 00:00 2024-04-25T11:06:21.173 INFO:teuthology.orchestra.run.smithi007.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.5 MB/s | 5.1 MB 00:02 2024-04-25T11:06:21.248 INFO:teuthology.orchestra.run.smithi007.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 149 kB/s | 51 kB 00:00 2024-04-25T11:06:21.290 INFO:teuthology.orchestra.run.smithi007.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 492 kB/s | 279 kB 00:00 2024-04-25T11:06:21.515 INFO:teuthology.orchestra.run.smithi007.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 157 kB/s | 54 kB 00:00 2024-04-25T11:06:21.557 INFO:teuthology.orchestra.run.smithi007.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 156 kB/s | 42 kB 00:00 2024-04-25T11:06:21.866 INFO:teuthology.orchestra.run.smithi007.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 126 kB/s | 44 kB 00:00 2024-04-25T11:06:22.133 INFO:teuthology.orchestra.run.smithi007.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 135 kB/s | 36 kB 00:00 2024-04-25T11:06:22.275 INFO:teuthology.orchestra.run.smithi007.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 308 kB/s | 221 kB 00:00 2024-04-25T11:06:22.475 INFO:teuthology.orchestra.run.smithi007.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 139 kB/s | 47 kB 00:00 2024-04-25T11:06:22.817 INFO:teuthology.orchestra.run.smithi007.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 169 kB/s | 58 kB 00:00 2024-04-25T11:06:23.101 INFO:teuthology.orchestra.run.smithi007.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 370 kB/s | 305 kB 00:00 2024-04-25T11:06:23.242 INFO:teuthology.orchestra.run.smithi007.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 221 kB/s | 94 kB 00:00 2024-04-25T11:06:23.451 INFO:teuthology.orchestra.run.smithi007.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 182 kB/s | 64 kB 00:00 2024-04-25T11:06:23.585 INFO:teuthology.orchestra.run.smithi007.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 131 kB/s | 45 kB 00:00 2024-04-25T11:06:23.827 INFO:teuthology.orchestra.run.smithi007.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.3 MB/s | 16 MB 00:02 2024-04-25T11:06:23.843 INFO:teuthology.orchestra.run.smithi007.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 57 kB/s | 22 kB 00:00 2024-04-25T11:06:24.086 INFO:teuthology.orchestra.run.smithi007.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-25T11:06:24.119 INFO:teuthology.orchestra.run.smithi007.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-25T11:06:24.144 INFO:teuthology.orchestra.run.smithi007.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 336 kB/s | 19 kB 00:00 2024-04-25T11:06:24.203 INFO:teuthology.orchestra.run.smithi007.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-04-25T11:06:24.262 INFO:teuthology.orchestra.run.smithi007.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 842 kB/s | 49 kB 00:00 2024-04-25T11:06:24.321 INFO:teuthology.orchestra.run.smithi007.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T11:06:24.429 INFO:teuthology.orchestra.run.smithi007.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-04-25T11:06:24.463 INFO:teuthology.orchestra.run.smithi007.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-25T11:06:24.522 INFO:teuthology.orchestra.run.smithi007.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.0 MB/s | 838 kB 00:00 2024-04-25T11:06:24.572 INFO:teuthology.orchestra.run.smithi007.stdout:(113/152): protobuf-compiler-3.14.0-13.el9.x86_ 875 kB/s | 863 kB 00:00 2024-04-25T11:06:24.597 INFO:teuthology.orchestra.run.smithi007.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 585 kB/s | 43 kB 00:00 2024-04-25T11:06:24.631 INFO:teuthology.orchestra.run.smithi007.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-25T11:06:24.769 INFO:teuthology.orchestra.run.smithi007.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 88 kB/s | 15 kB 00:00 2024-04-25T11:06:24.907 INFO:teuthology.orchestra.run.smithi007.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 96 kB/s | 32 kB 00:00 2024-04-25T11:06:24.940 INFO:teuthology.orchestra.run.smithi007.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 557 kB/s | 172 kB 00:00 2024-04-25T11:06:24.974 INFO:teuthology.orchestra.run.smithi007.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.7 MB/s | 358 kB 00:00 2024-04-25T11:06:24.999 INFO:teuthology.orchestra.run.smithi007.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 612 kB/s | 35 kB 00:00 2024-04-25T11:06:25.041 INFO:teuthology.orchestra.run.smithi007.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.1 MB/s | 206 kB 00:00 2024-04-25T11:06:25.083 INFO:teuthology.orchestra.run.smithi007.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-04-25T11:06:25.118 INFO:teuthology.orchestra.run.smithi007.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-25T11:06:25.142 INFO:teuthology.orchestra.run.smithi007.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2024-04-25T11:06:25.176 INFO:teuthology.orchestra.run.smithi007.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 304 kB/s | 18 kB 00:00 2024-04-25T11:06:25.209 INFO:teuthology.orchestra.run.smithi007.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 350 kB/s | 23 kB 00:00 2024-04-25T11:06:25.234 INFO:teuthology.orchestra.run.smithi007.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 337 kB/s | 19 kB 00:00 2024-04-25T11:06:25.293 INFO:teuthology.orchestra.run.smithi007.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.0 MB/s | 2.0 MB 00:00 2024-04-25T11:06:25.318 INFO:teuthology.orchestra.run.smithi007.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2024-04-25T11:06:25.344 INFO:teuthology.orchestra.run.smithi007.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 939 kB/s | 46 kB 00:00 2024-04-25T11:06:25.377 INFO:teuthology.orchestra.run.smithi007.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-25T11:06:25.394 INFO:teuthology.orchestra.run.smithi007.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T11:06:25.436 INFO:teuthology.orchestra.run.smithi007.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-04-25T11:06:25.453 INFO:teuthology.orchestra.run.smithi007.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 283 kB/s | 16 kB 00:00 2024-04-25T11:06:25.495 INFO:teuthology.orchestra.run.smithi007.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-25T11:06:25.520 INFO:teuthology.orchestra.run.smithi007.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 427 kB/s | 28 kB 00:00 2024-04-25T11:06:25.546 INFO:teuthology.orchestra.run.smithi007.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 828 kB/s | 90 kB 00:00 2024-04-25T11:06:25.571 INFO:teuthology.orchestra.run.smithi007.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 411 kB/s | 31 kB 00:00 2024-04-25T11:06:25.605 INFO:teuthology.orchestra.run.smithi007.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-25T11:06:25.630 INFO:teuthology.orchestra.run.smithi007.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 619 kB/s | 36 kB 00:00 2024-04-25T11:06:25.655 INFO:teuthology.orchestra.run.smithi007.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-04-25T11:06:25.705 INFO:teuthology.orchestra.run.smithi007.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 513 kB/s | 51 kB 00:00 2024-04-25T11:06:25.731 INFO:teuthology.orchestra.run.smithi007.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T11:06:25.764 INFO:teuthology.orchestra.run.smithi007.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-25T11:06:25.790 INFO:teuthology.orchestra.run.smithi007.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 345 kB/s | 20 kB 00:00 2024-04-25T11:06:25.840 INFO:teuthology.orchestra.run.smithi007.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-25T11:06:25.874 INFO:teuthology.orchestra.run.smithi007.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T11:06:25.974 INFO:teuthology.orchestra.run.smithi007.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-25T11:06:26.233 INFO:teuthology.orchestra.run.smithi007.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 1.3 MB 00:00 2024-04-25T11:06:26.375 INFO:teuthology.orchestra.run.smithi007.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-04-25T11:06:26.475 INFO:teuthology.orchestra.run.smithi007.stdout:(151/152): librados2-19.0.0-3291.g994e2f20.el9. 5.2 MB/s | 3.3 MB 00:00 2024-04-25T11:06:26.600 INFO:teuthology.orchestra.run.smithi007.stdout:(152/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 4.3 MB/s | 3.1 MB 00:00 2024-04-25T11:06:26.605 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:06:26.606 INFO:teuthology.orchestra.run.smithi007.stdout:Total 8.6 MB/s | 214 MB 00:25 2024-04-25T11:06:29.546 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-04-25T11:06:29.809 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-04-25T11:06:29.809 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-04-25T11:06:32.638 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-04-25T11:06:32.639 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-04-25T11:06:35.957 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 755 B/s | 16 kB 00:22 2024-04-25T11:06:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-04-25T11:06:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:06:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:06:39.354 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:06:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:06:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:06:39.598 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:06:40.037 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:06:40.156 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:06:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:06:40.382 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:06:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:06:40.557 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:06:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:06:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:06:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:06:41.200 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:06:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:06:41.365 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:06:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:06:41.474 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:06:41.778 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:06:41.841 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:06:42.047 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:06:42.213 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:06:42.257 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:06:42.304 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:06:42.406 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:06:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:06:42.525 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:06:42.575 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:06:42.614 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:06:42.737 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:06:42.815 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:06:42.850 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:06:42.911 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:06:43.028 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:06:43.069 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T11:06:43.155 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T11:06:43.194 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T11:06:43.324 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:06:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:06:43.518 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:06:43.694 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:06:43.758 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:06:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:06:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:06:44.624 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:06:44.669 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:06:44.717 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T11:06:44.777 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:06:45.014 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:06:45.086 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:06:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:06:45.270 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:06:45.310 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:06:45.350 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:06:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:06:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:06:45.539 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:06:45.640 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:06:45.858 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:06:45.915 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:06:45.952 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:06:46.012 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:06:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:06:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:06:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:06:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:06:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:06:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:06:46.720 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:06:46.764 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:06:46.804 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:06:46.851 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:06:46.920 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:06:46.979 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:06:47.028 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:06:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:06:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:06:47.209 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:06:47.264 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:06:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:06:47.477 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:06:47.520 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:06:47.591 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:06:47.742 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:06:47.808 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:06:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:06:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:06:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:06:48.966 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:06:49.037 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:06:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:06:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:06:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:06:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:06:49.566 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:06:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:06:50.617 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:06:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:06:52.396 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:06:52.439 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:06:52.482 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:06:52.901 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:06:52.951 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:06:53.076 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:06:53.108 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T11:06:53.167 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:06:53.733 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:06:53.762 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:06:53.845 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:06:53.872 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T11:06:56.037 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-25T11:06:56.628 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:06:56.677 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:06:57.386 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:06:57.456 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:06:57.643 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:06:57.644 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:06:57.644 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:06:57.644 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:06:57.690 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:07:01.974 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-04-25T11:07:04.386 INFO:teuthology.orchestra.run.smithi087.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:07:04.386 INFO:teuthology.orchestra.run.smithi087.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T11:07:04.404 INFO:teuthology.orchestra.run.smithi087.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T11:07:04.405 INFO:teuthology.orchestra.run.smithi087.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T11:07:04.406 INFO:teuthology.orchestra.run.smithi087.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T11:07:04.408 INFO:teuthology.orchestra.run.smithi087.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T11:07:04.410 INFO:teuthology.orchestra.run.smithi087.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T11:07:04.413 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T11:07:04.414 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T11:07:04.484 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-04-25T11:07:04.502 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2024-04-25T11:07:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2024-04-25T11:07:04.502 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2024-04-25T11:07:04.502 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-04-25T11:07:04.502 INFO:teuthology.orchestra.run.smithi087.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T11:07:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T11:07:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T11:07:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T11:07:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T11:07:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T11:07:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T11:07:04.503 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T11:07:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T11:07:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T11:07:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T11:07:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T11:07:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T11:07:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T11:07:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T11:07:04.504 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T11:07:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T11:07:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T11:07:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T11:07:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T11:07:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T11:07:04.505 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T11:07:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T11:07:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T11:07:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T11:07:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T11:07:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T11:07:04.506 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T11:07:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T11:07:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T11:07:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T11:07:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T11:07:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T11:07:04.507 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T11:07:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T11:07:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T11:07:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T11:07:04.508 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrading: 2024-04-25T11:07:04.508 INFO:teuthology.orchestra.run.smithi087.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T11:07:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T11:07:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T11:07:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T11:07:04.509 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2024-04-25T11:07:04.509 INFO:teuthology.orchestra.run.smithi087.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T11:07:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T11:07:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T11:07:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T11:07:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T11:07:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T11:07:04.510 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T11:07:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T11:07:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T11:07:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T11:07:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T11:07:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T11:07:04.511 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T11:07:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T11:07:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T11:07:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T11:07:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T11:07:04.512 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T11:07:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T11:07:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T11:07:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T11:07:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T11:07:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T11:07:04.513 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T11:07:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T11:07:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T11:07:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T11:07:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T11:07:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T11:07:04.514 INFO:teuthology.orchestra.run.smithi087.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T11:07:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T11:07:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T11:07:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T11:07:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T11:07:04.515 INFO:teuthology.orchestra.run.smithi087.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T11:07:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T11:07:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T11:07:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T11:07:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T11:07:04.516 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T11:07:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T11:07:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T11:07:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T11:07:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T11:07:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T11:07:04.517 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T11:07:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T11:07:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T11:07:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T11:07:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T11:07:04.518 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T11:07:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T11:07:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T11:07:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T11:07:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T11:07:04.519 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T11:07:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T11:07:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T11:07:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T11:07:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T11:07:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T11:07:04.520 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T11:07:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T11:07:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T11:07:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T11:07:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T11:07:04.521 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T11:07:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T11:07:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T11:07:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T11:07:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T11:07:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T11:07:04.522 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T11:07:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T11:07:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T11:07:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T11:07:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T11:07:04.523 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T11:07:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T11:07:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T11:07:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T11:07:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T11:07:04.524 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T11:07:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T11:07:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T11:07:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T11:07:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T11:07:04.525 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T11:07:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T11:07:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T11:07:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T11:07:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T11:07:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T11:07:04.526 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T11:07:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T11:07:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T11:07:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T11:07:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T11:07:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T11:07:04.527 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T11:07:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T11:07:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T11:07:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T11:07:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T11:07:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T11:07:04.528 INFO:teuthology.orchestra.run.smithi087.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T11:07:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T11:07:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T11:07:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T11:07:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T11:07:04.529 INFO:teuthology.orchestra.run.smithi087.stdout:Installing weak dependencies: 2024-04-25T11:07:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T11:07:04.529 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:07:04.530 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-04-25T11:07:04.530 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2024-04-25T11:07:04.530 INFO:teuthology.orchestra.run.smithi087.stdout:Install 148 Packages 2024-04-25T11:07:04.530 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrade 4 Packages 2024-04-25T11:07:04.530 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:07:04.531 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 214 M 2024-04-25T11:07:04.532 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-04-25T11:07:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:07:11.614 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-04-25T11:07:11.614 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-04-25T11:07:11.614 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-04-25T11:07:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-04-25T11:07:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-04-25T11:07:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-04-25T11:07:11.615 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-04-25T11:07:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:11.856 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:07:11.905 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:07:11.905 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:07:11.905 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:07:11.905 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:13.725 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:07:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:07:13.811 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:07:13.811 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:07:13.811 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:07:13.811 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:07:13.811 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:14.083 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:07:14.182 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:07:14.182 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:07:14.182 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:07:14.182 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:07:14.182 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:07:14.182 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:07:14.813 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:07:14.814 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:07:14.814 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:07:14.814 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:07:14.814 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:07:14.814 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:14.952 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:07:15.017 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:07:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:07:15.357 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:07:15.415 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:07:15.504 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:07:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:07:15.787 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:07:16.004 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:07:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:07:16.088 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:07:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:07:16.846 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:07:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T11:07:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T11:07:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:07:18.458 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:07:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:07:19.674 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:07:19.703 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:07:19.856 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:07:20.081 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T11:07:20.113 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:07:20.184 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:07:20.184 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:07:20.184 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:07:20.184 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:07:20.185 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:07:20.185 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:07:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:07:20.422 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T11:07:20.494 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T11:07:21.581 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:07:21.616 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:07:21.688 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:07:21.689 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:07:21.689 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:07:21.689 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:07:21.689 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:07:21.689 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:21.742 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:07:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:07:21.816 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:07:21.816 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:07:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:07:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:07:21.964 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:07:21.964 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:07:21.964 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:07:21.964 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:07:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:22.369 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:07:22.444 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:07:22.444 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:07:22.444 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:07:22.444 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:07:22.444 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:07:22.445 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T11:07:27.504 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T11:07:27.599 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T11:07:27.650 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:07:27.704 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:07:28.064 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T11:07:28.146 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:07:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T11:07:29.089 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:07:29.155 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:07:29.214 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:07:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T11:07:30.755 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:07:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:07:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:07:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:07:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:07:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:07:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:07:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:07:31.450 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:07:31.585 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:07:31.879 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:07:31.880 INFO:teuthology.orchestra.run.smithi007.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T11:07:31.880 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:37.910 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T11:07:37.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T11:07:37.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T11:07:37.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T11:07:37.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T11:07:37.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T11:07:37.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T11:07:37.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:07:37.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:07:37.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:07:37.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T11:07:37.911 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T11:07:37.912 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T11:07:37.913 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T11:07:37.913 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T11:07:37.925 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T11:07:37.925 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:07:37.926 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:07:37.927 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:07:37.928 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:07:37.929 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:07:37.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:07:37.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:07:37.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:07:37.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:07:37.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:07:37.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:07:37.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:07:37.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:07:37.930 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:07:37.931 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:07:37.932 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:07:37.933 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:07:37.934 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:07:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:07:40.308 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.309 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.310 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:07:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:07:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:07:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:07:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:07:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:07:40.311 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:07:40.312 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:07:40.313 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:07:40.314 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:07:40.315 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:07:40.316 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:07:40.317 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:07:40.318 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:07:40.319 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:07:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:07:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:07:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:07:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:07:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:07:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:07:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:07:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:07:40.320 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:07:40.321 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:07:40.321 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:07:40.321 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-04-25T11:07:40.972 DEBUG:teuthology.parallel:result is None 2024-04-25T11:07:43.347 INFO:teuthology.orchestra.run.smithi087.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-25T11:07:43.664 INFO:teuthology.orchestra.run.smithi087.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 2.7 MB/s | 863 kB 00:00 2024-04-25T11:07:43.756 INFO:teuthology.orchestra.run.smithi087.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.6 MB/s | 147 kB 00:00 2024-04-25T11:07:43.873 INFO:teuthology.orchestra.run.smithi087.stdout:(4/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 6.9 MB/s | 5.4 MB 00:00 2024-04-25T11:07:44.048 INFO:teuthology.orchestra.run.smithi087.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 8.6 MB/s | 1.5 MB 00:00 2024-04-25T11:07:44.248 INFO:teuthology.orchestra.run.smithi087.stdout:(6/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 4.6 MB/s | 2.3 MB 00:00 2024-04-25T11:07:44.548 INFO:teuthology.orchestra.run.smithi087.stdout:(7/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 9.8 MB/s | 4.9 MB 00:00 2024-04-25T11:07:45.057 INFO:teuthology.orchestra.run.smithi087.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 10 MB/s | 20 MB 00:01 2024-04-25T11:07:45.157 INFO:teuthology.orchestra.run.smithi087.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 254 kB/s | 25 kB 00:00 2024-04-25T11:07:46.483 INFO:teuthology.orchestra.run.smithi087.stdout:(10/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 5.2 MB/s | 10 MB 00:01 2024-04-25T11:07:46.599 INFO:teuthology.orchestra.run.smithi087.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.9 MB/s | 224 kB 00:00 2024-04-25T11:07:46.683 INFO:teuthology.orchestra.run.smithi087.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 327 kB/s | 27 kB 00:00 2024-04-25T11:07:46.775 INFO:teuthology.orchestra.run.smithi087.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 349 kB/s | 32 kB 00:00 2024-04-25T11:07:46.958 INFO:teuthology.orchestra.run.smithi087.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 4.0 MB/s | 756 kB 00:00 2024-04-25T11:07:47.067 INFO:teuthology.orchestra.run.smithi087.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.4 MB/s | 158 kB 00:00 2024-04-25T11:07:47.167 INFO:teuthology.orchestra.run.smithi087.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.3 MB/s | 127 kB 00:00 2024-04-25T11:07:47.334 INFO:teuthology.orchestra.run.smithi087.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 2.9 MB/s | 497 kB 00:00 2024-04-25T11:07:48.009 INFO:teuthology.orchestra.run.smithi087.stdout:(18/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 15 MB/s | 43 MB 00:02 2024-04-25T11:07:48.118 INFO:teuthology.orchestra.run.smithi087.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 420 kB/s | 45 kB 00:00 2024-04-25T11:07:48.227 INFO:teuthology.orchestra.run.smithi087.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.2 MB/s | 135 kB 00:00 2024-04-25T11:07:48.343 INFO:teuthology.orchestra.run.smithi087.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.4 MB/s | 161 kB 00:00 2024-04-25T11:07:48.527 INFO:teuthology.orchestra.run.smithi087.stdout:(22/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 4.3 MB/s | 5.2 MB 00:01 2024-04-25T11:07:48.778 INFO:teuthology.orchestra.run.smithi087.stdout:(23/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 3.9 MB/s | 18 MB 00:04 2024-04-25T11:07:48.811 INFO:teuthology.orchestra.run.smithi087.stdout:(24/152): python3-rados-19.0.0-3291.g994e2f20.e 687 kB/s | 321 kB 00:00 2024-04-25T11:07:48.845 INFO:teuthology.orchestra.run.smithi087.stdout:(25/152): python3-rbd-19.0.0-3291.g994e2f20.el9 942 kB/s | 298 kB 00:00 2024-04-25T11:07:48.870 INFO:teuthology.orchestra.run.smithi087.stdout:(26/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.1 MB/s | 100 kB 00:00 2024-04-25T11:07:48.903 INFO:teuthology.orchestra.run.smithi087.stdout:(27/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 940 kB/s | 86 kB 00:00 2024-04-25T11:07:48.970 INFO:teuthology.orchestra.run.smithi087.stdout:(28/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.7 MB/s | 169 kB 00:00 2024-04-25T11:07:48.987 INFO:teuthology.orchestra.run.smithi087.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3291.g 413 kB/s | 34 kB 00:00 2024-04-25T11:07:49.088 INFO:teuthology.orchestra.run.smithi087.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.2 MB/s | 142 kB 00:00 2024-04-25T11:07:49.413 INFO:teuthology.orchestra.run.smithi087.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 8.3 MB/s | 3.5 MB 00:00 2024-04-25T11:07:49.488 INFO:teuthology.orchestra.run.smithi087.stdout:(32/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 4.7 MB/s | 3.0 MB 00:00 2024-04-25T11:07:49.522 INFO:teuthology.orchestra.run.smithi087.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3291.g99 2.3 MB/s | 249 kB 00:00 2024-04-25T11:07:49.601 INFO:teuthology.orchestra.run.smithi087.stdout:(34/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 442 kB/s | 49 kB 00:00 2024-04-25T11:07:49.631 INFO:teuthology.orchestra.run.smithi087.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3291.g9 152 kB/s | 16 kB 00:00 2024-04-25T11:07:49.689 INFO:teuthology.orchestra.run.smithi087.stdout:(36/152): ceph-volume-19.0.0-3291.g994e2f20.el9 3.1 MB/s | 273 kB 00:00 2024-04-25T11:07:49.748 INFO:teuthology.orchestra.run.smithi087.stdout:(37/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 6.4 MB/s | 762 kB 00:00 2024-04-25T11:07:49.773 INFO:teuthology.orchestra.run.smithi087.stdout:(38/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 220 kB/s | 18 kB 00:00 2024-04-25T11:07:49.965 INFO:teuthology.orchestra.run.smithi087.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 372 kB/s | 80 kB 00:00 2024-04-25T11:07:50.032 INFO:teuthology.orchestra.run.smithi087.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-25T11:07:50.124 INFO:teuthology.orchestra.run.smithi087.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-25T11:07:50.258 INFO:teuthology.orchestra.run.smithi087.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 9.3 MB/s | 1.2 MB 00:00 2024-04-25T11:07:50.291 INFO:teuthology.orchestra.run.smithi087.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-04-25T11:07:50.317 INFO:teuthology.orchestra.run.smithi087.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T11:07:50.367 INFO:teuthology.orchestra.run.smithi087.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-25T11:07:50.392 INFO:teuthology.orchestra.run.smithi087.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-25T11:07:50.426 INFO:teuthology.orchestra.run.smithi087.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T11:07:50.607 INFO:teuthology.orchestra.run.smithi087.stdout:(48/152): ceph-mgr-diskprediction-local-19.0.0- 4.9 MB/s | 7.4 MB 00:01 2024-04-25T11:07:50.668 INFO:teuthology.orchestra.run.smithi087.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-25T11:07:50.718 INFO:teuthology.orchestra.run.smithi087.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-04-25T11:07:50.743 INFO:teuthology.orchestra.run.smithi087.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 788 kB/s | 107 kB 00:00 2024-04-25T11:07:50.769 INFO:teuthology.orchestra.run.smithi087.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 216 kB/s | 21 kB 00:00 2024-04-25T11:07:50.811 INFO:teuthology.orchestra.run.smithi087.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 493 kB/s | 33 kB 00:00 2024-04-25T11:07:50.844 INFO:teuthology.orchestra.run.smithi087.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-04-25T11:07:50.886 INFO:teuthology.orchestra.run.smithi087.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 272 kB/s | 11 kB 00:00 2024-04-25T11:07:50.911 INFO:teuthology.orchestra.run.smithi087.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 183 kB/s | 18 kB 00:00 2024-04-25T11:07:50.970 INFO:teuthology.orchestra.run.smithi087.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 788 kB/s | 45 kB 00:00 2024-04-25T11:07:50.995 INFO:teuthology.orchestra.run.smithi087.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 160 kB 00:00 2024-04-25T11:07:51.079 INFO:teuthology.orchestra.run.smithi087.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.7 MB/s | 3.0 MB 00:00 2024-04-25T11:07:51.129 INFO:teuthology.orchestra.run.smithi087.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 515 kB/s | 25 kB 00:00 2024-04-25T11:07:51.171 INFO:teuthology.orchestra.run.smithi087.stdout:(61/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-04-25T11:07:51.222 INFO:teuthology.orchestra.run.smithi087.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 2.5 MB/s | 582 kB 00:00 2024-04-25T11:07:51.239 INFO:teuthology.orchestra.run.smithi087.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 439 kB/s | 47 kB 00:00 2024-04-25T11:07:51.289 INFO:teuthology.orchestra.run.smithi087.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-04-25T11:07:51.306 INFO:teuthology.orchestra.run.smithi087.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 534 kB/s | 35 kB 00:00 2024-04-25T11:07:51.331 INFO:teuthology.orchestra.run.smithi087.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-25T11:07:51.473 INFO:teuthology.orchestra.run.smithi087.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T11:07:51.590 INFO:teuthology.orchestra.run.smithi087.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-04-25T11:07:51.690 INFO:teuthology.orchestra.run.smithi087.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-25T11:07:51.740 INFO:teuthology.orchestra.run.smithi087.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.3 MB/s | 911 kB 00:00 2024-04-25T11:07:51.766 INFO:teuthology.orchestra.run.smithi087.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 764 kB/s | 133 kB 00:00 2024-04-25T11:07:51.833 INFO:teuthology.orchestra.run.smithi087.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T11:07:51.883 INFO:teuthology.orchestra.run.smithi087.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-04-25T11:07:51.967 INFO:teuthology.orchestra.run.smithi087.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-25T11:07:52.101 INFO:teuthology.orchestra.run.smithi087.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-25T11:07:52.151 INFO:teuthology.orchestra.run.smithi087.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-04-25T11:07:52.192 INFO:teuthology.orchestra.run.smithi087.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 282 kB/s | 26 kB 00:00 2024-04-25T11:07:52.218 INFO:teuthology.orchestra.run.smithi087.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 291 kB/s | 19 kB 00:00 2024-04-25T11:07:52.334 INFO:teuthology.orchestra.run.smithi087.stdout:(79/152): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-25T11:07:52.360 INFO:teuthology.orchestra.run.smithi087.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 246 kB/s | 35 kB 00:00 2024-04-25T11:07:52.385 INFO:teuthology.orchestra.run.smithi087.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 900 kB/s | 172 kB 00:00 2024-04-25T11:07:52.452 INFO:teuthology.orchestra.run.smithi087.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-25T11:07:52.485 INFO:teuthology.orchestra.run.smithi087.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-25T11:07:52.544 INFO:teuthology.orchestra.run.smithi087.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 848 kB/s | 77 kB 00:00 2024-04-25T11:07:52.570 INFO:teuthology.orchestra.run.smithi087.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 509 kB/s | 42 kB 00:00 2024-04-25T11:07:52.695 INFO:teuthology.orchestra.run.smithi087.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-04-25T11:07:52.728 INFO:teuthology.orchestra.run.smithi087.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-25T11:07:52.762 INFO:teuthology.orchestra.run.smithi087.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 1.2 MB/s | 269 kB 00:00 2024-04-25T11:07:52.812 INFO:teuthology.orchestra.run.smithi087.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 616 kB/s | 51 kB 00:00 2024-04-25T11:07:52.837 INFO:teuthology.orchestra.run.smithi087.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 719 kB/s | 54 kB 00:00 2024-04-25T11:07:52.865 INFO:teuthology.orchestra.run.smithi087.stdout:(91/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-04-25T11:07:52.907 INFO:teuthology.orchestra.run.smithi087.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 609 kB/s | 42 kB 00:00 2024-04-25T11:07:52.924 INFO:teuthology.orchestra.run.smithi087.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 767 kB/s | 44 kB 00:00 2024-04-25T11:07:52.966 INFO:teuthology.orchestra.run.smithi087.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 879 kB/s | 36 kB 00:00 2024-04-25T11:07:52.999 INFO:teuthology.orchestra.run.smithi087.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-25T11:07:53.041 INFO:teuthology.orchestra.run.smithi087.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 639 kB/s | 47 kB 00:00 2024-04-25T11:07:53.100 INFO:teuthology.orchestra.run.smithi087.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-25T11:07:53.155 INFO:teuthology.orchestra.run.smithi087.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 516 kB/s | 58 kB 00:00 2024-04-25T11:07:53.226 INFO:teuthology.orchestra.run.smithi087.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 751 kB/s | 94 kB 00:00 2024-04-25T11:07:53.251 INFO:teuthology.orchestra.run.smithi087.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 669 kB/s | 64 kB 00:00 2024-04-25T11:07:53.318 INFO:teuthology.orchestra.run.smithi087.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 489 kB/s | 45 kB 00:00 2024-04-25T11:07:53.343 INFO:teuthology.orchestra.run.smithi087.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-04-25T11:07:53.452 INFO:teuthology.orchestra.run.smithi087.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.3 MB/s | 863 kB 00:00 2024-04-25T11:07:53.677 INFO:teuthology.orchestra.run.smithi087.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-04-25T11:07:53.811 INFO:teuthology.orchestra.run.smithi087.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-25T11:07:53.836 INFO:teuthology.orchestra.run.smithi087.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 123 kB/s | 19 kB 00:00 2024-04-25T11:07:53.911 INFO:teuthology.orchestra.run.smithi087.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 349 kB/s | 26 kB 00:00 2024-04-25T11:07:53.945 INFO:teuthology.orchestra.run.smithi087.stdout:(108/152): gperftools-libs-2.9.1-2.el9.x86_64.r 627 kB/s | 309 kB 00:00 2024-04-25T11:07:54.003 INFO:teuthology.orchestra.run.smithi087.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 533 kB/s | 49 kB 00:00 2024-04-25T11:07:54.020 INFO:teuthology.orchestra.run.smithi087.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 905 kB/s | 67 kB 00:00 2024-04-25T11:07:54.129 INFO:teuthology.orchestra.run.smithi087.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-25T11:07:54.188 INFO:teuthology.orchestra.run.smithi087.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.9 MB/s | 838 kB 00:00 2024-04-25T11:07:54.246 INFO:teuthology.orchestra.run.smithi087.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 756 kB/s | 43 kB 00:00 2024-04-25T11:07:54.355 INFO:teuthology.orchestra.run.smithi087.stdout:(114/152): libarrow-9.0.0-10.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-04-25T11:07:54.380 INFO:teuthology.orchestra.run.smithi087.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 242 kB/s | 32 kB 00:00 2024-04-25T11:07:54.413 INFO:teuthology.orchestra.run.smithi087.stdout:(116/152): python3-asyncssh-2.13.2-5.el9.noarch 1.9 MB/s | 548 kB 00:00 2024-04-25T11:07:54.439 INFO:teuthology.orchestra.run.smithi087.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 180 kB/s | 15 kB 00:00 2024-04-25T11:07:54.473 INFO:teuthology.orchestra.run.smithi087.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-25T11:07:54.506 INFO:teuthology.orchestra.run.smithi087.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-25T11:07:54.557 INFO:teuthology.orchestra.run.smithi087.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 425 kB/s | 35 kB 00:00 2024-04-25T11:07:54.590 INFO:teuthology.orchestra.run.smithi087.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-04-25T11:07:54.624 INFO:teuthology.orchestra.run.smithi087.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T11:07:54.682 INFO:teuthology.orchestra.run.smithi087.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.6 MB/s | 144 kB 00:00 2024-04-25T11:07:54.716 INFO:teuthology.orchestra.run.smithi087.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-04-25T11:07:54.741 INFO:teuthology.orchestra.run.smithi087.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-04-25T11:07:54.775 INFO:teuthology.orchestra.run.smithi087.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 401 kB/s | 23 kB 00:00 2024-04-25T11:07:54.800 INFO:teuthology.orchestra.run.smithi087.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 335 kB/s | 19 kB 00:00 2024-04-25T11:07:54.834 INFO:teuthology.orchestra.run.smithi087.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 341 kB/s | 20 kB 00:00 2024-04-25T11:07:54.909 INFO:teuthology.orchestra.run.smithi087.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.8 MB/s | 2.0 MB 00:00 2024-04-25T11:07:54.934 INFO:teuthology.orchestra.run.smithi087.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 466 kB/s | 46 kB 00:00 2024-04-25T11:07:54.968 INFO:teuthology.orchestra.run.smithi087.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-25T11:07:54.993 INFO:teuthology.orchestra.run.smithi087.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 991 kB/s | 58 kB 00:00 2024-04-25T11:07:55.035 INFO:teuthology.orchestra.run.smithi087.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-04-25T11:07:55.060 INFO:teuthology.orchestra.run.smithi087.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 247 kB/s | 16 kB 00:00 2024-04-25T11:07:55.094 INFO:teuthology.orchestra.run.smithi087.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-25T11:07:55.119 INFO:teuthology.orchestra.run.smithi087.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 487 kB/s | 28 kB 00:00 2024-04-25T11:07:55.145 INFO:teuthology.orchestra.run.smithi087.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-04-25T11:07:55.193 INFO:teuthology.orchestra.run.smithi087.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 315 kB/s | 31 kB 00:00 2024-04-25T11:07:55.228 INFO:teuthology.orchestra.run.smithi087.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 713 kB/s | 59 kB 00:00 2024-04-25T11:07:55.262 INFO:teuthology.orchestra.run.smithi087.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 527 kB/s | 36 kB 00:00 2024-04-25T11:07:55.287 INFO:teuthology.orchestra.run.smithi087.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 883 kB/s | 51 kB 00:00 2024-04-25T11:07:55.312 INFO:teuthology.orchestra.run.smithi087.stdout:(142/152): python3-routes-2.5.1-5.el9.noarch.rp 980 kB/s | 188 kB 00:00 2024-04-25T11:07:55.371 INFO:teuthology.orchestra.run.smithi087.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-25T11:07:55.396 INFO:teuthology.orchestra.run.smithi087.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 829 kB/s | 90 kB 00:00 2024-04-25T11:07:55.430 INFO:teuthology.orchestra.run.smithi087.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-25T11:07:55.455 INFO:teuthology.orchestra.run.smithi087.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-25T11:07:55.489 INFO:teuthology.orchestra.run.smithi087.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-25T11:07:55.597 INFO:teuthology.orchestra.run.smithi087.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-25T11:07:55.831 INFO:teuthology.orchestra.run.smithi087.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 1.3 MB 00:00 2024-04-25T11:07:55.906 INFO:teuthology.orchestra.run.smithi087.stdout:(150/152): librados2-19.0.0-3291.g994e2f20.el9. 7.4 MB/s | 3.3 MB 00:00 2024-04-25T11:07:55.990 INFO:teuthology.orchestra.run.smithi087.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-04-25T11:07:56.074 INFO:teuthology.orchestra.run.smithi087.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.9 MB/s | 2.4 MB 00:00 2024-04-25T11:07:56.079 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-04-25T11:07:56.080 INFO:teuthology.orchestra.run.smithi087.stdout:Total 4.2 MB/s | 214 MB 00:51 2024-04-25T11:07:58.891 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-04-25T11:07:59.135 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-04-25T11:07:59.135 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-04-25T11:08:01.781 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-04-25T11:08:01.782 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-04-25T11:08:08.084 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-04-25T11:08:08.188 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T11:08:08.273 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T11:08:08.405 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T11:08:08.478 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T11:08:08.516 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T11:08:08.666 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T11:08:09.114 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T11:08:09.229 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:08:09.383 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:08:09.414 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:08:09.536 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:08:09.571 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:08:09.905 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T11:08:09.977 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T11:08:10.010 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:08:10.165 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:08:10.194 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:08:10.290 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T11:08:10.320 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:08:10.387 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T11:08:10.654 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:08:10.713 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T11:08:10.910 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T11:08:11.034 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T11:08:11.084 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T11:08:11.128 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T11:08:11.229 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T11:08:11.274 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T11:08:11.330 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T11:08:11.421 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T11:08:11.462 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T11:08:11.560 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T11:08:11.639 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T11:08:11.679 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T11:08:11.730 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T11:08:11.833 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T11:08:11.866 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T11:08:11.947 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T11:08:11.981 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T11:08:12.098 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T11:08:12.150 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T11:08:12.251 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T11:08:12.439 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T11:08:12.512 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T11:08:13.214 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T11:08:13.327 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T11:08:13.372 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T11:08:13.449 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T11:08:13.491 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T11:08:13.553 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T11:08:13.803 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T11:08:13.865 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T11:08:13.968 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T11:08:14.073 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T11:08:14.124 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T11:08:14.163 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T11:08:14.232 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T11:08:14.279 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T11:08:14.342 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T11:08:14.410 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T11:08:14.627 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T11:08:14.677 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T11:08:14.714 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T11:08:14.771 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T11:08:14.900 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T11:08:14.956 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T11:08:15.006 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T11:08:15.037 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T11:08:15.062 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T11:08:15.243 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T11:08:15.431 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T11:08:15.477 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T11:08:15.515 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T11:08:15.562 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T11:08:15.622 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T11:08:15.672 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T11:08:15.714 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T11:08:15.759 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T11:08:15.800 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T11:08:15.884 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T11:08:15.950 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T11:08:16.106 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T11:08:16.171 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T11:08:16.215 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T11:08:16.276 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T11:08:16.425 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T11:08:16.495 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T11:08:17.473 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T11:08:17.528 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T11:08:17.560 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T11:08:17.686 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T11:08:17.756 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T11:08:17.862 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T11:08:17.904 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T11:08:18.135 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T11:08:18.167 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T11:08:18.270 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T11:08:19.106 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T11:08:19.292 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T11:08:20.915 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T11:08:21.069 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T11:08:21.131 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T11:08:21.196 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T11:08:21.629 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T11:08:21.703 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:08:21.868 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T11:08:21.928 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T11:08:22.013 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T11:08:22.674 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T11:08:22.731 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:08:22.832 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T11:08:22.883 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T11:08:25.465 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:08:25.529 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:08:26.193 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T11:08:26.275 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:08:26.466 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T11:08:26.466 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T11:08:26.467 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T11:08:26.467 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:08:26.509 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:08:39.569 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T11:08:39.569 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /sys 2024-04-25T11:08:39.569 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /proc 2024-04-25T11:08:39.569 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /mnt 2024-04-25T11:08:39.569 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /var/tmp 2024-04-25T11:08:39.569 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /home 2024-04-25T11:08:39.569 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /root 2024-04-25T11:08:39.569 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /tmp 2024-04-25T11:08:39.569 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:08:39.817 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:08:39.863 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T11:08:39.863 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:08:39.864 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T11:08:39.864 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:08:41.600 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:08:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T11:08:41.679 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:08:41.680 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T11:08:41.680 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:08:41.680 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T11:08:41.680 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:08:41.933 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:08:42.002 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T11:08:42.002 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:08:42.002 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T11:08:42.002 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:08:42.002 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T11:08:42.002 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:08:42.518 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:08:42.583 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T11:08:42.583 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:08:42.583 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T11:08:42.584 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:08:42.584 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T11:08:42.584 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:08:42.709 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T11:08:42.770 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T11:08:42.855 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T11:08:43.105 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T11:08:43.172 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T11:08:43.263 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T11:08:43.368 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:08:43.543 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:08:43.602 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T11:08:43.645 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T11:08:43.687 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T11:08:44.360 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:08:44.408 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T11:08:44.465 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T11:08:44.574 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T11:08:44.608 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:08:45.900 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T11:08:45.944 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:08:47.043 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T11:08:47.077 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:08:47.227 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T11:08:47.405 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T11:08:47.438 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:08:47.505 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T11:08:47.505 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:08:47.505 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T11:08:47.505 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:08:47.506 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T11:08:47.506 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:08:47.572 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:08:47.609 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T11:08:47.768 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T11:08:47.852 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T11:08:48.897 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T11:08:48.932 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:08:48.998 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T11:08:48.999 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:08:48.999 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T11:08:48.999 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:08:48.999 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T11:08:48.999 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:08:49.058 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:08:49.136 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T11:08:49.136 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:08:49.136 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T11:08:49.137 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:08:49.209 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:08:49.283 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T11:08:49.283 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:08:49.283 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T11:08:49.283 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:08:49.283 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T11:08:49.283 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:08:49.668 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:08:49.734 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T11:08:49.735 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T11:08:49.735 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T11:08:49.735 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:08:49.735 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T11:08:49.735 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:08:54.408 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T11:08:54.483 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T11:08:54.581 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T11:08:54.628 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T11:08:54.675 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T11:08:55.010 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T11:08:55.070 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T11:08:55.097 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T11:08:56.035 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:08:56.087 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:08:56.143 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T11:08:57.458 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T11:08:57.569 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T11:08:57.664 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T11:08:57.852 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T11:08:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T11:08:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:08:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T11:08:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:08:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:08:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T11:08:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:08:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:08:58.642 INFO:teuthology.orchestra.run.smithi087.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T11:08:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T11:09:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T11:09:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T11:09:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T11:09:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T11:09:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T11:09:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T11:09:04.364 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T11:09:04.365 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T11:09:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T11:09:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T11:09:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T11:09:04.366 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T11:09:04.378 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T11:09:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T11:09:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T11:09:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T11:09:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T11:09:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T11:09:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T11:09:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T11:09:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T11:09:04.379 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T11:09:04.380 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T11:09:04.381 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T11:09:04.382 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T11:09:04.383 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T11:09:04.384 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T11:09:04.385 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T11:09:04.386 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T11:09:04.387 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T11:09:04.388 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T11:09:06.861 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T11:09:06.861 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:06.861 INFO:teuthology.orchestra.run.smithi087.stdout:Upgraded: 2024-04-25T11:09:06.861 INFO:teuthology.orchestra.run.smithi087.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.861 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.861 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T11:09:06.861 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.862 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:09:06.863 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T11:09:06.864 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T11:09:06.865 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T11:09:06.866 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.867 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T11:09:06.868 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T11:09:06.869 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T11:09:06.870 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T11:09:06.871 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T11:09:06.872 INFO:teuthology.orchestra.run.smithi087.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T11:09:06.873 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T11:09:06.873 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T11:09:06.873 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T11:09:06.873 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:06.873 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-04-25T11:09:07.463 DEBUG:teuthology.parallel:result is None 2024-04-25T11:09:07.464 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:09:07.614 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:09:07.690 INFO:teuthology.orchestra.run.smithi007.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T11:09:07.691 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T11:09:07.691 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T11:09:07.693 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T11:09:07.844 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T11:09:07.915 INFO:teuthology.orchestra.run.smithi087.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T11:09:07.916 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T11:09:07.916 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T11:09:07.918 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T11:09:07.918 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:07.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:09:08.015 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:08.015 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T11:09:08.106 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T11:09:08.106 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:08.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:09:08.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:09:08.337 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:08.337 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T11:09:08.404 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T11:09:08.509 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T11:09:08.509 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:08.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:09:08.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:09:08.738 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:08.738 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T11:09:08.808 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T11:09:08.922 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T11:09:08.922 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:08.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:09:09.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:09:09.110 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:09.110 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T11:09:09.177 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T11:09:09.285 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T11:09:09.376 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T11:09:09.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-04-25T11:09:09.384 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /var/log/ceph 2024-04-25T11:09:09.471 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T11:09:09.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T11:09:09.479 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T11:09:09.561 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T11:09:09.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T11:09:09.568 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T11:09:09.658 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T11:09:09.659 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T11:09:09.659 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T11:09:09.659 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:09.659 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T11:09:09.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T11:09:09.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T11:09:10.006 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:10.006 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T11:09:10.050 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T11:09:10.221 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T11:09:10.286 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T11:09:10.290 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T11:09:10.290 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/client-limits}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673113', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBF9DBbDueiRm4eDdN0mrzKTJprD7x+i5FZv0vs3HthC+LUltS7s7AUgunI7ikqP83+TEI+YWv3AyIG6ogCgVHQ=', 'smithi087.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNeDhjrKhWH3UftX4hlFg2IblmuJy9LaZzw/IPWE2EoQR6d7VNAsXTvAtNKFmQsC1IYgilb3kTOBH94c524Blrg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'responding to mclientcaps\\(revoke\\)', 'not advance its oldest_client_tid', 'failing to advance its oldest client/flush tid', 'Too many inodes in cache', 'failing to respond to cache pressure', 'slow requests are blocked', 'failing to respond to capability release', 'MDS cache is too large', '\\(MDS_CLIENT_OLDEST_TID\\)', '\\(MDS_CACHE_OVERSIZED\\)'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_client_limits']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T11:09:10.290 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T11:09:10.298 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T11:09:10.350 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T11:09:10.358 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T11:09:10.446 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:10.447 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:09:10.503 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:09:10.503 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-04-25T11:09:10.561 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:09:10.561 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:09:10.561 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-25T11:09:10.561 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:09:10.561 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:09:10.561 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-04-25 11:07:32.380885425 +0000 2024-04-25T11:09:10.561 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-04-25 11:05:09.358945173 +0000 2024-04-25T11:09:10.561 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-04-25 11:05:09.358945173 +0000 2024-04-25T11:09:10.561 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-04-25 11:05:09.358945173 +0000 2024-04-25T11:09:10.562 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:09:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-04-25T11:09:10.647 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-04-25T11:09:10.648 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000264154 s, 1.9 MB/s 2024-04-25T11:09:10.651 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:09:10.707 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-04-25T11:09:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:09:10.761 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:09:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-25T11:09:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:09:10.761 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:09:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-04-25 11:07:32.381885410 +0000 2024-04-25T11:09:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-04-25 11:05:09.959935426 +0000 2024-04-25T11:09:10.762 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-04-25 11:05:09.959935426 +0000 2024-04-25T11:09:10.762 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-04-25 11:05:09.959935426 +0000 2024-04-25T11:09:10.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:09:10.845 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-04-25T11:09:10.845 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-04-25T11:09:10.845 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000257233 s, 2.0 MB/s 2024-04-25T11:09:10.849 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:09:10.911 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-04-25T11:09:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:09:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:09:10.965 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-25T11:09:10.965 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:09:10.965 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:09:10.965 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-04-25 11:07:32.381885410 +0000 2024-04-25T11:09:10.965 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-04-25 11:05:10.610924868 +0000 2024-04-25T11:09:10.966 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-04-25 11:05:10.610924868 +0000 2024-04-25T11:09:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-04-25 11:05:10.610924868 +0000 2024-04-25T11:09:10.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:09:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-04-25T11:09:11.053 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-04-25T11:09:11.053 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000268275 s, 1.9 MB/s 2024-04-25T11:09:11.057 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:09:11.116 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-04-25T11:09:11.169 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:09:11.170 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:09:11.170 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-25T11:09:11.170 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:09:11.170 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:09:11.170 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-04-25 11:07:32.382885394 +0000 2024-04-25T11:09:11.170 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-04-25 11:05:11.229914830 +0000 2024-04-25T11:09:11.170 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-04-25 11:05:11.229914830 +0000 2024-04-25T11:09:11.170 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-04-25 11:05:11.229914830 +0000 2024-04-25T11:09:11.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:09:11.252 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-04-25T11:09:11.252 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-04-25T11:09:11.252 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000257811 s, 2.0 MB/s 2024-04-25T11:09:11.256 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:09:11.315 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:09:11.315 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:11.315 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T11:09:11.361 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T11:09:11.361 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2024-04-25T11:09:11.442 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T11:09:11.443 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:09:11.443 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-25T11:09:11.443 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:09:11.443 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:09:11.443 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 11:08:59.149796911 +0000 2024-04-25T11:09:11.443 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 11:05:00.519190334 +0000 2024-04-25T11:09:11.443 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 11:05:00.519190334 +0000 2024-04-25T11:09:11.443 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-04-25 11:05:00.519190334 +0000 2024-04-25T11:09:11.443 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T11:09:11.542 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T11:09:11.542 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T11:09:11.542 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.00042317 s, 1.2 MB/s 2024-04-25T11:09:11.545 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T11:09:11.584 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2024-04-25T11:09:11.664 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T11:09:11.664 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:09:11.664 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-25T11:09:11.664 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:09:11.664 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:09:11.664 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 11:08:59.150796889 +0000 2024-04-25T11:09:11.664 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 11:05:01.102176922 +0000 2024-04-25T11:09:11.664 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 11:05:01.102176922 +0000 2024-04-25T11:09:11.664 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-04-25 11:05:01.102176922 +0000 2024-04-25T11:09:11.665 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T11:09:11.762 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T11:09:11.763 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T11:09:11.763 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000284377 s, 1.8 MB/s 2024-04-25T11:09:11.766 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T11:09:11.805 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2024-04-25T11:09:11.886 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T11:09:11.887 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:09:11.887 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-25T11:09:11.887 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:09:11.887 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:09:11.887 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 11:08:59.151796867 +0000 2024-04-25T11:09:11.887 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 11:05:01.717162773 +0000 2024-04-25T11:09:11.887 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 11:05:01.717162773 +0000 2024-04-25T11:09:11.887 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-04-25 11:05:01.717162773 +0000 2024-04-25T11:09:11.887 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T11:09:11.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T11:09:11.990 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T11:09:11.990 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000331303 s, 1.5 MB/s 2024-04-25T11:09:11.994 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T11:09:12.031 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2024-04-25T11:09:12.111 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T11:09:12.111 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T11:09:12.112 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-25T11:09:12.112 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T11:09:12.112 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T11:09:12.112 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 11:08:59.151796867 +0000 2024-04-25T11:09:12.112 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 11:05:02.302149315 +0000 2024-04-25T11:09:12.112 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 11:05:02.302149315 +0000 2024-04-25T11:09:12.112 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-04-25 11:05:02.302149315 +0000 2024-04-25T11:09:12.112 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T11:09:12.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T11:09:12.207 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T11:09:12.207 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000256069 s, 2.0 MB/s 2024-04-25T11:09:12.210 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T11:09:12.249 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:09:12.249 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T11:09:12.249 INFO:tasks.ceph:Generating config... 2024-04-25T11:09:12.250 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T11:09:12.250 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T11:09:12.251 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T11:09:12.251 INFO:tasks.ceph:[client] pid file = 2024-04-25T11:09:12.251 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T11:09:12.251 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T11:09:12.251 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T11:09:12.251 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T11:09:12.251 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T11:09:12.251 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T11:09:12.251 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T11:09:12.251 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T11:09:12.251 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T11:09:12.251 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T11:09:12.251 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T11:09:12.251 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T11:09:12.251 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T11:09:12.252 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T11:09:12.252 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T11:09:12.252 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T11:09:12.252 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T11:09:12.252 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T11:09:12.252 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T11:09:12.252 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T11:09:12.252 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T11:09:12.252 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T11:09:12.252 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T11:09:12.252 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T11:09:12.252 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T11:09:12.252 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T11:09:12.252 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T11:09:12.252 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T11:09:12.253 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T11:09:12.253 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T11:09:12.253 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T11:09:12.253 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T11:09:12.253 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T11:09:12.253 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T11:09:12.253 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T11:09:12.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T11:09:12.466 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T11:09:12.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T11:09:12.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:09:12.784 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.b', '172.21.15.87'), ('mon.c', '[v2:172.21.15.87:3301,v1:172.21.15.87:6790]')] 2024-04-25T11:09:12.785 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,172.21.15.87,[v2:172.21.15.87:3301,v1:172.21.15.87:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T11:09:12.785 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:12.786 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T11:09:12.845 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --add b 172.21.15.87 --addv c '[v2:172.21.15.87:3301,v1:172.21.15.87:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 4ee91e9b-5e2d-4052-ab33-0a56271c8c7f 2024-04-25T11:09:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-04-25T11:09:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2024-04-25T11:09:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 4ee91e9b-5e2d-4052-ab33-0a56271c8c7f 2024-04-25T11:09:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2024-04-25T11:09:13.077597+0000 2024-04-25T11:09:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-04-25T11:09:13.077597+0000 2024-04-25T11:09:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 17 (quincy) 2024-04-25T11:09:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2024-04-25T11:09:13.078 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-04-25T11:09:13.078 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.b 2024-04-25T11:09:13.078 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] mon.c 2024-04-25T11:09:13.078 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T11:09:13.086 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T11:09:13.139 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4ee91e9b-5e2d-4052-ab33-0a56271c8c7f... 2024-04-25T11:09:13.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T11:09:13.145 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T11:09:13.260 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-04-25T11:09:13.260 INFO:teuthology.orchestra.run.smithi087.stdout: chdir = "" 2024-04-25T11:09:13.260 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T11:09:13.260 INFO:teuthology.orchestra.run.smithi087.stdout: auth supported = cephx 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: filestore xattr use omap = true 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: mon clock drift allowed = 1.000 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: osd crush chooseleaf type = 0 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: auth debug = true 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on old message = true 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on bug = true 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: mon pg warn max object skew = 0 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.261 INFO:teuthology.orchestra.run.smithi087.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default size = 2 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow primary affinity = true 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow pg remap = true 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on legacy crush tunables = false 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on crush straw calc version zero = false 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on no sortbitwise = false 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on osd down out interval zero = false 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on too few osds = false 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T11:09:13.262 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: mon_allow_pool_size_one = true 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: osd default data pool replay window = 5 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: mon allow pool delete = true 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: mon cluster log file level = debug 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: debug asserts on shutdown = true 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: mon health detail to clog = false 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: mon host = "172.21.15.7,172.21.15.87,[v2:172.21.15.87:3301,v1:172.21.15.87:6790]" 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on skipped message = False 2024-04-25T11:09:13.263 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 4ee91e9b-5e2d-4052-ab33-0a56271c8c7f 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:[osd] 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: osd journal size = 100 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub load threshold = 5.0 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub max interval = 600 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: osd mclock profile = high_recovery_ops 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: osd recover clone overlap = true 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: osd recovery max chunk = 1048576 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug shutdown = true 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug op order = true 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify stray on activate = true 2024-04-25T11:09:13.264 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug trim objects = true 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: osd open classes on start = true 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug pg log writeout = true 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: osd deep scrub update digest min age = 30 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: osd map max advance = 10 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: journal zero on create = true 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: filestore ondisk finisher threads = 3 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: filestore apply finisher threads = 3 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: bdev debug aio = true 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug misdirected ops = true 2024-04-25T11:09:13.265 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore block size = 96636764160 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore fsck on mount = True 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluefs = 20 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluestore = 20 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: debug osd = 20 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: debug rocksdb = 10 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.9 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.8 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: osd failsafe full ratio = 0.95 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: osd objectstore = bluestore 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.266 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr] 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: debug mgr = 20 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: mgr/telemetry/nag = false 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout:[mon] 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: debug paxos = 20 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: mon data avail warn = 5 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: mon mgr mkfs grace = 240 2024-04-25T11:09:13.267 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd reporter subtree level = osd 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd prime pg temp = true 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: # 1m isn't quite enough 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: mon_down_mkfs_grace = 2m 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.268 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_filestore_osds = false 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: mon down mkfs grace = 300 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: mon op complaint time = 120 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout:[client] 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: rgw cache enabled = true 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable ops log = true 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable usage log = true 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: client mount timeout = 600 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = "" 2024-04-25T11:09:13.269 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.a] 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.b] 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.c] 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout:[mds] 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds = 20 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds balancer = 20 2024-04-25T11:09:13.270 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T11:09:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug frag = True 2024-04-25T11:09:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug scatterstat = True 2024-04-25T11:09:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: mds op complaint time = 180 2024-04-25T11:09:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: mds verify scatter = True 2024-04-25T11:09:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T11:09:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T11:09:13.271 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T11:09:13.283 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-04-25T11:09:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-04-25T11:09:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T11:09:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-04-25T11:09:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-04-25T11:09:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-04-25T11:09:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-04-25T11:09:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-04-25T11:09:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.87,[v2:172.21.15.87:3301,v1:172.21.15.87:6790]" 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on skipped message = False 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 4ee91e9b-5e2d-4052-ab33-0a56271c8c7f 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-04-25T11:09:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 20 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 20 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 10 2024-04-25T11:09:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-04-25T11:09:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T11:09:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: mon down mkfs grace = 300 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-04-25T11:09:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-04-25T11:09:13.292 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-04-25T11:09:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2024-04-25T11:09:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds balancer = 20 2024-04-25T11:09:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-04-25T11:09:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2024-04-25T11:09:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2024-04-25T11:09:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2024-04-25T11:09:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-04-25T11:09:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-04-25T11:09:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-04-25T11:09:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-04-25T11:09:13.321 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T11:09:13.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T11:09:13.544 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T11:09:13.544 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:13.544 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T11:09:13.599 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:13.599 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T11:09:13.654 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:09:13.654 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:13.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T11:09:13.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:09:13.789 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:13.789 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:13.886 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:09:13.886 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:13.886 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T11:09:13.886 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T11:09:13.986 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:13.987 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:14.071 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T11:09:14.071 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T11:09:14.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T11:09:14.303 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T11:09:14.319 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T11:09:14.586 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T11:09:14.601 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T11:09:14.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T11:09:14.850 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T11:09:14.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T11:09:14.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T11:09:15.190 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T11:09:15.206 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T11:09:15.296 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T11:09:15.513 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T11:09:15.526 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T11:09:15.608 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T11:09:15.825 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T11:09:15.839 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T11:09:15.916 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T11:09:15.916 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T11:09:16.119 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T11:09:16.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T11:09:16.381 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T11:09:16.448 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T11:09:16.633 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T11:09:16.677 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T11:09:16.870 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T11:09:16.920 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T11:09:16.920 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T11:09:16.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T11:09:17.017 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:09:17.017 INFO:tasks.ceph:role: osd.0 2024-04-25T11:09:17.017 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:09:17.017 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T11:09:17.101 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:09:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:09:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:09:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:09:17.102 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:09:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:09:17.102 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:09:17.102 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:09:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:09:17.102 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:09:17.135 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-04-25T11:09:17.141 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-04-25T11:09:17.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T11:09:17.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T11:09:17.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T11:09:17.411 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:09:17.411 INFO:tasks.ceph:role: osd.1 2024-04-25T11:09:17.411 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:09:17.411 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T11:09:17.491 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:09:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:09:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:09:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:09:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:09:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:09:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:09:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:09:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:09:17.492 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:09:17.525 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-04-25T11:09:17.532 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-04-25T11:09:17.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T11:09:17.640 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T11:09:17.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T11:09:17.796 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:09:17.796 INFO:tasks.ceph:role: osd.2 2024-04-25T11:09:17.796 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:09:17.796 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T11:09:17.883 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:09:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:09:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:09:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:09:17.884 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:09:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:09:17.884 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:09:17.884 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:09:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:09:17.884 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:09:17.918 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-04-25T11:09:17.923 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-04-25T11:09:17.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T11:09:18.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T11:09:18.112 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T11:09:18.196 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T11:09:18.196 INFO:tasks.ceph:role: osd.3 2024-04-25T11:09:18.197 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:09:18.197 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T11:09:18.280 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:09:18.280 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:09:18.280 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:09:18.280 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:09:18.280 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:09:18.280 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:09:18.280 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:09:18.280 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:09:18.280 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:09:18.280 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:09:18.313 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-04-25T11:09:18.319 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-04-25T11:09:18.319 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T11:09:18.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T11:09:18.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:18.714+0000 7f9a7761c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T11:09:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:18.714+0000 7f9a7761c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T11:09:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:18.714+0000 7f9a7761c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:09:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:18.714+0000 7f9a7761c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:09:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:18.714+0000 7f9a7761c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T11:09:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:18.718+0000 7f9a7761c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T11:09:21.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T11:09:21.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:22.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:22.037+0000 7f0aa0e0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T11:09:22.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:22.038+0000 7f0aa0e0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T11:09:22.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:22.038+0000 7f0aa0e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:09:22.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:22.038+0000 7f0aa0e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:09:22.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:22.038+0000 7f0aa0e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T11:09:22.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:22.040+0000 7f0aa0e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T11:09:25.280 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T11:09:25.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:25.564+0000 7f798191d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T11:09:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:25.565+0000 7f798191d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T11:09:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:25.565+0000 7f798191d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:09:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:25.565+0000 7f798191d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:09:25.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:25.565+0000 7f798191d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T11:09:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:25.567+0000 7f798191d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T11:09:28.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T11:09:28.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:29.110+0000 7faa3d0e3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T11:09:29.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:29.111+0000 7faa3d0e3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T11:09:29.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:29.111+0000 7faa3d0e3640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:09:29.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:29.111+0000 7faa3d0e3640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:09:29.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:29.111+0000 7faa3d0e3640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T11:09:29.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:29.113+0000 7faa3d0e3640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T11:09:32.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T11:09:32.425 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T11:09:32.516 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:09:32.516 INFO:tasks.ceph:role: osd.4 2024-04-25T11:09:32.516 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:09:32.517 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T11:09:32.606 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:09:32.606 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:09:32.606 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:09:32.607 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:09:32.607 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:09:32.607 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:09:32.607 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:09:32.607 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:09:32.607 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:09:32.607 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:09:33.012 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T11:09:33.021 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T11:09:33.021 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T11:09:33.192 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T11:09:33.281 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T11:09:33.350 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:09:33.350 INFO:tasks.ceph:role: osd.5 2024-04-25T11:09:33.350 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:09:33.350 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T11:09:33.423 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:09:33.424 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:09:33.424 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:09:33.424 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:09:33.424 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:09:33.424 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:09:33.424 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:09:33.424 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:09:33.424 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:09:33.424 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:09:33.779 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T11:09:33.784 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T11:09:33.784 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T11:09:33.891 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T11:09:33.964 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T11:09:34.027 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:09:34.027 INFO:tasks.ceph:role: osd.6 2024-04-25T11:09:34.027 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:09:34.027 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T11:09:34.097 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:09:34.097 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:09:34.097 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:09:34.098 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:09:34.098 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:09:34.098 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:09:34.098 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:09:34.098 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:09:34.098 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:09:34.098 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:09:34.487 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T11:09:34.492 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T11:09:34.492 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T11:09:34.598 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T11:09:34.668 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T11:09:34.729 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T11:09:34.729 INFO:tasks.ceph:role: osd.7 2024-04-25T11:09:34.730 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:09:34.730 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T11:09:34.805 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T11:09:34.805 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T11:09:34.805 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T11:09:34.805 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T11:09:34.805 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T11:09:34.806 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T11:09:34.806 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T11:09:34.806 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T11:09:34.806 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T11:09:34.806 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T11:09:35.227 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T11:09:35.234 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T11:09:35.234 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T11:09:35.392 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T11:09:35.486 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:35.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:35.708+0000 7efd28c11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T11:09:35.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:35.708+0000 7efd28c11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T11:09:35.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:35.708+0000 7efd28c11640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:09:35.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:35.708+0000 7efd28c11640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:09:35.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:35.708+0000 7efd28c11640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T11:09:35.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:35.712+0000 7efd28c11640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T11:09:38.946 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T11:09:39.034 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:39.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:39.228+0000 7f700d00f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T11:09:39.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:39.229+0000 7f700d00f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T11:09:39.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:39.229+0000 7f700d00f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:09:39.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:39.229+0000 7f700d00f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:09:39.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:39.229+0000 7f700d00f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T11:09:39.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:39.231+0000 7f700d00f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T11:09:40.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:09:40.300 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:09:42.195 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T11:09:42.276 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:42.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:42.481+0000 7fba90731640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T11:09:42.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:42.482+0000 7fba90731640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T11:09:42.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:42.482+0000 7fba90731640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:09:42.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:42.482+0000 7fba90731640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:09:42.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:42.482+0000 7fba90731640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T11:09:42.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:42.484+0000 7fba90731640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T11:09:45.721 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T11:09:45.804 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:46.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:46.002+0000 7fa1ebf2c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T11:09:46.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:46.003+0000 7fa1ebf2c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T11:09:46.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:46.003+0000 7fa1ebf2c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:09:46.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:46.003+0000 7fa1ebf2c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:09:46.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:46.003+0000 7fa1ebf2c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T11:09:46.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:46.005+0000 7fa1ebf2c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T11:09:49.241 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T11:09:49.324 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T11:09:49.324 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:49.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T11:09:49.415 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:49.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T11:09:49.500 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:49.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T11:09:49.591 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:49.591 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T11:09:49.683 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:49.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T11:09:49.772 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:49.772 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T11:09:49.852 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:49.852 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T11:09:49.930 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:49.930 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T11:09:50.020 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:50.020 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T11:09:50.106 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:50.106 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T11:09:50.186 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:50.186 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T11:09:50.263 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:50.264 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T11:09:50.343 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:50.343 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T11:09:50.419 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:50.419 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T11:09:50.499 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:50.499 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T11:09:50.559 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:50.559 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T11:09:50.615 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:50.616 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T11:09:50.674 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:50.674 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T11:09:50.724 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T11:09:50.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T11:09:50.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T11:09:50.809 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.y] 2024-04-25T11:09:50.809 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w== 2024-04-25T11:09:50.809 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-04-25T11:09:50.809 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w== 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw== 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.0] 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA== 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.1] 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg== 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.2] 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw== 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.3] 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ== 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.x] 2024-04-25T11:09:50.810 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw== 2024-04-25T11:09:50.813 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-04-25T11:09:50.814 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw== 2024-04-25T11:09:50.814 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-04-25T11:09:50.814 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A== 2024-04-25T11:09:50.815 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.4] 2024-04-25T11:09:50.815 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw== 2024-04-25T11:09:50.815 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.5] 2024-04-25T11:09:50.815 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng== 2024-04-25T11:09:50.815 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.6] 2024-04-25T11:09:50.815 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ== 2024-04-25T11:09:50.815 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.7] 2024-04-25T11:09:50.815 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A== 2024-04-25T11:09:50.815 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T11:09:50.815 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:09:50.815 INFO:teuthology.orchestra.run.smithi087.stdout:[client.2] 2024-04-25T11:09:50.816 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg== 2024-04-25T11:09:50.816 INFO:teuthology.orchestra.run.smithi087.stdout:[client.1] 2024-04-25T11:09:50.816 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA== 2024-04-25T11:09:50.816 INFO:teuthology.orchestra.run.smithi087.stdout:[client.3] 2024-04-25T11:09:50.816 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ== 2024-04-25T11:09:50.817 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2024-04-25T11:09:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w== 2024-04-25T11:09:50.818 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-04-25T11:09:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w== 2024-04-25T11:09:50.818 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-04-25T11:09:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw== 2024-04-25T11:09:50.818 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-04-25T11:09:50.818 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA== 2024-04-25T11:09:50.819 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-04-25T11:09:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg== 2024-04-25T11:09:50.819 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-04-25T11:09:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw== 2024-04-25T11:09:50.819 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-04-25T11:09:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ== 2024-04-25T11:09:50.819 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-04-25T11:09:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw== 2024-04-25T11:09:50.819 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-04-25T11:09:50.819 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw== 2024-04-25T11:09:50.819 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A== 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw== 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng== 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ== 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A== 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout:[client.2] 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg== 2024-04-25T11:09:50.820 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2024-04-25T11:09:50.821 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA== 2024-04-25T11:09:50.821 INFO:teuthology.orchestra.run.smithi007.stdout:[client.3] 2024-04-25T11:09:50.821 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ== 2024-04-25T11:09:50.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:09:50.866 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:09:51.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:09:51.088 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:09:51.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:09:51.308 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:09:51.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:51.529 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:51.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:51.750 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:51.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:51.963 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:52.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:52.177 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:52.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:09:52.389 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T11:09:52.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:09:52.607 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:09:52.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:09:52.824 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T11:09:53.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:53.042 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:53.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:53.255 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:53.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:53.471 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:53.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:53.684 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T11:09:53.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:09:53.901 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:09:54.112 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:09:54.119 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:09:54.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:09:54.343 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:09:54.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:09:54.562 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T11:09:54.776 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T11:09:54.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T11:09:54.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:09:55.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T11:09:55.176 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T11:09:55.266 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:09:55.495 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T11:09:55.559 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T11:09:55.633 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T11:09:55.849 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T11:09:55.927 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:55.936 DEBUG:teuthology.orchestra.run.smithi087:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T11:09:55.990 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T11:09:55.991 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T11:09:55.991 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T11:09:55.996 INFO:tasks.ceph.mon.a:Started 2024-04-25T11:09:55.997 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T11:09:55.997 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T11:09:56.002 INFO:tasks.ceph.mon.b:Started 2024-04-25T11:09:56.002 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T11:09:56.003 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T11:09:56.008 INFO:tasks.ceph.mon.c:Started 2024-04-25T11:09:56.008 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T11:09:56.008 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T11:09:56.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T11:09:56.014 INFO:tasks.ceph.mgr.y:Started 2024-04-25T11:09:56.014 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T11:09:56.015 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T11:09:56.019 INFO:tasks.ceph.mgr.x:Started 2024-04-25T11:09:56.020 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T11:09:56.020 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T11:09:56.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.266+0000 7f40d7200640 1 Processor -- start 2024-04-25T11:09:56.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.266+0000 7f40d7200640 1 -- start start 2024-04-25T11:09:56.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.268+0000 7f40d7200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 0x7f40d80b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.268+0000 7f40d7200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40d80d0ca0 con 0x7f40d80d0480 2024-04-25T11:09:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.268+0000 7f40d7200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40d80ce570 con 0x7f40d81583c0 2024-04-25T11:09:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.268+0000 7f40d7200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f40d815f190 con 0x7f40d80d0890 2024-04-25T11:09:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.269+0000 7f40cfe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f40d80d0480 0x7f40d806eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47846/0 (socket says 172.21.15.7:47846) 2024-04-25T11:09:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.269+0000 7f40cfe00640 1 -- 172.21.15.7:0/1320922152 learned_addr learned my addr 172.21.15.7:0/1320922152 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:09:56.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.269+0000 7f40d6800640 1 --2- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 0x7f40d80b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.269+0000 7f40d6800640 1 -- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 msgr2=0x7f40d80b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T11:09:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.269+0000 7f40d6800640 1 -- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 msgr2=0x7f40d80b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:09:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.269+0000 7f40d6800640 1 --2- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 0x7f40d80b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:09:56.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.269+0000 7f40d6800640 1 --2- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 0x7f40d80b0760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T11:09:56.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.471+0000 7f40d6800640 1 --2- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 0x7f40d80b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:56.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.471+0000 7f40d6800640 1 -- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 msgr2=0x7f40d80b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T11:09:56.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.471+0000 7f40d6800640 1 -- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 msgr2=0x7f40d80b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T11:09:56.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.471+0000 7f40d6800640 1 --2- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 0x7f40d80b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T11:09:56.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.471+0000 7f40d6800640 1 --2- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 0x7f40d80b0760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T11:09:56.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.539+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 145020085 0 0) 0x7f40d80ce570 con 0x7f40d81583c0 2024-04-25T11:09:56.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.539+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40bc003520 con 0x7f40d81583c0 2024-04-25T11:09:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.540+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 <== mon.1 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4238250720 0 0) 0x7f40bc003520 con 0x7f40d81583c0 2024-04-25T11:09:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.540+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 msgr2=0x7f40d80b0760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T11:09:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.540+0000 7f40d5400640 1 --2- 172.21.15.7:0/1320922152 >> v2:172.21.15.87:3300/0 conn(0x7f40d80d0890 0x7f40d80b0760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.540+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 >> v1:172.21.15.7:6789/0 conn(0x7f40d80d0480 legacy=0x7f40d806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.540+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d80ce850 con 0x7f40d81583c0 2024-04-25T11:09:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.540+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 <== mon.1 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f40c8001670 con 0x7f40d81583c0 2024-04-25T11:09:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.540+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 >> v1:172.21.15.87:6789/0 conn(0x7f40d81583c0 legacy=0x7f40d80cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.540+0000 7f40d5400640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40bc003a00 0x7f40bc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.541+0000 7f40d5400640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40bc004800 0x7f40bc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.541+0000 7f40d5400640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc005350 0x7f40bc00ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.541+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40d815f190 con 0x7f40bc003a00 2024-04-25T11:09:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.541+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc00c2e0 con 0x7f40bc004800 2024-04-25T11:09:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.541+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f40bc00c4e0 con 0x7f40bc005350 2024-04-25T11:09:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.542+0000 7f40cfe00640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40bc004800 0x7f40bc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.542+0000 7f40d6800640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc005350 0x7f40bc00ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.542+0000 7f40d5e00640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40bc003a00 0x7f40bc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.543+0000 7f40cfe00640 1 -- 172.21.15.7:0/1320922152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc005350 msgr2=0x7f40bc00ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:09:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.543+0000 7f40cfe00640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc005350 0x7f40bc00ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.543+0000 7f40cfe00640 1 -- 172.21.15.7:0/1320922152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40bc003a00 msgr2=0x7f40bc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:09:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.543+0000 7f40cfe00640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40bc003a00 0x7f40bc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.543+0000 7f40cfe00640 1 -- 172.21.15.7:0/1320922152 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc003520 con 0x7f40bc004800 2024-04-25T11:09:56.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.543+0000 7f40d6800640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc005350 0x7f40bc00ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:09:56.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.543+0000 7f40d6800640 1 -- 172.21.15.7:0/1320922152 reap_dead start 2024-04-25T11:09:56.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.545+0000 7f40cfe00640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40bc004800 0x7f40bc004c10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40d80b4740 tx=0x7f40bc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f834f0d4d692461 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:56.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.548+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0001460 con 0x7f40bc004800 2024-04-25T11:09:56.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.548+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c002e070 con 0x7f40bc004800 2024-04-25T11:09:56.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.548+0000 7f40d5400640 1 -- 172.21.15.7:0/1320922152 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c002ad30 con 0x7f40bc004800 2024-04-25T11:09:56.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.549+0000 7f40d7200640 1 -- 172.21.15.7:0/1320922152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40bc004800 msgr2=0x7f40bc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:56.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.549+0000 7f40d7200640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40bc004800 0x7f40bc004c10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f40d80b4740 tx=0x7f40bc005830 comp rx=0 tx=0).stop 2024-04-25T11:09:56.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.550+0000 7f40d7200640 1 -- 172.21.15.7:0/1320922152 shutdown_connections 2024-04-25T11:09:56.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.550+0000 7f40d7200640 1 --2- 172.21.15.7:0/1320922152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40bc004800 0x7f40bc004c10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:56.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.550+0000 7f40d7200640 1 -- 172.21.15.7:0/1320922152 >> 172.21.15.7:0/1320922152 conn(0x7f40d80740f0 msgr2=0x7f40d806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:56.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.550+0000 7f40d7200640 1 -- 172.21.15.7:0/1320922152 shutdown_connections 2024-04-25T11:09:56.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.550+0000 7f40d7200640 1 -- 172.21.15.7:0/1320922152 wait complete. 2024-04-25T11:09:56.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.553+0000 7f40d7200640 1 Processor -- start 2024-04-25T11:09:56.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.553+0000 7f40d7200640 1 -- start start 2024-04-25T11:09:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.555+0000 7f40d7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc004800 0x7f40d81fa070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.555+0000 7f40d7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d80c99e0 0x7f40d8143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.555+0000 7f40d7200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d806eea0 0x7f40d806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.555+0000 7f40d7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40d80cf0f0 con 0x7f40d80c99e0 2024-04-25T11:09:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.555+0000 7f40d7200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f40d80cee10 con 0x7f40d806eea0 2024-04-25T11:09:56.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.555+0000 7f40d7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f40d80cf610 con 0x7f40bc004800 2024-04-25T11:09:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.556+0000 7f40cfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d80c99e0 0x7f40d8143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.556+0000 7f40cfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d80c99e0 0x7f40d8143ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55792/0 (socket says 172.21.15.7:55792) 2024-04-25T11:09:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.556+0000 7f40cfe00640 1 -- 172.21.15.7:0/641745768 learned_addr learned my addr 172.21.15.7:0/641745768 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:09:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.556+0000 7f40d6800640 1 --2- 172.21.15.7:0/641745768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d806eea0 0x7f40d806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:56.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.556+0000 7f40d5e00640 1 --2- 172.21.15.7:0/641745768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc004800 0x7f40d81fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.557+0000 7f40cfe00640 1 -- 172.21.15.7:0/641745768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc004800 msgr2=0x7f40d81fa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.557+0000 7f40cfe00640 1 --2- 172.21.15.7:0/641745768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc004800 0x7f40d81fa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.557+0000 7f40cfe00640 1 -- 172.21.15.7:0/641745768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d806eea0 msgr2=0x7f40d806f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.557+0000 7f40cfe00640 1 --2- 172.21.15.7:0/641745768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d806eea0 0x7f40d806f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.557+0000 7f40cfe00640 1 -- 172.21.15.7:0/641745768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d80ca3a0 con 0x7f40d80c99e0 2024-04-25T11:09:56.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.557+0000 7f40cfe00640 1 --2- 172.21.15.7:0/641745768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d80c99e0 0x7f40d8143ed0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f40c00018d0 tx=0x7f40c0001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.562+0000 7f40ce000640 1 -- 172.21.15.7:0/641745768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0044070 con 0x7f40d80c99e0 2024-04-25T11:09:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.562+0000 7f40ce000640 1 -- 172.21.15.7:0/641745768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c0004c10 con 0x7f40d80c99e0 2024-04-25T11:09:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.562+0000 7f40ce000640 1 -- 172.21.15.7:0/641745768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0007070 con 0x7f40d80c99e0 2024-04-25T11:09:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.562+0000 7f40d7200640 1 -- 172.21.15.7:0/641745768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d806f7f0 con 0x7f40d80c99e0 2024-04-25T11:09:56.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.562+0000 7f40d7200640 1 -- 172.21.15.7:0/641745768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d80ca0c0 con 0x7f40d80c99e0 2024-04-25T11:09:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.563+0000 7f40ce000640 1 -- 172.21.15.7:0/641745768 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f40c0030040 con 0x7f40d80c99e0 2024-04-25T11:09:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.563+0000 7f40ce000640 1 -- 172.21.15.7:0/641745768 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f40c003fa60 con 0x7f40d80c99e0 2024-04-25T11:09:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.564+0000 7f40d7200640 1 -- 172.21.15.7:0/641745768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d8050740 con 0x7f40d80c99e0 2024-04-25T11:09:56.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.571+0000 7f40ce000640 1 -- 172.21.15.7:0/641745768 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f40d806f7f0 con 0x7f40d80c99e0 2024-04-25T11:09:56.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.646+0000 7f40d7200640 1 -- 172.21.15.7:0/641745768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f40d8070dd0 con 0x7f40d80c99e0 2024-04-25T11:09:56.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.650+0000 7f40ce000640 1 -- 172.21.15.7:0/641745768 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f40d80ca3a0 con 0x7f40d80c99e0 2024-04-25T11:09:56.650 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-04-25T11:09:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.652+0000 7f40d7200640 1 -- 172.21.15.7:0/641745768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d80c99e0 msgr2=0x7f40d8143ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.652+0000 7f40d7200640 1 --2- 172.21.15.7:0/641745768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d80c99e0 0x7f40d8143ed0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f40c00018d0 tx=0x7f40c0001190 comp rx=0 tx=0).stop 2024-04-25T11:09:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.652+0000 7f40d7200640 1 -- 172.21.15.7:0/641745768 shutdown_connections 2024-04-25T11:09:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.652+0000 7f40d7200640 1 --2- 172.21.15.7:0/641745768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40d806eea0 0x7f40d806f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.652+0000 7f40d7200640 1 --2- 172.21.15.7:0/641745768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d80c99e0 0x7f40d8143ed0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.652+0000 7f40d7200640 1 --2- 172.21.15.7:0/641745768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc004800 0x7f40d81fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.652+0000 7f40d7200640 1 -- 172.21.15.7:0/641745768 >> 172.21.15.7:0/641745768 conn(0x7f40d80740f0 msgr2=0x7f40d8070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:56.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.652+0000 7f40d7200640 1 -- 172.21.15.7:0/641745768 shutdown_connections 2024-04-25T11:09:56.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:09:56.652+0000 7f40d7200640 1 -- 172.21.15.7:0/641745768 wait complete. 2024-04-25T11:09:56.678 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T11:09:56.678 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T11:09:56.678 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:56.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T11:09:56.716 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:56.716+0000 7f5f4ba0f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:09:56.751 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:56.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T11:09:56.872 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:56.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T11:09:56.908 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:56.908+0000 7f5f4ba0f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:09:56.948 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:09:56.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T11:09:56.978 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:56.977+0000 7f5f4ba0f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:09:57.029 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:57.029+0000 7f7071be1040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T11:09:57.052 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:57.052 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T11:09:57.134 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:57.134 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T11:09:57.211 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:57.211 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T11:09:57.222 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:57.221+0000 7f7071be1040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:09:57.289 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:57.289+0000 7f5f4ba0f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:09:57.290 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:57.289+0000 7f7071be1040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T11:09:57.295 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:09:57.295 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T11:09:57.347 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:57.346+0000 7f5f4ba0f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:09:57.412 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new da1b353f-d9c1-458d-a27f-e53c50446727 0 2024-04-25T11:09:57.595 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:57.594+0000 7f7071be1040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T11:09:57.651 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:57.650+0000 7f7071be1040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T11:09:57.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.702+0000 7f6353e00640 1 Processor -- start 2024-04-25T11:09:57.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.702+0000 7f6353e00640 1 -- start start 2024-04-25T11:09:57.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.704+0000 7f6353e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63540d0fb0 0x7f635415fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:57.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.704+0000 7f6353e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63540b7af0 0x7f63540d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:57.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.704+0000 7f6353e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63540d13c0 con 0x7f63540b8e50 2024-04-25T11:09:57.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.704+0000 7f6353e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6354163f10 con 0x7f63540b7af0 2024-04-25T11:09:57.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.704+0000 7f6353e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f63541641f0 con 0x7f63540d0fb0 2024-04-25T11:09:57.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.704+0000 7f6352a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63540d0fb0 0x7f635415fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:57.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.704+0000 7f6352a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63540d0fb0 0x7f635415fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52048/0 (socket says 172.21.15.87:52048) 2024-04-25T11:09:57.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.704+0000 7f6352a00640 1 -- 172.21.15.87:0/4039452539 learned_addr learned my addr 172.21.15.87:0/4039452539 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T11:09:57.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.704+0000 7f6353400640 1 --2- 172.21.15.87:0/4039452539 >> v2:172.21.15.7:3300/0 conn(0x7f63540b7af0 0x7f63540d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:57.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.704+0000 7f6352a00640 1 -- 172.21.15.87:0/4039452539 >> v2:172.21.15.7:3300/0 conn(0x7f63540b7af0 msgr2=0x7f63540d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:09:57.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.704+0000 7f6352a00640 1 --2- 172.21.15.87:0/4039452539 >> v2:172.21.15.7:3300/0 conn(0x7f63540b7af0 0x7f63540d45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:57.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.705+0000 7f6352a00640 1 -- 172.21.15.87:0/4039452539 >> v1:172.21.15.7:6789/0 conn(0x7f63540b8e50 legacy=0x7f63540b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:09:57.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.705+0000 7f6352a00640 1 -- 172.21.15.87:0/4039452539 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63541644d0 con 0x7f63540d0fb0 2024-04-25T11:09:57.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.706+0000 7f6352a00640 1 --2- 172.21.15.87:0/4039452539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63540d0fb0 0x7f635415fea0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f63540d33d0 tx=0x7f634402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f541bf4089e317da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.706+0000 7f6352000640 1 -- 172.21.15.87:0/4039452539 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344037070 con 0x7f63540d0fb0 2024-04-25T11:09:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.706+0000 7f6352000640 1 -- 172.21.15.87:0/4039452539 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6344032440 con 0x7f63540d0fb0 2024-04-25T11:09:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.706+0000 7f6352000640 1 -- 172.21.15.87:0/4039452539 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344032810 con 0x7f63540d0fb0 2024-04-25T11:09:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.707+0000 7f6353e00640 1 -- 172.21.15.87:0/4039452539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63540d0fb0 msgr2=0x7f635415fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.707+0000 7f6353e00640 1 --2- 172.21.15.87:0/4039452539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63540d0fb0 0x7f635415fea0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f63540d33d0 tx=0x7f634402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:09:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.707+0000 7f6353e00640 1 -- 172.21.15.87:0/4039452539 shutdown_connections 2024-04-25T11:09:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.707+0000 7f6353e00640 1 --2- 172.21.15.87:0/4039452539 >> v2:172.21.15.7:3300/0 conn(0x7f63540b7af0 0x7f63540d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.707+0000 7f6353e00640 1 --2- 172.21.15.87:0/4039452539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63540d0fb0 0x7f635415fea0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.707+0000 7f6353e00640 1 -- 172.21.15.87:0/4039452539 >> 172.21.15.87:0/4039452539 conn(0x7f63540d23e0 msgr2=0x7f63540d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.707+0000 7f6353e00640 1 -- 172.21.15.87:0/4039452539 shutdown_connections 2024-04-25T11:09:57.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.707+0000 7f6353e00640 1 -- 172.21.15.87:0/4039452539 wait complete. 2024-04-25T11:09:57.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.711+0000 7f6353e00640 1 Processor -- start 2024-04-25T11:09:57.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.711+0000 7f6353e00640 1 -- start start 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.713+0000 7f6353e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63540b8e50 0x7f635415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.713+0000 7f6353e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63540d0fb0 0x7f63540c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.713+0000 7f6353e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f635415ff10 0x7f6354160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.713+0000 7f6353e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63540b9260 con 0x7f63540d0fb0 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.713+0000 7f6353e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f63540b4bc0 con 0x7f63540b8e50 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.713+0000 7f6353e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6354163a80 con 0x7f635415ff10 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f6352a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63540b8e50 0x7f635415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f6352a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63540b8e50 0x7f635415a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52062/0 (socket says 172.21.15.87:52062) 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f6352a00640 1 -- 172.21.15.87:0/2004467189 learned_addr learned my addr 172.21.15.87:0/2004467189 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f6353400640 1 --2- 172.21.15.87:0/2004467189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f635415ff10 0x7f6354160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f634be00640 1 --2- 172.21.15.87:0/2004467189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63540d0fb0 0x7f63540c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f6352a00640 1 -- 172.21.15.87:0/2004467189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f635415ff10 msgr2=0x7f6354160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:57.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f6352a00640 1 --2- 172.21.15.87:0/2004467189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f635415ff10 0x7f6354160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:57.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f6352a00640 1 -- 172.21.15.87:0/2004467189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63540d0fb0 msgr2=0x7f63540c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:57.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f6352a00640 1 --2- 172.21.15.87:0/2004467189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63540d0fb0 0x7f63540c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:57.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f6352a00640 1 -- 172.21.15.87:0/2004467189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63540c8af0 con 0x7f63540b8e50 2024-04-25T11:09:57.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f634be00640 1 --2- 172.21.15.87:0/2004467189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63540d0fb0 0x7f63540c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:09:57.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.714+0000 7f6352a00640 1 --2- 172.21.15.87:0/2004467189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63540b8e50 0x7f635415a5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f634402a1e0 tx=0x7f63440046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:57.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.715+0000 7f634aa00640 1 -- 172.21.15.87:0/2004467189 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344041020 con 0x7f63540b8e50 2024-04-25T11:09:57.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.715+0000 7f634aa00640 1 -- 172.21.15.87:0/2004467189 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63440306c0 con 0x7f63540b8e50 2024-04-25T11:09:57.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.715+0000 7f634aa00640 1 -- 172.21.15.87:0/2004467189 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344037040 con 0x7f63540b8e50 2024-04-25T11:09:57.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.715+0000 7f6353e00640 1 -- 172.21.15.87:0/2004467189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6354159530 con 0x7f63540b8e50 2024-04-25T11:09:57.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.715+0000 7f6353e00640 1 -- 172.21.15.87:0/2004467189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6354159af0 con 0x7f63540b8e50 2024-04-25T11:09:57.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.715+0000 7f634aa00640 1 -- 172.21.15.87:0/2004467189 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6344044020 con 0x7f63540b8e50 2024-04-25T11:09:57.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.715+0000 7f634aa00640 1 -- 172.21.15.87:0/2004467189 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f634403dda0 con 0x7f63540b8e50 2024-04-25T11:09:57.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.716+0000 7f6353e00640 1 -- 172.21.15.87:0/2004467189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6354050740 con 0x7f63540b8e50 2024-04-25T11:09:57.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.721+0000 7f634aa00640 1 -- 172.21.15.87:0/2004467189 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f634404b030 con 0x7f63540b8e50 2024-04-25T11:09:57.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.800+0000 7f6353e00640 1 -- 172.21.15.87:0/2004467189 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "da1b353f-d9c1-458d-a27f-e53c50446727", "id": 0} v 0) -- 0x7f63540d3d40 con 0x7f63540b8e50 2024-04-25T11:09:57.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.806+0000 7f634aa00640 1 -- 172.21.15.87:0/2004467189 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "da1b353f-d9c1-458d-a27f-e53c50446727", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f63540c8af0 con 0x7f63540b8e50 2024-04-25T11:09:57.807 INFO:teuthology.orchestra.run.smithi087.stdout:0 2024-04-25T11:09:57.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.808+0000 7f6353e00640 1 -- 172.21.15.87:0/2004467189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63540b8e50 msgr2=0x7f635415a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:57.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.808+0000 7f6353e00640 1 --2- 172.21.15.87:0/2004467189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63540b8e50 0x7f635415a5d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f634402a1e0 tx=0x7f63440046c0 comp rx=0 tx=0).stop 2024-04-25T11:09:57.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.808+0000 7f6353e00640 1 -- 172.21.15.87:0/2004467189 shutdown_connections 2024-04-25T11:09:57.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.808+0000 7f6353e00640 1 --2- 172.21.15.87:0/2004467189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f635415ff10 0x7f6354160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:57.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.808+0000 7f6353e00640 1 --2- 172.21.15.87:0/2004467189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63540d0fb0 0x7f63540c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:57.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.808+0000 7f6353e00640 1 --2- 172.21.15.87:0/2004467189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63540b8e50 0x7f635415a5d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:57.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.808+0000 7f6353e00640 1 -- 172.21.15.87:0/2004467189 >> 172.21.15.87:0/2004467189 conn(0x7f63540d23e0 msgr2=0x7f63540d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:57.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.809+0000 7f6353e00640 1 -- 172.21.15.87:0/2004467189 shutdown_connections 2024-04-25T11:09:57.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:57.809+0000 7f6353e00640 1 -- 172.21.15.87:0/2004467189 wait complete. 2024-04-25T11:09:57.833 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 0eca90e3-34f2-4488-96b0-249fe063693a 1 2024-04-25T11:09:58.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.077+0000 7fc016400640 1 Processor -- start 2024-04-25T11:09:58.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.077+0000 7fc016400640 1 -- start start 2024-04-25T11:09:58.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.079+0000 7fc016400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0100d0fb0 0x7fc01015fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc016400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc0100b7af0 0x7fc0100d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc016400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0100d13c0 con 0x7fc0100b8e50 2024-04-25T11:09:58.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc016400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc010163f10 con 0x7fc0100b7af0 2024-04-25T11:09:58.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc016400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc0101641f0 con 0x7fc0100d0fb0 2024-04-25T11:09:58.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc015000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0100d0fb0 0x7fc01015fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc015000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0100d0fb0 0x7fc01015fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52076/0 (socket says 172.21.15.87:52076) 2024-04-25T11:09:58.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc015000640 1 -- 172.21.15.87:0/467775402 learned_addr learned my addr 172.21.15.87:0/467775402 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T11:09:58.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc015000640 1 -- 172.21.15.87:0/467775402 >> v2:172.21.15.7:3300/0 conn(0x7fc0100b7af0 msgr2=0x7fc0100d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:09:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc015a00640 1 --2- 172.21.15.87:0/467775402 >> v2:172.21.15.7:3300/0 conn(0x7fc0100b7af0 0x7fc0100d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc015000640 1 --2- 172.21.15.87:0/467775402 >> v2:172.21.15.7:3300/0 conn(0x7fc0100b7af0 0x7fc0100d45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc015000640 1 -- 172.21.15.87:0/467775402 >> v1:172.21.15.7:6789/0 conn(0x7fc0100b8e50 legacy=0x7fc0100b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:09:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.080+0000 7fc015000640 1 -- 172.21.15.87:0/467775402 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0101644d0 con 0x7fc0100d0fb0 2024-04-25T11:09:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.081+0000 7fc015000640 1 --2- 172.21.15.87:0/467775402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0100d0fb0 0x7fc01015fea0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc0100d33d0 tx=0x7fc00002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c12e3dda8118b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:58.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.082+0000 7fc00fe00640 1 -- 172.21.15.87:0/467775402 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000036070 con 0x7fc0100d0fb0 2024-04-25T11:09:58.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.082+0000 7fc00fe00640 1 -- 172.21.15.87:0/467775402 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc000031600 con 0x7fc0100d0fb0 2024-04-25T11:09:58.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.082+0000 7fc00fe00640 1 -- 172.21.15.87:0/467775402 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0000319d0 con 0x7fc0100d0fb0 2024-04-25T11:09:58.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.083+0000 7fc016400640 1 -- 172.21.15.87:0/467775402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0100d0fb0 msgr2=0x7fc01015fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.083+0000 7fc016400640 1 --2- 172.21.15.87:0/467775402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0100d0fb0 0x7fc01015fea0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc0100d33d0 tx=0x7fc00002a660 comp rx=0 tx=0).stop 2024-04-25T11:09:58.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.083+0000 7fc016400640 1 -- 172.21.15.87:0/467775402 shutdown_connections 2024-04-25T11:09:58.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.083+0000 7fc016400640 1 --2- 172.21.15.87:0/467775402 >> v2:172.21.15.7:3300/0 conn(0x7fc0100b7af0 0x7fc0100d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.083+0000 7fc016400640 1 --2- 172.21.15.87:0/467775402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0100d0fb0 0x7fc01015fea0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.083+0000 7fc016400640 1 -- 172.21.15.87:0/467775402 >> 172.21.15.87:0/467775402 conn(0x7fc0100d23e0 msgr2=0x7fc0100d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:58.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.083+0000 7fc016400640 1 -- 172.21.15.87:0/467775402 shutdown_connections 2024-04-25T11:09:58.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.083+0000 7fc016400640 1 -- 172.21.15.87:0/467775402 wait complete. 2024-04-25T11:09:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.085+0000 7fc016400640 1 Processor -- start 2024-04-25T11:09:58.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.085+0000 7fc016400640 1 -- start start 2024-04-25T11:09:58.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.087+0000 7fc016400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0100b8e50 0x7fc0101581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.087+0000 7fc016400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0100d0fb0 0x7fc0100cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.087+0000 7fc016400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0101f98f0 0x7fc0101f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.087+0000 7fc016400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0100b9260 con 0x7fc0100d0fb0 2024-04-25T11:09:58.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.087+0000 7fc016400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc0100b4bc0 con 0x7fc0100b8e50 2024-04-25T11:09:58.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.087+0000 7fc016400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc010163a80 con 0x7fc0101f98f0 2024-04-25T11:09:58.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.087+0000 7fc015000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0100b8e50 0x7fc0101581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.087+0000 7fc015000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0100b8e50 0x7fc0101581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52090/0 (socket says 172.21.15.87:52090) 2024-04-25T11:09:58.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.087+0000 7fc015000640 1 -- 172.21.15.87:0/3837699242 learned_addr learned my addr 172.21.15.87:0/3837699242 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T11:09:58.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.087+0000 7fc015a00640 1 --2- 172.21.15.87:0/3837699242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0101f98f0 0x7fc0101f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.087+0000 7fc007e00640 1 --2- 172.21.15.87:0/3837699242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0100d0fb0 0x7fc0100cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.088+0000 7fc015000640 1 -- 172.21.15.87:0/3837699242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0101f98f0 msgr2=0x7fc0101f62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.088+0000 7fc015000640 1 --2- 172.21.15.87:0/3837699242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0101f98f0 0x7fc0101f62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.088+0000 7fc015000640 1 -- 172.21.15.87:0/3837699242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0100d0fb0 msgr2=0x7fc0100cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.088+0000 7fc015000640 1 --2- 172.21.15.87:0/3837699242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0100d0fb0 0x7fc0100cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.088+0000 7fc015000640 1 -- 172.21.15.87:0/3837699242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0101fa500 con 0x7fc0100b8e50 2024-04-25T11:09:58.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.088+0000 7fc015000640 1 --2- 172.21.15.87:0/3837699242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0100b8e50 0x7fc0101581d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc000009f40 tx=0x7fc000004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.088+0000 7fc00d600640 1 -- 172.21.15.87:0/3837699242 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000036050 con 0x7fc0100b8e50 2024-04-25T11:09:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.088+0000 7fc00d600640 1 -- 172.21.15.87:0/3837699242 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc000042070 con 0x7fc0100b8e50 2024-04-25T11:09:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.088+0000 7fc016400640 1 -- 172.21.15.87:0/3837699242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0101fa7e0 con 0x7fc0100b8e50 2024-04-25T11:09:58.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.088+0000 7fc00d600640 1 -- 172.21.15.87:0/3837699242 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000009510 con 0x7fc0100b8e50 2024-04-25T11:09:58.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.089+0000 7fc016400640 1 -- 172.21.15.87:0/3837699242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0101579d0 con 0x7fc0100b8e50 2024-04-25T11:09:58.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.089+0000 7fc00d600640 1 -- 172.21.15.87:0/3837699242 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc000007c50 con 0x7fc0100b8e50 2024-04-25T11:09:58.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.089+0000 7fc00d600640 1 -- 172.21.15.87:0/3837699242 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fc00003d800 con 0x7fc0100b8e50 2024-04-25T11:09:58.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.090+0000 7fc016400640 1 -- 172.21.15.87:0/3837699242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc010050740 con 0x7fc0100b8e50 2024-04-25T11:09:58.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.094+0000 7fc00d600640 1 -- 172.21.15.87:0/3837699242 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc00003da50 con 0x7fc0100b8e50 2024-04-25T11:09:58.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.174+0000 7fc016400640 1 -- 172.21.15.87:0/3837699242 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0eca90e3-34f2-4488-96b0-249fe063693a", "id": 1} v 0) -- 0x7fc0100d3d40 con 0x7fc0100b8e50 2024-04-25T11:09:58.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.180+0000 7fc00d600640 1 -- 172.21.15.87:0/3837699242 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0eca90e3-34f2-4488-96b0-249fe063693a", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fc00003dc30 con 0x7fc0100b8e50 2024-04-25T11:09:58.182 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:09:58.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.182+0000 7fc016400640 1 -- 172.21.15.87:0/3837699242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0100b8e50 msgr2=0x7fc0101581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.182+0000 7fc016400640 1 --2- 172.21.15.87:0/3837699242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0100b8e50 0x7fc0101581d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc000009f40 tx=0x7fc000004a40 comp rx=0 tx=0).stop 2024-04-25T11:09:58.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.182+0000 7fc016400640 1 -- 172.21.15.87:0/3837699242 shutdown_connections 2024-04-25T11:09:58.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.182+0000 7fc016400640 1 --2- 172.21.15.87:0/3837699242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0101f98f0 0x7fc0101f62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.182+0000 7fc016400640 1 --2- 172.21.15.87:0/3837699242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0100d0fb0 0x7fc0100cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.182+0000 7fc016400640 1 --2- 172.21.15.87:0/3837699242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0100b8e50 0x7fc0101581d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.182+0000 7fc016400640 1 -- 172.21.15.87:0/3837699242 >> 172.21.15.87:0/3837699242 conn(0x7fc0100d23e0 msgr2=0x7fc0100b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:58.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.182+0000 7fc016400640 1 -- 172.21.15.87:0/3837699242 shutdown_connections 2024-04-25T11:09:58.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.182+0000 7fc016400640 1 -- 172.21.15.87:0/3837699242 wait complete. 2024-04-25T11:09:58.207 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 1d3b267f-385c-4080-a237-0caf41de8a05 2 2024-04-25T11:09:58.366 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:58.366+0000 7f5f4ba0f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:09:58.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.440+0000 7f89e2c00640 1 Processor -- start 2024-04-25T11:09:58.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.441+0000 7f89e2c00640 1 -- start start 2024-04-25T11:09:58.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.443+0000 7f89e2c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f89dc160310 0x7f89dc0d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.443+0000 7f89e2c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f89dc0b9920 0x7f89dc0b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.443+0000 7f89e2c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89dc160720 con 0x7f89dc0b79f0 2024-04-25T11:09:58.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.443+0000 7f89e2c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f89dc0d5640 con 0x7f89dc0b9920 2024-04-25T11:09:58.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.443+0000 7f89e2c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f89dc0d0f40 con 0x7f89dc160310 2024-04-25T11:09:58.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.443+0000 7f89dbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f89dc160310 0x7f89dc0d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.443+0000 7f89d3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f89dc0b79f0 0x7f89dc0b7e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56838/0 (socket says 172.21.15.87:56838) 2024-04-25T11:09:58.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.443+0000 7f89d3400640 1 -- 172.21.15.87:0/3086804902 learned_addr learned my addr 172.21.15.87:0/3086804902 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T11:09:58.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.443+0000 7f89e0c00640 1 --2- 172.21.15.87:0/3086804902 >> v2:172.21.15.7:3300/0 conn(0x7f89dc0b9920 0x7f89dc0b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.444+0000 7f89dbe00640 1 -- 172.21.15.87:0/3086804902 >> v2:172.21.15.7:3300/0 conn(0x7f89dc0b9920 msgr2=0x7f89dc0b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:09:58.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.444+0000 7f89dbe00640 1 --2- 172.21.15.87:0/3086804902 >> v2:172.21.15.7:3300/0 conn(0x7f89dc0b9920 0x7f89dc0b9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.444+0000 7f89dbe00640 1 -- 172.21.15.87:0/3086804902 >> v1:172.21.15.87:6789/0 conn(0x7f89dc0b79f0 legacy=0x7f89dc0b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.444+0000 7f89db400640 1 -- 172.21.15.87:0/3086804902 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4236685793 0 0) 0x7f89dc160720 con 0x7f89dc0b79f0 2024-04-25T11:09:58.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.444+0000 7f89dbe00640 1 -- 172.21.15.87:0/3086804902 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc0d1220 con 0x7f89dc160310 2024-04-25T11:09:58.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.445+0000 7f89dbe00640 1 --2- 172.21.15.87:0/3086804902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc160310 0x7f89dc0d5100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d6be0 tx=0x7f89cc02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e0e15bcc53314a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:58.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.445+0000 7f89db400640 1 -- 172.21.15.87:0/3086804902 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc007740 con 0x7f89dc160310 2024-04-25T11:09:58.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.445+0000 7f89db400640 1 -- 172.21.15.87:0/3086804902 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89cc0078e0 con 0x7f89dc160310 2024-04-25T11:09:58.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.445+0000 7f89db400640 1 -- 172.21.15.87:0/3086804902 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc007cb0 con 0x7f89dc160310 2024-04-25T11:09:58.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.446+0000 7f89e2c00640 1 -- 172.21.15.87:0/3086804902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc160310 msgr2=0x7f89dc0d5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.446+0000 7f89e2c00640 1 --2- 172.21.15.87:0/3086804902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc160310 0x7f89dc0d5100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0d6be0 tx=0x7f89cc02a680 comp rx=0 tx=0).stop 2024-04-25T11:09:58.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.446+0000 7f89e2c00640 1 -- 172.21.15.87:0/3086804902 shutdown_connections 2024-04-25T11:09:58.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.446+0000 7f89e2c00640 1 --2- 172.21.15.87:0/3086804902 >> v2:172.21.15.7:3300/0 conn(0x7f89dc0b9920 0x7f89dc0b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.446+0000 7f89e2c00640 1 --2- 172.21.15.87:0/3086804902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc160310 0x7f89dc0d5100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.446+0000 7f89e2c00640 1 -- 172.21.15.87:0/3086804902 >> 172.21.15.87:0/3086804902 conn(0x7f89dc0d2a50 msgr2=0x7f89dc0d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:58.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.446+0000 7f89e2c00640 1 -- 172.21.15.87:0/3086804902 shutdown_connections 2024-04-25T11:09:58.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.446+0000 7f89e2c00640 1 -- 172.21.15.87:0/3086804902 wait complete. 2024-04-25T11:09:58.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.448+0000 7f89e2c00640 1 Processor -- start 2024-04-25T11:09:58.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.449+0000 7f89e2c00640 1 -- start start 2024-04-25T11:09:58.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.450+0000 7f89e2c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc0b79f0 0x7f89dc1f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.450+0000 7f89e2c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc0b9920 0x7f89dc0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.450+0000 7f89e2c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89dc160310 0x7f89dc0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.450+0000 7f89e2c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc0d0bf0 con 0x7f89dc0b9920 2024-04-25T11:09:58.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.450+0000 7f89e2c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc0b9400 con 0x7f89dc0b79f0 2024-04-25T11:09:58.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.450+0000 7f89e2c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89dc0cc950 con 0x7f89dc160310 2024-04-25T11:09:58.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.450+0000 7f89dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc0b79f0 0x7f89dc1f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.450+0000 7f89dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc0b79f0 0x7f89dc1f6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52108/0 (socket says 172.21.15.87:52108) 2024-04-25T11:09:58.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.450+0000 7f89dbe00640 1 -- 172.21.15.87:0/2185509837 learned_addr learned my addr 172.21.15.87:0/2185509837 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T11:09:58.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.450+0000 7f89e0c00640 1 --2- 172.21.15.87:0/2185509837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89dc160310 0x7f89dc0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.450+0000 7f89d3400640 1 --2- 172.21.15.87:0/2185509837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc0b9920 0x7f89dc0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.451+0000 7f89dbe00640 1 -- 172.21.15.87:0/2185509837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89dc160310 msgr2=0x7f89dc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.451+0000 7f89dbe00640 1 --2- 172.21.15.87:0/2185509837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89dc160310 0x7f89dc0cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.451+0000 7f89dbe00640 1 -- 172.21.15.87:0/2185509837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc0b9920 msgr2=0x7f89dc0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.451+0000 7f89dbe00640 1 --2- 172.21.15.87:0/2185509837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc0b9920 0x7f89dc0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.451+0000 7f89dbe00640 1 -- 172.21.15.87:0/2185509837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc1e5ba0 con 0x7f89dc0b79f0 2024-04-25T11:09:58.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.451+0000 7f89d3400640 1 --2- 172.21.15.87:0/2185509837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc0b9920 0x7f89dc0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:09:58.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.451+0000 7f89dbe00640 1 --2- 172.21.15.87:0/2185509837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc0b79f0 0x7f89dc1f6ed0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009e80 tx=0x7f89cc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:58.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.451+0000 7f89d8c00640 1 -- 172.21.15.87:0/2185509837 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc03f070 con 0x7f89dc0b79f0 2024-04-25T11:09:58.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.451+0000 7f89d8c00640 1 -- 172.21.15.87:0/2185509837 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89cc031c40 con 0x7f89dc0b79f0 2024-04-25T11:09:58.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.451+0000 7f89e2c00640 1 -- 172.21.15.87:0/2185509837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89dc1e5e80 con 0x7f89dc0b79f0 2024-04-25T11:09:58.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.451+0000 7f89d8c00640 1 -- 172.21.15.87:0/2185509837 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89cc044460 con 0x7f89dc0b79f0 2024-04-25T11:09:58.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.452+0000 7f89e2c00640 1 -- 172.21.15.87:0/2185509837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89dc1e6440 con 0x7f89dc0b79f0 2024-04-25T11:09:58.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.452+0000 7f89d8c00640 1 -- 172.21.15.87:0/2185509837 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f89cc044600 con 0x7f89dc0b79f0 2024-04-25T11:09:58.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.452+0000 7f89d8c00640 1 -- 172.21.15.87:0/2185509837 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f89cc044d10 con 0x7f89dc0b79f0 2024-04-25T11:09:58.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.453+0000 7f89e2c00640 1 -- 172.21.15.87:0/2185509837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89dc050740 con 0x7f89dc0b79f0 2024-04-25T11:09:58.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.455+0000 7f89d8c00640 1 -- 172.21.15.87:0/2185509837 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f89cc0448f0 con 0x7f89dc0b79f0 2024-04-25T11:09:58.475 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:58.474+0000 7f5f4ba0f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:09:58.527 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:58.527+0000 7f5f4ba0f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:09:58.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.532+0000 7f89e2c00640 1 -- 172.21.15.87:0/2185509837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1d3b267f-385c-4080-a237-0caf41de8a05", "id": 2} v 0) -- 0x7f89dc15f100 con 0x7f89dc0b79f0 2024-04-25T11:09:58.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.540+0000 7f89d8c00640 1 -- 172.21.15.87:0/2185509837 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1d3b267f-385c-4080-a237-0caf41de8a05", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f89cc03b5a0 con 0x7f89dc0b79f0 2024-04-25T11:09:58.541 INFO:teuthology.orchestra.run.smithi087.stdout:2 2024-04-25T11:09:58.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.541+0000 7f89e2c00640 1 -- 172.21.15.87:0/2185509837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc0b79f0 msgr2=0x7f89dc1f6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.541+0000 7f89e2c00640 1 --2- 172.21.15.87:0/2185509837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc0b79f0 0x7f89dc1f6ed0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009e80 tx=0x7f89cc004a40 comp rx=0 tx=0).stop 2024-04-25T11:09:58.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.542+0000 7f89e2c00640 1 -- 172.21.15.87:0/2185509837 shutdown_connections 2024-04-25T11:09:58.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.542+0000 7f89e2c00640 1 --2- 172.21.15.87:0/2185509837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89dc160310 0x7f89dc0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.542+0000 7f89e2c00640 1 --2- 172.21.15.87:0/2185509837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc0b9920 0x7f89dc0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.542+0000 7f89e2c00640 1 --2- 172.21.15.87:0/2185509837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc0b79f0 0x7f89dc1f6ed0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.542+0000 7f89e2c00640 1 -- 172.21.15.87:0/2185509837 >> 172.21.15.87:0/2185509837 conn(0x7f89dc0d2a50 msgr2=0x7f89dc15f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:58.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.542+0000 7f89e2c00640 1 -- 172.21.15.87:0/2185509837 shutdown_connections 2024-04-25T11:09:58.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.542+0000 7f89e2c00640 1 -- 172.21.15.87:0/2185509837 wait complete. 2024-04-25T11:09:58.566 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new fd328260-3895-4610-9bfb-171d15b180fd 3 2024-04-25T11:09:58.604 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:58.604+0000 7f7071be1040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T11:09:58.715 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:58.715+0000 7f7071be1040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T11:09:58.727 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:58.727+0000 7f5f4ba0f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:09:58.769 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:58.769+0000 7f7071be1040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T11:09:58.785 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:58.785+0000 7f5f4ba0f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:09:58.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.803+0000 7f4456200640 1 Processor -- start 2024-04-25T11:09:58.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.803+0000 7f4456200640 1 -- start start 2024-04-25T11:09:58.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.805+0000 7f4456200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f44500b7af0 0x7f445015fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.805+0000 7f4456200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f44500b8e50 0x7f44500d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.805+0000 7f4456200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44500d13c0 con 0x7f44500d0fb0 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.805+0000 7f4456200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4450163f10 con 0x7f44500b7af0 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.805+0000 7f4456200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f44501641f0 con 0x7f44500b8e50 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.805+0000 7f4454e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f44500d0fb0 0x7f44500b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56844/0 (socket says 172.21.15.87:56844) 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.805+0000 7f4454e00640 1 -- 172.21.15.87:0/2773734184 learned_addr learned my addr 172.21.15.87:0/2773734184 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.805+0000 7f4455800640 1 --2- 172.21.15.87:0/2773734184 >> v2:172.21.15.87:3300/0 conn(0x7f44500b8e50 0x7f44500d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.806+0000 7f444fe00640 1 --2- 172.21.15.87:0/2773734184 >> v2:172.21.15.7:3300/0 conn(0x7f44500b7af0 0x7f445015fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.806+0000 7f444f400640 1 -- 172.21.15.87:0/2773734184 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4208107339 0 0) 0x7f44500d13c0 con 0x7f44500d0fb0 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.806+0000 7f444f400640 1 -- 172.21.15.87:0/2773734184 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4438003520 con 0x7f44500d0fb0 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.806+0000 7f4455800640 1 -- 172.21.15.87:0/2773734184 >> v2:172.21.15.7:3300/0 conn(0x7f44500b7af0 msgr2=0x7f445015fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:09:58.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.806+0000 7f4455800640 1 --2- 172.21.15.87:0/2773734184 >> v2:172.21.15.7:3300/0 conn(0x7f44500b7af0 0x7f445015fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.806+0000 7f4455800640 1 -- 172.21.15.87:0/2773734184 >> v1:172.21.15.87:6789/0 conn(0x7f44500d0fb0 legacy=0x7f44500b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.806+0000 7f4455800640 1 -- 172.21.15.87:0/2773734184 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44501644d0 con 0x7f44500b8e50 2024-04-25T11:09:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.807+0000 7f4455800640 1 --2- 172.21.15.87:0/2773734184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44500b8e50 0x7f44500d45e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4450164a90 tx=0x7f444802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39344f4ecdb517d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.807+0000 7f444f400640 1 -- 172.21.15.87:0/2773734184 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448007740 con 0x7f44500b8e50 2024-04-25T11:09:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.807+0000 7f444f400640 1 -- 172.21.15.87:0/2773734184 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44480078e0 con 0x7f44500b8e50 2024-04-25T11:09:58.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.807+0000 7f444f400640 1 -- 172.21.15.87:0/2773734184 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448007cb0 con 0x7f44500b8e50 2024-04-25T11:09:58.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.809+0000 7f4456200640 1 -- 172.21.15.87:0/2773734184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44500b8e50 msgr2=0x7f44500d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.809+0000 7f4456200640 1 --2- 172.21.15.87:0/2773734184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44500b8e50 0x7f44500d45e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4450164a90 tx=0x7f444802a5f0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.809+0000 7f4456200640 1 -- 172.21.15.87:0/2773734184 shutdown_connections 2024-04-25T11:09:58.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.809+0000 7f4456200640 1 --2- 172.21.15.87:0/2773734184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44500b8e50 0x7f44500d45e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.809+0000 7f4456200640 1 --2- 172.21.15.87:0/2773734184 >> v2:172.21.15.7:3300/0 conn(0x7f44500b7af0 0x7f445015fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.809+0000 7f4456200640 1 -- 172.21.15.87:0/2773734184 >> 172.21.15.87:0/2773734184 conn(0x7f44500d23e0 msgr2=0x7f44500d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:58.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.809+0000 7f4456200640 1 -- 172.21.15.87:0/2773734184 shutdown_connections 2024-04-25T11:09:58.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.809+0000 7f4456200640 1 -- 172.21.15.87:0/2773734184 wait complete. 2024-04-25T11:09:58.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.811+0000 7f4456200640 1 Processor -- start 2024-04-25T11:09:58.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.811+0000 7f4456200640 1 -- start start 2024-04-25T11:09:58.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.813+0000 7f4456200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44500b8e50 0x7f44501565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.813+0000 7f4456200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44500d0fb0 0x7f445015a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.813+0000 7f4456200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445015bf30 0x7f445015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:58.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.813+0000 7f4456200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44500b9260 con 0x7f445015bf30 2024-04-25T11:09:58.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.813+0000 7f4456200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f44500b4bc0 con 0x7f44500b8e50 2024-04-25T11:09:58.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.813+0000 7f4456200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4450163a80 con 0x7f44500d0fb0 2024-04-25T11:09:58.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.813+0000 7f4454e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44500b8e50 0x7f44501565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.813+0000 7f4454e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44500b8e50 0x7f44501565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52126/0 (socket says 172.21.15.87:52126) 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.813+0000 7f4454e00640 1 -- 172.21.15.87:0/2686643019 learned_addr learned my addr 172.21.15.87:0/2686643019 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.813+0000 7f444fe00640 1 --2- 172.21.15.87:0/2686643019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44500d0fb0 0x7f445015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.813+0000 7f4455800640 1 --2- 172.21.15.87:0/2686643019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445015bf30 0x7f445015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f4454e00640 1 -- 172.21.15.87:0/2686643019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44500d0fb0 msgr2=0x7f445015a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f4454e00640 1 --2- 172.21.15.87:0/2686643019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44500d0fb0 0x7f445015a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f4454e00640 1 -- 172.21.15.87:0/2686643019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445015bf30 msgr2=0x7f445015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f4454e00640 1 --2- 172.21.15.87:0/2686643019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445015bf30 0x7f445015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f4454e00640 1 -- 172.21.15.87:0/2686643019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445015c880 con 0x7f44500b8e50 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f4455800640 1 --2- 172.21.15.87:0/2686643019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445015bf30 0x7f445015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f4454e00640 1 --2- 172.21.15.87:0/2686643019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44500b8e50 0x7f44501565e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f444400c390 tx=0x7f444400c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f444cc00640 1 -- 172.21.15.87:0/2686643019 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444017070 con 0x7f44500b8e50 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f444cc00640 1 -- 172.21.15.87:0/2686643019 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4444005ce0 con 0x7f44500b8e50 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f444cc00640 1 -- 172.21.15.87:0/2686643019 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44440072f0 con 0x7f44500b8e50 2024-04-25T11:09:58.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f4456200640 1 -- 172.21.15.87:0/2686643019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445015ff40 con 0x7f44500b8e50 2024-04-25T11:09:58.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f4456200640 1 -- 172.21.15.87:0/2686643019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44501604c0 con 0x7f44500b8e50 2024-04-25T11:09:58.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.814+0000 7f444cc00640 1 -- 172.21.15.87:0/2686643019 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f444401a070 con 0x7f44500b8e50 2024-04-25T11:09:58.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.815+0000 7f444cc00640 1 -- 172.21.15.87:0/2686643019 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f444401c8d0 con 0x7f44500b8e50 2024-04-25T11:09:58.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.815+0000 7f4456200640 1 -- 172.21.15.87:0/2686643019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4450050740 con 0x7f44500b8e50 2024-04-25T11:09:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.819+0000 7f444cc00640 1 -- 172.21.15.87:0/2686643019 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4444021070 con 0x7f44500b8e50 2024-04-25T11:09:58.849 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:58.849+0000 7f5f4ba0f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:09:58.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.898+0000 7f4456200640 1 -- 172.21.15.87:0/2686643019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fd328260-3895-4610-9bfb-171d15b180fd", "id": 3} v 0) -- 0x7f44500d3d40 con 0x7f44500b8e50 2024-04-25T11:09:58.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.905+0000 7f444cc00640 1 -- 172.21.15.87:0/2686643019 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fd328260-3895-4610-9bfb-171d15b180fd", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f445015c880 con 0x7f44500b8e50 2024-04-25T11:09:58.906 INFO:teuthology.orchestra.run.smithi087.stdout:3 2024-04-25T11:09:58.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.907+0000 7f4456200640 1 -- 172.21.15.87:0/2686643019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44500b8e50 msgr2=0x7f44501565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:58.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.907+0000 7f4456200640 1 --2- 172.21.15.87:0/2686643019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44500b8e50 0x7f44501565e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f444400c390 tx=0x7f444400c850 comp rx=0 tx=0).stop 2024-04-25T11:09:58.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.907+0000 7f4456200640 1 -- 172.21.15.87:0/2686643019 shutdown_connections 2024-04-25T11:09:58.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.907+0000 7f4456200640 1 --2- 172.21.15.87:0/2686643019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445015bf30 0x7f445015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.907+0000 7f4456200640 1 --2- 172.21.15.87:0/2686643019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44500d0fb0 0x7f445015a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.907+0000 7f4456200640 1 --2- 172.21.15.87:0/2686643019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44500b8e50 0x7f44501565e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:58.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.907+0000 7f4456200640 1 -- 172.21.15.87:0/2686643019 >> 172.21.15.87:0/2686643019 conn(0x7f44500d23e0 msgr2=0x7f44500d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:58.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.907+0000 7f4456200640 1 -- 172.21.15.87:0/2686643019 shutdown_connections 2024-04-25T11:09:58.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:58.907+0000 7f4456200640 1 -- 172.21.15.87:0/2686643019 wait complete. 2024-04-25T11:09:58.931 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 99bfd73d-2071-467e-b193-41909aa6b352 4 2024-04-25T11:09:58.976 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:58.973+0000 7f7071be1040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T11:09:58.983 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:58.983+0000 7f5f4ba0f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:09:59.046 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:59.046+0000 7f7071be1040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T11:09:59.111 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:59.110+0000 7f7071be1040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T11:09:59.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.231+0000 7f9de1a00640 1 Processor -- start 2024-04-25T11:09:59.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.232+0000 7f9de1a00640 1 -- start start 2024-04-25T11:09:59.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.234+0000 7f9de1a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ddc09b540 0x7f9ddc0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.234+0000 7f9de1a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9ddc161200 0x7f9ddc163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.234+0000 7f9de1a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ddc0b1e90 con 0x7f9ddc160df0 2024-04-25T11:09:59.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.234+0000 7f9de1a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9ddc0b2160 con 0x7f9ddc09b540 2024-04-25T11:09:59.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.234+0000 7f9de1a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9ddc0b2440 con 0x7f9ddc161200 2024-04-25T11:09:59.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.234+0000 7f9dd3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9ddc160df0 0x7f9ddc06d790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56852/0 (socket says 172.21.15.87:56852) 2024-04-25T11:09:59.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.234+0000 7f9dd3400640 1 -- 172.21.15.87:0/3289034756 learned_addr learned my addr 172.21.15.87:0/3289034756 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T11:09:59.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.234+0000 7f9de1000640 1 --2- 172.21.15.87:0/3289034756 >> v2:172.21.15.87:3300/0 conn(0x7f9ddc161200 0x7f9ddc163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.234+0000 7f9ddbe00640 1 --2- 172.21.15.87:0/3289034756 >> v2:172.21.15.7:3300/0 conn(0x7f9ddc09b540 0x7f9ddc0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.235+0000 7f9ddb400640 1 -- 172.21.15.87:0/3289034756 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887492701 0 0) 0x7f9ddc0b1e90 con 0x7f9ddc160df0 2024-04-25T11:09:59.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.235+0000 7f9ddb400640 1 -- 172.21.15.87:0/3289034756 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9dc0003520 con 0x7f9ddc160df0 2024-04-25T11:09:59.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.235+0000 7f9de1000640 1 -- 172.21.15.87:0/3289034756 >> v2:172.21.15.7:3300/0 conn(0x7f9ddc09b540 msgr2=0x7f9ddc0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:09:59.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.235+0000 7f9de1000640 1 --2- 172.21.15.87:0/3289034756 >> v2:172.21.15.7:3300/0 conn(0x7f9ddc09b540 0x7f9ddc0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.235+0000 7f9de1000640 1 -- 172.21.15.87:0/3289034756 >> v1:172.21.15.87:6789/0 conn(0x7f9ddc160df0 legacy=0x7f9ddc06d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.235+0000 7f9de1000640 1 -- 172.21.15.87:0/3289034756 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc1605f0 con 0x7f9ddc161200 2024-04-25T11:09:59.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.235+0000 7f9ddbe00640 1 --2- 172.21.15.87:0/3289034756 >> v2:172.21.15.7:3300/0 conn(0x7f9ddc09b540 0x7f9ddc0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:09:59.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.236+0000 7f9de1000640 1 --2- 172.21.15.87:0/3289034756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ddc161200 0x7f9ddc163ad0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0b39c0 tx=0x7f9dd402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2963bdc580f19958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:59.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.236+0000 7f9ddb400640 1 -- 172.21.15.87:0/3289034756 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4037070 con 0x7f9ddc161200 2024-04-25T11:09:59.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.236+0000 7f9ddb400640 1 -- 172.21.15.87:0/3289034756 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd4032440 con 0x7f9ddc161200 2024-04-25T11:09:59.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.236+0000 7f9ddb400640 1 -- 172.21.15.87:0/3289034756 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd402ae60 con 0x7f9ddc161200 2024-04-25T11:09:59.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.237+0000 7f9de1a00640 1 -- 172.21.15.87:0/3289034756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ddc161200 msgr2=0x7f9ddc163ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.237+0000 7f9de1a00640 1 --2- 172.21.15.87:0/3289034756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ddc161200 0x7f9ddc163ad0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc0b39c0 tx=0x7f9dd402a110 comp rx=0 tx=0).stop 2024-04-25T11:09:59.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.237+0000 7f9de1a00640 1 -- 172.21.15.87:0/3289034756 shutdown_connections 2024-04-25T11:09:59.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.237+0000 7f9de1a00640 1 --2- 172.21.15.87:0/3289034756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ddc161200 0x7f9ddc163ad0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.237+0000 7f9de1a00640 1 --2- 172.21.15.87:0/3289034756 >> v2:172.21.15.7:3300/0 conn(0x7f9ddc09b540 0x7f9ddc0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.237+0000 7f9de1a00640 1 -- 172.21.15.87:0/3289034756 >> 172.21.15.87:0/3289034756 conn(0x7f9ddc073b80 msgr2=0x7f9ddc0b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:59.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.237+0000 7f9de1a00640 1 -- 172.21.15.87:0/3289034756 shutdown_connections 2024-04-25T11:09:59.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.237+0000 7f9de1a00640 1 -- 172.21.15.87:0/3289034756 wait complete. 2024-04-25T11:09:59.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.239+0000 7f9de1a00640 1 Processor -- start 2024-04-25T11:09:59.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.239+0000 7f9de1a00640 1 -- start start 2024-04-25T11:09:59.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.241+0000 7f9de1a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ddc09b540 0x7f9ddc152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.241+0000 7f9de1a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ddc160df0 0x7f9ddc152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.241+0000 7f9de1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc161200 0x7f9ddc140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.241+0000 7f9de1a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc0b28a0 con 0x7f9ddc161200 2024-04-25T11:09:59.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.241+0000 7f9de1a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc0b2720 con 0x7f9ddc160df0 2024-04-25T11:09:59.243 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.241+0000 7f9de1a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9ddc15a7a0 con 0x7f9ddc09b540 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.241+0000 7f9ddbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ddc09b540 0x7f9ddc152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.241+0000 7f9dd3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ddc160df0 0x7f9ddc152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.241+0000 7f9ddbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ddc09b540 0x7f9ddc152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.87:41042/0 (socket says 172.21.15.87:41042) 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.241+0000 7f9ddbe00640 1 -- 172.21.15.87:0/2686988514 learned_addr learned my addr 172.21.15.87:0/2686988514 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.241+0000 7f9de1000640 1 --2- 172.21.15.87:0/2686988514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc161200 0x7f9ddc140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9dd3400640 1 -- 172.21.15.87:0/2686988514 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ddc09b540 msgr2=0x7f9ddc152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9dd3400640 1 --2- 172.21.15.87:0/2686988514 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ddc09b540 0x7f9ddc152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9dd3400640 1 -- 172.21.15.87:0/2686988514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc161200 msgr2=0x7f9ddc140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9dd3400640 1 --2- 172.21.15.87:0/2686988514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc161200 0x7f9ddc140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9dd3400640 1 -- 172.21.15.87:0/2686988514 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc152880 con 0x7f9ddc160df0 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9de1000640 1 --2- 172.21.15.87:0/2686988514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc161200 0x7f9ddc140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9dd3400640 1 --2- 172.21.15.87:0/2686988514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ddc160df0 0x7f9ddc152340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc073900 tx=0x7f9dc4001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9dd8c00640 1 -- 172.21.15.87:0/2686988514 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4017070 con 0x7f9ddc160df0 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9dd8c00640 1 -- 172.21.15.87:0/2686988514 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc4005ce0 con 0x7f9ddc160df0 2024-04-25T11:09:59.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9dd8c00640 1 -- 172.21.15.87:0/2686988514 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40072f0 con 0x7f9ddc160df0 2024-04-25T11:09:59.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9de1a00640 1 -- 172.21.15.87:0/2686988514 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc1556a0 con 0x7f9ddc160df0 2024-04-25T11:09:59.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9de1a00640 1 -- 172.21.15.87:0/2686988514 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc155c60 con 0x7f9ddc160df0 2024-04-25T11:09:59.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.242+0000 7f9dd8c00640 1 -- 172.21.15.87:0/2686988514 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9dc401a070 con 0x7f9ddc160df0 2024-04-25T11:09:59.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.243+0000 7f9dd8c00640 1 -- 172.21.15.87:0/2686988514 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f9dc4007490 con 0x7f9ddc160df0 2024-04-25T11:09:59.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.244+0000 7f9de1a00640 1 -- 172.21.15.87:0/2686988514 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ddc050740 con 0x7f9ddc160df0 2024-04-25T11:09:59.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.246+0000 7f9dd8c00640 1 -- 172.21.15.87:0/2686988514 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9dc4021070 con 0x7f9ddc160df0 2024-04-25T11:09:59.248 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:59.247+0000 7f7071be1040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T11:09:59.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.324+0000 7f9de1a00640 1 -- 172.21.15.87:0/2686988514 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "99bfd73d-2071-467e-b193-41909aa6b352", "id": 4} v 0) -- 0x7f9ddc0b0eb0 con 0x7f9ddc160df0 2024-04-25T11:09:59.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.334+0000 7f9dd8c00640 1 -- 172.21.15.87:0/2686988514 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "99bfd73d-2071-467e-b193-41909aa6b352", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f9dc401c440 con 0x7f9ddc160df0 2024-04-25T11:09:59.334 INFO:teuthology.orchestra.run.smithi087.stdout:4 2024-04-25T11:09:59.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.335+0000 7f9de1a00640 1 -- 172.21.15.87:0/2686988514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ddc160df0 msgr2=0x7f9ddc152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.335+0000 7f9de1a00640 1 --2- 172.21.15.87:0/2686988514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ddc160df0 0x7f9ddc152340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc073900 tx=0x7f9dc4001e70 comp rx=0 tx=0).stop 2024-04-25T11:09:59.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.335+0000 7f9de1a00640 1 -- 172.21.15.87:0/2686988514 shutdown_connections 2024-04-25T11:09:59.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.335+0000 7f9de1a00640 1 --2- 172.21.15.87:0/2686988514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ddc161200 0x7f9ddc140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.335+0000 7f9de1a00640 1 --2- 172.21.15.87:0/2686988514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ddc160df0 0x7f9ddc152340 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc073900 tx=0x7f9dc4001e70 comp rx=0 tx=0).stop 2024-04-25T11:09:59.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.335+0000 7f9de1a00640 1 --2- 172.21.15.87:0/2686988514 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ddc09b540 0x7f9ddc152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.335+0000 7f9de1a00640 1 -- 172.21.15.87:0/2686988514 >> 172.21.15.87:0/2686988514 conn(0x7f9ddc073b80 msgr2=0x7f9ddc0b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:59.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.335+0000 7f9de1a00640 1 -- 172.21.15.87:0/2686988514 shutdown_connections 2024-04-25T11:09:59.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.335+0000 7f9de1a00640 1 -- 172.21.15.87:0/2686988514 wait complete. 2024-04-25T11:09:59.359 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 9e92fd38-1801-468a-9bd6-e0bb0e885c84 5 2024-04-25T11:09:59.459 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:59.458+0000 7f5f4ba0f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:09:59.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.612+0000 7f188a800640 1 Processor -- start 2024-04-25T11:09:59.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.613+0000 7f188a800640 1 -- start start 2024-04-25T11:09:59.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.615+0000 7f188a800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18840ce8d0 0x7f18840cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f188a800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188415c2e0 0x7f188415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f188a800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18840b4830 con 0x7f188406f730 2024-04-25T11:09:59.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f188a800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f18840b6600 con 0x7f18840ce8d0 2024-04-25T11:09:59.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f188a800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18840b4b10 con 0x7f188415c2e0 2024-04-25T11:09:59.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f1888a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188415c2e0 0x7f188415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f1889e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f188406f730 0x7f188406fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56854/0 (socket says 172.21.15.87:56854) 2024-04-25T11:09:59.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f1889e00640 1 -- 172.21.15.87:0/223833466 learned_addr learned my addr 172.21.15.87:0/223833466 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T11:09:59.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f1889400640 1 --2- 172.21.15.87:0/223833466 >> v2:172.21.15.7:3300/0 conn(0x7f18840ce8d0 0x7f18840cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f1883e00640 1 -- 172.21.15.87:0/223833466 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 828094674 0 0) 0x7f18840b4830 con 0x7f188406f730 2024-04-25T11:09:59.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f1883e00640 1 -- 172.21.15.87:0/223833466 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f186c003520 con 0x7f188406f730 2024-04-25T11:09:59.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f1888a00640 1 -- 172.21.15.87:0/223833466 >> v2:172.21.15.7:3300/0 conn(0x7f18840ce8d0 msgr2=0x7f18840cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:09:59.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f1888a00640 1 --2- 172.21.15.87:0/223833466 >> v2:172.21.15.7:3300/0 conn(0x7f18840ce8d0 0x7f18840cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.616+0000 7f1888a00640 1 -- 172.21.15.87:0/223833466 >> v1:172.21.15.87:6789/0 conn(0x7f188406f730 legacy=0x7f188406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.617+0000 7f1888a00640 1 -- 172.21.15.87:0/223833466 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18840b4df0 con 0x7f188415c2e0 2024-04-25T11:09:59.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.617+0000 7f1889400640 1 --2- 172.21.15.87:0/223833466 >> v2:172.21.15.7:3300/0 conn(0x7f18840ce8d0 0x7f18840cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:09:59.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.618+0000 7f1888a00640 1 --2- 172.21.15.87:0/223833466 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188415c2e0 0x7f188415c6f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f187000a940 tx=0x7f187002b550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d3ab060db39f4db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:59.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.618+0000 7f1883e00640 1 -- 172.21.15.87:0/223833466 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187000b040 con 0x7f188415c2e0 2024-04-25T11:09:59.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.618+0000 7f1883e00640 1 -- 172.21.15.87:0/223833466 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1870030710 con 0x7f188415c2e0 2024-04-25T11:09:59.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.618+0000 7f1883e00640 1 -- 172.21.15.87:0/223833466 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870030ae0 con 0x7f188415c2e0 2024-04-25T11:09:59.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.619+0000 7f188a800640 1 -- 172.21.15.87:0/223833466 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188415c2e0 msgr2=0x7f188415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.619+0000 7f188a800640 1 --2- 172.21.15.87:0/223833466 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188415c2e0 0x7f188415c6f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f187000a940 tx=0x7f187002b550 comp rx=0 tx=0).stop 2024-04-25T11:09:59.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.620+0000 7f188a800640 1 -- 172.21.15.87:0/223833466 shutdown_connections 2024-04-25T11:09:59.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.620+0000 7f188a800640 1 --2- 172.21.15.87:0/223833466 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188415c2e0 0x7f188415c6f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.620+0000 7f188a800640 1 --2- 172.21.15.87:0/223833466 >> v2:172.21.15.7:3300/0 conn(0x7f18840ce8d0 0x7f18840cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.620+0000 7f188a800640 1 -- 172.21.15.87:0/223833466 >> 172.21.15.87:0/223833466 conn(0x7f18840740f0 msgr2=0x7f188406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:59.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.620+0000 7f188a800640 1 -- 172.21.15.87:0/223833466 shutdown_connections 2024-04-25T11:09:59.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.620+0000 7f188a800640 1 -- 172.21.15.87:0/223833466 wait complete. 2024-04-25T11:09:59.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.623+0000 7f188a800640 1 Processor -- start 2024-04-25T11:09:59.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.624+0000 7f188a800640 1 -- start start 2024-04-25T11:09:59.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.626+0000 7f188a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188406f730 0x7f1884145550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.626+0000 7f188a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18840ce8d0 0x7f18841f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.626+0000 7f188a800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188415c2e0 0x7f1884150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.626+0000 7f188a800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1884070ff0 con 0x7f18840ce8d0 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.626+0000 7f188a800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1884070e70 con 0x7f188406f730 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.626+0000 7f188a800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f188406eea0 con 0x7f188415c2e0 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.626+0000 7f1889400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188406f730 0x7f1884145550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.626+0000 7f1889400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188406f730 0x7f1884145550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52160/0 (socket says 172.21.15.87:52160) 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.626+0000 7f1889400640 1 -- 172.21.15.87:0/1420193391 learned_addr learned my addr 172.21.15.87:0/1420193391 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.626+0000 7f1889e00640 1 --2- 172.21.15.87:0/1420193391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188415c2e0 0x7f1884150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.627+0000 7f1888a00640 1 --2- 172.21.15.87:0/1420193391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18840ce8d0 0x7f18841f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.627+0000 7f1889400640 1 -- 172.21.15.87:0/1420193391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188415c2e0 msgr2=0x7f1884150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.627+0000 7f1889400640 1 --2- 172.21.15.87:0/1420193391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188415c2e0 0x7f1884150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.627+0000 7f1889400640 1 -- 172.21.15.87:0/1420193391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18840ce8d0 msgr2=0x7f18841f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.627+0000 7f1889400640 1 --2- 172.21.15.87:0/1420193391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18840ce8d0 0x7f18841f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.627+0000 7f1889400640 1 -- 172.21.15.87:0/1420193391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1884152fc0 con 0x7f188406f730 2024-04-25T11:09:59.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.627+0000 7f1889400640 1 --2- 172.21.15.87:0/1420193391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188406f730 0x7f1884145550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f187800d7d0 tx=0x7f187800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:59.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.627+0000 7f1881600640 1 -- 172.21.15.87:0/1420193391 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187800b660 con 0x7f188406f730 2024-04-25T11:09:59.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.628+0000 7f1881600640 1 -- 172.21.15.87:0/1420193391 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1878019070 con 0x7f188406f730 2024-04-25T11:09:59.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.628+0000 7f1881600640 1 -- 172.21.15.87:0/1420193391 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187800ba40 con 0x7f188406f730 2024-04-25T11:09:59.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.628+0000 7f188a800640 1 -- 172.21.15.87:0/1420193391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18841534c0 con 0x7f188406f730 2024-04-25T11:09:59.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.628+0000 7f188a800640 1 -- 172.21.15.87:0/1420193391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18841f75a0 con 0x7f188406f730 2024-04-25T11:09:59.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.628+0000 7f1881600640 1 -- 172.21.15.87:0/1420193391 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f187800bbe0 con 0x7f188406f730 2024-04-25T11:09:59.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.628+0000 7f1881600640 1 -- 172.21.15.87:0/1420193391 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f187801e9e0 con 0x7f188406f730 2024-04-25T11:09:59.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.629+0000 7f188a800640 1 -- 172.21.15.87:0/1420193391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1884050740 con 0x7f188406f730 2024-04-25T11:09:59.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.631+0000 7f1881600640 1 -- 172.21.15.87:0/1420193391 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f187801e460 con 0x7f188406f730 2024-04-25T11:09:59.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.710+0000 7f188a800640 1 -- 172.21.15.87:0/1420193391 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9e92fd38-1801-468a-9bd6-e0bb0e885c84", "id": 5} v 0) -- 0x7f18840709b0 con 0x7f188406f730 2024-04-25T11:09:59.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.717+0000 7f1881600640 1 -- 172.21.15.87:0/1420193391 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9e92fd38-1801-468a-9bd6-e0bb0e885c84", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f187801e640 con 0x7f188406f730 2024-04-25T11:09:59.717 INFO:teuthology.orchestra.run.smithi087.stdout:5 2024-04-25T11:09:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.718+0000 7f188a800640 1 -- 172.21.15.87:0/1420193391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188406f730 msgr2=0x7f1884145550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.718+0000 7f188a800640 1 --2- 172.21.15.87:0/1420193391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188406f730 0x7f1884145550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f187800d7d0 tx=0x7f187800dc90 comp rx=0 tx=0).stop 2024-04-25T11:09:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.718+0000 7f188a800640 1 -- 172.21.15.87:0/1420193391 shutdown_connections 2024-04-25T11:09:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.718+0000 7f188a800640 1 --2- 172.21.15.87:0/1420193391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188415c2e0 0x7f1884150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.718+0000 7f188a800640 1 --2- 172.21.15.87:0/1420193391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18840ce8d0 0x7f18841f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.718+0000 7f188a800640 1 --2- 172.21.15.87:0/1420193391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188406f730 0x7f1884145550 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.718+0000 7f188a800640 1 -- 172.21.15.87:0/1420193391 >> 172.21.15.87:0/1420193391 conn(0x7f18840740f0 msgr2=0x7f18840b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.718+0000 7f188a800640 1 -- 172.21.15.87:0/1420193391 shutdown_connections 2024-04-25T11:09:59.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.718+0000 7f188a800640 1 -- 172.21.15.87:0/1420193391 wait complete. 2024-04-25T11:09:59.737 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:09:59.736+0000 7f7071be1040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T11:09:59.743 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 9b0485e2-f51d-462b-81ad-e00f2549b9df 6 2024-04-25T11:09:59.768 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:59.768+0000 7f5f4ba0f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:09:59.831 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:59.831+0000 7f5f4ba0f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:09:59.969 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:09:59.969+0000 7f5f4ba0f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:09:59.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.991+0000 7fae67200640 1 Processor -- start 2024-04-25T11:09:59.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.991+0000 7fae67200640 1 -- start start 2024-04-25T11:09:59.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae67200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae600b7ac0 0x7fae600af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae67200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fae600b29e0 0x7fae600cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:09:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae67200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae600d1740 con 0x7fae601604c0 2024-04-25T11:09:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae67200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fae600cffe0 con 0x7fae600b29e0 2024-04-25T11:09:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae67200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fae600d2310 con 0x7fae600b7ac0 2024-04-25T11:09:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae65400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae600b7ac0 0x7fae600af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae66800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fae600b29e0 0x7fae600cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:09:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae65400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae600b7ac0 0x7fae600af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.87:41050/0 (socket says 172.21.15.87:41050) 2024-04-25T11:09:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae66800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fae600b29e0 0x7fae600cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52176/0 (socket says 172.21.15.87:52176) 2024-04-25T11:09:59.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae65400640 1 -- 172.21.15.87:0/540675588 learned_addr learned my addr 172.21.15.87:0/540675588 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T11:09:59.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae66800640 1 -- 172.21.15.87:0/540675588 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae600b7ac0 msgr2=0x7fae600af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:09:59.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae66800640 1 --2- 172.21.15.87:0/540675588 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae600b7ac0 0x7fae600af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.993+0000 7fae66800640 1 -- 172.21.15.87:0/540675588 >> v1:172.21.15.7:6789/0 conn(0x7fae601604c0 legacy=0x7fae600b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.994+0000 7fae66800640 1 -- 172.21.15.87:0/540675588 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae600d19e0 con 0x7fae600b29e0 2024-04-25T11:09:59.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.994+0000 7fae65400640 1 --2- 172.21.15.87:0/540675588 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae600b7ac0 0x7fae600af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:09:59.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.994+0000 7fae66800640 1 --2- 172.21.15.87:0/540675588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae600b29e0 0x7fae600cf840 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0097a0 tx=0x7fae5c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3502d8e3c809059 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:09:59.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.995+0000 7fae64a00640 1 -- 172.21.15.87:0/540675588 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c037070 con 0x7fae600b29e0 2024-04-25T11:09:59.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.995+0000 7fae64a00640 1 -- 172.21.15.87:0/540675588 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5c032440 con 0x7fae600b29e0 2024-04-25T11:09:59.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.995+0000 7fae64a00640 1 -- 172.21.15.87:0/540675588 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c032810 con 0x7fae600b29e0 2024-04-25T11:09:59.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.995+0000 7fae67200640 1 -- 172.21.15.87:0/540675588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae600b29e0 msgr2=0x7fae600cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:09:59.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.995+0000 7fae67200640 1 --2- 172.21.15.87:0/540675588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae600b29e0 0x7fae600cf840 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0097a0 tx=0x7fae5c02a180 comp rx=0 tx=0).stop 2024-04-25T11:09:59.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.996+0000 7fae67200640 1 -- 172.21.15.87:0/540675588 shutdown_connections 2024-04-25T11:09:59.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.996+0000 7fae67200640 1 --2- 172.21.15.87:0/540675588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae600b29e0 0x7fae600cf840 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.996+0000 7fae67200640 1 --2- 172.21.15.87:0/540675588 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae600b7ac0 0x7fae600af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:09:59.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.996+0000 7fae67200640 1 -- 172.21.15.87:0/540675588 >> 172.21.15.87:0/540675588 conn(0x7fae6009b540 msgr2=0x7fae6009b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:09:59.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.996+0000 7fae67200640 1 -- 172.21.15.87:0/540675588 shutdown_connections 2024-04-25T11:09:59.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.996+0000 7fae67200640 1 -- 172.21.15.87:0/540675588 wait complete. 2024-04-25T11:09:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.998+0000 7fae67200640 1 Processor -- start 2024-04-25T11:09:59.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:09:59.998+0000 7fae67200640 1 -- start start 2024-04-25T11:10:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae67200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae600b29e0 0x7fae6009d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae67200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae601604c0 0x7fae601f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae67200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae601f9d90 0x7fae601fa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae67200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae600d1fa0 con 0x7fae601f9d90 2024-04-25T11:10:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae67200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fae600cfd80 con 0x7fae600b29e0 2024-04-25T11:10:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae67200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fae600d27a0 con 0x7fae601604c0 2024-04-25T11:10:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae65e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae600b29e0 0x7fae6009d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae65400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae601604c0 0x7fae601f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae65e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae600b29e0 0x7fae6009d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52188/0 (socket says 172.21.15.87:52188) 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae65e00640 1 -- 172.21.15.87:0/253668760 learned_addr learned my addr 172.21.15.87:0/253668760 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae66800640 1 --2- 172.21.15.87:0/253668760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae601f9d90 0x7fae601fa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae65e00640 1 -- 172.21.15.87:0/253668760 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae601604c0 msgr2=0x7fae601f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae65e00640 1 --2- 172.21.15.87:0/253668760 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae601604c0 0x7fae601f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae65e00640 1 -- 172.21.15.87:0/253668760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae601f9d90 msgr2=0x7fae601fa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae65e00640 1 --2- 172.21.15.87:0/253668760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae601f9d90 0x7fae601fa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae65e00640 1 -- 172.21.15.87:0/253668760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae601fa6e0 con 0x7fae600b29e0 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.000+0000 7fae66800640 1 --2- 172.21.15.87:0/253668760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae601f9d90 0x7fae601fa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.001+0000 7fae65e00640 1 --2- 172.21.15.87:0/253668760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae600b29e0 0x7fae6009d460 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fae5800c730 tx=0x7fae5800cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.001+0000 7fae56000640 1 -- 172.21.15.87:0/253668760 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58016020 con 0x7fae600b29e0 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.001+0000 7fae56000640 1 -- 172.21.15.87:0/253668760 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae58006d50 con 0x7fae600b29e0 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.001+0000 7fae56000640 1 -- 172.21.15.87:0/253668760 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58003bb0 con 0x7fae600b29e0 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.001+0000 7fae67200640 1 -- 172.21.15.87:0/253668760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6009d9a0 con 0x7fae600b29e0 2024-04-25T11:10:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.001+0000 7fae67200640 1 -- 172.21.15.87:0/253668760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae601629f0 con 0x7fae600b29e0 2024-04-25T11:10:00.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.001+0000 7fae56000640 1 -- 172.21.15.87:0/253668760 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fae5800abe0 con 0x7fae600b29e0 2024-04-25T11:10:00.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.001+0000 7fae56000640 1 -- 172.21.15.87:0/253668760 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fae58003d50 con 0x7fae600b29e0 2024-04-25T11:10:00.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.002+0000 7fae67200640 1 -- 172.21.15.87:0/253668760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae60050740 con 0x7fae600b29e0 2024-04-25T11:10:00.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.005+0000 7fae56000640 1 -- 172.21.15.87:0/253668760 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fae58020020 con 0x7fae600b29e0 2024-04-25T11:10:00.048 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:00.048+0000 7f7071be1040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T11:10:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.083+0000 7fae67200640 1 -- 172.21.15.87:0/253668760 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9b0485e2-f51d-462b-81ad-e00f2549b9df", "id": 6} v 0) -- 0x7fae600cf290 con 0x7fae600b29e0 2024-04-25T11:10:00.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.094+0000 7fae56000640 1 -- 172.21.15.87:0/253668760 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9b0485e2-f51d-462b-81ad-e00f2549b9df", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fae601fa6e0 con 0x7fae600b29e0 2024-04-25T11:10:00.095 INFO:teuthology.orchestra.run.smithi087.stdout:6 2024-04-25T11:10:00.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.096+0000 7fae67200640 1 -- 172.21.15.87:0/253668760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae600b29e0 msgr2=0x7fae6009d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.096+0000 7fae67200640 1 --2- 172.21.15.87:0/253668760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae600b29e0 0x7fae6009d460 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fae5800c730 tx=0x7fae5800cbf0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.096+0000 7fae67200640 1 -- 172.21.15.87:0/253668760 shutdown_connections 2024-04-25T11:10:00.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.096+0000 7fae67200640 1 --2- 172.21.15.87:0/253668760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae601f9d90 0x7fae601fa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.096+0000 7fae67200640 1 --2- 172.21.15.87:0/253668760 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae601604c0 0x7fae601f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.096+0000 7fae67200640 1 --2- 172.21.15.87:0/253668760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae600b29e0 0x7fae6009d460 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.096+0000 7fae67200640 1 -- 172.21.15.87:0/253668760 >> 172.21.15.87:0/253668760 conn(0x7fae6009b540 msgr2=0x7fae6015f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:00.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.096+0000 7fae67200640 1 -- 172.21.15.87:0/253668760 shutdown_connections 2024-04-25T11:10:00.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.096+0000 7fae67200640 1 -- 172.21.15.87:0/253668760 wait complete. 2024-04-25T11:10:00.113 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:00.112+0000 7f7071be1040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T11:10:00.119 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 2a8ffed7-232e-4325-8580-5c86d07ecf97 7 2024-04-25T11:10:00.253 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:00.253+0000 7f7071be1040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T11:10:00.319 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:10:00.319+0000 7f5f4ba0f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:10:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.361+0000 7f36fa200640 1 Processor -- start 2024-04-25T11:10:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.361+0000 7f36fa200640 1 -- start start 2024-04-25T11:10:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.363+0000 7f36fa200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f36f40d0480 0x7f36f40d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.363+0000 7f36fa200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f36f406eea0 0x7f36f406f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36fa200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f40ce570 con 0x7f36f41583c0 2024-04-25T11:10:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36fa200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f36f406f7f0 con 0x7f36f40d0480 2024-04-25T11:10:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36fa200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f36f415f190 con 0x7f36f406eea0 2024-04-25T11:10:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36f8e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f36f41583c0 0x7f36f40cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:56856/0 (socket says 172.21.15.87:56856) 2024-04-25T11:10:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36f9800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f36f406eea0 0x7f36f406f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36f8e00640 1 -- 172.21.15.87:0/3612378779 learned_addr learned my addr 172.21.15.87:0/3612378779 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T11:10:00.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36f3e00640 1 --2- 172.21.15.87:0/3612378779 >> v2:172.21.15.7:3300/0 conn(0x7f36f40d0480 0x7f36f40d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36f9800640 1 -- 172.21.15.87:0/3612378779 >> v2:172.21.15.7:3300/0 conn(0x7f36f40d0480 msgr2=0x7f36f40d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:00.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36f9800640 1 --2- 172.21.15.87:0/3612378779 >> v2:172.21.15.7:3300/0 conn(0x7f36f40d0480 0x7f36f40d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36f9800640 1 -- 172.21.15.87:0/3612378779 >> v1:172.21.15.87:6789/0 conn(0x7f36f41583c0 legacy=0x7f36f40cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36f9800640 1 -- 172.21.15.87:0/3612378779 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f40ce850 con 0x7f36f406eea0 2024-04-25T11:10:00.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36f3400640 1 -- 172.21.15.87:0/3612378779 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410590380 0 0) 0x7f36f40ce570 con 0x7f36f41583c0 2024-04-25T11:10:00.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.364+0000 7f36f3e00640 1 --2- 172.21.15.87:0/3612378779 >> v2:172.21.15.7:3300/0 conn(0x7f36f40d0480 0x7f36f40d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:00.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.366+0000 7f36f9800640 1 --2- 172.21.15.87:0/3612378779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f406eea0 0x7f36f406f2b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f36f40d0e00 tx=0x7f36ec02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb2988fb7b1dac15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:00.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.366+0000 7f36f3400640 1 -- 172.21.15.87:0/3612378779 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec007740 con 0x7f36f406eea0 2024-04-25T11:10:00.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.366+0000 7f36f3400640 1 -- 172.21.15.87:0/3612378779 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36ec0078e0 con 0x7f36f406eea0 2024-04-25T11:10:00.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.366+0000 7f36f3400640 1 -- 172.21.15.87:0/3612378779 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ec007cb0 con 0x7f36f406eea0 2024-04-25T11:10:00.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.367+0000 7f36fa200640 1 -- 172.21.15.87:0/3612378779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f406eea0 msgr2=0x7f36f406f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.367+0000 7f36fa200640 1 --2- 172.21.15.87:0/3612378779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f406eea0 0x7f36f406f2b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f36f40d0e00 tx=0x7f36ec02a5f0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.367+0000 7f36fa200640 1 -- 172.21.15.87:0/3612378779 shutdown_connections 2024-04-25T11:10:00.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.367+0000 7f36fa200640 1 --2- 172.21.15.87:0/3612378779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f406eea0 0x7f36f406f2b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.367+0000 7f36fa200640 1 --2- 172.21.15.87:0/3612378779 >> v2:172.21.15.7:3300/0 conn(0x7f36f40d0480 0x7f36f40d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.367+0000 7f36fa200640 1 -- 172.21.15.87:0/3612378779 >> 172.21.15.87:0/3612378779 conn(0x7f36f40740f0 msgr2=0x7f36f406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:00.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.367+0000 7f36fa200640 1 -- 172.21.15.87:0/3612378779 shutdown_connections 2024-04-25T11:10:00.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.367+0000 7f36fa200640 1 -- 172.21.15.87:0/3612378779 wait complete. 2024-04-25T11:10:00.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.370+0000 7f36fa200640 1 Processor -- start 2024-04-25T11:10:00.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.370+0000 7f36fa200640 1 -- start start 2024-04-25T11:10:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.371+0000 7f36fa200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f406eea0 0x7f36f41522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.371+0000 7f36fa200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40d0480 0x7f36f4143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.371+0000 7f36fa200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f41583c0 0x7f36f41e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.371+0000 7f36fa200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36f40cf0f0 con 0x7f36f41583c0 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.371+0000 7f36fa200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f36f40cee10 con 0x7f36f40d0480 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.371+0000 7f36fa200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f36f40cfd20 con 0x7f36f406eea0 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.372+0000 7f36f3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40d0480 0x7f36f4143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.372+0000 7f36f3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40d0480 0x7f36f4143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52212/0 (socket says 172.21.15.87:52212) 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.372+0000 7f36f3e00640 1 -- 172.21.15.87:0/2521949916 learned_addr learned my addr 172.21.15.87:0/2521949916 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.372+0000 7f36f8e00640 1 --2- 172.21.15.87:0/2521949916 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f406eea0 0x7f36f41522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.372+0000 7f36f9800640 1 --2- 172.21.15.87:0/2521949916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f41583c0 0x7f36f41e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.372+0000 7f36f3e00640 1 -- 172.21.15.87:0/2521949916 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f406eea0 msgr2=0x7f36f41522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.372+0000 7f36f3e00640 1 --2- 172.21.15.87:0/2521949916 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f406eea0 0x7f36f41522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.372+0000 7f36f3e00640 1 -- 172.21.15.87:0/2521949916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f41583c0 msgr2=0x7f36f41e8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.372+0000 7f36f3e00640 1 --2- 172.21.15.87:0/2521949916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f41583c0 0x7f36f41e8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.372+0000 7f36f3e00640 1 -- 172.21.15.87:0/2521949916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f4151200 con 0x7f36f40d0480 2024-04-25T11:10:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.373+0000 7f36f3e00640 1 --2- 172.21.15.87:0/2521949916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40d0480 0x7f36f4143ef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f36e000d800 tx=0x7f36e000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.373+0000 7f36f0c00640 1 -- 172.21.15.87:0/2521949916 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00096a0 con 0x7f36f40d0480 2024-04-25T11:10:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.373+0000 7f36f0c00640 1 -- 172.21.15.87:0/2521949916 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e0019070 con 0x7f36f40d0480 2024-04-25T11:10:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.373+0000 7f36f0c00640 1 -- 172.21.15.87:0/2521949916 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0009a80 con 0x7f36f40d0480 2024-04-25T11:10:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.373+0000 7f36fa200640 1 -- 172.21.15.87:0/2521949916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f41514e0 con 0x7f36f40d0480 2024-04-25T11:10:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.373+0000 7f36fa200640 1 -- 172.21.15.87:0/2521949916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f4151aa0 con 0x7f36f40d0480 2024-04-25T11:10:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.373+0000 7f36f0c00640 1 -- 172.21.15.87:0/2521949916 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f36e0009c20 con 0x7f36f40d0480 2024-04-25T11:10:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.373+0000 7f36f0c00640 1 -- 172.21.15.87:0/2521949916 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f36e0014ad0 con 0x7f36f40d0480 2024-04-25T11:10:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.374+0000 7f36fa200640 1 -- 172.21.15.87:0/2521949916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f4050740 con 0x7f36f40d0480 2024-04-25T11:10:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.376+0000 7f36f0c00640 1 -- 172.21.15.87:0/2521949916 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f36e0014d70 con 0x7f36f40d0480 2024-04-25T11:10:00.378 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:10:00.378+0000 7f5f4ba0f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:10:00.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.451+0000 7f36fa200640 1 -- 172.21.15.87:0/2521949916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2a8ffed7-232e-4325-8580-5c86d07ecf97", "id": 7} v 0) -- 0x7f36f40b5a30 con 0x7f36f40d0480 2024-04-25T11:10:00.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.462+0000 7f36f0c00640 1 -- 172.21.15.87:0/2521949916 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2a8ffed7-232e-4325-8580-5c86d07ecf97", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f36e0014500 con 0x7f36f40d0480 2024-04-25T11:10:00.462 INFO:teuthology.orchestra.run.smithi087.stdout:7 2024-04-25T11:10:00.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.463+0000 7f36fa200640 1 -- 172.21.15.87:0/2521949916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40d0480 msgr2=0x7f36f4143ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.463+0000 7f36fa200640 1 --2- 172.21.15.87:0/2521949916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40d0480 0x7f36f4143ef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f36e000d800 tx=0x7f36e000dcc0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.463+0000 7f36fa200640 1 -- 172.21.15.87:0/2521949916 shutdown_connections 2024-04-25T11:10:00.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.463+0000 7f36fa200640 1 --2- 172.21.15.87:0/2521949916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36f41583c0 0x7f36f41e8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.463+0000 7f36fa200640 1 --2- 172.21.15.87:0/2521949916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40d0480 0x7f36f4143ef0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.463+0000 7f36fa200640 1 --2- 172.21.15.87:0/2521949916 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f406eea0 0x7f36f41522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.463+0000 7f36fa200640 1 -- 172.21.15.87:0/2521949916 >> 172.21.15.87:0/2521949916 conn(0x7f36f40740f0 msgr2=0x7f36f40cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:00.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.464+0000 7f36fa200640 1 -- 172.21.15.87:0/2521949916 shutdown_connections 2024-04-25T11:10:00.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:10:00.464+0000 7f36fa200640 1 -- 172.21.15.87:0/2521949916 wait complete. 2024-04-25T11:10:00.486 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T11:10:00.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T11:10:00.494 INFO:tasks.ceph.osd.0:Started 2024-04-25T11:10:00.494 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T11:10:00.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T11:10:00.504 INFO:tasks.ceph.osd.1:Started 2024-04-25T11:10:00.504 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T11:10:00.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T11:10:00.509 INFO:tasks.ceph.osd.2:Started 2024-04-25T11:10:00.510 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T11:10:00.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T11:10:00.514 INFO:tasks.ceph.osd.3:Started 2024-04-25T11:10:00.514 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T11:10:00.515 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T11:10:00.521 INFO:tasks.ceph.osd.4:Started 2024-04-25T11:10:00.521 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T11:10:00.521 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T11:10:00.528 INFO:tasks.ceph.osd.5:Started 2024-04-25T11:10:00.528 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T11:10:00.528 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T11:10:00.533 INFO:tasks.ceph.osd.6:Started 2024-04-25T11:10:00.533 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T11:10:00.533 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T11:10:00.539 INFO:tasks.ceph.osd.7:Started 2024-04-25T11:10:00.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:00.593 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:10:00.593+0000 7f5f4ba0f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:10:00.628 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:00.627+0000 7f7071be1040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T11:10:00.698 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:00.697+0000 7f7071be1040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T11:10:00.807 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:10:00.807+0000 7f5f4ba0f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:10:00.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.821+0000 7f5ef2c00640 1 Processor -- start 2024-04-25T11:10:00.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.821+0000 7f5ef2c00640 1 -- start start 2024-04-25T11:10:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.825+0000 7f5ef2c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5eec164380 0x7f5eec0b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.825+0000 7f5ef2c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec0b5d40 0x7f5eec167f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.825+0000 7f5ef2c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eec0b47d0 con 0x7f5eec06ffd0 2024-04-25T11:10:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.825+0000 7f5ef2c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5eec0b4a40 con 0x7f5eec164380 2024-04-25T11:10:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.825+0000 7f5ef2c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5eec0b4d20 con 0x7f5eec0b5d40 2024-04-25T11:10:00.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.825+0000 7f5ef1800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5eec06ffd0 0x7f5eec0af390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:38890/0 (socket says 172.21.15.7:38890) 2024-04-25T11:10:00.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.825+0000 7f5ef1800640 1 -- 172.21.15.7:0/3975075099 learned_addr learned my addr 172.21.15.7:0/3975075099 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:00.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.825+0000 7f5ef0e00640 1 --2- 172.21.15.7:0/3975075099 >> v2:172.21.15.87:3300/0 conn(0x7f5eec164380 0x7f5eec0b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.825+0000 7f5ef2200640 1 --2- 172.21.15.7:0/3975075099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec0b5d40 0x7f5eec167f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.826+0000 7f5ef0e00640 1 -- 172.21.15.7:0/3975075099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec0b5d40 msgr2=0x7f5eec167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.826+0000 7f5ef0e00640 1 --2- 172.21.15.7:0/3975075099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec0b5d40 0x7f5eec167f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.826+0000 7f5ef0e00640 1 -- 172.21.15.7:0/3975075099 >> v1:172.21.15.87:6789/0 conn(0x7f5eec06ffd0 legacy=0x7f5eec0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.826+0000 7f5ef0e00640 1 -- 172.21.15.7:0/3975075099 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec0b5000 con 0x7f5eec164380 2024-04-25T11:10:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.826+0000 7f5ef2200640 1 --2- 172.21.15.7:0/3975075099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec0b5d40 0x7f5eec167f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.827+0000 7f5ef0e00640 1 --2- 172.21.15.7:0/3975075099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec164380 0x7f5eec0b5800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80097a0 tx=0x7f5ed802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4217aaffb75976bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.827+0000 7f5eebe00640 1 -- 172.21.15.7:0/3975075099 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8037070 con 0x7f5eec164380 2024-04-25T11:10:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.827+0000 7f5eebe00640 1 -- 172.21.15.7:0/3975075099 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed802ad10 con 0x7f5eec164380 2024-04-25T11:10:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.827+0000 7f5eebe00640 1 -- 172.21.15.7:0/3975075099 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8032660 con 0x7f5eec164380 2024-04-25T11:10:00.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.828+0000 7f5ef2c00640 1 -- 172.21.15.7:0/3975075099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec164380 msgr2=0x7f5eec0b5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.828+0000 7f5ef2c00640 1 --2- 172.21.15.7:0/3975075099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec164380 0x7f5eec0b5800 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ed80097a0 tx=0x7f5ed802a080 comp rx=0 tx=0).stop 2024-04-25T11:10:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.829+0000 7f5ef2c00640 1 -- 172.21.15.7:0/3975075099 shutdown_connections 2024-04-25T11:10:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.829+0000 7f5ef2c00640 1 --2- 172.21.15.7:0/3975075099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec0b5d40 0x7f5eec167f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.829+0000 7f5ef2c00640 1 --2- 172.21.15.7:0/3975075099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec164380 0x7f5eec0b5800 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.829+0000 7f5ef2c00640 1 -- 172.21.15.7:0/3975075099 >> 172.21.15.7:0/3975075099 conn(0x7f5eec0b3820 msgr2=0x7f5eec0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.829+0000 7f5ef2c00640 1 -- 172.21.15.7:0/3975075099 shutdown_connections 2024-04-25T11:10:00.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.829+0000 7f5ef2c00640 1 -- 172.21.15.7:0/3975075099 wait complete. 2024-04-25T11:10:00.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.833+0000 7f5ef2c00640 1 Processor -- start 2024-04-25T11:10:00.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.833+0000 7f5ef2c00640 1 -- start start 2024-04-25T11:10:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef2c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec06ffd0 0x7f5eec15d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef2c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eec0b5d40 0x7f5eec1f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef2c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec164380 0x7f5eec1f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef2c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec0afaa0 con 0x7f5eec0b5d40 2024-04-25T11:10:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef2c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec164790 con 0x7f5eec164380 2024-04-25T11:10:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef2c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5eec06fa70 con 0x7f5eec06ffd0 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef1800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec06ffd0 0x7f5eec15d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eec0b5d40 0x7f5eec1f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef1800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec06ffd0 0x7f5eec15d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:54476/0 (socket says 172.21.15.7:54476) 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eec0b5d40 0x7f5eec1f0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57486/0 (socket says 172.21.15.7:57486) 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef1800640 1 -- 172.21.15.7:0/581462145 learned_addr learned my addr 172.21.15.7:0/581462145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.836+0000 7f5ef2200640 1 --2- 172.21.15.7:0/581462145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec164380 0x7f5eec1f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.837+0000 7f5ef1800640 1 -- 172.21.15.7:0/581462145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec164380 msgr2=0x7f5eec1f0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.837+0000 7f5ef1800640 1 --2- 172.21.15.7:0/581462145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec164380 0x7f5eec1f0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.837+0000 7f5ef1800640 1 -- 172.21.15.7:0/581462145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eec0b5d40 msgr2=0x7f5eec1f0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.837+0000 7f5ef1800640 1 --2- 172.21.15.7:0/581462145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eec0b5d40 0x7f5eec1f0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.837+0000 7f5ef1800640 1 -- 172.21.15.7:0/581462145 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec1eed80 con 0x7f5eec06ffd0 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.837+0000 7f5ef0e00640 1 --2- 172.21.15.7:0/581462145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eec0b5d40 0x7f5eec1f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.837+0000 7f5ef1800640 1 --2- 172.21.15.7:0/581462145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec06ffd0 0x7f5eec15d290 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000c570 tx=0x7f5ee000ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.837+0000 7f5ee9600640 1 -- 172.21.15.7:0/581462145 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0018070 con 0x7f5eec06ffd0 2024-04-25T11:10:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.837+0000 7f5ee9600640 1 -- 172.21.15.7:0/581462145 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee00074e0 con 0x7f5eec06ffd0 2024-04-25T11:10:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.837+0000 7f5ee9600640 1 -- 172.21.15.7:0/581462145 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0007690 con 0x7f5eec06ffd0 2024-04-25T11:10:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.838+0000 7f5ef2c00640 1 -- 172.21.15.7:0/581462145 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eec1ef060 con 0x7f5eec06ffd0 2024-04-25T11:10:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.838+0000 7f5ef2c00640 1 -- 172.21.15.7:0/581462145 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eec1efb10 con 0x7f5eec06ffd0 2024-04-25T11:10:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.838+0000 7f5ee9600640 1 -- 172.21.15.7:0/581462145 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5ee001b070 con 0x7f5eec06ffd0 2024-04-25T11:10:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.838+0000 7f5ee9600640 1 -- 172.21.15.7:0/581462145 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5ee001d440 con 0x7f5eec06ffd0 2024-04-25T11:10:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.839+0000 7f5ef2c00640 1 -- 172.21.15.7:0/581462145 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eec0ae9b0 con 0x7f5eec06ffd0 2024-04-25T11:10:00.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.842+0000 7f5ee9600640 1 -- 172.21.15.7:0/581462145 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5eec0ae9b0 con 0x7f5eec06ffd0 2024-04-25T11:10:00.880 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:10:00.879+0000 7f5f4ba0f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:10:00.899 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:00.899+0000 7f7071be1040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T11:10:00.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.908+0000 7f5ef2c00640 1 -- 172.21.15.7:0/581462145 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5eec050740 con 0x7f5eec06ffd0 2024-04-25T11:10:00.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.910+0000 7f5ee9600640 1 -- 172.21.15.7:0/581462145 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5ee0022080 con 0x7f5eec06ffd0 2024-04-25T11:10:00.910 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:00.910 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:00.454539+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:00.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.911+0000 7f5ef2c00640 1 -- 172.21.15.7:0/581462145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec06ffd0 msgr2=0x7f5eec15d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:00.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.911+0000 7f5ef2c00640 1 --2- 172.21.15.7:0/581462145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec06ffd0 0x7f5eec15d290 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000c570 tx=0x7f5ee000ca30 comp rx=0 tx=0).stop 2024-04-25T11:10:00.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.912+0000 7f5ef2c00640 1 -- 172.21.15.7:0/581462145 shutdown_connections 2024-04-25T11:10:00.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.912+0000 7f5ef2c00640 1 --2- 172.21.15.7:0/581462145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eec164380 0x7f5eec1f0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.912+0000 7f5ef2c00640 1 --2- 172.21.15.7:0/581462145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eec0b5d40 0x7f5eec1f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.912+0000 7f5ef2c00640 1 --2- 172.21.15.7:0/581462145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eec06ffd0 0x7f5eec15d290 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:00.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.912+0000 7f5ef2c00640 1 -- 172.21.15.7:0/581462145 >> 172.21.15.7:0/581462145 conn(0x7f5eec0b3820 msgr2=0x7f5eec0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:00.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.912+0000 7f5ef2c00640 1 -- 172.21.15.7:0/581462145 shutdown_connections 2024-04-25T11:10:00.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:00.912+0000 7f5ef2c00640 1 -- 172.21.15.7:0/581462145 wait complete. 2024-04-25T11:10:00.950 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T11:10:00.951 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T11:10:00.983 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:10:00.982+0000 7f5f4ba0f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:10:01.038 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:10:01.038+0000 7f5f4ba0f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:10:01.067 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:01.067+0000 7f7071be1040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T11:10:01.127 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:01.127+0000 7f7071be1040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T11:10:01.208 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:01.207+0000 7f7071be1040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T11:10:01.261 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:01.261+0000 7f7071be1040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T11:10:01.275 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-04-25T11:10:01.274+0000 7f1c7fed9640 -1 Falling back to public interface 2024-04-25T11:10:01.284 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-04-25T11:10:01.284+0000 7fa40962c640 -1 Falling back to public interface 2024-04-25T11:10:01.323 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-04-25T11:10:01.323+0000 7f30307c4640 -1 Falling back to public interface 2024-04-25T11:10:01.335 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-04-25T11:10:01.335+0000 7fd9fa80c640 -1 Falling back to public interface 2024-04-25T11:10:01.343 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-04-25T11:10:01.343+0000 7fb7c1add640 -1 Falling back to public interface 2024-04-25T11:10:01.352 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-04-25T11:10:01.352+0000 7fb81b73e640 -1 Falling back to public interface 2024-04-25T11:10:01.356 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-04-25T11:10:01.356+0000 7ffb13e14640 -1 Falling back to public interface 2024-04-25T11:10:01.363 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-04-25T11:10:01.363+0000 7f093a80d640 -1 Falling back to public interface 2024-04-25T11:10:01.608 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:10:01.608+0000 7f5f4ba0f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:10:01.745 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:10:01.745+0000 7f5f4ba0f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:10:01.756 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:01.816 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:01.815+0000 7f7071be1040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T11:10:01.952 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T11:10:01.951+0000 7f7071be1040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T11:10:01.978 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.977+0000 7f0d9e200640 1 Processor -- start 2024-04-25T11:10:01.978 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.978+0000 7f0d9e200640 1 -- start start 2024-04-25T11:10:01.981 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.979+0000 7f0d9e200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d980d1050 0x7f0d98162540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:01.981 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.979+0000 7f0d9e200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d9806d580 0x7f0d9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:01.981 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.979+0000 7f0d9e200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d9806ded0 con 0x7f0d9806f1e0 2024-04-25T11:10:01.981 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.979+0000 7f0d9e200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0d980706c0 con 0x7f0d9806d580 2024-04-25T11:10:01.982 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.979+0000 7f0d9e200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0d98074410 con 0x7f0d980d1050 2024-04-25T11:10:01.982 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.980+0000 7f0d8fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d9806d580 0x7f0d9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:01.982 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.980+0000 7f0d8fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d9806d580 0x7f0d9806d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57564/0 (socket says 172.21.15.7:57564) 2024-04-25T11:10:01.982 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.980+0000 7f0d8fe00640 1 -- 172.21.15.7:0/88524701 learned_addr learned my addr 172.21.15.7:0/88524701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:01.982 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.980+0000 7f0d8fe00640 1 -- 172.21.15.7:0/88524701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d980d1050 msgr2=0x7f0d98162540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:01.982 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.980+0000 7f0d8fe00640 1 --2- 172.21.15.7:0/88524701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d980d1050 0x7f0d98162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:01.983 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.980+0000 7f0d8fe00640 1 -- 172.21.15.7:0/88524701 >> v1:172.21.15.87:6789/0 conn(0x7f0d9806f1e0 legacy=0x7f0d9806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:01.983 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.980+0000 7f0d8fe00640 1 -- 172.21.15.7:0/88524701 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d98161a60 con 0x7f0d9806d580 2024-04-25T11:10:01.986 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.981+0000 7f0d8fe00640 1 --2- 172.21.15.7:0/88524701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9806d580 0x7f0d9806d990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009870 tx=0x7f0d8002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d6298b552c72e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:01.986 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.982+0000 7f0d97e00640 1 -- 172.21.15.7:0/88524701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80037070 con 0x7f0d9806d580 2024-04-25T11:10:01.986 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.982+0000 7f0d97e00640 1 -- 172.21.15.7:0/88524701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8002ae10 con 0x7f0d9806d580 2024-04-25T11:10:01.986 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.982+0000 7f0d97e00640 1 -- 172.21.15.7:0/88524701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d80032660 con 0x7f0d9806d580 2024-04-25T11:10:01.986 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.982+0000 7f0d9e200640 1 -- 172.21.15.7:0/88524701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9806d580 msgr2=0x7f0d9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:01.986 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.982+0000 7f0d9e200640 1 --2- 172.21.15.7:0/88524701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9806d580 0x7f0d9806d990 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009870 tx=0x7f0d8002a100 comp rx=0 tx=0).stop 2024-04-25T11:10:01.986 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.982+0000 7f0d9e200640 1 -- 172.21.15.7:0/88524701 shutdown_connections 2024-04-25T11:10:01.987 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.982+0000 7f0d9e200640 1 --2- 172.21.15.7:0/88524701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9806d580 0x7f0d9806d990 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:01.987 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.982+0000 7f0d9e200640 1 --2- 172.21.15.7:0/88524701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d980d1050 0x7f0d98162540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:01.987 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.982+0000 7f0d9e200640 1 -- 172.21.15.7:0/88524701 >> 172.21.15.7:0/88524701 conn(0x7f0d98073180 msgr2=0x7f0d98073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:01.987 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.983+0000 7f0d9e200640 1 -- 172.21.15.7:0/88524701 shutdown_connections 2024-04-25T11:10:01.987 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.983+0000 7f0d9e200640 1 -- 172.21.15.7:0/88524701 wait complete. 2024-04-25T11:10:01.987 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.985+0000 7f0d9e200640 1 Processor -- start 2024-04-25T11:10:01.987 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.985+0000 7f0d9e200640 1 -- start start 2024-04-25T11:10:01.990 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.986+0000 7f0d9e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9806d580 0x7f0d9815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:01.990 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.986+0000 7f0d9e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d9806f1e0 0x7f0d981500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:01.990 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.986+0000 7f0d9e200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d980d1050 0x7f0d981607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:01.990 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.986+0000 7f0d9e200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98070910 con 0x7f0d9806d580 2024-04-25T11:10:01.990 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.986+0000 7f0d9e200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0d98162020 con 0x7f0d9806f1e0 2024-04-25T11:10:01.990 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.986+0000 7f0d9e200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0d980b06c0 con 0x7f0d980d1050 2024-04-25T11:10:01.990 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.987+0000 7f0d9ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9806d580 0x7f0d9815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:01.990 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.987+0000 7f0d9ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9806d580 0x7f0d9815eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57566/0 (socket says 172.21.15.7:57566) 2024-04-25T11:10:01.990 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.987+0000 7f0d9ce00640 1 -- 172.21.15.7:0/3099457512 learned_addr learned my addr 172.21.15.7:0/3099457512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.987+0000 7f0d9ce00640 1 -- 172.21.15.7:0/3099457512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d980d1050 msgr2=0x7f0d981607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.987+0000 7f0d9ce00640 1 --2- 172.21.15.7:0/3099457512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d980d1050 0x7f0d981607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.987+0000 7f0d9ce00640 1 -- 172.21.15.7:0/3099457512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d9806f1e0 msgr2=0x7f0d981500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.987+0000 7f0d8fe00640 1 --2- 172.21.15.7:0/3099457512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d9806f1e0 0x7f0d981500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.987+0000 7f0d9ce00640 1 --2- 172.21.15.7:0/3099457512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d9806f1e0 0x7f0d981500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.987+0000 7f0d9ce00640 1 -- 172.21.15.7:0/3099457512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9815f720 con 0x7f0d9806d580 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.987+0000 7f0d8fe00640 1 --2- 172.21.15.7:0/3099457512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d9806f1e0 0x7f0d981500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.987+0000 7f0d9ce00640 1 --2- 172.21.15.7:0/3099457512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9806d580 0x7f0d9815eed0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800b440 tx=0x7f0d8800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.988+0000 7f0d95600640 1 -- 172.21.15.7:0/3099457512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88015020 con 0x7f0d9806d580 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.988+0000 7f0d95600640 1 -- 172.21.15.7:0/3099457512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d880118b0 con 0x7f0d9806d580 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.988+0000 7f0d95600640 1 -- 172.21.15.7:0/3099457512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88011ca0 con 0x7f0d9806d580 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.988+0000 7f0d9e200640 1 -- 172.21.15.7:0/3099457512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9815fa00 con 0x7f0d9806d580 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.988+0000 7f0d9e200640 1 -- 172.21.15.7:0/3099457512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9815ffc0 con 0x7f0d9806d580 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.989+0000 7f0d9e200640 1 -- 172.21.15.7:0/3099457512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9815d680 con 0x7f0d9806d580 2024-04-25T11:10:01.991 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.989+0000 7f0d95600640 1 -- 172.21.15.7:0/3099457512 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 2) ==== 48416+0+0 (secure 0 0 0) 0x7f0d88018020 con 0x7f0d9806d580 2024-04-25T11:10:01.992 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.989+0000 7f0d95600640 1 -- 172.21.15.7:0/3099457512 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0d8804f700 con 0x7f0d9806d580 2024-04-25T11:10:01.992 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:01.992+0000 7f0d95600640 1 -- 172.21.15.7:0/3099457512 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0d9815d680 con 0x7f0d9806d580 2024-04-25T11:10:02.057 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:02.057+0000 7f0d9e200640 1 -- 172.21.15.7:0/3099457512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d980769d0 con 0x7f0d9806d580 2024-04-25T11:10:02.058 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:02.058+0000 7f0d95600640 1 -- 172.21.15.7:0/3099457512 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0d8801f020 con 0x7f0d9806d580 2024-04-25T11:10:02.058 INFO:teuthology.misc.health.smithi007.stdout: 2024-04-25T11:10:02.059 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":10,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:00.454539+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:02.060 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:02.059+0000 7f0d9e200640 1 -- 172.21.15.7:0/3099457512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9806d580 msgr2=0x7f0d9815eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:02.060 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:02.059+0000 7f0d9e200640 1 --2- 172.21.15.7:0/3099457512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9806d580 0x7f0d9815eed0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800b440 tx=0x7f0d8800b900 comp rx=0 tx=0).stop 2024-04-25T11:10:02.060 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:02.059+0000 7f0d9e200640 1 -- 172.21.15.7:0/3099457512 shutdown_connections 2024-04-25T11:10:02.060 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:02.059+0000 7f0d9e200640 1 --2- 172.21.15.7:0/3099457512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d980d1050 0x7f0d981607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:02.060 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:02.059+0000 7f0d9e200640 1 --2- 172.21.15.7:0/3099457512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d9806f1e0 0x7f0d981500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:02.060 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:02.059+0000 7f0d9e200640 1 --2- 172.21.15.7:0/3099457512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9806d580 0x7f0d9815eed0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:02.060 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:02.059+0000 7f0d9e200640 1 -- 172.21.15.7:0/3099457512 >> 172.21.15.7:0/3099457512 conn(0x7f0d98073180 msgr2=0x7f0d9815cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:02.060 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:02.060+0000 7f0d9e200640 1 -- 172.21.15.7:0/3099457512 shutdown_connections 2024-04-25T11:10:02.060 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:02.060+0000 7f0d9e200640 1 -- 172.21.15.7:0/3099457512 wait complete. 2024-04-25T11:10:02.078 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T11:10:02.784 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:10:02.784+0000 7f5eeb200640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T11:10:02.785 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-04-25T11:10:02.785+0000 7f5eeb200640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T11:10:04.758 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-04-25T11:10:04.757+0000 7fb81b73e640 -1 osd.2 0 log_to_monitors true 2024-04-25T11:10:04.954 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-04-25T11:10:04.954+0000 7fb7c1add640 -1 osd.3 0 log_to_monitors true 2024-04-25T11:10:04.960 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-04-25T11:10:04.960+0000 7fa40962c640 -1 osd.4 0 log_to_monitors true 2024-04-25T11:10:04.972 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-04-25T11:10:04.972+0000 7f093a80d640 -1 osd.1 0 log_to_monitors true 2024-04-25T11:10:04.972 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-04-25T11:10:04.972+0000 7f30307c4640 -1 osd.5 0 log_to_monitors true 2024-04-25T11:10:05.144 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-04-25T11:10:05.143+0000 7f1c7fed9640 -1 osd.7 0 log_to_monitors true 2024-04-25T11:10:05.185 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-04-25T11:10:05.185+0000 7fd9fa80c640 -1 osd.0 0 log_to_monitors true 2024-04-25T11:10:05.215 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-04-25T11:10:05.215+0000 7ffb13e14640 -1 osd.6 0 log_to_monitors true 2024-04-25T11:10:06.886 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-04-25T11:10:06.886+0000 7fb816600640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T11:10:06.901 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-04-25T11:10:06.900+0000 7fb80c600640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:10:08.308 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-04-25T11:10:08.308+0000 7f0935a00640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T11:10:08.316 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-04-25T11:10:08.316+0000 7f092b000640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:10:08.353 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-04-25T11:10:08.352+0000 7fd9f5000640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T11:10:08.358 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-04-25T11:10:08.358+0000 7fb7bca00640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T11:10:08.359 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-04-25T11:10:08.359+0000 7fd9eb000640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:10:08.365 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-04-25T11:10:08.365+0000 7fb7b2a00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:10:08.886 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:09.149 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.148+0000 7f9ca8e00640 1 Processor -- start 2024-04-25T11:10:09.149 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.149+0000 7f9ca8e00640 1 -- start start 2024-04-25T11:10:09.152 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.151+0000 7f9ca8e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca406dc90 0x7f9ca4162540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:09.152 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.151+0000 7f9ca8e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9ca415ddc0 0x7f9ca415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:09.152 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.151+0000 7f9ca8e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca4074410 con 0x7f9ca40d1050 2024-04-25T11:10:09.152 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.151+0000 7f9ca8e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca40706c0 con 0x7f9ca415ddc0 2024-04-25T11:10:09.152 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.151+0000 7f9ca8e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca4161a60 con 0x7f9ca406dc90 2024-04-25T11:10:09.152 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.152+0000 7f9c9be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca406dc90 0x7f9ca4162540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:09.152 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.152+0000 7f9ca3400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9ca40d1050 0x7f9ca406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:38920/0 (socket says 172.21.15.7:38920) 2024-04-25T11:10:09.152 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.152+0000 7f9c9be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca406dc90 0x7f9ca4162540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:54614/0 (socket says 172.21.15.7:54614) 2024-04-25T11:10:09.152 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.152+0000 7f9ca3400640 1 -- 172.21.15.7:0/2388365457 learned_addr learned my addr 172.21.15.7:0/2388365457 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:09.152 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.152+0000 7f9ca3e00640 1 --2- 172.21.15.7:0/2388365457 >> v2:172.21.15.87:3300/0 conn(0x7f9ca415ddc0 0x7f9ca415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:09.153 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.152+0000 7f9c9be00640 1 -- 172.21.15.7:0/2388365457 >> v2:172.21.15.87:3300/0 conn(0x7f9ca415ddc0 msgr2=0x7f9ca415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:09.153 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.152+0000 7f9c9be00640 1 --2- 172.21.15.7:0/2388365457 >> v2:172.21.15.87:3300/0 conn(0x7f9ca415ddc0 0x7f9ca415e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:09.153 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.152+0000 7f9c9be00640 1 -- 172.21.15.7:0/2388365457 >> v1:172.21.15.87:6789/0 conn(0x7f9ca40d1050 legacy=0x7f9ca406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:09.153 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.152+0000 7f9c9be00640 1 -- 172.21.15.7:0/2388365457 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca4161d40 con 0x7f9ca406dc90 2024-04-25T11:10:09.153 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.152+0000 7f9ca3e00640 1 --2- 172.21.15.7:0/2388365457 >> v2:172.21.15.87:3300/0 conn(0x7f9ca415ddc0 0x7f9ca415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:10:09.153 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.153+0000 7f9c9be00640 1 --2- 172.21.15.7:0/2388365457 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca406dc90 0x7f9ca4162540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0097a0 tx=0x7f9c8c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b515ef0311a866ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:09.154 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.154+0000 7f9ca2a00640 1 -- 172.21.15.7:0/2388365457 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c037070 con 0x7f9ca406dc90 2024-04-25T11:10:09.154 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.154+0000 7f9ca2a00640 1 -- 172.21.15.7:0/2388365457 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8c02ae50 con 0x7f9ca406dc90 2024-04-25T11:10:09.154 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.154+0000 7f9ca2a00640 1 -- 172.21.15.7:0/2388365457 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c032660 con 0x7f9ca406dc90 2024-04-25T11:10:09.155 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.155+0000 7f9ca8e00640 1 -- 172.21.15.7:0/2388365457 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca406dc90 msgr2=0x7f9ca4162540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:09.155 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.155+0000 7f9ca8e00640 1 --2- 172.21.15.7:0/2388365457 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca406dc90 0x7f9ca4162540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c0097a0 tx=0x7f9c8c02a100 comp rx=0 tx=0).stop 2024-04-25T11:10:09.155 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.155+0000 7f9ca8e00640 1 -- 172.21.15.7:0/2388365457 shutdown_connections 2024-04-25T11:10:09.155 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.155+0000 7f9ca8e00640 1 --2- 172.21.15.7:0/2388365457 >> v2:172.21.15.87:3300/0 conn(0x7f9ca415ddc0 0x7f9ca415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:09.155 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.155+0000 7f9ca8e00640 1 --2- 172.21.15.7:0/2388365457 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca406dc90 0x7f9ca4162540 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:09.155 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.155+0000 7f9ca8e00640 1 -- 172.21.15.7:0/2388365457 >> 172.21.15.7:0/2388365457 conn(0x7f9ca4073180 msgr2=0x7f9ca4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:09.155 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.155+0000 7f9ca8e00640 1 -- 172.21.15.7:0/2388365457 shutdown_connections 2024-04-25T11:10:09.155 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.155+0000 7f9ca8e00640 1 -- 172.21.15.7:0/2388365457 wait complete. 2024-04-25T11:10:09.157 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.157+0000 7f9ca8e00640 1 Processor -- start 2024-04-25T11:10:09.157 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.157+0000 7f9ca8e00640 1 -- start start 2024-04-25T11:10:09.159 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca40d1050 0x7f9ca414d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:09.159 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca8e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca415ddc0 0x7f9ca40b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca40b58b0 0x7f9ca41583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca8e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca40b06c0 con 0x7f9ca40d1050 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca8e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca4070910 con 0x7f9ca40b58b0 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca8e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca40b0840 con 0x7f9ca415ddc0 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca40d1050 0x7f9ca414d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca40d1050 0x7f9ca414d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57616/0 (socket says 172.21.15.7:57616) 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca3400640 1 -- 172.21.15.7:0/262724971 learned_addr learned my addr 172.21.15.7:0/262724971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9c9be00640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca415ddc0 0x7f9ca40b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca3e00640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca40b58b0 0x7f9ca41583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca3400640 1 -- 172.21.15.7:0/262724971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca415ddc0 msgr2=0x7f9ca40b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca3400640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca415ddc0 0x7f9ca40b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca3400640 1 -- 172.21.15.7:0/262724971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca40b58b0 msgr2=0x7f9ca41583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca3400640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca40b58b0 0x7f9ca41583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:09.160 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.159+0000 7f9ca3400640 1 -- 172.21.15.7:0/262724971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca4158fa0 con 0x7f9ca40d1050 2024-04-25T11:10:09.161 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.160+0000 7f9ca3400640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca40d1050 0x7f9ca414d290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400c470 tx=0x7f9c9400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:09.161 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.160+0000 7f9c9b400640 1 -- 172.21.15.7:0/262724971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c94017070 con 0x7f9ca40d1050 2024-04-25T11:10:09.162 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.160+0000 7f9c9b400640 1 -- 172.21.15.7:0/262724971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c94005ce0 con 0x7f9ca40d1050 2024-04-25T11:10:09.162 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.160+0000 7f9c9b400640 1 -- 172.21.15.7:0/262724971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c940072d0 con 0x7f9ca40d1050 2024-04-25T11:10:09.162 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.160+0000 7f9ca8e00640 1 -- 172.21.15.7:0/262724971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca4159280 con 0x7f9ca40d1050 2024-04-25T11:10:09.162 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.160+0000 7f9ca8e00640 1 -- 172.21.15.7:0/262724971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca4159840 con 0x7f9ca40d1050 2024-04-25T11:10:09.162 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.161+0000 7f9ca8e00640 1 -- 172.21.15.7:0/262724971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca40769d0 con 0x7f9ca40d1050 2024-04-25T11:10:09.162 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.162+0000 7f9c9b400640 1 -- 172.21.15.7:0/262724971 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) ==== 96726+0+0 (secure 0 0 0) 0x7f9c9401a070 con 0x7f9ca40d1050 2024-04-25T11:10:09.163 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.163+0000 7f9c9b400640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c80074420 0x7f9c80076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:09.163 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.163+0000 7f9c9b400640 1 -- 172.21.15.7:0/262724971 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7f9c94093570 con 0x7f9ca40d1050 2024-04-25T11:10:09.164 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.164+0000 7f9c9be00640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c80074420 0x7f9c80076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:09.164 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.164+0000 7f9c9be00640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c80074420 0x7f9c80076930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c034000 tx=0x7f9c8c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:09.169 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.169+0000 7f9c9b400640 1 -- 172.21.15.7:0/262724971 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c9405e410 con 0x7f9ca40d1050 2024-04-25T11:10:09.309 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-04-25T11:10:09.309+0000 7f302b600640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T11:10:09.312 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-04-25T11:10:09.312+0000 7fa404400640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T11:10:09.317 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-04-25T11:10:09.316+0000 7f3021600640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:10:09.324 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-04-25T11:10:09.323+0000 7fa3f9a00640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:10:09.333 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-04-25T11:10:09.333+0000 7ffb0e600640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T11:10:09.340 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-04-25T11:10:09.340+0000 7ffb04600640 -1 osd.6 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:10:09.362 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-04-25T11:10:09.362+0000 7f1c7ae00640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T11:10:09.367 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-04-25T11:10:09.367+0000 7f1c70e00640 -1 osd.7 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T11:10:09.380 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.380+0000 7f9ca8e00640 1 -- 172.21.15.7:0/262724971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ca4074160 con 0x7f9ca40d1050 2024-04-25T11:10:09.382 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.382+0000 7f9c9b400640 1 -- 172.21.15.7:0/262724971 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+8952 (secure 0 0 0) 0x7f9c94062710 con 0x7f9ca40d1050 2024-04-25T11:10:09.382 INFO:teuthology.misc.health.smithi007.stdout: 2024-04-25T11:10:09.382 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":15,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:09.314123+0000","last_up_change":"2024-04-25T11:10:09.314123+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:09.385 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.384+0000 7f9ca8e00640 1 -- 172.21.15.7:0/262724971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c80074420 msgr2=0x7f9c80076930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:09.385 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.384+0000 7f9ca8e00640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c80074420 0x7f9c80076930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c034000 tx=0x7f9c8c02e000 comp rx=0 tx=0).stop 2024-04-25T11:10:09.385 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.384+0000 7f9ca8e00640 1 -- 172.21.15.7:0/262724971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca40d1050 msgr2=0x7f9ca414d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:09.385 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.384+0000 7f9ca8e00640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca40d1050 0x7f9ca414d290 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400c470 tx=0x7f9c9400c870 comp rx=0 tx=0).stop 2024-04-25T11:10:09.385 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.384+0000 7f9ca8e00640 1 -- 172.21.15.7:0/262724971 shutdown_connections 2024-04-25T11:10:09.385 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.384+0000 7f9ca8e00640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c80074420 0x7f9c80076930 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:09.385 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.384+0000 7f9ca8e00640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca40b58b0 0x7f9ca41583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:09.385 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.384+0000 7f9ca8e00640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca415ddc0 0x7f9ca40b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:09.385 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.384+0000 7f9ca8e00640 1 --2- 172.21.15.7:0/262724971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca40d1050 0x7f9ca414d290 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:09.385 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.384+0000 7f9ca8e00640 1 -- 172.21.15.7:0/262724971 >> 172.21.15.7:0/262724971 conn(0x7f9ca4073180 msgr2=0x7f9ca40b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:09.386 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.385+0000 7f9ca8e00640 1 -- 172.21.15.7:0/262724971 shutdown_connections 2024-04-25T11:10:09.386 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:09.385+0000 7f9ca8e00640 1 -- 172.21.15.7:0/262724971 wait complete. 2024-04-25T11:10:09.406 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-04-25T11:10:10.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:10:10.454 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:10:16.214 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:16.484 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.484+0000 7f8718c00640 1 Processor -- start 2024-04-25T11:10:16.485 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.484+0000 7f8718c00640 1 -- start start 2024-04-25T11:10:16.487 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.486+0000 7f8718c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87140d0a10 0x7f871415e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:16.487 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.487+0000 7f8718c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871406c990 con 0x7f871406ffa0 2024-04-25T11:10:16.487 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.487+0000 7f8718c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871406cc70 con 0x7f8714070ec0 2024-04-25T11:10:16.487 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.487+0000 7f8718c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f871406d5a0 con 0x7f87140d0a10 2024-04-25T11:10:16.487 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.487+0000 7f8709400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87140d0a10 0x7f871415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:16.487 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.487+0000 7f8709400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87140d0a10 0x7f871415e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54572/0 (socket says 172.21.15.7:54572) 2024-04-25T11:10:16.487 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.487+0000 7f8709400640 1 -- 172.21.15.7:0/385851456 learned_addr learned my addr 172.21.15.7:0/385851456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:16.488 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.487+0000 7f8709400640 1 -- 172.21.15.7:0/385851456 >> v1:172.21.15.87:6789/0 conn(0x7f8714070ec0 legacy=0x7f87140d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:16.488 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.487+0000 7f8709400640 1 -- 172.21.15.7:0/385851456 >> v1:172.21.15.7:6789/0 conn(0x7f871406ffa0 legacy=0x7f87140b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:16.488 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.487+0000 7f8709400640 1 -- 172.21.15.7:0/385851456 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871406cf50 con 0x7f87140d0a10 2024-04-25T11:10:16.488 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.487+0000 7f8711400640 1 -- 172.21.15.7:0/385851456 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4116782185 0 0) 0x7f871406c990 con 0x7f871406ffa0 2024-04-25T11:10:16.489 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.489+0000 7f8709400640 1 --2- 172.21.15.7:0/385851456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87140d0a10 0x7f871415e040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009870 tx=0x7f86fc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efec368c3a3530b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:16.489 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.489+0000 7f8711400640 1 -- 172.21.15.7:0/385851456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc037070 con 0x7f87140d0a10 2024-04-25T11:10:16.489 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.489+0000 7f8711400640 1 -- 172.21.15.7:0/385851456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86fc032440 con 0x7f87140d0a10 2024-04-25T11:10:16.489 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.489+0000 7f8711400640 1 -- 172.21.15.7:0/385851456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc032810 con 0x7f87140d0a10 2024-04-25T11:10:16.490 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.490+0000 7f8718c00640 1 -- 172.21.15.7:0/385851456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87140d0a10 msgr2=0x7f871415e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:16.491 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.490+0000 7f8718c00640 1 --2- 172.21.15.7:0/385851456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87140d0a10 0x7f871415e040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009870 tx=0x7f86fc02a210 comp rx=0 tx=0).stop 2024-04-25T11:10:16.491 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.490+0000 7f8718c00640 1 -- 172.21.15.7:0/385851456 shutdown_connections 2024-04-25T11:10:16.491 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.490+0000 7f8718c00640 1 --2- 172.21.15.7:0/385851456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87140d0a10 0x7f871415e040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:16.491 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.490+0000 7f8718c00640 1 -- 172.21.15.7:0/385851456 >> 172.21.15.7:0/385851456 conn(0x7f87140ce240 msgr2=0x7f87140ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:16.491 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.490+0000 7f8718c00640 1 -- 172.21.15.7:0/385851456 shutdown_connections 2024-04-25T11:10:16.491 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.491+0000 7f8718c00640 1 -- 172.21.15.7:0/385851456 wait complete. 2024-04-25T11:10:16.493 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.493+0000 7f8718c00640 1 Processor -- start 2024-04-25T11:10:16.493 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.493+0000 7f8718c00640 1 -- start start 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.495+0000 7f8718c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f871406ffa0 0x7f87140c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.495+0000 7f8718c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8714070ec0 0x7f87140c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.495+0000 7f8718c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87140c7c20 0x7f87140ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.495+0000 7f8718c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f871406c6f0 con 0x7f871406ffa0 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.495+0000 7f8718c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f871406c570 con 0x7f8714070ec0 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.495+0000 7f8718c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f871406da30 con 0x7f87140c7c20 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.495+0000 7f8711e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f871406ffa0 0x7f87140c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.495+0000 7f8711e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f871406ffa0 0x7f87140c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54580/0 (socket says 172.21.15.7:54580) 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.495+0000 7f8711e00640 1 -- 172.21.15.7:0/4240630313 learned_addr learned my addr 172.21.15.7:0/4240630313 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.496+0000 7f8709400640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8714070ec0 0x7f87140c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.496+0000 7f8711e00640 1 -- 172.21.15.7:0/4240630313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87140c7c20 msgr2=0x7f87140ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.496+0000 7f8712800640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87140c7c20 0x7f87140ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:16.497 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.496+0000 7f8711e00640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87140c7c20 0x7f87140ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:16.498 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.496+0000 7f8711e00640 1 -- 172.21.15.7:0/4240630313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8714070ec0 msgr2=0x7f87140c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:16.498 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.496+0000 7f8711e00640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8714070ec0 0x7f87140c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:16.498 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.496+0000 7f8711e00640 1 -- 172.21.15.7:0/4240630313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87141588a0 con 0x7f871406ffa0 2024-04-25T11:10:16.498 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.496+0000 7f8709400640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8714070ec0 0x7f87140c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:16.498 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.496+0000 7f8711e00640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f871406ffa0 0x7f87140c98a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f870400ba50 tx=0x7f870400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:16.498 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.496+0000 7f870aa00640 1 -- 172.21.15.7:0/4240630313 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704019070 con 0x7f871406ffa0 2024-04-25T11:10:16.498 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.496+0000 7f870aa00640 1 -- 172.21.15.7:0/4240630313 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870400cbc0 con 0x7f871406ffa0 2024-04-25T11:10:16.498 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.497+0000 7f8718c00640 1 -- 172.21.15.7:0/4240630313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8714158b80 con 0x7f871406ffa0 2024-04-25T11:10:16.498 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.497+0000 7f870aa00640 1 -- 172.21.15.7:0/4240630313 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704014440 con 0x7f871406ffa0 2024-04-25T11:10:16.498 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.497+0000 7f8718c00640 1 -- 172.21.15.7:0/4240630313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8714159140 con 0x7f871406ffa0 2024-04-25T11:10:16.499 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.498+0000 7f8718c00640 1 -- 172.21.15.7:0/4240630313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8714050740 con 0x7f871406ffa0 2024-04-25T11:10:16.499 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.499+0000 7f870aa00640 1 -- 172.21.15.7:0/4240630313 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f870401c070 con 0x7f871406ffa0 2024-04-25T11:10:16.500 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.500+0000 7f870aa00640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f86f00744b0 0x7f86f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:16.500 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.500+0000 7f870aa00640 1 -- 172.21.15.7:0/4240630313 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f8704094c20 con 0x7f871406ffa0 2024-04-25T11:10:16.500 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.500+0000 7f8709400640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f86f00744b0 0x7f86f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:16.501 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.501+0000 7f8709400640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f86f00744b0 0x7f86f00769c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f86fc034000 tx=0x7f86fc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:16.506 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.506+0000 7f870aa00640 1 -- 172.21.15.7:0/4240630313 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f870405f010 con 0x7f871406ffa0 2024-04-25T11:10:16.717 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.717+0000 7f8718c00640 1 -- 172.21.15.7:0/4240630313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87140c8070 con 0x7f871406ffa0 2024-04-25T11:10:16.720 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.719+0000 7f870aa00640 1 -- 172.21.15.7:0/4240630313 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12788 (secure 0 0 0) 0x7f8704063310 con 0x7f871406ffa0 2024-04-25T11:10:16.720 INFO:teuthology.misc.health.smithi007.stdout: 2024-04-25T11:10:16.720 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":18,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:12.339298+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:16.722 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 -- 172.21.15.7:0/4240630313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f86f00744b0 msgr2=0x7f86f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:16.722 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f86f00744b0 0x7f86f00769c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f86fc034000 tx=0x7f86fc02e000 comp rx=0 tx=0).stop 2024-04-25T11:10:16.723 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 -- 172.21.15.7:0/4240630313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f871406ffa0 msgr2=0x7f87140c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:16.723 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f871406ffa0 0x7f87140c98a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f870400ba50 tx=0x7f870400bf10 comp rx=0 tx=0).stop 2024-04-25T11:10:16.723 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 -- 172.21.15.7:0/4240630313 shutdown_connections 2024-04-25T11:10:16.723 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f86f00744b0 0x7f86f00769c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:16.723 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87140c7c20 0x7f87140ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:16.723 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8714070ec0 0x7f87140c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:16.723 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 --2- 172.21.15.7:0/4240630313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f871406ffa0 0x7f87140c98a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:16.723 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 -- 172.21.15.7:0/4240630313 >> 172.21.15.7:0/4240630313 conn(0x7f87140ce240 msgr2=0x7f87140b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:16.723 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 -- 172.21.15.7:0/4240630313 shutdown_connections 2024-04-25T11:10:16.723 INFO:teuthology.misc.health.smithi007.stderr:2024-04-25T11:10:16.722+0000 7f8718c00640 1 -- 172.21.15.7:0/4240630313 wait complete. 2024-04-25T11:10:16.746 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T11:10:16.746 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T11:10:16.746 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:10:16.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:10:16.751 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:10:16.752 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:10:16.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:10:16.756 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:10:16.756 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:10:16.756 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:10:16.764 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:10:16.764 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:10:16.764 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:10:16.771 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:10:16.772 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T11:10:16.772 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:16.975 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:10:16.980 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:10:16.990 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:10:16.992 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:10:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.027+0000 7fb991800640 1 Processor -- start 2024-04-25T11:10:17.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.027+0000 7fb991800640 1 -- start start 2024-04-25T11:10:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.029+0000 7fb991800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb98c15de70 0x7fb98c164640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.029+0000 7fb991800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb98c0b6790 0x7fb98c0b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.029+0000 7fb991800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb98c0b0a50 0x7fb98c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.029+0000 7fb991800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb98c0cdb70 con 0x7fb98c15de70 2024-04-25T11:10:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.029+0000 7fb991800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb98c0aed30 con 0x7fb98c0b6790 2024-04-25T11:10:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.029+0000 7fb991800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c0aea00 con 0x7fb98c0b0a50 2024-04-25T11:10:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.029+0000 7fb98aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb98c15de70 0x7fb98c164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.029+0000 7fb98aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb98c15de70 0x7fb98c164640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54620/0 (socket says 172.21.15.7:54620) 2024-04-25T11:10:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.029+0000 7fb98aa00640 1 -- 172.21.15.7:0/1660481022 learned_addr learned my addr 172.21.15.7:0/1660481022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.029+0000 7fb983e00640 1 --2- 172.21.15.7:0/1660481022 >> v2:172.21.15.87:3300/0 conn(0x7fb98c0b6790 0x7fb98c0b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.029+0000 7fb98b400640 1 --2- 172.21.15.7:0/1660481022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb98c0b0a50 0x7fb98c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.030+0000 7fb98aa00640 1 -- 172.21.15.7:0/1660481022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb98c0b0a50 msgr2=0x7fb98c0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.030+0000 7fb98aa00640 1 --2- 172.21.15.7:0/1660481022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb98c0b0a50 0x7fb98c0b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.030+0000 7fb98aa00640 1 -- 172.21.15.7:0/1660481022 >> v2:172.21.15.87:3300/0 conn(0x7fb98c0b6790 msgr2=0x7fb98c0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.030+0000 7fb98aa00640 1 --2- 172.21.15.7:0/1660481022 >> v2:172.21.15.87:3300/0 conn(0x7fb98c0b6790 0x7fb98c0b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.030+0000 7fb98aa00640 1 -- 172.21.15.7:0/1660481022 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c0b13e0 con 0x7fb98c15de70 2024-04-25T11:10:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.031+0000 7fb98aa00640 1 --2- 172.21.15.7:0/1660481022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb98c15de70 0x7fb98c164640 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb98c071470 tx=0x7fb97c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c044d0becad2083 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.031+0000 7fb98a000640 1 -- 172.21.15.7:0/1660481022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c037070 con 0x7fb98c15de70 2024-04-25T11:10:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.031+0000 7fb98a000640 1 -- 172.21.15.7:0/1660481022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c032440 con 0x7fb98c15de70 2024-04-25T11:10:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.031+0000 7fb98a000640 1 -- 172.21.15.7:0/1660481022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c032810 con 0x7fb98c15de70 2024-04-25T11:10:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.032+0000 7fb991800640 1 -- 172.21.15.7:0/1660481022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb98c15de70 msgr2=0x7fb98c164640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.032+0000 7fb991800640 1 --2- 172.21.15.7:0/1660481022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb98c15de70 0x7fb98c164640 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb98c071470 tx=0x7fb97c02a140 comp rx=0 tx=0).stop 2024-04-25T11:10:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.032+0000 7fb991800640 1 -- 172.21.15.7:0/1660481022 shutdown_connections 2024-04-25T11:10:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.032+0000 7fb991800640 1 --2- 172.21.15.7:0/1660481022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb98c0b0a50 0x7fb98c0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.032+0000 7fb991800640 1 --2- 172.21.15.7:0/1660481022 >> v2:172.21.15.87:3300/0 conn(0x7fb98c0b6790 0x7fb98c0b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.032+0000 7fb991800640 1 --2- 172.21.15.7:0/1660481022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb98c15de70 0x7fb98c164640 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.032+0000 7fb991800640 1 -- 172.21.15.7:0/1660481022 >> 172.21.15.7:0/1660481022 conn(0x7fb98c074200 msgr2=0x7fb98c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.032+0000 7fb991800640 1 -- 172.21.15.7:0/1660481022 shutdown_connections 2024-04-25T11:10:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.032+0000 7fb991800640 1 -- 172.21.15.7:0/1660481022 wait complete. 2024-04-25T11:10:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.034+0000 7fb991800640 1 Processor -- start 2024-04-25T11:10:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.034+0000 7fb991800640 1 -- start start 2024-04-25T11:10:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.036+0000 7fb991800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb98c0b0a50 0x7fb98c13ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.036+0000 7fb991800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb98c0b6790 0x7fb98c0ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.036+0000 7fb991800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb98c15de70 0x7fb98c15ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.036+0000 7fb991800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c0b0e60 con 0x7fb98c0b0a50 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.036+0000 7fb991800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c0ce0a0 con 0x7fb98c15de70 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.036+0000 7fb991800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c06f300 con 0x7fb98c0b6790 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.036+0000 7fb98aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb98c0b0a50 0x7fb98c13ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.036+0000 7fb98aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb98c0b0a50 0x7fb98c13ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54630/0 (socket says 172.21.15.7:54630) 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.036+0000 7fb98aa00640 1 -- 172.21.15.7:0/875172741 learned_addr learned my addr 172.21.15.7:0/875172741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb98b400640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb98c15de70 0x7fb98c15ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb983e00640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb98c0b6790 0x7fb98c0ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb98aa00640 1 -- 172.21.15.7:0/875172741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb98c0b6790 msgr2=0x7fb98c0ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb98aa00640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb98c0b6790 0x7fb98c0ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb98aa00640 1 -- 172.21.15.7:0/875172741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb98c15de70 msgr2=0x7fb98c15ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb98aa00640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb98c15de70 0x7fb98c15ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb98aa00640 1 -- 172.21.15.7:0/875172741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c0d0af0 con 0x7fb98c0b0a50 2024-04-25T11:10:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb98b400640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb98c15de70 0x7fb98c15ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb98aa00640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb98c0b0a50 0x7fb98c13ea70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb97c000c00 tx=0x7fb97c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb982a00640 1 -- 172.21.15.7:0/875172741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c041020 con 0x7fb98c0b0a50 2024-04-25T11:10:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb982a00640 1 -- 172.21.15.7:0/875172741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c03d620 con 0x7fb98c0b0a50 2024-04-25T11:10:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb982a00640 1 -- 172.21.15.7:0/875172741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c037040 con 0x7fb98c0b0a50 2024-04-25T11:10:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb991800640 1 -- 172.21.15.7:0/875172741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c0d0dd0 con 0x7fb98c0b0a50 2024-04-25T11:10:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.037+0000 7fb991800640 1 -- 172.21.15.7:0/875172741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c13f130 con 0x7fb98c0b0a50 2024-04-25T11:10:17.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.039+0000 7fb991800640 1 -- 172.21.15.7:0/875172741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c06a100 con 0x7fb98c0b0a50 2024-04-25T11:10:17.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.040+0000 7fb982a00640 1 -- 172.21.15.7:0/875172741 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb97c044020 con 0x7fb98c0b0a50 2024-04-25T11:10:17.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.041+0000 7fb982a00640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9680744e0 0x7fb9680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.041+0000 7fb982a00640 1 -- 172.21.15.7:0/875172741 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fb97c0b55e0 con 0x7fb98c0b0a50 2024-04-25T11:10:17.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.041+0000 7fb983e00640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9680744e0 0x7fb9680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.042+0000 7fb983e00640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9680744e0 0x7fb9680769f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb98c15fa20 tx=0x7fb974008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:17.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.046+0000 7fb982a00640 1 -- 172.21.15.7:0/875172741 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb97c04b030 con 0x7fb98c0b0a50 2024-04-25T11:10:17.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.249+0000 7fb991800640 1 -- 172.21.15.7:0/875172741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb98c0b5b20 con 0x7fb98c0b0a50 2024-04-25T11:10:17.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.252+0000 7fb982a00640 1 -- 172.21.15.7:0/875172741 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12788 (secure 0 0 0) 0x7fb97c0800b0 con 0x7fb98c0b0a50 2024-04-25T11:10:17.252 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:17.252 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:12.339298+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.254+0000 7fb991800640 1 -- 172.21.15.7:0/875172741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9680744e0 msgr2=0x7fb9680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.254+0000 7fb991800640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9680744e0 0x7fb9680769f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb98c15fa20 tx=0x7fb974008040 comp rx=0 tx=0).stop 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.254+0000 7fb991800640 1 -- 172.21.15.7:0/875172741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb98c0b0a50 msgr2=0x7fb98c13ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.254+0000 7fb991800640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb98c0b0a50 0x7fb98c13ea70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb97c000c00 tx=0x7fb97c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.254+0000 7fb991800640 1 -- 172.21.15.7:0/875172741 shutdown_connections 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.254+0000 7fb991800640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9680744e0 0x7fb9680769f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.254+0000 7fb991800640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb98c15de70 0x7fb98c15ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.254+0000 7fb991800640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb98c0b6790 0x7fb98c0ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.254+0000 7fb991800640 1 --2- 172.21.15.7:0/875172741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb98c0b0a50 0x7fb98c13ea70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.254+0000 7fb991800640 1 -- 172.21.15.7:0/875172741 >> 172.21.15.7:0/875172741 conn(0x7fb98c074200 msgr2=0x7fb98c15fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.255+0000 7fb991800640 1 -- 172.21.15.7:0/875172741 shutdown_connections 2024-04-25T11:10:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.255+0000 7fb991800640 1 -- 172.21.15.7:0/875172741 wait complete. 2024-04-25T11:10:17.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:10:17.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:10:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.559+0000 7fdf31800640 1 Processor -- start 2024-04-25T11:10:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.560+0000 7fdf31800640 1 -- start start 2024-04-25T11:10:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.562+0000 7fdf31800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c1583c0 0x7fdf2c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.563+0000 7fdf31800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdf2c06eea0 0x7fdf2c06f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.563+0000 7fdf31800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf2c0ce570 con 0x7fdf2c0cfb50 2024-04-25T11:10:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.563+0000 7fdf31800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fdf2c06f7f0 con 0x7fdf2c06eea0 2024-04-25T11:10:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.563+0000 7fdf31800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2c15f190 con 0x7fdf2c1583c0 2024-04-25T11:10:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.563+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c1583c0 0x7fdf2c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.563+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c1583c0 0x7fdf2c0cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:42822/0 (socket says 172.21.15.7:42822) 2024-04-25T11:10:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.563+0000 7fdf2be00640 1 -- 172.21.15.7:0/3132957419 learned_addr learned my addr 172.21.15.7:0/3132957419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.563+0000 7fdf30e00640 1 --2- 172.21.15.7:0/3132957419 >> v2:172.21.15.87:3300/0 conn(0x7fdf2c06eea0 0x7fdf2c06f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.564+0000 7fdf2be00640 1 -- 172.21.15.7:0/3132957419 >> v2:172.21.15.87:3300/0 conn(0x7fdf2c06eea0 msgr2=0x7fdf2c06f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.564+0000 7fdf2be00640 1 --2- 172.21.15.7:0/3132957419 >> v2:172.21.15.87:3300/0 conn(0x7fdf2c06eea0 0x7fdf2c06f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.564+0000 7fdf2be00640 1 -- 172.21.15.7:0/3132957419 >> v1:172.21.15.87:6789/0 conn(0x7fdf2c0cfb50 legacy=0x7fdf2c0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.564+0000 7fdf2be00640 1 -- 172.21.15.7:0/3132957419 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c0ce850 con 0x7fdf2c1583c0 2024-04-25T11:10:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.564+0000 7fdf30e00640 1 --2- 172.21.15.7:0/3132957419 >> v2:172.21.15.87:3300/0 conn(0x7fdf2c06eea0 0x7fdf2c06f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.565+0000 7fdf2be00640 1 --2- 172.21.15.7:0/3132957419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c1583c0 0x7fdf2c0cf610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009870 tx=0x7fdf1c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8430bd0326d17f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.566+0000 7fdf2aa00640 1 -- 172.21.15.7:0/3132957419 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c037070 con 0x7fdf2c1583c0 2024-04-25T11:10:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.566+0000 7fdf2aa00640 1 -- 172.21.15.7:0/3132957419 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c032440 con 0x7fdf2c1583c0 2024-04-25T11:10:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.566+0000 7fdf2aa00640 1 -- 172.21.15.7:0/3132957419 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c032810 con 0x7fdf2c1583c0 2024-04-25T11:10:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.566+0000 7fdf31800640 1 -- 172.21.15.7:0/3132957419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c1583c0 msgr2=0x7fdf2c0cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.566+0000 7fdf31800640 1 --2- 172.21.15.7:0/3132957419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c1583c0 0x7fdf2c0cf610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009870 tx=0x7fdf1c02a250 comp rx=0 tx=0).stop 2024-04-25T11:10:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.567+0000 7fdf31800640 1 -- 172.21.15.7:0/3132957419 shutdown_connections 2024-04-25T11:10:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.567+0000 7fdf31800640 1 --2- 172.21.15.7:0/3132957419 >> v2:172.21.15.87:3300/0 conn(0x7fdf2c06eea0 0x7fdf2c06f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.567+0000 7fdf31800640 1 --2- 172.21.15.7:0/3132957419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c1583c0 0x7fdf2c0cf610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.567+0000 7fdf31800640 1 -- 172.21.15.7:0/3132957419 >> 172.21.15.7:0/3132957419 conn(0x7fdf2c0740f0 msgr2=0x7fdf2c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.567+0000 7fdf31800640 1 -- 172.21.15.7:0/3132957419 shutdown_connections 2024-04-25T11:10:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.567+0000 7fdf31800640 1 -- 172.21.15.7:0/3132957419 wait complete. 2024-04-25T11:10:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.569+0000 7fdf31800640 1 Processor -- start 2024-04-25T11:10:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.569+0000 7fdf31800640 1 -- start start 2024-04-25T11:10:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.571+0000 7fdf31800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c06eea0 0x7fdf2c13dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.571+0000 7fdf31800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf2c0cfb50 0x7fdf2c158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.571+0000 7fdf31800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c1583c0 0x7fdf2c159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.571+0000 7fdf31800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c0cf0f0 con 0x7fdf2c1583c0 2024-04-25T11:10:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.571+0000 7fdf31800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c0cee10 con 0x7fdf2c0cfb50 2024-04-25T11:10:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.571+0000 7fdf31800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2c0d0b90 con 0x7fdf2c06eea0 2024-04-25T11:10:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.571+0000 7fdf30e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c1583c0 0x7fdf2c159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.571+0000 7fdf30e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c1583c0 0x7fdf2c159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54638/0 (socket says 172.21.15.7:54638) 2024-04-25T11:10:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.571+0000 7fdf30e00640 1 -- 172.21.15.7:0/1492836513 learned_addr learned my addr 172.21.15.7:0/1492836513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.571+0000 7fdf2be00640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c06eea0 0x7fdf2c13dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.571+0000 7fdf2b400640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf2c0cfb50 0x7fdf2c158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.572+0000 7fdf30e00640 1 -- 172.21.15.7:0/1492836513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c06eea0 msgr2=0x7fdf2c13dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.572+0000 7fdf30e00640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c06eea0 0x7fdf2c13dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.572+0000 7fdf30e00640 1 -- 172.21.15.7:0/1492836513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf2c0cfb50 msgr2=0x7fdf2c158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:17.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.572+0000 7fdf30e00640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf2c0cfb50 0x7fdf2c158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.572+0000 7fdf30e00640 1 -- 172.21.15.7:0/1492836513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c13e210 con 0x7fdf2c1583c0 2024-04-25T11:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.572+0000 7fdf2b400640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf2c0cfb50 0x7fdf2c158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.572+0000 7fdf30e00640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c1583c0 0x7fdf2c159080 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf24007e20 tx=0x7fdf2400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.572+0000 7fdf23e00640 1 -- 172.21.15.7:0/1492836513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2401a070 con 0x7fdf2c1583c0 2024-04-25T11:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.572+0000 7fdf23e00640 1 -- 172.21.15.7:0/1492836513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf24003b90 con 0x7fdf2c1583c0 2024-04-25T11:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.572+0000 7fdf23e00640 1 -- 172.21.15.7:0/1492836513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400de80 con 0x7fdf2c1583c0 2024-04-25T11:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.572+0000 7fdf31800640 1 -- 172.21.15.7:0/1492836513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2c1598c0 con 0x7fdf2c1583c0 2024-04-25T11:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.573+0000 7fdf31800640 1 -- 172.21.15.7:0/1492836513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2c159e80 con 0x7fdf2c1583c0 2024-04-25T11:10:17.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.574+0000 7fdf31800640 1 -- 172.21.15.7:0/1492836513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf2c159630 con 0x7fdf2c1583c0 2024-04-25T11:10:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.575+0000 7fdf23e00640 1 -- 172.21.15.7:0/1492836513 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf2401d070 con 0x7fdf2c1583c0 2024-04-25T11:10:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.576+0000 7fdf23e00640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf0c0744b0 0x7fdf0c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.576+0000 7fdf23e00640 1 -- 172.21.15.7:0/1492836513 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fdf24095050 con 0x7fdf2c1583c0 2024-04-25T11:10:17.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.576+0000 7fdf2be00640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf0c0744b0 0x7fdf0c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:17.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.577+0000 7fdf2be00640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf0c0744b0 0x7fdf0c0769c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c034000 tx=0x7fdf1c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.581+0000 7fdf23e00640 1 -- 172.21.15.7:0/1492836513 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf24010070 con 0x7fdf2c1583c0 2024-04-25T11:10:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.786+0000 7fdf31800640 1 -- 172.21.15.7:0/1492836513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf2c050740 con 0x7fdf2c1583c0 2024-04-25T11:10:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.787+0000 7fdf23e00640 1 -- 172.21.15.7:0/1492836513 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (secure 0 0 0) 0x7fdf2405f4e0 con 0x7fdf2c1583c0 2024-04-25T11:10:17.787 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:10:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 -- 172.21.15.7:0/1492836513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf0c0744b0 msgr2=0x7fdf0c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf0c0744b0 0x7fdf0c0769c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c034000 tx=0x7fdf1c02e000 comp rx=0 tx=0).stop 2024-04-25T11:10:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 -- 172.21.15.7:0/1492836513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c1583c0 msgr2=0x7fdf2c159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c1583c0 0x7fdf2c159080 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf24007e20 tx=0x7fdf2400d4a0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 -- 172.21.15.7:0/1492836513 shutdown_connections 2024-04-25T11:10:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf0c0744b0 0x7fdf0c0769c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf2c1583c0 0x7fdf2c159080 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf2c0cfb50 0x7fdf2c158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 --2- 172.21.15.7:0/1492836513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf2c06eea0 0x7fdf2c13dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 -- 172.21.15.7:0/1492836513 >> 172.21.15.7:0/1492836513 conn(0x7fdf2c0740f0 msgr2=0x7fdf2c15e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 -- 172.21.15.7:0/1492836513 shutdown_connections 2024-04-25T11:10:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:17.791+0000 7fdf31800640 1 -- 172.21.15.7:0/1492836513 wait complete. 2024-04-25T11:10:17.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:10:18.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.085+0000 7fc1a1600640 1 Processor -- start 2024-04-25T11:10:18.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.085+0000 7fc1a1600640 1 -- start start 2024-04-25T11:10:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.088+0000 7fc1a1600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc19c0b6790 0x7fc19c164640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.088+0000 7fc1a1600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc19c0b0c20 0x7fc19c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.088+0000 7fc1a1600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc19c0aea00 con 0x7fc19c15de70 2024-04-25T11:10:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.088+0000 7fc1a1600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc19c0cdb70 con 0x7fc19c0b0c20 2024-04-25T11:10:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.088+0000 7fc1a1600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc19c0aed30 con 0x7fc19c0b6790 2024-04-25T11:10:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.088+0000 7fc19aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc19c15de70 0x7fc19c0b0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55528/0 (socket says 172.21.15.7:55528) 2024-04-25T11:10:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.088+0000 7fc19aa00640 1 -- 172.21.15.7:0/3295911472 learned_addr learned my addr 172.21.15.7:0/3295911472 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.088+0000 7fc19b400640 1 --2- 172.21.15.7:0/3295911472 >> v2:172.21.15.7:3300/0 conn(0x7fc19c0b0c20 0x7fc19c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.088+0000 7fc19a000640 1 --2- 172.21.15.7:0/3295911472 >> v2:172.21.15.87:3300/0 conn(0x7fc19c0b6790 0x7fc19c164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.088+0000 7fc199600640 1 -- 172.21.15.7:0/3295911472 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965428030 0 0) 0x7fc19c0aea00 con 0x7fc19c15de70 2024-04-25T11:10:18.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.089+0000 7fc199600640 1 -- 172.21.15.7:0/3295911472 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc180003520 con 0x7fc19c15de70 2024-04-25T11:10:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.089+0000 7fc19b400640 1 -- 172.21.15.7:0/3295911472 >> v2:172.21.15.87:3300/0 conn(0x7fc19c0b6790 msgr2=0x7fc19c164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.089+0000 7fc19b400640 1 --2- 172.21.15.7:0/3295911472 >> v2:172.21.15.87:3300/0 conn(0x7fc19c0b6790 0x7fc19c164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.089+0000 7fc19b400640 1 -- 172.21.15.7:0/3295911472 >> v1:172.21.15.7:6789/0 conn(0x7fc19c15de70 legacy=0x7fc19c0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.089+0000 7fc19b400640 1 -- 172.21.15.7:0/3295911472 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c0b13e0 con 0x7fc19c0b0c20 2024-04-25T11:10:18.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.090+0000 7fc19b400640 1 --2- 172.21.15.7:0/3295911472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19c0b0c20 0x7fc19c1606a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0cdcf0 tx=0x7fc19402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2cf9aee8c88e64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:18.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.090+0000 7fc199600640 1 -- 172.21.15.7:0/3295911472 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194031600 con 0x7fc19c0b0c20 2024-04-25T11:10:18.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.090+0000 7fc199600640 1 -- 172.21.15.7:0/3295911472 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1940317a0 con 0x7fc19c0b0c20 2024-04-25T11:10:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.090+0000 7fc199600640 1 -- 172.21.15.7:0/3295911472 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194031b70 con 0x7fc19c0b0c20 2024-04-25T11:10:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.091+0000 7fc1a1600640 1 -- 172.21.15.7:0/3295911472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19c0b0c20 msgr2=0x7fc19c1606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.091+0000 7fc1a1600640 1 --2- 172.21.15.7:0/3295911472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19c0b0c20 0x7fc19c1606a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0cdcf0 tx=0x7fc19402a5d0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.091+0000 7fc1a1600640 1 -- 172.21.15.7:0/3295911472 shutdown_connections 2024-04-25T11:10:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.091+0000 7fc1a1600640 1 --2- 172.21.15.7:0/3295911472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19c0b0c20 0x7fc19c1606a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.091+0000 7fc1a1600640 1 --2- 172.21.15.7:0/3295911472 >> v2:172.21.15.87:3300/0 conn(0x7fc19c0b6790 0x7fc19c164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.091+0000 7fc1a1600640 1 -- 172.21.15.7:0/3295911472 >> 172.21.15.7:0/3295911472 conn(0x7fc19c074200 msgr2=0x7fc19c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.091+0000 7fc1a1600640 1 -- 172.21.15.7:0/3295911472 shutdown_connections 2024-04-25T11:10:18.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.091+0000 7fc1a1600640 1 -- 172.21.15.7:0/3295911472 wait complete. 2024-04-25T11:10:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.093+0000 7fc1a1600640 1 Processor -- start 2024-04-25T11:10:18.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.094+0000 7fc1a1600640 1 -- start start 2024-04-25T11:10:18.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.095+0000 7fc1a1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19c0b0c20 0x7fc19c0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.095+0000 7fc1a1600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc19c0b6790 0x7fc19c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.095+0000 7fc1a1600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc19c15de70 0x7fc19c147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.095+0000 7fc1a1600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c06f300 con 0x7fc19c0b0c20 2024-04-25T11:10:18.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.095+0000 7fc1a1600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c0ce0a0 con 0x7fc19c15de70 2024-04-25T11:10:18.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.095+0000 7fc1a1600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc19c06f480 con 0x7fc19c0b6790 2024-04-25T11:10:18.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.095+0000 7fc19aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19c0b0c20 0x7fc19c0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.095+0000 7fc19aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19c0b0c20 0x7fc19c0cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54664/0 (socket says 172.21.15.7:54664) 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.096+0000 7fc19aa00640 1 -- 172.21.15.7:0/1512784007 learned_addr learned my addr 172.21.15.7:0/1512784007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.096+0000 7fc19aa00640 1 -- 172.21.15.7:0/1512784007 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc19c0b6790 msgr2=0x7fc19c15cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.096+0000 7fc19a000640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc19c0b6790 0x7fc19c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.096+0000 7fc19aa00640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc19c0b6790 0x7fc19c15cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.096+0000 7fc19b400640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc19c15de70 0x7fc19c147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.096+0000 7fc19aa00640 1 -- 172.21.15.7:0/1512784007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc19c15de70 msgr2=0x7fc19c147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.096+0000 7fc19aa00640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc19c15de70 0x7fc19c147a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.096+0000 7fc19aa00640 1 -- 172.21.15.7:0/1512784007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c0d0650 con 0x7fc19c0b0c20 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.096+0000 7fc19aa00640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19c0b0c20 0x7fc19c0cfcd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00c390 tx=0x7fc18c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.097+0000 7fc192a00640 1 -- 172.21.15.7:0/1512784007 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c017070 con 0x7fc19c0b0c20 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.097+0000 7fc192a00640 1 -- 172.21.15.7:0/1512784007 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18c005d00 con 0x7fc19c0b0c20 2024-04-25T11:10:18.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.097+0000 7fc192a00640 1 -- 172.21.15.7:0/1512784007 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c007310 con 0x7fc19c0b0c20 2024-04-25T11:10:18.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.097+0000 7fc1a1600640 1 -- 172.21.15.7:0/1512784007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c0d0930 con 0x7fc19c0b0c20 2024-04-25T11:10:18.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.097+0000 7fc1a1600640 1 -- 172.21.15.7:0/1512784007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c145a00 con 0x7fc19c0b0c20 2024-04-25T11:10:18.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.098+0000 7fc1a1600640 1 -- 172.21.15.7:0/1512784007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19c06a100 con 0x7fc19c0b0c20 2024-04-25T11:10:18.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.099+0000 7fc192a00640 1 -- 172.21.15.7:0/1512784007 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc18c01a070 con 0x7fc19c0b0c20 2024-04-25T11:10:18.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.100+0000 7fc192a00640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc17c0744b0 0x7fc17c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.100+0000 7fc192a00640 1 -- 172.21.15.7:0/1512784007 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fc18c094200 con 0x7fc19c0b0c20 2024-04-25T11:10:18.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.100+0000 7fc19a000640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc17c0744b0 0x7fc17c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.101+0000 7fc19a000640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc17c0744b0 0x7fc17c0769c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc19c161b90 tx=0x7fc188008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:18.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.105+0000 7fc192a00640 1 -- 172.21.15.7:0/1512784007 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc18c05ed60 con 0x7fc19c0b0c20 2024-04-25T11:10:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.347+0000 7fc1a1600640 1 -- 172.21.15.7:0/1512784007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc19c0b1a30 con 0x7fc19c0b0c20 2024-04-25T11:10:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.348+0000 7fc192a00640 1 -- 172.21.15.7:0/1512784007 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3855 (secure 0 0 0) 0x7fc18c05e660 con 0x7fc19c0b0c20 2024-04-25T11:10:18.348 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3458223722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3458223722},{"type":"v1","addr":"172.21.15.87:6835","nonce":3458223722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1383067804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1383067804},{"type":"v1","addr":"172.21.15.7:6835","nonce":1383067804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/435824250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":435824250},{"type":"v1","addr":"172.21.15.7:6837","nonce":435824250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/1091275258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1091275258},{"type":"v1","addr":"172.21.15.87:6834","nonce":1091275258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T11:10:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.350+0000 7fc1a1600640 1 -- 172.21.15.7:0/1512784007 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc17c0744b0 msgr2=0x7fc17c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.350+0000 7fc1a1600640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc17c0744b0 0x7fc17c0769c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc19c161b90 tx=0x7fc188008040 comp rx=0 tx=0).stop 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.350+0000 7fc1a1600640 1 -- 172.21.15.7:0/1512784007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19c0b0c20 msgr2=0x7fc19c0cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.350+0000 7fc1a1600640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19c0b0c20 0x7fc19c0cfcd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00c390 tx=0x7fc18c00c850 comp rx=0 tx=0).stop 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.350+0000 7fc1a1600640 1 -- 172.21.15.7:0/1512784007 shutdown_connections 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.350+0000 7fc1a1600640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc17c0744b0 0x7fc17c0769c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.350+0000 7fc1a1600640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc19c15de70 0x7fc19c147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.350+0000 7fc1a1600640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc19c0b6790 0x7fc19c15cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.350+0000 7fc1a1600640 1 --2- 172.21.15.7:0/1512784007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19c0b0c20 0x7fc19c0cfcd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.350+0000 7fc1a1600640 1 -- 172.21.15.7:0/1512784007 >> 172.21.15.7:0/1512784007 conn(0x7fc19c074200 msgr2=0x7fc19c0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.351+0000 7fc1a1600640 1 -- 172.21.15.7:0/1512784007 shutdown_connections 2024-04-25T11:10:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.351+0000 7fc1a1600640 1 -- 172.21.15.7:0/1512784007 wait complete. 2024-04-25T11:10:18.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.659+0000 7f8dd2000640 1 Processor -- start 2024-04-25T11:10:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.659+0000 7f8dd2000640 1 -- start start 2024-04-25T11:10:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.661+0000 7f8dd2000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8dcc06cc80 0x7f8dcc0bae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.661+0000 7f8dd2000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dcc164120 con 0x7f8dcc163bb0 2024-04-25T11:10:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.661+0000 7f8dd2000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dcc164400 con 0x7f8dcc0d03a0 2024-04-25T11:10:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.661+0000 7f8dd2000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8dcc0b51c0 con 0x7f8dcc06cc80 2024-04-25T11:10:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.662+0000 7f8dd0c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8dcc163bb0 0x7f8dcc0b8d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55536/0 (socket says 172.21.15.7:55536) 2024-04-25T11:10:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.662+0000 7f8dd0c00640 1 -- 172.21.15.7:0/2349840201 learned_addr learned my addr 172.21.15.7:0/2349840201 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.662+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3499646957 0 0) 0x7f8dcc164120 con 0x7f8dcc163bb0 2024-04-25T11:10:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.662+0000 7f8dd1600640 1 --2- 172.21.15.7:0/2349840201 >> v2:172.21.15.87:3300/0 conn(0x7f8dcc06cc80 0x7f8dcc0bae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.662+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8db0003520 con 0x7f8dcc163bb0 2024-04-25T11:10:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1375053336 0 0) 0x7f8db0003520 con 0x7f8dcc163bb0 2024-04-25T11:10:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 >> v2:172.21.15.87:3300/0 conn(0x7f8dcc06cc80 msgr2=0x7f8dcc0bae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 --2- 172.21.15.7:0/2349840201 >> v2:172.21.15.87:3300/0 conn(0x7f8dcc06cc80 0x7f8dcc0bae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 >> v1:172.21.15.87:6789/0 conn(0x7f8dcc0d03a0 legacy=0x7f8dcc06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc1646e0 con 0x7f8dcc163bb0 2024-04-25T11:10:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f8dbc001670 con 0x7f8dcc163bb0 2024-04-25T11:10:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 >> v1:172.21.15.7:6789/0 conn(0x7f8dcc163bb0 legacy=0x7f8dcc0b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 --2- 172.21.15.7:0/2349840201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db00039b0 0x7f8db0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 --2- 172.21.15.7:0/2349840201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db0004800 0x7f8db0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 --2- 172.21.15.7:0/2349840201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8db00051c0 0x7f8db000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc0b51c0 con 0x7f8db0004800 2024-04-25T11:10:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8db000c160 con 0x7f8db00039b0 2024-04-25T11:10:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8db000c360 con 0x7f8db00051c0 2024-04-25T11:10:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcbe00640 1 --2- 172.21.15.7:0/2349840201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db0004800 0x7f8db0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcbe00640 1 -- 172.21.15.7:0/2349840201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8db00051c0 msgr2=0x7f8db000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcbe00640 1 --2- 172.21.15.7:0/2349840201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8db00051c0 0x7f8db000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcbe00640 1 -- 172.21.15.7:0/2349840201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db00039b0 msgr2=0x7f8db0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.663+0000 7f8dcbe00640 1 --2- 172.21.15.7:0/2349840201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8db00039b0 0x7f8db0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.664+0000 7f8dcbe00640 1 -- 172.21.15.7:0/2349840201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db0003520 con 0x7f8db0004800 2024-04-25T11:10:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.664+0000 7f8dd1600640 1 -- 172.21.15.7:0/2349840201 reap_dead start 2024-04-25T11:10:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.665+0000 7f8dcbe00640 1 --2- 172.21.15.7:0/2349840201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db0004800 0x7f8db0004c10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8db4001750 tx=0x7f8db0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc85db40fe1a3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.665+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4001370 con 0x7f8db0004800 2024-04-25T11:10:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.665+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db40052d0 con 0x7f8db0004800 2024-04-25T11:10:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.665+0000 7f8dcb400640 1 -- 172.21.15.7:0/2349840201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db402abc0 con 0x7f8db0004800 2024-04-25T11:10:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.665+0000 7f8dd2000640 1 -- 172.21.15.7:0/2349840201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db0004800 msgr2=0x7f8db0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.665+0000 7f8dd2000640 1 --2- 172.21.15.7:0/2349840201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db0004800 0x7f8db0004c10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8db4001750 tx=0x7f8db0005830 comp rx=0 tx=0).stop 2024-04-25T11:10:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.666+0000 7f8dd2000640 1 -- 172.21.15.7:0/2349840201 shutdown_connections 2024-04-25T11:10:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.666+0000 7f8dd2000640 1 --2- 172.21.15.7:0/2349840201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8db0004800 0x7f8db0004c10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.666+0000 7f8dd2000640 1 -- 172.21.15.7:0/2349840201 >> 172.21.15.7:0/2349840201 conn(0x7f8dcc06eb00 msgr2=0x7f8dcc0d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.666+0000 7f8dd2000640 1 -- 172.21.15.7:0/2349840201 shutdown_connections 2024-04-25T11:10:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.666+0000 7f8dd2000640 1 -- 172.21.15.7:0/2349840201 wait complete. 2024-04-25T11:10:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.668+0000 7f8dd2000640 1 Processor -- start 2024-04-25T11:10:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.668+0000 7f8dd2000640 1 -- start start 2024-04-25T11:10:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.670+0000 7f8dd2000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8db0004800 0x7f8dcc15c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.670+0000 7f8dd2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dcc15ebb0 0x7f8dcc15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.670+0000 7f8dd2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8dcc1618e0 0x7f8dcc161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.670+0000 7f8dd2000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc0d07b0 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.670+0000 7f8dd2000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc0d4170 con 0x7f8dcc1618e0 2024-04-25T11:10:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.670+0000 7f8dd2000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8dcc0bae90 con 0x7f8db0004800 2024-04-25T11:10:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.670+0000 7f8dcbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dcc15ebb0 0x7f8dcc15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.670+0000 7f8dcbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dcc15ebb0 0x7f8dcc15a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54690/0 (socket says 172.21.15.7:54690) 2024-04-25T11:10:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.670+0000 7f8dcbe00640 1 -- 172.21.15.7:0/1920300033 learned_addr learned my addr 172.21.15.7:0/1920300033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.670+0000 7f8dd0c00640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8db0004800 0x7f8dcc15c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.670+0000 7f8dd1600640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8dcc1618e0 0x7f8dcc161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.671+0000 7f8dcbe00640 1 -- 172.21.15.7:0/1920300033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8db0004800 msgr2=0x7f8dcc15c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.671+0000 7f8dcbe00640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8db0004800 0x7f8dcc15c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.671+0000 7f8dcbe00640 1 -- 172.21.15.7:0/1920300033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8dcc1618e0 msgr2=0x7f8dcc161cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.671+0000 7f8dcbe00640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8dcc1618e0 0x7f8dcc161cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.671+0000 7f8dcbe00640 1 -- 172.21.15.7:0/1920300033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc162230 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.671+0000 7f8dcbe00640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dcc15ebb0 0x7f8dcc15a980 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8db4000c00 tx=0x7f8db4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.671+0000 7f8dc8c00640 1 -- 172.21.15.7:0/1920300033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4037040 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.671+0000 7f8dc8c00640 1 -- 172.21.15.7:0/1920300033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db4005480 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.671+0000 7f8dc8c00640 1 -- 172.21.15.7:0/1920300033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4001370 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.671+0000 7f8dd2000640 1 -- 172.21.15.7:0/1920300033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dcc15cf10 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.672+0000 7f8dd2000640 1 -- 172.21.15.7:0/1920300033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dcc15d4d0 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.673+0000 7f8dd2000640 1 -- 172.21.15.7:0/1920300033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dcc050740 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.674+0000 7f8dc8c00640 1 -- 172.21.15.7:0/1920300033 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8db4043020 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.675+0000 7f8dc8c00640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8da80744e0 0x7f8da80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.675+0000 7f8dc8c00640 1 -- 172.21.15.7:0/1920300033 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f8db40b46b0 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.675+0000 7f8dd0c00640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8da80744e0 0x7f8da80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.676+0000 7f8dd0c00640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8da80744e0 0x7f8da80769f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0011e0 tx=0x7f8dbc0015b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:18.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.680+0000 7f8dc8c00640 1 -- 172.21.15.7:0/1920300033 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8dcc162230 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.878+0000 7f8dd2000640 1 -- 172.21.15.7:0/1920300033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8dcc15cc60 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.880+0000 7f8dc8c00640 1 -- 172.21.15.7:0/1920300033 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12788 (secure 0 0 0) 0x7f8db407f0f0 con 0x7f8dcc15ebb0 2024-04-25T11:10:18.880 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:18.880 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:12.339298+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:18.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.882+0000 7f8dd2000640 1 -- 172.21.15.7:0/1920300033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8da80744e0 msgr2=0x7f8da80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.882+0000 7f8dd2000640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8da80744e0 0x7f8da80769f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0011e0 tx=0x7f8dbc0015b0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.882+0000 7f8dd2000640 1 -- 172.21.15.7:0/1920300033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dcc15ebb0 msgr2=0x7f8dcc15a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:18.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.882+0000 7f8dd2000640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dcc15ebb0 0x7f8dcc15a980 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8db4000c00 tx=0x7f8db4001120 comp rx=0 tx=0).stop 2024-04-25T11:10:18.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.883+0000 7f8dd2000640 1 -- 172.21.15.7:0/1920300033 shutdown_connections 2024-04-25T11:10:18.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.883+0000 7f8dd2000640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8da80744e0 0x7f8da80769f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.883+0000 7f8dd2000640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8dcc1618e0 0x7f8dcc161cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.883+0000 7f8dd2000640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dcc15ebb0 0x7f8dcc15a980 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.883+0000 7f8dd2000640 1 --2- 172.21.15.7:0/1920300033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8db0004800 0x7f8dcc15c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:18.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.883+0000 7f8dd2000640 1 -- 172.21.15.7:0/1920300033 >> 172.21.15.7:0/1920300033 conn(0x7f8dcc06eb00 msgr2=0x7f8dcc0b5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:18.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.883+0000 7f8dd2000640 1 -- 172.21.15.7:0/1920300033 shutdown_connections 2024-04-25T11:10:18.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:18.883+0000 7f8dd2000640 1 -- 172.21.15.7:0/1920300033 wait complete. 2024-04-25T11:10:18.908 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:10:18.908 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:10:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.189+0000 7fc861600640 1 Processor -- start 2024-04-25T11:10:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.190+0000 7fc861600640 1 -- start start 2024-04-25T11:10:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.192+0000 7fc861600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc85c0cff00 0x7fc85c0d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.192+0000 7fc861600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc85c06f330 0x7fc85c06f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.192+0000 7fc861600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc85c0d09e0 con 0x7fc85c167de0 2024-04-25T11:10:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.192+0000 7fc861600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc85c06eea0 con 0x7fc85c06f330 2024-04-25T11:10:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.192+0000 7fc861600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc85c0d0cc0 con 0x7fc85c0cff00 2024-04-25T11:10:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.193+0000 7fc860c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc85c06f330 0x7fc85c06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.193+0000 7fc860c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc85c06f330 0x7fc85c06f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54702/0 (socket says 172.21.15.7:54702) 2024-04-25T11:10:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.193+0000 7fc860c00640 1 -- 172.21.15.7:0/972168819 learned_addr learned my addr 172.21.15.7:0/972168819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.193+0000 7fc85be00640 1 --2- 172.21.15.7:0/972168819 >> v2:172.21.15.87:3300/0 conn(0x7fc85c0cff00 0x7fc85c0d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.193+0000 7fc860c00640 1 -- 172.21.15.7:0/972168819 >> v2:172.21.15.87:3300/0 conn(0x7fc85c0cff00 msgr2=0x7fc85c0d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.193+0000 7fc860c00640 1 --2- 172.21.15.7:0/972168819 >> v2:172.21.15.87:3300/0 conn(0x7fc85c0cff00 0x7fc85c0d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.193+0000 7fc860c00640 1 -- 172.21.15.7:0/972168819 >> v1:172.21.15.87:6789/0 conn(0x7fc85c167de0 legacy=0x7fc85c1681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.193+0000 7fc860c00640 1 -- 172.21.15.7:0/972168819 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c0d0fa0 con 0x7fc85c06f330 2024-04-25T11:10:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.194+0000 7fc860c00640 1 --2- 172.21.15.7:0/972168819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85c06f330 0x7fc85c06f740 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc85c072330 tx=0x7fc85402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e88fa000b9b395 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.195+0000 7fc85aa00640 1 -- 172.21.15.7:0/972168819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854037070 con 0x7fc85c06f330 2024-04-25T11:10:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.195+0000 7fc85aa00640 1 -- 172.21.15.7:0/972168819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc854032440 con 0x7fc85c06f330 2024-04-25T11:10:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.195+0000 7fc85aa00640 1 -- 172.21.15.7:0/972168819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854032810 con 0x7fc85c06f330 2024-04-25T11:10:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.195+0000 7fc861600640 1 -- 172.21.15.7:0/972168819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85c06f330 msgr2=0x7fc85c06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.195+0000 7fc861600640 1 --2- 172.21.15.7:0/972168819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85c06f330 0x7fc85c06f740 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc85c072330 tx=0x7fc85402a270 comp rx=0 tx=0).stop 2024-04-25T11:10:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.196+0000 7fc861600640 1 -- 172.21.15.7:0/972168819 shutdown_connections 2024-04-25T11:10:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.196+0000 7fc861600640 1 --2- 172.21.15.7:0/972168819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85c06f330 0x7fc85c06f740 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.196+0000 7fc861600640 1 --2- 172.21.15.7:0/972168819 >> v2:172.21.15.87:3300/0 conn(0x7fc85c0cff00 0x7fc85c0d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.196+0000 7fc861600640 1 -- 172.21.15.7:0/972168819 >> 172.21.15.7:0/972168819 conn(0x7fc85c0740f0 msgr2=0x7fc85c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.196+0000 7fc861600640 1 -- 172.21.15.7:0/972168819 shutdown_connections 2024-04-25T11:10:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.196+0000 7fc861600640 1 -- 172.21.15.7:0/972168819 wait complete. 2024-04-25T11:10:19.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.198+0000 7fc861600640 1 Processor -- start 2024-04-25T11:10:19.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.198+0000 7fc861600640 1 -- start start 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc861600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc85c06f330 0x7fc85c1ee030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc861600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc85c0cff00 0x7fc85c13e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc861600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85c167de0 0x7fc85c1ef290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc861600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c168bb0 con 0x7fc85c167de0 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc861600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c168a30 con 0x7fc85c06f330 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc861600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc85c147340 con 0x7fc85c0cff00 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc860c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85c167de0 0x7fc85c1ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc860c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85c167de0 0x7fc85c1ef290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54704/0 (socket says 172.21.15.7:54704) 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc860c00640 1 -- 172.21.15.7:0/2951458820 learned_addr learned my addr 172.21.15.7:0/2951458820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc860c00640 1 -- 172.21.15.7:0/2951458820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc85c0cff00 msgr2=0x7fc85c13e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc860c00640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc85c0cff00 0x7fc85c13e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc860c00640 1 -- 172.21.15.7:0/2951458820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc85c06f330 msgr2=0x7fc85c1ee030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.200+0000 7fc85be00640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc85c06f330 0x7fc85c1ee030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:19.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.201+0000 7fc860c00640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc85c06f330 0x7fc85c1ee030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.201+0000 7fc860c00640 1 -- 172.21.15.7:0/2951458820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c146c40 con 0x7fc85c167de0 2024-04-25T11:10:19.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.201+0000 7fc860c00640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85c167de0 0x7fc85c1ef290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc85402a2b0 tx=0x7fc8540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:19.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.201+0000 7fc853e00640 1 -- 172.21.15.7:0/2951458820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854037040 con 0x7fc85c167de0 2024-04-25T11:10:19.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.201+0000 7fc853e00640 1 -- 172.21.15.7:0/2951458820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc854042070 con 0x7fc85c167de0 2024-04-25T11:10:19.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.201+0000 7fc861600640 1 -- 172.21.15.7:0/2951458820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85c13ec10 con 0x7fc85c167de0 2024-04-25T11:10:19.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.201+0000 7fc853e00640 1 -- 172.21.15.7:0/2951458820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854030920 con 0x7fc85c167de0 2024-04-25T11:10:19.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.201+0000 7fc861600640 1 -- 172.21.15.7:0/2951458820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85c15ba90 con 0x7fc85c167de0 2024-04-25T11:10:19.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.202+0000 7fc861600640 1 -- 172.21.15.7:0/2951458820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc85c050740 con 0x7fc85c167de0 2024-04-25T11:10:19.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.204+0000 7fc853e00640 1 -- 172.21.15.7:0/2951458820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc854030ac0 con 0x7fc85c167de0 2024-04-25T11:10:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.205+0000 7fc853e00640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8380744e0 0x7fc8380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.205+0000 7fc853e00640 1 -- 172.21.15.7:0/2951458820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fc8540b57c0 con 0x7fc85c167de0 2024-04-25T11:10:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.205+0000 7fc85be00640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8380744e0 0x7fc8380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:19.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.206+0000 7fc85be00640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8380744e0 0x7fc8380769f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc844006fd0 tx=0x7fc844008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.209+0000 7fc853e00640 1 -- 172.21.15.7:0/2951458820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc85407fc40 con 0x7fc85c167de0 2024-04-25T11:10:19.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.414+0000 7fc861600640 1 -- 172.21.15.7:0/2951458820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc85c0b1c00 con 0x7fc85c167de0 2024-04-25T11:10:19.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.415+0000 7fc853e00640 1 -- 172.21.15.7:0/2951458820 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (secure 0 0 0) 0x7fc854083f40 con 0x7fc85c167de0 2024-04-25T11:10:19.415 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:10:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.417+0000 7fc861600640 1 -- 172.21.15.7:0/2951458820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8380744e0 msgr2=0x7fc8380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.417+0000 7fc861600640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8380744e0 0x7fc8380769f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc844006fd0 tx=0x7fc844008040 comp rx=0 tx=0).stop 2024-04-25T11:10:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.417+0000 7fc861600640 1 -- 172.21.15.7:0/2951458820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85c167de0 msgr2=0x7fc85c1ef290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.417+0000 7fc861600640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85c167de0 0x7fc85c1ef290 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc85402a2b0 tx=0x7fc8540046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.418+0000 7fc861600640 1 -- 172.21.15.7:0/2951458820 shutdown_connections 2024-04-25T11:10:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.418+0000 7fc861600640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8380744e0 0x7fc8380769f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.418+0000 7fc861600640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc85c167de0 0x7fc85c1ef290 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.418+0000 7fc861600640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc85c0cff00 0x7fc85c13e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.418+0000 7fc861600640 1 --2- 172.21.15.7:0/2951458820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc85c06f330 0x7fc85c1ee030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.418+0000 7fc861600640 1 -- 172.21.15.7:0/2951458820 >> 172.21.15.7:0/2951458820 conn(0x7fc85c0740f0 msgr2=0x7fc85c1646f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.418+0000 7fc861600640 1 -- 172.21.15.7:0/2951458820 shutdown_connections 2024-04-25T11:10:19.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.418+0000 7fc861600640 1 -- 172.21.15.7:0/2951458820 wait complete. 2024-04-25T11:10:19.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:10:19.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.726+0000 7f715be00640 1 Processor -- start 2024-04-25T11:10:19.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.727+0000 7f715be00640 1 -- start start 2024-04-25T11:10:19.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.729+0000 7f715be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f715c06fb00 0x7f715c0b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.729+0000 7f715be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f715c06d580 0x7f715c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.729+0000 7f715be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f715c06ded0 con 0x7f715c15ff10 2024-04-25T11:10:19.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.729+0000 7f715be00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f715c0af2b0 con 0x7f715c06fb00 2024-04-25T11:10:19.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.729+0000 7f715be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f715c0af550 con 0x7f715c06d580 2024-04-25T11:10:19.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.729+0000 7f715aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f715c06fb00 0x7f715c0b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:19.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.729+0000 7f715aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f715c06fb00 0x7f715c0b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54718/0 (socket says 172.21.15.7:54718) 2024-04-25T11:10:19.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.729+0000 7f715aa00640 1 -- 172.21.15.7:0/1522271098 learned_addr learned my addr 172.21.15.7:0/1522271098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:19.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.730+0000 7f715aa00640 1 -- 172.21.15.7:0/1522271098 >> v2:172.21.15.87:3300/0 conn(0x7f715c06d580 msgr2=0x7f715c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:19.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.730+0000 7f715aa00640 1 --2- 172.21.15.7:0/1522271098 >> v2:172.21.15.87:3300/0 conn(0x7f715c06d580 0x7f715c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.730+0000 7f715aa00640 1 -- 172.21.15.7:0/1522271098 >> v1:172.21.15.87:6789/0 conn(0x7f715c15ff10 legacy=0x7f715c0b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:19.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.730+0000 7f715aa00640 1 -- 172.21.15.7:0/1522271098 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c0af830 con 0x7f715c06fb00 2024-04-25T11:10:19.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.731+0000 7f715aa00640 1 --2- 172.21.15.7:0/1522271098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb00 0x7f715c0b0030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f715c074920 tx=0x7f714c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2faf517c7f62400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:19.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.731+0000 7f715a000640 1 -- 172.21.15.7:0/1522271098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c037070 con 0x7f715c06fb00 2024-04-25T11:10:19.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.731+0000 7f715a000640 1 -- 172.21.15.7:0/1522271098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714c032440 con 0x7f715c06fb00 2024-04-25T11:10:19.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.731+0000 7f715a000640 1 -- 172.21.15.7:0/1522271098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c032810 con 0x7f715c06fb00 2024-04-25T11:10:19.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.732+0000 7f715be00640 1 -- 172.21.15.7:0/1522271098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb00 msgr2=0x7f715c0b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:19.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.732+0000 7f715be00640 1 --2- 172.21.15.7:0/1522271098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb00 0x7f715c0b0030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f715c074920 tx=0x7f714c02a230 comp rx=0 tx=0).stop 2024-04-25T11:10:19.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.733+0000 7f715be00640 1 -- 172.21.15.7:0/1522271098 shutdown_connections 2024-04-25T11:10:19.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.733+0000 7f715be00640 1 --2- 172.21.15.7:0/1522271098 >> v2:172.21.15.87:3300/0 conn(0x7f715c06d580 0x7f715c06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.733+0000 7f715be00640 1 --2- 172.21.15.7:0/1522271098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb00 0x7f715c0b0030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.733+0000 7f715be00640 1 -- 172.21.15.7:0/1522271098 >> 172.21.15.7:0/1522271098 conn(0x7f715c073090 msgr2=0x7f715c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:19.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.733+0000 7f715be00640 1 -- 172.21.15.7:0/1522271098 shutdown_connections 2024-04-25T11:10:19.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.733+0000 7f715be00640 1 -- 172.21.15.7:0/1522271098 wait complete. 2024-04-25T11:10:19.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.736+0000 7f715be00640 1 Processor -- start 2024-04-25T11:10:19.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.736+0000 7f715be00640 1 -- start start 2024-04-25T11:10:19.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.738+0000 7f715be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c06d580 0x7f715c0b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.738+0000 7f715be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb00 0x7f715c1f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.738+0000 7f715be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f715c15ff10 0x7f715c0b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.738+0000 7f715be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f715c160320 con 0x7f715c06fb00 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.738+0000 7f715be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f715c070070 con 0x7f715c06d580 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.738+0000 7f715be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f715c0b4690 con 0x7f715c15ff10 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.738+0000 7f7153e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb00 0x7f715c1f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.738+0000 7f7153e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb00 0x7f715c1f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54730/0 (socket says 172.21.15.7:54730) 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.738+0000 7f7153e00640 1 -- 172.21.15.7:0/297205661 learned_addr learned my addr 172.21.15.7:0/297205661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f715aa00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c06d580 0x7f715c0b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f7153e00640 1 -- 172.21.15.7:0/297205661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f715c15ff10 msgr2=0x7f715c0b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f7153e00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f715c15ff10 0x7f715c0b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f7153e00640 1 -- 172.21.15.7:0/297205661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c06d580 msgr2=0x7f715c0b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:19.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f7153e00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c06d580 0x7f715c0b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f7153e00640 1 -- 172.21.15.7:0/297205661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c1f7650 con 0x7f715c06fb00 2024-04-25T11:10:19.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f715aa00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c06d580 0x7f715c0b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:19.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f7153e00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb00 0x7f715c1f6920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f714400b3f0 tx=0x7f714400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:19.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f7152a00640 1 -- 172.21.15.7:0/297205661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144015020 con 0x7f715c06fb00 2024-04-25T11:10:19.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f7152a00640 1 -- 172.21.15.7:0/297205661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7144011830 con 0x7f715c06fb00 2024-04-25T11:10:19.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f715be00640 1 -- 172.21.15.7:0/297205661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c144a70 con 0x7f715c06fb00 2024-04-25T11:10:19.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.739+0000 7f7152a00640 1 -- 172.21.15.7:0/297205661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144011c20 con 0x7f715c06fb00 2024-04-25T11:10:19.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.740+0000 7f715be00640 1 -- 172.21.15.7:0/297205661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c145030 con 0x7f715c06fb00 2024-04-25T11:10:19.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.741+0000 7f715be00640 1 -- 172.21.15.7:0/297205661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715c0d0a40 con 0x7f715c06fb00 2024-04-25T11:10:19.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.742+0000 7f7152a00640 1 -- 172.21.15.7:0/297205661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7144018020 con 0x7f715c06fb00 2024-04-25T11:10:19.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.743+0000 7f7152a00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71380744b0 0x7f71380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:19.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.743+0000 7f7152a00640 1 -- 172.21.15.7:0/297205661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f7144094bb0 con 0x7f715c06fb00 2024-04-25T11:10:19.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.743+0000 7f715aa00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71380744b0 0x7f71380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:19.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.744+0000 7f715aa00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71380744b0 0x7f71380769c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f714c034000 tx=0x7f714c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:19.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.747+0000 7f7152a00640 1 -- 172.21.15.7:0/297205661 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f715c0d0a40 con 0x7f715c06fb00 2024-04-25T11:10:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.958+0000 7f715be00640 1 -- 172.21.15.7:0/297205661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f715c050740 con 0x7f715c06fb00 2024-04-25T11:10:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.959+0000 7f7152a00640 1 -- 172.21.15.7:0/297205661 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f714401f030 con 0x7f715c06fb00 2024-04-25T11:10:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:19.959 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-25T11:10:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:10:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-25T11:10:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-25T11:10:19.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:10:19.960 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-25T11:10:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.961+0000 7f715be00640 1 -- 172.21.15.7:0/297205661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71380744b0 msgr2=0x7f71380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.961+0000 7f715be00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71380744b0 0x7f71380769c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f714c034000 tx=0x7f714c02e000 comp rx=0 tx=0).stop 2024-04-25T11:10:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.962+0000 7f715be00640 1 -- 172.21.15.7:0/297205661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb00 msgr2=0x7f715c1f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.962+0000 7f715be00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb00 0x7f715c1f6920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f714400b3f0 tx=0x7f714400b8b0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.962+0000 7f715be00640 1 -- 172.21.15.7:0/297205661 shutdown_connections 2024-04-25T11:10:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.962+0000 7f715be00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71380744b0 0x7f71380769c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.962+0000 7f715be00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f715c15ff10 0x7f715c0b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.962+0000 7f715be00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb00 0x7f715c1f6920 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.962+0000 7f715be00640 1 --2- 172.21.15.7:0/297205661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f715c06d580 0x7f715c0b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.962+0000 7f715be00640 1 -- 172.21.15.7:0/297205661 >> 172.21.15.7:0/297205661 conn(0x7f715c073090 msgr2=0x7f715c15d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.962+0000 7f715be00640 1 -- 172.21.15.7:0/297205661 shutdown_connections 2024-04-25T11:10:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:19.962+0000 7f715be00640 1 -- 172.21.15.7:0/297205661 wait complete. 2024-04-25T11:10:19.988 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:10:19.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:10:20.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.281+0000 7f7e59200640 1 Processor -- start 2024-04-25T11:10:20.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.281+0000 7f7e59200640 1 -- start start 2024-04-25T11:10:20.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.283+0000 7f7e59200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e541620a0 0x7f7e540b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.283+0000 7f7e59200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5409f9c0 con 0x7f7e540b48a0 2024-04-25T11:10:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.283+0000 7f7e59200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e541682c0 con 0x7f7e540b0be0 2024-04-25T11:10:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.283+0000 7f7e59200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7e54168ed0 con 0x7f7e541620a0 2024-04-25T11:10:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.283+0000 7f7e51e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7e540b48a0 0x7f7e54070db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41446/0 (socket says 172.21.15.7:41446) 2024-04-25T11:10:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.283+0000 7f7e51e00640 1 -- 172.21.15.7:0/1779664064 learned_addr learned my addr 172.21.15.7:0/1779664064 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 434759958 0 0) 0x7f7e5409f9c0 con 0x7f7e540b48a0 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e38003520 con 0x7f7e540b48a0 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e4be00640 1 --2- 172.21.15.7:0/1779664064 >> v2:172.21.15.87:3300/0 conn(0x7f7e541620a0 0x7f7e540b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1239693930 0 0) 0x7f7e38003520 con 0x7f7e540b48a0 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 >> v2:172.21.15.87:3300/0 conn(0x7f7e541620a0 msgr2=0x7f7e540b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 --2- 172.21.15.7:0/1779664064 >> v2:172.21.15.87:3300/0 conn(0x7f7e541620a0 0x7f7e540b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 >> v1:172.21.15.87:6789/0 conn(0x7f7e540b0be0 legacy=0x7f7e54164ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e541685a0 con 0x7f7e540b48a0 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f7e44001670 con 0x7f7e540b48a0 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 >> v1:172.21.15.7:6789/0 conn(0x7f7e540b48a0 legacy=0x7f7e54070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 --2- 172.21.15.7:0/1779664064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e38003a30 0x7f7e38003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 --2- 172.21.15.7:0/1779664064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e38004800 0x7f7e38004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 --2- 172.21.15.7:0/1779664064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e380099b0 0x7f7e3800bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e54168ed0 con 0x7f7e38004800 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3800c3c0 con 0x7f7e380099b0 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.284+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7e3800c5c0 con 0x7f7e38003a30 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.285+0000 7f7e4be00640 1 --2- 172.21.15.7:0/1779664064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e38004800 0x7f7e38004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.285+0000 7f7e4be00640 1 -- 172.21.15.7:0/1779664064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e38003a30 msgr2=0x7f7e38003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.285+0000 7f7e4be00640 1 --2- 172.21.15.7:0/1779664064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e38003a30 0x7f7e38003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.285+0000 7f7e4be00640 1 -- 172.21.15.7:0/1779664064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e380099b0 msgr2=0x7f7e3800bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.285+0000 7f7e4be00640 1 --2- 172.21.15.7:0/1779664064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e380099b0 0x7f7e3800bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.285+0000 7f7e4be00640 1 -- 172.21.15.7:0/1779664064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e38003520 con 0x7f7e38004800 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.285+0000 7f7e52800640 1 -- 172.21.15.7:0/1779664064 reap_dead start 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.286+0000 7f7e4be00640 1 --2- 172.21.15.7:0/1779664064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e38004800 0x7f7e38004c10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00ba00 tx=0x7f7e38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fd42ff7e7ed80f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.286+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c030070 con 0x7f7e38004800 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.286+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e3c02ade0 con 0x7f7e38004800 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.286+0000 7f7e51400640 1 -- 172.21.15.7:0/1779664064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c00b600 con 0x7f7e38004800 2024-04-25T11:10:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.287+0000 7f7e59200640 1 -- 172.21.15.7:0/1779664064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e38004800 msgr2=0x7f7e38004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.287+0000 7f7e59200640 1 --2- 172.21.15.7:0/1779664064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e38004800 0x7f7e38004c10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00ba00 tx=0x7f7e38005830 comp rx=0 tx=0).stop 2024-04-25T11:10:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.287+0000 7f7e59200640 1 -- 172.21.15.7:0/1779664064 shutdown_connections 2024-04-25T11:10:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.287+0000 7f7e59200640 1 --2- 172.21.15.7:0/1779664064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e38004800 0x7f7e38004c10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.287+0000 7f7e59200640 1 -- 172.21.15.7:0/1779664064 >> 172.21.15.7:0/1779664064 conn(0x7f7e54073920 msgr2=0x7f7e54073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.287+0000 7f7e59200640 1 -- 172.21.15.7:0/1779664064 shutdown_connections 2024-04-25T11:10:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.287+0000 7f7e59200640 1 -- 172.21.15.7:0/1779664064 wait complete. 2024-04-25T11:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.290+0000 7f7e59200640 1 Processor -- start 2024-04-25T11:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.290+0000 7f7e59200640 1 -- start start 2024-04-25T11:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.291+0000 7f7e59200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e38004800 0x7f7e54159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.292+0000 7f7e59200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e5415c0b0 0x7f7e54163680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.292+0000 7f7e59200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e54163bc0 0x7f7e5415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.292+0000 7f7e59200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e541624b0 con 0x7f7e54163bc0 2024-04-25T11:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.292+0000 7f7e59200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7e54168b60 con 0x7f7e5415c0b0 2024-04-25T11:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.292+0000 7f7e59200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7e540b0ff0 con 0x7f7e38004800 2024-04-25T11:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.292+0000 7f7e52800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e54163bc0 0x7f7e5415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.292+0000 7f7e51e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e38004800 0x7f7e54159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.292+0000 7f7e52800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e54163bc0 0x7f7e5415a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48874/0 (socket says 172.21.15.7:48874) 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.292+0000 7f7e52800640 1 -- 172.21.15.7:0/2027246686 learned_addr learned my addr 172.21.15.7:0/2027246686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.292+0000 7f7e4be00640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e5415c0b0 0x7f7e54163680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.293+0000 7f7e52800640 1 -- 172.21.15.7:0/2027246686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e38004800 msgr2=0x7f7e54159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.293+0000 7f7e52800640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e38004800 0x7f7e54159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.293+0000 7f7e52800640 1 -- 172.21.15.7:0/2027246686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e5415c0b0 msgr2=0x7f7e54163680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.293+0000 7f7e52800640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e5415c0b0 0x7f7e54163680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.293+0000 7f7e52800640 1 -- 172.21.15.7:0/2027246686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e54163fd0 con 0x7f7e54163bc0 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.293+0000 7f7e52800640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e54163bc0 0x7f7e5415a4b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e54164b30 tx=0x7f7e4c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.293+0000 7f7e4a000640 1 -- 172.21.15.7:0/2027246686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c017070 con 0x7f7e54163bc0 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.293+0000 7f7e4a000640 1 -- 172.21.15.7:0/2027246686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c005d40 con 0x7f7e54163bc0 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.293+0000 7f7e4a000640 1 -- 172.21.15.7:0/2027246686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00a850 con 0x7f7e54163bc0 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.293+0000 7f7e59200640 1 -- 172.21.15.7:0/2027246686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e54159d80 con 0x7f7e54163bc0 2024-04-25T11:10:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.293+0000 7f7e59200640 1 -- 172.21.15.7:0/2027246686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e54166190 con 0x7f7e54163bc0 2024-04-25T11:10:20.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.295+0000 7f7e59200640 1 -- 172.21.15.7:0/2027246686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e54165c20 con 0x7f7e54163bc0 2024-04-25T11:10:20.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.296+0000 7f7e4a000640 1 -- 172.21.15.7:0/2027246686 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e4c01a070 con 0x7f7e54163bc0 2024-04-25T11:10:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.297+0000 7f7e4a000640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e300744e0 0x7f7e300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.297+0000 7f7e4a000640 1 -- 172.21.15.7:0/2027246686 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f7e4c0941d0 con 0x7f7e54163bc0 2024-04-25T11:10:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.297+0000 7f7e51e00640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e300744e0 0x7f7e300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.298+0000 7f7e51e00640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e300744e0 0x7f7e300769f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7e44001eb0 tx=0x7f7e54168490 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.302+0000 7f7e4a000640 1 -- 172.21.15.7:0/2027246686 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e54163fd0 con 0x7f7e54163bc0 2024-04-25T11:10:20.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:20.511+0000 7f7e59200640 1 -- 172.21.15.7:0/2027246686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7e540af6c0 con 0x7f7e54163bc0 2024-04-25T11:10:21.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.418+0000 7f7e4a000640 1 -- 172.21.15.7:0/2027246686 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7f7e540af6c0 con 0x7f7e54163bc0 2024-04-25T11:10:21.419 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-25T11:10:21.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.423+0000 7f7e59200640 1 -- 172.21.15.7:0/2027246686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e300744e0 msgr2=0x7f7e300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:21.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.423+0000 7f7e59200640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e300744e0 0x7f7e300769f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7e44001eb0 tx=0x7f7e54168490 comp rx=0 tx=0).stop 2024-04-25T11:10:21.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.423+0000 7f7e59200640 1 -- 172.21.15.7:0/2027246686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e54163bc0 msgr2=0x7f7e5415a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:21.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.423+0000 7f7e59200640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e54163bc0 0x7f7e5415a4b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e54164b30 tx=0x7f7e4c00cb90 comp rx=0 tx=0).stop 2024-04-25T11:10:21.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.423+0000 7f7e59200640 1 -- 172.21.15.7:0/2027246686 shutdown_connections 2024-04-25T11:10:21.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.423+0000 7f7e59200640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e300744e0 0x7f7e300769f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:21.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.423+0000 7f7e59200640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e54163bc0 0x7f7e5415a4b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:21.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.423+0000 7f7e59200640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e5415c0b0 0x7f7e54163680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:21.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.423+0000 7f7e59200640 1 --2- 172.21.15.7:0/2027246686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e38004800 0x7f7e54159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:21.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.423+0000 7f7e59200640 1 -- 172.21.15.7:0/2027246686 >> 172.21.15.7:0/2027246686 conn(0x7f7e54073920 msgr2=0x7f7e540b2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:21.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.424+0000 7f7e59200640 1 -- 172.21.15.7:0/2027246686 shutdown_connections 2024-04-25T11:10:21.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.424+0000 7f7e59200640 1 -- 172.21.15.7:0/2027246686 wait complete. 2024-04-25T11:10:21.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T11:10:21.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.725+0000 7facafc00640 1 Processor -- start 2024-04-25T11:10:21.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.725+0000 7facafc00640 1 -- start start 2024-04-25T11:10:21.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.728+0000 7facafc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faca8153e90 0x7faca8070a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:21.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.728+0000 7facafc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faca80b6370 0x7faca80b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:21.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.728+0000 7facafc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca80b0630 con 0x7faca806d580 2024-04-25T11:10:21.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.728+0000 7facafc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7faca80705d0 con 0x7faca80b6370 2024-04-25T11:10:21.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.728+0000 7facafc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faca80b08b0 con 0x7faca8153e90 2024-04-25T11:10:21.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.728+0000 7faca7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faca80b6370 0x7faca80b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:21.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.728+0000 7facadc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faca806d580 0x7faca806d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41452/0 (socket says 172.21.15.7:41452) 2024-04-25T11:10:21.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.728+0000 7facadc00640 1 -- 172.21.15.7:0/920302646 learned_addr learned my addr 172.21.15.7:0/920302646 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:21.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.728+0000 7facad200640 1 --2- 172.21.15.7:0/920302646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faca8153e90 0x7faca8070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:21.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.729+0000 7faca7e00640 1 -- 172.21.15.7:0/920302646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faca8153e90 msgr2=0x7faca8070a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:21.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.729+0000 7faca7e00640 1 --2- 172.21.15.7:0/920302646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faca8153e90 0x7faca8070a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:21.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.729+0000 7faca7e00640 1 -- 172.21.15.7:0/920302646 >> v1:172.21.15.7:6789/0 conn(0x7faca806d580 legacy=0x7faca806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.729+0000 7faca7e00640 1 -- 172.21.15.7:0/920302646 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca81542a0 con 0x7faca80b6370 2024-04-25T11:10:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.729+0000 7faca7400640 1 -- 172.21.15.7:0/920302646 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3210684014 0 0) 0x7faca0001490 con 0x7faca806d580 2024-04-25T11:10:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.730+0000 7faca7e00640 1 --2- 172.21.15.7:0/920302646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faca80b6370 0x7faca80b6780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac940097a0 tx=0x7fac9402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a096a83b87d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:21.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.730+0000 7faca7400640 1 -- 172.21.15.7:0/920302646 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94037070 con 0x7faca80b6370 2024-04-25T11:10:21.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.730+0000 7faca7400640 1 -- 172.21.15.7:0/920302646 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac94032440 con 0x7faca80b6370 2024-04-25T11:10:21.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.730+0000 7faca7400640 1 -- 172.21.15.7:0/920302646 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94032810 con 0x7faca80b6370 2024-04-25T11:10:21.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.731+0000 7facafc00640 1 -- 172.21.15.7:0/920302646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faca80b6370 msgr2=0x7faca80b6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:21.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.731+0000 7facafc00640 1 --2- 172.21.15.7:0/920302646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faca80b6370 0x7faca80b6780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac940097a0 tx=0x7fac9402a180 comp rx=0 tx=0).stop 2024-04-25T11:10:21.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.731+0000 7facafc00640 1 -- 172.21.15.7:0/920302646 shutdown_connections 2024-04-25T11:10:21.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.731+0000 7facafc00640 1 --2- 172.21.15.7:0/920302646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faca80b6370 0x7faca80b6780 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:21.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.731+0000 7facafc00640 1 --2- 172.21.15.7:0/920302646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faca8153e90 0x7faca8070a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:21.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.731+0000 7facafc00640 1 -- 172.21.15.7:0/920302646 >> 172.21.15.7:0/920302646 conn(0x7faca80d1d40 msgr2=0x7faca8073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:21.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.731+0000 7facafc00640 1 -- 172.21.15.7:0/920302646 shutdown_connections 2024-04-25T11:10:21.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.731+0000 7facafc00640 1 -- 172.21.15.7:0/920302646 wait complete. 2024-04-25T11:10:21.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.734+0000 7facafc00640 1 Processor -- start 2024-04-25T11:10:21.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.734+0000 7facafc00640 1 -- start start 2024-04-25T11:10:21.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.735+0000 7facafc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faca806d580 0x7faca81e5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:21.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.735+0000 7facafc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faca80b6370 0x7faca813f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:21.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.735+0000 7facafc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faca8153e90 0x7faca815be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.735+0000 7facafc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faca8070fa0 con 0x7faca8153e90 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.735+0000 7facafc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faca80b0e50 con 0x7faca806d580 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.735+0000 7facafc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faca8071120 con 0x7faca80b6370 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.736+0000 7facadc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faca8153e90 0x7faca815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.736+0000 7facadc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faca8153e90 0x7faca815be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48896/0 (socket says 172.21.15.7:48896) 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.736+0000 7facadc00640 1 -- 172.21.15.7:0/2513986257 learned_addr learned my addr 172.21.15.7:0/2513986257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.736+0000 7facad200640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faca806d580 0x7faca81e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.736+0000 7faca7e00640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faca80b6370 0x7faca813f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.736+0000 7facadc00640 1 -- 172.21.15.7:0/2513986257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faca80b6370 msgr2=0x7faca813f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.736+0000 7facadc00640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faca80b6370 0x7faca813f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.736+0000 7facadc00640 1 -- 172.21.15.7:0/2513986257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faca806d580 msgr2=0x7faca81e5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.736+0000 7facadc00640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faca806d580 0x7faca81e5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:21.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.736+0000 7facadc00640 1 -- 172.21.15.7:0/2513986257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca81e5d40 con 0x7faca8153e90 2024-04-25T11:10:21.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.737+0000 7facadc00640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faca8153e90 0x7faca815be70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faca000c700 tx=0x7faca000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:21.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.737+0000 7faca4c00640 1 -- 172.21.15.7:0/2513986257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0018070 con 0x7faca8153e90 2024-04-25T11:10:21.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.737+0000 7faca4c00640 1 -- 172.21.15.7:0/2513986257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca000ce10 con 0x7faca8153e90 2024-04-25T11:10:21.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.737+0000 7faca4c00640 1 -- 172.21.15.7:0/2513986257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca000a880 con 0x7faca8153e90 2024-04-25T11:10:21.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.737+0000 7facafc00640 1 -- 172.21.15.7:0/2513986257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca815f390 con 0x7faca8153e90 2024-04-25T11:10:21.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.737+0000 7facafc00640 1 -- 172.21.15.7:0/2513986257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca815f950 con 0x7faca8153e90 2024-04-25T11:10:21.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.738+0000 7facafc00640 1 -- 172.21.15.7:0/2513986257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca8050740 con 0x7faca8153e90 2024-04-25T11:10:21.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.740+0000 7faca4c00640 1 -- 172.21.15.7:0/2513986257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faca001b070 con 0x7faca8153e90 2024-04-25T11:10:21.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.741+0000 7faca4c00640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac880744b0 0x7fac880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.741+0000 7faca4c00640 1 -- 172.21.15.7:0/2513986257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7faca00945f0 con 0x7faca8153e90 2024-04-25T11:10:21.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.741+0000 7facad200640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac880744b0 0x7fac880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:21.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.742+0000 7facad200640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac880744b0 0x7fac880769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009a30 tx=0x7fac9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:21.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.746+0000 7faca4c00640 1 -- 172.21.15.7:0/2513986257 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faca80b0630 con 0x7faca8153e90 2024-04-25T11:10:21.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:21.952+0000 7facafc00640 1 -- 172.21.15.7:0/2513986257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7faca806fea0 con 0x7faca8153e90 2024-04-25T11:10:22.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.434+0000 7faca4c00640 1 -- 172.21.15.7:0/2513986257 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) ==== 171+0+0 (secure 0 0 0) 0x7faca005ef60 con 0x7faca8153e90 2024-04-25T11:10:22.435 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-25T11:10:22.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.438+0000 7facafc00640 1 -- 172.21.15.7:0/2513986257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac880744b0 msgr2=0x7fac880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:22.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.438+0000 7facafc00640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac880744b0 0x7fac880769c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009a30 tx=0x7fac9c008040 comp rx=0 tx=0).stop 2024-04-25T11:10:22.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.438+0000 7facafc00640 1 -- 172.21.15.7:0/2513986257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faca8153e90 msgr2=0x7faca815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:22.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.438+0000 7facafc00640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faca8153e90 0x7faca815be70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faca000c700 tx=0x7faca000cbc0 comp rx=0 tx=0).stop 2024-04-25T11:10:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.439+0000 7facafc00640 1 -- 172.21.15.7:0/2513986257 shutdown_connections 2024-04-25T11:10:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.439+0000 7facafc00640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac880744b0 0x7fac880769c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.439+0000 7facafc00640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faca8153e90 0x7faca815be70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:22.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.439+0000 7facafc00640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faca80b6370 0x7faca813f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:22.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.439+0000 7facafc00640 1 --2- 172.21.15.7:0/2513986257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faca806d580 0x7faca81e5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:22.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.439+0000 7facafc00640 1 -- 172.21.15.7:0/2513986257 >> 172.21.15.7:0/2513986257 conn(0x7faca80d1d40 msgr2=0x7faca8153340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:22.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.441+0000 7facafc00640 1 -- 172.21.15.7:0/2513986257 shutdown_connections 2024-04-25T11:10:22.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.441+0000 7facafc00640 1 -- 172.21.15.7:0/2513986257 wait complete. 2024-04-25T11:10:22.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:10:22.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.752+0000 7f4833e00640 1 Processor -- start 2024-04-25T11:10:22.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.752+0000 7f4833e00640 1 -- start start 2024-04-25T11:10:22.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.754+0000 7f4833e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4834146180 0x7f4834070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:22.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.754+0000 7f4833e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48341490f0 0x7f4834149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:22.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.754+0000 7f4833e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4834070860 con 0x7f48340b01d0 2024-04-25T11:10:22.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.754+0000 7f4833e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4834146590 con 0x7f4834146180 2024-04-25T11:10:22.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.754+0000 7f4833e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f483406f280 con 0x7f48341490f0 2024-04-25T11:10:22.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.755+0000 7f4833400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f48340b01d0 0x7f48340b05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41460/0 (socket says 172.21.15.7:41460) 2024-04-25T11:10:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.755+0000 7f4833400640 1 -- 172.21.15.7:0/629586064 learned_addr learned my addr 172.21.15.7:0/629586064 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.755+0000 7f4832a00640 1 --2- 172.21.15.7:0/629586064 >> v2:172.21.15.87:3300/0 conn(0x7f4834146180 0x7f4834070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.755+0000 7f4832000640 1 --2- 172.21.15.7:0/629586064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48341490f0 0x7f4834149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.755+0000 7f4831600640 1 -- 172.21.15.7:0/629586064 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2246004750 0 0) 0x7f4834070860 con 0x7f48340b01d0 2024-04-25T11:10:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.756+0000 7f4831600640 1 -- 172.21.15.7:0/629586064 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4818003520 con 0x7f48340b01d0 2024-04-25T11:10:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.756+0000 7f4832a00640 1 -- 172.21.15.7:0/629586064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48341490f0 msgr2=0x7f4834149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.756+0000 7f4832a00640 1 --2- 172.21.15.7:0/629586064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48341490f0 0x7f4834149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.756+0000 7f4832a00640 1 -- 172.21.15.7:0/629586064 >> v1:172.21.15.7:6789/0 conn(0x7f48340b01d0 legacy=0x7f48340b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.756+0000 7f4832a00640 1 -- 172.21.15.7:0/629586064 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483406f560 con 0x7f4834146180 2024-04-25T11:10:22.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.758+0000 7f4832a00640 1 --2- 172.21.15.7:0/629586064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4834146180 0x7f4834070320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4824009d90 tx=0x7f482402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2144f9ade9b49a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.758+0000 7f4831600640 1 -- 172.21.15.7:0/629586064 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824007740 con 0x7f4834146180 2024-04-25T11:10:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.758+0000 7f4831600640 1 -- 172.21.15.7:0/629586064 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48240078e0 con 0x7f4834146180 2024-04-25T11:10:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.758+0000 7f4831600640 1 -- 172.21.15.7:0/629586064 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824007cb0 con 0x7f4834146180 2024-04-25T11:10:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.759+0000 7f4833e00640 1 -- 172.21.15.7:0/629586064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4834146180 msgr2=0x7f4834070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.759+0000 7f4833e00640 1 --2- 172.21.15.7:0/629586064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4834146180 0x7f4834070320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4824009d90 tx=0x7f482402a660 comp rx=0 tx=0).stop 2024-04-25T11:10:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.759+0000 7f4833e00640 1 -- 172.21.15.7:0/629586064 shutdown_connections 2024-04-25T11:10:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.759+0000 7f4833e00640 1 --2- 172.21.15.7:0/629586064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48341490f0 0x7f4834149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.759+0000 7f4833e00640 1 --2- 172.21.15.7:0/629586064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4834146180 0x7f4834070320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.759+0000 7f4833e00640 1 -- 172.21.15.7:0/629586064 >> 172.21.15.7:0/629586064 conn(0x7f48340740f0 msgr2=0x7f483406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.759+0000 7f4833e00640 1 -- 172.21.15.7:0/629586064 shutdown_connections 2024-04-25T11:10:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.759+0000 7f4833e00640 1 -- 172.21.15.7:0/629586064 wait complete. 2024-04-25T11:10:22.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.761+0000 7f4833e00640 1 Processor -- start 2024-04-25T11:10:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.762+0000 7f4833e00640 1 -- start start 2024-04-25T11:10:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.763+0000 7f4833e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48340b01d0 0x7f48341417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.763+0000 7f4833e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4834146180 0x7f4834160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.763+0000 7f4833e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48341490f0 0x7f4834161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.763+0000 7f4833e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f483406fe00 con 0x7f48341490f0 2024-04-25T11:10:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.763+0000 7f4833e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f483406fb20 con 0x7f48340b01d0 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.763+0000 7f4833e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f483406eea0 con 0x7f4834146180 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.763+0000 7f4833400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48341490f0 0x7f4834161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.764+0000 7f4833400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48341490f0 0x7f4834161f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48906/0 (socket says 172.21.15.7:48906) 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.764+0000 7f4833400640 1 -- 172.21.15.7:0/787364096 learned_addr learned my addr 172.21.15.7:0/787364096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.764+0000 7f4832a00640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48340b01d0 0x7f48341417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.764+0000 7f4832000640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4834146180 0x7f4834160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.764+0000 7f4833400640 1 -- 172.21.15.7:0/787364096 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4834146180 msgr2=0x7f4834160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.764+0000 7f4833400640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4834146180 0x7f4834160430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.764+0000 7f4833400640 1 -- 172.21.15.7:0/787364096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48340b01d0 msgr2=0x7f48341417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.764+0000 7f4833400640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48340b01d0 0x7f48341417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.764+0000 7f4833400640 1 -- 172.21.15.7:0/787364096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48341faa40 con 0x7f48341490f0 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.764+0000 7f4832a00640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48340b01d0 0x7f48341417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.764+0000 7f4833400640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48341490f0 0x7f4834161f60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f482c00c3c0 tx=0x7f482c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.765+0000 7f482aa00640 1 -- 172.21.15.7:0/787364096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c017070 con 0x7f48341490f0 2024-04-25T11:10:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.765+0000 7f482aa00640 1 -- 172.21.15.7:0/787364096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482c005ce0 con 0x7f48341490f0 2024-04-25T11:10:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.765+0000 7f482aa00640 1 -- 172.21.15.7:0/787364096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c007310 con 0x7f48341490f0 2024-04-25T11:10:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.765+0000 7f4833e00640 1 -- 172.21.15.7:0/787364096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4834161040 con 0x7f48341490f0 2024-04-25T11:10:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.765+0000 7f4833e00640 1 -- 172.21.15.7:0/787364096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48340cdad0 con 0x7f48341490f0 2024-04-25T11:10:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.766+0000 7f4833e00640 1 -- 172.21.15.7:0/787364096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4834160db0 con 0x7f48341490f0 2024-04-25T11:10:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.767+0000 7f482aa00640 1 -- 172.21.15.7:0/787364096 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f482c01a070 con 0x7f48341490f0 2024-04-25T11:10:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.768+0000 7f482aa00640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48140744b0 0x7f48140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.769+0000 7f482aa00640 1 -- 172.21.15.7:0/787364096 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f482c094560 con 0x7f48341490f0 2024-04-25T11:10:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.769+0000 7f4832a00640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48140744b0 0x7f48140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.770+0000 7f4832a00640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48140744b0 0x7f48140769c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4824034000 tx=0x7f4824033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:22.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:22.774+0000 7f482aa00640 1 -- 172.21.15.7:0/787364096 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f482c05ecb0 con 0x7f48341490f0 2024-04-25T11:10:23.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.022+0000 7f4833e00640 1 -- 172.21.15.7:0/787364096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f48340b4620 con 0x7f48341490f0 2024-04-25T11:10:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.439+0000 7f482aa00640 1 -- 172.21.15.7:0/787364096 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f48340b4620 con 0x7f48341490f0 2024-04-25T11:10:23.439 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T11:10:23.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.442+0000 7f4833e00640 1 -- 172.21.15.7:0/787364096 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48140744b0 msgr2=0x7f48140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:23.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.442+0000 7f4833e00640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48140744b0 0x7f48140769c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4824034000 tx=0x7f4824033040 comp rx=0 tx=0).stop 2024-04-25T11:10:23.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.442+0000 7f4833e00640 1 -- 172.21.15.7:0/787364096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48341490f0 msgr2=0x7f4834161f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:23.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.442+0000 7f4833e00640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48341490f0 0x7f4834161f60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f482c00c3c0 tx=0x7f482c00c880 comp rx=0 tx=0).stop 2024-04-25T11:10:23.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.443+0000 7f4833e00640 1 -- 172.21.15.7:0/787364096 shutdown_connections 2024-04-25T11:10:23.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.443+0000 7f4833e00640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48140744b0 0x7f48140769c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:23.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.443+0000 7f4833e00640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48341490f0 0x7f4834161f60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.443+0000 7f4833e00640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4834146180 0x7f4834160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.443+0000 7f4833e00640 1 --2- 172.21.15.7:0/787364096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48340b01d0 0x7f48341417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.443+0000 7f4833e00640 1 -- 172.21.15.7:0/787364096 >> 172.21.15.7:0/787364096 conn(0x7f48340740f0 msgr2=0x7f4834148720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.443+0000 7f4833e00640 1 -- 172.21.15.7:0/787364096 shutdown_connections 2024-04-25T11:10:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.443+0000 7f4833e00640 1 -- 172.21.15.7:0/787364096 wait complete. 2024-04-25T11:10:23.507 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T11:10:23.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T11:10:23.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.763+0000 7f0df6a00640 1 Processor -- start 2024-04-25T11:10:23.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.763+0000 7f0df6a00640 1 -- start start 2024-04-25T11:10:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.766+0000 7f0df6a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0df006de20 0x7f0df015ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:23.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.766+0000 7f0df6a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0df006d220 0x7f0df00af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.766+0000 7f0df6a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df01646e0 con 0x7f0df006ce10 2024-04-25T11:10:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.766+0000 7f0df6a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0df006c610 con 0x7f0df006de20 2024-04-25T11:10:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.766+0000 7f0df6a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0df006c8f0 con 0x7f0df006d220 2024-04-25T11:10:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.767+0000 7f0df5600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0df006de20 0x7f0df015ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.767+0000 7f0df5600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0df006de20 0x7f0df015ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48912/0 (socket says 172.21.15.7:48912) 2024-04-25T11:10:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.767+0000 7f0df5600640 1 -- 172.21.15.7:0/990278723 learned_addr learned my addr 172.21.15.7:0/990278723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.767+0000 7f0df6000640 1 --2- 172.21.15.7:0/990278723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0df006d220 0x7f0df00af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.767+0000 7f0df5600640 1 -- 172.21.15.7:0/990278723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0df006d220 msgr2=0x7f0df00af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:23.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.767+0000 7f0df5600640 1 --2- 172.21.15.7:0/990278723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0df006d220 0x7f0df00af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:23.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.767+0000 7f0df5600640 1 -- 172.21.15.7:0/990278723 >> v1:172.21.15.87:6789/0 conn(0x7f0df006ce10 legacy=0x7f0df00af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:23.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.767+0000 7f0df5600640 1 -- 172.21.15.7:0/990278723 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df01649c0 con 0x7f0df006de20 2024-04-25T11:10:23.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.768+0000 7f0df5600640 1 --2- 172.21.15.7:0/990278723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df006de20 0x7f0df015ffe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009870 tx=0x7f0de402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2856e744219fd6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:23.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.769+0000 7f0defe00640 1 -- 172.21.15.7:0/990278723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4037070 con 0x7f0df006de20 2024-04-25T11:10:23.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.769+0000 7f0defe00640 1 -- 172.21.15.7:0/990278723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de4032440 con 0x7f0df006de20 2024-04-25T11:10:23.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.769+0000 7f0defe00640 1 -- 172.21.15.7:0/990278723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4032810 con 0x7f0df006de20 2024-04-25T11:10:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.769+0000 7f0df6a00640 1 -- 172.21.15.7:0/990278723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df006de20 msgr2=0x7f0df015ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.769+0000 7f0df6a00640 1 --2- 172.21.15.7:0/990278723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df006de20 0x7f0df015ffe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009870 tx=0x7f0de402a1c0 comp rx=0 tx=0).stop 2024-04-25T11:10:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.769+0000 7f0df6a00640 1 -- 172.21.15.7:0/990278723 shutdown_connections 2024-04-25T11:10:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.769+0000 7f0df6a00640 1 --2- 172.21.15.7:0/990278723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0df006d220 0x7f0df00af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.769+0000 7f0df6a00640 1 --2- 172.21.15.7:0/990278723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df006de20 0x7f0df015ffe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.769+0000 7f0df6a00640 1 -- 172.21.15.7:0/990278723 >> 172.21.15.7:0/990278723 conn(0x7f0df0073ba0 msgr2=0x7f0df00b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.770+0000 7f0df6a00640 1 -- 172.21.15.7:0/990278723 shutdown_connections 2024-04-25T11:10:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.770+0000 7f0df6a00640 1 -- 172.21.15.7:0/990278723 wait complete. 2024-04-25T11:10:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.772+0000 7f0df6a00640 1 Processor -- start 2024-04-25T11:10:23.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.772+0000 7f0df6a00640 1 -- start start 2024-04-25T11:10:23.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.774+0000 7f0df6a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0df006ce10 0x7f0df0151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.774+0000 7f0df6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0df006d220 0x7f0df01f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.774+0000 7f0df6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df006de20 0x7f0df0159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.774+0000 7f0df6a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0df00d1a80 con 0x7f0df006de20 2024-04-25T11:10:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.774+0000 7f0df6a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0df00d1900 con 0x7f0df006d220 2024-04-25T11:10:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.774+0000 7f0df6a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0df00d1c00 con 0x7f0df006ce10 2024-04-25T11:10:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.774+0000 7f0df6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df006de20 0x7f0df0159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.774+0000 7f0df6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df006de20 0x7f0df0159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48916/0 (socket says 172.21.15.7:48916) 2024-04-25T11:10:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.774+0000 7f0df6000640 1 -- 172.21.15.7:0/4245202698 learned_addr learned my addr 172.21.15.7:0/4245202698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.774+0000 7f0df5600640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0df006ce10 0x7f0df0151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:23.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.774+0000 7f0df4c00640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0df006d220 0x7f0df01f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.775+0000 7f0df6000640 1 -- 172.21.15.7:0/4245202698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0df006ce10 msgr2=0x7f0df0151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.775+0000 7f0df6000640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0df006ce10 0x7f0df0151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.775+0000 7f0df6000640 1 -- 172.21.15.7:0/4245202698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0df006d220 msgr2=0x7f0df01f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.775+0000 7f0df6000640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0df006d220 0x7f0df01f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.775+0000 7f0df6000640 1 -- 172.21.15.7:0/4245202698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df0150190 con 0x7f0df006de20 2024-04-25T11:10:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.775+0000 7f0df6000640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df006de20 0x7f0df0159af0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0de8007e90 tx=0x7f0de800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.775+0000 7f0ded600640 1 -- 172.21.15.7:0/4245202698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de8014420 con 0x7f0df006de20 2024-04-25T11:10:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.775+0000 7f0ded600640 1 -- 172.21.15.7:0/4245202698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de8019070 con 0x7f0df006de20 2024-04-25T11:10:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.775+0000 7f0ded600640 1 -- 172.21.15.7:0/4245202698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de8014800 con 0x7f0df006de20 2024-04-25T11:10:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.775+0000 7f0df6a00640 1 -- 172.21.15.7:0/4245202698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df0150470 con 0x7f0df006de20 2024-04-25T11:10:23.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.775+0000 7f0df6a00640 1 -- 172.21.15.7:0/4245202698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df0150a30 con 0x7f0df006de20 2024-04-25T11:10:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.776+0000 7f0df6a00640 1 -- 172.21.15.7:0/4245202698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df015fb80 con 0x7f0df006de20 2024-04-25T11:10:23.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.778+0000 7f0ded600640 1 -- 172.21.15.7:0/4245202698 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0de80149a0 con 0x7f0df006de20 2024-04-25T11:10:23.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.779+0000 7f0ded600640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0dd00744e0 0x7f0dd00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:23.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.779+0000 7f0ded600640 1 -- 172.21.15.7:0/4245202698 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0de80959e0 con 0x7f0df006de20 2024-04-25T11:10:23.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.779+0000 7f0df5600640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0dd00744e0 0x7f0dd00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:23.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.780+0000 7f0df5600640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0dd00744e0 0x7f0dd00769f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0de4034000 tx=0x7f0de402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:23.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:23.784+0000 7f0ded600640 1 -- 172.21.15.7:0/4245202698 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0df015fb80 con 0x7f0df006de20 2024-04-25T11:10:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.002+0000 7f0df6a00640 1 -- 172.21.15.7:0/4245202698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f0df0050740 con 0x7f0df006de20 2024-04-25T11:10:24.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.470+0000 7f0ded600640 1 -- 172.21.15.7:0/4245202698 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) ==== 157+0+0 (secure 0 0 0) 0x7f0de805fbb0 con 0x7f0df006de20 2024-04-25T11:10:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.475+0000 7f0df6a00640 1 -- 172.21.15.7:0/4245202698 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0dd00744e0 msgr2=0x7f0dd00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:24.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.475+0000 7f0df6a00640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0dd00744e0 0x7f0dd00769f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0de4034000 tx=0x7f0de402e000 comp rx=0 tx=0).stop 2024-04-25T11:10:24.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.475+0000 7f0df6a00640 1 -- 172.21.15.7:0/4245202698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df006de20 msgr2=0x7f0df0159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:24.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.475+0000 7f0df6a00640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df006de20 0x7f0df0159af0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0de8007e90 tx=0x7f0de800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:10:24.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.475+0000 7f0df6a00640 1 -- 172.21.15.7:0/4245202698 shutdown_connections 2024-04-25T11:10:24.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.475+0000 7f0df6a00640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0dd00744e0 0x7f0dd00769f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:24.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.475+0000 7f0df6a00640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0df006de20 0x7f0df0159af0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:24.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.475+0000 7f0df6a00640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0df006d220 0x7f0df01f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:24.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.475+0000 7f0df6a00640 1 --2- 172.21.15.7:0/4245202698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0df006ce10 0x7f0df0151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:24.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.475+0000 7f0df6a00640 1 -- 172.21.15.7:0/4245202698 >> 172.21.15.7:0/4245202698 conn(0x7f0df0073ba0 msgr2=0x7f0df0163aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:24.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.477+0000 7f0df6a00640 1 -- 172.21.15.7:0/4245202698 shutdown_connections 2024-04-25T11:10:24.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.477+0000 7f0df6a00640 1 -- 172.21.15.7:0/4245202698 wait complete. 2024-04-25T11:10:24.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:10:24.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.805+0000 7f0c44a00640 1 Processor -- start 2024-04-25T11:10:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.805+0000 7f0c44a00640 1 -- start start 2024-04-25T11:10:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.808+0000 7f0c44a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c400d5190 0x7f0c401601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.808+0000 7f0c44a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c40161030 con 0x7f0c400d4d80 2024-04-25T11:10:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.808+0000 7f0c44a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c40161310 con 0x7f0c400d2e90 2024-04-25T11:10:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.808+0000 7f0c44a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0c400d4580 con 0x7f0c400d5190 2024-04-25T11:10:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.808+0000 7f0c3e800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0c400d4d80 0x7f0c400d2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41466/0 (socket says 172.21.15.7:41466) 2024-04-25T11:10:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.808+0000 7f0c3e800640 1 -- 172.21.15.7:0/3248615589 learned_addr learned my addr 172.21.15.7:0/3248615589 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.809+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2281462607 0 0) 0x7f0c40161030 con 0x7f0c400d4d80 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.809+0000 7f0c3f200640 1 --2- 172.21.15.7:0/3248615589 >> v2:172.21.15.87:3300/0 conn(0x7f0c400d5190 0x7f0c401601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.809+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c24003520 con 0x7f0c400d4d80 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.809+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4084819607 0 0) 0x7f0c24003520 con 0x7f0c400d4d80 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.809+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 >> v2:172.21.15.87:3300/0 conn(0x7f0c400d5190 msgr2=0x7f0c401601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.809+0000 7f0c3de00640 1 --2- 172.21.15.7:0/3248615589 >> v2:172.21.15.87:3300/0 conn(0x7f0c400d5190 0x7f0c401601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.809+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 >> v1:172.21.15.87:6789/0 conn(0x7f0c400d2e90 legacy=0x7f0c401616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.809+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c400d4860 con 0x7f0c400d4d80 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.809+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0c30001670 con 0x7f0c400d4d80 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.809+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 >> v1:172.21.15.7:6789/0 conn(0x7f0c400d4d80 legacy=0x7f0c400d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.809+0000 7f0c3de00640 1 --2- 172.21.15.7:0/3248615589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 0x7f0c24003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3de00640 1 --2- 172.21.15.7:0/3248615589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c24004800 0x7f0c24004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3de00640 1 --2- 172.21.15.7:0/3248615589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c240051c0 0x7f0c2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c400d4580 con 0x7f0c240039b0 2024-04-25T11:10:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2400c160 con 0x7f0c24004800 2024-04-25T11:10:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2400c360 con 0x7f0c240051c0 2024-04-25T11:10:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3e800640 1 --2- 172.21.15.7:0/3248615589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 0x7f0c24003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3e800640 1 -- 172.21.15.7:0/3248615589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c240051c0 msgr2=0x7f0c2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3e800640 1 --2- 172.21.15.7:0/3248615589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c240051c0 0x7f0c2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3e800640 1 -- 172.21.15.7:0/3248615589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c24004800 msgr2=0x7f0c24004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3e800640 1 --2- 172.21.15.7:0/3248615589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c24004800 0x7f0c24004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3e800640 1 -- 172.21.15.7:0/3248615589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c24003520 con 0x7f0c240039b0 2024-04-25T11:10:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.810+0000 7f0c3f200640 1 -- 172.21.15.7:0/3248615589 reap_dead start 2024-04-25T11:10:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.811+0000 7f0c3e800640 1 --2- 172.21.15.7:0/3248615589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 0x7f0c24003e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000c240 tx=0x7f0c24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df832fd93d6b9721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.812+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30007cf0 con 0x7f0c240039b0 2024-04-25T11:10:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.812+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c30005af0 con 0x7f0c240039b0 2024-04-25T11:10:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.812+0000 7f0c3de00640 1 -- 172.21.15.7:0/3248615589 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30007070 con 0x7f0c240039b0 2024-04-25T11:10:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.812+0000 7f0c44a00640 1 -- 172.21.15.7:0/3248615589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 msgr2=0x7f0c24003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.812+0000 7f0c44a00640 1 --2- 172.21.15.7:0/3248615589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 0x7f0c24003e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000c240 tx=0x7f0c24005830 comp rx=0 tx=0).stop 2024-04-25T11:10:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.812+0000 7f0c44a00640 1 -- 172.21.15.7:0/3248615589 shutdown_connections 2024-04-25T11:10:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.812+0000 7f0c44a00640 1 --2- 172.21.15.7:0/3248615589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 0x7f0c24003e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.812+0000 7f0c44a00640 1 -- 172.21.15.7:0/3248615589 >> 172.21.15.7:0/3248615589 conn(0x7f0c4006c750 msgr2=0x7f0c4006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.813+0000 7f0c44a00640 1 -- 172.21.15.7:0/3248615589 shutdown_connections 2024-04-25T11:10:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.813+0000 7f0c44a00640 1 -- 172.21.15.7:0/3248615589 wait complete. 2024-04-25T11:10:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.815+0000 7f0c44a00640 1 Processor -- start 2024-04-25T11:10:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.815+0000 7f0c44a00640 1 -- start start 2024-04-25T11:10:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c44a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 0x7f0c400c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c44a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c401e6300 0x7f0c401499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c44a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c40149f00 0x7f0c401f9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c44a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c400d2900 con 0x7f0c240039b0 2024-04-25T11:10:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c44a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0c400d2780 con 0x7f0c401e6300 2024-04-25T11:10:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c44a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c400d2a80 con 0x7f0c40149f00 2024-04-25T11:10:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c3e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 0x7f0c400c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c3e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 0x7f0c400c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48930/0 (socket says 172.21.15.7:48930) 2024-04-25T11:10:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c3e800640 1 -- 172.21.15.7:0/2749897281 learned_addr learned my addr 172.21.15.7:0/2749897281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c3f200640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c40149f00 0x7f0c401f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:24.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c3e800640 1 -- 172.21.15.7:0/2749897281 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c40149f00 msgr2=0x7f0c401f9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c3e800640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c40149f00 0x7f0c401f9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c37e00640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c401e6300 0x7f0c401499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.817+0000 7f0c3e800640 1 -- 172.21.15.7:0/2749897281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c401e6300 msgr2=0x7f0c401499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.818+0000 7f0c3e800640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c401e6300 0x7f0c401499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.818+0000 7f0c3e800640 1 -- 172.21.15.7:0/2749897281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c400b66b0 con 0x7f0c240039b0 2024-04-25T11:10:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.818+0000 7f0c3f200640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c40149f00 0x7f0c401f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:24.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.818+0000 7f0c3e800640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 0x7f0c400c8500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000c2a0 tx=0x7f0c401614e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:24.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.818+0000 7f0c36a00640 1 -- 172.21.15.7:0/2749897281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30040030 con 0x7f0c240039b0 2024-04-25T11:10:24.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.818+0000 7f0c36a00640 1 -- 172.21.15.7:0/2749897281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c30001430 con 0x7f0c240039b0 2024-04-25T11:10:24.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.818+0000 7f0c36a00640 1 -- 172.21.15.7:0/2749897281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30037040 con 0x7f0c240039b0 2024-04-25T11:10:24.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.818+0000 7f0c44a00640 1 -- 172.21.15.7:0/2749897281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4014fd80 con 0x7f0c240039b0 2024-04-25T11:10:24.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.818+0000 7f0c44a00640 1 -- 172.21.15.7:0/2749897281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c400b6c30 con 0x7f0c240039b0 2024-04-25T11:10:24.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.820+0000 7f0c44a00640 1 -- 172.21.15.7:0/2749897281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4006d050 con 0x7f0c240039b0 2024-04-25T11:10:24.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.821+0000 7f0c36a00640 1 -- 172.21.15.7:0/2749897281 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0c30043070 con 0x7f0c240039b0 2024-04-25T11:10:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.822+0000 7f0c36a00640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c1c0744b0 0x7f0c1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.822+0000 7f0c36a00640 1 -- 172.21.15.7:0/2749897281 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6007+0+0 (secure 0 0 0) 0x7f0c300b5230 con 0x7f0c240039b0 2024-04-25T11:10:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.822+0000 7f0c37e00640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c1c0744b0 0x7f0c1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:24.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.823+0000 7f0c37e00640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c1c0744b0 0x7f0c1c0769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c400d32d0 tx=0x7f0c28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:24.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:24.827+0000 7f0c36a00640 1 -- 172.21.15.7:0/2749897281 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c4006d050 con 0x7f0c240039b0 2024-04-25T11:10:25.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:25.035+0000 7f0c44a00640 1 -- 172.21.15.7:0/2749897281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0c40050740 con 0x7f0c240039b0 2024-04-25T11:10:26.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.502+0000 7f0c36a00640 1 -- 172.21.15.7:0/2749897281 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) ==== 228+0+0 (secure 0 0 0) 0x7f0c3007f800 con 0x7f0c240039b0 2024-04-25T11:10:26.503 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' created 2024-04-25T11:10:26.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.508+0000 7f0c44a00640 1 -- 172.21.15.7:0/2749897281 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c1c0744b0 msgr2=0x7f0c1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:26.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.508+0000 7f0c44a00640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c1c0744b0 0x7f0c1c0769c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0c400d32d0 tx=0x7f0c28009040 comp rx=0 tx=0).stop 2024-04-25T11:10:26.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.508+0000 7f0c44a00640 1 -- 172.21.15.7:0/2749897281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 msgr2=0x7f0c400c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:26.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.508+0000 7f0c44a00640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 0x7f0c400c8500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000c2a0 tx=0x7f0c401614e0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.509+0000 7f0c44a00640 1 -- 172.21.15.7:0/2749897281 shutdown_connections 2024-04-25T11:10:26.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.509+0000 7f0c44a00640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c1c0744b0 0x7f0c1c0769c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.509+0000 7f0c44a00640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c40149f00 0x7f0c401f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.509+0000 7f0c44a00640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c401e6300 0x7f0c401499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.509+0000 7f0c44a00640 1 --2- 172.21.15.7:0/2749897281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c240039b0 0x7f0c400c8500 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.509+0000 7f0c44a00640 1 -- 172.21.15.7:0/2749897281 >> 172.21.15.7:0/2749897281 conn(0x7f0c4006c750 msgr2=0x7f0c400ca140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:26.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.509+0000 7f0c44a00640 1 -- 172.21.15.7:0/2749897281 shutdown_connections 2024-04-25T11:10:26.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.509+0000 7f0c44a00640 1 -- 172.21.15.7:0/2749897281 wait complete. 2024-04-25T11:10:26.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T11:10:26.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.808+0000 7f0168e00640 1 Processor -- start 2024-04-25T11:10:26.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.808+0000 7f0168e00640 1 -- start start 2024-04-25T11:10:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.810+0000 7f0168e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0164070110 0x7f016415c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.810+0000 7f0168e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f016406d7c0 0x7f0164070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.810+0000 7f0168e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f016406dbd0 0x7f01640ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.810+0000 7f0168e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f01640b5550 con 0x7f016406d7c0 2024-04-25T11:10:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.810+0000 7f0168e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f01640b4940 con 0x7f016406dbd0 2024-04-25T11:10:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.810+0000 7f0168e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f01640b4c20 con 0x7f0164070110 2024-04-25T11:10:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.811+0000 7f0161400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f016406d7c0 0x7f0164070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.811+0000 7f0161400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f016406d7c0 0x7f0164070f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48942/0 (socket says 172.21.15.7:48942) 2024-04-25T11:10:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.811+0000 7f0161400640 1 -- 172.21.15.7:0/2724978909 learned_addr learned my addr 172.21.15.7:0/2724978909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.811+0000 7f0161e00640 1 --2- 172.21.15.7:0/2724978909 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0164070110 0x7f016415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:26.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.811+0000 7f0162800640 1 --2- 172.21.15.7:0/2724978909 >> v2:172.21.15.87:3300/0 conn(0x7f016406dbd0 0x7f01640ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:26.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.811+0000 7f0161400640 1 -- 172.21.15.7:0/2724978909 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0164070110 msgr2=0x7f016415c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:26.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.811+0000 7f0161400640 1 --2- 172.21.15.7:0/2724978909 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0164070110 0x7f016415c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.811+0000 7f0161400640 1 -- 172.21.15.7:0/2724978909 >> v2:172.21.15.87:3300/0 conn(0x7f016406dbd0 msgr2=0x7f01640ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:26.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.811+0000 7f0161400640 1 --2- 172.21.15.7:0/2724978909 >> v2:172.21.15.87:3300/0 conn(0x7f016406dbd0 0x7f01640ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.811+0000 7f0161400640 1 -- 172.21.15.7:0/2724978909 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01640b4f00 con 0x7f016406d7c0 2024-04-25T11:10:26.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.811+0000 7f0161e00640 1 --2- 172.21.15.7:0/2724978909 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0164070110 0x7f016415c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:26.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.813+0000 7f0161400640 1 --2- 172.21.15.7:0/2724978909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016406d7c0 0x7f0164070f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f01640b54c0 tx=0x7f014c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bdc3a08bf609f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:26.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.813+0000 7f0160a00640 1 -- 172.21.15.7:0/2724978909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c036070 con 0x7f016406d7c0 2024-04-25T11:10:26.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.813+0000 7f0160a00640 1 -- 172.21.15.7:0/2724978909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014c031600 con 0x7f016406d7c0 2024-04-25T11:10:26.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.813+0000 7f0160a00640 1 -- 172.21.15.7:0/2724978909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0319d0 con 0x7f016406d7c0 2024-04-25T11:10:26.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.814+0000 7f0168e00640 1 -- 172.21.15.7:0/2724978909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016406d7c0 msgr2=0x7f0164070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:26.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.814+0000 7f0168e00640 1 --2- 172.21.15.7:0/2724978909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016406d7c0 0x7f0164070f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f01640b54c0 tx=0x7f014c02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.814+0000 7f0168e00640 1 -- 172.21.15.7:0/2724978909 shutdown_connections 2024-04-25T11:10:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.814+0000 7f0168e00640 1 --2- 172.21.15.7:0/2724978909 >> v2:172.21.15.87:3300/0 conn(0x7f016406dbd0 0x7f01640ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.814+0000 7f0168e00640 1 --2- 172.21.15.7:0/2724978909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016406d7c0 0x7f0164070f50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.814+0000 7f0168e00640 1 --2- 172.21.15.7:0/2724978909 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0164070110 0x7f016415c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.814+0000 7f0168e00640 1 -- 172.21.15.7:0/2724978909 >> 172.21.15.7:0/2724978909 conn(0x7f0164073920 msgr2=0x7f0164073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.814+0000 7f0168e00640 1 -- 172.21.15.7:0/2724978909 shutdown_connections 2024-04-25T11:10:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.814+0000 7f0168e00640 1 -- 172.21.15.7:0/2724978909 wait complete. 2024-04-25T11:10:26.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.817+0000 7f0168e00640 1 Processor -- start 2024-04-25T11:10:26.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.817+0000 7f0168e00640 1 -- start start 2024-04-25T11:10:26.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.818+0000 7f0168e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016406d7c0 0x7f0164151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:26.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.818+0000 7f0168e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016406dbd0 0x7f01641f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:26.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.818+0000 7f0168e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0164070110 0x7f016415e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:26.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.818+0000 7f0168e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0164071610 con 0x7f016406d7c0 2024-04-25T11:10:26.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.818+0000 7f0168e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0164071490 con 0x7f016406dbd0 2024-04-25T11:10:26.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.818+0000 7f0168e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f01640ce6d0 con 0x7f0164070110 2024-04-25T11:10:26.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.819+0000 7f0161e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016406d7c0 0x7f0164151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:26.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.819+0000 7f0161e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016406d7c0 0x7f0164151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48950/0 (socket says 172.21.15.7:48950) 2024-04-25T11:10:26.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.819+0000 7f0161e00640 1 -- 172.21.15.7:0/2144057317 learned_addr learned my addr 172.21.15.7:0/2144057317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:26.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.819+0000 7f0162800640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0164070110 0x7f016415e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.819+0000 7f0161400640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016406dbd0 0x7f01641f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.819+0000 7f0161e00640 1 -- 172.21.15.7:0/2144057317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0164070110 msgr2=0x7f016415e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.819+0000 7f0161e00640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0164070110 0x7f016415e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.819+0000 7f0161e00640 1 -- 172.21.15.7:0/2144057317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016406dbd0 msgr2=0x7f01641f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.819+0000 7f0161e00640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016406dbd0 0x7f01641f7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.819+0000 7f0161e00640 1 -- 172.21.15.7:0/2144057317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01641f82d0 con 0x7f016406d7c0 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.819+0000 7f0161400640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016406dbd0 0x7f01641f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.820+0000 7f0161e00640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016406d7c0 0x7f0164151680 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f015400d800 tx=0x7f015400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.820+0000 7f015a000640 1 -- 172.21.15.7:0/2144057317 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154004030 con 0x7f016406d7c0 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.820+0000 7f015a000640 1 -- 172.21.15.7:0/2144057317 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01540041d0 con 0x7f016406d7c0 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.820+0000 7f015a000640 1 -- 172.21.15.7:0/2144057317 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015400b6c0 con 0x7f016406d7c0 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.820+0000 7f0168e00640 1 -- 172.21.15.7:0/2144057317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01641f85b0 con 0x7f016406d7c0 2024-04-25T11:10:26.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.820+0000 7f0168e00640 1 -- 172.21.15.7:0/2144057317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016415dfb0 con 0x7f016406d7c0 2024-04-25T11:10:26.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.821+0000 7f0168e00640 1 -- 172.21.15.7:0/2144057317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016415e210 con 0x7f016406d7c0 2024-04-25T11:10:26.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.822+0000 7f015a000640 1 -- 172.21.15.7:0/2144057317 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f015400b860 con 0x7f016406d7c0 2024-04-25T11:10:26.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.824+0000 7f015a000640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0140074500 0x7f0140076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:26.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.824+0000 7f015a000640 1 -- 172.21.15.7:0/2144057317 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f0154095760 con 0x7f016406d7c0 2024-04-25T11:10:26.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.824+0000 7f0161400640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0140074500 0x7f0140076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:26.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.825+0000 7f0161400640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0140074500 0x7f0140076a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f014c033000 tx=0x7f014c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:26.830+0000 7f015a000640 1 -- 172.21.15.7:0/2144057317 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f015405fcb0 con 0x7f016406d7c0 2024-04-25T11:10:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.035+0000 7f0168e00640 1 -- 172.21.15.7:0/2144057317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f016406a100 con 0x7f016406d7c0 2024-04-25T11:10:27.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.507+0000 7f015a000640 1 -- 172.21.15.7:0/2144057317 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) ==== 169+0+0 (secure 0 0 0) 0x7f015405f5b0 con 0x7f016406d7c0 2024-04-25T11:10:27.508 INFO:teuthology.orchestra.run.smithi007.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T11:10:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.512+0000 7f0168e00640 1 -- 172.21.15.7:0/2144057317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0140074500 msgr2=0x7f0140076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.512+0000 7f0168e00640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0140074500 0x7f0140076a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f014c033000 tx=0x7f014c009000 comp rx=0 tx=0).stop 2024-04-25T11:10:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.512+0000 7f0168e00640 1 -- 172.21.15.7:0/2144057317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016406d7c0 msgr2=0x7f0164151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:27.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.512+0000 7f0168e00640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016406d7c0 0x7f0164151680 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f015400d800 tx=0x7f015400dcc0 comp rx=0 tx=0).stop 2024-04-25T11:10:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.513+0000 7f0168e00640 1 -- 172.21.15.7:0/2144057317 shutdown_connections 2024-04-25T11:10:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.513+0000 7f0168e00640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0140074500 0x7f0140076a10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.513+0000 7f0168e00640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0164070110 0x7f016415e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.513+0000 7f0168e00640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016406dbd0 0x7f01641f7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.513+0000 7f0168e00640 1 --2- 172.21.15.7:0/2144057317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016406d7c0 0x7f0164151680 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:27.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.513+0000 7f0168e00640 1 -- 172.21.15.7:0/2144057317 >> 172.21.15.7:0/2144057317 conn(0x7f0164073920 msgr2=0x7f01640b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.514+0000 7f0168e00640 1 -- 172.21.15.7:0/2144057317 shutdown_connections 2024-04-25T11:10:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.514+0000 7f0168e00640 1 -- 172.21.15.7:0/2144057317 wait complete. 2024-04-25T11:10:27.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T11:10:27.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.822+0000 7f8bdf200640 1 Processor -- start 2024-04-25T11:10:27.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.822+0000 7f8bdf200640 1 -- start start 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.824+0000 7f8bdf200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8be00d0fb0 0x7f8be015fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.824+0000 7f8bdf200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b7af0 0x7f8be00d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.824+0000 7f8bdf200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be00d13c0 con 0x7f8be00b8e50 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.824+0000 7f8bdf200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8be0163f10 con 0x7f8be00d0fb0 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.824+0000 7f8bdf200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8be01641f0 con 0x7f8be00b7af0 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.824+0000 7f8bdde00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8be00d0fb0 0x7f8be015fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.824+0000 7f8bdd400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8be00b8e50 0x7f8be00b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41476/0 (socket says 172.21.15.7:41476) 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.824+0000 7f8bdd400640 1 -- 172.21.15.7:0/1030215981 learned_addr learned my addr 172.21.15.7:0/1030215981 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.825+0000 7f8bde800640 1 --2- 172.21.15.7:0/1030215981 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b7af0 0x7f8be00d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.825+0000 7f8bdca00640 1 -- 172.21.15.7:0/1030215981 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2883222178 0 0) 0x7f8be00d13c0 con 0x7f8be00b8e50 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.825+0000 7f8bdca00640 1 -- 172.21.15.7:0/1030215981 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bc4003520 con 0x7f8be00b8e50 2024-04-25T11:10:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.825+0000 7f8bdde00640 1 -- 172.21.15.7:0/1030215981 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b7af0 msgr2=0x7f8be00d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.825+0000 7f8bdde00640 1 --2- 172.21.15.7:0/1030215981 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b7af0 0x7f8be00d45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.825+0000 7f8bdde00640 1 -- 172.21.15.7:0/1030215981 >> v1:172.21.15.7:6789/0 conn(0x7f8be00b8e50 legacy=0x7f8be00b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.825+0000 7f8bdde00640 1 -- 172.21.15.7:0/1030215981 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be01644d0 con 0x7f8be00d0fb0 2024-04-25T11:10:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.826+0000 7f8bdde00640 1 --2- 172.21.15.7:0/1030215981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be00d0fb0 0x7f8be015fea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00097a0 tx=0x7f8bd002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a3ebf55b23eaef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.826+0000 7f8bdca00640 1 -- 172.21.15.7:0/1030215981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0037070 con 0x7f8be00d0fb0 2024-04-25T11:10:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.826+0000 7f8bdca00640 1 -- 172.21.15.7:0/1030215981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd002ae50 con 0x7f8be00d0fb0 2024-04-25T11:10:27.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.826+0000 7f8bdca00640 1 -- 172.21.15.7:0/1030215981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0032660 con 0x7f8be00d0fb0 2024-04-25T11:10:27.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.827+0000 7f8bdf200640 1 -- 172.21.15.7:0/1030215981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be00d0fb0 msgr2=0x7f8be015fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:27.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.827+0000 7f8bdf200640 1 --2- 172.21.15.7:0/1030215981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be00d0fb0 0x7f8be015fea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00097a0 tx=0x7f8bd002a100 comp rx=0 tx=0).stop 2024-04-25T11:10:27.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.827+0000 7f8bdf200640 1 -- 172.21.15.7:0/1030215981 shutdown_connections 2024-04-25T11:10:27.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.827+0000 7f8bdf200640 1 --2- 172.21.15.7:0/1030215981 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b7af0 0x7f8be00d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:27.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.827+0000 7f8bdf200640 1 --2- 172.21.15.7:0/1030215981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be00d0fb0 0x7f8be015fea0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:27.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.827+0000 7f8bdf200640 1 -- 172.21.15.7:0/1030215981 >> 172.21.15.7:0/1030215981 conn(0x7f8be00d23e0 msgr2=0x7f8be00d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:27.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.827+0000 7f8bdf200640 1 -- 172.21.15.7:0/1030215981 shutdown_connections 2024-04-25T11:10:27.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.827+0000 7f8bdf200640 1 -- 172.21.15.7:0/1030215981 wait complete. 2024-04-25T11:10:27.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.830+0000 7f8bdf200640 1 Processor -- start 2024-04-25T11:10:27.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.830+0000 7f8bdf200640 1 -- start start 2024-04-25T11:10:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.831+0000 7f8bdf200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be00b8e50 0x7f8be01f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.831+0000 7f8bdf200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00d0fb0 0x7f8be00ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.831+0000 7f8bdf200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be014fe50 0x7f8be0150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.831+0000 7f8bdf200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00b9260 con 0x7f8be014fe50 2024-04-25T11:10:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.831+0000 7f8bdf200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00b4bc0 con 0x7f8be00b8e50 2024-04-25T11:10:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.831+0000 7f8bdf200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8be0163a80 con 0x7f8be00d0fb0 2024-04-25T11:10:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bde800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be014fe50 0x7f8be0150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bde800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be014fe50 0x7f8be0150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48978/0 (socket says 172.21.15.7:48978) 2024-04-25T11:10:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bde800640 1 -- 172.21.15.7:0/1498780407 learned_addr learned my addr 172.21.15.7:0/1498780407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bdde00640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be00b8e50 0x7f8be01f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bdd400640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00d0fb0 0x7f8be00ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bde800640 1 -- 172.21.15.7:0/1498780407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00d0fb0 msgr2=0x7f8be00ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bde800640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00d0fb0 0x7f8be00ac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bde800640 1 -- 172.21.15.7:0/1498780407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be00b8e50 msgr2=0x7f8be01f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bde800640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be00b8e50 0x7f8be01f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bde800640 1 -- 172.21.15.7:0/1498780407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be014edb0 con 0x7f8be014fe50 2024-04-25T11:10:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bdde00640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be00b8e50 0x7f8be01f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:27.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.832+0000 7f8bde800640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be014fe50 0x7f8be0150260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8007d80 tx=0x7f8bd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:27.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.833+0000 7f8bd6000640 1 -- 172.21.15.7:0/1498780407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8014420 con 0x7f8be014fe50 2024-04-25T11:10:27.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.833+0000 7f8bd6000640 1 -- 172.21.15.7:0/1498780407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd8019070 con 0x7f8be014fe50 2024-04-25T11:10:27.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.833+0000 7f8bd6000640 1 -- 172.21.15.7:0/1498780407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8014800 con 0x7f8be014fe50 2024-04-25T11:10:27.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.833+0000 7f8bdf200640 1 -- 172.21.15.7:0/1498780407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be014f090 con 0x7f8be014fe50 2024-04-25T11:10:27.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.833+0000 7f8bdf200640 1 -- 172.21.15.7:0/1498780407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be014f650 con 0x7f8be014fe50 2024-04-25T11:10:27.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.834+0000 7f8bdf200640 1 -- 172.21.15.7:0/1498780407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be0050740 con 0x7f8be014fe50 2024-04-25T11:10:27.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.835+0000 7f8bd6000640 1 -- 172.21.15.7:0/1498780407 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8bd80149a0 con 0x7f8be014fe50 2024-04-25T11:10:27.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.836+0000 7f8bd6000640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc00744e0 0x7f8bc00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:27.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.837+0000 7f8bd6000640 1 -- 172.21.15.7:0/1498780407 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7f8bd8095eb0 con 0x7f8be014fe50 2024-04-25T11:10:27.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.837+0000 7f8bdde00640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc00744e0 0x7f8bc00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:27.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.837+0000 7f8bdde00640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc00744e0 0x7f8bc00769f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0034000 tx=0x7f8bd002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:27.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:27.841+0000 7f8bd6000640 1 -- 172.21.15.7:0/1498780407 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8bd8060280 con 0x7f8be014fe50 2024-04-25T11:10:28.010 INFO:tasks.ceph.mds.d.smithi087.stderr:2024-04-25T11:10:28.009+0000 7fa8a3000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:10:28.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.083+0000 7f8bdf200640 1 -- 172.21.15.7:0/1498780407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f8be00d3d40 con 0x7f8be014fe50 2024-04-25T11:10:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.540+0000 7f8bd6000640 1 -- 172.21.15.7:0/1498780407 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f8bd805fb80 con 0x7f8be014fe50 2024-04-25T11:10:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:added data pool 4 to fsmap 2024-04-25T11:10:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 -- 172.21.15.7:0/1498780407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc00744e0 msgr2=0x7f8bc00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc00744e0 0x7f8bc00769f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0034000 tx=0x7f8bd002e000 comp rx=0 tx=0).stop 2024-04-25T11:10:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 -- 172.21.15.7:0/1498780407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be014fe50 msgr2=0x7f8be0150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be014fe50 0x7f8be0150260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8007d80 tx=0x7f8bd800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:10:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 -- 172.21.15.7:0/1498780407 shutdown_connections 2024-04-25T11:10:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc00744e0 0x7f8bc00769f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be014fe50 0x7f8be0150260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00d0fb0 0x7f8be00ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 --2- 172.21.15.7:0/1498780407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be00b8e50 0x7f8be01f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 -- 172.21.15.7:0/1498780407 >> 172.21.15.7:0/1498780407 conn(0x7f8be00d23e0 msgr2=0x7f8be00d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 -- 172.21.15.7:0/1498780407 shutdown_connections 2024-04-25T11:10:28.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.544+0000 7f8bdf200640 1 -- 172.21.15.7:0/1498780407 wait complete. 2024-04-25T11:10:28.571 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:10:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.848+0000 7f342fa00640 1 Processor -- start 2024-04-25T11:10:28.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.849+0000 7f342fa00640 1 -- start start 2024-04-25T11:10:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.851+0000 7f342fa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f342806dc90 0x7f34280afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.851+0000 7f342fa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f342806f300 0x7f34280b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.851+0000 7f342fa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34280b4980 con 0x7f3428160a30 2024-04-25T11:10:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.851+0000 7f342fa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f34280b4c60 con 0x7f342806f300 2024-04-25T11:10:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.851+0000 7f342fa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f34280b4f40 con 0x7f342806dc90 2024-04-25T11:10:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.852+0000 7f342d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f342806dc90 0x7f34280afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.852+0000 7f342d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f342806dc90 0x7f34280afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:59842/0 (socket says 172.21.15.7:59842) 2024-04-25T11:10:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.852+0000 7f342d000640 1 -- 172.21.15.7:0/1024272719 learned_addr learned my addr 172.21.15.7:0/1024272719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:28.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.852+0000 7f3427e00640 1 --2- 172.21.15.7:0/1024272719 >> v2:172.21.15.87:3300/0 conn(0x7f342806f300 0x7f34280b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:28.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.853+0000 7f342d000640 1 -- 172.21.15.7:0/1024272719 >> v2:172.21.15.87:3300/0 conn(0x7f342806f300 msgr2=0x7f34280b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:28.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.853+0000 7f342d000640 1 --2- 172.21.15.7:0/1024272719 >> v2:172.21.15.87:3300/0 conn(0x7f342806f300 0x7f34280b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:28.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.853+0000 7f342d000640 1 -- 172.21.15.7:0/1024272719 >> v1:172.21.15.87:6789/0 conn(0x7f3428160a30 legacy=0x7f34280ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:28.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.853+0000 7f342d000640 1 -- 172.21.15.7:0/1024272719 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34280b5220 con 0x7f342806dc90 2024-04-25T11:10:28.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.854+0000 7f342d000640 1 --2- 172.21.15.7:0/1024272719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f342806dc90 0x7f34280afb80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f341c009870 tx=0x7f341c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51c5507a9b394d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.854+0000 7f3427400640 1 -- 172.21.15.7:0/1024272719 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c037070 con 0x7f342806dc90 2024-04-25T11:10:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.855+0000 7f3427400640 1 -- 172.21.15.7:0/1024272719 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341c032440 con 0x7f342806dc90 2024-04-25T11:10:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.855+0000 7f3427400640 1 -- 172.21.15.7:0/1024272719 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c032810 con 0x7f342806dc90 2024-04-25T11:10:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.855+0000 7f342fa00640 1 -- 172.21.15.7:0/1024272719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f342806dc90 msgr2=0x7f34280afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.855+0000 7f342fa00640 1 --2- 172.21.15.7:0/1024272719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f342806dc90 0x7f34280afb80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f341c009870 tx=0x7f341c02a250 comp rx=0 tx=0).stop 2024-04-25T11:10:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.855+0000 7f342fa00640 1 -- 172.21.15.7:0/1024272719 shutdown_connections 2024-04-25T11:10:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.855+0000 7f342fa00640 1 --2- 172.21.15.7:0/1024272719 >> v2:172.21.15.87:3300/0 conn(0x7f342806f300 0x7f34280b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:28.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.855+0000 7f342fa00640 1 --2- 172.21.15.7:0/1024272719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f342806dc90 0x7f34280afb80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:28.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.855+0000 7f342fa00640 1 -- 172.21.15.7:0/1024272719 >> 172.21.15.7:0/1024272719 conn(0x7f3428074200 msgr2=0x7f342806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:28.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.855+0000 7f342fa00640 1 -- 172.21.15.7:0/1024272719 shutdown_connections 2024-04-25T11:10:28.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.856+0000 7f342fa00640 1 -- 172.21.15.7:0/1024272719 wait complete. 2024-04-25T11:10:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.858+0000 7f342fa00640 1 Processor -- start 2024-04-25T11:10:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.858+0000 7f342fa00640 1 -- start start 2024-04-25T11:10:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.859+0000 7f342fa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f342806f300 0x7f342815e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.859+0000 7f342fa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3428160a30 0x7f34281f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342fa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34281e6ab0 0x7f3428148a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342fa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34280b6a30 con 0x7f342806f300 2024-04-25T11:10:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342fa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f342815d220 con 0x7f34281e6ab0 2024-04-25T11:10:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342fa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f34280ce920 con 0x7f3428160a30 2024-04-25T11:10:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f342806f300 0x7f342815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f342806f300 0x7f342815e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48980/0 (socket says 172.21.15.7:48980) 2024-04-25T11:10:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342d000640 1 -- 172.21.15.7:0/4008863485 learned_addr learned my addr 172.21.15.7:0/4008863485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f3427e00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3428160a30 0x7f34281f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342d000640 1 -- 172.21.15.7:0/4008863485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3428160a30 msgr2=0x7f34281f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342d000640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3428160a30 0x7f34281f5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342d000640 1 -- 172.21.15.7:0/4008863485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34281e6ab0 msgr2=0x7f3428148a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342da00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34281e6ab0 0x7f3428148a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342d000640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34281e6ab0 0x7f3428148a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342d000640 1 -- 172.21.15.7:0/4008863485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342815f190 con 0x7f342806f300 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.860+0000 7f342da00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34281e6ab0 0x7f3428148a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.861+0000 7f342d000640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f342806f300 0x7f342815e970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f341c009870 tx=0x7f341c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.861+0000 7f3424c00640 1 -- 172.21.15.7:0/4008863485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c041070 con 0x7f342806f300 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.861+0000 7f342fa00640 1 -- 172.21.15.7:0/4008863485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342815f470 con 0x7f342806f300 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.861+0000 7f3424c00640 1 -- 172.21.15.7:0/4008863485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341c03c870 con 0x7f342806f300 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.861+0000 7f3424c00640 1 -- 172.21.15.7:0/4008863485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c037040 con 0x7f342806f300 2024-04-25T11:10:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.861+0000 7f342fa00640 1 -- 172.21.15.7:0/4008863485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342815eeb0 con 0x7f342806f300 2024-04-25T11:10:28.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.863+0000 7f342fa00640 1 -- 172.21.15.7:0/4008863485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342815f990 con 0x7f342806f300 2024-04-25T11:10:28.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.864+0000 7f3424c00640 1 -- 172.21.15.7:0/4008863485 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f341c044070 con 0x7f342806f300 2024-04-25T11:10:28.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.865+0000 7f3424c00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f34080744b0 0x7f34080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:28.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.865+0000 7f3424c00640 1 -- 172.21.15.7:0/4008863485 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f341c0b5f90 con 0x7f342806f300 2024-04-25T11:10:28.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.865+0000 7f3427e00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f34080744b0 0x7f34080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:28.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.866+0000 7f3427e00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f34080744b0 0x7f34080769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f342815c3f0 tx=0x7f3414008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:28.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:28.870+0000 7f3424c00640 1 -- 172.21.15.7:0/4008863485 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f341c0803d0 con 0x7f342806f300 2024-04-25T11:10:29.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.119+0000 7f342fa00640 1 -- 172.21.15.7:0/4008863485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3428146a30 con 0x7f342806f300 2024-04-25T11:10:29.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.120+0000 7f3424c00640 1 -- 172.21.15.7:0/4008863485 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5168 (secure 0 0 0) 0x7f341c07fcd0 con 0x7f342806f300 2024-04-25T11:10:29.121 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:29.121 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3458223722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3458223722},{"type":"v1","addr":"172.21.15.87:6835","nonce":3458223722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1383067804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1383067804},{"type":"v1","addr":"172.21.15.7:6835","nonce":1383067804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/435824250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":435824250},{"type":"v1","addr":"172.21.15.7:6837","nonce":435824250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:10:23.023233+0000","modified":"2024-04-25T11:10:28.085152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1091275258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1091275258},{"type":"v1","addr":"172.21.15.87:6834","nonce":1091275258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:10:29.121 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 7 2024-04-25T11:10:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.123+0000 7f342fa00640 1 -- 172.21.15.7:0/4008863485 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f34080744b0 msgr2=0x7f34080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.123+0000 7f342fa00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f34080744b0 0x7f34080769c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f342815c3f0 tx=0x7f3414008040 comp rx=0 tx=0).stop 2024-04-25T11:10:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.123+0000 7f342fa00640 1 -- 172.21.15.7:0/4008863485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f342806f300 msgr2=0x7f342815e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.123+0000 7f342fa00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f342806f300 0x7f342815e970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f341c009870 tx=0x7f341c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.123+0000 7f342fa00640 1 -- 172.21.15.7:0/4008863485 shutdown_connections 2024-04-25T11:10:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.123+0000 7f342fa00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f34080744b0 0x7f34080769c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.123+0000 7f342fa00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34281e6ab0 0x7f3428148a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.123+0000 7f342fa00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3428160a30 0x7f34281f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.123+0000 7f342fa00640 1 --2- 172.21.15.7:0/4008863485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f342806f300 0x7f342815e970 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.123+0000 7f342fa00640 1 -- 172.21.15.7:0/4008863485 >> 172.21.15.7:0/4008863485 conn(0x7f3428074200 msgr2=0x7f34280af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:29.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.124+0000 7f342fa00640 1 -- 172.21.15.7:0/4008863485 shutdown_connections 2024-04-25T11:10:29.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.124+0000 7f342fa00640 1 -- 172.21.15.7:0/4008863485 wait complete. 2024-04-25T11:10:29.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:29.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.435+0000 7f71d2800640 1 Processor -- start 2024-04-25T11:10:29.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.435+0000 7f71d2800640 1 -- start start 2024-04-25T11:10:29.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.437+0000 7f71d2800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71cc0aed40 0x7f71cc15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:29.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.437+0000 7f71d2800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71cc0b5220 0x7f71cc0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:29.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.437+0000 7f71d2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f71cc0af420 0x7f71cc0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:29.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.437+0000 7f71d2800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f71cc15e1f0 con 0x7f71cc0aed40 2024-04-25T11:10:29.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.437+0000 7f71d2800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f71cc0afd70 con 0x7f71cc0af420 2024-04-25T11:10:29.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.437+0000 7f71d2800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f71cc0b64b0 con 0x7f71cc0b5220 2024-04-25T11:10:29.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d1400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71cc0aed40 0x7f71cc15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:29.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d1400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71cc0aed40 0x7f71cc15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48990/0 (socket says 172.21.15.7:48990) 2024-04-25T11:10:29.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d1400640 1 -- 172.21.15.7:0/1184857873 learned_addr learned my addr 172.21.15.7:0/1184857873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:29.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d1400640 1 -- 172.21.15.7:0/1184857873 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71cc0b5220 msgr2=0x7f71cc0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:29.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d0a00640 1 --2- 172.21.15.7:0/1184857873 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71cc0b5220 0x7f71cc0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:29.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d1e00640 1 --2- 172.21.15.7:0/1184857873 >> v2:172.21.15.87:3300/0 conn(0x7f71cc0af420 0x7f71cc0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:29.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d1400640 1 --2- 172.21.15.7:0/1184857873 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71cc0b5220 0x7f71cc0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d1400640 1 -- 172.21.15.7:0/1184857873 >> v2:172.21.15.87:3300/0 conn(0x7f71cc0af420 msgr2=0x7f71cc0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:29.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d1400640 1 --2- 172.21.15.7:0/1184857873 >> v2:172.21.15.87:3300/0 conn(0x7f71cc0af420 0x7f71cc0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d1400640 1 -- 172.21.15.7:0/1184857873 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc06c610 con 0x7f71cc0aed40 2024-04-25T11:10:29.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d0a00640 1 --2- 172.21.15.7:0/1184857873 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71cc0b5220 0x7f71cc0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:29.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.438+0000 7f71d1e00640 1 --2- 172.21.15.7:0/1184857873 >> v2:172.21.15.87:3300/0 conn(0x7f71cc0af420 0x7f71cc0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:29.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.440+0000 7f71d1400640 1 --2- 172.21.15.7:0/1184857873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc0aed40 0x7f71cc15dcb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71cc073080 tx=0x7f71c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed321fbf36e770 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:29.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.440+0000 7f71cbe00640 1 -- 172.21.15.7:0/1184857873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0037070 con 0x7f71cc0aed40 2024-04-25T11:10:29.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.440+0000 7f71cbe00640 1 -- 172.21.15.7:0/1184857873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c0032440 con 0x7f71cc0aed40 2024-04-25T11:10:29.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.440+0000 7f71cbe00640 1 -- 172.21.15.7:0/1184857873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0032810 con 0x7f71cc0aed40 2024-04-25T11:10:29.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.441+0000 7f71d2800640 1 -- 172.21.15.7:0/1184857873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc0aed40 msgr2=0x7f71cc15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:29.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.441+0000 7f71d2800640 1 --2- 172.21.15.7:0/1184857873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc0aed40 0x7f71cc15dcb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f71cc073080 tx=0x7f71c002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.441+0000 7f71d2800640 1 -- 172.21.15.7:0/1184857873 shutdown_connections 2024-04-25T11:10:29.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.441+0000 7f71d2800640 1 --2- 172.21.15.7:0/1184857873 >> v2:172.21.15.87:3300/0 conn(0x7f71cc0af420 0x7f71cc0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.441+0000 7f71d2800640 1 --2- 172.21.15.7:0/1184857873 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71cc0b5220 0x7f71cc0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.441+0000 7f71d2800640 1 --2- 172.21.15.7:0/1184857873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc0aed40 0x7f71cc15dcb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.441+0000 7f71d2800640 1 -- 172.21.15.7:0/1184857873 >> 172.21.15.7:0/1184857873 conn(0x7f71cc0b3820 msgr2=0x7f71cc0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:29.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.441+0000 7f71d2800640 1 -- 172.21.15.7:0/1184857873 shutdown_connections 2024-04-25T11:10:29.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.441+0000 7f71d2800640 1 -- 172.21.15.7:0/1184857873 wait complete. 2024-04-25T11:10:29.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.444+0000 7f71d2800640 1 Processor -- start 2024-04-25T11:10:29.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.444+0000 7f71d2800640 1 -- start start 2024-04-25T11:10:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.445+0000 7f71d2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc0aed40 0x7f71cc150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.445+0000 7f71d2800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71cc0af420 0x7f71cc15fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.445+0000 7f71d2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71cc0b5220 0x7f71cc149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.445+0000 7f71d2800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc06ceb0 con 0x7f71cc0aed40 2024-04-25T11:10:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.445+0000 7f71d2800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc06cbd0 con 0x7f71cc0b5220 2024-04-25T11:10:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.445+0000 7f71d2800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f71cc0b6110 con 0x7f71cc0af420 2024-04-25T11:10:29.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.446+0000 7f71d1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc0aed40 0x7f71cc150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:29.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.446+0000 7f71d1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc0aed40 0x7f71cc150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49002/0 (socket says 172.21.15.7:49002) 2024-04-25T11:10:29.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.446+0000 7f71d1400640 1 -- 172.21.15.7:0/655008833 learned_addr learned my addr 172.21.15.7:0/655008833 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:29.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.446+0000 7f71d1e00640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71cc0b5220 0x7f71cc149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:29.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.446+0000 7f71d1400640 1 -- 172.21.15.7:0/655008833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71cc0af420 msgr2=0x7f71cc15fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:29.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.446+0000 7f71d1400640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71cc0af420 0x7f71cc15fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.446+0000 7f71d1400640 1 -- 172.21.15.7:0/655008833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71cc0b5220 msgr2=0x7f71cc149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:29.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.446+0000 7f71d1400640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71cc0b5220 0x7f71cc149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.446+0000 7f71d1400640 1 -- 172.21.15.7:0/655008833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc14d5d0 con 0x7f71cc0aed40 2024-04-25T11:10:29.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.447+0000 7f71d1400640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc0aed40 0x7f71cc150150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f71c002a220 tx=0x7f71c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:29.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.447+0000 7f71c9600640 1 -- 172.21.15.7:0/655008833 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0037040 con 0x7f71cc0aed40 2024-04-25T11:10:29.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.447+0000 7f71c9600640 1 -- 172.21.15.7:0/655008833 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c0042070 con 0x7f71cc0aed40 2024-04-25T11:10:29.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.447+0000 7f71c9600640 1 -- 172.21.15.7:0/655008833 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c0030bb0 con 0x7f71cc0aed40 2024-04-25T11:10:29.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.447+0000 7f71d2800640 1 -- 172.21.15.7:0/655008833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc14d050 con 0x7f71cc0aed40 2024-04-25T11:10:29.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.447+0000 7f71d2800640 1 -- 172.21.15.7:0/655008833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc14d2f0 con 0x7f71cc0aed40 2024-04-25T11:10:29.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.448+0000 7f71d2800640 1 -- 172.21.15.7:0/655008833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71cc050740 con 0x7f71cc0aed40 2024-04-25T11:10:29.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.450+0000 7f71c9600640 1 -- 172.21.15.7:0/655008833 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f71c0047460 con 0x7f71cc0aed40 2024-04-25T11:10:29.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.450+0000 7f71c9600640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71ac0744b0 0x7f71ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:29.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.451+0000 7f71c9600640 1 -- 172.21.15.7:0/655008833 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f71c00b6200 con 0x7f71cc0aed40 2024-04-25T11:10:29.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.451+0000 7f71d0a00640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71ac0744b0 0x7f71ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:29.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.452+0000 7f71d0a00640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71ac0744b0 0x7f71ac0769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f71b8006fd0 tx=0x7f71b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:29.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.456+0000 7f71c9600640 1 -- 172.21.15.7:0/655008833 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71c0080080 con 0x7f71cc0aed40 2024-04-25T11:10:29.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.666+0000 7f71d2800640 1 -- 172.21.15.7:0/655008833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71cc0aeaf0 con 0x7f71cc0aed40 2024-04-25T11:10:29.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.669+0000 7f71c9600640 1 -- 172.21.15.7:0/655008833 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18418 (secure 0 0 0) 0x7f71c0084380 con 0x7f71cc0aed40 2024-04-25T11:10:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:29.670 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:28.084739+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.672+0000 7f71d2800640 1 -- 172.21.15.7:0/655008833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71ac0744b0 msgr2=0x7f71ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.672+0000 7f71d2800640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71ac0744b0 0x7f71ac0769c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f71b8006fd0 tx=0x7f71b8008040 comp rx=0 tx=0).stop 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.672+0000 7f71d2800640 1 -- 172.21.15.7:0/655008833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc0aed40 msgr2=0x7f71cc150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.672+0000 7f71d2800640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc0aed40 0x7f71cc150150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f71c002a220 tx=0x7f71c00046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.672+0000 7f71d2800640 1 -- 172.21.15.7:0/655008833 shutdown_connections 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.672+0000 7f71d2800640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71ac0744b0 0x7f71ac0769c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.672+0000 7f71d2800640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71cc0b5220 0x7f71cc149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.672+0000 7f71d2800640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71cc0af420 0x7f71cc15fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.672+0000 7f71d2800640 1 --2- 172.21.15.7:0/655008833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71cc0aed40 0x7f71cc150150 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.672+0000 7f71d2800640 1 -- 172.21.15.7:0/655008833 >> 172.21.15.7:0/655008833 conn(0x7f71cc0b3820 msgr2=0x7f71cc15d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.672+0000 7f71d2800640 1 -- 172.21.15.7:0/655008833 shutdown_connections 2024-04-25T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.673+0000 7f71d2800640 1 -- 172.21.15.7:0/655008833 wait complete. 2024-04-25T11:10:29.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.990+0000 7f5f56c00640 1 Processor -- start 2024-04-25T11:10:29.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.990+0000 7f5f56c00640 1 -- start start 2024-04-25T11:10:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.993+0000 7f5f56c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f5006dc90 0x7f5f500afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.993+0000 7f5f56c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f50160a30 0x7f5f500b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.993+0000 7f5f56c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f500b4980 con 0x7f5f5006f300 2024-04-25T11:10:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.993+0000 7f5f56c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5f500b4c60 con 0x7f5f50160a30 2024-04-25T11:10:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.993+0000 7f5f56c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5f500b4f40 con 0x7f5f5006dc90 2024-04-25T11:10:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.993+0000 7f5f54c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f50160a30 0x7f5f500b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.993+0000 7f5f4f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5f5006f300 0x7f5f500ae960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45242/0 (socket says 172.21.15.7:45242) 2024-04-25T11:10:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.993+0000 7f5f4f400640 1 -- 172.21.15.7:0/395960679 learned_addr learned my addr 172.21.15.7:0/395960679 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.993+0000 7f5f4fe00640 1 --2- 172.21.15.7:0/395960679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f5006dc90 0x7f5f500afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.994+0000 7f5f54c00640 1 -- 172.21.15.7:0/395960679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f5006dc90 msgr2=0x7f5f500afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.994+0000 7f5f54c00640 1 --2- 172.21.15.7:0/395960679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f5006dc90 0x7f5f500afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.994+0000 7f5f54c00640 1 -- 172.21.15.7:0/395960679 >> v1:172.21.15.7:6789/0 conn(0x7f5f5006f300 legacy=0x7f5f500ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.994+0000 7f5f54c00640 1 -- 172.21.15.7:0/395960679 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f500b5220 con 0x7f5f50160a30 2024-04-25T11:10:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.995+0000 7f5f54c00640 1 --2- 172.21.15.7:0/395960679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f50160a30 0x7f5f500b00c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f480097a0 tx=0x7f5f4802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193ccf1f69915925 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.995+0000 7f5f4ea00640 1 -- 172.21.15.7:0/395960679 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48037070 con 0x7f5f50160a30 2024-04-25T11:10:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.995+0000 7f5f4ea00640 1 -- 172.21.15.7:0/395960679 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f48032440 con 0x7f5f50160a30 2024-04-25T11:10:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.995+0000 7f5f4ea00640 1 -- 172.21.15.7:0/395960679 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48032810 con 0x7f5f50160a30 2024-04-25T11:10:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.996+0000 7f5f56c00640 1 -- 172.21.15.7:0/395960679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f50160a30 msgr2=0x7f5f500b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.996+0000 7f5f56c00640 1 --2- 172.21.15.7:0/395960679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f50160a30 0x7f5f500b00c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5f480097a0 tx=0x7f5f4802a180 comp rx=0 tx=0).stop 2024-04-25T11:10:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.996+0000 7f5f56c00640 1 -- 172.21.15.7:0/395960679 shutdown_connections 2024-04-25T11:10:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.996+0000 7f5f56c00640 1 --2- 172.21.15.7:0/395960679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f50160a30 0x7f5f500b00c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.996+0000 7f5f56c00640 1 --2- 172.21.15.7:0/395960679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f5006dc90 0x7f5f500afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.996+0000 7f5f56c00640 1 -- 172.21.15.7:0/395960679 >> 172.21.15.7:0/395960679 conn(0x7f5f50074200 msgr2=0x7f5f5006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.996+0000 7f5f56c00640 1 -- 172.21.15.7:0/395960679 shutdown_connections 2024-04-25T11:10:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.996+0000 7f5f56c00640 1 -- 172.21.15.7:0/395960679 wait complete. 2024-04-25T11:10:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.998+0000 7f5f56c00640 1 Processor -- start 2024-04-25T11:10:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:29.999+0000 7f5f56c00640 1 -- start start 2024-04-25T11:10:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.000+0000 7f5f56c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f5006f300 0x7f5f50150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.000+0000 7f5f56c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f50160a30 0x7f5f5014c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.000+0000 7f5f56c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f500cc3a0 0x7f5f500cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.000+0000 7f5f56c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5f500b6a30 con 0x7f5f500cc3a0 2024-04-25T11:10:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.000+0000 7f5f56c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5015d220 con 0x7f5f50160a30 2024-04-25T11:10:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.000+0000 7f5f56c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5f500ce920 con 0x7f5f5006f300 2024-04-25T11:10:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.000+0000 7f5f54c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f500cc3a0 0x7f5f500cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.001+0000 7f5f54c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f500cc3a0 0x7f5f500cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36874/0 (socket says 172.21.15.7:36874) 2024-04-25T11:10:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.001+0000 7f5f54c00640 1 -- 172.21.15.7:0/307351880 learned_addr learned my addr 172.21.15.7:0/307351880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.001+0000 7f5f4fe00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f5006f300 0x7f5f50150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.001+0000 7f5f54c00640 1 -- 172.21.15.7:0/307351880 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f5006f300 msgr2=0x7f5f50150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.001+0000 7f5f54c00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f5006f300 0x7f5f50150440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.001+0000 7f5f54c00640 1 -- 172.21.15.7:0/307351880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f50160a30 msgr2=0x7f5f5014c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.001+0000 7f5f54c00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f50160a30 0x7f5f5014c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.001+0000 7f5f54c00640 1 -- 172.21.15.7:0/307351880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f50150980 con 0x7f5f500cc3a0 2024-04-25T11:10:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.001+0000 7f5f54c00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f500cc3a0 0x7f5f500cc7b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f480097a0 tx=0x7f5f480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.001+0000 7f5f47e00640 1 -- 172.21.15.7:0/307351880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48037040 con 0x7f5f500cc3a0 2024-04-25T11:10:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.002+0000 7f5f47e00640 1 -- 172.21.15.7:0/307351880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f48042070 con 0x7f5f500cc3a0 2024-04-25T11:10:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.002+0000 7f5f47e00640 1 -- 172.21.15.7:0/307351880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f48030b30 con 0x7f5f500cc3a0 2024-04-25T11:10:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.002+0000 7f5f56c00640 1 -- 172.21.15.7:0/307351880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5015e170 con 0x7f5f500cc3a0 2024-04-25T11:10:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.002+0000 7f5f56c00640 1 -- 172.21.15.7:0/307351880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f500cbba0 con 0x7f5f500cc3a0 2024-04-25T11:10:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.003+0000 7f5f56c00640 1 -- 172.21.15.7:0/307351880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5015dee0 con 0x7f5f500cc3a0 2024-04-25T11:10:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.004+0000 7f5f47e00640 1 -- 172.21.15.7:0/307351880 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f48030cd0 con 0x7f5f500cc3a0 2024-04-25T11:10:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.006+0000 7f5f47e00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f2c0744b0 0x7f5f2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.006+0000 7f5f47e00640 1 -- 172.21.15.7:0/307351880 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5f480b6180 con 0x7f5f500cc3a0 2024-04-25T11:10:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.006+0000 7f5f4fe00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f2c0744b0 0x7f5f2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.007+0000 7f5f4fe00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f2c0744b0 0x7f5f2c0769c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f40006fd0 tx=0x7f5f40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:30.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.011+0000 7f5f47e00640 1 -- 172.21.15.7:0/307351880 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f48032e20 con 0x7f5f500cc3a0 2024-04-25T11:10:30.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.215+0000 7f5f56c00640 1 -- 172.21.15.7:0/307351880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f500cb6b0 con 0x7f5f500cc3a0 2024-04-25T11:10:30.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.219+0000 7f5f47e00640 1 -- 172.21.15.7:0/307351880 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18418 (secure 0 0 0) 0x7f5f4807fee0 con 0x7f5f500cc3a0 2024-04-25T11:10:30.219 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:30.220 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:28.084739+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:30.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 -- 172.21.15.7:0/307351880 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f2c0744b0 msgr2=0x7f5f2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:30.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f2c0744b0 0x7f5f2c0769c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f40006fd0 tx=0x7f5f40008040 comp rx=0 tx=0).stop 2024-04-25T11:10:30.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 -- 172.21.15.7:0/307351880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f500cc3a0 msgr2=0x7f5f500cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:30.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f500cc3a0 0x7f5f500cc7b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5f480097a0 tx=0x7f5f480046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 -- 172.21.15.7:0/307351880 shutdown_connections 2024-04-25T11:10:30.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f2c0744b0 0x7f5f2c0769c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f500cc3a0 0x7f5f500cc7b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f50160a30 0x7f5f5014c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 --2- 172.21.15.7:0/307351880 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f5006f300 0x7f5f50150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 -- 172.21.15.7:0/307351880 >> 172.21.15.7:0/307351880 conn(0x7f5f50074200 msgr2=0x7f5f500686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:30.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 -- 172.21.15.7:0/307351880 shutdown_connections 2024-04-25T11:10:30.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.222+0000 7f5f56c00640 1 -- 172.21.15.7:0/307351880 wait complete. 2024-04-25T11:10:30.265 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:10:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.544+0000 7f4a02200640 1 Processor -- start 2024-04-25T11:10:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.545+0000 7f4a02200640 1 -- start start 2024-04-25T11:10:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.549+0000 7f4a02200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f49fc163bb0 0x7f49fc0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.549+0000 7f4a02200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 0x7f49fc06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.549+0000 7f4a02200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49fc06cec0 con 0x7f49fc0bae90 2024-04-25T11:10:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.549+0000 7f4a02200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f49fc0d08e0 con 0x7f49fc163bb0 2024-04-25T11:10:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.549+0000 7f4a02200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f49fc0b51c0 con 0x7f49fc06c570 2024-04-25T11:10:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.550+0000 7f4a01800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 0x7f49fc06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.550+0000 7f4a01800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 0x7f49fc06c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:36530/0 (socket says 172.21.15.7:36530) 2024-04-25T11:10:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.550+0000 7f4a01800640 1 -- 172.21.15.7:0/2816968384 learned_addr learned my addr 172.21.15.7:0/2816968384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.550+0000 7f4a00e00640 1 --2- 172.21.15.7:0/2816968384 >> v2:172.21.15.87:3300/0 conn(0x7f49fc163bb0 0x7f49fc0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:30.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.551+0000 7f4a00e00640 1 -- 172.21.15.7:0/2816968384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 msgr2=0x7f49fc06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:30.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.551+0000 7f4a00e00640 1 --2- 172.21.15.7:0/2816968384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 0x7f49fc06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.551+0000 7f4a00e00640 1 -- 172.21.15.7:0/2816968384 >> v1:172.21.15.87:6789/0 conn(0x7f49fc0bae90 legacy=0x7f49fc0b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:30.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.551+0000 7f4a00e00640 1 -- 172.21.15.7:0/2816968384 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc164120 con 0x7f49fc163bb0 2024-04-25T11:10:30.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.551+0000 7f4a01800640 1 --2- 172.21.15.7:0/2816968384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 0x7f49fc06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:10:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.552+0000 7f4a00e00640 1 --2- 172.21.15.7:0/2816968384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49fc163bb0 0x7f49fc0d03a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0097d0 tx=0x7f49ec02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b8fbe4805ee43ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:30.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.554+0000 7f49fbe00640 1 -- 172.21.15.7:0/2816968384 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec037070 con 0x7f49fc163bb0 2024-04-25T11:10:30.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.554+0000 7f49fbe00640 1 -- 172.21.15.7:0/2816968384 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ec032440 con 0x7f49fc163bb0 2024-04-25T11:10:30.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.554+0000 7f49fbe00640 1 -- 172.21.15.7:0/2816968384 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ec032810 con 0x7f49fc163bb0 2024-04-25T11:10:30.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.555+0000 7f4a02200640 1 -- 172.21.15.7:0/2816968384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49fc163bb0 msgr2=0x7f49fc0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:30.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.555+0000 7f4a02200640 1 --2- 172.21.15.7:0/2816968384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49fc163bb0 0x7f49fc0d03a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0097d0 tx=0x7f49ec02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.555+0000 7f4a02200640 1 -- 172.21.15.7:0/2816968384 shutdown_connections 2024-04-25T11:10:30.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.555+0000 7f4a02200640 1 --2- 172.21.15.7:0/2816968384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 0x7f49fc06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.555+0000 7f4a02200640 1 --2- 172.21.15.7:0/2816968384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49fc163bb0 0x7f49fc0d03a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.556+0000 7f4a02200640 1 -- 172.21.15.7:0/2816968384 >> 172.21.15.7:0/2816968384 conn(0x7f49fc06eb00 msgr2=0x7f49fc0d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:30.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.556+0000 7f4a02200640 1 -- 172.21.15.7:0/2816968384 shutdown_connections 2024-04-25T11:10:30.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.556+0000 7f4a02200640 1 -- 172.21.15.7:0/2816968384 wait complete. 2024-04-25T11:10:30.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.559+0000 7f4a02200640 1 Processor -- start 2024-04-25T11:10:30.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.559+0000 7f4a02200640 1 -- start start 2024-04-25T11:10:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.561+0000 7f4a02200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 0x7f49fc1f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.561+0000 7f4a02200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49fc0bae90 0x7f49fc0b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.561+0000 7f4a02200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49fc163bb0 0x7f49fc0b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.561+0000 7f4a02200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1649c0 con 0x7f49fc163bb0 2024-04-25T11:10:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.561+0000 7f4a02200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc1646e0 con 0x7f49fc0bae90 2024-04-25T11:10:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.561+0000 7f4a02200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f49fc0d4170 con 0x7f49fc06c570 2024-04-25T11:10:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.561+0000 7f4a01800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49fc163bb0 0x7f49fc0b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.561+0000 7f4a01800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49fc163bb0 0x7f49fc0b7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36890/0 (socket says 172.21.15.7:36890) 2024-04-25T11:10:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.561+0000 7f4a01800640 1 -- 172.21.15.7:0/4068045801 learned_addr learned my addr 172.21.15.7:0/4068045801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.561+0000 7f49f3e00640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49fc0bae90 0x7f49fc0b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:30.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.561+0000 7f4a00e00640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 0x7f49fc1f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f4a01800640 1 -- 172.21.15.7:0/4068045801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 msgr2=0x7f49fc1f9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f4a01800640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 0x7f49fc1f9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f4a01800640 1 -- 172.21.15.7:0/4068045801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49fc0bae90 msgr2=0x7f49fc0b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f4a01800640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49fc0bae90 0x7f49fc0b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f4a01800640 1 -- 172.21.15.7:0/4068045801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc152570 con 0x7f49fc163bb0 2024-04-25T11:10:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f4a00e00640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 0x7f49fc1f9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f49f3e00640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49fc0bae90 0x7f49fc0b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f4a01800640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49fc163bb0 0x7f49fc0b7b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f49f400efc0 tx=0x7f49f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:30.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f49f9600640 1 -- 172.21.15.7:0/4068045801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f400cbe0 con 0x7f49fc163bb0 2024-04-25T11:10:30.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f49f9600640 1 -- 172.21.15.7:0/4068045801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f400cd80 con 0x7f49fc163bb0 2024-04-25T11:10:30.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f49f9600640 1 -- 172.21.15.7:0/4068045801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4003bd0 con 0x7f49fc163bb0 2024-04-25T11:10:30.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.562+0000 7f4a02200640 1 -- 172.21.15.7:0/4068045801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc152850 con 0x7f49fc163bb0 2024-04-25T11:10:30.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.563+0000 7f4a02200640 1 -- 172.21.15.7:0/4068045801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc152e10 con 0x7f49fc163bb0 2024-04-25T11:10:30.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.564+0000 7f4a02200640 1 -- 172.21.15.7:0/4068045801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49fc1fa800 con 0x7f49fc163bb0 2024-04-25T11:10:30.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.566+0000 7f49f9600640 1 -- 172.21.15.7:0/4068045801 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f49f401b030 con 0x7f49fc163bb0 2024-04-25T11:10:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.567+0000 7f49f9600640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49d80744e0 0x7f49d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.567+0000 7f49f9600640 1 -- 172.21.15.7:0/4068045801 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f49f4095880 con 0x7f49fc163bb0 2024-04-25T11:10:30.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.568+0000 7f4a00e00640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49d80744e0 0x7f49d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:30.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.568+0000 7f4a00e00640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49d80744e0 0x7f49d80769f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49ec034000 tx=0x7f49ec02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:30.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.572+0000 7f49f9600640 1 -- 172.21.15.7:0/4068045801 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49f4022020 con 0x7f49fc163bb0 2024-04-25T11:10:30.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.817+0000 7f4a02200640 1 -- 172.21.15.7:0/4068045801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49fc0d3d20 con 0x7f49fc163bb0 2024-04-25T11:10:30.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.818+0000 7f49f9600640 1 -- 172.21.15.7:0/4068045801 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5168 (secure 0 0 0) 0x7f49f4010040 con 0x7f49fc163bb0 2024-04-25T11:10:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:30.819 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3458223722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3458223722},{"type":"v1","addr":"172.21.15.87:6835","nonce":3458223722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1383067804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1383067804},{"type":"v1","addr":"172.21.15.7:6835","nonce":1383067804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/435824250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":435824250},{"type":"v1","addr":"172.21.15.7:6837","nonce":435824250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:10:23.023233+0000","modified":"2024-04-25T11:10:28.085152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1091275258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1091275258},{"type":"v1","addr":"172.21.15.87:6834","nonce":1091275258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:10:30.819 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 7 2024-04-25T11:10:30.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.821+0000 7f4a02200640 1 -- 172.21.15.7:0/4068045801 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49d80744e0 msgr2=0x7f49d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:30.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.821+0000 7f4a02200640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49d80744e0 0x7f49d80769f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f49ec034000 tx=0x7f49ec02e000 comp rx=0 tx=0).stop 2024-04-25T11:10:30.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.821+0000 7f4a02200640 1 -- 172.21.15.7:0/4068045801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49fc163bb0 msgr2=0x7f49fc0b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:30.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.821+0000 7f4a02200640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49fc163bb0 0x7f49fc0b7b50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f49f400efc0 tx=0x7f49f400c490 comp rx=0 tx=0).stop 2024-04-25T11:10:30.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.822+0000 7f4a02200640 1 -- 172.21.15.7:0/4068045801 shutdown_connections 2024-04-25T11:10:30.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.822+0000 7f4a02200640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49d80744e0 0x7f49d80769f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.822+0000 7f4a02200640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49fc163bb0 0x7f49fc0b7b50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.822+0000 7f4a02200640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49fc0bae90 0x7f49fc0b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.822+0000 7f4a02200640 1 --2- 172.21.15.7:0/4068045801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49fc06c570 0x7f49fc1f9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:30.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.822+0000 7f4a02200640 1 -- 172.21.15.7:0/4068045801 >> 172.21.15.7:0/4068045801 conn(0x7f49fc06eb00 msgr2=0x7f49fc0cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:30.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.822+0000 7f4a02200640 1 -- 172.21.15.7:0/4068045801 shutdown_connections 2024-04-25T11:10:30.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:30.822+0000 7f4a02200640 1 -- 172.21.15.7:0/4068045801 wait complete. 2024-04-25T11:10:30.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.132+0000 7f1f75c00640 1 Processor -- start 2024-04-25T11:10:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.132+0000 7f1f75c00640 1 -- start start 2024-04-25T11:10:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.135+0000 7f1f75c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f700aed40 0x7f1f7015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.135+0000 7f1f75c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f700af420 0x7f1f700af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.135+0000 7f1f75c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f700cebb0 con 0x7f1f700b5220 2024-04-25T11:10:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.135+0000 7f1f75c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1f7015e1f0 con 0x7f1f700af420 2024-04-25T11:10:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.135+0000 7f1f75c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1f700afd70 con 0x7f1f700aed40 2024-04-25T11:10:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.135+0000 7f1f6f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f700b5220 0x7f1f700ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45244/0 (socket says 172.21.15.7:45244) 2024-04-25T11:10:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.135+0000 7f1f75200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f700af420 0x7f1f700af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.135+0000 7f1f6f400640 1 -- 172.21.15.7:0/1791515612 learned_addr learned my addr 172.21.15.7:0/1791515612 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.135+0000 7f1f6fe00640 1 --2- 172.21.15.7:0/1791515612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f700aed40 0x7f1f7015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.136+0000 7f1f6ea00640 1 -- 172.21.15.7:0/1791515612 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3698786996 0 0) 0x7f1f5c001110 con 0x7f1f700b5220 2024-04-25T11:10:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.136+0000 7f1f6ea00640 1 -- 172.21.15.7:0/1791515612 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f54003520 con 0x7f1f700b5220 2024-04-25T11:10:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.136+0000 7f1f75200640 1 -- 172.21.15.7:0/1791515612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f700aed40 msgr2=0x7f1f7015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.136+0000 7f1f75200640 1 --2- 172.21.15.7:0/1791515612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f700aed40 0x7f1f7015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.136+0000 7f1f75200640 1 -- 172.21.15.7:0/1791515612 >> v1:172.21.15.7:6789/0 conn(0x7f1f700b5220 legacy=0x7f1f700ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.136+0000 7f1f75200640 1 -- 172.21.15.7:0/1791515612 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f700b64b0 con 0x7f1f700af420 2024-04-25T11:10:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.137+0000 7f1f75200640 1 --2- 172.21.15.7:0/1791515612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f700af420 0x7f1f700af830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009d90 tx=0x7f1f6802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bac6764b8c0fd6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.137+0000 7f1f6ea00640 1 -- 172.21.15.7:0/1791515612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68007740 con 0x7f1f700af420 2024-04-25T11:10:31.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.137+0000 7f1f6ea00640 1 -- 172.21.15.7:0/1791515612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f680078e0 con 0x7f1f700af420 2024-04-25T11:10:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.137+0000 7f1f6ea00640 1 -- 172.21.15.7:0/1791515612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68007cb0 con 0x7f1f700af420 2024-04-25T11:10:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.138+0000 7f1f75c00640 1 -- 172.21.15.7:0/1791515612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f700af420 msgr2=0x7f1f700af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.138+0000 7f1f75c00640 1 --2- 172.21.15.7:0/1791515612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f700af420 0x7f1f700af830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009d90 tx=0x7f1f6802a660 comp rx=0 tx=0).stop 2024-04-25T11:10:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.138+0000 7f1f75c00640 1 -- 172.21.15.7:0/1791515612 shutdown_connections 2024-04-25T11:10:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.138+0000 7f1f75c00640 1 --2- 172.21.15.7:0/1791515612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f700af420 0x7f1f700af830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.138+0000 7f1f75c00640 1 --2- 172.21.15.7:0/1791515612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f700aed40 0x7f1f7015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.138+0000 7f1f75c00640 1 -- 172.21.15.7:0/1791515612 >> 172.21.15.7:0/1791515612 conn(0x7f1f700b3820 msgr2=0x7f1f700b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.138+0000 7f1f75c00640 1 -- 172.21.15.7:0/1791515612 shutdown_connections 2024-04-25T11:10:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.138+0000 7f1f75c00640 1 -- 172.21.15.7:0/1791515612 wait complete. 2024-04-25T11:10:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.141+0000 7f1f75c00640 1 Processor -- start 2024-04-25T11:10:31.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.141+0000 7f1f75c00640 1 -- start start 2024-04-25T11:10:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.142+0000 7f1f75c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f700aed40 0x7f1f701f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.142+0000 7f1f75c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f700af420 0x7f1f70160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.142+0000 7f1f75c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f700b5220 0x7f1f70160c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.142+0000 7f1f75c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7006cbd0 con 0x7f1f700aed40 2024-04-25T11:10:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.142+0000 7f1f75c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7006c8f0 con 0x7f1f700b5220 2024-04-25T11:10:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.142+0000 7f1f75c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7006ceb0 con 0x7f1f700af420 2024-04-25T11:10:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.143+0000 7f1f6fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f700aed40 0x7f1f701f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.143+0000 7f1f6fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f700aed40 0x7f1f701f5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36904/0 (socket says 172.21.15.7:36904) 2024-04-25T11:10:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.143+0000 7f1f6fe00640 1 -- 172.21.15.7:0/406751051 learned_addr learned my addr 172.21.15.7:0/406751051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.143+0000 7f1f6fe00640 1 -- 172.21.15.7:0/406751051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f700af420 msgr2=0x7f1f70160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.143+0000 7f1f6f400640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f700af420 0x7f1f70160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.143+0000 7f1f75200640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f700b5220 0x7f1f70160c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.143+0000 7f1f6fe00640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f700af420 0x7f1f70160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.143+0000 7f1f6fe00640 1 -- 172.21.15.7:0/406751051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f700b5220 msgr2=0x7f1f70160c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.143+0000 7f1f6fe00640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f700b5220 0x7f1f70160c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.143+0000 7f1f6fe00640 1 -- 172.21.15.7:0/406751051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f701541f0 con 0x7f1f700aed40 2024-04-25T11:10:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.144+0000 7f1f6fe00640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f700aed40 0x7f1f701f5c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000e870 tx=0x7f1f6000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.144+0000 7f1f67e00640 1 -- 172.21.15.7:0/406751051 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60017070 con 0x7f1f700aed40 2024-04-25T11:10:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.144+0000 7f1f67e00640 1 -- 172.21.15.7:0/406751051 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f6000c940 con 0x7f1f700aed40 2024-04-25T11:10:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.144+0000 7f1f75c00640 1 -- 172.21.15.7:0/406751051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f701544d0 con 0x7f1f700aed40 2024-04-25T11:10:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.144+0000 7f1f67e00640 1 -- 172.21.15.7:0/406751051 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000cd10 con 0x7f1f700aed40 2024-04-25T11:10:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.144+0000 7f1f75c00640 1 -- 172.21.15.7:0/406751051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f70154a90 con 0x7f1f700aed40 2024-04-25T11:10:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.145+0000 7f1f75c00640 1 -- 172.21.15.7:0/406751051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f70050740 con 0x7f1f700aed40 2024-04-25T11:10:31.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.146+0000 7f1f67e00640 1 -- 172.21.15.7:0/406751051 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1f6001a070 con 0x7f1f700aed40 2024-04-25T11:10:31.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.148+0000 7f1f67e00640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f50074530 0x7f1f50076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.148+0000 7f1f67e00640 1 -- 172.21.15.7:0/406751051 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1f60095730 con 0x7f1f700aed40 2024-04-25T11:10:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.148+0000 7f1f6f400640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f50074530 0x7f1f50076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.149+0000 7f1f6f400640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f50074530 0x7f1f50076a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0012b0 tx=0x7f1f5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.152+0000 7f1f67e00640 1 -- 172.21.15.7:0/406751051 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f6005fb60 con 0x7f1f700aed40 2024-04-25T11:10:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.358+0000 7f1f75c00640 1 -- 172.21.15.7:0/406751051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f700b3fb0 con 0x7f1f700aed40 2024-04-25T11:10:31.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.360+0000 7f1f67e00640 1 -- 172.21.15.7:0/406751051 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18418 (secure 0 0 0) 0x7f1f6005f460 con 0x7f1f700aed40 2024-04-25T11:10:31.361 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:31.361 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:28.084739+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.363+0000 7f1f75c00640 1 -- 172.21.15.7:0/406751051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f50074530 msgr2=0x7f1f50076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.363+0000 7f1f75c00640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f50074530 0x7f1f50076a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0012b0 tx=0x7f1f5c009040 comp rx=0 tx=0).stop 2024-04-25T11:10:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.363+0000 7f1f75c00640 1 -- 172.21.15.7:0/406751051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f700aed40 msgr2=0x7f1f701f5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.363+0000 7f1f75c00640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f700aed40 0x7f1f701f5c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000e870 tx=0x7f1f6000ed30 comp rx=0 tx=0).stop 2024-04-25T11:10:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.363+0000 7f1f75c00640 1 -- 172.21.15.7:0/406751051 shutdown_connections 2024-04-25T11:10:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.363+0000 7f1f75c00640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f50074530 0x7f1f50076a40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.364+0000 7f1f75c00640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f700b5220 0x7f1f70160c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.364+0000 7f1f75c00640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f700af420 0x7f1f70160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.364+0000 7f1f75c00640 1 --2- 172.21.15.7:0/406751051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f700aed40 0x7f1f701f5c40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.364+0000 7f1f75c00640 1 -- 172.21.15.7:0/406751051 >> 172.21.15.7:0/406751051 conn(0x7f1f700b3820 msgr2=0x7f1f700ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.364+0000 7f1f75c00640 1 -- 172.21.15.7:0/406751051 shutdown_connections 2024-04-25T11:10:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:31.364+0000 7f1f75c00640 1 -- 172.21.15.7:0/406751051 wait complete. 2024-04-25T11:10:31.792 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:10:23.023233+0000', 'modified': '2024-04-25T11:10:28.085152+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6834/1091275258', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6832', 'nonce': 1091275258}, {'type': 'v1', 'addr': '172.21.15.87:6834', 'nonce': 1091275258}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-04-25T11:10:31.792 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:10:31.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T11:10:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.089+0000 7fd2d5a00640 1 Processor -- start 2024-04-25T11:10:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.089+0000 7fd2d5a00640 1 -- start start 2024-04-25T11:10:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.091+0000 7fd2d5a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d006c570 0x7fd2d00b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.091+0000 7fd2d5a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2d00d0330 0x7fd2d00d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.091+0000 7fd2d5a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d00d0c80 con 0x7fd2d00cf110 2024-04-25T11:10:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.091+0000 7fd2d5a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d00b5c20 con 0x7fd2d00d0330 2024-04-25T11:10:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.091+0000 7fd2d5a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d00b5f00 con 0x7fd2d006c570 2024-04-25T11:10:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.092+0000 7fd2d5000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd2d00cf110 0x7fd2d00cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45254/0 (socket says 172.21.15.7:45254) 2024-04-25T11:10:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.092+0000 7fd2d5000640 1 -- 172.21.15.7:0/1226402231 learned_addr learned my addr 172.21.15.7:0/1226402231 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:32.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.092+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d006c570 0x7fd2d00b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.092+0000 7fd2cf400640 1 --2- 172.21.15.7:0/1226402231 >> v2:172.21.15.87:3300/0 conn(0x7fd2d00d0330 0x7fd2d00d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.092+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4244518210 0 0) 0x7fd2c0001110 con 0x7fd2d00cf110 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.092+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2b4003520 con 0x7fd2d00cf110 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 275515175 0 0) 0x7fd2b4003520 con 0x7fd2d00cf110 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d006c570 msgr2=0x7fd2d00b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d006c570 0x7fd2d00b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 >> v2:172.21.15.87:3300/0 conn(0x7fd2d00d0330 msgr2=0x7fd2d00d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 --2- 172.21.15.7:0/1226402231 >> v2:172.21.15.87:3300/0 conn(0x7fd2d00d0330 0x7fd2d00d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d00b61e0 con 0x7fd2d00cf110 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fd2c0001840 con 0x7fd2d00cf110 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 >> v1:172.21.15.7:6789/0 conn(0x7fd2d00cf110 legacy=0x7fd2d00cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2b40039b0 0x7fd2b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d006c570 0x7fd2b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b4004d40 0x7fd2b4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d00b5c20 con 0x7fd2b40039b0 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d00b5f00 con 0x7fd2b4004d40 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b400b9f0 con 0x7fd2d006c570 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.093+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2b40039b0 0x7fd2b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.094+0000 7fd2cf400640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d006c570 0x7fd2b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.094+0000 7fd2d5000640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b4004d40 0x7fd2b4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.094+0000 7fd2cfe00640 1 -- 172.21.15.7:0/1226402231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d006c570 msgr2=0x7fd2b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.094+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d006c570 0x7fd2b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.094+0000 7fd2cfe00640 1 -- 172.21.15.7:0/1226402231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b4004d40 msgr2=0x7fd2b4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.094+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b4004d40 0x7fd2b4005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.094+0000 7fd2cfe00640 1 -- 172.21.15.7:0/1226402231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b4003520 con 0x7fd2b40039b0 2024-04-25T11:10:32.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.095+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2b40039b0 0x7fd2b4003e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009f90 tx=0x7fd2b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3259c6557e71cc29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:32.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.095+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8009c10 con 0x7fd2b40039b0 2024-04-25T11:10:32.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.095+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c8036070 con 0x7fd2b40039b0 2024-04-25T11:10:32.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.095+0000 7fd2cea00640 1 -- 172.21.15.7:0/1226402231 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8031810 con 0x7fd2b40039b0 2024-04-25T11:10:32.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.096+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1226402231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2b40039b0 msgr2=0x7fd2b4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.096+0000 7fd2d5a00640 1 --2- 172.21.15.7:0/1226402231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2b40039b0 0x7fd2b4003e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009f90 tx=0x7fd2b4005830 comp rx=0 tx=0).stop 2024-04-25T11:10:32.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.096+0000 7fd2d5000640 1 -- 172.21.15.7:0/1226402231 reap_dead start 2024-04-25T11:10:32.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.096+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1226402231 shutdown_connections 2024-04-25T11:10:32.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.096+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1226402231 >> 172.21.15.7:0/1226402231 conn(0x7fd2d00d1540 msgr2=0x7fd2d00d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:32.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.096+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1226402231 shutdown_connections 2024-04-25T11:10:32.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.096+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1226402231 wait complete. 2024-04-25T11:10:32.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.099+0000 7fd2d5a00640 1 Processor -- start 2024-04-25T11:10:32.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.099+0000 7fd2d5a00640 1 -- start start 2024-04-25T11:10:32.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.100+0000 7fd2d5a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b40039b0 0x7fd2d013c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.100+0000 7fd2d5a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d0138ee0 0x7fd2d013aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.100+0000 7fd2d5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2d01392f0 0x7fd2d013afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.100+0000 7fd2d5a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01432a0 con 0x7fd2d01392f0 2024-04-25T11:10:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.100+0000 7fd2d5a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d00b67a0 con 0x7fd2b40039b0 2024-04-25T11:10:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.100+0000 7fd2d5a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d0143420 con 0x7fd2d0138ee0 2024-04-25T11:10:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2d5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2d01392f0 0x7fd2d013afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2d5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2d01392f0 0x7fd2d013afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36924/0 (socket says 172.21.15.7:36924) 2024-04-25T11:10:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2d5000640 1 -- 172.21.15.7:0/1301858949 learned_addr learned my addr 172.21.15.7:0/1301858949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b40039b0 0x7fd2d013c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2cf400640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d0138ee0 0x7fd2d013aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2d5000640 1 -- 172.21.15.7:0/1301858949 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d0138ee0 msgr2=0x7fd2d013aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2d5000640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d0138ee0 0x7fd2d013aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2d5000640 1 -- 172.21.15.7:0/1301858949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b40039b0 msgr2=0x7fd2d013c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2d5000640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b40039b0 0x7fd2d013c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2d5000640 1 -- 172.21.15.7:0/1301858949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d013cf40 con 0x7fd2d01392f0 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2cf400640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d0138ee0 0x7fd2d013aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.101+0000 7fd2d5000640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2d01392f0 0x7fd2d013afe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00cdb50 tx=0x7fd2c000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.102+0000 7fd2c7e00640 1 -- 172.21.15.7:0/1301858949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0019070 con 0x7fd2d01392f0 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.102+0000 7fd2c7e00640 1 -- 172.21.15.7:0/1301858949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c000c630 con 0x7fd2d01392f0 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.102+0000 7fd2c7e00640 1 -- 172.21.15.7:0/1301858949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c000ca20 con 0x7fd2d01392f0 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.102+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1301858949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d013fad0 con 0x7fd2d01392f0 2024-04-25T11:10:32.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.102+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1301858949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d0153a50 con 0x7fd2d01392f0 2024-04-25T11:10:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.104+0000 7fd2c7e00640 1 -- 172.21.15.7:0/1301858949 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd2c001c070 con 0x7fd2d01392f0 2024-04-25T11:10:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.105+0000 7fd2c7e00640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2ac0744b0 0x7fd2ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.106+0000 7fd2c7e00640 1 -- 172.21.15.7:0/1301858949 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd2c0094bb0 con 0x7fd2d01392f0 2024-04-25T11:10:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.106+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2ac0744b0 0x7fd2ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.106+0000 7fd2de03a740 1 Processor -- start 2024-04-25T11:10:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.106+0000 7fd2de03a740 1 -- start start 2024-04-25T11:10:32.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.107+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2ac0744b0 0x7fd2ac0769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8035000 tx=0x7fd2c802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:32.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b064064270 0x55b064431210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064043590 0x55b06444c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2de03a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b06436a600 con 0x55b06403b3b0 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2de03a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b064307100 con 0x55b064064270 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2de03a740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55b0643665c0 con 0x55b064043590 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2cf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2cf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36928/0 (socket says 172.21.15.7:36928) 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2cf400640 1 -- 172.21.15.7:0/2132415632 learned_addr learned my addr 172.21.15.7:0/2132415632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2d5000640 1 --2- 172.21.15.7:0/2132415632 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064043590 0x55b06444c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/2132415632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b064064270 0x55b064431210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2cf400640 1 -- 172.21.15.7:0/2132415632 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064043590 msgr2=0x55b06444c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2cf400640 1 --2- 172.21.15.7:0/2132415632 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064043590 0x55b06444c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.108+0000 7fd2cf400640 1 -- 172.21.15.7:0/2132415632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b064064270 msgr2=0x55b064431210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.109+0000 7fd2cf400640 1 --2- 172.21.15.7:0/2132415632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b064064270 0x55b064431210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.109+0000 7fd2cf400640 1 -- 172.21.15.7:0/2132415632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b06407c2d0 con 0x55b06403b3b0 2024-04-25T11:10:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.109+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/2132415632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b064064270 0x55b064431210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.109+0000 7fd2cf400640 1 --2- 172.21.15.7:0/2132415632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444bf40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80036f0 tx=0x7fd2b8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcbec5662e3f943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.109+0000 7fd2c4c00640 1 -- 172.21.15.7:0/2132415632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8019020 con 0x55b06403b3b0 2024-04-25T11:10:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.109+0000 7fd2c4c00640 1 -- 172.21.15.7:0/2132415632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b8009a80 con 0x55b06403b3b0 2024-04-25T11:10:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.109+0000 7fd2c4c00640 1 -- 172.21.15.7:0/2132415632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8009e30 con 0x55b06403b3b0 2024-04-25T11:10:32.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.110+0000 7fd2de03a740 1 -- 172.21.15.7:0/2132415632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 msgr2=0x55b06444bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.110+0000 7fd2de03a740 1 --2- 172.21.15.7:0/2132415632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444bf40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80036f0 tx=0x7fd2b8009190 comp rx=0 tx=0).stop 2024-04-25T11:10:32.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.110+0000 7fd2de03a740 1 -- 172.21.15.7:0/2132415632 shutdown_connections 2024-04-25T11:10:32.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.110+0000 7fd2de03a740 1 --2- 172.21.15.7:0/2132415632 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064043590 0x55b06444c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.110+0000 7fd2de03a740 1 --2- 172.21.15.7:0/2132415632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444bf40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.110+0000 7fd2de03a740 1 --2- 172.21.15.7:0/2132415632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b064064270 0x55b064431210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.110+0000 7fd2de03a740 1 -- 172.21.15.7:0/2132415632 >> 172.21.15.7:0/2132415632 conn(0x55b0640583c0 msgr2=0x55b064406870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:32.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.110+0000 7fd2de03a740 1 -- 172.21.15.7:0/2132415632 shutdown_connections 2024-04-25T11:10:32.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.110+0000 7fd2de03a740 1 -- 172.21.15.7:0/2132415632 wait complete. 2024-04-25T11:10:32.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.111+0000 7fd2de03a740 1 -- start start 2024-04-25T11:10:32.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.112+0000 7fd2de03a740 1 Processor -- start 2024-04-25T11:10:32.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.113+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.113+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b064043590 0x55b064433a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.113+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064064270 0x55b064433f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.113+0000 7fd2de03a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b0642f1f70 con 0x55b06403b3b0 2024-04-25T11:10:32.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.113+0000 7fd2de03a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b0644493b0 con 0x55b064043590 2024-04-25T11:10:32.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.113+0000 7fd2de03a740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55b0644468d0 con 0x55b064064270 2024-04-25T11:10:32.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.113+0000 7fd2de03a740 10 client.0 _ll_register_callbacks cb 0x55b064360480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:10:32.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.113+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.114+0000 7fd2cfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36932/0 (socket says 172.21.15.7:36932) 2024-04-25T11:10:32.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.114+0000 7fd2cfe00640 1 -- 172.21.15.7:0/1869679811 learned_addr learned my addr 172.21.15.7:0/1869679811 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:32.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.114+0000 7fd2cf400640 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b064043590 0x55b064433a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.114+0000 7fd2d5000640 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064064270 0x55b064433f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.114+0000 7fd2cfe00640 1 -- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064064270 msgr2=0x55b064433f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.114+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064064270 0x55b064433f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.114+0000 7fd2cfe00640 1 -- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b064043590 msgr2=0x55b064433a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.114+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b064043590 0x55b064433a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.114+0000 7fd2cfe00640 1 -- 172.21.15.7:0/1869679811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0641466b0 con 0x55b06403b3b0 2024-04-25T11:10:32.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.114+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444c630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd2c802aeb0 tx=0x7fd2c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99abe18d6059065 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:32.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.114+0000 7fd2bd600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:10:32.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.115+0000 7fd2bd600640 1 -- 172.21.15.7:0/1869679811 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8054060 con 0x55b06403b3b0 2024-04-25T11:10:32.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.115+0000 7fd2de03a740 1 -- 172.21.15.7:0/1869679811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2b8009a80 con 0x55b06403b3b0 2024-04-25T11:10:32.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.115+0000 7fd2bd600640 1 -- 172.21.15.7:0/1869679811 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd2c8045430 con 0x55b06403b3b0 2024-04-25T11:10:32.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.115+0000 7fd2bd600640 1 -- 172.21.15.7:0/1869679811 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fd2c805a050 con 0x55b06403b3b0 2024-04-25T11:10:32.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.115+0000 7fd2de03a740 10 client.4389 fetch_fsmap learned FSMap version 7 2024-04-25T11:10:32.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.115+0000 7fd2de03a740 1 -- 172.21.15.7:0/1869679811 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55b064447ba0 con 0x55b06403b3b0 2024-04-25T11:10:32.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.115+0000 7fd2bd600640 1 -- 172.21.15.7:0/1869679811 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b8009a80 con 0x55b06403b3b0 2024-04-25T11:10:32.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.116+0000 7fd2bd600640 1 -- 172.21.15.7:0/1869679811 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fd2c805d020 con 0x55b06403b3b0 2024-04-25T11:10:32.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.116+0000 7fd2de03a740 10 client.4389 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T11:10:32.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.116+0000 7fd2de03a740 10 client.4389 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:10:32.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.116+0000 7fd2de03a740 20 client.4389 populate_metadata read hostname 'smithi007' 2024-04-25T11:10:32.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.116+0000 7fd2de03a740 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x55b064022e40 0x55b064448390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.116+0000 7fd2de03a740 4 client.4389 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:10:32.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.116+0000 7fd2de03a740 1 -- 172.21.15.7:0/1869679811 --> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b064378210 con 0x55b064022e40 2024-04-25T11:10:32.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.116+0000 7fd2cf400640 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x55b064022e40 0x55b064448390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.117+0000 7fd2cf400640 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x55b064022e40 0x55b064448390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b46f7c7734f7645 server_cookie=837a7d3e2f814054 in_seq=0 out_seq=0 2024-04-25T11:10:32.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.117+0000 7fd2bd600640 10 client.4389 ms_handle_connect on v2:172.21.15.87:6832/1091275258 2024-04-25T11:10:32.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2bd600640 1 -- 172.21.15.7:0/1869679811 <== mds.0 v2:172.21.15.87:6832/1091275258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b064378210 con 0x55b064022e40 2024-04-25T11:10:32.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2bd600640 10 client.4389 handle_command_reply: tid=0 2024-04-25T11:10:32.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2de03a740 1 client.4389 shutdown 2024-04-25T11:10:32.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2de03a740 1 -- 172.21.15.7:0/1869679811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 msgr2=0x55b06444c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2de03a740 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444c630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd2c802aeb0 tx=0x7fd2c8004ab0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2de03a740 1 -- 172.21.15.7:0/1869679811 shutdown_connections 2024-04-25T11:10:32.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2de03a740 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x55b064022e40 0x55b064448390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2de03a740 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064064270 0x55b064433f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2de03a740 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b064043590 0x55b064433a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2de03a740 1 --2- 172.21.15.7:0/1869679811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b06403b3b0 0x55b06444c630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2bd600640 0 client.4389 ms_handle_reset on v2:172.21.15.87:6832/1091275258 2024-04-25T11:10:32.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.120+0000 7fd2de03a740 1 -- 172.21.15.7:0/1869679811 >> 172.21.15.7:0/1869679811 conn(0x55b0640583c0 msgr2=0x55b064431310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:32.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.121+0000 7fd2de03a740 1 -- 172.21.15.7:0/1869679811 shutdown_connections 2024-04-25T11:10:32.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.121+0000 7fd2de03a740 1 -- 172.21.15.7:0/1869679811 wait complete. 2024-04-25T11:10:32.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.121+0000 7fd2de03a740 20 client.4389 trim_cache size 0 max 16384 2024-04-25T11:10:32.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.142+0000 7fd2de03a740 1 Processor -- start 2024-04-25T11:10:32.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.142+0000 7fd2de03a740 1 -- start start 2024-04-25T11:10:32.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.143+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b06403b3b0 0x55b06442b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.143+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064043590 0x55b0644016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.143+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064064270 0x55b06446f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.143+0000 7fd2de03a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0201530 con 0x55b064043590 2024-04-25T11:10:32.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.143+0000 7fd2de03a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d02008c0 con 0x55b06403b3b0 2024-04-25T11:10:32.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.143+0000 7fd2de03a740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d0200f10 con 0x55b064064270 2024-04-25T11:10:32.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.144+0000 7fd2cf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064043590 0x55b0644016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.144+0000 7fd2cf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064043590 0x55b0644016d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36944/0 (socket says 172.21.15.7:36944) 2024-04-25T11:10:32.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.144+0000 7fd2cf400640 1 -- 172.21.15.7:0/329584983 learned_addr learned my addr 172.21.15.7:0/329584983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:32.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.144+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/329584983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b06403b3b0 0x55b06442b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.144+0000 7fd2d5000640 1 --2- 172.21.15.7:0/329584983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064064270 0x55b06446f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.144+0000 7fd2cf400640 1 -- 172.21.15.7:0/329584983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064064270 msgr2=0x55b06446f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.144+0000 7fd2cf400640 1 --2- 172.21.15.7:0/329584983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064064270 0x55b06446f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.144+0000 7fd2cf400640 1 -- 172.21.15.7:0/329584983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b06403b3b0 msgr2=0x55b06442b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.144+0000 7fd2cf400640 1 --2- 172.21.15.7:0/329584983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b06403b3b0 0x55b06442b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.144+0000 7fd2cf400640 1 -- 172.21.15.7:0/329584983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b064425b50 con 0x55b064043590 2024-04-25T11:10:32.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.145+0000 7fd2cf400640 1 --2- 172.21.15.7:0/329584983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064043590 0x55b0644016d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80036f0 tx=0x7fd2b800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88200478bb8d4b81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:32.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.145+0000 7fd2bfe00640 1 -- 172.21.15.7:0/329584983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b802e070 con 0x55b064043590 2024-04-25T11:10:32.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.145+0000 7fd2bfe00640 1 -- 172.21.15.7:0/329584983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b8009d40 con 0x55b064043590 2024-04-25T11:10:32.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.145+0000 7fd2bfe00640 1 -- 172.21.15.7:0/329584983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8019040 con 0x55b064043590 2024-04-25T11:10:32.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.146+0000 7fd2de03a740 1 -- 172.21.15.7:0/329584983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064043590 msgr2=0x55b0644016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.146+0000 7fd2de03a740 1 --2- 172.21.15.7:0/329584983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064043590 0x55b0644016d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80036f0 tx=0x7fd2b800f000 comp rx=0 tx=0).stop 2024-04-25T11:10:32.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.146+0000 7fd2de03a740 1 -- 172.21.15.7:0/329584983 shutdown_connections 2024-04-25T11:10:32.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.146+0000 7fd2de03a740 1 --2- 172.21.15.7:0/329584983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064064270 0x55b06446f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.146+0000 7fd2de03a740 1 --2- 172.21.15.7:0/329584983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064043590 0x55b0644016d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.146+0000 7fd2de03a740 1 --2- 172.21.15.7:0/329584983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b06403b3b0 0x55b06442b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.146+0000 7fd2de03a740 1 -- 172.21.15.7:0/329584983 >> 172.21.15.7:0/329584983 conn(0x55b0640583c0 msgr2=0x55b06444c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:32.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.146+0000 7fd2de03a740 1 -- 172.21.15.7:0/329584983 shutdown_connections 2024-04-25T11:10:32.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.146+0000 7fd2de03a740 1 -- 172.21.15.7:0/329584983 wait complete. 2024-04-25T11:10:32.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.147+0000 7fd2de03a740 1 -- start start 2024-04-25T11:10:32.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.147+0000 7fd2de03a740 1 Processor -- start 2024-04-25T11:10:32.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b06403b3b0 0x55b06446edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064043590 0x55b064424f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2de03a740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064064270 0x55b064425460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2de03a740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b06442be70 con 0x55b064064270 2024-04-25T11:10:32.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2de03a740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55b064449210 con 0x55b06403b3b0 2024-04-25T11:10:32.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2de03a740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55b06445f5a0 con 0x55b064043590 2024-04-25T11:10:32.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2de03a740 10 client.0 _ll_register_callbacks cb 0x7fd2b40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:10:32.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2d5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064064270 0x55b064425460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2d5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064064270 0x55b064425460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36954/0 (socket says 172.21.15.7:36954) 2024-04-25T11:10:32.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2d5000640 1 -- 172.21.15.7:0/3204760590 learned_addr learned my addr 172.21.15.7:0/3204760590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:32.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2cf400640 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064043590 0x55b064424f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.149+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b06403b3b0 0x55b06446edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.150+0000 7fd2d5000640 1 -- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064043590 msgr2=0x55b064424f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.150+0000 7fd2d5000640 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064043590 0x55b064424f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.150+0000 7fd2d5000640 1 -- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b06403b3b0 msgr2=0x55b06446edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.150+0000 7fd2d5000640 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b06403b3b0 0x55b06446edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.150+0000 7fd2d5000640 1 -- 172.21.15.7:0/3204760590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b064400310 con 0x55b064064270 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.150+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b06403b3b0 0x55b06446edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.150+0000 7fd2d5000640 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064064270 0x55b064425460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0001e30 tx=0x7fd2c005ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146f54e2e0a1582 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.150+0000 7fd2bd600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.150+0000 7fd2bd600640 1 -- 172.21.15.7:0/3204760590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c005ef60 con 0x55b064064270 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.150+0000 7fd2bd600640 1 -- 172.21.15.7:0/3204760590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd2c000cbc0 con 0x55b064064270 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.150+0000 7fd2de03a740 1 -- 172.21.15.7:0/3204760590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2b8009d40 con 0x55b064064270 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.151+0000 7fd2bd600640 1 -- 172.21.15.7:0/3204760590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fd2c0092050 con 0x55b064064270 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.151+0000 7fd2de03a740 10 client.4395 fetch_fsmap learned FSMap version 7 2024-04-25T11:10:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.151+0000 7fd2de03a740 1 -- 172.21.15.7:0/3204760590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55b0644d1bf0 con 0x55b064064270 2024-04-25T11:10:32.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.151+0000 7fd2bd600640 1 -- 172.21.15.7:0/3204760590 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b8009d40 con 0x55b064064270 2024-04-25T11:10:32.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.151+0000 7fd2bd600640 1 -- 172.21.15.7:0/3204760590 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fd2c005dac0 con 0x55b064064270 2024-04-25T11:10:32.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.151+0000 7fd2de03a740 10 client.4395 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T11:10:32.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.151+0000 7fd2de03a740 10 client.4395 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:10:32.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.151+0000 7fd2de03a740 20 client.4395 populate_metadata read hostname 'smithi007' 2024-04-25T11:10:32.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.151+0000 7fd2de03a740 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x55b064406560 0x55b064406970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.151+0000 7fd2de03a740 4 client.4395 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:10:32.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.151+0000 7fd2de03a740 1 -- 172.21.15.7:0/3204760590 --> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] -- command(tid 0: {"prefix": "status"}) -- 0x55b064372080 con 0x55b064406560 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.152+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x55b064406560 0x55b064406970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.152+0000 7fd2cfe00640 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x55b064406560 0x55b064406970 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=204f60bcf573106d server_cookie=bc27371c748c51ed in_seq=0 out_seq=0 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.152+0000 7fd2bd600640 10 client.4395 ms_handle_connect on v2:172.21.15.87:6832/1091275258 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.153+0000 7fd2bd600640 1 -- 172.21.15.7:0/3204760590 <== mds.0 v2:172.21.15.87:6832/1091275258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55b064372080 con 0x55b064406560 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.153+0000 7fd2bd600640 10 client.4395 handle_command_reply: tid=0 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.153+0000 7fd2de03a740 1 client.4395 shutdown 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.153+0000 7fd2de03a740 1 -- 172.21.15.7:0/3204760590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064064270 msgr2=0x55b064425460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.153+0000 7fd2de03a740 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064064270 0x55b064425460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0001e30 tx=0x7fd2c005ada0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.154+0000 7fd2de03a740 1 -- 172.21.15.7:0/3204760590 shutdown_connections 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.154+0000 7fd2de03a740 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x55b064406560 0x55b064406970 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.154+0000 7fd2de03a740 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b064064270 0x55b064425460 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.154+0000 7fd2de03a740 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55b064043590 0x55b064424f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.154+0000 7fd2de03a740 1 --2- 172.21.15.7:0/3204760590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55b06403b3b0 0x55b06446edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.154+0000 7fd2bd600640 0 client.4395 ms_handle_reset on v2:172.21.15.87:6832/1091275258 2024-04-25T11:10:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.154+0000 7fd2de03a740 1 -- 172.21.15.7:0/3204760590 >> 172.21.15.7:0/3204760590 conn(0x55b0640583c0 msgr2=0x55b0644050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.154+0000 7fd2de03a740 1 -- 172.21.15.7:0/3204760590 shutdown_connections 2024-04-25T11:10:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.154+0000 7fd2de03a740 1 -- 172.21.15.7:0/3204760590 wait complete. 2024-04-25T11:10:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.154+0000 7fd2de03a740 20 client.4395 trim_cache size 0 max 16384 2024-04-25T11:10:32.157 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:10:32.158 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:10:32.158 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:10:32.158 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4285, 2024-04-25T11:10:32.158 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:10:32.158 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:10:32.158 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:10:32.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 8.7107469450000004, 2024-04-25T11:10:32.159 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 7, 2024-04-25T11:10:32.159 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 26, 2024-04-25T11:10:32.159 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 26, 2024-04-25T11:10:32.159 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 15.158338895 2024-04-25T11:10:32.159 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:10:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1301858949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2ac0744b0 msgr2=0x7fd2ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2ac0744b0 0x7fd2ac0769c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8035000 tx=0x7fd2c802f000 comp rx=0 tx=0).stop 2024-04-25T11:10:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1301858949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2d01392f0 msgr2=0x7fd2d013afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2d01392f0 0x7fd2d013afe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd2d00cdb50 tx=0x7fd2c000eb90 comp rx=0 tx=0).stop 2024-04-25T11:10:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1301858949 shutdown_connections 2024-04-25T11:10:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2ac0744b0 0x7fd2ac0769c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2d01392f0 0x7fd2d013afe0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2d0138ee0 0x7fd2d013aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 --2- 172.21.15.7:0/1301858949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b40039b0 0x7fd2d013c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1301858949 >> 172.21.15.7:0/1301858949 conn(0x7fd2d00d1540 msgr2=0x7fd2d00b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1301858949 shutdown_connections 2024-04-25T11:10:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.155+0000 7fd2d5a00640 1 -- 172.21.15.7:0/1301858949 wait complete. 2024-04-25T11:10:32.183 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:10:32.183 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:10:32.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:10:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.463+0000 7fa9e9800640 1 Processor -- start 2024-04-25T11:10:32.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.463+0000 7fa9e9800640 1 -- start start 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.466+0000 7fa9e9800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9e408cc20 0x7fa9e40b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.466+0000 7fa9e9800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa9e40b7210 0x7fa9e409ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.466+0000 7fa9e9800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9e40d21c0 0x7fa9e40d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.466+0000 7fa9e9800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e40b9570 con 0x7fa9e40d21c0 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.466+0000 7fa9e9800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e40b8fb0 con 0x7fa9e40b7210 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.466+0000 7fa9e9800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e40cf730 con 0x7fa9e408cc20 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.466+0000 7fa9e3400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9e40d21c0 0x7fa9e40d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.466+0000 7fa9e3400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9e40d21c0 0x7fa9e40d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36962/0 (socket says 172.21.15.7:36962) 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.466+0000 7fa9e3400640 1 -- 172.21.15.7:0/1752811698 learned_addr learned my addr 172.21.15.7:0/1752811698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.466+0000 7fa9e2000640 1 --2- 172.21.15.7:0/1752811698 >> v2:172.21.15.87:3300/0 conn(0x7fa9e40b7210 0x7fa9e409ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.466+0000 7fa9e2a00640 1 --2- 172.21.15.7:0/1752811698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9e408cc20 0x7fa9e40b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.467+0000 7fa9e3400640 1 -- 172.21.15.7:0/1752811698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9e408cc20 msgr2=0x7fa9e40b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.467+0000 7fa9e3400640 1 --2- 172.21.15.7:0/1752811698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9e408cc20 0x7fa9e40b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.467+0000 7fa9e3400640 1 -- 172.21.15.7:0/1752811698 >> v2:172.21.15.87:3300/0 conn(0x7fa9e40b7210 msgr2=0x7fa9e409ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.467+0000 7fa9e3400640 1 --2- 172.21.15.7:0/1752811698 >> v2:172.21.15.87:3300/0 conn(0x7fa9e40b7210 0x7fa9e409ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.467+0000 7fa9e3400640 1 -- 172.21.15.7:0/1752811698 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e40cfa10 con 0x7fa9e40d21c0 2024-04-25T11:10:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.468+0000 7fa9e3400640 1 --2- 172.21.15.7:0/1752811698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9e40d21c0 0x7fa9e40d1580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009e50 tx=0x7fa9dc02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1296116c2a74681 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.468+0000 7fa9e1600640 1 -- 172.21.15.7:0/1752811698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc036070 con 0x7fa9e40d21c0 2024-04-25T11:10:32.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.468+0000 7fa9e1600640 1 -- 172.21.15.7:0/1752811698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9dc031600 con 0x7fa9e40d21c0 2024-04-25T11:10:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.468+0000 7fa9e1600640 1 -- 172.21.15.7:0/1752811698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0319d0 con 0x7fa9e40d21c0 2024-04-25T11:10:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.469+0000 7fa9e9800640 1 -- 172.21.15.7:0/1752811698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9e40d21c0 msgr2=0x7fa9e40d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.469+0000 7fa9e9800640 1 --2- 172.21.15.7:0/1752811698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9e40d21c0 0x7fa9e40d1580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009e50 tx=0x7fa9dc02a640 comp rx=0 tx=0).stop 2024-04-25T11:10:32.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.469+0000 7fa9e9800640 1 -- 172.21.15.7:0/1752811698 shutdown_connections 2024-04-25T11:10:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.469+0000 7fa9e9800640 1 --2- 172.21.15.7:0/1752811698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9e40d21c0 0x7fa9e40d1580 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.469+0000 7fa9e9800640 1 --2- 172.21.15.7:0/1752811698 >> v2:172.21.15.87:3300/0 conn(0x7fa9e40b7210 0x7fa9e409ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.469+0000 7fa9e9800640 1 --2- 172.21.15.7:0/1752811698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9e408cc20 0x7fa9e40b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.469+0000 7fa9e9800640 1 -- 172.21.15.7:0/1752811698 >> 172.21.15.7:0/1752811698 conn(0x7fa9e40d0430 msgr2=0x7fa9e40d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.469+0000 7fa9e9800640 1 -- 172.21.15.7:0/1752811698 shutdown_connections 2024-04-25T11:10:32.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.469+0000 7fa9e9800640 1 -- 172.21.15.7:0/1752811698 wait complete. 2024-04-25T11:10:32.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.472+0000 7fa9e9800640 1 Processor -- start 2024-04-25T11:10:32.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.472+0000 7fa9e9800640 1 -- start start 2024-04-25T11:10:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.473+0000 7fa9e9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9e408cc20 0x7fa9e414f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.473+0000 7fa9e9800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9e40b7210 0x7fa9e409d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.473+0000 7fa9e9800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e40d21c0 0x7fa9e409de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.473+0000 7fa9e9800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e40d11d0 con 0x7fa9e408cc20 2024-04-25T11:10:32.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.473+0000 7fa9e9800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e40d1050 con 0x7fa9e40d21c0 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.473+0000 7fa9e9800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e40d1ac0 con 0x7fa9e40b7210 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.473+0000 7fa9e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9e408cc20 0x7fa9e414f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.473+0000 7fa9e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9e408cc20 0x7fa9e414f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36972/0 (socket says 172.21.15.7:36972) 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.474+0000 7fa9e2a00640 1 -- 172.21.15.7:0/2798821099 learned_addr learned my addr 172.21.15.7:0/2798821099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.474+0000 7fa9e2000640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9e40b7210 0x7fa9e409d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.474+0000 7fa9e3400640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e40d21c0 0x7fa9e409de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.474+0000 7fa9e2a00640 1 -- 172.21.15.7:0/2798821099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9e40b7210 msgr2=0x7fa9e409d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.474+0000 7fa9e2a00640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9e40b7210 0x7fa9e409d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.474+0000 7fa9e2a00640 1 -- 172.21.15.7:0/2798821099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e40d21c0 msgr2=0x7fa9e409de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.474+0000 7fa9e2a00640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e40d21c0 0x7fa9e409de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.474+0000 7fa9e2a00640 1 -- 172.21.15.7:0/2798821099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e414f660 con 0x7fa9e408cc20 2024-04-25T11:10:32.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.474+0000 7fa9e2a00640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9e408cc20 0x7fa9e414f120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400d5d0 tx=0x7fa9d400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.475+0000 7fa9daa00640 1 -- 172.21.15.7:0/2798821099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4018020 con 0x7fa9e408cc20 2024-04-25T11:10:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.475+0000 7fa9daa00640 1 -- 172.21.15.7:0/2798821099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d4014620 con 0x7fa9e408cc20 2024-04-25T11:10:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.475+0000 7fa9e9800640 1 -- 172.21.15.7:0/2798821099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e409e6e0 con 0x7fa9e408cc20 2024-04-25T11:10:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.475+0000 7fa9daa00640 1 -- 172.21.15.7:0/2798821099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4014a10 con 0x7fa9e408cc20 2024-04-25T11:10:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.475+0000 7fa9e9800640 1 -- 172.21.15.7:0/2798821099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e409eca0 con 0x7fa9e408cc20 2024-04-25T11:10:32.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.476+0000 7fa9e9800640 1 -- 172.21.15.7:0/2798821099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e4050740 con 0x7fa9e408cc20 2024-04-25T11:10:32.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.478+0000 7fa9daa00640 1 -- 172.21.15.7:0/2798821099 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa9d401b020 con 0x7fa9e408cc20 2024-04-25T11:10:32.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.479+0000 7fa9daa00640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9c40744e0 0x7fa9c40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:32.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.479+0000 7fa9daa00640 1 -- 172.21.15.7:0/2798821099 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa9d4095600 con 0x7fa9e408cc20 2024-04-25T11:10:32.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.479+0000 7fa9e2000640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9c40744e0 0x7fa9c40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:32.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.480+0000 7fa9e2000640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9c40744e0 0x7fa9c40769f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0006fd0 tx=0x7fa9d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:32.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.484+0000 7fa9daa00640 1 -- 172.21.15.7:0/2798821099 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9d4022030 con 0x7fa9e408cc20 2024-04-25T11:10:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.729+0000 7fa9e9800640 1 -- 172.21.15.7:0/2798821099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa9e4076100 con 0x7fa9e408cc20 2024-04-25T11:10:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.730+0000 7fa9daa00640 1 -- 172.21.15.7:0/2798821099 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fa9d405fad0 con 0x7fa9e408cc20 2024-04-25T11:10:32.731 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T11:10:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.732+0000 7fa9e9800640 1 -- 172.21.15.7:0/2798821099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9c40744e0 msgr2=0x7fa9c40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.732+0000 7fa9e9800640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9c40744e0 0x7fa9c40769f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0006fd0 tx=0x7fa9d0008040 comp rx=0 tx=0).stop 2024-04-25T11:10:32.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.732+0000 7fa9e9800640 1 -- 172.21.15.7:0/2798821099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9e408cc20 msgr2=0x7fa9e414f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:32.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.732+0000 7fa9e9800640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9e408cc20 0x7fa9e414f120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400d5d0 tx=0x7fa9d400da90 comp rx=0 tx=0).stop 2024-04-25T11:10:32.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.733+0000 7fa9e9800640 1 -- 172.21.15.7:0/2798821099 shutdown_connections 2024-04-25T11:10:32.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.733+0000 7fa9e9800640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9c40744e0 0x7fa9c40769f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.733+0000 7fa9e9800640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e40d21c0 0x7fa9e409de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.733+0000 7fa9e9800640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9e40b7210 0x7fa9e409d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.733+0000 7fa9e9800640 1 --2- 172.21.15.7:0/2798821099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9e408cc20 0x7fa9e414f120 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.733+0000 7fa9e9800640 1 -- 172.21.15.7:0/2798821099 >> 172.21.15.7:0/2798821099 conn(0x7fa9e40d0430 msgr2=0x7fa9e40b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.733+0000 7fa9e9800640 1 -- 172.21.15.7:0/2798821099 shutdown_connections 2024-04-25T11:10:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32.733+0000 7fa9e9800640 1 -- 172.21.15.7:0/2798821099 wait complete. 2024-04-25T11:10:32.758 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:10:32.759 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:10:32.759 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:10:32.759 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T11:10:32.759 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:10:32.759 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:10:32.759 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:10:32.759 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:10:32.759 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T11:10:32.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:32.760 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:10:32.814 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:10:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:10:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:10:32.814 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42542sec preferred_lft 42542sec 2024-04-25T11:10:32.815 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:10:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:10:32.816 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:10:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:10:32.816 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:10:32.817 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T11:10:32.817 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:10:32.817 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:10:32.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-04-25T11:10:32.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-04-25T11:10:32.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-04-25T11:10:32.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T11:10:32.818 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:10:32.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:10:33.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:10:33.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:33.340 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:10:33.409 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:10:33.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:33.414 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:10:33.463 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:10:33.463 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:10:33.463 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:10:33.463 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:10:33.463 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:10:33.464 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:10:33.464 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:10:33.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:10:33.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:10:33.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:10:33.465 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:10:33.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:10:34.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:10:34.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:34.233 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:10:34.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:34.284 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-04-25T11:10:34.329 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:10:34.329 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:10:34.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:10:34.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T11:10:34.329 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:10:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:10:34.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:10:34.561 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T11:10:34.561 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:10:34.561 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:10:34.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T11:10:34.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T11:10:34.562 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T11:10:34.562 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T11:10:34.562 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T11:10:34.562 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:10:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:10:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:10:34.838 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:10:34.838 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:10:34.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:10:34.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T11:10:34.838 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:10:34.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:10:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:10:35.033 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T11:10:35.033 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:10:35.033 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:10:35.078 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T11:10:35.078 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:10:35.078 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:10:35.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-04-25T11:10:35.242 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:10:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-04-25T11:10:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-04-25T11:10:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-04-25T11:10:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-04-25T11:10:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-04-25T11:10:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:10:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:10:35.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:35.292 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-04-25T11:10:35.388 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T11:10:35.388 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T11:10:35.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-04-25T11:10:35.458 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:10:35.500 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-04-25T11:10:35.500 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-04-25T11:10:35.500 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-04-25T11:10:35.500 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-04-25T11:10:35.500 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-04-25T11:10:35.500 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-04-25T11:10:35.500 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-04-25T11:10:35.500 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-04-25T11:10:35.500 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-04-25T11:10:35.500 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T11:10:35.501 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-04-25T11:10:35.502 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-04-25T11:10:35.502 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:10:35.502 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:10:35.502 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:10:35.502 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:10:35.502 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-04-25T11:10:35.502 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:10:35.502 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:10:35.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:35.503 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-04-25T11:10:35.651 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:2024-04-25T11:10:35.650+0000 7f209d031480 -1 init, newargv = 0x55e26af51340 newargc=15 2024-04-25T11:10:35.651 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[38714]: starting ceph client 2024-04-25T11:10:35.665 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[38714]: starting fuse 2024-04-25T11:10:36.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-04-25T11:10:36.787 DEBUG:teuthology.orchestra.run.smithi007:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T11:10:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:/proc 2024-04-25T11:10:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:/sys 2024-04-25T11:10:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:/dev 2024-04-25T11:10:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/security 2024-04-25T11:10:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/shm 2024-04-25T11:10:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/pts 2024-04-25T11:10:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:/run 2024-04-25T11:10:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/cgroup 2024-04-25T11:10:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/pstore 2024-04-25T11:10:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/bpf 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/ 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/selinux 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/hugepages 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/dev/mqueue 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/debug 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/tracing 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/fs/fuse/connections 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/sys/kernel/config 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T11:10:36.849 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T11:10:36.850 INFO:teuthology.orchestra.run.smithi007.stdout:/run/user/1000 2024-04-25T11:10:36.850 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph 2024-04-25T11:10:36.850 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T11:10:36.850 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T11:10:36.850 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T11:10:36.850 INFO:teuthology.orchestra.run.smithi007.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T11:10:36.850 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns 2024-04-25T11:10:36.850 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:10:36.850 INFO:teuthology.orchestra.run.smithi007.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:10:36.850 INFO:teuthology.orchestra.run.smithi007.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T11:10:36.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:36.851 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-04-25T11:10:36.908 INFO:teuthology.orchestra.run.smithi007.stdout:42 2024-04-25T11:10:36.908 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-25T11:10:36.908 DEBUG:teuthology.orchestra.run.smithi007:> sudo stdin-killer -- python3 -c ' 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:10:36.909 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-04-25T11:10:36.910 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T11:10:36.910 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T11:10:36.910 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T11:10:36.910 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-04-25T11:10:36.910 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-04-25T11:10:36.910 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-04-25T11:10:36.910 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T11:10:36.910 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:10:36.910 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.admin")) 2024-04-25T11:10:36.910 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-04-25T11:10:37.089 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.admin.38714.asok 2024-04-25T11:10:37.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:10:37.106 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.38714.asok 2024-04-25T11:10:37.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:37.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.38714.asok status 2024-04-25T11:10:37.411 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "admin", 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "38714", 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4410, 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-04-25T11:10:37.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4410 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1439195879 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1439195879 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.4410 192.168.0.1:0/1439195879", 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/1439195879", 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2024-04-25T11:10:37.413 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 7, 2024-04-25T11:10:37.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 26, 2024-04-25T11:10:37.414 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-04-25T11:10:37.414 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-04-25T11:10:37.414 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-04-25T11:10:37.414 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:10:37.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:37.432 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-04-25T11:10:37.432 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:10:37.490 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-04-25T11:10:37.490 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:10:37.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:37.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:10:37.581 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T11:10:37.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:10:37.740 INFO:teuthology.orchestra.run.smithi007.stdout:# file: . 2024-04-25T11:10:37.741 INFO:teuthology.orchestra.run.smithi007.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T11:10:37.741 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:37.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:10:37.751 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:10:37.751 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:10:37.806 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:38.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.090+0000 7fad7fe00640 1 Processor -- start 2024-04-25T11:10:38.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.090+0000 7fad7fe00640 1 -- start start 2024-04-25T11:10:38.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.093+0000 7fad7fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad800b5220 0x7fad8015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:38.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.093+0000 7fad7fe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad800cebb0 con 0x7fad800aed40 2024-04-25T11:10:38.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.093+0000 7fad7fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad800b64b0 con 0x7fad800af420 2024-04-25T11:10:38.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.093+0000 7fad7fe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fad8015e1f0 con 0x7fad800b5220 2024-04-25T11:10:38.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.093+0000 7fad7ea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fad800aed40 0x7fad800ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45258/0 (socket says 172.21.15.7:45258) 2024-04-25T11:10:38.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.093+0000 7fad7ea00640 1 -- 172.21.15.7:0/2757005778 learned_addr learned my addr 172.21.15.7:0/2757005778 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:38.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.093+0000 7fad77e00640 1 --2- 172.21.15.7:0/2757005778 >> v2:172.21.15.7:3300/0 conn(0x7fad800b5220 0x7fad8015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:38.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.094+0000 7fad77e00640 1 -- 172.21.15.7:0/2757005778 >> v1:172.21.15.87:6789/0 conn(0x7fad800af420 legacy=0x7fad800af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:38.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.094+0000 7fad77e00640 1 -- 172.21.15.7:0/2757005778 >> v1:172.21.15.7:6789/0 conn(0x7fad800aed40 legacy=0x7fad800ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.094+0000 7fad77e00640 1 -- 172.21.15.7:0/2757005778 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8006c610 con 0x7fad800b5220 2024-04-25T11:10:38.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.095+0000 7fad77e00640 1 --2- 172.21.15.7:0/2757005778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad800b5220 0x7fad8015dcb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fad680097a0 tx=0x7fad6802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723bcdca1f9997cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:38.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.095+0000 7fad7e000640 1 -- 172.21.15.7:0/2757005778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68037070 con 0x7fad800b5220 2024-04-25T11:10:38.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.095+0000 7fad7e000640 1 -- 172.21.15.7:0/2757005778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad68032440 con 0x7fad800b5220 2024-04-25T11:10:38.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.095+0000 7fad7e000640 1 -- 172.21.15.7:0/2757005778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68032810 con 0x7fad800b5220 2024-04-25T11:10:38.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.096+0000 7fad7fe00640 1 -- 172.21.15.7:0/2757005778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad800b5220 msgr2=0x7fad8015dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.096+0000 7fad7fe00640 1 --2- 172.21.15.7:0/2757005778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad800b5220 0x7fad8015dcb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fad680097a0 tx=0x7fad6802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.096+0000 7fad7fe00640 1 -- 172.21.15.7:0/2757005778 shutdown_connections 2024-04-25T11:10:38.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.096+0000 7fad7fe00640 1 --2- 172.21.15.7:0/2757005778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad800b5220 0x7fad8015dcb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.096+0000 7fad7fe00640 1 -- 172.21.15.7:0/2757005778 >> 172.21.15.7:0/2757005778 conn(0x7fad800b3820 msgr2=0x7fad800b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:38.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.096+0000 7fad7fe00640 1 -- 172.21.15.7:0/2757005778 shutdown_connections 2024-04-25T11:10:38.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.096+0000 7fad7fe00640 1 -- 172.21.15.7:0/2757005778 wait complete. 2024-04-25T11:10:38.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.099+0000 7fad7fe00640 1 Processor -- start 2024-04-25T11:10:38.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.099+0000 7fad7fe00640 1 -- start start 2024-04-25T11:10:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.100+0000 7fad7fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad800aed40 0x7fad8009c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.100+0000 7fad7fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad800af420 0x7fad801f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.100+0000 7fad7fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad800b5220 0x7fad8014a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.100+0000 7fad7fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad8006ceb0 con 0x7fad800aed40 2024-04-25T11:10:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.100+0000 7fad7fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fad8006cbd0 con 0x7fad800af420 2024-04-25T11:10:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.101+0000 7fad7fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad800b6110 con 0x7fad800b5220 2024-04-25T11:10:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.101+0000 7fad7ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad800aed40 0x7fad8009c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.101+0000 7fad7ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad800aed40 0x7fad8009c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36992/0 (socket says 172.21.15.7:36992) 2024-04-25T11:10:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.101+0000 7fad7ea00640 1 -- 172.21.15.7:0/3437656911 learned_addr learned my addr 172.21.15.7:0/3437656911 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.101+0000 7fad77e00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad800af420 0x7fad801f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.101+0000 7fad7f400640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad800b5220 0x7fad8014a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.101+0000 7fad7ea00640 1 -- 172.21.15.7:0/3437656911 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad800b5220 msgr2=0x7fad8014a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.101+0000 7fad7ea00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad800b5220 0x7fad8014a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.101+0000 7fad7ea00640 1 -- 172.21.15.7:0/3437656911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad800af420 msgr2=0x7fad801f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.101+0000 7fad7ea00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad800af420 0x7fad801f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.102+0000 7fad7ea00640 1 -- 172.21.15.7:0/3437656911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad801f4fd0 con 0x7fad800aed40 2024-04-25T11:10:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.102+0000 7fad7ea00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad800aed40 0x7fad8009c910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c440 tx=0x7fad7000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.102+0000 7fad76a00640 1 -- 172.21.15.7:0/3437656911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70017070 con 0x7fad800aed40 2024-04-25T11:10:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.102+0000 7fad76a00640 1 -- 172.21.15.7:0/3437656911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad70005ce0 con 0x7fad800aed40 2024-04-25T11:10:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.102+0000 7fad76a00640 1 -- 172.21.15.7:0/3437656911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad700072f0 con 0x7fad800aed40 2024-04-25T11:10:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.102+0000 7fad7fe00640 1 -- 172.21.15.7:0/3437656911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8014a600 con 0x7fad800aed40 2024-04-25T11:10:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.102+0000 7fad7fe00640 1 -- 172.21.15.7:0/3437656911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8009bb50 con 0x7fad800aed40 2024-04-25T11:10:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.103+0000 7fad7fe00640 1 -- 172.21.15.7:0/3437656911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8009d130 con 0x7fad800aed40 2024-04-25T11:10:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.105+0000 7fad76a00640 1 -- 172.21.15.7:0/3437656911 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad7001a070 con 0x7fad800aed40 2024-04-25T11:10:38.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.107+0000 7fad76a00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad5c0744b0 0x7fad5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:38.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.107+0000 7fad76a00640 1 -- 172.21.15.7:0/3437656911 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fad70094980 con 0x7fad800aed40 2024-04-25T11:10:38.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.107+0000 7fad77e00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad5c0744b0 0x7fad5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:38.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.108+0000 7fad77e00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad5c0744b0 0x7fad5c0769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fad68034000 tx=0x7fad6802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:38.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.112+0000 7fad76a00640 1 -- 172.21.15.7:0/3437656911 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad801f4fd0 con 0x7fad800aed40 2024-04-25T11:10:38.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.317+0000 7fad7fe00640 1 -- 172.21.15.7:0/3437656911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad8015ae20 con 0x7fad800aed40 2024-04-25T11:10:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.318+0000 7fad76a00640 1 -- 172.21.15.7:0/3437656911 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18418 (secure 0 0 0) 0x7fad800cebb0 con 0x7fad800aed40 2024-04-25T11:10:38.320 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:38.321 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:28.084739+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.321+0000 7fad7fe00640 1 -- 172.21.15.7:0/3437656911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad5c0744b0 msgr2=0x7fad5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.321+0000 7fad7fe00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad5c0744b0 0x7fad5c0769c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fad68034000 tx=0x7fad6802e000 comp rx=0 tx=0).stop 2024-04-25T11:10:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.321+0000 7fad7fe00640 1 -- 172.21.15.7:0/3437656911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad800aed40 msgr2=0x7fad8009c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.321+0000 7fad7fe00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad800aed40 0x7fad8009c910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c440 tx=0x7fad7000c900 comp rx=0 tx=0).stop 2024-04-25T11:10:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.322+0000 7fad7fe00640 1 -- 172.21.15.7:0/3437656911 shutdown_connections 2024-04-25T11:10:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.322+0000 7fad7fe00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad5c0744b0 0x7fad5c0769c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.322+0000 7fad7fe00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad800b5220 0x7fad8014a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.322+0000 7fad7fe00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad800af420 0x7fad801f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.322+0000 7fad7fe00640 1 --2- 172.21.15.7:0/3437656911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad800aed40 0x7fad8009c910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.322+0000 7fad7fe00640 1 -- 172.21.15.7:0/3437656911 >> 172.21.15.7:0/3437656911 conn(0x7fad800b3820 msgr2=0x7fad800b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.322+0000 7fad7fe00640 1 -- 172.21.15.7:0/3437656911 shutdown_connections 2024-04-25T11:10:38.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.322+0000 7fad7fe00640 1 -- 172.21.15.7:0/3437656911 wait complete. 2024-04-25T11:10:38.352 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:10:38.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:10:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.647+0000 7fc8a2600640 1 Processor -- start 2024-04-25T11:10:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.647+0000 7fc8a2600640 1 -- start start 2024-04-25T11:10:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.649+0000 7fc8a2600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc89c161160 0x7fc89c0b7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.649+0000 7fc8a2600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc89c0b7980 0x7fc89c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.649+0000 7fc8a2600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc89c0b7d90 con 0x7fc89c15a7a0 2024-04-25T11:10:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.649+0000 7fc8a2600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc89c160cd0 con 0x7fc89c0b7980 2024-04-25T11:10:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.649+0000 7fc8a2600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc89c06cfc0 con 0x7fc89c161160 2024-04-25T11:10:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.650+0000 7fc893e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc89c161160 0x7fc89c0b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.650+0000 7fc8a1200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc89c15a7a0 0x7fc89c06c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:49430/0 (socket says 172.21.15.7:49430) 2024-04-25T11:10:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.650+0000 7fc8a1200640 1 -- 172.21.15.7:0/3963541544 learned_addr learned my addr 172.21.15.7:0/3963541544 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.650+0000 7fc8a1c00640 1 --2- 172.21.15.7:0/3963541544 >> v2:172.21.15.87:3300/0 conn(0x7fc89c0b7980 0x7fc89c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.651+0000 7fc89be00640 1 -- 172.21.15.7:0/3963541544 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422075140 0 0) 0x7fc88c001490 con 0x7fc89c15a7a0 2024-04-25T11:10:38.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.651+0000 7fc89be00640 1 -- 172.21.15.7:0/3963541544 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc880003520 con 0x7fc89c15a7a0 2024-04-25T11:10:38.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.651+0000 7fc893e00640 1 -- 172.21.15.7:0/3963541544 >> v2:172.21.15.87:3300/0 conn(0x7fc89c0b7980 msgr2=0x7fc89c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:38.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.651+0000 7fc893e00640 1 --2- 172.21.15.7:0/3963541544 >> v2:172.21.15.87:3300/0 conn(0x7fc89c0b7980 0x7fc89c0b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.651+0000 7fc893e00640 1 -- 172.21.15.7:0/3963541544 >> v1:172.21.15.87:6789/0 conn(0x7fc89c15a7a0 legacy=0x7fc89c06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.652+0000 7fc893e00640 1 -- 172.21.15.7:0/3963541544 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c06d2a0 con 0x7fc89c161160 2024-04-25T11:10:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.653+0000 7fc893e00640 1 --2- 172.21.15.7:0/3963541544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc89c161160 0x7fc89c0b7440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc8840097a0 tx=0x7fc88402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f10352345e1eedf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.654+0000 7fc89be00640 1 -- 172.21.15.7:0/3963541544 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884037070 con 0x7fc89c161160 2024-04-25T11:10:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.654+0000 7fc89be00640 1 -- 172.21.15.7:0/3963541544 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc88402ae50 con 0x7fc89c161160 2024-04-25T11:10:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.654+0000 7fc89be00640 1 -- 172.21.15.7:0/3963541544 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884032660 con 0x7fc89c161160 2024-04-25T11:10:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.655+0000 7fc8a2600640 1 -- 172.21.15.7:0/3963541544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc89c161160 msgr2=0x7fc89c0b7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.655+0000 7fc8a2600640 1 --2- 172.21.15.7:0/3963541544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc89c161160 0x7fc89c0b7440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc8840097a0 tx=0x7fc88402a100 comp rx=0 tx=0).stop 2024-04-25T11:10:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.655+0000 7fc8a2600640 1 -- 172.21.15.7:0/3963541544 shutdown_connections 2024-04-25T11:10:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.655+0000 7fc8a2600640 1 --2- 172.21.15.7:0/3963541544 >> v2:172.21.15.87:3300/0 conn(0x7fc89c0b7980 0x7fc89c0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.655+0000 7fc8a2600640 1 --2- 172.21.15.7:0/3963541544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc89c161160 0x7fc89c0b7440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.655+0000 7fc8a2600640 1 -- 172.21.15.7:0/3963541544 >> 172.21.15.7:0/3963541544 conn(0x7fc89c0d59c0 msgr2=0x7fc89c0d5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.655+0000 7fc8a2600640 1 -- 172.21.15.7:0/3963541544 shutdown_connections 2024-04-25T11:10:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.655+0000 7fc8a2600640 1 -- 172.21.15.7:0/3963541544 wait complete. 2024-04-25T11:10:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.657+0000 7fc8a2600640 1 Processor -- start 2024-04-25T11:10:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.658+0000 7fc8a2600640 1 -- start start 2024-04-25T11:10:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.659+0000 7fc8a2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c0b7980 0x7fc89c0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.659+0000 7fc8a2600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc89c15a7a0 0x7fc89c1f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.659+0000 7fc8a2600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc89c161160 0x7fc89c0cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.659+0000 7fc8a2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c06cc80 con 0x7fc89c0b7980 2024-04-25T11:10:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.659+0000 7fc8a2600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c161570 con 0x7fc89c15a7a0 2024-04-25T11:10:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.659+0000 7fc8a2600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc89c160610 con 0x7fc89c161160 2024-04-25T11:10:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.659+0000 7fc8a1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c0b7980 0x7fc89c0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.659+0000 7fc8a1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c0b7980 0x7fc89c0c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36994/0 (socket says 172.21.15.7:36994) 2024-04-25T11:10:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.659+0000 7fc8a1200640 1 -- 172.21.15.7:0/114760740 learned_addr learned my addr 172.21.15.7:0/114760740 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.660+0000 7fc893e00640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc89c15a7a0 0x7fc89c1f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.660+0000 7fc8a1c00640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc89c161160 0x7fc89c0cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.660+0000 7fc8a1200640 1 -- 172.21.15.7:0/114760740 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc89c161160 msgr2=0x7fc89c0cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.660+0000 7fc8a1200640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc89c161160 0x7fc89c0cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.660+0000 7fc8a1200640 1 -- 172.21.15.7:0/114760740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc89c15a7a0 msgr2=0x7fc89c1f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.660+0000 7fc8a1200640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc89c15a7a0 0x7fc89c1f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.660+0000 7fc8a1200640 1 -- 172.21.15.7:0/114760740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c0c9940 con 0x7fc89c0b7980 2024-04-25T11:10:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.660+0000 7fc8a1200640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c0b7980 0x7fc89c0c9400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00ca70 tx=0x7fc88c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.661+0000 7fc899600640 1 -- 172.21.15.7:0/114760740 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c017070 con 0x7fc89c0b7980 2024-04-25T11:10:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.661+0000 7fc899600640 1 -- 172.21.15.7:0/114760740 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc88c00abd0 con 0x7fc89c0b7980 2024-04-25T11:10:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.661+0000 7fc899600640 1 -- 172.21.15.7:0/114760740 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c012400 con 0x7fc89c0b7980 2024-04-25T11:10:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.661+0000 7fc8a2600640 1 -- 172.21.15.7:0/114760740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c0c9dc0 con 0x7fc89c0b7980 2024-04-25T11:10:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.661+0000 7fc8a2600640 1 -- 172.21.15.7:0/114760740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c0ca380 con 0x7fc89c0b7980 2024-04-25T11:10:38.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.662+0000 7fc8a2600640 1 -- 172.21.15.7:0/114760740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89c0d3ee0 con 0x7fc89c0b7980 2024-04-25T11:10:38.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.663+0000 7fc899600640 1 -- 172.21.15.7:0/114760740 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc88c01a070 con 0x7fc89c0b7980 2024-04-25T11:10:38.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.665+0000 7fc899600640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc878074530 0x7fc878076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:38.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.665+0000 7fc899600640 1 -- 172.21.15.7:0/114760740 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc88c0951b0 con 0x7fc89c0b7980 2024-04-25T11:10:38.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.665+0000 7fc893e00640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc878074530 0x7fc878076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:38.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.666+0000 7fc893e00640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc878074530 0x7fc878076a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc884034000 tx=0x7fc88402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.670+0000 7fc899600640 1 -- 172.21.15.7:0/114760740 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc89c0b7d90 con 0x7fc89c0b7980 2024-04-25T11:10:38.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.879+0000 7fc8a2600640 1 -- 172.21.15.7:0/114760740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc89c075ea0 con 0x7fc89c0b7980 2024-04-25T11:10:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.880+0000 7fc899600640 1 -- 172.21.15.7:0/114760740 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7fc88c05ef90 con 0x7fc89c0b7980 2024-04-25T11:10:38.908 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:10:38.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.882+0000 7fc8a2600640 1 -- 172.21.15.7:0/114760740 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc878074530 msgr2=0x7fc878076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.882+0000 7fc8a2600640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc878074530 0x7fc878076a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc884034000 tx=0x7fc88402e000 comp rx=0 tx=0).stop 2024-04-25T11:10:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.882+0000 7fc8a2600640 1 -- 172.21.15.7:0/114760740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c0b7980 msgr2=0x7fc89c0c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.882+0000 7fc8a2600640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c0b7980 0x7fc89c0c9400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00ca70 tx=0x7fc88c00cf30 comp rx=0 tx=0).stop 2024-04-25T11:10:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.883+0000 7fc8a2600640 1 -- 172.21.15.7:0/114760740 shutdown_connections 2024-04-25T11:10:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.883+0000 7fc8a2600640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc878074530 0x7fc878076a40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.883+0000 7fc8a2600640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc89c161160 0x7fc89c0cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.883+0000 7fc8a2600640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc89c15a7a0 0x7fc89c1f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.883+0000 7fc8a2600640 1 --2- 172.21.15.7:0/114760740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c0b7980 0x7fc89c0c9400 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.883+0000 7fc8a2600640 1 -- 172.21.15.7:0/114760740 >> 172.21.15.7:0/114760740 conn(0x7fc89c0d59c0 msgr2=0x7fc89c0c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.883+0000 7fc8a2600640 1 -- 172.21.15.7:0/114760740 shutdown_connections 2024-04-25T11:10:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:38.883+0000 7fc8a2600640 1 -- 172.21.15.7:0/114760740 wait complete. 2024-04-25T11:10:38.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:10:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.184+0000 7f7a66200640 1 Processor -- start 2024-04-25T11:10:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.184+0000 7f7a66200640 1 -- start start 2024-04-25T11:10:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.187+0000 7f7a66200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7a601611e0 0x7f7a6006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.187+0000 7f7a66200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a6006e950 con 0x7f7a6006e540 2024-04-25T11:10:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.187+0000 7f7a66200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a60160720 con 0x7f7a600b4db0 2024-04-25T11:10:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.187+0000 7f7a66200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7a600b7f00 con 0x7f7a601611e0 2024-04-25T11:10:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.187+0000 7f7a65800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7a6006e540 0x7f7a601618e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45268/0 (socket says 172.21.15.7:45268) 2024-04-25T11:10:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.187+0000 7f7a65800640 1 -- 172.21.15.7:0/2918423494 learned_addr learned my addr 172.21.15.7:0/2918423494 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.187+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055634770 0 0) 0x7f7a58001110 con 0x7f7a6006e540 2024-04-25T11:10:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.187+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a44003520 con 0x7f7a6006e540 2024-04-25T11:10:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.187+0000 7f7a5fe00640 1 --2- 172.21.15.7:0/2918423494 >> v2:172.21.15.87:3300/0 conn(0x7f7a601611e0 0x7f7a6006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1955018831 0 0) 0x7f7a44003520 con 0x7f7a6006e540 2024-04-25T11:10:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 >> v2:172.21.15.87:3300/0 conn(0x7f7a601611e0 msgr2=0x7f7a6006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 --2- 172.21.15.7:0/2918423494 >> v2:172.21.15.87:3300/0 conn(0x7f7a601611e0 0x7f7a6006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 >> v1:172.21.15.87:6789/0 conn(0x7f7a600b4db0 legacy=0x7f7a600b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a600b81e0 con 0x7f7a6006e540 2024-04-25T11:10:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f7a58001840 con 0x7f7a6006e540 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 >> v1:172.21.15.7:6789/0 conn(0x7f7a6006e540 legacy=0x7f7a601618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 --2- 172.21.15.7:0/2918423494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a44003a00 0x7f7a44003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 --2- 172.21.15.7:0/2918423494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a44004800 0x7f7a44004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 --2- 172.21.15.7:0/2918423494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a440053f0 0x7f7a4400bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a600b7f00 con 0x7f7a440053f0 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4400c390 con 0x7f7a44003a00 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4400c590 con 0x7f7a44004800 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.188+0000 7f7a65800640 1 --2- 172.21.15.7:0/2918423494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a440053f0 0x7f7a4400bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.189+0000 7f7a5fe00640 1 --2- 172.21.15.7:0/2918423494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a44004800 0x7f7a44004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.189+0000 7f7a64e00640 1 --2- 172.21.15.7:0/2918423494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a44003a00 0x7f7a44003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.189+0000 7f7a65800640 1 -- 172.21.15.7:0/2918423494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a44004800 msgr2=0x7f7a44004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.189+0000 7f7a65800640 1 --2- 172.21.15.7:0/2918423494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a44004800 0x7f7a44004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.189+0000 7f7a65800640 1 -- 172.21.15.7:0/2918423494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a44003a00 msgr2=0x7f7a44003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.189+0000 7f7a65800640 1 --2- 172.21.15.7:0/2918423494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a44003a00 0x7f7a44003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.189+0000 7f7a65800640 1 -- 172.21.15.7:0/2918423494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a44003520 con 0x7f7a440053f0 2024-04-25T11:10:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.190+0000 7f7a65800640 1 -- 172.21.15.7:0/2918423494 reap_dead start 2024-04-25T11:10:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.190+0000 7f7a65800640 1 --2- 172.21.15.7:0/2918423494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a440053f0 0x7f7a4400bab0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a58007fd0 tx=0x7f7a44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfec22b0d0056c38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.190+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58007e30 con 0x7f7a440053f0 2024-04-25T11:10:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.190+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a58005b10 con 0x7f7a440053f0 2024-04-25T11:10:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.190+0000 7f7a5f400640 1 -- 172.21.15.7:0/2918423494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58007070 con 0x7f7a440053f0 2024-04-25T11:10:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.191+0000 7f7a66200640 1 -- 172.21.15.7:0/2918423494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a440053f0 msgr2=0x7f7a4400bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.191+0000 7f7a66200640 1 --2- 172.21.15.7:0/2918423494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a440053f0 0x7f7a4400bab0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7a58007fd0 tx=0x7f7a44005830 comp rx=0 tx=0).stop 2024-04-25T11:10:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.191+0000 7f7a66200640 1 -- 172.21.15.7:0/2918423494 shutdown_connections 2024-04-25T11:10:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.191+0000 7f7a66200640 1 --2- 172.21.15.7:0/2918423494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a440053f0 0x7f7a4400bab0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.191+0000 7f7a66200640 1 -- 172.21.15.7:0/2918423494 >> 172.21.15.7:0/2918423494 conn(0x7f7a600d24b0 msgr2=0x7f7a600d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.191+0000 7f7a66200640 1 -- 172.21.15.7:0/2918423494 shutdown_connections 2024-04-25T11:10:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.191+0000 7f7a66200640 1 -- 172.21.15.7:0/2918423494 wait complete. 2024-04-25T11:10:39.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.193+0000 7f7a66200640 1 Processor -- start 2024-04-25T11:10:39.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.193+0000 7f7a66200640 1 -- start start 2024-04-25T11:10:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a66200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a6014c3b0 0x7f7a601e86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a66200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a601e8c10 0x7f7a6015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a66200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6015db80 0x7f7a601f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a66200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a600d3ef0 con 0x7f7a6015db80 2024-04-25T11:10:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a66200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7a601615f0 con 0x7f7a601e8c10 2024-04-25T11:10:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a66200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7a600d4070 con 0x7f7a6014c3b0 2024-04-25T11:10:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a65800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6015db80 0x7f7a601f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a65800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6015db80 0x7f7a601f98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37020/0 (socket says 172.21.15.7:37020) 2024-04-25T11:10:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a65800640 1 -- 172.21.15.7:0/2092108592 learned_addr learned my addr 172.21.15.7:0/2092108592 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a64e00640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a6014c3b0 0x7f7a601e86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a65800640 1 -- 172.21.15.7:0/2092108592 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a6014c3b0 msgr2=0x7f7a601e86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.195+0000 7f7a5fe00640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a601e8c10 0x7f7a6015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.196+0000 7f7a65800640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a6014c3b0 0x7f7a601e86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.196+0000 7f7a65800640 1 -- 172.21.15.7:0/2092108592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a601e8c10 msgr2=0x7f7a6015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.196+0000 7f7a65800640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a601e8c10 0x7f7a6015d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.196+0000 7f7a65800640 1 -- 172.21.15.7:0/2092108592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6015ca70 con 0x7f7a6015db80 2024-04-25T11:10:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.196+0000 7f7a5fe00640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a601e8c10 0x7f7a6015d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.196+0000 7f7a65800640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6015db80 0x7f7a601f98c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7a58000c00 tx=0x7f7a58001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.196+0000 7f7a5cc00640 1 -- 172.21.15.7:0/2092108592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58007480 con 0x7f7a6015db80 2024-04-25T11:10:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.196+0000 7f7a5cc00640 1 -- 172.21.15.7:0/2092108592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a58042070 con 0x7f7a6015db80 2024-04-25T11:10:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.196+0000 7f7a5cc00640 1 -- 172.21.15.7:0/2092108592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5803a410 con 0x7f7a6015db80 2024-04-25T11:10:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.196+0000 7f7a66200640 1 -- 172.21.15.7:0/2092108592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a601e9020 con 0x7f7a6015db80 2024-04-25T11:10:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.196+0000 7f7a66200640 1 -- 172.21.15.7:0/2092108592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a601e6620 con 0x7f7a6015db80 2024-04-25T11:10:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.197+0000 7f7a66200640 1 -- 172.21.15.7:0/2092108592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a60050740 con 0x7f7a6015db80 2024-04-25T11:10:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.199+0000 7f7a5cc00640 1 -- 172.21.15.7:0/2092108592 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7a5803a5b0 con 0x7f7a6015db80 2024-04-25T11:10:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.200+0000 7f7a5cc00640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7a3c074500 0x7f7a3c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.200+0000 7f7a5cc00640 1 -- 172.21.15.7:0/2092108592 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7a580b5a70 con 0x7f7a6015db80 2024-04-25T11:10:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.200+0000 7f7a64e00640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7a3c074500 0x7f7a3c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.201+0000 7f7a64e00640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7a3c074500 0x7f7a3c076a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a600cdd20 tx=0x7f7a50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.204+0000 7f7a5cc00640 1 -- 172.21.15.7:0/2092108592 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a601e9020 con 0x7f7a6015db80 2024-04-25T11:10:39.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.411+0000 7f7a66200640 1 -- 172.21.15.7:0/2092108592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7a600cdd50 con 0x7f7a6015db80 2024-04-25T11:10:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.412+0000 7f7a5cc00640 1 -- 172.21.15.7:0/2092108592 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f7a6006e950 con 0x7f7a6015db80 2024-04-25T11:10:39.413 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:10:39.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 -- 172.21.15.7:0/2092108592 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7a3c074500 msgr2=0x7f7a3c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7a3c074500 0x7f7a3c076a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a600cdd20 tx=0x7f7a50009040 comp rx=0 tx=0).stop 2024-04-25T11:10:39.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 -- 172.21.15.7:0/2092108592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6015db80 msgr2=0x7f7a601f98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6015db80 0x7f7a601f98c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7a58000c00 tx=0x7f7a58001b40 comp rx=0 tx=0).stop 2024-04-25T11:10:39.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 -- 172.21.15.7:0/2092108592 shutdown_connections 2024-04-25T11:10:39.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7a3c074500 0x7f7a3c076a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a6015db80 0x7f7a601f98c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a601e8c10 0x7f7a6015d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 --2- 172.21.15.7:0/2092108592 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a6014c3b0 0x7f7a601e86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 -- 172.21.15.7:0/2092108592 >> 172.21.15.7:0/2092108592 conn(0x7f7a600d24b0 msgr2=0x7f7a600ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:39.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 -- 172.21.15.7:0/2092108592 shutdown_connections 2024-04-25T11:10:39.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.415+0000 7f7a66200640 1 -- 172.21.15.7:0/2092108592 wait complete. 2024-04-25T11:10:39.442 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:10:39.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.724+0000 7f5d7ae00640 1 Processor -- start 2024-04-25T11:10:39.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.724+0000 7f5d7ae00640 1 -- start start 2024-04-25T11:10:39.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.727+0000 7f5d7ae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d7406fab0 0x7f5d74168e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.727+0000 7f5d7ae00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d7415e3c0 con 0x7f5d740b0470 2024-04-25T11:10:39.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.727+0000 7f5d7ae00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d740cf800 con 0x7f5d7415dfb0 2024-04-25T11:10:39.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.727+0000 7f5d7ae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5d740d1250 con 0x7f5d7406fab0 2024-04-25T11:10:39.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.727+0000 7f5d73400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5d740b0470 0x7f5d7406f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45274/0 (socket says 172.21.15.7:45274) 2024-04-25T11:10:39.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.727+0000 7f5d73400640 1 -- 172.21.15.7:0/841125386 learned_addr learned my addr 172.21.15.7:0/841125386 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:39.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.727+0000 7f5d78e00640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d7406fab0 0x7f5d74168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.727+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 162843075 0 0) 0x7f5d7415e3c0 con 0x7f5d740b0470 2024-04-25T11:10:39.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d58003520 con 0x7f5d740b0470 2024-04-25T11:10:39.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2412371642 0 0) 0x7f5d58003520 con 0x7f5d740b0470 2024-04-25T11:10:39.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d7406fab0 msgr2=0x7f5d74168e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:39.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d7406fab0 0x7f5d74168e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 >> v1:172.21.15.87:6789/0 conn(0x7f5d7415dfb0 legacy=0x7f5d740d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d740cfaa0 con 0x7f5d740b0470 2024-04-25T11:10:39.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f5d5c001670 con 0x7f5d740b0470 2024-04-25T11:10:39.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 >> v1:172.21.15.7:6789/0 conn(0x7f5d740b0470 legacy=0x7f5d7406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d580039e0 0x7f5d58003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 0x7f5d58004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d7406fab0 0x7f5d580052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d740d1250 con 0x7f5d58004800 2024-04-25T11:10:39.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5800bb80 con 0x7f5d580039e0 2024-04-25T11:10:39.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5d5800bd80 con 0x7f5d7406fab0 2024-04-25T11:10:39.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.728+0000 7f5d73400640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 0x7f5d58004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.729+0000 7f5d73e00640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d580039e0 0x7f5d58003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.729+0000 7f5d78e00640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d7406fab0 0x7f5d580052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.729+0000 7f5d73400640 1 -- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d7406fab0 msgr2=0x7f5d580052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:39.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.729+0000 7f5d73400640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d7406fab0 0x7f5d580052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.729+0000 7f5d73400640 1 -- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d580039e0 msgr2=0x7f5d58003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:39.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.729+0000 7f5d73400640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d580039e0 0x7f5d58003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.729+0000 7f5d73400640 1 -- 172.21.15.7:0/841125386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d58003520 con 0x7f5d58004800 2024-04-25T11:10:39.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.729+0000 7f5d78e00640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d7406fab0 0x7f5d580052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:39.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.730+0000 7f5d73400640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 0x7f5d58004c10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5d740b6c90 tx=0x7f5d58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c12d4c3005a3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:39.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.730+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c007c40 con 0x7f5d58004800 2024-04-25T11:10:39.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.730+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d5c005af0 con 0x7f5d58004800 2024-04-25T11:10:39.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.730+0000 7f5d72a00640 1 -- 172.21.15.7:0/841125386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c007070 con 0x7f5d58004800 2024-04-25T11:10:39.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.731+0000 7f5d7ae00640 1 -- 172.21.15.7:0/841125386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 msgr2=0x7f5d58004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.731+0000 7f5d7ae00640 1 --2- 172.21.15.7:0/841125386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 0x7f5d58004c10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5d740b6c90 tx=0x7f5d58005830 comp rx=0 tx=0).stop 2024-04-25T11:10:39.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.731+0000 7f5d78e00640 1 -- 172.21.15.7:0/841125386 reap_dead start 2024-04-25T11:10:39.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.731+0000 7f5d7ae00640 1 -- 172.21.15.7:0/841125386 shutdown_connections 2024-04-25T11:10:39.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.731+0000 7f5d7ae00640 1 -- 172.21.15.7:0/841125386 >> 172.21.15.7:0/841125386 conn(0x7f5d740742c0 msgr2=0x7f5d7406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:39.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.731+0000 7f5d7ae00640 1 -- 172.21.15.7:0/841125386 shutdown_connections 2024-04-25T11:10:39.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.731+0000 7f5d7ae00640 1 -- 172.21.15.7:0/841125386 wait complete. 2024-04-25T11:10:39.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.734+0000 7f5d7ae00640 1 Processor -- start 2024-04-25T11:10:39.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.734+0000 7f5d7ae00640 1 -- start start 2024-04-25T11:10:39.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.735+0000 7f5d7ae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 0x7f5d74154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.735+0000 7f5d7ae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d74154a60 0x7f5d74154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.735+0000 7f5d7ae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d74159c70 0x7f5d7415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d7ae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d740d0340 con 0x7f5d58004800 2024-04-25T11:10:39.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d7ae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5d740d0060 con 0x7f5d74154a60 2024-04-25T11:10:39.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d7ae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5d740d0620 con 0x7f5d74159c70 2024-04-25T11:10:39.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 0x7f5d74154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 0x7f5d74154520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37028/0 (socket says 172.21.15.7:37028) 2024-04-25T11:10:39.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d73e00640 1 -- 172.21.15.7:0/1110088169 learned_addr learned my addr 172.21.15.7:0/1110088169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:39.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d73400640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d74154a60 0x7f5d74154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d78e00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d74159c70 0x7f5d7415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d73e00640 1 -- 172.21.15.7:0/1110088169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d74159c70 msgr2=0x7f5d7415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d73e00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d74159c70 0x7f5d7415a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d73e00640 1 -- 172.21.15.7:0/1110088169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d74154a60 msgr2=0x7f5d74154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d73e00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d74154a60 0x7f5d74154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d73e00640 1 -- 172.21.15.7:0/1110088169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d741553b0 con 0x7f5d58004800 2024-04-25T11:10:39.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.736+0000 7f5d78e00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d74159c70 0x7f5d7415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:39.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.737+0000 7f5d73e00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 0x7f5d74154520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400f5a0 tx=0x7f5d6400fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:39.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.737+0000 7f5d6be00640 1 -- 172.21.15.7:0/1110088169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d640052e0 con 0x7f5d58004800 2024-04-25T11:10:39.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.737+0000 7f5d6be00640 1 -- 172.21.15.7:0/1110088169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d64018070 con 0x7f5d58004800 2024-04-25T11:10:39.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.737+0000 7f5d6be00640 1 -- 172.21.15.7:0/1110088169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d64003810 con 0x7f5d58004800 2024-04-25T11:10:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.737+0000 7f5d7ae00640 1 -- 172.21.15.7:0/1110088169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d7415f610 con 0x7f5d58004800 2024-04-25T11:10:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.737+0000 7f5d7ae00640 1 -- 172.21.15.7:0/1110088169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d7415fbd0 con 0x7f5d58004800 2024-04-25T11:10:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.738+0000 7f5d7ae00640 1 -- 172.21.15.7:0/1110088169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d74050740 con 0x7f5d58004800 2024-04-25T11:10:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.740+0000 7f5d6be00640 1 -- 172.21.15.7:0/1110088169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5d640039b0 con 0x7f5d58004800 2024-04-25T11:10:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.741+0000 7f5d6be00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d500744e0 0x7f5d500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:39.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.741+0000 7f5d6be00640 1 -- 172.21.15.7:0/1110088169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f5d64095370 con 0x7f5d58004800 2024-04-25T11:10:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.741+0000 7f5d73400640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d500744e0 0x7f5d500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.742+0000 7f5d73400640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d500744e0 0x7f5d500769f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c032000 tx=0x7f5d5c02d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.745+0000 7f5d6be00640 1 -- 172.21.15.7:0/1110088169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d6405f720 con 0x7f5d58004800 2024-04-25T11:10:39.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.950+0000 7f5d7ae00640 1 -- 172.21.15.7:0/1110088169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5d74157890 con 0x7f5d58004800 2024-04-25T11:10:39.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.951+0000 7f5d6be00640 1 -- 172.21.15.7:0/1110088169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7f5d6405f020 con 0x7f5d58004800 2024-04-25T11:10:39.952 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:10:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.953+0000 7f5d7ae00640 1 -- 172.21.15.7:0/1110088169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d500744e0 msgr2=0x7f5d500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.953+0000 7f5d7ae00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d500744e0 0x7f5d500769f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c032000 tx=0x7f5d5c02d000 comp rx=0 tx=0).stop 2024-04-25T11:10:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.954+0000 7f5d7ae00640 1 -- 172.21.15.7:0/1110088169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 msgr2=0x7f5d74154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.954+0000 7f5d7ae00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 0x7f5d74154520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400f5a0 tx=0x7f5d6400fa60 comp rx=0 tx=0).stop 2024-04-25T11:10:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.954+0000 7f5d7ae00640 1 -- 172.21.15.7:0/1110088169 shutdown_connections 2024-04-25T11:10:39.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.954+0000 7f5d7ae00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d500744e0 0x7f5d500769f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.954+0000 7f5d7ae00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d74159c70 0x7f5d7415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.954+0000 7f5d7ae00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d74154a60 0x7f5d74154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.954+0000 7f5d7ae00640 1 --2- 172.21.15.7:0/1110088169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d58004800 0x7f5d74154520 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:39.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.954+0000 7f5d7ae00640 1 -- 172.21.15.7:0/1110088169 >> 172.21.15.7:0/1110088169 conn(0x7f5d740742c0 msgr2=0x7f5d7406d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:39.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.954+0000 7f5d7ae00640 1 -- 172.21.15.7:0/1110088169 shutdown_connections 2024-04-25T11:10:39.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:39.954+0000 7f5d7ae00640 1 -- 172.21.15.7:0/1110088169 wait complete. 2024-04-25T11:10:39.982 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:10:40.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.269+0000 7fed7ce00640 1 Processor -- start 2024-04-25T11:10:40.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.269+0000 7fed7ce00640 1 -- start start 2024-04-25T11:10:40.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.271+0000 7fed7ce00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed780b73b0 0x7fed780b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.271+0000 7fed7ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fed7806d710 0x7fed7806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.271+0000 7fed7ce00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed780b7d00 con 0x7fed780ce7b0 2024-04-25T11:10:40.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.271+0000 7fed7ce00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fed780d3230 con 0x7fed7806d710 2024-04-25T11:10:40.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.271+0000 7fed7ce00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fed780b26c0 con 0x7fed780b73b0 2024-04-25T11:10:40.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.272+0000 7fed77400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fed780ce7b0 0x7fed780b1fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48686/0 (socket says 172.21.15.7:48686) 2024-04-25T11:10:40.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.272+0000 7fed77400640 1 -- 172.21.15.7:0/113569031 learned_addr learned my addr 172.21.15.7:0/113569031 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:40.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.272+0000 7fed6fe00640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed780b73b0 0x7fed780b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.272+0000 7fed77e00640 1 --2- 172.21.15.7:0/113569031 >> v2:172.21.15.87:3300/0 conn(0x7fed7806d710 0x7fed7806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2313103662 0 0) 0x7fed68001110 con 0x7fed780ce7b0 2024-04-25T11:10:40.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed5c003520 con 0x7fed780ce7b0 2024-04-25T11:10:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3311943371 0 0) 0x7fed5c003520 con 0x7fed780ce7b0 2024-04-25T11:10:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed780b73b0 msgr2=0x7fed780b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed780b73b0 0x7fed780b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 >> v2:172.21.15.87:3300/0 conn(0x7fed7806d710 msgr2=0x7fed7806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 --2- 172.21.15.7:0/113569031 >> v2:172.21.15.87:3300/0 conn(0x7fed7806d710 0x7fed7806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed780d2620 con 0x7fed780ce7b0 2024-04-25T11:10:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fed68001840 con 0x7fed780ce7b0 2024-04-25T11:10:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 >> v1:172.21.15.7:6789/0 conn(0x7fed780ce7b0 legacy=0x7fed780b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c0039b0 0x7fed5c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed780b73b0 0x7fed5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed5c004d40 0x7fed5c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed780b26c0 con 0x7fed5c0039b0 2024-04-25T11:10:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c00b9c0 con 0x7fed5c004d40 2024-04-25T11:10:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.273+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fed5c00bb70 con 0x7fed780b73b0 2024-04-25T11:10:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.274+0000 7fed77400640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c0039b0 0x7fed5c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.274+0000 7fed6fe00640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed780b73b0 0x7fed5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.274+0000 7fed77e00640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed5c004d40 0x7fed5c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.274+0000 7fed77400640 1 -- 172.21.15.7:0/113569031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed780b73b0 msgr2=0x7fed5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.274+0000 7fed77400640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed780b73b0 0x7fed5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.274+0000 7fed77400640 1 -- 172.21.15.7:0/113569031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed5c004d40 msgr2=0x7fed5c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.274+0000 7fed77400640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed5c004d40 0x7fed5c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.274+0000 7fed77400640 1 -- 172.21.15.7:0/113569031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c003520 con 0x7fed5c0039b0 2024-04-25T11:10:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.275+0000 7fed77400640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c0039b0 0x7fed5c003e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c2f0 tx=0x7fed5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12237fad68cb481 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.276+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68035070 con 0x7fed5c0039b0 2024-04-25T11:10:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.276+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed680058d0 con 0x7fed5c0039b0 2024-04-25T11:10:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.276+0000 7fed76a00640 1 -- 172.21.15.7:0/113569031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68007e60 con 0x7fed5c0039b0 2024-04-25T11:10:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.276+0000 7fed7ce00640 1 -- 172.21.15.7:0/113569031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c0039b0 msgr2=0x7fed5c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.276+0000 7fed7ce00640 1 --2- 172.21.15.7:0/113569031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c0039b0 0x7fed5c003e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fed6800c2f0 tx=0x7fed5c005830 comp rx=0 tx=0).stop 2024-04-25T11:10:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.276+0000 7fed77e00640 1 -- 172.21.15.7:0/113569031 reap_dead start 2024-04-25T11:10:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.276+0000 7fed7ce00640 1 -- 172.21.15.7:0/113569031 shutdown_connections 2024-04-25T11:10:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.276+0000 7fed7ce00640 1 -- 172.21.15.7:0/113569031 >> 172.21.15.7:0/113569031 conn(0x7fed7809b540 msgr2=0x7fed7809b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.277+0000 7fed7ce00640 1 -- 172.21.15.7:0/113569031 shutdown_connections 2024-04-25T11:10:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.277+0000 7fed7ce00640 1 -- 172.21.15.7:0/113569031 wait complete. 2024-04-25T11:10:40.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.279+0000 7fed7ce00640 1 Processor -- start 2024-04-25T11:10:40.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.279+0000 7fed7ce00640 1 -- start start 2024-04-25T11:10:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.281+0000 7fed7ce00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed5c0039b0 0x7fed7815eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.281+0000 7fed7ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed780af3e0 0x7fed780caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.281+0000 7fed7ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780cb530 0x7fed7815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.281+0000 7fed7ce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed780d2ec0 con 0x7fed780cb530 2024-04-25T11:10:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.281+0000 7fed7ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fed780d2be0 con 0x7fed780af3e0 2024-04-25T11:10:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.281+0000 7fed7ce00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fed780ccca0 con 0x7fed5c0039b0 2024-04-25T11:10:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.281+0000 7fed77e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780cb530 0x7fed7815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.282+0000 7fed77e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780cb530 0x7fed7815d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59680/0 (socket says 172.21.15.7:59680) 2024-04-25T11:10:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.282+0000 7fed77e00640 1 -- 172.21.15.7:0/1083464343 learned_addr learned my addr 172.21.15.7:0/1083464343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.282+0000 7fed77400640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed5c0039b0 0x7fed7815eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.282+0000 7fed6fe00640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed780af3e0 0x7fed780caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.282+0000 7fed77e00640 1 -- 172.21.15.7:0/1083464343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed5c0039b0 msgr2=0x7fed7815eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.282+0000 7fed77e00640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed5c0039b0 0x7fed7815eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.282+0000 7fed77e00640 1 -- 172.21.15.7:0/1083464343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed780af3e0 msgr2=0x7fed780caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.282+0000 7fed77e00640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed780af3e0 0x7fed780caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.282+0000 7fed77e00640 1 -- 172.21.15.7:0/1083464343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7815f400 con 0x7fed780cb530 2024-04-25T11:10:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.282+0000 7fed77e00640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780cb530 0x7fed7815d4a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fed780b85b0 tx=0x7fed700115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.283+0000 7fed6f400640 1 -- 172.21.15.7:0/1083464343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70011de0 con 0x7fed780cb530 2024-04-25T11:10:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.283+0000 7fed6f400640 1 -- 172.21.15.7:0/1083464343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7001b070 con 0x7fed780cb530 2024-04-25T11:10:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.283+0000 7fed6f400640 1 -- 172.21.15.7:0/1083464343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70003990 con 0x7fed780cb530 2024-04-25T11:10:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.283+0000 7fed7ce00640 1 -- 172.21.15.7:0/1083464343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed780b1750 con 0x7fed780cb530 2024-04-25T11:10:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.283+0000 7fed7ce00640 1 -- 172.21.15.7:0/1083464343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed780ca230 con 0x7fed780cb530 2024-04-25T11:10:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.284+0000 7fed7ce00640 1 -- 172.21.15.7:0/1083464343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed78050740 con 0x7fed780cb530 2024-04-25T11:10:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.285+0000 7fed6f400640 1 -- 172.21.15.7:0/1083464343 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed70005d10 con 0x7fed780cb530 2024-04-25T11:10:40.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.286+0000 7fed6f400640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed54074530 0x7fed54076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.286+0000 7fed6f400640 1 -- 172.21.15.7:0/1083464343 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fed700958d0 con 0x7fed780cb530 2024-04-25T11:10:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.287+0000 7fed77400640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed54074530 0x7fed54076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.288+0000 7fed77400640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed54074530 0x7fed54076a40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed68032000 tx=0x7fed68009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.291+0000 7fed6f400640 1 -- 172.21.15.7:0/1083464343 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed70025020 con 0x7fed780cb530 2024-04-25T11:10:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.494+0000 7fed7ce00640 1 -- 172.21.15.7:0/1083464343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fed780af7f0 con 0x7fed780cb530 2024-04-25T11:10:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.495+0000 7fed6f400640 1 -- 172.21.15.7:0/1083464343 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7fed7005fd90 con 0x7fed780cb530 2024-04-25T11:10:40.496 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.497+0000 7fed7ce00640 1 -- 172.21.15.7:0/1083464343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed54074530 msgr2=0x7fed54076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.497+0000 7fed7ce00640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed54074530 0x7fed54076a40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fed68032000 tx=0x7fed68009030 comp rx=0 tx=0).stop 2024-04-25T11:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.497+0000 7fed7ce00640 1 -- 172.21.15.7:0/1083464343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780cb530 msgr2=0x7fed7815d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.497+0000 7fed7ce00640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780cb530 0x7fed7815d4a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fed780b85b0 tx=0x7fed700115a0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.498+0000 7fed7ce00640 1 -- 172.21.15.7:0/1083464343 shutdown_connections 2024-04-25T11:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.498+0000 7fed7ce00640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed54074530 0x7fed54076a40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.498+0000 7fed7ce00640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780cb530 0x7fed7815d4a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.498+0000 7fed7ce00640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed780af3e0 0x7fed780caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.498+0000 7fed7ce00640 1 --2- 172.21.15.7:0/1083464343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed5c0039b0 0x7fed7815eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.498+0000 7fed7ce00640 1 -- 172.21.15.7:0/1083464343 >> 172.21.15.7:0/1083464343 conn(0x7fed7809b540 msgr2=0x7fed780b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:40.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.498+0000 7fed7ce00640 1 -- 172.21.15.7:0/1083464343 shutdown_connections 2024-04-25T11:10:40.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.498+0000 7fed7ce00640 1 -- 172.21.15.7:0/1083464343 wait complete. 2024-04-25T11:10:40.526 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:10:40.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:10:40.746 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:10:40.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.799+0000 7f5448400640 1 Processor -- start 2024-04-25T11:10:40.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.799+0000 7f5448400640 1 -- start start 2024-04-25T11:10:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.802+0000 7f5448400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f544006d580 0x7f544006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.802+0000 7f5448400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f544006f1e0 0x7f544006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.802+0000 7f5448400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f544006ded0 con 0x7f544015f180 2024-04-25T11:10:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.802+0000 7f5448400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f544015f590 con 0x7f544006d580 2024-04-25T11:10:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.802+0000 7f5448400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5440158ae0 con 0x7f544006f1e0 2024-04-25T11:10:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.802+0000 7f5445a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f544015f180 0x7f5440161c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48700/0 (socket says 172.21.15.7:48700) 2024-04-25T11:10:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.802+0000 7f5445a00640 1 -- 172.21.15.7:0/3984206727 learned_addr learned my addr 172.21.15.7:0/3984206727 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.802+0000 7f5445000640 1 --2- 172.21.15.7:0/3984206727 >> v2:172.21.15.87:3300/0 conn(0x7f544006d580 0x7f544006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.802+0000 7f5446400640 1 --2- 172.21.15.7:0/3984206727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f544006f1e0 0x7f544006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.803+0000 7f543fe00640 1 -- 172.21.15.7:0/3984206727 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 478833111 0 0) 0x7f5434001110 con 0x7f544015f180 2024-04-25T11:10:40.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.803+0000 7f543fe00640 1 -- 172.21.15.7:0/3984206727 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5428003520 con 0x7f544015f180 2024-04-25T11:10:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.803+0000 7f5445000640 1 -- 172.21.15.7:0/3984206727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f544006f1e0 msgr2=0x7f544006f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.803+0000 7f5445000640 1 --2- 172.21.15.7:0/3984206727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f544006f1e0 0x7f544006f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.803+0000 7f5445000640 1 -- 172.21.15.7:0/3984206727 >> v1:172.21.15.7:6789/0 conn(0x7f544015f180 legacy=0x7f5440161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:40.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.803+0000 7f5445000640 1 -- 172.21.15.7:0/3984206727 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5440162700 con 0x7f544006d580 2024-04-25T11:10:40.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.804+0000 7f5445000640 1 --2- 172.21.15.7:0/3984206727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f544006d580 0x7f544006d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f542c0097d0 tx=0x7f542c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf39f4e66f158f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:40.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.805+0000 7f543fe00640 1 -- 172.21.15.7:0/3984206727 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c037070 con 0x7f544006d580 2024-04-25T11:10:40.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.805+0000 7f543fe00640 1 -- 172.21.15.7:0/3984206727 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f542c032440 con 0x7f544006d580 2024-04-25T11:10:40.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.805+0000 7f543fe00640 1 -- 172.21.15.7:0/3984206727 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c032810 con 0x7f544006d580 2024-04-25T11:10:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.805+0000 7f5448400640 1 -- 172.21.15.7:0/3984206727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f544006d580 msgr2=0x7f544006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.805+0000 7f5448400640 1 --2- 172.21.15.7:0/3984206727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f544006d580 0x7f544006d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f542c0097d0 tx=0x7f542c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.806+0000 7f5448400640 1 -- 172.21.15.7:0/3984206727 shutdown_connections 2024-04-25T11:10:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.806+0000 7f5448400640 1 --2- 172.21.15.7:0/3984206727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f544006f1e0 0x7f544006f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.806+0000 7f5448400640 1 --2- 172.21.15.7:0/3984206727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f544006d580 0x7f544006d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.806+0000 7f5448400640 1 -- 172.21.15.7:0/3984206727 >> 172.21.15.7:0/3984206727 conn(0x7f54400732b0 msgr2=0x7f54400736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.806+0000 7f5448400640 1 -- 172.21.15.7:0/3984206727 shutdown_connections 2024-04-25T11:10:40.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.806+0000 7f5448400640 1 -- 172.21.15.7:0/3984206727 wait complete. 2024-04-25T11:10:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.808+0000 7f5448400640 1 Processor -- start 2024-04-25T11:10:40.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.808+0000 7f5448400640 1 -- start start 2024-04-25T11:10:40.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.810+0000 7f5448400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f544006d580 0x7f54401e8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.810+0000 7f5448400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544006f1e0 0x7f5440149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.810+0000 7f5448400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f544015f180 0x7f5440160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.810+0000 7f5448400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f544015ec60 con 0x7f544006f1e0 2024-04-25T11:10:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.810+0000 7f5448400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f544015e980 con 0x7f544015f180 2024-04-25T11:10:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.810+0000 7f5448400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f544015e670 con 0x7f544006d580 2024-04-25T11:10:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.810+0000 7f5445000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544006f1e0 0x7f5440149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.810+0000 7f5445000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544006f1e0 0x7f5440149590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59684/0 (socket says 172.21.15.7:59684) 2024-04-25T11:10:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.810+0000 7f5445000640 1 -- 172.21.15.7:0/2866450972 learned_addr learned my addr 172.21.15.7:0/2866450972 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.811+0000 7f5445a00640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f544006d580 0x7f54401e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.811+0000 7f5446400640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f544015f180 0x7f5440160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.811+0000 7f5445000640 1 -- 172.21.15.7:0/2866450972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f544006d580 msgr2=0x7f54401e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.811+0000 7f5445000640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f544006d580 0x7f54401e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.811+0000 7f5445000640 1 -- 172.21.15.7:0/2866450972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f544015f180 msgr2=0x7f5440160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.811+0000 7f5445000640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f544015f180 0x7f5440160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.811+0000 7f5445000640 1 -- 172.21.15.7:0/2866450972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54401e9820 con 0x7f544006f1e0 2024-04-25T11:10:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.811+0000 7f5445000640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544006f1e0 0x7f5440149590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f542c0098f0 tx=0x7f542c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.812+0000 7f543d600640 1 -- 172.21.15.7:0/2866450972 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c041020 con 0x7f544006f1e0 2024-04-25T11:10:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.812+0000 7f543d600640 1 -- 172.21.15.7:0/2866450972 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f542c03d6e0 con 0x7f544006f1e0 2024-04-25T11:10:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.812+0000 7f543d600640 1 -- 172.21.15.7:0/2866450972 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c037040 con 0x7f544006f1e0 2024-04-25T11:10:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.812+0000 7f5448400640 1 -- 172.21.15.7:0/2866450972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54401e9b00 con 0x7f544006f1e0 2024-04-25T11:10:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.812+0000 7f5448400640 1 -- 172.21.15.7:0/2866450972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54401ea0c0 con 0x7f544006f1e0 2024-04-25T11:10:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.813+0000 7f5448400640 1 -- 172.21.15.7:0/2866450972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5440050740 con 0x7f544006f1e0 2024-04-25T11:10:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.816+0000 7f543d600640 1 -- 172.21.15.7:0/2866450972 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f542c044020 con 0x7f544006f1e0 2024-04-25T11:10:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.817+0000 7f543d600640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5420074530 0x7f5420076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.817+0000 7f543d600640 1 -- 172.21.15.7:0/2866450972 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f542c0b62b0 con 0x7f544006f1e0 2024-04-25T11:10:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.818+0000 7f5445a00640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5420074530 0x7f5420076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:40.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.818+0000 7f5445a00640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5420074530 0x7f5420076a40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5434005880 tx=0x7f5434009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:40.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:40.821+0000 7f543d600640 1 -- 172.21.15.7:0/2866450972 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f542c04b030 con 0x7f544006f1e0 2024-04-25T11:10:41.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.039+0000 7f5448400640 1 -- 172.21.15.7:0/2866450972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f54401e9570 con 0x7f544006f1e0 2024-04-25T11:10:41.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.040+0000 7f543d600640 1 -- 172.21.15.7:0/2866450972 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f542c080220 con 0x7f544006f1e0 2024-04-25T11:10:41.042 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:10:41.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.042+0000 7f5448400640 1 -- 172.21.15.7:0/2866450972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5420074530 msgr2=0x7f5420076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:41.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.042+0000 7f5448400640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5420074530 0x7f5420076a40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5434005880 tx=0x7f5434009040 comp rx=0 tx=0).stop 2024-04-25T11:10:41.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.043+0000 7f5448400640 1 -- 172.21.15.7:0/2866450972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544006f1e0 msgr2=0x7f5440149590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:41.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.043+0000 7f5448400640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544006f1e0 0x7f5440149590 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f542c0098f0 tx=0x7f542c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.043+0000 7f5448400640 1 -- 172.21.15.7:0/2866450972 shutdown_connections 2024-04-25T11:10:41.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.043+0000 7f5448400640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5420074530 0x7f5420076a40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.043+0000 7f5448400640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f544015f180 0x7f5440160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.043+0000 7f5448400640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f544006f1e0 0x7f5440149590 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.043+0000 7f5448400640 1 --2- 172.21.15.7:0/2866450972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f544006d580 0x7f54401e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.043+0000 7f5448400640 1 -- 172.21.15.7:0/2866450972 >> 172.21.15.7:0/2866450972 conn(0x7f54400732b0 msgr2=0x7f54400684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:41.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.043+0000 7f5448400640 1 -- 172.21.15.7:0/2866450972 shutdown_connections 2024-04-25T11:10:41.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.043+0000 7f5448400640 1 -- 172.21.15.7:0/2866450972 wait complete. 2024-04-25T11:10:41.070 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:10:41.071 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:10:41.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:41.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.408+0000 7f360f200640 1 Processor -- start 2024-04-25T11:10:41.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.408+0000 7f360f200640 1 -- start start 2024-04-25T11:10:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.410+0000 7f360f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3610164690 0x7f3610070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.410+0000 7f360f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3610156cd0 0x7f3610163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.410+0000 7f360f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36101570e0 0x7f3610164010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.410+0000 7f360f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f36100b5080 con 0x7f3610156cd0 2024-04-25T11:10:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.410+0000 7f360f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f36100b5360 con 0x7f3610164690 2024-04-25T11:10:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.410+0000 7f360f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f36100b5640 con 0x7f36101570e0 2024-04-25T11:10:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.411+0000 7f360d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3610156cd0 0x7f3610163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.411+0000 7f360d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3610156cd0 0x7f3610163ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59698/0 (socket says 172.21.15.7:59698) 2024-04-25T11:10:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.411+0000 7f360d400640 1 -- 172.21.15.7:0/1793201160 learned_addr learned my addr 172.21.15.7:0/1793201160 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:41.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.411+0000 7f360e800640 1 --2- 172.21.15.7:0/1793201160 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36101570e0 0x7f3610164010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.411+0000 7f360d400640 1 -- 172.21.15.7:0/1793201160 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36101570e0 msgr2=0x7f3610164010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:41.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.411+0000 7f360d400640 1 --2- 172.21.15.7:0/1793201160 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36101570e0 0x7f3610164010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.411+0000 7f360d400640 1 -- 172.21.15.7:0/1793201160 >> v2:172.21.15.87:3300/0 conn(0x7f3610164690 msgr2=0x7f3610070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:41.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.411+0000 7f360d400640 1 --2- 172.21.15.7:0/1793201160 >> v2:172.21.15.87:3300/0 conn(0x7f3610164690 0x7f3610070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.411+0000 7f360d400640 1 -- 172.21.15.7:0/1793201160 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36100b5920 con 0x7f3610156cd0 2024-04-25T11:10:41.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.413+0000 7f360d400640 1 --2- 172.21.15.7:0/1793201160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3610156cd0 0x7f3610163ad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f361006f470 tx=0x7f35fc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db4b0288c62542a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:41.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.413+0000 7f360ca00640 1 -- 172.21.15.7:0/1793201160 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc037070 con 0x7f3610156cd0 2024-04-25T11:10:41.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.413+0000 7f360ca00640 1 -- 172.21.15.7:0/1793201160 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35fc032440 con 0x7f3610156cd0 2024-04-25T11:10:41.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.413+0000 7f360ca00640 1 -- 172.21.15.7:0/1793201160 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc032810 con 0x7f3610156cd0 2024-04-25T11:10:41.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.414+0000 7f360f200640 1 -- 172.21.15.7:0/1793201160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3610156cd0 msgr2=0x7f3610163ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:41.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.414+0000 7f360f200640 1 --2- 172.21.15.7:0/1793201160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3610156cd0 0x7f3610163ad0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f361006f470 tx=0x7f35fc02a140 comp rx=0 tx=0).stop 2024-04-25T11:10:41.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.414+0000 7f360f200640 1 -- 172.21.15.7:0/1793201160 shutdown_connections 2024-04-25T11:10:41.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.414+0000 7f360f200640 1 --2- 172.21.15.7:0/1793201160 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36101570e0 0x7f3610164010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.414+0000 7f360f200640 1 --2- 172.21.15.7:0/1793201160 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3610156cd0 0x7f3610163ad0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.414+0000 7f360f200640 1 --2- 172.21.15.7:0/1793201160 >> v2:172.21.15.87:3300/0 conn(0x7f3610164690 0x7f3610070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.414+0000 7f360f200640 1 -- 172.21.15.7:0/1793201160 >> 172.21.15.7:0/1793201160 conn(0x7f36100b3820 msgr2=0x7f36100b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:41.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.414+0000 7f360f200640 1 -- 172.21.15.7:0/1793201160 shutdown_connections 2024-04-25T11:10:41.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.414+0000 7f360f200640 1 -- 172.21.15.7:0/1793201160 wait complete. 2024-04-25T11:10:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.417+0000 7f360f200640 1 Processor -- start 2024-04-25T11:10:41.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.417+0000 7f360f200640 1 -- start start 2024-04-25T11:10:41.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.418+0000 7f360f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3610156cd0 0x7f3610143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.418+0000 7f360f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36101570e0 0x7f36101447c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.418+0000 7f360f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3610164690 0x7f3610144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.418+0000 7f360f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36100b0210 con 0x7f3610156cd0 2024-04-25T11:10:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.418+0000 7f360f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3610070f60 con 0x7f36101570e0 2024-04-25T11:10:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.418+0000 7f360f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f36100b0390 con 0x7f3610164690 2024-04-25T11:10:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.419+0000 7f360de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3610156cd0 0x7f3610143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.419+0000 7f360de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3610156cd0 0x7f3610143e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59704/0 (socket says 172.21.15.7:59704) 2024-04-25T11:10:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.419+0000 7f360de00640 1 -- 172.21.15.7:0/2137801436 learned_addr learned my addr 172.21.15.7:0/2137801436 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:41.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.419+0000 7f360de00640 1 -- 172.21.15.7:0/2137801436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3610164690 msgr2=0x7f3610144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:41.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.419+0000 7f360de00640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3610164690 0x7f3610144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.419+0000 7f360de00640 1 -- 172.21.15.7:0/2137801436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36101570e0 msgr2=0x7f36101447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:41.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.419+0000 7f360de00640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36101570e0 0x7f36101447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.419+0000 7f360de00640 1 -- 172.21.15.7:0/2137801436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361015ec00 con 0x7f3610156cd0 2024-04-25T11:10:41.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.419+0000 7f360de00640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3610156cd0 0x7f3610143e50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f360000ba80 tx=0x7f360000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.420+0000 7f3606000640 1 -- 172.21.15.7:0/2137801436 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600019070 con 0x7f3610156cd0 2024-04-25T11:10:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.420+0000 7f3606000640 1 -- 172.21.15.7:0/2137801436 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360000cc20 con 0x7f3610156cd0 2024-04-25T11:10:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.420+0000 7f3606000640 1 -- 172.21.15.7:0/2137801436 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600014440 con 0x7f3610156cd0 2024-04-25T11:10:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.420+0000 7f360f200640 1 -- 172.21.15.7:0/2137801436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361015eee0 con 0x7f3610156cd0 2024-04-25T11:10:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.420+0000 7f360f200640 1 -- 172.21.15.7:0/2137801436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361015f4a0 con 0x7f3610156cd0 2024-04-25T11:10:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.421+0000 7f360f200640 1 -- 172.21.15.7:0/2137801436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3610050740 con 0x7f3610156cd0 2024-04-25T11:10:41.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.422+0000 7f3606000640 1 -- 172.21.15.7:0/2137801436 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f360001c070 con 0x7f3610156cd0 2024-04-25T11:10:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.424+0000 7f3606000640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35f00744b0 0x7f35f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.424+0000 7f3606000640 1 -- 172.21.15.7:0/2137801436 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f36000952c0 con 0x7f3610156cd0 2024-04-25T11:10:41.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.424+0000 7f360d400640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35f00744b0 0x7f35f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.425+0000 7f360d400640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35f00744b0 0x7f35f00769c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f35fc034000 tx=0x7f35fc02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.429+0000 7f3606000640 1 -- 172.21.15.7:0/2137801436 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f360005f020 con 0x7f3610156cd0 2024-04-25T11:10:41.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.633+0000 7f360f200640 1 -- 172.21.15.7:0/2137801436 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36101568a0 con 0x7f3610156cd0 2024-04-25T11:10:41.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.635+0000 7f3606000640 1 -- 172.21.15.7:0/2137801436 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18418 (secure 0 0 0) 0x7f3600063320 con 0x7f3610156cd0 2024-04-25T11:10:41.637 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:41.637 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:28.084739+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:41.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.638+0000 7f360f200640 1 -- 172.21.15.7:0/2137801436 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35f00744b0 msgr2=0x7f35f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:41.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.638+0000 7f360f200640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35f00744b0 0x7f35f00769c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f35fc034000 tx=0x7f35fc02c000 comp rx=0 tx=0).stop 2024-04-25T11:10:41.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.638+0000 7f360f200640 1 -- 172.21.15.7:0/2137801436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3610156cd0 msgr2=0x7f3610143e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:41.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.638+0000 7f360f200640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3610156cd0 0x7f3610143e50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f360000ba80 tx=0x7f360000bf40 comp rx=0 tx=0).stop 2024-04-25T11:10:41.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.638+0000 7f360f200640 1 -- 172.21.15.7:0/2137801436 shutdown_connections 2024-04-25T11:10:41.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.638+0000 7f360f200640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35f00744b0 0x7f35f00769c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.638+0000 7f360f200640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3610164690 0x7f3610144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.638+0000 7f360f200640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36101570e0 0x7f36101447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.638+0000 7f360f200640 1 --2- 172.21.15.7:0/2137801436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3610156cd0 0x7f3610143e50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.639+0000 7f360f200640 1 -- 172.21.15.7:0/2137801436 >> 172.21.15.7:0/2137801436 conn(0x7f36100b3820 msgr2=0x7f3610154910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:41.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.639+0000 7f360f200640 1 -- 172.21.15.7:0/2137801436 shutdown_connections 2024-04-25T11:10:41.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.639+0000 7f360f200640 1 -- 172.21.15.7:0/2137801436 wait complete. 2024-04-25T11:10:41.665 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:10:41.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:10:41.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.942+0000 7f48f6e00640 1 Processor -- start 2024-04-25T11:10:41.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.943+0000 7f48f6e00640 1 -- start start 2024-04-25T11:10:41.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.945+0000 7f48f6e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f48f015a7a0 0x7f48f0161160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.945+0000 7f48f6e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f48f006c570 0x7f48f006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.945+0000 7f48f6e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f006cfc0 con 0x7f48f00b7440 2024-04-25T11:10:41.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.945+0000 7f48f6e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f48f0160cd0 con 0x7f48f006c570 2024-04-25T11:10:41.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.945+0000 7f48f6e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f48f006d2a0 con 0x7f48f015a7a0 2024-04-25T11:10:41.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.945+0000 7f48f6400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f48f00b7440 0x7f48f00b7850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48704/0 (socket says 172.21.15.7:48704) 2024-04-25T11:10:41.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.945+0000 7f48f6400640 1 -- 172.21.15.7:0/951061881 learned_addr learned my addr 172.21.15.7:0/951061881 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:41.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.945+0000 7f48f5a00640 1 --2- 172.21.15.7:0/951061881 >> v2:172.21.15.87:3300/0 conn(0x7f48f015a7a0 0x7f48f0161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.945+0000 7f48f5000640 1 --2- 172.21.15.7:0/951061881 >> v2:172.21.15.7:3300/0 conn(0x7f48f006c570 0x7f48f006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 789541062 0 0) 0x7f48f006cfc0 con 0x7f48f00b7440 2024-04-25T11:10:41.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48d8003520 con 0x7f48f00b7440 2024-04-25T11:10:41.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1926444842 0 0) 0x7f48d8003520 con 0x7f48f00b7440 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 >> v2:172.21.15.87:3300/0 conn(0x7f48f015a7a0 msgr2=0x7f48f0161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 --2- 172.21.15.7:0/951061881 >> v2:172.21.15.87:3300/0 conn(0x7f48f015a7a0 0x7f48f0161160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 >> v2:172.21.15.7:3300/0 conn(0x7f48f006c570 msgr2=0x7f48f006c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 --2- 172.21.15.7:0/951061881 >> v2:172.21.15.7:3300/0 conn(0x7f48f006c570 0x7f48f006c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f0160960 con 0x7f48f00b7440 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f48e80019f0 con 0x7f48f00b7440 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 >> v1:172.21.15.7:6789/0 conn(0x7f48f00b7440 legacy=0x7f48f00b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 --2- 172.21.15.7:0/951061881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d8003a00 0x7f48d8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 --2- 172.21.15.7:0/951061881 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d8004800 0x7f48d8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 --2- 172.21.15.7:0/951061881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48d8005150 0x7f48d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.946+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0160cd0 con 0x7f48d8003a00 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.947+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48f006d2a0 con 0x7f48d8005150 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.947+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48d800c140 con 0x7f48d8004800 2024-04-25T11:10:41.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.947+0000 7f48f5a00640 1 --2- 172.21.15.7:0/951061881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d8003a00 0x7f48d8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.947+0000 7f48f5000640 1 --2- 172.21.15.7:0/951061881 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d8004800 0x7f48d8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.947+0000 7f48f5a00640 1 -- 172.21.15.7:0/951061881 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d8004800 msgr2=0x7f48d8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.947+0000 7f48f5a00640 1 --2- 172.21.15.7:0/951061881 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d8004800 0x7f48d8004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.947+0000 7f48f5a00640 1 -- 172.21.15.7:0/951061881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48d8005150 msgr2=0x7f48d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.947+0000 7f48f5a00640 1 --2- 172.21.15.7:0/951061881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48d8005150 0x7f48d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.947+0000 7f48f5a00640 1 -- 172.21.15.7:0/951061881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d8003520 con 0x7f48d8003a00 2024-04-25T11:10:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.947+0000 7f48f6400640 1 -- 172.21.15.7:0/951061881 reap_dead start 2024-04-25T11:10:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.948+0000 7f48f5a00640 1 --2- 172.21.15.7:0/951061881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d8003a00 0x7f48d8003e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f48f00d5990 tx=0x7f48d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81dec7e08ada078 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.948+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4036070 con 0x7f48d8003a00 2024-04-25T11:10:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.949+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e4031400 con 0x7f48d8003a00 2024-04-25T11:10:41.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.949+0000 7f48efe00640 1 -- 172.21.15.7:0/951061881 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4009690 con 0x7f48d8003a00 2024-04-25T11:10:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.949+0000 7f48f6e00640 1 -- 172.21.15.7:0/951061881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d8003a00 msgr2=0x7f48d8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.949+0000 7f48f6e00640 1 --2- 172.21.15.7:0/951061881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d8003a00 0x7f48d8003e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f48f00d5990 tx=0x7f48d8005830 comp rx=0 tx=0).stop 2024-04-25T11:10:41.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.949+0000 7f48f6e00640 1 -- 172.21.15.7:0/951061881 shutdown_connections 2024-04-25T11:10:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.949+0000 7f48f6e00640 1 --2- 172.21.15.7:0/951061881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d8003a00 0x7f48d8003e70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.949+0000 7f48f6e00640 1 -- 172.21.15.7:0/951061881 >> 172.21.15.7:0/951061881 conn(0x7f48f00d59c0 msgr2=0x7f48f00d5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.949+0000 7f48f6e00640 1 -- 172.21.15.7:0/951061881 shutdown_connections 2024-04-25T11:10:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.949+0000 7f48f6e00640 1 -- 172.21.15.7:0/951061881 wait complete. 2024-04-25T11:10:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.952+0000 7f48f6e00640 1 Processor -- start 2024-04-25T11:10:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.952+0000 7f48f6e00640 1 -- start start 2024-04-25T11:10:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.953+0000 7f48f6e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d8003a00 0x7f48f00cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.953+0000 7f48f6e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0159e60 0x7f48f0162c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.953+0000 7f48f6e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f01631a0 0x7f48f00cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.953+0000 7f48f6e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48f00d09f0 con 0x7f48f01631a0 2024-04-25T11:10:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.953+0000 7f48f6e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48f00d0870 con 0x7f48f0159e60 2024-04-25T11:10:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.953+0000 7f48f6e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48f006c980 con 0x7f48d8003a00 2024-04-25T11:10:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.954+0000 7f48f6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f01631a0 0x7f48f00cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.954+0000 7f48f6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f01631a0 0x7f48f00cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59736/0 (socket says 172.21.15.7:59736) 2024-04-25T11:10:41.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.954+0000 7f48f6400640 1 -- 172.21.15.7:0/1045368022 learned_addr learned my addr 172.21.15.7:0/1045368022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.954+0000 7f48f5a00640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d8003a00 0x7f48f00cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.954+0000 7f48f5000640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0159e60 0x7f48f0162c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.954+0000 7f48f6400640 1 -- 172.21.15.7:0/1045368022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d8003a00 msgr2=0x7f48f00cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.954+0000 7f48f6400640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d8003a00 0x7f48f00cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.954+0000 7f48f6400640 1 -- 172.21.15.7:0/1045368022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0159e60 msgr2=0x7f48f0162c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.954+0000 7f48f6400640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0159e60 0x7f48f0162c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.954+0000 7f48f6400640 1 -- 172.21.15.7:0/1045368022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f01635b0 con 0x7f48f01631a0 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.954+0000 7f48f6400640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f01631a0 0x7f48f00cca00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48f0164d20 tx=0x7f48e800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.955+0000 7f48ed600640 1 -- 172.21.15.7:0/1045368022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e800ce70 con 0x7f48f01631a0 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.955+0000 7f48ed600640 1 -- 172.21.15.7:0/1045368022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e800a640 con 0x7f48f01631a0 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.955+0000 7f48ed600640 1 -- 172.21.15.7:0/1045368022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e800aa10 con 0x7f48f01631a0 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.955+0000 7f48f6e00640 1 -- 172.21.15.7:0/1045368022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f00cef40 con 0x7f48f01631a0 2024-04-25T11:10:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.955+0000 7f48f6e00640 1 -- 172.21.15.7:0/1045368022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f01f7070 con 0x7f48f01631a0 2024-04-25T11:10:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.956+0000 7f48f6e00640 1 -- 172.21.15.7:0/1045368022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f00cf1c0 con 0x7f48f01631a0 2024-04-25T11:10:41.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.957+0000 7f48ed600640 1 -- 172.21.15.7:0/1045368022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f48e800abb0 con 0x7f48f01631a0 2024-04-25T11:10:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.959+0000 7f48ed600640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d00744b0 0x7f48d00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.959+0000 7f48ed600640 1 -- 172.21.15.7:0/1045368022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f48e8094e50 con 0x7f48f01631a0 2024-04-25T11:10:41.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.959+0000 7f48f5a00640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d00744b0 0x7f48d00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:41.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.960+0000 7f48f5a00640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d00744b0 0x7f48d00769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48e4033000 tx=0x7f48e4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:41.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:41.963+0000 7f48ed600640 1 -- 172.21.15.7:0/1045368022 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48e801a020 con 0x7f48f01631a0 2024-04-25T11:10:42.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.163+0000 7f48f6e00640 1 -- 172.21.15.7:0/1045368022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f48f00b6500 con 0x7f48f01631a0 2024-04-25T11:10:42.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.164+0000 7f48ed600640 1 -- 172.21.15.7:0/1045368022 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f48f01635b0 con 0x7f48f01631a0 2024-04-25T11:10:42.166 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:10:42.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 -- 172.21.15.7:0/1045368022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d00744b0 msgr2=0x7f48d00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:42.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d00744b0 0x7f48d00769c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48e4033000 tx=0x7f48e4008030 comp rx=0 tx=0).stop 2024-04-25T11:10:42.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 -- 172.21.15.7:0/1045368022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f01631a0 msgr2=0x7f48f00cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:42.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f01631a0 0x7f48f00cca00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48f0164d20 tx=0x7f48e800cc20 comp rx=0 tx=0).stop 2024-04-25T11:10:42.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 -- 172.21.15.7:0/1045368022 shutdown_connections 2024-04-25T11:10:42.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d00744b0 0x7f48d00769c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f01631a0 0x7f48f00cca00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48f0159e60 0x7f48f0162c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 --2- 172.21.15.7:0/1045368022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d8003a00 0x7f48f00cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 -- 172.21.15.7:0/1045368022 >> 172.21.15.7:0/1045368022 conn(0x7f48f00d59c0 msgr2=0x7f48f0163a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:42.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 -- 172.21.15.7:0/1045368022 shutdown_connections 2024-04-25T11:10:42.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.167+0000 7f48f6e00640 1 -- 172.21.15.7:0/1045368022 wait complete. 2024-04-25T11:10:42.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:10:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.459+0000 7f5302000640 1 Processor -- start 2024-04-25T11:10:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.459+0000 7f5302000640 1 -- start start 2024-04-25T11:10:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.461+0000 7f5302000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52fc157320 0x7f52fc164710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.462+0000 7f5302000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52fc0b1390 con 0x7f52fc06f3a0 2024-04-25T11:10:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.462+0000 7f5302000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52fc0ae880 con 0x7f52fc0b6720 2024-04-25T11:10:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.462+0000 7f5302000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f52fc160a60 con 0x7f52fc157320 2024-04-25T11:10:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.462+0000 7f52fbe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f52fc06f3a0 0x7f52fc06f7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48710/0 (socket says 172.21.15.7:48710) 2024-04-25T11:10:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.462+0000 7f52fbe00640 1 -- 172.21.15.7:0/2796943228 learned_addr learned my addr 172.21.15.7:0/2796943228 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.462+0000 7f52faa00640 1 --2- 172.21.15.7:0/2796943228 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52fc157320 0x7f52fc164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.463+0000 7f52fa000640 1 -- 172.21.15.7:0/2796943228 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2446499113 0 0) 0x7f52fc0b1390 con 0x7f52fc06f3a0 2024-04-25T11:10:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.463+0000 7f52fa000640 1 -- 172.21.15.7:0/2796943228 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52e0003520 con 0x7f52fc06f3a0 2024-04-25T11:10:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.463+0000 7f52faa00640 1 -- 172.21.15.7:0/2796943228 >> v1:172.21.15.87:6789/0 conn(0x7f52fc0b6720 legacy=0x7f52fc160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.463+0000 7f52faa00640 1 -- 172.21.15.7:0/2796943228 >> v1:172.21.15.7:6789/0 conn(0x7f52fc06f3a0 legacy=0x7f52fc06f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:42.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.463+0000 7f52faa00640 1 -- 172.21.15.7:0/2796943228 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc160710 con 0x7f52fc157320 2024-04-25T11:10:42.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.464+0000 7f52faa00640 1 --2- 172.21.15.7:0/2796943228 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52fc157320 0x7f52fc164710 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f52fc160be0 tx=0x7f52e802a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a6c623c19b14f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.465+0000 7f52fa000640 1 -- 172.21.15.7:0/2796943228 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8037070 con 0x7f52fc157320 2024-04-25T11:10:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.465+0000 7f52fa000640 1 -- 172.21.15.7:0/2796943228 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e8032440 con 0x7f52fc157320 2024-04-25T11:10:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.465+0000 7f52fa000640 1 -- 172.21.15.7:0/2796943228 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8032810 con 0x7f52fc157320 2024-04-25T11:10:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.466+0000 7f5302000640 1 -- 172.21.15.7:0/2796943228 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52fc157320 msgr2=0x7f52fc164710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.466+0000 7f5302000640 1 --2- 172.21.15.7:0/2796943228 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52fc157320 0x7f52fc164710 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f52fc160be0 tx=0x7f52e802a1f0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.466+0000 7f5302000640 1 -- 172.21.15.7:0/2796943228 shutdown_connections 2024-04-25T11:10:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.466+0000 7f5302000640 1 --2- 172.21.15.7:0/2796943228 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52fc157320 0x7f52fc164710 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.466+0000 7f5302000640 1 -- 172.21.15.7:0/2796943228 >> 172.21.15.7:0/2796943228 conn(0x7f52fc0742c0 msgr2=0x7f52fc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.466+0000 7f5302000640 1 -- 172.21.15.7:0/2796943228 shutdown_connections 2024-04-25T11:10:42.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.466+0000 7f5302000640 1 -- 172.21.15.7:0/2796943228 wait complete. 2024-04-25T11:10:42.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.468+0000 7f5302000640 1 Processor -- start 2024-04-25T11:10:42.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.468+0000 7f5302000640 1 -- start start 2024-04-25T11:10:42.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.470+0000 7f5302000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc06f3a0 0x7f52fc149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:42.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.470+0000 7f5302000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52fc0b6720 0x7f52fc159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:42.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.470+0000 7f5302000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52fc157320 0x7f52fc159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:42.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.470+0000 7f5302000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc0aee00 con 0x7f52fc06f3a0 2024-04-25T11:10:42.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.470+0000 7f5302000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f52fc0aeb20 con 0x7f52fc0b6720 2024-04-25T11:10:42.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.470+0000 7f5302000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f52fc0b04a0 con 0x7f52fc157320 2024-04-25T11:10:42.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.470+0000 7f52fb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc06f3a0 0x7f52fc149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.470+0000 7f52fb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc06f3a0 0x7f52fc149220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59738/0 (socket says 172.21.15.7:59738) 2024-04-25T11:10:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.470+0000 7f52fb400640 1 -- 172.21.15.7:0/1904586957 learned_addr learned my addr 172.21.15.7:0/1904586957 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.470+0000 7f52fbe00640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52fc157320 0x7f52fc159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.471+0000 7f52faa00640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52fc0b6720 0x7f52fc159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.471+0000 7f52fb400640 1 -- 172.21.15.7:0/1904586957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52fc157320 msgr2=0x7f52fc159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.471+0000 7f52fb400640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52fc157320 0x7f52fc159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.471+0000 7f52fb400640 1 -- 172.21.15.7:0/1904586957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52fc0b6720 msgr2=0x7f52fc159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:42.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.471+0000 7f52fb400640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52fc0b6720 0x7f52fc159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.471+0000 7f52fb400640 1 -- 172.21.15.7:0/1904586957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52fc149760 con 0x7f52fc06f3a0 2024-04-25T11:10:42.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.471+0000 7f52fb400640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc06f3a0 0x7f52fc149220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00c440 tx=0x7f52ec00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:42.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.471+0000 7f52f3400640 1 -- 172.21.15.7:0/1904586957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec0052d0 con 0x7f52fc06f3a0 2024-04-25T11:10:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.471+0000 7f52f3400640 1 -- 172.21.15.7:0/1904586957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52ec005470 con 0x7f52fc06f3a0 2024-04-25T11:10:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.471+0000 7f52f3400640 1 -- 172.21.15.7:0/1904586957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ec007260 con 0x7f52fc06f3a0 2024-04-25T11:10:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.471+0000 7f5302000640 1 -- 172.21.15.7:0/1904586957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52fc149ea0 con 0x7f52fc06f3a0 2024-04-25T11:10:42.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.472+0000 7f5302000640 1 -- 172.21.15.7:0/1904586957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52fc14a460 con 0x7f52fc06f3a0 2024-04-25T11:10:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.473+0000 7f5302000640 1 -- 172.21.15.7:0/1904586957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52fc050740 con 0x7f52fc06f3a0 2024-04-25T11:10:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.474+0000 7f52f3400640 1 -- 172.21.15.7:0/1904586957 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f52ec0074c0 con 0x7f52fc06f3a0 2024-04-25T11:10:42.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.475+0000 7f52f3400640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f52dc0744b0 0x7f52dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.475+0000 7f52f3400640 1 -- 172.21.15.7:0/1904586957 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f52ec094b60 con 0x7f52fc06f3a0 2024-04-25T11:10:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.475+0000 7f52faa00640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f52dc0744b0 0x7f52dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:42.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.476+0000 7f52faa00640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f52dc0744b0 0x7f52dc0769c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f52e8034000 tx=0x7f52e802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:42.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.480+0000 7f52f3400640 1 -- 172.21.15.7:0/1904586957 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f52ec05ee90 con 0x7f52fc06f3a0 2024-04-25T11:10:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.687+0000 7f5302000640 1 -- 172.21.15.7:0/1904586957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f52fc068580 con 0x7f52fc06f3a0 2024-04-25T11:10:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.688+0000 7f52f3400640 1 -- 172.21.15.7:0/1904586957 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) ==== 104+0+11 (secure 0 0 0) 0x7f52ec05e790 con 0x7f52fc06f3a0 2024-04-25T11:10:42.689 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:10:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.690+0000 7f5302000640 1 -- 172.21.15.7:0/1904586957 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f52dc0744b0 msgr2=0x7f52dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.690+0000 7f5302000640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f52dc0744b0 0x7f52dc0769c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f52e8034000 tx=0x7f52e802e000 comp rx=0 tx=0).stop 2024-04-25T11:10:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.691+0000 7f5302000640 1 -- 172.21.15.7:0/1904586957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc06f3a0 msgr2=0x7f52fc149220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.691+0000 7f5302000640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc06f3a0 0x7f52fc149220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00c440 tx=0x7f52ec00c900 comp rx=0 tx=0).stop 2024-04-25T11:10:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.691+0000 7f5302000640 1 -- 172.21.15.7:0/1904586957 shutdown_connections 2024-04-25T11:10:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.691+0000 7f5302000640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f52dc0744b0 0x7f52dc0769c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.691+0000 7f5302000640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52fc157320 0x7f52fc159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.691+0000 7f5302000640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52fc0b6720 0x7f52fc159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.691+0000 7f5302000640 1 --2- 172.21.15.7:0/1904586957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc06f3a0 0x7f52fc149220 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.691+0000 7f5302000640 1 -- 172.21.15.7:0/1904586957 >> 172.21.15.7:0/1904586957 conn(0x7f52fc0742c0 msgr2=0x7f52fc0b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.691+0000 7f5302000640 1 -- 172.21.15.7:0/1904586957 shutdown_connections 2024-04-25T11:10:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:42.691+0000 7f5302000640 1 -- 172.21.15.7:0/1904586957 wait complete. 2024-04-25T11:10:42.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:10:43.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.007+0000 7fd252200640 1 Processor -- start 2024-04-25T11:10:43.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.007+0000 7fd252200640 1 -- start start 2024-04-25T11:10:43.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.009+0000 7fd252200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c15aaa0 0x7fd24c0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.009+0000 7fd252200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd24c06f5b0 con 0x7fd24c0b5020 2024-04-25T11:10:43.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.009+0000 7fd252200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd24c0d14a0 con 0x7fd24c0d0ee0 2024-04-25T11:10:43.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.009+0000 7fd252200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd24c0af290 con 0x7fd24c15aaa0 2024-04-25T11:10:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.010+0000 7fd24be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd24c0b5020 0x7fd24c160990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48712/0 (socket says 172.21.15.7:48712) 2024-04-25T11:10:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.010+0000 7fd24be00640 1 -- 172.21.15.7:0/2057072630 learned_addr learned my addr 172.21.15.7:0/2057072630 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.010+0000 7fd24b400640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c15aaa0 0x7fd24c0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.010+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1343348136 0 0) 0x7fd24c06f5b0 con 0x7fd24c0b5020 2024-04-25T11:10:43.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.010+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd230005730 con 0x7fd24c0b5020 2024-04-25T11:10:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1649405352 0 0) 0x7fd24c0d14a0 con 0x7fd24c0d0ee0 2024-04-25T11:10:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd24c06f5b0 con 0x7fd24c0d0ee0 2024-04-25T11:10:43.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2887104783 0 0) 0x7fd230005730 con 0x7fd24c0b5020 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c15aaa0 msgr2=0x7fd24c0affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c15aaa0 0x7fd24c0affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 >> v1:172.21.15.87:6789/0 conn(0x7fd24c0d0ee0 legacy=0x7fd24c0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c0af4e0 con 0x7fd24c0b5020 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fd2440019f0 con 0x7fd24c0b5020 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 >> v1:172.21.15.7:6789/0 conn(0x7fd24c0b5020 legacy=0x7fd24c160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 0x7fd2300061d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c15aaa0 0x7fd230006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd230006f50 0x7fd230007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c0af290 con 0x7fd230005d80 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd23000dcd0 con 0x7fd230006f50 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd23000ded0 con 0x7fd24c15aaa0 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.011+0000 7fd24b400640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 0x7fd2300061d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.012+0000 7fd24aa00640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c15aaa0 0x7fd230006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.012+0000 7fd24be00640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd230006f50 0x7fd230007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.012+0000 7fd24b400640 1 -- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c15aaa0 msgr2=0x7fd230006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:43.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.012+0000 7fd24b400640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c15aaa0 0x7fd230006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.012+0000 7fd24b400640 1 -- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd230006f50 msgr2=0x7fd230007360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:43.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.012+0000 7fd24b400640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd230006f50 0x7fd230007360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.012+0000 7fd24b400640 1 -- 172.21.15.7:0/2057072630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd230005730 con 0x7fd230005d80 2024-04-25T11:10:43.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.012+0000 7fd24be00640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd230006f50 0x7fd230007360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.013+0000 7fd24b400640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 0x7fd2300061d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009ac0 tx=0x7fd230007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df88e20fa94aafe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.013+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c036070 con 0x7fd230005d80 2024-04-25T11:10:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.013+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c031400 con 0x7fd230005d80 2024-04-25T11:10:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.013+0000 7fd24a000640 1 -- 172.21.15.7:0/2057072630 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c009690 con 0x7fd230005d80 2024-04-25T11:10:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.014+0000 7fd252200640 1 -- 172.21.15.7:0/2057072630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 msgr2=0x7fd2300061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.014+0000 7fd252200640 1 --2- 172.21.15.7:0/2057072630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 0x7fd2300061d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009ac0 tx=0x7fd230007a40 comp rx=0 tx=0).stop 2024-04-25T11:10:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.014+0000 7fd24be00640 1 -- 172.21.15.7:0/2057072630 reap_dead start 2024-04-25T11:10:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.014+0000 7fd252200640 1 -- 172.21.15.7:0/2057072630 shutdown_connections 2024-04-25T11:10:43.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.014+0000 7fd252200640 1 -- 172.21.15.7:0/2057072630 >> 172.21.15.7:0/2057072630 conn(0x7fd24c0742c0 msgr2=0x7fd24c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:43.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.014+0000 7fd252200640 1 -- 172.21.15.7:0/2057072630 shutdown_connections 2024-04-25T11:10:43.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.014+0000 7fd252200640 1 -- 172.21.15.7:0/2057072630 wait complete. 2024-04-25T11:10:43.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.016+0000 7fd252200640 1 Processor -- start 2024-04-25T11:10:43.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.017+0000 7fd252200640 1 -- start start 2024-04-25T11:10:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.018+0000 7fd252200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 0x7fd24c14a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.018+0000 7fd252200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd24c159c90 0x7fd24c15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.018+0000 7fd252200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c1e75f0 0x7fd24c1e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.018+0000 7fd252200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c0b5430 con 0x7fd230005d80 2024-04-25T11:10:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.018+0000 7fd252200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c0afaa0 con 0x7fd24c159c90 2024-04-25T11:10:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.018+0000 7fd252200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd24c0b55b0 con 0x7fd24c1e75f0 2024-04-25T11:10:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.018+0000 7fd24b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 0x7fd24c14a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.019+0000 7fd24b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 0x7fd24c14a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59758/0 (socket says 172.21.15.7:59758) 2024-04-25T11:10:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.019+0000 7fd24b400640 1 -- 172.21.15.7:0/76567544 learned_addr learned my addr 172.21.15.7:0/76567544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:43.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.019+0000 7fd24aa00640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd24c159c90 0x7fd24c15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.019+0000 7fd24b400640 1 -- 172.21.15.7:0/76567544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c1e75f0 msgr2=0x7fd24c1e7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.019+0000 7fd24b400640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c1e75f0 0x7fd24c1e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.019+0000 7fd24b400640 1 -- 172.21.15.7:0/76567544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd24c159c90 msgr2=0x7fd24c15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.019+0000 7fd24b400640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd24c159c90 0x7fd24c15a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.019+0000 7fd24b400640 1 -- 172.21.15.7:0/76567544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c1e7f40 con 0x7fd230005d80 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.019+0000 7fd24b400640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 0x7fd24c14a3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009ac0 tx=0x7fd23c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.020+0000 7fd243400640 1 -- 172.21.15.7:0/76567544 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c040020 con 0x7fd230005d80 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.020+0000 7fd243400640 1 -- 172.21.15.7:0/76567544 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c003bc0 con 0x7fd230005d80 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.020+0000 7fd252200640 1 -- 172.21.15.7:0/76567544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24c158170 con 0x7fd230005d80 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.020+0000 7fd243400640 1 -- 172.21.15.7:0/76567544 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c036040 con 0x7fd230005d80 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.020+0000 7fd252200640 1 -- 172.21.15.7:0/76567544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24c158730 con 0x7fd230005d80 2024-04-25T11:10:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.021+0000 7fd252200640 1 -- 172.21.15.7:0/76567544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24c050740 con 0x7fd230005d80 2024-04-25T11:10:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.022+0000 7fd243400640 1 -- 172.21.15.7:0/76567544 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd23c043020 con 0x7fd230005d80 2024-04-25T11:10:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.023+0000 7fd243400640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2280744e0 0x7fd2280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.023+0000 7fd243400640 1 -- 172.21.15.7:0/76567544 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd23c0b5880 con 0x7fd230005d80 2024-04-25T11:10:43.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.023+0000 7fd24aa00640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2280744e0 0x7fd2280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.024+0000 7fd24aa00640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2280744e0 0x7fd2280769f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd24c156ca0 tx=0x7fd234009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:43.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.028+0000 7fd243400640 1 -- 172.21.15.7:0/76567544 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd23c04a070 con 0x7fd230005d80 2024-04-25T11:10:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.232+0000 7fd252200640 1 -- 172.21.15.7:0/76567544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd24c070f20 con 0x7fd230005d80 2024-04-25T11:10:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.233+0000 7fd243400640 1 -- 172.21.15.7:0/76567544 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) ==== 100+0+11 (secure 0 0 0) 0x7fd24c1e7f40 con 0x7fd230005d80 2024-04-25T11:10:43.235 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:10:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 -- 172.21.15.7:0/76567544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2280744e0 msgr2=0x7fd2280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2280744e0 0x7fd2280769f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd24c156ca0 tx=0x7fd234009040 comp rx=0 tx=0).stop 2024-04-25T11:10:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 -- 172.21.15.7:0/76567544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 msgr2=0x7fd24c14a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 0x7fd24c14a3e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009ac0 tx=0x7fd23c004730 comp rx=0 tx=0).stop 2024-04-25T11:10:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 -- 172.21.15.7:0/76567544 shutdown_connections 2024-04-25T11:10:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2280744e0 0x7fd2280769f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c1e75f0 0x7fd24c1e7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd24c159c90 0x7fd24c15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 --2- 172.21.15.7:0/76567544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd230005d80 0x7fd24c14a3e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 -- 172.21.15.7:0/76567544 >> 172.21.15.7:0/76567544 conn(0x7fd24c0742c0 msgr2=0x7fd24c06cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 -- 172.21.15.7:0/76567544 shutdown_connections 2024-04-25T11:10:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.236+0000 7fd252200640 1 -- 172.21.15.7:0/76567544 wait complete. 2024-04-25T11:10:43.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:10:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.556+0000 7f7f67e00640 1 Processor -- start 2024-04-25T11:10:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.556+0000 7f7f67e00640 1 -- start start 2024-04-25T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.559+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f680d0fb0 0x7f7f6815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.559+0000 7f7f67e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f680b8e50 0x7f7f680b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.559+0000 7f7f67e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f680b7920 0x7f7f680d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.559+0000 7f7f67e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7f680b7d30 con 0x7f7f680b7920 2024-04-25T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.559+0000 7f7f67e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7f680d13c0 con 0x7f7f680b8e50 2024-04-25T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.559+0000 7f7f67e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f68163f10 con 0x7f7f680d0fb0 2024-04-25T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.559+0000 7f7f67400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f680b7920 0x7f7f680d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.559+0000 7f7f67400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f680b7920 0x7f7f680d45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59774/0 (socket says 172.21.15.7:59774) 2024-04-25T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.560+0000 7f7f67400640 1 -- 172.21.15.7:0/1068996522 learned_addr learned my addr 172.21.15.7:0/1068996522 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.560+0000 7f7f66a00640 1 --2- 172.21.15.7:0/1068996522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f680d0fb0 0x7f7f6815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.560+0000 7f7f66000640 1 --2- 172.21.15.7:0/1068996522 >> v2:172.21.15.87:3300/0 conn(0x7f7f680b8e50 0x7f7f680b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.560+0000 7f7f67400640 1 -- 172.21.15.7:0/1068996522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f680d0fb0 msgr2=0x7f7f6815fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.560+0000 7f7f67400640 1 --2- 172.21.15.7:0/1068996522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f680d0fb0 0x7f7f6815fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.560+0000 7f7f67400640 1 -- 172.21.15.7:0/1068996522 >> v2:172.21.15.87:3300/0 conn(0x7f7f680b8e50 msgr2=0x7f7f680b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.560+0000 7f7f67400640 1 --2- 172.21.15.7:0/1068996522 >> v2:172.21.15.87:3300/0 conn(0x7f7f680b8e50 0x7f7f680b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.560+0000 7f7f67400640 1 -- 172.21.15.7:0/1068996522 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f681641f0 con 0x7f7f680b7920 2024-04-25T11:10:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.561+0000 7f7f67400640 1 --2- 172.21.15.7:0/1068996522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f680b7920 0x7f7f680d45e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7f6806ef40 tx=0x7f7f6002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9e050fe2466eb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.562+0000 7f7f65600640 1 -- 172.21.15.7:0/1068996522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60036070 con 0x7f7f680b7920 2024-04-25T11:10:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.562+0000 7f7f65600640 1 -- 172.21.15.7:0/1068996522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f60031600 con 0x7f7f680b7920 2024-04-25T11:10:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.562+0000 7f7f65600640 1 -- 172.21.15.7:0/1068996522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f600319d0 con 0x7f7f680b7920 2024-04-25T11:10:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.563+0000 7f7f67e00640 1 -- 172.21.15.7:0/1068996522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f680b7920 msgr2=0x7f7f680d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.563+0000 7f7f67e00640 1 --2- 172.21.15.7:0/1068996522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f680b7920 0x7f7f680d45e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7f6806ef40 tx=0x7f7f6002a610 comp rx=0 tx=0).stop 2024-04-25T11:10:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.563+0000 7f7f67e00640 1 -- 172.21.15.7:0/1068996522 shutdown_connections 2024-04-25T11:10:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.563+0000 7f7f67e00640 1 --2- 172.21.15.7:0/1068996522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f680b7920 0x7f7f680d45e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.563+0000 7f7f67e00640 1 --2- 172.21.15.7:0/1068996522 >> v2:172.21.15.87:3300/0 conn(0x7f7f680b8e50 0x7f7f680b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.563+0000 7f7f67e00640 1 --2- 172.21.15.7:0/1068996522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f680d0fb0 0x7f7f6815fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.563+0000 7f7f67e00640 1 -- 172.21.15.7:0/1068996522 >> 172.21.15.7:0/1068996522 conn(0x7f7f680d23e0 msgr2=0x7f7f680d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.563+0000 7f7f67e00640 1 -- 172.21.15.7:0/1068996522 shutdown_connections 2024-04-25T11:10:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.563+0000 7f7f67e00640 1 -- 172.21.15.7:0/1068996522 wait complete. 2024-04-25T11:10:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.566+0000 7f7f67e00640 1 Processor -- start 2024-04-25T11:10:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.566+0000 7f7f67e00640 1 -- start start 2024-04-25T11:10:43.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.567+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f680b7920 0x7f7f681f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.567+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f680b8e50 0x7f7f680c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.567+0000 7f7f67e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f680d0fb0 0x7f7f680ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f67e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f680b4bc0 con 0x7f7f680d0fb0 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f67e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7f681647b0 con 0x7f7f680b8e50 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f67e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f680b9260 con 0x7f7f680b7920 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f67400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f680d0fb0 0x7f7f680ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f67400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f680d0fb0 0x7f7f680ca350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59778/0 (socket says 172.21.15.7:59778) 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f67400640 1 -- 172.21.15.7:0/2783976199 learned_addr learned my addr 172.21.15.7:0/2783976199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f66a00640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f680b7920 0x7f7f681f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f66000640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f680b8e50 0x7f7f680c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f67400640 1 -- 172.21.15.7:0/2783976199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f680b7920 msgr2=0x7f7f681f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f67400640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f680b7920 0x7f7f681f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f67400640 1 -- 172.21.15.7:0/2783976199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f680b8e50 msgr2=0x7f7f680c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f67400640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f680b8e50 0x7f7f680c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.568+0000 7f7f67400640 1 -- 172.21.15.7:0/2783976199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f680cab00 con 0x7f7f680d0fb0 2024-04-25T11:10:43.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.569+0000 7f7f67400640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f680d0fb0 0x7f7f680ca350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7f60034040 tx=0x7f7f60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:43.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.569+0000 7f7f5ea00640 1 -- 172.21.15.7:0/2783976199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60036050 con 0x7f7f680d0fb0 2024-04-25T11:10:43.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.569+0000 7f7f5ea00640 1 -- 172.21.15.7:0/2783976199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f60042070 con 0x7f7f680d0fb0 2024-04-25T11:10:43.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.569+0000 7f7f67e00640 1 -- 172.21.15.7:0/2783976199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f680cade0 con 0x7f7f680d0fb0 2024-04-25T11:10:43.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.569+0000 7f7f5ea00640 1 -- 172.21.15.7:0/2783976199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f60009510 con 0x7f7f680d0fb0 2024-04-25T11:10:43.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.569+0000 7f7f67e00640 1 -- 172.21.15.7:0/2783976199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f680ccad0 con 0x7f7f680d0fb0 2024-04-25T11:10:43.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.570+0000 7f7f67e00640 1 -- 172.21.15.7:0/2783976199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f68050740 con 0x7f7f680d0fb0 2024-04-25T11:10:43.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.572+0000 7f7f5ea00640 1 -- 172.21.15.7:0/2783976199 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7f60007bd0 con 0x7f7f680d0fb0 2024-04-25T11:10:43.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.573+0000 7f7f5ea00640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f480744b0 0x7f7f480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:43.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.573+0000 7f7f5ea00640 1 -- 172.21.15.7:0/2783976199 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7f600b63d0 con 0x7f7f680d0fb0 2024-04-25T11:10:43.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.573+0000 7f7f66a00640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f480744b0 0x7f7f480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:43.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.574+0000 7f7f66a00640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f480744b0 0x7f7f480769c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7f58006fd0 tx=0x7f7f58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:43.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.578+0000 7f7f5ea00640 1 -- 172.21.15.7:0/2783976199 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f600808a0 con 0x7f7f680d0fb0 2024-04-25T11:10:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.782+0000 7f7f67e00640 1 -- 172.21.15.7:0/2783976199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7f680ca890 con 0x7f7f680d0fb0 2024-04-25T11:10:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.783+0000 7f7f5ea00640 1 -- 172.21.15.7:0/2783976199 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f7f600801a0 con 0x7f7f680d0fb0 2024-04-25T11:10:43.785 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:10:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.786+0000 7f7f67e00640 1 -- 172.21.15.7:0/2783976199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f480744b0 msgr2=0x7f7f480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.787+0000 7f7f67e00640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f480744b0 0x7f7f480769c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7f58006fd0 tx=0x7f7f58008040 comp rx=0 tx=0).stop 2024-04-25T11:10:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.787+0000 7f7f67e00640 1 -- 172.21.15.7:0/2783976199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f680d0fb0 msgr2=0x7f7f680ca350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.787+0000 7f7f67e00640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f680d0fb0 0x7f7f680ca350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7f60034040 tx=0x7f7f60004a40 comp rx=0 tx=0).stop 2024-04-25T11:10:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.787+0000 7f7f67e00640 1 -- 172.21.15.7:0/2783976199 shutdown_connections 2024-04-25T11:10:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.787+0000 7f7f67e00640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f480744b0 0x7f7f480769c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.787+0000 7f7f67e00640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f680d0fb0 0x7f7f680ca350 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.787+0000 7f7f67e00640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f680b8e50 0x7f7f680c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.787+0000 7f7f67e00640 1 --2- 172.21.15.7:0/2783976199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f680b7920 0x7f7f681f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.787+0000 7f7f67e00640 1 -- 172.21.15.7:0/2783976199 >> 172.21.15.7:0/2783976199 conn(0x7f7f680d23e0 msgr2=0x7f7f680b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.787+0000 7f7f67e00640 1 -- 172.21.15.7:0/2783976199 shutdown_connections 2024-04-25T11:10:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:43.787+0000 7f7f67e00640 1 -- 172.21.15.7:0/2783976199 wait complete. 2024-04-25T11:10:43.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:10:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.097+0000 7efff7e00640 1 Processor -- start 2024-04-25T11:10:44.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.097+0000 7efff7e00640 1 -- start start 2024-04-25T11:10:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.099+0000 7efff7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efff8141290 0x7efff81416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.099+0000 7efff7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efff806d610 0x7efff806da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.099+0000 7efff7e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efff806c570 con 0x7efff815eed0 2024-04-25T11:10:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.099+0000 7efff7e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efff806c850 con 0x7efff806d610 2024-04-25T11:10:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.099+0000 7efff7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efff806cb30 con 0x7efff8141290 2024-04-25T11:10:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.100+0000 7efff6a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efff815eed0 0x7efff80cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48716/0 (socket says 172.21.15.7:48716) 2024-04-25T11:10:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.100+0000 7efff6a00640 1 -- 172.21.15.7:0/3796411863 learned_addr learned my addr 172.21.15.7:0/3796411863 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:44.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.100+0000 7efff7400640 1 --2- 172.21.15.7:0/3796411863 >> v2:172.21.15.7:3300/0 conn(0x7efff806d610 0x7efff806da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.100+0000 7efff6000640 1 -- 172.21.15.7:0/3796411863 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4204028005 0 0) 0x7efff806c570 con 0x7efff815eed0 2024-04-25T11:10:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.100+0000 7effefe00640 1 --2- 172.21.15.7:0/3796411863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efff8141290 0x7efff81416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.100+0000 7efff6000640 1 -- 172.21.15.7:0/3796411863 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effdc003520 con 0x7efff815eed0 2024-04-25T11:10:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.101+0000 7efff7400640 1 -- 172.21.15.7:0/3796411863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efff8141290 msgr2=0x7efff81416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.101+0000 7efff7400640 1 --2- 172.21.15.7:0/3796411863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efff8141290 0x7efff81416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.101+0000 7efff7400640 1 -- 172.21.15.7:0/3796411863 >> v1:172.21.15.7:6789/0 conn(0x7efff815eed0 legacy=0x7efff80cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:44.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.101+0000 7efff7400640 1 -- 172.21.15.7:0/3796411863 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff806ce10 con 0x7efff806d610 2024-04-25T11:10:44.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.102+0000 7efff7400640 1 --2- 172.21.15.7:0/3796411863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff806d610 0x7efff806da20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efff0009d90 tx=0x7efff002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdcb3214577a72c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:44.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.102+0000 7efff6000640 1 -- 172.21.15.7:0/3796411863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0007740 con 0x7efff806d610 2024-04-25T11:10:44.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.102+0000 7efff6000640 1 -- 172.21.15.7:0/3796411863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efff00078e0 con 0x7efff806d610 2024-04-25T11:10:44.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.102+0000 7efff6000640 1 -- 172.21.15.7:0/3796411863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0007c90 con 0x7efff806d610 2024-04-25T11:10:44.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.103+0000 7efff7e00640 1 -- 172.21.15.7:0/3796411863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff806d610 msgr2=0x7efff806da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:44.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.103+0000 7efff7e00640 1 --2- 172.21.15.7:0/3796411863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff806d610 0x7efff806da20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efff0009d90 tx=0x7efff002a660 comp rx=0 tx=0).stop 2024-04-25T11:10:44.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.103+0000 7efff7e00640 1 -- 172.21.15.7:0/3796411863 shutdown_connections 2024-04-25T11:10:44.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.103+0000 7efff7e00640 1 --2- 172.21.15.7:0/3796411863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff806d610 0x7efff806da20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:44.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.103+0000 7efff7e00640 1 --2- 172.21.15.7:0/3796411863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efff8141290 0x7efff81416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:44.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.103+0000 7efff7e00640 1 -- 172.21.15.7:0/3796411863 >> 172.21.15.7:0/3796411863 conn(0x7efff80d1540 msgr2=0x7efff80d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:44.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.103+0000 7efff7e00640 1 -- 172.21.15.7:0/3796411863 shutdown_connections 2024-04-25T11:10:44.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.103+0000 7efff7e00640 1 -- 172.21.15.7:0/3796411863 wait complete. 2024-04-25T11:10:44.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.106+0000 7efff7e00640 1 Processor -- start 2024-04-25T11:10:44.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.106+0000 7efff7e00640 1 -- start start 2024-04-25T11:10:44.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.107+0000 7efff7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff806d610 0x7efff8150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:44.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.107+0000 7efff7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efff8141290 0x7efff80d0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:44.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.107+0000 7efff7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efff815eed0 0x7efff815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:44.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.107+0000 7efff7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efff8140f20 con 0x7efff806d610 2024-04-25T11:10:44.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.107+0000 7efff7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efff8140da0 con 0x7efff8141290 2024-04-25T11:10:44.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.107+0000 7efff7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efff80b4590 con 0x7efff815eed0 2024-04-25T11:10:44.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.108+0000 7efff6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff806d610 0x7efff8150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:44.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.108+0000 7efff6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff806d610 0x7efff8150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59800/0 (socket says 172.21.15.7:59800) 2024-04-25T11:10:44.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.108+0000 7efff6a00640 1 -- 172.21.15.7:0/1273632848 learned_addr learned my addr 172.21.15.7:0/1273632848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:44.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.108+0000 7effefe00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efff8141290 0x7efff80d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:44.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.108+0000 7efff7400640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efff815eed0 0x7efff815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:44.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.108+0000 7efff6a00640 1 -- 172.21.15.7:0/1273632848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efff815eed0 msgr2=0x7efff815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:44.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.108+0000 7efff6a00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efff815eed0 0x7efff815ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:44.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.108+0000 7efff6a00640 1 -- 172.21.15.7:0/1273632848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efff8141290 msgr2=0x7efff80d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:44.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.108+0000 7efff6a00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efff8141290 0x7efff80d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:44.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.108+0000 7efff6a00640 1 -- 172.21.15.7:0/1273632848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff81536d0 con 0x7efff806d610 2024-04-25T11:10:44.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.109+0000 7efff6a00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff806d610 0x7efff8150ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7effe800c3c0 tx=0x7effe800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:44.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.109+0000 7effef400640 1 -- 172.21.15.7:0/1273632848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8017070 con 0x7efff806d610 2024-04-25T11:10:44.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.109+0000 7effef400640 1 -- 172.21.15.7:0/1273632848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe8005ce0 con 0x7efff806d610 2024-04-25T11:10:44.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.109+0000 7efff7e00640 1 -- 172.21.15.7:0/1273632848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff81539b0 con 0x7efff806d610 2024-04-25T11:10:44.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.109+0000 7effef400640 1 -- 172.21.15.7:0/1273632848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8007310 con 0x7efff806d610 2024-04-25T11:10:44.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.109+0000 7efff7e00640 1 -- 172.21.15.7:0/1273632848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff8153f70 con 0x7efff806d610 2024-04-25T11:10:44.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.110+0000 7efff7e00640 1 -- 172.21.15.7:0/1273632848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff8050740 con 0x7efff806d610 2024-04-25T11:10:44.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.111+0000 7effef400640 1 -- 172.21.15.7:0/1273632848 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7effe801a070 con 0x7efff806d610 2024-04-25T11:10:44.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.112+0000 7effef400640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effd40744b0 0x7effd40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:44.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.113+0000 7effef400640 1 -- 172.21.15.7:0/1273632848 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7effe8094980 con 0x7efff806d610 2024-04-25T11:10:44.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.113+0000 7effefe00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effd40744b0 0x7effd40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:44.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.114+0000 7effefe00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effd40744b0 0x7effd40769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7effe0006fd0 tx=0x7effe0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:44.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.118+0000 7effef400640 1 -- 172.21.15.7:0/1273632848 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effe805ed30 con 0x7efff806d610 2024-04-25T11:10:44.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.339+0000 7efff7e00640 1 -- 172.21.15.7:0/1273632848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efff8154b60 con 0x7efff806d610 2024-04-25T11:10:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.340+0000 7effef400640 1 -- 172.21.15.7:0/1273632848 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7effe805e630 con 0x7efff806d610 2024-04-25T11:10:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:10:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.342+0000 7efff7e00640 1 -- 172.21.15.7:0/1273632848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effd40744b0 msgr2=0x7effd40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.342+0000 7efff7e00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effd40744b0 0x7effd40769c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7effe0006fd0 tx=0x7effe0008040 comp rx=0 tx=0).stop 2024-04-25T11:10:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.342+0000 7efff7e00640 1 -- 172.21.15.7:0/1273632848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff806d610 msgr2=0x7efff8150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.342+0000 7efff7e00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff806d610 0x7efff8150ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7effe800c3c0 tx=0x7effe800c880 comp rx=0 tx=0).stop 2024-04-25T11:10:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.343+0000 7efff7e00640 1 -- 172.21.15.7:0/1273632848 shutdown_connections 2024-04-25T11:10:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.343+0000 7efff7e00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effd40744b0 0x7effd40769c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.343+0000 7efff7e00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efff815eed0 0x7efff815ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:44.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.343+0000 7efff7e00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efff8141290 0x7efff80d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:44.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.343+0000 7efff7e00640 1 --2- 172.21.15.7:0/1273632848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efff806d610 0x7efff8150ff0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:44.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.343+0000 7efff7e00640 1 -- 172.21.15.7:0/1273632848 >> 172.21.15.7:0/1273632848 conn(0x7efff80d1540 msgr2=0x7efff80d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:44.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.343+0000 7efff7e00640 1 -- 172.21.15.7:0/1273632848 shutdown_connections 2024-04-25T11:10:44.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:44.343+0000 7efff7e00640 1 -- 172.21.15.7:0/1273632848 wait complete. 2024-04-25T11:10:44.371 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-04-25T11:10:44.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:44.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T11:10:44.456 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi007.stderr:ceph-fuse[38714]: fuse finished with error 0 and tester_r 0 2024-04-25T11:10:44.460 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:10:50.562 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:10:50.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:10:50.563 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T11:10:50.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.889+0000 7fc4e1600640 1 Processor -- start 2024-04-25T11:10:50.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.889+0000 7fc4e1600640 1 -- start start 2024-04-25T11:10:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.891+0000 7fc4e1600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4dc0b04d0 0x7fc4dc0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.891+0000 7fc4e1600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4dc0b4a80 0x7fc4dc0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.891+0000 7fc4e1600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4dc15e190 con 0x7fc4dc070a70 2024-04-25T11:10:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.891+0000 7fc4e1600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc4dc15d610 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.891+0000 7fc4e1600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc4dc15d8f0 con 0x7fc4dc0b04d0 2024-04-25T11:10:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.892+0000 7fc4dbe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc4dc070a70 0x7fc4dc0ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50142/0 (socket says 172.21.15.7:50142) 2024-04-25T11:10:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.892+0000 7fc4dbe00640 1 -- 172.21.15.7:0/2213329811 learned_addr learned my addr 172.21.15.7:0/2213329811 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.892+0000 7fc4e0c00640 1 --2- 172.21.15.7:0/2213329811 >> v2:172.21.15.7:3300/0 conn(0x7fc4dc0b4a80 0x7fc4dc0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:50.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.892+0000 7fc4d3e00640 1 --2- 172.21.15.7:0/2213329811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4dc0b04d0 0x7fc4dc0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.892+0000 7fc4db400640 1 -- 172.21.15.7:0/2213329811 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2146955601 0 0) 0x7fc4dc15e190 con 0x7fc4dc070a70 2024-04-25T11:10:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.892+0000 7fc4db400640 1 -- 172.21.15.7:0/2213329811 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4c0005730 con 0x7fc4dc070a70 2024-04-25T11:10:50.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.892+0000 7fc4e0c00640 1 -- 172.21.15.7:0/2213329811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4dc0b04d0 msgr2=0x7fc4dc0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.892+0000 7fc4e0c00640 1 --2- 172.21.15.7:0/2213329811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4dc0b04d0 0x7fc4dc0b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.892+0000 7fc4e0c00640 1 -- 172.21.15.7:0/2213329811 >> v1:172.21.15.7:6789/0 conn(0x7fc4dc070a70 legacy=0x7fc4dc0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.892+0000 7fc4e0c00640 1 -- 172.21.15.7:0/2213329811 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4dc15dbd0 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.892+0000 7fc4d3e00640 1 --2- 172.21.15.7:0/2213329811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4dc0b04d0 0x7fc4dc0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.893+0000 7fc4e0c00640 1 --2- 172.21.15.7:0/2213329811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0b4a80 0x7fc4dc0b4e90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009d90 tx=0x7fc4d402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d09e155b32ae6c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:50.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.894+0000 7fc4db400640 1 -- 172.21.15.7:0/2213329811 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4007740 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.894+0000 7fc4db400640 1 -- 172.21.15.7:0/2213329811 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d40078e0 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.894+0000 7fc4db400640 1 -- 172.21.15.7:0/2213329811 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4007cb0 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.895+0000 7fc4e1600640 1 -- 172.21.15.7:0/2213329811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0b4a80 msgr2=0x7fc4dc0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:50.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.895+0000 7fc4e1600640 1 --2- 172.21.15.7:0/2213329811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0b4a80 0x7fc4dc0b4e90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009d90 tx=0x7fc4d402a660 comp rx=0 tx=0).stop 2024-04-25T11:10:50.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.895+0000 7fc4e1600640 1 -- 172.21.15.7:0/2213329811 shutdown_connections 2024-04-25T11:10:50.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.895+0000 7fc4e1600640 1 --2- 172.21.15.7:0/2213329811 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0b4a80 0x7fc4dc0b4e90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:50.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.895+0000 7fc4e1600640 1 --2- 172.21.15.7:0/2213329811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4dc0b04d0 0x7fc4dc0b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:50.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.895+0000 7fc4e1600640 1 -- 172.21.15.7:0/2213329811 >> 172.21.15.7:0/2213329811 conn(0x7fc4dc0b3820 msgr2=0x7fc4dc0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:50.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.895+0000 7fc4e1600640 1 -- 172.21.15.7:0/2213329811 shutdown_connections 2024-04-25T11:10:50.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.895+0000 7fc4e1600640 1 -- 172.21.15.7:0/2213329811 wait complete. 2024-04-25T11:10:50.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.897+0000 7fc4e1600640 1 Processor -- start 2024-04-25T11:10:50.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.897+0000 7fc4e1600640 1 -- start start 2024-04-25T11:10:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.899+0000 7fc4e1600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4dc070a70 0x7fc4dc0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.899+0000 7fc4e1600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4dc0b04d0 0x7fc4dc14f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.899+0000 7fc4e1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0b4a80 0x7fc4dc06ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.899+0000 7fc4e1600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc0b0e20 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.899+0000 7fc4e1600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc4dc0aef90 con 0x7fc4dc0b04d0 2024-04-25T11:10:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.899+0000 7fc4e1600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc4dc15d170 con 0x7fc4dc070a70 2024-04-25T11:10:50.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.899+0000 7fc4e0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0b4a80 0x7fc4dc06ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.899+0000 7fc4e0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0b4a80 0x7fc4dc06ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48578/0 (socket says 172.21.15.7:48578) 2024-04-25T11:10:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.899+0000 7fc4e0c00640 1 -- 172.21.15.7:0/3120973117 learned_addr learned my addr 172.21.15.7:0/3120973117 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4dbe00640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4dc070a70 0x7fc4dc0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4d3e00640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4dc0b04d0 0x7fc4dc14f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4e0c00640 1 -- 172.21.15.7:0/3120973117 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4dc070a70 msgr2=0x7fc4dc0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4e0c00640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4dc070a70 0x7fc4dc0cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4e0c00640 1 -- 172.21.15.7:0/3120973117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4dc0b04d0 msgr2=0x7fc4dc14f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4e0c00640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4dc0b04d0 0x7fc4dc14f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4e0c00640 1 -- 172.21.15.7:0/3120973117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4dc0cbb80 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4d3e00640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4dc0b04d0 0x7fc4dc14f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4e0c00640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0b4a80 0x7fc4dc06ce00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009eb0 tx=0x7fc4d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4d8c00640 1 -- 172.21.15.7:0/3120973117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d403f070 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4d8c00640 1 -- 172.21.15.7:0/3120973117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d4031bc0 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.900+0000 7fc4d8c00640 1 -- 172.21.15.7:0/3120973117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4009450 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.901+0000 7fc4e1600640 1 -- 172.21.15.7:0/3120973117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4dc0cbe60 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.901+0000 7fc4e1600640 1 -- 172.21.15.7:0/3120973117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4dc0cc420 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.902+0000 7fc4e1600640 1 -- 172.21.15.7:0/3120973117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4dc050740 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.903+0000 7fc4d8c00640 1 -- 172.21.15.7:0/3120973117 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc4d403b410 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.904+0000 7fc4d8c00640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4b80744e0 0x7fc4b80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.904+0000 7fc4d8c00640 1 -- 172.21.15.7:0/3120973117 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc4d40b6cf0 con 0x7fc4dc0b4a80 2024-04-25T11:10:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.905+0000 7fc4dbe00640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4b80744e0 0x7fc4b80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.905+0000 7fc4dbe00640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4b80744e0 0x7fc4b80769f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc007fd0 tx=0x7fc4cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:50.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:50.912+0000 7fc4d8c00640 1 -- 172.21.15.7:0/3120973117 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc4d40bb070 con 0x7fc4dc0b4a80 2024-04-25T11:10:51.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.116+0000 7fc4e1600640 1 -- 172.21.15.7:0/3120973117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4dc160150 con 0x7fc4dc0b4a80 2024-04-25T11:10:51.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.119+0000 7fc4d8c00640 1 -- 172.21.15.7:0/3120973117 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18418 (secure 0 0 0) 0x7fc4d4080b80 con 0x7fc4dc0b4a80 2024-04-25T11:10:51.121 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:51.121 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:28.084739+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:51.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 -- 172.21.15.7:0/3120973117 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4b80744e0 msgr2=0x7fc4b80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:51.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4b80744e0 0x7fc4b80769f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc007fd0 tx=0x7fc4cc009040 comp rx=0 tx=0).stop 2024-04-25T11:10:51.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 -- 172.21.15.7:0/3120973117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0b4a80 msgr2=0x7fc4dc06ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0b4a80 0x7fc4dc06ce00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009eb0 tx=0x7fc4d4004a40 comp rx=0 tx=0).stop 2024-04-25T11:10:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 -- 172.21.15.7:0/3120973117 shutdown_connections 2024-04-25T11:10:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4b80744e0 0x7fc4b80769f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0b4a80 0x7fc4dc06ce00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4dc0b04d0 0x7fc4dc14f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 --2- 172.21.15.7:0/3120973117 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4dc070a70 0x7fc4dc0cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 -- 172.21.15.7:0/3120973117 >> 172.21.15.7:0/3120973117 conn(0x7fc4dc0b3820 msgr2=0x7fc4dc0b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 -- 172.21.15.7:0/3120973117 shutdown_connections 2024-04-25T11:10:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.122+0000 7fc4e1600640 1 -- 172.21.15.7:0/3120973117 wait complete. 2024-04-25T11:10:51.153 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.435+0000 7fa473e00640 1 Processor -- start 2024-04-25T11:10:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.435+0000 7fa473e00640 1 -- start start 2024-04-25T11:10:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.437+0000 7fa473e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa474160590 0x7fa4740d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.437+0000 7fa473e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4740d35a0 con 0x7fa474163ac0 2024-04-25T11:10:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.438+0000 7fa473e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa474164310 con 0x7fa47406dcd0 2024-04-25T11:10:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.438+0000 7fa473e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa4741645f0 con 0x7fa474160590 2024-04-25T11:10:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.438+0000 7fa472a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa474163ac0 0x7fa4740b4b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50150/0 (socket says 172.21.15.7:50150) 2024-04-25T11:10:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.438+0000 7fa472a00640 1 -- 172.21.15.7:0/1000626770 learned_addr learned my addr 172.21.15.7:0/1000626770 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:51.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.438+0000 7fa473400640 1 --2- 172.21.15.7:0/1000626770 >> v2:172.21.15.7:3300/0 conn(0x7fa474160590 0x7fa4740d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:51.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.438+0000 7fa471600640 1 -- 172.21.15.7:0/1000626770 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 757761224 0 0) 0x7fa4740d35a0 con 0x7fa474163ac0 2024-04-25T11:10:51.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.438+0000 7fa471600640 1 -- 172.21.15.7:0/1000626770 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa458003520 con 0x7fa474163ac0 2024-04-25T11:10:51.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.438+0000 7fa473400640 1 -- 172.21.15.7:0/1000626770 >> v1:172.21.15.87:6789/0 conn(0x7fa47406dcd0 legacy=0x7fa47406e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:51.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.438+0000 7fa473400640 1 -- 172.21.15.7:0/1000626770 >> v1:172.21.15.7:6789/0 conn(0x7fa474163ac0 legacy=0x7fa4740b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:51.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.438+0000 7fa473400640 1 -- 172.21.15.7:0/1000626770 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4741648d0 con 0x7fa474160590 2024-04-25T11:10:51.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.440+0000 7fa473400640 1 --2- 172.21.15.7:0/1000626770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa474160590 0x7fa4740d3d30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4640097a0 tx=0x7fa46402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d2ef4be16d64ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:51.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.440+0000 7fa471600640 1 -- 172.21.15.7:0/1000626770 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464037070 con 0x7fa474160590 2024-04-25T11:10:51.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.440+0000 7fa471600640 1 -- 172.21.15.7:0/1000626770 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa464032440 con 0x7fa474160590 2024-04-25T11:10:51.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.440+0000 7fa471600640 1 -- 172.21.15.7:0/1000626770 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464032810 con 0x7fa474160590 2024-04-25T11:10:51.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.440+0000 7fa473e00640 1 -- 172.21.15.7:0/1000626770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa474160590 msgr2=0x7fa4740d3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:51.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.440+0000 7fa473e00640 1 --2- 172.21.15.7:0/1000626770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa474160590 0x7fa4740d3d30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4640097a0 tx=0x7fa46402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.441+0000 7fa473e00640 1 -- 172.21.15.7:0/1000626770 shutdown_connections 2024-04-25T11:10:51.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.441+0000 7fa473e00640 1 --2- 172.21.15.7:0/1000626770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa474160590 0x7fa4740d3d30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.441+0000 7fa473e00640 1 -- 172.21.15.7:0/1000626770 >> 172.21.15.7:0/1000626770 conn(0x7fa4740d29e0 msgr2=0x7fa4740d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:51.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.441+0000 7fa473e00640 1 -- 172.21.15.7:0/1000626770 shutdown_connections 2024-04-25T11:10:51.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.441+0000 7fa473e00640 1 -- 172.21.15.7:0/1000626770 wait complete. 2024-04-25T11:10:51.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.443+0000 7fa473e00640 1 Processor -- start 2024-04-25T11:10:51.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.443+0000 7fa473e00640 1 -- start start 2024-04-25T11:10:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.445+0000 7fa473e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa47406dcd0 0x7fa4740cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.445+0000 7fa473e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa474160590 0x7fa474156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.445+0000 7fa473e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa474163ac0 0x7fa474156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.445+0000 7fa473e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa4740d43f0 con 0x7fa47406dcd0 2024-04-25T11:10:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.445+0000 7fa473e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa4740d4270 con 0x7fa474163ac0 2024-04-25T11:10:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.445+0000 7fa473e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa4740d4570 con 0x7fa474160590 2024-04-25T11:10:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.445+0000 7fa472a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa47406dcd0 0x7fa4740cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.445+0000 7fa472a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa47406dcd0 0x7fa4740cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48584/0 (socket says 172.21.15.7:48584) 2024-04-25T11:10:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.445+0000 7fa472a00640 1 -- 172.21.15.7:0/1307271179 learned_addr learned my addr 172.21.15.7:0/1307271179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:51.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.445+0000 7fa473400640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa474163ac0 0x7fa474156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.445+0000 7fa472000640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa474160590 0x7fa474156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.446+0000 7fa472a00640 1 -- 172.21.15.7:0/1307271179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa474160590 msgr2=0x7fa474156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.446+0000 7fa472a00640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa474160590 0x7fa474156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.446+0000 7fa472a00640 1 -- 172.21.15.7:0/1307271179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa474163ac0 msgr2=0x7fa474156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.446+0000 7fa472a00640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa474163ac0 0x7fa474156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.446+0000 7fa472a00640 1 -- 172.21.15.7:0/1307271179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4741f63e0 con 0x7fa47406dcd0 2024-04-25T11:10:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.446+0000 7fa472000640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa474160590 0x7fa474156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:51.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.446+0000 7fa472a00640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa47406dcd0 0x7fa4740cfbc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00cbc0 tx=0x7fa46c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.446+0000 7fa46aa00640 1 -- 172.21.15.7:0/1307271179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c017070 con 0x7fa47406dcd0 2024-04-25T11:10:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.446+0000 7fa46aa00640 1 -- 172.21.15.7:0/1307271179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46c00ad40 con 0x7fa47406dcd0 2024-04-25T11:10:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.446+0000 7fa46aa00640 1 -- 172.21.15.7:0/1307271179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c012620 con 0x7fa47406dcd0 2024-04-25T11:10:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.446+0000 7fa473e00640 1 -- 172.21.15.7:0/1307271179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4741f66c0 con 0x7fa47406dcd0 2024-04-25T11:10:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.447+0000 7fa473e00640 1 -- 172.21.15.7:0/1307271179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4741f6c80 con 0x7fa47406dcd0 2024-04-25T11:10:51.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.448+0000 7fa473e00640 1 -- 172.21.15.7:0/1307271179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa474050740 con 0x7fa47406dcd0 2024-04-25T11:10:51.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.449+0000 7fa46aa00640 1 -- 172.21.15.7:0/1307271179 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa46c01a070 con 0x7fa47406dcd0 2024-04-25T11:10:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.450+0000 7fa46aa00640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4540744b0 0x7fa4540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.450+0000 7fa46aa00640 1 -- 172.21.15.7:0/1307271179 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa46c0953b0 con 0x7fa47406dcd0 2024-04-25T11:10:51.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.450+0000 7fa472000640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4540744b0 0x7fa4540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:51.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.451+0000 7fa472000640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4540744b0 0x7fa4540769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa474158890 tx=0x7fa460009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:51.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.455+0000 7fa46aa00640 1 -- 172.21.15.7:0/1307271179 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa46c05f7f0 con 0x7fa47406dcd0 2024-04-25T11:10:51.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.676+0000 7fa473e00640 1 -- 172.21.15.7:0/1307271179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa47415ff50 con 0x7fa47406dcd0 2024-04-25T11:10:51.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.680+0000 7fa46aa00640 1 -- 172.21.15.7:0/1307271179 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18418 (secure 0 0 0) 0x7fa46c05f0f0 con 0x7fa47406dcd0 2024-04-25T11:10:51.682 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:51.682 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:28.084739+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:51.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 -- 172.21.15.7:0/1307271179 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4540744b0 msgr2=0x7fa4540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4540744b0 0x7fa4540769c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa474158890 tx=0x7fa460009040 comp rx=0 tx=0).stop 2024-04-25T11:10:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 -- 172.21.15.7:0/1307271179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa47406dcd0 msgr2=0x7fa4740cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa47406dcd0 0x7fa4740cfbc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00cbc0 tx=0x7fa46c00a430 comp rx=0 tx=0).stop 2024-04-25T11:10:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 -- 172.21.15.7:0/1307271179 shutdown_connections 2024-04-25T11:10:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4540744b0 0x7fa4540769c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa474163ac0 0x7fa474156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa474160590 0x7fa474156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 --2- 172.21.15.7:0/1307271179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa47406dcd0 0x7fa4740cfbc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 -- 172.21.15.7:0/1307271179 >> 172.21.15.7:0/1307271179 conn(0x7fa4740d29e0 msgr2=0x7fa474161570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 -- 172.21.15.7:0/1307271179 shutdown_connections 2024-04-25T11:10:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.683+0000 7fa473e00640 1 -- 172.21.15.7:0/1307271179 wait complete. 2024-04-25T11:10:51.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:10:52.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.999+0000 7fade6e00640 1 Processor -- start 2024-04-25T11:10:52.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:51.999+0000 7fade6e00640 1 -- start start 2024-04-25T11:10:52.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.001+0000 7fade6e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fade01583c0 0x7fade00cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:52.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.001+0000 7fade6e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fade00cfb50 0x7fade00d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:52.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.001+0000 7fade6e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fade00ce570 con 0x7fade00d09c0 2024-04-25T11:10:52.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.001+0000 7fade6e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fade015f190 con 0x7fade01583c0 2024-04-25T11:10:52.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.001+0000 7fade6e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fade00ce850 con 0x7fade00cfb50 2024-04-25T11:10:52.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.001+0000 7fade5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fade01583c0 0x7fade00cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:52.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.002+0000 7fade5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fade01583c0 0x7fade00cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48592/0 (socket says 172.21.15.7:48592) 2024-04-25T11:10:52.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.002+0000 7fade5a00640 1 -- 172.21.15.7:0/2179134323 learned_addr learned my addr 172.21.15.7:0/2179134323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:52.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.002+0000 7fade5a00640 1 -- 172.21.15.7:0/2179134323 >> v2:172.21.15.87:3300/0 conn(0x7fade00cfb50 msgr2=0x7fade00d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:52.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.002+0000 7fade5a00640 1 --2- 172.21.15.7:0/2179134323 >> v2:172.21.15.87:3300/0 conn(0x7fade00cfb50 0x7fade00d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:52.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.002+0000 7fade5a00640 1 -- 172.21.15.7:0/2179134323 >> v1:172.21.15.87:6789/0 conn(0x7fade00d09c0 legacy=0x7fade006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:52.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.002+0000 7fade5a00640 1 -- 172.21.15.7:0/2179134323 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade00ceb30 con 0x7fade01583c0 2024-04-25T11:10:52.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.003+0000 7fade5a00640 1 --2- 172.21.15.7:0/2179134323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade01583c0 0x7fade00cf610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fade0070ad0 tx=0x7fadd402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bfea7eea791b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.004+0000 7faddfe00640 1 -- 172.21.15.7:0/2179134323 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4037070 con 0x7fade01583c0 2024-04-25T11:10:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.004+0000 7faddfe00640 1 -- 172.21.15.7:0/2179134323 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd4032440 con 0x7fade01583c0 2024-04-25T11:10:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.004+0000 7faddfe00640 1 -- 172.21.15.7:0/2179134323 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4032810 con 0x7fade01583c0 2024-04-25T11:10:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.004+0000 7fade6e00640 1 -- 172.21.15.7:0/2179134323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade01583c0 msgr2=0x7fade00cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.004+0000 7fade6e00640 1 --2- 172.21.15.7:0/2179134323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade01583c0 0x7fade00cf610 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fade0070ad0 tx=0x7fadd402a1c0 comp rx=0 tx=0).stop 2024-04-25T11:10:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.004+0000 7fade6e00640 1 -- 172.21.15.7:0/2179134323 shutdown_connections 2024-04-25T11:10:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.004+0000 7fade6e00640 1 --2- 172.21.15.7:0/2179134323 >> v2:172.21.15.87:3300/0 conn(0x7fade00cfb50 0x7fade00d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.004+0000 7fade6e00640 1 --2- 172.21.15.7:0/2179134323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade01583c0 0x7fade00cf610 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.004+0000 7fade6e00640 1 -- 172.21.15.7:0/2179134323 >> 172.21.15.7:0/2179134323 conn(0x7fade00740f0 msgr2=0x7fade006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.005+0000 7fade6e00640 1 -- 172.21.15.7:0/2179134323 shutdown_connections 2024-04-25T11:10:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.005+0000 7fade6e00640 1 -- 172.21.15.7:0/2179134323 wait complete. 2024-04-25T11:10:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.007+0000 7fade6e00640 1 Processor -- start 2024-04-25T11:10:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.007+0000 7fade6e00640 1 -- start start 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.008+0000 7fade6e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fade00cfb50 0x7fade0150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade6e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fade00d09c0 0x7fade014aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade6e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade01583c0 0x7fade01f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade6e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fade006f5b0 con 0x7fade01583c0 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade6e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fade00cf0f0 con 0x7fade00cfb50 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade6e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fade006f730 con 0x7fade00d09c0 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade01583c0 0x7fade01f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade01583c0 0x7fade01f7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48600/0 (socket says 172.21.15.7:48600) 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade6400640 1 -- 172.21.15.7:0/1509764059 learned_addr learned my addr 172.21.15.7:0/1509764059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade5a00640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fade00cfb50 0x7fade0150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade5000640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fade00d09c0 0x7fade014aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade6400640 1 -- 172.21.15.7:0/1509764059 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fade00d09c0 msgr2=0x7fade014aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade6400640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fade00d09c0 0x7fade014aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.009+0000 7fade6400640 1 -- 172.21.15.7:0/1509764059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fade00cfb50 msgr2=0x7fade0150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:52.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.010+0000 7fade6400640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fade00cfb50 0x7fade0150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:52.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.010+0000 7fade6400640 1 -- 172.21.15.7:0/1509764059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade01503e0 con 0x7fade01583c0 2024-04-25T11:10:52.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.010+0000 7fade6400640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade01583c0 0x7fade01f7800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fadd800b9e0 tx=0x7fadd800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:52.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.010+0000 7faddd600640 1 -- 172.21.15.7:0/1509764059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8019070 con 0x7fade01583c0 2024-04-25T11:10:52.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.010+0000 7faddd600640 1 -- 172.21.15.7:0/1509764059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd800cb20 con 0x7fade01583c0 2024-04-25T11:10:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.010+0000 7faddd600640 1 -- 172.21.15.7:0/1509764059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd8014440 con 0x7fade01583c0 2024-04-25T11:10:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.010+0000 7fade6e00640 1 -- 172.21.15.7:0/1509764059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade01506c0 con 0x7fade01583c0 2024-04-25T11:10:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.010+0000 7fade6e00640 1 -- 172.21.15.7:0/1509764059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade01e89e0 con 0x7fade01583c0 2024-04-25T11:10:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.012+0000 7fade6e00640 1 -- 172.21.15.7:0/1509764059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade0150150 con 0x7fade01583c0 2024-04-25T11:10:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.013+0000 7faddd600640 1 -- 172.21.15.7:0/1509764059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fadd801c070 con 0x7fade01583c0 2024-04-25T11:10:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.014+0000 7faddd600640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fadc00744e0 0x7fadc00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.014+0000 7faddd600640 1 -- 172.21.15.7:0/1509764059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fadd8095310 con 0x7fade01583c0 2024-04-25T11:10:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.015+0000 7fade5a00640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fadc00744e0 0x7fadc00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.016+0000 7fade5a00640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fadc00744e0 0x7fadc00769f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fadd4034000 tx=0x7fadd402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.019+0000 7faddd600640 1 -- 172.21.15.7:0/1509764059 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fadd805f6c0 con 0x7fade01583c0 2024-04-25T11:10:52.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.263+0000 7fade6e00640 1 -- 172.21.15.7:0/1509764059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fade0050740 con 0x7fade01583c0 2024-04-25T11:10:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.928+0000 7faddd600640 1 -- 172.21.15.7:0/1509764059 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fadd805efc0 con 0x7fade01583c0 2024-04-25T11:10:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.931+0000 7fade6e00640 1 -- 172.21.15.7:0/1509764059 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fadc00744e0 msgr2=0x7fadc00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.931+0000 7fade6e00640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fadc00744e0 0x7fadc00769f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fadd4034000 tx=0x7fadd402e000 comp rx=0 tx=0).stop 2024-04-25T11:10:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.931+0000 7fade6e00640 1 -- 172.21.15.7:0/1509764059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade01583c0 msgr2=0x7fade01f7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.931+0000 7fade6e00640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade01583c0 0x7fade01f7800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fadd800b9e0 tx=0x7fadd800bea0 comp rx=0 tx=0).stop 2024-04-25T11:10:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.932+0000 7fade6e00640 1 -- 172.21.15.7:0/1509764059 shutdown_connections 2024-04-25T11:10:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.932+0000 7fade6e00640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fadc00744e0 0x7fadc00769f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.932+0000 7fade6e00640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade01583c0 0x7fade01f7800 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.932+0000 7fade6e00640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fade00d09c0 0x7fade014aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.932+0000 7fade6e00640 1 --2- 172.21.15.7:0/1509764059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fade00cfb50 0x7fade0150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.932+0000 7fade6e00640 1 -- 172.21.15.7:0/1509764059 >> 172.21.15.7:0/1509764059 conn(0x7fade00740f0 msgr2=0x7fade006d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.932+0000 7fade6e00640 1 -- 172.21.15.7:0/1509764059 shutdown_connections 2024-04-25T11:10:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:52.932+0000 7fade6e00640 1 -- 172.21.15.7:0/1509764059 wait complete. 2024-04-25T11:10:52.966 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T11:10:52.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:10:53.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.259+0000 7f9a15c00640 1 Processor -- start 2024-04-25T11:10:53.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.259+0000 7f9a15c00640 1 -- start start 2024-04-25T11:10:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.262+0000 7f9a15c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9a100d0ee0 0x7f9a100affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.262+0000 7f9a15c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a100b5020 0x7f9a10160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.262+0000 7f9a15c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a1006f5b0 con 0x7f9a1015aaa0 2024-04-25T11:10:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.262+0000 7f9a15c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9a100af290 con 0x7f9a100d0ee0 2024-04-25T11:10:53.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.262+0000 7f9a15c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9a100d14a0 con 0x7f9a100b5020 2024-04-25T11:10:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.262+0000 7f9a0f400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a100b5020 0x7f9a10160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.262+0000 7f9a0f400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a100b5020 0x7f9a10160990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43176/0 (socket says 172.21.15.7:43176) 2024-04-25T11:10:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.262+0000 7f9a0f400640 1 -- 172.21.15.7:0/2983885492 learned_addr learned my addr 172.21.15.7:0/2983885492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.262+0000 7f9a06000640 1 --2- 172.21.15.7:0/2983885492 >> v2:172.21.15.87:3300/0 conn(0x7f9a100d0ee0 0x7f9a100affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.263+0000 7f9a0f400640 1 -- 172.21.15.7:0/2983885492 >> v2:172.21.15.87:3300/0 conn(0x7f9a100d0ee0 msgr2=0x7f9a100affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.263+0000 7f9a0f400640 1 --2- 172.21.15.7:0/2983885492 >> v2:172.21.15.87:3300/0 conn(0x7f9a100d0ee0 0x7f9a100affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.263+0000 7f9a0f400640 1 -- 172.21.15.7:0/2983885492 >> v1:172.21.15.87:6789/0 conn(0x7f9a1015aaa0 legacy=0x7f9a100b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:53.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.263+0000 7f9a0f400640 1 -- 172.21.15.7:0/2983885492 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a100af4e0 con 0x7f9a100b5020 2024-04-25T11:10:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.264+0000 7f9a0f400640 1 --2- 172.21.15.7:0/2983885492 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a100b5020 0x7f9a10160990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800d150 tx=0x7f9a0802c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6721863d7763f84f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.264+0000 7f9a0e000640 1 -- 172.21.15.7:0/2983885492 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08009600 con 0x7f9a100b5020 2024-04-25T11:10:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.264+0000 7f9a0e000640 1 -- 172.21.15.7:0/2983885492 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a08037070 con 0x7f9a100b5020 2024-04-25T11:10:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.264+0000 7f9a0e000640 1 -- 172.21.15.7:0/2983885492 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a080099c0 con 0x7f9a100b5020 2024-04-25T11:10:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.265+0000 7f9a15c00640 1 -- 172.21.15.7:0/2983885492 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a100b5020 msgr2=0x7f9a10160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.265+0000 7f9a15c00640 1 --2- 172.21.15.7:0/2983885492 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a100b5020 0x7f9a10160990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800d150 tx=0x7f9a0802c930 comp rx=0 tx=0).stop 2024-04-25T11:10:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.265+0000 7f9a15c00640 1 -- 172.21.15.7:0/2983885492 shutdown_connections 2024-04-25T11:10:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.265+0000 7f9a15c00640 1 --2- 172.21.15.7:0/2983885492 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a100b5020 0x7f9a10160990 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.265+0000 7f9a15c00640 1 --2- 172.21.15.7:0/2983885492 >> v2:172.21.15.87:3300/0 conn(0x7f9a100d0ee0 0x7f9a100affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.265+0000 7f9a15c00640 1 -- 172.21.15.7:0/2983885492 >> 172.21.15.7:0/2983885492 conn(0x7f9a100742c0 msgr2=0x7f9a1006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.265+0000 7f9a15c00640 1 -- 172.21.15.7:0/2983885492 shutdown_connections 2024-04-25T11:10:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.265+0000 7f9a15c00640 1 -- 172.21.15.7:0/2983885492 wait complete. 2024-04-25T11:10:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.268+0000 7f9a15c00640 1 Processor -- start 2024-04-25T11:10:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.268+0000 7f9a15c00640 1 -- start start 2024-04-25T11:10:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.269+0000 7f9a15c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a100b5020 0x7f9a101f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.269+0000 7f9a15c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a100d0ee0 0x7f9a100b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.269+0000 7f9a15c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a1015aaa0 0x7f9a1009e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.269+0000 7f9a15c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a100b5430 con 0x7f9a100b5020 2024-04-25T11:10:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.269+0000 7f9a15c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9a100afaa0 con 0x7f9a100d0ee0 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a15c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9a100b55b0 con 0x7f9a1015aaa0 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a0ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a100b5020 0x7f9a101f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a0ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a100b5020 0x7f9a101f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48608/0 (socket says 172.21.15.7:48608) 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a0ea00640 1 -- 172.21.15.7:0/615271220 learned_addr learned my addr 172.21.15.7:0/615271220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a0f400640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a1015aaa0 0x7f9a1009e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a06000640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a100d0ee0 0x7f9a100b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a0ea00640 1 -- 172.21.15.7:0/615271220 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a1015aaa0 msgr2=0x7f9a1009e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a0ea00640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a1015aaa0 0x7f9a1009e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a0ea00640 1 -- 172.21.15.7:0/615271220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a100d0ee0 msgr2=0x7f9a100b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a0ea00640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a100d0ee0 0x7f9a100b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a0ea00640 1 -- 172.21.15.7:0/615271220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a100b25a0 con 0x7f9a100b5020 2024-04-25T11:10:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.270+0000 7f9a06000640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a100d0ee0 0x7f9a100b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.271+0000 7f9a0ea00640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a100b5020 0x7f9a101f2b60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c3c0 tx=0x7f9a0000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.271+0000 7f9a07400640 1 -- 172.21.15.7:0/615271220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a000052d0 con 0x7f9a100b5020 2024-04-25T11:10:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.271+0000 7f9a07400640 1 -- 172.21.15.7:0/615271220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a00005490 con 0x7f9a100b5020 2024-04-25T11:10:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.271+0000 7f9a15c00640 1 -- 172.21.15.7:0/615271220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a100b2880 con 0x7f9a100b5020 2024-04-25T11:10:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.271+0000 7f9a07400640 1 -- 172.21.15.7:0/615271220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00007260 con 0x7f9a100b5020 2024-04-25T11:10:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.271+0000 7f9a15c00640 1 -- 172.21.15.7:0/615271220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a100b2e40 con 0x7f9a100b5020 2024-04-25T11:10:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.273+0000 7f9a15c00640 1 -- 172.21.15.7:0/615271220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a10050740 con 0x7f9a100b5020 2024-04-25T11:10:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.274+0000 7f9a07400640 1 -- 172.21.15.7:0/615271220 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9a00007400 con 0x7f9a100b5020 2024-04-25T11:10:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.275+0000 7f9a07400640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f99ec0744b0 0x7f99ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.275+0000 7f9a07400640 1 -- 172.21.15.7:0/615271220 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9a00094a30 con 0x7f9a100b5020 2024-04-25T11:10:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.275+0000 7f9a06000640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f99ec0744b0 0x7f99ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.276+0000 7f9a06000640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f99ec0744b0 0x7f99ec0769c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a1015c9d0 tx=0x7f99f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.280+0000 7f9a07400640 1 -- 172.21.15.7:0/615271220 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a0005ee70 con 0x7f9a100b5020 2024-04-25T11:10:53.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.523+0000 7f9a15c00640 1 -- 172.21.15.7:0/615271220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9a100d07d0 con 0x7f9a100b5020 2024-04-25T11:10:53.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.941+0000 7f9a07400640 1 -- 172.21.15.7:0/615271220 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f9a0005e770 con 0x7f9a100b5020 2024-04-25T11:10:53.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.944+0000 7f9a15c00640 1 -- 172.21.15.7:0/615271220 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f99ec0744b0 msgr2=0x7f99ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:53.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.944+0000 7f9a15c00640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f99ec0744b0 0x7f99ec0769c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a1015c9d0 tx=0x7f99f8008040 comp rx=0 tx=0).stop 2024-04-25T11:10:53.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.944+0000 7f9a15c00640 1 -- 172.21.15.7:0/615271220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a100b5020 msgr2=0x7f9a101f2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:53.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.944+0000 7f9a15c00640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a100b5020 0x7f9a101f2b60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000c3c0 tx=0x7f9a0000c880 comp rx=0 tx=0).stop 2024-04-25T11:10:53.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.944+0000 7f9a15c00640 1 -- 172.21.15.7:0/615271220 shutdown_connections 2024-04-25T11:10:53.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.944+0000 7f9a15c00640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f99ec0744b0 0x7f99ec0769c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:53.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.944+0000 7f9a15c00640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a1015aaa0 0x7f9a1009e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:53.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.944+0000 7f9a15c00640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a100d0ee0 0x7f9a100b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:53.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.944+0000 7f9a15c00640 1 --2- 172.21.15.7:0/615271220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a100b5020 0x7f9a101f2b60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:53.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.945+0000 7f9a15c00640 1 -- 172.21.15.7:0/615271220 >> 172.21.15.7:0/615271220 conn(0x7f9a100742c0 msgr2=0x7f9a1006cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:53.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.945+0000 7f9a15c00640 1 -- 172.21.15.7:0/615271220 shutdown_connections 2024-04-25T11:10:53.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:53.945+0000 7f9a15c00640 1 -- 172.21.15.7:0/615271220 wait complete. 2024-04-25T11:10:53.979 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:10:53.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:10:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.264+0000 7fdf2f200640 1 Processor -- start 2024-04-25T11:10:54.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.264+0000 7fdf2f200640 1 -- start start 2024-04-25T11:10:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.265+0000 7fdf2f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf300aed40 0x7fdf3015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.265+0000 7fdf2f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdf300b5220 0x7fdf300ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.265+0000 7fdf2f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf300b64b0 con 0x7fdf300af420 2024-04-25T11:10:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.265+0000 7fdf2f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fdf3015e1f0 con 0x7fdf300b5220 2024-04-25T11:10:54.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.265+0000 7fdf2f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdf3006c610 con 0x7fdf300aed40 2024-04-25T11:10:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.266+0000 7fdf2d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdf300b5220 0x7fdf300ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.266+0000 7fdf2de00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf300aed40 0x7fdf3015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.266+0000 7fdf2d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdf300b5220 0x7fdf300ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50136/0 (socket says 172.21.15.7:50136) 2024-04-25T11:10:54.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.266+0000 7fdf2d400640 1 -- 172.21.15.7:0/2903708026 learned_addr learned my addr 172.21.15.7:0/2903708026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.266+0000 7fdf2d400640 1 -- 172.21.15.7:0/2903708026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf300aed40 msgr2=0x7fdf3015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.266+0000 7fdf2d400640 1 --2- 172.21.15.7:0/2903708026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf300aed40 0x7fdf3015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.266+0000 7fdf2d400640 1 -- 172.21.15.7:0/2903708026 >> v1:172.21.15.87:6789/0 conn(0x7fdf300af420 legacy=0x7fdf300af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.266+0000 7fdf2d400640 1 -- 172.21.15.7:0/2903708026 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3006c8f0 con 0x7fdf300b5220 2024-04-25T11:10:54.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.268+0000 7fdf2d400640 1 --2- 172.21.15.7:0/2903708026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf300b5220 0x7fdf300ce4a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009870 tx=0x7fdf1c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e182209cc6513c30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.268+0000 7fdf2ca00640 1 -- 172.21.15.7:0/2903708026 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c037070 con 0x7fdf300b5220 2024-04-25T11:10:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.268+0000 7fdf2ca00640 1 -- 172.21.15.7:0/2903708026 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c032440 con 0x7fdf300b5220 2024-04-25T11:10:54.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.268+0000 7fdf2ca00640 1 -- 172.21.15.7:0/2903708026 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c032810 con 0x7fdf300b5220 2024-04-25T11:10:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.269+0000 7fdf2f200640 1 -- 172.21.15.7:0/2903708026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf300b5220 msgr2=0x7fdf300ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.269+0000 7fdf2f200640 1 --2- 172.21.15.7:0/2903708026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf300b5220 0x7fdf300ce4a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009870 tx=0x7fdf1c02a270 comp rx=0 tx=0).stop 2024-04-25T11:10:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.269+0000 7fdf2f200640 1 -- 172.21.15.7:0/2903708026 shutdown_connections 2024-04-25T11:10:54.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.269+0000 7fdf2f200640 1 --2- 172.21.15.7:0/2903708026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf300b5220 0x7fdf300ce4a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.269+0000 7fdf2f200640 1 --2- 172.21.15.7:0/2903708026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf300aed40 0x7fdf3015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.269+0000 7fdf2f200640 1 -- 172.21.15.7:0/2903708026 >> 172.21.15.7:0/2903708026 conn(0x7fdf300b3820 msgr2=0x7fdf300b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.269+0000 7fdf2f200640 1 -- 172.21.15.7:0/2903708026 shutdown_connections 2024-04-25T11:10:54.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.269+0000 7fdf2f200640 1 -- 172.21.15.7:0/2903708026 wait complete. 2024-04-25T11:10:54.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.273+0000 7fdf2f200640 1 Processor -- start 2024-04-25T11:10:54.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.273+0000 7fdf2f200640 1 -- start start 2024-04-25T11:10:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf300aed40 0x7fdf30150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf300af420 0x7fdf300709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf300b5220 0x7fdf30159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf300b6110 con 0x7fdf300af420 2024-04-25T11:10:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3006ceb0 con 0x7fdf300b5220 2024-04-25T11:10:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdf300b6290 con 0x7fdf300aed40 2024-04-25T11:10:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf300af420 0x7fdf300709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf300af420 0x7fdf300709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48624/0 (socket says 172.21.15.7:48624) 2024-04-25T11:10:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2d400640 1 -- 172.21.15.7:0/503317287 learned_addr learned my addr 172.21.15.7:0/503317287 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2de00640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf300aed40 0x7fdf30150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2e800640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf300b5220 0x7fdf30159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2d400640 1 -- 172.21.15.7:0/503317287 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf300aed40 msgr2=0x7fdf30150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:54.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.274+0000 7fdf2d400640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf300aed40 0x7fdf30150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.275+0000 7fdf2d400640 1 -- 172.21.15.7:0/503317287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf300b5220 msgr2=0x7fdf30159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:54.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.275+0000 7fdf2d400640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf300b5220 0x7fdf30159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.275+0000 7fdf2d400640 1 -- 172.21.15.7:0/503317287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3006fda0 con 0x7fdf300af420 2024-04-25T11:10:54.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.275+0000 7fdf2e800640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf300b5220 0x7fdf30159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:54.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.275+0000 7fdf2d400640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf300af420 0x7fdf300709a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c02a2b0 tx=0x7fdf1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:54.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.275+0000 7fdf26000640 1 -- 172.21.15.7:0/503317287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c041030 con 0x7fdf300af420 2024-04-25T11:10:54.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.275+0000 7fdf26000640 1 -- 172.21.15.7:0/503317287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1c030440 con 0x7fdf300af420 2024-04-25T11:10:54.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.275+0000 7fdf26000640 1 -- 172.21.15.7:0/503317287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c037040 con 0x7fdf300af420 2024-04-25T11:10:54.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.275+0000 7fdf2f200640 1 -- 172.21.15.7:0/503317287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3006f500 con 0x7fdf300af420 2024-04-25T11:10:54.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.275+0000 7fdf2f200640 1 -- 172.21.15.7:0/503317287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3006fac0 con 0x7fdf300af420 2024-04-25T11:10:54.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.277+0000 7fdf2f200640 1 -- 172.21.15.7:0/503317287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf3006f220 con 0x7fdf300af420 2024-04-25T11:10:54.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.279+0000 7fdf26000640 1 -- 172.21.15.7:0/503317287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf1c044070 con 0x7fdf300af420 2024-04-25T11:10:54.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.279+0000 7fdf26000640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf100744b0 0x7fdf100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.279+0000 7fdf26000640 1 -- 172.21.15.7:0/503317287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdf1c0b5ce0 con 0x7fdf300af420 2024-04-25T11:10:54.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.279+0000 7fdf2de00640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf100744b0 0x7fdf100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.280+0000 7fdf2de00640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf100744b0 0x7fdf100769c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf30071200 tx=0x7fdf20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:54.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.284+0000 7fdf26000640 1 -- 172.21.15.7:0/503317287 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf1c080090 con 0x7fdf300af420 2024-04-25T11:10:54.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.532+0000 7fdf2f200640 1 -- 172.21.15.7:0/503317287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf3006e0b0 con 0x7fdf300af420 2024-04-25T11:10:54.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.534+0000 7fdf26000640 1 -- 172.21.15.7:0/503317287 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7fdf1c088110 con 0x7fdf300af420 2024-04-25T11:10:54.534 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:54.535 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3458223722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3458223722},{"type":"v1","addr":"172.21.15.87:6835","nonce":3458223722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1383067804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1383067804},{"type":"v1","addr":"172.21.15.7:6835","nonce":1383067804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/435824250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":435824250},{"type":"v1","addr":"172.21.15.7:6837","nonce":435824250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:10:23.023233+0000","modified":"2024-04-25T11:10:53.526222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1091275258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1091275258},{"type":"v1","addr":"172.21.15.87:6834","nonce":1091275258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:10:54.535 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-04-25T11:10:54.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 -- 172.21.15.7:0/503317287 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf100744b0 msgr2=0x7fdf100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:54.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf100744b0 0x7fdf100769c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf30071200 tx=0x7fdf20008040 comp rx=0 tx=0).stop 2024-04-25T11:10:54.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 -- 172.21.15.7:0/503317287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf300af420 msgr2=0x7fdf300709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:54.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf300af420 0x7fdf300709a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c02a2b0 tx=0x7fdf1c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 -- 172.21.15.7:0/503317287 shutdown_connections 2024-04-25T11:10:54.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf100744b0 0x7fdf100769c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf300b5220 0x7fdf30159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf300af420 0x7fdf300709a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 --2- 172.21.15.7:0/503317287 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf300aed40 0x7fdf30150170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 -- 172.21.15.7:0/503317287 >> 172.21.15.7:0/503317287 conn(0x7fdf300b3820 msgr2=0x7fdf300b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 -- 172.21.15.7:0/503317287 shutdown_connections 2024-04-25T11:10:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.536+0000 7fdf2f200640 1 -- 172.21.15.7:0/503317287 wait complete. 2024-04-25T11:10:54.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.843+0000 7f66e7e00640 1 Processor -- start 2024-04-25T11:10:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.843+0000 7f66e7e00640 1 -- start start 2024-04-25T11:10:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66e8163ac0 0x7f66e80b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66e806dcd0 0x7f66e806e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e7e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66e80d35a0 con 0x7f66e80d0570 2024-04-25T11:10:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e7e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f66e806e620 con 0x7f66e806dcd0 2024-04-25T11:10:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e7e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f66e8164310 con 0x7f66e8163ac0 2024-04-25T11:10:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e6000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66e806dcd0 0x7f66e806e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e6000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66e806dcd0 0x7f66e806e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48634/0 (socket says 172.21.15.7:48634) 2024-04-25T11:10:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e6000640 1 -- 172.21.15.7:0/2192841955 learned_addr learned my addr 172.21.15.7:0/2192841955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e6a00640 1 --2- 172.21.15.7:0/2192841955 >> v2:172.21.15.87:3300/0 conn(0x7f66e8163ac0 0x7f66e80b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e6000640 1 -- 172.21.15.7:0/2192841955 >> v2:172.21.15.87:3300/0 conn(0x7f66e8163ac0 msgr2=0x7f66e80b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e6000640 1 --2- 172.21.15.7:0/2192841955 >> v2:172.21.15.87:3300/0 conn(0x7f66e8163ac0 0x7f66e80b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e6000640 1 -- 172.21.15.7:0/2192841955 >> v1:172.21.15.87:6789/0 conn(0x7f66e80d0570 legacy=0x7f66e80d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.845+0000 7f66e6000640 1 -- 172.21.15.7:0/2192841955 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e81645f0 con 0x7f66e806dcd0 2024-04-25T11:10:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.846+0000 7f66e6000640 1 --2- 172.21.15.7:0/2192841955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66e806dcd0 0x7f66e806e0e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f66e808e560 tx=0x7f66d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ce34c6c69e2521 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.846+0000 7f66e5600640 1 -- 172.21.15.7:0/2192841955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4037070 con 0x7f66e806dcd0 2024-04-25T11:10:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.847+0000 7f66e5600640 1 -- 172.21.15.7:0/2192841955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d4032440 con 0x7f66e806dcd0 2024-04-25T11:10:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.847+0000 7f66e5600640 1 -- 172.21.15.7:0/2192841955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4032810 con 0x7f66e806dcd0 2024-04-25T11:10:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.847+0000 7f66e7e00640 1 -- 172.21.15.7:0/2192841955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66e806dcd0 msgr2=0x7f66e806e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:54.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.847+0000 7f66e7e00640 1 --2- 172.21.15.7:0/2192841955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66e806dcd0 0x7f66e806e0e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f66e808e560 tx=0x7f66d402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.847+0000 7f66e7e00640 1 -- 172.21.15.7:0/2192841955 shutdown_connections 2024-04-25T11:10:54.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.847+0000 7f66e7e00640 1 --2- 172.21.15.7:0/2192841955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66e806dcd0 0x7f66e806e0e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.847+0000 7f66e7e00640 1 --2- 172.21.15.7:0/2192841955 >> v2:172.21.15.87:3300/0 conn(0x7f66e8163ac0 0x7f66e80b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.847+0000 7f66e7e00640 1 -- 172.21.15.7:0/2192841955 >> 172.21.15.7:0/2192841955 conn(0x7f66e80d29e0 msgr2=0x7f66e80d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:54.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.847+0000 7f66e7e00640 1 -- 172.21.15.7:0/2192841955 shutdown_connections 2024-04-25T11:10:54.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.848+0000 7f66e7e00640 1 -- 172.21.15.7:0/2192841955 wait complete. 2024-04-25T11:10:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.851+0000 7f66e7e00640 1 Processor -- start 2024-04-25T11:10:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.851+0000 7f66e7e00640 1 -- start start 2024-04-25T11:10:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.851+0000 7f66e7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66e806dcd0 0x7f66e80ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.851+0000 7f66e7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66e80d0570 0x7f66e815de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.851+0000 7f66e7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e8163ac0 0x7f66e81eaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.851+0000 7f66e7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8163ed0 con 0x7f66e80d0570 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.851+0000 7f66e7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f66e80b50d0 con 0x7f66e806dcd0 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.851+0000 7f66e7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66e8164050 con 0x7f66e8163ac0 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66e80d0570 0x7f66e815de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66e80d0570 0x7f66e815de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48650/0 (socket says 172.21.15.7:48650) 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e6000640 1 -- 172.21.15.7:0/2684884718 learned_addr learned my addr 172.21.15.7:0/2684884718 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e6a00640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66e806dcd0 0x7f66e80ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e6000640 1 -- 172.21.15.7:0/2684884718 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e8163ac0 msgr2=0x7f66e81eaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e7400640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e8163ac0 0x7f66e81eaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e6000640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e8163ac0 0x7f66e81eaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e6000640 1 -- 172.21.15.7:0/2684884718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66e806dcd0 msgr2=0x7f66e80ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e6000640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66e806dcd0 0x7f66e80ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e6000640 1 -- 172.21.15.7:0/2684884718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e80acc00 con 0x7f66e80d0570 2024-04-25T11:10:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e6a00640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66e806dcd0 0x7f66e80ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.852+0000 7f66e7400640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e8163ac0 0x7f66e81eaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.853+0000 7f66e6000640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66e80d0570 0x7f66e815de30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f66d402a220 tx=0x7f66d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.853+0000 7f66dea00640 1 -- 172.21.15.7:0/2684884718 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4041020 con 0x7f66e80d0570 2024-04-25T11:10:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.853+0000 7f66dea00640 1 -- 172.21.15.7:0/2684884718 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d40306c0 con 0x7f66e80d0570 2024-04-25T11:10:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.853+0000 7f66dea00640 1 -- 172.21.15.7:0/2684884718 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4037040 con 0x7f66e80d0570 2024-04-25T11:10:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.853+0000 7f66e7e00640 1 -- 172.21.15.7:0/2684884718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e80acee0 con 0x7f66e80d0570 2024-04-25T11:10:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.853+0000 7f66e7e00640 1 -- 172.21.15.7:0/2684884718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e80ad7a0 con 0x7f66e80d0570 2024-04-25T11:10:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.854+0000 7f66e7e00640 1 -- 172.21.15.7:0/2684884718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e80ac970 con 0x7f66e80d0570 2024-04-25T11:10:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.856+0000 7f66dea00640 1 -- 172.21.15.7:0/2684884718 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f66d4044020 con 0x7f66e80d0570 2024-04-25T11:10:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.857+0000 7f66dea00640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66c80744b0 0x7f66c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:54.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.857+0000 7f66dea00640 1 -- 172.21.15.7:0/2684884718 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f66d40b5e20 con 0x7f66e80d0570 2024-04-25T11:10:54.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.857+0000 7f66e6a00640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66c80744b0 0x7f66c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:54.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.858+0000 7f66e6a00640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66c80744b0 0x7f66c80769c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66d800b9f0 tx=0x7f66d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:54.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:54.862+0000 7f66dea00640 1 -- 172.21.15.7:0/2684884718 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66d404b030 con 0x7f66e80d0570 2024-04-25T11:10:55.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.066+0000 7f66e7e00640 1 -- 172.21.15.7:0/2684884718 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66e80ad590 con 0x7f66e80d0570 2024-04-25T11:10:55.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.070+0000 7f66dea00640 1 -- 172.21.15.7:0/2684884718 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18418 (secure 0 0 0) 0x7f66d40b2810 con 0x7f66e80d0570 2024-04-25T11:10:55.071 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:55.071 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:28.084739+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 -- 172.21.15.7:0/2684884718 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66c80744b0 msgr2=0x7f66c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66c80744b0 0x7f66c80769c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f66d800b9f0 tx=0x7f66d8008040 comp rx=0 tx=0).stop 2024-04-25T11:10:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 -- 172.21.15.7:0/2684884718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66e80d0570 msgr2=0x7f66e815de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66e80d0570 0x7f66e815de30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f66d402a220 tx=0x7f66d40046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 -- 172.21.15.7:0/2684884718 shutdown_connections 2024-04-25T11:10:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66c80744b0 0x7f66c80769c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e8163ac0 0x7f66e81eaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66e80d0570 0x7f66e815de30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 --2- 172.21.15.7:0/2684884718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66e806dcd0 0x7f66e80ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 -- 172.21.15.7:0/2684884718 >> 172.21.15.7:0/2684884718 conn(0x7f66e80d29e0 msgr2=0x7f66e80bace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 -- 172.21.15.7:0/2684884718 shutdown_connections 2024-04-25T11:10:55.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.073+0000 7f66e7e00640 1 -- 172.21.15.7:0/2684884718 wait complete. 2024-04-25T11:10:55.098 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:10:55.098 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:10:55.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.382+0000 7f57e2200640 1 Processor -- start 2024-04-25T11:10:55.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.382+0000 7f57e2200640 1 -- start start 2024-04-25T11:10:55.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.384+0000 7f57e2200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57dc0aed40 0x7f57dc15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.384+0000 7f57e2200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc0b5220 0x7f57dc0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.384+0000 7f57e2200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57dc0af420 0x7f57dc0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.384+0000 7f57e2200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f57dc15e1f0 con 0x7f57dc0aed40 2024-04-25T11:10:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.384+0000 7f57e2200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f57dc0afd70 con 0x7f57dc0af420 2024-04-25T11:10:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.384+0000 7f57e2200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57dc0b64b0 con 0x7f57dc0b5220 2024-04-25T11:10:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.385+0000 7f57e0e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57dc0aed40 0x7f57dc15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.385+0000 7f57e0e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57dc0aed40 0x7f57dc15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48654/0 (socket says 172.21.15.7:48654) 2024-04-25T11:10:55.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.385+0000 7f57e0e00640 1 -- 172.21.15.7:0/2787742524 learned_addr learned my addr 172.21.15.7:0/2787742524 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.385+0000 7f57e1800640 1 --2- 172.21.15.7:0/2787742524 >> v2:172.21.15.87:3300/0 conn(0x7f57dc0af420 0x7f57dc0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.385+0000 7f57dbe00640 1 --2- 172.21.15.7:0/2787742524 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc0b5220 0x7f57dc0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.385+0000 7f57e0e00640 1 -- 172.21.15.7:0/2787742524 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc0b5220 msgr2=0x7f57dc0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.385+0000 7f57e0e00640 1 --2- 172.21.15.7:0/2787742524 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc0b5220 0x7f57dc0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.385+0000 7f57e0e00640 1 -- 172.21.15.7:0/2787742524 >> v2:172.21.15.87:3300/0 conn(0x7f57dc0af420 msgr2=0x7f57dc0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.385+0000 7f57e0e00640 1 --2- 172.21.15.7:0/2787742524 >> v2:172.21.15.87:3300/0 conn(0x7f57dc0af420 0x7f57dc0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.385+0000 7f57e0e00640 1 -- 172.21.15.7:0/2787742524 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc06c610 con 0x7f57dc0aed40 2024-04-25T11:10:55.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.386+0000 7f57e0e00640 1 --2- 172.21.15.7:0/2787742524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57dc0aed40 0x7f57dc15dcb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f57dc073080 tx=0x7f57cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa7bde03caa9cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:55.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.387+0000 7f57db400640 1 -- 172.21.15.7:0/2787742524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc036070 con 0x7f57dc0aed40 2024-04-25T11:10:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.387+0000 7f57db400640 1 -- 172.21.15.7:0/2787742524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc031600 con 0x7f57dc0aed40 2024-04-25T11:10:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.387+0000 7f57db400640 1 -- 172.21.15.7:0/2787742524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0319d0 con 0x7f57dc0aed40 2024-04-25T11:10:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.388+0000 7f57e2200640 1 -- 172.21.15.7:0/2787742524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57dc0aed40 msgr2=0x7f57dc15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.388+0000 7f57e2200640 1 --2- 172.21.15.7:0/2787742524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57dc0aed40 0x7f57dc15dcb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f57dc073080 tx=0x7f57cc02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.388+0000 7f57e2200640 1 -- 172.21.15.7:0/2787742524 shutdown_connections 2024-04-25T11:10:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.388+0000 7f57e2200640 1 --2- 172.21.15.7:0/2787742524 >> v2:172.21.15.87:3300/0 conn(0x7f57dc0af420 0x7f57dc0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.388+0000 7f57e2200640 1 --2- 172.21.15.7:0/2787742524 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc0b5220 0x7f57dc0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.388+0000 7f57e2200640 1 --2- 172.21.15.7:0/2787742524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57dc0aed40 0x7f57dc15dcb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.388+0000 7f57e2200640 1 -- 172.21.15.7:0/2787742524 >> 172.21.15.7:0/2787742524 conn(0x7f57dc0b3820 msgr2=0x7f57dc0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:55.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.388+0000 7f57e2200640 1 -- 172.21.15.7:0/2787742524 shutdown_connections 2024-04-25T11:10:55.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.388+0000 7f57e2200640 1 -- 172.21.15.7:0/2787742524 wait complete. 2024-04-25T11:10:55.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.390+0000 7f57e2200640 1 Processor -- start 2024-04-25T11:10:55.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.390+0000 7f57e2200640 1 -- start start 2024-04-25T11:10:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.392+0000 7f57e2200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0aed40 0x7f57dc1fa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.392+0000 7f57e2200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc0af420 0x7f57dc070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.392+0000 7f57e2200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57dc0b5220 0x7f57dc15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.392+0000 7f57e2200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc06ceb0 con 0x7f57dc0b5220 2024-04-25T11:10:55.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.392+0000 7f57e2200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc06cbd0 con 0x7f57dc0aed40 2024-04-25T11:10:55.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.392+0000 7f57e2200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57dc0b6110 con 0x7f57dc0af420 2024-04-25T11:10:55.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.392+0000 7f57e1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57dc0b5220 0x7f57dc15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.393+0000 7f57e1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57dc0b5220 0x7f57dc15cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48666/0 (socket says 172.21.15.7:48666) 2024-04-25T11:10:55.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.393+0000 7f57e1800640 1 -- 172.21.15.7:0/2048978566 learned_addr learned my addr 172.21.15.7:0/2048978566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.393+0000 7f57dbe00640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc0af420 0x7f57dc070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.393+0000 7f57e1800640 1 -- 172.21.15.7:0/2048978566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc0af420 msgr2=0x7f57dc070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.393+0000 7f57e1800640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc0af420 0x7f57dc070ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.393+0000 7f57e1800640 1 -- 172.21.15.7:0/2048978566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0aed40 msgr2=0x7f57dc1fa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.393+0000 7f57e0e00640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0aed40 0x7f57dc1fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.393+0000 7f57e1800640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0aed40 0x7f57dc1fa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.393+0000 7f57e1800640 1 -- 172.21.15.7:0/2048978566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc070170 con 0x7f57dc0b5220 2024-04-25T11:10:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.393+0000 7f57dbe00640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc0af420 0x7f57dc070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.393+0000 7f57e1800640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57dc0b5220 0x7f57dc15cef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f57d400e820 tx=0x7f57d400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.394+0000 7f57d8c00640 1 -- 172.21.15.7:0/2048978566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4017070 con 0x7f57dc0b5220 2024-04-25T11:10:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.394+0000 7f57d8c00640 1 -- 172.21.15.7:0/2048978566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d400c730 con 0x7f57dc0b5220 2024-04-25T11:10:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.394+0000 7f57e2200640 1 -- 172.21.15.7:0/2048978566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc0cefe0 con 0x7f57dc0b5220 2024-04-25T11:10:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.394+0000 7f57d8c00640 1 -- 172.21.15.7:0/2048978566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d400cb20 con 0x7f57dc0b5220 2024-04-25T11:10:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.394+0000 7f57e2200640 1 -- 172.21.15.7:0/2048978566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc0cf560 con 0x7f57dc0b5220 2024-04-25T11:10:55.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.395+0000 7f57e2200640 1 -- 172.21.15.7:0/2048978566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57dc06fec0 con 0x7f57dc0b5220 2024-04-25T11:10:55.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.397+0000 7f57d8c00640 1 -- 172.21.15.7:0/2048978566 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f57d401a070 con 0x7f57dc0b5220 2024-04-25T11:10:55.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.398+0000 7f57d8c00640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57b80744e0 0x7f57b80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.398+0000 7f57d8c00640 1 -- 172.21.15.7:0/2048978566 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f57d40953f0 con 0x7f57dc0b5220 2024-04-25T11:10:55.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.398+0000 7f57e0e00640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57b80744e0 0x7f57b80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.399+0000 7f57e0e00640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57b80744e0 0x7f57b80769f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f57cc033000 tx=0x7f57cc009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:55.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.403+0000 7f57d8c00640 1 -- 172.21.15.7:0/2048978566 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57d405f7a0 con 0x7f57dc0b5220 2024-04-25T11:10:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.609+0000 7f57e2200640 1 -- 172.21.15.7:0/2048978566 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f57dc050740 con 0x7f57b80744e0 2024-04-25T11:10:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.620+0000 7f57d8c00640 1 -- 172.21.15.7:0/2048978566 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332526 (secure 0 0 0) 0x7f57dc050740 con 0x7f57b80744e0 2024-04-25T11:10:55.627 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:55.629 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:10:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.627+0000 7f57e2200640 1 -- 172.21.15.7:0/2048978566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57b80744e0 msgr2=0x7f57b80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.627+0000 7f57e2200640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57b80744e0 0x7f57b80769f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f57cc033000 tx=0x7f57cc009000 comp rx=0 tx=0).stop 2024-04-25T11:10:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.627+0000 7f57e2200640 1 -- 172.21.15.7:0/2048978566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57dc0b5220 msgr2=0x7f57dc15cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.627+0000 7f57e2200640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57dc0b5220 0x7f57dc15cef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f57d400e820 tx=0x7f57d400ece0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.628+0000 7f57e2200640 1 -- 172.21.15.7:0/2048978566 shutdown_connections 2024-04-25T11:10:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.628+0000 7f57e2200640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57b80744e0 0x7f57b80769f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.628+0000 7f57e2200640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57dc0b5220 0x7f57dc15cef0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.628+0000 7f57e2200640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc0af420 0x7f57dc070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.628+0000 7f57e2200640 1 --2- 172.21.15.7:0/2048978566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0aed40 0x7f57dc1fa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.628+0000 7f57e2200640 1 -- 172.21.15.7:0/2048978566 >> 172.21.15.7:0/2048978566 conn(0x7f57dc0b3820 msgr2=0x7f57dc15b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.628+0000 7f57e2200640 1 -- 172.21.15.7:0/2048978566 shutdown_connections 2024-04-25T11:10:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.628+0000 7f57e2200640 1 -- 172.21.15.7:0/2048978566 wait complete. 2024-04-25T11:10:55.656 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-25T11:10:53.791196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231960,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742760,"statfs":{"total":773094113280,"available":772856586240,"internally_reserved":0,"allocated":3383296,"data_stored":2025904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005642"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895493+0000","last_change":"2024-04-25T11:10:22.475060+0000","last_active":"2024-04-25T11:10:29.895493+0000","last_peered":"2024-04-25T11:10:29.895493+0000","last_clean":"2024-04-25T11:10:29.895493+0000","last_became_active":"2024-04-25T11:10:22.472574+0000","last_became_peered":"2024-04-25T11:10:22.472574+0000","last_unstale":"2024-04-25T11:10:29.895493+0000","last_undegraded":"2024-04-25T11:10:29.895493+0000","last_fullsized":"2024-04-25T11:10:29.895493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.835819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895751+0000","last_change":"2024-04-25T11:10:23.488367+0000","last_active":"2024-04-25T11:10:29.895751+0000","last_peered":"2024-04-25T11:10:29.895751+0000","last_clean":"2024-04-25T11:10:29.895751+0000","last_became_active":"2024-04-25T11:10:23.488034+0000","last_became_peered":"2024-04-25T11:10:23.488034+0000","last_unstale":"2024-04-25T11:10:29.895751+0000","last_undegraded":"2024-04-25T11:10:29.895751+0000","last_fullsized":"2024-04-25T11:10:29.895751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892242+0000","last_change":"2024-04-25T11:10:27.551067+0000","last_active":"2024-04-25T11:10:29.892242+0000","last_peered":"2024-04-25T11:10:29.892242+0000","last_clean":"2024-04-25T11:10:29.892242+0000","last_became_active":"2024-04-25T11:10:27.550247+0000","last_became_peered":"2024-04-25T11:10:27.550247+0000","last_unstale":"2024-04-25T11:10:29.892242+0000","last_undegraded":"2024-04-25T11:10:29.892242+0000","last_fullsized":"2024-04-25T11:10:29.892242+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:29.432696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895898+0000","last_change":"2024-04-25T11:10:22.463507+0000","last_active":"2024-04-25T11:10:29.895898+0000","last_peered":"2024-04-25T11:10:29.895898+0000","last_clean":"2024-04-25T11:10:29.895898+0000","last_became_active":"2024-04-25T11:10:22.463142+0000","last_became_peered":"2024-04-25T11:10:22.463142+0000","last_unstale":"2024-04-25T11:10:29.895898+0000","last_undegraded":"2024-04-25T11:10:29.895898+0000","last_fullsized":"2024-04-25T11:10:29.895898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:42.000742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894569+0000","last_change":"2024-04-25T11:10:23.500843+0000","last_active":"2024-04-25T11:10:29.894569+0000","last_peered":"2024-04-25T11:10:29.894569+0000","last_clean":"2024-04-25T11:10:29.894569+0000","last_became_active":"2024-04-25T11:10:23.500412+0000","last_became_peered":"2024-04-25T11:10:23.500412+0000","last_unstale":"2024-04-25T11:10:29.894569+0000","last_undegraded":"2024-04-25T11:10:29.894569+0000","last_fullsized":"2024-04-25T11:10:29.894569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534853+0000","last_change":"2024-04-25T11:10:27.565431+0000","last_active":"2024-04-25T11:10:28.534853+0000","last_peered":"2024-04-25T11:10:28.534853+0000","last_clean":"2024-04-25T11:10:28.534853+0000","last_became_active":"2024-04-25T11:10:27.564724+0000","last_became_peered":"2024-04-25T11:10:27.564724+0000","last_unstale":"2024-04-25T11:10:28.534853+0000","last_undegraded":"2024-04-25T11:10:28.534853+0000","last_fullsized":"2024-04-25T11:10:28.534853+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:51.033161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536873+0000","last_change":"2024-04-25T11:10:22.477443+0000","last_active":"2024-04-25T11:10:28.536873+0000","last_peered":"2024-04-25T11:10:28.536873+0000","last_clean":"2024-04-25T11:10:28.536873+0000","last_became_active":"2024-04-25T11:10:22.477131+0000","last_became_peered":"2024-04-25T11:10:22.477131+0000","last_unstale":"2024-04-25T11:10:28.536873+0000","last_undegraded":"2024-04-25T11:10:28.536873+0000","last_fullsized":"2024-04-25T11:10:28.536873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:57.426313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291476+0000","last_change":"2024-04-25T11:10:23.496542+0000","last_active":"2024-04-25T11:10:29.291476+0000","last_peered":"2024-04-25T11:10:29.291476+0000","last_clean":"2024-04-25T11:10:29.291476+0000","last_became_active":"2024-04-25T11:10:23.496211+0000","last_became_peered":"2024-04-25T11:10:23.496211+0000","last_unstale":"2024-04-25T11:10:29.291476+0000","last_undegraded":"2024-04-25T11:10:29.291476+0000","last_fullsized":"2024-04-25T11:10:29.291476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:15.460249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537266+0000","last_change":"2024-04-25T11:10:27.565126+0000","last_active":"2024-04-25T11:10:28.537266+0000","last_peered":"2024-04-25T11:10:28.537266+0000","last_clean":"2024-04-25T11:10:28.537266+0000","last_became_active":"2024-04-25T11:10:27.564664+0000","last_became_peered":"2024-04-25T11:10:27.564664+0000","last_unstale":"2024-04-25T11:10:28.537266+0000","last_undegraded":"2024-04-25T11:10:28.537266+0000","last_fullsized":"2024-04-25T11:10:28.537266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:44.604923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536194+0000","last_change":"2024-04-25T11:10:22.483531+0000","last_active":"2024-04-25T11:10:28.536194+0000","last_peered":"2024-04-25T11:10:28.536194+0000","last_clean":"2024-04-25T11:10:28.536194+0000","last_became_active":"2024-04-25T11:10:22.483204+0000","last_became_peered":"2024-04-25T11:10:22.483204+0000","last_unstale":"2024-04-25T11:10:28.536194+0000","last_undegraded":"2024-04-25T11:10:28.536194+0000","last_fullsized":"2024-04-25T11:10:28.536194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:03.866498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291328+0000","last_change":"2024-04-25T11:10:23.490550+0000","last_active":"2024-04-25T11:10:29.291328+0000","last_peered":"2024-04-25T11:10:29.291328+0000","last_clean":"2024-04-25T11:10:29.291328+0000","last_became_active":"2024-04-25T11:10:23.490300+0000","last_became_peered":"2024-04-25T11:10:23.490300+0000","last_unstale":"2024-04-25T11:10:29.291328+0000","last_undegraded":"2024-04-25T11:10:29.291328+0000","last_fullsized":"2024-04-25T11:10:29.291328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:20.100576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535567+0000","last_change":"2024-04-25T11:10:27.555808+0000","last_active":"2024-04-25T11:10:28.535567+0000","last_peered":"2024-04-25T11:10:28.535567+0000","last_clean":"2024-04-25T11:10:28.535567+0000","last_became_active":"2024-04-25T11:10:27.555277+0000","last_became_peered":"2024-04-25T11:10:27.555277+0000","last_unstale":"2024-04-25T11:10:28.535567+0000","last_undegraded":"2024-04-25T11:10:28.535567+0000","last_fullsized":"2024-04-25T11:10:28.535567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:52.665016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.890747+0000","last_change":"2024-04-25T11:10:23.493263+0000","last_active":"2024-04-25T11:10:29.890747+0000","last_peered":"2024-04-25T11:10:29.890747+0000","last_clean":"2024-04-25T11:10:29.890747+0000","last_became_active":"2024-04-25T11:10:23.489625+0000","last_became_peered":"2024-04-25T11:10:23.489625+0000","last_unstale":"2024-04-25T11:10:29.890747+0000","last_undegraded":"2024-04-25T11:10:29.890747+0000","last_fullsized":"2024-04-25T11:10:29.890747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535220+0000","last_change":"2024-04-25T11:10:22.475840+0000","last_active":"2024-04-25T11:10:28.535220+0000","last_peered":"2024-04-25T11:10:28.535220+0000","last_clean":"2024-04-25T11:10:28.535220+0000","last_became_active":"2024-04-25T11:10:22.475539+0000","last_became_peered":"2024-04-25T11:10:22.475539+0000","last_unstale":"2024-04-25T11:10:28.535220+0000","last_undegraded":"2024-04-25T11:10:28.535220+0000","last_fullsized":"2024-04-25T11:10:28.535220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:49.860961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538827+0000","last_change":"2024-04-25T11:10:27.558933+0000","last_active":"2024-04-25T11:10:28.538827+0000","last_peered":"2024-04-25T11:10:28.538827+0000","last_clean":"2024-04-25T11:10:28.538827+0000","last_became_active":"2024-04-25T11:10:27.558566+0000","last_became_peered":"2024-04-25T11:10:27.558566+0000","last_unstale":"2024-04-25T11:10:28.538827+0000","last_undegraded":"2024-04-25T11:10:28.538827+0000","last_fullsized":"2024-04-25T11:10:28.538827+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:45.815273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892352+0000","last_change":"2024-04-25T11:10:23.482991+0000","last_active":"2024-04-25T11:10:29.892352+0000","last_peered":"2024-04-25T11:10:29.892352+0000","last_clean":"2024-04-25T11:10:29.892352+0000","last_became_active":"2024-04-25T11:10:23.481972+0000","last_became_peered":"2024-04-25T11:10:23.481972+0000","last_unstale":"2024-04-25T11:10:29.892352+0000","last_undegraded":"2024-04-25T11:10:29.892352+0000","last_fullsized":"2024-04-25T11:10:29.892352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533468+0000","last_change":"2024-04-25T11:10:22.469445+0000","last_active":"2024-04-25T11:10:28.533468+0000","last_peered":"2024-04-25T11:10:28.533468+0000","last_clean":"2024-04-25T11:10:28.533468+0000","last_became_active":"2024-04-25T11:10:22.469095+0000","last_became_peered":"2024-04-25T11:10:22.469095+0000","last_unstale":"2024-04-25T11:10:28.533468+0000","last_undegraded":"2024-04-25T11:10:28.533468+0000","last_fullsized":"2024-04-25T11:10:28.533468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537528+0000","last_change":"2024-04-25T11:10:27.559144+0000","last_active":"2024-04-25T11:10:28.537528+0000","last_peered":"2024-04-25T11:10:28.537528+0000","last_clean":"2024-04-25T11:10:28.537528+0000","last_became_active":"2024-04-25T11:10:27.558635+0000","last_became_peered":"2024-04-25T11:10:27.558635+0000","last_unstale":"2024-04-25T11:10:28.537528+0000","last_undegraded":"2024-04-25T11:10:28.537528+0000","last_fullsized":"2024-04-25T11:10:28.537528+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:24.222645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892015+0000","last_change":"2024-04-25T11:10:23.460813+0000","last_active":"2024-04-25T11:10:29.892015+0000","last_peered":"2024-04-25T11:10:29.892015+0000","last_clean":"2024-04-25T11:10:29.892015+0000","last_became_active":"2024-04-25T11:10:23.460435+0000","last_became_peered":"2024-04-25T11:10:23.460435+0000","last_unstale":"2024-04-25T11:10:29.892015+0000","last_undegraded":"2024-04-25T11:10:29.892015+0000","last_fullsized":"2024-04-25T11:10:29.892015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531902+0000","last_change":"2024-04-25T11:10:22.466351+0000","last_active":"2024-04-25T11:10:28.531902+0000","last_peered":"2024-04-25T11:10:28.531902+0000","last_clean":"2024-04-25T11:10:28.531902+0000","last_became_active":"2024-04-25T11:10:22.464110+0000","last_became_peered":"2024-04-25T11:10:22.464110+0000","last_unstale":"2024-04-25T11:10:28.531902+0000","last_undegraded":"2024-04-25T11:10:28.531902+0000","last_fullsized":"2024-04-25T11:10:28.531902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534334+0000","last_change":"2024-04-25T11:10:27.555355+0000","last_active":"2024-04-25T11:10:28.534334+0000","last_peered":"2024-04-25T11:10:28.534334+0000","last_clean":"2024-04-25T11:10:28.534334+0000","last_became_active":"2024-04-25T11:10:27.554893+0000","last_became_peered":"2024-04-25T11:10:27.554893+0000","last_unstale":"2024-04-25T11:10:28.534334+0000","last_undegraded":"2024-04-25T11:10:28.534334+0000","last_fullsized":"2024-04-25T11:10:28.534334+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:51.174327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535363+0000","last_change":"2024-04-25T11:10:23.487680+0000","last_active":"2024-04-25T11:10:28.535363+0000","last_peered":"2024-04-25T11:10:28.535363+0000","last_clean":"2024-04-25T11:10:28.535363+0000","last_became_active":"2024-04-25T11:10:23.483024+0000","last_became_peered":"2024-04-25T11:10:23.483024+0000","last_unstale":"2024-04-25T11:10:28.535363+0000","last_undegraded":"2024-04-25T11:10:28.535363+0000","last_fullsized":"2024-04-25T11:10:28.535363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:06.246101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534055+0000","last_change":"2024-04-25T11:10:22.470867+0000","last_active":"2024-04-25T11:10:28.534055+0000","last_peered":"2024-04-25T11:10:28.534055+0000","last_clean":"2024-04-25T11:10:28.534055+0000","last_became_active":"2024-04-25T11:10:22.470501+0000","last_became_peered":"2024-04-25T11:10:22.470501+0000","last_unstale":"2024-04-25T11:10:28.534055+0000","last_undegraded":"2024-04-25T11:10:28.534055+0000","last_fullsized":"2024-04-25T11:10:28.534055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538997+0000","last_change":"2024-04-25T11:10:27.567703+0000","last_active":"2024-04-25T11:10:28.538997+0000","last_peered":"2024-04-25T11:10:28.538997+0000","last_clean":"2024-04-25T11:10:28.538997+0000","last_became_active":"2024-04-25T11:10:27.567208+0000","last_became_peered":"2024-04-25T11:10:27.567208+0000","last_unstale":"2024-04-25T11:10:28.538997+0000","last_undegraded":"2024-04-25T11:10:28.538997+0000","last_fullsized":"2024-04-25T11:10:28.538997+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:24.170044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895262+0000","last_change":"2024-04-25T11:10:23.488689+0000","last_active":"2024-04-25T11:10:29.895262+0000","last_peered":"2024-04-25T11:10:29.895262+0000","last_clean":"2024-04-25T11:10:29.895262+0000","last_became_active":"2024-04-25T11:10:23.479256+0000","last_became_peered":"2024-04-25T11:10:23.479256+0000","last_unstale":"2024-04-25T11:10:29.895262+0000","last_undegraded":"2024-04-25T11:10:29.895262+0000","last_fullsized":"2024-04-25T11:10:29.895262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892311+0000","last_change":"2024-04-25T11:10:22.486332+0000","last_active":"2024-04-25T11:10:29.892311+0000","last_peered":"2024-04-25T11:10:29.892311+0000","last_clean":"2024-04-25T11:10:29.892311+0000","last_became_active":"2024-04-25T11:10:22.485970+0000","last_became_peered":"2024-04-25T11:10:22.485970+0000","last_unstale":"2024-04-25T11:10:29.892311+0000","last_undegraded":"2024-04-25T11:10:29.892311+0000","last_fullsized":"2024-04-25T11:10:29.892311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534779+0000","last_change":"2024-04-25T11:10:27.549543+0000","last_active":"2024-04-25T11:10:28.534779+0000","last_peered":"2024-04-25T11:10:28.534779+0000","last_clean":"2024-04-25T11:10:28.534779+0000","last_became_active":"2024-04-25T11:10:27.549007+0000","last_became_peered":"2024-04-25T11:10:27.549007+0000","last_unstale":"2024-04-25T11:10:28.534779+0000","last_undegraded":"2024-04-25T11:10:28.534779+0000","last_fullsized":"2024-04-25T11:10:28.534779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:01.381820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896311+0000","last_change":"2024-04-25T11:10:23.465612+0000","last_active":"2024-04-25T11:10:29.896311+0000","last_peered":"2024-04-25T11:10:29.896311+0000","last_clean":"2024-04-25T11:10:29.896311+0000","last_became_active":"2024-04-25T11:10:23.465261+0000","last_became_peered":"2024-04-25T11:10:23.465261+0000","last_unstale":"2024-04-25T11:10:29.896311+0000","last_undegraded":"2024-04-25T11:10:29.896311+0000","last_fullsized":"2024-04-25T11:10:29.896311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534173+0000","last_change":"2024-04-25T11:10:22.483679+0000","last_active":"2024-04-25T11:10:28.534173+0000","last_peered":"2024-04-25T11:10:28.534173+0000","last_clean":"2024-04-25T11:10:28.534173+0000","last_became_active":"2024-04-25T11:10:22.483243+0000","last_became_peered":"2024-04-25T11:10:22.483243+0000","last_unstale":"2024-04-25T11:10:28.534173+0000","last_undegraded":"2024-04-25T11:10:28.534173+0000","last_fullsized":"2024-04-25T11:10:28.534173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:49.860961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891407+0000","last_change":"2024-04-25T11:10:27.550030+0000","last_active":"2024-04-25T11:10:29.891407+0000","last_peered":"2024-04-25T11:10:29.891407+0000","last_clean":"2024-04-25T11:10:29.891407+0000","last_became_active":"2024-04-25T11:10:27.549537+0000","last_became_peered":"2024-04-25T11:10:27.549537+0000","last_unstale":"2024-04-25T11:10:29.891407+0000","last_undegraded":"2024-04-25T11:10:29.891407+0000","last_fullsized":"2024-04-25T11:10:29.891407+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:51.538673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534183+0000","last_change":"2024-04-25T11:10:23.470926+0000","last_active":"2024-04-25T11:10:28.534183+0000","last_peered":"2024-04-25T11:10:28.534183+0000","last_clean":"2024-04-25T11:10:28.534183+0000","last_became_active":"2024-04-25T11:10:23.470483+0000","last_became_peered":"2024-04-25T11:10:23.470483+0000","last_unstale":"2024-04-25T11:10:28.534183+0000","last_undegraded":"2024-04-25T11:10:28.534183+0000","last_fullsized":"2024-04-25T11:10:28.534183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537196+0000","last_change":"2024-04-25T11:10:22.479703+0000","last_active":"2024-04-25T11:10:28.537196+0000","last_peered":"2024-04-25T11:10:28.537196+0000","last_clean":"2024-04-25T11:10:28.537196+0000","last_became_active":"2024-04-25T11:10:22.478929+0000","last_became_peered":"2024-04-25T11:10:22.478929+0000","last_unstale":"2024-04-25T11:10:28.537196+0000","last_undegraded":"2024-04-25T11:10:28.537196+0000","last_fullsized":"2024-04-25T11:10:28.537196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:08.833540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893409+0000","last_change":"2024-04-25T11:10:27.559919+0000","last_active":"2024-04-25T11:10:29.893409+0000","last_peered":"2024-04-25T11:10:29.893409+0000","last_clean":"2024-04-25T11:10:29.893409+0000","last_became_active":"2024-04-25T11:10:27.558805+0000","last_became_peered":"2024-04-25T11:10:27.558805+0000","last_unstale":"2024-04-25T11:10:29.893409+0000","last_undegraded":"2024-04-25T11:10:29.893409+0000","last_fullsized":"2024-04-25T11:10:29.893409+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291634+0000","last_change":"2024-04-25T11:10:23.471738+0000","last_active":"2024-04-25T11:10:29.291634+0000","last_peered":"2024-04-25T11:10:29.291634+0000","last_clean":"2024-04-25T11:10:29.291634+0000","last_became_active":"2024-04-25T11:10:23.471191+0000","last_became_peered":"2024-04-25T11:10:23.471191+0000","last_unstale":"2024-04-25T11:10:29.291634+0000","last_undegraded":"2024-04-25T11:10:29.291634+0000","last_fullsized":"2024-04-25T11:10:29.291634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:47.901333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534371+0000","last_change":"2024-04-25T11:10:22.467322+0000","last_active":"2024-04-25T11:10:28.534371+0000","last_peered":"2024-04-25T11:10:28.534371+0000","last_clean":"2024-04-25T11:10:28.534371+0000","last_became_active":"2024-04-25T11:10:22.466806+0000","last_became_peered":"2024-04-25T11:10:22.466806+0000","last_unstale":"2024-04-25T11:10:28.534371+0000","last_undegraded":"2024-04-25T11:10:28.534371+0000","last_fullsized":"2024-04-25T11:10:28.534371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533236+0000","last_change":"2024-04-25T11:10:27.546558+0000","last_active":"2024-04-25T11:10:28.533236+0000","last_peered":"2024-04-25T11:10:28.533236+0000","last_clean":"2024-04-25T11:10:28.533236+0000","last_became_active":"2024-04-25T11:10:27.544215+0000","last_became_peered":"2024-04-25T11:10:27.544215+0000","last_unstale":"2024-04-25T11:10:28.533236+0000","last_undegraded":"2024-04-25T11:10:28.533236+0000","last_fullsized":"2024-04-25T11:10:28.533236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.970801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537205+0000","last_change":"2024-04-25T11:10:23.489986+0000","last_active":"2024-04-25T11:10:28.537205+0000","last_peered":"2024-04-25T11:10:28.537205+0000","last_clean":"2024-04-25T11:10:28.537205+0000","last_became_active":"2024-04-25T11:10:23.489629+0000","last_became_peered":"2024-04-25T11:10:23.489629+0000","last_unstale":"2024-04-25T11:10:28.537205+0000","last_undegraded":"2024-04-25T11:10:28.537205+0000","last_fullsized":"2024-04-25T11:10:28.537205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:14.555604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291125+0000","last_change":"2024-04-25T11:10:22.482966+0000","last_active":"2024-04-25T11:10:29.291125+0000","last_peered":"2024-04-25T11:10:29.291125+0000","last_clean":"2024-04-25T11:10:29.291125+0000","last_became_active":"2024-04-25T11:10:22.480437+0000","last_became_peered":"2024-04-25T11:10:22.480437+0000","last_unstale":"2024-04-25T11:10:29.291125+0000","last_undegraded":"2024-04-25T11:10:29.291125+0000","last_fullsized":"2024-04-25T11:10:29.291125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534554+0000","last_change":"2024-04-25T11:10:27.549617+0000","last_active":"2024-04-25T11:10:28.534554+0000","last_peered":"2024-04-25T11:10:28.534554+0000","last_clean":"2024-04-25T11:10:28.534554+0000","last_became_active":"2024-04-25T11:10:27.549203+0000","last_became_peered":"2024-04-25T11:10:27.549203+0000","last_unstale":"2024-04-25T11:10:28.534554+0000","last_undegraded":"2024-04-25T11:10:28.534554+0000","last_fullsized":"2024-04-25T11:10:28.534554+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:55.043679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531961+0000","last_change":"2024-04-25T11:10:23.472089+0000","last_active":"2024-04-25T11:10:28.531961+0000","last_peered":"2024-04-25T11:10:28.531961+0000","last_clean":"2024-04-25T11:10:28.531961+0000","last_became_active":"2024-04-25T11:10:23.468460+0000","last_became_peered":"2024-04-25T11:10:23.468460+0000","last_unstale":"2024-04-25T11:10:28.531961+0000","last_undegraded":"2024-04-25T11:10:28.531961+0000","last_fullsized":"2024-04-25T11:10:28.531961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373356+0000","last_change":"2024-04-25T11:10:22.468201+0000","last_active":"2024-04-25T11:10:29.373356+0000","last_peered":"2024-04-25T11:10:29.373356+0000","last_clean":"2024-04-25T11:10:29.373356+0000","last_became_active":"2024-04-25T11:10:22.467845+0000","last_became_peered":"2024-04-25T11:10:22.467845+0000","last_unstale":"2024-04-25T11:10:29.373356+0000","last_undegraded":"2024-04-25T11:10:29.373356+0000","last_fullsized":"2024-04-25T11:10:29.373356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:50.983943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534837+0000","last_change":"2024-04-25T11:10:27.552725+0000","last_active":"2024-04-25T11:10:28.534837+0000","last_peered":"2024-04-25T11:10:28.534837+0000","last_clean":"2024-04-25T11:10:28.534837+0000","last_became_active":"2024-04-25T11:10:27.552073+0000","last_became_peered":"2024-04-25T11:10:27.552073+0000","last_unstale":"2024-04-25T11:10:28.534837+0000","last_undegraded":"2024-04-25T11:10:28.534837+0000","last_fullsized":"2024-04-25T11:10:28.534837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.402499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893257+0000","last_change":"2024-04-25T11:10:23.492738+0000","last_active":"2024-04-25T11:10:29.893257+0000","last_peered":"2024-04-25T11:10:29.893257+0000","last_clean":"2024-04-25T11:10:29.893257+0000","last_became_active":"2024-04-25T11:10:23.492416+0000","last_became_peered":"2024-04-25T11:10:23.492416+0000","last_unstale":"2024-04-25T11:10:29.893257+0000","last_undegraded":"2024-04-25T11:10:29.893257+0000","last_fullsized":"2024-04-25T11:10:29.893257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:00.331944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536183+0000","last_change":"2024-04-25T11:10:22.478976+0000","last_active":"2024-04-25T11:10:28.536183+0000","last_peered":"2024-04-25T11:10:28.536183+0000","last_clean":"2024-04-25T11:10:28.536183+0000","last_became_active":"2024-04-25T11:10:22.478595+0000","last_became_peered":"2024-04-25T11:10:22.478595+0000","last_unstale":"2024-04-25T11:10:28.536183+0000","last_undegraded":"2024-04-25T11:10:28.536183+0000","last_fullsized":"2024-04-25T11:10:28.536183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895071+0000","last_change":"2024-04-25T11:10:27.558873+0000","last_active":"2024-04-25T11:10:29.895071+0000","last_peered":"2024-04-25T11:10:29.895071+0000","last_clean":"2024-04-25T11:10:29.895071+0000","last_became_active":"2024-04-25T11:10:27.557790+0000","last_became_peered":"2024-04-25T11:10:27.557790+0000","last_unstale":"2024-04-25T11:10:29.895071+0000","last_undegraded":"2024-04-25T11:10:29.895071+0000","last_fullsized":"2024-04-25T11:10:29.895071+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.402499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893187+0000","last_change":"2024-04-25T11:10:23.470011+0000","last_active":"2024-04-25T11:10:29.893187+0000","last_peered":"2024-04-25T11:10:29.893187+0000","last_clean":"2024-04-25T11:10:29.893187+0000","last_became_active":"2024-04-25T11:10:23.469655+0000","last_became_peered":"2024-04-25T11:10:23.469655+0000","last_unstale":"2024-04-25T11:10:29.893187+0000","last_undegraded":"2024-04-25T11:10:29.893187+0000","last_fullsized":"2024-04-25T11:10:29.893187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291689+0000","last_change":"2024-04-25T11:10:22.480507+0000","last_active":"2024-04-25T11:10:29.291689+0000","last_peered":"2024-04-25T11:10:29.291689+0000","last_clean":"2024-04-25T11:10:29.291689+0000","last_became_active":"2024-04-25T11:10:22.479568+0000","last_became_peered":"2024-04-25T11:10:22.479568+0000","last_unstale":"2024-04-25T11:10:29.291689+0000","last_undegraded":"2024-04-25T11:10:29.291689+0000","last_fullsized":"2024-04-25T11:10:29.291689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":118,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536490+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:10:28.536490+0000","last_peered":"2024-04-25T11:10:28.536490+0000","last_clean":"2024-04-25T11:10:28.536490+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:10:28.536490+0000","last_undegraded":"2024-04-25T11:10:28.536490+0000","last_fullsized":"2024-04-25T11:10:28.536490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891956+0000","last_change":"2024-04-25T11:10:27.551611+0000","last_active":"2024-04-25T11:10:29.891956+0000","last_peered":"2024-04-25T11:10:29.891956+0000","last_clean":"2024-04-25T11:10:29.891956+0000","last_became_active":"2024-04-25T11:10:27.551117+0000","last_became_peered":"2024-04-25T11:10:27.551117+0000","last_unstale":"2024-04-25T11:10:29.891956+0000","last_undegraded":"2024-04-25T11:10:29.891956+0000","last_fullsized":"2024-04-25T11:10:29.891956+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536555+0000","last_change":"2024-04-25T11:10:23.469975+0000","last_active":"2024-04-25T11:10:28.536555+0000","last_peered":"2024-04-25T11:10:28.536555+0000","last_clean":"2024-04-25T11:10:28.536555+0000","last_became_active":"2024-04-25T11:10:23.469366+0000","last_became_peered":"2024-04-25T11:10:23.469366+0000","last_unstale":"2024-04-25T11:10:28.536555+0000","last_undegraded":"2024-04-25T11:10:28.536555+0000","last_fullsized":"2024-04-25T11:10:28.536555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:47.901333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373786+0000","last_change":"2024-04-25T11:10:22.480954+0000","last_active":"2024-04-25T11:10:29.373786+0000","last_peered":"2024-04-25T11:10:29.373786+0000","last_clean":"2024-04-25T11:10:29.373786+0000","last_became_active":"2024-04-25T11:10:22.480489+0000","last_became_peered":"2024-04-25T11:10:22.480489+0000","last_unstale":"2024-04-25T11:10:29.373786+0000","last_undegraded":"2024-04-25T11:10:29.373786+0000","last_fullsized":"2024-04-25T11:10:29.373786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:11.295052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891284+0000","last_change":"2024-04-25T11:10:27.548113+0000","last_active":"2024-04-25T11:10:29.891284+0000","last_peered":"2024-04-25T11:10:29.891284+0000","last_clean":"2024-04-25T11:10:29.891284+0000","last_became_active":"2024-04-25T11:10:27.547579+0000","last_became_peered":"2024-04-25T11:10:27.547579+0000","last_unstale":"2024-04-25T11:10:29.891284+0000","last_undegraded":"2024-04-25T11:10:29.891284+0000","last_fullsized":"2024-04-25T11:10:29.891284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:45.815273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291802+0000","last_change":"2024-04-25T11:10:23.478738+0000","last_active":"2024-04-25T11:10:29.291802+0000","last_peered":"2024-04-25T11:10:29.291802+0000","last_clean":"2024-04-25T11:10:29.291802+0000","last_became_active":"2024-04-25T11:10:23.468486+0000","last_became_peered":"2024-04-25T11:10:23.468486+0000","last_unstale":"2024-04-25T11:10:29.291802+0000","last_undegraded":"2024-04-25T11:10:29.291802+0000","last_fullsized":"2024-04-25T11:10:29.291802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:47.104859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533727+0000","last_change":"2024-04-25T11:10:22.473530+0000","last_active":"2024-04-25T11:10:28.533727+0000","last_peered":"2024-04-25T11:10:28.533727+0000","last_clean":"2024-04-25T11:10:28.533727+0000","last_became_active":"2024-04-25T11:10:22.473177+0000","last_became_peered":"2024-04-25T11:10:22.473177+0000","last_unstale":"2024-04-25T11:10:28.533727+0000","last_undegraded":"2024-04-25T11:10:28.533727+0000","last_fullsized":"2024-04-25T11:10:28.533727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894250+0000","last_change":"2024-04-25T11:10:27.559345+0000","last_active":"2024-04-25T11:10:29.894250+0000","last_peered":"2024-04-25T11:10:29.894250+0000","last_clean":"2024-04-25T11:10:29.894250+0000","last_became_active":"2024-04-25T11:10:27.558090+0000","last_became_peered":"2024-04-25T11:10:27.558090+0000","last_unstale":"2024-04-25T11:10:29.894250+0000","last_undegraded":"2024-04-25T11:10:29.894250+0000","last_fullsized":"2024-04-25T11:10:29.894250+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:51.174327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538883+0000","last_change":"2024-04-25T11:10:23.477613+0000","last_active":"2024-04-25T11:10:28.538883+0000","last_peered":"2024-04-25T11:10:28.538883+0000","last_clean":"2024-04-25T11:10:28.538883+0000","last_became_active":"2024-04-25T11:10:23.477226+0000","last_became_peered":"2024-04-25T11:10:23.477226+0000","last_unstale":"2024-04-25T11:10:28.538883+0000","last_undegraded":"2024-04-25T11:10:28.538883+0000","last_fullsized":"2024-04-25T11:10:28.538883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:07.701501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.374436+0000","last_change":"2024-04-25T11:10:22.468389+0000","last_active":"2024-04-25T11:10:29.374436+0000","last_peered":"2024-04-25T11:10:29.374436+0000","last_clean":"2024-04-25T11:10:29.374436+0000","last_became_active":"2024-04-25T11:10:22.467926+0000","last_became_peered":"2024-04-25T11:10:22.467926+0000","last_unstale":"2024-04-25T11:10:29.374436+0000","last_undegraded":"2024-04-25T11:10:29.374436+0000","last_fullsized":"2024-04-25T11:10:29.374436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.448477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290385+0000","last_change":"2024-04-25T11:10:27.547813+0000","last_active":"2024-04-25T11:10:29.290385+0000","last_peered":"2024-04-25T11:10:29.290385+0000","last_clean":"2024-04-25T11:10:29.290385+0000","last_became_active":"2024-04-25T11:10:27.547300+0000","last_became_peered":"2024-04-25T11:10:27.547300+0000","last_unstale":"2024-04-25T11:10:29.290385+0000","last_undegraded":"2024-04-25T11:10:29.290385+0000","last_fullsized":"2024-04-25T11:10:29.290385+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:51.538673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537521+0000","last_change":"2024-04-25T11:10:23.499736+0000","last_active":"2024-04-25T11:10:28.537521+0000","last_peered":"2024-04-25T11:10:28.537521+0000","last_clean":"2024-04-25T11:10:28.537521+0000","last_became_active":"2024-04-25T11:10:23.491282+0000","last_became_peered":"2024-04-25T11:10:23.491282+0000","last_unstale":"2024-04-25T11:10:28.537521+0000","last_undegraded":"2024-04-25T11:10:28.537521+0000","last_fullsized":"2024-04-25T11:10:28.537521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:15.460249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.539367+0000","last_change":"2024-04-25T11:10:22.477761+0000","last_active":"2024-04-25T11:10:28.539367+0000","last_peered":"2024-04-25T11:10:28.539367+0000","last_clean":"2024-04-25T11:10:28.539367+0000","last_became_active":"2024-04-25T11:10:22.477390+0000","last_became_peered":"2024-04-25T11:10:22.477390+0000","last_unstale":"2024-04-25T11:10:28.539367+0000","last_undegraded":"2024-04-25T11:10:28.539367+0000","last_fullsized":"2024-04-25T11:10:28.539367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:48.408404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535808+0000","last_change":"2024-04-25T11:10:27.559907+0000","last_active":"2024-04-25T11:10:28.535808+0000","last_peered":"2024-04-25T11:10:28.535808+0000","last_clean":"2024-04-25T11:10:28.535808+0000","last_became_active":"2024-04-25T11:10:27.559437+0000","last_became_peered":"2024-04-25T11:10:27.559437+0000","last_unstale":"2024-04-25T11:10:28.535808+0000","last_undegraded":"2024-04-25T11:10:28.535808+0000","last_fullsized":"2024-04-25T11:10:28.535808+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.067687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535432+0000","last_change":"2024-04-25T11:10:23.479907+0000","last_active":"2024-04-25T11:10:28.535432+0000","last_peered":"2024-04-25T11:10:28.535432+0000","last_clean":"2024-04-25T11:10:28.535432+0000","last_became_active":"2024-04-25T11:10:23.479578+0000","last_became_peered":"2024-04-25T11:10:23.479578+0000","last_unstale":"2024-04-25T11:10:28.535432+0000","last_undegraded":"2024-04-25T11:10:28.535432+0000","last_fullsized":"2024-04-25T11:10:28.535432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373126+0000","last_change":"2024-04-25T11:10:22.467360+0000","last_active":"2024-04-25T11:10:29.373126+0000","last_peered":"2024-04-25T11:10:29.373126+0000","last_clean":"2024-04-25T11:10:29.373126+0000","last_became_active":"2024-04-25T11:10:22.467002+0000","last_became_peered":"2024-04-25T11:10:22.467002+0000","last_unstale":"2024-04-25T11:10:29.373126+0000","last_undegraded":"2024-04-25T11:10:29.373126+0000","last_fullsized":"2024-04-25T11:10:29.373126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:17.490319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894874+0000","last_change":"2024-04-25T11:10:27.566981+0000","last_active":"2024-04-25T11:10:29.894874+0000","last_peered":"2024-04-25T11:10:29.894874+0000","last_clean":"2024-04-25T11:10:29.894874+0000","last_became_active":"2024-04-25T11:10:27.566434+0000","last_became_peered":"2024-04-25T11:10:27.566434+0000","last_unstale":"2024-04-25T11:10:29.894874+0000","last_undegraded":"2024-04-25T11:10:29.894874+0000","last_fullsized":"2024-04-25T11:10:29.894874+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:55.043679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891408+0000","last_change":"2024-04-25T11:10:23.489031+0000","last_active":"2024-04-25T11:10:29.891408+0000","last_peered":"2024-04-25T11:10:29.891408+0000","last_clean":"2024-04-25T11:10:29.891408+0000","last_became_active":"2024-04-25T11:10:23.487804+0000","last_became_peered":"2024-04-25T11:10:23.487804+0000","last_unstale":"2024-04-25T11:10:29.891408+0000","last_undegraded":"2024-04-25T11:10:29.891408+0000","last_fullsized":"2024-04-25T11:10:29.891408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891229+0000","last_change":"2024-04-25T11:10:22.489707+0000","last_active":"2024-04-25T11:10:29.891229+0000","last_peered":"2024-04-25T11:10:29.891229+0000","last_clean":"2024-04-25T11:10:29.891229+0000","last_became_active":"2024-04-25T11:10:22.489324+0000","last_became_peered":"2024-04-25T11:10:22.489324+0000","last_unstale":"2024-04-25T11:10:29.891229+0000","last_undegraded":"2024-04-25T11:10:29.891229+0000","last_fullsized":"2024-04-25T11:10:29.891229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896355+0000","last_change":"2024-04-25T11:10:27.556359+0000","last_active":"2024-04-25T11:10:29.896355+0000","last_peered":"2024-04-25T11:10:29.896355+0000","last_clean":"2024-04-25T11:10:29.896355+0000","last_became_active":"2024-04-25T11:10:27.555528+0000","last_became_peered":"2024-04-25T11:10:27.555528+0000","last_unstale":"2024-04-25T11:10:29.896355+0000","last_undegraded":"2024-04-25T11:10:29.896355+0000","last_fullsized":"2024-04-25T11:10:29.896355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:35.346179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535152+0000","last_change":"2024-04-25T11:10:23.478210+0000","last_active":"2024-04-25T11:10:28.535152+0000","last_peered":"2024-04-25T11:10:28.535152+0000","last_clean":"2024-04-25T11:10:28.535152+0000","last_became_active":"2024-04-25T11:10:23.477839+0000","last_became_peered":"2024-04-25T11:10:23.477839+0000","last_unstale":"2024-04-25T11:10:28.535152+0000","last_undegraded":"2024-04-25T11:10:28.535152+0000","last_fullsized":"2024-04-25T11:10:28.535152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:41.745805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290930+0000","last_change":"2024-04-25T11:10:22.487737+0000","last_active":"2024-04-25T11:10:29.290930+0000","last_peered":"2024-04-25T11:10:29.290930+0000","last_clean":"2024-04-25T11:10:29.290930+0000","last_became_active":"2024-04-25T11:10:22.485337+0000","last_became_peered":"2024-04-25T11:10:22.485337+0000","last_unstale":"2024-04-25T11:10:29.290930+0000","last_undegraded":"2024-04-25T11:10:29.290930+0000","last_fullsized":"2024-04-25T11:10:29.290930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894290+0000","last_change":"2024-04-25T11:10:27.562988+0000","last_active":"2024-04-25T11:10:29.894290+0000","last_peered":"2024-04-25T11:10:29.894290+0000","last_clean":"2024-04-25T11:10:29.894290+0000","last_became_active":"2024-04-25T11:10:27.562511+0000","last_became_peered":"2024-04-25T11:10:27.562511+0000","last_unstale":"2024-04-25T11:10:29.894290+0000","last_undegraded":"2024-04-25T11:10:29.894290+0000","last_fullsized":"2024-04-25T11:10:29.894290+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:32.363124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893441+0000","last_change":"2024-04-25T11:10:23.480645+0000","last_active":"2024-04-25T11:10:29.893441+0000","last_peered":"2024-04-25T11:10:29.893441+0000","last_clean":"2024-04-25T11:10:29.893441+0000","last_became_active":"2024-04-25T11:10:23.480244+0000","last_became_peered":"2024-04-25T11:10:23.480244+0000","last_unstale":"2024-04-25T11:10:29.893441+0000","last_undegraded":"2024-04-25T11:10:29.893441+0000","last_fullsized":"2024-04-25T11:10:29.893441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:46.490227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894602+0000","last_change":"2024-04-25T11:10:22.479455+0000","last_active":"2024-04-25T11:10:29.894602+0000","last_peered":"2024-04-25T11:10:29.894602+0000","last_clean":"2024-04-25T11:10:29.894602+0000","last_became_active":"2024-04-25T11:10:22.479024+0000","last_became_peered":"2024-04-25T11:10:22.479024+0000","last_unstale":"2024-04-25T11:10:29.894602+0000","last_undegraded":"2024-04-25T11:10:29.894602+0000","last_fullsized":"2024-04-25T11:10:29.894602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:56.918083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291635+0000","last_change":"2024-04-25T11:10:27.549545+0000","last_active":"2024-04-25T11:10:29.291635+0000","last_peered":"2024-04-25T11:10:29.291635+0000","last_clean":"2024-04-25T11:10:29.291635+0000","last_became_active":"2024-04-25T11:10:27.549085+0000","last_became_peered":"2024-04-25T11:10:27.549085+0000","last_unstale":"2024-04-25T11:10:29.291635+0000","last_undegraded":"2024-04-25T11:10:29.291635+0000","last_fullsized":"2024-04-25T11:10:29.291635+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.906264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536502+0000","last_change":"2024-04-25T11:10:23.472355+0000","last_active":"2024-04-25T11:10:28.536502+0000","last_peered":"2024-04-25T11:10:28.536502+0000","last_clean":"2024-04-25T11:10:28.536502+0000","last_became_active":"2024-04-25T11:10:23.470905+0000","last_became_peered":"2024-04-25T11:10:23.470905+0000","last_unstale":"2024-04-25T11:10:28.536502+0000","last_undegraded":"2024-04-25T11:10:28.536502+0000","last_fullsized":"2024-04-25T11:10:28.536502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:20.100576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:44.462567+0000","last_change":"2024-04-25T11:10:22.481299+0000","last_active":"2024-04-25T11:10:44.462567+0000","last_peered":"2024-04-25T11:10:44.462567+0000","last_clean":"2024-04-25T11:10:44.462567+0000","last_became_active":"2024-04-25T11:10:22.480741+0000","last_became_peered":"2024-04-25T11:10:22.480741+0000","last_unstale":"2024-04-25T11:10:44.462567+0000","last_undegraded":"2024-04-25T11:10:44.462567+0000","last_fullsized":"2024-04-25T11:10:44.462567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533006+0000","last_change":"2024-04-25T11:10:27.551729+0000","last_active":"2024-04-25T11:10:28.533006+0000","last_peered":"2024-04-25T11:10:28.533006+0000","last_clean":"2024-04-25T11:10:28.533006+0000","last_became_active":"2024-04-25T11:10:27.548417+0000","last_became_peered":"2024-04-25T11:10:27.548417+0000","last_unstale":"2024-04-25T11:10:28.533006+0000","last_undegraded":"2024-04-25T11:10:28.533006+0000","last_fullsized":"2024-04-25T11:10:28.533006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538108+0000","last_change":"2024-04-25T11:10:23.482351+0000","last_active":"2024-04-25T11:10:28.538108+0000","last_peered":"2024-04-25T11:10:28.538108+0000","last_clean":"2024-04-25T11:10:28.538108+0000","last_became_active":"2024-04-25T11:10:23.481665+0000","last_became_peered":"2024-04-25T11:10:23.481665+0000","last_unstale":"2024-04-25T11:10:28.538108+0000","last_undegraded":"2024-04-25T11:10:28.538108+0000","last_fullsized":"2024-04-25T11:10:28.538108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:50.877149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533327+0000","last_change":"2024-04-25T11:10:22.484130+0000","last_active":"2024-04-25T11:10:28.533327+0000","last_peered":"2024-04-25T11:10:28.533327+0000","last_clean":"2024-04-25T11:10:28.533327+0000","last_became_active":"2024-04-25T11:10:22.483811+0000","last_became_peered":"2024-04-25T11:10:22.483811+0000","last_unstale":"2024-04-25T11:10:28.533327+0000","last_undegraded":"2024-04-25T11:10:28.533327+0000","last_fullsized":"2024-04-25T11:10:28.533327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535359+0000","last_change":"2024-04-25T11:10:27.564379+0000","last_active":"2024-04-25T11:10:28.535359+0000","last_peered":"2024-04-25T11:10:28.535359+0000","last_clean":"2024-04-25T11:10:28.535359+0000","last_became_active":"2024-04-25T11:10:27.563892+0000","last_became_peered":"2024-04-25T11:10:27.563892+0000","last_unstale":"2024-04-25T11:10:28.535359+0000","last_undegraded":"2024-04-25T11:10:28.535359+0000","last_fullsized":"2024-04-25T11:10:28.535359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:06.999062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895182+0000","last_change":"2024-04-25T11:10:23.487092+0000","last_active":"2024-04-25T11:10:29.895182+0000","last_peered":"2024-04-25T11:10:29.895182+0000","last_clean":"2024-04-25T11:10:29.895182+0000","last_became_active":"2024-04-25T11:10:23.486583+0000","last_became_peered":"2024-04-25T11:10:23.486583+0000","last_unstale":"2024-04-25T11:10:29.895182+0000","last_undegraded":"2024-04-25T11:10:29.895182+0000","last_fullsized":"2024-04-25T11:10:29.895182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291509+0000","last_change":"2024-04-25T11:10:22.484333+0000","last_active":"2024-04-25T11:10:29.291509+0000","last_peered":"2024-04-25T11:10:29.291509+0000","last_clean":"2024-04-25T11:10:29.291509+0000","last_became_active":"2024-04-25T11:10:22.483865+0000","last_became_peered":"2024-04-25T11:10:22.483865+0000","last_unstale":"2024-04-25T11:10:29.291509+0000","last_undegraded":"2024-04-25T11:10:29.291509+0000","last_fullsized":"2024-04-25T11:10:29.291509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891783+0000","last_change":"2024-04-25T11:10:27.551066+0000","last_active":"2024-04-25T11:10:29.891783+0000","last_peered":"2024-04-25T11:10:29.891783+0000","last_clean":"2024-04-25T11:10:29.891783+0000","last_became_active":"2024-04-25T11:10:27.550625+0000","last_became_peered":"2024-04-25T11:10:27.550625+0000","last_unstale":"2024-04-25T11:10:29.891783+0000","last_undegraded":"2024-04-25T11:10:29.891783+0000","last_fullsized":"2024-04-25T11:10:29.891783+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.906264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291447+0000","last_change":"2024-04-25T11:10:23.485935+0000","last_active":"2024-04-25T11:10:29.291447+0000","last_peered":"2024-04-25T11:10:29.291447+0000","last_clean":"2024-04-25T11:10:29.291447+0000","last_became_active":"2024-04-25T11:10:23.485367+0000","last_became_peered":"2024-04-25T11:10:23.485367+0000","last_unstale":"2024-04-25T11:10:29.291447+0000","last_undegraded":"2024-04-25T11:10:29.291447+0000","last_fullsized":"2024-04-25T11:10:29.291447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:50.974212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892031+0000","last_change":"2024-04-25T11:10:22.482686+0000","last_active":"2024-04-25T11:10:29.892031+0000","last_peered":"2024-04-25T11:10:29.892031+0000","last_clean":"2024-04-25T11:10:29.892031+0000","last_became_active":"2024-04-25T11:10:22.482307+0000","last_became_peered":"2024-04-25T11:10:22.482307+0000","last_unstale":"2024-04-25T11:10:29.892031+0000","last_undegraded":"2024-04-25T11:10:29.892031+0000","last_fullsized":"2024-04-25T11:10:29.892031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892597+0000","last_change":"2024-04-25T11:10:27.563942+0000","last_active":"2024-04-25T11:10:29.892597+0000","last_peered":"2024-04-25T11:10:29.892597+0000","last_clean":"2024-04-25T11:10:29.892597+0000","last_became_active":"2024-04-25T11:10:27.561604+0000","last_became_peered":"2024-04-25T11:10:27.561604+0000","last_unstale":"2024-04-25T11:10:29.892597+0000","last_undegraded":"2024-04-25T11:10:29.892597+0000","last_fullsized":"2024-04-25T11:10:29.892597+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:59.323947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893904+0000","last_change":"2024-04-25T11:10:27.559342+0000","last_active":"2024-04-25T11:10:29.893904+0000","last_peered":"2024-04-25T11:10:29.893904+0000","last_clean":"2024-04-25T11:10:29.893904+0000","last_became_active":"2024-04-25T11:10:27.558417+0000","last_became_peered":"2024-04-25T11:10:27.558417+0000","last_unstale":"2024-04-25T11:10:29.893904+0000","last_undegraded":"2024-04-25T11:10:29.893904+0000","last_fullsized":"2024-04-25T11:10:29.893904+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.970801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531482+0000","last_change":"2024-04-25T11:10:23.472501+0000","last_active":"2024-04-25T11:10:28.531482+0000","last_peered":"2024-04-25T11:10:28.531482+0000","last_clean":"2024-04-25T11:10:28.531482+0000","last_became_active":"2024-04-25T11:10:23.472103+0000","last_became_peered":"2024-04-25T11:10:23.472103+0000","last_unstale":"2024-04-25T11:10:28.531482+0000","last_undegraded":"2024-04-25T11:10:28.531482+0000","last_fullsized":"2024-04-25T11:10:28.531482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290643+0000","last_change":"2024-04-25T11:10:22.487396+0000","last_active":"2024-04-25T11:10:29.290643+0000","last_peered":"2024-04-25T11:10:29.290643+0000","last_clean":"2024-04-25T11:10:29.290643+0000","last_became_active":"2024-04-25T11:10:22.484146+0000","last_became_peered":"2024-04-25T11:10:22.484146+0000","last_unstale":"2024-04-25T11:10:29.290643+0000","last_undegraded":"2024-04-25T11:10:29.290643+0000","last_fullsized":"2024-04-25T11:10:29.290643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891822+0000","last_change":"2024-04-25T11:10:27.545602+0000","last_active":"2024-04-25T11:10:29.891822+0000","last_peered":"2024-04-25T11:10:29.891822+0000","last_clean":"2024-04-25T11:10:29.891822+0000","last_became_active":"2024-04-25T11:10:27.545089+0000","last_became_peered":"2024-04-25T11:10:27.545089+0000","last_unstale":"2024-04-25T11:10:29.891822+0000","last_undegraded":"2024-04-25T11:10:29.891822+0000","last_fullsized":"2024-04-25T11:10:29.891822+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:24.170044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891897+0000","last_change":"2024-04-25T11:10:23.480332+0000","last_active":"2024-04-25T11:10:29.891897+0000","last_peered":"2024-04-25T11:10:29.891897+0000","last_clean":"2024-04-25T11:10:29.891897+0000","last_became_active":"2024-04-25T11:10:23.479866+0000","last_became_peered":"2024-04-25T11:10:23.479866+0000","last_unstale":"2024-04-25T11:10:29.891897+0000","last_undegraded":"2024-04-25T11:10:29.891897+0000","last_fullsized":"2024-04-25T11:10:29.891897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893193+0000","last_change":"2024-04-25T11:10:22.476980+0000","last_active":"2024-04-25T11:10:29.893193+0000","last_peered":"2024-04-25T11:10:29.893193+0000","last_clean":"2024-04-25T11:10:29.893193+0000","last_became_active":"2024-04-25T11:10:22.476483+0000","last_became_peered":"2024-04-25T11:10:22.476483+0000","last_unstale":"2024-04-25T11:10:29.893193+0000","last_undegraded":"2024-04-25T11:10:29.893193+0000","last_fullsized":"2024-04-25T11:10:29.893193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:56.918083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891422+0000","last_change":"2024-04-25T11:10:22.485058+0000","last_active":"2024-04-25T11:10:29.891422+0000","last_peered":"2024-04-25T11:10:29.891422+0000","last_clean":"2024-04-25T11:10:29.891422+0000","last_became_active":"2024-04-25T11:10:22.484647+0000","last_became_peered":"2024-04-25T11:10:22.484647+0000","last_unstale":"2024-04-25T11:10:29.891422+0000","last_undegraded":"2024-04-25T11:10:29.891422+0000","last_fullsized":"2024-04-25T11:10:29.891422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:42.000742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894489+0000","last_change":"2024-04-25T11:10:23.494632+0000","last_active":"2024-04-25T11:10:29.894489+0000","last_peered":"2024-04-25T11:10:29.894489+0000","last_clean":"2024-04-25T11:10:29.894489+0000","last_became_active":"2024-04-25T11:10:23.494214+0000","last_became_peered":"2024-04-25T11:10:23.494214+0000","last_unstale":"2024-04-25T11:10:29.894489+0000","last_undegraded":"2024-04-25T11:10:29.894489+0000","last_fullsized":"2024-04-25T11:10:29.894489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373551+0000","last_change":"2024-04-25T11:10:27.546702+0000","last_active":"2024-04-25T11:10:29.373551+0000","last_peered":"2024-04-25T11:10:29.373551+0000","last_clean":"2024-04-25T11:10:29.373551+0000","last_became_active":"2024-04-25T11:10:27.546093+0000","last_became_peered":"2024-04-25T11:10:27.546093+0000","last_unstale":"2024-04-25T11:10:29.373551+0000","last_undegraded":"2024-04-25T11:10:29.373551+0000","last_fullsized":"2024-04-25T11:10:29.373551+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:55.973871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892460+0000","last_change":"2024-04-25T11:10:22.470279+0000","last_active":"2024-04-25T11:10:29.892460+0000","last_peered":"2024-04-25T11:10:29.892460+0000","last_clean":"2024-04-25T11:10:29.892460+0000","last_became_active":"2024-04-25T11:10:22.469133+0000","last_became_peered":"2024-04-25T11:10:22.469133+0000","last_unstale":"2024-04-25T11:10:29.892460+0000","last_undegraded":"2024-04-25T11:10:29.892460+0000","last_fullsized":"2024-04-25T11:10:29.892460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.835819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536180+0000","last_change":"2024-04-25T11:10:23.482007+0000","last_active":"2024-04-25T11:10:28.536180+0000","last_peered":"2024-04-25T11:10:28.536180+0000","last_clean":"2024-04-25T11:10:28.536180+0000","last_became_active":"2024-04-25T11:10:23.475539+0000","last_became_peered":"2024-04-25T11:10:23.475539+0000","last_unstale":"2024-04-25T11:10:28.536180+0000","last_undegraded":"2024-04-25T11:10:28.536180+0000","last_fullsized":"2024-04-25T11:10:28.536180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:56.556443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895038+0000","last_change":"2024-04-25T11:10:27.555814+0000","last_active":"2024-04-25T11:10:29.895038+0000","last_peered":"2024-04-25T11:10:29.895038+0000","last_clean":"2024-04-25T11:10:29.895038+0000","last_became_active":"2024-04-25T11:10:27.554994+0000","last_became_peered":"2024-04-25T11:10:27.554994+0000","last_unstale":"2024-04-25T11:10:29.895038+0000","last_undegraded":"2024-04-25T11:10:29.895038+0000","last_fullsized":"2024-04-25T11:10:29.895038+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:11.895073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.890634+0000","last_change":"2024-04-25T11:10:23.487735+0000","last_active":"2024-04-25T11:10:29.890634+0000","last_peered":"2024-04-25T11:10:29.890634+0000","last_clean":"2024-04-25T11:10:29.890634+0000","last_became_active":"2024-04-25T11:10:23.487354+0000","last_became_peered":"2024-04-25T11:10:23.487354+0000","last_unstale":"2024-04-25T11:10:29.890634+0000","last_undegraded":"2024-04-25T11:10:29.890634+0000","last_fullsized":"2024-04-25T11:10:29.890634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373170+0000","last_change":"2024-04-25T11:10:23.476515+0000","last_active":"2024-04-25T11:10:29.373170+0000","last_peered":"2024-04-25T11:10:29.373170+0000","last_clean":"2024-04-25T11:10:29.373170+0000","last_became_active":"2024-04-25T11:10:23.476183+0000","last_became_peered":"2024-04-25T11:10:23.476183+0000","last_unstale":"2024-04-25T11:10:29.373170+0000","last_undegraded":"2024-04-25T11:10:29.373170+0000","last_fullsized":"2024-04-25T11:10:29.373170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:57.829487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290735+0000","last_change":"2024-04-25T11:10:23.477846+0000","last_active":"2024-04-25T11:10:29.290735+0000","last_peered":"2024-04-25T11:10:29.290735+0000","last_clean":"2024-04-25T11:10:29.290735+0000","last_became_active":"2024-04-25T11:10:23.476420+0000","last_became_peered":"2024-04-25T11:10:23.476420+0000","last_unstale":"2024-04-25T11:10:29.290735+0000","last_undegraded":"2024-04-25T11:10:29.290735+0000","last_fullsized":"2024-04-25T11:10:29.290735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894372+0000","last_change":"2024-04-25T11:10:23.485446+0000","last_active":"2024-04-25T11:10:29.894372+0000","last_peered":"2024-04-25T11:10:29.894372+0000","last_clean":"2024-04-25T11:10:29.894372+0000","last_became_active":"2024-04-25T11:10:23.485095+0000","last_became_peered":"2024-04-25T11:10:23.485095+0000","last_unstale":"2024-04-25T11:10:29.894372+0000","last_undegraded":"2024-04-25T11:10:29.894372+0000","last_fullsized":"2024-04-25T11:10:29.894372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895256+0000","last_change":"2024-04-25T11:10:23.490644+0000","last_active":"2024-04-25T11:10:29.895256+0000","last_peered":"2024-04-25T11:10:29.895256+0000","last_clean":"2024-04-25T11:10:29.895256+0000","last_became_active":"2024-04-25T11:10:23.487911+0000","last_became_peered":"2024-04-25T11:10:23.487911+0000","last_unstale":"2024-04-25T11:10:29.895256+0000","last_undegraded":"2024-04-25T11:10:29.895256+0000","last_fullsized":"2024-04-25T11:10:29.895256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:48.595548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.532685+0000","last_change":"2024-04-25T11:10:23.484155+0000","last_active":"2024-04-25T11:10:28.532685+0000","last_peered":"2024-04-25T11:10:28.532685+0000","last_clean":"2024-04-25T11:10:28.532685+0000","last_became_active":"2024-04-25T11:10:23.483906+0000","last_became_peered":"2024-04-25T11:10:23.483906+0000","last_unstale":"2024-04-25T11:10:28.532685+0000","last_undegraded":"2024-04-25T11:10:28.532685+0000","last_fullsized":"2024-04-25T11:10:28.532685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892230+0000","last_change":"2024-04-25T11:10:23.479315+0000","last_active":"2024-04-25T11:10:29.892230+0000","last_peered":"2024-04-25T11:10:29.892230+0000","last_clean":"2024-04-25T11:10:29.892230+0000","last_became_active":"2024-04-25T11:10:23.478986+0000","last_became_peered":"2024-04-25T11:10:23.478986+0000","last_unstale":"2024-04-25T11:10:29.892230+0000","last_undegraded":"2024-04-25T11:10:29.892230+0000","last_fullsized":"2024-04-25T11:10:29.892230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896160+0000","last_change":"2024-04-25T11:10:23.475790+0000","last_active":"2024-04-25T11:10:29.896160+0000","last_peered":"2024-04-25T11:10:29.896160+0000","last_clean":"2024-04-25T11:10:29.896160+0000","last_became_active":"2024-04-25T11:10:23.475049+0000","last_became_peered":"2024-04-25T11:10:23.475049+0000","last_unstale":"2024-04-25T11:10:29.896160+0000","last_undegraded":"2024-04-25T11:10:29.896160+0000","last_fullsized":"2024-04-25T11:10:29.896160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290626+0000","last_change":"2024-04-25T11:10:23.489591+0000","last_active":"2024-04-25T11:10:29.290626+0000","last_peered":"2024-04-25T11:10:29.290626+0000","last_clean":"2024-04-25T11:10:29.290626+0000","last_became_active":"2024-04-25T11:10:23.488935+0000","last_became_peered":"2024-04-25T11:10:23.488935+0000","last_unstale":"2024-04-25T11:10:29.290626+0000","last_undegraded":"2024-04-25T11:10:29.290626+0000","last_fullsized":"2024-04-25T11:10:29.290626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892914+0000","last_change":"2024-04-25T11:10:23.487850+0000","last_active":"2024-04-25T11:10:29.892914+0000","last_peered":"2024-04-25T11:10:29.892914+0000","last_clean":"2024-04-25T11:10:29.892914+0000","last_became_active":"2024-04-25T11:10:23.487234+0000","last_became_peered":"2024-04-25T11:10:23.487234+0000","last_unstale":"2024-04-25T11:10:29.892914+0000","last_undegraded":"2024-04-25T11:10:29.892914+0000","last_fullsized":"2024-04-25T11:10:29.892914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290514+0000","last_change":"2024-04-25T11:10:23.473547+0000","last_active":"2024-04-25T11:10:29.290514+0000","last_peered":"2024-04-25T11:10:29.290514+0000","last_clean":"2024-04-25T11:10:29.290514+0000","last_became_active":"2024-04-25T11:10:23.467594+0000","last_became_peered":"2024-04-25T11:10:23.467594+0000","last_unstale":"2024-04-25T11:10:29.290514+0000","last_undegraded":"2024-04-25T11:10:29.290514+0000","last_fullsized":"2024-04-25T11:10:29.290514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:48.595548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893781+0000","last_change":"2024-04-25T11:10:23.480740+0000","last_active":"2024-04-25T11:10:29.893781+0000","last_peered":"2024-04-25T11:10:29.893781+0000","last_clean":"2024-04-25T11:10:29.893781+0000","last_became_active":"2024-04-25T11:10:23.480405+0000","last_became_peered":"2024-04-25T11:10:23.480405+0000","last_unstale":"2024-04-25T11:10:29.893781+0000","last_undegraded":"2024-04-25T11:10:29.893781+0000","last_fullsized":"2024-04-25T11:10:29.893781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891342+0000","last_change":"2024-04-25T11:10:23.497261+0000","last_active":"2024-04-25T11:10:29.891342+0000","last_peered":"2024-04-25T11:10:29.891342+0000","last_clean":"2024-04-25T11:10:29.891342+0000","last_became_active":"2024-04-25T11:10:23.493494+0000","last_became_peered":"2024-04-25T11:10:23.493494+0000","last_unstale":"2024-04-25T11:10:29.891342+0000","last_undegraded":"2024-04-25T11:10:29.891342+0000","last_fullsized":"2024-04-25T11:10:29.891342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537112+0000","last_change":"2024-04-25T11:10:23.469676+0000","last_active":"2024-04-25T11:10:28.537112+0000","last_peered":"2024-04-25T11:10:28.537112+0000","last_clean":"2024-04-25T11:10:28.537112+0000","last_became_active":"2024-04-25T11:10:23.468887+0000","last_became_peered":"2024-04-25T11:10:23.468887+0000","last_unstale":"2024-04-25T11:10:28.537112+0000","last_undegraded":"2024-04-25T11:10:28.537112+0000","last_fullsized":"2024-04-25T11:10:28.537112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:47.104859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.374373+0000","last_change":"2024-04-25T11:10:23.491992+0000","last_active":"2024-04-25T11:10:29.374373+0000","last_peered":"2024-04-25T11:10:29.374373+0000","last_clean":"2024-04-25T11:10:29.374373+0000","last_became_active":"2024-04-25T11:10:23.491456+0000","last_became_peered":"2024-04-25T11:10:23.491456+0000","last_unstale":"2024-04-25T11:10:29.374373+0000","last_undegraded":"2024-04-25T11:10:29.374373+0000","last_fullsized":"2024-04-25T11:10:29.374373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.097249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.292111+0000","last_change":"2024-04-25T11:10:23.469823+0000","last_active":"2024-04-25T11:10:29.292111+0000","last_peered":"2024-04-25T11:10:29.292111+0000","last_clean":"2024-04-25T11:10:29.292111+0000","last_became_active":"2024-04-25T11:10:23.469420+0000","last_became_peered":"2024-04-25T11:10:23.469420+0000","last_unstale":"2024-04-25T11:10:29.292111+0000","last_undegraded":"2024-04-25T11:10:29.292111+0000","last_fullsized":"2024-04-25T11:10:29.292111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290180+0000","last_change":"2024-04-25T11:10:23.479442+0000","last_active":"2024-04-25T11:10:29.290180+0000","last_peered":"2024-04-25T11:10:29.290180+0000","last_clean":"2024-04-25T11:10:29.290180+0000","last_became_active":"2024-04-25T11:10:23.472659+0000","last_became_peered":"2024-04-25T11:10:23.472659+0000","last_unstale":"2024-04-25T11:10:29.290180+0000","last_undegraded":"2024-04-25T11:10:29.290180+0000","last_fullsized":"2024-04-25T11:10:29.290180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531629+0000","last_change":"2024-04-25T11:10:23.490775+0000","last_active":"2024-04-25T11:10:28.531629+0000","last_peered":"2024-04-25T11:10:28.531629+0000","last_clean":"2024-04-25T11:10:28.531629+0000","last_became_active":"2024-04-25T11:10:23.490138+0000","last_became_peered":"2024-04-25T11:10:23.490138+0000","last_unstale":"2024-04-25T11:10:28.531629+0000","last_undegraded":"2024-04-25T11:10:28.531629+0000","last_fullsized":"2024-04-25T11:10:28.531629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893293+0000","last_change":"2024-04-25T11:10:23.479882+0000","last_active":"2024-04-25T11:10:29.893293+0000","last_peered":"2024-04-25T11:10:29.893293+0000","last_clean":"2024-04-25T11:10:29.893293+0000","last_became_active":"2024-04-25T11:10:23.479534+0000","last_became_peered":"2024-04-25T11:10:23.479534+0000","last_unstale":"2024-04-25T11:10:29.893293+0000","last_undegraded":"2024-04-25T11:10:29.893293+0000","last_fullsized":"2024-04-25T11:10:29.893293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:04.105145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892215+0000","last_change":"2024-04-25T11:10:23.483845+0000","last_active":"2024-04-25T11:10:29.892215+0000","last_peered":"2024-04-25T11:10:29.892215+0000","last_clean":"2024-04-25T11:10:29.892215+0000","last_became_active":"2024-04-25T11:10:23.482401+0000","last_became_peered":"2024-04-25T11:10:23.482401+0000","last_unstale":"2024-04-25T11:10:29.892215+0000","last_undegraded":"2024-04-25T11:10:29.892215+0000","last_fullsized":"2024-04-25T11:10:29.892215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537880+0000","last_change":"2024-04-25T11:10:23.490750+0000","last_active":"2024-04-25T11:10:28.537880+0000","last_peered":"2024-04-25T11:10:28.537880+0000","last_clean":"2024-04-25T11:10:28.537880+0000","last_became_active":"2024-04-25T11:10:23.490489+0000","last_became_peered":"2024-04-25T11:10:23.490489+0000","last_unstale":"2024-04-25T11:10:28.537880+0000","last_undegraded":"2024-04-25T11:10:28.537880+0000","last_fullsized":"2024-04-25T11:10:28.537880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:30.205583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891988+0000","last_change":"2024-04-25T11:10:23.498316+0000","last_active":"2024-04-25T11:10:29.891988+0000","last_peered":"2024-04-25T11:10:29.891988+0000","last_clean":"2024-04-25T11:10:29.891988+0000","last_became_active":"2024-04-25T11:10:23.497414+0000","last_became_peered":"2024-04-25T11:10:23.497414+0000","last_unstale":"2024-04-25T11:10:29.891988+0000","last_undegraded":"2024-04-25T11:10:29.891988+0000","last_fullsized":"2024-04-25T11:10:29.891988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892093+0000","last_change":"2024-04-25T11:10:23.487588+0000","last_active":"2024-04-25T11:10:29.892093+0000","last_peered":"2024-04-25T11:10:29.892093+0000","last_clean":"2024-04-25T11:10:29.892093+0000","last_became_active":"2024-04-25T11:10:23.487280+0000","last_became_peered":"2024-04-25T11:10:23.487280+0000","last_unstale":"2024-04-25T11:10:29.892093+0000","last_undegraded":"2024-04-25T11:10:29.892093+0000","last_fullsized":"2024-04-25T11:10:29.892093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538694+0000","last_change":"2024-04-25T11:10:23.472328+0000","last_active":"2024-04-25T11:10:28.538694+0000","last_peered":"2024-04-25T11:10:28.538694+0000","last_clean":"2024-04-25T11:10:28.538694+0000","last_became_active":"2024-04-25T11:10:23.471692+0000","last_became_peered":"2024-04-25T11:10:23.471692+0000","last_unstale":"2024-04-25T11:10:28.538694+0000","last_undegraded":"2024-04-25T11:10:28.538694+0000","last_fullsized":"2024-04-25T11:10:28.538694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:50.974212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891640+0000","last_change":"2024-04-25T11:10:23.471703+0000","last_active":"2024-04-25T11:10:29.891640+0000","last_peered":"2024-04-25T11:10:29.891640+0000","last_clean":"2024-04-25T11:10:29.891640+0000","last_became_active":"2024-04-25T11:10:23.469570+0000","last_became_peered":"2024-04-25T11:10:23.469570+0000","last_unstale":"2024-04-25T11:10:29.891640+0000","last_undegraded":"2024-04-25T11:10:29.891640+0000","last_fullsized":"2024-04-25T11:10:29.891640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535108+0000","last_change":"2024-04-25T11:10:23.473106+0000","last_active":"2024-04-25T11:10:28.535108+0000","last_peered":"2024-04-25T11:10:28.535108+0000","last_clean":"2024-04-25T11:10:28.535108+0000","last_became_active":"2024-04-25T11:10:23.472627+0000","last_became_peered":"2024-04-25T11:10:23.472627+0000","last_unstale":"2024-04-25T11:10:28.535108+0000","last_undegraded":"2024-04-25T11:10:28.535108+0000","last_fullsized":"2024-04-25T11:10:28.535108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290419+0000","last_change":"2024-04-25T11:10:23.480283+0000","last_active":"2024-04-25T11:10:29.290419+0000","last_peered":"2024-04-25T11:10:29.290419+0000","last_clean":"2024-04-25T11:10:29.290419+0000","last_became_active":"2024-04-25T11:10:23.479889+0000","last_became_peered":"2024-04-25T11:10:23.479889+0000","last_unstale":"2024-04-25T11:10:29.290419+0000","last_undegraded":"2024-04-25T11:10:29.290419+0000","last_fullsized":"2024-04-25T11:10:29.290419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534607+0000","last_change":"2024-04-25T11:10:23.469104+0000","last_active":"2024-04-25T11:10:28.534607+0000","last_peered":"2024-04-25T11:10:28.534607+0000","last_clean":"2024-04-25T11:10:28.534607+0000","last_became_active":"2024-04-25T11:10:23.468743+0000","last_became_peered":"2024-04-25T11:10:23.468743+0000","last_unstale":"2024-04-25T11:10:28.534607+0000","last_undegraded":"2024-04-25T11:10:28.534607+0000","last_fullsized":"2024-04-25T11:10:28.534607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291058+0000","last_change":"2024-04-25T11:10:23.496747+0000","last_active":"2024-04-25T11:10:29.291058+0000","last_peered":"2024-04-25T11:10:29.291058+0000","last_clean":"2024-04-25T11:10:29.291058+0000","last_became_active":"2024-04-25T11:10:23.496354+0000","last_became_peered":"2024-04-25T11:10:23.496354+0000","last_unstale":"2024-04-25T11:10:29.291058+0000","last_undegraded":"2024-04-25T11:10:29.291058+0000","last_fullsized":"2024-04-25T11:10:29.291058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:41.745805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893849+0000","last_change":"2024-04-25T11:10:23.480571+0000","last_active":"2024-04-25T11:10:29.893849+0000","last_peered":"2024-04-25T11:10:29.893849+0000","last_clean":"2024-04-25T11:10:29.893849+0000","last_became_active":"2024-04-25T11:10:23.479502+0000","last_became_peered":"2024-04-25T11:10:23.479502+0000","last_unstale":"2024-04-25T11:10:29.893849+0000","last_undegraded":"2024-04-25T11:10:29.893849+0000","last_fullsized":"2024-04-25T11:10:29.893849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533940+0000","last_change":"2024-04-25T11:10:23.475376+0000","last_active":"2024-04-25T11:10:28.533940+0000","last_peered":"2024-04-25T11:10:28.533940+0000","last_clean":"2024-04-25T11:10:28.533940+0000","last_became_active":"2024-04-25T11:10:23.474750+0000","last_became_peered":"2024-04-25T11:10:23.474750+0000","last_unstale":"2024-04-25T11:10:28.533940+0000","last_undegraded":"2024-04-25T11:10:28.533940+0000","last_fullsized":"2024-04-25T11:10:28.533940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538538+0000","last_change":"2024-04-25T11:10:23.481991+0000","last_active":"2024-04-25T11:10:28.538538+0000","last_peered":"2024-04-25T11:10:28.538538+0000","last_clean":"2024-04-25T11:10:28.538538+0000","last_became_active":"2024-04-25T11:10:23.481639+0000","last_became_peered":"2024-04-25T11:10:23.481639+0000","last_unstale":"2024-04-25T11:10:28.538538+0000","last_undegraded":"2024-04-25T11:10:28.538538+0000","last_fullsized":"2024-04-25T11:10:28.538538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:13.333031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476745,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476746,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476745,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476746,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":104646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:10:55.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:10:55.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.986+0000 7f185e600640 1 Processor -- start 2024-04-25T11:10:55.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.986+0000 7f185e600640 1 -- start start 2024-04-25T11:10:55.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.988+0000 7f185e600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f18580aed40 0x7f185815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.988+0000 7f185e600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18580b5220 0x7f18580ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.988+0000 7f185e600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18580b64b0 con 0x7f18580af420 2024-04-25T11:10:55.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.988+0000 7f185e600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f185815e1f0 con 0x7f18580b5220 2024-04-25T11:10:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.989+0000 7f185e600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f185806c610 con 0x7f18580aed40 2024-04-25T11:10:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.989+0000 7f1857e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18580b5220 0x7f18580ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.989+0000 7f1857e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18580b5220 0x7f18580ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48682/0 (socket says 172.21.15.7:48682) 2024-04-25T11:10:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.989+0000 7f1857e00640 1 -- 172.21.15.7:0/3467505219 learned_addr learned my addr 172.21.15.7:0/3467505219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.989+0000 7f185d200640 1 --2- 172.21.15.7:0/3467505219 >> v2:172.21.15.87:3300/0 conn(0x7f18580aed40 0x7f185815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.989+0000 7f1857e00640 1 -- 172.21.15.7:0/3467505219 >> v2:172.21.15.87:3300/0 conn(0x7f18580aed40 msgr2=0x7f185815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.989+0000 7f1857e00640 1 --2- 172.21.15.7:0/3467505219 >> v2:172.21.15.87:3300/0 conn(0x7f18580aed40 0x7f185815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.990+0000 7f1857e00640 1 -- 172.21.15.7:0/3467505219 >> v1:172.21.15.87:6789/0 conn(0x7f18580af420 legacy=0x7f18580af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:55.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.990+0000 7f1857e00640 1 -- 172.21.15.7:0/3467505219 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185806c8f0 con 0x7f18580b5220 2024-04-25T11:10:55.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.991+0000 7f1857e00640 1 --2- 172.21.15.7:0/3467505219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18580b5220 0x7f18580ce4a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18580725e0 tx=0x7f184402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bfd7a2eace7e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:55.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.991+0000 7f1857400640 1 -- 172.21.15.7:0/3467505219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844037070 con 0x7f18580b5220 2024-04-25T11:10:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.991+0000 7f1857400640 1 -- 172.21.15.7:0/3467505219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1844032440 con 0x7f18580b5220 2024-04-25T11:10:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.991+0000 7f1857400640 1 -- 172.21.15.7:0/3467505219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844032810 con 0x7f18580b5220 2024-04-25T11:10:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.991+0000 7f185e600640 1 -- 172.21.15.7:0/3467505219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18580b5220 msgr2=0x7f18580ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.992+0000 7f185e600640 1 --2- 172.21.15.7:0/3467505219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18580b5220 0x7f18580ce4a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f18580725e0 tx=0x7f184402a270 comp rx=0 tx=0).stop 2024-04-25T11:10:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.992+0000 7f185e600640 1 -- 172.21.15.7:0/3467505219 shutdown_connections 2024-04-25T11:10:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.992+0000 7f185e600640 1 --2- 172.21.15.7:0/3467505219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18580b5220 0x7f18580ce4a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.992+0000 7f185e600640 1 --2- 172.21.15.7:0/3467505219 >> v2:172.21.15.87:3300/0 conn(0x7f18580aed40 0x7f185815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.992+0000 7f185e600640 1 -- 172.21.15.7:0/3467505219 >> 172.21.15.7:0/3467505219 conn(0x7f18580b3820 msgr2=0x7f18580b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.992+0000 7f185e600640 1 -- 172.21.15.7:0/3467505219 shutdown_connections 2024-04-25T11:10:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.992+0000 7f185e600640 1 -- 172.21.15.7:0/3467505219 wait complete. 2024-04-25T11:10:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.994+0000 7f185e600640 1 Processor -- start 2024-04-25T11:10:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.994+0000 7f185e600640 1 -- start start 2024-04-25T11:10:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18580aed40 0x7f1858156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18580af420 0x7f1858160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185e600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18580b5220 0x7f1858147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18580b6110 con 0x7f18580aed40 2024-04-25T11:10:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185e600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f185806ceb0 con 0x7f18580af420 2024-04-25T11:10:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185e600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18580b6290 con 0x7f18580b5220 2024-04-25T11:10:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18580aed40 0x7f1858156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18580aed40 0x7f1858156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48698/0 (socket says 172.21.15.7:48698) 2024-04-25T11:10:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185d200640 1 -- 172.21.15.7:0/90850169 learned_addr learned my addr 172.21.15.7:0/90850169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:55.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f1857e00640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18580af420 0x7f1858160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185d200640 1 -- 172.21.15.7:0/90850169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18580b5220 msgr2=0x7f1858147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:55.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185dc00640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18580b5220 0x7f1858147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185d200640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18580b5220 0x7f1858147a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185d200640 1 -- 172.21.15.7:0/90850169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18580af420 msgr2=0x7f1858160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185d200640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18580af420 0x7f1858160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185d200640 1 -- 172.21.15.7:0/90850169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1858154df0 con 0x7f18580aed40 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.996+0000 7f185dc00640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18580b5220 0x7f1858147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.997+0000 7f185d200640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18580aed40 0x7f1858156300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f184800d5a0 tx=0x7f184800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.997+0000 7f1854c00640 1 -- 172.21.15.7:0/90850169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18480125f0 con 0x7f18580aed40 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.997+0000 7f1854c00640 1 -- 172.21.15.7:0/90850169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1848019070 con 0x7f18580aed40 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.997+0000 7f185e600640 1 -- 172.21.15.7:0/90850169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1858155260 con 0x7f18580aed40 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.997+0000 7f1854c00640 1 -- 172.21.15.7:0/90850169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18480129d0 con 0x7f18580aed40 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.997+0000 7f185e600640 1 -- 172.21.15.7:0/90850169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1858155820 con 0x7f18580aed40 2024-04-25T11:10:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.998+0000 7f185e600640 1 -- 172.21.15.7:0/90850169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1858050740 con 0x7f18580aed40 2024-04-25T11:10:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:55.999+0000 7f1854c00640 1 -- 172.21.15.7:0/90850169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1848012b70 con 0x7f18580aed40 2024-04-25T11:10:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.000+0000 7f1854c00640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18380744e0 0x7f18380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.000+0000 7f1854c00640 1 -- 172.21.15.7:0/90850169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1848095990 con 0x7f18580aed40 2024-04-25T11:10:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.001+0000 7f1857e00640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18380744e0 0x7f18380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.001+0000 7f1857e00640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18380744e0 0x7f18380769f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1844034000 tx=0x7f184402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.005+0000 7f1854c00640 1 -- 172.21.15.7:0/90850169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f184805fdd0 con 0x7f18580aed40 2024-04-25T11:10:56.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.199+0000 7f185e600640 1 -- 172.21.15.7:0/90850169 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18580b4920 con 0x7f18380744e0 2024-04-25T11:10:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.209+0000 7f1854c00640 1 -- 172.21.15.7:0/90850169 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332526 (secure 0 0 0) 0x7f18580b4920 con 0x7f18380744e0 2024-04-25T11:10:56.212 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:56.215 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:10:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.216+0000 7f185e600640 1 -- 172.21.15.7:0/90850169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18380744e0 msgr2=0x7f18380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.216+0000 7f185e600640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18380744e0 0x7f18380769f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1844034000 tx=0x7f184402e000 comp rx=0 tx=0).stop 2024-04-25T11:10:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.216+0000 7f185e600640 1 -- 172.21.15.7:0/90850169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18580aed40 msgr2=0x7f1858156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.216+0000 7f185e600640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18580aed40 0x7f1858156300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f184800d5a0 tx=0x7f184800da60 comp rx=0 tx=0).stop 2024-04-25T11:10:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.216+0000 7f185e600640 1 -- 172.21.15.7:0/90850169 shutdown_connections 2024-04-25T11:10:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.216+0000 7f185e600640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18380744e0 0x7f18380769f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.216+0000 7f185e600640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18580b5220 0x7f1858147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.216+0000 7f185e600640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18580af420 0x7f1858160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.216+0000 7f185e600640 1 --2- 172.21.15.7:0/90850169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18580aed40 0x7f1858156300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.216+0000 7f185e600640 1 -- 172.21.15.7:0/90850169 >> 172.21.15.7:0/90850169 conn(0x7f18580b3820 msgr2=0x7f18580b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.217+0000 7f185e600640 1 -- 172.21.15.7:0/90850169 shutdown_connections 2024-04-25T11:10:56.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.217+0000 7f185e600640 1 -- 172.21.15.7:0/90850169 wait complete. 2024-04-25T11:10:56.240 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2024-04-25T11:10:55.791826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231960,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742760,"statfs":{"total":773094113280,"available":772856586240,"internally_reserved":0,"allocated":3383296,"data_stored":2025904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005469"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895493+0000","last_change":"2024-04-25T11:10:22.475060+0000","last_active":"2024-04-25T11:10:29.895493+0000","last_peered":"2024-04-25T11:10:29.895493+0000","last_clean":"2024-04-25T11:10:29.895493+0000","last_became_active":"2024-04-25T11:10:22.472574+0000","last_became_peered":"2024-04-25T11:10:22.472574+0000","last_unstale":"2024-04-25T11:10:29.895493+0000","last_undegraded":"2024-04-25T11:10:29.895493+0000","last_fullsized":"2024-04-25T11:10:29.895493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.835819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895751+0000","last_change":"2024-04-25T11:10:23.488367+0000","last_active":"2024-04-25T11:10:29.895751+0000","last_peered":"2024-04-25T11:10:29.895751+0000","last_clean":"2024-04-25T11:10:29.895751+0000","last_became_active":"2024-04-25T11:10:23.488034+0000","last_became_peered":"2024-04-25T11:10:23.488034+0000","last_unstale":"2024-04-25T11:10:29.895751+0000","last_undegraded":"2024-04-25T11:10:29.895751+0000","last_fullsized":"2024-04-25T11:10:29.895751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892242+0000","last_change":"2024-04-25T11:10:27.551067+0000","last_active":"2024-04-25T11:10:29.892242+0000","last_peered":"2024-04-25T11:10:29.892242+0000","last_clean":"2024-04-25T11:10:29.892242+0000","last_became_active":"2024-04-25T11:10:27.550247+0000","last_became_peered":"2024-04-25T11:10:27.550247+0000","last_unstale":"2024-04-25T11:10:29.892242+0000","last_undegraded":"2024-04-25T11:10:29.892242+0000","last_fullsized":"2024-04-25T11:10:29.892242+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:29.432696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895898+0000","last_change":"2024-04-25T11:10:22.463507+0000","last_active":"2024-04-25T11:10:29.895898+0000","last_peered":"2024-04-25T11:10:29.895898+0000","last_clean":"2024-04-25T11:10:29.895898+0000","last_became_active":"2024-04-25T11:10:22.463142+0000","last_became_peered":"2024-04-25T11:10:22.463142+0000","last_unstale":"2024-04-25T11:10:29.895898+0000","last_undegraded":"2024-04-25T11:10:29.895898+0000","last_fullsized":"2024-04-25T11:10:29.895898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:42.000742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894569+0000","last_change":"2024-04-25T11:10:23.500843+0000","last_active":"2024-04-25T11:10:29.894569+0000","last_peered":"2024-04-25T11:10:29.894569+0000","last_clean":"2024-04-25T11:10:29.894569+0000","last_became_active":"2024-04-25T11:10:23.500412+0000","last_became_peered":"2024-04-25T11:10:23.500412+0000","last_unstale":"2024-04-25T11:10:29.894569+0000","last_undegraded":"2024-04-25T11:10:29.894569+0000","last_fullsized":"2024-04-25T11:10:29.894569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534853+0000","last_change":"2024-04-25T11:10:27.565431+0000","last_active":"2024-04-25T11:10:28.534853+0000","last_peered":"2024-04-25T11:10:28.534853+0000","last_clean":"2024-04-25T11:10:28.534853+0000","last_became_active":"2024-04-25T11:10:27.564724+0000","last_became_peered":"2024-04-25T11:10:27.564724+0000","last_unstale":"2024-04-25T11:10:28.534853+0000","last_undegraded":"2024-04-25T11:10:28.534853+0000","last_fullsized":"2024-04-25T11:10:28.534853+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:51.033161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536873+0000","last_change":"2024-04-25T11:10:22.477443+0000","last_active":"2024-04-25T11:10:28.536873+0000","last_peered":"2024-04-25T11:10:28.536873+0000","last_clean":"2024-04-25T11:10:28.536873+0000","last_became_active":"2024-04-25T11:10:22.477131+0000","last_became_peered":"2024-04-25T11:10:22.477131+0000","last_unstale":"2024-04-25T11:10:28.536873+0000","last_undegraded":"2024-04-25T11:10:28.536873+0000","last_fullsized":"2024-04-25T11:10:28.536873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:57.426313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291476+0000","last_change":"2024-04-25T11:10:23.496542+0000","last_active":"2024-04-25T11:10:29.291476+0000","last_peered":"2024-04-25T11:10:29.291476+0000","last_clean":"2024-04-25T11:10:29.291476+0000","last_became_active":"2024-04-25T11:10:23.496211+0000","last_became_peered":"2024-04-25T11:10:23.496211+0000","last_unstale":"2024-04-25T11:10:29.291476+0000","last_undegraded":"2024-04-25T11:10:29.291476+0000","last_fullsized":"2024-04-25T11:10:29.291476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:15.460249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537266+0000","last_change":"2024-04-25T11:10:27.565126+0000","last_active":"2024-04-25T11:10:28.537266+0000","last_peered":"2024-04-25T11:10:28.537266+0000","last_clean":"2024-04-25T11:10:28.537266+0000","last_became_active":"2024-04-25T11:10:27.564664+0000","last_became_peered":"2024-04-25T11:10:27.564664+0000","last_unstale":"2024-04-25T11:10:28.537266+0000","last_undegraded":"2024-04-25T11:10:28.537266+0000","last_fullsized":"2024-04-25T11:10:28.537266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:44.604923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536194+0000","last_change":"2024-04-25T11:10:22.483531+0000","last_active":"2024-04-25T11:10:28.536194+0000","last_peered":"2024-04-25T11:10:28.536194+0000","last_clean":"2024-04-25T11:10:28.536194+0000","last_became_active":"2024-04-25T11:10:22.483204+0000","last_became_peered":"2024-04-25T11:10:22.483204+0000","last_unstale":"2024-04-25T11:10:28.536194+0000","last_undegraded":"2024-04-25T11:10:28.536194+0000","last_fullsized":"2024-04-25T11:10:28.536194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:03.866498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291328+0000","last_change":"2024-04-25T11:10:23.490550+0000","last_active":"2024-04-25T11:10:29.291328+0000","last_peered":"2024-04-25T11:10:29.291328+0000","last_clean":"2024-04-25T11:10:29.291328+0000","last_became_active":"2024-04-25T11:10:23.490300+0000","last_became_peered":"2024-04-25T11:10:23.490300+0000","last_unstale":"2024-04-25T11:10:29.291328+0000","last_undegraded":"2024-04-25T11:10:29.291328+0000","last_fullsized":"2024-04-25T11:10:29.291328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:20.100576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535567+0000","last_change":"2024-04-25T11:10:27.555808+0000","last_active":"2024-04-25T11:10:28.535567+0000","last_peered":"2024-04-25T11:10:28.535567+0000","last_clean":"2024-04-25T11:10:28.535567+0000","last_became_active":"2024-04-25T11:10:27.555277+0000","last_became_peered":"2024-04-25T11:10:27.555277+0000","last_unstale":"2024-04-25T11:10:28.535567+0000","last_undegraded":"2024-04-25T11:10:28.535567+0000","last_fullsized":"2024-04-25T11:10:28.535567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:52.665016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.890747+0000","last_change":"2024-04-25T11:10:23.493263+0000","last_active":"2024-04-25T11:10:29.890747+0000","last_peered":"2024-04-25T11:10:29.890747+0000","last_clean":"2024-04-25T11:10:29.890747+0000","last_became_active":"2024-04-25T11:10:23.489625+0000","last_became_peered":"2024-04-25T11:10:23.489625+0000","last_unstale":"2024-04-25T11:10:29.890747+0000","last_undegraded":"2024-04-25T11:10:29.890747+0000","last_fullsized":"2024-04-25T11:10:29.890747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535220+0000","last_change":"2024-04-25T11:10:22.475840+0000","last_active":"2024-04-25T11:10:28.535220+0000","last_peered":"2024-04-25T11:10:28.535220+0000","last_clean":"2024-04-25T11:10:28.535220+0000","last_became_active":"2024-04-25T11:10:22.475539+0000","last_became_peered":"2024-04-25T11:10:22.475539+0000","last_unstale":"2024-04-25T11:10:28.535220+0000","last_undegraded":"2024-04-25T11:10:28.535220+0000","last_fullsized":"2024-04-25T11:10:28.535220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:49.860961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538827+0000","last_change":"2024-04-25T11:10:27.558933+0000","last_active":"2024-04-25T11:10:28.538827+0000","last_peered":"2024-04-25T11:10:28.538827+0000","last_clean":"2024-04-25T11:10:28.538827+0000","last_became_active":"2024-04-25T11:10:27.558566+0000","last_became_peered":"2024-04-25T11:10:27.558566+0000","last_unstale":"2024-04-25T11:10:28.538827+0000","last_undegraded":"2024-04-25T11:10:28.538827+0000","last_fullsized":"2024-04-25T11:10:28.538827+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:45.815273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892352+0000","last_change":"2024-04-25T11:10:23.482991+0000","last_active":"2024-04-25T11:10:29.892352+0000","last_peered":"2024-04-25T11:10:29.892352+0000","last_clean":"2024-04-25T11:10:29.892352+0000","last_became_active":"2024-04-25T11:10:23.481972+0000","last_became_peered":"2024-04-25T11:10:23.481972+0000","last_unstale":"2024-04-25T11:10:29.892352+0000","last_undegraded":"2024-04-25T11:10:29.892352+0000","last_fullsized":"2024-04-25T11:10:29.892352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533468+0000","last_change":"2024-04-25T11:10:22.469445+0000","last_active":"2024-04-25T11:10:28.533468+0000","last_peered":"2024-04-25T11:10:28.533468+0000","last_clean":"2024-04-25T11:10:28.533468+0000","last_became_active":"2024-04-25T11:10:22.469095+0000","last_became_peered":"2024-04-25T11:10:22.469095+0000","last_unstale":"2024-04-25T11:10:28.533468+0000","last_undegraded":"2024-04-25T11:10:28.533468+0000","last_fullsized":"2024-04-25T11:10:28.533468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537528+0000","last_change":"2024-04-25T11:10:27.559144+0000","last_active":"2024-04-25T11:10:28.537528+0000","last_peered":"2024-04-25T11:10:28.537528+0000","last_clean":"2024-04-25T11:10:28.537528+0000","last_became_active":"2024-04-25T11:10:27.558635+0000","last_became_peered":"2024-04-25T11:10:27.558635+0000","last_unstale":"2024-04-25T11:10:28.537528+0000","last_undegraded":"2024-04-25T11:10:28.537528+0000","last_fullsized":"2024-04-25T11:10:28.537528+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:24.222645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892015+0000","last_change":"2024-04-25T11:10:23.460813+0000","last_active":"2024-04-25T11:10:29.892015+0000","last_peered":"2024-04-25T11:10:29.892015+0000","last_clean":"2024-04-25T11:10:29.892015+0000","last_became_active":"2024-04-25T11:10:23.460435+0000","last_became_peered":"2024-04-25T11:10:23.460435+0000","last_unstale":"2024-04-25T11:10:29.892015+0000","last_undegraded":"2024-04-25T11:10:29.892015+0000","last_fullsized":"2024-04-25T11:10:29.892015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531902+0000","last_change":"2024-04-25T11:10:22.466351+0000","last_active":"2024-04-25T11:10:28.531902+0000","last_peered":"2024-04-25T11:10:28.531902+0000","last_clean":"2024-04-25T11:10:28.531902+0000","last_became_active":"2024-04-25T11:10:22.464110+0000","last_became_peered":"2024-04-25T11:10:22.464110+0000","last_unstale":"2024-04-25T11:10:28.531902+0000","last_undegraded":"2024-04-25T11:10:28.531902+0000","last_fullsized":"2024-04-25T11:10:28.531902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534334+0000","last_change":"2024-04-25T11:10:27.555355+0000","last_active":"2024-04-25T11:10:28.534334+0000","last_peered":"2024-04-25T11:10:28.534334+0000","last_clean":"2024-04-25T11:10:28.534334+0000","last_became_active":"2024-04-25T11:10:27.554893+0000","last_became_peered":"2024-04-25T11:10:27.554893+0000","last_unstale":"2024-04-25T11:10:28.534334+0000","last_undegraded":"2024-04-25T11:10:28.534334+0000","last_fullsized":"2024-04-25T11:10:28.534334+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:51.174327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535363+0000","last_change":"2024-04-25T11:10:23.487680+0000","last_active":"2024-04-25T11:10:28.535363+0000","last_peered":"2024-04-25T11:10:28.535363+0000","last_clean":"2024-04-25T11:10:28.535363+0000","last_became_active":"2024-04-25T11:10:23.483024+0000","last_became_peered":"2024-04-25T11:10:23.483024+0000","last_unstale":"2024-04-25T11:10:28.535363+0000","last_undegraded":"2024-04-25T11:10:28.535363+0000","last_fullsized":"2024-04-25T11:10:28.535363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:06.246101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534055+0000","last_change":"2024-04-25T11:10:22.470867+0000","last_active":"2024-04-25T11:10:28.534055+0000","last_peered":"2024-04-25T11:10:28.534055+0000","last_clean":"2024-04-25T11:10:28.534055+0000","last_became_active":"2024-04-25T11:10:22.470501+0000","last_became_peered":"2024-04-25T11:10:22.470501+0000","last_unstale":"2024-04-25T11:10:28.534055+0000","last_undegraded":"2024-04-25T11:10:28.534055+0000","last_fullsized":"2024-04-25T11:10:28.534055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538997+0000","last_change":"2024-04-25T11:10:27.567703+0000","last_active":"2024-04-25T11:10:28.538997+0000","last_peered":"2024-04-25T11:10:28.538997+0000","last_clean":"2024-04-25T11:10:28.538997+0000","last_became_active":"2024-04-25T11:10:27.567208+0000","last_became_peered":"2024-04-25T11:10:27.567208+0000","last_unstale":"2024-04-25T11:10:28.538997+0000","last_undegraded":"2024-04-25T11:10:28.538997+0000","last_fullsized":"2024-04-25T11:10:28.538997+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:24.170044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895262+0000","last_change":"2024-04-25T11:10:23.488689+0000","last_active":"2024-04-25T11:10:29.895262+0000","last_peered":"2024-04-25T11:10:29.895262+0000","last_clean":"2024-04-25T11:10:29.895262+0000","last_became_active":"2024-04-25T11:10:23.479256+0000","last_became_peered":"2024-04-25T11:10:23.479256+0000","last_unstale":"2024-04-25T11:10:29.895262+0000","last_undegraded":"2024-04-25T11:10:29.895262+0000","last_fullsized":"2024-04-25T11:10:29.895262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892311+0000","last_change":"2024-04-25T11:10:22.486332+0000","last_active":"2024-04-25T11:10:29.892311+0000","last_peered":"2024-04-25T11:10:29.892311+0000","last_clean":"2024-04-25T11:10:29.892311+0000","last_became_active":"2024-04-25T11:10:22.485970+0000","last_became_peered":"2024-04-25T11:10:22.485970+0000","last_unstale":"2024-04-25T11:10:29.892311+0000","last_undegraded":"2024-04-25T11:10:29.892311+0000","last_fullsized":"2024-04-25T11:10:29.892311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534779+0000","last_change":"2024-04-25T11:10:27.549543+0000","last_active":"2024-04-25T11:10:28.534779+0000","last_peered":"2024-04-25T11:10:28.534779+0000","last_clean":"2024-04-25T11:10:28.534779+0000","last_became_active":"2024-04-25T11:10:27.549007+0000","last_became_peered":"2024-04-25T11:10:27.549007+0000","last_unstale":"2024-04-25T11:10:28.534779+0000","last_undegraded":"2024-04-25T11:10:28.534779+0000","last_fullsized":"2024-04-25T11:10:28.534779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:01.381820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896311+0000","last_change":"2024-04-25T11:10:23.465612+0000","last_active":"2024-04-25T11:10:29.896311+0000","last_peered":"2024-04-25T11:10:29.896311+0000","last_clean":"2024-04-25T11:10:29.896311+0000","last_became_active":"2024-04-25T11:10:23.465261+0000","last_became_peered":"2024-04-25T11:10:23.465261+0000","last_unstale":"2024-04-25T11:10:29.896311+0000","last_undegraded":"2024-04-25T11:10:29.896311+0000","last_fullsized":"2024-04-25T11:10:29.896311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534173+0000","last_change":"2024-04-25T11:10:22.483679+0000","last_active":"2024-04-25T11:10:28.534173+0000","last_peered":"2024-04-25T11:10:28.534173+0000","last_clean":"2024-04-25T11:10:28.534173+0000","last_became_active":"2024-04-25T11:10:22.483243+0000","last_became_peered":"2024-04-25T11:10:22.483243+0000","last_unstale":"2024-04-25T11:10:28.534173+0000","last_undegraded":"2024-04-25T11:10:28.534173+0000","last_fullsized":"2024-04-25T11:10:28.534173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:49.860961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891407+0000","last_change":"2024-04-25T11:10:27.550030+0000","last_active":"2024-04-25T11:10:29.891407+0000","last_peered":"2024-04-25T11:10:29.891407+0000","last_clean":"2024-04-25T11:10:29.891407+0000","last_became_active":"2024-04-25T11:10:27.549537+0000","last_became_peered":"2024-04-25T11:10:27.549537+0000","last_unstale":"2024-04-25T11:10:29.891407+0000","last_undegraded":"2024-04-25T11:10:29.891407+0000","last_fullsized":"2024-04-25T11:10:29.891407+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:51.538673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534183+0000","last_change":"2024-04-25T11:10:23.470926+0000","last_active":"2024-04-25T11:10:28.534183+0000","last_peered":"2024-04-25T11:10:28.534183+0000","last_clean":"2024-04-25T11:10:28.534183+0000","last_became_active":"2024-04-25T11:10:23.470483+0000","last_became_peered":"2024-04-25T11:10:23.470483+0000","last_unstale":"2024-04-25T11:10:28.534183+0000","last_undegraded":"2024-04-25T11:10:28.534183+0000","last_fullsized":"2024-04-25T11:10:28.534183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537196+0000","last_change":"2024-04-25T11:10:22.479703+0000","last_active":"2024-04-25T11:10:28.537196+0000","last_peered":"2024-04-25T11:10:28.537196+0000","last_clean":"2024-04-25T11:10:28.537196+0000","last_became_active":"2024-04-25T11:10:22.478929+0000","last_became_peered":"2024-04-25T11:10:22.478929+0000","last_unstale":"2024-04-25T11:10:28.537196+0000","last_undegraded":"2024-04-25T11:10:28.537196+0000","last_fullsized":"2024-04-25T11:10:28.537196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:08.833540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893409+0000","last_change":"2024-04-25T11:10:27.559919+0000","last_active":"2024-04-25T11:10:29.893409+0000","last_peered":"2024-04-25T11:10:29.893409+0000","last_clean":"2024-04-25T11:10:29.893409+0000","last_became_active":"2024-04-25T11:10:27.558805+0000","last_became_peered":"2024-04-25T11:10:27.558805+0000","last_unstale":"2024-04-25T11:10:29.893409+0000","last_undegraded":"2024-04-25T11:10:29.893409+0000","last_fullsized":"2024-04-25T11:10:29.893409+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291634+0000","last_change":"2024-04-25T11:10:23.471738+0000","last_active":"2024-04-25T11:10:29.291634+0000","last_peered":"2024-04-25T11:10:29.291634+0000","last_clean":"2024-04-25T11:10:29.291634+0000","last_became_active":"2024-04-25T11:10:23.471191+0000","last_became_peered":"2024-04-25T11:10:23.471191+0000","last_unstale":"2024-04-25T11:10:29.291634+0000","last_undegraded":"2024-04-25T11:10:29.291634+0000","last_fullsized":"2024-04-25T11:10:29.291634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:47.901333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534371+0000","last_change":"2024-04-25T11:10:22.467322+0000","last_active":"2024-04-25T11:10:28.534371+0000","last_peered":"2024-04-25T11:10:28.534371+0000","last_clean":"2024-04-25T11:10:28.534371+0000","last_became_active":"2024-04-25T11:10:22.466806+0000","last_became_peered":"2024-04-25T11:10:22.466806+0000","last_unstale":"2024-04-25T11:10:28.534371+0000","last_undegraded":"2024-04-25T11:10:28.534371+0000","last_fullsized":"2024-04-25T11:10:28.534371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533236+0000","last_change":"2024-04-25T11:10:27.546558+0000","last_active":"2024-04-25T11:10:28.533236+0000","last_peered":"2024-04-25T11:10:28.533236+0000","last_clean":"2024-04-25T11:10:28.533236+0000","last_became_active":"2024-04-25T11:10:27.544215+0000","last_became_peered":"2024-04-25T11:10:27.544215+0000","last_unstale":"2024-04-25T11:10:28.533236+0000","last_undegraded":"2024-04-25T11:10:28.533236+0000","last_fullsized":"2024-04-25T11:10:28.533236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.970801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537205+0000","last_change":"2024-04-25T11:10:23.489986+0000","last_active":"2024-04-25T11:10:28.537205+0000","last_peered":"2024-04-25T11:10:28.537205+0000","last_clean":"2024-04-25T11:10:28.537205+0000","last_became_active":"2024-04-25T11:10:23.489629+0000","last_became_peered":"2024-04-25T11:10:23.489629+0000","last_unstale":"2024-04-25T11:10:28.537205+0000","last_undegraded":"2024-04-25T11:10:28.537205+0000","last_fullsized":"2024-04-25T11:10:28.537205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:14.555604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291125+0000","last_change":"2024-04-25T11:10:22.482966+0000","last_active":"2024-04-25T11:10:29.291125+0000","last_peered":"2024-04-25T11:10:29.291125+0000","last_clean":"2024-04-25T11:10:29.291125+0000","last_became_active":"2024-04-25T11:10:22.480437+0000","last_became_peered":"2024-04-25T11:10:22.480437+0000","last_unstale":"2024-04-25T11:10:29.291125+0000","last_undegraded":"2024-04-25T11:10:29.291125+0000","last_fullsized":"2024-04-25T11:10:29.291125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534554+0000","last_change":"2024-04-25T11:10:27.549617+0000","last_active":"2024-04-25T11:10:28.534554+0000","last_peered":"2024-04-25T11:10:28.534554+0000","last_clean":"2024-04-25T11:10:28.534554+0000","last_became_active":"2024-04-25T11:10:27.549203+0000","last_became_peered":"2024-04-25T11:10:27.549203+0000","last_unstale":"2024-04-25T11:10:28.534554+0000","last_undegraded":"2024-04-25T11:10:28.534554+0000","last_fullsized":"2024-04-25T11:10:28.534554+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:55.043679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531961+0000","last_change":"2024-04-25T11:10:23.472089+0000","last_active":"2024-04-25T11:10:28.531961+0000","last_peered":"2024-04-25T11:10:28.531961+0000","last_clean":"2024-04-25T11:10:28.531961+0000","last_became_active":"2024-04-25T11:10:23.468460+0000","last_became_peered":"2024-04-25T11:10:23.468460+0000","last_unstale":"2024-04-25T11:10:28.531961+0000","last_undegraded":"2024-04-25T11:10:28.531961+0000","last_fullsized":"2024-04-25T11:10:28.531961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373356+0000","last_change":"2024-04-25T11:10:22.468201+0000","last_active":"2024-04-25T11:10:29.373356+0000","last_peered":"2024-04-25T11:10:29.373356+0000","last_clean":"2024-04-25T11:10:29.373356+0000","last_became_active":"2024-04-25T11:10:22.467845+0000","last_became_peered":"2024-04-25T11:10:22.467845+0000","last_unstale":"2024-04-25T11:10:29.373356+0000","last_undegraded":"2024-04-25T11:10:29.373356+0000","last_fullsized":"2024-04-25T11:10:29.373356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:50.983943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534837+0000","last_change":"2024-04-25T11:10:27.552725+0000","last_active":"2024-04-25T11:10:28.534837+0000","last_peered":"2024-04-25T11:10:28.534837+0000","last_clean":"2024-04-25T11:10:28.534837+0000","last_became_active":"2024-04-25T11:10:27.552073+0000","last_became_peered":"2024-04-25T11:10:27.552073+0000","last_unstale":"2024-04-25T11:10:28.534837+0000","last_undegraded":"2024-04-25T11:10:28.534837+0000","last_fullsized":"2024-04-25T11:10:28.534837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.402499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893257+0000","last_change":"2024-04-25T11:10:23.492738+0000","last_active":"2024-04-25T11:10:29.893257+0000","last_peered":"2024-04-25T11:10:29.893257+0000","last_clean":"2024-04-25T11:10:29.893257+0000","last_became_active":"2024-04-25T11:10:23.492416+0000","last_became_peered":"2024-04-25T11:10:23.492416+0000","last_unstale":"2024-04-25T11:10:29.893257+0000","last_undegraded":"2024-04-25T11:10:29.893257+0000","last_fullsized":"2024-04-25T11:10:29.893257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:00.331944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536183+0000","last_change":"2024-04-25T11:10:22.478976+0000","last_active":"2024-04-25T11:10:28.536183+0000","last_peered":"2024-04-25T11:10:28.536183+0000","last_clean":"2024-04-25T11:10:28.536183+0000","last_became_active":"2024-04-25T11:10:22.478595+0000","last_became_peered":"2024-04-25T11:10:22.478595+0000","last_unstale":"2024-04-25T11:10:28.536183+0000","last_undegraded":"2024-04-25T11:10:28.536183+0000","last_fullsized":"2024-04-25T11:10:28.536183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895071+0000","last_change":"2024-04-25T11:10:27.558873+0000","last_active":"2024-04-25T11:10:29.895071+0000","last_peered":"2024-04-25T11:10:29.895071+0000","last_clean":"2024-04-25T11:10:29.895071+0000","last_became_active":"2024-04-25T11:10:27.557790+0000","last_became_peered":"2024-04-25T11:10:27.557790+0000","last_unstale":"2024-04-25T11:10:29.895071+0000","last_undegraded":"2024-04-25T11:10:29.895071+0000","last_fullsized":"2024-04-25T11:10:29.895071+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.402499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893187+0000","last_change":"2024-04-25T11:10:23.470011+0000","last_active":"2024-04-25T11:10:29.893187+0000","last_peered":"2024-04-25T11:10:29.893187+0000","last_clean":"2024-04-25T11:10:29.893187+0000","last_became_active":"2024-04-25T11:10:23.469655+0000","last_became_peered":"2024-04-25T11:10:23.469655+0000","last_unstale":"2024-04-25T11:10:29.893187+0000","last_undegraded":"2024-04-25T11:10:29.893187+0000","last_fullsized":"2024-04-25T11:10:29.893187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291689+0000","last_change":"2024-04-25T11:10:22.480507+0000","last_active":"2024-04-25T11:10:29.291689+0000","last_peered":"2024-04-25T11:10:29.291689+0000","last_clean":"2024-04-25T11:10:29.291689+0000","last_became_active":"2024-04-25T11:10:22.479568+0000","last_became_peered":"2024-04-25T11:10:22.479568+0000","last_unstale":"2024-04-25T11:10:29.291689+0000","last_undegraded":"2024-04-25T11:10:29.291689+0000","last_fullsized":"2024-04-25T11:10:29.291689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":118,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536490+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:10:28.536490+0000","last_peered":"2024-04-25T11:10:28.536490+0000","last_clean":"2024-04-25T11:10:28.536490+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:10:28.536490+0000","last_undegraded":"2024-04-25T11:10:28.536490+0000","last_fullsized":"2024-04-25T11:10:28.536490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891956+0000","last_change":"2024-04-25T11:10:27.551611+0000","last_active":"2024-04-25T11:10:29.891956+0000","last_peered":"2024-04-25T11:10:29.891956+0000","last_clean":"2024-04-25T11:10:29.891956+0000","last_became_active":"2024-04-25T11:10:27.551117+0000","last_became_peered":"2024-04-25T11:10:27.551117+0000","last_unstale":"2024-04-25T11:10:29.891956+0000","last_undegraded":"2024-04-25T11:10:29.891956+0000","last_fullsized":"2024-04-25T11:10:29.891956+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536555+0000","last_change":"2024-04-25T11:10:23.469975+0000","last_active":"2024-04-25T11:10:28.536555+0000","last_peered":"2024-04-25T11:10:28.536555+0000","last_clean":"2024-04-25T11:10:28.536555+0000","last_became_active":"2024-04-25T11:10:23.469366+0000","last_became_peered":"2024-04-25T11:10:23.469366+0000","last_unstale":"2024-04-25T11:10:28.536555+0000","last_undegraded":"2024-04-25T11:10:28.536555+0000","last_fullsized":"2024-04-25T11:10:28.536555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:47.901333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373786+0000","last_change":"2024-04-25T11:10:22.480954+0000","last_active":"2024-04-25T11:10:29.373786+0000","last_peered":"2024-04-25T11:10:29.373786+0000","last_clean":"2024-04-25T11:10:29.373786+0000","last_became_active":"2024-04-25T11:10:22.480489+0000","last_became_peered":"2024-04-25T11:10:22.480489+0000","last_unstale":"2024-04-25T11:10:29.373786+0000","last_undegraded":"2024-04-25T11:10:29.373786+0000","last_fullsized":"2024-04-25T11:10:29.373786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:11.295052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891284+0000","last_change":"2024-04-25T11:10:27.548113+0000","last_active":"2024-04-25T11:10:29.891284+0000","last_peered":"2024-04-25T11:10:29.891284+0000","last_clean":"2024-04-25T11:10:29.891284+0000","last_became_active":"2024-04-25T11:10:27.547579+0000","last_became_peered":"2024-04-25T11:10:27.547579+0000","last_unstale":"2024-04-25T11:10:29.891284+0000","last_undegraded":"2024-04-25T11:10:29.891284+0000","last_fullsized":"2024-04-25T11:10:29.891284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:45.815273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291802+0000","last_change":"2024-04-25T11:10:23.478738+0000","last_active":"2024-04-25T11:10:29.291802+0000","last_peered":"2024-04-25T11:10:29.291802+0000","last_clean":"2024-04-25T11:10:29.291802+0000","last_became_active":"2024-04-25T11:10:23.468486+0000","last_became_peered":"2024-04-25T11:10:23.468486+0000","last_unstale":"2024-04-25T11:10:29.291802+0000","last_undegraded":"2024-04-25T11:10:29.291802+0000","last_fullsized":"2024-04-25T11:10:29.291802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:47.104859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533727+0000","last_change":"2024-04-25T11:10:22.473530+0000","last_active":"2024-04-25T11:10:28.533727+0000","last_peered":"2024-04-25T11:10:28.533727+0000","last_clean":"2024-04-25T11:10:28.533727+0000","last_became_active":"2024-04-25T11:10:22.473177+0000","last_became_peered":"2024-04-25T11:10:22.473177+0000","last_unstale":"2024-04-25T11:10:28.533727+0000","last_undegraded":"2024-04-25T11:10:28.533727+0000","last_fullsized":"2024-04-25T11:10:28.533727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894250+0000","last_change":"2024-04-25T11:10:27.559345+0000","last_active":"2024-04-25T11:10:29.894250+0000","last_peered":"2024-04-25T11:10:29.894250+0000","last_clean":"2024-04-25T11:10:29.894250+0000","last_became_active":"2024-04-25T11:10:27.558090+0000","last_became_peered":"2024-04-25T11:10:27.558090+0000","last_unstale":"2024-04-25T11:10:29.894250+0000","last_undegraded":"2024-04-25T11:10:29.894250+0000","last_fullsized":"2024-04-25T11:10:29.894250+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:51.174327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538883+0000","last_change":"2024-04-25T11:10:23.477613+0000","last_active":"2024-04-25T11:10:28.538883+0000","last_peered":"2024-04-25T11:10:28.538883+0000","last_clean":"2024-04-25T11:10:28.538883+0000","last_became_active":"2024-04-25T11:10:23.477226+0000","last_became_peered":"2024-04-25T11:10:23.477226+0000","last_unstale":"2024-04-25T11:10:28.538883+0000","last_undegraded":"2024-04-25T11:10:28.538883+0000","last_fullsized":"2024-04-25T11:10:28.538883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:07.701501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.374436+0000","last_change":"2024-04-25T11:10:22.468389+0000","last_active":"2024-04-25T11:10:29.374436+0000","last_peered":"2024-04-25T11:10:29.374436+0000","last_clean":"2024-04-25T11:10:29.374436+0000","last_became_active":"2024-04-25T11:10:22.467926+0000","last_became_peered":"2024-04-25T11:10:22.467926+0000","last_unstale":"2024-04-25T11:10:29.374436+0000","last_undegraded":"2024-04-25T11:10:29.374436+0000","last_fullsized":"2024-04-25T11:10:29.374436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.448477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290385+0000","last_change":"2024-04-25T11:10:27.547813+0000","last_active":"2024-04-25T11:10:29.290385+0000","last_peered":"2024-04-25T11:10:29.290385+0000","last_clean":"2024-04-25T11:10:29.290385+0000","last_became_active":"2024-04-25T11:10:27.547300+0000","last_became_peered":"2024-04-25T11:10:27.547300+0000","last_unstale":"2024-04-25T11:10:29.290385+0000","last_undegraded":"2024-04-25T11:10:29.290385+0000","last_fullsized":"2024-04-25T11:10:29.290385+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:51.538673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537521+0000","last_change":"2024-04-25T11:10:23.499736+0000","last_active":"2024-04-25T11:10:28.537521+0000","last_peered":"2024-04-25T11:10:28.537521+0000","last_clean":"2024-04-25T11:10:28.537521+0000","last_became_active":"2024-04-25T11:10:23.491282+0000","last_became_peered":"2024-04-25T11:10:23.491282+0000","last_unstale":"2024-04-25T11:10:28.537521+0000","last_undegraded":"2024-04-25T11:10:28.537521+0000","last_fullsized":"2024-04-25T11:10:28.537521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:15.460249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.539367+0000","last_change":"2024-04-25T11:10:22.477761+0000","last_active":"2024-04-25T11:10:28.539367+0000","last_peered":"2024-04-25T11:10:28.539367+0000","last_clean":"2024-04-25T11:10:28.539367+0000","last_became_active":"2024-04-25T11:10:22.477390+0000","last_became_peered":"2024-04-25T11:10:22.477390+0000","last_unstale":"2024-04-25T11:10:28.539367+0000","last_undegraded":"2024-04-25T11:10:28.539367+0000","last_fullsized":"2024-04-25T11:10:28.539367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:48.408404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535808+0000","last_change":"2024-04-25T11:10:27.559907+0000","last_active":"2024-04-25T11:10:28.535808+0000","last_peered":"2024-04-25T11:10:28.535808+0000","last_clean":"2024-04-25T11:10:28.535808+0000","last_became_active":"2024-04-25T11:10:27.559437+0000","last_became_peered":"2024-04-25T11:10:27.559437+0000","last_unstale":"2024-04-25T11:10:28.535808+0000","last_undegraded":"2024-04-25T11:10:28.535808+0000","last_fullsized":"2024-04-25T11:10:28.535808+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.067687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535432+0000","last_change":"2024-04-25T11:10:23.479907+0000","last_active":"2024-04-25T11:10:28.535432+0000","last_peered":"2024-04-25T11:10:28.535432+0000","last_clean":"2024-04-25T11:10:28.535432+0000","last_became_active":"2024-04-25T11:10:23.479578+0000","last_became_peered":"2024-04-25T11:10:23.479578+0000","last_unstale":"2024-04-25T11:10:28.535432+0000","last_undegraded":"2024-04-25T11:10:28.535432+0000","last_fullsized":"2024-04-25T11:10:28.535432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373126+0000","last_change":"2024-04-25T11:10:22.467360+0000","last_active":"2024-04-25T11:10:29.373126+0000","last_peered":"2024-04-25T11:10:29.373126+0000","last_clean":"2024-04-25T11:10:29.373126+0000","last_became_active":"2024-04-25T11:10:22.467002+0000","last_became_peered":"2024-04-25T11:10:22.467002+0000","last_unstale":"2024-04-25T11:10:29.373126+0000","last_undegraded":"2024-04-25T11:10:29.373126+0000","last_fullsized":"2024-04-25T11:10:29.373126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:17.490319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894874+0000","last_change":"2024-04-25T11:10:27.566981+0000","last_active":"2024-04-25T11:10:29.894874+0000","last_peered":"2024-04-25T11:10:29.894874+0000","last_clean":"2024-04-25T11:10:29.894874+0000","last_became_active":"2024-04-25T11:10:27.566434+0000","last_became_peered":"2024-04-25T11:10:27.566434+0000","last_unstale":"2024-04-25T11:10:29.894874+0000","last_undegraded":"2024-04-25T11:10:29.894874+0000","last_fullsized":"2024-04-25T11:10:29.894874+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:55.043679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891408+0000","last_change":"2024-04-25T11:10:23.489031+0000","last_active":"2024-04-25T11:10:29.891408+0000","last_peered":"2024-04-25T11:10:29.891408+0000","last_clean":"2024-04-25T11:10:29.891408+0000","last_became_active":"2024-04-25T11:10:23.487804+0000","last_became_peered":"2024-04-25T11:10:23.487804+0000","last_unstale":"2024-04-25T11:10:29.891408+0000","last_undegraded":"2024-04-25T11:10:29.891408+0000","last_fullsized":"2024-04-25T11:10:29.891408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891229+0000","last_change":"2024-04-25T11:10:22.489707+0000","last_active":"2024-04-25T11:10:29.891229+0000","last_peered":"2024-04-25T11:10:29.891229+0000","last_clean":"2024-04-25T11:10:29.891229+0000","last_became_active":"2024-04-25T11:10:22.489324+0000","last_became_peered":"2024-04-25T11:10:22.489324+0000","last_unstale":"2024-04-25T11:10:29.891229+0000","last_undegraded":"2024-04-25T11:10:29.891229+0000","last_fullsized":"2024-04-25T11:10:29.891229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896355+0000","last_change":"2024-04-25T11:10:27.556359+0000","last_active":"2024-04-25T11:10:29.896355+0000","last_peered":"2024-04-25T11:10:29.896355+0000","last_clean":"2024-04-25T11:10:29.896355+0000","last_became_active":"2024-04-25T11:10:27.555528+0000","last_became_peered":"2024-04-25T11:10:27.555528+0000","last_unstale":"2024-04-25T11:10:29.896355+0000","last_undegraded":"2024-04-25T11:10:29.896355+0000","last_fullsized":"2024-04-25T11:10:29.896355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:35.346179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535152+0000","last_change":"2024-04-25T11:10:23.478210+0000","last_active":"2024-04-25T11:10:28.535152+0000","last_peered":"2024-04-25T11:10:28.535152+0000","last_clean":"2024-04-25T11:10:28.535152+0000","last_became_active":"2024-04-25T11:10:23.477839+0000","last_became_peered":"2024-04-25T11:10:23.477839+0000","last_unstale":"2024-04-25T11:10:28.535152+0000","last_undegraded":"2024-04-25T11:10:28.535152+0000","last_fullsized":"2024-04-25T11:10:28.535152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:41.745805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290930+0000","last_change":"2024-04-25T11:10:22.487737+0000","last_active":"2024-04-25T11:10:29.290930+0000","last_peered":"2024-04-25T11:10:29.290930+0000","last_clean":"2024-04-25T11:10:29.290930+0000","last_became_active":"2024-04-25T11:10:22.485337+0000","last_became_peered":"2024-04-25T11:10:22.485337+0000","last_unstale":"2024-04-25T11:10:29.290930+0000","last_undegraded":"2024-04-25T11:10:29.290930+0000","last_fullsized":"2024-04-25T11:10:29.290930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894290+0000","last_change":"2024-04-25T11:10:27.562988+0000","last_active":"2024-04-25T11:10:29.894290+0000","last_peered":"2024-04-25T11:10:29.894290+0000","last_clean":"2024-04-25T11:10:29.894290+0000","last_became_active":"2024-04-25T11:10:27.562511+0000","last_became_peered":"2024-04-25T11:10:27.562511+0000","last_unstale":"2024-04-25T11:10:29.894290+0000","last_undegraded":"2024-04-25T11:10:29.894290+0000","last_fullsized":"2024-04-25T11:10:29.894290+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:32.363124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893441+0000","last_change":"2024-04-25T11:10:23.480645+0000","last_active":"2024-04-25T11:10:29.893441+0000","last_peered":"2024-04-25T11:10:29.893441+0000","last_clean":"2024-04-25T11:10:29.893441+0000","last_became_active":"2024-04-25T11:10:23.480244+0000","last_became_peered":"2024-04-25T11:10:23.480244+0000","last_unstale":"2024-04-25T11:10:29.893441+0000","last_undegraded":"2024-04-25T11:10:29.893441+0000","last_fullsized":"2024-04-25T11:10:29.893441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:46.490227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894602+0000","last_change":"2024-04-25T11:10:22.479455+0000","last_active":"2024-04-25T11:10:29.894602+0000","last_peered":"2024-04-25T11:10:29.894602+0000","last_clean":"2024-04-25T11:10:29.894602+0000","last_became_active":"2024-04-25T11:10:22.479024+0000","last_became_peered":"2024-04-25T11:10:22.479024+0000","last_unstale":"2024-04-25T11:10:29.894602+0000","last_undegraded":"2024-04-25T11:10:29.894602+0000","last_fullsized":"2024-04-25T11:10:29.894602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:56.918083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291635+0000","last_change":"2024-04-25T11:10:27.549545+0000","last_active":"2024-04-25T11:10:29.291635+0000","last_peered":"2024-04-25T11:10:29.291635+0000","last_clean":"2024-04-25T11:10:29.291635+0000","last_became_active":"2024-04-25T11:10:27.549085+0000","last_became_peered":"2024-04-25T11:10:27.549085+0000","last_unstale":"2024-04-25T11:10:29.291635+0000","last_undegraded":"2024-04-25T11:10:29.291635+0000","last_fullsized":"2024-04-25T11:10:29.291635+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.906264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536502+0000","last_change":"2024-04-25T11:10:23.472355+0000","last_active":"2024-04-25T11:10:28.536502+0000","last_peered":"2024-04-25T11:10:28.536502+0000","last_clean":"2024-04-25T11:10:28.536502+0000","last_became_active":"2024-04-25T11:10:23.470905+0000","last_became_peered":"2024-04-25T11:10:23.470905+0000","last_unstale":"2024-04-25T11:10:28.536502+0000","last_undegraded":"2024-04-25T11:10:28.536502+0000","last_fullsized":"2024-04-25T11:10:28.536502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:20.100576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:44.462567+0000","last_change":"2024-04-25T11:10:22.481299+0000","last_active":"2024-04-25T11:10:44.462567+0000","last_peered":"2024-04-25T11:10:44.462567+0000","last_clean":"2024-04-25T11:10:44.462567+0000","last_became_active":"2024-04-25T11:10:22.480741+0000","last_became_peered":"2024-04-25T11:10:22.480741+0000","last_unstale":"2024-04-25T11:10:44.462567+0000","last_undegraded":"2024-04-25T11:10:44.462567+0000","last_fullsized":"2024-04-25T11:10:44.462567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533006+0000","last_change":"2024-04-25T11:10:27.551729+0000","last_active":"2024-04-25T11:10:28.533006+0000","last_peered":"2024-04-25T11:10:28.533006+0000","last_clean":"2024-04-25T11:10:28.533006+0000","last_became_active":"2024-04-25T11:10:27.548417+0000","last_became_peered":"2024-04-25T11:10:27.548417+0000","last_unstale":"2024-04-25T11:10:28.533006+0000","last_undegraded":"2024-04-25T11:10:28.533006+0000","last_fullsized":"2024-04-25T11:10:28.533006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538108+0000","last_change":"2024-04-25T11:10:23.482351+0000","last_active":"2024-04-25T11:10:28.538108+0000","last_peered":"2024-04-25T11:10:28.538108+0000","last_clean":"2024-04-25T11:10:28.538108+0000","last_became_active":"2024-04-25T11:10:23.481665+0000","last_became_peered":"2024-04-25T11:10:23.481665+0000","last_unstale":"2024-04-25T11:10:28.538108+0000","last_undegraded":"2024-04-25T11:10:28.538108+0000","last_fullsized":"2024-04-25T11:10:28.538108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:50.877149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533327+0000","last_change":"2024-04-25T11:10:22.484130+0000","last_active":"2024-04-25T11:10:28.533327+0000","last_peered":"2024-04-25T11:10:28.533327+0000","last_clean":"2024-04-25T11:10:28.533327+0000","last_became_active":"2024-04-25T11:10:22.483811+0000","last_became_peered":"2024-04-25T11:10:22.483811+0000","last_unstale":"2024-04-25T11:10:28.533327+0000","last_undegraded":"2024-04-25T11:10:28.533327+0000","last_fullsized":"2024-04-25T11:10:28.533327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535359+0000","last_change":"2024-04-25T11:10:27.564379+0000","last_active":"2024-04-25T11:10:28.535359+0000","last_peered":"2024-04-25T11:10:28.535359+0000","last_clean":"2024-04-25T11:10:28.535359+0000","last_became_active":"2024-04-25T11:10:27.563892+0000","last_became_peered":"2024-04-25T11:10:27.563892+0000","last_unstale":"2024-04-25T11:10:28.535359+0000","last_undegraded":"2024-04-25T11:10:28.535359+0000","last_fullsized":"2024-04-25T11:10:28.535359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:06.999062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895182+0000","last_change":"2024-04-25T11:10:23.487092+0000","last_active":"2024-04-25T11:10:29.895182+0000","last_peered":"2024-04-25T11:10:29.895182+0000","last_clean":"2024-04-25T11:10:29.895182+0000","last_became_active":"2024-04-25T11:10:23.486583+0000","last_became_peered":"2024-04-25T11:10:23.486583+0000","last_unstale":"2024-04-25T11:10:29.895182+0000","last_undegraded":"2024-04-25T11:10:29.895182+0000","last_fullsized":"2024-04-25T11:10:29.895182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291509+0000","last_change":"2024-04-25T11:10:22.484333+0000","last_active":"2024-04-25T11:10:29.291509+0000","last_peered":"2024-04-25T11:10:29.291509+0000","last_clean":"2024-04-25T11:10:29.291509+0000","last_became_active":"2024-04-25T11:10:22.483865+0000","last_became_peered":"2024-04-25T11:10:22.483865+0000","last_unstale":"2024-04-25T11:10:29.291509+0000","last_undegraded":"2024-04-25T11:10:29.291509+0000","last_fullsized":"2024-04-25T11:10:29.291509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891783+0000","last_change":"2024-04-25T11:10:27.551066+0000","last_active":"2024-04-25T11:10:29.891783+0000","last_peered":"2024-04-25T11:10:29.891783+0000","last_clean":"2024-04-25T11:10:29.891783+0000","last_became_active":"2024-04-25T11:10:27.550625+0000","last_became_peered":"2024-04-25T11:10:27.550625+0000","last_unstale":"2024-04-25T11:10:29.891783+0000","last_undegraded":"2024-04-25T11:10:29.891783+0000","last_fullsized":"2024-04-25T11:10:29.891783+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.906264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291447+0000","last_change":"2024-04-25T11:10:23.485935+0000","last_active":"2024-04-25T11:10:29.291447+0000","last_peered":"2024-04-25T11:10:29.291447+0000","last_clean":"2024-04-25T11:10:29.291447+0000","last_became_active":"2024-04-25T11:10:23.485367+0000","last_became_peered":"2024-04-25T11:10:23.485367+0000","last_unstale":"2024-04-25T11:10:29.291447+0000","last_undegraded":"2024-04-25T11:10:29.291447+0000","last_fullsized":"2024-04-25T11:10:29.291447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:50.974212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892031+0000","last_change":"2024-04-25T11:10:22.482686+0000","last_active":"2024-04-25T11:10:29.892031+0000","last_peered":"2024-04-25T11:10:29.892031+0000","last_clean":"2024-04-25T11:10:29.892031+0000","last_became_active":"2024-04-25T11:10:22.482307+0000","last_became_peered":"2024-04-25T11:10:22.482307+0000","last_unstale":"2024-04-25T11:10:29.892031+0000","last_undegraded":"2024-04-25T11:10:29.892031+0000","last_fullsized":"2024-04-25T11:10:29.892031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892597+0000","last_change":"2024-04-25T11:10:27.563942+0000","last_active":"2024-04-25T11:10:29.892597+0000","last_peered":"2024-04-25T11:10:29.892597+0000","last_clean":"2024-04-25T11:10:29.892597+0000","last_became_active":"2024-04-25T11:10:27.561604+0000","last_became_peered":"2024-04-25T11:10:27.561604+0000","last_unstale":"2024-04-25T11:10:29.892597+0000","last_undegraded":"2024-04-25T11:10:29.892597+0000","last_fullsized":"2024-04-25T11:10:29.892597+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:59.323947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893904+0000","last_change":"2024-04-25T11:10:27.559342+0000","last_active":"2024-04-25T11:10:29.893904+0000","last_peered":"2024-04-25T11:10:29.893904+0000","last_clean":"2024-04-25T11:10:29.893904+0000","last_became_active":"2024-04-25T11:10:27.558417+0000","last_became_peered":"2024-04-25T11:10:27.558417+0000","last_unstale":"2024-04-25T11:10:29.893904+0000","last_undegraded":"2024-04-25T11:10:29.893904+0000","last_fullsized":"2024-04-25T11:10:29.893904+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.970801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531482+0000","last_change":"2024-04-25T11:10:23.472501+0000","last_active":"2024-04-25T11:10:28.531482+0000","last_peered":"2024-04-25T11:10:28.531482+0000","last_clean":"2024-04-25T11:10:28.531482+0000","last_became_active":"2024-04-25T11:10:23.472103+0000","last_became_peered":"2024-04-25T11:10:23.472103+0000","last_unstale":"2024-04-25T11:10:28.531482+0000","last_undegraded":"2024-04-25T11:10:28.531482+0000","last_fullsized":"2024-04-25T11:10:28.531482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290643+0000","last_change":"2024-04-25T11:10:22.487396+0000","last_active":"2024-04-25T11:10:29.290643+0000","last_peered":"2024-04-25T11:10:29.290643+0000","last_clean":"2024-04-25T11:10:29.290643+0000","last_became_active":"2024-04-25T11:10:22.484146+0000","last_became_peered":"2024-04-25T11:10:22.484146+0000","last_unstale":"2024-04-25T11:10:29.290643+0000","last_undegraded":"2024-04-25T11:10:29.290643+0000","last_fullsized":"2024-04-25T11:10:29.290643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891822+0000","last_change":"2024-04-25T11:10:27.545602+0000","last_active":"2024-04-25T11:10:29.891822+0000","last_peered":"2024-04-25T11:10:29.891822+0000","last_clean":"2024-04-25T11:10:29.891822+0000","last_became_active":"2024-04-25T11:10:27.545089+0000","last_became_peered":"2024-04-25T11:10:27.545089+0000","last_unstale":"2024-04-25T11:10:29.891822+0000","last_undegraded":"2024-04-25T11:10:29.891822+0000","last_fullsized":"2024-04-25T11:10:29.891822+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:24.170044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891897+0000","last_change":"2024-04-25T11:10:23.480332+0000","last_active":"2024-04-25T11:10:29.891897+0000","last_peered":"2024-04-25T11:10:29.891897+0000","last_clean":"2024-04-25T11:10:29.891897+0000","last_became_active":"2024-04-25T11:10:23.479866+0000","last_became_peered":"2024-04-25T11:10:23.479866+0000","last_unstale":"2024-04-25T11:10:29.891897+0000","last_undegraded":"2024-04-25T11:10:29.891897+0000","last_fullsized":"2024-04-25T11:10:29.891897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893193+0000","last_change":"2024-04-25T11:10:22.476980+0000","last_active":"2024-04-25T11:10:29.893193+0000","last_peered":"2024-04-25T11:10:29.893193+0000","last_clean":"2024-04-25T11:10:29.893193+0000","last_became_active":"2024-04-25T11:10:22.476483+0000","last_became_peered":"2024-04-25T11:10:22.476483+0000","last_unstale":"2024-04-25T11:10:29.893193+0000","last_undegraded":"2024-04-25T11:10:29.893193+0000","last_fullsized":"2024-04-25T11:10:29.893193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:56.918083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891422+0000","last_change":"2024-04-25T11:10:22.485058+0000","last_active":"2024-04-25T11:10:29.891422+0000","last_peered":"2024-04-25T11:10:29.891422+0000","last_clean":"2024-04-25T11:10:29.891422+0000","last_became_active":"2024-04-25T11:10:22.484647+0000","last_became_peered":"2024-04-25T11:10:22.484647+0000","last_unstale":"2024-04-25T11:10:29.891422+0000","last_undegraded":"2024-04-25T11:10:29.891422+0000","last_fullsized":"2024-04-25T11:10:29.891422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:42.000742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894489+0000","last_change":"2024-04-25T11:10:23.494632+0000","last_active":"2024-04-25T11:10:29.894489+0000","last_peered":"2024-04-25T11:10:29.894489+0000","last_clean":"2024-04-25T11:10:29.894489+0000","last_became_active":"2024-04-25T11:10:23.494214+0000","last_became_peered":"2024-04-25T11:10:23.494214+0000","last_unstale":"2024-04-25T11:10:29.894489+0000","last_undegraded":"2024-04-25T11:10:29.894489+0000","last_fullsized":"2024-04-25T11:10:29.894489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373551+0000","last_change":"2024-04-25T11:10:27.546702+0000","last_active":"2024-04-25T11:10:29.373551+0000","last_peered":"2024-04-25T11:10:29.373551+0000","last_clean":"2024-04-25T11:10:29.373551+0000","last_became_active":"2024-04-25T11:10:27.546093+0000","last_became_peered":"2024-04-25T11:10:27.546093+0000","last_unstale":"2024-04-25T11:10:29.373551+0000","last_undegraded":"2024-04-25T11:10:29.373551+0000","last_fullsized":"2024-04-25T11:10:29.373551+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:55.973871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892460+0000","last_change":"2024-04-25T11:10:22.470279+0000","last_active":"2024-04-25T11:10:29.892460+0000","last_peered":"2024-04-25T11:10:29.892460+0000","last_clean":"2024-04-25T11:10:29.892460+0000","last_became_active":"2024-04-25T11:10:22.469133+0000","last_became_peered":"2024-04-25T11:10:22.469133+0000","last_unstale":"2024-04-25T11:10:29.892460+0000","last_undegraded":"2024-04-25T11:10:29.892460+0000","last_fullsized":"2024-04-25T11:10:29.892460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.835819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536180+0000","last_change":"2024-04-25T11:10:23.482007+0000","last_active":"2024-04-25T11:10:28.536180+0000","last_peered":"2024-04-25T11:10:28.536180+0000","last_clean":"2024-04-25T11:10:28.536180+0000","last_became_active":"2024-04-25T11:10:23.475539+0000","last_became_peered":"2024-04-25T11:10:23.475539+0000","last_unstale":"2024-04-25T11:10:28.536180+0000","last_undegraded":"2024-04-25T11:10:28.536180+0000","last_fullsized":"2024-04-25T11:10:28.536180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:56.556443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895038+0000","last_change":"2024-04-25T11:10:27.555814+0000","last_active":"2024-04-25T11:10:29.895038+0000","last_peered":"2024-04-25T11:10:29.895038+0000","last_clean":"2024-04-25T11:10:29.895038+0000","last_became_active":"2024-04-25T11:10:27.554994+0000","last_became_peered":"2024-04-25T11:10:27.554994+0000","last_unstale":"2024-04-25T11:10:29.895038+0000","last_undegraded":"2024-04-25T11:10:29.895038+0000","last_fullsized":"2024-04-25T11:10:29.895038+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:11.895073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.890634+0000","last_change":"2024-04-25T11:10:23.487735+0000","last_active":"2024-04-25T11:10:29.890634+0000","last_peered":"2024-04-25T11:10:29.890634+0000","last_clean":"2024-04-25T11:10:29.890634+0000","last_became_active":"2024-04-25T11:10:23.487354+0000","last_became_peered":"2024-04-25T11:10:23.487354+0000","last_unstale":"2024-04-25T11:10:29.890634+0000","last_undegraded":"2024-04-25T11:10:29.890634+0000","last_fullsized":"2024-04-25T11:10:29.890634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373170+0000","last_change":"2024-04-25T11:10:23.476515+0000","last_active":"2024-04-25T11:10:29.373170+0000","last_peered":"2024-04-25T11:10:29.373170+0000","last_clean":"2024-04-25T11:10:29.373170+0000","last_became_active":"2024-04-25T11:10:23.476183+0000","last_became_peered":"2024-04-25T11:10:23.476183+0000","last_unstale":"2024-04-25T11:10:29.373170+0000","last_undegraded":"2024-04-25T11:10:29.373170+0000","last_fullsized":"2024-04-25T11:10:29.373170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:57.829487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290735+0000","last_change":"2024-04-25T11:10:23.477846+0000","last_active":"2024-04-25T11:10:29.290735+0000","last_peered":"2024-04-25T11:10:29.290735+0000","last_clean":"2024-04-25T11:10:29.290735+0000","last_became_active":"2024-04-25T11:10:23.476420+0000","last_became_peered":"2024-04-25T11:10:23.476420+0000","last_unstale":"2024-04-25T11:10:29.290735+0000","last_undegraded":"2024-04-25T11:10:29.290735+0000","last_fullsized":"2024-04-25T11:10:29.290735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894372+0000","last_change":"2024-04-25T11:10:23.485446+0000","last_active":"2024-04-25T11:10:29.894372+0000","last_peered":"2024-04-25T11:10:29.894372+0000","last_clean":"2024-04-25T11:10:29.894372+0000","last_became_active":"2024-04-25T11:10:23.485095+0000","last_became_peered":"2024-04-25T11:10:23.485095+0000","last_unstale":"2024-04-25T11:10:29.894372+0000","last_undegraded":"2024-04-25T11:10:29.894372+0000","last_fullsized":"2024-04-25T11:10:29.894372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895256+0000","last_change":"2024-04-25T11:10:23.490644+0000","last_active":"2024-04-25T11:10:29.895256+0000","last_peered":"2024-04-25T11:10:29.895256+0000","last_clean":"2024-04-25T11:10:29.895256+0000","last_became_active":"2024-04-25T11:10:23.487911+0000","last_became_peered":"2024-04-25T11:10:23.487911+0000","last_unstale":"2024-04-25T11:10:29.895256+0000","last_undegraded":"2024-04-25T11:10:29.895256+0000","last_fullsized":"2024-04-25T11:10:29.895256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:48.595548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.532685+0000","last_change":"2024-04-25T11:10:23.484155+0000","last_active":"2024-04-25T11:10:28.532685+0000","last_peered":"2024-04-25T11:10:28.532685+0000","last_clean":"2024-04-25T11:10:28.532685+0000","last_became_active":"2024-04-25T11:10:23.483906+0000","last_became_peered":"2024-04-25T11:10:23.483906+0000","last_unstale":"2024-04-25T11:10:28.532685+0000","last_undegraded":"2024-04-25T11:10:28.532685+0000","last_fullsized":"2024-04-25T11:10:28.532685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892230+0000","last_change":"2024-04-25T11:10:23.479315+0000","last_active":"2024-04-25T11:10:29.892230+0000","last_peered":"2024-04-25T11:10:29.892230+0000","last_clean":"2024-04-25T11:10:29.892230+0000","last_became_active":"2024-04-25T11:10:23.478986+0000","last_became_peered":"2024-04-25T11:10:23.478986+0000","last_unstale":"2024-04-25T11:10:29.892230+0000","last_undegraded":"2024-04-25T11:10:29.892230+0000","last_fullsized":"2024-04-25T11:10:29.892230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896160+0000","last_change":"2024-04-25T11:10:23.475790+0000","last_active":"2024-04-25T11:10:29.896160+0000","last_peered":"2024-04-25T11:10:29.896160+0000","last_clean":"2024-04-25T11:10:29.896160+0000","last_became_active":"2024-04-25T11:10:23.475049+0000","last_became_peered":"2024-04-25T11:10:23.475049+0000","last_unstale":"2024-04-25T11:10:29.896160+0000","last_undegraded":"2024-04-25T11:10:29.896160+0000","last_fullsized":"2024-04-25T11:10:29.896160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290626+0000","last_change":"2024-04-25T11:10:23.489591+0000","last_active":"2024-04-25T11:10:29.290626+0000","last_peered":"2024-04-25T11:10:29.290626+0000","last_clean":"2024-04-25T11:10:29.290626+0000","last_became_active":"2024-04-25T11:10:23.488935+0000","last_became_peered":"2024-04-25T11:10:23.488935+0000","last_unstale":"2024-04-25T11:10:29.290626+0000","last_undegraded":"2024-04-25T11:10:29.290626+0000","last_fullsized":"2024-04-25T11:10:29.290626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892914+0000","last_change":"2024-04-25T11:10:23.487850+0000","last_active":"2024-04-25T11:10:29.892914+0000","last_peered":"2024-04-25T11:10:29.892914+0000","last_clean":"2024-04-25T11:10:29.892914+0000","last_became_active":"2024-04-25T11:10:23.487234+0000","last_became_peered":"2024-04-25T11:10:23.487234+0000","last_unstale":"2024-04-25T11:10:29.892914+0000","last_undegraded":"2024-04-25T11:10:29.892914+0000","last_fullsized":"2024-04-25T11:10:29.892914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290514+0000","last_change":"2024-04-25T11:10:23.473547+0000","last_active":"2024-04-25T11:10:29.290514+0000","last_peered":"2024-04-25T11:10:29.290514+0000","last_clean":"2024-04-25T11:10:29.290514+0000","last_became_active":"2024-04-25T11:10:23.467594+0000","last_became_peered":"2024-04-25T11:10:23.467594+0000","last_unstale":"2024-04-25T11:10:29.290514+0000","last_undegraded":"2024-04-25T11:10:29.290514+0000","last_fullsized":"2024-04-25T11:10:29.290514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:48.595548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893781+0000","last_change":"2024-04-25T11:10:23.480740+0000","last_active":"2024-04-25T11:10:29.893781+0000","last_peered":"2024-04-25T11:10:29.893781+0000","last_clean":"2024-04-25T11:10:29.893781+0000","last_became_active":"2024-04-25T11:10:23.480405+0000","last_became_peered":"2024-04-25T11:10:23.480405+0000","last_unstale":"2024-04-25T11:10:29.893781+0000","last_undegraded":"2024-04-25T11:10:29.893781+0000","last_fullsized":"2024-04-25T11:10:29.893781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891342+0000","last_change":"2024-04-25T11:10:23.497261+0000","last_active":"2024-04-25T11:10:29.891342+0000","last_peered":"2024-04-25T11:10:29.891342+0000","last_clean":"2024-04-25T11:10:29.891342+0000","last_became_active":"2024-04-25T11:10:23.493494+0000","last_became_peered":"2024-04-25T11:10:23.493494+0000","last_unstale":"2024-04-25T11:10:29.891342+0000","last_undegraded":"2024-04-25T11:10:29.891342+0000","last_fullsized":"2024-04-25T11:10:29.891342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537112+0000","last_change":"2024-04-25T11:10:23.469676+0000","last_active":"2024-04-25T11:10:28.537112+0000","last_peered":"2024-04-25T11:10:28.537112+0000","last_clean":"2024-04-25T11:10:28.537112+0000","last_became_active":"2024-04-25T11:10:23.468887+0000","last_became_peered":"2024-04-25T11:10:23.468887+0000","last_unstale":"2024-04-25T11:10:28.537112+0000","last_undegraded":"2024-04-25T11:10:28.537112+0000","last_fullsized":"2024-04-25T11:10:28.537112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:47.104859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.374373+0000","last_change":"2024-04-25T11:10:23.491992+0000","last_active":"2024-04-25T11:10:29.374373+0000","last_peered":"2024-04-25T11:10:29.374373+0000","last_clean":"2024-04-25T11:10:29.374373+0000","last_became_active":"2024-04-25T11:10:23.491456+0000","last_became_peered":"2024-04-25T11:10:23.491456+0000","last_unstale":"2024-04-25T11:10:29.374373+0000","last_undegraded":"2024-04-25T11:10:29.374373+0000","last_fullsized":"2024-04-25T11:10:29.374373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.097249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.292111+0000","last_change":"2024-04-25T11:10:23.469823+0000","last_active":"2024-04-25T11:10:29.292111+0000","last_peered":"2024-04-25T11:10:29.292111+0000","last_clean":"2024-04-25T11:10:29.292111+0000","last_became_active":"2024-04-25T11:10:23.469420+0000","last_became_peered":"2024-04-25T11:10:23.469420+0000","last_unstale":"2024-04-25T11:10:29.292111+0000","last_undegraded":"2024-04-25T11:10:29.292111+0000","last_fullsized":"2024-04-25T11:10:29.292111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290180+0000","last_change":"2024-04-25T11:10:23.479442+0000","last_active":"2024-04-25T11:10:29.290180+0000","last_peered":"2024-04-25T11:10:29.290180+0000","last_clean":"2024-04-25T11:10:29.290180+0000","last_became_active":"2024-04-25T11:10:23.472659+0000","last_became_peered":"2024-04-25T11:10:23.472659+0000","last_unstale":"2024-04-25T11:10:29.290180+0000","last_undegraded":"2024-04-25T11:10:29.290180+0000","last_fullsized":"2024-04-25T11:10:29.290180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531629+0000","last_change":"2024-04-25T11:10:23.490775+0000","last_active":"2024-04-25T11:10:28.531629+0000","last_peered":"2024-04-25T11:10:28.531629+0000","last_clean":"2024-04-25T11:10:28.531629+0000","last_became_active":"2024-04-25T11:10:23.490138+0000","last_became_peered":"2024-04-25T11:10:23.490138+0000","last_unstale":"2024-04-25T11:10:28.531629+0000","last_undegraded":"2024-04-25T11:10:28.531629+0000","last_fullsized":"2024-04-25T11:10:28.531629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893293+0000","last_change":"2024-04-25T11:10:23.479882+0000","last_active":"2024-04-25T11:10:29.893293+0000","last_peered":"2024-04-25T11:10:29.893293+0000","last_clean":"2024-04-25T11:10:29.893293+0000","last_became_active":"2024-04-25T11:10:23.479534+0000","last_became_peered":"2024-04-25T11:10:23.479534+0000","last_unstale":"2024-04-25T11:10:29.893293+0000","last_undegraded":"2024-04-25T11:10:29.893293+0000","last_fullsized":"2024-04-25T11:10:29.893293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:04.105145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892215+0000","last_change":"2024-04-25T11:10:23.483845+0000","last_active":"2024-04-25T11:10:29.892215+0000","last_peered":"2024-04-25T11:10:29.892215+0000","last_clean":"2024-04-25T11:10:29.892215+0000","last_became_active":"2024-04-25T11:10:23.482401+0000","last_became_peered":"2024-04-25T11:10:23.482401+0000","last_unstale":"2024-04-25T11:10:29.892215+0000","last_undegraded":"2024-04-25T11:10:29.892215+0000","last_fullsized":"2024-04-25T11:10:29.892215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537880+0000","last_change":"2024-04-25T11:10:23.490750+0000","last_active":"2024-04-25T11:10:28.537880+0000","last_peered":"2024-04-25T11:10:28.537880+0000","last_clean":"2024-04-25T11:10:28.537880+0000","last_became_active":"2024-04-25T11:10:23.490489+0000","last_became_peered":"2024-04-25T11:10:23.490489+0000","last_unstale":"2024-04-25T11:10:28.537880+0000","last_undegraded":"2024-04-25T11:10:28.537880+0000","last_fullsized":"2024-04-25T11:10:28.537880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:30.205583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891988+0000","last_change":"2024-04-25T11:10:23.498316+0000","last_active":"2024-04-25T11:10:29.891988+0000","last_peered":"2024-04-25T11:10:29.891988+0000","last_clean":"2024-04-25T11:10:29.891988+0000","last_became_active":"2024-04-25T11:10:23.497414+0000","last_became_peered":"2024-04-25T11:10:23.497414+0000","last_unstale":"2024-04-25T11:10:29.891988+0000","last_undegraded":"2024-04-25T11:10:29.891988+0000","last_fullsized":"2024-04-25T11:10:29.891988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892093+0000","last_change":"2024-04-25T11:10:23.487588+0000","last_active":"2024-04-25T11:10:29.892093+0000","last_peered":"2024-04-25T11:10:29.892093+0000","last_clean":"2024-04-25T11:10:29.892093+0000","last_became_active":"2024-04-25T11:10:23.487280+0000","last_became_peered":"2024-04-25T11:10:23.487280+0000","last_unstale":"2024-04-25T11:10:29.892093+0000","last_undegraded":"2024-04-25T11:10:29.892093+0000","last_fullsized":"2024-04-25T11:10:29.892093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538694+0000","last_change":"2024-04-25T11:10:23.472328+0000","last_active":"2024-04-25T11:10:28.538694+0000","last_peered":"2024-04-25T11:10:28.538694+0000","last_clean":"2024-04-25T11:10:28.538694+0000","last_became_active":"2024-04-25T11:10:23.471692+0000","last_became_peered":"2024-04-25T11:10:23.471692+0000","last_unstale":"2024-04-25T11:10:28.538694+0000","last_undegraded":"2024-04-25T11:10:28.538694+0000","last_fullsized":"2024-04-25T11:10:28.538694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:50.974212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891640+0000","last_change":"2024-04-25T11:10:23.471703+0000","last_active":"2024-04-25T11:10:29.891640+0000","last_peered":"2024-04-25T11:10:29.891640+0000","last_clean":"2024-04-25T11:10:29.891640+0000","last_became_active":"2024-04-25T11:10:23.469570+0000","last_became_peered":"2024-04-25T11:10:23.469570+0000","last_unstale":"2024-04-25T11:10:29.891640+0000","last_undegraded":"2024-04-25T11:10:29.891640+0000","last_fullsized":"2024-04-25T11:10:29.891640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535108+0000","last_change":"2024-04-25T11:10:23.473106+0000","last_active":"2024-04-25T11:10:28.535108+0000","last_peered":"2024-04-25T11:10:28.535108+0000","last_clean":"2024-04-25T11:10:28.535108+0000","last_became_active":"2024-04-25T11:10:23.472627+0000","last_became_peered":"2024-04-25T11:10:23.472627+0000","last_unstale":"2024-04-25T11:10:28.535108+0000","last_undegraded":"2024-04-25T11:10:28.535108+0000","last_fullsized":"2024-04-25T11:10:28.535108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290419+0000","last_change":"2024-04-25T11:10:23.480283+0000","last_active":"2024-04-25T11:10:29.290419+0000","last_peered":"2024-04-25T11:10:29.290419+0000","last_clean":"2024-04-25T11:10:29.290419+0000","last_became_active":"2024-04-25T11:10:23.479889+0000","last_became_peered":"2024-04-25T11:10:23.479889+0000","last_unstale":"2024-04-25T11:10:29.290419+0000","last_undegraded":"2024-04-25T11:10:29.290419+0000","last_fullsized":"2024-04-25T11:10:29.290419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534607+0000","last_change":"2024-04-25T11:10:23.469104+0000","last_active":"2024-04-25T11:10:28.534607+0000","last_peered":"2024-04-25T11:10:28.534607+0000","last_clean":"2024-04-25T11:10:28.534607+0000","last_became_active":"2024-04-25T11:10:23.468743+0000","last_became_peered":"2024-04-25T11:10:23.468743+0000","last_unstale":"2024-04-25T11:10:28.534607+0000","last_undegraded":"2024-04-25T11:10:28.534607+0000","last_fullsized":"2024-04-25T11:10:28.534607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291058+0000","last_change":"2024-04-25T11:10:23.496747+0000","last_active":"2024-04-25T11:10:29.291058+0000","last_peered":"2024-04-25T11:10:29.291058+0000","last_clean":"2024-04-25T11:10:29.291058+0000","last_became_active":"2024-04-25T11:10:23.496354+0000","last_became_peered":"2024-04-25T11:10:23.496354+0000","last_unstale":"2024-04-25T11:10:29.291058+0000","last_undegraded":"2024-04-25T11:10:29.291058+0000","last_fullsized":"2024-04-25T11:10:29.291058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:41.745805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893849+0000","last_change":"2024-04-25T11:10:23.480571+0000","last_active":"2024-04-25T11:10:29.893849+0000","last_peered":"2024-04-25T11:10:29.893849+0000","last_clean":"2024-04-25T11:10:29.893849+0000","last_became_active":"2024-04-25T11:10:23.479502+0000","last_became_peered":"2024-04-25T11:10:23.479502+0000","last_unstale":"2024-04-25T11:10:29.893849+0000","last_undegraded":"2024-04-25T11:10:29.893849+0000","last_fullsized":"2024-04-25T11:10:29.893849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533940+0000","last_change":"2024-04-25T11:10:23.475376+0000","last_active":"2024-04-25T11:10:28.533940+0000","last_peered":"2024-04-25T11:10:28.533940+0000","last_clean":"2024-04-25T11:10:28.533940+0000","last_became_active":"2024-04-25T11:10:23.474750+0000","last_became_peered":"2024-04-25T11:10:23.474750+0000","last_unstale":"2024-04-25T11:10:28.533940+0000","last_undegraded":"2024-04-25T11:10:28.533940+0000","last_fullsized":"2024-04-25T11:10:28.533940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538538+0000","last_change":"2024-04-25T11:10:23.481991+0000","last_active":"2024-04-25T11:10:28.538538+0000","last_peered":"2024-04-25T11:10:28.538538+0000","last_clean":"2024-04-25T11:10:28.538538+0000","last_became_active":"2024-04-25T11:10:23.481639+0000","last_became_peered":"2024-04-25T11:10:23.481639+0000","last_unstale":"2024-04-25T11:10:28.538538+0000","last_undegraded":"2024-04-25T11:10:28.538538+0000","last_fullsized":"2024-04-25T11:10:28.538538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:13.333031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476745,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476746,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476745,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476746,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":104646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542155,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:10:56.246 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:10:56.246 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T11:10:56.246 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T11:10:56.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T11:10:56.247 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T11:10:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.561+0000 7f5600c00640 1 Processor -- start 2024-04-25T11:10:56.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.561+0000 7f5600c00640 1 -- start start 2024-04-25T11:10:56.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.563+0000 7f5600c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f55fc0aed40 0x7f55fc15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.563+0000 7f5600c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55fc0b5220 0x7f55fc0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.563+0000 7f5600c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55fc0b64b0 con 0x7f55fc0af420 2024-04-25T11:10:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.563+0000 7f5600c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f55fc15e1f0 con 0x7f55fc0aed40 2024-04-25T11:10:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.563+0000 7f5600c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f55fc06c610 con 0x7f55fc0b5220 2024-04-25T11:10:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.564+0000 7f55fbe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f55fc0af420 0x7f55fc0af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39212/0 (socket says 172.21.15.7:39212) 2024-04-25T11:10:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.564+0000 7f55fbe00640 1 -- 172.21.15.7:0/4231241431 learned_addr learned my addr 172.21.15.7:0/4231241431 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:56.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.564+0000 7f55fb400640 1 --2- 172.21.15.7:0/4231241431 >> v2:172.21.15.87:3300/0 conn(0x7f55fc0aed40 0x7f55fc15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.564+0000 7f55f2a00640 1 --2- 172.21.15.7:0/4231241431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55fc0b5220 0x7f55fc0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.565+0000 7f55faa00640 1 -- 172.21.15.7:0/4231241431 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 821713757 0 0) 0x7f55fc0b64b0 con 0x7f55fc0af420 2024-04-25T11:10:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.565+0000 7f55faa00640 1 -- 172.21.15.7:0/4231241431 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55e0003520 con 0x7f55fc0af420 2024-04-25T11:10:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.565+0000 7f55f2a00640 1 -- 172.21.15.7:0/4231241431 >> v2:172.21.15.87:3300/0 conn(0x7f55fc0aed40 msgr2=0x7f55fc15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.565+0000 7f55f2a00640 1 --2- 172.21.15.7:0/4231241431 >> v2:172.21.15.87:3300/0 conn(0x7f55fc0aed40 0x7f55fc15dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.565+0000 7f55f2a00640 1 -- 172.21.15.7:0/4231241431 >> v1:172.21.15.87:6789/0 conn(0x7f55fc0af420 legacy=0x7f55fc0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.565+0000 7f55f2a00640 1 -- 172.21.15.7:0/4231241431 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc06c8f0 con 0x7f55fc0b5220 2024-04-25T11:10:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.567+0000 7f55f2a00640 1 --2- 172.21.15.7:0/4231241431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55fc0b5220 0x7f55fc0ce4a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f55fc15e370 tx=0x7f55e402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db42fb66a805d22c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.567+0000 7f55faa00640 1 -- 172.21.15.7:0/4231241431 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4037070 con 0x7f55fc0b5220 2024-04-25T11:10:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.567+0000 7f55faa00640 1 -- 172.21.15.7:0/4231241431 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e4032440 con 0x7f55fc0b5220 2024-04-25T11:10:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.567+0000 7f55faa00640 1 -- 172.21.15.7:0/4231241431 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e40327f0 con 0x7f55fc0b5220 2024-04-25T11:10:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.568+0000 7f5600c00640 1 -- 172.21.15.7:0/4231241431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55fc0b5220 msgr2=0x7f55fc0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.568+0000 7f5600c00640 1 --2- 172.21.15.7:0/4231241431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55fc0b5220 0x7f55fc0ce4a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f55fc15e370 tx=0x7f55e402a180 comp rx=0 tx=0).stop 2024-04-25T11:10:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.568+0000 7f5600c00640 1 -- 172.21.15.7:0/4231241431 shutdown_connections 2024-04-25T11:10:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.568+0000 7f5600c00640 1 --2- 172.21.15.7:0/4231241431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55fc0b5220 0x7f55fc0ce4a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.568+0000 7f5600c00640 1 --2- 172.21.15.7:0/4231241431 >> v2:172.21.15.87:3300/0 conn(0x7f55fc0aed40 0x7f55fc15dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.568+0000 7f5600c00640 1 -- 172.21.15.7:0/4231241431 >> 172.21.15.7:0/4231241431 conn(0x7f55fc0b3820 msgr2=0x7f55fc0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:56.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.568+0000 7f5600c00640 1 -- 172.21.15.7:0/4231241431 shutdown_connections 2024-04-25T11:10:56.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.568+0000 7f5600c00640 1 -- 172.21.15.7:0/4231241431 wait complete. 2024-04-25T11:10:56.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.570+0000 7f5600c00640 1 Processor -- start 2024-04-25T11:10:56.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.571+0000 7f5600c00640 1 -- start start 2024-04-25T11:10:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.572+0000 7f5600c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55fc0aed40 0x7f55fc1f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.572+0000 7f5600c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55fc0af420 0x7f55fc1f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.572+0000 7f5600c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55fc0b5220 0x7f55fc1f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.572+0000 7f5600c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc0b6110 con 0x7f55fc0b5220 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.572+0000 7f5600c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f55fc06ceb0 con 0x7f55fc0af420 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.572+0000 7f5600c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f55fc0b6290 con 0x7f55fc0aed40 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.572+0000 7f55fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55fc0b5220 0x7f55fc1f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.572+0000 7f55fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55fc0b5220 0x7f55fc1f9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48700/0 (socket says 172.21.15.7:48700) 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.572+0000 7f55fbe00640 1 -- 172.21.15.7:0/1526615804 learned_addr learned my addr 172.21.15.7:0/1526615804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.573+0000 7f55f2a00640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55fc0af420 0x7f55fc1f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.573+0000 7f55fb400640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55fc0aed40 0x7f55fc1f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.573+0000 7f55fbe00640 1 -- 172.21.15.7:0/1526615804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55fc0aed40 msgr2=0x7f55fc1f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.573+0000 7f55fbe00640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55fc0aed40 0x7f55fc1f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.573+0000 7f55fbe00640 1 -- 172.21.15.7:0/1526615804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55fc0af420 msgr2=0x7f55fc1f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.573+0000 7f55fbe00640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55fc0af420 0x7f55fc1f9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.573+0000 7f55fbe00640 1 -- 172.21.15.7:0/1526615804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55fc06fff0 con 0x7f55fc0b5220 2024-04-25T11:10:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.573+0000 7f55fbe00640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55fc0b5220 0x7f55fc1f9e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f55f4001eb0 tx=0x7f55f400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.573+0000 7f55f3e00640 1 -- 172.21.15.7:0/1526615804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4017070 con 0x7f55fc0b5220 2024-04-25T11:10:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.574+0000 7f55f3e00640 1 -- 172.21.15.7:0/1526615804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f4005ce0 con 0x7f55fc0b5220 2024-04-25T11:10:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.574+0000 7f55f3e00640 1 -- 172.21.15.7:0/1526615804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f40072d0 con 0x7f55fc0b5220 2024-04-25T11:10:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.574+0000 7f5600c00640 1 -- 172.21.15.7:0/1526615804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55fc0702d0 con 0x7f55fc0b5220 2024-04-25T11:10:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.574+0000 7f5600c00640 1 -- 172.21.15.7:0/1526615804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55fc158240 con 0x7f55fc0b5220 2024-04-25T11:10:56.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.575+0000 7f5600c00640 1 -- 172.21.15.7:0/1526615804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55fc050740 con 0x7f55fc0b5220 2024-04-25T11:10:56.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.576+0000 7f55f3e00640 1 -- 172.21.15.7:0/1526615804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f55f401a070 con 0x7f55fc0b5220 2024-04-25T11:10:56.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.577+0000 7f55f3e00640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55d80744b0 0x7f55d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:56.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.577+0000 7f55f3e00640 1 -- 172.21.15.7:0/1526615804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f55f4094940 con 0x7f55fc0b5220 2024-04-25T11:10:56.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.577+0000 7f55fb400640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55d80744b0 0x7f55d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:56.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.578+0000 7f55fb400640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55d80744b0 0x7f55d80769c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f55fc071200 tx=0x7f55ec00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:56.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.582+0000 7f55f3e00640 1 -- 172.21.15.7:0/1526615804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f55f405e6a0 con 0x7f55fc0b5220 2024-04-25T11:10:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.837+0000 7f5600c00640 1 -- 172.21.15.7:0/1526615804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f55fc14cd00 con 0x7f55fc0b5220 2024-04-25T11:10:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.844+0000 7f55f3e00640 1 -- 172.21.15.7:0/1526615804 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186735 (secure 0 0 0) 0x7f55f40629a0 con 0x7f55fc0b5220 2024-04-25T11:10:56.846 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.849+0000 7f5600c00640 1 -- 172.21.15.7:0/1526615804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55d80744b0 msgr2=0x7f55d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.849+0000 7f5600c00640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55d80744b0 0x7f55d80769c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f55fc071200 tx=0x7f55ec00b040 comp rx=0 tx=0).stop 2024-04-25T11:10:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.849+0000 7f5600c00640 1 -- 172.21.15.7:0/1526615804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55fc0b5220 msgr2=0x7f55fc1f9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.849+0000 7f5600c00640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55fc0b5220 0x7f55fc1f9e40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f55f4001eb0 tx=0x7f55f400c820 comp rx=0 tx=0).stop 2024-04-25T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.850+0000 7f5600c00640 1 -- 172.21.15.7:0/1526615804 shutdown_connections 2024-04-25T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.850+0000 7f5600c00640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55d80744b0 0x7f55d80769c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.850+0000 7f5600c00640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55fc0b5220 0x7f55fc1f9e40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.850+0000 7f5600c00640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55fc0af420 0x7f55fc1f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.850+0000 7f5600c00640 1 --2- 172.21.15.7:0/1526615804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55fc0aed40 0x7f55fc1f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.850+0000 7f5600c00640 1 -- 172.21.15.7:0/1526615804 >> 172.21.15.7:0/1526615804 conn(0x7f55fc0b3820 msgr2=0x7f55fc0b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.850+0000 7f5600c00640 1 -- 172.21.15.7:0/1526615804 shutdown_connections 2024-04-25T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:56.850+0000 7f5600c00640 1 -- 172.21.15.7:0/1526615804 wait complete. 2024-04-25T11:10:56.874 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2350428036},{"type":"v1","addr":"172.21.15.7:6833","nonce":2350428036}]},"active_addr":"172.21.15.7:6833/2350428036","active_change":"2024-04-25T11:10:01.750894+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2565659384}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":4285364253}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":2853693262}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":107296270}]}]} 2024-04-25T11:10:56.880 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T11:10:56.880 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T11:10:56.880 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.156+0000 7fed8fe00640 1 Processor -- start 2024-04-25T11:10:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.156+0000 7fed8fe00640 1 -- start start 2024-04-25T11:10:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.158+0000 7fed8fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fed900aed40 0x7fed9015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.158+0000 7fed8fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed900b5220 0x7fed900ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.158+0000 7fed8fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed900af420 0x7fed900af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.158+0000 7fed8fe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fed9015e1f0 con 0x7fed900b5220 2024-04-25T11:10:57.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.158+0000 7fed8fe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fed900afd70 con 0x7fed900aed40 2024-04-25T11:10:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.158+0000 7fed8fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fed900b64b0 con 0x7fed900af420 2024-04-25T11:10:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.159+0000 7fed87e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed900b5220 0x7fed900ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.159+0000 7fed87e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed900b5220 0x7fed900ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48708/0 (socket says 172.21.15.7:48708) 2024-04-25T11:10:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.159+0000 7fed87e00640 1 -- 172.21.15.7:0/257745125 learned_addr learned my addr 172.21.15.7:0/257745125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.159+0000 7fed8ea00640 1 --2- 172.21.15.7:0/257745125 >> v2:172.21.15.87:3300/0 conn(0x7fed900aed40 0x7fed9015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.159+0000 7fed8f400640 1 --2- 172.21.15.7:0/257745125 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed900af420 0x7fed900af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.160+0000 7fed87e00640 1 -- 172.21.15.7:0/257745125 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed900af420 msgr2=0x7fed900af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.160+0000 7fed87e00640 1 --2- 172.21.15.7:0/257745125 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed900af420 0x7fed900af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.160+0000 7fed87e00640 1 -- 172.21.15.7:0/257745125 >> v2:172.21.15.87:3300/0 conn(0x7fed900aed40 msgr2=0x7fed9015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.160+0000 7fed87e00640 1 --2- 172.21.15.7:0/257745125 >> v2:172.21.15.87:3300/0 conn(0x7fed900aed40 0x7fed9015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.160+0000 7fed87e00640 1 -- 172.21.15.7:0/257745125 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9006c610 con 0x7fed900b5220 2024-04-25T11:10:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.161+0000 7fed87e00640 1 --2- 172.21.15.7:0/257745125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed900b5220 0x7fed900ce4a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fed900725e0 tx=0x7fed7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c816a7e76e8665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.162+0000 7fed8e000640 1 -- 172.21.15.7:0/257745125 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78037070 con 0x7fed900b5220 2024-04-25T11:10:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.162+0000 7fed8e000640 1 -- 172.21.15.7:0/257745125 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed78032440 con 0x7fed900b5220 2024-04-25T11:10:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.162+0000 7fed8e000640 1 -- 172.21.15.7:0/257745125 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78032810 con 0x7fed900b5220 2024-04-25T11:10:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.162+0000 7fed8fe00640 1 -- 172.21.15.7:0/257745125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed900b5220 msgr2=0x7fed900ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.162+0000 7fed8fe00640 1 --2- 172.21.15.7:0/257745125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed900b5220 0x7fed900ce4a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fed900725e0 tx=0x7fed7802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.163+0000 7fed8fe00640 1 -- 172.21.15.7:0/257745125 shutdown_connections 2024-04-25T11:10:57.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.163+0000 7fed8fe00640 1 --2- 172.21.15.7:0/257745125 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed900af420 0x7fed900af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.163+0000 7fed8fe00640 1 --2- 172.21.15.7:0/257745125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed900b5220 0x7fed900ce4a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.163+0000 7fed8fe00640 1 --2- 172.21.15.7:0/257745125 >> v2:172.21.15.87:3300/0 conn(0x7fed900aed40 0x7fed9015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.163+0000 7fed8fe00640 1 -- 172.21.15.7:0/257745125 >> 172.21.15.7:0/257745125 conn(0x7fed900b3820 msgr2=0x7fed900b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.163+0000 7fed8fe00640 1 -- 172.21.15.7:0/257745125 shutdown_connections 2024-04-25T11:10:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.163+0000 7fed8fe00640 1 -- 172.21.15.7:0/257745125 wait complete. 2024-04-25T11:10:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.166+0000 7fed8fe00640 1 Processor -- start 2024-04-25T11:10:57.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.166+0000 7fed8fe00640 1 -- start start 2024-04-25T11:10:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.168+0000 7fed8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed900aed40 0x7fed90158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.168+0000 7fed8fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed900af420 0x7fed9015f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.168+0000 7fed8fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed900b5220 0x7fed901604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.168+0000 7fed8fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed9006ceb0 con 0x7fed900aed40 2024-04-25T11:10:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.168+0000 7fed8fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fed9006cbd0 con 0x7fed900af420 2024-04-25T11:10:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.168+0000 7fed8fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fed900b6110 con 0x7fed900b5220 2024-04-25T11:10:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.168+0000 7fed8ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed900aed40 0x7fed90158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.168+0000 7fed8ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed900aed40 0x7fed90158cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48716/0 (socket says 172.21.15.7:48716) 2024-04-25T11:10:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.168+0000 7fed8ea00640 1 -- 172.21.15.7:0/294569816 learned_addr learned my addr 172.21.15.7:0/294569816 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.168+0000 7fed87e00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed900af420 0x7fed9015f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed8ea00640 1 -- 172.21.15.7:0/294569816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed900b5220 msgr2=0x7fed901604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed8ea00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed900b5220 0x7fed901604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed8ea00640 1 -- 172.21.15.7:0/294569816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed900af420 msgr2=0x7fed9015f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed8ea00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed900af420 0x7fed9015f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed8ea00640 1 -- 172.21.15.7:0/294569816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9015fc00 con 0x7fed900aed40 2024-04-25T11:10:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed87e00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed900af420 0x7fed9015f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed8ea00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed900aed40 0x7fed90158cf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fed8000e870 tx=0x7fed8000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed86a00640 1 -- 172.21.15.7:0/294569816 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80017070 con 0x7fed900aed40 2024-04-25T11:10:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed86a00640 1 -- 172.21.15.7:0/294569816 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed8000ee60 con 0x7fed900aed40 2024-04-25T11:10:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed86a00640 1 -- 172.21.15.7:0/294569816 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8000cb60 con 0x7fed900aed40 2024-04-25T11:10:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed8fe00640 1 -- 172.21.15.7:0/294569816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed901f70c0 con 0x7fed900aed40 2024-04-25T11:10:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.169+0000 7fed8fe00640 1 -- 172.21.15.7:0/294569816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed901f7680 con 0x7fed900aed40 2024-04-25T11:10:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.171+0000 7fed8fe00640 1 -- 172.21.15.7:0/294569816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed90050740 con 0x7fed900aed40 2024-04-25T11:10:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.172+0000 7fed86a00640 1 -- 172.21.15.7:0/294569816 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed8001a070 con 0x7fed900aed40 2024-04-25T11:10:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.173+0000 7fed86a00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed6c074580 0x7fed6c076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.173+0000 7fed86a00640 1 -- 172.21.15.7:0/294569816 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fed80095770 con 0x7fed900aed40 2024-04-25T11:10:57.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.173+0000 7fed87e00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed6c074580 0x7fed6c076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.174+0000 7fed87e00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed6c074580 0x7fed6c076a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fed78034000 tx=0x7fed7802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:57.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.178+0000 7fed86a00640 1 -- 172.21.15.7:0/294569816 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed8005fba0 con 0x7fed900aed40 2024-04-25T11:10:57.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.383+0000 7fed8fe00640 1 -- 172.21.15.7:0/294569816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed900cdf40 con 0x7fed900aed40 2024-04-25T11:10:57.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.384+0000 7fed86a00640 1 -- 172.21.15.7:0/294569816 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18418 (secure 0 0 0) 0x7fed8005f4a0 con 0x7fed900aed40 2024-04-25T11:10:57.385 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:57.385 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:28.084739+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.387+0000 7fed8fe00640 1 -- 172.21.15.7:0/294569816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed6c074580 msgr2=0x7fed6c076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.387+0000 7fed8fe00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed6c074580 0x7fed6c076a90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fed78034000 tx=0x7fed7802e000 comp rx=0 tx=0).stop 2024-04-25T11:10:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.387+0000 7fed8fe00640 1 -- 172.21.15.7:0/294569816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed900aed40 msgr2=0x7fed90158cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.387+0000 7fed8fe00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed900aed40 0x7fed90158cf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fed8000e870 tx=0x7fed8000ed30 comp rx=0 tx=0).stop 2024-04-25T11:10:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.388+0000 7fed8fe00640 1 -- 172.21.15.7:0/294569816 shutdown_connections 2024-04-25T11:10:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.388+0000 7fed8fe00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed6c074580 0x7fed6c076a90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.388+0000 7fed8fe00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed900b5220 0x7fed901604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.388+0000 7fed8fe00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed900af420 0x7fed9015f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.388+0000 7fed8fe00640 1 --2- 172.21.15.7:0/294569816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed900aed40 0x7fed90158cf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.388+0000 7fed8fe00640 1 -- 172.21.15.7:0/294569816 >> 172.21.15.7:0/294569816 conn(0x7fed900b3820 msgr2=0x7fed9006fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.388+0000 7fed8fe00640 1 -- 172.21.15.7:0/294569816 shutdown_connections 2024-04-25T11:10:57.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.388+0000 7fed8fe00640 1 -- 172.21.15.7:0/294569816 wait complete. 2024-04-25T11:10:57.417 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T11:10:57.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:10:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.701+0000 7f0065e00640 1 Processor -- start 2024-04-25T11:10:57.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.701+0000 7f0065e00640 1 -- start start 2024-04-25T11:10:57.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.704+0000 7f0065e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f006006dcd0 0x7f006006e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.704+0000 7f0065e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f006015f580 0x7f006015f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.704+0000 7f0065e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00600d35a0 con 0x7f0060163ac0 2024-04-25T11:10:57.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.704+0000 7f0065e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f006006e620 con 0x7f006006dcd0 2024-04-25T11:10:57.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.704+0000 7f0065e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0060164310 con 0x7f006015f580 2024-04-25T11:10:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.704+0000 7f005fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f006006dcd0 0x7f006006e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.704+0000 7f0065400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f006015f580 0x7f006015f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.704+0000 7f005fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f006006dcd0 0x7f006006e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50242/0 (socket says 172.21.15.7:50242) 2024-04-25T11:10:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.704+0000 7f005fe00640 1 -- 172.21.15.7:0/2521738063 learned_addr learned my addr 172.21.15.7:0/2521738063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.704+0000 7f0064a00640 1 --1- 172.21.15.7:0/2521738063 >> v1:172.21.15.87:6789/0 conn(0x7f0060163ac0 0x7f00600b4b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39224/0 (socket says 172.21.15.7:39224) 2024-04-25T11:10:57.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.705+0000 7f005fe00640 1 -- 172.21.15.7:0/2521738063 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f006015f580 msgr2=0x7f006015f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.705+0000 7f005fe00640 1 --2- 172.21.15.7:0/2521738063 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f006015f580 0x7f006015f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.705+0000 7f005fe00640 1 -- 172.21.15.7:0/2521738063 >> v1:172.21.15.87:6789/0 conn(0x7f0060163ac0 legacy=0x7f00600b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:57.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.705+0000 7f005fe00640 1 -- 172.21.15.7:0/2521738063 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00601645f0 con 0x7f006006dcd0 2024-04-25T11:10:57.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.707+0000 7f005fe00640 1 --2- 172.21.15.7:0/2521738063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006006dcd0 0x7f006006e0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f004c009870 tx=0x7f004c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7fca1f9d5ed3eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.707+0000 7f005f400640 1 -- 172.21.15.7:0/2521738063 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c037070 con 0x7f006006dcd0 2024-04-25T11:10:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.708+0000 7f005f400640 1 -- 172.21.15.7:0/2521738063 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f004c032440 con 0x7f006006dcd0 2024-04-25T11:10:57.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.708+0000 7f005f400640 1 -- 172.21.15.7:0/2521738063 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c032810 con 0x7f006006dcd0 2024-04-25T11:10:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.709+0000 7f0065e00640 1 -- 172.21.15.7:0/2521738063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006006dcd0 msgr2=0x7f006006e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.709+0000 7f0065e00640 1 --2- 172.21.15.7:0/2521738063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006006dcd0 0x7f006006e0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f004c009870 tx=0x7f004c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.709+0000 7f0065e00640 1 -- 172.21.15.7:0/2521738063 shutdown_connections 2024-04-25T11:10:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.709+0000 7f0065e00640 1 --2- 172.21.15.7:0/2521738063 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f006015f580 0x7f006015f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.709+0000 7f0065e00640 1 --2- 172.21.15.7:0/2521738063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006006dcd0 0x7f006006e0e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.709+0000 7f0065e00640 1 -- 172.21.15.7:0/2521738063 >> 172.21.15.7:0/2521738063 conn(0x7f00600d29e0 msgr2=0x7f00600d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.709+0000 7f0065e00640 1 -- 172.21.15.7:0/2521738063 shutdown_connections 2024-04-25T11:10:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.709+0000 7f0065e00640 1 -- 172.21.15.7:0/2521738063 wait complete. 2024-04-25T11:10:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.712+0000 7f0065e00640 1 Processor -- start 2024-04-25T11:10:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.712+0000 7f0065e00640 1 -- start start 2024-04-25T11:10:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.714+0000 7f0065e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006006dcd0 0x7f00600d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.714+0000 7f0065e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f006015f580 0x7f00600ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0065e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0060163ac0 0x7f00601f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0065e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0060164050 con 0x7f0060163ac0 2024-04-25T11:10:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0065e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0060163ed0 con 0x7f006006dcd0 2024-04-25T11:10:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0065e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f006015fed0 con 0x7f006015f580 2024-04-25T11:10:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0065400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0060163ac0 0x7f00601f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0065400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0060163ac0 0x7f00601f9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48722/0 (socket says 172.21.15.7:48722) 2024-04-25T11:10:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0065400640 1 -- 172.21.15.7:0/2781757438 learned_addr learned my addr 172.21.15.7:0/2781757438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0064a00640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006006dcd0 0x7f00600d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f005fe00640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f006015f580 0x7f00600ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0065400640 1 -- 172.21.15.7:0/2781757438 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f006015f580 msgr2=0x7f00600ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0065400640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f006015f580 0x7f00600ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0065400640 1 -- 172.21.15.7:0/2781757438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006006dcd0 msgr2=0x7f00600d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.715+0000 7f0065400640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006006dcd0 0x7f00600d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.716+0000 7f0065400640 1 -- 172.21.15.7:0/2781757438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00601e98e0 con 0x7f0060163ac0 2024-04-25T11:10:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.716+0000 7f0065400640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0060163ac0 0x7f00601f9b60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f005800e820 tx=0x7f005800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.716+0000 7f005cc00640 1 -- 172.21.15.7:0/2781757438 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058017070 con 0x7f0060163ac0 2024-04-25T11:10:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.716+0000 7f005cc00640 1 -- 172.21.15.7:0/2781757438 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f005800c730 con 0x7f0060163ac0 2024-04-25T11:10:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.716+0000 7f0065e00640 1 -- 172.21.15.7:0/2781757438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00600ad0a0 con 0x7f0060163ac0 2024-04-25T11:10:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.716+0000 7f005cc00640 1 -- 172.21.15.7:0/2781757438 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005800cb00 con 0x7f0060163ac0 2024-04-25T11:10:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.716+0000 7f0065e00640 1 -- 172.21.15.7:0/2781757438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00601e9ba0 con 0x7f0060163ac0 2024-04-25T11:10:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.718+0000 7f0065e00640 1 -- 172.21.15.7:0/2781757438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00600ace10 con 0x7f0060163ac0 2024-04-25T11:10:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.719+0000 7f005cc00640 1 -- 172.21.15.7:0/2781757438 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f005801a070 con 0x7f0060163ac0 2024-04-25T11:10:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.720+0000 7f005cc00640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00400744e0 0x7f00400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.720+0000 7f005cc00640 1 -- 172.21.15.7:0/2781757438 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0058095400 con 0x7f0060163ac0 2024-04-25T11:10:57.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.721+0000 7f0064a00640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00400744e0 0x7f00400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.721+0000 7f0064a00640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00400744e0 0x7f00400769f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f00500012e0 tx=0x7f0050005b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:57.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.725+0000 7f005cc00640 1 -- 172.21.15.7:0/2781757438 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f005805f850 con 0x7f0060163ac0 2024-04-25T11:10:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.930+0000 7f0065e00640 1 -- 172.21.15.7:0/2781757438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0060050740 con 0x7f0060163ac0 2024-04-25T11:10:57.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.934+0000 7f005cc00640 1 -- 172.21.15.7:0/2781757438 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18418 (secure 0 0 0) 0x7f005805f150 con 0x7f0060163ac0 2024-04-25T11:10:57.934 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:10:57.935 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:10:28.084739+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:10:57.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.936+0000 7f0065e00640 1 -- 172.21.15.7:0/2781757438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00400744e0 msgr2=0x7f00400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:57.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.936+0000 7f0065e00640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00400744e0 0x7f00400769f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f00500012e0 tx=0x7f0050005b70 comp rx=0 tx=0).stop 2024-04-25T11:10:57.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.936+0000 7f0065e00640 1 -- 172.21.15.7:0/2781757438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0060163ac0 msgr2=0x7f00601f9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:57.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.936+0000 7f0065e00640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0060163ac0 0x7f00601f9b60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f005800e820 tx=0x7f005800ece0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.937+0000 7f0065e00640 1 -- 172.21.15.7:0/2781757438 shutdown_connections 2024-04-25T11:10:57.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.937+0000 7f0065e00640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00400744e0 0x7f00400769f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.937+0000 7f0065e00640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0060163ac0 0x7f00601f9b60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.937+0000 7f0065e00640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f006015f580 0x7f00600ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.937+0000 7f0065e00640 1 --2- 172.21.15.7:0/2781757438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f006006dcd0 0x7f00600d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:57.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.937+0000 7f0065e00640 1 -- 172.21.15.7:0/2781757438 >> 172.21.15.7:0/2781757438 conn(0x7f00600d29e0 msgr2=0x7f006015d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:57.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.937+0000 7f0065e00640 1 -- 172.21.15.7:0/2781757438 shutdown_connections 2024-04-25T11:10:57.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:57.937+0000 7f0065e00640 1 -- 172.21.15.7:0/2781757438 wait complete. 2024-04-25T11:10:57.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T11:10:57.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T11:10:57.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T11:10:57.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T11:10:57.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T11:10:57.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T11:10:57.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T11:10:57.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T11:10:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.390+0000 7fdcd7200640 1 Processor -- start 2024-04-25T11:10:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.390+0000 7fdcd7200640 1 -- start start 2024-04-25T11:10:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.393+0000 7fdcd7200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdcd0146180 0x7fdcd0070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.393+0000 7fdcd7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd00b08e0 0x7fdcd0149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.394+0000 7fdcd7200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd0070860 con 0x7fdcd01490f0 2024-04-25T11:10:58.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.394+0000 7fdcd7200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fdcd0146590 con 0x7fdcd0146180 2024-04-25T11:10:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.394+0000 7fdcd7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd006f280 con 0x7fdcd00b08e0 2024-04-25T11:10:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.394+0000 7fdcd5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdcd0146180 0x7fdcd0070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.394+0000 7fdcd6800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd00b08e0 0x7fdcd0149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.394+0000 7fdcd5400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fdcd01490f0 0x7fdcd00b01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39234/0 (socket says 172.21.15.7:39234) 2024-04-25T11:10:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.394+0000 7fdcd5400640 1 -- 172.21.15.7:0/3793011512 learned_addr learned my addr 172.21.15.7:0/3793011512 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.396+0000 7fdcd6800640 1 -- 172.21.15.7:0/3793011512 >> v2:172.21.15.87:3300/0 conn(0x7fdcd0146180 msgr2=0x7fdcd0070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.393+0000 7f7b6a600640 1 Processor -- start 2024-04-25T11:10:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.393+0000 7f7b6a600640 1 -- start start 2024-04-25T11:10:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.396+0000 7f7b6a600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7b64168050 0x7f7b640b7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.397+0000 7f7b6a600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b640b7a60 0x7f7b640d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.397+0000 7f7b6a600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b641659c0 con 0x7f7b640d0b00 2024-04-25T11:10:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.397+0000 7f7b6a600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7b640b7e70 con 0x7f7b640b7a60 2024-04-25T11:10:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.397+0000 7f7b6a600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7b64164e40 con 0x7f7b64168050 2024-04-25T11:10:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.396+0000 7fdcd6800640 1 --2- 172.21.15.7:0/3793011512 >> v2:172.21.15.87:3300/0 conn(0x7fdcd0146180 0x7fdcd0070320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.396+0000 7fdcd6800640 1 -- 172.21.15.7:0/3793011512 >> v1:172.21.15.87:6789/0 conn(0x7fdcd01490f0 legacy=0x7fdcd00b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.396+0000 7fdcd6800640 1 -- 172.21.15.7:0/3793011512 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd006f560 con 0x7fdcd00b08e0 2024-04-25T11:10:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.397+0000 7f7b63400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7b640d0b00 0x7f7b64161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50156/0 (socket says 172.21.15.7:50156) 2024-04-25T11:10:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.397+0000 7f7b63400640 1 -- 172.21.15.7:0/2157184579 learned_addr learned my addr 172.21.15.7:0/2157184579 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.398+0000 7f50fa600640 1 Processor -- start 2024-04-25T11:10:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.398+0000 7fdcd6800640 1 --2- 172.21.15.7:0/3793011512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd00b08e0 0x7fdcd0149500 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009d90 tx=0x7fdccc02a570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37890ed0b033f7ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.398+0000 7fdcd4a00640 1 -- 172.21.15.7:0/3793011512 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc007740 con 0x7fdcd00b08e0 2024-04-25T11:10:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.399+0000 7fdcd4a00640 1 -- 172.21.15.7:0/3793011512 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdccc0078e0 con 0x7fdcd00b08e0 2024-04-25T11:10:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.399+0000 7fdcd4a00640 1 -- 172.21.15.7:0/3793011512 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdccc007cb0 con 0x7fdcd00b08e0 2024-04-25T11:10:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.400+0000 7fdcd7200640 1 -- 172.21.15.7:0/3793011512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd00b08e0 msgr2=0x7fdcd0149500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.400+0000 7fdcd7200640 1 --2- 172.21.15.7:0/3793011512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd00b08e0 0x7fdcd0149500 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009d90 tx=0x7fdccc02a570 comp rx=0 tx=0).stop 2024-04-25T11:10:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.400+0000 7fdcd7200640 1 -- 172.21.15.7:0/3793011512 shutdown_connections 2024-04-25T11:10:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.400+0000 7fdcd7200640 1 --2- 172.21.15.7:0/3793011512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd00b08e0 0x7fdcd0149500 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.400+0000 7fdcd7200640 1 --2- 172.21.15.7:0/3793011512 >> v2:172.21.15.87:3300/0 conn(0x7fdcd0146180 0x7fdcd0070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.400+0000 7fdcd7200640 1 -- 172.21.15.7:0/3793011512 >> 172.21.15.7:0/3793011512 conn(0x7fdcd00740f0 msgr2=0x7fdcd006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.397+0000 7f7b63e00640 1 --2- 172.21.15.7:0/2157184579 >> v2:172.21.15.7:3300/0 conn(0x7f7b640b7a60 0x7f7b640d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.398+0000 7f7b62a00640 1 --2- 172.21.15.7:0/2157184579 >> v2:172.21.15.87:3300/0 conn(0x7f7b64168050 0x7f7b640b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.399+0000 7f7b63e00640 1 -- 172.21.15.7:0/2157184579 >> v2:172.21.15.87:3300/0 conn(0x7f7b64168050 msgr2=0x7f7b640b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.399+0000 7f7b63e00640 1 --2- 172.21.15.7:0/2157184579 >> v2:172.21.15.87:3300/0 conn(0x7f7b64168050 0x7f7b640b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.399+0000 7f7b63e00640 1 -- 172.21.15.7:0/2157184579 >> v1:172.21.15.7:6789/0 conn(0x7f7b640d0b00 legacy=0x7f7b64161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.399+0000 7f7b63e00640 1 -- 172.21.15.7:0/2157184579 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b64165120 con 0x7f7b640b7a60 2024-04-25T11:10:58.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.398+0000 7f50fa600640 1 -- start start 2024-04-25T11:10:58.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.401+0000 7f50fa600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f50ec0e47c0 0x7f50ec037b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.401+0000 7f50fa600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50ec0380c0 0x7f50ec0384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.401+0000 7f50fa600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50ec057910 con 0x7f50ec0e7330 2024-04-25T11:10:58.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.401+0000 7f50fa600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f50ec03c190 con 0x7f50ec0e47c0 2024-04-25T11:10:58.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.401+0000 7f50fa600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f50ec057bf0 con 0x7f50ec0380c0 2024-04-25T11:10:58.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.402+0000 7f50f3e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f50ec0e7330 0x7f50ec0e7740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39238/0 (socket says 172.21.15.7:39238) 2024-04-25T11:10:58.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.402+0000 7f50f3e00640 1 -- 172.21.15.7:0/3675660666 learned_addr learned my addr 172.21.15.7:0/3675660666 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:58.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.401+0000 7fdcd7200640 1 -- 172.21.15.7:0/3793011512 shutdown_connections 2024-04-25T11:10:58.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.401+0000 7fdcd7200640 1 -- 172.21.15.7:0/3793011512 wait complete. 2024-04-25T11:10:58.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.402+0000 7f50f3400640 1 --2- 172.21.15.7:0/3675660666 >> v2:172.21.15.7:3300/0 conn(0x7f50ec0e47c0 0x7f50ec037b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.403+0000 7f7b63e00640 1 --2- 172.21.15.7:0/2157184579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b640b7a60 0x7f7b640d3f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b6408e5a0 tx=0x7f7b5c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b117d3c66523c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.404+0000 7f7b62000640 1 -- 172.21.15.7:0/2157184579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c007740 con 0x7f7b640b7a60 2024-04-25T11:10:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.404+0000 7f7b62000640 1 -- 172.21.15.7:0/2157184579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c0078e0 con 0x7f7b640b7a60 2024-04-25T11:10:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.404+0000 7f7b62000640 1 -- 172.21.15.7:0/2157184579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c007cf0 con 0x7f7b640b7a60 2024-04-25T11:10:58.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.404+0000 7fdcd7200640 1 Processor -- start 2024-04-25T11:10:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.402+0000 7f50f2a00640 1 --2- 172.21.15.7:0/3675660666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50ec0380c0 0x7f50ec0384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.402+0000 7f50f2000640 1 -- 172.21.15.7:0/3675660666 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4034515597 0 0) 0x7f50ec057910 con 0x7f50ec0e7330 2024-04-25T11:10:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.403+0000 7f50f2000640 1 -- 172.21.15.7:0/3675660666 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50dc003520 con 0x7f50ec0e7330 2024-04-25T11:10:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.403+0000 7f50f3400640 1 -- 172.21.15.7:0/3675660666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50ec0380c0 msgr2=0x7f50ec0384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.403+0000 7f50f3400640 1 --2- 172.21.15.7:0/3675660666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50ec0380c0 0x7f50ec0384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.403+0000 7f50f3400640 1 -- 172.21.15.7:0/3675660666 >> v1:172.21.15.87:6789/0 conn(0x7f50ec0e7330 legacy=0x7f50ec0e7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.403+0000 7f50f3400640 1 -- 172.21.15.7:0/3675660666 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec057ed0 con 0x7f50ec0e47c0 2024-04-25T11:10:58.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.405+0000 7fdcd7200640 1 -- start start 2024-04-25T11:10:58.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.405+0000 7f50f3400640 1 --2- 172.21.15.7:0/3675660666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50ec0e47c0 0x7f50ec037b80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009790 tx=0x7f50e802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5d4018b4c4c5c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.405+0000 7f9f90c00640 1 Processor -- start 2024-04-25T11:10:58.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.406+0000 7f50f2000640 1 -- 172.21.15.7:0/3675660666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e8037070 con 0x7f50ec0e47c0 2024-04-25T11:10:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.406+0000 7f9f90c00640 1 -- start start 2024-04-25T11:10:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.406+0000 7f7030c00640 1 Processor -- start 2024-04-25T11:10:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.406+0000 7f50f2000640 1 -- 172.21.15.7:0/3675660666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e8032440 con 0x7f50ec0e47c0 2024-04-25T11:10:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.406+0000 7f50f2000640 1 -- 172.21.15.7:0/3675660666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e802ae60 con 0x7f50ec0e47c0 2024-04-25T11:10:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.406+0000 7f16ce200640 1 Processor -- start 2024-04-25T11:10:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7f7b6a600640 1 -- 172.21.15.7:0/2157184579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b640b7a60 msgr2=0x7f7b640d3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7f7b6a600640 1 --2- 172.21.15.7:0/2157184579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b640b7a60 0x7f7b640d3f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b6408e5a0 tx=0x7f7b5c02a570 comp rx=0 tx=0).stop 2024-04-25T11:10:58.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7fdcd7200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcd00b08e0 0x7fdcd00d0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7fdcd7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd0146180 0x7fdcd0139060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7fdcd7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd01490f0 0x7fdcd01395a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7fdcd7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd006fe00 con 0x7fdcd0146180 2024-04-25T11:10:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7fdcd7200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd006fb20 con 0x7fdcd00b08e0 2024-04-25T11:10:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7fdcd7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd006eea0 con 0x7fdcd01490f0 2024-04-25T11:10:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7f50fa600640 1 -- 172.21.15.7:0/3675660666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50ec0e47c0 msgr2=0x7f50ec037b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7f50fa600640 1 --2- 172.21.15.7:0/3675660666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50ec0e47c0 0x7f50ec037b80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009790 tx=0x7f50e802a110 comp rx=0 tx=0).stop 2024-04-25T11:10:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7fdcd5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcd00b08e0 0x7fdcd00d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7f50fa600640 1 -- 172.21.15.7:0/3675660666 shutdown_connections 2024-04-25T11:10:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7f50fa600640 1 --2- 172.21.15.7:0/3675660666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50ec0380c0 0x7f50ec0384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7f50fa600640 1 --2- 172.21.15.7:0/3675660666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50ec0e47c0 0x7f50ec037b80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7f50fa600640 1 -- 172.21.15.7:0/3675660666 >> 172.21.15.7:0/3675660666 conn(0x7f50ec059110 msgr2=0x7f50ec059520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7fdcd5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd0146180 0x7fdcd0139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7fdcd5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd0146180 0x7fdcd0139060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48742/0 (socket says 172.21.15.7:48742) 2024-04-25T11:10:58.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7fdcd5400640 1 -- 172.21.15.7:0/1222750485 learned_addr learned my addr 172.21.15.7:0/1222750485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7fdcd6800640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd01490f0 0x7fdcd01395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7f7b6a600640 1 -- 172.21.15.7:0/2157184579 shutdown_connections 2024-04-25T11:10:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7f7b6a600640 1 --2- 172.21.15.7:0/2157184579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b640b7a60 0x7f7b640d3f70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7f7b6a600640 1 --2- 172.21.15.7:0/2157184579 >> v2:172.21.15.87:3300/0 conn(0x7f7b64168050 0x7f7b640b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7f7b6a600640 1 -- 172.21.15.7:0/2157184579 >> 172.21.15.7:0/2157184579 conn(0x7f7b640d2520 msgr2=0x7f7b640d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7f16ce200640 1 -- start start 2024-04-25T11:10:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.409+0000 7f50fa600640 1 -- 172.21.15.7:0/3675660666 shutdown_connections 2024-04-25T11:10:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.409+0000 7f50fa600640 1 -- 172.21.15.7:0/3675660666 wait complete. 2024-04-25T11:10:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.408+0000 7f7b6a600640 1 -- 172.21.15.7:0/2157184579 shutdown_connections 2024-04-25T11:10:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.409+0000 7fdcd5400640 1 -- 172.21.15.7:0/1222750485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd01490f0 msgr2=0x7fdcd01395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.409+0000 7fdcd5400640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd01490f0 0x7fdcd01395a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.409+0000 7f7b6a600640 1 -- 172.21.15.7:0/2157184579 wait complete. 2024-04-25T11:10:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.409+0000 7fdcd5400640 1 -- 172.21.15.7:0/1222750485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcd00b08e0 msgr2=0x7fdcd00d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.409+0000 7fdcd5400640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcd00b08e0 0x7fdcd00d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.409+0000 7fdcd5400640 1 -- 172.21.15.7:0/1222750485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd00cd880 con 0x7fdcd0146180 2024-04-25T11:10:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.409+0000 7fdcd5400640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd0146180 0x7fdcd0139060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400c5a0 tx=0x7fdcc400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.409+0000 7f9f90c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9f8c0d0310 0x7f9f8c06fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.409+0000 7f9f8b400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9f8c0cff00 0x7f9f8c167de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50158/0 (socket says 172.21.15.7:50158) 2024-04-25T11:10:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.410+0000 7f9f8b400640 1 -- 172.21.15.7:0/1194778784 learned_addr learned my addr 172.21.15.7:0/1194778784 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.410+0000 7f9f90c00640 1 -- 172.21.15.7:0/1194778784 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c0d0720 con 0x7f9f8c0cff00 2024-04-25T11:10:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.410+0000 7f9f90c00640 1 -- 172.21.15.7:0/1194778784 --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8c0d09e0 con 0x7f9f8c1684f0 2024-04-25T11:10:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.410+0000 7f16ce200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16c004f370 0x7f16c0033aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.410+0000 7f16ce200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16c004f780 con 0x7f16c0033fe0 2024-04-25T11:10:58.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.410+0000 7f16ce200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16c00343f0 con 0x7f16c0052fe0 2024-04-25T11:10:58.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.410+0000 7f16ce200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f16c00533f0 con 0x7f16c004f370 2024-04-25T11:10:58.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.410+0000 7f9f90c00640 1 -- 172.21.15.7:0/1194778784 --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9f8c06eea0 con 0x7f9f8c0d0310 2024-04-25T11:10:58.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.410+0000 7fdcca000640 1 -- 172.21.15.7:0/1222750485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4017070 con 0x7fdcd0146180 2024-04-25T11:10:58.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.411+0000 7fdcca000640 1 -- 172.21.15.7:0/1222750485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc40074e0 con 0x7fdcd0146180 2024-04-25T11:10:58.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.411+0000 7fdcca000640 1 -- 172.21.15.7:0/1222750485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4007690 con 0x7fdcd0146180 2024-04-25T11:10:58.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.411+0000 7fdcd7200640 1 -- 172.21.15.7:0/1222750485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd0136f70 con 0x7fdcd0146180 2024-04-25T11:10:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.411+0000 7fdcd7200640 1 -- 172.21.15.7:0/1222750485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd0137530 con 0x7fdcd0146180 2024-04-25T11:10:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.407+0000 7f7030c00640 1 -- start start 2024-04-25T11:10:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.411+0000 7f16c7400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f16c0052fe0 0x7f16c0007350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39250/0 (socket says 172.21.15.7:39250) 2024-04-25T11:10:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.411+0000 7f16c7400640 1 -- 172.21.15.7:0/3748734646 learned_addr learned my addr 172.21.15.7:0/3748734646 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.411+0000 7f16c6a00640 1 --2- 172.21.15.7:0/3748734646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16c004f370 0x7f16c0033aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.411+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 200655680 0 0) 0x7f9f8c0d0720 con 0x7f9f8c0cff00 2024-04-25T11:10:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.412+0000 7f7030c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f70240e47c0 0x7f7024037b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.412+0000 7f7030c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70240384d0 0x7f70240e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.412+0000 7f7030c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7024057910 con 0x7f70240380c0 2024-04-25T11:10:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.412+0000 7f7030c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f702403c190 con 0x7f70240e47c0 2024-04-25T11:10:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.412+0000 7f7030c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7024057bf0 con 0x7f70240384d0 2024-04-25T11:10:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.412+0000 7f7cbe600640 1 Processor -- start 2024-04-25T11:10:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.412+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f70003520 con 0x7f9f8c0cff00 2024-04-25T11:10:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.412+0000 7f9f8be00640 1 --2- 172.21.15.7:0/1194778784 >> v2:172.21.15.87:3300/0 conn(0x7f9f8c0d0310 0x7f9f8c06fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.412+0000 7f16c6000640 1 -- 172.21.15.7:0/3748734646 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3311023443 0 0) 0x7f16b4001110 con 0x7f16c0052fe0 2024-04-25T11:10:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.412+0000 7f16c6000640 1 -- 172.21.15.7:0/3748734646 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16b0003520 con 0x7f16c0052fe0 2024-04-25T11:10:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.413+0000 7f7b6a600640 1 Processor -- start 2024-04-25T11:10:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.413+0000 7f16c6a00640 1 -- 172.21.15.7:0/3748734646 >> v1:172.21.15.87:6789/0 conn(0x7f16c0052fe0 legacy=0x7f16c0007350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.413+0000 7f7b6a600640 1 -- start start 2024-04-25T11:10:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.412+0000 7f7cbe600640 1 -- start start 2024-04-25T11:10:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.413+0000 7f50fa600640 1 Processor -- start 2024-04-25T11:10:58.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.413+0000 7f702aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f70240380c0 0x7f70240e7330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39254/0 (socket says 172.21.15.7:39254) 2024-04-25T11:10:58.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.413+0000 7f702aa00640 1 -- 172.21.15.7:0/2261202741 learned_addr learned my addr 172.21.15.7:0/2261202741 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:58.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.413+0000 7f702be00640 1 --2- 172.21.15.7:0/2261202741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70240384d0 0x7f70240e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.413+0000 7f702b400640 1 --2- 172.21.15.7:0/2261202741 >> v2:172.21.15.87:3300/0 conn(0x7f70240e47c0 0x7f7024037b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3390682309 0 0) 0x7f9f70003520 con 0x7f9f8c0cff00 2024-04-25T11:10:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 >> v2:172.21.15.87:3300/0 conn(0x7f9f8c0d0310 msgr2=0x7f9f8c06fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 --2- 172.21.15.7:0/1194778784 >> v2:172.21.15.87:3300/0 conn(0x7f9f8c0d0310 0x7f9f8c06fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 >> v1:172.21.15.87:6789/0 conn(0x7f9f8c1684f0 legacy=0x7f9f8c06f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c0d0cc0 con 0x7f9f8c0cff00 2024-04-25T11:10:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f9f840019f0 con 0x7f9f8c0cff00 2024-04-25T11:10:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 >> v1:172.21.15.7:6789/0 conn(0x7f9f8c0cff00 legacy=0x7f9f8c167de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 --2- 172.21.15.7:0/1194778784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f700039b0 0x7f9f70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 --2- 172.21.15.7:0/1194778784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f70004800 0x7f9f70004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 --2- 172.21.15.7:0/1194778784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f700051a0 0x7f9f7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c06eea0 con 0x7f9f700039b0 2024-04-25T11:10:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7000c140 con 0x7f9f70004800 2024-04-25T11:10:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7000c340 con 0x7f9f700051a0 2024-04-25T11:10:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7fdcd7200640 1 -- 172.21.15.7:0/1222750485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdcd0139e00 con 0x7fdcd0146180 2024-04-25T11:10:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f9f8be00640 1 --2- 172.21.15.7:0/1194778784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f700051a0 0x7f9f7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f9f8b400640 1 --2- 172.21.15.7:0/1194778784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f700039b0 0x7f9f70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f9f8aa00640 1 --2- 172.21.15.7:0/1194778784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f70004800 0x7f9f70004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f9f8b400640 1 -- 172.21.15.7:0/1194778784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f700051a0 msgr2=0x7f9f7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f9f8b400640 1 --2- 172.21.15.7:0/1194778784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f700051a0 0x7f9f7000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f9f8b400640 1 -- 172.21.15.7:0/1194778784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f70004800 msgr2=0x7f9f70004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f9f8b400640 1 --2- 172.21.15.7:0/1194778784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f70004800 0x7f9f70004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f9f8b400640 1 -- 172.21.15.7:0/1194778784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c0d0cc0 con 0x7f9f700039b0 2024-04-25T11:10:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f7b6a600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b640b7a60 0x7f7b640ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f702b400640 1 -- 172.21.15.7:0/2261202741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70240384d0 msgr2=0x7f70240e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f702b400640 1 --2- 172.21.15.7:0/2261202741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70240384d0 0x7f70240e7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f702b400640 1 -- 172.21.15.7:0/2261202741 >> v1:172.21.15.87:6789/0 conn(0x7f70240380c0 legacy=0x7f70240e7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.415+0000 7f702b400640 1 -- 172.21.15.7:0/2261202741 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7024057ed0 con 0x7f70240e47c0 2024-04-25T11:10:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f9f8be00640 1 -- 172.21.15.7:0/1194778784 reap_dead start 2024-04-25T11:10:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.413+0000 7f16c6a00640 1 -- 172.21.15.7:0/3748734646 >> v1:172.21.15.7:6789/0 conn(0x7f16c0033fe0 legacy=0x7f16c0038370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.413+0000 7f16c6a00640 1 -- 172.21.15.7:0/3748734646 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c0007a60 con 0x7f16c004f370 2024-04-25T11:10:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f7b6a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b640d0b00 0x7f7b642059d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.414+0000 7f50fa600640 1 -- start start 2024-04-25T11:10:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f7b6a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b64168050 0x7f7b640cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f50fa600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50ec0380c0 0x7f50ec0de390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f50fa600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50ec0e47c0 0x7f50ec1898b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f50fa600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50ec0e7330 0x7f50ec03d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f50fa600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec0e4bd0 con 0x7f50ec0e47c0 2024-04-25T11:10:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f50fa600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec058490 con 0x7f50ec0380c0 2024-04-25T11:10:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f50fa600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f50ec0dec20 con 0x7f50ec0e7330 2024-04-25T11:10:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f7b6a600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b64164a60 con 0x7f7b64168050 2024-04-25T11:10:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f7b6a600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7b641656e0 con 0x7f7b640d0b00 2024-04-25T11:10:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f7b6a600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7b64164be0 con 0x7f7b640b7a60 2024-04-25T11:10:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f50f3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50ec0380c0 0x7f50ec0de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f50f3e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50ec0e7330 0x7f50ec03d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f50f3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50ec0380c0 0x7f50ec0de390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50312/0 (socket says 172.21.15.7:50312) 2024-04-25T11:10:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f50f3400640 1 -- 172.21.15.7:0/3271268637 learned_addr learned my addr 172.21.15.7:0/3271268637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.416+0000 7f7b63400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b640b7a60 0x7f7b640ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f7b63400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b640b7a60 0x7f7b640ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43390/0 (socket says 172.21.15.7:43390) 2024-04-25T11:10:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f7b63400640 1 -- 172.21.15.7:0/1203052249 learned_addr learned my addr 172.21.15.7:0/1203052249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f7cbe600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7cb80d2210 0x7f7cb80d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f7cbe600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7cb81625c0 0x7f7cb81629d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f7b62a00640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b640d0b00 0x7f7b642059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f50f2a00640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50ec0e47c0 0x7f50ec1898b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f7b63e00640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b64168050 0x7f7b640cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f16c6a00640 1 --2- 172.21.15.7:0/3748734646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16c004f370 0x7f16c0033aa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0097d0 tx=0x7f16bc029fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e6d5a0a40b0cdfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7fdcca000640 1 -- 172.21.15.7:0/1222750485 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdcc401a070 con 0x7fdcd0146180 2024-04-25T11:10:58.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f50f2a00640 1 -- 172.21.15.7:0/3271268637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50ec0e7330 msgr2=0x7f50ec03d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f50f2a00640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50ec0e7330 0x7f50ec03d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f50f2a00640 1 -- 172.21.15.7:0/3271268637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50ec0380c0 msgr2=0x7f50ec0de390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f7cbe600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7cb8162f10 0x7f7cb80ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f7cbe600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7cb80b8f10 con 0x7f7cb8162f10 2024-04-25T11:10:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f7cbe600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7cb809cda0 con 0x7f7cb80d2210 2024-04-25T11:10:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.417+0000 7f7cbe600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb809d080 con 0x7f7cb81625c0 2024-04-25T11:10:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f50f2a00640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50ec0380c0 0x7f50ec0de390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f50f2a00640 1 -- 172.21.15.7:0/3271268637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec0e6020 con 0x7f50ec0e47c0 2024-04-25T11:10:58.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f50f3400640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50ec0380c0 0x7f50ec0de390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:10:58.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f9f8b400640 1 --2- 172.21.15.7:0/1194778784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f700039b0 0x7f9f70003e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9f700057e0 tx=0x7f9f70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8108ec84dd9e7bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8400a4c0 con 0x7f9f700039b0 2024-04-25T11:10:58.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f16c6000640 1 -- 172.21.15.7:0/3748734646 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc037070 con 0x7f16c004f370 2024-04-25T11:10:58.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8402f070 con 0x7f9f700039b0 2024-04-25T11:10:58.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f9f8a000640 1 -- 172.21.15.7:0/1194778784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8402bc60 con 0x7f9f700039b0 2024-04-25T11:10:58.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f1ae4800640 1 Processor -- start 2024-04-25T11:10:58.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7b63400640 1 -- 172.21.15.7:0/1203052249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b640d0b00 msgr2=0x7f7b642059d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7b63400640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b640d0b00 0x7f7b642059d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7b63400640 1 -- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b64168050 msgr2=0x7f7b640cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7b63400640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b64168050 0x7f7b640cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7b63400640 1 -- 172.21.15.7:0/1203052249 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b642065c0 con 0x7f7b640b7a60 2024-04-25T11:10:58.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7b63e00640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b64168050 0x7f7b640cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:58.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7b62a00640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b640d0b00 0x7f7b642059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:58.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f1ae4800640 1 -- start start 2024-04-25T11:10:58.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f702b400640 1 --2- 172.21.15.7:0/2261202741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70240e47c0 0x7f7024037b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7020009790 tx=0x7f702002a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acda30841de850a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f50f2a00640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50ec0e47c0 0x7f50ec1898b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009db0 tx=0x7f50e000e540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f7b63400640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b640b7a60 0x7f7b640ce4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800c3f0 tx=0x7f7b5800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f16c6000640 1 -- 172.21.15.7:0/3748734646 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc02ac60 con 0x7f16c004f370 2024-04-25T11:10:58.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f16c6000640 1 -- 172.21.15.7:0/3748734646 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc037070 con 0x7f16c004f370 2024-04-25T11:10:58.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7cb7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7cb8162f10 0x7f7cb80ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7cb7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7cb8162f10 0x7f7cb80ce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48770/0 (socket says 172.21.15.7:48770) 2024-04-25T11:10:58.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7cb7e00640 1 -- 172.21.15.7:0/103595782 learned_addr learned my addr 172.21.15.7:0/103595782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7cb7e00640 1 -- 172.21.15.7:0/103595782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7cb81625c0 msgr2=0x7f7cb81629d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7cb7e00640 1 --2- 172.21.15.7:0/103595782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7cb81625c0 0x7f7cb81629d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.418+0000 7f7cb7e00640 1 -- 172.21.15.7:0/103595782 >> v2:172.21.15.87:3300/0 conn(0x7f7cb80d2210 msgr2=0x7f7cb80d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f7cb7e00640 1 --2- 172.21.15.7:0/103595782 >> v2:172.21.15.87:3300/0 conn(0x7f7cb80d2210 0x7f7cb80d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f7cb7e00640 1 -- 172.21.15.7:0/103595782 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb80710d0 con 0x7f7cb8162f10 2024-04-25T11:10:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f50e7400640 1 -- 172.21.15.7:0/3271268637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e000f040 con 0x7f50ec0e47c0 2024-04-25T11:10:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f50e7400640 1 -- 172.21.15.7:0/3271268637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e000ee50 con 0x7f50ec0e47c0 2024-04-25T11:10:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f50e7400640 1 -- 172.21.15.7:0/3271268637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0013610 con 0x7f50ec0e47c0 2024-04-25T11:10:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f50fa600640 1 -- 172.21.15.7:0/3271268637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ec0e6300 con 0x7f50ec0e47c0 2024-04-25T11:10:58.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7fdcca000640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdcb4074580 0x7fdcb4076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f50fa600640 1 -- 172.21.15.7:0/3271268637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ec0de960 con 0x7f50ec0e47c0 2024-04-25T11:10:58.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.420+0000 7f9f90c00640 1 -- 172.21.15.7:0/1194778784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f700039b0 msgr2=0x7f9f70003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.420+0000 7f9f90c00640 1 --2- 172.21.15.7:0/1194778784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f700039b0 0x7f9f70003e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9f700057e0 tx=0x7f9f70005830 comp rx=0 tx=0).stop 2024-04-25T11:10:58.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.420+0000 7f702a000640 1 -- 172.21.15.7:0/2261202741 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020037070 con 0x7f70240e47c0 2024-04-25T11:10:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f7b53400640 1 -- 172.21.15.7:0/1203052249 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58017070 con 0x7f7b640b7a60 2024-04-25T11:10:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7f7b6a600640 1 -- 172.21.15.7:0/1203052249 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b642068a0 con 0x7f7b640b7a60 2024-04-25T11:10:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.420+0000 7f7b6a600640 1 -- 172.21.15.7:0/1203052249 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b64206e60 con 0x7f7b640b7a60 2024-04-25T11:10:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.419+0000 7fdcca000640 1 -- 172.21.15.7:0/1222750485 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdcc40948d0 con 0x7fdcd0146180 2024-04-25T11:10:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.420+0000 7fdcca000640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] conn(0x7fdcb407dd40 0x7fdcb40801d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.420+0000 7fdcca000640 1 -- 172.21.15.7:0/1222750485 --> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdcc400ee50 con 0x7fdcb407dd40 2024-04-25T11:10:58.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.420+0000 7fdcca000640 1 -- 172.21.15.7:0/1222750485 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdcc40992a0 con 0x7fdcd0146180 2024-04-25T11:10:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f16ce200640 1 -- 172.21.15.7:0/3748734646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16c004f370 msgr2=0x7f16c0033aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7fdcd5e00640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdcb4074580 0x7fdcb4076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f16ce200640 1 --2- 172.21.15.7:0/3748734646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16c004f370 0x7f16c0033aa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0097d0 tx=0x7f16bc029fd0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f9f90c00640 1 -- 172.21.15.7:0/1194778784 shutdown_connections 2024-04-25T11:10:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f9f90c00640 1 --2- 172.21.15.7:0/1194778784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f700039b0 0x7f9f70003e00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f9f90c00640 1 -- 172.21.15.7:0/1194778784 >> 172.21.15.7:0/1194778784 conn(0x7f9f8c0740f0 msgr2=0x7f9f8c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f50e4c00640 1 -- 172.21.15.7:0/3271268637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f50c8001030 con 0x7f50ec0e47c0 2024-04-25T11:10:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f7b6a600640 1 -- 172.21.15.7:0/1203052249 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b64207240 con 0x7f7b640b7a60 2024-04-25T11:10:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f7b53400640 1 -- 172.21.15.7:0/1203052249 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b58005ce0 con 0x7f7b640b7a60 2024-04-25T11:10:58.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f7b53400640 1 -- 172.21.15.7:0/1203052249 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b580072f0 con 0x7f7b640b7a60 2024-04-25T11:10:58.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f702a000640 1 -- 172.21.15.7:0/2261202741 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702002ae40 con 0x7f70240e47c0 2024-04-25T11:10:58.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f702a000640 1 -- 172.21.15.7:0/2261202741 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020032660 con 0x7f70240e47c0 2024-04-25T11:10:58.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f16ce200640 1 -- 172.21.15.7:0/3748734646 shutdown_connections 2024-04-25T11:10:58.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f16ce200640 1 --2- 172.21.15.7:0/3748734646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16c004f370 0x7f16c0033aa0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f16ce200640 1 -- 172.21.15.7:0/3748734646 >> 172.21.15.7:0/3748734646 conn(0x7f16c0054d20 msgr2=0x7f16c0055130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f16ce200640 1 -- 172.21.15.7:0/3748734646 shutdown_connections 2024-04-25T11:10:58.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f16ce200640 1 -- 172.21.15.7:0/3748734646 wait complete. 2024-04-25T11:10:58.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7fdcd6800640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] conn(0x7fdcb407dd40 0x7fdcb40801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f9f90c00640 1 -- 172.21.15.7:0/1194778784 shutdown_connections 2024-04-25T11:10:58.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.421+0000 7f9f90c00640 1 -- 172.21.15.7:0/1194778784 wait complete. 2024-04-25T11:10:58.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.422+0000 7f7cb7e00640 1 --2- 172.21.15.7:0/103595782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb8162f10 0x7f7cb80ce760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8051770 tx=0x7f7cb002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a70079d7880d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.422+0000 7fdcd5e00640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdcb4074580 0x7fdcb4076a90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009aa0 tx=0x7fdcc0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.422+0000 7f1ae4800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1adc0b6790 0x7f1adc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.423+0000 7f1ae4800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1adc0b0430 0x7f1adc0b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.423+0000 7f1ae4800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1adc0b5630 con 0x7f1adc06fba0 2024-04-25T11:10:58.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.423+0000 7f1ae4800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1adc0b5910 con 0x7f1adc0b6790 2024-04-25T11:10:58.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.423+0000 7f1ae4800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1adc0b5bf0 con 0x7f1adc0b0430 2024-04-25T11:10:58.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.423+0000 7f1ae1e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1adc0b6790 0x7f1adc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f7030c00640 1 -- 172.21.15.7:0/2261202741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70240e47c0 msgr2=0x7f7024037b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f7030c00640 1 --2- 172.21.15.7:0/2261202741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70240e47c0 0x7f7024037b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7020009790 tx=0x7f702002a0f0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.423+0000 7f1ae1e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1adc0b6790 0x7f1adc0ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50330/0 (socket says 172.21.15.7:50330) 2024-04-25T11:10:58.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.423+0000 7f1ae1e00640 1 -- 172.21.15.7:0/4211609748 learned_addr learned my addr 172.21.15.7:0/4211609748 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.423+0000 7f1ae1400640 1 --2- 172.21.15.7:0/4211609748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1adc0b0430 0x7f1adc0b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f7030c00640 1 -- 172.21.15.7:0/2261202741 shutdown_connections 2024-04-25T11:10:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f7030c00640 1 --2- 172.21.15.7:0/2261202741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70240384d0 0x7f70240e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f7030c00640 1 --2- 172.21.15.7:0/2261202741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70240e47c0 0x7f7024037b80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f7030c00640 1 -- 172.21.15.7:0/2261202741 >> 172.21.15.7:0/2261202741 conn(0x7f7024059110 msgr2=0x7f7024059520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.423+0000 7f7cb6000640 1 -- 172.21.15.7:0/103595782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0036070 con 0x7f7cb8162f10 2024-04-25T11:10:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.423+0000 7f7cb6000640 1 -- 172.21.15.7:0/103595782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb0031600 con 0x7f7cb8162f10 2024-04-25T11:10:58.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.423+0000 7f7cb6000640 1 -- 172.21.15.7:0/103595782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb00319d0 con 0x7f7cb8162f10 2024-04-25T11:10:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f1ae1e00640 1 -- 172.21.15.7:0/4211609748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1adc0b0430 msgr2=0x7f1adc0b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f7cbe600640 1 -- 172.21.15.7:0/103595782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb8162f10 msgr2=0x7f7cb80ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f7cbe600640 1 --2- 172.21.15.7:0/103595782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb8162f10 0x7f7cb80ce760 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8051770 tx=0x7f7cb002a570 comp rx=0 tx=0).stop 2024-04-25T11:10:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f1ae1e00640 1 --2- 172.21.15.7:0/4211609748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1adc0b0430 0x7f1adc0b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f1ae1e00640 1 -- 172.21.15.7:0/4211609748 >> v1:172.21.15.7:6789/0 conn(0x7f1adc06fba0 legacy=0x7f1adc06ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.425+0000 7fdcd6800640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] conn(0x7fdcb407dd40 0x7fdcb40801d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.425+0000 7f7030c00640 1 -- 172.21.15.7:0/2261202741 shutdown_connections 2024-04-25T11:10:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f16ce200640 1 Processor -- start 2024-04-25T11:10:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.425+0000 7f16ce200640 1 -- start start 2024-04-25T11:10:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f1ae1e00640 1 -- 172.21.15.7:0/4211609748 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1adc06f3a0 con 0x7f1adc0b6790 2024-04-25T11:10:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.425+0000 7f7030c00640 1 -- 172.21.15.7:0/2261202741 wait complete. 2024-04-25T11:10:58.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f7cbe600640 1 -- 172.21.15.7:0/103595782 shutdown_connections 2024-04-25T11:10:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f7cbe600640 1 --2- 172.21.15.7:0/103595782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb8162f10 0x7f7cb80ce760 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.424+0000 7f7cbe600640 1 --2- 172.21.15.7:0/103595782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7cb81625c0 0x7f7cb81629d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.425+0000 7f7cbe600640 1 --2- 172.21.15.7:0/103595782 >> v2:172.21.15.87:3300/0 conn(0x7f7cb80d2210 0x7f7cb80d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.425+0000 7f7cbe600640 1 -- 172.21.15.7:0/103595782 >> 172.21.15.7:0/103595782 conn(0x7f7cb80d03a0 msgr2=0x7f7cb80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.425+0000 7f7cbe600640 1 -- 172.21.15.7:0/103595782 shutdown_connections 2024-04-25T11:10:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.425+0000 7f7cbe600640 1 -- 172.21.15.7:0/103595782 wait complete. 2024-04-25T11:10:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.426+0000 7f7b53400640 1 -- 172.21.15.7:0/1203052249 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7b5801a070 con 0x7f7b640b7a60 2024-04-25T11:10:58.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.426+0000 7fdcca000640 1 -- 172.21.15.7:0/1222750485 <== osd.1 v2:172.21.15.7:6822/1767080083 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdcc400ee50 con 0x7fdcb407dd40 2024-04-25T11:10:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.426+0000 7f9f90c00640 1 Processor -- start 2024-04-25T11:10:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.426+0000 7f1ae1e00640 1 --2- 172.21.15.7:0/4211609748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc0b6790 0x7f1adc0ae880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009870 tx=0x7f1acc02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f51e5054dde28523 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f9f90c00640 1 -- start start 2024-04-25T11:10:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f16ce200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16c0033fe0 0x7f16c00e0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f16ce200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16c004f370 0x7f16c0046c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f16ce200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16c0052fe0 0x7f16c00e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f16ce200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16c0006590 con 0x7f16c0033fe0 2024-04-25T11:10:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f16ce200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f16c00062b0 con 0x7f16c004f370 2024-04-25T11:10:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f16ce200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f16c0006870 con 0x7f16c0052fe0 2024-04-25T11:10:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f50e7400640 1 -- 172.21.15.7:0/3271268637 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f50e0026480 con 0x7f50ec0e47c0 2024-04-25T11:10:58.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f1ae0a00640 1 -- 172.21.15.7:0/4211609748 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc037070 con 0x7f1adc0b6790 2024-04-25T11:10:58.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f1ae0a00640 1 -- 172.21.15.7:0/4211609748 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1acc032440 con 0x7f1adc0b6790 2024-04-25T11:10:58.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f1ae0a00640 1 -- 172.21.15.7:0/4211609748 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc032810 con 0x7f1adc0b6790 2024-04-25T11:10:58.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.427+0000 7f16c7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16c0033fe0 0x7f16c00e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f16c7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16c0033fe0 0x7f16c00e0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48774/0 (socket says 172.21.15.7:48774) 2024-04-25T11:10:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f16c7400640 1 -- 172.21.15.7:0/4181687460 learned_addr learned my addr 172.21.15.7:0/4181687460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f16c6a00640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16c004f370 0x7f16c0046c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f7b53400640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7b480744e0 0x7f7b480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f7b53400640 1 -- 172.21.15.7:0/1203052249 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7b580950c0 con 0x7f7b640b7a60 2024-04-25T11:10:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f7b53400640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] conn(0x7f7b4807dc20 0x7f7b480800b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f7b53400640 1 -- 172.21.15.7:0/1203052249 --> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b5801ee50 con 0x7f7b4807dc20 2024-04-25T11:10:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f7b53400640 1 -- 172.21.15.7:0/1203052249 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b58095470 con 0x7f7b640b7a60 2024-04-25T11:10:58.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f7b62a00640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7b480744e0 0x7f7b480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f1ae4800640 1 -- 172.21.15.7:0/4211609748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc0b6790 msgr2=0x7f1adc0ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f1ae4800640 1 --2- 172.21.15.7:0/4211609748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc0b6790 0x7f1adc0ae880 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009870 tx=0x7f1acc02a210 comp rx=0 tx=0).stop 2024-04-25T11:10:58.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f1ae4800640 1 -- 172.21.15.7:0/4211609748 shutdown_connections 2024-04-25T11:10:58.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f1ae4800640 1 --2- 172.21.15.7:0/4211609748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1adc0b0430 0x7f1adc0b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f1ae4800640 1 --2- 172.21.15.7:0/4211609748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc0b6790 0x7f1adc0ae880 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f1ae4800640 1 -- 172.21.15.7:0/4211609748 >> 172.21.15.7:0/4211609748 conn(0x7f1adc074310 msgr2=0x7f1adc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f7cbe600640 1 Processor -- start 2024-04-25T11:10:58.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f7b63e00640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] conn(0x7f7b4807dc20 0x7f7b480800b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f1ae4800640 1 -- 172.21.15.7:0/4211609748 shutdown_connections 2024-04-25T11:10:58.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f50e7400640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50d4074580 0x7f50d4076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.430+0000 7f50f3400640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50d4074580 0x7f50d4076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.430+0000 7f50e7400640 1 -- 172.21.15.7:0/3271268637 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f50e00956f0 con 0x7f50ec0e47c0 2024-04-25T11:10:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.430+0000 7f50e7400640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] conn(0x7f50d407dd40 0x7f50d40801d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.430+0000 7f50e7400640 1 -- 172.21.15.7:0/3271268637 --> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50d40808a0 con 0x7f50d407dd40 2024-04-25T11:10:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.430+0000 7f50e7400640 1 -- 172.21.15.7:0/3271268637 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50e0095ae0 con 0x7f50ec0e47c0 2024-04-25T11:10:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.430+0000 7f7b62a00640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7b480744e0 0x7f7b480769f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b640d02f0 tx=0x7f7b54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.430+0000 7f7b63e00640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] conn(0x7f7b4807dc20 0x7f7b480800b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f7030c00640 1 Processor -- start 2024-04-25T11:10:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f7030c00640 1 -- start start 2024-04-25T11:10:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.428+0000 7f16c7e00640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16c0052fe0 0x7f16c00e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f16c6a00640 1 -- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16c0052fe0 msgr2=0x7f16c00e46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f16c6a00640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16c0052fe0 0x7f16c00e46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f16c6a00640 1 -- 172.21.15.7:0/4181687460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16c0033fe0 msgr2=0x7f16c00e0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f16c6a00640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16c0033fe0 0x7f16c00e0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f16c6a00640 1 -- 172.21.15.7:0/4181687460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c0171440 con 0x7f16c004f370 2024-04-25T11:10:58.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f1ae4800640 1 -- 172.21.15.7:0/4211609748 wait complete. 2024-04-25T11:10:58.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.430+0000 7f16c6a00640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16c004f370 0x7f16c0046c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f16bc031000 tx=0x7f16bc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.429+0000 7f7cbe600640 1 -- start start 2024-04-25T11:10:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f50f3e00640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] conn(0x7f50d407dd40 0x7f50d40801d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f50f3400640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50d4074580 0x7f50d4076a90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f50e802a150 tx=0x7f50e802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.432+0000 7f50f3e00640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] conn(0x7f50d407dd40 0x7f50d40801d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f7030c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70240380c0 0x7f7024038a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f7030c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70240384d0 0x7f70240d7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f7030c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70240e47c0 0x7f70240e6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f7030c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f70240388e0 con 0x7f70240380c0 2024-04-25T11:10:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f7030c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7024058490 con 0x7f70240e47c0 2024-04-25T11:10:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f7030c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f70240e4bd0 con 0x7f70240384d0 2024-04-25T11:10:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.432+0000 7f702b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70240380c0 0x7f7024038a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f16bb400640 1 -- 172.21.15.7:0/4181687460 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc037040 con 0x7f16c004f370 2024-04-25T11:10:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f16bb400640 1 -- 172.21.15.7:0/4181687460 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc042070 con 0x7f16c004f370 2024-04-25T11:10:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f16bb400640 1 -- 172.21.15.7:0/4181687460 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc03d680 con 0x7f16c004f370 2024-04-25T11:10:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.432+0000 7f702b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70240380c0 0x7f7024038a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48776/0 (socket says 172.21.15.7:48776) 2024-04-25T11:10:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.432+0000 7f702b400640 1 -- 172.21.15.7:0/1276321267 learned_addr learned my addr 172.21.15.7:0/1276321267 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f702b400640 1 -- 172.21.15.7:0/1276321267 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70240384d0 msgr2=0x7f70240d7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f1ae4800640 1 Processor -- start 2024-04-25T11:10:58.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f702be00640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70240e47c0 0x7f70240e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f9f90c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f8c15b910 0x7f9f8c14a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f1ae4800640 1 -- start start 2024-04-25T11:10:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f702b400640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70240384d0 0x7f70240d7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f702b400640 1 -- 172.21.15.7:0/1276321267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70240e47c0 msgr2=0x7f70240e6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f702b400640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70240e47c0 0x7f70240e6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f702b400640 1 -- 172.21.15.7:0/1276321267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7024039400 con 0x7f70240380c0 2024-04-25T11:10:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f9f90c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f8c15a510 0x7f9f8c165880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f9f90c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f8c165dc0 0x7f9f8c15d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f9f90c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c168a30 con 0x7f9f8c15b910 2024-04-25T11:10:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f9f90c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c0d1280 con 0x7f9f8c165dc0 2024-04-25T11:10:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f9f90c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9f8c168bb0 con 0x7f9f8c15a510 2024-04-25T11:10:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.434+0000 7f9f8b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f8c15b910 0x7f9f8c14a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.434+0000 7f9f8b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f8c15b910 0x7f9f8c14a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48792/0 (socket says 172.21.15.7:48792) 2024-04-25T11:10:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.434+0000 7f9f8b400640 1 -- 172.21.15.7:0/1313421467 learned_addr learned my addr 172.21.15.7:0/1313421467 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.434+0000 7f9f8be00640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f8c165dc0 0x7f9f8c15d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.434+0000 7f9f8aa00640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f8c15a510 0x7f9f8c165880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f16ce200640 1 -- 172.21.15.7:0/4181687460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c0171720 con 0x7f16c004f370 2024-04-25T11:10:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.431+0000 7f16ce200640 1 -- 172.21.15.7:0/4181687460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c0172200 con 0x7f16c004f370 2024-04-25T11:10:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f16ce200640 1 -- 172.21.15.7:0/4181687460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16c00054d0 con 0x7f16c004f370 2024-04-25T11:10:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.433+0000 7f7cbe600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb80d2210 0x7f7cb81faf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.434+0000 7f7cbe600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cb81625c0 0x7f7cb8154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.434+0000 7f7cbe600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7cb8162f10 0x7f7cb809f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.434+0000 7f7b53400640 1 -- 172.21.15.7:0/1203052249 <== osd.3 v2:172.21.15.7:6808/1382334542 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b5801ee50 con 0x7f7b4807dc20 2024-04-25T11:10:58.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.434+0000 7f50e7400640 1 -- 172.21.15.7:0/3271268637 <== osd.0 v2:172.21.15.7:6800/3278797987 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f50d40808a0 con 0x7f50d407dd40 2024-04-25T11:10:58.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.434+0000 7f7cbe600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb80d10d0 con 0x7f7cb80d2210 2024-04-25T11:10:58.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.434+0000 7f7cb7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb80d2210 0x7f7cb81faf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f7cb7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb80d2210 0x7f7cb81faf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48804/0 (socket says 172.21.15.7:48804) 2024-04-25T11:10:58.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f7cb6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cb81625c0 0x7f7cb8154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f7cb6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cb81625c0 0x7f7cb8154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50374/0 (socket says 172.21.15.7:50374) 2024-04-25T11:10:58.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f7cb6a00640 1 -- 172.21.15.7:0/3391884162 learned_addr learned my addr 172.21.15.7:0/3391884162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f7cb7e00640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7cb8162f10 0x7f7cb809f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb80b91c0 con 0x7f7cb81625c0 2024-04-25T11:10:58.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb80cfc50 con 0x7f7cb8162f10 2024-04-25T11:10:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f9f8be00640 1 -- 172.21.15.7:0/1313421467 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f8c15a510 msgr2=0x7f9f8c165880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f9f8be00640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f8c15a510 0x7f9f8c165880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f9f8be00640 1 -- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f8c15b910 msgr2=0x7f9f8c14a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f9f8be00640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f8c15b910 0x7f9f8c14a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.435+0000 7f9f8be00640 1 -- 172.21.15.7:0/1313421467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c1661d0 con 0x7f9f8c165dc0 2024-04-25T11:10:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.436+0000 7f702b400640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70240380c0 0x7f7024038a60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7020035040 tx=0x7f70200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.436+0000 7f9f8be00640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f8c165dc0 0x7f9f8c15d4d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c072b90 tx=0x7f9f7c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.436+0000 7f7cb7400640 1 -- 172.21.15.7:0/3391884162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7cb8162f10 msgr2=0x7f7cb809f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.436+0000 7f7cb7400640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7cb8162f10 0x7f7cb809f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.436+0000 7f7cb7400640 1 -- 172.21.15.7:0/3391884162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cb81625c0 msgr2=0x7f7cb8154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.436+0000 7f7cb7400640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cb81625c0 0x7f7cb8154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.436+0000 7f7cb7400640 1 -- 172.21.15.7:0/3391884162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb8155440 con 0x7f7cb80d2210 2024-04-25T11:10:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f9f83400640 1 -- 172.21.15.7:0/1313421467 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c004440 con 0x7f9f8c165dc0 2024-04-25T11:10:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f7cb6a00640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cb81625c0 0x7f7cb8154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:10:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f701f400640 1 -- 172.21.15.7:0/1276321267 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020041020 con 0x7f70240380c0 2024-04-25T11:10:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f1ae4800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1adc06fba0 0x7f1adc1441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f9f83400640 1 -- 172.21.15.7:0/1313421467 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7c01b070 con 0x7f9f8c165dc0 2024-04-25T11:10:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f9f83400640 1 -- 172.21.15.7:0/1313421467 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c003850 con 0x7f9f8c165dc0 2024-04-25T11:10:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f7cb7400640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb80d2210 0x7f7cb81faf80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800b3e0 tx=0x7f7ca800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f16bb400640 1 -- 172.21.15.7:0/4181687460 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f16bc03d820 con 0x7f16c004f370 2024-04-25T11:10:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f701f400640 1 -- 172.21.15.7:0/1276321267 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7020005b90 con 0x7f70240380c0 2024-04-25T11:10:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f701f400640 1 -- 172.21.15.7:0/1276321267 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7020037040 con 0x7f70240380c0 2024-04-25T11:10:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f1ae4800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1adc0b0430 0x7f1adc0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f9f90c00640 1 -- 172.21.15.7:0/1313421467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c15c820 con 0x7f9f8c165dc0 2024-04-25T11:10:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f9f90c00640 1 -- 172.21.15.7:0/1313421467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c15c260 con 0x7f9f8c165dc0 2024-04-25T11:10:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f1ae4800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc0b6790 0x7f1adc144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.438+0000 7f7caf400640 1 -- 172.21.15.7:0/3391884162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8015020 con 0x7f7cb80d2210 2024-04-25T11:10:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.438+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb80b6670 con 0x7f7cb80d2210 2024-04-25T11:10:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.439+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb80b6c30 con 0x7f7cb80d2210 2024-04-25T11:10:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f1ae4800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc1638f0 con 0x7f1adc0b0430 2024-04-25T11:10:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f1ae4800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc0aedc0 con 0x7f1adc0b6790 2024-04-25T11:10:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.437+0000 7f1ae4800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1adc0b0d80 con 0x7f1adc06fba0 2024-04-25T11:10:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.438+0000 7f1ae1e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1adc06fba0 0x7f1adc1441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.438+0000 7f1ae1e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1adc06fba0 0x7f1adc1441b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43480/0 (socket says 172.21.15.7:43480) 2024-04-25T11:10:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.438+0000 7f1ae1e00640 1 -- 172.21.15.7:0/2712707505 learned_addr learned my addr 172.21.15.7:0/2712707505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.439+0000 7f1ae1400640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1adc0b0430 0x7f1adc0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.439+0000 7f7caf400640 1 -- 172.21.15.7:0/3391884162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca8011830 con 0x7f7cb80d2210 2024-04-25T11:10:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.439+0000 7f1ae2800640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc0b6790 0x7f1adc144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.439+0000 7f1ae1e00640 1 -- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc0b6790 msgr2=0x7f1adc144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.439+0000 7f1ae1e00640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc0b6790 0x7f1adc144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.439+0000 7f1ae1e00640 1 -- 172.21.15.7:0/2712707505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1adc0b0430 msgr2=0x7f1adc0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.439+0000 7f1ae1e00640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1adc0b0430 0x7f1adc0d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.439+0000 7f1ae1e00640 1 -- 172.21.15.7:0/2712707505 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1adc1e9870 con 0x7f1adc06fba0 2024-04-25T11:10:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.440+0000 7f1ae2800640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc0b6790 0x7f1adc144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.440+0000 7f1ae1400640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1adc0b0430 0x7f1adc0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:10:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.440+0000 7f7030c00640 1 -- 172.21.15.7:0/1276321267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70240396e0 con 0x7f70240380c0 2024-04-25T11:10:58.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.440+0000 7f1ae1e00640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1adc06fba0 0x7f1adc1441b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009990 tx=0x7f1acc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.440+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7c90001030 con 0x7f7cb80d2210 2024-04-25T11:10:58.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.441+0000 7f9f90c00640 1 -- 172.21.15.7:0/1313421467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f8c15bd20 con 0x7f9f8c165dc0 2024-04-25T11:10:58.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.440+0000 7f7030c00640 1 -- 172.21.15.7:0/1276321267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7024039ca0 con 0x7f70240380c0 2024-04-25T11:10:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.441+0000 7f16bb400640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16a80744e0 0x7f16a80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.441+0000 7f16bb400640 1 -- 172.21.15.7:0/4181687460 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f16bc0b66f0 con 0x7f16c004f370 2024-04-25T11:10:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.441+0000 7f16bb400640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] conn(0x7f16a807dc20 0x7f16a80800b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.441+0000 7f16bb400640 1 -- 172.21.15.7:0/4181687460 --> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16bc02ac60 con 0x7f16a807dc20 2024-04-25T11:10:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.442+0000 7f16bb400640 1 -- 172.21.15.7:0/4181687460 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f16bc0b6aa0 con 0x7f16c004f370 2024-04-25T11:10:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.441+0000 7f7caf400640 1 -- 172.21.15.7:0/3391884162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8015020 con 0x7f7cb80d2210 2024-04-25T11:10:58.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.442+0000 7f16c7400640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16a80744e0 0x7f16a80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.441+0000 7f1ad6000640 1 -- 172.21.15.7:0/2712707505 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc041020 con 0x7f1adc06fba0 2024-04-25T11:10:58.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.441+0000 7f1ad6000640 1 -- 172.21.15.7:0/2712707505 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1acc03d6e0 con 0x7f1adc06fba0 2024-04-25T11:10:58.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.441+0000 7f1ad6000640 1 -- 172.21.15.7:0/2712707505 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc037040 con 0x7f1adc06fba0 2024-04-25T11:10:58.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.443+0000 7f1ae4800640 1 -- 172.21.15.7:0/2712707505 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1adc1e9b50 con 0x7f1adc06fba0 2024-04-25T11:10:58.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.443+0000 7f1ae4800640 1 -- 172.21.15.7:0/2712707505 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1adc0cef90 con 0x7f1adc06fba0 2024-04-25T11:10:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.443+0000 7f16c7400640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16a80744e0 0x7f16a80769f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16b40013b0 tx=0x7f16b4000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.443+0000 7f9f83400640 1 -- 172.21.15.7:0/1313421467 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9f7c0039f0 con 0x7f9f8c165dc0 2024-04-25T11:10:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.443+0000 7f7030c00640 1 -- 172.21.15.7:0/1276321267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f70240058f0 con 0x7f70240380c0 2024-04-25T11:10:58.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.444+0000 7f1ae4800640 1 -- 172.21.15.7:0/2712707505 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1adc1e9d50 con 0x7f1adc06fba0 2024-04-25T11:10:58.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.445+0000 7f9f83400640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f6c0744b0 0x7f9f6c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.445+0000 7f16c7e00640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] conn(0x7f16a807dc20 0x7f16a80800b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.445+0000 7f9f83400640 1 -- 172.21.15.7:0/1313421467 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f9f7c095290 con 0x7f9f8c165dc0 2024-04-25T11:10:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.445+0000 7f9f83400640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] conn(0x7f9f6c07dc70 0x7f9f6c080100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.445+0000 7f9f83400640 1 -- 172.21.15.7:0/1313421467 --> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f6c0807d0 con 0x7f9f6c07dc70 2024-04-25T11:10:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.445+0000 7f9f83400640 1 -- 172.21.15.7:0/1313421467 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9f7c09c050 con 0x7f9f8c165dc0 2024-04-25T11:10:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.445+0000 7f9f8b400640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f6c0744b0 0x7f9f6c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.446+0000 7f9f8aa00640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] conn(0x7f9f6c07dc70 0x7f9f6c080100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.446+0000 7f701f400640 1 -- 172.21.15.7:0/1276321267 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7020044020 con 0x7f70240380c0 2024-04-25T11:10:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.447+0000 7f16c7e00640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] conn(0x7f16a807dc20 0x7f16a80800b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.447+0000 7f7caf400640 1 -- 172.21.15.7:0/3391884162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7ca8018020 con 0x7f7cb80d2210 2024-04-25T11:10:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.447+0000 7f9f8b400640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f6c0744b0 0x7f9f6c0769c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0ceb30 tx=0x7f9f84009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.448+0000 7f9f8aa00640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] conn(0x7f9f6c07dc70 0x7f9f6c080100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.448+0000 7f1ad6000640 1 -- 172.21.15.7:0/2712707505 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1acc044020 con 0x7f1adc06fba0 2024-04-25T11:10:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.448+0000 7f701f400640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f700c0744b0 0x7f700c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.449+0000 7f7caf400640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7c980746e0 0x7f7c98076bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.449+0000 7f701f400640 1 -- 172.21.15.7:0/1276321267 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f70200b5570 con 0x7f70240380c0 2024-04-25T11:10:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.449+0000 7f701f400640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] conn(0x7f700c07dc00 0x7f700c080090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.449+0000 7f701f400640 1 -- 172.21.15.7:0/1276321267 --> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7020043e70 con 0x7f700c07dc00 2024-04-25T11:10:58.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.449+0000 7f701f400640 1 -- 172.21.15.7:0/1276321267 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f70200bc050 con 0x7f70240380c0 2024-04-25T11:10:58.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.449+0000 7f16bb400640 1 -- 172.21.15.7:0/4181687460 <== osd.5 v2:172.21.15.87:6816/2772334579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16bc02ac60 con 0x7f16a807dc20 2024-04-25T11:10:58.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.449+0000 7f7caf400640 1 -- 172.21.15.7:0/3391884162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7ca8095680 con 0x7f7cb80d2210 2024-04-25T11:10:58.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.449+0000 7f7cb6a00640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7c980746e0 0x7f7c98076bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.450+0000 7f7caf400640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] conn(0x7f7c9807de20 0x7f7c980802b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.450+0000 7f7caf400640 1 -- 172.21.15.7:0/3391884162 --> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ca8004450 con 0x7f7c9807de20 2024-04-25T11:10:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.450+0000 7f7caf400640 1 -- 172.21.15.7:0/3391884162 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7ca8095a70 con 0x7f7cb80d2210 2024-04-25T11:10:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.450+0000 7f1ad6000640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ac00744b0 0x7f1ac00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.451+0000 7f9f83400640 1 -- 172.21.15.7:0/1313421467 <== osd.2 v2:172.21.15.7:6816/3878005205 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f6c0807d0 con 0x7f9f6c07dc70 2024-04-25T11:10:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.450+0000 7f702aa00640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f700c0744b0 0x7f700c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.450+0000 7f702be00640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] conn(0x7f700c07dc00 0x7f700c080090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.451+0000 7f7cb7e00640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] conn(0x7f7c9807de20 0x7f7c980802b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.450+0000 7f1ad6000640 1 -- 172.21.15.7:0/2712707505 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1acc0b6140 con 0x7f1adc06fba0 2024-04-25T11:10:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.450+0000 7f1ad6000640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] conn(0x7f1ac007dc00 0x7f1ac0080090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.450+0000 7f1ad6000640 1 -- 172.21.15.7:0/2712707505 --> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1acc007e60 con 0x7f1ac007dc00 2024-04-25T11:10:58.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.450+0000 7f1ad6000640 1 -- 172.21.15.7:0/2712707505 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1acc0b65d0 con 0x7f1adc06fba0 2024-04-25T11:10:58.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.451+0000 7f1ae1400640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ac00744b0 0x7f1ac00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.451+0000 7f1ae2800640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] conn(0x7f1ac007dc00 0x7f1ac0080090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.452+0000 7f702aa00640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f700c0744b0 0x7f700c0769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7018000f60 tx=0x7f7018007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.452+0000 7f702be00640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] conn(0x7f700c07dc00 0x7f700c080090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.453+0000 7f7cb6a00640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7c980746e0 0x7f7c98076bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7cb809c430 tx=0x7f7ca4006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.453+0000 7f7cb7e00640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] conn(0x7f7c9807de20 0x7f7c980802b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.453+0000 7f1ae2800640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] conn(0x7f1ac007dc00 0x7f1ac0080090 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.453+0000 7f1ae1400640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ac00744b0 0x7f1ac00769c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009fd0 tx=0x7f1ad800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.455+0000 7f701f400640 1 -- 172.21.15.7:0/1276321267 <== osd.4 v2:172.21.15.87:6808/1933779372 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7020043e70 con 0x7f700c07dc00 2024-04-25T11:10:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.458+0000 7fdcd7200640 1 -- 172.21.15.7:0/1222750485 --> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdcd0050740 con 0x7fdcb407dd40 2024-04-25T11:10:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.455+0000 7f7caf400640 1 -- 172.21.15.7:0/3391884162 <== osd.6 v2:172.21.15.87:6824/960154076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ca8004450 con 0x7f7c9807de20 2024-04-25T11:10:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.456+0000 7f1ad6000640 1 -- 172.21.15.7:0/2712707505 <== osd.7 v2:172.21.15.87:6800/3856235626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1acc007e60 con 0x7f1ac007dc00 2024-04-25T11:10:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.460+0000 7fdcca000640 1 -- 172.21.15.7:0/1222750485 <== osd.1 v2:172.21.15.7:6822/1767080083 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdcd0050740 con 0x7fdcb407dd40 2024-04-25T11:10:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.460+0000 7fdcd7200640 1 -- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] conn(0x7fdcb407dd40 msgr2=0x7fdcb40801d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.460+0000 7fdcd7200640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] conn(0x7fdcb407dd40 0x7fdcb40801d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.462+0000 7fdcd7200640 1 -- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdcb4074580 msgr2=0x7fdcb4076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.462+0000 7fdcd7200640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdcb4074580 0x7fdcb4076a90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009aa0 tx=0x7fdcc0009190 comp rx=0 tx=0).stop 2024-04-25T11:10:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.463+0000 7fdcd7200640 1 -- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd0146180 msgr2=0x7fdcd0139060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.463+0000 7fdcd7200640 1 --2- 172.21.15.7:0/1222750485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd0146180 0x7fdcd0139060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400c5a0 tx=0x7fdcc400ca60 comp rx=0 tx=0).stop 2024-04-25T11:10:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.463+0000 7fdcd6800640 1 -- 172.21.15.7:0/1222750485 reap_dead start 2024-04-25T11:10:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.463+0000 7fdcd7200640 1 -- 172.21.15.7:0/1222750485 shutdown_connections 2024-04-25T11:10:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.463+0000 7fdcd7200640 1 -- 172.21.15.7:0/1222750485 >> 172.21.15.7:0/1222750485 conn(0x7fdcd00740f0 msgr2=0x7fdcd01489e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.464+0000 7fdcd7200640 1 -- 172.21.15.7:0/1222750485 shutdown_connections 2024-04-25T11:10:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.464+0000 7fdcd7200640 1 -- 172.21.15.7:0/1222750485 wait complete. 2024-04-25T11:10:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.464+0000 7f7b6a600640 1 -- 172.21.15.7:0/1203052249 --> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b64050740 con 0x7f7b4807dc20 2024-04-25T11:10:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.465+0000 7f7b53400640 1 -- 172.21.15.7:0/1203052249 <== osd.3 v2:172.21.15.7:6808/1382334542 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7b64050740 con 0x7f7b4807dc20 2024-04-25T11:10:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.466+0000 7f50e4c00640 1 -- 172.21.15.7:0/3271268637 --> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f50c8002dd0 con 0x7f50d407dd40 2024-04-25T11:10:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.467+0000 7f50e7400640 1 -- 172.21.15.7:0/3271268637 <== osd.0 v2:172.21.15.7:6800/3278797987 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f50c8002dd0 con 0x7f50d407dd40 2024-04-25T11:10:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.467+0000 7f7b6a600640 1 -- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] conn(0x7f7b4807dc20 msgr2=0x7f7b480800b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.467+0000 7f7b6a600640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] conn(0x7f7b4807dc20 0x7f7b480800b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.467+0000 7f50e4c00640 1 -- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] conn(0x7f50d407dd40 msgr2=0x7f50d40801d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.467+0000 7f50e4c00640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] conn(0x7f50d407dd40 0x7f50d40801d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.468+0000 7f50e4c00640 1 -- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50d4074580 msgr2=0x7f50d4076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.468+0000 7f50e4c00640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50d4074580 0x7f50d4076a90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f50e802a150 tx=0x7f50e802e000 comp rx=0 tx=0).stop 2024-04-25T11:10:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.468+0000 7f50e4c00640 1 -- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50ec0e47c0 msgr2=0x7f50ec1898b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.468+0000 7f50e4c00640 1 --2- 172.21.15.7:0/3271268637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50ec0e47c0 0x7f50ec1898b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009db0 tx=0x7f50e000e540 comp rx=0 tx=0).stop 2024-04-25T11:10:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.468+0000 7f50f3e00640 1 -- 172.21.15.7:0/3271268637 reap_dead start 2024-04-25T11:10:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.468+0000 7f50e4c00640 1 -- 172.21.15.7:0/3271268637 shutdown_connections 2024-04-25T11:10:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.468+0000 7f50e4c00640 1 -- 172.21.15.7:0/3271268637 >> 172.21.15.7:0/3271268637 conn(0x7f50ec059110 msgr2=0x7f50ec0e1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.468+0000 7f7b6a600640 1 -- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7b480744e0 msgr2=0x7f7b480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.468+0000 7f7b6a600640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7b480744e0 0x7f7b480769f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b640d02f0 tx=0x7f7b54008040 comp rx=0 tx=0).stop 2024-04-25T11:10:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.468+0000 7f7b6a600640 1 -- 172.21.15.7:0/1203052249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b640b7a60 msgr2=0x7f7b640ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.468+0000 7f7b6a600640 1 --2- 172.21.15.7:0/1203052249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b640b7a60 0x7f7b640ce4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800c3f0 tx=0x7f7b5800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.469+0000 7f7b63e00640 1 -- 172.21.15.7:0/1203052249 reap_dead start 2024-04-25T11:10:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.469+0000 7f7b6a600640 1 -- 172.21.15.7:0/1203052249 shutdown_connections 2024-04-25T11:10:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.469+0000 7f7b6a600640 1 -- 172.21.15.7:0/1203052249 >> 172.21.15.7:0/1203052249 conn(0x7f7b640d2520 msgr2=0x7f7b640b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.469+0000 7f50e4c00640 1 -- 172.21.15.7:0/3271268637 shutdown_connections 2024-04-25T11:10:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.469+0000 7f50e4c00640 1 -- 172.21.15.7:0/3271268637 wait complete. 2024-04-25T11:10:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.470+0000 7f7b6a600640 1 -- 172.21.15.7:0/1203052249 shutdown_connections 2024-04-25T11:10:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.470+0000 7f7b6a600640 1 -- 172.21.15.7:0/1203052249 wait complete. 2024-04-25T11:10:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.484+0000 7f16ce200640 1 -- 172.21.15.7:0/4181687460 --> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f16c0171fc0 con 0x7f16a807dc20 2024-04-25T11:10:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.485+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 --> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7c90002dd0 con 0x7f7c9807de20 2024-04-25T11:10:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.485+0000 7f9f90c00640 1 -- 172.21.15.7:0/1313421467 --> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f8c050740 con 0x7f9f6c07dc70 2024-04-25T11:10:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.486+0000 7f7caf400640 1 -- 172.21.15.7:0/3391884162 <== osd.6 v2:172.21.15.87:6824/960154076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7c90002dd0 con 0x7f7c9807de20 2024-04-25T11:10:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.486+0000 7f9f83400640 1 -- 172.21.15.7:0/1313421467 <== osd.2 v2:172.21.15.7:6816/3878005205 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9f8c050740 con 0x7f9f6c07dc70 2024-04-25T11:10:58.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.486+0000 7f16bb400640 1 -- 172.21.15.7:0/4181687460 <== osd.5 v2:172.21.15.87:6816/2772334579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f16c0171fc0 con 0x7f16a807dc20 2024-04-25T11:10:58.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.487+0000 7f9f90c00640 1 -- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] conn(0x7f9f6c07dc70 msgr2=0x7f9f6c080100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.487+0000 7f9f90c00640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] conn(0x7f9f6c07dc70 0x7f9f6c080100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.487+0000 7f9f90c00640 1 -- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f6c0744b0 msgr2=0x7f9f6c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.487+0000 7f9f90c00640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f6c0744b0 0x7f9f6c0769c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0ceb30 tx=0x7f9f84009030 comp rx=0 tx=0).stop 2024-04-25T11:10:58.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.487+0000 7f9f90c00640 1 -- 172.21.15.7:0/1313421467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f8c165dc0 msgr2=0x7f9f8c15d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.487+0000 7f9f90c00640 1 --2- 172.21.15.7:0/1313421467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f8c165dc0 0x7f9f8c15d4d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c072b90 tx=0x7f9f7c009f90 comp rx=0 tx=0).stop 2024-04-25T11:10:58.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.487+0000 7f9f8be00640 1 -- 172.21.15.7:0/1313421467 reap_dead start 2024-04-25T11:10:58.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.487+0000 7f9f90c00640 1 -- 172.21.15.7:0/1313421467 shutdown_connections 2024-04-25T11:10:58.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.487+0000 7f9f90c00640 1 -- 172.21.15.7:0/1313421467 >> 172.21.15.7:0/1313421467 conn(0x7f9f8c0740f0 msgr2=0x7f9f8c0b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.488+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 >> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] conn(0x7f7c9807de20 msgr2=0x7f7c980802b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.488+0000 7f7cbe600640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] conn(0x7f7c9807de20 0x7f7c980802b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.488+0000 7f9f90c00640 1 -- 172.21.15.7:0/1313421467 shutdown_connections 2024-04-25T11:10:58.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.488+0000 7f16ce200640 1 -- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] conn(0x7f16a807dc20 msgr2=0x7f16a80800b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.488+0000 7f16ce200640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] conn(0x7f16a807dc20 0x7f16a80800b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.488+0000 7f9f90c00640 1 -- 172.21.15.7:0/1313421467 wait complete. 2024-04-25T11:10:58.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.488+0000 7f16ce200640 1 -- 172.21.15.7:0/4181687460 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16a80744e0 msgr2=0x7f16a80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.488+0000 7f16ce200640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16a80744e0 0x7f16a80769f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f16b40013b0 tx=0x7f16b4000ed0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.488+0000 7f16ce200640 1 -- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16c004f370 msgr2=0x7f16c0046c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.488+0000 7f16ce200640 1 --2- 172.21.15.7:0/4181687460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16c004f370 0x7f16c0046c00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f16bc031000 tx=0x7f16bc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.489+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7c980746e0 msgr2=0x7f7c98076bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.489+0000 7f7cbe600640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7c980746e0 0x7f7c98076bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7cb809c430 tx=0x7f7ca4006f90 comp rx=0 tx=0).stop 2024-04-25T11:10:58.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.489+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb80d2210 msgr2=0x7f7cb81faf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.489+0000 7f7cbe600640 1 --2- 172.21.15.7:0/3391884162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7cb80d2210 0x7f7cb81faf80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800b3e0 tx=0x7f7ca800b8a0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.489+0000 7f16c7e00640 1 -- 172.21.15.7:0/4181687460 reap_dead start 2024-04-25T11:10:58.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.489+0000 7f7cb7e00640 1 -- 172.21.15.7:0/3391884162 reap_dead start 2024-04-25T11:10:58.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.489+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 shutdown_connections 2024-04-25T11:10:58.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.489+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 >> 172.21.15.7:0/3391884162 conn(0x7f7cb80d03a0 msgr2=0x7f7cb809bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.489+0000 7f16ce200640 1 -- 172.21.15.7:0/4181687460 shutdown_connections 2024-04-25T11:10:58.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.489+0000 7f16ce200640 1 -- 172.21.15.7:0/4181687460 >> 172.21.15.7:0/4181687460 conn(0x7f16c0054d20 msgr2=0x7f16c00350d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.490+0000 7f16ce200640 1 -- 172.21.15.7:0/4181687460 shutdown_connections 2024-04-25T11:10:58.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.490+0000 7f16ce200640 1 -- 172.21.15.7:0/4181687460 wait complete. 2024-04-25T11:10:58.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.490+0000 7f1ae4800640 1 -- 172.21.15.7:0/2712707505 --> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1adc0b1e30 con 0x7f1ac007dc00 2024-04-25T11:10:58.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.491+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 shutdown_connections 2024-04-25T11:10:58.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.491+0000 7f7cbe600640 1 -- 172.21.15.7:0/3391884162 wait complete. 2024-04-25T11:10:58.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.491+0000 7f1ad6000640 1 -- 172.21.15.7:0/2712707505 <== osd.7 v2:172.21.15.87:6800/3856235626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1adc0b1e30 con 0x7f1ac007dc00 2024-04-25T11:10:58.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.492+0000 7f1ae4800640 1 -- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] conn(0x7f1ac007dc00 msgr2=0x7f1ac0080090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.492+0000 7f1ae4800640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] conn(0x7f1ac007dc00 0x7f1ac0080090 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.492+0000 7f7030c00640 1 -- 172.21.15.7:0/1276321267 --> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f702404c960 con 0x7f700c07dc00 2024-04-25T11:10:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.493+0000 7f1ae4800640 1 -- 172.21.15.7:0/2712707505 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ac00744b0 msgr2=0x7f1ac00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.493+0000 7f1ae4800640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ac00744b0 0x7f1ac00769c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009fd0 tx=0x7f1ad800b040 comp rx=0 tx=0).stop 2024-04-25T11:10:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.493+0000 7f1ae4800640 1 -- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1adc06fba0 msgr2=0x7f1adc1441b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.493+0000 7f1ae4800640 1 --2- 172.21.15.7:0/2712707505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1adc06fba0 0x7f1adc1441b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009990 tx=0x7f1acc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.493+0000 7f1ae2800640 1 -- 172.21.15.7:0/2712707505 reap_dead start 2024-04-25T11:10:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.493+0000 7f1ae4800640 1 -- 172.21.15.7:0/2712707505 shutdown_connections 2024-04-25T11:10:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.493+0000 7f1ae4800640 1 -- 172.21.15.7:0/2712707505 >> 172.21.15.7:0/2712707505 conn(0x7f1adc074310 msgr2=0x7f1adc0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.493+0000 7f1ae4800640 1 -- 172.21.15.7:0/2712707505 shutdown_connections 2024-04-25T11:10:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.494+0000 7f1ae4800640 1 -- 172.21.15.7:0/2712707505 wait complete. 2024-04-25T11:10:58.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.494+0000 7f701f400640 1 -- 172.21.15.7:0/1276321267 <== osd.4 v2:172.21.15.87:6808/1933779372 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f702404c960 con 0x7f700c07dc00 2024-04-25T11:10:58.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.495+0000 7f7030c00640 1 -- 172.21.15.7:0/1276321267 >> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] conn(0x7f700c07dc00 msgr2=0x7f700c080090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.495+0000 7f7030c00640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] conn(0x7f700c07dc00 0x7f700c080090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.497+0000 7f7030c00640 1 -- 172.21.15.7:0/1276321267 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f700c0744b0 msgr2=0x7f700c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.497+0000 7f7030c00640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f700c0744b0 0x7f700c0769c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7018000f60 tx=0x7f7018007a80 comp rx=0 tx=0).stop 2024-04-25T11:10:58.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.497+0000 7f7030c00640 1 -- 172.21.15.7:0/1276321267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70240380c0 msgr2=0x7f7024038a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.497+0000 7f7030c00640 1 --2- 172.21.15.7:0/1276321267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70240380c0 0x7f7024038a60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7020035040 tx=0x7f70200046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.497+0000 7f702be00640 1 -- 172.21.15.7:0/1276321267 reap_dead start 2024-04-25T11:10:58.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.497+0000 7f7030c00640 1 -- 172.21.15.7:0/1276321267 shutdown_connections 2024-04-25T11:10:58.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.497+0000 7f7030c00640 1 -- 172.21.15.7:0/1276321267 >> 172.21.15.7:0/1276321267 conn(0x7f7024059110 msgr2=0x7f70240d9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.498+0000 7f7030c00640 1 -- 172.21.15.7:0/1276321267 shutdown_connections 2024-04-25T11:10:58.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.498+0000 7f7030c00640 1 -- 172.21.15.7:0/1276321267 wait complete. 2024-04-25T11:10:58.515 INFO:teuthology.orchestra.run.smithi007.stdout:64424509452 2024-04-25T11:10:58.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:10:58.515 INFO:teuthology.orchestra.run.smithi007.stdout:64424509451 2024-04-25T11:10:58.516 INFO:teuthology.orchestra.run.smithi007.stdout:64424509451 2024-04-25T11:10:58.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:10:58.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:10:58.522 INFO:teuthology.orchestra.run.smithi007.stdout:68719476747 2024-04-25T11:10:58.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T11:10:58.526 INFO:teuthology.orchestra.run.smithi007.stdout:68719476747 2024-04-25T11:10:58.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T11:10:58.530 INFO:teuthology.orchestra.run.smithi007.stdout:60129542156 2024-04-25T11:10:58.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:10:58.538 INFO:teuthology.orchestra.run.smithi007.stdout:68719476748 2024-04-25T11:10:58.538 INFO:teuthology.orchestra.run.smithi007.stdout:68719476748 2024-04-25T11:10:58.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T11:10:58.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T11:10:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.902+0000 7ff3b8400640 1 Processor -- start 2024-04-25T11:10:58.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.902+0000 7ff3b8400640 1 -- start start 2024-04-25T11:10:58.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.903+0000 7ff3eb800640 1 Processor -- start 2024-04-25T11:10:58.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.903+0000 7ff3eb800640 1 -- start start 2024-04-25T11:10:58.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.906+0000 7ff3b5000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff3a804f330 0x7ff3a8033a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50204/0 (socket says 172.21.15.7:50204) 2024-04-25T11:10:58.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.906+0000 7ff3b5a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff3a8052fa0 0x7ff3a8007310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39268/0 (socket says 172.21.15.7:39268) 2024-04-25T11:10:58.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.905+0000 7f4ba4a00640 1 Processor -- start 2024-04-25T11:10:58.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.906+0000 7f4ba4a00640 1 -- start start 2024-04-25T11:10:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.907+0000 7ff3eb800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3dc054160 0x7ff3dc0ebea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.907+0000 7ff3eb800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff3dc0575a0 0x7ff3dc0579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.906+0000 7ff3b5a00640 1 -- 172.21.15.7:0/1470167301 learned_addr learned my addr 172.21.15.7:0/1470167301 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.907+0000 7ff3b8400640 1 --2- 172.21.15.7:0/1470167301 >> v2:172.21.15.87:3300/0 conn(0x7ff3a8034170 0x7ff3a8038330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.907+0000 7ff3b8400640 1 -- 172.21.15.7:0/1470167301 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a804f740 con 0x7ff3a804f330 2024-04-25T11:10:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.907+0000 7ff3b8400640 1 -- 172.21.15.7:0/1470167301 --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a8007a20 con 0x7ff3a8052fa0 2024-04-25T11:10:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.907+0000 7ff3b8400640 1 -- 172.21.15.7:0/1470167301 --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff3a80533b0 con 0x7ff3a8034170 2024-04-25T11:10:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.907+0000 7ff3eb800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3dc007000 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.907+0000 7ff3eb800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff3dc007c10 con 0x7ff3dc0575a0 2024-04-25T11:10:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.907+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1708061833 0 0) 0x7ff3a804f740 con 0x7ff3a804f330 2024-04-25T11:10:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.907+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff398003520 con 0x7ff3a804f330 2024-04-25T11:10:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.907+0000 7ff3eb800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff3dc0072e0 con 0x7ff3dc054160 2024-04-25T11:10:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.908+0000 7ff3b6400640 1 --2- 172.21.15.7:0/1470167301 >> v2:172.21.15.87:3300/0 conn(0x7ff3a8034170 0x7ff3a8038330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.908+0000 7ff3e8e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff3dc0e57f0 0x7ff3dc0eb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39272/0 (socket says 172.21.15.7:39272) 2024-04-25T11:10:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.908+0000 7ff3e8e00640 1 -- 172.21.15.7:0/1523134532 learned_addr learned my addr 172.21.15.7:0/1523134532 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.908+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2385826542 0 0) 0x7ff398003520 con 0x7ff3a804f330 2024-04-25T11:10:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 >> v2:172.21.15.87:3300/0 conn(0x7ff3a8034170 msgr2=0x7ff3a8038330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 --2- 172.21.15.7:0/1470167301 >> v2:172.21.15.87:3300/0 conn(0x7ff3a8034170 0x7ff3a8038330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 >> v1:172.21.15.87:6789/0 conn(0x7ff3a8052fa0 legacy=0x7ff3a8007310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7f4ba4a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ba01646b0 0x7f4ba00d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7f4ba4a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4ba00d3590 0x7f4ba006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7f4ba4a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ba006eb90 con 0x7f4ba006e780 2024-04-25T11:10:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7f4ba4a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4ba00d5680 con 0x7f4ba01646b0 2024-04-25T11:10:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7f4ba4a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4ba00bb6d0 con 0x7f4ba00d3590 2024-04-25T11:10:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3e9800640 1 --2- 172.21.15.7:0/1523134532 >> v2:172.21.15.87:3300/0 conn(0x7ff3dc0575a0 0x7ff3dc0579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a80e8220 con 0x7ff3a804f330 2024-04-25T11:10:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7ff3a40015a0 con 0x7ff3a804f330 2024-04-25T11:10:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 >> v1:172.21.15.7:6789/0 conn(0x7ff3a804f330 legacy=0x7ff3a8033a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 --2- 172.21.15.7:0/1470167301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3980039b0 0x7ff398003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 --2- 172.21.15.7:0/1470167301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff398004800 0x7ff398004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 --2- 172.21.15.7:0/1470167301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3980053a0 0x7ff39800ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a80533b0 con 0x7ff3980053a0 2024-04-25T11:10:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff39800c330 con 0x7ff398004800 2024-04-25T11:10:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.909+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff39800c530 con 0x7ff3980039b0 2024-04-25T11:10:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.910+0000 7f4b9de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ba01646b0 0x7f4ba00d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.910+0000 7f4b9de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ba01646b0 0x7f4ba00d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48818/0 (socket says 172.21.15.7:48818) 2024-04-25T11:10:58.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.910+0000 7f4b9de00640 1 -- 172.21.15.7:0/3799515420 learned_addr learned my addr 172.21.15.7:0/3799515420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.910+0000 7f4b95400640 1 --2- 172.21.15.7:0/3799515420 >> v2:172.21.15.87:3300/0 conn(0x7f4ba00d3590 0x7f4ba006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.910+0000 7ff3e3400640 1 -- 172.21.15.7:0/1523134532 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497092351 0 0) 0x7ff3dc007000 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.910+0000 7ff3e3400640 1 -- 172.21.15.7:0/1523134532 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3cc003520 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.910+0000 7ff3e3e00640 1 --2- 172.21.15.7:0/1523134532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3dc054160 0x7ff3dc0ebea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.911+0000 7f4b9de00640 1 -- 172.21.15.7:0/3799515420 >> v2:172.21.15.87:3300/0 conn(0x7f4ba00d3590 msgr2=0x7f4ba006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.911+0000 7f4b9de00640 1 --2- 172.21.15.7:0/3799515420 >> v2:172.21.15.87:3300/0 conn(0x7f4ba00d3590 0x7f4ba006e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.911+0000 7f4b9de00640 1 -- 172.21.15.7:0/3799515420 >> v1:172.21.15.87:6789/0 conn(0x7f4ba006e780 legacy=0x7f4ba00b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.911+0000 7f4b9de00640 1 -- 172.21.15.7:0/3799515420 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba00d4a70 con 0x7f4ba01646b0 2024-04-25T11:10:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.911+0000 7ff3b5a00640 1 --2- 172.21.15.7:0/1470167301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3980039b0 0x7ff398003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.911+0000 7ff3b6400640 1 --2- 172.21.15.7:0/1470167301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3980053a0 0x7ff39800ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.911+0000 7ff3b5000640 1 --2- 172.21.15.7:0/1470167301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff398004800 0x7ff398004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.911+0000 7ff3e9800640 1 -- 172.21.15.7:0/1523134532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3dc054160 msgr2=0x7ff3dc0ebea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.911+0000 7ff3e9800640 1 --2- 172.21.15.7:0/1523134532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3dc054160 0x7ff3dc0ebea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.911+0000 7ff3e9800640 1 -- 172.21.15.7:0/1523134532 >> v1:172.21.15.87:6789/0 conn(0x7ff3dc0e57f0 legacy=0x7ff3dc0eb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.911+0000 7ff3e9800640 1 -- 172.21.15.7:0/1523134532 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3dc0075c0 con 0x7ff3dc0575a0 2024-04-25T11:10:58.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.912+0000 7ff3e9800640 1 --2- 172.21.15.7:0/1523134532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3dc0575a0 0x7ff3dc0579b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc007d90 tx=0x7ff3e40a70d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fde787c5df6b67d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.912+0000 7ff3b6400640 1 -- 172.21.15.7:0/1470167301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3980039b0 msgr2=0x7ff398003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.912+0000 7ff3b6400640 1 --2- 172.21.15.7:0/1470167301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3980039b0 0x7ff398003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.912+0000 7ff3b6400640 1 -- 172.21.15.7:0/1470167301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff398004800 msgr2=0x7ff398004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.912+0000 7ff3b6400640 1 --2- 172.21.15.7:0/1470167301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff398004800 0x7ff398004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.912+0000 7ff3b6400640 1 -- 172.21.15.7:0/1470167301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a80e8220 con 0x7ff3980053a0 2024-04-25T11:10:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.913+0000 7ff3e3400640 1 -- 172.21.15.7:0/1523134532 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40b4070 con 0x7ff3dc0575a0 2024-04-25T11:10:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.913+0000 7ff3e3400640 1 -- 172.21.15.7:0/1523134532 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e40a7d10 con 0x7ff3dc0575a0 2024-04-25T11:10:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.913+0000 7ff3e3400640 1 -- 172.21.15.7:0/1523134532 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40b4070 con 0x7ff3dc0575a0 2024-04-25T11:10:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.914+0000 7ff3b6400640 1 -- 172.21.15.7:0/1470167301 reap_dead start 2024-04-25T11:10:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.913+0000 7f4b9de00640 1 --2- 172.21.15.7:0/3799515420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba01646b0 0x7f4ba00d0a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00b8580 tx=0x7f4b9002a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f0cdd3f8f2cdcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.914+0000 7f63e8e00640 1 Processor -- start 2024-04-25T11:10:58.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.914+0000 7ff3b6400640 1 --2- 172.21.15.7:0/1470167301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3980053a0 0x7ff39800ba50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0000b60 tx=0x7ff398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea45bec5e8cc39ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.914+0000 7f4b9d400640 1 -- 172.21.15.7:0/3799515420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90036070 con 0x7f4ba01646b0 2024-04-25T11:10:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.915+0000 7ff3eb800640 1 -- 172.21.15.7:0/1523134532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3dc0575a0 msgr2=0x7ff3dc0579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.914+0000 7f4b9d400640 1 -- 172.21.15.7:0/3799515420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b90031600 con 0x7f4ba01646b0 2024-04-25T11:10:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.914+0000 7f4b9d400640 1 -- 172.21.15.7:0/3799515420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b900319d0 con 0x7f4ba01646b0 2024-04-25T11:10:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.915+0000 7ff3eb800640 1 --2- 172.21.15.7:0/1523134532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3dc0575a0 0x7ff3dc0579b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc007d90 tx=0x7ff3e40a70d0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.915+0000 7ff3eb800640 1 -- 172.21.15.7:0/1523134532 shutdown_connections 2024-04-25T11:10:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.915+0000 7ff3eb800640 1 --2- 172.21.15.7:0/1523134532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3dc0575a0 0x7ff3dc0579b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.915+0000 7ff3eb800640 1 --2- 172.21.15.7:0/1523134532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3dc054160 0x7ff3dc0ebea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.915+0000 7ff3eb800640 1 -- 172.21.15.7:0/1523134532 >> 172.21.15.7:0/1523134532 conn(0x7ff3dc059100 msgr2=0x7ff3dc059510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.915+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00b4070 con 0x7ff3980053a0 2024-04-25T11:10:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.915+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b00a7990 con 0x7ff3980053a0 2024-04-25T11:10:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.915+0000 7ff3afe00640 1 -- 172.21.15.7:0/1470167301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0063ff0 con 0x7ff3980053a0 2024-04-25T11:10:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.914+0000 7f63e8e00640 1 -- start start 2024-04-25T11:10:58.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.915+0000 7ff3eb800640 1 -- 172.21.15.7:0/1523134532 shutdown_connections 2024-04-25T11:10:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.916+0000 7f4ba4a00640 1 -- 172.21.15.7:0/3799515420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba01646b0 msgr2=0x7f4ba00d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.916+0000 7f4ba4a00640 1 --2- 172.21.15.7:0/3799515420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba01646b0 0x7f4ba00d0a00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00b8580 tx=0x7f4b9002a740 comp rx=0 tx=0).stop 2024-04-25T11:10:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7ff3b8400640 1 -- 172.21.15.7:0/1470167301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3980053a0 msgr2=0x7ff39800ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7ff3b8400640 1 --2- 172.21.15.7:0/1470167301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3980053a0 0x7ff39800ba50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0000b60 tx=0x7ff398005830 comp rx=0 tx=0).stop 2024-04-25T11:10:58.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7f4ba4a00640 1 -- 172.21.15.7:0/3799515420 shutdown_connections 2024-04-25T11:10:58.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7f4ba4a00640 1 --2- 172.21.15.7:0/3799515420 >> v2:172.21.15.87:3300/0 conn(0x7f4ba00d3590 0x7f4ba006e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7f4ba4a00640 1 --2- 172.21.15.7:0/3799515420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba01646b0 0x7f4ba00d0a00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7ff3b8400640 1 -- 172.21.15.7:0/1470167301 shutdown_connections 2024-04-25T11:10:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7ff3b8400640 1 --2- 172.21.15.7:0/1470167301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3980053a0 0x7ff39800ba50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7f4ba4a00640 1 -- 172.21.15.7:0/3799515420 >> 172.21.15.7:0/3799515420 conn(0x7f4ba00d26b0 msgr2=0x7f4ba00d2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7ff3b8400640 1 -- 172.21.15.7:0/1470167301 >> 172.21.15.7:0/1470167301 conn(0x7ff3a8054ce0 msgr2=0x7ff3a80550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7f63e8e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63e415ace0 0x7f63e415e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7ff3b8400640 1 -- 172.21.15.7:0/1470167301 shutdown_connections 2024-04-25T11:10:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.918+0000 7f4ba4a00640 1 -- 172.21.15.7:0/3799515420 shutdown_connections 2024-04-25T11:10:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.917+0000 7f63e8e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63e40b2d90 0x7f63e40d3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.918+0000 7f63e8e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 0x7f63e409b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.918+0000 7f63e8e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f63e40d2c80 con 0x7f63e40b2d90 2024-04-25T11:10:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.918+0000 7f63e8e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f63e409be90 con 0x7f63e415ace0 2024-04-25T11:10:58.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.915+0000 7ff3eb800640 1 -- 172.21.15.7:0/1523134532 wait complete. 2024-04-25T11:10:58.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.918+0000 7f63e8e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63e40d3650 con 0x7f63e409b540 2024-04-25T11:10:58.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.918+0000 7ff3b8400640 1 -- 172.21.15.7:0/1470167301 wait complete. 2024-04-25T11:10:58.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.919+0000 7f63e1e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63e415ace0 0x7f63e415e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.919+0000 7ff3eb800640 1 Processor -- start 2024-04-25T11:10:58.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.919+0000 7f63e1e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63e415ace0 0x7f63e415e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50428/0 (socket says 172.21.15.7:50428) 2024-04-25T11:10:58.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.919+0000 7f63e2800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 0x7f63e409b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.919+0000 7f63e1400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63e40b2d90 0x7f63e40d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.920+0000 7ff3eb800640 1 -- start start 2024-04-25T11:10:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.918+0000 7f4ba4a00640 1 -- 172.21.15.7:0/3799515420 wait complete. 2024-04-25T11:10:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.920+0000 7f63e2800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 0x7f63e409b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43508/0 (socket says 172.21.15.7:43508) 2024-04-25T11:10:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.920+0000 7f63e2800640 1 -- 172.21.15.7:0/4066265603 learned_addr learned my addr 172.21.15.7:0/4066265603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.921+0000 7f63e1400640 1 -- 172.21.15.7:0/4066265603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 msgr2=0x7f63e409b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.922+0000 7ff3b8400640 1 Processor -- start 2024-04-25T11:10:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.921+0000 7f63e1400640 1 --2- 172.21.15.7:0/4066265603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 0x7f63e409b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.923+0000 7ff3b8400640 1 -- start start 2024-04-25T11:10:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.923+0000 7ff3eb800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3dc054160 0x7ff3dc0d90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.923+0000 7f4ba4a00640 1 Processor -- start 2024-04-25T11:10:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.921+0000 7f63e1400640 1 -- 172.21.15.7:0/4066265603 >> v2:172.21.15.87:3300/0 conn(0x7f63e415ace0 msgr2=0x7f63e415e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.921+0000 7f63e1400640 1 --2- 172.21.15.7:0/4066265603 >> v2:172.21.15.87:3300/0 conn(0x7f63e415ace0 0x7f63e415e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.923+0000 7ff3eb800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3dc0575a0 0x7ff3dc0dc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.921+0000 7f63e1400640 1 -- 172.21.15.7:0/4066265603 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e40b3530 con 0x7f63e40b2d90 2024-04-25T11:10:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.923+0000 7f4ba4a00640 1 -- start start 2024-04-25T11:10:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.924+0000 7ff3eb800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3dc0e57f0 0x7ff3dc0dec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.923+0000 7f63e1e00640 1 --2- 172.21.15.7:0/4066265603 >> v2:172.21.15.87:3300/0 conn(0x7f63e415ace0 0x7f63e415e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.924+0000 7f63e1400640 1 --2- 172.21.15.7:0/4066265603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e40b2d90 0x7f63e40d3110 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63e40b7b70 tx=0x7f63d002a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c145038e301f26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.924+0000 7ff3e8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3dc054160 0x7ff3dc0d90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.924+0000 7ff3eb800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc0ec3e0 con 0x7ff3dc054160 2024-04-25T11:10:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.924+0000 7ff3eb800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff3dc054570 con 0x7ff3dc0575a0 2024-04-25T11:10:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.924+0000 7ff3eb800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff3dc0ec560 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.924+0000 7ff3e3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3dc0575a0 0x7ff3dc0dc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.924+0000 7ff3e3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3dc0575a0 0x7ff3dc0dc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50436/0 (socket says 172.21.15.7:50436) 2024-04-25T11:10:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.924+0000 7ff3e3e00640 1 -- 172.21.15.7:0/1093077544 learned_addr learned my addr 172.21.15.7:0/1093077544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.925+0000 7ff3e9800640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3dc0e57f0 0x7ff3dc0dec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.925+0000 7f63e0a00640 1 -- 172.21.15.7:0/4066265603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0037070 con 0x7f63e40b2d90 2024-04-25T11:10:58.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3e9800640 1 -- 172.21.15.7:0/1093077544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3dc0575a0 msgr2=0x7ff3dc0dc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3e9800640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3dc0575a0 0x7ff3dc0dc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3e9800640 1 -- 172.21.15.7:0/1093077544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3dc054160 msgr2=0x7ff3dc0d90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3e9800640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3dc054160 0x7ff3dc0d90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3b8400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3980053a0 0x7ff3a80d8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3e9800640 1 -- 172.21.15.7:0/1093077544 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3dc0d9ff0 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3e8e00640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3dc054160 0x7ff3dc0d90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:10:58.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3b8400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3a8048ee0 0x7ff3a80492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3b8400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3a8046800 0x7ff3a8046c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.925+0000 7f63e0a00640 1 -- 172.21.15.7:0/4066265603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63d0032440 con 0x7f63e40b2d90 2024-04-25T11:10:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7ff3e9800640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3dc0e57f0 0x7ff3dc0dec40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4069e70 tx=0x7ff3e406b1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.925+0000 7f63e0a00640 1 -- 172.21.15.7:0/4066265603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d002ae60 con 0x7f63e40b2d90 2024-04-25T11:10:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3b8400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8006830 con 0x7ff3a8046800 2024-04-25T11:10:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3b8400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8006550 con 0x7ff3a8048ee0 2024-04-25T11:10:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7ff3b8400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a8006b10 con 0x7ff3980053a0 2024-04-25T11:10:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7f63e8e00640 1 -- 172.21.15.7:0/4066265603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e40b2d90 msgr2=0x7f63e40d3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7f4ba4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ba006e780 0x7f4ba01569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.926+0000 7f63e8e00640 1 --2- 172.21.15.7:0/4066265603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e40b2d90 0x7f63e40d3110 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63e40b7b70 tx=0x7f63d002a320 comp rx=0 tx=0).stop 2024-04-25T11:10:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7f4ba4a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ba00d3590 0x7f4ba0156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7f4ba4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba01646b0 0x7f4ba0157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7f4ba4a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba00d5310 con 0x7f4ba01646b0 2024-04-25T11:10:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7f4ba4a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba00d5030 con 0x7f4ba006e780 2024-04-25T11:10:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7f4ba4a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba00b9a60 con 0x7f4ba00d3590 2024-04-25T11:10:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7ff3b5a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3980053a0 0x7ff3a80d8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7ff3b5000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3a8048ee0 0x7ff3a80492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7ff3b6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3a8046800 0x7ff3a8046c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7f63e8e00640 1 -- 172.21.15.7:0/4066265603 shutdown_connections 2024-04-25T11:10:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7f63e8e00640 1 --2- 172.21.15.7:0/4066265603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 0x7f63e409b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7f63e8e00640 1 --2- 172.21.15.7:0/4066265603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e40b2d90 0x7f63e40d3110 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7f63e8e00640 1 --2- 172.21.15.7:0/4066265603 >> v2:172.21.15.87:3300/0 conn(0x7f63e415ace0 0x7f63e415e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.927+0000 7f63e8e00640 1 -- 172.21.15.7:0/4066265603 >> 172.21.15.7:0/4066265603 conn(0x7f63e40b7560 msgr2=0x7f63e40b7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.928+0000 7ff3b5a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3980053a0 0x7ff3a80d8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43514/0 (socket says 172.21.15.7:43514) 2024-04-25T11:10:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.928+0000 7ff3e0c00640 1 -- 172.21.15.7:0/1093077544 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40b4040 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.928+0000 7ff3b5a00640 1 -- 172.21.15.7:0/2903955169 learned_addr learned my addr 172.21.15.7:0/2903955169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.928+0000 7ff3eb800640 1 -- 172.21.15.7:0/1093077544 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3dc0da2d0 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.928+0000 7ff3b6400640 1 -- 172.21.15.7:0/2903955169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3980053a0 msgr2=0x7ff3a80d8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.929+0000 7ff3e0c00640 1 -- 172.21.15.7:0/1093077544 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e40a7b10 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.928+0000 7ff3b6400640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3980053a0 0x7ff3a80d8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.928+0000 7ff3b6400640 1 -- 172.21.15.7:0/2903955169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3a8048ee0 msgr2=0x7ff3a80492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.928+0000 7ff3b6400640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3a8048ee0 0x7ff3a80492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.928+0000 7ff3b6400640 1 -- 172.21.15.7:0/2903955169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a8047150 con 0x7ff3a8046800 2024-04-25T11:10:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.929+0000 7ff3eb800640 1 -- 172.21.15.7:0/1093077544 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3dc0da890 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.929+0000 7ff3e0c00640 1 -- 172.21.15.7:0/1093077544 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40b8640 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.929+0000 7f4b95400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ba00d3590 0x7f4ba0156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.929+0000 7f4b9de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ba006e780 0x7f4ba01569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.929+0000 7f4b95400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ba00d3590 0x7f4ba0156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43516/0 (socket says 172.21.15.7:43516) 2024-04-25T11:10:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.930+0000 7ff3b6400640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3a8046800 0x7ff3a8046c10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0066010 tx=0x7ff3b0069570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.929+0000 7f4b95400640 1 -- 172.21.15.7:0/1675074108 learned_addr learned my addr 172.21.15.7:0/1675074108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.929+0000 7f4b9e800640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba01646b0 0x7f4ba0157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.930+0000 7ff3ad600640 1 -- 172.21.15.7:0/2903955169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00b4070 con 0x7ff3a8046800 2024-04-25T11:10:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.930+0000 7ff3ad600640 1 -- 172.21.15.7:0/2903955169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b00b5040 con 0x7ff3a8046800 2024-04-25T11:10:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.930+0000 7ff3ad600640 1 -- 172.21.15.7:0/2903955169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00b4070 con 0x7ff3a8046800 2024-04-25T11:10:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.931+0000 7f63e8e00640 1 -- 172.21.15.7:0/4066265603 shutdown_connections 2024-04-25T11:10:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.930+0000 7f4b9de00640 1 -- 172.21.15.7:0/1675074108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ba00d3590 msgr2=0x7f4ba0156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.930+0000 7f4b9de00640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ba00d3590 0x7f4ba0156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.930+0000 7f4b9de00640 1 -- 172.21.15.7:0/1675074108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba01646b0 msgr2=0x7f4ba0157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.931+0000 7f4b9de00640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba01646b0 0x7f4ba0157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.931+0000 7f4b9de00640 1 -- 172.21.15.7:0/1675074108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba01523e0 con 0x7f4ba006e780 2024-04-25T11:10:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.931+0000 7f4b9e800640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba01646b0 0x7f4ba0157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:10:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.932+0000 7ff3b8400640 1 -- 172.21.15.7:0/2903955169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a8049a10 con 0x7ff3a8046800 2024-04-25T11:10:58.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.932+0000 7ff3b8400640 1 -- 172.21.15.7:0/2903955169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a8180ec0 con 0x7ff3a8046800 2024-04-25T11:10:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.933+0000 7f4b9de00640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ba006e780 0x7f4ba01569b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b90030000 tx=0x7f4b90004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.934+0000 7ff3b8400640 1 -- 172.21.15.7:0/2903955169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a8049830 con 0x7ff3a8046800 2024-04-25T11:10:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.934+0000 7ff3e0c00640 1 -- 172.21.15.7:0/1093077544 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff3e40bf030 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.934+0000 7ff3eb800640 1 -- 172.21.15.7:0/1093077544 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3dc0058e0 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.931+0000 7f63e8e00640 1 -- 172.21.15.7:0/4066265603 wait complete. 2024-04-25T11:10:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.935+0000 7f63e8e00640 1 Processor -- start 2024-04-25T11:10:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.935+0000 7f63e8e00640 1 -- start start 2024-04-25T11:10:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.936+0000 7ff3e0c00640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3c4074580 0x7ff3c4076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.936+0000 7ff3ad600640 1 -- 172.21.15.7:0/2903955169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff3b00c1060 con 0x7ff3a8046800 2024-04-25T11:10:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.936+0000 7ff3e0c00640 1 -- 172.21.15.7:0/1093077544 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff3e4131850 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.937+0000 7ff3e8e00640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3c4074580 0x7ff3c4076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.938+0000 7f63e8e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 0x7f63e414b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.938+0000 7ff3ad600640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3900746b0 0x7ff390076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.939+0000 7f63e8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e40b2d90 0x7f63e4154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.939+0000 7f4b96a00640 1 -- 172.21.15.7:0/1675074108 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90036050 con 0x7f4ba006e780 2024-04-25T11:10:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.939+0000 7f4b96a00640 1 -- 172.21.15.7:0/1675074108 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b90037040 con 0x7f4ba006e780 2024-04-25T11:10:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.939+0000 7f4b96a00640 1 -- 172.21.15.7:0/1675074108 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90009510 con 0x7f4ba006e780 2024-04-25T11:10:58.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.939+0000 7f63e8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e415ace0 0x7f63e4155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.939+0000 7ff3e8e00640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3c4074580 0x7ff3c4076a90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8005880 tx=0x7ff3d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.939+0000 7f4ba4a00640 1 -- 172.21.15.7:0/1675074108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba0157a30 con 0x7f4ba006e780 2024-04-25T11:10:58.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.939+0000 7f4ba4a00640 1 -- 172.21.15.7:0/1675074108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba0155bf0 con 0x7f4ba006e780 2024-04-25T11:10:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.939+0000 7ff3ad600640 1 -- 172.21.15.7:0/2903955169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff3b0130c00 con 0x7ff3a8046800 2024-04-25T11:10:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.940+0000 7f63e8e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40d2910 con 0x7f63e415ace0 2024-04-25T11:10:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.940+0000 7f63e8e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40d2630 con 0x7f63e40b2d90 2024-04-25T11:10:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.940+0000 7f63e8e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63e40b31a0 con 0x7f63e409b540 2024-04-25T11:10:58.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.940+0000 7f63e1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e40b2d90 0x7f63e4154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.939+0000 7f63e1e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 0x7f63e414b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.940+0000 7f63e1e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 0x7f63e414b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43518/0 (socket says 172.21.15.7:43518) 2024-04-25T11:10:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.940+0000 7f63e1e00640 1 -- 172.21.15.7:0/135237321 learned_addr learned my addr 172.21.15.7:0/135237321 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.941+0000 7ff3b5a00640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3900746b0 0x7ff390076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.941+0000 7f4ba4a00640 1 -- 172.21.15.7:0/1675074108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba00747d0 con 0x7f4ba006e780 2024-04-25T11:10:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.941+0000 7f63e2800640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e415ace0 0x7f63e4155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.941+0000 7f63e1e00640 1 -- 172.21.15.7:0/135237321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e40b2d90 msgr2=0x7f63e4154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.941+0000 7f63e1e00640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e40b2d90 0x7f63e4154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.941+0000 7f63e1e00640 1 -- 172.21.15.7:0/135237321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e415ace0 msgr2=0x7f63e4155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.942+0000 7f63e1e00640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e415ace0 0x7f63e4155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.942+0000 7f63e1e00640 1 -- 172.21.15.7:0/135237321 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e40d0100 con 0x7f63e409b540 2024-04-25T11:10:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.942+0000 7ff3b5a00640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3900746b0 0x7ff390076bc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff39c007110 tx=0x7ff39c004910 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.942+0000 7f63e2800640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e415ace0 0x7f63e4155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:10:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.943+0000 7f63e1e00640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 0x7f63e414b2d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00d7d0 tx=0x7f63dc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.945+0000 7ff3ad600640 1 -- 172.21.15.7:0/2903955169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3a8049830 con 0x7ff3a8046800 2024-04-25T11:10:58.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.945+0000 7f4b96a00640 1 -- 172.21.15.7:0/1675074108 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4b90044020 con 0x7f4ba006e780 2024-04-25T11:10:58.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.947+0000 7f4b96a00640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b7c074500 0x7f4b7c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.947+0000 7f4b96a00640 1 -- 172.21.15.7:0/1675074108 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4b900b61e0 con 0x7f4ba006e780 2024-04-25T11:10:58.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.948+0000 7f4b95400640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b7c074500 0x7f4b7c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.949+0000 7f4b95400640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b7c074500 0x7f4b7c076a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00b9220 tx=0x7f4b88009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.954+0000 7f63da000640 1 -- 172.21.15.7:0/135237321 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc004030 con 0x7f63e409b540 2024-04-25T11:10:58.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.955+0000 7f63da000640 1 -- 172.21.15.7:0/135237321 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63dc0041d0 con 0x7f63e409b540 2024-04-25T11:10:58.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.955+0000 7f63da000640 1 -- 172.21.15.7:0/135237321 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0096a0 con 0x7f63e409b540 2024-04-25T11:10:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.955+0000 7ff3e0c00640 1 -- 172.21.15.7:0/1093077544 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3e40c6020 con 0x7ff3dc0e57f0 2024-04-25T11:10:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.955+0000 7f63e8e00640 1 -- 172.21.15.7:0/135237321 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e40cf350 con 0x7f63e409b540 2024-04-25T11:10:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.955+0000 7f63e8e00640 1 -- 172.21.15.7:0/135237321 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e40cf910 con 0x7f63e409b540 2024-04-25T11:10:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.957+0000 7f63e8e00640 1 -- 172.21.15.7:0/135237321 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e40d02d0 con 0x7f63e409b540 2024-04-25T11:10:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.960+0000 7f63da000640 1 -- 172.21.15.7:0/135237321 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f63dc005990 con 0x7f63e409b540 2024-04-25T11:10:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.961+0000 7f63da000640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63c40745b0 0x7f63c4076ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.961+0000 7f4b96a00640 1 -- 172.21.15.7:0/1675074108 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ba00747d0 con 0x7f4ba006e780 2024-04-25T11:10:58.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.961+0000 7f63da000640 1 -- 172.21.15.7:0/135237321 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f63dc095b90 con 0x7f63e409b540 2024-04-25T11:10:58.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.962+0000 7f63e1400640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63c40745b0 0x7f63c4076ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.963+0000 7f63e1400640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63c40745b0 0x7f63c4076ac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f63d0034000 tx=0x7f63d002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:58.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.973+0000 7f63da000640 1 -- 172.21.15.7:0/135237321 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63dc05fa00 con 0x7f63e409b540 2024-04-25T11:10:58.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.991+0000 7f4e34e00640 1 Processor -- start 2024-04-25T11:10:58.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.991+0000 7f4e34e00640 1 -- start start 2024-04-25T11:10:58.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.995+0000 7f4e34e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3006e5e0 0x7f4e30161130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:58.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.995+0000 7f4e2de00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f4e300b5130 0x7f4e3006ded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39288/0 (socket says 172.21.15.7:39288) 2024-04-25T11:10:58.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.995+0000 7f4e2de00640 1 -- 172.21.15.7:0/1025781446 learned_addr learned my addr 172.21.15.7:0/1025781446 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:58.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.996+0000 7f4e34e00640 1 -- 172.21.15.7:0/1025781446 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e30163e80 con 0x7f4e3015f7d0 2024-04-25T11:10:58.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.996+0000 7f4e34e00640 1 -- 172.21.15.7:0/1025781446 --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e30164160 con 0x7f4e300b5130 2024-04-25T11:10:58.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.996+0000 7fe5e2800640 1 Processor -- start 2024-04-25T11:10:58.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.996+0000 7f4e2d400640 1 --2- 172.21.15.7:0/1025781446 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3006e5e0 0x7f4e30161130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.996+0000 7f4e34e00640 1 -- 172.21.15.7:0/1025781446 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e30163c30 con 0x7f4e3006e5e0 2024-04-25T11:10:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.997+0000 7f4e2ca00640 1 -- 172.21.15.7:0/1025781446 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2650026194 0 0) 0x7f4e30164160 con 0x7f4e300b5130 2024-04-25T11:10:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.997+0000 7f4e2ca00640 1 -- 172.21.15.7:0/1025781446 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e14003520 con 0x7f4e300b5130 2024-04-25T11:10:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.997+0000 7f4e2ca00640 1 -- 172.21.15.7:0/1025781446 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109119533 0 0) 0x7f4e30163e80 con 0x7f4e3015f7d0 2024-04-25T11:10:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.997+0000 7f4e2ca00640 1 -- 172.21.15.7:0/1025781446 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e14003880 con 0x7f4e3015f7d0 2024-04-25T11:10:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.997+0000 7f4e2d400640 1 -- 172.21.15.7:0/1025781446 >> v1:172.21.15.87:6789/0 conn(0x7f4e300b5130 legacy=0x7f4e3006ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.997+0000 7f4e2d400640 1 -- 172.21.15.7:0/1025781446 >> v1:172.21.15.7:6789/0 conn(0x7f4e3015f7d0 legacy=0x7f4e3015fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.997+0000 7f4e2d400640 1 -- 172.21.15.7:0/1025781446 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e30164440 con 0x7f4e3006e5e0 2024-04-25T11:10:59.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:58.997+0000 7fe5e2800640 1 -- start start 2024-04-25T11:10:59.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.000+0000 7fe5e2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe5dc162330 0x7fe5dc06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.000+0000 7f4e2d400640 1 --2- 172.21.15.7:0/1025781446 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3006e5e0 0x7f4e30161130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4e200097d0 tx=0x7f4e2002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f38cbb94eaaf4407 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.001+0000 7fe5e2800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5dc070aa0 0x7fe5dc0b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.001+0000 7f4e2ca00640 1 -- 172.21.15.7:0/1025781446 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20037070 con 0x7f4e3006e5e0 2024-04-25T11:10:59.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.001+0000 7fe5db400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe5dc162330 0x7fe5dc06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.001+0000 7fe5daa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe5dc06dac0 0x7fe5dc070390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39304/0 (socket says 172.21.15.7:39304) 2024-04-25T11:10:59.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.001+0000 7fe5daa00640 1 -- 172.21.15.7:0/3001091508 learned_addr learned my addr 172.21.15.7:0/3001091508 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:10:59.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.001+0000 7fe5e2800640 1 -- 172.21.15.7:0/3001091508 --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5dc06ded0 con 0x7fe5dc06dac0 2024-04-25T11:10:59.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.002+0000 7fe5e2800640 1 -- 172.21.15.7:0/3001091508 --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe5dc070eb0 con 0x7fe5dc162330 2024-04-25T11:10:59.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.002+0000 7fb028600640 1 Processor -- start 2024-04-25T11:10:59.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.001+0000 7f4e2ca00640 1 -- 172.21.15.7:0/1025781446 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e20032440 con 0x7f4e3006e5e0 2024-04-25T11:10:59.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.002+0000 7f4e2ca00640 1 -- 172.21.15.7:0/1025781446 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20037070 con 0x7f4e3006e5e0 2024-04-25T11:10:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.002+0000 7fe5dbe00640 1 --2- 172.21.15.7:0/3001091508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5dc070aa0 0x7fe5dc0b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.002+0000 7fe5e2800640 1 -- 172.21.15.7:0/3001091508 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe5dc165b20 con 0x7fe5dc070aa0 2024-04-25T11:10:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.002+0000 7f4e34e00640 1 -- 172.21.15.7:0/1025781446 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3006e5e0 msgr2=0x7f4e30161130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.002+0000 7f4e34e00640 1 --2- 172.21.15.7:0/1025781446 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3006e5e0 0x7f4e30161130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4e200097d0 tx=0x7f4e2002a210 comp rx=0 tx=0).stop 2024-04-25T11:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.002+0000 7f3de2200640 1 Processor -- start 2024-04-25T11:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.003+0000 7f3de2200640 1 -- start start 2024-04-25T11:10:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.003+0000 7f4e34e00640 1 -- 172.21.15.7:0/1025781446 shutdown_connections 2024-04-25T11:10:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.003+0000 7f4e34e00640 1 --2- 172.21.15.7:0/1025781446 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3006e5e0 0x7f4e30161130 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.003+0000 7f4e34e00640 1 -- 172.21.15.7:0/1025781446 >> 172.21.15.7:0/1025781446 conn(0x7f4e300b7ad0 msgr2=0x7f4e300b7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.003+0000 7fe5db400640 1 -- 172.21.15.7:0/3001091508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5dc070aa0 msgr2=0x7fe5dc0b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.003+0000 7fe5db400640 1 --2- 172.21.15.7:0/3001091508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5dc070aa0 0x7fe5dc0b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.003+0000 7fe5db400640 1 -- 172.21.15.7:0/3001091508 >> v1:172.21.15.87:6789/0 conn(0x7fe5dc06dac0 legacy=0x7fe5dc070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.003+0000 7fe5db400640 1 -- 172.21.15.7:0/3001091508 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc06f2f0 con 0x7fe5dc162330 2024-04-25T11:10:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.005+0000 7f4e34e00640 1 -- 172.21.15.7:0/1025781446 shutdown_connections 2024-04-25T11:10:59.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.005+0000 7fe5db400640 1 --2- 172.21.15.7:0/3001091508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc162330 0x7fe5dc06d580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80097a0 tx=0x7fe5c802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e184e37230d0ce6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.005+0000 7f4e34e00640 1 -- 172.21.15.7:0/1025781446 wait complete. 2024-04-25T11:10:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.003+0000 7fb028600640 1 -- start start 2024-04-25T11:10:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.006+0000 7f3de2200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd403a510 0x7f3dd40e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.006+0000 7f3de2200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3dd4038c20 0x7f3dd40064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.006+0000 7f3de2200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd4050930 con 0x7f3dd4034140 2024-04-25T11:10:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.006+0000 7f3de2200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd4034550 con 0x7f3dd4038c20 2024-04-25T11:10:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.006+0000 7f3de2200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd403aaa0 con 0x7f3dd403a510 2024-04-25T11:10:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.006+0000 7f3de0e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd403a510 0x7f3dd40e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.006+0000 7f3de0e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd403a510 0x7f3dd40e16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43548/0 (socket says 172.21.15.7:43548) 2024-04-25T11:10:59.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.006+0000 7f3de0e00640 1 -- 172.21.15.7:0/2514257854 learned_addr learned my addr 172.21.15.7:0/2514257854 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:59.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.006+0000 7fe5da000640 1 -- 172.21.15.7:0/3001091508 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8037070 con 0x7fe5dc162330 2024-04-25T11:10:59.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.006+0000 7fe5da000640 1 -- 172.21.15.7:0/3001091508 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c8032440 con 0x7fe5dc162330 2024-04-25T11:10:59.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.007+0000 7fe5da000640 1 -- 172.21.15.7:0/3001091508 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8032850 con 0x7fe5dc162330 2024-04-25T11:10:59.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.006+0000 7fb028600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0200b4e30 0x7fb0200bb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.007+0000 7fb028600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0200b9470 0x7fb0200b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.007+0000 7fb028600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0200b7650 0x7fb0200b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.007+0000 7fb028600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb0200d0b50 con 0x7fb0200b9470 2024-04-25T11:10:59.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.007+0000 7fb028600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb02008e3a0 con 0x7fb0200b4e30 2024-04-25T11:10:59.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.007+0000 7fb028600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb0200bbb90 con 0x7fb0200b7650 2024-04-25T11:10:59.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.009+0000 7fe5e2800640 1 -- 172.21.15.7:0/3001091508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc162330 msgr2=0x7fe5dc06d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.009+0000 7fe5e2800640 1 --2- 172.21.15.7:0/3001091508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc162330 0x7fe5dc06d580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80097a0 tx=0x7fe5c802a1c0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.009+0000 7fe5e2800640 1 -- 172.21.15.7:0/3001091508 shutdown_connections 2024-04-25T11:10:59.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.009+0000 7fe5e2800640 1 --2- 172.21.15.7:0/3001091508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5dc070aa0 0x7fe5dc0b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.009+0000 7fe5e2800640 1 --2- 172.21.15.7:0/3001091508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc162330 0x7fe5dc06d580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.009+0000 7fe5e2800640 1 -- 172.21.15.7:0/3001091508 >> 172.21.15.7:0/3001091508 conn(0x7fe5dc0732b0 msgr2=0x7fe5dc0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.009+0000 7f4e34e00640 1 Processor -- start 2024-04-25T11:10:59.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.009+0000 7f4e34e00640 1 -- start start 2024-04-25T11:10:59.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.007+0000 7fb025c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0200b4e30 0x7fb0200bb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.007+0000 7fb025c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0200b4e30 0x7fb0200bb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50488/0 (socket says 172.21.15.7:50488) 2024-04-25T11:10:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.007+0000 7fb025c00640 1 -- 172.21.15.7:0/3798487247 learned_addr learned my addr 172.21.15.7:0/3798487247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.007+0000 7fb026600640 1 --2- 172.21.15.7:0/3798487247 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0200b7650 0x7fb0200b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.008+0000 7fb025200640 1 --2- 172.21.15.7:0/3798487247 >> v2:172.21.15.7:3300/0 conn(0x7fb0200b9470 0x7fb0200b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.008+0000 7fb025c00640 1 -- 172.21.15.7:0/3798487247 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0200b7650 msgr2=0x7fb0200b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.008+0000 7fb025c00640 1 --2- 172.21.15.7:0/3798487247 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0200b7650 0x7fb0200b7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.008+0000 7fb025c00640 1 -- 172.21.15.7:0/3798487247 >> v2:172.21.15.7:3300/0 conn(0x7fb0200b9470 msgr2=0x7fb0200b9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:59.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.008+0000 7fb025c00640 1 --2- 172.21.15.7:0/3798487247 >> v2:172.21.15.7:3300/0 conn(0x7fb0200b9470 0x7fb0200b9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.008+0000 7fb025c00640 1 -- 172.21.15.7:0/3798487247 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0200b7fa0 con 0x7fb0200b4e30 2024-04-25T11:10:59.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.010+0000 7fe5e2800640 1 -- 172.21.15.7:0/3001091508 shutdown_connections 2024-04-25T11:10:59.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.010+0000 7fe5e2800640 1 -- 172.21.15.7:0/3001091508 wait complete. 2024-04-25T11:10:59.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.008+0000 7f3de0e00640 1 -- 172.21.15.7:0/2514257854 >> v2:172.21.15.7:3300/0 conn(0x7f3dd4038c20 msgr2=0x7f3dd40064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:10:59.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.008+0000 7f3de0e00640 1 --2- 172.21.15.7:0/2514257854 >> v2:172.21.15.7:3300/0 conn(0x7f3dd4038c20 0x7f3dd40064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.008+0000 7f3de0e00640 1 -- 172.21.15.7:0/2514257854 >> v1:172.21.15.7:6789/0 conn(0x7f3dd4034140 legacy=0x7f3dd4038510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.008+0000 7f3de0e00640 1 -- 172.21.15.7:0/2514257854 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd4050bb0 con 0x7f3dd403a510 2024-04-25T11:10:59.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.011+0000 7fb025c00640 1 --2- 172.21.15.7:0/3798487247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200b4e30 0x7fb0200bb650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb02008e8e0 tx=0x7fb01002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b08f6bdc6dc223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.011+0000 7f3de0e00640 1 --2- 172.21.15.7:0/2514257854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd403a510 0x7f3dd40e16a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009870 tx=0x7f3dd002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f83d6df93132beb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.011+0000 7fb01fe00640 1 -- 172.21.15.7:0/3798487247 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010037070 con 0x7fb0200b4e30 2024-04-25T11:10:59.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.011+0000 7f4e34e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300b5130 0x7f4e30071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.011+0000 7f4e34e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3015f7d0 0x7f4e30072760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.011+0000 7f4e34e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e30072ca0 0x7f4e300c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.011+0000 7f4e34e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e300d4910 con 0x7f4e3015f7d0 2024-04-25T11:10:59.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.011+0000 7f4e34e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30164a90 con 0x7f4e30072ca0 2024-04-25T11:10:59.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.011+0000 7f4e34e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e300d4a90 con 0x7f4e300b5130 2024-04-25T11:10:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.011+0000 7fb01fe00640 1 -- 172.21.15.7:0/3798487247 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb010032440 con 0x7fb0200b4e30 2024-04-25T11:10:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.012+0000 7fb01fe00640 1 -- 172.21.15.7:0/3798487247 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010032870 con 0x7fb0200b4e30 2024-04-25T11:10:59.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.011+0000 7f3ddb400640 1 -- 172.21.15.7:0/2514257854 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0037070 con 0x7f3dd403a510 2024-04-25T11:10:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.012+0000 7f4e2d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3015f7d0 0x7f4e30072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.012+0000 7f4e2d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3015f7d0 0x7f4e30072760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48908/0 (socket says 172.21.15.7:48908) 2024-04-25T11:10:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.012+0000 7f4e2d400640 1 -- 172.21.15.7:0/955006551 learned_addr learned my addr 172.21.15.7:0/955006551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.012+0000 7f4e2de00640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300b5130 0x7f4e30071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7f4e2e800640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e30072ca0 0x7f4e300c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7fb028600640 1 -- 172.21.15.7:0/3798487247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200b4e30 msgr2=0x7fb0200bb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7f4e2de00640 1 -- 172.21.15.7:0/955006551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e30072ca0 msgr2=0x7f4e300c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7f4e2de00640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e30072ca0 0x7f4e300c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7fb028600640 1 --2- 172.21.15.7:0/3798487247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200b4e30 0x7fb0200bb650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb02008e8e0 tx=0x7fb01002a180 comp rx=0 tx=0).stop 2024-04-25T11:10:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7f4e2de00640 1 -- 172.21.15.7:0/955006551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3015f7d0 msgr2=0x7f4e30072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7f4e2de00640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3015f7d0 0x7f4e30072760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7f4e2de00640 1 -- 172.21.15.7:0/955006551 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e300730b0 con 0x7f4e300b5130 2024-04-25T11:10:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7fb028600640 1 -- 172.21.15.7:0/3798487247 shutdown_connections 2024-04-25T11:10:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7fb028600640 1 --2- 172.21.15.7:0/3798487247 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0200b7650 0x7fb0200b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7fb028600640 1 --2- 172.21.15.7:0/3798487247 >> v2:172.21.15.7:3300/0 conn(0x7fb0200b9470 0x7fb0200b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7fb028600640 1 --2- 172.21.15.7:0/3798487247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200b4e30 0x7fb0200bb650 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7fb028600640 1 -- 172.21.15.7:0/3798487247 >> 172.21.15.7:0/3798487247 conn(0x7fb0200d2650 msgr2=0x7fb0200d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7f3ddb400640 1 -- 172.21.15.7:0/2514257854 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dd0032440 con 0x7f3dd403a510 2024-04-25T11:10:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.013+0000 7f3ddb400640 1 -- 172.21.15.7:0/2514257854 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0032850 con 0x7f3dd403a510 2024-04-25T11:10:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.014+0000 7f3de2200640 1 -- 172.21.15.7:0/2514257854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd403a510 msgr2=0x7f3dd40e16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.014+0000 7f3de2200640 1 --2- 172.21.15.7:0/2514257854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd403a510 0x7f3dd40e16a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009870 tx=0x7f3dd002a210 comp rx=0 tx=0).stop 2024-04-25T11:10:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.014+0000 7f3de2200640 1 -- 172.21.15.7:0/2514257854 shutdown_connections 2024-04-25T11:10:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.014+0000 7f3de2200640 1 --2- 172.21.15.7:0/2514257854 >> v2:172.21.15.7:3300/0 conn(0x7f3dd4038c20 0x7f3dd40064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.014+0000 7f4e2de00640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300b5130 0x7f4e30071070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00c340 tx=0x7f4e1c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.014+0000 7f3de2200640 1 --2- 172.21.15.7:0/2514257854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd403a510 0x7f3dd40e16a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.014+0000 7f3de2200640 1 -- 172.21.15.7:0/2514257854 >> 172.21.15.7:0/2514257854 conn(0x7f3dd4054cf0 msgr2=0x7f3dd4055100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.015+0000 7fe5e2800640 1 Processor -- start 2024-04-25T11:10:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.015+0000 7fb028600640 1 -- 172.21.15.7:0/3798487247 shutdown_connections 2024-04-25T11:10:59.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.015+0000 7fb028600640 1 -- 172.21.15.7:0/3798487247 wait complete. 2024-04-25T11:10:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.015+0000 7fe5e2800640 1 -- start start 2024-04-25T11:10:59.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.015+0000 7f4e26000640 1 -- 172.21.15.7:0/955006551 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c0074e0 con 0x7f4e300b5130 2024-04-25T11:10:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.015+0000 7f4e26000640 1 -- 172.21.15.7:0/955006551 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e1c007680 con 0x7f4e300b5130 2024-04-25T11:10:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.015+0000 7f4e26000640 1 -- 172.21.15.7:0/955006551 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c004a20 con 0x7f4e300b5130 2024-04-25T11:10:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.016+0000 7f4e34e00640 1 -- 172.21.15.7:0/955006551 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e300715b0 con 0x7f4e300b5130 2024-04-25T11:10:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.016+0000 7f4e34e00640 1 -- 172.21.15.7:0/955006551 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e300c4dd0 con 0x7f4e300b5130 2024-04-25T11:10:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.017+0000 7f4e34e00640 1 -- 172.21.15.7:0/955006551 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e300b6410 con 0x7f4e300b5130 2024-04-25T11:10:59.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.018+0000 7fe5e2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5dc06dac0 0x7fe5dc0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.018+0000 7fe5e2800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc070aa0 0x7fe5dc160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.018+0000 7fe5e2800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5dc162330 0x7fe5dc0cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.018+0000 7fe5e2800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc06fb90 con 0x7fe5dc06dac0 2024-04-25T11:10:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.018+0000 7fe5e2800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc06f8b0 con 0x7fe5dc070aa0 2024-04-25T11:10:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.018+0000 7fe5e2800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe5dc0b5e80 con 0x7fe5dc162330 2024-04-25T11:10:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.015+0000 7f3de2200640 1 -- 172.21.15.7:0/2514257854 shutdown_connections 2024-04-25T11:10:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.015+0000 7f3de2200640 1 -- 172.21.15.7:0/2514257854 wait complete. 2024-04-25T11:10:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.019+0000 7fb028600640 1 Processor -- start 2024-04-25T11:10:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.019+0000 7fe5db400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5dc06dac0 0x7fe5dc0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.019+0000 7fb028600640 1 -- start start 2024-04-25T11:10:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.019+0000 7fe5db400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5dc06dac0 0x7fe5dc0af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48910/0 (socket says 172.21.15.7:48910) 2024-04-25T11:10:59.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.019+0000 7fe5db400640 1 -- 172.21.15.7:0/2851659638 learned_addr learned my addr 172.21.15.7:0/2851659638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.020+0000 7f3de2200640 1 Processor -- start 2024-04-25T11:10:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.021+0000 7f3de2200640 1 -- start start 2024-04-25T11:10:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.020+0000 7fe5daa00640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc070aa0 0x7fe5dc160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.020+0000 7fe5db400640 1 -- 172.21.15.7:0/2851659638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5dc162330 msgr2=0x7fe5dc0cf6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:10:59.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.020+0000 7fe5db400640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5dc162330 0x7fe5dc0cf6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.021+0000 7fe5db400640 1 -- 172.21.15.7:0/2851659638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc070aa0 msgr2=0x7fe5dc160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.021+0000 7fe5db400640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc070aa0 0x7fe5dc160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.021+0000 7fe5db400640 1 -- 172.21.15.7:0/2851659638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc0b0140 con 0x7fe5dc06dac0 2024-04-25T11:10:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.021+0000 7fe5daa00640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc070aa0 0x7fe5dc160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:10:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.021+0000 7fe5db400640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5dc06dac0 0x7fe5dc0af7d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009890 tx=0x7fe5c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.022+0000 7fe5d3400640 1 -- 172.21.15.7:0/2851659638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8037050 con 0x7fe5dc06dac0 2024-04-25T11:10:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.022+0000 7fe5d3400640 1 -- 172.21.15.7:0/2851659638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c8038040 con 0x7fe5dc06dac0 2024-04-25T11:10:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.022+0000 7fe5d3400640 1 -- 172.21.15.7:0/2851659638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c803c910 con 0x7fe5dc06dac0 2024-04-25T11:10:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.022+0000 7fe5e2800640 1 -- 172.21.15.7:0/2851659638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc14af90 con 0x7fe5dc06dac0 2024-04-25T11:10:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.022+0000 7fe5e2800640 1 -- 172.21.15.7:0/2851659638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc14b550 con 0x7fe5dc06dac0 2024-04-25T11:10:59.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.021+0000 7f4e26000640 1 -- 172.21.15.7:0/955006551 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4e1c011070 con 0x7f4e300b5130 2024-04-25T11:10:59.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.022+0000 7f4e26000640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 0x7f4e100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.023+0000 7f4e26000640 1 -- 172.21.15.7:0/955006551 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4e1c095630 con 0x7f4e300b5130 2024-04-25T11:10:59.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.024+0000 7fb028600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200b4e30 0x7fb0201ec640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.024+0000 7fb028600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0200b7650 0x7fb02015c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.024+0000 7f3de2200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd4034140 0x7f3dd4036750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7f3de2200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dd403a510 0x7f3dd4036c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7f3de2200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dd40d8f20 0x7f3dd40c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7f3de2200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4033f30 con 0x7f3dd40d8f20 2024-04-25T11:10:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7f3de2200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd4033db0 con 0x7f3dd403a510 2024-04-25T11:10:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7f3de2200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd4006a00 con 0x7f3dd4034140 2024-04-25T11:10:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7fb028600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0200b9470 0x7fb0201569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7f3de0e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd4034140 0x7f3dd4036750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7fb028600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0200d0500 con 0x7fb0200b7650 2024-04-25T11:10:59.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7f3de0e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd4034140 0x7f3dd4036750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43588/0 (socket says 172.21.15.7:43588) 2024-04-25T11:10:59.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7f3de0e00640 1 -- 172.21.15.7:0/637823249 learned_addr learned my addr 172.21.15.7:0/637823249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:59.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7fb028600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb0200d0220 con 0x7fb0200b4e30 2024-04-25T11:10:59.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7fb028600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb0200d07e0 con 0x7fb0200b9470 2024-04-25T11:10:59.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7f3ddbe00640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dd403a510 0x7f3dd4036c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.026+0000 7fb025c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200b4e30 0x7fb0201ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.025+0000 7fe5e2800640 1 -- 172.21.15.7:0/2851659638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b4005290 con 0x7fe5dc06dac0 2024-04-25T11:10:59.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.026+0000 7fb025c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200b4e30 0x7fb0201ec640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50520/0 (socket says 172.21.15.7:50520) 2024-04-25T11:10:59.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.026+0000 7fb025c00640 1 -- 172.21.15.7:0/1151779710 learned_addr learned my addr 172.21.15.7:0/1151779710 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:10:59.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.026+0000 7f3de0e00640 1 -- 172.21.15.7:0/637823249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dd403a510 msgr2=0x7f3dd4036c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.026+0000 7f3de0e00640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dd403a510 0x7f3dd4036c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.026+0000 7fb025200640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0200b7650 0x7fb02015c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.027+0000 7f4e2d400640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 0x7f4e100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.026+0000 7f3de0e00640 1 -- 172.21.15.7:0/637823249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dd40d8f20 msgr2=0x7f3dd40c91d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T11:10:59.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.026+0000 7f3de0e00640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dd40d8f20 0x7f3dd40c91d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.026+0000 7f3de0e00640 1 -- 172.21.15.7:0/637823249 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd40d9730 con 0x7f3dd4034140 2024-04-25T11:10:59.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.027+0000 7fb026600640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0200b9470 0x7fb0201569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.027+0000 7f3de0e00640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd4034140 0x7f3dd4036750 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0000c00 tx=0x7f3dd00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.027+0000 7fb025200640 1 -- 172.21.15.7:0/1151779710 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0200b9470 msgr2=0x7fb0201569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.027+0000 7fb025200640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0200b9470 0x7fb0201569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.027+0000 7fb025200640 1 -- 172.21.15.7:0/1151779710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200b4e30 msgr2=0x7fb0201ec640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.027+0000 7fb025200640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200b4e30 0x7fb0201ec640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.027+0000 7fb025200640 1 -- 172.21.15.7:0/1151779710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02015cf70 con 0x7fb0200b7650 2024-04-25T11:10:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.028+0000 7fb025200640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0200b7650 0x7fb02015c2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb01400d4f0 tx=0x7fb01400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.028+0000 7f4e2d400640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 0x7f4e100769c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009720 tx=0x7f4e2002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.028+0000 7fe5d3400640 1 -- 172.21.15.7:0/2851659638 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe5c8045030 con 0x7fe5dc06dac0 2024-04-25T11:10:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.029+0000 7fe5d3400640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5bc0744b0 0x7fe5bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.030+0000 7fe5d3400640 1 -- 172.21.15.7:0/2851659638 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe5c80b57f0 con 0x7fe5dc06dac0 2024-04-25T11:10:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.030+0000 7fe5daa00640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5bc0744b0 0x7fe5bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.031+0000 7fe5daa00640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5bc0744b0 0x7fe5bc0769c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc007fd0 tx=0x7fe5cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.034+0000 7f3dd8c00640 1 -- 172.21.15.7:0/637823249 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0041070 con 0x7f3dd4034140 2024-04-25T11:10:59.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.034+0000 7fb01d600640 1 -- 172.21.15.7:0/1151779710 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014004240 con 0x7fb0200b7650 2024-04-25T11:10:59.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.034+0000 7fb028600640 1 -- 172.21.15.7:0/1151779710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02015d250 con 0x7fb0200b7650 2024-04-25T11:10:59.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.034+0000 7f3dd8c00640 1 -- 172.21.15.7:0/637823249 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dd003c890 con 0x7f3dd4034140 2024-04-25T11:10:59.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.034+0000 7f3dd8c00640 1 -- 172.21.15.7:0/637823249 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0037040 con 0x7f3dd4034140 2024-04-25T11:10:59.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.035+0000 7fe5d3400640 1 -- 172.21.15.7:0/2851659638 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe5c804c060 con 0x7fe5dc06dac0 2024-04-25T11:10:59.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.036+0000 7f4e26000640 1 -- 172.21.15.7:0/955006551 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e300b6410 con 0x7f4e300b5130 2024-04-25T11:10:59.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.035+0000 7fb01d600640 1 -- 172.21.15.7:0/1151779710 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0140043e0 con 0x7fb0200b7650 2024-04-25T11:10:59.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.035+0000 7fb01d600640 1 -- 172.21.15.7:0/1151779710 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014003850 con 0x7fb0200b7650 2024-04-25T11:10:59.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.035+0000 7f3de2200640 1 -- 172.21.15.7:0/637823249 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd404b060 con 0x7f3dd4034140 2024-04-25T11:10:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.036+0000 7fb028600640 1 -- 172.21.15.7:0/1151779710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb020157fa0 con 0x7fb0200b7650 2024-04-25T11:10:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.036+0000 7f3de2200640 1 -- 172.21.15.7:0/637823249 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd404b610 con 0x7f3dd4034140 2024-04-25T11:10:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.037+0000 7f3de2200640 1 -- 172.21.15.7:0/637823249 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd40054a0 con 0x7f3dd4034140 2024-04-25T11:10:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.038+0000 7fb028600640 1 -- 172.21.15.7:0/1151779710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb020050740 con 0x7fb0200b7650 2024-04-25T11:10:59.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.041+0000 7fb01d600640 1 -- 172.21.15.7:0/1151779710 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb0140039f0 con 0x7fb0200b7650 2024-04-25T11:10:59.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.042+0000 7f3dd8c00640 1 -- 172.21.15.7:0/637823249 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3dd0044070 con 0x7f3dd4034140 2024-04-25T11:10:59.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.043+0000 7f3dd8c00640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dc00744e0 0x7f3dc00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.043+0000 7f3dd8c00640 1 -- 172.21.15.7:0/637823249 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3dd00b63c0 con 0x7f3dd4034140 2024-04-25T11:10:59.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.043+0000 7fb01d600640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0040744b0 0x7fb0040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:10:59.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.043+0000 7fb01d600640 1 -- 172.21.15.7:0/1151779710 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fb0140956d0 con 0x7fb0200b7650 2024-04-25T11:10:59.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.044+0000 7f3ddbe00640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dc00744e0 0x7f3dc00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.045+0000 7fb025c00640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0040744b0 0x7fb0040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:10:59.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.046+0000 7f3ddbe00640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dc00744e0 0x7f3dc00769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4048220 tx=0x7f3dcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.047+0000 7fb025c00640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0040744b0 0x7fb0040769c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb01002a1c0 tx=0x7fb01002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:10:59.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.051+0000 7fb01d600640 1 -- 172.21.15.7:0/1151779710 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb01405fa80 con 0x7fb0200b7650 2024-04-25T11:10:59.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.054+0000 7f3dd8c00640 1 -- 172.21.15.7:0/637823249 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3dd004b070 con 0x7f3dd4034140 2024-04-25T11:10:59.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.366+0000 7ff3b8400640 1 -- 172.21.15.7:0/2903955169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff3a8005490 con 0x7ff3a8046800 2024-04-25T11:10:59.370 INFO:teuthology.orchestra.run.smithi007.stdout:64424509451 2024-04-25T11:10:59.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.368+0000 7ff3ad600640 1 -- 172.21.15.7:0/2903955169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff3a8047150 con 0x7ff3a8046800 2024-04-25T11:10:59.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.372+0000 7ff3b8400640 1 -- 172.21.15.7:0/2903955169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3900746b0 msgr2=0x7ff390076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.372+0000 7ff3b8400640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3900746b0 0x7ff390076bc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff39c007110 tx=0x7ff39c004910 comp rx=0 tx=0).stop 2024-04-25T11:10:59.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.372+0000 7ff3b8400640 1 -- 172.21.15.7:0/2903955169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3a8046800 msgr2=0x7ff3a8046c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.372+0000 7ff3b8400640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3a8046800 0x7ff3a8046c10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0066010 tx=0x7ff3b0069570 comp rx=0 tx=0).stop 2024-04-25T11:10:59.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.373+0000 7ff3b8400640 1 -- 172.21.15.7:0/2903955169 shutdown_connections 2024-04-25T11:10:59.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.373+0000 7ff3b8400640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3900746b0 0x7ff390076bc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.374+0000 7ff3b8400640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3a8046800 0x7ff3a8046c10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.374+0000 7ff3b8400640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3a8048ee0 0x7ff3a80492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.374+0000 7ff3b8400640 1 --2- 172.21.15.7:0/2903955169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3980053a0 0x7ff3a80d8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.374+0000 7ff3b8400640 1 -- 172.21.15.7:0/2903955169 >> 172.21.15.7:0/2903955169 conn(0x7ff3a8054ce0 msgr2=0x7ff3a80e0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.375+0000 7ff3b8400640 1 -- 172.21.15.7:0/2903955169 shutdown_connections 2024-04-25T11:10:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.375+0000 7ff3b8400640 1 -- 172.21.15.7:0/2903955169 wait complete. 2024-04-25T11:10:59.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.412+0000 7ff3eb800640 1 -- 172.21.15.7:0/1093077544 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff3dc0581a0 con 0x7ff3dc0e57f0 2024-04-25T11:10:59.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.413+0000 7ff3e0c00640 1 -- 172.21.15.7:0/1093077544 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff3e40fb750 con 0x7ff3dc0e57f0 2024-04-25T11:10:59.418 INFO:teuthology.orchestra.run.smithi007.stdout:64424509450 2024-04-25T11:10:59.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509451 for osd.1 2024-04-25T11:10:59.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.416+0000 7ff3eb800640 1 -- 172.21.15.7:0/1093077544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3c4074580 msgr2=0x7ff3c4076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.416+0000 7ff3eb800640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3c4074580 0x7ff3c4076a90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8005880 tx=0x7ff3d8009040 comp rx=0 tx=0).stop 2024-04-25T11:10:59.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.416+0000 7ff3eb800640 1 -- 172.21.15.7:0/1093077544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3dc0e57f0 msgr2=0x7ff3dc0dec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.416+0000 7ff3eb800640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3dc0e57f0 0x7ff3dc0dec40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4069e70 tx=0x7ff3e406b1c0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.416+0000 7f4ba4a00640 1 -- 172.21.15.7:0/1675074108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4ba0050740 con 0x7f4ba006e780 2024-04-25T11:10:59.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.417+0000 7ff3eb800640 1 -- 172.21.15.7:0/1093077544 shutdown_connections 2024-04-25T11:10:59.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.417+0000 7ff3eb800640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3c4074580 0x7ff3c4076a90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.417+0000 7ff3eb800640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3dc0e57f0 0x7ff3dc0dec40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.417+0000 7ff3eb800640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3dc0575a0 0x7ff3dc0dc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.417+0000 7ff3eb800640 1 --2- 172.21.15.7:0/1093077544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3dc054160 0x7ff3dc0d90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.417+0000 7ff3eb800640 1 -- 172.21.15.7:0/1093077544 >> 172.21.15.7:0/1093077544 conn(0x7ff3dc059100 msgr2=0x7ff3dc0579b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.417+0000 7ff3eb800640 1 -- 172.21.15.7:0/1093077544 shutdown_connections 2024-04-25T11:10:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.417+0000 7ff3eb800640 1 -- 172.21.15.7:0/1093077544 wait complete. 2024-04-25T11:10:59.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.417+0000 7f4b96a00640 1 -- 172.21.15.7:0/1675074108 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4b9004b020 con 0x7f4ba006e780 2024-04-25T11:10:59.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.418+0000 7f63e8e00640 1 -- 172.21.15.7:0/135237321 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f63e406d8a0 con 0x7f63e409b540 2024-04-25T11:10:59.424 INFO:teuthology.orchestra.run.smithi007.stdout:68719476746 2024-04-25T11:10:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.419+0000 7f63da000640 1 -- 172.21.15.7:0/135237321 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f63dc068040 con 0x7f63e409b540 2024-04-25T11:10:59.424 INFO:teuthology.orchestra.run.smithi007.stdout:64424509450 2024-04-25T11:10:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.421+0000 7f4ba4a00640 1 -- 172.21.15.7:0/1675074108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b7c074500 msgr2=0x7f4b7c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.421+0000 7f4ba4a00640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b7c074500 0x7f4b7c076a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00b9220 tx=0x7f4b88009190 comp rx=0 tx=0).stop 2024-04-25T11:10:59.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.421+0000 7f4ba4a00640 1 -- 172.21.15.7:0/1675074108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ba006e780 msgr2=0x7f4ba01569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.421+0000 7f4ba4a00640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ba006e780 0x7f4ba01569b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b90030000 tx=0x7f4b90004a40 comp rx=0 tx=0).stop 2024-04-25T11:10:59.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f4ba4a00640 1 -- 172.21.15.7:0/1675074108 shutdown_connections 2024-04-25T11:10:59.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f4ba4a00640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b7c074500 0x7f4b7c076a10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f4ba4a00640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba01646b0 0x7f4ba0157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f4ba4a00640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ba00d3590 0x7f4ba0156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f4ba4a00640 1 --2- 172.21.15.7:0/1675074108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ba006e780 0x7f4ba01569b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f4ba4a00640 1 -- 172.21.15.7:0/1675074108 >> 172.21.15.7:0/1675074108 conn(0x7f4ba00d26b0 msgr2=0x7f4ba0163700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f4ba4a00640 1 -- 172.21.15.7:0/1675074108 shutdown_connections 2024-04-25T11:10:59.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f63e8e00640 1 -- 172.21.15.7:0/135237321 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63c40745b0 msgr2=0x7f63c4076ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f63e8e00640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63c40745b0 0x7f63c4076ac0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f63d0034000 tx=0x7f63d002e000 comp rx=0 tx=0).stop 2024-04-25T11:10:59.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f63e8e00640 1 -- 172.21.15.7:0/135237321 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 msgr2=0x7f63e414b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f63e8e00640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 0x7f63e414b2d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00d7d0 tx=0x7f63dc00dc90 comp rx=0 tx=0).stop 2024-04-25T11:10:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f63e8e00640 1 -- 172.21.15.7:0/135237321 shutdown_connections 2024-04-25T11:10:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f4ba4a00640 1 -- 172.21.15.7:0/1675074108 wait complete. 2024-04-25T11:10:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f63e8e00640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63c40745b0 0x7f63c4076ac0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f63e8e00640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63e415ace0 0x7f63e4155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f63e8e00640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63e40b2d90 0x7f63e4154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f63e8e00640 1 --2- 172.21.15.7:0/135237321 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63e409b540 0x7f63e414b2d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.422+0000 7f63e8e00640 1 -- 172.21.15.7:0/135237321 >> 172.21.15.7:0/135237321 conn(0x7f63e40b7560 msgr2=0x7f63e40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.423+0000 7f63e8e00640 1 -- 172.21.15.7:0/135237321 shutdown_connections 2024-04-25T11:10:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.423+0000 7f63e8e00640 1 -- 172.21.15.7:0/135237321 wait complete. 2024-04-25T11:10:59.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.0 2024-04-25T11:10:59.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.3 2024-04-25T11:10:59.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476746 for osd.5 2024-04-25T11:10:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.492+0000 7f4e34e00640 1 -- 172.21.15.7:0/955006551 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4e30050740 con 0x7f4e300b5130 2024-04-25T11:10:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.493+0000 7f4e26000640 1 -- 172.21.15.7:0/955006551 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4e300730b0 con 0x7f4e300b5130 2024-04-25T11:10:59.494 INFO:teuthology.orchestra.run.smithi007.stdout:60129542155 2024-04-25T11:10:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.496+0000 7f4e34e00640 1 -- 172.21.15.7:0/955006551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 msgr2=0x7f4e100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.496+0000 7f4e34e00640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 0x7f4e100769c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009720 tx=0x7f4e2002e000 comp rx=0 tx=0).stop 2024-04-25T11:10:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.496+0000 7f4e34e00640 1 -- 172.21.15.7:0/955006551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300b5130 msgr2=0x7f4e30071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.496+0000 7f4e34e00640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300b5130 0x7f4e30071070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c00c340 tx=0x7f4e1c00c800 comp rx=0 tx=0).stop 2024-04-25T11:10:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.496+0000 7fb028600640 1 -- 172.21.15.7:0/1151779710 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb02015cd00 con 0x7fb0200b7650 2024-04-25T11:10:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.497+0000 7f4e34e00640 1 -- 172.21.15.7:0/955006551 shutdown_connections 2024-04-25T11:10:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.497+0000 7f4e34e00640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 0x7f4e100769c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.497+0000 7f4e34e00640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e30072ca0 0x7f4e300c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.497+0000 7f4e34e00640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e3015f7d0 0x7f4e30072760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.497+0000 7f4e34e00640 1 --2- 172.21.15.7:0/955006551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300b5130 0x7f4e30071070 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.497+0000 7f4e34e00640 1 -- 172.21.15.7:0/955006551 >> 172.21.15.7:0/955006551 conn(0x7f4e300b7ad0 msgr2=0x7f4e3015ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.497+0000 7fb01d600640 1 -- 172.21.15.7:0/1151779710 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb01405f380 con 0x7fb0200b7650 2024-04-25T11:10:59.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.497+0000 7f4e34e00640 1 -- 172.21.15.7:0/955006551 shutdown_connections 2024-04-25T11:10:59.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.497+0000 7f4e34e00640 1 -- 172.21.15.7:0/955006551 wait complete. 2024-04-25T11:10:59.501 INFO:teuthology.orchestra.run.smithi007.stdout:68719476747 2024-04-25T11:10:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.499+0000 7fb028600640 1 -- 172.21.15.7:0/1151779710 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0040744b0 msgr2=0x7fb0040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.499+0000 7fb028600640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0040744b0 0x7fb0040769c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb01002a1c0 tx=0x7fb01002e000 comp rx=0 tx=0).stop 2024-04-25T11:10:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.499+0000 7fb028600640 1 -- 172.21.15.7:0/1151779710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0200b7650 msgr2=0x7fb02015c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.499+0000 7fb028600640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0200b7650 0x7fb02015c2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb01400d4f0 tx=0x7fb01400d9b0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.499+0000 7fb028600640 1 -- 172.21.15.7:0/1151779710 shutdown_connections 2024-04-25T11:10:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.500+0000 7fb028600640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0040744b0 0x7fb0040769c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.500+0000 7fb028600640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0200b9470 0x7fb0201569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.500+0000 7fb028600640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0200b7650 0x7fb02015c2f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.500+0000 7fb028600640 1 --2- 172.21.15.7:0/1151779710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200b4e30 0x7fb0201ec640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.500+0000 7fb028600640 1 -- 172.21.15.7:0/1151779710 >> 172.21.15.7:0/1151779710 conn(0x7fb0200d2650 msgr2=0x7fb0200b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.500+0000 7fb028600640 1 -- 172.21.15.7:0/1151779710 shutdown_connections 2024-04-25T11:10:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.500+0000 7fb028600640 1 -- 172.21.15.7:0/1151779710 wait complete. 2024-04-25T11:10:59.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.501+0000 7fe5e2800640 1 -- 172.21.15.7:0/2851659638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe5b4005580 con 0x7fe5dc06dac0 2024-04-25T11:10:59.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.501+0000 7fe5d3400640 1 -- 172.21.15.7:0/2851659638 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe5c807fc30 con 0x7fe5dc06dac0 2024-04-25T11:10:59.504 INFO:teuthology.orchestra.run.smithi007.stdout:68719476746 2024-04-25T11:10:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 -- 172.21.15.7:0/2851659638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5bc0744b0 msgr2=0x7fe5bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5bc0744b0 0x7fe5bc0769c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc007fd0 tx=0x7fe5cc009040 comp rx=0 tx=0).stop 2024-04-25T11:10:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 -- 172.21.15.7:0/2851659638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5dc06dac0 msgr2=0x7fe5dc0af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5dc06dac0 0x7fe5dc0af7d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009890 tx=0x7fe5c80046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 -- 172.21.15.7:0/2851659638 shutdown_connections 2024-04-25T11:10:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5bc0744b0 0x7fe5bc0769c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5dc162330 0x7fe5dc0cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5dc070aa0 0x7fe5dc160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 --2- 172.21.15.7:0/2851659638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5dc06dac0 0x7fe5dc0af7d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 -- 172.21.15.7:0/2851659638 >> 172.21.15.7:0/2851659638 conn(0x7fe5dc0732b0 msgr2=0x7fe5dc167fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 -- 172.21.15.7:0/2851659638 shutdown_connections 2024-04-25T11:10:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.505+0000 7fe5e2800640 1 -- 172.21.15.7:0/2851659638 wait complete. 2024-04-25T11:10:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.510+0000 7f3de2200640 1 -- 172.21.15.7:0/637823249 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3dd40380e0 con 0x7f3dd4034140 2024-04-25T11:10:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.511+0000 7f3dd8c00640 1 -- 172.21.15.7:0/637823249 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3dd40d9730 con 0x7f3dd4034140 2024-04-25T11:10:59.511 INFO:teuthology.orchestra.run.smithi007.stdout:68719476747 2024-04-25T11:10:59.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 -- 172.21.15.7:0/637823249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dc00744e0 msgr2=0x7f3dc00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dc00744e0 0x7f3dc00769f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4048220 tx=0x7f3dcc009040 comp rx=0 tx=0).stop 2024-04-25T11:10:59.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 -- 172.21.15.7:0/637823249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd4034140 msgr2=0x7f3dd4036750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:10:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd4034140 0x7f3dd4036750 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0000c00 tx=0x7f3dd00046c0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 -- 172.21.15.7:0/637823249 shutdown_connections 2024-04-25T11:10:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dc00744e0 0x7f3dc00769f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dd40d8f20 0x7f3dd40c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dd403a510 0x7f3dd4036c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 --2- 172.21.15.7:0/637823249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd4034140 0x7f3dd4036750 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:10:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 -- 172.21.15.7:0/637823249 >> 172.21.15.7:0/637823249 conn(0x7f3dd4054cf0 msgr2=0x7f3dd40dbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:10:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 -- 172.21.15.7:0/637823249 shutdown_connections 2024-04-25T11:10:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:10:59.514+0000 7f3de2200640 1 -- 172.21.15.7:0/637823249 wait complete. 2024-04-25T11:10:59.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542155 for osd.2 2024-04-25T11:10:59.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476746 for osd.7 2024-04-25T11:10:59.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476748 got 68719476747 for osd.6 2024-04-25T11:10:59.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476748 got 68719476747 for osd.4 2024-04-25T11:11:00.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T11:11:00.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T11:11:00.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T11:11:00.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T11:11:00.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T11:11:00.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T11:11:00.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T11:11:00.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T11:11:00.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.813+0000 7fd51a800640 1 Processor -- start 2024-04-25T11:11:00.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.813+0000 7fd51a800640 1 -- start start 2024-04-25T11:11:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.817+0000 7fd51a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd50c0e3e10 0x7fd50c054120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.818+0000 7fd51a800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd50c054a70 con 0x7fd50c054660 2024-04-25T11:11:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.818+0000 7fd51a800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd50c006c50 con 0x7fd50c0ddfe0 2024-04-25T11:11:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.818+0000 7fd513400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd50c0ddfe0 0x7fd50c006400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:49470/0 (socket says 172.21.15.7:49470) 2024-04-25T11:11:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.818+0000 7fd513400640 1 -- 172.21.15.7:0/2846396116 learned_addr learned my addr 172.21.15.7:0/2846396116 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.819+0000 7fd512a00640 1 --2- 172.21.15.7:0/2846396116 >> v2:172.21.15.87:3300/0 conn(0x7fd50c0e3e10 0x7fd50c054120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.819+0000 7fd51a800640 1 -- 172.21.15.7:0/2846396116 --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd50c006f30 con 0x7fd50c0e3e10 2024-04-25T11:11:00.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.820+0000 7fd512000640 1 -- 172.21.15.7:0/2846396116 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 581181415 0 0) 0x7fd50c006c50 con 0x7fd50c0ddfe0 2024-04-25T11:11:00.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.820+0000 7fd512000640 1 -- 172.21.15.7:0/2846396116 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4fc003520 con 0x7fd50c0ddfe0 2024-04-25T11:11:00.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.821+0000 7fd512a00640 1 -- 172.21.15.7:0/2846396116 >> v1:172.21.15.87:6789/0 conn(0x7fd50c0ddfe0 legacy=0x7fd50c006400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.821+0000 7fd512a00640 1 -- 172.21.15.7:0/2846396116 >> v1:172.21.15.7:6789/0 conn(0x7fd50c054660 legacy=0x7fd50c057550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.821+0000 7fd512a00640 1 -- 172.21.15.7:0/2846396116 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50c007210 con 0x7fd50c0e3e10 2024-04-25T11:11:00.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.823+0000 7fd512a00640 1 --2- 172.21.15.7:0/2846396116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd50c0e3e10 0x7fd50c054120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd5000097a0 tx=0x7fd50002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=239f1d6fb80f3891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.823+0000 7fd512000640 1 -- 172.21.15.7:0/2846396116 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500037070 con 0x7fd50c0e3e10 2024-04-25T11:11:00.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.823+0000 7fd512000640 1 -- 172.21.15.7:0/2846396116 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd500032440 con 0x7fd50c0e3e10 2024-04-25T11:11:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.824+0000 7fd512000640 1 -- 172.21.15.7:0/2846396116 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500037070 con 0x7fd50c0e3e10 2024-04-25T11:11:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.825+0000 7fd51a800640 1 -- 172.21.15.7:0/2846396116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd50c0e3e10 msgr2=0x7fd50c054120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.825+0000 7fd51a800640 1 --2- 172.21.15.7:0/2846396116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd50c0e3e10 0x7fd50c054120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd5000097a0 tx=0x7fd50002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.825+0000 7fd51a800640 1 -- 172.21.15.7:0/2846396116 shutdown_connections 2024-04-25T11:11:00.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.825+0000 7fd51a800640 1 --2- 172.21.15.7:0/2846396116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd50c0e3e10 0x7fd50c054120 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.826+0000 7fd51a800640 1 -- 172.21.15.7:0/2846396116 >> 172.21.15.7:0/2846396116 conn(0x7fd50c0563f0 msgr2=0x7fd50c059200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:00.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.826+0000 7fd51a800640 1 -- 172.21.15.7:0/2846396116 shutdown_connections 2024-04-25T11:11:00.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.826+0000 7fd51a800640 1 -- 172.21.15.7:0/2846396116 wait complete. 2024-04-25T11:11:00.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.831+0000 7fd51a800640 1 Processor -- start 2024-04-25T11:11:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.831+0000 7fd51a800640 1 -- start start 2024-04-25T11:11:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.834+0000 7fd51a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd50c054660 0x7fd50c0d5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.835+0000 7fd51a800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd50c0ddfe0 0x7fd50c052870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.835+0000 7fd513400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd50c054660 0x7fd50c0d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.835+0000 7fd513400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd50c054660 0x7fd50c0d5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43342/0 (socket says 172.21.15.7:43342) 2024-04-25T11:11:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.836+0000 7fd513400640 1 -- 172.21.15.7:0/124672614 learned_addr learned my addr 172.21.15.7:0/124672614 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.836+0000 7fd51a800640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd50c0e3e10 0x7fd50c03a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.836+0000 7fd512a00640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd50c0ddfe0 0x7fd50c052870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.837+0000 7fd513e00640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd50c0e3e10 0x7fd50c03a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.837+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c007cf0 con 0x7fd50c054660 2024-04-25T11:11:00.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.837+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c0077d0 con 0x7fd50c0e3e10 2024-04-25T11:11:00.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.837+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd50c057c60 con 0x7fd50c0ddfe0 2024-04-25T11:11:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.838+0000 7fd513400640 1 -- 172.21.15.7:0/124672614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd50c0ddfe0 msgr2=0x7fd50c052870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.838+0000 7fd513400640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd50c0ddfe0 0x7fd50c052870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.838+0000 7fd513400640 1 -- 172.21.15.7:0/124672614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd50c0e3e10 msgr2=0x7fd50c03a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.838+0000 7fd513400640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd50c0e3e10 0x7fd50c03a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.838+0000 7fd513400640 1 -- 172.21.15.7:0/124672614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50c050f30 con 0x7fd50c054660 2024-04-25T11:11:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.839+0000 7fd513400640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd50c054660 0x7fd50c0d5210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c390 tx=0x7fd50800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.840+0000 7fd507400640 1 -- 172.21.15.7:0/124672614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508017070 con 0x7fd50c054660 2024-04-25T11:11:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.840+0000 7fd507400640 1 -- 172.21.15.7:0/124672614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd508005ce0 con 0x7fd50c054660 2024-04-25T11:11:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.840+0000 7fd507400640 1 -- 172.21.15.7:0/124672614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5080072f0 con 0x7fd50c054660 2024-04-25T11:11:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.840+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50c051210 con 0x7fd50c054660 2024-04-25T11:11:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.840+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd50c0517d0 con 0x7fd50c054660 2024-04-25T11:11:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.843+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f0005290 con 0x7fd50c054660 2024-04-25T11:11:00.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.847+0000 7fd507400640 1 -- 172.21.15.7:0/124672614 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd50801a070 con 0x7fd50c054660 2024-04-25T11:11:00.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.848+0000 7fd507400640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4f4074500 0x7fd4f4076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.853+0000 7fd507400640 1 -- 172.21.15.7:0/124672614 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd50805adc0 con 0x7fd50c054660 2024-04-25T11:11:00.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.855+0000 7fd512a00640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4f4074500 0x7fd4f4076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.855+0000 7fd507400640 1 -- 172.21.15.7:0/124672614 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd508021020 con 0x7fd50c054660 2024-04-25T11:11:00.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.856+0000 7fd512a00640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4f4074500 0x7fd4f4076a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd500034000 tx=0x7fd50002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.856+0000 7f424ce00640 1 Processor -- start 2024-04-25T11:11:00.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.857+0000 7f424ce00640 1 -- start start 2024-04-25T11:11:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.860+0000 7f424ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f42480cdd20 0x7f42480cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.860+0000 7f424ce00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f424806d580 0x7f424806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.860+0000 7f424ce00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f424806ded0 con 0x7f42480d1140 2024-04-25T11:11:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.860+0000 7f424ce00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f42480d0a60 con 0x7f42480cdd20 2024-04-25T11:11:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.860+0000 7f424ce00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4248164020 con 0x7f424806d580 2024-04-25T11:11:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.861+0000 7f4245400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f42480cdd20 0x7f42480cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.861+0000 7f4245400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f42480cdd20 0x7f42480cf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:55458/0 (socket says 172.21.15.7:55458) 2024-04-25T11:11:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.861+0000 7f4245400640 1 -- 172.21.15.7:0/2343379769 learned_addr learned my addr 172.21.15.7:0/2343379769 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.861+0000 7f4246800640 1 --2- 172.21.15.7:0/2343379769 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f424806d580 0x7f424806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.862+0000 7f4245400640 1 -- 172.21.15.7:0/2343379769 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f424806d580 msgr2=0x7f424806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:00.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.862+0000 7fd053200640 1 Processor -- start 2024-04-25T11:11:00.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.862+0000 7f4245400640 1 --2- 172.21.15.7:0/2343379769 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f424806d580 0x7f424806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.862+0000 7f4245400640 1 -- 172.21.15.7:0/2343379769 >> v1:172.21.15.7:6789/0 conn(0x7f42480d1140 legacy=0x7f42481644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.862+0000 7f4245400640 1 -- 172.21.15.7:0/2343379769 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42480cfdb0 con 0x7f42480cdd20 2024-04-25T11:11:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.863+0000 7fd053200640 1 -- start start 2024-04-25T11:11:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.864+0000 7f4245400640 1 --2- 172.21.15.7:0/2343379769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42480cdd20 0x7f42480cf6f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4234009870 tx=0x7f423402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a2e861e8016157a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.865+0000 7f4244a00640 1 -- 172.21.15.7:0/2343379769 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234037070 con 0x7f42480cdd20 2024-04-25T11:11:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.865+0000 7f4244a00640 1 -- 172.21.15.7:0/2343379769 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4234032440 con 0x7f42480cdd20 2024-04-25T11:11:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.865+0000 7f4244a00640 1 -- 172.21.15.7:0/2343379769 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234032810 con 0x7f42480cdd20 2024-04-25T11:11:00.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.866+0000 7fd053200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd0540ceaf0 0x7fd05406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.866+0000 7fd053200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd054156460 0x7fd05415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.866+0000 7fd053200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd05415cee0 0x7fd0540b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.866+0000 7fd053200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd0541605f0 con 0x7fd05415cee0 2024-04-25T11:11:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.866+0000 7fd053200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd0540b0d70 con 0x7fd0540ceaf0 2024-04-25T11:11:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.866+0000 7fd053200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd054160860 con 0x7fd054156460 2024-04-25T11:11:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.867+0000 7f424ce00640 1 -- 172.21.15.7:0/2343379769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42480cdd20 msgr2=0x7f42480cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.867+0000 7f424ce00640 1 --2- 172.21.15.7:0/2343379769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42480cdd20 0x7f42480cf6f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4234009870 tx=0x7f423402a270 comp rx=0 tx=0).stop 2024-04-25T11:11:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.867+0000 7fd051400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd054156460 0x7fd05415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.867+0000 7f424ce00640 1 -- 172.21.15.7:0/2343379769 shutdown_connections 2024-04-25T11:11:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.867+0000 7f424ce00640 1 --2- 172.21.15.7:0/2343379769 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f424806d580 0x7f424806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.867+0000 7f424ce00640 1 --2- 172.21.15.7:0/2343379769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42480cdd20 0x7f42480cf6f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.867+0000 7fd051e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd0540ceaf0 0x7fd05406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.867+0000 7f424ce00640 1 -- 172.21.15.7:0/2343379769 >> 172.21.15.7:0/2343379769 conn(0x7f42480732b0 msgr2=0x7f42480736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.867+0000 7fd051400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd054156460 0x7fd05415c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:54928/0 (socket says 172.21.15.7:54928) 2024-04-25T11:11:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.867+0000 7fd051400640 1 -- 172.21.15.7:0/2843048608 learned_addr learned my addr 172.21.15.7:0/2843048608 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.868+0000 7fd052800640 1 --2- 172.21.15.7:0/2843048608 >> v2:172.21.15.7:3300/0 conn(0x7fd05415cee0 0x7fd0540b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.868+0000 7f424ce00640 1 -- 172.21.15.7:0/2343379769 shutdown_connections 2024-04-25T11:11:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.870+0000 7f39bec00640 1 Processor -- start 2024-04-25T11:11:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.870+0000 7f39bec00640 1 -- start start 2024-04-25T11:11:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.869+0000 7fd051400640 1 -- 172.21.15.7:0/2843048608 >> v2:172.21.15.87:3300/0 conn(0x7fd0540ceaf0 msgr2=0x7fd05406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.869+0000 7fd051400640 1 --2- 172.21.15.7:0/2843048608 >> v2:172.21.15.87:3300/0 conn(0x7fd0540ceaf0 0x7fd05406eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.869+0000 7fd051400640 1 -- 172.21.15.7:0/2843048608 >> v2:172.21.15.7:3300/0 conn(0x7fd05415cee0 msgr2=0x7fd0540b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.869+0000 7fd051400640 1 --2- 172.21.15.7:0/2843048608 >> v2:172.21.15.7:3300/0 conn(0x7fd05415cee0 0x7fd0540b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.869+0000 7fd051400640 1 -- 172.21.15.7:0/2843048608 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0540cef00 con 0x7fd054156460 2024-04-25T11:11:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.868+0000 7f424ce00640 1 -- 172.21.15.7:0/2343379769 wait complete. 2024-04-25T11:11:00.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.871+0000 7fd051e00640 1 --2- 172.21.15.7:0/2843048608 >> v2:172.21.15.87:3300/0 conn(0x7fd0540ceaf0 0x7fd05406eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:11:00.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.872+0000 7fd051400640 1 --2- 172.21.15.7:0/2843048608 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd054156460 0x7fd05415c9a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd05406eb50 tx=0x7fd04402e620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7352dd25bd201d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.872+0000 7f424ce00640 1 Processor -- start 2024-04-25T11:11:00.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.872+0000 7f424ce00640 1 -- start start 2024-04-25T11:11:00.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.872+0000 7fd050a00640 1 -- 172.21.15.7:0/2843048608 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04403a070 con 0x7fd054156460 2024-04-25T11:11:00.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.873+0000 7fd050a00640 1 -- 172.21.15.7:0/2843048608 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd044033710 con 0x7fd054156460 2024-04-25T11:11:00.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.873+0000 7fd050a00640 1 -- 172.21.15.7:0/2843048608 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044033b20 con 0x7fd054156460 2024-04-25T11:11:00.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.873+0000 7f39bec00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f39b806ddd0 0x7f39b80d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.874+0000 7fd053200640 1 -- 172.21.15.7:0/2843048608 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd054156460 msgr2=0x7fd05415c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.874+0000 7fd053200640 1 --2- 172.21.15.7:0/2843048608 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd054156460 0x7fd05415c9a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd05406eb50 tx=0x7fd04402e620 comp rx=0 tx=0).stop 2024-04-25T11:11:00.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7f424ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f424806d580 0x7f424815b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7fd053200640 1 -- 172.21.15.7:0/2843048608 shutdown_connections 2024-04-25T11:11:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7fd053200640 1 --2- 172.21.15.7:0/2843048608 >> v2:172.21.15.7:3300/0 conn(0x7fd05415cee0 0x7fd0540b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7fd053200640 1 --2- 172.21.15.7:0/2843048608 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd054156460 0x7fd05415c9a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7fd053200640 1 --2- 172.21.15.7:0/2843048608 >> v2:172.21.15.87:3300/0 conn(0x7fd0540ceaf0 0x7fd05406eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7fd053200640 1 -- 172.21.15.7:0/2843048608 >> 172.21.15.7:0/2843048608 conn(0x7fd0540740f0 msgr2=0x7fd05406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:00.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.873+0000 7f39bec00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b80d1f40 0x7f39b806cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.874+0000 7f39bec00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39b80d2350 con 0x7f39b80b0fd0 2024-04-25T11:11:00.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.874+0000 7f39bec00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f39b80b19e0 con 0x7f39b806ddd0 2024-04-25T11:11:00.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.874+0000 7f39bec00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f39b8075c40 con 0x7f39b80d1f40 2024-04-25T11:11:00.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7f39bd800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f39b80b0fd0 0x7f39b80b3330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33798/0 (socket says 172.21.15.7:33798) 2024-04-25T11:11:00.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7f39bd800640 1 -- 172.21.15.7:0/540959624 learned_addr learned my addr 172.21.15.7:0/540959624 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:00.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7f39bce00640 1 --2- 172.21.15.7:0/540959624 >> v2:172.21.15.87:3300/0 conn(0x7f39b806ddd0 0x7f39b80d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7f4245e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f424806d580 0x7f424815b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7f4245e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f424806d580 0x7f424815b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43356/0 (socket says 172.21.15.7:43356) 2024-04-25T11:11:00.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7f4245e00640 1 -- 172.21.15.7:0/3918114671 learned_addr learned my addr 172.21.15.7:0/3918114671 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.875+0000 7f39be200640 1 --2- 172.21.15.7:0/540959624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b80d1f40 0x7f39b806cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.876+0000 7f424ce00640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42480cdd20 0x7f42481456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.876+0000 7f424ce00640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42480d1140 0x7f42481427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.876+0000 7f39b7e00640 1 -- 172.21.15.7:0/540959624 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1379491332 0 0) 0x7f39a4001110 con 0x7f39b80b0fd0 2024-04-25T11:11:00.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.876+0000 7f39b7e00640 1 -- 172.21.15.7:0/540959624 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39a0003520 con 0x7f39b80b0fd0 2024-04-25T11:11:00.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.876+0000 7fd053200640 1 -- 172.21.15.7:0/2843048608 shutdown_connections 2024-04-25T11:11:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.876+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42480d0370 con 0x7f424806d580 2024-04-25T11:11:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.876+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f42480ce130 con 0x7f42480cdd20 2024-04-25T11:11:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.876+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4248164bc0 con 0x7f42480d1140 2024-04-25T11:11:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.876+0000 7f4246800640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42480d1140 0x7f42481427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.876+0000 7f39b7e00640 1 -- 172.21.15.7:0/540959624 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 794332988 0 0) 0x7f39a0003520 con 0x7f39b80b0fd0 2024-04-25T11:11:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.877+0000 7f39bce00640 1 -- 172.21.15.7:0/540959624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b80d1f40 msgr2=0x7f39b806cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.877+0000 7f39bce00640 1 --2- 172.21.15.7:0/540959624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b80d1f40 0x7f39b806cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.877+0000 7f39bce00640 1 -- 172.21.15.7:0/540959624 >> v1:172.21.15.7:6789/0 conn(0x7f39b80b0fd0 legacy=0x7f39b80b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.877+0000 7f39bce00640 1 -- 172.21.15.7:0/540959624 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b80b1c80 con 0x7f39b806ddd0 2024-04-25T11:11:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.877+0000 7f4245400640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42480cdd20 0x7f42481456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.878+0000 7f4245e00640 1 -- 172.21.15.7:0/3918114671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42480d1140 msgr2=0x7f42481427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.878+0000 7f4245e00640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42480d1140 0x7f42481427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.878+0000 7f4245e00640 1 -- 172.21.15.7:0/3918114671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42480cdd20 msgr2=0x7f42481456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.878+0000 7f4245e00640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42480cdd20 0x7f42481456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.878+0000 7f4245e00640 1 -- 172.21.15.7:0/3918114671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4248143550 con 0x7f424806d580 2024-04-25T11:11:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.879+0000 7f4245e00640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f424806d580 0x7f424815b010 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f423800c3f0 tx=0x7f423800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.879+0000 7f423e000640 1 -- 172.21.15.7:0/3918114671 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42380052d0 con 0x7f424806d580 2024-04-25T11:11:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.880+0000 7f423e000640 1 -- 172.21.15.7:0/3918114671 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4238005490 con 0x7f424806d580 2024-04-25T11:11:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.880+0000 7f423e000640 1 -- 172.21.15.7:0/3918114671 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238007260 con 0x7f424806d580 2024-04-25T11:11:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.880+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42480b1fc0 con 0x7f424806d580 2024-04-25T11:11:00.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.880+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42480b2580 con 0x7f424806d580 2024-04-25T11:11:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.876+0000 7fd053200640 1 -- 172.21.15.7:0/2843048608 wait complete. 2024-04-25T11:11:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.880+0000 7f39bce00640 1 --2- 172.21.15.7:0/540959624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b806ddd0 0x7f39b80d1a00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0097a0 tx=0x7f39ac02a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1711a240b15b3ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.880+0000 7f39b7e00640 1 -- 172.21.15.7:0/540959624 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac037070 con 0x7f39b806ddd0 2024-04-25T11:11:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.881+0000 7f39b7e00640 1 -- 172.21.15.7:0/540959624 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39ac02aa90 con 0x7f39b806ddd0 2024-04-25T11:11:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.881+0000 7fd053200640 1 Processor -- start 2024-04-25T11:11:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.881+0000 7f39b7e00640 1 -- 172.21.15.7:0/540959624 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4001840 con 0x7f39b806ddd0 2024-04-25T11:11:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.881+0000 7fd053200640 1 -- start start 2024-04-25T11:11:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.882+0000 7f39bec00640 1 -- 172.21.15.7:0/540959624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b806ddd0 msgr2=0x7f39b80d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.882+0000 7f39bec00640 1 --2- 172.21.15.7:0/540959624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b806ddd0 0x7f39b80d1a00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0097a0 tx=0x7f39ac02a000 comp rx=0 tx=0).stop 2024-04-25T11:11:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.883+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4248050740 con 0x7f424806d580 2024-04-25T11:11:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.883+0000 7f39bec00640 1 -- 172.21.15.7:0/540959624 shutdown_connections 2024-04-25T11:11:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.883+0000 7f39bec00640 1 --2- 172.21.15.7:0/540959624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b80d1f40 0x7f39b806cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.883+0000 7f39bec00640 1 --2- 172.21.15.7:0/540959624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b806ddd0 0x7f39b80d1a00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.883+0000 7f39bec00640 1 -- 172.21.15.7:0/540959624 >> 172.21.15.7:0/540959624 conn(0x7f39b8073b60 msgr2=0x7f39b80b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.884+0000 7f39bec00640 1 -- 172.21.15.7:0/540959624 shutdown_connections 2024-04-25T11:11:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.884+0000 7f39bec00640 1 -- 172.21.15.7:0/540959624 wait complete. 2024-04-25T11:11:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.884+0000 7fd053200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0540ceaf0 0x7fd05414da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.884+0000 7fd053200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd054156460 0x7fd0541464f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.885+0000 7fd053200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05415cee0 0x7fd05415e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.885+0000 7fd053200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0540b1050 con 0x7fd05415cee0 2024-04-25T11:11:00.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.885+0000 7fd053200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd0540b4bb0 con 0x7fd054156460 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.885+0000 7fd052800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05415cee0 0x7fd05415e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.885+0000 7fd052800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05415cee0 0x7fd05415e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43362/0 (socket says 172.21.15.7:43362) 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.885+0000 7fd051400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd054156460 0x7fd0541464f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.886+0000 7fd051400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd054156460 0x7fd0541464f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:55498/0 (socket says 172.21.15.7:55498) 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.886+0000 7fd051400640 1 -- 172.21.15.7:0/2692798259 learned_addr learned my addr 172.21.15.7:0/2692798259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.886+0000 7fd051e00640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0540ceaf0 0x7fd05414da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.886+0000 7fd053200640 1 -- 172.21.15.7:0/2692798259 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd054156870 con 0x7fd0540ceaf0 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.887+0000 7fd052800640 1 -- 172.21.15.7:0/2692798259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0540ceaf0 msgr2=0x7fd05414da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.887+0000 7fd052800640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0540ceaf0 0x7fd05414da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.887+0000 7fd052800640 1 -- 172.21.15.7:0/2692798259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd054156460 msgr2=0x7fd0541464f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.887+0000 7fd052800640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd054156460 0x7fd0541464f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.887+0000 7fd052800640 1 -- 172.21.15.7:0/2692798259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05414dfd0 con 0x7fd05415cee0 2024-04-25T11:11:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.887+0000 7f39bec00640 1 Processor -- start 2024-04-25T11:11:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.888+0000 7f39bec00640 1 -- start start 2024-04-25T11:11:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.888+0000 7fd052800640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05415cee0 0x7fd05415e7c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd04000e7f0 tx=0x7fd04000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.890+0000 7f39bec00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b806ddd0 0x7f39b815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.888+0000 7f423e000640 1 -- 172.21.15.7:0/3918114671 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4238007400 con 0x7f424806d580 2024-04-25T11:11:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.890+0000 7f423e000640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4228074580 0x7f4228076a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.890+0000 7f423e000640 1 -- 172.21.15.7:0/3918114671 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f4238094940 con 0x7f424806d580 2024-04-25T11:11:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.890+0000 7f39bec00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b80b0fd0 0x7f39b809e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.890+0000 7f39bec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b80d1f40 0x7f39b814da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.890+0000 7f39bec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39b80d26c0 con 0x7f39b80d1f40 2024-04-25T11:11:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.890+0000 7f39bec00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f39b809b990 con 0x7f39b80b0fd0 2024-04-25T11:11:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.890+0000 7f39bec00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f39b806d4c0 con 0x7f39b806ddd0 2024-04-25T11:11:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.891+0000 7f4245400640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4228074580 0x7f4228076a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.891+0000 7f39bd800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b806ddd0 0x7f39b815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.891+0000 7f39bce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b80b0fd0 0x7f39b809e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.891+0000 7f39be200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b80d1f40 0x7f39b814da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.891+0000 7f39bce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b80b0fd0 0x7f39b809e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:55500/0 (socket says 172.21.15.7:55500) 2024-04-25T11:11:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.891+0000 7f39bce00640 1 -- 172.21.15.7:0/2644451546 learned_addr learned my addr 172.21.15.7:0/2644451546 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.892+0000 7f39bce00640 1 -- 172.21.15.7:0/2644451546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b806ddd0 msgr2=0x7f39b815ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.892+0000 7f39bce00640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b806ddd0 0x7f39b815ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.892+0000 7f39bce00640 1 -- 172.21.15.7:0/2644451546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b80d1f40 msgr2=0x7f39b814da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.892+0000 7f39bce00640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b80d1f40 0x7f39b814da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.892+0000 7f39bce00640 1 -- 172.21.15.7:0/2644451546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b815b730 con 0x7f39b80b0fd0 2024-04-25T11:11:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.893+0000 7f4245400640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4228074580 0x7f4228076a90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4234034000 tx=0x7f423402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.893+0000 7f39bce00640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b80b0fd0 0x7f39b809e9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0098c0 tx=0x7f39ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.894+0000 7f39b5600640 1 -- 172.21.15.7:0/2644451546 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac037050 con 0x7f39b80b0fd0 2024-04-25T11:11:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.894+0000 7f39b5600640 1 -- 172.21.15.7:0/2644451546 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39ac038040 con 0x7f39b80b0fd0 2024-04-25T11:11:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.894+0000 7f39b5600640 1 -- 172.21.15.7:0/2644451546 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac004490 con 0x7f39b80b0fd0 2024-04-25T11:11:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.894+0000 7f39bec00640 1 -- 172.21.15.7:0/2644451546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b815ba10 con 0x7f39b80b0fd0 2024-04-25T11:11:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.894+0000 7f39bec00640 1 -- 172.21.15.7:0/2644451546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b809c020 con 0x7f39b80b0fd0 2024-04-25T11:11:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.896+0000 7f39bec00640 1 -- 172.21.15.7:0/2644451546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b815b4a0 con 0x7f39b80b0fd0 2024-04-25T11:11:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.897+0000 7fd04a000640 1 -- 172.21.15.7:0/2692798259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040017070 con 0x7fd05415cee0 2024-04-25T11:11:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.898+0000 7f423e000640 1 -- 172.21.15.7:0/3918114671 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f424806ded0 con 0x7f424806d580 2024-04-25T11:11:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.898+0000 7fd053200640 1 -- 172.21.15.7:0/2692798259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd054146a30 con 0x7fd05415cee0 2024-04-25T11:11:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.899+0000 7fd04a000640 1 -- 172.21.15.7:0/2692798259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd04000c730 con 0x7fd05415cee0 2024-04-25T11:11:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.899+0000 7fd04a000640 1 -- 172.21.15.7:0/2692798259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040017070 con 0x7fd05415cee0 2024-04-25T11:11:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.899+0000 7f39b5600640 1 -- 172.21.15.7:0/2644451546 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f39ac003610 con 0x7f39b80b0fd0 2024-04-25T11:11:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.901+0000 7f39b5600640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39980744b0 0x7f39980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.901+0000 7f39b5600640 1 -- 172.21.15.7:0/2644451546 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f39ac07cf00 con 0x7f39b80b0fd0 2024-04-25T11:11:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.902+0000 7f39bd800640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39980744b0 0x7f39980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.901+0000 7fd053200640 1 -- 172.21.15.7:0/2692798259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05414b330 con 0x7fd05415cee0 2024-04-25T11:11:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.903+0000 7fd053200640 1 -- 172.21.15.7:0/2692798259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd054050740 con 0x7fd05415cee0 2024-04-25T11:11:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.903+0000 7f39bd800640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39980744b0 0x7f39980769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39a4001eb0 tx=0x7f39a4001290 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.906+0000 7fd04a000640 1 -- 172.21.15.7:0/2692798259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd04001a070 con 0x7fd05415cee0 2024-04-25T11:11:00.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.908+0000 7fd04a000640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0340746e0 0x7fd034076bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.909+0000 7fd04a000640 1 -- 172.21.15.7:0/2692798259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd040099060 con 0x7fd05415cee0 2024-04-25T11:11:00.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.910+0000 7fd051e00640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0340746e0 0x7fd034076bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.911+0000 7fd051e00640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0340746e0 0x7fd034076bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd054160520 tx=0x7fd04c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.913+0000 7f39b5600640 1 -- 172.21.15.7:0/2644451546 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f39ac04c020 con 0x7f39b80b0fd0 2024-04-25T11:11:00.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.916+0000 7fd04a000640 1 -- 172.21.15.7:0/2692798259 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd04005e3b0 con 0x7fd05415cee0 2024-04-25T11:11:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.939+0000 7f84fd000640 1 Processor -- start 2024-04-25T11:11:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.939+0000 7f84fd000640 1 -- start start 2024-04-25T11:11:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.940+0000 7fe3e1200640 1 Processor -- start 2024-04-25T11:11:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.941+0000 7fe3e1200640 1 -- start start 2024-04-25T11:11:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.942+0000 7f84fd000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f84f80d2360 0x7f84f809cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.943+0000 7f84fd000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 0x7f84f809bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.943+0000 7f84f5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f84f80d2360 0x7f84f809cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.944+0000 7f84fd000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84f80cfba0 con 0x7f84f80b4b80 2024-04-25T11:11:00.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.944+0000 7f84fd000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f84f80cf000 con 0x7f84f80d2360 2024-04-25T11:11:00.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.944+0000 7fe3e1200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3d4050670 0x7fe3d40d9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.944+0000 7f84f6800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 0x7f84f809bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.944+0000 7fe3e1200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d4050a80 0x7fe3d40d9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.944+0000 7f84f6800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 0x7f84f809bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:54968/0 (socket says 172.21.15.7:54968) 2024-04-25T11:11:00.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.945+0000 7fe3d9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3d4050670 0x7fe3d40d9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.945+0000 7fe3e1200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d40e0c10 con 0x7fe3d404f370 2024-04-25T11:11:00.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.944+0000 7f84f6800640 1 -- 172.21.15.7:0/2581697983 learned_addr learned my addr 172.21.15.7:0/2581697983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.945+0000 7f84fd000640 1 -- 172.21.15.7:0/2581697983 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f84f80d2940 con 0x7f84f80b4f90 2024-04-25T11:11:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.945+0000 7fe3e1200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d4007120 con 0x7fe3d4050670 2024-04-25T11:11:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.945+0000 7fe3e1200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d40538b0 con 0x7fe3d4050a80 2024-04-25T11:11:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.945+0000 7fe3d9400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d4050a80 0x7fe3d40d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.945+0000 7fe3d9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3d4050670 0x7fe3d40d9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:55520/0 (socket says 172.21.15.7:55520) 2024-04-25T11:11:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.946+0000 7fe3d9400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d4050a80 0x7fe3d40d9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:54982/0 (socket says 172.21.15.7:54982) 2024-04-25T11:11:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.946+0000 7fe3d9400640 1 -- 172.21.15.7:0/4043101558 learned_addr learned my addr 172.21.15.7:0/4043101558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.947+0000 7f84f6800640 1 -- 172.21.15.7:0/2581697983 >> v2:172.21.15.87:3300/0 conn(0x7f84f80d2360 msgr2=0x7f84f809cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.947+0000 7f84f6800640 1 --2- 172.21.15.7:0/2581697983 >> v2:172.21.15.87:3300/0 conn(0x7f84f80d2360 0x7f84f809cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.947+0000 7f84f6800640 1 -- 172.21.15.7:0/2581697983 >> v1:172.21.15.7:6789/0 conn(0x7f84f80b4b80 legacy=0x7f84f80cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.947+0000 7f84f6800640 1 -- 172.21.15.7:0/2581697983 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f80ce9b0 con 0x7f84f80b4f90 2024-04-25T11:11:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.947+0000 7fe3d9e00640 1 -- 172.21.15.7:0/4043101558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d4050a80 msgr2=0x7fe3d40d9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.947+0000 7fe3d9e00640 1 --2- 172.21.15.7:0/4043101558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d4050a80 0x7fe3d40d9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.948+0000 7fe3d8a00640 1 -- 172.21.15.7:0/4043101558 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383748946 0 0) 0x7fe3d40e0c10 con 0x7fe3d404f370 2024-04-25T11:11:00.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.949+0000 7f84f6800640 1 --2- 172.21.15.7:0/2581697983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 0x7f84f809bae0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009870 tx=0x7f84e002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b11915ed09db24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.948+0000 7fe3d9e00640 1 -- 172.21.15.7:0/4043101558 >> v1:172.21.15.7:6789/0 conn(0x7fe3d404f370 legacy=0x7fe3d404f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.948+0000 7fe3d9e00640 1 -- 172.21.15.7:0/4043101558 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d4053b90 con 0x7fe3d4050670 2024-04-25T11:11:00.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.950+0000 7f84f4a00640 1 -- 172.21.15.7:0/2581697983 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e0037070 con 0x7f84f80b4f90 2024-04-25T11:11:00.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.950+0000 7f84f4a00640 1 -- 172.21.15.7:0/2581697983 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e0032440 con 0x7f84f80b4f90 2024-04-25T11:11:00.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.950+0000 7fe3d9e00640 1 --2- 172.21.15.7:0/4043101558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3d4050670 0x7fe3d40d9920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0097a0 tx=0x7fe3cc02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da914465840b26d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.951+0000 7fe3d8a00640 1 -- 172.21.15.7:0/4043101558 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc037070 con 0x7fe3d4050670 2024-04-25T11:11:00.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.951+0000 7fe3d8a00640 1 -- 172.21.15.7:0/4043101558 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3cc032440 con 0x7fe3d4050670 2024-04-25T11:11:00.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.951+0000 7fe3d8a00640 1 -- 172.21.15.7:0/4043101558 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc037070 con 0x7fe3d4050670 2024-04-25T11:11:00.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.951+0000 7f84fd000640 1 -- 172.21.15.7:0/2581697983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 msgr2=0x7f84f809bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.951+0000 7f84fd000640 1 --2- 172.21.15.7:0/2581697983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 0x7f84f809bae0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009870 tx=0x7f84e002a250 comp rx=0 tx=0).stop 2024-04-25T11:11:00.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.952+0000 7f84fd000640 1 -- 172.21.15.7:0/2581697983 shutdown_connections 2024-04-25T11:11:00.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.952+0000 7f84fd000640 1 --2- 172.21.15.7:0/2581697983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 0x7f84f809bae0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.952+0000 7f84fd000640 1 --2- 172.21.15.7:0/2581697983 >> v2:172.21.15.87:3300/0 conn(0x7f84f80d2360 0x7f84f809cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.952+0000 7f84fd000640 1 -- 172.21.15.7:0/2581697983 >> 172.21.15.7:0/2581697983 conn(0x7f84f80b54b0 msgr2=0x7f84f80b58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:00.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.952+0000 7fe3e1200640 1 -- 172.21.15.7:0/4043101558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3d4050670 msgr2=0x7fe3d40d9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.952+0000 7fe3e1200640 1 --2- 172.21.15.7:0/4043101558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3d4050670 0x7fe3d40d9920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0097a0 tx=0x7fe3cc02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.952+0000 7fe3e1200640 1 -- 172.21.15.7:0/4043101558 shutdown_connections 2024-04-25T11:11:00.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.952+0000 7fe3e1200640 1 --2- 172.21.15.7:0/4043101558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d4050a80 0x7fe3d40d9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.952+0000 7fe3e1200640 1 --2- 172.21.15.7:0/4043101558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3d4050670 0x7fe3d40d9920 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.952+0000 7fe3e1200640 1 -- 172.21.15.7:0/4043101558 >> 172.21.15.7:0/4043101558 conn(0x7fe3d4054d20 msgr2=0x7fe3d4055130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:00.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.953+0000 7fe3e1200640 1 -- 172.21.15.7:0/4043101558 shutdown_connections 2024-04-25T11:11:00.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.953+0000 7f84fd000640 1 -- 172.21.15.7:0/2581697983 shutdown_connections 2024-04-25T11:11:00.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.953+0000 7fe3e1200640 1 -- 172.21.15.7:0/4043101558 wait complete. 2024-04-25T11:11:00.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.953+0000 7f84fd000640 1 -- 172.21.15.7:0/2581697983 wait complete. 2024-04-25T11:11:00.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.954+0000 7f049b200640 1 Processor -- start 2024-04-25T11:11:00.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.954+0000 7f049b200640 1 -- start start 2024-04-25T11:11:00.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.957+0000 7f84fd000640 1 Processor -- start 2024-04-25T11:11:00.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.957+0000 7f049b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f049c0cf890 0x7f049c0d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.958+0000 7fe3e1200640 1 Processor -- start 2024-04-25T11:11:00.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.958+0000 7f84fd000640 1 -- start start 2024-04-25T11:11:00.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.958+0000 7f049b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c06eb50 0x7f049c0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.958+0000 7f0499e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f049c15d0e0 0x7f049c0cf180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:49478/0 (socket says 172.21.15.7:49478) 2024-04-25T11:11:00.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.958+0000 7fe3e1200640 1 -- start start 2024-04-25T11:11:00.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.958+0000 7f0499400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f049c0cf890 0x7f049c0d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.958+0000 7f0499400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f049c0cf890 0x7f049c0d24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:55532/0 (socket says 172.21.15.7:55532) 2024-04-25T11:11:00.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.959+0000 7f0499e00640 1 -- 172.21.15.7:0/813402816 learned_addr learned my addr 172.21.15.7:0/813402816 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:00.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.959+0000 7f049a800640 1 --2- 172.21.15.7:0/813402816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c06eb50 0x7f049c0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.959+0000 7f049b200640 1 -- 172.21.15.7:0/813402816 --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f049c0d2b00 con 0x7f049c15d0e0 2024-04-25T11:11:00.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.960+0000 7f049b200640 1 -- 172.21.15.7:0/813402816 --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f049c0d2de0 con 0x7f049c0cf890 2024-04-25T11:11:00.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.960+0000 7f049b200640 1 -- 172.21.15.7:0/813402816 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f049c0d30c0 con 0x7f049c06eb50 2024-04-25T11:11:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.960+0000 7f84fd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f80b4b80 0x7f84f809d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7f84fd000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 0x7f84f8153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7f84fd000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84f80d2360 0x7f84f80c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7f84fd000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80ce750 con 0x7f84f80b4b80 2024-04-25T11:11:00.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7f84fd000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80ce5d0 con 0x7f84f80d2360 2024-04-25T11:11:00.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7f84fd000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f84f809c020 con 0x7f84f80b4f90 2024-04-25T11:11:00.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7fe3e1200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d404f370 0x7fe3d404ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7fe3e1200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3d4050670 0x7fe3d4048b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7fe3e1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4049050 0x7fe3d4049460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7fe3e1200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d4054670 con 0x7fe3d4049050 2024-04-25T11:11:00.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7fe3e1200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d4054150 con 0x7fe3d4050670 2024-04-25T11:11:00.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7fe3e1200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d4049e50 con 0x7fe3d404f370 2024-04-25T11:11:00.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.961+0000 7f84f5400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 0x7f84f8153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.962+0000 7f84f5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f80b4b80 0x7f84f809d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.962+0000 7f84f5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f80b4b80 0x7f84f809d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43368/0 (socket says 172.21.15.7:43368) 2024-04-25T11:11:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.962+0000 7f84f5e00640 1 -- 172.21.15.7:0/4125106887 learned_addr learned my addr 172.21.15.7:0/4125106887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.962+0000 7f84f6800640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84f80d2360 0x7f84f80c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.962+0000 7fe3d9e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d404f370 0x7fe3d404ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.962+0000 7fe3da800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4049050 0x7fe3d4049460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.962+0000 7fe3d9e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d404f370 0x7fe3d404ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:55012/0 (socket says 172.21.15.7:55012) 2024-04-25T11:11:00.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.962+0000 7fe3d9e00640 1 -- 172.21.15.7:0/513872260 learned_addr learned my addr 172.21.15.7:0/513872260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.960+0000 7f0498a00640 1 -- 172.21.15.7:0/813402816 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3145675350 0 0) 0x7f049c0d2b00 con 0x7f049c15d0e0 2024-04-25T11:11:00.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7f0498a00640 1 -- 172.21.15.7:0/813402816 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0480003520 con 0x7f049c15d0e0 2024-04-25T11:11:00.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7fe3d9400640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3d4050670 0x7fe3d4048b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7f84f5e00640 1 -- 172.21.15.7:0/4125106887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 msgr2=0x7f84f8153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7f84f5e00640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 0x7f84f8153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7f84f5e00640 1 -- 172.21.15.7:0/4125106887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84f80d2360 msgr2=0x7f84f80c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7f84f5e00640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84f80d2360 0x7f84f80c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7f84f5e00640 1 -- 172.21.15.7:0/4125106887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f80cc740 con 0x7f84f80b4b80 2024-04-25T11:11:00.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7fe3da800640 1 -- 172.21.15.7:0/513872260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d404f370 msgr2=0x7fe3d404ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7fe3da800640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d404f370 0x7fe3d404ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7fe3da800640 1 -- 172.21.15.7:0/513872260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3d4050670 msgr2=0x7fe3d4048b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7fe3da800640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3d4050670 0x7fe3d4048b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7fe3da800640 1 -- 172.21.15.7:0/513872260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d40499a0 con 0x7fe3d4049050 2024-04-25T11:11:00.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.963+0000 7fe3d9400640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3d4050670 0x7fe3d4048b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:00.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.964+0000 7f84f5e00640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f80b4b80 0x7f84f809d9e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84e800e840 tx=0x7f84e800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.964+0000 7f0499400640 1 -- 172.21.15.7:0/813402816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c06eb50 msgr2=0x7f049c0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.964+0000 7f0499400640 1 --2- 172.21.15.7:0/813402816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c06eb50 0x7f049c0cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.964+0000 7f0498a00640 1 -- 172.21.15.7:0/813402816 <== mon.1 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2037862026 0 0) 0x7f0480003520 con 0x7f049c15d0e0 2024-04-25T11:11:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.964+0000 7f0499400640 1 -- 172.21.15.7:0/813402816 >> v1:172.21.15.87:6789/0 conn(0x7f049c15d0e0 legacy=0x7f049c0cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.964+0000 7fe3da800640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4049050 0x7fe3d4049460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc06aef0 tx=0x7fe3dc06ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.964+0000 7f0499400640 1 -- 172.21.15.7:0/813402816 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c0d33a0 con 0x7f049c0cf890 2024-04-25T11:11:00.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.965+0000 7fe3d2000640 1 -- 172.21.15.7:0/513872260 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc08cad0 con 0x7fe3d4049050 2024-04-25T11:11:00.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.965+0000 7fe3d2000640 1 -- 172.21.15.7:0/513872260 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3dc090030 con 0x7fe3d4049050 2024-04-25T11:11:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.964+0000 7f84ee000640 1 -- 172.21.15.7:0/4125106887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8017070 con 0x7f84f80b4b80 2024-04-25T11:11:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.964+0000 7f84ee000640 1 -- 172.21.15.7:0/4125106887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e800c6d0 con 0x7f84f80b4b80 2024-04-25T11:11:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.964+0000 7f84ee000640 1 -- 172.21.15.7:0/4125106887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e800caa0 con 0x7f84f80b4b80 2024-04-25T11:11:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.964+0000 7f84fd000640 1 -- 172.21.15.7:0/4125106887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f80cca20 con 0x7f84f80b4b80 2024-04-25T11:11:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.965+0000 7f84fd000640 1 -- 172.21.15.7:0/4125106887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f80ccfe0 con 0x7f84f80b4b80 2024-04-25T11:11:00.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.966+0000 7f0499400640 1 --2- 172.21.15.7:0/813402816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f049c0cf890 0x7f049c0d24b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f04840097a0 tx=0x7f048402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3fd2e5bd6125bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.966+0000 7fe3e1200640 1 -- 172.21.15.7:0/513872260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d404cff0 con 0x7fe3d4049050 2024-04-25T11:11:00.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.966+0000 7fe3e1200640 1 -- 172.21.15.7:0/513872260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d404bad0 con 0x7fe3d4049050 2024-04-25T11:11:00.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.967+0000 7f0498a00640 1 -- 172.21.15.7:0/813402816 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484037070 con 0x7f049c0cf890 2024-04-25T11:11:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.967+0000 7fe3d2000640 1 -- 172.21.15.7:0/513872260 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc075b10 con 0x7fe3d4049050 2024-04-25T11:11:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.968+0000 7fe3e1200640 1 -- 172.21.15.7:0/513872260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b4005290 con 0x7fe3d4049050 2024-04-25T11:11:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.967+0000 7f0498a00640 1 -- 172.21.15.7:0/813402816 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048402aba0 con 0x7f049c0cf890 2024-04-25T11:11:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.968+0000 7f049b200640 1 -- 172.21.15.7:0/813402816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f049c0cf890 msgr2=0x7f049c0d24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.970+0000 7f84fd000640 1 -- 172.21.15.7:0/4125106887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f8050740 con 0x7f84f80b4b80 2024-04-25T11:11:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.968+0000 7f049b200640 1 --2- 172.21.15.7:0/813402816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f049c0cf890 0x7f049c0d24b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f04840097a0 tx=0x7f048402a110 comp rx=0 tx=0).stop 2024-04-25T11:11:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.970+0000 7fa380800640 1 Processor -- start 2024-04-25T11:11:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.967+0000 7f0498a00640 1 -- 172.21.15.7:0/813402816 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c001690 con 0x7f049c0cf890 2024-04-25T11:11:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.972+0000 7f049b200640 1 -- 172.21.15.7:0/813402816 shutdown_connections 2024-04-25T11:11:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.972+0000 7f049b200640 1 --2- 172.21.15.7:0/813402816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c06eb50 0x7f049c0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.971+0000 7fe3d2000640 1 -- 172.21.15.7:0/513872260 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe3dc075cb0 con 0x7fe3d4049050 2024-04-25T11:11:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.972+0000 7fe3d2000640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3bc074500 0x7fe3bc076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.972+0000 7fe3d2000640 1 -- 172.21.15.7:0/513872260 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe3dc111990 con 0x7fe3d4049050 2024-04-25T11:11:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.973+0000 7f049b200640 1 --2- 172.21.15.7:0/813402816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f049c0cf890 0x7f049c0d24b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.973+0000 7f049b200640 1 -- 172.21.15.7:0/813402816 >> 172.21.15.7:0/813402816 conn(0x7f049c09b540 msgr2=0x7f049c09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.973+0000 7fe3d9e00640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3bc074500 0x7fe3bc076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.973+0000 7f049b200640 1 -- 172.21.15.7:0/813402816 shutdown_connections 2024-04-25T11:11:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.970+0000 7fa380800640 1 -- start start 2024-04-25T11:11:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.972+0000 7f84ee000640 1 -- 172.21.15.7:0/4125106887 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f84e801a070 con 0x7f84f80b4b80 2024-04-25T11:11:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.974+0000 7f84ee000640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84d40744e0 0x7f84d40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.975+0000 7fe3d9e00640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3bc074500 0x7fe3bc076a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc02a1e0 tx=0x7fe3cc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.975+0000 7fa380800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa37806d580 0x7fa37806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.975+0000 7f049b200640 1 -- 172.21.15.7:0/813402816 wait complete. 2024-04-25T11:11:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.975+0000 7fa380800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa37806ded0 con 0x7fa3780b4690 2024-04-25T11:11:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.975+0000 7fa380800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa37806ef80 con 0x7fa37814db80 2024-04-25T11:11:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.975+0000 7fa380800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa37806f260 con 0x7fa37806d580 2024-04-25T11:11:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.976+0000 7fa37d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa37806d580 0x7fa37806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.976+0000 7fa37d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa37806d580 0x7fa37806d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:55562/0 (socket says 172.21.15.7:55562) 2024-04-25T11:11:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.976+0000 7fa37d400640 1 -- 172.21.15.7:0/2309899898 learned_addr learned my addr 172.21.15.7:0/2309899898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.976+0000 7f84ee000640 1 -- 172.21.15.7:0/4125106887 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f84e80952f0 con 0x7f84f80b4b80 2024-04-25T11:11:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.976+0000 7f84f5400640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84d40744e0 0x7f84d40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.977+0000 7fa37d400640 1 -- 172.21.15.7:0/2309899898 >> v1:172.21.15.87:6789/0 conn(0x7fa37814db80 legacy=0x7fa3780b5e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T11:11:00.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.977+0000 7fa37d400640 1 -- 172.21.15.7:0/2309899898 >> v1:172.21.15.7:6789/0 conn(0x7fa3780b4690 legacy=0x7fa3780b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.977+0000 7fa37d400640 1 -- 172.21.15.7:0/2309899898 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37806f540 con 0x7fa37806d580 2024-04-25T11:11:00.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.979+0000 7f84f5400640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84d40744e0 0x7f84d40769f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f84f80d1280 tx=0x7f84f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.979+0000 7fa37d400640 1 --2- 172.21.15.7:0/2309899898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37806d580 0x7fa37806d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009870 tx=0x7fa36c02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a62f80875882f3ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.979+0000 7fa37ca00640 1 -- 172.21.15.7:0/2309899898 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c037070 con 0x7fa37806d580 2024-04-25T11:11:00.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.980+0000 7fa37ca00640 1 -- 172.21.15.7:0/2309899898 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa36c032440 con 0x7fa37806d580 2024-04-25T11:11:00.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.980+0000 7fa37ca00640 1 -- 172.21.15.7:0/2309899898 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c032850 con 0x7fa37806d580 2024-04-25T11:11:00.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.980+0000 7f049b200640 1 Processor -- start 2024-04-25T11:11:00.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.981+0000 7f049b200640 1 -- start start 2024-04-25T11:11:00.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.981+0000 7fe3d2000640 1 -- 172.21.15.7:0/513872260 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3d404cff0 con 0x7fe3d4049050 2024-04-25T11:11:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.981+0000 7fa380800640 1 -- 172.21.15.7:0/2309899898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37806d580 msgr2=0x7fa37806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.981+0000 7fa380800640 1 --2- 172.21.15.7:0/2309899898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37806d580 0x7fa37806d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009870 tx=0x7fa36c02a2c0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.981+0000 7fa380800640 1 -- 172.21.15.7:0/2309899898 shutdown_connections 2024-04-25T11:11:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.981+0000 7fa380800640 1 --2- 172.21.15.7:0/2309899898 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37806d580 0x7fa37806d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.982+0000 7fa380800640 1 -- 172.21.15.7:0/2309899898 >> 172.21.15.7:0/2309899898 conn(0x7fa378073160 msgr2=0x7fa378073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.982+0000 7fa380800640 1 -- 172.21.15.7:0/2309899898 shutdown_connections 2024-04-25T11:11:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.982+0000 7fa380800640 1 -- 172.21.15.7:0/2309899898 wait complete. 2024-04-25T11:11:00.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.984+0000 7f049b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f049c06eb50 0x7f049c0cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.984+0000 7f049b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c0cf890 0x7f049c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.984+0000 7f0499e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f049c06eb50 0x7f049c0cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.985+0000 7f049b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f049c0cc240 0x7f049c0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.985+0000 7f049b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f049c0cd7a0 con 0x7f049c06eb50 2024-04-25T11:11:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.985+0000 7f049b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f049c0b3f60 con 0x7f049c0cc240 2024-04-25T11:11:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.985+0000 7f049b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f049c0cd920 con 0x7f049c0cf890 2024-04-25T11:11:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.985+0000 7f0499400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c0cf890 0x7f049c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.985+0000 7f0499400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c0cf890 0x7f049c0caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:55016/0 (socket says 172.21.15.7:55016) 2024-04-25T11:11:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.985+0000 7f0499400640 1 -- 172.21.15.7:0/648484464 learned_addr learned my addr 172.21.15.7:0/648484464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.986+0000 7f049a800640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f049c0cc240 0x7f049c0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.986+0000 7fa380800640 1 Processor -- start 2024-04-25T11:11:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.986+0000 7fa380800640 1 -- start start 2024-04-25T11:11:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.986+0000 7f0499400640 1 -- 172.21.15.7:0/648484464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f049c0cc240 msgr2=0x7f049c0cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.986+0000 7f0499400640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f049c0cc240 0x7f049c0cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.986+0000 7f0499400640 1 -- 172.21.15.7:0/648484464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f049c06eb50 msgr2=0x7f049c0cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.986+0000 7f0499400640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f049c06eb50 0x7f049c0cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.986+0000 7f0499400640 1 -- 172.21.15.7:0/648484464 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c0b12f0 con 0x7f049c0cf890 2024-04-25T11:11:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.987+0000 7f0499400640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c0cf890 0x7f049c0caf70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f04840098c0 tx=0x7f04840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.988+0000 7f0492000640 1 -- 172.21.15.7:0/648484464 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484041020 con 0x7f049c0cf890 2024-04-25T11:11:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.988+0000 7f049b200640 1 -- 172.21.15.7:0/648484464 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049c0b15d0 con 0x7f049c0cf890 2024-04-25T11:11:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.988+0000 7f049b200640 1 -- 172.21.15.7:0/648484464 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f049c0b1b90 con 0x7f049c0cf890 2024-04-25T11:11:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.989+0000 7fa380800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37806d580 0x7fa378202b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.989+0000 7fa380800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa3780b4690 0x7fa378148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.990+0000 7fa380800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa37814db80 0x7fa3781600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:00.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.989+0000 7f0492000640 1 -- 172.21.15.7:0/648484464 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048403d620 con 0x7f049c0cf890 2024-04-25T11:11:00.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.989+0000 7f0492000640 1 -- 172.21.15.7:0/648484464 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484041020 con 0x7f049c0cf890 2024-04-25T11:11:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.990+0000 7fa380800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3780b6530 con 0x7fa37814db80 2024-04-25T11:11:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.990+0000 7fa380800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa37806fb00 con 0x7fa37806d580 2024-04-25T11:11:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.990+0000 7fa380800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa378070680 con 0x7fa3780b4690 2024-04-25T11:11:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.990+0000 7fa37de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37806d580 0x7fa378202b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.990+0000 7fa37de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37806d580 0x7fa378202b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:55588/0 (socket says 172.21.15.7:55588) 2024-04-25T11:11:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.990+0000 7fa37de00640 1 -- 172.21.15.7:0/3005936846 learned_addr learned my addr 172.21.15.7:0/3005936846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.991+0000 7fa37d400640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa3780b4690 0x7fa378148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.992+0000 7fa37de00640 1 -- 172.21.15.7:0/3005936846 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa3780b4690 msgr2=0x7fa378148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.992+0000 7fa37de00640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa3780b4690 0x7fa378148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.992+0000 7fa37de00640 1 -- 172.21.15.7:0/3005936846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa37814db80 msgr2=0x7fa3781600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:00.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.992+0000 7fa37de00640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa37814db80 0x7fa3781600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:00.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.992+0000 7fa37de00640 1 -- 172.21.15.7:0/3005936846 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa378149210 con 0x7fa37806d580 2024-04-25T11:11:00.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.993+0000 7f84ee000640 1 -- 172.21.15.7:0/4125106887 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f84e805f100 con 0x7f84f80b4b80 2024-04-25T11:11:00.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.994+0000 7fa37de00640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37806d580 0x7fa378202b10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa36400b3e0 tx=0x7fa36400b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:00.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.994+0000 7fa372000640 1 -- 172.21.15.7:0/3005936846 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364015020 con 0x7fa37806d580 2024-04-25T11:11:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.994+0000 7f049b200640 1 -- 172.21.15.7:0/648484464 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0474005290 con 0x7f049c0cf890 2024-04-25T11:11:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.995+0000 7fa380800640 1 -- 172.21.15.7:0/3005936846 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37815f040 con 0x7fa37806d580 2024-04-25T11:11:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.995+0000 7fa380800640 1 -- 172.21.15.7:0/3005936846 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37815f600 con 0x7fa37806d580 2024-04-25T11:11:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.995+0000 7fa372000640 1 -- 172.21.15.7:0/3005936846 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa364011830 con 0x7fa37806d580 2024-04-25T11:11:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.995+0000 7fa372000640 1 -- 172.21.15.7:0/3005936846 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364015020 con 0x7fa37806d580 2024-04-25T11:11:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.997+0000 7fa36b400640 1 -- 172.21.15.7:0/3005936846 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa378050740 con 0x7fa37806d580 2024-04-25T11:11:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:00.998+0000 7f0492000640 1 -- 172.21.15.7:0/648484464 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0484044020 con 0x7f049c0cf890 2024-04-25T11:11:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.000+0000 7f0492000640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04780746b0 0x7f0478076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.001+0000 7f0492000640 1 -- 172.21.15.7:0/648484464 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f04840b56f0 con 0x7f049c0cf890 2024-04-25T11:11:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.002+0000 7fa372000640 1 -- 172.21.15.7:0/3005936846 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa364018020 con 0x7fa37806d580 2024-04-25T11:11:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.001+0000 7f0499e00640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04780746b0 0x7f0478076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:01.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.003+0000 7fa372000640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3580746e0 0x7fa358076bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:01.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.004+0000 7f0499e00640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04780746b0 0x7f0478076bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f048c005880 tx=0x7f048c0019b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:01.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.004+0000 7fa372000640 1 -- 172.21.15.7:0/3005936846 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa364094f20 con 0x7fa37806d580 2024-04-25T11:11:01.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.005+0000 7fa37d400640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3580746e0 0x7fa358076bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:01.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.006+0000 7fa37d400640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3580746e0 0x7fa358076bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa36c034000 tx=0x7fa36c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:01.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.014+0000 7f0492000640 1 -- 172.21.15.7:0/648484464 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f048407f4e0 con 0x7f049c0cf890 2024-04-25T11:11:01.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.020+0000 7fa372000640 1 -- 172.21.15.7:0/3005936846 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa36401f030 con 0x7fa37806d580 2024-04-25T11:11:01.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.307+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd4f0005850 con 0x7fd50c054660 2024-04-25T11:11:01.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.308+0000 7fd507400640 1 -- 172.21.15.7:0/124672614 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd508067030 con 0x7fd50c054660 2024-04-25T11:11:01.310 INFO:teuthology.orchestra.run.smithi007.stdout:64424509452 2024-04-25T11:11:01.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.311+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4f4074500 msgr2=0x7fd4f4076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.311+0000 7fd51a800640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4f4074500 0x7fd4f4076a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd500034000 tx=0x7fd50002e000 comp rx=0 tx=0).stop 2024-04-25T11:11:01.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.311+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd50c054660 msgr2=0x7fd50c0d5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.311+0000 7fd51a800640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd50c054660 0x7fd50c0d5210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c390 tx=0x7fd50800c850 comp rx=0 tx=0).stop 2024-04-25T11:11:01.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.313+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 shutdown_connections 2024-04-25T11:11:01.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.313+0000 7fd51a800640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4f4074500 0x7fd4f4076a10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.313+0000 7fd51a800640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd50c0e3e10 0x7fd50c03a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.313+0000 7fd51a800640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd50c0ddfe0 0x7fd50c052870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.313+0000 7fd51a800640 1 --2- 172.21.15.7:0/124672614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd50c054660 0x7fd50c0d5210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.313+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 >> 172.21.15.7:0/124672614 conn(0x7fd50c0563f0 msgr2=0x7fd50c0dd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:01.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.313+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 shutdown_connections 2024-04-25T11:11:01.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.313+0000 7fd51a800640 1 -- 172.21.15.7:0/124672614 wait complete. 2024-04-25T11:11:01.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.324+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f42480697c0 con 0x7f424806d580 2024-04-25T11:11:01.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.325+0000 7f423e000640 1 -- 172.21.15.7:0/3918114671 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f423805e720 con 0x7f424806d580 2024-04-25T11:11:01.326 INFO:teuthology.orchestra.run.smithi007.stdout:64424509451 2024-04-25T11:11:01.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.329+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4228074580 msgr2=0x7f4228076a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.329+0000 7f424ce00640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4228074580 0x7f4228076a90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4234034000 tx=0x7f423402e000 comp rx=0 tx=0).stop 2024-04-25T11:11:01.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.330+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f424806d580 msgr2=0x7f424815b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.330+0000 7f424ce00640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f424806d580 0x7f424815b010 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f423800c3f0 tx=0x7f423800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.330+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 shutdown_connections 2024-04-25T11:11:01.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.330+0000 7f424ce00640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4228074580 0x7f4228076a90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.330+0000 7f424ce00640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42480d1140 0x7f42481427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.330+0000 7f424ce00640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42480cdd20 0x7f42481456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.330+0000 7f424ce00640 1 --2- 172.21.15.7:0/3918114671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f424806d580 0x7f424815b010 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.330+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 >> 172.21.15.7:0/3918114671 conn(0x7f42480732b0 msgr2=0x7f4248159360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:01.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.331+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 shutdown_connections 2024-04-25T11:11:01.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.331+0000 7f424ce00640 1 -- 172.21.15.7:0/3918114671 wait complete. 2024-04-25T11:11:01.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.341+0000 7fd053200640 1 -- 172.21.15.7:0/2692798259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd0540709d0 con 0x7fd05415cee0 2024-04-25T11:11:01.345 INFO:teuthology.orchestra.run.smithi007.stdout:64424509451 2024-04-25T11:11:01.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.342+0000 7fd04a000640 1 -- 172.21.15.7:0/2692798259 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd0400626b0 con 0x7fd05415cee0 2024-04-25T11:11:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.346+0000 7fd053200640 1 -- 172.21.15.7:0/2692798259 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0340746e0 msgr2=0x7fd034076bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.346+0000 7fd053200640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0340746e0 0x7fd034076bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd054160520 tx=0x7fd04c009040 comp rx=0 tx=0).stop 2024-04-25T11:11:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.346+0000 7fd053200640 1 -- 172.21.15.7:0/2692798259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05415cee0 msgr2=0x7fd05415e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.346+0000 7fd053200640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05415cee0 0x7fd05415e7c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd04000e7f0 tx=0x7fd04000ecb0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.346+0000 7fd053200640 1 -- 172.21.15.7:0/2692798259 shutdown_connections 2024-04-25T11:11:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.346+0000 7fd053200640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0340746e0 0x7fd034076bf0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd054160520 tx=0x7fd04c009040 comp rx=0 tx=0).stop 2024-04-25T11:11:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.346+0000 7fd053200640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd05415cee0 0x7fd05415e7c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.346+0000 7fd053200640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd054156460 0x7fd0541464f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.346+0000 7fd053200640 1 --2- 172.21.15.7:0/2692798259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0540ceaf0 0x7fd05414da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.346+0000 7fd053200640 1 -- 172.21.15.7:0/2692798259 >> 172.21.15.7:0/2692798259 conn(0x7fd0540740f0 msgr2=0x7fd05406f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.347+0000 7fd053200640 1 -- 172.21.15.7:0/2692798259 shutdown_connections 2024-04-25T11:11:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.347+0000 7fd053200640 1 -- 172.21.15.7:0/2692798259 wait complete. 2024-04-25T11:11:01.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.1 2024-04-25T11:11:01.352 DEBUG:teuthology.parallel:result is None 2024-04-25T11:11:01.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.358+0000 7f39bec00640 1 -- 172.21.15.7:0/2644451546 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f39b8076180 con 0x7f39b80b0fd0 2024-04-25T11:11:01.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.359+0000 7f39b5600640 1 -- 172.21.15.7:0/2644451546 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f39ac04c020 con 0x7f39b80b0fd0 2024-04-25T11:11:01.359 INFO:teuthology.orchestra.run.smithi007.stdout:68719476747 2024-04-25T11:11:01.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.363+0000 7f39bec00640 1 -- 172.21.15.7:0/2644451546 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39980744b0 msgr2=0x7f39980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.363+0000 7f39bec00640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39980744b0 0x7f39980769c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f39a4001eb0 tx=0x7f39a4001290 comp rx=0 tx=0).stop 2024-04-25T11:11:01.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.363+0000 7f39bec00640 1 -- 172.21.15.7:0/2644451546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b80b0fd0 msgr2=0x7f39b809e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.363+0000 7f39bec00640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b80b0fd0 0x7f39b809e9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0098c0 tx=0x7f39ac0046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.364+0000 7f39bec00640 1 -- 172.21.15.7:0/2644451546 shutdown_connections 2024-04-25T11:11:01.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.364+0000 7f39bec00640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39980744b0 0x7f39980769c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.364+0000 7f39bec00640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b80d1f40 0x7f39b814da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.364+0000 7f39bec00640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b80b0fd0 0x7f39b809e9a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.364+0000 7f39bec00640 1 --2- 172.21.15.7:0/2644451546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b806ddd0 0x7f39b815ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.364+0000 7f39bec00640 1 -- 172.21.15.7:0/2644451546 >> 172.21.15.7:0/2644451546 conn(0x7f39b8073b60 msgr2=0x7f39b80b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:01.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.364+0000 7f39bec00640 1 -- 172.21.15.7:0/2644451546 shutdown_connections 2024-04-25T11:11:01.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.364+0000 7f39bec00640 1 -- 172.21.15.7:0/2644451546 wait complete. 2024-04-25T11:11:01.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.0 2024-04-25T11:11:01.374 DEBUG:teuthology.parallel:result is None 2024-04-25T11:11:01.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.3 2024-04-25T11:11:01.376 DEBUG:teuthology.parallel:result is None 2024-04-25T11:11:01.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476747 for osd.5 2024-04-25T11:11:01.403 DEBUG:teuthology.parallel:result is None 2024-04-25T11:11:01.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.406+0000 7f049b200640 1 -- 172.21.15.7:0/648484464 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0474005580 con 0x7f049c0cf890 2024-04-25T11:11:01.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.407+0000 7f0492000640 1 -- 172.21.15.7:0/648484464 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f04840837e0 con 0x7f049c0cf890 2024-04-25T11:11:01.407 INFO:teuthology.orchestra.run.smithi007.stdout:68719476748 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.409+0000 7f049b200640 1 -- 172.21.15.7:0/648484464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04780746b0 msgr2=0x7f0478076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.409+0000 7f049b200640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04780746b0 0x7f0478076bc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f048c005880 tx=0x7f048c0019b0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.409+0000 7f049b200640 1 -- 172.21.15.7:0/648484464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c0cf890 msgr2=0x7f049c0caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.409+0000 7f049b200640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c0cf890 0x7f049c0caf70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f04840098c0 tx=0x7f04840046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.410+0000 7f049b200640 1 -- 172.21.15.7:0/648484464 shutdown_connections 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.410+0000 7f049b200640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04780746b0 0x7f0478076bc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.410+0000 7f049b200640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f049c0cc240 0x7f049c0cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.410+0000 7f049b200640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f049c0cf890 0x7f049c0caf70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.410+0000 7f049b200640 1 --2- 172.21.15.7:0/648484464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f049c06eb50 0x7f049c0cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.410+0000 7f049b200640 1 -- 172.21.15.7:0/648484464 >> 172.21.15.7:0/648484464 conn(0x7f049c09b540 msgr2=0x7f049c070260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.410+0000 7f049b200640 1 -- 172.21.15.7:0/648484464 shutdown_connections 2024-04-25T11:11:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.410+0000 7f049b200640 1 -- 172.21.15.7:0/648484464 wait complete. 2024-04-25T11:11:01.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.417+0000 7f84fd000640 1 -- 172.21.15.7:0/4125106887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f84f809c890 con 0x7f84f80b4b80 2024-04-25T11:11:01.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.417+0000 7f84ee000640 1 -- 172.21.15.7:0/4125106887 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f84e8063400 con 0x7f84f80b4b80 2024-04-25T11:11:01.417 INFO:teuthology.orchestra.run.smithi007.stdout:68719476747 2024-04-25T11:11:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.419+0000 7f84fd000640 1 -- 172.21.15.7:0/4125106887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84d40744e0 msgr2=0x7f84d40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.419+0000 7f84fd000640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84d40744e0 0x7f84d40769f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f84f80d1280 tx=0x7f84f0009040 comp rx=0 tx=0).stop 2024-04-25T11:11:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.419+0000 7f84fd000640 1 -- 172.21.15.7:0/4125106887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f80b4b80 msgr2=0x7f84f809d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.419+0000 7f84fd000640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f80b4b80 0x7f84f809d9e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84e800e840 tx=0x7f84e800ed00 comp rx=0 tx=0).stop 2024-04-25T11:11:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.419+0000 7f84fd000640 1 -- 172.21.15.7:0/4125106887 shutdown_connections 2024-04-25T11:11:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.419+0000 7f84fd000640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84d40744e0 0x7f84d40769f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.419+0000 7f84fd000640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84f80d2360 0x7f84f80c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.419+0000 7f84fd000640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84f80b4f90 0x7f84f8153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.419+0000 7f84fd000640 1 --2- 172.21.15.7:0/4125106887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84f80b4b80 0x7f84f809d9e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.419+0000 7f84fd000640 1 -- 172.21.15.7:0/4125106887 >> 172.21.15.7:0/4125106887 conn(0x7f84f80b54b0 msgr2=0x7f84f80b6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.420+0000 7f84fd000640 1 -- 172.21.15.7:0/4125106887 shutdown_connections 2024-04-25T11:11:01.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.420+0000 7f84fd000640 1 -- 172.21.15.7:0/4125106887 wait complete. 2024-04-25T11:11:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.434+0000 7fe3e1200640 1 -- 172.21.15.7:0/513872260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe3b4005850 con 0x7fe3d4049050 2024-04-25T11:11:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.434+0000 7fe3d2000640 1 -- 172.21.15.7:0/513872260 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe3dc0dbdd0 con 0x7fe3d4049050 2024-04-25T11:11:01.434 INFO:teuthology.orchestra.run.smithi007.stdout:60129542157 2024-04-25T11:11:01.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476748 got 68719476748 for osd.4 2024-04-25T11:11:01.436 DEBUG:teuthology.parallel:result is None 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.437+0000 7fe3e1200640 1 -- 172.21.15.7:0/513872260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3bc074500 msgr2=0x7fe3bc076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.437+0000 7fe3e1200640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3bc074500 0x7fe3bc076a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc02a1e0 tx=0x7fe3cc02e000 comp rx=0 tx=0).stop 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.437+0000 7fe3e1200640 1 -- 172.21.15.7:0/513872260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4049050 msgr2=0x7fe3d4049460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.437+0000 7fe3e1200640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4049050 0x7fe3d4049460 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc06aef0 tx=0x7fe3dc06ead0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.437+0000 7fe3e1200640 1 -- 172.21.15.7:0/513872260 shutdown_connections 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.437+0000 7fe3e1200640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3bc074500 0x7fe3bc076a10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.437+0000 7fe3e1200640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3d4049050 0x7fe3d4049460 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.437+0000 7fe3e1200640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3d4050670 0x7fe3d4048b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.437+0000 7fe3e1200640 1 --2- 172.21.15.7:0/513872260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3d404f370 0x7fe3d404ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.437+0000 7fe3e1200640 1 -- 172.21.15.7:0/513872260 >> 172.21.15.7:0/513872260 conn(0x7fe3d4054d20 msgr2=0x7fe3d40061e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.438+0000 7fe3e1200640 1 -- 172.21.15.7:0/513872260 shutdown_connections 2024-04-25T11:11:01.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.438+0000 7fe3e1200640 1 -- 172.21.15.7:0/513872260 wait complete. 2024-04-25T11:11:01.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476747 got 68719476747 for osd.7 2024-04-25T11:11:01.443 DEBUG:teuthology.parallel:result is None 2024-04-25T11:11:01.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.453+0000 7fa36b400640 1 -- 172.21.15.7:0/3005936846 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa378164cc0 con 0x7fa37806d580 2024-04-25T11:11:01.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.454+0000 7fa372000640 1 -- 172.21.15.7:0/3005936846 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa36405f3e0 con 0x7fa37806d580 2024-04-25T11:11:01.455 INFO:teuthology.orchestra.run.smithi007.stdout:68719476748 2024-04-25T11:11:01.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.456+0000 7fa36b400640 1 -- 172.21.15.7:0/3005936846 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3580746e0 msgr2=0x7fa358076bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.456+0000 7fa36b400640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3580746e0 0x7fa358076bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa36c034000 tx=0x7fa36c02e000 comp rx=0 tx=0).stop 2024-04-25T11:11:01.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.456+0000 7fa36b400640 1 -- 172.21.15.7:0/3005936846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37806d580 msgr2=0x7fa378202b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.456+0000 7fa36b400640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37806d580 0x7fa378202b10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa36400b3e0 tx=0x7fa36400b8a0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.457+0000 7fa36b400640 1 -- 172.21.15.7:0/3005936846 shutdown_connections 2024-04-25T11:11:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.457+0000 7fa36b400640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3580746e0 0x7fa358076bf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.457+0000 7fa36b400640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa37814db80 0x7fa3781600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.457+0000 7fa36b400640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa3780b4690 0x7fa378148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.457+0000 7fa36b400640 1 --2- 172.21.15.7:0/3005936846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa37806d580 0x7fa378202b10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.457+0000 7fa36b400640 1 -- 172.21.15.7:0/3005936846 >> 172.21.15.7:0/3005936846 conn(0x7fa378073160 msgr2=0x7fa3780b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:01.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.457+0000 7fa36b400640 1 -- 172.21.15.7:0/3005936846 shutdown_connections 2024-04-25T11:11:01.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.457+0000 7fa36b400640 1 -- 172.21.15.7:0/3005936846 wait complete. 2024-04-25T11:11:01.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542157 for osd.2 2024-04-25T11:11:01.462 DEBUG:teuthology.parallel:result is None 2024-04-25T11:11:01.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476748 got 68719476748 for osd.6 2024-04-25T11:11:01.482 DEBUG:teuthology.parallel:result is None 2024-04-25T11:11:01.482 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:11:01.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:11:01.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.781+0000 7f3da1600640 1 Processor -- start 2024-04-25T11:11:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.781+0000 7f3da1600640 1 -- start start 2024-04-25T11:11:01.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.784+0000 7f3da1600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d9c15ef90 0x7f3d9c09f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:01.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.784+0000 7f3da1600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d9c0af540 0x7f3d9c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:01.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.784+0000 7f3da1600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9c15f4e0 con 0x7f3d9c15bc40 2024-04-25T11:11:01.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.784+0000 7f3da1600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3d9c071190 con 0x7f3d9c0af540 2024-04-25T11:11:01.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.784+0000 7f3da1600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9c0b0560 con 0x7f3d9c15ef90 2024-04-25T11:11:01.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.784+0000 7f3d9b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d9c0af540 0x7f3d9c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:01.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.784+0000 7f3d9b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d9c0af540 0x7f3d9c0cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43398/0 (socket says 172.21.15.7:43398) 2024-04-25T11:11:01.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.784+0000 7f3d9b400640 1 -- 172.21.15.7:0/3850318259 learned_addr learned my addr 172.21.15.7:0/3850318259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:01.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.784+0000 7f3d9aa00640 1 --2- 172.21.15.7:0/3850318259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d9c15ef90 0x7f3d9c09f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:01.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.785+0000 7f3d9b400640 1 -- 172.21.15.7:0/3850318259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d9c15ef90 msgr2=0x7f3d9c09f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:01.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.785+0000 7f3d9b400640 1 --2- 172.21.15.7:0/3850318259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d9c15ef90 0x7f3d9c09f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.785+0000 7f3d9b400640 1 -- 172.21.15.7:0/3850318259 >> v1:172.21.15.87:6789/0 conn(0x7f3d9c15bc40 legacy=0x7f3d9c0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.785+0000 7f3d9b400640 1 -- 172.21.15.7:0/3850318259 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c0b0840 con 0x7f3d9c0af540 2024-04-25T11:11:01.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.785+0000 7f3d9aa00640 1 --2- 172.21.15.7:0/3850318259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d9c15ef90 0x7f3d9c09f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.786+0000 7f3d9b400640 1 --2- 172.21.15.7:0/3850318259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d9c0af540 0x7f3d9c0cf980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009e50 tx=0x7f3d9402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c471ceaa05aa1522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.786+0000 7f3d99600640 1 -- 172.21.15.7:0/3850318259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d94036070 con 0x7f3d9c0af540 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.786+0000 7f3d99600640 1 -- 172.21.15.7:0/3850318259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d94031600 con 0x7f3d9c0af540 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.786+0000 7f3d99600640 1 -- 172.21.15.7:0/3850318259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d940319d0 con 0x7f3d9c0af540 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.787+0000 7f3da1600640 1 -- 172.21.15.7:0/3850318259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d9c0af540 msgr2=0x7f3d9c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.787+0000 7f3da1600640 1 --2- 172.21.15.7:0/3850318259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d9c0af540 0x7f3d9c0cf980 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009e50 tx=0x7f3d9402a680 comp rx=0 tx=0).stop 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.787+0000 7f3da1600640 1 -- 172.21.15.7:0/3850318259 shutdown_connections 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.787+0000 7f3da1600640 1 --2- 172.21.15.7:0/3850318259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d9c0af540 0x7f3d9c0cf980 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.787+0000 7f3da1600640 1 --2- 172.21.15.7:0/3850318259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d9c15ef90 0x7f3d9c09f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.787+0000 7f3da1600640 1 -- 172.21.15.7:0/3850318259 >> 172.21.15.7:0/3850318259 conn(0x7f3d9c074200 msgr2=0x7f3d9c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.787+0000 7f3da1600640 1 -- 172.21.15.7:0/3850318259 shutdown_connections 2024-04-25T11:11:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.787+0000 7f3da1600640 1 -- 172.21.15.7:0/3850318259 wait complete. 2024-04-25T11:11:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.789+0000 7f3da1600640 1 Processor -- start 2024-04-25T11:11:01.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.790+0000 7f3da1600640 1 -- start start 2024-04-25T11:11:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.791+0000 7f3da1600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d9c15bc40 0x7f3d9c147270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.791+0000 7f3da1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d9c15ef90 0x7f3d9c0b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.791+0000 7f3da1600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d9c149a70 0x7f3d9c0b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.791+0000 7f3da1600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c0b1320 con 0x7f3d9c15ef90 2024-04-25T11:11:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.791+0000 7f3da1600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c0b0e00 con 0x7f3d9c15bc40 2024-04-25T11:11:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.791+0000 7f3da1600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9c0afb80 con 0x7f3d9c149a70 2024-04-25T11:11:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.791+0000 7f3d9a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d9c15ef90 0x7f3d9c0b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.791+0000 7f3d9a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d9c15ef90 0x7f3d9c0b2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43412/0 (socket says 172.21.15.7:43412) 2024-04-25T11:11:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.791+0000 7f3d9a000640 1 -- 172.21.15.7:0/3020996989 learned_addr learned my addr 172.21.15.7:0/3020996989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3d9aa00640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d9c15bc40 0x7f3d9c147270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3d9b400640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d9c149a70 0x7f3d9c0b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3d9a000640 1 -- 172.21.15.7:0/3020996989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d9c149a70 msgr2=0x7f3d9c0b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3d9a000640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d9c149a70 0x7f3d9c0b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3d9a000640 1 -- 172.21.15.7:0/3020996989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d9c15bc40 msgr2=0x7f3d9c147270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3d9a000640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d9c15bc40 0x7f3d9c147270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3d9a000640 1 -- 172.21.15.7:0/3020996989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c14a190 con 0x7f3d9c15ef90 2024-04-25T11:11:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3d9a000640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d9c15ef90 0x7f3d9c0b2a40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800c3c0 tx=0x7f3d8800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3d92a00640 1 -- 172.21.15.7:0/3020996989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d880052d0 con 0x7f3d9c15ef90 2024-04-25T11:11:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3d92a00640 1 -- 172.21.15.7:0/3020996989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d88005490 con 0x7f3d9c15ef90 2024-04-25T11:11:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3da1600640 1 -- 172.21.15.7:0/3020996989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9c14a470 con 0x7f3d9c15ef90 2024-04-25T11:11:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.792+0000 7f3d92a00640 1 -- 172.21.15.7:0/3020996989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88007260 con 0x7f3d9c15ef90 2024-04-25T11:11:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.793+0000 7f3da1600640 1 -- 172.21.15.7:0/3020996989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9c147b50 con 0x7f3d9c15ef90 2024-04-25T11:11:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.794+0000 7f3da1600640 1 -- 172.21.15.7:0/3020996989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9c0cdfb0 con 0x7f3d9c15ef90 2024-04-25T11:11:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.796+0000 7f3d92a00640 1 -- 172.21.15.7:0/3020996989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3d88007400 con 0x7f3d9c15ef90 2024-04-25T11:11:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.797+0000 7f3d92a00640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d7c074500 0x7f3d7c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.797+0000 7f3d92a00640 1 -- 172.21.15.7:0/3020996989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3d88094ed0 con 0x7f3d9c15ef90 2024-04-25T11:11:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.798+0000 7f3d9aa00640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d7c074500 0x7f3d7c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.798+0000 7f3d9aa00640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d7c074500 0x7f3d7c076a10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c006fd0 tx=0x7f3d8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:01.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.800+0000 7f3d92a00640 1 -- 172.21.15.7:0/3020996989 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d9c0cdfb0 con 0x7f3d9c15ef90 2024-04-25T11:11:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:01.992+0000 7f3da1600640 1 -- 172.21.15.7:0/3020996989 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d9c06a100 con 0x7f3d7c074500 2024-04-25T11:11:02.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.004+0000 7f3d92a00640 1 -- 172.21.15.7:0/3020996989 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332524 (secure 0 0 0) 0x7f3d9c06a100 con 0x7f3d7c074500 2024-04-25T11:11:02.012 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:11:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.012+0000 7f3da1600640 1 -- 172.21.15.7:0/3020996989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d7c074500 msgr2=0x7f3d7c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.012+0000 7f3da1600640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d7c074500 0x7f3d7c076a10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c006fd0 tx=0x7f3d8c008040 comp rx=0 tx=0).stop 2024-04-25T11:11:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.012+0000 7f3da1600640 1 -- 172.21.15.7:0/3020996989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d9c15ef90 msgr2=0x7f3d9c0b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.012+0000 7f3da1600640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d9c15ef90 0x7f3d9c0b2a40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800c3c0 tx=0x7f3d8800c880 comp rx=0 tx=0).stop 2024-04-25T11:11:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.013+0000 7f3da1600640 1 -- 172.21.15.7:0/3020996989 shutdown_connections 2024-04-25T11:11:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.013+0000 7f3da1600640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d7c074500 0x7f3d7c076a10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.013+0000 7f3da1600640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d9c149a70 0x7f3d9c0b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.013+0000 7f3da1600640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d9c15ef90 0x7f3d9c0b2a40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.013+0000 7f3da1600640 1 --2- 172.21.15.7:0/3020996989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d9c15bc40 0x7f3d9c147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.013+0000 7f3da1600640 1 -- 172.21.15.7:0/3020996989 >> 172.21.15.7:0/3020996989 conn(0x7f3d9c074200 msgr2=0x7f3d9c0aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.013+0000 7f3da1600640 1 -- 172.21.15.7:0/3020996989 shutdown_connections 2024-04-25T11:11:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.013+0000 7f3da1600640 1 -- 172.21.15.7:0/3020996989 wait complete. 2024-04-25T11:11:02.043 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":46,"stamp":"2024-04-25T11:11:01.795109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231960,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742760,"statfs":{"total":773094113280,"available":772856586240,"internally_reserved":0,"allocated":3383296,"data_stored":2025904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005714"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895493+0000","last_change":"2024-04-25T11:10:22.475060+0000","last_active":"2024-04-25T11:10:29.895493+0000","last_peered":"2024-04-25T11:10:29.895493+0000","last_clean":"2024-04-25T11:10:29.895493+0000","last_became_active":"2024-04-25T11:10:22.472574+0000","last_became_peered":"2024-04-25T11:10:22.472574+0000","last_unstale":"2024-04-25T11:10:29.895493+0000","last_undegraded":"2024-04-25T11:10:29.895493+0000","last_fullsized":"2024-04-25T11:10:29.895493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.835819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895751+0000","last_change":"2024-04-25T11:10:23.488367+0000","last_active":"2024-04-25T11:10:29.895751+0000","last_peered":"2024-04-25T11:10:29.895751+0000","last_clean":"2024-04-25T11:10:29.895751+0000","last_became_active":"2024-04-25T11:10:23.488034+0000","last_became_peered":"2024-04-25T11:10:23.488034+0000","last_unstale":"2024-04-25T11:10:29.895751+0000","last_undegraded":"2024-04-25T11:10:29.895751+0000","last_fullsized":"2024-04-25T11:10:29.895751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892242+0000","last_change":"2024-04-25T11:10:27.551067+0000","last_active":"2024-04-25T11:10:29.892242+0000","last_peered":"2024-04-25T11:10:29.892242+0000","last_clean":"2024-04-25T11:10:29.892242+0000","last_became_active":"2024-04-25T11:10:27.550247+0000","last_became_peered":"2024-04-25T11:10:27.550247+0000","last_unstale":"2024-04-25T11:10:29.892242+0000","last_undegraded":"2024-04-25T11:10:29.892242+0000","last_fullsized":"2024-04-25T11:10:29.892242+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:29.432696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895898+0000","last_change":"2024-04-25T11:10:22.463507+0000","last_active":"2024-04-25T11:10:29.895898+0000","last_peered":"2024-04-25T11:10:29.895898+0000","last_clean":"2024-04-25T11:10:29.895898+0000","last_became_active":"2024-04-25T11:10:22.463142+0000","last_became_peered":"2024-04-25T11:10:22.463142+0000","last_unstale":"2024-04-25T11:10:29.895898+0000","last_undegraded":"2024-04-25T11:10:29.895898+0000","last_fullsized":"2024-04-25T11:10:29.895898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:42.000742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894569+0000","last_change":"2024-04-25T11:10:23.500843+0000","last_active":"2024-04-25T11:10:29.894569+0000","last_peered":"2024-04-25T11:10:29.894569+0000","last_clean":"2024-04-25T11:10:29.894569+0000","last_became_active":"2024-04-25T11:10:23.500412+0000","last_became_peered":"2024-04-25T11:10:23.500412+0000","last_unstale":"2024-04-25T11:10:29.894569+0000","last_undegraded":"2024-04-25T11:10:29.894569+0000","last_fullsized":"2024-04-25T11:10:29.894569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534853+0000","last_change":"2024-04-25T11:10:27.565431+0000","last_active":"2024-04-25T11:10:28.534853+0000","last_peered":"2024-04-25T11:10:28.534853+0000","last_clean":"2024-04-25T11:10:28.534853+0000","last_became_active":"2024-04-25T11:10:27.564724+0000","last_became_peered":"2024-04-25T11:10:27.564724+0000","last_unstale":"2024-04-25T11:10:28.534853+0000","last_undegraded":"2024-04-25T11:10:28.534853+0000","last_fullsized":"2024-04-25T11:10:28.534853+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:51.033161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536873+0000","last_change":"2024-04-25T11:10:22.477443+0000","last_active":"2024-04-25T11:10:28.536873+0000","last_peered":"2024-04-25T11:10:28.536873+0000","last_clean":"2024-04-25T11:10:28.536873+0000","last_became_active":"2024-04-25T11:10:22.477131+0000","last_became_peered":"2024-04-25T11:10:22.477131+0000","last_unstale":"2024-04-25T11:10:28.536873+0000","last_undegraded":"2024-04-25T11:10:28.536873+0000","last_fullsized":"2024-04-25T11:10:28.536873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:57.426313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291476+0000","last_change":"2024-04-25T11:10:23.496542+0000","last_active":"2024-04-25T11:10:29.291476+0000","last_peered":"2024-04-25T11:10:29.291476+0000","last_clean":"2024-04-25T11:10:29.291476+0000","last_became_active":"2024-04-25T11:10:23.496211+0000","last_became_peered":"2024-04-25T11:10:23.496211+0000","last_unstale":"2024-04-25T11:10:29.291476+0000","last_undegraded":"2024-04-25T11:10:29.291476+0000","last_fullsized":"2024-04-25T11:10:29.291476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:15.460249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537266+0000","last_change":"2024-04-25T11:10:27.565126+0000","last_active":"2024-04-25T11:10:28.537266+0000","last_peered":"2024-04-25T11:10:28.537266+0000","last_clean":"2024-04-25T11:10:28.537266+0000","last_became_active":"2024-04-25T11:10:27.564664+0000","last_became_peered":"2024-04-25T11:10:27.564664+0000","last_unstale":"2024-04-25T11:10:28.537266+0000","last_undegraded":"2024-04-25T11:10:28.537266+0000","last_fullsized":"2024-04-25T11:10:28.537266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:44.604923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536194+0000","last_change":"2024-04-25T11:10:22.483531+0000","last_active":"2024-04-25T11:10:28.536194+0000","last_peered":"2024-04-25T11:10:28.536194+0000","last_clean":"2024-04-25T11:10:28.536194+0000","last_became_active":"2024-04-25T11:10:22.483204+0000","last_became_peered":"2024-04-25T11:10:22.483204+0000","last_unstale":"2024-04-25T11:10:28.536194+0000","last_undegraded":"2024-04-25T11:10:28.536194+0000","last_fullsized":"2024-04-25T11:10:28.536194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:03.866498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291328+0000","last_change":"2024-04-25T11:10:23.490550+0000","last_active":"2024-04-25T11:10:29.291328+0000","last_peered":"2024-04-25T11:10:29.291328+0000","last_clean":"2024-04-25T11:10:29.291328+0000","last_became_active":"2024-04-25T11:10:23.490300+0000","last_became_peered":"2024-04-25T11:10:23.490300+0000","last_unstale":"2024-04-25T11:10:29.291328+0000","last_undegraded":"2024-04-25T11:10:29.291328+0000","last_fullsized":"2024-04-25T11:10:29.291328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:20.100576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535567+0000","last_change":"2024-04-25T11:10:27.555808+0000","last_active":"2024-04-25T11:10:28.535567+0000","last_peered":"2024-04-25T11:10:28.535567+0000","last_clean":"2024-04-25T11:10:28.535567+0000","last_became_active":"2024-04-25T11:10:27.555277+0000","last_became_peered":"2024-04-25T11:10:27.555277+0000","last_unstale":"2024-04-25T11:10:28.535567+0000","last_undegraded":"2024-04-25T11:10:28.535567+0000","last_fullsized":"2024-04-25T11:10:28.535567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:52.665016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.890747+0000","last_change":"2024-04-25T11:10:23.493263+0000","last_active":"2024-04-25T11:10:29.890747+0000","last_peered":"2024-04-25T11:10:29.890747+0000","last_clean":"2024-04-25T11:10:29.890747+0000","last_became_active":"2024-04-25T11:10:23.489625+0000","last_became_peered":"2024-04-25T11:10:23.489625+0000","last_unstale":"2024-04-25T11:10:29.890747+0000","last_undegraded":"2024-04-25T11:10:29.890747+0000","last_fullsized":"2024-04-25T11:10:29.890747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535220+0000","last_change":"2024-04-25T11:10:22.475840+0000","last_active":"2024-04-25T11:10:28.535220+0000","last_peered":"2024-04-25T11:10:28.535220+0000","last_clean":"2024-04-25T11:10:28.535220+0000","last_became_active":"2024-04-25T11:10:22.475539+0000","last_became_peered":"2024-04-25T11:10:22.475539+0000","last_unstale":"2024-04-25T11:10:28.535220+0000","last_undegraded":"2024-04-25T11:10:28.535220+0000","last_fullsized":"2024-04-25T11:10:28.535220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:49.860961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538827+0000","last_change":"2024-04-25T11:10:27.558933+0000","last_active":"2024-04-25T11:10:28.538827+0000","last_peered":"2024-04-25T11:10:28.538827+0000","last_clean":"2024-04-25T11:10:28.538827+0000","last_became_active":"2024-04-25T11:10:27.558566+0000","last_became_peered":"2024-04-25T11:10:27.558566+0000","last_unstale":"2024-04-25T11:10:28.538827+0000","last_undegraded":"2024-04-25T11:10:28.538827+0000","last_fullsized":"2024-04-25T11:10:28.538827+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:45.815273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892352+0000","last_change":"2024-04-25T11:10:23.482991+0000","last_active":"2024-04-25T11:10:29.892352+0000","last_peered":"2024-04-25T11:10:29.892352+0000","last_clean":"2024-04-25T11:10:29.892352+0000","last_became_active":"2024-04-25T11:10:23.481972+0000","last_became_peered":"2024-04-25T11:10:23.481972+0000","last_unstale":"2024-04-25T11:10:29.892352+0000","last_undegraded":"2024-04-25T11:10:29.892352+0000","last_fullsized":"2024-04-25T11:10:29.892352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533468+0000","last_change":"2024-04-25T11:10:22.469445+0000","last_active":"2024-04-25T11:10:28.533468+0000","last_peered":"2024-04-25T11:10:28.533468+0000","last_clean":"2024-04-25T11:10:28.533468+0000","last_became_active":"2024-04-25T11:10:22.469095+0000","last_became_peered":"2024-04-25T11:10:22.469095+0000","last_unstale":"2024-04-25T11:10:28.533468+0000","last_undegraded":"2024-04-25T11:10:28.533468+0000","last_fullsized":"2024-04-25T11:10:28.533468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537528+0000","last_change":"2024-04-25T11:10:27.559144+0000","last_active":"2024-04-25T11:10:28.537528+0000","last_peered":"2024-04-25T11:10:28.537528+0000","last_clean":"2024-04-25T11:10:28.537528+0000","last_became_active":"2024-04-25T11:10:27.558635+0000","last_became_peered":"2024-04-25T11:10:27.558635+0000","last_unstale":"2024-04-25T11:10:28.537528+0000","last_undegraded":"2024-04-25T11:10:28.537528+0000","last_fullsized":"2024-04-25T11:10:28.537528+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:24.222645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892015+0000","last_change":"2024-04-25T11:10:23.460813+0000","last_active":"2024-04-25T11:10:29.892015+0000","last_peered":"2024-04-25T11:10:29.892015+0000","last_clean":"2024-04-25T11:10:29.892015+0000","last_became_active":"2024-04-25T11:10:23.460435+0000","last_became_peered":"2024-04-25T11:10:23.460435+0000","last_unstale":"2024-04-25T11:10:29.892015+0000","last_undegraded":"2024-04-25T11:10:29.892015+0000","last_fullsized":"2024-04-25T11:10:29.892015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531902+0000","last_change":"2024-04-25T11:10:22.466351+0000","last_active":"2024-04-25T11:10:28.531902+0000","last_peered":"2024-04-25T11:10:28.531902+0000","last_clean":"2024-04-25T11:10:28.531902+0000","last_became_active":"2024-04-25T11:10:22.464110+0000","last_became_peered":"2024-04-25T11:10:22.464110+0000","last_unstale":"2024-04-25T11:10:28.531902+0000","last_undegraded":"2024-04-25T11:10:28.531902+0000","last_fullsized":"2024-04-25T11:10:28.531902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534334+0000","last_change":"2024-04-25T11:10:27.555355+0000","last_active":"2024-04-25T11:10:28.534334+0000","last_peered":"2024-04-25T11:10:28.534334+0000","last_clean":"2024-04-25T11:10:28.534334+0000","last_became_active":"2024-04-25T11:10:27.554893+0000","last_became_peered":"2024-04-25T11:10:27.554893+0000","last_unstale":"2024-04-25T11:10:28.534334+0000","last_undegraded":"2024-04-25T11:10:28.534334+0000","last_fullsized":"2024-04-25T11:10:28.534334+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:51.174327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535363+0000","last_change":"2024-04-25T11:10:23.487680+0000","last_active":"2024-04-25T11:10:28.535363+0000","last_peered":"2024-04-25T11:10:28.535363+0000","last_clean":"2024-04-25T11:10:28.535363+0000","last_became_active":"2024-04-25T11:10:23.483024+0000","last_became_peered":"2024-04-25T11:10:23.483024+0000","last_unstale":"2024-04-25T11:10:28.535363+0000","last_undegraded":"2024-04-25T11:10:28.535363+0000","last_fullsized":"2024-04-25T11:10:28.535363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:06.246101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534055+0000","last_change":"2024-04-25T11:10:22.470867+0000","last_active":"2024-04-25T11:10:28.534055+0000","last_peered":"2024-04-25T11:10:28.534055+0000","last_clean":"2024-04-25T11:10:28.534055+0000","last_became_active":"2024-04-25T11:10:22.470501+0000","last_became_peered":"2024-04-25T11:10:22.470501+0000","last_unstale":"2024-04-25T11:10:28.534055+0000","last_undegraded":"2024-04-25T11:10:28.534055+0000","last_fullsized":"2024-04-25T11:10:28.534055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538997+0000","last_change":"2024-04-25T11:10:27.567703+0000","last_active":"2024-04-25T11:10:28.538997+0000","last_peered":"2024-04-25T11:10:28.538997+0000","last_clean":"2024-04-25T11:10:28.538997+0000","last_became_active":"2024-04-25T11:10:27.567208+0000","last_became_peered":"2024-04-25T11:10:27.567208+0000","last_unstale":"2024-04-25T11:10:28.538997+0000","last_undegraded":"2024-04-25T11:10:28.538997+0000","last_fullsized":"2024-04-25T11:10:28.538997+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:24.170044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895262+0000","last_change":"2024-04-25T11:10:23.488689+0000","last_active":"2024-04-25T11:10:29.895262+0000","last_peered":"2024-04-25T11:10:29.895262+0000","last_clean":"2024-04-25T11:10:29.895262+0000","last_became_active":"2024-04-25T11:10:23.479256+0000","last_became_peered":"2024-04-25T11:10:23.479256+0000","last_unstale":"2024-04-25T11:10:29.895262+0000","last_undegraded":"2024-04-25T11:10:29.895262+0000","last_fullsized":"2024-04-25T11:10:29.895262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892311+0000","last_change":"2024-04-25T11:10:22.486332+0000","last_active":"2024-04-25T11:10:29.892311+0000","last_peered":"2024-04-25T11:10:29.892311+0000","last_clean":"2024-04-25T11:10:29.892311+0000","last_became_active":"2024-04-25T11:10:22.485970+0000","last_became_peered":"2024-04-25T11:10:22.485970+0000","last_unstale":"2024-04-25T11:10:29.892311+0000","last_undegraded":"2024-04-25T11:10:29.892311+0000","last_fullsized":"2024-04-25T11:10:29.892311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534779+0000","last_change":"2024-04-25T11:10:27.549543+0000","last_active":"2024-04-25T11:10:28.534779+0000","last_peered":"2024-04-25T11:10:28.534779+0000","last_clean":"2024-04-25T11:10:28.534779+0000","last_became_active":"2024-04-25T11:10:27.549007+0000","last_became_peered":"2024-04-25T11:10:27.549007+0000","last_unstale":"2024-04-25T11:10:28.534779+0000","last_undegraded":"2024-04-25T11:10:28.534779+0000","last_fullsized":"2024-04-25T11:10:28.534779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:01.381820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896311+0000","last_change":"2024-04-25T11:10:23.465612+0000","last_active":"2024-04-25T11:10:29.896311+0000","last_peered":"2024-04-25T11:10:29.896311+0000","last_clean":"2024-04-25T11:10:29.896311+0000","last_became_active":"2024-04-25T11:10:23.465261+0000","last_became_peered":"2024-04-25T11:10:23.465261+0000","last_unstale":"2024-04-25T11:10:29.896311+0000","last_undegraded":"2024-04-25T11:10:29.896311+0000","last_fullsized":"2024-04-25T11:10:29.896311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534173+0000","last_change":"2024-04-25T11:10:22.483679+0000","last_active":"2024-04-25T11:10:28.534173+0000","last_peered":"2024-04-25T11:10:28.534173+0000","last_clean":"2024-04-25T11:10:28.534173+0000","last_became_active":"2024-04-25T11:10:22.483243+0000","last_became_peered":"2024-04-25T11:10:22.483243+0000","last_unstale":"2024-04-25T11:10:28.534173+0000","last_undegraded":"2024-04-25T11:10:28.534173+0000","last_fullsized":"2024-04-25T11:10:28.534173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:49.860961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891407+0000","last_change":"2024-04-25T11:10:27.550030+0000","last_active":"2024-04-25T11:10:29.891407+0000","last_peered":"2024-04-25T11:10:29.891407+0000","last_clean":"2024-04-25T11:10:29.891407+0000","last_became_active":"2024-04-25T11:10:27.549537+0000","last_became_peered":"2024-04-25T11:10:27.549537+0000","last_unstale":"2024-04-25T11:10:29.891407+0000","last_undegraded":"2024-04-25T11:10:29.891407+0000","last_fullsized":"2024-04-25T11:10:29.891407+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:51.538673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534183+0000","last_change":"2024-04-25T11:10:23.470926+0000","last_active":"2024-04-25T11:10:28.534183+0000","last_peered":"2024-04-25T11:10:28.534183+0000","last_clean":"2024-04-25T11:10:28.534183+0000","last_became_active":"2024-04-25T11:10:23.470483+0000","last_became_peered":"2024-04-25T11:10:23.470483+0000","last_unstale":"2024-04-25T11:10:28.534183+0000","last_undegraded":"2024-04-25T11:10:28.534183+0000","last_fullsized":"2024-04-25T11:10:28.534183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537196+0000","last_change":"2024-04-25T11:10:22.479703+0000","last_active":"2024-04-25T11:10:28.537196+0000","last_peered":"2024-04-25T11:10:28.537196+0000","last_clean":"2024-04-25T11:10:28.537196+0000","last_became_active":"2024-04-25T11:10:22.478929+0000","last_became_peered":"2024-04-25T11:10:22.478929+0000","last_unstale":"2024-04-25T11:10:28.537196+0000","last_undegraded":"2024-04-25T11:10:28.537196+0000","last_fullsized":"2024-04-25T11:10:28.537196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:08.833540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893409+0000","last_change":"2024-04-25T11:10:27.559919+0000","last_active":"2024-04-25T11:10:29.893409+0000","last_peered":"2024-04-25T11:10:29.893409+0000","last_clean":"2024-04-25T11:10:29.893409+0000","last_became_active":"2024-04-25T11:10:27.558805+0000","last_became_peered":"2024-04-25T11:10:27.558805+0000","last_unstale":"2024-04-25T11:10:29.893409+0000","last_undegraded":"2024-04-25T11:10:29.893409+0000","last_fullsized":"2024-04-25T11:10:29.893409+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291634+0000","last_change":"2024-04-25T11:10:23.471738+0000","last_active":"2024-04-25T11:10:29.291634+0000","last_peered":"2024-04-25T11:10:29.291634+0000","last_clean":"2024-04-25T11:10:29.291634+0000","last_became_active":"2024-04-25T11:10:23.471191+0000","last_became_peered":"2024-04-25T11:10:23.471191+0000","last_unstale":"2024-04-25T11:10:29.291634+0000","last_undegraded":"2024-04-25T11:10:29.291634+0000","last_fullsized":"2024-04-25T11:10:29.291634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:47.901333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534371+0000","last_change":"2024-04-25T11:10:22.467322+0000","last_active":"2024-04-25T11:10:28.534371+0000","last_peered":"2024-04-25T11:10:28.534371+0000","last_clean":"2024-04-25T11:10:28.534371+0000","last_became_active":"2024-04-25T11:10:22.466806+0000","last_became_peered":"2024-04-25T11:10:22.466806+0000","last_unstale":"2024-04-25T11:10:28.534371+0000","last_undegraded":"2024-04-25T11:10:28.534371+0000","last_fullsized":"2024-04-25T11:10:28.534371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533236+0000","last_change":"2024-04-25T11:10:27.546558+0000","last_active":"2024-04-25T11:10:28.533236+0000","last_peered":"2024-04-25T11:10:28.533236+0000","last_clean":"2024-04-25T11:10:28.533236+0000","last_became_active":"2024-04-25T11:10:27.544215+0000","last_became_peered":"2024-04-25T11:10:27.544215+0000","last_unstale":"2024-04-25T11:10:28.533236+0000","last_undegraded":"2024-04-25T11:10:28.533236+0000","last_fullsized":"2024-04-25T11:10:28.533236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.970801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537205+0000","last_change":"2024-04-25T11:10:23.489986+0000","last_active":"2024-04-25T11:10:28.537205+0000","last_peered":"2024-04-25T11:10:28.537205+0000","last_clean":"2024-04-25T11:10:28.537205+0000","last_became_active":"2024-04-25T11:10:23.489629+0000","last_became_peered":"2024-04-25T11:10:23.489629+0000","last_unstale":"2024-04-25T11:10:28.537205+0000","last_undegraded":"2024-04-25T11:10:28.537205+0000","last_fullsized":"2024-04-25T11:10:28.537205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:14.555604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291125+0000","last_change":"2024-04-25T11:10:22.482966+0000","last_active":"2024-04-25T11:10:29.291125+0000","last_peered":"2024-04-25T11:10:29.291125+0000","last_clean":"2024-04-25T11:10:29.291125+0000","last_became_active":"2024-04-25T11:10:22.480437+0000","last_became_peered":"2024-04-25T11:10:22.480437+0000","last_unstale":"2024-04-25T11:10:29.291125+0000","last_undegraded":"2024-04-25T11:10:29.291125+0000","last_fullsized":"2024-04-25T11:10:29.291125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534554+0000","last_change":"2024-04-25T11:10:27.549617+0000","last_active":"2024-04-25T11:10:28.534554+0000","last_peered":"2024-04-25T11:10:28.534554+0000","last_clean":"2024-04-25T11:10:28.534554+0000","last_became_active":"2024-04-25T11:10:27.549203+0000","last_became_peered":"2024-04-25T11:10:27.549203+0000","last_unstale":"2024-04-25T11:10:28.534554+0000","last_undegraded":"2024-04-25T11:10:28.534554+0000","last_fullsized":"2024-04-25T11:10:28.534554+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:55.043679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531961+0000","last_change":"2024-04-25T11:10:23.472089+0000","last_active":"2024-04-25T11:10:28.531961+0000","last_peered":"2024-04-25T11:10:28.531961+0000","last_clean":"2024-04-25T11:10:28.531961+0000","last_became_active":"2024-04-25T11:10:23.468460+0000","last_became_peered":"2024-04-25T11:10:23.468460+0000","last_unstale":"2024-04-25T11:10:28.531961+0000","last_undegraded":"2024-04-25T11:10:28.531961+0000","last_fullsized":"2024-04-25T11:10:28.531961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373356+0000","last_change":"2024-04-25T11:10:22.468201+0000","last_active":"2024-04-25T11:10:29.373356+0000","last_peered":"2024-04-25T11:10:29.373356+0000","last_clean":"2024-04-25T11:10:29.373356+0000","last_became_active":"2024-04-25T11:10:22.467845+0000","last_became_peered":"2024-04-25T11:10:22.467845+0000","last_unstale":"2024-04-25T11:10:29.373356+0000","last_undegraded":"2024-04-25T11:10:29.373356+0000","last_fullsized":"2024-04-25T11:10:29.373356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:50.983943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534837+0000","last_change":"2024-04-25T11:10:27.552725+0000","last_active":"2024-04-25T11:10:28.534837+0000","last_peered":"2024-04-25T11:10:28.534837+0000","last_clean":"2024-04-25T11:10:28.534837+0000","last_became_active":"2024-04-25T11:10:27.552073+0000","last_became_peered":"2024-04-25T11:10:27.552073+0000","last_unstale":"2024-04-25T11:10:28.534837+0000","last_undegraded":"2024-04-25T11:10:28.534837+0000","last_fullsized":"2024-04-25T11:10:28.534837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.402499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893257+0000","last_change":"2024-04-25T11:10:23.492738+0000","last_active":"2024-04-25T11:10:29.893257+0000","last_peered":"2024-04-25T11:10:29.893257+0000","last_clean":"2024-04-25T11:10:29.893257+0000","last_became_active":"2024-04-25T11:10:23.492416+0000","last_became_peered":"2024-04-25T11:10:23.492416+0000","last_unstale":"2024-04-25T11:10:29.893257+0000","last_undegraded":"2024-04-25T11:10:29.893257+0000","last_fullsized":"2024-04-25T11:10:29.893257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:00.331944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536183+0000","last_change":"2024-04-25T11:10:22.478976+0000","last_active":"2024-04-25T11:10:28.536183+0000","last_peered":"2024-04-25T11:10:28.536183+0000","last_clean":"2024-04-25T11:10:28.536183+0000","last_became_active":"2024-04-25T11:10:22.478595+0000","last_became_peered":"2024-04-25T11:10:22.478595+0000","last_unstale":"2024-04-25T11:10:28.536183+0000","last_undegraded":"2024-04-25T11:10:28.536183+0000","last_fullsized":"2024-04-25T11:10:28.536183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895071+0000","last_change":"2024-04-25T11:10:27.558873+0000","last_active":"2024-04-25T11:10:29.895071+0000","last_peered":"2024-04-25T11:10:29.895071+0000","last_clean":"2024-04-25T11:10:29.895071+0000","last_became_active":"2024-04-25T11:10:27.557790+0000","last_became_peered":"2024-04-25T11:10:27.557790+0000","last_unstale":"2024-04-25T11:10:29.895071+0000","last_undegraded":"2024-04-25T11:10:29.895071+0000","last_fullsized":"2024-04-25T11:10:29.895071+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.402499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893187+0000","last_change":"2024-04-25T11:10:23.470011+0000","last_active":"2024-04-25T11:10:29.893187+0000","last_peered":"2024-04-25T11:10:29.893187+0000","last_clean":"2024-04-25T11:10:29.893187+0000","last_became_active":"2024-04-25T11:10:23.469655+0000","last_became_peered":"2024-04-25T11:10:23.469655+0000","last_unstale":"2024-04-25T11:10:29.893187+0000","last_undegraded":"2024-04-25T11:10:29.893187+0000","last_fullsized":"2024-04-25T11:10:29.893187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291689+0000","last_change":"2024-04-25T11:10:22.480507+0000","last_active":"2024-04-25T11:10:29.291689+0000","last_peered":"2024-04-25T11:10:29.291689+0000","last_clean":"2024-04-25T11:10:29.291689+0000","last_became_active":"2024-04-25T11:10:22.479568+0000","last_became_peered":"2024-04-25T11:10:22.479568+0000","last_unstale":"2024-04-25T11:10:29.291689+0000","last_undegraded":"2024-04-25T11:10:29.291689+0000","last_fullsized":"2024-04-25T11:10:29.291689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":118,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536490+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:10:28.536490+0000","last_peered":"2024-04-25T11:10:28.536490+0000","last_clean":"2024-04-25T11:10:28.536490+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:10:28.536490+0000","last_undegraded":"2024-04-25T11:10:28.536490+0000","last_fullsized":"2024-04-25T11:10:28.536490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891956+0000","last_change":"2024-04-25T11:10:27.551611+0000","last_active":"2024-04-25T11:10:29.891956+0000","last_peered":"2024-04-25T11:10:29.891956+0000","last_clean":"2024-04-25T11:10:29.891956+0000","last_became_active":"2024-04-25T11:10:27.551117+0000","last_became_peered":"2024-04-25T11:10:27.551117+0000","last_unstale":"2024-04-25T11:10:29.891956+0000","last_undegraded":"2024-04-25T11:10:29.891956+0000","last_fullsized":"2024-04-25T11:10:29.891956+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536555+0000","last_change":"2024-04-25T11:10:23.469975+0000","last_active":"2024-04-25T11:10:28.536555+0000","last_peered":"2024-04-25T11:10:28.536555+0000","last_clean":"2024-04-25T11:10:28.536555+0000","last_became_active":"2024-04-25T11:10:23.469366+0000","last_became_peered":"2024-04-25T11:10:23.469366+0000","last_unstale":"2024-04-25T11:10:28.536555+0000","last_undegraded":"2024-04-25T11:10:28.536555+0000","last_fullsized":"2024-04-25T11:10:28.536555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:47.901333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373786+0000","last_change":"2024-04-25T11:10:22.480954+0000","last_active":"2024-04-25T11:10:29.373786+0000","last_peered":"2024-04-25T11:10:29.373786+0000","last_clean":"2024-04-25T11:10:29.373786+0000","last_became_active":"2024-04-25T11:10:22.480489+0000","last_became_peered":"2024-04-25T11:10:22.480489+0000","last_unstale":"2024-04-25T11:10:29.373786+0000","last_undegraded":"2024-04-25T11:10:29.373786+0000","last_fullsized":"2024-04-25T11:10:29.373786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:11.295052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891284+0000","last_change":"2024-04-25T11:10:27.548113+0000","last_active":"2024-04-25T11:10:29.891284+0000","last_peered":"2024-04-25T11:10:29.891284+0000","last_clean":"2024-04-25T11:10:29.891284+0000","last_became_active":"2024-04-25T11:10:27.547579+0000","last_became_peered":"2024-04-25T11:10:27.547579+0000","last_unstale":"2024-04-25T11:10:29.891284+0000","last_undegraded":"2024-04-25T11:10:29.891284+0000","last_fullsized":"2024-04-25T11:10:29.891284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:45.815273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291802+0000","last_change":"2024-04-25T11:10:23.478738+0000","last_active":"2024-04-25T11:10:29.291802+0000","last_peered":"2024-04-25T11:10:29.291802+0000","last_clean":"2024-04-25T11:10:29.291802+0000","last_became_active":"2024-04-25T11:10:23.468486+0000","last_became_peered":"2024-04-25T11:10:23.468486+0000","last_unstale":"2024-04-25T11:10:29.291802+0000","last_undegraded":"2024-04-25T11:10:29.291802+0000","last_fullsized":"2024-04-25T11:10:29.291802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:47.104859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533727+0000","last_change":"2024-04-25T11:10:22.473530+0000","last_active":"2024-04-25T11:10:28.533727+0000","last_peered":"2024-04-25T11:10:28.533727+0000","last_clean":"2024-04-25T11:10:28.533727+0000","last_became_active":"2024-04-25T11:10:22.473177+0000","last_became_peered":"2024-04-25T11:10:22.473177+0000","last_unstale":"2024-04-25T11:10:28.533727+0000","last_undegraded":"2024-04-25T11:10:28.533727+0000","last_fullsized":"2024-04-25T11:10:28.533727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894250+0000","last_change":"2024-04-25T11:10:27.559345+0000","last_active":"2024-04-25T11:10:29.894250+0000","last_peered":"2024-04-25T11:10:29.894250+0000","last_clean":"2024-04-25T11:10:29.894250+0000","last_became_active":"2024-04-25T11:10:27.558090+0000","last_became_peered":"2024-04-25T11:10:27.558090+0000","last_unstale":"2024-04-25T11:10:29.894250+0000","last_undegraded":"2024-04-25T11:10:29.894250+0000","last_fullsized":"2024-04-25T11:10:29.894250+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:51.174327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538883+0000","last_change":"2024-04-25T11:10:23.477613+0000","last_active":"2024-04-25T11:10:28.538883+0000","last_peered":"2024-04-25T11:10:28.538883+0000","last_clean":"2024-04-25T11:10:28.538883+0000","last_became_active":"2024-04-25T11:10:23.477226+0000","last_became_peered":"2024-04-25T11:10:23.477226+0000","last_unstale":"2024-04-25T11:10:28.538883+0000","last_undegraded":"2024-04-25T11:10:28.538883+0000","last_fullsized":"2024-04-25T11:10:28.538883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:07.701501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.374436+0000","last_change":"2024-04-25T11:10:22.468389+0000","last_active":"2024-04-25T11:10:29.374436+0000","last_peered":"2024-04-25T11:10:29.374436+0000","last_clean":"2024-04-25T11:10:29.374436+0000","last_became_active":"2024-04-25T11:10:22.467926+0000","last_became_peered":"2024-04-25T11:10:22.467926+0000","last_unstale":"2024-04-25T11:10:29.374436+0000","last_undegraded":"2024-04-25T11:10:29.374436+0000","last_fullsized":"2024-04-25T11:10:29.374436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.448477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290385+0000","last_change":"2024-04-25T11:10:27.547813+0000","last_active":"2024-04-25T11:10:29.290385+0000","last_peered":"2024-04-25T11:10:29.290385+0000","last_clean":"2024-04-25T11:10:29.290385+0000","last_became_active":"2024-04-25T11:10:27.547300+0000","last_became_peered":"2024-04-25T11:10:27.547300+0000","last_unstale":"2024-04-25T11:10:29.290385+0000","last_undegraded":"2024-04-25T11:10:29.290385+0000","last_fullsized":"2024-04-25T11:10:29.290385+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:51.538673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537521+0000","last_change":"2024-04-25T11:10:23.499736+0000","last_active":"2024-04-25T11:10:28.537521+0000","last_peered":"2024-04-25T11:10:28.537521+0000","last_clean":"2024-04-25T11:10:28.537521+0000","last_became_active":"2024-04-25T11:10:23.491282+0000","last_became_peered":"2024-04-25T11:10:23.491282+0000","last_unstale":"2024-04-25T11:10:28.537521+0000","last_undegraded":"2024-04-25T11:10:28.537521+0000","last_fullsized":"2024-04-25T11:10:28.537521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:15.460249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.539367+0000","last_change":"2024-04-25T11:10:22.477761+0000","last_active":"2024-04-25T11:10:28.539367+0000","last_peered":"2024-04-25T11:10:28.539367+0000","last_clean":"2024-04-25T11:10:28.539367+0000","last_became_active":"2024-04-25T11:10:22.477390+0000","last_became_peered":"2024-04-25T11:10:22.477390+0000","last_unstale":"2024-04-25T11:10:28.539367+0000","last_undegraded":"2024-04-25T11:10:28.539367+0000","last_fullsized":"2024-04-25T11:10:28.539367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:48.408404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535808+0000","last_change":"2024-04-25T11:10:27.559907+0000","last_active":"2024-04-25T11:10:28.535808+0000","last_peered":"2024-04-25T11:10:28.535808+0000","last_clean":"2024-04-25T11:10:28.535808+0000","last_became_active":"2024-04-25T11:10:27.559437+0000","last_became_peered":"2024-04-25T11:10:27.559437+0000","last_unstale":"2024-04-25T11:10:28.535808+0000","last_undegraded":"2024-04-25T11:10:28.535808+0000","last_fullsized":"2024-04-25T11:10:28.535808+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.067687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535432+0000","last_change":"2024-04-25T11:10:23.479907+0000","last_active":"2024-04-25T11:10:28.535432+0000","last_peered":"2024-04-25T11:10:28.535432+0000","last_clean":"2024-04-25T11:10:28.535432+0000","last_became_active":"2024-04-25T11:10:23.479578+0000","last_became_peered":"2024-04-25T11:10:23.479578+0000","last_unstale":"2024-04-25T11:10:28.535432+0000","last_undegraded":"2024-04-25T11:10:28.535432+0000","last_fullsized":"2024-04-25T11:10:28.535432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373126+0000","last_change":"2024-04-25T11:10:22.467360+0000","last_active":"2024-04-25T11:10:29.373126+0000","last_peered":"2024-04-25T11:10:29.373126+0000","last_clean":"2024-04-25T11:10:29.373126+0000","last_became_active":"2024-04-25T11:10:22.467002+0000","last_became_peered":"2024-04-25T11:10:22.467002+0000","last_unstale":"2024-04-25T11:10:29.373126+0000","last_undegraded":"2024-04-25T11:10:29.373126+0000","last_fullsized":"2024-04-25T11:10:29.373126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:17.490319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894874+0000","last_change":"2024-04-25T11:10:27.566981+0000","last_active":"2024-04-25T11:10:29.894874+0000","last_peered":"2024-04-25T11:10:29.894874+0000","last_clean":"2024-04-25T11:10:29.894874+0000","last_became_active":"2024-04-25T11:10:27.566434+0000","last_became_peered":"2024-04-25T11:10:27.566434+0000","last_unstale":"2024-04-25T11:10:29.894874+0000","last_undegraded":"2024-04-25T11:10:29.894874+0000","last_fullsized":"2024-04-25T11:10:29.894874+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:55.043679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891408+0000","last_change":"2024-04-25T11:10:23.489031+0000","last_active":"2024-04-25T11:10:29.891408+0000","last_peered":"2024-04-25T11:10:29.891408+0000","last_clean":"2024-04-25T11:10:29.891408+0000","last_became_active":"2024-04-25T11:10:23.487804+0000","last_became_peered":"2024-04-25T11:10:23.487804+0000","last_unstale":"2024-04-25T11:10:29.891408+0000","last_undegraded":"2024-04-25T11:10:29.891408+0000","last_fullsized":"2024-04-25T11:10:29.891408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891229+0000","last_change":"2024-04-25T11:10:22.489707+0000","last_active":"2024-04-25T11:10:29.891229+0000","last_peered":"2024-04-25T11:10:29.891229+0000","last_clean":"2024-04-25T11:10:29.891229+0000","last_became_active":"2024-04-25T11:10:22.489324+0000","last_became_peered":"2024-04-25T11:10:22.489324+0000","last_unstale":"2024-04-25T11:10:29.891229+0000","last_undegraded":"2024-04-25T11:10:29.891229+0000","last_fullsized":"2024-04-25T11:10:29.891229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896355+0000","last_change":"2024-04-25T11:10:27.556359+0000","last_active":"2024-04-25T11:10:29.896355+0000","last_peered":"2024-04-25T11:10:29.896355+0000","last_clean":"2024-04-25T11:10:29.896355+0000","last_became_active":"2024-04-25T11:10:27.555528+0000","last_became_peered":"2024-04-25T11:10:27.555528+0000","last_unstale":"2024-04-25T11:10:29.896355+0000","last_undegraded":"2024-04-25T11:10:29.896355+0000","last_fullsized":"2024-04-25T11:10:29.896355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:35.346179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535152+0000","last_change":"2024-04-25T11:10:23.478210+0000","last_active":"2024-04-25T11:10:28.535152+0000","last_peered":"2024-04-25T11:10:28.535152+0000","last_clean":"2024-04-25T11:10:28.535152+0000","last_became_active":"2024-04-25T11:10:23.477839+0000","last_became_peered":"2024-04-25T11:10:23.477839+0000","last_unstale":"2024-04-25T11:10:28.535152+0000","last_undegraded":"2024-04-25T11:10:28.535152+0000","last_fullsized":"2024-04-25T11:10:28.535152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:41.745805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290930+0000","last_change":"2024-04-25T11:10:22.487737+0000","last_active":"2024-04-25T11:10:29.290930+0000","last_peered":"2024-04-25T11:10:29.290930+0000","last_clean":"2024-04-25T11:10:29.290930+0000","last_became_active":"2024-04-25T11:10:22.485337+0000","last_became_peered":"2024-04-25T11:10:22.485337+0000","last_unstale":"2024-04-25T11:10:29.290930+0000","last_undegraded":"2024-04-25T11:10:29.290930+0000","last_fullsized":"2024-04-25T11:10:29.290930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894290+0000","last_change":"2024-04-25T11:10:27.562988+0000","last_active":"2024-04-25T11:10:29.894290+0000","last_peered":"2024-04-25T11:10:29.894290+0000","last_clean":"2024-04-25T11:10:29.894290+0000","last_became_active":"2024-04-25T11:10:27.562511+0000","last_became_peered":"2024-04-25T11:10:27.562511+0000","last_unstale":"2024-04-25T11:10:29.894290+0000","last_undegraded":"2024-04-25T11:10:29.894290+0000","last_fullsized":"2024-04-25T11:10:29.894290+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:32.363124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893441+0000","last_change":"2024-04-25T11:10:23.480645+0000","last_active":"2024-04-25T11:10:29.893441+0000","last_peered":"2024-04-25T11:10:29.893441+0000","last_clean":"2024-04-25T11:10:29.893441+0000","last_became_active":"2024-04-25T11:10:23.480244+0000","last_became_peered":"2024-04-25T11:10:23.480244+0000","last_unstale":"2024-04-25T11:10:29.893441+0000","last_undegraded":"2024-04-25T11:10:29.893441+0000","last_fullsized":"2024-04-25T11:10:29.893441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:46.490227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894602+0000","last_change":"2024-04-25T11:10:22.479455+0000","last_active":"2024-04-25T11:10:29.894602+0000","last_peered":"2024-04-25T11:10:29.894602+0000","last_clean":"2024-04-25T11:10:29.894602+0000","last_became_active":"2024-04-25T11:10:22.479024+0000","last_became_peered":"2024-04-25T11:10:22.479024+0000","last_unstale":"2024-04-25T11:10:29.894602+0000","last_undegraded":"2024-04-25T11:10:29.894602+0000","last_fullsized":"2024-04-25T11:10:29.894602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:56.918083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291635+0000","last_change":"2024-04-25T11:10:27.549545+0000","last_active":"2024-04-25T11:10:29.291635+0000","last_peered":"2024-04-25T11:10:29.291635+0000","last_clean":"2024-04-25T11:10:29.291635+0000","last_became_active":"2024-04-25T11:10:27.549085+0000","last_became_peered":"2024-04-25T11:10:27.549085+0000","last_unstale":"2024-04-25T11:10:29.291635+0000","last_undegraded":"2024-04-25T11:10:29.291635+0000","last_fullsized":"2024-04-25T11:10:29.291635+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.906264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536502+0000","last_change":"2024-04-25T11:10:23.472355+0000","last_active":"2024-04-25T11:10:28.536502+0000","last_peered":"2024-04-25T11:10:28.536502+0000","last_clean":"2024-04-25T11:10:28.536502+0000","last_became_active":"2024-04-25T11:10:23.470905+0000","last_became_peered":"2024-04-25T11:10:23.470905+0000","last_unstale":"2024-04-25T11:10:28.536502+0000","last_undegraded":"2024-04-25T11:10:28.536502+0000","last_fullsized":"2024-04-25T11:10:28.536502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:20.100576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:44.462567+0000","last_change":"2024-04-25T11:10:22.481299+0000","last_active":"2024-04-25T11:10:44.462567+0000","last_peered":"2024-04-25T11:10:44.462567+0000","last_clean":"2024-04-25T11:10:44.462567+0000","last_became_active":"2024-04-25T11:10:22.480741+0000","last_became_peered":"2024-04-25T11:10:22.480741+0000","last_unstale":"2024-04-25T11:10:44.462567+0000","last_undegraded":"2024-04-25T11:10:44.462567+0000","last_fullsized":"2024-04-25T11:10:44.462567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533006+0000","last_change":"2024-04-25T11:10:27.551729+0000","last_active":"2024-04-25T11:10:28.533006+0000","last_peered":"2024-04-25T11:10:28.533006+0000","last_clean":"2024-04-25T11:10:28.533006+0000","last_became_active":"2024-04-25T11:10:27.548417+0000","last_became_peered":"2024-04-25T11:10:27.548417+0000","last_unstale":"2024-04-25T11:10:28.533006+0000","last_undegraded":"2024-04-25T11:10:28.533006+0000","last_fullsized":"2024-04-25T11:10:28.533006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538108+0000","last_change":"2024-04-25T11:10:23.482351+0000","last_active":"2024-04-25T11:10:28.538108+0000","last_peered":"2024-04-25T11:10:28.538108+0000","last_clean":"2024-04-25T11:10:28.538108+0000","last_became_active":"2024-04-25T11:10:23.481665+0000","last_became_peered":"2024-04-25T11:10:23.481665+0000","last_unstale":"2024-04-25T11:10:28.538108+0000","last_undegraded":"2024-04-25T11:10:28.538108+0000","last_fullsized":"2024-04-25T11:10:28.538108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:50.877149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533327+0000","last_change":"2024-04-25T11:10:22.484130+0000","last_active":"2024-04-25T11:10:28.533327+0000","last_peered":"2024-04-25T11:10:28.533327+0000","last_clean":"2024-04-25T11:10:28.533327+0000","last_became_active":"2024-04-25T11:10:22.483811+0000","last_became_peered":"2024-04-25T11:10:22.483811+0000","last_unstale":"2024-04-25T11:10:28.533327+0000","last_undegraded":"2024-04-25T11:10:28.533327+0000","last_fullsized":"2024-04-25T11:10:28.533327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535359+0000","last_change":"2024-04-25T11:10:27.564379+0000","last_active":"2024-04-25T11:10:28.535359+0000","last_peered":"2024-04-25T11:10:28.535359+0000","last_clean":"2024-04-25T11:10:28.535359+0000","last_became_active":"2024-04-25T11:10:27.563892+0000","last_became_peered":"2024-04-25T11:10:27.563892+0000","last_unstale":"2024-04-25T11:10:28.535359+0000","last_undegraded":"2024-04-25T11:10:28.535359+0000","last_fullsized":"2024-04-25T11:10:28.535359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:06.999062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895182+0000","last_change":"2024-04-25T11:10:23.487092+0000","last_active":"2024-04-25T11:10:29.895182+0000","last_peered":"2024-04-25T11:10:29.895182+0000","last_clean":"2024-04-25T11:10:29.895182+0000","last_became_active":"2024-04-25T11:10:23.486583+0000","last_became_peered":"2024-04-25T11:10:23.486583+0000","last_unstale":"2024-04-25T11:10:29.895182+0000","last_undegraded":"2024-04-25T11:10:29.895182+0000","last_fullsized":"2024-04-25T11:10:29.895182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291509+0000","last_change":"2024-04-25T11:10:22.484333+0000","last_active":"2024-04-25T11:10:29.291509+0000","last_peered":"2024-04-25T11:10:29.291509+0000","last_clean":"2024-04-25T11:10:29.291509+0000","last_became_active":"2024-04-25T11:10:22.483865+0000","last_became_peered":"2024-04-25T11:10:22.483865+0000","last_unstale":"2024-04-25T11:10:29.291509+0000","last_undegraded":"2024-04-25T11:10:29.291509+0000","last_fullsized":"2024-04-25T11:10:29.291509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891783+0000","last_change":"2024-04-25T11:10:27.551066+0000","last_active":"2024-04-25T11:10:29.891783+0000","last_peered":"2024-04-25T11:10:29.891783+0000","last_clean":"2024-04-25T11:10:29.891783+0000","last_became_active":"2024-04-25T11:10:27.550625+0000","last_became_peered":"2024-04-25T11:10:27.550625+0000","last_unstale":"2024-04-25T11:10:29.891783+0000","last_undegraded":"2024-04-25T11:10:29.891783+0000","last_fullsized":"2024-04-25T11:10:29.891783+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.906264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291447+0000","last_change":"2024-04-25T11:10:23.485935+0000","last_active":"2024-04-25T11:10:29.291447+0000","last_peered":"2024-04-25T11:10:29.291447+0000","last_clean":"2024-04-25T11:10:29.291447+0000","last_became_active":"2024-04-25T11:10:23.485367+0000","last_became_peered":"2024-04-25T11:10:23.485367+0000","last_unstale":"2024-04-25T11:10:29.291447+0000","last_undegraded":"2024-04-25T11:10:29.291447+0000","last_fullsized":"2024-04-25T11:10:29.291447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:50.974212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892031+0000","last_change":"2024-04-25T11:10:22.482686+0000","last_active":"2024-04-25T11:10:29.892031+0000","last_peered":"2024-04-25T11:10:29.892031+0000","last_clean":"2024-04-25T11:10:29.892031+0000","last_became_active":"2024-04-25T11:10:22.482307+0000","last_became_peered":"2024-04-25T11:10:22.482307+0000","last_unstale":"2024-04-25T11:10:29.892031+0000","last_undegraded":"2024-04-25T11:10:29.892031+0000","last_fullsized":"2024-04-25T11:10:29.892031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892597+0000","last_change":"2024-04-25T11:10:27.563942+0000","last_active":"2024-04-25T11:10:29.892597+0000","last_peered":"2024-04-25T11:10:29.892597+0000","last_clean":"2024-04-25T11:10:29.892597+0000","last_became_active":"2024-04-25T11:10:27.561604+0000","last_became_peered":"2024-04-25T11:10:27.561604+0000","last_unstale":"2024-04-25T11:10:29.892597+0000","last_undegraded":"2024-04-25T11:10:29.892597+0000","last_fullsized":"2024-04-25T11:10:29.892597+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:59.323947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893904+0000","last_change":"2024-04-25T11:10:27.559342+0000","last_active":"2024-04-25T11:10:29.893904+0000","last_peered":"2024-04-25T11:10:29.893904+0000","last_clean":"2024-04-25T11:10:29.893904+0000","last_became_active":"2024-04-25T11:10:27.558417+0000","last_became_peered":"2024-04-25T11:10:27.558417+0000","last_unstale":"2024-04-25T11:10:29.893904+0000","last_undegraded":"2024-04-25T11:10:29.893904+0000","last_fullsized":"2024-04-25T11:10:29.893904+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.970801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531482+0000","last_change":"2024-04-25T11:10:23.472501+0000","last_active":"2024-04-25T11:10:28.531482+0000","last_peered":"2024-04-25T11:10:28.531482+0000","last_clean":"2024-04-25T11:10:28.531482+0000","last_became_active":"2024-04-25T11:10:23.472103+0000","last_became_peered":"2024-04-25T11:10:23.472103+0000","last_unstale":"2024-04-25T11:10:28.531482+0000","last_undegraded":"2024-04-25T11:10:28.531482+0000","last_fullsized":"2024-04-25T11:10:28.531482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290643+0000","last_change":"2024-04-25T11:10:22.487396+0000","last_active":"2024-04-25T11:10:29.290643+0000","last_peered":"2024-04-25T11:10:29.290643+0000","last_clean":"2024-04-25T11:10:29.290643+0000","last_became_active":"2024-04-25T11:10:22.484146+0000","last_became_peered":"2024-04-25T11:10:22.484146+0000","last_unstale":"2024-04-25T11:10:29.290643+0000","last_undegraded":"2024-04-25T11:10:29.290643+0000","last_fullsized":"2024-04-25T11:10:29.290643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891822+0000","last_change":"2024-04-25T11:10:27.545602+0000","last_active":"2024-04-25T11:10:29.891822+0000","last_peered":"2024-04-25T11:10:29.891822+0000","last_clean":"2024-04-25T11:10:29.891822+0000","last_became_active":"2024-04-25T11:10:27.545089+0000","last_became_peered":"2024-04-25T11:10:27.545089+0000","last_unstale":"2024-04-25T11:10:29.891822+0000","last_undegraded":"2024-04-25T11:10:29.891822+0000","last_fullsized":"2024-04-25T11:10:29.891822+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:24.170044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891897+0000","last_change":"2024-04-25T11:10:23.480332+0000","last_active":"2024-04-25T11:10:29.891897+0000","last_peered":"2024-04-25T11:10:29.891897+0000","last_clean":"2024-04-25T11:10:29.891897+0000","last_became_active":"2024-04-25T11:10:23.479866+0000","last_became_peered":"2024-04-25T11:10:23.479866+0000","last_unstale":"2024-04-25T11:10:29.891897+0000","last_undegraded":"2024-04-25T11:10:29.891897+0000","last_fullsized":"2024-04-25T11:10:29.891897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893193+0000","last_change":"2024-04-25T11:10:22.476980+0000","last_active":"2024-04-25T11:10:29.893193+0000","last_peered":"2024-04-25T11:10:29.893193+0000","last_clean":"2024-04-25T11:10:29.893193+0000","last_became_active":"2024-04-25T11:10:22.476483+0000","last_became_peered":"2024-04-25T11:10:22.476483+0000","last_unstale":"2024-04-25T11:10:29.893193+0000","last_undegraded":"2024-04-25T11:10:29.893193+0000","last_fullsized":"2024-04-25T11:10:29.893193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:56.918083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891422+0000","last_change":"2024-04-25T11:10:22.485058+0000","last_active":"2024-04-25T11:10:29.891422+0000","last_peered":"2024-04-25T11:10:29.891422+0000","last_clean":"2024-04-25T11:10:29.891422+0000","last_became_active":"2024-04-25T11:10:22.484647+0000","last_became_peered":"2024-04-25T11:10:22.484647+0000","last_unstale":"2024-04-25T11:10:29.891422+0000","last_undegraded":"2024-04-25T11:10:29.891422+0000","last_fullsized":"2024-04-25T11:10:29.891422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:42.000742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894489+0000","last_change":"2024-04-25T11:10:23.494632+0000","last_active":"2024-04-25T11:10:29.894489+0000","last_peered":"2024-04-25T11:10:29.894489+0000","last_clean":"2024-04-25T11:10:29.894489+0000","last_became_active":"2024-04-25T11:10:23.494214+0000","last_became_peered":"2024-04-25T11:10:23.494214+0000","last_unstale":"2024-04-25T11:10:29.894489+0000","last_undegraded":"2024-04-25T11:10:29.894489+0000","last_fullsized":"2024-04-25T11:10:29.894489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373551+0000","last_change":"2024-04-25T11:10:27.546702+0000","last_active":"2024-04-25T11:10:29.373551+0000","last_peered":"2024-04-25T11:10:29.373551+0000","last_clean":"2024-04-25T11:10:29.373551+0000","last_became_active":"2024-04-25T11:10:27.546093+0000","last_became_peered":"2024-04-25T11:10:27.546093+0000","last_unstale":"2024-04-25T11:10:29.373551+0000","last_undegraded":"2024-04-25T11:10:29.373551+0000","last_fullsized":"2024-04-25T11:10:29.373551+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:55.973871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892460+0000","last_change":"2024-04-25T11:10:22.470279+0000","last_active":"2024-04-25T11:10:29.892460+0000","last_peered":"2024-04-25T11:10:29.892460+0000","last_clean":"2024-04-25T11:10:29.892460+0000","last_became_active":"2024-04-25T11:10:22.469133+0000","last_became_peered":"2024-04-25T11:10:22.469133+0000","last_unstale":"2024-04-25T11:10:29.892460+0000","last_undegraded":"2024-04-25T11:10:29.892460+0000","last_fullsized":"2024-04-25T11:10:29.892460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.835819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536180+0000","last_change":"2024-04-25T11:10:23.482007+0000","last_active":"2024-04-25T11:10:28.536180+0000","last_peered":"2024-04-25T11:10:28.536180+0000","last_clean":"2024-04-25T11:10:28.536180+0000","last_became_active":"2024-04-25T11:10:23.475539+0000","last_became_peered":"2024-04-25T11:10:23.475539+0000","last_unstale":"2024-04-25T11:10:28.536180+0000","last_undegraded":"2024-04-25T11:10:28.536180+0000","last_fullsized":"2024-04-25T11:10:28.536180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:56.556443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895038+0000","last_change":"2024-04-25T11:10:27.555814+0000","last_active":"2024-04-25T11:10:29.895038+0000","last_peered":"2024-04-25T11:10:29.895038+0000","last_clean":"2024-04-25T11:10:29.895038+0000","last_became_active":"2024-04-25T11:10:27.554994+0000","last_became_peered":"2024-04-25T11:10:27.554994+0000","last_unstale":"2024-04-25T11:10:29.895038+0000","last_undegraded":"2024-04-25T11:10:29.895038+0000","last_fullsized":"2024-04-25T11:10:29.895038+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:11.895073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.890634+0000","last_change":"2024-04-25T11:10:23.487735+0000","last_active":"2024-04-25T11:10:29.890634+0000","last_peered":"2024-04-25T11:10:29.890634+0000","last_clean":"2024-04-25T11:10:29.890634+0000","last_became_active":"2024-04-25T11:10:23.487354+0000","last_became_peered":"2024-04-25T11:10:23.487354+0000","last_unstale":"2024-04-25T11:10:29.890634+0000","last_undegraded":"2024-04-25T11:10:29.890634+0000","last_fullsized":"2024-04-25T11:10:29.890634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373170+0000","last_change":"2024-04-25T11:10:23.476515+0000","last_active":"2024-04-25T11:10:29.373170+0000","last_peered":"2024-04-25T11:10:29.373170+0000","last_clean":"2024-04-25T11:10:29.373170+0000","last_became_active":"2024-04-25T11:10:23.476183+0000","last_became_peered":"2024-04-25T11:10:23.476183+0000","last_unstale":"2024-04-25T11:10:29.373170+0000","last_undegraded":"2024-04-25T11:10:29.373170+0000","last_fullsized":"2024-04-25T11:10:29.373170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:57.829487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290735+0000","last_change":"2024-04-25T11:10:23.477846+0000","last_active":"2024-04-25T11:10:29.290735+0000","last_peered":"2024-04-25T11:10:29.290735+0000","last_clean":"2024-04-25T11:10:29.290735+0000","last_became_active":"2024-04-25T11:10:23.476420+0000","last_became_peered":"2024-04-25T11:10:23.476420+0000","last_unstale":"2024-04-25T11:10:29.290735+0000","last_undegraded":"2024-04-25T11:10:29.290735+0000","last_fullsized":"2024-04-25T11:10:29.290735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894372+0000","last_change":"2024-04-25T11:10:23.485446+0000","last_active":"2024-04-25T11:10:29.894372+0000","last_peered":"2024-04-25T11:10:29.894372+0000","last_clean":"2024-04-25T11:10:29.894372+0000","last_became_active":"2024-04-25T11:10:23.485095+0000","last_became_peered":"2024-04-25T11:10:23.485095+0000","last_unstale":"2024-04-25T11:10:29.894372+0000","last_undegraded":"2024-04-25T11:10:29.894372+0000","last_fullsized":"2024-04-25T11:10:29.894372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895256+0000","last_change":"2024-04-25T11:10:23.490644+0000","last_active":"2024-04-25T11:10:29.895256+0000","last_peered":"2024-04-25T11:10:29.895256+0000","last_clean":"2024-04-25T11:10:29.895256+0000","last_became_active":"2024-04-25T11:10:23.487911+0000","last_became_peered":"2024-04-25T11:10:23.487911+0000","last_unstale":"2024-04-25T11:10:29.895256+0000","last_undegraded":"2024-04-25T11:10:29.895256+0000","last_fullsized":"2024-04-25T11:10:29.895256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:48.595548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.532685+0000","last_change":"2024-04-25T11:10:23.484155+0000","last_active":"2024-04-25T11:10:28.532685+0000","last_peered":"2024-04-25T11:10:28.532685+0000","last_clean":"2024-04-25T11:10:28.532685+0000","last_became_active":"2024-04-25T11:10:23.483906+0000","last_became_peered":"2024-04-25T11:10:23.483906+0000","last_unstale":"2024-04-25T11:10:28.532685+0000","last_undegraded":"2024-04-25T11:10:28.532685+0000","last_fullsized":"2024-04-25T11:10:28.532685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892230+0000","last_change":"2024-04-25T11:10:23.479315+0000","last_active":"2024-04-25T11:10:29.892230+0000","last_peered":"2024-04-25T11:10:29.892230+0000","last_clean":"2024-04-25T11:10:29.892230+0000","last_became_active":"2024-04-25T11:10:23.478986+0000","last_became_peered":"2024-04-25T11:10:23.478986+0000","last_unstale":"2024-04-25T11:10:29.892230+0000","last_undegraded":"2024-04-25T11:10:29.892230+0000","last_fullsized":"2024-04-25T11:10:29.892230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896160+0000","last_change":"2024-04-25T11:10:23.475790+0000","last_active":"2024-04-25T11:10:29.896160+0000","last_peered":"2024-04-25T11:10:29.896160+0000","last_clean":"2024-04-25T11:10:29.896160+0000","last_became_active":"2024-04-25T11:10:23.475049+0000","last_became_peered":"2024-04-25T11:10:23.475049+0000","last_unstale":"2024-04-25T11:10:29.896160+0000","last_undegraded":"2024-04-25T11:10:29.896160+0000","last_fullsized":"2024-04-25T11:10:29.896160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290626+0000","last_change":"2024-04-25T11:10:23.489591+0000","last_active":"2024-04-25T11:10:29.290626+0000","last_peered":"2024-04-25T11:10:29.290626+0000","last_clean":"2024-04-25T11:10:29.290626+0000","last_became_active":"2024-04-25T11:10:23.488935+0000","last_became_peered":"2024-04-25T11:10:23.488935+0000","last_unstale":"2024-04-25T11:10:29.290626+0000","last_undegraded":"2024-04-25T11:10:29.290626+0000","last_fullsized":"2024-04-25T11:10:29.290626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892914+0000","last_change":"2024-04-25T11:10:23.487850+0000","last_active":"2024-04-25T11:10:29.892914+0000","last_peered":"2024-04-25T11:10:29.892914+0000","last_clean":"2024-04-25T11:10:29.892914+0000","last_became_active":"2024-04-25T11:10:23.487234+0000","last_became_peered":"2024-04-25T11:10:23.487234+0000","last_unstale":"2024-04-25T11:10:29.892914+0000","last_undegraded":"2024-04-25T11:10:29.892914+0000","last_fullsized":"2024-04-25T11:10:29.892914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290514+0000","last_change":"2024-04-25T11:10:23.473547+0000","last_active":"2024-04-25T11:10:29.290514+0000","last_peered":"2024-04-25T11:10:29.290514+0000","last_clean":"2024-04-25T11:10:29.290514+0000","last_became_active":"2024-04-25T11:10:23.467594+0000","last_became_peered":"2024-04-25T11:10:23.467594+0000","last_unstale":"2024-04-25T11:10:29.290514+0000","last_undegraded":"2024-04-25T11:10:29.290514+0000","last_fullsized":"2024-04-25T11:10:29.290514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:48.595548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893781+0000","last_change":"2024-04-25T11:10:23.480740+0000","last_active":"2024-04-25T11:10:29.893781+0000","last_peered":"2024-04-25T11:10:29.893781+0000","last_clean":"2024-04-25T11:10:29.893781+0000","last_became_active":"2024-04-25T11:10:23.480405+0000","last_became_peered":"2024-04-25T11:10:23.480405+0000","last_unstale":"2024-04-25T11:10:29.893781+0000","last_undegraded":"2024-04-25T11:10:29.893781+0000","last_fullsized":"2024-04-25T11:10:29.893781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891342+0000","last_change":"2024-04-25T11:10:23.497261+0000","last_active":"2024-04-25T11:10:29.891342+0000","last_peered":"2024-04-25T11:10:29.891342+0000","last_clean":"2024-04-25T11:10:29.891342+0000","last_became_active":"2024-04-25T11:10:23.493494+0000","last_became_peered":"2024-04-25T11:10:23.493494+0000","last_unstale":"2024-04-25T11:10:29.891342+0000","last_undegraded":"2024-04-25T11:10:29.891342+0000","last_fullsized":"2024-04-25T11:10:29.891342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537112+0000","last_change":"2024-04-25T11:10:23.469676+0000","last_active":"2024-04-25T11:10:28.537112+0000","last_peered":"2024-04-25T11:10:28.537112+0000","last_clean":"2024-04-25T11:10:28.537112+0000","last_became_active":"2024-04-25T11:10:23.468887+0000","last_became_peered":"2024-04-25T11:10:23.468887+0000","last_unstale":"2024-04-25T11:10:28.537112+0000","last_undegraded":"2024-04-25T11:10:28.537112+0000","last_fullsized":"2024-04-25T11:10:28.537112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:47.104859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.374373+0000","last_change":"2024-04-25T11:10:23.491992+0000","last_active":"2024-04-25T11:10:29.374373+0000","last_peered":"2024-04-25T11:10:29.374373+0000","last_clean":"2024-04-25T11:10:29.374373+0000","last_became_active":"2024-04-25T11:10:23.491456+0000","last_became_peered":"2024-04-25T11:10:23.491456+0000","last_unstale":"2024-04-25T11:10:29.374373+0000","last_undegraded":"2024-04-25T11:10:29.374373+0000","last_fullsized":"2024-04-25T11:10:29.374373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.097249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.292111+0000","last_change":"2024-04-25T11:10:23.469823+0000","last_active":"2024-04-25T11:10:29.292111+0000","last_peered":"2024-04-25T11:10:29.292111+0000","last_clean":"2024-04-25T11:10:29.292111+0000","last_became_active":"2024-04-25T11:10:23.469420+0000","last_became_peered":"2024-04-25T11:10:23.469420+0000","last_unstale":"2024-04-25T11:10:29.292111+0000","last_undegraded":"2024-04-25T11:10:29.292111+0000","last_fullsized":"2024-04-25T11:10:29.292111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290180+0000","last_change":"2024-04-25T11:10:23.479442+0000","last_active":"2024-04-25T11:10:29.290180+0000","last_peered":"2024-04-25T11:10:29.290180+0000","last_clean":"2024-04-25T11:10:29.290180+0000","last_became_active":"2024-04-25T11:10:23.472659+0000","last_became_peered":"2024-04-25T11:10:23.472659+0000","last_unstale":"2024-04-25T11:10:29.290180+0000","last_undegraded":"2024-04-25T11:10:29.290180+0000","last_fullsized":"2024-04-25T11:10:29.290180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531629+0000","last_change":"2024-04-25T11:10:23.490775+0000","last_active":"2024-04-25T11:10:28.531629+0000","last_peered":"2024-04-25T11:10:28.531629+0000","last_clean":"2024-04-25T11:10:28.531629+0000","last_became_active":"2024-04-25T11:10:23.490138+0000","last_became_peered":"2024-04-25T11:10:23.490138+0000","last_unstale":"2024-04-25T11:10:28.531629+0000","last_undegraded":"2024-04-25T11:10:28.531629+0000","last_fullsized":"2024-04-25T11:10:28.531629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893293+0000","last_change":"2024-04-25T11:10:23.479882+0000","last_active":"2024-04-25T11:10:29.893293+0000","last_peered":"2024-04-25T11:10:29.893293+0000","last_clean":"2024-04-25T11:10:29.893293+0000","last_became_active":"2024-04-25T11:10:23.479534+0000","last_became_peered":"2024-04-25T11:10:23.479534+0000","last_unstale":"2024-04-25T11:10:29.893293+0000","last_undegraded":"2024-04-25T11:10:29.893293+0000","last_fullsized":"2024-04-25T11:10:29.893293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:04.105145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892215+0000","last_change":"2024-04-25T11:10:23.483845+0000","last_active":"2024-04-25T11:10:29.892215+0000","last_peered":"2024-04-25T11:10:29.892215+0000","last_clean":"2024-04-25T11:10:29.892215+0000","last_became_active":"2024-04-25T11:10:23.482401+0000","last_became_peered":"2024-04-25T11:10:23.482401+0000","last_unstale":"2024-04-25T11:10:29.892215+0000","last_undegraded":"2024-04-25T11:10:29.892215+0000","last_fullsized":"2024-04-25T11:10:29.892215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537880+0000","last_change":"2024-04-25T11:10:23.490750+0000","last_active":"2024-04-25T11:10:28.537880+0000","last_peered":"2024-04-25T11:10:28.537880+0000","last_clean":"2024-04-25T11:10:28.537880+0000","last_became_active":"2024-04-25T11:10:23.490489+0000","last_became_peered":"2024-04-25T11:10:23.490489+0000","last_unstale":"2024-04-25T11:10:28.537880+0000","last_undegraded":"2024-04-25T11:10:28.537880+0000","last_fullsized":"2024-04-25T11:10:28.537880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:30.205583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891988+0000","last_change":"2024-04-25T11:10:23.498316+0000","last_active":"2024-04-25T11:10:29.891988+0000","last_peered":"2024-04-25T11:10:29.891988+0000","last_clean":"2024-04-25T11:10:29.891988+0000","last_became_active":"2024-04-25T11:10:23.497414+0000","last_became_peered":"2024-04-25T11:10:23.497414+0000","last_unstale":"2024-04-25T11:10:29.891988+0000","last_undegraded":"2024-04-25T11:10:29.891988+0000","last_fullsized":"2024-04-25T11:10:29.891988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892093+0000","last_change":"2024-04-25T11:10:23.487588+0000","last_active":"2024-04-25T11:10:29.892093+0000","last_peered":"2024-04-25T11:10:29.892093+0000","last_clean":"2024-04-25T11:10:29.892093+0000","last_became_active":"2024-04-25T11:10:23.487280+0000","last_became_peered":"2024-04-25T11:10:23.487280+0000","last_unstale":"2024-04-25T11:10:29.892093+0000","last_undegraded":"2024-04-25T11:10:29.892093+0000","last_fullsized":"2024-04-25T11:10:29.892093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538694+0000","last_change":"2024-04-25T11:10:23.472328+0000","last_active":"2024-04-25T11:10:28.538694+0000","last_peered":"2024-04-25T11:10:28.538694+0000","last_clean":"2024-04-25T11:10:28.538694+0000","last_became_active":"2024-04-25T11:10:23.471692+0000","last_became_peered":"2024-04-25T11:10:23.471692+0000","last_unstale":"2024-04-25T11:10:28.538694+0000","last_undegraded":"2024-04-25T11:10:28.538694+0000","last_fullsized":"2024-04-25T11:10:28.538694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:50.974212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891640+0000","last_change":"2024-04-25T11:10:23.471703+0000","last_active":"2024-04-25T11:10:29.891640+0000","last_peered":"2024-04-25T11:10:29.891640+0000","last_clean":"2024-04-25T11:10:29.891640+0000","last_became_active":"2024-04-25T11:10:23.469570+0000","last_became_peered":"2024-04-25T11:10:23.469570+0000","last_unstale":"2024-04-25T11:10:29.891640+0000","last_undegraded":"2024-04-25T11:10:29.891640+0000","last_fullsized":"2024-04-25T11:10:29.891640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535108+0000","last_change":"2024-04-25T11:10:23.473106+0000","last_active":"2024-04-25T11:10:28.535108+0000","last_peered":"2024-04-25T11:10:28.535108+0000","last_clean":"2024-04-25T11:10:28.535108+0000","last_became_active":"2024-04-25T11:10:23.472627+0000","last_became_peered":"2024-04-25T11:10:23.472627+0000","last_unstale":"2024-04-25T11:10:28.535108+0000","last_undegraded":"2024-04-25T11:10:28.535108+0000","last_fullsized":"2024-04-25T11:10:28.535108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290419+0000","last_change":"2024-04-25T11:10:23.480283+0000","last_active":"2024-04-25T11:10:29.290419+0000","last_peered":"2024-04-25T11:10:29.290419+0000","last_clean":"2024-04-25T11:10:29.290419+0000","last_became_active":"2024-04-25T11:10:23.479889+0000","last_became_peered":"2024-04-25T11:10:23.479889+0000","last_unstale":"2024-04-25T11:10:29.290419+0000","last_undegraded":"2024-04-25T11:10:29.290419+0000","last_fullsized":"2024-04-25T11:10:29.290419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534607+0000","last_change":"2024-04-25T11:10:23.469104+0000","last_active":"2024-04-25T11:10:28.534607+0000","last_peered":"2024-04-25T11:10:28.534607+0000","last_clean":"2024-04-25T11:10:28.534607+0000","last_became_active":"2024-04-25T11:10:23.468743+0000","last_became_peered":"2024-04-25T11:10:23.468743+0000","last_unstale":"2024-04-25T11:10:28.534607+0000","last_undegraded":"2024-04-25T11:10:28.534607+0000","last_fullsized":"2024-04-25T11:10:28.534607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291058+0000","last_change":"2024-04-25T11:10:23.496747+0000","last_active":"2024-04-25T11:10:29.291058+0000","last_peered":"2024-04-25T11:10:29.291058+0000","last_clean":"2024-04-25T11:10:29.291058+0000","last_became_active":"2024-04-25T11:10:23.496354+0000","last_became_peered":"2024-04-25T11:10:23.496354+0000","last_unstale":"2024-04-25T11:10:29.291058+0000","last_undegraded":"2024-04-25T11:10:29.291058+0000","last_fullsized":"2024-04-25T11:10:29.291058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:41.745805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893849+0000","last_change":"2024-04-25T11:10:23.480571+0000","last_active":"2024-04-25T11:10:29.893849+0000","last_peered":"2024-04-25T11:10:29.893849+0000","last_clean":"2024-04-25T11:10:29.893849+0000","last_became_active":"2024-04-25T11:10:23.479502+0000","last_became_peered":"2024-04-25T11:10:23.479502+0000","last_unstale":"2024-04-25T11:10:29.893849+0000","last_undegraded":"2024-04-25T11:10:29.893849+0000","last_fullsized":"2024-04-25T11:10:29.893849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533940+0000","last_change":"2024-04-25T11:10:23.475376+0000","last_active":"2024-04-25T11:10:28.533940+0000","last_peered":"2024-04-25T11:10:28.533940+0000","last_clean":"2024-04-25T11:10:28.533940+0000","last_became_active":"2024-04-25T11:10:23.474750+0000","last_became_peered":"2024-04-25T11:10:23.474750+0000","last_unstale":"2024-04-25T11:10:28.533940+0000","last_undegraded":"2024-04-25T11:10:28.533940+0000","last_fullsized":"2024-04-25T11:10:28.533940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538538+0000","last_change":"2024-04-25T11:10:23.481991+0000","last_active":"2024-04-25T11:10:28.538538+0000","last_peered":"2024-04-25T11:10:28.538538+0000","last_clean":"2024-04-25T11:10:28.538538+0000","last_became_active":"2024-04-25T11:10:23.481639+0000","last_became_peered":"2024-04-25T11:10:23.481639+0000","last_unstale":"2024-04-25T11:10:28.538538+0000","last_undegraded":"2024-04-25T11:10:28.538538+0000","last_fullsized":"2024-04-25T11:10:28.538538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:13.333031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476748,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476749,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476748,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476749,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":104646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:11:02.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:11:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.343+0000 7f123ac00640 1 Processor -- start 2024-04-25T11:11:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.343+0000 7f123ac00640 1 -- start start 2024-04-25T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.345+0000 7f123ac00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12340b5220 0x7f123415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.345+0000 7f123ac00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12340cebb0 con 0x7f12340af420 2024-04-25T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.345+0000 7f123ac00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12340b64b0 con 0x7f12340aed40 2024-04-25T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.345+0000 7f123ac00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f123415e1f0 con 0x7f12340b5220 2024-04-25T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.346+0000 7f1238e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12340b5220 0x7f123415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.346+0000 7f123a200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f12340af420 0x7f12340af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33832/0 (socket says 172.21.15.7:33832) 2024-04-25T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.346+0000 7f123a200640 1 -- 172.21.15.7:0/2547354217 learned_addr learned my addr 172.21.15.7:0/2547354217 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.346+0000 7f1238e00640 1 --2- 172.21.15.7:0/2547354217 >> v2:172.21.15.7:3300/0 conn(0x7f12340b5220 0x7f123415dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43420/0 (socket says 172.21.15.7:43420) 2024-04-25T11:11:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.346+0000 7f1238e00640 1 -- 172.21.15.7:0/2547354217 >> v1:172.21.15.87:6789/0 conn(0x7f12340aed40 legacy=0x7f12340ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.346+0000 7f1238e00640 1 -- 172.21.15.7:0/2547354217 >> v1:172.21.15.7:6789/0 conn(0x7f12340af420 legacy=0x7f12340af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.346+0000 7f1238e00640 1 -- 172.21.15.7:0/2547354217 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123406c610 con 0x7f12340b5220 2024-04-25T11:11:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.347+0000 7f1238e00640 1 --2- 172.21.15.7:0/2547354217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12340b5220 0x7f123415dcb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f12240097a0 tx=0x7f122402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b1bf830461bdcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.348+0000 7f1233e00640 1 -- 172.21.15.7:0/2547354217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224037070 con 0x7f12340b5220 2024-04-25T11:11:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.348+0000 7f1233e00640 1 -- 172.21.15.7:0/2547354217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1224032440 con 0x7f12340b5220 2024-04-25T11:11:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.348+0000 7f1233e00640 1 -- 172.21.15.7:0/2547354217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224032810 con 0x7f12340b5220 2024-04-25T11:11:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.348+0000 7f123ac00640 1 -- 172.21.15.7:0/2547354217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12340b5220 msgr2=0x7f123415dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.348+0000 7f123ac00640 1 --2- 172.21.15.7:0/2547354217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12340b5220 0x7f123415dcb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f12240097a0 tx=0x7f122402a230 comp rx=0 tx=0).stop 2024-04-25T11:11:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.348+0000 7f123ac00640 1 -- 172.21.15.7:0/2547354217 shutdown_connections 2024-04-25T11:11:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.348+0000 7f123ac00640 1 --2- 172.21.15.7:0/2547354217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12340b5220 0x7f123415dcb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.348+0000 7f123ac00640 1 -- 172.21.15.7:0/2547354217 >> 172.21.15.7:0/2547354217 conn(0x7f12340b3820 msgr2=0x7f12340b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.349+0000 7f123ac00640 1 -- 172.21.15.7:0/2547354217 shutdown_connections 2024-04-25T11:11:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.349+0000 7f123ac00640 1 -- 172.21.15.7:0/2547354217 wait complete. 2024-04-25T11:11:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.351+0000 7f123ac00640 1 Processor -- start 2024-04-25T11:11:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.351+0000 7f123ac00640 1 -- start start 2024-04-25T11:11:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123ac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12340aed40 0x7f1234151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123ac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12340af420 0x7f12340cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123ac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12340b5220 0x7f12340d0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123ac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f123406ceb0 con 0x7f12340b5220 2024-04-25T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123ac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f123406cbd0 con 0x7f12340aed40 2024-04-25T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123ac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f12340b6110 con 0x7f12340af420 2024-04-25T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12340b5220 0x7f12340d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12340b5220 0x7f12340d0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43434/0 (socket says 172.21.15.7:43434) 2024-04-25T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123a200640 1 -- 172.21.15.7:0/2957200168 learned_addr learned my addr 172.21.15.7:0/2957200168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123a200640 1 -- 172.21.15.7:0/2957200168 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12340af420 msgr2=0x7f12340cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123a200640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12340af420 0x7f12340cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123a200640 1 -- 172.21.15.7:0/2957200168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12340aed40 msgr2=0x7f1234151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f1239800640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12340aed40 0x7f1234151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123a200640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12340aed40 0x7f1234151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.353+0000 7f123a200640 1 -- 172.21.15.7:0/2957200168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1234155080 con 0x7f12340b5220 2024-04-25T11:11:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.354+0000 7f123a200640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12340b5220 0x7f12340d0320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f122c00cab0 tx=0x7f122c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.354+0000 7f1231600640 1 -- 172.21.15.7:0/2957200168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c017070 con 0x7f12340b5220 2024-04-25T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.354+0000 7f1231600640 1 -- 172.21.15.7:0/2957200168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122c00ac30 con 0x7f12340b5220 2024-04-25T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.354+0000 7f1231600640 1 -- 172.21.15.7:0/2957200168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c012400 con 0x7f12340b5220 2024-04-25T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.354+0000 7f123ac00640 1 -- 172.21.15.7:0/2957200168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1234155360 con 0x7f12340b5220 2024-04-25T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.354+0000 7f123ac00640 1 -- 172.21.15.7:0/2957200168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1234155920 con 0x7f12340b5220 2024-04-25T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.355+0000 7f123ac00640 1 -- 172.21.15.7:0/2957200168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1234154de0 con 0x7f12340b5220 2024-04-25T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.356+0000 7f1231600640 1 -- 172.21.15.7:0/2957200168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f122c01a070 con 0x7f12340b5220 2024-04-25T11:11:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.358+0000 7f1231600640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f12180744e0 0x7f12180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.358+0000 7f1231600640 1 -- 172.21.15.7:0/2957200168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f122c095330 con 0x7f12340b5220 2024-04-25T11:11:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.358+0000 7f1239800640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f12180744e0 0x7f12180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.359+0000 7f1239800640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f12180744e0 0x7f12180769f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1228000f60 tx=0x7f1228009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.363+0000 7f1231600640 1 -- 172.21.15.7:0/2957200168 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1234154de0 con 0x7f12340b5220 2024-04-25T11:11:02.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.559+0000 7f123ac00640 1 -- 172.21.15.7:0/2957200168 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f123409f3c0 con 0x7f12180744e0 2024-04-25T11:11:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.569+0000 7f1231600640 1 -- 172.21.15.7:0/2957200168 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332524 (secure 0 0 0) 0x7f12280050d0 con 0x7f12180744e0 2024-04-25T11:11:02.572 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:02.576 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:11:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.576+0000 7f123ac00640 1 -- 172.21.15.7:0/2957200168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f12180744e0 msgr2=0x7f12180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.576+0000 7f123ac00640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f12180744e0 0x7f12180769f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1228000f60 tx=0x7f1228009040 comp rx=0 tx=0).stop 2024-04-25T11:11:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.576+0000 7f123ac00640 1 -- 172.21.15.7:0/2957200168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12340b5220 msgr2=0x7f12340d0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.576+0000 7f123ac00640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12340b5220 0x7f12340d0320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f122c00cab0 tx=0x7f122c00cf70 comp rx=0 tx=0).stop 2024-04-25T11:11:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.576+0000 7f123ac00640 1 -- 172.21.15.7:0/2957200168 shutdown_connections 2024-04-25T11:11:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.577+0000 7f123ac00640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f12180744e0 0x7f12180769f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.577+0000 7f123ac00640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12340b5220 0x7f12340d0320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.577+0000 7f123ac00640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12340af420 0x7f12340cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.577+0000 7f123ac00640 1 --2- 172.21.15.7:0/2957200168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12340aed40 0x7f1234151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.577+0000 7f123ac00640 1 -- 172.21.15.7:0/2957200168 >> 172.21.15.7:0/2957200168 conn(0x7f12340b3820 msgr2=0x7f123415b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.577+0000 7f123ac00640 1 -- 172.21.15.7:0/2957200168 shutdown_connections 2024-04-25T11:11:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.577+0000 7f123ac00640 1 -- 172.21.15.7:0/2957200168 wait complete. 2024-04-25T11:11:02.601 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-04-25T11:11:02.020478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":150,"num_write_kb":1647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231960,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742760,"statfs":{"total":773094113280,"available":772856586240,"internally_reserved":0,"allocated":3383296,"data_stored":2025904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.230196"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895493+0000","last_change":"2024-04-25T11:10:22.475060+0000","last_active":"2024-04-25T11:10:29.895493+0000","last_peered":"2024-04-25T11:10:29.895493+0000","last_clean":"2024-04-25T11:10:29.895493+0000","last_became_active":"2024-04-25T11:10:22.472574+0000","last_became_peered":"2024-04-25T11:10:22.472574+0000","last_unstale":"2024-04-25T11:10:29.895493+0000","last_undegraded":"2024-04-25T11:10:29.895493+0000","last_fullsized":"2024-04-25T11:10:29.895493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.835819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895751+0000","last_change":"2024-04-25T11:10:23.488367+0000","last_active":"2024-04-25T11:10:29.895751+0000","last_peered":"2024-04-25T11:10:29.895751+0000","last_clean":"2024-04-25T11:10:29.895751+0000","last_became_active":"2024-04-25T11:10:23.488034+0000","last_became_peered":"2024-04-25T11:10:23.488034+0000","last_unstale":"2024-04-25T11:10:29.895751+0000","last_undegraded":"2024-04-25T11:10:29.895751+0000","last_fullsized":"2024-04-25T11:10:29.895751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892242+0000","last_change":"2024-04-25T11:10:27.551067+0000","last_active":"2024-04-25T11:10:29.892242+0000","last_peered":"2024-04-25T11:10:29.892242+0000","last_clean":"2024-04-25T11:10:29.892242+0000","last_became_active":"2024-04-25T11:10:27.550247+0000","last_became_peered":"2024-04-25T11:10:27.550247+0000","last_unstale":"2024-04-25T11:10:29.892242+0000","last_undegraded":"2024-04-25T11:10:29.892242+0000","last_fullsized":"2024-04-25T11:10:29.892242+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:29.432696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895898+0000","last_change":"2024-04-25T11:10:22.463507+0000","last_active":"2024-04-25T11:10:29.895898+0000","last_peered":"2024-04-25T11:10:29.895898+0000","last_clean":"2024-04-25T11:10:29.895898+0000","last_became_active":"2024-04-25T11:10:22.463142+0000","last_became_peered":"2024-04-25T11:10:22.463142+0000","last_unstale":"2024-04-25T11:10:29.895898+0000","last_undegraded":"2024-04-25T11:10:29.895898+0000","last_fullsized":"2024-04-25T11:10:29.895898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:42.000742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894569+0000","last_change":"2024-04-25T11:10:23.500843+0000","last_active":"2024-04-25T11:10:29.894569+0000","last_peered":"2024-04-25T11:10:29.894569+0000","last_clean":"2024-04-25T11:10:29.894569+0000","last_became_active":"2024-04-25T11:10:23.500412+0000","last_became_peered":"2024-04-25T11:10:23.500412+0000","last_unstale":"2024-04-25T11:10:29.894569+0000","last_undegraded":"2024-04-25T11:10:29.894569+0000","last_fullsized":"2024-04-25T11:10:29.894569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534853+0000","last_change":"2024-04-25T11:10:27.565431+0000","last_active":"2024-04-25T11:10:28.534853+0000","last_peered":"2024-04-25T11:10:28.534853+0000","last_clean":"2024-04-25T11:10:28.534853+0000","last_became_active":"2024-04-25T11:10:27.564724+0000","last_became_peered":"2024-04-25T11:10:27.564724+0000","last_unstale":"2024-04-25T11:10:28.534853+0000","last_undegraded":"2024-04-25T11:10:28.534853+0000","last_fullsized":"2024-04-25T11:10:28.534853+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:51.033161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536873+0000","last_change":"2024-04-25T11:10:22.477443+0000","last_active":"2024-04-25T11:10:28.536873+0000","last_peered":"2024-04-25T11:10:28.536873+0000","last_clean":"2024-04-25T11:10:28.536873+0000","last_became_active":"2024-04-25T11:10:22.477131+0000","last_became_peered":"2024-04-25T11:10:22.477131+0000","last_unstale":"2024-04-25T11:10:28.536873+0000","last_undegraded":"2024-04-25T11:10:28.536873+0000","last_fullsized":"2024-04-25T11:10:28.536873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:57.426313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291476+0000","last_change":"2024-04-25T11:10:23.496542+0000","last_active":"2024-04-25T11:10:29.291476+0000","last_peered":"2024-04-25T11:10:29.291476+0000","last_clean":"2024-04-25T11:10:29.291476+0000","last_became_active":"2024-04-25T11:10:23.496211+0000","last_became_peered":"2024-04-25T11:10:23.496211+0000","last_unstale":"2024-04-25T11:10:29.291476+0000","last_undegraded":"2024-04-25T11:10:29.291476+0000","last_fullsized":"2024-04-25T11:10:29.291476+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:15.460249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537266+0000","last_change":"2024-04-25T11:10:27.565126+0000","last_active":"2024-04-25T11:10:28.537266+0000","last_peered":"2024-04-25T11:10:28.537266+0000","last_clean":"2024-04-25T11:10:28.537266+0000","last_became_active":"2024-04-25T11:10:27.564664+0000","last_became_peered":"2024-04-25T11:10:27.564664+0000","last_unstale":"2024-04-25T11:10:28.537266+0000","last_undegraded":"2024-04-25T11:10:28.537266+0000","last_fullsized":"2024-04-25T11:10:28.537266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:44.604923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536194+0000","last_change":"2024-04-25T11:10:22.483531+0000","last_active":"2024-04-25T11:10:28.536194+0000","last_peered":"2024-04-25T11:10:28.536194+0000","last_clean":"2024-04-25T11:10:28.536194+0000","last_became_active":"2024-04-25T11:10:22.483204+0000","last_became_peered":"2024-04-25T11:10:22.483204+0000","last_unstale":"2024-04-25T11:10:28.536194+0000","last_undegraded":"2024-04-25T11:10:28.536194+0000","last_fullsized":"2024-04-25T11:10:28.536194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:03.866498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291328+0000","last_change":"2024-04-25T11:10:23.490550+0000","last_active":"2024-04-25T11:10:29.291328+0000","last_peered":"2024-04-25T11:10:29.291328+0000","last_clean":"2024-04-25T11:10:29.291328+0000","last_became_active":"2024-04-25T11:10:23.490300+0000","last_became_peered":"2024-04-25T11:10:23.490300+0000","last_unstale":"2024-04-25T11:10:29.291328+0000","last_undegraded":"2024-04-25T11:10:29.291328+0000","last_fullsized":"2024-04-25T11:10:29.291328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:20.100576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535567+0000","last_change":"2024-04-25T11:10:27.555808+0000","last_active":"2024-04-25T11:10:28.535567+0000","last_peered":"2024-04-25T11:10:28.535567+0000","last_clean":"2024-04-25T11:10:28.535567+0000","last_became_active":"2024-04-25T11:10:27.555277+0000","last_became_peered":"2024-04-25T11:10:27.555277+0000","last_unstale":"2024-04-25T11:10:28.535567+0000","last_undegraded":"2024-04-25T11:10:28.535567+0000","last_fullsized":"2024-04-25T11:10:28.535567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:52.665016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.890747+0000","last_change":"2024-04-25T11:10:23.493263+0000","last_active":"2024-04-25T11:10:29.890747+0000","last_peered":"2024-04-25T11:10:29.890747+0000","last_clean":"2024-04-25T11:10:29.890747+0000","last_became_active":"2024-04-25T11:10:23.489625+0000","last_became_peered":"2024-04-25T11:10:23.489625+0000","last_unstale":"2024-04-25T11:10:29.890747+0000","last_undegraded":"2024-04-25T11:10:29.890747+0000","last_fullsized":"2024-04-25T11:10:29.890747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535220+0000","last_change":"2024-04-25T11:10:22.475840+0000","last_active":"2024-04-25T11:10:28.535220+0000","last_peered":"2024-04-25T11:10:28.535220+0000","last_clean":"2024-04-25T11:10:28.535220+0000","last_became_active":"2024-04-25T11:10:22.475539+0000","last_became_peered":"2024-04-25T11:10:22.475539+0000","last_unstale":"2024-04-25T11:10:28.535220+0000","last_undegraded":"2024-04-25T11:10:28.535220+0000","last_fullsized":"2024-04-25T11:10:28.535220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:49.860961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538827+0000","last_change":"2024-04-25T11:10:27.558933+0000","last_active":"2024-04-25T11:10:28.538827+0000","last_peered":"2024-04-25T11:10:28.538827+0000","last_clean":"2024-04-25T11:10:28.538827+0000","last_became_active":"2024-04-25T11:10:27.558566+0000","last_became_peered":"2024-04-25T11:10:27.558566+0000","last_unstale":"2024-04-25T11:10:28.538827+0000","last_undegraded":"2024-04-25T11:10:28.538827+0000","last_fullsized":"2024-04-25T11:10:28.538827+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:45.815273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892352+0000","last_change":"2024-04-25T11:10:23.482991+0000","last_active":"2024-04-25T11:10:29.892352+0000","last_peered":"2024-04-25T11:10:29.892352+0000","last_clean":"2024-04-25T11:10:29.892352+0000","last_became_active":"2024-04-25T11:10:23.481972+0000","last_became_peered":"2024-04-25T11:10:23.481972+0000","last_unstale":"2024-04-25T11:10:29.892352+0000","last_undegraded":"2024-04-25T11:10:29.892352+0000","last_fullsized":"2024-04-25T11:10:29.892352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533468+0000","last_change":"2024-04-25T11:10:22.469445+0000","last_active":"2024-04-25T11:10:28.533468+0000","last_peered":"2024-04-25T11:10:28.533468+0000","last_clean":"2024-04-25T11:10:28.533468+0000","last_became_active":"2024-04-25T11:10:22.469095+0000","last_became_peered":"2024-04-25T11:10:22.469095+0000","last_unstale":"2024-04-25T11:10:28.533468+0000","last_undegraded":"2024-04-25T11:10:28.533468+0000","last_fullsized":"2024-04-25T11:10:28.533468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537528+0000","last_change":"2024-04-25T11:10:27.559144+0000","last_active":"2024-04-25T11:10:28.537528+0000","last_peered":"2024-04-25T11:10:28.537528+0000","last_clean":"2024-04-25T11:10:28.537528+0000","last_became_active":"2024-04-25T11:10:27.558635+0000","last_became_peered":"2024-04-25T11:10:27.558635+0000","last_unstale":"2024-04-25T11:10:28.537528+0000","last_undegraded":"2024-04-25T11:10:28.537528+0000","last_fullsized":"2024-04-25T11:10:28.537528+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:24.222645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892015+0000","last_change":"2024-04-25T11:10:23.460813+0000","last_active":"2024-04-25T11:10:29.892015+0000","last_peered":"2024-04-25T11:10:29.892015+0000","last_clean":"2024-04-25T11:10:29.892015+0000","last_became_active":"2024-04-25T11:10:23.460435+0000","last_became_peered":"2024-04-25T11:10:23.460435+0000","last_unstale":"2024-04-25T11:10:29.892015+0000","last_undegraded":"2024-04-25T11:10:29.892015+0000","last_fullsized":"2024-04-25T11:10:29.892015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531902+0000","last_change":"2024-04-25T11:10:22.466351+0000","last_active":"2024-04-25T11:10:28.531902+0000","last_peered":"2024-04-25T11:10:28.531902+0000","last_clean":"2024-04-25T11:10:28.531902+0000","last_became_active":"2024-04-25T11:10:22.464110+0000","last_became_peered":"2024-04-25T11:10:22.464110+0000","last_unstale":"2024-04-25T11:10:28.531902+0000","last_undegraded":"2024-04-25T11:10:28.531902+0000","last_fullsized":"2024-04-25T11:10:28.531902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534334+0000","last_change":"2024-04-25T11:10:27.555355+0000","last_active":"2024-04-25T11:10:28.534334+0000","last_peered":"2024-04-25T11:10:28.534334+0000","last_clean":"2024-04-25T11:10:28.534334+0000","last_became_active":"2024-04-25T11:10:27.554893+0000","last_became_peered":"2024-04-25T11:10:27.554893+0000","last_unstale":"2024-04-25T11:10:28.534334+0000","last_undegraded":"2024-04-25T11:10:28.534334+0000","last_fullsized":"2024-04-25T11:10:28.534334+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:51.174327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535363+0000","last_change":"2024-04-25T11:10:23.487680+0000","last_active":"2024-04-25T11:10:28.535363+0000","last_peered":"2024-04-25T11:10:28.535363+0000","last_clean":"2024-04-25T11:10:28.535363+0000","last_became_active":"2024-04-25T11:10:23.483024+0000","last_became_peered":"2024-04-25T11:10:23.483024+0000","last_unstale":"2024-04-25T11:10:28.535363+0000","last_undegraded":"2024-04-25T11:10:28.535363+0000","last_fullsized":"2024-04-25T11:10:28.535363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:06.246101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534055+0000","last_change":"2024-04-25T11:10:22.470867+0000","last_active":"2024-04-25T11:10:28.534055+0000","last_peered":"2024-04-25T11:10:28.534055+0000","last_clean":"2024-04-25T11:10:28.534055+0000","last_became_active":"2024-04-25T11:10:22.470501+0000","last_became_peered":"2024-04-25T11:10:22.470501+0000","last_unstale":"2024-04-25T11:10:28.534055+0000","last_undegraded":"2024-04-25T11:10:28.534055+0000","last_fullsized":"2024-04-25T11:10:28.534055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538997+0000","last_change":"2024-04-25T11:10:27.567703+0000","last_active":"2024-04-25T11:10:28.538997+0000","last_peered":"2024-04-25T11:10:28.538997+0000","last_clean":"2024-04-25T11:10:28.538997+0000","last_became_active":"2024-04-25T11:10:27.567208+0000","last_became_peered":"2024-04-25T11:10:27.567208+0000","last_unstale":"2024-04-25T11:10:28.538997+0000","last_undegraded":"2024-04-25T11:10:28.538997+0000","last_fullsized":"2024-04-25T11:10:28.538997+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:24.170044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895262+0000","last_change":"2024-04-25T11:10:23.488689+0000","last_active":"2024-04-25T11:10:29.895262+0000","last_peered":"2024-04-25T11:10:29.895262+0000","last_clean":"2024-04-25T11:10:29.895262+0000","last_became_active":"2024-04-25T11:10:23.479256+0000","last_became_peered":"2024-04-25T11:10:23.479256+0000","last_unstale":"2024-04-25T11:10:29.895262+0000","last_undegraded":"2024-04-25T11:10:29.895262+0000","last_fullsized":"2024-04-25T11:10:29.895262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892311+0000","last_change":"2024-04-25T11:10:22.486332+0000","last_active":"2024-04-25T11:10:29.892311+0000","last_peered":"2024-04-25T11:10:29.892311+0000","last_clean":"2024-04-25T11:10:29.892311+0000","last_became_active":"2024-04-25T11:10:22.485970+0000","last_became_peered":"2024-04-25T11:10:22.485970+0000","last_unstale":"2024-04-25T11:10:29.892311+0000","last_undegraded":"2024-04-25T11:10:29.892311+0000","last_fullsized":"2024-04-25T11:10:29.892311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534779+0000","last_change":"2024-04-25T11:10:27.549543+0000","last_active":"2024-04-25T11:10:28.534779+0000","last_peered":"2024-04-25T11:10:28.534779+0000","last_clean":"2024-04-25T11:10:28.534779+0000","last_became_active":"2024-04-25T11:10:27.549007+0000","last_became_peered":"2024-04-25T11:10:27.549007+0000","last_unstale":"2024-04-25T11:10:28.534779+0000","last_undegraded":"2024-04-25T11:10:28.534779+0000","last_fullsized":"2024-04-25T11:10:28.534779+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:01.381820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896311+0000","last_change":"2024-04-25T11:10:23.465612+0000","last_active":"2024-04-25T11:10:29.896311+0000","last_peered":"2024-04-25T11:10:29.896311+0000","last_clean":"2024-04-25T11:10:29.896311+0000","last_became_active":"2024-04-25T11:10:23.465261+0000","last_became_peered":"2024-04-25T11:10:23.465261+0000","last_unstale":"2024-04-25T11:10:29.896311+0000","last_undegraded":"2024-04-25T11:10:29.896311+0000","last_fullsized":"2024-04-25T11:10:29.896311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534173+0000","last_change":"2024-04-25T11:10:22.483679+0000","last_active":"2024-04-25T11:10:28.534173+0000","last_peered":"2024-04-25T11:10:28.534173+0000","last_clean":"2024-04-25T11:10:28.534173+0000","last_became_active":"2024-04-25T11:10:22.483243+0000","last_became_peered":"2024-04-25T11:10:22.483243+0000","last_unstale":"2024-04-25T11:10:28.534173+0000","last_undegraded":"2024-04-25T11:10:28.534173+0000","last_fullsized":"2024-04-25T11:10:28.534173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:49.860961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891407+0000","last_change":"2024-04-25T11:10:27.550030+0000","last_active":"2024-04-25T11:10:29.891407+0000","last_peered":"2024-04-25T11:10:29.891407+0000","last_clean":"2024-04-25T11:10:29.891407+0000","last_became_active":"2024-04-25T11:10:27.549537+0000","last_became_peered":"2024-04-25T11:10:27.549537+0000","last_unstale":"2024-04-25T11:10:29.891407+0000","last_undegraded":"2024-04-25T11:10:29.891407+0000","last_fullsized":"2024-04-25T11:10:29.891407+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:51.538673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534183+0000","last_change":"2024-04-25T11:10:23.470926+0000","last_active":"2024-04-25T11:10:28.534183+0000","last_peered":"2024-04-25T11:10:28.534183+0000","last_clean":"2024-04-25T11:10:28.534183+0000","last_became_active":"2024-04-25T11:10:23.470483+0000","last_became_peered":"2024-04-25T11:10:23.470483+0000","last_unstale":"2024-04-25T11:10:28.534183+0000","last_undegraded":"2024-04-25T11:10:28.534183+0000","last_fullsized":"2024-04-25T11:10:28.534183+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537196+0000","last_change":"2024-04-25T11:10:22.479703+0000","last_active":"2024-04-25T11:10:28.537196+0000","last_peered":"2024-04-25T11:10:28.537196+0000","last_clean":"2024-04-25T11:10:28.537196+0000","last_became_active":"2024-04-25T11:10:22.478929+0000","last_became_peered":"2024-04-25T11:10:22.478929+0000","last_unstale":"2024-04-25T11:10:28.537196+0000","last_undegraded":"2024-04-25T11:10:28.537196+0000","last_fullsized":"2024-04-25T11:10:28.537196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:08.833540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893409+0000","last_change":"2024-04-25T11:10:27.559919+0000","last_active":"2024-04-25T11:10:29.893409+0000","last_peered":"2024-04-25T11:10:29.893409+0000","last_clean":"2024-04-25T11:10:29.893409+0000","last_became_active":"2024-04-25T11:10:27.558805+0000","last_became_peered":"2024-04-25T11:10:27.558805+0000","last_unstale":"2024-04-25T11:10:29.893409+0000","last_undegraded":"2024-04-25T11:10:29.893409+0000","last_fullsized":"2024-04-25T11:10:29.893409+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291634+0000","last_change":"2024-04-25T11:10:23.471738+0000","last_active":"2024-04-25T11:10:29.291634+0000","last_peered":"2024-04-25T11:10:29.291634+0000","last_clean":"2024-04-25T11:10:29.291634+0000","last_became_active":"2024-04-25T11:10:23.471191+0000","last_became_peered":"2024-04-25T11:10:23.471191+0000","last_unstale":"2024-04-25T11:10:29.291634+0000","last_undegraded":"2024-04-25T11:10:29.291634+0000","last_fullsized":"2024-04-25T11:10:29.291634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:47.901333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534371+0000","last_change":"2024-04-25T11:10:22.467322+0000","last_active":"2024-04-25T11:10:28.534371+0000","last_peered":"2024-04-25T11:10:28.534371+0000","last_clean":"2024-04-25T11:10:28.534371+0000","last_became_active":"2024-04-25T11:10:22.466806+0000","last_became_peered":"2024-04-25T11:10:22.466806+0000","last_unstale":"2024-04-25T11:10:28.534371+0000","last_undegraded":"2024-04-25T11:10:28.534371+0000","last_fullsized":"2024-04-25T11:10:28.534371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533236+0000","last_change":"2024-04-25T11:10:27.546558+0000","last_active":"2024-04-25T11:10:28.533236+0000","last_peered":"2024-04-25T11:10:28.533236+0000","last_clean":"2024-04-25T11:10:28.533236+0000","last_became_active":"2024-04-25T11:10:27.544215+0000","last_became_peered":"2024-04-25T11:10:27.544215+0000","last_unstale":"2024-04-25T11:10:28.533236+0000","last_undegraded":"2024-04-25T11:10:28.533236+0000","last_fullsized":"2024-04-25T11:10:28.533236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.970801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537205+0000","last_change":"2024-04-25T11:10:23.489986+0000","last_active":"2024-04-25T11:10:28.537205+0000","last_peered":"2024-04-25T11:10:28.537205+0000","last_clean":"2024-04-25T11:10:28.537205+0000","last_became_active":"2024-04-25T11:10:23.489629+0000","last_became_peered":"2024-04-25T11:10:23.489629+0000","last_unstale":"2024-04-25T11:10:28.537205+0000","last_undegraded":"2024-04-25T11:10:28.537205+0000","last_fullsized":"2024-04-25T11:10:28.537205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:14.555604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291125+0000","last_change":"2024-04-25T11:10:22.482966+0000","last_active":"2024-04-25T11:10:29.291125+0000","last_peered":"2024-04-25T11:10:29.291125+0000","last_clean":"2024-04-25T11:10:29.291125+0000","last_became_active":"2024-04-25T11:10:22.480437+0000","last_became_peered":"2024-04-25T11:10:22.480437+0000","last_unstale":"2024-04-25T11:10:29.291125+0000","last_undegraded":"2024-04-25T11:10:29.291125+0000","last_fullsized":"2024-04-25T11:10:29.291125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534554+0000","last_change":"2024-04-25T11:10:27.549617+0000","last_active":"2024-04-25T11:10:28.534554+0000","last_peered":"2024-04-25T11:10:28.534554+0000","last_clean":"2024-04-25T11:10:28.534554+0000","last_became_active":"2024-04-25T11:10:27.549203+0000","last_became_peered":"2024-04-25T11:10:27.549203+0000","last_unstale":"2024-04-25T11:10:28.534554+0000","last_undegraded":"2024-04-25T11:10:28.534554+0000","last_fullsized":"2024-04-25T11:10:28.534554+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:55.043679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531961+0000","last_change":"2024-04-25T11:10:23.472089+0000","last_active":"2024-04-25T11:10:28.531961+0000","last_peered":"2024-04-25T11:10:28.531961+0000","last_clean":"2024-04-25T11:10:28.531961+0000","last_became_active":"2024-04-25T11:10:23.468460+0000","last_became_peered":"2024-04-25T11:10:23.468460+0000","last_unstale":"2024-04-25T11:10:28.531961+0000","last_undegraded":"2024-04-25T11:10:28.531961+0000","last_fullsized":"2024-04-25T11:10:28.531961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373356+0000","last_change":"2024-04-25T11:10:22.468201+0000","last_active":"2024-04-25T11:10:29.373356+0000","last_peered":"2024-04-25T11:10:29.373356+0000","last_clean":"2024-04-25T11:10:29.373356+0000","last_became_active":"2024-04-25T11:10:22.467845+0000","last_became_peered":"2024-04-25T11:10:22.467845+0000","last_unstale":"2024-04-25T11:10:29.373356+0000","last_undegraded":"2024-04-25T11:10:29.373356+0000","last_fullsized":"2024-04-25T11:10:29.373356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:50.983943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534837+0000","last_change":"2024-04-25T11:10:27.552725+0000","last_active":"2024-04-25T11:10:28.534837+0000","last_peered":"2024-04-25T11:10:28.534837+0000","last_clean":"2024-04-25T11:10:28.534837+0000","last_became_active":"2024-04-25T11:10:27.552073+0000","last_became_peered":"2024-04-25T11:10:27.552073+0000","last_unstale":"2024-04-25T11:10:28.534837+0000","last_undegraded":"2024-04-25T11:10:28.534837+0000","last_fullsized":"2024-04-25T11:10:28.534837+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.402499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893257+0000","last_change":"2024-04-25T11:10:23.492738+0000","last_active":"2024-04-25T11:10:29.893257+0000","last_peered":"2024-04-25T11:10:29.893257+0000","last_clean":"2024-04-25T11:10:29.893257+0000","last_became_active":"2024-04-25T11:10:23.492416+0000","last_became_peered":"2024-04-25T11:10:23.492416+0000","last_unstale":"2024-04-25T11:10:29.893257+0000","last_undegraded":"2024-04-25T11:10:29.893257+0000","last_fullsized":"2024-04-25T11:10:29.893257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:00.331944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536183+0000","last_change":"2024-04-25T11:10:22.478976+0000","last_active":"2024-04-25T11:10:28.536183+0000","last_peered":"2024-04-25T11:10:28.536183+0000","last_clean":"2024-04-25T11:10:28.536183+0000","last_became_active":"2024-04-25T11:10:22.478595+0000","last_became_peered":"2024-04-25T11:10:22.478595+0000","last_unstale":"2024-04-25T11:10:28.536183+0000","last_undegraded":"2024-04-25T11:10:28.536183+0000","last_fullsized":"2024-04-25T11:10:28.536183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895071+0000","last_change":"2024-04-25T11:10:27.558873+0000","last_active":"2024-04-25T11:10:29.895071+0000","last_peered":"2024-04-25T11:10:29.895071+0000","last_clean":"2024-04-25T11:10:29.895071+0000","last_became_active":"2024-04-25T11:10:27.557790+0000","last_became_peered":"2024-04-25T11:10:27.557790+0000","last_unstale":"2024-04-25T11:10:29.895071+0000","last_undegraded":"2024-04-25T11:10:29.895071+0000","last_fullsized":"2024-04-25T11:10:29.895071+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:17.402499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893187+0000","last_change":"2024-04-25T11:10:23.470011+0000","last_active":"2024-04-25T11:10:29.893187+0000","last_peered":"2024-04-25T11:10:29.893187+0000","last_clean":"2024-04-25T11:10:29.893187+0000","last_became_active":"2024-04-25T11:10:23.469655+0000","last_became_peered":"2024-04-25T11:10:23.469655+0000","last_unstale":"2024-04-25T11:10:29.893187+0000","last_undegraded":"2024-04-25T11:10:29.893187+0000","last_fullsized":"2024-04-25T11:10:29.893187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291689+0000","last_change":"2024-04-25T11:10:22.480507+0000","last_active":"2024-04-25T11:10:29.291689+0000","last_peered":"2024-04-25T11:10:29.291689+0000","last_clean":"2024-04-25T11:10:29.291689+0000","last_became_active":"2024-04-25T11:10:22.479568+0000","last_became_peered":"2024-04-25T11:10:22.479568+0000","last_unstale":"2024-04-25T11:10:29.291689+0000","last_undegraded":"2024-04-25T11:10:29.291689+0000","last_fullsized":"2024-04-25T11:10:29.291689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":118,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536490+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:10:28.536490+0000","last_peered":"2024-04-25T11:10:28.536490+0000","last_clean":"2024-04-25T11:10:28.536490+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:10:28.536490+0000","last_undegraded":"2024-04-25T11:10:28.536490+0000","last_fullsized":"2024-04-25T11:10:28.536490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891956+0000","last_change":"2024-04-25T11:10:27.551611+0000","last_active":"2024-04-25T11:10:29.891956+0000","last_peered":"2024-04-25T11:10:29.891956+0000","last_clean":"2024-04-25T11:10:29.891956+0000","last_became_active":"2024-04-25T11:10:27.551117+0000","last_became_peered":"2024-04-25T11:10:27.551117+0000","last_unstale":"2024-04-25T11:10:29.891956+0000","last_undegraded":"2024-04-25T11:10:29.891956+0000","last_fullsized":"2024-04-25T11:10:29.891956+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536555+0000","last_change":"2024-04-25T11:10:23.469975+0000","last_active":"2024-04-25T11:10:28.536555+0000","last_peered":"2024-04-25T11:10:28.536555+0000","last_clean":"2024-04-25T11:10:28.536555+0000","last_became_active":"2024-04-25T11:10:23.469366+0000","last_became_peered":"2024-04-25T11:10:23.469366+0000","last_unstale":"2024-04-25T11:10:28.536555+0000","last_undegraded":"2024-04-25T11:10:28.536555+0000","last_fullsized":"2024-04-25T11:10:28.536555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:47.901333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373786+0000","last_change":"2024-04-25T11:10:22.480954+0000","last_active":"2024-04-25T11:10:29.373786+0000","last_peered":"2024-04-25T11:10:29.373786+0000","last_clean":"2024-04-25T11:10:29.373786+0000","last_became_active":"2024-04-25T11:10:22.480489+0000","last_became_peered":"2024-04-25T11:10:22.480489+0000","last_unstale":"2024-04-25T11:10:29.373786+0000","last_undegraded":"2024-04-25T11:10:29.373786+0000","last_fullsized":"2024-04-25T11:10:29.373786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:11.295052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891284+0000","last_change":"2024-04-25T11:10:27.548113+0000","last_active":"2024-04-25T11:10:29.891284+0000","last_peered":"2024-04-25T11:10:29.891284+0000","last_clean":"2024-04-25T11:10:29.891284+0000","last_became_active":"2024-04-25T11:10:27.547579+0000","last_became_peered":"2024-04-25T11:10:27.547579+0000","last_unstale":"2024-04-25T11:10:29.891284+0000","last_undegraded":"2024-04-25T11:10:29.891284+0000","last_fullsized":"2024-04-25T11:10:29.891284+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:45.815273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291802+0000","last_change":"2024-04-25T11:10:23.478738+0000","last_active":"2024-04-25T11:10:29.291802+0000","last_peered":"2024-04-25T11:10:29.291802+0000","last_clean":"2024-04-25T11:10:29.291802+0000","last_became_active":"2024-04-25T11:10:23.468486+0000","last_became_peered":"2024-04-25T11:10:23.468486+0000","last_unstale":"2024-04-25T11:10:29.291802+0000","last_undegraded":"2024-04-25T11:10:29.291802+0000","last_fullsized":"2024-04-25T11:10:29.291802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:47.104859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533727+0000","last_change":"2024-04-25T11:10:22.473530+0000","last_active":"2024-04-25T11:10:28.533727+0000","last_peered":"2024-04-25T11:10:28.533727+0000","last_clean":"2024-04-25T11:10:28.533727+0000","last_became_active":"2024-04-25T11:10:22.473177+0000","last_became_peered":"2024-04-25T11:10:22.473177+0000","last_unstale":"2024-04-25T11:10:28.533727+0000","last_undegraded":"2024-04-25T11:10:28.533727+0000","last_fullsized":"2024-04-25T11:10:28.533727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894250+0000","last_change":"2024-04-25T11:10:27.559345+0000","last_active":"2024-04-25T11:10:29.894250+0000","last_peered":"2024-04-25T11:10:29.894250+0000","last_clean":"2024-04-25T11:10:29.894250+0000","last_became_active":"2024-04-25T11:10:27.558090+0000","last_became_peered":"2024-04-25T11:10:27.558090+0000","last_unstale":"2024-04-25T11:10:29.894250+0000","last_undegraded":"2024-04-25T11:10:29.894250+0000","last_fullsized":"2024-04-25T11:10:29.894250+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:51.174327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538883+0000","last_change":"2024-04-25T11:10:23.477613+0000","last_active":"2024-04-25T11:10:28.538883+0000","last_peered":"2024-04-25T11:10:28.538883+0000","last_clean":"2024-04-25T11:10:28.538883+0000","last_became_active":"2024-04-25T11:10:23.477226+0000","last_became_peered":"2024-04-25T11:10:23.477226+0000","last_unstale":"2024-04-25T11:10:28.538883+0000","last_undegraded":"2024-04-25T11:10:28.538883+0000","last_fullsized":"2024-04-25T11:10:28.538883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:07.701501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.374436+0000","last_change":"2024-04-25T11:10:22.468389+0000","last_active":"2024-04-25T11:10:29.374436+0000","last_peered":"2024-04-25T11:10:29.374436+0000","last_clean":"2024-04-25T11:10:29.374436+0000","last_became_active":"2024-04-25T11:10:22.467926+0000","last_became_peered":"2024-04-25T11:10:22.467926+0000","last_unstale":"2024-04-25T11:10:29.374436+0000","last_undegraded":"2024-04-25T11:10:29.374436+0000","last_fullsized":"2024-04-25T11:10:29.374436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:42.448477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290385+0000","last_change":"2024-04-25T11:10:27.547813+0000","last_active":"2024-04-25T11:10:29.290385+0000","last_peered":"2024-04-25T11:10:29.290385+0000","last_clean":"2024-04-25T11:10:29.290385+0000","last_became_active":"2024-04-25T11:10:27.547300+0000","last_became_peered":"2024-04-25T11:10:27.547300+0000","last_unstale":"2024-04-25T11:10:29.290385+0000","last_undegraded":"2024-04-25T11:10:29.290385+0000","last_fullsized":"2024-04-25T11:10:29.290385+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:51.538673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537521+0000","last_change":"2024-04-25T11:10:23.499736+0000","last_active":"2024-04-25T11:10:28.537521+0000","last_peered":"2024-04-25T11:10:28.537521+0000","last_clean":"2024-04-25T11:10:28.537521+0000","last_became_active":"2024-04-25T11:10:23.491282+0000","last_became_peered":"2024-04-25T11:10:23.491282+0000","last_unstale":"2024-04-25T11:10:28.537521+0000","last_undegraded":"2024-04-25T11:10:28.537521+0000","last_fullsized":"2024-04-25T11:10:28.537521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:15.460249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.539367+0000","last_change":"2024-04-25T11:10:22.477761+0000","last_active":"2024-04-25T11:10:28.539367+0000","last_peered":"2024-04-25T11:10:28.539367+0000","last_clean":"2024-04-25T11:10:28.539367+0000","last_became_active":"2024-04-25T11:10:22.477390+0000","last_became_peered":"2024-04-25T11:10:22.477390+0000","last_unstale":"2024-04-25T11:10:28.539367+0000","last_undegraded":"2024-04-25T11:10:28.539367+0000","last_fullsized":"2024-04-25T11:10:28.539367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:48.408404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535808+0000","last_change":"2024-04-25T11:10:27.559907+0000","last_active":"2024-04-25T11:10:28.535808+0000","last_peered":"2024-04-25T11:10:28.535808+0000","last_clean":"2024-04-25T11:10:28.535808+0000","last_became_active":"2024-04-25T11:10:27.559437+0000","last_became_peered":"2024-04-25T11:10:27.559437+0000","last_unstale":"2024-04-25T11:10:28.535808+0000","last_undegraded":"2024-04-25T11:10:28.535808+0000","last_fullsized":"2024-04-25T11:10:28.535808+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:14.067687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535432+0000","last_change":"2024-04-25T11:10:23.479907+0000","last_active":"2024-04-25T11:10:28.535432+0000","last_peered":"2024-04-25T11:10:28.535432+0000","last_clean":"2024-04-25T11:10:28.535432+0000","last_became_active":"2024-04-25T11:10:23.479578+0000","last_became_peered":"2024-04-25T11:10:23.479578+0000","last_unstale":"2024-04-25T11:10:28.535432+0000","last_undegraded":"2024-04-25T11:10:28.535432+0000","last_fullsized":"2024-04-25T11:10:28.535432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":30,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373126+0000","last_change":"2024-04-25T11:10:22.467360+0000","last_active":"2024-04-25T11:10:29.373126+0000","last_peered":"2024-04-25T11:10:29.373126+0000","last_clean":"2024-04-25T11:10:29.373126+0000","last_became_active":"2024-04-25T11:10:22.467002+0000","last_became_peered":"2024-04-25T11:10:22.467002+0000","last_unstale":"2024-04-25T11:10:29.373126+0000","last_undegraded":"2024-04-25T11:10:29.373126+0000","last_fullsized":"2024-04-25T11:10:29.373126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:17.490319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894874+0000","last_change":"2024-04-25T11:10:27.566981+0000","last_active":"2024-04-25T11:10:29.894874+0000","last_peered":"2024-04-25T11:10:29.894874+0000","last_clean":"2024-04-25T11:10:29.894874+0000","last_became_active":"2024-04-25T11:10:27.566434+0000","last_became_peered":"2024-04-25T11:10:27.566434+0000","last_unstale":"2024-04-25T11:10:29.894874+0000","last_undegraded":"2024-04-25T11:10:29.894874+0000","last_fullsized":"2024-04-25T11:10:29.894874+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:55.043679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891408+0000","last_change":"2024-04-25T11:10:23.489031+0000","last_active":"2024-04-25T11:10:29.891408+0000","last_peered":"2024-04-25T11:10:29.891408+0000","last_clean":"2024-04-25T11:10:29.891408+0000","last_became_active":"2024-04-25T11:10:23.487804+0000","last_became_peered":"2024-04-25T11:10:23.487804+0000","last_unstale":"2024-04-25T11:10:29.891408+0000","last_undegraded":"2024-04-25T11:10:29.891408+0000","last_fullsized":"2024-04-25T11:10:29.891408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891229+0000","last_change":"2024-04-25T11:10:22.489707+0000","last_active":"2024-04-25T11:10:29.891229+0000","last_peered":"2024-04-25T11:10:29.891229+0000","last_clean":"2024-04-25T11:10:29.891229+0000","last_became_active":"2024-04-25T11:10:22.489324+0000","last_became_peered":"2024-04-25T11:10:22.489324+0000","last_unstale":"2024-04-25T11:10:29.891229+0000","last_undegraded":"2024-04-25T11:10:29.891229+0000","last_fullsized":"2024-04-25T11:10:29.891229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:55.540255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896355+0000","last_change":"2024-04-25T11:10:27.556359+0000","last_active":"2024-04-25T11:10:29.896355+0000","last_peered":"2024-04-25T11:10:29.896355+0000","last_clean":"2024-04-25T11:10:29.896355+0000","last_became_active":"2024-04-25T11:10:27.555528+0000","last_became_peered":"2024-04-25T11:10:27.555528+0000","last_unstale":"2024-04-25T11:10:29.896355+0000","last_undegraded":"2024-04-25T11:10:29.896355+0000","last_fullsized":"2024-04-25T11:10:29.896355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:35.346179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535152+0000","last_change":"2024-04-25T11:10:23.478210+0000","last_active":"2024-04-25T11:10:28.535152+0000","last_peered":"2024-04-25T11:10:28.535152+0000","last_clean":"2024-04-25T11:10:28.535152+0000","last_became_active":"2024-04-25T11:10:23.477839+0000","last_became_peered":"2024-04-25T11:10:23.477839+0000","last_unstale":"2024-04-25T11:10:28.535152+0000","last_undegraded":"2024-04-25T11:10:28.535152+0000","last_fullsized":"2024-04-25T11:10:28.535152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:41.745805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290930+0000","last_change":"2024-04-25T11:10:22.487737+0000","last_active":"2024-04-25T11:10:29.290930+0000","last_peered":"2024-04-25T11:10:29.290930+0000","last_clean":"2024-04-25T11:10:29.290930+0000","last_became_active":"2024-04-25T11:10:22.485337+0000","last_became_peered":"2024-04-25T11:10:22.485337+0000","last_unstale":"2024-04-25T11:10:29.290930+0000","last_undegraded":"2024-04-25T11:10:29.290930+0000","last_fullsized":"2024-04-25T11:10:29.290930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894290+0000","last_change":"2024-04-25T11:10:27.562988+0000","last_active":"2024-04-25T11:10:29.894290+0000","last_peered":"2024-04-25T11:10:29.894290+0000","last_clean":"2024-04-25T11:10:29.894290+0000","last_became_active":"2024-04-25T11:10:27.562511+0000","last_became_peered":"2024-04-25T11:10:27.562511+0000","last_unstale":"2024-04-25T11:10:29.894290+0000","last_undegraded":"2024-04-25T11:10:29.894290+0000","last_fullsized":"2024-04-25T11:10:29.894290+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:32.363124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893441+0000","last_change":"2024-04-25T11:10:23.480645+0000","last_active":"2024-04-25T11:10:29.893441+0000","last_peered":"2024-04-25T11:10:29.893441+0000","last_clean":"2024-04-25T11:10:29.893441+0000","last_became_active":"2024-04-25T11:10:23.480244+0000","last_became_peered":"2024-04-25T11:10:23.480244+0000","last_unstale":"2024-04-25T11:10:29.893441+0000","last_undegraded":"2024-04-25T11:10:29.893441+0000","last_fullsized":"2024-04-25T11:10:29.893441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:46.490227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894602+0000","last_change":"2024-04-25T11:10:22.479455+0000","last_active":"2024-04-25T11:10:29.894602+0000","last_peered":"2024-04-25T11:10:29.894602+0000","last_clean":"2024-04-25T11:10:29.894602+0000","last_became_active":"2024-04-25T11:10:22.479024+0000","last_became_peered":"2024-04-25T11:10:22.479024+0000","last_unstale":"2024-04-25T11:10:29.894602+0000","last_undegraded":"2024-04-25T11:10:29.894602+0000","last_fullsized":"2024-04-25T11:10:29.894602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:56.918083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291635+0000","last_change":"2024-04-25T11:10:27.549545+0000","last_active":"2024-04-25T11:10:29.291635+0000","last_peered":"2024-04-25T11:10:29.291635+0000","last_clean":"2024-04-25T11:10:29.291635+0000","last_became_active":"2024-04-25T11:10:27.549085+0000","last_became_peered":"2024-04-25T11:10:27.549085+0000","last_unstale":"2024-04-25T11:10:29.291635+0000","last_undegraded":"2024-04-25T11:10:29.291635+0000","last_fullsized":"2024-04-25T11:10:29.291635+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.906264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536502+0000","last_change":"2024-04-25T11:10:23.472355+0000","last_active":"2024-04-25T11:10:28.536502+0000","last_peered":"2024-04-25T11:10:28.536502+0000","last_clean":"2024-04-25T11:10:28.536502+0000","last_became_active":"2024-04-25T11:10:23.470905+0000","last_became_peered":"2024-04-25T11:10:23.470905+0000","last_unstale":"2024-04-25T11:10:28.536502+0000","last_undegraded":"2024-04-25T11:10:28.536502+0000","last_fullsized":"2024-04-25T11:10:28.536502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:20.100576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:44.462567+0000","last_change":"2024-04-25T11:10:22.481299+0000","last_active":"2024-04-25T11:10:44.462567+0000","last_peered":"2024-04-25T11:10:44.462567+0000","last_clean":"2024-04-25T11:10:44.462567+0000","last_became_active":"2024-04-25T11:10:22.480741+0000","last_became_peered":"2024-04-25T11:10:22.480741+0000","last_unstale":"2024-04-25T11:10:44.462567+0000","last_undegraded":"2024-04-25T11:10:44.462567+0000","last_fullsized":"2024-04-25T11:10:44.462567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533006+0000","last_change":"2024-04-25T11:10:27.551729+0000","last_active":"2024-04-25T11:10:28.533006+0000","last_peered":"2024-04-25T11:10:28.533006+0000","last_clean":"2024-04-25T11:10:28.533006+0000","last_became_active":"2024-04-25T11:10:27.548417+0000","last_became_peered":"2024-04-25T11:10:27.548417+0000","last_unstale":"2024-04-25T11:10:28.533006+0000","last_undegraded":"2024-04-25T11:10:28.533006+0000","last_fullsized":"2024-04-25T11:10:28.533006+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:19.529717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538108+0000","last_change":"2024-04-25T11:10:23.482351+0000","last_active":"2024-04-25T11:10:28.538108+0000","last_peered":"2024-04-25T11:10:28.538108+0000","last_clean":"2024-04-25T11:10:28.538108+0000","last_became_active":"2024-04-25T11:10:23.481665+0000","last_became_peered":"2024-04-25T11:10:23.481665+0000","last_unstale":"2024-04-25T11:10:28.538108+0000","last_undegraded":"2024-04-25T11:10:28.538108+0000","last_fullsized":"2024-04-25T11:10:28.538108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:50.877149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533327+0000","last_change":"2024-04-25T11:10:22.484130+0000","last_active":"2024-04-25T11:10:28.533327+0000","last_peered":"2024-04-25T11:10:28.533327+0000","last_clean":"2024-04-25T11:10:28.533327+0000","last_became_active":"2024-04-25T11:10:22.483811+0000","last_became_peered":"2024-04-25T11:10:22.483811+0000","last_unstale":"2024-04-25T11:10:28.533327+0000","last_undegraded":"2024-04-25T11:10:28.533327+0000","last_fullsized":"2024-04-25T11:10:28.533327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:05.229913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535359+0000","last_change":"2024-04-25T11:10:27.564379+0000","last_active":"2024-04-25T11:10:28.535359+0000","last_peered":"2024-04-25T11:10:28.535359+0000","last_clean":"2024-04-25T11:10:28.535359+0000","last_became_active":"2024-04-25T11:10:27.563892+0000","last_became_peered":"2024-04-25T11:10:27.563892+0000","last_unstale":"2024-04-25T11:10:28.535359+0000","last_undegraded":"2024-04-25T11:10:28.535359+0000","last_fullsized":"2024-04-25T11:10:28.535359+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:06.999062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895182+0000","last_change":"2024-04-25T11:10:23.487092+0000","last_active":"2024-04-25T11:10:29.895182+0000","last_peered":"2024-04-25T11:10:29.895182+0000","last_clean":"2024-04-25T11:10:29.895182+0000","last_became_active":"2024-04-25T11:10:23.486583+0000","last_became_peered":"2024-04-25T11:10:23.486583+0000","last_unstale":"2024-04-25T11:10:29.895182+0000","last_undegraded":"2024-04-25T11:10:29.895182+0000","last_fullsized":"2024-04-25T11:10:29.895182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291509+0000","last_change":"2024-04-25T11:10:22.484333+0000","last_active":"2024-04-25T11:10:29.291509+0000","last_peered":"2024-04-25T11:10:29.291509+0000","last_clean":"2024-04-25T11:10:29.291509+0000","last_became_active":"2024-04-25T11:10:22.483865+0000","last_became_peered":"2024-04-25T11:10:22.483865+0000","last_unstale":"2024-04-25T11:10:29.291509+0000","last_undegraded":"2024-04-25T11:10:29.291509+0000","last_fullsized":"2024-04-25T11:10:29.291509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:29.189395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891783+0000","last_change":"2024-04-25T11:10:27.551066+0000","last_active":"2024-04-25T11:10:29.891783+0000","last_peered":"2024-04-25T11:10:29.891783+0000","last_clean":"2024-04-25T11:10:29.891783+0000","last_became_active":"2024-04-25T11:10:27.550625+0000","last_became_peered":"2024-04-25T11:10:27.550625+0000","last_unstale":"2024-04-25T11:10:29.891783+0000","last_undegraded":"2024-04-25T11:10:29.891783+0000","last_fullsized":"2024-04-25T11:10:29.891783+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:03.906264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291447+0000","last_change":"2024-04-25T11:10:23.485935+0000","last_active":"2024-04-25T11:10:29.291447+0000","last_peered":"2024-04-25T11:10:29.291447+0000","last_clean":"2024-04-25T11:10:29.291447+0000","last_became_active":"2024-04-25T11:10:23.485367+0000","last_became_peered":"2024-04-25T11:10:23.485367+0000","last_unstale":"2024-04-25T11:10:29.291447+0000","last_undegraded":"2024-04-25T11:10:29.291447+0000","last_fullsized":"2024-04-25T11:10:29.291447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:50.974212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892031+0000","last_change":"2024-04-25T11:10:22.482686+0000","last_active":"2024-04-25T11:10:29.892031+0000","last_peered":"2024-04-25T11:10:29.892031+0000","last_clean":"2024-04-25T11:10:29.892031+0000","last_became_active":"2024-04-25T11:10:22.482307+0000","last_became_peered":"2024-04-25T11:10:22.482307+0000","last_unstale":"2024-04-25T11:10:29.892031+0000","last_undegraded":"2024-04-25T11:10:29.892031+0000","last_fullsized":"2024-04-25T11:10:29.892031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.539416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892597+0000","last_change":"2024-04-25T11:10:27.563942+0000","last_active":"2024-04-25T11:10:29.892597+0000","last_peered":"2024-04-25T11:10:29.892597+0000","last_clean":"2024-04-25T11:10:29.892597+0000","last_became_active":"2024-04-25T11:10:27.561604+0000","last_became_peered":"2024-04-25T11:10:27.561604+0000","last_unstale":"2024-04-25T11:10:29.892597+0000","last_undegraded":"2024-04-25T11:10:29.892597+0000","last_fullsized":"2024-04-25T11:10:29.892597+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:59.323947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893904+0000","last_change":"2024-04-25T11:10:27.559342+0000","last_active":"2024-04-25T11:10:29.893904+0000","last_peered":"2024-04-25T11:10:29.893904+0000","last_clean":"2024-04-25T11:10:29.893904+0000","last_became_active":"2024-04-25T11:10:27.558417+0000","last_became_peered":"2024-04-25T11:10:27.558417+0000","last_unstale":"2024-04-25T11:10:29.893904+0000","last_undegraded":"2024-04-25T11:10:29.893904+0000","last_fullsized":"2024-04-25T11:10:29.893904+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:51.970801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531482+0000","last_change":"2024-04-25T11:10:23.472501+0000","last_active":"2024-04-25T11:10:28.531482+0000","last_peered":"2024-04-25T11:10:28.531482+0000","last_clean":"2024-04-25T11:10:28.531482+0000","last_became_active":"2024-04-25T11:10:23.472103+0000","last_became_peered":"2024-04-25T11:10:23.472103+0000","last_unstale":"2024-04-25T11:10:28.531482+0000","last_undegraded":"2024-04-25T11:10:28.531482+0000","last_fullsized":"2024-04-25T11:10:28.531482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290643+0000","last_change":"2024-04-25T11:10:22.487396+0000","last_active":"2024-04-25T11:10:29.290643+0000","last_peered":"2024-04-25T11:10:29.290643+0000","last_clean":"2024-04-25T11:10:29.290643+0000","last_became_active":"2024-04-25T11:10:22.484146+0000","last_became_peered":"2024-04-25T11:10:22.484146+0000","last_unstale":"2024-04-25T11:10:29.290643+0000","last_undegraded":"2024-04-25T11:10:29.290643+0000","last_fullsized":"2024-04-25T11:10:29.290643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:11.967325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891822+0000","last_change":"2024-04-25T11:10:27.545602+0000","last_active":"2024-04-25T11:10:29.891822+0000","last_peered":"2024-04-25T11:10:29.891822+0000","last_clean":"2024-04-25T11:10:29.891822+0000","last_became_active":"2024-04-25T11:10:27.545089+0000","last_became_peered":"2024-04-25T11:10:27.545089+0000","last_unstale":"2024-04-25T11:10:29.891822+0000","last_undegraded":"2024-04-25T11:10:29.891822+0000","last_fullsized":"2024-04-25T11:10:29.891822+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:24.170044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891897+0000","last_change":"2024-04-25T11:10:23.480332+0000","last_active":"2024-04-25T11:10:29.891897+0000","last_peered":"2024-04-25T11:10:29.891897+0000","last_clean":"2024-04-25T11:10:29.891897+0000","last_became_active":"2024-04-25T11:10:23.479866+0000","last_became_peered":"2024-04-25T11:10:23.479866+0000","last_unstale":"2024-04-25T11:10:29.891897+0000","last_undegraded":"2024-04-25T11:10:29.891897+0000","last_fullsized":"2024-04-25T11:10:29.891897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893193+0000","last_change":"2024-04-25T11:10:22.476980+0000","last_active":"2024-04-25T11:10:29.893193+0000","last_peered":"2024-04-25T11:10:29.893193+0000","last_clean":"2024-04-25T11:10:29.893193+0000","last_became_active":"2024-04-25T11:10:22.476483+0000","last_became_peered":"2024-04-25T11:10:22.476483+0000","last_unstale":"2024-04-25T11:10:29.893193+0000","last_undegraded":"2024-04-25T11:10:29.893193+0000","last_fullsized":"2024-04-25T11:10:29.893193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:56.918083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891422+0000","last_change":"2024-04-25T11:10:22.485058+0000","last_active":"2024-04-25T11:10:29.891422+0000","last_peered":"2024-04-25T11:10:29.891422+0000","last_clean":"2024-04-25T11:10:29.891422+0000","last_became_active":"2024-04-25T11:10:22.484647+0000","last_became_peered":"2024-04-25T11:10:22.484647+0000","last_unstale":"2024-04-25T11:10:29.891422+0000","last_undegraded":"2024-04-25T11:10:29.891422+0000","last_fullsized":"2024-04-25T11:10:29.891422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:42.000742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894489+0000","last_change":"2024-04-25T11:10:23.494632+0000","last_active":"2024-04-25T11:10:29.894489+0000","last_peered":"2024-04-25T11:10:29.894489+0000","last_clean":"2024-04-25T11:10:29.894489+0000","last_became_active":"2024-04-25T11:10:23.494214+0000","last_became_peered":"2024-04-25T11:10:23.494214+0000","last_unstale":"2024-04-25T11:10:29.894489+0000","last_undegraded":"2024-04-25T11:10:29.894489+0000","last_fullsized":"2024-04-25T11:10:29.894489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373551+0000","last_change":"2024-04-25T11:10:27.546702+0000","last_active":"2024-04-25T11:10:29.373551+0000","last_peered":"2024-04-25T11:10:29.373551+0000","last_clean":"2024-04-25T11:10:29.373551+0000","last_became_active":"2024-04-25T11:10:27.546093+0000","last_became_peered":"2024-04-25T11:10:27.546093+0000","last_unstale":"2024-04-25T11:10:29.373551+0000","last_undegraded":"2024-04-25T11:10:29.373551+0000","last_fullsized":"2024-04-25T11:10:29.373551+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:55.973871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892460+0000","last_change":"2024-04-25T11:10:22.470279+0000","last_active":"2024-04-25T11:10:29.892460+0000","last_peered":"2024-04-25T11:10:29.892460+0000","last_clean":"2024-04-25T11:10:29.892460+0000","last_became_active":"2024-04-25T11:10:22.469133+0000","last_became_peered":"2024-04-25T11:10:22.469133+0000","last_unstale":"2024-04-25T11:10:29.892460+0000","last_undegraded":"2024-04-25T11:10:29.892460+0000","last_fullsized":"2024-04-25T11:10:29.892460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:21.404101+0000","last_clean_scrub_stamp":"2024-04-25T11:10:21.404101+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:46.835819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.536180+0000","last_change":"2024-04-25T11:10:23.482007+0000","last_active":"2024-04-25T11:10:28.536180+0000","last_peered":"2024-04-25T11:10:28.536180+0000","last_clean":"2024-04-25T11:10:28.536180+0000","last_became_active":"2024-04-25T11:10:23.475539+0000","last_became_peered":"2024-04-25T11:10:23.475539+0000","last_unstale":"2024-04-25T11:10:28.536180+0000","last_undegraded":"2024-04-25T11:10:28.536180+0000","last_fullsized":"2024-04-25T11:10:28.536180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:56.556443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895038+0000","last_change":"2024-04-25T11:10:27.555814+0000","last_active":"2024-04-25T11:10:29.895038+0000","last_peered":"2024-04-25T11:10:29.895038+0000","last_clean":"2024-04-25T11:10:29.895038+0000","last_became_active":"2024-04-25T11:10:27.554994+0000","last_became_peered":"2024-04-25T11:10:27.554994+0000","last_unstale":"2024-04-25T11:10:29.895038+0000","last_undegraded":"2024-04-25T11:10:29.895038+0000","last_fullsized":"2024-04-25T11:10:29.895038+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:26.489757+0000","last_clean_scrub_stamp":"2024-04-25T11:10:26.489757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:11.895073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.890634+0000","last_change":"2024-04-25T11:10:23.487735+0000","last_active":"2024-04-25T11:10:29.890634+0000","last_peered":"2024-04-25T11:10:29.890634+0000","last_clean":"2024-04-25T11:10:29.890634+0000","last_became_active":"2024-04-25T11:10:23.487354+0000","last_became_peered":"2024-04-25T11:10:23.487354+0000","last_unstale":"2024-04-25T11:10:29.890634+0000","last_undegraded":"2024-04-25T11:10:29.890634+0000","last_fullsized":"2024-04-25T11:10:29.890634+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.373170+0000","last_change":"2024-04-25T11:10:23.476515+0000","last_active":"2024-04-25T11:10:29.373170+0000","last_peered":"2024-04-25T11:10:29.373170+0000","last_clean":"2024-04-25T11:10:29.373170+0000","last_became_active":"2024-04-25T11:10:23.476183+0000","last_became_peered":"2024-04-25T11:10:23.476183+0000","last_unstale":"2024-04-25T11:10:29.373170+0000","last_undegraded":"2024-04-25T11:10:29.373170+0000","last_fullsized":"2024-04-25T11:10:29.373170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:57.829487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290735+0000","last_change":"2024-04-25T11:10:23.477846+0000","last_active":"2024-04-25T11:10:29.290735+0000","last_peered":"2024-04-25T11:10:29.290735+0000","last_clean":"2024-04-25T11:10:29.290735+0000","last_became_active":"2024-04-25T11:10:23.476420+0000","last_became_peered":"2024-04-25T11:10:23.476420+0000","last_unstale":"2024-04-25T11:10:29.290735+0000","last_undegraded":"2024-04-25T11:10:29.290735+0000","last_fullsized":"2024-04-25T11:10:29.290735+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.894372+0000","last_change":"2024-04-25T11:10:23.485446+0000","last_active":"2024-04-25T11:10:29.894372+0000","last_peered":"2024-04-25T11:10:29.894372+0000","last_clean":"2024-04-25T11:10:29.894372+0000","last_became_active":"2024-04-25T11:10:23.485095+0000","last_became_peered":"2024-04-25T11:10:23.485095+0000","last_unstale":"2024-04-25T11:10:29.894372+0000","last_undegraded":"2024-04-25T11:10:29.894372+0000","last_fullsized":"2024-04-25T11:10:29.894372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.895256+0000","last_change":"2024-04-25T11:10:23.490644+0000","last_active":"2024-04-25T11:10:29.895256+0000","last_peered":"2024-04-25T11:10:29.895256+0000","last_clean":"2024-04-25T11:10:29.895256+0000","last_became_active":"2024-04-25T11:10:23.487911+0000","last_became_peered":"2024-04-25T11:10:23.487911+0000","last_unstale":"2024-04-25T11:10:29.895256+0000","last_undegraded":"2024-04-25T11:10:29.895256+0000","last_fullsized":"2024-04-25T11:10:29.895256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:48.595548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.532685+0000","last_change":"2024-04-25T11:10:23.484155+0000","last_active":"2024-04-25T11:10:28.532685+0000","last_peered":"2024-04-25T11:10:28.532685+0000","last_clean":"2024-04-25T11:10:28.532685+0000","last_became_active":"2024-04-25T11:10:23.483906+0000","last_became_peered":"2024-04-25T11:10:23.483906+0000","last_unstale":"2024-04-25T11:10:28.532685+0000","last_undegraded":"2024-04-25T11:10:28.532685+0000","last_fullsized":"2024-04-25T11:10:28.532685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892230+0000","last_change":"2024-04-25T11:10:23.479315+0000","last_active":"2024-04-25T11:10:29.892230+0000","last_peered":"2024-04-25T11:10:29.892230+0000","last_clean":"2024-04-25T11:10:29.892230+0000","last_became_active":"2024-04-25T11:10:23.478986+0000","last_became_peered":"2024-04-25T11:10:23.478986+0000","last_unstale":"2024-04-25T11:10:29.892230+0000","last_undegraded":"2024-04-25T11:10:29.892230+0000","last_fullsized":"2024-04-25T11:10:29.892230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.896160+0000","last_change":"2024-04-25T11:10:23.475790+0000","last_active":"2024-04-25T11:10:29.896160+0000","last_peered":"2024-04-25T11:10:29.896160+0000","last_clean":"2024-04-25T11:10:29.896160+0000","last_became_active":"2024-04-25T11:10:23.475049+0000","last_became_peered":"2024-04-25T11:10:23.475049+0000","last_unstale":"2024-04-25T11:10:29.896160+0000","last_undegraded":"2024-04-25T11:10:29.896160+0000","last_fullsized":"2024-04-25T11:10:29.896160+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:09.998219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290626+0000","last_change":"2024-04-25T11:10:23.489591+0000","last_active":"2024-04-25T11:10:29.290626+0000","last_peered":"2024-04-25T11:10:29.290626+0000","last_clean":"2024-04-25T11:10:29.290626+0000","last_became_active":"2024-04-25T11:10:23.488935+0000","last_became_peered":"2024-04-25T11:10:23.488935+0000","last_unstale":"2024-04-25T11:10:29.290626+0000","last_undegraded":"2024-04-25T11:10:29.290626+0000","last_fullsized":"2024-04-25T11:10:29.290626+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892914+0000","last_change":"2024-04-25T11:10:23.487850+0000","last_active":"2024-04-25T11:10:29.892914+0000","last_peered":"2024-04-25T11:10:29.892914+0000","last_clean":"2024-04-25T11:10:29.892914+0000","last_became_active":"2024-04-25T11:10:23.487234+0000","last_became_peered":"2024-04-25T11:10:23.487234+0000","last_unstale":"2024-04-25T11:10:29.892914+0000","last_undegraded":"2024-04-25T11:10:29.892914+0000","last_fullsized":"2024-04-25T11:10:29.892914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290514+0000","last_change":"2024-04-25T11:10:23.473547+0000","last_active":"2024-04-25T11:10:29.290514+0000","last_peered":"2024-04-25T11:10:29.290514+0000","last_clean":"2024-04-25T11:10:29.290514+0000","last_became_active":"2024-04-25T11:10:23.467594+0000","last_became_peered":"2024-04-25T11:10:23.467594+0000","last_unstale":"2024-04-25T11:10:29.290514+0000","last_undegraded":"2024-04-25T11:10:29.290514+0000","last_fullsized":"2024-04-25T11:10:29.290514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:48.595548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893781+0000","last_change":"2024-04-25T11:10:23.480740+0000","last_active":"2024-04-25T11:10:29.893781+0000","last_peered":"2024-04-25T11:10:29.893781+0000","last_clean":"2024-04-25T11:10:29.893781+0000","last_became_active":"2024-04-25T11:10:23.480405+0000","last_became_peered":"2024-04-25T11:10:23.480405+0000","last_unstale":"2024-04-25T11:10:29.893781+0000","last_undegraded":"2024-04-25T11:10:29.893781+0000","last_fullsized":"2024-04-25T11:10:29.893781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:44.483052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891342+0000","last_change":"2024-04-25T11:10:23.497261+0000","last_active":"2024-04-25T11:10:29.891342+0000","last_peered":"2024-04-25T11:10:29.891342+0000","last_clean":"2024-04-25T11:10:29.891342+0000","last_became_active":"2024-04-25T11:10:23.493494+0000","last_became_peered":"2024-04-25T11:10:23.493494+0000","last_unstale":"2024-04-25T11:10:29.891342+0000","last_undegraded":"2024-04-25T11:10:29.891342+0000","last_fullsized":"2024-04-25T11:10:29.891342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:26.010634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537112+0000","last_change":"2024-04-25T11:10:23.469676+0000","last_active":"2024-04-25T11:10:28.537112+0000","last_peered":"2024-04-25T11:10:28.537112+0000","last_clean":"2024-04-25T11:10:28.537112+0000","last_became_active":"2024-04-25T11:10:23.468887+0000","last_became_peered":"2024-04-25T11:10:23.468887+0000","last_unstale":"2024-04-25T11:10:28.537112+0000","last_undegraded":"2024-04-25T11:10:28.537112+0000","last_fullsized":"2024-04-25T11:10:28.537112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:47.104859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.374373+0000","last_change":"2024-04-25T11:10:23.491992+0000","last_active":"2024-04-25T11:10:29.374373+0000","last_peered":"2024-04-25T11:10:29.374373+0000","last_clean":"2024-04-25T11:10:29.374373+0000","last_became_active":"2024-04-25T11:10:23.491456+0000","last_became_peered":"2024-04-25T11:10:23.491456+0000","last_unstale":"2024-04-25T11:10:29.374373+0000","last_undegraded":"2024-04-25T11:10:29.374373+0000","last_fullsized":"2024-04-25T11:10:29.374373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:38.097249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.292111+0000","last_change":"2024-04-25T11:10:23.469823+0000","last_active":"2024-04-25T11:10:29.292111+0000","last_peered":"2024-04-25T11:10:29.292111+0000","last_clean":"2024-04-25T11:10:29.292111+0000","last_became_active":"2024-04-25T11:10:23.469420+0000","last_became_peered":"2024-04-25T11:10:23.469420+0000","last_unstale":"2024-04-25T11:10:29.292111+0000","last_undegraded":"2024-04-25T11:10:29.292111+0000","last_fullsized":"2024-04-25T11:10:29.292111+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290180+0000","last_change":"2024-04-25T11:10:23.479442+0000","last_active":"2024-04-25T11:10:29.290180+0000","last_peered":"2024-04-25T11:10:29.290180+0000","last_clean":"2024-04-25T11:10:29.290180+0000","last_became_active":"2024-04-25T11:10:23.472659+0000","last_became_peered":"2024-04-25T11:10:23.472659+0000","last_unstale":"2024-04-25T11:10:29.290180+0000","last_undegraded":"2024-04-25T11:10:29.290180+0000","last_fullsized":"2024-04-25T11:10:29.290180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.531629+0000","last_change":"2024-04-25T11:10:23.490775+0000","last_active":"2024-04-25T11:10:28.531629+0000","last_peered":"2024-04-25T11:10:28.531629+0000","last_clean":"2024-04-25T11:10:28.531629+0000","last_became_active":"2024-04-25T11:10:23.490138+0000","last_became_peered":"2024-04-25T11:10:23.490138+0000","last_unstale":"2024-04-25T11:10:28.531629+0000","last_undegraded":"2024-04-25T11:10:28.531629+0000","last_fullsized":"2024-04-25T11:10:28.531629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:46.963693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893293+0000","last_change":"2024-04-25T11:10:23.479882+0000","last_active":"2024-04-25T11:10:29.893293+0000","last_peered":"2024-04-25T11:10:29.893293+0000","last_clean":"2024-04-25T11:10:29.893293+0000","last_became_active":"2024-04-25T11:10:23.479534+0000","last_became_peered":"2024-04-25T11:10:23.479534+0000","last_unstale":"2024-04-25T11:10:29.893293+0000","last_undegraded":"2024-04-25T11:10:29.893293+0000","last_fullsized":"2024-04-25T11:10:29.893293+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:04.105145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892215+0000","last_change":"2024-04-25T11:10:23.483845+0000","last_active":"2024-04-25T11:10:29.892215+0000","last_peered":"2024-04-25T11:10:29.892215+0000","last_clean":"2024-04-25T11:10:29.892215+0000","last_became_active":"2024-04-25T11:10:23.482401+0000","last_became_peered":"2024-04-25T11:10:23.482401+0000","last_unstale":"2024-04-25T11:10:29.892215+0000","last_undegraded":"2024-04-25T11:10:29.892215+0000","last_fullsized":"2024-04-25T11:10:29.892215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.537880+0000","last_change":"2024-04-25T11:10:23.490750+0000","last_active":"2024-04-25T11:10:28.537880+0000","last_peered":"2024-04-25T11:10:28.537880+0000","last_clean":"2024-04-25T11:10:28.537880+0000","last_became_active":"2024-04-25T11:10:23.490489+0000","last_became_peered":"2024-04-25T11:10:23.490489+0000","last_unstale":"2024-04-25T11:10:28.537880+0000","last_undegraded":"2024-04-25T11:10:28.537880+0000","last_fullsized":"2024-04-25T11:10:28.537880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:30.205583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891988+0000","last_change":"2024-04-25T11:10:23.498316+0000","last_active":"2024-04-25T11:10:29.891988+0000","last_peered":"2024-04-25T11:10:29.891988+0000","last_clean":"2024-04-25T11:10:29.891988+0000","last_became_active":"2024-04-25T11:10:23.497414+0000","last_became_peered":"2024-04-25T11:10:23.497414+0000","last_unstale":"2024-04-25T11:10:29.891988+0000","last_undegraded":"2024-04-25T11:10:29.891988+0000","last_fullsized":"2024-04-25T11:10:29.891988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.892093+0000","last_change":"2024-04-25T11:10:23.487588+0000","last_active":"2024-04-25T11:10:29.892093+0000","last_peered":"2024-04-25T11:10:29.892093+0000","last_clean":"2024-04-25T11:10:29.892093+0000","last_became_active":"2024-04-25T11:10:23.487280+0000","last_became_peered":"2024-04-25T11:10:23.487280+0000","last_unstale":"2024-04-25T11:10:29.892093+0000","last_undegraded":"2024-04-25T11:10:29.892093+0000","last_fullsized":"2024-04-25T11:10:29.892093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:57.312352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538694+0000","last_change":"2024-04-25T11:10:23.472328+0000","last_active":"2024-04-25T11:10:28.538694+0000","last_peered":"2024-04-25T11:10:28.538694+0000","last_clean":"2024-04-25T11:10:28.538694+0000","last_became_active":"2024-04-25T11:10:23.471692+0000","last_became_peered":"2024-04-25T11:10:23.471692+0000","last_unstale":"2024-04-25T11:10:28.538694+0000","last_undegraded":"2024-04-25T11:10:28.538694+0000","last_fullsized":"2024-04-25T11:10:28.538694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:50.974212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.891640+0000","last_change":"2024-04-25T11:10:23.471703+0000","last_active":"2024-04-25T11:10:29.891640+0000","last_peered":"2024-04-25T11:10:29.891640+0000","last_clean":"2024-04-25T11:10:29.891640+0000","last_became_active":"2024-04-25T11:10:23.469570+0000","last_became_peered":"2024-04-25T11:10:23.469570+0000","last_unstale":"2024-04-25T11:10:29.891640+0000","last_undegraded":"2024-04-25T11:10:29.891640+0000","last_fullsized":"2024-04-25T11:10:29.891640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:20.637180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.535108+0000","last_change":"2024-04-25T11:10:23.473106+0000","last_active":"2024-04-25T11:10:28.535108+0000","last_peered":"2024-04-25T11:10:28.535108+0000","last_clean":"2024-04-25T11:10:28.535108+0000","last_became_active":"2024-04-25T11:10:23.472627+0000","last_became_peered":"2024-04-25T11:10:23.472627+0000","last_unstale":"2024-04-25T11:10:28.535108+0000","last_undegraded":"2024-04-25T11:10:28.535108+0000","last_fullsized":"2024-04-25T11:10:28.535108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.535455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.290419+0000","last_change":"2024-04-25T11:10:23.480283+0000","last_active":"2024-04-25T11:10:29.290419+0000","last_peered":"2024-04-25T11:10:29.290419+0000","last_clean":"2024-04-25T11:10:29.290419+0000","last_became_active":"2024-04-25T11:10:23.479889+0000","last_became_peered":"2024-04-25T11:10:23.479889+0000","last_unstale":"2024-04-25T11:10:29.290419+0000","last_undegraded":"2024-04-25T11:10:29.290419+0000","last_fullsized":"2024-04-25T11:10:29.290419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.534607+0000","last_change":"2024-04-25T11:10:23.469104+0000","last_active":"2024-04-25T11:10:28.534607+0000","last_peered":"2024-04-25T11:10:28.534607+0000","last_clean":"2024-04-25T11:10:28.534607+0000","last_became_active":"2024-04-25T11:10:23.468743+0000","last_became_peered":"2024-04-25T11:10:23.468743+0000","last_unstale":"2024-04-25T11:10:28.534607+0000","last_undegraded":"2024-04-25T11:10:28.534607+0000","last_fullsized":"2024-04-25T11:10:28.534607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:20.153177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.291058+0000","last_change":"2024-04-25T11:10:23.496747+0000","last_active":"2024-04-25T11:10:29.291058+0000","last_peered":"2024-04-25T11:10:29.291058+0000","last_clean":"2024-04-25T11:10:29.291058+0000","last_became_active":"2024-04-25T11:10:23.496354+0000","last_became_peered":"2024-04-25T11:10:23.496354+0000","last_unstale":"2024-04-25T11:10:29.291058+0000","last_undegraded":"2024-04-25T11:10:29.291058+0000","last_fullsized":"2024-04-25T11:10:29.291058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:41.745805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:29.893849+0000","last_change":"2024-04-25T11:10:23.480571+0000","last_active":"2024-04-25T11:10:29.893849+0000","last_peered":"2024-04-25T11:10:29.893849+0000","last_clean":"2024-04-25T11:10:29.893849+0000","last_became_active":"2024-04-25T11:10:23.479502+0000","last_became_peered":"2024-04-25T11:10:23.479502+0000","last_unstale":"2024-04-25T11:10:29.893849+0000","last_undegraded":"2024-04-25T11:10:29.893849+0000","last_fullsized":"2024-04-25T11:10:29.893849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:41.107653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.533940+0000","last_change":"2024-04-25T11:10:23.475376+0000","last_active":"2024-04-25T11:10:28.533940+0000","last_peered":"2024-04-25T11:10:28.533940+0000","last_clean":"2024-04-25T11:10:28.533940+0000","last_became_active":"2024-04-25T11:10:23.474750+0000","last_became_peered":"2024-04-25T11:10:23.474750+0000","last_unstale":"2024-04-25T11:10:28.533940+0000","last_undegraded":"2024-04-25T11:10:28.533940+0000","last_fullsized":"2024-04-25T11:10:28.533940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:02.929595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T11:10:28.538538+0000","last_change":"2024-04-25T11:10:23.481991+0000","last_active":"2024-04-25T11:10:28.538538+0000","last_peered":"2024-04-25T11:10:28.538538+0000","last_clean":"2024-04-25T11:10:28.538538+0000","last_became_active":"2024-04-25T11:10:23.481639+0000","last_became_peered":"2024-04-25T11:10:23.481639+0000","last_unstale":"2024-04-25T11:10:28.538538+0000","last_undegraded":"2024-04-25T11:10:28.538538+0000","last_fullsized":"2024-04-25T11:10:28.538538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:22.420289+0000","last_clean_scrub_stamp":"2024-04-25T11:10:22.420289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:13.333031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476748,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":695436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476749,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476748,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476749,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":104612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":274432,"data_stored":104646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:11:02.606 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:11:02.606 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T11:11:02.606 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T11:11:02.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:11:02.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.918+0000 7f77a4c00640 1 Processor -- start 2024-04-25T11:11:02.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.918+0000 7f77a4c00640 1 -- start start 2024-04-25T11:11:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.921+0000 7f77a4c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77a015e2f0 0x7f77a0161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.921+0000 7f77a4c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f77a0162420 0x7f77a006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.921+0000 7f77a4c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77a015e700 con 0x7f77a015baa0 2024-04-25T11:11:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.921+0000 7f77a4c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f77a0162830 con 0x7f77a0162420 2024-04-25T11:11:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.921+0000 7f77a4c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f77a015e980 con 0x7f77a015e2f0 2024-04-25T11:11:02.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.921+0000 7f779fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f77a0162420 0x7f77a006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:02.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.921+0000 7f779fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f77a0162420 0x7f77a006d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43440/0 (socket says 172.21.15.7:43440) 2024-04-25T11:11:02.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.921+0000 7f779fe00640 1 -- 172.21.15.7:0/3936488745 learned_addr learned my addr 172.21.15.7:0/3936488745 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:02.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.922+0000 7f779ea00640 1 --2- 172.21.15.7:0/3936488745 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77a015e2f0 0x7f77a0161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:02.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.922+0000 7f779fe00640 1 -- 172.21.15.7:0/3936488745 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77a015e2f0 msgr2=0x7f77a0161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:02.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.922+0000 7f779fe00640 1 --2- 172.21.15.7:0/3936488745 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77a015e2f0 0x7f77a0161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.922+0000 7f779fe00640 1 -- 172.21.15.7:0/3936488745 >> v1:172.21.15.87:6789/0 conn(0x7f77a015baa0 legacy=0x7f77a0070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.922+0000 7f779fe00640 1 -- 172.21.15.7:0/3936488745 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a015ec60 con 0x7f77a0162420 2024-04-25T11:11:02.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.923+0000 7f779fe00640 1 --2- 172.21.15.7:0/3936488745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a0162420 0x7f77a006d580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7798009e50 tx=0x7f779802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe7b3ec4a0d70d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:02.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.924+0000 7f779e000640 1 -- 172.21.15.7:0/3936488745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798036070 con 0x7f77a0162420 2024-04-25T11:11:02.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.924+0000 7f779e000640 1 -- 172.21.15.7:0/3936488745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7798031600 con 0x7f77a0162420 2024-04-25T11:11:02.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.924+0000 7f779e000640 1 -- 172.21.15.7:0/3936488745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77980319d0 con 0x7f77a0162420 2024-04-25T11:11:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.925+0000 7f77a4c00640 1 -- 172.21.15.7:0/3936488745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a0162420 msgr2=0x7f77a006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.925+0000 7f77a4c00640 1 --2- 172.21.15.7:0/3936488745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a0162420 0x7f77a006d580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7798009e50 tx=0x7f779802a730 comp rx=0 tx=0).stop 2024-04-25T11:11:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.925+0000 7f77a4c00640 1 -- 172.21.15.7:0/3936488745 shutdown_connections 2024-04-25T11:11:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.925+0000 7f77a4c00640 1 --2- 172.21.15.7:0/3936488745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a0162420 0x7f77a006d580 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.925+0000 7f77a4c00640 1 --2- 172.21.15.7:0/3936488745 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77a015e2f0 0x7f77a0161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.925+0000 7f77a4c00640 1 -- 172.21.15.7:0/3936488745 >> 172.21.15.7:0/3936488745 conn(0x7f77a0073090 msgr2=0x7f77a00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.925+0000 7f77a4c00640 1 -- 172.21.15.7:0/3936488745 shutdown_connections 2024-04-25T11:11:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.925+0000 7f77a4c00640 1 -- 172.21.15.7:0/3936488745 wait complete. 2024-04-25T11:11:02.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.927+0000 7f77a4c00640 1 Processor -- start 2024-04-25T11:11:02.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.927+0000 7f77a4c00640 1 -- start start 2024-04-25T11:11:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.929+0000 7f77a4c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77a015e2f0 0x7f77a01e7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.929+0000 7f77a4c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a0162420 0x7f77a015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.929+0000 7f77a4c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a0161040 0x7f77a01e9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.929+0000 7f77a4c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77a006dac0 con 0x7f77a0162420 2024-04-25T11:11:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.929+0000 7f77a4c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f77a01619c0 con 0x7f77a0161040 2024-04-25T11:11:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.929+0000 7f77a4c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f77a006dc40 con 0x7f77a015e2f0 2024-04-25T11:11:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.929+0000 7f779ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a0162420 0x7f77a015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.929+0000 7f779ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a0162420 0x7f77a015a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43448/0 (socket says 172.21.15.7:43448) 2024-04-25T11:11:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.929+0000 7f779ea00640 1 -- 172.21.15.7:0/2691809056 learned_addr learned my addr 172.21.15.7:0/2691809056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f779f400640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77a015e2f0 0x7f77a01e7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f779fe00640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a0161040 0x7f77a01e9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:02.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f779ea00640 1 -- 172.21.15.7:0/2691809056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77a015e2f0 msgr2=0x7f77a01e7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f779ea00640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77a015e2f0 0x7f77a01e7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f779ea00640 1 -- 172.21.15.7:0/2691809056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a0161040 msgr2=0x7f77a01e9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:02.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f779ea00640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a0161040 0x7f77a01e9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:02.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f779ea00640 1 -- 172.21.15.7:0/2691809056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a015acc0 con 0x7f77a0162420 2024-04-25T11:11:02.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f779ea00640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a0162420 0x7f77a015a780 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f778c00d620 tx=0x7f778c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:02.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f7797400640 1 -- 172.21.15.7:0/2691809056 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c00b620 con 0x7f77a0162420 2024-04-25T11:11:02.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f7797400640 1 -- 172.21.15.7:0/2691809056 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778c019070 con 0x7f77a0162420 2024-04-25T11:11:02.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f77a4c00640 1 -- 172.21.15.7:0/2691809056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a01e8090 con 0x7f77a0162420 2024-04-25T11:11:02.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.930+0000 7f7797400640 1 -- 172.21.15.7:0/2691809056 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c00ba00 con 0x7f77a0162420 2024-04-25T11:11:02.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.931+0000 7f77a4c00640 1 -- 172.21.15.7:0/2691809056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a01e8650 con 0x7f77a0162420 2024-04-25T11:11:02.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.932+0000 7f77a4c00640 1 -- 172.21.15.7:0/2691809056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a0074020 con 0x7f77a0162420 2024-04-25T11:11:02.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.933+0000 7f7797400640 1 -- 172.21.15.7:0/2691809056 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f778c00bba0 con 0x7f77a0162420 2024-04-25T11:11:02.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.934+0000 7f7797400640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77800744b0 0x7f77800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:02.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.935+0000 7f7797400640 1 -- 172.21.15.7:0/2691809056 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f778c095ac0 con 0x7f77a0162420 2024-04-25T11:11:02.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.935+0000 7f779f400640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77800744b0 0x7f77800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:02.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.935+0000 7f779f400640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77800744b0 0x7f77800769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7790007fd0 tx=0x7f7790009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:02.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:02.939+0000 7f7797400640 1 -- 172.21.15.7:0/2691809056 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77a015acc0 con 0x7f77a0162420 2024-04-25T11:11:03.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.205+0000 7f77a4c00640 1 -- 172.21.15.7:0/2691809056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f77a0146860 con 0x7f77a0162420 2024-04-25T11:11:03.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.206+0000 7f7797400640 1 -- 172.21.15.7:0/2691809056 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f778c05fe40 con 0x7f77a0162420 2024-04-25T11:11:03.206 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:03.206 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:11:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.208+0000 7f77a4c00640 1 -- 172.21.15.7:0/2691809056 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77800744b0 msgr2=0x7f77800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.208+0000 7f77a4c00640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77800744b0 0x7f77800769c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7790007fd0 tx=0x7f7790009040 comp rx=0 tx=0).stop 2024-04-25T11:11:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.208+0000 7f77a4c00640 1 -- 172.21.15.7:0/2691809056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a0162420 msgr2=0x7f77a015a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.208+0000 7f77a4c00640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a0162420 0x7f77a015a780 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f778c00d620 tx=0x7f778c00dae0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.209+0000 7f77a4c00640 1 -- 172.21.15.7:0/2691809056 shutdown_connections 2024-04-25T11:11:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.209+0000 7f77a4c00640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77800744b0 0x7f77800769c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.209+0000 7f77a4c00640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77a0161040 0x7f77a01e9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.209+0000 7f77a4c00640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77a0162420 0x7f77a015a780 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.209+0000 7f77a4c00640 1 --2- 172.21.15.7:0/2691809056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77a015e2f0 0x7f77a01e7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.209+0000 7f77a4c00640 1 -- 172.21.15.7:0/2691809056 >> 172.21.15.7:0/2691809056 conn(0x7f77a0073090 msgr2=0x7f77a0147410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.209+0000 7f77a4c00640 1 -- 172.21.15.7:0/2691809056 shutdown_connections 2024-04-25T11:11:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.209+0000 7f77a4c00640 1 -- 172.21.15.7:0/2691809056 wait complete. 2024-04-25T11:11:03.235 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T11:11:03.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:03.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.524+0000 7fbfc2600640 1 Processor -- start 2024-04-25T11:11:03.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.524+0000 7fbfc2600640 1 -- start start 2024-04-25T11:11:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.526+0000 7fbfc2600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbfbc160350 0x7fbfbc0d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.526+0000 7fbfc2600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbfbc0b4730 0x7fbfbc0d28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.526+0000 7fbfc2600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfbc163ce0 con 0x7fbfbc0b4c10 2024-04-25T11:11:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.526+0000 7fbfc2600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbfbc0d2e20 con 0x7fbfbc0b4730 2024-04-25T11:11:03.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.526+0000 7fbfc2600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbfbc1645f0 con 0x7fbfbc160350 2024-04-25T11:11:03.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.527+0000 7fbfb3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbfbc0b4730 0x7fbfbc0d28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:03.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.527+0000 7fbfb3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbfbc0b4730 0x7fbfbc0d28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43464/0 (socket says 172.21.15.7:43464) 2024-04-25T11:11:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.527+0000 7fbfb3e00640 1 -- 172.21.15.7:0/3452871342 learned_addr learned my addr 172.21.15.7:0/3452871342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.527+0000 7fbfbb400640 1 --2- 172.21.15.7:0/3452871342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbfbc160350 0x7fbfbc0d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.527+0000 7fbfb3e00640 1 -- 172.21.15.7:0/3452871342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbfbc160350 msgr2=0x7fbfbc0d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.527+0000 7fbfb3e00640 1 --2- 172.21.15.7:0/3452871342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbfbc160350 0x7fbfbc0d23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.527+0000 7fbfb3e00640 1 -- 172.21.15.7:0/3452871342 >> v1:172.21.15.87:6789/0 conn(0x7fbfbc0b4c10 legacy=0x7fbfbc0d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:03.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.527+0000 7fbfb3e00640 1 -- 172.21.15.7:0/3452871342 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc0b83f0 con 0x7fbfbc0b4730 2024-04-25T11:11:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.528+0000 7fbfb3e00640 1 --2- 172.21.15.7:0/3452871342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc0b4730 0x7fbfbc0d28e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009870 tx=0x7fbfa402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7728a8f3c0377501 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.529+0000 7fbfbaa00640 1 -- 172.21.15.7:0/3452871342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4037070 con 0x7fbfbc0b4730 2024-04-25T11:11:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.529+0000 7fbfbaa00640 1 -- 172.21.15.7:0/3452871342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa4032440 con 0x7fbfbc0b4730 2024-04-25T11:11:03.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.529+0000 7fbfbaa00640 1 -- 172.21.15.7:0/3452871342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4032810 con 0x7fbfbc0b4730 2024-04-25T11:11:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.529+0000 7fbfc2600640 1 -- 172.21.15.7:0/3452871342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc0b4730 msgr2=0x7fbfbc0d28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.529+0000 7fbfc2600640 1 --2- 172.21.15.7:0/3452871342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc0b4730 0x7fbfbc0d28e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009870 tx=0x7fbfa402a1c0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.529+0000 7fbfc2600640 1 -- 172.21.15.7:0/3452871342 shutdown_connections 2024-04-25T11:11:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.529+0000 7fbfc2600640 1 --2- 172.21.15.7:0/3452871342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc0b4730 0x7fbfbc0d28e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.529+0000 7fbfc2600640 1 --2- 172.21.15.7:0/3452871342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbfbc160350 0x7fbfbc0d23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.529+0000 7fbfc2600640 1 -- 172.21.15.7:0/3452871342 >> 172.21.15.7:0/3452871342 conn(0x7fbfbc06edb0 msgr2=0x7fbfbc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.530+0000 7fbfc2600640 1 -- 172.21.15.7:0/3452871342 shutdown_connections 2024-04-25T11:11:03.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.530+0000 7fbfc2600640 1 -- 172.21.15.7:0/3452871342 wait complete. 2024-04-25T11:11:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.532+0000 7fbfc2600640 1 Processor -- start 2024-04-25T11:11:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.532+0000 7fbfc2600640 1 -- start start 2024-04-25T11:11:03.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfc2600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfbc0b4730 0x7fbfbc0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfc2600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbfbc0b4c10 0x7fbfbc14c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfc2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc160350 0x7fbfbc0cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfc2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc164280 con 0x7fbfbc160350 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfc2600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc0d54e0 con 0x7fbfbc0b4730 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfc2600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbfbc0b8080 con 0x7fbfbc0b4c10 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfbbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc160350 0x7fbfbc0cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfbbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc160350 0x7fbfbc0cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43472/0 (socket says 172.21.15.7:43472) 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfbbe00640 1 -- 172.21.15.7:0/3346312583 learned_addr learned my addr 172.21.15.7:0/3346312583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfbb400640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfbc0b4730 0x7fbfbc0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfb3e00640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbfbc0b4c10 0x7fbfbc14c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfbbe00640 1 -- 172.21.15.7:0/3346312583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbfbc0b4c10 msgr2=0x7fbfbc14c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfbbe00640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbfbc0b4c10 0x7fbfbc14c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfbbe00640 1 -- 172.21.15.7:0/3346312583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfbc0b4730 msgr2=0x7fbfbc0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfbbe00640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfbc0b4730 0x7fbfbc0cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.534+0000 7fbfbbe00640 1 -- 172.21.15.7:0/3346312583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc14d540 con 0x7fbfbc160350 2024-04-25T11:11:03.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.535+0000 7fbfbbe00640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc160350 0x7fbfbc0cbf10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400c730 tx=0x7fbfb400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:03.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.535+0000 7fbfb3400640 1 -- 172.21.15.7:0/3346312583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4003b70 con 0x7fbfbc160350 2024-04-25T11:11:03.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.535+0000 7fbfb3400640 1 -- 172.21.15.7:0/3346312583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb4003d10 con 0x7fbfbc160350 2024-04-25T11:11:03.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.535+0000 7fbfc2600640 1 -- 172.21.15.7:0/3346312583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfbc14a940 con 0x7fbfbc160350 2024-04-25T11:11:03.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.535+0000 7fbfb3400640 1 -- 172.21.15.7:0/3346312583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4005310 con 0x7fbfbc160350 2024-04-25T11:11:03.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.535+0000 7fbfc2600640 1 -- 172.21.15.7:0/3346312583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfbc14c4a0 con 0x7fbfbc160350 2024-04-25T11:11:03.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.536+0000 7fbfc2600640 1 -- 172.21.15.7:0/3346312583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfbc06a100 con 0x7fbfbc160350 2024-04-25T11:11:03.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.538+0000 7fbfb3400640 1 -- 172.21.15.7:0/3346312583 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbfb40056c0 con 0x7fbfbc160350 2024-04-25T11:11:03.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.539+0000 7fbfb3400640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf980744b0 0x7fbf980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:03.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.539+0000 7fbfb3400640 1 -- 172.21.15.7:0/3346312583 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fbfb40955f0 con 0x7fbfbc160350 2024-04-25T11:11:03.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.539+0000 7fbfbb400640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf980744b0 0x7fbf980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:03.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.540+0000 7fbfbb400640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf980744b0 0x7fbf980769c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009a30 tx=0x7fbfac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:03.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.544+0000 7fbfb3400640 1 -- 172.21.15.7:0/3346312583 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbfb405fa00 con 0x7fbfbc160350 2024-04-25T11:11:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.747+0000 7fbfc2600640 1 -- 172.21.15.7:0/3346312583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbfbc0d3ff0 con 0x7fbfbc160350 2024-04-25T11:11:03.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.749+0000 7fbfb3400640 1 -- 172.21.15.7:0/3346312583 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7fbfb405f300 con 0x7fbfbc160350 2024-04-25T11:11:03.750 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:03.750 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.752+0000 7fbfc2600640 1 -- 172.21.15.7:0/3346312583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf980744b0 msgr2=0x7fbf980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.752+0000 7fbfc2600640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf980744b0 0x7fbf980769c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009a30 tx=0x7fbfac008040 comp rx=0 tx=0).stop 2024-04-25T11:11:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.752+0000 7fbfc2600640 1 -- 172.21.15.7:0/3346312583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc160350 msgr2=0x7fbfbc0cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.752+0000 7fbfc2600640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc160350 0x7fbfbc0cbf10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400c730 tx=0x7fbfb400cbf0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.753+0000 7fbfc2600640 1 -- 172.21.15.7:0/3346312583 shutdown_connections 2024-04-25T11:11:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.753+0000 7fbfc2600640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf980744b0 0x7fbf980769c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.753+0000 7fbfc2600640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfbc160350 0x7fbfbc0cbf10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.753+0000 7fbfc2600640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbfbc0b4c10 0x7fbfbc14c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.753+0000 7fbfc2600640 1 --2- 172.21.15.7:0/3346312583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfbc0b4730 0x7fbfbc0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.753+0000 7fbfc2600640 1 -- 172.21.15.7:0/3346312583 >> 172.21.15.7:0/3346312583 conn(0x7fbfbc06edb0 msgr2=0x7fbfbc0b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.753+0000 7fbfc2600640 1 -- 172.21.15.7:0/3346312583 shutdown_connections 2024-04-25T11:11:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:03.753+0000 7fbfc2600640 1 -- 172.21.15.7:0/3346312583 wait complete. 2024-04-25T11:11:03.780 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:03.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:04.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.068+0000 7f1a7be00640 1 Processor -- start 2024-04-25T11:11:04.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.069+0000 7f1a7be00640 1 -- start start 2024-04-25T11:11:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.071+0000 7f1a7be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a7c146180 0x7f1a7c070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.071+0000 7f1a7be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1a7c0b08e0 0x7f1a7c149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:04.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.071+0000 7f1a7be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a7c070860 con 0x7f1a7c1490f0 2024-04-25T11:11:04.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.071+0000 7f1a7be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1a7c146590 con 0x7f1a7c0b08e0 2024-04-25T11:11:04.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.072+0000 7f1a7be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7c06f280 con 0x7f1a7c146180 2024-04-25T11:11:04.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.072+0000 7f1a7aa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a7c146180 0x7f1a7c070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:04.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.072+0000 7f1a7b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1a7c0b08e0 0x7f1a7c149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:04.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.072+0000 7f1a7a000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1a7c1490f0 0x7f1a7c0b01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:49550/0 (socket says 172.21.15.7:49550) 2024-04-25T11:11:04.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.072+0000 7f1a7a000640 1 -- 172.21.15.7:0/3832018571 learned_addr learned my addr 172.21.15.7:0/3832018571 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:04.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.073+0000 7f1a7b400640 1 -- 172.21.15.7:0/3832018571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a7c146180 msgr2=0x7f1a7c070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:04.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.073+0000 7f1a7b400640 1 --2- 172.21.15.7:0/3832018571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a7c146180 0x7f1a7c070320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.073+0000 7f1a7b400640 1 -- 172.21.15.7:0/3832018571 >> v1:172.21.15.87:6789/0 conn(0x7f1a7c1490f0 legacy=0x7f1a7c0b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.073+0000 7f1a7b400640 1 -- 172.21.15.7:0/3832018571 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c06f560 con 0x7f1a7c0b08e0 2024-04-25T11:11:04.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.073+0000 7f1a7aa00640 1 --2- 172.21.15.7:0/3832018571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a7c146180 0x7f1a7c070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:11:04.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.075+0000 7f1a7b400640 1 --2- 172.21.15.7:0/3832018571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a7c0b08e0 0x7f1a7c149500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009d90 tx=0x7f1a7402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e4fc382d4248b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:04.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.075+0000 7f1a79600640 1 -- 172.21.15.7:0/3832018571 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74007740 con 0x7f1a7c0b08e0 2024-04-25T11:11:04.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.075+0000 7f1a79600640 1 -- 172.21.15.7:0/3832018571 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a740078e0 con 0x7f1a7c0b08e0 2024-04-25T11:11:04.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.075+0000 7f1a79600640 1 -- 172.21.15.7:0/3832018571 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74007cb0 con 0x7f1a7c0b08e0 2024-04-25T11:11:04.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.076+0000 7f1a7be00640 1 -- 172.21.15.7:0/3832018571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a7c0b08e0 msgr2=0x7f1a7c149500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.076+0000 7f1a7be00640 1 --2- 172.21.15.7:0/3832018571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a7c0b08e0 0x7f1a7c149500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009d90 tx=0x7f1a7402a660 comp rx=0 tx=0).stop 2024-04-25T11:11:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.076+0000 7f1a7be00640 1 -- 172.21.15.7:0/3832018571 shutdown_connections 2024-04-25T11:11:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.076+0000 7f1a7be00640 1 --2- 172.21.15.7:0/3832018571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a7c0b08e0 0x7f1a7c149500 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.076+0000 7f1a7be00640 1 --2- 172.21.15.7:0/3832018571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a7c146180 0x7f1a7c070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.076+0000 7f1a7be00640 1 -- 172.21.15.7:0/3832018571 >> 172.21.15.7:0/3832018571 conn(0x7f1a7c0740f0 msgr2=0x7f1a7c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.076+0000 7f1a7be00640 1 -- 172.21.15.7:0/3832018571 shutdown_connections 2024-04-25T11:11:04.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.076+0000 7f1a7be00640 1 -- 172.21.15.7:0/3832018571 wait complete. 2024-04-25T11:11:04.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.079+0000 7f1a7be00640 1 Processor -- start 2024-04-25T11:11:04.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.079+0000 7f1a7be00640 1 -- start start 2024-04-25T11:11:04.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.080+0000 7f1a7be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a7c0b08e0 0x7f1a7c131b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:04.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a7c146180 0x7f1a7c1f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:04.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a7c1490f0 0x7f1a7c12ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:04.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c06fe00 con 0x7f1a7c146180 2024-04-25T11:11:04.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c06fb20 con 0x7f1a7c0b08e0 2024-04-25T11:11:04.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7c06eea0 con 0x7f1a7c1490f0 2024-04-25T11:11:04.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a7c146180 0x7f1a7c1f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:04.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a7c146180 0x7f1a7c1f9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43480/0 (socket says 172.21.15.7:43480) 2024-04-25T11:11:04.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7a000640 1 -- 172.21.15.7:0/4242105972 learned_addr learned my addr 172.21.15.7:0/4242105972 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:04.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7b400640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a7c1490f0 0x7f1a7c12ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7aa00640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a7c0b08e0 0x7f1a7c131b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7a000640 1 -- 172.21.15.7:0/4242105972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a7c1490f0 msgr2=0x7f1a7c12ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7a000640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a7c1490f0 0x7f1a7c12ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7a000640 1 -- 172.21.15.7:0/4242105972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a7c0b08e0 msgr2=0x7f1a7c131b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7a000640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a7c0b08e0 0x7f1a7c131b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.081+0000 7f1a7a000640 1 -- 172.21.15.7:0/4242105972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c1fa660 con 0x7f1a7c146180 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.082+0000 7f1a7a000640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a7c146180 0x7f1a7c1f9af0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c470 tx=0x7f1a6800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.082+0000 7f1a72a00640 1 -- 172.21.15.7:0/4242105972 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a68017070 con 0x7f1a7c146180 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.082+0000 7f1a72a00640 1 -- 172.21.15.7:0/4242105972 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a68005ce0 con 0x7f1a7c146180 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.082+0000 7f1a72a00640 1 -- 172.21.15.7:0/4242105972 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a680072d0 con 0x7f1a7c146180 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.082+0000 7f1a7be00640 1 -- 172.21.15.7:0/4242105972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7c1310a0 con 0x7f1a7c146180 2024-04-25T11:11:04.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.082+0000 7f1a7be00640 1 -- 172.21.15.7:0/4242105972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7c1352d0 con 0x7f1a7c146180 2024-04-25T11:11:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.083+0000 7f1a7be00640 1 -- 172.21.15.7:0/4242105972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a7c1fa380 con 0x7f1a7c146180 2024-04-25T11:11:04.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.085+0000 7f1a72a00640 1 -- 172.21.15.7:0/4242105972 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1a6801a070 con 0x7f1a7c146180 2024-04-25T11:11:04.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.086+0000 7f1a72a00640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1a5c0744b0 0x7f1a5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:04.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.086+0000 7f1a72a00640 1 -- 172.21.15.7:0/4242105972 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f1a68094810 con 0x7f1a7c146180 2024-04-25T11:11:04.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.086+0000 7f1a7aa00640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1a5c0744b0 0x7f1a5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:04.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.087+0000 7f1a7aa00640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1a5c0744b0 0x7f1a5c0769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009a30 tx=0x7f1a6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:04.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.091+0000 7f1a72a00640 1 -- 172.21.15.7:0/4242105972 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1a6805e540 con 0x7f1a7c146180 2024-04-25T11:11:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.293+0000 7f1a7be00640 1 -- 172.21.15.7:0/4242105972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a7c143ef0 con 0x7f1a7c146180 2024-04-25T11:11:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.294+0000 7f1a72a00640 1 -- 172.21.15.7:0/4242105972 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f1a68062840 con 0x7f1a7c146180 2024-04-25T11:11:04.294 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 -- 172.21.15.7:0/4242105972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1a5c0744b0 msgr2=0x7f1a5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1a5c0744b0 0x7f1a5c0769c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009a30 tx=0x7f1a6c008040 comp rx=0 tx=0).stop 2024-04-25T11:11:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 -- 172.21.15.7:0/4242105972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a7c146180 msgr2=0x7f1a7c1f9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a7c146180 0x7f1a7c1f9af0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a6800c470 tx=0x7f1a6800c930 comp rx=0 tx=0).stop 2024-04-25T11:11:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 -- 172.21.15.7:0/4242105972 shutdown_connections 2024-04-25T11:11:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1a5c0744b0 0x7f1a5c0769c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a7c1490f0 0x7f1a7c12ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a7c146180 0x7f1a7c1f9af0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 --2- 172.21.15.7:0/4242105972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a7c0b08e0 0x7f1a7c131b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 -- 172.21.15.7:0/4242105972 >> 172.21.15.7:0/4242105972 conn(0x7f1a7c0740f0 msgr2=0x7f1a7c0684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 -- 172.21.15.7:0/4242105972 shutdown_connections 2024-04-25T11:11:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.296+0000 7f1a7be00640 1 -- 172.21.15.7:0/4242105972 wait complete. 2024-04-25T11:11:04.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:04.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.602+0000 7fa398e00640 1 Processor -- start 2024-04-25T11:11:04.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.603+0000 7fa398e00640 1 -- start start 2024-04-25T11:11:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.605+0000 7fa398e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa39406fab0 0x7fa394168e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.605+0000 7fa398e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa39415e3c0 con 0x7fa39415dfb0 2024-04-25T11:11:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.605+0000 7fa398e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3940cf800 con 0x7fa3940b0470 2024-04-25T11:11:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.605+0000 7fa398e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa3940d1250 con 0x7fa39406fab0 2024-04-25T11:11:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.605+0000 7fa391e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa39415dfb0 0x7fa3940d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33842/0 (socket says 172.21.15.7:33842) 2024-04-25T11:11:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.605+0000 7fa391e00640 1 -- 172.21.15.7:0/1739533696 learned_addr learned my addr 172.21.15.7:0/1739533696 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.606+0000 7fa392800640 1 --2- 172.21.15.7:0/1739533696 >> v2:172.21.15.7:3300/0 conn(0x7fa39406fab0 0x7fa394168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:04.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.606+0000 7fa390a00640 1 -- 172.21.15.7:0/1739533696 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 923724979 0 0) 0x7fa39415e3c0 con 0x7fa39415dfb0 2024-04-25T11:11:04.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.606+0000 7fa390a00640 1 -- 172.21.15.7:0/1739533696 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa378003520 con 0x7fa39415dfb0 2024-04-25T11:11:04.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.606+0000 7fa392800640 1 -- 172.21.15.7:0/1739533696 >> v1:172.21.15.87:6789/0 conn(0x7fa3940b0470 legacy=0x7fa39406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.606+0000 7fa392800640 1 -- 172.21.15.7:0/1739533696 >> v1:172.21.15.7:6789/0 conn(0x7fa39415dfb0 legacy=0x7fa3940d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.606+0000 7fa390a00640 1 -- 172.21.15.7:0/1739533696 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3784721974 0 0) 0x7fa378003520 con 0x7fa39415dfb0 2024-04-25T11:11:04.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.606+0000 7fa392800640 1 -- 172.21.15.7:0/1739533696 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3940cfaa0 con 0x7fa39406fab0 2024-04-25T11:11:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.607+0000 7fa392800640 1 --2- 172.21.15.7:0/1739533696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa39406fab0 0x7fa394168e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa3940ceb10 tx=0x7fa38c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf8d2009f66a864 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.608+0000 7fa390a00640 1 -- 172.21.15.7:0/1739533696 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384001670 con 0x7fa39406fab0 2024-04-25T11:11:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.608+0000 7fa390a00640 1 -- 172.21.15.7:0/1739533696 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38c007540 con 0x7fa39406fab0 2024-04-25T11:11:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.608+0000 7fa390a00640 1 -- 172.21.15.7:0/1739533696 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c007910 con 0x7fa39406fab0 2024-04-25T11:11:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.608+0000 7fa398e00640 1 -- 172.21.15.7:0/1739533696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa39406fab0 msgr2=0x7fa394168e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.608+0000 7fa398e00640 1 --2- 172.21.15.7:0/1739533696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa39406fab0 0x7fa394168e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa3940ceb10 tx=0x7fa38c02a680 comp rx=0 tx=0).stop 2024-04-25T11:11:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.608+0000 7fa398e00640 1 -- 172.21.15.7:0/1739533696 shutdown_connections 2024-04-25T11:11:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.608+0000 7fa398e00640 1 --2- 172.21.15.7:0/1739533696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa39406fab0 0x7fa394168e60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.609+0000 7fa398e00640 1 -- 172.21.15.7:0/1739533696 >> 172.21.15.7:0/1739533696 conn(0x7fa3940742c0 msgr2=0x7fa39406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.609+0000 7fa398e00640 1 -- 172.21.15.7:0/1739533696 shutdown_connections 2024-04-25T11:11:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.609+0000 7fa398e00640 1 -- 172.21.15.7:0/1739533696 wait complete. 2024-04-25T11:11:04.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.611+0000 7fa398e00640 1 Processor -- start 2024-04-25T11:11:04.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.611+0000 7fa398e00640 1 -- start start 2024-04-25T11:11:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.612+0000 7fa398e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa39406fab0 0x7fa3940cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa398e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3940b0470 0x7fa3940cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa398e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa39415dfb0 0x7fa3941f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa398e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3940d0340 con 0x7fa39406fab0 2024-04-25T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa398e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa3940d0060 con 0x7fa3940b0470 2024-04-25T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa398e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa3940d0620 con 0x7fa39415dfb0 2024-04-25T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa391e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa39406fab0 0x7fa3940cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa391e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa39406fab0 0x7fa3940cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43492/0 (socket says 172.21.15.7:43492) 2024-04-25T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa391e00640 1 -- 172.21.15.7:0/551307311 learned_addr learned my addr 172.21.15.7:0/551307311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa391e00640 1 -- 172.21.15.7:0/551307311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa39415dfb0 msgr2=0x7fa3941f9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa391e00640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa39415dfb0 0x7fa3941f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa391e00640 1 -- 172.21.15.7:0/551307311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3940b0470 msgr2=0x7fa3940cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa391400640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3940b0470 0x7fa3940cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa391e00640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3940b0470 0x7fa3940cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa391e00640 1 -- 172.21.15.7:0/551307311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa394159360 con 0x7fa39406fab0 2024-04-25T11:11:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.613+0000 7fa391400640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3940b0470 0x7fa3940cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.614+0000 7fa391e00640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa39406fab0 0x7fa3940cb610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa3840011e0 tx=0x7fa38400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.614+0000 7fa38a000640 1 -- 172.21.15.7:0/551307311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384017070 con 0x7fa39406fab0 2024-04-25T11:11:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.614+0000 7fa38a000640 1 -- 172.21.15.7:0/551307311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa384005ce0 con 0x7fa39406fab0 2024-04-25T11:11:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.614+0000 7fa38a000640 1 -- 172.21.15.7:0/551307311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3840072b0 con 0x7fa39406fab0 2024-04-25T11:11:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.614+0000 7fa398e00640 1 -- 172.21.15.7:0/551307311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa394159640 con 0x7fa39406fab0 2024-04-25T11:11:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.614+0000 7fa398e00640 1 -- 172.21.15.7:0/551307311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa394159c00 con 0x7fa39406fab0 2024-04-25T11:11:04.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.615+0000 7fa398e00640 1 -- 172.21.15.7:0/551307311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa394050740 con 0x7fa39406fab0 2024-04-25T11:11:04.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.616+0000 7fa38a000640 1 -- 172.21.15.7:0/551307311 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa38401a070 con 0x7fa39406fab0 2024-04-25T11:11:04.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.617+0000 7fa38a000640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa374074530 0x7fa374076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:04.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.617+0000 7fa38a000640 1 -- 172.21.15.7:0/551307311 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fa3840948c0 con 0x7fa39406fab0 2024-04-25T11:11:04.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.618+0000 7fa391400640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa374074530 0x7fa374076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:04.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.618+0000 7fa391400640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa374074530 0x7fa374076a40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa380000f60 tx=0x7fa380009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.622+0000 7fa38a000640 1 -- 172.21.15.7:0/551307311 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa38405e670 con 0x7fa39406fab0 2024-04-25T11:11:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.829+0000 7fa398e00640 1 -- 172.21.15.7:0/551307311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa3940b0910 con 0x7fa39406fab0 2024-04-25T11:11:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.830+0000 7fa38a000640 1 -- 172.21.15.7:0/551307311 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fa384062970 con 0x7fa39406fab0 2024-04-25T11:11:04.830 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:04.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 -- 172.21.15.7:0/551307311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa374074530 msgr2=0x7fa374076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa374074530 0x7fa374076a40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa380000f60 tx=0x7fa380009040 comp rx=0 tx=0).stop 2024-04-25T11:11:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 -- 172.21.15.7:0/551307311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa39406fab0 msgr2=0x7fa3940cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa39406fab0 0x7fa3940cb610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa3840011e0 tx=0x7fa38400c960 comp rx=0 tx=0).stop 2024-04-25T11:11:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 -- 172.21.15.7:0/551307311 shutdown_connections 2024-04-25T11:11:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa374074530 0x7fa374076a40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa39415dfb0 0x7fa3941f9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa3940b0470 0x7fa3940cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 --2- 172.21.15.7:0/551307311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa39406fab0 0x7fa3940cb610 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 -- 172.21.15.7:0/551307311 >> 172.21.15.7:0/551307311 conn(0x7fa3940742c0 msgr2=0x7fa39406cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 -- 172.21.15.7:0/551307311 shutdown_connections 2024-04-25T11:11:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:04.833+0000 7fa398e00640 1 -- 172.21.15.7:0/551307311 wait complete. 2024-04-25T11:11:04.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.146+0000 7fc8c8e00640 1 Processor -- start 2024-04-25T11:11:05.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.146+0000 7fc8c8e00640 1 -- start start 2024-04-25T11:11:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.148+0000 7fc8c8e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c41602e0 0x7fc8c406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.148+0000 7fc8c8e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc8c406e8b0 0x7fc8c40d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.148+0000 7fc8c8e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c406ecc0 con 0x7fc8c40d4ca0 2024-04-25T11:11:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.148+0000 7fc8c8e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c40b9060 con 0x7fc8c406e8b0 2024-04-25T11:11:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.148+0000 7fc8c8e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c40d5500 con 0x7fc8c41602e0 2024-04-25T11:11:05.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.149+0000 7fc8c3400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c41602e0 0x7fc8c406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.149+0000 7fc8c3400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c41602e0 0x7fc8c406e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:55110/0 (socket says 172.21.15.7:55110) 2024-04-25T11:11:05.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.149+0000 7fc8c3400640 1 -- 172.21.15.7:0/3354407092 learned_addr learned my addr 172.21.15.7:0/3354407092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:05.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.149+0000 7fc8c2a00640 1 --2- 172.21.15.7:0/3354407092 >> v2:172.21.15.87:3300/0 conn(0x7fc8c406e8b0 0x7fc8c40d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.150+0000 7fc8c3400640 1 -- 172.21.15.7:0/3354407092 >> v2:172.21.15.87:3300/0 conn(0x7fc8c406e8b0 msgr2=0x7fc8c40d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:05.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.150+0000 7fc8c3400640 1 --2- 172.21.15.7:0/3354407092 >> v2:172.21.15.87:3300/0 conn(0x7fc8c406e8b0 0x7fc8c40d4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.150+0000 7fc8c3400640 1 -- 172.21.15.7:0/3354407092 >> v1:172.21.15.87:6789/0 conn(0x7fc8c40d4ca0 legacy=0x7fc8c40b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.150+0000 7fc8c3400640 1 -- 172.21.15.7:0/3354407092 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c40b9300 con 0x7fc8c41602e0 2024-04-25T11:11:05.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.150+0000 7fc8c2000640 1 -- 172.21.15.7:0/3354407092 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2765366084 0 0) 0x7fc8bc001490 con 0x7fc8c40d4ca0 2024-04-25T11:11:05.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.151+0000 7fc8c3400640 1 --2- 172.21.15.7:0/3354407092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c41602e0 0x7fc8c406e370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009870 tx=0x7fc8b402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e38a1daf7329d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:05.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.152+0000 7fc8c2000640 1 -- 172.21.15.7:0/3354407092 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4037070 con 0x7fc8c41602e0 2024-04-25T11:11:05.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.152+0000 7fc8c2000640 1 -- 172.21.15.7:0/3354407092 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b4032440 con 0x7fc8c41602e0 2024-04-25T11:11:05.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.152+0000 7fc8c2000640 1 -- 172.21.15.7:0/3354407092 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4032810 con 0x7fc8c41602e0 2024-04-25T11:11:05.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.153+0000 7fc8c8e00640 1 -- 172.21.15.7:0/3354407092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c41602e0 msgr2=0x7fc8c406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.153+0000 7fc8c8e00640 1 --2- 172.21.15.7:0/3354407092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c41602e0 0x7fc8c406e370 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009870 tx=0x7fc8b402a270 comp rx=0 tx=0).stop 2024-04-25T11:11:05.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.153+0000 7fc8c8e00640 1 -- 172.21.15.7:0/3354407092 shutdown_connections 2024-04-25T11:11:05.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.153+0000 7fc8c8e00640 1 --2- 172.21.15.7:0/3354407092 >> v2:172.21.15.87:3300/0 conn(0x7fc8c406e8b0 0x7fc8c40d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.153+0000 7fc8c8e00640 1 --2- 172.21.15.7:0/3354407092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c41602e0 0x7fc8c406e370 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.153+0000 7fc8c8e00640 1 -- 172.21.15.7:0/3354407092 >> 172.21.15.7:0/3354407092 conn(0x7fc8c40d2670 msgr2=0x7fc8c40d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:05.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.153+0000 7fc8c8e00640 1 -- 172.21.15.7:0/3354407092 shutdown_connections 2024-04-25T11:11:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.153+0000 7fc8c8e00640 1 -- 172.21.15.7:0/3354407092 wait complete. 2024-04-25T11:11:05.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.156+0000 7fc8c8e00640 1 Processor -- start 2024-04-25T11:11:05.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.156+0000 7fc8c8e00640 1 -- start start 2024-04-25T11:11:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.157+0000 7fc8c8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8c406e8b0 0x7fc8c40cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.157+0000 7fc8c8e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c40d4ca0 0x7fc8c4150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.157+0000 7fc8c8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c41602e0 0x7fc8c4150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.157+0000 7fc8c8e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c40d50b0 con 0x7fc8c406e8b0 2024-04-25T11:11:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.157+0000 7fc8c8e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c40b98c0 con 0x7fc8c41602e0 2024-04-25T11:11:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.157+0000 7fc8c8e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c40d5230 con 0x7fc8c40d4ca0 2024-04-25T11:11:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8c406e8b0 0x7fc8c40cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8c406e8b0 0x7fc8c40cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43506/0 (socket says 172.21.15.7:43506) 2024-04-25T11:11:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c3400640 1 -- 172.21.15.7:0/78066116 learned_addr learned my addr 172.21.15.7:0/78066116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c2a00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c40d4ca0 0x7fc8c4150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c3400640 1 -- 172.21.15.7:0/78066116 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c40d4ca0 msgr2=0x7fc8c4150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c3400640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c40d4ca0 0x7fc8c4150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c3e00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c41602e0 0x7fc8c4150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c3400640 1 -- 172.21.15.7:0/78066116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c41602e0 msgr2=0x7fc8c4150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c3400640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c41602e0 0x7fc8c4150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c3400640 1 -- 172.21.15.7:0/78066116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c40ce7a0 con 0x7fc8c406e8b0 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c2a00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c40d4ca0 0x7fc8c4150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.158+0000 7fc8c3e00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c41602e0 0x7fc8c4150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.159+0000 7fc8c3400640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8c406e8b0 0x7fc8c40cdf50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009870 tx=0x7fc8b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.159+0000 7fc8bb400640 1 -- 172.21.15.7:0/78066116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4041070 con 0x7fc8c406e8b0 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.159+0000 7fc8bb400640 1 -- 172.21.15.7:0/78066116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b403c870 con 0x7fc8c406e8b0 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.159+0000 7fc8bb400640 1 -- 172.21.15.7:0/78066116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4037040 con 0x7fc8c406e8b0 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.159+0000 7fc8c8e00640 1 -- 172.21.15.7:0/78066116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c40cea80 con 0x7fc8c406e8b0 2024-04-25T11:11:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.159+0000 7fc8c8e00640 1 -- 172.21.15.7:0/78066116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c40cf040 con 0x7fc8c406e8b0 2024-04-25T11:11:05.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.160+0000 7fc8c8e00640 1 -- 172.21.15.7:0/78066116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c4050740 con 0x7fc8c406e8b0 2024-04-25T11:11:05.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.162+0000 7fc8bb400640 1 -- 172.21.15.7:0/78066116 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc8b4044070 con 0x7fc8c406e8b0 2024-04-25T11:11:05.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.163+0000 7fc8bb400640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8a40744b0 0x7fc8a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.163+0000 7fc8bb400640 1 -- 172.21.15.7:0/78066116 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fc8b40b5f10 con 0x7fc8c406e8b0 2024-04-25T11:11:05.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.163+0000 7fc8c2a00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8a40744b0 0x7fc8a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.164+0000 7fc8c2a00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8a40744b0 0x7fc8a40769c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40cdea0 tx=0x7fc8b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:05.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.168+0000 7fc8bb400640 1 -- 172.21.15.7:0/78066116 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc8b4080330 con 0x7fc8c406e8b0 2024-04-25T11:11:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.376+0000 7fc8c8e00640 1 -- 172.21.15.7:0/78066116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc8c4069c50 con 0x7fc8c406e8b0 2024-04-25T11:11:05.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.377+0000 7fc8bb400640 1 -- 172.21.15.7:0/78066116 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fc8b407fc30 con 0x7fc8c406e8b0 2024-04-25T11:11:05.378 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.380+0000 7fc8c8e00640 1 -- 172.21.15.7:0/78066116 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8a40744b0 msgr2=0x7fc8a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.380+0000 7fc8c8e00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8a40744b0 0x7fc8a40769c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40cdea0 tx=0x7fc8b0008040 comp rx=0 tx=0).stop 2024-04-25T11:11:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.380+0000 7fc8c8e00640 1 -- 172.21.15.7:0/78066116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8c406e8b0 msgr2=0x7fc8c40cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.380+0000 7fc8c8e00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8c406e8b0 0x7fc8c40cdf50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009870 tx=0x7fc8b40046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.380+0000 7fc8c8e00640 1 -- 172.21.15.7:0/78066116 shutdown_connections 2024-04-25T11:11:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.380+0000 7fc8c8e00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8a40744b0 0x7fc8a40769c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.380+0000 7fc8c8e00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8c41602e0 0x7fc8c4150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.380+0000 7fc8c8e00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8c40d4ca0 0x7fc8c4150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.380+0000 7fc8c8e00640 1 --2- 172.21.15.7:0/78066116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8c406e8b0 0x7fc8c40cdf50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.380+0000 7fc8c8e00640 1 -- 172.21.15.7:0/78066116 >> 172.21.15.7:0/78066116 conn(0x7fc8c40d2670 msgr2=0x7fc8c415a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.381+0000 7fc8c8e00640 1 -- 172.21.15.7:0/78066116 shutdown_connections 2024-04-25T11:11:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.381+0000 7fc8c8e00640 1 -- 172.21.15.7:0/78066116 wait complete. 2024-04-25T11:11:05.410 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.699+0000 7fc65a800640 1 Processor -- start 2024-04-25T11:11:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.699+0000 7fc65a800640 1 -- start start 2024-04-25T11:11:05.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.702+0000 7fc65a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc6540d3cb0 0x7fc6540d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.702+0000 7fc65a800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6540d1a70 con 0x7fc6540ce400 2024-04-25T11:11:05.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.702+0000 7fc65a800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6540d28a0 con 0x7fc6540d1660 2024-04-25T11:11:05.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.702+0000 7fc65a800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc6540d1d20 con 0x7fc6540d3cb0 2024-04-25T11:11:05.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.702+0000 7fc659400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc6540ce400 0x7fc6540b2980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33844/0 (socket says 172.21.15.7:33844) 2024-04-25T11:11:05.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.702+0000 7fc659400640 1 -- 172.21.15.7:0/1910102485 learned_addr learned my addr 172.21.15.7:0/1910102485 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:05.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.702+0000 7fc659e00640 1 --2- 172.21.15.7:0/1910102485 >> v2:172.21.15.87:3300/0 conn(0x7fc6540d3cb0 0x7fc6540d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1656516589 0 0) 0x7fc6540d1a70 con 0x7fc6540ce400 2024-04-25T11:11:05.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc63c003520 con 0x7fc6540ce400 2024-04-25T11:11:05.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3663897895 0 0) 0x7fc63c003520 con 0x7fc6540ce400 2024-04-25T11:11:05.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 >> v2:172.21.15.87:3300/0 conn(0x7fc6540d3cb0 msgr2=0x7fc6540d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 --2- 172.21.15.7:0/1910102485 >> v2:172.21.15.87:3300/0 conn(0x7fc6540d3cb0 0x7fc6540d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 >> v1:172.21.15.87:6789/0 conn(0x7fc6540d1660 legacy=0x7fc6540d35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6540d2000 con 0x7fc6540ce400 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fc648001b70 con 0x7fc6540ce400 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 >> v1:172.21.15.7:6789/0 conn(0x7fc6540ce400 legacy=0x7fc6540b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 --2- 172.21.15.7:0/1910102485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc63c0039b0 0x7fc63c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 --2- 172.21.15.7:0/1910102485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc63c004800 0x7fc63c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 --2- 172.21.15.7:0/1910102485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc63c005250 0x7fc63c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc6540d1d20 con 0x7fc63c004800 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c00c1e0 con 0x7fc63c0039b0 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.703+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc63c00c3e0 con 0x7fc63c005250 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.704+0000 7fc658a00640 1 --2- 172.21.15.7:0/1910102485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc63c004800 0x7fc63c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.704+0000 7fc659400640 1 --2- 172.21.15.7:0/1910102485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc63c0039b0 0x7fc63c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.704+0000 7fc659e00640 1 --2- 172.21.15.7:0/1910102485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc63c005250 0x7fc63c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.704+0000 7fc658a00640 1 -- 172.21.15.7:0/1910102485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc63c005250 msgr2=0x7fc63c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:05.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.704+0000 7fc658a00640 1 --2- 172.21.15.7:0/1910102485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc63c005250 0x7fc63c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.704+0000 7fc658a00640 1 -- 172.21.15.7:0/1910102485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc63c0039b0 msgr2=0x7fc63c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:05.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.704+0000 7fc658a00640 1 --2- 172.21.15.7:0/1910102485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc63c0039b0 0x7fc63c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.704+0000 7fc658a00640 1 -- 172.21.15.7:0/1910102485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c003520 con 0x7fc63c004800 2024-04-25T11:11:05.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.704+0000 7fc659e00640 1 -- 172.21.15.7:0/1910102485 reap_dead start 2024-04-25T11:11:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.705+0000 7fc658a00640 1 --2- 172.21.15.7:0/1910102485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc63c004800 0x7fc63c004c10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc640001750 tx=0x7fc63c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26741bb087bf52c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.705+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640001370 con 0x7fc63c004800 2024-04-25T11:11:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.705+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6400052d0 con 0x7fc63c004800 2024-04-25T11:11:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.705+0000 7fc653e00640 1 -- 172.21.15.7:0/1910102485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64002abc0 con 0x7fc63c004800 2024-04-25T11:11:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.706+0000 7fc65a800640 1 -- 172.21.15.7:0/1910102485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc63c004800 msgr2=0x7fc63c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.706+0000 7fc65a800640 1 --2- 172.21.15.7:0/1910102485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc63c004800 0x7fc63c004c10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc640001750 tx=0x7fc63c005830 comp rx=0 tx=0).stop 2024-04-25T11:11:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.706+0000 7fc65a800640 1 -- 172.21.15.7:0/1910102485 shutdown_connections 2024-04-25T11:11:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.706+0000 7fc65a800640 1 --2- 172.21.15.7:0/1910102485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc63c004800 0x7fc63c004c10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.706+0000 7fc65a800640 1 -- 172.21.15.7:0/1910102485 >> 172.21.15.7:0/1910102485 conn(0x7fc65408ce10 msgr2=0x7fc65408d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.706+0000 7fc65a800640 1 -- 172.21.15.7:0/1910102485 shutdown_connections 2024-04-25T11:11:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.707+0000 7fc65a800640 1 -- 172.21.15.7:0/1910102485 wait complete. 2024-04-25T11:11:05.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.709+0000 7fc65a800640 1 Processor -- start 2024-04-25T11:11:05.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.709+0000 7fc65a800640 1 -- start start 2024-04-25T11:11:05.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.711+0000 7fc65a800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc63c004800 0x7fc65415da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.711+0000 7fc65a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc654160060 0x7fc654160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.711+0000 7fc65a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406ed40 0x7fc65406f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.711+0000 7fc65a800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc6540d0c20 con 0x7fc65406ed40 2024-04-25T11:11:05.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.711+0000 7fc65a800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc6540d25c0 con 0x7fc654160060 2024-04-25T11:11:05.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.711+0000 7fc65a800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc6540d0da0 con 0x7fc63c004800 2024-04-25T11:11:05.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.711+0000 7fc659e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406ed40 0x7fc65406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.711+0000 7fc659e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406ed40 0x7fc65406f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43534/0 (socket says 172.21.15.7:43534) 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.711+0000 7fc659e00640 1 -- 172.21.15.7:0/3822299763 learned_addr learned my addr 172.21.15.7:0/3822299763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.711+0000 7fc659400640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc63c004800 0x7fc65415da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.711+0000 7fc658a00640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc654160060 0x7fc654160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.712+0000 7fc659e00640 1 -- 172.21.15.7:0/3822299763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc63c004800 msgr2=0x7fc65415da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.712+0000 7fc659e00640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc63c004800 0x7fc65415da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.712+0000 7fc659e00640 1 -- 172.21.15.7:0/3822299763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc654160060 msgr2=0x7fc654160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.712+0000 7fc659e00640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc654160060 0x7fc654160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.712+0000 7fc659e00640 1 -- 172.21.15.7:0/3822299763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65406f690 con 0x7fc65406ed40 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.712+0000 7fc659e00640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406ed40 0x7fc65406f150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc6540d3030 tx=0x7fc64c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.712+0000 7fc651600640 1 -- 172.21.15.7:0/3822299763 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c004440 con 0x7fc65406ed40 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.712+0000 7fc651600640 1 -- 172.21.15.7:0/3822299763 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc64c01b070 con 0x7fc65406ed40 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.712+0000 7fc651600640 1 -- 172.21.15.7:0/3822299763 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c003850 con 0x7fc65406ed40 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.712+0000 7fc65a800640 1 -- 172.21.15.7:0/3822299763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6541f99f0 con 0x7fc65406ed40 2024-04-25T11:11:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.712+0000 7fc65a800640 1 -- 172.21.15.7:0/3822299763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6541f9fb0 con 0x7fc65406ed40 2024-04-25T11:11:05.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.714+0000 7fc65a800640 1 -- 172.21.15.7:0/3822299763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc654050740 con 0x7fc65406ed40 2024-04-25T11:11:05.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.715+0000 7fc651600640 1 -- 172.21.15.7:0/3822299763 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc64c0039f0 con 0x7fc65406ed40 2024-04-25T11:11:05.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.716+0000 7fc651600640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc6340744b0 0x7fc6340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:05.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.716+0000 7fc651600640 1 -- 172.21.15.7:0/3822299763 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fc64c095510 con 0x7fc65406ed40 2024-04-25T11:11:05.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.716+0000 7fc659400640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc6340744b0 0x7fc6340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:05.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.717+0000 7fc659400640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc6340744b0 0x7fc6340769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc648001700 tx=0x7fc648009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:05.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.721+0000 7fc651600640 1 -- 172.21.15.7:0/3822299763 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc64c024020 con 0x7fc65406ed40 2024-04-25T11:11:05.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.926+0000 7fc65a800640 1 -- 172.21.15.7:0/3822299763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc6540cc580 con 0x7fc65406ed40 2024-04-25T11:11:05.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.927+0000 7fc651600640 1 -- 172.21.15.7:0/3822299763 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fc65406f690 con 0x7fc65406ed40 2024-04-25T11:11:05.927 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.929+0000 7fc65a800640 1 -- 172.21.15.7:0/3822299763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc6340744b0 msgr2=0x7fc6340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.929+0000 7fc65a800640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc6340744b0 0x7fc6340769c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc648001700 tx=0x7fc648009040 comp rx=0 tx=0).stop 2024-04-25T11:11:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.929+0000 7fc65a800640 1 -- 172.21.15.7:0/3822299763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406ed40 msgr2=0x7fc65406f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.929+0000 7fc65a800640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406ed40 0x7fc65406f150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc6540d3030 tx=0x7fc64c009f90 comp rx=0 tx=0).stop 2024-04-25T11:11:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.929+0000 7fc65a800640 1 -- 172.21.15.7:0/3822299763 shutdown_connections 2024-04-25T11:11:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.929+0000 7fc65a800640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc6340744b0 0x7fc6340769c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.929+0000 7fc65a800640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc65406ed40 0x7fc65406f150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.929+0000 7fc65a800640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc654160060 0x7fc654160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.930+0000 7fc65a800640 1 --2- 172.21.15.7:0/3822299763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc63c004800 0x7fc65415da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:05.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.930+0000 7fc65a800640 1 -- 172.21.15.7:0/3822299763 >> 172.21.15.7:0/3822299763 conn(0x7fc65408ce10 msgr2=0x7fc6540b8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:05.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.930+0000 7fc65a800640 1 -- 172.21.15.7:0/3822299763 shutdown_connections 2024-04-25T11:11:05.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:05.930+0000 7fc65a800640 1 -- 172.21.15.7:0/3822299763 wait complete. 2024-04-25T11:11:05.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:11:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.238+0000 7f1873e00640 1 Processor -- start 2024-04-25T11:11:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.239+0000 7f1873e00640 1 -- start start 2024-04-25T11:11:06.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.241+0000 7f1873e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f187406ffd0 0x7f1874164380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.241+0000 7f1873e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18740af390 0x7f18740af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.241+0000 7f1873e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f18740b5800 0x7f18740b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.241+0000 7f1873e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f18740b47d0 con 0x7f187406ffd0 2024-04-25T11:11:06.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.241+0000 7f1873e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f18740b4a40 con 0x7f18740b5800 2024-04-25T11:11:06.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.241+0000 7f1873e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18740b4d20 con 0x7f18740af390 2024-04-25T11:11:06.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.241+0000 7f1872a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f187406ffd0 0x7f1874164380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.241+0000 7f1872a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f187406ffd0 0x7f1874164380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43546/0 (socket says 172.21.15.7:43546) 2024-04-25T11:11:06.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.241+0000 7f1872a00640 1 -- 172.21.15.7:0/1992081731 learned_addr learned my addr 172.21.15.7:0/1992081731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:06.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.242+0000 7f1873400640 1 --2- 172.21.15.7:0/1992081731 >> v2:172.21.15.87:3300/0 conn(0x7f18740b5800 0x7f18740b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.242+0000 7f186a000640 1 --2- 172.21.15.7:0/1992081731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18740af390 0x7f18740af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.242+0000 7f1872a00640 1 -- 172.21.15.7:0/1992081731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18740af390 msgr2=0x7f18740af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:06.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.242+0000 7f1872a00640 1 --2- 172.21.15.7:0/1992081731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18740af390 0x7f18740af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.242+0000 7f1872a00640 1 -- 172.21.15.7:0/1992081731 >> v2:172.21.15.87:3300/0 conn(0x7f18740b5800 msgr2=0x7f18740b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:06.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.242+0000 7f1872a00640 1 --2- 172.21.15.7:0/1992081731 >> v2:172.21.15.87:3300/0 conn(0x7f18740b5800 0x7f18740b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.242+0000 7f1872a00640 1 -- 172.21.15.7:0/1992081731 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18740b5000 con 0x7f187406ffd0 2024-04-25T11:11:06.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.243+0000 7f1872a00640 1 --2- 172.21.15.7:0/1992081731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187406ffd0 0x7f1874164380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f187406ef00 tx=0x7f186402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dee532d7b76b0c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:06.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.243+0000 7f1872000640 1 -- 172.21.15.7:0/1992081731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864037070 con 0x7f187406ffd0 2024-04-25T11:11:06.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.243+0000 7f1872000640 1 -- 172.21.15.7:0/1992081731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1864032440 con 0x7f187406ffd0 2024-04-25T11:11:06.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.243+0000 7f1872000640 1 -- 172.21.15.7:0/1992081731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864032810 con 0x7f187406ffd0 2024-04-25T11:11:06.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.244+0000 7f1873e00640 1 -- 172.21.15.7:0/1992081731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187406ffd0 msgr2=0x7f1874164380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:06.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.244+0000 7f1873e00640 1 --2- 172.21.15.7:0/1992081731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187406ffd0 0x7f1874164380 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f187406ef00 tx=0x7f186402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.244+0000 7f1873e00640 1 -- 172.21.15.7:0/1992081731 shutdown_connections 2024-04-25T11:11:06.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.244+0000 7f1873e00640 1 --2- 172.21.15.7:0/1992081731 >> v2:172.21.15.87:3300/0 conn(0x7f18740b5800 0x7f18740b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.244+0000 7f1873e00640 1 --2- 172.21.15.7:0/1992081731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18740af390 0x7f18740af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.244+0000 7f1873e00640 1 --2- 172.21.15.7:0/1992081731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187406ffd0 0x7f1874164380 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.244+0000 7f1873e00640 1 -- 172.21.15.7:0/1992081731 >> 172.21.15.7:0/1992081731 conn(0x7f18740b3820 msgr2=0x7f18740b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:06.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.244+0000 7f1873e00640 1 -- 172.21.15.7:0/1992081731 shutdown_connections 2024-04-25T11:11:06.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.245+0000 7f1873e00640 1 -- 172.21.15.7:0/1992081731 wait complete. 2024-04-25T11:11:06.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.247+0000 7f1873e00640 1 Processor -- start 2024-04-25T11:11:06.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.247+0000 7f1873e00640 1 -- start start 2024-04-25T11:11:06.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.248+0000 7f1873e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187406ffd0 0x7f18741f0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.248+0000 7f1873e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18740af390 0x7f1874157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.248+0000 7f1873e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18740b5800 0x7f18741ee780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.248+0000 7f1873e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f187406fbf0 con 0x7f187406ffd0 2024-04-25T11:11:06.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.248+0000 7f1873e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f187406fa70 con 0x7f18740af390 2024-04-25T11:11:06.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.248+0000 7f1873e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18740b6150 con 0x7f18740b5800 2024-04-25T11:11:06.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f1872a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187406ffd0 0x7f18741f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f1872a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187406ffd0 0x7f18741f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43552/0 (socket says 172.21.15.7:43552) 2024-04-25T11:11:06.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f1872a00640 1 -- 172.21.15.7:0/3511224438 learned_addr learned my addr 172.21.15.7:0/3511224438 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:06.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f186a000640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18740af390 0x7f1874157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f1873400640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18740b5800 0x7f18741ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f1872a00640 1 -- 172.21.15.7:0/3511224438 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18740b5800 msgr2=0x7f18741ee780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:06.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f1872a00640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18740b5800 0x7f18741ee780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f1872a00640 1 -- 172.21.15.7:0/3511224438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18740af390 msgr2=0x7f1874157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:06.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f1872a00640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18740af390 0x7f1874157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f1872a00640 1 -- 172.21.15.7:0/3511224438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18741ef960 con 0x7f187406ffd0 2024-04-25T11:11:06.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f186a000640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18740af390 0x7f1874157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:06.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.249+0000 7f1872a00640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187406ffd0 0x7f18741f0720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f186402a220 tx=0x7f18640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:06.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.250+0000 7f186be00640 1 -- 172.21.15.7:0/3511224438 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864041020 con 0x7f187406ffd0 2024-04-25T11:11:06.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.250+0000 7f186be00640 1 -- 172.21.15.7:0/3511224438 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18640306c0 con 0x7f187406ffd0 2024-04-25T11:11:06.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.250+0000 7f1873e00640 1 -- 172.21.15.7:0/3511224438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18741efc40 con 0x7f187406ffd0 2024-04-25T11:11:06.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.250+0000 7f186be00640 1 -- 172.21.15.7:0/3511224438 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864037040 con 0x7f187406ffd0 2024-04-25T11:11:06.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.250+0000 7f1873e00640 1 -- 172.21.15.7:0/3511224438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18741661b0 con 0x7f187406ffd0 2024-04-25T11:11:06.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.251+0000 7f1873e00640 1 -- 172.21.15.7:0/3511224438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1874050740 con 0x7f187406ffd0 2024-04-25T11:11:06.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.252+0000 7f186be00640 1 -- 172.21.15.7:0/3511224438 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1864044020 con 0x7f187406ffd0 2024-04-25T11:11:06.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.254+0000 7f186be00640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18500744b0 0x7f18500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.254+0000 7f186be00640 1 -- 172.21.15.7:0/3511224438 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f18640b5f90 con 0x7f187406ffd0 2024-04-25T11:11:06.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.254+0000 7f186a000640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18500744b0 0x7f18500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.255+0000 7f186a000640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18500744b0 0x7f18500769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f185c006fd0 tx=0x7f185c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:06.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.259+0000 7f186be00640 1 -- 172.21.15.7:0/3511224438 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f186404b030 con 0x7f187406ffd0 2024-04-25T11:11:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.492+0000 7f1873e00640 1 -- 172.21.15.7:0/3511224438 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18740b3e80 con 0x7f187406ffd0 2024-04-25T11:11:06.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.493+0000 7f186be00640 1 -- 172.21.15.7:0/3511224438 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7f1864080280 con 0x7f187406ffd0 2024-04-25T11:11:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:06.494 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3458223722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3458223722},{"type":"v1","addr":"172.21.15.87:6835","nonce":3458223722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1383067804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1383067804},{"type":"v1","addr":"172.21.15.7:6835","nonce":1383067804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/435824250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":435824250},{"type":"v1","addr":"172.21.15.7:6837","nonce":435824250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:10:23.023233+0000","modified":"2024-04-25T11:10:53.526222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1091275258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1091275258},{"type":"v1","addr":"172.21.15.87:6834","nonce":1091275258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:11:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 -- 172.21.15.7:0/3511224438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18500744b0 msgr2=0x7f18500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18500744b0 0x7f18500769c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f185c006fd0 tx=0x7f185c008040 comp rx=0 tx=0).stop 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 -- 172.21.15.7:0/3511224438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187406ffd0 msgr2=0x7f18741f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187406ffd0 0x7f18741f0720 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f186402a220 tx=0x7f18640046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 -- 172.21.15.7:0/3511224438 shutdown_connections 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18500744b0 0x7f18500769c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18740b5800 0x7f18741ee780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18740af390 0x7f1874157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 --2- 172.21.15.7:0/3511224438 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f187406ffd0 0x7f18741f0720 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 -- 172.21.15.7:0/3511224438 >> 172.21.15.7:0/3511224438 conn(0x7f18740b3820 msgr2=0x7f18740b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 -- 172.21.15.7:0/3511224438 shutdown_connections 2024-04-25T11:11:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.496+0000 7f1873e00640 1 -- 172.21.15.7:0/3511224438 wait complete. 2024-04-25T11:11:06.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:06.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.808+0000 7ff12aa00640 1 Processor -- start 2024-04-25T11:11:06.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.808+0000 7ff12aa00640 1 -- start start 2024-04-25T11:11:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.811+0000 7ff12aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1240b3960 0x7ff1240b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.811+0000 7ff12aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1240b08b0 0x7ff1240b7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.811+0000 7ff12aa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1240d2990 con 0x7ff1240b3d70 2024-04-25T11:11:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.811+0000 7ff12aa00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff1240d2c50 con 0x7ff1240b08b0 2024-04-25T11:11:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.811+0000 7ff12aa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff1240d2f30 con 0x7ff1240b3960 2024-04-25T11:11:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.811+0000 7ff12a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1240b08b0 0x7ff1240b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.811+0000 7ff12a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1240b08b0 0x7ff1240b7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43560/0 (socket says 172.21.15.7:43560) 2024-04-25T11:11:06.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.811+0000 7ff12a000640 1 -- 172.21.15.7:0/2338894568 learned_addr learned my addr 172.21.15.7:0/2338894568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:06.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.811+0000 7ff129600640 1 --2- 172.21.15.7:0/2338894568 >> v2:172.21.15.87:3300/0 conn(0x7ff1240b3960 0x7ff1240b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.812+0000 7ff12a000640 1 -- 172.21.15.7:0/2338894568 >> v2:172.21.15.87:3300/0 conn(0x7ff1240b3960 msgr2=0x7ff1240b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:06.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.812+0000 7ff12a000640 1 --2- 172.21.15.7:0/2338894568 >> v2:172.21.15.87:3300/0 conn(0x7ff1240b3960 0x7ff1240b73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.812+0000 7ff12a000640 1 -- 172.21.15.7:0/2338894568 >> v1:172.21.15.87:6789/0 conn(0x7ff1240b3d70 legacy=0x7ff1240d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:06.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.812+0000 7ff12a000640 1 -- 172.21.15.7:0/2338894568 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1240d3210 con 0x7ff1240b08b0 2024-04-25T11:11:06.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.813+0000 7ff12a000640 1 --2- 172.21.15.7:0/2338894568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1240b08b0 0x7ff1240b7910 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff1240ccc30 tx=0x7ff11c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58f5f015912025c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:06.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.813+0000 7ff123e00640 1 -- 172.21.15.7:0/2338894568 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c037070 con 0x7ff1240b08b0 2024-04-25T11:11:06.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.813+0000 7ff123e00640 1 -- 172.21.15.7:0/2338894568 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c032440 con 0x7ff1240b08b0 2024-04-25T11:11:06.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.813+0000 7ff123e00640 1 -- 172.21.15.7:0/2338894568 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c032810 con 0x7ff1240b08b0 2024-04-25T11:11:06.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.814+0000 7ff12aa00640 1 -- 172.21.15.7:0/2338894568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1240b08b0 msgr2=0x7ff1240b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:06.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.814+0000 7ff12aa00640 1 --2- 172.21.15.7:0/2338894568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1240b08b0 0x7ff1240b7910 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff1240ccc30 tx=0x7ff11c02a230 comp rx=0 tx=0).stop 2024-04-25T11:11:06.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.814+0000 7ff12aa00640 1 -- 172.21.15.7:0/2338894568 shutdown_connections 2024-04-25T11:11:06.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.814+0000 7ff12aa00640 1 --2- 172.21.15.7:0/2338894568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1240b08b0 0x7ff1240b7910 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.814+0000 7ff12aa00640 1 --2- 172.21.15.7:0/2338894568 >> v2:172.21.15.87:3300/0 conn(0x7ff1240b3960 0x7ff1240b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.814+0000 7ff12aa00640 1 -- 172.21.15.7:0/2338894568 >> 172.21.15.7:0/2338894568 conn(0x7ff12409b540 msgr2=0x7ff12409b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:06.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.814+0000 7ff12aa00640 1 -- 172.21.15.7:0/2338894568 shutdown_connections 2024-04-25T11:11:06.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.814+0000 7ff12aa00640 1 -- 172.21.15.7:0/2338894568 wait complete. 2024-04-25T11:11:06.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.816+0000 7ff12aa00640 1 Processor -- start 2024-04-25T11:11:06.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.817+0000 7ff12aa00640 1 -- start start 2024-04-25T11:11:06.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.818+0000 7ff12aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1240b08b0 0x7ff1240b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.818+0000 7ff12aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1240b3960 0x7ff1240cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.818+0000 7ff12aa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1240b3d70 0x7ff1240cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.818+0000 7ff12aa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff12406eb10 con 0x7ff1240b3960 2024-04-25T11:11:06.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.818+0000 7ff12aa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff12406e990 con 0x7ff1240b08b0 2024-04-25T11:11:06.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.818+0000 7ff12aa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff1240d1ea0 con 0x7ff1240b3d70 2024-04-25T11:11:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.818+0000 7ff128c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1240b3960 0x7ff1240cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.819+0000 7ff128c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1240b3960 0x7ff1240cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43564/0 (socket says 172.21.15.7:43564) 2024-04-25T11:11:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.819+0000 7ff128c00640 1 -- 172.21.15.7:0/2839391085 learned_addr learned my addr 172.21.15.7:0/2839391085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.819+0000 7ff129600640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1240b08b0 0x7ff1240b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.819+0000 7ff12a000640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1240b3d70 0x7ff1240cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.819+0000 7ff128c00640 1 -- 172.21.15.7:0/2839391085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1240b3d70 msgr2=0x7ff1240cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.819+0000 7ff128c00640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1240b3d70 0x7ff1240cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.819+0000 7ff128c00640 1 -- 172.21.15.7:0/2839391085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1240b08b0 msgr2=0x7ff1240b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.819+0000 7ff128c00640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1240b08b0 0x7ff1240b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.819+0000 7ff128c00640 1 -- 172.21.15.7:0/2839391085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1240b26a0 con 0x7ff1240b3960 2024-04-25T11:11:06.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.819+0000 7ff128c00640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1240b3960 0x7ff1240cdab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff11000b420 tx=0x7ff11000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.820+0000 7ff121600640 1 -- 172.21.15.7:0/2839391085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110011830 con 0x7ff1240b3960 2024-04-25T11:11:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.820+0000 7ff121600640 1 -- 172.21.15.7:0/2839391085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff110016070 con 0x7ff1240b3960 2024-04-25T11:11:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.820+0000 7ff121600640 1 -- 172.21.15.7:0/2839391085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110011c10 con 0x7ff1240b3960 2024-04-25T11:11:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.820+0000 7ff12aa00640 1 -- 172.21.15.7:0/2839391085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1240b2ba0 con 0x7ff1240b3960 2024-04-25T11:11:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.820+0000 7ff12aa00640 1 -- 172.21.15.7:0/2839391085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12415c3b0 con 0x7ff1240b3960 2024-04-25T11:11:06.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.821+0000 7ff12aa00640 1 -- 172.21.15.7:0/2839391085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1240b2910 con 0x7ff1240b3960 2024-04-25T11:11:06.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.822+0000 7ff121600640 1 -- 172.21.15.7:0/2839391085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff11001b460 con 0x7ff1240b3960 2024-04-25T11:11:06.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.823+0000 7ff121600640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1040744e0 0x7ff1040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:06.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.823+0000 7ff121600640 1 -- 172.21.15.7:0/2839391085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff110095180 con 0x7ff1240b3960 2024-04-25T11:11:06.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.823+0000 7ff129600640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1040744e0 0x7ff1040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:06.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.824+0000 7ff129600640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1040744e0 0x7ff1040769f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff1180099c0 tx=0x7ff118003990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:06.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:06.828+0000 7ff121600640 1 -- 172.21.15.7:0/2839391085 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff110020020 con 0x7ff1240b3960 2024-04-25T11:11:07.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.033+0000 7ff12aa00640 1 -- 172.21.15.7:0/2839391085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff124050740 con 0x7ff1240b3960 2024-04-25T11:11:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.035+0000 7ff121600640 1 -- 172.21.15.7:0/2839391085 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7ff11005f470 con 0x7ff1240b3960 2024-04-25T11:11:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:07.036 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 -- 172.21.15.7:0/2839391085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1040744e0 msgr2=0x7ff1040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1040744e0 0x7ff1040769f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff1180099c0 tx=0x7ff118003990 comp rx=0 tx=0).stop 2024-04-25T11:11:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 -- 172.21.15.7:0/2839391085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1240b3960 msgr2=0x7ff1240cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1240b3960 0x7ff1240cdab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff11000b420 tx=0x7ff11000b8e0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 -- 172.21.15.7:0/2839391085 shutdown_connections 2024-04-25T11:11:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1040744e0 0x7ff1040769f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1240b3d70 0x7ff1240cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1240b3960 0x7ff1240cdab0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 --2- 172.21.15.7:0/2839391085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1240b08b0 0x7ff1240b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 -- 172.21.15.7:0/2839391085 >> 172.21.15.7:0/2839391085 conn(0x7ff12409b540 msgr2=0x7ff1240b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 -- 172.21.15.7:0/2839391085 shutdown_connections 2024-04-25T11:11:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.038+0000 7ff12aa00640 1 -- 172.21.15.7:0/2839391085 wait complete. 2024-04-25T11:11:07.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:07.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:07.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.345+0000 7fa231c00640 1 Processor -- start 2024-04-25T11:11:07.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.346+0000 7fa231c00640 1 -- start start 2024-04-25T11:11:07.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.348+0000 7fa231c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa22c0b6240 0x7fa22c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.348+0000 7fa231c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa22c0aef90 0x7fa22c0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.348+0000 7fa231c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa22c164e00 con 0x7fa22c0b6920 2024-04-25T11:11:07.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.348+0000 7fa231c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa22c1650e0 con 0x7fa22c0b6240 2024-04-25T11:11:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.348+0000 7fa231c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa22c1653c0 con 0x7fa22c0aef90 2024-04-25T11:11:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.348+0000 7fa223e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa22c0b6920 0x7fa22c0ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33860/0 (socket says 172.21.15.7:33860) 2024-04-25T11:11:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.348+0000 7fa223e00640 1 -- 172.21.15.7:0/607284502 learned_addr learned my addr 172.21.15.7:0/607284502 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.349+0000 7fa22aa00640 1 --2- 172.21.15.7:0/607284502 >> v2:172.21.15.7:3300/0 conn(0x7fa22c0b6240 0x7fa22c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.349+0000 7fa22b400640 1 --2- 172.21.15.7:0/607284502 >> v2:172.21.15.87:3300/0 conn(0x7fa22c0aef90 0x7fa22c0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.349+0000 7fa22a000640 1 -- 172.21.15.7:0/607284502 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1635279075 0 0) 0x7fa22c164e00 con 0x7fa22c0b6920 2024-04-25T11:11:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.349+0000 7fa22a000640 1 -- 172.21.15.7:0/607284502 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa210003520 con 0x7fa22c0b6920 2024-04-25T11:11:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.349+0000 7fa22aa00640 1 -- 172.21.15.7:0/607284502 >> v2:172.21.15.87:3300/0 conn(0x7fa22c0aef90 msgr2=0x7fa22c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.349+0000 7fa22aa00640 1 --2- 172.21.15.7:0/607284502 >> v2:172.21.15.87:3300/0 conn(0x7fa22c0aef90 0x7fa22c0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.349+0000 7fa22aa00640 1 -- 172.21.15.7:0/607284502 >> v1:172.21.15.7:6789/0 conn(0x7fa22c0b6920 legacy=0x7fa22c0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.349+0000 7fa22aa00640 1 -- 172.21.15.7:0/607284502 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c1656a0 con 0x7fa22c0b6240 2024-04-25T11:11:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.350+0000 7fa22aa00640 1 --2- 172.21.15.7:0/607284502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22c0b6240 0x7fa22c15eb00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0afc80 tx=0x7fa21c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7353b26d2a815ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.351+0000 7fa22a000640 1 -- 172.21.15.7:0/607284502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c037070 con 0x7fa22c0b6240 2024-04-25T11:11:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.351+0000 7fa22a000640 1 -- 172.21.15.7:0/607284502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa21c032440 con 0x7fa22c0b6240 2024-04-25T11:11:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.351+0000 7fa22a000640 1 -- 172.21.15.7:0/607284502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c02ae60 con 0x7fa22c0b6240 2024-04-25T11:11:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.351+0000 7fa231c00640 1 -- 172.21.15.7:0/607284502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22c0b6240 msgr2=0x7fa22c15eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.351+0000 7fa231c00640 1 --2- 172.21.15.7:0/607284502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22c0b6240 0x7fa22c15eb00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0afc80 tx=0x7fa21c02a110 comp rx=0 tx=0).stop 2024-04-25T11:11:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.351+0000 7fa231c00640 1 -- 172.21.15.7:0/607284502 shutdown_connections 2024-04-25T11:11:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.351+0000 7fa231c00640 1 --2- 172.21.15.7:0/607284502 >> v2:172.21.15.87:3300/0 conn(0x7fa22c0aef90 0x7fa22c0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.351+0000 7fa231c00640 1 --2- 172.21.15.7:0/607284502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22c0b6240 0x7fa22c15eb00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.351+0000 7fa231c00640 1 -- 172.21.15.7:0/607284502 >> 172.21.15.7:0/607284502 conn(0x7fa22c0742c0 msgr2=0x7fa22c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.351+0000 7fa231c00640 1 -- 172.21.15.7:0/607284502 shutdown_connections 2024-04-25T11:11:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.351+0000 7fa231c00640 1 -- 172.21.15.7:0/607284502 wait complete. 2024-04-25T11:11:07.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.354+0000 7fa231c00640 1 Processor -- start 2024-04-25T11:11:07.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.354+0000 7fa231c00640 1 -- start start 2024-04-25T11:11:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.355+0000 7fa231c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22c0aef90 0x7fa22c145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.355+0000 7fa231c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa22c0b6240 0x7fa22c1fe120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.355+0000 7fa231c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa22c0b6920 0x7fa22c153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.355+0000 7fa231c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c164ba0 con 0x7fa22c0aef90 2024-04-25T11:11:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.355+0000 7fa231c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c164a20 con 0x7fa22c0b6920 2024-04-25T11:11:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.355+0000 7fa231c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa22c0b5130 con 0x7fa22c0b6240 2024-04-25T11:11:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.356+0000 7fa22aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22c0aef90 0x7fa22c145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.356+0000 7fa22aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22c0aef90 0x7fa22c145c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43580/0 (socket says 172.21.15.7:43580) 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.356+0000 7fa22aa00640 1 -- 172.21.15.7:0/2618396309 learned_addr learned my addr 172.21.15.7:0/2618396309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.356+0000 7fa223e00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa22c0b6240 0x7fa22c1fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.356+0000 7fa22b400640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa22c0b6920 0x7fa22c153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.356+0000 7fa22aa00640 1 -- 172.21.15.7:0/2618396309 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa22c0b6240 msgr2=0x7fa22c1fe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.356+0000 7fa22aa00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa22c0b6240 0x7fa22c1fe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.356+0000 7fa22aa00640 1 -- 172.21.15.7:0/2618396309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa22c0b6920 msgr2=0x7fa22c153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.356+0000 7fa22aa00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa22c0b6920 0x7fa22c153210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.356+0000 7fa22aa00640 1 -- 172.21.15.7:0/2618396309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c145180 con 0x7fa22c0aef90 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.356+0000 7fa22aa00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22c0aef90 0x7fa22c145c20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa21c000c00 tx=0x7fa21c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.357+0000 7fa222a00640 1 -- 172.21.15.7:0/2618396309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c037040 con 0x7fa22c0aef90 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.357+0000 7fa222a00640 1 -- 172.21.15.7:0/2618396309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa21c042070 con 0x7fa22c0aef90 2024-04-25T11:11:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.357+0000 7fa222a00640 1 -- 172.21.15.7:0/2618396309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c030920 con 0x7fa22c0aef90 2024-04-25T11:11:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.357+0000 7fa231c00640 1 -- 172.21.15.7:0/2618396309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c1febc0 con 0x7fa22c0aef90 2024-04-25T11:11:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.357+0000 7fa231c00640 1 -- 172.21.15.7:0/2618396309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c145420 con 0x7fa22c0aef90 2024-04-25T11:11:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.358+0000 7fa231c00640 1 -- 172.21.15.7:0/2618396309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa22c050740 con 0x7fa22c0aef90 2024-04-25T11:11:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.359+0000 7fa222a00640 1 -- 172.21.15.7:0/2618396309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa21c030ac0 con 0x7fa22c0aef90 2024-04-25T11:11:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.360+0000 7fa222a00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2080744b0 0x7fa2080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.360+0000 7fa222a00640 1 -- 172.21.15.7:0/2618396309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fa21c0b5f10 con 0x7fa22c0aef90 2024-04-25T11:11:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.360+0000 7fa223e00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2080744b0 0x7fa2080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.361+0000 7fa223e00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2080744b0 0x7fa2080769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa214007fd0 tx=0x7fa214009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:07.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.365+0000 7fa222a00640 1 -- 172.21.15.7:0/2618396309 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa21c07fd70 con 0x7fa22c0aef90 2024-04-25T11:11:07.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.574+0000 7fa231c00640 1 -- 172.21.15.7:0/2618396309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa22c0710c0 con 0x7fa22c0aef90 2024-04-25T11:11:07.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.576+0000 7fa222a00640 1 -- 172.21.15.7:0/2618396309 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fa21c084070 con 0x7fa22c0aef90 2024-04-25T11:11:07.576 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:07.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.578+0000 7fa231c00640 1 -- 172.21.15.7:0/2618396309 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2080744b0 msgr2=0x7fa2080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.578+0000 7fa231c00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2080744b0 0x7fa2080769c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa214007fd0 tx=0x7fa214009040 comp rx=0 tx=0).stop 2024-04-25T11:11:07.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.578+0000 7fa231c00640 1 -- 172.21.15.7:0/2618396309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22c0aef90 msgr2=0x7fa22c145c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.578+0000 7fa231c00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22c0aef90 0x7fa22c145c20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa21c000c00 tx=0x7fa21c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.579+0000 7fa231c00640 1 -- 172.21.15.7:0/2618396309 shutdown_connections 2024-04-25T11:11:07.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.579+0000 7fa231c00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2080744b0 0x7fa2080769c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.579+0000 7fa231c00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa22c0b6920 0x7fa22c153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.579+0000 7fa231c00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa22c0b6240 0x7fa22c1fe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.579+0000 7fa231c00640 1 --2- 172.21.15.7:0/2618396309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa22c0aef90 0x7fa22c145c20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.579+0000 7fa231c00640 1 -- 172.21.15.7:0/2618396309 >> 172.21.15.7:0/2618396309 conn(0x7fa22c0742c0 msgr2=0x7fa22c0b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:07.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.579+0000 7fa231c00640 1 -- 172.21.15.7:0/2618396309 shutdown_connections 2024-04-25T11:11:07.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.579+0000 7fa231c00640 1 -- 172.21.15.7:0/2618396309 wait complete. 2024-04-25T11:11:07.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:07.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.889+0000 7efcde600640 1 Processor -- start 2024-04-25T11:11:07.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.889+0000 7efcde600640 1 -- start start 2024-04-25T11:11:07.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.892+0000 7efcde600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcd80aef90 0x7efcd815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.892+0000 7efcde600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efcd80b6920 0x7efcd80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.892+0000 7efcde600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd8164e00 con 0x7efcd80b6240 2024-04-25T11:11:07.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.892+0000 7efcde600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efcd81650e0 con 0x7efcd80b6920 2024-04-25T11:11:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.892+0000 7efcde600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efcd81653c0 con 0x7efcd80aef90 2024-04-25T11:11:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.893+0000 7efcd7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efcd80b6920 0x7efcd80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.893+0000 7efcd7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efcd80b6920 0x7efcd80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43582/0 (socket says 172.21.15.7:43582) 2024-04-25T11:11:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.893+0000 7efcd7e00640 1 -- 172.21.15.7:0/725362029 learned_addr learned my addr 172.21.15.7:0/725362029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.893+0000 7efcd6a00640 1 --2- 172.21.15.7:0/725362029 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcd80aef90 0x7efcd815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.893+0000 7efcd7e00640 1 -- 172.21.15.7:0/725362029 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcd80aef90 msgr2=0x7efcd815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.893+0000 7efcd7e00640 1 --2- 172.21.15.7:0/725362029 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcd80aef90 0x7efcd815eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.893+0000 7efcd7e00640 1 -- 172.21.15.7:0/725362029 >> v1:172.21.15.87:6789/0 conn(0x7efcd80b6240 legacy=0x7efcd80ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.893+0000 7efcd7e00640 1 -- 172.21.15.7:0/725362029 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd81656a0 con 0x7efcd80b6920 2024-04-25T11:11:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.894+0000 7efcd7e00640 1 --2- 172.21.15.7:0/725362029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcd80b6920 0x7efcd80b4bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a14021926f69f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.895+0000 7efcd6000640 1 -- 172.21.15.7:0/725362029 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0037070 con 0x7efcd80b6920 2024-04-25T11:11:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.895+0000 7efcd6000640 1 -- 172.21.15.7:0/725362029 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd0032440 con 0x7efcd80b6920 2024-04-25T11:11:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.895+0000 7efcd6000640 1 -- 172.21.15.7:0/725362029 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0032810 con 0x7efcd80b6920 2024-04-25T11:11:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.895+0000 7efcde600640 1 -- 172.21.15.7:0/725362029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcd80b6920 msgr2=0x7efcd80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.895+0000 7efcde600640 1 --2- 172.21.15.7:0/725362029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcd80b6920 0x7efcd80b4bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd002a270 comp rx=0 tx=0).stop 2024-04-25T11:11:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.895+0000 7efcde600640 1 -- 172.21.15.7:0/725362029 shutdown_connections 2024-04-25T11:11:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.895+0000 7efcde600640 1 --2- 172.21.15.7:0/725362029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcd80b6920 0x7efcd80b4bf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.896+0000 7efcde600640 1 --2- 172.21.15.7:0/725362029 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcd80aef90 0x7efcd815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.896+0000 7efcde600640 1 -- 172.21.15.7:0/725362029 >> 172.21.15.7:0/725362029 conn(0x7efcd80742c0 msgr2=0x7efcd806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.896+0000 7efcde600640 1 -- 172.21.15.7:0/725362029 shutdown_connections 2024-04-25T11:11:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.896+0000 7efcde600640 1 -- 172.21.15.7:0/725362029 wait complete. 2024-04-25T11:11:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.898+0000 7efcde600640 1 Processor -- start 2024-04-25T11:11:07.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.899+0000 7efcde600640 1 -- start start 2024-04-25T11:11:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.900+0000 7efcde600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcd80aef90 0x7efcd814ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.900+0000 7efcde600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcd80b6240 0x7efcd814d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.900+0000 7efcde600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcd80b6920 0x7efcd8166bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.900+0000 7efcde600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8164ba0 con 0x7efcd80b6920 2024-04-25T11:11:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.900+0000 7efcde600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8164a20 con 0x7efcd80b6240 2024-04-25T11:11:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.900+0000 7efcde600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efcd80b5130 con 0x7efcd80aef90 2024-04-25T11:11:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.900+0000 7efcd7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcd80b6920 0x7efcd8166bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.901+0000 7efcd7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcd80b6920 0x7efcd8166bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43596/0 (socket says 172.21.15.7:43596) 2024-04-25T11:11:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.901+0000 7efcd7e00640 1 -- 172.21.15.7:0/111912955 learned_addr learned my addr 172.21.15.7:0/111912955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.901+0000 7efcd6a00640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcd80b6240 0x7efcd814d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.901+0000 7efcd7400640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcd80aef90 0x7efcd814ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.901+0000 7efcd7e00640 1 -- 172.21.15.7:0/111912955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcd80aef90 msgr2=0x7efcd814ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.901+0000 7efcd7e00640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcd80aef90 0x7efcd814ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.901+0000 7efcd7e00640 1 -- 172.21.15.7:0/111912955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcd80b6240 msgr2=0x7efcd814d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.901+0000 7efcd7e00640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcd80b6240 0x7efcd814d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.901+0000 7efcd7e00640 1 -- 172.21.15.7:0/111912955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd814e740 con 0x7efcd80b6920 2024-04-25T11:11:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.901+0000 7efcd7e00640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcd80b6920 0x7efcd8166bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:07.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.902+0000 7efccf400640 1 -- 172.21.15.7:0/111912955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0041020 con 0x7efcd80b6920 2024-04-25T11:11:07.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.902+0000 7efccf400640 1 -- 172.21.15.7:0/111912955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd00306c0 con 0x7efcd80b6920 2024-04-25T11:11:07.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.902+0000 7efccf400640 1 -- 172.21.15.7:0/111912955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0037040 con 0x7efcd80b6920 2024-04-25T11:11:07.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.902+0000 7efcde600640 1 -- 172.21.15.7:0/111912955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd81ecce0 con 0x7efcd80b6920 2024-04-25T11:11:07.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.902+0000 7efcde600640 1 -- 172.21.15.7:0/111912955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd81ed2a0 con 0x7efcd80b6920 2024-04-25T11:11:07.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.903+0000 7efcde600640 1 -- 172.21.15.7:0/111912955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd8050740 con 0x7efcd80b6920 2024-04-25T11:11:07.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.904+0000 7efccf400640 1 -- 172.21.15.7:0/111912955 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efcd0044020 con 0x7efcd80b6920 2024-04-25T11:11:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.905+0000 7efccf400640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efcb40744b0 0x7efcb40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.905+0000 7efccf400640 1 -- 172.21.15.7:0/111912955 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7efcd00b5d70 con 0x7efcd80b6920 2024-04-25T11:11:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.906+0000 7efcd7400640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efcb40744b0 0x7efcb40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:07.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.906+0000 7efcd7400640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efcb40744b0 0x7efcb40769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efcc80012e0 tx=0x7efcc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:07.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:07.910+0000 7efccf400640 1 -- 172.21.15.7:0/111912955 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcd004b030 con 0x7efcd80b6920 2024-04-25T11:11:08.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.123+0000 7efcde600640 1 -- 172.21.15.7:0/111912955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efcd8068a80 con 0x7efcd80b6920 2024-04-25T11:11:08.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.124+0000 7efccf400640 1 -- 172.21.15.7:0/111912955 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7efcd0080220 con 0x7efcd80b6920 2024-04-25T11:11:08.124 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:08.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 -- 172.21.15.7:0/111912955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efcb40744b0 msgr2=0x7efcb40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efcb40744b0 0x7efcb40769c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efcc80012e0 tx=0x7efcc8009040 comp rx=0 tx=0).stop 2024-04-25T11:11:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 -- 172.21.15.7:0/111912955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcd80b6920 msgr2=0x7efcd8166bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcd80b6920 0x7efcd8166bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd00046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 -- 172.21.15.7:0/111912955 shutdown_connections 2024-04-25T11:11:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efcb40744b0 0x7efcb40769c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcd80b6920 0x7efcd8166bd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcd80b6240 0x7efcd814d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 --2- 172.21.15.7:0/111912955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcd80aef90 0x7efcd814ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 -- 172.21.15.7:0/111912955 >> 172.21.15.7:0/111912955 conn(0x7efcd80742c0 msgr2=0x7efcd80b54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 -- 172.21.15.7:0/111912955 shutdown_connections 2024-04-25T11:11:08.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.127+0000 7efcde600640 1 -- 172.21.15.7:0/111912955 wait complete. 2024-04-25T11:11:08.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:08.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.416+0000 7f122f200640 1 Processor -- start 2024-04-25T11:11:08.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.416+0000 7f122f200640 1 -- start start 2024-04-25T11:11:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.418+0000 7f122f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1228070630 0x7f12280cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.418+0000 7f122f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f122815ecc0 0x7f12280d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.418+0000 7f122f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12280d0800 0x7f122806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.418+0000 7f122f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f12280d0c10 con 0x7f12280d0800 2024-04-25T11:11:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.418+0000 7f122f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f12280b0be0 con 0x7f1228070630 2024-04-25T11:11:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.418+0000 7f122f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f12280ce9e0 con 0x7f122815ecc0 2024-04-25T11:11:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.418+0000 7f122e800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12280d0800 0x7f122806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.418+0000 7f122e800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12280d0800 0x7f122806eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43604/0 (socket says 172.21.15.7:43604) 2024-04-25T11:11:08.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.418+0000 7f122e800640 1 -- 172.21.15.7:0/2775660483 learned_addr learned my addr 172.21.15.7:0/2775660483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:08.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.418+0000 7f122de00640 1 --2- 172.21.15.7:0/2775660483 >> v2:172.21.15.87:3300/0 conn(0x7f1228070630 0x7f12280cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.419+0000 7f122e800640 1 -- 172.21.15.7:0/2775660483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f122815ecc0 msgr2=0x7f12280d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:08.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.419+0000 7f122d400640 1 --2- 172.21.15.7:0/2775660483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f122815ecc0 0x7f12280d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.419+0000 7f122e800640 1 --2- 172.21.15.7:0/2775660483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f122815ecc0 0x7f12280d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.419+0000 7f122e800640 1 -- 172.21.15.7:0/2775660483 >> v2:172.21.15.87:3300/0 conn(0x7f1228070630 msgr2=0x7f12280cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:08.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.419+0000 7f122e800640 1 --2- 172.21.15.7:0/2775660483 >> v2:172.21.15.87:3300/0 conn(0x7f1228070630 0x7f12280cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.419+0000 7f122e800640 1 -- 172.21.15.7:0/2775660483 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12280cecc0 con 0x7f12280d0800 2024-04-25T11:11:08.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.419+0000 7f122d400640 1 --2- 172.21.15.7:0/2775660483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f122815ecc0 0x7f12280d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.420+0000 7f122e800640 1 --2- 172.21.15.7:0/2775660483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12280d0800 0x7f122806eea0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1224009e50 tx=0x7f122402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a806a659f92b963 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.420+0000 7f122ca00640 1 -- 172.21.15.7:0/2775660483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224036070 con 0x7f12280d0800 2024-04-25T11:11:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.420+0000 7f122ca00640 1 -- 172.21.15.7:0/2775660483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1224031600 con 0x7f12280d0800 2024-04-25T11:11:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.420+0000 7f122ca00640 1 -- 172.21.15.7:0/2775660483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240319d0 con 0x7f12280d0800 2024-04-25T11:11:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.421+0000 7f122f200640 1 -- 172.21.15.7:0/2775660483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12280d0800 msgr2=0x7f122806eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.421+0000 7f122f200640 1 --2- 172.21.15.7:0/2775660483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12280d0800 0x7f122806eea0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1224009e50 tx=0x7f122402a6c0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.421+0000 7f122f200640 1 -- 172.21.15.7:0/2775660483 shutdown_connections 2024-04-25T11:11:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.421+0000 7f122f200640 1 --2- 172.21.15.7:0/2775660483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12280d0800 0x7f122806eea0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.421+0000 7f122f200640 1 --2- 172.21.15.7:0/2775660483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f122815ecc0 0x7f12280d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.421+0000 7f122f200640 1 --2- 172.21.15.7:0/2775660483 >> v2:172.21.15.87:3300/0 conn(0x7f1228070630 0x7f12280cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.421+0000 7f122f200640 1 -- 172.21.15.7:0/2775660483 >> 172.21.15.7:0/2775660483 conn(0x7f1228074140 msgr2=0x7f122806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:08.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.421+0000 7f122f200640 1 -- 172.21.15.7:0/2775660483 shutdown_connections 2024-04-25T11:11:08.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.421+0000 7f122f200640 1 -- 172.21.15.7:0/2775660483 wait complete. 2024-04-25T11:11:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.423+0000 7f122f200640 1 Processor -- start 2024-04-25T11:11:08.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.423+0000 7f122f200640 1 -- start start 2024-04-25T11:11:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.425+0000 7f122f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1228070630 0x7f12281491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.425+0000 7f122f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12280d0800 0x7f12280aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.425+0000 7f122f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f122815ecc0 0x7f122806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.425+0000 7f122f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f12280cf560 con 0x7f1228070630 2024-04-25T11:11:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.425+0000 7f122f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f12280cf280 con 0x7f12280d0800 2024-04-25T11:11:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.425+0000 7f122f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f122815f0d0 con 0x7f122815ecc0 2024-04-25T11:11:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.425+0000 7f122de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1228070630 0x7f12281491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.425+0000 7f122de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1228070630 0x7f12281491d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43612/0 (socket says 172.21.15.7:43612) 2024-04-25T11:11:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.425+0000 7f122de00640 1 -- 172.21.15.7:0/2110465406 learned_addr learned my addr 172.21.15.7:0/2110465406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.425+0000 7f122d400640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12280d0800 0x7f12280aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f122de00640 1 -- 172.21.15.7:0/2110465406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f122815ecc0 msgr2=0x7f122806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f122de00640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f122815ecc0 0x7f122806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f122de00640 1 -- 172.21.15.7:0/2110465406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12280d0800 msgr2=0x7f12280aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f122de00640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12280d0800 0x7f12280aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f122de00640 1 -- 172.21.15.7:0/2110465406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12280ad460 con 0x7f1228070630 2024-04-25T11:11:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f122d400640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12280d0800 0x7f12280aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f122de00640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1228070630 0x7f12281491d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f121c00d5f0 tx=0x7f121c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f1222000640 1 -- 172.21.15.7:0/2110465406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c0125f0 con 0x7f1228070630 2024-04-25T11:11:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f1222000640 1 -- 172.21.15.7:0/2110465406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121c019070 con 0x7f1228070630 2024-04-25T11:11:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f1222000640 1 -- 172.21.15.7:0/2110465406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c0129d0 con 0x7f1228070630 2024-04-25T11:11:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f122f200640 1 -- 172.21.15.7:0/2110465406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122814a110 con 0x7f1228070630 2024-04-25T11:11:08.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.426+0000 7f122f200640 1 -- 172.21.15.7:0/2110465406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122814a6d0 con 0x7f1228070630 2024-04-25T11:11:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.427+0000 7f122f200640 1 -- 172.21.15.7:0/2110465406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1228050740 con 0x7f1228070630 2024-04-25T11:11:08.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.429+0000 7f1222000640 1 -- 172.21.15.7:0/2110465406 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f121c012b70 con 0x7f1228070630 2024-04-25T11:11:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.429+0000 7f1222000640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f120c0744e0 0x7f120c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.430+0000 7f1222000640 1 -- 172.21.15.7:0/2110465406 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f121c095b00 con 0x7f1228070630 2024-04-25T11:11:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.430+0000 7f122d400640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f120c0744e0 0x7f120c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.431+0000 7f122d400640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f120c0744e0 0x7f120c0769f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1218006fd0 tx=0x7f1218008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.434+0000 7f1222000640 1 -- 172.21.15.7:0/2110465406 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f121c05fdf0 con 0x7f1228070630 2024-04-25T11:11:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.629+0000 7f122f200640 1 -- 172.21.15.7:0/2110465406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1228161400 con 0x7f1228070630 2024-04-25T11:11:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.629+0000 7f1222000640 1 -- 172.21.15.7:0/2110465406 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f121c066e20 con 0x7f1228070630 2024-04-25T11:11:08.630 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 -- 172.21.15.7:0/2110465406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f120c0744e0 msgr2=0x7f120c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f120c0744e0 0x7f120c0769f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1218006fd0 tx=0x7f1218008040 comp rx=0 tx=0).stop 2024-04-25T11:11:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 -- 172.21.15.7:0/2110465406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1228070630 msgr2=0x7f12281491d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1228070630 0x7f12281491d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f121c00d5f0 tx=0x7f121c00dab0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 -- 172.21.15.7:0/2110465406 shutdown_connections 2024-04-25T11:11:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f120c0744e0 0x7f120c0769f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f122815ecc0 0x7f122806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12280d0800 0x7f12280aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 --2- 172.21.15.7:0/2110465406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1228070630 0x7f12281491d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 -- 172.21.15.7:0/2110465406 >> 172.21.15.7:0/2110465406 conn(0x7f1228074140 msgr2=0x7f12280b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 -- 172.21.15.7:0/2110465406 shutdown_connections 2024-04-25T11:11:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.632+0000 7f122f200640 1 -- 172.21.15.7:0/2110465406 wait complete. 2024-04-25T11:11:08.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:08.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.940+0000 7ff4d8a00640 1 Processor -- start 2024-04-25T11:11:08.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.940+0000 7ff4d8a00640 1 -- start start 2024-04-25T11:11:08.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.942+0000 7ff4d8a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4d40b4fd0 0x7ff4d40b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.942+0000 7ff4d8a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4d415fe80 0x7ff4d40af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.942+0000 7ff4d8a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4d406d580 0x7ff4d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.942+0000 7ff4d8a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d40b1470 con 0x7ff4d415fe80 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d8a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d406ded0 con 0x7ff4d40b4fd0 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d8a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d40b5650 con 0x7ff4d406d580 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d2a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4d415fe80 0x7ff4d40af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d2a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4d415fe80 0x7ff4d40af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43628/0 (socket says 172.21.15.7:43628) 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d2a00640 1 -- 172.21.15.7:0/664245922 learned_addr learned my addr 172.21.15.7:0/664245922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d3400640 1 --2- 172.21.15.7:0/664245922 >> v2:172.21.15.87:3300/0 conn(0x7ff4d40b4fd0 0x7ff4d40b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d3e00640 1 --2- 172.21.15.7:0/664245922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4d406d580 0x7ff4d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d2a00640 1 -- 172.21.15.7:0/664245922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4d406d580 msgr2=0x7ff4d406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d2a00640 1 --2- 172.21.15.7:0/664245922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4d406d580 0x7ff4d406d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d2a00640 1 -- 172.21.15.7:0/664245922 >> v2:172.21.15.87:3300/0 conn(0x7ff4d40b4fd0 msgr2=0x7ff4d40b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d2a00640 1 --2- 172.21.15.7:0/664245922 >> v2:172.21.15.87:3300/0 conn(0x7ff4d40b4fd0 0x7ff4d40b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d2a00640 1 -- 172.21.15.7:0/664245922 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d40b5930 con 0x7ff4d415fe80 2024-04-25T11:11:08.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.943+0000 7ff4d3e00640 1 --2- 172.21.15.7:0/664245922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4d406d580 0x7ff4d406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:08.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.945+0000 7ff4d2a00640 1 --2- 172.21.15.7:0/664245922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4d415fe80 0x7ff4d40af240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4072f80 tx=0x7ff4c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9370fecbf10e6dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:08.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.945+0000 7ff4d2000640 1 -- 172.21.15.7:0/664245922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0037070 con 0x7ff4d415fe80 2024-04-25T11:11:08.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.945+0000 7ff4d2000640 1 -- 172.21.15.7:0/664245922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c0032440 con 0x7ff4d415fe80 2024-04-25T11:11:08.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.945+0000 7ff4d2000640 1 -- 172.21.15.7:0/664245922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0032810 con 0x7ff4d415fe80 2024-04-25T11:11:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.946+0000 7ff4d8a00640 1 -- 172.21.15.7:0/664245922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4d415fe80 msgr2=0x7ff4d40af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.946+0000 7ff4d8a00640 1 --2- 172.21.15.7:0/664245922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4d415fe80 0x7ff4d40af240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4072f80 tx=0x7ff4c002a140 comp rx=0 tx=0).stop 2024-04-25T11:11:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.946+0000 7ff4d8a00640 1 -- 172.21.15.7:0/664245922 shutdown_connections 2024-04-25T11:11:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.946+0000 7ff4d8a00640 1 --2- 172.21.15.7:0/664245922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4d406d580 0x7ff4d406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.946+0000 7ff4d8a00640 1 --2- 172.21.15.7:0/664245922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4d415fe80 0x7ff4d40af240 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.946+0000 7ff4d8a00640 1 --2- 172.21.15.7:0/664245922 >> v2:172.21.15.87:3300/0 conn(0x7ff4d40b4fd0 0x7ff4d40b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.946+0000 7ff4d8a00640 1 -- 172.21.15.7:0/664245922 >> 172.21.15.7:0/664245922 conn(0x7ff4d40734d0 msgr2=0x7ff4d40738e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:08.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.946+0000 7ff4d8a00640 1 -- 172.21.15.7:0/664245922 shutdown_connections 2024-04-25T11:11:08.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.946+0000 7ff4d8a00640 1 -- 172.21.15.7:0/664245922 wait complete. 2024-04-25T11:11:08.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.949+0000 7ff4d8a00640 1 Processor -- start 2024-04-25T11:11:08.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.949+0000 7ff4d8a00640 1 -- start start 2024-04-25T11:11:08.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.951+0000 7ff4d8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4d406d580 0x7ff4d4160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.951+0000 7ff4d8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4d40b4fd0 0x7ff4d414bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.951+0000 7ff4d8a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4d415fe80 0x7ff4d4162560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.951+0000 7ff4d8a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40b61d0 con 0x7ff4d406d580 2024-04-25T11:11:08.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.951+0000 7ff4d8a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d40b5ef0 con 0x7ff4d40b4fd0 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.951+0000 7ff4d8a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d40af780 con 0x7ff4d415fe80 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.951+0000 7ff4d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4d406d580 0x7ff4d4160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.952+0000 7ff4d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4d406d580 0x7ff4d4160fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43636/0 (socket says 172.21.15.7:43636) 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.952+0000 7ff4d3400640 1 -- 172.21.15.7:0/710972570 learned_addr learned my addr 172.21.15.7:0/710972570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.952+0000 7ff4d2a00640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4d40b4fd0 0x7ff4d414bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.952+0000 7ff4d3e00640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4d415fe80 0x7ff4d4162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.952+0000 7ff4d3400640 1 -- 172.21.15.7:0/710972570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4d415fe80 msgr2=0x7ff4d4162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.952+0000 7ff4d3400640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4d415fe80 0x7ff4d4162560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.952+0000 7ff4d3400640 1 -- 172.21.15.7:0/710972570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4d40b4fd0 msgr2=0x7ff4d414bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.952+0000 7ff4d3400640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4d40b4fd0 0x7ff4d414bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.952+0000 7ff4d3400640 1 -- 172.21.15.7:0/710972570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d414c600 con 0x7ff4d406d580 2024-04-25T11:11:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.952+0000 7ff4d3400640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4d406d580 0x7ff4d4160fa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400d5d0 tx=0x7ff4c400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:08.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.953+0000 7ff4cb400640 1 -- 172.21.15.7:0/710972570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4004240 con 0x7ff4d406d580 2024-04-25T11:11:08.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.953+0000 7ff4cb400640 1 -- 172.21.15.7:0/710972570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c40043e0 con 0x7ff4d406d580 2024-04-25T11:11:08.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.953+0000 7ff4d8a00640 1 -- 172.21.15.7:0/710972570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d4161980 con 0x7ff4d406d580 2024-04-25T11:11:08.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.953+0000 7ff4cb400640 1 -- 172.21.15.7:0/710972570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4003830 con 0x7ff4d406d580 2024-04-25T11:11:08.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.953+0000 7ff4d8a00640 1 -- 172.21.15.7:0/710972570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d41552d0 con 0x7ff4d406d580 2024-04-25T11:11:08.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.954+0000 7ff4d8a00640 1 -- 172.21.15.7:0/710972570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d4068df0 con 0x7ff4d406d580 2024-04-25T11:11:08.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.955+0000 7ff4cb400640 1 -- 172.21.15.7:0/710972570 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff4c40039d0 con 0x7ff4d406d580 2024-04-25T11:11:08.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.956+0000 7ff4cb400640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4b40744e0 0x7ff4b40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:08.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.957+0000 7ff4cb400640 1 -- 172.21.15.7:0/710972570 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff4c40957e0 con 0x7ff4d406d580 2024-04-25T11:11:08.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.957+0000 7ff4d2a00640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4b40744e0 0x7ff4b40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:08.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.958+0000 7ff4d2a00640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4b40744e0 0x7ff4b40769f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0034000 tx=0x7ff4c002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:08.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:08.962+0000 7ff4cb400640 1 -- 172.21.15.7:0/710972570 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff4c405f510 con 0x7ff4d406d580 2024-04-25T11:11:09.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.163+0000 7ff4d8a00640 1 -- 172.21.15.7:0/710972570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff4d40b3460 con 0x7ff4d406d580 2024-04-25T11:11:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.164+0000 7ff4cb400640 1 -- 172.21.15.7:0/710972570 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7ff4c4063810 con 0x7ff4d406d580 2024-04-25T11:11:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 -- 172.21.15.7:0/710972570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4b40744e0 msgr2=0x7ff4b40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4b40744e0 0x7ff4b40769f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0034000 tx=0x7ff4c002e000 comp rx=0 tx=0).stop 2024-04-25T11:11:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 -- 172.21.15.7:0/710972570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4d406d580 msgr2=0x7ff4d4160fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4d406d580 0x7ff4d4160fa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400d5d0 tx=0x7ff4c400da90 comp rx=0 tx=0).stop 2024-04-25T11:11:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 -- 172.21.15.7:0/710972570 shutdown_connections 2024-04-25T11:11:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4b40744e0 0x7ff4b40769f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4d415fe80 0x7ff4d4162560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4d40b4fd0 0x7ff4d414bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 --2- 172.21.15.7:0/710972570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4d406d580 0x7ff4d4160fa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 -- 172.21.15.7:0/710972570 >> 172.21.15.7:0/710972570 conn(0x7ff4d40734d0 msgr2=0x7ff4d4164000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 -- 172.21.15.7:0/710972570 shutdown_connections 2024-04-25T11:11:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.167+0000 7ff4d8a00640 1 -- 172.21.15.7:0/710972570 wait complete. 2024-04-25T11:11:09.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:11:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.474+0000 7fc98fe00640 1 Processor -- start 2024-04-25T11:11:09.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.474+0000 7fc98fe00640 1 -- start start 2024-04-25T11:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.476+0000 7fc98fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc9900b5220 0x7fc99015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.476+0000 7fc98fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc9900af420 0x7fc9900af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.477+0000 7fc98fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9900cebb0 con 0x7fc9900aed40 2024-04-25T11:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.477+0000 7fc98fe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc99015e1f0 con 0x7fc9900af420 2024-04-25T11:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.477+0000 7fc98fe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc9900afd70 con 0x7fc9900b5220 2024-04-25T11:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.477+0000 7fc98f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc9900af420 0x7fc9900af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.477+0000 7fc98f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc9900af420 0x7fc9900af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43638/0 (socket says 172.21.15.7:43638) 2024-04-25T11:11:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.477+0000 7fc98f400640 1 -- 172.21.15.7:0/2468560508 learned_addr learned my addr 172.21.15.7:0/2468560508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.477+0000 7fc98e000640 1 --2- 172.21.15.7:0/2468560508 >> v2:172.21.15.87:3300/0 conn(0x7fc9900b5220 0x7fc99015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.477+0000 7fc98f400640 1 -- 172.21.15.7:0/2468560508 >> v2:172.21.15.87:3300/0 conn(0x7fc9900b5220 msgr2=0x7fc99015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.478+0000 7fc98f400640 1 --2- 172.21.15.7:0/2468560508 >> v2:172.21.15.87:3300/0 conn(0x7fc9900b5220 0x7fc99015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.478+0000 7fc98f400640 1 -- 172.21.15.7:0/2468560508 >> v1:172.21.15.87:6789/0 conn(0x7fc9900aed40 legacy=0x7fc9900ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:09.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.478+0000 7fc98f400640 1 -- 172.21.15.7:0/2468560508 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9900b64b0 con 0x7fc9900af420 2024-04-25T11:11:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.479+0000 7fc98f400640 1 --2- 172.21.15.7:0/2468560508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900af420 0x7fc9900af830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc9900b0a90 tx=0x7fc98802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ab70c012444536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.479+0000 7fc98d600640 1 -- 172.21.15.7:0/2468560508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988036070 con 0x7fc9900af420 2024-04-25T11:11:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.479+0000 7fc98d600640 1 -- 172.21.15.7:0/2468560508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc988031600 con 0x7fc9900af420 2024-04-25T11:11:09.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.479+0000 7fc98d600640 1 -- 172.21.15.7:0/2468560508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9880319d0 con 0x7fc9900af420 2024-04-25T11:11:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.480+0000 7fc98fe00640 1 -- 172.21.15.7:0/2468560508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900af420 msgr2=0x7fc9900af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.480+0000 7fc98fe00640 1 --2- 172.21.15.7:0/2468560508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900af420 0x7fc9900af830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc9900b0a90 tx=0x7fc98802a6f0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.480+0000 7fc98fe00640 1 -- 172.21.15.7:0/2468560508 shutdown_connections 2024-04-25T11:11:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.480+0000 7fc98fe00640 1 --2- 172.21.15.7:0/2468560508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900af420 0x7fc9900af830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.480+0000 7fc98fe00640 1 --2- 172.21.15.7:0/2468560508 >> v2:172.21.15.87:3300/0 conn(0x7fc9900b5220 0x7fc99015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.480+0000 7fc98fe00640 1 -- 172.21.15.7:0/2468560508 >> 172.21.15.7:0/2468560508 conn(0x7fc9900b3820 msgr2=0x7fc9900b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.480+0000 7fc98fe00640 1 -- 172.21.15.7:0/2468560508 shutdown_connections 2024-04-25T11:11:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.480+0000 7fc98fe00640 1 -- 172.21.15.7:0/2468560508 wait complete. 2024-04-25T11:11:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.483+0000 7fc98fe00640 1 Processor -- start 2024-04-25T11:11:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.483+0000 7fc98fe00640 1 -- start start 2024-04-25T11:11:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.485+0000 7fc98fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9900aed40 0x7fc99015edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.485+0000 7fc98fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9900af420 0x7fc990145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.485+0000 7fc98fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5220 0x7fc9901559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.485+0000 7fc98fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc99006cbd0 con 0x7fc9900b5220 2024-04-25T11:11:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.485+0000 7fc98fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc99006c8f0 con 0x7fc9900af420 2024-04-25T11:11:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.485+0000 7fc98fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc99006ceb0 con 0x7fc9900aed40 2024-04-25T11:11:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5220 0x7fc9901559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5220 0x7fc9901559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43648/0 (socket says 172.21.15.7:43648) 2024-04-25T11:11:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98f400640 1 -- 172.21.15.7:0/2541774814 learned_addr learned my addr 172.21.15.7:0/2541774814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:09.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98e000640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9900af420 0x7fc990145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98ea00640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9900aed40 0x7fc99015edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98f400640 1 -- 172.21.15.7:0/2541774814 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9900aed40 msgr2=0x7fc99015edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98f400640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9900aed40 0x7fc99015edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98f400640 1 -- 172.21.15.7:0/2541774814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9900af420 msgr2=0x7fc990145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98f400640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9900af420 0x7fc990145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98f400640 1 -- 172.21.15.7:0/2541774814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9900cf400 con 0x7fc9900b5220 2024-04-25T11:11:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98ea00640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9900aed40 0x7fc99015edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.486+0000 7fc98f400640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5220 0x7fc9901559a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc988009f40 tx=0x7fc988004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.487+0000 7fc986a00640 1 -- 172.21.15.7:0/2541774814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988036050 con 0x7fc9900b5220 2024-04-25T11:11:09.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.487+0000 7fc986a00640 1 -- 172.21.15.7:0/2541774814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc988042070 con 0x7fc9900b5220 2024-04-25T11:11:09.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.487+0000 7fc986a00640 1 -- 172.21.15.7:0/2541774814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988009510 con 0x7fc9900b5220 2024-04-25T11:11:09.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.487+0000 7fc98fe00640 1 -- 172.21.15.7:0/2541774814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9900cf6e0 con 0x7fc9900b5220 2024-04-25T11:11:09.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.487+0000 7fc98fe00640 1 -- 172.21.15.7:0/2541774814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9900cfca0 con 0x7fc9900b5220 2024-04-25T11:11:09.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.488+0000 7fc98fe00640 1 -- 172.21.15.7:0/2541774814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9900b1970 con 0x7fc9900b5220 2024-04-25T11:11:09.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.489+0000 7fc986a00640 1 -- 172.21.15.7:0/2541774814 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc988007bd0 con 0x7fc9900b5220 2024-04-25T11:11:09.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.490+0000 7fc986a00640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 0x7fc96c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:09.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.490+0000 7fc986a00640 1 -- 172.21.15.7:0/2541774814 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fc9880b6560 con 0x7fc9900b5220 2024-04-25T11:11:09.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.490+0000 7fc98ea00640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 0x7fc96c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.491+0000 7fc98ea00640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 0x7fc96c0769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc990071200 tx=0x7fc980008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:09.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.497+0000 7fc986a00640 1 -- 172.21.15.7:0/2541774814 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9900cebb0 con 0x7fc9900b5220 2024-04-25T11:11:09.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.737+0000 7fc98fe00640 1 -- 172.21.15.7:0/2541774814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc99006e0b0 con 0x7fc9900b5220 2024-04-25T11:11:09.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.738+0000 7fc986a00640 1 -- 172.21.15.7:0/2541774814 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7fc9880808f0 con 0x7fc9900b5220 2024-04-25T11:11:09.739 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:09.739 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3458223722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3458223722},{"type":"v1","addr":"172.21.15.87:6835","nonce":3458223722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1383067804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1383067804},{"type":"v1","addr":"172.21.15.7:6835","nonce":1383067804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/435824250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":435824250},{"type":"v1","addr":"172.21.15.7:6837","nonce":435824250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:10:23.023233+0000","modified":"2024-04-25T11:10:53.526222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1091275258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1091275258},{"type":"v1","addr":"172.21.15.87:6834","nonce":1091275258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:11:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-04-25T11:11:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.741+0000 7fc98fe00640 1 -- 172.21.15.7:0/2541774814 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 msgr2=0x7fc96c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.741+0000 7fc98fe00640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 0x7fc96c0769c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc990071200 tx=0x7fc980008040 comp rx=0 tx=0).stop 2024-04-25T11:11:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.741+0000 7fc98fe00640 1 -- 172.21.15.7:0/2541774814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5220 msgr2=0x7fc9901559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.741+0000 7fc98fe00640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5220 0x7fc9901559a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc988009f40 tx=0x7fc988004a40 comp rx=0 tx=0).stop 2024-04-25T11:11:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.741+0000 7fc98fe00640 1 -- 172.21.15.7:0/2541774814 shutdown_connections 2024-04-25T11:11:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.741+0000 7fc98fe00640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 0x7fc96c0769c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.741+0000 7fc98fe00640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5220 0x7fc9901559a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.741+0000 7fc98fe00640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9900af420 0x7fc990145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.741+0000 7fc98fe00640 1 --2- 172.21.15.7:0/2541774814 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9900aed40 0x7fc99015edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.741+0000 7fc98fe00640 1 -- 172.21.15.7:0/2541774814 >> 172.21.15.7:0/2541774814 conn(0x7fc9900b3820 msgr2=0x7fc9900b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.742+0000 7fc98fe00640 1 -- 172.21.15.7:0/2541774814 shutdown_connections 2024-04-25T11:11:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:09.742+0000 7fc98fe00640 1 -- 172.21.15.7:0/2541774814 wait complete. 2024-04-25T11:11:09.766 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.049+0000 7f778a800640 1 Processor -- start 2024-04-25T11:11:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.049+0000 7f778a800640 1 -- start start 2024-04-25T11:11:10.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.051+0000 7f778a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f778415ef90 0x7f778409f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.052+0000 7f778a800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f778415f4e0 con 0x7f77840af540 2024-04-25T11:11:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.052+0000 7f778a800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77840b0560 con 0x7f778415bc40 2024-04-25T11:11:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.052+0000 7f778a800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7784071190 con 0x7f778415ef90 2024-04-25T11:11:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.052+0000 7f7783e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f77840af540 0x7f77840cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54920/0 (socket says 172.21.15.7:54920) 2024-04-25T11:11:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.052+0000 7f7783e00640 1 -- 172.21.15.7:0/3576077862 learned_addr learned my addr 172.21.15.7:0/3576077862 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:10.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.052+0000 7f7783400640 1 --2- 172.21.15.7:0/3576077862 >> v2:172.21.15.87:3300/0 conn(0x7f778415ef90 0x7f778409f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.052+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 592778242 0 0) 0x7f778415f4e0 con 0x7f77840af540 2024-04-25T11:11:10.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7768003520 con 0x7f77840af540 2024-04-25T11:11:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4019728748 0 0) 0x7f7768003520 con 0x7f77840af540 2024-04-25T11:11:10.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 >> v2:172.21.15.87:3300/0 conn(0x7f778415ef90 msgr2=0x7f778409f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 --2- 172.21.15.7:0/3576077862 >> v2:172.21.15.87:3300/0 conn(0x7f778415ef90 0x7f778409f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 >> v1:172.21.15.87:6789/0 conn(0x7f778415bc40 legacy=0x7f77840aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77840b0840 con 0x7f77840af540 2024-04-25T11:11:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f777c0019f0 con 0x7f77840af540 2024-04-25T11:11:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 >> v1:172.21.15.7:6789/0 conn(0x7f77840af540 legacy=0x7f77840cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 --2- 172.21.15.7:0/3576077862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77680039b0 0x7f7768003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 --2- 172.21.15.7:0/3576077862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 0x7f7768004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 --2- 172.21.15.7:0/3576077862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7768005250 0x7f776800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7784071190 con 0x7f7768004800 2024-04-25T11:11:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f776800c1e0 con 0x7f77680039b0 2024-04-25T11:11:10.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f776800c3e0 con 0x7f7768005250 2024-04-25T11:11:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.053+0000 7f777aa00640 1 --2- 172.21.15.7:0/3576077862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 0x7f7768004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.054+0000 7f7783e00640 1 --2- 172.21.15.7:0/3576077862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7768005250 0x7f776800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.054+0000 7f7783400640 1 --2- 172.21.15.7:0/3576077862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77680039b0 0x7f7768003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.054+0000 7f777aa00640 1 -- 172.21.15.7:0/3576077862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7768005250 msgr2=0x7f776800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.054+0000 7f777aa00640 1 --2- 172.21.15.7:0/3576077862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7768005250 0x7f776800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.054+0000 7f777aa00640 1 -- 172.21.15.7:0/3576077862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77680039b0 msgr2=0x7f7768003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.054+0000 7f777aa00640 1 --2- 172.21.15.7:0/3576077862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77680039b0 0x7f7768003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.054+0000 7f777aa00640 1 -- 172.21.15.7:0/3576077862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7768003520 con 0x7f7768004800 2024-04-25T11:11:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.054+0000 7f7783e00640 1 -- 172.21.15.7:0/3576077862 reap_dead start 2024-04-25T11:11:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.055+0000 7f777aa00640 1 --2- 172.21.15.7:0/3576077862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 0x7f7768004c10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f776c001750 tx=0x7f7768005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c567373ce13d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.055+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c001370 con 0x7f7768004800 2024-04-25T11:11:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.055+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776c0052d0 con 0x7f7768004800 2024-04-25T11:11:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.055+0000 7f7782a00640 1 -- 172.21.15.7:0/3576077862 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c02abc0 con 0x7f7768004800 2024-04-25T11:11:10.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.056+0000 7f778a800640 1 -- 172.21.15.7:0/3576077862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 msgr2=0x7f7768004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.056+0000 7f778a800640 1 --2- 172.21.15.7:0/3576077862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 0x7f7768004c10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f776c001750 tx=0x7f7768005830 comp rx=0 tx=0).stop 2024-04-25T11:11:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.056+0000 7f778a800640 1 -- 172.21.15.7:0/3576077862 shutdown_connections 2024-04-25T11:11:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.056+0000 7f778a800640 1 --2- 172.21.15.7:0/3576077862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 0x7f7768004c10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.056+0000 7f778a800640 1 -- 172.21.15.7:0/3576077862 >> 172.21.15.7:0/3576077862 conn(0x7f7784074200 msgr2=0x7f778406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.056+0000 7f778a800640 1 -- 172.21.15.7:0/3576077862 shutdown_connections 2024-04-25T11:11:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.056+0000 7f778a800640 1 -- 172.21.15.7:0/3576077862 wait complete. 2024-04-25T11:11:10.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.059+0000 7f778a800640 1 Processor -- start 2024-04-25T11:11:10.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.059+0000 7f778a800640 1 -- start start 2024-04-25T11:11:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f778a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 0x7f77841483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f778a800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77840b56c0 0x7f77841f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f778a800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77841f6a00 0x7f7784157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f778a800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77840b1320 con 0x7f7768004800 2024-04-25T11:11:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f778a800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f77840b0e00 con 0x7f77840b56c0 2024-04-25T11:11:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f778a800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f77840afb80 con 0x7f77841f6a00 2024-04-25T11:11:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f7783400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 0x7f77841483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f7783400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 0x7f77841483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42848/0 (socket says 172.21.15.7:42848) 2024-04-25T11:11:10.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f7783400640 1 -- 172.21.15.7:0/3387448531 learned_addr learned my addr 172.21.15.7:0/3387448531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f777aa00640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77840b56c0 0x7f77841f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f7783e00640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77841f6a00 0x7f7784157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.061+0000 7f7783400640 1 -- 172.21.15.7:0/3387448531 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77841f6a00 msgr2=0x7f7784157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.062+0000 7f7783400640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77841f6a00 0x7f7784157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.062+0000 7f7783400640 1 -- 172.21.15.7:0/3387448531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77840b56c0 msgr2=0x7f77841f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.062+0000 7f7783400640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77840b56c0 0x7f77841f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.062+0000 7f7783400640 1 -- 172.21.15.7:0/3387448531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77840b6110 con 0x7f7768004800 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.062+0000 7f7783e00640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77841f6a00 0x7f7784157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.062+0000 7f7783400640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 0x7f77841483a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f77740114e0 tx=0x7f77740119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.062+0000 7f777be00640 1 -- 172.21.15.7:0/3387448531 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774015870 con 0x7f7768004800 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.062+0000 7f777be00640 1 -- 172.21.15.7:0/3387448531 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7774015a10 con 0x7f7768004800 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.062+0000 7f777be00640 1 -- 172.21.15.7:0/3387448531 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774015dc0 con 0x7f7768004800 2024-04-25T11:11:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.062+0000 7f778a800640 1 -- 172.21.15.7:0/3387448531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77840b63f0 con 0x7f7768004800 2024-04-25T11:11:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.062+0000 7f778a800640 1 -- 172.21.15.7:0/3387448531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77840b69b0 con 0x7f7768004800 2024-04-25T11:11:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.064+0000 7f778a800640 1 -- 172.21.15.7:0/3387448531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77840d15f0 con 0x7f7768004800 2024-04-25T11:11:10.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.065+0000 7f777be00640 1 -- 172.21.15.7:0/3387448531 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7774013400 con 0x7f7768004800 2024-04-25T11:11:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.066+0000 7f777be00640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77600744e0 0x7f77600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.066+0000 7f777be00640 1 -- 172.21.15.7:0/3387448531 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f7774095ff0 con 0x7f7768004800 2024-04-25T11:11:10.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.066+0000 7f777aa00640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77600744e0 0x7f77600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.067+0000 7f777aa00640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77600744e0 0x7f77600769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f776c032000 tx=0x7f776c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:10.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.071+0000 7f777be00640 1 -- 172.21.15.7:0/3387448531 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f77840d15f0 con 0x7f7768004800 2024-04-25T11:11:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.279+0000 7f778a800640 1 -- 172.21.15.7:0/3387448531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f778406a100 con 0x7f7768004800 2024-04-25T11:11:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.282+0000 7f777be00640 1 -- 172.21.15.7:0/3387448531 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7f7774019030 con 0x7f7768004800 2024-04-25T11:11:10.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:10.284 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 -- 172.21.15.7:0/3387448531 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77600744e0 msgr2=0x7f77600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77600744e0 0x7f77600769f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f776c032000 tx=0x7f776c02e000 comp rx=0 tx=0).stop 2024-04-25T11:11:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 -- 172.21.15.7:0/3387448531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 msgr2=0x7f77841483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 0x7f77841483a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f77740114e0 tx=0x7f77740119a0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 -- 172.21.15.7:0/3387448531 shutdown_connections 2024-04-25T11:11:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77600744e0 0x7f77600769f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77841f6a00 0x7f7784157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77840b56c0 0x7f77841f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 --2- 172.21.15.7:0/3387448531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7768004800 0x7f77841483a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 -- 172.21.15.7:0/3387448531 >> 172.21.15.7:0/3387448531 conn(0x7f7784074200 msgr2=0x7f77840d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 -- 172.21.15.7:0/3387448531 shutdown_connections 2024-04-25T11:11:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.285+0000 7f778a800640 1 -- 172.21.15.7:0/3387448531 wait complete. 2024-04-25T11:11:10.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:11:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.595+0000 7f9d8be00640 1 Processor -- start 2024-04-25T11:11:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.596+0000 7f9d8be00640 1 -- start start 2024-04-25T11:11:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.597+0000 7f9d8be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9d8c0afb30 0x7f9d8c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.598+0000 7f9d8be00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d8c0cebb0 con 0x7f9d8c0aed40 2024-04-25T11:11:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.598+0000 7f9d8be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d8c0b64b0 con 0x7f9d8c0b5220 2024-04-25T11:11:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.598+0000 7f9d8be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9d8c15e1f0 con 0x7f9d8c0afb30 2024-04-25T11:11:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.598+0000 7f9d8aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9d8c0aed40 0x7f9d8c0ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54924/0 (socket says 172.21.15.7:54924) 2024-04-25T11:11:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.598+0000 7f9d8aa00640 1 -- 172.21.15.7:0/2746060779 learned_addr learned my addr 172.21.15.7:0/2746060779 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.598+0000 7f9d8b400640 1 --2- 172.21.15.7:0/2746060779 >> v2:172.21.15.87:3300/0 conn(0x7f9d8c0afb30 0x7f9d8c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.598+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2386812649 0 0) 0x7f9d7c001110 con 0x7f9d8c0aed40 2024-04-25T11:11:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.598+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d70003520 con 0x7f9d8c0aed40 2024-04-25T11:11:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3429473188 0 0) 0x7f9d70003520 con 0x7f9d8c0aed40 2024-04-25T11:11:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 >> v2:172.21.15.87:3300/0 conn(0x7f9d8c0afb30 msgr2=0x7f9d8c15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 --2- 172.21.15.7:0/2746060779 >> v2:172.21.15.87:3300/0 conn(0x7f9d8c0afb30 0x7f9d8c15dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 >> v1:172.21.15.87:6789/0 conn(0x7f9d8c0b5220 legacy=0x7f9d8c0af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c06c610 con 0x7f9d8c0aed40 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f9d7c001840 con 0x7f9d8c0aed40 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 >> v1:172.21.15.7:6789/0 conn(0x7f9d8c0aed40 legacy=0x7f9d8c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d700039b0 0x7f9d70003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d70004800 0x7f9d70004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700051c0 0x7f9d7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c15e1f0 con 0x7f9d70004800 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7000c160 con 0x7f9d700039b0 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7000c360 con 0x7f9d700051c0 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.599+0000 7f9d83e00640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d70004800 0x7f9d70004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.600+0000 7f9d8aa00640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d700039b0 0x7f9d70003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.600+0000 7f9d8b400640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700051c0 0x7f9d7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.600+0000 7f9d83e00640 1 -- 172.21.15.7:0/2746060779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700051c0 msgr2=0x7f9d7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.600+0000 7f9d83e00640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700051c0 0x7f9d7000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.600+0000 7f9d83e00640 1 -- 172.21.15.7:0/2746060779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d700039b0 msgr2=0x7f9d70003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.600+0000 7f9d83e00640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d700039b0 0x7f9d70003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.600+0000 7f9d83e00640 1 -- 172.21.15.7:0/2746060779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d70003520 con 0x7f9d70004800 2024-04-25T11:11:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.600+0000 7f9d8aa00640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d700039b0 0x7f9d70003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:11:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.600+0000 7f9d8b400640 1 -- 172.21.15.7:0/2746060779 reap_dead start 2024-04-25T11:11:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.601+0000 7f9d83e00640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d70004800 0x7f9d70004c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9d74001750 tx=0x7f9d70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead9fa256e074260 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.601+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74001370 con 0x7f9d70004800 2024-04-25T11:11:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.601+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d740052d0 con 0x7f9d70004800 2024-04-25T11:11:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.601+0000 7f9d8a000640 1 -- 172.21.15.7:0/2746060779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7402abe0 con 0x7f9d70004800 2024-04-25T11:11:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.602+0000 7f9d8be00640 1 -- 172.21.15.7:0/2746060779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d70004800 msgr2=0x7f9d70004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.602+0000 7f9d8be00640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d70004800 0x7f9d70004c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9d74001750 tx=0x7f9d70005830 comp rx=0 tx=0).stop 2024-04-25T11:11:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.602+0000 7f9d8be00640 1 -- 172.21.15.7:0/2746060779 shutdown_connections 2024-04-25T11:11:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.602+0000 7f9d8be00640 1 --2- 172.21.15.7:0/2746060779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d70004800 0x7f9d70004c10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.602+0000 7f9d8be00640 1 -- 172.21.15.7:0/2746060779 >> 172.21.15.7:0/2746060779 conn(0x7f9d8c0b3820 msgr2=0x7f9d8c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.602+0000 7f9d8be00640 1 -- 172.21.15.7:0/2746060779 shutdown_connections 2024-04-25T11:11:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.602+0000 7f9d8be00640 1 -- 172.21.15.7:0/2746060779 wait complete. 2024-04-25T11:11:10.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.605+0000 7f9d8be00640 1 Processor -- start 2024-04-25T11:11:10.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.605+0000 7f9d8be00640 1 -- start start 2024-04-25T11:11:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.606+0000 7f9d8be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d70004800 0x7f9d8c15edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.606+0000 7f9d8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d8c152170 0x7f9d8c1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.606+0000 7f9d8be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d8c152580 0x7f9d8c156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.606+0000 7f9d8be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c06ceb0 con 0x7f9d8c152580 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.606+0000 7f9d8be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8c06cbd0 con 0x7f9d8c152170 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.606+0000 7f9d8be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9d8c0b6110 con 0x7f9d70004800 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.607+0000 7f9d8b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d8c152580 0x7f9d8c156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.607+0000 7f9d8b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d8c152580 0x7f9d8c156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42858/0 (socket says 172.21.15.7:42858) 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.607+0000 7f9d8b400640 1 -- 172.21.15.7:0/286259031 learned_addr learned my addr 172.21.15.7:0/286259031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.607+0000 7f9d8aa00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d70004800 0x7f9d8c15edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.607+0000 7f9d8b400640 1 -- 172.21.15.7:0/286259031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d70004800 msgr2=0x7f9d8c15edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.607+0000 7f9d8b400640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d70004800 0x7f9d8c15edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.607+0000 7f9d8b400640 1 -- 172.21.15.7:0/286259031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d8c152170 msgr2=0x7f9d8c1561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.607+0000 7f9d83e00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d8c152170 0x7f9d8c1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.607+0000 7f9d8b400640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d8c152170 0x7f9d8c1561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.607+0000 7f9d8b400640 1 -- 172.21.15.7:0/286259031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8c15f720 con 0x7f9d8c152580 2024-04-25T11:11:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.607+0000 7f9d8aa00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d70004800 0x7f9d8c15edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.608+0000 7f9d8b400640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d8c152580 0x7f9d8c156730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0b1350 tx=0x7f9d8400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.608+0000 7f9d82a00640 1 -- 172.21.15.7:0/286259031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84016620 con 0x7f9d8c152580 2024-04-25T11:11:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.608+0000 7f9d82a00640 1 -- 172.21.15.7:0/286259031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d8401b070 con 0x7f9d8c152580 2024-04-25T11:11:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.608+0000 7f9d82a00640 1 -- 172.21.15.7:0/286259031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84016a20 con 0x7f9d8c152580 2024-04-25T11:11:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.608+0000 7f9d8be00640 1 -- 172.21.15.7:0/286259031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8c15fa00 con 0x7f9d8c152580 2024-04-25T11:11:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.608+0000 7f9d8be00640 1 -- 172.21.15.7:0/286259031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d8c15ffc0 con 0x7f9d8c152580 2024-04-25T11:11:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.609+0000 7f9d8be00640 1 -- 172.21.15.7:0/286259031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8c15f490 con 0x7f9d8c152580 2024-04-25T11:11:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.610+0000 7f9d82a00640 1 -- 172.21.15.7:0/286259031 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9d84016bc0 con 0x7f9d8c152580 2024-04-25T11:11:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.612+0000 7f9d82a00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d680744b0 0x7f9d680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.612+0000 7f9d82a00640 1 -- 172.21.15.7:0/286259031 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f9d84096620 con 0x7f9d8c152580 2024-04-25T11:11:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.612+0000 7f9d8aa00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d680744b0 0x7f9d680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.613+0000 7f9d8aa00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d680744b0 0x7f9d680769c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c06c310 tx=0x7f9d7c000ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.617+0000 7f9d82a00640 1 -- 172.21.15.7:0/286259031 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d84025020 con 0x7f9d8c152580 2024-04-25T11:11:10.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.867+0000 7f9d8be00640 1 -- 172.21.15.7:0/286259031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d8c050740 con 0x7f9d8c152580 2024-04-25T11:11:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.868+0000 7f9d82a00640 1 -- 172.21.15.7:0/286259031 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7f9d84060a40 con 0x7f9d8c152580 2024-04-25T11:11:10.868 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:10.869 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3458223722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3458223722},{"type":"v1","addr":"172.21.15.87:6835","nonce":3458223722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1383067804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1383067804},{"type":"v1","addr":"172.21.15.7:6835","nonce":1383067804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/435824250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":435824250},{"type":"v1","addr":"172.21.15.7:6837","nonce":435824250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:10:23.023233+0000","modified":"2024-04-25T11:10:53.526222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1091275258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1091275258},{"type":"v1","addr":"172.21.15.87:6834","nonce":1091275258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:11:10.869 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-04-25T11:11:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.871+0000 7f9d8be00640 1 -- 172.21.15.7:0/286259031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d680744b0 msgr2=0x7f9d680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.871+0000 7f9d8be00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d680744b0 0x7f9d680769c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c06c310 tx=0x7f9d7c000ef0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.871+0000 7f9d8be00640 1 -- 172.21.15.7:0/286259031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d8c152580 msgr2=0x7f9d8c156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.871+0000 7f9d8be00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d8c152580 0x7f9d8c156730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0b1350 tx=0x7f9d8400f520 comp rx=0 tx=0).stop 2024-04-25T11:11:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.871+0000 7f9d8be00640 1 -- 172.21.15.7:0/286259031 shutdown_connections 2024-04-25T11:11:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.871+0000 7f9d8be00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d680744b0 0x7f9d680769c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.871+0000 7f9d8be00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d8c152580 0x7f9d8c156730 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.871+0000 7f9d8be00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d8c152170 0x7f9d8c1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.871+0000 7f9d8be00640 1 --2- 172.21.15.7:0/286259031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d70004800 0x7f9d8c15edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.871+0000 7f9d8be00640 1 -- 172.21.15.7:0/286259031 >> 172.21.15.7:0/286259031 conn(0x7f9d8c0b3820 msgr2=0x7f9d8c0af400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.872+0000 7f9d8be00640 1 -- 172.21.15.7:0/286259031 shutdown_connections 2024-04-25T11:11:10.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:10.872+0000 7f9d8be00640 1 -- 172.21.15.7:0/286259031 wait complete. 2024-04-25T11:11:10.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:11:10.938 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:11:11.306 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:10:23.023233+0000', 'modified': '2024-04-25T11:10:53.526222+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6834/1091275258', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6832', 'nonce': 1091275258}, {'type': 'v1', 'addr': '172.21.15.87:6834', 'nonce': 1091275258}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-04-25T11:11:11.306 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:11:11.306 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T11:11:11.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.607+0000 7f8dc6a00640 1 Processor -- start 2024-04-25T11:11:11.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.607+0000 7f8dc6a00640 1 -- start start 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.609+0000 7f8dc6a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dc006d580 0x7f8dc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.609+0000 7f8dc6a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8dc006f390 0x7f8dc006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.609+0000 7f8dc6a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dc00b55a0 con 0x7f8dc00b5190 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.609+0000 7f8dc6a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8dc006ded0 con 0x7f8dc006d580 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.609+0000 7f8dc6a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc015d560 con 0x7f8dc006f390 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.609+0000 7f8dbf400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dc006d580 0x7f8dc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.609+0000 7f8dbf400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8dc006d580 0x7f8dc006d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42862/0 (socket says 172.21.15.7:42862) 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.609+0000 7f8dbf400640 1 -- 172.21.15.7:0/1730131506 learned_addr learned my addr 172.21.15.7:0/1730131506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.610+0000 7f8dc4a00640 1 --2- 172.21.15.7:0/1730131506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8dc006f390 0x7f8dc006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.610+0000 7f8dbf400640 1 -- 172.21.15.7:0/1730131506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8dc006f390 msgr2=0x7f8dc006f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.610+0000 7f8dbf400640 1 --2- 172.21.15.7:0/1730131506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8dc006f390 0x7f8dc006f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.610+0000 7f8dbf400640 1 -- 172.21.15.7:0/1730131506 >> v1:172.21.15.87:6789/0 conn(0x7f8dc00b5190 legacy=0x7f8dc00d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:11.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.610+0000 7f8dbf400640 1 -- 172.21.15.7:0/1730131506 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc015d820 con 0x7f8dc006d580 2024-04-25T11:11:11.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.611+0000 7f8dbf400640 1 --2- 172.21.15.7:0/1730131506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dc006d580 0x7f8dc006d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009870 tx=0x7f8da802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf5cebcc7dbfd2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:11.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.611+0000 7f8dbea00640 1 -- 172.21.15.7:0/1730131506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8037070 con 0x7f8dc006d580 2024-04-25T11:11:11.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.611+0000 7f8dbea00640 1 -- 172.21.15.7:0/1730131506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da8032440 con 0x7f8dc006d580 2024-04-25T11:11:11.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.611+0000 7f8dbea00640 1 -- 172.21.15.7:0/1730131506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8032810 con 0x7f8dc006d580 2024-04-25T11:11:11.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.612+0000 7f8dc6a00640 1 -- 172.21.15.7:0/1730131506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dc006d580 msgr2=0x7f8dc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:11.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.612+0000 7f8dc6a00640 1 --2- 172.21.15.7:0/1730131506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dc006d580 0x7f8dc006d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009870 tx=0x7f8da802a270 comp rx=0 tx=0).stop 2024-04-25T11:11:11.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.612+0000 7f8dc6a00640 1 -- 172.21.15.7:0/1730131506 shutdown_connections 2024-04-25T11:11:11.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.612+0000 7f8dc6a00640 1 --2- 172.21.15.7:0/1730131506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8dc006f390 0x7f8dc006f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.612+0000 7f8dc6a00640 1 --2- 172.21.15.7:0/1730131506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dc006d580 0x7f8dc006d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.612+0000 7f8dc6a00640 1 -- 172.21.15.7:0/1730131506 >> 172.21.15.7:0/1730131506 conn(0x7f8dc00732b0 msgr2=0x7f8dc00736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.612+0000 7f8dc6a00640 1 -- 172.21.15.7:0/1730131506 shutdown_connections 2024-04-25T11:11:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.612+0000 7f8dc6a00640 1 -- 172.21.15.7:0/1730131506 wait complete. 2024-04-25T11:11:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.614+0000 7f8dc6a00640 1 Processor -- start 2024-04-25T11:11:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.615+0000 7f8dc6a00640 1 -- start start 2024-04-25T11:11:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.616+0000 7f8dc6a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8dc006d580 0x7f8dc014efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.616+0000 7f8dc6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8dc006f390 0x7f8dc0156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.616+0000 7f8dc6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dc00b5190 0x7f8dc00af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.616+0000 7f8dc6a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc015e0c0 con 0x7f8dc00b5190 2024-04-25T11:11:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.616+0000 7f8dc6a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc015dde0 con 0x7f8dc006f390 2024-04-25T11:11:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.616+0000 7f8dc6a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc009f8e0 con 0x7f8dc006d580 2024-04-25T11:11:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dc4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dc00b5190 0x7f8dc00af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dc4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dc00b5190 0x7f8dc00af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42864/0 (socket says 172.21.15.7:42864) 2024-04-25T11:11:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dc4a00640 1 -- 172.21.15.7:0/2042120083 learned_addr learned my addr 172.21.15.7:0/2042120083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dbf400640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8dc006f390 0x7f8dc0156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8dc006d580 0x7f8dc014efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dc4a00640 1 -- 172.21.15.7:0/2042120083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8dc006d580 msgr2=0x7f8dc014efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dc4a00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8dc006d580 0x7f8dc014efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dc4a00640 1 -- 172.21.15.7:0/2042120083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8dc006f390 msgr2=0x7f8dc0156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dc4a00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8dc006f390 0x7f8dc0156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dc4a00640 1 -- 172.21.15.7:0/2042120083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc014f520 con 0x7f8dc00b5190 2024-04-25T11:11:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8dc006d580 0x7f8dc014efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dbf400640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8dc006f390 0x7f8dc0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.617+0000 7f8dc4a00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dc00b5190 0x7f8dc00af8d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8db8007e60 tx=0x7f8db800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.618+0000 7f8db7e00640 1 -- 172.21.15.7:0/2042120083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8014420 con 0x7f8dc00b5190 2024-04-25T11:11:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.618+0000 7f8db7e00640 1 -- 172.21.15.7:0/2042120083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db8019070 con 0x7f8dc00b5190 2024-04-25T11:11:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.618+0000 7f8db7e00640 1 -- 172.21.15.7:0/2042120083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8014820 con 0x7f8dc00b5190 2024-04-25T11:11:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.618+0000 7f8dc6a00640 1 -- 172.21.15.7:0/2042120083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc0150410 con 0x7f8dc00b5190 2024-04-25T11:11:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.618+0000 7f8dc6a00640 1 -- 172.21.15.7:0/2042120083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc01509d0 con 0x7f8dc00b5190 2024-04-25T11:11:11.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.622+0000 7f8db7e00640 1 -- 172.21.15.7:0/2042120083 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8db80149c0 con 0x7f8dc00b5190 2024-04-25T11:11:11.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.623+0000 7f8db7e00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d9c0744e0 0x7f8d9c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.623+0000 7f8db7e00640 1 -- 172.21.15.7:0/2042120083 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f8db8095ea0 con 0x7f8dc00b5190 2024-04-25T11:11:11.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.623+0000 7f8dcda54740 1 Processor -- start 2024-04-25T11:11:11.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.623+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d9c0744e0 0x7f8d9c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.623+0000 7f8dcda54740 1 -- start start 2024-04-25T11:11:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.624+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d9c0744e0 0x7f8d9c0769f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000f60 tx=0x7f8db0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.625+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.625+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445ca3b0 0x5644449dad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.625+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445d2590 0x5644449db2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.625+0000 7f8dcda54740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5644448f9600 con 0x5644445f3270 2024-04-25T11:11:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.625+0000 7f8dcda54740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564444896100 con 0x5644445ca3b0 2024-04-25T11:11:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.625+0000 7f8dcda54740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5644448f55c0 con 0x5644445d2590 2024-04-25T11:11:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.625+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.625+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42876/0 (socket says 172.21.15.7:42876) 2024-04-25T11:11:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.625+0000 7f8dbfe00640 1 -- 172.21.15.7:0/2689842921 learned_addr learned my addr 172.21.15.7:0/2689842921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.626+0000 7f8dc4a00640 1 --2- 172.21.15.7:0/2689842921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445d2590 0x5644449db2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.626+0000 7f8dbf400640 1 --2- 172.21.15.7:0/2689842921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445ca3b0 0x5644449dad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.626+0000 7f8dbfe00640 1 -- 172.21.15.7:0/2689842921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445d2590 msgr2=0x5644449db2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.626+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/2689842921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445d2590 0x5644449db2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.626+0000 7f8dbfe00640 1 -- 172.21.15.7:0/2689842921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445ca3b0 msgr2=0x5644449dad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.626+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/2689842921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445ca3b0 0x5644449dad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.626+0000 7f8dbfe00640 1 -- 172.21.15.7:0/2689842921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56444460b2d0 con 0x5644445f3270 2024-04-25T11:11:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.626+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/2689842921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c0040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8db000da60 tx=0x7f8db0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506c70dfbd1c5832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.626+0000 7f8dafe00640 1 -- 172.21.15.7:0/2689842921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0024070 con 0x5644445f3270 2024-04-25T11:11:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.627+0000 7f8dafe00640 1 -- 172.21.15.7:0/2689842921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db0014ba0 con 0x5644445f3270 2024-04-25T11:11:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.627+0000 7f8dafe00640 1 -- 172.21.15.7:0/2689842921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db001f400 con 0x5644445f3270 2024-04-25T11:11:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.627+0000 7f8dcda54740 1 -- 172.21.15.7:0/2689842921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 msgr2=0x5644449c0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.627+0000 7f8dcda54740 1 --2- 172.21.15.7:0/2689842921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c0040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8db000da60 tx=0x7f8db0010000 comp rx=0 tx=0).stop 2024-04-25T11:11:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.628+0000 7f8dcda54740 1 -- 172.21.15.7:0/2689842921 shutdown_connections 2024-04-25T11:11:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.628+0000 7f8dcda54740 1 --2- 172.21.15.7:0/2689842921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445d2590 0x5644449db2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.628+0000 7f8dcda54740 1 --2- 172.21.15.7:0/2689842921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445ca3b0 0x5644449dad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.628+0000 7f8dcda54740 1 --2- 172.21.15.7:0/2689842921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c0040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.628+0000 7f8dcda54740 1 -- 172.21.15.7:0/2689842921 >> 172.21.15.7:0/2689842921 conn(0x5644445e73c0 msgr2=0x5644449956a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.628+0000 7f8dcda54740 1 -- 172.21.15.7:0/2689842921 shutdown_connections 2024-04-25T11:11:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.628+0000 7f8dcda54740 1 -- 172.21.15.7:0/2689842921 wait complete. 2024-04-25T11:11:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.629+0000 7f8dcda54740 1 -- start start 2024-04-25T11:11:11.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.629+0000 7f8dcda54740 1 Processor -- start 2024-04-25T11:11:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445ca3b0 0x5644449dada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445d2590 0x5644449edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dcda54740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564444880f70 con 0x5644445f3270 2024-04-25T11:11:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dcda54740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5644449d6110 con 0x5644445d2590 2024-04-25T11:11:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dcda54740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5644449d62b0 con 0x5644445ca3b0 2024-04-25T11:11:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dcda54740 10 client.0 _ll_register_callbacks cb 0x5644448ef480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dc4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dc4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42882/0 (socket says 172.21.15.7:42882) 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dc4a00640 1 -- 172.21.15.7:0/1523464289 learned_addr learned my addr 172.21.15.7:0/1523464289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dc4a00640 1 -- 172.21.15.7:0/1523464289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445ca3b0 msgr2=0x5644449dada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dc4a00640 1 --2- 172.21.15.7:0/1523464289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445ca3b0 0x5644449dada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dc4a00640 1 -- 172.21.15.7:0/1523464289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445d2590 msgr2=0x5644449edbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dc4a00640 1 --2- 172.21.15.7:0/1523464289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445d2590 0x5644449edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.631+0000 7f8dc4a00640 1 -- 172.21.15.7:0/1523464289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644446d56b0 con 0x5644445f3270 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.632+0000 7f8dc4a00640 1 --2- 172.21.15.7:0/1523464289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c2dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8db800efd0 tx=0x7f8db80682f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b448197701e20783 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.632+0000 7f8dacc00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.632+0000 7f8dacc00640 1 -- 172.21.15.7:0/1523464289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db805f910 con 0x5644445f3270 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.632+0000 7f8dacc00640 1 -- 172.21.15.7:0/1523464289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f8db805eda0 con 0x5644445f3270 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.632+0000 7f8dcda54740 1 -- 172.21.15.7:0/1523464289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8db0014ba0 con 0x5644445f3270 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.632+0000 7f8dacc00640 1 -- 172.21.15.7:0/1523464289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f8db8063e40 con 0x5644445f3270 2024-04-25T11:11:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.632+0000 7f8dcda54740 10 client.4779 fetch_fsmap learned FSMap version 9 2024-04-25T11:11:11.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.632+0000 7f8dcda54740 1 -- 172.21.15.7:0/1523464289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5644449c28a0 con 0x5644445f3270 2024-04-25T11:11:11.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.632+0000 7f8dacc00640 1 -- 172.21.15.7:0/1523464289 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db0014ba0 con 0x5644445f3270 2024-04-25T11:11:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.633+0000 7f8dacc00640 1 -- 172.21.15.7:0/1523464289 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f8db805b4b0 con 0x5644445f3270 2024-04-25T11:11:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.633+0000 7f8dcda54740 10 client.4779 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T11:11:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.633+0000 7f8dcda54740 10 client.4779 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:11:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.633+0000 7f8dcda54740 20 client.4779 populate_metadata read hostname 'smithi007' 2024-04-25T11:11:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.633+0000 7f8dcda54740 1 --2- 172.21.15.7:0/1523464289 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x5644445b1e40 0x5644449f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.633+0000 7f8dcda54740 4 client.4779 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:11:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.633+0000 7f8dcda54740 1 -- 172.21.15.7:0/1523464289 --> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564444907210 con 0x5644445b1e40 2024-04-25T11:11:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.633+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/1523464289 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x5644445b1e40 0x5644449f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.634+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/1523464289 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x5644445b1e40 0x5644449f83d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ba6d720de5407b server_cookie=8aa65c9e357df96f in_seq=0 out_seq=0 2024-04-25T11:11:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.634+0000 7f8dacc00640 10 client.4779 ms_handle_connect on v2:172.21.15.87:6832/1091275258 2024-04-25T11:11:11.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.636+0000 7f8dacc00640 1 -- 172.21.15.7:0/1523464289 <== mds.0 v2:172.21.15.87:6832/1091275258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564444907210 con 0x5644445b1e40 2024-04-25T11:11:11.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.636+0000 7f8dacc00640 10 client.4779 handle_command_reply: tid=0 2024-04-25T11:11:11.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.636+0000 7f8dcda54740 1 client.4779 shutdown 2024-04-25T11:11:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dcda54740 1 -- 172.21.15.7:0/1523464289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 msgr2=0x5644449c2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dcda54740 1 --2- 172.21.15.7:0/1523464289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c2dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8db800efd0 tx=0x7f8db80682f0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dcda54740 1 -- 172.21.15.7:0/1523464289 shutdown_connections 2024-04-25T11:11:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dcda54740 1 --2- 172.21.15.7:0/1523464289 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x5644445b1e40 0x5644449f83d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dcda54740 1 --2- 172.21.15.7:0/1523464289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445f3270 0x5644449c2dc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dcda54740 1 --2- 172.21.15.7:0/1523464289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445d2590 0x5644449edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dcda54740 1 --2- 172.21.15.7:0/1523464289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445ca3b0 0x5644449dada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dacc00640 0 client.4779 ms_handle_reset on v2:172.21.15.87:6832/1091275258 2024-04-25T11:11:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dcda54740 1 -- 172.21.15.7:0/1523464289 >> 172.21.15.7:0/1523464289 conn(0x5644445e73c0 msgr2=0x5644449956a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dcda54740 1 -- 172.21.15.7:0/1523464289 shutdown_connections 2024-04-25T11:11:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dcda54740 1 -- 172.21.15.7:0/1523464289 wait complete. 2024-04-25T11:11:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.637+0000 7f8dcda54740 20 client.4779 trim_cache size 0 max 16384 2024-04-25T11:11:11.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.658+0000 7f8dcda54740 1 Processor -- start 2024-04-25T11:11:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.658+0000 7f8dcda54740 1 -- start start 2024-04-25T11:11:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445e73c0 0x564444a4f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445ca3b0 0x5644449dada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445d2590 0x5644449db2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dcda54740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc02034f0 con 0x5644445e73c0 2024-04-25T11:11:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dcda54740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc01528a0 con 0x5644445ca3b0 2024-04-25T11:11:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dcda54740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc0202e50 con 0x5644445d2590 2024-04-25T11:11:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445e73c0 0x564444a4f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445e73c0 0x564444a4f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42886/0 (socket says 172.21.15.7:42886) 2024-04-25T11:11:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dbfe00640 1 -- 172.21.15.7:0/1450960749 learned_addr learned my addr 172.21.15.7:0/1450960749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dbfe00640 1 -- 172.21.15.7:0/1450960749 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445d2590 msgr2=0x5644449db2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/1450960749 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445d2590 0x5644449db2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dbfe00640 1 -- 172.21.15.7:0/1450960749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445ca3b0 msgr2=0x5644449dada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/1450960749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445ca3b0 0x5644449dada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.660+0000 7f8dbfe00640 1 -- 172.21.15.7:0/1450960749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644449d8a30 con 0x5644445e73c0 2024-04-25T11:11:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.661+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/1450960749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445e73c0 0x564444a4f980 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8db001f000 tx=0x7f8db0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c918e9d5e24d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.661+0000 7f8daf400640 1 -- 172.21.15.7:0/1450960749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0024070 con 0x5644445e73c0 2024-04-25T11:11:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.661+0000 7f8daf400640 1 -- 172.21.15.7:0/1450960749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db0032070 con 0x5644445e73c0 2024-04-25T11:11:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.661+0000 7f8daf400640 1 -- 172.21.15.7:0/1450960749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0007470 con 0x5644445e73c0 2024-04-25T11:11:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.662+0000 7f8dcda54740 1 -- 172.21.15.7:0/1450960749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445e73c0 msgr2=0x564444a4f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.662+0000 7f8dcda54740 1 --2- 172.21.15.7:0/1450960749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445e73c0 0x564444a4f980 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8db001f000 tx=0x7f8db0012000 comp rx=0 tx=0).stop 2024-04-25T11:11:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.662+0000 7f8dcda54740 1 -- 172.21.15.7:0/1450960749 shutdown_connections 2024-04-25T11:11:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.662+0000 7f8dcda54740 1 --2- 172.21.15.7:0/1450960749 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445d2590 0x5644449db2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.662+0000 7f8dcda54740 1 --2- 172.21.15.7:0/1450960749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445ca3b0 0x5644449dada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.662+0000 7f8dcda54740 1 --2- 172.21.15.7:0/1450960749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445e73c0 0x564444a4f980 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.662+0000 7f8dcda54740 1 -- 172.21.15.7:0/1450960749 >> 172.21.15.7:0/1450960749 conn(0x5644449c2ef0 msgr2=0x5644449d5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.662+0000 7f8dcda54740 1 -- 172.21.15.7:0/1450960749 shutdown_connections 2024-04-25T11:11:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.662+0000 7f8dcda54740 1 -- 172.21.15.7:0/1450960749 wait complete. 2024-04-25T11:11:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.663+0000 7f8dcda54740 1 -- start start 2024-04-25T11:11:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.663+0000 7f8dcda54740 1 Processor -- start 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.665+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445ca3b0 0x564444a4bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.665+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445d2590 0x5644449aa250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.665+0000 7f8dcda54740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445e73c0 0x5644449aa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.665+0000 7f8dcda54740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5644449c37d0 con 0x5644445ca3b0 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.665+0000 7f8dcda54740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5644449ee170 con 0x5644445d2590 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.665+0000 7f8dcda54740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5644449ae890 con 0x5644445e73c0 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.665+0000 7f8dcda54740 10 client.0 _ll_register_callbacks cb 0x7f8db805ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.665+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445ca3b0 0x564444a4bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.665+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445ca3b0 0x564444a4bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42890/0 (socket says 172.21.15.7:42890) 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.665+0000 7f8dbfe00640 1 -- 172.21.15.7:0/3140619946 learned_addr learned my addr 172.21.15.7:0/3140619946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dbf400640 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445d2590 0x5644449aa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dc4a00640 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445e73c0 0x5644449aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dbfe00640 1 -- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445e73c0 msgr2=0x5644449aa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445e73c0 0x5644449aa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dbfe00640 1 -- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445d2590 msgr2=0x5644449aa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445d2590 0x5644449aa250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dbfe00640 1 -- 172.21.15.7:0/3140619946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644449c1ef0 con 0x5644445ca3b0 2024-04-25T11:11:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dbfe00640 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445ca3b0 0x564444a4bb50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8db000da00 tx=0x7f8db0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fcdcb9cc0e761d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dacc00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dacc00640 1 -- 172.21.15.7:0/3140619946 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db003a030 con 0x5644445ca3b0 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dacc00640 1 -- 172.21.15.7:0/3140619946 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f8db0007240 con 0x5644445ca3b0 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.666+0000 7f8dcda54740 1 -- 172.21.15.7:0/3140619946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8db0032070 con 0x5644445ca3b0 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.667+0000 7f8dacc00640 1 -- 172.21.15.7:0/3140619946 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f8db0046050 con 0x5644445ca3b0 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.667+0000 7f8dcda54740 10 client.4785 fetch_fsmap learned FSMap version 9 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.667+0000 7f8dcda54740 1 -- 172.21.15.7:0/3140619946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5644449ee040 con 0x5644445ca3b0 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.667+0000 7f8dacc00640 1 -- 172.21.15.7:0/3140619946 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db0032070 con 0x5644445ca3b0 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.667+0000 7f8dacc00640 1 -- 172.21.15.7:0/3140619946 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f8db0049020 con 0x5644445ca3b0 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.667+0000 7f8dcda54740 10 client.4785 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.667+0000 7f8dcda54740 10 client.4785 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.667+0000 7f8dcda54740 20 client.4785 populate_metadata read hostname 'smithi007' 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.667+0000 7f8dcda54740 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x564444587280 0x5644449946c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.667+0000 7f8dcda54740 4 client.4785 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:11:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.667+0000 7f8dcda54740 1 -- 172.21.15.7:0/3140619946 --> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] -- command(tid 0: {"prefix": "status"}) -- 0x5644449da510 con 0x564444587280 2024-04-25T11:11:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.668+0000 7f8dbf400640 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x564444587280 0x5644449946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.669+0000 7f8dbf400640 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x564444587280 0x5644449946c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4300fcec5fb72b8b server_cookie=7d77416ecaf68007 in_seq=0 out_seq=0 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.669+0000 7f8dacc00640 10 client.4785 ms_handle_connect on v2:172.21.15.87:6832/1091275258 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dacc00640 1 -- 172.21.15.7:0/3140619946 <== mds.0 v2:172.21.15.87:6832/1091275258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5644449da510 con 0x564444587280 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dacc00640 10 client.4785 handle_command_reply: tid=0 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 1 client.4785 shutdown 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 1 -- 172.21.15.7:0/3140619946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445ca3b0 msgr2=0x564444a4bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445ca3b0 0x564444a4bb50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8db000da00 tx=0x7f8db0000ed0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 1 -- 172.21.15.7:0/3140619946 shutdown_connections 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:6832/1091275258,v1:172.21.15.87:6834/1091275258] conn(0x564444587280 0x5644449946c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5644445e73c0 0x5644449aa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5644445d2590 0x5644449aa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dacc00640 0 client.4785 ms_handle_reset on v2:172.21.15.87:6832/1091275258 2024-04-25T11:11:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 1 --2- 172.21.15.7:0/3140619946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5644445ca3b0 0x564444a4bb50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 1 -- 172.21.15.7:0/3140619946 >> 172.21.15.7:0/3140619946 conn(0x5644449c2ef0 msgr2=0x5644449bc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 1 -- 172.21.15.7:0/3140619946 shutdown_connections 2024-04-25T11:11:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 1 -- 172.21.15.7:0/3140619946 wait complete. 2024-04-25T11:11:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.670+0000 7f8dcda54740 20 client.4785 trim_cache size 0 max 16384 2024-04-25T11:11:11.672 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:11:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:11:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:11:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4285, 2024-04-25T11:11:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:11:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:11:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:11:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 48.227358522000003, 2024-04-25T11:11:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 9, 2024-04-25T11:11:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 28, 2024-04-25T11:11:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 26, 2024-04-25T11:11:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 54.674952673 2024-04-25T11:11:11.673 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:11:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 -- 172.21.15.7:0/2042120083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d9c0744e0 msgr2=0x7f8d9c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d9c0744e0 0x7f8d9c0769f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8db0000f60 tx=0x7f8db0009040 comp rx=0 tx=0).stop 2024-04-25T11:11:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 -- 172.21.15.7:0/2042120083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dc00b5190 msgr2=0x7f8dc00af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dc00b5190 0x7f8dc00af8d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8db8007e60 tx=0x7f8db800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 -- 172.21.15.7:0/2042120083 shutdown_connections 2024-04-25T11:11:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d9c0744e0 0x7f8d9c0769f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8dc00b5190 0x7f8dc00af8d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8dc006f390 0x7f8dc0156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 --2- 172.21.15.7:0/2042120083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8dc006d580 0x7f8dc014efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 -- 172.21.15.7:0/2042120083 >> 172.21.15.7:0/2042120083 conn(0x7f8dc00732b0 msgr2=0x7f8dc006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 -- 172.21.15.7:0/2042120083 shutdown_connections 2024-04-25T11:11:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:11.671+0000 7f8dc6a00640 1 -- 172.21.15.7:0/2042120083 wait complete. 2024-04-25T11:11:11.699 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T11:11:11.709 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T11:11:11.709 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T11:11:11.710 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T11:11:11.710 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T11:11:11.727 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T11:11:11.727 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T11:11:11.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:11.728 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:11:11.783 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T11:11:11.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:11.785 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T11:11:11.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:11.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-04-25T11:11:11.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:11.982 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:11:12.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:12.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-04-25T11:11:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-04-25T11:11:12.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:11:12.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:12.161 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:11:12.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:12.219 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-04-25T11:11:12.306 INFO:teuthology.orchestra.run.smithi087.stderr:Cannot find device "ceph-brx" 2024-04-25T11:11:12.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:11:12.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:12.310 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:11:12.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:12.363 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-04-25T11:11:12.445 INFO:teuthology.orchestra.run.smithi087.stderr:Cannot find device "ceph-brx" 2024-04-25T11:11:12.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:11:12.449 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-04-25T11:11:12.449 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:11:12.449 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:11:12.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:11:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.746+0000 7fa55ea00640 1 Processor -- start 2024-04-25T11:11:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.746+0000 7fa55ea00640 1 -- start start 2024-04-25T11:11:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.748+0000 7fa55ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa558070ca0 0x7fa558073dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.748+0000 7fa55ea00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa558160320 con 0x7fa5580b60a0 2024-04-25T11:11:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.748+0000 7fa55ea00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa558160600 con 0x7fa558146310 2024-04-25T11:11:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.749+0000 7fa55ea00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa5581608e0 con 0x7fa558070ca0 2024-04-25T11:11:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.749+0000 7fa55ca00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa5580b60a0 0x7fa5580b64b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54938/0 (socket says 172.21.15.7:54938) 2024-04-25T11:11:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.749+0000 7fa55ca00640 1 -- 172.21.15.7:0/1541019334 learned_addr learned my addr 172.21.15.7:0/1541019334 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.749+0000 7fa557400640 1 --2- 172.21.15.7:0/1541019334 >> v2:172.21.15.7:3300/0 conn(0x7fa558070ca0 0x7fa558073dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.749+0000 7fa556a00640 1 -- 172.21.15.7:0/1541019334 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 560242808 0 0) 0x7fa558160320 con 0x7fa5580b60a0 2024-04-25T11:11:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.750+0000 7fa557400640 1 -- 172.21.15.7:0/1541019334 >> v1:172.21.15.87:6789/0 conn(0x7fa558146310 legacy=0x7fa5580d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.750+0000 7fa557400640 1 -- 172.21.15.7:0/1541019334 >> v1:172.21.15.7:6789/0 conn(0x7fa5580b60a0 legacy=0x7fa5580b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.750+0000 7fa557400640 1 -- 172.21.15.7:0/1541019334 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa558160bc0 con 0x7fa558070ca0 2024-04-25T11:11:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.751+0000 7fa557400640 1 --2- 172.21.15.7:0/1541019334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa558070ca0 0x7fa558073dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa5440097a0 tx=0x7fa54402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de83c7da41600965 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.751+0000 7fa556a00640 1 -- 172.21.15.7:0/1541019334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544037070 con 0x7fa558070ca0 2024-04-25T11:11:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.751+0000 7fa556a00640 1 -- 172.21.15.7:0/1541019334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa544032640 con 0x7fa558070ca0 2024-04-25T11:11:12.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.751+0000 7fa556a00640 1 -- 172.21.15.7:0/1541019334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544032a10 con 0x7fa558070ca0 2024-04-25T11:11:12.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.752+0000 7fa55ea00640 1 -- 172.21.15.7:0/1541019334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa558070ca0 msgr2=0x7fa558073dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:12.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.752+0000 7fa55ea00640 1 --2- 172.21.15.7:0/1541019334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa558070ca0 0x7fa558073dd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa5440097a0 tx=0x7fa54402a2c0 comp rx=0 tx=0).stop 2024-04-25T11:11:12.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.752+0000 7fa55ea00640 1 -- 172.21.15.7:0/1541019334 shutdown_connections 2024-04-25T11:11:12.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.752+0000 7fa55ea00640 1 --2- 172.21.15.7:0/1541019334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa558070ca0 0x7fa558073dd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:12.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.752+0000 7fa55ea00640 1 -- 172.21.15.7:0/1541019334 >> 172.21.15.7:0/1541019334 conn(0x7fa5580732b0 msgr2=0x7fa5580736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:12.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.752+0000 7fa55ea00640 1 -- 172.21.15.7:0/1541019334 shutdown_connections 2024-04-25T11:11:12.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.752+0000 7fa55ea00640 1 -- 172.21.15.7:0/1541019334 wait complete. 2024-04-25T11:11:12.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.754+0000 7fa55ea00640 1 Processor -- start 2024-04-25T11:11:12.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.754+0000 7fa55ea00640 1 -- start start 2024-04-25T11:11:12.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.756+0000 7fa55ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa558070ca0 0x7fa55815b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.756+0000 7fa55ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5580b60a0 0x7fa5581e70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.756+0000 7fa55ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa558146310 0x7fa5581e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.756+0000 7fa55ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa55806f360 con 0x7fa558146310 2024-04-25T11:11:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.756+0000 7fa55ea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa55806f1e0 con 0x7fa5580b60a0 2024-04-25T11:11:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.756+0000 7fa55ea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa55806f4e0 con 0x7fa558070ca0 2024-04-25T11:11:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.756+0000 7fa55ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa558146310 0x7fa5581e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.756+0000 7fa55ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa558146310 0x7fa5581e7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42908/0 (socket says 172.21.15.7:42908) 2024-04-25T11:11:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.756+0000 7fa55ca00640 1 -- 172.21.15.7:0/380786376 learned_addr learned my addr 172.21.15.7:0/380786376 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.757+0000 7fa557e00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa558070ca0 0x7fa55815b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.757+0000 7fa557400640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5580b60a0 0x7fa5581e70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.757+0000 7fa55ca00640 1 -- 172.21.15.7:0/380786376 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa558070ca0 msgr2=0x7fa55815b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.757+0000 7fa55ca00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa558070ca0 0x7fa55815b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.757+0000 7fa55ca00640 1 -- 172.21.15.7:0/380786376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5580b60a0 msgr2=0x7fa5581e70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.757+0000 7fa55ca00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5580b60a0 0x7fa5581e70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.757+0000 7fa55ca00640 1 -- 172.21.15.7:0/380786376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55815bc50 con 0x7fa558146310 2024-04-25T11:11:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.757+0000 7fa55ca00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa558146310 0x7fa5581e7630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c7b0 tx=0x7fa55000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.758+0000 7fa54fe00640 1 -- 172.21.15.7:0/380786376 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa550018070 con 0x7fa558146310 2024-04-25T11:11:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.758+0000 7fa54fe00640 1 -- 172.21.15.7:0/380786376 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55000a5e0 con 0x7fa558146310 2024-04-25T11:11:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.758+0000 7fa54fe00640 1 -- 172.21.15.7:0/380786376 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55000ce60 con 0x7fa558146310 2024-04-25T11:11:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.758+0000 7fa55ea00640 1 -- 172.21.15.7:0/380786376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55815be50 con 0x7fa558146310 2024-04-25T11:11:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.758+0000 7fa55ea00640 1 -- 172.21.15.7:0/380786376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5581ea340 con 0x7fa558146310 2024-04-25T11:11:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.759+0000 7fa55ea00640 1 -- 172.21.15.7:0/380786376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa558050740 con 0x7fa558146310 2024-04-25T11:11:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.760+0000 7fa54fe00640 1 -- 172.21.15.7:0/380786376 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa55001b070 con 0x7fa558146310 2024-04-25T11:11:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.762+0000 7fa54fe00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa538074500 0x7fa538076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.762+0000 7fa54fe00640 1 -- 172.21.15.7:0/380786376 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fa550094ab0 con 0x7fa558146310 2024-04-25T11:11:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.762+0000 7fa557e00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa538074500 0x7fa538076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.763+0000 7fa557e00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa538074500 0x7fa538076a10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa548007fd0 tx=0x7fa548009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:12.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:12.767+0000 7fa54fe00640 1 -- 172.21.15.7:0/380786376 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa55815bc50 con 0x7fa558146310 2024-04-25T11:11:13.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.016+0000 7fa55ea00640 1 -- 172.21.15.7:0/380786376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa5580b4b60 con 0x7fa558146310 2024-04-25T11:11:13.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.017+0000 7fa54fe00640 1 -- 172.21.15.7:0/380786376 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fa55005efb0 con 0x7fa558146310 2024-04-25T11:11:13.017 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.019+0000 7fa55ea00640 1 -- 172.21.15.7:0/380786376 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa538074500 msgr2=0x7fa538076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.019+0000 7fa55ea00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa538074500 0x7fa538076a10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa548007fd0 tx=0x7fa548009040 comp rx=0 tx=0).stop 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.019+0000 7fa55ea00640 1 -- 172.21.15.7:0/380786376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa558146310 msgr2=0x7fa5581e7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.019+0000 7fa55ea00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa558146310 0x7fa5581e7630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c7b0 tx=0x7fa55000cc70 comp rx=0 tx=0).stop 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.019+0000 7fa55ea00640 1 -- 172.21.15.7:0/380786376 shutdown_connections 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.020+0000 7fa55ea00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa538074500 0x7fa538076a10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.020+0000 7fa55ea00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa558146310 0x7fa5581e7630 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.020+0000 7fa55ea00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5580b60a0 0x7fa5581e70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.020+0000 7fa55ea00640 1 --2- 172.21.15.7:0/380786376 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa558070ca0 0x7fa55815b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.020+0000 7fa55ea00640 1 -- 172.21.15.7:0/380786376 >> 172.21.15.7:0/380786376 conn(0x7fa5580732b0 msgr2=0x7fa558142fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.020+0000 7fa55ea00640 1 -- 172.21.15.7:0/380786376 shutdown_connections 2024-04-25T11:11:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13.020+0000 7fa55ea00640 1 -- 172.21.15.7:0/380786376 wait complete. 2024-04-25T11:11:13.050 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:11:13.050 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:11:13.050 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:11:13.050 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:11:13.050 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:11:13.050 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:11:13.051 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:11:13.051 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:11:13.051 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:11:13.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:13.051 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:11:13.103 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:11:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:11:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:11:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:11:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:13.103 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:11:13.103 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:13.103 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:11:13.104 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:11:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:11:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:11:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42502sec preferred_lft 42502sec 2024-04-25T11:11:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:11:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:13.104 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:13.104 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:11:13.105 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T11:11:13.105 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:13.105 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:11:13.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-04-25T11:11:13.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-04-25T11:11:13.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-04-25T11:11:13.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T11:11:13.105 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:11:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:13.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:13.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:13.359 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:11:13.426 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:11:13.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:13.430 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:11:13.479 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:11:13.479 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:11:13.479 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:11:13.479 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:11:13.479 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:11:13.480 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:13.480 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:11:13.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:11:13.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:11:13.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:11:13.480 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:11:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:13.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:13.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:13.695 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:11:13.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:13.746 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-04-25T11:11:13.800 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:13.800 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:11:13.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:11:13.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T11:11:13.800 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:11:13.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:13.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:13.989 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T11:11:13.989 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:13.990 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:11:13.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T11:11:13.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T11:11:13.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T11:11:13.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T11:11:13.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T11:11:13.990 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:11:14.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:14.262 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:14.263 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:11:14.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:11:14.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T11:11:14.263 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:11:14.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:14.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:14.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:11:14.468 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:11:14.521 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:11:14.521 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:11:14.522 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:11:14.573 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:11:14.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:14.573 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:14.782 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:14.782 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:11:15.573 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:15.573 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:11:15.573 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:11:15.573 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:11:15.573 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:11:15.577 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:11:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:11:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:11:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:11:15.577 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:11:15.584 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:11:15.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:15.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:11:15.670 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:15.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:15.802 INFO:teuthology.orchestra.run.smithi007.stdout:client4797 2024-04-25T11:11:15.802 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:11:15.802 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:15.858 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:15.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:15.975 INFO:teuthology.orchestra.run.smithi007.stdout:client4797 2024-04-25T11:11:15.976 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:16.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:16.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:16.097 INFO:teuthology.orchestra.run.smithi007.stdout:client4797 2024-04-25T11:11:16.098 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.client4797/status') 2024-04-25T11:11:16.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:16.235 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.4797 (3)192.168.0.1:0/589412829 2024-04-25T11:11:16.236 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-04-25T11:11:16.236 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-04-25T11:11:16.236 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-04-25T11:11:16.236 INFO:teuthology.orchestra.run.smithi007.stderr:68 bytes copied, 8.6271e-05 s, 788 kB/s 2024-04-25T11:11:16.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:16.248 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-04-25T11:11:16.249 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:11:16.249 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T11:11:16.249 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:11:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.528+0000 7f8795e00640 1 Processor -- start 2024-04-25T11:11:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.528+0000 7f8795e00640 1 -- start start 2024-04-25T11:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.530+0000 7f8795e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f879015a7a0 0x7f8790161160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.530+0000 7f8795e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f87900b7b50 0x7f879006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.530+0000 7f8795e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f879006cfc0 con 0x7f879006c570 2024-04-25T11:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.530+0000 7f8795e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8790160cd0 con 0x7f87900b7b50 2024-04-25T11:11:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.530+0000 7f8795e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f879006d2a0 con 0x7f879015a7a0 2024-04-25T11:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.531+0000 7f8794a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f879015a7a0 0x7f8790161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.531+0000 7f8794a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f879015a7a0 0x7f8790161160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:60380/0 (socket says 172.21.15.7:60380) 2024-04-25T11:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.531+0000 7f8794a00640 1 -- 172.21.15.7:0/411045340 learned_addr learned my addr 172.21.15.7:0/411045340 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:16.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.531+0000 7f8795400640 1 --2- 172.21.15.7:0/411045340 >> v2:172.21.15.87:3300/0 conn(0x7f87900b7b50 0x7f879006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.532+0000 7f8794a00640 1 -- 172.21.15.7:0/411045340 >> v2:172.21.15.87:3300/0 conn(0x7f87900b7b50 msgr2=0x7f879006c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.532+0000 7f878fe00640 1 -- 172.21.15.7:0/411045340 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 90396037 0 0) 0x7f879006cfc0 con 0x7f879006c570 2024-04-25T11:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.532+0000 7f8794a00640 1 --2- 172.21.15.7:0/411045340 >> v2:172.21.15.87:3300/0 conn(0x7f87900b7b50 0x7f879006c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.532+0000 7f8794a00640 1 -- 172.21.15.7:0/411045340 >> v1:172.21.15.87:6789/0 conn(0x7f879006c570 legacy=0x7f87900b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.532+0000 7f8794a00640 1 -- 172.21.15.7:0/411045340 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8790160960 con 0x7f879015a7a0 2024-04-25T11:11:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.532+0000 7f8795400640 1 --2- 172.21.15.7:0/411045340 >> v2:172.21.15.87:3300/0 conn(0x7f87900b7b50 0x7f879006c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:11:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.533+0000 7f8794a00640 1 --2- 172.21.15.7:0/411045340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f879015a7a0 0x7f8790161160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8780009870 tx=0x7f878002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c777d898fc7e4452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.534+0000 7f878fe00640 1 -- 172.21.15.7:0/411045340 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780037070 con 0x7f879015a7a0 2024-04-25T11:11:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.534+0000 7f878fe00640 1 -- 172.21.15.7:0/411045340 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8780032440 con 0x7f879015a7a0 2024-04-25T11:11:16.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.534+0000 7f878fe00640 1 -- 172.21.15.7:0/411045340 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780032810 con 0x7f879015a7a0 2024-04-25T11:11:16.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.534+0000 7f8795e00640 1 -- 172.21.15.7:0/411045340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f879015a7a0 msgr2=0x7f8790161160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:16.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.534+0000 7f8795e00640 1 --2- 172.21.15.7:0/411045340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f879015a7a0 0x7f8790161160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8780009870 tx=0x7f878002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:11:16.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.534+0000 7f8795e00640 1 -- 172.21.15.7:0/411045340 shutdown_connections 2024-04-25T11:11:16.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.534+0000 7f8795e00640 1 --2- 172.21.15.7:0/411045340 >> v2:172.21.15.87:3300/0 conn(0x7f87900b7b50 0x7f879006c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:16.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.534+0000 7f8795e00640 1 --2- 172.21.15.7:0/411045340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f879015a7a0 0x7f8790161160 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:16.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.534+0000 7f8795e00640 1 -- 172.21.15.7:0/411045340 >> 172.21.15.7:0/411045340 conn(0x7f87900d59c0 msgr2=0x7f87900d5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:16.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.535+0000 7f8795e00640 1 -- 172.21.15.7:0/411045340 shutdown_connections 2024-04-25T11:11:16.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.535+0000 7f8795e00640 1 -- 172.21.15.7:0/411045340 wait complete. 2024-04-25T11:11:16.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.537+0000 7f8795e00640 1 Processor -- start 2024-04-25T11:11:16.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.537+0000 7f8795e00640 1 -- start start 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.539+0000 7f8795e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f879006c570 0x7f87900bad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.539+0000 7f8795e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f879015a7a0 0x7f87900c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.539+0000 7f8795e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87900cb400 0x7f87900c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.539+0000 7f8795e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87900d09f0 con 0x7f879006c570 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.539+0000 7f8795e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f87900d0870 con 0x7f879015a7a0 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.539+0000 7f8795e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87900b43b0 con 0x7f87900cb400 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.539+0000 7f8794a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f879006c570 0x7f87900bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.539+0000 7f8794a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f879006c570 0x7f87900bad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42912/0 (socket says 172.21.15.7:42912) 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.539+0000 7f8794a00640 1 -- 172.21.15.7:0/3890943295 learned_addr learned my addr 172.21.15.7:0/3890943295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.540+0000 7f8795400640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87900cb400 0x7f87900c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.540+0000 7f8794a00640 1 -- 172.21.15.7:0/3890943295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87900cb400 msgr2=0x7f87900c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:16.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.540+0000 7f8794a00640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87900cb400 0x7f87900c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:16.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.540+0000 7f8794a00640 1 -- 172.21.15.7:0/3890943295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f879015a7a0 msgr2=0x7f87900c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:16.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.540+0000 7f8794a00640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f879015a7a0 0x7f87900c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:16.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.540+0000 7f8794a00640 1 -- 172.21.15.7:0/3890943295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87900cbb30 con 0x7f879006c570 2024-04-25T11:11:16.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.540+0000 7f8794a00640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f879006c570 0x7f87900bad90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f878002a200 tx=0x7f87800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:16.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.540+0000 7f878d600640 1 -- 172.21.15.7:0/3890943295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780037040 con 0x7f879006c570 2024-04-25T11:11:16.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.541+0000 7f878d600640 1 -- 172.21.15.7:0/3890943295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8780042070 con 0x7f879006c570 2024-04-25T11:11:16.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.541+0000 7f878d600640 1 -- 172.21.15.7:0/3890943295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780030880 con 0x7f879006c570 2024-04-25T11:11:16.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.541+0000 7f8795e00640 1 -- 172.21.15.7:0/3890943295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87900cbe10 con 0x7f879006c570 2024-04-25T11:11:16.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.541+0000 7f8795e00640 1 -- 172.21.15.7:0/3890943295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87900ce140 con 0x7f879006c570 2024-04-25T11:11:16.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.542+0000 7f8795e00640 1 -- 172.21.15.7:0/3890943295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87900cca00 con 0x7f879006c570 2024-04-25T11:11:16.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.544+0000 7f878d600640 1 -- 172.21.15.7:0/3890943295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f878002ad10 con 0x7f879006c570 2024-04-25T11:11:16.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.545+0000 7f878d600640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f876c0744b0 0x7f876c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:16.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.545+0000 7f878d600640 1 -- 172.21.15.7:0/3890943295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f87800b5d00 con 0x7f879006c570 2024-04-25T11:11:16.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.545+0000 7f8787e00640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f876c0744b0 0x7f876c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:16.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.546+0000 7f8787e00640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f876c0744b0 0x7f876c0769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8790154530 tx=0x7f8778007950 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:16.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.552+0000 7f878d600640 1 -- 172.21.15.7:0/3890943295 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f878007fff0 con 0x7f879006c570 2024-04-25T11:11:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.799+0000 7f8795e00640 1 -- 172.21.15.7:0/3890943295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f87900d11e0 con 0x7f879006c570 2024-04-25T11:11:16.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.800+0000 7f878d600640 1 -- 172.21.15.7:0/3890943295 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f87800b5070 con 0x7f879006c570 2024-04-25T11:11:16.800 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T11:11:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 -- 172.21.15.7:0/3890943295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f876c0744b0 msgr2=0x7f876c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f876c0744b0 0x7f876c0769c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8790154530 tx=0x7f8778007950 comp rx=0 tx=0).stop 2024-04-25T11:11:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 -- 172.21.15.7:0/3890943295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f879006c570 msgr2=0x7f87900bad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f879006c570 0x7f87900bad90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f878002a200 tx=0x7f87800046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 -- 172.21.15.7:0/3890943295 shutdown_connections 2024-04-25T11:11:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f876c0744b0 0x7f876c0769c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87900cb400 0x7f87900c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f879015a7a0 0x7f87900c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 --2- 172.21.15.7:0/3890943295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f879006c570 0x7f87900bad90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:16.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 -- 172.21.15.7:0/3890943295 >> 172.21.15.7:0/3890943295 conn(0x7f87900d59c0 msgr2=0x7f87900b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:16.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 -- 172.21.15.7:0/3890943295 shutdown_connections 2024-04-25T11:11:16.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:16.803+0000 7f8795e00640 1 -- 172.21.15.7:0/3890943295 wait complete. 2024-04-25T11:11:16.830 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:11:16.830 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:11:16.830 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:11:16.830 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T11:11:16.831 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T11:11:16.831 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:11:16.831 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:11:16.831 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:11:16.831 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T11:11:16.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:16.831 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:11:16.884 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:11:16.884 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:11:16.884 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:11:16.884 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42498sec preferred_lft 42498sec 2024-04-25T11:11:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6098:c7ff:fe8a:4a30/64 scope link 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::646a:35ff:fe7b:ad0/64 scope link 2024-04-25T11:11:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:16.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:16.888 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:11:16.970 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:11:16.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:16.974 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:11:17.028 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:11:17.028 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:11:17.028 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:11:17.029 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:11:17.029 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:11:17.030 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:17.030 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:11:17.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:11:17.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:11:17.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:11:17.030 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:11:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:17.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:17.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:17.252 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:11:17.300 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:11:17.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:17.301 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-04-25T11:11:17.348 INFO:teuthology.orchestra.run.smithi007.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T11:11:17.348 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:17.348 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:11:17.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:11:17.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T11:11:17.348 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:11:17.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:17.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:17.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T11:11:17.624 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:11:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:11:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:11:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:11:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:17.624 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:11:17.624 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0a:f2:df:d6:33:c6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:11:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:11:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::8f2:dfff:fed6:33c6/64 scope link 2024-04-25T11:11:17.625 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:17.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:17.628 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T11:11:17.712 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:11:17.712 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:11:17.712 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:11:17.712 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:17.712 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:11:17.713 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:17.713 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:11:17.713 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0a:f2:df:d6:33:c6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:11:17.713 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:11:17.713 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:17.713 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::8f2:dfff:fed6:33c6/64 scope link 2024-04-25T11:11:17.713 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:17.716 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T11:11:17.717 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:17.717 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:11:17.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T11:11:17.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T11:11:17.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T11:11:17.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T11:11:17.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T11:11:17.717 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:11:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:18.007 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:18.007 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:11:18.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.1 up 2024-04-25T11:11:18.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T11:11:18.008 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:11:18.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:18.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:18.227 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:11:18.227 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T11:11:18.275 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T11:11:18.275 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:11:18.275 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T11:11:18.371 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-04-25T11:11:18.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:18.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:18.592 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:18.592 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:11:18.637 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:18.637 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:11:18.637 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:11:18.637 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 2@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:11:18.637 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:11:18.639 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T11:11:18.639 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:11:18.639 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:11:18.640 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:11:18.640 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:11:18.644 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:11:18.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:18.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T11:11:18.726 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:18.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:18.853 INFO:teuthology.orchestra.run.smithi007.stdout:client4803 2024-04-25T11:11:18.853 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:11:18.854 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:18.907 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:19.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:19.030 INFO:teuthology.orchestra.run.smithi007.stdout:client4803 2024-04-25T11:11:19.030 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:19.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:19.155 INFO:teuthology.orchestra.run.smithi007.stdout:client4803 2024-04-25T11:11:19.155 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.client4803/status') 2024-04-25T11:11:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.4803 (3)192.168.0.2:0/845345914 2024-04-25T11:11:19.297 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-04-25T11:11:19.297 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-04-25T11:11:19.298 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-04-25T11:11:19.298 INFO:teuthology.orchestra.run.smithi007.stderr:68 bytes copied, 0.000109162 s, 623 kB/s 2024-04-25T11:11:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:19.311 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-04-25T11:11:19.311 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:11:19.311 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:11:19.311 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:11:19.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.598+0000 7fd247400640 1 Processor -- start 2024-04-25T11:11:19.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.599+0000 7fd247400640 1 -- start start 2024-04-25T11:11:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.601+0000 7fd247400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24015f180 0x7fd240161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.601+0000 7fd247400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2401621a0 0x7fd24006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.601+0000 7fd247400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd24006dac0 0x7fd24006f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.601+0000 7fd247400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd24015f590 con 0x7fd24006dac0 2024-04-25T11:11:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.601+0000 7fd247400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd240158ae0 con 0x7fd2401621a0 2024-04-25T11:11:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.601+0000 7fd247400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd24006ded0 con 0x7fd24015f180 2024-04-25T11:11:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.601+0000 7fd245400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd24006dac0 0x7fd24006f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.602+0000 7fd245400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd24006dac0 0x7fd24006f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42922/0 (socket says 172.21.15.7:42922) 2024-04-25T11:11:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.602+0000 7fd245400640 1 -- 172.21.15.7:0/3793056043 learned_addr learned my addr 172.21.15.7:0/3793056043 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.602+0000 7fd244a00640 1 --2- 172.21.15.7:0/3793056043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24015f180 0x7fd240161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:19.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.602+0000 7fd23fe00640 1 --2- 172.21.15.7:0/3793056043 >> v2:172.21.15.87:3300/0 conn(0x7fd2401621a0 0x7fd24006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.602+0000 7fd245400640 1 -- 172.21.15.7:0/3793056043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24015f180 msgr2=0x7fd240161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.602+0000 7fd245400640 1 --2- 172.21.15.7:0/3793056043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24015f180 0x7fd240161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.602+0000 7fd245400640 1 -- 172.21.15.7:0/3793056043 >> v2:172.21.15.87:3300/0 conn(0x7fd2401621a0 msgr2=0x7fd24006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.602+0000 7fd245400640 1 --2- 172.21.15.7:0/3793056043 >> v2:172.21.15.87:3300/0 conn(0x7fd2401621a0 0x7fd24006d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.602+0000 7fd245400640 1 -- 172.21.15.7:0/3793056043 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd240162700 con 0x7fd24006dac0 2024-04-25T11:11:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.603+0000 7fd245400640 1 --2- 172.21.15.7:0/3793056043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd24006dac0 0x7fd24006f1e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd240074ba0 tx=0x7fd23802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac267793728869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:19.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.604+0000 7fd23f400640 1 -- 172.21.15.7:0/3793056043 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238036070 con 0x7fd24006dac0 2024-04-25T11:11:19.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.604+0000 7fd23f400640 1 -- 172.21.15.7:0/3793056043 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd238031600 con 0x7fd24006dac0 2024-04-25T11:11:19.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.604+0000 7fd23f400640 1 -- 172.21.15.7:0/3793056043 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380319d0 con 0x7fd24006dac0 2024-04-25T11:11:19.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.605+0000 7fd247400640 1 -- 172.21.15.7:0/3793056043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd24006dac0 msgr2=0x7fd24006f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:19.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.605+0000 7fd247400640 1 --2- 172.21.15.7:0/3793056043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd24006dac0 0x7fd24006f1e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd240074ba0 tx=0x7fd23802a640 comp rx=0 tx=0).stop 2024-04-25T11:11:19.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.605+0000 7fd247400640 1 -- 172.21.15.7:0/3793056043 shutdown_connections 2024-04-25T11:11:19.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.605+0000 7fd247400640 1 --2- 172.21.15.7:0/3793056043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd24006dac0 0x7fd24006f1e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:19.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.605+0000 7fd247400640 1 --2- 172.21.15.7:0/3793056043 >> v2:172.21.15.87:3300/0 conn(0x7fd2401621a0 0x7fd24006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:19.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.605+0000 7fd247400640 1 --2- 172.21.15.7:0/3793056043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24015f180 0x7fd240161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.605+0000 7fd247400640 1 -- 172.21.15.7:0/3793056043 >> 172.21.15.7:0/3793056043 conn(0x7fd2400732b0 msgr2=0x7fd2400736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.605+0000 7fd247400640 1 -- 172.21.15.7:0/3793056043 shutdown_connections 2024-04-25T11:11:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.605+0000 7fd247400640 1 -- 172.21.15.7:0/3793056043 wait complete. 2024-04-25T11:11:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.608+0000 7fd247400640 1 Processor -- start 2024-04-25T11:11:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.609+0000 7fd247400640 1 -- start start 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.611+0000 7fd247400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24006dac0 0x7fd2401e8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.611+0000 7fd247400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd24015f180 0x7fd2401f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.611+0000 7fd247400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2401621a0 0x7fd24009e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.611+0000 7fd247400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd24015ec60 con 0x7fd24015f180 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.611+0000 7fd247400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd24015e980 con 0x7fd2401621a0 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.611+0000 7fd247400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd24015e670 con 0x7fd24006dac0 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.611+0000 7fd23fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd24015f180 0x7fd2401f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.611+0000 7fd23fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd24015f180 0x7fd2401f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42926/0 (socket says 172.21.15.7:42926) 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.611+0000 7fd23fe00640 1 -- 172.21.15.7:0/350414063 learned_addr learned my addr 172.21.15.7:0/350414063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.611+0000 7fd244a00640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24006dac0 0x7fd2401e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.611+0000 7fd245400640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2401621a0 0x7fd24009e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.612+0000 7fd23fe00640 1 -- 172.21.15.7:0/350414063 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24006dac0 msgr2=0x7fd2401e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.612+0000 7fd23fe00640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24006dac0 0x7fd2401e8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.612+0000 7fd23fe00640 1 -- 172.21.15.7:0/350414063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2401621a0 msgr2=0x7fd24009e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.612+0000 7fd23fe00640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2401621a0 0x7fd24009e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.612+0000 7fd23fe00640 1 -- 172.21.15.7:0/350414063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24009b870 con 0x7fd24015f180 2024-04-25T11:11:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.612+0000 7fd23fe00640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd24015f180 0x7fd2401f75d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd22c00d5d0 tx=0x7fd22c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.612+0000 7fd23cc00640 1 -- 172.21.15.7:0/350414063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c0125f0 con 0x7fd24015f180 2024-04-25T11:11:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.613+0000 7fd23cc00640 1 -- 172.21.15.7:0/350414063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd22c019070 con 0x7fd24015f180 2024-04-25T11:11:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.613+0000 7fd23cc00640 1 -- 172.21.15.7:0/350414063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c0129b0 con 0x7fd24015f180 2024-04-25T11:11:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.613+0000 7fd247400640 1 -- 172.21.15.7:0/350414063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2401f81b0 con 0x7fd24015f180 2024-04-25T11:11:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.613+0000 7fd247400640 1 -- 172.21.15.7:0/350414063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2401e7a70 con 0x7fd24015f180 2024-04-25T11:11:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.614+0000 7fd247400640 1 -- 172.21.15.7:0/350414063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2401f7f00 con 0x7fd24015f180 2024-04-25T11:11:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.615+0000 7fd23cc00640 1 -- 172.21.15.7:0/350414063 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd22c012b50 con 0x7fd24015f180 2024-04-25T11:11:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.616+0000 7fd23cc00640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2200744b0 0x7fd2200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.616+0000 7fd23cc00640 1 -- 172.21.15.7:0/350414063 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fd22c095980 con 0x7fd24015f180 2024-04-25T11:11:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.616+0000 7fd244a00640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2200744b0 0x7fd2200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.617+0000 7fd244a00640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2200744b0 0x7fd2200769c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd240070310 tx=0x7fd234008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:19.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.621+0000 7fd23cc00640 1 -- 172.21.15.7:0/350414063 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd24009b870 con 0x7fd24015f180 2024-04-25T11:11:19.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.866+0000 7fd247400640 1 -- 172.21.15.7:0/350414063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd240050740 con 0x7fd24015f180 2024-04-25T11:11:19.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.867+0000 7fd23cc00640 1 -- 172.21.15.7:0/350414063 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fd22c05f630 con 0x7fd24015f180 2024-04-25T11:11:19.867 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T11:11:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 -- 172.21.15.7:0/350414063 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2200744b0 msgr2=0x7fd2200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2200744b0 0x7fd2200769c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd240070310 tx=0x7fd234008040 comp rx=0 tx=0).stop 2024-04-25T11:11:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 -- 172.21.15.7:0/350414063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd24015f180 msgr2=0x7fd2401f75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd24015f180 0x7fd2401f75d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd22c00d5d0 tx=0x7fd22c00da90 comp rx=0 tx=0).stop 2024-04-25T11:11:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 -- 172.21.15.7:0/350414063 shutdown_connections 2024-04-25T11:11:19.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2200744b0 0x7fd2200769c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2401621a0 0x7fd24009e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd24015f180 0x7fd2401f75d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 --2- 172.21.15.7:0/350414063 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24006dac0 0x7fd2401e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 -- 172.21.15.7:0/350414063 >> 172.21.15.7:0/350414063 conn(0x7fd2400732b0 msgr2=0x7fd2400b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 -- 172.21.15.7:0/350414063 shutdown_connections 2024-04-25T11:11:19.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:19.870+0000 7fd247400640 1 -- 172.21.15.7:0/350414063 wait complete. 2024-04-25T11:11:19.896 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:11:19.897 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:11:19.897 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:11:19.897 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:11:19.897 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:11:19.897 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:11:19.897 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:11:19.897 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:11:19.897 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:11:19.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:19.897 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:11:19.956 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:11:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:11:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:11:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:11:19.956 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:19.957 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:11:19.957 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:19.957 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:11:19.958 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:19.958 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:11:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:11:19.958 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42496sec preferred_lft 42496sec 2024-04-25T11:11:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:11:19.959 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:19.959 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T11:11:19.959 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:19.960 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:11:19.960 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add name ceph-brx type bridge 2024-04-25T11:11:19.960 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr flush dev ceph-brx 2024-04-25T11:11:19.960 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set ceph-brx up 2024-04-25T11:11:19.960 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T11:11:19.960 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:11:20.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:20.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:20.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:20.437 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:11:20.503 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:11:20.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:20.507 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:11:20.560 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:11:20.560 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:11:20.560 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:11:20.561 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:11:20.561 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:11:20.561 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:20.561 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:11:20.561 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:11:20.561 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:11:20.561 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:11:20.561 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:11:20.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:21.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:21.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:21.200 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:11:21.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:21.247 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-04-25T11:11:21.338 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:21.338 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:11:21.338 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:11:21.338 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T11:11:21.338 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:11:21.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:21.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:21.502 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T11:11:21.503 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:21.503 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:11:21.503 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T11:11:21.503 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T11:11:21.503 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T11:11:21.503 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T11:11:21.503 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T11:11:21.503 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:11:21.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:21.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:21.776 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:21.777 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:11:21.777 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T11:11:21.777 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T11:11:21.777 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:11:21.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:21.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:21.963 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:11:21.963 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:11:22.002 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:11:22.002 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:11:22.002 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:11:22.093 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:11:22.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:22.094 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:22.308 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:22.308 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:11:23.006 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:23.007 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:11:23.007 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:11:23.007 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 1@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:11:23.007 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:11:23.009 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:11:23.009 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:11:23.009 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:11:23.009 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:11:23.009 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:11:23.015 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:11:23.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:23.015 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:11:23.089 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:23.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:23.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:23.210 INFO:teuthology.orchestra.run.smithi087.stdout:client4812 2024-04-25T11:11:23.211 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:11:23.211 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:23.262 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:23.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:23.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:23.429 INFO:teuthology.orchestra.run.smithi087.stdout:client4812 2024-04-25T11:11:23.430 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:23.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:23.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:23.547 INFO:teuthology.orchestra.run.smithi087.stdout:client4812 2024-04-25T11:11:23.548 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.client4812/status') 2024-04-25T11:11:23.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:23.677 INFO:teuthology.orchestra.run.smithi087.stdout:instance: client.4812 (3)172.21.15.87:0/3680557416 2024-04-25T11:11:23.677 INFO:teuthology.orchestra.run.smithi087.stdout:blocklisted: false 2024-04-25T11:11:23.677 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records in 2024-04-25T11:11:23.677 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records out 2024-04-25T11:11:23.677 INFO:teuthology.orchestra.run.smithi087.stderr:70 bytes copied, 9.3348e-05 s, 750 kB/s 2024-04-25T11:11:23.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:23.690 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-04-25T11:11:23.690 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T11:11:23.690 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T11:11:23.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:11:23.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.980+0000 7fd233c00640 1 Processor -- start 2024-04-25T11:11:23.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.980+0000 7fd233c00640 1 -- start start 2024-04-25T11:11:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.982+0000 7fd233c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd22c0b4e30 0x7fd22c0bb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.982+0000 7fd233c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd22c0b9470 0x7fd22c0b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.982+0000 7fd233c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd22c0b7650 0x7fd22c0b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.982+0000 7fd233c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd22c0d0b50 con 0x7fd22c0b7650 2024-04-25T11:11:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.982+0000 7fd233c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd22c08e3a0 con 0x7fd22c0b9470 2024-04-25T11:11:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.982+0000 7fd233c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd22c0bbb90 con 0x7fd22c0b4e30 2024-04-25T11:11:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.983+0000 7fd231c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd22c0b7650 0x7fd22c0b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.983+0000 7fd231c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd22c0b7650 0x7fd22c0b7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54250/0 (socket says 172.21.15.7:54250) 2024-04-25T11:11:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.983+0000 7fd231c00640 1 -- 172.21.15.7:0/2195060799 learned_addr learned my addr 172.21.15.7:0/2195060799 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.983+0000 7fd231c00640 1 -- 172.21.15.7:0/2195060799 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd22c0b4e30 msgr2=0x7fd22c0bb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.983+0000 7fd231c00640 1 --2- 172.21.15.7:0/2195060799 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd22c0b4e30 0x7fd22c0bb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.983+0000 7fd231c00640 1 -- 172.21.15.7:0/2195060799 >> v2:172.21.15.87:3300/0 conn(0x7fd22c0b9470 msgr2=0x7fd22c0b9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.983+0000 7fd231c00640 1 --2- 172.21.15.7:0/2195060799 >> v2:172.21.15.87:3300/0 conn(0x7fd22c0b9470 0x7fd22c0b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.983+0000 7fd231c00640 1 -- 172.21.15.7:0/2195060799 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c0b7fa0 con 0x7fd22c0b7650 2024-04-25T11:11:23.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.984+0000 7fd231c00640 1 --2- 172.21.15.7:0/2195060799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd22c0b7650 0x7fd22c0b7a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd22c08e1c0 tx=0x7fd21c02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c5d8cf3323e807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:23.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.985+0000 7fd22b400640 1 -- 172.21.15.7:0/2195060799 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c037070 con 0x7fd22c0b7650 2024-04-25T11:11:23.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.985+0000 7fd22b400640 1 -- 172.21.15.7:0/2195060799 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21c032440 con 0x7fd22c0b7650 2024-04-25T11:11:23.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.985+0000 7fd22b400640 1 -- 172.21.15.7:0/2195060799 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c032810 con 0x7fd22c0b7650 2024-04-25T11:11:23.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.985+0000 7fd233c00640 1 -- 172.21.15.7:0/2195060799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd22c0b7650 msgr2=0x7fd22c0b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:23.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.985+0000 7fd233c00640 1 --2- 172.21.15.7:0/2195060799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd22c0b7650 0x7fd22c0b7a60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd22c08e1c0 tx=0x7fd21c02a160 comp rx=0 tx=0).stop 2024-04-25T11:11:23.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.986+0000 7fd233c00640 1 -- 172.21.15.7:0/2195060799 shutdown_connections 2024-04-25T11:11:23.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.986+0000 7fd233c00640 1 --2- 172.21.15.7:0/2195060799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd22c0b7650 0x7fd22c0b7a60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:23.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.986+0000 7fd233c00640 1 --2- 172.21.15.7:0/2195060799 >> v2:172.21.15.87:3300/0 conn(0x7fd22c0b9470 0x7fd22c0b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:23.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.986+0000 7fd233c00640 1 --2- 172.21.15.7:0/2195060799 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd22c0b4e30 0x7fd22c0bb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:23.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.986+0000 7fd233c00640 1 -- 172.21.15.7:0/2195060799 >> 172.21.15.7:0/2195060799 conn(0x7fd22c0d2650 msgr2=0x7fd22c0d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:23.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.986+0000 7fd233c00640 1 -- 172.21.15.7:0/2195060799 shutdown_connections 2024-04-25T11:11:23.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.986+0000 7fd233c00640 1 -- 172.21.15.7:0/2195060799 wait complete. 2024-04-25T11:11:23.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.988+0000 7fd233c00640 1 Processor -- start 2024-04-25T11:11:23.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.988+0000 7fd233c00640 1 -- start start 2024-04-25T11:11:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd233c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd22c0b4e30 0x7fd22c156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd233c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd22c0b7650 0x7fd22c0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd233c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c0b9470 0x7fd22c162fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd233c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c0d0500 con 0x7fd22c0b4e30 2024-04-25T11:11:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd233c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c0d0220 con 0x7fd22c0b9470 2024-04-25T11:11:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd233c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd22c0d07e0 con 0x7fd22c0b7650 2024-04-25T11:11:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd231200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd22c0b4e30 0x7fd22c156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd231200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd22c0b4e30 0x7fd22c156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54260/0 (socket says 172.21.15.7:54260) 2024-04-25T11:11:23.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd231200640 1 -- 172.21.15.7:0/4293391274 learned_addr learned my addr 172.21.15.7:0/4293391274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd231200640 1 -- 172.21.15.7:0/4293391274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd22c0b7650 msgr2=0x7fd22c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd231200640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd22c0b7650 0x7fd22c0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd231200640 1 -- 172.21.15.7:0/4293391274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c0b9470 msgr2=0x7fd22c162fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.991+0000 7fd231200640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c0b9470 0x7fd22c162fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.992+0000 7fd231200640 1 -- 172.21.15.7:0/4293391274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c1634f0 con 0x7fd22c0b4e30 2024-04-25T11:11:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.992+0000 7fd231200640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd22c0b4e30 0x7fd22c156770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd22400b410 tx=0x7fd22400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.992+0000 7fd228c00640 1 -- 172.21.15.7:0/4293391274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224015020 con 0x7fd22c0b4e30 2024-04-25T11:11:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.992+0000 7fd228c00640 1 -- 172.21.15.7:0/4293391274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd224011830 con 0x7fd22c0b4e30 2024-04-25T11:11:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.992+0000 7fd228c00640 1 -- 172.21.15.7:0/4293391274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224011c20 con 0x7fd22c0b4e30 2024-04-25T11:11:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.992+0000 7fd233c00640 1 -- 172.21.15.7:0/4293391274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22c162170 con 0x7fd22c0b4e30 2024-04-25T11:11:23.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.992+0000 7fd233c00640 1 -- 172.21.15.7:0/4293391274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22c162730 con 0x7fd22c0b4e30 2024-04-25T11:11:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.994+0000 7fd233c00640 1 -- 172.21.15.7:0/4293391274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd22c050740 con 0x7fd22c0b4e30 2024-04-25T11:11:23.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.995+0000 7fd228c00640 1 -- 172.21.15.7:0/4293391274 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd224018020 con 0x7fd22c0b4e30 2024-04-25T11:11:23.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.996+0000 7fd228c00640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2080744e0 0x7fd2080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:23.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.996+0000 7fd228c00640 1 -- 172.21.15.7:0/4293391274 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fd2240954f0 con 0x7fd22c0b4e30 2024-04-25T11:11:23.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.996+0000 7fd22be00640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2080744e0 0x7fd2080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:23.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:23.997+0000 7fd22be00640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2080744e0 0x7fd2080769f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd22c1636e0 tx=0x7fd214009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.001+0000 7fd228c00640 1 -- 172.21.15.7:0/4293391274 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd22401f020 con 0x7fd22c0b4e30 2024-04-25T11:11:24.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.245+0000 7fd233c00640 1 -- 172.21.15.7:0/4293391274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd22c0b8ab0 con 0x7fd22c0b4e30 2024-04-25T11:11:24.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.246+0000 7fd228c00640 1 -- 172.21.15.7:0/4293391274 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fd22405f870 con 0x7fd22c0b4e30 2024-04-25T11:11:24.246 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T11:11:24.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.248+0000 7fd233c00640 1 -- 172.21.15.7:0/4293391274 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2080744e0 msgr2=0x7fd2080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:24.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.248+0000 7fd233c00640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2080744e0 0x7fd2080769f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd22c1636e0 tx=0x7fd214009250 comp rx=0 tx=0).stop 2024-04-25T11:11:24.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.248+0000 7fd233c00640 1 -- 172.21.15.7:0/4293391274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd22c0b4e30 msgr2=0x7fd22c156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:24.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.248+0000 7fd233c00640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd22c0b4e30 0x7fd22c156770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd22400b410 tx=0x7fd22400b8d0 comp rx=0 tx=0).stop 2024-04-25T11:11:24.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.248+0000 7fd233c00640 1 -- 172.21.15.7:0/4293391274 shutdown_connections 2024-04-25T11:11:24.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.248+0000 7fd233c00640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2080744e0 0x7fd2080769f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:24.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.248+0000 7fd233c00640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c0b9470 0x7fd22c162fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:24.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.248+0000 7fd233c00640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd22c0b7650 0x7fd22c0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:24.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.248+0000 7fd233c00640 1 --2- 172.21.15.7:0/4293391274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd22c0b4e30 0x7fd22c156770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:24.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.248+0000 7fd233c00640 1 -- 172.21.15.7:0/4293391274 >> 172.21.15.7:0/4293391274 conn(0x7fd22c0d2650 msgr2=0x7fd22c1557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:24.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.249+0000 7fd233c00640 1 -- 172.21.15.7:0/4293391274 shutdown_connections 2024-04-25T11:11:24.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:24.249+0000 7fd233c00640 1 -- 172.21.15.7:0/4293391274 wait complete. 2024-04-25T11:11:24.276 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T11:11:24.276 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:11:24.276 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:11:24.276 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T11:11:24.276 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T11:11:24.276 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T11:11:24.276 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T11:11:24.276 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:11:24.276 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T11:11:24.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:24.276 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:11:24.334 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:11:24.335 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:11:24.335 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:11:24.335 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:24.335 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:11:24.335 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:24.335 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:24.335 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:24.335 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:11:24.335 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:24.335 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42491sec preferred_lft 42491sec 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::58f2:6bff:fe8b:8b4c/64 scope link 2024-04-25T11:11:24.337 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:24.337 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:11:24.337 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:11:24.337 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::98fe:1bff:fe9f:d71c/64 scope link 2024-04-25T11:11:24.337 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:24.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:24.337 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:11:24.415 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:11:24.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:24.419 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:11:24.468 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:11:24.468 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:11:24.468 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:11:24.468 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:11:24.468 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:11:24.469 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:24.469 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:11:24.469 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:11:24.469 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:11:24.469 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:11:24.469 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:11:24.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:24.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:24.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:24.671 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:11:24.715 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:11:24.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:24.715 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-04-25T11:11:24.811 INFO:teuthology.orchestra.run.smithi087.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T11:11:24.812 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:24.812 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:11:24.812 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:11:24.812 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T11:11:24.812 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:11:24.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:24.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:25.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:25.000 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T11:11:25.075 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:11:25.075 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:11:25.075 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:11:25.075 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:25.075 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:11:25.075 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:25.075 INFO:teuthology.orchestra.run.smithi087.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:11:25.075 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3a:2d:53:0a:ec:e9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:11:25.075 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:11:25.075 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:25.075 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::382d:53ff:fe0a:ece9/64 scope link 2024-04-25T11:11:25.076 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:25.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:25.080 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T11:11:25.162 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:11:25.163 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:11:25.163 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:11:25.163 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:25.163 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:11:25.163 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:25.163 INFO:teuthology.orchestra.run.smithi087.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:11:25.163 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3a:2d:53:0a:ec:e9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T11:11:25.163 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T11:11:25.163 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:25.163 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::382d:53ff:fe0a:ece9/64 scope link 2024-04-25T11:11:25.163 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:11:25.167 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T11:11:25.167 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:25.167 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:11:25.167 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T11:11:25.167 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T11:11:25.167 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T11:11:25.167 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T11:11:25.167 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T11:11:25.167 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:11:25.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:25.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:25.437 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:11:25.437 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:11:25.437 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.1 up 2024-04-25T11:11:25.437 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T11:11:25.437 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:11:25.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:25.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:25.638 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:11:25.638 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T11:11:25.684 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T11:11:25.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:11:25.684 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T11:11:25.775 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-04-25T11:11:25.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:25.776 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:25.980 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:25.981 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:11:26.026 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:11:26.026 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:11:26.026 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:11:26.026 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 3@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:11:26.026 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:11:26.028 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T11:11:26.028 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:11:26.028 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:11:26.028 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:11:26.028 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:11:26.032 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:11:26.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:26.032 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T11:11:26.107 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:26.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:26.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:26.227 INFO:teuthology.orchestra.run.smithi087.stdout:client4821 2024-04-25T11:11:26.227 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:11:26.227 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:26.278 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:26.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:26.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:26.439 INFO:teuthology.orchestra.run.smithi087.stdout:client4821 2024-04-25T11:11:26.439 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:11:26.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:11:26.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:26.557 INFO:teuthology.orchestra.run.smithi087.stdout:client4821 2024-04-25T11:11:26.557 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.client4821/status') 2024-04-25T11:11:26.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:11:26.685 INFO:teuthology.orchestra.run.smithi087.stdout:instance: client.4821 (3)172.21.15.87:0/3716356736 2024-04-25T11:11:26.685 INFO:teuthology.orchestra.run.smithi087.stdout:blocklisted: false 2024-04-25T11:11:26.685 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records in 2024-04-25T11:11:26.685 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records out 2024-04-25T11:11:26.685 INFO:teuthology.orchestra.run.smithi087.stderr:70 bytes copied, 8.1108e-05 s, 863 kB/s 2024-04-25T11:11:26.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:11:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:11:26.697 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T11:11:26.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.010+0000 7fe2dd400640 1 Processor -- start 2024-04-25T11:11:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.010+0000 7fe2dd400640 1 -- start start 2024-04-25T11:11:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.012+0000 7fe2dd400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe2d80af540 0x7fe2d809f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.013+0000 7fe2dd400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe2d815bc40 0x7fe2d80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.013+0000 7fe2dd400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2d815f4e0 con 0x7fe2d815ef90 2024-04-25T11:11:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.013+0000 7fe2dd400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d8071190 con 0x7fe2d80af540 2024-04-25T11:11:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.013+0000 7fe2dd400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d80b0560 con 0x7fe2d815bc40 2024-04-25T11:11:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.013+0000 7fe2d5e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe2d80af540 0x7fe2d809f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:27.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.013+0000 7fe2d6800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe2d815ef90 0x7fe2d80aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45226/0 (socket says 172.21.15.7:45226) 2024-04-25T11:11:27.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.013+0000 7fe2d6800640 1 -- 172.21.15.7:0/3833702381 learned_addr learned my addr 172.21.15.7:0/3833702381 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:27.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.013+0000 7fe2d5e00640 1 --2- 172.21.15.7:0/3833702381 >> v2:172.21.15.7:3300/0 conn(0x7fe2d80af540 0x7fe2d809f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54272/0 (socket says 172.21.15.7:54272) 2024-04-25T11:11:27.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.013+0000 7fe2d7200640 1 --2- 172.21.15.7:0/3833702381 >> v2:172.21.15.87:3300/0 conn(0x7fe2d815bc40 0x7fe2d80cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:27.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.014+0000 7fe2d5e00640 1 -- 172.21.15.7:0/3833702381 >> v2:172.21.15.87:3300/0 conn(0x7fe2d815bc40 msgr2=0x7fe2d80cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:27.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.014+0000 7fe2d5e00640 1 --2- 172.21.15.7:0/3833702381 >> v2:172.21.15.87:3300/0 conn(0x7fe2d815bc40 0x7fe2d80cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.014+0000 7fe2d5e00640 1 -- 172.21.15.7:0/3833702381 >> v1:172.21.15.7:6789/0 conn(0x7fe2d815ef90 legacy=0x7fe2d80aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.014+0000 7fe2d5e00640 1 -- 172.21.15.7:0/3833702381 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d80b0840 con 0x7fe2d80af540 2024-04-25T11:11:27.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.014+0000 7fe2d5400640 1 -- 172.21.15.7:0/3833702381 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4148713347 0 0) 0x7fe2d815f4e0 con 0x7fe2d815ef90 2024-04-25T11:11:27.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.015+0000 7fe2d5e00640 1 --2- 172.21.15.7:0/3833702381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d80af540 0x7fe2d809f790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80cf210 tx=0x7fe2c002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f054e554ea01f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:27.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.015+0000 7fe2d5400640 1 -- 172.21.15.7:0/3833702381 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0037070 con 0x7fe2d80af540 2024-04-25T11:11:27.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.015+0000 7fe2d5400640 1 -- 172.21.15.7:0/3833702381 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c0032440 con 0x7fe2d80af540 2024-04-25T11:11:27.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.015+0000 7fe2d5400640 1 -- 172.21.15.7:0/3833702381 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c002ae60 con 0x7fe2d80af540 2024-04-25T11:11:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.016+0000 7fe2dd400640 1 -- 172.21.15.7:0/3833702381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d80af540 msgr2=0x7fe2d809f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.016+0000 7fe2dd400640 1 --2- 172.21.15.7:0/3833702381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d80af540 0x7fe2d809f790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe2d80cf210 tx=0x7fe2c002a110 comp rx=0 tx=0).stop 2024-04-25T11:11:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.016+0000 7fe2dd400640 1 -- 172.21.15.7:0/3833702381 shutdown_connections 2024-04-25T11:11:27.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.016+0000 7fe2dd400640 1 --2- 172.21.15.7:0/3833702381 >> v2:172.21.15.87:3300/0 conn(0x7fe2d815bc40 0x7fe2d80cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.016+0000 7fe2dd400640 1 --2- 172.21.15.7:0/3833702381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d80af540 0x7fe2d809f790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.016+0000 7fe2dd400640 1 -- 172.21.15.7:0/3833702381 >> 172.21.15.7:0/3833702381 conn(0x7fe2d8074200 msgr2=0x7fe2d806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.016+0000 7fe2dd400640 1 -- 172.21.15.7:0/3833702381 shutdown_connections 2024-04-25T11:11:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.016+0000 7fe2dd400640 1 -- 172.21.15.7:0/3833702381 wait complete. 2024-04-25T11:11:27.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.019+0000 7fe2dd400640 1 Processor -- start 2024-04-25T11:11:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.019+0000 7fe2dd400640 1 -- start start 2024-04-25T11:11:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.020+0000 7fe2dd400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2d815bc40 0x7fe2d81f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.020+0000 7fe2dd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d815ef90 0x7fe2d81521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.020+0000 7fe2dd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2d8160530 0x7fe2d8160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.020+0000 7fe2dd400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d80b1320 con 0x7fe2d815ef90 2024-04-25T11:11:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.020+0000 7fe2dd400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d80b0e00 con 0x7fe2d8160530 2024-04-25T11:11:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.020+0000 7fe2dd400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d80afb80 con 0x7fe2d815bc40 2024-04-25T11:11:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.021+0000 7fe2d5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d815ef90 0x7fe2d81521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.021+0000 7fe2d5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d815ef90 0x7fe2d81521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54282/0 (socket says 172.21.15.7:54282) 2024-04-25T11:11:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.021+0000 7fe2d5e00640 1 -- 172.21.15.7:0/990615311 learned_addr learned my addr 172.21.15.7:0/990615311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.021+0000 7fe2d5e00640 1 -- 172.21.15.7:0/990615311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2d815bc40 msgr2=0x7fe2d81f9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.021+0000 7fe2d5e00640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2d815bc40 0x7fe2d81f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.021+0000 7fe2d5e00640 1 -- 172.21.15.7:0/990615311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2d8160530 msgr2=0x7fe2d8160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.021+0000 7fe2d7200640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2d8160530 0x7fe2d8160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.021+0000 7fe2d5e00640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2d8160530 0x7fe2d8160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.021+0000 7fe2d5e00640 1 -- 172.21.15.7:0/990615311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d81fa480 con 0x7fe2d815ef90 2024-04-25T11:11:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.022+0000 7fe2d5e00640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d815ef90 0x7fe2d81521b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009890 tx=0x7fe2c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.022+0000 7fe2cea00640 1 -- 172.21.15.7:0/990615311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0037040 con 0x7fe2d815ef90 2024-04-25T11:11:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.022+0000 7fe2cea00640 1 -- 172.21.15.7:0/990615311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c0042070 con 0x7fe2d815ef90 2024-04-25T11:11:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.022+0000 7fe2dd400640 1 -- 172.21.15.7:0/990615311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d8160e80 con 0x7fe2d815ef90 2024-04-25T11:11:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.022+0000 7fe2cea00640 1 -- 172.21.15.7:0/990615311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0030bd0 con 0x7fe2d815ef90 2024-04-25T11:11:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.022+0000 7fe2dd400640 1 -- 172.21.15.7:0/990615311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d80cc8c0 con 0x7fe2d815ef90 2024-04-25T11:11:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.023+0000 7fe2dd400640 1 -- 172.21.15.7:0/990615311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d806a100 con 0x7fe2d815ef90 2024-04-25T11:11:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.025+0000 7fe2cea00640 1 -- 172.21.15.7:0/990615311 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe2c0047460 con 0x7fe2d815ef90 2024-04-25T11:11:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.026+0000 7fe2cea00640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2b40744b0 0x7fe2b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.026+0000 7fe2cea00640 1 -- 172.21.15.7:0/990615311 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fe2c00b6110 con 0x7fe2d815ef90 2024-04-25T11:11:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.026+0000 7fe2d6800640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2b40744b0 0x7fe2b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.027+0000 7fe2d6800640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2b40744b0 0x7fe2b40769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8007fd0 tx=0x7fe2c8007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.031+0000 7fe2cea00640 1 -- 172.21.15.7:0/990615311 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe2c004c020 con 0x7fe2d815ef90 2024-04-25T11:11:27.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.245+0000 7fe2dd400640 1 -- 172.21.15.7:0/990615311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2d815c280 con 0x7fe2d815ef90 2024-04-25T11:11:27.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.249+0000 7fe2cea00640 1 -- 172.21.15.7:0/990615311 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7fe2d8160e80 con 0x7fe2d815ef90 2024-04-25T11:11:27.249 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:27.249 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 -- 172.21.15.7:0/990615311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2b40744b0 msgr2=0x7fe2b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2b40744b0 0x7fe2b40769c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8007fd0 tx=0x7fe2c8007ce0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 -- 172.21.15.7:0/990615311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d815ef90 msgr2=0x7fe2d81521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d815ef90 0x7fe2d81521b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009890 tx=0x7fe2c00046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 -- 172.21.15.7:0/990615311 shutdown_connections 2024-04-25T11:11:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2b40744b0 0x7fe2b40769c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2d8160530 0x7fe2d8160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2d815ef90 0x7fe2d81521b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 --2- 172.21.15.7:0/990615311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2d815bc40 0x7fe2d81f9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 -- 172.21.15.7:0/990615311 >> 172.21.15.7:0/990615311 conn(0x7fe2d8074200 msgr2=0x7fe2d806d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 -- 172.21.15.7:0/990615311 shutdown_connections 2024-04-25T11:11:27.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.252+0000 7fe2dd400640 1 -- 172.21.15.7:0/990615311 wait complete. 2024-04-25T11:11:27.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:27.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:27.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.575+0000 7fd2e0a00640 1 Processor -- start 2024-04-25T11:11:27.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.575+0000 7fd2e0a00640 1 -- start start 2024-04-25T11:11:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.578+0000 7fd2e0a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2dc0ce7b0 0x7fd2dc0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.578+0000 7fd2e0a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2dc0b73b0 0x7fd2dc0b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.578+0000 7fd2e0a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2dc0b7d00 con 0x7fd2dc160a20 2024-04-25T11:11:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.578+0000 7fd2e0a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd2dc0d3230 con 0x7fd2dc0b73b0 2024-04-25T11:11:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.578+0000 7fd2e0a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd2dc06e720 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.578+0000 7fd2d2a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2dc0b73b0 0x7fd2dc0b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.578+0000 7fd2d2a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2dc0b73b0 0x7fd2dc0b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54290/0 (socket says 172.21.15.7:54290) 2024-04-25T11:11:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.578+0000 7fd2d2a00640 1 -- 172.21.15.7:0/2253812763 learned_addr learned my addr 172.21.15.7:0/2253812763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.579+0000 7fd2db400640 1 --2- 172.21.15.7:0/2253812763 >> v2:172.21.15.87:3300/0 conn(0x7fd2dc0ce7b0 0x7fd2dc0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.579+0000 7fd2d2a00640 1 -- 172.21.15.7:0/2253812763 >> v2:172.21.15.87:3300/0 conn(0x7fd2dc0ce7b0 msgr2=0x7fd2dc0b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.579+0000 7fd2d2a00640 1 --2- 172.21.15.7:0/2253812763 >> v2:172.21.15.87:3300/0 conn(0x7fd2dc0ce7b0 0x7fd2dc0b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.579+0000 7fd2d2a00640 1 -- 172.21.15.7:0/2253812763 >> v1:172.21.15.7:6789/0 conn(0x7fd2dc160a20 legacy=0x7fd2dc160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.579+0000 7fd2d2a00640 1 -- 172.21.15.7:0/2253812763 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2dc0d2620 con 0x7fd2dc0b73b0 2024-04-25T11:11:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.580+0000 7fd2d2a00640 1 --2- 172.21.15.7:0/2253812763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2dc0b73b0 0x7fd2dc0b77c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc074a80 tx=0x7fd2c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eeccda2ddae9df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.580+0000 7fd2daa00640 1 -- 172.21.15.7:0/2253812763 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4037070 con 0x7fd2dc0b73b0 2024-04-25T11:11:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.580+0000 7fd2daa00640 1 -- 172.21.15.7:0/2253812763 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c4032440 con 0x7fd2dc0b73b0 2024-04-25T11:11:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.580+0000 7fd2daa00640 1 -- 172.21.15.7:0/2253812763 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4032810 con 0x7fd2dc0b73b0 2024-04-25T11:11:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.581+0000 7fd2e0a00640 1 -- 172.21.15.7:0/2253812763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2dc0b73b0 msgr2=0x7fd2dc0b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.581+0000 7fd2e0a00640 1 --2- 172.21.15.7:0/2253812763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2dc0b73b0 0x7fd2dc0b77c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc074a80 tx=0x7fd2c402a270 comp rx=0 tx=0).stop 2024-04-25T11:11:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.581+0000 7fd2e0a00640 1 -- 172.21.15.7:0/2253812763 shutdown_connections 2024-04-25T11:11:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.581+0000 7fd2e0a00640 1 --2- 172.21.15.7:0/2253812763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2dc0b73b0 0x7fd2dc0b77c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.581+0000 7fd2e0a00640 1 --2- 172.21.15.7:0/2253812763 >> v2:172.21.15.87:3300/0 conn(0x7fd2dc0ce7b0 0x7fd2dc0b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.581+0000 7fd2e0a00640 1 -- 172.21.15.7:0/2253812763 >> 172.21.15.7:0/2253812763 conn(0x7fd2dc09b540 msgr2=0x7fd2dc09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.581+0000 7fd2e0a00640 1 -- 172.21.15.7:0/2253812763 shutdown_connections 2024-04-25T11:11:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.581+0000 7fd2e0a00640 1 -- 172.21.15.7:0/2253812763 wait complete. 2024-04-25T11:11:27.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.584+0000 7fd2e0a00640 1 Processor -- start 2024-04-25T11:11:27.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.584+0000 7fd2e0a00640 1 -- start start 2024-04-25T11:11:27.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.585+0000 7fd2e0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2dc0b73b0 0x7fd2dc1f9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.585+0000 7fd2e0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2dc0ce7b0 0x7fd2dc0c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.585+0000 7fd2e0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2dc160a20 0x7fd2dc15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.585+0000 7fd2e0a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc0d2ec0 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.585+0000 7fd2e0a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd2dc0d2be0 con 0x7fd2dc0b73b0 2024-04-25T11:11:27.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.585+0000 7fd2e0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd2dc0ccca0 con 0x7fd2dc160a20 2024-04-25T11:11:27.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.586+0000 7fd2d2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2dc0ce7b0 0x7fd2dc0c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:27.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.586+0000 7fd2d2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2dc0ce7b0 0x7fd2dc0c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54302/0 (socket says 172.21.15.7:54302) 2024-04-25T11:11:27.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.586+0000 7fd2d2a00640 1 -- 172.21.15.7:0/1132493691 learned_addr learned my addr 172.21.15.7:0/1132493691 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:27.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.586+0000 7fd2d2a00640 1 -- 172.21.15.7:0/1132493691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2dc160a20 msgr2=0x7fd2dc15dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.586+0000 7fd2d2a00640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2dc160a20 0x7fd2dc15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.586+0000 7fd2d2a00640 1 -- 172.21.15.7:0/1132493691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2dc0b73b0 msgr2=0x7fd2dc1f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.586+0000 7fd2d2a00640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2dc0b73b0 0x7fd2dc1f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.586+0000 7fd2d2a00640 1 -- 172.21.15.7:0/1132493691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2dc0c8040 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.586+0000 7fd2d2a00640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2dc0ce7b0 0x7fd2dc0c8ac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd2c402a2b0 tx=0x7fd2c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.587+0000 7fd2d3e00640 1 -- 172.21.15.7:0/1132493691 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4041020 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.587+0000 7fd2d3e00640 1 -- 172.21.15.7:0/1132493691 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c40306c0 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.587+0000 7fd2e0a00640 1 -- 172.21.15.7:0/1132493691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2dc1fa0f0 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.587+0000 7fd2d3e00640 1 -- 172.21.15.7:0/1132493691 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4037040 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.587+0000 7fd2e0a00640 1 -- 172.21.15.7:0/1132493691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2dc0c82c0 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.588+0000 7fd2e0a00640 1 -- 172.21.15.7:0/1132493691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2dc050740 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.590+0000 7fd2d3e00640 1 -- 172.21.15.7:0/1132493691 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd2c4044020 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.591+0000 7fd2d3e00640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2b80744e0 0x7fd2b80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.591+0000 7fd2d3e00640 1 -- 172.21.15.7:0/1132493691 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fd2c40b5f60 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.591+0000 7fd2db400640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2b80744e0 0x7fd2b80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:27.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.592+0000 7fd2db400640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2b80744e0 0x7fd2b80769f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0070b0 tx=0x7fd2cc007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.595+0000 7fd2d3e00640 1 -- 172.21.15.7:0/1132493691 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd2c404b030 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.799+0000 7fd2e0a00640 1 -- 172.21.15.7:0/1132493691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd2dc1f9e80 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.800+0000 7fd2d3e00640 1 -- 172.21.15.7:0/1132493691 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fd2dc0c8040 con 0x7fd2dc0ce7b0 2024-04-25T11:11:27.800 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:27.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 -- 172.21.15.7:0/1132493691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2b80744e0 msgr2=0x7fd2b80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2b80744e0 0x7fd2b80769f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0070b0 tx=0x7fd2cc007040 comp rx=0 tx=0).stop 2024-04-25T11:11:27.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 -- 172.21.15.7:0/1132493691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2dc0ce7b0 msgr2=0x7fd2dc0c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:27.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2dc0ce7b0 0x7fd2dc0c8ac0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd2c402a2b0 tx=0x7fd2c40046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 -- 172.21.15.7:0/1132493691 shutdown_connections 2024-04-25T11:11:27.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd2b80744e0 0x7fd2b80769f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2dc160a20 0x7fd2dc15dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2dc0ce7b0 0x7fd2dc0c8ac0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 --2- 172.21.15.7:0/1132493691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2dc0b73b0 0x7fd2dc1f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:27.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 -- 172.21.15.7:0/1132493691 >> 172.21.15.7:0/1132493691 conn(0x7fd2dc09b540 msgr2=0x7fd2dc071500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:27.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 -- 172.21.15.7:0/1132493691 shutdown_connections 2024-04-25T11:11:27.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:27.803+0000 7fd2e0a00640 1 -- 172.21.15.7:0/1132493691 wait complete. 2024-04-25T11:11:27.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:28.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.126+0000 7ff173000640 1 Processor -- start 2024-04-25T11:11:28.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.127+0000 7ff173000640 1 -- start start 2024-04-25T11:11:28.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.129+0000 7ff173000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff16c145e40 0x7ff16c146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.129+0000 7ff173000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16c144a80 0x7ff16c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.129+0000 7ff173000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff16c0b66b0 con 0x7ff16c0b4870 2024-04-25T11:11:28.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.129+0000 7ff173000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff16c145b70 con 0x7ff16c145e40 2024-04-25T11:11:28.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.129+0000 7ff173000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff16c0b58d0 con 0x7ff16c144a80 2024-04-25T11:11:28.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.129+0000 7ff171c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff16c145e40 0x7ff16c146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.129+0000 7ff171c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff16c145e40 0x7ff16c146250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54318/0 (socket says 172.21.15.7:54318) 2024-04-25T11:11:28.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.129+0000 7ff171c00640 1 -- 172.21.15.7:0/1983098089 learned_addr learned my addr 172.21.15.7:0/1983098089 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:28.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.130+0000 7ff172600640 1 --2- 172.21.15.7:0/1983098089 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16c144a80 0x7ff16c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.130+0000 7ff171c00640 1 -- 172.21.15.7:0/1983098089 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16c144a80 msgr2=0x7ff16c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:28.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.130+0000 7ff171c00640 1 --2- 172.21.15.7:0/1983098089 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16c144a80 0x7ff16c0b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.130+0000 7ff171c00640 1 -- 172.21.15.7:0/1983098089 >> v1:172.21.15.87:6789/0 conn(0x7ff16c0b4870 legacy=0x7ff16c0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:28.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.130+0000 7ff171c00640 1 -- 172.21.15.7:0/1983098089 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16c0b5b30 con 0x7ff16c145e40 2024-04-25T11:11:28.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.130+0000 7ff172600640 1 --2- 172.21.15.7:0/1983098089 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16c144a80 0x7ff16c0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:28.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.131+0000 7ff171c00640 1 --2- 172.21.15.7:0/1983098089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c145e40 0x7ff16c146250 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff16c06ded0 tx=0x7ff16002c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9ed872bab036dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:28.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.131+0000 7ff16be00640 1 -- 172.21.15.7:0/1983098089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160031710 con 0x7ff16c145e40 2024-04-25T11:11:28.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.131+0000 7ff16be00640 1 -- 172.21.15.7:0/1983098089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1600318b0 con 0x7ff16c145e40 2024-04-25T11:11:28.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.131+0000 7ff16be00640 1 -- 172.21.15.7:0/1983098089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160031ca0 con 0x7ff16c145e40 2024-04-25T11:11:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.132+0000 7ff173000640 1 -- 172.21.15.7:0/1983098089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c145e40 msgr2=0x7ff16c146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.132+0000 7ff173000640 1 --2- 172.21.15.7:0/1983098089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c145e40 0x7ff16c146250 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff16c06ded0 tx=0x7ff16002c800 comp rx=0 tx=0).stop 2024-04-25T11:11:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.132+0000 7ff173000640 1 -- 172.21.15.7:0/1983098089 shutdown_connections 2024-04-25T11:11:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.132+0000 7ff173000640 1 --2- 172.21.15.7:0/1983098089 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16c144a80 0x7ff16c0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.132+0000 7ff173000640 1 --2- 172.21.15.7:0/1983098089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c145e40 0x7ff16c146250 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.132+0000 7ff173000640 1 -- 172.21.15.7:0/1983098089 >> 172.21.15.7:0/1983098089 conn(0x7ff16c0740f0 msgr2=0x7ff16c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.132+0000 7ff173000640 1 -- 172.21.15.7:0/1983098089 shutdown_connections 2024-04-25T11:11:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.132+0000 7ff173000640 1 -- 172.21.15.7:0/1983098089 wait complete. 2024-04-25T11:11:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.135+0000 7ff173000640 1 Processor -- start 2024-04-25T11:11:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.135+0000 7ff173000640 1 -- start start 2024-04-25T11:11:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff173000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16c0b4870 0x7ff16c1e4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff173000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c144a80 0x7ff16c134610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff173000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c145e40 0x7ff16c136860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff173000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c0b63d0 con 0x7ff16c144a80 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff173000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff16c0b60f0 con 0x7ff16c145e40 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff173000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff16c06f730 con 0x7ff16c0b4870 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff171200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c144a80 0x7ff16c134610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff171200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c144a80 0x7ff16c134610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54328/0 (socket says 172.21.15.7:54328) 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff171200640 1 -- 172.21.15.7:0/2515865441 learned_addr learned my addr 172.21.15.7:0/2515865441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff171c00640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16c0b4870 0x7ff16c1e4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff172600640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c145e40 0x7ff16c136860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff171200640 1 -- 172.21.15.7:0/2515865441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16c0b4870 msgr2=0x7ff16c1e4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff171200640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16c0b4870 0x7ff16c1e4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff171200640 1 -- 172.21.15.7:0/2515865441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c145e40 msgr2=0x7ff16c136860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.137+0000 7ff171200640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c145e40 0x7ff16c136860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.138+0000 7ff171200640 1 -- 172.21.15.7:0/2515865441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16c1e3770 con 0x7ff16c144a80 2024-04-25T11:11:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.138+0000 7ff171200640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c144a80 0x7ff16c134610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff15800b3f0 tx=0x7ff15800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.138+0000 7ff169600640 1 -- 172.21.15.7:0/2515865441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158015020 con 0x7ff16c144a80 2024-04-25T11:11:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.138+0000 7ff169600640 1 -- 172.21.15.7:0/2515865441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff158011830 con 0x7ff16c144a80 2024-04-25T11:11:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.138+0000 7ff169600640 1 -- 172.21.15.7:0/2515865441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158011c20 con 0x7ff16c144a80 2024-04-25T11:11:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.138+0000 7ff173000640 1 -- 172.21.15.7:0/2515865441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16c1e3a50 con 0x7ff16c144a80 2024-04-25T11:11:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.138+0000 7ff173000640 1 -- 172.21.15.7:0/2515865441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16c1e4010 con 0x7ff16c144a80 2024-04-25T11:11:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.140+0000 7ff173000640 1 -- 172.21.15.7:0/2515865441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff16c1e34e0 con 0x7ff16c144a80 2024-04-25T11:11:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.141+0000 7ff169600640 1 -- 172.21.15.7:0/2515865441 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff158018020 con 0x7ff16c144a80 2024-04-25T11:11:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.142+0000 7ff169600640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff14c0744b0 0x7ff14c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.142+0000 7ff169600640 1 -- 172.21.15.7:0/2515865441 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff158095320 con 0x7ff16c144a80 2024-04-25T11:11:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.142+0000 7ff171c00640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff14c0744b0 0x7ff14c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.143+0000 7ff171c00640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff14c0744b0 0x7ff14c0769c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff160036000 tx=0x7ff160031000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:28.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.147+0000 7ff169600640 1 -- 172.21.15.7:0/2515865441 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff15801f030 con 0x7ff16c144a80 2024-04-25T11:11:28.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.354+0000 7ff173000640 1 -- 172.21.15.7:0/2515865441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff16c06eef0 con 0x7ff16c144a80 2024-04-25T11:11:28.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.355+0000 7ff169600640 1 -- 172.21.15.7:0/2515865441 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7ff16c06eef0 con 0x7ff16c144a80 2024-04-25T11:11:28.355 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:28.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.357+0000 7ff173000640 1 -- 172.21.15.7:0/2515865441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff14c0744b0 msgr2=0x7ff14c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:28.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.357+0000 7ff173000640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff14c0744b0 0x7ff14c0769c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff160036000 tx=0x7ff160031000 comp rx=0 tx=0).stop 2024-04-25T11:11:28.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.357+0000 7ff173000640 1 -- 172.21.15.7:0/2515865441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c144a80 msgr2=0x7ff16c134610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:28.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.357+0000 7ff173000640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c144a80 0x7ff16c134610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff15800b3f0 tx=0x7ff15800b8b0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.358+0000 7ff173000640 1 -- 172.21.15.7:0/2515865441 shutdown_connections 2024-04-25T11:11:28.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.358+0000 7ff173000640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff14c0744b0 0x7ff14c0769c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.358+0000 7ff173000640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff16c145e40 0x7ff16c136860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.358+0000 7ff173000640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff16c144a80 0x7ff16c134610 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.358+0000 7ff173000640 1 --2- 172.21.15.7:0/2515865441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16c0b4870 0x7ff16c1e4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.358+0000 7ff173000640 1 -- 172.21.15.7:0/2515865441 >> 172.21.15.7:0/2515865441 conn(0x7ff16c0740f0 msgr2=0x7ff16c06faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:28.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.358+0000 7ff173000640 1 -- 172.21.15.7:0/2515865441 shutdown_connections 2024-04-25T11:11:28.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.358+0000 7ff173000640 1 -- 172.21.15.7:0/2515865441 wait complete. 2024-04-25T11:11:28.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:28.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.669+0000 7fb46ce00640 1 Processor -- start 2024-04-25T11:11:28.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.669+0000 7fb46ce00640 1 -- start start 2024-04-25T11:11:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.672+0000 7fb46ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb468070630 0x7fb4680cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.672+0000 7fb46ce00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb46815ecc0 0x7fb4680d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.672+0000 7fb46ce00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4680d0c10 con 0x7fb4680d0800 2024-04-25T11:11:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.672+0000 7fb46ce00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb4680b0be0 con 0x7fb468070630 2024-04-25T11:11:28.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.672+0000 7fb46ce00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb4680ce9e0 con 0x7fb46815ecc0 2024-04-25T11:11:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.673+0000 7fb467e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb4680d0800 0x7fb46806eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45256/0 (socket says 172.21.15.7:45256) 2024-04-25T11:11:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.673+0000 7fb467e00640 1 -- 172.21.15.7:0/887174684 learned_addr learned my addr 172.21.15.7:0/887174684 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.673+0000 7fb45ea00640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb46815ecc0 0x7fb4680d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.673+0000 7fb467400640 1 --2- 172.21.15.7:0/887174684 >> v2:172.21.15.87:3300/0 conn(0x7fb468070630 0x7fb4680cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.673+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2590833773 0 0) 0x7fb460001490 con 0x7fb4680d0800 2024-04-25T11:11:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.673+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb44c003520 con 0x7fb4680d0800 2024-04-25T11:11:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.673+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3151772729 0 0) 0x7fb44c003520 con 0x7fb4680d0800 2024-04-25T11:11:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb46815ecc0 msgr2=0x7fb4680d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb46815ecc0 0x7fb4680d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 >> v2:172.21.15.87:3300/0 conn(0x7fb468070630 msgr2=0x7fb4680cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 --2- 172.21.15.7:0/887174684 >> v2:172.21.15.87:3300/0 conn(0x7fb468070630 0x7fb4680cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4680cecc0 con 0x7fb4680d0800 2024-04-25T11:11:28.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fb460001bc0 con 0x7fb4680d0800 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 >> v1:172.21.15.7:6789/0 conn(0x7fb4680d0800 legacy=0x7fb46806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb44c0039b0 0x7fb44c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c004800 0x7fb44c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb46815ecc0 0x7fb44c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4680b0be0 con 0x7fb44c004800 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb4680ce9e0 con 0x7fb44c0039b0 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb44c00b9c0 con 0x7fb46815ecc0 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.674+0000 7fb45ea00640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c004800 0x7fb44c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.675+0000 7fb467400640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb44c0039b0 0x7fb44c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.675+0000 7fb45ea00640 1 -- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb46815ecc0 msgr2=0x7fb44c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.675+0000 7fb467e00640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb46815ecc0 0x7fb44c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.675+0000 7fb45ea00640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb46815ecc0 0x7fb44c005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.675+0000 7fb45ea00640 1 -- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb44c0039b0 msgr2=0x7fb44c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.675+0000 7fb45ea00640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb44c0039b0 0x7fb44c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.675+0000 7fb45ea00640 1 -- 172.21.15.7:0/887174684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c003520 con 0x7fb44c004800 2024-04-25T11:11:28.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.675+0000 7fb467400640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb44c0039b0 0x7fb44c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.676+0000 7fb45ea00640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c004800 0x7fb44c004c10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb450009ac0 tx=0x7fb44c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b952fd8909c5e7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.676+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450036070 con 0x7fb44c004800 2024-04-25T11:11:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.677+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb450031400 con 0x7fb44c004800 2024-04-25T11:11:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.677+0000 7fb466a00640 1 -- 172.21.15.7:0/887174684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450009690 con 0x7fb44c004800 2024-04-25T11:11:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.677+0000 7fb46ce00640 1 -- 172.21.15.7:0/887174684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c004800 msgr2=0x7fb44c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.677+0000 7fb46ce00640 1 --2- 172.21.15.7:0/887174684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44c004800 0x7fb44c004c10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb450009ac0 tx=0x7fb44c005830 comp rx=0 tx=0).stop 2024-04-25T11:11:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.677+0000 7fb467e00640 1 -- 172.21.15.7:0/887174684 reap_dead start 2024-04-25T11:11:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.677+0000 7fb46ce00640 1 -- 172.21.15.7:0/887174684 shutdown_connections 2024-04-25T11:11:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.677+0000 7fb46ce00640 1 -- 172.21.15.7:0/887174684 >> 172.21.15.7:0/887174684 conn(0x7fb468074140 msgr2=0x7fb46806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.677+0000 7fb46ce00640 1 -- 172.21.15.7:0/887174684 shutdown_connections 2024-04-25T11:11:28.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.677+0000 7fb46ce00640 1 -- 172.21.15.7:0/887174684 wait complete. 2024-04-25T11:11:28.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.680+0000 7fb46ce00640 1 Processor -- start 2024-04-25T11:11:28.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.680+0000 7fb46ce00640 1 -- start start 2024-04-25T11:11:28.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb46ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb44c004800 0x7fb468154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb46ce00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4681592e0 0x7fb4681596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb46ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb468159c30 0x7fb468157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb46ce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4680cf560 con 0x7fb468159c30 2024-04-25T11:11:28.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb46ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb4680cf280 con 0x7fb44c004800 2024-04-25T11:11:28.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb46ce00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb46815f0d0 con 0x7fb4681592e0 2024-04-25T11:11:28.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb467e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb468159c30 0x7fb468157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb467e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb468159c30 0x7fb468157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54344/0 (socket says 172.21.15.7:54344) 2024-04-25T11:11:28.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb467e00640 1 -- 172.21.15.7:0/791610322 learned_addr learned my addr 172.21.15.7:0/791610322 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:28.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb467400640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb44c004800 0x7fb468154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb467e00640 1 -- 172.21.15.7:0/791610322 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4681592e0 msgr2=0x7fb4681596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:28.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb467e00640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4681592e0 0x7fb4681596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb467e00640 1 -- 172.21.15.7:0/791610322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb44c004800 msgr2=0x7fb468154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:28.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb467e00640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb44c004800 0x7fb468154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.682+0000 7fb467e00640 1 -- 172.21.15.7:0/791610322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46815a040 con 0x7fb468159c30 2024-04-25T11:11:28.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.683+0000 7fb467e00640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb468159c30 0x7fb468157090 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb4680aa3d0 tx=0x7fb46000b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:28.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.683+0000 7fb45fe00640 1 -- 172.21.15.7:0/791610322 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000b8d0 con 0x7fb468159c30 2024-04-25T11:11:28.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.683+0000 7fb45fe00640 1 -- 172.21.15.7:0/791610322 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb46000ba90 con 0x7fb468159c30 2024-04-25T11:11:28.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.683+0000 7fb45fe00640 1 -- 172.21.15.7:0/791610322 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000be80 con 0x7fb468159c30 2024-04-25T11:11:28.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.683+0000 7fb46ce00640 1 -- 172.21.15.7:0/791610322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4681e9770 con 0x7fb468159c30 2024-04-25T11:11:28.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.683+0000 7fb46ce00640 1 -- 172.21.15.7:0/791610322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4681e9d30 con 0x7fb468159c30 2024-04-25T11:11:28.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.684+0000 7fb46ce00640 1 -- 172.21.15.7:0/791610322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb468050740 con 0x7fb468159c30 2024-04-25T11:11:28.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.686+0000 7fb45fe00640 1 -- 172.21.15.7:0/791610322 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb46001b020 con 0x7fb468159c30 2024-04-25T11:11:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.687+0000 7fb45fe00640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4440744b0 0x7fb4440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.687+0000 7fb45fe00640 1 -- 172.21.15.7:0/791610322 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fb460095810 con 0x7fb468159c30 2024-04-25T11:11:28.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.687+0000 7fb467400640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4440744b0 0x7fb4440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:28.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.688+0000 7fb467400640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4440744b0 0x7fb4440769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb458009a40 tx=0x7fb458008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:28.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.692+0000 7fb45fe00640 1 -- 172.21.15.7:0/791610322 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4680d0c10 con 0x7fb468159c30 2024-04-25T11:11:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.897+0000 7fb46ce00640 1 -- 172.21.15.7:0/791610322 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb4681553f0 con 0x7fb468159c30 2024-04-25T11:11:28.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.898+0000 7fb45fe00640 1 -- 172.21.15.7:0/791610322 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fb46005fb00 con 0x7fb468159c30 2024-04-25T11:11:28.898 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:28.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.900+0000 7fb46ce00640 1 -- 172.21.15.7:0/791610322 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4440744b0 msgr2=0x7fb4440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:28.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.900+0000 7fb46ce00640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4440744b0 0x7fb4440769c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb458009a40 tx=0x7fb458008040 comp rx=0 tx=0).stop 2024-04-25T11:11:28.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.900+0000 7fb46ce00640 1 -- 172.21.15.7:0/791610322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb468159c30 msgr2=0x7fb468157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:28.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.900+0000 7fb46ce00640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb468159c30 0x7fb468157090 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb4680aa3d0 tx=0x7fb46000b0d0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.901+0000 7fb46ce00640 1 -- 172.21.15.7:0/791610322 shutdown_connections 2024-04-25T11:11:28.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.901+0000 7fb46ce00640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4440744b0 0x7fb4440769c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.901+0000 7fb46ce00640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb468159c30 0x7fb468157090 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.901+0000 7fb46ce00640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4681592e0 0x7fb4681596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.901+0000 7fb46ce00640 1 --2- 172.21.15.7:0/791610322 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb44c004800 0x7fb468154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.901+0000 7fb46ce00640 1 -- 172.21.15.7:0/791610322 >> 172.21.15.7:0/791610322 conn(0x7fb468074140 msgr2=0x7fb4680b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.901+0000 7fb46ce00640 1 -- 172.21.15.7:0/791610322 shutdown_connections 2024-04-25T11:11:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:28.901+0000 7fb46ce00640 1 -- 172.21.15.7:0/791610322 wait complete. 2024-04-25T11:11:28.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.221+0000 7f6081c00640 1 Processor -- start 2024-04-25T11:11:29.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.221+0000 7f6081c00640 1 -- start start 2024-04-25T11:11:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.223+0000 7f6081c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f607c1583c0 0x7f607c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.223+0000 7f6081c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f607c0cfb50 0x7f607c0d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.223+0000 7f6081c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f607c0d09c0 0x7f607c06eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.223+0000 7f6081c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f607c15f190 con 0x7f607c1583c0 2024-04-25T11:11:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.223+0000 7f6081c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f607c0ce570 con 0x7f607c0cfb50 2024-04-25T11:11:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.223+0000 7f6081c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f607c0ce850 con 0x7f607c0d09c0 2024-04-25T11:11:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.224+0000 7f607be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f607c1583c0 0x7f607c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.224+0000 7f607be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f607c1583c0 0x7f607c0cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54358/0 (socket says 172.21.15.7:54358) 2024-04-25T11:11:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.224+0000 7f607be00640 1 -- 172.21.15.7:0/4142200587 learned_addr learned my addr 172.21.15.7:0/4142200587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.224+0000 7f607be00640 1 -- 172.21.15.7:0/4142200587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f607c0d09c0 msgr2=0x7f607c06eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.224+0000 7f607be00640 1 --2- 172.21.15.7:0/4142200587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f607c0d09c0 0x7f607c06eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.224+0000 7f607be00640 1 -- 172.21.15.7:0/4142200587 >> v2:172.21.15.87:3300/0 conn(0x7f607c0cfb50 msgr2=0x7f607c0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.224+0000 7f607be00640 1 --2- 172.21.15.7:0/4142200587 >> v2:172.21.15.87:3300/0 conn(0x7f607c0cfb50 0x7f607c0d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.224+0000 7f607be00640 1 -- 172.21.15.7:0/4142200587 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c0ceb30 con 0x7f607c1583c0 2024-04-25T11:11:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.225+0000 7f607be00640 1 --2- 172.21.15.7:0/4142200587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607c1583c0 0x7f607c0cf610 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f607c070ad0 tx=0x7f606c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de986fb9593facf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:29.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.226+0000 7f607aa00640 1 -- 172.21.15.7:0/4142200587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c037070 con 0x7f607c1583c0 2024-04-25T11:11:29.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.226+0000 7f607aa00640 1 -- 172.21.15.7:0/4142200587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606c032440 con 0x7f607c1583c0 2024-04-25T11:11:29.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.226+0000 7f607aa00640 1 -- 172.21.15.7:0/4142200587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c032810 con 0x7f607c1583c0 2024-04-25T11:11:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.226+0000 7f6081c00640 1 -- 172.21.15.7:0/4142200587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607c1583c0 msgr2=0x7f607c0cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.226+0000 7f6081c00640 1 --2- 172.21.15.7:0/4142200587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607c1583c0 0x7f607c0cf610 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f607c070ad0 tx=0x7f606c02a200 comp rx=0 tx=0).stop 2024-04-25T11:11:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.226+0000 7f6081c00640 1 -- 172.21.15.7:0/4142200587 shutdown_connections 2024-04-25T11:11:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.226+0000 7f6081c00640 1 --2- 172.21.15.7:0/4142200587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f607c0d09c0 0x7f607c06eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.226+0000 7f6081c00640 1 --2- 172.21.15.7:0/4142200587 >> v2:172.21.15.87:3300/0 conn(0x7f607c0cfb50 0x7f607c0d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.226+0000 7f6081c00640 1 --2- 172.21.15.7:0/4142200587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607c1583c0 0x7f607c0cf610 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.226+0000 7f6081c00640 1 -- 172.21.15.7:0/4142200587 >> 172.21.15.7:0/4142200587 conn(0x7f607c0740f0 msgr2=0x7f607c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.227+0000 7f6081c00640 1 -- 172.21.15.7:0/4142200587 shutdown_connections 2024-04-25T11:11:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.227+0000 7f6081c00640 1 -- 172.21.15.7:0/4142200587 wait complete. 2024-04-25T11:11:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.229+0000 7f6081c00640 1 Processor -- start 2024-04-25T11:11:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.229+0000 7f6081c00640 1 -- start start 2024-04-25T11:11:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f6081c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607c0cfb50 0x7f607c13eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f6081c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f607c0d09c0 0x7f607c1ead20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f6081c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f607c1583c0 0x7f607c1f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f6081c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f607c06f3e0 con 0x7f607c0cfb50 2024-04-25T11:11:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f6081c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f607c0cf0f0 con 0x7f607c0d09c0 2024-04-25T11:11:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f6081c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f607c06f560 con 0x7f607c1583c0 2024-04-25T11:11:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f607be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607c0cfb50 0x7f607c13eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f607be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607c0cfb50 0x7f607c13eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54370/0 (socket says 172.21.15.7:54370) 2024-04-25T11:11:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f607be00640 1 -- 172.21.15.7:0/3490477231 learned_addr learned my addr 172.21.15.7:0/3490477231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f607be00640 1 -- 172.21.15.7:0/3490477231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f607c1583c0 msgr2=0x7f607c1f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f607be00640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f607c1583c0 0x7f607c1f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f607be00640 1 -- 172.21.15.7:0/3490477231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f607c0d09c0 msgr2=0x7f607c1ead20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f607be00640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f607c0d09c0 0x7f607c1ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.231+0000 7f607be00640 1 -- 172.21.15.7:0/3490477231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c1e9c80 con 0x7f607c0cfb50 2024-04-25T11:11:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.232+0000 7f607be00640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607c0cfb50 0x7f607c13eed0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f606c000c00 tx=0x7f606c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:29.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.232+0000 7f6073e00640 1 -- 172.21.15.7:0/3490477231 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c041070 con 0x7f607c0cfb50 2024-04-25T11:11:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.232+0000 7f6073e00640 1 -- 172.21.15.7:0/3490477231 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606c03c890 con 0x7f607c0cfb50 2024-04-25T11:11:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.232+0000 7f6073e00640 1 -- 172.21.15.7:0/3490477231 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606c037040 con 0x7f607c0cfb50 2024-04-25T11:11:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.232+0000 7f6081c00640 1 -- 172.21.15.7:0/3490477231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607c1e9f60 con 0x7f607c0cfb50 2024-04-25T11:11:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.232+0000 7f6081c00640 1 -- 172.21.15.7:0/3490477231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607c1ea520 con 0x7f607c0cfb50 2024-04-25T11:11:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.233+0000 7f6081c00640 1 -- 172.21.15.7:0/3490477231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607c050740 con 0x7f607c0cfb50 2024-04-25T11:11:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.235+0000 7f6073e00640 1 -- 172.21.15.7:0/3490477231 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f606c044070 con 0x7f607c0cfb50 2024-04-25T11:11:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.236+0000 7f6073e00640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f605c0744b0 0x7f605c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.236+0000 7f6073e00640 1 -- 172.21.15.7:0/3490477231 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f606c0b6200 con 0x7f607c0cfb50 2024-04-25T11:11:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.236+0000 7f607b400640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f605c0744b0 0x7f605c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.237+0000 7f607b400640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f605c0744b0 0x7f605c0769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f607c13f590 tx=0x7f6068006bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:29.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.240+0000 7f6073e00640 1 -- 172.21.15.7:0/3490477231 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f606c080610 con 0x7f607c0cfb50 2024-04-25T11:11:29.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.443+0000 7f6081c00640 1 -- 172.21.15.7:0/3490477231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f607c1626a0 con 0x7f607c0cfb50 2024-04-25T11:11:29.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.444+0000 7f6073e00640 1 -- 172.21.15.7:0/3490477231 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f606c07ff10 con 0x7f607c0cfb50 2024-04-25T11:11:29.444 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:29.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 -- 172.21.15.7:0/3490477231 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f605c0744b0 msgr2=0x7f605c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f605c0744b0 0x7f605c0769c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f607c13f590 tx=0x7f6068006bf0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 -- 172.21.15.7:0/3490477231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607c0cfb50 msgr2=0x7f607c13eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607c0cfb50 0x7f607c13eed0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f606c000c00 tx=0x7f606c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 -- 172.21.15.7:0/3490477231 shutdown_connections 2024-04-25T11:11:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f605c0744b0 0x7f605c0769c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f607c1583c0 0x7f607c1f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f607c0d09c0 0x7f607c1ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 --2- 172.21.15.7:0/3490477231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f607c0cfb50 0x7f607c13eed0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 -- 172.21.15.7:0/3490477231 >> 172.21.15.7:0/3490477231 conn(0x7f607c0740f0 msgr2=0x7f607c06c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 -- 172.21.15.7:0/3490477231 shutdown_connections 2024-04-25T11:11:29.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.447+0000 7f6081c00640 1 -- 172.21.15.7:0/3490477231 wait complete. 2024-04-25T11:11:29.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:29.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.751+0000 7f5b8b200640 1 Processor -- start 2024-04-25T11:11:29.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.751+0000 7f5b8b200640 1 -- start start 2024-04-25T11:11:29.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.754+0000 7f5b8b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c070890 0x7f5b8c15c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.754+0000 7f5b8b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5b8c0afca0 0x7f5b8c0b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.754+0000 7f5b8b200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b8c06ef10 con 0x7f5b8c0b5140 2024-04-25T11:11:29.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.754+0000 7f5b8b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8c06f210 con 0x7f5b8c0afca0 2024-04-25T11:11:29.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.754+0000 7f5b8b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c0b6390 con 0x7f5b8c070890 2024-04-25T11:11:29.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.754+0000 7f5b89e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c070890 0x7f5b8c15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:29.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.755+0000 7f5b89e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c070890 0x7f5b8c15c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:38572/0 (socket says 172.21.15.7:38572) 2024-04-25T11:11:29.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.755+0000 7f5b89e00640 1 -- 172.21.15.7:0/4252403627 learned_addr learned my addr 172.21.15.7:0/4252403627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:29.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.754+0000 7f5b8a800640 1 --2- 172.21.15.7:0/4252403627 >> v2:172.21.15.87:3300/0 conn(0x7f5b8c0afca0 0x7f5b8c0b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:29.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.755+0000 7f5b89400640 1 --1- 172.21.15.7:0/4252403627 >> v1:172.21.15.87:6789/0 conn(0x7f5b8c0b5140 0x7f5b8c06f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:60074/0 (socket says 172.21.15.7:60074) 2024-04-25T11:11:29.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.755+0000 7f5b8a800640 1 -- 172.21.15.7:0/4252403627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c070890 msgr2=0x7f5b8c15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:29.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.755+0000 7f5b8a800640 1 --2- 172.21.15.7:0/4252403627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c070890 0x7f5b8c15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.755+0000 7f5b8a800640 1 -- 172.21.15.7:0/4252403627 >> v1:172.21.15.87:6789/0 conn(0x7f5b8c0b5140 legacy=0x7f5b8c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:29.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.755+0000 7f5b8a800640 1 -- 172.21.15.7:0/4252403627 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c0b6670 con 0x7f5b8c0afca0 2024-04-25T11:11:29.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.757+0000 7f5b8a800640 1 --2- 172.21.15.7:0/4252403627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b8c0afca0 0x7f5b8c0b00b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009d90 tx=0x7f5b8402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69fe4b2ec3fc3be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:29.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.758+0000 7f5b88a00640 1 -- 172.21.15.7:0/4252403627 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84031600 con 0x7f5b8c0afca0 2024-04-25T11:11:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.758+0000 7f5b88a00640 1 -- 172.21.15.7:0/4252403627 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b840317a0 con 0x7f5b8c0afca0 2024-04-25T11:11:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.758+0000 7f5b88a00640 1 -- 172.21.15.7:0/4252403627 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84031b70 con 0x7f5b8c0afca0 2024-04-25T11:11:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.758+0000 7f5b8b200640 1 -- 172.21.15.7:0/4252403627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b8c0afca0 msgr2=0x7f5b8c0b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.758+0000 7f5b8b200640 1 --2- 172.21.15.7:0/4252403627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b8c0afca0 0x7f5b8c0b00b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5b84009d90 tx=0x7f5b8402a5d0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.759+0000 7f5b8b200640 1 -- 172.21.15.7:0/4252403627 shutdown_connections 2024-04-25T11:11:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.759+0000 7f5b8b200640 1 --2- 172.21.15.7:0/4252403627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b8c0afca0 0x7f5b8c0b00b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.759+0000 7f5b8b200640 1 --2- 172.21.15.7:0/4252403627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c070890 0x7f5b8c15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.759+0000 7f5b8b200640 1 -- 172.21.15.7:0/4252403627 >> 172.21.15.7:0/4252403627 conn(0x7f5b8c0b3820 msgr2=0x7f5b8c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.759+0000 7f5b8b200640 1 -- 172.21.15.7:0/4252403627 shutdown_connections 2024-04-25T11:11:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.759+0000 7f5b8b200640 1 -- 172.21.15.7:0/4252403627 wait complete. 2024-04-25T11:11:29.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.762+0000 7f5b8b200640 1 Processor -- start 2024-04-25T11:11:29.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.762+0000 7f5b8b200640 1 -- start start 2024-04-25T11:11:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.764+0000 7f5b8b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b8c070890 0x7f5b8c1f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.764+0000 7f5b8b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0afca0 0x7f5b8c1454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.764+0000 7f5b8b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c0b5140 0x7f5b8c145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.764+0000 7f5b8b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0b6950 con 0x7f5b8c0afca0 2024-04-25T11:11:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.764+0000 7f5b8b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c0b5550 con 0x7f5b8c070890 2024-04-25T11:11:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.764+0000 7f5b8b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8c0b05f0 con 0x7f5b8c0b5140 2024-04-25T11:11:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.764+0000 7f5b89400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0afca0 0x7f5b8c1454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.764+0000 7f5b89e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b8c070890 0x7f5b8c1f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.765+0000 7f5b89400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0afca0 0x7f5b8c1454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54378/0 (socket says 172.21.15.7:54378) 2024-04-25T11:11:29.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.765+0000 7f5b89400640 1 -- 172.21.15.7:0/1369708715 learned_addr learned my addr 172.21.15.7:0/1369708715 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:29.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.765+0000 7f5b8a800640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c0b5140 0x7f5b8c145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:29.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.765+0000 7f5b89e00640 1 -- 172.21.15.7:0/1369708715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c0b5140 msgr2=0x7f5b8c145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:29.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.765+0000 7f5b89e00640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c0b5140 0x7f5b8c145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.765+0000 7f5b89e00640 1 -- 172.21.15.7:0/1369708715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0afca0 msgr2=0x7f5b8c1454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:29.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.765+0000 7f5b89e00640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0afca0 0x7f5b8c1454d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.765+0000 7f5b89e00640 1 -- 172.21.15.7:0/1369708715 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c1f2e40 con 0x7f5b8c070890 2024-04-25T11:11:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.765+0000 7f5b8a800640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c0b5140 0x7f5b8c145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.765+0000 7f5b89400640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0afca0 0x7f5b8c1454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:11:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.765+0000 7f5b89e00640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b8c070890 0x7f5b8c1f2900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c010bf0 tx=0x7f5b7c00e730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.766+0000 7f5b82000640 1 -- 172.21.15.7:0/1369708715 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c01e070 con 0x7f5b8c070890 2024-04-25T11:11:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.766+0000 7f5b82000640 1 -- 172.21.15.7:0/1369708715 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b7c011650 con 0x7f5b8c070890 2024-04-25T11:11:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.766+0000 7f5b8b200640 1 -- 172.21.15.7:0/1369708715 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c1550b0 con 0x7f5b8c070890 2024-04-25T11:11:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.766+0000 7f5b82000640 1 -- 172.21.15.7:0/1369708715 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c011a00 con 0x7f5b8c070890 2024-04-25T11:11:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.766+0000 7f5b8b200640 1 -- 172.21.15.7:0/1369708715 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c155670 con 0x7f5b8c070890 2024-04-25T11:11:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.767+0000 7f5b8b200640 1 -- 172.21.15.7:0/1369708715 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c154bd0 con 0x7f5b8c070890 2024-04-25T11:11:29.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.769+0000 7f5b82000640 1 -- 172.21.15.7:0/1369708715 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5b7c021070 con 0x7f5b8c070890 2024-04-25T11:11:29.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.770+0000 7f5b82000640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b680744e0 0x7f5b680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:29.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.770+0000 7f5b82000640 1 -- 172.21.15.7:0/1369708715 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f5b7c098440 con 0x7f5b8c070890 2024-04-25T11:11:29.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.771+0000 7f5b89400640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b680744e0 0x7f5b680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:29.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.771+0000 7f5b89400640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b680744e0 0x7f5b680769f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c149b50 tx=0x7f5b74007ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:29.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.787+0000 7f5b82000640 1 -- 172.21.15.7:0/1369708715 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b8c154bd0 con 0x7f5b8c070890 2024-04-25T11:11:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.995+0000 7f5b8b200640 1 -- 172.21.15.7:0/1369708715 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b8c050740 con 0x7f5b8c070890 2024-04-25T11:11:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:29.999+0000 7f5b82000640 1 -- 172.21.15.7:0/1369708715 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7f5b8c1f2e40 con 0x7f5b8c070890 2024-04-25T11:11:30.000 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:30.000 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.001+0000 7f5b8b200640 1 -- 172.21.15.7:0/1369708715 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b680744e0 msgr2=0x7f5b680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.001+0000 7f5b8b200640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b680744e0 0x7f5b680769f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c149b50 tx=0x7f5b74007ee0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.001+0000 7f5b8b200640 1 -- 172.21.15.7:0/1369708715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b8c070890 msgr2=0x7f5b8c1f2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.001+0000 7f5b8b200640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b8c070890 0x7f5b8c1f2900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c010bf0 tx=0x7f5b7c00e730 comp rx=0 tx=0).stop 2024-04-25T11:11:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.002+0000 7f5b8b200640 1 -- 172.21.15.7:0/1369708715 shutdown_connections 2024-04-25T11:11:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.002+0000 7f5b8b200640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b680744e0 0x7f5b680769f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.002+0000 7f5b8b200640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b8c0b5140 0x7f5b8c145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.002+0000 7f5b8b200640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b8c0afca0 0x7f5b8c1454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.002+0000 7f5b8b200640 1 --2- 172.21.15.7:0/1369708715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b8c070890 0x7f5b8c1f2900 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.002+0000 7f5b8b200640 1 -- 172.21.15.7:0/1369708715 >> 172.21.15.7:0/1369708715 conn(0x7f5b8c0b3820 msgr2=0x7f5b8c15b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.002+0000 7f5b8b200640 1 -- 172.21.15.7:0/1369708715 shutdown_connections 2024-04-25T11:11:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.002+0000 7f5b8b200640 1 -- 172.21.15.7:0/1369708715 wait complete. 2024-04-25T11:11:30.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:30.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:30.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.330+0000 7f2e3bc00640 1 Processor -- start 2024-04-25T11:11:30.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.330+0000 7f2e3bc00640 1 -- start start 2024-04-25T11:11:30.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.332+0000 7f2e3bc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e340b0470 0x7f2e3406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:30.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.333+0000 7f2e3bc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e3415e3c0 con 0x7f2e3415dfb0 2024-04-25T11:11:30.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.333+0000 7f2e3bc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e340cf800 con 0x7f2e3406f8e0 2024-04-25T11:11:30.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.333+0000 7f2e3bc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2e340d1250 con 0x7f2e340b0470 2024-04-25T11:11:30.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.333+0000 7f2e33e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e340b0470 0x7f2e3406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:30.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.333+0000 7f2e33e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e340b0470 0x7f2e3406f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46162/0 (socket says 172.21.15.7:46162) 2024-04-25T11:11:30.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.333+0000 7f2e33e00640 1 -- 172.21.15.7:0/3589696871 learned_addr learned my addr 172.21.15.7:0/3589696871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:30.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.333+0000 7f2e39200640 1 --1- 172.21.15.7:0/3589696871 >> v1:172.21.15.7:6789/0 conn(0x7f2e3415dfb0 0x7f2e340d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58908/0 (socket says 172.21.15.7:58908) 2024-04-25T11:11:30.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.333+0000 7f2e33e00640 1 -- 172.21.15.7:0/3589696871 >> v1:172.21.15.87:6789/0 conn(0x7f2e3406f8e0 legacy=0x7f2e340aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:30.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.333+0000 7f2e33e00640 1 -- 172.21.15.7:0/3589696871 >> v1:172.21.15.7:6789/0 conn(0x7f2e3415dfb0 legacy=0x7f2e340d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:30.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.334+0000 7f2e33e00640 1 -- 172.21.15.7:0/3589696871 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e340cfaa0 con 0x7f2e340b0470 2024-04-25T11:11:30.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.334+0000 7f2e33400640 1 -- 172.21.15.7:0/3589696871 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 344523375 0 0) 0x7f2e3415e3c0 con 0x7f2e3415dfb0 2024-04-25T11:11:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.335+0000 7f2e33e00640 1 --2- 172.21.15.7:0/3589696871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340b0470 0x7f2e3406f3a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2e340b4670 tx=0x7f2e2002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21dd50cc0a800a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.335+0000 7f2e33400640 1 -- 172.21.15.7:0/3589696871 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20037070 con 0x7f2e340b0470 2024-04-25T11:11:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.335+0000 7f2e33400640 1 -- 172.21.15.7:0/3589696871 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e20032440 con 0x7f2e340b0470 2024-04-25T11:11:30.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.335+0000 7f2e33400640 1 -- 172.21.15.7:0/3589696871 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20032810 con 0x7f2e340b0470 2024-04-25T11:11:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.336+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3589696871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340b0470 msgr2=0x7f2e3406f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.336+0000 7f2e3bc00640 1 --2- 172.21.15.7:0/3589696871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340b0470 0x7f2e3406f3a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2e340b4670 tx=0x7f2e2002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.336+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3589696871 shutdown_connections 2024-04-25T11:11:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.336+0000 7f2e3bc00640 1 --2- 172.21.15.7:0/3589696871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340b0470 0x7f2e3406f3a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.336+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3589696871 >> 172.21.15.7:0/3589696871 conn(0x7f2e340742c0 msgr2=0x7f2e3406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.336+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3589696871 shutdown_connections 2024-04-25T11:11:30.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.336+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3589696871 wait complete. 2024-04-25T11:11:30.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.339+0000 7f2e3bc00640 1 Processor -- start 2024-04-25T11:11:30.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.339+0000 7f2e3bc00640 1 -- start start 2024-04-25T11:11:30.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.340+0000 7f2e3bc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e3406f8e0 0x7f2e34151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:30.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.340+0000 7f2e3bc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340b0470 0x7f2e34156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:30.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e3bc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3415dfb0 0x7f2e34156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:30.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e3bc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340d0340 con 0x7f2e340b0470 2024-04-25T11:11:30.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e3bc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2e340cfe40 con 0x7f2e3406f8e0 2024-04-25T11:11:30.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e3bc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2e340d0620 con 0x7f2e3415dfb0 2024-04-25T11:11:30.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340b0470 0x7f2e34156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:30.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340b0470 0x7f2e34156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46178/0 (socket says 172.21.15.7:46178) 2024-04-25T11:11:30.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e33e00640 1 -- 172.21.15.7:0/3232436137 learned_addr learned my addr 172.21.15.7:0/3232436137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:30.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e39200640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e3406f8e0 0x7f2e34151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:30.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e33e00640 1 -- 172.21.15.7:0/3232436137 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3415dfb0 msgr2=0x7f2e34156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e33e00640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3415dfb0 0x7f2e34156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e33e00640 1 -- 172.21.15.7:0/3232436137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e3406f8e0 msgr2=0x7f2e34151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e33e00640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e3406f8e0 0x7f2e34151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.341+0000 7f2e33e00640 1 -- 172.21.15.7:0/3232436137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e341e8430 con 0x7f2e340b0470 2024-04-25T11:11:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.342+0000 7f2e33e00640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340b0470 0x7f2e34156700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e2002a220 tx=0x7f2e200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.342+0000 7f2e30c00640 1 -- 172.21.15.7:0/3232436137 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20041070 con 0x7f2e340b0470 2024-04-25T11:11:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.342+0000 7f2e30c00640 1 -- 172.21.15.7:0/3232436137 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2003c870 con 0x7f2e340b0470 2024-04-25T11:11:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.342+0000 7f2e30c00640 1 -- 172.21.15.7:0/3232436137 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20037040 con 0x7f2e340b0470 2024-04-25T11:11:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.342+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3232436137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e341e8710 con 0x7f2e340b0470 2024-04-25T11:11:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.342+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3232436137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e341e7b30 con 0x7f2e340b0470 2024-04-25T11:11:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.344+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3232436137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3415a0c0 con 0x7f2e340b0470 2024-04-25T11:11:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.345+0000 7f2e30c00640 1 -- 172.21.15.7:0/3232436137 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2e20044070 con 0x7f2e340b0470 2024-04-25T11:11:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.346+0000 7f2e30c00640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2e140744e0 0x7f2e140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.346+0000 7f2e30c00640 1 -- 172.21.15.7:0/3232436137 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f2e200b6230 con 0x7f2e340b0470 2024-04-25T11:11:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.347+0000 7f2e39200640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2e140744e0 0x7f2e140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.347+0000 7f2e39200640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2e140744e0 0x7f2e140769f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e28000f60 tx=0x7f2e28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.351+0000 7f2e30c00640 1 -- 172.21.15.7:0/3232436137 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2e3415a0c0 con 0x7f2e340b0470 2024-04-25T11:11:30.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.558+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3232436137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2e341e81c0 con 0x7f2e340b0470 2024-04-25T11:11:30.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.559+0000 7f2e30c00640 1 -- 172.21.15.7:0/3232436137 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f2e200805b0 con 0x7f2e340b0470 2024-04-25T11:11:30.559 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.562+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3232436137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2e140744e0 msgr2=0x7f2e140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.562+0000 7f2e3bc00640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2e140744e0 0x7f2e140769f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e28000f60 tx=0x7f2e28009040 comp rx=0 tx=0).stop 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.562+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3232436137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340b0470 msgr2=0x7f2e34156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.562+0000 7f2e3bc00640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340b0470 0x7f2e34156700 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e2002a220 tx=0x7f2e200046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.562+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3232436137 shutdown_connections 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.562+0000 7f2e3bc00640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2e140744e0 0x7f2e140769f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.562+0000 7f2e3bc00640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3415dfb0 0x7f2e34156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.562+0000 7f2e3bc00640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340b0470 0x7f2e34156700 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.562+0000 7f2e3bc00640 1 --2- 172.21.15.7:0/3232436137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e3406f8e0 0x7f2e34151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.562+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3232436137 >> 172.21.15.7:0/3232436137 conn(0x7f2e340742c0 msgr2=0x7f2e340aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.563+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3232436137 shutdown_connections 2024-04-25T11:11:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.563+0000 7f2e3bc00640 1 -- 172.21.15.7:0/3232436137 wait complete. 2024-04-25T11:11:30.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.882+0000 7fdad6e00640 1 Processor -- start 2024-04-25T11:11:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.882+0000 7fdad6e00640 1 -- start start 2024-04-25T11:11:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.884+0000 7fdad6e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdad0145e40 0x7fdad0146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.884+0000 7fdad6e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdad00b4870 0x7fdad00b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.885+0000 7fdad6e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad00b66b0 con 0x7fdad00b51c0 2024-04-25T11:11:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.885+0000 7fdad6e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fdad0145b70 con 0x7fdad0145e40 2024-04-25T11:11:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.885+0000 7fdad6e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdad00b5b30 con 0x7fdad00b4870 2024-04-25T11:11:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.885+0000 7fdad5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdad0145e40 0x7fdad0146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.885+0000 7fdad5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdad0145e40 0x7fdad0146250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46190/0 (socket says 172.21.15.7:46190) 2024-04-25T11:11:30.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.885+0000 7fdad5a00640 1 -- 172.21.15.7:0/1568811887 learned_addr learned my addr 172.21.15.7:0/1568811887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:30.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.885+0000 7fdad5000640 1 --2- 172.21.15.7:0/1568811887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdad00b4870 0x7fdad00b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:30.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.885+0000 7fdad5a00640 1 -- 172.21.15.7:0/1568811887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdad00b4870 msgr2=0x7fdad00b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:30.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.885+0000 7fdad5a00640 1 --2- 172.21.15.7:0/1568811887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdad00b4870 0x7fdad00b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.885+0000 7fdad5a00640 1 -- 172.21.15.7:0/1568811887 >> v1:172.21.15.87:6789/0 conn(0x7fdad00b51c0 legacy=0x7fdad0143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:30.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.885+0000 7fdad5a00640 1 -- 172.21.15.7:0/1568811887 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad00b5e10 con 0x7fdad0145e40 2024-04-25T11:11:30.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.887+0000 7fdad5a00640 1 --2- 172.21.15.7:0/1568811887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdad0145e40 0x7fdad0146250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdad00b6880 tx=0x7fdac402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc0238e7fbc94e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:30.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.887+0000 7fdacfe00640 1 -- 172.21.15.7:0/1568811887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4037070 con 0x7fdad0145e40 2024-04-25T11:11:30.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.887+0000 7fdacfe00640 1 -- 172.21.15.7:0/1568811887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac4032440 con 0x7fdad0145e40 2024-04-25T11:11:30.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.887+0000 7fdacfe00640 1 -- 172.21.15.7:0/1568811887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4032810 con 0x7fdad0145e40 2024-04-25T11:11:30.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.888+0000 7fdad6e00640 1 -- 172.21.15.7:0/1568811887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdad0145e40 msgr2=0x7fdad0146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:30.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.888+0000 7fdad6e00640 1 --2- 172.21.15.7:0/1568811887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdad0145e40 0x7fdad0146250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdad00b6880 tx=0x7fdac402a160 comp rx=0 tx=0).stop 2024-04-25T11:11:30.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.888+0000 7fdad6e00640 1 -- 172.21.15.7:0/1568811887 shutdown_connections 2024-04-25T11:11:30.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.888+0000 7fdad6e00640 1 --2- 172.21.15.7:0/1568811887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdad00b4870 0x7fdad00b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.888+0000 7fdad6e00640 1 --2- 172.21.15.7:0/1568811887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdad0145e40 0x7fdad0146250 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.888+0000 7fdad6e00640 1 -- 172.21.15.7:0/1568811887 >> 172.21.15.7:0/1568811887 conn(0x7fdad00740f0 msgr2=0x7fdad006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:30.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.888+0000 7fdad6e00640 1 -- 172.21.15.7:0/1568811887 shutdown_connections 2024-04-25T11:11:30.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.888+0000 7fdad6e00640 1 -- 172.21.15.7:0/1568811887 wait complete. 2024-04-25T11:11:30.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.891+0000 7fdad6e00640 1 Processor -- start 2024-04-25T11:11:30.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.891+0000 7fdad6e00640 1 -- start start 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.892+0000 7fdad6e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdad00b4870 0x7fdad0141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad6e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad00b51c0 0x7fdad01eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad6e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdad0145e40 0x7fdad00a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad6e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdad00b55d0 con 0x7fdad00b4870 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad6e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdad00b63d0 con 0x7fdad00b51c0 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad6e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdad00b5750 con 0x7fdad0145e40 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdad00b4870 0x7fdad0141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdad00b4870 0x7fdad0141fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46198/0 (socket says 172.21.15.7:46198) 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad5a00640 1 -- 172.21.15.7:0/3927366233 learned_addr learned my addr 172.21.15.7:0/3927366233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad5000640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad00b51c0 0x7fdad01eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad5a00640 1 -- 172.21.15.7:0/3927366233 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdad0145e40 msgr2=0x7fdad00a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad5a00640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdad0145e40 0x7fdad00a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad5a00640 1 -- 172.21.15.7:0/3927366233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad00b51c0 msgr2=0x7fdad01eb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad5a00640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad00b51c0 0x7fdad01eb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad5a00640 1 -- 172.21.15.7:0/3927366233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad01eb5e0 con 0x7fdad00b4870 2024-04-25T11:11:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.893+0000 7fdad5000640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad00b51c0 0x7fdad01eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.894+0000 7fdad5a00640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdad00b4870 0x7fdad0141fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdac402a1a0 tx=0x7fdac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.894+0000 7fdacd600640 1 -- 172.21.15.7:0/3927366233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4037040 con 0x7fdad00b4870 2024-04-25T11:11:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.894+0000 7fdacd600640 1 -- 172.21.15.7:0/3927366233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac4042070 con 0x7fdad00b4870 2024-04-25T11:11:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.894+0000 7fdacd600640 1 -- 172.21.15.7:0/3927366233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4030bb0 con 0x7fdad00b4870 2024-04-25T11:11:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.894+0000 7fdad6e00640 1 -- 172.21.15.7:0/3927366233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad01501d0 con 0x7fdad00b4870 2024-04-25T11:11:30.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.894+0000 7fdad6e00640 1 -- 172.21.15.7:0/3927366233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad0150790 con 0x7fdad00b4870 2024-04-25T11:11:30.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.896+0000 7fdad6e00640 1 -- 172.21.15.7:0/3927366233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad0050740 con 0x7fdad00b4870 2024-04-25T11:11:30.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.897+0000 7fdacd600640 1 -- 172.21.15.7:0/3927366233 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdac4047460 con 0x7fdad00b4870 2024-04-25T11:11:30.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.898+0000 7fdacd600640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdab00744b0 0x7fdab00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:30.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.898+0000 7fdacd600640 1 -- 172.21.15.7:0/3927366233 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fdac40b63b0 con 0x7fdad00b4870 2024-04-25T11:11:30.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.898+0000 7fdad5000640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdab00744b0 0x7fdab00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:30.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.899+0000 7fdad5000640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdab00744b0 0x7fdab00769c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdad00ab020 tx=0x7fdabc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:30.903+0000 7fdacd600640 1 -- 172.21.15.7:0/3927366233 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdac40800e0 con 0x7fdad00b4870 2024-04-25T11:11:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.108+0000 7fdad6e00640 1 -- 172.21.15.7:0/3927366233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdad01eb810 con 0x7fdad00b4870 2024-04-25T11:11:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.109+0000 7fdacd600640 1 -- 172.21.15.7:0/3927366233 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fdac40843e0 con 0x7fdad00b4870 2024-04-25T11:11:31.109 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.111+0000 7fdad6e00640 1 -- 172.21.15.7:0/3927366233 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdab00744b0 msgr2=0x7fdab00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.111+0000 7fdad6e00640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdab00744b0 0x7fdab00769c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdad00ab020 tx=0x7fdabc008040 comp rx=0 tx=0).stop 2024-04-25T11:11:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.111+0000 7fdad6e00640 1 -- 172.21.15.7:0/3927366233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdad00b4870 msgr2=0x7fdad0141fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.111+0000 7fdad6e00640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdad00b4870 0x7fdad0141fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdac402a1a0 tx=0x7fdac40046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.111+0000 7fdad6e00640 1 -- 172.21.15.7:0/3927366233 shutdown_connections 2024-04-25T11:11:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.111+0000 7fdad6e00640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdab00744b0 0x7fdab00769c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.111+0000 7fdad6e00640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdad0145e40 0x7fdad00a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.111+0000 7fdad6e00640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdad00b51c0 0x7fdad01eb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.111+0000 7fdad6e00640 1 --2- 172.21.15.7:0/3927366233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdad00b4870 0x7fdad0141fd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.112+0000 7fdad6e00640 1 -- 172.21.15.7:0/3927366233 >> 172.21.15.7:0/3927366233 conn(0x7fdad00740f0 msgr2=0x7fdad00ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.112+0000 7fdad6e00640 1 -- 172.21.15.7:0/3927366233 shutdown_connections 2024-04-25T11:11:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.112+0000 7fdad6e00640 1 -- 172.21.15.7:0/3927366233 wait complete. 2024-04-25T11:11:31.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.425+0000 7f3176a00640 1 Processor -- start 2024-04-25T11:11:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.425+0000 7f3176a00640 1 -- start start 2024-04-25T11:11:31.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.427+0000 7f3176a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f31700d09d0 0x7f317006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.427+0000 7f3176a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31700b0be0 con 0x7f31700cfa80 2024-04-25T11:11:31.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.427+0000 7f3176a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31700cfe90 con 0x7f3170070630 2024-04-25T11:11:31.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.427+0000 7f3176a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f31700ce9e0 con 0x7f31700d09d0 2024-04-25T11:11:31.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.428+0000 7f3174c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f31700cfa80 0x7f31700d02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58910/0 (socket says 172.21.15.7:58910) 2024-04-25T11:11:31.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.428+0000 7f3174c00640 1 -- 172.21.15.7:0/2191489505 learned_addr learned my addr 172.21.15.7:0/2191489505 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:31.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.428+0000 7f3176000640 1 --2- 172.21.15.7:0/2191489505 >> v2:172.21.15.7:3300/0 conn(0x7f31700d09d0 0x7f317006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.428+0000 7f316fe00640 1 -- 172.21.15.7:0/2191489505 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1616433947 0 0) 0x7f31700b0be0 con 0x7f31700cfa80 2024-04-25T11:11:31.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.428+0000 7f316fe00640 1 -- 172.21.15.7:0/2191489505 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3154003520 con 0x7f31700cfa80 2024-04-25T11:11:31.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.428+0000 7f3176000640 1 -- 172.21.15.7:0/2191489505 >> v1:172.21.15.87:6789/0 conn(0x7f3170070630 legacy=0x7f317015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.428+0000 7f3176000640 1 -- 172.21.15.7:0/2191489505 >> v1:172.21.15.7:6789/0 conn(0x7f31700cfa80 legacy=0x7f31700d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.428+0000 7f3176000640 1 -- 172.21.15.7:0/2191489505 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31700cecc0 con 0x7f31700d09d0 2024-04-25T11:11:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.430+0000 7f3176000640 1 --2- 172.21.15.7:0/2191489505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31700d09d0 0x7f317006eea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3168009d90 tx=0x7f316802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428028624db71d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.430+0000 7f316fe00640 1 -- 172.21.15.7:0/2191489505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168007740 con 0x7f31700d09d0 2024-04-25T11:11:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.430+0000 7f316fe00640 1 -- 172.21.15.7:0/2191489505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31680078e0 con 0x7f31700d09d0 2024-04-25T11:11:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.430+0000 7f316fe00640 1 -- 172.21.15.7:0/2191489505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168007c90 con 0x7f31700d09d0 2024-04-25T11:11:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.431+0000 7f3176a00640 1 -- 172.21.15.7:0/2191489505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31700d09d0 msgr2=0x7f317006eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.431+0000 7f3176a00640 1 --2- 172.21.15.7:0/2191489505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31700d09d0 0x7f317006eea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3168009d90 tx=0x7f316802a660 comp rx=0 tx=0).stop 2024-04-25T11:11:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.431+0000 7f3176a00640 1 -- 172.21.15.7:0/2191489505 shutdown_connections 2024-04-25T11:11:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.431+0000 7f3176a00640 1 --2- 172.21.15.7:0/2191489505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31700d09d0 0x7f317006eea0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.431+0000 7f3176a00640 1 -- 172.21.15.7:0/2191489505 >> 172.21.15.7:0/2191489505 conn(0x7f3170074140 msgr2=0x7f317006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:31.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.431+0000 7f3176a00640 1 -- 172.21.15.7:0/2191489505 shutdown_connections 2024-04-25T11:11:31.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.431+0000 7f3176a00640 1 -- 172.21.15.7:0/2191489505 wait complete. 2024-04-25T11:11:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.433+0000 7f3176a00640 1 Processor -- start 2024-04-25T11:11:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.433+0000 7f3176a00640 1 -- start start 2024-04-25T11:11:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.435+0000 7f3176a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3170070630 0x7f3170153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.435+0000 7f3176a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31700cfa80 0x7f3170151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.435+0000 7f3176a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31700b5310 0x7f3170152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.435+0000 7f3176a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31700cf560 con 0x7f31700b5310 2024-04-25T11:11:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.435+0000 7f3176a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f31700cf280 con 0x7f3170070630 2024-04-25T11:11:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.435+0000 7f3176a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f31700ce590 con 0x7f31700cfa80 2024-04-25T11:11:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.435+0000 7f3176000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31700b5310 0x7f3170152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.435+0000 7f3176000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31700b5310 0x7f3170152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46222/0 (socket says 172.21.15.7:46222) 2024-04-25T11:11:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.435+0000 7f3176000640 1 -- 172.21.15.7:0/2578913838 learned_addr learned my addr 172.21.15.7:0/2578913838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.435+0000 7f3174c00640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31700cfa80 0x7f3170151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.435+0000 7f3175600640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3170070630 0x7f3170153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.436+0000 7f3176000640 1 -- 172.21.15.7:0/2578913838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31700cfa80 msgr2=0x7f3170151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.436+0000 7f3176000640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31700cfa80 0x7f3170151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.436+0000 7f3176000640 1 -- 172.21.15.7:0/2578913838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3170070630 msgr2=0x7f3170153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.436+0000 7f3176000640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3170070630 0x7f3170153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.436+0000 7f3176000640 1 -- 172.21.15.7:0/2578913838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3170154920 con 0x7f31700b5310 2024-04-25T11:11:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.436+0000 7f3175600640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3170070630 0x7f3170153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:11:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.436+0000 7f3176000640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31700b5310 0x7f3170152340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3168009d90 tx=0x7f3168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.436+0000 7f316d600640 1 -- 172.21.15.7:0/2578913838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316803f070 con 0x7f31700b5310 2024-04-25T11:11:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.437+0000 7f316d600640 1 -- 172.21.15.7:0/2578913838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3168031c40 con 0x7f31700b5310 2024-04-25T11:11:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.437+0000 7f316d600640 1 -- 172.21.15.7:0/2578913838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168009450 con 0x7f31700b5310 2024-04-25T11:11:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.437+0000 7f3176a00640 1 -- 172.21.15.7:0/2578913838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3170154c00 con 0x7f31700b5310 2024-04-25T11:11:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.437+0000 7f3176a00640 1 -- 172.21.15.7:0/2578913838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31700b4b10 con 0x7f31700b5310 2024-04-25T11:11:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.438+0000 7f3176a00640 1 -- 172.21.15.7:0/2578913838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3170050740 con 0x7f31700b5310 2024-04-25T11:11:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.439+0000 7f316d600640 1 -- 172.21.15.7:0/2578913838 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3168003990 con 0x7f31700b5310 2024-04-25T11:11:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.440+0000 7f316d600640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3150074500 0x7f3150076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.440+0000 7f316d600640 1 -- 172.21.15.7:0/2578913838 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f31680b66a0 con 0x7f31700b5310 2024-04-25T11:11:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.440+0000 7f3175600640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3150074500 0x7f3150076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.441+0000 7f3175600640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3150074500 0x7f3150076a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3160007fd0 tx=0x7f3160009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.446+0000 7f316d600640 1 -- 172.21.15.7:0/2578913838 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f31680803c0 con 0x7f31700b5310 2024-04-25T11:11:31.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.656+0000 7f3176a00640 1 -- 172.21.15.7:0/2578913838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3170070dd0 con 0x7f31700b5310 2024-04-25T11:11:31.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.657+0000 7f316d600640 1 -- 172.21.15.7:0/2578913838 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f31680846c0 con 0x7f31700b5310 2024-04-25T11:11:31.658 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:31.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.660+0000 7f3176a00640 1 -- 172.21.15.7:0/2578913838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3150074500 msgr2=0x7f3150076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.660+0000 7f3176a00640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3150074500 0x7f3150076a10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3160007fd0 tx=0x7f3160009040 comp rx=0 tx=0).stop 2024-04-25T11:11:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.660+0000 7f3176a00640 1 -- 172.21.15.7:0/2578913838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31700b5310 msgr2=0x7f3170152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.660+0000 7f3176a00640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31700b5310 0x7f3170152340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3168009d90 tx=0x7f3168004a40 comp rx=0 tx=0).stop 2024-04-25T11:11:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.660+0000 7f3176a00640 1 -- 172.21.15.7:0/2578913838 shutdown_connections 2024-04-25T11:11:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.660+0000 7f3176a00640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3150074500 0x7f3150076a10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.660+0000 7f3176a00640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31700b5310 0x7f3170152340 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.660+0000 7f3176a00640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31700cfa80 0x7f3170151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.660+0000 7f3176a00640 1 --2- 172.21.15.7:0/2578913838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3170070630 0x7f3170153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.660+0000 7f3176a00640 1 -- 172.21.15.7:0/2578913838 >> 172.21.15.7:0/2578913838 conn(0x7f3170074140 msgr2=0x7f317006eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.660+0000 7f3176a00640 1 -- 172.21.15.7:0/2578913838 shutdown_connections 2024-04-25T11:11:31.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.661+0000 7f3176a00640 1 -- 172.21.15.7:0/2578913838 wait complete. 2024-04-25T11:11:31.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.976+0000 7fd6c0a00640 1 Processor -- start 2024-04-25T11:11:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.977+0000 7fd6c0a00640 1 -- start start 2024-04-25T11:11:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.979+0000 7fd6c0a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd6bc15dfb0 0x7fd6bc0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.979+0000 7fd6c0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc0d0b40 0x7fd6bc06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.979+0000 7fd6c0a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6bc15e3c0 con 0x7fd6bc0d0f50 2024-04-25T11:11:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.979+0000 7fd6c0a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd6bc0b09b0 con 0x7fd6bc15dfb0 2024-04-25T11:11:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.979+0000 7fd6c0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6bc0cf800 con 0x7fd6bc0d0b40 2024-04-25T11:11:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.979+0000 7fd6ba800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd6bc0d0f50 0x7fd6bc161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58922/0 (socket says 172.21.15.7:58922) 2024-04-25T11:11:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.979+0000 7fd6ba800640 1 -- 172.21.15.7:0/1089424169 learned_addr learned my addr 172.21.15.7:0/1089424169 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.979+0000 7fd6b9e00640 1 --2- 172.21.15.7:0/1089424169 >> v2:172.21.15.87:3300/0 conn(0x7fd6bc15dfb0 0x7fd6bc0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.979+0000 7fd6b1400640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc0d0b40 0x7fd6bc06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 230730190 0 0) 0x7fd6bc15e3c0 con 0x7fd6bc0d0f50 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6a0003520 con 0x7fd6bc0d0f50 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1390681580 0 0) 0x7fd6a0003520 con 0x7fd6bc0d0f50 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc0d0b40 msgr2=0x7fd6bc06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc0d0b40 0x7fd6bc06f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 >> v2:172.21.15.87:3300/0 conn(0x7fd6bc15dfb0 msgr2=0x7fd6bc0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 --2- 172.21.15.7:0/1089424169 >> v2:172.21.15.87:3300/0 conn(0x7fd6bc15dfb0 0x7fd6bc0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc0cfaa0 con 0x7fd6bc0d0f50 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fd6b40019f0 con 0x7fd6bc0d0f50 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 >> v1:172.21.15.7:6789/0 conn(0x7fd6bc0d0f50 legacy=0x7fd6bc161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 0x7fd6a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc0d0b40 0x7fd6a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6a0004d40 0x7fd6a0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc0b09b0 con 0x7fd6a00039b0 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc0cf800 con 0x7fd6a0004d40 2024-04-25T11:11:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.980+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a000b9f0 con 0x7fd6bc0d0b40 2024-04-25T11:11:31.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.981+0000 7fd6b9e00640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 0x7fd6a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.981+0000 7fd6ba800640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6a0004d40 0x7fd6a0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.981+0000 7fd6b1400640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc0d0b40 0x7fd6a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.981+0000 7fd6b9e00640 1 -- 172.21.15.7:0/1089424169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc0d0b40 msgr2=0x7fd6a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:31.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.981+0000 7fd6b9e00640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc0d0b40 0x7fd6a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.981+0000 7fd6b9e00640 1 -- 172.21.15.7:0/1089424169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6a0004d40 msgr2=0x7fd6a0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:31.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.981+0000 7fd6b9e00640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6a0004d40 0x7fd6a0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.981+0000 7fd6b9e00640 1 -- 172.21.15.7:0/1089424169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a0003520 con 0x7fd6a00039b0 2024-04-25T11:11:31.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.983+0000 7fd6b9e00640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 0x7fd6a0003e20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc072df0 tx=0x7fd6a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282605180f996d6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:31.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.983+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac036070 con 0x7fd6a00039b0 2024-04-25T11:11:31.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.983+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6ac031400 con 0x7fd6a00039b0 2024-04-25T11:11:31.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.983+0000 7fd6b9400640 1 -- 172.21.15.7:0/1089424169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac009690 con 0x7fd6a00039b0 2024-04-25T11:11:31.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.984+0000 7fd6c0a00640 1 -- 172.21.15.7:0/1089424169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 msgr2=0x7fd6a0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.984+0000 7fd6c0a00640 1 --2- 172.21.15.7:0/1089424169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 0x7fd6a0003e20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc072df0 tx=0x7fd6a0005830 comp rx=0 tx=0).stop 2024-04-25T11:11:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.984+0000 7fd6ba800640 1 -- 172.21.15.7:0/1089424169 reap_dead start 2024-04-25T11:11:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.984+0000 7fd6c0a00640 1 -- 172.21.15.7:0/1089424169 shutdown_connections 2024-04-25T11:11:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.984+0000 7fd6c0a00640 1 -- 172.21.15.7:0/1089424169 >> 172.21.15.7:0/1089424169 conn(0x7fd6bc0742c0 msgr2=0x7fd6bc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.984+0000 7fd6c0a00640 1 -- 172.21.15.7:0/1089424169 shutdown_connections 2024-04-25T11:11:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.984+0000 7fd6c0a00640 1 -- 172.21.15.7:0/1089424169 wait complete. 2024-04-25T11:11:31.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.987+0000 7fd6c0a00640 1 Processor -- start 2024-04-25T11:11:31.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.987+0000 7fd6c0a00640 1 -- start start 2024-04-25T11:11:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6c0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 0x7fd6bc151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6c0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc1527c0 0x7fd6bc0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6c0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6bc15e3c0 0x7fd6bc160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6c0a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc0d0340 con 0x7fd6a00039b0 2024-04-25T11:11:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6c0a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd6bc0d0060 con 0x7fd6bc15e3c0 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6c0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6bc0d0620 con 0x7fd6bc1527c0 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6b9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 0x7fd6bc151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6b9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 0x7fd6bc151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46244/0 (socket says 172.21.15.7:46244) 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6b9e00640 1 -- 172.21.15.7:0/3069692246 learned_addr learned my addr 172.21.15.7:0/3069692246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6b1400640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc1527c0 0x7fd6bc0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6b9e00640 1 -- 172.21.15.7:0/3069692246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc1527c0 msgr2=0x7fd6bc0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6b9e00640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc1527c0 0x7fd6bc0cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6b9e00640 1 -- 172.21.15.7:0/3069692246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6bc15e3c0 msgr2=0x7fd6bc160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.989+0000 7fd6ba800640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6bc15e3c0 0x7fd6bc160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.990+0000 7fd6b9e00640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6bc15e3c0 0x7fd6bc160990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.990+0000 7fd6b9e00640 1 -- 172.21.15.7:0/3069692246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6bc0cbfc0 con 0x7fd6a00039b0 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.990+0000 7fd6b1400640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc1527c0 0x7fd6bc0cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.990+0000 7fd6b9e00640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 0x7fd6bc151230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac02ab50 tx=0x7fd6ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.990+0000 7fd6b2a00640 1 -- 172.21.15.7:0/3069692246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac041030 con 0x7fd6a00039b0 2024-04-25T11:11:31.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.990+0000 7fd6b2a00640 1 -- 172.21.15.7:0/3069692246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6ac03c470 con 0x7fd6a00039b0 2024-04-25T11:11:31.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.990+0000 7fd6b2a00640 1 -- 172.21.15.7:0/3069692246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac036040 con 0x7fd6a00039b0 2024-04-25T11:11:31.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.990+0000 7fd6c0a00640 1 -- 172.21.15.7:0/3069692246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6bc15f050 con 0x7fd6a00039b0 2024-04-25T11:11:31.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.990+0000 7fd6c0a00640 1 -- 172.21.15.7:0/3069692246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6bc0cc580 con 0x7fd6a00039b0 2024-04-25T11:11:31.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.992+0000 7fd6c0a00640 1 -- 172.21.15.7:0/3069692246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6bc050740 con 0x7fd6a00039b0 2024-04-25T11:11:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.993+0000 7fd6b2a00640 1 -- 172.21.15.7:0/3069692246 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd6ac044070 con 0x7fd6a00039b0 2024-04-25T11:11:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.994+0000 7fd6b2a00640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 0x7fd6980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.994+0000 7fd6b2a00640 1 -- 172.21.15.7:0/3069692246 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fd6ac0b5500 con 0x7fd6a00039b0 2024-04-25T11:11:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.994+0000 7fd6b1400640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 0x7fd6980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:31.995+0000 7fd6b1400640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 0x7fd6980769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc159240 tx=0x7fd6a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:31.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.000+0000 7fd6b2a00640 1 -- 172.21.15.7:0/3069692246 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6ac07f230 con 0x7fd6a00039b0 2024-04-25T11:11:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.207+0000 7fd6c0a00640 1 -- 172.21.15.7:0/3069692246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd6bc06d830 con 0x7fd6a00039b0 2024-04-25T11:11:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.208+0000 7fd6b2a00640 1 -- 172.21.15.7:0/3069692246 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fd6ac083530 con 0x7fd6a00039b0 2024-04-25T11:11:32.209 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:32.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.211+0000 7fd6c0a00640 1 -- 172.21.15.7:0/3069692246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 msgr2=0x7fd6980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:32.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.211+0000 7fd6c0a00640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 0x7fd6980769c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc159240 tx=0x7fd6a4008040 comp rx=0 tx=0).stop 2024-04-25T11:11:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.211+0000 7fd6c0a00640 1 -- 172.21.15.7:0/3069692246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 msgr2=0x7fd6bc151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.211+0000 7fd6c0a00640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 0x7fd6bc151230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac02ab50 tx=0x7fd6ac004730 comp rx=0 tx=0).stop 2024-04-25T11:11:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.212+0000 7fd6c0a00640 1 -- 172.21.15.7:0/3069692246 shutdown_connections 2024-04-25T11:11:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.212+0000 7fd6c0a00640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 0x7fd6980769c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.212+0000 7fd6c0a00640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6bc15e3c0 0x7fd6bc160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.212+0000 7fd6c0a00640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6bc1527c0 0x7fd6bc0cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.212+0000 7fd6c0a00640 1 --2- 172.21.15.7:0/3069692246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a00039b0 0x7fd6bc151230 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.212+0000 7fd6c0a00640 1 -- 172.21.15.7:0/3069692246 >> 172.21.15.7:0/3069692246 conn(0x7fd6bc0742c0 msgr2=0x7fd6bc0b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.212+0000 7fd6c0a00640 1 -- 172.21.15.7:0/3069692246 shutdown_connections 2024-04-25T11:11:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.212+0000 7fd6c0a00640 1 -- 172.21.15.7:0/3069692246 wait complete. 2024-04-25T11:11:32.242 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:32.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.531+0000 7f687d400640 1 Processor -- start 2024-04-25T11:11:32.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.531+0000 7f687d400640 1 -- start start 2024-04-25T11:11:32.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.533+0000 7f687d400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68780aed40 0x7f687815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:32.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.533+0000 7f687d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f68780b5220 0x7f68780ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:32.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.534+0000 7f687d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68780af420 0x7f68780af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:32.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.534+0000 7f687d400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f687815e1f0 con 0x7f68780af420 2024-04-25T11:11:32.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.534+0000 7f687d400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f68780afd70 con 0x7f68780b5220 2024-04-25T11:11:32.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.534+0000 7f687d400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f68780b64b0 con 0x7f68780aed40 2024-04-25T11:11:32.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.534+0000 7f687ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68780af420 0x7f68780af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:32.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.534+0000 7f687ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f68780af420 0x7f68780af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46250/0 (socket says 172.21.15.7:46250) 2024-04-25T11:11:32.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.534+0000 7f687ca00640 1 -- 172.21.15.7:0/169764085 learned_addr learned my addr 172.21.15.7:0/169764085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:32.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.534+0000 7f6877400640 1 --2- 172.21.15.7:0/169764085 >> v2:172.21.15.87:3300/0 conn(0x7f68780b5220 0x7f68780ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:32.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.534+0000 7f6877e00640 1 --2- 172.21.15.7:0/169764085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68780aed40 0x7f687815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:32.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.534+0000 7f687ca00640 1 -- 172.21.15.7:0/169764085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68780aed40 msgr2=0x7f687815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:32.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.535+0000 7f687ca00640 1 --2- 172.21.15.7:0/169764085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68780aed40 0x7f687815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.535+0000 7f687ca00640 1 -- 172.21.15.7:0/169764085 >> v2:172.21.15.87:3300/0 conn(0x7f68780b5220 msgr2=0x7f68780ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:32.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.535+0000 7f687ca00640 1 --2- 172.21.15.7:0/169764085 >> v2:172.21.15.87:3300/0 conn(0x7f68780b5220 0x7f68780ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.535+0000 7f687ca00640 1 -- 172.21.15.7:0/169764085 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687806c610 con 0x7f68780af420 2024-04-25T11:11:32.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.535+0000 7f6877e00640 1 --2- 172.21.15.7:0/169764085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68780aed40 0x7f687815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:32.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.536+0000 7f687ca00640 1 --2- 172.21.15.7:0/169764085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68780af420 0x7f68780af830 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6878074450 tx=0x7f687002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e45dcc1c02d706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:32.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.536+0000 7f6876a00640 1 -- 172.21.15.7:0/169764085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870037070 con 0x7f68780af420 2024-04-25T11:11:32.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.536+0000 7f6876a00640 1 -- 172.21.15.7:0/169764085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6870032440 con 0x7f68780af420 2024-04-25T11:11:32.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.536+0000 7f6876a00640 1 -- 172.21.15.7:0/169764085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870032810 con 0x7f68780af420 2024-04-25T11:11:32.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.537+0000 7f687d400640 1 -- 172.21.15.7:0/169764085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68780af420 msgr2=0x7f68780af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:32.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.537+0000 7f687d400640 1 --2- 172.21.15.7:0/169764085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68780af420 0x7f68780af830 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6878074450 tx=0x7f687002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.537+0000 7f687d400640 1 -- 172.21.15.7:0/169764085 shutdown_connections 2024-04-25T11:11:32.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.537+0000 7f687d400640 1 --2- 172.21.15.7:0/169764085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68780af420 0x7f68780af830 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.537+0000 7f687d400640 1 --2- 172.21.15.7:0/169764085 >> v2:172.21.15.87:3300/0 conn(0x7f68780b5220 0x7f68780ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.537+0000 7f687d400640 1 --2- 172.21.15.7:0/169764085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68780aed40 0x7f687815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.537+0000 7f687d400640 1 -- 172.21.15.7:0/169764085 >> 172.21.15.7:0/169764085 conn(0x7f68780b3820 msgr2=0x7f68780b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:32.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.537+0000 7f687d400640 1 -- 172.21.15.7:0/169764085 shutdown_connections 2024-04-25T11:11:32.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.537+0000 7f687d400640 1 -- 172.21.15.7:0/169764085 wait complete. 2024-04-25T11:11:32.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.541+0000 7f687d400640 1 Processor -- start 2024-04-25T11:11:32.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.541+0000 7f687d400640 1 -- start start 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.543+0000 7f687d400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68780aed40 0x7f68781472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.543+0000 7f687d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68780af420 0x7f687809b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.543+0000 7f687d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68780b5220 0x7f6878159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.543+0000 7f687d400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f687806ceb0 con 0x7f68780b5220 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.543+0000 7f687d400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f687806cbd0 con 0x7f68780af420 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.543+0000 7f687d400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f68780b6110 con 0x7f68780aed40 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.543+0000 7f687ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68780b5220 0x7f6878159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.543+0000 7f687ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68780b5220 0x7f6878159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46254/0 (socket says 172.21.15.7:46254) 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.543+0000 7f687ca00640 1 -- 172.21.15.7:0/833147826 learned_addr learned my addr 172.21.15.7:0/833147826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.544+0000 7f6877400640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68780af420 0x7f687809b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.544+0000 7f6877e00640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68780aed40 0x7f68781472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:32.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.544+0000 7f687ca00640 1 -- 172.21.15.7:0/833147826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68780aed40 msgr2=0x7f68781472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:32.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.544+0000 7f687ca00640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68780aed40 0x7f68781472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.544+0000 7f687ca00640 1 -- 172.21.15.7:0/833147826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68780af420 msgr2=0x7f687809b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:32.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.544+0000 7f687ca00640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68780af420 0x7f687809b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.544+0000 7f687ca00640 1 -- 172.21.15.7:0/833147826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687809e400 con 0x7f68780b5220 2024-04-25T11:11:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.544+0000 7f687ca00640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68780b5220 0x7f6878159f80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f687002a220 tx=0x7f68700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.545+0000 7f686fe00640 1 -- 172.21.15.7:0/833147826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870037040 con 0x7f68780b5220 2024-04-25T11:11:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.545+0000 7f686fe00640 1 -- 172.21.15.7:0/833147826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6870042070 con 0x7f68780b5220 2024-04-25T11:11:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.545+0000 7f686fe00640 1 -- 172.21.15.7:0/833147826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870030920 con 0x7f68780b5220 2024-04-25T11:11:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.545+0000 7f687d400640 1 -- 172.21.15.7:0/833147826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68781477f0 con 0x7f68780b5220 2024-04-25T11:11:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.545+0000 7f687d400640 1 -- 172.21.15.7:0/833147826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687809e980 con 0x7f68780b5220 2024-04-25T11:11:32.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.546+0000 7f687d400640 1 -- 172.21.15.7:0/833147826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6878050740 con 0x7f68780b5220 2024-04-25T11:11:32.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.547+0000 7f686fe00640 1 -- 172.21.15.7:0/833147826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6870030ac0 con 0x7f68780b5220 2024-04-25T11:11:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.549+0000 7f686fe00640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68540744b0 0x7f68540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.549+0000 7f686fe00640 1 -- 172.21.15.7:0/833147826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f68700b5ff0 con 0x7f68780b5220 2024-04-25T11:11:32.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.550+0000 7f6877e00640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68540744b0 0x7f68540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:32.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.550+0000 7f6877e00640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68540744b0 0x7f68540769c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f68680070b0 tx=0x7f6868007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:32.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.555+0000 7f686fe00640 1 -- 172.21.15.7:0/833147826 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f687007fcb0 con 0x7f68780b5220 2024-04-25T11:11:32.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.761+0000 7f687d400640 1 -- 172.21.15.7:0/833147826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f687809bee0 con 0x7f68780b5220 2024-04-25T11:11:32.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.764+0000 7f686fe00640 1 -- 172.21.15.7:0/833147826 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7f6870083fb0 con 0x7f68780b5220 2024-04-25T11:11:32.766 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:32.766 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:32.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.766+0000 7f687d400640 1 -- 172.21.15.7:0/833147826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68540744b0 msgr2=0x7f68540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:32.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.766+0000 7f687d400640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68540744b0 0x7f68540769c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f68680070b0 tx=0x7f6868007040 comp rx=0 tx=0).stop 2024-04-25T11:11:32.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.766+0000 7f687d400640 1 -- 172.21.15.7:0/833147826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68780b5220 msgr2=0x7f6878159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:32.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.766+0000 7f687d400640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68780b5220 0x7f6878159f80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f687002a220 tx=0x7f68700046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.766+0000 7f687d400640 1 -- 172.21.15.7:0/833147826 shutdown_connections 2024-04-25T11:11:32.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.766+0000 7f687d400640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68540744b0 0x7f68540769c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.767+0000 7f687d400640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68780b5220 0x7f6878159f80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.767+0000 7f687d400640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68780af420 0x7f687809b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.767+0000 7f687d400640 1 --2- 172.21.15.7:0/833147826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68780aed40 0x7f68781472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:32.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.767+0000 7f687d400640 1 -- 172.21.15.7:0/833147826 >> 172.21.15.7:0/833147826 conn(0x7f68780b3820 msgr2=0x7f68780b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:32.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.767+0000 7f687d400640 1 -- 172.21.15.7:0/833147826 shutdown_connections 2024-04-25T11:11:32.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:32.767+0000 7f687d400640 1 -- 172.21.15.7:0/833147826 wait complete. 2024-04-25T11:11:32.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:32.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.084+0000 7f69d1400640 1 Processor -- start 2024-04-25T11:11:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.084+0000 7f69d1400640 1 -- start start 2024-04-25T11:11:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.087+0000 7f69d1400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69cc15dfb0 0x7f69cc0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.087+0000 7f69d1400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69cc06fab0 0x7f69cc0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.087+0000 7f69d1400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69cc15e3c0 con 0x7f69cc0d0b40 2024-04-25T11:11:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.087+0000 7f69d1400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f69cc0b09b0 con 0x7f69cc06fab0 2024-04-25T11:11:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.087+0000 7f69d1400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f69cc0cf800 con 0x7f69cc15dfb0 2024-04-25T11:11:33.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.087+0000 7f69cb200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69cc06fab0 0x7f69cc0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:33.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.087+0000 7f69cb200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69cc06fab0 0x7f69cc0aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46266/0 (socket says 172.21.15.7:46266) 2024-04-25T11:11:33.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.087+0000 7f69cb200640 1 -- 172.21.15.7:0/788385776 learned_addr learned my addr 172.21.15.7:0/788385776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:33.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.088+0000 7f69ca800640 1 --2- 172.21.15.7:0/788385776 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69cc15dfb0 0x7f69cc0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:33.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.088+0000 7f69cb200640 1 -- 172.21.15.7:0/788385776 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69cc15dfb0 msgr2=0x7f69cc0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:33.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.088+0000 7f69cb200640 1 --2- 172.21.15.7:0/788385776 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69cc15dfb0 0x7f69cc0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.088+0000 7f69cb200640 1 -- 172.21.15.7:0/788385776 >> v1:172.21.15.87:6789/0 conn(0x7f69cc0d0b40 legacy=0x7f69cc06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.088+0000 7f69cb200640 1 -- 172.21.15.7:0/788385776 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc0cfaa0 con 0x7f69cc06fab0 2024-04-25T11:11:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.089+0000 7f69cb200640 1 --2- 172.21.15.7:0/788385776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69cc06fab0 0x7f69cc0aea90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009e50 tx=0x7f69c402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5124666115141ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.089+0000 7f69c9400640 1 -- 172.21.15.7:0/788385776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4036070 con 0x7f69cc06fab0 2024-04-25T11:11:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.089+0000 7f69c9400640 1 -- 172.21.15.7:0/788385776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c4031600 con 0x7f69cc06fab0 2024-04-25T11:11:33.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.089+0000 7f69c9400640 1 -- 172.21.15.7:0/788385776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c40319d0 con 0x7f69cc06fab0 2024-04-25T11:11:33.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.090+0000 7f69d1400640 1 -- 172.21.15.7:0/788385776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69cc06fab0 msgr2=0x7f69cc0aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:33.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.090+0000 7f69d1400640 1 --2- 172.21.15.7:0/788385776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69cc06fab0 0x7f69cc0aea90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009e50 tx=0x7f69c402a710 comp rx=0 tx=0).stop 2024-04-25T11:11:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.090+0000 7f69d1400640 1 -- 172.21.15.7:0/788385776 shutdown_connections 2024-04-25T11:11:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.090+0000 7f69d1400640 1 --2- 172.21.15.7:0/788385776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69cc06fab0 0x7f69cc0aea90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.090+0000 7f69d1400640 1 --2- 172.21.15.7:0/788385776 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69cc15dfb0 0x7f69cc0b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.090+0000 7f69d1400640 1 -- 172.21.15.7:0/788385776 >> 172.21.15.7:0/788385776 conn(0x7f69cc0742c0 msgr2=0x7f69cc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.090+0000 7f69d1400640 1 -- 172.21.15.7:0/788385776 shutdown_connections 2024-04-25T11:11:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.090+0000 7f69d1400640 1 -- 172.21.15.7:0/788385776 wait complete. 2024-04-25T11:11:33.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.093+0000 7f69d1400640 1 Processor -- start 2024-04-25T11:11:33.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.093+0000 7f69d1400640 1 -- start start 2024-04-25T11:11:33.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.094+0000 7f69d1400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69cc06fab0 0x7f69cc0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.094+0000 7f69d1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69cc0d0b40 0x7f69cc15fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.094+0000 7f69d1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69cc15dfb0 0x7f69cc15ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.094+0000 7f69d1400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc0d0340 con 0x7f69cc15dfb0 2024-04-25T11:11:33.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.094+0000 7f69d1400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f69cc0d0060 con 0x7f69cc0d0b40 2024-04-25T11:11:33.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69d1400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f69cc0d0620 con 0x7f69cc06fab0 2024-04-25T11:11:33.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69cb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69cc15dfb0 0x7f69cc15ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:33.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69cb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69cc15dfb0 0x7f69cc15ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46272/0 (socket says 172.21.15.7:46272) 2024-04-25T11:11:33.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69cb200640 1 -- 172.21.15.7:0/2193966640 learned_addr learned my addr 172.21.15.7:0/2193966640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69ca800640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69cc06fab0 0x7f69cc0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69c9e00640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69cc0d0b40 0x7f69cc15fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69cb200640 1 -- 172.21.15.7:0/2193966640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69cc06fab0 msgr2=0x7f69cc0cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69cb200640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69cc06fab0 0x7f69cc0cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69cb200640 1 -- 172.21.15.7:0/2193966640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69cc0d0b40 msgr2=0x7f69cc15fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69cb200640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69cc0d0b40 0x7f69cc15fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69cb200640 1 -- 172.21.15.7:0/2193966640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69cc0cb590 con 0x7f69cc15dfb0 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.095+0000 7f69c9e00640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69cc0d0b40 0x7f69cc15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.096+0000 7f69cb200640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69cc15dfb0 0x7f69cc15ffb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f69c4034040 tx=0x7f69c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.096+0000 7f69c2a00640 1 -- 172.21.15.7:0/2193966640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4036050 con 0x7f69cc15dfb0 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.096+0000 7f69c2a00640 1 -- 172.21.15.7:0/2193966640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69c4042070 con 0x7f69cc15dfb0 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.096+0000 7f69d1400640 1 -- 172.21.15.7:0/2193966640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69cc0cb870 con 0x7f69cc15dfb0 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.096+0000 7f69c2a00640 1 -- 172.21.15.7:0/2193966640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c4009510 con 0x7f69cc15dfb0 2024-04-25T11:11:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.096+0000 7f69d1400640 1 -- 172.21.15.7:0/2193966640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69cc0cbe30 con 0x7f69cc15dfb0 2024-04-25T11:11:33.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.097+0000 7f69d1400640 1 -- 172.21.15.7:0/2193966640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69cc0cb300 con 0x7f69cc15dfb0 2024-04-25T11:11:33.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.098+0000 7f69c2a00640 1 -- 172.21.15.7:0/2193966640 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f69c4007bd0 con 0x7f69cc15dfb0 2024-04-25T11:11:33.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.099+0000 7f69c2a00640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69ac0744b0 0x7f69ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.099+0000 7f69c2a00640 1 -- 172.21.15.7:0/2193966640 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f69c40b64f0 con 0x7f69cc15dfb0 2024-04-25T11:11:33.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.100+0000 7f69ca800640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69ac0744b0 0x7f69ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:33.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.100+0000 7f69ca800640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69ac0744b0 0x7f69ac0769c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69bc006fd0 tx=0x7f69bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:33.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.104+0000 7f69c2a00640 1 -- 172.21.15.7:0/2193966640 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f69c4080870 con 0x7f69cc15dfb0 2024-04-25T11:11:33.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.307+0000 7f69d1400640 1 -- 172.21.15.7:0/2193966640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69cc050740 con 0x7f69cc15dfb0 2024-04-25T11:11:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.308+0000 7f69c2a00640 1 -- 172.21.15.7:0/2193966640 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f69c4080170 con 0x7f69cc15dfb0 2024-04-25T11:11:33.308 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:33.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 -- 172.21.15.7:0/2193966640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69ac0744b0 msgr2=0x7f69ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:33.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69ac0744b0 0x7f69ac0769c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69bc006fd0 tx=0x7f69bc008040 comp rx=0 tx=0).stop 2024-04-25T11:11:33.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 -- 172.21.15.7:0/2193966640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69cc15dfb0 msgr2=0x7f69cc15ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:33.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69cc15dfb0 0x7f69cc15ffb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f69c4034040 tx=0x7f69c4004a40 comp rx=0 tx=0).stop 2024-04-25T11:11:33.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 -- 172.21.15.7:0/2193966640 shutdown_connections 2024-04-25T11:11:33.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69ac0744b0 0x7f69ac0769c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69cc15dfb0 0x7f69cc15ffb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69cc0d0b40 0x7f69cc15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 --2- 172.21.15.7:0/2193966640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69cc06fab0 0x7f69cc0cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 -- 172.21.15.7:0/2193966640 >> 172.21.15.7:0/2193966640 conn(0x7f69cc0742c0 msgr2=0x7f69cc0b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:33.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 -- 172.21.15.7:0/2193966640 shutdown_connections 2024-04-25T11:11:33.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.311+0000 7f69d1400640 1 -- 172.21.15.7:0/2193966640 wait complete. 2024-04-25T11:11:33.337 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.618+0000 7f6795000640 1 Processor -- start 2024-04-25T11:11:33.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.618+0000 7f6795000640 1 -- start start 2024-04-25T11:11:33.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.621+0000 7f6795000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67901583c0 0x7f67900cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.621+0000 7f6795000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67900cfb50 0x7f67900d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.621+0000 7f6795000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f67900d09c0 0x7f679006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.621+0000 7f6795000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f679015f190 con 0x7f67900cfb50 2024-04-25T11:11:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.621+0000 7f6795000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f67900ce570 con 0x7f67900d09c0 2024-04-25T11:11:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.621+0000 7f6795000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f67900ce850 con 0x7f67901583c0 2024-04-25T11:11:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.621+0000 7f678ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67900cfb50 0x7f67900d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.621+0000 7f678ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67900cfb50 0x7f67900d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46274/0 (socket says 172.21.15.7:46274) 2024-04-25T11:11:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.621+0000 7f678ea00640 1 -- 172.21.15.7:0/2974157819 learned_addr learned my addr 172.21.15.7:0/2974157819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.622+0000 7f678fe00640 1 --2- 172.21.15.7:0/2974157819 >> v2:172.21.15.87:3300/0 conn(0x7f67900d09c0 0x7f679006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.622+0000 7f678f400640 1 --2- 172.21.15.7:0/2974157819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67901583c0 0x7f67900cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.622+0000 7f678ea00640 1 -- 172.21.15.7:0/2974157819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67901583c0 msgr2=0x7f67900cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.622+0000 7f678ea00640 1 --2- 172.21.15.7:0/2974157819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67901583c0 0x7f67900cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.622+0000 7f678ea00640 1 -- 172.21.15.7:0/2974157819 >> v2:172.21.15.87:3300/0 conn(0x7f67900d09c0 msgr2=0x7f679006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.622+0000 7f678ea00640 1 --2- 172.21.15.7:0/2974157819 >> v2:172.21.15.87:3300/0 conn(0x7f67900d09c0 0x7f679006eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.622+0000 7f678ea00640 1 -- 172.21.15.7:0/2974157819 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67900ceb30 con 0x7f67900cfb50 2024-04-25T11:11:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.623+0000 7f678ea00640 1 --2- 172.21.15.7:0/2974157819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900cfb50 0x7f67900d0480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f67900cdfa0 tx=0x7f677c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6607cef17d638f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.623+0000 7f678e000640 1 -- 172.21.15.7:0/2974157819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c037070 con 0x7f67900cfb50 2024-04-25T11:11:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.623+0000 7f678e000640 1 -- 172.21.15.7:0/2974157819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c032440 con 0x7f67900cfb50 2024-04-25T11:11:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.624+0000 7f678e000640 1 -- 172.21.15.7:0/2974157819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c032810 con 0x7f67900cfb50 2024-04-25T11:11:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.624+0000 7f6795000640 1 -- 172.21.15.7:0/2974157819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900cfb50 msgr2=0x7f67900d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.624+0000 7f6795000640 1 --2- 172.21.15.7:0/2974157819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900cfb50 0x7f67900d0480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f67900cdfa0 tx=0x7f677c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.624+0000 7f6795000640 1 -- 172.21.15.7:0/2974157819 shutdown_connections 2024-04-25T11:11:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.624+0000 7f6795000640 1 --2- 172.21.15.7:0/2974157819 >> v2:172.21.15.87:3300/0 conn(0x7f67900d09c0 0x7f679006eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.624+0000 7f6795000640 1 --2- 172.21.15.7:0/2974157819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900cfb50 0x7f67900d0480 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.624+0000 7f6795000640 1 --2- 172.21.15.7:0/2974157819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67901583c0 0x7f67900cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.624+0000 7f6795000640 1 -- 172.21.15.7:0/2974157819 >> 172.21.15.7:0/2974157819 conn(0x7f67900740f0 msgr2=0x7f679006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.624+0000 7f6795000640 1 -- 172.21.15.7:0/2974157819 shutdown_connections 2024-04-25T11:11:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.624+0000 7f6795000640 1 -- 172.21.15.7:0/2974157819 wait complete. 2024-04-25T11:11:33.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.627+0000 7f6795000640 1 Processor -- start 2024-04-25T11:11:33.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.627+0000 7f6795000640 1 -- start start 2024-04-25T11:11:33.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.628+0000 7f6795000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900cfb50 0x7f6790141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f6795000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67900d09c0 0x7f67901f9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f6795000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67901583c0 0x7f679015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f6795000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f679006f3e0 con 0x7f67900cfb50 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f6795000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f67900cf0f0 con 0x7f67901583c0 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f6795000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f679006f560 con 0x7f67900d09c0 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f678f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900cfb50 0x7f6790141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f678f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900cfb50 0x7f6790141d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46290/0 (socket says 172.21.15.7:46290) 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f678f400640 1 -- 172.21.15.7:0/554331474 learned_addr learned my addr 172.21.15.7:0/554331474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f678f400640 1 -- 172.21.15.7:0/554331474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67900d09c0 msgr2=0x7f67901f9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f678f400640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67900d09c0 0x7f67901f9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f678f400640 1 -- 172.21.15.7:0/554331474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67901583c0 msgr2=0x7f679015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f678f400640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67901583c0 0x7f679015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.629+0000 7f678f400640 1 -- 172.21.15.7:0/554331474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67901fa5d0 con 0x7f67900cfb50 2024-04-25T11:11:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.630+0000 7f678f400640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900cfb50 0x7f6790141d10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f678000d5a0 tx=0x7f678000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:33.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.630+0000 7f6787400640 1 -- 172.21.15.7:0/554331474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780019070 con 0x7f67900cfb50 2024-04-25T11:11:33.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.630+0000 7f6787400640 1 -- 172.21.15.7:0/554331474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6780004240 con 0x7f67900cfb50 2024-04-25T11:11:33.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.630+0000 7f6787400640 1 -- 172.21.15.7:0/554331474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780003610 con 0x7f67900cfb50 2024-04-25T11:11:33.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.630+0000 7f6795000640 1 -- 172.21.15.7:0/554331474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67900b4f90 con 0x7f67900cfb50 2024-04-25T11:11:33.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.630+0000 7f6795000640 1 -- 172.21.15.7:0/554331474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67901429a0 con 0x7f67900cfb50 2024-04-25T11:11:33.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.631+0000 7f6795000640 1 -- 172.21.15.7:0/554331474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6790050740 con 0x7f67900cfb50 2024-04-25T11:11:33.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.632+0000 7f6787400640 1 -- 172.21.15.7:0/554331474 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f678001c070 con 0x7f67900cfb50 2024-04-25T11:11:33.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.633+0000 7f6787400640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744e0 0x7f67700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.634+0000 7f6787400640 1 -- 172.21.15.7:0/554331474 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f6780094ee0 con 0x7f67900cfb50 2024-04-25T11:11:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.634+0000 7f678ea00640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744e0 0x7f67700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:33.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.635+0000 7f678ea00640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744e0 0x7f67700769f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f677c034000 tx=0x7f677c02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:33.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.639+0000 7f6787400640 1 -- 172.21.15.7:0/554331474 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f678005ec10 con 0x7f67900cfb50 2024-04-25T11:11:33.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.842+0000 7f6795000640 1 -- 172.21.15.7:0/554331474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6790142c00 con 0x7f67900cfb50 2024-04-25T11:11:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.843+0000 7f6787400640 1 -- 172.21.15.7:0/554331474 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f6780062f10 con 0x7f67900cfb50 2024-04-25T11:11:33.843 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 -- 172.21.15.7:0/554331474 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744e0 msgr2=0x7f67700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744e0 0x7f67700769f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f677c034000 tx=0x7f677c02c000 comp rx=0 tx=0).stop 2024-04-25T11:11:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 -- 172.21.15.7:0/554331474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900cfb50 msgr2=0x7f6790141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900cfb50 0x7f6790141d10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f678000d5a0 tx=0x7f678000da60 comp rx=0 tx=0).stop 2024-04-25T11:11:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 -- 172.21.15.7:0/554331474 shutdown_connections 2024-04-25T11:11:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744e0 0x7f67700769f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67901583c0 0x7f679015cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67900d09c0 0x7f67901f9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 --2- 172.21.15.7:0/554331474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900cfb50 0x7f6790141d10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 -- 172.21.15.7:0/554331474 >> 172.21.15.7:0/554331474 conn(0x7f67900740f0 msgr2=0x7f6790160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 -- 172.21.15.7:0/554331474 shutdown_connections 2024-04-25T11:11:33.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:33.846+0000 7f6795000640 1 -- 172.21.15.7:0/554331474 wait complete. 2024-04-25T11:11:33.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:34.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.161+0000 7ff6d7800640 1 Processor -- start 2024-04-25T11:11:34.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.161+0000 7ff6d7800640 1 -- start start 2024-04-25T11:11:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.163+0000 7ff6d7800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6d015dfb0 0x7ff6d00b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.163+0000 7ff6d7800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6d015e3c0 con 0x7ff6d00d0b40 2024-04-25T11:11:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.163+0000 7ff6d7800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6d00cf800 con 0x7ff6d006fab0 2024-04-25T11:11:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.163+0000 7ff6d7800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff6d00b09b0 con 0x7ff6d015dfb0 2024-04-25T11:11:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.163+0000 7ff6d4e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6d015dfb0 0x7ff6d00b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.163+0000 7ff6c7e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff6d00d0b40 0x7ff6d006f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58936/0 (socket says 172.21.15.7:58936) 2024-04-25T11:11:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.163+0000 7ff6c7e00640 1 -- 172.21.15.7:0/2792060192 learned_addr learned my addr 172.21.15.7:0/2792060192 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.164+0000 7ff6d4e00640 1 -- 172.21.15.7:0/2792060192 >> v1:172.21.15.87:6789/0 conn(0x7ff6d006fab0 legacy=0x7ff6d00aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.164+0000 7ff6d4e00640 1 -- 172.21.15.7:0/2792060192 >> v1:172.21.15.7:6789/0 conn(0x7ff6d00d0b40 legacy=0x7ff6d006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.164+0000 7ff6d4e00640 1 -- 172.21.15.7:0/2792060192 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d00cfaa0 con 0x7ff6d015dfb0 2024-04-25T11:11:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.164+0000 7ff6cfe00640 1 -- 172.21.15.7:0/2792060192 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1415413679 0 0) 0x7ff6d015e3c0 con 0x7ff6d00d0b40 2024-04-25T11:11:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.165+0000 7ff6d4e00640 1 --2- 172.21.15.7:0/2792060192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d015dfb0 0x7ff6d00b0470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00ceb10 tx=0x7ff6c002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cfac0b9b11b4325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.165+0000 7ff6cfe00640 1 -- 172.21.15.7:0/2792060192 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0037070 con 0x7ff6d015dfb0 2024-04-25T11:11:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.165+0000 7ff6cfe00640 1 -- 172.21.15.7:0/2792060192 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6c0032440 con 0x7ff6d015dfb0 2024-04-25T11:11:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.165+0000 7ff6cfe00640 1 -- 172.21.15.7:0/2792060192 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0032810 con 0x7ff6d015dfb0 2024-04-25T11:11:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.166+0000 7ff6d7800640 1 -- 172.21.15.7:0/2792060192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d015dfb0 msgr2=0x7ff6d00b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.166+0000 7ff6d7800640 1 --2- 172.21.15.7:0/2792060192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d015dfb0 0x7ff6d00b0470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00ceb10 tx=0x7ff6c002a210 comp rx=0 tx=0).stop 2024-04-25T11:11:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.166+0000 7ff6d7800640 1 -- 172.21.15.7:0/2792060192 shutdown_connections 2024-04-25T11:11:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.166+0000 7ff6d7800640 1 --2- 172.21.15.7:0/2792060192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d015dfb0 0x7ff6d00b0470 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.166+0000 7ff6d7800640 1 -- 172.21.15.7:0/2792060192 >> 172.21.15.7:0/2792060192 conn(0x7ff6d00742c0 msgr2=0x7ff6d006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.166+0000 7ff6d7800640 1 -- 172.21.15.7:0/2792060192 shutdown_connections 2024-04-25T11:11:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.166+0000 7ff6d7800640 1 -- 172.21.15.7:0/2792060192 wait complete. 2024-04-25T11:11:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.168+0000 7ff6d7800640 1 Processor -- start 2024-04-25T11:11:34.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.169+0000 7ff6d7800640 1 -- start start 2024-04-25T11:11:34.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.170+0000 7ff6d7800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6d006fab0 0x7ff6d0142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.170+0000 7ff6d7800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d00d0b40 0x7ff6d01e7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.170+0000 7ff6d7800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6d015dfb0 0x7ff6d0159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.170+0000 7ff6d7800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d00d0340 con 0x7ff6d00d0b40 2024-04-25T11:11:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.170+0000 7ff6d7800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d00d0060 con 0x7ff6d015dfb0 2024-04-25T11:11:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.170+0000 7ff6d7800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d00d0620 con 0x7ff6d006fab0 2024-04-25T11:11:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.170+0000 7ff6c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d00d0b40 0x7ff6d01e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.170+0000 7ff6c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d00d0b40 0x7ff6d01e7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46310/0 (socket says 172.21.15.7:46310) 2024-04-25T11:11:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.170+0000 7ff6c7e00640 1 -- 172.21.15.7:0/3500412315 learned_addr learned my addr 172.21.15.7:0/3500412315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.171+0000 7ff6d4e00640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6d006fab0 0x7ff6d0142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:34.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.171+0000 7ff6d5800640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6d015dfb0 0x7ff6d0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:34.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.171+0000 7ff6c7e00640 1 -- 172.21.15.7:0/3500412315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6d006fab0 msgr2=0x7ff6d0142240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.171+0000 7ff6c7e00640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6d006fab0 0x7ff6d0142240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.171+0000 7ff6c7e00640 1 -- 172.21.15.7:0/3500412315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6d015dfb0 msgr2=0x7ff6d0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.171+0000 7ff6c7e00640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6d015dfb0 0x7ff6d0159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.171+0000 7ff6c7e00640 1 -- 172.21.15.7:0/3500412315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d01ea320 con 0x7ff6d00d0b40 2024-04-25T11:11:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.171+0000 7ff6c7e00640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d00d0b40 0x7ff6d01e7910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800c3e0 tx=0x7ff6b800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.172+0000 7ff6cd600640 1 -- 172.21.15.7:0/3500412315 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b80074e0 con 0x7ff6d00d0b40 2024-04-25T11:11:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.172+0000 7ff6cd600640 1 -- 172.21.15.7:0/3500412315 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6b80047e0 con 0x7ff6d00d0b40 2024-04-25T11:11:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.172+0000 7ff6cd600640 1 -- 172.21.15.7:0/3500412315 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b8004bd0 con 0x7ff6d00d0b40 2024-04-25T11:11:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.172+0000 7ff6d7800640 1 -- 172.21.15.7:0/3500412315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d01ea600 con 0x7ff6d00d0b40 2024-04-25T11:11:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.172+0000 7ff6d7800640 1 -- 172.21.15.7:0/3500412315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d0155540 con 0x7ff6d00d0b40 2024-04-25T11:11:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.173+0000 7ff6d7800640 1 -- 172.21.15.7:0/3500412315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d0050740 con 0x7ff6d00d0b40 2024-04-25T11:11:34.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.175+0000 7ff6cd600640 1 -- 172.21.15.7:0/3500412315 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff6b8011070 con 0x7ff6d00d0b40 2024-04-25T11:11:34.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.176+0000 7ff6cd600640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6ac0744b0 0x7ff6ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:34.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.176+0000 7ff6cd600640 1 -- 172.21.15.7:0/3500412315 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff6b8095290 con 0x7ff6d00d0b40 2024-04-25T11:11:34.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.177+0000 7ff6d4e00640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6ac0744b0 0x7ff6ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:34.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.178+0000 7ff6d4e00640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6ac0744b0 0x7ff6ac0769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0034000 tx=0x7ff6c002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:34.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.181+0000 7ff6cd600640 1 -- 172.21.15.7:0/3500412315 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff6b805f610 con 0x7ff6d00d0b40 2024-04-25T11:11:34.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.388+0000 7ff6d7800640 1 -- 172.21.15.7:0/3500412315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff6d006c820 con 0x7ff6d00d0b40 2024-04-25T11:11:34.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.389+0000 7ff6cd600640 1 -- 172.21.15.7:0/3500412315 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7ff6b805ef10 con 0x7ff6d00d0b40 2024-04-25T11:11:34.390 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.392+0000 7ff6d7800640 1 -- 172.21.15.7:0/3500412315 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6ac0744b0 msgr2=0x7ff6ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.392+0000 7ff6d7800640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6ac0744b0 0x7ff6ac0769c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0034000 tx=0x7ff6c002e000 comp rx=0 tx=0).stop 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.392+0000 7ff6d7800640 1 -- 172.21.15.7:0/3500412315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d00d0b40 msgr2=0x7ff6d01e7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.392+0000 7ff6d7800640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d00d0b40 0x7ff6d01e7910 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800c3e0 tx=0x7ff6b800c8a0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.392+0000 7ff6d7800640 1 -- 172.21.15.7:0/3500412315 shutdown_connections 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.392+0000 7ff6d7800640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6ac0744b0 0x7ff6ac0769c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.392+0000 7ff6d7800640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6d015dfb0 0x7ff6d0159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.392+0000 7ff6d7800640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d00d0b40 0x7ff6d01e7910 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.392+0000 7ff6d7800640 1 --2- 172.21.15.7:0/3500412315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6d006fab0 0x7ff6d0142240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.392+0000 7ff6d7800640 1 -- 172.21.15.7:0/3500412315 >> 172.21.15.7:0/3500412315 conn(0x7ff6d00742c0 msgr2=0x7ff6d015a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.393+0000 7ff6d7800640 1 -- 172.21.15.7:0/3500412315 shutdown_connections 2024-04-25T11:11:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.393+0000 7ff6d7800640 1 -- 172.21.15.7:0/3500412315 wait complete. 2024-04-25T11:11:34.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.708+0000 7f0897000640 1 Processor -- start 2024-04-25T11:11:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.708+0000 7f0897000640 1 -- start start 2024-04-25T11:11:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.711+0000 7f0897000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08900b0470 0x7f089006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.711+0000 7f0897000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f089006f8e0 0x7f08900aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.711+0000 7f0897000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f089015e3c0 con 0x7f089015dfb0 2024-04-25T11:11:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.711+0000 7f0897000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f08900cf800 con 0x7f089006f8e0 2024-04-25T11:11:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.711+0000 7f0897000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f08900d1250 con 0x7f08900b0470 2024-04-25T11:11:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.712+0000 7f0895000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f089006f8e0 0x7f08900aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.712+0000 7f0895000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f089006f8e0 0x7f08900aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46322/0 (socket says 172.21.15.7:46322) 2024-04-25T11:11:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.712+0000 7f0895000640 1 -- 172.21.15.7:0/3388661668 learned_addr learned my addr 172.21.15.7:0/3388661668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.712+0000 7f0887400640 1 --2- 172.21.15.7:0/3388661668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08900b0470 0x7f089006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.712+0000 7f0895000640 1 -- 172.21.15.7:0/3388661668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08900b0470 msgr2=0x7f089006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.712+0000 7f0895000640 1 --2- 172.21.15.7:0/3388661668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08900b0470 0x7f089006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.712+0000 7f0895000640 1 -- 172.21.15.7:0/3388661668 >> v1:172.21.15.87:6789/0 conn(0x7f089015dfb0 legacy=0x7f08900d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.712+0000 7f0895000640 1 -- 172.21.15.7:0/3388661668 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08900cfaa0 con 0x7f089006f8e0 2024-04-25T11:11:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.713+0000 7f0895000640 1 --2- 172.21.15.7:0/3388661668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f089006f8e0 0x7f08900aea90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0888009870 tx=0x7f088802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959dcb997ececbe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.714+0000 7f088f400640 1 -- 172.21.15.7:0/3388661668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888037070 con 0x7f089006f8e0 2024-04-25T11:11:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.714+0000 7f088f400640 1 -- 172.21.15.7:0/3388661668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0888032440 con 0x7f089006f8e0 2024-04-25T11:11:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.714+0000 7f088f400640 1 -- 172.21.15.7:0/3388661668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888032810 con 0x7f089006f8e0 2024-04-25T11:11:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.715+0000 7f0897000640 1 -- 172.21.15.7:0/3388661668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f089006f8e0 msgr2=0x7f08900aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.715+0000 7f0897000640 1 --2- 172.21.15.7:0/3388661668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f089006f8e0 0x7f08900aea90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0888009870 tx=0x7f088802a270 comp rx=0 tx=0).stop 2024-04-25T11:11:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.715+0000 7f0897000640 1 -- 172.21.15.7:0/3388661668 shutdown_connections 2024-04-25T11:11:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.715+0000 7f0897000640 1 --2- 172.21.15.7:0/3388661668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f089006f8e0 0x7f08900aea90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.715+0000 7f0897000640 1 --2- 172.21.15.7:0/3388661668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08900b0470 0x7f089006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.715+0000 7f0897000640 1 -- 172.21.15.7:0/3388661668 >> 172.21.15.7:0/3388661668 conn(0x7f08900742c0 msgr2=0x7f089006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.715+0000 7f0897000640 1 -- 172.21.15.7:0/3388661668 shutdown_connections 2024-04-25T11:11:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.715+0000 7f0897000640 1 -- 172.21.15.7:0/3388661668 wait complete. 2024-04-25T11:11:34.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.717+0000 7f0897000640 1 Processor -- start 2024-04-25T11:11:34.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.718+0000 7f0897000640 1 -- start start 2024-04-25T11:11:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.719+0000 7f0897000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f089006f8e0 0x7f089009f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.719+0000 7f0897000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08900b0470 0x7f089014a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:34.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.719+0000 7f0897000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f089015dfb0 0x7f089014cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.719+0000 7f0897000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08900d0340 con 0x7f089015dfb0 2024-04-25T11:11:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.719+0000 7f0897000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f08900d0060 con 0x7f089006f8e0 2024-04-25T11:11:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.719+0000 7f0897000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f08900d0620 con 0x7f08900b0470 2024-04-25T11:11:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.719+0000 7f0895000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f089015dfb0 0x7f089014cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.719+0000 7f0895000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f089015dfb0 0x7f089014cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46328/0 (socket says 172.21.15.7:46328) 2024-04-25T11:11:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.719+0000 7f0895000640 1 -- 172.21.15.7:0/793309598 learned_addr learned my addr 172.21.15.7:0/793309598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.720+0000 7f0887400640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08900b0470 0x7f089014a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.720+0000 7f088fe00640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f089006f8e0 0x7f089009f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:34.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.720+0000 7f0895000640 1 -- 172.21.15.7:0/793309598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08900b0470 msgr2=0x7f089014a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.720+0000 7f0895000640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08900b0470 0x7f089014a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.720+0000 7f0895000640 1 -- 172.21.15.7:0/793309598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f089006f8e0 msgr2=0x7f089009f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.720+0000 7f0895000640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f089006f8e0 0x7f089009f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.720+0000 7f0895000640 1 -- 172.21.15.7:0/793309598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0890160d50 con 0x7f089015dfb0 2024-04-25T11:11:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.720+0000 7f0895000640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f089015dfb0 0x7f089014cde0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f088802a2b0 tx=0x7f08880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.721+0000 7f088cc00640 1 -- 172.21.15.7:0/793309598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888037040 con 0x7f089015dfb0 2024-04-25T11:11:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.721+0000 7f088cc00640 1 -- 172.21.15.7:0/793309598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0888042070 con 0x7f089015dfb0 2024-04-25T11:11:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.721+0000 7f088cc00640 1 -- 172.21.15.7:0/793309598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888030920 con 0x7f089015dfb0 2024-04-25T11:11:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.721+0000 7f0897000640 1 -- 172.21.15.7:0/793309598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089009b6e0 con 0x7f089015dfb0 2024-04-25T11:11:34.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.721+0000 7f0897000640 1 -- 172.21.15.7:0/793309598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089014a770 con 0x7f089015dfb0 2024-04-25T11:11:34.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.722+0000 7f0897000640 1 -- 172.21.15.7:0/793309598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0890160ac0 con 0x7f089015dfb0 2024-04-25T11:11:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.724+0000 7f088cc00640 1 -- 172.21.15.7:0/793309598 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0888030ac0 con 0x7f089015dfb0 2024-04-25T11:11:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.725+0000 7f088cc00640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f086c0744e0 0x7f086c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.725+0000 7f088cc00640 1 -- 172.21.15.7:0/793309598 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f08880b5fe0 con 0x7f089015dfb0 2024-04-25T11:11:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.725+0000 7f088fe00640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f086c0744e0 0x7f086c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:34.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.726+0000 7f088fe00640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f086c0744e0 0x7f086c0769f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f08800012e0 tx=0x7f0880009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:34.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.730+0000 7f088cc00640 1 -- 172.21.15.7:0/793309598 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0888080380 con 0x7f089015dfb0 2024-04-25T11:11:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.935+0000 7f0897000640 1 -- 172.21.15.7:0/793309598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0890050740 con 0x7f089015dfb0 2024-04-25T11:11:34.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.936+0000 7f088cc00640 1 -- 172.21.15.7:0/793309598 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f088807fc80 con 0x7f089015dfb0 2024-04-25T11:11:34.936 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.938+0000 7f0897000640 1 -- 172.21.15.7:0/793309598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f086c0744e0 msgr2=0x7f086c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.938+0000 7f0897000640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f086c0744e0 0x7f086c0769f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f08800012e0 tx=0x7f0880009040 comp rx=0 tx=0).stop 2024-04-25T11:11:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.938+0000 7f0897000640 1 -- 172.21.15.7:0/793309598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f089015dfb0 msgr2=0x7f089014cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.938+0000 7f0897000640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f089015dfb0 0x7f089014cde0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f088802a2b0 tx=0x7f08880046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.938+0000 7f0897000640 1 -- 172.21.15.7:0/793309598 shutdown_connections 2024-04-25T11:11:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.938+0000 7f0897000640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f086c0744e0 0x7f086c0769f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.938+0000 7f0897000640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f089015dfb0 0x7f089014cde0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.938+0000 7f0897000640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08900b0470 0x7f089014a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.938+0000 7f0897000640 1 --2- 172.21.15.7:0/793309598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f089006f8e0 0x7f089009f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.939+0000 7f0897000640 1 -- 172.21.15.7:0/793309598 >> 172.21.15.7:0/793309598 conn(0x7f08900742c0 msgr2=0x7f089006c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.939+0000 7f0897000640 1 -- 172.21.15.7:0/793309598 shutdown_connections 2024-04-25T11:11:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:34.939+0000 7f0897000640 1 -- 172.21.15.7:0/793309598 wait complete. 2024-04-25T11:11:34.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:35.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.243+0000 7fb28de00640 1 Processor -- start 2024-04-25T11:11:35.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.243+0000 7fb28de00640 1 -- start start 2024-04-25T11:11:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.246+0000 7fb28de00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28806dc90 0x7fb2880afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.246+0000 7fb28de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb28806f300 0x7fb2880b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.246+0000 7fb28de00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2880b4980 con 0x7fb288160a30 2024-04-25T11:11:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.246+0000 7fb28de00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb2880b4c60 con 0x7fb28806f300 2024-04-25T11:11:35.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.246+0000 7fb28de00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb2880b4f40 con 0x7fb28806dc90 2024-04-25T11:11:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.246+0000 7fb287400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb288160a30 0x7fb2880ae960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58944/0 (socket says 172.21.15.7:58944) 2024-04-25T11:11:35.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.246+0000 7fb287400640 1 -- 172.21.15.7:0/963108529 learned_addr learned my addr 172.21.15.7:0/963108529 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2683753618 0 0) 0x7fb2880b4980 con 0x7fb288160a30 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb26c003520 con 0x7fb288160a30 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286a00640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28806dc90 0x7fb2880afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb27fe00640 1 --2- 172.21.15.7:0/963108529 >> v2:172.21.15.87:3300/0 conn(0x7fb28806f300 0x7fb2880b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1537144568 0 0) 0x7fb26c003520 con 0x7fb288160a30 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28806dc90 msgr2=0x7fb2880afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28806dc90 0x7fb2880afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 >> v2:172.21.15.87:3300/0 conn(0x7fb28806f300 msgr2=0x7fb2880b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 --2- 172.21.15.7:0/963108529 >> v2:172.21.15.87:3300/0 conn(0x7fb28806f300 0x7fb2880b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2880b5220 con 0x7fb288160a30 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fb2800019f0 con 0x7fb288160a30 2024-04-25T11:11:35.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 >> v1:172.21.15.7:6789/0 conn(0x7fb288160a30 legacy=0x7fb2880ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb26c0039b0 0x7fb26c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb26c004360 0x7fb26c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb26c004e80 0x7fb26c00ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2880b4c60 con 0x7fb26c004e80 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb2880b4f40 con 0x7fb26c0039b0 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.247+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb26c005670 con 0x7fb26c004360 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.248+0000 7fb287400640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb26c004e80 0x7fb26c00ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.248+0000 7fb286a00640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb26c0039b0 0x7fb26c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.248+0000 7fb27fe00640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb26c004360 0x7fb26c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.248+0000 7fb287400640 1 -- 172.21.15.7:0/963108529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb26c004360 msgr2=0x7fb26c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.248+0000 7fb287400640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb26c004360 0x7fb26c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.248+0000 7fb287400640 1 -- 172.21.15.7:0/963108529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb26c0039b0 msgr2=0x7fb26c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.248+0000 7fb287400640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb26c0039b0 0x7fb26c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.248+0000 7fb287400640 1 -- 172.21.15.7:0/963108529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c003520 con 0x7fb26c004e80 2024-04-25T11:11:35.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.249+0000 7fb287400640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb26c004e80 0x7fb26c00ffd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2880d18a0 tx=0x7fb26c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2132babd8e3998a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:35.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.250+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28000a3d0 con 0x7fb26c004e80 2024-04-25T11:11:35.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.250+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28002f070 con 0x7fb26c004e80 2024-04-25T11:11:35.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.250+0000 7fb286000640 1 -- 172.21.15.7:0/963108529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28002bbe0 con 0x7fb26c004e80 2024-04-25T11:11:35.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.250+0000 7fb28de00640 1 -- 172.21.15.7:0/963108529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb26c004e80 msgr2=0x7fb26c00ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:35.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.250+0000 7fb28de00640 1 --2- 172.21.15.7:0/963108529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb26c004e80 0x7fb26c00ffd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2880d18a0 tx=0x7fb26c005830 comp rx=0 tx=0).stop 2024-04-25T11:11:35.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.250+0000 7fb287400640 1 -- 172.21.15.7:0/963108529 reap_dead start 2024-04-25T11:11:35.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.250+0000 7fb28de00640 1 -- 172.21.15.7:0/963108529 shutdown_connections 2024-04-25T11:11:35.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.250+0000 7fb28de00640 1 -- 172.21.15.7:0/963108529 >> 172.21.15.7:0/963108529 conn(0x7fb288074200 msgr2=0x7fb28806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:35.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.251+0000 7fb28de00640 1 -- 172.21.15.7:0/963108529 shutdown_connections 2024-04-25T11:11:35.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.251+0000 7fb28de00640 1 -- 172.21.15.7:0/963108529 wait complete. 2024-04-25T11:11:35.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.253+0000 7fb28de00640 1 Processor -- start 2024-04-25T11:11:35.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.253+0000 7fb28de00640 1 -- start start 2024-04-25T11:11:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.255+0000 7fb28de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28815a8b0 0x7fb28814eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.255+0000 7fb28de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28809ec40 0x7fb288152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.255+0000 7fb28de00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28809f050 0x7fb288152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.255+0000 7fb28de00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb2880b6a30 con 0x7fb28815a8b0 2024-04-25T11:11:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.255+0000 7fb28de00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb28815d220 con 0x7fb28809ec40 2024-04-25T11:11:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.255+0000 7fb28de00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb2880ce920 con 0x7fb28809f050 2024-04-25T11:11:35.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.255+0000 7fb286a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28815a8b0 0x7fb28814eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.255+0000 7fb286a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28815a8b0 0x7fb28814eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46352/0 (socket says 172.21.15.7:46352) 2024-04-25T11:11:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.255+0000 7fb286a00640 1 -- 172.21.15.7:0/2307669292 learned_addr learned my addr 172.21.15.7:0/2307669292 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.256+0000 7fb27fe00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28809ec40 0x7fb288152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.256+0000 7fb286a00640 1 -- 172.21.15.7:0/2307669292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28809f050 msgr2=0x7fb288152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.256+0000 7fb286a00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28809f050 0x7fb288152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.256+0000 7fb286a00640 1 -- 172.21.15.7:0/2307669292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28809ec40 msgr2=0x7fb288152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.256+0000 7fb286a00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28809ec40 0x7fb288152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.256+0000 7fb286a00640 1 -- 172.21.15.7:0/2307669292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb288159810 con 0x7fb28815a8b0 2024-04-25T11:11:35.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.256+0000 7fb286a00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28815a8b0 0x7fb28814eec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb27800f460 tx=0x7fb27800f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:35.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.256+0000 7fb27f400640 1 -- 172.21.15.7:0/2307669292 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278019020 con 0x7fb28815a8b0 2024-04-25T11:11:35.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.256+0000 7fb27f400640 1 -- 172.21.15.7:0/2307669292 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb278014870 con 0x7fb28815a8b0 2024-04-25T11:11:35.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.256+0000 7fb27f400640 1 -- 172.21.15.7:0/2307669292 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278014c20 con 0x7fb28815a8b0 2024-04-25T11:11:35.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.256+0000 7fb28de00640 1 -- 172.21.15.7:0/2307669292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb288159af0 con 0x7fb28815a8b0 2024-04-25T11:11:35.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.257+0000 7fb28de00640 1 -- 172.21.15.7:0/2307669292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28815a0b0 con 0x7fb28815a8b0 2024-04-25T11:11:35.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.258+0000 7fb28de00640 1 -- 172.21.15.7:0/2307669292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28806a100 con 0x7fb28815a8b0 2024-04-25T11:11:35.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.259+0000 7fb27f400640 1 -- 172.21.15.7:0/2307669292 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb27801c020 con 0x7fb28815a8b0 2024-04-25T11:11:35.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.260+0000 7fb27f400640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2640744b0 0x7fb2640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.260+0000 7fb27f400640 1 -- 172.21.15.7:0/2307669292 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fb278095ca0 con 0x7fb28815a8b0 2024-04-25T11:11:35.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.260+0000 7fb27fe00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2640744b0 0x7fb2640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.261+0000 7fb27fe00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2640744b0 0x7fb2640769c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb270006fd0 tx=0x7fb270008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:35.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.265+0000 7fb27f400640 1 -- 172.21.15.7:0/2307669292 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb278023020 con 0x7fb28815a8b0 2024-04-25T11:11:35.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.470+0000 7fb28de00640 1 -- 172.21.15.7:0/2307669292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb288070fb0 con 0x7fb28815a8b0 2024-04-25T11:11:35.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.473+0000 7fb27f400640 1 -- 172.21.15.7:0/2307669292 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7fb278060150 con 0x7fb28815a8b0 2024-04-25T11:11:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:35.508 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:35.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 -- 172.21.15.7:0/2307669292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2640744b0 msgr2=0x7fb2640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:35.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2640744b0 0x7fb2640769c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb270006fd0 tx=0x7fb270008040 comp rx=0 tx=0).stop 2024-04-25T11:11:35.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 -- 172.21.15.7:0/2307669292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28815a8b0 msgr2=0x7fb28814eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28815a8b0 0x7fb28814eec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb27800f460 tx=0x7fb27800f920 comp rx=0 tx=0).stop 2024-04-25T11:11:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 -- 172.21.15.7:0/2307669292 shutdown_connections 2024-04-25T11:11:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2640744b0 0x7fb2640769c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28809f050 0x7fb288152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28809ec40 0x7fb288152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 --2- 172.21.15.7:0/2307669292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28815a8b0 0x7fb28814eec0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 -- 172.21.15.7:0/2307669292 >> 172.21.15.7:0/2307669292 conn(0x7fb288074200 msgr2=0x7fb2880af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 -- 172.21.15.7:0/2307669292 shutdown_connections 2024-04-25T11:11:35.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.476+0000 7fb28de00640 1 -- 172.21.15.7:0/2307669292 wait complete. 2024-04-25T11:11:35.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:35.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.786+0000 7f4863e00640 1 Processor -- start 2024-04-25T11:11:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.786+0000 7f4863e00640 1 -- start start 2024-04-25T11:11:35.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.788+0000 7f4863e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f486406cac0 0x7f486406ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.788+0000 7f4863e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f486415f630 0x7f486415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.788+0000 7f4863e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486406d410 con 0x7f48640d12a0 2024-04-25T11:11:35.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.788+0000 7f4863e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f48640d5420 con 0x7f486415f630 2024-04-25T11:11:35.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.788+0000 7f4863e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48640b8ef0 con 0x7f486406cac0 2024-04-25T11:11:35.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.789+0000 7f4862a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f48640d12a0 0x7f48640d4310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:52790/0 (socket says 172.21.15.7:52790) 2024-04-25T11:11:35.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.789+0000 7f4862a00640 1 -- 172.21.15.7:0/2957321834 learned_addr learned my addr 172.21.15.7:0/2957321834 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:35.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.789+0000 7f485a000640 1 --2- 172.21.15.7:0/2957321834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f486406cac0 0x7f486406ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.789+0000 7f4863400640 1 --2- 172.21.15.7:0/2957321834 >> v2:172.21.15.87:3300/0 conn(0x7f486415f630 0x7f486415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.790+0000 7f485a000640 1 -- 172.21.15.7:0/2957321834 >> v2:172.21.15.87:3300/0 conn(0x7f486415f630 msgr2=0x7f486415fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:35.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.790+0000 7f485a000640 1 --2- 172.21.15.7:0/2957321834 >> v2:172.21.15.87:3300/0 conn(0x7f486415f630 0x7f486415fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.790+0000 7f485a000640 1 -- 172.21.15.7:0/2957321834 >> v1:172.21.15.87:6789/0 conn(0x7f48640d12a0 legacy=0x7f48640d4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:35.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.790+0000 7f485a000640 1 -- 172.21.15.7:0/2957321834 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48640d4e60 con 0x7f486406cac0 2024-04-25T11:11:35.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.790+0000 7f4862000640 1 -- 172.21.15.7:0/2957321834 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1056643764 0 0) 0x7f4854001110 con 0x7f48640d12a0 2024-04-25T11:11:35.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.791+0000 7f485a000640 1 --2- 172.21.15.7:0/2957321834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f486406cac0 0x7f486406ced0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f484c0097d0 tx=0x7f484c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccf76365d765e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:35.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.792+0000 7f4862000640 1 -- 172.21.15.7:0/2957321834 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c037070 con 0x7f486406cac0 2024-04-25T11:11:35.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.792+0000 7f4862000640 1 -- 172.21.15.7:0/2957321834 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484c032440 con 0x7f486406cac0 2024-04-25T11:11:35.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.792+0000 7f4862000640 1 -- 172.21.15.7:0/2957321834 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c032810 con 0x7f486406cac0 2024-04-25T11:11:35.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.792+0000 7f4863e00640 1 -- 172.21.15.7:0/2957321834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f486406cac0 msgr2=0x7f486406ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:35.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.792+0000 7f4863e00640 1 --2- 172.21.15.7:0/2957321834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f486406cac0 0x7f486406ced0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f484c0097d0 tx=0x7f484c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.793+0000 7f4863e00640 1 -- 172.21.15.7:0/2957321834 shutdown_connections 2024-04-25T11:11:35.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.793+0000 7f4863e00640 1 --2- 172.21.15.7:0/2957321834 >> v2:172.21.15.87:3300/0 conn(0x7f486415f630 0x7f486415fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.793+0000 7f4863e00640 1 --2- 172.21.15.7:0/2957321834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f486406cac0 0x7f486406ced0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.793+0000 7f4863e00640 1 -- 172.21.15.7:0/2957321834 >> 172.21.15.7:0/2957321834 conn(0x7f48640d2880 msgr2=0x7f48640d2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:35.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.793+0000 7f4863e00640 1 -- 172.21.15.7:0/2957321834 shutdown_connections 2024-04-25T11:11:35.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.793+0000 7f4863e00640 1 -- 172.21.15.7:0/2957321834 wait complete. 2024-04-25T11:11:35.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.795+0000 7f4863e00640 1 Processor -- start 2024-04-25T11:11:35.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.795+0000 7f4863e00640 1 -- start start 2024-04-25T11:11:35.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.797+0000 7f4863e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f486406cac0 0x7f48641e8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.797+0000 7f4863e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48640d12a0 0x7f486414f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.797+0000 7f4863e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f486415f630 0x7f486414fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.797+0000 7f4863e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48640d3e70 con 0x7f486415f630 2024-04-25T11:11:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.797+0000 7f4863e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48640b9470 con 0x7f48640d12a0 2024-04-25T11:11:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.797+0000 7f4863e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48640d4a20 con 0x7f486406cac0 2024-04-25T11:11:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.797+0000 7f4863400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f486415f630 0x7f486414fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.797+0000 7f4863400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f486415f630 0x7f486414fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46366/0 (socket says 172.21.15.7:46366) 2024-04-25T11:11:35.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.797+0000 7f4863400640 1 -- 172.21.15.7:0/4085196858 learned_addr learned my addr 172.21.15.7:0/4085196858 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.797+0000 7f485a000640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48640d12a0 0x7f486414f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.798+0000 7f4862a00640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f486406cac0 0x7f48641e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.798+0000 7f4863400640 1 -- 172.21.15.7:0/4085196858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f486406cac0 msgr2=0x7f48641e8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.798+0000 7f4863400640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f486406cac0 0x7f48641e8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.798+0000 7f4863400640 1 -- 172.21.15.7:0/4085196858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48640d12a0 msgr2=0x7f486414f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:35.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.798+0000 7f4863400640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48640d12a0 0x7f486414f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.798+0000 7f4863400640 1 -- 172.21.15.7:0/4085196858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4864161c10 con 0x7f486415f630 2024-04-25T11:11:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.798+0000 7f485a000640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48640d12a0 0x7f486414f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:11:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.798+0000 7f4863400640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f486415f630 0x7f486414fb20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f485c00efc0 tx=0x7f485c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.798+0000 7f485b400640 1 -- 172.21.15.7:0/4085196858 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c018020 con 0x7f486415f630 2024-04-25T11:11:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.799+0000 7f485b400640 1 -- 172.21.15.7:0/4085196858 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485c014480 con 0x7f486415f630 2024-04-25T11:11:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.799+0000 7f485b400640 1 -- 172.21.15.7:0/4085196858 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c009e60 con 0x7f486415f630 2024-04-25T11:11:35.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.799+0000 7f4863e00640 1 -- 172.21.15.7:0/4085196858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4864150060 con 0x7f486415f630 2024-04-25T11:11:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.799+0000 7f4863e00640 1 -- 172.21.15.7:0/4085196858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4864162160 con 0x7f486415f630 2024-04-25T11:11:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.800+0000 7f4863e00640 1 -- 172.21.15.7:0/4085196858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4864050740 con 0x7f486415f630 2024-04-25T11:11:35.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.801+0000 7f485b400640 1 -- 172.21.15.7:0/4085196858 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f485c005d10 con 0x7f486415f630 2024-04-25T11:11:35.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.802+0000 7f485b400640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48400744e0 0x7f48400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.802+0000 7f485b400640 1 -- 172.21.15.7:0/4085196858 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f485c0963f0 con 0x7f486415f630 2024-04-25T11:11:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.803+0000 7f4862a00640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48400744e0 0x7f48400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:35.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.803+0000 7f4862a00640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48400744e0 0x7f48400769f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4854000f60 tx=0x7f4854007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:35.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:35.807+0000 7f485b400640 1 -- 172.21.15.7:0/4085196858 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f485c022030 con 0x7f486415f630 2024-04-25T11:11:36.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.011+0000 7f4863e00640 1 -- 172.21.15.7:0/4085196858 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f48640b8540 con 0x7f486415f630 2024-04-25T11:11:36.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.012+0000 7f485b400640 1 -- 172.21.15.7:0/4085196858 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f485c010040 con 0x7f486415f630 2024-04-25T11:11:36.012 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 -- 172.21.15.7:0/4085196858 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48400744e0 msgr2=0x7f48400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48400744e0 0x7f48400769f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4854000f60 tx=0x7f4854007c00 comp rx=0 tx=0).stop 2024-04-25T11:11:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 -- 172.21.15.7:0/4085196858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f486415f630 msgr2=0x7f486414fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f486415f630 0x7f486414fb20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f485c00efc0 tx=0x7f485c00c490 comp rx=0 tx=0).stop 2024-04-25T11:11:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 -- 172.21.15.7:0/4085196858 shutdown_connections 2024-04-25T11:11:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48400744e0 0x7f48400769f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f486415f630 0x7f486414fb20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48640d12a0 0x7f486414f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 --2- 172.21.15.7:0/4085196858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f486406cac0 0x7f48641e8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 -- 172.21.15.7:0/4085196858 >> 172.21.15.7:0/4085196858 conn(0x7f48640d2880 msgr2=0x7f48640cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:36.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 -- 172.21.15.7:0/4085196858 shutdown_connections 2024-04-25T11:11:36.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.015+0000 7f4863e00640 1 -- 172.21.15.7:0/4085196858 wait complete. 2024-04-25T11:11:36.042 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.320+0000 7ff77da00640 1 Processor -- start 2024-04-25T11:11:36.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.321+0000 7ff77da00640 1 -- start start 2024-04-25T11:11:36.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.323+0000 7ff77da00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff7781583c0 0x7ff7780cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.323+0000 7ff77da00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff77806eea0 0x7ff77806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.323+0000 7ff77da00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7780ce570 con 0x7ff7780cfb50 2024-04-25T11:11:36.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.323+0000 7ff77da00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff77806f7f0 con 0x7ff77806eea0 2024-04-25T11:11:36.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.323+0000 7ff77da00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff77815f190 con 0x7ff7781583c0 2024-04-25T11:11:36.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.323+0000 7ff76fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff7780cfb50 0x7ff7780d0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58960/0 (socket says 172.21.15.7:58960) 2024-04-25T11:11:36.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.324+0000 7ff76fe00640 1 -- 172.21.15.7:0/1223494686 learned_addr learned my addr 172.21.15.7:0/1223494686 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:36.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.324+0000 7ff77d000640 1 --2- 172.21.15.7:0/1223494686 >> v2:172.21.15.7:3300/0 conn(0x7ff77806eea0 0x7ff77806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:36.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.324+0000 7ff777e00640 1 --2- 172.21.15.7:0/1223494686 >> v2:172.21.15.87:3300/0 conn(0x7ff7781583c0 0x7ff7780cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:36.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.324+0000 7ff777400640 1 -- 172.21.15.7:0/1223494686 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 975143073 0 0) 0x7ff7780ce570 con 0x7ff7780cfb50 2024-04-25T11:11:36.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.324+0000 7ff777400640 1 -- 172.21.15.7:0/1223494686 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff75c003520 con 0x7ff7780cfb50 2024-04-25T11:11:36.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.324+0000 7ff77d000640 1 -- 172.21.15.7:0/1223494686 >> v2:172.21.15.87:3300/0 conn(0x7ff7781583c0 msgr2=0x7ff7780cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:36.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.324+0000 7ff77d000640 1 --2- 172.21.15.7:0/1223494686 >> v2:172.21.15.87:3300/0 conn(0x7ff7781583c0 0x7ff7780cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.324+0000 7ff77d000640 1 -- 172.21.15.7:0/1223494686 >> v1:172.21.15.7:6789/0 conn(0x7ff7780cfb50 legacy=0x7ff7780d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.324+0000 7ff77d000640 1 -- 172.21.15.7:0/1223494686 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7780ce850 con 0x7ff77806eea0 2024-04-25T11:11:36.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.325+0000 7ff77d000640 1 --2- 172.21.15.7:0/1223494686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff77806eea0 0x7ff77806f2b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff778072330 tx=0x7ff77002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b692d18be7d3f2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:36.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.325+0000 7ff777400640 1 -- 172.21.15.7:0/1223494686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770007740 con 0x7ff77806eea0 2024-04-25T11:11:36.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.326+0000 7ff777400640 1 -- 172.21.15.7:0/1223494686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7700078e0 con 0x7ff77806eea0 2024-04-25T11:11:36.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.326+0000 7ff777400640 1 -- 172.21.15.7:0/1223494686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770007cb0 con 0x7ff77806eea0 2024-04-25T11:11:36.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.326+0000 7ff77da00640 1 -- 172.21.15.7:0/1223494686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff77806eea0 msgr2=0x7ff77806f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.326+0000 7ff77da00640 1 --2- 172.21.15.7:0/1223494686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff77806eea0 0x7ff77806f2b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff778072330 tx=0x7ff77002a5f0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.327+0000 7ff77da00640 1 -- 172.21.15.7:0/1223494686 shutdown_connections 2024-04-25T11:11:36.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.327+0000 7ff77da00640 1 --2- 172.21.15.7:0/1223494686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff77806eea0 0x7ff77806f2b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.327+0000 7ff77da00640 1 --2- 172.21.15.7:0/1223494686 >> v2:172.21.15.87:3300/0 conn(0x7ff7781583c0 0x7ff7780cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.327+0000 7ff77da00640 1 -- 172.21.15.7:0/1223494686 >> 172.21.15.7:0/1223494686 conn(0x7ff7780740f0 msgr2=0x7ff77806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:36.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.327+0000 7ff77da00640 1 -- 172.21.15.7:0/1223494686 shutdown_connections 2024-04-25T11:11:36.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.327+0000 7ff77da00640 1 -- 172.21.15.7:0/1223494686 wait complete. 2024-04-25T11:11:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.329+0000 7ff77da00640 1 Processor -- start 2024-04-25T11:11:36.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.329+0000 7ff77da00640 1 -- start start 2024-04-25T11:11:36.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.330+0000 7ff77da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff77806eea0 0x7ff77813add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.330+0000 7ff77da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7780cfb50 0x7ff7781e7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff77da00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7781583c0 0x7ff77815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff77da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff7780cf0f0 con 0x7ff77806eea0 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff77da00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff7780cee10 con 0x7ff7780cfb50 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff77da00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff7780d0b90 con 0x7ff7781583c0 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff777e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff77806eea0 0x7ff77813add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff777e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff77806eea0 0x7ff77813add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46382/0 (socket says 172.21.15.7:46382) 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff777e00640 1 -- 172.21.15.7:0/2474169695 learned_addr learned my addr 172.21.15.7:0/2474169695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff777e00640 1 -- 172.21.15.7:0/2474169695 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7781583c0 msgr2=0x7ff77815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff777e00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7781583c0 0x7ff77815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff777e00640 1 -- 172.21.15.7:0/2474169695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7780cfb50 msgr2=0x7ff7781e7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff777e00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7780cfb50 0x7ff7781e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.331+0000 7ff777e00640 1 -- 172.21.15.7:0/2474169695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7781f6440 con 0x7ff77806eea0 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.332+0000 7ff777e00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff77806eea0 0x7ff77813add0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff76800d5a0 tx=0x7ff76800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.332+0000 7ff774c00640 1 -- 172.21.15.7:0/2474169695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768019070 con 0x7ff77806eea0 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.332+0000 7ff774c00640 1 -- 172.21.15.7:0/2474169695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff768004240 con 0x7ff77806eea0 2024-04-25T11:11:36.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.332+0000 7ff774c00640 1 -- 172.21.15.7:0/2474169695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768003610 con 0x7ff77806eea0 2024-04-25T11:11:36.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.332+0000 7ff77da00640 1 -- 172.21.15.7:0/2474169695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7781f6720 con 0x7ff77806eea0 2024-04-25T11:11:36.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.332+0000 7ff77da00640 1 -- 172.21.15.7:0/2474169695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7781f6ce0 con 0x7ff77806eea0 2024-04-25T11:11:36.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.333+0000 7ff77da00640 1 -- 172.21.15.7:0/2474169695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff778050740 con 0x7ff77806eea0 2024-04-25T11:11:36.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.334+0000 7ff774c00640 1 -- 172.21.15.7:0/2474169695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff76801c070 con 0x7ff77806eea0 2024-04-25T11:11:36.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.335+0000 7ff774c00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7540744b0 0x7ff7540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.335+0000 7ff774c00640 1 -- 172.21.15.7:0/2474169695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff768094e50 con 0x7ff77806eea0 2024-04-25T11:11:36.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.336+0000 7ff76fe00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7540744b0 0x7ff7540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:36.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.336+0000 7ff76fe00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7540744b0 0x7ff7540769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff760000f60 tx=0x7ff7600078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:36.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.340+0000 7ff774c00640 1 -- 172.21.15.7:0/2474169695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff76805ec10 con 0x7ff77806eea0 2024-04-25T11:11:36.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.543+0000 7ff77da00640 1 -- 172.21.15.7:0/2474169695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff778071110 con 0x7ff77806eea0 2024-04-25T11:11:36.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.544+0000 7ff774c00640 1 -- 172.21.15.7:0/2474169695 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7ff768062f10 con 0x7ff77806eea0 2024-04-25T11:11:36.545 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:36.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.547+0000 7ff77da00640 1 -- 172.21.15.7:0/2474169695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7540744b0 msgr2=0x7ff7540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.547+0000 7ff77da00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7540744b0 0x7ff7540769c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff760000f60 tx=0x7ff7600078f0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.547+0000 7ff77da00640 1 -- 172.21.15.7:0/2474169695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff77806eea0 msgr2=0x7ff77813add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.547+0000 7ff77da00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff77806eea0 0x7ff77813add0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff76800d5a0 tx=0x7ff76800da60 comp rx=0 tx=0).stop 2024-04-25T11:11:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.547+0000 7ff77da00640 1 -- 172.21.15.7:0/2474169695 shutdown_connections 2024-04-25T11:11:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.547+0000 7ff77da00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7540744b0 0x7ff7540769c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.547+0000 7ff77da00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7781583c0 0x7ff77815ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.547+0000 7ff77da00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7780cfb50 0x7ff7781e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.547+0000 7ff77da00640 1 --2- 172.21.15.7:0/2474169695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff77806eea0 0x7ff77813add0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.547+0000 7ff77da00640 1 -- 172.21.15.7:0/2474169695 >> 172.21.15.7:0/2474169695 conn(0x7ff7780740f0 msgr2=0x7ff77815f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.547+0000 7ff77da00640 1 -- 172.21.15.7:0/2474169695 shutdown_connections 2024-04-25T11:11:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.548+0000 7ff77da00640 1 -- 172.21.15.7:0/2474169695 wait complete. 2024-04-25T11:11:36.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:36.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.851+0000 7fdcda600640 1 Processor -- start 2024-04-25T11:11:36.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.851+0000 7fdcda600640 1 -- start start 2024-04-25T11:11:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.854+0000 7fdcda600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd40b0470 0x7fdcd406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.854+0000 7fdcda600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdcd406f8e0 0x7fdcd40aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.854+0000 7fdcda600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcd415e3c0 con 0x7fdcd415dfb0 2024-04-25T11:11:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.854+0000 7fdcda600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fdcd40cf800 con 0x7fdcd406f8e0 2024-04-25T11:11:36.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.854+0000 7fdcda600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd40d1250 con 0x7fdcd40b0470 2024-04-25T11:11:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.854+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd40b0470 0x7fdcd406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.854+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd40b0470 0x7fdcd406f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43550/0 (socket says 172.21.15.7:43550) 2024-04-25T11:11:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.854+0000 7fdcd2a00640 1 -- 172.21.15.7:0/897369184 learned_addr learned my addr 172.21.15.7:0/897369184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.855+0000 7fdcd3e00640 1 --2- 172.21.15.7:0/897369184 >> v2:172.21.15.87:3300/0 conn(0x7fdcd406f8e0 0x7fdcd40aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.855+0000 7fdcd2a00640 1 -- 172.21.15.7:0/897369184 >> v2:172.21.15.87:3300/0 conn(0x7fdcd406f8e0 msgr2=0x7fdcd40aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.855+0000 7fdcd2a00640 1 --2- 172.21.15.7:0/897369184 >> v2:172.21.15.87:3300/0 conn(0x7fdcd406f8e0 0x7fdcd40aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.855+0000 7fdcd2a00640 1 -- 172.21.15.7:0/897369184 >> v1:172.21.15.87:6789/0 conn(0x7fdcd415dfb0 legacy=0x7fdcd40d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.855+0000 7fdcd2a00640 1 -- 172.21.15.7:0/897369184 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd40cfaa0 con 0x7fdcd40b0470 2024-04-25T11:11:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.857+0000 7fdcd2a00640 1 --2- 172.21.15.7:0/897369184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd40b0470 0x7fdcd406f3a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009870 tx=0x7fdcc002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6a0860e8f0387ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.857+0000 7fdcd2000640 1 -- 172.21.15.7:0/897369184 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0037070 con 0x7fdcd40b0470 2024-04-25T11:11:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.857+0000 7fdcd2000640 1 -- 172.21.15.7:0/897369184 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc0032440 con 0x7fdcd40b0470 2024-04-25T11:11:36.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.857+0000 7fdcd2000640 1 -- 172.21.15.7:0/897369184 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0032810 con 0x7fdcd40b0470 2024-04-25T11:11:36.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.858+0000 7fdcda600640 1 -- 172.21.15.7:0/897369184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd40b0470 msgr2=0x7fdcd406f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.858+0000 7fdcda600640 1 --2- 172.21.15.7:0/897369184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd40b0470 0x7fdcd406f3a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009870 tx=0x7fdcc002a250 comp rx=0 tx=0).stop 2024-04-25T11:11:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.858+0000 7fdcda600640 1 -- 172.21.15.7:0/897369184 shutdown_connections 2024-04-25T11:11:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.858+0000 7fdcda600640 1 --2- 172.21.15.7:0/897369184 >> v2:172.21.15.87:3300/0 conn(0x7fdcd406f8e0 0x7fdcd40aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.858+0000 7fdcda600640 1 --2- 172.21.15.7:0/897369184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd40b0470 0x7fdcd406f3a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.858+0000 7fdcda600640 1 -- 172.21.15.7:0/897369184 >> 172.21.15.7:0/897369184 conn(0x7fdcd40742c0 msgr2=0x7fdcd406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.858+0000 7fdcda600640 1 -- 172.21.15.7:0/897369184 shutdown_connections 2024-04-25T11:11:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.858+0000 7fdcda600640 1 -- 172.21.15.7:0/897369184 wait complete. 2024-04-25T11:11:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.862+0000 7fdcda600640 1 Processor -- start 2024-04-25T11:11:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.862+0000 7fdcda600640 1 -- start start 2024-04-25T11:11:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.864+0000 7fdcda600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcd406f8e0 0x7fdcd41580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.864+0000 7fdcda600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd40b0470 0x7fdcd41f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.864+0000 7fdcda600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd415dfb0 0x7fdcd41f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.864+0000 7fdcda600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd40d0340 con 0x7fdcd40b0470 2024-04-25T11:11:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.864+0000 7fdcda600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd40d0060 con 0x7fdcd406f8e0 2024-04-25T11:11:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.864+0000 7fdcda600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd40d0620 con 0x7fdcd415dfb0 2024-04-25T11:11:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.864+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd40b0470 0x7fdcd41f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.865+0000 7fdcd2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd40b0470 0x7fdcd41f7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46394/0 (socket says 172.21.15.7:46394) 2024-04-25T11:11:36.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.865+0000 7fdcd2a00640 1 -- 172.21.15.7:0/246910812 learned_addr learned my addr 172.21.15.7:0/246910812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.865+0000 7fdcd3400640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcd406f8e0 0x7fdcd41580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.865+0000 7fdcd3e00640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd415dfb0 0x7fdcd41f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.865+0000 7fdcd2a00640 1 -- 172.21.15.7:0/246910812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd415dfb0 msgr2=0x7fdcd41f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.865+0000 7fdcd2a00640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd415dfb0 0x7fdcd41f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.865+0000 7fdcd2a00640 1 -- 172.21.15.7:0/246910812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcd406f8e0 msgr2=0x7fdcd41580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.865+0000 7fdcd2a00640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcd406f8e0 0x7fdcd41580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.865+0000 7fdcd2a00640 1 -- 172.21.15.7:0/246910812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd4141ad0 con 0x7fdcd40b0470 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.865+0000 7fdcd3e00640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd415dfb0 0x7fdcd41f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.866+0000 7fdcd2a00640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd40b0470 0x7fdcd41f7470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009990 tx=0x7fdcc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.866+0000 7fdccb400640 1 -- 172.21.15.7:0/246910812 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0037040 con 0x7fdcd40b0470 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.866+0000 7fdccb400640 1 -- 172.21.15.7:0/246910812 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcc0042070 con 0x7fdcd40b0470 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.866+0000 7fdccb400640 1 -- 172.21.15.7:0/246910812 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0030940 con 0x7fdcd40b0470 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.866+0000 7fdcda600640 1 -- 172.21.15.7:0/246910812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd4158c90 con 0x7fdcd40b0470 2024-04-25T11:11:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.866+0000 7fdcda600640 1 -- 172.21.15.7:0/246910812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd4159250 con 0x7fdcd40b0470 2024-04-25T11:11:36.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.867+0000 7fdcda600640 1 -- 172.21.15.7:0/246910812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd4158a00 con 0x7fdcd40b0470 2024-04-25T11:11:36.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.868+0000 7fdccb400640 1 -- 172.21.15.7:0/246910812 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdcc0030ae0 con 0x7fdcd40b0470 2024-04-25T11:11:36.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.869+0000 7fdccb400640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdcb40744b0 0x7fdcb40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:36.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.870+0000 7fdccb400640 1 -- 172.21.15.7:0/246910812 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fdcc00b5ec0 con 0x7fdcd40b0470 2024-04-25T11:11:36.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.870+0000 7fdcd3400640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdcb40744b0 0x7fdcb40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:36.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.871+0000 7fdcd3400640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdcb40744b0 0x7fdcb40769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4007fd0 tx=0x7fdcc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:36.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:36.875+0000 7fdccb400640 1 -- 172.21.15.7:0/246910812 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdcd4141ad0 con 0x7fdcd40b0470 2024-04-25T11:11:37.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.083+0000 7fdcda600640 1 -- 172.21.15.7:0/246910812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdcd4050740 con 0x7fdcd40b0470 2024-04-25T11:11:37.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.084+0000 7fdccb400640 1 -- 172.21.15.7:0/246910812 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fdcc0080240 con 0x7fdcd40b0470 2024-04-25T11:11:37.085 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.086+0000 7fdcda600640 1 -- 172.21.15.7:0/246910812 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdcb40744b0 msgr2=0x7fdcb40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.086+0000 7fdcda600640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdcb40744b0 0x7fdcb40769c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4007fd0 tx=0x7fdcc4009040 comp rx=0 tx=0).stop 2024-04-25T11:11:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.087+0000 7fdcda600640 1 -- 172.21.15.7:0/246910812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd40b0470 msgr2=0x7fdcd41f7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.087+0000 7fdcda600640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd40b0470 0x7fdcd41f7470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009990 tx=0x7fdcc00046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.087+0000 7fdcda600640 1 -- 172.21.15.7:0/246910812 shutdown_connections 2024-04-25T11:11:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.087+0000 7fdcda600640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdcb40744b0 0x7fdcb40769c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.087+0000 7fdcda600640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcd415dfb0 0x7fdcd41f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.087+0000 7fdcda600640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcd40b0470 0x7fdcd41f7470 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.087+0000 7fdcda600640 1 --2- 172.21.15.7:0/246910812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcd406f8e0 0x7fdcd41580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.087+0000 7fdcda600640 1 -- 172.21.15.7:0/246910812 >> 172.21.15.7:0/246910812 conn(0x7fdcd40742c0 msgr2=0x7fdcd40b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.087+0000 7fdcda600640 1 -- 172.21.15.7:0/246910812 shutdown_connections 2024-04-25T11:11:37.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.087+0000 7fdcda600640 1 -- 172.21.15.7:0/246910812 wait complete. 2024-04-25T11:11:37.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:37.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.401+0000 7f8bef200640 1 Processor -- start 2024-04-25T11:11:37.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.401+0000 7f8bef200640 1 -- start start 2024-04-25T11:11:37.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.403+0000 7f8bef200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be80d1f80 0x7f8be815dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.403+0000 7f8bef200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8be8161230 0x7f8be80b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.404+0000 7f8bef200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be80b2400 con 0x7f8be80b1ff0 2024-04-25T11:11:37.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.404+0000 7f8bef200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8be806c570 con 0x7f8be8161230 2024-04-25T11:11:37.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.404+0000 7f8bef200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8be80b29a0 con 0x7f8be80d1f80 2024-04-25T11:11:37.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.404+0000 7f8bed200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8be80b1ff0 0x7f8be8163da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58976/0 (socket says 172.21.15.7:58976) 2024-04-25T11:11:37.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.404+0000 7f8bed200640 1 -- 172.21.15.7:0/3615976789 learned_addr learned my addr 172.21.15.7:0/3615976789 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:37.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.404+0000 7f8be7400640 1 --2- 172.21.15.7:0/3615976789 >> v2:172.21.15.7:3300/0 conn(0x7f8be8161230 0x7f8be80b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:37.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.404+0000 7f8be7e00640 1 --2- 172.21.15.7:0/3615976789 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be80d1f80 0x7f8be815dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.405+0000 7f8be7400640 1 -- 172.21.15.7:0/3615976789 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be80d1f80 msgr2=0x7f8be815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.405+0000 7f8be7400640 1 --2- 172.21.15.7:0/3615976789 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be80d1f80 0x7f8be815dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.405+0000 7f8be7400640 1 -- 172.21.15.7:0/3615976789 >> v1:172.21.15.7:6789/0 conn(0x7f8be80b1ff0 legacy=0x7f8be8163da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.405+0000 7f8be7400640 1 -- 172.21.15.7:0/3615976789 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be80d2540 con 0x7f8be8161230 2024-04-25T11:11:37.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.406+0000 7f8be7400640 1 --2- 172.21.15.7:0/3615976789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be8161230 0x7f8be80b1ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009d90 tx=0x7f8bd402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c80723077a474b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.406+0000 7f8be6a00640 1 -- 172.21.15.7:0/3615976789 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4031600 con 0x7f8be8161230 2024-04-25T11:11:37.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.406+0000 7f8be6a00640 1 -- 172.21.15.7:0/3615976789 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd40317a0 con 0x7f8be8161230 2024-04-25T11:11:37.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.406+0000 7f8be6a00640 1 -- 172.21.15.7:0/3615976789 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4031b70 con 0x7f8be8161230 2024-04-25T11:11:37.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.407+0000 7f8bef200640 1 -- 172.21.15.7:0/3615976789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be8161230 msgr2=0x7f8be80b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:37.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.407+0000 7f8bef200640 1 --2- 172.21.15.7:0/3615976789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be8161230 0x7f8be80b1ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009d90 tx=0x7f8bd402a5d0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.407+0000 7f8bef200640 1 -- 172.21.15.7:0/3615976789 shutdown_connections 2024-04-25T11:11:37.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.407+0000 7f8bef200640 1 --2- 172.21.15.7:0/3615976789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be8161230 0x7f8be80b1ab0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.407+0000 7f8bef200640 1 --2- 172.21.15.7:0/3615976789 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be80d1f80 0x7f8be815dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.407+0000 7f8bef200640 1 -- 172.21.15.7:0/3615976789 >> 172.21.15.7:0/3615976789 conn(0x7f8be8073d30 msgr2=0x7f8be809b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:37.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.407+0000 7f8bef200640 1 -- 172.21.15.7:0/3615976789 shutdown_connections 2024-04-25T11:11:37.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.407+0000 7f8bef200640 1 -- 172.21.15.7:0/3615976789 wait complete. 2024-04-25T11:11:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.409+0000 7f8bef200640 1 Processor -- start 2024-04-25T11:11:37.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.410+0000 7f8bef200640 1 -- start start 2024-04-25T11:11:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.411+0000 7f8bef200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be80b1ff0 0x7f8be8148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.411+0000 7f8bef200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be80d1f80 0x7f8be809edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.411+0000 7f8bef200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be8161230 0x7f8be815af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.411+0000 7f8bef200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8be80d2de0 con 0x7f8be80b1ff0 2024-04-25T11:11:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.411+0000 7f8bef200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8be80d2b00 con 0x7f8be80d1f80 2024-04-25T11:11:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.411+0000 7f8bef200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8be8161640 con 0x7f8be8161230 2024-04-25T11:11:37.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.411+0000 7f8be7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be80b1ff0 0x7f8be8148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.411+0000 7f8be7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be80b1ff0 0x7f8be8148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46406/0 (socket says 172.21.15.7:46406) 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.411+0000 7f8be7e00640 1 -- 172.21.15.7:0/3082263345 learned_addr learned my addr 172.21.15.7:0/3082263345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.412+0000 7f8be7e00640 1 -- 172.21.15.7:0/3082263345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be8161230 msgr2=0x7f8be815af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.412+0000 7f8bed200640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be8161230 0x7f8be815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.412+0000 7f8be7400640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be80d1f80 0x7f8be809edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.412+0000 7f8be7e00640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be8161230 0x7f8be815af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.412+0000 7f8be7e00640 1 -- 172.21.15.7:0/3082263345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be80d1f80 msgr2=0x7f8be809edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.412+0000 7f8be7e00640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be80d1f80 0x7f8be809edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.412+0000 7f8be7e00640 1 -- 172.21.15.7:0/3082263345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be809c230 con 0x7f8be80b1ff0 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.412+0000 7f8be7e00640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be80b1ff0 0x7f8be8148a70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800d5a0 tx=0x7f8bd800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.413+0000 7f8bdfe00640 1 -- 172.21.15.7:0/3082263345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8019070 con 0x7f8be80b1ff0 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.413+0000 7f8bdfe00640 1 -- 172.21.15.7:0/3082263345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd8004240 con 0x7f8be80b1ff0 2024-04-25T11:11:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.413+0000 7f8bdfe00640 1 -- 172.21.15.7:0/3082263345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8003610 con 0x7f8be80b1ff0 2024-04-25T11:11:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.413+0000 7f8bef200640 1 -- 172.21.15.7:0/3082263345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be8148fb0 con 0x7f8be80b1ff0 2024-04-25T11:11:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.413+0000 7f8bef200640 1 -- 172.21.15.7:0/3082263345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be809fb50 con 0x7f8be80b1ff0 2024-04-25T11:11:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.414+0000 7f8bef200640 1 -- 172.21.15.7:0/3082263345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be8050740 con 0x7f8be80b1ff0 2024-04-25T11:11:37.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.415+0000 7f8bdfe00640 1 -- 172.21.15.7:0/3082263345 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8bd801c070 con 0x7f8be80b1ff0 2024-04-25T11:11:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.416+0000 7f8bdfe00640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc80744b0 0x7f8bc80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.416+0000 7f8bdfe00640 1 -- 172.21.15.7:0/3082263345 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f8bd8094e90 con 0x7f8be80b1ff0 2024-04-25T11:11:37.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.416+0000 7f8be7400640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc80744b0 0x7f8bc80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:37.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.417+0000 7f8be7400640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc80744b0 0x7f8bc80769c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4034000 tx=0x7f8bd4033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:37.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.421+0000 7f8bdfe00640 1 -- 172.21.15.7:0/3082263345 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8bd805ebc0 con 0x7f8be80b1ff0 2024-04-25T11:11:37.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.623+0000 7f8bef200640 1 -- 172.21.15.7:0/3082263345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8be806db60 con 0x7f8be80b1ff0 2024-04-25T11:11:37.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.624+0000 7f8bdfe00640 1 -- 172.21.15.7:0/3082263345 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f8bd80960a0 con 0x7f8be80b1ff0 2024-04-25T11:11:37.625 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.626+0000 7f8bef200640 1 -- 172.21.15.7:0/3082263345 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc80744b0 msgr2=0x7f8bc80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.626+0000 7f8bef200640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc80744b0 0x7f8bc80769c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4034000 tx=0x7f8bd4033040 comp rx=0 tx=0).stop 2024-04-25T11:11:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.626+0000 7f8bef200640 1 -- 172.21.15.7:0/3082263345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be80b1ff0 msgr2=0x7f8be8148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.626+0000 7f8bef200640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be80b1ff0 0x7f8be8148a70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800d5a0 tx=0x7f8bd800da60 comp rx=0 tx=0).stop 2024-04-25T11:11:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.626+0000 7f8bef200640 1 -- 172.21.15.7:0/3082263345 shutdown_connections 2024-04-25T11:11:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.626+0000 7f8bef200640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8bc80744b0 0x7f8bc80769c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.626+0000 7f8bef200640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be8161230 0x7f8be815af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.626+0000 7f8bef200640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be80d1f80 0x7f8be809edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.626+0000 7f8bef200640 1 --2- 172.21.15.7:0/3082263345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8be80b1ff0 0x7f8be8148a70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.627+0000 7f8bef200640 1 -- 172.21.15.7:0/3082263345 >> 172.21.15.7:0/3082263345 conn(0x7f8be8073d30 msgr2=0x7f8be80684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:37.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.627+0000 7f8bef200640 1 -- 172.21.15.7:0/3082263345 shutdown_connections 2024-04-25T11:11:37.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.627+0000 7f8bef200640 1 -- 172.21.15.7:0/3082263345 wait complete. 2024-04-25T11:11:37.674 INFO:tasks.cephfs_test_runner:Starting test: test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) Mostly readdir acquires caps faster than the mds recalls, so the cap 2024-04-25T11:11:37.674 INFO:tasks.cephfs_test_runner:test_cap_acquisition_throttle_readdir (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-25T11:11:37.675 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:37.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.954+0000 7f48c5600640 1 Processor -- start 2024-04-25T11:11:37.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.954+0000 7f48c5600640 1 -- start start 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.956+0000 7f48c5600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f48c009c110 0x7f48c00cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.956+0000 7f48c5600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f48c00d31a0 0x7f48c00ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.956+0000 7f48c5600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48c00d1cf0 con 0x7f48c00d2d90 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.956+0000 7f48c5600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f48c00d1fd0 con 0x7f48c009c110 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.956+0000 7f48c5600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f48c00d22b0 con 0x7f48c00d31a0 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.956+0000 7f48bf400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f48c009c110 0x7f48c00cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.956+0000 7f48bf400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f48c009c110 0x7f48c00cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46416/0 (socket says 172.21.15.7:46416) 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.956+0000 7f48bf400640 1 -- 172.21.15.7:0/364851 learned_addr learned my addr 172.21.15.7:0/364851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.956+0000 7f48bf400640 1 -- 172.21.15.7:0/364851 >> v2:172.21.15.87:3300/0 conn(0x7f48c00d31a0 msgr2=0x7f48c00ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.956+0000 7f48bf400640 1 --2- 172.21.15.7:0/364851 >> v2:172.21.15.87:3300/0 conn(0x7f48c00d31a0 0x7f48c00ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.957+0000 7f48bf400640 1 -- 172.21.15.7:0/364851 >> v1:172.21.15.87:6789/0 conn(0x7f48c00d2d90 legacy=0x7f48c009ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:37.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.957+0000 7f48bf400640 1 -- 172.21.15.7:0/364851 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c00d2590 con 0x7f48c009c110 2024-04-25T11:11:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.958+0000 7f48bf400640 1 --2- 172.21.15.7:0/364851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c009c110 0x7f48c00cddb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f48c009cb90 tx=0x7f48a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f80941d86f69d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.958+0000 7f48bea00640 1 -- 172.21.15.7:0/364851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8037070 con 0x7f48c009c110 2024-04-25T11:11:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.958+0000 7f48bea00640 1 -- 172.21.15.7:0/364851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a8032440 con 0x7f48c009c110 2024-04-25T11:11:37.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.958+0000 7f48bea00640 1 -- 172.21.15.7:0/364851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8032810 con 0x7f48c009c110 2024-04-25T11:11:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.959+0000 7f48c5600640 1 -- 172.21.15.7:0/364851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c009c110 msgr2=0x7f48c00cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.959+0000 7f48c5600640 1 --2- 172.21.15.7:0/364851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c009c110 0x7f48c00cddb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f48c009cb90 tx=0x7f48a802a230 comp rx=0 tx=0).stop 2024-04-25T11:11:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.959+0000 7f48c5600640 1 -- 172.21.15.7:0/364851 shutdown_connections 2024-04-25T11:11:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.959+0000 7f48c5600640 1 --2- 172.21.15.7:0/364851 >> v2:172.21.15.87:3300/0 conn(0x7f48c00d31a0 0x7f48c00ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.959+0000 7f48c5600640 1 --2- 172.21.15.7:0/364851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c009c110 0x7f48c00cddb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.959+0000 7f48c5600640 1 -- 172.21.15.7:0/364851 >> 172.21.15.7:0/364851 conn(0x7f48c00b5250 msgr2=0x7f48c00b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:37.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.959+0000 7f48c5600640 1 -- 172.21.15.7:0/364851 shutdown_connections 2024-04-25T11:11:37.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.959+0000 7f48c5600640 1 -- 172.21.15.7:0/364851 wait complete. 2024-04-25T11:11:37.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.961+0000 7f48c5600640 1 Processor -- start 2024-04-25T11:11:37.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.961+0000 7f48c5600640 1 -- start start 2024-04-25T11:11:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48c5600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48c00d2d90 0x7f48c01f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48c5600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c00d31a0 0x7f48c00b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48c5600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48c0161d50 0x7f48c00b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48c5600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48c00b91e0 con 0x7f48c00d31a0 2024-04-25T11:11:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48c5600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48c00d19a0 con 0x7f48c00d2d90 2024-04-25T11:11:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48c5600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48c006dce0 con 0x7f48c0161d50 2024-04-25T11:11:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48bf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c00d31a0 0x7f48c00b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48bf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c00d31a0 0x7f48c00b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46420/0 (socket says 172.21.15.7:46420) 2024-04-25T11:11:37.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48bf400640 1 -- 172.21.15.7:0/3343546755 learned_addr learned my addr 172.21.15.7:0/3343546755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48c4c00640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48c0161d50 0x7f48c00b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48bf400640 1 -- 172.21.15.7:0/3343546755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48c0161d50 msgr2=0x7f48c00b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48bfe00640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48c00d2d90 0x7f48c01f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48bf400640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48c0161d50 0x7f48c00b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48bf400640 1 -- 172.21.15.7:0/3343546755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48c00d2d90 msgr2=0x7f48c01f6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48bf400640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48c00d2d90 0x7f48c01f6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.963+0000 7f48bf400640 1 -- 172.21.15.7:0/3343546755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c00c7310 con 0x7f48c00d31a0 2024-04-25T11:11:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.964+0000 7f48c4c00640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48c0161d50 0x7f48c00b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.964+0000 7f48bfe00640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48c00d2d90 0x7f48c01f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.964+0000 7f48bf400640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c00d31a0 0x7f48c00b43a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48a802a270 tx=0x7f48a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:37.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.964+0000 7f48b7e00640 1 -- 172.21.15.7:0/3343546755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8041070 con 0x7f48c00d31a0 2024-04-25T11:11:37.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.964+0000 7f48b7e00640 1 -- 172.21.15.7:0/3343546755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48a803c870 con 0x7f48c00d31a0 2024-04-25T11:11:37.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.964+0000 7f48b7e00640 1 -- 172.21.15.7:0/3343546755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8037040 con 0x7f48c00d31a0 2024-04-25T11:11:37.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.964+0000 7f48c5600640 1 -- 172.21.15.7:0/3343546755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c01f7410 con 0x7f48c00d31a0 2024-04-25T11:11:37.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.964+0000 7f48c5600640 1 -- 172.21.15.7:0/3343546755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c00c7580 con 0x7f48c00d31a0 2024-04-25T11:11:37.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.965+0000 7f48c5600640 1 -- 172.21.15.7:0/3343546755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c009c6e0 con 0x7f48c00d31a0 2024-04-25T11:11:37.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.967+0000 7f48b7e00640 1 -- 172.21.15.7:0/3343546755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f48a8044070 con 0x7f48c00d31a0 2024-04-25T11:11:37.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.967+0000 7f48b7e00640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f489c0744e0 0x7f489c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:37.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.968+0000 7f48b7e00640 1 -- 172.21.15.7:0/3343546755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f48a80b61b0 con 0x7f48c00d31a0 2024-04-25T11:11:37.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.968+0000 7f48bfe00640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f489c0744e0 0x7f489c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:37.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.969+0000 7f48bfe00640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f489c0744e0 0x7f489c0769f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48b000a810 tx=0x7f48b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:37.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:37.972+0000 7f48b7e00640 1 -- 172.21.15.7:0/3343546755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48c01f7410 con 0x7f48c00d31a0 2024-04-25T11:11:38.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.167+0000 7f48c5600640 1 -- 172.21.15.7:0/3343546755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48c00c9aa0 con 0x7f48c00d31a0 2024-04-25T11:11:38.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.171+0000 7f48b7e00640 1 -- 172.21.15.7:0/3343546755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7f48a80805c0 con 0x7f48c00d31a0 2024-04-25T11:11:38.173 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:38.173 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 -- 172.21.15.7:0/3343546755 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f489c0744e0 msgr2=0x7f489c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f489c0744e0 0x7f489c0769f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f48b000a810 tx=0x7f48b0008040 comp rx=0 tx=0).stop 2024-04-25T11:11:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 -- 172.21.15.7:0/3343546755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c00d31a0 msgr2=0x7f48c00b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c00d31a0 0x7f48c00b43a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f48a802a270 tx=0x7f48a80046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 -- 172.21.15.7:0/3343546755 shutdown_connections 2024-04-25T11:11:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f489c0744e0 0x7f489c0769f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48c0161d50 0x7f48c00b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c00d31a0 0x7f48c00b43a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 --2- 172.21.15.7:0/3343546755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48c00d2d90 0x7f48c01f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 -- 172.21.15.7:0/3343546755 >> 172.21.15.7:0/3343546755 conn(0x7f48c00b5250 msgr2=0x7f48c00b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:38.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 -- 172.21.15.7:0/3343546755 shutdown_connections 2024-04-25T11:11:38.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.174+0000 7f48c5600640 1 -- 172.21.15.7:0/3343546755 wait complete. 2024-04-25T11:11:38.201 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:38.201 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:38.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.476+0000 7f8af1200640 1 Processor -- start 2024-04-25T11:11:38.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.476+0000 7f8af1200640 1 -- start start 2024-04-25T11:11:38.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.478+0000 7f8af1200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8aec145e40 0x7f8aec146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:38.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.478+0000 7f8af1200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8aec144a80 0x7f8aec0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:38.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.478+0000 7f8af1200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aec0b66b0 con 0x7f8aec0b4870 2024-04-25T11:11:38.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.478+0000 7f8af1200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8aec145b70 con 0x7f8aec145e40 2024-04-25T11:11:38.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.478+0000 7f8af1200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8aec0b58d0 con 0x7f8aec144a80 2024-04-25T11:11:38.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.479+0000 7f8aebe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8aec144a80 0x7f8aec0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:38.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.479+0000 7f8aebe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8aec144a80 0x7f8aec0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43600/0 (socket says 172.21.15.7:43600) 2024-04-25T11:11:38.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.479+0000 7f8aebe00640 1 -- 172.21.15.7:0/1289709060 learned_addr learned my addr 172.21.15.7:0/1289709060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:38.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.480+0000 7f8aeb400640 1 --2- 172.21.15.7:0/1289709060 >> v2:172.21.15.87:3300/0 conn(0x7f8aec145e40 0x7f8aec146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:38.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.480+0000 7f8aebe00640 1 -- 172.21.15.7:0/1289709060 >> v2:172.21.15.87:3300/0 conn(0x7f8aec145e40 msgr2=0x7f8aec146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:38.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.480+0000 7f8aebe00640 1 --2- 172.21.15.7:0/1289709060 >> v2:172.21.15.87:3300/0 conn(0x7f8aec145e40 0x7f8aec146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.480+0000 7f8aebe00640 1 -- 172.21.15.7:0/1289709060 >> v1:172.21.15.87:6789/0 conn(0x7f8aec0b4870 legacy=0x7f8aec0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:38.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.480+0000 7f8aebe00640 1 -- 172.21.15.7:0/1289709060 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec0b5b30 con 0x7f8aec144a80 2024-04-25T11:11:38.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.481+0000 7f8aebe00640 1 --2- 172.21.15.7:0/1289709060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8aec144a80 0x7f8aec0b5390 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009e50 tx=0x7f8ae402a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ea5fad78b58d498 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.482+0000 7f8aea000640 1 -- 172.21.15.7:0/1289709060 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4036070 con 0x7f8aec144a80 2024-04-25T11:11:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.482+0000 7f8aea000640 1 -- 172.21.15.7:0/1289709060 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae4031600 con 0x7f8aec144a80 2024-04-25T11:11:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.482+0000 7f8aea000640 1 -- 172.21.15.7:0/1289709060 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40319d0 con 0x7f8aec144a80 2024-04-25T11:11:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.483+0000 7f8af1200640 1 -- 172.21.15.7:0/1289709060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8aec144a80 msgr2=0x7f8aec0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.483+0000 7f8af1200640 1 --2- 172.21.15.7:0/1289709060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8aec144a80 0x7f8aec0b5390 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009e50 tx=0x7f8ae402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.483+0000 7f8af1200640 1 -- 172.21.15.7:0/1289709060 shutdown_connections 2024-04-25T11:11:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.483+0000 7f8af1200640 1 --2- 172.21.15.7:0/1289709060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8aec144a80 0x7f8aec0b5390 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.483+0000 7f8af1200640 1 --2- 172.21.15.7:0/1289709060 >> v2:172.21.15.87:3300/0 conn(0x7f8aec145e40 0x7f8aec146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.483+0000 7f8af1200640 1 -- 172.21.15.7:0/1289709060 >> 172.21.15.7:0/1289709060 conn(0x7f8aec0740f0 msgr2=0x7f8aec06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.483+0000 7f8af1200640 1 -- 172.21.15.7:0/1289709060 shutdown_connections 2024-04-25T11:11:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.483+0000 7f8af1200640 1 -- 172.21.15.7:0/1289709060 wait complete. 2024-04-25T11:11:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.485+0000 7f8af1200640 1 Processor -- start 2024-04-25T11:11:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.485+0000 7f8af1200640 1 -- start start 2024-04-25T11:11:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.487+0000 7f8af1200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8aec0b4870 0x7f8aec1cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.487+0000 7f8af1200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8aec144a80 0x7f8aec1ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.487+0000 7f8af1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec145e40 0x7f8aec1cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.487+0000 7f8af1200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec0b63d0 con 0x7f8aec145e40 2024-04-25T11:11:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.487+0000 7f8af1200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec0b60f0 con 0x7f8aec0b4870 2024-04-25T11:11:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.487+0000 7f8af1200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8aec06f730 con 0x7f8aec144a80 2024-04-25T11:11:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.487+0000 7f8aebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec145e40 0x7f8aec1cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.487+0000 7f8aebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec145e40 0x7f8aec1cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46422/0 (socket says 172.21.15.7:46422) 2024-04-25T11:11:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.487+0000 7f8aebe00640 1 -- 172.21.15.7:0/2283952193 learned_addr learned my addr 172.21.15.7:0/2283952193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.488+0000 7f8aeb400640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8aec0b4870 0x7f8aec1cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:38.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.488+0000 7f8aebe00640 1 -- 172.21.15.7:0/2283952193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8aec144a80 msgr2=0x7f8aec1ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:38.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.488+0000 7f8aebe00640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8aec144a80 0x7f8aec1ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.488+0000 7f8aebe00640 1 -- 172.21.15.7:0/2283952193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8aec0b4870 msgr2=0x7f8aec1cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:38.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.488+0000 7f8aebe00640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8aec0b4870 0x7f8aec1cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.488+0000 7f8aebe00640 1 -- 172.21.15.7:0/2283952193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aec1e7870 con 0x7f8aec145e40 2024-04-25T11:11:38.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.488+0000 7f8aebe00640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec145e40 0x7f8aec1cd110 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009f70 tx=0x7f8ae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:38.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.488+0000 7f8ae3400640 1 -- 172.21.15.7:0/2283952193 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae403f020 con 0x7f8aec145e40 2024-04-25T11:11:38.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.489+0000 7f8ae3400640 1 -- 172.21.15.7:0/2283952193 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae4037040 con 0x7f8aec145e40 2024-04-25T11:11:38.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.489+0000 7f8af1200640 1 -- 172.21.15.7:0/2283952193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aec1cf510 con 0x7f8aec145e40 2024-04-25T11:11:38.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.489+0000 7f8ae3400640 1 -- 172.21.15.7:0/2283952193 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4036070 con 0x7f8aec145e40 2024-04-25T11:11:38.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.489+0000 7f8af1200640 1 -- 172.21.15.7:0/2283952193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec1cfad0 con 0x7f8aec145e40 2024-04-25T11:11:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.490+0000 7f8af1200640 1 -- 172.21.15.7:0/2283952193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aec050740 con 0x7f8aec145e40 2024-04-25T11:11:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.491+0000 7f8ae3400640 1 -- 172.21.15.7:0/2283952193 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8ae4042020 con 0x7f8aec145e40 2024-04-25T11:11:38.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.492+0000 7f8ae3400640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8acc0744b0 0x7f8acc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:38.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.493+0000 7f8ae3400640 1 -- 172.21.15.7:0/2283952193 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f8ae40b5d60 con 0x7f8aec145e40 2024-04-25T11:11:38.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.493+0000 7f8aeb400640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8acc0744b0 0x7f8acc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:38.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.493+0000 7f8aeb400640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8acc0744b0 0x7f8acc0769c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009a30 tx=0x7f8adc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:38.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.497+0000 7f8ae3400640 1 -- 172.21.15.7:0/2283952193 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8ae4049020 con 0x7f8aec145e40 2024-04-25T11:11:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.699+0000 7f8af1200640 1 -- 172.21.15.7:0/2283952193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8aec06ceb0 con 0x7f8aec145e40 2024-04-25T11:11:38.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.700+0000 7f8ae3400640 1 -- 172.21.15.7:0/2283952193 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f8ae407fb20 con 0x7f8aec145e40 2024-04-25T11:11:38.701 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.702+0000 7f8af1200640 1 -- 172.21.15.7:0/2283952193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8acc0744b0 msgr2=0x7f8acc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.702+0000 7f8af1200640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8acc0744b0 0x7f8acc0769c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8adc009a30 tx=0x7f8adc008040 comp rx=0 tx=0).stop 2024-04-25T11:11:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.702+0000 7f8af1200640 1 -- 172.21.15.7:0/2283952193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec145e40 msgr2=0x7f8aec1cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.702+0000 7f8af1200640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec145e40 0x7f8aec1cd110 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4009f70 tx=0x7f8ae4004a40 comp rx=0 tx=0).stop 2024-04-25T11:11:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.703+0000 7f8af1200640 1 -- 172.21.15.7:0/2283952193 shutdown_connections 2024-04-25T11:11:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.703+0000 7f8af1200640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8acc0744b0 0x7f8acc0769c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.703+0000 7f8af1200640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8aec145e40 0x7f8aec1cd110 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.703+0000 7f8af1200640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8aec144a80 0x7f8aec1ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.703+0000 7f8af1200640 1 --2- 172.21.15.7:0/2283952193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8aec0b4870 0x7f8aec1cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.703+0000 7f8af1200640 1 -- 172.21.15.7:0/2283952193 >> 172.21.15.7:0/2283952193 conn(0x7f8aec0740f0 msgr2=0x7f8aec142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.703+0000 7f8af1200640 1 -- 172.21.15.7:0/2283952193 shutdown_connections 2024-04-25T11:11:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:38.703+0000 7f8af1200640 1 -- 172.21.15.7:0/2283952193 wait complete. 2024-04-25T11:11:38.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.016+0000 7f9fad000640 1 Processor -- start 2024-04-25T11:11:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.017+0000 7f9fad000640 1 -- start start 2024-04-25T11:11:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.019+0000 7f9fad000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fa80d2c60 0x7f9fa80d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.019+0000 7f9fad000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa8164090 con 0x7f9fa8160180 2024-04-25T11:11:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.019+0000 7f9fad000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fa8164bb0 con 0x7f9fa8163c80 2024-04-25T11:11:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.019+0000 7f9fad000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa80d0c00 con 0x7f9fa80d2c60 2024-04-25T11:11:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.019+0000 7f9fa6800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fa80d2c60 0x7f9fa80d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.020+0000 7f9fa5e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9fa8160180 0x7f9fa80b9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58982/0 (socket says 172.21.15.7:58982) 2024-04-25T11:11:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.020+0000 7f9fa6800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fa80d2c60 0x7f9fa80d06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46424/0 (socket says 172.21.15.7:46424) 2024-04-25T11:11:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.020+0000 7f9fa5e00640 1 -- 172.21.15.7:0/3584654324 learned_addr learned my addr 172.21.15.7:0/3584654324 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.020+0000 7f9fa6800640 1 -- 172.21.15.7:0/3584654324 >> v1:172.21.15.87:6789/0 conn(0x7f9fa8163c80 legacy=0x7f9fa80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.020+0000 7f9fa6800640 1 -- 172.21.15.7:0/3584654324 >> v1:172.21.15.7:6789/0 conn(0x7f9fa8160180 legacy=0x7f9fa80b9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.020+0000 7f9fa6800640 1 -- 172.21.15.7:0/3584654324 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa8164310 con 0x7f9fa80d2c60 2024-04-25T11:11:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.021+0000 7f9fa6800640 1 --2- 172.21.15.7:0/3584654324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa80d2c60 0x7f9fa80d06c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00097a0 tx=0x7f9fa002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94766cc483e66221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.021+0000 7f9fa5400640 1 -- 172.21.15.7:0/3584654324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0037070 con 0x7f9fa80d2c60 2024-04-25T11:11:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.022+0000 7f9fa5400640 1 -- 172.21.15.7:0/3584654324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa0032440 con 0x7f9fa80d2c60 2024-04-25T11:11:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.022+0000 7f9fa5400640 1 -- 172.21.15.7:0/3584654324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0032810 con 0x7f9fa80d2c60 2024-04-25T11:11:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.022+0000 7f9fad000640 1 -- 172.21.15.7:0/3584654324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa80d2c60 msgr2=0x7f9fa80d06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.022+0000 7f9fad000640 1 --2- 172.21.15.7:0/3584654324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa80d2c60 0x7f9fa80d06c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00097a0 tx=0x7f9fa002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.022+0000 7f9fad000640 1 -- 172.21.15.7:0/3584654324 shutdown_connections 2024-04-25T11:11:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.022+0000 7f9fad000640 1 --2- 172.21.15.7:0/3584654324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa80d2c60 0x7f9fa80d06c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.022+0000 7f9fad000640 1 -- 172.21.15.7:0/3584654324 >> 172.21.15.7:0/3584654324 conn(0x7f9fa806c750 msgr2=0x7f9fa806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.022+0000 7f9fad000640 1 -- 172.21.15.7:0/3584654324 shutdown_connections 2024-04-25T11:11:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.023+0000 7f9fad000640 1 -- 172.21.15.7:0/3584654324 wait complete. 2024-04-25T11:11:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.025+0000 7f9fad000640 1 Processor -- start 2024-04-25T11:11:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.025+0000 7f9fad000640 1 -- start start 2024-04-25T11:11:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.027+0000 7f9fad000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fa80d2c60 0x7f9fa80cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.027+0000 7f9fad000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa8160180 0x7f9fa80c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.027+0000 7f9fad000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa8163c80 0x7f9fa80c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.027+0000 7f9fad000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa81648d0 con 0x7f9fa8163c80 2024-04-25T11:11:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.027+0000 7f9fad000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80d4510 con 0x7f9fa80d2c60 2024-04-25T11:11:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.027+0000 7f9fad000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa80d49a0 con 0x7f9fa8160180 2024-04-25T11:11:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.027+0000 7f9fa6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa8163c80 0x7f9fa80c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.027+0000 7f9fa6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa8163c80 0x7f9fa80c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46434/0 (socket says 172.21.15.7:46434) 2024-04-25T11:11:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.027+0000 7f9fa6800640 1 -- 172.21.15.7:0/760539654 learned_addr learned my addr 172.21.15.7:0/760539654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:39.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.027+0000 7f9f9d400640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa8160180 0x7f9fa80c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.027+0000 7f9fa5e00640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fa80d2c60 0x7f9fa80cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.028+0000 7f9fa6800640 1 -- 172.21.15.7:0/760539654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa8160180 msgr2=0x7f9fa80c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.028+0000 7f9fa6800640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa8160180 0x7f9fa80c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.028+0000 7f9fa6800640 1 -- 172.21.15.7:0/760539654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fa80d2c60 msgr2=0x7f9fa80cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.028+0000 7f9fa6800640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fa80d2c60 0x7f9fa80cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.028+0000 7f9fa6800640 1 -- 172.21.15.7:0/760539654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa815d9a0 con 0x7f9fa8163c80 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.028+0000 7f9fa6800640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa8163c80 0x7f9fa80c9d50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00097a0 tx=0x7f9fa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.028+0000 7f9f9ea00640 1 -- 172.21.15.7:0/760539654 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0037040 con 0x7f9fa8163c80 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.028+0000 7f9f9ea00640 1 -- 172.21.15.7:0/760539654 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa0042070 con 0x7f9fa8163c80 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.028+0000 7f9fad000640 1 -- 172.21.15.7:0/760539654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa81ea4f0 con 0x7f9fa8163c80 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.028+0000 7f9f9ea00640 1 -- 172.21.15.7:0/760539654 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0030b30 con 0x7f9fa8163c80 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.029+0000 7f9fad000640 1 -- 172.21.15.7:0/760539654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa81ead00 con 0x7f9fa8163c80 2024-04-25T11:11:39.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.030+0000 7f9fad000640 1 -- 172.21.15.7:0/760539654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa81ea790 con 0x7f9fa8163c80 2024-04-25T11:11:39.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.031+0000 7f9f9ea00640 1 -- 172.21.15.7:0/760539654 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9fa0030cd0 con 0x7f9fa8163c80 2024-04-25T11:11:39.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.032+0000 7f9f9ea00640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f840744b0 0x7f9f840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:39.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.032+0000 7f9f9ea00640 1 -- 172.21.15.7:0/760539654 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f9fa00b6240 con 0x7f9fa8163c80 2024-04-25T11:11:39.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.032+0000 7f9fa5e00640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f840744b0 0x7f9f840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:39.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.033+0000 7f9fa5e00640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f840744b0 0x7f9f840769c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f980012e0 tx=0x7f9f98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:39.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.037+0000 7f9f9ea00640 1 -- 172.21.15.7:0/760539654 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9fa004c020 con 0x7f9fa8163c80 2024-04-25T11:11:39.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.237+0000 7f9fad000640 1 -- 172.21.15.7:0/760539654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9fa8050740 con 0x7f9fa8163c80 2024-04-25T11:11:39.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.238+0000 7f9f9ea00640 1 -- 172.21.15.7:0/760539654 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f9fa0080530 con 0x7f9fa8163c80 2024-04-25T11:11:39.240 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.241+0000 7f9fad000640 1 -- 172.21.15.7:0/760539654 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f840744b0 msgr2=0x7f9f840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.241+0000 7f9fad000640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f840744b0 0x7f9f840769c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f980012e0 tx=0x7f9f98009040 comp rx=0 tx=0).stop 2024-04-25T11:11:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.241+0000 7f9fad000640 1 -- 172.21.15.7:0/760539654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa8163c80 msgr2=0x7f9fa80c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.241+0000 7f9fad000640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa8163c80 0x7f9fa80c9d50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00097a0 tx=0x7f9fa00046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.241+0000 7f9fad000640 1 -- 172.21.15.7:0/760539654 shutdown_connections 2024-04-25T11:11:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.241+0000 7f9fad000640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f840744b0 0x7f9f840769c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.241+0000 7f9fad000640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa8163c80 0x7f9fa80c9d50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.241+0000 7f9fad000640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa8160180 0x7f9fa80c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.241+0000 7f9fad000640 1 --2- 172.21.15.7:0/760539654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fa80d2c60 0x7f9fa80cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.241+0000 7f9fad000640 1 -- 172.21.15.7:0/760539654 >> 172.21.15.7:0/760539654 conn(0x7f9fa806c750 msgr2=0x7f9fa815f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.242+0000 7f9fad000640 1 -- 172.21.15.7:0/760539654 shutdown_connections 2024-04-25T11:11:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.242+0000 7f9fad000640 1 -- 172.21.15.7:0/760539654 wait complete. 2024-04-25T11:11:39.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:39.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.558+0000 7effe7200640 1 Processor -- start 2024-04-25T11:11:39.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.559+0000 7effe7200640 1 -- start start 2024-04-25T11:11:39.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.561+0000 7effe7200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effe80b4ab0 0x7effe80d0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:39.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.561+0000 7effe7200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe8163cd0 con 0x7effe80b7b40 2024-04-25T11:11:39.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.561+0000 7effe7200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe8163fb0 con 0x7effe8160080 2024-04-25T11:11:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.561+0000 7effe7200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7effe8163a80 con 0x7effe80b4ab0 2024-04-25T11:11:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.562+0000 7effe5e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effe80b4ab0 0x7effe80d0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.562+0000 7effe5e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effe80b4ab0 0x7effe80d0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46444/0 (socket says 172.21.15.7:46444) 2024-04-25T11:11:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.562+0000 7effe5e00640 1 -- 172.21.15.7:0/1130396095 learned_addr learned my addr 172.21.15.7:0/1130396095 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.562+0000 7effe5e00640 1 -- 172.21.15.7:0/1130396095 >> v1:172.21.15.87:6789/0 conn(0x7effe8160080 legacy=0x7effe80b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.562+0000 7effe4a00640 1 -- 172.21.15.7:0/1130396095 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 485383138 0 0) 0x7effe8163cd0 con 0x7effe80b7b40 2024-04-25T11:11:39.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.563+0000 7effe5e00640 1 -- 172.21.15.7:0/1130396095 >> v1:172.21.15.7:6789/0 conn(0x7effe80b7b40 legacy=0x7effe80d4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.563+0000 7effe5e00640 1 -- 172.21.15.7:0/1130396095 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe8164290 con 0x7effe80b4ab0 2024-04-25T11:11:39.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.564+0000 7effe5e00640 1 --2- 172.21.15.7:0/1130396095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80b4ab0 0x7effe80d0f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7effd0009870 tx=0x7effd002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf11c77d543f097 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:39.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.564+0000 7effe4a00640 1 -- 172.21.15.7:0/1130396095 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0037070 con 0x7effe80b4ab0 2024-04-25T11:11:39.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.564+0000 7effe4a00640 1 -- 172.21.15.7:0/1130396095 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd002ae50 con 0x7effe80b4ab0 2024-04-25T11:11:39.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.564+0000 7effe4a00640 1 -- 172.21.15.7:0/1130396095 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0032860 con 0x7effe80b4ab0 2024-04-25T11:11:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.565+0000 7effe7200640 1 -- 172.21.15.7:0/1130396095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80b4ab0 msgr2=0x7effe80d0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.565+0000 7effe7200640 1 --2- 172.21.15.7:0/1130396095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80b4ab0 0x7effe80d0f40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7effd0009870 tx=0x7effd002a300 comp rx=0 tx=0).stop 2024-04-25T11:11:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.565+0000 7effe7200640 1 -- 172.21.15.7:0/1130396095 shutdown_connections 2024-04-25T11:11:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.565+0000 7effe7200640 1 --2- 172.21.15.7:0/1130396095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe80b4ab0 0x7effe80d0f40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.565+0000 7effe7200640 1 -- 172.21.15.7:0/1130396095 >> 172.21.15.7:0/1130396095 conn(0x7effe80d2610 msgr2=0x7effe80d2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.565+0000 7effe7200640 1 -- 172.21.15.7:0/1130396095 shutdown_connections 2024-04-25T11:11:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.565+0000 7effe7200640 1 -- 172.21.15.7:0/1130396095 wait complete. 2024-04-25T11:11:39.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.568+0000 7effe7200640 1 Processor -- start 2024-04-25T11:11:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.568+0000 7effe7200640 1 -- start start 2024-04-25T11:11:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe7200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe80b4ab0 0x7effe815b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe8160080 0x7effe80ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe80ce6d0 0x7effe80c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7effe80b8e80 con 0x7effe8160080 2024-04-25T11:11:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe7200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7effe80d1480 con 0x7effe80b4ab0 2024-04-25T11:11:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7effe80b9000 con 0x7effe80ce6d0 2024-04-25T11:11:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe8160080 0x7effe80ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe8160080 0x7effe80ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46448/0 (socket says 172.21.15.7:46448) 2024-04-25T11:11:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe5400640 1 -- 172.21.15.7:0/603969325 learned_addr learned my addr 172.21.15.7:0/603969325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe5e00640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe80b4ab0 0x7effe815b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe6800640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe80ce6d0 0x7effe80c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe5400640 1 -- 172.21.15.7:0/603969325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe80ce6d0 msgr2=0x7effe80c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe5400640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe80ce6d0 0x7effe80c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe5400640 1 -- 172.21.15.7:0/603969325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe80b4ab0 msgr2=0x7effe815b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe5400640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe80b4ab0 0x7effe815b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.570+0000 7effe5400640 1 -- 172.21.15.7:0/603969325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe80ceec0 con 0x7effe8160080 2024-04-25T11:11:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.571+0000 7effe6800640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe80ce6d0 0x7effe80c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.571+0000 7effe5e00640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe80b4ab0 0x7effe815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.571+0000 7effe5400640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe8160080 0x7effe80ce190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7effd800c400 tx=0x7effd800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.571+0000 7effde000640 1 -- 172.21.15.7:0/603969325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8017070 con 0x7effe8160080 2024-04-25T11:11:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.571+0000 7effde000640 1 -- 172.21.15.7:0/603969325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd80052d0 con 0x7effe8160080 2024-04-25T11:11:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.571+0000 7effde000640 1 -- 172.21.15.7:0/603969325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8003830 con 0x7effe8160080 2024-04-25T11:11:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.571+0000 7effe7200640 1 -- 172.21.15.7:0/603969325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe80cf1a0 con 0x7effe8160080 2024-04-25T11:11:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.571+0000 7effe7200640 1 -- 172.21.15.7:0/603969325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe80cf760 con 0x7effe8160080 2024-04-25T11:11:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.573+0000 7effe7200640 1 -- 172.21.15.7:0/603969325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe8050740 con 0x7effe8160080 2024-04-25T11:11:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.574+0000 7effde000640 1 -- 172.21.15.7:0/603969325 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7effd801a070 con 0x7effe8160080 2024-04-25T11:11:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.575+0000 7effde000640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 0x7effc40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.575+0000 7effde000640 1 -- 172.21.15.7:0/603969325 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7effd80949c0 con 0x7effe8160080 2024-04-25T11:11:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.575+0000 7effe5e00640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 0x7effc40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.576+0000 7effe5e00640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 0x7effc40769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7effd0034000 tx=0x7effd002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:39.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.580+0000 7effde000640 1 -- 172.21.15.7:0/603969325 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effd8096e20 con 0x7effe8160080 2024-04-25T11:11:39.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.788+0000 7effe7200640 1 -- 172.21.15.7:0/603969325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7effe80cec50 con 0x7effe8160080 2024-04-25T11:11:39.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.789+0000 7effde000640 1 -- 172.21.15.7:0/603969325 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7effd805ed50 con 0x7effe8160080 2024-04-25T11:11:39.791 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:39.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 -- 172.21.15.7:0/603969325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 msgr2=0x7effc40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 0x7effc40769c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7effd0034000 tx=0x7effd002e000 comp rx=0 tx=0).stop 2024-04-25T11:11:39.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 -- 172.21.15.7:0/603969325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe8160080 msgr2=0x7effe80ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:39.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe8160080 0x7effe80ce190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7effd800c400 tx=0x7effd800c8c0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 -- 172.21.15.7:0/603969325 shutdown_connections 2024-04-25T11:11:39.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 0x7effc40769c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe80ce6d0 0x7effe80c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe8160080 0x7effe80ce190 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 --2- 172.21.15.7:0/603969325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe80b4ab0 0x7effe815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:39.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 -- 172.21.15.7:0/603969325 >> 172.21.15.7:0/603969325 conn(0x7effe80d2610 msgr2=0x7effe815ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:39.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 -- 172.21.15.7:0/603969325 shutdown_connections 2024-04-25T11:11:39.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:39.792+0000 7effe7200640 1 -- 172.21.15.7:0/603969325 wait complete. 2024-04-25T11:11:39.823 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:40.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.110+0000 7f7615a00640 1 Processor -- start 2024-04-25T11:11:40.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.110+0000 7f7615a00640 1 -- start start 2024-04-25T11:11:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.112+0000 7f7615a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76100d0ee0 0x7f76100affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.112+0000 7f7615a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76100b5020 0x7f7610160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.112+0000 7f7615a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761006f5b0 con 0x7f761015aaa0 2024-04-25T11:11:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.112+0000 7f7615a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f76100af290 con 0x7f76100b5020 2024-04-25T11:11:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.112+0000 7f7615a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f76100d14a0 con 0x7f76100d0ee0 2024-04-25T11:11:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.113+0000 7f760ea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f761015aaa0 0x7f76100b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59140/0 (socket says 172.21.15.7:59140) 2024-04-25T11:11:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.113+0000 7f760ea00640 1 -- 172.21.15.7:0/1794533172 learned_addr learned my addr 172.21.15.7:0/1794533172 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.113+0000 7f760f400640 1 --2- 172.21.15.7:0/1794533172 >> v2:172.21.15.7:3300/0 conn(0x7f76100b5020 0x7f7610160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.113+0000 7f760e000640 1 --2- 172.21.15.7:0/1794533172 >> v2:172.21.15.87:3300/0 conn(0x7f76100d0ee0 0x7f76100affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.113+0000 7f760d600640 1 -- 172.21.15.7:0/1794533172 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1902531466 0 0) 0x7f761006f5b0 con 0x7f761015aaa0 2024-04-25T11:11:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.113+0000 7f760d600640 1 -- 172.21.15.7:0/1794533172 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75f4003520 con 0x7f761015aaa0 2024-04-25T11:11:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.113+0000 7f760f400640 1 -- 172.21.15.7:0/1794533172 >> v2:172.21.15.87:3300/0 conn(0x7f76100d0ee0 msgr2=0x7f76100affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.113+0000 7f760f400640 1 --2- 172.21.15.7:0/1794533172 >> v2:172.21.15.87:3300/0 conn(0x7f76100d0ee0 0x7f76100affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.114+0000 7f760f400640 1 -- 172.21.15.7:0/1794533172 >> v1:172.21.15.7:6789/0 conn(0x7f761015aaa0 legacy=0x7f76100b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.114+0000 7f760f400640 1 -- 172.21.15.7:0/1794533172 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76100af4e0 con 0x7f76100b5020 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.115+0000 7f760f400640 1 --2- 172.21.15.7:0/1794533172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76100b5020 0x7f7610160990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f761006eba0 tx=0x7f760002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da9532f588512f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.115+0000 7f760d600640 1 -- 172.21.15.7:0/1794533172 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600037070 con 0x7f76100b5020 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.115+0000 7f760d600640 1 -- 172.21.15.7:0/1794533172 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7600032440 con 0x7f76100b5020 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.115+0000 7f760d600640 1 -- 172.21.15.7:0/1794533172 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000327f0 con 0x7f76100b5020 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.116+0000 7f7615a00640 1 -- 172.21.15.7:0/1794533172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76100b5020 msgr2=0x7f7610160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.116+0000 7f7615a00640 1 --2- 172.21.15.7:0/1794533172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76100b5020 0x7f7610160990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f761006eba0 tx=0x7f760002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.116+0000 7f7615a00640 1 -- 172.21.15.7:0/1794533172 shutdown_connections 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.116+0000 7f7615a00640 1 --2- 172.21.15.7:0/1794533172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76100b5020 0x7f7610160990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.116+0000 7f7615a00640 1 --2- 172.21.15.7:0/1794533172 >> v2:172.21.15.87:3300/0 conn(0x7f76100d0ee0 0x7f76100affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.116+0000 7f7615a00640 1 -- 172.21.15.7:0/1794533172 >> 172.21.15.7:0/1794533172 conn(0x7f76100742c0 msgr2=0x7f761006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.116+0000 7f7615a00640 1 -- 172.21.15.7:0/1794533172 shutdown_connections 2024-04-25T11:11:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.116+0000 7f7615a00640 1 -- 172.21.15.7:0/1794533172 wait complete. 2024-04-25T11:11:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.118+0000 7f7615a00640 1 Processor -- start 2024-04-25T11:11:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.119+0000 7f7615a00640 1 -- start start 2024-04-25T11:11:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.120+0000 7f7615a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76100b5020 0x7f76101518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.120+0000 7f7615a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76100d0ee0 0x7f761006fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.120+0000 7f7615a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f761015aaa0 0x7f7610070170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.120+0000 7f7615a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76100b5430 con 0x7f761015aaa0 2024-04-25T11:11:40.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.120+0000 7f7615a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f76100afaa0 con 0x7f76100b5020 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.120+0000 7f7615a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f76100b55b0 con 0x7f76100d0ee0 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f761015aaa0 0x7f7610070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f761015aaa0 0x7f7610070170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40264/0 (socket says 172.21.15.7:40264) 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760f400640 1 -- 172.21.15.7:0/611292280 learned_addr learned my addr 172.21.15.7:0/611292280 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760ea00640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76100b5020 0x7f76101518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760e000640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76100d0ee0 0x7f761006fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760f400640 1 -- 172.21.15.7:0/611292280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76100d0ee0 msgr2=0x7f761006fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760f400640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76100d0ee0 0x7f761006fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760f400640 1 -- 172.21.15.7:0/611292280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76100b5020 msgr2=0x7f76101518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760f400640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76100b5020 0x7f76101518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760f400640 1 -- 172.21.15.7:0/611292280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76100cf3a0 con 0x7f761015aaa0 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760ea00640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76100b5020 0x7f76101518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.121+0000 7f760e000640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76100d0ee0 0x7f761006fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:40.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.122+0000 7f760f400640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f761015aaa0 0x7f7610070170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7600035040 tx=0x7f76000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:40.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.122+0000 7f7606a00640 1 -- 172.21.15.7:0/611292280 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600037040 con 0x7f761015aaa0 2024-04-25T11:11:40.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.122+0000 7f7606a00640 1 -- 172.21.15.7:0/611292280 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7600042070 con 0x7f761015aaa0 2024-04-25T11:11:40.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.122+0000 7f7606a00640 1 -- 172.21.15.7:0/611292280 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600030840 con 0x7f761015aaa0 2024-04-25T11:11:40.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.122+0000 7f7615a00640 1 -- 172.21.15.7:0/611292280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76100cf680 con 0x7f761015aaa0 2024-04-25T11:11:40.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.122+0000 7f7615a00640 1 -- 172.21.15.7:0/611292280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76100cfc40 con 0x7f761015aaa0 2024-04-25T11:11:40.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.123+0000 7f7615a00640 1 -- 172.21.15.7:0/611292280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7610050740 con 0x7f761015aaa0 2024-04-25T11:11:40.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.124+0000 7f7606a00640 1 -- 172.21.15.7:0/611292280 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f76000309e0 con 0x7f761015aaa0 2024-04-25T11:11:40.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.125+0000 7f7606a00640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f00744e0 0x7f75f00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.126+0000 7f7606a00640 1 -- 172.21.15.7:0/611292280 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f76000b5f20 con 0x7f761015aaa0 2024-04-25T11:11:40.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.126+0000 7f760ea00640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f00744e0 0x7f75f00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.127+0000 7f760ea00640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f00744e0 0x7f75f00769f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7608005880 tx=0x7f7608009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:40.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.131+0000 7f7606a00640 1 -- 172.21.15.7:0/611292280 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f760004c020 con 0x7f761015aaa0 2024-04-25T11:11:40.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.340+0000 7f7615a00640 1 -- 172.21.15.7:0/611292280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7610070f20 con 0x7f761015aaa0 2024-04-25T11:11:40.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.341+0000 7f7606a00640 1 -- 172.21.15.7:0/611292280 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f7600080210 con 0x7f761015aaa0 2024-04-25T11:11:40.342 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 -- 172.21.15.7:0/611292280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f00744e0 msgr2=0x7f75f00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f00744e0 0x7f75f00769f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7608005880 tx=0x7f7608009040 comp rx=0 tx=0).stop 2024-04-25T11:11:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 -- 172.21.15.7:0/611292280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f761015aaa0 msgr2=0x7f7610070170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f761015aaa0 0x7f7610070170 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7600035040 tx=0x7f76000046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 -- 172.21.15.7:0/611292280 shutdown_connections 2024-04-25T11:11:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f00744e0 0x7f75f00769f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f761015aaa0 0x7f7610070170 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76100d0ee0 0x7f761006fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 --2- 172.21.15.7:0/611292280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76100b5020 0x7f76101518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 -- 172.21.15.7:0/611292280 >> 172.21.15.7:0/611292280 conn(0x7f76100742c0 msgr2=0x7f761015be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:40.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 -- 172.21.15.7:0/611292280 shutdown_connections 2024-04-25T11:11:40.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.344+0000 7f7615a00640 1 -- 172.21.15.7:0/611292280 wait complete. 2024-04-25T11:11:40.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2024-04-25T11:11:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.657+0000 7f7fa6200640 1 Processor -- start 2024-04-25T11:11:40.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.657+0000 7f7fa6200640 1 -- start start 2024-04-25T11:11:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.660+0000 7f7fa6200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7fa00b5a90 0x7f7fa0167930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.660+0000 7f7fa6200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa015e190 con 0x7f7fa0070a70 2024-04-25T11:11:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.660+0000 7f7fa6200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa015d610 con 0x7f7fa00b04d0 2024-04-25T11:11:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.660+0000 7f7fa6200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa015d8f0 con 0x7f7fa00b5a90 2024-04-25T11:11:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.660+0000 7f7fa4e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7fa0070a70 0x7f7fa00ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59148/0 (socket says 172.21.15.7:59148) 2024-04-25T11:11:40.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.660+0000 7f7fa4e00640 1 -- 172.21.15.7:0/1917703831 learned_addr learned my addr 172.21.15.7:0/1917703831 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005101958 0 0) 0x7f7fa015e190 con 0x7f7fa0070a70 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f84005730 con 0x7f7fa0070a70 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7fa5800640 1 --2- 172.21.15.7:0/1917703831 >> v2:172.21.15.87:3300/0 conn(0x7f7fa00b5a90 0x7f7fa0167930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1059952715 0 0) 0x7f7f84005730 con 0x7f7fa0070a70 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 >> v2:172.21.15.87:3300/0 conn(0x7f7fa00b5a90 msgr2=0x7f7fa0167930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 --2- 172.21.15.7:0/1917703831 >> v2:172.21.15.87:3300/0 conn(0x7f7fa00b5a90 0x7f7fa0167930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 >> v1:172.21.15.87:6789/0 conn(0x7f7fa00b04d0 legacy=0x7f7fa00b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa015dbd0 con 0x7f7fa0070a70 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f7f90001670 con 0x7f7fa0070a70 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 >> v1:172.21.15.7:6789/0 conn(0x7f7fa0070a70 legacy=0x7f7fa00ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 --2- 172.21.15.7:0/1917703831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f84005bc0 0x7f7f84006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 --2- 172.21.15.7:0/1917703831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 0x7f7f84006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 --2- 172.21.15.7:0/1917703831 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f84007480 0x7f7f8400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa015d8f0 con 0x7f7f84006a10 2024-04-25T11:11:40.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8400e410 con 0x7f7f84005bc0 2024-04-25T11:11:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.661+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f8400e610 con 0x7f7f84007480 2024-04-25T11:11:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.662+0000 7f7f97e00640 1 --2- 172.21.15.7:0/1917703831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 0x7f7f84006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.662+0000 7f7fa4e00640 1 --2- 172.21.15.7:0/1917703831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f84005bc0 0x7f7f84006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.662+0000 7f7f97e00640 1 -- 172.21.15.7:0/1917703831 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f84007480 msgr2=0x7f7f8400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.662+0000 7f7f97e00640 1 --2- 172.21.15.7:0/1917703831 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f84007480 0x7f7f8400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.662+0000 7f7f97e00640 1 -- 172.21.15.7:0/1917703831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f84005bc0 msgr2=0x7f7f84006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.662+0000 7f7f97e00640 1 --2- 172.21.15.7:0/1917703831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f84005bc0 0x7f7f84006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.662+0000 7f7f97e00640 1 -- 172.21.15.7:0/1917703831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f84005730 con 0x7f7f84006a10 2024-04-25T11:11:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.662+0000 7f7fa5800640 1 -- 172.21.15.7:0/1917703831 reap_dead start 2024-04-25T11:11:40.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.662+0000 7f7fa4e00640 1 --2- 172.21.15.7:0/1917703831 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f84005bc0 0x7f7f84006030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.663+0000 7f7f97e00640 1 --2- 172.21.15.7:0/1917703831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 0x7f7f84006e20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f88001750 tx=0x7f7f84007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20c30947465705f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.664+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88001370 con 0x7f7f84006a10 2024-04-25T11:11:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.664+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f880052d0 con 0x7f7f84006a10 2024-04-25T11:11:40.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.664+0000 7f7f9fe00640 1 -- 172.21.15.7:0/1917703831 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8802abc0 con 0x7f7f84006a10 2024-04-25T11:11:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.664+0000 7f7fa6200640 1 -- 172.21.15.7:0/1917703831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 msgr2=0x7f7f84006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.664+0000 7f7fa6200640 1 --2- 172.21.15.7:0/1917703831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 0x7f7f84006e20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f88001750 tx=0x7f7f84007a40 comp rx=0 tx=0).stop 2024-04-25T11:11:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.664+0000 7f7fa6200640 1 -- 172.21.15.7:0/1917703831 shutdown_connections 2024-04-25T11:11:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.664+0000 7f7fa6200640 1 --2- 172.21.15.7:0/1917703831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 0x7f7f84006e20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.664+0000 7f7fa6200640 1 -- 172.21.15.7:0/1917703831 >> 172.21.15.7:0/1917703831 conn(0x7f7fa00b3820 msgr2=0x7f7fa00b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.665+0000 7f7fa6200640 1 -- 172.21.15.7:0/1917703831 shutdown_connections 2024-04-25T11:11:40.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.665+0000 7f7fa6200640 1 -- 172.21.15.7:0/1917703831 wait complete. 2024-04-25T11:11:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.667+0000 7f7fa6200640 1 Processor -- start 2024-04-25T11:11:40.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.667+0000 7f7fa6200640 1 -- start start 2024-04-25T11:11:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.669+0000 7f7fa6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 0x7f7fa015bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.669+0000 7f7fa6200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fa0148fb0 0x7f7fa00ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.669+0000 7f7fa6200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa01493c0 0x7f7fa00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.669+0000 7f7fa6200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa015d2f0 con 0x7f7f84006a10 2024-04-25T11:11:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.669+0000 7f7fa6200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa015d170 con 0x7f7fa01493c0 2024-04-25T11:11:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.669+0000 7f7fa6200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa00b08e0 con 0x7f7fa0148fb0 2024-04-25T11:11:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.669+0000 7f7fa4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 0x7f7fa015bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.669+0000 7f7fa4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 0x7f7fa015bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40270/0 (socket says 172.21.15.7:40270) 2024-04-25T11:11:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.669+0000 7f7fa4e00640 1 -- 172.21.15.7:0/3406325478 learned_addr learned my addr 172.21.15.7:0/3406325478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:40.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7fa5800640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa01493c0 0x7f7fa00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7f97e00640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fa0148fb0 0x7f7fa00ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7fa4e00640 1 -- 172.21.15.7:0/3406325478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fa0148fb0 msgr2=0x7f7fa00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7fa4e00640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fa0148fb0 0x7f7fa00ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7fa4e00640 1 -- 172.21.15.7:0/3406325478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa01493c0 msgr2=0x7f7fa00cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7fa4e00640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa01493c0 0x7f7fa00cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7fa4e00640 1 -- 172.21.15.7:0/3406325478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa015c260 con 0x7f7f84006a10 2024-04-25T11:11:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7fa5800640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa01493c0 0x7f7fa00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7f97e00640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fa0148fb0 0x7f7fa00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:40.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7fa4e00640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 0x7f7fa015bd20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000d7a0 tx=0x7f7f9000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7f9d600640 1 -- 172.21.15.7:0/3406325478 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90019070 con 0x7f7f84006a10 2024-04-25T11:11:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7f9d600640 1 -- 172.21.15.7:0/3406325478 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f900074e0 con 0x7f7f84006a10 2024-04-25T11:11:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7fa6200640 1 -- 172.21.15.7:0/3406325478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa015a3a0 con 0x7f7f84006a10 2024-04-25T11:11:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.670+0000 7f7f9d600640 1 -- 172.21.15.7:0/3406325478 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90004a20 con 0x7f7f84006a10 2024-04-25T11:11:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.671+0000 7f7fa6200640 1 -- 172.21.15.7:0/3406325478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa015a960 con 0x7f7f84006a10 2024-04-25T11:11:40.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.672+0000 7f7fa6200640 1 -- 172.21.15.7:0/3406325478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa0050740 con 0x7f7f84006a10 2024-04-25T11:11:40.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.673+0000 7f7f9d600640 1 -- 172.21.15.7:0/3406325478 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7f9001c070 con 0x7f7f84006a10 2024-04-25T11:11:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.674+0000 7f7f9d600640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f7c0744b0 0x7f7f7c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.674+0000 7f7f9d600640 1 -- 172.21.15.7:0/3406325478 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f7f900950e0 con 0x7f7f84006a10 2024-04-25T11:11:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.674+0000 7f7f97e00640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f7c0744b0 0x7f7f7c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:40.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.675+0000 7f7f97e00640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f7c0744b0 0x7f7f7c0769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f88032000 tx=0x7f7f8802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.679+0000 7f7f9d600640 1 -- 172.21.15.7:0/3406325478 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f9005f3d0 con 0x7f7f84006a10 2024-04-25T11:11:40.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:40.962+0000 7f7fa6200640 1 -- 172.21.15.7:0/3406325478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) -- 0x7f7fa015f810 con 0x7f7f84006a10 2024-04-25T11:11:41.064 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:11:41.072 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:11:41.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.270+0000 7f7f9d600640 1 -- 172.21.15.7:0/3406325478 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v90) ==== 168+0+0 (secure 0 0 0) 0x7f7f9005ecd0 con 0x7f7f84006a10 2024-04-25T11:11:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.274+0000 7f7fa6200640 1 -- 172.21.15.7:0/3406325478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f7c0744b0 msgr2=0x7f7f7c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.274+0000 7f7fa6200640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f7c0744b0 0x7f7f7c0769c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7f88032000 tx=0x7f7f8802e000 comp rx=0 tx=0).stop 2024-04-25T11:11:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.274+0000 7f7fa6200640 1 -- 172.21.15.7:0/3406325478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 msgr2=0x7f7fa015bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.274+0000 7f7fa6200640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 0x7f7fa015bd20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000d7a0 tx=0x7f7f9000dc60 comp rx=0 tx=0).stop 2024-04-25T11:11:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.274+0000 7f7fa6200640 1 -- 172.21.15.7:0/3406325478 shutdown_connections 2024-04-25T11:11:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.274+0000 7f7fa6200640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f7c0744b0 0x7f7f7c0769c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.274+0000 7f7fa6200640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fa01493c0 0x7f7fa00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.274+0000 7f7fa6200640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fa0148fb0 0x7f7fa00ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.274+0000 7f7fa6200640 1 --2- 172.21.15.7:0/3406325478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f84006a10 0x7f7fa015bd20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.274+0000 7f7fa6200640 1 -- 172.21.15.7:0/3406325478 >> 172.21.15.7:0/3406325478 conn(0x7f7fa00b3820 msgr2=0x7f7fa00b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.275+0000 7f7fa6200640 1 -- 172.21.15.7:0/3406325478 shutdown_connections 2024-04-25T11:11:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.275+0000 7f7fa6200640 1 -- 172.21.15.7:0/3406325478 wait complete. 2024-04-25T11:11:41.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:11:41.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.598+0000 7f710e800640 1 Processor -- start 2024-04-25T11:11:41.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.599+0000 7f710e800640 1 -- start start 2024-04-25T11:11:41.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.601+0000 7f710e800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71081583c0 0x7f71080cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:41.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.601+0000 7f710e800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71080cfb50 0x7f71080d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:41.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.601+0000 7f710e800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71080ce570 con 0x7f71080d09c0 2024-04-25T11:11:41.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.601+0000 7f710e800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f710815f190 con 0x7f71081583c0 2024-04-25T11:11:41.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.601+0000 7f710e800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f71080ce850 con 0x7f71080cfb50 2024-04-25T11:11:41.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.601+0000 7f710d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71081583c0 0x7f71080cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:41.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.602+0000 7f710d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71081583c0 0x7f71080cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40286/0 (socket says 172.21.15.7:40286) 2024-04-25T11:11:41.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.602+0000 7f710d400640 1 -- 172.21.15.7:0/1602762326 learned_addr learned my addr 172.21.15.7:0/1602762326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:41.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.602+0000 7f710de00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f71080d09c0 0x7f710806eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59162/0 (socket says 172.21.15.7:59162) 2024-04-25T11:11:41.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.602+0000 7f710ca00640 1 --2- 172.21.15.7:0/1602762326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71080cfb50 0x7f71080d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:41.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.602+0000 7f710d400640 1 -- 172.21.15.7:0/1602762326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71080cfb50 msgr2=0x7f71080d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:41.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.602+0000 7f710d400640 1 --2- 172.21.15.7:0/1602762326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71080cfb50 0x7f71080d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.602+0000 7f710d400640 1 -- 172.21.15.7:0/1602762326 >> v1:172.21.15.7:6789/0 conn(0x7f71080d09c0 legacy=0x7f710806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:41.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.602+0000 7f710d400640 1 -- 172.21.15.7:0/1602762326 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71080ceb30 con 0x7f71081583c0 2024-04-25T11:11:41.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.603+0000 7f710d400640 1 --2- 172.21.15.7:0/1602762326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71081583c0 0x7f71080cf610 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009e50 tx=0x7f70fc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e4e040f2ed81ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:41.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.603+0000 7f7107e00640 1 -- 172.21.15.7:0/1602762326 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc036070 con 0x7f71081583c0 2024-04-25T11:11:41.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.604+0000 7f7107e00640 1 -- 172.21.15.7:0/1602762326 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70fc031600 con 0x7f71081583c0 2024-04-25T11:11:41.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.604+0000 7f7107e00640 1 -- 172.21.15.7:0/1602762326 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0319d0 con 0x7f71081583c0 2024-04-25T11:11:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.604+0000 7f710e800640 1 -- 172.21.15.7:0/1602762326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71081583c0 msgr2=0x7f71080cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.604+0000 7f710e800640 1 --2- 172.21.15.7:0/1602762326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71081583c0 0x7f71080cf610 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009e50 tx=0x7f70fc02a710 comp rx=0 tx=0).stop 2024-04-25T11:11:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.604+0000 7f710e800640 1 -- 172.21.15.7:0/1602762326 shutdown_connections 2024-04-25T11:11:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.604+0000 7f710e800640 1 --2- 172.21.15.7:0/1602762326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71080cfb50 0x7f71080d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.604+0000 7f710e800640 1 --2- 172.21.15.7:0/1602762326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71081583c0 0x7f71080cf610 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.604+0000 7f710e800640 1 -- 172.21.15.7:0/1602762326 >> 172.21.15.7:0/1602762326 conn(0x7f71080740f0 msgr2=0x7f710806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.605+0000 7f710e800640 1 -- 172.21.15.7:0/1602762326 shutdown_connections 2024-04-25T11:11:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.605+0000 7f710e800640 1 -- 172.21.15.7:0/1602762326 wait complete. 2024-04-25T11:11:41.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.607+0000 7f710e800640 1 Processor -- start 2024-04-25T11:11:41.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.607+0000 7f710e800640 1 -- start start 2024-04-25T11:11:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.609+0000 7f710e800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71080cfb50 0x7f7108161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.609+0000 7f710e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71080d09c0 0x7f71081f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.609+0000 7f710e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71081583c0 0x7f71081ea8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.609+0000 7f710e800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f710806f5b0 con 0x7f71081583c0 2024-04-25T11:11:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.609+0000 7f710e800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f71080cf0f0 con 0x7f71080d09c0 2024-04-25T11:11:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.609+0000 7f710e800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f710806f730 con 0x7f71080cfb50 2024-04-25T11:11:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.609+0000 7f710de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71081583c0 0x7f71081ea8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.609+0000 7f710de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71081583c0 0x7f71081ea8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40288/0 (socket says 172.21.15.7:40288) 2024-04-25T11:11:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.609+0000 7f710de00640 1 -- 172.21.15.7:0/3405453985 learned_addr learned my addr 172.21.15.7:0/3405453985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.609+0000 7f710ca00640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71080d09c0 0x7f71081f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.609+0000 7f710d400640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71080cfb50 0x7f7108161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.610+0000 7f710de00640 1 -- 172.21.15.7:0/3405453985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71080cfb50 msgr2=0x7f7108161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.610+0000 7f710de00640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71080cfb50 0x7f7108161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.610+0000 7f710de00640 1 -- 172.21.15.7:0/3405453985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71080d09c0 msgr2=0x7f71081f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.610+0000 7f710de00640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71080d09c0 0x7f71081f5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.610+0000 7f710de00640 1 -- 172.21.15.7:0/3405453985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71081621e0 con 0x7f71081583c0 2024-04-25T11:11:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.610+0000 7f710de00640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71081583c0 0x7f71081ea8a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f710000c390 tx=0x7f710000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:41.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.610+0000 7f7105600640 1 -- 172.21.15.7:0/3405453985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000074e0 con 0x7f71081583c0 2024-04-25T11:11:41.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.610+0000 7f7105600640 1 -- 172.21.15.7:0/3405453985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7100007680 con 0x7f71081583c0 2024-04-25T11:11:41.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.610+0000 7f7105600640 1 -- 172.21.15.7:0/3405453985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71000049e0 con 0x7f71081583c0 2024-04-25T11:11:41.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.610+0000 7f710e800640 1 -- 172.21.15.7:0/3405453985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71081f6c30 con 0x7f71081583c0 2024-04-25T11:11:41.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.611+0000 7f710e800640 1 -- 172.21.15.7:0/3405453985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71081f7210 con 0x7f71081583c0 2024-04-25T11:11:41.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.612+0000 7f710e800640 1 -- 172.21.15.7:0/3405453985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7108050740 con 0x7f71081583c0 2024-04-25T11:11:41.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.613+0000 7f7105600640 1 -- 172.21.15.7:0/3405453985 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f710001a020 con 0x7f71081583c0 2024-04-25T11:11:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.615+0000 7f7105600640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70e80744e0 0x7f70e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.615+0000 7f7105600640 1 -- 172.21.15.7:0/3405453985 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f7100094e50 con 0x7f71081583c0 2024-04-25T11:11:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.615+0000 7f710d400640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70e80744e0 0x7f70e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:41.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.616+0000 7f710d400640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70e80744e0 0x7f70e80769f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f70fc033000 tx=0x7f70fc009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:41.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.620+0000 7f7105600640 1 -- 172.21.15.7:0/3405453985 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f710005f140 con 0x7f71081583c0 2024-04-25T11:11:41.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.823+0000 7f710e800640 1 -- 172.21.15.7:0/3405453985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f71081f69c0 con 0x7f71081583c0 2024-04-25T11:11:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.830+0000 7f7105600640 1 -- 172.21.15.7:0/3405453985 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f710005ea40 con 0x7f71081583c0 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.833+0000 7f710e800640 1 -- 172.21.15.7:0/3405453985 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70e80744e0 msgr2=0x7f70e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.833+0000 7f710e800640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70e80744e0 0x7f70e80769f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f70fc033000 tx=0x7f70fc009000 comp rx=0 tx=0).stop 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.833+0000 7f710e800640 1 -- 172.21.15.7:0/3405453985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71081583c0 msgr2=0x7f71081ea8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.833+0000 7f710e800640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71081583c0 0x7f71081ea8a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f710000c390 tx=0x7f710000c850 comp rx=0 tx=0).stop 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.833+0000 7f710e800640 1 -- 172.21.15.7:0/3405453985 shutdown_connections 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.833+0000 7f710e800640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70e80744e0 0x7f70e80769f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.833+0000 7f710e800640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71081583c0 0x7f71081ea8a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.833+0000 7f710e800640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71080d09c0 0x7f71081f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.833+0000 7f710e800640 1 --2- 172.21.15.7:0/3405453985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71080cfb50 0x7f7108161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.833+0000 7f710e800640 1 -- 172.21.15.7:0/3405453985 >> 172.21.15.7:0/3405453985 conn(0x7f71080740f0 msgr2=0x7f710815ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.834+0000 7f710e800640 1 -- 172.21.15.7:0/3405453985 shutdown_connections 2024-04-25T11:11:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:41.834+0000 7f710e800640 1 -- 172.21.15.7:0/3405453985 wait complete. 2024-04-25T11:11:41.860 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:11:41.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:11:41.971 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:11:42.082 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:11:42.082 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:11:42.082 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:42.143 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:11:42.143 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:42.199 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:11:42.199 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:42.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.580+0000 7f76a6600640 1 Processor -- start 2024-04-25T11:11:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.580+0000 7f76a6600640 1 -- start start 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.583+0000 7f76a6600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76a00cf930 0x7f76a00b8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.583+0000 7f76a6600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76a00d1c80 0x7f76a00d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.583+0000 7f76a6600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76a00d25d0 0x7f76a00b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.583+0000 7f76a6600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f76a00d18f0 con 0x7f76a00cf930 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.583+0000 7f76a6600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f76a00d1210 con 0x7f76a00d25d0 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.583+0000 7f76a6600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f76a00b9340 con 0x7f76a00d1c80 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.583+0000 7f76a5200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76a00cf930 0x7f76a00b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.583+0000 7f76a5200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76a00cf930 0x7f76a00b8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40292/0 (socket says 172.21.15.7:40292) 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.583+0000 7f76a5200640 1 -- 172.21.15.7:0/1161696727 learned_addr learned my addr 172.21.15.7:0/1161696727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.583+0000 7f769fe00640 1 --2- 172.21.15.7:0/1161696727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76a00d1c80 0x7f76a00d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.584+0000 7f76a5c00640 1 --2- 172.21.15.7:0/1161696727 >> v2:172.21.15.87:3300/0 conn(0x7f76a00d25d0 0x7f76a00b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.584+0000 7f76a5200640 1 -- 172.21.15.7:0/1161696727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76a00d1c80 msgr2=0x7f76a00d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.584+0000 7f76a5200640 1 --2- 172.21.15.7:0/1161696727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76a00d1c80 0x7f76a00d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.584+0000 7f76a5200640 1 -- 172.21.15.7:0/1161696727 >> v2:172.21.15.87:3300/0 conn(0x7f76a00d25d0 msgr2=0x7f76a00b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.584+0000 7f76a5200640 1 --2- 172.21.15.7:0/1161696727 >> v2:172.21.15.87:3300/0 conn(0x7f76a00d25d0 0x7f76a00b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.584+0000 7f76a5200640 1 -- 172.21.15.7:0/1161696727 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a00d1610 con 0x7f76a00cf930 2024-04-25T11:11:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.584+0000 7f769fe00640 1 --2- 172.21.15.7:0/1161696727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76a00d1c80 0x7f76a00d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.584+0000 7f76a5c00640 1 --2- 172.21.15.7:0/1161696727 >> v2:172.21.15.87:3300/0 conn(0x7f76a00d25d0 0x7f76a00b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.585+0000 7f76a5200640 1 --2- 172.21.15.7:0/1161696727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a00cf930 0x7f76a00b8900 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f76a0074670 tx=0x7f769402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd82082971c6fbc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.585+0000 7f769f400640 1 -- 172.21.15.7:0/1161696727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694037070 con 0x7f76a00cf930 2024-04-25T11:11:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.585+0000 7f769f400640 1 -- 172.21.15.7:0/1161696727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7694032440 con 0x7f76a00cf930 2024-04-25T11:11:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.585+0000 7f769f400640 1 -- 172.21.15.7:0/1161696727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7694032810 con 0x7f76a00cf930 2024-04-25T11:11:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.586+0000 7f76a6600640 1 -- 172.21.15.7:0/1161696727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a00cf930 msgr2=0x7f76a00b8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.586+0000 7f76a6600640 1 --2- 172.21.15.7:0/1161696727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a00cf930 0x7f76a00b8900 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f76a0074670 tx=0x7f769402a140 comp rx=0 tx=0).stop 2024-04-25T11:11:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.586+0000 7f76a6600640 1 -- 172.21.15.7:0/1161696727 shutdown_connections 2024-04-25T11:11:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.586+0000 7f76a6600640 1 --2- 172.21.15.7:0/1161696727 >> v2:172.21.15.87:3300/0 conn(0x7f76a00d25d0 0x7f76a00b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.586+0000 7f76a6600640 1 --2- 172.21.15.7:0/1161696727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76a00d1c80 0x7f76a00d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.586+0000 7f76a6600640 1 --2- 172.21.15.7:0/1161696727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a00cf930 0x7f76a00b8900 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.586+0000 7f76a6600640 1 -- 172.21.15.7:0/1161696727 >> 172.21.15.7:0/1161696727 conn(0x7f76a00d0270 msgr2=0x7f76a00d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.586+0000 7f76a6600640 1 -- 172.21.15.7:0/1161696727 shutdown_connections 2024-04-25T11:11:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.586+0000 7f76a6600640 1 -- 172.21.15.7:0/1161696727 wait complete. 2024-04-25T11:11:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.589+0000 7f76a6600640 1 Processor -- start 2024-04-25T11:11:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.589+0000 7f76a6600640 1 -- start start 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f76a6600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76a00cf930 0x7f76a0156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f76a6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a00d1c80 0x7f76a0159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f76a6600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76a00d25d0 0x7f76a015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f76a6600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76a00b7590 con 0x7f76a00d1c80 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f76a6600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f76a00b8fd0 con 0x7f76a00d25d0 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f76a6600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f76a00b7710 con 0x7f76a00cf930 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f769fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a00d1c80 0x7f76a0159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f769fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a00d1c80 0x7f76a0159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40300/0 (socket says 172.21.15.7:40300) 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f769fe00640 1 -- 172.21.15.7:0/4056627670 learned_addr learned my addr 172.21.15.7:0/4056627670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f76a5200640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76a00cf930 0x7f76a0156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f76a5c00640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76a00d25d0 0x7f76a015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f769fe00640 1 -- 172.21.15.7:0/4056627670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76a00cf930 msgr2=0x7f76a0156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.591+0000 7f769fe00640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76a00cf930 0x7f76a0156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.592+0000 7f769fe00640 1 -- 172.21.15.7:0/4056627670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76a00d25d0 msgr2=0x7f76a015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.592+0000 7f769fe00640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76a00d25d0 0x7f76a015ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.592+0000 7f769fe00640 1 -- 172.21.15.7:0/4056627670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a0155d40 con 0x7f76a00d1c80 2024-04-25T11:11:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.592+0000 7f769fe00640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a00d1c80 0x7f76a0159d30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f768c00c7f0 tx=0x7f768c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.592+0000 7f769cc00640 1 -- 172.21.15.7:0/4056627670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c015070 con 0x7f76a00d1c80 2024-04-25T11:11:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.592+0000 7f769cc00640 1 -- 172.21.15.7:0/4056627670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f768c0077c0 con 0x7f76a00d1c80 2024-04-25T11:11:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.592+0000 7f769cc00640 1 -- 172.21.15.7:0/4056627670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c007bb0 con 0x7f76a00d1c80 2024-04-25T11:11:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.592+0000 7f76a6600640 1 -- 172.21.15.7:0/4056627670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a0156020 con 0x7f76a00d1c80 2024-04-25T11:11:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.592+0000 7f76a6600640 1 -- 172.21.15.7:0/4056627670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a01565e0 con 0x7f76a00d1c80 2024-04-25T11:11:42.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.594+0000 7f76a6600640 1 -- 172.21.15.7:0/4056627670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a006f360 con 0x7f76a00d1c80 2024-04-25T11:11:42.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.595+0000 7f769cc00640 1 -- 172.21.15.7:0/4056627670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f768c018070 con 0x7f76a00d1c80 2024-04-25T11:11:42.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.596+0000 7f769cc00640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76800744e0 0x7f76800769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:42.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.596+0000 7f769cc00640 1 -- 172.21.15.7:0/4056627670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f768c095710 con 0x7f76a00d1c80 2024-04-25T11:11:42.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.596+0000 7f76a5200640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76800744e0 0x7f76800769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:42.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.597+0000 7f76a5200640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76800744e0 0x7f76800769f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7694034000 tx=0x7f769402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.601+0000 7f769cc00640 1 -- 172.21.15.7:0/4056627670 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f768c05fa10 con 0x7f76a00d1c80 2024-04-25T11:11:42.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.805+0000 7f76a6600640 1 -- 172.21.15.7:0/4056627670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76a0050740 con 0x7f76a00d1c80 2024-04-25T11:11:42.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.809+0000 7f769cc00640 1 -- 172.21.15.7:0/4056627670 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7f768c05f310 con 0x7f76a00d1c80 2024-04-25T11:11:42.811 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:42.811 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:42.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.811+0000 7f76a6600640 1 -- 172.21.15.7:0/4056627670 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76800744e0 msgr2=0x7f76800769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:42.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.811+0000 7f76a6600640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76800744e0 0x7f76800769f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7694034000 tx=0x7f769402e000 comp rx=0 tx=0).stop 2024-04-25T11:11:42.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.812+0000 7f76a6600640 1 -- 172.21.15.7:0/4056627670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a00d1c80 msgr2=0x7f76a0159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:42.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.812+0000 7f76a6600640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a00d1c80 0x7f76a0159d30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f768c00c7f0 tx=0x7f768c00ccb0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.812+0000 7f76a6600640 1 -- 172.21.15.7:0/4056627670 shutdown_connections 2024-04-25T11:11:42.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.812+0000 7f76a6600640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76800744e0 0x7f76800769f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.812+0000 7f76a6600640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76a00d25d0 0x7f76a015ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.812+0000 7f76a6600640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a00d1c80 0x7f76a0159d30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.812+0000 7f76a6600640 1 --2- 172.21.15.7:0/4056627670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76a00cf930 0x7f76a0156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:42.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.812+0000 7f76a6600640 1 -- 172.21.15.7:0/4056627670 >> 172.21.15.7:0/4056627670 conn(0x7f76a00d0270 msgr2=0x7f76a00716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:42.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.812+0000 7f76a6600640 1 -- 172.21.15.7:0/4056627670 shutdown_connections 2024-04-25T11:11:42.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:42.812+0000 7f76a6600640 1 -- 172.21.15.7:0/4056627670 wait complete. 2024-04-25T11:11:42.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:42.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:43.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.133+0000 7fe546000640 1 Processor -- start 2024-04-25T11:11:43.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.134+0000 7fe546000640 1 -- start start 2024-04-25T11:11:43.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.136+0000 7fe546000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe5400b0470 0x7fe54006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.136+0000 7fe546000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 0x7fe5400aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.136+0000 7fe546000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe54015e3c0 con 0x7fe54015dfb0 2024-04-25T11:11:43.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.136+0000 7fe546000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe5400cf800 con 0x7fe5400b0470 2024-04-25T11:11:43.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.136+0000 7fe546000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe5400d1250 con 0x7fe54006f8e0 2024-04-25T11:11:43.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.137+0000 7fe53fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 0x7fe5400aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.137+0000 7fe53f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe54015dfb0 0x7fe5400d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39808/0 (socket says 172.21.15.7:39808) 2024-04-25T11:11:43.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.137+0000 7fe53f400640 1 -- 172.21.15.7:0/3462825026 learned_addr learned my addr 172.21.15.7:0/3462825026 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:43.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.137+0000 7fe53fe00640 1 -- 172.21.15.7:0/3462825026 >> v2:172.21.15.87:3300/0 conn(0x7fe5400b0470 msgr2=0x7fe54006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:43.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.138+0000 7fe53ea00640 1 --2- 172.21.15.7:0/3462825026 >> v2:172.21.15.87:3300/0 conn(0x7fe5400b0470 0x7fe54006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.138+0000 7fe53fe00640 1 --2- 172.21.15.7:0/3462825026 >> v2:172.21.15.87:3300/0 conn(0x7fe5400b0470 0x7fe54006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.138+0000 7fe53fe00640 1 -- 172.21.15.7:0/3462825026 >> v1:172.21.15.87:6789/0 conn(0x7fe54015dfb0 legacy=0x7fe5400d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.138+0000 7fe53fe00640 1 -- 172.21.15.7:0/3462825026 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5400cfaa0 con 0x7fe54006f8e0 2024-04-25T11:11:43.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.138+0000 7fe53ea00640 1 --2- 172.21.15.7:0/3462825026 >> v2:172.21.15.87:3300/0 conn(0x7fe5400b0470 0x7fe54006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:43.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.139+0000 7fe53fe00640 1 --2- 172.21.15.7:0/3462825026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 0x7fe5400aea90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe5380097a0 tx=0x7fe53802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec99b69c8ce4a391 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.139+0000 7fe53e000640 1 -- 172.21.15.7:0/3462825026 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538037070 con 0x7fe54006f8e0 2024-04-25T11:11:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.139+0000 7fe53e000640 1 -- 172.21.15.7:0/3462825026 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe538032440 con 0x7fe54006f8e0 2024-04-25T11:11:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.139+0000 7fe53e000640 1 -- 172.21.15.7:0/3462825026 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538032810 con 0x7fe54006f8e0 2024-04-25T11:11:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.140+0000 7fe546000640 1 -- 172.21.15.7:0/3462825026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 msgr2=0x7fe5400aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.140+0000 7fe546000640 1 --2- 172.21.15.7:0/3462825026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 0x7fe5400aea90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe5380097a0 tx=0x7fe53802a180 comp rx=0 tx=0).stop 2024-04-25T11:11:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.140+0000 7fe546000640 1 -- 172.21.15.7:0/3462825026 shutdown_connections 2024-04-25T11:11:43.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.140+0000 7fe546000640 1 --2- 172.21.15.7:0/3462825026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 0x7fe5400aea90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.140+0000 7fe546000640 1 --2- 172.21.15.7:0/3462825026 >> v2:172.21.15.87:3300/0 conn(0x7fe5400b0470 0x7fe54006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.140+0000 7fe546000640 1 -- 172.21.15.7:0/3462825026 >> 172.21.15.7:0/3462825026 conn(0x7fe5400742c0 msgr2=0x7fe54006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.140+0000 7fe546000640 1 -- 172.21.15.7:0/3462825026 shutdown_connections 2024-04-25T11:11:43.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.140+0000 7fe546000640 1 -- 172.21.15.7:0/3462825026 wait complete. 2024-04-25T11:11:43.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.143+0000 7fe546000640 1 Processor -- start 2024-04-25T11:11:43.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.143+0000 7fe546000640 1 -- start start 2024-04-25T11:11:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.144+0000 7fe546000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 0x7fe540148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.144+0000 7fe546000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5400b0470 0x7fe540155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.144+0000 7fe546000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe54015dfb0 0x7fe5401564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.144+0000 7fe546000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5400d0340 con 0x7fe54015dfb0 2024-04-25T11:11:43.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.144+0000 7fe546000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe5400d0060 con 0x7fe5400b0470 2024-04-25T11:11:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.144+0000 7fe546000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe5400d0620 con 0x7fe54006f8e0 2024-04-25T11:11:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.145+0000 7fe53f400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 0x7fe540148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.145+0000 7fe53f400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 0x7fe540148dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:50888/0 (socket says 172.21.15.7:50888) 2024-04-25T11:11:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.145+0000 7fe53f400640 1 -- 172.21.15.7:0/3762894927 learned_addr learned my addr 172.21.15.7:0/3762894927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.145+0000 7fe53fe00640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe54015dfb0 0x7fe5401564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.145+0000 7fe53ea00640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5400b0470 0x7fe540155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.145+0000 7fe53f400640 1 -- 172.21.15.7:0/3762894927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5400b0470 msgr2=0x7fe540155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.145+0000 7fe53f400640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5400b0470 0x7fe540155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.145+0000 7fe53f400640 1 -- 172.21.15.7:0/3762894927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe54015dfb0 msgr2=0x7fe5401564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.145+0000 7fe53f400640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe54015dfb0 0x7fe5401564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.145+0000 7fe53f400640 1 -- 172.21.15.7:0/3762894927 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5400cd1d0 con 0x7fe54006f8e0 2024-04-25T11:11:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.146+0000 7fe53f400640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 0x7fe540148dd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe53000cb40 tx=0x7fe53000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.146+0000 7fe537400640 1 -- 172.21.15.7:0/3762894927 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530017070 con 0x7fe54006f8e0 2024-04-25T11:11:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.146+0000 7fe537400640 1 -- 172.21.15.7:0/3762894927 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53000aca0 con 0x7fe54006f8e0 2024-04-25T11:11:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.146+0000 7fe546000640 1 -- 172.21.15.7:0/3762894927 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5400b6400 con 0x7fe54006f8e0 2024-04-25T11:11:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.146+0000 7fe537400640 1 -- 172.21.15.7:0/3762894927 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530012600 con 0x7fe54006f8e0 2024-04-25T11:11:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.146+0000 7fe546000640 1 -- 172.21.15.7:0/3762894927 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5400b69c0 con 0x7fe54006f8e0 2024-04-25T11:11:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.147+0000 7fe546000640 1 -- 172.21.15.7:0/3762894927 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe540050740 con 0x7fe54006f8e0 2024-04-25T11:11:43.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.149+0000 7fe537400640 1 -- 172.21.15.7:0/3762894927 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe53001a070 con 0x7fe54006f8e0 2024-04-25T11:11:43.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.150+0000 7fe537400640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe51c0744e0 0x7fe51c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.150+0000 7fe537400640 1 -- 172.21.15.7:0/3762894927 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fe5300957e0 con 0x7fe54006f8e0 2024-04-25T11:11:43.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.151+0000 7fe53ea00640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe51c0744e0 0x7fe51c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.151+0000 7fe53ea00640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe51c0744e0 0x7fe51c0769f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe540149b50 tx=0x7fe528008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.166+0000 7fe537400640 1 -- 172.21.15.7:0/3762894927 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe53005f510 con 0x7fe54006f8e0 2024-04-25T11:11:43.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.370+0000 7fe546000640 1 -- 172.21.15.7:0/3762894927 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe540070fb0 con 0x7fe54006f8e0 2024-04-25T11:11:43.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.373+0000 7fe537400640 1 -- 172.21.15.7:0/3762894927 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fe530063810 con 0x7fe54006f8e0 2024-04-25T11:11:43.373 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:43.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.375+0000 7fe546000640 1 -- 172.21.15.7:0/3762894927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe51c0744e0 msgr2=0x7fe51c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.375+0000 7fe546000640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe51c0744e0 0x7fe51c0769f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe540149b50 tx=0x7fe528008040 comp rx=0 tx=0).stop 2024-04-25T11:11:43.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.375+0000 7fe546000640 1 -- 172.21.15.7:0/3762894927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 msgr2=0x7fe540148dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.375+0000 7fe546000640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 0x7fe540148dd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe53000cb40 tx=0x7fe53000a060 comp rx=0 tx=0).stop 2024-04-25T11:11:43.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.375+0000 7fe546000640 1 -- 172.21.15.7:0/3762894927 shutdown_connections 2024-04-25T11:11:43.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.375+0000 7fe546000640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe51c0744e0 0x7fe51c0769f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.375+0000 7fe546000640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe54015dfb0 0x7fe5401564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.375+0000 7fe546000640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5400b0470 0x7fe540155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.375+0000 7fe546000640 1 --2- 172.21.15.7:0/3762894927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe54006f8e0 0x7fe540148dd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.375+0000 7fe546000640 1 -- 172.21.15.7:0/3762894927 >> 172.21.15.7:0/3762894927 conn(0x7fe5400742c0 msgr2=0x7fe54006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:43.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.376+0000 7fe546000640 1 -- 172.21.15.7:0/3762894927 shutdown_connections 2024-04-25T11:11:43.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.376+0000 7fe546000640 1 -- 172.21.15.7:0/3762894927 wait complete. 2024-04-25T11:11:43.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:43.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.690+0000 7fdd9f200640 1 Processor -- start 2024-04-25T11:11:43.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.690+0000 7fdd9f200640 1 -- start start 2024-04-25T11:11:43.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.693+0000 7fdd9f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdda01583c0 0x7fdda00cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.693+0000 7fdd9f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdda00cfb50 0x7fdda00d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.693+0000 7fdd9f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdda00ce570 con 0x7fdda00d09c0 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.693+0000 7fdd9f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fdda015f190 con 0x7fdda01583c0 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.693+0000 7fdd9f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fdda00ce850 con 0x7fdda00cfb50 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.693+0000 7fdd9de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdda01583c0 0x7fdda00cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.693+0000 7fdd9e800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdda00d09c0 0x7fdda006eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59178/0 (socket says 172.21.15.7:59178) 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.693+0000 7fdd9e800640 1 -- 172.21.15.7:0/3929077863 learned_addr learned my addr 172.21.15.7:0/3929077863 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.693+0000 7fdd9d400640 1 --2- 172.21.15.7:0/3929077863 >> v2:172.21.15.87:3300/0 conn(0x7fdda00cfb50 0x7fdda00d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.694+0000 7fdd9de00640 1 -- 172.21.15.7:0/3929077863 >> v2:172.21.15.87:3300/0 conn(0x7fdda00cfb50 msgr2=0x7fdda00d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.694+0000 7fdd9de00640 1 --2- 172.21.15.7:0/3929077863 >> v2:172.21.15.87:3300/0 conn(0x7fdda00cfb50 0x7fdda00d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.694+0000 7fdd9de00640 1 -- 172.21.15.7:0/3929077863 >> v1:172.21.15.7:6789/0 conn(0x7fdda00d09c0 legacy=0x7fdda006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.694+0000 7fdd9de00640 1 -- 172.21.15.7:0/3929077863 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda00ceb30 con 0x7fdda01583c0 2024-04-25T11:11:43.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.694+0000 7fdd9ca00640 1 -- 172.21.15.7:0/3929077863 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982620464 0 0) 0x7fdda00ce570 con 0x7fdda00d09c0 2024-04-25T11:11:43.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.695+0000 7fdd9de00640 1 --2- 172.21.15.7:0/3929077863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda01583c0 0x7fdda00cf610 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdda0070ad0 tx=0x7fdd9002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bc8884705db438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:43.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.695+0000 7fdd9ca00640 1 -- 172.21.15.7:0/3929077863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90037070 con 0x7fdda01583c0 2024-04-25T11:11:43.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.695+0000 7fdd9ca00640 1 -- 172.21.15.7:0/3929077863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd90032440 con 0x7fdda01583c0 2024-04-25T11:11:43.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.695+0000 7fdd9ca00640 1 -- 172.21.15.7:0/3929077863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9002ae60 con 0x7fdda01583c0 2024-04-25T11:11:43.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.696+0000 7fdd9f200640 1 -- 172.21.15.7:0/3929077863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda01583c0 msgr2=0x7fdda00cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.696+0000 7fdd9f200640 1 --2- 172.21.15.7:0/3929077863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda01583c0 0x7fdda00cf610 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdda0070ad0 tx=0x7fdd9002a110 comp rx=0 tx=0).stop 2024-04-25T11:11:43.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.696+0000 7fdd9f200640 1 -- 172.21.15.7:0/3929077863 shutdown_connections 2024-04-25T11:11:43.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.696+0000 7fdd9f200640 1 --2- 172.21.15.7:0/3929077863 >> v2:172.21.15.87:3300/0 conn(0x7fdda00cfb50 0x7fdda00d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.696+0000 7fdd9f200640 1 --2- 172.21.15.7:0/3929077863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda01583c0 0x7fdda00cf610 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.696+0000 7fdd9f200640 1 -- 172.21.15.7:0/3929077863 >> 172.21.15.7:0/3929077863 conn(0x7fdda00740f0 msgr2=0x7fdda006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:43.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.696+0000 7fdd9f200640 1 -- 172.21.15.7:0/3929077863 shutdown_connections 2024-04-25T11:11:43.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.696+0000 7fdd9f200640 1 -- 172.21.15.7:0/3929077863 wait complete. 2024-04-25T11:11:43.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.698+0000 7fdd9f200640 1 Processor -- start 2024-04-25T11:11:43.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.699+0000 7fdd9f200640 1 -- start start 2024-04-25T11:11:43.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.700+0000 7fdd9f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdda00cfb50 0x7fdda01e9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.700+0000 7fdd9f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda00d09c0 0x7fdda01e8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.700+0000 7fdd9f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda01583c0 0x7fdda01e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.700+0000 7fdd9f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdda006f5b0 con 0x7fdda00d09c0 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.700+0000 7fdd9f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdda00cf0f0 con 0x7fdda01583c0 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.700+0000 7fdd9f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdda006f730 con 0x7fdda00cfb50 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.700+0000 7fdd9d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda00d09c0 0x7fdda01e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.700+0000 7fdd9d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda00d09c0 0x7fdda01e8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40326/0 (socket says 172.21.15.7:40326) 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.700+0000 7fdd9d400640 1 -- 172.21.15.7:0/2769886853 learned_addr learned my addr 172.21.15.7:0/2769886853 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd9e800640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda01583c0 0x7fdda01e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd9d400640 1 -- 172.21.15.7:0/2769886853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdda00cfb50 msgr2=0x7fdda01e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd9de00640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdda00cfb50 0x7fdda01e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd9d400640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdda00cfb50 0x7fdda01e9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd9d400640 1 -- 172.21.15.7:0/2769886853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda01583c0 msgr2=0x7fdda01e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd9d400640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda01583c0 0x7fdda01e8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd9d400640 1 -- 172.21.15.7:0/2769886853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda0152dc0 con 0x7fdda00d09c0 2024-04-25T11:11:43.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd9e800640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda01583c0 0x7fdda01e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:43.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd9de00640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdda00cfb50 0x7fdda01e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:43.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd9d400640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda00d09c0 0x7fdda01e8660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00d5a0 tx=0x7fdd8c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:43.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd96000640 1 -- 172.21.15.7:0/2769886853 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c019070 con 0x7fdda00d09c0 2024-04-25T11:11:43.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd96000640 1 -- 172.21.15.7:0/2769886853 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8c004240 con 0x7fdda00d09c0 2024-04-25T11:11:43.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd96000640 1 -- 172.21.15.7:0/2769886853 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c003610 con 0x7fdda00d09c0 2024-04-25T11:11:43.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.701+0000 7fdd9f200640 1 -- 172.21.15.7:0/2769886853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda0150c30 con 0x7fdda00d09c0 2024-04-25T11:11:43.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.702+0000 7fdd9f200640 1 -- 172.21.15.7:0/2769886853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda0153360 con 0x7fdda00d09c0 2024-04-25T11:11:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.703+0000 7fdd9f200640 1 -- 172.21.15.7:0/2769886853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda0050740 con 0x7fdda00d09c0 2024-04-25T11:11:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.704+0000 7fdd96000640 1 -- 172.21.15.7:0/2769886853 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdd8c01c070 con 0x7fdda00d09c0 2024-04-25T11:11:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.705+0000 7fdd96000640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd800744b0 0x7fdd800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.706+0000 7fdd96000640 1 -- 172.21.15.7:0/2769886853 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fdd8c094f30 con 0x7fdda00d09c0 2024-04-25T11:11:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.706+0000 7fdd9de00640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd800744b0 0x7fdd800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.706+0000 7fdd9de00640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd800744b0 0x7fdd800769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd90034000 tx=0x7fdd9002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.710+0000 7fdd96000640 1 -- 172.21.15.7:0/2769886853 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd8c05ec10 con 0x7fdda00d09c0 2024-04-25T11:11:43.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.912+0000 7fdd9f200640 1 -- 172.21.15.7:0/2769886853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdda0143d00 con 0x7fdda00d09c0 2024-04-25T11:11:43.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.913+0000 7fdd96000640 1 -- 172.21.15.7:0/2769886853 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7fdd8c062f10 con 0x7fdda00d09c0 2024-04-25T11:11:43.914 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 -- 172.21.15.7:0/2769886853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd800744b0 msgr2=0x7fdd800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd800744b0 0x7fdd800769c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd90034000 tx=0x7fdd9002e000 comp rx=0 tx=0).stop 2024-04-25T11:11:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 -- 172.21.15.7:0/2769886853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda00d09c0 msgr2=0x7fdda01e8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:43.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda00d09c0 0x7fdda01e8660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00d5a0 tx=0x7fdd8c00da60 comp rx=0 tx=0).stop 2024-04-25T11:11:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 -- 172.21.15.7:0/2769886853 shutdown_connections 2024-04-25T11:11:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd800744b0 0x7fdd800769c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdda01583c0 0x7fdda01e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdda00d09c0 0x7fdda01e8660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 --2- 172.21.15.7:0/2769886853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdda00cfb50 0x7fdda01e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 -- 172.21.15.7:0/2769886853 >> 172.21.15.7:0/2769886853 conn(0x7fdda00740f0 msgr2=0x7fdda006d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 -- 172.21.15.7:0/2769886853 shutdown_connections 2024-04-25T11:11:43.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:43.916+0000 7fdd9f200640 1 -- 172.21.15.7:0/2769886853 wait complete. 2024-04-25T11:11:43.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:44.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.221+0000 7fcb2ea00640 1 Processor -- start 2024-04-25T11:11:44.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.222+0000 7fcb2ea00640 1 -- start start 2024-04-25T11:11:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.224+0000 7fcb2ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb28070630 0x7fcb280cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.224+0000 7fcb2ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb280d09d0 0x7fcb2806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.224+0000 7fcb2ea00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb280b0be0 con 0x7fcb2815ecc0 2024-04-25T11:11:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.224+0000 7fcb2ea00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fcb280ce9e0 con 0x7fcb28070630 2024-04-25T11:11:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.224+0000 7fcb2ea00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcb280cecc0 con 0x7fcb280d09d0 2024-04-25T11:11:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.224+0000 7fcb2d600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb28070630 0x7fcb280cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.225+0000 7fcb2d600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb28070630 0x7fcb280cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40336/0 (socket says 172.21.15.7:40336) 2024-04-25T11:11:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.225+0000 7fcb2d600640 1 -- 172.21.15.7:0/2912163175 learned_addr learned my addr 172.21.15.7:0/2912163175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:44.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.225+0000 7fcb2cc00640 1 --1- 172.21.15.7:0/2912163175 >> v1:172.21.15.7:6789/0 conn(0x7fcb2815ecc0 0x7fcb280d02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59188/0 (socket says 172.21.15.7:59188) 2024-04-25T11:11:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.225+0000 7fcb2e000640 1 --2- 172.21.15.7:0/2912163175 >> v2:172.21.15.87:3300/0 conn(0x7fcb280d09d0 0x7fcb2806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.225+0000 7fcb2d600640 1 -- 172.21.15.7:0/2912163175 >> v2:172.21.15.87:3300/0 conn(0x7fcb280d09d0 msgr2=0x7fcb2806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.225+0000 7fcb2d600640 1 --2- 172.21.15.7:0/2912163175 >> v2:172.21.15.87:3300/0 conn(0x7fcb280d09d0 0x7fcb2806eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.225+0000 7fcb2d600640 1 -- 172.21.15.7:0/2912163175 >> v1:172.21.15.7:6789/0 conn(0x7fcb2815ecc0 legacy=0x7fcb280d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.225+0000 7fcb27e00640 1 -- 172.21.15.7:0/2912163175 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 104289220 0 0) 0x7fcb280b0be0 con 0x7fcb2815ecc0 2024-04-25T11:11:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.225+0000 7fcb2d600640 1 -- 172.21.15.7:0/2912163175 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb280cefa0 con 0x7fcb28070630 2024-04-25T11:11:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.226+0000 7fcb2d600640 1 --2- 172.21.15.7:0/2912163175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb28070630 0x7fcb280cfa80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcb280730b0 tx=0x7fcb1c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3414ea79c3d0e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.226+0000 7fcb27e00640 1 -- 172.21.15.7:0/2912163175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c037070 con 0x7fcb28070630 2024-04-25T11:11:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.226+0000 7fcb27e00640 1 -- 172.21.15.7:0/2912163175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb1c032440 con 0x7fcb28070630 2024-04-25T11:11:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.226+0000 7fcb27e00640 1 -- 172.21.15.7:0/2912163175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c032810 con 0x7fcb28070630 2024-04-25T11:11:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.227+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2912163175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb28070630 msgr2=0x7fcb280cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.227+0000 7fcb2ea00640 1 --2- 172.21.15.7:0/2912163175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb28070630 0x7fcb280cfa80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcb280730b0 tx=0x7fcb1c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.227+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2912163175 shutdown_connections 2024-04-25T11:11:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.227+0000 7fcb2ea00640 1 --2- 172.21.15.7:0/2912163175 >> v2:172.21.15.87:3300/0 conn(0x7fcb280d09d0 0x7fcb2806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.227+0000 7fcb2ea00640 1 --2- 172.21.15.7:0/2912163175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb28070630 0x7fcb280cfa80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.227+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2912163175 >> 172.21.15.7:0/2912163175 conn(0x7fcb28074140 msgr2=0x7fcb2806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.227+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2912163175 shutdown_connections 2024-04-25T11:11:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.227+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2912163175 wait complete. 2024-04-25T11:11:44.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.230+0000 7fcb2ea00640 1 Processor -- start 2024-04-25T11:11:44.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.230+0000 7fcb2ea00640 1 -- start start 2024-04-25T11:11:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.232+0000 7fcb2ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb28070630 0x7fcb28147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.232+0000 7fcb2ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb2815ecc0 0x7fcb281f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.232+0000 7fcb2ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb281f66a0 0x7fcb2806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.232+0000 7fcb2ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2815f0d0 con 0x7fcb28070630 2024-04-25T11:11:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.232+0000 7fcb2ea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcb280cf560 con 0x7fcb281f66a0 2024-04-25T11:11:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.232+0000 7fcb2ea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcb281f6cb0 con 0x7fcb2815ecc0 2024-04-25T11:11:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.232+0000 7fcb2d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb28070630 0x7fcb28147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.232+0000 7fcb2d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb28070630 0x7fcb28147ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40340/0 (socket says 172.21.15.7:40340) 2024-04-25T11:11:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.232+0000 7fcb2d600640 1 -- 172.21.15.7:0/2392137240 learned_addr learned my addr 172.21.15.7:0/2392137240 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:44.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.233+0000 7fcb2d600640 1 -- 172.21.15.7:0/2392137240 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb2815ecc0 msgr2=0x7fcb281f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.233+0000 7fcb2d600640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb2815ecc0 0x7fcb281f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.233+0000 7fcb2d600640 1 -- 172.21.15.7:0/2392137240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb281f66a0 msgr2=0x7fcb2806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.233+0000 7fcb2e000640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb281f66a0 0x7fcb2806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.233+0000 7fcb2d600640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb281f66a0 0x7fcb2806c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.233+0000 7fcb2d600640 1 -- 172.21.15.7:0/2392137240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb28148020 con 0x7fcb28070630 2024-04-25T11:11:44.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.233+0000 7fcb2d600640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb28070630 0x7fcb28147ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c000c00 tx=0x7fcb1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.233+0000 7fcb25600640 1 -- 172.21.15.7:0/2392137240 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c041020 con 0x7fcb28070630 2024-04-25T11:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.233+0000 7fcb25600640 1 -- 172.21.15.7:0/2392137240 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb1c0306c0 con 0x7fcb28070630 2024-04-25T11:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.234+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2392137240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb281f6f00 con 0x7fcb28070630 2024-04-25T11:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.234+0000 7fcb25600640 1 -- 172.21.15.7:0/2392137240 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c037040 con 0x7fcb28070630 2024-04-25T11:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.234+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2392137240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2814a8c0 con 0x7fcb28070630 2024-04-25T11:11:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.235+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2392137240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb280d0f10 con 0x7fcb28070630 2024-04-25T11:11:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.236+0000 7fcb25600640 1 -- 172.21.15.7:0/2392137240 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcb1c044020 con 0x7fcb28070630 2024-04-25T11:11:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.237+0000 7fcb25600640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb080744b0 0x7fcb080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.237+0000 7fcb25600640 1 -- 172.21.15.7:0/2392137240 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fcb1c0b5ef0 con 0x7fcb28070630 2024-04-25T11:11:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.237+0000 7fcb2cc00640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb080744b0 0x7fcb080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.238+0000 7fcb2cc00640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb080744b0 0x7fcb080769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb14007fd0 tx=0x7fcb14007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.241+0000 7fcb25600640 1 -- 172.21.15.7:0/2392137240 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb280d0f10 con 0x7fcb28070630 2024-04-25T11:11:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.445+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2392137240 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb280ce590 con 0x7fcb28070630 2024-04-25T11:11:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.445+0000 7fcb25600640 1 -- 172.21.15.7:0/2392137240 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fcb280ce590 con 0x7fcb28070630 2024-04-25T11:11:44.447 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2392137240 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb080744b0 msgr2=0x7fcb080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb080744b0 0x7fcb080769c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb14007fd0 tx=0x7fcb14007960 comp rx=0 tx=0).stop 2024-04-25T11:11:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2392137240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb28070630 msgr2=0x7fcb28147ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb28070630 0x7fcb28147ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c000c00 tx=0x7fcb1c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2392137240 shutdown_connections 2024-04-25T11:11:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb080744b0 0x7fcb080769c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb281f66a0 0x7fcb2806c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb2815ecc0 0x7fcb281f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 --2- 172.21.15.7:0/2392137240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb28070630 0x7fcb28147ae0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2392137240 >> 172.21.15.7:0/2392137240 conn(0x7fcb28074140 msgr2=0x7fcb280b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2392137240 shutdown_connections 2024-04-25T11:11:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.448+0000 7fcb2ea00640 1 -- 172.21.15.7:0/2392137240 wait complete. 2024-04-25T11:11:44.477 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:44.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.762+0000 7f5507200640 1 Processor -- start 2024-04-25T11:11:44.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.762+0000 7f5507200640 1 -- start start 2024-04-25T11:11:44.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.765+0000 7f5507200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55081583c0 0x7f55080cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.765+0000 7f5507200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55080ce570 con 0x7f55080cfb50 2024-04-25T11:11:44.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.765+0000 7f5507200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55080ce850 con 0x7f550806eea0 2024-04-25T11:11:44.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.765+0000 7f5507200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f550815f190 con 0x7f55081583c0 2024-04-25T11:11:44.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.765+0000 7f54fd400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f55080cfb50 0x7f55080d0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59204/0 (socket says 172.21.15.7:59204) 2024-04-25T11:11:44.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.765+0000 7f54fd400640 1 -- 172.21.15.7:0/2190829971 learned_addr learned my addr 172.21.15.7:0/2190829971 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505e00640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55081583c0 0x7f55080cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 818600864 0 0) 0x7f55080ce570 con 0x7f55080cfb50 2024-04-25T11:11:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54ec003520 con 0x7f55080cfb50 2024-04-25T11:11:44.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 875896184 0 0) 0x7f54ec003520 con 0x7f55080cfb50 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55081583c0 msgr2=0x7f55080cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55081583c0 0x7f55080cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 >> v1:172.21.15.87:6789/0 conn(0x7f550806eea0 legacy=0x7f55080b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55080ceb30 con 0x7f55080cfb50 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f54f0001670 con 0x7f55080cfb50 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 >> v1:172.21.15.7:6789/0 conn(0x7f55080cfb50 legacy=0x7f55080d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54ec0039e0 0x7f54ec003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55081583c0 0x7f54ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ec004d40 0x7f54ec0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f550815f190 con 0x7f54ec0039e0 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec00bb50 con 0x7f54ec004d40 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.766+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f54ec00bd50 con 0x7f55081583c0 2024-04-25T11:11:44.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.767+0000 7f5505e00640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54ec0039e0 0x7f54ec003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.767+0000 7f54fd400640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55081583c0 0x7f54ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.767+0000 7f5506800640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ec004d40 0x7f54ec0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.767+0000 7f5505e00640 1 -- 172.21.15.7:0/2190829971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55081583c0 msgr2=0x7f54ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:44.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.767+0000 7f5505e00640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55081583c0 0x7f54ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.767+0000 7f5505e00640 1 -- 172.21.15.7:0/2190829971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ec004d40 msgr2=0x7f54ec0051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:44.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.767+0000 7f5505e00640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ec004d40 0x7f54ec0051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.767+0000 7f5505e00640 1 -- 172.21.15.7:0/2190829971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec003520 con 0x7f54ec0039e0 2024-04-25T11:11:44.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.768+0000 7f5505e00640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54ec0039e0 0x7f54ec003e50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f55080b43c0 tx=0x7f54ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cab08b12f0893e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:44.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.769+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8009c10 con 0x7f54ec0039e0 2024-04-25T11:11:44.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.769+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f8036070 con 0x7f54ec0039e0 2024-04-25T11:11:44.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.769+0000 7f5505400640 1 -- 172.21.15.7:0/2190829971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80317f0 con 0x7f54ec0039e0 2024-04-25T11:11:44.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.770+0000 7f5507200640 1 -- 172.21.15.7:0/2190829971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54ec0039e0 msgr2=0x7f54ec003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.770+0000 7f5507200640 1 --2- 172.21.15.7:0/2190829971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54ec0039e0 0x7f54ec003e50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f55080b43c0 tx=0x7f54ec005830 comp rx=0 tx=0).stop 2024-04-25T11:11:44.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.770+0000 7f5506800640 1 -- 172.21.15.7:0/2190829971 reap_dead start 2024-04-25T11:11:44.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.770+0000 7f5507200640 1 -- 172.21.15.7:0/2190829971 shutdown_connections 2024-04-25T11:11:44.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.770+0000 7f5507200640 1 -- 172.21.15.7:0/2190829971 >> 172.21.15.7:0/2190829971 conn(0x7f55080740f0 msgr2=0x7f550806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:44.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.770+0000 7f5507200640 1 -- 172.21.15.7:0/2190829971 shutdown_connections 2024-04-25T11:11:44.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.770+0000 7f5507200640 1 -- 172.21.15.7:0/2190829971 wait complete. 2024-04-25T11:11:44.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.772+0000 7f5507200640 1 Processor -- start 2024-04-25T11:11:44.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.772+0000 7f5507200640 1 -- start start 2024-04-25T11:11:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.774+0000 7f5507200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ec0039e0 0x7f5508140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.774+0000 7f5507200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5508144b60 0x7f5508155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.774+0000 7f5507200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55080b5a50 0x7f55080b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.774+0000 7f5507200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f55080d0b90 con 0x7f5508144b60 2024-04-25T11:11:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.774+0000 7f5507200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f55080cf0f0 con 0x7f54ec0039e0 2024-04-25T11:11:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.774+0000 7f5507200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f55080b0e70 con 0x7f55080b5a50 2024-04-25T11:11:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.774+0000 7f54fd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5508144b60 0x7f5508155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.774+0000 7f54fd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5508144b60 0x7f5508155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40356/0 (socket says 172.21.15.7:40356) 2024-04-25T11:11:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.774+0000 7f54fd400640 1 -- 172.21.15.7:0/3437648152 learned_addr learned my addr 172.21.15.7:0/3437648152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.775+0000 7f5505e00640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ec0039e0 0x7f5508140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.775+0000 7f5506800640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55080b5a50 0x7f55080b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.775+0000 7f54fd400640 1 -- 172.21.15.7:0/3437648152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55080b5a50 msgr2=0x7f55080b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.775+0000 7f54fd400640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55080b5a50 0x7f55080b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.775+0000 7f54fd400640 1 -- 172.21.15.7:0/3437648152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ec0039e0 msgr2=0x7f5508140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.775+0000 7f54fd400640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ec0039e0 0x7f5508140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.775+0000 7f54fd400640 1 -- 172.21.15.7:0/3437648152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5508154360 con 0x7f5508144b60 2024-04-25T11:11:44.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.775+0000 7f54fd400640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5508144b60 0x7f5508155120 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e790 tx=0x7f54f000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.776+0000 7f54fea00640 1 -- 172.21.15.7:0/3437648152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f000eea0 con 0x7f5508144b60 2024-04-25T11:11:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.776+0000 7f54fea00640 1 -- 172.21.15.7:0/3437648152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f0018070 con 0x7f5508144b60 2024-04-25T11:11:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.776+0000 7f54fea00640 1 -- 172.21.15.7:0/3437648152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f000caf0 con 0x7f5508144b60 2024-04-25T11:11:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.776+0000 7f5507200640 1 -- 172.21.15.7:0/3437648152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5508154640 con 0x7f5508144b60 2024-04-25T11:11:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.776+0000 7f5507200640 1 -- 172.21.15.7:0/3437648152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5508154c00 con 0x7f5508144b60 2024-04-25T11:11:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.777+0000 7f5507200640 1 -- 172.21.15.7:0/3437648152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5508050740 con 0x7f5508144b60 2024-04-25T11:11:44.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.778+0000 7f54fea00640 1 -- 172.21.15.7:0/3437648152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f54f000cc90 con 0x7f5508144b60 2024-04-25T11:11:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.779+0000 7f54fea00640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54e40744b0 0x7f54e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.780+0000 7f54fea00640 1 -- 172.21.15.7:0/3437648152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f54f0095460 con 0x7f5508144b60 2024-04-25T11:11:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.780+0000 7f5505e00640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54e40744b0 0x7f54e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.781+0000 7f5505e00640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54e40744b0 0x7f54e40769c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f55080b43c0 tx=0x7f54f802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:44.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.784+0000 7f54fea00640 1 -- 172.21.15.7:0/3437648152 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54f005f880 con 0x7f5508144b60 2024-04-25T11:11:44.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.987+0000 7f5507200640 1 -- 172.21.15.7:0/3437648152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f55080b6540 con 0x7f5508144b60 2024-04-25T11:11:44.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.989+0000 7f54fea00640 1 -- 172.21.15.7:0/3437648152 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f54f005f180 con 0x7f5508144b60 2024-04-25T11:11:44.990 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 -- 172.21.15.7:0/3437648152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54e40744b0 msgr2=0x7f54e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54e40744b0 0x7f54e40769c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f55080b43c0 tx=0x7f54f802f000 comp rx=0 tx=0).stop 2024-04-25T11:11:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 -- 172.21.15.7:0/3437648152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5508144b60 msgr2=0x7f5508155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5508144b60 0x7f5508155120 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f54f000e790 tx=0x7f54f000ec50 comp rx=0 tx=0).stop 2024-04-25T11:11:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 -- 172.21.15.7:0/3437648152 shutdown_connections 2024-04-25T11:11:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54e40744b0 0x7f54e40769c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55080b5a50 0x7f55080b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5508144b60 0x7f5508155120 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 --2- 172.21.15.7:0/3437648152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54ec0039e0 0x7f5508140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 -- 172.21.15.7:0/3437648152 >> 172.21.15.7:0/3437648152 conn(0x7f55080740f0 msgr2=0x7f55080b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 -- 172.21.15.7:0/3437648152 shutdown_connections 2024-04-25T11:11:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:44.992+0000 7f5507200640 1 -- 172.21.15.7:0/3437648152 wait complete. 2024-04-25T11:11:45.020 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:11:45.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.301+0000 7fec35600640 1 Processor -- start 2024-04-25T11:11:45.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.301+0000 7fec35600640 1 -- start start 2024-04-25T11:11:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.304+0000 7fec35600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec30075d30 0x7fec30149af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.304+0000 7fec35600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fec300744d0 0x7fec3006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.304+0000 7fec35600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fec3006dac0 0x7fec30163b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.304+0000 7fec35600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fec3006ded0 con 0x7fec300744d0 2024-04-25T11:11:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.304+0000 7fec35600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fec300d0de0 con 0x7fec3006dac0 2024-04-25T11:11:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.304+0000 7fec35600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fec300d10c0 con 0x7fec30075d30 2024-04-25T11:11:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.305+0000 7fec2f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fec300744d0 0x7fec3006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.305+0000 7fec2f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fec300744d0 0x7fec3006d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40366/0 (socket says 172.21.15.7:40366) 2024-04-25T11:11:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.305+0000 7fec2f400640 1 -- 172.21.15.7:0/1440881321 learned_addr learned my addr 172.21.15.7:0/1440881321 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.305+0000 7fec2fe00640 1 --2- 172.21.15.7:0/1440881321 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec30075d30 0x7fec30149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:45.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.305+0000 7fec34c00640 1 --2- 172.21.15.7:0/1440881321 >> v2:172.21.15.87:3300/0 conn(0x7fec3006dac0 0x7fec30163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:45.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.305+0000 7fec2f400640 1 -- 172.21.15.7:0/1440881321 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec30075d30 msgr2=0x7fec30149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:45.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.305+0000 7fec2f400640 1 --2- 172.21.15.7:0/1440881321 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec30075d30 0x7fec30149af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.305+0000 7fec2f400640 1 -- 172.21.15.7:0/1440881321 >> v2:172.21.15.87:3300/0 conn(0x7fec3006dac0 msgr2=0x7fec30163b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:45.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.305+0000 7fec2f400640 1 --2- 172.21.15.7:0/1440881321 >> v2:172.21.15.87:3300/0 conn(0x7fec3006dac0 0x7fec30163b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.305+0000 7fec2f400640 1 -- 172.21.15.7:0/1440881321 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec300d13a0 con 0x7fec300744d0 2024-04-25T11:11:45.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.307+0000 7fec2f400640 1 --2- 172.21.15.7:0/1440881321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec300744d0 0x7fec3006d580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fec30164c10 tx=0x7fec1c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d1df4a91f01355 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:45.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.307+0000 7fec2ea00640 1 -- 172.21.15.7:0/1440881321 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c037070 con 0x7fec300744d0 2024-04-25T11:11:45.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.307+0000 7fec2ea00640 1 -- 172.21.15.7:0/1440881321 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1c02ae00 con 0x7fec300744d0 2024-04-25T11:11:45.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.307+0000 7fec2ea00640 1 -- 172.21.15.7:0/1440881321 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c032660 con 0x7fec300744d0 2024-04-25T11:11:45.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.308+0000 7fec35600640 1 -- 172.21.15.7:0/1440881321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec300744d0 msgr2=0x7fec3006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.308+0000 7fec35600640 1 --2- 172.21.15.7:0/1440881321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec300744d0 0x7fec3006d580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fec30164c10 tx=0x7fec1c02a0b0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.308+0000 7fec35600640 1 -- 172.21.15.7:0/1440881321 shutdown_connections 2024-04-25T11:11:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.308+0000 7fec35600640 1 --2- 172.21.15.7:0/1440881321 >> v2:172.21.15.87:3300/0 conn(0x7fec3006dac0 0x7fec30163b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.308+0000 7fec35600640 1 --2- 172.21.15.7:0/1440881321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec300744d0 0x7fec3006d580 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.308+0000 7fec35600640 1 --2- 172.21.15.7:0/1440881321 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec30075d30 0x7fec30149af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.308+0000 7fec35600640 1 -- 172.21.15.7:0/1440881321 >> 172.21.15.7:0/1440881321 conn(0x7fec300739b0 msgr2=0x7fec30073dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.308+0000 7fec35600640 1 -- 172.21.15.7:0/1440881321 shutdown_connections 2024-04-25T11:11:45.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.308+0000 7fec35600640 1 -- 172.21.15.7:0/1440881321 wait complete. 2024-04-25T11:11:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.311+0000 7fec35600640 1 Processor -- start 2024-04-25T11:11:45.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.311+0000 7fec35600640 1 -- start start 2024-04-25T11:11:45.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec35600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3006dac0 0x7fec3015f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:45.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec35600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec30075d30 0x7fec301ed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec35600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec301330d0 0x7fec301334e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec35600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec300d1ba0 con 0x7fec301330d0 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec35600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fec300d0a00 con 0x7fec3006dac0 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec35600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fec30133ba0 con 0x7fec30075d30 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec34c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec301330d0 0x7fec301334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec34c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec301330d0 0x7fec301334e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40376/0 (socket says 172.21.15.7:40376) 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec34c00640 1 -- 172.21.15.7:0/1094728840 learned_addr learned my addr 172.21.15.7:0/1094728840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec2f400640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec30075d30 0x7fec301ed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec2fe00640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3006dac0 0x7fec3015f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec34c00640 1 -- 172.21.15.7:0/1094728840 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec30075d30 msgr2=0x7fec301ed1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.313+0000 7fec34c00640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec30075d30 0x7fec301ed1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.314+0000 7fec34c00640 1 -- 172.21.15.7:0/1094728840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3006dac0 msgr2=0x7fec3015f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:45.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.314+0000 7fec34c00640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3006dac0 0x7fec3015f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.314+0000 7fec34c00640 1 -- 172.21.15.7:0/1094728840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3015fa50 con 0x7fec301330d0 2024-04-25T11:11:45.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.314+0000 7fec34c00640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec301330d0 0x7fec301334e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fec2800ef00 tx=0x7fec2800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:45.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.314+0000 7fec27e00640 1 -- 172.21.15.7:0/1094728840 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28018020 con 0x7fec301330d0 2024-04-25T11:11:45.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.314+0000 7fec27e00640 1 -- 172.21.15.7:0/1094728840 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2800bd20 con 0x7fec301330d0 2024-04-25T11:11:45.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.314+0000 7fec35600640 1 -- 172.21.15.7:0/1094728840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec301ebcf0 con 0x7fec301330d0 2024-04-25T11:11:45.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.314+0000 7fec27e00640 1 -- 172.21.15.7:0/1094728840 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28014640 con 0x7fec301330d0 2024-04-25T11:11:45.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.314+0000 7fec35600640 1 -- 172.21.15.7:0/1094728840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec301ec2b0 con 0x7fec301330d0 2024-04-25T11:11:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.315+0000 7fec35600640 1 -- 172.21.15.7:0/1094728840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec300b80f0 con 0x7fec301330d0 2024-04-25T11:11:45.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.317+0000 7fec27e00640 1 -- 172.21.15.7:0/1094728840 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fec28005d10 con 0x7fec301330d0 2024-04-25T11:11:45.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.318+0000 7fec27e00640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec100744b0 0x7fec100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:45.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.318+0000 7fec27e00640 1 -- 172.21.15.7:0/1094728840 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fec28095c00 con 0x7fec301330d0 2024-04-25T11:11:45.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.318+0000 7fec2fe00640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec100744b0 0x7fec100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:45.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.319+0000 7fec2fe00640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec100744b0 0x7fec100769c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fec20009a30 tx=0x7fec20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:45.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.322+0000 7fec27e00640 1 -- 172.21.15.7:0/1094728840 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec300b80f0 con 0x7fec301330d0 2024-04-25T11:11:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.538+0000 7fec35600640 1 -- 172.21.15.7:0/1094728840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fec30050740 con 0x7fec301330d0 2024-04-25T11:11:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.539+0000 7fec27e00640 1 -- 172.21.15.7:0/1094728840 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fec28022070 con 0x7fec301330d0 2024-04-25T11:11:45.540 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:11:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 -- 172.21.15.7:0/1094728840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec100744b0 msgr2=0x7fec100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec100744b0 0x7fec100769c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fec20009a30 tx=0x7fec20008040 comp rx=0 tx=0).stop 2024-04-25T11:11:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 -- 172.21.15.7:0/1094728840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec301330d0 msgr2=0x7fec301334e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec301330d0 0x7fec301334e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fec2800ef00 tx=0x7fec2800b480 comp rx=0 tx=0).stop 2024-04-25T11:11:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 -- 172.21.15.7:0/1094728840 shutdown_connections 2024-04-25T11:11:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec100744b0 0x7fec100769c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec301330d0 0x7fec301334e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec30075d30 0x7fec301ed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 --2- 172.21.15.7:0/1094728840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3006dac0 0x7fec3015f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 -- 172.21.15.7:0/1094728840 >> 172.21.15.7:0/1094728840 conn(0x7fec300739b0 msgr2=0x7fec301458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 -- 172.21.15.7:0/1094728840 shutdown_connections 2024-04-25T11:11:45.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:45.542+0000 7fec35600640 1 -- 172.21.15.7:0/1094728840 wait complete. 2024-04-25T11:11:45.573 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:11:45.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:45.573 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T11:11:45.655 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:11:45.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:45.655 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:11:45.702 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:11:45.702 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:45.762 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:11:45.762 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:45.814 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:11:45.814 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:45.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:46.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.160+0000 7fadb3800640 1 Processor -- start 2024-04-25T11:11:46.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.161+0000 7fadb3800640 1 -- start start 2024-04-25T11:11:46.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.163+0000 7fadb3800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fadac0af540 0x7fadac09f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.163+0000 7fadb3800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadac15f4e0 con 0x7fadac15ef90 2024-04-25T11:11:46.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.163+0000 7fadb3800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadac0b0560 con 0x7fadac15bc40 2024-04-25T11:11:46.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.163+0000 7fadb3800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fadac071190 con 0x7fadac0af540 2024-04-25T11:11:46.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.163+0000 7fadb0e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fadac15ef90 0x7fadac0aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59206/0 (socket says 172.21.15.7:59206) 2024-04-25T11:11:46.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.163+0000 7fadb0e00640 1 -- 172.21.15.7:0/3856912757 learned_addr learned my addr 172.21.15.7:0/3856912757 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.164+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1354238365 0 0) 0x7fadac15f4e0 con 0x7fadac15ef90 2024-04-25T11:11:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.164+0000 7fadabe00640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fadac0af540 0x7fadac09f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.164+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad94003520 con 0x7fadac15ef90 2024-04-25T11:11:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.164+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3160498141 0 0) 0x7fad94003520 con 0x7fadac15ef90 2024-04-25T11:11:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.164+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fadac0af540 msgr2=0x7fadac09f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.164+0000 7fadab400640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fadac0af540 0x7fadac09f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.164+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 >> v1:172.21.15.87:6789/0 conn(0x7fadac15bc40 legacy=0x7fadac0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.164+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac0b0840 con 0x7fadac15ef90 2024-04-25T11:11:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.164+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fada0001670 con 0x7fadac15ef90 2024-04-25T11:11:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.164+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 >> v1:172.21.15.7:6789/0 conn(0x7fadac15ef90 legacy=0x7fadac0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadab400640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940039e0 0x7fad94003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadab400640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad94004800 0x7fad94004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadab400640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad94005150 0x7fad9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fadac071190 con 0x7fad940039e0 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fad9400c000 con 0x7fad94005150 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad9400c200 con 0x7fad94004800 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadb0e00640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940039e0 0x7fad94003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadabe00640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad94004800 0x7fad94004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadb1800640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad94005150 0x7fad9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadb0e00640 1 -- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad94004800 msgr2=0x7fad94004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadb0e00640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad94004800 0x7fad94004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadb0e00640 1 -- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad94005150 msgr2=0x7fad9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadb0e00640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad94005150 0x7fad9400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadb0e00640 1 -- 172.21.15.7:0/3856912757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad94003520 con 0x7fad940039e0 2024-04-25T11:11:46.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.165+0000 7fadb1800640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad94005150 0x7fad9400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:46.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.166+0000 7fadb0e00640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940039e0 0x7fad94003e50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fada000c0e0 tx=0x7fad94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519282e58d56a11c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:46.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.167+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0007c20 con 0x7fad940039e0 2024-04-25T11:11:46.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.167+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada0005af0 con 0x7fad940039e0 2024-04-25T11:11:46.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.167+0000 7fadab400640 1 -- 172.21.15.7:0/3856912757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0007070 con 0x7fad940039e0 2024-04-25T11:11:46.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.167+0000 7fadb3800640 1 -- 172.21.15.7:0/3856912757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940039e0 msgr2=0x7fad94003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.167+0000 7fadb3800640 1 --2- 172.21.15.7:0/3856912757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940039e0 0x7fad94003e50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fada000c0e0 tx=0x7fad94005830 comp rx=0 tx=0).stop 2024-04-25T11:11:46.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.167+0000 7fadb1800640 1 -- 172.21.15.7:0/3856912757 reap_dead start 2024-04-25T11:11:46.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.168+0000 7fadb3800640 1 -- 172.21.15.7:0/3856912757 shutdown_connections 2024-04-25T11:11:46.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.168+0000 7fadb3800640 1 -- 172.21.15.7:0/3856912757 >> 172.21.15.7:0/3856912757 conn(0x7fadac074200 msgr2=0x7fadac06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:46.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.168+0000 7fadb3800640 1 -- 172.21.15.7:0/3856912757 shutdown_connections 2024-04-25T11:11:46.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.168+0000 7fadb3800640 1 -- 172.21.15.7:0/3856912757 wait complete. 2024-04-25T11:11:46.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.170+0000 7fadb3800640 1 Processor -- start 2024-04-25T11:11:46.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.170+0000 7fadb3800640 1 -- start start 2024-04-25T11:11:46.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.172+0000 7fadb3800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad940039e0 0x7fadac156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.172+0000 7fadb3800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadac1574b0 0x7fadac151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.172+0000 7fadb3800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac158d90 0x7fadac152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.172+0000 7fadb3800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0b1320 con 0x7fadac1574b0 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.172+0000 7fadb3800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fadac0b0e00 con 0x7fadac158d90 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.172+0000 7fadb3800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fadac0afb80 con 0x7fad940039e0 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadabe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadac1574b0 0x7fadac151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadabe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadac1574b0 0x7fadac151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40388/0 (socket says 172.21.15.7:40388) 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadabe00640 1 -- 172.21.15.7:0/2977005727 learned_addr learned my addr 172.21.15.7:0/2977005727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadb0e00640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad940039e0 0x7fadac156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadb1800640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac158d90 0x7fadac152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadabe00640 1 -- 172.21.15.7:0/2977005727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad940039e0 msgr2=0x7fadac156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadabe00640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad940039e0 0x7fadac156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadabe00640 1 -- 172.21.15.7:0/2977005727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac158d90 msgr2=0x7fadac152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadabe00640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac158d90 0x7fadac152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadabe00640 1 -- 172.21.15.7:0/2977005727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadac15a070 con 0x7fadac1574b0 2024-04-25T11:11:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadb0e00640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad940039e0 0x7fadac156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:11:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.173+0000 7fadabe00640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadac1574b0 0x7fadac151f10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fad9c011600 tx=0x7fad9c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.174+0000 7fada8c00640 1 -- 172.21.15.7:0/2977005727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c00f920 con 0x7fadac1574b0 2024-04-25T11:11:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.174+0000 7fada8c00640 1 -- 172.21.15.7:0/2977005727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9c00fac0 con 0x7fadac1574b0 2024-04-25T11:11:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.174+0000 7fada8c00640 1 -- 172.21.15.7:0/2977005727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c00fe70 con 0x7fadac1574b0 2024-04-25T11:11:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.174+0000 7fadb3800640 1 -- 172.21.15.7:0/2977005727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadac15a350 con 0x7fadac1574b0 2024-04-25T11:11:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.174+0000 7fadb3800640 1 -- 172.21.15.7:0/2977005727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadac15a910 con 0x7fadac1574b0 2024-04-25T11:11:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.175+0000 7fadb3800640 1 -- 172.21.15.7:0/2977005727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadac06a100 con 0x7fadac1574b0 2024-04-25T11:11:46.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.176+0000 7fada8c00640 1 -- 172.21.15.7:0/2977005727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad9c015460 con 0x7fadac1574b0 2024-04-25T11:11:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.178+0000 7fada8c00640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad900744e0 0x7fad900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.178+0000 7fada8c00640 1 -- 172.21.15.7:0/2977005727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fad9c095b80 con 0x7fadac1574b0 2024-04-25T11:11:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.178+0000 7fadb0e00640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad900744e0 0x7fad900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.179+0000 7fadb0e00640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad900744e0 0x7fad900769f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fada0032000 tx=0x7fada0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.183+0000 7fada8c00640 1 -- 172.21.15.7:0/2977005727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad9c019030 con 0x7fadac1574b0 2024-04-25T11:11:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.388+0000 7fadb3800640 1 -- 172.21.15.7:0/2977005727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadac15e190 con 0x7fadac1574b0 2024-04-25T11:11:46.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.391+0000 7fada8c00640 1 -- 172.21.15.7:0/2977005727 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7fad9c05ff00 con 0x7fadac1574b0 2024-04-25T11:11:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:46.393 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.394+0000 7fadb3800640 1 -- 172.21.15.7:0/2977005727 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad900744e0 msgr2=0x7fad900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.394+0000 7fadb3800640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad900744e0 0x7fad900769f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fada0032000 tx=0x7fada0009030 comp rx=0 tx=0).stop 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.394+0000 7fadb3800640 1 -- 172.21.15.7:0/2977005727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadac1574b0 msgr2=0x7fadac151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.394+0000 7fadb3800640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadac1574b0 0x7fadac151f10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fad9c011600 tx=0x7fad9c011ac0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.394+0000 7fadb3800640 1 -- 172.21.15.7:0/2977005727 shutdown_connections 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.394+0000 7fadb3800640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad900744e0 0x7fad900769f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.394+0000 7fadb3800640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac158d90 0x7fadac152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.394+0000 7fadb3800640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadac1574b0 0x7fadac151f10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.394+0000 7fadb3800640 1 --2- 172.21.15.7:0/2977005727 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad940039e0 0x7fadac156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.394+0000 7fadb3800640 1 -- 172.21.15.7:0/2977005727 >> 172.21.15.7:0/2977005727 conn(0x7fadac074200 msgr2=0x7fadac0b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.394+0000 7fadb3800640 1 -- 172.21.15.7:0/2977005727 shutdown_connections 2024-04-25T11:11:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.395+0000 7fadb3800640 1 -- 172.21.15.7:0/2977005727 wait complete. 2024-04-25T11:11:46.420 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:46.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:46.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.700+0000 7fe456600640 1 Processor -- start 2024-04-25T11:11:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.700+0000 7fe456600640 1 -- start start 2024-04-25T11:11:46.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.702+0000 7fe456600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4500b6240 0x7fe45015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.702+0000 7fe456600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe4500aef90 0x7fe4500b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.702+0000 7fe456600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe450164e00 con 0x7fe4500b6920 2024-04-25T11:11:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.702+0000 7fe456600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe4501650e0 con 0x7fe4500aef90 2024-04-25T11:11:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.702+0000 7fe456600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe4501653c0 con 0x7fe4500b6240 2024-04-25T11:11:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.703+0000 7fe44ea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe4500b6920 0x7fe4500ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59210/0 (socket says 172.21.15.7:59210) 2024-04-25T11:11:46.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.703+0000 7fe44ea00640 1 -- 172.21.15.7:0/3936351802 learned_addr learned my addr 172.21.15.7:0/3936351802 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:46.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.703+0000 7fe44fe00640 1 --2- 172.21.15.7:0/3936351802 >> v2:172.21.15.87:3300/0 conn(0x7fe4500aef90 0x7fe4500b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.703+0000 7fe44f400640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4500b6240 0x7fe45015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.703+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3317954579 0 0) 0x7fe450164e00 con 0x7fe4500b6920 2024-04-25T11:11:46.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.703+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe434003520 con 0x7fe4500b6920 2024-04-25T11:11:46.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4098126346 0 0) 0x7fe434003520 con 0x7fe4500b6920 2024-04-25T11:11:46.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4500b6240 msgr2=0x7fe45015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4500b6240 0x7fe45015eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 >> v2:172.21.15.87:3300/0 conn(0x7fe4500aef90 msgr2=0x7fe4500b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 --2- 172.21.15.7:0/3936351802 >> v2:172.21.15.87:3300/0 conn(0x7fe4500aef90 0x7fe4500b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4501656a0 con 0x7fe4500b6920 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fe43c001670 con 0x7fe4500b6920 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 >> v1:172.21.15.7:6789/0 conn(0x7fe4500b6920 legacy=0x7fe4500ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4340039b0 0x7fe434003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4500b6240 0x7fe434004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe434004d40 0x7fe434005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe4501650e0 con 0x7fe4340039b0 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe4501653c0 con 0x7fe434004d40 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe43400b9f0 con 0x7fe4500b6240 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.704+0000 7fe44f400640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4340039b0 0x7fe434003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.705+0000 7fe44ea00640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4500b6240 0x7fe434004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.705+0000 7fe44fe00640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe434004d40 0x7fe434005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.705+0000 7fe44f400640 1 -- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4500b6240 msgr2=0x7fe434004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:46.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.705+0000 7fe44f400640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4500b6240 0x7fe434004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.705+0000 7fe44f400640 1 -- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe434004d40 msgr2=0x7fe434005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:46.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.705+0000 7fe44f400640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe434004d40 0x7fe434005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.705+0000 7fe44f400640 1 -- 172.21.15.7:0/3936351802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe434003520 con 0x7fe4340039b0 2024-04-25T11:11:46.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.705+0000 7fe44ea00640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4500b6240 0x7fe434004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:11:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.706+0000 7fe44f400640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4340039b0 0x7fe434003e20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe4500714e0 tx=0x7fe434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4d8d8b11d9bdf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.706+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440036070 con 0x7fe4340039b0 2024-04-25T11:11:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.707+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe440031400 con 0x7fe4340039b0 2024-04-25T11:11:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.707+0000 7fe44e000640 1 -- 172.21.15.7:0/3936351802 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440009690 con 0x7fe4340039b0 2024-04-25T11:11:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.707+0000 7fe456600640 1 -- 172.21.15.7:0/3936351802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4340039b0 msgr2=0x7fe434003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.707+0000 7fe456600640 1 --2- 172.21.15.7:0/3936351802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4340039b0 0x7fe434003e20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe4500714e0 tx=0x7fe434005830 comp rx=0 tx=0).stop 2024-04-25T11:11:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.707+0000 7fe44fe00640 1 -- 172.21.15.7:0/3936351802 reap_dead start 2024-04-25T11:11:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.707+0000 7fe456600640 1 -- 172.21.15.7:0/3936351802 shutdown_connections 2024-04-25T11:11:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.707+0000 7fe456600640 1 -- 172.21.15.7:0/3936351802 >> 172.21.15.7:0/3936351802 conn(0x7fe4500742c0 msgr2=0x7fe45006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.707+0000 7fe456600640 1 -- 172.21.15.7:0/3936351802 shutdown_connections 2024-04-25T11:11:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.707+0000 7fe456600640 1 -- 172.21.15.7:0/3936351802 wait complete. 2024-04-25T11:11:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.710+0000 7fe456600640 1 Processor -- start 2024-04-25T11:11:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.710+0000 7fe456600640 1 -- start start 2024-04-25T11:11:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe456600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4340039b0 0x7fe4501fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe456600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4501fa270 0x7fe45006c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe456600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45006cab0 0x7fe45009d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe456600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe450164ba0 con 0x7fe45006cab0 2024-04-25T11:11:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe456600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe450164a20 con 0x7fe4501fa270 2024-04-25T11:11:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe456600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe45015eb00 con 0x7fe4340039b0 2024-04-25T11:11:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe44fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45006cab0 0x7fe45009d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe44fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45006cab0 0x7fe45009d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40404/0 (socket says 172.21.15.7:40404) 2024-04-25T11:11:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe44fe00640 1 -- 172.21.15.7:0/3293218758 learned_addr learned my addr 172.21.15.7:0/3293218758 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe44f400640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4340039b0 0x7fe4501fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe44ea00640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4501fa270 0x7fe45006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe44fe00640 1 -- 172.21.15.7:0/3293218758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4340039b0 msgr2=0x7fe4501fdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe44fe00640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4340039b0 0x7fe4501fdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe44fe00640 1 -- 172.21.15.7:0/3293218758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4501fa270 msgr2=0x7fe45006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.712+0000 7fe44fe00640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4501fa270 0x7fe45006c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.713+0000 7fe44fe00640 1 -- 172.21.15.7:0/3293218758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45006cec0 con 0x7fe45006cab0 2024-04-25T11:11:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.713+0000 7fe44ea00640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4501fa270 0x7fe45006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.713+0000 7fe44fe00640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45006cab0 0x7fe45009d710 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe4500b0420 tx=0x7fe44800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.713+0000 7fe447400640 1 -- 172.21.15.7:0/3293218758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44800fde0 con 0x7fe45006cab0 2024-04-25T11:11:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.713+0000 7fe447400640 1 -- 172.21.15.7:0/3293218758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe448015430 con 0x7fe45006cab0 2024-04-25T11:11:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.713+0000 7fe447400640 1 -- 172.21.15.7:0/3293218758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448015820 con 0x7fe45006cab0 2024-04-25T11:11:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.713+0000 7fe456600640 1 -- 172.21.15.7:0/3293218758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4501fe9f0 con 0x7fe45006cab0 2024-04-25T11:11:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.713+0000 7fe456600640 1 -- 172.21.15.7:0/3293218758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4501ecdc0 con 0x7fe45006cab0 2024-04-25T11:11:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.715+0000 7fe456600640 1 -- 172.21.15.7:0/3293218758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4501ecbb0 con 0x7fe45006cab0 2024-04-25T11:11:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.716+0000 7fe447400640 1 -- 172.21.15.7:0/3293218758 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe4480159c0 con 0x7fe45006cab0 2024-04-25T11:11:46.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.717+0000 7fe447400640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4300744b0 0x7fe4300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:46.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.717+0000 7fe447400640 1 -- 172.21.15.7:0/3293218758 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fe4480963f0 con 0x7fe45006cab0 2024-04-25T11:11:46.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.718+0000 7fe44f400640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4300744b0 0x7fe4300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:46.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.718+0000 7fe44f400640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4300744b0 0x7fe4300769c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe440033000 tx=0x7fe440008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:46.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.722+0000 7fe447400640 1 -- 172.21.15.7:0/3293218758 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe45006cec0 con 0x7fe45006cab0 2024-04-25T11:11:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.925+0000 7fe456600640 1 -- 172.21.15.7:0/3293218758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe4500710c0 con 0x7fe45006cab0 2024-04-25T11:11:46.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.926+0000 7fe447400640 1 -- 172.21.15.7:0/3293218758 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7fe448019030 con 0x7fe45006cab0 2024-04-25T11:11:46.928 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.929+0000 7fe456600640 1 -- 172.21.15.7:0/3293218758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4300744b0 msgr2=0x7fe4300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.929+0000 7fe456600640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4300744b0 0x7fe4300769c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe440033000 tx=0x7fe440008030 comp rx=0 tx=0).stop 2024-04-25T11:11:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.929+0000 7fe456600640 1 -- 172.21.15.7:0/3293218758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45006cab0 msgr2=0x7fe45009d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:46.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.929+0000 7fe456600640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45006cab0 0x7fe45009d710 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe4500b0420 tx=0x7fe44800f580 comp rx=0 tx=0).stop 2024-04-25T11:11:46.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.931+0000 7fe456600640 1 -- 172.21.15.7:0/3293218758 shutdown_connections 2024-04-25T11:11:46.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.931+0000 7fe456600640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4300744b0 0x7fe4300769c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.931+0000 7fe456600640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe45006cab0 0x7fe45009d710 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.931+0000 7fe456600640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4501fa270 0x7fe45006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.931+0000 7fe456600640 1 --2- 172.21.15.7:0/3293218758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4340039b0 0x7fe4501fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:46.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.931+0000 7fe456600640 1 -- 172.21.15.7:0/3293218758 >> 172.21.15.7:0/3293218758 conn(0x7fe4500742c0 msgr2=0x7fe4500b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:46.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.931+0000 7fe456600640 1 -- 172.21.15.7:0/3293218758 shutdown_connections 2024-04-25T11:11:46.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:46.931+0000 7fe456600640 1 -- 172.21.15.7:0/3293218758 wait complete. 2024-04-25T11:11:46.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:47.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.246+0000 7f2286600640 1 Processor -- start 2024-04-25T11:11:47.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.246+0000 7f2286600640 1 -- start start 2024-04-25T11:11:47.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.249+0000 7f2286600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22800d0ee0 0x7f22800affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.249+0000 7f2286600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22800b5020 0x7f2280160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.249+0000 7f2286600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f228006f5b0 con 0x7f228015aaa0 2024-04-25T11:11:47.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.249+0000 7f2286600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f22800af290 con 0x7f22800d0ee0 2024-04-25T11:11:47.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.249+0000 7f2286600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f22800d14a0 con 0x7f22800b5020 2024-04-25T11:11:47.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.250+0000 7f227fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22800b5020 0x7f2280160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.250+0000 7f227ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22800d0ee0 0x7f22800affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.250+0000 7f227f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f228015aaa0 0x7f22800b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39848/0 (socket says 172.21.15.7:39848) 2024-04-25T11:11:47.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.250+0000 7f227f400640 1 -- 172.21.15.7:0/1639523082 learned_addr learned my addr 172.21.15.7:0/1639523082 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:47.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.251+0000 7f227fe00640 1 -- 172.21.15.7:0/1639523082 >> v2:172.21.15.87:3300/0 conn(0x7f22800d0ee0 msgr2=0x7f22800affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:47.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.251+0000 7f227fe00640 1 --2- 172.21.15.7:0/1639523082 >> v2:172.21.15.87:3300/0 conn(0x7f22800d0ee0 0x7f22800affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.251+0000 7f227fe00640 1 -- 172.21.15.7:0/1639523082 >> v1:172.21.15.87:6789/0 conn(0x7f228015aaa0 legacy=0x7f22800b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:47.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.251+0000 7f227fe00640 1 -- 172.21.15.7:0/1639523082 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22800af4e0 con 0x7f22800b5020 2024-04-25T11:11:47.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.252+0000 7f227fe00640 1 --2- 172.21.15.7:0/1639523082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22800b5020 0x7f2280160990 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2278009d90 tx=0x7f227802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1718c7585b61161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:47.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.252+0000 7f227e000640 1 -- 172.21.15.7:0/1639523082 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278007740 con 0x7f22800b5020 2024-04-25T11:11:47.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.252+0000 7f227e000640 1 -- 172.21.15.7:0/1639523082 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22780078e0 con 0x7f22800b5020 2024-04-25T11:11:47.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.252+0000 7f227e000640 1 -- 172.21.15.7:0/1639523082 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278007cb0 con 0x7f22800b5020 2024-04-25T11:11:47.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.253+0000 7f2286600640 1 -- 172.21.15.7:0/1639523082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22800b5020 msgr2=0x7f2280160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:47.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.253+0000 7f2286600640 1 --2- 172.21.15.7:0/1639523082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22800b5020 0x7f2280160990 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2278009d90 tx=0x7f227802a660 comp rx=0 tx=0).stop 2024-04-25T11:11:47.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.253+0000 7f2286600640 1 -- 172.21.15.7:0/1639523082 shutdown_connections 2024-04-25T11:11:47.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.253+0000 7f2286600640 1 --2- 172.21.15.7:0/1639523082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22800b5020 0x7f2280160990 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.253+0000 7f2286600640 1 --2- 172.21.15.7:0/1639523082 >> v2:172.21.15.87:3300/0 conn(0x7f22800d0ee0 0x7f22800affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.253+0000 7f2286600640 1 -- 172.21.15.7:0/1639523082 >> 172.21.15.7:0/1639523082 conn(0x7f22800742c0 msgr2=0x7f228006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:47.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.254+0000 7f2286600640 1 -- 172.21.15.7:0/1639523082 shutdown_connections 2024-04-25T11:11:47.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.254+0000 7f2286600640 1 -- 172.21.15.7:0/1639523082 wait complete. 2024-04-25T11:11:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.257+0000 7f2286600640 1 Processor -- start 2024-04-25T11:11:47.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.257+0000 7f2286600640 1 -- start start 2024-04-25T11:11:47.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.259+0000 7f2286600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22800b5020 0x7f22801f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.259+0000 7f2286600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22800d0ee0 0x7f2280146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.259+0000 7f2286600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f228015aaa0 0x7f228006fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.259+0000 7f2286600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22800b5430 con 0x7f22800b5020 2024-04-25T11:11:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.259+0000 7f2286600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f22800afaa0 con 0x7f22800d0ee0 2024-04-25T11:11:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.259+0000 7f2286600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f22800b55b0 con 0x7f228015aaa0 2024-04-25T11:11:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.259+0000 7f227f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22800b5020 0x7f22801f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.259+0000 7f227f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22800b5020 0x7f22801f8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40408/0 (socket says 172.21.15.7:40408) 2024-04-25T11:11:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.259+0000 7f227f400640 1 -- 172.21.15.7:0/139050992 learned_addr learned my addr 172.21.15.7:0/139050992 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.260+0000 7f227ea00640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22800d0ee0 0x7f2280146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.260+0000 7f227fe00640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f228015aaa0 0x7f228006fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.260+0000 7f227f400640 1 -- 172.21.15.7:0/139050992 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f228015aaa0 msgr2=0x7f228006fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:47.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.260+0000 7f227f400640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f228015aaa0 0x7f228006fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.260+0000 7f227f400640 1 -- 172.21.15.7:0/139050992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22800d0ee0 msgr2=0x7f2280146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:47.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.260+0000 7f227f400640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22800d0ee0 0x7f2280146160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.260+0000 7f227f400640 1 -- 172.21.15.7:0/139050992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2280148c90 con 0x7f22800b5020 2024-04-25T11:11:47.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.260+0000 7f227fe00640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f228015aaa0 0x7f228006fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:11:47.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.260+0000 7f227f400640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22800b5020 0x7f22801f8270 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f227000c480 tx=0x7f227000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:47.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.261+0000 7f2277400640 1 -- 172.21.15.7:0/139050992 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270017070 con 0x7f22800b5020 2024-04-25T11:11:47.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.261+0000 7f2277400640 1 -- 172.21.15.7:0/139050992 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2270005ce0 con 0x7f22800b5020 2024-04-25T11:11:47.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.261+0000 7f2277400640 1 -- 172.21.15.7:0/139050992 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22700072f0 con 0x7f22800b5020 2024-04-25T11:11:47.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.261+0000 7f2286600640 1 -- 172.21.15.7:0/139050992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22801e3750 con 0x7f22800b5020 2024-04-25T11:11:47.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.261+0000 7f2286600640 1 -- 172.21.15.7:0/139050992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22801e3d10 con 0x7f22800b5020 2024-04-25T11:11:47.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.262+0000 7f2286600640 1 -- 172.21.15.7:0/139050992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2280050740 con 0x7f22800b5020 2024-04-25T11:11:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.264+0000 7f2277400640 1 -- 172.21.15.7:0/139050992 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f227001a070 con 0x7f22800b5020 2024-04-25T11:11:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.266+0000 7f2277400640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22600744b0 0x7f22600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.266+0000 7f2277400640 1 -- 172.21.15.7:0/139050992 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f2270094d90 con 0x7f22800b5020 2024-04-25T11:11:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.266+0000 7f227ea00640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22600744b0 0x7f22600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.267+0000 7f227ea00640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22600744b0 0x7f22600769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2280070870 tx=0x7f226c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.269+0000 7f2277400640 1 -- 172.21.15.7:0/139050992 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f227005eb50 con 0x7f22800b5020 2024-04-25T11:11:47.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.476+0000 7f2286600640 1 -- 172.21.15.7:0/139050992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2280070f20 con 0x7f22800b5020 2024-04-25T11:11:47.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.477+0000 7f2277400640 1 -- 172.21.15.7:0/139050992 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f2270062e50 con 0x7f22800b5020 2024-04-25T11:11:47.478 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.479+0000 7f2286600640 1 -- 172.21.15.7:0/139050992 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22600744b0 msgr2=0x7f22600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.479+0000 7f2286600640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22600744b0 0x7f22600769c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2280070870 tx=0x7f226c009190 comp rx=0 tx=0).stop 2024-04-25T11:11:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.479+0000 7f2286600640 1 -- 172.21.15.7:0/139050992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22800b5020 msgr2=0x7f22801f8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.479+0000 7f2286600640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22800b5020 0x7f22801f8270 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f227000c480 tx=0x7f227000c940 comp rx=0 tx=0).stop 2024-04-25T11:11:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.480+0000 7f2286600640 1 -- 172.21.15.7:0/139050992 shutdown_connections 2024-04-25T11:11:47.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.480+0000 7f2286600640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22600744b0 0x7f22600769c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.480+0000 7f2286600640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f228015aaa0 0x7f228006fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.480+0000 7f2286600640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22800d0ee0 0x7f2280146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.480+0000 7f2286600640 1 --2- 172.21.15.7:0/139050992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22800b5020 0x7f22801f8270 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.480+0000 7f2286600640 1 -- 172.21.15.7:0/139050992 >> 172.21.15.7:0/139050992 conn(0x7f22800742c0 msgr2=0x7f228015afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:47.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.480+0000 7f2286600640 1 -- 172.21.15.7:0/139050992 shutdown_connections 2024-04-25T11:11:47.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.480+0000 7f2286600640 1 -- 172.21.15.7:0/139050992 wait complete. 2024-04-25T11:11:47.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.789+0000 7fd9f5800640 1 Processor -- start 2024-04-25T11:11:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.789+0000 7fd9f5800640 1 -- start start 2024-04-25T11:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.792+0000 7fd9f5800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9f00d09d0 0x7fd9f006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.792+0000 7fd9f5800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f00b0be0 con 0x7fd9f00cfa80 2024-04-25T11:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.792+0000 7fd9f5800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f00cfe90 con 0x7fd9f0070630 2024-04-25T11:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.792+0000 7fd9f5800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f00ce9e0 con 0x7fd9f00d09d0 2024-04-25T11:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.792+0000 7fd9e7e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd9f00cfa80 0x7fd9f00d02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59220/0 (socket says 172.21.15.7:59220) 2024-04-25T11:11:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.792+0000 7fd9e7e00640 1 -- 172.21.15.7:0/3747893005 learned_addr learned my addr 172.21.15.7:0/3747893005 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.792+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 280118308 0 0) 0x7fd9f00b0be0 con 0x7fd9f00cfa80 2024-04-25T11:11:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9d4003520 con 0x7fd9f00cfa80 2024-04-25T11:11:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9f4e00640 1 --2- 172.21.15.7:0/3747893005 >> v2:172.21.15.87:3300/0 conn(0x7fd9f00d09d0 0x7fd9f006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1898659492 0 0) 0x7fd9d4003520 con 0x7fd9f00cfa80 2024-04-25T11:11:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 >> v2:172.21.15.87:3300/0 conn(0x7fd9f00d09d0 msgr2=0x7fd9f006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 --2- 172.21.15.7:0/3747893005 >> v2:172.21.15.87:3300/0 conn(0x7fd9f00d09d0 0x7fd9f006eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 >> v1:172.21.15.87:6789/0 conn(0x7fd9f0070630 legacy=0x7fd9f015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f00cecc0 con 0x7fd9f00cfa80 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fd9d8001670 con 0x7fd9f00cfa80 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 >> v1:172.21.15.7:6789/0 conn(0x7fd9f00cfa80 legacy=0x7fd9f00d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 0x7fd9d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9d4004800 0x7fd9d4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9d4005300 0x7fd9d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f00ce9e0 con 0x7fd9d40039b0 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d400c2a0 con 0x7fd9d4005300 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.793+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d400c4a0 con 0x7fd9d4004800 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.794+0000 7fd9efe00640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 0x7fd9d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.794+0000 7fd9e7e00640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9d4004800 0x7fd9d4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.794+0000 7fd9efe00640 1 -- 172.21.15.7:0/3747893005 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9d4004800 msgr2=0x7fd9d4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.794+0000 7fd9efe00640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9d4004800 0x7fd9d4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.794+0000 7fd9efe00640 1 -- 172.21.15.7:0/3747893005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9d4005300 msgr2=0x7fd9d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.794+0000 7fd9f4e00640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9d4005300 0x7fd9d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.794+0000 7fd9efe00640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9d4005300 0x7fd9d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.794+0000 7fd9efe00640 1 -- 172.21.15.7:0/3747893005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d4003520 con 0x7fd9d40039b0 2024-04-25T11:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.794+0000 7fd9f4e00640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9d4005300 0x7fd9d400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.794+0000 7fd9f4e00640 1 -- 172.21.15.7:0/3747893005 reap_dead start 2024-04-25T11:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.795+0000 7fd9efe00640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 0x7fd9d4003e20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0001750 tx=0x7fd9d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9deb9bbbb6cea5e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.796+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0001370 con 0x7fd9d40039b0 2024-04-25T11:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.796+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e00052d0 con 0x7fd9d40039b0 2024-04-25T11:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.796+0000 7fd9ef400640 1 -- 172.21.15.7:0/3747893005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e002abc0 con 0x7fd9d40039b0 2024-04-25T11:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.796+0000 7fd9f5800640 1 -- 172.21.15.7:0/3747893005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 msgr2=0x7fd9d4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.796+0000 7fd9f5800640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 0x7fd9d4003e20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0001750 tx=0x7fd9d4005830 comp rx=0 tx=0).stop 2024-04-25T11:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.796+0000 7fd9f5800640 1 -- 172.21.15.7:0/3747893005 shutdown_connections 2024-04-25T11:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.797+0000 7fd9f5800640 1 --2- 172.21.15.7:0/3747893005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 0x7fd9d4003e20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.797+0000 7fd9f5800640 1 -- 172.21.15.7:0/3747893005 >> 172.21.15.7:0/3747893005 conn(0x7fd9f0074140 msgr2=0x7fd9f006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.797+0000 7fd9f5800640 1 -- 172.21.15.7:0/3747893005 shutdown_connections 2024-04-25T11:11:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.797+0000 7fd9f5800640 1 -- 172.21.15.7:0/3747893005 wait complete. 2024-04-25T11:11:47.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.799+0000 7fd9f5800640 1 Processor -- start 2024-04-25T11:11:47.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.799+0000 7fd9f5800640 1 -- start start 2024-04-25T11:11:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.801+0000 7fd9f5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 0x7fd9f0150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.801+0000 7fd9f5800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9f0142820 0x7fd9f0160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.801+0000 7fd9f5800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9f0142c30 0x7fd9f0161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.801+0000 7fd9f5800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f00cf560 con 0x7fd9d40039b0 2024-04-25T11:11:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.801+0000 7fd9f5800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f00cf280 con 0x7fd9f0142820 2024-04-25T11:11:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.801+0000 7fd9f5800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f00ce590 con 0x7fd9f0142c30 2024-04-25T11:11:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.801+0000 7fd9efe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 0x7fd9f0150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.801+0000 7fd9efe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 0x7fd9f0150dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40428/0 (socket says 172.21.15.7:40428) 2024-04-25T11:11:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.801+0000 7fd9efe00640 1 -- 172.21.15.7:0/1230755082 learned_addr learned my addr 172.21.15.7:0/1230755082 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.802+0000 7fd9e7e00640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9f0142820 0x7fd9f0160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.802+0000 7fd9f4e00640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9f0142c30 0x7fd9f0161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.802+0000 7fd9efe00640 1 -- 172.21.15.7:0/1230755082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9f0142c30 msgr2=0x7fd9f0161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.802+0000 7fd9efe00640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9f0142c30 0x7fd9f0161370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.802+0000 7fd9efe00640 1 -- 172.21.15.7:0/1230755082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9f0142820 msgr2=0x7fd9f0160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.802+0000 7fd9efe00640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9f0142820 0x7fd9f0160e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.802+0000 7fd9efe00640 1 -- 172.21.15.7:0/1230755082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f0153a10 con 0x7fd9d40039b0 2024-04-25T11:11:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.802+0000 7fd9efe00640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 0x7fd9f0150dd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0001870 tx=0x7fd9e0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.803+0000 7fd9ecc00640 1 -- 172.21.15.7:0/1230755082 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0037040 con 0x7fd9d40039b0 2024-04-25T11:11:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.803+0000 7fd9ecc00640 1 -- 172.21.15.7:0/1230755082 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e0040070 con 0x7fd9d40039b0 2024-04-25T11:11:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.803+0000 7fd9ecc00640 1 -- 172.21.15.7:0/1230755082 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0005480 con 0x7fd9d40039b0 2024-04-25T11:11:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.803+0000 7fd9f5800640 1 -- 172.21.15.7:0/1230755082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f0151310 con 0x7fd9d40039b0 2024-04-25T11:11:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.803+0000 7fd9f5800640 1 -- 172.21.15.7:0/1230755082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f014f340 con 0x7fd9d40039b0 2024-04-25T11:11:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.804+0000 7fd9f5800640 1 -- 172.21.15.7:0/1230755082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f0050740 con 0x7fd9d40039b0 2024-04-25T11:11:47.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.805+0000 7fd9ecc00640 1 -- 172.21.15.7:0/1230755082 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd9e00067d0 con 0x7fd9d40039b0 2024-04-25T11:11:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.806+0000 7fd9ecc00640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9cc0744b0 0x7fd9cc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.806+0000 7fd9ecc00640 1 -- 172.21.15.7:0/1230755082 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fd9e00b5450 con 0x7fd9d40039b0 2024-04-25T11:11:47.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.806+0000 7fd9e7e00640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9cc0744b0 0x7fd9cc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:47.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.807+0000 7fd9e7e00640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9cc0744b0 0x7fd9cc0769c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8001cc0 tx=0x7fd9d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:47.811+0000 7fd9ecc00640 1 -- 172.21.15.7:0/1230755082 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9e007f7e0 con 0x7fd9d40039b0 2024-04-25T11:11:48.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.013+0000 7fd9f5800640 1 -- 172.21.15.7:0/1230755082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd9f0070dd0 con 0x7fd9d40039b0 2024-04-25T11:11:48.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.014+0000 7fd9ecc00640 1 -- 172.21.15.7:0/1230755082 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fd9e007f0e0 con 0x7fd9d40039b0 2024-04-25T11:11:48.015 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:48.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.016+0000 7fd9f5800640 1 -- 172.21.15.7:0/1230755082 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9cc0744b0 msgr2=0x7fd9cc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.016+0000 7fd9f5800640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9cc0744b0 0x7fd9cc0769c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8001cc0 tx=0x7fd9d8009040 comp rx=0 tx=0).stop 2024-04-25T11:11:48.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.016+0000 7fd9f5800640 1 -- 172.21.15.7:0/1230755082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 msgr2=0x7fd9f0150dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.017+0000 7fd9f5800640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 0x7fd9f0150dd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0001870 tx=0x7fd9e0001120 comp rx=0 tx=0).stop 2024-04-25T11:11:48.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.017+0000 7fd9f5800640 1 -- 172.21.15.7:0/1230755082 shutdown_connections 2024-04-25T11:11:48.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.017+0000 7fd9f5800640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9cc0744b0 0x7fd9cc0769c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.017+0000 7fd9f5800640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9f0142c30 0x7fd9f0161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.017+0000 7fd9f5800640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9f0142820 0x7fd9f0160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.017+0000 7fd9f5800640 1 --2- 172.21.15.7:0/1230755082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9d40039b0 0x7fd9f0150dd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.017+0000 7fd9f5800640 1 -- 172.21.15.7:0/1230755082 >> 172.21.15.7:0/1230755082 conn(0x7fd9f0074140 msgr2=0x7fd9f0161d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:48.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.017+0000 7fd9f5800640 1 -- 172.21.15.7:0/1230755082 shutdown_connections 2024-04-25T11:11:48.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.017+0000 7fd9f5800640 1 -- 172.21.15.7:0/1230755082 wait complete. 2024-04-25T11:11:48.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.330+0000 7fe674400640 1 Processor -- start 2024-04-25T11:11:48.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.330+0000 7fe674400640 1 -- start start 2024-04-25T11:11:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.333+0000 7fe674400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe66c0b21c0 0x7fe66c15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.333+0000 7fe674400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe66c161230 0x7fe66c06ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.333+0000 7fe674400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c0b29a0 con 0x7fe66c0d1f80 2024-04-25T11:11:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.333+0000 7fe674400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe66c06c570 con 0x7fe66c161230 2024-04-25T11:11:48.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.333+0000 7fe674400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c0d2540 con 0x7fe66c0b21c0 2024-04-25T11:11:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.333+0000 7fe672400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe66c161230 0x7fe66c06ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.333+0000 7fe671a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe66c0d1f80 0x7fe66c0b1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59230/0 (socket says 172.21.15.7:59230) 2024-04-25T11:11:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.333+0000 7fe671a00640 1 -- 172.21.15.7:0/1911494603 learned_addr learned my addr 172.21.15.7:0/1911494603 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:48.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.334+0000 7fe671000640 1 --2- 172.21.15.7:0/1911494603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe66c0b21c0 0x7fe66c15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.334+0000 7fe672400640 1 -- 172.21.15.7:0/1911494603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe66c0b21c0 msgr2=0x7fe66c15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.334+0000 7fe672400640 1 --2- 172.21.15.7:0/1911494603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe66c0b21c0 0x7fe66c15dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.334+0000 7fe672400640 1 -- 172.21.15.7:0/1911494603 >> v1:172.21.15.7:6789/0 conn(0x7fe66c0d1f80 legacy=0x7fe66c0b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.334+0000 7fe672400640 1 -- 172.21.15.7:0/1911494603 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c0d2820 con 0x7fe66c161230 2024-04-25T11:11:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.334+0000 7fe66be00640 1 -- 172.21.15.7:0/1911494603 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1558584493 0 0) 0x7fe66c0b29a0 con 0x7fe66c0d1f80 2024-04-25T11:11:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.335+0000 7fe672400640 1 --2- 172.21.15.7:0/1911494603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe66c161230 0x7fe66c06ca00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe6640097a0 tx=0x7fe66402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590cfe9fe40edc31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:48.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.335+0000 7fe66be00640 1 -- 172.21.15.7:0/1911494603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664037070 con 0x7fe66c161230 2024-04-25T11:11:48.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.335+0000 7fe66be00640 1 -- 172.21.15.7:0/1911494603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66402ae50 con 0x7fe66c161230 2024-04-25T11:11:48.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.335+0000 7fe66be00640 1 -- 172.21.15.7:0/1911494603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664032660 con 0x7fe66c161230 2024-04-25T11:11:48.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.336+0000 7fe674400640 1 -- 172.21.15.7:0/1911494603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe66c161230 msgr2=0x7fe66c06ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.336+0000 7fe674400640 1 --2- 172.21.15.7:0/1911494603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe66c161230 0x7fe66c06ca00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe6640097a0 tx=0x7fe66402a100 comp rx=0 tx=0).stop 2024-04-25T11:11:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.336+0000 7fe674400640 1 -- 172.21.15.7:0/1911494603 shutdown_connections 2024-04-25T11:11:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.336+0000 7fe674400640 1 --2- 172.21.15.7:0/1911494603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe66c161230 0x7fe66c06ca00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.336+0000 7fe674400640 1 --2- 172.21.15.7:0/1911494603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe66c0b21c0 0x7fe66c15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.336+0000 7fe674400640 1 -- 172.21.15.7:0/1911494603 >> 172.21.15.7:0/1911494603 conn(0x7fe66c073d30 msgr2=0x7fe66c09b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.336+0000 7fe674400640 1 -- 172.21.15.7:0/1911494603 shutdown_connections 2024-04-25T11:11:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.336+0000 7fe674400640 1 -- 172.21.15.7:0/1911494603 wait complete. 2024-04-25T11:11:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.339+0000 7fe674400640 1 Processor -- start 2024-04-25T11:11:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.339+0000 7fe674400640 1 -- start start 2024-04-25T11:11:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe674400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe66c0d1f80 0x7fe66c162be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe674400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe66c161230 0x7fe66c14b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe674400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe66c0cf460 0x7fe66c0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe674400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c161640 con 0x7fe66c0d1f80 2024-04-25T11:11:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe674400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0d2de0 con 0x7fe66c0cf460 2024-04-25T11:11:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe674400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c06cf40 con 0x7fe66c161230 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe671a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe66c0d1f80 0x7fe66c162be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe671a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe66c0d1f80 0x7fe66c162be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40432/0 (socket says 172.21.15.7:40432) 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe671a00640 1 -- 172.21.15.7:0/2098247417 learned_addr learned my addr 172.21.15.7:0/2098247417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe671000640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe66c161230 0x7fe66c14b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe672400640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe66c0cf460 0x7fe66c0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe671a00640 1 -- 172.21.15.7:0/2098247417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe66c161230 msgr2=0x7fe66c14b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe671a00640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe66c161230 0x7fe66c14b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe671a00640 1 -- 172.21.15.7:0/2098247417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe66c0cf460 msgr2=0x7fe66c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe671a00640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe66c0cf460 0x7fe66c0cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe671a00640 1 -- 172.21.15.7:0/2098247417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c161b40 con 0x7fe66c0d1f80 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.341+0000 7fe671000640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe66c161230 0x7fe66c14b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.342+0000 7fe671a00640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe66c0d1f80 0x7fe66c162be0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe66000cac0 tx=0x7fe66000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.342+0000 7fe669600640 1 -- 172.21.15.7:0/2098247417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660017070 con 0x7fe66c0d1f80 2024-04-25T11:11:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.342+0000 7fe669600640 1 -- 172.21.15.7:0/2098247417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66000ac30 con 0x7fe66c0d1f80 2024-04-25T11:11:48.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.342+0000 7fe669600640 1 -- 172.21.15.7:0/2098247417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660012400 con 0x7fe66c0d1f80 2024-04-25T11:11:48.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.342+0000 7fe674400640 1 -- 172.21.15.7:0/2098247417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c161e20 con 0x7fe66c0d1f80 2024-04-25T11:11:48.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.342+0000 7fe674400640 1 -- 172.21.15.7:0/2098247417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c1623e0 con 0x7fe66c0d1f80 2024-04-25T11:11:48.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.343+0000 7fe674400640 1 -- 172.21.15.7:0/2098247417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c070bb0 con 0x7fe66c0d1f80 2024-04-25T11:11:48.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.345+0000 7fe669600640 1 -- 172.21.15.7:0/2098247417 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe66001a070 con 0x7fe66c0d1f80 2024-04-25T11:11:48.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.346+0000 7fe669600640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe64c0744b0 0x7fe64c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.346+0000 7fe669600640 1 -- 172.21.15.7:0/2098247417 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fe660095450 con 0x7fe66c0d1f80 2024-04-25T11:11:48.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.346+0000 7fe671000640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe64c0744b0 0x7fe64c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.347+0000 7fe671000640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe64c0744b0 0x7fe64c0769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe658006fd0 tx=0x7fe658008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:48.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.350+0000 7fe669600640 1 -- 172.21.15.7:0/2098247417 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe66c070bb0 con 0x7fe66c0d1f80 2024-04-25T11:11:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.549+0000 7fe674400640 1 -- 172.21.15.7:0/2098247417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe66c050740 con 0x7fe66c0d1f80 2024-04-25T11:11:48.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.550+0000 7fe669600640 1 -- 172.21.15.7:0/2098247417 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fe66005f740 con 0x7fe66c0d1f80 2024-04-25T11:11:48.551 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.552+0000 7fe674400640 1 -- 172.21.15.7:0/2098247417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe64c0744b0 msgr2=0x7fe64c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.552+0000 7fe674400640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe64c0744b0 0x7fe64c0769c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe658006fd0 tx=0x7fe658008040 comp rx=0 tx=0).stop 2024-04-25T11:11:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.553+0000 7fe674400640 1 -- 172.21.15.7:0/2098247417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe66c0d1f80 msgr2=0x7fe66c162be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.553+0000 7fe674400640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe66c0d1f80 0x7fe66c162be0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe66000cac0 tx=0x7fe66000cf80 comp rx=0 tx=0).stop 2024-04-25T11:11:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.553+0000 7fe674400640 1 -- 172.21.15.7:0/2098247417 shutdown_connections 2024-04-25T11:11:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.553+0000 7fe674400640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe64c0744b0 0x7fe64c0769c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.553+0000 7fe674400640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe66c0cf460 0x7fe66c0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.553+0000 7fe674400640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe66c161230 0x7fe66c14b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.553+0000 7fe674400640 1 --2- 172.21.15.7:0/2098247417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe66c0d1f80 0x7fe66c162be0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.553+0000 7fe674400640 1 -- 172.21.15.7:0/2098247417 >> 172.21.15.7:0/2098247417 conn(0x7fe66c073d30 msgr2=0x7fe66c0af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.553+0000 7fe674400640 1 -- 172.21.15.7:0/2098247417 shutdown_connections 2024-04-25T11:11:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.553+0000 7fe674400640 1 -- 172.21.15.7:0/2098247417 wait complete. 2024-04-25T11:11:48.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:11:48.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.850+0000 7fdd85a00640 1 Processor -- start 2024-04-25T11:11:48.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.850+0000 7fdd85a00640 1 -- start start 2024-04-25T11:11:48.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.852+0000 7fdd85a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdd8015eed0 0x7fdd800cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.852+0000 7fdd85a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd80141290 0x7fdd801416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.852+0000 7fdd85a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd8006c570 con 0x7fdd8006d610 2024-04-25T11:11:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.852+0000 7fdd85a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fdd8006c850 con 0x7fdd8015eed0 2024-04-25T11:11:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.852+0000 7fdd85a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8006cb30 con 0x7fdd80141290 2024-04-25T11:11:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.852+0000 7fdd7fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdd8015eed0 0x7fdd800cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.853+0000 7fdd7fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdd8015eed0 0x7fdd800cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40448/0 (socket says 172.21.15.7:40448) 2024-04-25T11:11:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.853+0000 7fdd7fe00640 1 -- 172.21.15.7:0/3511717643 learned_addr learned my addr 172.21.15.7:0/3511717643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:48.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.853+0000 7fdd77e00640 1 --2- 172.21.15.7:0/3511717643 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd80141290 0x7fdd801416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.853+0000 7fdd7fe00640 1 -- 172.21.15.7:0/3511717643 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd80141290 msgr2=0x7fdd801416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.853+0000 7fdd7fe00640 1 --2- 172.21.15.7:0/3511717643 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd80141290 0x7fdd801416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.853+0000 7fdd7fe00640 1 -- 172.21.15.7:0/3511717643 >> v1:172.21.15.87:6789/0 conn(0x7fdd8006d610 legacy=0x7fdd800b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.853+0000 7fdd7fe00640 1 -- 172.21.15.7:0/3511717643 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8006ce10 con 0x7fdd8015eed0 2024-04-25T11:11:48.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.853+0000 7fdd77e00640 1 --2- 172.21.15.7:0/3511717643 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd80141290 0x7fdd801416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:11:48.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.855+0000 7fdd7fe00640 1 --2- 172.21.15.7:0/3511717643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd8015eed0 0x7fdd800cd780 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009870 tx=0x7fdd7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7abac58910803c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:48.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.855+0000 7fdd7f400640 1 -- 172.21.15.7:0/3511717643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70037070 con 0x7fdd8015eed0 2024-04-25T11:11:48.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.855+0000 7fdd7f400640 1 -- 172.21.15.7:0/3511717643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd70032440 con 0x7fdd8015eed0 2024-04-25T11:11:48.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.855+0000 7fdd7f400640 1 -- 172.21.15.7:0/3511717643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70032810 con 0x7fdd8015eed0 2024-04-25T11:11:48.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.856+0000 7fdd85a00640 1 -- 172.21.15.7:0/3511717643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd8015eed0 msgr2=0x7fdd800cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.856+0000 7fdd85a00640 1 --2- 172.21.15.7:0/3511717643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd8015eed0 0x7fdd800cd780 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009870 tx=0x7fdd7002a270 comp rx=0 tx=0).stop 2024-04-25T11:11:48.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.856+0000 7fdd85a00640 1 -- 172.21.15.7:0/3511717643 shutdown_connections 2024-04-25T11:11:48.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.856+0000 7fdd85a00640 1 --2- 172.21.15.7:0/3511717643 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd80141290 0x7fdd801416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.856+0000 7fdd85a00640 1 --2- 172.21.15.7:0/3511717643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd8015eed0 0x7fdd800cd780 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.856+0000 7fdd85a00640 1 -- 172.21.15.7:0/3511717643 >> 172.21.15.7:0/3511717643 conn(0x7fdd800d1540 msgr2=0x7fdd800d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:48.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.856+0000 7fdd85a00640 1 -- 172.21.15.7:0/3511717643 shutdown_connections 2024-04-25T11:11:48.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.856+0000 7fdd85a00640 1 -- 172.21.15.7:0/3511717643 wait complete. 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.858+0000 7fdd85a00640 1 Processor -- start 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.858+0000 7fdd85a00640 1 -- start start 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.860+0000 7fdd85a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd8006d610 0x7fdd80131280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.860+0000 7fdd85a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80141290 0x7fdd8012e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.860+0000 7fdd85a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd8015eed0 0x7fdd80152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.860+0000 7fdd85a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdd80140da0 con 0x7fdd80141290 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.860+0000 7fdd85a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdd800cdcc0 con 0x7fdd8006d610 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.860+0000 7fdd85a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdd80140f20 con 0x7fdd8015eed0 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.860+0000 7fdd77e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80141290 0x7fdd8012e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.860+0000 7fdd77e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80141290 0x7fdd8012e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40454/0 (socket says 172.21.15.7:40454) 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.860+0000 7fdd77e00640 1 -- 172.21.15.7:0/63852860 learned_addr learned my addr 172.21.15.7:0/63852860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.860+0000 7fdd85000640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd8015eed0 0x7fdd80152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.861+0000 7fdd7fe00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd8006d610 0x7fdd80131280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.861+0000 7fdd77e00640 1 -- 172.21.15.7:0/63852860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd8015eed0 msgr2=0x7fdd80152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.861+0000 7fdd77e00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd8015eed0 0x7fdd80152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.861+0000 7fdd77e00640 1 -- 172.21.15.7:0/63852860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd8006d610 msgr2=0x7fdd80131280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.861+0000 7fdd77e00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd8006d610 0x7fdd80131280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.861+0000 7fdd77e00640 1 -- 172.21.15.7:0/63852860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd80154620 con 0x7fdd80141290 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.861+0000 7fdd77e00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80141290 0x7fdd8012e650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800d7d0 tx=0x7fdd6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.861+0000 7fdd7cc00640 1 -- 172.21.15.7:0/63852860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68004030 con 0x7fdd80141290 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.861+0000 7fdd7cc00640 1 -- 172.21.15.7:0/63852860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd680041d0 con 0x7fdd80141290 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.861+0000 7fdd7cc00640 1 -- 172.21.15.7:0/63852860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6800b6c0 con 0x7fdd80141290 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.861+0000 7fdd85a00640 1 -- 172.21.15.7:0/63852860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd80154900 con 0x7fdd80141290 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.862+0000 7fdd85a00640 1 -- 172.21.15.7:0/63852860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd80152370 con 0x7fdd80141290 2024-04-25T11:11:48.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.863+0000 7fdd85a00640 1 -- 172.21.15.7:0/63852860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd80050740 con 0x7fdd80141290 2024-04-25T11:11:48.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.864+0000 7fdd7cc00640 1 -- 172.21.15.7:0/63852860 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdd6800b860 con 0x7fdd80141290 2024-04-25T11:11:48.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.865+0000 7fdd7cc00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd5c0744e0 0x7fdd5c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:48.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.865+0000 7fdd7cc00640 1 -- 172.21.15.7:0/63852860 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fdd68095900 con 0x7fdd80141290 2024-04-25T11:11:48.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.865+0000 7fdd7fe00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd5c0744e0 0x7fdd5c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:48.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.866+0000 7fdd7fe00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd5c0744e0 0x7fdd5c0769f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd70034000 tx=0x7fdd7002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:48.870+0000 7fdd7cc00640 1 -- 172.21.15.7:0/63852860 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdd6805fbf0 con 0x7fdd80141290 2024-04-25T11:11:49.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.079+0000 7fdd85a00640 1 -- 172.21.15.7:0/63852860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdd80154120 con 0x7fdd80141290 2024-04-25T11:11:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.080+0000 7fdd7cc00640 1 -- 172.21.15.7:0/63852860 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fdd6805f4f0 con 0x7fdd80141290 2024-04-25T11:11:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:11:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.082+0000 7fdd85a00640 1 -- 172.21.15.7:0/63852860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd5c0744e0 msgr2=0x7fdd5c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.082+0000 7fdd85a00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd5c0744e0 0x7fdd5c0769f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd70034000 tx=0x7fdd7002e000 comp rx=0 tx=0).stop 2024-04-25T11:11:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.082+0000 7fdd85a00640 1 -- 172.21.15.7:0/63852860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80141290 msgr2=0x7fdd8012e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.082+0000 7fdd85a00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80141290 0x7fdd8012e650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800d7d0 tx=0x7fdd6800dc90 comp rx=0 tx=0).stop 2024-04-25T11:11:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.083+0000 7fdd85a00640 1 -- 172.21.15.7:0/63852860 shutdown_connections 2024-04-25T11:11:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.083+0000 7fdd85a00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdd5c0744e0 0x7fdd5c0769f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.083+0000 7fdd85a00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd8015eed0 0x7fdd80152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.083+0000 7fdd85a00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80141290 0x7fdd8012e650 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.083+0000 7fdd85a00640 1 --2- 172.21.15.7:0/63852860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd8006d610 0x7fdd80131280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.083+0000 7fdd85a00640 1 -- 172.21.15.7:0/63852860 >> 172.21.15.7:0/63852860 conn(0x7fdd800d1540 msgr2=0x7fdd8015fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:49.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.083+0000 7fdd85a00640 1 -- 172.21.15.7:0/63852860 shutdown_connections 2024-04-25T11:11:49.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.083+0000 7fdd85a00640 1 -- 172.21.15.7:0/63852860 wait complete. 2024-04-25T11:11:49.109 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:11:49.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:49.110 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T11:11:49.209 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:11:49.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:49.209 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:11:49.268 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:11:49.268 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:49.325 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:11:49.325 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:49.381 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:11:49.381 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:49.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.724+0000 7f08bf000640 1 Processor -- start 2024-04-25T11:11:49.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.724+0000 7f08bf000640 1 -- start start 2024-04-25T11:11:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.726+0000 7f08bf000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08b815c390 0x7f08b80b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:49.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.726+0000 7f08bf000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08b806ce00 0x7f08b806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.726+0000 7f08bf000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b806ef10 con 0x7f08b8070890 2024-04-25T11:11:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.726+0000 7f08bf000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f08b806f210 con 0x7f08b815c390 2024-04-25T11:11:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.726+0000 7f08bf000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f08b806d750 con 0x7f08b806ce00 2024-04-25T11:11:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.727+0000 7f08bd200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08b815c390 0x7f08b80b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.727+0000 7f08bd200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08b815c390 0x7f08b80b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40460/0 (socket says 172.21.15.7:40460) 2024-04-25T11:11:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.727+0000 7f08bd200640 1 -- 172.21.15.7:0/1391325871 learned_addr learned my addr 172.21.15.7:0/1391325871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:49.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.727+0000 7f08bdc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f08b8070890 0x7f08b806f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59234/0 (socket says 172.21.15.7:59234) 2024-04-25T11:11:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.727+0000 7f08bd200640 1 -- 172.21.15.7:0/1391325871 >> v2:172.21.15.87:3300/0 conn(0x7f08b806ce00 msgr2=0x7f08b806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.727+0000 7f08be600640 1 --2- 172.21.15.7:0/1391325871 >> v2:172.21.15.87:3300/0 conn(0x7f08b806ce00 0x7f08b806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.727+0000 7f08bd200640 1 --2- 172.21.15.7:0/1391325871 >> v2:172.21.15.87:3300/0 conn(0x7f08b806ce00 0x7f08b806d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.727+0000 7f08bd200640 1 -- 172.21.15.7:0/1391325871 >> v1:172.21.15.7:6789/0 conn(0x7f08b8070890 legacy=0x7f08b806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.727+0000 7f08b7e00640 1 -- 172.21.15.7:0/1391325871 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 970968677 0 0) 0x7f08b806ef10 con 0x7f08b8070890 2024-04-25T11:11:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.727+0000 7f08bd200640 1 -- 172.21.15.7:0/1391325871 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b80b6390 con 0x7f08b815c390 2024-04-25T11:11:49.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.727+0000 7f08be600640 1 --2- 172.21.15.7:0/1391325871 >> v2:172.21.15.87:3300/0 conn(0x7f08b806ce00 0x7f08b806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:49.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.728+0000 7f08bd200640 1 --2- 172.21.15.7:0/1391325871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b815c390 0x7f08b80b5140 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f08b80722d0 tx=0x7f08a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b4a210dd26ee87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:49.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.729+0000 7f08b7e00640 1 -- 172.21.15.7:0/1391325871 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4037070 con 0x7f08b815c390 2024-04-25T11:11:49.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.729+0000 7f08b7e00640 1 -- 172.21.15.7:0/1391325871 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a4032440 con 0x7f08b815c390 2024-04-25T11:11:49.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.729+0000 7f08b7e00640 1 -- 172.21.15.7:0/1391325871 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4032810 con 0x7f08b815c390 2024-04-25T11:11:49.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.729+0000 7f08bf000640 1 -- 172.21.15.7:0/1391325871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b815c390 msgr2=0x7f08b80b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:49.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.729+0000 7f08bf000640 1 --2- 172.21.15.7:0/1391325871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b815c390 0x7f08b80b5140 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f08b80722d0 tx=0x7f08a402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.729+0000 7f08bf000640 1 -- 172.21.15.7:0/1391325871 shutdown_connections 2024-04-25T11:11:49.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.729+0000 7f08bf000640 1 --2- 172.21.15.7:0/1391325871 >> v2:172.21.15.87:3300/0 conn(0x7f08b806ce00 0x7f08b806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.729+0000 7f08bf000640 1 --2- 172.21.15.7:0/1391325871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b815c390 0x7f08b80b5140 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.730+0000 7f08bf000640 1 -- 172.21.15.7:0/1391325871 >> 172.21.15.7:0/1391325871 conn(0x7f08b80b3820 msgr2=0x7f08b80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:49.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.730+0000 7f08bf000640 1 -- 172.21.15.7:0/1391325871 shutdown_connections 2024-04-25T11:11:49.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.730+0000 7f08bf000640 1 -- 172.21.15.7:0/1391325871 wait complete. 2024-04-25T11:11:49.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.732+0000 7f08bf000640 1 Processor -- start 2024-04-25T11:11:49.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.732+0000 7f08bf000640 1 -- start start 2024-04-25T11:11:49.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08bf000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b806ce00 0x7f08b815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08bf000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08b8070890 0x7f08b8152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08bf000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b815c390 0x7f08b815dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08bf000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08b815c7a0 con 0x7f08b815c390 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08bf000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f08b806fdb0 con 0x7f08b806ce00 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08bf000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f08b80b6950 con 0x7f08b8070890 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08be600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b815c390 0x7f08b815dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08be600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b815c390 0x7f08b815dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40472/0 (socket says 172.21.15.7:40472) 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08be600640 1 -- 172.21.15.7:0/3653214164 learned_addr learned my addr 172.21.15.7:0/3653214164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08bdc00640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b806ce00 0x7f08b815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08be600640 1 -- 172.21.15.7:0/3653214164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08b8070890 msgr2=0x7f08b8152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08be600640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08b8070890 0x7f08b8152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08be600640 1 -- 172.21.15.7:0/3653214164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b806ce00 msgr2=0x7f08b815a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:49.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08be600640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b806ce00 0x7f08b815a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08be600640 1 -- 172.21.15.7:0/3653214164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b815ac60 con 0x7f08b815c390 2024-04-25T11:11:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.734+0000 7f08bdc00640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b806ce00 0x7f08b815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.735+0000 7f08be600640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b815c390 0x7f08b815dbd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f08b000c7f0 tx=0x7f08b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:49.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.735+0000 7f08b5600640 1 -- 172.21.15.7:0/3653214164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0015020 con 0x7f08b815c390 2024-04-25T11:11:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.735+0000 7f08b5600640 1 -- 172.21.15.7:0/3653214164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b0004230 con 0x7f08b815c390 2024-04-25T11:11:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.735+0000 7f08bf000640 1 -- 172.21.15.7:0/3653214164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b815af40 con 0x7f08b815c390 2024-04-25T11:11:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.735+0000 7f08b5600640 1 -- 172.21.15.7:0/3653214164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0007620 con 0x7f08b815c390 2024-04-25T11:11:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.735+0000 7f08bf000640 1 -- 172.21.15.7:0/3653214164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b815b500 con 0x7f08b815c390 2024-04-25T11:11:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.736+0000 7f08bf000640 1 -- 172.21.15.7:0/3653214164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b815a9d0 con 0x7f08b815c390 2024-04-25T11:11:49.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.738+0000 7f08b5600640 1 -- 172.21.15.7:0/3653214164 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f08b0018020 con 0x7f08b815c390 2024-04-25T11:11:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.739+0000 7f08b5600640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08980744b0 0x7f08980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.739+0000 7f08b5600640 1 -- 172.21.15.7:0/3653214164 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f08b0095930 con 0x7f08b815c390 2024-04-25T11:11:49.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.740+0000 7f08bdc00640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08980744b0 0x7f08980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:49.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.740+0000 7f08bdc00640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08980744b0 0x7f08980769c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08a8007fd0 tx=0x7f08a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:49.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.744+0000 7f08b5600640 1 -- 172.21.15.7:0/3653214164 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f08b001f020 con 0x7f08b815c390 2024-04-25T11:11:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.951+0000 7f08bf000640 1 -- 172.21.15.7:0/3653214164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08b8050740 con 0x7f08b815c390 2024-04-25T11:11:49.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.953+0000 7f08b5600640 1 -- 172.21.15.7:0/3653214164 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7f08b005fd70 con 0x7f08b815c390 2024-04-25T11:11:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:49.955 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:49.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.956+0000 7f08bf000640 1 -- 172.21.15.7:0/3653214164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08980744b0 msgr2=0x7f08980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:49.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.956+0000 7f08bf000640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08980744b0 0x7f08980769c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f08a8007fd0 tx=0x7f08a8009040 comp rx=0 tx=0).stop 2024-04-25T11:11:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.956+0000 7f08bf000640 1 -- 172.21.15.7:0/3653214164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b815c390 msgr2=0x7f08b815dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.956+0000 7f08bf000640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b815c390 0x7f08b815dbd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f08b000c7f0 tx=0x7f08b000ccb0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.956+0000 7f08bf000640 1 -- 172.21.15.7:0/3653214164 shutdown_connections 2024-04-25T11:11:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.957+0000 7f08bf000640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08980744b0 0x7f08980769c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.957+0000 7f08bf000640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08b815c390 0x7f08b815dbd0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.957+0000 7f08bf000640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08b8070890 0x7f08b8152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.957+0000 7f08bf000640 1 --2- 172.21.15.7:0/3653214164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08b806ce00 0x7f08b815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.957+0000 7f08bf000640 1 -- 172.21.15.7:0/3653214164 >> 172.21.15.7:0/3653214164 conn(0x7f08b80b3820 msgr2=0x7f08b80684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.957+0000 7f08bf000640 1 -- 172.21.15.7:0/3653214164 shutdown_connections 2024-04-25T11:11:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:49.957+0000 7f08bf000640 1 -- 172.21.15.7:0/3653214164 wait complete. 2024-04-25T11:11:49.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:49.989 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:50.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.288+0000 7f957a600640 1 Processor -- start 2024-04-25T11:11:50.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.288+0000 7f957a600640 1 -- start start 2024-04-25T11:11:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.290+0000 7f957a600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95740b0470 0x7f957406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.290+0000 7f957a600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f957406f8e0 0x7f95740aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.290+0000 7f957a600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f957415e3c0 con 0x7f957415dfb0 2024-04-25T11:11:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.290+0000 7f957a600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f95740cf800 con 0x7f957406f8e0 2024-04-25T11:11:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.290+0000 7f957a600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f95740d1250 con 0x7f95740b0470 2024-04-25T11:11:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.291+0000 7f9573400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f957415dfb0 0x7f95740d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51568/0 (socket says 172.21.15.7:51568) 2024-04-25T11:11:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.291+0000 7f9573400640 1 -- 172.21.15.7:0/1618979695 learned_addr learned my addr 172.21.15.7:0/1618979695 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.291+0000 7f9573e00640 1 --2- 172.21.15.7:0/1618979695 >> v2:172.21.15.7:3300/0 conn(0x7f957406f8e0 0x7f95740aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.291+0000 7f9572a00640 1 --2- 172.21.15.7:0/1618979695 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95740b0470 0x7f957406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.292+0000 7f9573e00640 1 -- 172.21.15.7:0/1618979695 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95740b0470 msgr2=0x7f957406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.292+0000 7f9573e00640 1 --2- 172.21.15.7:0/1618979695 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95740b0470 0x7f957406f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.291+0000 7f9572000640 1 -- 172.21.15.7:0/1618979695 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3939194338 0 0) 0x7f957415e3c0 con 0x7f957415dfb0 2024-04-25T11:11:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.292+0000 7f9573e00640 1 -- 172.21.15.7:0/1618979695 >> v1:172.21.15.7:6789/0 conn(0x7f957415dfb0 legacy=0x7f95740d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.292+0000 7f9573e00640 1 -- 172.21.15.7:0/1618979695 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95740cfaa0 con 0x7f957406f8e0 2024-04-25T11:11:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.293+0000 7f9573e00640 1 --2- 172.21.15.7:0/1618979695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f957406f8e0 0x7f95740aea90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f956c0097a0 tx=0x7f956c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f17a29b550c5e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.293+0000 7f9572000640 1 -- 172.21.15.7:0/1618979695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c037070 con 0x7f957406f8e0 2024-04-25T11:11:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.293+0000 7f9572000640 1 -- 172.21.15.7:0/1618979695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c032440 con 0x7f957406f8e0 2024-04-25T11:11:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.293+0000 7f9572000640 1 -- 172.21.15.7:0/1618979695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c032810 con 0x7f957406f8e0 2024-04-25T11:11:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.294+0000 7f957a600640 1 -- 172.21.15.7:0/1618979695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f957406f8e0 msgr2=0x7f95740aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.294+0000 7f957a600640 1 --2- 172.21.15.7:0/1618979695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f957406f8e0 0x7f95740aea90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f956c0097a0 tx=0x7f956c02a180 comp rx=0 tx=0).stop 2024-04-25T11:11:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.294+0000 7f957a600640 1 -- 172.21.15.7:0/1618979695 shutdown_connections 2024-04-25T11:11:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.294+0000 7f957a600640 1 --2- 172.21.15.7:0/1618979695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f957406f8e0 0x7f95740aea90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.294+0000 7f957a600640 1 --2- 172.21.15.7:0/1618979695 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95740b0470 0x7f957406f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.294+0000 7f957a600640 1 -- 172.21.15.7:0/1618979695 >> 172.21.15.7:0/1618979695 conn(0x7f95740742c0 msgr2=0x7f957406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.294+0000 7f957a600640 1 -- 172.21.15.7:0/1618979695 shutdown_connections 2024-04-25T11:11:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.294+0000 7f957a600640 1 -- 172.21.15.7:0/1618979695 wait complete. 2024-04-25T11:11:50.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.296+0000 7f957a600640 1 Processor -- start 2024-04-25T11:11:50.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.297+0000 7f957a600640 1 -- start start 2024-04-25T11:11:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.298+0000 7f957a600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f957406f8e0 0x7f9574153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.298+0000 7f957a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95740b0470 0x7f95741604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.298+0000 7f957a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f957415dfb0 0x7f95741609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.298+0000 7f957a600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95740d0340 con 0x7f957415dfb0 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.298+0000 7f957a600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f95740d0060 con 0x7f95740b0470 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.298+0000 7f957a600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f95740d0620 con 0x7f957406f8e0 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.299+0000 7f9573e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f957415dfb0 0x7f95741609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.299+0000 7f9573e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f957415dfb0 0x7f95741609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58484/0 (socket says 172.21.15.7:58484) 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.299+0000 7f9573e00640 1 -- 172.21.15.7:0/1195033816 learned_addr learned my addr 172.21.15.7:0/1195033816 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.299+0000 7f9572a00640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95740b0470 0x7f95741604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.299+0000 7f9573400640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f957406f8e0 0x7f9574153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.299+0000 7f9573e00640 1 -- 172.21.15.7:0/1195033816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f957406f8e0 msgr2=0x7f9574153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.299+0000 7f9573e00640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f957406f8e0 0x7f9574153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.299+0000 7f9573e00640 1 -- 172.21.15.7:0/1195033816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95740b0470 msgr2=0x7f95741604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.299+0000 7f9573e00640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95740b0470 0x7f95741604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.299+0000 7f9573e00640 1 -- 172.21.15.7:0/1195033816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95740cbe80 con 0x7f957415dfb0 2024-04-25T11:11:50.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.299+0000 7f9572a00640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95740b0470 0x7f95741604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:50.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.300+0000 7f9573e00640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f957415dfb0 0x7f95741609e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f956c0098c0 tx=0x7f956c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:50.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.300+0000 7f956b400640 1 -- 172.21.15.7:0/1195033816 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c037040 con 0x7f957415dfb0 2024-04-25T11:11:50.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.300+0000 7f956b400640 1 -- 172.21.15.7:0/1195033816 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c042070 con 0x7f957415dfb0 2024-04-25T11:11:50.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.300+0000 7f956b400640 1 -- 172.21.15.7:0/1195033816 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c030b30 con 0x7f957415dfb0 2024-04-25T11:11:50.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.300+0000 7f957a600640 1 -- 172.21.15.7:0/1195033816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95740cc160 con 0x7f957415dfb0 2024-04-25T11:11:50.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.300+0000 7f957a600640 1 -- 172.21.15.7:0/1195033816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957415aa70 con 0x7f957415dfb0 2024-04-25T11:11:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.302+0000 7f957a600640 1 -- 172.21.15.7:0/1195033816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957415acd0 con 0x7f957415dfb0 2024-04-25T11:11:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.303+0000 7f956b400640 1 -- 172.21.15.7:0/1195033816 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f956c030cd0 con 0x7f957415dfb0 2024-04-25T11:11:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.304+0000 7f956b400640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9550074500 0x7f9550076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.304+0000 7f956b400640 1 -- 172.21.15.7:0/1195033816 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f956c0b5ea0 con 0x7f957415dfb0 2024-04-25T11:11:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.304+0000 7f9573400640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9550074500 0x7f9550076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.305+0000 7f9573400640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9550074500 0x7f9550076a10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9564000f60 tx=0x7f9564009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:50.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.308+0000 7f956b400640 1 -- 172.21.15.7:0/1195033816 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f956c04c020 con 0x7f957415dfb0 2024-04-25T11:11:50.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.518+0000 7f957a600640 1 -- 172.21.15.7:0/1195033816 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9574050740 con 0x7f957415dfb0 2024-04-25T11:11:50.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.519+0000 7f956b400640 1 -- 172.21.15.7:0/1195033816 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f956c032e20 con 0x7f957415dfb0 2024-04-25T11:11:50.521 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 -- 172.21.15.7:0/1195033816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9550074500 msgr2=0x7f9550076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9550074500 0x7f9550076a10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9564000f60 tx=0x7f9564009040 comp rx=0 tx=0).stop 2024-04-25T11:11:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 -- 172.21.15.7:0/1195033816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f957415dfb0 msgr2=0x7f95741609e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f957415dfb0 0x7f95741609e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f956c0098c0 tx=0x7f956c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 -- 172.21.15.7:0/1195033816 shutdown_connections 2024-04-25T11:11:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9550074500 0x7f9550076a10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f957415dfb0 0x7f95741609e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95740b0470 0x7f95741604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 --2- 172.21.15.7:0/1195033816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f957406f8e0 0x7f9574153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 -- 172.21.15.7:0/1195033816 >> 172.21.15.7:0/1195033816 conn(0x7f95740742c0 msgr2=0x7f95740b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:50.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 -- 172.21.15.7:0/1195033816 shutdown_connections 2024-04-25T11:11:50.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.523+0000 7f957a600640 1 -- 172.21.15.7:0/1195033816 wait complete. 2024-04-25T11:11:50.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:50.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.838+0000 7f525ac00640 1 Processor -- start 2024-04-25T11:11:50.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.839+0000 7f525ac00640 1 -- start start 2024-04-25T11:11:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.841+0000 7f525ac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f525415dfb0 0x7f52540b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.841+0000 7f525ac00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f525415e3c0 con 0x7f52540d0b40 2024-04-25T11:11:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.841+0000 7f525ac00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52540cf800 con 0x7f525406fab0 2024-04-25T11:11:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.841+0000 7f525ac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f52540b09b0 con 0x7f525415dfb0 2024-04-25T11:11:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.842+0000 7f5253400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f52540d0b40 0x7f525406f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51574/0 (socket says 172.21.15.7:51574) 2024-04-25T11:11:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.842+0000 7f5253400640 1 -- 172.21.15.7:0/3158439233 learned_addr learned my addr 172.21.15.7:0/3158439233 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.842+0000 7f5253e00640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f525415dfb0 0x7f52540b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.842+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323718198 0 0) 0x7f525415e3c0 con 0x7f52540d0b40 2024-04-25T11:11:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.842+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5238003520 con 0x7f52540d0b40 2024-04-25T11:11:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2630206072 0 0) 0x7f5238003520 con 0x7f52540d0b40 2024-04-25T11:11:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f525415dfb0 msgr2=0x7f52540b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f525415dfb0 0x7f52540b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 >> v1:172.21.15.87:6789/0 conn(0x7f525406fab0 legacy=0x7f52540aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52540cfaa0 con 0x7f52540d0b40 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f5240001670 con 0x7f52540d0b40 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 >> v1:172.21.15.7:6789/0 conn(0x7f52540d0b40 legacy=0x7f525406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f525415dfb0 0x7f52380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5238003fb0 0x7f5238004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5238004d40 0x7f52380051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52540b09b0 con 0x7f5238003fb0 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f523800bbb0 con 0x7f5238004d40 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f523800bdb0 con 0x7f525415dfb0 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5253400640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5238003fb0 0x7f5238004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.843+0000 7f5258c00640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5238004d40 0x7f52380051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.844+0000 7f5253e00640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f525415dfb0 0x7f52380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.844+0000 7f5253400640 1 -- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f525415dfb0 msgr2=0x7f52380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.844+0000 7f5253400640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f525415dfb0 0x7f52380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.844+0000 7f5253400640 1 -- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5238004d40 msgr2=0x7f52380051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.844+0000 7f5253400640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5238004d40 0x7f52380051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.844+0000 7f5253400640 1 -- 172.21.15.7:0/3158439233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5238003520 con 0x7f5238003fb0 2024-04-25T11:11:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.844+0000 7f5258c00640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5238004d40 0x7f52380051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:11:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.845+0000 7f5253400640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5238003fb0 0x7f5238004800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f524000c0e0 tx=0x7f5238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c516b07b989d51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.846+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240007c40 con 0x7f5238003fb0 2024-04-25T11:11:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.846+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5240005af0 con 0x7f5238003fb0 2024-04-25T11:11:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.846+0000 7f5252a00640 1 -- 172.21.15.7:0/3158439233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5240007070 con 0x7f5238003fb0 2024-04-25T11:11:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.846+0000 7f525ac00640 1 -- 172.21.15.7:0/3158439233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5238003fb0 msgr2=0x7f5238004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.847+0000 7f525ac00640 1 --2- 172.21.15.7:0/3158439233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5238003fb0 0x7f5238004800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f524000c0e0 tx=0x7f5238005830 comp rx=0 tx=0).stop 2024-04-25T11:11:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.847+0000 7f5258c00640 1 -- 172.21.15.7:0/3158439233 reap_dead start 2024-04-25T11:11:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.847+0000 7f525ac00640 1 -- 172.21.15.7:0/3158439233 shutdown_connections 2024-04-25T11:11:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.847+0000 7f525ac00640 1 -- 172.21.15.7:0/3158439233 >> 172.21.15.7:0/3158439233 conn(0x7f52540742c0 msgr2=0x7f525406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.847+0000 7f525ac00640 1 -- 172.21.15.7:0/3158439233 shutdown_connections 2024-04-25T11:11:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.847+0000 7f525ac00640 1 -- 172.21.15.7:0/3158439233 wait complete. 2024-04-25T11:11:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.849+0000 7f525ac00640 1 Processor -- start 2024-04-25T11:11:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.849+0000 7f525ac00640 1 -- start start 2024-04-25T11:11:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.851+0000 7f525ac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5238003fb0 0x7f5254144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.851+0000 7f525ac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f525409e6a0 0x7f5254146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.851+0000 7f525ac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52541f5f90 0x7f52541469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.851+0000 7f525ac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52540d0340 con 0x7f52541f5f90 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.851+0000 7f525ac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f52540d0060 con 0x7f525409e6a0 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.851+0000 7f525ac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f52540d0620 con 0x7f5238003fb0 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.851+0000 7f5258c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52541f5f90 0x7f52541469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.851+0000 7f5258c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52541f5f90 0x7f52541469c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58512/0 (socket says 172.21.15.7:58512) 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.851+0000 7f5258c00640 1 -- 172.21.15.7:0/1520659154 learned_addr learned my addr 172.21.15.7:0/1520659154 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.852+0000 7f5253400640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f525409e6a0 0x7f5254146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.852+0000 7f5253e00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5238003fb0 0x7f5254144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.852+0000 7f5258c00640 1 -- 172.21.15.7:0/1520659154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5238003fb0 msgr2=0x7f5254144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.852+0000 7f5258c00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5238003fb0 0x7f5254144930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.852+0000 7f5258c00640 1 -- 172.21.15.7:0/1520659154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f525409e6a0 msgr2=0x7f5254146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.852+0000 7f5258c00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f525409e6a0 0x7f5254146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.852+0000 7f5258c00640 1 -- 172.21.15.7:0/1520659154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525409eab0 con 0x7f52541f5f90 2024-04-25T11:11:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.852+0000 7f5258c00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52541f5f90 0x7f52541469c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f524c000f80 tx=0x7f524c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.852+0000 7f524be00640 1 -- 172.21.15.7:0/1520659154 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c003970 con 0x7f52541f5f90 2024-04-25T11:11:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.853+0000 7f524be00640 1 -- 172.21.15.7:0/1520659154 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524c018070 con 0x7f52541f5f90 2024-04-25T11:11:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.853+0000 7f524be00640 1 -- 172.21.15.7:0/1520659154 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c003d30 con 0x7f52541f5f90 2024-04-25T11:11:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.853+0000 7f525ac00640 1 -- 172.21.15.7:0/1520659154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52541450f0 con 0x7f52541f5f90 2024-04-25T11:11:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.853+0000 7f525ac00640 1 -- 172.21.15.7:0/1520659154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52541456b0 con 0x7f52541f5f90 2024-04-25T11:11:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.854+0000 7f525ac00640 1 -- 172.21.15.7:0/1520659154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5254050740 con 0x7f52541f5f90 2024-04-25T11:11:50.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.855+0000 7f524be00640 1 -- 172.21.15.7:0/1520659154 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f524c0050d0 con 0x7f52541f5f90 2024-04-25T11:11:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.857+0000 7f524be00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5234074500 0x7f5234076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.857+0000 7f524be00640 1 -- 172.21.15.7:0/1520659154 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f524c095380 con 0x7f52541f5f90 2024-04-25T11:11:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.857+0000 7f5253e00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5234074500 0x7f5234076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:50.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.858+0000 7f5253e00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5234074500 0x7f5234076a10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5254072df0 tx=0x7f5244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:50.862+0000 7f524be00640 1 -- 172.21.15.7:0/1520659154 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f524c05f6f0 con 0x7f52541f5f90 2024-04-25T11:11:51.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.068+0000 7f525ac00640 1 -- 172.21.15.7:0/1520659154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5254144e80 con 0x7f52541f5f90 2024-04-25T11:11:51.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.069+0000 7f524be00640 1 -- 172.21.15.7:0/1520659154 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f524c05eff0 con 0x7f52541f5f90 2024-04-25T11:11:51.070 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.071+0000 7f525ac00640 1 -- 172.21.15.7:0/1520659154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5234074500 msgr2=0x7f5234076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.071+0000 7f525ac00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5234074500 0x7f5234076a10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5254072df0 tx=0x7f5244008040 comp rx=0 tx=0).stop 2024-04-25T11:11:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.071+0000 7f525ac00640 1 -- 172.21.15.7:0/1520659154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52541f5f90 msgr2=0x7f52541469c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.071+0000 7f525ac00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52541f5f90 0x7f52541469c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f524c000f80 tx=0x7f524c00ec10 comp rx=0 tx=0).stop 2024-04-25T11:11:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.071+0000 7f525ac00640 1 -- 172.21.15.7:0/1520659154 shutdown_connections 2024-04-25T11:11:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.071+0000 7f525ac00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5234074500 0x7f5234076a10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.071+0000 7f525ac00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52541f5f90 0x7f52541469c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.071+0000 7f525ac00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f525409e6a0 0x7f5254146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.071+0000 7f525ac00640 1 --2- 172.21.15.7:0/1520659154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5238003fb0 0x7f5254144930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.072+0000 7f525ac00640 1 -- 172.21.15.7:0/1520659154 >> 172.21.15.7:0/1520659154 conn(0x7f52540742c0 msgr2=0x7f52540b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:51.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.072+0000 7f525ac00640 1 -- 172.21.15.7:0/1520659154 shutdown_connections 2024-04-25T11:11:51.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.072+0000 7f525ac00640 1 -- 172.21.15.7:0/1520659154 wait complete. 2024-04-25T11:11:51.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:51.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.383+0000 7f2480800640 1 Processor -- start 2024-04-25T11:11:51.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.383+0000 7f2480800640 1 -- start start 2024-04-25T11:11:51.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.385+0000 7f2480800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f24780b6720 0x7f2478164710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.385+0000 7f2480800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2478157320 0x7f2478160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.385+0000 7f2480800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24780b1390 con 0x7f247806f3a0 2024-04-25T11:11:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.385+0000 7f2480800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2478160a60 con 0x7f24780b6720 2024-04-25T11:11:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.385+0000 7f2480800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f24780ae880 con 0x7f2478157320 2024-04-25T11:11:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.386+0000 7f247e800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f247806f3a0 0x7f247806f7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51588/0 (socket says 172.21.15.7:51588) 2024-04-25T11:11:51.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.386+0000 7f247e800640 1 -- 172.21.15.7:0/2796251540 learned_addr learned my addr 172.21.15.7:0/2796251540 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.386+0000 7f247de00640 1 --2- 172.21.15.7:0/2796251540 >> v2:172.21.15.87:3300/0 conn(0x7f24780b6720 0x7f2478164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.386+0000 7f247d400640 1 --2- 172.21.15.7:0/2796251540 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2478157320 0x7f2478160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.386+0000 7f247ca00640 1 -- 172.21.15.7:0/2796251540 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3171884880 0 0) 0x7f24780b1390 con 0x7f247806f3a0 2024-04-25T11:11:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.386+0000 7f247ca00640 1 -- 172.21.15.7:0/2796251540 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2464003520 con 0x7f247806f3a0 2024-04-25T11:11:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.387+0000 7f247de00640 1 -- 172.21.15.7:0/2796251540 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2478157320 msgr2=0x7f2478160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.387+0000 7f247de00640 1 --2- 172.21.15.7:0/2796251540 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2478157320 0x7f2478160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.387+0000 7f247de00640 1 -- 172.21.15.7:0/2796251540 >> v1:172.21.15.7:6789/0 conn(0x7f247806f3a0 legacy=0x7f247806f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.387+0000 7f247ca00640 1 -- 172.21.15.7:0/2796251540 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 577916230 0 0) 0x7f2464003520 con 0x7f247806f3a0 2024-04-25T11:11:51.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.387+0000 7f247de00640 1 -- 172.21.15.7:0/2796251540 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2478160710 con 0x7f24780b6720 2024-04-25T11:11:51.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.388+0000 7f247de00640 1 --2- 172.21.15.7:0/2796251540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24780b6720 0x7f2478164710 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2470009d90 tx=0x7f247002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec8f3befad1f5e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.389+0000 7f247ca00640 1 -- 172.21.15.7:0/2796251540 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474001670 con 0x7f24780b6720 2024-04-25T11:11:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.389+0000 7f247ca00640 1 -- 172.21.15.7:0/2796251540 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2470007540 con 0x7f24780b6720 2024-04-25T11:11:51.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.389+0000 7f247ca00640 1 -- 172.21.15.7:0/2796251540 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470007910 con 0x7f24780b6720 2024-04-25T11:11:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.390+0000 7f2480800640 1 -- 172.21.15.7:0/2796251540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24780b6720 msgr2=0x7f2478164710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.390+0000 7f2480800640 1 --2- 172.21.15.7:0/2796251540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24780b6720 0x7f2478164710 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2470009d90 tx=0x7f247002a5f0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.390+0000 7f2480800640 1 -- 172.21.15.7:0/2796251540 shutdown_connections 2024-04-25T11:11:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.390+0000 7f2480800640 1 --2- 172.21.15.7:0/2796251540 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2478157320 0x7f2478160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.390+0000 7f2480800640 1 --2- 172.21.15.7:0/2796251540 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24780b6720 0x7f2478164710 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.390+0000 7f2480800640 1 -- 172.21.15.7:0/2796251540 >> 172.21.15.7:0/2796251540 conn(0x7f24780742c0 msgr2=0x7f247806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.390+0000 7f2480800640 1 -- 172.21.15.7:0/2796251540 shutdown_connections 2024-04-25T11:11:51.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.390+0000 7f2480800640 1 -- 172.21.15.7:0/2796251540 wait complete. 2024-04-25T11:11:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.392+0000 7f2480800640 1 Processor -- start 2024-04-25T11:11:51.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.392+0000 7f2480800640 1 -- start start 2024-04-25T11:11:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.394+0000 7f2480800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f247806f3a0 0x7f24780cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.394+0000 7f2480800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24780b6720 0x7f247814dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.394+0000 7f2480800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2478157320 0x7f2478152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.394+0000 7f2480800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2478161430 con 0x7f24780b6720 2024-04-25T11:11:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.394+0000 7f2480800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f24780aee00 con 0x7f247806f3a0 2024-04-25T11:11:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.394+0000 7f2480800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f24781615b0 con 0x7f2478157320 2024-04-25T11:11:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.394+0000 7f247d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24780b6720 0x7f247814dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.394+0000 7f247d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24780b6720 0x7f247814dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58528/0 (socket says 172.21.15.7:58528) 2024-04-25T11:11:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.394+0000 7f247d400640 1 -- 172.21.15.7:0/3239262840 learned_addr learned my addr 172.21.15.7:0/3239262840 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f247de00640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f247806f3a0 0x7f24780cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f247d400640 1 -- 172.21.15.7:0/3239262840 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2478157320 msgr2=0x7f2478152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f247e800640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2478157320 0x7f2478152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f247d400640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2478157320 0x7f2478152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f247d400640 1 -- 172.21.15.7:0/3239262840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f247806f3a0 msgr2=0x7f24780cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f247d400640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f247806f3a0 0x7f24780cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f247d400640 1 -- 172.21.15.7:0/3239262840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247815af20 con 0x7f24780b6720 2024-04-25T11:11:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f247de00640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f247806f3a0 0x7f24780cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f247d400640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24780b6720 0x7f247814dd20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f246800e870 tx=0x7f246800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f246e000640 1 -- 172.21.15.7:0/3239262840 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2468017070 con 0x7f24780b6720 2024-04-25T11:11:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f246e000640 1 -- 172.21.15.7:0/3239262840 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246800c940 con 0x7f24780b6720 2024-04-25T11:11:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f246e000640 1 -- 172.21.15.7:0/3239262840 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246800cd30 con 0x7f24780b6720 2024-04-25T11:11:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.395+0000 7f2480800640 1 -- 172.21.15.7:0/3239262840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247815b200 con 0x7f24780b6720 2024-04-25T11:11:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.396+0000 7f2480800640 1 -- 172.21.15.7:0/3239262840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247815b7c0 con 0x7f24780b6720 2024-04-25T11:11:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.396+0000 7f2480800640 1 -- 172.21.15.7:0/3239262840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2478050740 con 0x7f24780b6720 2024-04-25T11:11:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.398+0000 7f246e000640 1 -- 172.21.15.7:0/3239262840 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f246801a070 con 0x7f24780b6720 2024-04-25T11:11:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.399+0000 7f246e000640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f245c0744b0 0x7f245c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.399+0000 7f246e000640 1 -- 172.21.15.7:0/3239262840 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f2468095980 con 0x7f24780b6720 2024-04-25T11:11:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.400+0000 7f247de00640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f245c0744b0 0x7f245c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.400+0000 7f247de00640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f245c0744b0 0x7f245c0769c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2470034000 tx=0x7f2470033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:51.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.405+0000 7f246e000640 1 -- 172.21.15.7:0/3239262840 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f246805fc70 con 0x7f24780b6720 2024-04-25T11:11:51.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.612+0000 7f2480800640 1 -- 172.21.15.7:0/3239262840 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f247815acb0 con 0x7f24780b6720 2024-04-25T11:11:51.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.613+0000 7f246e000640 1 -- 172.21.15.7:0/3239262840 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f246805f570 con 0x7f24780b6720 2024-04-25T11:11:51.614 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.616+0000 7f2480800640 1 -- 172.21.15.7:0/3239262840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f245c0744b0 msgr2=0x7f245c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.616+0000 7f2480800640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f245c0744b0 0x7f245c0769c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2470034000 tx=0x7f2470033040 comp rx=0 tx=0).stop 2024-04-25T11:11:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.616+0000 7f2480800640 1 -- 172.21.15.7:0/3239262840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24780b6720 msgr2=0x7f247814dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.616+0000 7f2480800640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24780b6720 0x7f247814dd20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f246800e870 tx=0x7f246800ed30 comp rx=0 tx=0).stop 2024-04-25T11:11:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.616+0000 7f2480800640 1 -- 172.21.15.7:0/3239262840 shutdown_connections 2024-04-25T11:11:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.616+0000 7f2480800640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f245c0744b0 0x7f245c0769c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.616+0000 7f2480800640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2478157320 0x7f2478152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.616+0000 7f2480800640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24780b6720 0x7f247814dd20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.616+0000 7f2480800640 1 --2- 172.21.15.7:0/3239262840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f247806f3a0 0x7f24780cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.616+0000 7f2480800640 1 -- 172.21.15.7:0/3239262840 >> 172.21.15.7:0/3239262840 conn(0x7f24780742c0 msgr2=0x7f2478154db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.617+0000 7f2480800640 1 -- 172.21.15.7:0/3239262840 shutdown_connections 2024-04-25T11:11:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.617+0000 7f2480800640 1 -- 172.21.15.7:0/3239262840 wait complete. 2024-04-25T11:11:51.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.931+0000 7f8951600640 1 Processor -- start 2024-04-25T11:11:51.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.932+0000 7f8951600640 1 -- start start 2024-04-25T11:11:51.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.934+0000 7f8951600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f894c1583c0 0x7f894c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.934+0000 7f8951600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f894c06eea0 0x7f894c06f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.934+0000 7f8951600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f894c0ce570 con 0x7f894c0cfb50 2024-04-25T11:11:51.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.934+0000 7f8951600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f894c06f7f0 con 0x7f894c06eea0 2024-04-25T11:11:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.934+0000 7f8951600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f894c15f190 con 0x7f894c1583c0 2024-04-25T11:11:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.935+0000 7f8950c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f894c06eea0 0x7f894c06f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.935+0000 7f8950c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f894c06eea0 0x7f894c06f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58538/0 (socket says 172.21.15.7:58538) 2024-04-25T11:11:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.935+0000 7f8950c00640 1 -- 172.21.15.7:0/507105084 learned_addr learned my addr 172.21.15.7:0/507105084 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.935+0000 7f894be00640 1 --2- 172.21.15.7:0/507105084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f894c1583c0 0x7f894c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.935+0000 7f8950c00640 1 -- 172.21.15.7:0/507105084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f894c1583c0 msgr2=0x7f894c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.935+0000 7f8950c00640 1 --2- 172.21.15.7:0/507105084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f894c1583c0 0x7f894c0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.935+0000 7f8950c00640 1 -- 172.21.15.7:0/507105084 >> v1:172.21.15.87:6789/0 conn(0x7f894c0cfb50 legacy=0x7f894c0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.935+0000 7f8950c00640 1 -- 172.21.15.7:0/507105084 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c0ce850 con 0x7f894c06eea0 2024-04-25T11:11:51.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.936+0000 7f8950c00640 1 --2- 172.21.15.7:0/507105084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f894c06eea0 0x7f894c06f2b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8944009e50 tx=0x7f894402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ff3402696309e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.937+0000 7f894aa00640 1 -- 172.21.15.7:0/507105084 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944036070 con 0x7f894c06eea0 2024-04-25T11:11:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.937+0000 7f894aa00640 1 -- 172.21.15.7:0/507105084 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8944031600 con 0x7f894c06eea0 2024-04-25T11:11:51.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.937+0000 7f894aa00640 1 -- 172.21.15.7:0/507105084 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89440319d0 con 0x7f894c06eea0 2024-04-25T11:11:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.938+0000 7f8951600640 1 -- 172.21.15.7:0/507105084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f894c06eea0 msgr2=0x7f894c06f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.938+0000 7f8951600640 1 --2- 172.21.15.7:0/507105084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f894c06eea0 0x7f894c06f2b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8944009e50 tx=0x7f894402a710 comp rx=0 tx=0).stop 2024-04-25T11:11:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.938+0000 7f8951600640 1 -- 172.21.15.7:0/507105084 shutdown_connections 2024-04-25T11:11:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.938+0000 7f8951600640 1 --2- 172.21.15.7:0/507105084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f894c06eea0 0x7f894c06f2b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.938+0000 7f8951600640 1 --2- 172.21.15.7:0/507105084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f894c1583c0 0x7f894c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.938+0000 7f8951600640 1 -- 172.21.15.7:0/507105084 >> 172.21.15.7:0/507105084 conn(0x7f894c0740f0 msgr2=0x7f894c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.938+0000 7f8951600640 1 -- 172.21.15.7:0/507105084 shutdown_connections 2024-04-25T11:11:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.938+0000 7f8951600640 1 -- 172.21.15.7:0/507105084 wait complete. 2024-04-25T11:11:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.940+0000 7f8951600640 1 Processor -- start 2024-04-25T11:11:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.940+0000 7f8951600640 1 -- start start 2024-04-25T11:11:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.942+0000 7f8951600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f894c06eea0 0x7f894c14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.942+0000 7f8951600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f894c0cfb50 0x7f894c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.942+0000 7f8951600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f894c1583c0 0x7f894c15d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.942+0000 7f8951600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f894c0cf0f0 con 0x7f894c1583c0 2024-04-25T11:11:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.942+0000 7f8951600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f894c0cee10 con 0x7f894c0cfb50 2024-04-25T11:11:51.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.942+0000 7f8951600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f894c0d0b90 con 0x7f894c06eea0 2024-04-25T11:11:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.942+0000 7f8950c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f894c1583c0 0x7f894c15d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.942+0000 7f8950c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f894c1583c0 0x7f894c15d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58540/0 (socket says 172.21.15.7:58540) 2024-04-25T11:11:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.942+0000 7f8950c00640 1 -- 172.21.15.7:0/1871207194 learned_addr learned my addr 172.21.15.7:0/1871207194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.943+0000 7f894b400640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f894c0cfb50 0x7f894c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.943+0000 7f894be00640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f894c06eea0 0x7f894c14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.943+0000 7f8950c00640 1 -- 172.21.15.7:0/1871207194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f894c06eea0 msgr2=0x7f894c14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.943+0000 7f8950c00640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f894c06eea0 0x7f894c14bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.943+0000 7f8950c00640 1 -- 172.21.15.7:0/1871207194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f894c0cfb50 msgr2=0x7f894c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.943+0000 7f8950c00640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f894c0cfb50 0x7f894c15cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.943+0000 7f8950c00640 1 -- 172.21.15.7:0/1871207194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894c1e8110 con 0x7f894c1583c0 2024-04-25T11:11:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.943+0000 7f8950c00640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f894c1583c0 0x7f894c15d1b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f894402ad00 tx=0x7f8944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.943+0000 7f8943e00640 1 -- 172.21.15.7:0/1871207194 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894403f020 con 0x7f894c1583c0 2024-04-25T11:11:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.944+0000 7f8943e00640 1 -- 172.21.15.7:0/1871207194 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8944037040 con 0x7f894c1583c0 2024-04-25T11:11:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.944+0000 7f8951600640 1 -- 172.21.15.7:0/1871207194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894c1e83f0 con 0x7f894c1583c0 2024-04-25T11:11:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.944+0000 7f8943e00640 1 -- 172.21.15.7:0/1871207194 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8944036070 con 0x7f894c1583c0 2024-04-25T11:11:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.944+0000 7f8951600640 1 -- 172.21.15.7:0/1871207194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f894c14c2a0 con 0x7f894c1583c0 2024-04-25T11:11:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.945+0000 7f8951600640 1 -- 172.21.15.7:0/1871207194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f894c050740 con 0x7f894c1583c0 2024-04-25T11:11:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.946+0000 7f8943e00640 1 -- 172.21.15.7:0/1871207194 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8944042020 con 0x7f894c1583c0 2024-04-25T11:11:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.947+0000 7f8943e00640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89280744e0 0x7f89280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.947+0000 7f8943e00640 1 -- 172.21.15.7:0/1871207194 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f89440b5ea0 con 0x7f894c1583c0 2024-04-25T11:11:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.947+0000 7f894be00640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89280744e0 0x7f89280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.948+0000 7f894be00640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89280744e0 0x7f89280769f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f893c006fd0 tx=0x7f893c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:51.952+0000 7f8943e00640 1 -- 172.21.15.7:0/1871207194 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8944049020 con 0x7f894c1583c0 2024-04-25T11:11:52.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.158+0000 7f8951600640 1 -- 172.21.15.7:0/1871207194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f894c070dd0 con 0x7f894c1583c0 2024-04-25T11:11:52.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.159+0000 7f8943e00640 1 -- 172.21.15.7:0/1871207194 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f8944080220 con 0x7f894c1583c0 2024-04-25T11:11:52.160 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:52.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.161+0000 7f8951600640 1 -- 172.21.15.7:0/1871207194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89280744e0 msgr2=0x7f89280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:52.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.161+0000 7f8951600640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89280744e0 0x7f89280769f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f893c006fd0 tx=0x7f893c008040 comp rx=0 tx=0).stop 2024-04-25T11:11:52.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.161+0000 7f8951600640 1 -- 172.21.15.7:0/1871207194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f894c1583c0 msgr2=0x7f894c15d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:52.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.161+0000 7f8951600640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f894c1583c0 0x7f894c15d1b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f894402ad00 tx=0x7f8944004a40 comp rx=0 tx=0).stop 2024-04-25T11:11:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.162+0000 7f8951600640 1 -- 172.21.15.7:0/1871207194 shutdown_connections 2024-04-25T11:11:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.162+0000 7f8951600640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89280744e0 0x7f89280769f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.162+0000 7f8951600640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f894c1583c0 0x7f894c15d1b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.162+0000 7f8951600640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f894c0cfb50 0x7f894c15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.162+0000 7f8951600640 1 --2- 172.21.15.7:0/1871207194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f894c06eea0 0x7f894c14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.162+0000 7f8951600640 1 -- 172.21.15.7:0/1871207194 >> 172.21.15.7:0/1871207194 conn(0x7f894c0740f0 msgr2=0x7f894c1614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.162+0000 7f8951600640 1 -- 172.21.15.7:0/1871207194 shutdown_connections 2024-04-25T11:11:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.162+0000 7f8951600640 1 -- 172.21.15.7:0/1871207194 wait complete. 2024-04-25T11:11:52.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:11:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.482+0000 7fb788a00640 1 Processor -- start 2024-04-25T11:11:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.482+0000 7fb788a00640 1 -- start start 2024-04-25T11:11:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.484+0000 7fb788a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb78415f180 0x7fb784161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.484+0000 7fb788a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb7841621a0 0x7fb78406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.484+0000 7fb788a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78406ded0 con 0x7fb78406dac0 2024-04-25T11:11:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.484+0000 7fb788a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb78415f590 con 0x7fb78415f180 2024-04-25T11:11:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.484+0000 7fb788a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb784158ae0 con 0x7fb7841621a0 2024-04-25T11:11:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.485+0000 7fb781e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb78415f180 0x7fb784161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.485+0000 7fb782800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb78406dac0 0x7fb78406f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51602/0 (socket says 172.21.15.7:51602) 2024-04-25T11:11:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.485+0000 7fb781e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb78415f180 0x7fb784161c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58550/0 (socket says 172.21.15.7:58550) 2024-04-25T11:11:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.485+0000 7fb782800640 1 -- 172.21.15.7:0/1126669903 learned_addr learned my addr 172.21.15.7:0/1126669903 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.485+0000 7fb781400640 1 --2- 172.21.15.7:0/1126669903 >> v2:172.21.15.87:3300/0 conn(0x7fb7841621a0 0x7fb78406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.485+0000 7fb781e00640 1 -- 172.21.15.7:0/1126669903 >> v2:172.21.15.87:3300/0 conn(0x7fb7841621a0 msgr2=0x7fb78406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.485+0000 7fb781e00640 1 --2- 172.21.15.7:0/1126669903 >> v2:172.21.15.87:3300/0 conn(0x7fb7841621a0 0x7fb78406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.485+0000 7fb781e00640 1 -- 172.21.15.7:0/1126669903 >> v1:172.21.15.7:6789/0 conn(0x7fb78406dac0 legacy=0x7fb78406f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.485+0000 7fb781e00640 1 -- 172.21.15.7:0/1126669903 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb784162700 con 0x7fb78415f180 2024-04-25T11:11:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.485+0000 7fb781400640 1 --2- 172.21.15.7:0/1126669903 >> v2:172.21.15.87:3300/0 conn(0x7fb7841621a0 0x7fb78406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.486+0000 7fb781e00640 1 --2- 172.21.15.7:0/1126669903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb78415f180 0x7fb784161c60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb784071740 tx=0x7fb77402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfea89d7fe58327c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.487+0000 7fb780a00640 1 -- 172.21.15.7:0/1126669903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774037070 con 0x7fb78415f180 2024-04-25T11:11:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.487+0000 7fb780a00640 1 -- 172.21.15.7:0/1126669903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77402ae50 con 0x7fb78415f180 2024-04-25T11:11:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.487+0000 7fb780a00640 1 -- 172.21.15.7:0/1126669903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774032660 con 0x7fb78415f180 2024-04-25T11:11:52.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.488+0000 7fb788a00640 1 -- 172.21.15.7:0/1126669903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb78415f180 msgr2=0x7fb784161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:52.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.488+0000 7fb788a00640 1 --2- 172.21.15.7:0/1126669903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb78415f180 0x7fb784161c60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb784071740 tx=0x7fb77402a140 comp rx=0 tx=0).stop 2024-04-25T11:11:52.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.488+0000 7fb788a00640 1 -- 172.21.15.7:0/1126669903 shutdown_connections 2024-04-25T11:11:52.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.488+0000 7fb788a00640 1 --2- 172.21.15.7:0/1126669903 >> v2:172.21.15.87:3300/0 conn(0x7fb7841621a0 0x7fb78406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.488+0000 7fb788a00640 1 --2- 172.21.15.7:0/1126669903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb78415f180 0x7fb784161c60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.488+0000 7fb788a00640 1 -- 172.21.15.7:0/1126669903 >> 172.21.15.7:0/1126669903 conn(0x7fb7840732b0 msgr2=0x7fb7840736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.488+0000 7fb788a00640 1 -- 172.21.15.7:0/1126669903 shutdown_connections 2024-04-25T11:11:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.488+0000 7fb788a00640 1 -- 172.21.15.7:0/1126669903 wait complete. 2024-04-25T11:11:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.490+0000 7fb788a00640 1 Processor -- start 2024-04-25T11:11:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.490+0000 7fb788a00640 1 -- start start 2024-04-25T11:11:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.492+0000 7fb788a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb78406dac0 0x7fb784145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.492+0000 7fb788a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb78415f180 0x7fb7840b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.492+0000 7fb788a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7841621a0 0x7fb78415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.492+0000 7fb788a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb78415ec60 con 0x7fb78406dac0 2024-04-25T11:11:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.492+0000 7fb788a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb78415e980 con 0x7fb7841621a0 2024-04-25T11:11:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.492+0000 7fb788a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb78415e670 con 0x7fb78415f180 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.492+0000 7fb781e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb78406dac0 0x7fb784145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.492+0000 7fb781e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb78406dac0 0x7fb784145ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58554/0 (socket says 172.21.15.7:58554) 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.492+0000 7fb781e00640 1 -- 172.21.15.7:0/3762613177 learned_addr learned my addr 172.21.15.7:0/3762613177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.493+0000 7fb781400640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb78415f180 0x7fb7840b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.493+0000 7fb781e00640 1 -- 172.21.15.7:0/3762613177 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb78415f180 msgr2=0x7fb7840b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.493+0000 7fb781e00640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb78415f180 0x7fb7840b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.493+0000 7fb781e00640 1 -- 172.21.15.7:0/3762613177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7841621a0 msgr2=0x7fb78415d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.493+0000 7fb782800640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7841621a0 0x7fb78415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.493+0000 7fb781e00640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7841621a0 0x7fb78415d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.493+0000 7fb781e00640 1 -- 172.21.15.7:0/3762613177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7840b5f80 con 0x7fb78406dac0 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.493+0000 7fb781400640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb78415f180 0x7fb7840b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:52.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.493+0000 7fb781e00640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb78406dac0 0x7fb784145ee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb774035040 tx=0x7fb7740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:52.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.494+0000 7fb77a000640 1 -- 172.21.15.7:0/3762613177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774041020 con 0x7fb78406dac0 2024-04-25T11:11:52.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.494+0000 7fb77a000640 1 -- 172.21.15.7:0/3762613177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77403c670 con 0x7fb78406dac0 2024-04-25T11:11:52.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.494+0000 7fb77a000640 1 -- 172.21.15.7:0/3762613177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774037040 con 0x7fb78406dac0 2024-04-25T11:11:52.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.494+0000 7fb788a00640 1 -- 172.21.15.7:0/3762613177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78414df20 con 0x7fb78406dac0 2024-04-25T11:11:52.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.494+0000 7fb788a00640 1 -- 172.21.15.7:0/3762613177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78414e4a0 con 0x7fb78406dac0 2024-04-25T11:11:52.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.495+0000 7fb788a00640 1 -- 172.21.15.7:0/3762613177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7840d0c10 con 0x7fb78406dac0 2024-04-25T11:11:52.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.496+0000 7fb77a000640 1 -- 172.21.15.7:0/3762613177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb774044020 con 0x7fb78406dac0 2024-04-25T11:11:52.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.498+0000 7fb77a000640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7600744e0 0x7fb7600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:52.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.498+0000 7fb77a000640 1 -- 172.21.15.7:0/3762613177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fb7740b5a10 con 0x7fb78406dac0 2024-04-25T11:11:52.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.498+0000 7fb781400640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7600744e0 0x7fb7600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:52.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.499+0000 7fb781400640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7600744e0 0x7fb7600769f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb76c006fd0 tx=0x7fb76c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:52.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.503+0000 7fb77a000640 1 -- 172.21.15.7:0/3762613177 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7840d0c10 con 0x7fb78406dac0 2024-04-25T11:11:52.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.721+0000 7fb788a00640 1 -- 172.21.15.7:0/3762613177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb784050740 con 0x7fb78406dac0 2024-04-25T11:11:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.722+0000 7fb77a000640 1 -- 172.21.15.7:0/3762613177 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fb77404b020 con 0x7fb78406dac0 2024-04-25T11:11:52.723 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:11:52.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.724+0000 7fb788a00640 1 -- 172.21.15.7:0/3762613177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7600744e0 msgr2=0x7fb7600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:52.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.724+0000 7fb788a00640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7600744e0 0x7fb7600769f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb76c006fd0 tx=0x7fb76c008040 comp rx=0 tx=0).stop 2024-04-25T11:11:52.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.724+0000 7fb788a00640 1 -- 172.21.15.7:0/3762613177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb78406dac0 msgr2=0x7fb784145ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:52.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.724+0000 7fb788a00640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb78406dac0 0x7fb784145ee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb774035040 tx=0x7fb7740046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.724+0000 7fb788a00640 1 -- 172.21.15.7:0/3762613177 shutdown_connections 2024-04-25T11:11:52.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.725+0000 7fb788a00640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7600744e0 0x7fb7600769f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.725+0000 7fb788a00640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7841621a0 0x7fb78415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.725+0000 7fb788a00640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb78415f180 0x7fb7840b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.725+0000 7fb788a00640 1 --2- 172.21.15.7:0/3762613177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb78406dac0 0x7fb784145ee0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:52.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.725+0000 7fb788a00640 1 -- 172.21.15.7:0/3762613177 >> 172.21.15.7:0/3762613177 conn(0x7fb7840732b0 msgr2=0x7fb78406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.725+0000 7fb788a00640 1 -- 172.21.15.7:0/3762613177 shutdown_connections 2024-04-25T11:11:52.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:52.725+0000 7fb788a00640 1 -- 172.21.15.7:0/3762613177 wait complete. 2024-04-25T11:11:52.752 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T11:11:52.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:52.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T11:11:52.839 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:11:52.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:52.840 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T11:11:52.887 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:11:52.888 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:52.944 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:11:52.945 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:52.997 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:11:52.998 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:11:53.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:53.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.337+0000 7f2a35600640 1 Processor -- start 2024-04-25T11:11:53.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.337+0000 7f2a35600640 1 -- start start 2024-04-25T11:11:53.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.339+0000 7f2a35600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2a300b9010 0x7f2a30161340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.340+0000 7f2a35600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a3009c000 0x7f2a300d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.340+0000 7f2a35600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a30160740 con 0x7f2a300b2ae0 2024-04-25T11:11:53.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.340+0000 7f2a35600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2a30161880 con 0x7f2a300b9010 2024-04-25T11:11:53.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.340+0000 7f2a35600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3006dc20 con 0x7f2a3009c000 2024-04-25T11:11:53.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.340+0000 7f2a34c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a3009c000 0x7f2a300d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:53.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.340+0000 7f2a34c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a3009c000 0x7f2a300d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:50124/0 (socket says 172.21.15.7:50124) 2024-04-25T11:11:53.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.340+0000 7f2a34c00640 1 -- 172.21.15.7:0/3956881800 learned_addr learned my addr 172.21.15.7:0/3956881800 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:53.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.340+0000 7f2a2fe00640 1 --2- 172.21.15.7:0/3956881800 >> v2:172.21.15.87:3300/0 conn(0x7f2a300b9010 0x7f2a30161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.341+0000 7f2a2fe00640 1 -- 172.21.15.7:0/3956881800 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a3009c000 msgr2=0x7f2a300d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.341+0000 7f2a2fe00640 1 --2- 172.21.15.7:0/3956881800 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a3009c000 0x7f2a300d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.341+0000 7f2a2fe00640 1 -- 172.21.15.7:0/3956881800 >> v1:172.21.15.87:6789/0 conn(0x7f2a300b2ae0 legacy=0x7f2a3009b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.341+0000 7f2a2fe00640 1 -- 172.21.15.7:0/3956881800 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3006df00 con 0x7f2a300b9010 2024-04-25T11:11:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.343+0000 7f2a2fe00640 1 --2- 172.21.15.7:0/3956881800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a300b9010 0x7f2a30161340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009d90 tx=0x7f2a2002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c85c997f4d5cf51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.343+0000 7f2a2ea00640 1 -- 172.21.15.7:0/3956881800 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20031600 con 0x7f2a300b9010 2024-04-25T11:11:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.343+0000 7f2a2ea00640 1 -- 172.21.15.7:0/3956881800 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a200317a0 con 0x7f2a300b9010 2024-04-25T11:11:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.343+0000 7f2a2ea00640 1 -- 172.21.15.7:0/3956881800 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20031b70 con 0x7f2a300b9010 2024-04-25T11:11:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.344+0000 7f2a35600640 1 -- 172.21.15.7:0/3956881800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a300b9010 msgr2=0x7f2a30161340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.344+0000 7f2a35600640 1 --2- 172.21.15.7:0/3956881800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a300b9010 0x7f2a30161340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009d90 tx=0x7f2a2002a5d0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.344+0000 7f2a35600640 1 -- 172.21.15.7:0/3956881800 shutdown_connections 2024-04-25T11:11:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.344+0000 7f2a35600640 1 --2- 172.21.15.7:0/3956881800 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a3009c000 0x7f2a300d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.344+0000 7f2a35600640 1 --2- 172.21.15.7:0/3956881800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a300b9010 0x7f2a30161340 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.344+0000 7f2a35600640 1 -- 172.21.15.7:0/3956881800 >> 172.21.15.7:0/3956881800 conn(0x7f2a300b5210 msgr2=0x7f2a300b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.344+0000 7f2a35600640 1 -- 172.21.15.7:0/3956881800 shutdown_connections 2024-04-25T11:11:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.344+0000 7f2a35600640 1 -- 172.21.15.7:0/3956881800 wait complete. 2024-04-25T11:11:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.346+0000 7f2a35600640 1 Processor -- start 2024-04-25T11:11:53.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.347+0000 7f2a35600640 1 -- start start 2024-04-25T11:11:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.348+0000 7f2a35600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a300b2ae0 0x7f2a30158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.348+0000 7f2a35600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a300b9010 0x7f2a3015ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.348+0000 7f2a35600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a3015ce50 0x7f2a300b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.348+0000 7f2a35600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3006e7a0 con 0x7f2a300b9010 2024-04-25T11:11:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.348+0000 7f2a35600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3006e4c0 con 0x7f2a3015ce50 2024-04-25T11:11:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.348+0000 7f2a35600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3006ea80 con 0x7f2a300b2ae0 2024-04-25T11:11:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.349+0000 7f2a2f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a300b9010 0x7f2a3015ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.349+0000 7f2a2f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a300b9010 0x7f2a3015ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58570/0 (socket says 172.21.15.7:58570) 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.349+0000 7f2a2f400640 1 -- 172.21.15.7:0/712578990 learned_addr learned my addr 172.21.15.7:0/712578990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.349+0000 7f2a2fe00640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a300b2ae0 0x7f2a30158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.349+0000 7f2a34c00640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a3015ce50 0x7f2a300b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.349+0000 7f2a2f400640 1 -- 172.21.15.7:0/712578990 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a300b2ae0 msgr2=0x7f2a30158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.349+0000 7f2a2f400640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a300b2ae0 0x7f2a30158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.349+0000 7f2a2f400640 1 -- 172.21.15.7:0/712578990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a3015ce50 msgr2=0x7f2a300b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.349+0000 7f2a2f400640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a3015ce50 0x7f2a300b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.349+0000 7f2a2f400640 1 -- 172.21.15.7:0/712578990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a30151650 con 0x7f2a300b9010 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.349+0000 7f2a34c00640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a3015ce50 0x7f2a300b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.350+0000 7f2a2f400640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a300b9010 0x7f2a3015ee00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800c440 tx=0x7f2a1800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.350+0000 7f2a27e00640 1 -- 172.21.15.7:0/712578990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a180052d0 con 0x7f2a300b9010 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.350+0000 7f2a27e00640 1 -- 172.21.15.7:0/712578990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a18005470 con 0x7f2a300b9010 2024-04-25T11:11:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.350+0000 7f2a35600640 1 -- 172.21.15.7:0/712578990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a30159140 con 0x7f2a300b9010 2024-04-25T11:11:53.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.350+0000 7f2a27e00640 1 -- 172.21.15.7:0/712578990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18007260 con 0x7f2a300b9010 2024-04-25T11:11:53.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.350+0000 7f2a35600640 1 -- 172.21.15.7:0/712578990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a30159700 con 0x7f2a300b9010 2024-04-25T11:11:53.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.351+0000 7f2a35600640 1 -- 172.21.15.7:0/712578990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a30158eb0 con 0x7f2a300b9010 2024-04-25T11:11:53.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.353+0000 7f2a27e00640 1 -- 172.21.15.7:0/712578990 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2a18007400 con 0x7f2a300b9010 2024-04-25T11:11:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.354+0000 7f2a27e00640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744e0 0x7f2a0c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.354+0000 7f2a27e00640 1 -- 172.21.15.7:0/712578990 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f2a18094a20 con 0x7f2a300b9010 2024-04-25T11:11:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.354+0000 7f2a2fe00640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744e0 0x7f2a0c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.355+0000 7f2a2fe00640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744e0 0x7f2a0c0769f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2a20034000 tx=0x7f2a20033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:53.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.361+0000 7f2a27e00640 1 -- 172.21.15.7:0/712578990 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a1805ee40 con 0x7f2a300b9010 2024-04-25T11:11:53.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.565+0000 7f2a35600640 1 -- 172.21.15.7:0/712578990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a3009c730 con 0x7f2a300b9010 2024-04-25T11:11:53.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.569+0000 7f2a27e00640 1 -- 172.21.15.7:0/712578990 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7f2a1805e740 con 0x7f2a300b9010 2024-04-25T11:11:53.570 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:53.571 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.571+0000 7f2a35600640 1 -- 172.21.15.7:0/712578990 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744e0 msgr2=0x7f2a0c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.572+0000 7f2a35600640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744e0 0x7f2a0c0769f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2a20034000 tx=0x7f2a20033040 comp rx=0 tx=0).stop 2024-04-25T11:11:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.572+0000 7f2a35600640 1 -- 172.21.15.7:0/712578990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a300b9010 msgr2=0x7f2a3015ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.572+0000 7f2a35600640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a300b9010 0x7f2a3015ee00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800c440 tx=0x7f2a1800c900 comp rx=0 tx=0).stop 2024-04-25T11:11:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.572+0000 7f2a35600640 1 -- 172.21.15.7:0/712578990 shutdown_connections 2024-04-25T11:11:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.572+0000 7f2a35600640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744e0 0x7f2a0c0769f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.572+0000 7f2a35600640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a3015ce50 0x7f2a300b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.572+0000 7f2a35600640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a300b9010 0x7f2a3015ee00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.572+0000 7f2a35600640 1 --2- 172.21.15.7:0/712578990 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a300b2ae0 0x7f2a30158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.572+0000 7f2a35600640 1 -- 172.21.15.7:0/712578990 >> 172.21.15.7:0/712578990 conn(0x7f2a300b5210 msgr2=0x7f2a301610a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.572+0000 7f2a35600640 1 -- 172.21.15.7:0/712578990 shutdown_connections 2024-04-25T11:11:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.572+0000 7f2a35600640 1 -- 172.21.15.7:0/712578990 wait complete. 2024-04-25T11:11:53.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:53.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.886+0000 7f78aac00640 1 Processor -- start 2024-04-25T11:11:53.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.887+0000 7f78aac00640 1 -- start start 2024-04-25T11:11:53.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.889+0000 7f78aac00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f78a40d0fb0 0x7f78a415fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.889+0000 7f78aac00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78a40d13c0 con 0x7f78a40b8e50 2024-04-25T11:11:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.889+0000 7f78aac00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78a4163f10 con 0x7f78a40b7af0 2024-04-25T11:11:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.889+0000 7f78aac00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f78a41641f0 con 0x7f78a40d0fb0 2024-04-25T11:11:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.889+0000 7f78a8e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f78a40b8e50 0x7f78a40b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51604/0 (socket says 172.21.15.7:51604) 2024-04-25T11:11:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.889+0000 7f78a8e00640 1 -- 172.21.15.7:0/3870075182 learned_addr learned my addr 172.21.15.7:0/3870075182 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 692586415 0 0) 0x7f78a40d13c0 con 0x7f78a40b8e50 2024-04-25T11:11:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7888003520 con 0x7f78a40b8e50 2024-04-25T11:11:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a9800640 1 --2- 172.21.15.7:0/3870075182 >> v2:172.21.15.87:3300/0 conn(0x7f78a40d0fb0 0x7f78a415fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3879205040 0 0) 0x7f7888003520 con 0x7f78a40b8e50 2024-04-25T11:11:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 >> v2:172.21.15.87:3300/0 conn(0x7f78a40d0fb0 msgr2=0x7f78a415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 --2- 172.21.15.7:0/3870075182 >> v2:172.21.15.87:3300/0 conn(0x7f78a40d0fb0 0x7f78a415fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 >> v1:172.21.15.87:6789/0 conn(0x7f78a40b7af0 legacy=0x7f78a40d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a41644d0 con 0x7f78a40b8e50 2024-04-25T11:11:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f7890001670 con 0x7f78a40b8e50 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 >> v1:172.21.15.7:6789/0 conn(0x7f78a40b8e50 legacy=0x7f78a40b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 --2- 172.21.15.7:0/3870075182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 0x7f7888003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 --2- 172.21.15.7:0/3870075182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7888004800 0x7f7888004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 --2- 172.21.15.7:0/3870075182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7888005300 0x7f788800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78a41641f0 con 0x7f78880039b0 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f788800c2a0 con 0x7f7888004800 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f788800c4a0 con 0x7f7888005300 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.890+0000 7f78a9800640 1 --2- 172.21.15.7:0/3870075182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 0x7f7888003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.891+0000 7f78a8e00640 1 --2- 172.21.15.7:0/3870075182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7888004800 0x7f7888004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.891+0000 7f78a9800640 1 -- 172.21.15.7:0/3870075182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7888005300 msgr2=0x7f788800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.891+0000 7f78a9800640 1 --2- 172.21.15.7:0/3870075182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7888005300 0x7f788800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.891+0000 7f78a9800640 1 -- 172.21.15.7:0/3870075182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7888004800 msgr2=0x7f7888004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.891+0000 7f78a9800640 1 --2- 172.21.15.7:0/3870075182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7888004800 0x7f7888004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.891+0000 7f78a9800640 1 -- 172.21.15.7:0/3870075182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7888003520 con 0x7f78880039b0 2024-04-25T11:11:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.891+0000 7f78aa200640 1 -- 172.21.15.7:0/3870075182 reap_dead start 2024-04-25T11:11:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.892+0000 7f78a9800640 1 --2- 172.21.15.7:0/3870075182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 0x7f7888003e20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f789400ba30 tx=0x7f7888005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eeba5cc45f65674 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.892+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894030070 con 0x7f78880039b0 2024-04-25T11:11:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.892+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789402ae10 con 0x7f78880039b0 2024-04-25T11:11:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.892+0000 7f78a3e00640 1 -- 172.21.15.7:0/3870075182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789400b600 con 0x7f78880039b0 2024-04-25T11:11:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.893+0000 7f78aac00640 1 -- 172.21.15.7:0/3870075182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 msgr2=0x7f7888003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.893+0000 7f78aac00640 1 --2- 172.21.15.7:0/3870075182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 0x7f7888003e20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f789400ba30 tx=0x7f7888005830 comp rx=0 tx=0).stop 2024-04-25T11:11:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.893+0000 7f78aac00640 1 -- 172.21.15.7:0/3870075182 shutdown_connections 2024-04-25T11:11:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.893+0000 7f78aac00640 1 --2- 172.21.15.7:0/3870075182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 0x7f7888003e20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.893+0000 7f78aac00640 1 -- 172.21.15.7:0/3870075182 >> 172.21.15.7:0/3870075182 conn(0x7f78a40d23e0 msgr2=0x7f78a40d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.893+0000 7f78aac00640 1 -- 172.21.15.7:0/3870075182 shutdown_connections 2024-04-25T11:11:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.893+0000 7f78aac00640 1 -- 172.21.15.7:0/3870075182 wait complete. 2024-04-25T11:11:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.896+0000 7f78aac00640 1 Processor -- start 2024-04-25T11:11:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.896+0000 7f78aac00640 1 -- start start 2024-04-25T11:11:53.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.897+0000 7f78aac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 0x7f78a40c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78aac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78a414dbc0 0x7f78a40bad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78aac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78a40bb2d0 0x7f78a414c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78aac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78a40b9260 con 0x7f78880039b0 2024-04-25T11:11:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78aac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f78a40b4bc0 con 0x7f78a40bb2d0 2024-04-25T11:11:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78aac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f78a4163a80 con 0x7f78a414dbc0 2024-04-25T11:11:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78a9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 0x7f78a40c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78a9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 0x7f78a40c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58592/0 (socket says 172.21.15.7:58592) 2024-04-25T11:11:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78a9800640 1 -- 172.21.15.7:0/1030860935 learned_addr learned my addr 172.21.15.7:0/1030860935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:53.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78a9800640 1 -- 172.21.15.7:0/1030860935 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78a414dbc0 msgr2=0x7f78a40bad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78a9800640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78a414dbc0 0x7f78a40bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78a9800640 1 -- 172.21.15.7:0/1030860935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78a40bb2d0 msgr2=0x7f78a414c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78a9800640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78a40bb2d0 0x7f78a414c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.898+0000 7f78a9800640 1 -- 172.21.15.7:0/1030860935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a414b120 con 0x7f78880039b0 2024-04-25T11:11:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.899+0000 7f78a9800640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 0x7f78a40c9470 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f789402a900 tx=0x7f7894004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.899+0000 7f78a1600640 1 -- 172.21.15.7:0/1030860935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894043030 con 0x7f78880039b0 2024-04-25T11:11:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.899+0000 7f78a1600640 1 -- 172.21.15.7:0/1030860935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7894034e50 con 0x7f78880039b0 2024-04-25T11:11:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.899+0000 7f78aac00640 1 -- 172.21.15.7:0/1030860935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a414b620 con 0x7f78880039b0 2024-04-25T11:11:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.899+0000 7f78a1600640 1 -- 172.21.15.7:0/1030860935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894004020 con 0x7f78880039b0 2024-04-25T11:11:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.899+0000 7f78aac00640 1 -- 172.21.15.7:0/1030860935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a414bbe0 con 0x7f78880039b0 2024-04-25T11:11:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.901+0000 7f78aac00640 1 -- 172.21.15.7:0/1030860935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a4050740 con 0x7f78880039b0 2024-04-25T11:11:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.902+0000 7f78a1600640 1 -- 172.21.15.7:0/1030860935 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7894046070 con 0x7f78880039b0 2024-04-25T11:11:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.903+0000 7f78a1600640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78840744e0 0x7f78840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.903+0000 7f78a1600640 1 -- 172.21.15.7:0/1030860935 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f78940b61c0 con 0x7f78880039b0 2024-04-25T11:11:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.903+0000 7f78a8e00640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78840744e0 0x7f78840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.904+0000 7f78a8e00640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78840744e0 0x7f78840769f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f78900017e0 tx=0x7f7890009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:53.908+0000 7f78a1600640 1 -- 172.21.15.7:0/1030860935 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7894080540 con 0x7f78880039b0 2024-04-25T11:11:54.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.112+0000 7f78aac00640 1 -- 172.21.15.7:0/1030860935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f78a40d3d40 con 0x7f78880039b0 2024-04-25T11:11:54.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.113+0000 7f78a1600640 1 -- 172.21.15.7:0/1030860935 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f789407fe40 con 0x7f78880039b0 2024-04-25T11:11:54.115 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:54.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.115+0000 7f78aac00640 1 -- 172.21.15.7:0/1030860935 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78840744e0 msgr2=0x7f78840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:54.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.115+0000 7f78aac00640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78840744e0 0x7f78840769f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f78900017e0 tx=0x7f7890009040 comp rx=0 tx=0).stop 2024-04-25T11:11:54.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.115+0000 7f78aac00640 1 -- 172.21.15.7:0/1030860935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 msgr2=0x7f78a40c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:54.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.115+0000 7f78aac00640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 0x7f78a40c9470 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f789402a900 tx=0x7f7894004730 comp rx=0 tx=0).stop 2024-04-25T11:11:54.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.116+0000 7f78aac00640 1 -- 172.21.15.7:0/1030860935 shutdown_connections 2024-04-25T11:11:54.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.116+0000 7f78aac00640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78840744e0 0x7f78840769f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.116+0000 7f78aac00640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78a40bb2d0 0x7f78a414c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.116+0000 7f78aac00640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78a414dbc0 0x7f78a40bad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.116+0000 7f78aac00640 1 --2- 172.21.15.7:0/1030860935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78880039b0 0x7f78a40c9470 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.116+0000 7f78aac00640 1 -- 172.21.15.7:0/1030860935 >> 172.21.15.7:0/1030860935 conn(0x7f78a40d23e0 msgr2=0x7f78a415bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:54.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.116+0000 7f78aac00640 1 -- 172.21.15.7:0/1030860935 shutdown_connections 2024-04-25T11:11:54.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.116+0000 7f78aac00640 1 -- 172.21.15.7:0/1030860935 wait complete. 2024-04-25T11:11:54.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.429+0000 7f57ad600640 1 Processor -- start 2024-04-25T11:11:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.430+0000 7f57ad600640 1 -- start start 2024-04-25T11:11:54.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.432+0000 7f57ad600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a80aef90 0x7f57a815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.432+0000 7f57ad600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57a80b6920 0x7f57a80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.432+0000 7f57ad600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a8164e00 con 0x7f57a80b6240 2024-04-25T11:11:54.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.432+0000 7f57ad600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f57a81650e0 con 0x7f57a80b6920 2024-04-25T11:11:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.432+0000 7f57ad600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57a81653c0 con 0x7f57a80aef90 2024-04-25T11:11:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.432+0000 7f57a7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57a80b6920 0x7f57a80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.432+0000 7f57a7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57a80b6920 0x7f57a80b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58596/0 (socket says 172.21.15.7:58596) 2024-04-25T11:11:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.433+0000 7f57a7400640 1 -- 172.21.15.7:0/1578176912 learned_addr learned my addr 172.21.15.7:0/1578176912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.433+0000 7f579fe00640 1 --2- 172.21.15.7:0/1578176912 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a80aef90 0x7f57a815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.433+0000 7f57a7400640 1 -- 172.21.15.7:0/1578176912 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a80aef90 msgr2=0x7f57a815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.433+0000 7f57a7400640 1 --2- 172.21.15.7:0/1578176912 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a80aef90 0x7f57a815eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.433+0000 7f57a7400640 1 -- 172.21.15.7:0/1578176912 >> v1:172.21.15.87:6789/0 conn(0x7f57a80b6240 legacy=0x7f57a80ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.433+0000 7f57a7400640 1 -- 172.21.15.7:0/1578176912 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a81656a0 con 0x7f57a80b6920 2024-04-25T11:11:54.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.434+0000 7f57a7400640 1 --2- 172.21.15.7:0/1578176912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a80b6920 0x7f57a80b4bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009e50 tx=0x7f57a002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f28fd943bcf935 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:54.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.434+0000 7f57a6000640 1 -- 172.21.15.7:0/1578176912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a0036070 con 0x7f57a80b6920 2024-04-25T11:11:54.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.435+0000 7f57a6000640 1 -- 172.21.15.7:0/1578176912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a0031600 con 0x7f57a80b6920 2024-04-25T11:11:54.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.435+0000 7f57a6000640 1 -- 172.21.15.7:0/1578176912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a00319d0 con 0x7f57a80b6920 2024-04-25T11:11:54.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.435+0000 7f57ad600640 1 -- 172.21.15.7:0/1578176912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a80b6920 msgr2=0x7f57a80b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:54.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.435+0000 7f57ad600640 1 --2- 172.21.15.7:0/1578176912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a80b6920 0x7f57a80b4bf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009e50 tx=0x7f57a002a6c0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.436+0000 7f57ad600640 1 -- 172.21.15.7:0/1578176912 shutdown_connections 2024-04-25T11:11:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.436+0000 7f57ad600640 1 --2- 172.21.15.7:0/1578176912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a80b6920 0x7f57a80b4bf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.436+0000 7f57ad600640 1 --2- 172.21.15.7:0/1578176912 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a80aef90 0x7f57a815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.436+0000 7f57ad600640 1 -- 172.21.15.7:0/1578176912 >> 172.21.15.7:0/1578176912 conn(0x7f57a80742c0 msgr2=0x7f57a806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.436+0000 7f57ad600640 1 -- 172.21.15.7:0/1578176912 shutdown_connections 2024-04-25T11:11:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.436+0000 7f57ad600640 1 -- 172.21.15.7:0/1578176912 wait complete. 2024-04-25T11:11:54.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.439+0000 7f57ad600640 1 Processor -- start 2024-04-25T11:11:54.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.439+0000 7f57ad600640 1 -- start start 2024-04-25T11:11:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.441+0000 7f57ad600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a80aef90 0x7f57a81fac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.441+0000 7f57ad600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a80b6240 0x7f57a814fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.441+0000 7f57ad600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57a80b6920 0x7f57a8155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.441+0000 7f57ad600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57a8164ba0 con 0x7f57a80aef90 2024-04-25T11:11:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.441+0000 7f57ad600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57a8164a20 con 0x7f57a80b6920 2024-04-25T11:11:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.441+0000 7f57ad600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57a80b5130 con 0x7f57a80b6240 2024-04-25T11:11:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.441+0000 7f57a6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a80aef90 0x7f57a81fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.441+0000 7f57a6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a80aef90 0x7f57a81fac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58604/0 (socket says 172.21.15.7:58604) 2024-04-25T11:11:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.441+0000 7f57a6a00640 1 -- 172.21.15.7:0/1730241834 learned_addr learned my addr 172.21.15.7:0/1730241834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.442+0000 7f579fe00640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a80b6240 0x7f57a814fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.442+0000 7f57a7400640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57a80b6920 0x7f57a8155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.442+0000 7f57a6a00640 1 -- 172.21.15.7:0/1730241834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a80b6240 msgr2=0x7f57a814fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:54.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.442+0000 7f57a6a00640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a80b6240 0x7f57a814fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.442+0000 7f57a6a00640 1 -- 172.21.15.7:0/1730241834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57a80b6920 msgr2=0x7f57a8155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:54.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.442+0000 7f57a6a00640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57a80b6920 0x7f57a8155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.442+0000 7f57a6a00640 1 -- 172.21.15.7:0/1730241834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a8150050 con 0x7f57a80aef90 2024-04-25T11:11:54.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.442+0000 7f57a6a00640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a80aef90 0x7f57a81fac30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f579800b3e0 tx=0x7f579800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:54.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.442+0000 7f579ea00640 1 -- 172.21.15.7:0/1730241834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798015020 con 0x7f57a80aef90 2024-04-25T11:11:54.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.442+0000 7f579ea00640 1 -- 172.21.15.7:0/1730241834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5798011830 con 0x7f57a80aef90 2024-04-25T11:11:54.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.442+0000 7f579ea00640 1 -- 172.21.15.7:0/1730241834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798011c20 con 0x7f57a80aef90 2024-04-25T11:11:54.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.443+0000 7f57ad600640 1 -- 172.21.15.7:0/1730241834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a8153fb0 con 0x7f57a80aef90 2024-04-25T11:11:54.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.443+0000 7f57ad600640 1 -- 172.21.15.7:0/1730241834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a8154570 con 0x7f57a80aef90 2024-04-25T11:11:54.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.444+0000 7f57ad600640 1 -- 172.21.15.7:0/1730241834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a80b5bc0 con 0x7f57a80aef90 2024-04-25T11:11:54.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.445+0000 7f579ea00640 1 -- 172.21.15.7:0/1730241834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5798018020 con 0x7f57a80aef90 2024-04-25T11:11:54.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.446+0000 7f579ea00640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57840744e0 0x7f57840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.446+0000 7f579ea00640 1 -- 172.21.15.7:0/1730241834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f5798095580 con 0x7f57a80aef90 2024-04-25T11:11:54.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.446+0000 7f579fe00640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57840744e0 0x7f57840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.447+0000 7f579fe00640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57840744e0 0x7f57840769f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57a80b18d0 tx=0x7f5790008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:54.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.451+0000 7f579ea00640 1 -- 172.21.15.7:0/1730241834 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57a80b5bc0 con 0x7f57a80aef90 2024-04-25T11:11:54.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.656+0000 7f57ad600640 1 -- 172.21.15.7:0/1730241834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f57a8050740 con 0x7f57a80aef90 2024-04-25T11:11:54.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.657+0000 7f579ea00640 1 -- 172.21.15.7:0/1730241834 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f579801f020 con 0x7f57a80aef90 2024-04-25T11:11:54.658 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.659+0000 7f57ad600640 1 -- 172.21.15.7:0/1730241834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57840744e0 msgr2=0x7f57840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.659+0000 7f57ad600640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57840744e0 0x7f57840769f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57a80b18d0 tx=0x7f5790008040 comp rx=0 tx=0).stop 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.659+0000 7f57ad600640 1 -- 172.21.15.7:0/1730241834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a80aef90 msgr2=0x7f57a81fac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.659+0000 7f57ad600640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a80aef90 0x7f57a81fac30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f579800b3e0 tx=0x7f579800b8a0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.660+0000 7f57ad600640 1 -- 172.21.15.7:0/1730241834 shutdown_connections 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.660+0000 7f57ad600640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57840744e0 0x7f57840769f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.660+0000 7f57ad600640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57a80b6920 0x7f57a8155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.660+0000 7f57ad600640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a80b6240 0x7f57a814fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.660+0000 7f57ad600640 1 --2- 172.21.15.7:0/1730241834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57a80aef90 0x7f57a81fac30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.660+0000 7f57ad600640 1 -- 172.21.15.7:0/1730241834 >> 172.21.15.7:0/1730241834 conn(0x7f57a80742c0 msgr2=0x7f57a80b54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.660+0000 7f57ad600640 1 -- 172.21.15.7:0/1730241834 shutdown_connections 2024-04-25T11:11:54.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.660+0000 7f57ad600640 1 -- 172.21.15.7:0/1730241834 wait complete. 2024-04-25T11:11:54.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:54.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.961+0000 7fb8dfe00640 1 Processor -- start 2024-04-25T11:11:54.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.961+0000 7fb8dfe00640 1 -- start start 2024-04-25T11:11:54.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.963+0000 7fb8dfe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e01602e0 0x7fb8e006e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.963+0000 7fb8dfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8e006e8b0 0x7fb8e00d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.963+0000 7fb8dfe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb8e00d4ca0 0x7fb8e00b7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.963+0000 7fb8dfe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e006ecc0 con 0x7fb8e00d4ca0 2024-04-25T11:11:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.963+0000 7fb8dfe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e00b9060 con 0x7fb8e006e8b0 2024-04-25T11:11:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.963+0000 7fb8dfe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e00d5500 con 0x7fb8e01602e0 2024-04-25T11:11:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.964+0000 7fb8df400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb8e00d4ca0 0x7fb8e00b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.964+0000 7fb8df400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb8e00d4ca0 0x7fb8e00b7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58608/0 (socket says 172.21.15.7:58608) 2024-04-25T11:11:54.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.964+0000 7fb8df400640 1 -- 172.21.15.7:0/735878646 learned_addr learned my addr 172.21.15.7:0/735878646 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.964+0000 7fb8df400640 1 -- 172.21.15.7:0/735878646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e01602e0 msgr2=0x7fb8e006e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.964+0000 7fb8df400640 1 --2- 172.21.15.7:0/735878646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e01602e0 0x7fb8e006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.964+0000 7fb8df400640 1 -- 172.21.15.7:0/735878646 >> v2:172.21.15.87:3300/0 conn(0x7fb8e006e8b0 msgr2=0x7fb8e00d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.964+0000 7fb8de000640 1 --2- 172.21.15.7:0/735878646 >> v2:172.21.15.87:3300/0 conn(0x7fb8e006e8b0 0x7fb8e00d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.964+0000 7fb8df400640 1 --2- 172.21.15.7:0/735878646 >> v2:172.21.15.87:3300/0 conn(0x7fb8e006e8b0 0x7fb8e00d4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.964+0000 7fb8df400640 1 -- 172.21.15.7:0/735878646 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e00b9300 con 0x7fb8e00d4ca0 2024-04-25T11:11:54.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.965+0000 7fb8df400640 1 --2- 172.21.15.7:0/735878646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e00d4ca0 0x7fb8e00b7490 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb8e008dfe0 tx=0x7fb8d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37182933ca1240d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:54.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.966+0000 7fb8dd600640 1 -- 172.21.15.7:0/735878646 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8037070 con 0x7fb8e00d4ca0 2024-04-25T11:11:54.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.966+0000 7fb8dd600640 1 -- 172.21.15.7:0/735878646 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d8032440 con 0x7fb8e00d4ca0 2024-04-25T11:11:54.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.966+0000 7fb8dd600640 1 -- 172.21.15.7:0/735878646 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8032810 con 0x7fb8e00d4ca0 2024-04-25T11:11:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.966+0000 7fb8dfe00640 1 -- 172.21.15.7:0/735878646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e00d4ca0 msgr2=0x7fb8e00b7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.966+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/735878646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e00d4ca0 0x7fb8e00b7490 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb8e008dfe0 tx=0x7fb8d802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.967+0000 7fb8dfe00640 1 -- 172.21.15.7:0/735878646 shutdown_connections 2024-04-25T11:11:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.967+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/735878646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e00d4ca0 0x7fb8e00b7490 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.967+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/735878646 >> v2:172.21.15.87:3300/0 conn(0x7fb8e006e8b0 0x7fb8e00d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.967+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/735878646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e01602e0 0x7fb8e006e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.967+0000 7fb8dfe00640 1 -- 172.21.15.7:0/735878646 >> 172.21.15.7:0/735878646 conn(0x7fb8e00d2670 msgr2=0x7fb8e00d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.967+0000 7fb8dfe00640 1 -- 172.21.15.7:0/735878646 shutdown_connections 2024-04-25T11:11:54.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.967+0000 7fb8dfe00640 1 -- 172.21.15.7:0/735878646 wait complete. 2024-04-25T11:11:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.969+0000 7fb8dfe00640 1 Processor -- start 2024-04-25T11:11:54.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.969+0000 7fb8dfe00640 1 -- start start 2024-04-25T11:11:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.970+0000 7fb8dfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e006e8b0 0x7fb8e0160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.970+0000 7fb8dfe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e00d4ca0 0x7fb8e00cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e01602e0 0x7fb8e01563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dfe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00d50b0 con 0x7fb8e006e8b0 2024-04-25T11:11:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dfe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00b98c0 con 0x7fb8e01602e0 2024-04-25T11:11:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dfe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e00d5230 con 0x7fb8e00d4ca0 2024-04-25T11:11:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e006e8b0 0x7fb8e0160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e006e8b0 0x7fb8e0160740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58624/0 (socket says 172.21.15.7:58624) 2024-04-25T11:11:54.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dea00640 1 -- 172.21.15.7:0/1351300200 learned_addr learned my addr 172.21.15.7:0/1351300200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:54.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8de000640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e00d4ca0 0x7fb8e00cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8df400640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e01602e0 0x7fb8e01563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dea00640 1 -- 172.21.15.7:0/1351300200 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e00d4ca0 msgr2=0x7fb8e00cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dea00640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e00d4ca0 0x7fb8e00cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dea00640 1 -- 172.21.15.7:0/1351300200 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e01602e0 msgr2=0x7fb8e01563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dea00640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e01602e0 0x7fb8e01563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8dea00640 1 -- 172.21.15.7:0/1351300200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e0161ce0 con 0x7fb8e006e8b0 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.971+0000 7fb8df400640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e01602e0 0x7fb8e01563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.972+0000 7fb8dea00640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e006e8b0 0x7fb8e0160740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000ba50 tx=0x7fb8d000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.972+0000 7fb8d7400640 1 -- 172.21.15.7:0/1351300200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0019070 con 0x7fb8e006e8b0 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.972+0000 7fb8d7400640 1 -- 172.21.15.7:0/1351300200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d000cc00 con 0x7fb8e006e8b0 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.972+0000 7fb8d7400640 1 -- 172.21.15.7:0/1351300200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0014440 con 0x7fb8e006e8b0 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.972+0000 7fb8dfe00640 1 -- 172.21.15.7:0/1351300200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e0160f00 con 0x7fb8e006e8b0 2024-04-25T11:11:54.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.972+0000 7fb8dfe00640 1 -- 172.21.15.7:0/1351300200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e01614c0 con 0x7fb8e006e8b0 2024-04-25T11:11:54.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.973+0000 7fb8dfe00640 1 -- 172.21.15.7:0/1351300200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e0050740 con 0x7fb8e006e8b0 2024-04-25T11:11:54.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.974+0000 7fb8d7400640 1 -- 172.21.15.7:0/1351300200 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb8d001c070 con 0x7fb8e006e8b0 2024-04-25T11:11:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.975+0000 7fb8d7400640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8c00744b0 0x7fb8c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.976+0000 7fb8d7400640 1 -- 172.21.15.7:0/1351300200 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fb8d00954e0 con 0x7fb8e006e8b0 2024-04-25T11:11:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.976+0000 7fb8de000640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8c00744b0 0x7fb8c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:54.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.976+0000 7fb8de000640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8c00744b0 0x7fb8c00769c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb8e01f9ae0 tx=0x7fb8cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:54.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:54.980+0000 7fb8d7400640 1 -- 172.21.15.7:0/1351300200 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8d005f7d0 con 0x7fb8e006e8b0 2024-04-25T11:11:55.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.176+0000 7fb8dfe00640 1 -- 172.21.15.7:0/1351300200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb8e0069c50 con 0x7fb8e006e8b0 2024-04-25T11:11:55.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.177+0000 7fb8d7400640 1 -- 172.21.15.7:0/1351300200 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7fb8d005f0d0 con 0x7fb8e006e8b0 2024-04-25T11:11:55.178 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:55.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.179+0000 7fb8dfe00640 1 -- 172.21.15.7:0/1351300200 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8c00744b0 msgr2=0x7fb8c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:55.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.179+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8c00744b0 0x7fb8c00769c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb8e01f9ae0 tx=0x7fb8cc008040 comp rx=0 tx=0).stop 2024-04-25T11:11:55.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.179+0000 7fb8dfe00640 1 -- 172.21.15.7:0/1351300200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e006e8b0 msgr2=0x7fb8e0160740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:55.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.179+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e006e8b0 0x7fb8e0160740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000ba50 tx=0x7fb8d000bf10 comp rx=0 tx=0).stop 2024-04-25T11:11:55.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.180+0000 7fb8dfe00640 1 -- 172.21.15.7:0/1351300200 shutdown_connections 2024-04-25T11:11:55.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.180+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8c00744b0 0x7fb8c00769c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.180+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e01602e0 0x7fb8e01563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.180+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e00d4ca0 0x7fb8e00cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.180+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/1351300200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e006e8b0 0x7fb8e0160740 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.180+0000 7fb8dfe00640 1 -- 172.21.15.7:0/1351300200 >> 172.21.15.7:0/1351300200 conn(0x7fb8e00d2670 msgr2=0x7fb8e00bb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.180+0000 7fb8dfe00640 1 -- 172.21.15.7:0/1351300200 shutdown_connections 2024-04-25T11:11:55.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.180+0000 7fb8dfe00640 1 -- 172.21.15.7:0/1351300200 wait complete. 2024-04-25T11:11:55.206 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.476+0000 7f9fa4a00640 1 Processor -- start 2024-04-25T11:11:55.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.476+0000 7f9fa4a00640 1 -- start start 2024-04-25T11:11:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.478+0000 7f9fa4a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa0145e40 0x7f9fa0146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.478+0000 7f9fa4a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9fa00b4870 0x7f9fa00b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:55.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.478+0000 7f9fa4a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fa00b51c0 0x7f9fa0143a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.478+0000 7f9fa4a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa0145b70 con 0x7f9fa00b51c0 2024-04-25T11:11:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.478+0000 7f9fa4a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9fa00b66b0 con 0x7f9fa00b4870 2024-04-25T11:11:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.478+0000 7f9fa4a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa00b5b30 con 0x7f9fa0145e40 2024-04-25T11:11:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.479+0000 7f9f9fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fa00b51c0 0x7f9fa0143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.479+0000 7f9f9fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fa00b51c0 0x7f9fa0143a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58638/0 (socket says 172.21.15.7:58638) 2024-04-25T11:11:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.479+0000 7f9f9fe00640 1 -- 172.21.15.7:0/3474739088 learned_addr learned my addr 172.21.15.7:0/3474739088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.479+0000 7f9f9f400640 1 --2- 172.21.15.7:0/3474739088 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa0145e40 0x7f9fa0146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.479+0000 7f9f9fe00640 1 -- 172.21.15.7:0/3474739088 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa0145e40 msgr2=0x7f9fa0146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.479+0000 7f9f9fe00640 1 --2- 172.21.15.7:0/3474739088 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa0145e40 0x7f9fa0146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.479+0000 7f9f9fe00640 1 -- 172.21.15.7:0/3474739088 >> v2:172.21.15.87:3300/0 conn(0x7f9fa00b4870 msgr2=0x7f9fa00b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.479+0000 7f9f9fe00640 1 --2- 172.21.15.7:0/3474739088 >> v2:172.21.15.87:3300/0 conn(0x7f9fa00b4870 0x7f9fa00b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.479+0000 7f9f9fe00640 1 -- 172.21.15.7:0/3474739088 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa00b5e10 con 0x7f9fa00b51c0 2024-04-25T11:11:55.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.480+0000 7f9f9fe00640 1 --2- 172.21.15.7:0/3474739088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa00b51c0 0x7f9fa0143a70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00b4390 tx=0x7f9f9802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b56f6fd07878be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:55.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.481+0000 7f9f9e000640 1 -- 172.21.15.7:0/3474739088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98037070 con 0x7f9fa00b51c0 2024-04-25T11:11:55.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.481+0000 7f9f9e000640 1 -- 172.21.15.7:0/3474739088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f98032440 con 0x7f9fa00b51c0 2024-04-25T11:11:55.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.481+0000 7f9f9e000640 1 -- 172.21.15.7:0/3474739088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f98032810 con 0x7f9fa00b51c0 2024-04-25T11:11:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.482+0000 7f9fa4a00640 1 -- 172.21.15.7:0/3474739088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa00b51c0 msgr2=0x7f9fa0143a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.482+0000 7f9fa4a00640 1 --2- 172.21.15.7:0/3474739088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa00b51c0 0x7f9fa0143a70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00b4390 tx=0x7f9f9802a140 comp rx=0 tx=0).stop 2024-04-25T11:11:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.482+0000 7f9fa4a00640 1 -- 172.21.15.7:0/3474739088 shutdown_connections 2024-04-25T11:11:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.482+0000 7f9fa4a00640 1 --2- 172.21.15.7:0/3474739088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa00b51c0 0x7f9fa0143a70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.482+0000 7f9fa4a00640 1 --2- 172.21.15.7:0/3474739088 >> v2:172.21.15.87:3300/0 conn(0x7f9fa00b4870 0x7f9fa00b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.482+0000 7f9fa4a00640 1 --2- 172.21.15.7:0/3474739088 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa0145e40 0x7f9fa0146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.482+0000 7f9fa4a00640 1 -- 172.21.15.7:0/3474739088 >> 172.21.15.7:0/3474739088 conn(0x7f9fa00740f0 msgr2=0x7f9fa006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.482+0000 7f9fa4a00640 1 -- 172.21.15.7:0/3474739088 shutdown_connections 2024-04-25T11:11:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.482+0000 7f9fa4a00640 1 -- 172.21.15.7:0/3474739088 wait complete. 2024-04-25T11:11:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.484+0000 7f9fa4a00640 1 Processor -- start 2024-04-25T11:11:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.484+0000 7f9fa4a00640 1 -- start start 2024-04-25T11:11:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9fa4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa00b4870 0x7f9fa013fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9fa4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fa00b51c0 0x7f9fa01e5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9fa4a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa0145e40 0x7f9fa01e7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9fa4a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa00b55d0 con 0x7f9fa00b4870 2024-04-25T11:11:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9fa4a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa00b63d0 con 0x7f9fa00b51c0 2024-04-25T11:11:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9fa4a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa00b5750 con 0x7f9fa0145e40 2024-04-25T11:11:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9f9f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa00b4870 0x7f9fa013fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9f9f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa00b4870 0x7f9fa013fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58640/0 (socket says 172.21.15.7:58640) 2024-04-25T11:11:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9f9f400640 1 -- 172.21.15.7:0/1713339951 learned_addr learned my addr 172.21.15.7:0/1713339951 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9f9f400640 1 -- 172.21.15.7:0/1713339951 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa0145e40 msgr2=0x7f9fa01e7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9f9f400640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa0145e40 0x7f9fa01e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9f9f400640 1 -- 172.21.15.7:0/1713339951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fa00b51c0 msgr2=0x7f9fa01e5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9f9f400640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fa00b51c0 0x7f9fa01e5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.486+0000 7f9f9f400640 1 -- 172.21.15.7:0/1713339951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa01e64b0 con 0x7f9fa00b4870 2024-04-25T11:11:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.487+0000 7f9f9f400640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa00b4870 0x7f9fa013fe20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f90007e70 tx=0x7f9f9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.487+0000 7f9f97400640 1 -- 172.21.15.7:0/1713339951 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90014420 con 0x7f9fa00b4870 2024-04-25T11:11:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.487+0000 7f9f97400640 1 -- 172.21.15.7:0/1713339951 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f90019070 con 0x7f9fa00b4870 2024-04-25T11:11:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.487+0000 7f9f97400640 1 -- 172.21.15.7:0/1713339951 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90014800 con 0x7f9fa00b4870 2024-04-25T11:11:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.487+0000 7f9fa4a00640 1 -- 172.21.15.7:0/1713339951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa01e6790 con 0x7f9fa00b4870 2024-04-25T11:11:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.487+0000 7f9fa4a00640 1 -- 172.21.15.7:0/1713339951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa01e6d50 con 0x7f9fa00b4870 2024-04-25T11:11:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.488+0000 7f9fa4a00640 1 -- 172.21.15.7:0/1713339951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa0050740 con 0x7f9fa00b4870 2024-04-25T11:11:55.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.489+0000 7f9f97400640 1 -- 172.21.15.7:0/1713339951 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9f900149a0 con 0x7f9fa00b4870 2024-04-25T11:11:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.490+0000 7f9f97400640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f800744b0 0x7f9f800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.491+0000 7f9f97400640 1 -- 172.21.15.7:0/1713339951 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f9f90095f30 con 0x7f9fa00b4870 2024-04-25T11:11:55.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.491+0000 7f9f9ea00640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f800744b0 0x7f9f800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:55.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.491+0000 7f9f9ea00640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f800744b0 0x7f9f800769c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9fa015ff10 tx=0x7f9f8c006bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.498+0000 7f9f97400640 1 -- 172.21.15.7:0/1713339951 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9f90060220 con 0x7f9fa00b4870 2024-04-25T11:11:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.705+0000 7f9fa4a00640 1 -- 172.21.15.7:0/1713339951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9fa00cfdd0 con 0x7f9fa00b4870 2024-04-25T11:11:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.706+0000 7f9f97400640 1 -- 172.21.15.7:0/1713339951 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7f9f9005fb20 con 0x7f9fa00b4870 2024-04-25T11:11:55.708 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:55.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 -- 172.21.15.7:0/1713339951 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f800744b0 msgr2=0x7f9f800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:55.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f800744b0 0x7f9f800769c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9fa015ff10 tx=0x7f9f8c006bf0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 -- 172.21.15.7:0/1713339951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa00b4870 msgr2=0x7f9fa013fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa00b4870 0x7f9fa013fe20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f90007e70 tx=0x7f9f9000e4b0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 -- 172.21.15.7:0/1713339951 shutdown_connections 2024-04-25T11:11:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9f800744b0 0x7f9f800769c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fa0145e40 0x7f9fa01e7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fa00b51c0 0x7f9fa01e5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 --2- 172.21.15.7:0/1713339951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fa00b4870 0x7f9fa013fe20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 -- 172.21.15.7:0/1713339951 >> 172.21.15.7:0/1713339951 conn(0x7f9fa00740f0 msgr2=0x7f9fa00cb140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 -- 172.21.15.7:0/1713339951 shutdown_connections 2024-04-25T11:11:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:55.709+0000 7f9fa4a00640 1 -- 172.21.15.7:0/1713339951 wait complete. 2024-04-25T11:11:55.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:11:56.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.014+0000 7fe819e00640 1 Processor -- start 2024-04-25T11:11:56.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.014+0000 7fe819e00640 1 -- start start 2024-04-25T11:11:56.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.016+0000 7fe819e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe81415f180 0x7fe814161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.016+0000 7fe819e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8141621a0 0x7fe81406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.016+0000 7fe819e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe81406ded0 con 0x7fe81406dac0 2024-04-25T11:11:56.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.016+0000 7fe819e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe81415f590 con 0x7fe8141621a0 2024-04-25T11:11:56.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.016+0000 7fe819e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe814158ae0 con 0x7fe81415f180 2024-04-25T11:11:56.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.017+0000 7fe812000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8141621a0 0x7fe81406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:56.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.017+0000 7fe812000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8141621a0 0x7fe81406d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58646/0 (socket says 172.21.15.7:58646) 2024-04-25T11:11:56.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.017+0000 7fe812000640 1 -- 172.21.15.7:0/1389524197 learned_addr learned my addr 172.21.15.7:0/1389524197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:56.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.017+0000 7fe812000640 1 -- 172.21.15.7:0/1389524197 >> v2:172.21.15.87:3300/0 conn(0x7fe81415f180 msgr2=0x7fe814161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:56.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.017+0000 7fe812000640 1 --2- 172.21.15.7:0/1389524197 >> v2:172.21.15.87:3300/0 conn(0x7fe81415f180 0x7fe814161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.017+0000 7fe812000640 1 -- 172.21.15.7:0/1389524197 >> v1:172.21.15.87:6789/0 conn(0x7fe81406dac0 legacy=0x7fe81406f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:56.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.017+0000 7fe812000640 1 -- 172.21.15.7:0/1389524197 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe814162700 con 0x7fe8141621a0 2024-04-25T11:11:56.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.018+0000 7fe812000640 1 --2- 172.21.15.7:0/1389524197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8141621a0 0x7fe81406d580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe814070340 tx=0x7fe80402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10b6a09e49235a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:56.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.019+0000 7fe811600640 1 -- 172.21.15.7:0/1389524197 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804036070 con 0x7fe8141621a0 2024-04-25T11:11:56.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.019+0000 7fe811600640 1 -- 172.21.15.7:0/1389524197 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe804031600 con 0x7fe8141621a0 2024-04-25T11:11:56.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.019+0000 7fe811600640 1 -- 172.21.15.7:0/1389524197 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80402ae60 con 0x7fe8141621a0 2024-04-25T11:11:56.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.019+0000 7fe819e00640 1 -- 172.21.15.7:0/1389524197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8141621a0 msgr2=0x7fe81406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:56.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.019+0000 7fe819e00640 1 --2- 172.21.15.7:0/1389524197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8141621a0 0x7fe81406d580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe814070340 tx=0x7fe80402a680 comp rx=0 tx=0).stop 2024-04-25T11:11:56.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.019+0000 7fe819e00640 1 -- 172.21.15.7:0/1389524197 shutdown_connections 2024-04-25T11:11:56.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.019+0000 7fe819e00640 1 --2- 172.21.15.7:0/1389524197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8141621a0 0x7fe81406d580 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.019+0000 7fe819e00640 1 --2- 172.21.15.7:0/1389524197 >> v2:172.21.15.87:3300/0 conn(0x7fe81415f180 0x7fe814161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.019+0000 7fe819e00640 1 -- 172.21.15.7:0/1389524197 >> 172.21.15.7:0/1389524197 conn(0x7fe8140732b0 msgr2=0x7fe8140736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:56.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.020+0000 7fe819e00640 1 -- 172.21.15.7:0/1389524197 shutdown_connections 2024-04-25T11:11:56.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.020+0000 7fe819e00640 1 -- 172.21.15.7:0/1389524197 wait complete. 2024-04-25T11:11:56.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.022+0000 7fe819e00640 1 Processor -- start 2024-04-25T11:11:56.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.022+0000 7fe819e00640 1 -- start start 2024-04-25T11:11:56.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe819e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81406dac0 0x7fe814145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe819e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81415f180 0x7fe8141420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe819e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8141621a0 0x7fe814160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe819e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe81415ec60 con 0x7fe8141621a0 2024-04-25T11:11:56.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe819e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe81415e980 con 0x7fe81415f180 2024-04-25T11:11:56.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe819e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe81415e670 con 0x7fe81406dac0 2024-04-25T11:11:56.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe813400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8141621a0 0x7fe814160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe813400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8141621a0 0x7fe814160450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58656/0 (socket says 172.21.15.7:58656) 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe813400640 1 -- 172.21.15.7:0/34202128 learned_addr learned my addr 172.21.15.7:0/34202128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe812000640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81415f180 0x7fe8141420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe813400640 1 -- 172.21.15.7:0/34202128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81406dac0 msgr2=0x7fe814145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe813400640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81406dac0 0x7fe814145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe813400640 1 -- 172.21.15.7:0/34202128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81415f180 msgr2=0x7fe8141420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe813400640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81415f180 0x7fe8141420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.024+0000 7fe813400640 1 -- 172.21.15.7:0/34202128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8141458d0 con 0x7fe8141621a0 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.025+0000 7fe813400640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8141621a0 0x7fe814160450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00b410 tx=0x7fe80c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.025+0000 7fe80b400640 1 -- 172.21.15.7:0/34202128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c015030 con 0x7fe8141621a0 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.025+0000 7fe80b400640 1 -- 172.21.15.7:0/34202128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe80c004020 con 0x7fe8141621a0 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.025+0000 7fe80b400640 1 -- 172.21.15.7:0/34202128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c004450 con 0x7fe8141621a0 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.025+0000 7fe819e00640 1 -- 172.21.15.7:0/34202128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe814142610 con 0x7fe8141621a0 2024-04-25T11:11:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.025+0000 7fe819e00640 1 -- 172.21.15.7:0/34202128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8141e91a0 con 0x7fe8141621a0 2024-04-25T11:11:56.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.026+0000 7fe819e00640 1 -- 172.21.15.7:0/34202128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe814050740 con 0x7fe8141621a0 2024-04-25T11:11:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.028+0000 7fe80b400640 1 -- 172.21.15.7:0/34202128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe80c018070 con 0x7fe8141621a0 2024-04-25T11:11:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.029+0000 7fe80b400640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7f00744b0 0x7fe7f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.029+0000 7fe80b400640 1 -- 172.21.15.7:0/34202128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fe80c0951a0 con 0x7fe8141621a0 2024-04-25T11:11:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.029+0000 7fe812a00640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7f00744b0 0x7fe7f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.030+0000 7fe812a00640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7f00744b0 0x7fe7f00769c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe814070310 tx=0x7fe7fc006bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:56.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.034+0000 7fe80b400640 1 -- 172.21.15.7:0/34202128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe814142610 con 0x7fe8141621a0 2024-04-25T11:11:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.247+0000 7fe819e00640 1 -- 172.21.15.7:0/34202128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe81409f9e0 con 0x7fe8141621a0 2024-04-25T11:11:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.248+0000 7fe80b400640 1 -- 172.21.15.7:0/34202128 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fe8141458d0 con 0x7fe8141621a0 2024-04-25T11:11:56.250 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:11:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 -- 172.21.15.7:0/34202128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7f00744b0 msgr2=0x7fe7f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7f00744b0 0x7fe7f00769c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe814070310 tx=0x7fe7fc006bf0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 -- 172.21.15.7:0/34202128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8141621a0 msgr2=0x7fe814160450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8141621a0 0x7fe814160450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00b410 tx=0x7fe80c00b8d0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 -- 172.21.15.7:0/34202128 shutdown_connections 2024-04-25T11:11:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7f00744b0 0x7fe7f00769c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8141621a0 0x7fe814160450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81415f180 0x7fe8141420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 --2- 172.21.15.7:0/34202128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81406dac0 0x7fe814145390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 -- 172.21.15.7:0/34202128 >> 172.21.15.7:0/34202128 conn(0x7fe8140732b0 msgr2=0x7fe81406d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 -- 172.21.15.7:0/34202128 shutdown_connections 2024-04-25T11:11:56.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.251+0000 7fe819e00640 1 -- 172.21.15.7:0/34202128 wait complete. 2024-04-25T11:11:56.280 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T11:11:56.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:56.281 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T11:11:56.376 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:11:56.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:11:56.377 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T11:11:56.425 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:11:56.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:11:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.715+0000 7fb0ef200640 1 Processor -- start 2024-04-25T11:11:56.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.715+0000 7fb0ef200640 1 -- start start 2024-04-25T11:11:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0ef200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0f00b5220 0x7fb0f015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0ef200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0f00af420 0x7fb0f00af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0ef200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0f00cebb0 con 0x7fb0f00aed40 2024-04-25T11:11:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0ef200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb0f015e1f0 con 0x7fb0f00b5220 2024-04-25T11:11:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0ef200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f00afd70 con 0x7fb0f00af420 2024-04-25T11:11:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0ede00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb0f00aed40 0x7fb0f00ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51618/0 (socket says 172.21.15.7:51618) 2024-04-25T11:11:56.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0ede00640 1 -- 172.21.15.7:0/4047227384 learned_addr learned my addr 172.21.15.7:0/4047227384 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:56.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0e5400640 1 --2- 172.21.15.7:0/4047227384 >> v2:172.21.15.87:3300/0 conn(0x7fb0f00b5220 0x7fb0f015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:56.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0ee800640 1 --2- 172.21.15.7:0/4047227384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0f00af420 0x7fb0f00af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:56.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0ed400640 1 -- 172.21.15.7:0/4047227384 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1291439171 0 0) 0x7fb0e0001490 con 0x7fb0f00aed40 2024-04-25T11:11:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0ed400640 1 -- 172.21.15.7:0/4047227384 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0d4003520 con 0x7fb0f00aed40 2024-04-25T11:11:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0e5400640 1 -- 172.21.15.7:0/4047227384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0f00af420 msgr2=0x7fb0f00af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0e5400640 1 --2- 172.21.15.7:0/4047227384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0f00af420 0x7fb0f00af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0e5400640 1 -- 172.21.15.7:0/4047227384 >> v1:172.21.15.7:6789/0 conn(0x7fb0f00aed40 legacy=0x7fb0f00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.718+0000 7fb0e5400640 1 -- 172.21.15.7:0/4047227384 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f00b64b0 con 0x7fb0f00b5220 2024-04-25T11:11:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.720+0000 7fb0e5400640 1 --2- 172.21.15.7:0/4047227384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00b5220 0x7fb0f015dcb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80097a0 tx=0x7fb0d802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecc66b35fa3db38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.720+0000 7fb0ed400640 1 -- 172.21.15.7:0/4047227384 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8037070 con 0x7fb0f00b5220 2024-04-25T11:11:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.720+0000 7fb0ed400640 1 -- 172.21.15.7:0/4047227384 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d8032440 con 0x7fb0f00b5220 2024-04-25T11:11:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.720+0000 7fb0ed400640 1 -- 172.21.15.7:0/4047227384 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8032810 con 0x7fb0f00b5220 2024-04-25T11:11:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.721+0000 7fb0ef200640 1 -- 172.21.15.7:0/4047227384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00b5220 msgr2=0x7fb0f015dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.721+0000 7fb0ef200640 1 --2- 172.21.15.7:0/4047227384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00b5220 0x7fb0f015dcb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80097a0 tx=0x7fb0d802a180 comp rx=0 tx=0).stop 2024-04-25T11:11:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.721+0000 7fb0ef200640 1 -- 172.21.15.7:0/4047227384 shutdown_connections 2024-04-25T11:11:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.721+0000 7fb0ef200640 1 --2- 172.21.15.7:0/4047227384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0f00af420 0x7fb0f00af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.721+0000 7fb0ef200640 1 --2- 172.21.15.7:0/4047227384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00b5220 0x7fb0f015dcb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.721+0000 7fb0ef200640 1 -- 172.21.15.7:0/4047227384 >> 172.21.15.7:0/4047227384 conn(0x7fb0f00b3820 msgr2=0x7fb0f00b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:56.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.721+0000 7fb0ef200640 1 -- 172.21.15.7:0/4047227384 shutdown_connections 2024-04-25T11:11:56.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.721+0000 7fb0ef200640 1 -- 172.21.15.7:0/4047227384 wait complete. 2024-04-25T11:11:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.725+0000 7fb0ef200640 1 Processor -- start 2024-04-25T11:11:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.726+0000 7fb0ef200640 1 -- start start 2024-04-25T11:11:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0ef200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0f00aed40 0x7fb0f009e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0ef200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0f00af420 0x7fb0f015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0ef200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00b5220 0x7fb0f0160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0ef200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f006cbd0 con 0x7fb0f00af420 2024-04-25T11:11:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0ef200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f006c8f0 con 0x7fb0f00b5220 2024-04-25T11:11:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0ef200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f006ceb0 con 0x7fb0f00aed40 2024-04-25T11:11:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0f00af420 0x7fb0f015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0f00af420 0x7fb0f015a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58670/0 (socket says 172.21.15.7:58670) 2024-04-25T11:11:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0e5400640 1 -- 172.21.15.7:0/3633802079 learned_addr learned my addr 172.21.15.7:0/3633802079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0ede00640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0f00aed40 0x7fb0f009e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0ee800640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00b5220 0x7fb0f0160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0e5400640 1 -- 172.21.15.7:0/3633802079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0f00aed40 msgr2=0x7fb0f009e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0e5400640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0f00aed40 0x7fb0f009e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0e5400640 1 -- 172.21.15.7:0/3633802079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00b5220 msgr2=0x7fb0f0160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0e5400640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00b5220 0x7fb0f0160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0e5400640 1 -- 172.21.15.7:0/3633802079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f009f150 con 0x7fb0f00af420 2024-04-25T11:11:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.728+0000 7fb0e5400640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0f00af420 0x7fb0f015a8c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb0d802a770 tx=0x7fb0d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:56.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.729+0000 7fb0e6a00640 1 -- 172.21.15.7:0/3633802079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8037050 con 0x7fb0f00af420 2024-04-25T11:11:56.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.729+0000 7fb0e6a00640 1 -- 172.21.15.7:0/3633802079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d8043070 con 0x7fb0f00af420 2024-04-25T11:11:56.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.729+0000 7fb0ef200640 1 -- 172.21.15.7:0/3633802079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f01f75d0 con 0x7fb0f00af420 2024-04-25T11:11:56.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.729+0000 7fb0e6a00640 1 -- 172.21.15.7:0/3633802079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d803cb30 con 0x7fb0f00af420 2024-04-25T11:11:56.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.729+0000 7fb0ef200640 1 -- 172.21.15.7:0/3633802079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f01f7b90 con 0x7fb0f00af420 2024-04-25T11:11:56.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.730+0000 7fb0ef200640 1 -- 172.21.15.7:0/3633802079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f009eec0 con 0x7fb0f00af420 2024-04-25T11:11:56.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.732+0000 7fb0e6a00640 1 -- 172.21.15.7:0/3633802079 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb0d803ccd0 con 0x7fb0f00af420 2024-04-25T11:11:56.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.732+0000 7fb0e6a00640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0cc0744e0 0x7fb0cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:56.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.733+0000 7fb0e6a00640 1 -- 172.21.15.7:0/3633802079 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fb0d80b6350 con 0x7fb0f00af420 2024-04-25T11:11:56.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.733+0000 7fb0ede00640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0cc0744e0 0x7fb0cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:56.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.735+0000 7fb0ede00640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0cc0744e0 0x7fb0cc0769f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005c00 tx=0x7fb0e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.738+0000 7fb0e6a00640 1 -- 172.21.15.7:0/3633802079 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb0d8080640 con 0x7fb0f00af420 2024-04-25T11:11:56.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.984+0000 7fb0ef200640 1 -- 172.21.15.7:0/3633802079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0f0050740 con 0x7fb0f00af420 2024-04-25T11:11:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.986+0000 7fb0e6a00640 1 -- 172.21.15.7:0/3633802079 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7fb0d807ff40 con 0x7fb0f00af420 2024-04-25T11:11:56.987 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:56.987 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3458223722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3458223722},{"type":"v1","addr":"172.21.15.87:6835","nonce":3458223722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1383067804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1383067804},{"type":"v1","addr":"172.21.15.7:6835","nonce":1383067804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/435824250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":435824250},{"type":"v1","addr":"172.21.15.7:6837","nonce":435824250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:10:23.023233+0000","modified":"2024-04-25T11:10:53.526222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1091275258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1091275258},{"type":"v1","addr":"172.21.15.87:6834","nonce":1091275258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:11:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.988+0000 7fb0ef200640 1 -- 172.21.15.7:0/3633802079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0cc0744e0 msgr2=0x7fb0cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.988+0000 7fb0ef200640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0cc0744e0 0x7fb0cc0769f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005c00 tx=0x7fb0e0009040 comp rx=0 tx=0).stop 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.988+0000 7fb0ef200640 1 -- 172.21.15.7:0/3633802079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0f00af420 msgr2=0x7fb0f015a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.988+0000 7fb0ef200640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0f00af420 0x7fb0f015a8c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb0d802a770 tx=0x7fb0d80046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.988+0000 7fb0ef200640 1 -- 172.21.15.7:0/3633802079 shutdown_connections 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.988+0000 7fb0ef200640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0cc0744e0 0x7fb0cc0769f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.988+0000 7fb0ef200640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00b5220 0x7fb0f0160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.988+0000 7fb0ef200640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0f00af420 0x7fb0f015a8c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.988+0000 7fb0ef200640 1 --2- 172.21.15.7:0/3633802079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0f00aed40 0x7fb0f009e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.988+0000 7fb0ef200640 1 -- 172.21.15.7:0/3633802079 >> 172.21.15.7:0/3633802079 conn(0x7fb0f00b3820 msgr2=0x7fb0f00ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.989+0000 7fb0ef200640 1 -- 172.21.15.7:0/3633802079 shutdown_connections 2024-04-25T11:11:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:56.989+0000 7fb0ef200640 1 -- 172.21.15.7:0/3633802079 wait complete. 2024-04-25T11:11:57.021 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:11:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.306+0000 7f0435c00640 1 Processor -- start 2024-04-25T11:11:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.306+0000 7f0435c00640 1 -- start start 2024-04-25T11:11:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.307+0000 7f0435c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f043009c000 0x7f0430161340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.307+0000 7f0435c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300b2ae0 0x7f04300d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.307+0000 7f0435c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0430160740 con 0x7f04300b9010 2024-04-25T11:11:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.307+0000 7f0435c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0430161880 con 0x7f043009c000 2024-04-25T11:11:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.307+0000 7f0435c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f043006dc20 con 0x7f04300b2ae0 2024-04-25T11:11:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.308+0000 7f042fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f04300b9010 0x7f043009b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51626/0 (socket says 172.21.15.7:51626) 2024-04-25T11:11:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.308+0000 7f042fe00640 1 -- 172.21.15.7:0/3525496801 learned_addr learned my addr 172.21.15.7:0/3525496801 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:57.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.308+0000 7f0427e00640 1 --2- 172.21.15.7:0/3525496801 >> v2:172.21.15.87:3300/0 conn(0x7f043009c000 0x7f0430161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.308+0000 7f0435200640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300b2ae0 0x7f04300d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.308+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1057807436 0 0) 0x7f0430160740 con 0x7f04300b9010 2024-04-25T11:11:57.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0414003520 con 0x7f04300b9010 2024-04-25T11:11:57.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1720887103 0 0) 0x7f0414003520 con 0x7f04300b9010 2024-04-25T11:11:57.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300b2ae0 msgr2=0x7f04300d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:57.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300b2ae0 0x7f04300d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 >> v2:172.21.15.87:3300/0 conn(0x7f043009c000 msgr2=0x7f0430161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:57.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 --2- 172.21.15.7:0/3525496801 >> v2:172.21.15.87:3300/0 conn(0x7f043009c000 0x7f0430161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043006df00 con 0x7f04300b9010 2024-04-25T11:11:57.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0420001670 con 0x7f04300b9010 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 >> v1:172.21.15.7:6789/0 conn(0x7f04300b9010 legacy=0x7f043009b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04140039b0 0x7f0414003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0414004800 0x7f0414004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300b2ae0 0x7f0414005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0430161880 con 0x7f0414004800 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f043006dc20 con 0x7f04140039b0 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f041400ba40 con 0x7f04300b2ae0 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.309+0000 7f0427e00640 1 --2- 172.21.15.7:0/3525496801 >> v2:172.21.15.87:3300/0 conn(0x7f043009c000 0x7f0430161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.310+0000 7f0427e00640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0414004800 0x7f0414004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.310+0000 7f0435200640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300b2ae0 0x7f0414005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.310+0000 7f042fe00640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04140039b0 0x7f0414003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.310+0000 7f0427e00640 1 -- 172.21.15.7:0/3525496801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300b2ae0 msgr2=0x7f0414005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:57.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.310+0000 7f0427e00640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300b2ae0 0x7f0414005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.310+0000 7f0427e00640 1 -- 172.21.15.7:0/3525496801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04140039b0 msgr2=0x7f0414003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:57.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.310+0000 7f0427e00640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04140039b0 0x7f0414003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.311+0000 7f0427e00640 1 -- 172.21.15.7:0/3525496801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0414003520 con 0x7f0414004800 2024-04-25T11:11:57.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.312+0000 7f0427e00640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0414004800 0x7f0414004c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0418009c40 tx=0x7f0414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e90deceb7e51850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.312+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418009840 con 0x7f0414004800 2024-04-25T11:11:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.312+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0418004020 con 0x7f0414004800 2024-04-25T11:11:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.312+0000 7f042f400640 1 -- 172.21.15.7:0/3525496801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04180043d0 con 0x7f0414004800 2024-04-25T11:11:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.313+0000 7f0435c00640 1 -- 172.21.15.7:0/3525496801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0414004800 msgr2=0x7f0414004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.313+0000 7f0435c00640 1 --2- 172.21.15.7:0/3525496801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0414004800 0x7f0414004c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0418009c40 tx=0x7f0414005830 comp rx=0 tx=0).stop 2024-04-25T11:11:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.313+0000 7f0435200640 1 -- 172.21.15.7:0/3525496801 reap_dead start 2024-04-25T11:11:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.313+0000 7f0435c00640 1 -- 172.21.15.7:0/3525496801 shutdown_connections 2024-04-25T11:11:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.313+0000 7f0435c00640 1 -- 172.21.15.7:0/3525496801 >> 172.21.15.7:0/3525496801 conn(0x7f04300b5210 msgr2=0x7f04300b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.313+0000 7f0435c00640 1 -- 172.21.15.7:0/3525496801 shutdown_connections 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.313+0000 7f0435c00640 1 -- 172.21.15.7:0/3525496801 wait complete. 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.317+0000 7f0435c00640 1 Processor -- start 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.317+0000 7f0435c00640 1 -- start start 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0414004800 0x7f04300cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300d0320 0x7f04300d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f043015afb0 0x7f043015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f043006e7a0 con 0x7f043015afb0 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f043006e4c0 con 0x7f0414004800 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f043006ea80 con 0x7f04300d0320 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f043015afb0 0x7f043015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f043015afb0 0x7f043015b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58702/0 (socket says 172.21.15.7:58702) 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435200640 1 -- 172.21.15.7:0/2040856744 learned_addr learned my addr 172.21.15.7:0/2040856744 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0427e00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300d0320 0x7f04300d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f042fe00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0414004800 0x7f04300cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435200640 1 -- 172.21.15.7:0/2040856744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300d0320 msgr2=0x7f04300d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435200640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300d0320 0x7f04300d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435200640 1 -- 172.21.15.7:0/2040856744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0414004800 msgr2=0x7f04300cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.318+0000 7f0435200640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0414004800 0x7f04300cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.319+0000 7f0435200640 1 -- 172.21.15.7:0/2040856744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04300cf7d0 con 0x7f043015afb0 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.319+0000 7f0427e00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300d0320 0x7f04300d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.319+0000 7f042fe00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0414004800 0x7f04300cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.319+0000 7f0435200640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f043015afb0 0x7f043015b3c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f04300b9420 tx=0x7f042800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.319+0000 7f042cc00640 1 -- 172.21.15.7:0/2040856744 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428015430 con 0x7f043015afb0 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.319+0000 7f042cc00640 1 -- 172.21.15.7:0/2040856744 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f042801d070 con 0x7f043015afb0 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.319+0000 7f042cc00640 1 -- 172.21.15.7:0/2040856744 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0428015830 con 0x7f043015afb0 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.319+0000 7f0435c00640 1 -- 172.21.15.7:0/2040856744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04300ce730 con 0x7f043015afb0 2024-04-25T11:11:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.319+0000 7f0435c00640 1 -- 172.21.15.7:0/2040856744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04300cecf0 con 0x7f043015afb0 2024-04-25T11:11:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.321+0000 7f0435c00640 1 -- 172.21.15.7:0/2040856744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f043009c730 con 0x7f043015afb0 2024-04-25T11:11:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.322+0000 7f042cc00640 1 -- 172.21.15.7:0/2040856744 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f04280159d0 con 0x7f043015afb0 2024-04-25T11:11:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.323+0000 7f042cc00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f040c0744b0 0x7f040c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.323+0000 7f042cc00640 1 -- 172.21.15.7:0/2040856744 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f0428095fc0 con 0x7f043015afb0 2024-04-25T11:11:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.323+0000 7f042fe00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f040c0744b0 0x7f040c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.324+0000 7f042fe00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f040c0744b0 0x7f040c0769c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0420000fd0 tx=0x7f0420001bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:57.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.329+0000 7f042cc00640 1 -- 172.21.15.7:0/2040856744 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0428010040 con 0x7f043015afb0 2024-04-25T11:11:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.536+0000 7f0435c00640 1 -- 172.21.15.7:0/2040856744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04300ce4c0 con 0x7f043015afb0 2024-04-25T11:11:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.539+0000 7f042cc00640 1 -- 172.21.15.7:0/2040856744 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7f0428060340 con 0x7f043015afb0 2024-04-25T11:11:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:11:57.540 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.542+0000 7f0435c00640 1 -- 172.21.15.7:0/2040856744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f040c0744b0 msgr2=0x7f040c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.542+0000 7f0435c00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f040c0744b0 0x7f040c0769c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0420000fd0 tx=0x7f0420001bd0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.542+0000 7f0435c00640 1 -- 172.21.15.7:0/2040856744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f043015afb0 msgr2=0x7f043015b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.542+0000 7f0435c00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f043015afb0 0x7f043015b3c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f04300b9420 tx=0x7f042800f580 comp rx=0 tx=0).stop 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.542+0000 7f0435c00640 1 -- 172.21.15.7:0/2040856744 shutdown_connections 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.542+0000 7f0435c00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f040c0744b0 0x7f040c0769c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.542+0000 7f0435c00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f043015afb0 0x7f043015b3c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.542+0000 7f0435c00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04300d0320 0x7f04300d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.542+0000 7f0435c00640 1 --2- 172.21.15.7:0/2040856744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0414004800 0x7f04300cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.542+0000 7f0435c00640 1 -- 172.21.15.7:0/2040856744 >> 172.21.15.7:0/2040856744 conn(0x7f04300b5210 msgr2=0x7f04300d2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.543+0000 7f0435c00640 1 -- 172.21.15.7:0/2040856744 shutdown_connections 2024-04-25T11:11:57.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.543+0000 7f0435c00640 1 -- 172.21.15.7:0/2040856744 wait complete. 2024-04-25T11:11:57.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:10:20.528603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:10:21.975446+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T11:10:25.509879+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T11:11:57.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:11:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.860+0000 7f284f200640 1 Processor -- start 2024-04-25T11:11:57.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.860+0000 7f284f200640 1 -- start start 2024-04-25T11:11:57.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.863+0000 7f284f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2850144a80 0x7f2850146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.863+0000 7f284f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f28500b4f80 0x7f28500b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.863+0000 7f284f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28500b66b0 con 0x7f2850145e40 2024-04-25T11:11:57.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.863+0000 7f284f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2850145b70 con 0x7f28500b4f80 2024-04-25T11:11:57.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.863+0000 7f284f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f28500b58d0 con 0x7f2850144a80 2024-04-25T11:11:57.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.863+0000 7f284e800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f28500b4f80 0x7f28500b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.863+0000 7f284de00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2850145e40 0x7f28500b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51628/0 (socket says 172.21.15.7:51628) 2024-04-25T11:11:57.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.863+0000 7f284de00640 1 -- 172.21.15.7:0/1081408342 learned_addr learned my addr 172.21.15.7:0/1081408342 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:57.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.863+0000 7f284d400640 1 --2- 172.21.15.7:0/1081408342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2850144a80 0x7f2850146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.864+0000 7f284e800640 1 -- 172.21.15.7:0/1081408342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2850144a80 msgr2=0x7f2850146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:57.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.864+0000 7f284e800640 1 --2- 172.21.15.7:0/1081408342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2850144a80 0x7f2850146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.864+0000 7f284e800640 1 -- 172.21.15.7:0/1081408342 >> v1:172.21.15.7:6789/0 conn(0x7f2850145e40 legacy=0x7f28500b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:57.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.864+0000 7f284e800640 1 -- 172.21.15.7:0/1081408342 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28500b5b30 con 0x7f28500b4f80 2024-04-25T11:11:57.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.865+0000 7f284e800640 1 --2- 172.21.15.7:0/1081408342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28500b4f80 0x7f28500b5390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f28500b6880 tx=0x7f284802a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d43db890ef5e674 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:57.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.865+0000 7f284ca00640 1 -- 172.21.15.7:0/1081408342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848007740 con 0x7f28500b4f80 2024-04-25T11:11:57.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.865+0000 7f284ca00640 1 -- 172.21.15.7:0/1081408342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28480078e0 con 0x7f28500b4f80 2024-04-25T11:11:57.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.865+0000 7f284ca00640 1 -- 172.21.15.7:0/1081408342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848007cb0 con 0x7f28500b4f80 2024-04-25T11:11:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.866+0000 7f284f200640 1 -- 172.21.15.7:0/1081408342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28500b4f80 msgr2=0x7f28500b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.866+0000 7f284f200640 1 --2- 172.21.15.7:0/1081408342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28500b4f80 0x7f28500b5390 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f28500b6880 tx=0x7f284802a5b0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.866+0000 7f284f200640 1 -- 172.21.15.7:0/1081408342 shutdown_connections 2024-04-25T11:11:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.866+0000 7f284f200640 1 --2- 172.21.15.7:0/1081408342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28500b4f80 0x7f28500b5390 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.866+0000 7f284f200640 1 --2- 172.21.15.7:0/1081408342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2850144a80 0x7f2850146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.866+0000 7f284f200640 1 -- 172.21.15.7:0/1081408342 >> 172.21.15.7:0/1081408342 conn(0x7f28500740f0 msgr2=0x7f285006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.866+0000 7f284f200640 1 -- 172.21.15.7:0/1081408342 shutdown_connections 2024-04-25T11:11:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.866+0000 7f284f200640 1 -- 172.21.15.7:0/1081408342 wait complete. 2024-04-25T11:11:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.868+0000 7f284f200640 1 Processor -- start 2024-04-25T11:11:57.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.869+0000 7f284f200640 1 -- start start 2024-04-25T11:11:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.870+0000 7f284f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28500b4f80 0x7f285013aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.870+0000 7f284f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2850144a80 0x7f2850166b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.870+0000 7f284f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2850145e40 0x7f28501468f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.870+0000 7f284f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f28500b63d0 con 0x7f2850145e40 2024-04-25T11:11:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.870+0000 7f284f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f28500b60f0 con 0x7f28500b4f80 2024-04-25T11:11:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.870+0000 7f284f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f285006f730 con 0x7f2850144a80 2024-04-25T11:11:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.870+0000 7f284e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2850145e40 0x7f28501468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.870+0000 7f284e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2850145e40 0x7f28501468f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58712/0 (socket says 172.21.15.7:58712) 2024-04-25T11:11:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.870+0000 7f284e800640 1 -- 172.21.15.7:0/2989446337 learned_addr learned my addr 172.21.15.7:0/2989446337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.871+0000 7f284de00640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28500b4f80 0x7f285013aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.871+0000 7f284d400640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2850144a80 0x7f2850166b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.871+0000 7f284e800640 1 -- 172.21.15.7:0/2989446337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2850144a80 msgr2=0x7f2850166b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.871+0000 7f284e800640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2850144a80 0x7f2850166b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.871+0000 7f284e800640 1 -- 172.21.15.7:0/2989446337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28500b4f80 msgr2=0x7f285013aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.871+0000 7f284e800640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28500b4f80 0x7f285013aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.871+0000 7f284e800640 1 -- 172.21.15.7:0/2989446337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2850168c80 con 0x7f2850145e40 2024-04-25T11:11:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.871+0000 7f284d400640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2850144a80 0x7f2850166b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.871+0000 7f284e800640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2850145e40 0x7f28501468f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2848009e50 tx=0x7f2848004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:57.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.872+0000 7f2846000640 1 -- 172.21.15.7:0/2989446337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284803f070 con 0x7f2850145e40 2024-04-25T11:11:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.872+0000 7f2846000640 1 -- 172.21.15.7:0/2989446337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2848031bf0 con 0x7f2850145e40 2024-04-25T11:11:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.872+0000 7f2846000640 1 -- 172.21.15.7:0/2989446337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848044460 con 0x7f2850145e40 2024-04-25T11:11:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.872+0000 7f284f200640 1 -- 172.21.15.7:0/2989446337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2850167750 con 0x7f2850145e40 2024-04-25T11:11:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.872+0000 7f284f200640 1 -- 172.21.15.7:0/2989446337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2850167d10 con 0x7f2850145e40 2024-04-25T11:11:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.873+0000 7f284f200640 1 -- 172.21.15.7:0/2989446337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2850050740 con 0x7f2850145e40 2024-04-25T11:11:57.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.874+0000 7f2846000640 1 -- 172.21.15.7:0/2989446337 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2848007790 con 0x7f2850145e40 2024-04-25T11:11:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.875+0000 7f2846000640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28300744b0 0x7f28300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:57.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.875+0000 7f2846000640 1 -- 172.21.15.7:0/2989446337 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f28480b6ec0 con 0x7f2850145e40 2024-04-25T11:11:57.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.876+0000 7f284de00640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28300744b0 0x7f28300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:57.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.877+0000 7f284de00640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28300744b0 0x7f28300769c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2840000f60 tx=0x7f2840009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:57.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:57.880+0000 7f2846000640 1 -- 172.21.15.7:0/2989446337 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2850168c80 con 0x7f2850145e40 2024-04-25T11:11:58.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.088+0000 7f284f200640 1 -- 172.21.15.7:0/2989446337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f285006d390 con 0x7f2850145e40 2024-04-25T11:11:58.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.090+0000 7f2846000640 1 -- 172.21.15.7:0/2989446337 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7f2848089230 con 0x7f2850145e40 2024-04-25T11:11:58.090 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:11:58.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 -- 172.21.15.7:0/2989446337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28300744b0 msgr2=0x7f28300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28300744b0 0x7f28300769c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2840000f60 tx=0x7f2840009040 comp rx=0 tx=0).stop 2024-04-25T11:11:58.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 -- 172.21.15.7:0/2989446337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2850145e40 msgr2=0x7f28501468f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2850145e40 0x7f28501468f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2848009e50 tx=0x7f2848004a40 comp rx=0 tx=0).stop 2024-04-25T11:11:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 -- 172.21.15.7:0/2989446337 shutdown_connections 2024-04-25T11:11:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28300744b0 0x7f28300769c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2850145e40 0x7f28501468f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2850144a80 0x7f2850166b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 --2- 172.21.15.7:0/2989446337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28500b4f80 0x7f285013aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 -- 172.21.15.7:0/2989446337 >> 172.21.15.7:0/2989446337 conn(0x7f28500740f0 msgr2=0x7f285006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 -- 172.21.15.7:0/2989446337 shutdown_connections 2024-04-25T11:11:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.093+0000 7f284f200640 1 -- 172.21.15.7:0/2989446337 wait complete. 2024-04-25T11:11:58.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:11:58.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.406+0000 7f62df000640 1 Processor -- start 2024-04-25T11:11:58.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.406+0000 7f62df000640 1 -- start start 2024-04-25T11:11:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.408+0000 7f62df000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d80b3eb0 0x7f62d806ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.408+0000 7f62df000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d80d1830 con 0x7f62d80b7c00 2024-04-25T11:11:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.408+0000 7f62df000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d80d1ad0 con 0x7f62d806c570 2024-04-25T11:11:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.408+0000 7f62df000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f62d80d1db0 con 0x7f62d80b3eb0 2024-04-25T11:11:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.408+0000 7f62de600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f62d80b7c00 0x7f62d80d2b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51630/0 (socket says 172.21.15.7:51630) 2024-04-25T11:11:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.408+0000 7f62de600640 1 -- 172.21.15.7:0/963780056 learned_addr learned my addr 172.21.15.7:0/963780056 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:11:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.408+0000 7f62ddc00640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d80b3eb0 0x7f62d806ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2494879876 0 0) 0x7f62d80d1830 con 0x7f62d80b7c00 2024-04-25T11:11:58.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62c0003520 con 0x7f62d80b7c00 2024-04-25T11:11:58.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2812802167 0 0) 0x7f62c0003520 con 0x7f62d80b7c00 2024-04-25T11:11:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d80b3eb0 msgr2=0x7f62d806ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d80b3eb0 0x7f62d806ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 >> v1:172.21.15.87:6789/0 conn(0x7f62d806c570 legacy=0x7f62d80b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d80d2090 con 0x7f62d80b7c00 2024-04-25T11:11:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f62d00019f0 con 0x7f62d80b7c00 2024-04-25T11:11:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 >> v1:172.21.15.7:6789/0 conn(0x7f62d80b7c00 legacy=0x7f62d80d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 0x7f62c0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d80b3eb0 0x7f62c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62c0004d40 0x7f62c0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62d80d1db0 con 0x7f62c00039e0 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f62c000baf0 con 0x7f62c0004d40 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.409+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f62c000bcf0 con 0x7f62d80b3eb0 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.410+0000 7f62ddc00640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 0x7f62c0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.410+0000 7f62de600640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62c0004d40 0x7f62c0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.410+0000 7f62ddc00640 1 -- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d80b3eb0 msgr2=0x7f62c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.410+0000 7f62dd200640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d80b3eb0 0x7f62c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.410+0000 7f62ddc00640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d80b3eb0 0x7f62c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.410+0000 7f62ddc00640 1 -- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62c0004d40 msgr2=0x7f62c0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.410+0000 7f62ddc00640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62c0004d40 0x7f62c0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.410+0000 7f62ddc00640 1 -- 172.21.15.7:0/963780056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c0003520 con 0x7f62c00039e0 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.410+0000 7f62de600640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62c0004d40 0x7f62c0005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.410+0000 7f62dd200640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d80b3eb0 0x7f62c0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:58.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.411+0000 7f62ddc00640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 0x7f62c0003e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009a90 tx=0x7f62c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd1914da7f103dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.412+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc036070 con 0x7f62c00039e0 2024-04-25T11:11:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.412+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62cc031400 con 0x7f62c00039e0 2024-04-25T11:11:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.412+0000 7f62d7e00640 1 -- 172.21.15.7:0/963780056 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc009690 con 0x7f62c00039e0 2024-04-25T11:11:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.412+0000 7f62df000640 1 -- 172.21.15.7:0/963780056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 msgr2=0x7f62c0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.412+0000 7f62df000640 1 --2- 172.21.15.7:0/963780056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 0x7f62c0003e50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009a90 tx=0x7f62c0005830 comp rx=0 tx=0).stop 2024-04-25T11:11:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.412+0000 7f62de600640 1 -- 172.21.15.7:0/963780056 reap_dead start 2024-04-25T11:11:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.413+0000 7f62df000640 1 -- 172.21.15.7:0/963780056 shutdown_connections 2024-04-25T11:11:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.413+0000 7f62df000640 1 -- 172.21.15.7:0/963780056 >> 172.21.15.7:0/963780056 conn(0x7f62d809b540 msgr2=0x7f62d809b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.413+0000 7f62df000640 1 -- 172.21.15.7:0/963780056 shutdown_connections 2024-04-25T11:11:58.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.413+0000 7f62df000640 1 -- 172.21.15.7:0/963780056 wait complete. 2024-04-25T11:11:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.415+0000 7f62df000640 1 Processor -- start 2024-04-25T11:11:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.415+0000 7f62df000640 1 -- start start 2024-04-25T11:11:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.417+0000 7f62df000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 0x7f62d8161070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.417+0000 7f62df000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d8156410 0x7f62d814cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.417+0000 7f62df000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62d8162b70 0x7f62d8162f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.417+0000 7f62df000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62d80cf900 con 0x7f62c00039e0 2024-04-25T11:11:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.417+0000 7f62df000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f62d80d2650 con 0x7f62d8162b70 2024-04-25T11:11:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.417+0000 7f62df000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f62d80cfa80 con 0x7f62d8156410 2024-04-25T11:11:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.417+0000 7f62ddc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 0x7f62d8161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.417+0000 7f62ddc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 0x7f62d8161070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58732/0 (socket says 172.21.15.7:58732) 2024-04-25T11:11:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.417+0000 7f62ddc00640 1 -- 172.21.15.7:0/344784204 learned_addr learned my addr 172.21.15.7:0/344784204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62de600640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62d8162b70 0x7f62d8162f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.417+0000 7f62dd200640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d8156410 0x7f62d814cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62ddc00640 1 -- 172.21.15.7:0/344784204 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d8156410 msgr2=0x7f62d814cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62ddc00640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d8156410 0x7f62d814cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62ddc00640 1 -- 172.21.15.7:0/344784204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62d8162b70 msgr2=0x7f62d8162f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62ddc00640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62d8162b70 0x7f62d8162f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62ddc00640 1 -- 172.21.15.7:0/344784204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d81634c0 con 0x7f62c00039e0 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62de600640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62d8162b70 0x7f62d8162f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62ddc00640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 0x7f62d8161070 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f62cc000c00 tx=0x7f62cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62d5600640 1 -- 172.21.15.7:0/344784204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc040020 con 0x7f62c00039e0 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62d5600640 1 -- 172.21.15.7:0/344784204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62cc03b680 con 0x7f62c00039e0 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62d5600640 1 -- 172.21.15.7:0/344784204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc036040 con 0x7f62c00039e0 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.418+0000 7f62df000640 1 -- 172.21.15.7:0/344784204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d81560c0 con 0x7f62c00039e0 2024-04-25T11:11:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.419+0000 7f62df000640 1 -- 172.21.15.7:0/344784204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d815adb0 con 0x7f62c00039e0 2024-04-25T11:11:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.420+0000 7f62df000640 1 -- 172.21.15.7:0/344784204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d8050740 con 0x7f62c00039e0 2024-04-25T11:11:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.421+0000 7f62d5600640 1 -- 172.21.15.7:0/344784204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f62cc043020 con 0x7f62c00039e0 2024-04-25T11:11:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.422+0000 7f62d5600640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62b80744b0 0x7f62b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.422+0000 7f62d5600640 1 -- 172.21.15.7:0/344784204 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f62cc0b5c30 con 0x7f62c00039e0 2024-04-25T11:11:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.422+0000 7f62dd200640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62b80744b0 0x7f62b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.423+0000 7f62dd200640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62b80744b0 0x7f62b80769c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f62c4007fd0 tx=0x7f62c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.427+0000 7f62d5600640 1 -- 172.21.15.7:0/344784204 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f62cc04a070 con 0x7f62c00039e0 2024-04-25T11:11:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.631+0000 7f62df000640 1 -- 172.21.15.7:0/344784204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f62d8156850 con 0x7f62c00039e0 2024-04-25T11:11:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.632+0000 7f62d5600640 1 -- 172.21.15.7:0/344784204 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f62d81634c0 con 0x7f62c00039e0 2024-04-25T11:11:58.633 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:58.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.635+0000 7f62df000640 1 -- 172.21.15.7:0/344784204 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62b80744b0 msgr2=0x7f62b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.635+0000 7f62df000640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62b80744b0 0x7f62b80769c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f62c4007fd0 tx=0x7f62c4009040 comp rx=0 tx=0).stop 2024-04-25T11:11:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.635+0000 7f62df000640 1 -- 172.21.15.7:0/344784204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 msgr2=0x7f62d8161070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.635+0000 7f62df000640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 0x7f62d8161070 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f62cc000c00 tx=0x7f62cc004730 comp rx=0 tx=0).stop 2024-04-25T11:11:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.635+0000 7f62df000640 1 -- 172.21.15.7:0/344784204 shutdown_connections 2024-04-25T11:11:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.635+0000 7f62df000640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62b80744b0 0x7f62b80769c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.635+0000 7f62df000640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62d8162b70 0x7f62d8162f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.635+0000 7f62df000640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62d8156410 0x7f62d814cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.635+0000 7f62df000640 1 --2- 172.21.15.7:0/344784204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62c00039e0 0x7f62d8161070 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.635+0000 7f62df000640 1 -- 172.21.15.7:0/344784204 >> 172.21.15.7:0/344784204 conn(0x7f62d809b540 msgr2=0x7f62d80b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.636+0000 7f62df000640 1 -- 172.21.15.7:0/344784204 shutdown_connections 2024-04-25T11:11:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.636+0000 7f62df000640 1 -- 172.21.15.7:0/344784204 wait complete. 2024-04-25T11:11:58.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:11:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.954+0000 7f6d11400640 1 Processor -- start 2024-04-25T11:11:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.954+0000 7f6d11400640 1 -- start start 2024-04-25T11:11:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.957+0000 7f6d11400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6d0c15dfb0 0x7f6d0c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.957+0000 7f6d11400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d0c0d0b40 0x7f6d0c06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.957+0000 7f6d11400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6d0c0d0f50 0x7f6d0c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.957+0000 7f6d11400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6d0c0b09b0 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.957+0000 7f6d11400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6d0c15e3c0 con 0x7f6d0c15dfb0 2024-04-25T11:11:58.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.957+0000 7f6d11400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0c0cf800 con 0x7f6d0c0d0b40 2024-04-25T11:11:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.957+0000 7f6d0b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6d0c0d0f50 0x7f6d0c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.957+0000 7f6d0b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6d0c0d0f50 0x7f6d0c06f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58734/0 (socket says 172.21.15.7:58734) 2024-04-25T11:11:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.958+0000 7f6d0b200640 1 -- 172.21.15.7:0/540490959 learned_addr learned my addr 172.21.15.7:0/540490959 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.958+0000 7f6d0a800640 1 --2- 172.21.15.7:0/540490959 >> v2:172.21.15.87:3300/0 conn(0x7f6d0c15dfb0 0x7f6d0c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.958+0000 7f6d03e00640 1 --2- 172.21.15.7:0/540490959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d0c0d0b40 0x7f6d0c06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.958+0000 7f6d0b200640 1 -- 172.21.15.7:0/540490959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d0c0d0b40 msgr2=0x7f6d0c06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.958+0000 7f6d0b200640 1 --2- 172.21.15.7:0/540490959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d0c0d0b40 0x7f6d0c06f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.958+0000 7f6d0b200640 1 -- 172.21.15.7:0/540490959 >> v2:172.21.15.87:3300/0 conn(0x7f6d0c15dfb0 msgr2=0x7f6d0c0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.958+0000 7f6d0b200640 1 --2- 172.21.15.7:0/540490959 >> v2:172.21.15.87:3300/0 conn(0x7f6d0c15dfb0 0x7f6d0c0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.958+0000 7f6d0b200640 1 -- 172.21.15.7:0/540490959 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c0cfaa0 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.960+0000 7f6d0b200640 1 --2- 172.21.15.7:0/540490959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0c0d0f50 0x7f6d0c06f8e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c072b10 tx=0x7f6d0402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62f0f3e5e6a7db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.960+0000 7f6d09e00640 1 -- 172.21.15.7:0/540490959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04036070 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.960+0000 7f6d09e00640 1 -- 172.21.15.7:0/540490959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d04031600 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.960+0000 7f6d09e00640 1 -- 172.21.15.7:0/540490959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d040319d0 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.961+0000 7f6d11400640 1 -- 172.21.15.7:0/540490959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0c0d0f50 msgr2=0x7f6d0c06f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.961+0000 7f6d11400640 1 --2- 172.21.15.7:0/540490959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0c0d0f50 0x7f6d0c06f8e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c072b10 tx=0x7f6d0402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.961+0000 7f6d11400640 1 -- 172.21.15.7:0/540490959 shutdown_connections 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.961+0000 7f6d11400640 1 --2- 172.21.15.7:0/540490959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0c0d0f50 0x7f6d0c06f8e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.961+0000 7f6d11400640 1 --2- 172.21.15.7:0/540490959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d0c0d0b40 0x7f6d0c06f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.961+0000 7f6d11400640 1 --2- 172.21.15.7:0/540490959 >> v2:172.21.15.87:3300/0 conn(0x7f6d0c15dfb0 0x7f6d0c0b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.961+0000 7f6d11400640 1 -- 172.21.15.7:0/540490959 >> 172.21.15.7:0/540490959 conn(0x7f6d0c0742c0 msgr2=0x7f6d0c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.962+0000 7f6d11400640 1 -- 172.21.15.7:0/540490959 shutdown_connections 2024-04-25T11:11:58.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.962+0000 7f6d11400640 1 -- 172.21.15.7:0/540490959 wait complete. 2024-04-25T11:11:58.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.964+0000 7f6d11400640 1 Processor -- start 2024-04-25T11:11:58.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.964+0000 7f6d11400640 1 -- start start 2024-04-25T11:11:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.966+0000 7f6d11400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d0c0d0b40 0x7f6d0c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.966+0000 7f6d11400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0c0d0f50 0x7f6d0c151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.966+0000 7f6d11400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d0c15dfb0 0x7f6d0c1f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.966+0000 7f6d11400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c0d0340 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.966+0000 7f6d11400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c0d0060 con 0x7f6d0c15dfb0 2024-04-25T11:11:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.966+0000 7f6d11400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0c0d0620 con 0x7f6d0c0d0b40 2024-04-25T11:11:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.966+0000 7f6d03e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0c0d0f50 0x7f6d0c151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.966+0000 7f6d03e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0c0d0f50 0x7f6d0c151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58748/0 (socket says 172.21.15.7:58748) 2024-04-25T11:11:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.966+0000 7f6d03e00640 1 -- 172.21.15.7:0/994482867 learned_addr learned my addr 172.21.15.7:0/994482867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.966+0000 7f6d0b200640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d0c15dfb0 0x7f6d0c1f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.967+0000 7f6d0a800640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d0c0d0b40 0x7f6d0c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.967+0000 7f6d03e00640 1 -- 172.21.15.7:0/994482867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d0c0d0b40 msgr2=0x7f6d0c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.967+0000 7f6d03e00640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d0c0d0b40 0x7f6d0c0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.967+0000 7f6d03e00640 1 -- 172.21.15.7:0/994482867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d0c15dfb0 msgr2=0x7f6d0c1f9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.967+0000 7f6d03e00640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d0c15dfb0 0x7f6d0c1f9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.967+0000 7f6d03e00640 1 -- 172.21.15.7:0/994482867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c15c870 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.967+0000 7f6d03e00640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0c0d0f50 0x7f6d0c151ff0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400e8a0 tx=0x7f6cf400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:58.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.967+0000 7f6d02a00640 1 -- 172.21.15.7:0/994482867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4017070 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.967+0000 7f6d02a00640 1 -- 172.21.15.7:0/994482867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf400ee90 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.967+0000 7f6d02a00640 1 -- 172.21.15.7:0/994482867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf400cb80 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.968+0000 7f6d11400640 1 -- 172.21.15.7:0/994482867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0c15cb50 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.968+0000 7f6d11400640 1 -- 172.21.15.7:0/994482867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0c15d110 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.969+0000 7f6d11400640 1 -- 172.21.15.7:0/994482867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d0c15c5e0 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.970+0000 7f6d02a00640 1 -- 172.21.15.7:0/994482867 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6cf401a070 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.971+0000 7f6d02a00640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ce80744b0 0x7f6ce80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:58.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.971+0000 7f6d02a00640 1 -- 172.21.15.7:0/994482867 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f6cf4095920 con 0x7f6d0c0d0f50 2024-04-25T11:11:58.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.972+0000 7f6d0a800640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ce80744b0 0x7f6ce80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:58.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.972+0000 7f6d0a800640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ce80744b0 0x7f6ce80769c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c072df0 tx=0x7f6cfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:58.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:58.977+0000 7f6d02a00640 1 -- 172.21.15.7:0/994482867 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cf405fc10 con 0x7f6d0c0d0f50 2024-04-25T11:11:59.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.185+0000 7f6d11400640 1 -- 172.21.15.7:0/994482867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6d0c0b1320 con 0x7f6d0c0d0f50 2024-04-25T11:11:59.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.186+0000 7f6d02a00640 1 -- 172.21.15.7:0/994482867 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f6d0c0b1320 con 0x7f6d0c0d0f50 2024-04-25T11:11:59.186 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:11:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 -- 172.21.15.7:0/994482867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ce80744b0 msgr2=0x7f6ce80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:59.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ce80744b0 0x7f6ce80769c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c072df0 tx=0x7f6cfc008040 comp rx=0 tx=0).stop 2024-04-25T11:11:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 -- 172.21.15.7:0/994482867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0c0d0f50 msgr2=0x7f6d0c151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0c0d0f50 0x7f6d0c151ff0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400e8a0 tx=0x7f6cf400ed60 comp rx=0 tx=0).stop 2024-04-25T11:11:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 -- 172.21.15.7:0/994482867 shutdown_connections 2024-04-25T11:11:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ce80744b0 0x7f6ce80769c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d0c15dfb0 0x7f6d0c1f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0c0d0f50 0x7f6d0c151ff0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 --2- 172.21.15.7:0/994482867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d0c0d0b40 0x7f6d0c0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 -- 172.21.15.7:0/994482867 >> 172.21.15.7:0/994482867 conn(0x7f6d0c0742c0 msgr2=0x7f6d0c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 -- 172.21.15.7:0/994482867 shutdown_connections 2024-04-25T11:11:59.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.189+0000 7f6d11400640 1 -- 172.21.15.7:0/994482867 wait complete. 2024-04-25T11:11:59.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T11:11:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.496+0000 7ff4b1a00640 1 Processor -- start 2024-04-25T11:11:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.496+0000 7ff4b1a00640 1 -- start start 2024-04-25T11:11:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.498+0000 7ff4b1a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4ac0ce7b0 0x7ff4ac0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.498+0000 7ff4b1a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4ac0b73b0 0x7ff4ac0b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.498+0000 7ff4b1a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ac0b7d00 con 0x7ff4ac160a20 2024-04-25T11:11:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.498+0000 7ff4b1a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff4ac0d3230 con 0x7ff4ac0b73b0 2024-04-25T11:11:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.498+0000 7ff4b1a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac06e720 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.498+0000 7ff4ab400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4ac0b73b0 0x7ff4ac0b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.498+0000 7ff4ab400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4ac0b73b0 0x7ff4ac0b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58752/0 (socket says 172.21.15.7:58752) 2024-04-25T11:11:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.498+0000 7ff4ab400640 1 -- 172.21.15.7:0/2619716875 learned_addr learned my addr 172.21.15.7:0/2619716875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.498+0000 7ff4abe00640 1 --2- 172.21.15.7:0/2619716875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4ac0ce7b0 0x7ff4ac0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.499+0000 7ff4ab400640 1 -- 172.21.15.7:0/2619716875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4ac0ce7b0 msgr2=0x7ff4ac0b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.499+0000 7ff4ab400640 1 --2- 172.21.15.7:0/2619716875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4ac0ce7b0 0x7ff4ac0b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.499+0000 7ff4ab400640 1 -- 172.21.15.7:0/2619716875 >> v1:172.21.15.87:6789/0 conn(0x7ff4ac160a20 legacy=0x7ff4ac160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:11:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.499+0000 7ff4ab400640 1 -- 172.21.15.7:0/2619716875 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac0d2620 con 0x7ff4ac0b73b0 2024-04-25T11:11:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.499+0000 7ff4abe00640 1 --2- 172.21.15.7:0/2619716875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4ac0ce7b0 0x7ff4ac0b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.500+0000 7ff4ab400640 1 --2- 172.21.15.7:0/2619716875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4ac0b73b0 0x7ff4ac0b77c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff498009870 tx=0x7ff49802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f372390bd4e6e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.500+0000 7ff4aaa00640 1 -- 172.21.15.7:0/2619716875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498037070 con 0x7ff4ac0b73b0 2024-04-25T11:11:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.500+0000 7ff4aaa00640 1 -- 172.21.15.7:0/2619716875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff498032440 con 0x7ff4ac0b73b0 2024-04-25T11:11:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.500+0000 7ff4aaa00640 1 -- 172.21.15.7:0/2619716875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498032810 con 0x7ff4ac0b73b0 2024-04-25T11:11:59.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.501+0000 7ff4b1a00640 1 -- 172.21.15.7:0/2619716875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4ac0b73b0 msgr2=0x7ff4ac0b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:59.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.501+0000 7ff4b1a00640 1 --2- 172.21.15.7:0/2619716875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4ac0b73b0 0x7ff4ac0b77c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff498009870 tx=0x7ff49802a200 comp rx=0 tx=0).stop 2024-04-25T11:11:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.501+0000 7ff4b1a00640 1 -- 172.21.15.7:0/2619716875 shutdown_connections 2024-04-25T11:11:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.501+0000 7ff4b1a00640 1 --2- 172.21.15.7:0/2619716875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4ac0b73b0 0x7ff4ac0b77c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.501+0000 7ff4b1a00640 1 --2- 172.21.15.7:0/2619716875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4ac0ce7b0 0x7ff4ac0b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.501+0000 7ff4b1a00640 1 -- 172.21.15.7:0/2619716875 >> 172.21.15.7:0/2619716875 conn(0x7ff4ac09b540 msgr2=0x7ff4ac09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.501+0000 7ff4b1a00640 1 -- 172.21.15.7:0/2619716875 shutdown_connections 2024-04-25T11:11:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.501+0000 7ff4b1a00640 1 -- 172.21.15.7:0/2619716875 wait complete. 2024-04-25T11:11:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.504+0000 7ff4b1a00640 1 Processor -- start 2024-04-25T11:11:59.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.504+0000 7ff4b1a00640 1 -- start start 2024-04-25T11:11:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.505+0000 7ff4b1a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4ac0b73b0 0x7ff4ac1f9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.505+0000 7ff4b1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4ac0ce7b0 0x7ff4ac152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.505+0000 7ff4b1a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4ac160a20 0x7ff4ac1f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4b1a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac0d2ec0 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4b1a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac0d2be0 con 0x7ff4ac160a20 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4b1a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac0ccca0 con 0x7ff4ac0b73b0 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4ab400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4ac0ce7b0 0x7ff4ac152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4ab400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4ac0ce7b0 0x7ff4ac152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58768/0 (socket says 172.21.15.7:58768) 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4ab400640 1 -- 172.21.15.7:0/1799778924 learned_addr learned my addr 172.21.15.7:0/1799778924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4abe00640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4ac0b73b0 0x7ff4ac1f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4b1000640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4ac160a20 0x7ff4ac1f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4ab400640 1 -- 172.21.15.7:0/1799778924 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4ac0b73b0 msgr2=0x7ff4ac1f9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4ab400640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4ac0b73b0 0x7ff4ac1f9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4ab400640 1 -- 172.21.15.7:0/1799778924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4ac160a20 msgr2=0x7ff4ac1f9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4ab400640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4ac160a20 0x7ff4ac1f9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4ab400640 1 -- 172.21.15.7:0/1799778924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac152b50 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.506+0000 7ff4b1000640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4ac160a20 0x7ff4ac1f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.507+0000 7ff4ab400640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4ac0ce7b0 0x7ff4ac152270 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff498009870 tx=0x7ff4980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.507+0000 7ff4a3e00640 1 -- 172.21.15.7:0/1799778924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498041020 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.507+0000 7ff4a3e00640 1 -- 172.21.15.7:0/1799778924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff498030970 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.507+0000 7ff4b1a00640 1 -- 172.21.15.7:0/1799778924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac152e30 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.507+0000 7ff4a3e00640 1 -- 172.21.15.7:0/1799778924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff498037040 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.507+0000 7ff4b1a00640 1 -- 172.21.15.7:0/1799778924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac15d4d0 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.509+0000 7ff4b1a00640 1 -- 172.21.15.7:0/1799778924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ac050740 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.510+0000 7ff4a3e00640 1 -- 172.21.15.7:0/1799778924 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff498044020 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.511+0000 7ff4a3e00640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff48c0744e0 0x7ff48c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:11:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.512+0000 7ff4a3e00640 1 -- 172.21.15.7:0/1799778924 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7ff4980b60a0 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.512+0000 7ff4abe00640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff48c0744e0 0x7ff48c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:11:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.513+0000 7ff4abe00640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff48c0744e0 0x7ff48c0769f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00a950 tx=0x7ff49c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:11:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.517+0000 7ff4a3e00640 1 -- 172.21.15.7:0/1799778924 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff49804b030 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.717+0000 7ff4b1a00640 1 -- 172.21.15.7:0/1799778924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff4ac06c850 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.718+0000 7ff4a3e00640 1 -- 172.21.15.7:0/1799778924 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7ff4ac0b7d00 con 0x7ff4ac0ce7b0 2024-04-25T11:11:59.718 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:11:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.721+0000 7ff4b1a00640 1 -- 172.21.15.7:0/1799778924 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff48c0744e0 msgr2=0x7ff48c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.721+0000 7ff4b1a00640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff48c0744e0 0x7ff48c0769f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00a950 tx=0x7ff49c008040 comp rx=0 tx=0).stop 2024-04-25T11:11:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.721+0000 7ff4b1a00640 1 -- 172.21.15.7:0/1799778924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4ac0ce7b0 msgr2=0x7ff4ac152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:11:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.721+0000 7ff4b1a00640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4ac0ce7b0 0x7ff4ac152270 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff498009870 tx=0x7ff4980046c0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.721+0000 7ff4b1a00640 1 -- 172.21.15.7:0/1799778924 shutdown_connections 2024-04-25T11:11:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.721+0000 7ff4b1a00640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff48c0744e0 0x7ff48c0769f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.721+0000 7ff4b1a00640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4ac160a20 0x7ff4ac1f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.722+0000 7ff4b1a00640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4ac0ce7b0 0x7ff4ac152270 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.722+0000 7ff4b1a00640 1 --2- 172.21.15.7:0/1799778924 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4ac0b73b0 0x7ff4ac1f9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:11:59.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.722+0000 7ff4b1a00640 1 -- 172.21.15.7:0/1799778924 >> 172.21.15.7:0/1799778924 conn(0x7ff4ac09b540 msgr2=0x7ff4ac071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:11:59.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.722+0000 7ff4b1a00640 1 -- 172.21.15.7:0/1799778924 shutdown_connections 2024-04-25T11:11:59.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:11:59.722+0000 7ff4b1a00640 1 -- 172.21.15.7:0/1799778924 wait complete. 2024-04-25T11:11:59.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:12:00.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.031+0000 7f0c92c00640 1 Processor -- start 2024-04-25T11:12:00.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.031+0000 7f0c92c00640 1 -- start start 2024-04-25T11:12:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.033+0000 7f0c92c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c8c0aed40 0x7f0c8c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.033+0000 7f0c92c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c8c0b5220 0x7f0c8c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.033+0000 7f0c92c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c8c0af420 0x7f0c8c0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.033+0000 7f0c92c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0c8c15e1f0 con 0x7f0c8c0b5220 2024-04-25T11:12:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.033+0000 7f0c92c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0c8c0afd70 con 0x7f0c8c0aed40 2024-04-25T11:12:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.033+0000 7f0c92c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8c0b64b0 con 0x7f0c8c0af420 2024-04-25T11:12:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.034+0000 7f0c90e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c8c0b5220 0x7f0c8c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.034+0000 7f0c90e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c8c0b5220 0x7f0c8c0ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34798/0 (socket says 172.21.15.7:34798) 2024-04-25T11:12:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.034+0000 7f0c90e00640 1 -- 172.21.15.7:0/3495371331 learned_addr learned my addr 172.21.15.7:0/3495371331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.034+0000 7f0c91800640 1 --2- 172.21.15.7:0/3495371331 >> v2:172.21.15.87:3300/0 conn(0x7f0c8c0aed40 0x7f0c8c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.034+0000 7f0c92200640 1 --2- 172.21.15.7:0/3495371331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c8c0af420 0x7f0c8c0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.034+0000 7f0c90e00640 1 -- 172.21.15.7:0/3495371331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c8c0af420 msgr2=0x7f0c8c0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.034+0000 7f0c90e00640 1 --2- 172.21.15.7:0/3495371331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c8c0af420 0x7f0c8c0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.035+0000 7f0c90e00640 1 -- 172.21.15.7:0/3495371331 >> v2:172.21.15.87:3300/0 conn(0x7f0c8c0aed40 msgr2=0x7f0c8c15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.035+0000 7f0c90e00640 1 --2- 172.21.15.7:0/3495371331 >> v2:172.21.15.87:3300/0 conn(0x7f0c8c0aed40 0x7f0c8c15dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.035+0000 7f0c90e00640 1 -- 172.21.15.7:0/3495371331 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c06c610 con 0x7f0c8c0b5220 2024-04-25T11:12:00.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.035+0000 7f0c91800640 1 --2- 172.21.15.7:0/3495371331 >> v2:172.21.15.87:3300/0 conn(0x7f0c8c0aed40 0x7f0c8c15dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:12:00.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.036+0000 7f0c90e00640 1 --2- 172.21.15.7:0/3495371331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c0b5220 0x7f0c8c0ce4a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0725e0 tx=0x7f0c7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1017731ca6e81be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:00.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.036+0000 7f0c8be00640 1 -- 172.21.15.7:0/3495371331 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78037070 con 0x7f0c8c0b5220 2024-04-25T11:12:00.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.036+0000 7f0c8be00640 1 -- 172.21.15.7:0/3495371331 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c78032440 con 0x7f0c8c0b5220 2024-04-25T11:12:00.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.036+0000 7f0c8be00640 1 -- 172.21.15.7:0/3495371331 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78032810 con 0x7f0c8c0b5220 2024-04-25T11:12:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.037+0000 7f0c92c00640 1 -- 172.21.15.7:0/3495371331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c0b5220 msgr2=0x7f0c8c0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.037+0000 7f0c92c00640 1 --2- 172.21.15.7:0/3495371331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c0b5220 0x7f0c8c0ce4a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0725e0 tx=0x7f0c7802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.037+0000 7f0c92c00640 1 -- 172.21.15.7:0/3495371331 shutdown_connections 2024-04-25T11:12:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.037+0000 7f0c92c00640 1 --2- 172.21.15.7:0/3495371331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c8c0af420 0x7f0c8c0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.037+0000 7f0c92c00640 1 --2- 172.21.15.7:0/3495371331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c0b5220 0x7f0c8c0ce4a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.037+0000 7f0c92c00640 1 --2- 172.21.15.7:0/3495371331 >> v2:172.21.15.87:3300/0 conn(0x7f0c8c0aed40 0x7f0c8c15dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.037+0000 7f0c92c00640 1 -- 172.21.15.7:0/3495371331 >> 172.21.15.7:0/3495371331 conn(0x7f0c8c0b3820 msgr2=0x7f0c8c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.037+0000 7f0c92c00640 1 -- 172.21.15.7:0/3495371331 shutdown_connections 2024-04-25T11:12:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.037+0000 7f0c92c00640 1 -- 172.21.15.7:0/3495371331 wait complete. 2024-04-25T11:12:00.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.039+0000 7f0c92c00640 1 Processor -- start 2024-04-25T11:12:00.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.039+0000 7f0c92c00640 1 -- start start 2024-04-25T11:12:00.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.041+0000 7f0c92c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c8c0aed40 0x7f0c8c14b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.041+0000 7f0c92c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c0af420 0x7f0c8c1f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.041+0000 7f0c92c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c8c0b5220 0x7f0c8c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.041+0000 7f0c92c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c06ceb0 con 0x7f0c8c0af420 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.041+0000 7f0c92c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c06cbd0 con 0x7f0c8c0aed40 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.041+0000 7f0c92c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8c0b6110 con 0x7f0c8c0b5220 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.041+0000 7f0c90e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c0af420 0x7f0c8c1f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.041+0000 7f0c90e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c0af420 0x7f0c8c1f7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34810/0 (socket says 172.21.15.7:34810) 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.041+0000 7f0c90e00640 1 -- 172.21.15.7:0/516097213 learned_addr learned my addr 172.21.15.7:0/516097213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.042+0000 7f0c91800640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c8c0aed40 0x7f0c8c14b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.042+0000 7f0c90e00640 1 -- 172.21.15.7:0/516097213 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c8c0b5220 msgr2=0x7f0c8c159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.042+0000 7f0c90e00640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c8c0b5220 0x7f0c8c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.042+0000 7f0c90e00640 1 -- 172.21.15.7:0/516097213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c8c0aed40 msgr2=0x7f0c8c14b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.042+0000 7f0c90e00640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c8c0aed40 0x7f0c8c14b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.042+0000 7f0c90e00640 1 -- 172.21.15.7:0/516097213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c15e5c0 con 0x7f0c8c0af420 2024-04-25T11:12:00.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.042+0000 7f0c91800640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c8c0aed40 0x7f0c8c14b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:00.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.042+0000 7f0c90e00640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c0af420 0x7f0c8c1f7460 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c78000c00 tx=0x7f0c780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:00.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.042+0000 7f0c89600640 1 -- 172.21.15.7:0/516097213 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78041020 con 0x7f0c8c0af420 2024-04-25T11:12:00.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.042+0000 7f0c89600640 1 -- 172.21.15.7:0/516097213 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c780306c0 con 0x7f0c8c0af420 2024-04-25T11:12:00.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.042+0000 7f0c89600640 1 -- 172.21.15.7:0/516097213 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c78037040 con 0x7f0c8c0af420 2024-04-25T11:12:00.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.043+0000 7f0c92c00640 1 -- 172.21.15.7:0/516097213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8c15e8a0 con 0x7f0c8c0af420 2024-04-25T11:12:00.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.043+0000 7f0c92c00640 1 -- 172.21.15.7:0/516097213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8c14b830 con 0x7f0c8c0af420 2024-04-25T11:12:00.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.044+0000 7f0c92c00640 1 -- 172.21.15.7:0/516097213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8c1f7f20 con 0x7f0c8c0af420 2024-04-25T11:12:00.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.045+0000 7f0c89600640 1 -- 172.21.15.7:0/516097213 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0c78044020 con 0x7f0c8c0af420 2024-04-25T11:12:00.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.046+0000 7f0c89600640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c6c0744b0 0x7f0c6c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.046+0000 7f0c89600640 1 -- 172.21.15.7:0/516097213 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f0c780b5f90 con 0x7f0c8c0af420 2024-04-25T11:12:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.046+0000 7f0c91800640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c6c0744b0 0x7f0c6c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.047+0000 7f0c91800640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c6c0744b0 0x7f0c6c0769c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c80006fd0 tx=0x7f0c80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:00.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.051+0000 7f0c89600640 1 -- 172.21.15.7:0/516097213 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0c7804b030 con 0x7f0c8c0af420 2024-04-25T11:12:00.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.296+0000 7f0c92c00640 1 -- 172.21.15.7:0/516097213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c8c050740 con 0x7f0c8c0af420 2024-04-25T11:12:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.298+0000 7f0c89600640 1 -- 172.21.15.7:0/516097213 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7f0c78080280 con 0x7f0c8c0af420 2024-04-25T11:12:00.298 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:00.298 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3458223722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3458223722},{"type":"v1","addr":"172.21.15.87:6835","nonce":3458223722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1383067804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1383067804},{"type":"v1","addr":"172.21.15.7:6835","nonce":1383067804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/435824250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":435824250},{"type":"v1","addr":"172.21.15.7:6837","nonce":435824250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:10:23.023233+0000","modified":"2024-04-25T11:10:53.526222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1091275258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1091275258},{"type":"v1","addr":"172.21.15.87:6834","nonce":1091275258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:12:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-04-25T11:12:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.300+0000 7f0c92c00640 1 -- 172.21.15.7:0/516097213 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c6c0744b0 msgr2=0x7f0c6c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.300+0000 7f0c92c00640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c6c0744b0 0x7f0c6c0769c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0c80006fd0 tx=0x7f0c80008040 comp rx=0 tx=0).stop 2024-04-25T11:12:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.300+0000 7f0c92c00640 1 -- 172.21.15.7:0/516097213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c0af420 msgr2=0x7f0c8c1f7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.300+0000 7f0c92c00640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c0af420 0x7f0c8c1f7460 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c78000c00 tx=0x7f0c780046c0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.300+0000 7f0c92c00640 1 -- 172.21.15.7:0/516097213 shutdown_connections 2024-04-25T11:12:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.300+0000 7f0c92c00640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0c6c0744b0 0x7f0c6c0769c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.300+0000 7f0c92c00640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c8c0b5220 0x7f0c8c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.300+0000 7f0c92c00640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c8c0af420 0x7f0c8c1f7460 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.300+0000 7f0c92c00640 1 --2- 172.21.15.7:0/516097213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c8c0aed40 0x7f0c8c14b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.301+0000 7f0c92c00640 1 -- 172.21.15.7:0/516097213 >> 172.21.15.7:0/516097213 conn(0x7f0c8c0b3820 msgr2=0x7f0c8c1575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.301+0000 7f0c92c00640 1 -- 172.21.15.7:0/516097213 shutdown_connections 2024-04-25T11:12:00.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.301+0000 7f0c92c00640 1 -- 172.21.15.7:0/516097213 wait complete. 2024-04-25T11:12:00.327 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:12:00.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.609+0000 7f6feb200640 1 Processor -- start 2024-04-25T11:12:00.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.609+0000 7f6feb200640 1 -- start start 2024-04-25T11:12:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.611+0000 7f6feb200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6fe4070890 0x7f6fe415c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.612+0000 7f6feb200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fe40b5140 0x7f6fe406f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.612+0000 7f6feb200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fe406ef10 con 0x7f6fe406fbe0 2024-04-25T11:12:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.612+0000 7f6feb200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6fe406f210 con 0x7f6fe4070890 2024-04-25T11:12:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.612+0000 7f6feb200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe40b6390 con 0x7f6fe40b5140 2024-04-25T11:12:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.612+0000 7f6fe9e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6fe4070890 0x7f6fe415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.612+0000 7f6fe9e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6fe4070890 0x7f6fe415c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34822/0 (socket says 172.21.15.7:34822) 2024-04-25T11:12:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.612+0000 7f6fe9e00640 1 -- 172.21.15.7:0/4208443042 learned_addr learned my addr 172.21.15.7:0/4208443042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:00.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.612+0000 7f6fe9400640 1 --2- 172.21.15.7:0/4208443042 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fe40b5140 0x7f6fe406f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.612+0000 7f6fe9e00640 1 -- 172.21.15.7:0/4208443042 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fe40b5140 msgr2=0x7f6fe406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:00.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.612+0000 7f6fe9e00640 1 --2- 172.21.15.7:0/4208443042 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fe40b5140 0x7f6fe406f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.612+0000 7f6fe9e00640 1 -- 172.21.15.7:0/4208443042 >> v1:172.21.15.87:6789/0 conn(0x7f6fe406fbe0 legacy=0x7f6fe406ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:00.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.613+0000 7f6fe9e00640 1 -- 172.21.15.7:0/4208443042 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe40b6670 con 0x7f6fe4070890 2024-04-25T11:12:00.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.613+0000 7f6fe9400640 1 --2- 172.21.15.7:0/4208443042 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fe40b5140 0x7f6fe406f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:00.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.614+0000 7f6fe9e00640 1 --2- 172.21.15.7:0/4208443042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe4070890 0x7f6fe415c390 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400bb00 tx=0x7f6fd402c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7500d3af40d23a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:00.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.614+0000 7f6fe8a00640 1 -- 172.21.15.7:0/4208443042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd403a070 con 0x7f6fe4070890 2024-04-25T11:12:00.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.614+0000 7f6fe8a00640 1 -- 172.21.15.7:0/4208443042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fd4032480 con 0x7f6fe4070890 2024-04-25T11:12:00.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.614+0000 7f6fe8a00640 1 -- 172.21.15.7:0/4208443042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4032850 con 0x7f6fe4070890 2024-04-25T11:12:00.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.615+0000 7f6feb200640 1 -- 172.21.15.7:0/4208443042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe4070890 msgr2=0x7f6fe415c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:00.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.615+0000 7f6feb200640 1 --2- 172.21.15.7:0/4208443042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe4070890 0x7f6fe415c390 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400bb00 tx=0x7f6fd402c600 comp rx=0 tx=0).stop 2024-04-25T11:12:00.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.615+0000 7f6feb200640 1 -- 172.21.15.7:0/4208443042 shutdown_connections 2024-04-25T11:12:00.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.615+0000 7f6feb200640 1 --2- 172.21.15.7:0/4208443042 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fe40b5140 0x7f6fe406f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.615+0000 7f6feb200640 1 --2- 172.21.15.7:0/4208443042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe4070890 0x7f6fe415c390 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.615+0000 7f6feb200640 1 -- 172.21.15.7:0/4208443042 >> 172.21.15.7:0/4208443042 conn(0x7f6fe40b3820 msgr2=0x7f6fe40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:00.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.615+0000 7f6feb200640 1 -- 172.21.15.7:0/4208443042 shutdown_connections 2024-04-25T11:12:00.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.615+0000 7f6feb200640 1 -- 172.21.15.7:0/4208443042 wait complete. 2024-04-25T11:12:00.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.617+0000 7f6feb200640 1 Processor -- start 2024-04-25T11:12:00.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.618+0000 7f6feb200640 1 -- start start 2024-04-25T11:12:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.619+0000 7f6feb200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fe406fbe0 0x7f6fe41593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.619+0000 7f6feb200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fe4070890 0x7f6fe409c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.619+0000 7f6feb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe40b5140 0x7f6fe415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.619+0000 7f6feb200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe406d510 con 0x7f6fe40b5140 2024-04-25T11:12:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.619+0000 7f6feb200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe40b6950 con 0x7f6fe4070890 2024-04-25T11:12:00.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.619+0000 7f6feb200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe406d690 con 0x7f6fe406fbe0 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.620+0000 7f6fea800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe40b5140 0x7f6fe415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.620+0000 7f6fea800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe40b5140 0x7f6fe415d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34838/0 (socket says 172.21.15.7:34838) 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.620+0000 7f6fea800640 1 -- 172.21.15.7:0/3436340196 learned_addr learned my addr 172.21.15.7:0/3436340196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.620+0000 7f6fe9e00640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fe406fbe0 0x7f6fe41593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.620+0000 7f6fe9400640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fe4070890 0x7f6fe409c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.620+0000 7f6fea800640 1 -- 172.21.15.7:0/3436340196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fe406fbe0 msgr2=0x7f6fe41593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.620+0000 7f6fea800640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fe406fbe0 0x7f6fe41593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.620+0000 7f6fea800640 1 -- 172.21.15.7:0/3436340196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fe4070890 msgr2=0x7f6fe409c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.620+0000 7f6fea800640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fe4070890 0x7f6fe409c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.620+0000 7f6fea800640 1 -- 172.21.15.7:0/3436340196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe409d4b0 con 0x7f6fe40b5140 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.620+0000 7f6fe9400640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fe4070890 0x7f6fe409c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.621+0000 7f6fea800640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe40b5140 0x7f6fe415d8b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000b9a0 tx=0x7f6fe000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.621+0000 7f6fde000640 1 -- 172.21.15.7:0/3436340196 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0019070 con 0x7f6fe40b5140 2024-04-25T11:12:00.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.621+0000 7f6fde000640 1 -- 172.21.15.7:0/3436340196 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe000cba0 con 0x7f6fe40b5140 2024-04-25T11:12:00.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.621+0000 7f6feb200640 1 -- 172.21.15.7:0/3436340196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe409d790 con 0x7f6fe40b5140 2024-04-25T11:12:00.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.621+0000 7f6fde000640 1 -- 172.21.15.7:0/3436340196 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0014440 con 0x7f6fe40b5140 2024-04-25T11:12:00.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.621+0000 7f6feb200640 1 -- 172.21.15.7:0/3436340196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe409dd50 con 0x7f6fe40b5140 2024-04-25T11:12:00.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.622+0000 7f6feb200640 1 -- 172.21.15.7:0/3436340196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe409d220 con 0x7f6fe40b5140 2024-04-25T11:12:00.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.624+0000 7f6fde000640 1 -- 172.21.15.7:0/3436340196 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6fe001c070 con 0x7f6fe40b5140 2024-04-25T11:12:00.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.625+0000 7f6fde000640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fc80744b0 0x7f6fc80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:00.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.625+0000 7f6fde000640 1 -- 172.21.15.7:0/3436340196 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f6fe00952f0 con 0x7f6fe40b5140 2024-04-25T11:12:00.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.625+0000 7f6fe9e00640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fc80744b0 0x7f6fc80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:00.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.626+0000 7f6fe9e00640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fc80744b0 0x7f6fc80769c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4037000 tx=0x7f6fd4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:00.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.629+0000 7f6fde000640 1 -- 172.21.15.7:0/3436340196 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fe005ef90 con 0x7f6fe40b5140 2024-04-25T11:12:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.833+0000 7f6feb200640 1 -- 172.21.15.7:0/3436340196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fe4050740 con 0x7f6fe40b5140 2024-04-25T11:12:00.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.837+0000 7f6fde000640 1 -- 172.21.15.7:0/3436340196 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7f6fe0063290 con 0x7f6fe40b5140 2024-04-25T11:12:00.838 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:00.838 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:12:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 -- 172.21.15.7:0/3436340196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fc80744b0 msgr2=0x7f6fc80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fc80744b0 0x7f6fc80769c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4037000 tx=0x7f6fd4030000 comp rx=0 tx=0).stop 2024-04-25T11:12:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 -- 172.21.15.7:0/3436340196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe40b5140 msgr2=0x7f6fe415d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe40b5140 0x7f6fe415d8b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000b9a0 tx=0x7f6fe000be60 comp rx=0 tx=0).stop 2024-04-25T11:12:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 -- 172.21.15.7:0/3436340196 shutdown_connections 2024-04-25T11:12:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fc80744b0 0x7f6fc80769c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fe40b5140 0x7f6fe415d8b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fe4070890 0x7f6fe409c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 --2- 172.21.15.7:0/3436340196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fe406fbe0 0x7f6fe41593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 -- 172.21.15.7:0/3436340196 >> 172.21.15.7:0/3436340196 conn(0x7f6fe40b3820 msgr2=0x7f6fe415b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 -- 172.21.15.7:0/3436340196 shutdown_connections 2024-04-25T11:12:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:00.840+0000 7f6feb200640 1 -- 172.21.15.7:0/3436340196 wait complete. 2024-04-25T11:12:00.866 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:12:00.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:12:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.140+0000 7f4a79600640 1 Processor -- start 2024-04-25T11:12:01.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.141+0000 7f4a79600640 1 -- start start 2024-04-25T11:12:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.143+0000 7f4a79600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a7415ef90 0x7f4a7409f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.143+0000 7f4a79600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a740af540 0x7f4a740cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.143+0000 7f4a79600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a7415f4e0 con 0x7f4a7415bc40 2024-04-25T11:12:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.143+0000 7f4a79600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4a74071190 con 0x7f4a740af540 2024-04-25T11:12:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.143+0000 7f4a79600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a740b0560 con 0x7f4a7415ef90 2024-04-25T11:12:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.143+0000 7f4a73400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a740af540 0x7f4a740cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.144+0000 7f4a73400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a740af540 0x7f4a740cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34850/0 (socket says 172.21.15.7:34850) 2024-04-25T11:12:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.144+0000 7f4a73400640 1 -- 172.21.15.7:0/831469027 learned_addr learned my addr 172.21.15.7:0/831469027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:01.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.144+0000 7f4a72a00640 1 --2- 172.21.15.7:0/831469027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a7415ef90 0x7f4a7409f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.144+0000 7f4a73400640 1 -- 172.21.15.7:0/831469027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a7415ef90 msgr2=0x7f4a7409f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:01.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.144+0000 7f4a73400640 1 --2- 172.21.15.7:0/831469027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a7415ef90 0x7f4a7409f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.144+0000 7f4a73400640 1 -- 172.21.15.7:0/831469027 >> v1:172.21.15.87:6789/0 conn(0x7f4a7415bc40 legacy=0x7f4a740aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.144+0000 7f4a73400640 1 -- 172.21.15.7:0/831469027 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a740b0840 con 0x7f4a740af540 2024-04-25T11:12:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.145+0000 7f4a73400640 1 --2- 172.21.15.7:0/831469027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a740af540 0x7f4a740cf980 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009e50 tx=0x7f4a6c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fc36f1a107daac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.146+0000 7f4a72000640 1 -- 172.21.15.7:0/831469027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c036070 con 0x7f4a740af540 2024-04-25T11:12:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.146+0000 7f4a72000640 1 -- 172.21.15.7:0/831469027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a6c031600 con 0x7f4a740af540 2024-04-25T11:12:01.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.146+0000 7f4a72000640 1 -- 172.21.15.7:0/831469027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c0319d0 con 0x7f4a740af540 2024-04-25T11:12:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.146+0000 7f4a79600640 1 -- 172.21.15.7:0/831469027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a740af540 msgr2=0x7f4a740cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.146+0000 7f4a79600640 1 --2- 172.21.15.7:0/831469027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a740af540 0x7f4a740cf980 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009e50 tx=0x7f4a6c02a680 comp rx=0 tx=0).stop 2024-04-25T11:12:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.147+0000 7f4a79600640 1 -- 172.21.15.7:0/831469027 shutdown_connections 2024-04-25T11:12:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.147+0000 7f4a79600640 1 --2- 172.21.15.7:0/831469027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a740af540 0x7f4a740cf980 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.147+0000 7f4a79600640 1 --2- 172.21.15.7:0/831469027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a7415ef90 0x7f4a7409f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.147+0000 7f4a79600640 1 -- 172.21.15.7:0/831469027 >> 172.21.15.7:0/831469027 conn(0x7f4a74074200 msgr2=0x7f4a7406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.147+0000 7f4a79600640 1 -- 172.21.15.7:0/831469027 shutdown_connections 2024-04-25T11:12:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.147+0000 7f4a79600640 1 -- 172.21.15.7:0/831469027 wait complete. 2024-04-25T11:12:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.150+0000 7f4a79600640 1 Processor -- start 2024-04-25T11:12:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.150+0000 7f4a79600640 1 -- start start 2024-04-25T11:12:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.152+0000 7f4a79600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a7415bc40 0x7f4a7415dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.152+0000 7f4a79600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a7415ef90 0x7f4a74149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.152+0000 7f4a79600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a741f9960 0x7f4a740b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.152+0000 7f4a79600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a740b1320 con 0x7f4a7415bc40 2024-04-25T11:12:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.152+0000 7f4a79600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4a740b0e00 con 0x7f4a7415ef90 2024-04-25T11:12:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.152+0000 7f4a79600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a740afb80 con 0x7f4a741f9960 2024-04-25T11:12:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.152+0000 7f4a72a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a7415bc40 0x7f4a7415dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.152+0000 7f4a72a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a7415bc40 0x7f4a7415dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34862/0 (socket says 172.21.15.7:34862) 2024-04-25T11:12:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.152+0000 7f4a72a00640 1 -- 172.21.15.7:0/1519228617 learned_addr learned my addr 172.21.15.7:0/1519228617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.152+0000 7f4a6be00640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a7415ef90 0x7f4a74149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.152+0000 7f4a73400640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a741f9960 0x7f4a740b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.153+0000 7f4a72a00640 1 -- 172.21.15.7:0/1519228617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a741f9960 msgr2=0x7f4a740b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.153+0000 7f4a72a00640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a741f9960 0x7f4a740b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.153+0000 7f4a72a00640 1 -- 172.21.15.7:0/1519228617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a7415ef90 msgr2=0x7f4a74149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.153+0000 7f4a72a00640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a7415ef90 0x7f4a74149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.153+0000 7f4a72a00640 1 -- 172.21.15.7:0/1519228617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7414a390 con 0x7f4a7415bc40 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.153+0000 7f4a72a00640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a7415bc40 0x7f4a7415dc80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400d620 tx=0x7f4a6400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.153+0000 7f4a6aa00640 1 -- 172.21.15.7:0/1519228617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400b620 con 0x7f4a7415bc40 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.153+0000 7f4a6aa00640 1 -- 172.21.15.7:0/1519228617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a64019070 con 0x7f4a7415bc40 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.153+0000 7f4a79600640 1 -- 172.21.15.7:0/1519228617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a741fa140 con 0x7f4a7415bc40 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.153+0000 7f4a6aa00640 1 -- 172.21.15.7:0/1519228617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400ba00 con 0x7f4a7415bc40 2024-04-25T11:12:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.153+0000 7f4a79600640 1 -- 172.21.15.7:0/1519228617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a741fa700 con 0x7f4a7415bc40 2024-04-25T11:12:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.155+0000 7f4a79600640 1 -- 172.21.15.7:0/1519228617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a740cecc0 con 0x7f4a7415bc40 2024-04-25T11:12:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.156+0000 7f4a6aa00640 1 -- 172.21.15.7:0/1519228617 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4a6400bba0 con 0x7f4a7415bc40 2024-04-25T11:12:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.157+0000 7f4a6aa00640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4a50074500 0x7f4a50076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.158+0000 7f4a6aa00640 1 -- 172.21.15.7:0/1519228617 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f4a64095a80 con 0x7f4a7415bc40 2024-04-25T11:12:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.158+0000 7f4a6be00640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4a50074500 0x7f4a50076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.158+0000 7f4a6be00640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4a50074500 0x7f4a50076a10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c007fd0 tx=0x7f4a5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.163+0000 7f4a6aa00640 1 -- 172.21.15.7:0/1519228617 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a740cecc0 con 0x7f4a7415bc40 2024-04-25T11:12:01.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.405+0000 7f4a79600640 1 -- 172.21.15.7:0/1519228617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a7406a100 con 0x7f4a7415bc40 2024-04-25T11:12:01.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.407+0000 7f4a6aa00640 1 -- 172.21.15.7:0/1519228617 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7f4a64066e20 con 0x7f4a7415bc40 2024-04-25T11:12:01.407 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:01.408 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3458223722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3458223722},{"type":"v1","addr":"172.21.15.87:6835","nonce":3458223722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1383067804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1383067804},{"type":"v1","addr":"172.21.15.7:6835","nonce":1383067804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/435824250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":435824250},{"type":"v1","addr":"172.21.15.7:6837","nonce":435824250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:10:23.023233+0000","modified":"2024-04-25T11:10:53.526222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1091275258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1091275258},{"type":"v1","addr":"172.21.15.87:6834","nonce":1091275258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:12:01.408 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-04-25T11:12:01.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.409+0000 7f4a79600640 1 -- 172.21.15.7:0/1519228617 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4a50074500 msgr2=0x7f4a50076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.409+0000 7f4a79600640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4a50074500 0x7f4a50076a10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c007fd0 tx=0x7f4a5c009040 comp rx=0 tx=0).stop 2024-04-25T11:12:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.409+0000 7f4a79600640 1 -- 172.21.15.7:0/1519228617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a7415bc40 msgr2=0x7f4a7415dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.409+0000 7f4a79600640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a7415bc40 0x7f4a7415dc80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400d620 tx=0x7f4a6400dae0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.410+0000 7f4a79600640 1 -- 172.21.15.7:0/1519228617 shutdown_connections 2024-04-25T11:12:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.410+0000 7f4a79600640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4a50074500 0x7f4a50076a10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.410+0000 7f4a79600640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a741f9960 0x7f4a740b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.410+0000 7f4a79600640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a7415ef90 0x7f4a74149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.410+0000 7f4a79600640 1 --2- 172.21.15.7:0/1519228617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a7415bc40 0x7f4a7415dc80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.410+0000 7f4a79600640 1 -- 172.21.15.7:0/1519228617 >> 172.21.15.7:0/1519228617 conn(0x7f4a74074200 msgr2=0x7f4a7406d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:01.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.410+0000 7f4a79600640 1 -- 172.21.15.7:0/1519228617 shutdown_connections 2024-04-25T11:12:01.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.410+0000 7f4a79600640 1 -- 172.21.15.7:0/1519228617 wait complete. 2024-04-25T11:12:01.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:12:01.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.716+0000 7f9980c00640 1 Processor -- start 2024-04-25T11:12:01.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.716+0000 7f9980c00640 1 -- start start 2024-04-25T11:12:01.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.718+0000 7f9980c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f997c06cc80 0x7f997c0bae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.718+0000 7f9980c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f997c164120 con 0x7f997c163bb0 2024-04-25T11:12:01.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.718+0000 7f9980c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f997c164400 con 0x7f997c0d03a0 2024-04-25T11:12:01.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.718+0000 7f9980c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f997c0b51c0 con 0x7f997c06cc80 2024-04-25T11:12:01.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.719+0000 7f997b400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f997c163bb0 0x7f997c0b8d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59834/0 (socket says 172.21.15.7:59834) 2024-04-25T11:12:01.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.719+0000 7f997b400640 1 -- 172.21.15.7:0/1600663904 learned_addr learned my addr 172.21.15.7:0/1600663904 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:01.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.719+0000 7f997be00640 1 --2- 172.21.15.7:0/1600663904 >> v2:172.21.15.87:3300/0 conn(0x7f997c06cc80 0x7f997c0bae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.719+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797071637 0 0) 0x7f997c164120 con 0x7f997c163bb0 2024-04-25T11:12:01.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.719+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9960003520 con 0x7f997c163bb0 2024-04-25T11:12:01.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2074716283 0 0) 0x7f9960003520 con 0x7f997c163bb0 2024-04-25T11:12:01.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 >> v2:172.21.15.87:3300/0 conn(0x7f997c06cc80 msgr2=0x7f997c0bae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:01.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 --2- 172.21.15.7:0/1600663904 >> v2:172.21.15.87:3300/0 conn(0x7f997c06cc80 0x7f997c0bae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 >> v1:172.21.15.87:6789/0 conn(0x7f997c0d03a0 legacy=0x7f997c06c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c1646e0 con 0x7f997c163bb0 2024-04-25T11:12:01.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f996c001670 con 0x7f997c163bb0 2024-04-25T11:12:01.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 >> v1:172.21.15.7:6789/0 conn(0x7f997c163bb0 legacy=0x7f997c0b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99600039b0 0x7f9960003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9960004800 0x7f9960004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 0x7f996000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f997c0b51c0 con 0x7f99600051c0 2024-04-25T11:12:01.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f996000c160 con 0x7f9960004800 2024-04-25T11:12:01.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f996000c360 con 0x7f99600039b0 2024-04-25T11:12:01.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.720+0000 7f997be00640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 0x7f996000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.721+0000 7f997aa00640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9960004800 0x7f9960004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.721+0000 7f997b400640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99600039b0 0x7f9960003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.721+0000 7f997be00640 1 -- 172.21.15.7:0/1600663904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99600039b0 msgr2=0x7f9960003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:01.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.721+0000 7f997be00640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99600039b0 0x7f9960003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.721+0000 7f997be00640 1 -- 172.21.15.7:0/1600663904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9960004800 msgr2=0x7f9960004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:01.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.721+0000 7f997be00640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9960004800 0x7f9960004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.721+0000 7f997be00640 1 -- 172.21.15.7:0/1600663904 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9960003520 con 0x7f99600051c0 2024-04-25T11:12:01.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.721+0000 7f997b400640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99600039b0 0x7f9960003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.722+0000 7f997be00640 1 -- 172.21.15.7:0/1600663904 reap_dead start 2024-04-25T11:12:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.722+0000 7f997be00640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 0x7f996000b9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9974009fc0 tx=0x7f9960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dacbeddb9c241b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.722+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974009c10 con 0x7f99600051c0 2024-04-25T11:12:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.722+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9974003bd0 con 0x7f99600051c0 2024-04-25T11:12:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.722+0000 7f997a000640 1 -- 172.21.15.7:0/1600663904 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974004020 con 0x7f99600051c0 2024-04-25T11:12:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.723+0000 7f9980c00640 1 -- 172.21.15.7:0/1600663904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 msgr2=0x7f996000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.723+0000 7f9980c00640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 0x7f996000b9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9974009fc0 tx=0x7f9960005830 comp rx=0 tx=0).stop 2024-04-25T11:12:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.723+0000 7f9980c00640 1 -- 172.21.15.7:0/1600663904 shutdown_connections 2024-04-25T11:12:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.723+0000 7f9980c00640 1 --2- 172.21.15.7:0/1600663904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 0x7f996000b9c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.723+0000 7f9980c00640 1 -- 172.21.15.7:0/1600663904 >> 172.21.15.7:0/1600663904 conn(0x7f997c06eb00 msgr2=0x7f997c0d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.723+0000 7f9980c00640 1 -- 172.21.15.7:0/1600663904 shutdown_connections 2024-04-25T11:12:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.723+0000 7f9980c00640 1 -- 172.21.15.7:0/1600663904 wait complete. 2024-04-25T11:12:01.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.726+0000 7f9980c00640 1 Processor -- start 2024-04-25T11:12:01.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.726+0000 7f9980c00640 1 -- start start 2024-04-25T11:12:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.727+0000 7f9980c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 0x7f997c14e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.727+0000 7f9980c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c159a80 0x7f997c159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.727+0000 7f9980c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f997c0cde70 0x7f997c0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f9980c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f997c0d07b0 con 0x7f99600051c0 2024-04-25T11:12:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f9980c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f997c0d4170 con 0x7f997c159a80 2024-04-25T11:12:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f9980c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f997c0bae90 con 0x7f997c0cde70 2024-04-25T11:12:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f997b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 0x7f997c14e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f997b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 0x7f997c14e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34886/0 (socket says 172.21.15.7:34886) 2024-04-25T11:12:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f997b400640 1 -- 172.21.15.7:0/182558760 learned_addr learned my addr 172.21.15.7:0/182558760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f997aa00640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c159a80 0x7f997c159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f997b400640 1 -- 172.21.15.7:0/182558760 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f997c0cde70 msgr2=0x7f997c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f997b400640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f997c0cde70 0x7f997c0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f997b400640 1 -- 172.21.15.7:0/182558760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c159a80 msgr2=0x7f997c159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f997b400640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c159a80 0x7f997c159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f997b400640 1 -- 172.21.15.7:0/182558760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997c15a3d0 con 0x7f99600051c0 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.728+0000 7f997aa00640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c159a80 0x7f997c159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.729+0000 7f997b400640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 0x7f997c14e150 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f996c00d770 tx=0x7f996c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.729+0000 7f9973400640 1 -- 172.21.15.7:0/182558760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c019070 con 0x7f99600051c0 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.729+0000 7f9973400640 1 -- 172.21.15.7:0/182558760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996c0074e0 con 0x7f99600051c0 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.729+0000 7f9973400640 1 -- 172.21.15.7:0/182558760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c0049e0 con 0x7f99600051c0 2024-04-25T11:12:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.729+0000 7f9980c00640 1 -- 172.21.15.7:0/182558760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997c0ccdd0 con 0x7f99600051c0 2024-04-25T11:12:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.729+0000 7f9980c00640 1 -- 172.21.15.7:0/182558760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997c0cd390 con 0x7f99600051c0 2024-04-25T11:12:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.730+0000 7f9980c00640 1 -- 172.21.15.7:0/182558760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997c050740 con 0x7f99600051c0 2024-04-25T11:12:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.732+0000 7f9973400640 1 -- 172.21.15.7:0/182558760 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f996c01c070 con 0x7f99600051c0 2024-04-25T11:12:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.733+0000 7f9973400640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f995c0744b0 0x7f995c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.733+0000 7f9973400640 1 -- 172.21.15.7:0/182558760 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f996c094f20 con 0x7f99600051c0 2024-04-25T11:12:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.733+0000 7f997aa00640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f995c0744b0 0x7f995c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.734+0000 7f997aa00640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f995c0744b0 0x7f995c0769c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9968000c00 tx=0x7f9968009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:01.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.738+0000 7f9973400640 1 -- 172.21.15.7:0/182558760 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f997c15a3d0 con 0x7f99600051c0 2024-04-25T11:12:01.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.986+0000 7f9980c00640 1 -- 172.21.15.7:0/182558760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f997c0d5630 con 0x7f99600051c0 2024-04-25T11:12:01.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.987+0000 7f9973400640 1 -- 172.21.15.7:0/182558760 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5168 (secure 0 0 0) 0x7f996c05f3d0 con 0x7f99600051c0 2024-04-25T11:12:01.988 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:01.988 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3458223722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3458223722},{"type":"v1","addr":"172.21.15.87:6835","nonce":3458223722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1383067804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1383067804},{"type":"v1","addr":"172.21.15.7:6835","nonce":1383067804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/435824250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":435824250},{"type":"v1","addr":"172.21.15.7:6837","nonce":435824250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:10:23.023233+0000","modified":"2024-04-25T11:10:53.526222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1091275258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1091275258},{"type":"v1","addr":"172.21.15.87:6834","nonce":1091275258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-25T11:12:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2024-04-25T11:12:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.990+0000 7f9980c00640 1 -- 172.21.15.7:0/182558760 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f995c0744b0 msgr2=0x7f995c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.990+0000 7f9980c00640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f995c0744b0 0x7f995c0769c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9968000c00 tx=0x7f9968009040 comp rx=0 tx=0).stop 2024-04-25T11:12:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.990+0000 7f9980c00640 1 -- 172.21.15.7:0/182558760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 msgr2=0x7f997c14e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.990+0000 7f9980c00640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 0x7f997c14e150 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f996c00d770 tx=0x7f996c00dc30 comp rx=0 tx=0).stop 2024-04-25T11:12:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.991+0000 7f9980c00640 1 -- 172.21.15.7:0/182558760 shutdown_connections 2024-04-25T11:12:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.991+0000 7f9980c00640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f995c0744b0 0x7f995c0769c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.991+0000 7f9980c00640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f997c0cde70 0x7f997c0ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.991+0000 7f9980c00640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f997c159a80 0x7f997c159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.991+0000 7f9980c00640 1 --2- 172.21.15.7:0/182558760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99600051c0 0x7f997c14e150 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.991+0000 7f9980c00640 1 -- 172.21.15.7:0/182558760 >> 172.21.15.7:0/182558760 conn(0x7f997c06eb00 msgr2=0x7f997c0b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.991+0000 7f9980c00640 1 -- 172.21.15.7:0/182558760 shutdown_connections 2024-04-25T11:12:01.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:01.991+0000 7f9980c00640 1 -- 172.21.15.7:0/182558760 wait complete. 2024-04-25T11:12:02.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:12:02.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.302+0000 7f9c89000640 1 Processor -- start 2024-04-25T11:12:02.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.302+0000 7f9c89000640 1 -- start start 2024-04-25T11:12:02.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.304+0000 7f9c89000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840aed40 0x7f9c8415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.304+0000 7f9c89000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c840af420 0x7f9c840af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.304+0000 7f9c89000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c840cebb0 con 0x7f9c840b5220 2024-04-25T11:12:02.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.304+0000 7f9c89000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8415e1f0 con 0x7f9c840af420 2024-04-25T11:12:02.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.304+0000 7f9c89000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9c840afd70 con 0x7f9c840aed40 2024-04-25T11:12:02.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.304+0000 7f9c83e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c840af420 0x7f9c840af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.304+0000 7f9c83e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c840af420 0x7f9c840af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34898/0 (socket says 172.21.15.7:34898) 2024-04-25T11:12:02.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.304+0000 7f9c83e00640 1 -- 172.21.15.7:0/342093694 learned_addr learned my addr 172.21.15.7:0/342093694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:02.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.304+0000 7f9c82a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9c840b5220 0x7f9c840ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59838/0 (socket says 172.21.15.7:59838) 2024-04-25T11:12:02.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.305+0000 7f9c83400640 1 --2- 172.21.15.7:0/342093694 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840aed40 0x7f9c8415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.305+0000 7f9c83e00640 1 -- 172.21.15.7:0/342093694 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840aed40 msgr2=0x7f9c8415dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:02.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.305+0000 7f9c83e00640 1 --2- 172.21.15.7:0/342093694 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840aed40 0x7f9c8415dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.305+0000 7f9c83e00640 1 -- 172.21.15.7:0/342093694 >> v1:172.21.15.7:6789/0 conn(0x7f9c840b5220 legacy=0x7f9c840ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:02.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.305+0000 7f9c83e00640 1 -- 172.21.15.7:0/342093694 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c840b64b0 con 0x7f9c840af420 2024-04-25T11:12:02.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.305+0000 7f9c82000640 1 -- 172.21.15.7:0/342093694 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 422737196 0 0) 0x7f9c70001110 con 0x7f9c840b5220 2024-04-25T11:12:02.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.305+0000 7f9c83400640 1 --2- 172.21.15.7:0/342093694 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840aed40 0x7f9c8415dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:12:02.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.306+0000 7f9c83e00640 1 --2- 172.21.15.7:0/342093694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c840af420 0x7f9c840af830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009e50 tx=0x7f9c7c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b04b6f418fe18ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:02.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.306+0000 7f9c82000640 1 -- 172.21.15.7:0/342093694 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c036070 con 0x7f9c840af420 2024-04-25T11:12:02.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.306+0000 7f9c82000640 1 -- 172.21.15.7:0/342093694 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c031600 con 0x7f9c840af420 2024-04-25T11:12:02.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.306+0000 7f9c82000640 1 -- 172.21.15.7:0/342093694 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0319d0 con 0x7f9c840af420 2024-04-25T11:12:02.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.307+0000 7f9c89000640 1 -- 172.21.15.7:0/342093694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c840af420 msgr2=0x7f9c840af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:02.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.307+0000 7f9c89000640 1 --2- 172.21.15.7:0/342093694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c840af420 0x7f9c840af830 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009e50 tx=0x7f9c7c02a710 comp rx=0 tx=0).stop 2024-04-25T11:12:02.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.307+0000 7f9c89000640 1 -- 172.21.15.7:0/342093694 shutdown_connections 2024-04-25T11:12:02.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.307+0000 7f9c89000640 1 --2- 172.21.15.7:0/342093694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c840af420 0x7f9c840af830 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.307+0000 7f9c89000640 1 --2- 172.21.15.7:0/342093694 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840aed40 0x7f9c8415dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.307+0000 7f9c89000640 1 -- 172.21.15.7:0/342093694 >> 172.21.15.7:0/342093694 conn(0x7f9c840b3820 msgr2=0x7f9c840b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:02.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.307+0000 7f9c89000640 1 -- 172.21.15.7:0/342093694 shutdown_connections 2024-04-25T11:12:02.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.307+0000 7f9c89000640 1 -- 172.21.15.7:0/342093694 wait complete. 2024-04-25T11:12:02.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.310+0000 7f9c89000640 1 Processor -- start 2024-04-25T11:12:02.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.310+0000 7f9c89000640 1 -- start start 2024-04-25T11:12:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.312+0000 7f9c89000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c840aed40 0x7f9c841430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.312+0000 7f9c89000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840af420 0x7f9c84141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.312+0000 7f9c89000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c840b5220 0x7f9c840d0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.312+0000 7f9c89000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8406cbd0 con 0x7f9c840b5220 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.312+0000 7f9c89000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8406c8f0 con 0x7f9c840aed40 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.312+0000 7f9c89000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8406ceb0 con 0x7f9c840af420 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.312+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c840b5220 0x7f9c840d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.312+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c840b5220 0x7f9c840d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34914/0 (socket says 172.21.15.7:34914) 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.312+0000 7f9c83e00640 1 -- 172.21.15.7:0/113580245 learned_addr learned my addr 172.21.15.7:0/113580245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.312+0000 7f9c82a00640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840af420 0x7f9c84141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c83e00640 1 -- 172.21.15.7:0/113580245 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840af420 msgr2=0x7f9c84141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c83400640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c840aed40 0x7f9c841430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c83e00640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840af420 0x7f9c84141070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c83e00640 1 -- 172.21.15.7:0/113580245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c840aed40 msgr2=0x7f9c841430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c83e00640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c840aed40 0x7f9c841430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c83e00640 1 -- 172.21.15.7:0/113580245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c841601a0 con 0x7f9c840b5220 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c82a00640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840af420 0x7f9c84141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c83e00640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c840b5220 0x7f9c840d0690 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c034040 tx=0x7f9c7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:02.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c7b400640 1 -- 172.21.15.7:0/113580245 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c041020 con 0x7f9c840b5220 2024-04-25T11:12:02.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c7b400640 1 -- 172.21.15.7:0/113580245 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c037040 con 0x7f9c840b5220 2024-04-25T11:12:02.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c89000640 1 -- 172.21.15.7:0/113580245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c84143950 con 0x7f9c840b5220 2024-04-25T11:12:02.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.313+0000 7f9c7b400640 1 -- 172.21.15.7:0/113580245 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c036050 con 0x7f9c840b5220 2024-04-25T11:12:02.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.314+0000 7f9c89000640 1 -- 172.21.15.7:0/113580245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c84143f10 con 0x7f9c840b5220 2024-04-25T11:12:02.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.315+0000 7f9c89000640 1 -- 172.21.15.7:0/113580245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c841436c0 con 0x7f9c840b5220 2024-04-25T11:12:02.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.316+0000 7f9c7b400640 1 -- 172.21.15.7:0/113580245 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c7c007c30 con 0x7f9c840b5220 2024-04-25T11:12:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.317+0000 7f9c7b400640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c640744b0 0x7f9c640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.317+0000 7f9c7b400640 1 -- 172.21.15.7:0/113580245 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f9c7c0b5e30 con 0x7f9c840b5220 2024-04-25T11:12:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.317+0000 7f9c83400640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c640744b0 0x7f9c640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.318+0000 7f9c83400640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c640744b0 0x7f9c640769c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c74006fd0 tx=0x7f9c74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:02.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.323+0000 7f9c7b400640 1 -- 172.21.15.7:0/113580245 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c7c04b060 con 0x7f9c840b5220 2024-04-25T11:12:02.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.531+0000 7f9c89000640 1 -- 172.21.15.7:0/113580245 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c84050740 con 0x7f9c840b5220 2024-04-25T11:12:02.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.534+0000 7f9c7b400640 1 -- 172.21.15.7:0/113580245 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18808 (secure 0 0 0) 0x7f9c7c0801b0 con 0x7f9c840b5220 2024-04-25T11:12:02.535 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:02.535 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:11:03.036524+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:10:20.528603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T11:10:21.975446+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T11:10:25.509879+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.24","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:12:02.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 -- 172.21.15.7:0/113580245 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c640744b0 msgr2=0x7f9c640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:02.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c640744b0 0x7f9c640769c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c74006fd0 tx=0x7f9c74008040 comp rx=0 tx=0).stop 2024-04-25T11:12:02.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 -- 172.21.15.7:0/113580245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c840b5220 msgr2=0x7f9c840d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:02.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c840b5220 0x7f9c840d0690 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c034040 tx=0x7f9c7c004a40 comp rx=0 tx=0).stop 2024-04-25T11:12:02.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 -- 172.21.15.7:0/113580245 shutdown_connections 2024-04-25T11:12:02.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c640744b0 0x7f9c640769c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c840b5220 0x7f9c840d0690 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c840af420 0x7f9c84141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 --2- 172.21.15.7:0/113580245 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c840aed40 0x7f9c841430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 -- 172.21.15.7:0/113580245 >> 172.21.15.7:0/113580245 conn(0x7f9c840b3820 msgr2=0x7f9c840ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:02.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 -- 172.21.15.7:0/113580245 shutdown_connections 2024-04-25T11:12:02.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.537+0000 7f9c89000640 1 -- 172.21.15.7:0/113580245 wait complete. 2024-04-25T11:12:02.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:12:02.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.852+0000 7fdc5b000640 1 Processor -- start 2024-04-25T11:12:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.852+0000 7fdc5b000640 1 -- start start 2024-04-25T11:12:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.854+0000 7fdc5b000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc54141290 0x7fdc541416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.854+0000 7fdc5b000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc5406d610 0x7fdc5406da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.854+0000 7fdc5b000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc5406c570 con 0x7fdc5415eed0 2024-04-25T11:12:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.854+0000 7fdc5b000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5406c850 con 0x7fdc54141290 2024-04-25T11:12:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.854+0000 7fdc5b000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5406cb30 con 0x7fdc5406d610 2024-04-25T11:12:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.854+0000 7fdc59200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc54141290 0x7fdc541416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.855+0000 7fdc59200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc54141290 0x7fdc541416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34930/0 (socket says 172.21.15.7:34930) 2024-04-25T11:12:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.855+0000 7fdc59200640 1 -- 172.21.15.7:0/31740145 learned_addr learned my addr 172.21.15.7:0/31740145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.855+0000 7fdc5a600640 1 --2- 172.21.15.7:0/31740145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc5406d610 0x7fdc5406da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.855+0000 7fdc59200640 1 -- 172.21.15.7:0/31740145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc5406d610 msgr2=0x7fdc5406da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:02.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.855+0000 7fdc59200640 1 --2- 172.21.15.7:0/31740145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc5406d610 0x7fdc5406da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.855+0000 7fdc59200640 1 -- 172.21.15.7:0/31740145 >> v1:172.21.15.87:6789/0 conn(0x7fdc5415eed0 legacy=0x7fdc540cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:02.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.855+0000 7fdc59200640 1 -- 172.21.15.7:0/31740145 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5406ce10 con 0x7fdc54141290 2024-04-25T11:12:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.856+0000 7fdc59200640 1 --2- 172.21.15.7:0/31740145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc54141290 0x7fdc541416a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdc5406eed0 tx=0x7fdc4002a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7eb2eb362e51a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.857+0000 7fdc53e00640 1 -- 172.21.15.7:0/31740145 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40037070 con 0x7fdc54141290 2024-04-25T11:12:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.857+0000 7fdc53e00640 1 -- 172.21.15.7:0/31740145 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc40032440 con 0x7fdc54141290 2024-04-25T11:12:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.857+0000 7fdc53e00640 1 -- 172.21.15.7:0/31740145 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40032810 con 0x7fdc54141290 2024-04-25T11:12:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.857+0000 7fdc5b000640 1 -- 172.21.15.7:0/31740145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc54141290 msgr2=0x7fdc541416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:02.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.857+0000 7fdc5b000640 1 --2- 172.21.15.7:0/31740145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc54141290 0x7fdc541416a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdc5406eed0 tx=0x7fdc4002a1f0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.857+0000 7fdc5b000640 1 -- 172.21.15.7:0/31740145 shutdown_connections 2024-04-25T11:12:02.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.857+0000 7fdc5b000640 1 --2- 172.21.15.7:0/31740145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc5406d610 0x7fdc5406da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.857+0000 7fdc5b000640 1 --2- 172.21.15.7:0/31740145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc54141290 0x7fdc541416a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.857+0000 7fdc5b000640 1 -- 172.21.15.7:0/31740145 >> 172.21.15.7:0/31740145 conn(0x7fdc540d1540 msgr2=0x7fdc540d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:02.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.858+0000 7fdc5b000640 1 -- 172.21.15.7:0/31740145 shutdown_connections 2024-04-25T11:12:02.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.858+0000 7fdc5b000640 1 -- 172.21.15.7:0/31740145 wait complete. 2024-04-25T11:12:02.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.860+0000 7fdc5b000640 1 Processor -- start 2024-04-25T11:12:02.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.860+0000 7fdc5b000640 1 -- start start 2024-04-25T11:12:02.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.862+0000 7fdc5b000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc5406d610 0x7fdc541e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.862+0000 7fdc5b000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc54141290 0x7fdc541e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.862+0000 7fdc5b000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc5415eed0 0x7fdc54133b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.862+0000 7fdc5b000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc54140f20 con 0x7fdc54141290 2024-04-25T11:12:02.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.862+0000 7fdc5b000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdc54140da0 con 0x7fdc5415eed0 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.862+0000 7fdc5b000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdc540b4590 con 0x7fdc5406d610 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.862+0000 7fdc59200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc54141290 0x7fdc541e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.862+0000 7fdc59200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc54141290 0x7fdc541e7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34942/0 (socket says 172.21.15.7:34942) 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.862+0000 7fdc59200640 1 -- 172.21.15.7:0/4103342109 learned_addr learned my addr 172.21.15.7:0/4103342109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.862+0000 7fdc59c00640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc5406d610 0x7fdc541e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.863+0000 7fdc5a600640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc5415eed0 0x7fdc54133b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.863+0000 7fdc59200640 1 -- 172.21.15.7:0/4103342109 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc5406d610 msgr2=0x7fdc541e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.863+0000 7fdc59200640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc5406d610 0x7fdc541e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.863+0000 7fdc59200640 1 -- 172.21.15.7:0/4103342109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc5415eed0 msgr2=0x7fdc54133b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.863+0000 7fdc59200640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc5415eed0 0x7fdc54133b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.863+0000 7fdc59200640 1 -- 172.21.15.7:0/4103342109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc541e82e0 con 0x7fdc54141290 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.863+0000 7fdc59c00640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc5406d610 0x7fdc541e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:12:02.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.863+0000 7fdc59200640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc54141290 0x7fdc541e7da0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdc4002a230 tx=0x7fdc400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.864+0000 7fdc51600640 1 -- 172.21.15.7:0/4103342109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40041070 con 0x7fdc54141290 2024-04-25T11:12:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.864+0000 7fdc51600640 1 -- 172.21.15.7:0/4103342109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc4003c870 con 0x7fdc54141290 2024-04-25T11:12:02.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.864+0000 7fdc5b000640 1 -- 172.21.15.7:0/4103342109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc541340b0 con 0x7fdc54141290 2024-04-25T11:12:02.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.864+0000 7fdc51600640 1 -- 172.21.15.7:0/4103342109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40037040 con 0x7fdc54141290 2024-04-25T11:12:02.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.864+0000 7fdc5b000640 1 -- 172.21.15.7:0/4103342109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc541e6fe0 con 0x7fdc54141290 2024-04-25T11:12:02.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.865+0000 7fdc5b000640 1 -- 172.21.15.7:0/4103342109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc54050740 con 0x7fdc54141290 2024-04-25T11:12:02.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.866+0000 7fdc51600640 1 -- 172.21.15.7:0/4103342109 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdc40044070 con 0x7fdc54141290 2024-04-25T11:12:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.867+0000 7fdc51600640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc340744b0 0x7fdc340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.867+0000 7fdc51600640 1 -- 172.21.15.7:0/4103342109 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fdc400b6410 con 0x7fdc54141290 2024-04-25T11:12:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.868+0000 7fdc59c00640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc340744b0 0x7fdc340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.868+0000 7fdc59c00640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc340744b0 0x7fdc340769c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdc48006fd0 tx=0x7fdc48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:02.873+0000 7fdc51600640 1 -- 172.21.15.7:0/4103342109 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc40080680 con 0x7fdc54141290 2024-04-25T11:12:03.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.118+0000 7fdc5b000640 1 -- 172.21.15.7:0/4103342109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdc541e6a90 con 0x7fdc54141290 2024-04-25T11:12:03.120 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-04-25T11:12:03.120+0000 7fac34e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:12:03.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.123+0000 7fdc51600640 1 -- 172.21.15.7:0/4103342109 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fdc4007ff80 con 0x7fdc54141290 2024-04-25T11:12:03.124 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:12:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.127+0000 7fdc5b000640 1 -- 172.21.15.7:0/4103342109 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc340744b0 msgr2=0x7fdc340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.127+0000 7fdc5b000640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc340744b0 0x7fdc340769c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdc48006fd0 tx=0x7fdc48008040 comp rx=0 tx=0).stop 2024-04-25T11:12:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.127+0000 7fdc5b000640 1 -- 172.21.15.7:0/4103342109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc54141290 msgr2=0x7fdc541e7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:03.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.127+0000 7fdc5b000640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc54141290 0x7fdc541e7da0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdc4002a230 tx=0x7fdc400046c0 comp rx=0 tx=0).stop 2024-04-25T11:12:03.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.127+0000 7fdc5b000640 1 -- 172.21.15.7:0/4103342109 shutdown_connections 2024-04-25T11:12:03.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.127+0000 7fdc5b000640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc340744b0 0x7fdc340769c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:03.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.127+0000 7fdc5b000640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc5415eed0 0x7fdc54133b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:03.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.127+0000 7fdc5b000640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc54141290 0x7fdc541e7da0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:03.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.127+0000 7fdc5b000640 1 --2- 172.21.15.7:0/4103342109 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc5406d610 0x7fdc541e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:03.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.127+0000 7fdc5b000640 1 -- 172.21.15.7:0/4103342109 >> 172.21.15.7:0/4103342109 conn(0x7fdc540d1540 msgr2=0x7fdc540cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:03.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.127+0000 7fdc5b000640 1 -- 172.21.15.7:0/4103342109 shutdown_connections 2024-04-25T11:12:03.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.128+0000 7fdc5b000640 1 -- 172.21.15.7:0/4103342109 wait complete. 2024-04-25T11:12:03.158 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:12:03.166 INFO:tasks.ceph.mds.d.smithi087.stderr: -7298> 2024-04-25T11:10:28.009+0000 7fa8a3000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:12:03.331 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:12:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.438+0000 7ff85be00640 1 Processor -- start 2024-04-25T11:12:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.438+0000 7ff85be00640 1 -- start start 2024-04-25T11:12:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.441+0000 7ff85be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff85c15fea0 0x7ff85c0b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.441+0000 7ff85be00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff85c0d13c0 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.441+0000 7ff85be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff85c163f10 con 0x7ff85c0b7af0 2024-04-25T11:12:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.441+0000 7ff85be00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff85c0b9390 con 0x7ff85c15fea0 2024-04-25T11:12:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.441+0000 7ff85aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff85c0d0fb0 0x7ff85c0b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59854/0 (socket says 172.21.15.7:59854) 2024-04-25T11:12:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.441+0000 7ff85aa00640 1 -- 172.21.15.7:0/3134910375 learned_addr learned my addr 172.21.15.7:0/3134910375 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.441+0000 7ff85b400640 1 --2- 172.21.15.7:0/3134910375 >> v2:172.21.15.7:3300/0 conn(0x7ff85c15fea0 0x7ff85c0b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.441+0000 7ff859600640 1 -- 172.21.15.7:0/3134910375 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3269289221 0 0) 0x7ff85c0d13c0 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.441+0000 7ff859600640 1 -- 172.21.15.7:0/3134910375 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff840003520 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.442+0000 7ff85b400640 1 -- 172.21.15.7:0/3134910375 >> v1:172.21.15.87:6789/0 conn(0x7ff85c0b7af0 legacy=0x7ff85c0d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.442+0000 7ff85b400640 1 -- 172.21.15.7:0/3134910375 >> v1:172.21.15.7:6789/0 conn(0x7ff85c0d0fb0 legacy=0x7ff85c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.442+0000 7ff85b400640 1 -- 172.21.15.7:0/3134910375 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c1641f0 con 0x7ff85c15fea0 2024-04-25T11:12:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.443+0000 7ff85b400640 1 --2- 172.21.15.7:0/3134910375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff85c15fea0 0x7ff85c0b8e50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0b9510 tx=0x7ff85402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5396cbfe542c8a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.443+0000 7ff859600640 1 -- 172.21.15.7:0/3134910375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854037070 con 0x7ff85c15fea0 2024-04-25T11:12:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.443+0000 7ff859600640 1 -- 172.21.15.7:0/3134910375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff854032440 con 0x7ff85c15fea0 2024-04-25T11:12:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.443+0000 7ff859600640 1 -- 172.21.15.7:0/3134910375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854032810 con 0x7ff85c15fea0 2024-04-25T11:12:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.444+0000 7ff85be00640 1 -- 172.21.15.7:0/3134910375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff85c15fea0 msgr2=0x7ff85c0b8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.444+0000 7ff85be00640 1 --2- 172.21.15.7:0/3134910375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff85c15fea0 0x7ff85c0b8e50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0b9510 tx=0x7ff85402a180 comp rx=0 tx=0).stop 2024-04-25T11:12:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.444+0000 7ff85be00640 1 -- 172.21.15.7:0/3134910375 shutdown_connections 2024-04-25T11:12:03.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.444+0000 7ff85be00640 1 --2- 172.21.15.7:0/3134910375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff85c15fea0 0x7ff85c0b8e50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.444+0000 7ff85be00640 1 -- 172.21.15.7:0/3134910375 >> 172.21.15.7:0/3134910375 conn(0x7ff85c0d23e0 msgr2=0x7ff85c0d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.444+0000 7ff85be00640 1 -- 172.21.15.7:0/3134910375 shutdown_connections 2024-04-25T11:12:03.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.444+0000 7ff85be00640 1 -- 172.21.15.7:0/3134910375 wait complete. 2024-04-25T11:12:03.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.446+0000 7ff85be00640 1 Processor -- start 2024-04-25T11:12:03.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.447+0000 7ff85be00640 1 -- start start 2024-04-25T11:12:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.448+0000 7ff85be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff85c0d0fb0 0x7ff85c159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.448+0000 7ff85be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff85c15fea0 0x7ff85c156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.448+0000 7ff85be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff85c15be30 0x7ff85c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.448+0000 7ff85be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1602b0 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.448+0000 7ff85be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1647b0 con 0x7ff85c15fea0 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.448+0000 7ff85be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff85c0b4bc0 con 0x7ff85c15be30 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.449+0000 7ff85aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff85c0d0fb0 0x7ff85c159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.449+0000 7ff85aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff85c0d0fb0 0x7ff85c159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34958/0 (socket says 172.21.15.7:34958) 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.449+0000 7ff85aa00640 1 -- 172.21.15.7:0/4004503569 learned_addr learned my addr 172.21.15.7:0/4004503569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.449+0000 7ff85a000640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff85c15fea0 0x7ff85c156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.449+0000 7ff85b400640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff85c15be30 0x7ff85c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.449+0000 7ff85aa00640 1 -- 172.21.15.7:0/4004503569 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff85c15be30 msgr2=0x7ff85c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.449+0000 7ff85aa00640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff85c15be30 0x7ff85c15c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.449+0000 7ff85aa00640 1 -- 172.21.15.7:0/4004503569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff85c15fea0 msgr2=0x7ff85c156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.449+0000 7ff85aa00640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff85c15fea0 0x7ff85c156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.449+0000 7ff85aa00640 1 -- 172.21.15.7:0/4004503569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c15c780 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.449+0000 7ff85a000640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff85c15fea0 0x7ff85c156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.450+0000 7ff85aa00640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff85c0d0fb0 0x7ff85c159e40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00cbf0 tx=0x7ff84c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.450+0000 7ff852a00640 1 -- 172.21.15.7:0/4004503569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c017070 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.450+0000 7ff852a00640 1 -- 172.21.15.7:0/4004503569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84c00ad40 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.450+0000 7ff85be00640 1 -- 172.21.15.7:0/4004503569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c15a380 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.450+0000 7ff852a00640 1 -- 172.21.15.7:0/4004503569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c012620 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.450+0000 7ff85be00640 1 -- 172.21.15.7:0/4004503569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c0cc610 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.451+0000 7ff85be00640 1 -- 172.21.15.7:0/4004503569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85c050740 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.452+0000 7ff852a00640 1 -- 172.21.15.7:0/4004503569 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff84c01a070 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.453+0000 7ff852a00640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8380744b0 0x7ff8380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.454+0000 7ff852a00640 1 -- 172.21.15.7:0/4004503569 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6821+0+0 (secure 0 0 0) 0x7ff84c095420 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.454+0000 7ff85a000640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8380744b0 0x7ff8380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.454+0000 7ff85a000640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8380744b0 0x7ff8380769c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff85c15cc50 tx=0x7ff844009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:03.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.458+0000 7ff852a00640 1 -- 172.21.15.7:0/4004503569 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff85c15c780 con 0x7ff85c0d0fb0 2024-04-25T11:12:03.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:03.715+0000 7ff85be00640 1 -- 172.21.15.7:0/4004503569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff85c0d3d40 con 0x7ff85c0d0fb0 2024-04-25T11:12:04.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.128+0000 7ff852a00640 1 -- 172.21.15.7:0/4004503569 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7ff84c05f7e0 con 0x7ff85c0d0fb0 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.131+0000 7ff85be00640 1 -- 172.21.15.7:0/4004503569 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8380744b0 msgr2=0x7ff8380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.131+0000 7ff85be00640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8380744b0 0x7ff8380769c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff85c15cc50 tx=0x7ff844009040 comp rx=0 tx=0).stop 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.131+0000 7ff85be00640 1 -- 172.21.15.7:0/4004503569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff85c0d0fb0 msgr2=0x7ff85c159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.131+0000 7ff85be00640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff85c0d0fb0 0x7ff85c159e40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00cbf0 tx=0x7ff84c00a430 comp rx=0 tx=0).stop 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.131+0000 7ff85be00640 1 -- 172.21.15.7:0/4004503569 shutdown_connections 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.131+0000 7ff85be00640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8380744b0 0x7ff8380769c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.131+0000 7ff85be00640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff85c15be30 0x7ff85c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.131+0000 7ff85be00640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff85c15fea0 0x7ff85c156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.131+0000 7ff85be00640 1 --2- 172.21.15.7:0/4004503569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff85c0d0fb0 0x7ff85c159e40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.131+0000 7ff85be00640 1 -- 172.21.15.7:0/4004503569 >> 172.21.15.7:0/4004503569 conn(0x7ff85c0d23e0 msgr2=0x7ff85c0b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.132+0000 7ff85be00640 1 -- 172.21.15.7:0/4004503569 shutdown_connections 2024-04-25T11:12:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.132+0000 7ff85be00640 1 -- 172.21.15.7:0/4004503569 wait complete. 2024-04-25T11:12:04.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:12:04.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.461+0000 7f5207800640 1 Processor -- start 2024-04-25T11:12:04.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.461+0000 7f5207800640 1 -- start start 2024-04-25T11:12:04.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.464+0000 7f5207800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52000aec60 0x7f52000b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:04.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.464+0000 7f5207800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52000b0940 0x7f52000b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:04.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.464+0000 7f5207800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52000b1290 con 0x7f52000b48a0 2024-04-25T11:12:04.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.464+0000 7f5207800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f52000b4da0 con 0x7f52000aec60 2024-04-25T11:12:04.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.464+0000 7f5207800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f52000b5080 con 0x7f52000b0940 2024-04-25T11:12:04.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.464+0000 7f51ffe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52000aec60 0x7f52000b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:04.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.464+0000 7f51ffe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52000aec60 0x7f52000b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34974/0 (socket says 172.21.15.7:34974) 2024-04-25T11:12:04.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.464+0000 7f51ffe00640 1 -- 172.21.15.7:0/3054142343 learned_addr learned my addr 172.21.15.7:0/3054142343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:04.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.464+0000 7f5205800640 1 --2- 172.21.15.7:0/3054142343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52000b0940 0x7f52000b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:04.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.465+0000 7f51ffe00640 1 -- 172.21.15.7:0/3054142343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52000b0940 msgr2=0x7f52000b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:04.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.465+0000 7f51ffe00640 1 --2- 172.21.15.7:0/3054142343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52000b0940 0x7f52000b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:04.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.465+0000 7f51ffe00640 1 -- 172.21.15.7:0/3054142343 >> v1:172.21.15.87:6789/0 conn(0x7f52000b48a0 legacy=0x7f520015dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:04.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.465+0000 7f51ffe00640 1 -- 172.21.15.7:0/3054142343 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52000b5360 con 0x7f52000aec60 2024-04-25T11:12:04.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.467+0000 7f51ffe00640 1 --2- 172.21.15.7:0/3054142343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52000aec60 0x7f52000b5e40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009870 tx=0x7f51ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e481871d5598a1e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:04.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.468+0000 7f51ff400640 1 -- 172.21.15.7:0/3054142343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec037070 con 0x7f52000aec60 2024-04-25T11:12:04.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.468+0000 7f51ff400640 1 -- 172.21.15.7:0/3054142343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51ec032440 con 0x7f52000aec60 2024-04-25T11:12:04.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.468+0000 7f51ff400640 1 -- 172.21.15.7:0/3054142343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec032810 con 0x7f52000aec60 2024-04-25T11:12:04.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.469+0000 7f5207800640 1 -- 172.21.15.7:0/3054142343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52000aec60 msgr2=0x7f52000b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:04.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.469+0000 7f5207800640 1 --2- 172.21.15.7:0/3054142343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52000aec60 0x7f52000b5e40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009870 tx=0x7f51ec02a270 comp rx=0 tx=0).stop 2024-04-25T11:12:04.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.469+0000 7f5207800640 1 -- 172.21.15.7:0/3054142343 shutdown_connections 2024-04-25T11:12:04.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.469+0000 7f5207800640 1 --2- 172.21.15.7:0/3054142343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52000b0940 0x7f52000b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:04.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.469+0000 7f5207800640 1 --2- 172.21.15.7:0/3054142343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52000aec60 0x7f52000b5e40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:04.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.469+0000 7f5207800640 1 -- 172.21.15.7:0/3054142343 >> 172.21.15.7:0/3054142343 conn(0x7f5200073920 msgr2=0x7f5200073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:04.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.469+0000 7f5207800640 1 -- 172.21.15.7:0/3054142343 shutdown_connections 2024-04-25T11:12:04.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.469+0000 7f5207800640 1 -- 172.21.15.7:0/3054142343 wait complete. 2024-04-25T11:12:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.472+0000 7f5207800640 1 Processor -- start 2024-04-25T11:12:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.472+0000 7f5207800640 1 -- start start 2024-04-25T11:12:04.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.473+0000 7f5207800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52000b0940 0x7f5200160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.473+0000 7f5207800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52000b48a0 0x7f5200148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.473+0000 7f5207800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52001457c0 0x7f5200145bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.473+0000 7f5207800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52000b6380 con 0x7f52000b0940 2024-04-25T11:12:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.473+0000 7f5207800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f52000b5920 con 0x7f52001457c0 2024-04-25T11:12:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.473+0000 7f5207800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f52000b03b0 con 0x7f52000b48a0 2024-04-25T11:12:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.474+0000 7f5204e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52000b0940 0x7f5200160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.474+0000 7f5204e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52000b0940 0x7f5200160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34982/0 (socket says 172.21.15.7:34982) 2024-04-25T11:12:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.474+0000 7f5204e00640 1 -- 172.21.15.7:0/3040766818 learned_addr learned my addr 172.21.15.7:0/3040766818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:04.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.474+0000 7f5205800640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52001457c0 0x7f5200145bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:04.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.474+0000 7f51ffe00640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52000b48a0 0x7f5200148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:04.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.474+0000 7f5204e00640 1 -- 172.21.15.7:0/3040766818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52000b48a0 msgr2=0x7f5200148960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.474+0000 7f5204e00640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52000b48a0 0x7f5200148960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.474+0000 7f5204e00640 1 -- 172.21.15.7:0/3040766818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52001457c0 msgr2=0x7f5200145bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.474+0000 7f5204e00640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52001457c0 0x7f5200145bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.474+0000 7f5204e00640 1 -- 172.21.15.7:0/3040766818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5200146110 con 0x7f52000b0940 2024-04-25T11:12:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.475+0000 7f5204e00640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52000b0940 0x7f5200160e90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c440 tx=0x7f51f400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.475+0000 7f51fcc00640 1 -- 172.21.15.7:0/3040766818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f40052d0 con 0x7f52000b0940 2024-04-25T11:12:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.475+0000 7f51fcc00640 1 -- 172.21.15.7:0/3040766818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f4005470 con 0x7f52000b0940 2024-04-25T11:12:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.475+0000 7f51fcc00640 1 -- 172.21.15.7:0/3040766818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4007260 con 0x7f52000b0940 2024-04-25T11:12:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.475+0000 7f5207800640 1 -- 172.21.15.7:0/3040766818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52000b1510 con 0x7f52000b0940 2024-04-25T11:12:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.475+0000 7f5207800640 1 -- 172.21.15.7:0/3040766818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52000b1ad0 con 0x7f52000b0940 2024-04-25T11:12:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.476+0000 7f5207800640 1 -- 172.21.15.7:0/3040766818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f520006a100 con 0x7f52000b0940 2024-04-25T11:12:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.478+0000 7f51fcc00640 1 -- 172.21.15.7:0/3040766818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f51f4007400 con 0x7f52000b0940 2024-04-25T11:12:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.478+0000 7f51fcc00640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51e00744e0 0x7f51e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.479+0000 7f51fcc00640 1 -- 172.21.15.7:0/3040766818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6821+0+0 (secure 0 0 0) 0x7f51f4094b20 con 0x7f52000b0940 2024-04-25T11:12:04.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.479+0000 7f51ffe00640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51e00744e0 0x7f51e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:04.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.480+0000 7f51ffe00640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51e00744e0 0x7f51e00769f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f51ec034000 tx=0x7f51ec02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.484+0000 7f51fcc00640 1 -- 172.21.15.7:0/3040766818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5200146110 con 0x7f52000b0940 2024-04-25T11:12:04.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:04.691+0000 7f5207800640 1 -- 172.21.15.7:0/3040766818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5200159db0 con 0x7f52000b0940 2024-04-25T11:12:05.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.450+0000 7f51fcc00640 1 -- 172.21.15.7:0/3040766818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7f52000b1290 con 0x7f52000b0940 2024-04-25T11:12:05.451 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:12:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.455+0000 7f5207800640 1 -- 172.21.15.7:0/3040766818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51e00744e0 msgr2=0x7f51e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.455+0000 7f5207800640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51e00744e0 0x7f51e00769f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f51ec034000 tx=0x7f51ec02e000 comp rx=0 tx=0).stop 2024-04-25T11:12:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.455+0000 7f5207800640 1 -- 172.21.15.7:0/3040766818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52000b0940 msgr2=0x7f5200160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.455+0000 7f5207800640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52000b0940 0x7f5200160e90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c440 tx=0x7f51f400c900 comp rx=0 tx=0).stop 2024-04-25T11:12:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.455+0000 7f5207800640 1 -- 172.21.15.7:0/3040766818 shutdown_connections 2024-04-25T11:12:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.455+0000 7f5207800640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51e00744e0 0x7f51e00769f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.455+0000 7f5207800640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52001457c0 0x7f5200145bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.455+0000 7f5207800640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52000b48a0 0x7f5200148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.455+0000 7f5207800640 1 --2- 172.21.15.7:0/3040766818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52000b0940 0x7f5200160e90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.455+0000 7f5207800640 1 -- 172.21.15.7:0/3040766818 >> 172.21.15.7:0/3040766818 conn(0x7f5200073920 msgr2=0x7f520006d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.456+0000 7f5207800640 1 -- 172.21.15.7:0/3040766818 shutdown_connections 2024-04-25T11:12:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.456+0000 7f5207800640 1 -- 172.21.15.7:0/3040766818 wait complete. 2024-04-25T11:12:05.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:12:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.758+0000 7f73b3000640 1 Processor -- start 2024-04-25T11:12:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.758+0000 7f73b3000640 1 -- start start 2024-04-25T11:12:05.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.760+0000 7f73b3000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac1440a0 0x7f73ac15f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:05.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.760+0000 7f73b3000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f73ac06dc90 0x7f73ac0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:05.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.760+0000 7f73b3000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73ac15e620 con 0x7f73ac0cf370 2024-04-25T11:12:05.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.760+0000 7f73b3000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f73ac15e900 con 0x7f73ac06dc90 2024-04-25T11:12:05.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.760+0000 7f73b3000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f73ac15ebe0 con 0x7f73ac1440a0 2024-04-25T11:12:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.761+0000 7f73b1c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac1440a0 0x7f73ac15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.761+0000 7f73b2600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f73ac06dc90 0x7f73ac0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.761+0000 7f73b1c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac1440a0 0x7f73ac15f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:45454/0 (socket says 172.21.15.7:45454) 2024-04-25T11:12:05.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.761+0000 7f73b1c00640 1 -- 172.21.15.7:0/63411678 learned_addr learned my addr 172.21.15.7:0/63411678 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.762+0000 7f73b1c00640 1 -- 172.21.15.7:0/63411678 >> v2:172.21.15.87:3300/0 conn(0x7f73ac06dc90 msgr2=0x7f73ac0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.762+0000 7f73b1c00640 1 --2- 172.21.15.7:0/63411678 >> v2:172.21.15.87:3300/0 conn(0x7f73ac06dc90 0x7f73ac0cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.762+0000 7f73b1c00640 1 -- 172.21.15.7:0/63411678 >> v1:172.21.15.87:6789/0 conn(0x7f73ac0cf370 legacy=0x7f73ac06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.762+0000 7f73b1c00640 1 -- 172.21.15.7:0/63411678 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac15eec0 con 0x7f73ac1440a0 2024-04-25T11:12:05.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.763+0000 7f73b1c00640 1 --2- 172.21.15.7:0/63411678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac1440a0 0x7f73ac15f230 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009870 tx=0x7f73a002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e45ad29d7c0b3666 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.764+0000 7f73abe00640 1 -- 172.21.15.7:0/63411678 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0037070 con 0x7f73ac1440a0 2024-04-25T11:12:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.764+0000 7f73abe00640 1 -- 172.21.15.7:0/63411678 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a0032440 con 0x7f73ac1440a0 2024-04-25T11:12:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.764+0000 7f73abe00640 1 -- 172.21.15.7:0/63411678 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0032810 con 0x7f73ac1440a0 2024-04-25T11:12:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.765+0000 7f73b3000640 1 -- 172.21.15.7:0/63411678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac1440a0 msgr2=0x7f73ac15f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.765+0000 7f73b3000640 1 --2- 172.21.15.7:0/63411678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac1440a0 0x7f73ac15f230 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009870 tx=0x7f73a002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:12:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.765+0000 7f73b3000640 1 -- 172.21.15.7:0/63411678 shutdown_connections 2024-04-25T11:12:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.765+0000 7f73b3000640 1 --2- 172.21.15.7:0/63411678 >> v2:172.21.15.87:3300/0 conn(0x7f73ac06dc90 0x7f73ac0cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.765+0000 7f73b3000640 1 --2- 172.21.15.7:0/63411678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac1440a0 0x7f73ac15f230 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.765+0000 7f73b3000640 1 -- 172.21.15.7:0/63411678 >> 172.21.15.7:0/63411678 conn(0x7f73ac073160 msgr2=0x7f73ac073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.765+0000 7f73b3000640 1 -- 172.21.15.7:0/63411678 shutdown_connections 2024-04-25T11:12:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.765+0000 7f73b3000640 1 -- 172.21.15.7:0/63411678 wait complete. 2024-04-25T11:12:05.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.767+0000 7f73b3000640 1 Processor -- start 2024-04-25T11:12:05.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.767+0000 7f73b3000640 1 -- start start 2024-04-25T11:12:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.769+0000 7f73b3000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac0cf370 0x7f73ac1383c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.769+0000 7f73b3000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73ac1440a0 0x7f73ac1e7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.769+0000 7f73b3000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73ac1f9990 0x7f73ac1f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.769+0000 7f73b3000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac15f770 con 0x7f73ac1440a0 2024-04-25T11:12:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.769+0000 7f73b3000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac15e200 con 0x7f73ac1f9990 2024-04-25T11:12:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.769+0000 7f73b3000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f73ac133ec0 con 0x7f73ac0cf370 2024-04-25T11:12:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.769+0000 7f73b1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73ac1440a0 0x7f73ac1e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.769+0000 7f73b1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73ac1440a0 0x7f73ac1e7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34990/0 (socket says 172.21.15.7:34990) 2024-04-25T11:12:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.769+0000 7f73b1200640 1 -- 172.21.15.7:0/1209566969 learned_addr learned my addr 172.21.15.7:0/1209566969 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.770+0000 7f73b2600640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73ac1f9990 0x7f73ac1f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.770+0000 7f73b1c00640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac0cf370 0x7f73ac1383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.770+0000 7f73b1200640 1 -- 172.21.15.7:0/1209566969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac0cf370 msgr2=0x7f73ac1383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.770+0000 7f73b1200640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac0cf370 0x7f73ac1383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.770+0000 7f73b1200640 1 -- 172.21.15.7:0/1209566969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73ac1f9990 msgr2=0x7f73ac1f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.770+0000 7f73b1200640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73ac1f9990 0x7f73ac1f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.770+0000 7f73b1200640 1 -- 172.21.15.7:0/1209566969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac1fa2e0 con 0x7f73ac1440a0 2024-04-25T11:12:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.770+0000 7f73b1c00640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac0cf370 0x7f73ac1383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:12:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.770+0000 7f73b1200640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73ac1440a0 0x7f73ac1e7490 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f739800c3f0 tx=0x7f739800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.771+0000 7f73a9600640 1 -- 172.21.15.7:0/1209566969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73980052d0 con 0x7f73ac1440a0 2024-04-25T11:12:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.771+0000 7f73a9600640 1 -- 172.21.15.7:0/1209566969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7398005490 con 0x7f73ac1440a0 2024-04-25T11:12:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.771+0000 7f73a9600640 1 -- 172.21.15.7:0/1209566969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7398007260 con 0x7f73ac1440a0 2024-04-25T11:12:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.771+0000 7f73b3000640 1 -- 172.21.15.7:0/1209566969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac138e20 con 0x7f73ac1440a0 2024-04-25T11:12:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.771+0000 7f73b3000640 1 -- 172.21.15.7:0/1209566969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac1e8000 con 0x7f73ac1440a0 2024-04-25T11:12:05.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.772+0000 7f73b3000640 1 -- 172.21.15.7:0/1209566969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ac15f580 con 0x7f73ac1440a0 2024-04-25T11:12:05.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.773+0000 7f73a9600640 1 -- 172.21.15.7:0/1209566969 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7398007400 con 0x7f73ac1440a0 2024-04-25T11:12:05.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.774+0000 7f73a9600640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f738c0744b0 0x7f738c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:05.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.774+0000 7f73a9600640 1 -- 172.21.15.7:0/1209566969 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6383+0+0 (secure 0 0 0) 0x7f7398094970 con 0x7f73ac1440a0 2024-04-25T11:12:05.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.775+0000 7f73b1c00640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f738c0744b0 0x7f738c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:05.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.775+0000 7f73b1c00640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f738c0744b0 0x7f738c0769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f73a0034000 tx=0x7f73a002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:05.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.782+0000 7f73a9600640 1 -- 172.21.15.7:0/1209566969 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f73ac1fa2e0 con 0x7f73ac1440a0 2024-04-25T11:12:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:05.989+0000 7f73b3000640 1 -- 172.21.15.7:0/1209566969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f73ac1e7d90 con 0x7f73ac1440a0 2024-04-25T11:12:06.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.477+0000 7f73a9600640 1 -- 172.21.15.7:0/1209566969 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7f739805e770 con 0x7f73ac1440a0 2024-04-25T11:12:06.478 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-25T11:12:06.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.481+0000 7f73b3000640 1 -- 172.21.15.7:0/1209566969 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f738c0744b0 msgr2=0x7f738c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:06.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.481+0000 7f73b3000640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f738c0744b0 0x7f738c0769c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f73a0034000 tx=0x7f73a002e000 comp rx=0 tx=0).stop 2024-04-25T11:12:06.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.481+0000 7f73b3000640 1 -- 172.21.15.7:0/1209566969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73ac1440a0 msgr2=0x7f73ac1e7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:06.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.481+0000 7f73b3000640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73ac1440a0 0x7f73ac1e7490 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f739800c3f0 tx=0x7f739800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:12:06.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.481+0000 7f73b3000640 1 -- 172.21.15.7:0/1209566969 shutdown_connections 2024-04-25T11:12:06.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.481+0000 7f73b3000640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f738c0744b0 0x7f738c0769c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:06.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.481+0000 7f73b3000640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73ac1f9990 0x7f73ac1f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:06.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.481+0000 7f73b3000640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73ac1440a0 0x7f73ac1e7490 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:06.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.481+0000 7f73b3000640 1 --2- 172.21.15.7:0/1209566969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73ac0cf370 0x7f73ac1383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:06.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.481+0000 7f73b3000640 1 -- 172.21.15.7:0/1209566969 >> 172.21.15.7:0/1209566969 conn(0x7f73ac073160 msgr2=0x7f73ac06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:06.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.481+0000 7f73b3000640 1 -- 172.21.15.7:0/1209566969 shutdown_connections 2024-04-25T11:12:06.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.482+0000 7f73b3000640 1 -- 172.21.15.7:0/1209566969 wait complete. 2024-04-25T11:12:06.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T11:12:06.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.782+0000 7ff88b000640 1 Processor -- start 2024-04-25T11:12:06.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.783+0000 7ff88b000640 1 -- start start 2024-04-25T11:12:06.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.785+0000 7ff88b000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 0x7ff8840b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:06.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.785+0000 7ff88b000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff8840b60b0 0x7ff88406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.785+0000 7ff88b000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88406ded0 con 0x7ff88406dac0 2024-04-25T11:12:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.785+0000 7ff88b000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff8840b4630 con 0x7ff8840b60b0 2024-04-25T11:12:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.785+0000 7ff88b000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff8840b48d0 con 0x7ff8841610d0 2024-04-25T11:12:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.786+0000 7ff889c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 0x7ff8840b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.786+0000 7ff889200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff8840b60b0 0x7ff88406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.786+0000 7ff889c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 0x7ff8840b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:45472/0 (socket says 172.21.15.7:45472) 2024-04-25T11:12:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.786+0000 7ff889c00640 1 -- 172.21.15.7:0/197921410 learned_addr learned my addr 172.21.15.7:0/197921410 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.787+0000 7ff889c00640 1 -- 172.21.15.7:0/197921410 >> v2:172.21.15.87:3300/0 conn(0x7ff8840b60b0 msgr2=0x7ff88406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.787+0000 7ff889c00640 1 --2- 172.21.15.7:0/197921410 >> v2:172.21.15.87:3300/0 conn(0x7ff8840b60b0 0x7ff88406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.787+0000 7ff889c00640 1 -- 172.21.15.7:0/197921410 >> v1:172.21.15.87:6789/0 conn(0x7ff88406dac0 legacy=0x7ff884161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.787+0000 7ff889c00640 1 -- 172.21.15.7:0/197921410 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8840b4bb0 con 0x7ff8841610d0 2024-04-25T11:12:06.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.788+0000 7ff889c00640 1 --2- 172.21.15.7:0/197921410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 0x7ff8840b5b70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff878009870 tx=0x7ff87802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8592f70ed0ea2a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:06.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.789+0000 7ff883e00640 1 -- 172.21.15.7:0/197921410 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878037070 con 0x7ff8841610d0 2024-04-25T11:12:06.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.789+0000 7ff883e00640 1 -- 172.21.15.7:0/197921410 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff878032440 con 0x7ff8841610d0 2024-04-25T11:12:06.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.789+0000 7ff883e00640 1 -- 172.21.15.7:0/197921410 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878032810 con 0x7ff8841610d0 2024-04-25T11:12:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.790+0000 7ff88b000640 1 -- 172.21.15.7:0/197921410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 msgr2=0x7ff8840b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.790+0000 7ff88b000640 1 --2- 172.21.15.7:0/197921410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 0x7ff8840b5b70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff878009870 tx=0x7ff87802a250 comp rx=0 tx=0).stop 2024-04-25T11:12:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.790+0000 7ff88b000640 1 -- 172.21.15.7:0/197921410 shutdown_connections 2024-04-25T11:12:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.790+0000 7ff88b000640 1 --2- 172.21.15.7:0/197921410 >> v2:172.21.15.87:3300/0 conn(0x7ff8840b60b0 0x7ff88406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.790+0000 7ff88b000640 1 --2- 172.21.15.7:0/197921410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 0x7ff8840b5b70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.790+0000 7ff88b000640 1 -- 172.21.15.7:0/197921410 >> 172.21.15.7:0/197921410 conn(0x7ff884073160 msgr2=0x7ff884073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.790+0000 7ff88b000640 1 -- 172.21.15.7:0/197921410 shutdown_connections 2024-04-25T11:12:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.790+0000 7ff88b000640 1 -- 172.21.15.7:0/197921410 wait complete. 2024-04-25T11:12:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.794+0000 7ff88b000640 1 Processor -- start 2024-04-25T11:12:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.794+0000 7ff88b000640 1 -- start start 2024-04-25T11:12:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.796+0000 7ff88b000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88406dac0 0x7ff884155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.796+0000 7ff88b000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8840b60b0 0x7ff884150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.796+0000 7ff88b000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 0x7ff8841f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.796+0000 7ff88b000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8841614e0 con 0x7ff88406dac0 2024-04-25T11:12:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.796+0000 7ff88b000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff8840b5170 con 0x7ff8840b60b0 2024-04-25T11:12:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.796+0000 7ff88b000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff88415f0d0 con 0x7ff8841610d0 2024-04-25T11:12:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.797+0000 7ff889c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88406dac0 0x7ff884155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.797+0000 7ff889c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88406dac0 0x7ff884155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34998/0 (socket says 172.21.15.7:34998) 2024-04-25T11:12:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.797+0000 7ff889c00640 1 -- 172.21.15.7:0/560782343 learned_addr learned my addr 172.21.15.7:0/560782343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.797+0000 7ff889200640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8840b60b0 0x7ff884150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.797+0000 7ff88a600640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 0x7ff8841f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.797+0000 7ff889c00640 1 -- 172.21.15.7:0/560782343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 msgr2=0x7ff8841f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.797+0000 7ff889c00640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 0x7ff8841f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.797+0000 7ff889c00640 1 -- 172.21.15.7:0/560782343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8840b60b0 msgr2=0x7ff884150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.797+0000 7ff889c00640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8840b60b0 0x7ff884150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.797+0000 7ff889c00640 1 -- 172.21.15.7:0/560782343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8841561a0 con 0x7ff88406dac0 2024-04-25T11:12:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.797+0000 7ff889200640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8840b60b0 0x7ff884150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:12:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.798+0000 7ff889c00640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88406dac0 0x7ff884155830 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff878009990 tx=0x7ff8780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:06.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.798+0000 7ff881600640 1 -- 172.21.15.7:0/560782343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878042020 con 0x7ff88406dac0 2024-04-25T11:12:06.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.798+0000 7ff881600640 1 -- 172.21.15.7:0/560782343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87803d750 con 0x7ff88406dac0 2024-04-25T11:12:06.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.798+0000 7ff881600640 1 -- 172.21.15.7:0/560782343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff878037040 con 0x7ff88406dac0 2024-04-25T11:12:06.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.798+0000 7ff88b000640 1 -- 172.21.15.7:0/560782343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff884156480 con 0x7ff88406dac0 2024-04-25T11:12:06.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.798+0000 7ff88b000640 1 -- 172.21.15.7:0/560782343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff884150e90 con 0x7ff88406dac0 2024-04-25T11:12:06.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.799+0000 7ff88b000640 1 -- 172.21.15.7:0/560782343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff884050740 con 0x7ff88406dac0 2024-04-25T11:12:06.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.801+0000 7ff881600640 1 -- 172.21.15.7:0/560782343 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff878045020 con 0x7ff88406dac0 2024-04-25T11:12:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.802+0000 7ff881600640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744e0 0x7ff8640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.802+0000 7ff881600640 1 -- 172.21.15.7:0/560782343 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5853+0+0 (secure 0 0 0) 0x7ff8780b5dc0 con 0x7ff88406dac0 2024-04-25T11:12:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.802+0000 7ff889200640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744e0 0x7ff8640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.803+0000 7ff889200640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744e0 0x7ff8640769f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff88415ae80 tx=0x7ff870008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:06.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:06.810+0000 7ff881600640 1 -- 172.21.15.7:0/560782343 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff87804c030 con 0x7ff88406dac0 2024-04-25T11:12:07.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.013+0000 7ff88b000640 1 -- 172.21.15.7:0/560782343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff8841555f0 con 0x7ff88406dac0 2024-04-25T11:12:07.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.481+0000 7ff881600640 1 -- 172.21.15.7:0/560782343 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7ff8780804c0 con 0x7ff88406dac0 2024-04-25T11:12:07.481 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data_ec' removed 2024-04-25T11:12:07.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.485+0000 7ff88b000640 1 -- 172.21.15.7:0/560782343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744e0 msgr2=0x7ff8640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.485+0000 7ff88b000640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744e0 0x7ff8640769f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff88415ae80 tx=0x7ff870008040 comp rx=0 tx=0).stop 2024-04-25T11:12:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.485+0000 7ff88b000640 1 -- 172.21.15.7:0/560782343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88406dac0 msgr2=0x7ff884155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.485+0000 7ff88b000640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88406dac0 0x7ff884155830 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff878009990 tx=0x7ff8780046c0 comp rx=0 tx=0).stop 2024-04-25T11:12:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.486+0000 7ff88b000640 1 -- 172.21.15.7:0/560782343 shutdown_connections 2024-04-25T11:12:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.486+0000 7ff88b000640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744e0 0x7ff8640769f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:07.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.486+0000 7ff88b000640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8841610d0 0x7ff8841f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:07.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.486+0000 7ff88b000640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8840b60b0 0x7ff884150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:07.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.486+0000 7ff88b000640 1 --2- 172.21.15.7:0/560782343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88406dac0 0x7ff884155830 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:07.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.486+0000 7ff88b000640 1 -- 172.21.15.7:0/560782343 >> 172.21.15.7:0/560782343 conn(0x7ff884073160 msgr2=0x7ff884073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:07.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.486+0000 7ff88b000640 1 -- 172.21.15.7:0/560782343 shutdown_connections 2024-04-25T11:12:07.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.486+0000 7ff88b000640 1 -- 172.21.15.7:0/560782343 wait complete. 2024-04-25T11:12:07.529 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:12:07.529 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:12:07.529 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:12:07.529 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:12:07.529 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:12:07.529 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:12:07.530 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:12:07.530 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:12:07.530 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:12:07.530 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:12:07.530 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:12:07.530 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:12:07.530 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:12:07.531 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:12:07.531 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:12:07.531 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:12:07.554 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:12:07.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:12:07.557 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:12:07.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:12:07.560 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:12:07.560 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:12:07.585 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:12:07.585 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:12:07.601 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:12:07.602 DEBUG:teuthology.parallel:result is None 2024-04-25T11:12:07.605 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:12:07.605 DEBUG:teuthology.parallel:result is None 2024-04-25T11:12:07.633 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:12:07.633 DEBUG:teuthology.parallel:result is None 2024-04-25T11:12:07.637 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:12:07.637 DEBUG:teuthology.parallel:result is None 2024-04-25T11:12:07.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:12:07.831 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:12:07.832 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:12:07.849 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:12:07.852 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:12:07.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.890+0000 7f2a16600640 1 Processor -- start 2024-04-25T11:12:07.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.890+0000 7f2a16600640 1 -- start start 2024-04-25T11:12:07.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.892+0000 7f2a16600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2a100b5190 0x7f2a100d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:07.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.892+0000 7f2a16600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a1006d580 0x7f2a1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.892+0000 7f2a16600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a100b55a0 con 0x7f2a10160b50 2024-04-25T11:12:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.892+0000 7f2a16600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2a1006ded0 con 0x7f2a100b5190 2024-04-25T11:12:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.892+0000 7f2a16600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1015d560 con 0x7f2a1006d580 2024-04-25T11:12:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.893+0000 7f2a0fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2a10160b50 0x7f2a10160f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:52594/0 (socket says 172.21.15.7:52594) 2024-04-25T11:12:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.893+0000 7f2a0fe00640 1 -- 172.21.15.7:0/446159156 learned_addr learned my addr 172.21.15.7:0/446159156 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.893+0000 7f2a07e00640 1 --2- 172.21.15.7:0/446159156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a1006d580 0x7f2a1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.893+0000 7f2a0f400640 1 --2- 172.21.15.7:0/446159156 >> v2:172.21.15.87:3300/0 conn(0x7f2a100b5190 0x7f2a100d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.893+0000 7f2a0ea00640 1 -- 172.21.15.7:0/446159156 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3661935478 0 0) 0x7f2a100b55a0 con 0x7f2a10160b50 2024-04-25T11:12:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.893+0000 7f2a0ea00640 1 -- 172.21.15.7:0/446159156 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29f4003520 con 0x7f2a10160b50 2024-04-25T11:12:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.894+0000 7f2a07e00640 1 -- 172.21.15.7:0/446159156 >> v2:172.21.15.87:3300/0 conn(0x7f2a100b5190 msgr2=0x7f2a100d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.894+0000 7f2a07e00640 1 --2- 172.21.15.7:0/446159156 >> v2:172.21.15.87:3300/0 conn(0x7f2a100b5190 0x7f2a100d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.894+0000 7f2a07e00640 1 -- 172.21.15.7:0/446159156 >> v1:172.21.15.87:6789/0 conn(0x7f2a10160b50 legacy=0x7f2a10160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.894+0000 7f2a07e00640 1 -- 172.21.15.7:0/446159156 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1015d820 con 0x7f2a1006d580 2024-04-25T11:12:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.894+0000 7f2a0f400640 1 --2- 172.21.15.7:0/446159156 >> v2:172.21.15.87:3300/0 conn(0x7f2a100b5190 0x7f2a100d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:12:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.895+0000 7f2a07e00640 1 --2- 172.21.15.7:0/446159156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a1006d580 0x7f2a1006d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29f80097a0 tx=0x7f29f802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f1aefc5067ce8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:07.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.895+0000 7f2a0ea00640 1 -- 172.21.15.7:0/446159156 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8037070 con 0x7f2a1006d580 2024-04-25T11:12:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.895+0000 7f2a0ea00640 1 -- 172.21.15.7:0/446159156 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f8032440 con 0x7f2a1006d580 2024-04-25T11:12:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.895+0000 7f2a0ea00640 1 -- 172.21.15.7:0/446159156 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f80327f0 con 0x7f2a1006d580 2024-04-25T11:12:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.896+0000 7f2a16600640 1 -- 172.21.15.7:0/446159156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a1006d580 msgr2=0x7f2a1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.896+0000 7f2a16600640 1 --2- 172.21.15.7:0/446159156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a1006d580 0x7f2a1006d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29f80097a0 tx=0x7f29f802a180 comp rx=0 tx=0).stop 2024-04-25T11:12:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.896+0000 7f2a16600640 1 -- 172.21.15.7:0/446159156 shutdown_connections 2024-04-25T11:12:07.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.896+0000 7f2a16600640 1 --2- 172.21.15.7:0/446159156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a1006d580 0x7f2a1006d990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.896+0000 7f2a16600640 1 --2- 172.21.15.7:0/446159156 >> v2:172.21.15.87:3300/0 conn(0x7f2a100b5190 0x7f2a100d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.896+0000 7f2a16600640 1 -- 172.21.15.7:0/446159156 >> 172.21.15.7:0/446159156 conn(0x7f2a100732b0 msgr2=0x7f2a100736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.896+0000 7f2a16600640 1 -- 172.21.15.7:0/446159156 shutdown_connections 2024-04-25T11:12:07.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.896+0000 7f2a16600640 1 -- 172.21.15.7:0/446159156 wait complete. 2024-04-25T11:12:07.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.899+0000 7f2a16600640 1 Processor -- start 2024-04-25T11:12:07.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.899+0000 7f2a16600640 1 -- start start 2024-04-25T11:12:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.900+0000 7f2a16600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a1006d580 0x7f2a1013e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.900+0000 7f2a16600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a100b5190 0x7f2a1009dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.900+0000 7f2a16600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10160b50 0x7f2a1009e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.900+0000 7f2a16600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1015e0c0 con 0x7f2a1006d580 2024-04-25T11:12:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.900+0000 7f2a16600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1015dde0 con 0x7f2a10160b50 2024-04-25T11:12:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.900+0000 7f2a16600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2a100d1530 con 0x7f2a100b5190 2024-04-25T11:12:07.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.901+0000 7f2a0f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a1006d580 0x7f2a1013e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.901+0000 7f2a0f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a1006d580 0x7f2a1013e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35050/0 (socket says 172.21.15.7:35050) 2024-04-25T11:12:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.901+0000 7f2a0f400640 1 -- 172.21.15.7:0/3448803804 learned_addr learned my addr 172.21.15.7:0/3448803804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.901+0000 7f2a0fe00640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10160b50 0x7f2a1009e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.901+0000 7f2a07e00640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a100b5190 0x7f2a1009dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.901+0000 7f2a0f400640 1 -- 172.21.15.7:0/3448803804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a100b5190 msgr2=0x7f2a1009dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.901+0000 7f2a0f400640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a100b5190 0x7f2a1009dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.901+0000 7f2a0f400640 1 -- 172.21.15.7:0/3448803804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10160b50 msgr2=0x7f2a1009e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.901+0000 7f2a0f400640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10160b50 0x7f2a1009e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:07.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.901+0000 7f2a0f400640 1 -- 172.21.15.7:0/3448803804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a100aead0 con 0x7f2a1006d580 2024-04-25T11:12:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.902+0000 7f2a0f400640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a1006d580 0x7f2a1013e000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000e910 tx=0x7f2a0000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.902+0000 7f2a07400640 1 -- 172.21.15.7:0/3448803804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00017070 con 0x7f2a1006d580 2024-04-25T11:12:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.902+0000 7f2a07400640 1 -- 172.21.15.7:0/3448803804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0000ca00 con 0x7f2a1006d580 2024-04-25T11:12:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.902+0000 7f2a07400640 1 -- 172.21.15.7:0/3448803804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000cdb0 con 0x7f2a1006d580 2024-04-25T11:12:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.902+0000 7f2a16600640 1 -- 172.21.15.7:0/3448803804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a100aedb0 con 0x7f2a1006d580 2024-04-25T11:12:07.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.902+0000 7f2a16600640 1 -- 172.21.15.7:0/3448803804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a100af370 con 0x7f2a1006d580 2024-04-25T11:12:07.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.903+0000 7f2a16600640 1 -- 172.21.15.7:0/3448803804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a10050740 con 0x7f2a1006d580 2024-04-25T11:12:07.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.905+0000 7f2a07400640 1 -- 172.21.15.7:0/3448803804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2a0001a070 con 0x7f2a1006d580 2024-04-25T11:12:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.906+0000 7f2a07400640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29ec0744b0 0x7f29ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.906+0000 7f2a07400640 1 -- 172.21.15.7:0/3448803804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7f2a00094f70 con 0x7f2a1006d580 2024-04-25T11:12:07.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.906+0000 7f2a07e00640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29ec0744b0 0x7f29ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:07.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.907+0000 7f2a07e00640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29ec0744b0 0x7f29ec0769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29f8034000 tx=0x7f29f802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:07.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:07.911+0000 7f2a07400640 1 -- 172.21.15.7:0/3448803804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a0005f860 con 0x7f2a1006d580 2024-04-25T11:12:08.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.118+0000 7f2a16600640 1 -- 172.21.15.7:0/3448803804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2a100d0d10 con 0x7f2a1006d580 2024-04-25T11:12:08.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.523+0000 7f2a07400640 1 -- 172.21.15.7:0/3448803804 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7f2a0005f160 con 0x7f2a1006d580 2024-04-25T11:12:08.523 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-04-25T11:12:08.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.526+0000 7f2a16600640 1 -- 172.21.15.7:0/3448803804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29ec0744b0 msgr2=0x7f29ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:08.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.526+0000 7f2a16600640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29ec0744b0 0x7f29ec0769c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29f8034000 tx=0x7f29f802e000 comp rx=0 tx=0).stop 2024-04-25T11:12:08.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.527+0000 7f2a16600640 1 -- 172.21.15.7:0/3448803804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a1006d580 msgr2=0x7f2a1013e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:08.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.527+0000 7f2a16600640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a1006d580 0x7f2a1013e000 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000e910 tx=0x7f2a0000edd0 comp rx=0 tx=0).stop 2024-04-25T11:12:08.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.527+0000 7f2a16600640 1 -- 172.21.15.7:0/3448803804 shutdown_connections 2024-04-25T11:12:08.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.527+0000 7f2a16600640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29ec0744b0 0x7f29ec0769c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:08.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.527+0000 7f2a16600640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10160b50 0x7f2a1009e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:08.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.527+0000 7f2a16600640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a100b5190 0x7f2a1009dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:08.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.527+0000 7f2a16600640 1 --2- 172.21.15.7:0/3448803804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a1006d580 0x7f2a1013e000 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:08.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.527+0000 7f2a16600640 1 -- 172.21.15.7:0/3448803804 >> 172.21.15.7:0/3448803804 conn(0x7f2a100732b0 msgr2=0x7f2a100b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:08.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.528+0000 7f2a16600640 1 -- 172.21.15.7:0/3448803804 shutdown_connections 2024-04-25T11:12:08.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.528+0000 7f2a16600640 1 -- 172.21.15.7:0/3448803804 wait complete. 2024-04-25T11:12:08.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:12:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.838+0000 7f3b57000640 1 Processor -- start 2024-04-25T11:12:08.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.838+0000 7f3b57000640 1 -- start start 2024-04-25T11:12:08.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.840+0000 7f3b57000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3b5006d580 0x7f3b5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:08.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.840+0000 7f3b57000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b5015e700 con 0x7f3b5015baa0 2024-04-25T11:12:08.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.840+0000 7f3b57000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b5006ded0 con 0x7f3b5015e2f0 2024-04-25T11:12:08.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.840+0000 7f3b57000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3b5015e980 con 0x7f3b5006d580 2024-04-25T11:12:08.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.840+0000 7f3b55c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3b5015baa0 0x7f3b50070640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59872/0 (socket says 172.21.15.7:59872) 2024-04-25T11:12:08.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.840+0000 7f3b55c00640 1 -- 172.21.15.7:0/1339441610 learned_addr learned my addr 172.21.15.7:0/1339441610 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:08.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.841+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519957163 0 0) 0x7f3b44001110 con 0x7f3b5015baa0 2024-04-25T11:12:08.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.841+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b38003520 con 0x7f3b5015baa0 2024-04-25T11:12:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.841+0000 7f3b56600640 1 --2- 172.21.15.7:0/1339441610 >> v2:172.21.15.87:3300/0 conn(0x7f3b5006d580 0x7f3b5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.841+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2009729033 0 0) 0x7f3b38003520 con 0x7f3b5015baa0 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.841+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 >> v2:172.21.15.87:3300/0 conn(0x7f3b5006d580 msgr2=0x7f3b5006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.841+0000 7f3b4fe00640 1 --2- 172.21.15.7:0/1339441610 >> v2:172.21.15.87:3300/0 conn(0x7f3b5006d580 0x7f3b5006d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.841+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 >> v1:172.21.15.87:6789/0 conn(0x7f3b5015e2f0 legacy=0x7f3b50161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.841+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5015ec60 con 0x7f3b5015baa0 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.841+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f3b44001840 con 0x7f3b5015baa0 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.841+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 >> v1:172.21.15.7:6789/0 conn(0x7f3b5015baa0 legacy=0x7f3b50070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.841+0000 7f3b4fe00640 1 --2- 172.21.15.7:0/1339441610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b380039b0 0x7f3b38003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b4fe00640 1 --2- 172.21.15.7:0/1339441610 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b38004800 0x7f3b38004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b4fe00640 1 --2- 172.21.15.7:0/1339441610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b38005350 0x7f3b3800b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5015e980 con 0x7f3b38005350 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3800c2c0 con 0x7f3b380039b0 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3800c4c0 con 0x7f3b38004800 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b56600640 1 --2- 172.21.15.7:0/1339441610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b38005350 0x7f3b3800b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:08.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b55c00640 1 --2- 172.21.15.7:0/1339441610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b380039b0 0x7f3b38003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b56600640 1 -- 172.21.15.7:0/1339441610 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b38004800 msgr2=0x7f3b38004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b56600640 1 --2- 172.21.15.7:0/1339441610 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b38004800 0x7f3b38004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b56600640 1 -- 172.21.15.7:0/1339441610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b380039b0 msgr2=0x7f3b38003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b56600640 1 --2- 172.21.15.7:0/1339441610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b380039b0 0x7f3b38003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.842+0000 7f3b56600640 1 -- 172.21.15.7:0/1339441610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b38003520 con 0x7f3b38005350 2024-04-25T11:12:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.843+0000 7f3b56600640 1 -- 172.21.15.7:0/1339441610 reap_dead start 2024-04-25T11:12:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.843+0000 7f3b56600640 1 --2- 172.21.15.7:0/1339441610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b38005350 0x7f3b3800b9e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3b48007f30 tx=0x7f3b38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25afe0f564f50fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.844+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48038070 con 0x7f3b38005350 2024-04-25T11:12:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.844+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4802b810 con 0x7f3b38005350 2024-04-25T11:12:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.844+0000 7f3b4fe00640 1 -- 172.21.15.7:0/1339441610 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48007b30 con 0x7f3b38005350 2024-04-25T11:12:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.844+0000 7f3b57000640 1 -- 172.21.15.7:0/1339441610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b38005350 msgr2=0x7f3b3800b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.844+0000 7f3b57000640 1 --2- 172.21.15.7:0/1339441610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b38005350 0x7f3b3800b9e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3b48007f30 tx=0x7f3b38005830 comp rx=0 tx=0).stop 2024-04-25T11:12:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.844+0000 7f3b57000640 1 -- 172.21.15.7:0/1339441610 shutdown_connections 2024-04-25T11:12:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.844+0000 7f3b57000640 1 --2- 172.21.15.7:0/1339441610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b38005350 0x7f3b3800b9e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.844+0000 7f3b57000640 1 -- 172.21.15.7:0/1339441610 >> 172.21.15.7:0/1339441610 conn(0x7f3b50073090 msgr2=0x7f3b500734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.844+0000 7f3b57000640 1 -- 172.21.15.7:0/1339441610 shutdown_connections 2024-04-25T11:12:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.844+0000 7f3b57000640 1 -- 172.21.15.7:0/1339441610 wait complete. 2024-04-25T11:12:08.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.847+0000 7f3b57000640 1 Processor -- start 2024-04-25T11:12:08.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.847+0000 7f3b57000640 1 -- start start 2024-04-25T11:12:08.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.848+0000 7f3b57000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b38005350 0x7f3b5015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:08.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.848+0000 7f3b57000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b500b5aa0 0x7f3b500b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:08.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.848+0000 7f3b57000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b500aed70 0x7f3b500af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:08.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.848+0000 7f3b57000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5015baa0 con 0x7f3b500aed70 2024-04-25T11:12:08.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.848+0000 7f3b57000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3b501619c0 con 0x7f3b500b5aa0 2024-04-25T11:12:08.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.848+0000 7f3b57000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5015bc20 con 0x7f3b38005350 2024-04-25T11:12:08.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.849+0000 7f3b56600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b500aed70 0x7f3b500af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:08.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.849+0000 7f3b56600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b500aed70 0x7f3b500af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35078/0 (socket says 172.21.15.7:35078) 2024-04-25T11:12:08.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.849+0000 7f3b56600640 1 -- 172.21.15.7:0/1662739466 learned_addr learned my addr 172.21.15.7:0/1662739466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.849+0000 7f3b56600640 1 -- 172.21.15.7:0/1662739466 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b38005350 msgr2=0x7f3b5015a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.849+0000 7f3b56600640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b38005350 0x7f3b5015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.849+0000 7f3b56600640 1 -- 172.21.15.7:0/1662739466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b500b5aa0 msgr2=0x7f3b500b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.849+0000 7f3b56600640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b500b5aa0 0x7f3b500b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.849+0000 7f3b56600640 1 -- 172.21.15.7:0/1662739466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b500af6c0 con 0x7f3b500aed70 2024-04-25T11:12:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.850+0000 7f3b56600640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b500aed70 0x7f3b500af180 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800c400 tx=0x7f3b48004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.850+0000 7f3b4d600640 1 -- 172.21.15.7:0/1662739466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48038040 con 0x7f3b500aed70 2024-04-25T11:12:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.850+0000 7f3b4d600640 1 -- 172.21.15.7:0/1662739466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b48043070 con 0x7f3b500aed70 2024-04-25T11:12:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.850+0000 7f3b4d600640 1 -- 172.21.15.7:0/1662739466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b48007410 con 0x7f3b500aed70 2024-04-25T11:12:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.850+0000 7f3b57000640 1 -- 172.21.15.7:0/1662739466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b501440c0 con 0x7f3b500aed70 2024-04-25T11:12:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.850+0000 7f3b57000640 1 -- 172.21.15.7:0/1662739466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b50144680 con 0x7f3b500aed70 2024-04-25T11:12:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.851+0000 7f3b57000640 1 -- 172.21.15.7:0/1662739466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b50050740 con 0x7f3b500aed70 2024-04-25T11:12:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.852+0000 7f3b4d600640 1 -- 172.21.15.7:0/1662739466 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3b4803b070 con 0x7f3b500aed70 2024-04-25T11:12:08.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.853+0000 7f3b4d600640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b300744b0 0x7f3b300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:08.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.853+0000 7f3b4d600640 1 -- 172.21.15.7:0/1662739466 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f3b480b58e0 con 0x7f3b500aed70 2024-04-25T11:12:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.854+0000 7f3b55c00640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b300744b0 0x7f3b300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.854+0000 7f3b55c00640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b300744b0 0x7f3b300769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3b44000f60 tx=0x7f3b44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:08.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:08.858+0000 7f3b4d600640 1 -- 172.21.15.7:0/1662739466 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b500af6c0 con 0x7f3b500aed70 2024-04-25T11:12:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.128+0000 7f3b57000640 1 -- 172.21.15.7:0/1662739466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3b50074020 con 0x7f3b500aed70 2024-04-25T11:12:09.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.129+0000 7f3b4d600640 1 -- 172.21.15.7:0/1662739466 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f3b48080190 con 0x7f3b500aed70 2024-04-25T11:12:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:09.130 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:12:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-04-25T11:12:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-04-25T11:12:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w==", 2024-04-25T11:12:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:12:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:12:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:12:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw==", 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw==", 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A==", 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:12:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA==", 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg==", 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw==", 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ==", 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw==", 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng==", 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:12:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:12:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-04-25T11:12:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ==", 2024-04-25T11:12:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:12:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:12:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-04-25T11:12:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A==", 2024-04-25T11:12:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:12:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:12:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg==", 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA==", 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg==", 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ==", 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:12:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDZOSpmS8ObHxAAkXXXg1OI1YFC2/OZER3woA==", 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mds", 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEOipm0DqVHRAAQynqObbNBx6UOYcnvjnzuw==", 2024-04-25T11:12:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEOipmTVeVHRAAlP8siMs2fvNLttMBg9N8bg==", 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-osd", 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEOipmy3CVHRAACSI7/gKbVUSmEYYV2j73OA==", 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEOipmKIqVHRAArBxHLofDd51tdjN27X6g5Q==", 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEOipmMKmVHRAA6UqbE2//9rnqrx6hGWWuPw==", 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEOipm+MOVHRAACzFeB2NIJuzjbXQF8TQ3Fw==", 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw==", 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w==", 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:12:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:12:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:12:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-25T11:12:09.147 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:12:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 -- 172.21.15.7:0/1662739466 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b300744b0 msgr2=0x7f3b300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b300744b0 0x7f3b300769c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3b44000f60 tx=0x7f3b44009040 comp rx=0 tx=0).stop 2024-04-25T11:12:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 -- 172.21.15.7:0/1662739466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b500aed70 msgr2=0x7f3b500af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b500aed70 0x7f3b500af180 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3b4800c400 tx=0x7f3b48004ab0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 -- 172.21.15.7:0/1662739466 shutdown_connections 2024-04-25T11:12:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b300744b0 0x7f3b300769c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b500aed70 0x7f3b500af180 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b500b5aa0 0x7f3b500b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 --2- 172.21.15.7:0/1662739466 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b38005350 0x7f3b5015a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 -- 172.21.15.7:0/1662739466 >> 172.21.15.7:0/1662739466 conn(0x7f3b50073090 msgr2=0x7f3b5015bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 -- 172.21.15.7:0/1662739466 shutdown_connections 2024-04-25T11:12:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.132+0000 7f3b57000640 1 -- 172.21.15.7:0/1662739466 wait complete. 2024-04-25T11:12:09.158 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T11:12:09.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.435+0000 7fbe8fc00640 1 Processor -- start 2024-04-25T11:12:09.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.435+0000 7fbe8fc00640 1 -- start start 2024-04-25T11:12:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.437+0000 7fbe8fc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe880b0470 0x7fbe8806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.437+0000 7fbe8fc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbe8806f8e0 0x7fbe880aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.437+0000 7fbe8fc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe8815e3c0 con 0x7fbe8815dfb0 2024-04-25T11:12:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.437+0000 7fbe8fc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbe880cf800 con 0x7fbe8806f8e0 2024-04-25T11:12:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.437+0000 7fbe8fc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe880d1250 con 0x7fbe880b0470 2024-04-25T11:12:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.438+0000 7fbe87e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe880b0470 0x7fbe8806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.438+0000 7fbe8d200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbe8815dfb0 0x7fbe880d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:52620/0 (socket says 172.21.15.7:52620) 2024-04-25T11:12:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.438+0000 7fbe8d200640 1 -- 172.21.15.7:0/4066230104 learned_addr learned my addr 172.21.15.7:0/4066230104 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.438+0000 7fbe8dc00640 1 --2- 172.21.15.7:0/4066230104 >> v2:172.21.15.87:3300/0 conn(0x7fbe8806f8e0 0x7fbe880aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:09.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.439+0000 7fbe87400640 1 -- 172.21.15.7:0/4066230104 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2459447732 0 0) 0x7fbe8815e3c0 con 0x7fbe8815dfb0 2024-04-25T11:12:09.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.439+0000 7fbe87400640 1 -- 172.21.15.7:0/4066230104 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe70003520 con 0x7fbe8815dfb0 2024-04-25T11:12:09.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.439+0000 7fbe87e00640 1 -- 172.21.15.7:0/4066230104 >> v2:172.21.15.87:3300/0 conn(0x7fbe8806f8e0 msgr2=0x7fbe880aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:09.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.439+0000 7fbe87e00640 1 --2- 172.21.15.7:0/4066230104 >> v2:172.21.15.87:3300/0 conn(0x7fbe8806f8e0 0x7fbe880aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.439+0000 7fbe87e00640 1 -- 172.21.15.7:0/4066230104 >> v1:172.21.15.87:6789/0 conn(0x7fbe8815dfb0 legacy=0x7fbe880d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:09.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.439+0000 7fbe87e00640 1 -- 172.21.15.7:0/4066230104 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe880cfaa0 con 0x7fbe880b0470 2024-04-25T11:12:09.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.440+0000 7fbe87e00640 1 --2- 172.21.15.7:0/4066230104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe880b0470 0x7fbe8806f3a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe740097a0 tx=0x7fbe7402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6faf0db8dca15d48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:09.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.441+0000 7fbe87400640 1 -- 172.21.15.7:0/4066230104 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74037070 con 0x7fbe880b0470 2024-04-25T11:12:09.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.441+0000 7fbe87400640 1 -- 172.21.15.7:0/4066230104 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe74032440 con 0x7fbe880b0470 2024-04-25T11:12:09.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.441+0000 7fbe87400640 1 -- 172.21.15.7:0/4066230104 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74032810 con 0x7fbe880b0470 2024-04-25T11:12:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.442+0000 7fbe8fc00640 1 -- 172.21.15.7:0/4066230104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe880b0470 msgr2=0x7fbe8806f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.442+0000 7fbe8fc00640 1 --2- 172.21.15.7:0/4066230104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe880b0470 0x7fbe8806f3a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe740097a0 tx=0x7fbe7402a180 comp rx=0 tx=0).stop 2024-04-25T11:12:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.442+0000 7fbe8fc00640 1 -- 172.21.15.7:0/4066230104 shutdown_connections 2024-04-25T11:12:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.442+0000 7fbe8fc00640 1 --2- 172.21.15.7:0/4066230104 >> v2:172.21.15.87:3300/0 conn(0x7fbe8806f8e0 0x7fbe880aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.442+0000 7fbe8fc00640 1 --2- 172.21.15.7:0/4066230104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe880b0470 0x7fbe8806f3a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.442+0000 7fbe8fc00640 1 -- 172.21.15.7:0/4066230104 >> 172.21.15.7:0/4066230104 conn(0x7fbe880742c0 msgr2=0x7fbe8806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.442+0000 7fbe8fc00640 1 -- 172.21.15.7:0/4066230104 shutdown_connections 2024-04-25T11:12:09.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.442+0000 7fbe8fc00640 1 -- 172.21.15.7:0/4066230104 wait complete. 2024-04-25T11:12:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.444+0000 7fbe8fc00640 1 Processor -- start 2024-04-25T11:12:09.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.444+0000 7fbe8fc00640 1 -- start start 2024-04-25T11:12:09.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.446+0000 7fbe8fc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8806f8e0 0x7fbe881f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:09.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.446+0000 7fbe8fc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe880b0470 0x7fbe881e7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:09.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.446+0000 7fbe8fc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe8815dfb0 0x7fbe8815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:09.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.446+0000 7fbe8fc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe880d0340 con 0x7fbe8806f8e0 2024-04-25T11:12:09.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.446+0000 7fbe8fc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbe880d0060 con 0x7fbe880b0470 2024-04-25T11:12:09.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.446+0000 7fbe8fc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe880d0620 con 0x7fbe8815dfb0 2024-04-25T11:12:09.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.446+0000 7fbe8d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8806f8e0 0x7fbe881f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:09.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.446+0000 7fbe8d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8806f8e0 0x7fbe881f3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35080/0 (socket says 172.21.15.7:35080) 2024-04-25T11:12:09.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.446+0000 7fbe8d200640 1 -- 172.21.15.7:0/3404293099 learned_addr learned my addr 172.21.15.7:0/3404293099 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:09.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.447+0000 7fbe87e00640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe880b0470 0x7fbe881e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:09.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.447+0000 7fbe8d200640 1 -- 172.21.15.7:0/3404293099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe8815dfb0 msgr2=0x7fbe8815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:09.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.447+0000 7fbe8d200640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe8815dfb0 0x7fbe8815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.447+0000 7fbe8d200640 1 -- 172.21.15.7:0/3404293099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe880b0470 msgr2=0x7fbe881e7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:09.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.447+0000 7fbe8d200640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe880b0470 0x7fbe881e7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.447+0000 7fbe8d200640 1 -- 172.21.15.7:0/3404293099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe881608f0 con 0x7fbe8806f8e0 2024-04-25T11:12:09.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.447+0000 7fbe8d200640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8806f8e0 0x7fbe881f3b00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c001eb0 tx=0x7fbe7c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:09.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.447+0000 7fbe84c00640 1 -- 172.21.15.7:0/3404293099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c017070 con 0x7fbe8806f8e0 2024-04-25T11:12:09.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.447+0000 7fbe84c00640 1 -- 172.21.15.7:0/3404293099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7c005d00 con 0x7fbe8806f8e0 2024-04-25T11:12:09.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.448+0000 7fbe8fc00640 1 -- 172.21.15.7:0/3404293099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe881e7e50 con 0x7fbe8806f8e0 2024-04-25T11:12:09.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.448+0000 7fbe84c00640 1 -- 172.21.15.7:0/3404293099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c007310 con 0x7fbe8806f8e0 2024-04-25T11:12:09.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.448+0000 7fbe8fc00640 1 -- 172.21.15.7:0/3404293099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe88160eb0 con 0x7fbe8806f8e0 2024-04-25T11:12:09.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.449+0000 7fbe8fc00640 1 -- 172.21.15.7:0/3404293099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe88050740 con 0x7fbe8806f8e0 2024-04-25T11:12:09.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.450+0000 7fbe84c00640 1 -- 172.21.15.7:0/3404293099 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbe7c01a070 con 0x7fbe8806f8e0 2024-04-25T11:12:09.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.451+0000 7fbe84c00640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe680744b0 0x7fbe680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:09.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.451+0000 7fbe84c00640 1 -- 172.21.15.7:0/3404293099 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fbe7c094370 con 0x7fbe8806f8e0 2024-04-25T11:12:09.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.452+0000 7fbe87e00640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe680744b0 0x7fbe680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.452+0000 7fbe87e00640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe680744b0 0x7fbe680769c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbe74034000 tx=0x7fbe7402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.456+0000 7fbe84c00640 1 -- 172.21.15.7:0/3404293099 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe7c05ed40 con 0x7fbe8806f8e0 2024-04-25T11:12:09.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.728+0000 7fbe8fc00640 1 -- 172.21.15.7:0/3404293099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fbe88070fb0 con 0x7fbe8806f8e0 2024-04-25T11:12:09.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.735+0000 7fbe84c00640 1 -- 172.21.15.7:0/3404293099 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fbe7c05e640 con 0x7fbe8806f8e0 2024-04-25T11:12:09.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 -- 172.21.15.7:0/3404293099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe680744b0 msgr2=0x7fbe680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe680744b0 0x7fbe680769c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbe74034000 tx=0x7fbe7402e000 comp rx=0 tx=0).stop 2024-04-25T11:12:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 -- 172.21.15.7:0/3404293099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8806f8e0 msgr2=0x7fbe881f3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8806f8e0 0x7fbe881f3b00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c001eb0 tx=0x7fbe7c00c820 comp rx=0 tx=0).stop 2024-04-25T11:12:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 -- 172.21.15.7:0/3404293099 shutdown_connections 2024-04-25T11:12:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe680744b0 0x7fbe680769c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe8815dfb0 0x7fbe8815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe880b0470 0x7fbe881e7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 --2- 172.21.15.7:0/3404293099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe8806f8e0 0x7fbe881f3b00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 -- 172.21.15.7:0/3404293099 >> 172.21.15.7:0/3404293099 conn(0x7fbe880742c0 msgr2=0x7fbe8806cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 -- 172.21.15.7:0/3404293099 shutdown_connections 2024-04-25T11:12:09.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:09.738+0000 7fbe8fc00640 1 -- 172.21.15.7:0/3404293099 wait complete. 2024-04-25T11:12:09.766 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T11:12:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.033+0000 7fec5cc00640 1 Processor -- start 2024-04-25T11:12:10.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.033+0000 7fec5cc00640 1 -- start start 2024-04-25T11:12:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.035+0000 7fec5cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fec5815dfb0 0x7fec580b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.036+0000 7fec5cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5806fab0 0x7fec580aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.036+0000 7fec5cc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec5815e3c0 con 0x7fec580d0b40 2024-04-25T11:12:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.036+0000 7fec5cc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fec580b09b0 con 0x7fec5815dfb0 2024-04-25T11:12:10.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.036+0000 7fec5cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fec580cf800 con 0x7fec5806fab0 2024-04-25T11:12:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.036+0000 7fec55400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fec580d0b40 0x7fec5806f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59562/0 (socket says 172.21.15.7:59562) 2024-04-25T11:12:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.036+0000 7fec55400640 1 -- 172.21.15.7:0/2349889962 learned_addr learned my addr 172.21.15.7:0/2349889962 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.036+0000 7fec55e00640 1 --2- 172.21.15.7:0/2349889962 >> v2:172.21.15.87:3300/0 conn(0x7fec5815dfb0 0x7fec580b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.036+0000 7fec56800640 1 --2- 172.21.15.7:0/2349889962 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5806fab0 0x7fec580aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1347518208 0 0) 0x7fec5815e3c0 con 0x7fec580d0b40 2024-04-25T11:12:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec3c003520 con 0x7fec580d0b40 2024-04-25T11:12:10.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2334444965 0 0) 0x7fec3c003520 con 0x7fec580d0b40 2024-04-25T11:12:10.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5806fab0 msgr2=0x7fec580aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:10.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 --2- 172.21.15.7:0/2349889962 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5806fab0 0x7fec580aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 >> v2:172.21.15.87:3300/0 conn(0x7fec5815dfb0 msgr2=0x7fec580b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 --2- 172.21.15.7:0/2349889962 >> v2:172.21.15.87:3300/0 conn(0x7fec5815dfb0 0x7fec580b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec580cfaa0 con 0x7fec580d0b40 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fec40001670 con 0x7fec580d0b40 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 >> v1:172.21.15.7:6789/0 conn(0x7fec580d0b40 legacy=0x7fec5806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 --2- 172.21.15.7:0/2349889962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec3c0039b0 0x7fec3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 --2- 172.21.15.7:0/2349889962 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5806fab0 0x7fec3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.037+0000 7fec54a00640 1 --2- 172.21.15.7:0/2349889962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3c004d40 0x7fec3c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.038+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec580b09b0 con 0x7fec3c0039b0 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.038+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fec580cf800 con 0x7fec3c004d40 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.038+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c00ba40 con 0x7fec5806fab0 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.038+0000 7fec55e00640 1 --2- 172.21.15.7:0/2349889962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec3c0039b0 0x7fec3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.038+0000 7fec55e00640 1 -- 172.21.15.7:0/2349889962 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5806fab0 msgr2=0x7fec3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.038+0000 7fec55e00640 1 --2- 172.21.15.7:0/2349889962 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5806fab0 0x7fec3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.038+0000 7fec55e00640 1 -- 172.21.15.7:0/2349889962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3c004d40 msgr2=0x7fec3c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:10.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.038+0000 7fec55e00640 1 --2- 172.21.15.7:0/2349889962 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec3c004d40 0x7fec3c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.038+0000 7fec55e00640 1 -- 172.21.15.7:0/2349889962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c003520 con 0x7fec3c0039b0 2024-04-25T11:12:10.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.039+0000 7fec55e00640 1 --2- 172.21.15.7:0/2349889962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec3c0039b0 0x7fec3c003e20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fec48009f90 tx=0x7fec3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23102eab6a91b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:10.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.040+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48009c10 con 0x7fec3c0039b0 2024-04-25T11:12:10.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.040+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec48036070 con 0x7fec3c0039b0 2024-04-25T11:12:10.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.040+0000 7fec54a00640 1 -- 172.21.15.7:0/2349889962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48031840 con 0x7fec3c0039b0 2024-04-25T11:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.041+0000 7fec5cc00640 1 -- 172.21.15.7:0/2349889962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec3c0039b0 msgr2=0x7fec3c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.041+0000 7fec5cc00640 1 --2- 172.21.15.7:0/2349889962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec3c0039b0 0x7fec3c003e20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fec48009f90 tx=0x7fec3c005830 comp rx=0 tx=0).stop 2024-04-25T11:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.041+0000 7fec56800640 1 -- 172.21.15.7:0/2349889962 reap_dead start 2024-04-25T11:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.041+0000 7fec5cc00640 1 -- 172.21.15.7:0/2349889962 shutdown_connections 2024-04-25T11:12:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.041+0000 7fec5cc00640 1 -- 172.21.15.7:0/2349889962 >> 172.21.15.7:0/2349889962 conn(0x7fec580742c0 msgr2=0x7fec5806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.041+0000 7fec5cc00640 1 -- 172.21.15.7:0/2349889962 shutdown_connections 2024-04-25T11:12:10.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.041+0000 7fec5cc00640 1 -- 172.21.15.7:0/2349889962 wait complete. 2024-04-25T11:12:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.044+0000 7fec5cc00640 1 Processor -- start 2024-04-25T11:12:10.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.044+0000 7fec5cc00640 1 -- start start 2024-04-25T11:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.045+0000 7fec5cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec3c0039b0 0x7fec58151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.045+0000 7fec5cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5815edd0 0x7fec58153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.045+0000 7fec5cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec581541c0 0x7fec581f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.045+0000 7fec5cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec580d0340 con 0x7fec5815edd0 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.045+0000 7fec5cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fec580d0060 con 0x7fec581541c0 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.045+0000 7fec5cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fec580d0620 con 0x7fec3c0039b0 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.046+0000 7fec55400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5815edd0 0x7fec58153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.046+0000 7fec55400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5815edd0 0x7fec58153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50782/0 (socket says 172.21.15.7:50782) 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.046+0000 7fec55400640 1 -- 172.21.15.7:0/1176727946 learned_addr learned my addr 172.21.15.7:0/1176727946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.046+0000 7fec55e00640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec3c0039b0 0x7fec58151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.046+0000 7fec56800640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec581541c0 0x7fec581f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.046+0000 7fec55400640 1 -- 172.21.15.7:0/1176727946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec3c0039b0 msgr2=0x7fec58151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.046+0000 7fec55400640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec3c0039b0 0x7fec58151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.046+0000 7fec55400640 1 -- 172.21.15.7:0/1176727946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec581541c0 msgr2=0x7fec581f9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.046+0000 7fec55400640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec581541c0 0x7fec581f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.046+0000 7fec55400640 1 -- 172.21.15.7:0/1176727946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec58155290 con 0x7fec5815edd0 2024-04-25T11:12:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.047+0000 7fec55400640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5815edd0 0x7fec58153c80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fec4000c760 tx=0x7fec4000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.047+0000 7fec4e000640 1 -- 172.21.15.7:0/1176727946 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40018070 con 0x7fec5815edd0 2024-04-25T11:12:10.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.047+0000 7fec4e000640 1 -- 172.21.15.7:0/1176727946 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec40007500 con 0x7fec5815edd0 2024-04-25T11:12:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.047+0000 7fec4e000640 1 -- 172.21.15.7:0/1176727946 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40004a20 con 0x7fec5815edd0 2024-04-25T11:12:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.047+0000 7fec5cc00640 1 -- 172.21.15.7:0/1176727946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec58151a70 con 0x7fec5815edd0 2024-04-25T11:12:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.047+0000 7fec5cc00640 1 -- 172.21.15.7:0/1176727946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec58160130 con 0x7fec5815edd0 2024-04-25T11:12:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.048+0000 7fec5cc00640 1 -- 172.21.15.7:0/1176727946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58162490 con 0x7fec5815edd0 2024-04-25T11:12:10.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.049+0000 7fec4e000640 1 -- 172.21.15.7:0/1176727946 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fec40004bc0 con 0x7fec5815edd0 2024-04-25T11:12:10.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.050+0000 7fec4e000640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec340744b0 0x7fec340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.050+0000 7fec4e000640 1 -- 172.21.15.7:0/1176727946 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fec400946e0 con 0x7fec5815edd0 2024-04-25T11:12:10.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.051+0000 7fec55e00640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec340744b0 0x7fec340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.052+0000 7fec55e00640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec340744b0 0x7fec340769c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fec48035000 tx=0x7fec4802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:10.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.056+0000 7fec4e000640 1 -- 172.21.15.7:0/1176727946 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fec58162490 con 0x7fec5815edd0 2024-04-25T11:12:10.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.337+0000 7fec5cc00640 1 -- 172.21.15.7:0/1176727946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fec580cf3e0 con 0x7fec5815edd0 2024-04-25T11:12:10.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.343+0000 7fec4e000640 1 -- 172.21.15.7:0/1176727946 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fec580cf3e0 con 0x7fec5815edd0 2024-04-25T11:12:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 -- 172.21.15.7:0/1176727946 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec340744b0 msgr2=0x7fec340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec340744b0 0x7fec340769c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fec48035000 tx=0x7fec4802f000 comp rx=0 tx=0).stop 2024-04-25T11:12:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 -- 172.21.15.7:0/1176727946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5815edd0 msgr2=0x7fec58153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5815edd0 0x7fec58153c80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fec4000c760 tx=0x7fec4000cc20 comp rx=0 tx=0).stop 2024-04-25T11:12:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 -- 172.21.15.7:0/1176727946 shutdown_connections 2024-04-25T11:12:10.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fec340744b0 0x7fec340769c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec581541c0 0x7fec581f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec5815edd0 0x7fec58153c80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 --2- 172.21.15.7:0/1176727946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec3c0039b0 0x7fec58151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 -- 172.21.15.7:0/1176727946 >> 172.21.15.7:0/1176727946 conn(0x7fec580742c0 msgr2=0x7fec580b46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 -- 172.21.15.7:0/1176727946 shutdown_connections 2024-04-25T11:12:10.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.347+0000 7fec5cc00640 1 -- 172.21.15.7:0/1176727946 wait complete. 2024-04-25T11:12:10.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T11:12:10.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.664+0000 7fa44de00640 1 Processor -- start 2024-04-25T11:12:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.664+0000 7fa44de00640 1 -- start start 2024-04-25T11:12:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.667+0000 7fa44de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa4481583c0 0x7fa4480cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.667+0000 7fa44de00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa44806eea0 0x7fa44806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.667+0000 7fa44de00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4480ce570 con 0x7fa4480cfb50 2024-04-25T11:12:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.667+0000 7fa44de00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa44806f7f0 con 0x7fa4481583c0 2024-04-25T11:12:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.667+0000 7fa44de00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa44815f190 con 0x7fa44806eea0 2024-04-25T11:12:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.668+0000 7fa44d400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa44806eea0 0x7fa44806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.668+0000 7fa44d400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa44806eea0 0x7fa44806f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:32854/0 (socket says 172.21.15.7:32854) 2024-04-25T11:12:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.668+0000 7fa44d400640 1 -- 172.21.15.7:0/3540412642 learned_addr learned my addr 172.21.15.7:0/3540412642 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:10.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.668+0000 7fa44ca00640 1 --2- 172.21.15.7:0/3540412642 >> v2:172.21.15.87:3300/0 conn(0x7fa4481583c0 0x7fa4480cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.669+0000 7fa44d400640 1 -- 172.21.15.7:0/3540412642 >> v2:172.21.15.87:3300/0 conn(0x7fa4481583c0 msgr2=0x7fa4480cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.669+0000 7fa44d400640 1 --2- 172.21.15.7:0/3540412642 >> v2:172.21.15.87:3300/0 conn(0x7fa4481583c0 0x7fa4480cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.669+0000 7fa44d400640 1 -- 172.21.15.7:0/3540412642 >> v1:172.21.15.87:6789/0 conn(0x7fa4480cfb50 legacy=0x7fa4480d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.669+0000 7fa44d400640 1 -- 172.21.15.7:0/3540412642 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4480ce850 con 0x7fa44806eea0 2024-04-25T11:12:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.670+0000 7fa44d400640 1 --2- 172.21.15.7:0/3540412642 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa44806eea0 0x7fa44806f2b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa440009e50 tx=0x7fa44002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea91bc3c61ab721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.671+0000 7fa447400640 1 -- 172.21.15.7:0/3540412642 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440036070 con 0x7fa44806eea0 2024-04-25T11:12:10.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.671+0000 7fa447400640 1 -- 172.21.15.7:0/3540412642 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa440031600 con 0x7fa44806eea0 2024-04-25T11:12:10.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.671+0000 7fa447400640 1 -- 172.21.15.7:0/3540412642 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4400319d0 con 0x7fa44806eea0 2024-04-25T11:12:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.671+0000 7fa44de00640 1 -- 172.21.15.7:0/3540412642 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa44806eea0 msgr2=0x7fa44806f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.671+0000 7fa44de00640 1 --2- 172.21.15.7:0/3540412642 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa44806eea0 0x7fa44806f2b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa440009e50 tx=0x7fa44002a710 comp rx=0 tx=0).stop 2024-04-25T11:12:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.672+0000 7fa44de00640 1 -- 172.21.15.7:0/3540412642 shutdown_connections 2024-04-25T11:12:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.672+0000 7fa44de00640 1 --2- 172.21.15.7:0/3540412642 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa44806eea0 0x7fa44806f2b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.672+0000 7fa44de00640 1 --2- 172.21.15.7:0/3540412642 >> v2:172.21.15.87:3300/0 conn(0x7fa4481583c0 0x7fa4480cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.672+0000 7fa44de00640 1 -- 172.21.15.7:0/3540412642 >> 172.21.15.7:0/3540412642 conn(0x7fa4480740f0 msgr2=0x7fa44806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.672+0000 7fa44de00640 1 -- 172.21.15.7:0/3540412642 shutdown_connections 2024-04-25T11:12:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.672+0000 7fa44de00640 1 -- 172.21.15.7:0/3540412642 wait complete. 2024-04-25T11:12:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.674+0000 7fa44de00640 1 Processor -- start 2024-04-25T11:12:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.674+0000 7fa44de00640 1 -- start start 2024-04-25T11:12:10.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa44806eea0 0x7fa4481e8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44de00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4480cfb50 0x7fa448143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4481583c0 0x7fa44815cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44de00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa4480cf0f0 con 0x7fa44806eea0 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44de00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa4480cee10 con 0x7fa4481583c0 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44de00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa4480d0b90 con 0x7fa4480cfb50 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa44806eea0 0x7fa4481e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa44806eea0 0x7fa4481e8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50796/0 (socket says 172.21.15.7:50796) 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44ca00640 1 -- 172.21.15.7:0/1707169754 learned_addr learned my addr 172.21.15.7:0/1707169754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa447e00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4480cfb50 0x7fa448143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44d400640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4481583c0 0x7fa44815cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44ca00640 1 -- 172.21.15.7:0/1707169754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4480cfb50 msgr2=0x7fa448143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44ca00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4480cfb50 0x7fa448143f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.676+0000 7fa44ca00640 1 -- 172.21.15.7:0/1707169754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4481583c0 msgr2=0x7fa44815cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.677+0000 7fa44ca00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4481583c0 0x7fa44815cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.677+0000 7fa44ca00640 1 -- 172.21.15.7:0/1707169754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4481444a0 con 0x7fa44806eea0 2024-04-25T11:12:10.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.677+0000 7fa447e00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4480cfb50 0x7fa448143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.677+0000 7fa44ca00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa44806eea0 0x7fa4481e8660 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00e8a0 tx=0x7fa43c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.677+0000 7fa444c00640 1 -- 172.21.15.7:0/1707169754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c017070 con 0x7fa44806eea0 2024-04-25T11:12:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.677+0000 7fa444c00640 1 -- 172.21.15.7:0/1707169754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43c00ee90 con 0x7fa44806eea0 2024-04-25T11:12:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.677+0000 7fa44de00640 1 -- 172.21.15.7:0/1707169754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4481e8fd0 con 0x7fa44806eea0 2024-04-25T11:12:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.677+0000 7fa444c00640 1 -- 172.21.15.7:0/1707169754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c00cb40 con 0x7fa44806eea0 2024-04-25T11:12:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.677+0000 7fa44de00640 1 -- 172.21.15.7:0/1707169754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4481e9590 con 0x7fa44806eea0 2024-04-25T11:12:10.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.679+0000 7fa44de00640 1 -- 172.21.15.7:0/1707169754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa448050740 con 0x7fa44806eea0 2024-04-25T11:12:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.680+0000 7fa444c00640 1 -- 172.21.15.7:0/1707169754 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa43c01a070 con 0x7fa44806eea0 2024-04-25T11:12:10.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.682+0000 7fa444c00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4280744e0 0x7fa4280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:10.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.682+0000 7fa444c00640 1 -- 172.21.15.7:0/1707169754 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fa43c094fb0 con 0x7fa44806eea0 2024-04-25T11:12:10.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.682+0000 7fa447e00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4280744e0 0x7fa4280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.683+0000 7fa447e00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4280744e0 0x7fa4280769f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa448157bd0 tx=0x7fa434009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:10.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.687+0000 7fa444c00640 1 -- 172.21.15.7:0/1707169754 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa43c05f870 con 0x7fa44806eea0 2024-04-25T11:12:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.963+0000 7fa44de00640 1 -- 172.21.15.7:0/1707169754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fa448162550 con 0x7fa44806eea0 2024-04-25T11:12:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.969+0000 7fa444c00640 1 -- 172.21.15.7:0/1707169754 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fa43c05f170 con 0x7fa44806eea0 2024-04-25T11:12:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 -- 172.21.15.7:0/1707169754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4280744e0 msgr2=0x7fa4280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4280744e0 0x7fa4280769f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa448157bd0 tx=0x7fa434009040 comp rx=0 tx=0).stop 2024-04-25T11:12:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 -- 172.21.15.7:0/1707169754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa44806eea0 msgr2=0x7fa4481e8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa44806eea0 0x7fa4481e8660 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa43c00e8a0 tx=0x7fa43c00ed60 comp rx=0 tx=0).stop 2024-04-25T11:12:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 -- 172.21.15.7:0/1707169754 shutdown_connections 2024-04-25T11:12:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4280744e0 0x7fa4280769f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4481583c0 0x7fa44815cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4480cfb50 0x7fa448143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 --2- 172.21.15.7:0/1707169754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa44806eea0 0x7fa4481e8660 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 -- 172.21.15.7:0/1707169754 >> 172.21.15.7:0/1707169754 conn(0x7fa4480740f0 msgr2=0x7fa44806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 -- 172.21.15.7:0/1707169754 shutdown_connections 2024-04-25T11:12:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:10.972+0000 7fa44de00640 1 -- 172.21.15.7:0/1707169754 wait complete. 2024-04-25T11:12:11.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T11:12:11.194 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:12:11.202 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:12:11.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.281+0000 7f0354800640 1 Processor -- start 2024-04-25T11:12:11.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.282+0000 7f0354800640 1 -- start start 2024-04-25T11:12:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.283+0000 7f0354800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f034c15aaa0 0x7f034c0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.284+0000 7f0354800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f034c0d0ee0 0x7f034c0b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.284+0000 7f0354800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f034c06f5b0 con 0x7f034c0b4e50 2024-04-25T11:12:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.284+0000 7f0354800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f034c0af290 con 0x7f034c15aaa0 2024-04-25T11:12:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.284+0000 7f0354800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f034c0d14a0 con 0x7f034c0d0ee0 2024-04-25T11:12:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.284+0000 7f0352800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f034c0b4e50 0x7f034c15d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59572/0 (socket says 172.21.15.7:59572) 2024-04-25T11:12:11.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.284+0000 7f0352800640 1 -- 172.21.15.7:0/2837291417 learned_addr learned my addr 172.21.15.7:0/2837291417 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.284+0000 7f0351400640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f034c0d0ee0 0x7f034c0b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.284+0000 7f0351e00640 1 --2- 172.21.15.7:0/2837291417 >> v2:172.21.15.87:3300/0 conn(0x7f034c15aaa0 0x7f034c0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2917605866 0 0) 0x7f034c06f5b0 con 0x7f034c0b4e50 2024-04-25T11:12:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0334003520 con 0x7f034c0b4e50 2024-04-25T11:12:11.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2332791423 0 0) 0x7f0334003520 con 0x7f034c0b4e50 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f034c0d0ee0 msgr2=0x7f034c0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f034c0d0ee0 0x7f034c0b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 >> v2:172.21.15.87:3300/0 conn(0x7f034c15aaa0 msgr2=0x7f034c0affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 --2- 172.21.15.7:0/2837291417 >> v2:172.21.15.87:3300/0 conn(0x7f034c15aaa0 0x7f034c0affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c0af4e0 con 0x7f034c0b4e50 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f03480019f0 con 0x7f034c0b4e50 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 >> v1:172.21.15.7:6789/0 conn(0x7f034c0b4e50 legacy=0x7f034c15d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03340039b0 0x7f0334003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0334004800 0x7f0334004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f034c0d0ee0 0x7f0334005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f034c0af290 con 0x7f03340039b0 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f034c0d14a0 con 0x7f0334004800 2024-04-25T11:12:11.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f033400ba70 con 0x7f034c0d0ee0 2024-04-25T11:12:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.285+0000 7f0351e00640 1 --2- 172.21.15.7:0/2837291417 >> v2:172.21.15.87:3300/0 conn(0x7f034c15aaa0 0x7f034c0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:12:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.286+0000 7f0351e00640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03340039b0 0x7f0334003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.286+0000 7f0351400640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0334004800 0x7f0334004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.286+0000 7f0352800640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f034c0d0ee0 0x7f0334005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.286+0000 7f0351400640 1 -- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f034c0d0ee0 msgr2=0x7f0334005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.286+0000 7f0351400640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f034c0d0ee0 0x7f0334005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.286+0000 7f0351400640 1 -- 172.21.15.7:0/2837291417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03340039b0 msgr2=0x7f0334003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.286+0000 7f0351400640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03340039b0 0x7f0334003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.286+0000 7f0351400640 1 -- 172.21.15.7:0/2837291417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0334003520 con 0x7f0334004800 2024-04-25T11:12:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.286+0000 7f0351e00640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03340039b0 0x7f0334003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:12:11.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.287+0000 7f0351400640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0334004800 0x7f0334004c10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f033c009af0 tx=0x7f0334005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91fae18993f0ef87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.288+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c036070 con 0x7f0334004800 2024-04-25T11:12:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.288+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033c031400 con 0x7f0334004800 2024-04-25T11:12:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.288+0000 7f0350a00640 1 -- 172.21.15.7:0/2837291417 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c009690 con 0x7f0334004800 2024-04-25T11:12:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.288+0000 7f0354800640 1 -- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0334004800 msgr2=0x7f0334004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:11.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.288+0000 7f0354800640 1 --2- 172.21.15.7:0/2837291417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0334004800 0x7f0334004c10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f033c009af0 tx=0x7f0334005830 comp rx=0 tx=0).stop 2024-04-25T11:12:11.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.288+0000 7f0352800640 1 -- 172.21.15.7:0/2837291417 reap_dead start 2024-04-25T11:12:11.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.288+0000 7f0354800640 1 -- 172.21.15.7:0/2837291417 shutdown_connections 2024-04-25T11:12:11.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.288+0000 7f0354800640 1 -- 172.21.15.7:0/2837291417 >> 172.21.15.7:0/2837291417 conn(0x7f034c0742c0 msgr2=0x7f034c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:11.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.289+0000 7f0354800640 1 -- 172.21.15.7:0/2837291417 shutdown_connections 2024-04-25T11:12:11.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.289+0000 7f0354800640 1 -- 172.21.15.7:0/2837291417 wait complete. 2024-04-25T11:12:11.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.291+0000 7f0354800640 1 Processor -- start 2024-04-25T11:12:11.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.291+0000 7f0354800640 1 -- start start 2024-04-25T11:12:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.292+0000 7f0354800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0334004800 0x7f034c1475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.292+0000 7f0354800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c14a2c0 0x7f034c0b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0354800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c0b18d0 0x7f034c1550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0354800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f034c0b4910 con 0x7f034c14a2c0 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0354800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f034c0afaa0 con 0x7f034c0b18d0 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0354800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f034c0b4a90 con 0x7f0334004800 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0351400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c14a2c0 0x7f034c0b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0351400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c14a2c0 0x7f034c0b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50812/0 (socket says 172.21.15.7:50812) 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0351400640 1 -- 172.21.15.7:0/2942154197 learned_addr learned my addr 172.21.15.7:0/2942154197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0351e00640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0334004800 0x7f034c1475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0352800640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c0b18d0 0x7f034c1550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0351400640 1 -- 172.21.15.7:0/2942154197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0334004800 msgr2=0x7f034c1475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0351400640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0334004800 0x7f034c1475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0351400640 1 -- 172.21.15.7:0/2942154197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c0b18d0 msgr2=0x7f034c1550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0351400640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c0b18d0 0x7f034c1550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0351400640 1 -- 172.21.15.7:0/2942154197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c0b1ce0 con 0x7f034c14a2c0 2024-04-25T11:12:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.293+0000 7f0352800640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c0b18d0 0x7f034c1550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.294+0000 7f0351400640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c14a2c0 0x7f034c0b1390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f033c009a90 tx=0x7f033c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.294+0000 7f0346000640 1 -- 172.21.15.7:0/2942154197 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c041020 con 0x7f034c14a2c0 2024-04-25T11:12:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.294+0000 7f0346000640 1 -- 172.21.15.7:0/2942154197 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033c03c470 con 0x7f034c14a2c0 2024-04-25T11:12:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.294+0000 7f0354800640 1 -- 172.21.15.7:0/2942154197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034c147b20 con 0x7f034c14a2c0 2024-04-25T11:12:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.294+0000 7f0346000640 1 -- 172.21.15.7:0/2942154197 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c036040 con 0x7f034c14a2c0 2024-04-25T11:12:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.294+0000 7f0354800640 1 -- 172.21.15.7:0/2942154197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034c159160 con 0x7f034c14a2c0 2024-04-25T11:12:11.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.295+0000 7f0354800640 1 -- 172.21.15.7:0/2942154197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034c050740 con 0x7f034c14a2c0 2024-04-25T11:12:11.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.296+0000 7f0346000640 1 -- 172.21.15.7:0/2942154197 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f033c044020 con 0x7f034c14a2c0 2024-04-25T11:12:11.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.297+0000 7f0346000640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03300744e0 0x7f03300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.298+0000 7f0346000640 1 -- 172.21.15.7:0/2942154197 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f033c0b53d0 con 0x7f034c14a2c0 2024-04-25T11:12:11.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.298+0000 7f0351e00640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03300744e0 0x7f03300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.299+0000 7f0351e00640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03300744e0 0x7f03300769f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0340009e30 tx=0x7f0340008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:11.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.302+0000 7f0346000640 1 -- 172.21.15.7:0/2942154197 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f033c04b020 con 0x7f034c14a2c0 2024-04-25T11:12:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.581+0000 7f0354800640 1 -- 172.21.15.7:0/2942154197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f034c070f20 con 0x7f034c14a2c0 2024-04-25T11:12:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.586+0000 7f0346000640 1 -- 172.21.15.7:0/2942154197 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f034c0b1ce0 con 0x7f034c14a2c0 2024-04-25T11:12:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.588+0000 7f0354800640 1 -- 172.21.15.7:0/2942154197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03300744e0 msgr2=0x7f03300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.589+0000 7f0354800640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03300744e0 0x7f03300769f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0340009e30 tx=0x7f0340008040 comp rx=0 tx=0).stop 2024-04-25T11:12:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.589+0000 7f0354800640 1 -- 172.21.15.7:0/2942154197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c14a2c0 msgr2=0x7f034c0b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.589+0000 7f0354800640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c14a2c0 0x7f034c0b1390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f033c009a90 tx=0x7f033c004730 comp rx=0 tx=0).stop 2024-04-25T11:12:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.589+0000 7f0354800640 1 -- 172.21.15.7:0/2942154197 shutdown_connections 2024-04-25T11:12:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.589+0000 7f0354800640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03300744e0 0x7f03300769f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.589+0000 7f0354800640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f034c0b18d0 0x7f034c1550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.589+0000 7f0354800640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c14a2c0 0x7f034c0b1390 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.589+0000 7f0354800640 1 --2- 172.21.15.7:0/2942154197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0334004800 0x7f034c1475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.589+0000 7f0354800640 1 -- 172.21.15.7:0/2942154197 >> 172.21.15.7:0/2942154197 conn(0x7f034c0742c0 msgr2=0x7f034c06fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.589+0000 7f0354800640 1 -- 172.21.15.7:0/2942154197 shutdown_connections 2024-04-25T11:12:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.589+0000 7f0354800640 1 -- 172.21.15.7:0/2942154197 wait complete. 2024-04-25T11:12:11.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T11:12:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.894+0000 7f104b200640 1 Processor -- start 2024-04-25T11:12:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.894+0000 7f104b200640 1 -- start start 2024-04-25T11:12:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.897+0000 7f104b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f104c1490f0 0x7f104c149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.897+0000 7f104b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f104c0b01d0 0x7f104c0b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.897+0000 7f104b200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f104c06f280 con 0x7f104c146180 2024-04-25T11:12:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.897+0000 7f104b200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f104c146590 con 0x7f104c1490f0 2024-04-25T11:12:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.897+0000 7f104b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f104c06f560 con 0x7f104c0b01d0 2024-04-25T11:12:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.897+0000 7f1043e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f104c1490f0 0x7f104c149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.897+0000 7f1043e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f104c1490f0 0x7f104c149500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50822/0 (socket says 172.21.15.7:50822) 2024-04-25T11:12:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.897+0000 7f1043e00640 1 -- 172.21.15.7:0/1472378933 learned_addr learned my addr 172.21.15.7:0/1472378933 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.897+0000 7f1049e00640 1 --1- 172.21.15.7:0/1472378933 >> v1:172.21.15.7:6789/0 conn(0x7f104c146180 0x7f104c070320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59588/0 (socket says 172.21.15.7:59588) 2024-04-25T11:12:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.898+0000 7f104a800640 1 --2- 172.21.15.7:0/1472378933 >> v2:172.21.15.87:3300/0 conn(0x7f104c0b01d0 0x7f104c0b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.898+0000 7f1043e00640 1 -- 172.21.15.7:0/1472378933 >> v2:172.21.15.87:3300/0 conn(0x7f104c0b01d0 msgr2=0x7f104c0b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.898+0000 7f1043e00640 1 --2- 172.21.15.7:0/1472378933 >> v2:172.21.15.87:3300/0 conn(0x7f104c0b01d0 0x7f104c0b05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.898+0000 7f1043e00640 1 -- 172.21.15.7:0/1472378933 >> v1:172.21.15.7:6789/0 conn(0x7f104c146180 legacy=0x7f104c070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.898+0000 7f1043e00640 1 -- 172.21.15.7:0/1472378933 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c06f840 con 0x7f104c1490f0 2024-04-25T11:12:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.899+0000 7f1043e00640 1 --2- 172.21.15.7:0/1472378933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f104c1490f0 0x7f104c149500 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f104c070d90 tx=0x7f103402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80628c258a619ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.899+0000 7f1049400640 1 -- 172.21.15.7:0/1472378933 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034037070 con 0x7f104c1490f0 2024-04-25T11:12:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.899+0000 7f1049400640 1 -- 172.21.15.7:0/1472378933 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1034032440 con 0x7f104c1490f0 2024-04-25T11:12:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.899+0000 7f1049400640 1 -- 172.21.15.7:0/1472378933 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034032810 con 0x7f104c1490f0 2024-04-25T11:12:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.900+0000 7f104b200640 1 -- 172.21.15.7:0/1472378933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f104c1490f0 msgr2=0x7f104c149500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.900+0000 7f104b200640 1 --2- 172.21.15.7:0/1472378933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f104c1490f0 0x7f104c149500 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f104c070d90 tx=0x7f103402a270 comp rx=0 tx=0).stop 2024-04-25T11:12:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.900+0000 7f104b200640 1 -- 172.21.15.7:0/1472378933 shutdown_connections 2024-04-25T11:12:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.900+0000 7f104b200640 1 --2- 172.21.15.7:0/1472378933 >> v2:172.21.15.87:3300/0 conn(0x7f104c0b01d0 0x7f104c0b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.900+0000 7f104b200640 1 --2- 172.21.15.7:0/1472378933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f104c1490f0 0x7f104c149500 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.900+0000 7f104b200640 1 -- 172.21.15.7:0/1472378933 >> 172.21.15.7:0/1472378933 conn(0x7f104c0740f0 msgr2=0x7f104c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.900+0000 7f104b200640 1 -- 172.21.15.7:0/1472378933 shutdown_connections 2024-04-25T11:12:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.900+0000 7f104b200640 1 -- 172.21.15.7:0/1472378933 wait complete. 2024-04-25T11:12:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.903+0000 7f104b200640 1 Processor -- start 2024-04-25T11:12:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.903+0000 7f104b200640 1 -- start start 2024-04-25T11:12:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.904+0000 7f104b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f104c0b01d0 0x7f104c1ea7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.904+0000 7f104b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f104c146180 0x7f104c1eafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.904+0000 7f104b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f104c1490f0 0x7f104c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.904+0000 7f104b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f104c0b0b20 con 0x7f104c146180 2024-04-25T11:12:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.904+0000 7f104b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f104c06fe00 con 0x7f104c1490f0 2024-04-25T11:12:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.904+0000 7f104b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f104c06eea0 con 0x7f104c0b01d0 2024-04-25T11:12:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.905+0000 7f1043e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f104c146180 0x7f104c1eafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.905+0000 7f1043e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f104c146180 0x7f104c1eafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50828/0 (socket says 172.21.15.7:50828) 2024-04-25T11:12:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.905+0000 7f1043e00640 1 -- 172.21.15.7:0/1612729504 learned_addr learned my addr 172.21.15.7:0/1612729504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.905+0000 7f1049e00640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f104c0b01d0 0x7f104c1ea7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.905+0000 7f104a800640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f104c1490f0 0x7f104c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.905+0000 7f1043e00640 1 -- 172.21.15.7:0/1612729504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f104c0b01d0 msgr2=0x7f104c1ea7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.905+0000 7f1043e00640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f104c0b01d0 0x7f104c1ea7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.905+0000 7f1043e00640 1 -- 172.21.15.7:0/1612729504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f104c1490f0 msgr2=0x7f104c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.905+0000 7f1043e00640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f104c1490f0 0x7f104c15f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.905+0000 7f1043e00640 1 -- 172.21.15.7:0/1612729504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c15fb90 con 0x7f104c146180 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.905+0000 7f104a800640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f104c1490f0 0x7f104c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.906+0000 7f1043e00640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f104c146180 0x7f104c1eafd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f103402a2b0 tx=0x7f10340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.906+0000 7f1042000640 1 -- 172.21.15.7:0/1612729504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034037040 con 0x7f104c146180 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.906+0000 7f1042000640 1 -- 172.21.15.7:0/1612729504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1034042070 con 0x7f104c146180 2024-04-25T11:12:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.906+0000 7f1042000640 1 -- 172.21.15.7:0/1612729504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034030bb0 con 0x7f104c146180 2024-04-25T11:12:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.906+0000 7f104b200640 1 -- 172.21.15.7:0/1612729504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104c1eb700 con 0x7f104c146180 2024-04-25T11:12:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.906+0000 7f104b200640 1 -- 172.21.15.7:0/1612729504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104c1ebcc0 con 0x7f104c146180 2024-04-25T11:12:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.907+0000 7f104b200640 1 -- 172.21.15.7:0/1612729504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104c050740 con 0x7f104c146180 2024-04-25T11:12:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.909+0000 7f1042000640 1 -- 172.21.15.7:0/1612729504 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1034047460 con 0x7f104c146180 2024-04-25T11:12:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.910+0000 7f1042000640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 0x7f10280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.910+0000 7f1042000640 1 -- 172.21.15.7:0/1612729504 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f10340b5d30 con 0x7f104c146180 2024-04-25T11:12:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.910+0000 7f1049e00640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 0x7f10280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.911+0000 7f1049e00640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 0x7f10280769c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f103c000f60 tx=0x7f103c007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:11.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:11.914+0000 7f1042000640 1 -- 172.21.15.7:0/1612729504 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f104c15fb90 con 0x7f104c146180 2024-04-25T11:12:12.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.193+0000 7f104b200640 1 -- 172.21.15.7:0/1612729504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f104c144f60 con 0x7f104c146180 2024-04-25T11:12:12.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.198+0000 7f1042000640 1 -- 172.21.15.7:0/1612729504 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f1034080020 con 0x7f104c146180 2024-04-25T11:12:12.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.200+0000 7f104b200640 1 -- 172.21.15.7:0/1612729504 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 msgr2=0x7f10280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:12.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.200+0000 7f104b200640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 0x7f10280769c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f103c000f60 tx=0x7f103c007ce0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.200+0000 7f104b200640 1 -- 172.21.15.7:0/1612729504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f104c146180 msgr2=0x7f104c1eafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:12.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.200+0000 7f104b200640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f104c146180 0x7f104c1eafd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f103402a2b0 tx=0x7f10340046c0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.201+0000 7f104b200640 1 -- 172.21.15.7:0/1612729504 shutdown_connections 2024-04-25T11:12:12.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.201+0000 7f104b200640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 0x7f10280769c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.201+0000 7f104b200640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f104c1490f0 0x7f104c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.201+0000 7f104b200640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f104c146180 0x7f104c1eafd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.201+0000 7f104b200640 1 --2- 172.21.15.7:0/1612729504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f104c0b01d0 0x7f104c1ea7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.201+0000 7f104b200640 1 -- 172.21.15.7:0/1612729504 >> 172.21.15.7:0/1612729504 conn(0x7f104c0740f0 msgr2=0x7f104c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.201+0000 7f104b200640 1 -- 172.21.15.7:0/1612729504 shutdown_connections 2024-04-25T11:12:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.201+0000 7f104b200640 1 -- 172.21.15.7:0/1612729504 wait complete. 2024-04-25T11:12:12.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T11:12:12.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.501+0000 7f3aaca00640 1 Processor -- start 2024-04-25T11:12:12.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.502+0000 7f3aaca00640 1 -- start start 2024-04-25T11:12:12.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.504+0000 7f3aaca00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa806fab0 0x7f3aa8168e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:12.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.504+0000 7f3aaca00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa815e3c0 con 0x7f3aa80b0470 2024-04-25T11:12:12.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.504+0000 7f3aaca00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa80cf800 con 0x7f3aa815dfb0 2024-04-25T11:12:12.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.504+0000 7f3aaca00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa80d1250 con 0x7f3aa806fab0 2024-04-25T11:12:12.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.504+0000 7f3a9fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3aa80b0470 0x7f3aa806f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59598/0 (socket says 172.21.15.7:59598) 2024-04-25T11:12:12.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.504+0000 7f3a9fe00640 1 -- 172.21.15.7:0/339660564 learned_addr learned my addr 172.21.15.7:0/339660564 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:12.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa6800640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa806fab0 0x7f3aa8168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 934791163 0 0) 0x7f3aa815e3c0 con 0x7f3aa80b0470 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a8c003520 con 0x7f3aa80b0470 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 29300502 0 0) 0x7f3a8c003520 con 0x7f3aa80b0470 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa806fab0 msgr2=0x7f3aa8168e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa5400640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa806fab0 0x7f3aa8168e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 >> v1:172.21.15.87:6789/0 conn(0x7f3aa815dfb0 legacy=0x7f3aa80d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa80cfaa0 con 0x7f3aa80b0470 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f3a90001670 con 0x7f3aa80b0470 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa6800640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa806fab0 0x7f3aa8168e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 >> v1:172.21.15.7:6789/0 conn(0x7f3aa80b0470 legacy=0x7f3aa806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.505+0000 7f3aa5400640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa806fab0 0x7f3a8c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa5400640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a8c004040 0x7f3a8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa5400640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 0x7f3a8c005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa80d1250 con 0x7f3a8c004d90 2024-04-25T11:12:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c00bc20 con 0x7f3a8c004040 2024-04-25T11:12:12.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3a8c00be20 con 0x7f3aa806fab0 2024-04-25T11:12:12.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa6800640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 0x7f3a8c005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:12.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa5e00640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa806fab0 0x7f3a8c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:12.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3a9fe00640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a8c004040 0x7f3a8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:12.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa6800640 1 -- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa806fab0 msgr2=0x7f3a8c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:12.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa6800640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa806fab0 0x7f3a8c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa6800640 1 -- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a8c004040 msgr2=0x7f3a8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:12.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa6800640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a8c004040 0x7f3a8c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.506+0000 7f3aa6800640 1 -- 172.21.15.7:0/339660564 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c003520 con 0x7f3a8c004d90 2024-04-25T11:12:12.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.507+0000 7f3aa6800640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 0x7f3a8c005260 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00073e0 tx=0x7f3a8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ee780ffc62b61a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:12.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.508+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0009c80 con 0x7f3a8c004d90 2024-04-25T11:12:12.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.508+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3aa002c620 con 0x7f3a8c004d90 2024-04-25T11:12:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.508+0000 7f3aa5400640 1 -- 172.21.15.7:0/339660564 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa002c9d0 con 0x7f3a8c004d90 2024-04-25T11:12:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.508+0000 7f3aaca00640 1 -- 172.21.15.7:0/339660564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 msgr2=0x7f3a8c005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.508+0000 7f3aaca00640 1 --2- 172.21.15.7:0/339660564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 0x7f3a8c005260 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00073e0 tx=0x7f3a8c005830 comp rx=0 tx=0).stop 2024-04-25T11:12:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.508+0000 7f3aa6800640 1 -- 172.21.15.7:0/339660564 reap_dead start 2024-04-25T11:12:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.508+0000 7f3aaca00640 1 -- 172.21.15.7:0/339660564 shutdown_connections 2024-04-25T11:12:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.508+0000 7f3aaca00640 1 -- 172.21.15.7:0/339660564 >> 172.21.15.7:0/339660564 conn(0x7f3aa80742c0 msgr2=0x7f3aa806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.509+0000 7f3aaca00640 1 -- 172.21.15.7:0/339660564 shutdown_connections 2024-04-25T11:12:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.509+0000 7f3aaca00640 1 -- 172.21.15.7:0/339660564 wait complete. 2024-04-25T11:12:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.511+0000 7f3aaca00640 1 Processor -- start 2024-04-25T11:12:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.511+0000 7f3aaca00640 1 -- start start 2024-04-25T11:12:12.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.512+0000 7f3aaca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 0x7f3aa8150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.512+0000 7f3aaca00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa81e8320 0x7f3aa806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.512+0000 7f3aaca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3aa81e8730 0x7f3aa806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.512+0000 7f3aaca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa80d0340 con 0x7f3a8c004d90 2024-04-25T11:12:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.512+0000 7f3aaca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa80d0060 con 0x7f3aa81e8730 2024-04-25T11:12:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.512+0000 7f3aaca00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa80d0620 con 0x7f3aa81e8320 2024-04-25T11:12:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.513+0000 7f3aa5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 0x7f3aa8150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.513+0000 7f3aa5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 0x7f3aa8150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50852/0 (socket says 172.21.15.7:50852) 2024-04-25T11:12:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.513+0000 7f3aa5e00640 1 -- 172.21.15.7:0/2882166757 learned_addr learned my addr 172.21.15.7:0/2882166757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.513+0000 7f3aa5e00640 1 -- 172.21.15.7:0/2882166757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa81e8320 msgr2=0x7f3aa806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.513+0000 7f3aa5e00640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa81e8320 0x7f3aa806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.513+0000 7f3aa5e00640 1 -- 172.21.15.7:0/2882166757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3aa81e8730 msgr2=0x7f3aa806d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.513+0000 7f3aa5e00640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3aa81e8730 0x7f3aa806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.513+0000 7f3aa5e00640 1 -- 172.21.15.7:0/2882166757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa81e9210 con 0x7f3a8c004d90 2024-04-25T11:12:12.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.513+0000 7f3aa5e00640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 0x7f3aa8150dc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800e490 tx=0x7f3a9800e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:12.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.514+0000 7f3a9e000640 1 -- 172.21.15.7:0/2882166757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98019070 con 0x7f3a8c004d90 2024-04-25T11:12:12.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.514+0000 7f3a9e000640 1 -- 172.21.15.7:0/2882166757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a980052e0 con 0x7f3a8c004d90 2024-04-25T11:12:12.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.514+0000 7f3a9e000640 1 -- 172.21.15.7:0/2882166757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98005490 con 0x7f3a8c004d90 2024-04-25T11:12:12.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.514+0000 7f3aaca00640 1 -- 172.21.15.7:0/2882166757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa81e8f30 con 0x7f3a8c004d90 2024-04-25T11:12:12.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.514+0000 7f3aaca00640 1 -- 172.21.15.7:0/2882166757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa815fcf0 con 0x7f3a8c004d90 2024-04-25T11:12:12.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.515+0000 7f3aaca00640 1 -- 172.21.15.7:0/2882166757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa8050740 con 0x7f3a8c004d90 2024-04-25T11:12:12.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.516+0000 7f3a9e000640 1 -- 172.21.15.7:0/2882166757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3a9801c070 con 0x7f3a8c004d90 2024-04-25T11:12:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.517+0000 7f3a9e000640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3a840744e0 0x7f3a840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.518+0000 7f3a9e000640 1 -- 172.21.15.7:0/2882166757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f3a980945f0 con 0x7f3a8c004d90 2024-04-25T11:12:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.518+0000 7f3a9fe00640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3a840744e0 0x7f3a840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.518+0000 7f3a9fe00640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3a840744e0 0x7f3a840769f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80af520 tx=0x7f3a90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:12.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.522+0000 7f3a9e000640 1 -- 172.21.15.7:0/2882166757 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3aa81e9210 con 0x7f3a8c004d90 2024-04-25T11:12:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.797+0000 7f3aaca00640 1 -- 172.21.15.7:0/2882166757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f3aa8070fb0 con 0x7f3a8c004d90 2024-04-25T11:12:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.801+0000 7f3a9e000640 1 -- 172.21.15.7:0/2882166757 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f3a9805e8e0 con 0x7f3a8c004d90 2024-04-25T11:12:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.804+0000 7f3aaca00640 1 -- 172.21.15.7:0/2882166757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3a840744e0 msgr2=0x7f3a840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.804+0000 7f3aaca00640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3a840744e0 0x7f3a840769f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80af520 tx=0x7f3a90009040 comp rx=0 tx=0).stop 2024-04-25T11:12:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.804+0000 7f3aaca00640 1 -- 172.21.15.7:0/2882166757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 msgr2=0x7f3aa8150dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.804+0000 7f3aaca00640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 0x7f3aa8150dc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800e490 tx=0x7f3a9800e950 comp rx=0 tx=0).stop 2024-04-25T11:12:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.804+0000 7f3aaca00640 1 -- 172.21.15.7:0/2882166757 shutdown_connections 2024-04-25T11:12:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.804+0000 7f3aaca00640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3a840744e0 0x7f3a840769f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.804+0000 7f3aaca00640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3aa81e8730 0x7f3aa806d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.804+0000 7f3aaca00640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3aa81e8320 0x7f3aa806cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.804+0000 7f3aaca00640 1 --2- 172.21.15.7:0/2882166757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a8c004d90 0x7f3aa8150dc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.804+0000 7f3aaca00640 1 -- 172.21.15.7:0/2882166757 >> 172.21.15.7:0/2882166757 conn(0x7f3aa80742c0 msgr2=0x7f3aa815b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.805+0000 7f3aaca00640 1 -- 172.21.15.7:0/2882166757 shutdown_connections 2024-04-25T11:12:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:12.805+0000 7f3aaca00640 1 -- 172.21.15.7:0/2882166757 wait complete. 2024-04-25T11:12:12.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:12:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.108+0000 7f114d400640 1 Processor -- start 2024-04-25T11:12:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.108+0000 7f114d400640 1 -- start start 2024-04-25T11:12:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.110+0000 7f114d400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11480aed40 0x7f114815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.110+0000 7f114d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11480b5220 0x7f11480ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.110+0000 7f114d400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11480b64b0 con 0x7f11480af420 2024-04-25T11:12:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.110+0000 7f114d400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f114815e1f0 con 0x7f11480b5220 2024-04-25T11:12:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.110+0000 7f114d400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f114806c610 con 0x7f11480aed40 2024-04-25T11:12:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.110+0000 7f1147400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11480b5220 0x7f11480ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.110+0000 7f1147400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11480b5220 0x7f11480ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50858/0 (socket says 172.21.15.7:50858) 2024-04-25T11:12:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.110+0000 7f1147400640 1 -- 172.21.15.7:0/2320868341 learned_addr learned my addr 172.21.15.7:0/2320868341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.110+0000 7f114ca00640 1 --1- 172.21.15.7:0/2320868341 >> v1:172.21.15.7:6789/0 conn(0x7f11480af420 0x7f11480af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59614/0 (socket says 172.21.15.7:59614) 2024-04-25T11:12:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.111+0000 7f1147e00640 1 --2- 172.21.15.7:0/2320868341 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11480aed40 0x7f114815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.111+0000 7f1147400640 1 -- 172.21.15.7:0/2320868341 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11480aed40 msgr2=0x7f114815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.111+0000 7f1147400640 1 --2- 172.21.15.7:0/2320868341 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11480aed40 0x7f114815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.111+0000 7f1147400640 1 -- 172.21.15.7:0/2320868341 >> v1:172.21.15.7:6789/0 conn(0x7f11480af420 legacy=0x7f11480af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.111+0000 7f1147400640 1 -- 172.21.15.7:0/2320868341 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114806c8f0 con 0x7f11480b5220 2024-04-25T11:12:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.111+0000 7f1146a00640 1 -- 172.21.15.7:0/2320868341 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 973692720 0 0) 0x7f11480b64b0 con 0x7f11480af420 2024-04-25T11:12:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.112+0000 7f1147400640 1 --2- 172.21.15.7:0/2320868341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11480b5220 0x7f11480ce4a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1130009870 tx=0x7f113002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f9c4e6b3f8ef3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.112+0000 7f1146a00640 1 -- 172.21.15.7:0/2320868341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130037070 con 0x7f11480b5220 2024-04-25T11:12:13.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.112+0000 7f1146a00640 1 -- 172.21.15.7:0/2320868341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1130032440 con 0x7f11480b5220 2024-04-25T11:12:13.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.112+0000 7f1146a00640 1 -- 172.21.15.7:0/2320868341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130032810 con 0x7f11480b5220 2024-04-25T11:12:13.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.113+0000 7f114d400640 1 -- 172.21.15.7:0/2320868341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11480b5220 msgr2=0x7f11480ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.113+0000 7f114d400640 1 --2- 172.21.15.7:0/2320868341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11480b5220 0x7f11480ce4a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1130009870 tx=0x7f113002a270 comp rx=0 tx=0).stop 2024-04-25T11:12:13.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.113+0000 7f114d400640 1 -- 172.21.15.7:0/2320868341 shutdown_connections 2024-04-25T11:12:13.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.113+0000 7f114d400640 1 --2- 172.21.15.7:0/2320868341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11480b5220 0x7f11480ce4a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.113+0000 7f114d400640 1 --2- 172.21.15.7:0/2320868341 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11480aed40 0x7f114815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.113+0000 7f114d400640 1 -- 172.21.15.7:0/2320868341 >> 172.21.15.7:0/2320868341 conn(0x7f11480b3820 msgr2=0x7f11480b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:13.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.113+0000 7f114d400640 1 -- 172.21.15.7:0/2320868341 shutdown_connections 2024-04-25T11:12:13.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.113+0000 7f114d400640 1 -- 172.21.15.7:0/2320868341 wait complete. 2024-04-25T11:12:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.115+0000 7f114d400640 1 Processor -- start 2024-04-25T11:12:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.116+0000 7f114d400640 1 -- start start 2024-04-25T11:12:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.117+0000 7f114d400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11480aed40 0x7f11481587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.117+0000 7f114d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11480af420 0x7f114809efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.117+0000 7f114d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11480b5220 0x7f114809c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.117+0000 7f114d400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f11480b6110 con 0x7f11480af420 2024-04-25T11:12:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.117+0000 7f114d400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f114806ceb0 con 0x7f11480b5220 2024-04-25T11:12:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.117+0000 7f114d400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f11480b6290 con 0x7f11480aed40 2024-04-25T11:12:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.117+0000 7f1147400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11480af420 0x7f114809efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:13.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.117+0000 7f1147400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11480af420 0x7f114809efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50872/0 (socket says 172.21.15.7:50872) 2024-04-25T11:12:13.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.117+0000 7f1147400640 1 -- 172.21.15.7:0/658721888 learned_addr learned my addr 172.21.15.7:0/658721888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:13.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.118+0000 7f1147e00640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11480aed40 0x7f11481587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:13.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.118+0000 7f1147400640 1 -- 172.21.15.7:0/658721888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11480aed40 msgr2=0x7f11481587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.118+0000 7f1147400640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11480aed40 0x7f11481587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.118+0000 7f1147400640 1 -- 172.21.15.7:0/658721888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11480b5220 msgr2=0x7f114809c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.118+0000 7f1147400640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11480b5220 0x7f114809c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.118+0000 7f1147400640 1 -- 172.21.15.7:0/658721888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114809df10 con 0x7f11480af420 2024-04-25T11:12:13.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.118+0000 7f1147e00640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11480aed40 0x7f11481587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:13.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.118+0000 7f1147400640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11480af420 0x7f114809efb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1130035040 tx=0x7f11300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:13.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.118+0000 7f113fe00640 1 -- 172.21.15.7:0/658721888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130037040 con 0x7f11480af420 2024-04-25T11:12:13.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.118+0000 7f113fe00640 1 -- 172.21.15.7:0/658721888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1130004030 con 0x7f11480af420 2024-04-25T11:12:13.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.119+0000 7f114d400640 1 -- 172.21.15.7:0/658721888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114809e1f0 con 0x7f11480af420 2024-04-25T11:12:13.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.119+0000 7f113fe00640 1 -- 172.21.15.7:0/658721888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130004420 con 0x7f11480af420 2024-04-25T11:12:13.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.119+0000 7f114d400640 1 -- 172.21.15.7:0/658721888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114809e7b0 con 0x7f11480af420 2024-04-25T11:12:13.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.120+0000 7f114d400640 1 -- 172.21.15.7:0/658721888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1148158d80 con 0x7f11480af420 2024-04-25T11:12:13.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.121+0000 7f113fe00640 1 -- 172.21.15.7:0/658721888 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f113003a070 con 0x7f11480af420 2024-04-25T11:12:13.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.122+0000 7f113fe00640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11240744b0 0x7f11240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.122+0000 7f113fe00640 1 -- 172.21.15.7:0/658721888 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f11300b5760 con 0x7f11480af420 2024-04-25T11:12:13.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.122+0000 7f1147e00640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11240744b0 0x7f11240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:13.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.123+0000 7f1147e00640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11240744b0 0x7f11240769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1138006fd0 tx=0x7f1138008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:13.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.127+0000 7f113fe00640 1 -- 172.21.15.7:0/658721888 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1130041020 con 0x7f11480af420 2024-04-25T11:12:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.325+0000 7f114d400640 1 -- 172.21.15.7:0/658721888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1148050740 con 0x7f11480af420 2024-04-25T11:12:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.327+0000 7f113fe00640 1 -- 172.21.15.7:0/658721888 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+13068 (secure 0 0 0) 0x7f1130080010 con 0x7f11480af420 2024-04-25T11:12:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:13.328 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:12:08.541865+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:12:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.330+0000 7f114d400640 1 -- 172.21.15.7:0/658721888 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11240744b0 msgr2=0x7f11240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.330+0000 7f114d400640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11240744b0 0x7f11240769c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1138006fd0 tx=0x7f1138008040 comp rx=0 tx=0).stop 2024-04-25T11:12:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.330+0000 7f114d400640 1 -- 172.21.15.7:0/658721888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11480af420 msgr2=0x7f114809efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.330+0000 7f114d400640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11480af420 0x7f114809efb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1130035040 tx=0x7f11300046c0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.330+0000 7f114d400640 1 -- 172.21.15.7:0/658721888 shutdown_connections 2024-04-25T11:12:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.330+0000 7f114d400640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11240744b0 0x7f11240769c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.330+0000 7f114d400640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11480b5220 0x7f114809c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.330+0000 7f114d400640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11480af420 0x7f114809efb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.330+0000 7f114d400640 1 --2- 172.21.15.7:0/658721888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11480aed40 0x7f11481587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.330+0000 7f114d400640 1 -- 172.21.15.7:0/658721888 >> 172.21.15.7:0/658721888 conn(0x7f11480b3820 msgr2=0x7f114806d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.331+0000 7f114d400640 1 -- 172.21.15.7:0/658721888 shutdown_connections 2024-04-25T11:12:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.331+0000 7f114d400640 1 -- 172.21.15.7:0/658721888 wait complete. 2024-04-25T11:12:13.356 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:12:13.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:12:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.643+0000 7f3912200640 1 Processor -- start 2024-04-25T11:12:13.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.643+0000 7f3912200640 1 -- start start 2024-04-25T11:12:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.646+0000 7f3912200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f390c15aaa0 0x7f390c0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.646+0000 7f3912200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f390c0d0ee0 0x7f390c0b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.646+0000 7f3912200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f390c06f5b0 con 0x7f390c0b4e50 2024-04-25T11:12:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.646+0000 7f3912200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f390c0af290 con 0x7f390c15aaa0 2024-04-25T11:12:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.646+0000 7f3912200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f390c0d14a0 con 0x7f390c0d0ee0 2024-04-25T11:12:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.646+0000 7f390b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f390c15aaa0 0x7f390c0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.646+0000 7f390b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f390c15aaa0 0x7f390c0affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50876/0 (socket says 172.21.15.7:50876) 2024-04-25T11:12:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.646+0000 7f390b400640 1 -- 172.21.15.7:0/2464207444 learned_addr learned my addr 172.21.15.7:0/2464207444 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.646+0000 7f390be00640 1 --1- 172.21.15.7:0/2464207444 >> v1:172.21.15.7:6789/0 conn(0x7f390c0b4e50 0x7f390c15d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59622/0 (socket says 172.21.15.7:59622) 2024-04-25T11:12:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.647+0000 7f390aa00640 1 --2- 172.21.15.7:0/2464207444 >> v2:172.21.15.87:3300/0 conn(0x7f390c0d0ee0 0x7f390c0b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.647+0000 7f390b400640 1 -- 172.21.15.7:0/2464207444 >> v2:172.21.15.87:3300/0 conn(0x7f390c0d0ee0 msgr2=0x7f390c0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.647+0000 7f390b400640 1 --2- 172.21.15.7:0/2464207444 >> v2:172.21.15.87:3300/0 conn(0x7f390c0d0ee0 0x7f390c0b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.647+0000 7f390b400640 1 -- 172.21.15.7:0/2464207444 >> v1:172.21.15.7:6789/0 conn(0x7f390c0b4e50 legacy=0x7f390c15d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.647+0000 7f390b400640 1 -- 172.21.15.7:0/2464207444 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c0af4e0 con 0x7f390c15aaa0 2024-04-25T11:12:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.647+0000 7f390aa00640 1 --2- 172.21.15.7:0/2464207444 >> v2:172.21.15.87:3300/0 conn(0x7f390c0d0ee0 0x7f390c0b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.647+0000 7f390a000640 1 -- 172.21.15.7:0/2464207444 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4000327624 0 0) 0x7f390c06f5b0 con 0x7f390c0b4e50 2024-04-25T11:12:13.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.648+0000 7f390b400640 1 --2- 172.21.15.7:0/2464207444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c15aaa0 0x7f390c0affc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f390c0af410 tx=0x7f38fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e3b559dba18a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:13.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.648+0000 7f390a000640 1 -- 172.21.15.7:0/2464207444 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc037070 con 0x7f390c15aaa0 2024-04-25T11:12:13.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.649+0000 7f390a000640 1 -- 172.21.15.7:0/2464207444 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc032440 con 0x7f390c15aaa0 2024-04-25T11:12:13.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.649+0000 7f390a000640 1 -- 172.21.15.7:0/2464207444 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc032810 con 0x7f390c15aaa0 2024-04-25T11:12:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.649+0000 7f3912200640 1 -- 172.21.15.7:0/2464207444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c15aaa0 msgr2=0x7f390c0affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.649+0000 7f3912200640 1 --2- 172.21.15.7:0/2464207444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c15aaa0 0x7f390c0affc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f390c0af410 tx=0x7f38fc02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.649+0000 7f3912200640 1 -- 172.21.15.7:0/2464207444 shutdown_connections 2024-04-25T11:12:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.649+0000 7f3912200640 1 --2- 172.21.15.7:0/2464207444 >> v2:172.21.15.87:3300/0 conn(0x7f390c0d0ee0 0x7f390c0b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.649+0000 7f3912200640 1 --2- 172.21.15.7:0/2464207444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c15aaa0 0x7f390c0affc0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.649+0000 7f3912200640 1 -- 172.21.15.7:0/2464207444 >> 172.21.15.7:0/2464207444 conn(0x7f390c0742c0 msgr2=0x7f390c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.649+0000 7f3912200640 1 -- 172.21.15.7:0/2464207444 shutdown_connections 2024-04-25T11:12:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.650+0000 7f3912200640 1 -- 172.21.15.7:0/2464207444 wait complete. 2024-04-25T11:12:13.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.652+0000 7f3912200640 1 Processor -- start 2024-04-25T11:12:13.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.652+0000 7f3912200640 1 -- start start 2024-04-25T11:12:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f3912200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f390c0b4e50 0x7f390c158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f3912200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c0d0ee0 0x7f390c148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f3912200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c15aaa0 0x7f390c06fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f3912200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f390c0b5260 con 0x7f390c15aaa0 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f3912200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f390c0afaa0 con 0x7f390c0d0ee0 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f3912200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f390c0b53e0 con 0x7f390c0b4e50 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f390be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c15aaa0 0x7f390c06fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f390be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c15aaa0 0x7f390c06fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50888/0 (socket says 172.21.15.7:50888) 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f390be00640 1 -- 172.21.15.7:0/2411551291 learned_addr learned my addr 172.21.15.7:0/2411551291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f390b400640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f390c0b4e50 0x7f390c158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f390aa00640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c0d0ee0 0x7f390c148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f390be00640 1 -- 172.21.15.7:0/2411551291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f390c0b4e50 msgr2=0x7f390c158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f390be00640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f390c0b4e50 0x7f390c158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f390be00640 1 -- 172.21.15.7:0/2411551291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c0d0ee0 msgr2=0x7f390c148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.654+0000 7f390be00640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c0d0ee0 0x7f390c148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.655+0000 7f390be00640 1 -- 172.21.15.7:0/2411551291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c0b2140 con 0x7f390c15aaa0 2024-04-25T11:12:13.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.655+0000 7f390aa00640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c0d0ee0 0x7f390c148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:13.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.655+0000 7f390b400640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f390c0b4e50 0x7f390c158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:13.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.655+0000 7f390be00640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c15aaa0 0x7f390c06fc10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f390400c700 tx=0x7f390400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:13.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.655+0000 7f3903400640 1 -- 172.21.15.7:0/2411551291 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904018070 con 0x7f390c15aaa0 2024-04-25T11:12:13.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.655+0000 7f3903400640 1 -- 172.21.15.7:0/2411551291 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f390400ce10 con 0x7f390c15aaa0 2024-04-25T11:12:13.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.655+0000 7f3912200640 1 -- 172.21.15.7:0/2411551291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c0b2420 con 0x7f390c15aaa0 2024-04-25T11:12:13.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.655+0000 7f3903400640 1 -- 172.21.15.7:0/2411551291 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390400a880 con 0x7f390c15aaa0 2024-04-25T11:12:13.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.655+0000 7f3912200640 1 -- 172.21.15.7:0/2411551291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c0b29e0 con 0x7f390c15aaa0 2024-04-25T11:12:13.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.656+0000 7f3912200640 1 -- 172.21.15.7:0/2411551291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c050740 con 0x7f390c15aaa0 2024-04-25T11:12:13.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.658+0000 7f3903400640 1 -- 172.21.15.7:0/2411551291 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f390401b070 con 0x7f390c15aaa0 2024-04-25T11:12:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.659+0000 7f3903400640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38ec0744b0 0x7f38ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.659+0000 7f3903400640 1 -- 172.21.15.7:0/2411551291 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f39040945c0 con 0x7f390c15aaa0 2024-04-25T11:12:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.659+0000 7f390b400640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38ec0744b0 0x7f38ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.660+0000 7f390b400640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38ec0744b0 0x7f38ec0769c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f38fc034000 tx=0x7f38fc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:13.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.663+0000 7f3903400640 1 -- 172.21.15.7:0/2411551291 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f390405ef00 con 0x7f390c15aaa0 2024-04-25T11:12:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.860+0000 7f3912200640 1 -- 172.21.15.7:0/2411551291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f390c0b1ed0 con 0x7f390c15aaa0 2024-04-25T11:12:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.861+0000 7f3903400640 1 -- 172.21.15.7:0/2411551291 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f39040650d0 con 0x7f390c15aaa0 2024-04-25T11:12:13.861 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.863+0000 7f3912200640 1 -- 172.21.15.7:0/2411551291 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38ec0744b0 msgr2=0x7f38ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.863+0000 7f3912200640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38ec0744b0 0x7f38ec0769c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f38fc034000 tx=0x7f38fc02e000 comp rx=0 tx=0).stop 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.863+0000 7f3912200640 1 -- 172.21.15.7:0/2411551291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c15aaa0 msgr2=0x7f390c06fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.863+0000 7f3912200640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c15aaa0 0x7f390c06fc10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f390400c700 tx=0x7f390400cbc0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.863+0000 7f3912200640 1 -- 172.21.15.7:0/2411551291 shutdown_connections 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.863+0000 7f3912200640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38ec0744b0 0x7f38ec0769c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.863+0000 7f3912200640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f390c15aaa0 0x7f390c06fc10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.863+0000 7f3912200640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c0d0ee0 0x7f390c148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.863+0000 7f3912200640 1 --2- 172.21.15.7:0/2411551291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f390c0b4e50 0x7f390c158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.863+0000 7f3912200640 1 -- 172.21.15.7:0/2411551291 >> 172.21.15.7:0/2411551291 conn(0x7f390c0742c0 msgr2=0x7f390c06cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.863+0000 7f3912200640 1 -- 172.21.15.7:0/2411551291 shutdown_connections 2024-04-25T11:12:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:13.864+0000 7f3912200640 1 -- 172.21.15.7:0/2411551291 wait complete. 2024-04-25T11:12:13.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:12:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.165+0000 7f218cc00640 1 Processor -- start 2024-04-25T11:12:14.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.165+0000 7f218cc00640 1 -- start start 2024-04-25T11:12:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.167+0000 7f218cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2188070c20 0x7f21880d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.168+0000 7f218cc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21880cffa0 con 0x7f2188070810 2024-04-25T11:12:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.168+0000 7f218cc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2188160d90 con 0x7f218806dc90 2024-04-25T11:12:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.168+0000 7f218cc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2188161070 con 0x7f2188070c20 2024-04-25T11:12:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.168+0000 7f2187400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2188070810 0x7f218806d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59632/0 (socket says 172.21.15.7:59632) 2024-04-25T11:12:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.168+0000 7f2187400640 1 -- 172.21.15.7:0/4100474310 learned_addr learned my addr 172.21.15.7:0/4100474310 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.168+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3582661044 0 0) 0x7f2178001110 con 0x7f2188070810 2024-04-25T11:12:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.168+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f216c005730 con 0x7f2188070810 2024-04-25T11:12:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2187e00640 1 --2- 172.21.15.7:0/4100474310 >> v2:172.21.15.87:3300/0 conn(0x7f2188070c20 0x7f21880d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3729177735 0 0) 0x7f216c005730 con 0x7f2188070810 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 >> v2:172.21.15.87:3300/0 conn(0x7f2188070c20 msgr2=0x7f21880d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 --2- 172.21.15.7:0/4100474310 >> v2:172.21.15.87:3300/0 conn(0x7f2188070c20 0x7f21880d0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 >> v1:172.21.15.87:6789/0 conn(0x7f218806dc90 legacy=0x7f21880d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2188161350 con 0x7f2188070810 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2178001840 con 0x7f2188070810 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 >> v1:172.21.15.7:6789/0 conn(0x7f2188070810 legacy=0x7f218806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 --2- 172.21.15.7:0/4100474310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f216c005c10 0x7f216c006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 --2- 172.21.15.7:0/4100474310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 0x7f216c006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 --2- 172.21.15.7:0/4100474310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f216c007560 0x7f216c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2188161070 con 0x7f216c006a10 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f216c00e460 con 0x7f216c007560 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f216c00e660 con 0x7f216c005c10 2024-04-25T11:12:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.169+0000 7f2186a00640 1 --2- 172.21.15.7:0/4100474310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 0x7f216c006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.170+0000 7f2187400640 1 --2- 172.21.15.7:0/4100474310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f216c005c10 0x7f216c006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.170+0000 7f2186a00640 1 -- 172.21.15.7:0/4100474310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f216c005c10 msgr2=0x7f216c006080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.170+0000 7f2186a00640 1 --2- 172.21.15.7:0/4100474310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f216c005c10 0x7f216c006080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.170+0000 7f2186a00640 1 -- 172.21.15.7:0/4100474310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f216c007560 msgr2=0x7f216c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.170+0000 7f2186a00640 1 --2- 172.21.15.7:0/4100474310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f216c007560 0x7f216c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.170+0000 7f2187e00640 1 -- 172.21.15.7:0/4100474310 reap_dead start 2024-04-25T11:12:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.170+0000 7f2186a00640 1 -- 172.21.15.7:0/4100474310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216c005730 con 0x7f216c006a10 2024-04-25T11:12:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.171+0000 7f2186a00640 1 --2- 172.21.15.7:0/4100474310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 0x7f216c006e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2174001750 tx=0x7f216c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b14c91d76ed9f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.171+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2174001370 con 0x7f216c006a10 2024-04-25T11:12:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.171+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21740052d0 con 0x7f216c006a10 2024-04-25T11:12:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.171+0000 7f2186000640 1 -- 172.21.15.7:0/4100474310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217402abc0 con 0x7f216c006a10 2024-04-25T11:12:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.172+0000 7f218cc00640 1 -- 172.21.15.7:0/4100474310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 msgr2=0x7f216c006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.172+0000 7f218cc00640 1 --2- 172.21.15.7:0/4100474310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 0x7f216c006e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2174001750 tx=0x7f216c007a40 comp rx=0 tx=0).stop 2024-04-25T11:12:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.172+0000 7f218cc00640 1 -- 172.21.15.7:0/4100474310 shutdown_connections 2024-04-25T11:12:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.172+0000 7f218cc00640 1 --2- 172.21.15.7:0/4100474310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 0x7f216c006e20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.172+0000 7f218cc00640 1 -- 172.21.15.7:0/4100474310 >> 172.21.15.7:0/4100474310 conn(0x7f2188073090 msgr2=0x7f21880734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.172+0000 7f218cc00640 1 -- 172.21.15.7:0/4100474310 shutdown_connections 2024-04-25T11:12:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.172+0000 7f218cc00640 1 -- 172.21.15.7:0/4100474310 wait complete. 2024-04-25T11:12:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.174+0000 7f218cc00640 1 Processor -- start 2024-04-25T11:12:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.175+0000 7f218cc00640 1 -- start start 2024-04-25T11:12:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.176+0000 7f218cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 0x7f218813f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.176+0000 7f218cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2188144050 0x7f2188144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.176+0000 7f218cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2188140cd0 0x7f21881410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.176+0000 7f218cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2188143530 con 0x7f216c006a10 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.176+0000 7f218cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f21881433b0 con 0x7f2188144050 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.176+0000 7f218cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f218815e4d0 con 0x7f2188140cd0 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.176+0000 7f2187400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 0x7f218813f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.177+0000 7f2187400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 0x7f218813f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50902/0 (socket says 172.21.15.7:50902) 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.177+0000 7f2187400640 1 -- 172.21.15.7:0/2298497719 learned_addr learned my addr 172.21.15.7:0/2298497719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.177+0000 7f2187e00640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2188140cd0 0x7f21881410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.177+0000 7f2186a00640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2188144050 0x7f2188144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.177+0000 7f2187400640 1 -- 172.21.15.7:0/2298497719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2188140cd0 msgr2=0x7f21881410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.177+0000 7f2187400640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2188140cd0 0x7f21881410e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.177+0000 7f2187400640 1 -- 172.21.15.7:0/2298497719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2188144050 msgr2=0x7f2188144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.177+0000 7f2187400640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2188144050 0x7f2188144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.177+0000 7f2187400640 1 -- 172.21.15.7:0/2298497719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218813f590 con 0x7f216c006a10 2024-04-25T11:12:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.177+0000 7f2187e00640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2188140cd0 0x7f21881410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.177+0000 7f2187400640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 0x7f218813f050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2178001dc0 tx=0x7f217800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.178+0000 7f217f400640 1 -- 172.21.15.7:0/2298497719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217800c660 con 0x7f216c006a10 2024-04-25T11:12:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.178+0000 7f217f400640 1 -- 172.21.15.7:0/2298497719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2178018070 con 0x7f216c006a10 2024-04-25T11:12:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.178+0000 7f217f400640 1 -- 172.21.15.7:0/2298497719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217800ca60 con 0x7f216c006a10 2024-04-25T11:12:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.178+0000 7f218cc00640 1 -- 172.21.15.7:0/2298497719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2188141620 con 0x7f216c006a10 2024-04-25T11:12:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.178+0000 7f218cc00640 1 -- 172.21.15.7:0/2298497719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2188145080 con 0x7f216c006a10 2024-04-25T11:12:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.179+0000 7f218cc00640 1 -- 172.21.15.7:0/2298497719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2188050740 con 0x7f216c006a10 2024-04-25T11:12:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.180+0000 7f217f400640 1 -- 172.21.15.7:0/2298497719 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f217800cc00 con 0x7f216c006a10 2024-04-25T11:12:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.181+0000 7f217f400640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f21680744b0 0x7f21680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.181+0000 7f217f400640 1 -- 172.21.15.7:0/2298497719 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f2178094f60 con 0x7f216c006a10 2024-04-25T11:12:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.181+0000 7f2186a00640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f21680744b0 0x7f21680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.182+0000 7f2186a00640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f21680744b0 0x7f21680769c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2174032000 tx=0x7f217402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.186+0000 7f217f400640 1 -- 172.21.15.7:0/2298497719 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2188141620 con 0x7f216c006a10 2024-04-25T11:12:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.391+0000 7f218cc00640 1 -- 172.21.15.7:0/2298497719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f218815b4e0 con 0x7f216c006a10 2024-04-25T11:12:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.392+0000 7f217f400640 1 -- 172.21.15.7:0/2298497719 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7f218813f590 con 0x7f216c006a10 2024-04-25T11:12:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:14.393 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-25T11:12:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:12:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-25T11:12:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-25T11:12:14.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:12:14.393 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-25T11:12:14.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.394+0000 7f218cc00640 1 -- 172.21.15.7:0/2298497719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f21680744b0 msgr2=0x7f21680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.395+0000 7f218cc00640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f21680744b0 0x7f21680769c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2174032000 tx=0x7f217402e000 comp rx=0 tx=0).stop 2024-04-25T11:12:14.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.395+0000 7f218cc00640 1 -- 172.21.15.7:0/2298497719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 msgr2=0x7f218813f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.395+0000 7f218cc00640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 0x7f218813f050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2178001dc0 tx=0x7f217800eb90 comp rx=0 tx=0).stop 2024-04-25T11:12:14.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.395+0000 7f218cc00640 1 -- 172.21.15.7:0/2298497719 shutdown_connections 2024-04-25T11:12:14.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.395+0000 7f218cc00640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f21680744b0 0x7f21680769c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.395+0000 7f218cc00640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2188140cd0 0x7f21881410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.395+0000 7f218cc00640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2188144050 0x7f2188144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.395+0000 7f218cc00640 1 --2- 172.21.15.7:0/2298497719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f216c006a10 0x7f218813f050 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.395+0000 7f218cc00640 1 -- 172.21.15.7:0/2298497719 >> 172.21.15.7:0/2298497719 conn(0x7f2188073090 msgr2=0x7f218806f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.395+0000 7f218cc00640 1 -- 172.21.15.7:0/2298497719 shutdown_connections 2024-04-25T11:12:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.395+0000 7f218cc00640 1 -- 172.21.15.7:0/2298497719 wait complete. 2024-04-25T11:12:14.421 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:12:14.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:12:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.712+0000 7fb8dfe00640 1 Processor -- start 2024-04-25T11:12:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.713+0000 7fb8dfe00640 1 -- start start 2024-04-25T11:12:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.715+0000 7fb8dfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8e00b52f0 0x7fb8e0160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.715+0000 7fb8dfe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8e0070320 con 0x7fb8e00b5a40 2024-04-25T11:12:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.715+0000 7fb8dfe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8e00b6270 con 0x7fb8e006d580 2024-04-25T11:12:14.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.715+0000 7fb8dfe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e006eee0 con 0x7fb8e00b52f0 2024-04-25T11:12:14.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.716+0000 7fb8de000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb8e00b5a40 0x7fb8e006fc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59644/0 (socket says 172.21.15.7:59644) 2024-04-25T11:12:14.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.716+0000 7fb8de000640 1 -- 172.21.15.7:0/4020496154 learned_addr learned my addr 172.21.15.7:0/4020496154 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.716+0000 7fb8dea00640 1 --2- 172.21.15.7:0/4020496154 >> v2:172.21.15.87:3300/0 conn(0x7fb8e00b52f0 0x7fb8e0160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.716+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4044917965 0 0) 0x7fb8e0070320 con 0x7fb8e00b5a40 2024-04-25T11:12:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.716+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8c4003520 con 0x7fb8e00b5a40 2024-04-25T11:12:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3303479987 0 0) 0x7fb8c4003520 con 0x7fb8e00b5a40 2024-04-25T11:12:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 >> v2:172.21.15.87:3300/0 conn(0x7fb8e00b52f0 msgr2=0x7fb8e0160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 --2- 172.21.15.7:0/4020496154 >> v2:172.21.15.87:3300/0 conn(0x7fb8e00b52f0 0x7fb8e0160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 >> v1:172.21.15.87:6789/0 conn(0x7fb8e006d580 legacy=0x7fb8e006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e006f130 con 0x7fb8e00b5a40 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fb8c8001670 con 0x7fb8e00b5a40 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 >> v1:172.21.15.7:6789/0 conn(0x7fb8e00b5a40 legacy=0x7fb8e006fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 --2- 172.21.15.7:0/4020496154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c4003a00 0x7fb8c4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 --2- 172.21.15.7:0/4020496154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8c4004800 0x7fb8c4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 --2- 172.21.15.7:0/4020496154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8c4005270 0x7fb8c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e006eee0 con 0x7fb8c4003a00 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c400c190 con 0x7fb8c4004800 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c400c390 con 0x7fb8c4005270 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.717+0000 7fb8dea00640 1 --2- 172.21.15.7:0/4020496154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c4003a00 0x7fb8c4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.718+0000 7fb8df400640 1 --2- 172.21.15.7:0/4020496154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8c4005270 0x7fb8c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.718+0000 7fb8de000640 1 --2- 172.21.15.7:0/4020496154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8c4004800 0x7fb8c4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.718+0000 7fb8dea00640 1 -- 172.21.15.7:0/4020496154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8c4005270 msgr2=0x7fb8c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:14.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.718+0000 7fb8dea00640 1 --2- 172.21.15.7:0/4020496154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8c4005270 0x7fb8c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.718+0000 7fb8dea00640 1 -- 172.21.15.7:0/4020496154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8c4004800 msgr2=0x7fb8c4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:14.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.718+0000 7fb8dea00640 1 --2- 172.21.15.7:0/4020496154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8c4004800 0x7fb8c4004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.718+0000 7fb8dea00640 1 -- 172.21.15.7:0/4020496154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c4003520 con 0x7fb8c4003a00 2024-04-25T11:12:14.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.718+0000 7fb8df400640 1 -- 172.21.15.7:0/4020496154 reap_dead start 2024-04-25T11:12:14.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.719+0000 7fb8dea00640 1 --2- 172.21.15.7:0/4020496154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c4003a00 0x7fb8c4003e70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009a90 tx=0x7fb8c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189093ec157ce111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:14.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.719+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0036070 con 0x7fb8c4003a00 2024-04-25T11:12:14.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.719+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d0031400 con 0x7fb8c4003a00 2024-04-25T11:12:14.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.719+0000 7fb8dd600640 1 -- 172.21.15.7:0/4020496154 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0009690 con 0x7fb8c4003a00 2024-04-25T11:12:14.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.720+0000 7fb8dfe00640 1 -- 172.21.15.7:0/4020496154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c4003a00 msgr2=0x7fb8c4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.720+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/4020496154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c4003a00 0x7fb8c4003e70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009a90 tx=0x7fb8c4005830 comp rx=0 tx=0).stop 2024-04-25T11:12:14.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.720+0000 7fb8dfe00640 1 -- 172.21.15.7:0/4020496154 shutdown_connections 2024-04-25T11:12:14.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.720+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/4020496154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c4003a00 0x7fb8c4003e70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.720+0000 7fb8dfe00640 1 -- 172.21.15.7:0/4020496154 >> 172.21.15.7:0/4020496154 conn(0x7fb8e0073160 msgr2=0x7fb8e0073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:14.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.720+0000 7fb8dfe00640 1 -- 172.21.15.7:0/4020496154 shutdown_connections 2024-04-25T11:12:14.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.720+0000 7fb8dfe00640 1 -- 172.21.15.7:0/4020496154 wait complete. 2024-04-25T11:12:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.723+0000 7fb8dfe00640 1 Processor -- start 2024-04-25T11:12:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.723+0000 7fb8dfe00640 1 -- start start 2024-04-25T11:12:14.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.724+0000 7fb8dfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8c4003a00 0x7fb8e014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.724+0000 7fb8dfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e014d4e0 0x7fb8e014d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8dfe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e014de30 0x7fb8e014e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8dfe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e00b5e50 con 0x7fb8e014d4e0 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8dfe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e006f6f0 con 0x7fb8c4003a00 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8dfe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e00b5fd0 con 0x7fb8e014de30 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8de000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e014d4e0 0x7fb8e014d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8de000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e014d4e0 0x7fb8e014d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50932/0 (socket says 172.21.15.7:50932) 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8de000640 1 -- 172.21.15.7:0/3730988224 learned_addr learned my addr 172.21.15.7:0/3730988224 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8dea00640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8c4003a00 0x7fb8e014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8df400640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e014de30 0x7fb8e014e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8de000640 1 -- 172.21.15.7:0/3730988224 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e014de30 msgr2=0x7fb8e014e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8de000640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e014de30 0x7fb8e014e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8de000640 1 -- 172.21.15.7:0/3730988224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8c4003a00 msgr2=0x7fb8e014c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:14.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8de000640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8c4003a00 0x7fb8e014c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:14.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.725+0000 7fb8de000640 1 -- 172.21.15.7:0/3730988224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e014c560 con 0x7fb8e014d4e0 2024-04-25T11:12:14.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.726+0000 7fb8de000640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e014d4e0 0x7fb8e014d8f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800d7a0 tx=0x7fb8c800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:14.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.726+0000 7fb8d6a00640 1 -- 172.21.15.7:0/3730988224 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8019070 con 0x7fb8e014d4e0 2024-04-25T11:12:14.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.726+0000 7fb8d6a00640 1 -- 172.21.15.7:0/3730988224 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c80072d0 con 0x7fb8e014d4e0 2024-04-25T11:12:14.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.726+0000 7fb8d6a00640 1 -- 172.21.15.7:0/3730988224 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c80047e0 con 0x7fb8e014d4e0 2024-04-25T11:12:14.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.726+0000 7fb8dfe00640 1 -- 172.21.15.7:0/3730988224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e00c7d40 con 0x7fb8e014d4e0 2024-04-25T11:12:14.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.726+0000 7fb8dfe00640 1 -- 172.21.15.7:0/3730988224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e014cce0 con 0x7fb8e014d4e0 2024-04-25T11:12:14.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.728+0000 7fb8dfe00640 1 -- 172.21.15.7:0/3730988224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e00c7a90 con 0x7fb8e014d4e0 2024-04-25T11:12:14.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.729+0000 7fb8d6a00640 1 -- 172.21.15.7:0/3730988224 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb8c8004980 con 0x7fb8e014d4e0 2024-04-25T11:12:14.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.730+0000 7fb8d6a00640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8bc0744b0 0x7fb8bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:14.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.730+0000 7fb8d6a00640 1 -- 172.21.15.7:0/3730988224 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fb8c8094930 con 0x7fb8e014d4e0 2024-04-25T11:12:14.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.730+0000 7fb8dea00640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8bc0744b0 0x7fb8bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:14.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.731+0000 7fb8dea00640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8bc0744b0 0x7fb8bc0769c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0033000 tx=0x7fb8d0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:14.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.735+0000 7fb8d6a00640 1 -- 172.21.15.7:0/3730988224 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8c805f270 con 0x7fb8e014d4e0 2024-04-25T11:12:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:14.942+0000 7fb8dfe00640 1 -- 172.21.15.7:0/3730988224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb8e0050740 con 0x7fb8e014d4e0 2024-04-25T11:12:15.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.569+0000 7fb8d6a00640 1 -- 172.21.15.7:0/3730988224 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7fb8c805eb70 con 0x7fb8e014d4e0 2024-04-25T11:12:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-25T11:12:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.573+0000 7fb8dfe00640 1 -- 172.21.15.7:0/3730988224 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8bc0744b0 msgr2=0x7fb8bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.573+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8bc0744b0 0x7fb8bc0769c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0033000 tx=0x7fb8d0008030 comp rx=0 tx=0).stop 2024-04-25T11:12:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.573+0000 7fb8dfe00640 1 -- 172.21.15.7:0/3730988224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e014d4e0 msgr2=0x7fb8e014d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.573+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e014d4e0 0x7fb8e014d8f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb8c800d7a0 tx=0x7fb8c800dc60 comp rx=0 tx=0).stop 2024-04-25T11:12:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.574+0000 7fb8dfe00640 1 -- 172.21.15.7:0/3730988224 shutdown_connections 2024-04-25T11:12:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.574+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8bc0744b0 0x7fb8bc0769c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.574+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e014de30 0x7fb8e014e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.574+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8e014d4e0 0x7fb8e014d8f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.574+0000 7fb8dfe00640 1 --2- 172.21.15.7:0/3730988224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8c4003a00 0x7fb8e014c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.574+0000 7fb8dfe00640 1 -- 172.21.15.7:0/3730988224 >> 172.21.15.7:0/3730988224 conn(0x7fb8e0073160 msgr2=0x7fb8e00d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.574+0000 7fb8dfe00640 1 -- 172.21.15.7:0/3730988224 shutdown_connections 2024-04-25T11:12:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.574+0000 7fb8dfe00640 1 -- 172.21.15.7:0/3730988224 wait complete. 2024-04-25T11:12:15.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:12:15.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.881+0000 7fcc99a00640 1 Processor -- start 2024-04-25T11:12:15.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.881+0000 7fcc99a00640 1 -- start start 2024-04-25T11:12:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.884+0000 7fcc99a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcc94073930 0x7fcc94073d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.884+0000 7fcc99a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc940afe30 0x7fcc940b0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.884+0000 7fcc99a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc94074280 con 0x7fcc9415c720 2024-04-25T11:12:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.884+0000 7fcc99a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fcc940b6a00 con 0x7fcc940afe30 2024-04-25T11:12:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.884+0000 7fcc99a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcc940b4840 con 0x7fcc94073930 2024-04-25T11:12:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.884+0000 7fcc93400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc940afe30 0x7fcc940b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:15.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.884+0000 7fcc93400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc940afe30 0x7fcc940b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50934/0 (socket says 172.21.15.7:50934) 2024-04-25T11:12:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.884+0000 7fcc93400640 1 -- 172.21.15.7:0/3130966995 learned_addr learned my addr 172.21.15.7:0/3130966995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.885+0000 7fcc92000640 1 --2- 172.21.15.7:0/3130966995 >> v2:172.21.15.87:3300/0 conn(0x7fcc94073930 0x7fcc94073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.885+0000 7fcc93400640 1 -- 172.21.15.7:0/3130966995 >> v2:172.21.15.87:3300/0 conn(0x7fcc94073930 msgr2=0x7fcc94073d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.885+0000 7fcc93400640 1 --2- 172.21.15.7:0/3130966995 >> v2:172.21.15.87:3300/0 conn(0x7fcc94073930 0x7fcc94073d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.885+0000 7fcc93400640 1 -- 172.21.15.7:0/3130966995 >> v1:172.21.15.87:6789/0 conn(0x7fcc9415c720 legacy=0x7fcc940b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:15.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.885+0000 7fcc93400640 1 -- 172.21.15.7:0/3130966995 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc940b4b20 con 0x7fcc940afe30 2024-04-25T11:12:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.886+0000 7fcc93400640 1 --2- 172.21.15.7:0/3130966995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc940afe30 0x7fcc940b0240 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcc940af300 tx=0x7fcc8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22711687d3645a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.886+0000 7fcc91600640 1 -- 172.21.15.7:0/3130966995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c036070 con 0x7fcc940afe30 2024-04-25T11:12:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.886+0000 7fcc91600640 1 -- 172.21.15.7:0/3130966995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc8c031600 con 0x7fcc940afe30 2024-04-25T11:12:15.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.886+0000 7fcc91600640 1 -- 172.21.15.7:0/3130966995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c0319d0 con 0x7fcc940afe30 2024-04-25T11:12:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.887+0000 7fcc99a00640 1 -- 172.21.15.7:0/3130966995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc940afe30 msgr2=0x7fcc940b0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.887+0000 7fcc99a00640 1 --2- 172.21.15.7:0/3130966995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc940afe30 0x7fcc940b0240 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcc940af300 tx=0x7fcc8c02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:12:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.887+0000 7fcc99a00640 1 -- 172.21.15.7:0/3130966995 shutdown_connections 2024-04-25T11:12:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.887+0000 7fcc99a00640 1 --2- 172.21.15.7:0/3130966995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc940afe30 0x7fcc940b0240 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.887+0000 7fcc99a00640 1 --2- 172.21.15.7:0/3130966995 >> v2:172.21.15.87:3300/0 conn(0x7fcc94073930 0x7fcc94073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.887+0000 7fcc99a00640 1 -- 172.21.15.7:0/3130966995 >> 172.21.15.7:0/3130966995 conn(0x7fcc940d1d40 msgr2=0x7fcc94073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.887+0000 7fcc99a00640 1 -- 172.21.15.7:0/3130966995 shutdown_connections 2024-04-25T11:12:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.887+0000 7fcc99a00640 1 -- 172.21.15.7:0/3130966995 wait complete. 2024-04-25T11:12:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.890+0000 7fcc99a00640 1 Processor -- start 2024-04-25T11:12:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.890+0000 7fcc99a00640 1 -- start start 2024-04-25T11:12:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc99a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc94073930 0x7fcc94151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc99a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcc940afe30 0x7fcc941539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc99a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc9415c720 0x7fcc941514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc99a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc940b5600 con 0x7fcc94073930 2024-04-25T11:12:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc99a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcc940b50e0 con 0x7fcc9415c720 2024-04-25T11:12:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc99a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcc940b5780 con 0x7fcc940afe30 2024-04-25T11:12:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc92a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc94073930 0x7fcc94151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc92a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc94073930 0x7fcc94151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50950/0 (socket says 172.21.15.7:50950) 2024-04-25T11:12:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc92a00640 1 -- 172.21.15.7:0/3184582460 learned_addr learned my addr 172.21.15.7:0/3184582460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc92000640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcc940afe30 0x7fcc941539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc92a00640 1 -- 172.21.15.7:0/3184582460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcc940afe30 msgr2=0x7fcc941539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc92a00640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcc940afe30 0x7fcc941539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc92a00640 1 -- 172.21.15.7:0/3184582460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc9415c720 msgr2=0x7fcc941514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.892+0000 7fcc92a00640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc9415c720 0x7fcc941514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.893+0000 7fcc92a00640 1 -- 172.21.15.7:0/3184582460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc94153f10 con 0x7fcc94073930 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.893+0000 7fcc92000640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcc940afe30 0x7fcc941539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.893+0000 7fcc92a00640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc94073930 0x7fcc94151f90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c3c0 tx=0x7fcc8400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.893+0000 7fcc8aa00640 1 -- 172.21.15.7:0/3184582460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc840052d0 con 0x7fcc94073930 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.893+0000 7fcc8aa00640 1 -- 172.21.15.7:0/3184582460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc84005490 con 0x7fcc94073930 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.893+0000 7fcc8aa00640 1 -- 172.21.15.7:0/3184582460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84007260 con 0x7fcc94073930 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.893+0000 7fcc99a00640 1 -- 172.21.15.7:0/3184582460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9415f040 con 0x7fcc94073930 2024-04-25T11:12:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.893+0000 7fcc99a00640 1 -- 172.21.15.7:0/3184582460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc9415f2e0 con 0x7fcc94073930 2024-04-25T11:12:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.895+0000 7fcc99a00640 1 -- 172.21.15.7:0/3184582460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc94070df0 con 0x7fcc94073930 2024-04-25T11:12:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.897+0000 7fcc8aa00640 1 -- 172.21.15.7:0/3184582460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcc84007400 con 0x7fcc94073930 2024-04-25T11:12:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.897+0000 7fcc8aa00640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcc740744e0 0x7fcc740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.898+0000 7fcc8aa00640 1 -- 172.21.15.7:0/3184582460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7fcc84094c00 con 0x7fcc94073930 2024-04-25T11:12:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.898+0000 7fcc92000640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcc740744e0 0x7fcc740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.899+0000 7fcc92000640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcc740744e0 0x7fcc740769f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcc80006fd0 tx=0x7fcc80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:15.902+0000 7fcc8aa00640 1 -- 172.21.15.7:0/3184582460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc94074280 con 0x7fcc94073930 2024-04-25T11:12:16.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.107+0000 7fcc99a00640 1 -- 172.21.15.7:0/3184582460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcc940b1fd0 con 0x7fcc94073930 2024-04-25T11:12:16.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.585+0000 7fcc8aa00640 1 -- 172.21.15.7:0/3184582460 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7fcc8405f3c0 con 0x7fcc94073930 2024-04-25T11:12:16.586 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-25T11:12:16.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.589+0000 7fcc99a00640 1 -- 172.21.15.7:0/3184582460 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcc740744e0 msgr2=0x7fcc740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:16.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.589+0000 7fcc99a00640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcc740744e0 0x7fcc740769f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcc80006fd0 tx=0x7fcc80008040 comp rx=0 tx=0).stop 2024-04-25T11:12:16.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.590+0000 7fcc99a00640 1 -- 172.21.15.7:0/3184582460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc94073930 msgr2=0x7fcc94151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:16.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.590+0000 7fcc99a00640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc94073930 0x7fcc94151f90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c3c0 tx=0x7fcc8400c880 comp rx=0 tx=0).stop 2024-04-25T11:12:16.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.590+0000 7fcc99a00640 1 -- 172.21.15.7:0/3184582460 shutdown_connections 2024-04-25T11:12:16.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.590+0000 7fcc99a00640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcc740744e0 0x7fcc740769f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:16.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.590+0000 7fcc99a00640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc9415c720 0x7fcc941514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:16.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.590+0000 7fcc99a00640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcc940afe30 0x7fcc941539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:16.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.590+0000 7fcc99a00640 1 --2- 172.21.15.7:0/3184582460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc94073930 0x7fcc94151f90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:16.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.590+0000 7fcc99a00640 1 -- 172.21.15.7:0/3184582460 >> 172.21.15.7:0/3184582460 conn(0x7fcc940d1d40 msgr2=0x7fcc9406d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:16.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.591+0000 7fcc99a00640 1 -- 172.21.15.7:0/3184582460 shutdown_connections 2024-04-25T11:12:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.591+0000 7fcc99a00640 1 -- 172.21.15.7:0/3184582460 wait complete. 2024-04-25T11:12:16.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:12:16.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.891+0000 7f7b67e00640 1 Processor -- start 2024-04-25T11:12:16.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.891+0000 7f7b67e00640 1 -- start start 2024-04-25T11:12:16.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.893+0000 7f7b67e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b68144a80 0x7f7b68146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:16.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.893+0000 7f7b67e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b680b4f80 0x7f7b680b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:16.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.893+0000 7f7b67e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b680b66b0 con 0x7f7b68145e40 2024-04-25T11:12:16.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.893+0000 7f7b67e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7b68145b70 con 0x7f7b680b4f80 2024-04-25T11:12:16.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.893+0000 7f7b67e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7b680b58d0 con 0x7f7b68144a80 2024-04-25T11:12:16.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.893+0000 7f7b67400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b680b4f80 0x7f7b680b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:16.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.894+0000 7f7b66a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7b68145e40 0x7f7b680b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59650/0 (socket says 172.21.15.7:59650) 2024-04-25T11:12:16.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.894+0000 7f7b66a00640 1 -- 172.21.15.7:0/4291853670 learned_addr learned my addr 172.21.15.7:0/4291853670 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:16.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.894+0000 7f7b5fe00640 1 --2- 172.21.15.7:0/4291853670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b68144a80 0x7f7b68146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:16.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.894+0000 7f7b67400640 1 -- 172.21.15.7:0/4291853670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b68144a80 msgr2=0x7f7b68146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:16.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.894+0000 7f7b67400640 1 --2- 172.21.15.7:0/4291853670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b68144a80 0x7f7b68146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:16.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.894+0000 7f7b67400640 1 -- 172.21.15.7:0/4291853670 >> v1:172.21.15.7:6789/0 conn(0x7f7b68145e40 legacy=0x7f7b680b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:16.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.894+0000 7f7b67400640 1 -- 172.21.15.7:0/4291853670 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b680b5b30 con 0x7f7b680b4f80 2024-04-25T11:12:16.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.894+0000 7f7b66000640 1 -- 172.21.15.7:0/4291853670 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 138803039 0 0) 0x7f7b680b66b0 con 0x7f7b68145e40 2024-04-25T11:12:16.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.895+0000 7f7b67400640 1 --2- 172.21.15.7:0/4291853670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b680b4f80 0x7f7b680b5390 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7b680b6880 tx=0x7f7b6002a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1483eccbf04f2daa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:16.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.896+0000 7f7b66000640 1 -- 172.21.15.7:0/4291853670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60007740 con 0x7f7b680b4f80 2024-04-25T11:12:16.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.896+0000 7f7b66000640 1 -- 172.21.15.7:0/4291853670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b600078e0 con 0x7f7b680b4f80 2024-04-25T11:12:16.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.896+0000 7f7b66000640 1 -- 172.21.15.7:0/4291853670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60007cb0 con 0x7f7b680b4f80 2024-04-25T11:12:16.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.896+0000 7f7b67e00640 1 -- 172.21.15.7:0/4291853670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b680b4f80 msgr2=0x7f7b680b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:16.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.896+0000 7f7b67e00640 1 --2- 172.21.15.7:0/4291853670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b680b4f80 0x7f7b680b5390 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7b680b6880 tx=0x7f7b6002a5b0 comp rx=0 tx=0).stop 2024-04-25T11:12:16.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.897+0000 7f7b67e00640 1 -- 172.21.15.7:0/4291853670 shutdown_connections 2024-04-25T11:12:16.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.897+0000 7f7b67e00640 1 --2- 172.21.15.7:0/4291853670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b680b4f80 0x7f7b680b5390 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:16.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.897+0000 7f7b67e00640 1 --2- 172.21.15.7:0/4291853670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b68144a80 0x7f7b68146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:16.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.897+0000 7f7b67e00640 1 -- 172.21.15.7:0/4291853670 >> 172.21.15.7:0/4291853670 conn(0x7f7b680740f0 msgr2=0x7f7b6806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:16.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.897+0000 7f7b67e00640 1 -- 172.21.15.7:0/4291853670 shutdown_connections 2024-04-25T11:12:16.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.897+0000 7f7b67e00640 1 -- 172.21.15.7:0/4291853670 wait complete. 2024-04-25T11:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.899+0000 7f7b67e00640 1 Processor -- start 2024-04-25T11:12:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.900+0000 7f7b67e00640 1 -- start start 2024-04-25T11:12:16.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.901+0000 7f7b67e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b680b4f80 0x7f7b6815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:16.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.901+0000 7f7b67e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b68144a80 0x7f7b68157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.901+0000 7f7b67e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b68145e40 0x7f7b681580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.901+0000 7f7b67e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b680b63d0 con 0x7f7b68145e40 2024-04-25T11:12:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.901+0000 7f7b67e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7b680b60f0 con 0x7f7b68144a80 2024-04-25T11:12:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.901+0000 7f7b67e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6806f730 con 0x7f7b680b4f80 2024-04-25T11:12:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.901+0000 7f7b67400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b68145e40 0x7f7b681580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.901+0000 7f7b67400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b68145e40 0x7f7b681580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50970/0 (socket says 172.21.15.7:50970) 2024-04-25T11:12:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.901+0000 7f7b67400640 1 -- 172.21.15.7:0/192077896 learned_addr learned my addr 172.21.15.7:0/192077896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:16.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.902+0000 7f7b66a00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b680b4f80 0x7f7b6815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:16.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.902+0000 7f7b5fe00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b68144a80 0x7f7b68157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:16.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.902+0000 7f7b67400640 1 -- 172.21.15.7:0/192077896 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b680b4f80 msgr2=0x7f7b6815cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.902+0000 7f7b67400640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b680b4f80 0x7f7b6815cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.902+0000 7f7b67400640 1 -- 172.21.15.7:0/192077896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b68144a80 msgr2=0x7f7b68157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.902+0000 7f7b67400640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b68144a80 0x7f7b68157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.902+0000 7f7b67400640 1 -- 172.21.15.7:0/192077896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6815d080 con 0x7f7b68145e40 2024-04-25T11:12:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.902+0000 7f7b66a00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b680b4f80 0x7f7b6815cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.902+0000 7f7b67400640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b68145e40 0x7f7b681580d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7b6002a5f0 tx=0x7f7b60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.903+0000 7f7b5ea00640 1 -- 172.21.15.7:0/192077896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6003f070 con 0x7f7b68145e40 2024-04-25T11:12:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.903+0000 7f7b5ea00640 1 -- 172.21.15.7:0/192077896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b60031bc0 con 0x7f7b68145e40 2024-04-25T11:12:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.903+0000 7f7b5ea00640 1 -- 172.21.15.7:0/192077896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60009450 con 0x7f7b68145e40 2024-04-25T11:12:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.903+0000 7f7b67e00640 1 -- 172.21.15.7:0/192077896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6813c470 con 0x7f7b68145e40 2024-04-25T11:12:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.903+0000 7f7b67e00640 1 -- 172.21.15.7:0/192077896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6813d700 con 0x7f7b68145e40 2024-04-25T11:12:16.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.905+0000 7f7b67e00640 1 -- 172.21.15.7:0/192077896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b68050740 con 0x7f7b68145e40 2024-04-25T11:12:16.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.906+0000 7f7b5ea00640 1 -- 172.21.15.7:0/192077896 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7b60003990 con 0x7f7b68145e40 2024-04-25T11:12:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.907+0000 7f7b5ea00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7b440744e0 0x7f7b440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.907+0000 7f7b5ea00640 1 -- 172.21.15.7:0/192077896 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7f7b600b6170 con 0x7f7b68145e40 2024-04-25T11:12:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.908+0000 7f7b66a00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7b440744e0 0x7f7b440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.908+0000 7f7b66a00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7b440744e0 0x7f7b440769f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7b58000f60 tx=0x7f7b58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:16.913+0000 7f7b5ea00640 1 -- 172.21.15.7:0/192077896 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7b60080860 con 0x7f7b68145e40 2024-04-25T11:12:17.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.155+0000 7f7b67e00640 1 -- 172.21.15.7:0/192077896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7b68160400 con 0x7f7b68145e40 2024-04-25T11:12:17.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.590+0000 7f7b5ea00640 1 -- 172.21.15.7:0/192077896 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f7b60080160 con 0x7f7b68145e40 2024-04-25T11:12:17.591 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T11:12:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.594+0000 7f7b67e00640 1 -- 172.21.15.7:0/192077896 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7b440744e0 msgr2=0x7f7b440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.594+0000 7f7b67e00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7b440744e0 0x7f7b440769f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7b58000f60 tx=0x7f7b58009040 comp rx=0 tx=0).stop 2024-04-25T11:12:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.594+0000 7f7b67e00640 1 -- 172.21.15.7:0/192077896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b68145e40 msgr2=0x7f7b681580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.594+0000 7f7b67e00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b68145e40 0x7f7b681580d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7b6002a5f0 tx=0x7f7b60004a40 comp rx=0 tx=0).stop 2024-04-25T11:12:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.594+0000 7f7b67e00640 1 -- 172.21.15.7:0/192077896 shutdown_connections 2024-04-25T11:12:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.595+0000 7f7b67e00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7b440744e0 0x7f7b440769f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.595+0000 7f7b67e00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b68145e40 0x7f7b681580d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.595+0000 7f7b67e00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b68144a80 0x7f7b68157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.595+0000 7f7b67e00640 1 --2- 172.21.15.7:0/192077896 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b680b4f80 0x7f7b6815cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.595+0000 7f7b67e00640 1 -- 172.21.15.7:0/192077896 >> 172.21.15.7:0/192077896 conn(0x7f7b680740f0 msgr2=0x7f7b680684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.595+0000 7f7b67e00640 1 -- 172.21.15.7:0/192077896 shutdown_connections 2024-04-25T11:12:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.595+0000 7f7b67e00640 1 -- 172.21.15.7:0/192077896 wait complete. 2024-04-25T11:12:17.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:12:17.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.894+0000 7fd50d000640 1 Processor -- start 2024-04-25T11:12:17.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.894+0000 7fd50d000640 1 -- start start 2024-04-25T11:12:17.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.896+0000 7fd50d000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd5080aed40 0x7fd50815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:17.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.896+0000 7fd50d000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd5080af420 0x7fd5080af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:17.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.896+0000 7fd50d000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5080cebb0 con 0x7fd5080b5220 2024-04-25T11:12:17.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.896+0000 7fd50d000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd50815e1f0 con 0x7fd5080af420 2024-04-25T11:12:17.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.896+0000 7fd50d000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd5080afd70 con 0x7fd5080aed40 2024-04-25T11:12:17.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.896+0000 7fd507e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd5080af420 0x7fd5080af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:17.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.897+0000 7fd507e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd5080af420 0x7fd5080af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50974/0 (socket says 172.21.15.7:50974) 2024-04-25T11:12:17.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.897+0000 7fd507e00640 1 -- 172.21.15.7:0/2584992831 learned_addr learned my addr 172.21.15.7:0/2584992831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:17.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.897+0000 7fd507400640 1 --2- 172.21.15.7:0/2584992831 >> v2:172.21.15.87:3300/0 conn(0x7fd5080aed40 0x7fd50815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:17.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.897+0000 7fd507e00640 1 -- 172.21.15.7:0/2584992831 >> v2:172.21.15.87:3300/0 conn(0x7fd5080aed40 msgr2=0x7fd50815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:17.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.897+0000 7fd507e00640 1 --2- 172.21.15.7:0/2584992831 >> v2:172.21.15.87:3300/0 conn(0x7fd5080aed40 0x7fd50815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:17.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.897+0000 7fd507e00640 1 -- 172.21.15.7:0/2584992831 >> v1:172.21.15.7:6789/0 conn(0x7fd5080b5220 legacy=0x7fd5080ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:17.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.897+0000 7fd507e00640 1 -- 172.21.15.7:0/2584992831 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5080b64b0 con 0x7fd5080af420 2024-04-25T11:12:17.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.897+0000 7fd506000640 1 -- 172.21.15.7:0/2584992831 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3847646912 0 0) 0x7fd4f4001110 con 0x7fd5080b5220 2024-04-25T11:12:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.898+0000 7fd507e00640 1 --2- 172.21.15.7:0/2584992831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5080af420 0x7fd5080af830 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd5080725e0 tx=0x7fd50002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f768a1ec8f1ea3c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.899+0000 7fd506000640 1 -- 172.21.15.7:0/2584992831 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd500036070 con 0x7fd5080af420 2024-04-25T11:12:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.899+0000 7fd506000640 1 -- 172.21.15.7:0/2584992831 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd500031600 con 0x7fd5080af420 2024-04-25T11:12:17.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.899+0000 7fd506000640 1 -- 172.21.15.7:0/2584992831 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5000319d0 con 0x7fd5080af420 2024-04-25T11:12:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.899+0000 7fd50d000640 1 -- 172.21.15.7:0/2584992831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5080af420 msgr2=0x7fd5080af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.899+0000 7fd50d000640 1 --2- 172.21.15.7:0/2584992831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5080af420 0x7fd5080af830 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd5080725e0 tx=0x7fd50002a730 comp rx=0 tx=0).stop 2024-04-25T11:12:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.899+0000 7fd50d000640 1 -- 172.21.15.7:0/2584992831 shutdown_connections 2024-04-25T11:12:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.899+0000 7fd50d000640 1 --2- 172.21.15.7:0/2584992831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5080af420 0x7fd5080af830 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.899+0000 7fd50d000640 1 --2- 172.21.15.7:0/2584992831 >> v2:172.21.15.87:3300/0 conn(0x7fd5080aed40 0x7fd50815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:17.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.899+0000 7fd50d000640 1 -- 172.21.15.7:0/2584992831 >> 172.21.15.7:0/2584992831 conn(0x7fd5080b3820 msgr2=0x7fd5080b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.899+0000 7fd50d000640 1 -- 172.21.15.7:0/2584992831 shutdown_connections 2024-04-25T11:12:17.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.899+0000 7fd50d000640 1 -- 172.21.15.7:0/2584992831 wait complete. 2024-04-25T11:12:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.902+0000 7fd50d000640 1 Processor -- start 2024-04-25T11:12:17.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.902+0000 7fd50d000640 1 -- start start 2024-04-25T11:12:17.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.903+0000 7fd50d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5080aed40 0x7fd50815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd50d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5080af420 0x7fd50815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd50d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5080b5220 0x7fd508156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd50d000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd50806cbd0 con 0x7fd5080af420 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd50d000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd50806c8f0 con 0x7fd5080aed40 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd50d000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd50806ceb0 con 0x7fd5080b5220 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd506a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5080af420 0x7fd50815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd506a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5080af420 0x7fd50815ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50978/0 (socket says 172.21.15.7:50978) 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd506a00640 1 -- 172.21.15.7:0/919972193 learned_addr learned my addr 172.21.15.7:0/919972193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd506a00640 1 -- 172.21.15.7:0/919972193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5080b5220 msgr2=0x7fd508156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd507400640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5080aed40 0x7fd50815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd507e00640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5080b5220 0x7fd508156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd506a00640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5080b5220 0x7fd508156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:17.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd506a00640 1 -- 172.21.15.7:0/919972193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5080aed40 msgr2=0x7fd50815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd506a00640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5080aed40 0x7fd50815c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.904+0000 7fd506a00640 1 -- 172.21.15.7:0/919972193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd50815fcb0 con 0x7fd5080af420 2024-04-25T11:12:17.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.905+0000 7fd506a00640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5080af420 0x7fd50815ef80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c3f0 tx=0x7fd4f400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.905+0000 7fd4ff400640 1 -- 172.21.15.7:0/919972193 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4017070 con 0x7fd5080af420 2024-04-25T11:12:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.905+0000 7fd4ff400640 1 -- 172.21.15.7:0/919972193 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f40035f0 con 0x7fd5080af420 2024-04-25T11:12:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.905+0000 7fd50d000640 1 -- 172.21.15.7:0/919972193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd50815ff90 con 0x7fd5080af420 2024-04-25T11:12:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.905+0000 7fd4ff400640 1 -- 172.21.15.7:0/919972193 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40039e0 con 0x7fd5080af420 2024-04-25T11:12:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.905+0000 7fd50d000640 1 -- 172.21.15.7:0/919972193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5080cc1f0 con 0x7fd5080af420 2024-04-25T11:12:17.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.906+0000 7fd50d000640 1 -- 172.21.15.7:0/919972193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5080b5010 con 0x7fd5080af420 2024-04-25T11:12:17.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.908+0000 7fd4ff400640 1 -- 172.21.15.7:0/919972193 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd4f401a070 con 0x7fd5080af420 2024-04-25T11:12:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.909+0000 7fd4ff400640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4e80744e0 0x7fd4e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.909+0000 7fd4ff400640 1 -- 172.21.15.7:0/919972193 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fd4f4094810 con 0x7fd5080af420 2024-04-25T11:12:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.909+0000 7fd507400640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4e80744e0 0x7fd4e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:17.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.910+0000 7fd507400640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4e80744e0 0x7fd4e80769f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8006fd0 tx=0x7fd4f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:17.914+0000 7fd4ff400640 1 -- 172.21.15.7:0/919972193 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd5080b5010 con 0x7fd5080af420 2024-04-25T11:12:18.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.108+0000 7fd50d000640 1 -- 172.21.15.7:0/919972193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd50815fa40 con 0x7fd5080af420 2024-04-25T11:12:18.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.111+0000 7fd4ff400640 1 -- 172.21.15.7:0/919972193 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16859 (secure 0 0 0) 0x7fd5080cebb0 con 0x7fd5080af420 2024-04-25T11:12:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:18.112 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":39,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:12:17.156704+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:12:14.961811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T11:12:16.128549+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:12:18.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 -- 172.21.15.7:0/919972193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4e80744e0 msgr2=0x7fd4e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4e80744e0 0x7fd4e80769f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8006fd0 tx=0x7fd4f8008040 comp rx=0 tx=0).stop 2024-04-25T11:12:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 -- 172.21.15.7:0/919972193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5080af420 msgr2=0x7fd50815ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5080af420 0x7fd50815ef80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c3f0 tx=0x7fd4f400c8b0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 -- 172.21.15.7:0/919972193 shutdown_connections 2024-04-25T11:12:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4e80744e0 0x7fd4e80769f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5080b5220 0x7fd508156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5080af420 0x7fd50815ef80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 --2- 172.21.15.7:0/919972193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5080aed40 0x7fd50815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 -- 172.21.15.7:0/919972193 >> 172.21.15.7:0/919972193 conn(0x7fd5080b3820 msgr2=0x7fd5080b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 -- 172.21.15.7:0/919972193 shutdown_connections 2024-04-25T11:12:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.114+0000 7fd50d000640 1 -- 172.21.15.7:0/919972193 wait complete. 2024-04-25T11:12:18.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:12:18.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.418+0000 7f136be00640 1 Processor -- start 2024-04-25T11:12:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.419+0000 7f136be00640 1 -- start start 2024-04-25T11:12:18.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.421+0000 7f136be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f136415ef90 0x7f136409f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.421+0000 7f136be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13640af540 0x7f13640cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.421+0000 7f136be00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f136415f4e0 con 0x7f136415bc40 2024-04-25T11:12:18.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.421+0000 7f136be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1364071190 con 0x7f13640af540 2024-04-25T11:12:18.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.421+0000 7f136be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13640b0560 con 0x7f136415ef90 2024-04-25T11:12:18.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.421+0000 7f1368a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f136415bc40 0x7f13640aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59672/0 (socket says 172.21.15.7:59672) 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.421+0000 7f1368a00640 1 -- 172.21.15.7:0/804534511 learned_addr learned my addr 172.21.15.7:0/804534511 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.422+0000 7f1369400640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f136415ef90 0x7f136409f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.422+0000 7f1369e00640 1 --2- 172.21.15.7:0/804534511 >> v2:172.21.15.87:3300/0 conn(0x7f13640af540 0x7f13640cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.422+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1595695281 0 0) 0x7f136415f4e0 con 0x7f136415bc40 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.422+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f134c003520 con 0x7f136415bc40 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.422+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 581231496 0 0) 0x7f134c003520 con 0x7f136415bc40 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.422+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f136415ef90 msgr2=0x7f136409f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f136415ef90 0x7f136409f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 >> v2:172.21.15.87:3300/0 conn(0x7f13640af540 msgr2=0x7f13640cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 --2- 172.21.15.7:0/804534511 >> v2:172.21.15.87:3300/0 conn(0x7f13640af540 0x7f13640cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13640b0840 con 0x7f136415bc40 2024-04-25T11:12:18.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1350001670 con 0x7f136415bc40 2024-04-25T11:12:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 >> v1:172.21.15.7:6789/0 conn(0x7f136415bc40 legacy=0x7f13640aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f136415ef90 0x7f134c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f134c003f80 0x7f134c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f134c004d40 0x7f134c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1364071190 con 0x7f134c004d40 2024-04-25T11:12:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13640b0560 con 0x7f134c003f80 2024-04-25T11:12:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f134c00ff30 con 0x7f136415ef90 2024-04-25T11:12:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1369e00640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f134c004d40 0x7f134c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.423+0000 7f1369400640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f136415ef90 0x7f134c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.424+0000 7f1368a00640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f134c003f80 0x7f134c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.424+0000 7f1369e00640 1 -- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f136415ef90 msgr2=0x7f134c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.424+0000 7f1369e00640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f136415ef90 0x7f134c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.424+0000 7f1369e00640 1 -- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f134c003f80 msgr2=0x7f134c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.424+0000 7f1369e00640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f134c003f80 0x7f134c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.424+0000 7f1369e00640 1 -- 172.21.15.7:0/804534511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c003520 con 0x7f134c004d40 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.424+0000 7f1368a00640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f134c003f80 0x7f134c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.425+0000 7f1369e00640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f134c004d40 0x7f134c005150 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f135c009f90 tx=0x7f134c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a2d7b8417990e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.425+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c009c10 con 0x7f134c004d40 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.425+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135c036070 con 0x7f134c004d40 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.425+0000 7f1363e00640 1 -- 172.21.15.7:0/804534511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c031880 con 0x7f134c004d40 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.426+0000 7f136be00640 1 -- 172.21.15.7:0/804534511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f134c004d40 msgr2=0x7f134c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.426+0000 7f136be00640 1 --2- 172.21.15.7:0/804534511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f134c004d40 0x7f134c005150 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f135c009f90 tx=0x7f134c005830 comp rx=0 tx=0).stop 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.426+0000 7f1369e00640 1 -- 172.21.15.7:0/804534511 reap_dead start 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.426+0000 7f136be00640 1 -- 172.21.15.7:0/804534511 shutdown_connections 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.426+0000 7f136be00640 1 -- 172.21.15.7:0/804534511 >> 172.21.15.7:0/804534511 conn(0x7f1364074200 msgr2=0x7f136406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.426+0000 7f136be00640 1 -- 172.21.15.7:0/804534511 shutdown_connections 2024-04-25T11:12:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.426+0000 7f136be00640 1 -- 172.21.15.7:0/804534511 wait complete. 2024-04-25T11:12:18.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.428+0000 7f136be00640 1 Processor -- start 2024-04-25T11:12:18.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.428+0000 7f136be00640 1 -- start start 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.430+0000 7f136be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13641579e0 0x7f13641529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.430+0000 7f136be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1364152f10 0x7f1364159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.430+0000 7f136be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1364159b80 0x7f13641f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.430+0000 7f136be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13640b1320 con 0x7f1364152f10 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.430+0000 7f136be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13640b0e00 con 0x7f13641579e0 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.430+0000 7f136be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13640afb80 con 0x7f1364159b80 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.430+0000 7f1368a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1364152f10 0x7f1364159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.430+0000 7f1368a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1364152f10 0x7f1364159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50992/0 (socket says 172.21.15.7:50992) 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.430+0000 7f1368a00640 1 -- 172.21.15.7:0/2940934080 learned_addr learned my addr 172.21.15.7:0/2940934080 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.431+0000 7f1369400640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13641579e0 0x7f13641529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.431+0000 7f1369e00640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1364159b80 0x7f13641f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.431+0000 7f1368a00640 1 -- 172.21.15.7:0/2940934080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1364159b80 msgr2=0x7f13641f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.431+0000 7f1368a00640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1364159b80 0x7f13641f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.431+0000 7f1368a00640 1 -- 172.21.15.7:0/2940934080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13641579e0 msgr2=0x7f13641529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.431+0000 7f1368a00640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13641579e0 0x7f13641529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.431+0000 7f1368a00640 1 -- 172.21.15.7:0/2940934080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1364159f90 con 0x7f1364152f10 2024-04-25T11:12:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.431+0000 7f1369e00640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1364159b80 0x7f13641f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.431+0000 7f1368a00640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1364152f10 0x7f1364159640 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f135000d8e0 tx=0x7f135000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.431+0000 7f1361600640 1 -- 172.21.15.7:0/2940934080 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350019070 con 0x7f1364152f10 2024-04-25T11:12:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.432+0000 7f1361600640 1 -- 172.21.15.7:0/2940934080 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1350004cb0 con 0x7f1364152f10 2024-04-25T11:12:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.432+0000 7f1361600640 1 -- 172.21.15.7:0/2940934080 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350004e60 con 0x7f1364152f10 2024-04-25T11:12:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.432+0000 7f136be00640 1 -- 172.21.15.7:0/2940934080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13641f5a30 con 0x7f1364152f10 2024-04-25T11:12:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.432+0000 7f136be00640 1 -- 172.21.15.7:0/2940934080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13641571e0 con 0x7f1364152f10 2024-04-25T11:12:18.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.433+0000 7f136be00640 1 -- 172.21.15.7:0/2940934080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136406a100 con 0x7f1364152f10 2024-04-25T11:12:18.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.434+0000 7f1361600640 1 -- 172.21.15.7:0/2940934080 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f135001c070 con 0x7f1364152f10 2024-04-25T11:12:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.435+0000 7f1361600640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f13440744b0 0x7f13440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.435+0000 7f1361600640 1 -- 172.21.15.7:0/2940934080 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1350094fe0 con 0x7f1364152f10 2024-04-25T11:12:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.435+0000 7f1369400640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f13440744b0 0x7f13440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.436+0000 7f1369400640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f13440744b0 0x7f13440769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f13640cdc80 tx=0x7f1358008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:18.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.440+0000 7f1361600640 1 -- 172.21.15.7:0/2940934080 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1364159f90 con 0x7f1364152f10 2024-04-25T11:12:18.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.654+0000 7f136be00640 1 -- 172.21.15.7:0/2940934080 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13640d07f0 con 0x7f1364152f10 2024-04-25T11:12:18.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.657+0000 7f1361600640 1 -- 172.21.15.7:0/2940934080 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16846 (secure 0 0 0) 0x7f135005f680 con 0x7f1364152f10 2024-04-25T11:12:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:18.658 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:12:18.606045+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:12:14.961811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T11:12:16.128549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:12:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 -- 172.21.15.7:0/2940934080 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f13440744b0 msgr2=0x7f13440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f13440744b0 0x7f13440769c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f13640cdc80 tx=0x7f1358008040 comp rx=0 tx=0).stop 2024-04-25T11:12:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 -- 172.21.15.7:0/2940934080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1364152f10 msgr2=0x7f1364159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1364152f10 0x7f1364159640 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f135000d8e0 tx=0x7f135000dda0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 -- 172.21.15.7:0/2940934080 shutdown_connections 2024-04-25T11:12:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f13440744b0 0x7f13440769c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1364159b80 0x7f13641f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1364152f10 0x7f1364159640 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 --2- 172.21.15.7:0/2940934080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13641579e0 0x7f13641529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 -- 172.21.15.7:0/2940934080 >> 172.21.15.7:0/2940934080 conn(0x7f1364074200 msgr2=0x7f13640b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 -- 172.21.15.7:0/2940934080 shutdown_connections 2024-04-25T11:12:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.660+0000 7f136be00640 1 -- 172.21.15.7:0/2940934080 wait complete. 2024-04-25T11:12:18.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:12:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.965+0000 7fa7cbe00640 1 Processor -- start 2024-04-25T11:12:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.965+0000 7fa7cbe00640 1 -- start start 2024-04-25T11:12:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.967+0000 7fa7cbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa7cc0b4a10 0x7fa7cc0d53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.967+0000 7fa7cbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc160090 0x7fa7cc0b9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.967+0000 7fa7cbe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7cc1604a0 con 0x7fa7cc0b7440 2024-04-25T11:12:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.967+0000 7fa7cbe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa7cc0d1490 con 0x7fa7cc0b4a10 2024-04-25T11:12:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.967+0000 7fa7cbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa7cc0b9550 con 0x7fa7cc160090 2024-04-25T11:12:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.968+0000 7fa7ca000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc160090 0x7fa7cc0b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.968+0000 7fa7cb400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa7cc0b7440 0x7fa7cc0b7850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:48886/0 (socket says 172.21.15.7:48886) 2024-04-25T11:12:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.968+0000 7fa7caa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa7cc0b4a10 0x7fa7cc0d53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.968+0000 7fa7cb400640 1 -- 172.21.15.7:0/4198384959 learned_addr learned my addr 172.21.15.7:0/4198384959 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.969+0000 7fa7c9600640 1 -- 172.21.15.7:0/4198384959 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 251766499 0 0) 0x7fa7cc1604a0 con 0x7fa7cc0b7440 2024-04-25T11:12:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.969+0000 7fa7c9600640 1 -- 172.21.15.7:0/4198384959 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7b0003520 con 0x7fa7cc0b7440 2024-04-25T11:12:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.969+0000 7fa7ca000640 1 -- 172.21.15.7:0/4198384959 >> v2:172.21.15.87:3300/0 conn(0x7fa7cc0b4a10 msgr2=0x7fa7cc0d53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.969+0000 7fa7ca000640 1 --2- 172.21.15.7:0/4198384959 >> v2:172.21.15.87:3300/0 conn(0x7fa7cc0b4a10 0x7fa7cc0d53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.969+0000 7fa7ca000640 1 -- 172.21.15.7:0/4198384959 >> v1:172.21.15.87:6789/0 conn(0x7fa7cc0b7440 legacy=0x7fa7cc0b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.969+0000 7fa7ca000640 1 -- 172.21.15.7:0/4198384959 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc0d0880 con 0x7fa7cc160090 2024-04-25T11:12:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.969+0000 7fa7caa00640 1 --2- 172.21.15.7:0/4198384959 >> v2:172.21.15.87:3300/0 conn(0x7fa7cc0b4a10 0x7fa7cc0d53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:12:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.971+0000 7fa7ca000640 1 --2- 172.21.15.7:0/4198384959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc160090 0x7fa7cc0b9010 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40097a0 tx=0x7fa7b402a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207a765b8df7ad90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.971+0000 7fa7c9600640 1 -- 172.21.15.7:0/4198384959 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4037070 con 0x7fa7cc160090 2024-04-25T11:12:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.972+0000 7fa7c9600640 1 -- 172.21.15.7:0/4198384959 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b402ae40 con 0x7fa7cc160090 2024-04-25T11:12:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.972+0000 7fa7c9600640 1 -- 172.21.15.7:0/4198384959 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4032660 con 0x7fa7cc160090 2024-04-25T11:12:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.973+0000 7fa7cbe00640 1 -- 172.21.15.7:0/4198384959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc160090 msgr2=0x7fa7cc0b9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.973+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/4198384959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc160090 0x7fa7cc0b9010 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40097a0 tx=0x7fa7b402a0f0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.973+0000 7fa7cbe00640 1 -- 172.21.15.7:0/4198384959 shutdown_connections 2024-04-25T11:12:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.973+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/4198384959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc160090 0x7fa7cc0b9010 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.973+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/4198384959 >> v2:172.21.15.87:3300/0 conn(0x7fa7cc0b4a10 0x7fa7cc0d53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.973+0000 7fa7cbe00640 1 -- 172.21.15.7:0/4198384959 >> 172.21.15.7:0/4198384959 conn(0x7fa7cc0d5de0 msgr2=0x7fa7cc0d61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.973+0000 7fa7cbe00640 1 -- 172.21.15.7:0/4198384959 shutdown_connections 2024-04-25T11:12:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.973+0000 7fa7cbe00640 1 -- 172.21.15.7:0/4198384959 wait complete. 2024-04-25T11:12:18.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.976+0000 7fa7cbe00640 1 Processor -- start 2024-04-25T11:12:18.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.976+0000 7fa7cbe00640 1 -- start start 2024-04-25T11:12:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.978+0000 7fa7cbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7cc0b4a10 0x7fa7cc160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.978+0000 7fa7cbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7cc0b7440 0x7fa7cc0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.978+0000 7fa7cbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc160090 0x7fa7cc162020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.978+0000 7fa7cbe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc0d1120 con 0x7fa7cc0b7440 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.978+0000 7fa7cbe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc0d0e40 con 0x7fa7cc0b4a10 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.978+0000 7fa7cbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa7cc163d40 con 0x7fa7cc160090 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.978+0000 7fa7ca000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7cc0b7440 0x7fa7cc0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.979+0000 7fa7ca000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7cc0b7440 0x7fa7cc0cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51004/0 (socket says 172.21.15.7:51004) 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.979+0000 7fa7ca000640 1 -- 172.21.15.7:0/3482263026 learned_addr learned my addr 172.21.15.7:0/3482263026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.979+0000 7fa7caa00640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7cc0b4a10 0x7fa7cc160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.979+0000 7fa7cb400640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc160090 0x7fa7cc162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.979+0000 7fa7ca000640 1 -- 172.21.15.7:0/3482263026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc160090 msgr2=0x7fa7cc162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.979+0000 7fa7ca000640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc160090 0x7fa7cc162020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.979+0000 7fa7ca000640 1 -- 172.21.15.7:0/3482263026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7cc0b4a10 msgr2=0x7fa7cc160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.979+0000 7fa7ca000640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7cc0b4a10 0x7fa7cc160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.979+0000 7fa7ca000640 1 -- 172.21.15.7:0/3482263026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc162ef0 con 0x7fa7cc0b7440 2024-04-25T11:12:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.979+0000 7fa7ca000640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7cc0b7440 0x7fa7cc0cee60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009890 tx=0x7fa7b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.980+0000 7fa7c2a00640 1 -- 172.21.15.7:0/3482263026 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4041020 con 0x7fa7cc0b7440 2024-04-25T11:12:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.980+0000 7fa7c2a00640 1 -- 172.21.15.7:0/3482263026 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b403d690 con 0x7fa7cc0b7440 2024-04-25T11:12:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.980+0000 7fa7cbe00640 1 -- 172.21.15.7:0/3482263026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7cc1631d0 con 0x7fa7cc0b7440 2024-04-25T11:12:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.980+0000 7fa7c2a00640 1 -- 172.21.15.7:0/3482263026 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4037040 con 0x7fa7cc0b7440 2024-04-25T11:12:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.980+0000 7fa7cbe00640 1 -- 172.21.15.7:0/3482263026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7cc163790 con 0x7fa7cc0b7440 2024-04-25T11:12:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.981+0000 7fa7cbe00640 1 -- 172.21.15.7:0/3482263026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7cc050740 con 0x7fa7cc0b7440 2024-04-25T11:12:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.983+0000 7fa7c2a00640 1 -- 172.21.15.7:0/3482263026 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa7b4044020 con 0x7fa7cc0b7440 2024-04-25T11:12:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.984+0000 7fa7c2a00640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 0x7fa7a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.984+0000 7fa7c2a00640 1 -- 172.21.15.7:0/3482263026 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fa7b40b57f0 con 0x7fa7cc0b7440 2024-04-25T11:12:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.984+0000 7fa7caa00640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 0x7fa7a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.985+0000 7fa7caa00640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 0x7fa7a80769c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0d1cb0 tx=0x7fa7bc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:18.989+0000 7fa7c2a00640 1 -- 172.21.15.7:0/3482263026 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa7b404b030 con 0x7fa7cc0b7440 2024-04-25T11:12:19.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.239+0000 7fa7cbe00640 1 -- 172.21.15.7:0/3482263026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa7cc15d320 con 0x7fa7cc0b7440 2024-04-25T11:12:19.539 INFO:tasks.ceph.mds.d.smithi087.stderr:2024-04-25T11:12:19.538+0000 7fb8aa200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:12:19.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.638+0000 7fa7c2a00640 1 -- 172.21.15.7:0/3482263026 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fa7b4086e20 con 0x7fa7cc0b7440 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.642+0000 7fa7cbe00640 1 -- 172.21.15.7:0/3482263026 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 msgr2=0x7fa7a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.642+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 0x7fa7a80769c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0d1cb0 tx=0x7fa7bc00b040 comp rx=0 tx=0).stop 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.642+0000 7fa7cbe00640 1 -- 172.21.15.7:0/3482263026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7cc0b7440 msgr2=0x7fa7cc0cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.642+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7cc0b7440 0x7fa7cc0cee60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009890 tx=0x7fa7b40046c0 comp rx=0 tx=0).stop 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.642+0000 7fa7cbe00640 1 -- 172.21.15.7:0/3482263026 shutdown_connections 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.642+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 0x7fa7a80769c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.642+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc160090 0x7fa7cc162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.642+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7cc0b7440 0x7fa7cc0cee60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.642+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/3482263026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7cc0b4a10 0x7fa7cc160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.642+0000 7fa7cbe00640 1 -- 172.21.15.7:0/3482263026 >> 172.21.15.7:0/3482263026 conn(0x7fa7cc0d5de0 msgr2=0x7fa7cc0d49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.643+0000 7fa7cbe00640 1 -- 172.21.15.7:0/3482263026 shutdown_connections 2024-04-25T11:12:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.643+0000 7fa7cbe00640 1 -- 172.21.15.7:0/3482263026 wait complete. 2024-04-25T11:12:19.669 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:12:19.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:12:19.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.957+0000 7f0b53e00640 1 Processor -- start 2024-04-25T11:12:19.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.957+0000 7f0b53e00640 1 -- start start 2024-04-25T11:12:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.959+0000 7f0b53e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0b540d0480 0x7f0b540d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.959+0000 7f0b53e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b540ce570 con 0x7f0b541583c0 2024-04-25T11:12:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.959+0000 7f0b53e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b540ce850 con 0x7f0b5406eea0 2024-04-25T11:12:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.959+0000 7f0b53e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0b5415f190 con 0x7f0b540d0480 2024-04-25T11:12:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.959+0000 7f0b52a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0b541583c0 0x7f0b540cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53000/0 (socket says 172.21.15.7:53000) 2024-04-25T11:12:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.959+0000 7f0b52a00640 1 -- 172.21.15.7:0/352272895 learned_addr learned my addr 172.21.15.7:0/352272895 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2623938859 0 0) 0x7f0b540ce570 con 0x7f0b541583c0 2024-04-25T11:12:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b52000640 1 --2- 172.21.15.7:0/352272895 >> v2:172.21.15.87:3300/0 conn(0x7f0b540d0480 0x7f0b540d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b38003520 con 0x7f0b541583c0 2024-04-25T11:12:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2228845630 0 0) 0x7f0b38003520 con 0x7f0b541583c0 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 >> v2:172.21.15.87:3300/0 conn(0x7f0b540d0480 msgr2=0x7f0b540d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 --2- 172.21.15.7:0/352272895 >> v2:172.21.15.87:3300/0 conn(0x7f0b540d0480 0x7f0b540d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 >> v1:172.21.15.87:6789/0 conn(0x7f0b5406eea0 legacy=0x7f0b540b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b540ceb30 con 0x7f0b541583c0 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0b4c0019f0 con 0x7f0b541583c0 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 >> v1:172.21.15.7:6789/0 conn(0x7f0b541583c0 legacy=0x7f0b540cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b38003a00 0x7f0b38003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b38004800 0x7f0b38004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b38005330 0x7f0b3800b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5415f190 con 0x7f0b38004800 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3800c2c0 con 0x7f0b38005330 2024-04-25T11:12:19.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.960+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3800c4c0 con 0x7f0b38003a00 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.961+0000 7f0b52000640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b38004800 0x7f0b38004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.961+0000 7f0b52a00640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b38003a00 0x7f0b38003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.961+0000 7f0b53400640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b38005330 0x7f0b3800b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.961+0000 7f0b52000640 1 -- 172.21.15.7:0/352272895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b38003a00 msgr2=0x7f0b38003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.961+0000 7f0b52000640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b38003a00 0x7f0b38003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.961+0000 7f0b52000640 1 -- 172.21.15.7:0/352272895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b38005330 msgr2=0x7f0b3800b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.961+0000 7f0b52000640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b38005330 0x7f0b3800b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.961+0000 7f0b52000640 1 -- 172.21.15.7:0/352272895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b38003520 con 0x7f0b38004800 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.961+0000 7f0b53400640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b38005330 0x7f0b3800b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.961+0000 7f0b53400640 1 -- 172.21.15.7:0/352272895 reap_dead start 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.962+0000 7f0b52000640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b38004800 0x7f0b38004c10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0b54072ef0 tx=0x7f0b38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7da0227b44befb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.962+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c030070 con 0x7f0b38004800 2024-04-25T11:12:19.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.962+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3c02ad80 con 0x7f0b38004800 2024-04-25T11:12:19.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.962+0000 7f0b51600640 1 -- 172.21.15.7:0/352272895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00b610 con 0x7f0b38004800 2024-04-25T11:12:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.963+0000 7f0b53e00640 1 -- 172.21.15.7:0/352272895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b38004800 msgr2=0x7f0b38004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.963+0000 7f0b53e00640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b38004800 0x7f0b38004c10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0b54072ef0 tx=0x7f0b38005830 comp rx=0 tx=0).stop 2024-04-25T11:12:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.963+0000 7f0b53e00640 1 -- 172.21.15.7:0/352272895 shutdown_connections 2024-04-25T11:12:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.963+0000 7f0b53e00640 1 --2- 172.21.15.7:0/352272895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b38004800 0x7f0b38004c10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.963+0000 7f0b53e00640 1 -- 172.21.15.7:0/352272895 >> 172.21.15.7:0/352272895 conn(0x7f0b540740f0 msgr2=0x7f0b5406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.963+0000 7f0b53e00640 1 -- 172.21.15.7:0/352272895 shutdown_connections 2024-04-25T11:12:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.963+0000 7f0b53e00640 1 -- 172.21.15.7:0/352272895 wait complete. 2024-04-25T11:12:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.965+0000 7f0b53e00640 1 Processor -- start 2024-04-25T11:12:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.966+0000 7f0b53e00640 1 -- start start 2024-04-25T11:12:19.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.967+0000 7f0b53e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b38004800 0x7f0b541403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:19.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.967+0000 7f0b53e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b541e9150 0x7f0b541420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:19.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.967+0000 7f0b53e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b541e9a50 0x7f0b54142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:19.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.967+0000 7f0b53e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0b540d0890 con 0x7f0b541e9150 2024-04-25T11:12:19.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.967+0000 7f0b53e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0b540cf0f0 con 0x7f0b38004800 2024-04-25T11:12:19.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.967+0000 7f0b53e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0b540d0a10 con 0x7f0b541e9a50 2024-04-25T11:12:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.967+0000 7f0b52000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b541e9150 0x7f0b541420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.967+0000 7f0b52000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b541e9150 0x7f0b541420c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53994/0 (socket says 172.21.15.7:53994) 2024-04-25T11:12:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.967+0000 7f0b52000640 1 -- 172.21.15.7:0/1227172641 learned_addr learned my addr 172.21.15.7:0/1227172641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.968+0000 7f0b52a00640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b38004800 0x7f0b541403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.968+0000 7f0b53400640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b541e9a50 0x7f0b54142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.968+0000 7f0b52000640 1 -- 172.21.15.7:0/1227172641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b541e9a50 msgr2=0x7f0b54142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.968+0000 7f0b52000640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b541e9a50 0x7f0b54142600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.968+0000 7f0b52000640 1 -- 172.21.15.7:0/1227172641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b38004800 msgr2=0x7f0b541403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.968+0000 7f0b52000640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b38004800 0x7f0b541403e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.968+0000 7f0b52000640 1 -- 172.21.15.7:0/1227172641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b54141020 con 0x7f0b541e9150 2024-04-25T11:12:19.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.968+0000 7f0b52000640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b541e9150 0x7f0b541420c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c02a870 tx=0x7f0b3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:19.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.969+0000 7f0b4aa00640 1 -- 172.21.15.7:0/1227172641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c043020 con 0x7f0b541e9150 2024-04-25T11:12:19.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.969+0000 7f0b4aa00640 1 -- 172.21.15.7:0/1227172641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3c03f680 con 0x7f0b541e9150 2024-04-25T11:12:19.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.969+0000 7f0b4aa00640 1 -- 172.21.15.7:0/1227172641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00b610 con 0x7f0b541e9150 2024-04-25T11:12:19.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.969+0000 7f0b53e00640 1 -- 172.21.15.7:0/1227172641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b54141300 con 0x7f0b541e9150 2024-04-25T11:12:19.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.969+0000 7f0b53e00640 1 -- 172.21.15.7:0/1227172641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b541418c0 con 0x7f0b541e9150 2024-04-25T11:12:19.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.970+0000 7f0b53e00640 1 -- 172.21.15.7:0/1227172641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b54050740 con 0x7f0b541e9150 2024-04-25T11:12:19.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.971+0000 7f0b4aa00640 1 -- 172.21.15.7:0/1227172641 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0b3c046020 con 0x7f0b541e9150 2024-04-25T11:12:19.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.972+0000 7f0b4aa00640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0b300744b0 0x7f0b300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:19.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.972+0000 7f0b4aa00640 1 -- 172.21.15.7:0/1227172641 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f0b3c0b6100 con 0x7f0b541e9150 2024-04-25T11:12:19.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.973+0000 7f0b52a00640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0b300744b0 0x7f0b300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:19.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.973+0000 7f0b52a00640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0b300744b0 0x7f0b300769c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b540aa990 tx=0x7f0b4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:19.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:19.977+0000 7f0b4aa00640 1 -- 172.21.15.7:0/1227172641 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0b3c04d030 con 0x7f0b541e9150 2024-04-25T11:12:20.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.223+0000 7f0b53e00640 1 -- 172.21.15.7:0/1227172641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0b54070dd0 con 0x7f0b541e9150 2024-04-25T11:12:20.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.652+0000 7f0b4aa00640 1 -- 172.21.15.7:0/1227172641 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f0b3c080660 con 0x7f0b541e9150 2024-04-25T11:12:20.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 -- 172.21.15.7:0/1227172641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0b300744b0 msgr2=0x7f0b300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:20.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0b300744b0 0x7f0b300769c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b540aa990 tx=0x7f0b4c009040 comp rx=0 tx=0).stop 2024-04-25T11:12:20.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 -- 172.21.15.7:0/1227172641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b541e9150 msgr2=0x7f0b541420c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:20.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b541e9150 0x7f0b541420c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c02a870 tx=0x7f0b3c004730 comp rx=0 tx=0).stop 2024-04-25T11:12:20.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 -- 172.21.15.7:0/1227172641 shutdown_connections 2024-04-25T11:12:20.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0b300744b0 0x7f0b300769c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:20.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b541e9a50 0x7f0b54142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:20.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b541e9150 0x7f0b541420c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:20.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 --2- 172.21.15.7:0/1227172641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b38004800 0x7f0b541403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:20.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 -- 172.21.15.7:0/1227172641 >> 172.21.15.7:0/1227172641 conn(0x7f0b540740f0 msgr2=0x7f0b5406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:20.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 -- 172.21.15.7:0/1227172641 shutdown_connections 2024-04-25T11:12:20.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.655+0000 7f0b53e00640 1 -- 172.21.15.7:0/1227172641 wait complete. 2024-04-25T11:12:20.684 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:12:20.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:12:20.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.967+0000 7f1e41c00640 1 Processor -- start 2024-04-25T11:12:20.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.967+0000 7f1e41c00640 1 -- start start 2024-04-25T11:12:20.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.969+0000 7f1e41c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e3c15ef90 0x7f1e3c09f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:20.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.970+0000 7f1e41c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1e3c0af540 0x7f1e3c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:20.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.970+0000 7f1e41c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e3c15f4e0 con 0x7f1e3c15bc40 2024-04-25T11:12:20.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.970+0000 7f1e41c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1e3c071190 con 0x7f1e3c0af540 2024-04-25T11:12:20.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.970+0000 7f1e41c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1e3c0b0560 con 0x7f1e3c15ef90 2024-04-25T11:12:20.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.970+0000 7f1e3aa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e3c15ef90 0x7f1e3c09f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:20.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.970+0000 7f1e3b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1e3c0af540 0x7f1e3c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:20.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.971+0000 7f1e3b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1e3c0af540 0x7f1e3c0cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:59206/0 (socket says 172.21.15.7:59206) 2024-04-25T11:12:20.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.971+0000 7f1e3b400640 1 -- 172.21.15.7:0/3612822437 learned_addr learned my addr 172.21.15.7:0/3612822437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:20.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.971+0000 7f1e3a000640 1 --1- 172.21.15.7:0/3612822437 >> v1:172.21.15.87:6789/0 conn(0x7f1e3c15bc40 0x7f1e3c0aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39898/0 (socket says 172.21.15.7:39898) 2024-04-25T11:12:20.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.971+0000 7f1e3b400640 1 -- 172.21.15.7:0/3612822437 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e3c15ef90 msgr2=0x7f1e3c09f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:20.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.971+0000 7f1e3b400640 1 --2- 172.21.15.7:0/3612822437 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e3c15ef90 0x7f1e3c09f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:20.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.971+0000 7f1e3b400640 1 -- 172.21.15.7:0/3612822437 >> v1:172.21.15.87:6789/0 conn(0x7f1e3c15bc40 legacy=0x7f1e3c0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:20.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.971+0000 7f1e3b400640 1 -- 172.21.15.7:0/3612822437 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3c0b0840 con 0x7f1e3c0af540 2024-04-25T11:12:20.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.973+0000 7f1e3b400640 1 --2- 172.21.15.7:0/3612822437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e3c0af540 0x7f1e3c0cf980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009870 tx=0x7f1e3402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19fa4fd0993361b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:20.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.973+0000 7f1e39600640 1 -- 172.21.15.7:0/3612822437 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34037070 con 0x7f1e3c0af540 2024-04-25T11:12:20.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.973+0000 7f1e39600640 1 -- 172.21.15.7:0/3612822437 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e34032440 con 0x7f1e3c0af540 2024-04-25T11:12:20.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.973+0000 7f1e39600640 1 -- 172.21.15.7:0/3612822437 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34032810 con 0x7f1e3c0af540 2024-04-25T11:12:20.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.974+0000 7f1e41c00640 1 -- 172.21.15.7:0/3612822437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e3c0af540 msgr2=0x7f1e3c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:20.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.974+0000 7f1e41c00640 1 --2- 172.21.15.7:0/3612822437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e3c0af540 0x7f1e3c0cf980 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1e34009870 tx=0x7f1e3402a1c0 comp rx=0 tx=0).stop 2024-04-25T11:12:20.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.974+0000 7f1e41c00640 1 -- 172.21.15.7:0/3612822437 shutdown_connections 2024-04-25T11:12:20.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.974+0000 7f1e41c00640 1 --2- 172.21.15.7:0/3612822437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e3c0af540 0x7f1e3c0cf980 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:20.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.974+0000 7f1e41c00640 1 --2- 172.21.15.7:0/3612822437 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e3c15ef90 0x7f1e3c09f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:20.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.974+0000 7f1e41c00640 1 -- 172.21.15.7:0/3612822437 >> 172.21.15.7:0/3612822437 conn(0x7f1e3c074200 msgr2=0x7f1e3c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:20.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.974+0000 7f1e41c00640 1 -- 172.21.15.7:0/3612822437 shutdown_connections 2024-04-25T11:12:20.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.974+0000 7f1e41c00640 1 -- 172.21.15.7:0/3612822437 wait complete. 2024-04-25T11:12:20.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.977+0000 7f1e41c00640 1 Processor -- start 2024-04-25T11:12:20.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.977+0000 7f1e41c00640 1 -- start start 2024-04-25T11:12:20.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.978+0000 7f1e41c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e3c15bc40 0x7f1e3c154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.978+0000 7f1e41c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e3c15ef90 0x7f1e3c14a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.978+0000 7f1e41c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e3c159620 0x7f1e3c159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.978+0000 7f1e41c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c0b1320 con 0x7f1e3c159620 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.978+0000 7f1e41c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1e3c0b0e00 con 0x7f1e3c15bc40 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.978+0000 7f1e41c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1e3c0afb80 con 0x7f1e3c15ef90 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.979+0000 7f1e3b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e3c159620 0x7f1e3c159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.979+0000 7f1e3b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e3c159620 0x7f1e3c159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54008/0 (socket says 172.21.15.7:54008) 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.979+0000 7f1e3b400640 1 -- 172.21.15.7:0/3216903794 learned_addr learned my addr 172.21.15.7:0/3216903794 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.979+0000 7f1e3a000640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e3c15ef90 0x7f1e3c14a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.979+0000 7f1e3aa00640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e3c15bc40 0x7f1e3c154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.979+0000 7f1e3b400640 1 -- 172.21.15.7:0/3216903794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e3c15ef90 msgr2=0x7f1e3c14a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.979+0000 7f1e3b400640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e3c15ef90 0x7f1e3c14a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.979+0000 7f1e3b400640 1 -- 172.21.15.7:0/3216903794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e3c15bc40 msgr2=0x7f1e3c154c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.979+0000 7f1e3b400640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e3c15bc40 0x7f1e3c154c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:20.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.979+0000 7f1e3b400640 1 -- 172.21.15.7:0/3216903794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3c159f70 con 0x7f1e3c159620 2024-04-25T11:12:20.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.979+0000 7f1e3aa00640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e3c15bc40 0x7f1e3c154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:12:20.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.980+0000 7f1e3b400640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e3c159620 0x7f1e3c159a30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1e3402a200 tx=0x7f1e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:20.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.980+0000 7f1e32a00640 1 -- 172.21.15.7:0/3216903794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34041020 con 0x7f1e3c159620 2024-04-25T11:12:20.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.980+0000 7f1e32a00640 1 -- 172.21.15.7:0/3216903794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e3403d640 con 0x7f1e3c159620 2024-04-25T11:12:20.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.980+0000 7f1e32a00640 1 -- 172.21.15.7:0/3216903794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e34037040 con 0x7f1e3c159620 2024-04-25T11:12:20.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.980+0000 7f1e41c00640 1 -- 172.21.15.7:0/3216903794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e3c1556b0 con 0x7f1e3c159620 2024-04-25T11:12:20.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.980+0000 7f1e41c00640 1 -- 172.21.15.7:0/3216903794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e3c155c70 con 0x7f1e3c159620 2024-04-25T11:12:20.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.982+0000 7f1e41c00640 1 -- 172.21.15.7:0/3216903794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e3c06a100 con 0x7f1e3c159620 2024-04-25T11:12:20.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.983+0000 7f1e32a00640 1 -- 172.21.15.7:0/3216903794 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1e34044020 con 0x7f1e3c159620 2024-04-25T11:12:20.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.984+0000 7f1e32a00640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1e1c0744b0 0x7f1e1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:20.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.984+0000 7f1e32a00640 1 -- 172.21.15.7:0/3216903794 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f1e340b5c00 con 0x7f1e3c159620 2024-04-25T11:12:20.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.984+0000 7f1e3aa00640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1e1c0744b0 0x7f1e1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:20.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.985+0000 7f1e3aa00640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1e1c0744b0 0x7f1e1c0769c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009af0 tx=0x7f1e28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:20.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:20.989+0000 7f1e32a00640 1 -- 172.21.15.7:0/3216903794 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e3404b030 con 0x7f1e3c159620 2024-04-25T11:12:21.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.232+0000 7f1e41c00640 1 -- 172.21.15.7:0/3216903794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e3c161530 con 0x7f1e3c159620 2024-04-25T11:12:21.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.234+0000 7f1e32a00640 1 -- 172.21.15.7:0/3216903794 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5172 (secure 0 0 0) 0x7f1e3c159f70 con 0x7f1e3c159620 2024-04-25T11:12:21.234 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:21.234 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2575517671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2575517671},{"type":"v1","addr":"172.21.15.87:6835","nonce":2575517671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/3746682343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3746682343},{"type":"v1","addr":"172.21.15.7:6836","nonce":3746682343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2132415318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2132415318},{"type":"v1","addr":"172.21.15.7:6837","nonce":2132415318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:12:17.156571+0000","modified":"2024-04-25T11:12:20.224854+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6833/389115631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":389115631},{"type":"v1","addr":"172.21.15.87:6833","nonce":389115631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5283,"qdb_cluster":[5283]},"id":2}]} 2024-04-25T11:12:21.235 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2024-04-25T11:12:21.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.236+0000 7f1e41c00640 1 -- 172.21.15.7:0/3216903794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1e1c0744b0 msgr2=0x7f1e1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:21.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.236+0000 7f1e41c00640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1e1c0744b0 0x7f1e1c0769c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009af0 tx=0x7f1e28008040 comp rx=0 tx=0).stop 2024-04-25T11:12:21.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.236+0000 7f1e41c00640 1 -- 172.21.15.7:0/3216903794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e3c159620 msgr2=0x7f1e3c159a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:21.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.236+0000 7f1e41c00640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e3c159620 0x7f1e3c159a30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1e3402a200 tx=0x7f1e340046c0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.237+0000 7f1e41c00640 1 -- 172.21.15.7:0/3216903794 shutdown_connections 2024-04-25T11:12:21.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.237+0000 7f1e41c00640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1e1c0744b0 0x7f1e1c0769c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.237+0000 7f1e41c00640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e3c159620 0x7f1e3c159a30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.237+0000 7f1e41c00640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e3c15ef90 0x7f1e3c14a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.237+0000 7f1e41c00640 1 --2- 172.21.15.7:0/3216903794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e3c15bc40 0x7f1e3c154c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.237+0000 7f1e41c00640 1 -- 172.21.15.7:0/3216903794 >> 172.21.15.7:0/3216903794 conn(0x7f1e3c074200 msgr2=0x7f1e3c0d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:21.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.237+0000 7f1e41c00640 1 -- 172.21.15.7:0/3216903794 shutdown_connections 2024-04-25T11:12:21.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.237+0000 7f1e41c00640 1 -- 172.21.15.7:0/3216903794 wait complete. 2024-04-25T11:12:21.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:12:21.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.561+0000 7f3681a00640 1 Processor -- start 2024-04-25T11:12:21.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.561+0000 7f3681a00640 1 -- start start 2024-04-25T11:12:21.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.563+0000 7f3681a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f367c0b04d0 0x7f367c0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:21.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.563+0000 7f3681a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f367c15e190 con 0x7f367c0b4a80 2024-04-25T11:12:21.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.563+0000 7f3681a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f367c15d610 con 0x7f367c070a70 2024-04-25T11:12:21.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.563+0000 7f3681a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f367c15d8f0 con 0x7f367c0b04d0 2024-04-25T11:12:21.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.563+0000 7f3681000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f367c0b4a80 0x7f367c0b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53016/0 (socket says 172.21.15.7:53016) 2024-04-25T11:12:21.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.563+0000 7f3681000640 1 -- 172.21.15.7:0/3794394150 learned_addr learned my addr 172.21.15.7:0/3794394150 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.564+0000 7f367b400640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f367c0b04d0 0x7f367c0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.564+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1104028796 0 0) 0x7f367c15e190 con 0x7f367c0b4a80 2024-04-25T11:12:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.564+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3660003520 con 0x7f367c0b4a80 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.564+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2049715761 0 0) 0x7f3660003520 con 0x7f367c0b4a80 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.564+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f367c0b04d0 msgr2=0x7f367c0b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.564+0000 7f367aa00640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f367c0b04d0 0x7f367c0b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.564+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 >> v1:172.21.15.87:6789/0 conn(0x7f367c070a70 legacy=0x7f367c0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.564+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c15dbd0 con 0x7f367c0b4a80 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.564+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f3674001690 con 0x7f367c0b4a80 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.564+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 >> v1:172.21.15.7:6789/0 conn(0x7f367c0b4a80 legacy=0x7f367c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f367aa00640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36600039e0 0x7f3660003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f367aa00640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f367c0b04d0 0x7f3660004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f367aa00640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3660004d40 0x7f3660005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f367c15d8f0 con 0x7f3660004d40 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f366000ba70 con 0x7f36600039e0 2024-04-25T11:12:21.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f366000bc70 con 0x7f367c0b04d0 2024-04-25T11:12:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f3681000640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3660004d40 0x7f3660005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f367be00640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36600039e0 0x7f3660003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f367b400640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f367c0b04d0 0x7f3660004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f3681000640 1 -- 172.21.15.7:0/3794394150 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f367c0b04d0 msgr2=0x7f3660004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f3681000640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f367c0b04d0 0x7f3660004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f3681000640 1 -- 172.21.15.7:0/3794394150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36600039e0 msgr2=0x7f3660003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f3681000640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36600039e0 0x7f3660003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.565+0000 7f3681000640 1 -- 172.21.15.7:0/3794394150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3660003520 con 0x7f3660004d40 2024-04-25T11:12:21.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.566+0000 7f3681000640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3660004d40 0x7f3660005150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f367c072ec0 tx=0x7f3660005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2be622bf74798d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:21.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.567+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674007ce0 con 0x7f3660004d40 2024-04-25T11:12:21.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.567+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3674005ad0 con 0x7f3660004d40 2024-04-25T11:12:21.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.567+0000 7f367aa00640 1 -- 172.21.15.7:0/3794394150 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674007070 con 0x7f3660004d40 2024-04-25T11:12:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.567+0000 7f3681a00640 1 -- 172.21.15.7:0/3794394150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3660004d40 msgr2=0x7f3660005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.567+0000 7f3681a00640 1 --2- 172.21.15.7:0/3794394150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3660004d40 0x7f3660005150 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f367c072ec0 tx=0x7f3660005830 comp rx=0 tx=0).stop 2024-04-25T11:12:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.568+0000 7f3681000640 1 -- 172.21.15.7:0/3794394150 reap_dead start 2024-04-25T11:12:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.568+0000 7f3681a00640 1 -- 172.21.15.7:0/3794394150 shutdown_connections 2024-04-25T11:12:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.568+0000 7f3681a00640 1 -- 172.21.15.7:0/3794394150 >> 172.21.15.7:0/3794394150 conn(0x7f367c0b3820 msgr2=0x7f367c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.568+0000 7f3681a00640 1 -- 172.21.15.7:0/3794394150 shutdown_connections 2024-04-25T11:12:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.568+0000 7f3681a00640 1 -- 172.21.15.7:0/3794394150 wait complete. 2024-04-25T11:12:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.570+0000 7f3681a00640 1 Processor -- start 2024-04-25T11:12:21.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.570+0000 7f3681a00640 1 -- start start 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.572+0000 7f3681a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3660004d40 0x7f367c153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.572+0000 7f3681a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f367c0cbe00 0x7f367c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.572+0000 7f3681a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0b1ed0 0x7f367c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.572+0000 7f3681a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f367c15d170 con 0x7f367c0b1ed0 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.572+0000 7f3681a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f367c0aef90 con 0x7f367c0cbe00 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.572+0000 7f3681a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f367c15d2f0 con 0x7f3660004d40 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.572+0000 7f3681000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0b1ed0 0x7f367c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.572+0000 7f3681000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0b1ed0 0x7f367c0b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54026/0 (socket says 172.21.15.7:54026) 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.572+0000 7f3681000640 1 -- 172.21.15.7:0/2578820051 learned_addr learned my addr 172.21.15.7:0/2578820051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.573+0000 7f367be00640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3660004d40 0x7f367c153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.573+0000 7f367b400640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f367c0cbe00 0x7f367c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:21.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.573+0000 7f3681000640 1 -- 172.21.15.7:0/2578820051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3660004d40 msgr2=0x7f367c153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.573+0000 7f3681000640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3660004d40 0x7f367c153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.573+0000 7f3681000640 1 -- 172.21.15.7:0/2578820051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f367c0cbe00 msgr2=0x7f367c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.573+0000 7f3681000640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f367c0cbe00 0x7f367c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.573+0000 7f3681000640 1 -- 172.21.15.7:0/2578820051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c0b2820 con 0x7f367c0b1ed0 2024-04-25T11:12:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.573+0000 7f3681000640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0b1ed0 0x7f367c0b22e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f367c070ea0 tx=0x7f367c0b4730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.573+0000 7f3673e00640 1 -- 172.21.15.7:0/2578820051 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674037040 con 0x7f367c0b1ed0 2024-04-25T11:12:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.574+0000 7f3673e00640 1 -- 172.21.15.7:0/2578820051 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3674041070 con 0x7f367c0b1ed0 2024-04-25T11:12:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.574+0000 7f3673e00640 1 -- 172.21.15.7:0/2578820051 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674007070 con 0x7f367c0b1ed0 2024-04-25T11:12:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.574+0000 7f3681a00640 1 -- 172.21.15.7:0/2578820051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367c1535b0 con 0x7f367c0b1ed0 2024-04-25T11:12:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.574+0000 7f3681a00640 1 -- 172.21.15.7:0/2578820051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367c15e7b0 con 0x7f367c0b1ed0 2024-04-25T11:12:21.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.575+0000 7f3681a00640 1 -- 172.21.15.7:0/2578820051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f367c15e240 con 0x7f367c0b1ed0 2024-04-25T11:12:21.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.576+0000 7f3673e00640 1 -- 172.21.15.7:0/2578820051 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3674007210 con 0x7f367c0b1ed0 2024-04-25T11:12:21.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.577+0000 7f3673e00640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36580744e0 0x7f36580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:21.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.577+0000 7f3673e00640 1 -- 172.21.15.7:0/2578820051 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f36740b5380 con 0x7f367c0b1ed0 2024-04-25T11:12:21.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.578+0000 7f367be00640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36580744e0 0x7f36580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:21.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.578+0000 7f367be00640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36580744e0 0x7f36580769f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f366c005540 tx=0x7f366c007c50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:21.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.582+0000 7f3673e00640 1 -- 172.21.15.7:0/2578820051 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f367c0b2820 con 0x7f367c0b1ed0 2024-04-25T11:12:21.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.792+0000 7f3681a00640 1 -- 172.21.15.7:0/2578820051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f367c050740 con 0x7f367c0b1ed0 2024-04-25T11:12:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.795+0000 7f3673e00640 1 -- 172.21.15.7:0/2578820051 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16846 (secure 0 0 0) 0x7f367407f8e0 con 0x7f367c0b1ed0 2024-04-25T11:12:21.796 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:21.796 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:12:18.606045+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:12:14.961811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T11:12:16.128549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:12:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.798+0000 7f3681a00640 1 -- 172.21.15.7:0/2578820051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36580744e0 msgr2=0x7f36580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.798+0000 7f3681a00640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36580744e0 0x7f36580769f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f366c005540 tx=0x7f366c007c50 comp rx=0 tx=0).stop 2024-04-25T11:12:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.798+0000 7f3681a00640 1 -- 172.21.15.7:0/2578820051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0b1ed0 msgr2=0x7f367c0b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.798+0000 7f3681a00640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0b1ed0 0x7f367c0b22e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f367c070ea0 tx=0x7f367c0b4730 comp rx=0 tx=0).stop 2024-04-25T11:12:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.798+0000 7f3681a00640 1 -- 172.21.15.7:0/2578820051 shutdown_connections 2024-04-25T11:12:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.798+0000 7f3681a00640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36580744e0 0x7f36580769f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.799+0000 7f3681a00640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f367c0b1ed0 0x7f367c0b22e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.799+0000 7f3681a00640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f367c0cbe00 0x7f367c0cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.799+0000 7f3681a00640 1 --2- 172.21.15.7:0/2578820051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3660004d40 0x7f367c153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.799+0000 7f3681a00640 1 -- 172.21.15.7:0/2578820051 >> 172.21.15.7:0/2578820051 conn(0x7f367c0b3820 msgr2=0x7f367c1608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.799+0000 7f3681a00640 1 -- 172.21.15.7:0/2578820051 shutdown_connections 2024-04-25T11:12:21.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:21.799+0000 7f3681a00640 1 -- 172.21.15.7:0/2578820051 wait complete. 2024-04-25T11:12:21.827 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:12:21.827 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:12:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.109+0000 7f9b0a000640 1 Processor -- start 2024-04-25T11:12:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.110+0000 7f9b0a000640 1 -- start start 2024-04-25T11:12:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.112+0000 7f9b0a000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b040b5a90 0x7f9b04167930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.112+0000 7f9b0a000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0415e190 con 0x7f9b04070a70 2024-04-25T11:12:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.112+0000 7f9b0a000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0415d610 con 0x7f9b040b04d0 2024-04-25T11:12:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.112+0000 7f9b0a000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b0415d8f0 con 0x7f9b040b5a90 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.112+0000 7f9b08c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9b04070a70 0x7f9b040ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53030/0 (socket says 172.21.15.7:53030) 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.112+0000 7f9b08c00640 1 -- 172.21.15.7:0/3604319431 learned_addr learned my addr 172.21.15.7:0/3604319431 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3410678753 0 0) 0x7f9b0415e190 con 0x7f9b04070a70 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ae8003520 con 0x7f9b04070a70 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b09600640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b040b5a90 0x7f9b04167930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2555944464 0 0) 0x7f9ae8003520 con 0x7f9b04070a70 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b040b5a90 msgr2=0x7f9b04167930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b040b5a90 0x7f9b04167930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 >> v1:172.21.15.87:6789/0 conn(0x7f9b040b04d0 legacy=0x7f9b040b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0415dbd0 con 0x7f9b04070a70 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f9af4001670 con 0x7f9b04070a70 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 >> v1:172.21.15.7:6789/0 conn(0x7f9b04070a70 legacy=0x7f9b040ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ae80039e0 0x7f9ae8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae8004800 0x7f9ae8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b040b5a90 0x7f9ae80052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0415d8f0 con 0x7f9ae8004800 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae800dde0 con 0x7f9ae80039e0 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.113+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae800dfe0 con 0x7f9b040b5a90 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.114+0000 7f9afbe00640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae8004800 0x7f9ae8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.114+0000 7f9b08c00640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ae80039e0 0x7f9ae8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.114+0000 7f9b09600640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b040b5a90 0x7f9ae80052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.114+0000 7f9afbe00640 1 -- 172.21.15.7:0/3604319431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b040b5a90 msgr2=0x7f9ae80052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.114+0000 7f9afbe00640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b040b5a90 0x7f9ae80052d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.114+0000 7f9afbe00640 1 -- 172.21.15.7:0/3604319431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ae80039e0 msgr2=0x7f9ae8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.114+0000 7f9afbe00640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ae80039e0 0x7f9ae8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.114+0000 7f9afbe00640 1 -- 172.21.15.7:0/3604319431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae8003520 con 0x7f9ae8004800 2024-04-25T11:12:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.115+0000 7f9afbe00640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae8004800 0x7f9ae8004c10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9aec001750 tx=0x7f9ae8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babe66c0e9a14388 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.115+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec001370 con 0x7f9ae8004800 2024-04-25T11:12:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.115+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aec0052d0 con 0x7f9ae8004800 2024-04-25T11:12:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.115+0000 7f9b03e00640 1 -- 172.21.15.7:0/3604319431 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec02abe0 con 0x7f9ae8004800 2024-04-25T11:12:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.116+0000 7f9b0a000640 1 -- 172.21.15.7:0/3604319431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae8004800 msgr2=0x7f9ae8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.116+0000 7f9b0a000640 1 --2- 172.21.15.7:0/3604319431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ae8004800 0x7f9ae8004c10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9aec001750 tx=0x7f9ae8005830 comp rx=0 tx=0).stop 2024-04-25T11:12:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.116+0000 7f9b09600640 1 -- 172.21.15.7:0/3604319431 reap_dead start 2024-04-25T11:12:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.116+0000 7f9b0a000640 1 -- 172.21.15.7:0/3604319431 shutdown_connections 2024-04-25T11:12:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.116+0000 7f9b0a000640 1 -- 172.21.15.7:0/3604319431 >> 172.21.15.7:0/3604319431 conn(0x7f9b040b3820 msgr2=0x7f9b040b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.116+0000 7f9b0a000640 1 -- 172.21.15.7:0/3604319431 shutdown_connections 2024-04-25T11:12:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.116+0000 7f9b0a000640 1 -- 172.21.15.7:0/3604319431 wait complete. 2024-04-25T11:12:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.119+0000 7f9b0a000640 1 Processor -- start 2024-04-25T11:12:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.119+0000 7f9b0a000640 1 -- start start 2024-04-25T11:12:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.120+0000 7f9b0a000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ae8004800 0x7f9b04155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.120+0000 7f9b0a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b040b1de0 0x7f9b040b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.120+0000 7f9b0a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b04159010 0x7f9b04159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.120+0000 7f9b0a000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0415d170 con 0x7f9b040b1de0 2024-04-25T11:12:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.120+0000 7f9b0a000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b040aef90 con 0x7f9b04159010 2024-04-25T11:12:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.120+0000 7f9b0a000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b0415d2f0 con 0x7f9ae8004800 2024-04-25T11:12:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.121+0000 7f9afbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b040b1de0 0x7f9b040b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.121+0000 7f9afbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b040b1de0 0x7f9b040b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54046/0 (socket says 172.21.15.7:54046) 2024-04-25T11:12:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.121+0000 7f9afbe00640 1 -- 172.21.15.7:0/3624432423 learned_addr learned my addr 172.21.15.7:0/3624432423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:22.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.121+0000 7f9b08c00640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ae8004800 0x7f9b04155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.121+0000 7f9b09600640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b04159010 0x7f9b04159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.121+0000 7f9afbe00640 1 -- 172.21.15.7:0/3624432423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ae8004800 msgr2=0x7f9b04155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.121+0000 7f9afbe00640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ae8004800 0x7f9b04155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.121+0000 7f9afbe00640 1 -- 172.21.15.7:0/3624432423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b04159010 msgr2=0x7f9b04159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.121+0000 7f9afbe00640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b04159010 0x7f9b04159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.121+0000 7f9afbe00640 1 -- 172.21.15.7:0/3624432423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b04159960 con 0x7f9b040b1de0 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.122+0000 7f9afbe00640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b040b1de0 0x7f9b040b21f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9aec001870 tx=0x7f9aec001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.122+0000 7f9b01600640 1 -- 172.21.15.7:0/3624432423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec037040 con 0x7f9b040b1de0 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.122+0000 7f9b01600640 1 -- 172.21.15.7:0/3624432423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aec005480 con 0x7f9b040b1de0 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.122+0000 7f9b0a000640 1 -- 172.21.15.7:0/3624432423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b040b2b10 con 0x7f9b040b1de0 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.122+0000 7f9b01600640 1 -- 172.21.15.7:0/3624432423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec007a30 con 0x7f9b040b1de0 2024-04-25T11:12:22.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.122+0000 7f9b0a000640 1 -- 172.21.15.7:0/3624432423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b040cad00 con 0x7f9b040b1de0 2024-04-25T11:12:22.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.123+0000 7f9b0a000640 1 -- 172.21.15.7:0/3624432423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b04050740 con 0x7f9b040b1de0 2024-04-25T11:12:22.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.124+0000 7f9b01600640 1 -- 172.21.15.7:0/3624432423 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9aec044020 con 0x7f9b040b1de0 2024-04-25T11:12:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.125+0000 7f9b01600640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ae00744b0 0x7f9ae00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.125+0000 7f9b01600640 1 -- 172.21.15.7:0/3624432423 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f9aec0b50c0 con 0x7f9b040b1de0 2024-04-25T11:12:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.126+0000 7f9b08c00640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ae00744b0 0x7f9ae00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.127+0000 7f9b08c00640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ae00744b0 0x7f9ae00769c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9af4001050 tx=0x7f9af4001150 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:22.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.130+0000 7f9b01600640 1 -- 172.21.15.7:0/3624432423 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b04159960 con 0x7f9b040b1de0 2024-04-25T11:12:22.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.331+0000 7f9b0a000640 1 -- 172.21.15.7:0/3624432423 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b040b28a0 con 0x7f9ae00744b0 2024-04-25T11:12:22.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.343+0000 7f9b01600640 1 -- 172.21.15.7:0/3624432423 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7f9b040b28a0 con 0x7f9ae00744b0 2024-04-25T11:12:22.350 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:22.352 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:12:22.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.350+0000 7f9b0a000640 1 -- 172.21.15.7:0/3624432423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ae00744b0 msgr2=0x7f9ae00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.350+0000 7f9b0a000640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ae00744b0 0x7f9ae00769c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9af4001050 tx=0x7f9af4001150 comp rx=0 tx=0).stop 2024-04-25T11:12:22.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.350+0000 7f9b0a000640 1 -- 172.21.15.7:0/3624432423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b040b1de0 msgr2=0x7f9b040b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.350+0000 7f9b0a000640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b040b1de0 0x7f9b040b21f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9aec001870 tx=0x7f9aec001120 comp rx=0 tx=0).stop 2024-04-25T11:12:22.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.350+0000 7f9b0a000640 1 -- 172.21.15.7:0/3624432423 shutdown_connections 2024-04-25T11:12:22.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.350+0000 7f9b0a000640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ae00744b0 0x7f9ae00769c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.350+0000 7f9b0a000640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b04159010 0x7f9b04159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.350+0000 7f9b0a000640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b040b1de0 0x7f9b040b21f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.350+0000 7f9b0a000640 1 --2- 172.21.15.7:0/3624432423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ae8004800 0x7f9b04155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.351+0000 7f9b0a000640 1 -- 172.21.15.7:0/3624432423 >> 172.21.15.7:0/3624432423 conn(0x7f9b040b3820 msgr2=0x7f9b040b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.351+0000 7f9b0a000640 1 -- 172.21.15.7:0/3624432423 shutdown_connections 2024-04-25T11:12:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.351+0000 7f9b0a000640 1 -- 172.21.15.7:0/3624432423 wait complete. 2024-04-25T11:12:22.379 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-25T11:12:21.827129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233392,"kb_used_data":4720,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741328,"statfs":{"total":773094113280,"available":772855119872,"internally_reserved":0,"allocated":4833280,"data_stored":2726364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002024"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.390284+0000","last_change":"2024-04-25T11:12:17.641255+0000","last_active":"2024-04-25T11:12:19.390284+0000","last_peered":"2024-04-25T11:12:19.390284+0000","last_clean":"2024-04-25T11:12:19.390284+0000","last_became_active":"2024-04-25T11:12:17.640883+0000","last_became_peered":"2024-04-25T11:12:17.640883+0000","last_unstale":"2024-04-25T11:12:19.390284+0000","last_undegraded":"2024-04-25T11:12:19.390284+0000","last_fullsized":"2024-04-25T11:12:19.390284+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:50.949873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389432+0000","last_change":"2024-04-25T11:12:17.634849+0000","last_active":"2024-04-25T11:12:19.389432+0000","last_peered":"2024-04-25T11:12:19.389432+0000","last_clean":"2024-04-25T11:12:19.389432+0000","last_became_active":"2024-04-25T11:12:17.634414+0000","last_became_peered":"2024-04-25T11:12:17.634414+0000","last_unstale":"2024-04-25T11:12:19.389432+0000","last_undegraded":"2024-04-25T11:12:19.389432+0000","last_fullsized":"2024-04-25T11:12:19.389432+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:35.196880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029403+0000","last_change":"2024-04-25T11:12:17.647463+0000","last_active":"2024-04-25T11:12:19.029403+0000","last_peered":"2024-04-25T11:12:19.029403+0000","last_clean":"2024-04-25T11:12:19.029403+0000","last_became_active":"2024-04-25T11:12:17.647178+0000","last_became_peered":"2024-04-25T11:12:17.647178+0000","last_unstale":"2024-04-25T11:12:19.029403+0000","last_undegraded":"2024-04-25T11:12:19.029403+0000","last_fullsized":"2024-04-25T11:12:19.029403+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.157140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.633010+0000","last_change":"2024-04-25T11:12:17.624701+0000","last_active":"2024-04-25T11:12:17.633010+0000","last_peered":"2024-04-25T11:12:17.633010+0000","last_clean":"2024-04-25T11:12:17.633010+0000","last_became_active":"2024-04-25T11:12:17.623517+0000","last_became_peered":"2024-04-25T11:12:17.623517+0000","last_unstale":"2024-04-25T11:12:17.633010+0000","last_undegraded":"2024-04-25T11:12:17.633010+0000","last_fullsized":"2024-04-25T11:12:17.633010+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:02.203277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.630060+0000","last_change":"2024-04-25T11:12:17.619749+0000","last_active":"2024-04-25T11:12:17.630060+0000","last_peered":"2024-04-25T11:12:17.630060+0000","last_clean":"2024-04-25T11:12:17.630060+0000","last_became_active":"2024-04-25T11:12:17.618350+0000","last_became_peered":"2024-04-25T11:12:17.618350+0000","last_unstale":"2024-04-25T11:12:17.630060+0000","last_undegraded":"2024-04-25T11:12:17.630060+0000","last_fullsized":"2024-04-25T11:12:17.630060+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:47.435885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615063+0000","last_change":"2024-04-25T11:12:17.629076+0000","last_active":"2024-04-25T11:12:18.615063+0000","last_peered":"2024-04-25T11:12:18.615063+0000","last_clean":"2024-04-25T11:12:18.615063+0000","last_became_active":"2024-04-25T11:12:17.628760+0000","last_became_peered":"2024-04-25T11:12:17.628760+0000","last_unstale":"2024-04-25T11:12:18.615063+0000","last_undegraded":"2024-04-25T11:12:18.615063+0000","last_fullsized":"2024-04-25T11:12:18.615063+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:42.282697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216232+0000","last_change":"2024-04-25T11:12:17.639438+0000","last_active":"2024-04-25T11:12:19.216232+0000","last_peered":"2024-04-25T11:12:19.216232+0000","last_clean":"2024-04-25T11:12:19.216232+0000","last_became_active":"2024-04-25T11:12:17.634264+0000","last_became_peered":"2024-04-25T11:12:17.634264+0000","last_unstale":"2024-04-25T11:12:19.216232+0000","last_undegraded":"2024-04-25T11:12:19.216232+0000","last_fullsized":"2024-04-25T11:12:19.216232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:31.557454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387464+0000","last_change":"2024-04-25T11:12:17.644433+0000","last_active":"2024-04-25T11:12:19.387464+0000","last_peered":"2024-04-25T11:12:19.387464+0000","last_clean":"2024-04-25T11:12:19.387464+0000","last_became_active":"2024-04-25T11:12:17.644037+0000","last_became_peered":"2024-04-25T11:12:17.644037+0000","last_unstale":"2024-04-25T11:12:19.387464+0000","last_undegraded":"2024-04-25T11:12:19.387464+0000","last_fullsized":"2024-04-25T11:12:19.387464+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:13.310787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029728+0000","last_change":"2024-04-25T11:12:17.649413+0000","last_active":"2024-04-25T11:12:19.029728+0000","last_peered":"2024-04-25T11:12:19.029728+0000","last_clean":"2024-04-25T11:12:19.029728+0000","last_became_active":"2024-04-25T11:12:17.649039+0000","last_became_peered":"2024-04-25T11:12:17.649039+0000","last_unstale":"2024-04-25T11:12:19.029728+0000","last_undegraded":"2024-04-25T11:12:19.029728+0000","last_fullsized":"2024-04-25T11:12:19.029728+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:20.586779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217424+0000","last_change":"2024-04-25T11:12:17.631797+0000","last_active":"2024-04-25T11:12:19.217424+0000","last_peered":"2024-04-25T11:12:19.217424+0000","last_clean":"2024-04-25T11:12:19.217424+0000","last_became_active":"2024-04-25T11:12:17.631467+0000","last_became_peered":"2024-04-25T11:12:17.631467+0000","last_unstale":"2024-04-25T11:12:19.217424+0000","last_undegraded":"2024-04-25T11:12:19.217424+0000","last_fullsized":"2024-04-25T11:12:19.217424+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:42.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217310+0000","last_change":"2024-04-25T11:12:17.634537+0000","last_active":"2024-04-25T11:12:19.217310+0000","last_peered":"2024-04-25T11:12:19.217310+0000","last_clean":"2024-04-25T11:12:19.217310+0000","last_became_active":"2024-04-25T11:12:17.634127+0000","last_became_peered":"2024-04-25T11:12:17.634127+0000","last_unstale":"2024-04-25T11:12:19.217310+0000","last_undegraded":"2024-04-25T11:12:19.217310+0000","last_fullsized":"2024-04-25T11:12:19.217310+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:38.365436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029099+0000","last_change":"2024-04-25T11:12:17.634556+0000","last_active":"2024-04-25T11:12:19.029099+0000","last_peered":"2024-04-25T11:12:19.029099+0000","last_clean":"2024-04-25T11:12:19.029099+0000","last_became_active":"2024-04-25T11:12:17.634259+0000","last_became_peered":"2024-04-25T11:12:17.634259+0000","last_unstale":"2024-04-25T11:12:19.029099+0000","last_undegraded":"2024-04-25T11:12:19.029099+0000","last_fullsized":"2024-04-25T11:12:19.029099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:11.180085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216341+0000","last_change":"2024-04-25T11:12:17.627970+0000","last_active":"2024-04-25T11:12:19.216341+0000","last_peered":"2024-04-25T11:12:19.216341+0000","last_clean":"2024-04-25T11:12:19.216341+0000","last_became_active":"2024-04-25T11:12:17.627655+0000","last_became_peered":"2024-04-25T11:12:17.627655+0000","last_unstale":"2024-04-25T11:12:19.216341+0000","last_undegraded":"2024-04-25T11:12:19.216341+0000","last_fullsized":"2024-04-25T11:12:19.216341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:13.310787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387126+0000","last_change":"2024-04-25T11:12:17.635455+0000","last_active":"2024-04-25T11:12:19.387126+0000","last_peered":"2024-04-25T11:12:19.387126+0000","last_clean":"2024-04-25T11:12:19.387126+0000","last_became_active":"2024-04-25T11:12:17.635133+0000","last_became_peered":"2024-04-25T11:12:17.635133+0000","last_unstale":"2024-04-25T11:12:19.387126+0000","last_undegraded":"2024-04-25T11:12:19.387126+0000","last_fullsized":"2024-04-25T11:12:19.387126+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:50.586215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216429+0000","last_change":"2024-04-25T11:12:17.634659+0000","last_active":"2024-04-25T11:12:19.216429+0000","last_peered":"2024-04-25T11:12:19.216429+0000","last_clean":"2024-04-25T11:12:19.216429+0000","last_became_active":"2024-04-25T11:12:17.632268+0000","last_became_peered":"2024-04-25T11:12:17.632268+0000","last_unstale":"2024-04-25T11:12:19.216429+0000","last_undegraded":"2024-04-25T11:12:19.216429+0000","last_fullsized":"2024-04-25T11:12:19.216429+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:31.231032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.617824+0000","last_change":"2024-04-25T11:12:17.643973+0000","last_active":"2024-04-25T11:12:18.617824+0000","last_peered":"2024-04-25T11:12:18.617824+0000","last_clean":"2024-04-25T11:12:18.617824+0000","last_became_active":"2024-04-25T11:12:17.643734+0000","last_became_peered":"2024-04-25T11:12:17.643734+0000","last_unstale":"2024-04-25T11:12:18.617824+0000","last_undegraded":"2024-04-25T11:12:18.617824+0000","last_fullsized":"2024-04-25T11:12:18.617824+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.157140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618809+0000","last_change":"2024-04-25T11:12:17.642118+0000","last_active":"2024-04-25T11:12:18.618809+0000","last_peered":"2024-04-25T11:12:18.618809+0000","last_clean":"2024-04-25T11:12:18.618809+0000","last_became_active":"2024-04-25T11:12:17.641802+0000","last_became_peered":"2024-04-25T11:12:17.641802+0000","last_unstale":"2024-04-25T11:12:18.618809+0000","last_undegraded":"2024-04-25T11:12:18.618809+0000","last_fullsized":"2024-04-25T11:12:18.618809+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:20.586779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621102+0000","last_change":"2024-04-25T11:12:17.626836+0000","last_active":"2024-04-25T11:12:18.621102+0000","last_peered":"2024-04-25T11:12:18.621102+0000","last_clean":"2024-04-25T11:12:18.621102+0000","last_became_active":"2024-04-25T11:12:17.625823+0000","last_became_peered":"2024-04-25T11:12:17.625823+0000","last_unstale":"2024-04-25T11:12:18.621102+0000","last_undegraded":"2024-04-25T11:12:18.621102+0000","last_fullsized":"2024-04-25T11:12:18.621102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:49.885461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.622328+0000","last_change":"2024-04-25T11:12:17.615647+0000","last_active":"2024-04-25T11:12:17.622328+0000","last_peered":"2024-04-25T11:12:17.622328+0000","last_clean":"2024-04-25T11:12:17.622328+0000","last_became_active":"2024-04-25T11:12:17.612194+0000","last_became_peered":"2024-04-25T11:12:17.612194+0000","last_unstale":"2024-04-25T11:12:17.622328+0000","last_undegraded":"2024-04-25T11:12:17.622328+0000","last_fullsized":"2024-04-25T11:12:17.622328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:35.393864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621181+0000","last_change":"2024-04-25T11:12:17.630309+0000","last_active":"2024-04-25T11:12:18.621181+0000","last_peered":"2024-04-25T11:12:18.621181+0000","last_clean":"2024-04-25T11:12:18.621181+0000","last_became_active":"2024-04-25T11:12:17.629536+0000","last_became_peered":"2024-04-25T11:12:17.629536+0000","last_unstale":"2024-04-25T11:12:18.621181+0000","last_undegraded":"2024-04-25T11:12:18.621181+0000","last_fullsized":"2024-04-25T11:12:18.621181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:50.586215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216728+0000","last_change":"2024-04-25T11:12:17.645960+0000","last_active":"2024-04-25T11:12:19.216728+0000","last_peered":"2024-04-25T11:12:19.216728+0000","last_clean":"2024-04-25T11:12:19.216728+0000","last_became_active":"2024-04-25T11:12:17.634389+0000","last_became_peered":"2024-04-25T11:12:17.634389+0000","last_unstale":"2024-04-25T11:12:19.216728+0000","last_undegraded":"2024-04-25T11:12:19.216728+0000","last_fullsized":"2024-04-25T11:12:19.216728+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:57.080529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388378+0000","last_change":"2024-04-25T11:12:17.617332+0000","last_active":"2024-04-25T11:12:19.388378+0000","last_peered":"2024-04-25T11:12:19.388378+0000","last_clean":"2024-04-25T11:12:19.388378+0000","last_became_active":"2024-04-25T11:12:17.616974+0000","last_became_peered":"2024-04-25T11:12:17.616974+0000","last_unstale":"2024-04-25T11:12:19.388378+0000","last_undegraded":"2024-04-25T11:12:19.388378+0000","last_fullsized":"2024-04-25T11:12:19.388378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:42.809695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618916+0000","last_change":"2024-04-25T11:12:17.632581+0000","last_active":"2024-04-25T11:12:18.618916+0000","last_peered":"2024-04-25T11:12:18.618916+0000","last_clean":"2024-04-25T11:12:18.618916+0000","last_became_active":"2024-04-25T11:12:17.632267+0000","last_became_peered":"2024-04-25T11:12:17.632267+0000","last_unstale":"2024-04-25T11:12:18.618916+0000","last_undegraded":"2024-04-25T11:12:18.618916+0000","last_fullsized":"2024-04-25T11:12:18.618916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:20.675156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.633055+0000","last_change":"2024-04-25T11:12:17.624993+0000","last_active":"2024-04-25T11:12:17.633055+0000","last_peered":"2024-04-25T11:12:17.633055+0000","last_clean":"2024-04-25T11:12:17.633055+0000","last_became_active":"2024-04-25T11:12:17.624291+0000","last_became_peered":"2024-04-25T11:12:17.624291+0000","last_unstale":"2024-04-25T11:12:17.633055+0000","last_undegraded":"2024-04-25T11:12:17.633055+0000","last_fullsized":"2024-04-25T11:12:17.633055+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.079603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621740+0000","last_change":"2024-04-25T11:12:17.641534+0000","last_active":"2024-04-25T11:12:18.621740+0000","last_peered":"2024-04-25T11:12:18.621740+0000","last_clean":"2024-04-25T11:12:18.621740+0000","last_became_active":"2024-04-25T11:12:17.640142+0000","last_became_peered":"2024-04-25T11:12:17.640142+0000","last_unstale":"2024-04-25T11:12:18.621740+0000","last_undegraded":"2024-04-25T11:12:18.621740+0000","last_fullsized":"2024-04-25T11:12:18.621740+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:28.331253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615848+0000","last_change":"2024-04-25T11:12:17.635251+0000","last_active":"2024-04-25T11:12:18.615848+0000","last_peered":"2024-04-25T11:12:18.615848+0000","last_clean":"2024-04-25T11:12:18.615848+0000","last_became_active":"2024-04-25T11:12:17.634913+0000","last_became_peered":"2024-04-25T11:12:17.634913+0000","last_unstale":"2024-04-25T11:12:18.615848+0000","last_undegraded":"2024-04-25T11:12:18.615848+0000","last_fullsized":"2024-04-25T11:12:18.615848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:11.180085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621953+0000","last_change":"2024-04-25T11:12:17.631494+0000","last_active":"2024-04-25T11:12:18.621953+0000","last_peered":"2024-04-25T11:12:18.621953+0000","last_clean":"2024-04-25T11:12:18.621953+0000","last_became_active":"2024-04-25T11:12:17.631010+0000","last_became_peered":"2024-04-25T11:12:17.631010+0000","last_unstale":"2024-04-25T11:12:18.621953+0000","last_undegraded":"2024-04-25T11:12:18.621953+0000","last_fullsized":"2024-04-25T11:12:18.621953+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:37.354273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216432+0000","last_change":"2024-04-25T11:12:17.640460+0000","last_active":"2024-04-25T11:12:19.216432+0000","last_peered":"2024-04-25T11:12:19.216432+0000","last_clean":"2024-04-25T11:12:19.216432+0000","last_became_active":"2024-04-25T11:12:17.640112+0000","last_became_peered":"2024-04-25T11:12:17.640112+0000","last_unstale":"2024-04-25T11:12:19.216432+0000","last_undegraded":"2024-04-25T11:12:19.216432+0000","last_fullsized":"2024-04-25T11:12:19.216432+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:07.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621755+0000","last_change":"2024-04-25T11:12:17.633084+0000","last_active":"2024-04-25T11:12:18.621755+0000","last_peered":"2024-04-25T11:12:18.621755+0000","last_clean":"2024-04-25T11:12:18.621755+0000","last_became_active":"2024-04-25T11:12:17.632763+0000","last_became_peered":"2024-04-25T11:12:17.632763+0000","last_unstale":"2024-04-25T11:12:18.621755+0000","last_undegraded":"2024-04-25T11:12:18.621755+0000","last_fullsized":"2024-04-25T11:12:18.621755+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:16.975633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.620664+0000","last_change":"2024-04-25T11:12:17.633820+0000","last_active":"2024-04-25T11:12:18.620664+0000","last_peered":"2024-04-25T11:12:18.620664+0000","last_clean":"2024-04-25T11:12:18.620664+0000","last_became_active":"2024-04-25T11:12:17.629365+0000","last_became_peered":"2024-04-25T11:12:17.629365+0000","last_unstale":"2024-04-25T11:12:18.620664+0000","last_undegraded":"2024-04-25T11:12:18.620664+0000","last_fullsized":"2024-04-25T11:12:18.620664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:35.028447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.619407+0000","last_change":"2024-04-25T11:12:17.621908+0000","last_active":"2024-04-25T11:12:18.619407+0000","last_peered":"2024-04-25T11:12:18.619407+0000","last_clean":"2024-04-25T11:12:18.619407+0000","last_became_active":"2024-04-25T11:12:17.621566+0000","last_became_peered":"2024-04-25T11:12:17.621566+0000","last_unstale":"2024-04-25T11:12:18.619407+0000","last_undegraded":"2024-04-25T11:12:18.619407+0000","last_fullsized":"2024-04-25T11:12:18.619407+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:48.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615085+0000","last_change":"2024-04-25T11:12:17.620893+0000","last_active":"2024-04-25T11:12:18.615085+0000","last_peered":"2024-04-25T11:12:18.615085+0000","last_clean":"2024-04-25T11:12:18.615085+0000","last_became_active":"2024-04-25T11:12:17.620562+0000","last_became_peered":"2024-04-25T11:12:17.620562+0000","last_unstale":"2024-04-25T11:12:18.615085+0000","last_undegraded":"2024-04-25T11:12:18.615085+0000","last_fullsized":"2024-04-25T11:12:18.615085+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:42.809695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618885+0000","last_change":"2024-04-25T11:12:17.644751+0000","last_active":"2024-04-25T11:12:18.618885+0000","last_peered":"2024-04-25T11:12:18.618885+0000","last_clean":"2024-04-25T11:12:18.618885+0000","last_became_active":"2024-04-25T11:12:17.644448+0000","last_became_peered":"2024-04-25T11:12:17.644448+0000","last_unstale":"2024-04-25T11:12:18.618885+0000","last_undegraded":"2024-04-25T11:12:18.618885+0000","last_fullsized":"2024-04-25T11:12:18.618885+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:29.679865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621562+0000","last_change":"2024-04-25T11:12:16.633833+0000","last_active":"2024-04-25T11:12:18.621562+0000","last_peered":"2024-04-25T11:12:18.621562+0000","last_clean":"2024-04-25T11:12:18.621562+0000","last_became_active":"2024-04-25T11:12:16.633462+0000","last_became_peered":"2024-04-25T11:12:16.633462+0000","last_unstale":"2024-04-25T11:12:18.621562+0000","last_undegraded":"2024-04-25T11:12:18.621562+0000","last_fullsized":"2024-04-25T11:12:18.621562+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.828983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.616203+0000","last_change":"2024-04-25T11:12:17.635395+0000","last_active":"2024-04-25T11:12:18.616203+0000","last_peered":"2024-04-25T11:12:18.616203+0000","last_clean":"2024-04-25T11:12:18.616203+0000","last_became_active":"2024-04-25T11:12:17.635054+0000","last_became_peered":"2024-04-25T11:12:17.635054+0000","last_unstale":"2024-04-25T11:12:18.616203+0000","last_undegraded":"2024-04-25T11:12:18.616203+0000","last_fullsized":"2024-04-25T11:12:18.616203+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.157140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029586+0000","last_change":"2024-04-25T11:12:16.626706+0000","last_active":"2024-04-25T11:12:19.029586+0000","last_peered":"2024-04-25T11:12:19.029586+0000","last_clean":"2024-04-25T11:12:19.029586+0000","last_became_active":"2024-04-25T11:12:16.623128+0000","last_became_peered":"2024-04-25T11:12:16.623128+0000","last_unstale":"2024-04-25T11:12:19.029586+0000","last_undegraded":"2024-04-25T11:12:19.029586+0000","last_fullsized":"2024-04-25T11:12:19.029586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:07.877880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388103+0000","last_change":"2024-04-25T11:12:17.631438+0000","last_active":"2024-04-25T11:12:19.388103+0000","last_peered":"2024-04-25T11:12:19.388103+0000","last_clean":"2024-04-25T11:12:19.388103+0000","last_became_active":"2024-04-25T11:12:17.630702+0000","last_became_peered":"2024-04-25T11:12:17.630702+0000","last_unstale":"2024-04-25T11:12:19.388103+0000","last_undegraded":"2024-04-25T11:12:19.388103+0000","last_fullsized":"2024-04-25T11:12:19.388103+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:42.282697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.616546+0000","last_change":"2024-04-25T11:12:16.633736+0000","last_active":"2024-04-25T11:12:18.616546+0000","last_peered":"2024-04-25T11:12:18.616546+0000","last_clean":"2024-04-25T11:12:18.616546+0000","last_became_active":"2024-04-25T11:12:16.633410+0000","last_became_peered":"2024-04-25T11:12:16.633410+0000","last_unstale":"2024-04-25T11:12:18.616546+0000","last_undegraded":"2024-04-25T11:12:18.616546+0000","last_fullsized":"2024-04-25T11:12:18.616546+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:03.462442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.390168+0000","last_change":"2024-04-25T11:12:17.638817+0000","last_active":"2024-04-25T11:12:19.390168+0000","last_peered":"2024-04-25T11:12:19.390168+0000","last_clean":"2024-04-25T11:12:19.390168+0000","last_became_active":"2024-04-25T11:12:17.638481+0000","last_became_peered":"2024-04-25T11:12:17.638481+0000","last_unstale":"2024-04-25T11:12:19.390168+0000","last_undegraded":"2024-04-25T11:12:19.390168+0000","last_fullsized":"2024-04-25T11:12:19.390168+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:27.521860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387630+0000","last_change":"2024-04-25T11:12:16.631138+0000","last_active":"2024-04-25T11:12:19.387630+0000","last_peered":"2024-04-25T11:12:19.387630+0000","last_clean":"2024-04-25T11:12:19.387630+0000","last_became_active":"2024-04-25T11:12:16.630006+0000","last_became_peered":"2024-04-25T11:12:16.630006+0000","last_unstale":"2024-04-25T11:12:19.387630+0000","last_undegraded":"2024-04-25T11:12:19.387630+0000","last_fullsized":"2024-04-25T11:12:19.387630+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:41.402376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.028397+0000","last_change":"2024-04-25T11:12:17.637173+0000","last_active":"2024-04-25T11:12:19.028397+0000","last_peered":"2024-04-25T11:12:19.028397+0000","last_clean":"2024-04-25T11:12:19.028397+0000","last_became_active":"2024-04-25T11:12:17.636870+0000","last_became_peered":"2024-04-25T11:12:17.636870+0000","last_unstale":"2024-04-25T11:12:19.028397+0000","last_undegraded":"2024-04-25T11:12:19.028397+0000","last_fullsized":"2024-04-25T11:12:19.028397+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:45.703997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216493+0000","last_change":"2024-04-25T11:12:16.629015+0000","last_active":"2024-04-25T11:12:19.216493+0000","last_peered":"2024-04-25T11:12:19.216493+0000","last_clean":"2024-04-25T11:12:19.216493+0000","last_became_active":"2024-04-25T11:12:16.628692+0000","last_became_peered":"2024-04-25T11:12:16.628692+0000","last_unstale":"2024-04-25T11:12:19.216493+0000","last_undegraded":"2024-04-25T11:12:19.216493+0000","last_fullsized":"2024-04-25T11:12:19.216493+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:48.867738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216768+0000","last_change":"2024-04-25T11:12:17.639756+0000","last_active":"2024-04-25T11:12:19.216768+0000","last_peered":"2024-04-25T11:12:19.216768+0000","last_clean":"2024-04-25T11:12:19.216768+0000","last_became_active":"2024-04-25T11:12:17.639081+0000","last_became_peered":"2024-04-25T11:12:17.639081+0000","last_unstale":"2024-04-25T11:12:19.216768+0000","last_undegraded":"2024-04-25T11:12:19.216768+0000","last_fullsized":"2024-04-25T11:12:19.216768+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:35.196880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.619998+0000","last_change":"2024-04-25T11:12:16.634253+0000","last_active":"2024-04-25T11:12:18.619998+0000","last_peered":"2024-04-25T11:12:18.619998+0000","last_clean":"2024-04-25T11:12:18.619998+0000","last_became_active":"2024-04-25T11:12:16.633894+0000","last_became_peered":"2024-04-25T11:12:16.633894+0000","last_unstale":"2024-04-25T11:12:18.619998+0000","last_undegraded":"2024-04-25T11:12:18.619998+0000","last_fullsized":"2024-04-25T11:12:18.619998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:03.462442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387702+0000","last_change":"2024-04-25T11:12:17.640260+0000","last_active":"2024-04-25T11:12:19.387702+0000","last_peered":"2024-04-25T11:12:19.387702+0000","last_clean":"2024-04-25T11:12:19.387702+0000","last_became_active":"2024-04-25T11:12:17.639569+0000","last_became_peered":"2024-04-25T11:12:17.639569+0000","last_unstale":"2024-04-25T11:12:19.387702+0000","last_undegraded":"2024-04-25T11:12:19.387702+0000","last_fullsized":"2024-04-25T11:12:19.387702+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:16.975633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615364+0000","last_change":"2024-04-25T11:12:16.626419+0000","last_active":"2024-04-25T11:12:18.615364+0000","last_peered":"2024-04-25T11:12:18.615364+0000","last_clean":"2024-04-25T11:12:18.615364+0000","last_became_active":"2024-04-25T11:12:16.626101+0000","last_became_peered":"2024-04-25T11:12:16.626101+0000","last_unstale":"2024-04-25T11:12:18.615364+0000","last_undegraded":"2024-04-25T11:12:18.615364+0000","last_fullsized":"2024-04-25T11:12:18.615364+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:07.877880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388844+0000","last_change":"2024-04-25T11:12:17.633901+0000","last_active":"2024-04-25T11:12:19.388844+0000","last_peered":"2024-04-25T11:12:19.388844+0000","last_clean":"2024-04-25T11:12:19.388844+0000","last_became_active":"2024-04-25T11:12:17.633561+0000","last_became_peered":"2024-04-25T11:12:17.633561+0000","last_unstale":"2024-04-25T11:12:19.388844+0000","last_undegraded":"2024-04-25T11:12:19.388844+0000","last_fullsized":"2024-04-25T11:12:19.388844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:07.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.386681+0000","last_change":"2024-04-25T11:12:16.627875+0000","last_active":"2024-04-25T11:12:19.386681+0000","last_peered":"2024-04-25T11:12:19.386681+0000","last_clean":"2024-04-25T11:12:19.386681+0000","last_became_active":"2024-04-25T11:12:16.627521+0000","last_became_peered":"2024-04-25T11:12:16.627521+0000","last_unstale":"2024-04-25T11:12:19.386681+0000","last_undegraded":"2024-04-25T11:12:19.386681+0000","last_fullsized":"2024-04-25T11:12:19.386681+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:51.916261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.631722+0000","last_change":"2024-04-25T11:12:17.623540+0000","last_active":"2024-04-25T11:12:17.631722+0000","last_peered":"2024-04-25T11:12:17.631722+0000","last_clean":"2024-04-25T11:12:17.631722+0000","last_became_active":"2024-04-25T11:12:17.622946+0000","last_became_peered":"2024-04-25T11:12:17.622946+0000","last_unstale":"2024-04-25T11:12:17.631722+0000","last_undegraded":"2024-04-25T11:12:17.631722+0000","last_fullsized":"2024-04-25T11:12:17.631722+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:54.933443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.390476+0000","last_change":"2024-04-25T11:12:16.623790+0000","last_active":"2024-04-25T11:12:19.390476+0000","last_peered":"2024-04-25T11:12:19.390476+0000","last_clean":"2024-04-25T11:12:19.390476+0000","last_became_active":"2024-04-25T11:12:16.623452+0000","last_became_peered":"2024-04-25T11:12:16.623452+0000","last_unstale":"2024-04-25T11:12:19.390476+0000","last_undegraded":"2024-04-25T11:12:19.390476+0000","last_fullsized":"2024-04-25T11:12:19.390476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:41.791972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.620021+0000","last_change":"2024-04-25T11:12:17.634206+0000","last_active":"2024-04-25T11:12:18.620021+0000","last_peered":"2024-04-25T11:12:18.620021+0000","last_clean":"2024-04-25T11:12:18.620021+0000","last_became_active":"2024-04-25T11:12:17.633879+0000","last_became_peered":"2024-04-25T11:12:17.633879+0000","last_unstale":"2024-04-25T11:12:18.620021+0000","last_undegraded":"2024-04-25T11:12:18.620021+0000","last_fullsized":"2024-04-25T11:12:18.620021+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:31.557454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217588+0000","last_change":"2024-04-25T11:12:16.623724+0000","last_active":"2024-04-25T11:12:19.217588+0000","last_peered":"2024-04-25T11:12:19.217588+0000","last_clean":"2024-04-25T11:12:19.217588+0000","last_became_active":"2024-04-25T11:12:16.622672+0000","last_became_peered":"2024-04-25T11:12:16.622672+0000","last_unstale":"2024-04-25T11:12:19.217588+0000","last_undegraded":"2024-04-25T11:12:19.217588+0000","last_fullsized":"2024-04-25T11:12:19.217588+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:15.957910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029390+0000","last_change":"2024-04-25T11:12:17.636237+0000","last_active":"2024-04-25T11:12:19.029390+0000","last_peered":"2024-04-25T11:12:19.029390+0000","last_clean":"2024-04-25T11:12:19.029390+0000","last_became_active":"2024-04-25T11:12:17.628700+0000","last_became_peered":"2024-04-25T11:12:17.628700+0000","last_unstale":"2024-04-25T11:12:19.029390+0000","last_undegraded":"2024-04-25T11:12:19.029390+0000","last_fullsized":"2024-04-25T11:12:19.029390+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:29.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217521+0000","last_change":"2024-04-25T11:12:16.628284+0000","last_active":"2024-04-25T11:12:19.217521+0000","last_peered":"2024-04-25T11:12:19.217521+0000","last_clean":"2024-04-25T11:12:19.217521+0000","last_became_active":"2024-04-25T11:12:16.625899+0000","last_became_peered":"2024-04-25T11:12:16.625899+0000","last_unstale":"2024-04-25T11:12:19.217521+0000","last_undegraded":"2024-04-25T11:12:19.217521+0000","last_fullsized":"2024-04-25T11:12:19.217521+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:34.010724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388022+0000","last_change":"2024-04-25T11:12:17.620298+0000","last_active":"2024-04-25T11:12:19.388022+0000","last_peered":"2024-04-25T11:12:19.388022+0000","last_clean":"2024-04-25T11:12:19.388022+0000","last_became_active":"2024-04-25T11:12:17.620023+0000","last_became_peered":"2024-04-25T11:12:17.620023+0000","last_unstale":"2024-04-25T11:12:19.388022+0000","last_undegraded":"2024-04-25T11:12:19.388022+0000","last_fullsized":"2024-04-25T11:12:19.388022+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:45.703997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389193+0000","last_change":"2024-04-25T11:12:16.624743+0000","last_active":"2024-04-25T11:12:19.389193+0000","last_peered":"2024-04-25T11:12:19.389193+0000","last_clean":"2024-04-25T11:12:19.389193+0000","last_became_active":"2024-04-25T11:12:16.624340+0000","last_became_peered":"2024-04-25T11:12:16.624340+0000","last_unstale":"2024-04-25T11:12:19.389193+0000","last_undegraded":"2024-04-25T11:12:19.389193+0000","last_fullsized":"2024-04-25T11:12:19.389193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:41.264974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.028520+0000","last_change":"2024-04-25T11:12:17.649420+0000","last_active":"2024-04-25T11:12:19.028520+0000","last_peered":"2024-04-25T11:12:19.028520+0000","last_clean":"2024-04-25T11:12:19.028520+0000","last_became_active":"2024-04-25T11:12:17.649092+0000","last_became_peered":"2024-04-25T11:12:17.649092+0000","last_unstale":"2024-04-25T11:12:19.028520+0000","last_undegraded":"2024-04-25T11:12:19.028520+0000","last_fullsized":"2024-04-25T11:12:19.028520+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:42.282697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389269+0000","last_change":"2024-04-25T11:12:16.626640+0000","last_active":"2024-04-25T11:12:19.389269+0000","last_peered":"2024-04-25T11:12:19.389269+0000","last_clean":"2024-04-25T11:12:19.389269+0000","last_became_active":"2024-04-25T11:12:16.626323+0000","last_became_peered":"2024-04-25T11:12:16.626323+0000","last_unstale":"2024-04-25T11:12:19.389269+0000","last_undegraded":"2024-04-25T11:12:19.389269+0000","last_fullsized":"2024-04-25T11:12:19.389269+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:44.686274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388293+0000","last_change":"2024-04-25T11:12:17.639174+0000","last_active":"2024-04-25T11:12:19.388293+0000","last_peered":"2024-04-25T11:12:19.388293+0000","last_clean":"2024-04-25T11:12:19.388293+0000","last_became_active":"2024-04-25T11:12:17.638842+0000","last_became_peered":"2024-04-25T11:12:17.638842+0000","last_unstale":"2024-04-25T11:12:19.388293+0000","last_undegraded":"2024-04-25T11:12:19.388293+0000","last_fullsized":"2024-04-25T11:12:19.388293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:31.231032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615189+0000","last_change":"2024-04-25T11:12:16.626699+0000","last_active":"2024-04-25T11:12:18.615189+0000","last_peered":"2024-04-25T11:12:18.615189+0000","last_clean":"2024-04-25T11:12:18.615189+0000","last_became_active":"2024-04-25T11:12:16.625059+0000","last_became_peered":"2024-04-25T11:12:16.625059+0000","last_unstale":"2024-04-25T11:12:18.615189+0000","last_undegraded":"2024-04-25T11:12:18.615189+0000","last_fullsized":"2024-04-25T11:12:18.615189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:00.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.619817+0000","last_change":"2024-04-25T11:12:17.613099+0000","last_active":"2024-04-25T11:12:18.619817+0000","last_peered":"2024-04-25T11:12:18.619817+0000","last_clean":"2024-04-25T11:12:18.619817+0000","last_became_active":"2024-04-25T11:12:17.612698+0000","last_became_peered":"2024-04-25T11:12:17.612698+0000","last_unstale":"2024-04-25T11:12:18.619817+0000","last_undegraded":"2024-04-25T11:12:18.619817+0000","last_fullsized":"2024-04-25T11:12:18.619817+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:33.935499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618748+0000","last_change":"2024-04-25T11:12:16.625452+0000","last_active":"2024-04-25T11:12:18.618748+0000","last_peered":"2024-04-25T11:12:18.618748+0000","last_clean":"2024-04-25T11:12:18.618748+0000","last_became_active":"2024-04-25T11:12:16.625104+0000","last_became_peered":"2024-04-25T11:12:16.625104+0000","last_unstale":"2024-04-25T11:12:18.618748+0000","last_undegraded":"2024-04-25T11:12:18.618748+0000","last_fullsized":"2024-04-25T11:12:18.618748+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:05.439617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388846+0000","last_change":"2024-04-25T11:12:17.631758+0000","last_active":"2024-04-25T11:12:19.388846+0000","last_peered":"2024-04-25T11:12:19.388846+0000","last_clean":"2024-04-25T11:12:19.388846+0000","last_became_active":"2024-04-25T11:12:17.631272+0000","last_became_peered":"2024-04-25T11:12:17.631272+0000","last_unstale":"2024-04-25T11:12:19.388846+0000","last_undegraded":"2024-04-25T11:12:19.388846+0000","last_fullsized":"2024-04-25T11:12:19.388846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:03.197485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.617731+0000","last_change":"2024-04-25T11:12:16.631114+0000","last_active":"2024-04-25T11:12:18.617731+0000","last_peered":"2024-04-25T11:12:18.617731+0000","last_clean":"2024-04-25T11:12:18.617731+0000","last_became_active":"2024-04-25T11:12:16.630322+0000","last_became_peered":"2024-04-25T11:12:16.630322+0000","last_unstale":"2024-04-25T11:12:18.617731+0000","last_undegraded":"2024-04-25T11:12:18.617731+0000","last_fullsized":"2024-04-25T11:12:18.617731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:34.010724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389139+0000","last_change":"2024-04-25T11:12:17.640574+0000","last_active":"2024-04-25T11:12:19.389139+0000","last_peered":"2024-04-25T11:12:19.389139+0000","last_clean":"2024-04-25T11:12:19.389139+0000","last_became_active":"2024-04-25T11:12:17.640233+0000","last_became_peered":"2024-04-25T11:12:17.640233+0000","last_unstale":"2024-04-25T11:12:19.389139+0000","last_undegraded":"2024-04-25T11:12:19.389139+0000","last_fullsized":"2024-04-25T11:12:19.389139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:57.080529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217584+0000","last_change":"2024-04-25T11:12:16.634084+0000","last_active":"2024-04-25T11:12:19.217584+0000","last_peered":"2024-04-25T11:12:19.217584+0000","last_clean":"2024-04-25T11:12:19.217584+0000","last_became_active":"2024-04-25T11:12:16.633713+0000","last_became_peered":"2024-04-25T11:12:16.633713+0000","last_unstale":"2024-04-25T11:12:19.217584+0000","last_undegraded":"2024-04-25T11:12:19.217584+0000","last_fullsized":"2024-04-25T11:12:19.217584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.568492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217492+0000","last_change":"2024-04-25T11:12:17.639082+0000","last_active":"2024-04-25T11:12:19.217492+0000","last_peered":"2024-04-25T11:12:19.217492+0000","last_clean":"2024-04-25T11:12:19.217492+0000","last_became_active":"2024-04-25T11:12:17.638766+0000","last_became_peered":"2024-04-25T11:12:17.638766+0000","last_unstale":"2024-04-25T11:12:19.217492+0000","last_undegraded":"2024-04-25T11:12:19.217492+0000","last_fullsized":"2024-04-25T11:12:19.217492+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:27.521860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388197+0000","last_change":"2024-04-25T11:12:16.625999+0000","last_active":"2024-04-25T11:12:19.388197+0000","last_peered":"2024-04-25T11:12:19.388197+0000","last_clean":"2024-04-25T11:12:19.388197+0000","last_became_active":"2024-04-25T11:12:16.625512+0000","last_became_peered":"2024-04-25T11:12:16.625512+0000","last_unstale":"2024-04-25T11:12:19.388197+0000","last_undegraded":"2024-04-25T11:12:19.388197+0000","last_fullsized":"2024-04-25T11:12:19.388197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:50.720849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387986+0000","last_change":"2024-04-25T11:12:17.631746+0000","last_active":"2024-04-25T11:12:19.387986+0000","last_peered":"2024-04-25T11:12:19.387986+0000","last_clean":"2024-04-25T11:12:19.387986+0000","last_became_active":"2024-04-25T11:12:17.631127+0000","last_became_peered":"2024-04-25T11:12:17.631127+0000","last_unstale":"2024-04-25T11:12:19.387986+0000","last_undegraded":"2024-04-25T11:12:19.387986+0000","last_fullsized":"2024-04-25T11:12:19.387986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:37.354273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.628092+0000","last_change":"2024-04-25T11:12:16.622650+0000","last_active":"2024-04-25T11:12:17.628092+0000","last_peered":"2024-04-25T11:12:17.628092+0000","last_clean":"2024-04-25T11:12:17.628092+0000","last_became_active":"2024-04-25T11:12:16.622326+0000","last_became_peered":"2024-04-25T11:12:16.622326+0000","last_unstale":"2024-04-25T11:12:17.628092+0000","last_undegraded":"2024-04-25T11:12:17.628092+0000","last_fullsized":"2024-04-25T11:12:17.628092+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:29.897108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.634283+0000","last_change":"2024-04-25T11:12:17.627698+0000","last_active":"2024-04-25T11:12:17.634283+0000","last_peered":"2024-04-25T11:12:17.634283+0000","last_clean":"2024-04-25T11:12:17.634283+0000","last_became_active":"2024-04-25T11:12:17.626871+0000","last_became_peered":"2024-04-25T11:12:17.626871+0000","last_unstale":"2024-04-25T11:12:17.634283+0000","last_undegraded":"2024-04-25T11:12:17.634283+0000","last_fullsized":"2024-04-25T11:12:17.634283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:08.842985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615540+0000","last_change":"2024-04-25T11:12:16.631034+0000","last_active":"2024-04-25T11:12:18.615540+0000","last_peered":"2024-04-25T11:12:18.615540+0000","last_clean":"2024-04-25T11:12:18.615540+0000","last_became_active":"2024-04-25T11:12:16.630509+0000","last_became_peered":"2024-04-25T11:12:16.630509+0000","last_unstale":"2024-04-25T11:12:18.615540+0000","last_undegraded":"2024-04-25T11:12:18.615540+0000","last_fullsized":"2024-04-25T11:12:18.615540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:28.137337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618299+0000","last_change":"2024-04-25T11:12:17.623664+0000","last_active":"2024-04-25T11:12:18.618299+0000","last_peered":"2024-04-25T11:12:18.618299+0000","last_clean":"2024-04-25T11:12:18.618299+0000","last_became_active":"2024-04-25T11:12:17.622056+0000","last_became_peered":"2024-04-25T11:12:17.622056+0000","last_unstale":"2024-04-25T11:12:18.618299+0000","last_undegraded":"2024-04-25T11:12:18.618299+0000","last_fullsized":"2024-04-25T11:12:18.618299+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:29.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.632472+0000","last_change":"2024-04-25T11:12:16.631385+0000","last_active":"2024-04-25T11:12:17.632472+0000","last_peered":"2024-04-25T11:12:17.632472+0000","last_clean":"2024-04-25T11:12:17.632472+0000","last_became_active":"2024-04-25T11:12:16.631053+0000","last_became_peered":"2024-04-25T11:12:16.631053+0000","last_unstale":"2024-04-25T11:12:17.632472+0000","last_undegraded":"2024-04-25T11:12:17.632472+0000","last_fullsized":"2024-04-25T11:12:17.632472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:26.584206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.028799+0000","last_change":"2024-04-25T11:12:17.630867+0000","last_active":"2024-04-25T11:12:19.028799+0000","last_peered":"2024-04-25T11:12:19.028799+0000","last_clean":"2024-04-25T11:12:19.028799+0000","last_became_active":"2024-04-25T11:12:17.630555+0000","last_became_peered":"2024-04-25T11:12:17.630555+0000","last_unstale":"2024-04-25T11:12:19.028799+0000","last_undegraded":"2024-04-25T11:12:19.028799+0000","last_fullsized":"2024-04-25T11:12:19.028799+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:25.139484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.597715+0000","last_change":"2024-04-25T11:12:16.626672+0000","last_active":"2024-04-25T11:12:17.597715+0000","last_peered":"2024-04-25T11:12:17.597715+0000","last_clean":"2024-04-25T11:12:17.597715+0000","last_became_active":"2024-04-25T11:12:16.625675+0000","last_became_peered":"2024-04-25T11:12:16.625675+0000","last_unstale":"2024-04-25T11:12:17.597715+0000","last_undegraded":"2024-04-25T11:12:17.597715+0000","last_fullsized":"2024-04-25T11:12:17.597715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:29.065743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216721+0000","last_change":"2024-04-25T11:12:17.646203+0000","last_active":"2024-04-25T11:12:19.216721+0000","last_peered":"2024-04-25T11:12:19.216721+0000","last_clean":"2024-04-25T11:12:19.216721+0000","last_became_active":"2024-04-25T11:12:17.640854+0000","last_became_peered":"2024-04-25T11:12:17.640854+0000","last_unstale":"2024-04-25T11:12:19.216721+0000","last_undegraded":"2024-04-25T11:12:19.216721+0000","last_fullsized":"2024-04-25T11:12:19.216721+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:03.197485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.619491+0000","last_change":"2024-04-25T11:12:16.624621+0000","last_active":"2024-04-25T11:12:18.619491+0000","last_peered":"2024-04-25T11:12:18.619491+0000","last_clean":"2024-04-25T11:12:18.619491+0000","last_became_active":"2024-04-25T11:12:16.624205+0000","last_became_peered":"2024-04-25T11:12:16.624205+0000","last_unstale":"2024-04-25T11:12:18.619491+0000","last_undegraded":"2024-04-25T11:12:18.619491+0000","last_fullsized":"2024-04-25T11:12:18.619491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:40.712090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389563+0000","last_change":"2024-04-25T11:12:17.632030+0000","last_active":"2024-04-25T11:12:19.389563+0000","last_peered":"2024-04-25T11:12:19.389563+0000","last_clean":"2024-04-25T11:12:19.389563+0000","last_became_active":"2024-04-25T11:12:17.631501+0000","last_became_peered":"2024-04-25T11:12:17.631501+0000","last_unstale":"2024-04-25T11:12:19.389563+0000","last_undegraded":"2024-04-25T11:12:19.389563+0000","last_fullsized":"2024-04-25T11:12:19.389563+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:38.365436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.598273+0000","last_change":"2024-04-25T11:12:16.631342+0000","last_active":"2024-04-25T11:12:17.598273+0000","last_peered":"2024-04-25T11:12:17.598273+0000","last_clean":"2024-04-25T11:12:17.598273+0000","last_became_active":"2024-04-25T11:12:16.631017+0000","last_became_peered":"2024-04-25T11:12:16.631017+0000","last_unstale":"2024-04-25T11:12:17.598273+0000","last_undegraded":"2024-04-25T11:12:17.598273+0000","last_fullsized":"2024-04-25T11:12:17.598273+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:58.555440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615742+0000","last_change":"2024-04-25T11:12:17.647803+0000","last_active":"2024-04-25T11:12:18.615742+0000","last_peered":"2024-04-25T11:12:18.615742+0000","last_clean":"2024-04-25T11:12:18.615742+0000","last_became_active":"2024-04-25T11:12:17.647327+0000","last_became_peered":"2024-04-25T11:12:17.647327+0000","last_unstale":"2024-04-25T11:12:18.615742+0000","last_undegraded":"2024-04-25T11:12:18.615742+0000","last_fullsized":"2024-04-25T11:12:18.615742+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:25.139484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217924+0000","last_change":"2024-04-25T11:12:16.624934+0000","last_active":"2024-04-25T11:12:19.217924+0000","last_peered":"2024-04-25T11:12:19.217924+0000","last_clean":"2024-04-25T11:12:19.217924+0000","last_became_active":"2024-04-25T11:12:16.623319+0000","last_became_peered":"2024-04-25T11:12:16.623319+0000","last_unstale":"2024-04-25T11:12:19.217924+0000","last_undegraded":"2024-04-25T11:12:19.217924+0000","last_fullsized":"2024-04-25T11:12:19.217924+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:27.313530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.620519+0000","last_change":"2024-04-25T11:12:17.629748+0000","last_active":"2024-04-25T11:12:18.620519+0000","last_peered":"2024-04-25T11:12:18.620519+0000","last_clean":"2024-04-25T11:12:18.620519+0000","last_became_active":"2024-04-25T11:12:17.629428+0000","last_became_peered":"2024-04-25T11:12:17.629428+0000","last_unstale":"2024-04-25T11:12:18.620519+0000","last_undegraded":"2024-04-25T11:12:18.620519+0000","last_fullsized":"2024-04-25T11:12:18.620519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:31.231032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.600794+0000","last_change":"2024-04-25T11:12:16.626094+0000","last_active":"2024-04-25T11:12:17.600794+0000","last_peered":"2024-04-25T11:12:17.600794+0000","last_clean":"2024-04-25T11:12:17.600794+0000","last_became_active":"2024-04-25T11:12:16.625415+0000","last_became_peered":"2024-04-25T11:12:16.625415+0000","last_unstale":"2024-04-25T11:12:17.600794+0000","last_undegraded":"2024-04-25T11:12:17.600794+0000","last_fullsized":"2024-04-25T11:12:17.600794+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:30.818798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389609+0000","last_change":"2024-04-25T11:12:17.635245+0000","last_active":"2024-04-25T11:12:19.389609+0000","last_peered":"2024-04-25T11:12:19.389609+0000","last_clean":"2024-04-25T11:12:19.389609+0000","last_became_active":"2024-04-25T11:12:17.634831+0000","last_became_peered":"2024-04-25T11:12:17.634831+0000","last_unstale":"2024-04-25T11:12:19.389609+0000","last_undegraded":"2024-04-25T11:12:19.389609+0000","last_fullsized":"2024-04-25T11:12:19.389609+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:42.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029281+0000","last_change":"2024-04-25T11:12:16.625237+0000","last_active":"2024-04-25T11:12:19.029281+0000","last_peered":"2024-04-25T11:12:19.029281+0000","last_clean":"2024-04-25T11:12:19.029281+0000","last_became_active":"2024-04-25T11:12:16.624927+0000","last_became_peered":"2024-04-25T11:12:16.624927+0000","last_unstale":"2024-04-25T11:12:19.029281+0000","last_undegraded":"2024-04-25T11:12:19.029281+0000","last_fullsized":"2024-04-25T11:12:19.029281+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:00.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.028612+0000","last_change":"2024-04-25T11:12:17.642157+0000","last_active":"2024-04-25T11:12:19.028612+0000","last_peered":"2024-04-25T11:12:19.028612+0000","last_clean":"2024-04-25T11:12:19.028612+0000","last_became_active":"2024-04-25T11:12:17.641908+0000","last_became_peered":"2024-04-25T11:12:17.641908+0000","last_unstale":"2024-04-25T11:12:19.028612+0000","last_undegraded":"2024-04-25T11:12:19.028612+0000","last_fullsized":"2024-04-25T11:12:19.028612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:42.809695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029263+0000","last_change":"2024-04-25T11:12:16.630633+0000","last_active":"2024-04-25T11:12:19.029263+0000","last_peered":"2024-04-25T11:12:19.029263+0000","last_clean":"2024-04-25T11:12:19.029263+0000","last_became_active":"2024-04-25T11:12:16.630089+0000","last_became_peered":"2024-04-25T11:12:16.630089+0000","last_unstale":"2024-04-25T11:12:19.029263+0000","last_undegraded":"2024-04-25T11:12:19.029263+0000","last_fullsized":"2024-04-25T11:12:19.029263+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:50.720849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.614928+0000","last_change":"2024-04-25T11:12:17.638972+0000","last_active":"2024-04-25T11:12:18.614928+0000","last_peered":"2024-04-25T11:12:18.614928+0000","last_clean":"2024-04-25T11:12:18.614928+0000","last_became_active":"2024-04-25T11:12:17.638630+0000","last_became_peered":"2024-04-25T11:12:17.638630+0000","last_unstale":"2024-04-25T11:12:18.614928+0000","last_undegraded":"2024-04-25T11:12:18.614928+0000","last_fullsized":"2024-04-25T11:12:18.614928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:45.703997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.620448+0000","last_change":"2024-04-25T11:12:16.623148+0000","last_active":"2024-04-25T11:12:18.620448+0000","last_peered":"2024-04-25T11:12:18.620448+0000","last_clean":"2024-04-25T11:12:18.620448+0000","last_became_active":"2024-04-25T11:12:16.622264+0000","last_became_peered":"2024-04-25T11:12:16.622264+0000","last_unstale":"2024-04-25T11:12:18.620448+0000","last_undegraded":"2024-04-25T11:12:18.620448+0000","last_fullsized":"2024-04-25T11:12:18.620448+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:41.402376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387169+0000","last_change":"2024-04-25T11:12:17.632213+0000","last_active":"2024-04-25T11:12:19.387169+0000","last_peered":"2024-04-25T11:12:19.387169+0000","last_clean":"2024-04-25T11:12:19.387169+0000","last_became_active":"2024-04-25T11:12:17.631877+0000","last_became_peered":"2024-04-25T11:12:17.631877+0000","last_unstale":"2024-04-25T11:12:19.387169+0000","last_undegraded":"2024-04-25T11:12:19.387169+0000","last_fullsized":"2024-04-25T11:12:19.387169+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:28.331253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.386708+0000","last_change":"2024-04-25T11:12:16.626722+0000","last_active":"2024-04-25T11:12:19.386708+0000","last_peered":"2024-04-25T11:12:19.386708+0000","last_clean":"2024-04-25T11:12:19.386708+0000","last_became_active":"2024-04-25T11:12:16.622129+0000","last_became_peered":"2024-04-25T11:12:16.622129+0000","last_unstale":"2024-04-25T11:12:19.386708+0000","last_undegraded":"2024-04-25T11:12:19.386708+0000","last_fullsized":"2024-04-25T11:12:19.386708+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:05.439617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.620501+0000","last_change":"2024-04-25T11:12:17.639962+0000","last_active":"2024-04-25T11:12:18.620501+0000","last_peered":"2024-04-25T11:12:18.620501+0000","last_clean":"2024-04-25T11:12:18.620501+0000","last_became_active":"2024-04-25T11:12:17.639536+0000","last_became_peered":"2024-04-25T11:12:17.639536+0000","last_unstale":"2024-04-25T11:12:18.620501+0000","last_undegraded":"2024-04-25T11:12:18.620501+0000","last_fullsized":"2024-04-25T11:12:18.620501+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:13.310787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388375+0000","last_change":"2024-04-25T11:12:16.623982+0000","last_active":"2024-04-25T11:12:19.388375+0000","last_peered":"2024-04-25T11:12:19.388375+0000","last_clean":"2024-04-25T11:12:19.388375+0000","last_became_active":"2024-04-25T11:12:16.623065+0000","last_became_peered":"2024-04-25T11:12:16.623065+0000","last_unstale":"2024-04-25T11:12:19.388375+0000","last_undegraded":"2024-04-25T11:12:19.388375+0000","last_fullsized":"2024-04-25T11:12:19.388375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:36.336550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217632+0000","last_change":"2024-04-25T11:12:17.638776+0000","last_active":"2024-04-25T11:12:19.217632+0000","last_peered":"2024-04-25T11:12:19.217632+0000","last_clean":"2024-04-25T11:12:19.217632+0000","last_became_active":"2024-04-25T11:12:17.638429+0000","last_became_peered":"2024-04-25T11:12:17.638429+0000","last_unstale":"2024-04-25T11:12:19.217632+0000","last_undegraded":"2024-04-25T11:12:19.217632+0000","last_fullsized":"2024-04-25T11:12:19.217632+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:50.949873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.619049+0000","last_change":"2024-04-25T11:12:16.635693+0000","last_active":"2024-04-25T11:12:18.619049+0000","last_peered":"2024-04-25T11:12:18.619049+0000","last_clean":"2024-04-25T11:12:18.619049+0000","last_became_active":"2024-04-25T11:12:16.635386+0000","last_became_peered":"2024-04-25T11:12:16.635386+0000","last_unstale":"2024-04-25T11:12:18.619049+0000","last_undegraded":"2024-04-25T11:12:18.619049+0000","last_fullsized":"2024-04-25T11:12:18.619049+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:28.137337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":146,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618109+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:12:18.618109+0000","last_peered":"2024-04-25T11:12:18.618109+0000","last_clean":"2024-04-25T11:12:18.618109+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:12:18.618109+0000","last_undegraded":"2024-04-25T11:12:18.618109+0000","last_fullsized":"2024-04-25T11:12:18.618109+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476764,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":786324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476765,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":194856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476764,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":194386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476765,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":194878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":189228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":454656,"data_stored":194364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":784676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":187652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:12:22.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:12:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.708+0000 7f1e21c00640 1 Processor -- start 2024-04-25T11:12:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.708+0000 7f1e21c00640 1 -- start start 2024-04-25T11:12:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.710+0000 7f1e21c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c0b3d30 0x7f1e1c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.710+0000 7f1e21c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e1c0cf290 con 0x7f1e1c0b7ae0 2024-04-25T11:12:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.710+0000 7f1e21c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e1c0cf570 con 0x7f1e1c0b3920 2024-04-25T11:12:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.710+0000 7f1e21c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1e1c0cf850 con 0x7f1e1c0b3d30 2024-04-25T11:12:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.710+0000 7f1e1b400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1e1c0b7ae0 0x7f1e1c0ccda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53042/0 (socket says 172.21.15.7:53042) 2024-04-25T11:12:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.710+0000 7f1e1b400640 1 -- 172.21.15.7:0/307635051 learned_addr learned my addr 172.21.15.7:0/307635051 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.711+0000 7f1e21200640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c0b3d30 0x7f1e1c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.711+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3302460235 0 0) 0x7f1e1c0cf290 con 0x7f1e1c0b7ae0 2024-04-25T11:12:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.711+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e00003520 con 0x7f1e1c0b7ae0 2024-04-25T11:12:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.711+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1211643536 0 0) 0x7f1e00003520 con 0x7f1e1c0b7ae0 2024-04-25T11:12:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.711+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c0b3d30 msgr2=0x7f1e1c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.711+0000 7f1e1aa00640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c0b3d30 0x7f1e1c0cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.712+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 >> v1:172.21.15.87:6789/0 conn(0x7f1e1c0b3920 legacy=0x7f1e1c0b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.712+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1c0cfb30 con 0x7f1e1c0b7ae0 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.712+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1e04001670 con 0x7f1e1c0b7ae0 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.712+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 >> v1:172.21.15.7:6789/0 conn(0x7f1e1c0b7ae0 legacy=0x7f1e1c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.712+0000 7f1e1aa00640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c0b3d30 0x7f1e000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.712+0000 7f1e1aa00640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e00004010 0x7f1e00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.712+0000 7f1e1aa00640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00004d40 0x7f1e00005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.712+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c0cf850 con 0x7f1e00004d40 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.712+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0000bcf0 con 0x7f1e00004010 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.712+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0000bef0 con 0x7f1e1c0b3d30 2024-04-25T11:12:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.712+0000 7f1e21200640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00004d40 0x7f1e00005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.713+0000 7f1e1b400640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e00004010 0x7f1e00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.713+0000 7f1e21200640 1 -- 172.21.15.7:0/307635051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c0b3d30 msgr2=0x7f1e000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.713+0000 7f1e21200640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c0b3d30 0x7f1e000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.713+0000 7f1e21200640 1 -- 172.21.15.7:0/307635051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e00004010 msgr2=0x7f1e00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.713+0000 7f1e21200640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e00004010 0x7f1e00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.713+0000 7f1e21200640 1 -- 172.21.15.7:0/307635051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e00003520 con 0x7f1e00004d40 2024-04-25T11:12:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.713+0000 7f1e1b400640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e00004010 0x7f1e00004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.714+0000 7f1e21200640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00004d40 0x7f1e00005210 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0c96d0 tx=0x7f1e00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35448a29f8842814 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.714+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14033070 con 0x7f1e00004d40 2024-04-25T11:12:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.714+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1402f650 con 0x7f1e00004d40 2024-04-25T11:12:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.714+0000 7f1e1aa00640 1 -- 172.21.15.7:0/307635051 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14009be0 con 0x7f1e00004d40 2024-04-25T11:12:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.715+0000 7f1e21c00640 1 -- 172.21.15.7:0/307635051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00004d40 msgr2=0x7f1e00005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.715+0000 7f1e21c00640 1 --2- 172.21.15.7:0/307635051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e00004d40 0x7f1e00005210 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0c96d0 tx=0x7f1e00005830 comp rx=0 tx=0).stop 2024-04-25T11:12:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.715+0000 7f1e21200640 1 -- 172.21.15.7:0/307635051 reap_dead start 2024-04-25T11:12:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.715+0000 7f1e21c00640 1 -- 172.21.15.7:0/307635051 shutdown_connections 2024-04-25T11:12:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.715+0000 7f1e21c00640 1 -- 172.21.15.7:0/307635051 >> 172.21.15.7:0/307635051 conn(0x7f1e1c09b540 msgr2=0x7f1e1c09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.715+0000 7f1e21c00640 1 -- 172.21.15.7:0/307635051 shutdown_connections 2024-04-25T11:12:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.715+0000 7f1e21c00640 1 -- 172.21.15.7:0/307635051 wait complete. 2024-04-25T11:12:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.718+0000 7f1e21c00640 1 Processor -- start 2024-04-25T11:12:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.718+0000 7f1e21c00640 1 -- start start 2024-04-25T11:12:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.719+0000 7f1e21c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c152c80 0x7f1e1c09fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.719+0000 7f1e21c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e1c071570 0x7f1e1c071980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.719+0000 7f1e21c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c1613a0 0x7f1e1c1617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.719+0000 7f1e21c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c0cd630 con 0x7f1e1c071570 2024-04-25T11:12:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.719+0000 7f1e21c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c0cd4b0 con 0x7f1e1c152c80 2024-04-25T11:12:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.719+0000 7f1e21c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1e1c0cd7b0 con 0x7f1e1c1613a0 2024-04-25T11:12:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.720+0000 7f1e1b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e1c071570 0x7f1e1c071980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.720+0000 7f1e1b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e1c071570 0x7f1e1c071980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54058/0 (socket says 172.21.15.7:54058) 2024-04-25T11:12:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.720+0000 7f1e1b400640 1 -- 172.21.15.7:0/2707790533 learned_addr learned my addr 172.21.15.7:0/2707790533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.720+0000 7f1e1be00640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c152c80 0x7f1e1c09fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.720+0000 7f1e21200640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c1613a0 0x7f1e1c1617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.720+0000 7f1e1b400640 1 -- 172.21.15.7:0/2707790533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c1613a0 msgr2=0x7f1e1c1617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.720+0000 7f1e1b400640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c1613a0 0x7f1e1c1617b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.720+0000 7f1e1b400640 1 -- 172.21.15.7:0/2707790533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c152c80 msgr2=0x7f1e1c09fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.720+0000 7f1e1b400640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c152c80 0x7f1e1c09fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.720+0000 7f1e1b400640 1 -- 172.21.15.7:0/2707790533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1c071ec0 con 0x7f1e1c071570 2024-04-25T11:12:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.720+0000 7f1e1b400640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e1c071570 0x7f1e1c071980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400d8c0 tx=0x7f1e0400dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.721+0000 7f1e13e00640 1 -- 172.21.15.7:0/2707790533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04019070 con 0x7f1e1c071570 2024-04-25T11:12:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.721+0000 7f1e13e00640 1 -- 172.21.15.7:0/2707790533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e040074e0 con 0x7f1e1c071570 2024-04-25T11:12:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.721+0000 7f1e13e00640 1 -- 172.21.15.7:0/2707790533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e040049e0 con 0x7f1e1c071570 2024-04-25T11:12:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.721+0000 7f1e21c00640 1 -- 172.21.15.7:0/2707790533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1c1628b0 con 0x7f1e1c071570 2024-04-25T11:12:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.721+0000 7f1e21c00640 1 -- 172.21.15.7:0/2707790533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e1c162e70 con 0x7f1e1c071570 2024-04-25T11:12:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.722+0000 7f1e21c00640 1 -- 172.21.15.7:0/2707790533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e1c050740 con 0x7f1e1c071570 2024-04-25T11:12:22.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.723+0000 7f1e13e00640 1 -- 172.21.15.7:0/2707790533 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1e04004b80 con 0x7f1e1c071570 2024-04-25T11:12:22.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.724+0000 7f1e13e00640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1df80744b0 0x7f1df80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:22.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.725+0000 7f1e13e00640 1 -- 172.21.15.7:0/2707790533 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f1e04094de0 con 0x7f1e1c071570 2024-04-25T11:12:22.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.725+0000 7f1e1be00640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1df80744b0 0x7f1df80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:22.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.725+0000 7f1e1be00640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1df80744b0 0x7f1df80769c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c1f9070 tx=0x7f1e0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:22.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.729+0000 7f1e13e00640 1 -- 172.21.15.7:0/2707790533 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1e1c071ec0 con 0x7f1e1c071570 2024-04-25T11:12:22.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.928+0000 7f1e21c00640 1 -- 172.21.15.7:0/2707790533 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e1c0ce330 con 0x7f1df80744b0 2024-04-25T11:12:22.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.949+0000 7f1e13e00640 1 -- 172.21.15.7:0/2707790533 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7f1e1c0ce330 con 0x7f1df80744b0 2024-04-25T11:12:22.952 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:22.954 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:12:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.955+0000 7f1e21c00640 1 -- 172.21.15.7:0/2707790533 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1df80744b0 msgr2=0x7f1df80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.955+0000 7f1e21c00640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1df80744b0 0x7f1df80769c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c1f9070 tx=0x7f1e0c009040 comp rx=0 tx=0).stop 2024-04-25T11:12:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.956+0000 7f1e21c00640 1 -- 172.21.15.7:0/2707790533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e1c071570 msgr2=0x7f1e1c071980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.956+0000 7f1e21c00640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e1c071570 0x7f1e1c071980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400d8c0 tx=0x7f1e0400dd80 comp rx=0 tx=0).stop 2024-04-25T11:12:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.956+0000 7f1e21c00640 1 -- 172.21.15.7:0/2707790533 shutdown_connections 2024-04-25T11:12:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.956+0000 7f1e21c00640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1df80744b0 0x7f1df80769c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.956+0000 7f1e21c00640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e1c1613a0 0x7f1e1c1617b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.956+0000 7f1e21c00640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e1c071570 0x7f1e1c071980 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.956+0000 7f1e21c00640 1 --2- 172.21.15.7:0/2707790533 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e1c152c80 0x7f1e1c09fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.956+0000 7f1e21c00640 1 -- 172.21.15.7:0/2707790533 >> 172.21.15.7:0/2707790533 conn(0x7f1e1c09b540 msgr2=0x7f1e1c09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.956+0000 7f1e21c00640 1 -- 172.21.15.7:0/2707790533 shutdown_connections 2024-04-25T11:12:22.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:22.956+0000 7f1e21c00640 1 -- 172.21.15.7:0/2707790533 wait complete. 2024-04-25T11:12:22.983 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-25T11:12:21.827129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233392,"kb_used_data":4720,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741328,"statfs":{"total":773094113280,"available":772855119872,"internally_reserved":0,"allocated":4833280,"data_stored":2726364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002024"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.390284+0000","last_change":"2024-04-25T11:12:17.641255+0000","last_active":"2024-04-25T11:12:19.390284+0000","last_peered":"2024-04-25T11:12:19.390284+0000","last_clean":"2024-04-25T11:12:19.390284+0000","last_became_active":"2024-04-25T11:12:17.640883+0000","last_became_peered":"2024-04-25T11:12:17.640883+0000","last_unstale":"2024-04-25T11:12:19.390284+0000","last_undegraded":"2024-04-25T11:12:19.390284+0000","last_fullsized":"2024-04-25T11:12:19.390284+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:50.949873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389432+0000","last_change":"2024-04-25T11:12:17.634849+0000","last_active":"2024-04-25T11:12:19.389432+0000","last_peered":"2024-04-25T11:12:19.389432+0000","last_clean":"2024-04-25T11:12:19.389432+0000","last_became_active":"2024-04-25T11:12:17.634414+0000","last_became_peered":"2024-04-25T11:12:17.634414+0000","last_unstale":"2024-04-25T11:12:19.389432+0000","last_undegraded":"2024-04-25T11:12:19.389432+0000","last_fullsized":"2024-04-25T11:12:19.389432+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:35.196880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029403+0000","last_change":"2024-04-25T11:12:17.647463+0000","last_active":"2024-04-25T11:12:19.029403+0000","last_peered":"2024-04-25T11:12:19.029403+0000","last_clean":"2024-04-25T11:12:19.029403+0000","last_became_active":"2024-04-25T11:12:17.647178+0000","last_became_peered":"2024-04-25T11:12:17.647178+0000","last_unstale":"2024-04-25T11:12:19.029403+0000","last_undegraded":"2024-04-25T11:12:19.029403+0000","last_fullsized":"2024-04-25T11:12:19.029403+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.157140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.633010+0000","last_change":"2024-04-25T11:12:17.624701+0000","last_active":"2024-04-25T11:12:17.633010+0000","last_peered":"2024-04-25T11:12:17.633010+0000","last_clean":"2024-04-25T11:12:17.633010+0000","last_became_active":"2024-04-25T11:12:17.623517+0000","last_became_peered":"2024-04-25T11:12:17.623517+0000","last_unstale":"2024-04-25T11:12:17.633010+0000","last_undegraded":"2024-04-25T11:12:17.633010+0000","last_fullsized":"2024-04-25T11:12:17.633010+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:02.203277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.630060+0000","last_change":"2024-04-25T11:12:17.619749+0000","last_active":"2024-04-25T11:12:17.630060+0000","last_peered":"2024-04-25T11:12:17.630060+0000","last_clean":"2024-04-25T11:12:17.630060+0000","last_became_active":"2024-04-25T11:12:17.618350+0000","last_became_peered":"2024-04-25T11:12:17.618350+0000","last_unstale":"2024-04-25T11:12:17.630060+0000","last_undegraded":"2024-04-25T11:12:17.630060+0000","last_fullsized":"2024-04-25T11:12:17.630060+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:47.435885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615063+0000","last_change":"2024-04-25T11:12:17.629076+0000","last_active":"2024-04-25T11:12:18.615063+0000","last_peered":"2024-04-25T11:12:18.615063+0000","last_clean":"2024-04-25T11:12:18.615063+0000","last_became_active":"2024-04-25T11:12:17.628760+0000","last_became_peered":"2024-04-25T11:12:17.628760+0000","last_unstale":"2024-04-25T11:12:18.615063+0000","last_undegraded":"2024-04-25T11:12:18.615063+0000","last_fullsized":"2024-04-25T11:12:18.615063+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:42.282697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216232+0000","last_change":"2024-04-25T11:12:17.639438+0000","last_active":"2024-04-25T11:12:19.216232+0000","last_peered":"2024-04-25T11:12:19.216232+0000","last_clean":"2024-04-25T11:12:19.216232+0000","last_became_active":"2024-04-25T11:12:17.634264+0000","last_became_peered":"2024-04-25T11:12:17.634264+0000","last_unstale":"2024-04-25T11:12:19.216232+0000","last_undegraded":"2024-04-25T11:12:19.216232+0000","last_fullsized":"2024-04-25T11:12:19.216232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:31.557454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387464+0000","last_change":"2024-04-25T11:12:17.644433+0000","last_active":"2024-04-25T11:12:19.387464+0000","last_peered":"2024-04-25T11:12:19.387464+0000","last_clean":"2024-04-25T11:12:19.387464+0000","last_became_active":"2024-04-25T11:12:17.644037+0000","last_became_peered":"2024-04-25T11:12:17.644037+0000","last_unstale":"2024-04-25T11:12:19.387464+0000","last_undegraded":"2024-04-25T11:12:19.387464+0000","last_fullsized":"2024-04-25T11:12:19.387464+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:13.310787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029728+0000","last_change":"2024-04-25T11:12:17.649413+0000","last_active":"2024-04-25T11:12:19.029728+0000","last_peered":"2024-04-25T11:12:19.029728+0000","last_clean":"2024-04-25T11:12:19.029728+0000","last_became_active":"2024-04-25T11:12:17.649039+0000","last_became_peered":"2024-04-25T11:12:17.649039+0000","last_unstale":"2024-04-25T11:12:19.029728+0000","last_undegraded":"2024-04-25T11:12:19.029728+0000","last_fullsized":"2024-04-25T11:12:19.029728+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:20.586779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217424+0000","last_change":"2024-04-25T11:12:17.631797+0000","last_active":"2024-04-25T11:12:19.217424+0000","last_peered":"2024-04-25T11:12:19.217424+0000","last_clean":"2024-04-25T11:12:19.217424+0000","last_became_active":"2024-04-25T11:12:17.631467+0000","last_became_peered":"2024-04-25T11:12:17.631467+0000","last_unstale":"2024-04-25T11:12:19.217424+0000","last_undegraded":"2024-04-25T11:12:19.217424+0000","last_fullsized":"2024-04-25T11:12:19.217424+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:42.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217310+0000","last_change":"2024-04-25T11:12:17.634537+0000","last_active":"2024-04-25T11:12:19.217310+0000","last_peered":"2024-04-25T11:12:19.217310+0000","last_clean":"2024-04-25T11:12:19.217310+0000","last_became_active":"2024-04-25T11:12:17.634127+0000","last_became_peered":"2024-04-25T11:12:17.634127+0000","last_unstale":"2024-04-25T11:12:19.217310+0000","last_undegraded":"2024-04-25T11:12:19.217310+0000","last_fullsized":"2024-04-25T11:12:19.217310+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:38.365436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029099+0000","last_change":"2024-04-25T11:12:17.634556+0000","last_active":"2024-04-25T11:12:19.029099+0000","last_peered":"2024-04-25T11:12:19.029099+0000","last_clean":"2024-04-25T11:12:19.029099+0000","last_became_active":"2024-04-25T11:12:17.634259+0000","last_became_peered":"2024-04-25T11:12:17.634259+0000","last_unstale":"2024-04-25T11:12:19.029099+0000","last_undegraded":"2024-04-25T11:12:19.029099+0000","last_fullsized":"2024-04-25T11:12:19.029099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:11.180085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216341+0000","last_change":"2024-04-25T11:12:17.627970+0000","last_active":"2024-04-25T11:12:19.216341+0000","last_peered":"2024-04-25T11:12:19.216341+0000","last_clean":"2024-04-25T11:12:19.216341+0000","last_became_active":"2024-04-25T11:12:17.627655+0000","last_became_peered":"2024-04-25T11:12:17.627655+0000","last_unstale":"2024-04-25T11:12:19.216341+0000","last_undegraded":"2024-04-25T11:12:19.216341+0000","last_fullsized":"2024-04-25T11:12:19.216341+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:13.310787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387126+0000","last_change":"2024-04-25T11:12:17.635455+0000","last_active":"2024-04-25T11:12:19.387126+0000","last_peered":"2024-04-25T11:12:19.387126+0000","last_clean":"2024-04-25T11:12:19.387126+0000","last_became_active":"2024-04-25T11:12:17.635133+0000","last_became_peered":"2024-04-25T11:12:17.635133+0000","last_unstale":"2024-04-25T11:12:19.387126+0000","last_undegraded":"2024-04-25T11:12:19.387126+0000","last_fullsized":"2024-04-25T11:12:19.387126+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:50.586215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216429+0000","last_change":"2024-04-25T11:12:17.634659+0000","last_active":"2024-04-25T11:12:19.216429+0000","last_peered":"2024-04-25T11:12:19.216429+0000","last_clean":"2024-04-25T11:12:19.216429+0000","last_became_active":"2024-04-25T11:12:17.632268+0000","last_became_peered":"2024-04-25T11:12:17.632268+0000","last_unstale":"2024-04-25T11:12:19.216429+0000","last_undegraded":"2024-04-25T11:12:19.216429+0000","last_fullsized":"2024-04-25T11:12:19.216429+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:31.231032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.617824+0000","last_change":"2024-04-25T11:12:17.643973+0000","last_active":"2024-04-25T11:12:18.617824+0000","last_peered":"2024-04-25T11:12:18.617824+0000","last_clean":"2024-04-25T11:12:18.617824+0000","last_became_active":"2024-04-25T11:12:17.643734+0000","last_became_peered":"2024-04-25T11:12:17.643734+0000","last_unstale":"2024-04-25T11:12:18.617824+0000","last_undegraded":"2024-04-25T11:12:18.617824+0000","last_fullsized":"2024-04-25T11:12:18.617824+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.157140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618809+0000","last_change":"2024-04-25T11:12:17.642118+0000","last_active":"2024-04-25T11:12:18.618809+0000","last_peered":"2024-04-25T11:12:18.618809+0000","last_clean":"2024-04-25T11:12:18.618809+0000","last_became_active":"2024-04-25T11:12:17.641802+0000","last_became_peered":"2024-04-25T11:12:17.641802+0000","last_unstale":"2024-04-25T11:12:18.618809+0000","last_undegraded":"2024-04-25T11:12:18.618809+0000","last_fullsized":"2024-04-25T11:12:18.618809+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:20.586779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621102+0000","last_change":"2024-04-25T11:12:17.626836+0000","last_active":"2024-04-25T11:12:18.621102+0000","last_peered":"2024-04-25T11:12:18.621102+0000","last_clean":"2024-04-25T11:12:18.621102+0000","last_became_active":"2024-04-25T11:12:17.625823+0000","last_became_peered":"2024-04-25T11:12:17.625823+0000","last_unstale":"2024-04-25T11:12:18.621102+0000","last_undegraded":"2024-04-25T11:12:18.621102+0000","last_fullsized":"2024-04-25T11:12:18.621102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:49.885461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.622328+0000","last_change":"2024-04-25T11:12:17.615647+0000","last_active":"2024-04-25T11:12:17.622328+0000","last_peered":"2024-04-25T11:12:17.622328+0000","last_clean":"2024-04-25T11:12:17.622328+0000","last_became_active":"2024-04-25T11:12:17.612194+0000","last_became_peered":"2024-04-25T11:12:17.612194+0000","last_unstale":"2024-04-25T11:12:17.622328+0000","last_undegraded":"2024-04-25T11:12:17.622328+0000","last_fullsized":"2024-04-25T11:12:17.622328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:35.393864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621181+0000","last_change":"2024-04-25T11:12:17.630309+0000","last_active":"2024-04-25T11:12:18.621181+0000","last_peered":"2024-04-25T11:12:18.621181+0000","last_clean":"2024-04-25T11:12:18.621181+0000","last_became_active":"2024-04-25T11:12:17.629536+0000","last_became_peered":"2024-04-25T11:12:17.629536+0000","last_unstale":"2024-04-25T11:12:18.621181+0000","last_undegraded":"2024-04-25T11:12:18.621181+0000","last_fullsized":"2024-04-25T11:12:18.621181+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:50.586215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216728+0000","last_change":"2024-04-25T11:12:17.645960+0000","last_active":"2024-04-25T11:12:19.216728+0000","last_peered":"2024-04-25T11:12:19.216728+0000","last_clean":"2024-04-25T11:12:19.216728+0000","last_became_active":"2024-04-25T11:12:17.634389+0000","last_became_peered":"2024-04-25T11:12:17.634389+0000","last_unstale":"2024-04-25T11:12:19.216728+0000","last_undegraded":"2024-04-25T11:12:19.216728+0000","last_fullsized":"2024-04-25T11:12:19.216728+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:57.080529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388378+0000","last_change":"2024-04-25T11:12:17.617332+0000","last_active":"2024-04-25T11:12:19.388378+0000","last_peered":"2024-04-25T11:12:19.388378+0000","last_clean":"2024-04-25T11:12:19.388378+0000","last_became_active":"2024-04-25T11:12:17.616974+0000","last_became_peered":"2024-04-25T11:12:17.616974+0000","last_unstale":"2024-04-25T11:12:19.388378+0000","last_undegraded":"2024-04-25T11:12:19.388378+0000","last_fullsized":"2024-04-25T11:12:19.388378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:42.809695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618916+0000","last_change":"2024-04-25T11:12:17.632581+0000","last_active":"2024-04-25T11:12:18.618916+0000","last_peered":"2024-04-25T11:12:18.618916+0000","last_clean":"2024-04-25T11:12:18.618916+0000","last_became_active":"2024-04-25T11:12:17.632267+0000","last_became_peered":"2024-04-25T11:12:17.632267+0000","last_unstale":"2024-04-25T11:12:18.618916+0000","last_undegraded":"2024-04-25T11:12:18.618916+0000","last_fullsized":"2024-04-25T11:12:18.618916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:20.675156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.633055+0000","last_change":"2024-04-25T11:12:17.624993+0000","last_active":"2024-04-25T11:12:17.633055+0000","last_peered":"2024-04-25T11:12:17.633055+0000","last_clean":"2024-04-25T11:12:17.633055+0000","last_became_active":"2024-04-25T11:12:17.624291+0000","last_became_peered":"2024-04-25T11:12:17.624291+0000","last_unstale":"2024-04-25T11:12:17.633055+0000","last_undegraded":"2024-04-25T11:12:17.633055+0000","last_fullsized":"2024-04-25T11:12:17.633055+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.079603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621740+0000","last_change":"2024-04-25T11:12:17.641534+0000","last_active":"2024-04-25T11:12:18.621740+0000","last_peered":"2024-04-25T11:12:18.621740+0000","last_clean":"2024-04-25T11:12:18.621740+0000","last_became_active":"2024-04-25T11:12:17.640142+0000","last_became_peered":"2024-04-25T11:12:17.640142+0000","last_unstale":"2024-04-25T11:12:18.621740+0000","last_undegraded":"2024-04-25T11:12:18.621740+0000","last_fullsized":"2024-04-25T11:12:18.621740+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:28.331253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615848+0000","last_change":"2024-04-25T11:12:17.635251+0000","last_active":"2024-04-25T11:12:18.615848+0000","last_peered":"2024-04-25T11:12:18.615848+0000","last_clean":"2024-04-25T11:12:18.615848+0000","last_became_active":"2024-04-25T11:12:17.634913+0000","last_became_peered":"2024-04-25T11:12:17.634913+0000","last_unstale":"2024-04-25T11:12:18.615848+0000","last_undegraded":"2024-04-25T11:12:18.615848+0000","last_fullsized":"2024-04-25T11:12:18.615848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:11.180085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621953+0000","last_change":"2024-04-25T11:12:17.631494+0000","last_active":"2024-04-25T11:12:18.621953+0000","last_peered":"2024-04-25T11:12:18.621953+0000","last_clean":"2024-04-25T11:12:18.621953+0000","last_became_active":"2024-04-25T11:12:17.631010+0000","last_became_peered":"2024-04-25T11:12:17.631010+0000","last_unstale":"2024-04-25T11:12:18.621953+0000","last_undegraded":"2024-04-25T11:12:18.621953+0000","last_fullsized":"2024-04-25T11:12:18.621953+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:37.354273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216432+0000","last_change":"2024-04-25T11:12:17.640460+0000","last_active":"2024-04-25T11:12:19.216432+0000","last_peered":"2024-04-25T11:12:19.216432+0000","last_clean":"2024-04-25T11:12:19.216432+0000","last_became_active":"2024-04-25T11:12:17.640112+0000","last_became_peered":"2024-04-25T11:12:17.640112+0000","last_unstale":"2024-04-25T11:12:19.216432+0000","last_undegraded":"2024-04-25T11:12:19.216432+0000","last_fullsized":"2024-04-25T11:12:19.216432+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:07.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621755+0000","last_change":"2024-04-25T11:12:17.633084+0000","last_active":"2024-04-25T11:12:18.621755+0000","last_peered":"2024-04-25T11:12:18.621755+0000","last_clean":"2024-04-25T11:12:18.621755+0000","last_became_active":"2024-04-25T11:12:17.632763+0000","last_became_peered":"2024-04-25T11:12:17.632763+0000","last_unstale":"2024-04-25T11:12:18.621755+0000","last_undegraded":"2024-04-25T11:12:18.621755+0000","last_fullsized":"2024-04-25T11:12:18.621755+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:16.975633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.620664+0000","last_change":"2024-04-25T11:12:17.633820+0000","last_active":"2024-04-25T11:12:18.620664+0000","last_peered":"2024-04-25T11:12:18.620664+0000","last_clean":"2024-04-25T11:12:18.620664+0000","last_became_active":"2024-04-25T11:12:17.629365+0000","last_became_peered":"2024-04-25T11:12:17.629365+0000","last_unstale":"2024-04-25T11:12:18.620664+0000","last_undegraded":"2024-04-25T11:12:18.620664+0000","last_fullsized":"2024-04-25T11:12:18.620664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:35.028447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.619407+0000","last_change":"2024-04-25T11:12:17.621908+0000","last_active":"2024-04-25T11:12:18.619407+0000","last_peered":"2024-04-25T11:12:18.619407+0000","last_clean":"2024-04-25T11:12:18.619407+0000","last_became_active":"2024-04-25T11:12:17.621566+0000","last_became_peered":"2024-04-25T11:12:17.621566+0000","last_unstale":"2024-04-25T11:12:18.619407+0000","last_undegraded":"2024-04-25T11:12:18.619407+0000","last_fullsized":"2024-04-25T11:12:18.619407+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:48.108640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615085+0000","last_change":"2024-04-25T11:12:17.620893+0000","last_active":"2024-04-25T11:12:18.615085+0000","last_peered":"2024-04-25T11:12:18.615085+0000","last_clean":"2024-04-25T11:12:18.615085+0000","last_became_active":"2024-04-25T11:12:17.620562+0000","last_became_peered":"2024-04-25T11:12:17.620562+0000","last_unstale":"2024-04-25T11:12:18.615085+0000","last_undegraded":"2024-04-25T11:12:18.615085+0000","last_fullsized":"2024-04-25T11:12:18.615085+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:42.809695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618885+0000","last_change":"2024-04-25T11:12:17.644751+0000","last_active":"2024-04-25T11:12:18.618885+0000","last_peered":"2024-04-25T11:12:18.618885+0000","last_clean":"2024-04-25T11:12:18.618885+0000","last_became_active":"2024-04-25T11:12:17.644448+0000","last_became_peered":"2024-04-25T11:12:17.644448+0000","last_unstale":"2024-04-25T11:12:18.618885+0000","last_undegraded":"2024-04-25T11:12:18.618885+0000","last_fullsized":"2024-04-25T11:12:18.618885+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:29.679865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.621562+0000","last_change":"2024-04-25T11:12:16.633833+0000","last_active":"2024-04-25T11:12:18.621562+0000","last_peered":"2024-04-25T11:12:18.621562+0000","last_clean":"2024-04-25T11:12:18.621562+0000","last_became_active":"2024-04-25T11:12:16.633462+0000","last_became_peered":"2024-04-25T11:12:16.633462+0000","last_unstale":"2024-04-25T11:12:18.621562+0000","last_undegraded":"2024-04-25T11:12:18.621562+0000","last_fullsized":"2024-04-25T11:12:18.621562+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.828983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.616203+0000","last_change":"2024-04-25T11:12:17.635395+0000","last_active":"2024-04-25T11:12:18.616203+0000","last_peered":"2024-04-25T11:12:18.616203+0000","last_clean":"2024-04-25T11:12:18.616203+0000","last_became_active":"2024-04-25T11:12:17.635054+0000","last_became_peered":"2024-04-25T11:12:17.635054+0000","last_unstale":"2024-04-25T11:12:18.616203+0000","last_undegraded":"2024-04-25T11:12:18.616203+0000","last_fullsized":"2024-04-25T11:12:18.616203+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:23.157140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029586+0000","last_change":"2024-04-25T11:12:16.626706+0000","last_active":"2024-04-25T11:12:19.029586+0000","last_peered":"2024-04-25T11:12:19.029586+0000","last_clean":"2024-04-25T11:12:19.029586+0000","last_became_active":"2024-04-25T11:12:16.623128+0000","last_became_peered":"2024-04-25T11:12:16.623128+0000","last_unstale":"2024-04-25T11:12:19.029586+0000","last_undegraded":"2024-04-25T11:12:19.029586+0000","last_fullsized":"2024-04-25T11:12:19.029586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:07.877880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388103+0000","last_change":"2024-04-25T11:12:17.631438+0000","last_active":"2024-04-25T11:12:19.388103+0000","last_peered":"2024-04-25T11:12:19.388103+0000","last_clean":"2024-04-25T11:12:19.388103+0000","last_became_active":"2024-04-25T11:12:17.630702+0000","last_became_peered":"2024-04-25T11:12:17.630702+0000","last_unstale":"2024-04-25T11:12:19.388103+0000","last_undegraded":"2024-04-25T11:12:19.388103+0000","last_fullsized":"2024-04-25T11:12:19.388103+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:42.282697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.616546+0000","last_change":"2024-04-25T11:12:16.633736+0000","last_active":"2024-04-25T11:12:18.616546+0000","last_peered":"2024-04-25T11:12:18.616546+0000","last_clean":"2024-04-25T11:12:18.616546+0000","last_became_active":"2024-04-25T11:12:16.633410+0000","last_became_peered":"2024-04-25T11:12:16.633410+0000","last_unstale":"2024-04-25T11:12:18.616546+0000","last_undegraded":"2024-04-25T11:12:18.616546+0000","last_fullsized":"2024-04-25T11:12:18.616546+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:03.462442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.390168+0000","last_change":"2024-04-25T11:12:17.638817+0000","last_active":"2024-04-25T11:12:19.390168+0000","last_peered":"2024-04-25T11:12:19.390168+0000","last_clean":"2024-04-25T11:12:19.390168+0000","last_became_active":"2024-04-25T11:12:17.638481+0000","last_became_peered":"2024-04-25T11:12:17.638481+0000","last_unstale":"2024-04-25T11:12:19.390168+0000","last_undegraded":"2024-04-25T11:12:19.390168+0000","last_fullsized":"2024-04-25T11:12:19.390168+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:27.521860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387630+0000","last_change":"2024-04-25T11:12:16.631138+0000","last_active":"2024-04-25T11:12:19.387630+0000","last_peered":"2024-04-25T11:12:19.387630+0000","last_clean":"2024-04-25T11:12:19.387630+0000","last_became_active":"2024-04-25T11:12:16.630006+0000","last_became_peered":"2024-04-25T11:12:16.630006+0000","last_unstale":"2024-04-25T11:12:19.387630+0000","last_undegraded":"2024-04-25T11:12:19.387630+0000","last_fullsized":"2024-04-25T11:12:19.387630+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:41.402376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.028397+0000","last_change":"2024-04-25T11:12:17.637173+0000","last_active":"2024-04-25T11:12:19.028397+0000","last_peered":"2024-04-25T11:12:19.028397+0000","last_clean":"2024-04-25T11:12:19.028397+0000","last_became_active":"2024-04-25T11:12:17.636870+0000","last_became_peered":"2024-04-25T11:12:17.636870+0000","last_unstale":"2024-04-25T11:12:19.028397+0000","last_undegraded":"2024-04-25T11:12:19.028397+0000","last_fullsized":"2024-04-25T11:12:19.028397+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:45.703997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216493+0000","last_change":"2024-04-25T11:12:16.629015+0000","last_active":"2024-04-25T11:12:19.216493+0000","last_peered":"2024-04-25T11:12:19.216493+0000","last_clean":"2024-04-25T11:12:19.216493+0000","last_became_active":"2024-04-25T11:12:16.628692+0000","last_became_peered":"2024-04-25T11:12:16.628692+0000","last_unstale":"2024-04-25T11:12:19.216493+0000","last_undegraded":"2024-04-25T11:12:19.216493+0000","last_fullsized":"2024-04-25T11:12:19.216493+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:48.867738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216768+0000","last_change":"2024-04-25T11:12:17.639756+0000","last_active":"2024-04-25T11:12:19.216768+0000","last_peered":"2024-04-25T11:12:19.216768+0000","last_clean":"2024-04-25T11:12:19.216768+0000","last_became_active":"2024-04-25T11:12:17.639081+0000","last_became_peered":"2024-04-25T11:12:17.639081+0000","last_unstale":"2024-04-25T11:12:19.216768+0000","last_undegraded":"2024-04-25T11:12:19.216768+0000","last_fullsized":"2024-04-25T11:12:19.216768+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:35.196880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.619998+0000","last_change":"2024-04-25T11:12:16.634253+0000","last_active":"2024-04-25T11:12:18.619998+0000","last_peered":"2024-04-25T11:12:18.619998+0000","last_clean":"2024-04-25T11:12:18.619998+0000","last_became_active":"2024-04-25T11:12:16.633894+0000","last_became_peered":"2024-04-25T11:12:16.633894+0000","last_unstale":"2024-04-25T11:12:18.619998+0000","last_undegraded":"2024-04-25T11:12:18.619998+0000","last_fullsized":"2024-04-25T11:12:18.619998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:03.462442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387702+0000","last_change":"2024-04-25T11:12:17.640260+0000","last_active":"2024-04-25T11:12:19.387702+0000","last_peered":"2024-04-25T11:12:19.387702+0000","last_clean":"2024-04-25T11:12:19.387702+0000","last_became_active":"2024-04-25T11:12:17.639569+0000","last_became_peered":"2024-04-25T11:12:17.639569+0000","last_unstale":"2024-04-25T11:12:19.387702+0000","last_undegraded":"2024-04-25T11:12:19.387702+0000","last_fullsized":"2024-04-25T11:12:19.387702+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:16.975633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615364+0000","last_change":"2024-04-25T11:12:16.626419+0000","last_active":"2024-04-25T11:12:18.615364+0000","last_peered":"2024-04-25T11:12:18.615364+0000","last_clean":"2024-04-25T11:12:18.615364+0000","last_became_active":"2024-04-25T11:12:16.626101+0000","last_became_peered":"2024-04-25T11:12:16.626101+0000","last_unstale":"2024-04-25T11:12:18.615364+0000","last_undegraded":"2024-04-25T11:12:18.615364+0000","last_fullsized":"2024-04-25T11:12:18.615364+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:07.877880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388844+0000","last_change":"2024-04-25T11:12:17.633901+0000","last_active":"2024-04-25T11:12:19.388844+0000","last_peered":"2024-04-25T11:12:19.388844+0000","last_clean":"2024-04-25T11:12:19.388844+0000","last_became_active":"2024-04-25T11:12:17.633561+0000","last_became_peered":"2024-04-25T11:12:17.633561+0000","last_unstale":"2024-04-25T11:12:19.388844+0000","last_undegraded":"2024-04-25T11:12:19.388844+0000","last_fullsized":"2024-04-25T11:12:19.388844+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:07.920796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.386681+0000","last_change":"2024-04-25T11:12:16.627875+0000","last_active":"2024-04-25T11:12:19.386681+0000","last_peered":"2024-04-25T11:12:19.386681+0000","last_clean":"2024-04-25T11:12:19.386681+0000","last_became_active":"2024-04-25T11:12:16.627521+0000","last_became_peered":"2024-04-25T11:12:16.627521+0000","last_unstale":"2024-04-25T11:12:19.386681+0000","last_undegraded":"2024-04-25T11:12:19.386681+0000","last_fullsized":"2024-04-25T11:12:19.386681+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:51.916261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.631722+0000","last_change":"2024-04-25T11:12:17.623540+0000","last_active":"2024-04-25T11:12:17.631722+0000","last_peered":"2024-04-25T11:12:17.631722+0000","last_clean":"2024-04-25T11:12:17.631722+0000","last_became_active":"2024-04-25T11:12:17.622946+0000","last_became_peered":"2024-04-25T11:12:17.622946+0000","last_unstale":"2024-04-25T11:12:17.631722+0000","last_undegraded":"2024-04-25T11:12:17.631722+0000","last_fullsized":"2024-04-25T11:12:17.631722+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:54.933443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.390476+0000","last_change":"2024-04-25T11:12:16.623790+0000","last_active":"2024-04-25T11:12:19.390476+0000","last_peered":"2024-04-25T11:12:19.390476+0000","last_clean":"2024-04-25T11:12:19.390476+0000","last_became_active":"2024-04-25T11:12:16.623452+0000","last_became_peered":"2024-04-25T11:12:16.623452+0000","last_unstale":"2024-04-25T11:12:19.390476+0000","last_undegraded":"2024-04-25T11:12:19.390476+0000","last_fullsized":"2024-04-25T11:12:19.390476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:41.791972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.620021+0000","last_change":"2024-04-25T11:12:17.634206+0000","last_active":"2024-04-25T11:12:18.620021+0000","last_peered":"2024-04-25T11:12:18.620021+0000","last_clean":"2024-04-25T11:12:18.620021+0000","last_became_active":"2024-04-25T11:12:17.633879+0000","last_became_peered":"2024-04-25T11:12:17.633879+0000","last_unstale":"2024-04-25T11:12:18.620021+0000","last_undegraded":"2024-04-25T11:12:18.620021+0000","last_fullsized":"2024-04-25T11:12:18.620021+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:31.557454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217588+0000","last_change":"2024-04-25T11:12:16.623724+0000","last_active":"2024-04-25T11:12:19.217588+0000","last_peered":"2024-04-25T11:12:19.217588+0000","last_clean":"2024-04-25T11:12:19.217588+0000","last_became_active":"2024-04-25T11:12:16.622672+0000","last_became_peered":"2024-04-25T11:12:16.622672+0000","last_unstale":"2024-04-25T11:12:19.217588+0000","last_undegraded":"2024-04-25T11:12:19.217588+0000","last_fullsized":"2024-04-25T11:12:19.217588+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:15.957910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029390+0000","last_change":"2024-04-25T11:12:17.636237+0000","last_active":"2024-04-25T11:12:19.029390+0000","last_peered":"2024-04-25T11:12:19.029390+0000","last_clean":"2024-04-25T11:12:19.029390+0000","last_became_active":"2024-04-25T11:12:17.628700+0000","last_became_peered":"2024-04-25T11:12:17.628700+0000","last_unstale":"2024-04-25T11:12:19.029390+0000","last_undegraded":"2024-04-25T11:12:19.029390+0000","last_fullsized":"2024-04-25T11:12:19.029390+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:29.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217521+0000","last_change":"2024-04-25T11:12:16.628284+0000","last_active":"2024-04-25T11:12:19.217521+0000","last_peered":"2024-04-25T11:12:19.217521+0000","last_clean":"2024-04-25T11:12:19.217521+0000","last_became_active":"2024-04-25T11:12:16.625899+0000","last_became_peered":"2024-04-25T11:12:16.625899+0000","last_unstale":"2024-04-25T11:12:19.217521+0000","last_undegraded":"2024-04-25T11:12:19.217521+0000","last_fullsized":"2024-04-25T11:12:19.217521+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:34.010724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388022+0000","last_change":"2024-04-25T11:12:17.620298+0000","last_active":"2024-04-25T11:12:19.388022+0000","last_peered":"2024-04-25T11:12:19.388022+0000","last_clean":"2024-04-25T11:12:19.388022+0000","last_became_active":"2024-04-25T11:12:17.620023+0000","last_became_peered":"2024-04-25T11:12:17.620023+0000","last_unstale":"2024-04-25T11:12:19.388022+0000","last_undegraded":"2024-04-25T11:12:19.388022+0000","last_fullsized":"2024-04-25T11:12:19.388022+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:45.703997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389193+0000","last_change":"2024-04-25T11:12:16.624743+0000","last_active":"2024-04-25T11:12:19.389193+0000","last_peered":"2024-04-25T11:12:19.389193+0000","last_clean":"2024-04-25T11:12:19.389193+0000","last_became_active":"2024-04-25T11:12:16.624340+0000","last_became_peered":"2024-04-25T11:12:16.624340+0000","last_unstale":"2024-04-25T11:12:19.389193+0000","last_undegraded":"2024-04-25T11:12:19.389193+0000","last_fullsized":"2024-04-25T11:12:19.389193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:41.264974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.028520+0000","last_change":"2024-04-25T11:12:17.649420+0000","last_active":"2024-04-25T11:12:19.028520+0000","last_peered":"2024-04-25T11:12:19.028520+0000","last_clean":"2024-04-25T11:12:19.028520+0000","last_became_active":"2024-04-25T11:12:17.649092+0000","last_became_peered":"2024-04-25T11:12:17.649092+0000","last_unstale":"2024-04-25T11:12:19.028520+0000","last_undegraded":"2024-04-25T11:12:19.028520+0000","last_fullsized":"2024-04-25T11:12:19.028520+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:42.282697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389269+0000","last_change":"2024-04-25T11:12:16.626640+0000","last_active":"2024-04-25T11:12:19.389269+0000","last_peered":"2024-04-25T11:12:19.389269+0000","last_clean":"2024-04-25T11:12:19.389269+0000","last_became_active":"2024-04-25T11:12:16.626323+0000","last_became_peered":"2024-04-25T11:12:16.626323+0000","last_unstale":"2024-04-25T11:12:19.389269+0000","last_undegraded":"2024-04-25T11:12:19.389269+0000","last_fullsized":"2024-04-25T11:12:19.389269+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:44.686274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388293+0000","last_change":"2024-04-25T11:12:17.639174+0000","last_active":"2024-04-25T11:12:19.388293+0000","last_peered":"2024-04-25T11:12:19.388293+0000","last_clean":"2024-04-25T11:12:19.388293+0000","last_became_active":"2024-04-25T11:12:17.638842+0000","last_became_peered":"2024-04-25T11:12:17.638842+0000","last_unstale":"2024-04-25T11:12:19.388293+0000","last_undegraded":"2024-04-25T11:12:19.388293+0000","last_fullsized":"2024-04-25T11:12:19.388293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:31.231032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615189+0000","last_change":"2024-04-25T11:12:16.626699+0000","last_active":"2024-04-25T11:12:18.615189+0000","last_peered":"2024-04-25T11:12:18.615189+0000","last_clean":"2024-04-25T11:12:18.615189+0000","last_became_active":"2024-04-25T11:12:16.625059+0000","last_became_peered":"2024-04-25T11:12:16.625059+0000","last_unstale":"2024-04-25T11:12:18.615189+0000","last_undegraded":"2024-04-25T11:12:18.615189+0000","last_fullsized":"2024-04-25T11:12:18.615189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:00.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.619817+0000","last_change":"2024-04-25T11:12:17.613099+0000","last_active":"2024-04-25T11:12:18.619817+0000","last_peered":"2024-04-25T11:12:18.619817+0000","last_clean":"2024-04-25T11:12:18.619817+0000","last_became_active":"2024-04-25T11:12:17.612698+0000","last_became_peered":"2024-04-25T11:12:17.612698+0000","last_unstale":"2024-04-25T11:12:18.619817+0000","last_undegraded":"2024-04-25T11:12:18.619817+0000","last_fullsized":"2024-04-25T11:12:18.619817+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:33.935499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618748+0000","last_change":"2024-04-25T11:12:16.625452+0000","last_active":"2024-04-25T11:12:18.618748+0000","last_peered":"2024-04-25T11:12:18.618748+0000","last_clean":"2024-04-25T11:12:18.618748+0000","last_became_active":"2024-04-25T11:12:16.625104+0000","last_became_peered":"2024-04-25T11:12:16.625104+0000","last_unstale":"2024-04-25T11:12:18.618748+0000","last_undegraded":"2024-04-25T11:12:18.618748+0000","last_fullsized":"2024-04-25T11:12:18.618748+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:05.439617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388846+0000","last_change":"2024-04-25T11:12:17.631758+0000","last_active":"2024-04-25T11:12:19.388846+0000","last_peered":"2024-04-25T11:12:19.388846+0000","last_clean":"2024-04-25T11:12:19.388846+0000","last_became_active":"2024-04-25T11:12:17.631272+0000","last_became_peered":"2024-04-25T11:12:17.631272+0000","last_unstale":"2024-04-25T11:12:19.388846+0000","last_undegraded":"2024-04-25T11:12:19.388846+0000","last_fullsized":"2024-04-25T11:12:19.388846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:03.197485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.617731+0000","last_change":"2024-04-25T11:12:16.631114+0000","last_active":"2024-04-25T11:12:18.617731+0000","last_peered":"2024-04-25T11:12:18.617731+0000","last_clean":"2024-04-25T11:12:18.617731+0000","last_became_active":"2024-04-25T11:12:16.630322+0000","last_became_peered":"2024-04-25T11:12:16.630322+0000","last_unstale":"2024-04-25T11:12:18.617731+0000","last_undegraded":"2024-04-25T11:12:18.617731+0000","last_fullsized":"2024-04-25T11:12:18.617731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:34.010724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389139+0000","last_change":"2024-04-25T11:12:17.640574+0000","last_active":"2024-04-25T11:12:19.389139+0000","last_peered":"2024-04-25T11:12:19.389139+0000","last_clean":"2024-04-25T11:12:19.389139+0000","last_became_active":"2024-04-25T11:12:17.640233+0000","last_became_peered":"2024-04-25T11:12:17.640233+0000","last_unstale":"2024-04-25T11:12:19.389139+0000","last_undegraded":"2024-04-25T11:12:19.389139+0000","last_fullsized":"2024-04-25T11:12:19.389139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:57.080529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217584+0000","last_change":"2024-04-25T11:12:16.634084+0000","last_active":"2024-04-25T11:12:19.217584+0000","last_peered":"2024-04-25T11:12:19.217584+0000","last_clean":"2024-04-25T11:12:19.217584+0000","last_became_active":"2024-04-25T11:12:16.633713+0000","last_became_peered":"2024-04-25T11:12:16.633713+0000","last_unstale":"2024-04-25T11:12:19.217584+0000","last_undegraded":"2024-04-25T11:12:19.217584+0000","last_fullsized":"2024-04-25T11:12:19.217584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.568492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217492+0000","last_change":"2024-04-25T11:12:17.639082+0000","last_active":"2024-04-25T11:12:19.217492+0000","last_peered":"2024-04-25T11:12:19.217492+0000","last_clean":"2024-04-25T11:12:19.217492+0000","last_became_active":"2024-04-25T11:12:17.638766+0000","last_became_peered":"2024-04-25T11:12:17.638766+0000","last_unstale":"2024-04-25T11:12:19.217492+0000","last_undegraded":"2024-04-25T11:12:19.217492+0000","last_fullsized":"2024-04-25T11:12:19.217492+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:27.521860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388197+0000","last_change":"2024-04-25T11:12:16.625999+0000","last_active":"2024-04-25T11:12:19.388197+0000","last_peered":"2024-04-25T11:12:19.388197+0000","last_clean":"2024-04-25T11:12:19.388197+0000","last_became_active":"2024-04-25T11:12:16.625512+0000","last_became_peered":"2024-04-25T11:12:16.625512+0000","last_unstale":"2024-04-25T11:12:19.388197+0000","last_undegraded":"2024-04-25T11:12:19.388197+0000","last_fullsized":"2024-04-25T11:12:19.388197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:50.720849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387986+0000","last_change":"2024-04-25T11:12:17.631746+0000","last_active":"2024-04-25T11:12:19.387986+0000","last_peered":"2024-04-25T11:12:19.387986+0000","last_clean":"2024-04-25T11:12:19.387986+0000","last_became_active":"2024-04-25T11:12:17.631127+0000","last_became_peered":"2024-04-25T11:12:17.631127+0000","last_unstale":"2024-04-25T11:12:19.387986+0000","last_undegraded":"2024-04-25T11:12:19.387986+0000","last_fullsized":"2024-04-25T11:12:19.387986+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:37.354273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.628092+0000","last_change":"2024-04-25T11:12:16.622650+0000","last_active":"2024-04-25T11:12:17.628092+0000","last_peered":"2024-04-25T11:12:17.628092+0000","last_clean":"2024-04-25T11:12:17.628092+0000","last_became_active":"2024-04-25T11:12:16.622326+0000","last_became_peered":"2024-04-25T11:12:16.622326+0000","last_unstale":"2024-04-25T11:12:17.628092+0000","last_undegraded":"2024-04-25T11:12:17.628092+0000","last_fullsized":"2024-04-25T11:12:17.628092+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:29.897108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.634283+0000","last_change":"2024-04-25T11:12:17.627698+0000","last_active":"2024-04-25T11:12:17.634283+0000","last_peered":"2024-04-25T11:12:17.634283+0000","last_clean":"2024-04-25T11:12:17.634283+0000","last_became_active":"2024-04-25T11:12:17.626871+0000","last_became_peered":"2024-04-25T11:12:17.626871+0000","last_unstale":"2024-04-25T11:12:17.634283+0000","last_undegraded":"2024-04-25T11:12:17.634283+0000","last_fullsized":"2024-04-25T11:12:17.634283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:08.842985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615540+0000","last_change":"2024-04-25T11:12:16.631034+0000","last_active":"2024-04-25T11:12:18.615540+0000","last_peered":"2024-04-25T11:12:18.615540+0000","last_clean":"2024-04-25T11:12:18.615540+0000","last_became_active":"2024-04-25T11:12:16.630509+0000","last_became_peered":"2024-04-25T11:12:16.630509+0000","last_unstale":"2024-04-25T11:12:18.615540+0000","last_undegraded":"2024-04-25T11:12:18.615540+0000","last_fullsized":"2024-04-25T11:12:18.615540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:28.137337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618299+0000","last_change":"2024-04-25T11:12:17.623664+0000","last_active":"2024-04-25T11:12:18.618299+0000","last_peered":"2024-04-25T11:12:18.618299+0000","last_clean":"2024-04-25T11:12:18.618299+0000","last_became_active":"2024-04-25T11:12:17.622056+0000","last_became_peered":"2024-04-25T11:12:17.622056+0000","last_unstale":"2024-04-25T11:12:18.618299+0000","last_undegraded":"2024-04-25T11:12:18.618299+0000","last_fullsized":"2024-04-25T11:12:18.618299+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:29.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.632472+0000","last_change":"2024-04-25T11:12:16.631385+0000","last_active":"2024-04-25T11:12:17.632472+0000","last_peered":"2024-04-25T11:12:17.632472+0000","last_clean":"2024-04-25T11:12:17.632472+0000","last_became_active":"2024-04-25T11:12:16.631053+0000","last_became_peered":"2024-04-25T11:12:16.631053+0000","last_unstale":"2024-04-25T11:12:17.632472+0000","last_undegraded":"2024-04-25T11:12:17.632472+0000","last_fullsized":"2024-04-25T11:12:17.632472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:26.584206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.028799+0000","last_change":"2024-04-25T11:12:17.630867+0000","last_active":"2024-04-25T11:12:19.028799+0000","last_peered":"2024-04-25T11:12:19.028799+0000","last_clean":"2024-04-25T11:12:19.028799+0000","last_became_active":"2024-04-25T11:12:17.630555+0000","last_became_peered":"2024-04-25T11:12:17.630555+0000","last_unstale":"2024-04-25T11:12:19.028799+0000","last_undegraded":"2024-04-25T11:12:19.028799+0000","last_fullsized":"2024-04-25T11:12:19.028799+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:25.139484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.597715+0000","last_change":"2024-04-25T11:12:16.626672+0000","last_active":"2024-04-25T11:12:17.597715+0000","last_peered":"2024-04-25T11:12:17.597715+0000","last_clean":"2024-04-25T11:12:17.597715+0000","last_became_active":"2024-04-25T11:12:16.625675+0000","last_became_peered":"2024-04-25T11:12:16.625675+0000","last_unstale":"2024-04-25T11:12:17.597715+0000","last_undegraded":"2024-04-25T11:12:17.597715+0000","last_fullsized":"2024-04-25T11:12:17.597715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:29.065743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.216721+0000","last_change":"2024-04-25T11:12:17.646203+0000","last_active":"2024-04-25T11:12:19.216721+0000","last_peered":"2024-04-25T11:12:19.216721+0000","last_clean":"2024-04-25T11:12:19.216721+0000","last_became_active":"2024-04-25T11:12:17.640854+0000","last_became_peered":"2024-04-25T11:12:17.640854+0000","last_unstale":"2024-04-25T11:12:19.216721+0000","last_undegraded":"2024-04-25T11:12:19.216721+0000","last_fullsized":"2024-04-25T11:12:19.216721+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:03.197485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.619491+0000","last_change":"2024-04-25T11:12:16.624621+0000","last_active":"2024-04-25T11:12:18.619491+0000","last_peered":"2024-04-25T11:12:18.619491+0000","last_clean":"2024-04-25T11:12:18.619491+0000","last_became_active":"2024-04-25T11:12:16.624205+0000","last_became_peered":"2024-04-25T11:12:16.624205+0000","last_unstale":"2024-04-25T11:12:18.619491+0000","last_undegraded":"2024-04-25T11:12:18.619491+0000","last_fullsized":"2024-04-25T11:12:18.619491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:40.712090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389563+0000","last_change":"2024-04-25T11:12:17.632030+0000","last_active":"2024-04-25T11:12:19.389563+0000","last_peered":"2024-04-25T11:12:19.389563+0000","last_clean":"2024-04-25T11:12:19.389563+0000","last_became_active":"2024-04-25T11:12:17.631501+0000","last_became_peered":"2024-04-25T11:12:17.631501+0000","last_unstale":"2024-04-25T11:12:19.389563+0000","last_undegraded":"2024-04-25T11:12:19.389563+0000","last_fullsized":"2024-04-25T11:12:19.389563+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:38.365436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.598273+0000","last_change":"2024-04-25T11:12:16.631342+0000","last_active":"2024-04-25T11:12:17.598273+0000","last_peered":"2024-04-25T11:12:17.598273+0000","last_clean":"2024-04-25T11:12:17.598273+0000","last_became_active":"2024-04-25T11:12:16.631017+0000","last_became_peered":"2024-04-25T11:12:16.631017+0000","last_unstale":"2024-04-25T11:12:17.598273+0000","last_undegraded":"2024-04-25T11:12:17.598273+0000","last_fullsized":"2024-04-25T11:12:17.598273+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:58.555440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.615742+0000","last_change":"2024-04-25T11:12:17.647803+0000","last_active":"2024-04-25T11:12:18.615742+0000","last_peered":"2024-04-25T11:12:18.615742+0000","last_clean":"2024-04-25T11:12:18.615742+0000","last_became_active":"2024-04-25T11:12:17.647327+0000","last_became_peered":"2024-04-25T11:12:17.647327+0000","last_unstale":"2024-04-25T11:12:18.615742+0000","last_undegraded":"2024-04-25T11:12:18.615742+0000","last_fullsized":"2024-04-25T11:12:18.615742+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:25.139484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217924+0000","last_change":"2024-04-25T11:12:16.624934+0000","last_active":"2024-04-25T11:12:19.217924+0000","last_peered":"2024-04-25T11:12:19.217924+0000","last_clean":"2024-04-25T11:12:19.217924+0000","last_became_active":"2024-04-25T11:12:16.623319+0000","last_became_peered":"2024-04-25T11:12:16.623319+0000","last_unstale":"2024-04-25T11:12:19.217924+0000","last_undegraded":"2024-04-25T11:12:19.217924+0000","last_fullsized":"2024-04-25T11:12:19.217924+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:27.313530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.620519+0000","last_change":"2024-04-25T11:12:17.629748+0000","last_active":"2024-04-25T11:12:18.620519+0000","last_peered":"2024-04-25T11:12:18.620519+0000","last_clean":"2024-04-25T11:12:18.620519+0000","last_became_active":"2024-04-25T11:12:17.629428+0000","last_became_peered":"2024-04-25T11:12:17.629428+0000","last_unstale":"2024-04-25T11:12:18.620519+0000","last_undegraded":"2024-04-25T11:12:18.620519+0000","last_fullsized":"2024-04-25T11:12:18.620519+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:31.231032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T11:12:17.600794+0000","last_change":"2024-04-25T11:12:16.626094+0000","last_active":"2024-04-25T11:12:17.600794+0000","last_peered":"2024-04-25T11:12:17.600794+0000","last_clean":"2024-04-25T11:12:17.600794+0000","last_became_active":"2024-04-25T11:12:16.625415+0000","last_became_peered":"2024-04-25T11:12:16.625415+0000","last_unstale":"2024-04-25T11:12:17.600794+0000","last_undegraded":"2024-04-25T11:12:17.600794+0000","last_fullsized":"2024-04-25T11:12:17.600794+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:30.818798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.389609+0000","last_change":"2024-04-25T11:12:17.635245+0000","last_active":"2024-04-25T11:12:19.389609+0000","last_peered":"2024-04-25T11:12:19.389609+0000","last_clean":"2024-04-25T11:12:19.389609+0000","last_became_active":"2024-04-25T11:12:17.634831+0000","last_became_peered":"2024-04-25T11:12:17.634831+0000","last_unstale":"2024-04-25T11:12:19.389609+0000","last_undegraded":"2024-04-25T11:12:19.389609+0000","last_fullsized":"2024-04-25T11:12:19.389609+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:42.047113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029281+0000","last_change":"2024-04-25T11:12:16.625237+0000","last_active":"2024-04-25T11:12:19.029281+0000","last_peered":"2024-04-25T11:12:19.029281+0000","last_clean":"2024-04-25T11:12:19.029281+0000","last_became_active":"2024-04-25T11:12:16.624927+0000","last_became_peered":"2024-04-25T11:12:16.624927+0000","last_unstale":"2024-04-25T11:12:19.029281+0000","last_undegraded":"2024-04-25T11:12:19.029281+0000","last_fullsized":"2024-04-25T11:12:19.029281+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:00.519503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.028612+0000","last_change":"2024-04-25T11:12:17.642157+0000","last_active":"2024-04-25T11:12:19.028612+0000","last_peered":"2024-04-25T11:12:19.028612+0000","last_clean":"2024-04-25T11:12:19.028612+0000","last_became_active":"2024-04-25T11:12:17.641908+0000","last_became_peered":"2024-04-25T11:12:17.641908+0000","last_unstale":"2024-04-25T11:12:19.028612+0000","last_undegraded":"2024-04-25T11:12:19.028612+0000","last_fullsized":"2024-04-25T11:12:19.028612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:42.809695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.029263+0000","last_change":"2024-04-25T11:12:16.630633+0000","last_active":"2024-04-25T11:12:19.029263+0000","last_peered":"2024-04-25T11:12:19.029263+0000","last_clean":"2024-04-25T11:12:19.029263+0000","last_became_active":"2024-04-25T11:12:16.630089+0000","last_became_peered":"2024-04-25T11:12:16.630089+0000","last_unstale":"2024-04-25T11:12:19.029263+0000","last_undegraded":"2024-04-25T11:12:19.029263+0000","last_fullsized":"2024-04-25T11:12:19.029263+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:50.720849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.614928+0000","last_change":"2024-04-25T11:12:17.638972+0000","last_active":"2024-04-25T11:12:18.614928+0000","last_peered":"2024-04-25T11:12:18.614928+0000","last_clean":"2024-04-25T11:12:18.614928+0000","last_became_active":"2024-04-25T11:12:17.638630+0000","last_became_peered":"2024-04-25T11:12:17.638630+0000","last_unstale":"2024-04-25T11:12:18.614928+0000","last_undegraded":"2024-04-25T11:12:18.614928+0000","last_fullsized":"2024-04-25T11:12:18.614928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:45.703997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.620448+0000","last_change":"2024-04-25T11:12:16.623148+0000","last_active":"2024-04-25T11:12:18.620448+0000","last_peered":"2024-04-25T11:12:18.620448+0000","last_clean":"2024-04-25T11:12:18.620448+0000","last_became_active":"2024-04-25T11:12:16.622264+0000","last_became_peered":"2024-04-25T11:12:16.622264+0000","last_unstale":"2024-04-25T11:12:18.620448+0000","last_undegraded":"2024-04-25T11:12:18.620448+0000","last_fullsized":"2024-04-25T11:12:18.620448+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:41.402376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.387169+0000","last_change":"2024-04-25T11:12:17.632213+0000","last_active":"2024-04-25T11:12:19.387169+0000","last_peered":"2024-04-25T11:12:19.387169+0000","last_clean":"2024-04-25T11:12:19.387169+0000","last_became_active":"2024-04-25T11:12:17.631877+0000","last_became_peered":"2024-04-25T11:12:17.631877+0000","last_unstale":"2024-04-25T11:12:19.387169+0000","last_undegraded":"2024-04-25T11:12:19.387169+0000","last_fullsized":"2024-04-25T11:12:19.387169+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:28.331253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.386708+0000","last_change":"2024-04-25T11:12:16.626722+0000","last_active":"2024-04-25T11:12:19.386708+0000","last_peered":"2024-04-25T11:12:19.386708+0000","last_clean":"2024-04-25T11:12:19.386708+0000","last_became_active":"2024-04-25T11:12:16.622129+0000","last_became_peered":"2024-04-25T11:12:16.622129+0000","last_unstale":"2024-04-25T11:12:19.386708+0000","last_undegraded":"2024-04-25T11:12:19.386708+0000","last_fullsized":"2024-04-25T11:12:19.386708+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:05.439617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.620501+0000","last_change":"2024-04-25T11:12:17.639962+0000","last_active":"2024-04-25T11:12:18.620501+0000","last_peered":"2024-04-25T11:12:18.620501+0000","last_clean":"2024-04-25T11:12:18.620501+0000","last_became_active":"2024-04-25T11:12:17.639536+0000","last_became_peered":"2024-04-25T11:12:17.639536+0000","last_unstale":"2024-04-25T11:12:18.620501+0000","last_undegraded":"2024-04-25T11:12:18.620501+0000","last_fullsized":"2024-04-25T11:12:18.620501+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:13.310787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.388375+0000","last_change":"2024-04-25T11:12:16.623982+0000","last_active":"2024-04-25T11:12:19.388375+0000","last_peered":"2024-04-25T11:12:19.388375+0000","last_clean":"2024-04-25T11:12:19.388375+0000","last_became_active":"2024-04-25T11:12:16.623065+0000","last_became_peered":"2024-04-25T11:12:16.623065+0000","last_unstale":"2024-04-25T11:12:19.388375+0000","last_undegraded":"2024-04-25T11:12:19.388375+0000","last_fullsized":"2024-04-25T11:12:19.388375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:36.336550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:19.217632+0000","last_change":"2024-04-25T11:12:17.638776+0000","last_active":"2024-04-25T11:12:19.217632+0000","last_peered":"2024-04-25T11:12:19.217632+0000","last_clean":"2024-04-25T11:12:19.217632+0000","last_became_active":"2024-04-25T11:12:17.638429+0000","last_became_peered":"2024-04-25T11:12:17.638429+0000","last_unstale":"2024-04-25T11:12:19.217632+0000","last_undegraded":"2024-04-25T11:12:19.217632+0000","last_fullsized":"2024-04-25T11:12:19.217632+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:16.570622+0000","last_clean_scrub_stamp":"2024-04-25T11:12:16.570622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:50.949873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.619049+0000","last_change":"2024-04-25T11:12:16.635693+0000","last_active":"2024-04-25T11:12:18.619049+0000","last_peered":"2024-04-25T11:12:18.619049+0000","last_clean":"2024-04-25T11:12:18.619049+0000","last_became_active":"2024-04-25T11:12:16.635386+0000","last_became_peered":"2024-04-25T11:12:16.635386+0000","last_unstale":"2024-04-25T11:12:18.619049+0000","last_undegraded":"2024-04-25T11:12:18.619049+0000","last_fullsized":"2024-04-25T11:12:18.619049+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:12:15.552899+0000","last_clean_scrub_stamp":"2024-04-25T11:12:15.552899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:28.137337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":146,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T11:12:18.618109+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:12:18.618109+0000","last_peered":"2024-04-25T11:12:18.618109+0000","last_clean":"2024-04-25T11:12:18.618109+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:12:18.618109+0000","last_undegraded":"2024-04-25T11:12:18.618109+0000","last_fullsized":"2024-04-25T11:12:18.618109+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476764,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":786324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476765,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":194856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476764,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":194386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476765,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":194878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":189228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":454656,"data_stored":194364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":784676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":187652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:12:22.987 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:12:22.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:12:23.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.326+0000 7f9cbb000640 1 Processor -- start 2024-04-25T11:12:23.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.327+0000 7f9cbb000640 1 -- start start 2024-04-25T11:12:23.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.329+0000 7f9cbb000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9cb409f790 0x7f9cb415dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.329+0000 7f9cbb000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb4070d60 0x7f9cb406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.329+0000 7f9cbb000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9cb406dac0 0x7f9cb40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.329+0000 7f9cbb000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb415e2a0 con 0x7f9cb409f790 2024-04-25T11:12:23.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.329+0000 7f9cbb000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb4074550 con 0x7f9cb406dac0 2024-04-25T11:12:23.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.329+0000 7f9cbb000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb406f220 con 0x7f9cb4070d60 2024-04-25T11:12:23.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.330+0000 7f9cb3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9cb409f790 0x7f9cb415dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:23.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.330+0000 7f9cb3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9cb409f790 0x7f9cb415dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54066/0 (socket says 172.21.15.7:54066) 2024-04-25T11:12:23.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.330+0000 7f9cb3e00640 1 -- 172.21.15.7:0/487532831 learned_addr learned my addr 172.21.15.7:0/487532831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:23.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.330+0000 7f9cb9000640 1 --2- 172.21.15.7:0/487532831 >> v2:172.21.15.87:3300/0 conn(0x7f9cb406dac0 0x7f9cb40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:23.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.330+0000 7f9cabe00640 1 --2- 172.21.15.7:0/487532831 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb4070d60 0x7f9cb406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:23.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.330+0000 7f9cb3e00640 1 -- 172.21.15.7:0/487532831 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb4070d60 msgr2=0x7f9cb406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:23.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.330+0000 7f9cb3e00640 1 --2- 172.21.15.7:0/487532831 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb4070d60 0x7f9cb406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.330+0000 7f9cb3e00640 1 -- 172.21.15.7:0/487532831 >> v2:172.21.15.87:3300/0 conn(0x7f9cb406dac0 msgr2=0x7f9cb40ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:23.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.330+0000 7f9cb3e00640 1 --2- 172.21.15.7:0/487532831 >> v2:172.21.15.87:3300/0 conn(0x7f9cb406dac0 0x7f9cb40ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.330+0000 7f9cb3e00640 1 -- 172.21.15.7:0/487532831 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb406ded0 con 0x7f9cb409f790 2024-04-25T11:12:23.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.331+0000 7f9cb3e00640 1 --2- 172.21.15.7:0/487532831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb409f790 0x7f9cb415dd60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40b0cc0 tx=0x7f9ca402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623384c4bfd71ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:23.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.332+0000 7f9cb3400640 1 -- 172.21.15.7:0/487532831 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4037070 con 0x7f9cb409f790 2024-04-25T11:12:23.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.332+0000 7f9cb3400640 1 -- 172.21.15.7:0/487532831 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca4032440 con 0x7f9cb409f790 2024-04-25T11:12:23.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.332+0000 7f9cb3400640 1 -- 172.21.15.7:0/487532831 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4032810 con 0x7f9cb409f790 2024-04-25T11:12:23.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.333+0000 7f9cbb000640 1 -- 172.21.15.7:0/487532831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb409f790 msgr2=0x7f9cb415dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:23.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.333+0000 7f9cbb000640 1 --2- 172.21.15.7:0/487532831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb409f790 0x7f9cb415dd60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9cb40b0cc0 tx=0x7f9ca402a210 comp rx=0 tx=0).stop 2024-04-25T11:12:23.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.333+0000 7f9cbb000640 1 -- 172.21.15.7:0/487532831 shutdown_connections 2024-04-25T11:12:23.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.333+0000 7f9cbb000640 1 --2- 172.21.15.7:0/487532831 >> v2:172.21.15.87:3300/0 conn(0x7f9cb406dac0 0x7f9cb40ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.333+0000 7f9cbb000640 1 --2- 172.21.15.7:0/487532831 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb4070d60 0x7f9cb406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.333+0000 7f9cbb000640 1 --2- 172.21.15.7:0/487532831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb409f790 0x7f9cb415dd60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.333+0000 7f9cbb000640 1 -- 172.21.15.7:0/487532831 >> 172.21.15.7:0/487532831 conn(0x7f9cb40d1d40 msgr2=0x7f9cb4073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:23.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.333+0000 7f9cbb000640 1 -- 172.21.15.7:0/487532831 shutdown_connections 2024-04-25T11:12:23.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.333+0000 7f9cbb000640 1 -- 172.21.15.7:0/487532831 wait complete. 2024-04-25T11:12:23.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.336+0000 7f9cbb000640 1 Processor -- start 2024-04-25T11:12:23.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.336+0000 7f9cbb000640 1 -- start start 2024-04-25T11:12:23.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.337+0000 7f9cbb000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb406dac0 0x7f9cb4147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.337+0000 7f9cbb000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb4070d60 0x7f9cb40cff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.337+0000 7f9cbb000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb409f790 0x7f9cb40d0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.337+0000 7f9cbb000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb406fa30 con 0x7f9cb406dac0 2024-04-25T11:12:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.337+0000 7f9cbb000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb406f750 con 0x7f9cb4070d60 2024-04-25T11:12:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.337+0000 7f9cbb000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb406fd10 con 0x7f9cb409f790 2024-04-25T11:12:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.337+0000 7f9cb3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb406dac0 0x7f9cb4147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.337+0000 7f9cb3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb406dac0 0x7f9cb4147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54070/0 (socket says 172.21.15.7:54070) 2024-04-25T11:12:23.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.337+0000 7f9cb3e00640 1 -- 172.21.15.7:0/2202335752 learned_addr learned my addr 172.21.15.7:0/2202335752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:23.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.338+0000 7f9cb3e00640 1 -- 172.21.15.7:0/2202335752 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb409f790 msgr2=0x7f9cb40d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:23.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.338+0000 7f9cb3e00640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb409f790 0x7f9cb40d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.338+0000 7f9cb3e00640 1 -- 172.21.15.7:0/2202335752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb4070d60 msgr2=0x7f9cb40cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:23.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.338+0000 7f9cb3e00640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb4070d60 0x7f9cb40cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.338+0000 7f9cb3e00640 1 -- 172.21.15.7:0/2202335752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb41509d0 con 0x7f9cb406dac0 2024-04-25T11:12:23.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.338+0000 7f9cb3e00640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb406dac0 0x7f9cb4147230 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4000c00 tx=0x7f9ca40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:23.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.339+0000 7f9cb0c00640 1 -- 172.21.15.7:0/2202335752 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4041020 con 0x7f9cb406dac0 2024-04-25T11:12:23.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.339+0000 7f9cb0c00640 1 -- 172.21.15.7:0/2202335752 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca40306c0 con 0x7f9cb406dac0 2024-04-25T11:12:23.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.339+0000 7f9cb0c00640 1 -- 172.21.15.7:0/2202335752 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4037040 con 0x7f9cb406dac0 2024-04-25T11:12:23.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.339+0000 7f9cbb000640 1 -- 172.21.15.7:0/2202335752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb4150c60 con 0x7f9cb406dac0 2024-04-25T11:12:23.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.339+0000 7f9cbb000640 1 -- 172.21.15.7:0/2202335752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb4151220 con 0x7f9cb406dac0 2024-04-25T11:12:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.340+0000 7f9cbb000640 1 -- 172.21.15.7:0/2202335752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb4050740 con 0x7f9cb406dac0 2024-04-25T11:12:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.342+0000 7f9cb0c00640 1 -- 172.21.15.7:0/2202335752 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ca4044020 con 0x7f9cb406dac0 2024-04-25T11:12:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.343+0000 7f9cb0c00640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c90074500 0x7f9c90076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.343+0000 7f9cb0c00640 1 -- 172.21.15.7:0/2202335752 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f9ca40b5c10 con 0x7f9cb406dac0 2024-04-25T11:12:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.343+0000 7f9cabe00640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c90074500 0x7f9c90076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:23.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.344+0000 7f9cabe00640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c90074500 0x7f9c90076a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c006fd0 tx=0x7f9c9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:23.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.348+0000 7f9cb0c00640 1 -- 172.21.15.7:0/2202335752 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ca404b030 con 0x7f9cb406dac0 2024-04-25T11:12:23.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.634+0000 7f9cbb000640 1 -- 172.21.15.7:0/2202335752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9cb415b710 con 0x7f9cb406dac0 2024-04-25T11:12:23.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.642+0000 7f9cb0c00640 1 -- 172.21.15.7:0/2202335752 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f9cb406ded0 con 0x7f9cb406dac0 2024-04-25T11:12:23.642 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-25T11:12:23.642 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:12:23.642 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-04-25T11:12:23.642 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-04-25T11:12:23.642 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:12:23.642 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-04-25T11:12:23.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 -- 172.21.15.7:0/2202335752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c90074500 msgr2=0x7f9c90076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:23.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c90074500 0x7f9c90076a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c006fd0 tx=0x7f9c9c008040 comp rx=0 tx=0).stop 2024-04-25T11:12:23.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 -- 172.21.15.7:0/2202335752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb406dac0 msgr2=0x7f9cb4147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:23.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb406dac0 0x7f9cb4147230 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4000c00 tx=0x7f9ca40046c0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 -- 172.21.15.7:0/2202335752 shutdown_connections 2024-04-25T11:12:23.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c90074500 0x7f9c90076a10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb409f790 0x7f9cb40d0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb4070d60 0x7f9cb40cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 --2- 172.21.15.7:0/2202335752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb406dac0 0x7f9cb4147230 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 -- 172.21.15.7:0/2202335752 >> 172.21.15.7:0/2202335752 conn(0x7f9cb40d1d40 msgr2=0x7f9cb415b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:23.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 -- 172.21.15.7:0/2202335752 shutdown_connections 2024-04-25T11:12:23.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.645+0000 7f9cbb000640 1 -- 172.21.15.7:0/2202335752 wait complete. 2024-04-25T11:12:23.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:12:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.951+0000 7feabbe00640 1 Processor -- start 2024-04-25T11:12:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.951+0000 7feabbe00640 1 -- start start 2024-04-25T11:12:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.954+0000 7feabbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feabc06ffd0 0x7feabc164380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.954+0000 7feabbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feabc0b5f10 0x7feabc0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.954+0000 7feabbe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feabc0b47d0 con 0x7feabc0af390 2024-04-25T11:12:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.954+0000 7feabbe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7feabc0b6320 con 0x7feabc06ffd0 2024-04-25T11:12:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.954+0000 7feabbe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7feabc0b4a40 con 0x7feabc0b5f10 2024-04-25T11:12:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.954+0000 7feaba000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7feabc0af390 0x7feabc0b5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53054/0 (socket says 172.21.15.7:53054) 2024-04-25T11:12:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.954+0000 7feaba000640 1 -- 172.21.15.7:0/1461970700 learned_addr learned my addr 172.21.15.7:0/1461970700 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.954+0000 7feabaa00640 1 --2- 172.21.15.7:0/1461970700 >> v2:172.21.15.7:3300/0 conn(0x7feabc06ffd0 0x7feabc164380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.955+0000 7feabb400640 1 --2- 172.21.15.7:0/1461970700 >> v2:172.21.15.87:3300/0 conn(0x7feabc0b5f10 0x7feabc0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.955+0000 7feabaa00640 1 -- 172.21.15.7:0/1461970700 >> v2:172.21.15.87:3300/0 conn(0x7feabc0b5f10 msgr2=0x7feabc0af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.955+0000 7feabaa00640 1 --2- 172.21.15.7:0/1461970700 >> v2:172.21.15.87:3300/0 conn(0x7feabc0b5f10 0x7feabc0af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.955+0000 7feabaa00640 1 -- 172.21.15.7:0/1461970700 >> v1:172.21.15.7:6789/0 conn(0x7feabc0af390 legacy=0x7feabc0b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.955+0000 7feabaa00640 1 -- 172.21.15.7:0/1461970700 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc0b4d20 con 0x7feabc06ffd0 2024-04-25T11:12:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.955+0000 7feab9600640 1 -- 172.21.15.7:0/1461970700 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2926881977 0 0) 0x7feabc0b47d0 con 0x7feabc0af390 2024-04-25T11:12:23.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.956+0000 7feabaa00640 1 --2- 172.21.15.7:0/1461970700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feabc06ffd0 0x7feabc164380 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7feabc0712c0 tx=0x7feaac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9125c1544a0d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:23.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.956+0000 7feab9600640 1 -- 172.21.15.7:0/1461970700 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac037070 con 0x7feabc06ffd0 2024-04-25T11:12:23.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.956+0000 7feab9600640 1 -- 172.21.15.7:0/1461970700 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaac032440 con 0x7feabc06ffd0 2024-04-25T11:12:23.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.956+0000 7feab9600640 1 -- 172.21.15.7:0/1461970700 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac032810 con 0x7feabc06ffd0 2024-04-25T11:12:23.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.957+0000 7feabbe00640 1 -- 172.21.15.7:0/1461970700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feabc06ffd0 msgr2=0x7feabc164380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:23.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.957+0000 7feabbe00640 1 --2- 172.21.15.7:0/1461970700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feabc06ffd0 0x7feabc164380 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7feabc0712c0 tx=0x7feaac02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.957+0000 7feabbe00640 1 -- 172.21.15.7:0/1461970700 shutdown_connections 2024-04-25T11:12:23.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.957+0000 7feabbe00640 1 --2- 172.21.15.7:0/1461970700 >> v2:172.21.15.87:3300/0 conn(0x7feabc0b5f10 0x7feabc0af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.957+0000 7feabbe00640 1 --2- 172.21.15.7:0/1461970700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feabc06ffd0 0x7feabc164380 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.957+0000 7feabbe00640 1 -- 172.21.15.7:0/1461970700 >> 172.21.15.7:0/1461970700 conn(0x7feabc0b3820 msgr2=0x7feabc0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:23.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.957+0000 7feabbe00640 1 -- 172.21.15.7:0/1461970700 shutdown_connections 2024-04-25T11:12:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.957+0000 7feabbe00640 1 -- 172.21.15.7:0/1461970700 wait complete. 2024-04-25T11:12:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.960+0000 7feabbe00640 1 Processor -- start 2024-04-25T11:12:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.960+0000 7feabbe00640 1 -- start start 2024-04-25T11:12:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.961+0000 7feabbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feabc06ffd0 0x7feabc1519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.961+0000 7feabbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feabc0af390 0x7feabc15e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.961+0000 7feabbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feabc0b5f10 0x7feabc0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.961+0000 7feabbe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feabc06fa70 con 0x7feabc06ffd0 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.961+0000 7feabbe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feabc0b52e0 con 0x7feabc0af390 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.961+0000 7feabbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feabc06fbf0 con 0x7feabc0b5f10 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.962+0000 7feabaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feabc06ffd0 0x7feabc1519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.962+0000 7feabaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feabc06ffd0 0x7feabc1519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54098/0 (socket says 172.21.15.7:54098) 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.962+0000 7feabaa00640 1 -- 172.21.15.7:0/1077048359 learned_addr learned my addr 172.21.15.7:0/1077048359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.962+0000 7feabaa00640 1 -- 172.21.15.7:0/1077048359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feabc0b5f10 msgr2=0x7feabc0b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.962+0000 7feabaa00640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feabc0b5f10 0x7feabc0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.962+0000 7feabaa00640 1 -- 172.21.15.7:0/1077048359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feabc0af390 msgr2=0x7feabc15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.962+0000 7feabaa00640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feabc0af390 0x7feabc15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.962+0000 7feabaa00640 1 -- 172.21.15.7:0/1077048359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feabc151f20 con 0x7feabc06ffd0 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.962+0000 7feabaa00640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feabc06ffd0 0x7feabc1519e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feaac035040 tx=0x7feaac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.963+0000 7feab3400640 1 -- 172.21.15.7:0/1077048359 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac037050 con 0x7feabc06ffd0 2024-04-25T11:12:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.963+0000 7feab3400640 1 -- 172.21.15.7:0/1077048359 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaac038040 con 0x7feabc06ffd0 2024-04-25T11:12:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.963+0000 7feab3400640 1 -- 172.21.15.7:0/1077048359 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac004470 con 0x7feabc06ffd0 2024-04-25T11:12:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.963+0000 7feabbe00640 1 -- 172.21.15.7:0/1077048359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feabc15eff0 con 0x7feabc06ffd0 2024-04-25T11:12:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.963+0000 7feabbe00640 1 -- 172.21.15.7:0/1077048359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feabc15f5b0 con 0x7feabc06ffd0 2024-04-25T11:12:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.964+0000 7feabbe00640 1 -- 172.21.15.7:0/1077048359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feabc050740 con 0x7feabc06ffd0 2024-04-25T11:12:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.966+0000 7feab3400640 1 -- 172.21.15.7:0/1077048359 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feaac030900 con 0x7feabc06ffd0 2024-04-25T11:12:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.967+0000 7feab3400640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea980744b0 0x7fea980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.967+0000 7feab3400640 1 -- 172.21.15.7:0/1077048359 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7feaac0b5e80 con 0x7feabc06ffd0 2024-04-25T11:12:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.967+0000 7feaba000640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea980744b0 0x7fea980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.968+0000 7feaba000640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea980744b0 0x7fea980769c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feaa4007fd0 tx=0x7feaa4007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:23.972+0000 7feab3400640 1 -- 172.21.15.7:0/1077048359 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feabc151f20 con 0x7feabc06ffd0 2024-04-25T11:12:24.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.220+0000 7feabbe00640 1 -- 172.21.15.7:0/1077048359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feabc161d80 con 0x7feabc06ffd0 2024-04-25T11:12:24.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.222+0000 7feab3400640 1 -- 172.21.15.7:0/1077048359 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5172 (secure 0 0 0) 0x7feaac080470 con 0x7feabc06ffd0 2024-04-25T11:12:24.222 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:12:24.222 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2575517671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2575517671},{"type":"v1","addr":"172.21.15.87:6835","nonce":2575517671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/3746682343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3746682343},{"type":"v1","addr":"172.21.15.7:6836","nonce":3746682343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2132415318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2132415318},{"type":"v1","addr":"172.21.15.7:6837","nonce":2132415318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:12:17.156571+0000","modified":"2024-04-25T11:12:20.224854+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6833/389115631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":389115631},{"type":"v1","addr":"172.21.15.87:6833","nonce":389115631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5283,"qdb_cluster":[5283]},"id":2}]} 2024-04-25T11:12:24.222 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2024-04-25T11:12:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.224+0000 7feabbe00640 1 -- 172.21.15.7:0/1077048359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea980744b0 msgr2=0x7fea980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.224+0000 7feabbe00640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea980744b0 0x7fea980769c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feaa4007fd0 tx=0x7feaa4007960 comp rx=0 tx=0).stop 2024-04-25T11:12:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.224+0000 7feabbe00640 1 -- 172.21.15.7:0/1077048359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feabc06ffd0 msgr2=0x7feabc1519e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.224+0000 7feabbe00640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feabc06ffd0 0x7feabc1519e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7feaac035040 tx=0x7feaac0046c0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.225+0000 7feabbe00640 1 -- 172.21.15.7:0/1077048359 shutdown_connections 2024-04-25T11:12:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.225+0000 7feabbe00640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea980744b0 0x7fea980769c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.225+0000 7feabbe00640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feabc0b5f10 0x7feabc0b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.225+0000 7feabbe00640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feabc0af390 0x7feabc15e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.225+0000 7feabbe00640 1 --2- 172.21.15.7:0/1077048359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feabc06ffd0 0x7feabc1519e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.225+0000 7feabbe00640 1 -- 172.21.15.7:0/1077048359 >> 172.21.15.7:0/1077048359 conn(0x7feabc0b3820 msgr2=0x7feabc15d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.225+0000 7feabbe00640 1 -- 172.21.15.7:0/1077048359 shutdown_connections 2024-04-25T11:12:24.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.225+0000 7feabbe00640 1 -- 172.21.15.7:0/1077048359 wait complete. 2024-04-25T11:12:24.654 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:12:17.156571+0000', 'modified': '2024-04-25T11:12:20.224854+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5283}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5283': {'gid': 5283, 'name': 'd', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.87:6833/389115631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6832', 'nonce': 389115631}, {'type': 'v1', 'addr': '172.21.15.87:6833', 'nonce': 389115631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5283, 'qdb_cluster': [5283]} 2024-04-25T11:12:24.654 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:12:24.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T11:12:24.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.952+0000 7f3132400640 1 Processor -- start 2024-04-25T11:12:24.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.952+0000 7f3132400640 1 -- start start 2024-04-25T11:12:24.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.954+0000 7f3132400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f312c15eed0 0x7f312c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.954+0000 7f3132400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f312c1416a0 0x7f312c0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.954+0000 7f3132400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f312c06c570 con 0x7f312c141290 2024-04-25T11:12:24.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.954+0000 7f3132400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f312c06c850 con 0x7f312c15eed0 2024-04-25T11:12:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.954+0000 7f3132400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f312c06cb30 con 0x7f312c1416a0 2024-04-25T11:12:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.955+0000 7f3131a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f312c1416a0 0x7f312c0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.955+0000 7f3123e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f312c141290 0x7f312c06d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39942/0 (socket says 172.21.15.7:39942) 2024-04-25T11:12:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.955+0000 7f3123e00640 1 -- 172.21.15.7:0/3585917684 learned_addr learned my addr 172.21.15.7:0/3585917684 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:12:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.955+0000 7f3131000640 1 --2- 172.21.15.7:0/3585917684 >> v2:172.21.15.87:3300/0 conn(0x7f312c15eed0 0x7f312c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.956+0000 7f312be00640 1 -- 172.21.15.7:0/3585917684 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1305584272 0 0) 0x7f312c06c570 con 0x7f312c141290 2024-04-25T11:12:24.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.956+0000 7f312be00640 1 -- 172.21.15.7:0/3585917684 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3110003520 con 0x7f312c141290 2024-04-25T11:12:24.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.956+0000 7f3131a00640 1 -- 172.21.15.7:0/3585917684 >> v2:172.21.15.87:3300/0 conn(0x7f312c15eed0 msgr2=0x7f312c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:24.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.956+0000 7f3131a00640 1 --2- 172.21.15.7:0/3585917684 >> v2:172.21.15.87:3300/0 conn(0x7f312c15eed0 0x7f312c0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.956+0000 7f3131a00640 1 -- 172.21.15.7:0/3585917684 >> v1:172.21.15.87:6789/0 conn(0x7f312c141290 legacy=0x7f312c06d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:24.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.956+0000 7f3131a00640 1 -- 172.21.15.7:0/3585917684 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c06ce10 con 0x7f312c1416a0 2024-04-25T11:12:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.958+0000 7f3131a00640 1 --2- 172.21.15.7:0/3585917684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f312c1416a0 0x7f312c0b4590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3124009d90 tx=0x7f312402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eafced25da7ac229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.958+0000 7f312be00640 1 -- 172.21.15.7:0/3585917684 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124007740 con 0x7f312c1416a0 2024-04-25T11:12:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.959+0000 7f312be00640 1 -- 172.21.15.7:0/3585917684 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31240078e0 con 0x7f312c1416a0 2024-04-25T11:12:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.959+0000 7f312be00640 1 -- 172.21.15.7:0/3585917684 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124007cb0 con 0x7f312c1416a0 2024-04-25T11:12:24.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.959+0000 7f3132400640 1 -- 172.21.15.7:0/3585917684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f312c1416a0 msgr2=0x7f312c0b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.959+0000 7f3132400640 1 --2- 172.21.15.7:0/3585917684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f312c1416a0 0x7f312c0b4590 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3124009d90 tx=0x7f312402a660 comp rx=0 tx=0).stop 2024-04-25T11:12:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.959+0000 7f3132400640 1 -- 172.21.15.7:0/3585917684 shutdown_connections 2024-04-25T11:12:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.959+0000 7f3132400640 1 --2- 172.21.15.7:0/3585917684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f312c1416a0 0x7f312c0b4590 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.959+0000 7f3132400640 1 --2- 172.21.15.7:0/3585917684 >> v2:172.21.15.87:3300/0 conn(0x7f312c15eed0 0x7f312c0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.960+0000 7f3132400640 1 -- 172.21.15.7:0/3585917684 >> 172.21.15.7:0/3585917684 conn(0x7f312c0d1540 msgr2=0x7f312c0d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.960+0000 7f3132400640 1 -- 172.21.15.7:0/3585917684 shutdown_connections 2024-04-25T11:12:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.960+0000 7f3132400640 1 -- 172.21.15.7:0/3585917684 wait complete. 2024-04-25T11:12:24.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.963+0000 7f3132400640 1 Processor -- start 2024-04-25T11:12:24.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.963+0000 7f3132400640 1 -- start start 2024-04-25T11:12:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.965+0000 7f3132400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f312c141290 0x7f312c153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.965+0000 7f3132400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f312c1416a0 0x7f312c1e7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.965+0000 7f3132400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f312c15eed0 0x7f312c155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.965+0000 7f3132400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f312c141ab0 con 0x7f312c141290 2024-04-25T11:12:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.965+0000 7f3132400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f312c0cdcc0 con 0x7f312c1416a0 2024-04-25T11:12:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.965+0000 7f3132400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f312c06dd20 con 0x7f312c15eed0 2024-04-25T11:12:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.965+0000 7f3131000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f312c141290 0x7f312c153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.965+0000 7f3131000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f312c141290 0x7f312c153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54104/0 (socket says 172.21.15.7:54104) 2024-04-25T11:12:24.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.965+0000 7f3131000640 1 -- 172.21.15.7:0/3638803865 learned_addr learned my addr 172.21.15.7:0/3638803865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.965+0000 7f3131a00640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f312c15eed0 0x7f312c155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.966+0000 7f3131000640 1 -- 172.21.15.7:0/3638803865 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f312c15eed0 msgr2=0x7f312c155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.966+0000 7f3123e00640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f312c1416a0 0x7f312c1e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.966+0000 7f3131000640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f312c15eed0 0x7f312c155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.966+0000 7f3131000640 1 -- 172.21.15.7:0/3638803865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f312c1416a0 msgr2=0x7f312c1e7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.966+0000 7f3131000640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f312c1416a0 0x7f312c1e7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.966+0000 7f3131000640 1 -- 172.21.15.7:0/3638803865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c1e8330 con 0x7f312c141290 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.966+0000 7f3131000640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f312c141290 0x7f312c153370 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f311c00e8a0 tx=0x7f311c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.966+0000 7f3129600640 1 -- 172.21.15.7:0/3638803865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c017070 con 0x7f312c141290 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.966+0000 7f3129600640 1 -- 172.21.15.7:0/3638803865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c00ee90 con 0x7f312c141290 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.966+0000 7f3132400640 1 -- 172.21.15.7:0/3638803865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c1e8610 con 0x7f312c141290 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.966+0000 7f3129600640 1 -- 172.21.15.7:0/3638803865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c00cb40 con 0x7f312c141290 2024-04-25T11:12:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.967+0000 7f3132400640 1 -- 172.21.15.7:0/3638803865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c1e8bd0 con 0x7f312c141290 2024-04-25T11:12:24.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.970+0000 7f3129600640 1 -- 172.21.15.7:0/3638803865 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f311c01a070 con 0x7f312c141290 2024-04-25T11:12:24.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.971+0000 7f3129600640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31080744b0 0x7f31080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.971+0000 7f3129600640 1 -- 172.21.15.7:0/3638803865 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f311c095670 con 0x7f312c141290 2024-04-25T11:12:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.972+0000 7f3123e00640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31080744b0 0x7f31080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.972+0000 7f313aa4e740 1 Processor -- start 2024-04-25T11:12:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.972+0000 7f313aa4e740 1 -- start start 2024-04-25T11:12:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.973+0000 7f3123e00640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31080744b0 0x7f31080769c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f312c130800 tx=0x7f3114009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:24.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb2524270 0x564cb28f1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb290bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2503590 0x564cb290c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f313aa4e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564cb282a600 con 0x564cb24fb3b0 2024-04-25T11:12:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f313aa4e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564cb27c7100 con 0x564cb2503590 2024-04-25T11:12:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f313aa4e740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x564cb28265c0 con 0x564cb2524270 2024-04-25T11:12:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f3123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb290bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f3123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb290bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54112/0 (socket says 172.21.15.7:54112) 2024-04-25T11:12:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f3123e00640 1 -- 172.21.15.7:0/2025217611 learned_addr learned my addr 172.21.15.7:0/2025217611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f3131000640 1 --2- 172.21.15.7:0/2025217611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb2524270 0x564cb28f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f3131a00640 1 --2- 172.21.15.7:0/2025217611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2503590 0x564cb290c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f3123e00640 1 -- 172.21.15.7:0/2025217611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb2524270 msgr2=0x564cb28f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f3123e00640 1 --2- 172.21.15.7:0/2025217611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb2524270 0x564cb28f1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f3123e00640 1 -- 172.21.15.7:0/2025217611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2503590 msgr2=0x564cb290c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f3123e00640 1 --2- 172.21.15.7:0/2025217611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2503590 0x564cb290c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.974+0000 7f3123e00640 1 -- 172.21.15.7:0/2025217611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb253c2d0 con 0x564cb24fb3b0 2024-04-25T11:12:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.975+0000 7f3123e00640 1 --2- 172.21.15.7:0/2025217611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb290bf40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3114005c10 tx=0x7f3114010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f10f451a7ef0469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.975+0000 7f3121600640 1 -- 172.21.15.7:0/2025217611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114005c40 con 0x564cb24fb3b0 2024-04-25T11:12:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.975+0000 7f3121600640 1 -- 172.21.15.7:0/2025217611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3114005de0 con 0x564cb24fb3b0 2024-04-25T11:12:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.975+0000 7f3121600640 1 -- 172.21.15.7:0/2025217611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114014bb0 con 0x564cb24fb3b0 2024-04-25T11:12:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.976+0000 7f313aa4e740 1 -- 172.21.15.7:0/2025217611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 msgr2=0x564cb290bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.976+0000 7f313aa4e740 1 --2- 172.21.15.7:0/2025217611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb290bf40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3114005c10 tx=0x7f3114010000 comp rx=0 tx=0).stop 2024-04-25T11:12:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.976+0000 7f313aa4e740 1 -- 172.21.15.7:0/2025217611 shutdown_connections 2024-04-25T11:12:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.976+0000 7f313aa4e740 1 --2- 172.21.15.7:0/2025217611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2503590 0x564cb290c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.976+0000 7f313aa4e740 1 --2- 172.21.15.7:0/2025217611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb290bf40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.976+0000 7f313aa4e740 1 --2- 172.21.15.7:0/2025217611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb2524270 0x564cb28f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.976+0000 7f313aa4e740 1 -- 172.21.15.7:0/2025217611 >> 172.21.15.7:0/2025217611 conn(0x564cb25183c0 msgr2=0x564cb28c6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.976+0000 7f313aa4e740 1 -- 172.21.15.7:0/2025217611 shutdown_connections 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.976+0000 7f313aa4e740 1 -- 172.21.15.7:0/2025217611 wait complete. 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.977+0000 7f313aa4e740 1 -- start start 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.978+0000 7f313aa4e740 1 Processor -- start 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.979+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb24fb3b0 0x564cb28ece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.979+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2503590 0x564cb28f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.979+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2524270 0x564cb290fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.979+0000 7f313aa4e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564cb27b1f70 con 0x564cb2503590 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.979+0000 7f313aa4e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564cb291c860 con 0x564cb2524270 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.979+0000 7f313aa4e740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x564cb2903b70 con 0x564cb24fb3b0 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.979+0000 7f313aa4e740 10 client.0 _ll_register_callbacks cb 0x564cb2820480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.980+0000 7f3123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2503590 0x564cb28f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.980+0000 7f3123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2503590 0x564cb28f3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54128/0 (socket says 172.21.15.7:54128) 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.980+0000 7f3123e00640 1 -- 172.21.15.7:0/446450048 learned_addr learned my addr 172.21.15.7:0/446450048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.980+0000 7f3131000640 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb24fb3b0 0x564cb28ece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.980+0000 7f3131a00640 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2524270 0x564cb290fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.980+0000 7f3123e00640 1 -- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb24fb3b0 msgr2=0x564cb28ece10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.980+0000 7f3123e00640 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb24fb3b0 0x564cb28ece10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.980+0000 7f3123e00640 1 -- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2524270 msgr2=0x564cb290fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.980+0000 7f3123e00640 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2524270 0x564cb290fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.980+0000 7f3123e00640 1 -- 172.21.15.7:0/446450048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb26066b0 con 0x564cb2503590 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.980+0000 7f3131000640 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb24fb3b0 0x564cb28ece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.981+0000 7f3123e00640 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2503590 0x564cb28f3b20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3114001eb0 tx=0x7f3114012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e0f3381d7faefe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.981+0000 7f311a000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.981+0000 7f311a000640 1 -- 172.21.15.7:0/446450048 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114005c40 con 0x564cb2503590 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.981+0000 7f313aa4e740 1 -- 172.21.15.7:0/446450048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3114005de0 con 0x564cb2503590 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.981+0000 7f311a000640 1 -- 172.21.15.7:0/446450048 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f3114020650 con 0x564cb2503590 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.981+0000 7f311a000640 1 -- 172.21.15.7:0/446450048 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3114034050 con 0x564cb2503590 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.981+0000 7f313aa4e740 10 client.5442 fetch_fsmap learned FSMap version 18 2024-04-25T11:12:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.981+0000 7f313aa4e740 1 -- 172.21.15.7:0/446450048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x564cb29103e0 con 0x564cb2503590 2024-04-25T11:12:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.982+0000 7f311a000640 1 -- 172.21.15.7:0/446450048 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3114005de0 con 0x564cb2503590 2024-04-25T11:12:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.982+0000 7f311a000640 1 -- 172.21.15.7:0/446450048 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f3114037020 con 0x564cb2503590 2024-04-25T11:12:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.982+0000 7f313aa4e740 10 client.5442 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T11:12:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.982+0000 7f313aa4e740 10 client.5442 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:12:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.982+0000 7f313aa4e740 20 client.5442 populate_metadata read hostname 'smithi007' 2024-04-25T11:12:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.982+0000 7f313aa4e740 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x564cb24e2e40 0x564cb291b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.982+0000 7f313aa4e740 4 client.5442 mds_command: new command op to 5283 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:12:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.982+0000 7f313aa4e740 1 -- 172.21.15.7:0/446450048 --> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cb2838210 con 0x564cb24e2e40 2024-04-25T11:12:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.983+0000 7f3131000640 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x564cb24e2e40 0x564cb291b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.984+0000 7f3131000640 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x564cb24e2e40 0x564cb291b8b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f383a1f2c3b3ea server_cookie=4361d30c8947b2d4 in_seq=0 out_seq=0 2024-04-25T11:12:24.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.984+0000 7f311a000640 10 client.5442 ms_handle_connect on v2:172.21.15.87:6832/389115631 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.986+0000 7f311a000640 1 -- 172.21.15.7:0/446450048 <== mds.0 v2:172.21.15.87:6832/389115631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564cb2838210 con 0x564cb24e2e40 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f311a000640 10 client.5442 handle_command_reply: tid=0 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f313aa4e740 1 client.5442 shutdown 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f313aa4e740 1 -- 172.21.15.7:0/446450048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2503590 msgr2=0x564cb28f3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f313aa4e740 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2503590 0x564cb28f3b20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3114001eb0 tx=0x7f3114012000 comp rx=0 tx=0).stop 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f313aa4e740 1 -- 172.21.15.7:0/446450048 shutdown_connections 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f313aa4e740 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x564cb24e2e40 0x564cb291b8b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f313aa4e740 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2524270 0x564cb290fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f313aa4e740 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2503590 0x564cb28f3b20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f313aa4e740 1 --2- 172.21.15.7:0/446450048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb24fb3b0 0x564cb28ece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f311a000640 0 client.5442 ms_handle_reset on v2:172.21.15.87:6832/389115631 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f313aa4e740 1 -- 172.21.15.7:0/446450048 >> 172.21.15.7:0/446450048 conn(0x564cb25183c0 msgr2=0x564cb28f1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f313aa4e740 1 -- 172.21.15.7:0/446450048 shutdown_connections 2024-04-25T11:12:24.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.987+0000 7f313aa4e740 1 -- 172.21.15.7:0/446450048 wait complete. 2024-04-25T11:12:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:24.988+0000 7f313aa4e740 20 client.5442 trim_cache size 0 max 16384 2024-04-25T11:12:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.008+0000 7f313aa4e740 1 Processor -- start 2024-04-25T11:12:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.009+0000 7f313aa4e740 1 -- start start 2024-04-25T11:12:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.010+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb292f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.010+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb2503590 0x564cb28f4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.010+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2524270 0x564cb29247c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.010+0000 7f313aa4e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f312c202150 con 0x564cb24fb3b0 2024-04-25T11:12:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.010+0000 7f313aa4e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f312c1e71f0 con 0x564cb2524270 2024-04-25T11:12:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.010+0000 7f313aa4e740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f312c1e9e90 con 0x564cb2503590 2024-04-25T11:12:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.010+0000 7f3131000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb292f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.010+0000 7f3131000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb292f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54134/0 (socket says 172.21.15.7:54134) 2024-04-25T11:12:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.010+0000 7f3131000640 1 -- 172.21.15.7:0/1256281025 learned_addr learned my addr 172.21.15.7:0/1256281025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:25.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.011+0000 7f3123e00640 1 --2- 172.21.15.7:0/1256281025 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb2503590 0x564cb28f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.011+0000 7f3131a00640 1 --2- 172.21.15.7:0/1256281025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2524270 0x564cb29247c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.011+0000 7f3131000640 1 -- 172.21.15.7:0/1256281025 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb2503590 msgr2=0x564cb28f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.011+0000 7f3131000640 1 --2- 172.21.15.7:0/1256281025 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb2503590 0x564cb28f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.011+0000 7f3131000640 1 -- 172.21.15.7:0/1256281025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2524270 msgr2=0x564cb29247c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.011+0000 7f3131000640 1 --2- 172.21.15.7:0/1256281025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2524270 0x564cb29247c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.011+0000 7f3131000640 1 -- 172.21.15.7:0/1256281025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb28f30f0 con 0x564cb24fb3b0 2024-04-25T11:12:25.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.011+0000 7f3131a00640 1 --2- 172.21.15.7:0/1256281025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2524270 0x564cb29247c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.011+0000 7f3131000640 1 --2- 172.21.15.7:0/1256281025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb292f040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f311c00c520 tx=0x7f311c05f4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374a3c3c78491b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.012+0000 7f3120c00640 1 -- 172.21.15.7:0/1256281025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c021040 con 0x564cb24fb3b0 2024-04-25T11:12:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.012+0000 7f3120c00640 1 -- 172.21.15.7:0/1256281025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c078030 con 0x564cb24fb3b0 2024-04-25T11:12:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.012+0000 7f3120c00640 1 -- 172.21.15.7:0/1256281025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c068330 con 0x564cb24fb3b0 2024-04-25T11:12:25.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.012+0000 7f313aa4e740 1 -- 172.21.15.7:0/1256281025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 msgr2=0x564cb292f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:25.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.012+0000 7f313aa4e740 1 --2- 172.21.15.7:0/1256281025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb292f040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f311c00c520 tx=0x7f311c05f4d0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.013+0000 7f313aa4e740 1 -- 172.21.15.7:0/1256281025 shutdown_connections 2024-04-25T11:12:25.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.013+0000 7f313aa4e740 1 --2- 172.21.15.7:0/1256281025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2524270 0x564cb29247c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.013+0000 7f313aa4e740 1 --2- 172.21.15.7:0/1256281025 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb2503590 0x564cb28f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.013+0000 7f313aa4e740 1 --2- 172.21.15.7:0/1256281025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb24fb3b0 0x564cb292f040 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.013+0000 7f313aa4e740 1 -- 172.21.15.7:0/1256281025 >> 172.21.15.7:0/1256281025 conn(0x564cb25183c0 msgr2=0x564cb291bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:25.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.013+0000 7f313aa4e740 1 -- 172.21.15.7:0/1256281025 shutdown_connections 2024-04-25T11:12:25.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.013+0000 7f313aa4e740 1 -- 172.21.15.7:0/1256281025 wait complete. 2024-04-25T11:12:25.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.014+0000 7f313aa4e740 1 -- start start 2024-04-25T11:12:25.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.014+0000 7f313aa4e740 1 Processor -- start 2024-04-25T11:12:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2503590 0x564cb292db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2524270 0x564cb28c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f313aa4e740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb28c2b40 0x564cb28ecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f313aa4e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564cb28f21c0 con 0x564cb2524270 2024-04-25T11:12:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f313aa4e740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564cb2910670 con 0x564cb2503590 2024-04-25T11:12:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f313aa4e740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x564cb28f27b0 con 0x564cb28c2b40 2024-04-25T11:12:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f313aa4e740 10 client.0 _ll_register_callbacks cb 0x7f3114005ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:12:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f3123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2524270 0x564cb28c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f3123e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2524270 0x564cb28c0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54140/0 (socket says 172.21.15.7:54140) 2024-04-25T11:12:25.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f3123e00640 1 -- 172.21.15.7:0/2472657669 learned_addr learned my addr 172.21.15.7:0/2472657669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:25.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f3131a00640 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb28c2b40 0x564cb28ecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f3131000640 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2503590 0x564cb292db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f3123e00640 1 -- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb28c2b40 msgr2=0x564cb28ecc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:25.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f3123e00640 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb28c2b40 0x564cb28ecc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f3123e00640 1 -- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2503590 msgr2=0x564cb292db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f3123e00640 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2503590 0x564cb292db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.016+0000 7f3123e00640 1 -- 172.21.15.7:0/2472657669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb28eae10 con 0x564cb2524270 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.017+0000 7f3123e00640 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2524270 0x564cb28c0a50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f31140070a0 tx=0x7f3114000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6b87f39ba78a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.017+0000 7f311a000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.017+0000 7f311a000640 1 -- 172.21.15.7:0/2472657669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114014800 con 0x564cb2524270 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.017+0000 7f311a000640 1 -- 172.21.15.7:0/2472657669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f3114005b90 con 0x564cb2524270 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.017+0000 7f313aa4e740 1 -- 172.21.15.7:0/2472657669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f311c078030 con 0x564cb2524270 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.017+0000 7f311a000640 1 -- 172.21.15.7:0/2472657669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3114005d90 con 0x564cb2524270 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.017+0000 7f313aa4e740 10 client.5448 fetch_fsmap learned FSMap version 18 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.018+0000 7f313aa4e740 1 -- 172.21.15.7:0/2472657669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x564cb2989540 con 0x564cb2524270 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.018+0000 7f311a000640 1 -- 172.21.15.7:0/2472657669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c078030 con 0x564cb2524270 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.018+0000 7f311a000640 1 -- 172.21.15.7:0/2472657669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f3114037050 con 0x564cb2524270 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.018+0000 7f313aa4e740 10 client.5448 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T11:12:25.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.018+0000 7f313aa4e740 10 client.5448 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:12:25.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.018+0000 7f313aa4e740 20 client.5448 populate_metadata read hostname 'smithi007' 2024-04-25T11:12:25.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.018+0000 7f313aa4e740 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x564cb291cc00 0x564cb291d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.018+0000 7f313aa4e740 4 client.5448 mds_command: new command op to 5283 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:12:25.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.018+0000 7f313aa4e740 1 -- 172.21.15.7:0/2472657669 --> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] -- command(tid 0: {"prefix": "status"}) -- 0x564cb2832080 con 0x564cb291cc00 2024-04-25T11:12:25.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.019+0000 7f3131000640 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x564cb291cc00 0x564cb291d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.020+0000 7f3131000640 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x564cb291cc00 0x564cb291d010 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4823e61939209ed2 server_cookie=9392f096d99c05c in_seq=0 out_seq=0 2024-04-25T11:12:25.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.020+0000 7f311a000640 10 client.5448 ms_handle_connect on v2:172.21.15.87:6832/389115631 2024-04-25T11:12:25.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.020+0000 7f311a000640 1 -- 172.21.15.7:0/2472657669 <== mds.0 v2:172.21.15.87:6832/389115631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x564cb2832080 con 0x564cb291cc00 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.020+0000 7f311a000640 10 client.5448 handle_command_reply: tid=0 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.020+0000 7f313aa4e740 1 client.5448 shutdown 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f313aa4e740 1 -- 172.21.15.7:0/2472657669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2524270 msgr2=0x564cb28c0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f313aa4e740 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2524270 0x564cb28c0a50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f31140070a0 tx=0x7f3114000bc0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f313aa4e740 1 -- 172.21.15.7:0/2472657669 shutdown_connections 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f313aa4e740 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x564cb291cc00 0x564cb291d010 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f313aa4e740 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564cb28c2b40 0x564cb28ecc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f313aa4e740 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cb2524270 0x564cb28c0a50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f313aa4e740 1 --2- 172.21.15.7:0/2472657669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564cb2503590 0x564cb292db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f311a000640 0 client.5448 ms_handle_reset on v2:172.21.15.87:6832/389115631 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f313aa4e740 1 -- 172.21.15.7:0/2472657669 >> 172.21.15.7:0/2472657669 conn(0x564cb25183c0 msgr2=0x564cb292ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f313aa4e740 1 -- 172.21.15.7:0/2472657669 shutdown_connections 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f313aa4e740 1 -- 172.21.15.7:0/2472657669 wait complete. 2024-04-25T11:12:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.021+0000 7f313aa4e740 20 client.5448 trim_cache size 0 max 16384 2024-04-25T11:12:25.023 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:12:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:12:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:12:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5283, 2024-04-25T11:12:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:12:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:12:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:12:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.4275654470000001, 2024-04-25T11:12:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 18, 2024-04-25T11:12:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 40, 2024-04-25T11:12:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 40, 2024-04-25T11:12:25.024 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 17.166957665999998 2024-04-25T11:12:25.024 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:12:25.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 -- 172.21.15.7:0/3638803865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31080744b0 msgr2=0x7f31080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:25.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31080744b0 0x7f31080769c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f312c130800 tx=0x7f3114009040 comp rx=0 tx=0).stop 2024-04-25T11:12:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 -- 172.21.15.7:0/3638803865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f312c141290 msgr2=0x7f312c153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f312c141290 0x7f312c153370 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f311c00e8a0 tx=0x7f311c00ed60 comp rx=0 tx=0).stop 2024-04-25T11:12:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 -- 172.21.15.7:0/3638803865 shutdown_connections 2024-04-25T11:12:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31080744b0 0x7f31080769c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f312c15eed0 0x7f312c155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f312c1416a0 0x7f312c1e7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 --2- 172.21.15.7:0/3638803865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f312c141290 0x7f312c153370 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 -- 172.21.15.7:0/3638803865 >> 172.21.15.7:0/3638803865 conn(0x7f312c0d1540 msgr2=0x7f312c0b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 -- 172.21.15.7:0/3638803865 shutdown_connections 2024-04-25T11:12:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.022+0000 7f3132400640 1 -- 172.21.15.7:0/3638803865 wait complete. 2024-04-25T11:12:25.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:12:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.319+0000 7f88edc00640 1 Processor -- start 2024-04-25T11:12:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.319+0000 7f88edc00640 1 -- start start 2024-04-25T11:12:25.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.321+0000 7f88edc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88e806dc90 0x7f88e80d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.321+0000 7f88edc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88e8070c20 0x7f88e80d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.321+0000 7f88edc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e80cffa0 con 0x7f88e8070810 2024-04-25T11:12:25.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.322+0000 7f88edc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f88e8160d90 con 0x7f88e806dc90 2024-04-25T11:12:25.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.322+0000 7f88edc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f88e8161070 con 0x7f88e8070c20 2024-04-25T11:12:25.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.322+0000 7f88e7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88e806dc90 0x7f88e80d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.322+0000 7f88e7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88e806dc90 0x7f88e80d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54144/0 (socket says 172.21.15.7:54144) 2024-04-25T11:12:25.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.322+0000 7f88e7400640 1 -- 172.21.15.7:0/1028989110 learned_addr learned my addr 172.21.15.7:0/1028989110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.322+0000 7f88ed200640 1 --2- 172.21.15.7:0/1028989110 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88e8070c20 0x7f88e80d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.322+0000 7f88e7400640 1 -- 172.21.15.7:0/1028989110 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88e8070c20 msgr2=0x7f88e80d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.322+0000 7f88e7400640 1 --2- 172.21.15.7:0/1028989110 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88e8070c20 0x7f88e80d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.322+0000 7f88e7400640 1 -- 172.21.15.7:0/1028989110 >> v1:172.21.15.87:6789/0 conn(0x7f88e8070810 legacy=0x7f88e806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.322+0000 7f88e7400640 1 -- 172.21.15.7:0/1028989110 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e8161350 con 0x7f88e806dc90 2024-04-25T11:12:25.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.322+0000 7f88ed200640 1 --2- 172.21.15.7:0/1028989110 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88e8070c20 0x7f88e80d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:25.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.324+0000 7f88e7400640 1 --2- 172.21.15.7:0/1028989110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88e806dc90 0x7f88e80d0460 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009870 tx=0x7f88d802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eca355e9fc4d4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:25.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.324+0000 7f88e6a00640 1 -- 172.21.15.7:0/1028989110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8037070 con 0x7f88e806dc90 2024-04-25T11:12:25.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.324+0000 7f88e6a00640 1 -- 172.21.15.7:0/1028989110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d8032440 con 0x7f88e806dc90 2024-04-25T11:12:25.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.324+0000 7f88e6a00640 1 -- 172.21.15.7:0/1028989110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8032810 con 0x7f88e806dc90 2024-04-25T11:12:25.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.325+0000 7f88edc00640 1 -- 172.21.15.7:0/1028989110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88e806dc90 msgr2=0x7f88e80d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:25.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.325+0000 7f88edc00640 1 --2- 172.21.15.7:0/1028989110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88e806dc90 0x7f88e80d0460 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009870 tx=0x7f88d802a200 comp rx=0 tx=0).stop 2024-04-25T11:12:25.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.325+0000 7f88edc00640 1 -- 172.21.15.7:0/1028989110 shutdown_connections 2024-04-25T11:12:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.325+0000 7f88edc00640 1 --2- 172.21.15.7:0/1028989110 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88e8070c20 0x7f88e80d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.325+0000 7f88edc00640 1 --2- 172.21.15.7:0/1028989110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88e806dc90 0x7f88e80d0460 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.325+0000 7f88edc00640 1 -- 172.21.15.7:0/1028989110 >> 172.21.15.7:0/1028989110 conn(0x7f88e8073090 msgr2=0x7f88e80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.325+0000 7f88edc00640 1 -- 172.21.15.7:0/1028989110 shutdown_connections 2024-04-25T11:12:25.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.325+0000 7f88edc00640 1 -- 172.21.15.7:0/1028989110 wait complete. 2024-04-25T11:12:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.328+0000 7f88edc00640 1 Processor -- start 2024-04-25T11:12:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.329+0000 7f88edc00640 1 -- start start 2024-04-25T11:12:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.330+0000 7f88edc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88e8070810 0x7f88e81446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.330+0000 7f88edc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88e8070c20 0x7f88e80ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.330+0000 7f88edc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88e80aead0 0x7f88e8141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88edc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88e8143530 con 0x7f88e8070810 2024-04-25T11:12:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88edc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f88e81433b0 con 0x7f88e80aead0 2024-04-25T11:12:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88edc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f88e815e4d0 con 0x7f88e8070c20 2024-04-25T11:12:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88e7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88e8070810 0x7f88e81446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88e7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88e8070810 0x7f88e81446e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54146/0 (socket says 172.21.15.7:54146) 2024-04-25T11:12:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88e7e00640 1 -- 172.21.15.7:0/3190129576 learned_addr learned my addr 172.21.15.7:0/3190129576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88e7400640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88e8070c20 0x7f88e80ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88e7e00640 1 -- 172.21.15.7:0/3190129576 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88e8070c20 msgr2=0x7f88e80ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88e7e00640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88e8070c20 0x7f88e80ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88ed200640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88e80aead0 0x7f88e8141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88e7e00640 1 -- 172.21.15.7:0/3190129576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88e80aead0 msgr2=0x7f88e8141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88e7e00640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88e80aead0 0x7f88e8141740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.331+0000 7f88e7e00640 1 -- 172.21.15.7:0/3190129576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e81452c0 con 0x7f88e8070810 2024-04-25T11:12:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.332+0000 7f88e7e00640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88e8070810 0x7f88e81446e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f88d000b3f0 tx=0x7f88d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.332+0000 7f88dfe00640 1 -- 172.21.15.7:0/3190129576 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0015020 con 0x7f88e8070810 2024-04-25T11:12:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.332+0000 7f88dfe00640 1 -- 172.21.15.7:0/3190129576 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d0011830 con 0x7f88e8070810 2024-04-25T11:12:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.332+0000 7f88dfe00640 1 -- 172.21.15.7:0/3190129576 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0011c20 con 0x7f88e8070810 2024-04-25T11:12:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.332+0000 7f88edc00640 1 -- 172.21.15.7:0/3190129576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e81455a0 con 0x7f88e8070810 2024-04-25T11:12:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.332+0000 7f88edc00640 1 -- 172.21.15.7:0/3190129576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e8141220 con 0x7f88e8070810 2024-04-25T11:12:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.334+0000 7f88edc00640 1 -- 172.21.15.7:0/3190129576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e8070020 con 0x7f88e8070810 2024-04-25T11:12:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.335+0000 7f88dfe00640 1 -- 172.21.15.7:0/3190129576 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f88d0018020 con 0x7f88e8070810 2024-04-25T11:12:25.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.336+0000 7f88dfe00640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88c40744e0 0x7f88c40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:25.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.336+0000 7f88dfe00640 1 -- 172.21.15.7:0/3190129576 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f88d0095040 con 0x7f88e8070810 2024-04-25T11:12:25.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.337+0000 7f88e7400640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88c40744e0 0x7f88c40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.337+0000 7f88e7400640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88c40744e0 0x7f88c40769f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88d8034000 tx=0x7f88d802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:25.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.341+0000 7f88dfe00640 1 -- 172.21.15.7:0/3190129576 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88e8070020 con 0x7f88e8070810 2024-04-25T11:12:25.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.585+0000 7f88edc00640 1 -- 172.21.15.7:0/3190129576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f88e8050740 con 0x7f88e8070810 2024-04-25T11:12:25.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.586+0000 7f88dfe00640 1 -- 172.21.15.7:0/3190129576 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f88d001f030 con 0x7f88e8070810 2024-04-25T11:12:25.586 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:12:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.588+0000 7f88edc00640 1 -- 172.21.15.7:0/3190129576 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88c40744e0 msgr2=0x7f88c40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:25.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.588+0000 7f88edc00640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88c40744e0 0x7f88c40769f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88d8034000 tx=0x7f88d802e000 comp rx=0 tx=0).stop 2024-04-25T11:12:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.588+0000 7f88edc00640 1 -- 172.21.15.7:0/3190129576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88e8070810 msgr2=0x7f88e81446e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.588+0000 7f88edc00640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88e8070810 0x7f88e81446e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f88d000b3f0 tx=0x7f88d000b8b0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.588+0000 7f88edc00640 1 -- 172.21.15.7:0/3190129576 shutdown_connections 2024-04-25T11:12:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.588+0000 7f88edc00640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88c40744e0 0x7f88c40769f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.588+0000 7f88edc00640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88e80aead0 0x7f88e8141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.588+0000 7f88edc00640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88e8070c20 0x7f88e80ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.588+0000 7f88edc00640 1 --2- 172.21.15.7:0/3190129576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88e8070810 0x7f88e81446e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.588+0000 7f88edc00640 1 -- 172.21.15.7:0/3190129576 >> 172.21.15.7:0/3190129576 conn(0x7f88e8073090 msgr2=0x7f88e80b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.588+0000 7f88edc00640 1 -- 172.21.15.7:0/3190129576 shutdown_connections 2024-04-25T11:12:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25.589+0000 7f88edc00640 1 -- 172.21.15.7:0/3190129576 wait complete. 2024-04-25T11:12:25.618 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:12:25.618 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:12:25.618 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:12:25.618 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:12:25.618 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:12:25.618 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:12:25.618 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:12:25.618 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:12:25.618 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:12:25.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:25.619 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:12:25.668 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:12:25.669 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42429sec preferred_lft 42429sec 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6098:c7ff:fe8a:4a30/64 scope link 2024-04-25T11:12:25.670 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:25.671 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:12:25.671 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:12:25.671 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::646a:35ff:fe7b:ad0/64 scope link 2024-04-25T11:12:25.671 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:25.671 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:12:25.671 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:12:25.671 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4465:c4ff:fe62:f75f/64 scope link 2024-04-25T11:12:25.671 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:25.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:25.671 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:12:25.741 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:12:25.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:25.745 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:12:25.799 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:12:25.799 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:12:25.799 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:12:25.799 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:12:25.799 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:12:25.800 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:12:25.800 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:12:25.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:12:25.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:12:25.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:12:25.800 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:12:25.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:12:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:12:26.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:26.018 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:12:26.066 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:12:26.066 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:12:26.067 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:12:26.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:26.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:12:26.154 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:12:26.154 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:12:26.210 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:12:26.210 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:12:26.210 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:12:26.262 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:12:26.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:26.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:12:26.469 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:12:26.469 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:12:26.520 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:12:26.520 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:12:26.521 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:12:26.521 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:12:26.521 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:12:26.523 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:12:26.523 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:12:26.523 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:12:26.523 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:12:26.523 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:12:26.530 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:12:26.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:26.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:12:26.617 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:12:26.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:12:26.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:12:26.772 INFO:teuthology.orchestra.run.smithi007.stdout:client4862 2024-04-25T11:12:26.772 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:12:26.772 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:12:26.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:12:27.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.119+0000 7fc7b6400640 1 Processor -- start 2024-04-25T11:12:27.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.119+0000 7fc7b6400640 1 -- start start 2024-04-25T11:12:27.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.121+0000 7fc7b6400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7b00d0fb0 0x7fc7b015fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:27.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.121+0000 7fc7b6400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b00d13c0 con 0x7fc7b00b8e50 2024-04-25T11:12:27.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.121+0000 7fc7b6400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b0163f10 con 0x7fc7b00b7af0 2024-04-25T11:12:27.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.121+0000 7fc7b6400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b01641f0 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.122+0000 7fc7b5000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7b00d0fb0 0x7fc7b015fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:27.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.122+0000 7fc7b5000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7b00d0fb0 0x7fc7b015fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54162/0 (socket says 172.21.15.7:54162) 2024-04-25T11:12:27.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.122+0000 7fc7b5000640 1 -- 172.21.15.7:0/3592596595 learned_addr learned my addr 172.21.15.7:0/3592596595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:27.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.122+0000 7fc7afe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc7b00b8e50 0x7fc7b00b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53060/0 (socket says 172.21.15.7:53060) 2024-04-25T11:12:27.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.122+0000 7fc7b5000640 1 -- 172.21.15.7:0/3592596595 >> v1:172.21.15.87:6789/0 conn(0x7fc7b00b7af0 legacy=0x7fc7b00d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:27.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.122+0000 7fc7b5000640 1 -- 172.21.15.7:0/3592596595 >> v1:172.21.15.7:6789/0 conn(0x7fc7b00b8e50 legacy=0x7fc7b00b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:27.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.122+0000 7fc7b5000640 1 -- 172.21.15.7:0/3592596595 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b01644d0 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.123+0000 7fc7b5000640 1 --2- 172.21.15.7:0/3592596595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7b00d0fb0 0x7fc7b015fea0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009870 tx=0x7fc7a402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8cd461db65535b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:27.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.124+0000 7fc7af400640 1 -- 172.21.15.7:0/3592596595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4037070 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.124+0000 7fc7af400640 1 -- 172.21.15.7:0/3592596595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a4032440 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.124+0000 7fc7af400640 1 -- 172.21.15.7:0/3592596595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4032810 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.125+0000 7fc7b6400640 1 -- 172.21.15.7:0/3592596595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7b00d0fb0 msgr2=0x7fc7b015fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:27.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.125+0000 7fc7b6400640 1 --2- 172.21.15.7:0/3592596595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7b00d0fb0 0x7fc7b015fea0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009870 tx=0x7fc7a402a210 comp rx=0 tx=0).stop 2024-04-25T11:12:27.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.125+0000 7fc7b6400640 1 -- 172.21.15.7:0/3592596595 shutdown_connections 2024-04-25T11:12:27.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.125+0000 7fc7b6400640 1 --2- 172.21.15.7:0/3592596595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7b00d0fb0 0x7fc7b015fea0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:27.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.125+0000 7fc7b6400640 1 -- 172.21.15.7:0/3592596595 >> 172.21.15.7:0/3592596595 conn(0x7fc7b00d23e0 msgr2=0x7fc7b00d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:27.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.125+0000 7fc7b6400640 1 -- 172.21.15.7:0/3592596595 shutdown_connections 2024-04-25T11:12:27.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.125+0000 7fc7b6400640 1 -- 172.21.15.7:0/3592596595 wait complete. 2024-04-25T11:12:27.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.127+0000 7fc7b6400640 1 Processor -- start 2024-04-25T11:12:27.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.127+0000 7fc7b6400640 1 -- start start 2024-04-25T11:12:27.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.129+0000 7fc7b6400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7b00b8e50 0x7fc7b00c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:27.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.129+0000 7fc7b6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7b00d0fb0 0x7fc7b014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:27.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.129+0000 7fc7b6400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7b01f63c0 0x7fc7b01f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:27.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.129+0000 7fc7b6400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b00b9260 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.129+0000 7fc7b6400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b00b4bc0 con 0x7fc7b01f63c0 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.129+0000 7fc7b6400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b0163a80 con 0x7fc7b00b8e50 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.129+0000 7fc7afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7b00d0fb0 0x7fc7b014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.129+0000 7fc7afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7b00d0fb0 0x7fc7b014fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54174/0 (socket says 172.21.15.7:54174) 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.129+0000 7fc7afe00640 1 -- 172.21.15.7:0/2282790568 learned_addr learned my addr 172.21.15.7:0/2282790568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7b5a00640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7b01f63c0 0x7fc7b01f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7b5000640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7b00b8e50 0x7fc7b00c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7afe00640 1 -- 172.21.15.7:0/2282790568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7b00b8e50 msgr2=0x7fc7b00c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7afe00640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7b00b8e50 0x7fc7b00c9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7afe00640 1 -- 172.21.15.7:0/2282790568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7b01f63c0 msgr2=0x7fc7b01f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7afe00640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7b01f63c0 0x7fc7b01f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7afe00640 1 -- 172.21.15.7:0/2282790568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b00ca2c0 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7b5a00640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7b01f63c0 0x7fc7b01f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:12:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7afe00640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7b00d0fb0 0x7fc7b014fe10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00c3c0 tx=0x7fc79c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7acc00640 1 -- 172.21.15.7:0/2282790568 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0074e0 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7acc00640 1 -- 172.21.15.7:0/2282790568 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc79c007680 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7b6400640 1 -- 172.21.15.7:0/2282790568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b01620b0 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.130+0000 7fc7acc00640 1 -- 172.21.15.7:0/2282790568 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c004a00 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.131+0000 7fc7b6400640 1 -- 172.21.15.7:0/2282790568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b0162670 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.132+0000 7fc7b6400640 1 -- 172.21.15.7:0/2282790568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b0161e20 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.133+0000 7fc7acc00640 1 -- 172.21.15.7:0/2282790568 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc79c01a020 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.135+0000 7fc7acc00640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7900744b0 0x7fc7900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:27.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.135+0000 7fc7acc00640 1 -- 172.21.15.7:0/2282790568 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fc79c094ca0 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.135+0000 7fc7b5000640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7900744b0 0x7fc7900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:27.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.136+0000 7fc7b5000640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7900744b0 0x7fc7900769c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4034000 tx=0x7fc7a402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:27.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.140+0000 7fc7acc00640 1 -- 172.21.15.7:0/2282790568 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc79c05f200 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.383+0000 7fc7b6400640 1 -- 172.21.15.7:0/2282790568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc7b0050740 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.384+0000 7fc7acc00640 1 -- 172.21.15.7:0/2282790568 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7fc79c05eb00 con 0x7fc7b00d0fb0 2024-04-25T11:12:27.384 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:12:27.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.386+0000 7fc7b6400640 1 -- 172.21.15.7:0/2282790568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7900744b0 msgr2=0x7fc7900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:27.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.386+0000 7fc7b6400640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7900744b0 0x7fc7900769c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4034000 tx=0x7fc7a402e000 comp rx=0 tx=0).stop 2024-04-25T11:12:27.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.386+0000 7fc7b6400640 1 -- 172.21.15.7:0/2282790568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7b00d0fb0 msgr2=0x7fc7b014fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:27.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.386+0000 7fc7b6400640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7b00d0fb0 0x7fc7b014fe10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00c3c0 tx=0x7fc79c00c880 comp rx=0 tx=0).stop 2024-04-25T11:12:27.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.386+0000 7fc7b6400640 1 -- 172.21.15.7:0/2282790568 shutdown_connections 2024-04-25T11:12:27.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.386+0000 7fc7b6400640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7900744b0 0x7fc7900769c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:27.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.386+0000 7fc7b6400640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7b01f63c0 0x7fc7b01f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:27.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.386+0000 7fc7b6400640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7b00d0fb0 0x7fc7b014fe10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:27.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.386+0000 7fc7b6400640 1 --2- 172.21.15.7:0/2282790568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7b00b8e50 0x7fc7b00c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:27.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.386+0000 7fc7b6400640 1 -- 172.21.15.7:0/2282790568 >> 172.21.15.7:0/2282790568 conn(0x7fc7b00d23e0 msgr2=0x7fc7b015f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:27.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.387+0000 7fc7b6400640 1 -- 172.21.15.7:0/2282790568 shutdown_connections 2024-04-25T11:12:27.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:27.387+0000 7fc7b6400640 1 -- 172.21.15.7:0/2282790568 wait complete. 2024-04-25T11:12:27.412 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:12:27.412 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:12:27.412 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:12:27.412 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:12:27.412 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:12:27.412 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:12:27.412 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:12:27.412 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:12:27.412 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:12:27.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:27.413 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:12:27.471 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:12:27.472 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42428sec preferred_lft 42428sec 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::58f2:6bff:fe8b:8b4c/64 scope link 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:12:27.473 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::98fe:1bff:fe9f:d71c/64 scope link 2024-04-25T11:12:27.474 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:27.474 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:12:27.474 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:12:27.474 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::3c38:a8ff:fec7:ea00/64 scope link 2024-04-25T11:12:27.474 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:12:27.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:27.475 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:12:27.556 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:12:27.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:27.561 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:12:27.614 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:12:27.614 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:12:27.614 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:12:27.615 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:12:27.615 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:12:27.615 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:12:27.615 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:12:27.615 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:12:27.615 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:12:27.615 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:12:27.615 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:12:27.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:12:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:12:27.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:12:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:12:27.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:27.823 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:12:27.874 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:12:27.875 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:12:27.875 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:12:27.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:27.875 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T11:12:27.955 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:12:27.956 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:12:28.009 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:12:28.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:12:28.009 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:12:28.060 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:12:28.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:28.061 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:12:28.258 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:12:28.258 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:12:28.297 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:12:28.298 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:12:28.298 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:12:28.298 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 1@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:12:28.298 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:12:28.299 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:12:28.299 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:12:28.299 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:12:28.299 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:12:28.299 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:12:28.303 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:12:28.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:12:28.304 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:12:28.381 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:12:28.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:12:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:12:28.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:12:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:12:28.493 INFO:teuthology.orchestra.run.smithi087.stdout:client4865 2024-04-25T11:12:28.494 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:12:28.494 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:12:28.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 1 2024-04-25T11:12:28.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.841+0000 7fcc23200640 1 Processor -- start 2024-04-25T11:12:28.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.841+0000 7fcc23200640 1 -- start start 2024-04-25T11:12:28.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.844+0000 7fcc23200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc1c15dfb0 0x7fcc1c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.844+0000 7fcc23200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcc1c06fab0 0x7fcc1c0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.844+0000 7fcc23200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc1c15e3c0 con 0x7fcc1c0d0b40 2024-04-25T11:12:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.844+0000 7fcc23200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fcc1c0b09b0 con 0x7fcc1c15dfb0 2024-04-25T11:12:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.844+0000 7fcc23200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcc1c0cf800 con 0x7fcc1c06fab0 2024-04-25T11:12:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.845+0000 7fcc1be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc1c15dfb0 0x7fcc1c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.845+0000 7fcc1be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc1c15dfb0 0x7fcc1c0b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54190/0 (socket says 172.21.15.7:54190) 2024-04-25T11:12:28.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.845+0000 7fcc1be00640 1 -- 172.21.15.7:0/3227541274 learned_addr learned my addr 172.21.15.7:0/3227541274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:28.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.845+0000 7fcc21200640 1 --2- 172.21.15.7:0/3227541274 >> v2:172.21.15.87:3300/0 conn(0x7fcc1c06fab0 0x7fcc1c0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:28.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.845+0000 7fcc1be00640 1 -- 172.21.15.7:0/3227541274 >> v2:172.21.15.87:3300/0 conn(0x7fcc1c06fab0 msgr2=0x7fcc1c0aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:28.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.845+0000 7fcc1be00640 1 --2- 172.21.15.7:0/3227541274 >> v2:172.21.15.87:3300/0 conn(0x7fcc1c06fab0 0x7fcc1c0aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:28.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.845+0000 7fcc1be00640 1 -- 172.21.15.7:0/3227541274 >> v1:172.21.15.7:6789/0 conn(0x7fcc1c0d0b40 legacy=0x7fcc1c06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:28.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.845+0000 7fcc1be00640 1 -- 172.21.15.7:0/3227541274 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c0cfaa0 con 0x7fcc1c15dfb0 2024-04-25T11:12:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.846+0000 7fcc1be00640 1 --2- 172.21.15.7:0/3227541274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1c15dfb0 0x7fcc1c0b0470 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c09fb50 tx=0x7fcc0402a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ad24d77fe84832 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.847+0000 7fcc1aa00640 1 -- 172.21.15.7:0/3227541274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04037070 con 0x7fcc1c15dfb0 2024-04-25T11:12:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.847+0000 7fcc1aa00640 1 -- 172.21.15.7:0/3227541274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc04032440 con 0x7fcc1c15dfb0 2024-04-25T11:12:28.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.847+0000 7fcc1aa00640 1 -- 172.21.15.7:0/3227541274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04032810 con 0x7fcc1c15dfb0 2024-04-25T11:12:28.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.848+0000 7fcc23200640 1 -- 172.21.15.7:0/3227541274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1c15dfb0 msgr2=0x7fcc1c0b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:28.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.848+0000 7fcc23200640 1 --2- 172.21.15.7:0/3227541274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1c15dfb0 0x7fcc1c0b0470 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c09fb50 tx=0x7fcc0402a280 comp rx=0 tx=0).stop 2024-04-25T11:12:28.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.848+0000 7fcc23200640 1 -- 172.21.15.7:0/3227541274 shutdown_connections 2024-04-25T11:12:28.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.848+0000 7fcc23200640 1 --2- 172.21.15.7:0/3227541274 >> v2:172.21.15.87:3300/0 conn(0x7fcc1c06fab0 0x7fcc1c0aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:28.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.848+0000 7fcc23200640 1 --2- 172.21.15.7:0/3227541274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1c15dfb0 0x7fcc1c0b0470 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:28.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.848+0000 7fcc23200640 1 -- 172.21.15.7:0/3227541274 >> 172.21.15.7:0/3227541274 conn(0x7fcc1c0742c0 msgr2=0x7fcc1c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:28.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.848+0000 7fcc23200640 1 -- 172.21.15.7:0/3227541274 shutdown_connections 2024-04-25T11:12:28.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.848+0000 7fcc23200640 1 -- 172.21.15.7:0/3227541274 wait complete. 2024-04-25T11:12:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.850+0000 7fcc23200640 1 Processor -- start 2024-04-25T11:12:28.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.851+0000 7fcc23200640 1 -- start start 2024-04-25T11:12:28.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.852+0000 7fcc23200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcc1c06fab0 0x7fcc1c1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:28.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.852+0000 7fcc23200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1c0d0b40 0x7fcc1c1594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:28.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.852+0000 7fcc23200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc1c15dfb0 0x7fcc1c15c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.852+0000 7fcc23200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c0d0340 con 0x7fcc1c0d0b40 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.852+0000 7fcc23200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c0d0060 con 0x7fcc1c15dfb0 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.852+0000 7fcc23200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcc1c0d0620 con 0x7fcc1c06fab0 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.853+0000 7fcc1b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1c0d0b40 0x7fcc1c1594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.853+0000 7fcc1be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcc1c06fab0 0x7fcc1c1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.853+0000 7fcc1b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1c0d0b40 0x7fcc1c1594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54192/0 (socket says 172.21.15.7:54192) 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.853+0000 7fcc1b400640 1 -- 172.21.15.7:0/2563441449 learned_addr learned my addr 172.21.15.7:0/2563441449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.853+0000 7fcc21200640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc1c15dfb0 0x7fcc1c15c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.853+0000 7fcc1b400640 1 -- 172.21.15.7:0/2563441449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcc1c06fab0 msgr2=0x7fcc1c1536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.853+0000 7fcc1b400640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcc1c06fab0 0x7fcc1c1536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.853+0000 7fcc1b400640 1 -- 172.21.15.7:0/2563441449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc1c15dfb0 msgr2=0x7fcc1c15c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.853+0000 7fcc1b400640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc1c15dfb0 0x7fcc1c15c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.853+0000 7fcc1b400640 1 -- 172.21.15.7:0/2563441449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c15b880 con 0x7fcc1c0d0b40 2024-04-25T11:12:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.854+0000 7fcc1b400640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1c0d0b40 0x7fcc1c1594d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00c6b0 tx=0x7fcc0c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.854+0000 7fcc13e00640 1 -- 172.21.15.7:0/2563441449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c016020 con 0x7fcc1c0d0b40 2024-04-25T11:12:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.854+0000 7fcc13e00640 1 -- 172.21.15.7:0/2563441449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc0c006d50 con 0x7fcc1c0d0b40 2024-04-25T11:12:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.854+0000 7fcc23200640 1 -- 172.21.15.7:0/2563441449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c15bb60 con 0x7fcc1c0d0b40 2024-04-25T11:12:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.854+0000 7fcc13e00640 1 -- 172.21.15.7:0/2563441449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c003bb0 con 0x7fcc1c0d0b40 2024-04-25T11:12:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.854+0000 7fcc23200640 1 -- 172.21.15.7:0/2563441449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1c15c120 con 0x7fcc1c0d0b40 2024-04-25T11:12:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.856+0000 7fcc23200640 1 -- 172.21.15.7:0/2563441449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1c050740 con 0x7fcc1c0d0b40 2024-04-25T11:12:28.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.857+0000 7fcc13e00640 1 -- 172.21.15.7:0/2563441449 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcc0c0050d0 con 0x7fcc1c0d0b40 2024-04-25T11:12:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.858+0000 7fcc13e00640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbf80744e0 0x7fcbf80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.858+0000 7fcc13e00640 1 -- 172.21.15.7:0/2563441449 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fcc0c094ee0 con 0x7fcc1c0d0b40 2024-04-25T11:12:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.858+0000 7fcc1be00640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbf80744e0 0x7fcbf80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.859+0000 7fcc1be00640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbf80744e0 0x7fcbf80769f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc04034000 tx=0x7fcc0402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:28.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:28.863+0000 7fcc13e00640 1 -- 172.21.15.7:0/2563441449 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcc0c020020 con 0x7fcc1c0d0b40 2024-04-25T11:12:29.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.068+0000 7fcc23200640 1 -- 172.21.15.7:0/2563441449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) -- 0x7fcc1c070fb0 con 0x7fcc1c0d0b40 2024-04-25T11:12:29.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.074+0000 7fcc13e00640 1 -- 172.21.15.7:0/2563441449 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v11)=0 v11) ==== 121+0+0 (secure 0 0 0) 0x7fcc0c05f560 con 0x7fcc1c0d0b40 2024-04-25T11:12:29.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.077+0000 7fcc23200640 1 -- 172.21.15.7:0/2563441449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbf80744e0 msgr2=0x7fcbf80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:29.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.077+0000 7fcc23200640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbf80744e0 0x7fcbf80769f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc04034000 tx=0x7fcc0402e000 comp rx=0 tx=0).stop 2024-04-25T11:12:29.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.077+0000 7fcc23200640 1 -- 172.21.15.7:0/2563441449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1c0d0b40 msgr2=0x7fcc1c1594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:29.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.077+0000 7fcc23200640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1c0d0b40 0x7fcc1c1594d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00c6b0 tx=0x7fcc0c00cb70 comp rx=0 tx=0).stop 2024-04-25T11:12:29.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.077+0000 7fcc23200640 1 -- 172.21.15.7:0/2563441449 shutdown_connections 2024-04-25T11:12:29.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.077+0000 7fcc23200640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbf80744e0 0x7fcbf80769f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.077+0000 7fcc23200640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc1c15dfb0 0x7fcc1c15c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.077+0000 7fcc23200640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1c0d0b40 0x7fcc1c1594d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.077+0000 7fcc23200640 1 --2- 172.21.15.7:0/2563441449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcc1c06fab0 0x7fcc1c1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.077+0000 7fcc23200640 1 -- 172.21.15.7:0/2563441449 >> 172.21.15.7:0/2563441449 conn(0x7fcc1c0742c0 msgr2=0x7fcc1c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:29.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.078+0000 7fcc23200640 1 -- 172.21.15.7:0/2563441449 shutdown_connections 2024-04-25T11:12:29.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.078+0000 7fcc23200640 1 -- 172.21.15.7:0/2563441449 wait complete. 2024-04-25T11:12:29.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_session_cap_acquisition_throttle 37 2024-04-25T11:12:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.398+0000 7f82d9800640 1 Processor -- start 2024-04-25T11:12:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.398+0000 7f82d9800640 1 -- start start 2024-04-25T11:12:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.400+0000 7f82d9800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d41583c0 0x7f82d40cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.400+0000 7f82d9800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f82d40cfb50 0x7f82d40d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:29.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.400+0000 7f82d9800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82d40d09c0 0x7f82d406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.400+0000 7f82d9800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f82d415f190 con 0x7f82d40d09c0 2024-04-25T11:12:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.400+0000 7f82d9800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f82d40ce570 con 0x7f82d40cfb50 2024-04-25T11:12:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.400+0000 7f82d9800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82d40ce850 con 0x7f82d41583c0 2024-04-25T11:12:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.400+0000 7f82d8e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82d40d09c0 0x7f82d406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.400+0000 7f82d8e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82d40d09c0 0x7f82d406eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54196/0 (socket says 172.21.15.7:54196) 2024-04-25T11:12:29.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.400+0000 7f82d8e00640 1 -- 172.21.15.7:0/2621843606 learned_addr learned my addr 172.21.15.7:0/2621843606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.401+0000 7f82d3e00640 1 --2- 172.21.15.7:0/2621843606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d41583c0 0x7f82d40cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.401+0000 7f82d8e00640 1 -- 172.21.15.7:0/2621843606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d41583c0 msgr2=0x7f82d40cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.401+0000 7f82d8e00640 1 --2- 172.21.15.7:0/2621843606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d41583c0 0x7f82d40cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.401+0000 7f82d8e00640 1 -- 172.21.15.7:0/2621843606 >> v2:172.21.15.87:3300/0 conn(0x7f82d40cfb50 msgr2=0x7f82d40d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:12:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.401+0000 7f82d8e00640 1 --2- 172.21.15.7:0/2621843606 >> v2:172.21.15.87:3300/0 conn(0x7f82d40cfb50 0x7f82d40d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.401+0000 7f82d8e00640 1 -- 172.21.15.7:0/2621843606 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d40ceb30 con 0x7f82d40d09c0 2024-04-25T11:12:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.402+0000 7f82d8e00640 1 --2- 172.21.15.7:0/2621843606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d40d09c0 0x7f82d406eea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f82d406e0f0 tx=0x7f82cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60e338244792119 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.403+0000 7f82d2a00640 1 -- 172.21.15.7:0/2621843606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc036070 con 0x7f82d40d09c0 2024-04-25T11:12:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.403+0000 7f82d2a00640 1 -- 172.21.15.7:0/2621843606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82cc031600 con 0x7f82d40d09c0 2024-04-25T11:12:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.403+0000 7f82d2a00640 1 -- 172.21.15.7:0/2621843606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0319d0 con 0x7f82d40d09c0 2024-04-25T11:12:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.404+0000 7f82d9800640 1 -- 172.21.15.7:0/2621843606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d40d09c0 msgr2=0x7f82d406eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.404+0000 7f82d9800640 1 --2- 172.21.15.7:0/2621843606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d40d09c0 0x7f82d406eea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f82d406e0f0 tx=0x7f82cc02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.404+0000 7f82d9800640 1 -- 172.21.15.7:0/2621843606 shutdown_connections 2024-04-25T11:12:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.404+0000 7f82d9800640 1 --2- 172.21.15.7:0/2621843606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d40d09c0 0x7f82d406eea0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.404+0000 7f82d9800640 1 --2- 172.21.15.7:0/2621843606 >> v2:172.21.15.87:3300/0 conn(0x7f82d40cfb50 0x7f82d40d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.404+0000 7f82d9800640 1 --2- 172.21.15.7:0/2621843606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d41583c0 0x7f82d40cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.404+0000 7f82d9800640 1 -- 172.21.15.7:0/2621843606 >> 172.21.15.7:0/2621843606 conn(0x7f82d40740f0 msgr2=0x7f82d406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.404+0000 7f82d9800640 1 -- 172.21.15.7:0/2621843606 shutdown_connections 2024-04-25T11:12:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.404+0000 7f82d9800640 1 -- 172.21.15.7:0/2621843606 wait complete. 2024-04-25T11:12:29.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.406+0000 7f82d9800640 1 Processor -- start 2024-04-25T11:12:29.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.407+0000 7f82d9800640 1 -- start start 2024-04-25T11:12:29.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.408+0000 7f82d9800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d40cfb50 0x7f82d41f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:29.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.408+0000 7f82d9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d40d09c0 0x7f82d414fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:29.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.408+0000 7f82d9800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d41583c0 0x7f82d415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:29.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.408+0000 7f82d9800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82d406f3e0 con 0x7f82d40d09c0 2024-04-25T11:12:29.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.408+0000 7f82d9800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82d40cf0f0 con 0x7f82d41583c0 2024-04-25T11:12:29.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.408+0000 7f82d9800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82d406f560 con 0x7f82d40cfb50 2024-04-25T11:12:29.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.408+0000 7f82d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d40d09c0 0x7f82d414fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:29.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.409+0000 7f82d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d40d09c0 0x7f82d414fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54212/0 (socket says 172.21.15.7:54212) 2024-04-25T11:12:29.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.409+0000 7f82d3400640 1 -- 172.21.15.7:0/788943609 learned_addr learned my addr 172.21.15.7:0/788943609 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:12:29.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.409+0000 7f82d3e00640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d40cfb50 0x7f82d41f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:29.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.409+0000 7f82d8e00640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d41583c0 0x7f82d415a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:29.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.409+0000 7f82d3400640 1 -- 172.21.15.7:0/788943609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d40cfb50 msgr2=0x7f82d41f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:29.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.409+0000 7f82d3400640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d40cfb50 0x7f82d41f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.409+0000 7f82d3400640 1 -- 172.21.15.7:0/788943609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d41583c0 msgr2=0x7f82d415a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:29.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.409+0000 7f82d3400640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d41583c0 0x7f82d415a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.409+0000 7f82d3400640 1 -- 172.21.15.7:0/788943609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d41500a0 con 0x7f82d40d09c0 2024-04-25T11:12:29.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.409+0000 7f82d3400640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d40d09c0 0x7f82d414fb60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00b410 tx=0x7f82bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:29.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.410+0000 7f82cbe00640 1 -- 172.21.15.7:0/788943609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc015020 con 0x7f82d40d09c0 2024-04-25T11:12:29.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.410+0000 7f82cbe00640 1 -- 172.21.15.7:0/788943609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc011850 con 0x7f82d40d09c0 2024-04-25T11:12:29.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.410+0000 7f82cbe00640 1 -- 172.21.15.7:0/788943609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc011c40 con 0x7f82d40d09c0 2024-04-25T11:12:29.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.410+0000 7f82d9800640 1 -- 172.21.15.7:0/788943609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d41e7790 con 0x7f82d40d09c0 2024-04-25T11:12:29.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.410+0000 7f82d9800640 1 -- 172.21.15.7:0/788943609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d41e7d50 con 0x7f82d40d09c0 2024-04-25T11:12:29.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.411+0000 7f82d9800640 1 -- 172.21.15.7:0/788943609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d41595d0 con 0x7f82d40d09c0 2024-04-25T11:12:29.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.412+0000 7f82cbe00640 1 -- 172.21.15.7:0/788943609 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f82bc018020 con 0x7f82d40d09c0 2024-04-25T11:12:29.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.414+0000 7f82cbe00640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82b00744e0 0x7f82b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:12:29.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.414+0000 7f82cbe00640 1 -- 172.21.15.7:0/788943609 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f82bc0952f0 con 0x7f82d40d09c0 2024-04-25T11:12:29.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.414+0000 7f82d3e00640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82b00744e0 0x7f82b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:12:29.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.415+0000 7f82d3e00640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82b00744e0 0x7f82b00769f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f82c4006fd0 tx=0x7f82c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:12:29.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.419+0000 7f82cbe00640 1 -- 172.21.15.7:0/788943609 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82d41595d0 con 0x7f82d40d09c0 2024-04-25T11:12:29.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.616+0000 7f82d9800640 1 -- 172.21.15.7:0/788943609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_session_cap_acquisition_throttle}] v 0) -- 0x7f82d4050740 con 0x7f82d40d09c0 2024-04-25T11:12:29.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.622+0000 7f82cbe00640 1 -- 172.21.15.7:0/788943609 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_session_cap_acquisition_throttle}]=0 v12)=0 v12) ==== 135+0+0 (secure 0 0 0) 0x7f82bc01f020 con 0x7f82d40d09c0 2024-04-25T11:12:29.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 -- 172.21.15.7:0/788943609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82b00744e0 msgr2=0x7f82b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:29.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82b00744e0 0x7f82b00769f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f82c4006fd0 tx=0x7f82c4008040 comp rx=0 tx=0).stop 2024-04-25T11:12:29.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 -- 172.21.15.7:0/788943609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d40d09c0 msgr2=0x7f82d414fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:12:29.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d40d09c0 0x7f82d414fb60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00b410 tx=0x7f82bc00b8d0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 -- 172.21.15.7:0/788943609 shutdown_connections 2024-04-25T11:12:29.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82b00744e0 0x7f82b00769f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d41583c0 0x7f82d415a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82d40d09c0 0x7f82d414fb60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 --2- 172.21.15.7:0/788943609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d40cfb50 0x7f82d41f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:12:29.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 -- 172.21.15.7:0/788943609 >> 172.21.15.7:0/788943609 conn(0x7f82d40740f0 msgr2=0x7f82d415f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:12:29.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 -- 172.21.15.7:0/788943609 shutdown_connections 2024-04-25T11:12:29.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:29.625+0000 7f82d9800640 1 -- 172.21.15.7:0/788943609 wait complete. 2024-04-25T11:12:29.651 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:12:29.651 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:12:29.704 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-04-25T11:12:29.704 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-04-25T11:12:29.704 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2024-04-25T11:12:29.704 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:29.704 DEBUG:teuthology.orchestra.run.smithi007:> n = 25 2024-04-25T11:12:29.704 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2024-04-25T11:12:29.704 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2024-04-25T11:12:29.704 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/dir1/file" 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> if True: 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-04-25T11:12:29.705 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2024-04-25T11:12:29.706 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-04-25T11:12:30.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:12:30.305 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:12:30.305 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:12:30.406 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-04-25T11:12:30.406 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-04-25T11:12:30.406 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2024-04-25T11:12:30.406 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:30.406 DEBUG:teuthology.orchestra.run.smithi007:> n = 25 2024-04-25T11:12:30.406 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2024-04-25T11:12:30.406 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2024-04-25T11:12:30.406 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2024-04-25T11:12:30.406 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/dir2/file" 2024-04-25T11:12:30.406 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:30.406 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> if True: 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2024-04-25T11:12:30.407 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:30.408 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2024-04-25T11:12:30.408 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2024-04-25T11:12:30.408 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2024-04-25T11:12:30.408 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T11:12:30.408 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:30.408 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2024-04-25T11:12:30.408 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2024-04-25T11:12:30.408 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2024-04-25T11:12:30.408 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-04-25T11:12:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:12:30.972 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:12:30.972 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:12:31.073 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> n = 25 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/dir3/file" 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:31.074 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> if True: 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T11:12:31.075 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:31.076 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2024-04-25T11:12:31.076 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2024-04-25T11:12:31.076 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2024-04-25T11:12:31.076 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-04-25T11:12:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:12:31.629 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:12:31.629 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:12:31.727 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-04-25T11:12:31.727 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-04-25T11:12:31.727 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2024-04-25T11:12:31.727 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:31.727 DEBUG:teuthology.orchestra.run.smithi007:> n = 25 2024-04-25T11:12:31.727 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2024-04-25T11:12:31.727 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2024-04-25T11:12:31.727 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2024-04-25T11:12:31.727 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/dir4/file" 2024-04-25T11:12:31.727 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> if True: 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2024-04-25T11:12:31.728 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:31.729 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2024-04-25T11:12:31.729 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:31.729 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2024-04-25T11:12:31.729 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2024-04-25T11:12:31.729 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2024-04-25T11:12:31.729 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T11:12:31.729 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:12:31.729 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2024-04-25T11:12:31.729 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2024-04-25T11:12:31.729 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2024-04-25T11:12:31.729 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-04-25T11:12:32.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:12:32.304 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:12:32.304 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:12:32.404 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:12:32.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:12:32.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:12:32.534 INFO:teuthology.orchestra.run.smithi007.stdout:client4862 2024-04-25T11:12:32.535 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'find . | wc') 2024-04-25T11:12:32.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:12:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:12:41.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:12:41.326 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:13:11.458 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:13:11.466 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:13:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: 105 105 1490 2024-04-25T11:13:11.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:13:11.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:13:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:11.998+0000 7f8793200640 1 Processor -- start 2024-04-25T11:13:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:11.998+0000 7f8793200640 1 -- start start 2024-04-25T11:13:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.000+0000 7f8793200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87940aed40 0x7f879415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.000+0000 7f8793200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87940b5220 0x7f87940ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.000+0000 7f8793200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87940b64b0 con 0x7f87940af420 2024-04-25T11:13:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.000+0000 7f8793200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f879415e1f0 con 0x7f87940b5220 2024-04-25T11:13:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.000+0000 7f8793200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f879406c610 con 0x7f87940aed40 2024-04-25T11:13:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.001+0000 7f8789400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87940b5220 0x7f87940ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.001+0000 7f8789400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87940b5220 0x7f87940ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50408/0 (socket says 172.21.15.7:50408) 2024-04-25T11:13:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.001+0000 7f8789400640 1 -- 172.21.15.7:0/3779096513 learned_addr learned my addr 172.21.15.7:0/3779096513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.001+0000 7f8791e00640 1 --2- 172.21.15.7:0/3779096513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87940aed40 0x7f879415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.001+0000 7f8789400640 1 -- 172.21.15.7:0/3779096513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87940aed40 msgr2=0x7f879415dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.001+0000 7f8789400640 1 --2- 172.21.15.7:0/3779096513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87940aed40 0x7f879415dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.001+0000 7f8789400640 1 -- 172.21.15.7:0/3779096513 >> v1:172.21.15.7:6789/0 conn(0x7f87940af420 legacy=0x7f87940af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.001+0000 7f8789400640 1 -- 172.21.15.7:0/3779096513 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879406c8f0 con 0x7f87940b5220 2024-04-25T11:13:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.002+0000 7f8789400640 1 --2- 172.21.15.7:0/3779096513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87940b5220 0x7f87940ce4a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f877c009870 tx=0x7f877c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff1d2d37acca1d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.003+0000 7f8791400640 1 -- 172.21.15.7:0/3779096513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c037070 con 0x7f87940b5220 2024-04-25T11:13:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.003+0000 7f8791400640 1 -- 172.21.15.7:0/3779096513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f877c032440 con 0x7f87940b5220 2024-04-25T11:13:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.003+0000 7f8791400640 1 -- 172.21.15.7:0/3779096513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c032810 con 0x7f87940b5220 2024-04-25T11:13:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.003+0000 7f8793200640 1 -- 172.21.15.7:0/3779096513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87940b5220 msgr2=0x7f87940ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.003+0000 7f8793200640 1 --2- 172.21.15.7:0/3779096513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87940b5220 0x7f87940ce4a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f877c009870 tx=0x7f877c02a270 comp rx=0 tx=0).stop 2024-04-25T11:13:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.004+0000 7f8793200640 1 -- 172.21.15.7:0/3779096513 shutdown_connections 2024-04-25T11:13:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.004+0000 7f8793200640 1 --2- 172.21.15.7:0/3779096513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87940b5220 0x7f87940ce4a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.004+0000 7f8793200640 1 --2- 172.21.15.7:0/3779096513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87940aed40 0x7f879415dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.004+0000 7f8793200640 1 -- 172.21.15.7:0/3779096513 >> 172.21.15.7:0/3779096513 conn(0x7f87940b3820 msgr2=0x7f87940b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.004+0000 7f8793200640 1 -- 172.21.15.7:0/3779096513 shutdown_connections 2024-04-25T11:13:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.004+0000 7f8793200640 1 -- 172.21.15.7:0/3779096513 wait complete. 2024-04-25T11:13:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.007+0000 7f8793200640 1 Processor -- start 2024-04-25T11:13:12.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.007+0000 7f8793200640 1 -- start start 2024-04-25T11:13:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.009+0000 7f8793200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87940aed40 0x7f87941424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.009+0000 7f8793200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87940af420 0x7f879415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8793200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87940b5220 0x7f879415fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8793200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87940b6110 con 0x7f87940aed40 2024-04-25T11:13:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8793200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f879406ceb0 con 0x7f87940b5220 2024-04-25T11:13:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8793200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87940b6290 con 0x7f87940af420 2024-04-25T11:13:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8791e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87940aed40 0x7f87941424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8791e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87940aed40 0x7f87941424d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50414/0 (socket says 172.21.15.7:50414) 2024-04-25T11:13:12.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8791e00640 1 -- 172.21.15.7:0/476325669 learned_addr learned my addr 172.21.15.7:0/476325669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:12.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8791e00640 1 -- 172.21.15.7:0/476325669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87940af420 msgr2=0x7f879415f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8791e00640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87940af420 0x7f879415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8791e00640 1 -- 172.21.15.7:0/476325669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87940b5220 msgr2=0x7f879415fc50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T11:13:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8791e00640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87940b5220 0x7f879415fc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.010+0000 7f8791e00640 1 -- 172.21.15.7:0/476325669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879415ad00 con 0x7f87940aed40 2024-04-25T11:13:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.011+0000 7f8791e00640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87940aed40 0x7f87941424d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f878400c820 tx=0x7f878400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.011+0000 7f878aa00640 1 -- 172.21.15.7:0/476325669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784004030 con 0x7f87940aed40 2024-04-25T11:13:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.011+0000 7f878aa00640 1 -- 172.21.15.7:0/476325669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87840041d0 con 0x7f87940aed40 2024-04-25T11:13:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.011+0000 7f878aa00640 1 -- 172.21.15.7:0/476325669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784007830 con 0x7f87940aed40 2024-04-25T11:13:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.011+0000 7f8793200640 1 -- 172.21.15.7:0/476325669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879415e950 con 0x7f87940aed40 2024-04-25T11:13:12.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.011+0000 7f8793200640 1 -- 172.21.15.7:0/476325669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879415ef10 con 0x7f87940aed40 2024-04-25T11:13:12.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.013+0000 7f8793200640 1 -- 172.21.15.7:0/476325669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8794157720 con 0x7f87940aed40 2024-04-25T11:13:12.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.014+0000 7f878aa00640 1 -- 172.21.15.7:0/476325669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f87840079d0 con 0x7f87940aed40 2024-04-25T11:13:12.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.015+0000 7f878aa00640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87700744b0 0x7f87700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:12.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.015+0000 7f878aa00640 1 -- 172.21.15.7:0/476325669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f8784095370 con 0x7f87940aed40 2024-04-25T11:13:12.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.015+0000 7f8789400640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87700744b0 0x7f87700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:12.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.016+0000 7f8789400640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87700744b0 0x7f87700769c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f877c034000 tx=0x7f877c02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:12.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.020+0000 7f878aa00640 1 -- 172.21.15.7:0/476325669 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8784015020 con 0x7f87940aed40 2024-04-25T11:13:12.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.265+0000 7f8793200640 1 -- 172.21.15.7:0/476325669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8794142a10 con 0x7f87940aed40 2024-04-25T11:13:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.267+0000 7f878aa00640 1 -- 172.21.15.7:0/476325669 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5172 (secure 0 0 0) 0x7f878405f940 con 0x7f87940aed40 2024-04-25T11:13:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:12.267 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2575517671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2575517671},{"type":"v1","addr":"172.21.15.87:6835","nonce":2575517671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/3746682343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3746682343},{"type":"v1","addr":"172.21.15.7:6836","nonce":3746682343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2132415318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2132415318},{"type":"v1","addr":"172.21.15.7:6837","nonce":2132415318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:12:17.156571+0000","modified":"2024-04-25T11:12:20.224854+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6833/389115631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":389115631},{"type":"v1","addr":"172.21.15.87:6833","nonce":389115631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5283,"qdb_cluster":[5283]},"id":2}]} 2024-04-25T11:13:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.269+0000 7f8793200640 1 -- 172.21.15.7:0/476325669 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87700744b0 msgr2=0x7f87700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.269+0000 7f8793200640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87700744b0 0x7f87700769c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f877c034000 tx=0x7f877c02c000 comp rx=0 tx=0).stop 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.269+0000 7f8793200640 1 -- 172.21.15.7:0/476325669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87940aed40 msgr2=0x7f87941424d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.269+0000 7f8793200640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87940aed40 0x7f87941424d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f878400c820 tx=0x7f878400cce0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.269+0000 7f8793200640 1 -- 172.21.15.7:0/476325669 shutdown_connections 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.269+0000 7f8793200640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87700744b0 0x7f87700769c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.269+0000 7f8793200640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87940b5220 0x7f879415fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.269+0000 7f8793200640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87940af420 0x7f879415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.269+0000 7f8793200640 1 --2- 172.21.15.7:0/476325669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87940aed40 0x7f87941424d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.269+0000 7f8793200640 1 -- 172.21.15.7:0/476325669 >> 172.21.15.7:0/476325669 conn(0x7f87940b3820 msgr2=0x7f87940af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.270+0000 7f8793200640 1 -- 172.21.15.7:0/476325669 shutdown_connections 2024-04-25T11:13:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.270+0000 7f8793200640 1 -- 172.21.15.7:0/476325669 wait complete. 2024-04-25T11:13:12.297 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json perf dump 2024-04-25T11:13:12.606 INFO:teuthology.orchestra.run.smithi087.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":147,"msgr_send_messages":99,"msgr_recv_bytes":152223,"msgr_send_bytes":39058,"msgr_created_connections":7,"msgr_active_connections":3,"msgr_running_total_time":0.040611193,"msgr_running_send_time":0.013614285,"msgr_running_recv_time":0.024451796,"msgr_running_fast_dispatch_time":0.005357884,"msgr_send_messages_queue_lat":{"avgcount":99,"sum":0.115434085,"avgtime":0.001166000},"msgr_handle_ack_lat":{"avgcount":64,"sum":0.000024789,"avgtime":0.000000387},"msgr_recv_encrypted_bytes":126112,"msgr_send_encrypted_bytes":19552},"AsyncMessenger::Worker-1":{"msgr_recv_messages":11,"msgr_send_messages":26,"msgr_recv_bytes":2872,"msgr_send_bytes":84414,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.008421019,"msgr_running_send_time":0.001883972,"msgr_running_recv_time":0.005539082,"msgr_running_fast_dispatch_time":0.000369984,"msgr_send_messages_queue_lat":{"avgcount":24,"sum":0.092793284,"avgtime":0.003866386},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":79760},"AsyncMessenger::Worker-2":{"msgr_recv_messages":522,"msgr_send_messages":594,"msgr_recv_bytes":149282,"msgr_send_bytes":764147,"msgr_created_connections":8,"msgr_active_connections":7,"msgr_running_total_time":0.138437394,"msgr_running_send_time":0.056758918,"msgr_running_recv_time":0.080158993,"msgr_running_fast_dispatch_time":0.011423627,"msgr_send_messages_queue_lat":{"avgcount":593,"sum":0.145072893,"avgtime":0.000244642},"msgr_handle_ack_lat":{"avgcount":842,"sum":0.000665458,"avgtime":0.000000790},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":270,"sum":0.156680835,"avgtime":0.000580299}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":2,"sum":0.000025286,"avgtime":0.000012643}},"mds":{"request":148,"reply":148,"reply_latency":{"avgcount":148,"sum":39.233897484,"avgtime":0.265093901},"slow_reply":0,"forward":0,"dir_fetch_complete":0,"dir_fetch_keys":0,"dir_commit":12,"dir_split":0,"dir_merge":0,"inodes":114,"inodes_top":0,"inodes_bottom":114,"inodes_pin_tail":0,"inodes_pinned":114,"inodes_expired":0,"inodes_with_caps":100,"caps":101,"subtrees":2,"traverse":188,"traverse_hit":174,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":100,"root_rbytes":160,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":4,"ceph_cap_op_grant":112,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":104,"handle_client_caps_dirty":100,"handle_client_cap_release":1,"process_request_cap_release":14},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_quiesce_path":0,"ireq_quiesce_inode":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evlrg":0,"evadd":219,"evex":0,"evtrm":0,"ev":219,"evexg":0,"evexd":0,"segadd":2,"segex":0,"segtrm":0,"seg":2,"segmjr":2,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4640660,"rdpos":0,"jlat":{"avgcount":110,"sum":0.579225889,"avgtime":0.005265689},"replayed":0},"mds_mem":{"ino":117,"ino+":117,"ino-":0,"dir":16,"dir+":16,"dir-":0,"dn":114,"dn+":115,"dn-":1,"cap":101,"cap+":106,"cap-":5,"rss":116992,"heap":231804},"mds_server":{"dispatch_client_request":230,"dispatch_server_request":0,"handle_client_request":148,"handle_client_session":106,"handle_peer_request":0,"req_create_latency":{"avgcount":100,"sum":0.170685808,"avgtime":0.001706858},"req_getattr_latency":{"avgcount":32,"sum":0.014378773,"avgtime":0.000449336},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":4,"sum":0.001313161,"avgtime":0.000328290},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":4,"sum":0.016602850,"avgtime":0.004150712},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":2,"sum":0.000504899,"avgtime":0.000252449},"req_readdir_latency":{"avgcount":5,"sum":39.029761087,"avgtime":7.805952217},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":1,"sum":0.000650906,"avgtime":0.000650906},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":78,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":2,"session_add":2,"session_remove":0,"sessions_open":2,"sessions_stale":0,"total_load":92,"average_load":46,"avg_session_uptime":40,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":249008,"buffer_anon_items":218,"buffer_meta_bytes":1584,"buffer_meta_items":18,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5520,"osdmap_items":78,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":420136,"mds_co_items":4385,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":153,"op_send_bytes":464235,"op_resend":37,"op_reply":153,"op_latency":{"avgcount":153,"sum":1.909957864,"avgtime":0.012483384},"op_inflight":0,"oplen_avg":{"avgcount":153,"sum":181},"op":153,"op_r":0,"op_w":153,"op_rmw":0,"op_pg":0,"osdop_stat":0,"osdop_create":11,"osdop_read":0,"osdop_write":110,"osdop_writefull":8,"osdop_writesame":0,"osdop_append":0,"osdop_zero":0,"osdop_truncate":0,"osdop_delete":6,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":0,"osdop_setxattr":22,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":42,"map_full":0,"map_inc":3,"osd_sessions":8,"osd_session_open":8,"osd_session_close":0,"osd_laggy":0,"omap_wr":22,"omap_rd":0,"omap_del":2},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":2},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":676,"get_sum":238525,"get_or_fail_fail":0,"get_or_fail_success":676,"take":0,"take_sum":0,"put":676,"put_sum":238525,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":153,"take_sum":464235,"put":153,"put_sum":464235,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":153,"take_sum":153,"put":153,"put_sum":153,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x560cfb19c750":{"val":0,"max":3758096384,"get_started":0,"get":219,"get_sum":446356,"get_or_fail_fail":0,"get_or_fail_success":219,"take":0,"take_sum":0,"put":110,"put_sum":446356,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-04-25T11:13:12.607 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 3, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 64, "avgtime": 3.87e-07, "sum": 2.4789e-05 }, "msgr_recv_bytes": 152223, "msgr_recv_encrypted_bytes": 126112, "msgr_recv_messages": 147, "msgr_running_fast_dispatch_time": 0.005357884, "msgr_running_recv_time": 0.024451796, "msgr_running_send_time": 0.013614285, "msgr_running_total_time": 0.040611193, "msgr_send_bytes": 39058, "msgr_send_encrypted_bytes": 19552, "msgr_send_messages": 99, "msgr_send_messages_queue_lat": { "avgcount": 99, "avgtime": 0.001166, "sum": 0.115434085 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 2872, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 11, "msgr_running_fast_dispatch_time": 0.000369984, "msgr_running_recv_time": 0.005539082, "msgr_running_send_time": 0.001883972, "msgr_running_total_time": 0.008421019, "msgr_send_bytes": 84414, "msgr_send_encrypted_bytes": 79760, "msgr_send_messages": 26, "msgr_send_messages_queue_lat": { "avgcount": 24, "avgtime": 0.003866386, "sum": 0.092793284 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 7, "msgr_created_connections": 8, "msgr_handle_ack_lat": { "avgcount": 842, "avgtime": 7.9e-07, "sum": 0.000665458 }, "msgr_recv_bytes": 149282, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 522, "msgr_running_fast_dispatch_time": 0.011423627, "msgr_running_recv_time": 0.080158993, "msgr_running_send_time": 0.056758918, "msgr_running_total_time": 0.138437394, "msgr_send_bytes": 764147, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 594, "msgr_send_messages_queue_lat": { "avgcount": 593, "avgtime": 0.000244642, "sum": 0.145072893 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 270, "avgtime": 0.000580299, "sum": 0.156680835 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 2, "avgtime": 1.2643e-05, "sum": 2.5286e-05 }, "queue_len": 0 }, "mds": { "caps": 101, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 112, "ceph_cap_op_revoke": 4, "ceph_cap_op_trunc": 0, "dir_commit": 12, "dir_fetch_complete": 0, "dir_fetch_keys": 0, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 1, "handle_client_caps": 104, "handle_client_caps_dirty": 100, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 114, "inodes_bottom": 114, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 114, "inodes_top": 0, "inodes_with_caps": 100, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 14, "q": 0, "reply": 148, "reply_latency": { "avgcount": 148, "avgtime": 0.265093901, "sum": 39.233897484 }, "request": 148, "root_rbytes": 160, "root_rfiles": 100, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 188, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 174, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "ireq_quiesce_inode": 0, "ireq_quiesce_path": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 219, "evadd": 219, "evex": 0, "evexd": 0, "evexg": 0, "evlrg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 110, "avgtime": 0.005265689, "sum": 0.579225889 }, "rdpos": 0, "replayed": 0, "seg": 2, "segadd": 2, "segex": 0, "segexd": 0, "segexg": 0, "segmjr": 2, "segtrm": 0, "wrpos": 4640660 }, "mds_mem": { "cap": 101, "cap+": 106, "cap-": 5, "dir": 16, "dir+": 16, "dir-": 0, "dn": 114, "dn+": 115, "dn-": 1, "heap": 231804, "ino": 117, "ino+": 117, "ino-": 0, "rss": 116992 }, "mds_server": { "cap_acquisition_throttle": 78, "cap_revoke_eviction": 0, "dispatch_client_request": 230, "dispatch_server_request": 0, "handle_client_request": 148, "handle_client_session": 106, "handle_peer_request": 0, "req_create_latency": { "avgcount": 100, "avgtime": 0.001706858, "sum": 0.170685808 }, "req_getattr_latency": { "avgcount": 32, "avgtime": 0.000449336, "sum": 0.014378773 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 4, "avgtime": 0.00032829, "sum": 0.001313161 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 4, "avgtime": 0.004150712, "sum": 0.01660285 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 2, "avgtime": 0.000252449, "sum": 0.000504899 }, "req_readdir_latency": { "avgcount": 5, "avgtime": 7.805952217, "sum": 39.029761087 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 1, "avgtime": 0.000650906, "sum": 0.000650906 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 46, "avg_session_uptime": 40, "mdthresh_evicted": 0, "session_add": 2, "session_count": 2, "session_remove": 0, "sessions_open": 2, "sessions_stale": 0, "total_load": 92 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 249008, "buffer_anon_items": 218, "buffer_meta_bytes": 1584, "buffer_meta_items": 18, "mds_co_bytes": 420136, "mds_co_items": 4385, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5520, "osdmap_items": 78, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 42, "map_full": 0, "map_inc": 3, "omap_del": 2, "omap_rd": 0, "omap_wr": 22, "op": 153, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 153, "avgtime": 0.012483384, "sum": 1.909957864 }, "op_pg": 0, "op_r": 0, "op_reply": 153, "op_resend": 37, "op_rmw": 0, "op_send": 153, "op_send_bytes": 464235, "op_w": 153, "oplen_avg": { "avgcount": 153, "sum": 181 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 8, "osd_sessions": 8, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 11, "osdop_delete": 6, "osdop_getxattr": 0, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 0, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 22, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 0, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 110, "osdop_writefull": 8, "osdop_writesame": 0, "osdop_zero": 0, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 1, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 2 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 676, "get_or_fail_fail": 0, "get_or_fail_success": 676, "get_started": 0, "get_sum": 238525, "max": 104857600, "put": 676, "put_sum": 238525, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 153, "put_sum": 464235, "take": 153, "take_sum": 464235, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 153, "put_sum": 153, "take": 153, "take_sum": 153, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x560cfb19c750": { "get": 219, "get_or_fail_fail": 0, "get_or_fail_success": 219, "get_started": 0, "get_sum": 446356, "max": 3758096384, "put": 110, "put_sum": 446356, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-04-25T11:13:12.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:13:12.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.887+0000 7ff56a400640 1 Processor -- start 2024-04-25T11:13:12.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.887+0000 7ff56a400640 1 -- start start 2024-04-25T11:13:12.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.890+0000 7ff56a400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5640b5220 0x7ff56415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:12.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.890+0000 7ff56a400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5640cebb0 con 0x7ff5640aed40 2024-04-25T11:13:12.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.890+0000 7ff56a400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5640b64b0 con 0x7ff5640af420 2024-04-25T11:13:12.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.890+0000 7ff56a400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff56415e1f0 con 0x7ff5640b5220 2024-04-25T11:13:12.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.890+0000 7ff55be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5640b5220 0x7ff56415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:12.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.891+0000 7ff55be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5640b5220 0x7ff56415dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50430/0 (socket says 172.21.15.7:50430) 2024-04-25T11:13:12.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.891+0000 7ff55be00640 1 -- 172.21.15.7:0/2321589055 learned_addr learned my addr 172.21.15.7:0/2321589055 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:12.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.891+0000 7ff55be00640 1 -- 172.21.15.7:0/2321589055 >> v1:172.21.15.87:6789/0 conn(0x7ff5640af420 legacy=0x7ff5640af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:12.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.891+0000 7ff55be00640 1 -- 172.21.15.7:0/2321589055 >> v1:172.21.15.7:6789/0 conn(0x7ff5640aed40 legacy=0x7ff5640ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:12.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.891+0000 7ff55be00640 1 -- 172.21.15.7:0/2321589055 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56406c610 con 0x7ff5640b5220 2024-04-25T11:13:12.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.892+0000 7ff55be00640 1 --2- 172.21.15.7:0/2321589055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b5220 0x7ff56415dcb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009870 tx=0x7ff54c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570baf0923f3d522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:12.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.892+0000 7ff563e00640 1 -- 172.21.15.7:0/2321589055 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c037070 con 0x7ff5640b5220 2024-04-25T11:13:12.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.892+0000 7ff563e00640 1 -- 172.21.15.7:0/2321589055 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff54c032440 con 0x7ff5640b5220 2024-04-25T11:13:12.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.893+0000 7ff563e00640 1 -- 172.21.15.7:0/2321589055 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c032810 con 0x7ff5640b5220 2024-04-25T11:13:12.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.893+0000 7ff56a400640 1 -- 172.21.15.7:0/2321589055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b5220 msgr2=0x7ff56415dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.893+0000 7ff56a400640 1 --2- 172.21.15.7:0/2321589055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b5220 0x7ff56415dcb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009870 tx=0x7ff54c02a2a0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.893+0000 7ff56a400640 1 -- 172.21.15.7:0/2321589055 shutdown_connections 2024-04-25T11:13:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.893+0000 7ff56a400640 1 --2- 172.21.15.7:0/2321589055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640b5220 0x7ff56415dcb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.894+0000 7ff56a400640 1 -- 172.21.15.7:0/2321589055 >> 172.21.15.7:0/2321589055 conn(0x7ff5640b3820 msgr2=0x7ff5640b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.894+0000 7ff56a400640 1 -- 172.21.15.7:0/2321589055 shutdown_connections 2024-04-25T11:13:12.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.894+0000 7ff56a400640 1 -- 172.21.15.7:0/2321589055 wait complete. 2024-04-25T11:13:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.896+0000 7ff56a400640 1 Processor -- start 2024-04-25T11:13:12.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.896+0000 7ff56a400640 1 -- start start 2024-04-25T11:13:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.897+0000 7ff56a400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640aed40 0x7ff564157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff56a400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5640af420 0x7ff56415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff56a400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5640b5220 0x7ff56415f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff56a400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff56406ceb0 con 0x7ff5640aed40 2024-04-25T11:13:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff56a400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff56406cbd0 con 0x7ff5640b5220 2024-04-25T11:13:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff56a400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5640b6110 con 0x7ff5640af420 2024-04-25T11:13:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff569000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640aed40 0x7ff564157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff569000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640aed40 0x7ff564157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50438/0 (socket says 172.21.15.7:50438) 2024-04-25T11:13:12.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff569000640 1 -- 172.21.15.7:0/1280070286 learned_addr learned my addr 172.21.15.7:0/1280070286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff569000640 1 -- 172.21.15.7:0/1280070286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5640af420 msgr2=0x7ff56415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff569000640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5640af420 0x7ff56415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff569000640 1 -- 172.21.15.7:0/1280070286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5640b5220 msgr2=0x7ff56415f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff569a00640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5640b5220 0x7ff56415f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff569000640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5640b5220 0x7ff56415f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.898+0000 7ff569000640 1 -- 172.21.15.7:0/1280070286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff564158480 con 0x7ff5640aed40 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.899+0000 7ff569a00640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5640b5220 0x7ff56415f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.899+0000 7ff569000640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640aed40 0x7ff564157f40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c470 tx=0x7ff55400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.899+0000 7ff561600640 1 -- 172.21.15.7:0/1280070286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5540052d0 con 0x7ff5640aed40 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.899+0000 7ff561600640 1 -- 172.21.15.7:0/1280070286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff554005490 con 0x7ff5640aed40 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.899+0000 7ff561600640 1 -- 172.21.15.7:0/1280070286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554007260 con 0x7ff5640aed40 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.899+0000 7ff56a400640 1 -- 172.21.15.7:0/1280070286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5640cc760 con 0x7ff5640aed40 2024-04-25T11:13:12.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.899+0000 7ff56a400640 1 -- 172.21.15.7:0/1280070286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5640ccd20 con 0x7ff5640aed40 2024-04-25T11:13:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.900+0000 7ff56a400640 1 -- 172.21.15.7:0/1280070286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5640ce6f0 con 0x7ff5640aed40 2024-04-25T11:13:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.902+0000 7ff561600640 1 -- 172.21.15.7:0/1280070286 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff554007400 con 0x7ff5640aed40 2024-04-25T11:13:12.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.903+0000 7ff561600640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5400744e0 0x7ff5400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:12.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.903+0000 7ff561600640 1 -- 172.21.15.7:0/1280070286 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7ff554094900 con 0x7ff5640aed40 2024-04-25T11:13:12.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.903+0000 7ff55be00640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5400744e0 0x7ff5400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:12.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.904+0000 7ff55be00640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5400744e0 0x7ff5400769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff54c034000 tx=0x7ff54c02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:12.908+0000 7ff561600640 1 -- 172.21.15.7:0/1280070286 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5640cebb0 con 0x7ff5640aed40 2024-04-25T11:13:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.150+0000 7ff56a400640 1 -- 172.21.15.7:0/1280070286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5640b1680 con 0x7ff5640aed40 2024-04-25T11:13:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.152+0000 7ff561600640 1 -- 172.21.15.7:0/1280070286 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5172 (secure 0 0 0) 0x7ff55405ee40 con 0x7ff5640aed40 2024-04-25T11:13:13.152 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:13.153 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2575517671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2575517671},{"type":"v1","addr":"172.21.15.87:6835","nonce":2575517671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/3746682343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3746682343},{"type":"v1","addr":"172.21.15.7:6836","nonce":3746682343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2132415318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2132415318},{"type":"v1","addr":"172.21.15.7:6837","nonce":2132415318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:12:17.156571+0000","modified":"2024-04-25T11:12:20.224854+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6833/389115631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":389115631},{"type":"v1","addr":"172.21.15.87:6833","nonce":389115631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5283,"qdb_cluster":[5283]},"id":2}]} 2024-04-25T11:13:13.153 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2024-04-25T11:13:13.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.154+0000 7ff56a400640 1 -- 172.21.15.7:0/1280070286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5400744e0 msgr2=0x7ff5400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.154+0000 7ff56a400640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5400744e0 0x7ff5400769f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff54c034000 tx=0x7ff54c02c000 comp rx=0 tx=0).stop 2024-04-25T11:13:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.154+0000 7ff56a400640 1 -- 172.21.15.7:0/1280070286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640aed40 msgr2=0x7ff564157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.154+0000 7ff56a400640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640aed40 0x7ff564157f40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c470 tx=0x7ff55400c930 comp rx=0 tx=0).stop 2024-04-25T11:13:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.155+0000 7ff56a400640 1 -- 172.21.15.7:0/1280070286 shutdown_connections 2024-04-25T11:13:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.155+0000 7ff56a400640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5400744e0 0x7ff5400769f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.155+0000 7ff56a400640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5640b5220 0x7ff56415f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.155+0000 7ff56a400640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5640af420 0x7ff56415ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.155+0000 7ff56a400640 1 --2- 172.21.15.7:0/1280070286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5640aed40 0x7ff564157f40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.155+0000 7ff56a400640 1 -- 172.21.15.7:0/1280070286 >> 172.21.15.7:0/1280070286 conn(0x7ff5640b3820 msgr2=0x7ff5640b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.155+0000 7ff56a400640 1 -- 172.21.15.7:0/1280070286 shutdown_connections 2024-04-25T11:13:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.155+0000 7ff56a400640 1 -- 172.21.15.7:0/1280070286 wait complete. 2024-04-25T11:13:13.182 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-25T11:13:13.490 INFO:teuthology.orchestra.run.smithi087.stdout:[{"id":4865,"entity":{"name":{"type":"client","num":4865},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":931265222}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":2,"uptime":45.182004161999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":8.1513475881859581,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.4865 192.168.0.1:0/931265222","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi087","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":4862,"entity":{"name":{"type":"client","num":4862},"addr":{"type":"any","addr":"172.21.15.7:0","nonce":2300151670}},"state":"open","num_leases":1,"num_caps":100,"request_load_avg":88,"uptime":46.968964776,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":9.4671474941542701,"halflife":60},"release_caps":{"value":3.2298682653499986,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":194.01856171888411,"halflife":300},"cap_acquisition":{"value":59.291635974640627,"halflife":30},"last_trim_completed_requests_tid":144,"last_trim_completed_flushes_tid":101,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.4862 172.21.15.7:0/2300151670","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000025c","length":899}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi007","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T11:13:13.491 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi087", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 931265222, "type": "any" }, "name": { "num": 4865, "type": "client" } }, "id": 4865, "inst": "client.4865 192.168.0.1:0/931265222", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 8.151347588185958 }, "state": "open", "uptime": 45.182004162 }, { "cap_acquisition": { "halflife": 30, "value": 59.29163597464063 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi007", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.7:0", "nonce": 2300151670, "type": "any" }, "name": { "num": 4862, "type": "client" } }, "id": 4862, "inst": "client.4862 172.21.15.7:0/2300151670", "last_trim_completed_flushes_tid": 101, "last_trim_completed_requests_tid": 144, "num_caps": 100, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 899, "start": "0x1000000025c" } ], "recall_caps": { "halflife": 60, "value": 9.46714749415427 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 3.2298682653499986 }, "request_load_avg": 88, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 194.01856171888411 }, "state": "open", "uptime": 46.968964776 } ] 2024-04-25T11:13:13.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump_historic_ops 2024-04-25T11:13:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.789+0000 7f8113200640 1 Processor -- start 2024-04-25T11:13:13.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.789+0000 7f8113200640 1 -- start start 2024-04-25T11:13:13.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.791+0000 7f8113200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81141583c0 0x7f81140cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.791+0000 7f8113200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140cfb50 0x7f81140d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.792+0000 7f8113200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81140ce570 con 0x7f81140d09c0 2024-04-25T11:13:13.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.792+0000 7f8113200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f811415f190 con 0x7f81141583c0 2024-04-25T11:13:13.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.792+0000 7f8113200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f81140ce850 con 0x7f81140cfb50 2024-04-25T11:13:13.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.792+0000 7f8111400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140cfb50 0x7f81140d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.792+0000 7f8111400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140cfb50 0x7f81140d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:39022/0 (socket says 172.21.15.7:39022) 2024-04-25T11:13:13.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.792+0000 7f8111400640 1 -- 172.21.15.7:0/3078045402 learned_addr learned my addr 172.21.15.7:0/3078045402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:13.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.793+0000 7f8111e00640 1 --2- 172.21.15.7:0/3078045402 >> v2:172.21.15.87:3300/0 conn(0x7f81141583c0 0x7f81140cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.793+0000 7f8111400640 1 -- 172.21.15.7:0/3078045402 >> v2:172.21.15.87:3300/0 conn(0x7f81141583c0 msgr2=0x7f81140cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:13.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.793+0000 7f8111400640 1 --2- 172.21.15.7:0/3078045402 >> v2:172.21.15.87:3300/0 conn(0x7f81141583c0 0x7f81140cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.793+0000 7f8111400640 1 -- 172.21.15.7:0/3078045402 >> v1:172.21.15.87:6789/0 conn(0x7f81140d09c0 legacy=0x7f811406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.793+0000 7f8111400640 1 -- 172.21.15.7:0/3078045402 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81140ceb30 con 0x7f81140cfb50 2024-04-25T11:13:13.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.795+0000 7f8111400640 1 --2- 172.21.15.7:0/3078045402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140cfb50 0x7f81140d0480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8100009870 tx=0x7f810002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a52550cfd52fbe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:13.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.795+0000 7f8110a00640 1 -- 172.21.15.7:0/3078045402 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100037070 con 0x7f81140cfb50 2024-04-25T11:13:13.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.795+0000 7f8110a00640 1 -- 172.21.15.7:0/3078045402 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8100032440 con 0x7f81140cfb50 2024-04-25T11:13:13.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.795+0000 7f8110a00640 1 -- 172.21.15.7:0/3078045402 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100032810 con 0x7f81140cfb50 2024-04-25T11:13:13.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.796+0000 7f8113200640 1 -- 172.21.15.7:0/3078045402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140cfb50 msgr2=0x7f81140d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.796+0000 7f8113200640 1 --2- 172.21.15.7:0/3078045402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140cfb50 0x7f81140d0480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8100009870 tx=0x7f810002a250 comp rx=0 tx=0).stop 2024-04-25T11:13:13.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.796+0000 7f8113200640 1 -- 172.21.15.7:0/3078045402 shutdown_connections 2024-04-25T11:13:13.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.796+0000 7f8113200640 1 --2- 172.21.15.7:0/3078045402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140cfb50 0x7f81140d0480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.796+0000 7f8113200640 1 --2- 172.21.15.7:0/3078045402 >> v2:172.21.15.87:3300/0 conn(0x7f81141583c0 0x7f81140cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.796+0000 7f8113200640 1 -- 172.21.15.7:0/3078045402 >> 172.21.15.7:0/3078045402 conn(0x7f81140740f0 msgr2=0x7f811406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:13.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.796+0000 7f8113200640 1 -- 172.21.15.7:0/3078045402 shutdown_connections 2024-04-25T11:13:13.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.796+0000 7f8113200640 1 -- 172.21.15.7:0/3078045402 wait complete. 2024-04-25T11:13:13.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.799+0000 7f8113200640 1 Processor -- start 2024-04-25T11:13:13.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.799+0000 7f8113200640 1 -- start start 2024-04-25T11:13:13.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.801+0000 7f8113200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81140cfb50 0x7f81141508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.801+0000 7f8113200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140d09c0 0x7f8114141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.801+0000 7f8113200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81141583c0 0x7f811415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.801+0000 7f8113200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f811406f5b0 con 0x7f81141583c0 2024-04-25T11:13:13.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.801+0000 7f8113200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81140cf0f0 con 0x7f81140cfb50 2024-04-25T11:13:13.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.801+0000 7f8113200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f811406f730 con 0x7f81140d09c0 2024-04-25T11:13:13.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.801+0000 7f8112800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81141583c0 0x7f811415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.801+0000 7f8112800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81141583c0 0x7f811415ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50440/0 (socket says 172.21.15.7:50440) 2024-04-25T11:13:13.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.801+0000 7f8112800640 1 -- 172.21.15.7:0/2572556407 learned_addr learned my addr 172.21.15.7:0/2572556407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:13.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.801+0000 7f8111400640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140d09c0 0x7f8114141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.801+0000 7f8111e00640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81140cfb50 0x7f81141508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.802+0000 7f8112800640 1 -- 172.21.15.7:0/2572556407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140d09c0 msgr2=0x7f8114141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.802+0000 7f8112800640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140d09c0 0x7f8114141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.802+0000 7f8112800640 1 -- 172.21.15.7:0/2572556407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81140cfb50 msgr2=0x7f81141508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.802+0000 7f8112800640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81140cfb50 0x7f81141508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.802+0000 7f8112800640 1 -- 172.21.15.7:0/2572556407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8114151920 con 0x7f81141583c0 2024-04-25T11:13:13.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.802+0000 7f8112800640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81141583c0 0x7f811415ccb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f810c00c6e0 tx=0x7f810c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:13.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.802+0000 7f810a000640 1 -- 172.21.15.7:0/2572556407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c003b70 con 0x7f81141583c0 2024-04-25T11:13:13.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.802+0000 7f810a000640 1 -- 172.21.15.7:0/2572556407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f810c003d30 con 0x7f81141583c0 2024-04-25T11:13:13.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.802+0000 7f8113200640 1 -- 172.21.15.7:0/2572556407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81141513a0 con 0x7f81141583c0 2024-04-25T11:13:13.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.802+0000 7f810a000640 1 -- 172.21.15.7:0/2572556407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c005310 con 0x7f81141583c0 2024-04-25T11:13:13.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.802+0000 7f8113200640 1 -- 172.21.15.7:0/2572556407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81140b4e80 con 0x7f81141583c0 2024-04-25T11:13:13.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.805+0000 7f810a000640 1 -- 172.21.15.7:0/2572556407 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f810c0054b0 con 0x7f81141583c0 2024-04-25T11:13:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.806+0000 7f810a000640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80f40744e0 0x7f80f40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.806+0000 7f810a000640 1 -- 172.21.15.7:0/2572556407 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f810c095360 con 0x7f81141583c0 2024-04-25T11:13:13.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.806+0000 7f8111e00640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80f40744e0 0x7f80f40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.806+0000 7f811fdff740 1 Processor -- start 2024-04-25T11:13:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.807+0000 7f811fdff740 1 -- start start 2024-04-25T11:13:13.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.807+0000 7f8111e00640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80f40744e0 0x7f80f40769f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8104006fd0 tx=0x7f8104008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:13.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.808+0000 7f811fdff740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 0x561f2194c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.808+0000 7f811fdff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f21948a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.808+0000 7f811fdff740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f21921d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.808+0000 7f811fdff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561f21885690 con 0x561f215564d0 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.808+0000 7f811fdff740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x561f21821fa0 con 0x561f2155e6b0 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.808+0000 7f811fdff740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x561f21881650 con 0x561f2157f390 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.808+0000 7f8111400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f21948a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.809+0000 7f8111400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f21948a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50454/0 (socket says 172.21.15.7:50454) 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.809+0000 7f8111400640 1 -- 172.21.15.7:0/260132473 learned_addr learned my addr 172.21.15.7:0/260132473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.809+0000 7f8111e00640 1 --2- 172.21.15.7:0/260132473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 0x561f2194c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.809+0000 7f8112800640 1 --2- 172.21.15.7:0/260132473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f21921d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.809+0000 7f8111400640 1 -- 172.21.15.7:0/260132473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 msgr2=0x561f2194c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.809+0000 7f8111400640 1 --2- 172.21.15.7:0/260132473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 0x561f2194c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.809+0000 7f8111400640 1 -- 172.21.15.7:0/260132473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 msgr2=0x561f21921d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.809+0000 7f8111400640 1 --2- 172.21.15.7:0/260132473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f21921d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.809+0000 7f8111400640 1 -- 172.21.15.7:0/260132473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f215973f0 con 0x561f215564d0 2024-04-25T11:13:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.809+0000 7f8111400640 1 --2- 172.21.15.7:0/260132473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f21948a70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8100009990 tx=0x7f810002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd98112c1edd3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.810+0000 7f80fea00640 1 -- 172.21.15.7:0/260132473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100037040 con 0x561f215564d0 2024-04-25T11:13:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.810+0000 7f80fea00640 1 -- 172.21.15.7:0/260132473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8100043070 con 0x561f215564d0 2024-04-25T11:13:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.810+0000 7f80fea00640 1 -- 172.21.15.7:0/260132473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81000329c0 con 0x561f215564d0 2024-04-25T11:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.810+0000 7f811fdff740 1 -- 172.21.15.7:0/260132473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 msgr2=0x561f21948a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.810+0000 7f811fdff740 1 --2- 172.21.15.7:0/260132473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f21948a70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8100009990 tx=0x7f810002c000 comp rx=0 tx=0).stop 2024-04-25T11:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.810+0000 7f811fdff740 1 -- 172.21.15.7:0/260132473 shutdown_connections 2024-04-25T11:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.810+0000 7f811fdff740 1 --2- 172.21.15.7:0/260132473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f21921d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.810+0000 7f811fdff740 1 --2- 172.21.15.7:0/260132473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f21948a70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.810+0000 7f811fdff740 1 --2- 172.21.15.7:0/260132473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 0x561f2194c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.810+0000 7f811fdff740 1 -- 172.21.15.7:0/260132473 >> 172.21.15.7:0/260132473 conn(0x561f215734e0 msgr2=0x561f219214c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.811+0000 7f811fdff740 1 -- 172.21.15.7:0/260132473 shutdown_connections 2024-04-25T11:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.811+0000 7f811fdff740 1 -- 172.21.15.7:0/260132473 wait complete. 2024-04-25T11:13:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.812+0000 7f811fdff740 1 -- start start 2024-04-25T11:13:13.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.812+0000 7f811fdff740 1 Processor -- start 2024-04-25T11:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f811fdff740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f215564d0 0x561f21967430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f811fdff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2155e6b0 0x561f219487a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f811fdff740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 0x561f21964be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f811fdff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561f2180cef0 con 0x561f2155e6b0 2024-04-25T11:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f811fdff740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x561f21965820 con 0x561f215564d0 2024-04-25T11:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f811fdff740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x561f2194cd70 con 0x561f2157f390 2024-04-25T11:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f811fdff740 10 client.0 _ll_register_callbacks cb 0x561f2187c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f8111400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2155e6b0 0x561f219487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f8111400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2155e6b0 0x561f219487a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50458/0 (socket says 172.21.15.7:50458) 2024-04-25T11:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f8111400640 1 -- 172.21.15.7:0/3260959430 learned_addr learned my addr 172.21.15.7:0/3260959430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f8111e00640 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f215564d0 0x561f21967430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f8111400640 1 -- 172.21.15.7:0/3260959430 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 msgr2=0x561f21964be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f8111400640 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 0x561f21964be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f8111400640 1 -- 172.21.15.7:0/3260959430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f215564d0 msgr2=0x561f21967430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f8111400640 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f215564d0 0x561f21967430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.814+0000 7f8111400640 1 -- 172.21.15.7:0/3260959430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2165f760 con 0x561f2155e6b0 2024-04-25T11:13:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.815+0000 7f8111400640 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2155e6b0 0x561f219487a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f810002a290 tx=0x7f8100004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475e3ca9da49954c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.815+0000 7f80f3400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:13:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.815+0000 7f80f3400640 1 -- 172.21.15.7:0/3260959430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810004b030 con 0x561f2155e6b0 2024-04-25T11:13:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.815+0000 7f80f3400640 1 -- 172.21.15.7:0/3260959430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f81000038e0 con 0x561f2155e6b0 2024-04-25T11:13:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.815+0000 7f811fdff740 1 -- 172.21.15.7:0/3260959430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8100043070 con 0x561f2155e6b0 2024-04-25T11:13:13.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.815+0000 7f80f3400640 1 -- 172.21.15.7:0/3260959430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f8100054050 con 0x561f2155e6b0 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.815+0000 7f811fdff740 10 client.5505 fetch_fsmap learned FSMap version 18 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.815+0000 7f811fdff740 1 -- 172.21.15.7:0/3260959430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x561f21965560 con 0x561f2155e6b0 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.816+0000 7f80f3400640 1 -- 172.21.15.7:0/3260959430 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8100043070 con 0x561f2155e6b0 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.816+0000 7f80f3400640 1 -- 172.21.15.7:0/3260959430 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f8100057020 con 0x561f2155e6b0 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.816+0000 7f811fdff740 10 client.5505 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.816+0000 7f811fdff740 10 client.5505 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.816+0000 7f811fdff740 20 client.5505 populate_metadata read hostname 'smithi007' 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.816+0000 7f811fdff740 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x561f2153df60 0x561f21940d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.816+0000 7f811fdff740 4 client.5505 mds_command: new command op to 5283 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.816+0000 7f811fdff740 1 -- 172.21.15.7:0/3260959430 --> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f218932a0 con 0x561f2153df60 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.817+0000 7f8111e00640 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x561f2153df60 0x561f21940d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.818+0000 7f8111e00640 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x561f2153df60 0x561f21940d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea34f03eca54ad4 server_cookie=dca8482b659691e7 in_seq=0 out_seq=0 2024-04-25T11:13:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.818+0000 7f80f3400640 10 client.5505 ms_handle_connect on v2:172.21.15.87:6832/389115631 2024-04-25T11:13:13.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f80f3400640 1 -- 172.21.15.7:0/3260959430 <== mds.0 v2:172.21.15.87:6832/389115631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561f218932a0 con 0x561f2153df60 2024-04-25T11:13:13.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f80f3400640 10 client.5505 handle_command_reply: tid=0 2024-04-25T11:13:13.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 1 client.5505 shutdown 2024-04-25T11:13:13.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 1 -- 172.21.15.7:0/3260959430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2155e6b0 msgr2=0x561f219487a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2155e6b0 0x561f219487a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f810002a290 tx=0x7f8100004730 comp rx=0 tx=0).stop 2024-04-25T11:13:13.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 1 -- 172.21.15.7:0/3260959430 shutdown_connections 2024-04-25T11:13:13.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x561f2153df60 0x561f21940d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 0x561f21964be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2155e6b0 0x561f219487a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 1 --2- 172.21.15.7:0/3260959430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f215564d0 0x561f21967430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f80f3400640 0 client.5505 ms_handle_reset on v2:172.21.15.87:6832/389115631 2024-04-25T11:13:13.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 1 -- 172.21.15.7:0/3260959430 >> 172.21.15.7:0/3260959430 conn(0x561f215734e0 msgr2=0x561f21920860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:13.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 1 -- 172.21.15.7:0/3260959430 shutdown_connections 2024-04-25T11:13:13.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 1 -- 172.21.15.7:0/3260959430 wait complete. 2024-04-25T11:13:13.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.820+0000 7f811fdff740 20 client.5505 trim_cache size 0 max 16384 2024-04-25T11:13:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.837+0000 7f811fdff740 1 Processor -- start 2024-04-25T11:13:13.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.837+0000 7f811fdff740 1 -- start start 2024-04-25T11:13:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.838+0000 7f811fdff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f2191c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.838+0000 7f811fdff740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f21988860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.838+0000 7f811fdff740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 0x561f21988da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.838+0000 7f811fdff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81142030f0 con 0x561f215564d0 2024-04-25T11:13:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.838+0000 7f811fdff740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81142020f0 con 0x561f2155e6b0 2024-04-25T11:13:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.838+0000 7f811fdff740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f81140b6010 con 0x561f2157f390 2024-04-25T11:13:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.839+0000 7f8111e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f2191c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.839+0000 7f8111e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f2191c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50472/0 (socket says 172.21.15.7:50472) 2024-04-25T11:13:13.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.839+0000 7f8111e00640 1 -- 172.21.15.7:0/936307315 learned_addr learned my addr 172.21.15.7:0/936307315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:13.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.839+0000 7f8111e00640 1 -- 172.21.15.7:0/936307315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 msgr2=0x561f21988da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:13.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.839+0000 7f8111e00640 1 --2- 172.21.15.7:0/936307315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 0x561f21988da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.839+0000 7f8111e00640 1 -- 172.21.15.7:0/936307315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 msgr2=0x561f21988860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:13.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.839+0000 7f8111e00640 1 --2- 172.21.15.7:0/936307315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f21988860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.839+0000 7f8111e00640 1 -- 172.21.15.7:0/936307315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f2193fe60 con 0x561f215564d0 2024-04-25T11:13:13.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.839+0000 7f8111e00640 1 --2- 172.21.15.7:0/936307315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f2191c0b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x561f21965310 tx=0x7f8104015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986cae962cc3e2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:13.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.840+0000 7f80fe000640 1 -- 172.21.15.7:0/936307315 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810401b030 con 0x561f215564d0 2024-04-25T11:13:13.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.840+0000 7f80fe000640 1 -- 172.21.15.7:0/936307315 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8104028070 con 0x561f215564d0 2024-04-25T11:13:13.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.840+0000 7f80fe000640 1 -- 172.21.15.7:0/936307315 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104014d50 con 0x561f215564d0 2024-04-25T11:13:13.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.840+0000 7f811fdff740 1 -- 172.21.15.7:0/936307315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 msgr2=0x561f2191c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.840+0000 7f811fdff740 1 --2- 172.21.15.7:0/936307315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f2191c0b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x561f21965310 tx=0x7f8104015000 comp rx=0 tx=0).stop 2024-04-25T11:13:13.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.840+0000 7f811fdff740 1 -- 172.21.15.7:0/936307315 shutdown_connections 2024-04-25T11:13:13.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.840+0000 7f811fdff740 1 --2- 172.21.15.7:0/936307315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f2157f390 0x561f21988da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.840+0000 7f811fdff740 1 --2- 172.21.15.7:0/936307315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f21988860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.840+0000 7f811fdff740 1 --2- 172.21.15.7:0/936307315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f215564d0 0x561f2191c0b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.841+0000 7f811fdff740 1 -- 172.21.15.7:0/936307315 >> 172.21.15.7:0/936307315 conn(0x561f215734e0 msgr2=0x561f21948680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:13.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.841+0000 7f811fdff740 1 -- 172.21.15.7:0/936307315 shutdown_connections 2024-04-25T11:13:13.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.841+0000 7f811fdff740 1 -- 172.21.15.7:0/936307315 wait complete. 2024-04-25T11:13:13.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.842+0000 7f811fdff740 1 -- start start 2024-04-25T11:13:13.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.842+0000 7f811fdff740 1 Processor -- start 2024-04-25T11:13:13.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.843+0000 7f811fdff740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f215564d0 0x561f2194c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.843+0000 7f811fdff740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f2191c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.843+0000 7f811fdff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2157f390 0x561f219e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.843+0000 7f811fdff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561f219892e0 con 0x561f2157f390 2024-04-25T11:13:13.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.843+0000 7f811fdff740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81140b6660 con 0x561f2155e6b0 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.843+0000 7f811fdff740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x561f2191f270 con 0x561f215564d0 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.843+0000 7f811fdff740 10 client.0 _ll_register_callbacks cb 0x7f81000096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.844+0000 7f8112800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2157f390 0x561f219e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.844+0000 7f8112800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2157f390 0x561f219e6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50480/0 (socket says 172.21.15.7:50480) 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.844+0000 7f8112800640 1 -- 172.21.15.7:0/337853941 learned_addr learned my addr 172.21.15.7:0/337853941 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.844+0000 7f8111e00640 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f215564d0 0x561f2194c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.844+0000 7f8111400640 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f2191c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.844+0000 7f8112800640 1 -- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f215564d0 msgr2=0x561f2194c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.844+0000 7f8112800640 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f215564d0 0x561f2194c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.844+0000 7f8112800640 1 -- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 msgr2=0x561f2191c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.844+0000 7f8112800640 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f2191c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.844+0000 7f8112800640 1 -- 172.21.15.7:0/337853941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f219e73b0 con 0x561f2157f390 2024-04-25T11:13:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.844+0000 7f8111400640 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f2191c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f8112800640 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2157f390 0x561f219e6e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f810c05b710 tx=0x7f810c05ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bc0feb8e84e534 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f80f3400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:13:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f80f3400640 1 -- 172.21.15.7:0/337853941 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c0634d0 con 0x561f2157f390 2024-04-25T11:13:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f80f3400640 1 -- 172.21.15.7:0/337853941 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f810c063170 con 0x561f2157f390 2024-04-25T11:13:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f811fdff740 1 -- 172.21.15.7:0/337853941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8104028070 con 0x561f2157f390 2024-04-25T11:13:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f80f3400640 1 -- 172.21.15.7:0/337853941 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f810c067f80 con 0x561f2157f390 2024-04-25T11:13:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f811fdff740 10 client.5511 fetch_fsmap learned FSMap version 18 2024-04-25T11:13:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f811fdff740 1 -- 172.21.15.7:0/337853941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x561f21983810 con 0x561f2157f390 2024-04-25T11:13:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f80f3400640 1 -- 172.21.15.7:0/337853941 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8104028070 con 0x561f2157f390 2024-04-25T11:13:13.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f80f3400640 1 -- 172.21.15.7:0/337853941 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f810c05b3b0 con 0x561f2157f390 2024-04-25T11:13:13.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f811fdff740 10 client.5511 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T11:13:13.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f811fdff740 10 client.5511 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-04-25T11:13:13.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f811fdff740 20 client.5511 populate_metadata read hostname 'smithi007' 2024-04-25T11:13:13.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.845+0000 7f811fdff740 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x561f21921c20 0x561f21922030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:13.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.846+0000 7f811fdff740 4 client.5511 mds_command: new command op to 5283 tid=0 multi_id=0 [{"prefix": "dump_historic_ops"}] 2024-04-25T11:13:13.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.846+0000 7f811fdff740 1 -- 172.21.15.7:0/337853941 --> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] -- command(tid 0: {"prefix": "dump_historic_ops"}) -- 0x561f2188d110 con 0x561f21921c20 2024-04-25T11:13:13.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.846+0000 7f8111e00640 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x561f21921c20 0x561f21922030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:13.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.847+0000 7f8111e00640 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x561f21921c20 0x561f21922030 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cb1b4b2fad6964c server_cookie=4c8c1fced4c973f4 in_seq=0 out_seq=0 2024-04-25T11:13:13.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.847+0000 7f80f3400640 10 client.5511 ms_handle_connect on v2:172.21.15.87:6832/389115631 2024-04-25T11:13:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.852+0000 7f80f3400640 1 -- 172.21.15.7:0/337853941 <== mds.0 v2:172.21.15.87:6832/389115631 1 ==== command_reply(tid 0: 0 ) ==== 8+0+71530 (crc 0 0 0) 0x561f2188d110 con 0x561f21921c20 2024-04-25T11:13:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.852+0000 7f80f3400640 10 client.5511 handle_command_reply: tid=0 2024-04-25T11:13:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.852+0000 7f811fdff740 1 client.5511 shutdown 2024-04-25T11:13:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.852+0000 7f811fdff740 1 -- 172.21.15.7:0/337853941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2157f390 msgr2=0x561f219e6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.852+0000 7f811fdff740 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2157f390 0x561f219e6e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f810c05b710 tx=0x7f810c05ecf0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.852+0000 7f811fdff740 1 -- 172.21.15.7:0/337853941 shutdown_connections 2024-04-25T11:13:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.852+0000 7f811fdff740 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:6832/389115631,v1:172.21.15.87:6833/389115631] conn(0x561f21921c20 0x561f21922030 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.852+0000 7f811fdff740 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561f2157f390 0x561f219e6e70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.852+0000 7f811fdff740 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x561f2155e6b0 0x561f2191c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.852+0000 7f811fdff740 1 --2- 172.21.15.7:0/337853941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x561f215564d0 0x561f2194c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.852+0000 7f80f3400640 0 client.5511 ms_handle_reset on v2:172.21.15.87:6832/389115631 2024-04-25T11:13:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.853+0000 7f811fdff740 1 -- 172.21.15.7:0/337853941 >> 172.21.15.7:0/337853941 conn(0x561f215734e0 msgr2=0x561f21964900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.853+0000 7f811fdff740 1 -- 172.21.15.7:0/337853941 shutdown_connections 2024-04-25T11:13:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.853+0000 7f811fdff740 1 -- 172.21.15.7:0/337853941 wait complete. 2024-04-25T11:13:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.853+0000 7f811fdff740 20 client.5511 trim_cache size 0 max 16384 2024-04-25T11:13:13.855 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:13:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-04-25T11:13:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-04-25T11:13:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-04-25T11:13:13.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:2 setattr mode=041777 #0x1 2024-04-25T11:12:26.611195+0000 caller_uid=0, caller_gid=0{0,})", 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:26.613274+0000", 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.235456663999997, 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.248943079, 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:26.613274+0000", 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:26.613274+0000", 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:26.613275+0000", 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:26.613293+0000", 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:26.613384+0000", 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:26.613773+0000", 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:26.613931+0000", 2024-04-25T11:13:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:26.613933+0000", 2024-04-25T11:13:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:27.861990+0000", 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:27.862037+0000", 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:27.862192+0000", 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:27.862216+0000", 2024-04-25T11:13:13.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:27.862217+0000", 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:27 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_0 2024-04-25T11:12:29.875143+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:29.876826+0000", 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.971904297000002, 2024-04-25T11:13:13.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024083163000000001, 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 27 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.876826+0000", 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.876826+0000", 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.876828+0000", 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.876842+0000", 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.876952+0000", 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.877495+0000", 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.878305+0000", 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.878307+0000", 2024-04-25T11:13:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.900024+0000", 2024-04-25T11:13:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.900440+0000", 2024-04-25T11:13:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.900671+0000", 2024-04-25T11:13:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.900908+0000", 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.900909+0000", 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.867 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:28 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_1 2024-04-25T11:12:29.900142+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:29.901338+0000", 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.947392499999999, 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017605164999999999, 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 28 2024-04-25T11:13:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.901338+0000", 2024-04-25T11:13:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.901338+0000", 2024-04-25T11:13:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.901340+0000", 2024-04-25T11:13:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.901355+0000", 2024-04-25T11:13:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.901515+0000", 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.902178+0000", 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.903475+0000", 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.903477+0000", 2024-04-25T11:13:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.918100+0000", 2024-04-25T11:13:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.918478+0000", 2024-04-25T11:13:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.918710+0000", 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.918941+0000", 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.918943+0000", 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:32 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_5 2024-04-25T11:12:29.964141+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:29.965177+0000", 2024-04-25T11:13:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.883553667999998, 2024-04-25T11:13:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017308647999999999, 2024-04-25T11:13:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 32 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.965177+0000", 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.965177+0000", 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.965179+0000", 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.965201+0000", 2024-04-25T11:13:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.965312+0000", 2024-04-25T11:13:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.965788+0000", 2024-04-25T11:13:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.966687+0000", 2024-04-25T11:13:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.966689+0000", 2024-04-25T11:13:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.981558+0000", 2024-04-25T11:13:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.981937+0000", 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.982244+0000", 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.982484+0000", 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:29.982485+0000", 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:35 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_8 2024-04-25T11:12:30.012140+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:30.013500+0000", 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.835230144999997, 2024-04-25T11:13:13.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017506771000000001, 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 35 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.881 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.013500+0000", 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.013500+0000", 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.013502+0000", 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.013521+0000", 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.013625+0000", 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.014159+0000", 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.015043+0000", 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.015045+0000", 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.030174+0000", 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.030547+0000", 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.030773+0000", 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.031006+0000", 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.031007+0000", 2024-04-25T11:13:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:36 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_9 2024-04-25T11:12:30.030140+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:30.031453+0000", 2024-04-25T11:13:13.886 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.817277247, 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017036154000000001, 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 36 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.031453+0000", 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.031453+0000", 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.031455+0000", 2024-04-25T11:13:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.031475+0000", 2024-04-25T11:13:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.031548+0000", 2024-04-25T11:13:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.032048+0000", 2024-04-25T11:13:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.032946+0000", 2024-04-25T11:13:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.032948+0000", 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.047538+0000", 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.047977+0000", 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.048259+0000", 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.048488+0000", 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.048489+0000", 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:37 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_10 2024-04-25T11:12:30.048140+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:30.048975+0000", 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.799755363000003, 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017432090000000001, 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 37 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.048975+0000", 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.048975+0000", 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.048977+0000", 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.048992+0000", 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.049132+0000", 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.049627+0000", 2024-04-25T11:13:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.050550+0000", 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.050552+0000", 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.065516+0000", 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.065896+0000", 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.066162+0000", 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.066406+0000", 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.066407+0000", 2024-04-25T11:13:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:42 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_15 2024-04-25T11:12:30.127139+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:30.128438+0000", 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.720292223999998, 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017156672000000001, 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 42 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.128438+0000", 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.128438+0000", 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.128440+0000", 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.128453+0000", 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.128532+0000", 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.129007+0000", 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.129905+0000", 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.129907+0000", 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.144684+0000", 2024-04-25T11:13:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.145135+0000", 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.145371+0000", 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.145593+0000", 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.145595+0000", 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:46 create owner_uid=1000, owner_gid=1316 #0x10000000000/file_19 2024-04-25T11:12:30.189138+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:30.190326+0000", 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.658404656999998, 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016162803, 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 46 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.190326+0000", 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.190326+0000", 2024-04-25T11:13:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.190327+0000", 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.190345+0000", 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.190451+0000", 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.190970+0000", 2024-04-25T11:13:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.192006+0000", 2024-04-25T11:13:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.192008+0000", 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.205600+0000", 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.205987+0000", 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.206266+0000", 2024-04-25T11:13:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.206487+0000", 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.206488+0000", 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:54 mkdir owner_uid=1000, owner_gid=1316 #0x1/dir2 2024-04-25T11:12:30.566132+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:30.567588+0000", 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.281142000000003, 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016295446000000002, 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54 2024-04-25T11:13:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.567588+0000", 2024-04-25T11:13:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.567588+0000", 2024-04-25T11:13:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.567590+0000", 2024-04-25T11:13:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.567603+0000", 2024-04-25T11:13:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.567673+0000", 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.568286+0000", 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "failed to wrlock, waiting" 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.569461+0000", 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.570246+0000", 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.570248+0000", 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.583250+0000", 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.583550+0000", 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.583785+0000", 2024-04-25T11:13:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.583882+0000", 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.583884+0000", 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:78 create owner_uid=1000, owner_gid=1316 #0x1000000020e/file_23 2024-04-25T11:12:30.915126+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:30.916426+0000", 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.932304443, 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015987622999999999, 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 78 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.916426+0000", 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.916426+0000", 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.916428+0000", 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.916447+0000", 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.916538+0000", 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.917779+0000", 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.918770+0000", 2024-04-25T11:13:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.918772+0000", 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.931057+0000", 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.931529+0000", 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.931953+0000", 2024-04-25T11:13:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.932409+0000", 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:30.932414+0000", 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:82 mkdir owner_uid=1000, owner_gid=1316 #0x1/dir3 2024-04-25T11:12:31.223121+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:31.224135+0000", 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.624595272999997, 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016964674999999999, 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 82 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.224135+0000", 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.224135+0000", 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.224144+0000", 2024-04-25T11:13:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.224160+0000", 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.224270+0000", 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.224962+0000", 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "failed to wrlock, waiting" 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.226099+0000", 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.226924+0000", 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.226926+0000", 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.240348+0000", 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.240656+0000", 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.240889+0000", 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.241097+0000", 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.241100+0000", 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:97 create owner_uid=1000, owner_gid=1316 #0x10000000228/file_14 2024-04-25T11:12:31.434118+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:31.435199+0000", 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.413531581000001, 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016027815000000001, 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 97 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.435199+0000", 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.435199+0000", 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.435200+0000", 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.435219+0000", 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.435310+0000", 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.435820+0000", 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.436822+0000", 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.436824+0000", 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.450315+0000", 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.450718+0000", 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.450951+0000", 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.451225+0000", 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.451227+0000", 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:101 create owner_uid=1000, owner_gid=1316 #0x10000000228/file_18 2024-04-25T11:12:31.496117+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:31.497192+0000", 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.351538626999996, 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015821524, 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 101 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.497192+0000", 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.497192+0000", 2024-04-25T11:13:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.497194+0000", 2024-04-25T11:13:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.497212+0000", 2024-04-25T11:13:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.497318+0000", 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.497826+0000", 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.498839+0000", 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.498841+0000", 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.511319+0000", 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.511910+0000", 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.512646+0000", 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.513011+0000", 2024-04-25T11:13:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.513013+0000", 2024-04-25T11:13:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:110 mkdir owner_uid=1000, owner_gid=1316 #0x1/dir4 2024-04-25T11:12:31.893110+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:31.894396+0000", 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.954334490999997, 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017658844, 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 110 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.894396+0000", 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.894396+0000", 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.894397+0000", 2024-04-25T11:13:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.894412+0000", 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.894522+0000", 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.895172+0000", 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "failed to wrlock, waiting" 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.896371+0000", 2024-04-25T11:13:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.897199+0000", 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.897201+0000", 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.911426+0000", 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.911729+0000", 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.911962+0000", 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.912053+0000", 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.912055+0000", 2024-04-25T11:13:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:112 create owner_uid=1000, owner_gid=1316 #0x10000000242/file_1 2024-04-25T11:12:31.912110+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:31.913214+0000", 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.935516757000002, 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01592551, 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 112 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.913214+0000", 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.913214+0000", 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.913215+0000", 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.913230+0000", 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.913363+0000", 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.914267+0000", 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.915405+0000", 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.915407+0000", 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.928029+0000", 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.928467+0000", 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.928750+0000", 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.929136+0000", 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:31.929139+0000", 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:118 create owner_uid=1000, owner_gid=1316 #0x10000000242/file_7 2024-04-25T11:12:32.005109+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:32.006596+0000", 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.842134559000002, 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015787160000000001, 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 118 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.006596+0000", 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.006596+0000", 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.006597+0000", 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.006615+0000", 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.006724+0000", 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.007441+0000", 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.008396+0000", 2024-04-25T11:13:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.008398+0000", 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.021254+0000", 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.021696+0000", 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.021990+0000", 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.022380+0000", 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.022383+0000", 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:122 create owner_uid=1000, owner_gid=1316 #0x10000000242/file_11 2024-04-25T11:12:32.067108+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:32.068610+0000", 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.780120314999998, 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016821091999999999, 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 122 2024-04-25T11:13:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.068610+0000", 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.068610+0000", 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.068612+0000", 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.068632+0000", 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.068737+0000", 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.069557+0000", 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.070591+0000", 2024-04-25T11:13:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "early_replied" 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.070593+0000", 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "submit entry: journal_and_reply" 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.083671+0000", 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "journal_committed: " 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.084373+0000", 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.084835+0000", 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.085425+0000", 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.085431+0000", 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:143 readdir #0x10000000242 2024-04-25T11:12:32.665098+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:32.665982+0000", 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.182748834999998, 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 16.509332429000001, 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 143 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.665982+0000", 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.665982+0000", 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.665983+0000", 2024-04-25T11:13:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.665994+0000", 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.666104+0000", 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:32.666385+0000", 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:33.166587+0000", 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:33.666829+0000", 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:34.167050+0000", 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:34.667261+0000", 2024-04-25T11:13:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:35.167462+0000", 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:35.667665+0000", 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:36.167890+0000", 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:36.668136+0000", 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:37.168359+0000", 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:37.668558+0000", 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:38.168786+0000", 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:38.668992+0000", 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:39.169219+0000", 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:39.669426+0000", 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:40.169648+0000", 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:40.669886+0000", 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:41.170177+0000", 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:41.670386+0000", 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:42.170564+0000", 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:42.670772+0000", 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:43.170976+0000", 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:43.671188+0000", 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:44.171370+0000", 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:44.671568+0000", 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:45.171799+0000", 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:45.671998+0000", 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:46.172236+0000", 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:46.672438+0000", 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:47.172680+0000", 2024-04-25T11:13:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:47.672895+0000", 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:48.173152+0000", 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:48.673300+0000", 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.173695+0000", 2024-04-25T11:13:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.174810+0000", 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.174924+0000", 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.175309+0000", 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.175314+0000", 2024-04-25T11:13:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "client_request(client.4862:144 readdir #0x10000000228 2024-04-25T11:12:49.174835+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})", 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-04-25T11:12:49.176384+0000", 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.672346705999999, 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 22.517156227000001, 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0, 2024-04-25T11:13:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "done", 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "reqid": { 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": { 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4862 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 144 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "op_type": "client_request", 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.176384+0000", 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated" 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.176384+0000", 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read" 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.176384+0000", 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled" 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.176394+0000", 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read" 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.176584+0000", 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched" 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.176839+0000", 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:49.676987+0000", 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:50.177200+0000", 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:50.677424+0000", 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:51.177629+0000", 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:51.677842+0000", 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:52.178052+0000", 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:52.678247+0000", 2024-04-25T11:13:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:53.178452+0000", 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:53.678640+0000", 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:54.178840+0000", 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:54.679021+0000", 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:55.179226+0000", 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:55.679435+0000", 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:56.179636+0000", 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:56.679846+0000", 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:57.180046+0000", 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:57.680244+0000", 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:58.180399+0000", 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:58.680592+0000", 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:59.180795+0000", 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:12:59.680985+0000", 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:00.181180+0000", 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:00.681366+0000", 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:01.181515+0000", 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:01.681710+0000", 2024-04-25T11:13:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:02.181855+0000", 2024-04-25T11:13:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:02.682052+0000", 2024-04-25T11:13:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:03.182276+0000", 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:03.682468+0000", 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:04.182675+0000", 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:04.682827+0000", 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:05.183044+0000", 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:05.683269+0000", 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:06.183466+0000", 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:06.683843+0000", 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:07.184042+0000", 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:07.685444+0000", 2024-04-25T11:13:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:08.185634+0000", 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:08.687054+0000", 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:09.187241+0000", 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:09.688629+0000", 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:10.188838+0000", 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:10.690216+0000", 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:11.190428+0000", 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cap_acquisition_throttle" 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:11.692029+0000", 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "acquired locks" 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:11.693140+0000", 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "replying" 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:11.693257+0000", 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "finishing request" 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:11.693536+0000", 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "cleaned up request" 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-04-25T11:13:11.693540+0000", 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done" 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "locks": null 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-25T11:13:13.988 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:13:13.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 -- 172.21.15.7:0/2572556407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80f40744e0 msgr2=0x7f80f40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80f40744e0 0x7f80f40769f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8104006fd0 tx=0x7f8104008040 comp rx=0 tx=0).stop 2024-04-25T11:13:13.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 -- 172.21.15.7:0/2572556407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81141583c0 msgr2=0x7f811415ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:13.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81141583c0 0x7f811415ccb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f810c00c6e0 tx=0x7f810c00cba0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 -- 172.21.15.7:0/2572556407 shutdown_connections 2024-04-25T11:13:13.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80f40744e0 0x7f80f40769f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81141583c0 0x7f811415ccb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81140d09c0 0x7f8114141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 --2- 172.21.15.7:0/2572556407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81140cfb50 0x7f81141508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:13.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 -- 172.21.15.7:0/2572556407 >> 172.21.15.7:0/2572556407 conn(0x7f81140740f0 msgr2=0x7f811406eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:13.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 -- 172.21.15.7:0/2572556407 shutdown_connections 2024-04-25T11:13:13.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:13.854+0000 7f8113200640 1 -- 172.21.15.7:0/2572556407 wait complete. 2024-04-25T11:13:13.991 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:13:13.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:13:14.110 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:13:14.226 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:13:14.226 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:13:14.227 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:13:14.227 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:14.288 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:13:14.289 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:14.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.629+0000 7f4afd600640 1 Processor -- start 2024-04-25T11:13:14.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.629+0000 7f4afd600640 1 -- start start 2024-04-25T11:13:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.631+0000 7f4afd600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4af806ffd0 0x7f4af8164380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.631+0000 7f4afd600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4af80af390 0x7f4af80af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.631+0000 7f4afd600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4af80b47d0 con 0x7f4af80b5800 2024-04-25T11:13:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.631+0000 7f4afd600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4af80b6320 con 0x7f4af806ffd0 2024-04-25T11:13:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.631+0000 7f4afd600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4af80b4a40 con 0x7f4af80af390 2024-04-25T11:13:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.632+0000 7f4afcc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4af80b5800 0x7f4af80b5c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41526/0 (socket says 172.21.15.7:41526) 2024-04-25T11:13:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.632+0000 7f4afcc00640 1 -- 172.21.15.7:0/2599398330 learned_addr learned my addr 172.21.15.7:0/2599398330 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.632+0000 7f4af7e00640 1 --2- 172.21.15.7:0/2599398330 >> v2:172.21.15.7:3300/0 conn(0x7f4af806ffd0 0x7f4af8164380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.632+0000 7f4af7400640 1 --2- 172.21.15.7:0/2599398330 >> v2:172.21.15.87:3300/0 conn(0x7f4af80af390 0x7f4af80af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.632+0000 7f4af6a00640 1 -- 172.21.15.7:0/2599398330 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3192064060 0 0) 0x7f4af80b47d0 con 0x7f4af80b5800 2024-04-25T11:13:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.632+0000 7f4af6a00640 1 -- 172.21.15.7:0/2599398330 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4adc003520 con 0x7f4af80b5800 2024-04-25T11:13:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.632+0000 7f4af7e00640 1 -- 172.21.15.7:0/2599398330 >> v2:172.21.15.87:3300/0 conn(0x7f4af80af390 msgr2=0x7f4af80af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.632+0000 7f4af7e00640 1 --2- 172.21.15.7:0/2599398330 >> v2:172.21.15.87:3300/0 conn(0x7f4af80af390 0x7f4af80af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.632+0000 7f4af7e00640 1 -- 172.21.15.7:0/2599398330 >> v1:172.21.15.7:6789/0 conn(0x7f4af80b5800 legacy=0x7f4af80b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.633+0000 7f4af7e00640 1 -- 172.21.15.7:0/2599398330 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af80b4d20 con 0x7f4af806ffd0 2024-04-25T11:13:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.634+0000 7f4af7e00640 1 --2- 172.21.15.7:0/2599398330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806ffd0 0x7f4af8164380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4af806f8b0 tx=0x7f4ae802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d358408140a4ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.634+0000 7f4af6a00640 1 -- 172.21.15.7:0/2599398330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8037070 con 0x7f4af806ffd0 2024-04-25T11:13:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.634+0000 7f4af6a00640 1 -- 172.21.15.7:0/2599398330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae8032440 con 0x7f4af806ffd0 2024-04-25T11:13:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.634+0000 7f4af6a00640 1 -- 172.21.15.7:0/2599398330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8032810 con 0x7f4af806ffd0 2024-04-25T11:13:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.635+0000 7f4afd600640 1 -- 172.21.15.7:0/2599398330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806ffd0 msgr2=0x7f4af8164380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.635+0000 7f4afd600640 1 --2- 172.21.15.7:0/2599398330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806ffd0 0x7f4af8164380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4af806f8b0 tx=0x7f4ae802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.635+0000 7f4afd600640 1 -- 172.21.15.7:0/2599398330 shutdown_connections 2024-04-25T11:13:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.635+0000 7f4afd600640 1 --2- 172.21.15.7:0/2599398330 >> v2:172.21.15.87:3300/0 conn(0x7f4af80af390 0x7f4af80af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.635+0000 7f4afd600640 1 --2- 172.21.15.7:0/2599398330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806ffd0 0x7f4af8164380 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.635+0000 7f4afd600640 1 -- 172.21.15.7:0/2599398330 >> 172.21.15.7:0/2599398330 conn(0x7f4af80b3820 msgr2=0x7f4af80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.635+0000 7f4afd600640 1 -- 172.21.15.7:0/2599398330 shutdown_connections 2024-04-25T11:13:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.635+0000 7f4afd600640 1 -- 172.21.15.7:0/2599398330 wait complete. 2024-04-25T11:13:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.637+0000 7f4afd600640 1 Processor -- start 2024-04-25T11:13:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.637+0000 7f4afd600640 1 -- start start 2024-04-25T11:13:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.639+0000 7f4afd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806ffd0 0x7f4af815c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.639+0000 7f4afd600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4af80af390 0x7f4af80ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.639+0000 7f4afd600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4af80b5800 0x7f4af815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.639+0000 7f4afd600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4af806fa70 con 0x7f4af806ffd0 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.639+0000 7f4afd600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4af80b52e0 con 0x7f4af80af390 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.639+0000 7f4afd600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4af806fbf0 con 0x7f4af80b5800 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.639+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806ffd0 0x7f4af815c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.639+0000 7f4af7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806ffd0 0x7f4af815c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50500/0 (socket says 172.21.15.7:50500) 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.639+0000 7f4af7e00640 1 -- 172.21.15.7:0/2928403304 learned_addr learned my addr 172.21.15.7:0/2928403304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.639+0000 7f4af7400640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4af80af390 0x7f4af80ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.640+0000 7f4af7e00640 1 -- 172.21.15.7:0/2928403304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4af80b5800 msgr2=0x7f4af815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.640+0000 7f4afcc00640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4af80b5800 0x7f4af815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.640+0000 7f4af7e00640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4af80b5800 0x7f4af815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.640+0000 7f4af7e00640 1 -- 172.21.15.7:0/2928403304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4af80af390 msgr2=0x7f4af80ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.640+0000 7f4af7e00640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4af80af390 0x7f4af80ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.640+0000 7f4af7e00640 1 -- 172.21.15.7:0/2928403304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af815f0e0 con 0x7f4af806ffd0 2024-04-25T11:13:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.640+0000 7f4afcc00640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4af80b5800 0x7f4af815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.640+0000 7f4af7e00640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806ffd0 0x7f4af815c990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8000c00 tx=0x7f4ae80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.640+0000 7f4aefe00640 1 -- 172.21.15.7:0/2928403304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8037040 con 0x7f4af806ffd0 2024-04-25T11:13:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.640+0000 7f4aefe00640 1 -- 172.21.15.7:0/2928403304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae8042070 con 0x7f4af806ffd0 2024-04-25T11:13:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.640+0000 7f4aefe00640 1 -- 172.21.15.7:0/2928403304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8030960 con 0x7f4af806ffd0 2024-04-25T11:13:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.641+0000 7f4afd600640 1 -- 172.21.15.7:0/2928403304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af815f3c0 con 0x7f4af806ffd0 2024-04-25T11:13:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.641+0000 7f4afd600640 1 -- 172.21.15.7:0/2928403304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af815f980 con 0x7f4af806ffd0 2024-04-25T11:13:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.642+0000 7f4afd600640 1 -- 172.21.15.7:0/2928403304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af8050740 con 0x7f4af806ffd0 2024-04-25T11:13:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.643+0000 7f4aefe00640 1 -- 172.21.15.7:0/2928403304 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ae8030b00 con 0x7f4af806ffd0 2024-04-25T11:13:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.644+0000 7f4aefe00640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4ad80744b0 0x7f4ad80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.645+0000 7f4aefe00640 1 -- 172.21.15.7:0/2928403304 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f4ae80b5e00 con 0x7f4af806ffd0 2024-04-25T11:13:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.645+0000 7f4af7400640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4ad80744b0 0x7f4ad80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.645+0000 7f4af7400640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4ad80744b0 0x7f4ad80769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4af8074930 tx=0x7f4ae40039b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.649+0000 7f4aefe00640 1 -- 172.21.15.7:0/2928403304 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ae8080340 con 0x7f4af806ffd0 2024-04-25T11:13:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.855+0000 7f4afd600640 1 -- 172.21.15.7:0/2928403304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4af8168b90 con 0x7f4af806ffd0 2024-04-25T11:13:14.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.857+0000 7f4aefe00640 1 -- 172.21.15.7:0/2928403304 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16839 (secure 0 0 0) 0x7f4ae807fc40 con 0x7f4af806ffd0 2024-04-25T11:13:14.858 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:14.858 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:04.193129+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:12:14.961811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T11:12:16.128549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.860+0000 7f4afd600640 1 -- 172.21.15.7:0/2928403304 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4ad80744b0 msgr2=0x7f4ad80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.860+0000 7f4afd600640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4ad80744b0 0x7f4ad80769c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4af8074930 tx=0x7f4ae40039b0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.860+0000 7f4afd600640 1 -- 172.21.15.7:0/2928403304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806ffd0 msgr2=0x7f4af815c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.860+0000 7f4afd600640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806ffd0 0x7f4af815c990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8000c00 tx=0x7f4ae80046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.861+0000 7f4afd600640 1 -- 172.21.15.7:0/2928403304 shutdown_connections 2024-04-25T11:13:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.861+0000 7f4afd600640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4ad80744b0 0x7f4ad80769c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.861+0000 7f4afd600640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4af80b5800 0x7f4af815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.861+0000 7f4afd600640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4af80af390 0x7f4af80ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.861+0000 7f4afd600640 1 --2- 172.21.15.7:0/2928403304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4af806ffd0 0x7f4af815c990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.861+0000 7f4afd600640 1 -- 172.21.15.7:0/2928403304 >> 172.21.15.7:0/2928403304 conn(0x7f4af80b3820 msgr2=0x7f4af8166960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.861+0000 7f4afd600640 1 -- 172.21.15.7:0/2928403304 shutdown_connections 2024-04-25T11:13:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:14.861+0000 7f4afd600640 1 -- 172.21.15.7:0/2928403304 wait complete. 2024-04-25T11:13:14.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:12:14.961811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:12:16.128549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:13:14.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:13:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.176+0000 7f3f0d200640 1 Processor -- start 2024-04-25T11:13:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.176+0000 7f3f0d200640 1 -- start start 2024-04-25T11:13:15.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.179+0000 7f3f0d200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3f08146180 0x7f3f08070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.179+0000 7f3f0d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3f081490f0 0x7f3f08149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.179+0000 7f3f0d200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f08070860 con 0x7f3f080b01d0 2024-04-25T11:13:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.179+0000 7f3f0d200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3f08146590 con 0x7f3f08146180 2024-04-25T11:13:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.179+0000 7f3f0d200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3f0806f280 con 0x7f3f081490f0 2024-04-25T11:13:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.179+0000 7f3f07400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3f08146180 0x7f3f08070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.179+0000 7f3f07e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3f080b01d0 0x7f3f080b05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41538/0 (socket says 172.21.15.7:41538) 2024-04-25T11:13:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.179+0000 7f3f07e00640 1 -- 172.21.15.7:0/4228528982 learned_addr learned my addr 172.21.15.7:0/4228528982 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:15.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.180+0000 7f3effe00640 1 --2- 172.21.15.7:0/4228528982 >> v2:172.21.15.87:3300/0 conn(0x7f3f081490f0 0x7f3f08149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.180+0000 7f3f06a00640 1 -- 172.21.15.7:0/4228528982 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751965533 0 0) 0x7f3f08070860 con 0x7f3f080b01d0 2024-04-25T11:13:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.180+0000 7f3f06a00640 1 -- 172.21.15.7:0/4228528982 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3eec003520 con 0x7f3f080b01d0 2024-04-25T11:13:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.180+0000 7f3f07400640 1 -- 172.21.15.7:0/4228528982 >> v2:172.21.15.87:3300/0 conn(0x7f3f081490f0 msgr2=0x7f3f08149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.180+0000 7f3f07400640 1 --2- 172.21.15.7:0/4228528982 >> v2:172.21.15.87:3300/0 conn(0x7f3f081490f0 0x7f3f08149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.180+0000 7f3f07400640 1 -- 172.21.15.7:0/4228528982 >> v1:172.21.15.7:6789/0 conn(0x7f3f080b01d0 legacy=0x7f3f080b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.180+0000 7f3f07400640 1 -- 172.21.15.7:0/4228528982 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0806f560 con 0x7f3f08146180 2024-04-25T11:13:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.181+0000 7f3f07400640 1 --2- 172.21.15.7:0/4228528982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f08146180 0x7f3f08070320 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3f08072610 tx=0x7f3ef802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7932c09e02d752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.182+0000 7f3f06a00640 1 -- 172.21.15.7:0/4228528982 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8037070 con 0x7f3f08146180 2024-04-25T11:13:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.182+0000 7f3f06a00640 1 -- 172.21.15.7:0/4228528982 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef8032440 con 0x7f3f08146180 2024-04-25T11:13:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.182+0000 7f3f06a00640 1 -- 172.21.15.7:0/4228528982 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8032810 con 0x7f3f08146180 2024-04-25T11:13:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.182+0000 7f3f0d200640 1 -- 172.21.15.7:0/4228528982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f08146180 msgr2=0x7f3f08070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.182+0000 7f3f0d200640 1 --2- 172.21.15.7:0/4228528982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f08146180 0x7f3f08070320 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3f08072610 tx=0x7f3ef802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.183+0000 7f3f0d200640 1 -- 172.21.15.7:0/4228528982 shutdown_connections 2024-04-25T11:13:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.183+0000 7f3f0d200640 1 --2- 172.21.15.7:0/4228528982 >> v2:172.21.15.87:3300/0 conn(0x7f3f081490f0 0x7f3f08149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.183+0000 7f3f0d200640 1 --2- 172.21.15.7:0/4228528982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f08146180 0x7f3f08070320 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.183+0000 7f3f0d200640 1 -- 172.21.15.7:0/4228528982 >> 172.21.15.7:0/4228528982 conn(0x7f3f080740f0 msgr2=0x7f3f0806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.183+0000 7f3f0d200640 1 -- 172.21.15.7:0/4228528982 shutdown_connections 2024-04-25T11:13:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.183+0000 7f3f0d200640 1 -- 172.21.15.7:0/4228528982 wait complete. 2024-04-25T11:13:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.185+0000 7f3f0d200640 1 Processor -- start 2024-04-25T11:13:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.185+0000 7f3f0d200640 1 -- start start 2024-04-25T11:13:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.187+0000 7f3f0d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f080b01d0 0x7f3f0815f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.187+0000 7f3f0d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08146180 0x7f3f08162b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.187+0000 7f3f0d200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f081490f0 0x7f3f08163050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.187+0000 7f3f0d200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0806fe00 con 0x7f3f080b01d0 2024-04-25T11:13:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.187+0000 7f3f0d200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0806fb20 con 0x7f3f08146180 2024-04-25T11:13:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.187+0000 7f3f0d200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0806eea0 con 0x7f3f081490f0 2024-04-25T11:13:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.187+0000 7f3f07400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f080b01d0 0x7f3f0815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.187+0000 7f3f07400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f080b01d0 0x7f3f0815f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50516/0 (socket says 172.21.15.7:50516) 2024-04-25T11:13:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.187+0000 7f3f07400640 1 -- 172.21.15.7:0/1263428829 learned_addr learned my addr 172.21.15.7:0/1263428829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.188+0000 7f3f07e00640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f081490f0 0x7f3f08163050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.188+0000 7f3effe00640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08146180 0x7f3f08162b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.188+0000 7f3f07400640 1 -- 172.21.15.7:0/1263428829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f081490f0 msgr2=0x7f3f08163050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.188+0000 7f3f07400640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f081490f0 0x7f3f08163050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.188+0000 7f3f07400640 1 -- 172.21.15.7:0/1263428829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08146180 msgr2=0x7f3f08162b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.188+0000 7f3f07400640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08146180 0x7f3f08162b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.188+0000 7f3f07400640 1 -- 172.21.15.7:0/1263428829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f080cde10 con 0x7f3f080b01d0 2024-04-25T11:13:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.188+0000 7f3f07400640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f080b01d0 0x7f3f0815f290 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8000c00 tx=0x7f3ef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.189+0000 7f3eff400640 1 -- 172.21.15.7:0/1263428829 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8037040 con 0x7f3f080b01d0 2024-04-25T11:13:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.189+0000 7f3eff400640 1 -- 172.21.15.7:0/1263428829 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef8042070 con 0x7f3f080b01d0 2024-04-25T11:13:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.189+0000 7f3eff400640 1 -- 172.21.15.7:0/1263428829 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8030960 con 0x7f3f080b01d0 2024-04-25T11:13:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.189+0000 7f3f0d200640 1 -- 172.21.15.7:0/1263428829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f080ce0f0 con 0x7f3f080b01d0 2024-04-25T11:13:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.189+0000 7f3f0d200640 1 -- 172.21.15.7:0/1263428829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f080ce6b0 con 0x7f3f080b01d0 2024-04-25T11:13:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.190+0000 7f3f0d200640 1 -- 172.21.15.7:0/1263428829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f08050740 con 0x7f3f080b01d0 2024-04-25T11:13:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.191+0000 7f3eff400640 1 -- 172.21.15.7:0/1263428829 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3ef8030b00 con 0x7f3f080b01d0 2024-04-25T11:13:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.193+0000 7f3eff400640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ee40744b0 0x7f3ee40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.193+0000 7f3eff400640 1 -- 172.21.15.7:0/1263428829 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f3ef80b5e80 con 0x7f3f080b01d0 2024-04-25T11:13:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.193+0000 7f3effe00640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ee40744b0 0x7f3ee40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.194+0000 7f3effe00640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ee40744b0 0x7f3ee40769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0006fd0 tx=0x7f3ef0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.198+0000 7f3eff400640 1 -- 172.21.15.7:0/1263428829 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ef80803c0 con 0x7f3f080b01d0 2024-04-25T11:13:15.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.405+0000 7f3f0d200640 1 -- 172.21.15.7:0/1263428829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f08145d50 con 0x7f3f080b01d0 2024-04-25T11:13:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.406+0000 7f3eff400640 1 -- 172.21.15.7:0/1263428829 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7f3ef807fcc0 con 0x7f3f080b01d0 2024-04-25T11:13:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:13:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.408+0000 7f3f0d200640 1 -- 172.21.15.7:0/1263428829 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ee40744b0 msgr2=0x7f3ee40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.408+0000 7f3f0d200640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ee40744b0 0x7f3ee40769c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0006fd0 tx=0x7f3ef0008040 comp rx=0 tx=0).stop 2024-04-25T11:13:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.408+0000 7f3f0d200640 1 -- 172.21.15.7:0/1263428829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f080b01d0 msgr2=0x7f3f0815f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.408+0000 7f3f0d200640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f080b01d0 0x7f3f0815f290 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8000c00 tx=0x7f3ef80046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.409+0000 7f3f0d200640 1 -- 172.21.15.7:0/1263428829 shutdown_connections 2024-04-25T11:13:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.409+0000 7f3f0d200640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ee40744b0 0x7f3ee40769c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.409+0000 7f3f0d200640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f081490f0 0x7f3f08163050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.409+0000 7f3f0d200640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f08146180 0x7f3f08162b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.409+0000 7f3f0d200640 1 --2- 172.21.15.7:0/1263428829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f080b01d0 0x7f3f0815f290 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.409+0000 7f3f0d200640 1 -- 172.21.15.7:0/1263428829 >> 172.21.15.7:0/1263428829 conn(0x7f3f080740f0 msgr2=0x7f3f080b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.409+0000 7f3f0d200640 1 -- 172.21.15.7:0/1263428829 shutdown_connections 2024-04-25T11:13:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.409+0000 7f3f0d200640 1 -- 172.21.15.7:0/1263428829 wait complete. 2024-04-25T11:13:15.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:13:15.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.730+0000 7faa36200640 1 Processor -- start 2024-04-25T11:13:15.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.730+0000 7faa36200640 1 -- start start 2024-04-25T11:13:15.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.732+0000 7faa36200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faa3006e4b0 0x7faa3006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.732+0000 7faa36200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa300b3ac0 0x7faa300b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.732+0000 7faa36200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faa301644f0 0x7faa30164900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.732+0000 7faa36200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7faa300b92e0 con 0x7faa3006e4b0 2024-04-25T11:13:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.732+0000 7faa36200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7faa3006d810 con 0x7faa301644f0 2024-04-25T11:13:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.732+0000 7faa36200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faa3009cb10 con 0x7faa300b3ac0 2024-04-25T11:13:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.733+0000 7faa34e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faa3006e4b0 0x7faa3006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.733+0000 7faa34e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faa3006e4b0 0x7faa3006e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50518/0 (socket says 172.21.15.7:50518) 2024-04-25T11:13:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.733+0000 7faa34e00640 1 -- 172.21.15.7:0/2191649084 learned_addr learned my addr 172.21.15.7:0/2191649084 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:15.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.733+0000 7faa35800640 1 --2- 172.21.15.7:0/2191649084 >> v2:172.21.15.87:3300/0 conn(0x7faa301644f0 0x7faa30164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.733+0000 7faa2fe00640 1 --2- 172.21.15.7:0/2191649084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa300b3ac0 0x7faa300b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.733+0000 7faa34e00640 1 -- 172.21.15.7:0/2191649084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa300b3ac0 msgr2=0x7faa300b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:15.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.733+0000 7faa34e00640 1 --2- 172.21.15.7:0/2191649084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa300b3ac0 0x7faa300b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.733+0000 7faa34e00640 1 -- 172.21.15.7:0/2191649084 >> v2:172.21.15.87:3300/0 conn(0x7faa301644f0 msgr2=0x7faa30164900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:15.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.733+0000 7faa34e00640 1 --2- 172.21.15.7:0/2191649084 >> v2:172.21.15.87:3300/0 conn(0x7faa301644f0 0x7faa30164900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.733+0000 7faa34e00640 1 -- 172.21.15.7:0/2191649084 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3009d010 con 0x7faa3006e4b0 2024-04-25T11:13:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.735+0000 7faa34e00640 1 --2- 172.21.15.7:0/2191649084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa3006e4b0 0x7faa3006e8c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7faa300ca720 tx=0x7faa2002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7aba8dfef0c3a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.735+0000 7faa2f400640 1 -- 172.21.15.7:0/2191649084 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa20036070 con 0x7faa3006e4b0 2024-04-25T11:13:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.735+0000 7faa2f400640 1 -- 172.21.15.7:0/2191649084 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa20031600 con 0x7faa3006e4b0 2024-04-25T11:13:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.735+0000 7faa2f400640 1 -- 172.21.15.7:0/2191649084 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa200319d0 con 0x7faa3006e4b0 2024-04-25T11:13:15.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.736+0000 7faa36200640 1 -- 172.21.15.7:0/2191649084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa3006e4b0 msgr2=0x7faa3006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:15.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.736+0000 7faa36200640 1 --2- 172.21.15.7:0/2191649084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa3006e4b0 0x7faa3006e8c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7faa300ca720 tx=0x7faa2002a610 comp rx=0 tx=0).stop 2024-04-25T11:13:15.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.736+0000 7faa36200640 1 -- 172.21.15.7:0/2191649084 shutdown_connections 2024-04-25T11:13:15.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.736+0000 7faa36200640 1 --2- 172.21.15.7:0/2191649084 >> v2:172.21.15.87:3300/0 conn(0x7faa301644f0 0x7faa30164900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.736+0000 7faa36200640 1 --2- 172.21.15.7:0/2191649084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa300b3ac0 0x7faa300b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.736+0000 7faa36200640 1 --2- 172.21.15.7:0/2191649084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa3006e4b0 0x7faa3006e8c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.736+0000 7faa36200640 1 -- 172.21.15.7:0/2191649084 >> 172.21.15.7:0/2191649084 conn(0x7faa300d01d0 msgr2=0x7faa300d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:15.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.736+0000 7faa36200640 1 -- 172.21.15.7:0/2191649084 shutdown_connections 2024-04-25T11:13:15.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.736+0000 7faa36200640 1 -- 172.21.15.7:0/2191649084 wait complete. 2024-04-25T11:13:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.738+0000 7faa36200640 1 Processor -- start 2024-04-25T11:13:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.738+0000 7faa36200640 1 -- start start 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.740+0000 7faa36200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa3006e4b0 0x7faa3015fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.740+0000 7faa36200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa300b3ac0 0x7faa300c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.740+0000 7faa36200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa301644f0 0x7faa30161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.740+0000 7faa36200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa300b8fd0 con 0x7faa3006e4b0 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.740+0000 7faa36200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faa3006da60 con 0x7faa301644f0 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.740+0000 7faa36200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faa300cda60 con 0x7faa300b3ac0 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.740+0000 7faa34e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa3006e4b0 0x7faa3015fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.740+0000 7faa34e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa3006e4b0 0x7faa3015fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50532/0 (socket says 172.21.15.7:50532) 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.740+0000 7faa34e00640 1 -- 172.21.15.7:0/3008302214 learned_addr learned my addr 172.21.15.7:0/3008302214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.741+0000 7faa2fe00640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa300b3ac0 0x7faa300c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.741+0000 7faa34e00640 1 -- 172.21.15.7:0/3008302214 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa300b3ac0 msgr2=0x7faa300c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.741+0000 7faa34e00640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa300b3ac0 0x7faa300c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.741+0000 7faa34e00640 1 -- 172.21.15.7:0/3008302214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa301644f0 msgr2=0x7faa30161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.741+0000 7faa34e00640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa301644f0 0x7faa30161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.741+0000 7faa34e00640 1 -- 172.21.15.7:0/3008302214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa300b6f20 con 0x7faa3006e4b0 2024-04-25T11:13:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.741+0000 7faa34e00640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa3006e4b0 0x7faa3015fed0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faa20000f80 tx=0x7faa20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.742+0000 7faa2cc00640 1 -- 172.21.15.7:0/3008302214 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa20036050 con 0x7faa3006e4b0 2024-04-25T11:13:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.742+0000 7faa2cc00640 1 -- 172.21.15.7:0/3008302214 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa20042070 con 0x7faa3006e4b0 2024-04-25T11:13:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.742+0000 7faa2cc00640 1 -- 172.21.15.7:0/3008302214 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa20009510 con 0x7faa3006e4b0 2024-04-25T11:13:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.742+0000 7faa36200640 1 -- 172.21.15.7:0/3008302214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa300b7200 con 0x7faa3006e4b0 2024-04-25T11:13:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.742+0000 7faa36200640 1 -- 172.21.15.7:0/3008302214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa300b77c0 con 0x7faa3006e4b0 2024-04-25T11:13:15.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.743+0000 7faa36200640 1 -- 172.21.15.7:0/3008302214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa30050740 con 0x7faa3006e4b0 2024-04-25T11:13:15.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.744+0000 7faa2cc00640 1 -- 172.21.15.7:0/3008302214 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faa20007c70 con 0x7faa3006e4b0 2024-04-25T11:13:15.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.745+0000 7faa2cc00640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa0c0744b0 0x7faa0c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:15.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.745+0000 7faa2cc00640 1 -- 172.21.15.7:0/3008302214 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7faa200b63b0 con 0x7faa3006e4b0 2024-04-25T11:13:15.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.746+0000 7faa2fe00640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa0c0744b0 0x7faa0c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:15.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.747+0000 7faa2fe00640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa0c0744b0 0x7faa0c0769c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faa18006fd0 tx=0x7faa18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:15.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.750+0000 7faa2cc00640 1 -- 172.21.15.7:0/3008302214 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa200808f0 con 0x7faa3006e4b0 2024-04-25T11:13:15.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.951+0000 7faa36200640 1 -- 172.21.15.7:0/3008302214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faa300d0fa0 con 0x7faa3006e4b0 2024-04-25T11:13:15.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.953+0000 7faa2cc00640 1 -- 172.21.15.7:0/3008302214 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) ==== 104+0+11 (secure 0 0 0) 0x7faa200801f0 con 0x7faa3006e4b0 2024-04-25T11:13:15.954 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:13:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.955+0000 7faa36200640 1 -- 172.21.15.7:0/3008302214 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa0c0744b0 msgr2=0x7faa0c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.955+0000 7faa36200640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa0c0744b0 0x7faa0c0769c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faa18006fd0 tx=0x7faa18008040 comp rx=0 tx=0).stop 2024-04-25T11:13:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.955+0000 7faa36200640 1 -- 172.21.15.7:0/3008302214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa3006e4b0 msgr2=0x7faa3015fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.955+0000 7faa36200640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa3006e4b0 0x7faa3015fed0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faa20000f80 tx=0x7faa20004a40 comp rx=0 tx=0).stop 2024-04-25T11:13:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.956+0000 7faa36200640 1 -- 172.21.15.7:0/3008302214 shutdown_connections 2024-04-25T11:13:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.956+0000 7faa36200640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa0c0744b0 0x7faa0c0769c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.956+0000 7faa36200640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa301644f0 0x7faa30161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.956+0000 7faa36200640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa300b3ac0 0x7faa300c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.956+0000 7faa36200640 1 --2- 172.21.15.7:0/3008302214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa3006e4b0 0x7faa3015fed0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.956+0000 7faa36200640 1 -- 172.21.15.7:0/3008302214 >> 172.21.15.7:0/3008302214 conn(0x7faa300d01d0 msgr2=0x7faa300cdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.956+0000 7faa36200640 1 -- 172.21.15.7:0/3008302214 shutdown_connections 2024-04-25T11:13:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:15.956+0000 7faa36200640 1 -- 172.21.15.7:0/3008302214 wait complete. 2024-04-25T11:13:15.983 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:13:16.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.271+0000 7f2f85200640 1 Processor -- start 2024-04-25T11:13:16.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.271+0000 7f2f85200640 1 -- start start 2024-04-25T11:13:16.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.273+0000 7f2f85200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2f8006fab0 0x7f2f80164710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.273+0000 7f2f85200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f800b1390 con 0x7f2f800b6720 2024-04-25T11:13:16.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.273+0000 7f2f85200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f800ae880 con 0x7f2f80157320 2024-04-25T11:13:16.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.273+0000 7f2f85200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2f80160a60 con 0x7f2f8006fab0 2024-04-25T11:13:16.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.274+0000 7f2f7de00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2f800b6720 0x7f2f80160ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41542/0 (socket says 172.21.15.7:41542) 2024-04-25T11:13:16.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.274+0000 7f2f7de00640 1 -- 172.21.15.7:0/1073321680 learned_addr learned my addr 172.21.15.7:0/1073321680 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.274+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1609451250 0 0) 0x7f2f800b1390 con 0x7f2f800b6720 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.274+0000 7f2f7e800640 1 --2- 172.21.15.7:0/1073321680 >> v2:172.21.15.87:3300/0 conn(0x7f2f8006fab0 0x7f2f80164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.274+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f64003520 con 0x7f2f800b6720 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2678036320 0 0) 0x7f2f64003520 con 0x7f2f800b6720 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 >> v2:172.21.15.87:3300/0 conn(0x7f2f8006fab0 msgr2=0x7f2f80164710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 --2- 172.21.15.7:0/1073321680 >> v2:172.21.15.87:3300/0 conn(0x7f2f8006fab0 0x7f2f80164710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 >> v1:172.21.15.87:6789/0 conn(0x7f2f80157320 legacy=0x7f2f8006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f80160710 con 0x7f2f800b6720 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2f70001670 con 0x7f2f800b6720 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 >> v1:172.21.15.7:6789/0 conn(0x7f2f800b6720 legacy=0x7f2f80160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 --2- 172.21.15.7:0/1073321680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 0x7f2f64003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 --2- 172.21.15.7:0/1073321680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f64004800 0x7f2f64004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 --2- 172.21.15.7:0/1073321680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f64005220 0x7f2f6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f80160a60 con 0x7f2f64003a00 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6400c1b0 con 0x7f2f64005220 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6400c3b0 con 0x7f2f64004800 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.275+0000 7f2f7de00640 1 --2- 172.21.15.7:0/1073321680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 0x7f2f64003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.276+0000 7f2f7e800640 1 --2- 172.21.15.7:0/1073321680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f64005220 0x7f2f6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.276+0000 7f2f7d400640 1 --2- 172.21.15.7:0/1073321680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f64004800 0x7f2f64004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.276+0000 7f2f7de00640 1 -- 172.21.15.7:0/1073321680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f64004800 msgr2=0x7f2f64004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.276+0000 7f2f7de00640 1 --2- 172.21.15.7:0/1073321680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f64004800 0x7f2f64004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.276+0000 7f2f7de00640 1 -- 172.21.15.7:0/1073321680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f64005220 msgr2=0x7f2f6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.276+0000 7f2f7de00640 1 --2- 172.21.15.7:0/1073321680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f64005220 0x7f2f6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.276+0000 7f2f7de00640 1 -- 172.21.15.7:0/1073321680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f64003520 con 0x7f2f64003a00 2024-04-25T11:13:16.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.276+0000 7f2f7e800640 1 -- 172.21.15.7:0/1073321680 reap_dead start 2024-04-25T11:13:16.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.277+0000 7f2f7de00640 1 --2- 172.21.15.7:0/1073321680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 0x7f2f64003e90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000c0e0 tx=0x7f2f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839891369c6b2fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.277+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70007c40 con 0x7f2f64003a00 2024-04-25T11:13:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.277+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f70005af0 con 0x7f2f64003a00 2024-04-25T11:13:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.277+0000 7f2f7ca00640 1 -- 172.21.15.7:0/1073321680 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70007070 con 0x7f2f64003a00 2024-04-25T11:13:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.278+0000 7f2f85200640 1 -- 172.21.15.7:0/1073321680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 msgr2=0x7f2f64003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.278+0000 7f2f85200640 1 --2- 172.21.15.7:0/1073321680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 0x7f2f64003e90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000c0e0 tx=0x7f2f64005830 comp rx=0 tx=0).stop 2024-04-25T11:13:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.278+0000 7f2f85200640 1 -- 172.21.15.7:0/1073321680 shutdown_connections 2024-04-25T11:13:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.278+0000 7f2f85200640 1 --2- 172.21.15.7:0/1073321680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 0x7f2f64003e90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.278+0000 7f2f85200640 1 -- 172.21.15.7:0/1073321680 >> 172.21.15.7:0/1073321680 conn(0x7f2f800742c0 msgr2=0x7f2f8006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.278+0000 7f2f85200640 1 -- 172.21.15.7:0/1073321680 shutdown_connections 2024-04-25T11:13:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.278+0000 7f2f85200640 1 -- 172.21.15.7:0/1073321680 wait complete. 2024-04-25T11:13:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.281+0000 7f2f85200640 1 Processor -- start 2024-04-25T11:13:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.281+0000 7f2f85200640 1 -- start start 2024-04-25T11:13:16.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.282+0000 7f2f85200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 0x7f2f8014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.282+0000 7f2f85200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f80159700 0x7f2f801578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f85200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f80157e30 0x7f2f801532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f85200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f80164710 con 0x7f2f64003a00 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f85200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2f800aee00 con 0x7f2f80159700 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f85200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f80164890 con 0x7f2f80157e30 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f7de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 0x7f2f8014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f7de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 0x7f2f8014e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50552/0 (socket says 172.21.15.7:50552) 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f7de00640 1 -- 172.21.15.7:0/2318657494 learned_addr learned my addr 172.21.15.7:0/2318657494 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f7e800640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f80157e30 0x7f2f801532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f7de00640 1 -- 172.21.15.7:0/2318657494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f80157e30 msgr2=0x7f2f801532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f7de00640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f80157e30 0x7f2f801532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f7d400640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f80159700 0x7f2f801578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f7de00640 1 -- 172.21.15.7:0/2318657494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f80159700 msgr2=0x7f2f801578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f7de00640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f80159700 0x7f2f801578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.283+0000 7f2f7de00640 1 -- 172.21.15.7:0/2318657494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f80158660 con 0x7f2f64003a00 2024-04-25T11:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.284+0000 7f2f7de00640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 0x7f2f8014e400 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000c200 tx=0x7f2f800af0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.284+0000 7f2f76000640 1 -- 172.21.15.7:0/2318657494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70037040 con 0x7f2f64003a00 2024-04-25T11:13:16.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.284+0000 7f2f76000640 1 -- 172.21.15.7:0/2318657494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f70041070 con 0x7f2f64003a00 2024-04-25T11:13:16.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.284+0000 7f2f76000640 1 -- 172.21.15.7:0/2318657494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f70001730 con 0x7f2f64003a00 2024-04-25T11:13:16.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.284+0000 7f2f85200640 1 -- 172.21.15.7:0/2318657494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f80158940 con 0x7f2f64003a00 2024-04-25T11:13:16.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.284+0000 7f2f85200640 1 -- 172.21.15.7:0/2318657494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f80158f00 con 0x7f2f64003a00 2024-04-25T11:13:16.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.285+0000 7f2f85200640 1 -- 172.21.15.7:0/2318657494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f80050740 con 0x7f2f64003a00 2024-04-25T11:13:16.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.287+0000 7f2f76000640 1 -- 172.21.15.7:0/2318657494 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2f70004a20 con 0x7f2f64003a00 2024-04-25T11:13:16.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.288+0000 7f2f76000640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2f600744b0 0x7f2f600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.288+0000 7f2f76000640 1 -- 172.21.15.7:0/2318657494 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f2f700b5ef0 con 0x7f2f64003a00 2024-04-25T11:13:16.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.288+0000 7f2f7d400640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2f600744b0 0x7f2f600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.289+0000 7f2f7d400640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2f600744b0 0x7f2f600769c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2f8015f560 tx=0x7f2f6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:16.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.291+0000 7f2f76000640 1 -- 172.21.15.7:0/2318657494 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2f700804b0 con 0x7f2f64003a00 2024-04-25T11:13:16.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.500+0000 7f2f85200640 1 -- 172.21.15.7:0/2318657494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2f8015ab20 con 0x7f2f64003a00 2024-04-25T11:13:16.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.501+0000 7f2f76000640 1 -- 172.21.15.7:0/2318657494 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) ==== 100+0+11 (secure 0 0 0) 0x7f2f7007fdb0 con 0x7f2f64003a00 2024-04-25T11:13:16.501 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:13:16.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 -- 172.21.15.7:0/2318657494 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2f600744b0 msgr2=0x7f2f600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2f600744b0 0x7f2f600769c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2f8015f560 tx=0x7f2f6c009040 comp rx=0 tx=0).stop 2024-04-25T11:13:16.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 -- 172.21.15.7:0/2318657494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 msgr2=0x7f2f8014e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 0x7f2f8014e400 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000c200 tx=0x7f2f800af0e0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 -- 172.21.15.7:0/2318657494 shutdown_connections 2024-04-25T11:13:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2f600744b0 0x7f2f600769c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f80157e30 0x7f2f801532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f80159700 0x7f2f801578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 --2- 172.21.15.7:0/2318657494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f64003a00 0x7f2f8014e400 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 -- 172.21.15.7:0/2318657494 >> 172.21.15.7:0/2318657494 conn(0x7f2f800742c0 msgr2=0x7f2f800b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 -- 172.21.15.7:0/2318657494 shutdown_connections 2024-04-25T11:13:16.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.504+0000 7f2f85200640 1 -- 172.21.15.7:0/2318657494 wait complete. 2024-04-25T11:13:16.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:13:16.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.818+0000 7fcf6fe00640 1 Processor -- start 2024-04-25T11:13:16.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.818+0000 7fcf6fe00640 1 -- start start 2024-04-25T11:13:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.821+0000 7fcf6fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eed0 0x7fcf700cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.821+0000 7fcf6fe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf7006c570 con 0x7fcf70141290 2024-04-25T11:13:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.821+0000 7fcf6fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf7006c850 con 0x7fcf701416a0 2024-04-25T11:13:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.821+0000 7fcf6fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7006cb30 con 0x7fcf7015eed0 2024-04-25T11:13:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.821+0000 7fcf6e000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcf70141290 0x7fcf7006d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41554/0 (socket says 172.21.15.7:41554) 2024-04-25T11:13:16.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.821+0000 7fcf6e000640 1 -- 172.21.15.7:0/3365465973 learned_addr learned my addr 172.21.15.7:0/3365465973 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:16.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6ea00640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eed0 0x7fcf700cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3063111440 0 0) 0x7fcf7006c570 con 0x7fcf70141290 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf54003520 con 0x7fcf70141290 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2577736012 0 0) 0x7fcf54003520 con 0x7fcf70141290 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eed0 msgr2=0x7fcf700cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eed0 0x7fcf700cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 >> v1:172.21.15.87:6789/0 conn(0x7fcf701416a0 legacy=0x7fcf700b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7006ce10 con 0x7fcf70141290 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fcf5c001670 con 0x7fcf70141290 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 >> v1:172.21.15.7:6789/0 conn(0x7fcf70141290 legacy=0x7fcf7006d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf540039e0 0x7fcf54003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eed0 0x7fcf54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf54004d40 0x7fcf54005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7006cb30 con 0x7fcf54004d40 2024-04-25T11:13:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5400baf0 con 0x7fcf540039e0 2024-04-25T11:13:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.822+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5400bcf0 con 0x7fcf7015eed0 2024-04-25T11:13:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.823+0000 7fcf6f400640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf54004d40 0x7fcf54005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.823+0000 7fcf6ea00640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf540039e0 0x7fcf54003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.823+0000 7fcf6e000640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eed0 0x7fcf54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.823+0000 7fcf6f400640 1 -- 172.21.15.7:0/3365465973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eed0 msgr2=0x7fcf54004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.823+0000 7fcf6f400640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eed0 0x7fcf54004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.823+0000 7fcf6f400640 1 -- 172.21.15.7:0/3365465973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf540039e0 msgr2=0x7fcf54003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.823+0000 7fcf6f400640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf540039e0 0x7fcf54003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.823+0000 7fcf6f400640 1 -- 172.21.15.7:0/3365465973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf54003520 con 0x7fcf54004d40 2024-04-25T11:13:16.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.824+0000 7fcf6f400640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf54004d40 0x7fcf54005150 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcf68001e20 tx=0x7fcf54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bd20edb887b80a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.825+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68001a70 con 0x7fcf54004d40 2024-04-25T11:13:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.825+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf6802ae40 con 0x7fcf54004d40 2024-04-25T11:13:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.825+0000 7fcf6d600640 1 -- 172.21.15.7:0/3365465973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68003bb0 con 0x7fcf54004d40 2024-04-25T11:13:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.825+0000 7fcf6fe00640 1 -- 172.21.15.7:0/3365465973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf54004d40 msgr2=0x7fcf54005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.825+0000 7fcf6fe00640 1 --2- 172.21.15.7:0/3365465973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf54004d40 0x7fcf54005150 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcf68001e20 tx=0x7fcf54005830 comp rx=0 tx=0).stop 2024-04-25T11:13:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.825+0000 7fcf6f400640 1 -- 172.21.15.7:0/3365465973 reap_dead start 2024-04-25T11:13:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.826+0000 7fcf6fe00640 1 -- 172.21.15.7:0/3365465973 shutdown_connections 2024-04-25T11:13:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.826+0000 7fcf6fe00640 1 -- 172.21.15.7:0/3365465973 >> 172.21.15.7:0/3365465973 conn(0x7fcf700d1540 msgr2=0x7fcf700d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.826+0000 7fcf6fe00640 1 -- 172.21.15.7:0/3365465973 shutdown_connections 2024-04-25T11:13:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.826+0000 7fcf6fe00640 1 -- 172.21.15.7:0/3365465973 wait complete. 2024-04-25T11:13:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.828+0000 7fcf6fe00640 1 Processor -- start 2024-04-25T11:13:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.828+0000 7fcf6fe00640 1 -- start start 2024-04-25T11:13:16.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.830+0000 7fcf6fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf70133330 0x7fcf701f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.830+0000 7fcf6fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf701318b0 0x7fcf70133940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.830+0000 7fcf6fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf70131cc0 0x7fcf70133e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.830+0000 7fcf6fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7006dd20 con 0x7fcf701318b0 2024-04-25T11:13:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.830+0000 7fcf6fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf70141ab0 con 0x7fcf70133330 2024-04-25T11:13:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.830+0000 7fcf6fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7006dea0 con 0x7fcf70131cc0 2024-04-25T11:13:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.830+0000 7fcf6e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf701318b0 0x7fcf70133940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.830+0000 7fcf6e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf701318b0 0x7fcf70133940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50570/0 (socket says 172.21.15.7:50570) 2024-04-25T11:13:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.830+0000 7fcf6e000640 1 -- 172.21.15.7:0/1140327053 learned_addr learned my addr 172.21.15.7:0/1140327053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:16.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.831+0000 7fcf6ea00640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf70133330 0x7fcf701f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.831+0000 7fcf6f400640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf70131cc0 0x7fcf70133e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.831+0000 7fcf6e000640 1 -- 172.21.15.7:0/1140327053 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf70131cc0 msgr2=0x7fcf70133e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.831+0000 7fcf6e000640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf70131cc0 0x7fcf70133e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.831+0000 7fcf6e000640 1 -- 172.21.15.7:0/1140327053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf70133330 msgr2=0x7fcf701f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.831+0000 7fcf6e000640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf70133330 0x7fcf701f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.831+0000 7fcf6e000640 1 -- 172.21.15.7:0/1140327053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf701f5ff0 con 0x7fcf701318b0 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.831+0000 7fcf6f400640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf70131cc0 0x7fcf70133e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.831+0000 7fcf6e000640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf701318b0 0x7fcf70133940 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00eb20 tx=0x7fcf5c0016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.832+0000 7fcf66a00640 1 -- 172.21.15.7:0/1140327053 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c00c8d0 con 0x7fcf701318b0 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.832+0000 7fcf66a00640 1 -- 172.21.15.7:0/1140327053 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5c018070 con 0x7fcf701318b0 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.832+0000 7fcf6fe00640 1 -- 172.21.15.7:0/1140327053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf70132290 con 0x7fcf701318b0 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.832+0000 7fcf66a00640 1 -- 172.21.15.7:0/1140327053 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c00ccb0 con 0x7fcf701318b0 2024-04-25T11:13:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.832+0000 7fcf6fe00640 1 -- 172.21.15.7:0/1140327053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf70132850 con 0x7fcf701318b0 2024-04-25T11:13:16.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.833+0000 7fcf6fe00640 1 -- 172.21.15.7:0/1140327053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf70050740 con 0x7fcf701318b0 2024-04-25T11:13:16.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.834+0000 7fcf66a00640 1 -- 172.21.15.7:0/1140327053 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf5c01d460 con 0x7fcf701318b0 2024-04-25T11:13:16.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.835+0000 7fcf66a00640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf500744b0 0x7fcf500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:16.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.835+0000 7fcf66a00640 1 -- 172.21.15.7:0/1140327053 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fcf5c095350 con 0x7fcf701318b0 2024-04-25T11:13:16.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.836+0000 7fcf6ea00640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf500744b0 0x7fcf500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:16.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.836+0000 7fcf6ea00640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf500744b0 0x7fcf500769c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcf70153ea0 tx=0x7fcf60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:16.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:16.841+0000 7fcf66a00640 1 -- 172.21.15.7:0/1140327053 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf5c05f9b0 con 0x7fcf701318b0 2024-04-25T11:13:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.065+0000 7fcf6fe00640 1 -- 172.21.15.7:0/1140327053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcf700b3070 con 0x7fcf701318b0 2024-04-25T11:13:17.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.066+0000 7fcf66a00640 1 -- 172.21.15.7:0/1140327053 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7fcf5c05f2b0 con 0x7fcf701318b0 2024-04-25T11:13:17.066 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:13:17.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.068+0000 7fcf6fe00640 1 -- 172.21.15.7:0/1140327053 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf500744b0 msgr2=0x7fcf500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:17.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.068+0000 7fcf6fe00640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf500744b0 0x7fcf500769c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcf70153ea0 tx=0x7fcf60008040 comp rx=0 tx=0).stop 2024-04-25T11:13:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.068+0000 7fcf6fe00640 1 -- 172.21.15.7:0/1140327053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf701318b0 msgr2=0x7fcf70133940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.068+0000 7fcf6fe00640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf701318b0 0x7fcf70133940 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00eb20 tx=0x7fcf5c0016f0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.069+0000 7fcf6fe00640 1 -- 172.21.15.7:0/1140327053 shutdown_connections 2024-04-25T11:13:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.069+0000 7fcf6fe00640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf500744b0 0x7fcf500769c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.069+0000 7fcf6fe00640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf70131cc0 0x7fcf70133e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.069+0000 7fcf6fe00640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf701318b0 0x7fcf70133940 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.069+0000 7fcf6fe00640 1 --2- 172.21.15.7:0/1140327053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf70133330 0x7fcf701f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.069+0000 7fcf6fe00640 1 -- 172.21.15.7:0/1140327053 >> 172.21.15.7:0/1140327053 conn(0x7fcf700d1540 msgr2=0x7fcf700b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.069+0000 7fcf6fe00640 1 -- 172.21.15.7:0/1140327053 shutdown_connections 2024-04-25T11:13:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.069+0000 7fcf6fe00640 1 -- 172.21.15.7:0/1140327053 wait complete. 2024-04-25T11:13:17.096 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:13:17.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:17.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:13:17.286 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:13:17.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:17.287 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:13:17.326 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:13:17.326 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:13:17.326 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:17.365 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:13:17.365 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:17.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:17.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.746+0000 7f15d9c00640 1 Processor -- start 2024-04-25T11:13:17.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.746+0000 7f15d9c00640 1 -- start start 2024-04-25T11:13:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.749+0000 7f15d9c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d415ef90 0x7f15d409f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.749+0000 7f15d9c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f15d415bc40 0x7f15d40aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.749+0000 7f15d9c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15d40af370 0x7f15d4161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.749+0000 7f15d9c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f15d4071190 con 0x7f15d40af370 2024-04-25T11:13:17.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.749+0000 7f15d9c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f15d415f4e0 con 0x7f15d415bc40 2024-04-25T11:13:17.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.749+0000 7f15d9c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f15d40af780 con 0x7f15d415ef90 2024-04-25T11:13:17.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.749+0000 7f15d3400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15d40af370 0x7f15d4161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:17.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.749+0000 7f15d3400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15d40af370 0x7f15d4161b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50580/0 (socket says 172.21.15.7:50580) 2024-04-25T11:13:17.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.749+0000 7f15d3400640 1 -- 172.21.15.7:0/3682793537 learned_addr learned my addr 172.21.15.7:0/3682793537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.750+0000 7f15d2000640 1 --2- 172.21.15.7:0/3682793537 >> v2:172.21.15.87:3300/0 conn(0x7f15d415bc40 0x7f15d40aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.750+0000 7f15d2a00640 1 --2- 172.21.15.7:0/3682793537 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d415ef90 0x7f15d409f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.750+0000 7f15d3400640 1 -- 172.21.15.7:0/3682793537 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d415ef90 msgr2=0x7f15d409f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.750+0000 7f15d3400640 1 --2- 172.21.15.7:0/3682793537 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d415ef90 0x7f15d409f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.750+0000 7f15d3400640 1 -- 172.21.15.7:0/3682793537 >> v2:172.21.15.87:3300/0 conn(0x7f15d415bc40 msgr2=0x7f15d40aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.750+0000 7f15d3400640 1 --2- 172.21.15.7:0/3682793537 >> v2:172.21.15.87:3300/0 conn(0x7f15d415bc40 0x7f15d40aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.750+0000 7f15d3400640 1 -- 172.21.15.7:0/3682793537 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d40b0560 con 0x7f15d40af370 2024-04-25T11:13:17.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.750+0000 7f15d2000640 1 --2- 172.21.15.7:0/3682793537 >> v2:172.21.15.87:3300/0 conn(0x7f15d415bc40 0x7f15d40aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:13:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.751+0000 7f15d3400640 1 --2- 172.21.15.7:0/3682793537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d40af370 0x7f15d4161b00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f15d40b6e50 tx=0x7f15cc02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79329765ec56fd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.752+0000 7f15d1600640 1 -- 172.21.15.7:0/3682793537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc036070 con 0x7f15d40af370 2024-04-25T11:13:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.752+0000 7f15d1600640 1 -- 172.21.15.7:0/3682793537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15cc031600 con 0x7f15d40af370 2024-04-25T11:13:17.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.752+0000 7f15d1600640 1 -- 172.21.15.7:0/3682793537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0319d0 con 0x7f15d40af370 2024-04-25T11:13:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.753+0000 7f15d9c00640 1 -- 172.21.15.7:0/3682793537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d40af370 msgr2=0x7f15d4161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.753+0000 7f15d9c00640 1 --2- 172.21.15.7:0/3682793537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d40af370 0x7f15d4161b00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f15d40b6e50 tx=0x7f15cc02a630 comp rx=0 tx=0).stop 2024-04-25T11:13:17.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.753+0000 7f15d9c00640 1 -- 172.21.15.7:0/3682793537 shutdown_connections 2024-04-25T11:13:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.753+0000 7f15d9c00640 1 --2- 172.21.15.7:0/3682793537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d40af370 0x7f15d4161b00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.753+0000 7f15d9c00640 1 --2- 172.21.15.7:0/3682793537 >> v2:172.21.15.87:3300/0 conn(0x7f15d415bc40 0x7f15d40aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.753+0000 7f15d9c00640 1 --2- 172.21.15.7:0/3682793537 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d415ef90 0x7f15d409f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.753+0000 7f15d9c00640 1 -- 172.21.15.7:0/3682793537 >> 172.21.15.7:0/3682793537 conn(0x7f15d4074200 msgr2=0x7f15d406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.753+0000 7f15d9c00640 1 -- 172.21.15.7:0/3682793537 shutdown_connections 2024-04-25T11:13:17.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.753+0000 7f15d9c00640 1 -- 172.21.15.7:0/3682793537 wait complete. 2024-04-25T11:13:17.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.757+0000 7f15d9c00640 1 Processor -- start 2024-04-25T11:13:17.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.757+0000 7f15d9c00640 1 -- start start 2024-04-25T11:13:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.759+0000 7f15d9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d40af370 0x7f15d415a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.759+0000 7f15d9c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d415bc40 0x7f15d4149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.759+0000 7f15d9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d415ef90 0x7f15d40d0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.759+0000 7f15d9c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15d40b0e00 con 0x7f15d415ef90 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.759+0000 7f15d9c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f15d40b0b20 con 0x7f15d40af370 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.759+0000 7f15d9c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f15d40b1320 con 0x7f15d415bc40 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.760+0000 7f15d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d415ef90 0x7f15d40d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.760+0000 7f15d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d415ef90 0x7f15d40d0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50588/0 (socket says 172.21.15.7:50588) 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.760+0000 7f15d3400640 1 -- 172.21.15.7:0/147273790 learned_addr learned my addr 172.21.15.7:0/147273790 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.760+0000 7f15d2a00640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d40af370 0x7f15d415a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.760+0000 7f15d2000640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d415bc40 0x7f15d4149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.760+0000 7f15d3400640 1 -- 172.21.15.7:0/147273790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d415bc40 msgr2=0x7f15d4149270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.760+0000 7f15d3400640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d415bc40 0x7f15d4149270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.760+0000 7f15d3400640 1 -- 172.21.15.7:0/147273790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d40af370 msgr2=0x7f15d415a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.760+0000 7f15d3400640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d40af370 0x7f15d415a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.760+0000 7f15d3400640 1 -- 172.21.15.7:0/147273790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d41f9b10 con 0x7f15d415ef90 2024-04-25T11:13:17.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.760+0000 7f15d2a00640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d40af370 0x7f15d415a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:13:17.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.761+0000 7f15d3400640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d415ef90 0x7f15d40d0950 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009f70 tx=0x7f15cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:17.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.761+0000 7f15caa00640 1 -- 172.21.15.7:0/147273790 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc041020 con 0x7f15d415ef90 2024-04-25T11:13:17.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.761+0000 7f15caa00640 1 -- 172.21.15.7:0/147273790 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15cc008730 con 0x7f15d415ef90 2024-04-25T11:13:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.761+0000 7f15caa00640 1 -- 172.21.15.7:0/147273790 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc036040 con 0x7f15d415ef90 2024-04-25T11:13:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.761+0000 7f15d9c00640 1 -- 172.21.15.7:0/147273790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d41f9df0 con 0x7f15d415ef90 2024-04-25T11:13:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.761+0000 7f15d9c00640 1 -- 172.21.15.7:0/147273790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d41fa3b0 con 0x7f15d415ef90 2024-04-25T11:13:17.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.762+0000 7f15d9c00640 1 -- 172.21.15.7:0/147273790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d406a100 con 0x7f15d415ef90 2024-04-25T11:13:17.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.764+0000 7f15caa00640 1 -- 172.21.15.7:0/147273790 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f15cc044020 con 0x7f15d415ef90 2024-04-25T11:13:17.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.766+0000 7f15caa00640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b40744b0 0x7f15b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:17.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.766+0000 7f15caa00640 1 -- 172.21.15.7:0/147273790 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f15cc0b6060 con 0x7f15d415ef90 2024-04-25T11:13:17.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.766+0000 7f15d2a00640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b40744b0 0x7f15b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:17.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.767+0000 7f15d2a00640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b40744b0 0x7f15b40769c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f15c4006fd0 tx=0x7f15c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.771+0000 7f15caa00640 1 -- 172.21.15.7:0/147273790 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15cc04b030 con 0x7f15d415ef90 2024-04-25T11:13:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.982+0000 7f15d9c00640 1 -- 172.21.15.7:0/147273790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15d41f9860 con 0x7f15d415ef90 2024-04-25T11:13:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.985+0000 7f15caa00640 1 -- 172.21.15.7:0/147273790 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16839 (secure 0 0 0) 0x7f15cc080630 con 0x7f15d415ef90 2024-04-25T11:13:17.986 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:17.986 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:04.193129+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:12:14.961811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T11:12:16.128549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.988+0000 7f15d9c00640 1 -- 172.21.15.7:0/147273790 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b40744b0 msgr2=0x7f15b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.988+0000 7f15d9c00640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b40744b0 0x7f15b40769c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f15c4006fd0 tx=0x7f15c4008040 comp rx=0 tx=0).stop 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.988+0000 7f15d9c00640 1 -- 172.21.15.7:0/147273790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d415ef90 msgr2=0x7f15d40d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.988+0000 7f15d9c00640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d415ef90 0x7f15d40d0950 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009f70 tx=0x7f15cc004a40 comp rx=0 tx=0).stop 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.988+0000 7f15d9c00640 1 -- 172.21.15.7:0/147273790 shutdown_connections 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.988+0000 7f15d9c00640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b40744b0 0x7f15b40769c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.988+0000 7f15d9c00640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d415ef90 0x7f15d40d0950 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.988+0000 7f15d9c00640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d415bc40 0x7f15d4149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.988+0000 7f15d9c00640 1 --2- 172.21.15.7:0/147273790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d40af370 0x7f15d415a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.988+0000 7f15d9c00640 1 -- 172.21.15.7:0/147273790 >> 172.21.15.7:0/147273790 conn(0x7f15d4074200 msgr2=0x7f15d4070190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.989+0000 7f15d9c00640 1 -- 172.21.15.7:0/147273790 shutdown_connections 2024-04-25T11:13:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:17.989+0000 7f15d9c00640 1 -- 172.21.15.7:0/147273790 wait complete. 2024-04-25T11:13:18.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:12:14.961811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:12:16.128549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:13:18.017 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:13:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.306+0000 7fef6f200640 1 Processor -- start 2024-04-25T11:13:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.306+0000 7fef6f200640 1 -- start start 2024-04-25T11:13:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.308+0000 7fef6f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef700cff00 0x7fef700d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.308+0000 7fef6f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fef70167de0 0x7fef701681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.309+0000 7fef6f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef70168730 con 0x7fef701498d0 2024-04-25T11:13:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.309+0000 7fef6f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fef7006eea0 con 0x7fef700cff00 2024-04-25T11:13:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.309+0000 7fef6f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fef700d09e0 con 0x7fef70167de0 2024-04-25T11:13:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.309+0000 7fef6de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef700cff00 0x7fef700d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.309+0000 7fef6de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef700cff00 0x7fef700d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50590/0 (socket says 172.21.15.7:50590) 2024-04-25T11:13:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.309+0000 7fef6de00640 1 -- 172.21.15.7:0/1656393569 learned_addr learned my addr 172.21.15.7:0/1656393569 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.309+0000 7fef65400640 1 --2- 172.21.15.7:0/1656393569 >> v2:172.21.15.87:3300/0 conn(0x7fef70167de0 0x7fef701681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.309+0000 7fef6de00640 1 -- 172.21.15.7:0/1656393569 >> v2:172.21.15.87:3300/0 conn(0x7fef70167de0 msgr2=0x7fef701681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.310+0000 7fef6de00640 1 --2- 172.21.15.7:0/1656393569 >> v2:172.21.15.87:3300/0 conn(0x7fef70167de0 0x7fef701681f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.310+0000 7fef6de00640 1 -- 172.21.15.7:0/1656393569 >> v1:172.21.15.87:6789/0 conn(0x7fef701498d0 legacy=0x7fef70149ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.310+0000 7fef6de00640 1 -- 172.21.15.7:0/1656393569 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef700d0cc0 con 0x7fef700cff00 2024-04-25T11:13:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.311+0000 7fef6de00640 1 --2- 172.21.15.7:0/1656393569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef700cff00 0x7fef700d0310 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fef700b6760 tx=0x7fef6002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1a7ae4008d0218 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.311+0000 7fef6d400640 1 -- 172.21.15.7:0/1656393569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60037070 con 0x7fef700cff00 2024-04-25T11:13:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.311+0000 7fef6d400640 1 -- 172.21.15.7:0/1656393569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef60032440 con 0x7fef700cff00 2024-04-25T11:13:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.311+0000 7fef6d400640 1 -- 172.21.15.7:0/1656393569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60032810 con 0x7fef700cff00 2024-04-25T11:13:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.312+0000 7fef6f200640 1 -- 172.21.15.7:0/1656393569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef700cff00 msgr2=0x7fef700d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.312+0000 7fef6f200640 1 --2- 172.21.15.7:0/1656393569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef700cff00 0x7fef700d0310 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fef700b6760 tx=0x7fef6002a230 comp rx=0 tx=0).stop 2024-04-25T11:13:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.312+0000 7fef6f200640 1 -- 172.21.15.7:0/1656393569 shutdown_connections 2024-04-25T11:13:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.312+0000 7fef6f200640 1 --2- 172.21.15.7:0/1656393569 >> v2:172.21.15.87:3300/0 conn(0x7fef70167de0 0x7fef701681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.312+0000 7fef6f200640 1 --2- 172.21.15.7:0/1656393569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef700cff00 0x7fef700d0310 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.312+0000 7fef6f200640 1 -- 172.21.15.7:0/1656393569 >> 172.21.15.7:0/1656393569 conn(0x7fef700740f0 msgr2=0x7fef7006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.312+0000 7fef6f200640 1 -- 172.21.15.7:0/1656393569 shutdown_connections 2024-04-25T11:13:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.312+0000 7fef6f200640 1 -- 172.21.15.7:0/1656393569 wait complete. 2024-04-25T11:13:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.315+0000 7fef6f200640 1 Processor -- start 2024-04-25T11:13:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.315+0000 7fef6f200640 1 -- start start 2024-04-25T11:13:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.316+0000 7fef6f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef700cff00 0x7fef7014d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.316+0000 7fef6f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef701498d0 0x7fef7014d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.316+0000 7fef6f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef70167de0 0x7fef70145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.316+0000 7fef6f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fef70168a30 con 0x7fef700cff00 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.316+0000 7fef6f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fef700d1280 con 0x7fef70167de0 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.316+0000 7fef6f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fef70168bb0 con 0x7fef701498d0 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.317+0000 7fef6de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef700cff00 0x7fef7014d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.317+0000 7fef6de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef700cff00 0x7fef7014d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50592/0 (socket says 172.21.15.7:50592) 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.317+0000 7fef6de00640 1 -- 172.21.15.7:0/3212079192 learned_addr learned my addr 172.21.15.7:0/3212079192 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.317+0000 7fef6de00640 1 -- 172.21.15.7:0/3212079192 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef701498d0 msgr2=0x7fef7014d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.317+0000 7fef6de00640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef701498d0 0x7fef7014d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.317+0000 7fef6de00640 1 -- 172.21.15.7:0/3212079192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef70167de0 msgr2=0x7fef70145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.317+0000 7fef6e800640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef70167de0 0x7fef70145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.317+0000 7fef6de00640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef70167de0 0x7fef70145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.317+0000 7fef6de00640 1 -- 172.21.15.7:0/3212079192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef701fbc30 con 0x7fef700cff00 2024-04-25T11:13:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.317+0000 7fef6e800640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef70167de0 0x7fef70145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.318+0000 7fef6de00640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef700cff00 0x7fef7014d160 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fef60000c00 tx=0x7fef600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.318+0000 7fef66a00640 1 -- 172.21.15.7:0/3212079192 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60041020 con 0x7fef700cff00 2024-04-25T11:13:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.318+0000 7fef66a00640 1 -- 172.21.15.7:0/3212079192 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef60030970 con 0x7fef700cff00 2024-04-25T11:13:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.318+0000 7fef66a00640 1 -- 172.21.15.7:0/3212079192 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60037040 con 0x7fef700cff00 2024-04-25T11:13:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.318+0000 7fef6f200640 1 -- 172.21.15.7:0/3212079192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef70143210 con 0x7fef700cff00 2024-04-25T11:13:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.318+0000 7fef6f200640 1 -- 172.21.15.7:0/3212079192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7014dbe0 con 0x7fef700cff00 2024-04-25T11:13:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.319+0000 7fef6f200640 1 -- 172.21.15.7:0/3212079192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef70050740 con 0x7fef700cff00 2024-04-25T11:13:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.321+0000 7fef66a00640 1 -- 172.21.15.7:0/3212079192 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fef60044020 con 0x7fef700cff00 2024-04-25T11:13:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.322+0000 7fef66a00640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef4c0744b0 0x7fef4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.322+0000 7fef66a00640 1 -- 172.21.15.7:0/3212079192 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fef600b5ed0 con 0x7fef700cff00 2024-04-25T11:13:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.322+0000 7fef65400640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef4c0744b0 0x7fef4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.323+0000 7fef65400640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef4c0744b0 0x7fef4c0769c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fef70166df0 tx=0x7fef58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.327+0000 7fef66a00640 1 -- 172.21.15.7:0/3212079192 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef6004b030 con 0x7fef700cff00 2024-04-25T11:13:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.531+0000 7fef6f200640 1 -- 172.21.15.7:0/3212079192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef7014b1b0 con 0x7fef700cff00 2024-04-25T11:13:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.532+0000 7fef66a00640 1 -- 172.21.15.7:0/3212079192 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7fef70168730 con 0x7fef700cff00 2024-04-25T11:13:18.532 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:13:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.535+0000 7fef6f200640 1 -- 172.21.15.7:0/3212079192 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef4c0744b0 msgr2=0x7fef4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.535+0000 7fef6f200640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef4c0744b0 0x7fef4c0769c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fef70166df0 tx=0x7fef58008040 comp rx=0 tx=0).stop 2024-04-25T11:13:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.535+0000 7fef6f200640 1 -- 172.21.15.7:0/3212079192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef700cff00 msgr2=0x7fef7014d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.535+0000 7fef6f200640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef700cff00 0x7fef7014d160 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fef60000c00 tx=0x7fef600046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.535+0000 7fef6f200640 1 -- 172.21.15.7:0/3212079192 shutdown_connections 2024-04-25T11:13:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.535+0000 7fef6f200640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef4c0744b0 0x7fef4c0769c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.535+0000 7fef6f200640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef70167de0 0x7fef70145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.535+0000 7fef6f200640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef701498d0 0x7fef7014d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.535+0000 7fef6f200640 1 --2- 172.21.15.7:0/3212079192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef700cff00 0x7fef7014d160 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.535+0000 7fef6f200640 1 -- 172.21.15.7:0/3212079192 >> 172.21.15.7:0/3212079192 conn(0x7fef700740f0 msgr2=0x7fef700b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.535+0000 7fef6f200640 1 -- 172.21.15.7:0/3212079192 shutdown_connections 2024-04-25T11:13:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.536+0000 7fef6f200640 1 -- 172.21.15.7:0/3212079192 wait complete. 2024-04-25T11:13:18.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:13:18.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.850+0000 7f667c000640 1 Processor -- start 2024-04-25T11:13:18.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.851+0000 7f667c000640 1 -- start start 2024-04-25T11:13:18.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.853+0000 7f667c000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f667415dfb0 0x7f66740b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.853+0000 7f667c000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66740d0b40 0x7f667406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.853+0000 7f667c000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66740d0f50 0x7f667406f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.853+0000 7f667c000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f66740b09b0 con 0x7f667415dfb0 2024-04-25T11:13:18.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.853+0000 7f667c000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f667415e3c0 con 0x7f66740d0f50 2024-04-25T11:13:18.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.853+0000 7f667c000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66740cf800 con 0x7f66740d0b40 2024-04-25T11:13:18.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.854+0000 7f6679600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f667415dfb0 0x7f66740b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:18.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.854+0000 7f6679600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f667415dfb0 0x7f66740b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50602/0 (socket says 172.21.15.7:50602) 2024-04-25T11:13:18.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.854+0000 7f6679600640 1 -- 172.21.15.7:0/2826160943 learned_addr learned my addr 172.21.15.7:0/2826160943 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:18.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.854+0000 7f6678c00640 1 --2- 172.21.15.7:0/2826160943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66740d0b40 0x7f667406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:18.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.854+0000 7f667a000640 1 --2- 172.21.15.7:0/2826160943 >> v2:172.21.15.87:3300/0 conn(0x7f66740d0f50 0x7f667406f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:18.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.854+0000 7f6679600640 1 -- 172.21.15.7:0/2826160943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66740d0b40 msgr2=0x7f667406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:18.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.854+0000 7f6679600640 1 --2- 172.21.15.7:0/2826160943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66740d0b40 0x7f667406f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.854+0000 7f6679600640 1 -- 172.21.15.7:0/2826160943 >> v2:172.21.15.87:3300/0 conn(0x7f66740d0f50 msgr2=0x7f667406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:18.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.854+0000 7f6679600640 1 --2- 172.21.15.7:0/2826160943 >> v2:172.21.15.87:3300/0 conn(0x7f66740d0f50 0x7f667406f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.854+0000 7f6679600640 1 -- 172.21.15.7:0/2826160943 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66740cfaa0 con 0x7f667415dfb0 2024-04-25T11:13:18.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.855+0000 7f6679600640 1 --2- 172.21.15.7:0/2826160943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f667415dfb0 0x7f66740b0470 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6674071810 tx=0x7f666802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c716ba162de78aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:18.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.856+0000 7f6673e00640 1 -- 172.21.15.7:0/2826160943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668037070 con 0x7f667415dfb0 2024-04-25T11:13:18.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.856+0000 7f6673e00640 1 -- 172.21.15.7:0/2826160943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6668032440 con 0x7f667415dfb0 2024-04-25T11:13:18.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.856+0000 7f6673e00640 1 -- 172.21.15.7:0/2826160943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668032810 con 0x7f667415dfb0 2024-04-25T11:13:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.856+0000 7f667c000640 1 -- 172.21.15.7:0/2826160943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f667415dfb0 msgr2=0x7f66740b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.857+0000 7f667c000640 1 --2- 172.21.15.7:0/2826160943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f667415dfb0 0x7f66740b0470 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6674071810 tx=0x7f666802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.857+0000 7f667c000640 1 -- 172.21.15.7:0/2826160943 shutdown_connections 2024-04-25T11:13:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.857+0000 7f667c000640 1 --2- 172.21.15.7:0/2826160943 >> v2:172.21.15.87:3300/0 conn(0x7f66740d0f50 0x7f667406f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.857+0000 7f667c000640 1 --2- 172.21.15.7:0/2826160943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66740d0b40 0x7f667406f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.857+0000 7f667c000640 1 --2- 172.21.15.7:0/2826160943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f667415dfb0 0x7f66740b0470 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.857+0000 7f667c000640 1 -- 172.21.15.7:0/2826160943 >> 172.21.15.7:0/2826160943 conn(0x7f66740742c0 msgr2=0x7f667406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:18.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.857+0000 7f667c000640 1 -- 172.21.15.7:0/2826160943 shutdown_connections 2024-04-25T11:13:18.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.857+0000 7f667c000640 1 -- 172.21.15.7:0/2826160943 wait complete. 2024-04-25T11:13:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.860+0000 7f667c000640 1 Processor -- start 2024-04-25T11:13:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.860+0000 7f667c000640 1 -- start start 2024-04-25T11:13:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f667c000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66740d0b40 0x7f6674151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f667c000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66740d0f50 0x7f66741580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f667c000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f667415dfb0 0x7f66741450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f667c000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66740d0340 con 0x7f66740d0f50 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f667c000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f66740d0060 con 0x7f667415dfb0 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f667c000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66740d0620 con 0x7f66740d0b40 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f6678c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66740d0f50 0x7f66741580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f6678c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66740d0f50 0x7f66741580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50610/0 (socket says 172.21.15.7:50610) 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f6678c00640 1 -- 172.21.15.7:0/713092189 learned_addr learned my addr 172.21.15.7:0/713092189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f6679600640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66740d0b40 0x7f6674151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f6678c00640 1 -- 172.21.15.7:0/713092189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66740d0b40 msgr2=0x7f6674151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f6678c00640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66740d0b40 0x7f6674151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f6678c00640 1 -- 172.21.15.7:0/713092189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f667415dfb0 msgr2=0x7f66741450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f6678c00640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f667415dfb0 0x7f66741450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f6678c00640 1 -- 172.21.15.7:0/713092189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6674151350 con 0x7f66740d0f50 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.862+0000 7f6679600640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66740d0b40 0x7f6674151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.863+0000 7f6678c00640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66740d0f50 0x7f66741580b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f666000d800 tx=0x7f666000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.863+0000 7f6671600640 1 -- 172.21.15.7:0/713092189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660018030 con 0x7f66740d0f50 2024-04-25T11:13:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.863+0000 7f6671600640 1 -- 172.21.15.7:0/713092189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66600096a0 con 0x7f66740d0f50 2024-04-25T11:13:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.863+0000 7f667c000640 1 -- 172.21.15.7:0/713092189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66740b4e80 con 0x7f66740d0f50 2024-04-25T11:13:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.863+0000 7f6671600640 1 -- 172.21.15.7:0/713092189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660009870 con 0x7f66740d0f50 2024-04-25T11:13:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.863+0000 7f667c000640 1 -- 172.21.15.7:0/713092189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66741562a0 con 0x7f66740d0f50 2024-04-25T11:13:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.864+0000 7f667c000640 1 -- 172.21.15.7:0/713092189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66740b5b50 con 0x7f66740d0f50 2024-04-25T11:13:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.866+0000 7f6671600640 1 -- 172.21.15.7:0/713092189 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f666001b070 con 0x7f66740d0f50 2024-04-25T11:13:18.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.867+0000 7f6671600640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66540744b0 0x7f66540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:18.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.867+0000 7f6671600640 1 -- 172.21.15.7:0/713092189 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f6660095610 con 0x7f66740d0f50 2024-04-25T11:13:18.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.867+0000 7f6679600640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66540744b0 0x7f66540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:18.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.868+0000 7f6679600640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66540744b0 0x7f66540769c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6668034000 tx=0x7f666802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:18.872+0000 7f6671600640 1 -- 172.21.15.7:0/713092189 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f666005fc70 con 0x7f66740d0f50 2024-04-25T11:13:19.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.073+0000 7f667c000640 1 -- 172.21.15.7:0/713092189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6674050740 con 0x7f66740d0f50 2024-04-25T11:13:19.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.074+0000 7f6671600640 1 -- 172.21.15.7:0/713092189 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) ==== 104+0+11 (secure 0 0 0) 0x7f666005f570 con 0x7f66740d0f50 2024-04-25T11:13:19.074 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:13:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 -- 172.21.15.7:0/713092189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66540744b0 msgr2=0x7f66540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66540744b0 0x7f66540769c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6668034000 tx=0x7f666802e000 comp rx=0 tx=0).stop 2024-04-25T11:13:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 -- 172.21.15.7:0/713092189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66740d0f50 msgr2=0x7f66741580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66740d0f50 0x7f66741580b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f666000d800 tx=0x7f666000dcc0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 -- 172.21.15.7:0/713092189 shutdown_connections 2024-04-25T11:13:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66540744b0 0x7f66540769c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f667415dfb0 0x7f66741450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66740d0f50 0x7f66741580b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 --2- 172.21.15.7:0/713092189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66740d0b40 0x7f6674151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 -- 172.21.15.7:0/713092189 >> 172.21.15.7:0/713092189 conn(0x7f66740742c0 msgr2=0x7f6674157760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 -- 172.21.15.7:0/713092189 shutdown_connections 2024-04-25T11:13:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.077+0000 7f667c000640 1 -- 172.21.15.7:0/713092189 wait complete. 2024-04-25T11:13:19.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:13:19.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.385+0000 7f3411200640 1 Processor -- start 2024-04-25T11:13:19.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.385+0000 7f3411200640 1 -- start start 2024-04-25T11:13:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.387+0000 7f3411200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f340c15dfb0 0x7f340c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.387+0000 7f3411200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f340c0d0b40 0x7f340c06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.387+0000 7f3411200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f340c0d0f50 0x7f340c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.387+0000 7f3411200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f340c0b09b0 con 0x7f340c0d0b40 2024-04-25T11:13:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.387+0000 7f3411200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f340c15e3c0 con 0x7f340c0d0f50 2024-04-25T11:13:19.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.387+0000 7f3411200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f340c0cf800 con 0x7f340c15dfb0 2024-04-25T11:13:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.388+0000 7f3409400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f340c0d0b40 0x7f340c06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.388+0000 7f3409400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f340c0d0b40 0x7f340c06f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50622/0 (socket says 172.21.15.7:50622) 2024-04-25T11:13:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.388+0000 7f3409400640 1 -- 172.21.15.7:0/3880804522 learned_addr learned my addr 172.21.15.7:0/3880804522 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.388+0000 7f3409e00640 1 --2- 172.21.15.7:0/3880804522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f340c15dfb0 0x7f340c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.388+0000 7f340a800640 1 --2- 172.21.15.7:0/3880804522 >> v2:172.21.15.87:3300/0 conn(0x7f340c0d0f50 0x7f340c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.388+0000 7f3409400640 1 -- 172.21.15.7:0/3880804522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f340c15dfb0 msgr2=0x7f340c0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.388+0000 7f3409400640 1 --2- 172.21.15.7:0/3880804522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f340c15dfb0 0x7f340c0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.388+0000 7f3409400640 1 -- 172.21.15.7:0/3880804522 >> v2:172.21.15.87:3300/0 conn(0x7f340c0d0f50 msgr2=0x7f340c06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.388+0000 7f3409400640 1 --2- 172.21.15.7:0/3880804522 >> v2:172.21.15.87:3300/0 conn(0x7f340c0d0f50 0x7f340c06f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.388+0000 7f3409400640 1 -- 172.21.15.7:0/3880804522 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c0cfaa0 con 0x7f340c0d0b40 2024-04-25T11:13:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.389+0000 7f3409400640 1 --2- 172.21.15.7:0/3880804522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f340c0d0b40 0x7f340c06f3a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f340c161a10 tx=0x7f33f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be218c9a20b9b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.390+0000 7f3408a00640 1 -- 172.21.15.7:0/3880804522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4037070 con 0x7f340c0d0b40 2024-04-25T11:13:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.390+0000 7f3408a00640 1 -- 172.21.15.7:0/3880804522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f4032440 con 0x7f340c0d0b40 2024-04-25T11:13:19.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.390+0000 7f3408a00640 1 -- 172.21.15.7:0/3880804522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4032810 con 0x7f340c0d0b40 2024-04-25T11:13:19.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.391+0000 7f3411200640 1 -- 172.21.15.7:0/3880804522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f340c0d0b40 msgr2=0x7f340c06f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:19.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.391+0000 7f3411200640 1 --2- 172.21.15.7:0/3880804522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f340c0d0b40 0x7f340c06f3a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f340c161a10 tx=0x7f33f402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.391+0000 7f3411200640 1 -- 172.21.15.7:0/3880804522 shutdown_connections 2024-04-25T11:13:19.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.391+0000 7f3411200640 1 --2- 172.21.15.7:0/3880804522 >> v2:172.21.15.87:3300/0 conn(0x7f340c0d0f50 0x7f340c06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.391+0000 7f3411200640 1 --2- 172.21.15.7:0/3880804522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f340c0d0b40 0x7f340c06f3a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.391+0000 7f3411200640 1 --2- 172.21.15.7:0/3880804522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f340c15dfb0 0x7f340c0b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.391+0000 7f3411200640 1 -- 172.21.15.7:0/3880804522 >> 172.21.15.7:0/3880804522 conn(0x7f340c0742c0 msgr2=0x7f340c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.391+0000 7f3411200640 1 -- 172.21.15.7:0/3880804522 shutdown_connections 2024-04-25T11:13:19.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.391+0000 7f3411200640 1 -- 172.21.15.7:0/3880804522 wait complete. 2024-04-25T11:13:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.393+0000 7f3411200640 1 Processor -- start 2024-04-25T11:13:19.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.394+0000 7f3411200640 1 -- start start 2024-04-25T11:13:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.395+0000 7f3411200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f340c0d0b40 0x7f340c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.395+0000 7f3411200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f340c0d0f50 0x7f340c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.395+0000 7f3411200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f340c15dfb0 0x7f340c1f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.395+0000 7f3411200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f340c0d0340 con 0x7f340c0d0b40 2024-04-25T11:13:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.395+0000 7f3411200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f340c0d0060 con 0x7f340c0d0f50 2024-04-25T11:13:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.395+0000 7f3411200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f340c0d0620 con 0x7f340c15dfb0 2024-04-25T11:13:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.395+0000 7f3409e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f340c0d0b40 0x7f340c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.395+0000 7f3409e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f340c0d0b40 0x7f340c0cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50624/0 (socket says 172.21.15.7:50624) 2024-04-25T11:13:19.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.395+0000 7f3409e00640 1 -- 172.21.15.7:0/379909913 learned_addr learned my addr 172.21.15.7:0/379909913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.396+0000 7f340a800640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f340c15dfb0 0x7f340c1f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.396+0000 7f3409400640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f340c0d0f50 0x7f340c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.396+0000 7f3409e00640 1 -- 172.21.15.7:0/379909913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f340c15dfb0 msgr2=0x7f340c1f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.396+0000 7f3409e00640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f340c15dfb0 0x7f340c1f9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.396+0000 7f3409e00640 1 -- 172.21.15.7:0/379909913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f340c0d0f50 msgr2=0x7f340c153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.396+0000 7f3409e00640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f340c0d0f50 0x7f340c153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.396+0000 7f3409e00640 1 -- 172.21.15.7:0/379909913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c0caee0 con 0x7f340c0d0b40 2024-04-25T11:13:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.396+0000 7f3409400640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f340c0d0f50 0x7f340c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:19.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.396+0000 7f3409e00640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f340c0d0b40 0x7f340c0cb620 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00d5d0 tx=0x7f33fc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.396+0000 7f3402000640 1 -- 172.21.15.7:0/379909913 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc004240 con 0x7f340c0d0b40 2024-04-25T11:13:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.397+0000 7f3402000640 1 -- 172.21.15.7:0/379909913 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33fc0043e0 con 0x7f340c0d0b40 2024-04-25T11:13:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.397+0000 7f3411200640 1 -- 172.21.15.7:0/379909913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340c0cb1c0 con 0x7f340c0d0b40 2024-04-25T11:13:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.397+0000 7f3402000640 1 -- 172.21.15.7:0/379909913 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc003830 con 0x7f340c0d0b40 2024-04-25T11:13:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.397+0000 7f3411200640 1 -- 172.21.15.7:0/379909913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340c0cdeb0 con 0x7f340c0d0b40 2024-04-25T11:13:19.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.398+0000 7f3411200640 1 -- 172.21.15.7:0/379909913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f340c050740 con 0x7f340c0d0b40 2024-04-25T11:13:19.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.399+0000 7f3402000640 1 -- 172.21.15.7:0/379909913 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f33fc0039d0 con 0x7f340c0d0b40 2024-04-25T11:13:19.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.400+0000 7f3402000640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33e80744e0 0x7f33e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.400+0000 7f3402000640 1 -- 172.21.15.7:0/379909913 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f33fc095610 con 0x7f340c0d0b40 2024-04-25T11:13:19.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.400+0000 7f3409400640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33e80744e0 0x7f33e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.401+0000 7f3409400640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33e80744e0 0x7f33e80769f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f33f4034000 tx=0x7f33f402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:19.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.405+0000 7f3402000640 1 -- 172.21.15.7:0/379909913 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f33fc05fc70 con 0x7f340c0d0b40 2024-04-25T11:13:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.610+0000 7f3411200640 1 -- 172.21.15.7:0/379909913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f340c070fb0 con 0x7f340c0d0b40 2024-04-25T11:13:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.611+0000 7f3402000640 1 -- 172.21.15.7:0/379909913 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) ==== 100+0+11 (secure 0 0 0) 0x7f33fc05f570 con 0x7f340c0d0b40 2024-04-25T11:13:19.612 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.614+0000 7f3411200640 1 -- 172.21.15.7:0/379909913 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33e80744e0 msgr2=0x7f33e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.614+0000 7f3411200640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33e80744e0 0x7f33e80769f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f33f4034000 tx=0x7f33f402e000 comp rx=0 tx=0).stop 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.614+0000 7f3411200640 1 -- 172.21.15.7:0/379909913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f340c0d0b40 msgr2=0x7f340c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.614+0000 7f3411200640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f340c0d0b40 0x7f340c0cb620 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00d5d0 tx=0x7f33fc00da90 comp rx=0 tx=0).stop 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.614+0000 7f3411200640 1 -- 172.21.15.7:0/379909913 shutdown_connections 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.614+0000 7f3411200640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33e80744e0 0x7f33e80769f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.614+0000 7f3411200640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f340c15dfb0 0x7f340c1f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.614+0000 7f3411200640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f340c0d0f50 0x7f340c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.614+0000 7f3411200640 1 --2- 172.21.15.7:0/379909913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f340c0d0b40 0x7f340c0cb620 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.614+0000 7f3411200640 1 -- 172.21.15.7:0/379909913 >> 172.21.15.7:0/379909913 conn(0x7f340c0742c0 msgr2=0x7f340c06f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.614+0000 7f3411200640 1 -- 172.21.15.7:0/379909913 shutdown_connections 2024-04-25T11:13:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.615+0000 7f3411200640 1 -- 172.21.15.7:0/379909913 wait complete. 2024-04-25T11:13:19.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:13:19.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.925+0000 7f8164200640 1 Processor -- start 2024-04-25T11:13:19.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.925+0000 7f8164200640 1 -- start start 2024-04-25T11:13:19.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.927+0000 7f8164200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f815c15f180 0x7f815c161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.927+0000 7f8164200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f815c1621a0 0x7f815c06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.927+0000 7f8164200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f815c06dac0 0x7f815c06f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.927+0000 7f8164200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f815c15f590 con 0x7f815c1621a0 2024-04-25T11:13:19.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.927+0000 7f8164200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f815c158ae0 con 0x7f815c15f180 2024-04-25T11:13:19.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.927+0000 7f8164200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f815c06ded0 con 0x7f815c06dac0 2024-04-25T11:13:19.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.927+0000 7f8160e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f815c1621a0 0x7f815c06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.928+0000 7f8160e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f815c1621a0 0x7f815c06d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45118/0 (socket says 172.21.15.7:45118) 2024-04-25T11:13:19.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.928+0000 7f8160e00640 1 -- 172.21.15.7:0/20831552 learned_addr learned my addr 172.21.15.7:0/20831552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:19.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.928+0000 7f8161800640 1 --2- 172.21.15.7:0/20831552 >> v2:172.21.15.87:3300/0 conn(0x7f815c15f180 0x7f815c161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.928+0000 7f8162200640 1 --2- 172.21.15.7:0/20831552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f815c06dac0 0x7f815c06f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.928+0000 7f8160e00640 1 -- 172.21.15.7:0/20831552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f815c06dac0 msgr2=0x7f815c06f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:19.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.928+0000 7f8160e00640 1 --2- 172.21.15.7:0/20831552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f815c06dac0 0x7f815c06f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.928+0000 7f8160e00640 1 -- 172.21.15.7:0/20831552 >> v2:172.21.15.87:3300/0 conn(0x7f815c15f180 msgr2=0x7f815c161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:19.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.928+0000 7f8160e00640 1 --2- 172.21.15.7:0/20831552 >> v2:172.21.15.87:3300/0 conn(0x7f815c15f180 0x7f815c161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.928+0000 7f8160e00640 1 -- 172.21.15.7:0/20831552 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c162700 con 0x7f815c1621a0 2024-04-25T11:13:19.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.929+0000 7f8160e00640 1 --2- 172.21.15.7:0/20831552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f815c1621a0 0x7f815c06d580 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f815c070340 tx=0x7f814802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0e709fd5d2317f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:19.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.930+0000 7f815be00640 1 -- 172.21.15.7:0/20831552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148037070 con 0x7f815c1621a0 2024-04-25T11:13:19.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.930+0000 7f815be00640 1 -- 172.21.15.7:0/20831552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8148032440 con 0x7f815c1621a0 2024-04-25T11:13:19.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.930+0000 7f815be00640 1 -- 172.21.15.7:0/20831552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148032810 con 0x7f815c1621a0 2024-04-25T11:13:19.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.931+0000 7f8164200640 1 -- 172.21.15.7:0/20831552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f815c1621a0 msgr2=0x7f815c06d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:19.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.931+0000 7f8164200640 1 --2- 172.21.15.7:0/20831552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f815c1621a0 0x7f815c06d580 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f815c070340 tx=0x7f814802a210 comp rx=0 tx=0).stop 2024-04-25T11:13:19.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.931+0000 7f8164200640 1 -- 172.21.15.7:0/20831552 shutdown_connections 2024-04-25T11:13:19.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.931+0000 7f8164200640 1 --2- 172.21.15.7:0/20831552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f815c06dac0 0x7f815c06f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.931+0000 7f8164200640 1 --2- 172.21.15.7:0/20831552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f815c1621a0 0x7f815c06d580 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.931+0000 7f8164200640 1 --2- 172.21.15.7:0/20831552 >> v2:172.21.15.87:3300/0 conn(0x7f815c15f180 0x7f815c161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.931+0000 7f8164200640 1 -- 172.21.15.7:0/20831552 >> 172.21.15.7:0/20831552 conn(0x7f815c0732b0 msgr2=0x7f815c0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:19.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.931+0000 7f8164200640 1 -- 172.21.15.7:0/20831552 shutdown_connections 2024-04-25T11:13:19.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.931+0000 7f8164200640 1 -- 172.21.15.7:0/20831552 wait complete. 2024-04-25T11:13:19.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.933+0000 7f8164200640 1 Processor -- start 2024-04-25T11:13:19.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.934+0000 7f8164200640 1 -- start start 2024-04-25T11:13:19.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.935+0000 7f8164200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f815c06dac0 0x7f815c1469c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.935+0000 7f8164200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f815c15f180 0x7f815c157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.935+0000 7f8164200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f815c1621a0 0x7f815c06f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.935+0000 7f8164200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f815c15ec60 con 0x7f815c15f180 2024-04-25T11:13:19.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.935+0000 7f8164200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f815c15e980 con 0x7f815c1621a0 2024-04-25T11:13:19.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.935+0000 7f8164200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f815c15e670 con 0x7f815c06dac0 2024-04-25T11:13:19.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.935+0000 7f8160e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f815c15f180 0x7f815c157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.935+0000 7f8160e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f815c15f180 0x7f815c157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45128/0 (socket says 172.21.15.7:45128) 2024-04-25T11:13:19.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.935+0000 7f8160e00640 1 -- 172.21.15.7:0/1584656973 learned_addr learned my addr 172.21.15.7:0/1584656973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:19.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.936+0000 7f8161800640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f815c06dac0 0x7f815c1469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.936+0000 7f8162200640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f815c1621a0 0x7f815c06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.936+0000 7f8160e00640 1 -- 172.21.15.7:0/1584656973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f815c06dac0 msgr2=0x7f815c1469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:19.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.936+0000 7f8160e00640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f815c06dac0 0x7f815c1469c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.936+0000 7f8160e00640 1 -- 172.21.15.7:0/1584656973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f815c1621a0 msgr2=0x7f815c06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:19.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.936+0000 7f8160e00640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f815c1621a0 0x7f815c06f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:19.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.936+0000 7f8160e00640 1 -- 172.21.15.7:0/1584656973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c158110 con 0x7f815c15f180 2024-04-25T11:13:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.936+0000 7f8160e00640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f815c15f180 0x7f815c157bd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8148000c00 tx=0x7f81480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.937+0000 7f8159600640 1 -- 172.21.15.7:0/1584656973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148037040 con 0x7f815c15f180 2024-04-25T11:13:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.937+0000 7f8159600640 1 -- 172.21.15.7:0/1584656973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8148042070 con 0x7f815c15f180 2024-04-25T11:13:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.937+0000 7f8164200640 1 -- 172.21.15.7:0/1584656973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c149940 con 0x7f815c15f180 2024-04-25T11:13:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.937+0000 7f8159600640 1 -- 172.21.15.7:0/1584656973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148030940 con 0x7f815c15f180 2024-04-25T11:13:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.937+0000 7f8164200640 1 -- 172.21.15.7:0/1584656973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c149010 con 0x7f815c15f180 2024-04-25T11:13:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.938+0000 7f8164200640 1 -- 172.21.15.7:0/1584656973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815c050740 con 0x7f815c15f180 2024-04-25T11:13:19.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.940+0000 7f8159600640 1 -- 172.21.15.7:0/1584656973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8148030ae0 con 0x7f815c15f180 2024-04-25T11:13:19.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.940+0000 7f8159600640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f813c0744e0 0x7f813c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.941+0000 7f8159600640 1 -- 172.21.15.7:0/1584656973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f81480b5c10 con 0x7f815c15f180 2024-04-25T11:13:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.941+0000 7f8161800640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f813c0744e0 0x7f813c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.942+0000 7f8161800640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f813c0744e0 0x7f813c0769f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f815c070310 tx=0x7f8150008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:19.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:19.946+0000 7f8159600640 1 -- 172.21.15.7:0/1584656973 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f814804c020 con 0x7f815c15f180 2024-04-25T11:13:20.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.163+0000 7f8164200640 1 -- 172.21.15.7:0/1584656973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f815c074250 con 0x7f815c15f180 2024-04-25T11:13:20.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.164+0000 7f8159600640 1 -- 172.21.15.7:0/1584656973 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7f8148080150 con 0x7f815c15f180 2024-04-25T11:13:20.164 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:13:20.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.166+0000 7f8164200640 1 -- 172.21.15.7:0/1584656973 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f813c0744e0 msgr2=0x7f813c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:20.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.166+0000 7f8164200640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f813c0744e0 0x7f813c0769f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f815c070310 tx=0x7f8150008040 comp rx=0 tx=0).stop 2024-04-25T11:13:20.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.166+0000 7f8164200640 1 -- 172.21.15.7:0/1584656973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f815c15f180 msgr2=0x7f815c157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:20.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.166+0000 7f8164200640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f815c15f180 0x7f815c157bd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8148000c00 tx=0x7f81480046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:20.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.166+0000 7f8164200640 1 -- 172.21.15.7:0/1584656973 shutdown_connections 2024-04-25T11:13:20.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.166+0000 7f8164200640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f813c0744e0 0x7f813c0769f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:20.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.166+0000 7f8164200640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f815c1621a0 0x7f815c06f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:20.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.166+0000 7f8164200640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f815c15f180 0x7f815c157bd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:20.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.166+0000 7f8164200640 1 --2- 172.21.15.7:0/1584656973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f815c06dac0 0x7f815c1469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:20.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.166+0000 7f8164200640 1 -- 172.21.15.7:0/1584656973 >> 172.21.15.7:0/1584656973 conn(0x7f815c0732b0 msgr2=0x7f815c1617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:20.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.167+0000 7f8164200640 1 -- 172.21.15.7:0/1584656973 shutdown_connections 2024-04-25T11:13:20.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.167+0000 7f8164200640 1 -- 172.21.15.7:0/1584656973 wait complete. 2024-04-25T11:13:20.195 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:13:20.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:20.195 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T11:13:20.298 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:13:20.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:20.298 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:13:20.349 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:13:20.349 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:13:20.349 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:20.408 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:13:20.408 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:13:20.409 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:13:20.409 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:20.466 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:13:20.466 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:13:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.759+0000 7fc791000640 1 Processor -- start 2024-04-25T11:13:20.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.759+0000 7fc791000640 1 -- start start 2024-04-25T11:13:20.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.761+0000 7fc791000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc78c09c110 0x7fc78c0cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:20.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.761+0000 7fc791000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0d31a0 0x7fc78c0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:20.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.761+0000 7fc791000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc78c0d1cf0 con 0x7fc78c0d2d90 2024-04-25T11:13:20.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.761+0000 7fc791000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc78c0d1fd0 con 0x7fc78c09c110 2024-04-25T11:13:20.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.761+0000 7fc791000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc78c0d22b0 con 0x7fc78c0d31a0 2024-04-25T11:13:20.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.761+0000 7fc78b400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc78c0d2d90 0x7fc78c09ba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42066/0 (socket says 172.21.15.7:42066) 2024-04-25T11:13:20.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.761+0000 7fc78b400640 1 -- 172.21.15.7:0/3335628332 learned_addr learned my addr 172.21.15.7:0/3335628332 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.761+0000 7fc782a00640 1 --2- 172.21.15.7:0/3335628332 >> v2:172.21.15.87:3300/0 conn(0x7fc78c09c110 0x7fc78c0cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.761+0000 7fc78be00640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0d31a0 0x7fc78c0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 182718293 0 0) 0x7fc78c0d1cf0 con 0x7fc78c0d2d90 2024-04-25T11:13:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc770003520 con 0x7fc78c0d2d90 2024-04-25T11:13:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 871305733 0 0) 0x7fc770003520 con 0x7fc78c0d2d90 2024-04-25T11:13:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0d31a0 msgr2=0x7fc78c0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0d31a0 0x7fc78c0ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 >> v2:172.21.15.87:3300/0 conn(0x7fc78c09c110 msgr2=0x7fc78c0cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 --2- 172.21.15.7:0/3335628332 >> v2:172.21.15.87:3300/0 conn(0x7fc78c09c110 0x7fc78c0cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:20.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c0d2590 con 0x7fc78c0d2d90 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fc77c0017f0 con 0x7fc78c0d2d90 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 >> v1:172.21.15.7:6789/0 conn(0x7fc78c0d2d90 legacy=0x7fc78c09ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700039b0 0x7fc770003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0d31a0 0x7fc770004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc770004d40 0x7fc770005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c0d1fd0 con 0x7fc7700039b0 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c0d22b0 con 0x7fc770004d40 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.762+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc77000b9c0 con 0x7fc78c0d31a0 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.763+0000 7fc78b400640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700039b0 0x7fc770003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.763+0000 7fc782a00640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0d31a0 0x7fc770004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.763+0000 7fc78be00640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc770004d40 0x7fc770005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.763+0000 7fc78b400640 1 -- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0d31a0 msgr2=0x7fc770004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.763+0000 7fc78b400640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0d31a0 0x7fc770004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:20.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.763+0000 7fc78b400640 1 -- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc770004d40 msgr2=0x7fc770005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.763+0000 7fc78b400640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc770004d40 0x7fc770005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.763+0000 7fc78b400640 1 -- 172.21.15.7:0/3335628332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc770003520 con 0x7fc7700039b0 2024-04-25T11:13:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.763+0000 7fc782a00640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0d31a0 0x7fc770004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:13:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.764+0000 7fc78b400640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700039b0 0x7fc770003e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c240 tx=0x7fc770005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ab80303a2d5075 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.765+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c007d30 con 0x7fc7700039b0 2024-04-25T11:13:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.765+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77c007070 con 0x7fc7700039b0 2024-04-25T11:13:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.765+0000 7fc78aa00640 1 -- 172.21.15.7:0/3335628332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c007480 con 0x7fc7700039b0 2024-04-25T11:13:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.765+0000 7fc791000640 1 -- 172.21.15.7:0/3335628332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700039b0 msgr2=0x7fc770003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.765+0000 7fc791000640 1 --2- 172.21.15.7:0/3335628332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700039b0 0x7fc770003e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00c240 tx=0x7fc770005830 comp rx=0 tx=0).stop 2024-04-25T11:13:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.765+0000 7fc78be00640 1 -- 172.21.15.7:0/3335628332 reap_dead start 2024-04-25T11:13:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.766+0000 7fc791000640 1 -- 172.21.15.7:0/3335628332 shutdown_connections 2024-04-25T11:13:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.766+0000 7fc791000640 1 -- 172.21.15.7:0/3335628332 >> 172.21.15.7:0/3335628332 conn(0x7fc78c0b5250 msgr2=0x7fc78c0b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.766+0000 7fc791000640 1 -- 172.21.15.7:0/3335628332 shutdown_connections 2024-04-25T11:13:20.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.766+0000 7fc791000640 1 -- 172.21.15.7:0/3335628332 wait complete. 2024-04-25T11:13:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.768+0000 7fc791000640 1 Processor -- start 2024-04-25T11:13:20.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.768+0000 7fc791000640 1 -- start start 2024-04-25T11:13:20.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.770+0000 7fc791000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700039b0 0x7fc78c1f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.770+0000 7fc791000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0c75e0 0x7fc78c0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.770+0000 7fc791000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c0b1b10 0x7fc78c15dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.770+0000 7fc791000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c0b91e0 con 0x7fc78c0b1b10 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.770+0000 7fc791000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c0d19a0 con 0x7fc7700039b0 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.770+0000 7fc791000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc78c0ce2f0 con 0x7fc78c0c75e0 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.770+0000 7fc78be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c0b1b10 0x7fc78c15dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.770+0000 7fc78be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c0b1b10 0x7fc78c15dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45144/0 (socket says 172.21.15.7:45144) 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.770+0000 7fc78be00640 1 -- 172.21.15.7:0/980277757 learned_addr learned my addr 172.21.15.7:0/980277757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.770+0000 7fc78b400640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700039b0 0x7fc78c1f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.770+0000 7fc782a00640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0c75e0 0x7fc78c0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.771+0000 7fc78be00640 1 -- 172.21.15.7:0/980277757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0c75e0 msgr2=0x7fc78c0b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.771+0000 7fc78be00640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0c75e0 0x7fc78c0b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:20.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.771+0000 7fc78be00640 1 -- 172.21.15.7:0/980277757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700039b0 msgr2=0x7fc78c1f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:20.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.771+0000 7fc78be00640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700039b0 0x7fc78c1f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:20.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.771+0000 7fc78be00640 1 -- 172.21.15.7:0/980277757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c0b1f20 con 0x7fc78c0b1b10 2024-04-25T11:13:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.771+0000 7fc78be00640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c0b1b10 0x7fc78c15dbf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc78c09cbb0 tx=0x7fc7840115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.771+0000 7fc783e00640 1 -- 172.21.15.7:0/980277757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784011e40 con 0x7fc78c0b1b10 2024-04-25T11:13:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.771+0000 7fc783e00640 1 -- 172.21.15.7:0/980277757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78401b070 con 0x7fc78c0b1b10 2024-04-25T11:13:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.771+0000 7fc791000640 1 -- 172.21.15.7:0/980277757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c0c9e90 con 0x7fc78c0b1b10 2024-04-25T11:13:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.772+0000 7fc783e00640 1 -- 172.21.15.7:0/980277757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc784003990 con 0x7fc78c0b1b10 2024-04-25T11:13:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.772+0000 7fc791000640 1 -- 172.21.15.7:0/980277757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c1f94e0 con 0x7fc78c0b1b10 2024-04-25T11:13:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.773+0000 7fc791000640 1 -- 172.21.15.7:0/980277757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78c0c99b0 con 0x7fc78c0b1b10 2024-04-25T11:13:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.774+0000 7fc783e00640 1 -- 172.21.15.7:0/980277757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc784005d10 con 0x7fc78c0b1b10 2024-04-25T11:13:20.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.775+0000 7fc783e00640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7680744e0 0x7fc7680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:20.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.775+0000 7fc783e00640 1 -- 172.21.15.7:0/980277757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fc7840957b0 con 0x7fc78c0b1b10 2024-04-25T11:13:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.776+0000 7fc78b400640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7680744e0 0x7fc7680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:20.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.776+0000 7fc78b400640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7680744e0 0x7fc7680769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc77c033000 tx=0x7fc77c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:20.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:20.780+0000 7fc783e00640 1 -- 172.21.15.7:0/980277757 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc78c0c99b0 con 0x7fc78c0b1b10 2024-04-25T11:13:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.028+0000 7fc791000640 1 -- 172.21.15.7:0/980277757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc78c050740 con 0x7fc78c0b1b10 2024-04-25T11:13:21.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.030+0000 7fc783e00640 1 -- 172.21.15.7:0/980277757 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5172 (secure 0 0 0) 0x7fc78c0b1f20 con 0x7fc78c0b1b10 2024-04-25T11:13:21.030 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:21.031 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2575517671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2575517671},{"type":"v1","addr":"172.21.15.87:6835","nonce":2575517671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/3746682343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3746682343},{"type":"v1","addr":"172.21.15.7:6836","nonce":3746682343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2132415318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2132415318},{"type":"v1","addr":"172.21.15.7:6837","nonce":2132415318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:12:17.156571+0000","modified":"2024-04-25T11:12:20.224854+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6833/389115631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":389115631},{"type":"v1","addr":"172.21.15.87:6833","nonce":389115631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5283,"qdb_cluster":[5283]},"id":2}]} 2024-04-25T11:13:21.031 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2024-04-25T11:13:21.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.032+0000 7fc791000640 1 -- 172.21.15.7:0/980277757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7680744e0 msgr2=0x7fc7680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.032+0000 7fc791000640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7680744e0 0x7fc7680769f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc77c033000 tx=0x7fc77c009030 comp rx=0 tx=0).stop 2024-04-25T11:13:21.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.032+0000 7fc791000640 1 -- 172.21.15.7:0/980277757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c0b1b10 msgr2=0x7fc78c15dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.032+0000 7fc791000640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c0b1b10 0x7fc78c15dbf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc78c09cbb0 tx=0x7fc7840115a0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.032+0000 7fc791000640 1 -- 172.21.15.7:0/980277757 shutdown_connections 2024-04-25T11:13:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.032+0000 7fc791000640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7680744e0 0x7fc7680769f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.032+0000 7fc791000640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc78c0b1b10 0x7fc78c15dbf0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.032+0000 7fc791000640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc78c0c75e0 0x7fc78c0b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.032+0000 7fc791000640 1 --2- 172.21.15.7:0/980277757 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700039b0 0x7fc78c1f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.032+0000 7fc791000640 1 -- 172.21.15.7:0/980277757 >> 172.21.15.7:0/980277757 conn(0x7fc78c0b5250 msgr2=0x7fc78c0cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.033+0000 7fc791000640 1 -- 172.21.15.7:0/980277757 shutdown_connections 2024-04-25T11:13:21.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.033+0000 7fc791000640 1 -- 172.21.15.7:0/980277757 wait complete. 2024-04-25T11:13:21.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:21.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.344+0000 7f8575400640 1 Processor -- start 2024-04-25T11:13:21.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.344+0000 7f8575400640 1 -- start start 2024-04-25T11:13:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.347+0000 7f8575400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f85700aed40 0x7f857015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.347+0000 7f8575400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85700cebb0 con 0x7f85700b5220 2024-04-25T11:13:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.347+0000 7f8575400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85700b64b0 con 0x7f85700af420 2024-04-25T11:13:21.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.347+0000 7f8575400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f857015e1f0 con 0x7f85700aed40 2024-04-25T11:13:21.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.348+0000 7f856f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f85700b5220 0x7f85700ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42072/0 (socket says 172.21.15.7:42072) 2024-04-25T11:13:21.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.348+0000 7f856f400640 1 -- 172.21.15.7:0/3516781579 learned_addr learned my addr 172.21.15.7:0/3516781579 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:21.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.348+0000 7f856fe00640 1 --2- 172.21.15.7:0/3516781579 >> v2:172.21.15.87:3300/0 conn(0x7f85700aed40 0x7f857015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.348+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3086739617 0 0) 0x7f855c001110 con 0x7f85700b5220 2024-04-25T11:13:21.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.348+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8554003520 con 0x7f85700b5220 2024-04-25T11:13:21.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1318273457 0 0) 0x7f8554003520 con 0x7f85700b5220 2024-04-25T11:13:21.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 >> v2:172.21.15.87:3300/0 conn(0x7f85700aed40 msgr2=0x7f857015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:21.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 --2- 172.21.15.7:0/3516781579 >> v2:172.21.15.87:3300/0 conn(0x7f85700aed40 0x7f857015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 >> v1:172.21.15.87:6789/0 conn(0x7f85700af420 legacy=0x7f85700af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857006c610 con 0x7f85700b5220 2024-04-25T11:13:21.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f855c001840 con 0x7f85700b5220 2024-04-25T11:13:21.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 >> v1:172.21.15.7:6789/0 conn(0x7f85700b5220 legacy=0x7f85700ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85540039b0 0x7f8554003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8554004800 0x7f8554004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85540051c0 0x7f855400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f857015e1f0 con 0x7f8554004800 2024-04-25T11:13:21.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f855400c160 con 0x7f85540039b0 2024-04-25T11:13:21.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8554010770 con 0x7f85540051c0 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856f400640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8554004800 0x7f8554004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f856fe00640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85540039b0 0x7f8554003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.349+0000 7f8574a00640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85540051c0 0x7f855400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.350+0000 7f856f400640 1 -- 172.21.15.7:0/3516781579 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85540051c0 msgr2=0x7f855400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.350+0000 7f856f400640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85540051c0 0x7f855400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.350+0000 7f856f400640 1 -- 172.21.15.7:0/3516781579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85540039b0 msgr2=0x7f8554003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.350+0000 7f856f400640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85540039b0 0x7f8554003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.350+0000 7f856f400640 1 -- 172.21.15.7:0/3516781579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8554003520 con 0x7f8554004800 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.350+0000 7f8574a00640 1 -- 172.21.15.7:0/3516781579 reap_dead start 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.350+0000 7f856fe00640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85540039b0 0x7f8554003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.351+0000 7f856f400640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8554004800 0x7f8554004c10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f855c00c4c0 tx=0x7f8554005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8755b773fba9b9e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:21.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.351+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c00c0e0 con 0x7f8554004800 2024-04-25T11:13:21.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.351+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c034070 con 0x7f8554004800 2024-04-25T11:13:21.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.351+0000 7f856ea00640 1 -- 172.21.15.7:0/3516781579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c005d70 con 0x7f8554004800 2024-04-25T11:13:21.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.351+0000 7f8575400640 1 -- 172.21.15.7:0/3516781579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8554004800 msgr2=0x7f8554004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.351+0000 7f8575400640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8554004800 0x7f8554004c10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f855c00c4c0 tx=0x7f8554005830 comp rx=0 tx=0).stop 2024-04-25T11:13:21.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.352+0000 7f8575400640 1 -- 172.21.15.7:0/3516781579 shutdown_connections 2024-04-25T11:13:21.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.352+0000 7f8575400640 1 --2- 172.21.15.7:0/3516781579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8554004800 0x7f8554004c10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.352+0000 7f8575400640 1 -- 172.21.15.7:0/3516781579 >> 172.21.15.7:0/3516781579 conn(0x7f85700b3820 msgr2=0x7f85700b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:21.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.352+0000 7f8575400640 1 -- 172.21.15.7:0/3516781579 shutdown_connections 2024-04-25T11:13:21.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.352+0000 7f8575400640 1 -- 172.21.15.7:0/3516781579 wait complete. 2024-04-25T11:13:21.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.354+0000 7f8575400640 1 Processor -- start 2024-04-25T11:13:21.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.354+0000 7f8575400640 1 -- start start 2024-04-25T11:13:21.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.356+0000 7f8575400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8554004800 0x7f857015f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.356+0000 7f8575400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85701452b0 0x7f85701561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.356+0000 7f8575400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8570151280 0x7f8570161700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.356+0000 7f8575400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f857006ceb0 con 0x7f85701452b0 2024-04-25T11:13:21.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.356+0000 7f8575400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f857006cbd0 con 0x7f8554004800 2024-04-25T11:13:21.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.356+0000 7f8575400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f85700b6110 con 0x7f8570151280 2024-04-25T11:13:21.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.356+0000 7f856f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85701452b0 0x7f85701561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.356+0000 7f856f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85701452b0 0x7f85701561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45162/0 (socket says 172.21.15.7:45162) 2024-04-25T11:13:21.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.356+0000 7f856f400640 1 -- 172.21.15.7:0/3341958656 learned_addr learned my addr 172.21.15.7:0/3341958656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:21.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.356+0000 7f856fe00640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8554004800 0x7f857015f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.356+0000 7f8574a00640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8570151280 0x7f8570161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f856f400640 1 -- 172.21.15.7:0/3341958656 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8570151280 msgr2=0x7f8570161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f856f400640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8570151280 0x7f8570161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f856f400640 1 -- 172.21.15.7:0/3341958656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8554004800 msgr2=0x7f857015f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f856f400640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8554004800 0x7f857015f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f856f400640 1 -- 172.21.15.7:0/3341958656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85700d13b0 con 0x7f85701452b0 2024-04-25T11:13:21.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f8574a00640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8570151280 0x7f8570161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:21.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f856f400640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85701452b0 0x7f85701561a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f855c031040 tx=0x7f857006c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:21.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f8567e00640 1 -- 172.21.15.7:0/3341958656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c007270 con 0x7f85701452b0 2024-04-25T11:13:21.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f8567e00640 1 -- 172.21.15.7:0/3341958656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c03f030 con 0x7f85701452b0 2024-04-25T11:13:21.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f8567e00640 1 -- 172.21.15.7:0/3341958656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c007630 con 0x7f85701452b0 2024-04-25T11:13:21.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f8575400640 1 -- 172.21.15.7:0/3341958656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857015e9a0 con 0x7f85701452b0 2024-04-25T11:13:21.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.357+0000 7f8575400640 1 -- 172.21.15.7:0/3341958656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857015ef60 con 0x7f85701452b0 2024-04-25T11:13:21.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.359+0000 7f8575400640 1 -- 172.21.15.7:0/3341958656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85700b0360 con 0x7f85701452b0 2024-04-25T11:13:21.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.359+0000 7f8567e00640 1 -- 172.21.15.7:0/3341958656 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f855c02f460 con 0x7f85701452b0 2024-04-25T11:13:21.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.360+0000 7f8567e00640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85500744b0 0x7f85500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.361+0000 7f8567e00640 1 -- 172.21.15.7:0/3341958656 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f855c0b5e40 con 0x7f85701452b0 2024-04-25T11:13:21.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.361+0000 7f856fe00640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85500744b0 0x7f85500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.361+0000 7f856fe00640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85500744b0 0x7f85500769c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f857006c310 tx=0x7f8560008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:21.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.366+0000 7f8567e00640 1 -- 172.21.15.7:0/3341958656 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85700b0360 con 0x7f85701452b0 2024-04-25T11:13:21.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.565+0000 7f8575400640 1 -- 172.21.15.7:0/3341958656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8570050740 con 0x7f85701452b0 2024-04-25T11:13:21.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.568+0000 7f8567e00640 1 -- 172.21.15.7:0/3341958656 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16839 (secure 0 0 0) 0x7f855c049030 con 0x7f85701452b0 2024-04-25T11:13:21.569 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:21.569 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:04.193129+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:12:14.961811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T11:12:16.128549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 -- 172.21.15.7:0/3341958656 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85500744b0 msgr2=0x7f85500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85500744b0 0x7f85500769c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f857006c310 tx=0x7f8560008040 comp rx=0 tx=0).stop 2024-04-25T11:13:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 -- 172.21.15.7:0/3341958656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85701452b0 msgr2=0x7f85701561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85701452b0 0x7f85701561a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f855c031040 tx=0x7f857006c7e0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 -- 172.21.15.7:0/3341958656 shutdown_connections 2024-04-25T11:13:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85500744b0 0x7f85500769c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8570151280 0x7f8570161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85701452b0 0x7f85701561a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 --2- 172.21.15.7:0/3341958656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8554004800 0x7f857015f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 -- 172.21.15.7:0/3341958656 >> 172.21.15.7:0/3341958656 conn(0x7f85700b3820 msgr2=0x7f85700ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 -- 172.21.15.7:0/3341958656 shutdown_connections 2024-04-25T11:13:21.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.571+0000 7f8575400640 1 -- 172.21.15.7:0/3341958656 wait complete. 2024-04-25T11:13:21.597 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:12:14.961811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:12:16.128549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:13:21.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.882+0000 7fc325600640 1 Processor -- start 2024-04-25T11:13:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.882+0000 7fc325600640 1 -- start start 2024-04-25T11:13:21.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.884+0000 7fc325600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc320144a80 0x7fc320146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.884+0000 7fc325600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc3200b4f80 0x7fc3200b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.884+0000 7fc325600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3200b66b0 con 0x7fc320145e40 2024-04-25T11:13:21.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.884+0000 7fc325600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc320145b70 con 0x7fc320144a80 2024-04-25T11:13:21.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.884+0000 7fc325600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc3200b58d0 con 0x7fc3200b4f80 2024-04-25T11:13:21.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.885+0000 7fc31f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc320144a80 0x7fc320146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.885+0000 7fc31f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc320144a80 0x7fc320146250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45164/0 (socket says 172.21.15.7:45164) 2024-04-25T11:13:21.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.885+0000 7fc31f400640 1 -- 172.21.15.7:0/2073597854 learned_addr learned my addr 172.21.15.7:0/2073597854 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:21.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.885+0000 7fc324c00640 1 --2- 172.21.15.7:0/2073597854 >> v2:172.21.15.87:3300/0 conn(0x7fc3200b4f80 0x7fc3200b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.885+0000 7fc31f400640 1 -- 172.21.15.7:0/2073597854 >> v2:172.21.15.87:3300/0 conn(0x7fc3200b4f80 msgr2=0x7fc3200b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:21.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.885+0000 7fc31f400640 1 --2- 172.21.15.7:0/2073597854 >> v2:172.21.15.87:3300/0 conn(0x7fc3200b4f80 0x7fc3200b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.885+0000 7fc31f400640 1 -- 172.21.15.7:0/2073597854 >> v1:172.21.15.87:6789/0 conn(0x7fc320145e40 legacy=0x7fc3200b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:21.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.886+0000 7fc31f400640 1 -- 172.21.15.7:0/2073597854 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3200b5b30 con 0x7fc320144a80 2024-04-25T11:13:21.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.886+0000 7fc324c00640 1 --2- 172.21.15.7:0/2073597854 >> v2:172.21.15.87:3300/0 conn(0x7fc3200b4f80 0x7fc3200b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:21.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.887+0000 7fc31f400640 1 --2- 172.21.15.7:0/2073597854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc320144a80 0x7fc320146250 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc3200cfca0 tx=0x7fc30c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d197633e0d8ed166 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:21.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.887+0000 7fc31ea00640 1 -- 172.21.15.7:0/2073597854 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c037070 con 0x7fc320144a80 2024-04-25T11:13:21.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.887+0000 7fc31ea00640 1 -- 172.21.15.7:0/2073597854 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc30c032440 con 0x7fc320144a80 2024-04-25T11:13:21.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.887+0000 7fc31ea00640 1 -- 172.21.15.7:0/2073597854 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30c032810 con 0x7fc320144a80 2024-04-25T11:13:21.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.888+0000 7fc325600640 1 -- 172.21.15.7:0/2073597854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc320144a80 msgr2=0x7fc320146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.888+0000 7fc325600640 1 --2- 172.21.15.7:0/2073597854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc320144a80 0x7fc320146250 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc3200cfca0 tx=0x7fc30c02a210 comp rx=0 tx=0).stop 2024-04-25T11:13:21.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.888+0000 7fc325600640 1 -- 172.21.15.7:0/2073597854 shutdown_connections 2024-04-25T11:13:21.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.888+0000 7fc325600640 1 --2- 172.21.15.7:0/2073597854 >> v2:172.21.15.87:3300/0 conn(0x7fc3200b4f80 0x7fc3200b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.888+0000 7fc325600640 1 --2- 172.21.15.7:0/2073597854 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc320144a80 0x7fc320146250 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.888+0000 7fc325600640 1 -- 172.21.15.7:0/2073597854 >> 172.21.15.7:0/2073597854 conn(0x7fc3200740f0 msgr2=0x7fc32006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:21.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.888+0000 7fc325600640 1 -- 172.21.15.7:0/2073597854 shutdown_connections 2024-04-25T11:13:21.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.888+0000 7fc325600640 1 -- 172.21.15.7:0/2073597854 wait complete. 2024-04-25T11:13:21.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.890+0000 7fc325600640 1 Processor -- start 2024-04-25T11:13:21.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.890+0000 7fc325600640 1 -- start start 2024-04-25T11:13:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.892+0000 7fc325600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200b4f80 0x7fc32015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.892+0000 7fc325600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc320144a80 0x7fc32013ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.892+0000 7fc325600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc320145e40 0x7fc32013f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.892+0000 7fc325600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3200b63d0 con 0x7fc3200b4f80 2024-04-25T11:13:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.892+0000 7fc325600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc3200b60f0 con 0x7fc320144a80 2024-04-25T11:13:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.892+0000 7fc325600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc3201606f0 con 0x7fc320145e40 2024-04-25T11:13:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.892+0000 7fc31fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200b4f80 0x7fc32015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.892+0000 7fc31fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200b4f80 0x7fc32015c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45172/0 (socket says 172.21.15.7:45172) 2024-04-25T11:13:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.892+0000 7fc31fe00640 1 -- 172.21.15.7:0/39827776 learned_addr learned my addr 172.21.15.7:0/39827776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.893+0000 7fc31f400640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc320144a80 0x7fc32013ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.893+0000 7fc324c00640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc320145e40 0x7fc32013f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.893+0000 7fc31fe00640 1 -- 172.21.15.7:0/39827776 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc320145e40 msgr2=0x7fc32013f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.893+0000 7fc31fe00640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc320145e40 0x7fc32013f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.893+0000 7fc31fe00640 1 -- 172.21.15.7:0/39827776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc320144a80 msgr2=0x7fc32013ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:21.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.893+0000 7fc31fe00640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc320144a80 0x7fc32013ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:21.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.893+0000 7fc31fe00640 1 -- 172.21.15.7:0/39827776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32013d8c0 con 0x7fc3200b4f80 2024-04-25T11:13:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.893+0000 7fc31fe00640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200b4f80 0x7fc32015c410 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc31000c390 tx=0x7fc31000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.893+0000 7fc317e00640 1 -- 172.21.15.7:0/39827776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310017070 con 0x7fc3200b4f80 2024-04-25T11:13:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.894+0000 7fc317e00640 1 -- 172.21.15.7:0/39827776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3100052d0 con 0x7fc3200b4f80 2024-04-25T11:13:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.894+0000 7fc317e00640 1 -- 172.21.15.7:0/39827776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310003830 con 0x7fc3200b4f80 2024-04-25T11:13:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.894+0000 7fc325600640 1 -- 172.21.15.7:0/39827776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32013dba0 con 0x7fc3200b4f80 2024-04-25T11:13:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.894+0000 7fc325600640 1 -- 172.21.15.7:0/39827776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32013e160 con 0x7fc3200b4f80 2024-04-25T11:13:21.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.895+0000 7fc325600640 1 -- 172.21.15.7:0/39827776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc320050740 con 0x7fc3200b4f80 2024-04-25T11:13:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.896+0000 7fc317e00640 1 -- 172.21.15.7:0/39827776 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc31001a070 con 0x7fc3200b4f80 2024-04-25T11:13:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.897+0000 7fc317e00640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3000744b0 0x7fc3000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.897+0000 7fc317e00640 1 -- 172.21.15.7:0/39827776 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fc3100946d0 con 0x7fc3200b4f80 2024-04-25T11:13:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.897+0000 7fc31f400640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3000744b0 0x7fc3000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:21.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.898+0000 7fc31f400640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3000744b0 0x7fc3000769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc30c034000 tx=0x7fc30c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:21.902+0000 7fc317e00640 1 -- 172.21.15.7:0/39827776 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc31005ed40 con 0x7fc3200b4f80 2024-04-25T11:13:22.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.106+0000 7fc325600640 1 -- 172.21.15.7:0/39827776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc32015f830 con 0x7fc3200b4f80 2024-04-25T11:13:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.107+0000 7fc317e00640 1 -- 172.21.15.7:0/39827776 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7fc31005e640 con 0x7fc3200b4f80 2024-04-25T11:13:22.107 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:13:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 -- 172.21.15.7:0/39827776 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3000744b0 msgr2=0x7fc3000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3000744b0 0x7fc3000769c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc30c034000 tx=0x7fc30c02e000 comp rx=0 tx=0).stop 2024-04-25T11:13:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 -- 172.21.15.7:0/39827776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200b4f80 msgr2=0x7fc32015c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200b4f80 0x7fc32015c410 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc31000c390 tx=0x7fc31000c850 comp rx=0 tx=0).stop 2024-04-25T11:13:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 -- 172.21.15.7:0/39827776 shutdown_connections 2024-04-25T11:13:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3000744b0 0x7fc3000769c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc320145e40 0x7fc32013f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc320144a80 0x7fc32013ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 --2- 172.21.15.7:0/39827776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200b4f80 0x7fc32015c410 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 -- 172.21.15.7:0/39827776 >> 172.21.15.7:0/39827776 conn(0x7fc3200740f0 msgr2=0x7fc32015fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 -- 172.21.15.7:0/39827776 shutdown_connections 2024-04-25T11:13:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.110+0000 7fc325600640 1 -- 172.21.15.7:0/39827776 wait complete. 2024-04-25T11:13:22.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:13:22.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.425+0000 7f8cf2400640 1 Processor -- start 2024-04-25T11:13:22.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.425+0000 7f8cf2400640 1 -- start start 2024-04-25T11:13:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.428+0000 7f8cf2400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8cec15dfb0 0x7f8cec0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.428+0000 7f8cf2400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cec15e3c0 con 0x7f8cec06fab0 2024-04-25T11:13:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.428+0000 7f8cf2400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cec0cf800 con 0x7f8cec0d0b40 2024-04-25T11:13:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.428+0000 7f8cf2400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8cec0b09b0 con 0x7f8cec15dfb0 2024-04-25T11:13:22.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.428+0000 7f8cebe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8cec06fab0 0x7f8cec0aea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42080/0 (socket says 172.21.15.7:42080) 2024-04-25T11:13:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.428+0000 7f8cebe00640 1 -- 172.21.15.7:0/2662203280 learned_addr learned my addr 172.21.15.7:0/2662203280 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:22.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.429+0000 7f8ceb400640 1 --2- 172.21.15.7:0/2662203280 >> v2:172.21.15.87:3300/0 conn(0x7f8cec15dfb0 0x7f8cec0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.429+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1719976372 0 0) 0x7f8cec15e3c0 con 0x7f8cec06fab0 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.429+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8cd0003520 con 0x7f8cec06fab0 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.429+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2096519296 0 0) 0x7f8cd0003520 con 0x7f8cec06fab0 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.429+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 >> v2:172.21.15.87:3300/0 conn(0x7f8cec15dfb0 msgr2=0x7f8cec0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.429+0000 7f8ceaa00640 1 --2- 172.21.15.7:0/2662203280 >> v2:172.21.15.87:3300/0 conn(0x7f8cec15dfb0 0x7f8cec0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.429+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 >> v1:172.21.15.87:6789/0 conn(0x7f8cec0d0b40 legacy=0x7f8cec06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.429+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cec0cfaa0 con 0x7f8cec06fab0 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.430+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f8ce40019f0 con 0x7f8cec06fab0 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.430+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 >> v1:172.21.15.7:6789/0 conn(0x7f8cec06fab0 legacy=0x7f8cec0aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.430+0000 7f8ceaa00640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8cd00039b0 0x7f8cd0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.430+0000 7f8ceaa00640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8cd0004800 0x7f8cd0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.430+0000 7f8ceaa00640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd0005270 0x7f8cd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.430+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec0b09b0 con 0x7f8cd0005270 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.430+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd000c190 con 0x7f8cd00039b0 2024-04-25T11:13:22.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.430+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd000c390 con 0x7f8cd0004800 2024-04-25T11:13:22.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.430+0000 7f8cebe00640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd0005270 0x7f8cd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.430+0000 7f8ceb400640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8cd00039b0 0x7f8cd0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.430+0000 7f8ce3e00640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8cd0004800 0x7f8cd0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.431+0000 7f8cebe00640 1 -- 172.21.15.7:0/2662203280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8cd0004800 msgr2=0x7f8cd0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:22.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.431+0000 7f8cebe00640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8cd0004800 0x7f8cd0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.431+0000 7f8cebe00640 1 -- 172.21.15.7:0/2662203280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8cd00039b0 msgr2=0x7f8cd0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:22.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.431+0000 7f8cebe00640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8cd00039b0 0x7f8cd0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.431+0000 7f8cebe00640 1 -- 172.21.15.7:0/2662203280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd0003520 con 0x7f8cd0005270 2024-04-25T11:13:22.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.431+0000 7f8ce3e00640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8cd0004800 0x7f8cd0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.431+0000 7f8cebe00640 1 -- 172.21.15.7:0/2662203280 reap_dead start 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.432+0000 7f8cebe00640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd0005270 0x7f8cd000b9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400a860 tx=0x7f8cd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da547065d1f01fed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.432+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400a480 con 0x7f8cd0005270 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.432+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ce402f070 con 0x7f8cd0005270 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.432+0000 7f8ceaa00640 1 -- 172.21.15.7:0/2662203280 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce402bcc0 con 0x7f8cd0005270 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.433+0000 7f8cf2400640 1 -- 172.21.15.7:0/2662203280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd0005270 msgr2=0x7f8cd000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.433+0000 7f8cf2400640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd0005270 0x7f8cd000b9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400a860 tx=0x7f8cd0005830 comp rx=0 tx=0).stop 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.433+0000 7f8cf2400640 1 -- 172.21.15.7:0/2662203280 shutdown_connections 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.433+0000 7f8cf2400640 1 --2- 172.21.15.7:0/2662203280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd0005270 0x7f8cd000b9c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.433+0000 7f8cf2400640 1 -- 172.21.15.7:0/2662203280 >> 172.21.15.7:0/2662203280 conn(0x7f8cec0742c0 msgr2=0x7f8cec06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.433+0000 7f8cf2400640 1 -- 172.21.15.7:0/2662203280 shutdown_connections 2024-04-25T11:13:22.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.433+0000 7f8cf2400640 1 -- 172.21.15.7:0/2662203280 wait complete. 2024-04-25T11:13:22.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.435+0000 7f8cf2400640 1 Processor -- start 2024-04-25T11:13:22.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.435+0000 7f8cf2400640 1 -- start start 2024-04-25T11:13:22.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.437+0000 7f8cf2400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8cd0005270 0x7f8cec140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.437+0000 7f8cf2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec15b770 0x7f8cec155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.437+0000 7f8cf2400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8cec15bb80 0x7f8cec155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.437+0000 7f8cf2400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec0d0340 con 0x7f8cec15b770 2024-04-25T11:13:22.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.437+0000 7f8cf2400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8cec0d0060 con 0x7f8cec15bb80 2024-04-25T11:13:22.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.437+0000 7f8cf2400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8cec0d0620 con 0x7f8cd0005270 2024-04-25T11:13:22.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.437+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec15b770 0x7f8cec155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.437+0000 7f8ce3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec15b770 0x7f8cec155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45198/0 (socket says 172.21.15.7:45198) 2024-04-25T11:13:22.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.437+0000 7f8ce3e00640 1 -- 172.21.15.7:0/1763976822 learned_addr learned my addr 172.21.15.7:0/1763976822 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8cebe00640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8cec15bb80 0x7f8cec155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8ceb400640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8cd0005270 0x7f8cec140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8ce3e00640 1 -- 172.21.15.7:0/1763976822 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8cd0005270 msgr2=0x7f8cec140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8ce3e00640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8cd0005270 0x7f8cec140a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8ce3e00640 1 -- 172.21.15.7:0/1763976822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8cec15bb80 msgr2=0x7f8cec155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8ce3e00640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8cec15bb80 0x7f8cec155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8ce3e00640 1 -- 172.21.15.7:0/1763976822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cec15e460 con 0x7f8cec15b770 2024-04-25T11:13:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8ceb400640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8cd0005270 0x7f8cec140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8ce3e00640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec15b770 0x7f8cec155590 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40016d0 tx=0x7f8cd4001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:22.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8ce3400640 1 -- 172.21.15.7:0/1763976822 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd401c070 con 0x7f8cec15b770 2024-04-25T11:13:22.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8ce3400640 1 -- 172.21.15.7:0/1763976822 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cd4011660 con 0x7f8cec15b770 2024-04-25T11:13:22.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8ce3400640 1 -- 172.21.15.7:0/1763976822 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4011a30 con 0x7f8cec15b770 2024-04-25T11:13:22.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.438+0000 7f8cf2400640 1 -- 172.21.15.7:0/1763976822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cec15c240 con 0x7f8cec15b770 2024-04-25T11:13:22.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.439+0000 7f8cf2400640 1 -- 172.21.15.7:0/1763976822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cec15e9b0 con 0x7f8cec15b770 2024-04-25T11:13:22.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.440+0000 7f8cf2400640 1 -- 172.21.15.7:0/1763976822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cec050740 con 0x7f8cec15b770 2024-04-25T11:13:22.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.441+0000 7f8ce3400640 1 -- 172.21.15.7:0/1763976822 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8cd401f070 con 0x7f8cec15b770 2024-04-25T11:13:22.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.443+0000 7f8ce3400640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8cc80744e0 0x7f8cc80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.443+0000 7f8ce3400640 1 -- 172.21.15.7:0/1763976822 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f8cd4098a50 con 0x7f8cec15b770 2024-04-25T11:13:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.443+0000 7f8ceb400640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8cc80744e0 0x7f8cc80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.444+0000 7f8ceb400640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8cc80744e0 0x7f8cc80769f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8cec072df0 tx=0x7f8cdc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.448+0000 7f8ce3400640 1 -- 172.21.15.7:0/1763976822 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8cd4062a90 con 0x7f8cec15b770 2024-04-25T11:13:22.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.653+0000 7f8cf2400640 1 -- 172.21.15.7:0/1763976822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8cec070fb0 con 0x7f8cec15b770 2024-04-25T11:13:22.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.654+0000 7f8ce3400640 1 -- 172.21.15.7:0/1763976822 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) ==== 104+0+11 (secure 0 0 0) 0x7f8cd4066d90 con 0x7f8cec15b770 2024-04-25T11:13:22.654 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:13:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 -- 172.21.15.7:0/1763976822 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8cc80744e0 msgr2=0x7f8cc80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8cc80744e0 0x7f8cc80769f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8cec072df0 tx=0x7f8cdc008040 comp rx=0 tx=0).stop 2024-04-25T11:13:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 -- 172.21.15.7:0/1763976822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec15b770 msgr2=0x7f8cec155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec15b770 0x7f8cec155590 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40016d0 tx=0x7f8cd4001b90 comp rx=0 tx=0).stop 2024-04-25T11:13:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 -- 172.21.15.7:0/1763976822 shutdown_connections 2024-04-25T11:13:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8cc80744e0 0x7f8cc80769f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8cec15bb80 0x7f8cec155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cec15b770 0x7f8cec155590 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 --2- 172.21.15.7:0/1763976822 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8cd0005270 0x7f8cec140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 -- 172.21.15.7:0/1763976822 >> 172.21.15.7:0/1763976822 conn(0x7f8cec0742c0 msgr2=0x7f8cec0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 -- 172.21.15.7:0/1763976822 shutdown_connections 2024-04-25T11:13:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.657+0000 7f8cf2400640 1 -- 172.21.15.7:0/1763976822 wait complete. 2024-04-25T11:13:22.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:13:22.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.972+0000 7fc7c6a00640 1 Processor -- start 2024-04-25T11:13:22.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.973+0000 7fc7c6a00640 1 -- start start 2024-04-25T11:13:22.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.975+0000 7fc7c6a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7c00b5580 0x7fc7c00cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.975+0000 7fc7c6a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7c00cf560 0x7fc7c0144220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.975+0000 7fc7c6a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7c00b5990 con 0x7fc7c00b4690 2024-04-25T11:13:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.975+0000 7fc7c6a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c00cf970 con 0x7fc7c00cf560 2024-04-25T11:13:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.975+0000 7fc7c6a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c00b5c40 con 0x7fc7c00b5580 2024-04-25T11:13:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.976+0000 7fc7c6000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7c00cf560 0x7fc7c0144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.976+0000 7fc7c6000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7c00cf560 0x7fc7c0144220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45202/0 (socket says 172.21.15.7:45202) 2024-04-25T11:13:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.976+0000 7fc7c6000640 1 -- 172.21.15.7:0/981875381 learned_addr learned my addr 172.21.15.7:0/981875381 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:22.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.976+0000 7fc7c4c00640 1 --2- 172.21.15.7:0/981875381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7c00b5580 0x7fc7c00cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.976+0000 7fc7c6000640 1 -- 172.21.15.7:0/981875381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7c00b5580 msgr2=0x7fc7c00cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:22.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.976+0000 7fc7c6000640 1 --2- 172.21.15.7:0/981875381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7c00b5580 0x7fc7c00cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.976+0000 7fc7c6000640 1 -- 172.21.15.7:0/981875381 >> v1:172.21.15.87:6789/0 conn(0x7fc7c00b4690 legacy=0x7fc7c0143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.976+0000 7fc7c6000640 1 -- 172.21.15.7:0/981875381 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c00b5f20 con 0x7fc7c00cf560 2024-04-25T11:13:22.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.977+0000 7fc7c6000640 1 --2- 172.21.15.7:0/981875381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7c00cf560 0x7fc7c0144220 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009e50 tx=0x7fc7b802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49948518450b1302 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.978+0000 7fc7bfe00640 1 -- 172.21.15.7:0/981875381 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8036070 con 0x7fc7c00cf560 2024-04-25T11:13:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.978+0000 7fc7bfe00640 1 -- 172.21.15.7:0/981875381 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b8031600 con 0x7fc7c00cf560 2024-04-25T11:13:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.978+0000 7fc7bfe00640 1 -- 172.21.15.7:0/981875381 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80319d0 con 0x7fc7c00cf560 2024-04-25T11:13:22.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.979+0000 7fc7c6a00640 1 -- 172.21.15.7:0/981875381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7c00cf560 msgr2=0x7fc7c0144220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.979+0000 7fc7c6a00640 1 --2- 172.21.15.7:0/981875381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7c00cf560 0x7fc7c0144220 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009e50 tx=0x7fc7b802a7a0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.979+0000 7fc7c6a00640 1 -- 172.21.15.7:0/981875381 shutdown_connections 2024-04-25T11:13:22.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.979+0000 7fc7c6a00640 1 --2- 172.21.15.7:0/981875381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7c00cf560 0x7fc7c0144220 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.979+0000 7fc7c6a00640 1 --2- 172.21.15.7:0/981875381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7c00b5580 0x7fc7c00cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.979+0000 7fc7c6a00640 1 -- 172.21.15.7:0/981875381 >> 172.21.15.7:0/981875381 conn(0x7fc7c0074140 msgr2=0x7fc7c006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:22.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.979+0000 7fc7c6a00640 1 -- 172.21.15.7:0/981875381 shutdown_connections 2024-04-25T11:13:22.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.979+0000 7fc7c6a00640 1 -- 172.21.15.7:0/981875381 wait complete. 2024-04-25T11:13:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.981+0000 7fc7c6a00640 1 Processor -- start 2024-04-25T11:13:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.982+0000 7fc7c6a00640 1 -- start start 2024-04-25T11:13:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.983+0000 7fc7c6a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7c00b4690 0x7fc7c01fde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.983+0000 7fc7c6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c00b5580 0x7fc7c01fe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.983+0000 7fc7c6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7c00cf560 0x7fc7c015d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.983+0000 7fc7c6a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c00b67c0 con 0x7fc7c00cf560 2024-04-25T11:13:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.983+0000 7fc7c6a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c00b64e0 con 0x7fc7c00b5580 2024-04-25T11:13:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.983+0000 7fc7c6a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c0070fa0 con 0x7fc7c00b4690 2024-04-25T11:13:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.983+0000 7fc7c6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7c00cf560 0x7fc7c015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.984+0000 7fc7c6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7c00cf560 0x7fc7c015d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45208/0 (socket says 172.21.15.7:45208) 2024-04-25T11:13:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.984+0000 7fc7c6000640 1 -- 172.21.15.7:0/2916178633 learned_addr learned my addr 172.21.15.7:0/2916178633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.984+0000 7fc7c5600640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7c00b4690 0x7fc7c01fde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.984+0000 7fc7c4c00640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c00b5580 0x7fc7c01fe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.984+0000 7fc7c6000640 1 -- 172.21.15.7:0/2916178633 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7c00b4690 msgr2=0x7fc7c01fde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.984+0000 7fc7c6000640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7c00b4690 0x7fc7c01fde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.984+0000 7fc7c6000640 1 -- 172.21.15.7:0/2916178633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c00b5580 msgr2=0x7fc7c01fe3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.984+0000 7fc7c6000640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c00b5580 0x7fc7c01fe3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.984+0000 7fc7c6000640 1 -- 172.21.15.7:0/2916178633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c013c240 con 0x7fc7c00cf560 2024-04-25T11:13:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.985+0000 7fc7c6000640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7c00cf560 0x7fc7c015d960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009f70 tx=0x7fc7b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.985+0000 7fc7bd600640 1 -- 172.21.15.7:0/2916178633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8036050 con 0x7fc7c00cf560 2024-04-25T11:13:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.985+0000 7fc7bd600640 1 -- 172.21.15.7:0/2916178633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b8042070 con 0x7fc7c00cf560 2024-04-25T11:13:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.985+0000 7fc7bd600640 1 -- 172.21.15.7:0/2916178633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8009510 con 0x7fc7c00cf560 2024-04-25T11:13:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.985+0000 7fc7c6a00640 1 -- 172.21.15.7:0/2916178633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c013c520 con 0x7fc7c00cf560 2024-04-25T11:13:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.985+0000 7fc7c6a00640 1 -- 172.21.15.7:0/2916178633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c013cae0 con 0x7fc7c00cf560 2024-04-25T11:13:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.986+0000 7fc7c6a00640 1 -- 172.21.15.7:0/2916178633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c01f4d90 con 0x7fc7c00cf560 2024-04-25T11:13:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.988+0000 7fc7bd600640 1 -- 172.21.15.7:0/2916178633 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc7b8007bb0 con 0x7fc7c00cf560 2024-04-25T11:13:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.989+0000 7fc7bd600640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7a00744b0 0x7fc7a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.989+0000 7fc7bd600640 1 -- 172.21.15.7:0/2916178633 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fc7b80b6330 con 0x7fc7c00cf560 2024-04-25T11:13:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.989+0000 7fc7c5600640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7a00744b0 0x7fc7a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.990+0000 7fc7c5600640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7a00744b0 0x7fc7a00769c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4000f60 tx=0x7fc7b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:22.994+0000 7fc7bd600640 1 -- 172.21.15.7:0/2916178633 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7b8080870 con 0x7fc7c00cf560 2024-04-25T11:13:23.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.197+0000 7fc7c6a00640 1 -- 172.21.15.7:0/2916178633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc7c00b7980 con 0x7fc7c00cf560 2024-04-25T11:13:23.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.198+0000 7fc7bd600640 1 -- 172.21.15.7:0/2916178633 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) ==== 100+0+11 (secure 0 0 0) 0x7fc7c00b7980 con 0x7fc7c00cf560 2024-04-25T11:13:23.198 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:13:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.200+0000 7fc7c6a00640 1 -- 172.21.15.7:0/2916178633 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7a00744b0 msgr2=0x7fc7a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.200+0000 7fc7c6a00640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7a00744b0 0x7fc7a00769c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4000f60 tx=0x7fc7b4009040 comp rx=0 tx=0).stop 2024-04-25T11:13:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.200+0000 7fc7c6a00640 1 -- 172.21.15.7:0/2916178633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7c00cf560 msgr2=0x7fc7c015d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.200+0000 7fc7c6a00640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7c00cf560 0x7fc7c015d960 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009f70 tx=0x7fc7b8004a40 comp rx=0 tx=0).stop 2024-04-25T11:13:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.201+0000 7fc7c6a00640 1 -- 172.21.15.7:0/2916178633 shutdown_connections 2024-04-25T11:13:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.201+0000 7fc7c6a00640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7a00744b0 0x7fc7a00769c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.201+0000 7fc7c6a00640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7c00cf560 0x7fc7c015d960 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.201+0000 7fc7c6a00640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7c00b5580 0x7fc7c01fe3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.201+0000 7fc7c6a00640 1 --2- 172.21.15.7:0/2916178633 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7c00b4690 0x7fc7c01fde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.201+0000 7fc7c6a00640 1 -- 172.21.15.7:0/2916178633 >> 172.21.15.7:0/2916178633 conn(0x7fc7c0074140 msgr2=0x7fc7c0139cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:23.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.201+0000 7fc7c6a00640 1 -- 172.21.15.7:0/2916178633 shutdown_connections 2024-04-25T11:13:23.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.201+0000 7fc7c6a00640 1 -- 172.21.15.7:0/2916178633 wait complete. 2024-04-25T11:13:23.227 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:13:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.505+0000 7f20ae600640 1 Processor -- start 2024-04-25T11:13:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.505+0000 7f20ae600640 1 -- start start 2024-04-25T11:13:23.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.508+0000 7f20ae600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20a80aed40 0x7f20a815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:23.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.508+0000 7f20ae600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f20a80af420 0x7f20a80af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:23.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.508+0000 7f20ae600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20a80cebb0 con 0x7f20a80b5220 2024-04-25T11:13:23.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.508+0000 7f20ae600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f20a815e1f0 con 0x7f20a80af420 2024-04-25T11:13:23.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.508+0000 7f20ae600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f20a80afd70 con 0x7f20a80aed40 2024-04-25T11:13:23.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.508+0000 7f209fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f20a80b5220 0x7f20a80ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42090/0 (socket says 172.21.15.7:42090) 2024-04-25T11:13:23.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.508+0000 7f209fe00640 1 -- 172.21.15.7:0/3648543382 learned_addr learned my addr 172.21.15.7:0/3648543382 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.508+0000 7f20adc00640 1 --2- 172.21.15.7:0/3648543382 >> v2:172.21.15.7:3300/0 conn(0x7f20a80af420 0x7f20a80af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.509+0000 7f20ad200640 1 --2- 172.21.15.7:0/3648543382 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20a80aed40 0x7f20a815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.509+0000 7f20a7e00640 1 -- 172.21.15.7:0/3648543382 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91415458 0 0) 0x7f2090001110 con 0x7f20a80b5220 2024-04-25T11:13:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.509+0000 7f20a7e00640 1 -- 172.21.15.7:0/3648543382 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f208c003520 con 0x7f20a80b5220 2024-04-25T11:13:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.509+0000 7f20adc00640 1 -- 172.21.15.7:0/3648543382 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20a80aed40 msgr2=0x7f20a815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.509+0000 7f20adc00640 1 --2- 172.21.15.7:0/3648543382 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20a80aed40 0x7f20a815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.509+0000 7f20adc00640 1 -- 172.21.15.7:0/3648543382 >> v1:172.21.15.7:6789/0 conn(0x7f20a80b5220 legacy=0x7f20a80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.509+0000 7f20adc00640 1 -- 172.21.15.7:0/3648543382 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a80b64b0 con 0x7f20a80af420 2024-04-25T11:13:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.510+0000 7f20adc00640 1 --2- 172.21.15.7:0/3648543382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a80af420 0x7f20a80af830 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f20a00097a0 tx=0x7f20a002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a96e29445cdd7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.510+0000 7f20a7e00640 1 -- 172.21.15.7:0/3648543382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0037070 con 0x7f20a80af420 2024-04-25T11:13:23.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.510+0000 7f20a7e00640 1 -- 172.21.15.7:0/3648543382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a0032440 con 0x7f20a80af420 2024-04-25T11:13:23.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.510+0000 7f20a7e00640 1 -- 172.21.15.7:0/3648543382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0032810 con 0x7f20a80af420 2024-04-25T11:13:23.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.511+0000 7f20ae600640 1 -- 172.21.15.7:0/3648543382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a80af420 msgr2=0x7f20a80af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:23.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.511+0000 7f20ae600640 1 --2- 172.21.15.7:0/3648543382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a80af420 0x7f20a80af830 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f20a00097a0 tx=0x7f20a002a180 comp rx=0 tx=0).stop 2024-04-25T11:13:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.511+0000 7f20ae600640 1 -- 172.21.15.7:0/3648543382 shutdown_connections 2024-04-25T11:13:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.511+0000 7f20ae600640 1 --2- 172.21.15.7:0/3648543382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a80af420 0x7f20a80af830 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.511+0000 7f20ae600640 1 --2- 172.21.15.7:0/3648543382 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20a80aed40 0x7f20a815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.511+0000 7f20ae600640 1 -- 172.21.15.7:0/3648543382 >> 172.21.15.7:0/3648543382 conn(0x7f20a80b3820 msgr2=0x7f20a80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.511+0000 7f20ae600640 1 -- 172.21.15.7:0/3648543382 shutdown_connections 2024-04-25T11:13:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.511+0000 7f20ae600640 1 -- 172.21.15.7:0/3648543382 wait complete. 2024-04-25T11:13:23.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.514+0000 7f20ae600640 1 Processor -- start 2024-04-25T11:13:23.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.514+0000 7f20ae600640 1 -- start start 2024-04-25T11:13:23.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.516+0000 7f20ae600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a80aed40 0x7f20a815fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:23.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.516+0000 7f20ae600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20a80af420 0x7f20a81fa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:23.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.516+0000 7f20ae600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20a80b5220 0x7f20a8159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.516+0000 7f20ae600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f20a806cbd0 con 0x7f20a80aed40 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.516+0000 7f20ae600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f20a806c8f0 con 0x7f20a80af420 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.516+0000 7f20ae600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f20a806ceb0 con 0x7f20a80b5220 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.516+0000 7f20ad200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a80aed40 0x7f20a815fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.516+0000 7f20ad200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a80aed40 0x7f20a815fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45226/0 (socket says 172.21.15.7:45226) 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.516+0000 7f20ad200640 1 -- 172.21.15.7:0/2978576249 learned_addr learned my addr 172.21.15.7:0/2978576249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.516+0000 7f209fe00640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20a80af420 0x7f20a81fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20adc00640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20a80b5220 0x7f20a8159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20ad200640 1 -- 172.21.15.7:0/2978576249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20a80b5220 msgr2=0x7f20a8159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20ad200640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20a80b5220 0x7f20a8159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20ad200640 1 -- 172.21.15.7:0/2978576249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20a80af420 msgr2=0x7f20a81fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20ad200640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20a80af420 0x7f20a81fa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20ad200640 1 -- 172.21.15.7:0/2978576249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a81603d0 con 0x7f20a80aed40 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20adc00640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20a80b5220 0x7f20a8159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20ad200640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a80aed40 0x7f20a815fe90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2098007e60 tx=0x7f209800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20a5600640 1 -- 172.21.15.7:0/2978576249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209801a070 con 0x7f20a80aed40 2024-04-25T11:13:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20a5600640 1 -- 172.21.15.7:0/2978576249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2098003b90 con 0x7f20a80aed40 2024-04-25T11:13:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20a5600640 1 -- 172.21.15.7:0/2978576249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098004030 con 0x7f20a80aed40 2024-04-25T11:13:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.517+0000 7f20ae600640 1 -- 172.21.15.7:0/2978576249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a8146ae0 con 0x7f20a80aed40 2024-04-25T11:13:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.518+0000 7f20ae600640 1 -- 172.21.15.7:0/2978576249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a81f9c50 con 0x7f20a80aed40 2024-04-25T11:13:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.519+0000 7f20ae600640 1 -- 172.21.15.7:0/2978576249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a8050740 con 0x7f20a80aed40 2024-04-25T11:13:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.520+0000 7f20a5600640 1 -- 172.21.15.7:0/2978576249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f209801d070 con 0x7f20a80aed40 2024-04-25T11:13:23.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.522+0000 7f20a5600640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20840744b0 0x7f20840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:23.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.522+0000 7f20a5600640 1 -- 172.21.15.7:0/2978576249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f2098095560 con 0x7f20a80aed40 2024-04-25T11:13:23.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.522+0000 7f209fe00640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20840744b0 0x7f20840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:23.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.523+0000 7f209fe00640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20840744b0 0x7f20840769c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f20a8159c40 tx=0x7f2090009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:23.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.527+0000 7f20a5600640 1 -- 172.21.15.7:0/2978576249 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f209805f450 con 0x7f20a80aed40 2024-04-25T11:13:23.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.769+0000 7f20ae600640 1 -- 172.21.15.7:0/2978576249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20a80d12c0 con 0x7f20a80aed40 2024-04-25T11:13:23.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.770+0000 7f20a5600640 1 -- 172.21.15.7:0/2978576249 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5172 (secure 0 0 0) 0x7f2098063750 con 0x7f20a80aed40 2024-04-25T11:13:23.770 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:23.770 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2575517671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2575517671},{"type":"v1","addr":"172.21.15.87:6835","nonce":2575517671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/3746682343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3746682343},{"type":"v1","addr":"172.21.15.7:6836","nonce":3746682343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2132415318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2132415318},{"type":"v1","addr":"172.21.15.7:6837","nonce":2132415318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:12:17.156571+0000","modified":"2024-04-25T11:12:20.224854+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6833/389115631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":389115631},{"type":"v1","addr":"172.21.15.87:6833","nonce":389115631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5283,"qdb_cluster":[5283]},"id":2}]} 2024-04-25T11:13:23.771 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2024-04-25T11:13:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 -- 172.21.15.7:0/2978576249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20840744b0 msgr2=0x7f20840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20840744b0 0x7f20840769c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f20a8159c40 tx=0x7f2090009040 comp rx=0 tx=0).stop 2024-04-25T11:13:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 -- 172.21.15.7:0/2978576249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a80aed40 msgr2=0x7f20a815fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a80aed40 0x7f20a815fe90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2098007e60 tx=0x7f209800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 -- 172.21.15.7:0/2978576249 shutdown_connections 2024-04-25T11:13:23.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20840744b0 0x7f20840769c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20a80b5220 0x7f20a8159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20a80af420 0x7f20a81fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 --2- 172.21.15.7:0/2978576249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20a80aed40 0x7f20a815fe90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 -- 172.21.15.7:0/2978576249 >> 172.21.15.7:0/2978576249 conn(0x7f20a80b3820 msgr2=0x7f20a80ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 -- 172.21.15.7:0/2978576249 shutdown_connections 2024-04-25T11:13:23.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:23.773+0000 7f20ae600640 1 -- 172.21.15.7:0/2978576249 wait complete. 2024-04-25T11:13:23.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:24.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.075+0000 7f5a98c00640 1 Processor -- start 2024-04-25T11:13:24.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.075+0000 7f5a98c00640 1 -- start start 2024-04-25T11:13:24.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.077+0000 7f5a98c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5a940d1540 0x7f5a940ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:24.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.077+0000 7f5a98c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a940cfb10 0x7f5a940b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:24.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a98c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a940ce7b0 con 0x7f5a9409ba00 2024-04-25T11:13:24.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a98c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5a940cea70 con 0x7f5a940cfb10 2024-04-25T11:13:24.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a98c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5a940ced50 con 0x7f5a940d1540 2024-04-25T11:13:24.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a8aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a940cfb10 0x7f5a940b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:24.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a8aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a940cfb10 0x7f5a940b8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45236/0 (socket says 172.21.15.7:45236) 2024-04-25T11:13:24.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a8aa00640 1 -- 172.21.15.7:0/2272130498 learned_addr learned my addr 172.21.15.7:0/2272130498 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:24.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a93e00640 1 --1- 172.21.15.7:0/2272130498 >> v1:172.21.15.7:6789/0 conn(0x7f5a9409ba00 0x7f5a9409be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42092/0 (socket says 172.21.15.7:42092) 2024-04-25T11:13:24.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a93400640 1 --2- 172.21.15.7:0/2272130498 >> v2:172.21.15.87:3300/0 conn(0x7f5a940d1540 0x7f5a940ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:24.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a8aa00640 1 -- 172.21.15.7:0/2272130498 >> v2:172.21.15.87:3300/0 conn(0x7f5a940d1540 msgr2=0x7f5a940ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:24.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a8aa00640 1 --2- 172.21.15.7:0/2272130498 >> v2:172.21.15.87:3300/0 conn(0x7f5a940d1540 0x7f5a940ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a8aa00640 1 -- 172.21.15.7:0/2272130498 >> v1:172.21.15.7:6789/0 conn(0x7f5a9409ba00 legacy=0x7f5a9409be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a8aa00640 1 -- 172.21.15.7:0/2272130498 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a940cf030 con 0x7f5a940cfb10 2024-04-25T11:13:24.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.078+0000 7f5a93400640 1 --2- 172.21.15.7:0/2272130498 >> v2:172.21.15.87:3300/0 conn(0x7f5a940d1540 0x7f5a940ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:24.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.079+0000 7f5a8aa00640 1 --2- 172.21.15.7:0/2272130498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a940cfb10 0x7f5a940b8d20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5a94073dd0 tx=0x7f5a7c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3d769e137b392a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.080+0000 7f5a92a00640 1 -- 172.21.15.7:0/2272130498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c037070 con 0x7f5a940cfb10 2024-04-25T11:13:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.080+0000 7f5a92a00640 1 -- 172.21.15.7:0/2272130498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a7c032440 con 0x7f5a940cfb10 2024-04-25T11:13:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.080+0000 7f5a92a00640 1 -- 172.21.15.7:0/2272130498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c032810 con 0x7f5a940cfb10 2024-04-25T11:13:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.081+0000 7f5a98c00640 1 -- 172.21.15.7:0/2272130498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a940cfb10 msgr2=0x7f5a940b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.081+0000 7f5a98c00640 1 --2- 172.21.15.7:0/2272130498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a940cfb10 0x7f5a940b8d20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5a94073dd0 tx=0x7f5a7c02a200 comp rx=0 tx=0).stop 2024-04-25T11:13:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.081+0000 7f5a98c00640 1 -- 172.21.15.7:0/2272130498 shutdown_connections 2024-04-25T11:13:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.081+0000 7f5a98c00640 1 --2- 172.21.15.7:0/2272130498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a940cfb10 0x7f5a940b8d20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.081+0000 7f5a98c00640 1 --2- 172.21.15.7:0/2272130498 >> v2:172.21.15.87:3300/0 conn(0x7f5a940d1540 0x7f5a940ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.081+0000 7f5a98c00640 1 -- 172.21.15.7:0/2272130498 >> 172.21.15.7:0/2272130498 conn(0x7f5a940b5250 msgr2=0x7f5a940b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.081+0000 7f5a98c00640 1 -- 172.21.15.7:0/2272130498 shutdown_connections 2024-04-25T11:13:24.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.081+0000 7f5a98c00640 1 -- 172.21.15.7:0/2272130498 wait complete. 2024-04-25T11:13:24.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.083+0000 7f5a98c00640 1 Processor -- start 2024-04-25T11:13:24.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.083+0000 7f5a98c00640 1 -- start start 2024-04-25T11:13:24.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.085+0000 7f5a98c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9409ba00 0x7f5a9409d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:24.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.085+0000 7f5a98c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a940cfb10 0x7f5a9415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:24.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.085+0000 7f5a98c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a940d1540 0x7f5a941f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:24.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.085+0000 7f5a98c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a940d2680 con 0x7f5a9409ba00 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.085+0000 7f5a98c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5a940b9260 con 0x7f5a940cfb10 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.085+0000 7f5a98c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5a940b65b0 con 0x7f5a940d1540 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.085+0000 7f5a93400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9409ba00 0x7f5a9409d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.085+0000 7f5a93400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9409ba00 0x7f5a9409d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45242/0 (socket says 172.21.15.7:45242) 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.085+0000 7f5a93400640 1 -- 172.21.15.7:0/1286249666 learned_addr learned my addr 172.21.15.7:0/1286249666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a93400640 1 -- 172.21.15.7:0/1286249666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a940d1540 msgr2=0x7f5a941f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a93400640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a940d1540 0x7f5a941f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a93400640 1 -- 172.21.15.7:0/1286249666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a940cfb10 msgr2=0x7f5a9415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a8aa00640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a940cfb10 0x7f5a9415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a93400640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a940cfb10 0x7f5a9415c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a93400640 1 -- 172.21.15.7:0/1286249666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9415c880 con 0x7f5a9409ba00 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a8aa00640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a940cfb10 0x7f5a9415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a93400640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9409ba00 0x7f5a9409d870 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400ced0 tx=0x7f5a8400a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a8be00640 1 -- 172.21.15.7:0/1286249666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8400e860 con 0x7f5a9409ba00 2024-04-25T11:13:24.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a8be00640 1 -- 172.21.15.7:0/1286249666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a8400ea00 con 0x7f5a9409ba00 2024-04-25T11:13:24.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a8be00640 1 -- 172.21.15.7:0/1286249666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8400edf0 con 0x7f5a9409ba00 2024-04-25T11:13:24.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.086+0000 7f5a98c00640 1 -- 172.21.15.7:0/1286249666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9409e250 con 0x7f5a9409ba00 2024-04-25T11:13:24.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.087+0000 7f5a98c00640 1 -- 172.21.15.7:0/1286249666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a9409e810 con 0x7f5a9409ba00 2024-04-25T11:13:24.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.088+0000 7f5a98c00640 1 -- 172.21.15.7:0/1286249666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a94050740 con 0x7f5a9409ba00 2024-04-25T11:13:24.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.089+0000 7f5a8be00640 1 -- 172.21.15.7:0/1286249666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5a84016420 con 0x7f5a9409ba00 2024-04-25T11:13:24.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.090+0000 7f5a8be00640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a700744e0 0x7f5a700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:24.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.090+0000 7f5a8be00640 1 -- 172.21.15.7:0/1286249666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f5a840961a0 con 0x7f5a9409ba00 2024-04-25T11:13:24.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.090+0000 7f5a8aa00640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a700744e0 0x7f5a700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:24.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.091+0000 7f5a8aa00640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a700744e0 0x7f5a700769f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c034000 tx=0x7f5a7c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:24.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.094+0000 7f5a8be00640 1 -- 172.21.15.7:0/1286249666 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a840606e0 con 0x7f5a9409ba00 2024-04-25T11:13:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.301+0000 7f5a98c00640 1 -- 172.21.15.7:0/1286249666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a940b8920 con 0x7f5a9409ba00 2024-04-25T11:13:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.304+0000 7f5a8be00640 1 -- 172.21.15.7:0/1286249666 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16839 (secure 0 0 0) 0x7f5a8405ffe0 con 0x7f5a9409ba00 2024-04-25T11:13:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:24.305 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:04.193129+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:12:14.961811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T11:12:16.128549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:24.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 -- 172.21.15.7:0/1286249666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a700744e0 msgr2=0x7f5a700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a700744e0 0x7f5a700769f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c034000 tx=0x7f5a7c02e000 comp rx=0 tx=0).stop 2024-04-25T11:13:24.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 -- 172.21.15.7:0/1286249666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9409ba00 msgr2=0x7f5a9409d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9409ba00 0x7f5a9409d870 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400ced0 tx=0x7f5a8400a6c0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 -- 172.21.15.7:0/1286249666 shutdown_connections 2024-04-25T11:13:24.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a700744e0 0x7f5a700769f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a940d1540 0x7f5a941f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a940cfb10 0x7f5a9415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 --2- 172.21.15.7:0/1286249666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a9409ba00 0x7f5a9409d870 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 -- 172.21.15.7:0/1286249666 >> 172.21.15.7:0/1286249666 conn(0x7f5a940b5250 msgr2=0x7f5a940b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:24.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 -- 172.21.15.7:0/1286249666 shutdown_connections 2024-04-25T11:13:24.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.307+0000 7f5a98c00640 1 -- 172.21.15.7:0/1286249666 wait complete. 2024-04-25T11:13:24.335 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:13:24.335 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:13:24.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.620+0000 7fd87b800640 1 Processor -- start 2024-04-25T11:13:24.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.620+0000 7fd87b800640 1 -- start start 2024-04-25T11:13:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.622+0000 7fd87b800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8740b16d0 0x7fd8740d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.622+0000 7fd87b800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8740d1ed0 con 0x7fd874160730 2024-04-25T11:13:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.622+0000 7fd87b800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8740d0c10 con 0x7fd874145880 2024-04-25T11:13:24.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.622+0000 7fd87b800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd8740b1ae0 con 0x7fd8740b16d0 2024-04-25T11:13:24.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.622+0000 7fd878e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8740b16d0 0x7fd8740d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:24.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.622+0000 7fd879800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd874160730 0x7fd874160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42094/0 (socket says 172.21.15.7:42094) 2024-04-25T11:13:24.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.622+0000 7fd879800640 1 -- 172.21.15.7:0/1828145668 learned_addr learned my addr 172.21.15.7:0/1828145668 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:24.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.623+0000 7fd878e00640 1 -- 172.21.15.7:0/1828145668 >> v1:172.21.15.87:6789/0 conn(0x7fd874145880 legacy=0x7fd874145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:24.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.623+0000 7fd878e00640 1 -- 172.21.15.7:0/1828145668 >> v1:172.21.15.7:6789/0 conn(0x7fd874160730 legacy=0x7fd874160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.623+0000 7fd873400640 1 -- 172.21.15.7:0/1828145668 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2411714052 0 0) 0x7fd8740d1ed0 con 0x7fd874160730 2024-04-25T11:13:24.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.623+0000 7fd878e00640 1 -- 172.21.15.7:0/1828145668 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd874161460 con 0x7fd8740b16d0 2024-04-25T11:13:24.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.624+0000 7fd878e00640 1 --2- 172.21.15.7:0/1828145668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8740b16d0 0x7fd8740d1990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0097a0 tx=0x7fd85c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ffbf6dfae46114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:24.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.624+0000 7fd873400640 1 -- 172.21.15.7:0/1828145668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c037070 con 0x7fd8740b16d0 2024-04-25T11:13:24.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.624+0000 7fd873400640 1 -- 172.21.15.7:0/1828145668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c032440 con 0x7fd8740b16d0 2024-04-25T11:13:24.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.624+0000 7fd873400640 1 -- 172.21.15.7:0/1828145668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c032810 con 0x7fd8740b16d0 2024-04-25T11:13:24.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.625+0000 7fd87b800640 1 -- 172.21.15.7:0/1828145668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8740b16d0 msgr2=0x7fd8740d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.625+0000 7fd87b800640 1 --2- 172.21.15.7:0/1828145668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8740b16d0 0x7fd8740d1990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0097a0 tx=0x7fd85c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.625+0000 7fd87b800640 1 -- 172.21.15.7:0/1828145668 shutdown_connections 2024-04-25T11:13:24.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.625+0000 7fd87b800640 1 --2- 172.21.15.7:0/1828145668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8740b16d0 0x7fd8740d1990 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.625+0000 7fd87b800640 1 -- 172.21.15.7:0/1828145668 >> 172.21.15.7:0/1828145668 conn(0x7fd874073b80 msgr2=0x7fd874073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:24.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.625+0000 7fd87b800640 1 -- 172.21.15.7:0/1828145668 shutdown_connections 2024-04-25T11:13:24.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.626+0000 7fd87b800640 1 -- 172.21.15.7:0/1828145668 wait complete. 2024-04-25T11:13:24.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.628+0000 7fd87b800640 1 Processor -- start 2024-04-25T11:13:24.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.628+0000 7fd87b800640 1 -- start start 2024-04-25T11:13:24.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd87b800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8740b16d0 0x7fd8740cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:24.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd87b800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd874145880 0x7fd874148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:24.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd87b800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd874160730 0x7fd874144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:24.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd87b800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8740d1190 con 0x7fd8740b16d0 2024-04-25T11:13:24.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd87b800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd8740d1470 con 0x7fd874160730 2024-04-25T11:13:24.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd87b800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd874146820 con 0x7fd874145880 2024-04-25T11:13:24.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd878e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8740b16d0 0x7fd8740cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:24.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd878e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8740b16d0 0x7fd8740cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45270/0 (socket says 172.21.15.7:45270) 2024-04-25T11:13:24.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd878e00640 1 -- 172.21.15.7:0/3577117330 learned_addr learned my addr 172.21.15.7:0/3577117330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:24.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd878e00640 1 -- 172.21.15.7:0/3577117330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd874145880 msgr2=0x7fd874148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd878e00640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd874145880 0x7fd874148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd878e00640 1 -- 172.21.15.7:0/3577117330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd874160730 msgr2=0x7fd874144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd878e00640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd874160730 0x7fd874144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.630+0000 7fd878e00640 1 -- 172.21.15.7:0/3577117330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741f7a10 con 0x7fd8740b16d0 2024-04-25T11:13:24.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.631+0000 7fd878e00640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8740b16d0 0x7fd8740cdf60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd85c035040 tx=0x7fd85c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:24.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.631+0000 7fd870c00640 1 -- 172.21.15.7:0/3577117330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c037040 con 0x7fd8740b16d0 2024-04-25T11:13:24.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.631+0000 7fd870c00640 1 -- 172.21.15.7:0/3577117330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c004250 con 0x7fd8740b16d0 2024-04-25T11:13:24.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.631+0000 7fd87b800640 1 -- 172.21.15.7:0/3577117330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd874149500 con 0x7fd8740b16d0 2024-04-25T11:13:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.631+0000 7fd870c00640 1 -- 172.21.15.7:0/3577117330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c032e20 con 0x7fd8740b16d0 2024-04-25T11:13:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.631+0000 7fd87b800640 1 -- 172.21.15.7:0/3577117330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8741fa210 con 0x7fd8740b16d0 2024-04-25T11:13:24.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.632+0000 7fd87b800640 1 -- 172.21.15.7:0/3577117330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87406a100 con 0x7fd8740b16d0 2024-04-25T11:13:24.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.634+0000 7fd870c00640 1 -- 172.21.15.7:0/3577117330 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd85c030770 con 0x7fd8740b16d0 2024-04-25T11:13:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.635+0000 7fd870c00640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 0x7fd8500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.635+0000 7fd870c00640 1 -- 172.21.15.7:0/3577117330 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fd85c0b64a0 con 0x7fd8740b16d0 2024-04-25T11:13:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.635+0000 7fd873e00640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 0x7fd8500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:24.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.636+0000 7fd873e00640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 0x7fd8500769c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd87415a1f0 tx=0x7fd864009230 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:24.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.643+0000 7fd870c00640 1 -- 172.21.15.7:0/3577117330 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd85c080a70 con 0x7fd8740b16d0 2024-04-25T11:13:24.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.878+0000 7fd87b800640 1 -- 172.21.15.7:0/3577117330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd87415fe40 con 0x7fd8740b16d0 2024-04-25T11:13:24.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.880+0000 7fd870c00640 1 -- 172.21.15.7:0/3577117330 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5172 (secure 0 0 0) 0x7fd85c080370 con 0x7fd8740b16d0 2024-04-25T11:13:24.880 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:24.881 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2575517671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2575517671},{"type":"v1","addr":"172.21.15.87:6835","nonce":2575517671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/3746682343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3746682343},{"type":"v1","addr":"172.21.15.7:6836","nonce":3746682343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2132415318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2132415318},{"type":"v1","addr":"172.21.15.7:6837","nonce":2132415318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:12:17.156571+0000","modified":"2024-04-25T11:12:20.224854+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6833/389115631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":389115631},{"type":"v1","addr":"172.21.15.87:6833","nonce":389115631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5283,"qdb_cluster":[5283]},"id":2}]} 2024-04-25T11:13:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2024-04-25T11:13:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.882+0000 7fd87b800640 1 -- 172.21.15.7:0/3577117330 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 msgr2=0x7fd8500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.882+0000 7fd87b800640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 0x7fd8500769c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd87415a1f0 tx=0x7fd864009230 comp rx=0 tx=0).stop 2024-04-25T11:13:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.882+0000 7fd87b800640 1 -- 172.21.15.7:0/3577117330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8740b16d0 msgr2=0x7fd8740cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.882+0000 7fd87b800640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8740b16d0 0x7fd8740cdf60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd85c035040 tx=0x7fd85c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.882+0000 7fd87b800640 1 -- 172.21.15.7:0/3577117330 shutdown_connections 2024-04-25T11:13:24.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.882+0000 7fd87b800640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 0x7fd8500769c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.882+0000 7fd87b800640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd874160730 0x7fd874144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.882+0000 7fd87b800640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd874145880 0x7fd874148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.882+0000 7fd87b800640 1 --2- 172.21.15.7:0/3577117330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8740b16d0 0x7fd8740cdf60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:24.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.882+0000 7fd87b800640 1 -- 172.21.15.7:0/3577117330 >> 172.21.15.7:0/3577117330 conn(0x7fd874073b80 msgr2=0x7fd874142930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:24.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.883+0000 7fd87b800640 1 -- 172.21.15.7:0/3577117330 shutdown_connections 2024-04-25T11:13:24.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:24.883+0000 7fd87b800640 1 -- 172.21.15.7:0/3577117330 wait complete. 2024-04-25T11:13:24.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:13:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.194+0000 7fac7b600640 1 Processor -- start 2024-04-25T11:13:25.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.194+0000 7fac7b600640 1 -- start start 2024-04-25T11:13:25.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.196+0000 7fac7b600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac740b6790 0x7fac74164640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.197+0000 7fac7b600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac740aea00 con 0x7fac740b0c20 2024-04-25T11:13:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.197+0000 7fac7b600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac740b13e0 con 0x7fac7415de70 2024-04-25T11:13:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.197+0000 7fac7b600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fac740cdb70 con 0x7fac740b6790 2024-04-25T11:13:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.197+0000 7fac79600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fac740b0c20 0x7fac741606a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42104/0 (socket says 172.21.15.7:42104) 2024-04-25T11:13:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.197+0000 7fac79600640 1 -- 172.21.15.7:0/551239360 learned_addr learned my addr 172.21.15.7:0/551239360 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.197+0000 7fac73e00640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac740b6790 0x7fac74164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.197+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2895861655 0 0) 0x7fac740aea00 con 0x7fac740b0c20 2024-04-25T11:13:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac58003520 con 0x7fac740b0c20 2024-04-25T11:13:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2664893476 0 0) 0x7fac58003520 con 0x7fac740b0c20 2024-04-25T11:13:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac740b6790 msgr2=0x7fac74164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac740b6790 0x7fac74164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 >> v1:172.21.15.87:6789/0 conn(0x7fac7415de70 legacy=0x7fac740b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac740cddc0 con 0x7fac740b0c20 2024-04-25T11:13:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fac6c0019f0 con 0x7fac740b0c20 2024-04-25T11:13:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 >> v1:172.21.15.7:6789/0 conn(0x7fac740b0c20 legacy=0x7fac741606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac740b6790 0x7fac580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac58003f60 0x7fac58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac58004d40 0x7fac58005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac740cdb70 con 0x7fac58003f60 2024-04-25T11:13:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fac5800bc00 con 0x7fac58004d40 2024-04-25T11:13:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.198+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fac5800be00 con 0x7fac740b6790 2024-04-25T11:13:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.199+0000 7fac73e00640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac58003f60 0x7fac58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.199+0000 7fac78c00640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac740b6790 0x7fac580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.199+0000 7fac79600640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac58004d40 0x7fac58005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.199+0000 7fac73e00640 1 -- 172.21.15.7:0/551239360 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac740b6790 msgr2=0x7fac580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.199+0000 7fac73e00640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac740b6790 0x7fac580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.199+0000 7fac73e00640 1 -- 172.21.15.7:0/551239360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac58004d40 msgr2=0x7fac58005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.199+0000 7fac73e00640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac58004d40 0x7fac58005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.199+0000 7fac73e00640 1 -- 172.21.15.7:0/551239360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac58003520 con 0x7fac58003f60 2024-04-25T11:13:25.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.200+0000 7fac73e00640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac58003f60 0x7fac58004800 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fac64009ac0 tx=0x7fac58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e58c1f0d0e81c19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.200+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64036070 con 0x7fac58003f60 2024-04-25T11:13:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.201+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac64031400 con 0x7fac58003f60 2024-04-25T11:13:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.201+0000 7fac73400640 1 -- 172.21.15.7:0/551239360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64009690 con 0x7fac58003f60 2024-04-25T11:13:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.201+0000 7fac7b600640 1 -- 172.21.15.7:0/551239360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac58003f60 msgr2=0x7fac58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.201+0000 7fac7b600640 1 --2- 172.21.15.7:0/551239360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac58003f60 0x7fac58004800 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fac64009ac0 tx=0x7fac58005830 comp rx=0 tx=0).stop 2024-04-25T11:13:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.201+0000 7fac79600640 1 -- 172.21.15.7:0/551239360 reap_dead start 2024-04-25T11:13:25.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.201+0000 7fac7b600640 1 -- 172.21.15.7:0/551239360 shutdown_connections 2024-04-25T11:13:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.201+0000 7fac7b600640 1 -- 172.21.15.7:0/551239360 >> 172.21.15.7:0/551239360 conn(0x7fac74074200 msgr2=0x7fac7406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.201+0000 7fac7b600640 1 -- 172.21.15.7:0/551239360 shutdown_connections 2024-04-25T11:13:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.201+0000 7fac7b600640 1 -- 172.21.15.7:0/551239360 wait complete. 2024-04-25T11:13:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.204+0000 7fac7b600640 1 Processor -- start 2024-04-25T11:13:25.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.204+0000 7fac7b600640 1 -- start start 2024-04-25T11:13:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.205+0000 7fac7b600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac58003f60 0x7fac74155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.205+0000 7fac7b600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac741483a0 0x7fac7415c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac7b600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac740cec60 0x7fac740cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac7b600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac7406f300 con 0x7fac740cec60 2024-04-25T11:13:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac7b600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fac740aed30 con 0x7fac58003f60 2024-04-25T11:13:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac7b600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fac7406f480 con 0x7fac741483a0 2024-04-25T11:13:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac79600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac740cec60 0x7fac740cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac79600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac740cec60 0x7fac740cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45280/0 (socket says 172.21.15.7:45280) 2024-04-25T11:13:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac79600640 1 -- 172.21.15.7:0/4111395551 learned_addr learned my addr 172.21.15.7:0/4111395551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac73e00640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac741483a0 0x7fac7415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac79600640 1 -- 172.21.15.7:0/4111395551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac741483a0 msgr2=0x7fac7415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac79600640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac741483a0 0x7fac7415c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac78c00640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac58003f60 0x7fac74155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac79600640 1 -- 172.21.15.7:0/4111395551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac58003f60 msgr2=0x7fac74155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac79600640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac58003f60 0x7fac74155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.206+0000 7fac79600640 1 -- 172.21.15.7:0/4111395551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac740cf5b0 con 0x7fac740cec60 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.207+0000 7fac79600640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac740cec60 0x7fac740cf070 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fac74072d60 tx=0x7fac6c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.207+0000 7fac70c00640 1 -- 172.21.15.7:0/4111395551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00bb80 con 0x7fac740cec60 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.207+0000 7fac70c00640 1 -- 172.21.15.7:0/4111395551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6c00bd20 con 0x7fac740cec60 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.207+0000 7fac70c00640 1 -- 172.21.15.7:0/4111395551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0072b0 con 0x7fac740cec60 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.207+0000 7fac7b600640 1 -- 172.21.15.7:0/4111395551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac74153bd0 con 0x7fac740cec60 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.207+0000 7fac7b600640 1 -- 172.21.15.7:0/4111395551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac741543b0 con 0x7fac740cec60 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.208+0000 7fac7b600640 1 -- 172.21.15.7:0/4111395551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac74153e40 con 0x7fac740cec60 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.209+0000 7fac70c00640 1 -- 172.21.15.7:0/4111395551 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fac6c007450 con 0x7fac740cec60 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.210+0000 7fac70c00640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac500744b0 0x7fac500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.210+0000 7fac70c00640 1 -- 172.21.15.7:0/4111395551 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fac6c0953d0 con 0x7fac740cec60 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.211+0000 7fac78c00640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac500744b0 0x7fac500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.211+0000 7fac78c00640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac500744b0 0x7fac500769c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fac5c007fd0 tx=0x7fac5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:25.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.215+0000 7fac70c00640 1 -- 172.21.15.7:0/4111395551 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac6c05f9a0 con 0x7fac740cec60 2024-04-25T11:13:25.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.457+0000 7fac7b600640 1 -- 172.21.15.7:0/4111395551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac74158db0 con 0x7fac740cec60 2024-04-25T11:13:25.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.459+0000 7fac70c00640 1 -- 172.21.15.7:0/4111395551 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5172 (secure 0 0 0) 0x7fac6c05f2a0 con 0x7fac740cec60 2024-04-25T11:13:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:25.459 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2575517671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2575517671},{"type":"v1","addr":"172.21.15.87:6835","nonce":2575517671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/3746682343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3746682343},{"type":"v1","addr":"172.21.15.7:6836","nonce":3746682343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2132415318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2132415318},{"type":"v1","addr":"172.21.15.7:6837","nonce":2132415318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:12:17.156571+0000","modified":"2024-04-25T11:12:20.224854+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5283},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5283":{"gid":5283,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.87:6833/389115631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":389115631},{"type":"v1","addr":"172.21.15.87:6833","nonce":389115631}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5283,"qdb_cluster":[5283]},"id":2}]} 2024-04-25T11:13:25.460 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2024-04-25T11:13:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.461+0000 7fac7b600640 1 -- 172.21.15.7:0/4111395551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac500744b0 msgr2=0x7fac500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.461+0000 7fac7b600640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac500744b0 0x7fac500769c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fac5c007fd0 tx=0x7fac5c009040 comp rx=0 tx=0).stop 2024-04-25T11:13:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.461+0000 7fac7b600640 1 -- 172.21.15.7:0/4111395551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac740cec60 msgr2=0x7fac740cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.461+0000 7fac7b600640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac740cec60 0x7fac740cf070 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fac74072d60 tx=0x7fac6c00b4a0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.461+0000 7fac7b600640 1 -- 172.21.15.7:0/4111395551 shutdown_connections 2024-04-25T11:13:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.461+0000 7fac7b600640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac500744b0 0x7fac500769c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.461+0000 7fac7b600640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac740cec60 0x7fac740cf070 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.461+0000 7fac7b600640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac741483a0 0x7fac7415c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.461+0000 7fac7b600640 1 --2- 172.21.15.7:0/4111395551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac58003f60 0x7fac74155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.461+0000 7fac7b600640 1 -- 172.21.15.7:0/4111395551 >> 172.21.15.7:0/4111395551 conn(0x7fac74074200 msgr2=0x7fac7415d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.461+0000 7fac7b600640 1 -- 172.21.15.7:0/4111395551 shutdown_connections 2024-04-25T11:13:25.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.462+0000 7fac7b600640 1 -- 172.21.15.7:0/4111395551 wait complete. 2024-04-25T11:13:25.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.748+0000 7fd874400640 1 Processor -- start 2024-04-25T11:13:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.748+0000 7fd874400640 1 -- start start 2024-04-25T11:13:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.750+0000 7fd874400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd86c0b8980 0x7fd86c0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.750+0000 7fd874400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd86c0d1f60 0x7fd86c0d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.750+0000 7fd874400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd86c0ce9e0 0x7fd86c0cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.750+0000 7fd874400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd86c09d050 con 0x7fd86c0b8980 2024-04-25T11:13:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.750+0000 7fd874400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd86c0d0f20 con 0x7fd86c0d1f60 2024-04-25T11:13:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.750+0000 7fd874400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd86c09cd70 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.751+0000 7fd871a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd86c0b8980 0x7fd86c0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.751+0000 7fd871a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd86c0b8980 0x7fd86c0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45288/0 (socket says 172.21.15.7:45288) 2024-04-25T11:13:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.751+0000 7fd871a00640 1 -- 172.21.15.7:0/1177044779 learned_addr learned my addr 172.21.15.7:0/1177044779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.751+0000 7fd872400640 1 --2- 172.21.15.7:0/1177044779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd86c0ce9e0 0x7fd86c0cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.751+0000 7fd871a00640 1 -- 172.21.15.7:0/1177044779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd86c0ce9e0 msgr2=0x7fd86c0cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.751+0000 7fd871a00640 1 --2- 172.21.15.7:0/1177044779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd86c0ce9e0 0x7fd86c0cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.751+0000 7fd871a00640 1 -- 172.21.15.7:0/1177044779 >> v2:172.21.15.87:3300/0 conn(0x7fd86c0d1f60 msgr2=0x7fd86c0d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.751+0000 7fd871000640 1 --2- 172.21.15.7:0/1177044779 >> v2:172.21.15.87:3300/0 conn(0x7fd86c0d1f60 0x7fd86c0d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.751+0000 7fd871a00640 1 --2- 172.21.15.7:0/1177044779 >> v2:172.21.15.87:3300/0 conn(0x7fd86c0d1f60 0x7fd86c0d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.751+0000 7fd871a00640 1 -- 172.21.15.7:0/1177044779 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c06e830 con 0x7fd86c0b8980 2024-04-25T11:13:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.751+0000 7fd871000640 1 --2- 172.21.15.7:0/1177044779 >> v2:172.21.15.87:3300/0 conn(0x7fd86c0d1f60 0x7fd86c0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.752+0000 7fd871a00640 1 --2- 172.21.15.7:0/1177044779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd86c0b8980 0x7fd86c0d2940 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0b96a0 tx=0x7fd86002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99080aa15a00144 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.753+0000 7fd86be00640 1 -- 172.21.15.7:0/1177044779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860037070 con 0x7fd86c0b8980 2024-04-25T11:13:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.753+0000 7fd86be00640 1 -- 172.21.15.7:0/1177044779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd860032440 con 0x7fd86c0b8980 2024-04-25T11:13:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.753+0000 7fd86be00640 1 -- 172.21.15.7:0/1177044779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd860032810 con 0x7fd86c0b8980 2024-04-25T11:13:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.753+0000 7fd874400640 1 -- 172.21.15.7:0/1177044779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd86c0b8980 msgr2=0x7fd86c0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.753+0000 7fd874400640 1 --2- 172.21.15.7:0/1177044779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd86c0b8980 0x7fd86c0d2940 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd86c0b96a0 tx=0x7fd86002a140 comp rx=0 tx=0).stop 2024-04-25T11:13:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.754+0000 7fd874400640 1 -- 172.21.15.7:0/1177044779 shutdown_connections 2024-04-25T11:13:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.754+0000 7fd874400640 1 --2- 172.21.15.7:0/1177044779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd86c0ce9e0 0x7fd86c0cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.754+0000 7fd874400640 1 --2- 172.21.15.7:0/1177044779 >> v2:172.21.15.87:3300/0 conn(0x7fd86c0d1f60 0x7fd86c0d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.754+0000 7fd874400640 1 --2- 172.21.15.7:0/1177044779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd86c0b8980 0x7fd86c0d2940 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.754+0000 7fd874400640 1 -- 172.21.15.7:0/1177044779 >> 172.21.15.7:0/1177044779 conn(0x7fd86c0d0400 msgr2=0x7fd86c0d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.754+0000 7fd874400640 1 -- 172.21.15.7:0/1177044779 shutdown_connections 2024-04-25T11:13:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.754+0000 7fd874400640 1 -- 172.21.15.7:0/1177044779 wait complete. 2024-04-25T11:13:25.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.756+0000 7fd874400640 1 Processor -- start 2024-04-25T11:13:25.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.756+0000 7fd874400640 1 -- start start 2024-04-25T11:13:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.758+0000 7fd874400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd86c0b8980 0x7fd86c1d6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.758+0000 7fd874400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd86c0ce9e0 0x7fd86c0c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.758+0000 7fd874400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0d1f60 0x7fd86c0c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.758+0000 7fd874400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c0b8ed0 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.758+0000 7fd874400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c0cf330 con 0x7fd86c0d1f60 2024-04-25T11:13:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.758+0000 7fd874400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd86c0b9050 con 0x7fd86c0b8980 2024-04-25T11:13:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.758+0000 7fd871000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd86c0ce9e0 0x7fd86c0c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.758+0000 7fd871000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd86c0ce9e0 0x7fd86c0c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45298/0 (socket says 172.21.15.7:45298) 2024-04-25T11:13:25.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.758+0000 7fd871000640 1 -- 172.21.15.7:0/2020019225 learned_addr learned my addr 172.21.15.7:0/2020019225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd872400640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0d1f60 0x7fd86c0c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd871a00640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd86c0b8980 0x7fd86c1d6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd871000640 1 -- 172.21.15.7:0/2020019225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd86c0b8980 msgr2=0x7fd86c1d6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd871000640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd86c0b8980 0x7fd86c1d6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd871000640 1 -- 172.21.15.7:0/2020019225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0d1f60 msgr2=0x7fd86c0c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd871000640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0d1f60 0x7fd86c0c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd871000640 1 -- 172.21.15.7:0/2020019225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c0c83e0 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd872400640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0d1f60 0x7fd86c0c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:25.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd871000640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd86c0ce9e0 0x7fd86c0c7ea0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00ce00 tx=0x7fd85c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd869600640 1 -- 172.21.15.7:0/2020019225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00f680 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd869600640 1 -- 172.21.15.7:0/2020019225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c00f820 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd869600640 1 -- 172.21.15.7:0/2020019225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00fc30 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd874400640 1 -- 172.21.15.7:0/2020019225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86c0c88f0 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.759+0000 7fd874400640 1 -- 172.21.15.7:0/2020019225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd86c1d74b0 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.761+0000 7fd874400640 1 -- 172.21.15.7:0/2020019225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd86c06a100 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.764+0000 7fd869600640 1 -- 172.21.15.7:0/2020019225 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd85c018400 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.765+0000 7fd869600640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 0x7fd8500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:25.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.766+0000 7fd869600640 1 -- 172.21.15.7:0/2020019225 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7fd85c0960b0 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.766+0000 7fd871a00640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 0x7fd8500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:25.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.767+0000 7fd871a00640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 0x7fd8500769c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd860034000 tx=0x7fd86002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:25.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.771+0000 7fd869600640 1 -- 172.21.15.7:0/2020019225 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd85c014030 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.976+0000 7fd874400640 1 -- 172.21.15.7:0/2020019225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd86c0b4f10 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.980+0000 7fd869600640 1 -- 172.21.15.7:0/2020019225 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16839 (secure 0 0 0) 0x7fd85c0600d0 con 0x7fd86c0ce9e0 2024-04-25T11:13:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:25.981 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":42,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:04.193129+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:12:14.961811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T11:12:16.128549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 -- 172.21.15.7:0/2020019225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 msgr2=0x7fd8500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 0x7fd8500769c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd860034000 tx=0x7fd86002e000 comp rx=0 tx=0).stop 2024-04-25T11:13:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 -- 172.21.15.7:0/2020019225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd86c0ce9e0 msgr2=0x7fd86c0c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd86c0ce9e0 0x7fd86c0c7ea0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00ce00 tx=0x7fd85c00a650 comp rx=0 tx=0).stop 2024-04-25T11:13:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 -- 172.21.15.7:0/2020019225 shutdown_connections 2024-04-25T11:13:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8500744b0 0x7fd8500769c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0d1f60 0x7fd86c0c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd86c0ce9e0 0x7fd86c0c7ea0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 --2- 172.21.15.7:0/2020019225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd86c0b8980 0x7fd86c1d6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 -- 172.21.15.7:0/2020019225 >> 172.21.15.7:0/2020019225 conn(0x7fd86c0d0400 msgr2=0x7fd86c0d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 -- 172.21.15.7:0/2020019225 shutdown_connections 2024-04-25T11:13:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:25.983+0000 7fd874400640 1 -- 172.21.15.7:0/2020019225 wait complete. 2024-04-25T11:13:26.010 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:13:26.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.281+0000 7f05b2400640 1 Processor -- start 2024-04-25T11:13:26.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.281+0000 7f05b2400640 1 -- start start 2024-04-25T11:13:26.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.284+0000 7f05b2400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05ac15eed0 0x7f05ac0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.284+0000 7f05b2400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05ac141290 0x7f05ac1416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.284+0000 7f05b2400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05ac06c570 con 0x7f05ac06d610 2024-04-25T11:13:26.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.284+0000 7f05b2400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f05ac06c850 con 0x7f05ac141290 2024-04-25T11:13:26.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.284+0000 7f05b2400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f05ac06cb30 con 0x7f05ac15eed0 2024-04-25T11:13:26.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.284+0000 7f05abe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05ac141290 0x7f05ac1416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:26.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.284+0000 7f05abe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05ac141290 0x7f05ac1416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45306/0 (socket says 172.21.15.7:45306) 2024-04-25T11:13:26.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.284+0000 7f05abe00640 1 -- 172.21.15.7:0/1013516004 learned_addr learned my addr 172.21.15.7:0/1013516004 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:26.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.285+0000 7f05b1000640 1 --2- 172.21.15.7:0/1013516004 >> v2:172.21.15.87:3300/0 conn(0x7f05ac15eed0 0x7f05ac0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:26.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.285+0000 7f05abe00640 1 -- 172.21.15.7:0/1013516004 >> v2:172.21.15.87:3300/0 conn(0x7f05ac15eed0 msgr2=0x7f05ac0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:26.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.285+0000 7f05abe00640 1 --2- 172.21.15.7:0/1013516004 >> v2:172.21.15.87:3300/0 conn(0x7f05ac15eed0 0x7f05ac0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.285+0000 7f05abe00640 1 -- 172.21.15.7:0/1013516004 >> v1:172.21.15.87:6789/0 conn(0x7f05ac06d610 legacy=0x7f05ac0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:26.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.285+0000 7f05abe00640 1 -- 172.21.15.7:0/1013516004 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac06ce10 con 0x7f05ac141290 2024-04-25T11:13:26.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.285+0000 7f05b1000640 1 --2- 172.21.15.7:0/1013516004 >> v2:172.21.15.87:3300/0 conn(0x7f05ac15eed0 0x7f05ac0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:26.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.286+0000 7f05abe00640 1 --2- 172.21.15.7:0/1013516004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ac141290 0x7f05ac1416a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f05ac074220 tx=0x7f059802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59ffeef7fb79aef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:26.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.286+0000 7f05ab400640 1 -- 172.21.15.7:0/1013516004 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598037070 con 0x7f05ac141290 2024-04-25T11:13:26.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.286+0000 7f05ab400640 1 -- 172.21.15.7:0/1013516004 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0598032440 con 0x7f05ac141290 2024-04-25T11:13:26.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.286+0000 7f05ab400640 1 -- 172.21.15.7:0/1013516004 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598032810 con 0x7f05ac141290 2024-04-25T11:13:26.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.287+0000 7f05b2400640 1 -- 172.21.15.7:0/1013516004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ac141290 msgr2=0x7f05ac1416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:26.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.287+0000 7f05b2400640 1 --2- 172.21.15.7:0/1013516004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ac141290 0x7f05ac1416a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f05ac074220 tx=0x7f059802a230 comp rx=0 tx=0).stop 2024-04-25T11:13:26.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.287+0000 7f05b2400640 1 -- 172.21.15.7:0/1013516004 shutdown_connections 2024-04-25T11:13:26.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.287+0000 7f05b2400640 1 --2- 172.21.15.7:0/1013516004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ac141290 0x7f05ac1416a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.287+0000 7f05b2400640 1 --2- 172.21.15.7:0/1013516004 >> v2:172.21.15.87:3300/0 conn(0x7f05ac15eed0 0x7f05ac0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.287+0000 7f05b2400640 1 -- 172.21.15.7:0/1013516004 >> 172.21.15.7:0/1013516004 conn(0x7f05ac0d1540 msgr2=0x7f05ac0d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:26.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.287+0000 7f05b2400640 1 -- 172.21.15.7:0/1013516004 shutdown_connections 2024-04-25T11:13:26.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.287+0000 7f05b2400640 1 -- 172.21.15.7:0/1013516004 wait complete. 2024-04-25T11:13:26.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.290+0000 7f05b2400640 1 Processor -- start 2024-04-25T11:13:26.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.290+0000 7f05b2400640 1 -- start start 2024-04-25T11:13:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.292+0000 7f05b2400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05ac06d610 0x7f05ac1f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.292+0000 7f05b2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ac141290 0x7f05ac14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.292+0000 7f05b2400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ac15eed0 0x7f05ac136dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.292+0000 7f05b2400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac140da0 con 0x7f05ac141290 2024-04-25T11:13:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.292+0000 7f05b2400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac0cdcc0 con 0x7f05ac15eed0 2024-04-25T11:13:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05b2400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f05ac140f20 con 0x7f05ac06d610 2024-04-25T11:13:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05abe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ac141290 0x7f05ac14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05abe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ac141290 0x7f05ac14fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45316/0 (socket says 172.21.15.7:45316) 2024-04-25T11:13:26.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05abe00640 1 -- 172.21.15.7:0/1995814185 learned_addr learned my addr 172.21.15.7:0/1995814185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05b1000640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05ac06d610 0x7f05ac1f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05b1a00640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ac15eed0 0x7f05ac136dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05abe00640 1 -- 172.21.15.7:0/1995814185 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05ac06d610 msgr2=0x7f05ac1f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05abe00640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05ac06d610 0x7f05ac1f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05abe00640 1 -- 172.21.15.7:0/1995814185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ac15eed0 msgr2=0x7f05ac136dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05abe00640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ac15eed0 0x7f05ac136dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05abe00640 1 -- 172.21.15.7:0/1995814185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac150830 con 0x7f05ac141290 2024-04-25T11:13:26.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.293+0000 7f05b1a00640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ac15eed0 0x7f05ac136dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.294+0000 7f05abe00640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ac141290 0x7f05ac14fb00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0598000c00 tx=0x7f05980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.294+0000 7f05a8c00640 1 -- 172.21.15.7:0/1995814185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598041070 con 0x7f05ac141290 2024-04-25T11:13:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.294+0000 7f05a8c00640 1 -- 172.21.15.7:0/1995814185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059803c870 con 0x7f05ac141290 2024-04-25T11:13:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.294+0000 7f05a8c00640 1 -- 172.21.15.7:0/1995814185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598037040 con 0x7f05ac141290 2024-04-25T11:13:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.294+0000 7f05b2400640 1 -- 172.21.15.7:0/1995814185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ac150b10 con 0x7f05ac141290 2024-04-25T11:13:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.294+0000 7f05b2400640 1 -- 172.21.15.7:0/1995814185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ac1510d0 con 0x7f05ac141290 2024-04-25T11:13:26.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.295+0000 7f05b2400640 1 -- 172.21.15.7:0/1995814185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ac1505a0 con 0x7f05ac141290 2024-04-25T11:13:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.297+0000 7f05a8c00640 1 -- 172.21.15.7:0/1995814185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0598044070 con 0x7f05ac141290 2024-04-25T11:13:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.298+0000 7f05a8c00640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f058c0744b0 0x7f058c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.298+0000 7f05a8c00640 1 -- 172.21.15.7:0/1995814185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6130+0+0 (secure 0 0 0) 0x7f05980b5ff0 con 0x7f05ac141290 2024-04-25T11:13:26.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.298+0000 7f05b1000640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f058c0744b0 0x7f058c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:26.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.298+0000 7f05b1000640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f058c0744b0 0x7f058c0769c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f05ac071fe0 tx=0x7f05a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.302+0000 7f05a8c00640 1 -- 172.21.15.7:0/1995814185 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05980805e0 con 0x7f05ac141290 2024-04-25T11:13:26.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.538+0000 7f05b2400640 1 -- 172.21.15.7:0/1995814185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f05ac050740 con 0x7f05ac141290 2024-04-25T11:13:26.540 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-04-25T11:13:26.540+0000 7fac34e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:13:26.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.544+0000 7f05a8c00640 1 -- 172.21.15.7:0/1995814185 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f059807fee0 con 0x7f05ac141290 2024-04-25T11:13:26.544 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.546+0000 7f05b2400640 1 -- 172.21.15.7:0/1995814185 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f058c0744b0 msgr2=0x7f058c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.546+0000 7f05b2400640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f058c0744b0 0x7f058c0769c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f05ac071fe0 tx=0x7f05a0008040 comp rx=0 tx=0).stop 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.546+0000 7f05b2400640 1 -- 172.21.15.7:0/1995814185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ac141290 msgr2=0x7f05ac14fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.546+0000 7f05b2400640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ac141290 0x7f05ac14fb00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0598000c00 tx=0x7f05980046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.546+0000 7f05b2400640 1 -- 172.21.15.7:0/1995814185 shutdown_connections 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.546+0000 7f05b2400640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f058c0744b0 0x7f058c0769c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.546+0000 7f05b2400640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05ac15eed0 0x7f05ac136dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.546+0000 7f05b2400640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05ac141290 0x7f05ac14fb00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.546+0000 7f05b2400640 1 --2- 172.21.15.7:0/1995814185 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05ac06d610 0x7f05ac1f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.546+0000 7f05b2400640 1 -- 172.21.15.7:0/1995814185 >> 172.21.15.7:0/1995814185 conn(0x7f05ac0d1540 msgr2=0x7f05ac06f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.547+0000 7f05b2400640 1 -- 172.21.15.7:0/1995814185 shutdown_connections 2024-04-25T11:13:26.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.547+0000 7f05b2400640 1 -- 172.21.15.7:0/1995814185 wait complete. 2024-04-25T11:13:26.573 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:13:26.747 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:13:26.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.837+0000 7f7f97e00640 1 Processor -- start 2024-04-25T11:13:26.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.837+0000 7f7f97e00640 1 -- start start 2024-04-25T11:13:26.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.839+0000 7f7f97e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f980b2550 0x7f7f980b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.839+0000 7f7f97e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f9809b960 0x7f7f9809bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.839+0000 7f7f97e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f9809c2b0 con 0x7f7f980b2aa0 2024-04-25T11:13:26.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.839+0000 7f7f97e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7f980d2320 con 0x7f7f980b2550 2024-04-25T11:13:26.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.840+0000 7f7f97e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f98160770 con 0x7f7f9809b960 2024-04-25T11:13:26.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.840+0000 7f7f96a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f980b2550 0x7f7f980b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:26.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.840+0000 7f7f96a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f980b2550 0x7f7f980b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:58920/0 (socket says 172.21.15.7:58920) 2024-04-25T11:13:26.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.840+0000 7f7f96a00640 1 -- 172.21.15.7:0/2963964085 learned_addr learned my addr 172.21.15.7:0/2963964085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:26.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.840+0000 7f7f97400640 1 --2- 172.21.15.7:0/2963964085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f9809b960 0x7f7f9809bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:26.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.841+0000 7f7f96a00640 1 -- 172.21.15.7:0/2963964085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f9809b960 msgr2=0x7f7f9809bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:26.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.841+0000 7f7f96a00640 1 --2- 172.21.15.7:0/2963964085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f9809b960 0x7f7f9809bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.841+0000 7f7f96a00640 1 -- 172.21.15.7:0/2963964085 >> v1:172.21.15.87:6789/0 conn(0x7f7f980b2aa0 legacy=0x7f7f980b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:26.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.841+0000 7f7f96a00640 1 -- 172.21.15.7:0/2963964085 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9806e550 con 0x7f7f980b2550 2024-04-25T11:13:26.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.842+0000 7f7f96a00640 1 --2- 172.21.15.7:0/2963964085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f980b2550 0x7f7f980b2ff0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009870 tx=0x7f7f8802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe14e5e1a3b0f017 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:26.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.843+0000 7f7f95600640 1 -- 172.21.15.7:0/2963964085 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88037070 con 0x7f7f980b2550 2024-04-25T11:13:26.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.843+0000 7f7f95600640 1 -- 172.21.15.7:0/2963964085 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f88032440 con 0x7f7f980b2550 2024-04-25T11:13:26.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.843+0000 7f7f95600640 1 -- 172.21.15.7:0/2963964085 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88032810 con 0x7f7f980b2550 2024-04-25T11:13:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.843+0000 7f7f97e00640 1 -- 172.21.15.7:0/2963964085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f980b2550 msgr2=0x7f7f980b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.843+0000 7f7f97e00640 1 --2- 172.21.15.7:0/2963964085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f980b2550 0x7f7f980b2ff0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009870 tx=0x7f7f8802a270 comp rx=0 tx=0).stop 2024-04-25T11:13:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.844+0000 7f7f97e00640 1 -- 172.21.15.7:0/2963964085 shutdown_connections 2024-04-25T11:13:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.844+0000 7f7f97e00640 1 --2- 172.21.15.7:0/2963964085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f9809b960 0x7f7f9809bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.844+0000 7f7f97e00640 1 --2- 172.21.15.7:0/2963964085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f980b2550 0x7f7f980b2ff0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.844+0000 7f7f97e00640 1 -- 172.21.15.7:0/2963964085 >> 172.21.15.7:0/2963964085 conn(0x7f7f980b5240 msgr2=0x7f7f980b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.844+0000 7f7f97e00640 1 -- 172.21.15.7:0/2963964085 shutdown_connections 2024-04-25T11:13:26.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.844+0000 7f7f97e00640 1 -- 172.21.15.7:0/2963964085 wait complete. 2024-04-25T11:13:26.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.847+0000 7f7f97e00640 1 Processor -- start 2024-04-25T11:13:26.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.847+0000 7f7f97e00640 1 -- start start 2024-04-25T11:13:26.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.849+0000 7f7f97e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9809b960 0x7f7f980c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.849+0000 7f7f97e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f980b2550 0x7f7f9815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.849+0000 7f7f97e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f980b2aa0 0x7f7f981f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.849+0000 7f7f97e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f980d1fb0 con 0x7f7f9809b960 2024-04-25T11:13:26.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.849+0000 7f7f97e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7f980d1cd0 con 0x7f7f980b2aa0 2024-04-25T11:13:26.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.849+0000 7f7f97e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f9806eb10 con 0x7f7f980b2550 2024-04-25T11:13:26.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.849+0000 7f7f96a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9809b960 0x7f7f980c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:26.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.850+0000 7f7f96a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9809b960 0x7f7f980c8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45330/0 (socket says 172.21.15.7:45330) 2024-04-25T11:13:26.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.850+0000 7f7f96a00640 1 -- 172.21.15.7:0/807645577 learned_addr learned my addr 172.21.15.7:0/807645577 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:26.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.850+0000 7f7f96a00640 1 -- 172.21.15.7:0/807645577 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f980b2550 msgr2=0x7f7f9815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:26.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.850+0000 7f7f96000640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f980b2550 0x7f7f9815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:26.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.850+0000 7f7f96a00640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f980b2550 0x7f7f9815c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.850+0000 7f7f96a00640 1 -- 172.21.15.7:0/807645577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f980b2aa0 msgr2=0x7f7f981f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:26.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.850+0000 7f7f96a00640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f980b2aa0 0x7f7f981f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:26.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.850+0000 7f7f96a00640 1 -- 172.21.15.7:0/807645577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f98163580 con 0x7f7f9809b960 2024-04-25T11:13:26.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.850+0000 7f7f96a00640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9809b960 0x7f7f980c8c90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7f88000c00 tx=0x7f7f880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:26.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.851+0000 7f7f8f400640 1 -- 172.21.15.7:0/807645577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88041020 con 0x7f7f9809b960 2024-04-25T11:13:26.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.851+0000 7f7f8f400640 1 -- 172.21.15.7:0/807645577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f880306c0 con 0x7f7f9809b960 2024-04-25T11:13:26.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.851+0000 7f7f8f400640 1 -- 172.21.15.7:0/807645577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88037040 con 0x7f7f9809b960 2024-04-25T11:13:26.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.851+0000 7f7f97e00640 1 -- 172.21.15.7:0/807645577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f98163860 con 0x7f7f9809b960 2024-04-25T11:13:26.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.851+0000 7f7f97e00640 1 -- 172.21.15.7:0/807645577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f980c8770 con 0x7f7f9809b960 2024-04-25T11:13:26.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.852+0000 7f7f97e00640 1 -- 172.21.15.7:0/807645577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f98050740 con 0x7f7f9809b960 2024-04-25T11:13:26.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.853+0000 7f7f8f400640 1 -- 172.21.15.7:0/807645577 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7f88044020 con 0x7f7f9809b960 2024-04-25T11:13:26.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.854+0000 7f7f8f400640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f740744b0 0x7f7f740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:26.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.855+0000 7f7f8f400640 1 -- 172.21.15.7:0/807645577 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6216+0+0 (secure 0 0 0) 0x7f7f880b5d60 con 0x7f7f9809b960 2024-04-25T11:13:26.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.855+0000 7f7f96000640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f740744b0 0x7f7f740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:26.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.855+0000 7f7f96000640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f740744b0 0x7f7f740769c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7f980c9df0 tx=0x7f7f80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:26.860+0000 7f7f8f400640 1 -- 172.21.15.7:0/807645577 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f8804b030 con 0x7f7f9809b960 2024-04-25T11:13:27.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.101+0000 7f7f97e00640 1 -- 172.21.15.7:0/807645577 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7f9809c6d0 con 0x7f7f9809b960 2024-04-25T11:13:27.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.548+0000 7f7f8f400640 1 -- 172.21.15.7:0/807645577 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f7f88080250 con 0x7f7f9809b960 2024-04-25T11:13:27.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.552+0000 7f7f97e00640 1 -- 172.21.15.7:0/807645577 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f740744b0 msgr2=0x7f7f740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:27.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.552+0000 7f7f97e00640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f740744b0 0x7f7f740769c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7f980c9df0 tx=0x7f7f80009040 comp rx=0 tx=0).stop 2024-04-25T11:13:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.552+0000 7f7f97e00640 1 -- 172.21.15.7:0/807645577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9809b960 msgr2=0x7f7f980c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.552+0000 7f7f97e00640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9809b960 0x7f7f980c8c90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7f88000c00 tx=0x7f7f880046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.553+0000 7f7f97e00640 1 -- 172.21.15.7:0/807645577 shutdown_connections 2024-04-25T11:13:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.553+0000 7f7f97e00640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f740744b0 0x7f7f740769c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.553+0000 7f7f97e00640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f980b2aa0 0x7f7f981f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.553+0000 7f7f97e00640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f980b2550 0x7f7f9815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.553+0000 7f7f97e00640 1 --2- 172.21.15.7:0/807645577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f9809b960 0x7f7f980c8c90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.553+0000 7f7f97e00640 1 -- 172.21.15.7:0/807645577 >> 172.21.15.7:0/807645577 conn(0x7f7f980b5240 msgr2=0x7f7f980b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.553+0000 7f7f97e00640 1 -- 172.21.15.7:0/807645577 shutdown_connections 2024-04-25T11:13:27.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.553+0000 7f7f97e00640 1 -- 172.21.15.7:0/807645577 wait complete. 2024-04-25T11:13:27.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:13:27.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.863+0000 7fc3c9c00640 1 Processor -- start 2024-04-25T11:13:27.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.863+0000 7fc3c9c00640 1 -- start start 2024-04-25T11:13:27.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.865+0000 7fc3c9c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c40b48a0 0x7fc3c415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:27.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.866+0000 7fc3c9c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c40b4da0 con 0x7fc3c40b0940 2024-04-25T11:13:27.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.866+0000 7fc3c9c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c40b5080 con 0x7fc3c40aec60 2024-04-25T11:13:27.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.866+0000 7fc3c9c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c40b5360 con 0x7fc3c40b48a0 2024-04-25T11:13:27.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.866+0000 7fc3c3400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc3c40b0940 0x7fc3c40b0d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42118/0 (socket says 172.21.15.7:42118) 2024-04-25T11:13:27.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.866+0000 7fc3c3400640 1 -- 172.21.15.7:0/2618439128 learned_addr learned my addr 172.21.15.7:0/2618439128 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:27.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.866+0000 7fc3c2a00640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c40b48a0 0x7fc3c415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082921368 0 0) 0x7fc3c40b4da0 con 0x7fc3c40b0940 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3a8003520 con 0x7fc3c40b0940 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688765104 0 0) 0x7fc3c40b5080 con 0x7fc3c40aec60 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3c40b4da0 con 0x7fc3c40aec60 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3330062551 0 0) 0x7fc3a8003520 con 0x7fc3c40b0940 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c40b48a0 msgr2=0x7fc3c415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c40b48a0 0x7fc3c415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 >> v1:172.21.15.87:6789/0 conn(0x7fc3c40aec60 legacy=0x7fc3c40b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c40b5640 con 0x7fc3c40b0940 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fc3bc0019f0 con 0x7fc3c40b0940 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 >> v1:172.21.15.7:6789/0 conn(0x7fc3c40b0940 legacy=0x7fc3c40b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3a8003b70 0x7fc3a8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c40b48a0 0x7fc3a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8004d40 0x7fc3a8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c40b5360 con 0x7fc3a8004d40 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a800bac0 con 0x7fc3a8003b70 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.867+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a800bcc0 con 0x7fc3c40b48a0 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.868+0000 7fc3c3400640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8004d40 0x7fc3a8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.868+0000 7fc3c2a00640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3a8003b70 0x7fc3a8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.868+0000 7fc3bbe00640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c40b48a0 0x7fc3a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.868+0000 7fc3c3400640 1 -- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c40b48a0 msgr2=0x7fc3a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.868+0000 7fc3c3400640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c40b48a0 0x7fc3a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.868+0000 7fc3c3400640 1 -- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3a8003b70 msgr2=0x7fc3a8003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.868+0000 7fc3c3400640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3a8003b70 0x7fc3a8003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.868+0000 7fc3c3400640 1 -- 172.21.15.7:0/2618439128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a8003520 con 0x7fc3a8004d40 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.868+0000 7fc3c2a00640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3a8003b70 0x7fc3a8003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:13:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.868+0000 7fc3bbe00640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c40b48a0 0x7fc3a8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:27.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.869+0000 7fc3c3400640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8004d40 0x7fc3a8005150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00a800 tx=0x7fc3a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da91ff2ee3cc4146 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:27.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.870+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc00a450 con 0x7fc3a8004d40 2024-04-25T11:13:27.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.870+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3bc02f070 con 0x7fc3a8004d40 2024-04-25T11:13:27.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.870+0000 7fc3c2000640 1 -- 172.21.15.7:0/2618439128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc02bc90 con 0x7fc3a8004d40 2024-04-25T11:13:27.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.870+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2618439128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8004d40 msgr2=0x7fc3a8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:27.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.870+0000 7fc3c9c00640 1 --2- 172.21.15.7:0/2618439128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a8004d40 0x7fc3a8005150 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00a800 tx=0x7fc3a8005830 comp rx=0 tx=0).stop 2024-04-25T11:13:27.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.870+0000 7fc3c3400640 1 -- 172.21.15.7:0/2618439128 reap_dead start 2024-04-25T11:13:27.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.871+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2618439128 shutdown_connections 2024-04-25T11:13:27.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.871+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2618439128 >> 172.21.15.7:0/2618439128 conn(0x7fc3c4073920 msgr2=0x7fc3c4073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:27.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.871+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2618439128 shutdown_connections 2024-04-25T11:13:27.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.871+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2618439128 wait complete. 2024-04-25T11:13:27.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.873+0000 7fc3c9c00640 1 Processor -- start 2024-04-25T11:13:27.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.873+0000 7fc3c9c00640 1 -- start start 2024-04-25T11:13:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.875+0000 7fc3c9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3a8004d40 0x7fc3c409c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.875+0000 7fc3c9c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c415b340 0x7fc3c415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.875+0000 7fc3c9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c415a560 0x7fc3c415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.875+0000 7fc3c9c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c40b0530 con 0x7fc3c415a560 2024-04-25T11:13:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.875+0000 7fc3c9c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c40b03b0 con 0x7fc3a8004d40 2024-04-25T11:13:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.875+0000 7fc3c9c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c415dc90 con 0x7fc3c415b340 2024-04-25T11:13:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.875+0000 7fc3c3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c415a560 0x7fc3c415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:27.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.875+0000 7fc3c3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c415a560 0x7fc3c415cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45344/0 (socket says 172.21.15.7:45344) 2024-04-25T11:13:27.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.875+0000 7fc3c3400640 1 -- 172.21.15.7:0/2373055020 learned_addr learned my addr 172.21.15.7:0/2373055020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:27.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3c2a00640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3a8004d40 0x7fc3c409c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:27.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3bbe00640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c415b340 0x7fc3c415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:27.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3c3400640 1 -- 172.21.15.7:0/2373055020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c415b340 msgr2=0x7fc3c415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:27.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3c3400640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c415b340 0x7fc3c415a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:27.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3c3400640 1 -- 172.21.15.7:0/2373055020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3a8004d40 msgr2=0x7fc3c409c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:27.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3c3400640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3a8004d40 0x7fc3c409c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:27.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3c3400640 1 -- 172.21.15.7:0/2373055020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c409b750 con 0x7fc3c415a560 2024-04-25T11:13:27.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3c3400640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c415a560 0x7fc3c415cf60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc02b4c0 tx=0x7fc3c40b5250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:27.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3baa00640 1 -- 172.21.15.7:0/2373055020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc044070 con 0x7fc3c415a560 2024-04-25T11:13:27.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3baa00640 1 -- 172.21.15.7:0/2373055020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3bc001250 con 0x7fc3c415a560 2024-04-25T11:13:27.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3baa00640 1 -- 172.21.15.7:0/2373055020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc00a450 con 0x7fc3c415a560 2024-04-25T11:13:27.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.876+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2373055020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c409ba30 con 0x7fc3c415a560 2024-04-25T11:13:27.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.877+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2373055020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c409bff0 con 0x7fc3c415a560 2024-04-25T11:13:27.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.878+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2373055020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c406a100 con 0x7fc3c415a560 2024-04-25T11:13:27.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.879+0000 7fc3baa00640 1 -- 172.21.15.7:0/2373055020 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc3bc047070 con 0x7fc3c415a560 2024-04-25T11:13:27.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.881+0000 7fc3baa00640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3a00744b0 0x7fc3a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:27.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.881+0000 7fc3baa00640 1 -- 172.21.15.7:0/2373055020 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6216+0+0 (secure 0 0 0) 0x7fc3bc0b5610 con 0x7fc3c415a560 2024-04-25T11:13:27.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.881+0000 7fc3c2a00640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3a00744b0 0x7fc3a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:27.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.882+0000 7fc3c2a00640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3a00744b0 0x7fc3a00769c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40cf3f0 tx=0x7fc3b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:27.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:27.886+0000 7fc3baa00640 1 -- 172.21.15.7:0/2373055020 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc3bc07fb00 con 0x7fc3c415a560 2024-04-25T11:13:28.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.093+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2373055020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc3c40b68d0 con 0x7fc3c415a560 2024-04-25T11:13:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.559+0000 7fc3baa00640 1 -- 172.21.15.7:0/2373055020 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) ==== 181+0+0 (secure 0 0 0) 0x7fc3bc07f400 con 0x7fc3c415a560 2024-04-25T11:13:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:13:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.564+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2373055020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3a00744b0 msgr2=0x7fc3a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.564+0000 7fc3c9c00640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3a00744b0 0x7fc3a00769c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40cf3f0 tx=0x7fc3b4008040 comp rx=0 tx=0).stop 2024-04-25T11:13:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.564+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2373055020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c415a560 msgr2=0x7fc3c415cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.564+0000 7fc3c9c00640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c415a560 0x7fc3c415cf60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc02b4c0 tx=0x7fc3c40b5250 comp rx=0 tx=0).stop 2024-04-25T11:13:28.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.564+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2373055020 shutdown_connections 2024-04-25T11:13:28.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.565+0000 7fc3c9c00640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3a00744b0 0x7fc3a00769c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:28.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.565+0000 7fc3c9c00640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c415a560 0x7fc3c415cf60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.565+0000 7fc3c9c00640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3c415b340 0x7fc3c415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.565+0000 7fc3c9c00640 1 --2- 172.21.15.7:0/2373055020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3a8004d40 0x7fc3c409c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.565+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2373055020 >> 172.21.15.7:0/2373055020 conn(0x7fc3c4073920 msgr2=0x7fc3c406d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.566+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2373055020 shutdown_connections 2024-04-25T11:13:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.566+0000 7fc3c9c00640 1 -- 172.21.15.7:0/2373055020 wait complete. 2024-04-25T11:13:28.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:13:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.887+0000 7f3226400640 1 Processor -- start 2024-04-25T11:13:28.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.887+0000 7f3226400640 1 -- start start 2024-04-25T11:13:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.890+0000 7f3226400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f322014d090 0x7f322006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.890+0000 7f3226400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322006d580 0x7f322006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.890+0000 7f3226400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f322006ded0 con 0x7f32200cee40 2024-04-25T11:13:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.890+0000 7f3226400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f322006f4c0 con 0x7f322014d090 2024-04-25T11:13:28.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.890+0000 7f3226400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f322006f7a0 con 0x7f322006d580 2024-04-25T11:13:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.891+0000 7f3225a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322006d580 0x7f322006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.891+0000 7f3225a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322006d580 0x7f322006d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:36862/0 (socket says 172.21.15.7:36862) 2024-04-25T11:13:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.891+0000 7f3225a00640 1 -- 172.21.15.7:0/3910836183 learned_addr learned my addr 172.21.15.7:0/3910836183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:28.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.891+0000 7f3225000640 1 --2- 172.21.15.7:0/3910836183 >> v2:172.21.15.87:3300/0 conn(0x7f322014d090 0x7f322006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:28.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.892+0000 7f3225a00640 1 -- 172.21.15.7:0/3910836183 >> v2:172.21.15.87:3300/0 conn(0x7f322014d090 msgr2=0x7f322006eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:28.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.892+0000 7f3225a00640 1 --2- 172.21.15.7:0/3910836183 >> v2:172.21.15.87:3300/0 conn(0x7f322014d090 0x7f322006eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:28.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.892+0000 7f3225a00640 1 -- 172.21.15.7:0/3910836183 >> v1:172.21.15.87:6789/0 conn(0x7f32200cee40 legacy=0x7f3220070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:28.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.892+0000 7f3225a00640 1 -- 172.21.15.7:0/3910836183 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f322006fa80 con 0x7f322006d580 2024-04-25T11:13:28.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.893+0000 7f3225a00640 1 --2- 172.21.15.7:0/3910836183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322006d580 0x7f322006d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3218009e50 tx=0x7f321802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee4478a9e98da5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.894+0000 7f321f400640 1 -- 172.21.15.7:0/3910836183 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218036070 con 0x7f322006d580 2024-04-25T11:13:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.894+0000 7f321f400640 1 -- 172.21.15.7:0/3910836183 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3218031600 con 0x7f322006d580 2024-04-25T11:13:28.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.894+0000 7f321f400640 1 -- 172.21.15.7:0/3910836183 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32180319d0 con 0x7f322006d580 2024-04-25T11:13:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.895+0000 7f3226400640 1 -- 172.21.15.7:0/3910836183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322006d580 msgr2=0x7f322006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.895+0000 7f3226400640 1 --2- 172.21.15.7:0/3910836183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322006d580 0x7f322006d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3218009e50 tx=0x7f321802a680 comp rx=0 tx=0).stop 2024-04-25T11:13:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.895+0000 7f3226400640 1 -- 172.21.15.7:0/3910836183 shutdown_connections 2024-04-25T11:13:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.895+0000 7f3226400640 1 --2- 172.21.15.7:0/3910836183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322006d580 0x7f322006d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.895+0000 7f3226400640 1 --2- 172.21.15.7:0/3910836183 >> v2:172.21.15.87:3300/0 conn(0x7f322014d090 0x7f322006eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.895+0000 7f3226400640 1 -- 172.21.15.7:0/3910836183 >> 172.21.15.7:0/3910836183 conn(0x7f3220073160 msgr2=0x7f3220073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.895+0000 7f3226400640 1 -- 172.21.15.7:0/3910836183 shutdown_connections 2024-04-25T11:13:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.895+0000 7f3226400640 1 -- 172.21.15.7:0/3910836183 wait complete. 2024-04-25T11:13:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.897+0000 7f3226400640 1 Processor -- start 2024-04-25T11:13:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.898+0000 7f3226400640 1 -- start start 2024-04-25T11:13:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.899+0000 7f3226400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322006d580 0x7f32201464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.899+0000 7f3226400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200cee40 0x7f3220148b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.899+0000 7f3226400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322014d090 0x7f32201490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.899+0000 7f3226400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32200cf250 con 0x7f32200cee40 2024-04-25T11:13:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.899+0000 7f3226400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f32200b4690 con 0x7f322006d580 2024-04-25T11:13:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.899+0000 7f3226400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f32200b4bb0 con 0x7f322014d090 2024-04-25T11:13:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.900+0000 7f321fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200cee40 0x7f3220148b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.900+0000 7f321fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200cee40 0x7f3220148b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45352/0 (socket says 172.21.15.7:45352) 2024-04-25T11:13:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.900+0000 7f321fe00640 1 -- 172.21.15.7:0/2186189406 learned_addr learned my addr 172.21.15.7:0/2186189406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.900+0000 7f3225a00640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322014d090 0x7f32201490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.900+0000 7f3225000640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322006d580 0x7f32201464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.900+0000 7f321fe00640 1 -- 172.21.15.7:0/2186189406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322014d090 msgr2=0x7f32201490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.900+0000 7f321fe00640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322014d090 0x7f32201490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.900+0000 7f321fe00640 1 -- 172.21.15.7:0/2186189406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322006d580 msgr2=0x7f32201464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.900+0000 7f321fe00640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322006d580 0x7f32201464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.900+0000 7f321fe00640 1 -- 172.21.15.7:0/2186189406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32201469f0 con 0x7f32200cee40 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.900+0000 7f3225a00640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322014d090 0x7f32201490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.901+0000 7f321fe00640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200cee40 0x7f3220148b70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f320c00c3f0 tx=0x7f320c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.901+0000 7f321cc00640 1 -- 172.21.15.7:0/2186189406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0052d0 con 0x7f32200cee40 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.901+0000 7f321cc00640 1 -- 172.21.15.7:0/2186189406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320c005490 con 0x7f32200cee40 2024-04-25T11:13:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.901+0000 7f321cc00640 1 -- 172.21.15.7:0/2186189406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c007260 con 0x7f32200cee40 2024-04-25T11:13:28.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.901+0000 7f3226400640 1 -- 172.21.15.7:0/2186189406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3220166780 con 0x7f32200cee40 2024-04-25T11:13:28.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.901+0000 7f3226400640 1 -- 172.21.15.7:0/2186189406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3220166d40 con 0x7f32200cee40 2024-04-25T11:13:28.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.903+0000 7f3226400640 1 -- 172.21.15.7:0/2186189406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3220149bf0 con 0x7f32200cee40 2024-04-25T11:13:28.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.904+0000 7f321cc00640 1 -- 172.21.15.7:0/2186189406 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f320c007400 con 0x7f32200cee40 2024-04-25T11:13:28.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.905+0000 7f321cc00640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744e0 0x7f32000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:28.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.905+0000 7f321cc00640 1 -- 172.21.15.7:0/2186189406 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5778+0+0 (secure 0 0 0) 0x7f320c094690 con 0x7f32200cee40 2024-04-25T11:13:28.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.905+0000 7f3225000640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744e0 0x7f32000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:28.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.906+0000 7f3225000640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744e0 0x7f32000769f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3214009a30 tx=0x7f3214008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:28.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:28.910+0000 7f321cc00640 1 -- 172.21.15.7:0/2186189406 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f322006ded0 con 0x7f32200cee40 2024-04-25T11:13:29.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.115+0000 7f3226400640 1 -- 172.21.15.7:0/2186189406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3220050740 con 0x7f32200cee40 2024-04-25T11:13:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.563+0000 7f321cc00640 1 -- 172.21.15.7:0/2186189406 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) ==== 169+0+0 (secure 0 0 0) 0x7f320c05ee50 con 0x7f32200cee40 2024-04-25T11:13:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-25T11:13:29.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.567+0000 7f3226400640 1 -- 172.21.15.7:0/2186189406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744e0 msgr2=0x7f32000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:29.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.567+0000 7f3226400640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744e0 0x7f32000769f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3214009a30 tx=0x7f3214008040 comp rx=0 tx=0).stop 2024-04-25T11:13:29.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.567+0000 7f3226400640 1 -- 172.21.15.7:0/2186189406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200cee40 msgr2=0x7f3220148b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:29.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.567+0000 7f3226400640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200cee40 0x7f3220148b70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f320c00c3f0 tx=0x7f320c00c8b0 comp rx=0 tx=0).stop 2024-04-25T11:13:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.567+0000 7f3226400640 1 -- 172.21.15.7:0/2186189406 shutdown_connections 2024-04-25T11:13:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.567+0000 7f3226400640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744e0 0x7f32000769f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.568+0000 7f3226400640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322014d090 0x7f32201490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.568+0000 7f3226400640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200cee40 0x7f3220148b70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.568+0000 7f3226400640 1 --2- 172.21.15.7:0/2186189406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f322006d580 0x7f32201464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.568+0000 7f3226400640 1 -- 172.21.15.7:0/2186189406 >> 172.21.15.7:0/2186189406 conn(0x7f3220073160 msgr2=0x7f3220167d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:29.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.568+0000 7f3226400640 1 -- 172.21.15.7:0/2186189406 shutdown_connections 2024-04-25T11:13:29.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.568+0000 7f3226400640 1 -- 172.21.15.7:0/2186189406 wait complete. 2024-04-25T11:13:29.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_session_cap_acquisition_throttle 2024-04-25T11:13:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.874+0000 7f7c1d200640 1 Processor -- start 2024-04-25T11:13:29.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.874+0000 7f7c1d200640 1 -- start start 2024-04-25T11:13:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.877+0000 7f7c1d200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c180b7440 0x7f7c180d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.877+0000 7f7c1d200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c180c8700 con 0x7f7c181600e0 2024-04-25T11:13:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.877+0000 7f7c1d200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c180c89a0 con 0x7f7c180b1660 2024-04-25T11:13:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.877+0000 7f7c1d200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7c180c6cc0 con 0x7f7c180b7440 2024-04-25T11:13:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.877+0000 7f7c17e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7c181600e0 0x7f7c180c7710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52386/0 (socket says 172.21.15.7:52386) 2024-04-25T11:13:29.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.877+0000 7f7c17e00640 1 -- 172.21.15.7:0/3261734837 learned_addr learned my addr 172.21.15.7:0/3261734837 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.877+0000 7f7c16a00640 1 --2- 172.21.15.7:0/3261734837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c180b7440 0x7f7c180d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.878+0000 7f7c16000640 1 -- 172.21.15.7:0/3261734837 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2069222410 0 0) 0x7f7c180c8700 con 0x7f7c181600e0 2024-04-25T11:13:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.878+0000 7f7c16000640 1 -- 172.21.15.7:0/3261734837 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bfc003520 con 0x7f7c181600e0 2024-04-25T11:13:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.878+0000 7f7c16000640 1 -- 172.21.15.7:0/3261734837 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1392757216 0 0) 0x7f7c180c89a0 con 0x7f7c180b1660 2024-04-25T11:13:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.878+0000 7f7c16000640 1 -- 172.21.15.7:0/3261734837 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c180c8700 con 0x7f7c180b1660 2024-04-25T11:13:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.878+0000 7f7c16000640 1 -- 172.21.15.7:0/3261734837 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1910502290 0 0) 0x7f7bfc003520 con 0x7f7c181600e0 2024-04-25T11:13:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.878+0000 7f7c16a00640 1 -- 172.21.15.7:0/3261734837 >> v1:172.21.15.87:6789/0 conn(0x7f7c180b1660 legacy=0x7f7c1806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.878+0000 7f7c16a00640 1 -- 172.21.15.7:0/3261734837 >> v1:172.21.15.7:6789/0 conn(0x7f7c181600e0 legacy=0x7f7c180c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.878+0000 7f7c16a00640 1 -- 172.21.15.7:0/3261734837 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c180c6f10 con 0x7f7c180b7440 2024-04-25T11:13:29.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.880+0000 7f7c16a00640 1 --2- 172.21.15.7:0/3261734837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c180b7440 0x7f7c180d1c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c040097d0 tx=0x7f7c0402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2208ea2de6699fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:29.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.880+0000 7f7c16000640 1 -- 172.21.15.7:0/3261734837 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04037070 con 0x7f7c180b7440 2024-04-25T11:13:29.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.880+0000 7f7c16000640 1 -- 172.21.15.7:0/3261734837 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0402ac30 con 0x7f7c180b7440 2024-04-25T11:13:29.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.880+0000 7f7c16000640 1 -- 172.21.15.7:0/3261734837 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c100019f0 con 0x7f7c180b7440 2024-04-25T11:13:29.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.881+0000 7f7c1d200640 1 -- 172.21.15.7:0/3261734837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c180b7440 msgr2=0x7f7c180d1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:29.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.881+0000 7f7c1d200640 1 --2- 172.21.15.7:0/3261734837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c180b7440 0x7f7c180d1c90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7c040097d0 tx=0x7f7c0402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:13:29.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.881+0000 7f7c1d200640 1 -- 172.21.15.7:0/3261734837 shutdown_connections 2024-04-25T11:13:29.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.881+0000 7f7c1d200640 1 --2- 172.21.15.7:0/3261734837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c180b7440 0x7f7c180d1c90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:29.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.881+0000 7f7c1d200640 1 -- 172.21.15.7:0/3261734837 >> 172.21.15.7:0/3261734837 conn(0x7f7c1809b540 msgr2=0x7f7c180b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:29.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.882+0000 7f7c1d200640 1 -- 172.21.15.7:0/3261734837 shutdown_connections 2024-04-25T11:13:29.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.882+0000 7f7c1d200640 1 -- 172.21.15.7:0/3261734837 wait complete. 2024-04-25T11:13:29.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.884+0000 7f7c1d200640 1 Processor -- start 2024-04-25T11:13:29.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.884+0000 7f7c1d200640 1 -- start start 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.886+0000 7f7c1d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c180b1660 0x7f7c181eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.886+0000 7f7c1d200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c181600e0 0x7f7c180af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.886+0000 7f7c1d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c180b05a0 0x7f7c18157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.886+0000 7f7c1d200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c180c7280 con 0x7f7c180b05a0 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.886+0000 7f7c1d200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7c180b3890 con 0x7f7c180b1660 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.886+0000 7f7c1d200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7c180d21d0 con 0x7f7c181600e0 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.886+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c180b05a0 0x7f7c18157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.886+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c180b05a0 0x7f7c18157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44210/0 (socket says 172.21.15.7:44210) 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.886+0000 7f7c17e00640 1 -- 172.21.15.7:0/294016145 learned_addr learned my addr 172.21.15.7:0/294016145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.886+0000 7f7c17400640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c180b1660 0x7f7c181eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.886+0000 7f7c16a00640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c181600e0 0x7f7c180af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.887+0000 7f7c17e00640 1 -- 172.21.15.7:0/294016145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c181600e0 msgr2=0x7f7c180af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:29.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.887+0000 7f7c17e00640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c181600e0 0x7f7c180af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:29.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.887+0000 7f7c17e00640 1 -- 172.21.15.7:0/294016145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c180b1660 msgr2=0x7f7c181eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:29.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.887+0000 7f7c17e00640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c180b1660 0x7f7c181eb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:29.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.887+0000 7f7c17e00640 1 -- 172.21.15.7:0/294016145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c181ebf00 con 0x7f7c180b05a0 2024-04-25T11:13:29.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.887+0000 7f7c17e00640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c180b05a0 0x7f7c18157860 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000cba0 tx=0x7f7c1000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:29.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.887+0000 7f7c0f400640 1 -- 172.21.15.7:0/294016145 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10017070 con 0x7f7c180b05a0 2024-04-25T11:13:29.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.887+0000 7f7c0f400640 1 -- 172.21.15.7:0/294016145 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1000ad40 con 0x7f7c180b05a0 2024-04-25T11:13:29.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.887+0000 7f7c1d200640 1 -- 172.21.15.7:0/294016145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c181ec1e0 con 0x7f7c180b05a0 2024-04-25T11:13:29.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.887+0000 7f7c0f400640 1 -- 172.21.15.7:0/294016145 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10012640 con 0x7f7c180b05a0 2024-04-25T11:13:29.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.887+0000 7f7c1d200640 1 -- 172.21.15.7:0/294016145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c180af010 con 0x7f7c180b05a0 2024-04-25T11:13:29.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.889+0000 7f7c1d200640 1 -- 172.21.15.7:0/294016145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c18050740 con 0x7f7c180b05a0 2024-04-25T11:13:29.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.890+0000 7f7c0f400640 1 -- 172.21.15.7:0/294016145 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7c1001a070 con 0x7f7c180b05a0 2024-04-25T11:13:29.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.891+0000 7f7c0f400640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7bf80744b0 0x7f7bf80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:29.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.892+0000 7f7c0f400640 1 -- 172.21.15.7:0/294016145 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f7c10094ff0 con 0x7f7c180b05a0 2024-04-25T11:13:29.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.892+0000 7f7c17400640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7bf80744b0 0x7f7bf80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:29.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.892+0000 7f7c17400640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7bf80744b0 0x7f7bf80769c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7c080015b0 tx=0x7f7c08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:29.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:29.896+0000 7f7c0f400640 1 -- 172.21.15.7:0/294016145 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c1005f900 con 0x7f7c180b05a0 2024-04-25T11:13:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.101+0000 7f7c1d200640 1 -- 172.21.15.7:0/294016145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"} v 0) -- 0x7f7c180d2930 con 0x7f7c180b05a0 2024-04-25T11:13:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.109+0000 7f7c0f400640 1 -- 172.21.15.7:0/294016145 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_session_cap_acquisition_throttle"}]=0 v13) ==== 119+0+0 (secure 0 0 0) 0x7f7c1005f200 con 0x7f7c180b05a0 2024-04-25T11:13:30.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.113+0000 7f7c1d200640 1 -- 172.21.15.7:0/294016145 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7bf80744b0 msgr2=0x7f7bf80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.113+0000 7f7c1d200640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7bf80744b0 0x7f7bf80769c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7c080015b0 tx=0x7f7c08009040 comp rx=0 tx=0).stop 2024-04-25T11:13:30.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.113+0000 7f7c1d200640 1 -- 172.21.15.7:0/294016145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c180b05a0 msgr2=0x7f7c18157860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.113+0000 7f7c1d200640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c180b05a0 0x7f7c18157860 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000cba0 tx=0x7f7c1000a430 comp rx=0 tx=0).stop 2024-04-25T11:13:30.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.113+0000 7f7c1d200640 1 -- 172.21.15.7:0/294016145 shutdown_connections 2024-04-25T11:13:30.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.113+0000 7f7c1d200640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7bf80744b0 0x7f7bf80769c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.113+0000 7f7c1d200640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c180b05a0 0x7f7c18157860 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.113+0000 7f7c1d200640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c181600e0 0x7f7c180af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.113+0000 7f7c1d200640 1 --2- 172.21.15.7:0/294016145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c180b1660 0x7f7c181eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.113+0000 7f7c1d200640 1 -- 172.21.15.7:0/294016145 >> 172.21.15.7:0/294016145 conn(0x7f7c1809b540 msgr2=0x7f7c18070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.113+0000 7f7c1d200640 1 -- 172.21.15.7:0/294016145 shutdown_connections 2024-04-25T11:13:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.114+0000 7f7c1d200640 1 -- 172.21.15.7:0/294016145 wait complete. 2024-04-25T11:13:30.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2024-04-25T11:13:30.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.424+0000 7f6a48400640 1 Processor -- start 2024-04-25T11:13:30.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.424+0000 7f6a48400640 1 -- start start 2024-04-25T11:13:30.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.427+0000 7f6a48400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6a4015dfb0 0x7f6a400b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.427+0000 7f6a48400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a400d0b40 0x7f6a4006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.427+0000 7f6a48400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a400d0f50 0x7f6a4006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.427+0000 7f6a48400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6a400b09b0 con 0x7f6a400d0f50 2024-04-25T11:13:30.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.427+0000 7f6a48400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6a4015e3c0 con 0x7f6a4015dfb0 2024-04-25T11:13:30.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.427+0000 7f6a48400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6a400cf800 con 0x7f6a400d0b40 2024-04-25T11:13:30.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.427+0000 7f6a46400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a400d0f50 0x7f6a4006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.427+0000 7f6a46400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a400d0f50 0x7f6a4006f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44224/0 (socket says 172.21.15.7:44224) 2024-04-25T11:13:30.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.427+0000 7f6a46400640 1 -- 172.21.15.7:0/2518325017 learned_addr learned my addr 172.21.15.7:0/2518325017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:30.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.428+0000 7f6a45000640 1 --2- 172.21.15.7:0/2518325017 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a400d0b40 0x7f6a4006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.428+0000 7f6a45a00640 1 --2- 172.21.15.7:0/2518325017 >> v2:172.21.15.87:3300/0 conn(0x7f6a4015dfb0 0x7f6a400b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.428+0000 7f6a46400640 1 -- 172.21.15.7:0/2518325017 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a400d0b40 msgr2=0x7f6a4006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:30.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.428+0000 7f6a46400640 1 --2- 172.21.15.7:0/2518325017 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a400d0b40 0x7f6a4006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.428+0000 7f6a46400640 1 -- 172.21.15.7:0/2518325017 >> v2:172.21.15.87:3300/0 conn(0x7f6a4015dfb0 msgr2=0x7f6a400b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:30.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.428+0000 7f6a46400640 1 --2- 172.21.15.7:0/2518325017 >> v2:172.21.15.87:3300/0 conn(0x7f6a4015dfb0 0x7f6a400b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.428+0000 7f6a46400640 1 -- 172.21.15.7:0/2518325017 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a400cfaa0 con 0x7f6a400d0f50 2024-04-25T11:13:30.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.428+0000 7f6a45a00640 1 --2- 172.21.15.7:0/2518325017 >> v2:172.21.15.87:3300/0 conn(0x7f6a4015dfb0 0x7f6a400b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:30.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.429+0000 7f6a46400640 1 --2- 172.21.15.7:0/2518325017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a400d0f50 0x7f6a4006f8e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6a40072b10 tx=0x7f6a3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb763813aee245db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:30.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.429+0000 7f6a3fe00640 1 -- 172.21.15.7:0/2518325017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38037070 con 0x7f6a400d0f50 2024-04-25T11:13:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.429+0000 7f6a3fe00640 1 -- 172.21.15.7:0/2518325017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a38032440 con 0x7f6a400d0f50 2024-04-25T11:13:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.429+0000 7f6a3fe00640 1 -- 172.21.15.7:0/2518325017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38032810 con 0x7f6a400d0f50 2024-04-25T11:13:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.430+0000 7f6a48400640 1 -- 172.21.15.7:0/2518325017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a400d0f50 msgr2=0x7f6a4006f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.430+0000 7f6a48400640 1 --2- 172.21.15.7:0/2518325017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a400d0f50 0x7f6a4006f8e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6a40072b10 tx=0x7f6a3802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.430+0000 7f6a48400640 1 -- 172.21.15.7:0/2518325017 shutdown_connections 2024-04-25T11:13:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.430+0000 7f6a48400640 1 --2- 172.21.15.7:0/2518325017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a400d0f50 0x7f6a4006f8e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.430+0000 7f6a48400640 1 --2- 172.21.15.7:0/2518325017 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a400d0b40 0x7f6a4006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.430+0000 7f6a48400640 1 --2- 172.21.15.7:0/2518325017 >> v2:172.21.15.87:3300/0 conn(0x7f6a4015dfb0 0x7f6a400b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.430+0000 7f6a48400640 1 -- 172.21.15.7:0/2518325017 >> 172.21.15.7:0/2518325017 conn(0x7f6a400742c0 msgr2=0x7f6a4006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.430+0000 7f6a48400640 1 -- 172.21.15.7:0/2518325017 shutdown_connections 2024-04-25T11:13:30.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.430+0000 7f6a48400640 1 -- 172.21.15.7:0/2518325017 wait complete. 2024-04-25T11:13:30.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.433+0000 7f6a48400640 1 Processor -- start 2024-04-25T11:13:30.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.433+0000 7f6a48400640 1 -- start start 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.435+0000 7f6a48400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a400d0b40 0x7f6a4014bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.435+0000 7f6a48400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a400d0f50 0x7f6a40160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.435+0000 7f6a48400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a4015dfb0 0x7f6a4015e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.435+0000 7f6a48400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a400d0340 con 0x7f6a4015dfb0 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.435+0000 7f6a48400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6a400d0060 con 0x7f6a400d0f50 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.435+0000 7f6a48400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6a400d0620 con 0x7f6a400d0b40 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.435+0000 7f6a46400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a4015dfb0 0x7f6a4015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.435+0000 7f6a46400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a4015dfb0 0x7f6a4015e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44234/0 (socket says 172.21.15.7:44234) 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.435+0000 7f6a46400640 1 -- 172.21.15.7:0/3281393022 learned_addr learned my addr 172.21.15.7:0/3281393022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.435+0000 7f6a45a00640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a400d0b40 0x7f6a4014bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.435+0000 7f6a45000640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a400d0f50 0x7f6a40160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a46400640 1 -- 172.21.15.7:0/3281393022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a400d0b40 msgr2=0x7f6a4014bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a46400640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a400d0b40 0x7f6a4014bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a46400640 1 -- 172.21.15.7:0/3281393022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a400d0f50 msgr2=0x7f6a40160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a46400640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a400d0f50 0x7f6a40160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a46400640 1 -- 172.21.15.7:0/3281393022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4014c220 con 0x7f6a4015dfb0 2024-04-25T11:13:30.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a45000640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a400d0f50 0x7f6a40160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:30.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a46400640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a4015dfb0 0x7f6a4015e540 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6a380090b0 tx=0x7f6a380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:30.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a3d600640 1 -- 172.21.15.7:0/3281393022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38037040 con 0x7f6a4015dfb0 2024-04-25T11:13:30.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a3d600640 1 -- 172.21.15.7:0/3281393022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a38042070 con 0x7f6a4015dfb0 2024-04-25T11:13:30.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a3d600640 1 -- 172.21.15.7:0/3281393022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38030920 con 0x7f6a4015dfb0 2024-04-25T11:13:30.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a48400640 1 -- 172.21.15.7:0/3281393022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4015fbd0 con 0x7f6a4015dfb0 2024-04-25T11:13:30.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.436+0000 7f6a48400640 1 -- 172.21.15.7:0/3281393022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4015f330 con 0x7f6a4015dfb0 2024-04-25T11:13:30.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.438+0000 7f6a48400640 1 -- 172.21.15.7:0/3281393022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a40050740 con 0x7f6a4015dfb0 2024-04-25T11:13:30.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.439+0000 7f6a3d600640 1 -- 172.21.15.7:0/3281393022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6a38030ac0 con 0x7f6a4015dfb0 2024-04-25T11:13:30.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.440+0000 7f6a3d600640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a200744e0 0x7f6a200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.440+0000 7f6a3d600640 1 -- 172.21.15.7:0/3281393022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f6a380b5ad0 con 0x7f6a4015dfb0 2024-04-25T11:13:30.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.440+0000 7f6a45a00640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a200744e0 0x7f6a200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.441+0000 7f6a45a00640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a200744e0 0x7f6a200769f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6a340070b0 tx=0x7f6a34007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:30.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.445+0000 7f6a3d600640 1 -- 172.21.15.7:0/3281393022 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a4014c220 con 0x7f6a4015dfb0 2024-04-25T11:13:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.652+0000 7f6a48400640 1 -- 172.21.15.7:0/3281393022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) -- 0x7f6a40070fb0 con 0x7f6a4015dfb0 2024-04-25T11:13:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.660+0000 7f6a3d600640 1 -- 172.21.15.7:0/3281393022 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v14) ==== 106+0+0 (secure 0 0 0) 0x7f6a3807fce0 con 0x7f6a4015dfb0 2024-04-25T11:13:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.663+0000 7f6a48400640 1 -- 172.21.15.7:0/3281393022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a200744e0 msgr2=0x7f6a200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.663+0000 7f6a48400640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a200744e0 0x7f6a200769f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6a340070b0 tx=0x7f6a34007040 comp rx=0 tx=0).stop 2024-04-25T11:13:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.663+0000 7f6a48400640 1 -- 172.21.15.7:0/3281393022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a4015dfb0 msgr2=0x7f6a4015e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.663+0000 7f6a48400640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a4015dfb0 0x7f6a4015e540 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6a380090b0 tx=0x7f6a380046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.663+0000 7f6a48400640 1 -- 172.21.15.7:0/3281393022 shutdown_connections 2024-04-25T11:13:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.663+0000 7f6a48400640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a200744e0 0x7f6a200769f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.663+0000 7f6a48400640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a4015dfb0 0x7f6a4015e540 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.663+0000 7f6a48400640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a400d0f50 0x7f6a40160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.663+0000 7f6a48400640 1 --2- 172.21.15.7:0/3281393022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a400d0b40 0x7f6a4014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.663+0000 7f6a48400640 1 -- 172.21.15.7:0/3281393022 >> 172.21.15.7:0/3281393022 conn(0x7f6a400742c0 msgr2=0x7f6a401578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.664+0000 7f6a48400640 1 -- 172.21.15.7:0/3281393022 shutdown_connections 2024-04-25T11:13:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.664+0000 7f6a48400640 1 -- 172.21.15.7:0/3281393022 wait complete. 2024-04-25T11:13:30.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:13:30.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.980+0000 7f3c57a00640 1 Processor -- start 2024-04-25T11:13:30.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.980+0000 7f3c57a00640 1 -- start start 2024-04-25T11:13:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.983+0000 7f3c57a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3c500b0470 0x7f3c5006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.983+0000 7f3c57a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c5015e3c0 con 0x7f3c5006f8e0 2024-04-25T11:13:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.983+0000 7f3c57a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c500cf800 con 0x7f3c5015dfb0 2024-04-25T11:13:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.983+0000 7f3c57a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3c500d1250 con 0x7f3c500b0470 2024-04-25T11:13:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.983+0000 7f3c55a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3c5006f8e0 0x7f3c500aea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52390/0 (socket says 172.21.15.7:52390) 2024-04-25T11:13:30.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.983+0000 7f3c55a00640 1 -- 172.21.15.7:0/3887196027 learned_addr learned my addr 172.21.15.7:0/3887196027 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:30.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4fe00640 1 --2- 172.21.15.7:0/3887196027 >> v2:172.21.15.87:3300/0 conn(0x7f3c500b0470 0x7f3c5006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3514557322 0 0) 0x7f3c5015e3c0 con 0x7f3c5006f8e0 2024-04-25T11:13:30.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c34003520 con 0x7f3c5006f8e0 2024-04-25T11:13:30.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2308513245 0 0) 0x7f3c34003520 con 0x7f3c5006f8e0 2024-04-25T11:13:30.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 >> v2:172.21.15.87:3300/0 conn(0x7f3c500b0470 msgr2=0x7f3c5006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 --2- 172.21.15.7:0/3887196027 >> v2:172.21.15.87:3300/0 conn(0x7f3c500b0470 0x7f3c5006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 >> v1:172.21.15.87:6789/0 conn(0x7f3c5015dfb0 legacy=0x7f3c500d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c500cfaa0 con 0x7f3c5006f8e0 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f3c48001670 con 0x7f3c5006f8e0 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 >> v1:172.21.15.7:6789/0 conn(0x7f3c5006f8e0 legacy=0x7f3c500aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 --2- 172.21.15.7:0/3887196027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c34003a00 0x7f3c34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 --2- 172.21.15.7:0/3887196027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 0x7f3c34004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 --2- 172.21.15.7:0/3887196027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c340052a0 0x7f3c3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3c500d1250 con 0x7f3c34004800 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3400c230 con 0x7f3c34003a00 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.984+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3c3400c430 con 0x7f3c340052a0 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.985+0000 7f3c4fe00640 1 --2- 172.21.15.7:0/3887196027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 0x7f3c34004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.985+0000 7f3c55000640 1 --2- 172.21.15.7:0/3887196027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c34003a00 0x7f3c34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.985+0000 7f3c4fe00640 1 -- 172.21.15.7:0/3887196027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c340052a0 msgr2=0x7f3c3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:30.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.985+0000 7f3c55a00640 1 --2- 172.21.15.7:0/3887196027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c340052a0 0x7f3c3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.985+0000 7f3c4fe00640 1 --2- 172.21.15.7:0/3887196027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c340052a0 0x7f3c3400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.985+0000 7f3c4fe00640 1 -- 172.21.15.7:0/3887196027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c34003a00 msgr2=0x7f3c34003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.985+0000 7f3c4fe00640 1 --2- 172.21.15.7:0/3887196027 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c34003a00 0x7f3c34003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.985+0000 7f3c4fe00640 1 -- 172.21.15.7:0/3887196027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c34003520 con 0x7f3c34004800 2024-04-25T11:13:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.985+0000 7f3c55a00640 1 -- 172.21.15.7:0/3887196027 reap_dead start 2024-04-25T11:13:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.986+0000 7f3c4fe00640 1 --2- 172.21.15.7:0/3887196027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 0x7f3c34004c10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3c500b6c90 tx=0x7f3c34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73907d5ad7eb8c17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.987+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38037070 con 0x7f3c34004800 2024-04-25T11:13:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.987+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c3802e050 con 0x7f3c34004800 2024-04-25T11:13:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.987+0000 7f3c4f400640 1 -- 172.21.15.7:0/3887196027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38009c10 con 0x7f3c34004800 2024-04-25T11:13:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.987+0000 7f3c57a00640 1 -- 172.21.15.7:0/3887196027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 msgr2=0x7f3c34004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.987+0000 7f3c57a00640 1 --2- 172.21.15.7:0/3887196027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 0x7f3c34004c10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3c500b6c90 tx=0x7f3c34005830 comp rx=0 tx=0).stop 2024-04-25T11:13:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.988+0000 7f3c57a00640 1 -- 172.21.15.7:0/3887196027 shutdown_connections 2024-04-25T11:13:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.988+0000 7f3c57a00640 1 --2- 172.21.15.7:0/3887196027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 0x7f3c34004c10 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.988+0000 7f3c57a00640 1 -- 172.21.15.7:0/3887196027 >> 172.21.15.7:0/3887196027 conn(0x7f3c500742c0 msgr2=0x7f3c5006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.988+0000 7f3c57a00640 1 -- 172.21.15.7:0/3887196027 shutdown_connections 2024-04-25T11:13:30.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.988+0000 7f3c57a00640 1 -- 172.21.15.7:0/3887196027 wait complete. 2024-04-25T11:13:30.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.990+0000 7f3c57a00640 1 Processor -- start 2024-04-25T11:13:30.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.990+0000 7f3c57a00640 1 -- start start 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.992+0000 7f3c57a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 0x7f3c50157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.992+0000 7f3c57a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c5015b630 0x7f3c50154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.992+0000 7f3c57a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c5015ce00 0x7f3c50155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.992+0000 7f3c57a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3c500d0340 con 0x7f3c34004800 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.992+0000 7f3c57a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3c500cfde0 con 0x7f3c5015ce00 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.992+0000 7f3c57a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3c500d0620 con 0x7f3c5015b630 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.992+0000 7f3c55000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 0x7f3c50157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.992+0000 7f3c55000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 0x7f3c50157530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44264/0 (socket says 172.21.15.7:44264) 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.992+0000 7f3c55000640 1 -- 172.21.15.7:0/2945801037 learned_addr learned my addr 172.21.15.7:0/2945801037 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.993+0000 7f3c55a00640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c5015ce00 0x7f3c50155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.993+0000 7f3c4fe00640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c5015b630 0x7f3c50154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.993+0000 7f3c55000640 1 -- 172.21.15.7:0/2945801037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c5015b630 msgr2=0x7f3c50154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.993+0000 7f3c55000640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c5015b630 0x7f3c50154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.993+0000 7f3c55000640 1 -- 172.21.15.7:0/2945801037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c5015ce00 msgr2=0x7f3c50155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.993+0000 7f3c55000640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c5015ce00 0x7f3c50155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:30.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.993+0000 7f3c55000640 1 -- 172.21.15.7:0/2945801037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5015bd60 con 0x7f3c34004800 2024-04-25T11:13:30.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.993+0000 7f3c55000640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 0x7f3c50157530 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000e4e0 tx=0x7f3c4000e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:30.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.993+0000 7f3c4cc00640 1 -- 172.21.15.7:0/2945801037 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400053a0 con 0x7f3c34004800 2024-04-25T11:13:30.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.993+0000 7f3c4cc00640 1 -- 172.21.15.7:0/2945801037 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c40007020 con 0x7f3c34004800 2024-04-25T11:13:30.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.993+0000 7f3c4cc00640 1 -- 172.21.15.7:0/2945801037 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40007210 con 0x7f3c34004800 2024-04-25T11:13:30.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.994+0000 7f3c57a00640 1 -- 172.21.15.7:0/2945801037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5015c040 con 0x7f3c34004800 2024-04-25T11:13:30.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.994+0000 7f3c57a00640 1 -- 172.21.15.7:0/2945801037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5015c600 con 0x7f3c34004800 2024-04-25T11:13:30.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.995+0000 7f3c57a00640 1 -- 172.21.15.7:0/2945801037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c50050740 con 0x7f3c34004800 2024-04-25T11:13:30.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.996+0000 7f3c4cc00640 1 -- 172.21.15.7:0/2945801037 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3c4001a020 con 0x7f3c34004800 2024-04-25T11:13:30.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.997+0000 7f3c4cc00640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c2c0744b0 0x7f3c2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:30.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.998+0000 7f3c4cc00640 1 -- 172.21.15.7:0/2945801037 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f3c400947f0 con 0x7f3c34004800 2024-04-25T11:13:30.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.998+0000 7f3c4fe00640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c2c0744b0 0x7f3c2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:30.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:30.999+0000 7f3c4fe00640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c2c0744b0 0x7f3c2c0769c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3c38034000 tx=0x7f3c3802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:31.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.003+0000 7f3c4cc00640 1 -- 172.21.15.7:0/2945801037 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c4005f050 con 0x7f3c34004800 2024-04-25T11:13:31.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.208+0000 7f3c57a00640 1 -- 172.21.15.7:0/2945801037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f3c501547f0 con 0x7f3c34004800 2024-04-25T11:13:31.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.215+0000 7f3c4cc00640 1 -- 172.21.15.7:0/2945801037 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f3c4005e950 con 0x7f3c34004800 2024-04-25T11:13:31.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 -- 172.21.15.7:0/2945801037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c2c0744b0 msgr2=0x7f3c2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:31.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c2c0744b0 0x7f3c2c0769c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3c38034000 tx=0x7f3c3802f000 comp rx=0 tx=0).stop 2024-04-25T11:13:31.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 -- 172.21.15.7:0/2945801037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 msgr2=0x7f3c50157530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 0x7f3c50157530 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000e4e0 tx=0x7f3c4000e9a0 comp rx=0 tx=0).stop 2024-04-25T11:13:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 -- 172.21.15.7:0/2945801037 shutdown_connections 2024-04-25T11:13:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c2c0744b0 0x7f3c2c0769c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c5015ce00 0x7f3c50155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c5015b630 0x7f3c50154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 --2- 172.21.15.7:0/2945801037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c34004800 0x7f3c50157530 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 -- 172.21.15.7:0/2945801037 >> 172.21.15.7:0/2945801037 conn(0x7f3c500742c0 msgr2=0x7f3c5006fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 -- 172.21.15.7:0/2945801037 shutdown_connections 2024-04-25T11:13:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.218+0000 7f3c57a00640 1 -- 172.21.15.7:0/2945801037 wait complete. 2024-04-25T11:13:31.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir' 2024-04-25T11:13:31.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.524+0000 7fc8b6400640 1 Processor -- start 2024-04-25T11:13:31.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.524+0000 7fc8b6400640 1 -- start start 2024-04-25T11:13:31.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.527+0000 7fc8b6400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8b015e2f0 0x7fc8b0161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:31.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.527+0000 7fc8b6400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc8b0162420 0x7fc8b006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:31.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.527+0000 7fc8b6400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b015e700 con 0x7fc8b015baa0 2024-04-25T11:13:31.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.527+0000 7fc8b6400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b0162830 con 0x7fc8b0162420 2024-04-25T11:13:31.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.527+0000 7fc8b6400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b015e980 con 0x7fc8b015e2f0 2024-04-25T11:13:31.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.527+0000 7fc8b5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc8b0162420 0x7fc8b006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:31.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.527+0000 7fc8b5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc8b0162420 0x7fc8b006d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44268/0 (socket says 172.21.15.7:44268) 2024-04-25T11:13:31.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.527+0000 7fc8b5a00640 1 -- 172.21.15.7:0/1152072927 learned_addr learned my addr 172.21.15.7:0/1152072927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:31.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.528+0000 7fc8afe00640 1 --2- 172.21.15.7:0/1152072927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8b015e2f0 0x7fc8b0161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:31.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.528+0000 7fc8b5a00640 1 -- 172.21.15.7:0/1152072927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8b015e2f0 msgr2=0x7fc8b0161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:31.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.528+0000 7fc8b5a00640 1 --2- 172.21.15.7:0/1152072927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8b015e2f0 0x7fc8b0161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:31.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.528+0000 7fc8b5a00640 1 -- 172.21.15.7:0/1152072927 >> v1:172.21.15.87:6789/0 conn(0x7fc8b015baa0 legacy=0x7fc8b0070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:31.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.528+0000 7fc8b5a00640 1 -- 172.21.15.7:0/1152072927 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b015ec60 con 0x7fc8b0162420 2024-04-25T11:13:31.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.529+0000 7fc8b5a00640 1 --2- 172.21.15.7:0/1152072927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b0162420 0x7fc8b006d580 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009e50 tx=0x7fc8a802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce525857c7077f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:31.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.529+0000 7fc8af400640 1 -- 172.21.15.7:0/1152072927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8036070 con 0x7fc8b0162420 2024-04-25T11:13:31.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.530+0000 7fc8af400640 1 -- 172.21.15.7:0/1152072927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a8031600 con 0x7fc8b0162420 2024-04-25T11:13:31.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.530+0000 7fc8af400640 1 -- 172.21.15.7:0/1152072927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a80319d0 con 0x7fc8b0162420 2024-04-25T11:13:31.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.530+0000 7fc8b6400640 1 -- 172.21.15.7:0/1152072927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b0162420 msgr2=0x7fc8b006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.530+0000 7fc8b6400640 1 --2- 172.21.15.7:0/1152072927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b0162420 0x7fc8b006d580 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009e50 tx=0x7fc8a802a6c0 comp rx=0 tx=0).stop 2024-04-25T11:13:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.530+0000 7fc8b6400640 1 -- 172.21.15.7:0/1152072927 shutdown_connections 2024-04-25T11:13:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.530+0000 7fc8b6400640 1 --2- 172.21.15.7:0/1152072927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b0162420 0x7fc8b006d580 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.530+0000 7fc8b6400640 1 --2- 172.21.15.7:0/1152072927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8b015e2f0 0x7fc8b0161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.530+0000 7fc8b6400640 1 -- 172.21.15.7:0/1152072927 >> 172.21.15.7:0/1152072927 conn(0x7fc8b0073090 msgr2=0x7fc8b00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.531+0000 7fc8b6400640 1 -- 172.21.15.7:0/1152072927 shutdown_connections 2024-04-25T11:13:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.531+0000 7fc8b6400640 1 -- 172.21.15.7:0/1152072927 wait complete. 2024-04-25T11:13:31.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.533+0000 7fc8b6400640 1 Processor -- start 2024-04-25T11:13:31.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.533+0000 7fc8b6400640 1 -- start start 2024-04-25T11:13:31.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.535+0000 7fc8b6400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8b015e2f0 0x7fc8b01e7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:31.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.535+0000 7fc8b6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b0162420 0x7fc8b01e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:31.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.535+0000 7fc8b6400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8b01ea2f0 0x7fc8b01367f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:31.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.535+0000 7fc8b6400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b006dac0 con 0x7fc8b0162420 2024-04-25T11:13:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.535+0000 7fc8b6400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b01619c0 con 0x7fc8b015e2f0 2024-04-25T11:13:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.535+0000 7fc8b6400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b006dc40 con 0x7fc8b01ea2f0 2024-04-25T11:13:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.535+0000 7fc8afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b0162420 0x7fc8b01e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.535+0000 7fc8afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b0162420 0x7fc8b01e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44272/0 (socket says 172.21.15.7:44272) 2024-04-25T11:13:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.535+0000 7fc8afe00640 1 -- 172.21.15.7:0/1378591677 learned_addr learned my addr 172.21.15.7:0/1378591677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.535+0000 7fc8b5000640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8b015e2f0 0x7fc8b01e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.535+0000 7fc8b5a00640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8b01ea2f0 0x7fc8b01367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:31.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.536+0000 7fc8afe00640 1 -- 172.21.15.7:0/1378591677 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8b01ea2f0 msgr2=0x7fc8b01367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:31.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.536+0000 7fc8afe00640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8b01ea2f0 0x7fc8b01367f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:31.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.536+0000 7fc8afe00640 1 -- 172.21.15.7:0/1378591677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8b015e2f0 msgr2=0x7fc8b01e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:31.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.536+0000 7fc8afe00640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8b015e2f0 0x7fc8b01e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.536+0000 7fc8afe00640 1 -- 172.21.15.7:0/1378591677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b0134230 con 0x7fc8b0162420 2024-04-25T11:13:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.536+0000 7fc8afe00640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b0162420 0x7fc8b01e9db0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00d5f0 tx=0x7fc89c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.536+0000 7fc8acc00640 1 -- 172.21.15.7:0/1378591677 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0125f0 con 0x7fc8b0162420 2024-04-25T11:13:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.536+0000 7fc8acc00640 1 -- 172.21.15.7:0/1378591677 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc89c019070 con 0x7fc8b0162420 2024-04-25T11:13:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.536+0000 7fc8b6400640 1 -- 172.21.15.7:0/1378591677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b0134510 con 0x7fc8b0162420 2024-04-25T11:13:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.536+0000 7fc8acc00640 1 -- 172.21.15.7:0/1378591677 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0129d0 con 0x7fc8b0162420 2024-04-25T11:13:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.537+0000 7fc8b6400640 1 -- 172.21.15.7:0/1378591677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b0134ad0 con 0x7fc8b0162420 2024-04-25T11:13:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.538+0000 7fc8b6400640 1 -- 172.21.15.7:0/1378591677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b0074020 con 0x7fc8b0162420 2024-04-25T11:13:31.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.539+0000 7fc8acc00640 1 -- 172.21.15.7:0/1378591677 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc89c012b70 con 0x7fc8b0162420 2024-04-25T11:13:31.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.540+0000 7fc8acc00640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc890074530 0x7fc890076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:31.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.540+0000 7fc8acc00640 1 -- 172.21.15.7:0/1378591677 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fc89c095470 con 0x7fc8b0162420 2024-04-25T11:13:31.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.540+0000 7fc8b5000640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc890074530 0x7fc890076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:31.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.541+0000 7fc8b5000640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc890074530 0x7fc890076a40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4000f60 tx=0x7fc8a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:31.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.545+0000 7fc8acc00640 1 -- 172.21.15.7:0/1378591677 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc89c05fd50 con 0x7fc8b0162420 2024-04-25T11:13:31.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:31.824+0000 7fc8b6400640 1 -- 172.21.15.7:0/1378591677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]} v 0) -- 0x7fc8b0161f80 con 0x7fc8b0162420 2024-04-25T11:13:32.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.606+0000 7fc8acc00640 1 -- 172.21.15.7:0/1378591677 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_cap_acquisition_throttle_readdir"]}]=0 v179) ==== 165+0+0 (secure 0 0 0) 0x7fc89c05f650 con 0x7fc8b0162420 2024-04-25T11:13:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.609+0000 7fc8b6400640 1 -- 172.21.15.7:0/1378591677 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc890074530 msgr2=0x7fc890076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.609+0000 7fc8b6400640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc890074530 0x7fc890076a40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4000f60 tx=0x7fc8a4009040 comp rx=0 tx=0).stop 2024-04-25T11:13:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.609+0000 7fc8b6400640 1 -- 172.21.15.7:0/1378591677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b0162420 msgr2=0x7fc8b01e9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.609+0000 7fc8b6400640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b0162420 0x7fc8b01e9db0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00d5f0 tx=0x7fc89c00dab0 comp rx=0 tx=0).stop 2024-04-25T11:13:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.610+0000 7fc8b6400640 1 -- 172.21.15.7:0/1378591677 shutdown_connections 2024-04-25T11:13:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.610+0000 7fc8b6400640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc890074530 0x7fc890076a40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:32.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.610+0000 7fc8b6400640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8b01ea2f0 0x7fc8b01367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.610+0000 7fc8b6400640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8b0162420 0x7fc8b01e9db0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.610+0000 7fc8b6400640 1 --2- 172.21.15.7:0/1378591677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8b015e2f0 0x7fc8b01e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.610+0000 7fc8b6400640 1 -- 172.21.15.7:0/1378591677 >> 172.21.15.7:0/1378591677 conn(0x7fc8b0073090 msgr2=0x7fc8b006cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.610+0000 7fc8b6400640 1 -- 172.21.15.7:0/1378591677 shutdown_connections 2024-04-25T11:13:32.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.610+0000 7fc8b6400640 1 -- 172.21.15.7:0/1378591677 wait complete. 2024-04-25T11:13:32.645 INFO:tasks.cephfs_test_runner:Mostly readdir acquires caps faster than the mds recalls, so the cap ... ok 2024-04-25T11:13:32.645 INFO:tasks.cephfs_test_runner:Starting test: test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) that a client is blocklisted when its encoded session metadata exceeds the 2024-04-25T11:13:32.646 INFO:tasks.cephfs_test_runner:test_client_blocklisted_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-25T11:13:32.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:32.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.931+0000 7fc90f000640 1 Processor -- start 2024-04-25T11:13:32.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.931+0000 7fc90f000640 1 -- start start 2024-04-25T11:13:32.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.933+0000 7fc90f000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9080af540 0x7fc90809f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:32.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.933+0000 7fc90f000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc90815bc40 0x7fc9080cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.933+0000 7fc90f000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90815f4e0 con 0x7fc90815ef90 2024-04-25T11:13:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.933+0000 7fc90f000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc908071190 con 0x7fc90815bc40 2024-04-25T11:13:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.933+0000 7fc90f000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc9080b0560 con 0x7fc9080af540 2024-04-25T11:13:32.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.934+0000 7fc90d000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc90815bc40 0x7fc9080cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.934+0000 7fc907e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fc90815ef90 0x7fc9080aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:43348/0 (socket says 172.21.15.7:43348) 2024-04-25T11:13:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.934+0000 7fc907e00640 1 -- 172.21.15.7:0/3956948760 learned_addr learned my addr 172.21.15.7:0/3956948760 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.934+0000 7fc8ffe00640 1 --2- 172.21.15.7:0/3956948760 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9080af540 0x7fc90809f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:32.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.935+0000 7fc90d000640 1 -- 172.21.15.7:0/3956948760 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9080af540 msgr2=0x7fc90809f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.935+0000 7fc90d000640 1 --2- 172.21.15.7:0/3956948760 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9080af540 0x7fc90809f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.935+0000 7fc90d000640 1 -- 172.21.15.7:0/3956948760 >> v1:172.21.15.87:6789/0 conn(0x7fc90815ef90 legacy=0x7fc9080aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.935+0000 7fc90d000640 1 -- 172.21.15.7:0/3956948760 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9080b0840 con 0x7fc90815bc40 2024-04-25T11:13:32.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.935+0000 7fc907400640 1 -- 172.21.15.7:0/3956948760 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1800009422 0 0) 0x7fc90815f4e0 con 0x7fc90815ef90 2024-04-25T11:13:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.937+0000 7fc90d000640 1 --2- 172.21.15.7:0/3956948760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc90815bc40 0x7fc9080cf980 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc900009d90 tx=0x7fc90002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d50363b24f029416 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:32.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.937+0000 7fc907400640 1 -- 172.21.15.7:0/3956948760 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900031600 con 0x7fc90815bc40 2024-04-25T11:13:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.937+0000 7fc907400640 1 -- 172.21.15.7:0/3956948760 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9000317a0 con 0x7fc90815bc40 2024-04-25T11:13:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.937+0000 7fc907400640 1 -- 172.21.15.7:0/3956948760 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900031b70 con 0x7fc90815bc40 2024-04-25T11:13:32.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.938+0000 7fc90f000640 1 -- 172.21.15.7:0/3956948760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc90815bc40 msgr2=0x7fc9080cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:32.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.938+0000 7fc90f000640 1 --2- 172.21.15.7:0/3956948760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc90815bc40 0x7fc9080cf980 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc900009d90 tx=0x7fc90002a5d0 comp rx=0 tx=0).stop 2024-04-25T11:13:32.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.938+0000 7fc90f000640 1 -- 172.21.15.7:0/3956948760 shutdown_connections 2024-04-25T11:13:32.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.938+0000 7fc90f000640 1 --2- 172.21.15.7:0/3956948760 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc90815bc40 0x7fc9080cf980 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:32.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.938+0000 7fc90f000640 1 --2- 172.21.15.7:0/3956948760 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9080af540 0x7fc90809f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:32.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.938+0000 7fc90f000640 1 -- 172.21.15.7:0/3956948760 >> 172.21.15.7:0/3956948760 conn(0x7fc908074200 msgr2=0x7fc90806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:32.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.939+0000 7fc90f000640 1 -- 172.21.15.7:0/3956948760 shutdown_connections 2024-04-25T11:13:32.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.939+0000 7fc90f000640 1 -- 172.21.15.7:0/3956948760 wait complete. 2024-04-25T11:13:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.941+0000 7fc90f000640 1 Processor -- start 2024-04-25T11:13:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.941+0000 7fc90f000640 1 -- start start 2024-04-25T11:13:32.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.943+0000 7fc90f000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc90815bc40 0x7fc9081e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:32.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.943+0000 7fc90f000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc90815ef90 0x7fc908147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:32.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.943+0000 7fc90f000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc90815da00 0x7fc908147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:32.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.943+0000 7fc90f000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9080b1320 con 0x7fc90815da00 2024-04-25T11:13:32.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.943+0000 7fc90f000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc9080b0e00 con 0x7fc90815ef90 2024-04-25T11:13:32.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.943+0000 7fc90f000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc9080afb80 con 0x7fc90815bc40 2024-04-25T11:13:32.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.943+0000 7fc90d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc90815da00 0x7fc908147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:32.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.943+0000 7fc90d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc90815da00 0x7fc908147b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44288/0 (socket says 172.21.15.7:44288) 2024-04-25T11:13:32.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.943+0000 7fc90d000640 1 -- 172.21.15.7:0/1378197759 learned_addr learned my addr 172.21.15.7:0/1378197759 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:32.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.943+0000 7fc907e00640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc90815bc40 0x7fc9081e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:32.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.944+0000 7fc8ffe00640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc90815ef90 0x7fc908147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:32.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.944+0000 7fc90d000640 1 -- 172.21.15.7:0/1378197759 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc90815bc40 msgr2=0x7fc9081e8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:32.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.944+0000 7fc90d000640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc90815bc40 0x7fc9081e8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:32.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.944+0000 7fc90d000640 1 -- 172.21.15.7:0/1378197759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc90815ef90 msgr2=0x7fc908147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:32.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.944+0000 7fc90d000640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc90815ef90 0x7fc908147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:32.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.944+0000 7fc90d000640 1 -- 172.21.15.7:0/1378197759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9081e7940 con 0x7fc90815da00 2024-04-25T11:13:32.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.944+0000 7fc90d000640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc90815da00 0x7fc908147b30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc900009eb0 tx=0x7fc900004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:32.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.944+0000 7fc904c00640 1 -- 172.21.15.7:0/1378197759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90003f070 con 0x7fc90815da00 2024-04-25T11:13:32.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.944+0000 7fc904c00640 1 -- 172.21.15.7:0/1378197759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc900030b60 con 0x7fc90815da00 2024-04-25T11:13:32.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.944+0000 7fc904c00640 1 -- 172.21.15.7:0/1378197759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900009450 con 0x7fc90815da00 2024-04-25T11:13:32.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.944+0000 7fc90f000640 1 -- 172.21.15.7:0/1378197759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9081e7db0 con 0x7fc90815da00 2024-04-25T11:13:32.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.945+0000 7fc90f000640 1 -- 172.21.15.7:0/1378197759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9081e8370 con 0x7fc90815da00 2024-04-25T11:13:32.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.946+0000 7fc90f000640 1 -- 172.21.15.7:0/1378197759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90806a100 con 0x7fc90815da00 2024-04-25T11:13:32.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.947+0000 7fc904c00640 1 -- 172.21.15.7:0/1378197759 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc90003b410 con 0x7fc90815da00 2024-04-25T11:13:32.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.948+0000 7fc904c00640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8e40744b0 0x7fc8e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:32.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.948+0000 7fc904c00640 1 -- 172.21.15.7:0/1378197759 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fc9000b6750 con 0x7fc90815da00 2024-04-25T11:13:32.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.949+0000 7fc907e00640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8e40744b0 0x7fc8e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:32.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.949+0000 7fc907e00640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8e40744b0 0x7fc8e40769c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8000f60 tx=0x7fc8f8007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:32.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:32.954+0000 7fc904c00640 1 -- 172.21.15.7:0/1378197759 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9000809f0 con 0x7fc90815da00 2024-04-25T11:13:33.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.158+0000 7fc90f000640 1 -- 172.21.15.7:0/1378197759 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9081e7b10 con 0x7fc90815da00 2024-04-25T11:13:33.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.161+0000 7fc904c00640 1 -- 172.21.15.7:0/1378197759 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+13195 (secure 0 0 0) 0x7fc900084cf0 con 0x7fc90815da00 2024-04-25T11:13:33.161 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:33.161 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":45,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:29.559629+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6833/389115631":"2024-04-26T11:13:26.540354+0000","172.21.15.87:6832/389115631":"2024-04-26T11:13:26.540354+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 -- 172.21.15.7:0/1378197759 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8e40744b0 msgr2=0x7fc8e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8e40744b0 0x7fc8e40769c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8000f60 tx=0x7fc8f8007c00 comp rx=0 tx=0).stop 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 -- 172.21.15.7:0/1378197759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc90815da00 msgr2=0x7fc908147b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc90815da00 0x7fc908147b30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc900009eb0 tx=0x7fc900004a40 comp rx=0 tx=0).stop 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 -- 172.21.15.7:0/1378197759 shutdown_connections 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc8e40744b0 0x7fc8e40769c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc90815da00 0x7fc908147b30 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc90815ef90 0x7fc908147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 --2- 172.21.15.7:0/1378197759 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc90815bc40 0x7fc9081e8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 -- 172.21.15.7:0/1378197759 >> 172.21.15.7:0/1378197759 conn(0x7fc908074200 msgr2=0x7fc9080b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 -- 172.21.15.7:0/1378197759 shutdown_connections 2024-04-25T11:13:33.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.163+0000 7fc90f000640 1 -- 172.21.15.7:0/1378197759 wait complete. 2024-04-25T11:13:33.192 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:13:33.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:13:33.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.475+0000 7f38a1c00640 1 Processor -- start 2024-04-25T11:13:33.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.475+0000 7f38a1c00640 1 -- start start 2024-04-25T11:13:33.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.478+0000 7f38a1c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c144a80 0x7f389c146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.478+0000 7f38a1c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389c0b66b0 con 0x7f389c145e40 2024-04-25T11:13:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.478+0000 7f38a1c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389c0b5b30 con 0x7f389c0b4f80 2024-04-25T11:13:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.478+0000 7f38a1c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f389c145b70 con 0x7f389c144a80 2024-04-25T11:13:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.478+0000 7f389be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f389c145e40 0x7f389c0b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52400/0 (socket says 172.21.15.7:52400) 2024-04-25T11:13:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.478+0000 7f389be00640 1 -- 172.21.15.7:0/977184018 learned_addr learned my addr 172.21.15.7:0/977184018 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:33.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f3893400640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c144a80 0x7f389c146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:33.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2806398384 0 0) 0x7f389c0b66b0 con 0x7f389c145e40 2024-04-25T11:13:33.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3880003520 con 0x7f389c145e40 2024-04-25T11:13:33.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1721416282 0 0) 0x7f3880003520 con 0x7f389c145e40 2024-04-25T11:13:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c144a80 msgr2=0x7f389c146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c144a80 0x7f389c146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 >> v1:172.21.15.87:6789/0 conn(0x7f389c0b4f80 legacy=0x7f389c1606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c0b5e10 con 0x7f389c145e40 2024-04-25T11:13:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f388c001670 con 0x7f389c145e40 2024-04-25T11:13:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 >> v1:172.21.15.7:6789/0 conn(0x7f389c145e40 legacy=0x7f389c0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38800039e0 0x7f3880003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 0x7f3880004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:33.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c144a80 0x7f38800052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:33.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f388000dd90 con 0x7f3880004800 2024-04-25T11:13:33.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f388000df40 con 0x7f38800039e0 2024-04-25T11:13:33.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.479+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f388000e140 con 0x7f389c144a80 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.480+0000 7f3893400640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 0x7f3880004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.480+0000 7f38a1200640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c144a80 0x7f38800052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.480+0000 7f389be00640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38800039e0 0x7f3880003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.480+0000 7f3893400640 1 -- 172.21.15.7:0/977184018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c144a80 msgr2=0x7f38800052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.480+0000 7f3893400640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c144a80 0x7f38800052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.480+0000 7f3893400640 1 -- 172.21.15.7:0/977184018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38800039e0 msgr2=0x7f3880003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.480+0000 7f3893400640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38800039e0 0x7f3880003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.480+0000 7f3893400640 1 -- 172.21.15.7:0/977184018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3880003520 con 0x7f3880004800 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.481+0000 7f3893400640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 0x7f3880004c10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3884009fc0 tx=0x7f3880005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539108105f0e2296 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.482+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884034020 con 0x7f3880004800 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.482+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38840305f0 con 0x7f3880004800 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.482+0000 7f389b400640 1 -- 172.21.15.7:0/977184018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884009c10 con 0x7f3880004800 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.482+0000 7f38a1c00640 1 -- 172.21.15.7:0/977184018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 msgr2=0x7f3880004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.482+0000 7f38a1c00640 1 --2- 172.21.15.7:0/977184018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 0x7f3880004c10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3884009fc0 tx=0x7f3880005830 comp rx=0 tx=0).stop 2024-04-25T11:13:33.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.482+0000 7f38a1200640 1 -- 172.21.15.7:0/977184018 reap_dead start 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.482+0000 7f38a1c00640 1 -- 172.21.15.7:0/977184018 shutdown_connections 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.482+0000 7f38a1c00640 1 -- 172.21.15.7:0/977184018 >> 172.21.15.7:0/977184018 conn(0x7f389c0740f0 msgr2=0x7f389c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.483+0000 7f38a1c00640 1 -- 172.21.15.7:0/977184018 shutdown_connections 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.483+0000 7f38a1c00640 1 -- 172.21.15.7:0/977184018 wait complete. 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.485+0000 7f38a1c00640 1 Processor -- start 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.485+0000 7f38a1c00640 1 -- start start 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.486+0000 7f38a1c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 0x7f389c1596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f38a1c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c159e70 0x7f389c13ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f38a1c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f389c13f110 0x7f389c13a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f38a1c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f389c161e10 con 0x7f3880004800 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f38a1c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f389c0b63d0 con 0x7f389c13f110 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f38a1c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f389c161f90 con 0x7f389c159e70 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f389be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 0x7f389c1596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f389be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 0x7f389c1596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44298/0 (socket says 172.21.15.7:44298) 2024-04-25T11:13:33.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f389be00640 1 -- 172.21.15.7:0/2547458152 learned_addr learned my addr 172.21.15.7:0/2547458152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f3893400640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c159e70 0x7f389c13ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f38a1200640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f389c13f110 0x7f389c13a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f389be00640 1 -- 172.21.15.7:0/2547458152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c159e70 msgr2=0x7f389c13ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f389be00640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c159e70 0x7f389c13ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f389be00640 1 -- 172.21.15.7:0/2547458152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f389c13f110 msgr2=0x7f389c13a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f389be00640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f389c13f110 0x7f389c13a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f389be00640 1 -- 172.21.15.7:0/2547458152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c13f520 con 0x7f3880004800 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f38a1200640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f389c13f110 0x7f389c13a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.487+0000 7f3893400640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c159e70 0x7f389c13ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.488+0000 7f389be00640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 0x7f389c1596c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f388c00e820 tx=0x7f388c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.488+0000 7f3898c00640 1 -- 172.21.15.7:0/2547458152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c00c8d0 con 0x7f3880004800 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.488+0000 7f3898c00640 1 -- 172.21.15.7:0/2547458152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c018070 con 0x7f3880004800 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.488+0000 7f3898c00640 1 -- 172.21.15.7:0/2547458152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c00ccb0 con 0x7f3880004800 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.488+0000 7f38a1c00640 1 -- 172.21.15.7:0/2547458152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c15a930 con 0x7f3880004800 2024-04-25T11:13:33.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.488+0000 7f38a1c00640 1 -- 172.21.15.7:0/2547458152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c15aef0 con 0x7f3880004800 2024-04-25T11:13:33.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.489+0000 7f38a1c00640 1 -- 172.21.15.7:0/2547458152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389c050740 con 0x7f3880004800 2024-04-25T11:13:33.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.491+0000 7f3898c00640 1 -- 172.21.15.7:0/2547458152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f388c01d460 con 0x7f3880004800 2024-04-25T11:13:33.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.492+0000 7f3898c00640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38780744b0 0x7f38780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:33.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.492+0000 7f3898c00640 1 -- 172.21.15.7:0/2547458152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f388c094fd0 con 0x7f3880004800 2024-04-25T11:13:33.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.492+0000 7f3893400640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38780744b0 0x7f38780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:33.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.493+0000 7f3893400640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38780744b0 0x7f38780769c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3884035000 tx=0x7f388402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:33.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.496+0000 7f3898c00640 1 -- 172.21.15.7:0/2547458152 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f389c13f520 con 0x7f3880004800 2024-04-25T11:13:33.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.693+0000 7f38a1c00640 1 -- 172.21.15.7:0/2547458152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f389c161bf0 con 0x7f3880004800 2024-04-25T11:13:33.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.694+0000 7f3898c00640 1 -- 172.21.15.7:0/2547458152 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f388c05f8d0 con 0x7f3880004800 2024-04-25T11:13:33.698 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:13:33.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 -- 172.21.15.7:0/2547458152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38780744b0 msgr2=0x7f38780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:33.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38780744b0 0x7f38780769c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3884035000 tx=0x7f388402f000 comp rx=0 tx=0).stop 2024-04-25T11:13:33.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 -- 172.21.15.7:0/2547458152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 msgr2=0x7f389c1596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:33.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 0x7f389c1596c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f388c00e820 tx=0x7f388c00ece0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 -- 172.21.15.7:0/2547458152 shutdown_connections 2024-04-25T11:13:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f38780744b0 0x7f38780769c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f389c13f110 0x7f389c13a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f389c159e70 0x7f389c13ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 --2- 172.21.15.7:0/2547458152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3880004800 0x7f389c1596c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 -- 172.21.15.7:0/2547458152 >> 172.21.15.7:0/2547458152 conn(0x7f389c0740f0 msgr2=0x7f389c15c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 -- 172.21.15.7:0/2547458152 shutdown_connections 2024-04-25T11:13:33.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.697+0000 7f38a1c00640 1 -- 172.21.15.7:0/2547458152 wait complete. 2024-04-25T11:13:33.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2024-04-25T11:13:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.997+0000 7f9b82000640 1 Processor -- start 2024-04-25T11:13:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.997+0000 7f9b82000640 1 -- start start 2024-04-25T11:13:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.998+0000 7f9b82000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c15a7a0 0x7f9b7c161160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.998+0000 7f9b82000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9b7c0b7b50 0x7f9b7c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.998+0000 7f9b82000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b7c06cfc0 con 0x7f9b7c06c570 2024-04-25T11:13:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.998+0000 7f9b82000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9b7c160cd0 con 0x7f9b7c0b7b50 2024-04-25T11:13:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.999+0000 7f9b82000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7c06d2a0 con 0x7f9b7c15a7a0 2024-04-25T11:13:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.999+0000 7f9b7be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9b7c06c570 0x7f9b7c0b7440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52416/0 (socket says 172.21.15.7:52416) 2024-04-25T11:13:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.999+0000 7f9b7be00640 1 -- 172.21.15.7:0/4133348682 learned_addr learned my addr 172.21.15.7:0/4133348682 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.999+0000 7f9b80c00640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c15a7a0 0x7f9b7c161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.999+0000 7f9b81600640 1 --2- 172.21.15.7:0/4133348682 >> v2:172.21.15.87:3300/0 conn(0x7f9b7c0b7b50 0x7f9b7c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:33.999+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2158432549 0 0) 0x7f9b7c06cfc0 con 0x7f9b7c06c570 2024-04-25T11:13:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b64003520 con 0x7f9b7c06c570 2024-04-25T11:13:34.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 870437935 0 0) 0x7f9b64003520 con 0x7f9b7c06c570 2024-04-25T11:13:34.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c15a7a0 msgr2=0x7f9b7c161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:34.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c15a7a0 0x7f9b7c161160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 >> v2:172.21.15.87:3300/0 conn(0x7f9b7c0b7b50 msgr2=0x7f9b7c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:34.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 --2- 172.21.15.7:0/4133348682 >> v2:172.21.15.87:3300/0 conn(0x7f9b7c0b7b50 0x7f9b7c06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c160960 con 0x7f9b7c06c570 2024-04-25T11:13:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f9b68001690 con 0x7f9b7c06c570 2024-04-25T11:13:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 >> v1:172.21.15.7:6789/0 conn(0x7f9b7c06c570 legacy=0x7f9b7c0b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b640039b0 0x7f9b64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c15a7a0 0x7f9b64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b64004d40 0x7f9b64005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c160cd0 con 0x7f9b64004d40 2024-04-25T11:13:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c06d2a0 con 0x7f9b640039b0 2024-04-25T11:13:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6400ba40 con 0x7f9b7c15a7a0 2024-04-25T11:13:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.000+0000 7f9b81600640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b64004d40 0x7f9b64005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.001+0000 7f9b80c00640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b640039b0 0x7f9b64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.001+0000 7f9b7be00640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c15a7a0 0x7f9b64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.001+0000 7f9b81600640 1 -- 172.21.15.7:0/4133348682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c15a7a0 msgr2=0x7f9b64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.001+0000 7f9b81600640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c15a7a0 0x7f9b64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.001+0000 7f9b81600640 1 -- 172.21.15.7:0/4133348682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b640039b0 msgr2=0x7f9b64003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.001+0000 7f9b81600640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b640039b0 0x7f9b64003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.001+0000 7f9b81600640 1 -- 172.21.15.7:0/4133348682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b64003520 con 0x7f9b64004d40 2024-04-25T11:13:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.002+0000 7f9b81600640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b64004d40 0x7f9b64005150 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009f90 tx=0x7f9b64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1de2f66fc7e1c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.002+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74009c10 con 0x7f9b64004d40 2024-04-25T11:13:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.002+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b74036070 con 0x7f9b64004d40 2024-04-25T11:13:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.002+0000 7f9b7b400640 1 -- 172.21.15.7:0/4133348682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74031840 con 0x7f9b64004d40 2024-04-25T11:13:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.003+0000 7f9b82000640 1 -- 172.21.15.7:0/4133348682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b64004d40 msgr2=0x7f9b64005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.003+0000 7f9b82000640 1 --2- 172.21.15.7:0/4133348682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b64004d40 0x7f9b64005150 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009f90 tx=0x7f9b64005830 comp rx=0 tx=0).stop 2024-04-25T11:13:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.003+0000 7f9b81600640 1 -- 172.21.15.7:0/4133348682 reap_dead start 2024-04-25T11:13:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.003+0000 7f9b82000640 1 -- 172.21.15.7:0/4133348682 shutdown_connections 2024-04-25T11:13:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.003+0000 7f9b82000640 1 -- 172.21.15.7:0/4133348682 >> 172.21.15.7:0/4133348682 conn(0x7f9b7c0d59c0 msgr2=0x7f9b7c0d5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.003+0000 7f9b82000640 1 -- 172.21.15.7:0/4133348682 shutdown_connections 2024-04-25T11:13:34.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.003+0000 7f9b82000640 1 -- 172.21.15.7:0/4133348682 wait complete. 2024-04-25T11:13:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.005+0000 7f9b82000640 1 Processor -- start 2024-04-25T11:13:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.006+0000 7f9b82000640 1 -- start start 2024-04-25T11:13:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.007+0000 7f9b82000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b64004d40 0x7f9b7c0cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.007+0000 7f9b82000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b7c0cd0a0 0x7f9b7c163a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.007+0000 7f9b82000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c163f60 0x7f9b7c158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.007+0000 7f9b82000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c0d09f0 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.007+0000 7f9b82000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c0d0870 con 0x7f9b64004d40 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.007+0000 7f9b82000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7c161160 con 0x7f9b7c163f60 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.007+0000 7f9b7be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b7c0cd0a0 0x7f9b7c163a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.007+0000 7f9b7be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b7c0cd0a0 0x7f9b7c163a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44320/0 (socket says 172.21.15.7:44320) 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.007+0000 7f9b7be00640 1 -- 172.21.15.7:0/1255428932 learned_addr learned my addr 172.21.15.7:0/1255428932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.008+0000 7f9b80c00640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b64004d40 0x7f9b7c0cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.008+0000 7f9b81600640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c163f60 0x7f9b7c158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.008+0000 7f9b7be00640 1 -- 172.21.15.7:0/1255428932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c163f60 msgr2=0x7f9b7c158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.008+0000 7f9b7be00640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c163f60 0x7f9b7c158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.008+0000 7f9b7be00640 1 -- 172.21.15.7:0/1255428932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b64004d40 msgr2=0x7f9b7c0cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.008+0000 7f9b7be00640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b64004d40 0x7f9b7c0cf4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.008+0000 7f9b7be00640 1 -- 172.21.15.7:0/1255428932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c164370 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.008+0000 7f9b81600640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c163f60 0x7f9b7c158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.008+0000 7f9b7be00640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b7c0cd0a0 0x7f9b7c163a20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9b68001b80 tx=0x7f9b6800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.008+0000 7f9b78c00640 1 -- 172.21.15.7:0/1255428932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800c8d0 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.008+0000 7f9b78c00640 1 -- 172.21.15.7:0/1255428932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b68018070 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.009+0000 7f9b78c00640 1 -- 172.21.15.7:0/1255428932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800ccb0 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.009+0000 7f9b82000640 1 -- 172.21.15.7:0/1255428932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c1f6b90 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.009+0000 7f9b82000640 1 -- 172.21.15.7:0/1255428932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c1f7150 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.010+0000 7f9b82000640 1 -- 172.21.15.7:0/1255428932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7c162eb0 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.011+0000 7f9b78c00640 1 -- 172.21.15.7:0/1255428932 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9b6801d460 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.012+0000 7f9b78c00640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9b5c0744b0 0x7f9b5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.012+0000 7f9b78c00640 1 -- 172.21.15.7:0/1255428932 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f9b68094fe0 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.012+0000 7f9b80c00640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9b5c0744b0 0x7f9b5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.013+0000 7f9b80c00640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9b5c0744b0 0x7f9b5c0769c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0c9390 tx=0x7f9b70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.017+0000 7f9b78c00640 1 -- 172.21.15.7:0/1255428932 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9b7c164370 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.299+0000 7f9b82000640 1 -- 172.21.15.7:0/1255428932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) -- 0x7f9b7c1f6920 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.612+0000 7f9b78c00640 1 -- 172.21.15.7:0/1255428932 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v181) ==== 165+0+0 (secure 0 0 0) 0x7f9b6805f9f0 con 0x7f9b7c0cd0a0 2024-04-25T11:13:34.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.616+0000 7f9b82000640 1 -- 172.21.15.7:0/1255428932 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9b5c0744b0 msgr2=0x7f9b5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:34.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.616+0000 7f9b82000640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9b5c0744b0 0x7f9b5c0769c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0c9390 tx=0x7f9b70008040 comp rx=0 tx=0).stop 2024-04-25T11:13:34.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.616+0000 7f9b82000640 1 -- 172.21.15.7:0/1255428932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b7c0cd0a0 msgr2=0x7f9b7c163a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:34.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.616+0000 7f9b82000640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b7c0cd0a0 0x7f9b7c163a20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9b68001b80 tx=0x7f9b6800ed50 comp rx=0 tx=0).stop 2024-04-25T11:13:34.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.616+0000 7f9b82000640 1 -- 172.21.15.7:0/1255428932 shutdown_connections 2024-04-25T11:13:34.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.616+0000 7f9b82000640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9b5c0744b0 0x7f9b5c0769c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.616+0000 7f9b82000640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b7c163f60 0x7f9b7c158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.616+0000 7f9b82000640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b7c0cd0a0 0x7f9b7c163a20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.616+0000 7f9b82000640 1 --2- 172.21.15.7:0/1255428932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b64004d40 0x7f9b7c0cf4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.616+0000 7f9b82000640 1 -- 172.21.15.7:0/1255428932 >> 172.21.15.7:0/1255428932 conn(0x7f9b7c0d59c0 msgr2=0x7f9b7c161550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:34.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.617+0000 7f9b82000640 1 -- 172.21.15.7:0/1255428932 shutdown_connections 2024-04-25T11:13:34.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.617+0000 7f9b82000640 1 -- 172.21.15.7:0/1255428932 wait complete. 2024-04-25T11:13:34.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:13:34.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.925+0000 7fee65c00640 1 Processor -- start 2024-04-25T11:13:34.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.926+0000 7fee65c00640 1 -- start start 2024-04-25T11:13:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.928+0000 7fee65c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee6015aaa0 0x7fee600affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.928+0000 7fee65c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee600b5020 0x7fee60160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.928+0000 7fee65c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee6006f5b0 con 0x7fee600d0ee0 2024-04-25T11:13:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.928+0000 7fee65c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fee600af290 con 0x7fee6015aaa0 2024-04-25T11:13:34.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.928+0000 7fee65c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fee600d14a0 con 0x7fee600b5020 2024-04-25T11:13:34.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.929+0000 7fee5ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee6015aaa0 0x7fee600affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.929+0000 7fee5e000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fee600d0ee0 0x7fee600b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52422/0 (socket says 172.21.15.7:52422) 2024-04-25T11:13:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.929+0000 7fee5e000640 1 -- 172.21.15.7:0/3389304867 learned_addr learned my addr 172.21.15.7:0/3389304867 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.929+0000 7fee5f400640 1 --2- 172.21.15.7:0/3389304867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee600b5020 0x7fee60160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.929+0000 7fee5ea00640 1 -- 172.21.15.7:0/3389304867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee600b5020 msgr2=0x7fee60160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.929+0000 7fee5ea00640 1 --2- 172.21.15.7:0/3389304867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee600b5020 0x7fee60160990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.929+0000 7fee5ea00640 1 -- 172.21.15.7:0/3389304867 >> v1:172.21.15.7:6789/0 conn(0x7fee600d0ee0 legacy=0x7fee600b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.929+0000 7fee5ea00640 1 -- 172.21.15.7:0/3389304867 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee600af4e0 con 0x7fee6015aaa0 2024-04-25T11:13:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.930+0000 7fee5ea00640 1 --2- 172.21.15.7:0/3389304867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee6015aaa0 0x7fee600affc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fee500097a0 tx=0x7fee5002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96aff98a9b9267a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.931+0000 7fee5d600640 1 -- 172.21.15.7:0/3389304867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50037070 con 0x7fee6015aaa0 2024-04-25T11:13:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.931+0000 7fee5d600640 1 -- 172.21.15.7:0/3389304867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee50032440 con 0x7fee6015aaa0 2024-04-25T11:13:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.931+0000 7fee5d600640 1 -- 172.21.15.7:0/3389304867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50032810 con 0x7fee6015aaa0 2024-04-25T11:13:34.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.931+0000 7fee65c00640 1 -- 172.21.15.7:0/3389304867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee6015aaa0 msgr2=0x7fee600affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:34.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.931+0000 7fee65c00640 1 --2- 172.21.15.7:0/3389304867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee6015aaa0 0x7fee600affc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fee500097a0 tx=0x7fee5002a180 comp rx=0 tx=0).stop 2024-04-25T11:13:34.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.932+0000 7fee65c00640 1 -- 172.21.15.7:0/3389304867 shutdown_connections 2024-04-25T11:13:34.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.932+0000 7fee65c00640 1 --2- 172.21.15.7:0/3389304867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee600b5020 0x7fee60160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.932+0000 7fee65c00640 1 --2- 172.21.15.7:0/3389304867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee6015aaa0 0x7fee600affc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.932+0000 7fee65c00640 1 -- 172.21.15.7:0/3389304867 >> 172.21.15.7:0/3389304867 conn(0x7fee600742c0 msgr2=0x7fee6006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.932+0000 7fee65c00640 1 -- 172.21.15.7:0/3389304867 shutdown_connections 2024-04-25T11:13:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.932+0000 7fee65c00640 1 -- 172.21.15.7:0/3389304867 wait complete. 2024-04-25T11:13:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.934+0000 7fee65c00640 1 Processor -- start 2024-04-25T11:13:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.934+0000 7fee65c00640 1 -- start start 2024-04-25T11:13:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.936+0000 7fee65c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee600b5020 0x7fee60140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.936+0000 7fee65c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee600d0ee0 0x7fee60158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.936+0000 7fee65c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee6015aaa0 0x7fee60158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.936+0000 7fee65c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fee600b5430 con 0x7fee600b5020 2024-04-25T11:13:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.936+0000 7fee65c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fee600afaa0 con 0x7fee600d0ee0 2024-04-25T11:13:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.936+0000 7fee65c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fee600b55b0 con 0x7fee6015aaa0 2024-04-25T11:13:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.936+0000 7fee5ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee600b5020 0x7fee60140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.936+0000 7fee5ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee600b5020 0x7fee60140c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44332/0 (socket says 172.21.15.7:44332) 2024-04-25T11:13:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.936+0000 7fee5ea00640 1 -- 172.21.15.7:0/2450081098 learned_addr learned my addr 172.21.15.7:0/2450081098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.936+0000 7fee5f400640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee6015aaa0 0x7fee60158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee5ea00640 1 -- 172.21.15.7:0/2450081098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee6015aaa0 msgr2=0x7fee60158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee5ea00640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee6015aaa0 0x7fee60158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee5ea00640 1 -- 172.21.15.7:0/2450081098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee600d0ee0 msgr2=0x7fee60158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee5e000640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee600d0ee0 0x7fee60158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee5ea00640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee600d0ee0 0x7fee60158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee5ea00640 1 -- 172.21.15.7:0/2450081098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee60141170 con 0x7fee600b5020 2024-04-25T11:13:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee5f400640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee6015aaa0 0x7fee60158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee5ea00640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee600b5020 0x7fee60140c30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fee50035040 tx=0x7fee500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee56a00640 1 -- 172.21.15.7:0/2450081098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50037040 con 0x7fee600b5020 2024-04-25T11:13:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee56a00640 1 -- 172.21.15.7:0/2450081098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee50042070 con 0x7fee600b5020 2024-04-25T11:13:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee56a00640 1 -- 172.21.15.7:0/2450081098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee50030bb0 con 0x7fee600b5020 2024-04-25T11:13:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.937+0000 7fee65c00640 1 -- 172.21.15.7:0/2450081098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee60141530 con 0x7fee600b5020 2024-04-25T11:13:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.938+0000 7fee65c00640 1 -- 172.21.15.7:0/2450081098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee60141af0 con 0x7fee600b5020 2024-04-25T11:13:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.939+0000 7fee65c00640 1 -- 172.21.15.7:0/2450081098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee60050740 con 0x7fee600b5020 2024-04-25T11:13:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.940+0000 7fee56a00640 1 -- 172.21.15.7:0/2450081098 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fee50047460 con 0x7fee600b5020 2024-04-25T11:13:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.941+0000 7fee56a00640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee400744b0 0x7fee400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.941+0000 7fee56a00640 1 -- 172.21.15.7:0/2450081098 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7fee500b5d10 con 0x7fee600b5020 2024-04-25T11:13:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.941+0000 7fee5e000640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee400744b0 0x7fee400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.942+0000 7fee5e000640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee400744b0 0x7fee400769c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fee6015dc20 tx=0x7fee4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:34.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:34.946+0000 7fee56a00640 1 -- 172.21.15.7:0/2450081098 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fee500bb020 con 0x7fee600b5020 2024-04-25T11:13:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.148+0000 7fee65c00640 1 -- 172.21.15.7:0/2450081098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fee6015df70 con 0x7fee600b5020 2024-04-25T11:13:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.156+0000 7fee56a00640 1 -- 172.21.15.7:0/2450081098 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fee50080570 con 0x7fee600b5020 2024-04-25T11:13:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 -- 172.21.15.7:0/2450081098 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee400744b0 msgr2=0x7fee400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee400744b0 0x7fee400769c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fee6015dc20 tx=0x7fee4c009040 comp rx=0 tx=0).stop 2024-04-25T11:13:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 -- 172.21.15.7:0/2450081098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee600b5020 msgr2=0x7fee60140c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee600b5020 0x7fee60140c30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fee50035040 tx=0x7fee500046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 -- 172.21.15.7:0/2450081098 shutdown_connections 2024-04-25T11:13:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee400744b0 0x7fee400769c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee6015aaa0 0x7fee60158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee600d0ee0 0x7fee60158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 --2- 172.21.15.7:0/2450081098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee600b5020 0x7fee60140c30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 -- 172.21.15.7:0/2450081098 >> 172.21.15.7:0/2450081098 conn(0x7fee600742c0 msgr2=0x7fee6006d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 -- 172.21.15.7:0/2450081098 shutdown_connections 2024-04-25T11:13:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.159+0000 7fee65c00640 1 -- 172.21.15.7:0/2450081098 wait complete. 2024-04-25T11:13:35.186 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:13:35.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:13:35.295 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:13:35.432 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:13:35.432 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:13:35.432 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:35.495 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:13:35.495 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:13:35.495 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:35.552 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:13:35.552 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:13:35.552 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:35.613 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:13:35.613 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:13:35.613 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:35.669 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:13:35.670 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:13:35.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:13:35.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.962+0000 7f5861600640 1 Processor -- start 2024-04-25T11:13:35.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.962+0000 7f5861600640 1 -- start start 2024-04-25T11:13:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.965+0000 7f5861600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f585c0b04d0 0x7f585c0b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.965+0000 7f5861600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f585c15e190 con 0x7f585c0b4a80 2024-04-25T11:13:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.965+0000 7f5861600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f585c15d610 con 0x7f585c070a70 2024-04-25T11:13:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.965+0000 7f5861600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f585c15d8f0 con 0x7f585c0b04d0 2024-04-25T11:13:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.965+0000 7f585b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f585c0b04d0 0x7f585c0b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.965+0000 7f5860c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f585c0b4a80 0x7f585c0b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52438/0 (socket says 172.21.15.7:52438) 2024-04-25T11:13:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.965+0000 7f5860c00640 1 -- 172.21.15.7:0/2769190152 learned_addr learned my addr 172.21.15.7:0/2769190152 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:35.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.966+0000 7f585b400640 1 -- 172.21.15.7:0/2769190152 >> v1:172.21.15.87:6789/0 conn(0x7f585c070a70 legacy=0x7f585c0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:35.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.966+0000 7f585b400640 1 -- 172.21.15.7:0/2769190152 >> v1:172.21.15.7:6789/0 conn(0x7f585c0b4a80 legacy=0x7f585c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:35.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.966+0000 7f585b400640 1 -- 172.21.15.7:0/2769190152 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c15dbd0 con 0x7f585c0b04d0 2024-04-25T11:13:35.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.966+0000 7f585aa00640 1 -- 172.21.15.7:0/2769190152 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2685159889 0 0) 0x7f585c15e190 con 0x7f585c0b4a80 2024-04-25T11:13:35.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.967+0000 7f585b400640 1 --2- 172.21.15.7:0/2769190152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c0b04d0 0x7f585c0b08e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f58480097a0 tx=0x7f584802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c925d83c029a9429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:35.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.967+0000 7f585aa00640 1 -- 172.21.15.7:0/2769190152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848037070 con 0x7f585c0b04d0 2024-04-25T11:13:35.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.967+0000 7f585aa00640 1 -- 172.21.15.7:0/2769190152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5848032440 con 0x7f585c0b04d0 2024-04-25T11:13:35.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.967+0000 7f585aa00640 1 -- 172.21.15.7:0/2769190152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848032810 con 0x7f585c0b04d0 2024-04-25T11:13:35.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.968+0000 7f5861600640 1 -- 172.21.15.7:0/2769190152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c0b04d0 msgr2=0x7f585c0b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:35.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.968+0000 7f5861600640 1 --2- 172.21.15.7:0/2769190152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c0b04d0 0x7f585c0b08e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f58480097a0 tx=0x7f584802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:13:35.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.968+0000 7f5861600640 1 -- 172.21.15.7:0/2769190152 shutdown_connections 2024-04-25T11:13:35.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.968+0000 7f5861600640 1 --2- 172.21.15.7:0/2769190152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c0b04d0 0x7f585c0b08e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:35.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.968+0000 7f5861600640 1 -- 172.21.15.7:0/2769190152 >> 172.21.15.7:0/2769190152 conn(0x7f585c0b3820 msgr2=0x7f585c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:35.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.968+0000 7f5861600640 1 -- 172.21.15.7:0/2769190152 shutdown_connections 2024-04-25T11:13:35.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.968+0000 7f5861600640 1 -- 172.21.15.7:0/2769190152 wait complete. 2024-04-25T11:13:35.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.971+0000 7f5861600640 1 Processor -- start 2024-04-25T11:13:35.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.971+0000 7f5861600640 1 -- start start 2024-04-25T11:13:35.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f5861600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c070a70 0x7f585c1f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:35.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f5861600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f585c0b04d0 0x7f585c1f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:35.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f5861600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f585c0b4a80 0x7f585c15bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f5861600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f585c0b0e20 con 0x7f585c070a70 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f5861600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f585c0aef90 con 0x7f585c0b04d0 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f5861600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f585c15d170 con 0x7f585c0b4a80 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f585be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c070a70 0x7f585c1f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f585be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c070a70 0x7f585c1f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44342/0 (socket says 172.21.15.7:44342) 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f585be00640 1 -- 172.21.15.7:0/40707750 learned_addr learned my addr 172.21.15.7:0/40707750 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f585b400640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f585c0b04d0 0x7f585c1f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f585be00640 1 -- 172.21.15.7:0/40707750 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f585c0b4a80 msgr2=0x7f585c15bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f585be00640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f585c0b4a80 0x7f585c15bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f585be00640 1 -- 172.21.15.7:0/40707750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f585c0b04d0 msgr2=0x7f585c1f8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f585be00640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f585c0b04d0 0x7f585c1f8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.973+0000 7f585be00640 1 -- 172.21.15.7:0/40707750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c1f8f60 con 0x7f585c070a70 2024-04-25T11:13:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.974+0000 7f585be00640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c070a70 0x7f585c1f6710 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f584c00b3f0 tx=0x7f584c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.974+0000 7f5853e00640 1 -- 172.21.15.7:0/40707750 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c015020 con 0x7f585c070a70 2024-04-25T11:13:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.974+0000 7f5853e00640 1 -- 172.21.15.7:0/40707750 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f584c011830 con 0x7f585c070a70 2024-04-25T11:13:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.974+0000 7f5853e00640 1 -- 172.21.15.7:0/40707750 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c011c20 con 0x7f585c070a70 2024-04-25T11:13:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.974+0000 7f5861600640 1 -- 172.21.15.7:0/40707750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585c158250 con 0x7f585c070a70 2024-04-25T11:13:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.974+0000 7f5861600640 1 -- 172.21.15.7:0/40707750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585c0cafa0 con 0x7f585c070a70 2024-04-25T11:13:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.976+0000 7f5861600640 1 -- 172.21.15.7:0/40707750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585c0caab0 con 0x7f585c070a70 2024-04-25T11:13:35.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.977+0000 7f5853e00640 1 -- 172.21.15.7:0/40707750 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f584c018020 con 0x7f585c070a70 2024-04-25T11:13:35.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.978+0000 7f5853e00640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f583c0744e0 0x7f583c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:35.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.978+0000 7f5853e00640 1 -- 172.21.15.7:0/40707750 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5335+0+0 (secure 0 0 0) 0x7f584c094de0 con 0x7f585c070a70 2024-04-25T11:13:35.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.978+0000 7f585b400640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f583c0744e0 0x7f583c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:35.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.979+0000 7f585b400640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f583c0744e0 0x7f583c0769f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5848034000 tx=0x7f584802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:35.983+0000 7f5853e00640 1 -- 172.21.15.7:0/40707750 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f584c01f030 con 0x7f585c070a70 2024-04-25T11:13:36.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.225+0000 7f5861600640 1 -- 172.21.15.7:0/40707750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f585c050740 con 0x7f585c070a70 2024-04-25T11:13:36.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.226+0000 7f5853e00640 1 -- 172.21.15.7:0/40707750 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3863 (secure 0 0 0) 0x7f584c05f640 con 0x7f585c070a70 2024-04-25T11:13:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:36.227 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/1868898207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1868898207},{"type":"v1","addr":"172.21.15.87:6833","nonce":1868898207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2575517671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2575517671},{"type":"v1","addr":"172.21.15.87:6835","nonce":2575517671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/3746682343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3746682343},{"type":"v1","addr":"172.21.15.7:6836","nonce":3746682343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5271,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2132415318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2132415318},{"type":"v1","addr":"172.21.15.7:6837","nonce":2132415318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-04-25T11:13:36.227 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2024-04-25T11:13:36.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.228+0000 7f5861600640 1 -- 172.21.15.7:0/40707750 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f583c0744e0 msgr2=0x7f583c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:36.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.228+0000 7f5861600640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f583c0744e0 0x7f583c0769f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5848034000 tx=0x7f584802e000 comp rx=0 tx=0).stop 2024-04-25T11:13:36.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.228+0000 7f5861600640 1 -- 172.21.15.7:0/40707750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c070a70 msgr2=0x7f585c1f6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:36.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.228+0000 7f5861600640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c070a70 0x7f585c1f6710 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f584c00b3f0 tx=0x7f584c00b8b0 comp rx=0 tx=0).stop 2024-04-25T11:13:36.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.229+0000 7f5861600640 1 -- 172.21.15.7:0/40707750 shutdown_connections 2024-04-25T11:13:36.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.229+0000 7f5861600640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f583c0744e0 0x7f583c0769f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:36.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.229+0000 7f5861600640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f585c0b4a80 0x7f585c15bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:36.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.229+0000 7f5861600640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f585c0b04d0 0x7f585c1f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:36.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.229+0000 7f5861600640 1 --2- 172.21.15.7:0/40707750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c070a70 0x7f585c1f6710 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:36.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.229+0000 7f5861600640 1 -- 172.21.15.7:0/40707750 >> 172.21.15.7:0/40707750 conn(0x7f585c0b3820 msgr2=0x7f585c06eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:36.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.229+0000 7f5861600640 1 -- 172.21.15.7:0/40707750 shutdown_connections 2024-04-25T11:13:36.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.229+0000 7f5861600640 1 -- 172.21.15.7:0/40707750 wait complete. 2024-04-25T11:13:36.257 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:13:36.257 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:13:36.257 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:13:36.257 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:13:36.258 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:13:36.258 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:13:36.258 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:13:36.258 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:13:36.258 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:13:36.258 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:13:36.259 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:13:36.259 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:13:36.259 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:13:36.259 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:13:36.259 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:13:36.259 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:13:36.284 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:13:36.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:13:36.288 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:13:36.288 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:13:36.292 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:13:36.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:13:36.298 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:13:36.298 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:13:36.339 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:13:36.340 DEBUG:teuthology.parallel:result is None 2024-04-25T11:13:36.343 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:13:36.343 DEBUG:teuthology.parallel:result is None 2024-04-25T11:13:36.346 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:13:36.346 DEBUG:teuthology.parallel:result is None 2024-04-25T11:13:36.347 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:13:36.347 DEBUG:teuthology.parallel:result is None 2024-04-25T11:13:36.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:13:36.566 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:13:36.567 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:13:36.568 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:13:36.568 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:13:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.606+0000 7f6436200640 1 Processor -- start 2024-04-25T11:13:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.606+0000 7f6436200640 1 -- start start 2024-04-25T11:13:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.608+0000 7f6436200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f643006d580 0x7f643006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.608+0000 7f6436200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643006f1e0 0x7f643006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.608+0000 7f6436200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f643006ded0 con 0x7f643015f180 2024-04-25T11:13:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.608+0000 7f6436200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f643015f590 con 0x7f643006d580 2024-04-25T11:13:36.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.608+0000 7f6436200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6430158ae0 con 0x7f643006f1e0 2024-04-25T11:13:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.608+0000 7f6427e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f643006d580 0x7f643006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.608+0000 7f6427e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f643006d580 0x7f643006d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44380/0 (socket says 172.21.15.7:44380) 2024-04-25T11:13:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.608+0000 7f6427e00640 1 -- 172.21.15.7:0/1736274837 learned_addr learned my addr 172.21.15.7:0/1736274837 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.608+0000 7f642fe00640 1 --2- 172.21.15.7:0/1736274837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643006f1e0 0x7f643006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.609+0000 7f6427e00640 1 -- 172.21.15.7:0/1736274837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643006f1e0 msgr2=0x7f643006f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.609+0000 7f6427e00640 1 --2- 172.21.15.7:0/1736274837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643006f1e0 0x7f643006f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.609+0000 7f642ea00640 1 -- 172.21.15.7:0/1736274837 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279169293 0 0) 0x7f6420001110 con 0x7f643015f180 2024-04-25T11:13:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.609+0000 7f6427e00640 1 -- 172.21.15.7:0/1736274837 >> v1:172.21.15.87:6789/0 conn(0x7f643015f180 legacy=0x7f6430161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.609+0000 7f6427e00640 1 -- 172.21.15.7:0/1736274837 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6430162700 con 0x7f643006d580 2024-04-25T11:13:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.610+0000 7f6427e00640 1 --2- 172.21.15.7:0/1736274837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643006d580 0x7f643006d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f64180098a0 tx=0x7f641802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0af9050099bfd43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.610+0000 7f642ea00640 1 -- 172.21.15.7:0/1736274837 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418037070 con 0x7f643006d580 2024-04-25T11:13:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.610+0000 7f642ea00640 1 -- 172.21.15.7:0/1736274837 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6418032440 con 0x7f643006d580 2024-04-25T11:13:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.610+0000 7f642ea00640 1 -- 172.21.15.7:0/1736274837 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418032810 con 0x7f643006d580 2024-04-25T11:13:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.611+0000 7f6436200640 1 -- 172.21.15.7:0/1736274837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643006d580 msgr2=0x7f643006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.611+0000 7f6436200640 1 --2- 172.21.15.7:0/1736274837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643006d580 0x7f643006d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f64180098a0 tx=0x7f641802a290 comp rx=0 tx=0).stop 2024-04-25T11:13:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.611+0000 7f6436200640 1 -- 172.21.15.7:0/1736274837 shutdown_connections 2024-04-25T11:13:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.611+0000 7f6436200640 1 --2- 172.21.15.7:0/1736274837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643006f1e0 0x7f643006f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.611+0000 7f6436200640 1 --2- 172.21.15.7:0/1736274837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643006d580 0x7f643006d990 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.611+0000 7f6436200640 1 -- 172.21.15.7:0/1736274837 >> 172.21.15.7:0/1736274837 conn(0x7f64300732b0 msgr2=0x7f64300736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.611+0000 7f6436200640 1 -- 172.21.15.7:0/1736274837 shutdown_connections 2024-04-25T11:13:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.611+0000 7f6436200640 1 -- 172.21.15.7:0/1736274837 wait complete. 2024-04-25T11:13:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.613+0000 7f6436200640 1 Processor -- start 2024-04-25T11:13:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.614+0000 7f6436200640 1 -- start start 2024-04-25T11:13:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.615+0000 7f6436200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f643006d580 0x7f64301512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.615+0000 7f6436200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643006f1e0 0x7f64300b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.615+0000 7f6436200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643015f180 0x7f64300b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.615+0000 7f6436200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f643015ec60 con 0x7f643006f1e0 2024-04-25T11:13:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.615+0000 7f6436200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f643015e980 con 0x7f643006d580 2024-04-25T11:13:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.615+0000 7f6436200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f643015e670 con 0x7f643015f180 2024-04-25T11:13:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.615+0000 7f6427e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643006f1e0 0x7f64300b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.615+0000 7f6427e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643006f1e0 0x7f64300b49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44386/0 (socket says 172.21.15.7:44386) 2024-04-25T11:13:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.616+0000 7f6427e00640 1 -- 172.21.15.7:0/2355535875 learned_addr learned my addr 172.21.15.7:0/2355535875 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.616+0000 7f642f400640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f643006d580 0x7f64301512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.616+0000 7f642fe00640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643015f180 0x7f64300b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.616+0000 7f6427e00640 1 -- 172.21.15.7:0/2355535875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643015f180 msgr2=0x7f64300b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.616+0000 7f6427e00640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643015f180 0x7f64300b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.616+0000 7f6427e00640 1 -- 172.21.15.7:0/2355535875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f643006d580 msgr2=0x7f64301512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.616+0000 7f6427e00640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f643006d580 0x7f64301512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.616+0000 7f6427e00640 1 -- 172.21.15.7:0/2355535875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64301517e0 con 0x7f643006f1e0 2024-04-25T11:13:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.616+0000 7f642f400640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f643006d580 0x7f64301512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:13:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.617+0000 7f6427e00640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643006f1e0 0x7f64300b49a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f64180097f0 tx=0x7f64180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.620+0000 7f6427400640 1 -- 172.21.15.7:0/2355535875 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418041020 con 0x7f643006f1e0 2024-04-25T11:13:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.620+0000 7f6436200640 1 -- 172.21.15.7:0/2355535875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64301578d0 con 0x7f643006f1e0 2024-04-25T11:13:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.620+0000 7f6427400640 1 -- 172.21.15.7:0/2355535875 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f641803d6e0 con 0x7f643006f1e0 2024-04-25T11:13:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.620+0000 7f6427400640 1 -- 172.21.15.7:0/2355535875 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418037040 con 0x7f643006f1e0 2024-04-25T11:13:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.620+0000 7f6436200640 1 -- 172.21.15.7:0/2355535875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643015fb20 con 0x7f643006f1e0 2024-04-25T11:13:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.621+0000 7f6436200640 1 -- 172.21.15.7:0/2355535875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6430050740 con 0x7f643006f1e0 2024-04-25T11:13:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.623+0000 7f6427400640 1 -- 172.21.15.7:0/2355535875 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6418044020 con 0x7f643006f1e0 2024-04-25T11:13:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.624+0000 7f6427400640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f640c0744b0 0x7f640c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.624+0000 7f6427400640 1 -- 172.21.15.7:0/2355535875 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5335+0+0 (secure 0 0 0) 0x7f64180b57d0 con 0x7f643006f1e0 2024-04-25T11:13:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.624+0000 7f642f400640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f640c0744b0 0x7f640c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.625+0000 7f642f400640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f640c0744b0 0x7f640c0769c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6420000f60 tx=0x7f6420009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:36.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.629+0000 7f6427400640 1 -- 172.21.15.7:0/2355535875 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f641804b030 con 0x7f643006f1e0 2024-04-25T11:13:36.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:36.847+0000 7f6436200640 1 -- 172.21.15.7:0/2355535875 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6430074250 con 0x7f643006f1e0 2024-04-25T11:13:37.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.658+0000 7f6427400640 1 -- 172.21.15.7:0/2355535875 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) ==== 97+0+0 (secure 0 0 0) 0x7f641807f9f0 con 0x7f643006f1e0 2024-04-25T11:13:37.659 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-04-25T11:13:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.663+0000 7f6436200640 1 -- 172.21.15.7:0/2355535875 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f640c0744b0 msgr2=0x7f640c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.663+0000 7f6436200640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f640c0744b0 0x7f640c0769c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6420000f60 tx=0x7f6420009040 comp rx=0 tx=0).stop 2024-04-25T11:13:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.663+0000 7f6436200640 1 -- 172.21.15.7:0/2355535875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643006f1e0 msgr2=0x7f64300b49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.663+0000 7f6436200640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643006f1e0 0x7f64300b49a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f64180097f0 tx=0x7f64180046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.663+0000 7f6436200640 1 -- 172.21.15.7:0/2355535875 shutdown_connections 2024-04-25T11:13:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.663+0000 7f6436200640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f640c0744b0 0x7f640c0769c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.663+0000 7f6436200640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643015f180 0x7f64300b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.663+0000 7f6436200640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f643006f1e0 0x7f64300b49a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.663+0000 7f6436200640 1 --2- 172.21.15.7:0/2355535875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f643006d580 0x7f64301512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.663+0000 7f6436200640 1 -- 172.21.15.7:0/2355535875 >> 172.21.15.7:0/2355535875 conn(0x7f64300732b0 msgr2=0x7f64300b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.664+0000 7f6436200640 1 -- 172.21.15.7:0/2355535875 shutdown_connections 2024-04-25T11:13:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.664+0000 7f6436200640 1 -- 172.21.15.7:0/2355535875 wait complete. 2024-04-25T11:13:37.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:13:37.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.974+0000 7f7100a00640 1 Processor -- start 2024-04-25T11:13:37.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.974+0000 7f7100a00640 1 -- start start 2024-04-25T11:13:37.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.977+0000 7f7100a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f70fc15e2f0 0x7f70fc161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:37.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.977+0000 7f7100a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70fc162420 0x7f70fc06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:37.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.977+0000 7f7100a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70fc15e700 con 0x7f70fc15baa0 2024-04-25T11:13:37.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.977+0000 7f7100a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f70fc162830 con 0x7f70fc15e2f0 2024-04-25T11:13:37.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.977+0000 7f7100a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f70fc15e980 con 0x7f70fc162420 2024-04-25T11:13:37.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.978+0000 7f70f3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f70fc15e2f0 0x7f70fc161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:37.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.978+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70fc162420 0x7f70fc06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:37.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.978+0000 7f70f3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f70fc15e2f0 0x7f70fc161ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:47502/0 (socket says 172.21.15.7:47502) 2024-04-25T11:13:37.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.978+0000 7f70f3e00640 1 -- 172.21.15.7:0/174445398 learned_addr learned my addr 172.21.15.7:0/174445398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:37.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.978+0000 7f70f3e00640 1 -- 172.21.15.7:0/174445398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70fc162420 msgr2=0x7f70fc06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:37.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.978+0000 7f70f3e00640 1 --2- 172.21.15.7:0/174445398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70fc162420 0x7f70fc06d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:37.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.978+0000 7f70f3e00640 1 -- 172.21.15.7:0/174445398 >> v1:172.21.15.87:6789/0 conn(0x7f70fc15baa0 legacy=0x7f70fc070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:37.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.978+0000 7f70f3e00640 1 -- 172.21.15.7:0/174445398 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc15ec60 con 0x7f70fc15e2f0 2024-04-25T11:13:37.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.980+0000 7f70f3e00640 1 --2- 172.21.15.7:0/174445398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70fc15e2f0 0x7f70fc161ee0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009870 tx=0x7f70e402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e41d521df182d2cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:37.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.980+0000 7f70faa00640 1 -- 172.21.15.7:0/174445398 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4037070 con 0x7f70fc15e2f0 2024-04-25T11:13:37.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.981+0000 7f70faa00640 1 -- 172.21.15.7:0/174445398 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e4032440 con 0x7f70fc15e2f0 2024-04-25T11:13:37.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.981+0000 7f70faa00640 1 -- 172.21.15.7:0/174445398 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4032810 con 0x7f70fc15e2f0 2024-04-25T11:13:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.981+0000 7f7100a00640 1 -- 172.21.15.7:0/174445398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70fc15e2f0 msgr2=0x7f70fc161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.981+0000 7f7100a00640 1 --2- 172.21.15.7:0/174445398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70fc15e2f0 0x7f70fc161ee0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f70e4009870 tx=0x7f70e402a270 comp rx=0 tx=0).stop 2024-04-25T11:13:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.981+0000 7f7100a00640 1 -- 172.21.15.7:0/174445398 shutdown_connections 2024-04-25T11:13:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.981+0000 7f7100a00640 1 --2- 172.21.15.7:0/174445398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70fc162420 0x7f70fc06d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.981+0000 7f7100a00640 1 --2- 172.21.15.7:0/174445398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70fc15e2f0 0x7f70fc161ee0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.981+0000 7f7100a00640 1 -- 172.21.15.7:0/174445398 >> 172.21.15.7:0/174445398 conn(0x7f70fc073090 msgr2=0x7f70fc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.982+0000 7f7100a00640 1 -- 172.21.15.7:0/174445398 shutdown_connections 2024-04-25T11:13:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.982+0000 7f7100a00640 1 -- 172.21.15.7:0/174445398 wait complete. 2024-04-25T11:13:37.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.984+0000 7f7100a00640 1 Processor -- start 2024-04-25T11:13:37.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.984+0000 7f7100a00640 1 -- start start 2024-04-25T11:13:37.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.986+0000 7f7100a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70fc15e2f0 0x7f70fc0b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:37.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.986+0000 7f7100a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70fc162420 0x7f70fc0af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:37.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.986+0000 7f7100a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70fc1601d0 0x7f70fc1605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:37.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.986+0000 7f7100a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc06dac0 con 0x7f70fc1601d0 2024-04-25T11:13:37.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.986+0000 7f7100a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc1619c0 con 0x7f70fc15e2f0 2024-04-25T11:13:37.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.986+0000 7f7100a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f70fc06dc40 con 0x7f70fc162420 2024-04-25T11:13:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.986+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70fc1601d0 0x7f70fc1605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.986+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70fc1601d0 0x7f70fc1605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44390/0 (socket says 172.21.15.7:44390) 2024-04-25T11:13:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.986+0000 7f70fbe00640 1 -- 172.21.15.7:0/2970408395 learned_addr learned my addr 172.21.15.7:0/2970408395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.986+0000 7f70f3e00640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70fc162420 0x7f70fc0af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.986+0000 7f70fb400640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70fc15e2f0 0x7f70fc0b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.987+0000 7f70fbe00640 1 -- 172.21.15.7:0/2970408395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70fc162420 msgr2=0x7f70fc0af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.987+0000 7f70fbe00640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70fc162420 0x7f70fc0af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.987+0000 7f70fbe00640 1 -- 172.21.15.7:0/2970408395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70fc15e2f0 msgr2=0x7f70fc0b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.987+0000 7f70fbe00640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70fc15e2f0 0x7f70fc0b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.987+0000 7f70fbe00640 1 -- 172.21.15.7:0/2970408395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc160b20 con 0x7f70fc1601d0 2024-04-25T11:13:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.987+0000 7f70fb400640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70fc15e2f0 0x7f70fc0b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:13:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.987+0000 7f70fbe00640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70fc1601d0 0x7f70fc1605e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f70f400efc0 tx=0x7f70f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.988+0000 7f70f3400640 1 -- 172.21.15.7:0/2970408395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4018020 con 0x7f70fc1601d0 2024-04-25T11:13:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.988+0000 7f70f3400640 1 -- 172.21.15.7:0/2970408395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f4014480 con 0x7f70fc1601d0 2024-04-25T11:13:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.988+0000 7f7100a00640 1 -- 172.21.15.7:0/2970408395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70fc0af4f0 con 0x7f70fc1601d0 2024-04-25T11:13:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.988+0000 7f70f3400640 1 -- 172.21.15.7:0/2970408395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4009e60 con 0x7f70fc1601d0 2024-04-25T11:13:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.988+0000 7f7100a00640 1 -- 172.21.15.7:0/2970408395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70fc0b0120 con 0x7f70fc1601d0 2024-04-25T11:13:37.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.989+0000 7f7100a00640 1 -- 172.21.15.7:0/2970408395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70fc074020 con 0x7f70fc1601d0 2024-04-25T11:13:37.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.990+0000 7f70f3400640 1 -- 172.21.15.7:0/2970408395 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f70f4005d10 con 0x7f70fc1601d0 2024-04-25T11:13:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.991+0000 7f70f3400640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70d80744b0 0x7f70d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:37.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.992+0000 7f70f3400640 1 -- 172.21.15.7:0/2970408395 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f70f4095d60 con 0x7f70fc1601d0 2024-04-25T11:13:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.992+0000 7f70fb400640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70d80744b0 0x7f70d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:37.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.992+0000 7f70fb400640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70d80744b0 0x7f70d80769c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f70ec007fd0 tx=0x7f70ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:37.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:37.997+0000 7f70f3400640 1 -- 172.21.15.7:0/2970408395 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f70f4022030 con 0x7f70fc1601d0 2024-04-25T11:13:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.272+0000 7f7100a00640 1 -- 172.21.15.7:0/2970408395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f70fc1f6fb0 con 0x7f70fc1601d0 2024-04-25T11:13:38.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.273+0000 7f70f3400640 1 -- 172.21.15.7:0/2970408395 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f70fc160b20 con 0x7f70fc1601d0 2024-04-25T11:13:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:38.273 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:13:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-04-25T11:13:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-04-25T11:13:38.273 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w==", 2024-04-25T11:13:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:13:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:13:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:13:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-04-25T11:13:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw==", 2024-04-25T11:13:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.275 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:13:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:13:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:13:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.276 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-04-25T11:13:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw==", 2024-04-25T11:13:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:13:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:13:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:13:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A==", 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-04-25T11:13:38.278 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA==", 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg==", 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:13:38.279 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-04-25T11:13:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw==", 2024-04-25T11:13:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:13:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:13:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ==", 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw==", 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:13:38.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng==", 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-04-25T11:13:38.282 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ==", 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A==", 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg==", 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA==", 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:13:38.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg==", 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-04-25T11:13:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ==", 2024-04-25T11:13:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:13:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:13:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:13:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:13:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDZOSpmS8ObHxAAkXXXg1OI1YFC2/OZER3woA==", 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw==", 2024-04-25T11:13:38.287 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w==", 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:13:38.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:38.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-25T11:13:38.289 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:13:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.275+0000 7f7100a00640 1 -- 172.21.15.7:0/2970408395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70d80744b0 msgr2=0x7f70d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.275+0000 7f7100a00640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70d80744b0 0x7f70d80769c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f70ec007fd0 tx=0x7f70ec009040 comp rx=0 tx=0).stop 2024-04-25T11:13:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.275+0000 7f7100a00640 1 -- 172.21.15.7:0/2970408395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70fc1601d0 msgr2=0x7f70fc1605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.275+0000 7f7100a00640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70fc1601d0 0x7f70fc1605e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f70f400efc0 tx=0x7f70f400c490 comp rx=0 tx=0).stop 2024-04-25T11:13:38.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.275+0000 7f7100a00640 1 -- 172.21.15.7:0/2970408395 shutdown_connections 2024-04-25T11:13:38.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.275+0000 7f7100a00640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f70d80744b0 0x7f70d80769c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.275+0000 7f7100a00640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70fc1601d0 0x7f70fc1605e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.275+0000 7f7100a00640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f70fc162420 0x7f70fc0af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.275+0000 7f7100a00640 1 --2- 172.21.15.7:0/2970408395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70fc15e2f0 0x7f70fc0b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.275+0000 7f7100a00640 1 -- 172.21.15.7:0/2970408395 >> 172.21.15.7:0/2970408395 conn(0x7f70fc073090 msgr2=0x7f70fc15c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:38.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.276+0000 7f7100a00640 1 -- 172.21.15.7:0/2970408395 shutdown_connections 2024-04-25T11:13:38.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.276+0000 7f7100a00640 1 -- 172.21.15.7:0/2970408395 wait complete. 2024-04-25T11:13:38.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.571+0000 7fbdb1a00640 1 Processor -- start 2024-04-25T11:13:38.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.571+0000 7fbdb1a00640 1 -- start start 2024-04-25T11:13:38.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.573+0000 7fbdb1a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdac0b5220 0x7fbdac15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:38.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.573+0000 7fbdb1a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdac0af420 0x7fbdac0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.573+0000 7fbdb1a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdac0cebb0 con 0x7fbdac0aed40 2024-04-25T11:13:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.573+0000 7fbdb1a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbdac15e1f0 con 0x7fbdac0b5220 2024-04-25T11:13:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.573+0000 7fbdb1a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbdac0afd70 con 0x7fbdac0af420 2024-04-25T11:13:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.573+0000 7fbdab400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdac0b5220 0x7fbdac15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.574+0000 7fbdab400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdac0b5220 0x7fbdac15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44404/0 (socket says 172.21.15.7:44404) 2024-04-25T11:13:38.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.574+0000 7fbdab400640 1 -- 172.21.15.7:0/3966445687 learned_addr learned my addr 172.21.15.7:0/3966445687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:38.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.574+0000 7fbdb1000640 1 --2- 172.21.15.7:0/3966445687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdac0af420 0x7fbdac0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:38.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.574+0000 7fbdab400640 1 -- 172.21.15.7:0/3966445687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdac0af420 msgr2=0x7fbdac0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:38.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.574+0000 7fbdab400640 1 --2- 172.21.15.7:0/3966445687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdac0af420 0x7fbdac0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.574+0000 7fbdab400640 1 -- 172.21.15.7:0/3966445687 >> v1:172.21.15.87:6789/0 conn(0x7fbdac0aed40 legacy=0x7fbdac0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:38.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.574+0000 7fbdab400640 1 -- 172.21.15.7:0/3966445687 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdac0b64b0 con 0x7fbdac0b5220 2024-04-25T11:13:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.575+0000 7fbdab400640 1 --2- 172.21.15.7:0/3966445687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdac0b5220 0x7fbdac15dcb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009870 tx=0x7fbd9402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95e69fc458a585b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.576+0000 7fbdaaa00640 1 -- 172.21.15.7:0/3966445687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94037070 con 0x7fbdac0b5220 2024-04-25T11:13:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.576+0000 7fbdaaa00640 1 -- 172.21.15.7:0/3966445687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd94032440 con 0x7fbdac0b5220 2024-04-25T11:13:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.576+0000 7fbdaaa00640 1 -- 172.21.15.7:0/3966445687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94032810 con 0x7fbdac0b5220 2024-04-25T11:13:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.576+0000 7fbdb1a00640 1 -- 172.21.15.7:0/3966445687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdac0b5220 msgr2=0x7fbdac15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.576+0000 7fbdb1a00640 1 --2- 172.21.15.7:0/3966445687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdac0b5220 0x7fbdac15dcb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009870 tx=0x7fbd9402a210 comp rx=0 tx=0).stop 2024-04-25T11:13:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.576+0000 7fbdb1a00640 1 -- 172.21.15.7:0/3966445687 shutdown_connections 2024-04-25T11:13:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.576+0000 7fbdb1a00640 1 --2- 172.21.15.7:0/3966445687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdac0af420 0x7fbdac0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.576+0000 7fbdb1a00640 1 --2- 172.21.15.7:0/3966445687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdac0b5220 0x7fbdac15dcb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.576+0000 7fbdb1a00640 1 -- 172.21.15.7:0/3966445687 >> 172.21.15.7:0/3966445687 conn(0x7fbdac0b3820 msgr2=0x7fbdac0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.577+0000 7fbdb1a00640 1 -- 172.21.15.7:0/3966445687 shutdown_connections 2024-04-25T11:13:38.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.577+0000 7fbdb1a00640 1 -- 172.21.15.7:0/3966445687 wait complete. 2024-04-25T11:13:38.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.579+0000 7fbdb1a00640 1 Processor -- start 2024-04-25T11:13:38.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.579+0000 7fbdb1a00640 1 -- start start 2024-04-25T11:13:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.581+0000 7fbdb1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdac0aed40 0x7fbdac159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.581+0000 7fbdb1a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdac0af420 0x7fbdac0d0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.581+0000 7fbdb1a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdac0b5220 0x7fbdac0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.581+0000 7fbdb1a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac06cbd0 con 0x7fbdac0aed40 2024-04-25T11:13:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.581+0000 7fbdb1a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac06c8f0 con 0x7fbdac0af420 2024-04-25T11:13:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.581+0000 7fbdb1a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbdac06ceb0 con 0x7fbdac0b5220 2024-04-25T11:13:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.581+0000 7fbdabe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdac0aed40 0x7fbdac159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.581+0000 7fbdabe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdac0aed40 0x7fbdac159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44408/0 (socket says 172.21.15.7:44408) 2024-04-25T11:13:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.581+0000 7fbdabe00640 1 -- 172.21.15.7:0/2587824664 learned_addr learned my addr 172.21.15.7:0/2587824664 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbdab400640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdac0af420 0x7fbdac0d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:38.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbdb1000640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdac0b5220 0x7fbdac0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbdabe00640 1 -- 172.21.15.7:0/2587824664 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdac0b5220 msgr2=0x7fbdac0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbdabe00640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdac0b5220 0x7fbdac0d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbdabe00640 1 -- 172.21.15.7:0/2587824664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdac0af420 msgr2=0x7fbdac0d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbdabe00640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdac0af420 0x7fbdac0d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbdabe00640 1 -- 172.21.15.7:0/2587824664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdac0d0db0 con 0x7fbdac0aed40 2024-04-25T11:13:38.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbdabe00640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdac0aed40 0x7fbdac159110 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00b9f0 tx=0x7fbd9c00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:38.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbda3e00640 1 -- 172.21.15.7:0/2587824664 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c00cbd0 con 0x7fbdac0aed40 2024-04-25T11:13:38.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbda3e00640 1 -- 172.21.15.7:0/2587824664 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9c00cd70 con 0x7fbdac0aed40 2024-04-25T11:13:38.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbda3e00640 1 -- 172.21.15.7:0/2587824664 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c003bb0 con 0x7fbdac0aed40 2024-04-25T11:13:38.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbdb1a00640 1 -- 172.21.15.7:0/2587824664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdac151520 con 0x7fbdac0aed40 2024-04-25T11:13:38.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.582+0000 7fbdb1a00640 1 -- 172.21.15.7:0/2587824664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdac0cf850 con 0x7fbdac0aed40 2024-04-25T11:13:38.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.584+0000 7fbdb1a00640 1 -- 172.21.15.7:0/2587824664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdac0b4a60 con 0x7fbdac0aed40 2024-04-25T11:13:38.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.585+0000 7fbda3e00640 1 -- 172.21.15.7:0/2587824664 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd9c014440 con 0x7fbdac0aed40 2024-04-25T11:13:38.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.586+0000 7fbda3e00640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd880744e0 0x7fbd880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:38.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.586+0000 7fbda3e00640 1 -- 172.21.15.7:0/2587824664 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7fbd9c095000 con 0x7fbdac0aed40 2024-04-25T11:13:38.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.587+0000 7fbdab400640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd880744e0 0x7fbd880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:38.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.587+0000 7fbdab400640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd880744e0 0x7fbd880769f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbd94034000 tx=0x7fbd9402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:38.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.591+0000 7fbda3e00640 1 -- 172.21.15.7:0/2587824664 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd9c05f8b0 con 0x7fbdac0aed40 2024-04-25T11:13:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.799+0000 7fbdb1a00640 1 -- 172.21.15.7:0/2587824664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdac06e0b0 con 0x7fbdac0aed40 2024-04-25T11:13:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.801+0000 7fbda3e00640 1 -- 172.21.15.7:0/2587824664 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13068 (secure 0 0 0) 0x7fbd9c05f1b0 con 0x7fbdac0aed40 2024-04-25T11:13:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:38.802 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:37.652832+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 -- 172.21.15.7:0/2587824664 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd880744e0 msgr2=0x7fbd880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd880744e0 0x7fbd880769f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbd94034000 tx=0x7fbd9402e000 comp rx=0 tx=0).stop 2024-04-25T11:13:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 -- 172.21.15.7:0/2587824664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdac0aed40 msgr2=0x7fbdac159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdac0aed40 0x7fbdac159110 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00b9f0 tx=0x7fbd9c00beb0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 -- 172.21.15.7:0/2587824664 shutdown_connections 2024-04-25T11:13:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd880744e0 0x7fbd880769f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdac0b5220 0x7fbdac0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdac0af420 0x7fbdac0d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 --2- 172.21.15.7:0/2587824664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdac0aed40 0x7fbdac159110 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 -- 172.21.15.7:0/2587824664 >> 172.21.15.7:0/2587824664 conn(0x7fbdac0b3820 msgr2=0x7fbdac0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 -- 172.21.15.7:0/2587824664 shutdown_connections 2024-04-25T11:13:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:38.804+0000 7fbdb1a00640 1 -- 172.21.15.7:0/2587824664 wait complete. 2024-04-25T11:13:38.833 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:13:38.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:13:39.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.116+0000 7f72b3200640 1 Processor -- start 2024-04-25T11:13:39.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.116+0000 7f72b3200640 1 -- start start 2024-04-25T11:13:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.118+0000 7f72b3200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72b41583c0 0x7f72b40cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.118+0000 7f72b3200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f72b40cfb50 0x7f72b40d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.118+0000 7f72b3200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b40ce570 con 0x7f72b40d09c0 2024-04-25T11:13:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.118+0000 7f72b3200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f72b415f190 con 0x7f72b41583c0 2024-04-25T11:13:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.118+0000 7f72b3200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f72b40ce850 con 0x7f72b40cfb50 2024-04-25T11:13:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.119+0000 7f72b1e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72b41583c0 0x7f72b40cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.119+0000 7f72b2800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f72b40d09c0 0x7f72b406eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52468/0 (socket says 172.21.15.7:52468) 2024-04-25T11:13:39.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.119+0000 7f72b2800640 1 -- 172.21.15.7:0/374559332 learned_addr learned my addr 172.21.15.7:0/374559332 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.119+0000 7f72b1e00640 1 -- 172.21.15.7:0/374559332 >> v2:172.21.15.87:3300/0 conn(0x7f72b40cfb50 msgr2=0x7f72b40d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.119+0000 7f72a9400640 1 --2- 172.21.15.7:0/374559332 >> v2:172.21.15.87:3300/0 conn(0x7f72b40cfb50 0x7f72b40d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.119+0000 7f72b1e00640 1 --2- 172.21.15.7:0/374559332 >> v2:172.21.15.87:3300/0 conn(0x7f72b40cfb50 0x7f72b40d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.119+0000 7f72b1e00640 1 -- 172.21.15.7:0/374559332 >> v1:172.21.15.7:6789/0 conn(0x7f72b40d09c0 legacy=0x7f72b406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.119+0000 7f72b1400640 1 -- 172.21.15.7:0/374559332 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178941260 0 0) 0x7f72b40ce570 con 0x7f72b40d09c0 2024-04-25T11:13:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.119+0000 7f72b1e00640 1 -- 172.21.15.7:0/374559332 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b40ceb30 con 0x7f72b41583c0 2024-04-25T11:13:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.119+0000 7f72a9400640 1 --2- 172.21.15.7:0/374559332 >> v2:172.21.15.87:3300/0 conn(0x7f72b40cfb50 0x7f72b40d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.121+0000 7f72b1e00640 1 --2- 172.21.15.7:0/374559332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72b41583c0 0x7f72b40cf610 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f72b4070ad0 tx=0x7f72a402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc863be9af79e4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.121+0000 7f72b1400640 1 -- 172.21.15.7:0/374559332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4037070 con 0x7f72b41583c0 2024-04-25T11:13:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.121+0000 7f72b1400640 1 -- 172.21.15.7:0/374559332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a4032440 con 0x7f72b41583c0 2024-04-25T11:13:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.121+0000 7f72b1400640 1 -- 172.21.15.7:0/374559332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a402ae60 con 0x7f72b41583c0 2024-04-25T11:13:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.121+0000 7f72b3200640 1 -- 172.21.15.7:0/374559332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72b41583c0 msgr2=0x7f72b40cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.121+0000 7f72b3200640 1 --2- 172.21.15.7:0/374559332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72b41583c0 0x7f72b40cf610 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f72b4070ad0 tx=0x7f72a402a110 comp rx=0 tx=0).stop 2024-04-25T11:13:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.122+0000 7f72b3200640 1 -- 172.21.15.7:0/374559332 shutdown_connections 2024-04-25T11:13:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.122+0000 7f72b3200640 1 --2- 172.21.15.7:0/374559332 >> v2:172.21.15.87:3300/0 conn(0x7f72b40cfb50 0x7f72b40d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.122+0000 7f72b3200640 1 --2- 172.21.15.7:0/374559332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72b41583c0 0x7f72b40cf610 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.122+0000 7f72b3200640 1 -- 172.21.15.7:0/374559332 >> 172.21.15.7:0/374559332 conn(0x7f72b40740f0 msgr2=0x7f72b406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.122+0000 7f72b3200640 1 -- 172.21.15.7:0/374559332 shutdown_connections 2024-04-25T11:13:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.122+0000 7f72b3200640 1 -- 172.21.15.7:0/374559332 wait complete. 2024-04-25T11:13:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.124+0000 7f72b3200640 1 Processor -- start 2024-04-25T11:13:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.124+0000 7f72b3200640 1 -- start start 2024-04-25T11:13:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.126+0000 7f72b3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72b40cfb50 0x7f72b4150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.126+0000 7f72b3200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72b40d09c0 0x7f72b41f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.126+0000 7f72b3200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72b41583c0 0x7f72b41e9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.126+0000 7f72b3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72b406f5b0 con 0x7f72b40cfb50 2024-04-25T11:13:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.126+0000 7f72b3200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f72b40cf0f0 con 0x7f72b40d09c0 2024-04-25T11:13:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.126+0000 7f72b3200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f72b406f730 con 0x7f72b41583c0 2024-04-25T11:13:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.126+0000 7f72b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72b40cfb50 0x7f72b4150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.126+0000 7f72b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72b40cfb50 0x7f72b4150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44430/0 (socket says 172.21.15.7:44430) 2024-04-25T11:13:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.126+0000 7f72b1e00640 1 -- 172.21.15.7:0/2487949805 learned_addr learned my addr 172.21.15.7:0/2487949805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.127+0000 7f72a9400640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72b40d09c0 0x7f72b41f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.127+0000 7f72b1e00640 1 -- 172.21.15.7:0/2487949805 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72b41583c0 msgr2=0x7f72b41e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.127+0000 7f72b1e00640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72b41583c0 0x7f72b41e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.127+0000 7f72b1e00640 1 -- 172.21.15.7:0/2487949805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72b40d09c0 msgr2=0x7f72b41f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.127+0000 7f72b1e00640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72b40d09c0 0x7f72b41f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.127+0000 7f72b1e00640 1 -- 172.21.15.7:0/2487949805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b4156020 con 0x7f72b40cfb50 2024-04-25T11:13:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.127+0000 7f72a9400640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72b40d09c0 0x7f72b41f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.127+0000 7f72b1e00640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72b40cfb50 0x7f72b4150240 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009890 tx=0x7f72a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.128+0000 7f72aaa00640 1 -- 172.21.15.7:0/2487949805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4037040 con 0x7f72b40cfb50 2024-04-25T11:13:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.128+0000 7f72aaa00640 1 -- 172.21.15.7:0/2487949805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a4042070 con 0x7f72b40cfb50 2024-04-25T11:13:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.128+0000 7f72aaa00640 1 -- 172.21.15.7:0/2487949805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4030bd0 con 0x7f72b40cfb50 2024-04-25T11:13:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.128+0000 7f72b3200640 1 -- 172.21.15.7:0/2487949805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b4156300 con 0x7f72b40cfb50 2024-04-25T11:13:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.128+0000 7f72b3200640 1 -- 172.21.15.7:0/2487949805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b41568c0 con 0x7f72b40cfb50 2024-04-25T11:13:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.129+0000 7f72b3200640 1 -- 172.21.15.7:0/2487949805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b4050740 con 0x7f72b40cfb50 2024-04-25T11:13:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.130+0000 7f72aaa00640 1 -- 172.21.15.7:0/2487949805 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f72a4047460 con 0x7f72b40cfb50 2024-04-25T11:13:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.131+0000 7f72aaa00640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72900744b0 0x7f72900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.131+0000 7f72aaa00640 1 -- 172.21.15.7:0/2487949805 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f72a40b5df0 con 0x7f72b40cfb50 2024-04-25T11:13:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.131+0000 7f72a9400640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72900744b0 0x7f72900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.132+0000 7f72a9400640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72900744b0 0x7f72900769c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f729c006fd0 tx=0x7f729c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.138+0000 7f72aaa00640 1 -- 172.21.15.7:0/2487949805 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72a4032e20 con 0x7f72b40cfb50 2024-04-25T11:13:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.344+0000 7f72b3200640 1 -- 172.21.15.7:0/2487949805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72b4070dd0 con 0x7f72b40cfb50 2024-04-25T11:13:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.345+0000 7f72aaa00640 1 -- 172.21.15.7:0/2487949805 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7f72a4080730 con 0x7f72b40cfb50 2024-04-25T11:13:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:13:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.347+0000 7f72b3200640 1 -- 172.21.15.7:0/2487949805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72900744b0 msgr2=0x7f72900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.347+0000 7f72b3200640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72900744b0 0x7f72900769c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f729c006fd0 tx=0x7f729c008040 comp rx=0 tx=0).stop 2024-04-25T11:13:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.347+0000 7f72b3200640 1 -- 172.21.15.7:0/2487949805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72b40cfb50 msgr2=0x7f72b4150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.347+0000 7f72b3200640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72b40cfb50 0x7f72b4150240 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009890 tx=0x7f72a40046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.348+0000 7f72b3200640 1 -- 172.21.15.7:0/2487949805 shutdown_connections 2024-04-25T11:13:39.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.348+0000 7f72b3200640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72900744b0 0x7f72900769c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.348+0000 7f72b3200640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72b41583c0 0x7f72b41e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.348+0000 7f72b3200640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72b40d09c0 0x7f72b41f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.348+0000 7f72b3200640 1 --2- 172.21.15.7:0/2487949805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72b40cfb50 0x7f72b4150240 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.348+0000 7f72b3200640 1 -- 172.21.15.7:0/2487949805 >> 172.21.15.7:0/2487949805 conn(0x7f72b40740f0 msgr2=0x7f72b406eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:39.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.348+0000 7f72b3200640 1 -- 172.21.15.7:0/2487949805 shutdown_connections 2024-04-25T11:13:39.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.348+0000 7f72b3200640 1 -- 172.21.15.7:0/2487949805 wait complete. 2024-04-25T11:13:39.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:13:39.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.654+0000 7f6721800640 1 Processor -- start 2024-04-25T11:13:39.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.654+0000 7f6721800640 1 -- start start 2024-04-25T11:13:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.657+0000 7f6721800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f671c15f180 0x7f671c161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.657+0000 7f6721800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f671c1621a0 0x7f671c06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.657+0000 7f6721800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f671c06ded0 con 0x7f671c06dac0 2024-04-25T11:13:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.657+0000 7f6721800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f671c15f590 con 0x7f671c15f180 2024-04-25T11:13:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.657+0000 7f6721800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f671c158ae0 con 0x7f671c1621a0 2024-04-25T11:13:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.657+0000 7f671aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f671c15f180 0x7f671c161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:39.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.658+0000 7f671a000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f671c1621a0 0x7f671c06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.658+0000 7f671b400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f671c06dac0 0x7f671c06f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52474/0 (socket says 172.21.15.7:52474) 2024-04-25T11:13:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.658+0000 7f671aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f671c15f180 0x7f671c161c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44440/0 (socket says 172.21.15.7:44440) 2024-04-25T11:13:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.658+0000 7f671b400640 1 -- 172.21.15.7:0/1283140440 learned_addr learned my addr 172.21.15.7:0/1283140440 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.658+0000 7f671aa00640 1 -- 172.21.15.7:0/1283140440 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f671c1621a0 msgr2=0x7f671c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.658+0000 7f671aa00640 1 --2- 172.21.15.7:0/1283140440 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f671c1621a0 0x7f671c06d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.658+0000 7f671aa00640 1 -- 172.21.15.7:0/1283140440 >> v1:172.21.15.7:6789/0 conn(0x7f671c06dac0 legacy=0x7f671c06f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.658+0000 7f671aa00640 1 -- 172.21.15.7:0/1283140440 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c162700 con 0x7f671c15f180 2024-04-25T11:13:39.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.658+0000 7f6719600640 1 -- 172.21.15.7:0/1283140440 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333599243 0 0) 0x7f6714001490 con 0x7f671c06dac0 2024-04-25T11:13:39.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.660+0000 7f671aa00640 1 --2- 172.21.15.7:0/1283140440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c15f180 0x7f671c161c60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f670c0097d0 tx=0x7f670c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2935cadaf05b61e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.661+0000 7f6719600640 1 -- 172.21.15.7:0/1283140440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c037070 con 0x7f671c15f180 2024-04-25T11:13:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.661+0000 7f6719600640 1 -- 172.21.15.7:0/1283140440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670c032440 con 0x7f671c15f180 2024-04-25T11:13:39.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.661+0000 7f6719600640 1 -- 172.21.15.7:0/1283140440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c032810 con 0x7f671c15f180 2024-04-25T11:13:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.662+0000 7f6721800640 1 -- 172.21.15.7:0/1283140440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c15f180 msgr2=0x7f671c161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.662+0000 7f6721800640 1 --2- 172.21.15.7:0/1283140440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c15f180 0x7f671c161c60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f670c0097d0 tx=0x7f670c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.662+0000 7f6721800640 1 -- 172.21.15.7:0/1283140440 shutdown_connections 2024-04-25T11:13:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.662+0000 7f6721800640 1 --2- 172.21.15.7:0/1283140440 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f671c1621a0 0x7f671c06d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.662+0000 7f6721800640 1 --2- 172.21.15.7:0/1283140440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c15f180 0x7f671c161c60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.662+0000 7f6721800640 1 -- 172.21.15.7:0/1283140440 >> 172.21.15.7:0/1283140440 conn(0x7f671c0732b0 msgr2=0x7f671c0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.662+0000 7f6721800640 1 -- 172.21.15.7:0/1283140440 shutdown_connections 2024-04-25T11:13:39.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.662+0000 7f6721800640 1 -- 172.21.15.7:0/1283140440 wait complete. 2024-04-25T11:13:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.666+0000 7f6721800640 1 Processor -- start 2024-04-25T11:13:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.666+0000 7f6721800640 1 -- start start 2024-04-25T11:13:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.668+0000 7f6721800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c06dac0 0x7f671c1f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.668+0000 7f6721800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f671c15f180 0x7f671c09d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.668+0000 7f6721800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c1621a0 0x7f671c1e9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.668+0000 7f6721800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f671c15ec60 con 0x7f671c06dac0 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.668+0000 7f6721800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f671c15e980 con 0x7f671c1621a0 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.668+0000 7f6721800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f671c15e670 con 0x7f671c15f180 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.669+0000 7f671aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c06dac0 0x7f671c1f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.669+0000 7f671aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c06dac0 0x7f671c1f3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44442/0 (socket says 172.21.15.7:44442) 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.669+0000 7f671aa00640 1 -- 172.21.15.7:0/2215394553 learned_addr learned my addr 172.21.15.7:0/2215394553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.669+0000 7f671a000640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f671c15f180 0x7f671c09d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.669+0000 7f671aa00640 1 -- 172.21.15.7:0/2215394553 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f671c15f180 msgr2=0x7f671c09d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.669+0000 7f671b400640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c1621a0 0x7f671c1e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.669+0000 7f671aa00640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f671c15f180 0x7f671c09d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.669+0000 7f671aa00640 1 -- 172.21.15.7:0/2215394553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c1621a0 msgr2=0x7f671c1e9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.669+0000 7f671aa00640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c1621a0 0x7f671c1e9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.669+0000 7f671aa00640 1 -- 172.21.15.7:0/2215394553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c09e5d0 con 0x7f671c06dac0 2024-04-25T11:13:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.670+0000 7f671aa00640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c06dac0 0x7f671c1f3dc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f670c0097d0 tx=0x7f670c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.670+0000 7f6712a00640 1 -- 172.21.15.7:0/2215394553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c037050 con 0x7f671c06dac0 2024-04-25T11:13:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.670+0000 7f6712a00640 1 -- 172.21.15.7:0/2215394553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670c038040 con 0x7f671c06dac0 2024-04-25T11:13:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.670+0000 7f6712a00640 1 -- 172.21.15.7:0/2215394553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c004470 con 0x7f671c06dac0 2024-04-25T11:13:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.670+0000 7f6721800640 1 -- 172.21.15.7:0/2215394553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c09e8b0 con 0x7f671c06dac0 2024-04-25T11:13:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.670+0000 7f6721800640 1 -- 172.21.15.7:0/2215394553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c09ee70 con 0x7f671c06dac0 2024-04-25T11:13:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.671+0000 7f6721800640 1 -- 172.21.15.7:0/2215394553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671c050740 con 0x7f671c06dac0 2024-04-25T11:13:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.673+0000 7f6712a00640 1 -- 172.21.15.7:0/2215394553 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f670c03c6d0 con 0x7f671c06dac0 2024-04-25T11:13:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.674+0000 7f6712a00640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66fc0744e0 0x7f66fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.674+0000 7f6712a00640 1 -- 172.21.15.7:0/2215394553 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f670c0b5b90 con 0x7f671c06dac0 2024-04-25T11:13:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.674+0000 7f671a000640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66fc0744e0 0x7f66fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.675+0000 7f671a000640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66fc0744e0 0x7f66fc0769f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f671c1ea390 tx=0x7f6708008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.681+0000 7f6712a00640 1 -- 172.21.15.7:0/2215394553 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f670c080440 con 0x7f671c06dac0 2024-04-25T11:13:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.888+0000 7f6721800640 1 -- 172.21.15.7:0/2215394553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f671c074250 con 0x7f671c06dac0 2024-04-25T11:13:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.889+0000 7f6712a00640 1 -- 172.21.15.7:0/2215394553 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) ==== 84+0+65 (secure 0 0 0) 0x7f670c07fd40 con 0x7f671c06dac0 2024-04-25T11:13:39.889 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:39.890 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-25T11:13:39.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:13:39.890 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-25T11:13:39.890 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-25T11:13:39.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:13:39.890 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-25T11:13:39.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 -- 172.21.15.7:0/2215394553 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66fc0744e0 msgr2=0x7f66fc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66fc0744e0 0x7f66fc0769f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f671c1ea390 tx=0x7f6708008040 comp rx=0 tx=0).stop 2024-04-25T11:13:39.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 -- 172.21.15.7:0/2215394553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c06dac0 msgr2=0x7f671c1f3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c06dac0 0x7f671c1f3dc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f670c0097d0 tx=0x7f670c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 -- 172.21.15.7:0/2215394553 shutdown_connections 2024-04-25T11:13:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66fc0744e0 0x7f66fc0769f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f671c1621a0 0x7f671c1e9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f671c15f180 0x7f671c09d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 --2- 172.21.15.7:0/2215394553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c06dac0 0x7f671c1f3dc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 -- 172.21.15.7:0/2215394553 >> 172.21.15.7:0/2215394553 conn(0x7f671c0732b0 msgr2=0x7f671c0b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 -- 172.21.15.7:0/2215394553 shutdown_connections 2024-04-25T11:13:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:39.892+0000 7f6721800640 1 -- 172.21.15.7:0/2215394553 wait complete. 2024-04-25T11:13:39.918 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:13:39.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:13:40.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.206+0000 7fcb55800640 1 Processor -- start 2024-04-25T11:13:40.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.206+0000 7fcb55800640 1 -- start start 2024-04-25T11:13:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.209+0000 7fcb55800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb50070110 0x7fcb5015c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.209+0000 7fcb55800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb50071360 0x7fcb500af070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.209+0000 7fcb55800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb500b4940 con 0x7fcb50070f50 2024-04-25T11:13:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.209+0000 7fcb55800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcb500b5550 con 0x7fcb50070110 2024-04-25T11:13:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.209+0000 7fcb55800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcb500b4c20 con 0x7fcb50071360 2024-04-25T11:13:40.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.209+0000 7fcb4ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb50070110 0x7fcb5015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.209+0000 7fcb4f400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb50071360 0x7fcb500af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.210+0000 7fcb4ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb50070110 0x7fcb5015c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:33146/0 (socket says 172.21.15.7:33146) 2024-04-25T11:13:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.210+0000 7fcb4ea00640 1 -- 172.21.15.7:0/2074193346 learned_addr learned my addr 172.21.15.7:0/2074193346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.210+0000 7fcb4f400640 1 -- 172.21.15.7:0/2074193346 >> v2:172.21.15.87:3300/0 conn(0x7fcb50070110 msgr2=0x7fcb5015c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.210+0000 7fcb4f400640 1 --2- 172.21.15.7:0/2074193346 >> v2:172.21.15.87:3300/0 conn(0x7fcb50070110 0x7fcb5015c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.210+0000 7fcb4f400640 1 -- 172.21.15.7:0/2074193346 >> v1:172.21.15.87:6789/0 conn(0x7fcb50070f50 legacy=0x7fcb500ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.210+0000 7fcb4f400640 1 -- 172.21.15.7:0/2074193346 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb500b4f00 con 0x7fcb50071360 2024-04-25T11:13:40.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.212+0000 7fcb4f400640 1 --2- 172.21.15.7:0/2074193346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb50071360 0x7fcb500af070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800aea0 tx=0x7fcb4802b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62559b70bb4b2d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:40.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.212+0000 7fcb4d600640 1 -- 172.21.15.7:0/2074193346 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48037070 con 0x7fcb50071360 2024-04-25T11:13:40.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.212+0000 7fcb4d600640 1 -- 172.21.15.7:0/2074193346 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb48008640 con 0x7fcb50071360 2024-04-25T11:13:40.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.212+0000 7fcb4d600640 1 -- 172.21.15.7:0/2074193346 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48008a10 con 0x7fcb50071360 2024-04-25T11:13:40.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.213+0000 7fcb55800640 1 -- 172.21.15.7:0/2074193346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb50071360 msgr2=0x7fcb500af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.213+0000 7fcb55800640 1 --2- 172.21.15.7:0/2074193346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb50071360 0x7fcb500af070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcb4800aea0 tx=0x7fcb4802b7b0 comp rx=0 tx=0).stop 2024-04-25T11:13:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.213+0000 7fcb55800640 1 -- 172.21.15.7:0/2074193346 shutdown_connections 2024-04-25T11:13:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.214+0000 7fcb55800640 1 --2- 172.21.15.7:0/2074193346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb50071360 0x7fcb500af070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.214+0000 7fcb55800640 1 --2- 172.21.15.7:0/2074193346 >> v2:172.21.15.87:3300/0 conn(0x7fcb50070110 0x7fcb5015c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.214+0000 7fcb55800640 1 -- 172.21.15.7:0/2074193346 >> 172.21.15.7:0/2074193346 conn(0x7fcb50073920 msgr2=0x7fcb50073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.214+0000 7fcb55800640 1 -- 172.21.15.7:0/2074193346 shutdown_connections 2024-04-25T11:13:40.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.214+0000 7fcb55800640 1 -- 172.21.15.7:0/2074193346 wait complete. 2024-04-25T11:13:40.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.217+0000 7fcb55800640 1 Processor -- start 2024-04-25T11:13:40.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.217+0000 7fcb55800640 1 -- start start 2024-04-25T11:13:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.219+0000 7fcb55800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb50070110 0x7fcb50142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.219+0000 7fcb55800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb50070f50 0x7fcb501494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.219+0000 7fcb55800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb50071360 0x7fcb500b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.219+0000 7fcb55800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb500af5b0 con 0x7fcb50071360 2024-04-25T11:13:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.219+0000 7fcb55800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcb500ce8a0 con 0x7fcb50070110 2024-04-25T11:13:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.219+0000 7fcb55800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcb500af730 con 0x7fcb50070f50 2024-04-25T11:13:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.219+0000 7fcb4f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb50071360 0x7fcb500b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.220+0000 7fcb4f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb50071360 0x7fcb500b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40934/0 (socket says 172.21.15.7:40934) 2024-04-25T11:13:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.220+0000 7fcb4f400640 1 -- 172.21.15.7:0/2598212079 learned_addr learned my addr 172.21.15.7:0/2598212079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.220+0000 7fcb4ea00640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb50070110 0x7fcb50142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.220+0000 7fcb4e000640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb50070f50 0x7fcb501494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.220+0000 7fcb4f400640 1 -- 172.21.15.7:0/2598212079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb50070f50 msgr2=0x7fcb501494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.220+0000 7fcb4f400640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb50070f50 0x7fcb501494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.220+0000 7fcb4f400640 1 -- 172.21.15.7:0/2598212079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb50070110 msgr2=0x7fcb50142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.220+0000 7fcb4f400640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb50070110 0x7fcb50142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.220+0000 7fcb4f400640 1 -- 172.21.15.7:0/2598212079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb50143770 con 0x7fcb50071360 2024-04-25T11:13:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.220+0000 7fcb4f400640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb50071360 0x7fcb500b2590 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcb4802bda0 tx=0x7fcb48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.221+0000 7fcb46a00640 1 -- 172.21.15.7:0/2598212079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48042020 con 0x7fcb50071360 2024-04-25T11:13:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.221+0000 7fcb46a00640 1 -- 172.21.15.7:0/2598212079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb48033740 con 0x7fcb50071360 2024-04-25T11:13:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.221+0000 7fcb55800640 1 -- 172.21.15.7:0/2598212079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb50143200 con 0x7fcb50071360 2024-04-25T11:13:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.221+0000 7fcb46a00640 1 -- 172.21.15.7:0/2598212079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb48037040 con 0x7fcb50071360 2024-04-25T11:13:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.221+0000 7fcb55800640 1 -- 172.21.15.7:0/2598212079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5014f070 con 0x7fcb50071360 2024-04-25T11:13:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.222+0000 7fcb55800640 1 -- 172.21.15.7:0/2598212079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5006a100 con 0x7fcb50071360 2024-04-25T11:13:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.224+0000 7fcb46a00640 1 -- 172.21.15.7:0/2598212079 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcb48045020 con 0x7fcb50071360 2024-04-25T11:13:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.225+0000 7fcb46a00640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb300744b0 0x7fcb300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.225+0000 7fcb46a00640 1 -- 172.21.15.7:0/2598212079 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7fcb480b5f00 con 0x7fcb50071360 2024-04-25T11:13:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.225+0000 7fcb4ea00640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb300744b0 0x7fcb300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.226+0000 7fcb4ea00640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb300744b0 0x7fcb300769c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009ac0 tx=0x7fcb40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:40.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.229+0000 7fcb46a00640 1 -- 172.21.15.7:0/2598212079 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcb4804c030 con 0x7fcb50071360 2024-04-25T11:13:40.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.428+0000 7fcb55800640 1 -- 172.21.15.7:0/2598212079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcb500ce4e0 con 0x7fcb50071360 2024-04-25T11:13:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.679+0000 7fcb46a00640 1 -- 172.21.15.7:0/2598212079 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) ==== 138+0+0 (secure 0 0 0) 0x7fcb50143770 con 0x7fcb50071360 2024-04-25T11:13:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-25T11:13:40.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.683+0000 7fcb55800640 1 -- 172.21.15.7:0/2598212079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb300744b0 msgr2=0x7fcb300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:40.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.683+0000 7fcb55800640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb300744b0 0x7fcb300769c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009ac0 tx=0x7fcb40008040 comp rx=0 tx=0).stop 2024-04-25T11:13:40.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.683+0000 7fcb55800640 1 -- 172.21.15.7:0/2598212079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb50071360 msgr2=0x7fcb500b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:40.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.683+0000 7fcb55800640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb50071360 0x7fcb500b2590 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcb4802bda0 tx=0x7fcb48004a40 comp rx=0 tx=0).stop 2024-04-25T11:13:40.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.684+0000 7fcb55800640 1 -- 172.21.15.7:0/2598212079 shutdown_connections 2024-04-25T11:13:40.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.684+0000 7fcb55800640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcb300744b0 0x7fcb300769c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:40.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.684+0000 7fcb55800640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb50071360 0x7fcb500b2590 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:40.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.684+0000 7fcb55800640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb50070f50 0x7fcb501494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:40.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.684+0000 7fcb55800640 1 --2- 172.21.15.7:0/2598212079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb50070110 0x7fcb50142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:40.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.684+0000 7fcb55800640 1 -- 172.21.15.7:0/2598212079 >> 172.21.15.7:0/2598212079 conn(0x7fcb50073920 msgr2=0x7fcb500687c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:40.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.684+0000 7fcb55800640 1 -- 172.21.15.7:0/2598212079 shutdown_connections 2024-04-25T11:13:40.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:40.684+0000 7fcb55800640 1 -- 172.21.15.7:0/2598212079 wait complete. 2024-04-25T11:13:40.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:13:41.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.010+0000 7f5385000640 1 Processor -- start 2024-04-25T11:13:41.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.010+0000 7f5385000640 1 -- start start 2024-04-25T11:13:41.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.012+0000 7f5385000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f538009f790 0x7f538015dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:41.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.012+0000 7f5385000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5380070d60 0x7f538006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:41.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.012+0000 7f5385000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f538006ded0 con 0x7f538006dac0 2024-04-25T11:13:41.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.012+0000 7f5385000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f538015e2a0 con 0x7f538009f790 2024-04-25T11:13:41.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.012+0000 7f5385000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5380074550 con 0x7f5380070d60 2024-04-25T11:13:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.013+0000 7f537de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f538009f790 0x7f538015dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.013+0000 7f537de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f538009f790 0x7f538015dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40938/0 (socket says 172.21.15.7:40938) 2024-04-25T11:13:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.013+0000 7f537de00640 1 -- 172.21.15.7:0/4126570041 learned_addr learned my addr 172.21.15.7:0/4126570041 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.013+0000 7f537d400640 1 --2- 172.21.15.7:0/4126570041 >> v2:172.21.15.87:3300/0 conn(0x7f5380070d60 0x7f538006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.014+0000 7f537de00640 1 -- 172.21.15.7:0/4126570041 >> v2:172.21.15.87:3300/0 conn(0x7f5380070d60 msgr2=0x7f538006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.014+0000 7f537de00640 1 --2- 172.21.15.7:0/4126570041 >> v2:172.21.15.87:3300/0 conn(0x7f5380070d60 0x7f538006d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.014+0000 7f537de00640 1 -- 172.21.15.7:0/4126570041 >> v1:172.21.15.87:6789/0 conn(0x7f538006dac0 legacy=0x7f53800ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.014+0000 7f537de00640 1 -- 172.21.15.7:0/4126570041 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538006f470 con 0x7f538009f790 2024-04-25T11:13:41.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.014+0000 7f537d400640 1 --2- 172.21.15.7:0/4126570041 >> v2:172.21.15.87:3300/0 conn(0x7f5380070d60 0x7f538006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:13:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.015+0000 7f537de00640 1 --2- 172.21.15.7:0/4126570041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f538009f790 0x7f538015dd60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f53800731d0 tx=0x7f537002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcad4071b211c5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.015+0000 7f537ca00640 1 -- 172.21.15.7:0/4126570041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370036070 con 0x7f538009f790 2024-04-25T11:13:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.015+0000 7f537ca00640 1 -- 172.21.15.7:0/4126570041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5370031600 con 0x7f538009f790 2024-04-25T11:13:41.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.015+0000 7f537ca00640 1 -- 172.21.15.7:0/4126570041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53700319d0 con 0x7f538009f790 2024-04-25T11:13:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.016+0000 7f5385000640 1 -- 172.21.15.7:0/4126570041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f538009f790 msgr2=0x7f538015dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.016+0000 7f5385000640 1 --2- 172.21.15.7:0/4126570041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f538009f790 0x7f538015dd60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f53800731d0 tx=0x7f537002a730 comp rx=0 tx=0).stop 2024-04-25T11:13:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.016+0000 7f5385000640 1 -- 172.21.15.7:0/4126570041 shutdown_connections 2024-04-25T11:13:41.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.016+0000 7f5385000640 1 --2- 172.21.15.7:0/4126570041 >> v2:172.21.15.87:3300/0 conn(0x7f5380070d60 0x7f538006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.016+0000 7f5385000640 1 --2- 172.21.15.7:0/4126570041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f538009f790 0x7f538015dd60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.016+0000 7f5385000640 1 -- 172.21.15.7:0/4126570041 >> 172.21.15.7:0/4126570041 conn(0x7f53800d1d40 msgr2=0x7f5380073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.016+0000 7f5385000640 1 -- 172.21.15.7:0/4126570041 shutdown_connections 2024-04-25T11:13:41.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.016+0000 7f5385000640 1 -- 172.21.15.7:0/4126570041 wait complete. 2024-04-25T11:13:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.019+0000 7f5385000640 1 Processor -- start 2024-04-25T11:13:41.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.019+0000 7f5385000640 1 -- start start 2024-04-25T11:13:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.020+0000 7f5385000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f538006dac0 0x7f5380160620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.020+0000 7f5385000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5380070d60 0x7f5380160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.020+0000 7f5385000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f538009f790 0x7f53801f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.020+0000 7f5385000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f538006fa30 con 0x7f5380070d60 2024-04-25T11:13:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.020+0000 7f5385000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5380070080 con 0x7f538009f790 2024-04-25T11:13:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.020+0000 7f5385000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f538006fd10 con 0x7f538006dac0 2024-04-25T11:13:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.021+0000 7f537d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5380070d60 0x7f5380160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.021+0000 7f537d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5380070d60 0x7f5380160b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40950/0 (socket says 172.21.15.7:40950) 2024-04-25T11:13:41.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.021+0000 7f537d400640 1 -- 172.21.15.7:0/120466630 learned_addr learned my addr 172.21.15.7:0/120466630 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.021+0000 7f537e800640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f538009f790 0x7f53801f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.021+0000 7f537d400640 1 -- 172.21.15.7:0/120466630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f538006dac0 msgr2=0x7f5380160620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.021+0000 7f537d400640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f538006dac0 0x7f5380160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.021+0000 7f537d400640 1 -- 172.21.15.7:0/120466630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f538009f790 msgr2=0x7f53801f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.021+0000 7f537d400640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f538009f790 0x7f53801f98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.021+0000 7f537d400640 1 -- 172.21.15.7:0/120466630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53800b47f0 con 0x7f5380070d60 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.022+0000 7f537d400640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5380070d60 0x7f5380160b60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f536c00d800 tx=0x7f536c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.022+0000 7f5376000640 1 -- 172.21.15.7:0/120466630 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c004030 con 0x7f5380070d60 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.022+0000 7f5376000640 1 -- 172.21.15.7:0/120466630 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f536c0041d0 con 0x7f5380070d60 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.022+0000 7f5385000640 1 -- 172.21.15.7:0/120466630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53800cc260 con 0x7f5380070d60 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.022+0000 7f5376000640 1 -- 172.21.15.7:0/120466630 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0096a0 con 0x7f5380070d60 2024-04-25T11:13:41.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.022+0000 7f5385000640 1 -- 172.21.15.7:0/120466630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538015fb40 con 0x7f5380070d60 2024-04-25T11:13:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.023+0000 7f5385000640 1 -- 172.21.15.7:0/120466630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53800cbcf0 con 0x7f5380070d60 2024-04-25T11:13:41.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.026+0000 7f5376000640 1 -- 172.21.15.7:0/120466630 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f536c009840 con 0x7f5380070d60 2024-04-25T11:13:41.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.027+0000 7f5376000640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53600744e0 0x7f53600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:41.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.027+0000 7f5376000640 1 -- 172.21.15.7:0/120466630 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5619+0+0 (secure 0 0 0) 0x7f536c095ff0 con 0x7f5380070d60 2024-04-25T11:13:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.028+0000 7f537de00640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53600744e0 0x7f53600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.029+0000 7f537de00640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53600744e0 0x7f53600769f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5370033000 tx=0x7f537002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.031+0000 7f5376000640 1 -- 172.21.15.7:0/120466630 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f536c060750 con 0x7f5380070d60 2024-04-25T11:13:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.245+0000 7f5385000640 1 -- 172.21.15.7:0/120466630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5380050740 con 0x7f5380070d60 2024-04-25T11:13:41.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:13:41.597 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:13:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.683+0000 7f5376000640 1 -- 172.21.15.7:0/120466630 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) ==== 170+0+0 (secure 0 0 0) 0x7f536c060050 con 0x7f5380070d60 2024-04-25T11:13:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-25T11:13:41.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.686+0000 7f5385000640 1 -- 172.21.15.7:0/120466630 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53600744e0 msgr2=0x7f53600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:41.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.686+0000 7f5385000640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53600744e0 0x7f53600769f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5370033000 tx=0x7f537002e000 comp rx=0 tx=0).stop 2024-04-25T11:13:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.686+0000 7f5385000640 1 -- 172.21.15.7:0/120466630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5380070d60 msgr2=0x7f5380160b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.686+0000 7f5385000640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5380070d60 0x7f5380160b60 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f536c00d800 tx=0x7f536c00dcc0 comp rx=0 tx=0).stop 2024-04-25T11:13:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.687+0000 7f5385000640 1 -- 172.21.15.7:0/120466630 shutdown_connections 2024-04-25T11:13:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.687+0000 7f5385000640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53600744e0 0x7f53600769f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.687+0000 7f5385000640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f538009f790 0x7f53801f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.687+0000 7f5385000640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5380070d60 0x7f5380160b60 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.687+0000 7f5385000640 1 --2- 172.21.15.7:0/120466630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f538006dac0 0x7f5380160620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.687+0000 7f5385000640 1 -- 172.21.15.7:0/120466630 >> 172.21.15.7:0/120466630 conn(0x7f53800d1d40 msgr2=0x7f538006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.687+0000 7f5385000640 1 -- 172.21.15.7:0/120466630 shutdown_connections 2024-04-25T11:13:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:41.687+0000 7f5385000640 1 -- 172.21.15.7:0/120466630 wait complete. 2024-04-25T11:13:41.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:13:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.038+0000 7fd52e800640 1 Processor -- start 2024-04-25T11:13:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.039+0000 7fd52e800640 1 -- start start 2024-04-25T11:13:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.041+0000 7fd52e800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd5280d0480 0x7fd5280d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.041+0000 7fd52e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd52806eea0 0x7fd52806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.041+0000 7fd52e800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5280ce570 con 0x7fd5281583c0 2024-04-25T11:13:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.041+0000 7fd52e800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd52806f7f0 con 0x7fd5280d0480 2024-04-25T11:13:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.041+0000 7fd52e800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd52815f190 con 0x7fd52806eea0 2024-04-25T11:13:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.042+0000 7fd52ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd5280d0480 0x7fd5280d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.042+0000 7fd52ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd5280d0480 0x7fd5280d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40960/0 (socket says 172.21.15.7:40960) 2024-04-25T11:13:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.042+0000 7fd52d400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd5281583c0 0x7fd5280cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47648/0 (socket says 172.21.15.7:47648) 2024-04-25T11:13:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.042+0000 7fd52ca00640 1 -- 172.21.15.7:0/444637632 learned_addr learned my addr 172.21.15.7:0/444637632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.042+0000 7fd52de00640 1 --2- 172.21.15.7:0/444637632 >> v2:172.21.15.87:3300/0 conn(0x7fd52806eea0 0x7fd52806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.042+0000 7fd52ca00640 1 -- 172.21.15.7:0/444637632 >> v2:172.21.15.87:3300/0 conn(0x7fd52806eea0 msgr2=0x7fd52806f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.042+0000 7fd52ca00640 1 --2- 172.21.15.7:0/444637632 >> v2:172.21.15.87:3300/0 conn(0x7fd52806eea0 0x7fd52806f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.042+0000 7fd52ca00640 1 -- 172.21.15.7:0/444637632 >> v1:172.21.15.7:6789/0 conn(0x7fd5281583c0 legacy=0x7fd5280cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.042+0000 7fd52ca00640 1 -- 172.21.15.7:0/444637632 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5280ce850 con 0x7fd5280d0480 2024-04-25T11:13:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.044+0000 7fd52ca00640 1 --2- 172.21.15.7:0/444637632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5280d0480 0x7fd5280d0890 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd5280d0e00 tx=0x7fd51402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2219db1316cf2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.044+0000 7fd527e00640 1 -- 172.21.15.7:0/444637632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514037070 con 0x7fd5280d0480 2024-04-25T11:13:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.044+0000 7fd527e00640 1 -- 172.21.15.7:0/444637632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd514032440 con 0x7fd5280d0480 2024-04-25T11:13:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.044+0000 7fd527e00640 1 -- 172.21.15.7:0/444637632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514032810 con 0x7fd5280d0480 2024-04-25T11:13:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.045+0000 7fd52e800640 1 -- 172.21.15.7:0/444637632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5280d0480 msgr2=0x7fd5280d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.045+0000 7fd52e800640 1 --2- 172.21.15.7:0/444637632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5280d0480 0x7fd5280d0890 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd5280d0e00 tx=0x7fd51402a1f0 comp rx=0 tx=0).stop 2024-04-25T11:13:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.045+0000 7fd52e800640 1 -- 172.21.15.7:0/444637632 shutdown_connections 2024-04-25T11:13:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.045+0000 7fd52e800640 1 --2- 172.21.15.7:0/444637632 >> v2:172.21.15.87:3300/0 conn(0x7fd52806eea0 0x7fd52806f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.045+0000 7fd52e800640 1 --2- 172.21.15.7:0/444637632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5280d0480 0x7fd5280d0890 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.045+0000 7fd52e800640 1 -- 172.21.15.7:0/444637632 >> 172.21.15.7:0/444637632 conn(0x7fd5280740f0 msgr2=0x7fd52806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.045+0000 7fd52e800640 1 -- 172.21.15.7:0/444637632 shutdown_connections 2024-04-25T11:13:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.045+0000 7fd52e800640 1 -- 172.21.15.7:0/444637632 wait complete. 2024-04-25T11:13:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.047+0000 7fd52e800640 1 Processor -- start 2024-04-25T11:13:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.047+0000 7fd52e800640 1 -- start start 2024-04-25T11:13:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.049+0000 7fd52e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52806eea0 0x7fd528152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.049+0000 7fd52e800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5280d0480 0x7fd528153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.049+0000 7fd52e800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5281583c0 0x7fd52814c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.049+0000 7fd52e800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280cf0f0 con 0x7fd52806eea0 2024-04-25T11:13:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.049+0000 7fd52e800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280cee10 con 0x7fd5281583c0 2024-04-25T11:13:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.049+0000 7fd52e800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd5280cfd20 con 0x7fd5280d0480 2024-04-25T11:13:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.049+0000 7fd52d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52806eea0 0x7fd528152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.049+0000 7fd52d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52806eea0 0x7fd528152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40968/0 (socket says 172.21.15.7:40968) 2024-04-25T11:13:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.049+0000 7fd52d400640 1 -- 172.21.15.7:0/3026599046 learned_addr learned my addr 172.21.15.7:0/3026599046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.050+0000 7fd52ca00640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5280d0480 0x7fd528153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.050+0000 7fd52de00640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5281583c0 0x7fd52814c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.050+0000 7fd52d400640 1 -- 172.21.15.7:0/3026599046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5280d0480 msgr2=0x7fd528153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.050+0000 7fd52d400640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5280d0480 0x7fd528153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.050+0000 7fd52d400640 1 -- 172.21.15.7:0/3026599046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5281583c0 msgr2=0x7fd52814c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.050+0000 7fd52d400640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5281583c0 0x7fd52814c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.050+0000 7fd52d400640 1 -- 172.21.15.7:0/3026599046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5281510e0 con 0x7fd52806eea0 2024-04-25T11:13:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.050+0000 7fd52d400640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52806eea0 0x7fd528152110 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c3c0 tx=0x7fd51c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.051+0000 7fd525600640 1 -- 172.21.15.7:0/3026599046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0074e0 con 0x7fd52806eea0 2024-04-25T11:13:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.051+0000 7fd525600640 1 -- 172.21.15.7:0/3026599046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51c007680 con 0x7fd52806eea0 2024-04-25T11:13:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.051+0000 7fd525600640 1 -- 172.21.15.7:0/3026599046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c004a00 con 0x7fd52806eea0 2024-04-25T11:13:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.051+0000 7fd52e800640 1 -- 172.21.15.7:0/3026599046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd528151350 con 0x7fd52806eea0 2024-04-25T11:13:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.051+0000 7fd52e800640 1 -- 172.21.15.7:0/3026599046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd528151910 con 0x7fd52806eea0 2024-04-25T11:13:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.052+0000 7fd52e800640 1 -- 172.21.15.7:0/3026599046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd528050740 con 0x7fd52806eea0 2024-04-25T11:13:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.054+0000 7fd525600640 1 -- 172.21.15.7:0/3026599046 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd51c01a020 con 0x7fd52806eea0 2024-04-25T11:13:42.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.055+0000 7fd525600640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd5080744e0 0x7fd5080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:42.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.055+0000 7fd525600640 1 -- 172.21.15.7:0/3026599046 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6001+0+0 (secure 0 0 0) 0x7fd51c094b00 con 0x7fd52806eea0 2024-04-25T11:13:42.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.055+0000 7fd52ca00640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd5080744e0 0x7fd5080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.056+0000 7fd52ca00640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd5080744e0 0x7fd5080769f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd514034000 tx=0x7fd51402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:42.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.061+0000 7fd525600640 1 -- 172.21.15.7:0/3026599046 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd51c05f1f0 con 0x7fd52806eea0 2024-04-25T11:13:42.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.307+0000 7fd52e800640 1 -- 172.21.15.7:0/3026599046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd5280b6780 con 0x7fd52806eea0 2024-04-25T11:13:42.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.727+0000 7fd525600640 1 -- 172.21.15.7:0/3026599046 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v23) ==== 172+0+0 (secure 0 0 0) 0x7fd51c05eaf0 con 0x7fd52806eea0 2024-04-25T11:13:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T11:13:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.733+0000 7fd52e800640 1 -- 172.21.15.7:0/3026599046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd5080744e0 msgr2=0x7fd5080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.733+0000 7fd52e800640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd5080744e0 0x7fd5080769f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd514034000 tx=0x7fd51402e000 comp rx=0 tx=0).stop 2024-04-25T11:13:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.733+0000 7fd52e800640 1 -- 172.21.15.7:0/3026599046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52806eea0 msgr2=0x7fd528152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.733+0000 7fd52e800640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52806eea0 0x7fd528152110 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c3c0 tx=0x7fd51c00c7c0 comp rx=0 tx=0).stop 2024-04-25T11:13:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.734+0000 7fd52e800640 1 -- 172.21.15.7:0/3026599046 shutdown_connections 2024-04-25T11:13:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.734+0000 7fd52e800640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd5080744e0 0x7fd5080769f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.734+0000 7fd52e800640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5281583c0 0x7fd52814c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.734+0000 7fd52e800640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5280d0480 0x7fd528153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.734+0000 7fd52e800640 1 --2- 172.21.15.7:0/3026599046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd52806eea0 0x7fd528152110 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.734+0000 7fd52e800640 1 -- 172.21.15.7:0/3026599046 >> 172.21.15.7:0/3026599046 conn(0x7fd5280740f0 msgr2=0x7fd52815e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.735+0000 7fd52e800640 1 -- 172.21.15.7:0/3026599046 shutdown_connections 2024-04-25T11:13:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:42.735+0000 7fd52e800640 1 -- 172.21.15.7:0/3026599046 wait complete. 2024-04-25T11:13:42.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:43.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.039+0000 7fd6d6a00640 1 Processor -- start 2024-04-25T11:13:43.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.039+0000 7fd6d6a00640 1 -- start start 2024-04-25T11:13:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.041+0000 7fd6d6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd6d006e510 0x7fd6d00b7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.041+0000 7fd6d6a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6d006e920 0x7fd6d00b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.041+0000 7fd6d6a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd6d00d0750 0x7fd6d00d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.041+0000 7fd6d6a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d00d4cc0 con 0x7fd6d00d0750 2024-04-25T11:13:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.041+0000 7fd6d6a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd6d00d4390 con 0x7fd6d006e510 2024-04-25T11:13:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.041+0000 7fd6d6a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d00d4670 con 0x7fd6d006e920 2024-04-25T11:13:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.042+0000 7fd6d6000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd6d00d0750 0x7fd6d00d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.042+0000 7fd6d6000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd6d00d0750 0x7fd6d00d0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40972/0 (socket says 172.21.15.7:40972) 2024-04-25T11:13:43.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.042+0000 7fd6d6000640 1 -- 172.21.15.7:0/3363844994 learned_addr learned my addr 172.21.15.7:0/3363844994 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.042+0000 7fd6d4c00640 1 --2- 172.21.15.7:0/3363844994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6d006e920 0x7fd6d00b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.042+0000 7fd6d5600640 1 --2- 172.21.15.7:0/3363844994 >> v2:172.21.15.87:3300/0 conn(0x7fd6d006e510 0x7fd6d00b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.042+0000 7fd6d6000640 1 -- 172.21.15.7:0/3363844994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6d006e920 msgr2=0x7fd6d00b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.042+0000 7fd6d6000640 1 --2- 172.21.15.7:0/3363844994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6d006e920 0x7fd6d00b7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.042+0000 7fd6d6000640 1 -- 172.21.15.7:0/3363844994 >> v2:172.21.15.87:3300/0 conn(0x7fd6d006e510 msgr2=0x7fd6d00b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.042+0000 7fd6d6000640 1 --2- 172.21.15.7:0/3363844994 >> v2:172.21.15.87:3300/0 conn(0x7fd6d006e510 0x7fd6d00b7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.042+0000 7fd6d6000640 1 -- 172.21.15.7:0/3363844994 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d00d4950 con 0x7fd6d00d0750 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.042+0000 7fd6d4c00640 1 --2- 172.21.15.7:0/3363844994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6d006e920 0x7fd6d00b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.043+0000 7fd6d6000640 1 --2- 172.21.15.7:0/3363844994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6d00d0750 0x7fd6d00d0b60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00d4880 tx=0x7fd6c002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b397eed4a9296bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.044+0000 7fd6cfe00640 1 -- 172.21.15.7:0/3363844994 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0037070 con 0x7fd6d00d0750 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.044+0000 7fd6cfe00640 1 -- 172.21.15.7:0/3363844994 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c0032440 con 0x7fd6d00d0750 2024-04-25T11:13:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.044+0000 7fd6cfe00640 1 -- 172.21.15.7:0/3363844994 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0032810 con 0x7fd6d00d0750 2024-04-25T11:13:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.045+0000 7fd6d6a00640 1 -- 172.21.15.7:0/3363844994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6d00d0750 msgr2=0x7fd6d00d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.045+0000 7fd6d6a00640 1 --2- 172.21.15.7:0/3363844994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6d00d0750 0x7fd6d00d0b60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00d4880 tx=0x7fd6c002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.045+0000 7fd6d6a00640 1 -- 172.21.15.7:0/3363844994 shutdown_connections 2024-04-25T11:13:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.045+0000 7fd6d6a00640 1 --2- 172.21.15.7:0/3363844994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6d00d0750 0x7fd6d00d0b60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.045+0000 7fd6d6a00640 1 --2- 172.21.15.7:0/3363844994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6d006e920 0x7fd6d00b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.045+0000 7fd6d6a00640 1 --2- 172.21.15.7:0/3363844994 >> v2:172.21.15.87:3300/0 conn(0x7fd6d006e510 0x7fd6d00b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.045+0000 7fd6d6a00640 1 -- 172.21.15.7:0/3363844994 >> 172.21.15.7:0/3363844994 conn(0x7fd6d00d27b0 msgr2=0x7fd6d00d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.045+0000 7fd6d6a00640 1 -- 172.21.15.7:0/3363844994 shutdown_connections 2024-04-25T11:13:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.045+0000 7fd6d6a00640 1 -- 172.21.15.7:0/3363844994 wait complete. 2024-04-25T11:13:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.047+0000 7fd6d6a00640 1 Processor -- start 2024-04-25T11:13:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.047+0000 7fd6d6a00640 1 -- start start 2024-04-25T11:13:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.049+0000 7fd6d6a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6d006e510 0x7fd6d0154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.049+0000 7fd6d6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6d006e920 0x7fd6d0157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.049+0000 7fd6d6a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6d00d0750 0x7fd6d0161c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.049+0000 7fd6d6a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d00d1520 con 0x7fd6d006e920 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.049+0000 7fd6d6a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d00d13a0 con 0x7fd6d006e510 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.049+0000 7fd6d6a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d00d5150 con 0x7fd6d00d0750 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.049+0000 7fd6d4c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6d006e920 0x7fd6d0157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.049+0000 7fd6d4c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6d006e920 0x7fd6d0157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40976/0 (socket says 172.21.15.7:40976) 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.049+0000 7fd6d4c00640 1 -- 172.21.15.7:0/989296414 learned_addr learned my addr 172.21.15.7:0/989296414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6d5600640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6d006e510 0x7fd6d0154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6d4c00640 1 -- 172.21.15.7:0/989296414 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6d00d0750 msgr2=0x7fd6d0161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6d4c00640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6d00d0750 0x7fd6d0161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6d4c00640 1 -- 172.21.15.7:0/989296414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6d006e510 msgr2=0x7fd6d0154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6d4c00640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6d006e510 0x7fd6d0154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6d4c00640 1 -- 172.21.15.7:0/989296414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d0158220 con 0x7fd6d006e920 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6d5600640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6d006e510 0x7fd6d0154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6d4c00640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6d006e920 0x7fd6d0157760 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c820 tx=0x7fd6bc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6cd600640 1 -- 172.21.15.7:0/989296414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc015070 con 0x7fd6d006e920 2024-04-25T11:13:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6cd600640 1 -- 172.21.15.7:0/989296414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6bc007a30 con 0x7fd6d006e920 2024-04-25T11:13:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6d6a00640 1 -- 172.21.15.7:0/989296414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d01f9c60 con 0x7fd6d006e920 2024-04-25T11:13:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6cd600640 1 -- 172.21.15.7:0/989296414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc007de0 con 0x7fd6d006e920 2024-04-25T11:13:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.050+0000 7fd6d6a00640 1 -- 172.21.15.7:0/989296414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d01fa220 con 0x7fd6d006e920 2024-04-25T11:13:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.052+0000 7fd6d6a00640 1 -- 172.21.15.7:0/989296414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d0157f90 con 0x7fd6d006e920 2024-04-25T11:13:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.053+0000 7fd6cd600640 1 -- 172.21.15.7:0/989296414 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd6bc018070 con 0x7fd6d006e920 2024-04-25T11:13:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.054+0000 7fd6cd600640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6b00744e0 0x7fd6b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.054+0000 7fd6cd600640 1 -- 172.21.15.7:0/989296414 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fd6bc0952c0 con 0x7fd6d006e920 2024-04-25T11:13:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.055+0000 7fd6d5600640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6b00744e0 0x7fd6b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.055+0000 7fd6d5600640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6b00744e0 0x7fd6b00769f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8007da0 tx=0x7fd6c80072f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.059+0000 7fd6cd600640 1 -- 172.21.15.7:0/989296414 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6bc05f8b0 con 0x7fd6d006e920 2024-04-25T11:13:43.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.261+0000 7fd6d6a00640 1 -- 172.21.15.7:0/989296414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6d0050740 con 0x7fd6d006e920 2024-04-25T11:13:43.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.263+0000 7fd6cd600640 1 -- 172.21.15.7:0/989296414 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16859 (secure 0 0 0) 0x7fd6bc05f1b0 con 0x7fd6d006e920 2024-04-25T11:13:43.265 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:43.265 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:42.310312+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:13:40.446828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T11:13:41.263101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 -- 172.21.15.7:0/989296414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6b00744e0 msgr2=0x7fd6b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6b00744e0 0x7fd6b00769f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8007da0 tx=0x7fd6c80072f0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 -- 172.21.15.7:0/989296414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6d006e920 msgr2=0x7fd6d0157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6d006e920 0x7fd6d0157760 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c820 tx=0x7fd6bc00cce0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 -- 172.21.15.7:0/989296414 shutdown_connections 2024-04-25T11:13:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6b00744e0 0x7fd6b00769f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6d00d0750 0x7fd6d0161c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6d006e920 0x7fd6d0157760 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 --2- 172.21.15.7:0/989296414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6d006e510 0x7fd6d0154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 -- 172.21.15.7:0/989296414 >> 172.21.15.7:0/989296414 conn(0x7fd6d00d27b0 msgr2=0x7fd6d00b8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 -- 172.21.15.7:0/989296414 shutdown_connections 2024-04-25T11:13:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.266+0000 7fd6d6a00640 1 -- 172.21.15.7:0/989296414 wait complete. 2024-04-25T11:13:43.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:43.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.550+0000 7fbd10000640 1 Processor -- start 2024-04-25T11:13:43.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.550+0000 7fbd10000640 1 -- start start 2024-04-25T11:13:43.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.552+0000 7fbd10000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd0806dc90 0x7fbd080afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.553+0000 7fbd10000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd080af780 con 0x7fbd0806f300 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.553+0000 7fbd10000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd080b4980 con 0x7fbd08160a30 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.553+0000 7fbd10000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbd080b4c60 con 0x7fbd0806dc90 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.553+0000 7fbd0cc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbd0806f300 0x7fbd080ae960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47660/0 (socket says 172.21.15.7:47660) 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.553+0000 7fbd0cc00640 1 -- 172.21.15.7:0/945115663 learned_addr learned my addr 172.21.15.7:0/945115663 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.553+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 117851076 0 0) 0x7fbd080af780 con 0x7fbd0806f300 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.553+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcf0005730 con 0x7fbd0806f300 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.553+0000 7fbd0d600640 1 --2- 172.21.15.7:0/945115663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd0806dc90 0x7fbd080afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3062330230 0 0) 0x7fbcf0005730 con 0x7fbd0806f300 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd0806dc90 msgr2=0x7fbd080afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 --2- 172.21.15.7:0/945115663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd0806dc90 0x7fbd080afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 >> v1:172.21.15.87:6789/0 conn(0x7fbd08160a30 legacy=0x7fbd080af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd080b4f40 con 0x7fbd0806f300 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fbcf8001670 con 0x7fbd0806f300 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 >> v1:172.21.15.7:6789/0 conn(0x7fbd0806f300 legacy=0x7fbd080ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 --2- 172.21.15.7:0/945115663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcf0005bf0 0x7fbcf0006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 --2- 172.21.15.7:0/945115663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcf0006a10 0x7fbcf0006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 --2- 172.21.15.7:0/945115663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf0007360 0x7fbcf000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd080b4c60 con 0x7fbcf0007360 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf000e210 con 0x7fbcf0005bf0 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf000e410 con 0x7fbcf0006a10 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.554+0000 7fbd0e000640 1 --2- 172.21.15.7:0/945115663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf0007360 0x7fbcf000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.555+0000 7fbd0d600640 1 --2- 172.21.15.7:0/945115663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcf0005bf0 0x7fbcf0006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.555+0000 7fbd0e000640 1 -- 172.21.15.7:0/945115663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcf0006a10 msgr2=0x7fbcf0006e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.555+0000 7fbd0e000640 1 --2- 172.21.15.7:0/945115663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcf0006a10 0x7fbcf0006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.555+0000 7fbd0e000640 1 -- 172.21.15.7:0/945115663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcf0005bf0 msgr2=0x7fbcf0006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.555+0000 7fbd0e000640 1 --2- 172.21.15.7:0/945115663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcf0005bf0 0x7fbcf0006060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.555+0000 7fbd0e000640 1 -- 172.21.15.7:0/945115663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf0005730 con 0x7fbcf0007360 2024-04-25T11:13:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.556+0000 7fbd0e000640 1 --2- 172.21.15.7:0/945115663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf0007360 0x7fbcf000dbd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbd080d18a0 tx=0x7fbcf0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6718c8edfbfd90cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.556+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00001a70 con 0x7fbcf0007360 2024-04-25T11:13:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.556+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd0002ae10 con 0x7fbcf0007360 2024-04-25T11:13:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.556+0000 7fbd07e00640 1 -- 172.21.15.7:0/945115663 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00003bb0 con 0x7fbcf0007360 2024-04-25T11:13:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.556+0000 7fbd10000640 1 -- 172.21.15.7:0/945115663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf0007360 msgr2=0x7fbcf000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.556+0000 7fbd10000640 1 --2- 172.21.15.7:0/945115663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcf0007360 0x7fbcf000dbd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbd080d18a0 tx=0x7fbcf0007a40 comp rx=0 tx=0).stop 2024-04-25T11:13:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.556+0000 7fbd0e000640 1 -- 172.21.15.7:0/945115663 reap_dead start 2024-04-25T11:13:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.556+0000 7fbd10000640 1 -- 172.21.15.7:0/945115663 shutdown_connections 2024-04-25T11:13:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.556+0000 7fbd10000640 1 -- 172.21.15.7:0/945115663 >> 172.21.15.7:0/945115663 conn(0x7fbd08074200 msgr2=0x7fbd0806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.557+0000 7fbd10000640 1 -- 172.21.15.7:0/945115663 shutdown_connections 2024-04-25T11:13:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.557+0000 7fbd10000640 1 -- 172.21.15.7:0/945115663 wait complete. 2024-04-25T11:13:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.559+0000 7fbd10000640 1 Processor -- start 2024-04-25T11:13:43.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.559+0000 7fbd10000640 1 -- start start 2024-04-25T11:13:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.560+0000 7fbd10000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcf0007360 0x7fbd0814c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.560+0000 7fbd10000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd0815fc00 0x7fbd081476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.560+0000 7fbd10000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd08150b50 0x7fbd08150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.560+0000 7fbd10000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0815d220 con 0x7fbd08150b50 2024-04-25T11:13:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd10000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbd080b5500 con 0x7fbd0815fc00 2024-04-25T11:13:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd10000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbd080b6a30 con 0x7fbcf0007360 2024-04-25T11:13:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd0e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd08150b50 0x7fbd08150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd0e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd08150b50 0x7fbd08150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41002/0 (socket says 172.21.15.7:41002) 2024-04-25T11:13:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd0e000640 1 -- 172.21.15.7:0/3749391008 learned_addr learned my addr 172.21.15.7:0/3749391008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd0d600640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcf0007360 0x7fbd0814c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd0cc00640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd0815fc00 0x7fbd081476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd0e000640 1 -- 172.21.15.7:0/3749391008 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcf0007360 msgr2=0x7fbd0814c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd0e000640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcf0007360 0x7fbd0814c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd0e000640 1 -- 172.21.15.7:0/3749391008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd0815fc00 msgr2=0x7fbd081476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd0e000640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd0815fc00 0x7fbd081476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.561+0000 7fbd0e000640 1 -- 172.21.15.7:0/3749391008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd081514a0 con 0x7fbd08150b50 2024-04-25T11:13:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.562+0000 7fbd0e000640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd08150b50 0x7fbd08150f60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbd00001f10 tx=0x7fbd000014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.562+0000 7fbd05600640 1 -- 172.21.15.7:0/3749391008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00037040 con 0x7fbd08150b50 2024-04-25T11:13:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.562+0000 7fbd05600640 1 -- 172.21.15.7:0/3749391008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd0002ac10 con 0x7fbd08150b50 2024-04-25T11:13:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.562+0000 7fbd05600640 1 -- 172.21.15.7:0/3749391008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00001a70 con 0x7fbd08150b50 2024-04-25T11:13:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.562+0000 7fbd10000640 1 -- 172.21.15.7:0/3749391008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd081491a0 con 0x7fbd08150b50 2024-04-25T11:13:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.562+0000 7fbd10000640 1 -- 172.21.15.7:0/3749391008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd08149760 con 0x7fbd08150b50 2024-04-25T11:13:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.563+0000 7fbd10000640 1 -- 172.21.15.7:0/3749391008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd08148f10 con 0x7fbd08150b50 2024-04-25T11:13:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.564+0000 7fbd05600640 1 -- 172.21.15.7:0/3749391008 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd00042020 con 0x7fbd08150b50 2024-04-25T11:13:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.565+0000 7fbd05600640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbcec0744b0 0x7fbcec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.565+0000 7fbd05600640 1 -- 172.21.15.7:0/3749391008 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fbd000b4b30 con 0x7fbd08150b50 2024-04-25T11:13:43.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.566+0000 7fbd0d600640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbcec0744b0 0x7fbcec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:43.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.566+0000 7fbd0d600640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbcec0744b0 0x7fbcec0769c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc006fd0 tx=0x7fbcfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:43.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.570+0000 7fbd05600640 1 -- 172.21.15.7:0/3749391008 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd081514a0 con 0x7fbd08150b50 2024-04-25T11:13:43.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.778+0000 7fbd10000640 1 -- 172.21.15.7:0/3749391008 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd080b25a0 con 0x7fbd08150b50 2024-04-25T11:13:43.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.779+0000 7fbd05600640 1 -- 172.21.15.7:0/3749391008 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16846 (secure 0 0 0) 0x7fbd0007f110 con 0x7fbd08150b50 2024-04-25T11:13:43.781 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:43.781 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:43.726826+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:13:40.446828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T11:13:41.263101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 -- 172.21.15.7:0/3749391008 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbcec0744b0 msgr2=0x7fbcec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbcec0744b0 0x7fbcec0769c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc006fd0 tx=0x7fbcfc008040 comp rx=0 tx=0).stop 2024-04-25T11:13:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 -- 172.21.15.7:0/3749391008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd08150b50 msgr2=0x7fbd08150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd08150b50 0x7fbd08150f60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbd00001f10 tx=0x7fbd000014a0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 -- 172.21.15.7:0/3749391008 shutdown_connections 2024-04-25T11:13:43.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbcec0744b0 0x7fbcec0769c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd08150b50 0x7fbd08150f60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd0815fc00 0x7fbd081476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 --2- 172.21.15.7:0/3749391008 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcf0007360 0x7fbd0814c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:43.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 -- 172.21.15.7:0/3749391008 >> 172.21.15.7:0/3749391008 conn(0x7fbd08074200 msgr2=0x7fbd0806fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:43.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 -- 172.21.15.7:0/3749391008 shutdown_connections 2024-04-25T11:13:43.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:43.783+0000 7fbd10000640 1 -- 172.21.15.7:0/3749391008 wait complete. 2024-04-25T11:13:43.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:13:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.078+0000 7fbeaf000640 1 Processor -- start 2024-04-25T11:13:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.078+0000 7fbeaf000640 1 -- start start 2024-04-25T11:13:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.080+0000 7fbeaf000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbea8145e40 0x7fbea8146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.080+0000 7fbeaf000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea80b4870 0x7fbea80b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeaf000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbea80b51c0 0x7fbea8143a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeaf000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbea80b5b30 con 0x7fbea80b51c0 2024-04-25T11:13:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeaf000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbea80b5e10 con 0x7fbea8145e40 2024-04-25T11:13:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeaf000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbea80b60f0 con 0x7fbea80b4870 2024-04-25T11:13:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeae600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbea80b51c0 0x7fbea8143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeae600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbea80b51c0 0x7fbea8143a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41008/0 (socket says 172.21.15.7:41008) 2024-04-25T11:13:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeae600640 1 -- 172.21.15.7:0/574633411 learned_addr learned my addr 172.21.15.7:0/574633411 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:44.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeadc00640 1 --2- 172.21.15.7:0/574633411 >> v2:172.21.15.87:3300/0 conn(0x7fbea8145e40 0x7fbea8146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:44.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbead200640 1 --2- 172.21.15.7:0/574633411 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea80b4870 0x7fbea80b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:44.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeae600640 1 -- 172.21.15.7:0/574633411 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea80b4870 msgr2=0x7fbea80b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:44.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeae600640 1 --2- 172.21.15.7:0/574633411 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea80b4870 0x7fbea80b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:44.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeae600640 1 -- 172.21.15.7:0/574633411 >> v2:172.21.15.87:3300/0 conn(0x7fbea8145e40 msgr2=0x7fbea8146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:44.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeae600640 1 --2- 172.21.15.7:0/574633411 >> v2:172.21.15.87:3300/0 conn(0x7fbea8145e40 0x7fbea8146250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:44.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbeae600640 1 -- 172.21.15.7:0/574633411 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea80b63d0 con 0x7fbea80b51c0 2024-04-25T11:13:44.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.081+0000 7fbead200640 1 --2- 172.21.15.7:0/574633411 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea80b4870 0x7fbea80b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:44.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.082+0000 7fbeae600640 1 --2- 172.21.15.7:0/574633411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea80b51c0 0x7fbea8143a70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbea806ded0 tx=0x7fbea002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefd6327defff941 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.083+0000 7fbea7e00640 1 -- 172.21.15.7:0/574633411 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0036070 con 0x7fbea80b51c0 2024-04-25T11:13:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.083+0000 7fbea7e00640 1 -- 172.21.15.7:0/574633411 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea0031600 con 0x7fbea80b51c0 2024-04-25T11:13:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.083+0000 7fbea7e00640 1 -- 172.21.15.7:0/574633411 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00319d0 con 0x7fbea80b51c0 2024-04-25T11:13:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.084+0000 7fbeaf000640 1 -- 172.21.15.7:0/574633411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea80b51c0 msgr2=0x7fbea8143a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.084+0000 7fbeaf000640 1 --2- 172.21.15.7:0/574633411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea80b51c0 0x7fbea8143a70 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbea806ded0 tx=0x7fbea002a610 comp rx=0 tx=0).stop 2024-04-25T11:13:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.084+0000 7fbeaf000640 1 -- 172.21.15.7:0/574633411 shutdown_connections 2024-04-25T11:13:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.084+0000 7fbeaf000640 1 --2- 172.21.15.7:0/574633411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea80b51c0 0x7fbea8143a70 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.084+0000 7fbeaf000640 1 --2- 172.21.15.7:0/574633411 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea80b4870 0x7fbea80b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.084+0000 7fbeaf000640 1 --2- 172.21.15.7:0/574633411 >> v2:172.21.15.87:3300/0 conn(0x7fbea8145e40 0x7fbea8146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:44.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.084+0000 7fbeaf000640 1 -- 172.21.15.7:0/574633411 >> 172.21.15.7:0/574633411 conn(0x7fbea80740f0 msgr2=0x7fbea806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:44.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.084+0000 7fbeaf000640 1 -- 172.21.15.7:0/574633411 shutdown_connections 2024-04-25T11:13:44.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.084+0000 7fbeaf000640 1 -- 172.21.15.7:0/574633411 wait complete. 2024-04-25T11:13:44.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.086+0000 7fbeaf000640 1 Processor -- start 2024-04-25T11:13:44.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.086+0000 7fbeaf000640 1 -- start start 2024-04-25T11:13:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.088+0000 7fbeaf000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea80b4870 0x7fbea81e82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.088+0000 7fbeaf000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea80b51c0 0x7fbea80aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.088+0000 7fbeaf000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8145e40 0x7fbea80aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.088+0000 7fbeaf000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8143fb0 con 0x7fbea8145e40 2024-04-25T11:13:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.088+0000 7fbeaf000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbea80b57e0 con 0x7fbea80b4870 2024-04-25T11:13:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.088+0000 7fbeaf000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbea8144130 con 0x7fbea80b51c0 2024-04-25T11:13:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.088+0000 7fbeae600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8145e40 0x7fbea80aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.088+0000 7fbeae600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8145e40 0x7fbea80aace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41010/0 (socket says 172.21.15.7:41010) 2024-04-25T11:13:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.088+0000 7fbeae600640 1 -- 172.21.15.7:0/2380048560 learned_addr learned my addr 172.21.15.7:0/2380048560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbeadc00640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea80b4870 0x7fbea81e82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbead200640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea80b51c0 0x7fbea80aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbeae600640 1 -- 172.21.15.7:0/2380048560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea80b51c0 msgr2=0x7fbea80aa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbeae600640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea80b51c0 0x7fbea80aa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbeae600640 1 -- 172.21.15.7:0/2380048560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea80b4870 msgr2=0x7fbea81e82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbeae600640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea80b4870 0x7fbea81e82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbeae600640 1 -- 172.21.15.7:0/2380048560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea80ac780 con 0x7fbea8145e40 2024-04-25T11:13:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbeae600640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8145e40 0x7fbea80aace0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbea002ac00 tx=0x7fbea0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:44.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbea5600640 1 -- 172.21.15.7:0/2380048560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea003f020 con 0x7fbea8145e40 2024-04-25T11:13:44.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbea5600640 1 -- 172.21.15.7:0/2380048560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea0037040 con 0x7fbea8145e40 2024-04-25T11:13:44.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbea5600640 1 -- 172.21.15.7:0/2380048560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0036070 con 0x7fbea8145e40 2024-04-25T11:13:44.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.089+0000 7fbeaf000640 1 -- 172.21.15.7:0/2380048560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea80aca60 con 0x7fbea8145e40 2024-04-25T11:13:44.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.090+0000 7fbeaf000640 1 -- 172.21.15.7:0/2380048560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea80ab820 con 0x7fbea8145e40 2024-04-25T11:13:44.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.091+0000 7fbeaf000640 1 -- 172.21.15.7:0/2380048560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea8050740 con 0x7fbea8145e40 2024-04-25T11:13:44.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.092+0000 7fbea5600640 1 -- 172.21.15.7:0/2380048560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbea0042020 con 0x7fbea8145e40 2024-04-25T11:13:44.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.093+0000 7fbea5600640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 0x7fbe880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:44.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.093+0000 7fbea5600640 1 -- 172.21.15.7:0/2380048560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fbea00b5a60 con 0x7fbea8145e40 2024-04-25T11:13:44.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.093+0000 7fbeadc00640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 0x7fbe880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:44.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.094+0000 7fbeadc00640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 0x7fbe880769c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c006fd0 tx=0x7fbe9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:44.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.097+0000 7fbea5600640 1 -- 172.21.15.7:0/2380048560 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbea0049030 con 0x7fbea8145e40 2024-04-25T11:13:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.331+0000 7fbeaf000640 1 -- 172.21.15.7:0/2380048560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbea80b1f20 con 0x7fbea8145e40 2024-04-25T11:13:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.751+0000 7fbea5600640 1 -- 172.21.15.7:0/2380048560 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7fbea007f970 con 0x7fbea8145e40 2024-04-25T11:13:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.754+0000 7fbeaf000640 1 -- 172.21.15.7:0/2380048560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 msgr2=0x7fbe880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.754+0000 7fbeaf000640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 0x7fbe880769c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c006fd0 tx=0x7fbe9c008040 comp rx=0 tx=0).stop 2024-04-25T11:13:44.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.754+0000 7fbeaf000640 1 -- 172.21.15.7:0/2380048560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8145e40 msgr2=0x7fbea80aace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:44.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.754+0000 7fbeaf000640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8145e40 0x7fbea80aace0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbea002ac00 tx=0x7fbea0004a40 comp rx=0 tx=0).stop 2024-04-25T11:13:44.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.754+0000 7fbeaf000640 1 -- 172.21.15.7:0/2380048560 shutdown_connections 2024-04-25T11:13:44.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.754+0000 7fbeaf000640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 0x7fbe880769c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:44.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.754+0000 7fbeaf000640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea8145e40 0x7fbea80aace0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:44.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.754+0000 7fbeaf000640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea80b51c0 0x7fbea80aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:44.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.754+0000 7fbeaf000640 1 --2- 172.21.15.7:0/2380048560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea80b4870 0x7fbea81e82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:44.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.754+0000 7fbeaf000640 1 -- 172.21.15.7:0/2380048560 >> 172.21.15.7:0/2380048560 conn(0x7fbea80740f0 msgr2=0x7fbea81e4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:44.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.755+0000 7fbeaf000640 1 -- 172.21.15.7:0/2380048560 shutdown_connections 2024-04-25T11:13:44.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:44.755+0000 7fbeaf000640 1 -- 172.21.15.7:0/2380048560 wait complete. 2024-04-25T11:13:44.789 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:13:44.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:13:45.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.074+0000 7f48eaa00640 1 Processor -- start 2024-04-25T11:13:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.074+0000 7f48eaa00640 1 -- start start 2024-04-25T11:13:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.077+0000 7f48eaa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e406fab0 0x7f48e4168e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.077+0000 7f48eaa00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48e415e3c0 con 0x7f48e40b0470 2024-04-25T11:13:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.077+0000 7f48eaa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48e40cf800 con 0x7f48e415dfb0 2024-04-25T11:13:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.077+0000 7f48eaa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48e40d1250 con 0x7f48e406fab0 2024-04-25T11:13:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.077+0000 7f48e3400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f48e40b0470 0x7f48e406f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47664/0 (socket says 172.21.15.7:47664) 2024-04-25T11:13:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.077+0000 7f48e3400640 1 -- 172.21.15.7:0/2656908551 learned_addr learned my addr 172.21.15.7:0/2656908551 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:45.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.078+0000 7f48e8a00640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e406fab0 0x7f48e4168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:45.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.078+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3535497042 0 0) 0x7f48e415e3c0 con 0x7f48e40b0470 2024-04-25T11:13:45.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.078+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48c8003520 con 0x7f48e40b0470 2024-04-25T11:13:45.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.078+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2643106420 0 0) 0x7f48c8003520 con 0x7f48e40b0470 2024-04-25T11:13:45.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.078+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e406fab0 msgr2=0x7f48e4168e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:45.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.078+0000 7f48e2a00640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e406fab0 0x7f48e4168e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:45.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.078+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 >> v1:172.21.15.87:6789/0 conn(0x7f48e415dfb0 legacy=0x7f48e40d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.078+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e40cfaa0 con 0x7f48e40b0470 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.078+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f48cc001670 con 0x7f48e40b0470 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.078+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 >> v1:172.21.15.7:6789/0 conn(0x7f48e40b0470 legacy=0x7f48e406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.079+0000 7f48e2a00640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 0x7f48c8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.079+0000 7f48e2a00640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48c8004800 0x7f48c8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.079+0000 7f48e2a00640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e406fab0 0x7f48c8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.079+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48e40d1250 con 0x7f48c80039e0 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.079+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48c800bb20 con 0x7f48c8004800 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.079+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48c800bd20 con 0x7f48e406fab0 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.079+0000 7f48e3e00640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 0x7f48c8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.079+0000 7f48e8a00640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e406fab0 0x7f48c8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.079+0000 7f48e3400640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48c8004800 0x7f48c8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.079+0000 7f48e3e00640 1 -- 172.21.15.7:0/2656908551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e406fab0 msgr2=0x7f48c8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.079+0000 7f48e3e00640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e406fab0 0x7f48c8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.080+0000 7f48e3e00640 1 -- 172.21.15.7:0/2656908551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48c8004800 msgr2=0x7f48c8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.080+0000 7f48e3e00640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48c8004800 0x7f48c8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.080+0000 7f48e3e00640 1 -- 172.21.15.7:0/2656908551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c8003520 con 0x7f48c80039e0 2024-04-25T11:13:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.081+0000 7f48e3e00640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 0x7f48c8003e50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f48e4072df0 tx=0x7f48c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c397465af2d4d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.081+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc001a70 con 0x7f48c80039e0 2024-04-25T11:13:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.081+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48dc02ae10 con 0x7f48c80039e0 2024-04-25T11:13:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.081+0000 7f48e2a00640 1 -- 172.21.15.7:0/2656908551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc003bb0 con 0x7f48c80039e0 2024-04-25T11:13:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.082+0000 7f48eaa00640 1 -- 172.21.15.7:0/2656908551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 msgr2=0x7f48c8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.082+0000 7f48eaa00640 1 --2- 172.21.15.7:0/2656908551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 0x7f48c8003e50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f48e4072df0 tx=0x7f48c8005830 comp rx=0 tx=0).stop 2024-04-25T11:13:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.082+0000 7f48e8a00640 1 -- 172.21.15.7:0/2656908551 reap_dead start 2024-04-25T11:13:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.082+0000 7f48eaa00640 1 -- 172.21.15.7:0/2656908551 shutdown_connections 2024-04-25T11:13:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.082+0000 7f48eaa00640 1 -- 172.21.15.7:0/2656908551 >> 172.21.15.7:0/2656908551 conn(0x7f48e40742c0 msgr2=0x7f48e406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.082+0000 7f48eaa00640 1 -- 172.21.15.7:0/2656908551 shutdown_connections 2024-04-25T11:13:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.082+0000 7f48eaa00640 1 -- 172.21.15.7:0/2656908551 wait complete. 2024-04-25T11:13:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.084+0000 7f48eaa00640 1 Processor -- start 2024-04-25T11:13:45.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.085+0000 7f48eaa00640 1 -- start start 2024-04-25T11:13:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.086+0000 7f48eaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 0x7f48e4160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.086+0000 7f48eaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151000 0x7f48e41f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.086+0000 7f48eaa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e4151410 0x7f48e41f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.086+0000 7f48eaa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48e40d0340 con 0x7f48c80039e0 2024-04-25T11:13:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.086+0000 7f48eaa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48e40d0060 con 0x7f48e4151000 2024-04-25T11:13:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.086+0000 7f48eaa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48e40d0620 con 0x7f48e4151410 2024-04-25T11:13:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.086+0000 7f48e3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 0x7f48e4160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.086+0000 7f48e3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 0x7f48e4160760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41030/0 (socket says 172.21.15.7:41030) 2024-04-25T11:13:45.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.086+0000 7f48e3e00640 1 -- 172.21.15.7:0/975554384 learned_addr learned my addr 172.21.15.7:0/975554384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.087+0000 7f48e8a00640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e4151410 0x7f48e41f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.087+0000 7f48e3400640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151000 0x7f48e41f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.087+0000 7f48e3e00640 1 -- 172.21.15.7:0/975554384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e4151410 msgr2=0x7f48e41f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.087+0000 7f48e3e00640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e4151410 0x7f48e41f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.087+0000 7f48e3e00640 1 -- 172.21.15.7:0/975554384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151000 msgr2=0x7f48e41f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.087+0000 7f48e3e00640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151000 0x7f48e41f7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.087+0000 7f48e3e00640 1 -- 172.21.15.7:0/975554384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e4154fd0 con 0x7f48c80039e0 2024-04-25T11:13:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.087+0000 7f48e3e00640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 0x7f48e4160760 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f48dc02a8c0 tx=0x7f48dc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.088+0000 7f48dbe00640 1 -- 172.21.15.7:0/975554384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc03f030 con 0x7f48c80039e0 2024-04-25T11:13:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.088+0000 7f48dbe00640 1 -- 172.21.15.7:0/975554384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48dc02ac10 con 0x7f48c80039e0 2024-04-25T11:13:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.088+0000 7f48dbe00640 1 -- 172.21.15.7:0/975554384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc037040 con 0x7f48c80039e0 2024-04-25T11:13:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.088+0000 7f48eaa00640 1 -- 172.21.15.7:0/975554384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e4161100 con 0x7f48c80039e0 2024-04-25T11:13:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.088+0000 7f48eaa00640 1 -- 172.21.15.7:0/975554384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e41e7d90 con 0x7f48c80039e0 2024-04-25T11:13:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.089+0000 7f48eaa00640 1 -- 172.21.15.7:0/975554384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e40cf3e0 con 0x7f48c80039e0 2024-04-25T11:13:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.090+0000 7f48dbe00640 1 -- 172.21.15.7:0/975554384 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f48dc042070 con 0x7f48c80039e0 2024-04-25T11:13:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.091+0000 7f48dbe00640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48c00744b0 0x7f48c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.091+0000 7f48dbe00640 1 -- 172.21.15.7:0/975554384 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f48dc0b5270 con 0x7f48c80039e0 2024-04-25T11:13:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.092+0000 7f48e3400640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48c00744b0 0x7f48c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:45.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.092+0000 7f48e3400640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48c00744b0 0x7f48c00769c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f48cc001050 tx=0x7f48cc001d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.096+0000 7f48dbe00640 1 -- 172.21.15.7:0/975554384 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48e40cf3e0 con 0x7f48c80039e0 2024-04-25T11:13:45.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.339+0000 7f48eaa00640 1 -- 172.21.15.7:0/975554384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f48e4050740 con 0x7f48c80039e0 2024-04-25T11:13:45.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.768+0000 7f48dbe00640 1 -- 172.21.15.7:0/975554384 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f48dc07f8e0 con 0x7f48c80039e0 2024-04-25T11:13:45.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.772+0000 7f48eaa00640 1 -- 172.21.15.7:0/975554384 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48c00744b0 msgr2=0x7f48c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.772+0000 7f48eaa00640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48c00744b0 0x7f48c00769c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f48cc001050 tx=0x7f48cc001d70 comp rx=0 tx=0).stop 2024-04-25T11:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.772+0000 7f48eaa00640 1 -- 172.21.15.7:0/975554384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 msgr2=0x7f48e4160760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.772+0000 7f48eaa00640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 0x7f48e4160760 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f48dc02a8c0 tx=0x7f48dc0014a0 comp rx=0 tx=0).stop 2024-04-25T11:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.772+0000 7f48eaa00640 1 -- 172.21.15.7:0/975554384 shutdown_connections 2024-04-25T11:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.772+0000 7f48eaa00640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48c00744b0 0x7f48c00769c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.772+0000 7f48eaa00640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e4151410 0x7f48e41f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.772+0000 7f48eaa00640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e4151000 0x7f48e41f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.772+0000 7f48eaa00640 1 --2- 172.21.15.7:0/975554384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48c80039e0 0x7f48e4160760 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.772+0000 7f48eaa00640 1 -- 172.21.15.7:0/975554384 >> 172.21.15.7:0/975554384 conn(0x7f48e40742c0 msgr2=0x7f48e406d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.773+0000 7f48eaa00640 1 -- 172.21.15.7:0/975554384 shutdown_connections 2024-04-25T11:13:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:45.773+0000 7f48eaa00640 1 -- 172.21.15.7:0/975554384 wait complete. 2024-04-25T11:13:45.806 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:13:45.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:13:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.091+0000 7f73a7e00640 1 Processor -- start 2024-04-25T11:13:46.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.092+0000 7f73a7e00640 1 -- start start 2024-04-25T11:13:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.093+0000 7f73a7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a80aed40 0x7f73a815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.094+0000 7f73a7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f73a80af420 0x7f73a80af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.094+0000 7f73a7e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73a80cebb0 con 0x7f73a80b5220 2024-04-25T11:13:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.094+0000 7f73a7e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f73a815e1f0 con 0x7f73a80af420 2024-04-25T11:13:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.094+0000 7f73a7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f73a80afd70 con 0x7f73a80aed40 2024-04-25T11:13:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.094+0000 7f739fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f73a80b5220 0x7f73a80ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47670/0 (socket says 172.21.15.7:47670) 2024-04-25T11:13:46.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.094+0000 7f739fe00640 1 -- 172.21.15.7:0/3122597060 learned_addr learned my addr 172.21.15.7:0/3122597060 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:46.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.094+0000 7f73a6a00640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a80aed40 0x7f73a815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.094+0000 7f73a7400640 1 --2- 172.21.15.7:0/3122597060 >> v2:172.21.15.87:3300/0 conn(0x7f73a80af420 0x7f73a80af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.094+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237013861 0 0) 0x7f7390001110 con 0x7f73a80b5220 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.094+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f738c003520 con 0x7f73a80b5220 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3134163764 0 0) 0x7f738c003520 con 0x7f73a80b5220 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a80aed40 msgr2=0x7f73a815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a80aed40 0x7f73a815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 >> v2:172.21.15.87:3300/0 conn(0x7f73a80af420 msgr2=0x7f73a80af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 --2- 172.21.15.7:0/3122597060 >> v2:172.21.15.87:3300/0 conn(0x7f73a80af420 0x7f73a80af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a80b64b0 con 0x7f73a80b5220 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f7390001840 con 0x7f73a80b5220 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 >> v1:172.21.15.7:6789/0 conn(0x7f73a80b5220 legacy=0x7f73a80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738c0039b0 0x7f738c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a80aed40 0x7f738c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 0x7f738c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73a815e1f0 con 0x7f738c004d40 2024-04-25T11:13:46.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f73a80afd70 con 0x7f738c0039b0 2024-04-25T11:13:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f738c00baa0 con 0x7f73a80aed40 2024-04-25T11:13:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.095+0000 7f73a7400640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 0x7f738c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.096+0000 7f73a6a00640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738c0039b0 0x7f738c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.096+0000 7f739fe00640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a80aed40 0x7f738c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.096+0000 7f73a7400640 1 -- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a80aed40 msgr2=0x7f738c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.096+0000 7f73a7400640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a80aed40 0x7f738c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.096+0000 7f73a7400640 1 -- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738c0039b0 msgr2=0x7f738c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.096+0000 7f73a7400640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738c0039b0 0x7f738c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.096+0000 7f73a7400640 1 -- 172.21.15.7:0/3122597060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c003520 con 0x7f738c004d40 2024-04-25T11:13:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.096+0000 7f73a6a00640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f738c0039b0 0x7f738c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:46.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.097+0000 7f73a7400640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 0x7f738c005150 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f73a0007360 tx=0x7f738c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dc674878925ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:46.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.097+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a0009be0 con 0x7f738c004d40 2024-04-25T11:13:46.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.097+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a002e620 con 0x7f738c004d40 2024-04-25T11:13:46.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.097+0000 7f73a6000640 1 -- 172.21.15.7:0/3122597060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a002e9f0 con 0x7f738c004d40 2024-04-25T11:13:46.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.098+0000 7f73a7e00640 1 -- 172.21.15.7:0/3122597060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 msgr2=0x7f738c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:46.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.098+0000 7f73a7e00640 1 --2- 172.21.15.7:0/3122597060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 0x7f738c005150 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f73a0007360 tx=0x7f738c005830 comp rx=0 tx=0).stop 2024-04-25T11:13:46.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.098+0000 7f73a7400640 1 -- 172.21.15.7:0/3122597060 reap_dead start 2024-04-25T11:13:46.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.098+0000 7f73a7e00640 1 -- 172.21.15.7:0/3122597060 shutdown_connections 2024-04-25T11:13:46.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.098+0000 7f73a7e00640 1 -- 172.21.15.7:0/3122597060 >> 172.21.15.7:0/3122597060 conn(0x7f73a80b3820 msgr2=0x7f73a80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:46.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.098+0000 7f73a7e00640 1 -- 172.21.15.7:0/3122597060 shutdown_connections 2024-04-25T11:13:46.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.098+0000 7f73a7e00640 1 -- 172.21.15.7:0/3122597060 wait complete. 2024-04-25T11:13:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.100+0000 7f73a7e00640 1 Processor -- start 2024-04-25T11:13:46.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.100+0000 7f73a7e00640 1 -- start start 2024-04-25T11:13:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.102+0000 7f73a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 0x7f73a815d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.102+0000 7f73a7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a815a450 0x7f73a8145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.102+0000 7f73a7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a8147d00 0x7f73a81486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.102+0000 7f73a7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73a806cbd0 con 0x7f738c004d40 2024-04-25T11:13:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.102+0000 7f73a7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f73a806c8f0 con 0x7f73a8147d00 2024-04-25T11:13:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.102+0000 7f73a7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f73a806ceb0 con 0x7f73a815a450 2024-04-25T11:13:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.102+0000 7f73a6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 0x7f73a815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.103+0000 7f73a6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 0x7f73a815d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41050/0 (socket says 172.21.15.7:41050) 2024-04-25T11:13:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.103+0000 7f73a6a00640 1 -- 172.21.15.7:0/4290489818 learned_addr learned my addr 172.21.15.7:0/4290489818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.103+0000 7f739fe00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a815a450 0x7f73a8145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.103+0000 7f73a7400640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a8147d00 0x7f73a81486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.103+0000 7f73a6a00640 1 -- 172.21.15.7:0/4290489818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a815a450 msgr2=0x7f73a8145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:46.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.103+0000 7f73a6a00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a815a450 0x7f73a8145c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.103+0000 7f73a6a00640 1 -- 172.21.15.7:0/4290489818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a8147d00 msgr2=0x7f73a81486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:46.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.103+0000 7f73a6a00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a8147d00 0x7f73a81486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.103+0000 7f73a6a00640 1 -- 172.21.15.7:0/4290489818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a815d570 con 0x7f738c004d40 2024-04-25T11:13:46.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.103+0000 7f73a6a00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 0x7f73a815d030 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f73980108b0 tx=0x7f7398010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:46.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.104+0000 7f739ea00640 1 -- 172.21.15.7:0/4290489818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739800f040 con 0x7f738c004d40 2024-04-25T11:13:46.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.104+0000 7f739ea00640 1 -- 172.21.15.7:0/4290489818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f739800eb60 con 0x7f738c004d40 2024-04-25T11:13:46.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.104+0000 7f739ea00640 1 -- 172.21.15.7:0/4290489818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739801c400 con 0x7f738c004d40 2024-04-25T11:13:46.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.104+0000 7f73a7e00640 1 -- 172.21.15.7:0/4290489818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a8146c60 con 0x7f738c004d40 2024-04-25T11:13:46.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.104+0000 7f73a7e00640 1 -- 172.21.15.7:0/4290489818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a8147220 con 0x7f738c004d40 2024-04-25T11:13:46.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.105+0000 7f73a7e00640 1 -- 172.21.15.7:0/4290489818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a8050740 con 0x7f738c004d40 2024-04-25T11:13:46.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.106+0000 7f739ea00640 1 -- 172.21.15.7:0/4290489818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f739801c5a0 con 0x7f738c004d40 2024-04-25T11:13:46.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.107+0000 7f739ea00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73840744b0 0x7f73840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.107+0000 7f739ea00640 1 -- 172.21.15.7:0/4290489818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f7398095b00 con 0x7f738c004d40 2024-04-25T11:13:46.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.107+0000 7f739fe00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73840744b0 0x7f73840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.108+0000 7f739fe00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73840744b0 0x7f73840769c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f73900019b0 tx=0x7f7390009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.112+0000 7f739ea00640 1 -- 172.21.15.7:0/4290489818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f73a815d570 con 0x7f738c004d40 2024-04-25T11:13:46.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.353+0000 7f73a7e00640 1 -- 172.21.15.7:0/4290489818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73a80d0510 con 0x7f738c004d40 2024-04-25T11:13:46.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.355+0000 7f739ea00640 1 -- 172.21.15.7:0/4290489818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5172 (secure 0 0 0) 0x7f7398018030 con 0x7f738c004d40 2024-04-25T11:13:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:46.356 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2225639104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2225639104},{"type":"v1","addr":"172.21.15.87:6834","nonce":2225639104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2625900548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2625900548},{"type":"v1","addr":"172.21.15.7:6837","nonce":2625900548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/717352569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":717352569},{"type":"v1","addr":"172.21.15.87:6835","nonce":717352569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:13:42.310229+0000","modified":"2024-04-25T11:13:45.341610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6836/3503942462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3503942462},{"type":"v1","addr":"172.21.15.7:6836","nonce":3503942462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-25T11:13:46.357 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-25T11:13:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 -- 172.21.15.7:0/4290489818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73840744b0 msgr2=0x7f73840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73840744b0 0x7f73840769c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f73900019b0 tx=0x7f7390009040 comp rx=0 tx=0).stop 2024-04-25T11:13:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 -- 172.21.15.7:0/4290489818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 msgr2=0x7f73a815d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 0x7f73a815d030 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f73980108b0 tx=0x7f7398010d70 comp rx=0 tx=0).stop 2024-04-25T11:13:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 -- 172.21.15.7:0/4290489818 shutdown_connections 2024-04-25T11:13:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73840744b0 0x7f73840769c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73a8147d00 0x7f73a81486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73a815a450 0x7f73a8145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 --2- 172.21.15.7:0/4290489818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f738c004d40 0x7f73a815d030 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 -- 172.21.15.7:0/4290489818 >> 172.21.15.7:0/4290489818 conn(0x7f73a80b3820 msgr2=0x7f73a80ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 -- 172.21.15.7:0/4290489818 shutdown_connections 2024-04-25T11:13:46.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.358+0000 7f73a7e00640 1 -- 172.21.15.7:0/4290489818 wait complete. 2024-04-25T11:13:46.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.653+0000 7fd282a00640 1 Processor -- start 2024-04-25T11:13:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.654+0000 7fd282a00640 1 -- start start 2024-04-25T11:13:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.656+0000 7fd282a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd27c156cd0 0x7fd27c163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.656+0000 7fd282a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd27c1570e0 0x7fd27c164010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.656+0000 7fd282a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd27c0b5080 con 0x7fd27c164690 2024-04-25T11:13:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.656+0000 7fd282a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd27c0b5360 con 0x7fd27c156cd0 2024-04-25T11:13:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.656+0000 7fd282a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd27c0b5640 con 0x7fd27c1570e0 2024-04-25T11:13:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.657+0000 7fd280c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd27c156cd0 0x7fd27c163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.657+0000 7fd280c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd27c156cd0 0x7fd27c163ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41062/0 (socket says 172.21.15.7:41062) 2024-04-25T11:13:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.657+0000 7fd280c00640 1 -- 172.21.15.7:0/3348527198 learned_addr learned my addr 172.21.15.7:0/3348527198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.657+0000 7fd282000640 1 --2- 172.21.15.7:0/3348527198 >> v2:172.21.15.87:3300/0 conn(0x7fd27c1570e0 0x7fd27c164010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.657+0000 7fd280c00640 1 -- 172.21.15.7:0/3348527198 >> v2:172.21.15.87:3300/0 conn(0x7fd27c1570e0 msgr2=0x7fd27c164010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.657+0000 7fd280c00640 1 --2- 172.21.15.7:0/3348527198 >> v2:172.21.15.87:3300/0 conn(0x7fd27c1570e0 0x7fd27c164010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.657+0000 7fd280c00640 1 -- 172.21.15.7:0/3348527198 >> v1:172.21.15.87:6789/0 conn(0x7fd27c164690 legacy=0x7fd27c070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.657+0000 7fd280c00640 1 -- 172.21.15.7:0/3348527198 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c0b5920 con 0x7fd27c156cd0 2024-04-25T11:13:46.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.658+0000 7fd280c00640 1 --2- 172.21.15.7:0/3348527198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd27c156cd0 0x7fd27c163ad0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd27c06f8d0 tx=0x7fd26802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad9ab102576480f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:46.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.659+0000 7fd27be00640 1 -- 172.21.15.7:0/3348527198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268037070 con 0x7fd27c156cd0 2024-04-25T11:13:46.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.659+0000 7fd27be00640 1 -- 172.21.15.7:0/3348527198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd268032440 con 0x7fd27c156cd0 2024-04-25T11:13:46.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.659+0000 7fd27be00640 1 -- 172.21.15.7:0/3348527198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268032810 con 0x7fd27c156cd0 2024-04-25T11:13:46.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.659+0000 7fd282a00640 1 -- 172.21.15.7:0/3348527198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd27c156cd0 msgr2=0x7fd27c163ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:46.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.659+0000 7fd282a00640 1 --2- 172.21.15.7:0/3348527198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd27c156cd0 0x7fd27c163ad0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd27c06f8d0 tx=0x7fd26802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.659+0000 7fd282a00640 1 -- 172.21.15.7:0/3348527198 shutdown_connections 2024-04-25T11:13:46.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.659+0000 7fd282a00640 1 --2- 172.21.15.7:0/3348527198 >> v2:172.21.15.87:3300/0 conn(0x7fd27c1570e0 0x7fd27c164010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.659+0000 7fd282a00640 1 --2- 172.21.15.7:0/3348527198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd27c156cd0 0x7fd27c163ad0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.659+0000 7fd282a00640 1 -- 172.21.15.7:0/3348527198 >> 172.21.15.7:0/3348527198 conn(0x7fd27c0b3820 msgr2=0x7fd27c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:46.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.659+0000 7fd282a00640 1 -- 172.21.15.7:0/3348527198 shutdown_connections 2024-04-25T11:13:46.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.660+0000 7fd282a00640 1 -- 172.21.15.7:0/3348527198 wait complete. 2024-04-25T11:13:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.662+0000 7fd282a00640 1 Processor -- start 2024-04-25T11:13:46.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.662+0000 7fd282a00640 1 -- start start 2024-04-25T11:13:46.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.663+0000 7fd282a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd27c156cd0 0x7fd27c15e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.663+0000 7fd282a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd27c1570e0 0x7fd27c14ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.663+0000 7fd282a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd27c164690 0x7fd27c1f9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.663+0000 7fd282a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c0b0390 con 0x7fd27c1570e0 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.663+0000 7fd282a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c0b0210 con 0x7fd27c156cd0 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.663+0000 7fd282a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd27c0b0510 con 0x7fd27c164690 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.664+0000 7fd280c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd27c1570e0 0x7fd27c14ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.664+0000 7fd280c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd27c1570e0 0x7fd27c14ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41074/0 (socket says 172.21.15.7:41074) 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.664+0000 7fd280c00640 1 -- 172.21.15.7:0/2794306877 learned_addr learned my addr 172.21.15.7:0/2794306877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.664+0000 7fd282000640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd27c164690 0x7fd27c1f9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.664+0000 7fd281600640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd27c156cd0 0x7fd27c15e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.664+0000 7fd280c00640 1 -- 172.21.15.7:0/2794306877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd27c164690 msgr2=0x7fd27c1f9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.664+0000 7fd280c00640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd27c164690 0x7fd27c1f9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.664+0000 7fd280c00640 1 -- 172.21.15.7:0/2794306877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd27c156cd0 msgr2=0x7fd27c15e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:46.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.664+0000 7fd280c00640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd27c156cd0 0x7fd27c15e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.664+0000 7fd280c00640 1 -- 172.21.15.7:0/2794306877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c158740 con 0x7fd27c1570e0 2024-04-25T11:13:46.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.665+0000 7fd280c00640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd27c1570e0 0x7fd27c14ece0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd268000c00 tx=0x7fd2680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.665+0000 7fd279600640 1 -- 172.21.15.7:0/2794306877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268037040 con 0x7fd27c1570e0 2024-04-25T11:13:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.665+0000 7fd279600640 1 -- 172.21.15.7:0/2794306877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd268042070 con 0x7fd27c1570e0 2024-04-25T11:13:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.665+0000 7fd279600640 1 -- 172.21.15.7:0/2794306877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268030b90 con 0x7fd27c1570e0 2024-04-25T11:13:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.665+0000 7fd282a00640 1 -- 172.21.15.7:0/2794306877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27c158a20 con 0x7fd27c1570e0 2024-04-25T11:13:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.665+0000 7fd282a00640 1 -- 172.21.15.7:0/2794306877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27c15ed80 con 0x7fd27c1570e0 2024-04-25T11:13:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.666+0000 7fd282a00640 1 -- 172.21.15.7:0/2794306877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27c050740 con 0x7fd27c1570e0 2024-04-25T11:13:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.668+0000 7fd279600640 1 -- 172.21.15.7:0/2794306877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd268047460 con 0x7fd27c1570e0 2024-04-25T11:13:46.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.668+0000 7fd279600640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd25c0744b0 0x7fd25c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:46.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.669+0000 7fd279600640 1 -- 172.21.15.7:0/2794306877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fd2680b6110 con 0x7fd27c1570e0 2024-04-25T11:13:46.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.669+0000 7fd281600640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd25c0744b0 0x7fd25c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:46.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.669+0000 7fd281600640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd25c0744b0 0x7fd25c0769c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd27000a730 tx=0x7fd270008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:46.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.674+0000 7fd279600640 1 -- 172.21.15.7:0/2794306877 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd268080710 con 0x7fd27c1570e0 2024-04-25T11:13:46.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.882+0000 7fd282a00640 1 -- 172.21.15.7:0/2794306877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd27c06f930 con 0x7fd27c1570e0 2024-04-25T11:13:46.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.886+0000 7fd279600640 1 -- 172.21.15.7:0/2794306877 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16846 (secure 0 0 0) 0x7fd268080010 con 0x7fd27c1570e0 2024-04-25T11:13:46.888 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:46.888 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:43.726826+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:13:40.446828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T11:13:41.263101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 -- 172.21.15.7:0/2794306877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd25c0744b0 msgr2=0x7fd25c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd25c0744b0 0x7fd25c0769c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd27000a730 tx=0x7fd270008040 comp rx=0 tx=0).stop 2024-04-25T11:13:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 -- 172.21.15.7:0/2794306877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd27c1570e0 msgr2=0x7fd27c14ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:46.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd27c1570e0 0x7fd27c14ece0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd268000c00 tx=0x7fd2680046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 -- 172.21.15.7:0/2794306877 shutdown_connections 2024-04-25T11:13:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd25c0744b0 0x7fd25c0769c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd27c164690 0x7fd27c1f9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd27c1570e0 0x7fd27c14ece0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 --2- 172.21.15.7:0/2794306877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd27c156cd0 0x7fd27c15e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 -- 172.21.15.7:0/2794306877 >> 172.21.15.7:0/2794306877 conn(0x7fd27c0b3820 msgr2=0x7fd27c06ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 -- 172.21.15.7:0/2794306877 shutdown_connections 2024-04-25T11:13:46.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:46.889+0000 7fd282a00640 1 -- 172.21.15.7:0/2794306877 wait complete. 2024-04-25T11:13:46.917 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:13:46.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.185+0000 7fc76fe00640 1 Processor -- start 2024-04-25T11:13:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.185+0000 7fc76fe00640 1 -- start start 2024-04-25T11:13:47.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.187+0000 7fc76fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700b5220 0x7fc77015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.187+0000 7fc76fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc7700af420 0x7fc7700af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.187+0000 7fc76fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7700cebb0 con 0x7fc7700aed40 2024-04-25T11:13:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.188+0000 7fc76fe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc77015e1f0 con 0x7fc7700af420 2024-04-25T11:13:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.188+0000 7fc76fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc7700afd70 con 0x7fc7700b5220 2024-04-25T11:13:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.188+0000 7fc76f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc7700af420 0x7fc7700af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.188+0000 7fc76e000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700b5220 0x7fc77015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.188+0000 7fc76e000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700b5220 0x7fc77015dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:48896/0 (socket says 172.21.15.7:48896) 2024-04-25T11:13:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.188+0000 7fc76e000640 1 -- 172.21.15.7:0/434326250 learned_addr learned my addr 172.21.15.7:0/434326250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.189+0000 7fc76e000640 1 -- 172.21.15.7:0/434326250 >> v2:172.21.15.87:3300/0 conn(0x7fc7700af420 msgr2=0x7fc7700af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.189+0000 7fc76e000640 1 --2- 172.21.15.7:0/434326250 >> v2:172.21.15.87:3300/0 conn(0x7fc7700af420 0x7fc7700af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.189+0000 7fc76e000640 1 -- 172.21.15.7:0/434326250 >> v1:172.21.15.87:6789/0 conn(0x7fc7700aed40 legacy=0x7fc7700ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.189+0000 7fc76e000640 1 -- 172.21.15.7:0/434326250 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7700b64b0 con 0x7fc7700b5220 2024-04-25T11:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.189+0000 7fc76f400640 1 --2- 172.21.15.7:0/434326250 >> v2:172.21.15.87:3300/0 conn(0x7fc7700af420 0x7fc7700af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:13:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.191+0000 7fc76e000640 1 --2- 172.21.15.7:0/434326250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700b5220 0x7fc77015dcb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009870 tx=0x7fc75c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da1a301ef235362b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.191+0000 7fc76d600640 1 -- 172.21.15.7:0/434326250 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c037070 con 0x7fc7700b5220 2024-04-25T11:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.191+0000 7fc76d600640 1 -- 172.21.15.7:0/434326250 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c032440 con 0x7fc7700b5220 2024-04-25T11:13:47.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.191+0000 7fc76d600640 1 -- 172.21.15.7:0/434326250 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c032810 con 0x7fc7700b5220 2024-04-25T11:13:47.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.192+0000 7fc76fe00640 1 -- 172.21.15.7:0/434326250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700b5220 msgr2=0x7fc77015dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:47.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.192+0000 7fc76fe00640 1 --2- 172.21.15.7:0/434326250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700b5220 0x7fc77015dcb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009870 tx=0x7fc75c02a250 comp rx=0 tx=0).stop 2024-04-25T11:13:47.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.192+0000 7fc76fe00640 1 -- 172.21.15.7:0/434326250 shutdown_connections 2024-04-25T11:13:47.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.192+0000 7fc76fe00640 1 --2- 172.21.15.7:0/434326250 >> v2:172.21.15.87:3300/0 conn(0x7fc7700af420 0x7fc7700af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.192+0000 7fc76fe00640 1 --2- 172.21.15.7:0/434326250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700b5220 0x7fc77015dcb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.192+0000 7fc76fe00640 1 -- 172.21.15.7:0/434326250 >> 172.21.15.7:0/434326250 conn(0x7fc7700b3820 msgr2=0x7fc7700b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:47.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.192+0000 7fc76fe00640 1 -- 172.21.15.7:0/434326250 shutdown_connections 2024-04-25T11:13:47.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.192+0000 7fc76fe00640 1 -- 172.21.15.7:0/434326250 wait complete. 2024-04-25T11:13:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.195+0000 7fc76fe00640 1 Processor -- start 2024-04-25T11:13:47.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.195+0000 7fc76fe00640 1 -- start start 2024-04-25T11:13:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.196+0000 7fc76fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700aed40 0x7fc770149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.196+0000 7fc76fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700af420 0x7fc77009de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.196+0000 7fc76fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700b5220 0x7fc770160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.196+0000 7fc76fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc77006cbd0 con 0x7fc7700aed40 2024-04-25T11:13:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.196+0000 7fc76fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc77006c8f0 con 0x7fc7700b5220 2024-04-25T11:13:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.196+0000 7fc76fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc77006ceb0 con 0x7fc7700af420 2024-04-25T11:13:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.197+0000 7fc76ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700aed40 0x7fc770149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.197+0000 7fc76ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700aed40 0x7fc770149860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41082/0 (socket says 172.21.15.7:41082) 2024-04-25T11:13:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.197+0000 7fc76ea00640 1 -- 172.21.15.7:0/3297200475 learned_addr learned my addr 172.21.15.7:0/3297200475 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.197+0000 7fc76f400640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700b5220 0x7fc770160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.197+0000 7fc76e000640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700af420 0x7fc77009de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.197+0000 7fc76ea00640 1 -- 172.21.15.7:0/3297200475 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700af420 msgr2=0x7fc77009de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.197+0000 7fc76ea00640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700af420 0x7fc77009de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.197+0000 7fc76ea00640 1 -- 172.21.15.7:0/3297200475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700b5220 msgr2=0x7fc770160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.197+0000 7fc76ea00640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700b5220 0x7fc770160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.197+0000 7fc76ea00640 1 -- 172.21.15.7:0/3297200475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77009e3b0 con 0x7fc7700aed40 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.197+0000 7fc76f400640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700b5220 0x7fc770160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.198+0000 7fc76ea00640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700aed40 0x7fc770149860 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc76000c3c0 tx=0x7fc76000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.198+0000 7fc766a00640 1 -- 172.21.15.7:0/3297200475 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760017070 con 0x7fc7700aed40 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.198+0000 7fc766a00640 1 -- 172.21.15.7:0/3297200475 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7600052f0 con 0x7fc7700aed40 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.198+0000 7fc766a00640 1 -- 172.21.15.7:0/3297200475 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760003830 con 0x7fc7700aed40 2024-04-25T11:13:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.198+0000 7fc76fe00640 1 -- 172.21.15.7:0/3297200475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77014a090 con 0x7fc7700aed40 2024-04-25T11:13:47.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.198+0000 7fc76fe00640 1 -- 172.21.15.7:0/3297200475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77014a650 con 0x7fc7700aed40 2024-04-25T11:13:47.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.199+0000 7fc76fe00640 1 -- 172.21.15.7:0/3297200475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc770050740 con 0x7fc7700aed40 2024-04-25T11:13:47.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.201+0000 7fc766a00640 1 -- 172.21.15.7:0/3297200475 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc76001a070 con 0x7fc7700aed40 2024-04-25T11:13:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.201+0000 7fc766a00640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7500744b0 0x7fc7500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.202+0000 7fc766a00640 1 -- 172.21.15.7:0/3297200475 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc760094760 con 0x7fc7700aed40 2024-04-25T11:13:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.202+0000 7fc76e000640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7500744b0 0x7fc7500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.203+0000 7fc76e000640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7500744b0 0x7fc7500769c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc75c034000 tx=0x7fc75c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:47.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.207+0000 7fc766a00640 1 -- 172.21.15.7:0/3297200475 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc7700cebb0 con 0x7fc7700aed40 2024-04-25T11:13:47.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.403+0000 7fc76fe00640 1 -- 172.21.15.7:0/3297200475 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc77006dd00 con 0x7fc7500744b0 2024-04-25T11:13:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.423+0000 7fc766a00640 1 -- 172.21.15.7:0/3297200475 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246803 (secure 0 0 0) 0x7fc75c032840 con 0x7fc7500744b0 2024-04-25T11:13:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:47.429 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:13:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.428+0000 7fc76fe00640 1 -- 172.21.15.7:0/3297200475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7500744b0 msgr2=0x7fc7500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.428+0000 7fc76fe00640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7500744b0 0x7fc7500769c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc75c034000 tx=0x7fc75c02e000 comp rx=0 tx=0).stop 2024-04-25T11:13:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.428+0000 7fc76fe00640 1 -- 172.21.15.7:0/3297200475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700aed40 msgr2=0x7fc770149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.428+0000 7fc76fe00640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700aed40 0x7fc770149860 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc76000c3c0 tx=0x7fc76000c880 comp rx=0 tx=0).stop 2024-04-25T11:13:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.429+0000 7fc76fe00640 1 -- 172.21.15.7:0/3297200475 shutdown_connections 2024-04-25T11:13:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.429+0000 7fc76fe00640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc7500744b0 0x7fc7500769c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.429+0000 7fc76fe00640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700b5220 0x7fc770160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.429+0000 7fc76fe00640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7700af420 0x7fc77009de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.429+0000 7fc76fe00640 1 --2- 172.21.15.7:0/3297200475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7700aed40 0x7fc770149860 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.429+0000 7fc76fe00640 1 -- 172.21.15.7:0/3297200475 >> 172.21.15.7:0/3297200475 conn(0x7fc7700b3820 msgr2=0x7fc7700ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:47.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.429+0000 7fc76fe00640 1 -- 172.21.15.7:0/3297200475 shutdown_connections 2024-04-25T11:13:47.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.429+0000 7fc76fe00640 1 -- 172.21.15.7:0/3297200475 wait complete. 2024-04-25T11:13:47.459 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-04-25T11:13:45.861156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592036,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":131,"num_write_kb":1639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":142,"acting":142,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242948,"kb_used_data":5952,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731772,"statfs":{"total":773094113280,"available":772845334528,"internally_reserved":0,"allocated":6094848,"data_stored":3325104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.179785"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.918077+0000","last_change":"2024-04-25T11:13:42.737280+0000","last_active":"2024-04-25T11:13:43.918077+0000","last_peered":"2024-04-25T11:13:43.918077+0000","last_clean":"2024-04-25T11:13:43.918077+0000","last_became_active":"2024-04-25T11:13:42.736721+0000","last_became_peered":"2024-04-25T11:13:42.736721+0000","last_unstale":"2024-04-25T11:13:43.918077+0000","last_undegraded":"2024-04-25T11:13:43.918077+0000","last_fullsized":"2024-04-25T11:13:43.918077+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:57.039186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763435+0000","last_change":"2024-04-25T11:13:42.757285+0000","last_active":"2024-04-25T11:13:43.763435+0000","last_peered":"2024-04-25T11:13:43.763435+0000","last_clean":"2024-04-25T11:13:43.763435+0000","last_became_active":"2024-04-25T11:13:42.756624+0000","last_became_peered":"2024-04-25T11:13:42.756624+0000","last_unstale":"2024-04-25T11:13:43.763435+0000","last_undegraded":"2024-04-25T11:13:43.763435+0000","last_fullsized":"2024-04-25T11:13:43.763435+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:55.417276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914607+0000","last_change":"2024-04-25T11:13:42.738461+0000","last_active":"2024-04-25T11:13:43.914607+0000","last_peered":"2024-04-25T11:13:43.914607+0000","last_clean":"2024-04-25T11:13:43.914607+0000","last_became_active":"2024-04-25T11:13:42.736638+0000","last_became_peered":"2024-04-25T11:13:42.736638+0000","last_unstale":"2024-04-25T11:13:43.914607+0000","last_undegraded":"2024-04-25T11:13:43.914607+0000","last_fullsized":"2024-04-25T11:13:43.914607+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:16.546115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.762942+0000","last_change":"2024-04-25T11:13:42.754847+0000","last_active":"2024-04-25T11:13:43.762942+0000","last_peered":"2024-04-25T11:13:43.762942+0000","last_clean":"2024-04-25T11:13:43.762942+0000","last_became_active":"2024-04-25T11:13:42.754513+0000","last_became_peered":"2024-04-25T11:13:42.754513+0000","last_unstale":"2024-04-25T11:13:43.762942+0000","last_undegraded":"2024-04-25T11:13:43.762942+0000","last_fullsized":"2024-04-25T11:13:43.762942+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:17.667799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.913603+0000","last_change":"2024-04-25T11:13:42.749655+0000","last_active":"2024-04-25T11:13:43.913603+0000","last_peered":"2024-04-25T11:13:43.913603+0000","last_clean":"2024-04-25T11:13:43.913603+0000","last_became_active":"2024-04-25T11:13:42.748078+0000","last_became_peered":"2024-04-25T11:13:42.748078+0000","last_unstale":"2024-04-25T11:13:43.913603+0000","last_undegraded":"2024-04-25T11:13:43.913603+0000","last_fullsized":"2024-04-25T11:13:43.913603+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:12.515416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915516+0000","last_change":"2024-04-25T11:13:42.775520+0000","last_active":"2024-04-25T11:13:43.915516+0000","last_peered":"2024-04-25T11:13:43.915516+0000","last_clean":"2024-04-25T11:13:43.915516+0000","last_became_active":"2024-04-25T11:13:42.775179+0000","last_became_peered":"2024-04-25T11:13:42.775179+0000","last_unstale":"2024-04-25T11:13:43.915516+0000","last_undegraded":"2024-04-25T11:13:43.915516+0000","last_fullsized":"2024-04-25T11:13:43.915516+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:38.398871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.917770+0000","last_change":"2024-04-25T11:13:42.751944+0000","last_active":"2024-04-25T11:13:43.917770+0000","last_peered":"2024-04-25T11:13:43.917770+0000","last_clean":"2024-04-25T11:13:43.917770+0000","last_became_active":"2024-04-25T11:13:42.751643+0000","last_became_peered":"2024-04-25T11:13:42.751643+0000","last_unstale":"2024-04-25T11:13:43.917770+0000","last_undegraded":"2024-04-25T11:13:43.917770+0000","last_fullsized":"2024-04-25T11:13:43.917770+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:52.656544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T11:13:42.741968+0000","last_change":"2024-04-25T11:13:42.737111+0000","last_active":"2024-04-25T11:13:42.741968+0000","last_peered":"2024-04-25T11:13:42.741968+0000","last_clean":"2024-04-25T11:13:42.741968+0000","last_became_active":"2024-04-25T11:13:42.736738+0000","last_became_peered":"2024-04-25T11:13:42.736738+0000","last_unstale":"2024-04-25T11:13:42.741968+0000","last_undegraded":"2024-04-25T11:13:42.741968+0000","last_fullsized":"2024-04-25T11:13:42.741968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:48.571574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916844+0000","last_change":"2024-04-25T11:13:42.738528+0000","last_active":"2024-04-25T11:13:43.916844+0000","last_peered":"2024-04-25T11:13:43.916844+0000","last_clean":"2024-04-25T11:13:43.916844+0000","last_became_active":"2024-04-25T11:13:42.737779+0000","last_became_peered":"2024-04-25T11:13:42.737779+0000","last_unstale":"2024-04-25T11:13:43.916844+0000","last_undegraded":"2024-04-25T11:13:43.916844+0000","last_fullsized":"2024-04-25T11:13:43.916844+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:30.695665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914552+0000","last_change":"2024-04-25T11:13:42.745999+0000","last_active":"2024-04-25T11:13:43.914552+0000","last_peered":"2024-04-25T11:13:43.914552+0000","last_clean":"2024-04-25T11:13:43.914552+0000","last_became_active":"2024-04-25T11:13:42.745650+0000","last_became_peered":"2024-04-25T11:13:42.745650+0000","last_unstale":"2024-04-25T11:13:43.914552+0000","last_undegraded":"2024-04-25T11:13:43.914552+0000","last_fullsized":"2024-04-25T11:13:43.914552+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:36.535614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.762889+0000","last_change":"2024-04-25T11:13:42.752470+0000","last_active":"2024-04-25T11:13:43.762889+0000","last_peered":"2024-04-25T11:13:43.762889+0000","last_clean":"2024-04-25T11:13:43.762889+0000","last_became_active":"2024-04-25T11:13:42.751181+0000","last_became_peered":"2024-04-25T11:13:42.751181+0000","last_unstale":"2024-04-25T11:13:43.762889+0000","last_undegraded":"2024-04-25T11:13:43.762889+0000","last_fullsized":"2024-04-25T11:13:43.762889+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:30.840906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.913530+0000","last_change":"2024-04-25T11:13:42.754006+0000","last_active":"2024-04-25T11:13:43.913530+0000","last_peered":"2024-04-25T11:13:43.913530+0000","last_clean":"2024-04-25T11:13:43.913530+0000","last_became_active":"2024-04-25T11:13:42.749343+0000","last_became_peered":"2024-04-25T11:13:42.749343+0000","last_unstale":"2024-04-25T11:13:43.913530+0000","last_undegraded":"2024-04-25T11:13:43.913530+0000","last_fullsized":"2024-04-25T11:13:43.913530+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:05.594877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764769+0000","last_change":"2024-04-25T11:13:42.766529+0000","last_active":"2024-04-25T11:13:43.764769+0000","last_peered":"2024-04-25T11:13:43.764769+0000","last_clean":"2024-04-25T11:13:43.764769+0000","last_became_active":"2024-04-25T11:13:42.765967+0000","last_became_peered":"2024-04-25T11:13:42.765967+0000","last_unstale":"2024-04-25T11:13:43.764769+0000","last_undegraded":"2024-04-25T11:13:43.764769+0000","last_fullsized":"2024-04-25T11:13:43.764769+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:30.695665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763976+0000","last_change":"2024-04-25T11:13:42.756989+0000","last_active":"2024-04-25T11:13:43.763976+0000","last_peered":"2024-04-25T11:13:43.763976+0000","last_clean":"2024-04-25T11:13:43.763976+0000","last_became_active":"2024-04-25T11:13:42.756284+0000","last_became_peered":"2024-04-25T11:13:42.756284+0000","last_unstale":"2024-04-25T11:13:43.763976+0000","last_undegraded":"2024-04-25T11:13:43.763976+0000","last_fullsized":"2024-04-25T11:13:43.763976+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:11.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.913772+0000","last_change":"2024-04-25T11:13:42.752831+0000","last_active":"2024-04-25T11:13:43.913772+0000","last_peered":"2024-04-25T11:13:43.913772+0000","last_clean":"2024-04-25T11:13:43.913772+0000","last_became_active":"2024-04-25T11:13:42.752510+0000","last_became_peered":"2024-04-25T11:13:42.752510+0000","last_unstale":"2024-04-25T11:13:43.913772+0000","last_undegraded":"2024-04-25T11:13:43.913772+0000","last_fullsized":"2024-04-25T11:13:43.913772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:55.417276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T11:13:42.758458+0000","last_change":"2024-04-25T11:13:42.750761+0000","last_active":"2024-04-25T11:13:42.758458+0000","last_peered":"2024-04-25T11:13:42.758458+0000","last_clean":"2024-04-25T11:13:42.758458+0000","last_became_active":"2024-04-25T11:13:42.750426+0000","last_became_peered":"2024-04-25T11:13:42.750426+0000","last_unstale":"2024-04-25T11:13:42.758458+0000","last_undegraded":"2024-04-25T11:13:42.758458+0000","last_fullsized":"2024-04-25T11:13:42.758458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:53.877082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.762879+0000","last_change":"2024-04-25T11:13:42.766414+0000","last_active":"2024-04-25T11:13:43.762879+0000","last_peered":"2024-04-25T11:13:43.762879+0000","last_clean":"2024-04-25T11:13:43.762879+0000","last_became_active":"2024-04-25T11:13:42.766013+0000","last_became_peered":"2024-04-25T11:13:42.766013+0000","last_unstale":"2024-04-25T11:13:43.762879+0000","last_undegraded":"2024-04-25T11:13:43.762879+0000","last_fullsized":"2024-04-25T11:13:43.762879+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:36.546299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764720+0000","last_change":"2024-04-25T11:13:42.773787+0000","last_active":"2024-04-25T11:13:43.764720+0000","last_peered":"2024-04-25T11:13:43.764720+0000","last_clean":"2024-04-25T11:13:43.764720+0000","last_became_active":"2024-04-25T11:13:42.773426+0000","last_became_peered":"2024-04-25T11:13:42.773426+0000","last_unstale":"2024-04-25T11:13:43.764720+0000","last_undegraded":"2024-04-25T11:13:43.764720+0000","last_fullsized":"2024-04-25T11:13:43.764720+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.499757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916555+0000","last_change":"2024-04-25T11:13:42.738002+0000","last_active":"2024-04-25T11:13:43.916555+0000","last_peered":"2024-04-25T11:13:43.916555+0000","last_clean":"2024-04-25T11:13:43.916555+0000","last_became_active":"2024-04-25T11:13:42.737072+0000","last_became_peered":"2024-04-25T11:13:42.737072+0000","last_unstale":"2024-04-25T11:13:43.916555+0000","last_undegraded":"2024-04-25T11:13:43.916555+0000","last_fullsized":"2024-04-25T11:13:43.916555+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:18.120611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.918381+0000","last_change":"2024-04-25T11:13:42.770514+0000","last_active":"2024-04-25T11:13:43.918381+0000","last_peered":"2024-04-25T11:13:43.918381+0000","last_clean":"2024-04-25T11:13:43.918381+0000","last_became_active":"2024-04-25T11:13:42.770134+0000","last_became_peered":"2024-04-25T11:13:42.770134+0000","last_unstale":"2024-04-25T11:13:43.918381+0000","last_undegraded":"2024-04-25T11:13:43.918381+0000","last_fullsized":"2024-04-25T11:13:43.918381+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:36.622499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763267+0000","last_change":"2024-04-25T11:13:42.749403+0000","last_active":"2024-04-25T11:13:43.763267+0000","last_peered":"2024-04-25T11:13:43.763267+0000","last_clean":"2024-04-25T11:13:43.763267+0000","last_became_active":"2024-04-25T11:13:42.748845+0000","last_became_peered":"2024-04-25T11:13:42.748845+0000","last_unstale":"2024-04-25T11:13:43.763267+0000","last_undegraded":"2024-04-25T11:13:43.763267+0000","last_fullsized":"2024-04-25T11:13:43.763267+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:34.576151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.762948+0000","last_change":"2024-04-25T11:13:42.749883+0000","last_active":"2024-04-25T11:13:43.762948+0000","last_peered":"2024-04-25T11:13:43.762948+0000","last_clean":"2024-04-25T11:13:43.762948+0000","last_became_active":"2024-04-25T11:13:42.749562+0000","last_became_peered":"2024-04-25T11:13:42.749562+0000","last_unstale":"2024-04-25T11:13:43.762948+0000","last_undegraded":"2024-04-25T11:13:43.762948+0000","last_fullsized":"2024-04-25T11:13:43.762948+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:43.973719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.696996+0000","last_change":"2024-04-25T11:13:40.696996+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.696996+0000","last_undegraded":"2024-04-25T11:13:40.696996+0000","last_fullsized":"2024-04-25T11:13:40.696996+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.917878+0000","last_change":"2024-04-25T11:13:41.724218+0000","last_active":"2024-04-25T11:13:43.917878+0000","last_peered":"2024-04-25T11:13:43.917878+0000","last_clean":"2024-04-25T11:13:43.917878+0000","last_became_active":"2024-04-25T11:13:41.723900+0000","last_became_peered":"2024-04-25T11:13:41.723900+0000","last_unstale":"2024-04-25T11:13:43.917878+0000","last_undegraded":"2024-04-25T11:13:43.917878+0000","last_fullsized":"2024-04-25T11:13:43.917878+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:44.400287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.918294+0000","last_change":"2024-04-25T11:13:41.725519+0000","last_active":"2024-04-25T11:13:43.918294+0000","last_peered":"2024-04-25T11:13:43.918294+0000","last_clean":"2024-04-25T11:13:43.918294+0000","last_became_active":"2024-04-25T11:13:41.725185+0000","last_became_peered":"2024-04-25T11:13:41.725185+0000","last_unstale":"2024-04-25T11:13:43.918294+0000","last_undegraded":"2024-04-25T11:13:43.918294+0000","last_fullsized":"2024-04-25T11:13:43.918294+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:35.512679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.913466+0000","last_change":"2024-04-25T11:13:42.739790+0000","last_active":"2024-04-25T11:13:43.913466+0000","last_peered":"2024-04-25T11:13:43.913466+0000","last_clean":"2024-04-25T11:13:43.913466+0000","last_became_active":"2024-04-25T11:13:42.738134+0000","last_became_peered":"2024-04-25T11:13:42.738134+0000","last_unstale":"2024-04-25T11:13:43.913466+0000","last_undegraded":"2024-04-25T11:13:43.913466+0000","last_fullsized":"2024-04-25T11:13:43.913466+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:34.576151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.698353+0000","last_change":"2024-04-25T11:13:40.698353+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.698353+0000","last_undegraded":"2024-04-25T11:13:40.698353+0000","last_fullsized":"2024-04-25T11:13:40.698353+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763273+0000","last_change":"2024-04-25T11:13:41.723371+0000","last_active":"2024-04-25T11:13:43.763273+0000","last_peered":"2024-04-25T11:13:43.763273+0000","last_clean":"2024-04-25T11:13:43.763273+0000","last_became_active":"2024-04-25T11:13:41.722761+0000","last_became_peered":"2024-04-25T11:13:41.722761+0000","last_unstale":"2024-04-25T11:13:43.763273+0000","last_undegraded":"2024-04-25T11:13:43.763273+0000","last_fullsized":"2024-04-25T11:13:43.763273+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:35.521194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.703026+0000","last_change":"2024-04-25T11:13:40.703026+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.703026+0000","last_undegraded":"2024-04-25T11:13:40.703026+0000","last_fullsized":"2024-04-25T11:13:40.703026+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916677+0000","last_change":"2024-04-25T11:13:42.738320+0000","last_active":"2024-04-25T11:13:43.916677+0000","last_peered":"2024-04-25T11:13:43.916677+0000","last_clean":"2024-04-25T11:13:43.916677+0000","last_became_active":"2024-04-25T11:13:42.737573+0000","last_became_peered":"2024-04-25T11:13:42.737573+0000","last_unstale":"2024-04-25T11:13:43.916677+0000","last_undegraded":"2024-04-25T11:13:43.916677+0000","last_fullsized":"2024-04-25T11:13:43.916677+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.499757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.693705+0000","last_change":"2024-04-25T11:13:40.693705+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.693705+0000","last_undegraded":"2024-04-25T11:13:40.693705+0000","last_fullsized":"2024-04-25T11:13:40.693705+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.762678+0000","last_change":"2024-04-25T11:13:41.720512+0000","last_active":"2024-04-25T11:13:43.762678+0000","last_peered":"2024-04-25T11:13:43.762678+0000","last_clean":"2024-04-25T11:13:43.762678+0000","last_became_active":"2024-04-25T11:13:41.720199+0000","last_became_peered":"2024-04-25T11:13:41.720199+0000","last_unstale":"2024-04-25T11:13:43.762678+0000","last_undegraded":"2024-04-25T11:13:43.762678+0000","last_fullsized":"2024-04-25T11:13:43.762678+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:14.874387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.687037+0000","last_change":"2024-04-25T11:13:40.687037+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.687037+0000","last_undegraded":"2024-04-25T11:13:40.687037+0000","last_fullsized":"2024-04-25T11:13:40.687037+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.703117+0000","last_change":"2024-04-25T11:13:40.703117+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.703117+0000","last_undegraded":"2024-04-25T11:13:40.703117+0000","last_fullsized":"2024-04-25T11:13:40.703117+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763378+0000","last_change":"2024-04-25T11:13:42.752177+0000","last_active":"2024-04-25T11:13:43.763378+0000","last_peered":"2024-04-25T11:13:43.763378+0000","last_clean":"2024-04-25T11:13:43.763378+0000","last_became_active":"2024-04-25T11:13:42.751415+0000","last_became_peered":"2024-04-25T11:13:42.751415+0000","last_unstale":"2024-04-25T11:13:43.763378+0000","last_undegraded":"2024-04-25T11:13:43.763378+0000","last_fullsized":"2024-04-25T11:13:43.763378+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:12.515416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.688486+0000","last_change":"2024-04-25T11:13:40.688486+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.688486+0000","last_undegraded":"2024-04-25T11:13:40.688486+0000","last_fullsized":"2024-04-25T11:13:40.688486+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916901+0000","last_change":"2024-04-25T11:13:41.718732+0000","last_active":"2024-04-25T11:13:43.916901+0000","last_peered":"2024-04-25T11:13:43.916901+0000","last_clean":"2024-04-25T11:13:43.916901+0000","last_became_active":"2024-04-25T11:13:41.718194+0000","last_became_peered":"2024-04-25T11:13:41.718194+0000","last_unstale":"2024-04-25T11:13:43.916901+0000","last_undegraded":"2024-04-25T11:13:43.916901+0000","last_fullsized":"2024-04-25T11:13:43.916901+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:35.521194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763488+0000","last_change":"2024-04-25T11:13:41.718434+0000","last_active":"2024-04-25T11:13:43.763488+0000","last_peered":"2024-04-25T11:13:43.763488+0000","last_clean":"2024-04-25T11:13:43.763488+0000","last_became_active":"2024-04-25T11:13:41.718102+0000","last_became_peered":"2024-04-25T11:13:41.718102+0000","last_unstale":"2024-04-25T11:13:43.763488+0000","last_undegraded":"2024-04-25T11:13:43.763488+0000","last_fullsized":"2024-04-25T11:13:43.763488+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:54.604025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764155+0000","last_change":"2024-04-25T11:13:42.755555+0000","last_active":"2024-04-25T11:13:43.764155+0000","last_peered":"2024-04-25T11:13:43.764155+0000","last_clean":"2024-04-25T11:13:43.764155+0000","last_became_active":"2024-04-25T11:13:42.755167+0000","last_became_peered":"2024-04-25T11:13:42.755167+0000","last_unstale":"2024-04-25T11:13:43.764155+0000","last_undegraded":"2024-04-25T11:13:43.764155+0000","last_fullsized":"2024-04-25T11:13:43.764155+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:25.629982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.694008+0000","last_change":"2024-04-25T11:13:40.694008+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.694008+0000","last_undegraded":"2024-04-25T11:13:40.694008+0000","last_fullsized":"2024-04-25T11:13:40.694008+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916425+0000","last_change":"2024-04-25T11:13:42.743697+0000","last_active":"2024-04-25T11:13:43.916425+0000","last_peered":"2024-04-25T11:13:43.916425+0000","last_clean":"2024-04-25T11:13:43.916425+0000","last_became_active":"2024-04-25T11:13:42.743098+0000","last_became_peered":"2024-04-25T11:13:42.743098+0000","last_unstale":"2024-04-25T11:13:43.916425+0000","last_undegraded":"2024-04-25T11:13:43.916425+0000","last_fullsized":"2024-04-25T11:13:43.916425+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:25.406529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.687097+0000","last_change":"2024-04-25T11:13:40.687097+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.687097+0000","last_undegraded":"2024-04-25T11:13:40.687097+0000","last_fullsized":"2024-04-25T11:13:40.687097+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764539+0000","last_change":"2024-04-25T11:13:42.766027+0000","last_active":"2024-04-25T11:13:43.764539+0000","last_peered":"2024-04-25T11:13:43.764539+0000","last_clean":"2024-04-25T11:13:43.764539+0000","last_became_active":"2024-04-25T11:13:42.757054+0000","last_became_peered":"2024-04-25T11:13:42.757054+0000","last_unstale":"2024-04-25T11:13:43.764539+0000","last_undegraded":"2024-04-25T11:13:43.764539+0000","last_fullsized":"2024-04-25T11:13:43.764539+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:18.120611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914065+0000","last_change":"2024-04-25T11:13:41.716579+0000","last_active":"2024-04-25T11:13:43.914065+0000","last_peered":"2024-04-25T11:13:43.914065+0000","last_clean":"2024-04-25T11:13:43.914065+0000","last_became_active":"2024-04-25T11:13:41.716270+0000","last_became_peered":"2024-04-25T11:13:41.716270+0000","last_unstale":"2024-04-25T11:13:43.914065+0000","last_undegraded":"2024-04-25T11:13:43.914065+0000","last_fullsized":"2024-04-25T11:13:43.914065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:03.187710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.913889+0000","last_change":"2024-04-25T11:13:42.739057+0000","last_active":"2024-04-25T11:13:43.913889+0000","last_peered":"2024-04-25T11:13:43.913889+0000","last_clean":"2024-04-25T11:13:43.913889+0000","last_became_active":"2024-04-25T11:13:42.738176+0000","last_became_peered":"2024-04-25T11:13:42.738176+0000","last_unstale":"2024-04-25T11:13:43.913889+0000","last_undegraded":"2024-04-25T11:13:43.913889+0000","last_fullsized":"2024-04-25T11:13:43.913889+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:51.420658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764835+0000","last_change":"2024-04-25T11:13:41.723957+0000","last_active":"2024-04-25T11:13:43.764835+0000","last_peered":"2024-04-25T11:13:43.764835+0000","last_clean":"2024-04-25T11:13:43.764835+0000","last_became_active":"2024-04-25T11:13:41.723640+0000","last_became_peered":"2024-04-25T11:13:41.723640+0000","last_unstale":"2024-04-25T11:13:43.764835+0000","last_undegraded":"2024-04-25T11:13:43.764835+0000","last_fullsized":"2024-04-25T11:13:43.764835+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:15.531695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914225+0000","last_change":"2024-04-25T11:13:42.769360+0000","last_active":"2024-04-25T11:13:43.914225+0000","last_peered":"2024-04-25T11:13:43.914225+0000","last_clean":"2024-04-25T11:13:43.914225+0000","last_became_active":"2024-04-25T11:13:42.768556+0000","last_became_peered":"2024-04-25T11:13:42.768556+0000","last_unstale":"2024-04-25T11:13:43.914225+0000","last_undegraded":"2024-04-25T11:13:43.914225+0000","last_fullsized":"2024-04-25T11:13:43.914225+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.822899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T11:13:42.769616+0000","last_change":"2024-04-25T11:13:41.714878+0000","last_active":"2024-04-25T11:13:42.769616+0000","last_peered":"2024-04-25T11:13:42.769616+0000","last_clean":"2024-04-25T11:13:42.769616+0000","last_became_active":"2024-04-25T11:13:41.714405+0000","last_became_peered":"2024-04-25T11:13:41.714405+0000","last_unstale":"2024-04-25T11:13:42.769616+0000","last_undegraded":"2024-04-25T11:13:42.769616+0000","last_fullsized":"2024-04-25T11:13:42.769616+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:33.520154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.694826+0000","last_change":"2024-04-25T11:13:40.694826+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.694826+0000","last_undegraded":"2024-04-25T11:13:40.694826+0000","last_fullsized":"2024-04-25T11:13:40.694826+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914675+0000","last_change":"2024-04-25T11:13:42.747006+0000","last_active":"2024-04-25T11:13:43.914675+0000","last_peered":"2024-04-25T11:13:43.914675+0000","last_clean":"2024-04-25T11:13:43.914675+0000","last_became_active":"2024-04-25T11:13:42.742764+0000","last_became_peered":"2024-04-25T11:13:42.742764+0000","last_unstale":"2024-04-25T11:13:43.914675+0000","last_undegraded":"2024-04-25T11:13:43.914675+0000","last_fullsized":"2024-04-25T11:13:43.914675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:04.202130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.694027+0000","last_change":"2024-04-25T11:13:40.694027+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.694027+0000","last_undegraded":"2024-04-25T11:13:40.694027+0000","last_fullsized":"2024-04-25T11:13:40.694027+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916043+0000","last_change":"2024-04-25T11:13:41.721219+0000","last_active":"2024-04-25T11:13:43.916043+0000","last_peered":"2024-04-25T11:13:43.916043+0000","last_clean":"2024-04-25T11:13:43.916043+0000","last_became_active":"2024-04-25T11:13:41.717206+0000","last_became_peered":"2024-04-25T11:13:41.717206+0000","last_unstale":"2024-04-25T11:13:43.916043+0000","last_undegraded":"2024-04-25T11:13:43.916043+0000","last_fullsized":"2024-04-25T11:13:43.916043+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:52.108620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914096+0000","last_change":"2024-04-25T11:13:41.720009+0000","last_active":"2024-04-25T11:13:43.914096+0000","last_peered":"2024-04-25T11:13:43.914096+0000","last_clean":"2024-04-25T11:13:43.914096+0000","last_became_active":"2024-04-25T11:13:41.719708+0000","last_became_peered":"2024-04-25T11:13:41.719708+0000","last_unstale":"2024-04-25T11:13:43.914096+0000","last_undegraded":"2024-04-25T11:13:43.914096+0000","last_fullsized":"2024-04-25T11:13:43.914096+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:48.907207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764282+0000","last_change":"2024-04-25T11:13:42.752941+0000","last_active":"2024-04-25T11:13:43.764282+0000","last_peered":"2024-04-25T11:13:43.764282+0000","last_clean":"2024-04-25T11:13:43.764282+0000","last_became_active":"2024-04-25T11:13:42.752586+0000","last_became_peered":"2024-04-25T11:13:42.752586+0000","last_unstale":"2024-04-25T11:13:43.764282+0000","last_undegraded":"2024-04-25T11:13:43.764282+0000","last_fullsized":"2024-04-25T11:13:43.764282+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:25.406529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763558+0000","last_change":"2024-04-25T11:13:41.730518+0000","last_active":"2024-04-25T11:13:43.763558+0000","last_peered":"2024-04-25T11:13:43.763558+0000","last_clean":"2024-04-25T11:13:43.763558+0000","last_became_active":"2024-04-25T11:13:41.728941+0000","last_became_peered":"2024-04-25T11:13:41.728941+0000","last_unstale":"2024-04-25T11:13:43.763558+0000","last_undegraded":"2024-04-25T11:13:43.763558+0000","last_fullsized":"2024-04-25T11:13:43.763558+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:23.646637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.918570+0000","last_change":"2024-04-25T11:13:42.750276+0000","last_active":"2024-04-25T11:13:43.918570+0000","last_peered":"2024-04-25T11:13:43.918570+0000","last_clean":"2024-04-25T11:13:43.918570+0000","last_became_active":"2024-04-25T11:13:42.748743+0000","last_became_peered":"2024-04-25T11:13:42.748743+0000","last_unstale":"2024-04-25T11:13:43.918570+0000","last_undegraded":"2024-04-25T11:13:43.918570+0000","last_fullsized":"2024-04-25T11:13:43.918570+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:02.782547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T11:13:42.724706+0000","last_change":"2024-04-25T11:13:41.715709+0000","last_active":"2024-04-25T11:13:42.724706+0000","last_peered":"2024-04-25T11:13:42.724706+0000","last_clean":"2024-04-25T11:13:42.724706+0000","last_became_active":"2024-04-25T11:13:41.715171+0000","last_became_peered":"2024-04-25T11:13:41.715171+0000","last_unstale":"2024-04-25T11:13:42.724706+0000","last_undegraded":"2024-04-25T11:13:42.724706+0000","last_fullsized":"2024-04-25T11:13:42.724706+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:37.869593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.695390+0000","last_change":"2024-04-25T11:13:40.695390+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.695390+0000","last_undegraded":"2024-04-25T11:13:40.695390+0000","last_fullsized":"2024-04-25T11:13:40.695390+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915894+0000","last_change":"2024-04-25T11:13:42.738876+0000","last_active":"2024-04-25T11:13:43.915894+0000","last_peered":"2024-04-25T11:13:43.915894+0000","last_clean":"2024-04-25T11:13:43.915894+0000","last_became_active":"2024-04-25T11:13:42.738029+0000","last_became_peered":"2024-04-25T11:13:42.738029+0000","last_unstale":"2024-04-25T11:13:43.915894+0000","last_undegraded":"2024-04-25T11:13:43.915894+0000","last_fullsized":"2024-04-25T11:13:43.915894+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:49.921627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.687123+0000","last_change":"2024-04-25T11:13:40.687123+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.687123+0000","last_undegraded":"2024-04-25T11:13:40.687123+0000","last_fullsized":"2024-04-25T11:13:40.687123+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915180+0000","last_change":"2024-04-25T11:13:42.740535+0000","last_active":"2024-04-25T11:13:43.915180+0000","last_peered":"2024-04-25T11:13:43.915180+0000","last_clean":"2024-04-25T11:13:43.915180+0000","last_became_active":"2024-04-25T11:13:42.740225+0000","last_became_peered":"2024-04-25T11:13:42.740225+0000","last_unstale":"2024-04-25T11:13:43.915180+0000","last_undegraded":"2024-04-25T11:13:43.915180+0000","last_fullsized":"2024-04-25T11:13:43.915180+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:11.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763748+0000","last_change":"2024-04-25T11:13:41.720592+0000","last_active":"2024-04-25T11:13:43.763748+0000","last_peered":"2024-04-25T11:13:43.763748+0000","last_clean":"2024-04-25T11:13:43.763748+0000","last_became_active":"2024-04-25T11:13:41.720256+0000","last_became_peered":"2024-04-25T11:13:41.720256+0000","last_unstale":"2024-04-25T11:13:43.763748+0000","last_undegraded":"2024-04-25T11:13:43.763748+0000","last_fullsized":"2024-04-25T11:13:43.763748+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:56.024766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763557+0000","last_change":"2024-04-25T11:13:42.773169+0000","last_active":"2024-04-25T11:13:43.763557+0000","last_peered":"2024-04-25T11:13:43.763557+0000","last_clean":"2024-04-25T11:13:43.763557+0000","last_became_active":"2024-04-25T11:13:42.772849+0000","last_became_peered":"2024-04-25T11:13:42.772849+0000","last_unstale":"2024-04-25T11:13:43.763557+0000","last_undegraded":"2024-04-25T11:13:43.763557+0000","last_fullsized":"2024-04-25T11:13:43.763557+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:51.420658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915474+0000","last_change":"2024-04-25T11:13:41.720500+0000","last_active":"2024-04-25T11:13:43.915474+0000","last_peered":"2024-04-25T11:13:43.915474+0000","last_clean":"2024-04-25T11:13:43.915474+0000","last_became_active":"2024-04-25T11:13:41.717767+0000","last_became_peered":"2024-04-25T11:13:41.717767+0000","last_unstale":"2024-04-25T11:13:43.915474+0000","last_undegraded":"2024-04-25T11:13:43.915474+0000","last_fullsized":"2024-04-25T11:13:43.915474+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:28.808479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914412+0000","last_change":"2024-04-25T11:13:41.723363+0000","last_active":"2024-04-25T11:13:43.914412+0000","last_peered":"2024-04-25T11:13:43.914412+0000","last_clean":"2024-04-25T11:13:43.914412+0000","last_became_active":"2024-04-25T11:13:41.722993+0000","last_became_peered":"2024-04-25T11:13:41.722993+0000","last_unstale":"2024-04-25T11:13:43.914412+0000","last_undegraded":"2024-04-25T11:13:43.914412+0000","last_fullsized":"2024-04-25T11:13:43.914412+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:51.642124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764123+0000","last_change":"2024-04-25T11:13:42.771648+0000","last_active":"2024-04-25T11:13:43.764123+0000","last_peered":"2024-04-25T11:13:43.764123+0000","last_clean":"2024-04-25T11:13:43.764123+0000","last_became_active":"2024-04-25T11:13:42.767080+0000","last_became_peered":"2024-04-25T11:13:42.767080+0000","last_unstale":"2024-04-25T11:13:43.764123+0000","last_undegraded":"2024-04-25T11:13:43.764123+0000","last_fullsized":"2024-04-25T11:13:43.764123+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:38.398871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.703482+0000","last_change":"2024-04-25T11:13:40.703482+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.703482+0000","last_undegraded":"2024-04-25T11:13:40.703482+0000","last_fullsized":"2024-04-25T11:13:40.703482+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915723+0000","last_change":"2024-04-25T11:13:42.744578+0000","last_active":"2024-04-25T11:13:43.915723+0000","last_peered":"2024-04-25T11:13:43.915723+0000","last_clean":"2024-04-25T11:13:43.915723+0000","last_became_active":"2024-04-25T11:13:42.744253+0000","last_became_peered":"2024-04-25T11:13:42.744253+0000","last_unstale":"2024-04-25T11:13:43.915723+0000","last_undegraded":"2024-04-25T11:13:43.915723+0000","last_fullsized":"2024-04-25T11:13:43.915723+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:25.629982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T11:13:40.697131+0000","last_change":"2024-04-25T11:13:40.697131+0000","last_active":"2024-04-25T11:13:40.665043+0000","last_peered":"2024-04-25T11:13:40.665043+0000","last_clean":"2024-04-25T11:13:40.665043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:40.697131+0000","last_undegraded":"2024-04-25T11:13:40.697131+0000","last_fullsized":"2024-04-25T11:13:40.697131+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T11:13:41.679463+0000","last_change":"2024-04-25T11:13:41.679463+0000","last_active":"2024-04-25T11:13:41.679463+0000","last_peered":"2024-04-25T11:13:41.679463+0000","last_clean":"2024-04-25T11:13:41.679463+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:13:41.679463+0000","last_undegraded":"2024-04-25T11:13:41.679463+0000","last_fullsized":"2024-04-25T11:13:41.679463+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.767628+0000","last_change":"2024-04-25T11:13:41.719711+0000","last_active":"2024-04-25T11:13:43.767628+0000","last_peered":"2024-04-25T11:13:43.767628+0000","last_clean":"2024-04-25T11:13:43.767628+0000","last_became_active":"2024-04-25T11:13:41.719331+0000","last_became_peered":"2024-04-25T11:13:41.719331+0000","last_unstale":"2024-04-25T11:13:43.767628+0000","last_undegraded":"2024-04-25T11:13:43.767628+0000","last_fullsized":"2024-04-25T11:13:43.767628+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:51.642124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":168,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T11:13:40.689393+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:13:40.689393+0000","last_peered":"2024-04-25T11:13:40.689393+0000","last_clean":"2024-04-25T11:13:40.689393+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:13:40.689393+0000","last_undegraded":"2024-04-25T11:13:40.689393+0000","last_fullsized":"2024-04-25T11:13:40.689393+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":76,"acting":76,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476780,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1187840,"data_stored":850107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476781,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":253530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476780,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":593920,"data_stored":259739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476781,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":253530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509485,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":647168,"data_stored":281899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509486,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":638976,"data_stored":280425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1232896,"data_stored":870689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":630784,"data_stored":275185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:13:47.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:13:47.654 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-04-25T11:13:47.653+0000 7f673e800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:13:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.790+0000 7fd85be00640 1 Processor -- start 2024-04-25T11:13:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.790+0000 7fd85be00640 1 -- start start 2024-04-25T11:13:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.793+0000 7fd85be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd85c164690 0x7fd85c070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.793+0000 7fd85be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd85c1641e0 0x7fd85c0b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.793+0000 7fd85be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd85c0b5080 con 0x7fd85c156cd0 2024-04-25T11:13:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.793+0000 7fd85be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd85c0b5360 con 0x7fd85c164690 2024-04-25T11:13:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.793+0000 7fd85be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd85c0b5640 con 0x7fd85c1641e0 2024-04-25T11:13:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.793+0000 7fd85aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd85c164690 0x7fd85c070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.793+0000 7fd85aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd85c164690 0x7fd85c070a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:33360/0 (socket says 172.21.15.7:33360) 2024-04-25T11:13:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.793+0000 7fd85a000640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd85c156cd0 0x7fd85c163ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:39674/0 (socket says 172.21.15.7:39674) 2024-04-25T11:13:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.793+0000 7fd85aa00640 1 -- 172.21.15.7:0/1602659236 learned_addr learned my addr 172.21.15.7:0/1602659236 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.793+0000 7fd85b400640 1 --2- 172.21.15.7:0/1602659236 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd85c1641e0 0x7fd85c0b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.794+0000 7fd85aa00640 1 -- 172.21.15.7:0/1602659236 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd85c1641e0 msgr2=0x7fd85c0b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.794+0000 7fd85aa00640 1 --2- 172.21.15.7:0/1602659236 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd85c1641e0 0x7fd85c0b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.794+0000 7fd85aa00640 1 -- 172.21.15.7:0/1602659236 >> v1:172.21.15.87:6789/0 conn(0x7fd85c156cd0 legacy=0x7fd85c163ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.794+0000 7fd85aa00640 1 -- 172.21.15.7:0/1602659236 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c0b5920 con 0x7fd85c164690 2024-04-25T11:13:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.796+0000 7fd85aa00640 1 --2- 172.21.15.7:0/1602659236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85c164690 0x7fd85c070a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009870 tx=0x7fd84c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fbdece1bd22cd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.796+0000 7fd859600640 1 -- 172.21.15.7:0/1602659236 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c037070 con 0x7fd85c164690 2024-04-25T11:13:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.796+0000 7fd859600640 1 -- 172.21.15.7:0/1602659236 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84c032440 con 0x7fd85c164690 2024-04-25T11:13:47.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.796+0000 7fd859600640 1 -- 172.21.15.7:0/1602659236 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c032810 con 0x7fd85c164690 2024-04-25T11:13:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.797+0000 7fd85be00640 1 -- 172.21.15.7:0/1602659236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85c164690 msgr2=0x7fd85c070a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.797+0000 7fd85be00640 1 --2- 172.21.15.7:0/1602659236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85c164690 0x7fd85c070a20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009870 tx=0x7fd84c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.797+0000 7fd85be00640 1 -- 172.21.15.7:0/1602659236 shutdown_connections 2024-04-25T11:13:47.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.797+0000 7fd85be00640 1 --2- 172.21.15.7:0/1602659236 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd85c1641e0 0x7fd85c0b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.797+0000 7fd85be00640 1 --2- 172.21.15.7:0/1602659236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85c164690 0x7fd85c070a20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.797+0000 7fd85be00640 1 -- 172.21.15.7:0/1602659236 >> 172.21.15.7:0/1602659236 conn(0x7fd85c0b3820 msgr2=0x7fd85c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.797+0000 7fd85be00640 1 -- 172.21.15.7:0/1602659236 shutdown_connections 2024-04-25T11:13:47.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.797+0000 7fd85be00640 1 -- 172.21.15.7:0/1602659236 wait complete. 2024-04-25T11:13:47.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.800+0000 7fd85be00640 1 Processor -- start 2024-04-25T11:13:47.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.800+0000 7fd85be00640 1 -- start start 2024-04-25T11:13:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.801+0000 7fd85be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd85c156cd0 0x7fd85c15f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.801+0000 7fd85be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c164690 0x7fd85c144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.801+0000 7fd85be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85c1e9fe0 0x7fd85c1ea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.801+0000 7fd85be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1570e0 con 0x7fd85c164690 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.801+0000 7fd85be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c070f60 con 0x7fd85c1e9fe0 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.801+0000 7fd85be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd85c157260 con 0x7fd85c156cd0 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.802+0000 7fd85a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c164690 0x7fd85c144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.802+0000 7fd85a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c164690 0x7fd85c144f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41092/0 (socket says 172.21.15.7:41092) 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.802+0000 7fd85a000640 1 -- 172.21.15.7:0/1783641021 learned_addr learned my addr 172.21.15.7:0/1783641021 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.802+0000 7fd85aa00640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd85c156cd0 0x7fd85c15f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.802+0000 7fd85b400640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85c1e9fe0 0x7fd85c1ea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.802+0000 7fd85a000640 1 -- 172.21.15.7:0/1783641021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd85c156cd0 msgr2=0x7fd85c15f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.802+0000 7fd85a000640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd85c156cd0 0x7fd85c15f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.802+0000 7fd85a000640 1 -- 172.21.15.7:0/1783641021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85c1e9fe0 msgr2=0x7fd85c1ea3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.802+0000 7fd85a000640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85c1e9fe0 0x7fd85c1ea3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.802+0000 7fd85a000640 1 -- 172.21.15.7:0/1783641021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c1ea930 con 0x7fd85c164690 2024-04-25T11:13:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.802+0000 7fd85b400640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85c1e9fe0 0x7fd85c1ea3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.803+0000 7fd85a000640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c164690 0x7fd85c144f90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd84800c3f0 tx=0x7fd84800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.803+0000 7fd852a00640 1 -- 172.21.15.7:0/1783641021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8480074e0 con 0x7fd85c164690 2024-04-25T11:13:47.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.803+0000 7fd852a00640 1 -- 172.21.15.7:0/1783641021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd848007680 con 0x7fd85c164690 2024-04-25T11:13:47.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.803+0000 7fd852a00640 1 -- 172.21.15.7:0/1783641021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd848004a00 con 0x7fd85c164690 2024-04-25T11:13:47.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.803+0000 7fd85be00640 1 -- 172.21.15.7:0/1783641021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85c153c80 con 0x7fd85c164690 2024-04-25T11:13:47.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.803+0000 7fd85be00640 1 -- 172.21.15.7:0/1783641021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85c154240 con 0x7fd85c164690 2024-04-25T11:13:47.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.805+0000 7fd85be00640 1 -- 172.21.15.7:0/1783641021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd85c050740 con 0x7fd85c164690 2024-04-25T11:13:47.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.806+0000 7fd852a00640 1 -- 172.21.15.7:0/1783641021 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd84801a020 con 0x7fd85c164690 2024-04-25T11:13:47.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.807+0000 7fd852a00640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd83c0744e0 0x7fd83c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:47.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.807+0000 7fd852a00640 1 -- 172.21.15.7:0/1783641021 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fd848094b80 con 0x7fd85c164690 2024-04-25T11:13:47.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.807+0000 7fd85aa00640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd83c0744e0 0x7fd83c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:47.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.808+0000 7fd85aa00640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd83c0744e0 0x7fd83c0769f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd84c034000 tx=0x7fd84c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:47.811+0000 7fd852a00640 1 -- 172.21.15.7:0/1783641021 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd85c1ea930 con 0x7fd85c164690 2024-04-25T11:13:48.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.015+0000 7fd85be00640 1 -- 172.21.15.7:0/1783641021 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd85c153a10 con 0x7fd83c0744e0 2024-04-25T11:13:48.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.036+0000 7fd852a00640 1 -- 172.21.15.7:0/1783641021 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7fd85c153a10 con 0x7fd83c0744e0 2024-04-25T11:13:48.040 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:48.042 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 -- 172.21.15.7:0/1783641021 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd83c0744e0 msgr2=0x7fd83c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd83c0744e0 0x7fd83c0769f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd84c034000 tx=0x7fd84c02e000 comp rx=0 tx=0).stop 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 -- 172.21.15.7:0/1783641021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c164690 msgr2=0x7fd85c144f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c164690 0x7fd85c144f90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd84800c3f0 tx=0x7fd84800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 -- 172.21.15.7:0/1783641021 shutdown_connections 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd83c0744e0 0x7fd83c0769f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd85c1e9fe0 0x7fd85c1ea3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd85c164690 0x7fd85c144f90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 --2- 172.21.15.7:0/1783641021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd85c156cd0 0x7fd85c15f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 -- 172.21.15.7:0/1783641021 >> 172.21.15.7:0/1783641021 conn(0x7fd85c0b3820 msgr2=0x7fd85c163840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 -- 172.21.15.7:0/1783641021 shutdown_connections 2024-04-25T11:13:48.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.041+0000 7fd85be00640 1 -- 172.21.15.7:0/1783641021 wait complete. 2024-04-25T11:13:48.070 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-04-25T11:13:47.861943+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243152,"kb_used_data":6140,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731568,"statfs":{"total":773094113280,"available":772845125632,"internally_reserved":0,"allocated":6287360,"data_stored":3420984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.175818"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.918077+0000","last_change":"2024-04-25T11:13:42.737280+0000","last_active":"2024-04-25T11:13:43.918077+0000","last_peered":"2024-04-25T11:13:43.918077+0000","last_clean":"2024-04-25T11:13:43.918077+0000","last_became_active":"2024-04-25T11:13:42.736721+0000","last_became_peered":"2024-04-25T11:13:42.736721+0000","last_unstale":"2024-04-25T11:13:43.918077+0000","last_undegraded":"2024-04-25T11:13:43.918077+0000","last_fullsized":"2024-04-25T11:13:43.918077+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:57.039186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.755252+0000","last_change":"2024-04-25T11:13:42.743842+0000","last_active":"2024-04-25T11:13:43.755252+0000","last_peered":"2024-04-25T11:13:43.755252+0000","last_clean":"2024-04-25T11:13:43.755252+0000","last_became_active":"2024-04-25T11:13:42.743519+0000","last_became_peered":"2024-04-25T11:13:42.743519+0000","last_unstale":"2024-04-25T11:13:43.755252+0000","last_undegraded":"2024-04-25T11:13:43.755252+0000","last_fullsized":"2024-04-25T11:13:43.755252+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:35.367915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763435+0000","last_change":"2024-04-25T11:13:42.757285+0000","last_active":"2024-04-25T11:13:43.763435+0000","last_peered":"2024-04-25T11:13:43.763435+0000","last_clean":"2024-04-25T11:13:43.763435+0000","last_became_active":"2024-04-25T11:13:42.756624+0000","last_became_peered":"2024-04-25T11:13:42.756624+0000","last_unstale":"2024-04-25T11:13:43.763435+0000","last_undegraded":"2024-04-25T11:13:43.763435+0000","last_fullsized":"2024-04-25T11:13:43.763435+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:55.417276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.755027+0000","last_change":"2024-04-25T11:13:42.741756+0000","last_active":"2024-04-25T11:13:43.755027+0000","last_peered":"2024-04-25T11:13:43.755027+0000","last_clean":"2024-04-25T11:13:43.755027+0000","last_became_active":"2024-04-25T11:13:42.741394+0000","last_became_peered":"2024-04-25T11:13:42.741394+0000","last_unstale":"2024-04-25T11:13:43.755027+0000","last_undegraded":"2024-04-25T11:13:43.755027+0000","last_fullsized":"2024-04-25T11:13:43.755027+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:33.692200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.752448+0000","last_change":"2024-04-25T11:13:42.750914+0000","last_active":"2024-04-25T11:13:43.752448+0000","last_peered":"2024-04-25T11:13:43.752448+0000","last_clean":"2024-04-25T11:13:43.752448+0000","last_became_active":"2024-04-25T11:13:42.750537+0000","last_became_peered":"2024-04-25T11:13:42.750537+0000","last_unstale":"2024-04-25T11:13:43.752448+0000","last_undegraded":"2024-04-25T11:13:43.752448+0000","last_fullsized":"2024-04-25T11:13:43.752448+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:21.401223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914607+0000","last_change":"2024-04-25T11:13:42.738461+0000","last_active":"2024-04-25T11:13:43.914607+0000","last_peered":"2024-04-25T11:13:43.914607+0000","last_clean":"2024-04-25T11:13:43.914607+0000","last_became_active":"2024-04-25T11:13:42.736638+0000","last_became_peered":"2024-04-25T11:13:42.736638+0000","last_unstale":"2024-04-25T11:13:43.914607+0000","last_undegraded":"2024-04-25T11:13:43.914607+0000","last_fullsized":"2024-04-25T11:13:43.914607+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:16.546115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.762942+0000","last_change":"2024-04-25T11:13:42.754847+0000","last_active":"2024-04-25T11:13:43.762942+0000","last_peered":"2024-04-25T11:13:43.762942+0000","last_clean":"2024-04-25T11:13:43.762942+0000","last_became_active":"2024-04-25T11:13:42.754513+0000","last_became_peered":"2024-04-25T11:13:42.754513+0000","last_unstale":"2024-04-25T11:13:43.762942+0000","last_undegraded":"2024-04-25T11:13:43.762942+0000","last_fullsized":"2024-04-25T11:13:43.762942+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:17.667799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.913603+0000","last_change":"2024-04-25T11:13:42.749655+0000","last_active":"2024-04-25T11:13:43.913603+0000","last_peered":"2024-04-25T11:13:43.913603+0000","last_clean":"2024-04-25T11:13:43.913603+0000","last_became_active":"2024-04-25T11:13:42.748078+0000","last_became_peered":"2024-04-25T11:13:42.748078+0000","last_unstale":"2024-04-25T11:13:43.913603+0000","last_undegraded":"2024-04-25T11:13:43.913603+0000","last_fullsized":"2024-04-25T11:13:43.913603+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:12.515416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915516+0000","last_change":"2024-04-25T11:13:42.775520+0000","last_active":"2024-04-25T11:13:43.915516+0000","last_peered":"2024-04-25T11:13:43.915516+0000","last_clean":"2024-04-25T11:13:43.915516+0000","last_became_active":"2024-04-25T11:13:42.775179+0000","last_became_peered":"2024-04-25T11:13:42.775179+0000","last_unstale":"2024-04-25T11:13:43.915516+0000","last_undegraded":"2024-04-25T11:13:43.915516+0000","last_fullsized":"2024-04-25T11:13:43.915516+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:38.398871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.917770+0000","last_change":"2024-04-25T11:13:42.751944+0000","last_active":"2024-04-25T11:13:43.917770+0000","last_peered":"2024-04-25T11:13:43.917770+0000","last_clean":"2024-04-25T11:13:43.917770+0000","last_became_active":"2024-04-25T11:13:42.751643+0000","last_became_peered":"2024-04-25T11:13:42.751643+0000","last_unstale":"2024-04-25T11:13:43.917770+0000","last_undegraded":"2024-04-25T11:13:43.917770+0000","last_fullsized":"2024-04-25T11:13:43.917770+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:52.656544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T11:13:42.741968+0000","last_change":"2024-04-25T11:13:42.737111+0000","last_active":"2024-04-25T11:13:42.741968+0000","last_peered":"2024-04-25T11:13:42.741968+0000","last_clean":"2024-04-25T11:13:42.741968+0000","last_became_active":"2024-04-25T11:13:42.736738+0000","last_became_peered":"2024-04-25T11:13:42.736738+0000","last_unstale":"2024-04-25T11:13:42.741968+0000","last_undegraded":"2024-04-25T11:13:42.741968+0000","last_fullsized":"2024-04-25T11:13:42.741968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:48.571574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916844+0000","last_change":"2024-04-25T11:13:42.738528+0000","last_active":"2024-04-25T11:13:43.916844+0000","last_peered":"2024-04-25T11:13:43.916844+0000","last_clean":"2024-04-25T11:13:43.916844+0000","last_became_active":"2024-04-25T11:13:42.737779+0000","last_became_peered":"2024-04-25T11:13:42.737779+0000","last_unstale":"2024-04-25T11:13:43.916844+0000","last_undegraded":"2024-04-25T11:13:43.916844+0000","last_fullsized":"2024-04-25T11:13:43.916844+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:30.695665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914552+0000","last_change":"2024-04-25T11:13:42.745999+0000","last_active":"2024-04-25T11:13:43.914552+0000","last_peered":"2024-04-25T11:13:43.914552+0000","last_clean":"2024-04-25T11:13:43.914552+0000","last_became_active":"2024-04-25T11:13:42.745650+0000","last_became_peered":"2024-04-25T11:13:42.745650+0000","last_unstale":"2024-04-25T11:13:43.914552+0000","last_undegraded":"2024-04-25T11:13:43.914552+0000","last_fullsized":"2024-04-25T11:13:43.914552+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:36.535614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.762889+0000","last_change":"2024-04-25T11:13:42.752470+0000","last_active":"2024-04-25T11:13:43.762889+0000","last_peered":"2024-04-25T11:13:43.762889+0000","last_clean":"2024-04-25T11:13:43.762889+0000","last_became_active":"2024-04-25T11:13:42.751181+0000","last_became_peered":"2024-04-25T11:13:42.751181+0000","last_unstale":"2024-04-25T11:13:43.762889+0000","last_undegraded":"2024-04-25T11:13:43.762889+0000","last_fullsized":"2024-04-25T11:13:43.762889+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:30.840906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.752523+0000","last_change":"2024-04-25T11:13:42.734876+0000","last_active":"2024-04-25T11:13:43.752523+0000","last_peered":"2024-04-25T11:13:43.752523+0000","last_clean":"2024-04-25T11:13:43.752523+0000","last_became_active":"2024-04-25T11:13:42.734520+0000","last_became_peered":"2024-04-25T11:13:42.734520+0000","last_unstale":"2024-04-25T11:13:43.752523+0000","last_undegraded":"2024-04-25T11:13:43.752523+0000","last_fullsized":"2024-04-25T11:13:43.752523+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:57.039186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.913530+0000","last_change":"2024-04-25T11:13:42.754006+0000","last_active":"2024-04-25T11:13:43.913530+0000","last_peered":"2024-04-25T11:13:43.913530+0000","last_clean":"2024-04-25T11:13:43.913530+0000","last_became_active":"2024-04-25T11:13:42.749343+0000","last_became_peered":"2024-04-25T11:13:42.749343+0000","last_unstale":"2024-04-25T11:13:43.913530+0000","last_undegraded":"2024-04-25T11:13:43.913530+0000","last_fullsized":"2024-04-25T11:13:43.913530+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:05.594877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764769+0000","last_change":"2024-04-25T11:13:42.766529+0000","last_active":"2024-04-25T11:13:43.764769+0000","last_peered":"2024-04-25T11:13:43.764769+0000","last_clean":"2024-04-25T11:13:43.764769+0000","last_became_active":"2024-04-25T11:13:42.765967+0000","last_became_peered":"2024-04-25T11:13:42.765967+0000","last_unstale":"2024-04-25T11:13:43.764769+0000","last_undegraded":"2024-04-25T11:13:43.764769+0000","last_fullsized":"2024-04-25T11:13:43.764769+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:30.695665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.755594+0000","last_change":"2024-04-25T11:13:42.748716+0000","last_active":"2024-04-25T11:13:43.755594+0000","last_peered":"2024-04-25T11:13:43.755594+0000","last_clean":"2024-04-25T11:13:43.755594+0000","last_became_active":"2024-04-25T11:13:42.748357+0000","last_became_peered":"2024-04-25T11:13:42.748357+0000","last_unstale":"2024-04-25T11:13:43.755594+0000","last_undegraded":"2024-04-25T11:13:43.755594+0000","last_fullsized":"2024-04-25T11:13:43.755594+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:36.546299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763976+0000","last_change":"2024-04-25T11:13:42.756989+0000","last_active":"2024-04-25T11:13:43.763976+0000","last_peered":"2024-04-25T11:13:43.763976+0000","last_clean":"2024-04-25T11:13:43.763976+0000","last_became_active":"2024-04-25T11:13:42.756284+0000","last_became_peered":"2024-04-25T11:13:42.756284+0000","last_unstale":"2024-04-25T11:13:43.763976+0000","last_undegraded":"2024-04-25T11:13:43.763976+0000","last_fullsized":"2024-04-25T11:13:43.763976+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:11.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915207+0000","last_change":"2024-04-25T11:13:42.748734+0000","last_active":"2024-04-25T11:13:43.915207+0000","last_peered":"2024-04-25T11:13:43.915207+0000","last_clean":"2024-04-25T11:13:43.915207+0000","last_became_active":"2024-04-25T11:13:42.747677+0000","last_became_peered":"2024-04-25T11:13:42.747677+0000","last_unstale":"2024-04-25T11:13:43.915207+0000","last_undegraded":"2024-04-25T11:13:43.915207+0000","last_fullsized":"2024-04-25T11:13:43.915207+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:16.546115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914940+0000","last_change":"2024-04-25T11:13:42.746026+0000","last_active":"2024-04-25T11:13:43.914940+0000","last_peered":"2024-04-25T11:13:43.914940+0000","last_clean":"2024-04-25T11:13:43.914940+0000","last_became_active":"2024-04-25T11:13:42.745679+0000","last_became_peered":"2024-04-25T11:13:42.745679+0000","last_unstale":"2024-04-25T11:13:43.914940+0000","last_undegraded":"2024-04-25T11:13:43.914940+0000","last_fullsized":"2024-04-25T11:13:43.914940+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:05.594877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.913772+0000","last_change":"2024-04-25T11:13:42.752831+0000","last_active":"2024-04-25T11:13:43.913772+0000","last_peered":"2024-04-25T11:13:43.913772+0000","last_clean":"2024-04-25T11:13:43.913772+0000","last_became_active":"2024-04-25T11:13:42.752510+0000","last_became_peered":"2024-04-25T11:13:42.752510+0000","last_unstale":"2024-04-25T11:13:43.913772+0000","last_undegraded":"2024-04-25T11:13:43.913772+0000","last_fullsized":"2024-04-25T11:13:43.913772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:55.417276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T11:13:42.758458+0000","last_change":"2024-04-25T11:13:42.750761+0000","last_active":"2024-04-25T11:13:42.758458+0000","last_peered":"2024-04-25T11:13:42.758458+0000","last_clean":"2024-04-25T11:13:42.758458+0000","last_became_active":"2024-04-25T11:13:42.750426+0000","last_became_peered":"2024-04-25T11:13:42.750426+0000","last_unstale":"2024-04-25T11:13:42.758458+0000","last_undegraded":"2024-04-25T11:13:42.758458+0000","last_fullsized":"2024-04-25T11:13:42.758458+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:53.877082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.762879+0000","last_change":"2024-04-25T11:13:42.766414+0000","last_active":"2024-04-25T11:13:43.762879+0000","last_peered":"2024-04-25T11:13:43.762879+0000","last_clean":"2024-04-25T11:13:43.762879+0000","last_became_active":"2024-04-25T11:13:42.766013+0000","last_became_peered":"2024-04-25T11:13:42.766013+0000","last_unstale":"2024-04-25T11:13:43.762879+0000","last_undegraded":"2024-04-25T11:13:43.762879+0000","last_fullsized":"2024-04-25T11:13:43.762879+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:36.546299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764720+0000","last_change":"2024-04-25T11:13:42.773787+0000","last_active":"2024-04-25T11:13:43.764720+0000","last_peered":"2024-04-25T11:13:43.764720+0000","last_clean":"2024-04-25T11:13:43.764720+0000","last_became_active":"2024-04-25T11:13:42.773426+0000","last_became_peered":"2024-04-25T11:13:42.773426+0000","last_unstale":"2024-04-25T11:13:43.764720+0000","last_undegraded":"2024-04-25T11:13:43.764720+0000","last_fullsized":"2024-04-25T11:13:43.764720+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.499757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916555+0000","last_change":"2024-04-25T11:13:42.738002+0000","last_active":"2024-04-25T11:13:43.916555+0000","last_peered":"2024-04-25T11:13:43.916555+0000","last_clean":"2024-04-25T11:13:43.916555+0000","last_became_active":"2024-04-25T11:13:42.737072+0000","last_became_peered":"2024-04-25T11:13:42.737072+0000","last_unstale":"2024-04-25T11:13:43.916555+0000","last_undegraded":"2024-04-25T11:13:43.916555+0000","last_fullsized":"2024-04-25T11:13:43.916555+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:18.120611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.918381+0000","last_change":"2024-04-25T11:13:42.770514+0000","last_active":"2024-04-25T11:13:43.918381+0000","last_peered":"2024-04-25T11:13:43.918381+0000","last_clean":"2024-04-25T11:13:43.918381+0000","last_became_active":"2024-04-25T11:13:42.770134+0000","last_became_peered":"2024-04-25T11:13:42.770134+0000","last_unstale":"2024-04-25T11:13:43.918381+0000","last_undegraded":"2024-04-25T11:13:43.918381+0000","last_fullsized":"2024-04-25T11:13:43.918381+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:36.622499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915012+0000","last_change":"2024-04-25T11:13:42.739831+0000","last_active":"2024-04-25T11:13:43.915012+0000","last_peered":"2024-04-25T11:13:43.915012+0000","last_clean":"2024-04-25T11:13:43.915012+0000","last_became_active":"2024-04-25T11:13:42.739477+0000","last_became_peered":"2024-04-25T11:13:42.739477+0000","last_unstale":"2024-04-25T11:13:43.915012+0000","last_undegraded":"2024-04-25T11:13:43.915012+0000","last_fullsized":"2024-04-25T11:13:43.915012+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.822899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763267+0000","last_change":"2024-04-25T11:13:42.749403+0000","last_active":"2024-04-25T11:13:43.763267+0000","last_peered":"2024-04-25T11:13:43.763267+0000","last_clean":"2024-04-25T11:13:43.763267+0000","last_became_active":"2024-04-25T11:13:42.748845+0000","last_became_peered":"2024-04-25T11:13:42.748845+0000","last_unstale":"2024-04-25T11:13:43.763267+0000","last_undegraded":"2024-04-25T11:13:43.763267+0000","last_fullsized":"2024-04-25T11:13:43.763267+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:34.576151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.754220+0000","last_change":"2024-04-25T11:13:42.738259+0000","last_active":"2024-04-25T11:13:43.754220+0000","last_peered":"2024-04-25T11:13:43.754220+0000","last_clean":"2024-04-25T11:13:43.754220+0000","last_became_active":"2024-04-25T11:13:42.737902+0000","last_became_peered":"2024-04-25T11:13:42.737902+0000","last_unstale":"2024-04-25T11:13:43.754220+0000","last_undegraded":"2024-04-25T11:13:43.754220+0000","last_fullsized":"2024-04-25T11:13:43.754220+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:02.782547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.762948+0000","last_change":"2024-04-25T11:13:42.749883+0000","last_active":"2024-04-25T11:13:43.762948+0000","last_peered":"2024-04-25T11:13:43.762948+0000","last_clean":"2024-04-25T11:13:43.762948+0000","last_became_active":"2024-04-25T11:13:42.749562+0000","last_became_peered":"2024-04-25T11:13:42.749562+0000","last_unstale":"2024-04-25T11:13:43.762948+0000","last_undegraded":"2024-04-25T11:13:43.762948+0000","last_fullsized":"2024-04-25T11:13:43.762948+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:43.973719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.748109+0000","last_change":"2024-04-25T11:13:42.739595+0000","last_active":"2024-04-25T11:13:43.748109+0000","last_peered":"2024-04-25T11:13:43.748109+0000","last_clean":"2024-04-25T11:13:43.748109+0000","last_became_active":"2024-04-25T11:13:42.738373+0000","last_became_peered":"2024-04-25T11:13:42.738373+0000","last_unstale":"2024-04-25T11:13:43.748109+0000","last_undegraded":"2024-04-25T11:13:43.748109+0000","last_fullsized":"2024-04-25T11:13:43.748109+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:57.039186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.753602+0000","last_change":"2024-04-25T11:13:42.736861+0000","last_active":"2024-04-25T11:13:43.753602+0000","last_peered":"2024-04-25T11:13:43.753602+0000","last_clean":"2024-04-25T11:13:43.753602+0000","last_became_active":"2024-04-25T11:13:42.736544+0000","last_became_peered":"2024-04-25T11:13:42.736544+0000","last_unstale":"2024-04-25T11:13:43.753602+0000","last_undegraded":"2024-04-25T11:13:43.753602+0000","last_fullsized":"2024-04-25T11:13:43.753602+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:36.622499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914706+0000","last_change":"2024-04-25T11:13:41.717285+0000","last_active":"2024-04-25T11:13:43.914706+0000","last_peered":"2024-04-25T11:13:43.914706+0000","last_clean":"2024-04-25T11:13:43.914706+0000","last_became_active":"2024-04-25T11:13:41.716963+0000","last_became_peered":"2024-04-25T11:13:41.716963+0000","last_unstale":"2024-04-25T11:13:43.914706+0000","last_undegraded":"2024-04-25T11:13:43.914706+0000","last_fullsized":"2024-04-25T11:13:43.914706+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:52.108620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.748599+0000","last_change":"2024-04-25T11:13:42.729115+0000","last_active":"2024-04-25T11:13:43.748599+0000","last_peered":"2024-04-25T11:13:43.748599+0000","last_clean":"2024-04-25T11:13:43.748599+0000","last_became_active":"2024-04-25T11:13:42.722418+0000","last_became_peered":"2024-04-25T11:13:42.722418+0000","last_unstale":"2024-04-25T11:13:43.748599+0000","last_undegraded":"2024-04-25T11:13:43.748599+0000","last_fullsized":"2024-04-25T11:13:43.748599+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:02.782547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.917878+0000","last_change":"2024-04-25T11:13:41.724218+0000","last_active":"2024-04-25T11:13:43.917878+0000","last_peered":"2024-04-25T11:13:43.917878+0000","last_clean":"2024-04-25T11:13:43.917878+0000","last_became_active":"2024-04-25T11:13:41.723900+0000","last_became_peered":"2024-04-25T11:13:41.723900+0000","last_unstale":"2024-04-25T11:13:43.917878+0000","last_undegraded":"2024-04-25T11:13:43.917878+0000","last_fullsized":"2024-04-25T11:13:43.917878+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:44.400287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914904+0000","last_change":"2024-04-25T11:13:42.746631+0000","last_active":"2024-04-25T11:13:43.914904+0000","last_peered":"2024-04-25T11:13:43.914904+0000","last_clean":"2024-04-25T11:13:43.914904+0000","last_became_active":"2024-04-25T11:13:42.746304+0000","last_became_peered":"2024-04-25T11:13:42.746304+0000","last_unstale":"2024-04-25T11:13:43.914904+0000","last_undegraded":"2024-04-25T11:13:43.914904+0000","last_fullsized":"2024-04-25T11:13:43.914904+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:11.059144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.918294+0000","last_change":"2024-04-25T11:13:41.725519+0000","last_active":"2024-04-25T11:13:43.918294+0000","last_peered":"2024-04-25T11:13:43.918294+0000","last_clean":"2024-04-25T11:13:43.918294+0000","last_became_active":"2024-04-25T11:13:41.725185+0000","last_became_peered":"2024-04-25T11:13:41.725185+0000","last_unstale":"2024-04-25T11:13:43.918294+0000","last_undegraded":"2024-04-25T11:13:43.918294+0000","last_fullsized":"2024-04-25T11:13:43.918294+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:35.512679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.913466+0000","last_change":"2024-04-25T11:13:42.739790+0000","last_active":"2024-04-25T11:13:43.913466+0000","last_peered":"2024-04-25T11:13:43.913466+0000","last_clean":"2024-04-25T11:13:43.913466+0000","last_became_active":"2024-04-25T11:13:42.738134+0000","last_became_peered":"2024-04-25T11:13:42.738134+0000","last_unstale":"2024-04-25T11:13:43.913466+0000","last_undegraded":"2024-04-25T11:13:43.913466+0000","last_fullsized":"2024-04-25T11:13:43.913466+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:34.576151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.749886+0000","last_change":"2024-04-25T11:13:41.706976+0000","last_active":"2024-04-25T11:13:43.749886+0000","last_peered":"2024-04-25T11:13:43.749886+0000","last_clean":"2024-04-25T11:13:43.749886+0000","last_became_active":"2024-04-25T11:13:41.706305+0000","last_became_peered":"2024-04-25T11:13:41.706305+0000","last_unstale":"2024-04-25T11:13:43.749886+0000","last_undegraded":"2024-04-25T11:13:43.749886+0000","last_fullsized":"2024-04-25T11:13:43.749886+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:10.044724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.753570+0000","last_change":"2024-04-25T11:13:42.774244+0000","last_active":"2024-04-25T11:13:43.753570+0000","last_peered":"2024-04-25T11:13:43.753570+0000","last_clean":"2024-04-25T11:13:43.753570+0000","last_became_active":"2024-04-25T11:13:42.773856+0000","last_became_peered":"2024-04-25T11:13:42.773856+0000","last_unstale":"2024-04-25T11:13:43.753570+0000","last_undegraded":"2024-04-25T11:13:43.753570+0000","last_fullsized":"2024-04-25T11:13:43.753570+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:46.968619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763273+0000","last_change":"2024-04-25T11:13:41.723371+0000","last_active":"2024-04-25T11:13:43.763273+0000","last_peered":"2024-04-25T11:13:43.763273+0000","last_clean":"2024-04-25T11:13:43.763273+0000","last_became_active":"2024-04-25T11:13:41.722761+0000","last_became_peered":"2024-04-25T11:13:41.722761+0000","last_unstale":"2024-04-25T11:13:43.763273+0000","last_undegraded":"2024-04-25T11:13:43.763273+0000","last_fullsized":"2024-04-25T11:13:43.763273+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:35.521194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.754814+0000","last_change":"2024-04-25T11:13:42.744186+0000","last_active":"2024-04-25T11:13:43.754814+0000","last_peered":"2024-04-25T11:13:43.754814+0000","last_clean":"2024-04-25T11:13:43.754814+0000","last_became_active":"2024-04-25T11:13:42.742407+0000","last_became_peered":"2024-04-25T11:13:42.742407+0000","last_unstale":"2024-04-25T11:13:43.754814+0000","last_undegraded":"2024-04-25T11:13:43.754814+0000","last_fullsized":"2024-04-25T11:13:43.754814+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:01.636873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.751388+0000","last_change":"2024-04-25T11:13:41.716321+0000","last_active":"2024-04-25T11:13:43.751388+0000","last_peered":"2024-04-25T11:13:43.751388+0000","last_clean":"2024-04-25T11:13:43.751388+0000","last_became_active":"2024-04-25T11:13:41.715992+0000","last_became_peered":"2024-04-25T11:13:41.715992+0000","last_unstale":"2024-04-25T11:13:43.751388+0000","last_undegraded":"2024-04-25T11:13:43.751388+0000","last_fullsized":"2024-04-25T11:13:43.751388+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:22.964946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916677+0000","last_change":"2024-04-25T11:13:42.738320+0000","last_active":"2024-04-25T11:13:43.916677+0000","last_peered":"2024-04-25T11:13:43.916677+0000","last_clean":"2024-04-25T11:13:43.916677+0000","last_became_active":"2024-04-25T11:13:42.737573+0000","last_became_peered":"2024-04-25T11:13:42.737573+0000","last_unstale":"2024-04-25T11:13:43.916677+0000","last_undegraded":"2024-04-25T11:13:43.916677+0000","last_fullsized":"2024-04-25T11:13:43.916677+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.499757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914310+0000","last_change":"2024-04-25T11:13:41.716141+0000","last_active":"2024-04-25T11:13:43.914310+0000","last_peered":"2024-04-25T11:13:43.914310+0000","last_clean":"2024-04-25T11:13:43.914310+0000","last_became_active":"2024-04-25T11:13:41.715363+0000","last_became_peered":"2024-04-25T11:13:41.715363+0000","last_unstale":"2024-04-25T11:13:43.914310+0000","last_undegraded":"2024-04-25T11:13:43.914310+0000","last_fullsized":"2024-04-25T11:13:43.914310+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:48.092463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.754989+0000","last_change":"2024-04-25T11:13:42.768013+0000","last_active":"2024-04-25T11:13:43.754989+0000","last_peered":"2024-04-25T11:13:43.754989+0000","last_clean":"2024-04-25T11:13:43.754989+0000","last_became_active":"2024-04-25T11:13:42.767671+0000","last_became_peered":"2024-04-25T11:13:42.767671+0000","last_unstale":"2024-04-25T11:13:43.754989+0000","last_undegraded":"2024-04-25T11:13:43.754989+0000","last_fullsized":"2024-04-25T11:13:43.754989+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:23.728362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.762678+0000","last_change":"2024-04-25T11:13:41.720512+0000","last_active":"2024-04-25T11:13:43.762678+0000","last_peered":"2024-04-25T11:13:43.762678+0000","last_clean":"2024-04-25T11:13:43.762678+0000","last_became_active":"2024-04-25T11:13:41.720199+0000","last_became_peered":"2024-04-25T11:13:41.720199+0000","last_unstale":"2024-04-25T11:13:43.762678+0000","last_undegraded":"2024-04-25T11:13:43.762678+0000","last_fullsized":"2024-04-25T11:13:43.762678+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:14.874387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.751124+0000","last_change":"2024-04-25T11:13:42.748221+0000","last_active":"2024-04-25T11:13:43.751124+0000","last_peered":"2024-04-25T11:13:43.751124+0000","last_clean":"2024-04-25T11:13:43.751124+0000","last_became_active":"2024-04-25T11:13:42.747875+0000","last_became_peered":"2024-04-25T11:13:42.747875+0000","last_unstale":"2024-04-25T11:13:43.751124+0000","last_undegraded":"2024-04-25T11:13:43.751124+0000","last_fullsized":"2024-04-25T11:13:43.751124+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:15.888807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.754693+0000","last_change":"2024-04-25T11:13:41.713780+0000","last_active":"2024-04-25T11:13:43.754693+0000","last_peered":"2024-04-25T11:13:43.754693+0000","last_clean":"2024-04-25T11:13:43.754693+0000","last_became_active":"2024-04-25T11:13:41.713456+0000","last_became_peered":"2024-04-25T11:13:41.713456+0000","last_unstale":"2024-04-25T11:13:43.754693+0000","last_undegraded":"2024-04-25T11:13:43.754693+0000","last_fullsized":"2024-04-25T11:13:43.754693+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:45.954199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.748961+0000","last_change":"2024-04-25T11:13:42.748242+0000","last_active":"2024-04-25T11:13:43.748961+0000","last_peered":"2024-04-25T11:13:43.748961+0000","last_clean":"2024-04-25T11:13:43.748961+0000","last_became_active":"2024-04-25T11:13:42.747881+0000","last_became_peered":"2024-04-25T11:13:42.747881+0000","last_unstale":"2024-04-25T11:13:43.748961+0000","last_undegraded":"2024-04-25T11:13:43.748961+0000","last_fullsized":"2024-04-25T11:13:43.748961+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:01.636873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.754009+0000","last_change":"2024-04-25T11:13:41.724956+0000","last_active":"2024-04-25T11:13:43.754009+0000","last_peered":"2024-04-25T11:13:43.754009+0000","last_clean":"2024-04-25T11:13:43.754009+0000","last_became_active":"2024-04-25T11:13:41.724624+0000","last_became_peered":"2024-04-25T11:13:41.724624+0000","last_unstale":"2024-04-25T11:13:43.754009+0000","last_undegraded":"2024-04-25T11:13:43.754009+0000","last_fullsized":"2024-04-25T11:13:43.754009+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:48.092463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763378+0000","last_change":"2024-04-25T11:13:42.752177+0000","last_active":"2024-04-25T11:13:43.763378+0000","last_peered":"2024-04-25T11:13:43.763378+0000","last_clean":"2024-04-25T11:13:43.763378+0000","last_became_active":"2024-04-25T11:13:42.751415+0000","last_became_peered":"2024-04-25T11:13:42.751415+0000","last_unstale":"2024-04-25T11:13:43.763378+0000","last_undegraded":"2024-04-25T11:13:43.763378+0000","last_fullsized":"2024-04-25T11:13:43.763378+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:12.515416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.754955+0000","last_change":"2024-04-25T11:13:41.725962+0000","last_active":"2024-04-25T11:13:43.754955+0000","last_peered":"2024-04-25T11:13:43.754955+0000","last_clean":"2024-04-25T11:13:43.754955+0000","last_became_active":"2024-04-25T11:13:41.725698+0000","last_became_peered":"2024-04-25T11:13:41.725698+0000","last_unstale":"2024-04-25T11:13:43.754955+0000","last_undegraded":"2024-04-25T11:13:43.754955+0000","last_fullsized":"2024-04-25T11:13:43.754955+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:29.534817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.752543+0000","last_change":"2024-04-25T11:13:42.745646+0000","last_active":"2024-04-25T11:13:43.752543+0000","last_peered":"2024-04-25T11:13:43.752543+0000","last_clean":"2024-04-25T11:13:43.752543+0000","last_became_active":"2024-04-25T11:13:42.745274+0000","last_became_peered":"2024-04-25T11:13:42.745274+0000","last_unstale":"2024-04-25T11:13:43.752543+0000","last_undegraded":"2024-04-25T11:13:43.752543+0000","last_fullsized":"2024-04-25T11:13:43.752543+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:30.549237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916901+0000","last_change":"2024-04-25T11:13:41.718732+0000","last_active":"2024-04-25T11:13:43.916901+0000","last_peered":"2024-04-25T11:13:43.916901+0000","last_clean":"2024-04-25T11:13:43.916901+0000","last_became_active":"2024-04-25T11:13:41.718194+0000","last_became_peered":"2024-04-25T11:13:41.718194+0000","last_unstale":"2024-04-25T11:13:43.916901+0000","last_undegraded":"2024-04-25T11:13:43.916901+0000","last_fullsized":"2024-04-25T11:13:43.916901+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:35.521194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.749241+0000","last_change":"2024-04-25T11:13:42.722762+0000","last_active":"2024-04-25T11:13:43.749241+0000","last_peered":"2024-04-25T11:13:43.749241+0000","last_clean":"2024-04-25T11:13:43.749241+0000","last_became_active":"2024-04-25T11:13:42.721720+0000","last_became_peered":"2024-04-25T11:13:42.721720+0000","last_unstale":"2024-04-25T11:13:43.749241+0000","last_undegraded":"2024-04-25T11:13:43.749241+0000","last_fullsized":"2024-04-25T11:13:43.749241+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:33.692200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763488+0000","last_change":"2024-04-25T11:13:41.718434+0000","last_active":"2024-04-25T11:13:43.763488+0000","last_peered":"2024-04-25T11:13:43.763488+0000","last_clean":"2024-04-25T11:13:43.763488+0000","last_became_active":"2024-04-25T11:13:41.718102+0000","last_became_peered":"2024-04-25T11:13:41.718102+0000","last_unstale":"2024-04-25T11:13:43.763488+0000","last_undegraded":"2024-04-25T11:13:43.763488+0000","last_fullsized":"2024-04-25T11:13:43.763488+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:54.604025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764155+0000","last_change":"2024-04-25T11:13:42.755555+0000","last_active":"2024-04-25T11:13:43.764155+0000","last_peered":"2024-04-25T11:13:43.764155+0000","last_clean":"2024-04-25T11:13:43.764155+0000","last_became_active":"2024-04-25T11:13:42.755167+0000","last_became_peered":"2024-04-25T11:13:42.755167+0000","last_unstale":"2024-04-25T11:13:43.764155+0000","last_undegraded":"2024-04-25T11:13:43.764155+0000","last_fullsized":"2024-04-25T11:13:43.764155+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:25.629982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914383+0000","last_change":"2024-04-25T11:13:41.725017+0000","last_active":"2024-04-25T11:13:43.914383+0000","last_peered":"2024-04-25T11:13:43.914383+0000","last_clean":"2024-04-25T11:13:43.914383+0000","last_became_active":"2024-04-25T11:13:41.724660+0000","last_became_peered":"2024-04-25T11:13:41.724660+0000","last_unstale":"2024-04-25T11:13:43.914383+0000","last_undegraded":"2024-04-25T11:13:43.914383+0000","last_fullsized":"2024-04-25T11:13:43.914383+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:39.617193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916425+0000","last_change":"2024-04-25T11:13:42.743697+0000","last_active":"2024-04-25T11:13:43.916425+0000","last_peered":"2024-04-25T11:13:43.916425+0000","last_clean":"2024-04-25T11:13:43.916425+0000","last_became_active":"2024-04-25T11:13:42.743098+0000","last_became_peered":"2024-04-25T11:13:42.743098+0000","last_unstale":"2024-04-25T11:13:43.916425+0000","last_undegraded":"2024-04-25T11:13:43.916425+0000","last_fullsized":"2024-04-25T11:13:43.916425+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:25.406529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.752703+0000","last_change":"2024-04-25T11:13:41.713664+0000","last_active":"2024-04-25T11:13:43.752703+0000","last_peered":"2024-04-25T11:13:43.752703+0000","last_clean":"2024-04-25T11:13:43.752703+0000","last_became_active":"2024-04-25T11:13:41.713319+0000","last_became_peered":"2024-04-25T11:13:41.713319+0000","last_unstale":"2024-04-25T11:13:43.752703+0000","last_undegraded":"2024-04-25T11:13:43.752703+0000","last_fullsized":"2024-04-25T11:13:43.752703+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:20.386803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764539+0000","last_change":"2024-04-25T11:13:42.766027+0000","last_active":"2024-04-25T11:13:43.764539+0000","last_peered":"2024-04-25T11:13:43.764539+0000","last_clean":"2024-04-25T11:13:43.764539+0000","last_became_active":"2024-04-25T11:13:42.757054+0000","last_became_peered":"2024-04-25T11:13:42.757054+0000","last_unstale":"2024-04-25T11:13:43.764539+0000","last_undegraded":"2024-04-25T11:13:43.764539+0000","last_fullsized":"2024-04-25T11:13:43.764539+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:18.120611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914065+0000","last_change":"2024-04-25T11:13:41.716579+0000","last_active":"2024-04-25T11:13:43.914065+0000","last_peered":"2024-04-25T11:13:43.914065+0000","last_clean":"2024-04-25T11:13:43.914065+0000","last_became_active":"2024-04-25T11:13:41.716270+0000","last_became_peered":"2024-04-25T11:13:41.716270+0000","last_unstale":"2024-04-25T11:13:43.914065+0000","last_undegraded":"2024-04-25T11:13:43.914065+0000","last_fullsized":"2024-04-25T11:13:43.914065+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:03.187710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.913889+0000","last_change":"2024-04-25T11:13:42.739057+0000","last_active":"2024-04-25T11:13:43.913889+0000","last_peered":"2024-04-25T11:13:43.913889+0000","last_clean":"2024-04-25T11:13:43.913889+0000","last_became_active":"2024-04-25T11:13:42.738176+0000","last_became_peered":"2024-04-25T11:13:42.738176+0000","last_unstale":"2024-04-25T11:13:43.913889+0000","last_undegraded":"2024-04-25T11:13:43.913889+0000","last_fullsized":"2024-04-25T11:13:43.913889+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:51.420658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764835+0000","last_change":"2024-04-25T11:13:41.723957+0000","last_active":"2024-04-25T11:13:43.764835+0000","last_peered":"2024-04-25T11:13:43.764835+0000","last_clean":"2024-04-25T11:13:43.764835+0000","last_became_active":"2024-04-25T11:13:41.723640+0000","last_became_peered":"2024-04-25T11:13:41.723640+0000","last_unstale":"2024-04-25T11:13:43.764835+0000","last_undegraded":"2024-04-25T11:13:43.764835+0000","last_fullsized":"2024-04-25T11:13:43.764835+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:15.531695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914225+0000","last_change":"2024-04-25T11:13:42.769360+0000","last_active":"2024-04-25T11:13:43.914225+0000","last_peered":"2024-04-25T11:13:43.914225+0000","last_clean":"2024-04-25T11:13:43.914225+0000","last_became_active":"2024-04-25T11:13:42.768556+0000","last_became_peered":"2024-04-25T11:13:42.768556+0000","last_unstale":"2024-04-25T11:13:43.914225+0000","last_undegraded":"2024-04-25T11:13:43.914225+0000","last_fullsized":"2024-04-25T11:13:43.914225+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:29.822899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T11:13:42.769616+0000","last_change":"2024-04-25T11:13:41.714878+0000","last_active":"2024-04-25T11:13:42.769616+0000","last_peered":"2024-04-25T11:13:42.769616+0000","last_clean":"2024-04-25T11:13:42.769616+0000","last_became_active":"2024-04-25T11:13:41.714405+0000","last_became_peered":"2024-04-25T11:13:41.714405+0000","last_unstale":"2024-04-25T11:13:42.769616+0000","last_undegraded":"2024-04-25T11:13:42.769616+0000","last_fullsized":"2024-04-25T11:13:42.769616+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:33.520154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.751218+0000","last_change":"2024-04-25T11:13:42.774281+0000","last_active":"2024-04-25T11:13:43.751218+0000","last_peered":"2024-04-25T11:13:43.751218+0000","last_clean":"2024-04-25T11:13:43.751218+0000","last_became_active":"2024-04-25T11:13:42.773870+0000","last_became_peered":"2024-04-25T11:13:42.773870+0000","last_unstale":"2024-04-25T11:13:43.751218+0000","last_undegraded":"2024-04-25T11:13:43.751218+0000","last_fullsized":"2024-04-25T11:13:43.751218+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:55.618445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915007+0000","last_change":"2024-04-25T11:13:41.725766+0000","last_active":"2024-04-25T11:13:43.915007+0000","last_peered":"2024-04-25T11:13:43.915007+0000","last_clean":"2024-04-25T11:13:43.915007+0000","last_became_active":"2024-04-25T11:13:41.725411+0000","last_became_peered":"2024-04-25T11:13:41.725411+0000","last_unstale":"2024-04-25T11:13:43.915007+0000","last_undegraded":"2024-04-25T11:13:43.915007+0000","last_fullsized":"2024-04-25T11:13:43.915007+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:44.400287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914675+0000","last_change":"2024-04-25T11:13:42.747006+0000","last_active":"2024-04-25T11:13:43.914675+0000","last_peered":"2024-04-25T11:13:43.914675+0000","last_clean":"2024-04-25T11:13:43.914675+0000","last_became_active":"2024-04-25T11:13:42.742764+0000","last_became_peered":"2024-04-25T11:13:42.742764+0000","last_unstale":"2024-04-25T11:13:43.914675+0000","last_undegraded":"2024-04-25T11:13:43.914675+0000","last_fullsized":"2024-04-25T11:13:43.914675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:04.202130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915257+0000","last_change":"2024-04-25T11:13:41.716905+0000","last_active":"2024-04-25T11:13:43.915257+0000","last_peered":"2024-04-25T11:13:43.915257+0000","last_clean":"2024-04-25T11:13:43.915257+0000","last_became_active":"2024-04-25T11:13:41.716578+0000","last_became_peered":"2024-04-25T11:13:41.716578+0000","last_unstale":"2024-04-25T11:13:43.915257+0000","last_undegraded":"2024-04-25T11:13:43.915257+0000","last_fullsized":"2024-04-25T11:13:43.915257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:35.512679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.749721+0000","last_change":"2024-04-25T11:13:42.739498+0000","last_active":"2024-04-25T11:13:43.749721+0000","last_peered":"2024-04-25T11:13:43.749721+0000","last_clean":"2024-04-25T11:13:43.749721+0000","last_became_active":"2024-04-25T11:13:42.739094+0000","last_became_peered":"2024-04-25T11:13:42.739094+0000","last_unstale":"2024-04-25T11:13:43.749721+0000","last_undegraded":"2024-04-25T11:13:43.749721+0000","last_fullsized":"2024-04-25T11:13:43.749721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:35.367915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.916043+0000","last_change":"2024-04-25T11:13:41.721219+0000","last_active":"2024-04-25T11:13:43.916043+0000","last_peered":"2024-04-25T11:13:43.916043+0000","last_clean":"2024-04-25T11:13:43.916043+0000","last_became_active":"2024-04-25T11:13:41.717206+0000","last_became_peered":"2024-04-25T11:13:41.717206+0000","last_unstale":"2024-04-25T11:13:43.916043+0000","last_undegraded":"2024-04-25T11:13:43.916043+0000","last_fullsized":"2024-04-25T11:13:43.916043+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:52.108620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915195+0000","last_change":"2024-04-25T11:13:42.747370+0000","last_active":"2024-04-25T11:13:43.915195+0000","last_peered":"2024-04-25T11:13:43.915195+0000","last_clean":"2024-04-25T11:13:43.915195+0000","last_became_active":"2024-04-25T11:13:42.745255+0000","last_became_peered":"2024-04-25T11:13:42.745255+0000","last_unstale":"2024-04-25T11:13:43.915195+0000","last_undegraded":"2024-04-25T11:13:43.915195+0000","last_fullsized":"2024-04-25T11:13:43.915195+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:36.535614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914096+0000","last_change":"2024-04-25T11:13:41.720009+0000","last_active":"2024-04-25T11:13:43.914096+0000","last_peered":"2024-04-25T11:13:43.914096+0000","last_clean":"2024-04-25T11:13:43.914096+0000","last_became_active":"2024-04-25T11:13:41.719708+0000","last_became_peered":"2024-04-25T11:13:41.719708+0000","last_unstale":"2024-04-25T11:13:43.914096+0000","last_undegraded":"2024-04-25T11:13:43.914096+0000","last_fullsized":"2024-04-25T11:13:43.914096+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:48.907207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764282+0000","last_change":"2024-04-25T11:13:42.752941+0000","last_active":"2024-04-25T11:13:43.764282+0000","last_peered":"2024-04-25T11:13:43.764282+0000","last_clean":"2024-04-25T11:13:43.764282+0000","last_became_active":"2024-04-25T11:13:42.752586+0000","last_became_peered":"2024-04-25T11:13:42.752586+0000","last_unstale":"2024-04-25T11:13:43.764282+0000","last_undegraded":"2024-04-25T11:13:43.764282+0000","last_fullsized":"2024-04-25T11:13:43.764282+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:25.406529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763558+0000","last_change":"2024-04-25T11:13:41.730518+0000","last_active":"2024-04-25T11:13:43.763558+0000","last_peered":"2024-04-25T11:13:43.763558+0000","last_clean":"2024-04-25T11:13:43.763558+0000","last_became_active":"2024-04-25T11:13:41.728941+0000","last_became_peered":"2024-04-25T11:13:41.728941+0000","last_unstale":"2024-04-25T11:13:43.763558+0000","last_undegraded":"2024-04-25T11:13:43.763558+0000","last_fullsized":"2024-04-25T11:13:43.763558+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:23.646637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.918570+0000","last_change":"2024-04-25T11:13:42.750276+0000","last_active":"2024-04-25T11:13:43.918570+0000","last_peered":"2024-04-25T11:13:43.918570+0000","last_clean":"2024-04-25T11:13:43.918570+0000","last_became_active":"2024-04-25T11:13:42.748743+0000","last_became_peered":"2024-04-25T11:13:42.748743+0000","last_unstale":"2024-04-25T11:13:43.918570+0000","last_undegraded":"2024-04-25T11:13:43.918570+0000","last_fullsized":"2024-04-25T11:13:43.918570+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:02.782547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T11:13:42.724706+0000","last_change":"2024-04-25T11:13:41.715709+0000","last_active":"2024-04-25T11:13:42.724706+0000","last_peered":"2024-04-25T11:13:42.724706+0000","last_clean":"2024-04-25T11:13:42.724706+0000","last_became_active":"2024-04-25T11:13:41.715171+0000","last_became_peered":"2024-04-25T11:13:41.715171+0000","last_unstale":"2024-04-25T11:13:42.724706+0000","last_undegraded":"2024-04-25T11:13:42.724706+0000","last_fullsized":"2024-04-25T11:13:42.724706+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:37.869593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.748177+0000","last_change":"2024-04-25T11:13:42.737293+0000","last_active":"2024-04-25T11:13:43.748177+0000","last_peered":"2024-04-25T11:13:43.748177+0000","last_clean":"2024-04-25T11:13:43.748177+0000","last_became_active":"2024-04-25T11:13:42.736952+0000","last_became_peered":"2024-04-25T11:13:42.736952+0000","last_unstale":"2024-04-25T11:13:43.748177+0000","last_undegraded":"2024-04-25T11:13:43.748177+0000","last_fullsized":"2024-04-25T11:13:43.748177+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:36.622499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914825+0000","last_change":"2024-04-25T11:13:41.728200+0000","last_active":"2024-04-25T11:13:43.914825+0000","last_peered":"2024-04-25T11:13:43.914825+0000","last_clean":"2024-04-25T11:13:43.914825+0000","last_became_active":"2024-04-25T11:13:41.727902+0000","last_became_peered":"2024-04-25T11:13:41.727902+0000","last_unstale":"2024-04-25T11:13:43.914825+0000","last_undegraded":"2024-04-25T11:13:43.914825+0000","last_fullsized":"2024-04-25T11:13:43.914825+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:57.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915894+0000","last_change":"2024-04-25T11:13:42.738876+0000","last_active":"2024-04-25T11:13:43.915894+0000","last_peered":"2024-04-25T11:13:43.915894+0000","last_clean":"2024-04-25T11:13:43.915894+0000","last_became_active":"2024-04-25T11:13:42.738029+0000","last_became_peered":"2024-04-25T11:13:42.738029+0000","last_unstale":"2024-04-25T11:13:43.915894+0000","last_undegraded":"2024-04-25T11:13:43.915894+0000","last_fullsized":"2024-04-25T11:13:43.915894+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:49.921627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.755125+0000","last_change":"2024-04-25T11:13:41.714268+0000","last_active":"2024-04-25T11:13:43.755125+0000","last_peered":"2024-04-25T11:13:43.755125+0000","last_clean":"2024-04-25T11:13:43.755125+0000","last_became_active":"2024-04-25T11:13:41.713936+0000","last_became_peered":"2024-04-25T11:13:41.713936+0000","last_unstale":"2024-04-25T11:13:43.755125+0000","last_undegraded":"2024-04-25T11:13:43.755125+0000","last_fullsized":"2024-04-25T11:13:43.755125+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:07.013666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915180+0000","last_change":"2024-04-25T11:13:42.740535+0000","last_active":"2024-04-25T11:13:43.915180+0000","last_peered":"2024-04-25T11:13:43.915180+0000","last_clean":"2024-04-25T11:13:43.915180+0000","last_became_active":"2024-04-25T11:13:42.740225+0000","last_became_peered":"2024-04-25T11:13:42.740225+0000","last_unstale":"2024-04-25T11:13:43.915180+0000","last_undegraded":"2024-04-25T11:13:43.915180+0000","last_fullsized":"2024-04-25T11:13:43.915180+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:11.467567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763748+0000","last_change":"2024-04-25T11:13:41.720592+0000","last_active":"2024-04-25T11:13:43.763748+0000","last_peered":"2024-04-25T11:13:43.763748+0000","last_clean":"2024-04-25T11:13:43.763748+0000","last_became_active":"2024-04-25T11:13:41.720256+0000","last_became_peered":"2024-04-25T11:13:41.720256+0000","last_unstale":"2024-04-25T11:13:43.763748+0000","last_undegraded":"2024-04-25T11:13:43.763748+0000","last_fullsized":"2024-04-25T11:13:43.763748+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:56.024766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.763557+0000","last_change":"2024-04-25T11:13:42.773169+0000","last_active":"2024-04-25T11:13:43.763557+0000","last_peered":"2024-04-25T11:13:43.763557+0000","last_clean":"2024-04-25T11:13:43.763557+0000","last_became_active":"2024-04-25T11:13:42.772849+0000","last_became_peered":"2024-04-25T11:13:42.772849+0000","last_unstale":"2024-04-25T11:13:43.763557+0000","last_undegraded":"2024-04-25T11:13:43.763557+0000","last_fullsized":"2024-04-25T11:13:43.763557+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:51.420658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915474+0000","last_change":"2024-04-25T11:13:41.720500+0000","last_active":"2024-04-25T11:13:43.915474+0000","last_peered":"2024-04-25T11:13:43.915474+0000","last_clean":"2024-04-25T11:13:43.915474+0000","last_became_active":"2024-04-25T11:13:41.717767+0000","last_became_peered":"2024-04-25T11:13:41.717767+0000","last_unstale":"2024-04-25T11:13:43.915474+0000","last_undegraded":"2024-04-25T11:13:43.915474+0000","last_fullsized":"2024-04-25T11:13:43.915474+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:28.808479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.749092+0000","last_change":"2024-04-25T11:13:42.738736+0000","last_active":"2024-04-25T11:13:43.749092+0000","last_peered":"2024-04-25T11:13:43.749092+0000","last_clean":"2024-04-25T11:13:43.749092+0000","last_became_active":"2024-04-25T11:13:42.735935+0000","last_became_peered":"2024-04-25T11:13:42.735935+0000","last_unstale":"2024-04-25T11:13:43.749092+0000","last_undegraded":"2024-04-25T11:13:43.749092+0000","last_fullsized":"2024-04-25T11:13:43.749092+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:23.728362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914412+0000","last_change":"2024-04-25T11:13:41.723363+0000","last_active":"2024-04-25T11:13:43.914412+0000","last_peered":"2024-04-25T11:13:43.914412+0000","last_clean":"2024-04-25T11:13:43.914412+0000","last_became_active":"2024-04-25T11:13:41.722993+0000","last_became_peered":"2024-04-25T11:13:41.722993+0000","last_unstale":"2024-04-25T11:13:43.914412+0000","last_undegraded":"2024-04-25T11:13:43.914412+0000","last_fullsized":"2024-04-25T11:13:43.914412+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:51.642124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.764123+0000","last_change":"2024-04-25T11:13:42.771648+0000","last_active":"2024-04-25T11:13:43.764123+0000","last_peered":"2024-04-25T11:13:43.764123+0000","last_clean":"2024-04-25T11:13:43.764123+0000","last_became_active":"2024-04-25T11:13:42.767080+0000","last_became_peered":"2024-04-25T11:13:42.767080+0000","last_unstale":"2024-04-25T11:13:43.764123+0000","last_undegraded":"2024-04-25T11:13:43.764123+0000","last_fullsized":"2024-04-25T11:13:43.764123+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:38.398871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.751476+0000","last_change":"2024-04-25T11:13:41.722803+0000","last_active":"2024-04-25T11:13:43.751476+0000","last_peered":"2024-04-25T11:13:43.751476+0000","last_clean":"2024-04-25T11:13:43.751476+0000","last_became_active":"2024-04-25T11:13:41.722483+0000","last_became_peered":"2024-04-25T11:13:41.722483+0000","last_unstale":"2024-04-25T11:13:43.751476+0000","last_undegraded":"2024-04-25T11:13:43.751476+0000","last_fullsized":"2024-04-25T11:13:43.751476+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:37.813749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.915723+0000","last_change":"2024-04-25T11:13:42.744578+0000","last_active":"2024-04-25T11:13:43.915723+0000","last_peered":"2024-04-25T11:13:43.915723+0000","last_clean":"2024-04-25T11:13:43.915723+0000","last_became_active":"2024-04-25T11:13:42.744253+0000","last_became_peered":"2024-04-25T11:13:42.744253+0000","last_unstale":"2024-04-25T11:13:43.915723+0000","last_undegraded":"2024-04-25T11:13:43.915723+0000","last_fullsized":"2024-04-25T11:13:43.915723+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:25.629982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.748346+0000","last_change":"2024-04-25T11:13:41.726599+0000","last_active":"2024-04-25T11:13:43.748346+0000","last_peered":"2024-04-25T11:13:43.748346+0000","last_clean":"2024-04-25T11:13:43.748346+0000","last_became_active":"2024-04-25T11:13:41.726301+0000","last_became_peered":"2024-04-25T11:13:41.726301+0000","last_unstale":"2024-04-25T11:13:43.748346+0000","last_undegraded":"2024-04-25T11:13:43.748346+0000","last_fullsized":"2024-04-25T11:13:43.748346+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:07.013666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.914592+0000","last_change":"2024-04-25T11:13:42.751440+0000","last_active":"2024-04-25T11:13:43.914592+0000","last_peered":"2024-04-25T11:13:43.914592+0000","last_clean":"2024-04-25T11:13:43.914592+0000","last_became_active":"2024-04-25T11:13:42.750173+0000","last_became_peered":"2024-04-25T11:13:42.750173+0000","last_unstale":"2024-04-25T11:13:43.914592+0000","last_undegraded":"2024-04-25T11:13:43.914592+0000","last_fullsized":"2024-04-25T11:13:43.914592+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:41.679463+0000","last_clean_scrub_stamp":"2024-04-25T11:13:41.679463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:08.028086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.767628+0000","last_change":"2024-04-25T11:13:41.719711+0000","last_active":"2024-04-25T11:13:43.767628+0000","last_peered":"2024-04-25T11:13:43.767628+0000","last_clean":"2024-04-25T11:13:43.767628+0000","last_became_active":"2024-04-25T11:13:41.719331+0000","last_became_peered":"2024-04-25T11:13:41.719331+0000","last_unstale":"2024-04-25T11:13:43.767628+0000","last_undegraded":"2024-04-25T11:13:43.767628+0000","last_fullsized":"2024-04-25T11:13:43.767628+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:13:40.665043+0000","last_clean_scrub_stamp":"2024-04-25T11:13:40.665043+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:51.642124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T11:13:43.753563+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:13:43.753563+0000","last_peered":"2024-04-25T11:13:43.753563+0000","last_clean":"2024-04-25T11:13:43.753563+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:13:43.753563+0000","last_undegraded":"2024-04-25T11:13:43.753563+0000","last_fullsized":"2024-04-25T11:13:43.753563+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476781,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1232896,"data_stored":870757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476782,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":638976,"data_stored":280917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476781,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476782,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":634880,"data_stored":280847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509485,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":647168,"data_stored":281899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509486,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":638976,"data_stored":280425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509485,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1232896,"data_stored":870689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":630784,"data_stored":275185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:13:48.074 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:13:48.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:13:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.412+0000 7f018b400640 1 Processor -- start 2024-04-25T11:13:48.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.412+0000 7f018b400640 1 -- start start 2024-04-25T11:13:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.414+0000 7f018b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f018409f790 0x7f018415dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.414+0000 7f018b400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f018406dc90 0x7f01840ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.415+0000 7f018b400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f018406f470 con 0x7f0184070d60 2024-04-25T11:13:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.415+0000 7f018b400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f018415e2a0 con 0x7f018409f790 2024-04-25T11:13:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.415+0000 7f018b400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0184074550 con 0x7f018406dc90 2024-04-25T11:13:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.415+0000 7f0183e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0184070d60 0x7f018406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47678/0 (socket says 172.21.15.7:47678) 2024-04-25T11:13:48.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.415+0000 7f0183e00640 1 -- 172.21.15.7:0/3885100569 learned_addr learned my addr 172.21.15.7:0/3885100569 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.415+0000 7f0189400640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f018406dc90 0x7f01840ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.415+0000 7f0188a00640 1 --2- 172.21.15.7:0/3885100569 >> v2:172.21.15.87:3300/0 conn(0x7f018409f790 0x7f018415dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.415+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 108954211 0 0) 0x7f018406f470 con 0x7f0184070d60 2024-04-25T11:13:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f016c003520 con 0x7f0184070d60 2024-04-25T11:13:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3850793611 0 0) 0x7f016c003520 con 0x7f0184070d60 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f018406dc90 msgr2=0x7f01840ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f018406dc90 0x7f01840ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 >> v2:172.21.15.87:3300/0 conn(0x7f018409f790 msgr2=0x7f018415dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 --2- 172.21.15.7:0/3885100569 >> v2:172.21.15.87:3300/0 conn(0x7f018409f790 0x7f018415dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f018406f750 con 0x7f0184070d60 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0170001670 con 0x7f0184070d60 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 >> v1:172.21.15.7:6789/0 conn(0x7f0184070d60 legacy=0x7f018406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f016c0039b0 0x7f016c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016c004360 0x7f016c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016c004ef0 0x7f016c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0184074550 con 0x7f016c004360 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f016c00bf00 con 0x7f016c004ef0 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.416+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f016c00c080 con 0x7f016c0039b0 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.417+0000 7f0183e00640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016c004360 0x7f016c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.417+0000 7f0188a00640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f016c0039b0 0x7f016c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.417+0000 7f0189400640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016c004ef0 0x7f016c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.417+0000 7f0183e00640 1 -- 172.21.15.7:0/3885100569 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f016c0039b0 msgr2=0x7f016c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.417+0000 7f0183e00640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f016c0039b0 0x7f016c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.417+0000 7f0183e00640 1 -- 172.21.15.7:0/3885100569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016c004ef0 msgr2=0x7f016c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.417+0000 7f0183e00640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016c004ef0 0x7f016c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.417+0000 7f0183e00640 1 -- 172.21.15.7:0/3885100569 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c003520 con 0x7f016c004360 2024-04-25T11:13:48.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.418+0000 7f0183e00640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016c004360 0x7f016c004880 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f017000c240 tx=0x7f016c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4eaf1e1c5e5bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.419+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017002e070 con 0x7f016c004360 2024-04-25T11:13:48.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.419+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0170005ad0 con 0x7f016c004360 2024-04-25T11:13:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.419+0000 7f0183400640 1 -- 172.21.15.7:0/3885100569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170007d30 con 0x7f016c004360 2024-04-25T11:13:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.420+0000 7f018b400640 1 -- 172.21.15.7:0/3885100569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016c004360 msgr2=0x7f016c004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.420+0000 7f018b400640 1 --2- 172.21.15.7:0/3885100569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f016c004360 0x7f016c004880 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f017000c240 tx=0x7f016c005830 comp rx=0 tx=0).stop 2024-04-25T11:13:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.420+0000 7f0189400640 1 -- 172.21.15.7:0/3885100569 reap_dead start 2024-04-25T11:13:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.420+0000 7f018b400640 1 -- 172.21.15.7:0/3885100569 shutdown_connections 2024-04-25T11:13:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.420+0000 7f018b400640 1 -- 172.21.15.7:0/3885100569 >> 172.21.15.7:0/3885100569 conn(0x7f01840d1d40 msgr2=0x7f0184073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.420+0000 7f018b400640 1 -- 172.21.15.7:0/3885100569 shutdown_connections 2024-04-25T11:13:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.420+0000 7f018b400640 1 -- 172.21.15.7:0/3885100569 wait complete. 2024-04-25T11:13:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.422+0000 7f018b400640 1 Processor -- start 2024-04-25T11:13:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.423+0000 7f018b400640 1 -- start start 2024-04-25T11:13:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.424+0000 7f018b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016c004360 0x7f0184155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.424+0000 7f018b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01841f4260 0x7f01841f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.424+0000 7f018b400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01840ceaa0 0x7f01840ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.424+0000 7f018b400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0184070080 con 0x7f01841f4260 2024-04-25T11:13:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.424+0000 7f018b400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f018406f220 con 0x7f016c004360 2024-04-25T11:13:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.424+0000 7f018b400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f018406fd10 con 0x7f01840ceaa0 2024-04-25T11:13:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.424+0000 7f0183e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01841f4260 0x7f01841f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.424+0000 7f0183e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01841f4260 0x7f01841f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41104/0 (socket says 172.21.15.7:41104) 2024-04-25T11:13:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.424+0000 7f0183e00640 1 -- 172.21.15.7:0/2527439299 learned_addr learned my addr 172.21.15.7:0/2527439299 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:48.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.425+0000 7f0188a00640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016c004360 0x7f0184155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:48.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.425+0000 7f0189400640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01840ceaa0 0x7f01840ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:48.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.425+0000 7f0183e00640 1 -- 172.21.15.7:0/2527439299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01840ceaa0 msgr2=0x7f01840ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:48.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.425+0000 7f0183e00640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01840ceaa0 0x7f01840ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.425+0000 7f0183e00640 1 -- 172.21.15.7:0/2527439299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016c004360 msgr2=0x7f0184155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:48.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.425+0000 7f0183e00640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016c004360 0x7f0184155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.425+0000 7f0183e00640 1 -- 172.21.15.7:0/2527439299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0184154ce0 con 0x7f01841f4260 2024-04-25T11:13:48.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.425+0000 7f0183e00640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01841f4260 0x7f01841f4670 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f017000c360 tx=0x7f018406f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.426+0000 7f0180c00640 1 -- 172.21.15.7:0/2527439299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170042070 con 0x7f01841f4260 2024-04-25T11:13:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.426+0000 7f0180c00640 1 -- 172.21.15.7:0/2527439299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0170007480 con 0x7f01841f4260 2024-04-25T11:13:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.426+0000 7f0180c00640 1 -- 172.21.15.7:0/2527439299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0170007d30 con 0x7f01841f4260 2024-04-25T11:13:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.426+0000 7f018b400640 1 -- 172.21.15.7:0/2527439299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0184154fc0 con 0x7f01841f4260 2024-04-25T11:13:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.426+0000 7f018b400640 1 -- 172.21.15.7:0/2527439299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01841549b0 con 0x7f01841f4260 2024-04-25T11:13:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.427+0000 7f018b400640 1 -- 172.21.15.7:0/2527439299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0184146710 con 0x7f01841f4260 2024-04-25T11:13:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.429+0000 7f0180c00640 1 -- 172.21.15.7:0/2527439299 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0170045070 con 0x7f01841f4260 2024-04-25T11:13:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.430+0000 7f0180c00640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f01640744e0 0x7f01640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.430+0000 7f0180c00640 1 -- 172.21.15.7:0/2527439299 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f01700b5eb0 con 0x7f01841f4260 2024-04-25T11:13:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.430+0000 7f0188a00640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f01640744e0 0x7f01640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:48.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.431+0000 7f0188a00640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f01640744e0 0x7f01640769f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01840b0460 tx=0x7f0178008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:48.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.434+0000 7f0180c00640 1 -- 172.21.15.7:0/2527439299 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f01700804b0 con 0x7f01841f4260 2024-04-25T11:13:48.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.710+0000 7f018b400640 1 -- 172.21.15.7:0/2527439299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0184050740 con 0x7f01841f4260 2024-04-25T11:13:48.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.715+0000 7f0180c00640 1 -- 172.21.15.7:0/2527439299 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f017007fdb0 con 0x7f01841f4260 2024-04-25T11:13:48.716 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-25T11:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-04-25T11:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-04-25T11:13:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:13:48.717 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-04-25T11:13:48.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.718+0000 7f018b400640 1 -- 172.21.15.7:0/2527439299 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f01640744e0 msgr2=0x7f01640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:48.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.718+0000 7f018b400640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f01640744e0 0x7f01640769f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01840b0460 tx=0x7f0178008040 comp rx=0 tx=0).stop 2024-04-25T11:13:48.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.718+0000 7f018b400640 1 -- 172.21.15.7:0/2527439299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01841f4260 msgr2=0x7f01841f4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:48.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.718+0000 7f018b400640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01841f4260 0x7f01841f4670 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f017000c360 tx=0x7f018406f920 comp rx=0 tx=0).stop 2024-04-25T11:13:48.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.718+0000 7f018b400640 1 -- 172.21.15.7:0/2527439299 shutdown_connections 2024-04-25T11:13:48.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.718+0000 7f018b400640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f01640744e0 0x7f01640769f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.718+0000 7f018b400640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01840ceaa0 0x7f01840ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.719+0000 7f018b400640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01841f4260 0x7f01841f4670 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.719+0000 7f018b400640 1 --2- 172.21.15.7:0/2527439299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f016c004360 0x7f0184155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:48.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.719+0000 7f018b400640 1 -- 172.21.15.7:0/2527439299 >> 172.21.15.7:0/2527439299 conn(0x7f01840d1d40 msgr2=0x7f018415a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:48.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.719+0000 7f018b400640 1 -- 172.21.15.7:0/2527439299 shutdown_connections 2024-04-25T11:13:48.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:48.719+0000 7f018b400640 1 -- 172.21.15.7:0/2527439299 wait complete. 2024-04-25T11:13:48.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:13:49.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.022+0000 7ff42f200640 1 Processor -- start 2024-04-25T11:13:49.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.022+0000 7ff42f200640 1 -- start start 2024-04-25T11:13:49.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.025+0000 7ff42f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4280af370 0x7ff4280ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:49.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.025+0000 7ff42f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4280ce860 0x7ff42806fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:49.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.025+0000 7ff42f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4280af850 con 0x7ff42815c7a0 2024-04-25T11:13:49.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.025+0000 7ff42f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff4280b5a80 con 0x7ff4280af370 2024-04-25T11:13:49.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.025+0000 7ff42f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff4280b5430 con 0x7ff4280ce860 2024-04-25T11:13:49.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.025+0000 7ff427e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4280af370 0x7ff4280ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:49.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.025+0000 7ff427e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff4280af370 0x7ff4280ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41118/0 (socket says 172.21.15.7:41118) 2024-04-25T11:13:49.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.025+0000 7ff427e00640 1 -- 172.21.15.7:0/1433678168 learned_addr learned my addr 172.21.15.7:0/1433678168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:49.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.025+0000 7ff427400640 1 --2- 172.21.15.7:0/1433678168 >> v2:172.21.15.87:3300/0 conn(0x7ff4280ce860 0x7ff42806fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:49.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.026+0000 7ff427e00640 1 -- 172.21.15.7:0/1433678168 >> v2:172.21.15.87:3300/0 conn(0x7ff4280ce860 msgr2=0x7ff42806fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:49.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.026+0000 7ff427e00640 1 --2- 172.21.15.7:0/1433678168 >> v2:172.21.15.87:3300/0 conn(0x7ff4280ce860 0x7ff42806fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:49.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.026+0000 7ff427e00640 1 -- 172.21.15.7:0/1433678168 >> v1:172.21.15.87:6789/0 conn(0x7ff42815c7a0 legacy=0x7ff4280b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:49.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.026+0000 7ff427e00640 1 -- 172.21.15.7:0/1433678168 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4280b5710 con 0x7ff4280af370 2024-04-25T11:13:49.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.027+0000 7ff427e00640 1 --2- 172.21.15.7:0/1433678168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4280af370 0x7ff4280ce320 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff4280730d0 tx=0x7ff41802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f21414784072596 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:49.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.027+0000 7ff426a00640 1 -- 172.21.15.7:0/1433678168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418037070 con 0x7ff4280af370 2024-04-25T11:13:49.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.027+0000 7ff426a00640 1 -- 172.21.15.7:0/1433678168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff418032440 con 0x7ff4280af370 2024-04-25T11:13:49.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.027+0000 7ff426a00640 1 -- 172.21.15.7:0/1433678168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418032810 con 0x7ff4280af370 2024-04-25T11:13:49.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.028+0000 7ff42f200640 1 -- 172.21.15.7:0/1433678168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4280af370 msgr2=0x7ff4280ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:49.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.028+0000 7ff42f200640 1 --2- 172.21.15.7:0/1433678168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4280af370 0x7ff4280ce320 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff4280730d0 tx=0x7ff41802a230 comp rx=0 tx=0).stop 2024-04-25T11:13:49.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.028+0000 7ff42f200640 1 -- 172.21.15.7:0/1433678168 shutdown_connections 2024-04-25T11:13:49.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.028+0000 7ff42f200640 1 --2- 172.21.15.7:0/1433678168 >> v2:172.21.15.87:3300/0 conn(0x7ff4280ce860 0x7ff42806fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:49.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.028+0000 7ff42f200640 1 --2- 172.21.15.7:0/1433678168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4280af370 0x7ff4280ce320 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:49.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.028+0000 7ff42f200640 1 -- 172.21.15.7:0/1433678168 >> 172.21.15.7:0/1433678168 conn(0x7ff428074200 msgr2=0x7ff42806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:49.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.028+0000 7ff42f200640 1 -- 172.21.15.7:0/1433678168 shutdown_connections 2024-04-25T11:13:49.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.028+0000 7ff42f200640 1 -- 172.21.15.7:0/1433678168 wait complete. 2024-04-25T11:13:49.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.030+0000 7ff42f200640 1 Processor -- start 2024-04-25T11:13:49.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.031+0000 7ff42f200640 1 -- start start 2024-04-25T11:13:49.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.032+0000 7ff42f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4280af370 0x7ff428148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:49.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.032+0000 7ff42f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4280ce860 0x7ff428141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:49.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.032+0000 7ff42f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42815c7a0 0x7ff42815f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.032+0000 7ff42f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff428070d70 con 0x7ff4280ce860 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.032+0000 7ff42f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff428070bf0 con 0x7ff4280af370 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.032+0000 7ff42f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff4280b6620 con 0x7ff42815c7a0 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.032+0000 7ff427400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4280ce860 0x7ff428141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.032+0000 7ff427400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4280ce860 0x7ff428141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41130/0 (socket says 172.21.15.7:41130) 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.033+0000 7ff427400640 1 -- 172.21.15.7:0/547988599 learned_addr learned my addr 172.21.15.7:0/547988599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.033+0000 7ff427e00640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4280af370 0x7ff428148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.033+0000 7ff42d200640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42815c7a0 0x7ff42815f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.033+0000 7ff427400640 1 -- 172.21.15.7:0/547988599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42815c7a0 msgr2=0x7ff42815f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.033+0000 7ff427400640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42815c7a0 0x7ff42815f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.033+0000 7ff427400640 1 -- 172.21.15.7:0/547988599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4280af370 msgr2=0x7ff428148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:49.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.033+0000 7ff427400640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4280af370 0x7ff428148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:49.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.033+0000 7ff427400640 1 -- 172.21.15.7:0/547988599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff428148fe0 con 0x7ff4280ce860 2024-04-25T11:13:49.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.033+0000 7ff42d200640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42815c7a0 0x7ff42815f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:49.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.033+0000 7ff427400640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4280ce860 0x7ff428141250 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff414007e40 tx=0x7ff41400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:49.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.034+0000 7ff41fe00640 1 -- 172.21.15.7:0/547988599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414014620 con 0x7ff4280ce860 2024-04-25T11:13:49.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.034+0000 7ff41fe00640 1 -- 172.21.15.7:0/547988599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff414019070 con 0x7ff4280ce860 2024-04-25T11:13:49.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.034+0000 7ff42f200640 1 -- 172.21.15.7:0/547988599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4281492c0 con 0x7ff4280ce860 2024-04-25T11:13:49.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.034+0000 7ff41fe00640 1 -- 172.21.15.7:0/547988599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414014a00 con 0x7ff4280ce860 2024-04-25T11:13:49.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.034+0000 7ff42f200640 1 -- 172.21.15.7:0/547988599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff428149880 con 0x7ff4280ce860 2024-04-25T11:13:49.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.035+0000 7ff42f200640 1 -- 172.21.15.7:0/547988599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff428148d50 con 0x7ff4280ce860 2024-04-25T11:13:49.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.036+0000 7ff41fe00640 1 -- 172.21.15.7:0/547988599 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff414014ba0 con 0x7ff4280ce860 2024-04-25T11:13:49.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.037+0000 7ff41fe00640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4080744b0 0x7ff4080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:49.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.037+0000 7ff41fe00640 1 -- 172.21.15.7:0/547988599 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7ff414095b90 con 0x7ff4280ce860 2024-04-25T11:13:49.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.037+0000 7ff427e00640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4080744b0 0x7ff4080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:49.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.038+0000 7ff427e00640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4080744b0 0x7ff4080769c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff418034000 tx=0x7ff41802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:49.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.042+0000 7ff41fe00640 1 -- 172.21.15.7:0/547988599 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff414060220 con 0x7ff4280ce860 2024-04-25T11:13:49.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.276+0000 7ff42f200640 1 -- 172.21.15.7:0/547988599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff42806a100 con 0x7ff4280ce860 2024-04-25T11:13:49.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.278+0000 7ff41fe00640 1 -- 172.21.15.7:0/547988599 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5172 (secure 0 0 0) 0x7ff41405fb20 con 0x7ff4280ce860 2024-04-25T11:13:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:49.279 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2225639104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2225639104},{"type":"v1","addr":"172.21.15.87:6834","nonce":2225639104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2625900548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2625900548},{"type":"v1","addr":"172.21.15.7:6837","nonce":2625900548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/717352569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":717352569},{"type":"v1","addr":"172.21.15.87:6835","nonce":717352569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:13:42.310229+0000","modified":"2024-04-25T11:13:45.341610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6836/3503942462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3503942462},{"type":"v1","addr":"172.21.15.7:6836","nonce":3503942462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-25T11:13:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-25T11:13:49.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 -- 172.21.15.7:0/547988599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4080744b0 msgr2=0x7ff4080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:49.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4080744b0 0x7ff4080769c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff418034000 tx=0x7ff41802e000 comp rx=0 tx=0).stop 2024-04-25T11:13:49.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 -- 172.21.15.7:0/547988599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4280ce860 msgr2=0x7ff428141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4280ce860 0x7ff428141250 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff414007e40 tx=0x7ff41400e4b0 comp rx=0 tx=0).stop 2024-04-25T11:13:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 -- 172.21.15.7:0/547988599 shutdown_connections 2024-04-25T11:13:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4080744b0 0x7ff4080769c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42815c7a0 0x7ff42815f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4280ce860 0x7ff428141250 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 --2- 172.21.15.7:0/547988599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4280af370 0x7ff428148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:49.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 -- 172.21.15.7:0/547988599 >> 172.21.15.7:0/547988599 conn(0x7ff428074200 msgr2=0x7ff4280cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:49.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 -- 172.21.15.7:0/547988599 shutdown_connections 2024-04-25T11:13:49.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:49.281+0000 7ff42f200640 1 -- 172.21.15.7:0/547988599 wait complete. 2024-04-25T11:13:49.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:13:42.310229+0000', 'modified': '2024-04-25T11:13:45.341610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5742}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5742': {'gid': 5742, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6836/3503942462', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 3503942462}, {'type': 'v1', 'addr': '172.21.15.7:6836', 'nonce': 3503942462}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5742, 'qdb_cluster': [5742]} 2024-04-25T11:13:49.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:13:49.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T11:13:50.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.009+0000 7f30ff000640 1 Processor -- start 2024-04-25T11:13:50.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.009+0000 7f30ff000640 1 -- start start 2024-04-25T11:13:50.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.011+0000 7f30ff000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f30f81600f0 0x7f30f80b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.011+0000 7f30ff000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f80b7d70 con 0x7f30f806ed90 2024-04-25T11:13:50.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.011+0000 7f30ff000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30f80cf850 con 0x7f30f80b7960 2024-04-25T11:13:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.011+0000 7f30ff000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f30f80af730 con 0x7f30f81600f0 2024-04-25T11:13:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.012+0000 7f30fdc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f30f806ed90 0x7f30f80b3960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55524/0 (socket says 172.21.15.7:55524) 2024-04-25T11:13:50.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.012+0000 7f30fdc00640 1 -- 172.21.15.7:0/151550139 learned_addr learned my addr 172.21.15.7:0/151550139 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:50.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.012+0000 7f30fd200640 1 --2- 172.21.15.7:0/151550139 >> v2:172.21.15.87:3300/0 conn(0x7f30f81600f0 0x7f30f80b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.012+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4279051616 0 0) 0x7f30ec001110 con 0x7f30f806ed90 2024-04-25T11:13:50.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.012+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30e0003520 con 0x7f30f806ed90 2024-04-25T11:13:50.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2367787689 0 0) 0x7f30e0003520 con 0x7f30f806ed90 2024-04-25T11:13:50.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 >> v2:172.21.15.87:3300/0 conn(0x7f30f81600f0 msgr2=0x7f30f80b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 --2- 172.21.15.7:0/151550139 >> v2:172.21.15.87:3300/0 conn(0x7f30f81600f0 0x7f30f80b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 >> v1:172.21.15.87:6789/0 conn(0x7f30f80b7960 legacy=0x7f30f80afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f80cfb30 con 0x7f30f806ed90 2024-04-25T11:13:50.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f30ec001840 con 0x7f30f806ed90 2024-04-25T11:13:50.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 >> v1:172.21.15.7:6789/0 conn(0x7f30f806ed90 legacy=0x7f30f80b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30e00039b0 0x7f30e0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 0x7f30e0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0005270 0x7f30e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30f80af730 con 0x7f30e0004800 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f30e000c190 con 0x7f30e0005270 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f30e000c390 con 0x7f30e00039b0 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30fd200640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 0x7f30e0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.013+0000 7f30fdc00640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30e00039b0 0x7f30e0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.014+0000 7f30fe600640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0005270 0x7f30e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.014+0000 7f30fd200640 1 -- 172.21.15.7:0/151550139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30e00039b0 msgr2=0x7f30e0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.014+0000 7f30fd200640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30e00039b0 0x7f30e0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.014+0000 7f30fd200640 1 -- 172.21.15.7:0/151550139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0005270 msgr2=0x7f30e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.014+0000 7f30fd200640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0005270 0x7f30e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.014+0000 7f30fd200640 1 -- 172.21.15.7:0/151550139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e0003520 con 0x7f30e0004800 2024-04-25T11:13:50.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.014+0000 7f30fe600640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30e0005270 0x7f30e000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:50.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.014+0000 7f30fe600640 1 -- 172.21.15.7:0/151550139 reap_dead start 2024-04-25T11:13:50.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.015+0000 7f30fd200640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 0x7f30e0004c10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009a90 tx=0x7f30e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bbf610253ff580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:50.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.015+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8036070 con 0x7f30e0004800 2024-04-25T11:13:50.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.015+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e8031400 con 0x7f30e0004800 2024-04-25T11:13:50.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.015+0000 7f30f7e00640 1 -- 172.21.15.7:0/151550139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8009690 con 0x7f30e0004800 2024-04-25T11:13:50.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.016+0000 7f30ff000640 1 -- 172.21.15.7:0/151550139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 msgr2=0x7f30e0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.016+0000 7f30ff000640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 0x7f30e0004c10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009a90 tx=0x7f30e0005830 comp rx=0 tx=0).stop 2024-04-25T11:13:50.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.016+0000 7f30ff000640 1 -- 172.21.15.7:0/151550139 shutdown_connections 2024-04-25T11:13:50.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.016+0000 7f30ff000640 1 --2- 172.21.15.7:0/151550139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 0x7f30e0004c10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.016+0000 7f30ff000640 1 -- 172.21.15.7:0/151550139 >> 172.21.15.7:0/151550139 conn(0x7f30f809b540 msgr2=0x7f30f809b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:50.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.016+0000 7f30ff000640 1 -- 172.21.15.7:0/151550139 shutdown_connections 2024-04-25T11:13:50.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.016+0000 7f30ff000640 1 -- 172.21.15.7:0/151550139 wait complete. 2024-04-25T11:13:50.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.018+0000 7f30ff000640 1 Processor -- start 2024-04-25T11:13:50.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.019+0000 7f30ff000640 1 -- start start 2024-04-25T11:13:50.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.020+0000 7f30ff000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 0x7f30f80c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.020+0000 7f30ff000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f80c6ac0 0x7f30f80cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.020+0000 7f30ff000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30f80cd620 0x7f30f80cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.020+0000 7f30ff000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30f80b7420 con 0x7f30e0004800 2024-04-25T11:13:50.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.020+0000 7f30ff000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f30f8160500 con 0x7f30f80c6ac0 2024-04-25T11:13:50.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.020+0000 7f30ff000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f30f80b75a0 con 0x7f30f80cd620 2024-04-25T11:13:50.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.020+0000 7f30fdc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 0x7f30f80c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.020+0000 7f30fdc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 0x7f30f80c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44338/0 (socket says 172.21.15.7:44338) 2024-04-25T11:13:50.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.020+0000 7f30fdc00640 1 -- 172.21.15.7:0/2354901432 learned_addr learned my addr 172.21.15.7:0/2354901432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:50.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.021+0000 7f30fe600640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30f80cd620 0x7f30f80cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.021+0000 7f30fd200640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f80c6ac0 0x7f30f80cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.021+0000 7f30fdc00640 1 -- 172.21.15.7:0/2354901432 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30f80cd620 msgr2=0x7f30f80cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.021+0000 7f30fdc00640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30f80cd620 0x7f30f80cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.021+0000 7f30fdc00640 1 -- 172.21.15.7:0/2354901432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f80c6ac0 msgr2=0x7f30f80cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.021+0000 7f30fdc00640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f80c6ac0 0x7f30f80cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.021+0000 7f30fdc00640 1 -- 172.21.15.7:0/2354901432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f80cdf70 con 0x7f30e0004800 2024-04-25T11:13:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.021+0000 7f30fd200640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f80c6ac0 0x7f30f80cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.021+0000 7f30fdc00640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 0x7f30f80c8ac0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f30ec007300 tx=0x7f30ec0077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.022+0000 7f30f5600640 1 -- 172.21.15.7:0/2354901432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00e950 con 0x7f30e0004800 2024-04-25T11:13:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.022+0000 7f30f5600640 1 -- 172.21.15.7:0/2354901432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30ec018070 con 0x7f30e0004800 2024-04-25T11:13:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.022+0000 7f30f5600640 1 -- 172.21.15.7:0/2354901432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00ed30 con 0x7f30e0004800 2024-04-25T11:13:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.022+0000 7f30ff000640 1 -- 172.21.15.7:0/2354901432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f80c7c80 con 0x7f30e0004800 2024-04-25T11:13:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.022+0000 7f30ff000640 1 -- 172.21.15.7:0/2354901432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f80c8240 con 0x7f30e0004800 2024-04-25T11:13:50.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.025+0000 7f30f5600640 1 -- 172.21.15.7:0/2354901432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f30ec01d460 con 0x7f30e0004800 2024-04-25T11:13:50.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.026+0000 7f30f5600640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f30dc0744e0 0x7f30dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.026+0000 7f30f5600640 1 -- 172.21.15.7:0/2354901432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f30ec0954c0 con 0x7f30e0004800 2024-04-25T11:13:50.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.026+0000 7f30fd200640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f30dc0744e0 0x7f30dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.026+0000 7f31075da740 1 Processor -- start 2024-04-25T11:13:50.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.026+0000 7f31075da740 1 -- start start 2024-04-25T11:13:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.027+0000 7f30fd200640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f30dc0744e0 0x7f30dc0769f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f30e8033000 tx=0x7f30e8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f31075da740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3cd7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f31075da740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3cf1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f31075da740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e9590 0x5568c3cf2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f31075da740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5568c3c10600 con 0x5568c38e13b0 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f31075da740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5568c3bad100 con 0x5568c38e9590 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f31075da740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5568c3c0c5c0 con 0x5568c390a270 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f30fd200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3cf1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f30fd200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3cf1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44352/0 (socket says 172.21.15.7:44352) 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f30fd200640 1 -- 172.21.15.7:0/1967049120 learned_addr learned my addr 172.21.15.7:0/1967049120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f30fe600640 1 --2- 172.21.15.7:0/1967049120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e9590 0x5568c3cf2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f30fdc00640 1 --2- 172.21.15.7:0/1967049120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3cd7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f30fd200640 1 -- 172.21.15.7:0/1967049120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 msgr2=0x5568c3cd7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f30fd200640 1 --2- 172.21.15.7:0/1967049120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3cd7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f30fd200640 1 -- 172.21.15.7:0/1967049120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e9590 msgr2=0x5568c3cf2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.028+0000 7f30fd200640 1 --2- 172.21.15.7:0/1967049120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e9590 0x5568c3cf2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.029+0000 7f30fd200640 1 -- 172.21.15.7:0/1967049120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568c39222d0 con 0x5568c38e13b0 2024-04-25T11:13:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.029+0000 7f30fd200640 1 --2- 172.21.15.7:0/1967049120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3cf1f40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009b80 tx=0x7f30e802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f395d8061a3814 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:50.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.029+0000 7f30e6000640 1 -- 172.21.15.7:0/1967049120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e804d070 con 0x5568c38e13b0 2024-04-25T11:13:50.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.029+0000 7f30e6000640 1 -- 172.21.15.7:0/1967049120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e8003b10 con 0x5568c38e13b0 2024-04-25T11:13:50.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.029+0000 7f30e6000640 1 -- 172.21.15.7:0/1967049120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8036040 con 0x5568c38e13b0 2024-04-25T11:13:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.030+0000 7f31075da740 1 -- 172.21.15.7:0/1967049120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 msgr2=0x5568c3cf1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.030+0000 7f31075da740 1 --2- 172.21.15.7:0/1967049120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3cf1f40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009b80 tx=0x7f30e802e000 comp rx=0 tx=0).stop 2024-04-25T11:13:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.030+0000 7f31075da740 1 -- 172.21.15.7:0/1967049120 shutdown_connections 2024-04-25T11:13:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.030+0000 7f31075da740 1 --2- 172.21.15.7:0/1967049120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e9590 0x5568c3cf2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.030+0000 7f31075da740 1 --2- 172.21.15.7:0/1967049120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3cf1f40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.030+0000 7f31075da740 1 --2- 172.21.15.7:0/1967049120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3cd7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.030+0000 7f31075da740 1 -- 172.21.15.7:0/1967049120 >> 172.21.15.7:0/1967049120 conn(0x5568c38fe3c0 msgr2=0x5568c3cac870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.030+0000 7f31075da740 1 -- 172.21.15.7:0/1967049120 shutdown_connections 2024-04-25T11:13:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.030+0000 7f31075da740 1 -- 172.21.15.7:0/1967049120 wait complete. 2024-04-25T11:13:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.031+0000 7f31075da740 1 -- start start 2024-04-25T11:13:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.032+0000 7f31075da740 1 Processor -- start 2024-04-25T11:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.033+0000 7f31075da740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e13b0 0x5568c3cedc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.033+0000 7f31075da740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c38e9590 0x5568c3cacdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.033+0000 7f31075da740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c390a270 0x5568c3cad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.033+0000 7f31075da740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5568c3b97f70 con 0x5568c390a270 2024-04-25T11:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.033+0000 7f31075da740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5568c3cd95a0 con 0x5568c38e13b0 2024-04-25T11:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.033+0000 7f31075da740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5568c3cee1d0 con 0x5568c38e9590 2024-04-25T11:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.033+0000 7f31075da740 10 client.0 _ll_register_callbacks cb 0x5568c3c06480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.034+0000 7f30fe600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c390a270 0x5568c3cad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.034+0000 7f30fe600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c390a270 0x5568c3cad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44354/0 (socket says 172.21.15.7:44354) 2024-04-25T11:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.034+0000 7f30fe600640 1 -- 172.21.15.7:0/1127278554 learned_addr learned my addr 172.21.15.7:0/1127278554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:50.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.034+0000 7f30fdc00640 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e13b0 0x5568c3cedc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.034+0000 7f30fd200640 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c38e9590 0x5568c3cacdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.034+0000 7f30fe600640 1 -- 172.21.15.7:0/1127278554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c38e9590 msgr2=0x5568c3cacdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.034+0000 7f30fe600640 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c38e9590 0x5568c3cacdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.034+0000 7f30fe600640 1 -- 172.21.15.7:0/1127278554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e13b0 msgr2=0x5568c3cedc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.034+0000 7f30fe600640 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e13b0 0x5568c3cedc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.034+0000 7f30fe600640 1 -- 172.21.15.7:0/1127278554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568c39ec6b0 con 0x5568c390a270 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.035+0000 7f30fe600640 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c390a270 0x5568c3cad2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f30f0002220 tx=0x7f30f00106d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ab05085b053513 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.035+0000 7f30daa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.035+0000 7f30daa00640 1 -- 172.21.15.7:0/1127278554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0003b80 con 0x5568c390a270 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.035+0000 7f30daa00640 1 -- 172.21.15.7:0/1127278554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f30f0003d20 con 0x5568c390a270 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.035+0000 7f31075da740 1 -- 172.21.15.7:0/1127278554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e8003b10 con 0x5568c390a270 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.035+0000 7f30daa00640 1 -- 172.21.15.7:0/1127278554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f30f0006b50 con 0x5568c390a270 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.035+0000 7f31075da740 10 client.5862 fetch_fsmap learned FSMap version 27 2024-04-25T11:13:50.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.035+0000 7f31075da740 1 -- 172.21.15.7:0/1127278554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x5568c3cd2470 con 0x5568c390a270 2024-04-25T11:13:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.035+0000 7f30daa00640 1 -- 172.21.15.7:0/1127278554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e8003b10 con 0x5568c390a270 2024-04-25T11:13:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.036+0000 7f30daa00640 1 -- 172.21.15.7:0/1127278554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f30f00050d0 con 0x5568c390a270 2024-04-25T11:13:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.036+0000 7f31075da740 10 client.5862 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T11:13:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.036+0000 7f31075da740 10 client.5862 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:13:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.036+0000 7f31075da740 20 client.5862 populate_metadata read hostname 'smithi007' 2024-04-25T11:13:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.036+0000 7f31075da740 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.7:6834/3503942462,v1:172.21.15.7:6836/3503942462] conn(0x5568c38c8e40 0x5568c3ce9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.036+0000 7f31075da740 4 client.5862 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:13:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.036+0000 7f31075da740 1 -- 172.21.15.7:0/1127278554 --> [v2:172.21.15.7:6834/3503942462,v1:172.21.15.7:6836/3503942462] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568c3c1e210 con 0x5568c38c8e40 2024-04-25T11:13:50.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.036+0000 7f30fdc00640 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.7:6834/3503942462,v1:172.21.15.7:6836/3503942462] conn(0x5568c38c8e40 0x5568c3ce9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.037+0000 7f30daa00640 10 client.5862 ms_handle_connect on v2:172.21.15.7:6834/3503942462 2024-04-25T11:13:50.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.037+0000 7f30fdc00640 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.7:6834/3503942462,v1:172.21.15.7:6836/3503942462] conn(0x5568c38c8e40 0x5568c3ce9580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=723e5e95706f9525 server_cookie=556e3cf89b12278f in_seq=0 out_seq=0 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.037+0000 7f30daa00640 1 -- 172.21.15.7:0/1127278554 <== mds.0 v2:172.21.15.7:6834/3503942462 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5568c3c1e210 con 0x5568c38c8e40 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.037+0000 7f30daa00640 10 client.5862 handle_command_reply: tid=0 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 1 client.5862 shutdown 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 1 -- 172.21.15.7:0/1127278554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c390a270 msgr2=0x5568c3cad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c390a270 0x5568c3cad2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f30f0002220 tx=0x7f30f00106d0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 1 -- 172.21.15.7:0/1127278554 shutdown_connections 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.7:6834/3503942462,v1:172.21.15.7:6836/3503942462] conn(0x5568c38c8e40 0x5568c3ce9580 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c390a270 0x5568c3cad2f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c38e9590 0x5568c3cacdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 1 --2- 172.21.15.7:0/1127278554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e13b0 0x5568c3cedc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 1 -- 172.21.15.7:0/1127278554 >> 172.21.15.7:0/1127278554 conn(0x5568c38fe3c0 msgr2=0x5568c3cab630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f30daa00640 0 client.5862 ms_handle_reset on v2:172.21.15.7:6834/3503942462 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 1 -- 172.21.15.7:0/1127278554 shutdown_connections 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 1 -- 172.21.15.7:0/1127278554 wait complete. 2024-04-25T11:13:50.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.038+0000 7f31075da740 20 client.5862 trim_cache size 0 max 16384 2024-04-25T11:13:50.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.059+0000 7f31075da740 1 Processor -- start 2024-04-25T11:13:50.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.059+0000 7f31075da740 1 -- start start 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f31075da740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e13b0 0x5568c3cd7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f31075da740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e9590 0x5568c3d152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f31075da740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3ca6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f31075da740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30f8202850 con 0x5568c38e9590 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f31075da740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f30f8201cd0 con 0x5568c38e13b0 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f31075da740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f30f8202200 con 0x5568c390a270 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f30fd200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e9590 0x5568c3d152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f30fd200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e9590 0x5568c3d152f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44362/0 (socket says 172.21.15.7:44362) 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f30fd200640 1 -- 172.21.15.7:0/3719238097 learned_addr learned my addr 172.21.15.7:0/3719238097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f30fdc00640 1 --2- 172.21.15.7:0/3719238097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e13b0 0x5568c3cd7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f30fd200640 1 -- 172.21.15.7:0/3719238097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 msgr2=0x5568c3ca6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f30fe600640 1 --2- 172.21.15.7:0/3719238097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3ca6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.061+0000 7f30fd200640 1 --2- 172.21.15.7:0/3719238097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3ca6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.062+0000 7f30fd200640 1 -- 172.21.15.7:0/3719238097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e13b0 msgr2=0x5568c3cd7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.062+0000 7f30fd200640 1 --2- 172.21.15.7:0/3719238097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e13b0 0x5568c3cd7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.062+0000 7f30fd200640 1 -- 172.21.15.7:0/3719238097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568c3caa8c0 con 0x5568c38e9590 2024-04-25T11:13:50.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.062+0000 7f30fdc00640 1 --2- 172.21.15.7:0/3719238097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e13b0 0x5568c3cd7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:50.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.062+0000 7f30fd200640 1 --2- 172.21.15.7:0/3719238097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e9590 0x5568c3d152f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f30e8047000 tx=0x7f30e803a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bd7c7f1f7e5508 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:50.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.062+0000 7f30e5600640 1 -- 172.21.15.7:0/3719238097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e804d070 con 0x5568c38e9590 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.062+0000 7f30e5600640 1 -- 172.21.15.7:0/3719238097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e8003c50 con 0x5568c38e9590 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.062+0000 7f30e5600640 1 -- 172.21.15.7:0/3719238097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8036040 con 0x5568c38e9590 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.063+0000 7f31075da740 1 -- 172.21.15.7:0/3719238097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e9590 msgr2=0x5568c3d152f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.063+0000 7f31075da740 1 --2- 172.21.15.7:0/3719238097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e9590 0x5568c3d152f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f30e8047000 tx=0x7f30e803a000 comp rx=0 tx=0).stop 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.063+0000 7f31075da740 1 -- 172.21.15.7:0/3719238097 shutdown_connections 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.063+0000 7f31075da740 1 --2- 172.21.15.7:0/3719238097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3ca6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.063+0000 7f31075da740 1 --2- 172.21.15.7:0/3719238097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e9590 0x5568c3d152f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.063+0000 7f31075da740 1 --2- 172.21.15.7:0/3719238097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e13b0 0x5568c3cd7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.063+0000 7f31075da740 1 -- 172.21.15.7:0/3719238097 >> 172.21.15.7:0/3719238097 conn(0x5568c38fe3c0 msgr2=0x5568c3ced4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.063+0000 7f31075da740 1 -- 172.21.15.7:0/3719238097 shutdown_connections 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.063+0000 7f31075da740 1 -- 172.21.15.7:0/3719238097 wait complete. 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.064+0000 7f31075da740 1 -- start start 2024-04-25T11:13:50.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.064+0000 7f31075da740 1 Processor -- start 2024-04-25T11:13:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.066+0000 7f31075da740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3d72640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.066+0000 7f31075da740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e9590 0x5568c3ce8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.066+0000 7f31075da740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3cda5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.066+0000 7f31075da740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5568c3cd8740 con 0x5568c38e13b0 2024-04-25T11:13:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.066+0000 7f31075da740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5568c3ced330 con 0x5568c38e9590 2024-04-25T11:13:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.066+0000 7f31075da740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5568c3cd8d30 con 0x5568c390a270 2024-04-25T11:13:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.066+0000 7f31075da740 10 client.0 _ll_register_callbacks cb 0x7f30f0007e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:13:50.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.066+0000 7f30fdc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3d72640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.066+0000 7f30fdc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3d72640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44364/0 (socket says 172.21.15.7:44364) 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.066+0000 7f30fdc00640 1 -- 172.21.15.7:0/627212764 learned_addr learned my addr 172.21.15.7:0/627212764 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30fe600640 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3cda5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30fd200640 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e9590 0x5568c3ce8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30fdc00640 1 -- 172.21.15.7:0/627212764 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 msgr2=0x5568c3cda5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30fdc00640 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3cda5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30fdc00640 1 -- 172.21.15.7:0/627212764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e9590 msgr2=0x5568c3ce8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30fdc00640 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e9590 0x5568c3ce8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30fdc00640 1 -- 172.21.15.7:0/627212764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568c3d15540 con 0x5568c38e13b0 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30fd200640 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e9590 0x5568c3ce8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30fdc00640 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3d72640 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f30ec05f3d0 tx=0x7f30ec01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddb19a01f739eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30daa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30daa00640 1 -- 172.21.15.7:0/627212764 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec091060 con 0x5568c38e13b0 2024-04-25T11:13:50.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f30daa00640 1 -- 172.21.15.7:0/627212764 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f30ec0950b0 con 0x5568c38e13b0 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.067+0000 7f31075da740 1 -- 172.21.15.7:0/627212764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e8003c50 con 0x5568c38e13b0 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f30daa00640 1 -- 172.21.15.7:0/627212764 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f30ec07d050 con 0x5568c38e13b0 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f31075da740 10 client.5868 fetch_fsmap learned FSMap version 27 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f31075da740 1 -- 172.21.15.7:0/627212764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x5568c3d72b80 con 0x5568c38e13b0 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f30daa00640 1 -- 172.21.15.7:0/627212764 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30e8003c50 con 0x5568c38e13b0 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f30daa00640 1 -- 172.21.15.7:0/627212764 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f30ec078ba0 con 0x5568c38e13b0 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f31075da740 10 client.5868 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f31075da740 10 client.5868 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f31075da740 20 client.5868 populate_metadata read hostname 'smithi007' 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f31075da740 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.7:6834/3503942462,v1:172.21.15.7:6836/3503942462] conn(0x5568c3cdaca0 0x5568c3cf1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f31075da740 4 client.5868 mds_command: new command op to 5742 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f31075da740 1 -- 172.21.15.7:0/627212764 --> [v2:172.21.15.7:6834/3503942462,v1:172.21.15.7:6836/3503942462] -- command(tid 0: {"prefix": "status"}) -- 0x5568c3c18080 con 0x5568c3cdaca0 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.068+0000 7f30fd200640 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.7:6834/3503942462,v1:172.21.15.7:6836/3503942462] conn(0x5568c3cdaca0 0x5568c3cf1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.069+0000 7f30fd200640 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.7:6834/3503942462,v1:172.21.15.7:6836/3503942462] conn(0x5568c3cdaca0 0x5568c3cf1040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d1935a9aff0cc server_cookie=4308c308dc570550 in_seq=0 out_seq=0 2024-04-25T11:13:50.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.069+0000 7f30daa00640 10 client.5868 ms_handle_connect on v2:172.21.15.7:6834/3503942462 2024-04-25T11:13:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.069+0000 7f30daa00640 1 -- 172.21.15.7:0/627212764 <== mds.0 v2:172.21.15.7:6834/3503942462 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5568c3c18080 con 0x5568c3cdaca0 2024-04-25T11:13:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.069+0000 7f30daa00640 10 client.5868 handle_command_reply: tid=0 2024-04-25T11:13:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.069+0000 7f31075da740 1 client.5868 shutdown 2024-04-25T11:13:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.069+0000 7f31075da740 1 -- 172.21.15.7:0/627212764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 msgr2=0x5568c3d72640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.069+0000 7f31075da740 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3d72640 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f30ec05f3d0 tx=0x7f30ec01f040 comp rx=0 tx=0).stop 2024-04-25T11:13:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f31075da740 1 -- 172.21.15.7:0/627212764 shutdown_connections 2024-04-25T11:13:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f31075da740 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.7:6834/3503942462,v1:172.21.15.7:6836/3503942462] conn(0x5568c3cdaca0 0x5568c3cf1040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f31075da740 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5568c390a270 0x5568c3cda5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f31075da740 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5568c38e9590 0x5568c3ce8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f31075da740 1 --2- 172.21.15.7:0/627212764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5568c38e13b0 0x5568c3d72640 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f31075da740 1 -- 172.21.15.7:0/627212764 >> 172.21.15.7:0/627212764 conn(0x5568c38fe3c0 msgr2=0x5568c3ca63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:50.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f30daa00640 0 client.5868 ms_handle_reset on v2:172.21.15.7:6834/3503942462 2024-04-25T11:13:50.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f31075da740 1 -- 172.21.15.7:0/627212764 shutdown_connections 2024-04-25T11:13:50.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f31075da740 1 -- 172.21.15.7:0/627212764 wait complete. 2024-04-25T11:13:50.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f31075da740 20 client.5868 trim_cache size 0 max 16384 2024-04-25T11:13:50.075 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:13:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:13:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:13:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5742, 2024-04-25T11:13:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:13:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:13:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:13:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.3410129739999999, 2024-04-25T11:13:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 27, 2024-04-25T11:13:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 54, 2024-04-25T11:13:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 54, 2024-04-25T11:13:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.497264980000001 2024-04-25T11:13:50.076 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f30ff000640 1 -- 172.21.15.7:0/2354901432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f30dc0744e0 msgr2=0x7f30dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f30ff000640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f30dc0744e0 0x7f30dc0769f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f30e8033000 tx=0x7f30e8008030 comp rx=0 tx=0).stop 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f30ff000640 1 -- 172.21.15.7:0/2354901432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 msgr2=0x7f30f80c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.070+0000 7f30ff000640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 0x7f30f80c8ac0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f30ec007300 tx=0x7f30ec0077c0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.071+0000 7f30ff000640 1 -- 172.21.15.7:0/2354901432 shutdown_connections 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.071+0000 7f30ff000640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f30dc0744e0 0x7f30dc0769f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.071+0000 7f30ff000640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30f80cd620 0x7f30f80cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.071+0000 7f30ff000640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f80c6ac0 0x7f30f80cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.071+0000 7f30ff000640 1 --2- 172.21.15.7:0/2354901432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30e0004800 0x7f30f80c8ac0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.071+0000 7f30ff000640 1 -- 172.21.15.7:0/2354901432 >> 172.21.15.7:0/2354901432 conn(0x7f30f809b540 msgr2=0x7f30f80ce470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.071+0000 7f30ff000640 1 -- 172.21.15.7:0/2354901432 shutdown_connections 2024-04-25T11:13:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.071+0000 7f30ff000640 1 -- 172.21.15.7:0/2354901432 wait complete. 2024-04-25T11:13:50.100 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.378+0000 7f39b0200640 1 Processor -- start 2024-04-25T11:13:50.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.379+0000 7f39b0200640 1 -- start start 2024-04-25T11:13:50.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.381+0000 7f39b0200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f39a815f180 0x7f39a8161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.381+0000 7f39b0200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f39a806dc90 0x7f39a806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.381+0000 7f39b0200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a8162700 con 0x7f39a81621a0 2024-04-25T11:13:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.381+0000 7f39b0200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f39a815f590 con 0x7f39a806dc90 2024-04-25T11:13:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.381+0000 7f39b0200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f39a8158ae0 con 0x7f39a815f180 2024-04-25T11:13:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.382+0000 7f39ace00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f39a81621a0 0x7f39a806d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55534/0 (socket says 172.21.15.7:55534) 2024-04-25T11:13:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.382+0000 7f39ace00640 1 -- 172.21.15.7:0/3200051374 learned_addr learned my addr 172.21.15.7:0/3200051374 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.382+0000 7f39ae200640 1 --2- 172.21.15.7:0/3200051374 >> v2:172.21.15.7:3300/0 conn(0x7f39a806dc90 0x7f39a806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.382+0000 7f39ad800640 1 --2- 172.21.15.7:0/3200051374 >> v2:172.21.15.87:3300/0 conn(0x7f39a815f180 0x7f39a8161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.382+0000 7f39a7e00640 1 -- 172.21.15.7:0/3200051374 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279856874 0 0) 0x7f39a8162700 con 0x7f39a81621a0 2024-04-25T11:13:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.382+0000 7f39a7e00640 1 -- 172.21.15.7:0/3200051374 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3990003520 con 0x7f39a81621a0 2024-04-25T11:13:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.382+0000 7f39ae200640 1 -- 172.21.15.7:0/3200051374 >> v2:172.21.15.87:3300/0 conn(0x7f39a815f180 msgr2=0x7f39a8161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.382+0000 7f39ae200640 1 --2- 172.21.15.7:0/3200051374 >> v2:172.21.15.87:3300/0 conn(0x7f39a815f180 0x7f39a8161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.382+0000 7f39ae200640 1 -- 172.21.15.7:0/3200051374 >> v1:172.21.15.7:6789/0 conn(0x7f39a81621a0 legacy=0x7f39a806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.382+0000 7f39ae200640 1 -- 172.21.15.7:0/3200051374 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a81629e0 con 0x7f39a806dc90 2024-04-25T11:13:50.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.383+0000 7f39ae200640 1 --2- 172.21.15.7:0/3200051374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a806dc90 0x7f39a806f1e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f39a8070340 tx=0x7f39a002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9da0bcaed3fc4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:50.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.384+0000 7f39a7e00640 1 -- 172.21.15.7:0/3200051374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0031600 con 0x7f39a806dc90 2024-04-25T11:13:50.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.384+0000 7f39a7e00640 1 -- 172.21.15.7:0/3200051374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a00317a0 con 0x7f39a806dc90 2024-04-25T11:13:50.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.384+0000 7f39a7e00640 1 -- 172.21.15.7:0/3200051374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0031b70 con 0x7f39a806dc90 2024-04-25T11:13:50.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.384+0000 7f39b0200640 1 -- 172.21.15.7:0/3200051374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a806dc90 msgr2=0x7f39a806f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.384+0000 7f39b0200640 1 --2- 172.21.15.7:0/3200051374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a806dc90 0x7f39a806f1e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f39a8070340 tx=0x7f39a002a610 comp rx=0 tx=0).stop 2024-04-25T11:13:50.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.385+0000 7f39b0200640 1 -- 172.21.15.7:0/3200051374 shutdown_connections 2024-04-25T11:13:50.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.385+0000 7f39b0200640 1 --2- 172.21.15.7:0/3200051374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a806dc90 0x7f39a806f1e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.385+0000 7f39b0200640 1 --2- 172.21.15.7:0/3200051374 >> v2:172.21.15.87:3300/0 conn(0x7f39a815f180 0x7f39a8161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.385+0000 7f39b0200640 1 -- 172.21.15.7:0/3200051374 >> 172.21.15.7:0/3200051374 conn(0x7f39a80732b0 msgr2=0x7f39a80736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:50.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.385+0000 7f39b0200640 1 -- 172.21.15.7:0/3200051374 shutdown_connections 2024-04-25T11:13:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.385+0000 7f39b0200640 1 -- 172.21.15.7:0/3200051374 wait complete. 2024-04-25T11:13:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.387+0000 7f39b0200640 1 Processor -- start 2024-04-25T11:13:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.387+0000 7f39b0200640 1 -- start start 2024-04-25T11:13:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.389+0000 7f39b0200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815f180 0x7f39a809b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.389+0000 7f39b0200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39a81621a0 0x7f39a81f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.389+0000 7f39b0200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39a8143370 0x7f39a81f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.389+0000 7f39b0200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39a815e670 con 0x7f39a815f180 2024-04-25T11:13:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.389+0000 7f39b0200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f39a815ec60 con 0x7f39a81621a0 2024-04-25T11:13:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.389+0000 7f39b0200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f39a80d1310 con 0x7f39a8143370 2024-04-25T11:13:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.389+0000 7f39ad800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815f180 0x7f39a809b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.389+0000 7f39ad800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815f180 0x7f39a809b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44392/0 (socket says 172.21.15.7:44392) 2024-04-25T11:13:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.389+0000 7f39ad800640 1 -- 172.21.15.7:0/2630909736 learned_addr learned my addr 172.21.15.7:0/2630909736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:50.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.389+0000 7f39ace00640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39a81621a0 0x7f39a81f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.389+0000 7f39ae200640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39a8143370 0x7f39a81f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39ad800640 1 -- 172.21.15.7:0/2630909736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39a8143370 msgr2=0x7f39a81f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39ad800640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39a8143370 0x7f39a81f7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39ad800640 1 -- 172.21.15.7:0/2630909736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39a81621a0 msgr2=0x7f39a81f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39ad800640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39a81621a0 0x7f39a81f7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39ad800640 1 -- 172.21.15.7:0/2630909736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a809d4e0 con 0x7f39a815f180 2024-04-25T11:13:50.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39ace00640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39a81621a0 0x7f39a81f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:50.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39ad800640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815f180 0x7f39a809b540 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f399c00d600 tx=0x7f399c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:50.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39a5600640 1 -- 172.21.15.7:0/2630909736 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c004240 con 0x7f39a815f180 2024-04-25T11:13:50.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39a5600640 1 -- 172.21.15.7:0/2630909736 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f399c0043e0 con 0x7f39a815f180 2024-04-25T11:13:50.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39a5600640 1 -- 172.21.15.7:0/2630909736 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c003830 con 0x7f39a815f180 2024-04-25T11:13:50.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39b0200640 1 -- 172.21.15.7:0/2630909736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a809d7c0 con 0x7f39a815f180 2024-04-25T11:13:50.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.390+0000 7f39b0200640 1 -- 172.21.15.7:0/2630909736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a809dd80 con 0x7f39a815f180 2024-04-25T11:13:50.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.392+0000 7f39b0200640 1 -- 172.21.15.7:0/2630909736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a8050740 con 0x7f39a815f180 2024-04-25T11:13:50.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.393+0000 7f39a5600640 1 -- 172.21.15.7:0/2630909736 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f399c0039d0 con 0x7f39a815f180 2024-04-25T11:13:50.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.394+0000 7f39a5600640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39880744e0 0x7f39880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:50.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.394+0000 7f39a5600640 1 -- 172.21.15.7:0/2630909736 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f399c0956c0 con 0x7f39a815f180 2024-04-25T11:13:50.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.394+0000 7f39ace00640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39880744e0 0x7f39880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:50.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.395+0000 7f39ace00640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39880744e0 0x7f39880769f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f39a806d7a0 tx=0x7f3994009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:50.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.399+0000 7f39a5600640 1 -- 172.21.15.7:0/2630909736 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f399c05fcb0 con 0x7f39a815f180 2024-04-25T11:13:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.644+0000 7f39b0200640 1 -- 172.21.15.7:0/2630909736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f39a80cdbd0 con 0x7f39a815f180 2024-04-25T11:13:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.645+0000 7f39a5600640 1 -- 172.21.15.7:0/2630909736 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f399c05f5b0 con 0x7f39a815f180 2024-04-25T11:13:50.646 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:13:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.647+0000 7f39b0200640 1 -- 172.21.15.7:0/2630909736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39880744e0 msgr2=0x7f39880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.647+0000 7f39b0200640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39880744e0 0x7f39880769f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f39a806d7a0 tx=0x7f3994009040 comp rx=0 tx=0).stop 2024-04-25T11:13:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.647+0000 7f39b0200640 1 -- 172.21.15.7:0/2630909736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815f180 msgr2=0x7f39a809b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.647+0000 7f39b0200640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815f180 0x7f39a809b540 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f399c00d600 tx=0x7f399c00dac0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.647+0000 7f39b0200640 1 -- 172.21.15.7:0/2630909736 shutdown_connections 2024-04-25T11:13:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.647+0000 7f39b0200640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39880744e0 0x7f39880769f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.647+0000 7f39b0200640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39a8143370 0x7f39a81f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.647+0000 7f39b0200640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39a81621a0 0x7f39a81f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.647+0000 7f39b0200640 1 --2- 172.21.15.7:0/2630909736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a815f180 0x7f39a809b540 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.647+0000 7f39b0200640 1 -- 172.21.15.7:0/2630909736 >> 172.21.15.7:0/2630909736 conn(0x7f39a80732b0 msgr2=0x7f39a8157100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.648+0000 7f39b0200640 1 -- 172.21.15.7:0/2630909736 shutdown_connections 2024-04-25T11:13:50.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50.648+0000 7f39b0200640 1 -- 172.21.15.7:0/2630909736 wait complete. 2024-04-25T11:13:50.680 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:13:50.680 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:13:50.680 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:13:50.680 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:13:50.680 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:13:50.680 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:13:50.680 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:13:50.680 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:13:50.681 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:13:50.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:50.681 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:13:50.736 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:13:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:13:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:13:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:13:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:50.737 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:13:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:13:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:13:50.737 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:13:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:13:50.737 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42344sec preferred_lft 42344sec 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:13:50.738 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6098:c7ff:fe8a:4a30/64 scope link 2024-04-25T11:13:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:50.740 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:13:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:13:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::646a:35ff:fe7b:ad0/64 scope link 2024-04-25T11:13:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:50.740 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:13:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:13:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4465:c4ff:fe62:f75f/64 scope link 2024-04-25T11:13:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:50.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:50.741 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:13:50.826 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:13:50.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:50.831 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:13:50.885 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:13:50.886 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:13:50.886 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:13:50.886 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:13:50.886 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:13:50.887 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:13:50.887 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:13:50.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:13:50.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:13:50.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:13:50.887 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:13:50.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:13:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:13:51.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:51.112 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:13:51.164 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:13:51.164 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:13:51.164 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:13:51.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:51.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:13:51.247 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:13:51.247 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:13:51.306 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:13:51.306 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:13:51.307 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:13:51.358 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:13:51.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:51.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:13:51.567 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:13:51.567 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:13:51.607 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:13:51.608 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:13:51.608 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:13:51.608 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:13:51.608 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:13:51.610 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:13:51.610 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:13:51.610 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:13:51.610 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:13:51.610 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:13:51.613 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:13:51.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:51.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:13:51.688 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:13:51.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:13:51.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:13:51.833 INFO:teuthology.orchestra.run.smithi007.stdout:client5880 2024-04-25T11:13:51.833 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:13:51.833 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:51.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:13:52.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.216+0000 7f183ca00640 1 Processor -- start 2024-04-25T11:13:52.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.216+0000 7f183ca00640 1 -- start start 2024-04-25T11:13:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.219+0000 7f183ca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f183815f180 0x7f1838161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.219+0000 7f183ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f183806dc90 0x7f183806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.219+0000 7f183ca00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1838162700 con 0x7f18381621a0 2024-04-25T11:13:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.219+0000 7f183ca00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f183815f590 con 0x7f183806dc90 2024-04-25T11:13:52.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.219+0000 7f183ca00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1838158ae0 con 0x7f183815f180 2024-04-25T11:13:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.220+0000 7f1836800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f183806dc90 0x7f183806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:52.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.220+0000 7f1836800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f183806dc90 0x7f183806f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44406/0 (socket says 172.21.15.7:44406) 2024-04-25T11:13:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.220+0000 7f1836800640 1 -- 172.21.15.7:0/1348593588 learned_addr learned my addr 172.21.15.7:0/1348593588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.220+0000 7f1835e00640 1 --2- 172.21.15.7:0/1348593588 >> v2:172.21.15.87:3300/0 conn(0x7f183815f180 0x7f1838161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.220+0000 7f1836800640 1 -- 172.21.15.7:0/1348593588 >> v2:172.21.15.87:3300/0 conn(0x7f183815f180 msgr2=0x7f1838161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.220+0000 7f1836800640 1 --2- 172.21.15.7:0/1348593588 >> v2:172.21.15.87:3300/0 conn(0x7f183815f180 0x7f1838161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.220+0000 7f1836800640 1 -- 172.21.15.7:0/1348593588 >> v1:172.21.15.87:6789/0 conn(0x7f18381621a0 legacy=0x7f183806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.220+0000 7f1836800640 1 -- 172.21.15.7:0/1348593588 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18381629e0 con 0x7f183806dc90 2024-04-25T11:13:52.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.220+0000 7f1835e00640 1 --2- 172.21.15.7:0/1348593588 >> v2:172.21.15.87:3300/0 conn(0x7f183815f180 0x7f1838161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:13:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.221+0000 7f1836800640 1 --2- 172.21.15.7:0/1348593588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f183806dc90 0x7f183806f1e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1838070340 tx=0x7f183002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e10c76226e32598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.222+0000 7f1835400640 1 -- 172.21.15.7:0/1348593588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830036070 con 0x7f183806dc90 2024-04-25T11:13:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.222+0000 7f1835400640 1 -- 172.21.15.7:0/1348593588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1830031600 con 0x7f183806dc90 2024-04-25T11:13:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.222+0000 7f1835400640 1 -- 172.21.15.7:0/1348593588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18300319d0 con 0x7f183806dc90 2024-04-25T11:13:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.223+0000 7f183ca00640 1 -- 172.21.15.7:0/1348593588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f183806dc90 msgr2=0x7f183806f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.223+0000 7f183ca00640 1 --2- 172.21.15.7:0/1348593588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f183806dc90 0x7f183806f1e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1838070340 tx=0x7f183002a6a0 comp rx=0 tx=0).stop 2024-04-25T11:13:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.223+0000 7f183ca00640 1 -- 172.21.15.7:0/1348593588 shutdown_connections 2024-04-25T11:13:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.223+0000 7f183ca00640 1 --2- 172.21.15.7:0/1348593588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f183806dc90 0x7f183806f1e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.223+0000 7f183ca00640 1 --2- 172.21.15.7:0/1348593588 >> v2:172.21.15.87:3300/0 conn(0x7f183815f180 0x7f1838161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.223+0000 7f183ca00640 1 -- 172.21.15.7:0/1348593588 >> 172.21.15.7:0/1348593588 conn(0x7f18380732b0 msgr2=0x7f18380736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.223+0000 7f183ca00640 1 -- 172.21.15.7:0/1348593588 shutdown_connections 2024-04-25T11:13:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.223+0000 7f183ca00640 1 -- 172.21.15.7:0/1348593588 wait complete. 2024-04-25T11:13:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.225+0000 7f183ca00640 1 Processor -- start 2024-04-25T11:13:52.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.225+0000 7f183ca00640 1 -- start start 2024-04-25T11:13:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.227+0000 7f183ca00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f183815f180 0x7f18381509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.227+0000 7f183ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18381621a0 0x7f18381480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.227+0000 7f183ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18381f9670 0x7f18381f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.227+0000 7f183ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f183815e670 con 0x7f18381621a0 2024-04-25T11:13:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.227+0000 7f183ca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f183815ec60 con 0x7f18381f9670 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.227+0000 7f183ca00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18380d1310 con 0x7f183815f180 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.227+0000 7f182fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18381621a0 0x7f18381480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.227+0000 7f182fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18381621a0 0x7f18381480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44410/0 (socket says 172.21.15.7:44410) 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.227+0000 7f182fe00640 1 -- 172.21.15.7:0/2306229303 learned_addr learned my addr 172.21.15.7:0/2306229303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.228+0000 7f1835e00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f183815f180 0x7f18381509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.228+0000 7f1836800640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18381f9670 0x7f18381f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.228+0000 7f182fe00640 1 -- 172.21.15.7:0/2306229303 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f183815f180 msgr2=0x7f18381509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.228+0000 7f182fe00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f183815f180 0x7f18381509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.228+0000 7f182fe00640 1 -- 172.21.15.7:0/2306229303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18381f9670 msgr2=0x7f18381f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.228+0000 7f182fe00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18381f9670 0x7f18381f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.228+0000 7f182fe00640 1 -- 172.21.15.7:0/2306229303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18381f9fc0 con 0x7f18381621a0 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.228+0000 7f182fe00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18381621a0 0x7f18381480a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f182000b3f0 tx=0x7f182000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.229+0000 7f182e000640 1 -- 172.21.15.7:0/2306229303 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820015020 con 0x7f18381621a0 2024-04-25T11:13:52.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.229+0000 7f182e000640 1 -- 172.21.15.7:0/2306229303 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1820011830 con 0x7f18381621a0 2024-04-25T11:13:52.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.229+0000 7f183ca00640 1 -- 172.21.15.7:0/2306229303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1838156e30 con 0x7f18381621a0 2024-04-25T11:13:52.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.229+0000 7f182e000640 1 -- 172.21.15.7:0/2306229303 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820011c20 con 0x7f18381621a0 2024-04-25T11:13:52.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.229+0000 7f183ca00640 1 -- 172.21.15.7:0/2306229303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183815fc30 con 0x7f18381621a0 2024-04-25T11:13:52.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.230+0000 7f183ca00640 1 -- 172.21.15.7:0/2306229303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1838157000 con 0x7f18381621a0 2024-04-25T11:13:52.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.231+0000 7f182e000640 1 -- 172.21.15.7:0/2306229303 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1820018020 con 0x7f18381621a0 2024-04-25T11:13:52.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.233+0000 7f182e000640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18140744e0 0x7f18140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:52.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.233+0000 7f182e000640 1 -- 172.21.15.7:0/2306229303 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f18200950b0 con 0x7f18381621a0 2024-04-25T11:13:52.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.234+0000 7f1835e00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18140744e0 0x7f18140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:52.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.234+0000 7f1835e00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18140744e0 0x7f18140769f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1828006fd0 tx=0x7f1828008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:52.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.239+0000 7f182e000640 1 -- 172.21.15.7:0/2306229303 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1838157000 con 0x7f18381621a0 2024-04-25T11:13:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.485+0000 7f183ca00640 1 -- 172.21.15.7:0/2306229303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1838050740 con 0x7f18381621a0 2024-04-25T11:13:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.486+0000 7f182e000640 1 -- 172.21.15.7:0/2306229303 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f182001f030 con 0x7f18381621a0 2024-04-25T11:13:52.487 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:13:52.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 -- 172.21.15.7:0/2306229303 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18140744e0 msgr2=0x7f18140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18140744e0 0x7f18140769f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1828006fd0 tx=0x7f1828008040 comp rx=0 tx=0).stop 2024-04-25T11:13:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 -- 172.21.15.7:0/2306229303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18381621a0 msgr2=0x7f18381480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18381621a0 0x7f18381480a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f182000b3f0 tx=0x7f182000b8b0 comp rx=0 tx=0).stop 2024-04-25T11:13:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 -- 172.21.15.7:0/2306229303 shutdown_connections 2024-04-25T11:13:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18140744e0 0x7f18140769f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18381f9670 0x7f18381f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18381621a0 0x7f18381480a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 --2- 172.21.15.7:0/2306229303 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f183815f180 0x7f18381509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 -- 172.21.15.7:0/2306229303 >> 172.21.15.7:0/2306229303 conn(0x7f18380732b0 msgr2=0x7f183815dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 -- 172.21.15.7:0/2306229303 shutdown_connections 2024-04-25T11:13:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:52.489+0000 7f183ca00640 1 -- 172.21.15.7:0/2306229303 wait complete. 2024-04-25T11:13:52.520 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:13:52.520 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:13:52.520 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:13:52.520 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:13:52.520 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:13:52.520 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:13:52.520 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:13:52.520 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:13:52.521 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:13:52.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:52.521 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:13:52.579 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:13:52.579 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:13:52.579 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:13:52.579 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:13:52.580 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42343sec preferred_lft 42343sec 2024-04-25T11:13:52.581 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:13:52.581 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:52.581 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:13:52.581 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:13:52.581 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:13:52.581 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:52.581 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::58f2:6bff:fe8b:8b4c/64 scope link 2024-04-25T11:13:52.581 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:52.581 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:13:52.581 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:13:52.581 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::98fe:1bff:fe9f:d71c/64 scope link 2024-04-25T11:13:52.582 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:52.582 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:13:52.582 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:13:52.582 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::3c38:a8ff:fec7:ea00/64 scope link 2024-04-25T11:13:52.582 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:13:52.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:52.583 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:13:52.667 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:13:52.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:52.672 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:13:52.725 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:13:52.726 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:13:52.726 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:13:52.726 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:13:52.726 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:13:52.727 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:13:52.727 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:13:52.727 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:13:52.727 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:13:52.728 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:13:52.728 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:13:52.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:13:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:13:52.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:13:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:13:52.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:52.941 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:13:52.994 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:13:52.994 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:13:52.995 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:13:52.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:52.995 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T11:13:53.077 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:13:53.077 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:13:53.131 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:13:53.133 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:13:53.133 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:13:53.183 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:13:53.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:53.184 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:13:53.388 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:13:53.388 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:13:53.440 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:13:53.441 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:13:53.441 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:13:53.441 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 1@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:13:53.441 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:13:53.443 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:13:53.443 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:13:53.443 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:13:53.443 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:13:53.443 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:13:53.448 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:13:53.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:53.448 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:13:53.522 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:13:53.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:13:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:13:53.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:13:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:13:53.640 INFO:teuthology.orchestra.run.smithi087.stdout:client5090 2024-04-25T11:13:53.640 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:13:53.641 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:53.692 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:13:53.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:13:53.805 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:13:53.915 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:13:53.915 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:13:53.915 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:13:53.915 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:53.978 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:13:53.978 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:54.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.322+0000 7f8d53200640 1 Processor -- start 2024-04-25T11:13:54.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.322+0000 7f8d53200640 1 -- start start 2024-04-25T11:13:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.324+0000 7f8d53200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8d4c0d1540 0x7f8d4c0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.324+0000 7f8d53200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d4c0cfb10 0x7f8d4c0b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.324+0000 7f8d53200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d4c0ce7b0 con 0x7f8d4c09ba00 2024-04-25T11:13:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.325+0000 7f8d53200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8d4c0cea70 con 0x7f8d4c0d1540 2024-04-25T11:13:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.325+0000 7f8d53200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8d4c0ced50 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.325+0000 7f8d51e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8d4c0d1540 0x7f8d4c0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.325+0000 7f8d52800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8d4c09ba00 0x7f8d4c09be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55536/0 (socket says 172.21.15.7:55536) 2024-04-25T11:13:54.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.325+0000 7f8d52800640 1 -- 172.21.15.7:0/1392878957 learned_addr learned my addr 172.21.15.7:0/1392878957 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.326+0000 7f8d51400640 1 --2- 172.21.15.7:0/1392878957 >> v2:172.21.15.87:3300/0 conn(0x7f8d4c0cfb10 0x7f8d4c0b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.326+0000 7f8d50a00640 1 -- 172.21.15.7:0/1392878957 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639507883 0 0) 0x7f8d4c0ce7b0 con 0x7f8d4c09ba00 2024-04-25T11:13:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.326+0000 7f8d50a00640 1 -- 172.21.15.7:0/1392878957 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d34003520 con 0x7f8d4c09ba00 2024-04-25T11:13:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.326+0000 7f8d51e00640 1 -- 172.21.15.7:0/1392878957 >> v2:172.21.15.87:3300/0 conn(0x7f8d4c0cfb10 msgr2=0x7f8d4c0b8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.326+0000 7f8d51e00640 1 --2- 172.21.15.7:0/1392878957 >> v2:172.21.15.87:3300/0 conn(0x7f8d4c0cfb10 0x7f8d4c0b8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.326+0000 7f8d51e00640 1 -- 172.21.15.7:0/1392878957 >> v1:172.21.15.7:6789/0 conn(0x7f8d4c09ba00 legacy=0x7f8d4c09be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.326+0000 7f8d51e00640 1 -- 172.21.15.7:0/1392878957 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c0cf030 con 0x7f8d4c0d1540 2024-04-25T11:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.327+0000 7f8d51e00640 1 --2- 172.21.15.7:0/1392878957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c0d1540 0x7f8d4c0ce270 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0cd980 tx=0x7f8d4002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a44ffa346fbc9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.327+0000 7f8d50a00640 1 -- 172.21.15.7:0/1392878957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40037070 con 0x7f8d4c0d1540 2024-04-25T11:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.327+0000 7f8d50a00640 1 -- 172.21.15.7:0/1392878957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d40032440 con 0x7f8d4c0d1540 2024-04-25T11:13:54.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.327+0000 7f8d50a00640 1 -- 172.21.15.7:0/1392878957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4002ae60 con 0x7f8d4c0d1540 2024-04-25T11:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.328+0000 7f8d53200640 1 -- 172.21.15.7:0/1392878957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c0d1540 msgr2=0x7f8d4c0ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.328+0000 7f8d53200640 1 --2- 172.21.15.7:0/1392878957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c0d1540 0x7f8d4c0ce270 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0cd980 tx=0x7f8d4002a110 comp rx=0 tx=0).stop 2024-04-25T11:13:54.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.328+0000 7f8d53200640 1 -- 172.21.15.7:0/1392878957 shutdown_connections 2024-04-25T11:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.328+0000 7f8d53200640 1 --2- 172.21.15.7:0/1392878957 >> v2:172.21.15.87:3300/0 conn(0x7f8d4c0cfb10 0x7f8d4c0b8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.328+0000 7f8d53200640 1 --2- 172.21.15.7:0/1392878957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c0d1540 0x7f8d4c0ce270 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.328+0000 7f8d53200640 1 -- 172.21.15.7:0/1392878957 >> 172.21.15.7:0/1392878957 conn(0x7f8d4c0b5250 msgr2=0x7f8d4c0b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.328+0000 7f8d53200640 1 -- 172.21.15.7:0/1392878957 shutdown_connections 2024-04-25T11:13:54.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.328+0000 7f8d53200640 1 -- 172.21.15.7:0/1392878957 wait complete. 2024-04-25T11:13:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.331+0000 7f8d53200640 1 Processor -- start 2024-04-25T11:13:54.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.331+0000 7f8d53200640 1 -- start start 2024-04-25T11:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.332+0000 7f8d53200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d4c09ba00 0x7f8d4c1563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.332+0000 7f8d53200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c0cfb10 0x7f8d4c09d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.332+0000 7f8d53200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d4c0d1540 0x7f8d4c1f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d53200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c0d2680 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d53200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c0b9260 con 0x7f8d4c09ba00 2024-04-25T11:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d53200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8d4c0b65b0 con 0x7f8d4c0d1540 2024-04-25T11:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d51400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c0cfb10 0x7f8d4c09d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d51400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c0cfb10 0x7f8d4c09d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44416/0 (socket says 172.21.15.7:44416) 2024-04-25T11:13:54.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d51400640 1 -- 172.21.15.7:0/3326527120 learned_addr learned my addr 172.21.15.7:0/3326527120 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d51e00640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d4c09ba00 0x7f8d4c1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d51400640 1 -- 172.21.15.7:0/3326527120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d4c0d1540 msgr2=0x7f8d4c1f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d52800640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d4c0d1540 0x7f8d4c1f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d51400640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d4c0d1540 0x7f8d4c1f7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d51400640 1 -- 172.21.15.7:0/3326527120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d4c09ba00 msgr2=0x7f8d4c1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d51400640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d4c09ba00 0x7f8d4c1563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.333+0000 7f8d51400640 1 -- 172.21.15.7:0/3326527120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c152ae0 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.334+0000 7f8d51400640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c0cfb10 0x7f8d4c09d560 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00d620 tx=0x7f8d3c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.334+0000 7f8d46000640 1 -- 172.21.15.7:0/3326527120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00b620 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.334+0000 7f8d46000640 1 -- 172.21.15.7:0/3326527120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3c019070 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.334+0000 7f8d46000640 1 -- 172.21.15.7:0/3326527120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00ba00 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.334+0000 7f8d53200640 1 -- 172.21.15.7:0/3326527120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4c152dc0 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.334+0000 7f8d53200640 1 -- 172.21.15.7:0/3326527120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4c09dba0 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.336+0000 7f8d53200640 1 -- 172.21.15.7:0/3326527120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4c050740 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.337+0000 7f8d46000640 1 -- 172.21.15.7:0/3326527120 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8d3c00bba0 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.338+0000 7f8d46000640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d300744b0 0x7f8d300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.339+0000 7f8d46000640 1 -- 172.21.15.7:0/3326527120 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f8d3c095800 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.339+0000 7f8d51e00640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d300744b0 0x7f8d300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.339+0000 7f8d51e00640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d300744b0 0x7f8d300769c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8d40034000 tx=0x7f8d4002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:54.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.344+0000 7f8d46000640 1 -- 172.21.15.7:0/3326527120 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d3c05fdf0 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.553+0000 7f8d53200640 1 -- 172.21.15.7:0/3326527120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d4c152870 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.554+0000 7f8d46000640 1 -- 172.21.15.7:0/3326527120 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16846 (secure 0 0 0) 0x7f8d3c05f6f0 con 0x7f8d4c0cfb10 2024-04-25T11:13:54.556 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:54.556 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:43.726826+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:13:40.446828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T11:13:41.263101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.557+0000 7f8d53200640 1 -- 172.21.15.7:0/3326527120 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d300744b0 msgr2=0x7f8d300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.557+0000 7f8d53200640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d300744b0 0x7f8d300769c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8d40034000 tx=0x7f8d4002e000 comp rx=0 tx=0).stop 2024-04-25T11:13:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.557+0000 7f8d53200640 1 -- 172.21.15.7:0/3326527120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c0cfb10 msgr2=0x7f8d4c09d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.557+0000 7f8d53200640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c0cfb10 0x7f8d4c09d560 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00d620 tx=0x7f8d3c00dae0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.557+0000 7f8d53200640 1 -- 172.21.15.7:0/3326527120 shutdown_connections 2024-04-25T11:13:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.557+0000 7f8d53200640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d300744b0 0x7f8d300769c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.558+0000 7f8d53200640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d4c0d1540 0x7f8d4c1f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.558+0000 7f8d53200640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d4c0cfb10 0x7f8d4c09d560 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.558+0000 7f8d53200640 1 --2- 172.21.15.7:0/3326527120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d4c09ba00 0x7f8d4c1563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.558+0000 7f8d53200640 1 -- 172.21.15.7:0/3326527120 >> 172.21.15.7:0/3326527120 conn(0x7f8d4c0b5250 msgr2=0x7f8d4c0d19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:54.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.558+0000 7f8d53200640 1 -- 172.21.15.7:0/3326527120 shutdown_connections 2024-04-25T11:13:54.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.558+0000 7f8d53200640 1 -- 172.21.15.7:0/3326527120 wait complete. 2024-04-25T11:13:54.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:13:40.446828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:13:41.263101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:13:54.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:13:54.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.875+0000 7fb942400640 1 Processor -- start 2024-04-25T11:13:54.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.875+0000 7fb942400640 1 -- start start 2024-04-25T11:13:54.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.878+0000 7fb942400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb93c15dfb0 0x7fb93c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.878+0000 7fb942400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb93c06fab0 0x7fb93c0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.878+0000 7fb942400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb93c15e3c0 con 0x7fb93c0d0b40 2024-04-25T11:13:54.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.878+0000 7fb942400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb93c0b09b0 con 0x7fb93c15dfb0 2024-04-25T11:13:54.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.878+0000 7fb942400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb93c0cf800 con 0x7fb93c06fab0 2024-04-25T11:13:54.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.878+0000 7fb93aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb93c0d0b40 0x7fb93c06f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55544/0 (socket says 172.21.15.7:55544) 2024-04-25T11:13:54.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.878+0000 7fb93aa00640 1 -- 172.21.15.7:0/2782715716 learned_addr learned my addr 172.21.15.7:0/2782715716 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:54.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.879+0000 7fb93b400640 1 --2- 172.21.15.7:0/2782715716 >> v2:172.21.15.87:3300/0 conn(0x7fb93c15dfb0 0x7fb93c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.879+0000 7fb93be00640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb93c06fab0 0x7fb93c0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.879+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2766702178 0 0) 0x7fb93c15e3c0 con 0x7fb93c0d0b40 2024-04-25T11:13:54.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.879+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb920003520 con 0x7fb93c0d0b40 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.879+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3298316568 0 0) 0x7fb920003520 con 0x7fb93c0d0b40 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.879+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb93c06fab0 msgr2=0x7fb93c0aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.879+0000 7fb93a000640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb93c06fab0 0x7fb93c0aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.879+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 >> v2:172.21.15.87:3300/0 conn(0x7fb93c15dfb0 msgr2=0x7fb93c0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.879+0000 7fb93a000640 1 --2- 172.21.15.7:0/2782715716 >> v2:172.21.15.87:3300/0 conn(0x7fb93c15dfb0 0x7fb93c0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c0cfaa0 con 0x7fb93c0d0b40 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fb924001670 con 0x7fb93c0d0b40 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 >> v1:172.21.15.7:6789/0 conn(0x7fb93c0d0b40 legacy=0x7fb93c06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93a000640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb93c06fab0 0x7fb9200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93a000640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb920003f80 0x7fb920004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93a000640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb920004d40 0x7fb920005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c0b09b0 con 0x7fb920003f80 2024-04-25T11:13:54.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c0cf800 con 0x7fb920004d40 2024-04-25T11:13:54.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb92000ba70 con 0x7fb93c06fab0 2024-04-25T11:13:54.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93aa00640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb920003f80 0x7fb920004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93b400640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb93c06fab0 0x7fb9200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.880+0000 7fb93be00640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb920004d40 0x7fb920005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.881+0000 7fb93aa00640 1 -- 172.21.15.7:0/2782715716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb93c06fab0 msgr2=0x7fb9200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:54.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.881+0000 7fb93aa00640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb93c06fab0 0x7fb9200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.881+0000 7fb93aa00640 1 -- 172.21.15.7:0/2782715716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb920004d40 msgr2=0x7fb920005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:54.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.881+0000 7fb93aa00640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb920004d40 0x7fb920005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.881+0000 7fb93aa00640 1 -- 172.21.15.7:0/2782715716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb920003520 con 0x7fb920003f80 2024-04-25T11:13:54.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.882+0000 7fb93aa00640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb920003f80 0x7fb920004800 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0732f0 tx=0x7fb920005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfbaa97d30c6e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:54.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.882+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924007c80 con 0x7fb920003f80 2024-04-25T11:13:54.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.882+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb924005ad0 con 0x7fb920003f80 2024-04-25T11:13:54.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.882+0000 7fb93a000640 1 -- 172.21.15.7:0/2782715716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924007070 con 0x7fb920003f80 2024-04-25T11:13:54.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.883+0000 7fb942400640 1 -- 172.21.15.7:0/2782715716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb920003f80 msgr2=0x7fb920004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:54.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.883+0000 7fb942400640 1 --2- 172.21.15.7:0/2782715716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb920003f80 0x7fb920004800 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0732f0 tx=0x7fb920005830 comp rx=0 tx=0).stop 2024-04-25T11:13:54.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.883+0000 7fb93be00640 1 -- 172.21.15.7:0/2782715716 reap_dead start 2024-04-25T11:13:54.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.883+0000 7fb942400640 1 -- 172.21.15.7:0/2782715716 shutdown_connections 2024-04-25T11:13:54.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.883+0000 7fb942400640 1 -- 172.21.15.7:0/2782715716 >> 172.21.15.7:0/2782715716 conn(0x7fb93c0742c0 msgr2=0x7fb93c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:54.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.884+0000 7fb942400640 1 -- 172.21.15.7:0/2782715716 shutdown_connections 2024-04-25T11:13:54.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.884+0000 7fb942400640 1 -- 172.21.15.7:0/2782715716 wait complete. 2024-04-25T11:13:54.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.886+0000 7fb942400640 1 Processor -- start 2024-04-25T11:13:54.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.886+0000 7fb942400640 1 -- start start 2024-04-25T11:13:54.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.888+0000 7fb942400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb920003f80 0x7fb93c09ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.888+0000 7fb942400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb93c1545c0 0x7fb93c157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.888+0000 7fb942400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb93c157a40 0x7fb93c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.888+0000 7fb942400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c0d0340 con 0x7fb93c1545c0 2024-04-25T11:13:54.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.888+0000 7fb942400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c0d0060 con 0x7fb93c157a40 2024-04-25T11:13:54.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.888+0000 7fb942400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb93c0d0620 con 0x7fb920003f80 2024-04-25T11:13:54.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.889+0000 7fb93aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb93c1545c0 0x7fb93c157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.889+0000 7fb93aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb93c1545c0 0x7fb93c157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44428/0 (socket says 172.21.15.7:44428) 2024-04-25T11:13:54.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.889+0000 7fb93aa00640 1 -- 172.21.15.7:0/210665395 learned_addr learned my addr 172.21.15.7:0/210665395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:54.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.889+0000 7fb93b400640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb920003f80 0x7fb93c09ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.889+0000 7fb93be00640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb93c157a40 0x7fb93c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.889+0000 7fb93aa00640 1 -- 172.21.15.7:0/210665395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb920003f80 msgr2=0x7fb93c09ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.889+0000 7fb93aa00640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb920003f80 0x7fb93c09ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.889+0000 7fb93aa00640 1 -- 172.21.15.7:0/210665395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb93c157a40 msgr2=0x7fb93c154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.889+0000 7fb93aa00640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb93c157a40 0x7fb93c154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.889+0000 7fb93aa00640 1 -- 172.21.15.7:0/210665395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c15b8b0 con 0x7fb93c1545c0 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.890+0000 7fb93aa00640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb93c1545c0 0x7fb93c157500 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb92402ad00 tx=0x7fb93c0cfc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.890+0000 7fb933400640 1 -- 172.21.15.7:0/210665395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924001630 con 0x7fb93c1545c0 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.890+0000 7fb933400640 1 -- 172.21.15.7:0/210665395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb924042070 con 0x7fb93c1545c0 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.890+0000 7fb933400640 1 -- 172.21.15.7:0/210665395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924007270 con 0x7fb93c1545c0 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.890+0000 7fb942400640 1 -- 172.21.15.7:0/210665395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c157e50 con 0x7fb93c1545c0 2024-04-25T11:13:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.890+0000 7fb942400640 1 -- 172.21.15.7:0/210665395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c15be20 con 0x7fb93c1545c0 2024-04-25T11:13:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.891+0000 7fb942400640 1 -- 172.21.15.7:0/210665395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb93c050740 con 0x7fb93c1545c0 2024-04-25T11:13:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.893+0000 7fb933400640 1 -- 172.21.15.7:0/210665395 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb924007410 con 0x7fb93c1545c0 2024-04-25T11:13:54.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.894+0000 7fb933400640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9180744b0 0x7fb9180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:54.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.894+0000 7fb933400640 1 -- 172.21.15.7:0/210665395 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fb9240b5510 con 0x7fb93c1545c0 2024-04-25T11:13:54.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.894+0000 7fb93b400640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9180744b0 0x7fb9180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.895+0000 7fb93b400640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9180744b0 0x7fb9180769c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb93c072df0 tx=0x7fb92c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:54.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:54.899+0000 7fb933400640 1 -- 172.21.15.7:0/210665395 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb93c157e50 con 0x7fb93c1545c0 2024-04-25T11:13:55.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.105+0000 7fb942400640 1 -- 172.21.15.7:0/210665395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb93c0b6600 con 0x7fb93c1545c0 2024-04-25T11:13:55.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.106+0000 7fb933400640 1 -- 172.21.15.7:0/210665395 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7fb924005aa0 con 0x7fb93c1545c0 2024-04-25T11:13:55.108 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.109+0000 7fb942400640 1 -- 172.21.15.7:0/210665395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9180744b0 msgr2=0x7fb9180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.109+0000 7fb942400640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9180744b0 0x7fb9180769c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb93c072df0 tx=0x7fb92c00b040 comp rx=0 tx=0).stop 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.109+0000 7fb942400640 1 -- 172.21.15.7:0/210665395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb93c1545c0 msgr2=0x7fb93c157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.109+0000 7fb942400640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb93c1545c0 0x7fb93c157500 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb92402ad00 tx=0x7fb93c0cfc70 comp rx=0 tx=0).stop 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.110+0000 7fb942400640 1 -- 172.21.15.7:0/210665395 shutdown_connections 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.110+0000 7fb942400640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb9180744b0 0x7fb9180769c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.110+0000 7fb942400640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb93c157a40 0x7fb93c154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.110+0000 7fb942400640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb93c1545c0 0x7fb93c157500 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.110+0000 7fb942400640 1 --2- 172.21.15.7:0/210665395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb920003f80 0x7fb93c09ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.110+0000 7fb942400640 1 -- 172.21.15.7:0/210665395 >> 172.21.15.7:0/210665395 conn(0x7fb93c0742c0 msgr2=0x7fb93c0b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.110+0000 7fb942400640 1 -- 172.21.15.7:0/210665395 shutdown_connections 2024-04-25T11:13:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.110+0000 7fb942400640 1 -- 172.21.15.7:0/210665395 wait complete. 2024-04-25T11:13:55.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:13:55.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.429+0000 7fa8e7200640 1 Processor -- start 2024-04-25T11:13:55.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.429+0000 7fa8e7200640 1 -- start start 2024-04-25T11:13:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.432+0000 7fa8e7200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa8e015aaa0 0x7fa8e00affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.432+0000 7fa8e7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e00d0ee0 0x7fa8e00b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.432+0000 7fa8e7200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8e006f5b0 con 0x7fa8e00b4e50 2024-04-25T11:13:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.432+0000 7fa8e7200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e00af290 con 0x7fa8e015aaa0 2024-04-25T11:13:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.432+0000 7fa8e7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e00d14a0 con 0x7fa8e00d0ee0 2024-04-25T11:13:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.432+0000 7fa8e5200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa8e00b4e50 0x7fa8e015d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55558/0 (socket says 172.21.15.7:55558) 2024-04-25T11:13:55.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.432+0000 7fa8e5200640 1 -- 172.21.15.7:0/4174250199 learned_addr learned my addr 172.21.15.7:0/4174250199 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:55.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dfe00640 1 --2- 172.21.15.7:0/4174250199 >> v2:172.21.15.87:3300/0 conn(0x7fa8e015aaa0 0x7fa8e00affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:55.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8df400640 1 --2- 172.21.15.7:0/4174250199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e00d0ee0 0x7fa8e00b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1263035836 0 0) 0x7fa8e006f5b0 con 0x7fa8e00b4e50 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8c4003520 con 0x7fa8e00b4e50 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 205904832 0 0) 0x7fa8c4003520 con 0x7fa8e00b4e50 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e00d0ee0 msgr2=0x7fa8e00b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 --2- 172.21.15.7:0/4174250199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e00d0ee0 0x7fa8e00b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 >> v2:172.21.15.87:3300/0 conn(0x7fa8e015aaa0 msgr2=0x7fa8e00affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 --2- 172.21.15.7:0/4174250199 >> v2:172.21.15.87:3300/0 conn(0x7fa8e015aaa0 0x7fa8e00affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e00af4e0 con 0x7fa8e00b4e50 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fa8d8001670 con 0x7fa8e00b4e50 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 >> v1:172.21.15.7:6789/0 conn(0x7fa8e00b4e50 legacy=0x7fa8e015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 --2- 172.21.15.7:0/4174250199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 0x7fa8c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.433+0000 7fa8dea00640 1 --2- 172.21.15.7:0/4174250199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8c4004800 0x7fa8c4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.434+0000 7fa8dea00640 1 --2- 172.21.15.7:0/4174250199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e00d0ee0 0x7fa8c4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.434+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e00af290 con 0x7fa8c40039b0 2024-04-25T11:13:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.434+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e00d14a0 con 0x7fa8c4004800 2024-04-25T11:13:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.434+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c400dcb0 con 0x7fa8e00d0ee0 2024-04-25T11:13:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.434+0000 7fa8dfe00640 1 --2- 172.21.15.7:0/4174250199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 0x7fa8c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.434+0000 7fa8df400640 1 --2- 172.21.15.7:0/4174250199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8c4004800 0x7fa8c4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.434+0000 7fa8dfe00640 1 -- 172.21.15.7:0/4174250199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e00d0ee0 msgr2=0x7fa8c4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.434+0000 7fa8dfe00640 1 --2- 172.21.15.7:0/4174250199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e00d0ee0 0x7fa8c4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.434+0000 7fa8dfe00640 1 -- 172.21.15.7:0/4174250199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8c4004800 msgr2=0x7fa8c4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.434+0000 7fa8dfe00640 1 --2- 172.21.15.7:0/4174250199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8c4004800 0x7fa8c4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.434+0000 7fa8dfe00640 1 -- 172.21.15.7:0/4174250199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c4003520 con 0x7fa8c40039b0 2024-04-25T11:13:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.435+0000 7fa8dfe00640 1 --2- 172.21.15.7:0/4174250199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 0x7fa8c4003e20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009fc0 tx=0x7fa8c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decb8a100e7fcf4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.436+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0009c10 con 0x7fa8c40039b0 2024-04-25T11:13:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.436+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d0036070 con 0x7fa8c40039b0 2024-04-25T11:13:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.436+0000 7fa8dea00640 1 -- 172.21.15.7:0/4174250199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d002d070 con 0x7fa8c40039b0 2024-04-25T11:13:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.436+0000 7fa8e7200640 1 -- 172.21.15.7:0/4174250199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 msgr2=0x7fa8c4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.436+0000 7fa8e7200640 1 --2- 172.21.15.7:0/4174250199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 0x7fa8c4003e20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009fc0 tx=0x7fa8c4005830 comp rx=0 tx=0).stop 2024-04-25T11:13:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.436+0000 7fa8e5200640 1 -- 172.21.15.7:0/4174250199 reap_dead start 2024-04-25T11:13:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.437+0000 7fa8e7200640 1 -- 172.21.15.7:0/4174250199 shutdown_connections 2024-04-25T11:13:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.437+0000 7fa8e7200640 1 -- 172.21.15.7:0/4174250199 >> 172.21.15.7:0/4174250199 conn(0x7fa8e00742c0 msgr2=0x7fa8e006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.437+0000 7fa8e7200640 1 -- 172.21.15.7:0/4174250199 shutdown_connections 2024-04-25T11:13:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.437+0000 7fa8e7200640 1 -- 172.21.15.7:0/4174250199 wait complete. 2024-04-25T11:13:55.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.439+0000 7fa8e7200640 1 Processor -- start 2024-04-25T11:13:55.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.439+0000 7fa8e7200640 1 -- start start 2024-04-25T11:13:55.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.441+0000 7fa8e7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 0x7fa8e0156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.441+0000 7fa8e7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e0156880 0x7fa8e006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.441+0000 7fa8e7200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8e0070490 0x7fa8e01f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.441+0000 7fa8e7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e00b4910 con 0x7fa8c40039b0 2024-04-25T11:13:55.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.441+0000 7fa8e7200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e00afaa0 con 0x7fa8e0070490 2024-04-25T11:13:55.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.441+0000 7fa8e7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e00b4a90 con 0x7fa8e0156880 2024-04-25T11:13:55.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.441+0000 7fa8dfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 0x7fa8e0156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:55.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.441+0000 7fa8dfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 0x7fa8e0156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44444/0 (socket says 172.21.15.7:44444) 2024-04-25T11:13:55.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.441+0000 7fa8dfe00640 1 -- 172.21.15.7:0/2996213784 learned_addr learned my addr 172.21.15.7:0/2996213784 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.441+0000 7fa8e5200640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8e0070490 0x7fa8e01f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.442+0000 7fa8df400640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e0156880 0x7fa8e006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.442+0000 7fa8dfe00640 1 -- 172.21.15.7:0/2996213784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e0156880 msgr2=0x7fa8e006ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.442+0000 7fa8dfe00640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e0156880 0x7fa8e006ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.442+0000 7fa8dfe00640 1 -- 172.21.15.7:0/2996213784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8e0070490 msgr2=0x7fa8e01f1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.442+0000 7fa8dfe00640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8e0070490 0x7fa8e01f1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.442+0000 7fa8dfe00640 1 -- 172.21.15.7:0/2996213784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e015ec30 con 0x7fa8c40039b0 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.442+0000 7fa8df400640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e0156880 0x7fa8e006ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.442+0000 7fa8dfe00640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 0x7fa8e0156010 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e230 tx=0x7fa8d0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.442+0000 7fa8d7e00640 1 -- 172.21.15.7:0/2996213784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0009c10 con 0x7fa8c40039b0 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.442+0000 7fa8d7e00640 1 -- 172.21.15.7:0/2996213784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d0003b90 con 0x7fa8c40039b0 2024-04-25T11:13:55.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.442+0000 7fa8d7e00640 1 -- 172.21.15.7:0/2996213784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0004020 con 0x7fa8c40039b0 2024-04-25T11:13:55.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.443+0000 7fa8e7200640 1 -- 172.21.15.7:0/2996213784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e00708a0 con 0x7fa8c40039b0 2024-04-25T11:13:55.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.443+0000 7fa8e7200640 1 -- 172.21.15.7:0/2996213784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e015e950 con 0x7fa8c40039b0 2024-04-25T11:13:55.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.444+0000 7fa8e7200640 1 -- 172.21.15.7:0/2996213784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e0050740 con 0x7fa8c40039b0 2024-04-25T11:13:55.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.445+0000 7fa8d7e00640 1 -- 172.21.15.7:0/2996213784 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa8d0005d10 con 0x7fa8c40039b0 2024-04-25T11:13:55.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.446+0000 7fa8d7e00640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8bc0744e0 0x7fa8bc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.446+0000 7fa8d7e00640 1 -- 172.21.15.7:0/2996213784 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fa8d00b6a60 con 0x7fa8c40039b0 2024-04-25T11:13:55.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.447+0000 7fa8df400640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8bc0744e0 0x7fa8bc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:55.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.448+0000 7fa8df400640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8bc0744e0 0x7fa8bc0769f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00b2430 tx=0x7fa8c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:55.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.453+0000 7fa8d7e00640 1 -- 172.21.15.7:0/2996213784 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8e00708a0 con 0x7fa8c40039b0 2024-04-25T11:13:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.666+0000 7fa8e7200640 1 -- 172.21.15.7:0/2996213784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa8e0070f20 con 0x7fa8c40039b0 2024-04-25T11:13:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.667+0000 7fa8d7e00640 1 -- 172.21.15.7:0/2996213784 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) ==== 104+0+11 (secure 0 0 0) 0x7fa8d0080fc0 con 0x7fa8c40039b0 2024-04-25T11:13:55.668 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:13:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 -- 172.21.15.7:0/2996213784 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8bc0744e0 msgr2=0x7fa8bc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8bc0744e0 0x7fa8bc0769f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00b2430 tx=0x7fa8c8008040 comp rx=0 tx=0).stop 2024-04-25T11:13:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 -- 172.21.15.7:0/2996213784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 msgr2=0x7fa8e0156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 0x7fa8e0156010 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000e230 tx=0x7fa8d0009750 comp rx=0 tx=0).stop 2024-04-25T11:13:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 -- 172.21.15.7:0/2996213784 shutdown_connections 2024-04-25T11:13:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8bc0744e0 0x7fa8bc0769f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8e0070490 0x7fa8e01f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa8e0156880 0x7fa8e006ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 --2- 172.21.15.7:0/2996213784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8c40039b0 0x7fa8e0156010 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 -- 172.21.15.7:0/2996213784 >> 172.21.15.7:0/2996213784 conn(0x7fa8e00742c0 msgr2=0x7fa8e015b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 -- 172.21.15.7:0/2996213784 shutdown_connections 2024-04-25T11:13:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.670+0000 7fa8e7200640 1 -- 172.21.15.7:0/2996213784 wait complete. 2024-04-25T11:13:55.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:13:55.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.987+0000 7fcfc0400640 1 Processor -- start 2024-04-25T11:13:55.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.987+0000 7fcfc0400640 1 -- start start 2024-04-25T11:13:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.989+0000 7fcfc0400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcfb80b6240 0x7fcfb815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.989+0000 7fcfc0400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80aef90 0x7fcfb80b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.989+0000 7fcfc0400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfb8164e00 con 0x7fcfb80b6920 2024-04-25T11:13:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.989+0000 7fcfc0400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fcfb81650e0 con 0x7fcfb80b6240 2024-04-25T11:13:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.989+0000 7fcfc0400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb81653c0 con 0x7fcfb80aef90 2024-04-25T11:13:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.990+0000 7fcfbda00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcfb80b6240 0x7fcfb815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.990+0000 7fcfbda00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcfb80b6240 0x7fcfb815eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44446/0 (socket says 172.21.15.7:44446) 2024-04-25T11:13:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.990+0000 7fcfbda00640 1 -- 172.21.15.7:0/3150217767 learned_addr learned my addr 172.21.15.7:0/3150217767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:55.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.990+0000 7fcfbe400640 1 --2- 172.21.15.7:0/3150217767 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80aef90 0x7fcfb80b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:55.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.991+0000 7fcfbda00640 1 -- 172.21.15.7:0/3150217767 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80aef90 msgr2=0x7fcfb80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:55.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.991+0000 7fcfbda00640 1 --2- 172.21.15.7:0/3150217767 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80aef90 0x7fcfb80b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.991+0000 7fcfbda00640 1 -- 172.21.15.7:0/3150217767 >> v1:172.21.15.87:6789/0 conn(0x7fcfb80b6920 legacy=0x7fcfb80ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:55.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.991+0000 7fcfbda00640 1 -- 172.21.15.7:0/3150217767 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb81656a0 con 0x7fcfb80b6240 2024-04-25T11:13:55.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.992+0000 7fcfbda00640 1 --2- 172.21.15.7:0/3150217767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80b6240 0x7fcfb815eb00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009870 tx=0x7fcfa802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202527b6ce108286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:55.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.992+0000 7fcfb7e00640 1 -- 172.21.15.7:0/3150217767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8037070 con 0x7fcfb80b6240 2024-04-25T11:13:55.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.992+0000 7fcfb7e00640 1 -- 172.21.15.7:0/3150217767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa8032440 con 0x7fcfb80b6240 2024-04-25T11:13:55.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.992+0000 7fcfb7e00640 1 -- 172.21.15.7:0/3150217767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8032810 con 0x7fcfb80b6240 2024-04-25T11:13:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.993+0000 7fcfc0400640 1 -- 172.21.15.7:0/3150217767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80b6240 msgr2=0x7fcfb815eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.993+0000 7fcfc0400640 1 --2- 172.21.15.7:0/3150217767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80b6240 0x7fcfb815eb00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009870 tx=0x7fcfa802a250 comp rx=0 tx=0).stop 2024-04-25T11:13:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.993+0000 7fcfc0400640 1 -- 172.21.15.7:0/3150217767 shutdown_connections 2024-04-25T11:13:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.993+0000 7fcfc0400640 1 --2- 172.21.15.7:0/3150217767 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80aef90 0x7fcfb80b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.993+0000 7fcfc0400640 1 --2- 172.21.15.7:0/3150217767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80b6240 0x7fcfb815eb00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.993+0000 7fcfc0400640 1 -- 172.21.15.7:0/3150217767 >> 172.21.15.7:0/3150217767 conn(0x7fcfb80742c0 msgr2=0x7fcfb806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.993+0000 7fcfc0400640 1 -- 172.21.15.7:0/3150217767 shutdown_connections 2024-04-25T11:13:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.993+0000 7fcfc0400640 1 -- 172.21.15.7:0/3150217767 wait complete. 2024-04-25T11:13:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.996+0000 7fcfc0400640 1 Processor -- start 2024-04-25T11:13:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.996+0000 7fcfc0400640 1 -- start start 2024-04-25T11:13:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.997+0000 7fcfc0400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80aef90 0x7fcfb814f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfc0400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80b6240 0x7fcfb81581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfc0400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb80b6920 0x7fcfb81fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfc0400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb8164ba0 con 0x7fcfb80aef90 2024-04-25T11:13:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfc0400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb8164a20 con 0x7fcfb80b6920 2024-04-25T11:13:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfc0400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb80b5130 con 0x7fcfb80b6240 2024-04-25T11:13:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfbda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80aef90 0x7fcfb814f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfbda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80aef90 0x7fcfb814f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44462/0 (socket says 172.21.15.7:44462) 2024-04-25T11:13:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfbda00640 1 -- 172.21.15.7:0/2385603016 learned_addr learned my addr 172.21.15.7:0/2385603016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:56.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfbe400640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb80b6920 0x7fcfb81fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfbda00640 1 -- 172.21.15.7:0/2385603016 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80b6240 msgr2=0x7fcfb81581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfbda00640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80b6240 0x7fcfb81581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfbda00640 1 -- 172.21.15.7:0/2385603016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb80b6920 msgr2=0x7fcfb81fdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfbda00640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb80b6920 0x7fcfb81fdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.998+0000 7fcfbda00640 1 -- 172.21.15.7:0/2385603016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb8158d40 con 0x7fcfb80aef90 2024-04-25T11:13:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.999+0000 7fcfbda00640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80aef90 0x7fcfb814f910 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009870 tx=0x7fcfa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.999+0000 7fcfb5600640 1 -- 172.21.15.7:0/2385603016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8041020 con 0x7fcfb80aef90 2024-04-25T11:13:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.999+0000 7fcfb5600640 1 -- 172.21.15.7:0/2385603016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa80306c0 con 0x7fcfb80aef90 2024-04-25T11:13:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.999+0000 7fcfb5600640 1 -- 172.21.15.7:0/2385603016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8037040 con 0x7fcfb80aef90 2024-04-25T11:13:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.999+0000 7fcfc0400640 1 -- 172.21.15.7:0/2385603016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb8159020 con 0x7fcfb80aef90 2024-04-25T11:13:56.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:55.999+0000 7fcfc0400640 1 -- 172.21.15.7:0/2385603016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb81595e0 con 0x7fcfb80aef90 2024-04-25T11:13:56.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.001+0000 7fcfc0400640 1 -- 172.21.15.7:0/2385603016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb8050740 con 0x7fcfb80aef90 2024-04-25T11:13:56.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.002+0000 7fcfb5600640 1 -- 172.21.15.7:0/2385603016 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcfa8044020 con 0x7fcfb80aef90 2024-04-25T11:13:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.003+0000 7fcfb5600640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf980744b0 0x7fcf980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.004+0000 7fcfb5600640 1 -- 172.21.15.7:0/2385603016 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fcfa80b5c60 con 0x7fcfb80aef90 2024-04-25T11:13:56.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.004+0000 7fcfbd000640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf980744b0 0x7fcf980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:56.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.005+0000 7fcfbd000640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf980744b0 0x7fcf980769c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4000f60 tx=0x7fcfa40079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:56.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.009+0000 7fcfb5600640 1 -- 172.21.15.7:0/2385603016 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfa804b030 con 0x7fcfb80aef90 2024-04-25T11:13:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.210+0000 7fcfc0400640 1 -- 172.21.15.7:0/2385603016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcfb81537d0 con 0x7fcfb80aef90 2024-04-25T11:13:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.211+0000 7fcfb5600640 1 -- 172.21.15.7:0/2385603016 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) ==== 100+0+11 (secure 0 0 0) 0x7fcfa8080250 con 0x7fcfb80aef90 2024-04-25T11:13:56.212 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:13:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 -- 172.21.15.7:0/2385603016 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf980744b0 msgr2=0x7fcf980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf980744b0 0x7fcf980769c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4000f60 tx=0x7fcfa40079d0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 -- 172.21.15.7:0/2385603016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80aef90 msgr2=0x7fcfb814f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80aef90 0x7fcfb814f910 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009870 tx=0x7fcfa80046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 -- 172.21.15.7:0/2385603016 shutdown_connections 2024-04-25T11:13:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf980744b0 0x7fcf980769c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb80b6920 0x7fcfb81fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80b6240 0x7fcfb81581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 --2- 172.21.15.7:0/2385603016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80aef90 0x7fcfb814f910 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 -- 172.21.15.7:0/2385603016 >> 172.21.15.7:0/2385603016 conn(0x7fcfb80742c0 msgr2=0x7fcfb80b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 -- 172.21.15.7:0/2385603016 shutdown_connections 2024-04-25T11:13:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.214+0000 7fcfc0400640 1 -- 172.21.15.7:0/2385603016 wait complete. 2024-04-25T11:13:56.242 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:13:56.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.528+0000 7f04cb600640 1 Processor -- start 2024-04-25T11:13:56.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.529+0000 7f04cb600640 1 -- start start 2024-04-25T11:13:56.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.531+0000 7f04cb600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f04c406f1e0 0x7f04c406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.531+0000 7f04cb600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c4162700 con 0x7f04c406d580 2024-04-25T11:13:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.531+0000 7f04cb600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04c41629e0 con 0x7f04c415f180 2024-04-25T11:13:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.531+0000 7f04cb600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f04c415f590 con 0x7f04c406f1e0 2024-04-25T11:13:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.531+0000 7f04c3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f04c406d580 0x7f04c406d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55562/0 (socket says 172.21.15.7:55562) 2024-04-25T11:13:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.531+0000 7f04c3e00640 1 -- 172.21.15.7:0/237485583 learned_addr learned my addr 172.21.15.7:0/237485583 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.531+0000 7f04c9600640 1 --2- 172.21.15.7:0/237485583 >> v2:172.21.15.87:3300/0 conn(0x7f04c406f1e0 0x7f04c406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1096673939 0 0) 0x7f04c4162700 con 0x7f04c406d580 2024-04-25T11:13:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04ac003520 con 0x7f04c406d580 2024-04-25T11:13:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1032765893 0 0) 0x7f04ac003520 con 0x7f04c406d580 2024-04-25T11:13:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 >> v2:172.21.15.87:3300/0 conn(0x7f04c406f1e0 msgr2=0x7f04c406f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 --2- 172.21.15.7:0/237485583 >> v2:172.21.15.87:3300/0 conn(0x7f04c406f1e0 0x7f04c406f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 >> v1:172.21.15.87:6789/0 conn(0x7f04c415f180 legacy=0x7f04c4161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c415e980 con 0x7f04c406d580 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f04b0001670 con 0x7f04c406d580 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 >> v1:172.21.15.7:6789/0 conn(0x7f04c406d580 legacy=0x7f04c406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac003a00 0x7f04ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac004800 0x7f04ac004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04ac0052a0 0x7f04ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04c415f590 con 0x7f04ac004800 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac00c230 con 0x7f04ac003a00 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.532+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f04ac00c430 con 0x7f04ac0052a0 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.533+0000 7f04c3e00640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac004800 0x7f04ac004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.533+0000 7f04c8c00640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac003a00 0x7f04ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.533+0000 7f04c9600640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04ac0052a0 0x7f04ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.533+0000 7f04c3e00640 1 -- 172.21.15.7:0/237485583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04ac0052a0 msgr2=0x7f04ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.533+0000 7f04c3e00640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04ac0052a0 0x7f04ac00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.533+0000 7f04c3e00640 1 -- 172.21.15.7:0/237485583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac003a00 msgr2=0x7f04ac003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.533+0000 7f04c3e00640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac003a00 0x7f04ac003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.533+0000 7f04c3e00640 1 -- 172.21.15.7:0/237485583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac003520 con 0x7f04ac004800 2024-04-25T11:13:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.533+0000 7f04c8c00640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac003a00 0x7f04ac003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.533+0000 7f04c9600640 1 -- 172.21.15.7:0/237485583 reap_dead start 2024-04-25T11:13:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.534+0000 7f04c3e00640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac004800 0x7f04ac004c10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c0e0 tx=0x7f04ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2918b8d7899607 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.535+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0007c20 con 0x7f04ac004800 2024-04-25T11:13:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.535+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b0005af0 con 0x7f04ac004800 2024-04-25T11:13:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.535+0000 7f04c3400640 1 -- 172.21.15.7:0/237485583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0007070 con 0x7f04ac004800 2024-04-25T11:13:56.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.535+0000 7f04cb600640 1 -- 172.21.15.7:0/237485583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac004800 msgr2=0x7f04ac004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:56.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.535+0000 7f04cb600640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac004800 0x7f04ac004c10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c0e0 tx=0x7f04ac005830 comp rx=0 tx=0).stop 2024-04-25T11:13:56.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.535+0000 7f04cb600640 1 -- 172.21.15.7:0/237485583 shutdown_connections 2024-04-25T11:13:56.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.535+0000 7f04cb600640 1 --2- 172.21.15.7:0/237485583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04ac004800 0x7f04ac004c10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.535+0000 7f04cb600640 1 -- 172.21.15.7:0/237485583 >> 172.21.15.7:0/237485583 conn(0x7f04c40732b0 msgr2=0x7f04c40736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:56.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.535+0000 7f04cb600640 1 -- 172.21.15.7:0/237485583 shutdown_connections 2024-04-25T11:13:56.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.536+0000 7f04cb600640 1 -- 172.21.15.7:0/237485583 wait complete. 2024-04-25T11:13:56.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.538+0000 7f04cb600640 1 Processor -- start 2024-04-25T11:13:56.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.538+0000 7f04cb600640 1 -- start start 2024-04-25T11:13:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.540+0000 7f04cb600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac004800 0x7f04c4152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.540+0000 7f04cb600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c41512c0 0x7f04c4153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.540+0000 7f04cb600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c4154180 0x7f04c406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.540+0000 7f04cb600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04c415e670 con 0x7f04c41512c0 2024-04-25T11:13:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.540+0000 7f04cb600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4158ae0 con 0x7f04ac004800 2024-04-25T11:13:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.540+0000 7f04cb600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f04c40d1310 con 0x7f04c4154180 2024-04-25T11:13:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.540+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c41512c0 0x7f04c4153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.540+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c41512c0 0x7f04c4153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44482/0 (socket says 172.21.15.7:44482) 2024-04-25T11:13:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.540+0000 7f04c3e00640 1 -- 172.21.15.7:0/4087385277 learned_addr learned my addr 172.21.15.7:0/4087385277 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.540+0000 7f04c9600640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c4154180 0x7f04c406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.540+0000 7f04c8c00640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac004800 0x7f04c4152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.541+0000 7f04c3e00640 1 -- 172.21.15.7:0/4087385277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c4154180 msgr2=0x7f04c406f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.541+0000 7f04c3e00640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c4154180 0x7f04c406f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.541+0000 7f04c3e00640 1 -- 172.21.15.7:0/4087385277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac004800 msgr2=0x7f04c4152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.541+0000 7f04c3e00640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac004800 0x7f04c4152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.541+0000 7f04c3e00640 1 -- 172.21.15.7:0/4087385277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c4151bc0 con 0x7f04c41512c0 2024-04-25T11:13:56.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.541+0000 7f04c3e00640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c41512c0 0x7f04c4153c40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c200 tx=0x7f04c415eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:56.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.541+0000 7f04c0c00640 1 -- 172.21.15.7:0/4087385277 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0042070 con 0x7f04c41512c0 2024-04-25T11:13:56.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.541+0000 7f04c0c00640 1 -- 172.21.15.7:0/4087385277 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b00047e0 con 0x7f04c41512c0 2024-04-25T11:13:56.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.541+0000 7f04c0c00640 1 -- 172.21.15.7:0/4087385277 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0037040 con 0x7f04c41512c0 2024-04-25T11:13:56.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.541+0000 7f04cb600640 1 -- 172.21.15.7:0/4087385277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c4151e60 con 0x7f04c41512c0 2024-04-25T11:13:56.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.542+0000 7f04cb600640 1 -- 172.21.15.7:0/4087385277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c4152420 con 0x7f04c41512c0 2024-04-25T11:13:56.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.543+0000 7f04cb600640 1 -- 172.21.15.7:0/4087385277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c4050740 con 0x7f04c41512c0 2024-04-25T11:13:56.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.544+0000 7f04c0c00640 1 -- 172.21.15.7:0/4087385277 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f04b0045070 con 0x7f04c41512c0 2024-04-25T11:13:56.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.545+0000 7f04c0c00640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04a40744e0 0x7f04a40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:56.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.545+0000 7f04c0c00640 1 -- 172.21.15.7:0/4087385277 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f04b00b4ab0 con 0x7f04c41512c0 2024-04-25T11:13:56.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.546+0000 7f04c8c00640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04a40744e0 0x7f04a40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:56.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.546+0000 7f04c8c00640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04a40744e0 0x7f04a40769f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f04c406c310 tx=0x7f04b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:56.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.550+0000 7f04c0c00640 1 -- 172.21.15.7:0/4087385277 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04c4151bc0 con 0x7f04c41512c0 2024-04-25T11:13:56.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.768+0000 7f04cb600640 1 -- 172.21.15.7:0/4087385277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f04c4074250 con 0x7f04c41512c0 2024-04-25T11:13:56.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.769+0000 7f04c0c00640 1 -- 172.21.15.7:0/4087385277 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) ==== 80+0+0 (secure 0 0 0) 0x7f04b007ea50 con 0x7f04c41512c0 2024-04-25T11:13:56.770 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:13:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 -- 172.21.15.7:0/4087385277 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04a40744e0 msgr2=0x7f04a40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04a40744e0 0x7f04a40769f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f04c406c310 tx=0x7f04b8009040 comp rx=0 tx=0).stop 2024-04-25T11:13:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 -- 172.21.15.7:0/4087385277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c41512c0 msgr2=0x7f04c4153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c41512c0 0x7f04c4153c40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c200 tx=0x7f04c415eb50 comp rx=0 tx=0).stop 2024-04-25T11:13:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 -- 172.21.15.7:0/4087385277 shutdown_connections 2024-04-25T11:13:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f04a40744e0 0x7f04a40769f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c4154180 0x7f04c406f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c41512c0 0x7f04c4153c40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 --2- 172.21.15.7:0/4087385277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04ac004800 0x7f04c4152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:56.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 -- 172.21.15.7:0/4087385277 >> 172.21.15.7:0/4087385277 conn(0x7f04c40732b0 msgr2=0x7f04c415dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:56.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 -- 172.21.15.7:0/4087385277 shutdown_connections 2024-04-25T11:13:56.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:56.772+0000 7f04cb600640 1 -- 172.21.15.7:0/4087385277 wait complete. 2024-04-25T11:13:56.798 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:13:56.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:56.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:13:56.893 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:13:56.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:56.893 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:13:56.947 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:13:56.947 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:13:56.947 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:57.004 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:13:57.004 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:13:57.058 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:13:57.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.352+0000 7faf67200640 1 Processor -- start 2024-04-25T11:13:57.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.352+0000 7faf67200640 1 -- start start 2024-04-25T11:13:57.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.355+0000 7faf67200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf6006d510 0x7faf600b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.355+0000 7faf67200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf6006ef10 con 0x7faf6015c390 2024-04-25T11:13:57.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.355+0000 7faf67200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf600b6390 con 0x7faf60070890 2024-04-25T11:13:57.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.355+0000 7faf67200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7faf6006f210 con 0x7faf6006d510 2024-04-25T11:13:57.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.355+0000 7faf65400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faf6015c390 0x7faf6006ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55566/0 (socket says 172.21.15.7:55566) 2024-04-25T11:13:57.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.355+0000 7faf65400640 1 -- 172.21.15.7:0/3589802843 learned_addr learned my addr 172.21.15.7:0/3589802843 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:57.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.355+0000 7faf66800640 1 --2- 172.21.15.7:0/3589802843 >> v2:172.21.15.7:3300/0 conn(0x7faf6006d510 0x7faf600b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:57.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.356+0000 7faf66800640 1 -- 172.21.15.7:0/3589802843 >> v1:172.21.15.87:6789/0 conn(0x7faf60070890 legacy=0x7faf6006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:57.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.356+0000 7faf66800640 1 -- 172.21.15.7:0/3589802843 >> v1:172.21.15.7:6789/0 conn(0x7faf6015c390 legacy=0x7faf6006ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:57.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.356+0000 7faf66800640 1 -- 172.21.15.7:0/3589802843 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf600b6670 con 0x7faf6006d510 2024-04-25T11:13:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.357+0000 7faf66800640 1 --2- 172.21.15.7:0/3589802843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf6006d510 0x7faf600b5140 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009d90 tx=0x7faf5c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f1898fdd39dd84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:57.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.357+0000 7faf64a00640 1 -- 172.21.15.7:0/3589802843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c031600 con 0x7faf6006d510 2024-04-25T11:13:57.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.357+0000 7faf64a00640 1 -- 172.21.15.7:0/3589802843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf5c0317a0 con 0x7faf6006d510 2024-04-25T11:13:57.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.357+0000 7faf64a00640 1 -- 172.21.15.7:0/3589802843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c031b70 con 0x7faf6006d510 2024-04-25T11:13:57.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.358+0000 7faf67200640 1 -- 172.21.15.7:0/3589802843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf6006d510 msgr2=0x7faf600b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.358+0000 7faf67200640 1 --2- 172.21.15.7:0/3589802843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf6006d510 0x7faf600b5140 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009d90 tx=0x7faf5c02a610 comp rx=0 tx=0).stop 2024-04-25T11:13:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.358+0000 7faf67200640 1 -- 172.21.15.7:0/3589802843 shutdown_connections 2024-04-25T11:13:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.358+0000 7faf67200640 1 --2- 172.21.15.7:0/3589802843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf6006d510 0x7faf600b5140 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.358+0000 7faf67200640 1 -- 172.21.15.7:0/3589802843 >> 172.21.15.7:0/3589802843 conn(0x7faf600b3820 msgr2=0x7faf600b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.358+0000 7faf67200640 1 -- 172.21.15.7:0/3589802843 shutdown_connections 2024-04-25T11:13:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.358+0000 7faf67200640 1 -- 172.21.15.7:0/3589802843 wait complete. 2024-04-25T11:13:57.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.361+0000 7faf67200640 1 Processor -- start 2024-04-25T11:13:57.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.361+0000 7faf67200640 1 -- start start 2024-04-25T11:13:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.362+0000 7faf67200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf60070890 0x7faf6014ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.362+0000 7faf67200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf6015c390 0x7faf6009e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.362+0000 7faf67200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf6009d050 0x7faf6009eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.362+0000 7faf67200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf600b6950 con 0x7faf6015c390 2024-04-25T11:13:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.362+0000 7faf67200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faf6015c7a0 con 0x7faf60070890 2024-04-25T11:13:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.362+0000 7faf67200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faf6006c570 con 0x7faf6009d050 2024-04-25T11:13:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.363+0000 7faf65400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf6015c390 0x7faf6009e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.363+0000 7faf65400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf6015c390 0x7faf6009e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44492/0 (socket says 172.21.15.7:44492) 2024-04-25T11:13:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.363+0000 7faf65400640 1 -- 172.21.15.7:0/3281252953 learned_addr learned my addr 172.21.15.7:0/3281252953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:57.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.363+0000 7faf65e00640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf60070890 0x7faf6014ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:57.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.363+0000 7faf65400640 1 -- 172.21.15.7:0/3281252953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf6009d050 msgr2=0x7faf6009eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:57.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.363+0000 7faf65400640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf6009d050 0x7faf6009eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.363+0000 7faf65400640 1 -- 172.21.15.7:0/3281252953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf60070890 msgr2=0x7faf6014ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.363+0000 7faf65400640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf60070890 0x7faf6014ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.363+0000 7faf65400640 1 -- 172.21.15.7:0/3281252953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6009f790 con 0x7faf6015c390 2024-04-25T11:13:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.363+0000 7faf65e00640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf60070890 0x7faf6014ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.364+0000 7faf65400640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf6015c390 0x7faf6009e980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c3f0 tx=0x7faf5400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.364+0000 7faf5a000640 1 -- 172.21.15.7:0/3281252953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf54017070 con 0x7faf6015c390 2024-04-25T11:13:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.364+0000 7faf5a000640 1 -- 172.21.15.7:0/3281252953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf54005ce0 con 0x7faf6015c390 2024-04-25T11:13:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.364+0000 7faf5a000640 1 -- 172.21.15.7:0/3281252953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf540072f0 con 0x7faf6015c390 2024-04-25T11:13:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.364+0000 7faf67200640 1 -- 172.21.15.7:0/3281252953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6009c840 con 0x7faf6015c390 2024-04-25T11:13:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.364+0000 7faf67200640 1 -- 172.21.15.7:0/3281252953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf600cf510 con 0x7faf6015c390 2024-04-25T11:13:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.365+0000 7faf67200640 1 -- 172.21.15.7:0/3281252953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf60050740 con 0x7faf6015c390 2024-04-25T11:13:57.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.366+0000 7faf5a000640 1 -- 172.21.15.7:0/3281252953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faf5401a070 con 0x7faf6015c390 2024-04-25T11:13:57.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.367+0000 7faf5a000640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf440744b0 0x7faf440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.368+0000 7faf5a000640 1 -- 172.21.15.7:0/3281252953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7faf54094760 con 0x7faf6015c390 2024-04-25T11:13:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.368+0000 7faf65e00640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf440744b0 0x7faf440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.369+0000 7faf65e00640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf440744b0 0x7faf440769c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faf6009fbd0 tx=0x7faf50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:57.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.372+0000 7faf5a000640 1 -- 172.21.15.7:0/3281252953 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf5405ed50 con 0x7faf6015c390 2024-04-25T11:13:57.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.576+0000 7faf67200640 1 -- 172.21.15.7:0/3281252953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf60070440 con 0x7faf6015c390 2024-04-25T11:13:57.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.579+0000 7faf5a000640 1 -- 172.21.15.7:0/3281252953 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16846 (secure 0 0 0) 0x7faf5405e650 con 0x7faf6015c390 2024-04-25T11:13:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:13:57.581 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:43.726826+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:13:40.446828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T11:13:41.263101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:13:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 -- 172.21.15.7:0/3281252953 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf440744b0 msgr2=0x7faf440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf440744b0 0x7faf440769c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faf6009fbd0 tx=0x7faf50008040 comp rx=0 tx=0).stop 2024-04-25T11:13:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 -- 172.21.15.7:0/3281252953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf6015c390 msgr2=0x7faf6009e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf6015c390 0x7faf6009e980 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7faf5400c3f0 tx=0x7faf5400c8b0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 -- 172.21.15.7:0/3281252953 shutdown_connections 2024-04-25T11:13:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf440744b0 0x7faf440769c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf6009d050 0x7faf6009eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf6015c390 0x7faf6009e980 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 --2- 172.21.15.7:0/3281252953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf60070890 0x7faf6014ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 -- 172.21.15.7:0/3281252953 >> 172.21.15.7:0/3281252953 conn(0x7faf600b3820 msgr2=0x7faf600b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 -- 172.21.15.7:0/3281252953 shutdown_connections 2024-04-25T11:13:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.582+0000 7faf67200640 1 -- 172.21.15.7:0/3281252953 wait complete. 2024-04-25T11:13:57.610 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:13:40.446828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:13:41.263101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:13:57.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:13:57.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.905+0000 7f5002a00640 1 Processor -- start 2024-04-25T11:13:57.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.906+0000 7f5002a00640 1 -- start start 2024-04-25T11:13:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.908+0000 7f5002a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ffc06fab0 0x7f4ffc168e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.908+0000 7f5002a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ffc15e3c0 con 0x7f4ffc0b0470 2024-04-25T11:13:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.908+0000 7f5002a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ffc0cf800 con 0x7f4ffc15dfb0 2024-04-25T11:13:57.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.908+0000 7f5002a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4ffc0d1250 con 0x7f4ffc06fab0 2024-04-25T11:13:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.909+0000 7f4ff3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4ffc0b0470 0x7f4ffc06f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55568/0 (socket says 172.21.15.7:55568) 2024-04-25T11:13:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.909+0000 7f4ff3e00640 1 -- 172.21.15.7:0/489389381 learned_addr learned my addr 172.21.15.7:0/489389381 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.909+0000 7f5000a00640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ffc06fab0 0x7f4ffc168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.909+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3145035105 0 0) 0x7f4ffc15e3c0 con 0x7f4ffc0b0470 2024-04-25T11:13:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.909+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fe0003520 con 0x7f4ffc0b0470 2024-04-25T11:13:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 225302726 0 0) 0x7f4fe0003520 con 0x7f4ffc0b0470 2024-04-25T11:13:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ffc06fab0 msgr2=0x7f4ffc168e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ffc06fab0 0x7f4ffc168e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 >> v1:172.21.15.87:6789/0 conn(0x7f4ffc15dfb0 legacy=0x7f4ffc0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ffc0cfaa0 con 0x7f4ffc0b0470 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f4fe4001670 con 0x7f4ffc0b0470 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 >> v1:172.21.15.7:6789/0 conn(0x7f4ffc0b0470 legacy=0x7f4ffc06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fe00039e0 0x7f4fe0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fe0004800 0x7f4fe0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ffc06fab0 0x7f4fe0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc0d1250 con 0x7f4fe00039e0 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe000bb20 con 0x7f4fe0004800 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe000bd20 con 0x7f4ffc06fab0 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.910+0000 7f4ffbe00640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fe00039e0 0x7f4fe0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.911+0000 7f4ff3e00640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fe0004800 0x7f4fe0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.911+0000 7f5000a00640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ffc06fab0 0x7f4fe0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.911+0000 7f4ffbe00640 1 -- 172.21.15.7:0/489389381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ffc06fab0 msgr2=0x7f4fe0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.911+0000 7f4ffbe00640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ffc06fab0 0x7f4fe0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.911+0000 7f4ffbe00640 1 -- 172.21.15.7:0/489389381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fe0004800 msgr2=0x7f4fe0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.911+0000 7f4ffbe00640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fe0004800 0x7f4fe0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.911+0000 7f4ffbe00640 1 -- 172.21.15.7:0/489389381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe0003520 con 0x7f4fe00039e0 2024-04-25T11:13:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.912+0000 7f4ffbe00640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fe00039e0 0x7f4fe0003e50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc072df0 tx=0x7f4fe0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0723273f10e597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.912+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec001370 con 0x7f4fe00039e0 2024-04-25T11:13:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.912+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fec0052d0 con 0x7f4fe00039e0 2024-04-25T11:13:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.912+0000 7f4ffb400640 1 -- 172.21.15.7:0/489389381 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec02ab90 con 0x7f4fe00039e0 2024-04-25T11:13:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.913+0000 7f5002a00640 1 -- 172.21.15.7:0/489389381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fe00039e0 msgr2=0x7f4fe0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.913+0000 7f5002a00640 1 --2- 172.21.15.7:0/489389381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fe00039e0 0x7f4fe0003e50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc072df0 tx=0x7f4fe0005830 comp rx=0 tx=0).stop 2024-04-25T11:13:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.913+0000 7f5000a00640 1 -- 172.21.15.7:0/489389381 reap_dead start 2024-04-25T11:13:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.913+0000 7f5002a00640 1 -- 172.21.15.7:0/489389381 shutdown_connections 2024-04-25T11:13:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.913+0000 7f5002a00640 1 -- 172.21.15.7:0/489389381 >> 172.21.15.7:0/489389381 conn(0x7f4ffc0742c0 msgr2=0x7f4ffc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.913+0000 7f5002a00640 1 -- 172.21.15.7:0/489389381 shutdown_connections 2024-04-25T11:13:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.913+0000 7f5002a00640 1 -- 172.21.15.7:0/489389381 wait complete. 2024-04-25T11:13:57.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.916+0000 7f5002a00640 1 Processor -- start 2024-04-25T11:13:57.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.916+0000 7f5002a00640 1 -- start start 2024-04-25T11:13:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.917+0000 7f5002a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fe00039e0 0x7f4ffc15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.917+0000 7f5002a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ffc155bb0 0x7f4ffc151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5002a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ffc0b4f50 0x7f4ffc0b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5002a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc0d0340 con 0x7f4ffc0b4f50 2024-04-25T11:13:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5002a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc0d0060 con 0x7f4ffc155bb0 2024-04-25T11:13:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5002a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4ffc0d0620 con 0x7f4fe00039e0 2024-04-25T11:13:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5000a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ffc0b4f50 0x7f4ffc0b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5000a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ffc0b4f50 0x7f4ffc0b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44512/0 (socket says 172.21.15.7:44512) 2024-04-25T11:13:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5000a00640 1 -- 172.21.15.7:0/806811637 learned_addr learned my addr 172.21.15.7:0/806811637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f4ff3e00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ffc155bb0 0x7f4ffc151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5000a00640 1 -- 172.21.15.7:0/806811637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fe00039e0 msgr2=0x7f4ffc15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5000a00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fe00039e0 0x7f4ffc15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5000a00640 1 -- 172.21.15.7:0/806811637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ffc155bb0 msgr2=0x7f4ffc151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5000a00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ffc155bb0 0x7f4ffc151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.918+0000 7f5000a00640 1 -- 172.21.15.7:0/806811637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ffc0b58a0 con 0x7f4ffc0b4f50 2024-04-25T11:13:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.919+0000 7f5000a00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ffc0b4f50 0x7f4ffc0b5360 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc074730 tx=0x7f4ff40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.919+0000 7f4ff8c00640 1 -- 172.21.15.7:0/806811637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4003990 con 0x7f4ffc0b4f50 2024-04-25T11:13:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.919+0000 7f4ff8c00640 1 -- 172.21.15.7:0/806811637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ff401b070 con 0x7f4ffc0b4f50 2024-04-25T11:13:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.919+0000 7f4ff8c00640 1 -- 172.21.15.7:0/806811637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4003b70 con 0x7f4ffc0b4f50 2024-04-25T11:13:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.919+0000 7f5002a00640 1 -- 172.21.15.7:0/806811637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ffc15e900 con 0x7f4ffc0b4f50 2024-04-25T11:13:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.919+0000 7f5002a00640 1 -- 172.21.15.7:0/806811637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ffc1fa030 con 0x7f4ffc0b4f50 2024-04-25T11:13:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.920+0000 7f5002a00640 1 -- 172.21.15.7:0/806811637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ffc1f9ac0 con 0x7f4ffc0b4f50 2024-04-25T11:13:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.922+0000 7f4ff8c00640 1 -- 172.21.15.7:0/806811637 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ff4005d10 con 0x7f4ffc0b4f50 2024-04-25T11:13:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.923+0000 7f4ff8c00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4fd80744b0 0x7f4fd80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:57.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.923+0000 7f4ff8c00640 1 -- 172.21.15.7:0/806811637 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f4ff4095800 con 0x7f4ffc0b4f50 2024-04-25T11:13:57.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.923+0000 7f4ffbe00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4fd80744b0 0x7f4fd80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:57.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.924+0000 7f4ffbe00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4fd80744b0 0x7f4fd80769c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4fec032000 tx=0x7f4fec02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:57.928+0000 7f4ff8c00640 1 -- 172.21.15.7:0/806811637 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ffc0b58a0 con 0x7f4ffc0b4f50 2024-04-25T11:13:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.132+0000 7f5002a00640 1 -- 172.21.15.7:0/806811637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4ffc15ead0 con 0x7f4ffc0b4f50 2024-04-25T11:13:58.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.133+0000 7f4ff8c00640 1 -- 172.21.15.7:0/806811637 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f4ff405fdf0 con 0x7f4ffc0b4f50 2024-04-25T11:13:58.134 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:13:58.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.135+0000 7f5002a00640 1 -- 172.21.15.7:0/806811637 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4fd80744b0 msgr2=0x7f4fd80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.136+0000 7f5002a00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4fd80744b0 0x7f4fd80769c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4fec032000 tx=0x7f4fec02c000 comp rx=0 tx=0).stop 2024-04-25T11:13:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.136+0000 7f5002a00640 1 -- 172.21.15.7:0/806811637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ffc0b4f50 msgr2=0x7f4ffc0b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.136+0000 7f5002a00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ffc0b4f50 0x7f4ffc0b5360 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc074730 tx=0x7f4ff40115a0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.136+0000 7f5002a00640 1 -- 172.21.15.7:0/806811637 shutdown_connections 2024-04-25T11:13:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.136+0000 7f5002a00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4fd80744b0 0x7f4fd80769c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.136+0000 7f5002a00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ffc0b4f50 0x7f4ffc0b5360 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.136+0000 7f5002a00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ffc155bb0 0x7f4ffc151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.136+0000 7f5002a00640 1 --2- 172.21.15.7:0/806811637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fe00039e0 0x7f4ffc15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.136+0000 7f5002a00640 1 -- 172.21.15.7:0/806811637 >> 172.21.15.7:0/806811637 conn(0x7f4ffc0742c0 msgr2=0x7f4ffc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.136+0000 7f5002a00640 1 -- 172.21.15.7:0/806811637 shutdown_connections 2024-04-25T11:13:58.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.136+0000 7f5002a00640 1 -- 172.21.15.7:0/806811637 wait complete. 2024-04-25T11:13:58.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:13:58.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.453+0000 7f71c1400640 1 Processor -- start 2024-04-25T11:13:58.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.454+0000 7f71c1400640 1 -- start start 2024-04-25T11:13:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.459+0000 7f71c1400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71bc144a80 0x7f71bc146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.459+0000 7f71c1400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71bc0b5b30 con 0x7f71bc0b4f80 2024-04-25T11:13:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.459+0000 7f71c0a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f71bc0b4f80 0x7f71bc166720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55578/0 (socket says 172.21.15.7:55578) 2024-04-25T11:13:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.459+0000 7f71c0a00640 1 -- 172.21.15.7:0/713352997 learned_addr learned my addr 172.21.15.7:0/713352997 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.460+0000 7f71c1400640 1 -- 172.21.15.7:0/713352997 --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71bc0b5e10 con 0x7f71bc145e40 2024-04-25T11:13:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.460+0000 7f71c1400640 1 -- 172.21.15.7:0/713352997 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f71bc0b60f0 con 0x7f71bc144a80 2024-04-25T11:13:58.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.460+0000 7f71bb400640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71bc144a80 0x7f71bc146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:58.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.460+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3499175643 0 0) 0x7f71bc0b5b30 con 0x7f71bc0b4f80 2024-04-25T11:13:58.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.460+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71a0003520 con 0x7f71bc0b4f80 2024-04-25T11:13:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2324233486 0 0) 0x7f71a0003520 con 0x7f71bc0b4f80 2024-04-25T11:13:58.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71bc144a80 msgr2=0x7f71bc146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71bc144a80 0x7f71bc146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 >> v1:172.21.15.87:6789/0 conn(0x7f71bc145e40 legacy=0x7f71bc0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc0b63d0 con 0x7f71bc0b4f80 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f71b40019f0 con 0x7f71bc0b4f80 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 >> v1:172.21.15.7:6789/0 conn(0x7f71bc0b4f80 legacy=0x7f71bc166720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a00039e0 0x7f71a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71a0004800 0x7f71a0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71bc144a80 0x7f71a00052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc0b60f0 con 0x7f71a00039e0 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f71a000bbb0 con 0x7f71a0004800 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f71a000bdb0 con 0x7f71bc144a80 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71bbe00640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a00039e0 0x7f71a0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.461+0000 7f71c0a00640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71bc144a80 0x7f71a00052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:58.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.462+0000 7f71bb400640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71a0004800 0x7f71a0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:58.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.462+0000 7f71bbe00640 1 -- 172.21.15.7:0/713352997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71bc144a80 msgr2=0x7f71a00052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:58.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.462+0000 7f71bbe00640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71bc144a80 0x7f71a00052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.462+0000 7f71bbe00640 1 -- 172.21.15.7:0/713352997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71a0004800 msgr2=0x7f71a0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:58.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.462+0000 7f71bbe00640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71a0004800 0x7f71a0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.462+0000 7f71bbe00640 1 -- 172.21.15.7:0/713352997 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a0003520 con 0x7f71a00039e0 2024-04-25T11:13:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.463+0000 7f71bbe00640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a00039e0 0x7f71a0003e50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f71ac001720 tx=0x7f71a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e611eee7ae75d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.463+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac001370 con 0x7f71a00039e0 2024-04-25T11:13:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.464+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71ac0052d0 con 0x7f71a00039e0 2024-04-25T11:13:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.464+0000 7f71baa00640 1 -- 172.21.15.7:0/713352997 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac02abc0 con 0x7f71a00039e0 2024-04-25T11:13:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.464+0000 7f71c1400640 1 -- 172.21.15.7:0/713352997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a00039e0 msgr2=0x7f71a0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.464+0000 7f71c1400640 1 --2- 172.21.15.7:0/713352997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71a00039e0 0x7f71a0003e50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f71ac001720 tx=0x7f71a0005830 comp rx=0 tx=0).stop 2024-04-25T11:13:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.464+0000 7f71c0a00640 1 -- 172.21.15.7:0/713352997 reap_dead start 2024-04-25T11:13:58.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.465+0000 7f71c1400640 1 -- 172.21.15.7:0/713352997 shutdown_connections 2024-04-25T11:13:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.465+0000 7f71c1400640 1 -- 172.21.15.7:0/713352997 >> 172.21.15.7:0/713352997 conn(0x7f71bc0740f0 msgr2=0x7f71bc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.465+0000 7f71c1400640 1 -- 172.21.15.7:0/713352997 shutdown_connections 2024-04-25T11:13:58.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.465+0000 7f71c1400640 1 -- 172.21.15.7:0/713352997 wait complete. 2024-04-25T11:13:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.467+0000 7f71c1400640 1 Processor -- start 2024-04-25T11:13:58.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.467+0000 7f71c1400640 1 -- start start 2024-04-25T11:13:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.469+0000 7f71c1400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71a00039e0 0x7f71bc1e3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.469+0000 7f71c1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc13f720 0x7f71bc0abd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.469+0000 7f71c1400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71bc0ac270 0x7f71bc1318e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.469+0000 7f71c1400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc146620 con 0x7f71bc13f720 2024-04-25T11:13:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.469+0000 7f71c1400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1464a0 con 0x7f71bc0ac270 2024-04-25T11:13:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.469+0000 7f71c1400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f71bc0b5390 con 0x7f71a00039e0 2024-04-25T11:13:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.469+0000 7f71bb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc13f720 0x7f71bc0abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.469+0000 7f71bb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc13f720 0x7f71bc0abd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44520/0 (socket says 172.21.15.7:44520) 2024-04-25T11:13:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.469+0000 7f71bb400640 1 -- 172.21.15.7:0/3443158544 learned_addr learned my addr 172.21.15.7:0/3443158544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.470+0000 7f71bbe00640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71a00039e0 0x7f71bc1e3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.470+0000 7f71bb400640 1 -- 172.21.15.7:0/3443158544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71a00039e0 msgr2=0x7f71bc1e3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.470+0000 7f71bb400640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71a00039e0 0x7f71bc1e3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.470+0000 7f71bb400640 1 -- 172.21.15.7:0/3443158544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71bc0ac270 msgr2=0x7f71bc1318e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.470+0000 7f71c0a00640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71bc0ac270 0x7f71bc1318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.470+0000 7f71bb400640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71bc0ac270 0x7f71bc1318e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.470+0000 7f71bb400640 1 -- 172.21.15.7:0/3443158544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc0ad9c0 con 0x7f71bc13f720 2024-04-25T11:13:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.470+0000 7f71bb400640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc13f720 0x7f71bc0abd30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f71a80089e0 tx=0x7f71a8008ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.471+0000 7f71b3e00640 1 -- 172.21.15.7:0/3443158544 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8009b10 con 0x7f71bc13f720 2024-04-25T11:13:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.471+0000 7f71b3e00640 1 -- 172.21.15.7:0/3443158544 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a8009cb0 con 0x7f71bc13f720 2024-04-25T11:13:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.471+0000 7f71b3e00640 1 -- 172.21.15.7:0/3443158544 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8009e60 con 0x7f71bc13f720 2024-04-25T11:13:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.471+0000 7f71c1400640 1 -- 172.21.15.7:0/3443158544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc0adca0 con 0x7f71bc13f720 2024-04-25T11:13:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.471+0000 7f71c1400640 1 -- 172.21.15.7:0/3443158544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc12f880 con 0x7f71bc13f720 2024-04-25T11:13:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.472+0000 7f71c1400640 1 -- 172.21.15.7:0/3443158544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71bc050740 con 0x7f71bc13f720 2024-04-25T11:13:58.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.474+0000 7f71b3e00640 1 -- 172.21.15.7:0/3443158544 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f71a8017650 con 0x7f71bc13f720 2024-04-25T11:13:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.475+0000 7f71b3e00640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f719c0744b0 0x7f719c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.475+0000 7f71b3e00640 1 -- 172.21.15.7:0/3443158544 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f71a80950a0 con 0x7f71bc13f720 2024-04-25T11:13:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.475+0000 7f71bbe00640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f719c0744b0 0x7f719c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.476+0000 7f71bbe00640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f719c0744b0 0x7f719c0769c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f71ac032000 tx=0x7f71ac02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.480+0000 7f71b3e00640 1 -- 172.21.15.7:0/3443158544 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71a805f720 con 0x7f71bc13f720 2024-04-25T11:13:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.684+0000 7f71c1400640 1 -- 172.21.15.7:0/3443158544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f71bc06da70 con 0x7f71bc13f720 2024-04-25T11:13:58.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.685+0000 7f71b3e00640 1 -- 172.21.15.7:0/3443158544 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) ==== 104+0+11 (secure 0 0 0) 0x7f71a805f020 con 0x7f71bc13f720 2024-04-25T11:13:58.686 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:13:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.687+0000 7f71c1400640 1 -- 172.21.15.7:0/3443158544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f719c0744b0 msgr2=0x7f719c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.687+0000 7f71c1400640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f719c0744b0 0x7f719c0769c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f71ac032000 tx=0x7f71ac02e000 comp rx=0 tx=0).stop 2024-04-25T11:13:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.687+0000 7f71c1400640 1 -- 172.21.15.7:0/3443158544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc13f720 msgr2=0x7f71bc0abd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.687+0000 7f71c1400640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc13f720 0x7f71bc0abd30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f71a80089e0 tx=0x7f71a8008ad0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.687+0000 7f71c1400640 1 -- 172.21.15.7:0/3443158544 shutdown_connections 2024-04-25T11:13:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.687+0000 7f71c1400640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f719c0744b0 0x7f719c0769c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.687+0000 7f71c1400640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71bc0ac270 0x7f71bc1318e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.688+0000 7f71c1400640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71bc13f720 0x7f71bc0abd30 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.688+0000 7f71c1400640 1 --2- 172.21.15.7:0/3443158544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71a00039e0 0x7f71bc1e3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.688+0000 7f71c1400640 1 -- 172.21.15.7:0/3443158544 >> 172.21.15.7:0/3443158544 conn(0x7f71bc0740f0 msgr2=0x7f71bc0b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.688+0000 7f71c1400640 1 -- 172.21.15.7:0/3443158544 shutdown_connections 2024-04-25T11:13:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.688+0000 7f71c1400640 1 -- 172.21.15.7:0/3443158544 wait complete. 2024-04-25T11:13:58.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:13:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.997+0000 7f2a30c00640 1 Processor -- start 2024-04-25T11:13:58.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:58.997+0000 7f2a30c00640 1 -- start start 2024-04-25T11:13:59.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.000+0000 7f2a30c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0cfa80 0x7f2a2c0d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.000+0000 7f2a30c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2a2c0d0800 0x7f2a2c06eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.000+0000 7f2a30c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a2c0d0c10 con 0x7f2a2c070630 2024-04-25T11:13:59.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.000+0000 7f2a30c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2a2c0b0be0 con 0x7f2a2c0d0800 2024-04-25T11:13:59.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.000+0000 7f2a30c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2c0cfe90 con 0x7f2a2c0cfa80 2024-04-25T11:13:59.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.000+0000 7f2a2b400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2a2c070630 0x7f2a2c15ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55582/0 (socket says 172.21.15.7:55582) 2024-04-25T11:13:59.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.000+0000 7f2a2b400640 1 -- 172.21.15.7:0/1364902667 learned_addr learned my addr 172.21.15.7:0/1364902667 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:13:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.001+0000 7f2a2aa00640 1 --2- 172.21.15.7:0/1364902667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0cfa80 0x7f2a2c0d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.001+0000 7f2a2be00640 1 --2- 172.21.15.7:0/1364902667 >> v2:172.21.15.87:3300/0 conn(0x7f2a2c0d0800 0x7f2a2c06eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.001+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1571365303 0 0) 0x7f2a1c001110 con 0x7f2a2c070630 2024-04-25T11:13:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.001+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a10003520 con 0x7f2a2c070630 2024-04-25T11:13:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.001+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1057291233 0 0) 0x7f2a10003520 con 0x7f2a2c070630 2024-04-25T11:13:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.001+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0cfa80 msgr2=0x7f2a2c0d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.001+0000 7f2a2a000640 1 --2- 172.21.15.7:0/1364902667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0cfa80 0x7f2a2c0d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.001+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 >> v2:172.21.15.87:3300/0 conn(0x7f2a2c0d0800 msgr2=0x7f2a2c06eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:59.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.001+0000 7f2a2a000640 1 --2- 172.21.15.7:0/1364902667 >> v2:172.21.15.87:3300/0 conn(0x7f2a2c0d0800 0x7f2a2c06eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.001+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c0ce9e0 con 0x7f2a2c070630 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.001+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2a1c001840 con 0x7f2a2c070630 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 >> v1:172.21.15.7:6789/0 conn(0x7f2a2c070630 legacy=0x7f2a2c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2a000640 1 --2- 172.21.15.7:0/1364902667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0cfa80 0x7f2a100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2a000640 1 --2- 172.21.15.7:0/1364902667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a10003f30 0x7f2a10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2a000640 1 --2- 172.21.15.7:0/1364902667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10004d40 0x7f2a10005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c0b0be0 con 0x7f2a10003f30 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c0cfe90 con 0x7f2a10004d40 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1000ba10 con 0x7f2a2c0cfa80 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2aa00640 1 --2- 172.21.15.7:0/1364902667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a10003f30 0x7f2a10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2b400640 1 --2- 172.21.15.7:0/1364902667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0cfa80 0x7f2a100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2aa00640 1 -- 172.21.15.7:0/1364902667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0cfa80 msgr2=0x7f2a100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2aa00640 1 --2- 172.21.15.7:0/1364902667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0cfa80 0x7f2a100039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2aa00640 1 -- 172.21.15.7:0/1364902667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10004d40 msgr2=0x7f2a10005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.002+0000 7f2a2aa00640 1 --2- 172.21.15.7:0/1364902667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10004d40 0x7f2a10005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.003+0000 7f2a2aa00640 1 -- 172.21.15.7:0/1364902667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a10003520 con 0x7f2a10003f30 2024-04-25T11:13:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.004+0000 7f2a2aa00640 1 --2- 172.21.15.7:0/1364902667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a10003f30 0x7f2a10004800 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009a90 tx=0x7f2a10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf9bc08eb6abe0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.004+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18036070 con 0x7f2a10003f30 2024-04-25T11:13:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.004+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a18031400 con 0x7f2a10003f30 2024-04-25T11:13:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.004+0000 7f2a2a000640 1 -- 172.21.15.7:0/1364902667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18009690 con 0x7f2a10003f30 2024-04-25T11:13:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.005+0000 7f2a30c00640 1 -- 172.21.15.7:0/1364902667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a10003f30 msgr2=0x7f2a10004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.005+0000 7f2a30c00640 1 --2- 172.21.15.7:0/1364902667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a10003f30 0x7f2a10004800 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009a90 tx=0x7f2a10005830 comp rx=0 tx=0).stop 2024-04-25T11:13:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.005+0000 7f2a2be00640 1 -- 172.21.15.7:0/1364902667 reap_dead start 2024-04-25T11:13:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.005+0000 7f2a30c00640 1 -- 172.21.15.7:0/1364902667 shutdown_connections 2024-04-25T11:13:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.005+0000 7f2a30c00640 1 -- 172.21.15.7:0/1364902667 >> 172.21.15.7:0/1364902667 conn(0x7f2a2c074140 msgr2=0x7f2a2c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.005+0000 7f2a30c00640 1 -- 172.21.15.7:0/1364902667 shutdown_connections 2024-04-25T11:13:59.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.005+0000 7f2a30c00640 1 -- 172.21.15.7:0/1364902667 wait complete. 2024-04-25T11:13:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.007+0000 7f2a30c00640 1 Processor -- start 2024-04-25T11:13:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.007+0000 7f2a30c00640 1 -- start start 2024-04-25T11:13:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.009+0000 7f2a30c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10003f30 0x7f2a2c1f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.009+0000 7f2a30c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a2c157bb0 0x7f2a2c0b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.009+0000 7f2a30c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0b6590 0x7f2a2c15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.009+0000 7f2a30c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c0cf280 con 0x7f2a2c157bb0 2024-04-25T11:13:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.009+0000 7f2a30c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2c0cefa0 con 0x7f2a10003f30 2024-04-25T11:13:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.009+0000 7f2a30c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2c0cf560 con 0x7f2a2c0b6590 2024-04-25T11:13:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.009+0000 7f2a2aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a2c157bb0 0x7f2a2c0b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.009+0000 7f2a2aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a2c157bb0 0x7f2a2c0b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44536/0 (socket says 172.21.15.7:44536) 2024-04-25T11:13:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.009+0000 7f2a2aa00640 1 -- 172.21.15.7:0/325851873 learned_addr learned my addr 172.21.15.7:0/325851873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.010+0000 7f2a2b400640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10003f30 0x7f2a2c1f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.010+0000 7f2a2be00640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0b6590 0x7f2a2c15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.010+0000 7f2a2aa00640 1 -- 172.21.15.7:0/325851873 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0b6590 msgr2=0x7f2a2c15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.010+0000 7f2a2aa00640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0b6590 0x7f2a2c15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.010+0000 7f2a2aa00640 1 -- 172.21.15.7:0/325851873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10003f30 msgr2=0x7f2a2c1f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.010+0000 7f2a2aa00640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10003f30 0x7f2a2c1f6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.010+0000 7f2a2aa00640 1 -- 172.21.15.7:0/325851873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2c15b940 con 0x7f2a2c157bb0 2024-04-25T11:13:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.010+0000 7f2a2be00640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0b6590 0x7f2a2c15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:13:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.010+0000 7f2a2aa00640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a2c157bb0 0x7f2a2c0b6050 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2a18034040 tx=0x7f2a18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.010+0000 7f2a23400640 1 -- 172.21.15.7:0/325851873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18040020 con 0x7f2a2c157bb0 2024-04-25T11:13:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.010+0000 7f2a23400640 1 -- 172.21.15.7:0/325851873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a18003a40 con 0x7f2a2c157bb0 2024-04-25T11:13:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.011+0000 7f2a23400640 1 -- 172.21.15.7:0/325851873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a18036040 con 0x7f2a2c157bb0 2024-04-25T11:13:59.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.011+0000 7f2a30c00640 1 -- 172.21.15.7:0/325851873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2c15bc20 con 0x7f2a2c157bb0 2024-04-25T11:13:59.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.011+0000 7f2a30c00640 1 -- 172.21.15.7:0/325851873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a2c159b30 con 0x7f2a2c157bb0 2024-04-25T11:13:59.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.012+0000 7f2a30c00640 1 -- 172.21.15.7:0/325851873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a2c050740 con 0x7f2a2c157bb0 2024-04-25T11:13:59.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.013+0000 7f2a23400640 1 -- 172.21.15.7:0/325851873 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2a18043020 con 0x7f2a2c157bb0 2024-04-25T11:13:59.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.014+0000 7f2a23400640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744b0 0x7f2a0c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.014+0000 7f2a23400640 1 -- 172.21.15.7:0/325851873 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f2a180b53b0 con 0x7f2a2c157bb0 2024-04-25T11:13:59.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.014+0000 7f2a2b400640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744b0 0x7f2a0c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:59.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.015+0000 7f2a2b400640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744b0 0x7f2a0c0769c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0019b0 tx=0x7f2a1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:59.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.019+0000 7f2a23400640 1 -- 172.21.15.7:0/325851873 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a1804a070 con 0x7f2a2c157bb0 2024-04-25T11:13:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.214+0000 7f2a30c00640 1 -- 172.21.15.7:0/325851873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2a2c070dd0 con 0x7f2a2c157bb0 2024-04-25T11:13:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.215+0000 7f2a23400640 1 -- 172.21.15.7:0/325851873 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) ==== 100+0+11 (secure 0 0 0) 0x7f2a1807f9a0 con 0x7f2a2c157bb0 2024-04-25T11:13:59.217 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:13:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 -- 172.21.15.7:0/325851873 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744b0 msgr2=0x7f2a0c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744b0 0x7f2a0c0769c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0019b0 tx=0x7f2a1c009040 comp rx=0 tx=0).stop 2024-04-25T11:13:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 -- 172.21.15.7:0/325851873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a2c157bb0 msgr2=0x7f2a2c0b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a2c157bb0 0x7f2a2c0b6050 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2a18034040 tx=0x7f2a18004730 comp rx=0 tx=0).stop 2024-04-25T11:13:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 -- 172.21.15.7:0/325851873 shutdown_connections 2024-04-25T11:13:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2a0c0744b0 0x7f2a0c0769c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a2c0b6590 0x7f2a2c15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a2c157bb0 0x7f2a2c0b6050 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 --2- 172.21.15.7:0/325851873 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a10003f30 0x7f2a2c1f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 -- 172.21.15.7:0/325851873 >> 172.21.15.7:0/325851873 conn(0x7f2a2c074140 msgr2=0x7f2a2c0b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 -- 172.21.15.7:0/325851873 shutdown_connections 2024-04-25T11:13:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.218+0000 7f2a30c00640 1 -- 172.21.15.7:0/325851873 wait complete. 2024-04-25T11:13:59.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:13:59.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.530+0000 7f0d70600640 1 Processor -- start 2024-04-25T11:13:59.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.530+0000 7f0d70600640 1 -- start start 2024-04-25T11:13:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.533+0000 7f0d70600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d6815f180 0x7f0d68161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.533+0000 7f0d70600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d68162700 con 0x7f0d6806dc90 2024-04-25T11:13:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.533+0000 7f0d70600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d681629e0 con 0x7f0d681621a0 2024-04-25T11:13:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.533+0000 7f0d70600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0d6815f590 con 0x7f0d6815f180 2024-04-25T11:13:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.533+0000 7f0d6dc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d6815f180 0x7f0d68161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.533+0000 7f0d6dc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d6815f180 0x7f0d68161c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44538/0 (socket says 172.21.15.7:44538) 2024-04-25T11:13:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.533+0000 7f0d6dc00640 1 -- 172.21.15.7:0/199825537 learned_addr learned my addr 172.21.15.7:0/199825537 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.534+0000 7f0d6dc00640 1 -- 172.21.15.7:0/199825537 >> v1:172.21.15.87:6789/0 conn(0x7f0d681621a0 legacy=0x7f0d6806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:13:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.534+0000 7f0d6dc00640 1 -- 172.21.15.7:0/199825537 >> v1:172.21.15.7:6789/0 conn(0x7f0d6806dc90 legacy=0x7f0d6806f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.534+0000 7f0d6dc00640 1 -- 172.21.15.7:0/199825537 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6815e980 con 0x7f0d6815f180 2024-04-25T11:13:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.534+0000 7f0d67e00640 1 -- 172.21.15.7:0/199825537 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2850488204 0 0) 0x7f0d68162700 con 0x7f0d6806dc90 2024-04-25T11:13:59.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.535+0000 7f0d6dc00640 1 --2- 172.21.15.7:0/199825537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d6815f180 0x7f0d68161c60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009870 tx=0x7f0d5c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ccd775b7792570 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:59.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.535+0000 7f0d67e00640 1 -- 172.21.15.7:0/199825537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c037070 con 0x7f0d6815f180 2024-04-25T11:13:59.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.535+0000 7f0d67e00640 1 -- 172.21.15.7:0/199825537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5c032440 con 0x7f0d6815f180 2024-04-25T11:13:59.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.535+0000 7f0d67e00640 1 -- 172.21.15.7:0/199825537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c032810 con 0x7f0d6815f180 2024-04-25T11:13:59.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.536+0000 7f0d70600640 1 -- 172.21.15.7:0/199825537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d6815f180 msgr2=0x7f0d68161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.536+0000 7f0d70600640 1 --2- 172.21.15.7:0/199825537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d6815f180 0x7f0d68161c60 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009870 tx=0x7f0d5c02a210 comp rx=0 tx=0).stop 2024-04-25T11:13:59.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.536+0000 7f0d70600640 1 -- 172.21.15.7:0/199825537 shutdown_connections 2024-04-25T11:13:59.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.536+0000 7f0d70600640 1 --2- 172.21.15.7:0/199825537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d6815f180 0x7f0d68161c60 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.536+0000 7f0d70600640 1 -- 172.21.15.7:0/199825537 >> 172.21.15.7:0/199825537 conn(0x7f0d680732b0 msgr2=0x7f0d680736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:59.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.536+0000 7f0d70600640 1 -- 172.21.15.7:0/199825537 shutdown_connections 2024-04-25T11:13:59.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.536+0000 7f0d70600640 1 -- 172.21.15.7:0/199825537 wait complete. 2024-04-25T11:13:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.538+0000 7f0d70600640 1 Processor -- start 2024-04-25T11:13:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.539+0000 7f0d70600640 1 -- start start 2024-04-25T11:13:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.540+0000 7f0d70600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d6815f180 0x7f0d6809b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.540+0000 7f0d70600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d681621a0 0x7f0d68146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.540+0000 7f0d70600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d6809eee0 0x7f0d68155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.540+0000 7f0d70600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6815e670 con 0x7f0d6815f180 2024-04-25T11:13:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.540+0000 7f0d70600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0d68158ae0 con 0x7f0d6809eee0 2024-04-25T11:13:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.540+0000 7f0d70600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0d680d1310 con 0x7f0d681621a0 2024-04-25T11:13:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.540+0000 7f0d6dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d6815f180 0x7f0d6809b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.540+0000 7f0d6dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d6815f180 0x7f0d6809b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44546/0 (socket says 172.21.15.7:44546) 2024-04-25T11:13:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.540+0000 7f0d6dc00640 1 -- 172.21.15.7:0/2395066580 learned_addr learned my addr 172.21.15.7:0/2395066580 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:13:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.541+0000 7f0d6dc00640 1 -- 172.21.15.7:0/2395066580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d681621a0 msgr2=0x7f0d68146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.541+0000 7f0d6dc00640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d681621a0 0x7f0d68146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.541+0000 7f0d6dc00640 1 -- 172.21.15.7:0/2395066580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d6809eee0 msgr2=0x7f0d68155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.541+0000 7f0d6dc00640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d6809eee0 0x7f0d68155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.541+0000 7f0d6dc00640 1 -- 172.21.15.7:0/2395066580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d68156de0 con 0x7f0d6815f180 2024-04-25T11:13:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.541+0000 7f0d6dc00640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d6815f180 0x7f0d6809b540 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c000c00 tx=0x7f0d5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.542+0000 7f0d65600640 1 -- 172.21.15.7:0/2395066580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c037040 con 0x7f0d6815f180 2024-04-25T11:13:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.542+0000 7f0d65600640 1 -- 172.21.15.7:0/2395066580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5c042070 con 0x7f0d6815f180 2024-04-25T11:13:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.542+0000 7f0d70600640 1 -- 172.21.15.7:0/2395066580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6809ba80 con 0x7f0d6815f180 2024-04-25T11:13:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.542+0000 7f0d65600640 1 -- 172.21.15.7:0/2395066580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c030bb0 con 0x7f0d6815f180 2024-04-25T11:13:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.542+0000 7f0d70600640 1 -- 172.21.15.7:0/2395066580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d68156b00 con 0x7f0d6815f180 2024-04-25T11:13:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.543+0000 7f0d70600640 1 -- 172.21.15.7:0/2395066580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d68050740 con 0x7f0d6815f180 2024-04-25T11:13:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.544+0000 7f0d65600640 1 -- 172.21.15.7:0/2395066580 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0d5c047460 con 0x7f0d6815f180 2024-04-25T11:13:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.546+0000 7f0d65600640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d480744b0 0x7f0d480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:13:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.546+0000 7f0d65600640 1 -- 172.21.15.7:0/2395066580 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f0d5c0b6060 con 0x7f0d6815f180 2024-04-25T11:13:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.546+0000 7f0d6d200640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d480744b0 0x7f0d480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:13:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.547+0000 7f0d6d200640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d480744b0 0x7f0d480769c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0d54006fd0 tx=0x7f0d54006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:13:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.551+0000 7f0d65600640 1 -- 172.21.15.7:0/2395066580 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0d5c080650 con 0x7f0d6815f180 2024-04-25T11:13:59.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.769+0000 7f0d70600640 1 -- 172.21.15.7:0/2395066580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0d68157790 con 0x7f0d6815f180 2024-04-25T11:13:59.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.769+0000 7f0d65600640 1 -- 172.21.15.7:0/2395066580 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) ==== 80+0+0 (secure 0 0 0) 0x7f0d5c07ff50 con 0x7f0d6815f180 2024-04-25T11:13:59.771 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:13:59.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.772+0000 7f0d70600640 1 -- 172.21.15.7:0/2395066580 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d480744b0 msgr2=0x7f0d480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.772+0000 7f0d70600640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d480744b0 0x7f0d480769c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0d54006fd0 tx=0x7f0d54006bc0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.772+0000 7f0d70600640 1 -- 172.21.15.7:0/2395066580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d6815f180 msgr2=0x7f0d6809b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:13:59.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.772+0000 7f0d70600640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d6815f180 0x7f0d6809b540 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c000c00 tx=0x7f0d5c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.772+0000 7f0d70600640 1 -- 172.21.15.7:0/2395066580 shutdown_connections 2024-04-25T11:13:59.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.772+0000 7f0d70600640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d480744b0 0x7f0d480769c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.772+0000 7f0d70600640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d6809eee0 0x7f0d68155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.772+0000 7f0d70600640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d681621a0 0x7f0d68146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.772+0000 7f0d70600640 1 --2- 172.21.15.7:0/2395066580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d6815f180 0x7f0d6809b540 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:13:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.772+0000 7f0d70600640 1 -- 172.21.15.7:0/2395066580 >> 172.21.15.7:0/2395066580 conn(0x7f0d680732b0 msgr2=0x7f0d6806d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:13:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.773+0000 7f0d70600640 1 -- 172.21.15.7:0/2395066580 shutdown_connections 2024-04-25T11:13:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:13:59.773+0000 7f0d70600640 1 -- 172.21.15.7:0/2395066580 wait complete. 2024-04-25T11:13:59.800 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:13:59.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:59.800 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T11:13:59.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:13:59.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:13:59.898 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:13:59.950 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:13:59.950 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:13:59.951 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:00.008 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:14:00.008 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:14:00.008 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:14:00.008 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:00.065 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:14:00.065 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.355+0000 7fd846600640 1 Processor -- start 2024-04-25T11:14:00.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.355+0000 7fd846600640 1 -- start start 2024-04-25T11:14:00.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.357+0000 7fd846600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8400d2d90 0x7fd84009ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.357+0000 7fd846600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8400d31a0 0x7fd84009bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.357+0000 7fd846600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8400d1cf0 con 0x7fd8400cddb0 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.357+0000 7fd846600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd8400d1fd0 con 0x7fd8400d31a0 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.357+0000 7fd846600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd8400d22b0 con 0x7fd8400d2d90 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.357+0000 7fd83fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8400d31a0 0x7fd84009bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.357+0000 7fd83fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8400d31a0 0x7fd84009bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37802/0 (socket says 172.21.15.7:37802) 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.357+0000 7fd83fe00640 1 -- 172.21.15.7:0/3308765977 learned_addr learned my addr 172.21.15.7:0/3308765977 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.357+0000 7fd845200640 1 --2- 172.21.15.7:0/3308765977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8400d2d90 0x7fd84009ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.358+0000 7fd83fe00640 1 -- 172.21.15.7:0/3308765977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8400d2d90 msgr2=0x7fd84009ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.358+0000 7fd83fe00640 1 --2- 172.21.15.7:0/3308765977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8400d2d90 0x7fd84009ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.358+0000 7fd83fe00640 1 -- 172.21.15.7:0/3308765977 >> v1:172.21.15.87:6789/0 conn(0x7fd8400cddb0 legacy=0x7fd8400ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.358+0000 7fd83fe00640 1 -- 172.21.15.7:0/3308765977 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8400d2590 con 0x7fd8400d31a0 2024-04-25T11:14:00.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.358+0000 7fd845200640 1 --2- 172.21.15.7:0/3308765977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8400d2d90 0x7fd84009ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.359+0000 7fd83fe00640 1 --2- 172.21.15.7:0/3308765977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8400d31a0 0x7fd84009bf40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009870 tx=0x7fd82c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f182353870e315b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.359+0000 7fd83f400640 1 -- 172.21.15.7:0/3308765977 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c037070 con 0x7fd8400d31a0 2024-04-25T11:14:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.359+0000 7fd83f400640 1 -- 172.21.15.7:0/3308765977 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd82c032440 con 0x7fd8400d31a0 2024-04-25T11:14:00.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.359+0000 7fd83f400640 1 -- 172.21.15.7:0/3308765977 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c032810 con 0x7fd8400d31a0 2024-04-25T11:14:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.360+0000 7fd846600640 1 -- 172.21.15.7:0/3308765977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8400d31a0 msgr2=0x7fd84009bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.360+0000 7fd846600640 1 --2- 172.21.15.7:0/3308765977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8400d31a0 0x7fd84009bf40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009870 tx=0x7fd82c02a210 comp rx=0 tx=0).stop 2024-04-25T11:14:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.360+0000 7fd846600640 1 -- 172.21.15.7:0/3308765977 shutdown_connections 2024-04-25T11:14:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.360+0000 7fd846600640 1 --2- 172.21.15.7:0/3308765977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8400d31a0 0x7fd84009bf40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.360+0000 7fd846600640 1 --2- 172.21.15.7:0/3308765977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8400d2d90 0x7fd84009ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.360+0000 7fd846600640 1 -- 172.21.15.7:0/3308765977 >> 172.21.15.7:0/3308765977 conn(0x7fd8400b5250 msgr2=0x7fd8400b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:00.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.360+0000 7fd846600640 1 -- 172.21.15.7:0/3308765977 shutdown_connections 2024-04-25T11:14:00.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.360+0000 7fd846600640 1 -- 172.21.15.7:0/3308765977 wait complete. 2024-04-25T11:14:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.362+0000 7fd846600640 1 Processor -- start 2024-04-25T11:14:00.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.363+0000 7fd846600640 1 -- start start 2024-04-25T11:14:00.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.364+0000 7fd846600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8400cddb0 0x7fd84015f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.364+0000 7fd846600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8400d2d90 0x7fd8400c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.364+0000 7fd846600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8400d31a0 0x7fd8401eb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.364+0000 7fd846600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8400b91e0 con 0x7fd8400d2d90 2024-04-25T11:14:00.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.364+0000 7fd846600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd8400d19a0 con 0x7fd8400d31a0 2024-04-25T11:14:00.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.364+0000 7fd846600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd84006dce0 con 0x7fd8400cddb0 2024-04-25T11:14:00.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.364+0000 7fd83fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8400d2d90 0x7fd8400c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:00.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.365+0000 7fd83fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8400d2d90 0x7fd8400c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37818/0 (socket says 172.21.15.7:37818) 2024-04-25T11:14:00.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.365+0000 7fd83fe00640 1 -- 172.21.15.7:0/2258691164 learned_addr learned my addr 172.21.15.7:0/2258691164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:00.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.365+0000 7fd845200640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8400cddb0 0x7fd84015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:00.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.365+0000 7fd83fe00640 1 -- 172.21.15.7:0/2258691164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8400cddb0 msgr2=0x7fd84015f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:00.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.365+0000 7fd83fe00640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8400cddb0 0x7fd84015f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.365+0000 7fd83fe00640 1 -- 172.21.15.7:0/2258691164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8400d31a0 msgr2=0x7fd8401eb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:00.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.365+0000 7fd83fe00640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8400d31a0 0x7fd8401eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.365+0000 7fd83fe00640 1 -- 172.21.15.7:0/2258691164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8400c8010 con 0x7fd8400d2d90 2024-04-25T11:14:00.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.365+0000 7fd83fe00640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8400d2d90 0x7fd8400c9880 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0090b0 tx=0x7fd82c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:00.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.366+0000 7fd83cc00640 1 -- 172.21.15.7:0/2258691164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c041020 con 0x7fd8400d2d90 2024-04-25T11:14:00.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.366+0000 7fd83cc00640 1 -- 172.21.15.7:0/2258691164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd82c030990 con 0x7fd8400d2d90 2024-04-25T11:14:00.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.366+0000 7fd83cc00640 1 -- 172.21.15.7:0/2258691164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c037040 con 0x7fd8400d2d90 2024-04-25T11:14:00.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.366+0000 7fd846600640 1 -- 172.21.15.7:0/2258691164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8400c82f0 con 0x7fd8400d2d90 2024-04-25T11:14:00.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.366+0000 7fd846600640 1 -- 172.21.15.7:0/2258691164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8400c88b0 con 0x7fd8400d2d90 2024-04-25T11:14:00.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.367+0000 7fd846600640 1 -- 172.21.15.7:0/2258691164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84009c6e0 con 0x7fd8400d2d90 2024-04-25T11:14:00.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.368+0000 7fd83cc00640 1 -- 172.21.15.7:0/2258691164 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd82c044020 con 0x7fd8400d2d90 2024-04-25T11:14:00.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.369+0000 7fd83cc00640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8200744b0 0x7fd8200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.369+0000 7fd83cc00640 1 -- 172.21.15.7:0/2258691164 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fd82c0b5f70 con 0x7fd8400d2d90 2024-04-25T11:14:00.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.370+0000 7fd845200640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8200744b0 0x7fd8200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:00.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.370+0000 7fd845200640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8200744b0 0x7fd8200769c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd830008f30 tx=0x7fd830008480 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:00.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.374+0000 7fd83cc00640 1 -- 172.21.15.7:0/2258691164 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd82c04b030 con 0x7fd8400d2d90 2024-04-25T11:14:00.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.624+0000 7fd846600640 1 -- 172.21.15.7:0/2258691164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8400b8d30 con 0x7fd8400d2d90 2024-04-25T11:14:00.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.626+0000 7fd83cc00640 1 -- 172.21.15.7:0/2258691164 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5172 (secure 0 0 0) 0x7fd82c0804d0 con 0x7fd8400d2d90 2024-04-25T11:14:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:00.628 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2225639104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2225639104},{"type":"v1","addr":"172.21.15.87:6834","nonce":2225639104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2625900548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2625900548},{"type":"v1","addr":"172.21.15.7:6837","nonce":2625900548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/717352569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":717352569},{"type":"v1","addr":"172.21.15.87:6835","nonce":717352569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:13:42.310229+0000","modified":"2024-04-25T11:13:45.341610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6836/3503942462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3503942462},{"type":"v1","addr":"172.21.15.7:6836","nonce":3503942462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-25T11:14:00.628 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-25T11:14:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.629+0000 7fd846600640 1 -- 172.21.15.7:0/2258691164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8200744b0 msgr2=0x7fd8200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.629+0000 7fd846600640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8200744b0 0x7fd8200769c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd830008f30 tx=0x7fd830008480 comp rx=0 tx=0).stop 2024-04-25T11:14:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.629+0000 7fd846600640 1 -- 172.21.15.7:0/2258691164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8400d2d90 msgr2=0x7fd8400c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.629+0000 7fd846600640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8400d2d90 0x7fd8400c9880 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0090b0 tx=0x7fd82c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.629+0000 7fd846600640 1 -- 172.21.15.7:0/2258691164 shutdown_connections 2024-04-25T11:14:00.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.629+0000 7fd846600640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8200744b0 0x7fd8200769c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.629+0000 7fd846600640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8400d31a0 0x7fd8401eb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.629+0000 7fd846600640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8400d2d90 0x7fd8400c9880 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.629+0000 7fd846600640 1 --2- 172.21.15.7:0/2258691164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8400cddb0 0x7fd84015f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.629+0000 7fd846600640 1 -- 172.21.15.7:0/2258691164 >> 172.21.15.7:0/2258691164 conn(0x7fd8400b5250 msgr2=0x7fd84006e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:00.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.629+0000 7fd846600640 1 -- 172.21.15.7:0/2258691164 shutdown_connections 2024-04-25T11:14:00.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.630+0000 7fd846600640 1 -- 172.21.15.7:0/2258691164 wait complete. 2024-04-25T11:14:00.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:00.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.936+0000 7fabe2c00640 1 Processor -- start 2024-04-25T11:14:00.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.936+0000 7fabe2c00640 1 -- start start 2024-04-25T11:14:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.939+0000 7fabe2c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabdc0b4aa0 0x7fabdc0ae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.939+0000 7fabe2c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabdc164710 0x7fabdc0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.939+0000 7fabe2c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabdc164c80 con 0x7fabdc0b0da0 2024-04-25T11:14:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.939+0000 7fabe2c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fabdc1655b0 con 0x7fabdc164710 2024-04-25T11:14:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.939+0000 7fabe2c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fabdc165a40 con 0x7fabdc0b4aa0 2024-04-25T11:14:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.939+0000 7fabd3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabdc164710 0x7fabdc0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.939+0000 7fabd3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabdc164710 0x7fabdc0b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37826/0 (socket says 172.21.15.7:37826) 2024-04-25T11:14:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.939+0000 7fabd3e00640 1 -- 172.21.15.7:0/4255054196 learned_addr learned my addr 172.21.15.7:0/4255054196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.940+0000 7fabdbe00640 1 --2- 172.21.15.7:0/4255054196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabdc0b4aa0 0x7fabdc0ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.940+0000 7fabd3e00640 1 -- 172.21.15.7:0/4255054196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabdc0b4aa0 msgr2=0x7fabdc0ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.940+0000 7fabd3e00640 1 --2- 172.21.15.7:0/4255054196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabdc0b4aa0 0x7fabdc0ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.940+0000 7fabd3e00640 1 -- 172.21.15.7:0/4255054196 >> v1:172.21.15.7:6789/0 conn(0x7fabdc0b0da0 legacy=0x7fabdc0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.940+0000 7fabd3e00640 1 -- 172.21.15.7:0/4255054196 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc164f60 con 0x7fabdc164710 2024-04-25T11:14:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.940+0000 7fabdbe00640 1 --2- 172.21.15.7:0/4255054196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabdc0b4aa0 0x7fabdc0ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:14:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.941+0000 7fabd3e00640 1 --2- 172.21.15.7:0/4255054196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabdc164710 0x7fabdc0b0860 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009870 tx=0x7fabc402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a53f2fff1a78a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.942+0000 7fabdb400640 1 -- 172.21.15.7:0/4255054196 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4037070 con 0x7fabdc164710 2024-04-25T11:14:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.942+0000 7fabdb400640 1 -- 172.21.15.7:0/4255054196 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc4032440 con 0x7fabdc164710 2024-04-25T11:14:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.942+0000 7fabdb400640 1 -- 172.21.15.7:0/4255054196 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4032810 con 0x7fabdc164710 2024-04-25T11:14:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.942+0000 7fabe2c00640 1 -- 172.21.15.7:0/4255054196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabdc164710 msgr2=0x7fabdc0b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.942+0000 7fabe2c00640 1 --2- 172.21.15.7:0/4255054196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabdc164710 0x7fabdc0b0860 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009870 tx=0x7fabc402a250 comp rx=0 tx=0).stop 2024-04-25T11:14:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.943+0000 7fabe2c00640 1 -- 172.21.15.7:0/4255054196 shutdown_connections 2024-04-25T11:14:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.943+0000 7fabe2c00640 1 --2- 172.21.15.7:0/4255054196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabdc164710 0x7fabdc0b0860 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.943+0000 7fabe2c00640 1 --2- 172.21.15.7:0/4255054196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabdc0b4aa0 0x7fabdc0ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.943+0000 7fabe2c00640 1 -- 172.21.15.7:0/4255054196 >> 172.21.15.7:0/4255054196 conn(0x7fabdc074200 msgr2=0x7fabdc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.943+0000 7fabe2c00640 1 -- 172.21.15.7:0/4255054196 shutdown_connections 2024-04-25T11:14:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.943+0000 7fabe2c00640 1 -- 172.21.15.7:0/4255054196 wait complete. 2024-04-25T11:14:00.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.945+0000 7fabe2c00640 1 Processor -- start 2024-04-25T11:14:00.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.946+0000 7fabe2c00640 1 -- start start 2024-04-25T11:14:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.947+0000 7fabe2c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabdc0b0da0 0x7fabdc1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.948+0000 7fabe2c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabdc0b4aa0 0x7fabdc159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.948+0000 7fabe2c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabdc164710 0x7fabdc15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.948+0000 7fabe2c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc06f300 con 0x7fabdc164710 2024-04-25T11:14:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.948+0000 7fabe2c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc0af340 con 0x7fabdc0b4aa0 2024-04-25T11:14:00.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.948+0000 7fabe2c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fabdc0b11b0 con 0x7fabdc0b0da0 2024-04-25T11:14:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.948+0000 7fabe0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabdc164710 0x7fabdc15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.948+0000 7fabe0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabdc164710 0x7fabdc15a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37830/0 (socket says 172.21.15.7:37830) 2024-04-25T11:14:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.948+0000 7fabe0c00640 1 -- 172.21.15.7:0/937743434 learned_addr learned my addr 172.21.15.7:0/937743434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.948+0000 7fabdbe00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabdc0b0da0 0x7fabdc1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:00.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.948+0000 7fabd3e00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabdc0b4aa0 0x7fabdc159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.949+0000 7fabe0c00640 1 -- 172.21.15.7:0/937743434 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabdc0b0da0 msgr2=0x7fabdc1585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.949+0000 7fabe0c00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabdc0b0da0 0x7fabdc1585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.949+0000 7fabe0c00640 1 -- 172.21.15.7:0/937743434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabdc0b4aa0 msgr2=0x7fabdc159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.949+0000 7fabe0c00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabdc0b4aa0 0x7fabdc159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.949+0000 7fabe0c00640 1 -- 172.21.15.7:0/937743434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc15ad70 con 0x7fabdc164710 2024-04-25T11:14:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.949+0000 7fabe0c00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabdc164710 0x7fabdc15a530 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fabd400c760 tx=0x7fabd400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.949+0000 7fabd8c00640 1 -- 172.21.15.7:0/937743434 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd4003b70 con 0x7fabdc164710 2024-04-25T11:14:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.949+0000 7fabd8c00640 1 -- 172.21.15.7:0/937743434 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabd4003d10 con 0x7fabdc164710 2024-04-25T11:14:00.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.949+0000 7fabd8c00640 1 -- 172.21.15.7:0/937743434 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabd400aa90 con 0x7fabdc164710 2024-04-25T11:14:00.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.949+0000 7fabe2c00640 1 -- 172.21.15.7:0/937743434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabdc15b050 con 0x7fabdc164710 2024-04-25T11:14:00.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.950+0000 7fabe2c00640 1 -- 172.21.15.7:0/937743434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabdc15b610 con 0x7fabdc164710 2024-04-25T11:14:00.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.951+0000 7fabe2c00640 1 -- 172.21.15.7:0/937743434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabdc15aae0 con 0x7fabdc164710 2024-04-25T11:14:00.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.952+0000 7fabd8c00640 1 -- 172.21.15.7:0/937743434 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fabd400ac30 con 0x7fabdc164710 2024-04-25T11:14:00.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.953+0000 7fabd8c00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fabb80744b0 0x7fabb80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:00.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.953+0000 7fabd8c00640 1 -- 172.21.15.7:0/937743434 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fabd40951c0 con 0x7fabdc164710 2024-04-25T11:14:00.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.953+0000 7fabdbe00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fabb80744b0 0x7fabb80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:00.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.954+0000 7fabdbe00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fabb80744b0 0x7fabb80769c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fabcc006fd0 tx=0x7fabcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:00.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:00.958+0000 7fabd8c00640 1 -- 172.21.15.7:0/937743434 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fabd405f720 con 0x7fabdc164710 2024-04-25T11:14:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.164+0000 7fabe2c00640 1 -- 172.21.15.7:0/937743434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabdc06a100 con 0x7fabdc164710 2024-04-25T11:14:01.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.167+0000 7fabd8c00640 1 -- 172.21.15.7:0/937743434 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16846 (secure 0 0 0) 0x7fabd405f020 con 0x7fabdc164710 2024-04-25T11:14:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:01.169 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:13:43.726826+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:13:40.446828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T11:13:41.263101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.170+0000 7fabe2c00640 1 -- 172.21.15.7:0/937743434 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fabb80744b0 msgr2=0x7fabb80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.170+0000 7fabe2c00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fabb80744b0 0x7fabb80769c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fabcc006fd0 tx=0x7fabcc008040 comp rx=0 tx=0).stop 2024-04-25T11:14:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.170+0000 7fabe2c00640 1 -- 172.21.15.7:0/937743434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabdc164710 msgr2=0x7fabdc15a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.170+0000 7fabe2c00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabdc164710 0x7fabdc15a530 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fabd400c760 tx=0x7fabd400cc20 comp rx=0 tx=0).stop 2024-04-25T11:14:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.170+0000 7fabe2c00640 1 -- 172.21.15.7:0/937743434 shutdown_connections 2024-04-25T11:14:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.170+0000 7fabe2c00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fabb80744b0 0x7fabb80769c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.170+0000 7fabe2c00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabdc164710 0x7fabdc15a530 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.170+0000 7fabe2c00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabdc0b4aa0 0x7fabdc159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.170+0000 7fabe2c00640 1 --2- 172.21.15.7:0/937743434 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabdc0b0da0 0x7fabdc1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.170+0000 7fabe2c00640 1 -- 172.21.15.7:0/937743434 >> 172.21.15.7:0/937743434 conn(0x7fabdc074200 msgr2=0x7fabdc162e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:01.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.171+0000 7fabe2c00640 1 -- 172.21.15.7:0/937743434 shutdown_connections 2024-04-25T11:14:01.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.171+0000 7fabe2c00640 1 -- 172.21.15.7:0/937743434 wait complete. 2024-04-25T11:14:01.200 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:13:40.446828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:13:41.263101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:14:01.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:14:01.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.479+0000 7f2643200640 1 Processor -- start 2024-04-25T11:14:01.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.479+0000 7f2643200640 1 -- start start 2024-04-25T11:14:01.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.481+0000 7f2643200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26441583c0 0x7f26440cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:01.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.481+0000 7f2643200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26440cfb50 0x7f26440d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:01.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.481+0000 7f2643200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f26440d09c0 0x7f264406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:01.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.481+0000 7f2643200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f264415f190 con 0x7f26441583c0 2024-04-25T11:14:01.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.481+0000 7f2643200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f26440ce570 con 0x7f26440d09c0 2024-04-25T11:14:01.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.481+0000 7f2643200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f26440ce850 con 0x7f26440cfb50 2024-04-25T11:14:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.482+0000 7f2641e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26441583c0 0x7f26440cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.482+0000 7f2641e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26441583c0 0x7f26440cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37832/0 (socket says 172.21.15.7:37832) 2024-04-25T11:14:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.482+0000 7f2641e00640 1 -- 172.21.15.7:0/653720864 learned_addr learned my addr 172.21.15.7:0/653720864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.482+0000 7f2641400640 1 --2- 172.21.15.7:0/653720864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26440cfb50 0x7f26440d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.482+0000 7f2642800640 1 --2- 172.21.15.7:0/653720864 >> v2:172.21.15.87:3300/0 conn(0x7f26440d09c0 0x7f264406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.483+0000 7f2641e00640 1 -- 172.21.15.7:0/653720864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26440cfb50 msgr2=0x7f26440d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.483+0000 7f2641e00640 1 --2- 172.21.15.7:0/653720864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26440cfb50 0x7f26440d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.483+0000 7f2641e00640 1 -- 172.21.15.7:0/653720864 >> v2:172.21.15.87:3300/0 conn(0x7f26440d09c0 msgr2=0x7f264406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.483+0000 7f2641e00640 1 --2- 172.21.15.7:0/653720864 >> v2:172.21.15.87:3300/0 conn(0x7f26440d09c0 0x7f264406eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.483+0000 7f2641e00640 1 -- 172.21.15.7:0/653720864 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26440ceb30 con 0x7f26441583c0 2024-04-25T11:14:01.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.483+0000 7f2642800640 1 --2- 172.21.15.7:0/653720864 >> v2:172.21.15.87:3300/0 conn(0x7f26440d09c0 0x7f264406eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:14:01.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.484+0000 7f2641e00640 1 --2- 172.21.15.7:0/653720864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26441583c0 0x7f26440cf610 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f26440cdfa0 tx=0x7f262c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6c2f911fa0b582 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:01.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.484+0000 7f2640a00640 1 -- 172.21.15.7:0/653720864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c037070 con 0x7f26441583c0 2024-04-25T11:14:01.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.484+0000 7f2640a00640 1 -- 172.21.15.7:0/653720864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262c032440 con 0x7f26441583c0 2024-04-25T11:14:01.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.484+0000 7f2640a00640 1 -- 172.21.15.7:0/653720864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c032810 con 0x7f26441583c0 2024-04-25T11:14:01.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.485+0000 7f2643200640 1 -- 172.21.15.7:0/653720864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26441583c0 msgr2=0x7f26440cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:01.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.485+0000 7f2643200640 1 --2- 172.21.15.7:0/653720864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26441583c0 0x7f26440cf610 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f26440cdfa0 tx=0x7f262c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.485+0000 7f2643200640 1 -- 172.21.15.7:0/653720864 shutdown_connections 2024-04-25T11:14:01.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.485+0000 7f2643200640 1 --2- 172.21.15.7:0/653720864 >> v2:172.21.15.87:3300/0 conn(0x7f26440d09c0 0x7f264406eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.485+0000 7f2643200640 1 --2- 172.21.15.7:0/653720864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26440cfb50 0x7f26440d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.485+0000 7f2643200640 1 --2- 172.21.15.7:0/653720864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26441583c0 0x7f26440cf610 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.485+0000 7f2643200640 1 -- 172.21.15.7:0/653720864 >> 172.21.15.7:0/653720864 conn(0x7f26440740f0 msgr2=0x7f264406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:01.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.486+0000 7f2643200640 1 -- 172.21.15.7:0/653720864 shutdown_connections 2024-04-25T11:14:01.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.486+0000 7f2643200640 1 -- 172.21.15.7:0/653720864 wait complete. 2024-04-25T11:14:01.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.488+0000 7f2643200640 1 Processor -- start 2024-04-25T11:14:01.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.488+0000 7f2643200640 1 -- start start 2024-04-25T11:14:01.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.489+0000 7f2643200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26440cfb50 0x7f264413feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:01.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.489+0000 7f2643200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26440d09c0 0x7f2644160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:01.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.489+0000 7f2643200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26441583c0 0x7f2644160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:01.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.489+0000 7f2643200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f264406f3e0 con 0x7f26440d09c0 2024-04-25T11:14:01.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.489+0000 7f2643200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f26440cf0f0 con 0x7f26440cfb50 2024-04-25T11:14:01.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.489+0000 7f2643200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f264406f560 con 0x7f26441583c0 2024-04-25T11:14:01.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.490+0000 7f2641400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26440d09c0 0x7f2644160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:01.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.490+0000 7f2641400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26440d09c0 0x7f2644160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37846/0 (socket says 172.21.15.7:37846) 2024-04-25T11:14:01.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.490+0000 7f2641400640 1 -- 172.21.15.7:0/623433598 learned_addr learned my addr 172.21.15.7:0/623433598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:01.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.490+0000 7f2641e00640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26440cfb50 0x7f264413feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:01.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.490+0000 7f2641400640 1 -- 172.21.15.7:0/623433598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26441583c0 msgr2=0x7f2644160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:01.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.490+0000 7f2641400640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26441583c0 0x7f2644160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.490+0000 7f2641400640 1 -- 172.21.15.7:0/623433598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26440cfb50 msgr2=0x7f264413feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:01.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.490+0000 7f2641400640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26440cfb50 0x7f264413feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.490+0000 7f2641400640 1 -- 172.21.15.7:0/623433598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264414d3d0 con 0x7f26440d09c0 2024-04-25T11:14:01.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.490+0000 7f2641400640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26440d09c0 0x7f2644160600 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f263400e840 tx=0x7f263400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.491+0000 7f263a000640 1 -- 172.21.15.7:0/623433598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634017070 con 0x7f26440d09c0 2024-04-25T11:14:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.491+0000 7f263a000640 1 -- 172.21.15.7:0/623433598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263400c730 con 0x7f26440d09c0 2024-04-25T11:14:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.491+0000 7f263a000640 1 -- 172.21.15.7:0/623433598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263400cb20 con 0x7f26440d09c0 2024-04-25T11:14:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.491+0000 7f2643200640 1 -- 172.21.15.7:0/623433598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264414d6b0 con 0x7f26440d09c0 2024-04-25T11:14:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.491+0000 7f2643200640 1 -- 172.21.15.7:0/623433598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f264414dc70 con 0x7f26440d09c0 2024-04-25T11:14:01.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.492+0000 7f2643200640 1 -- 172.21.15.7:0/623433598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2644050740 con 0x7f26440d09c0 2024-04-25T11:14:01.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.493+0000 7f263a000640 1 -- 172.21.15.7:0/623433598 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f263401a070 con 0x7f26440d09c0 2024-04-25T11:14:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.494+0000 7f263a000640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26200744b0 0x7f26200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.494+0000 7f263a000640 1 -- 172.21.15.7:0/623433598 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f26340955d0 con 0x7f26440d09c0 2024-04-25T11:14:01.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.494+0000 7f2641e00640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26200744b0 0x7f26200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:01.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.495+0000 7f2641e00640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26200744b0 0x7f26200769c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f262c034000 tx=0x7f262c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.499+0000 7f263a000640 1 -- 172.21.15.7:0/623433598 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f263405fb30 con 0x7f26440d09c0 2024-04-25T11:14:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.697+0000 7f2643200640 1 -- 172.21.15.7:0/623433598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f26441610e0 con 0x7f26440d09c0 2024-04-25T11:14:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.698+0000 7f263a000640 1 -- 172.21.15.7:0/623433598 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f263405f430 con 0x7f26440d09c0 2024-04-25T11:14:01.699 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:14:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.700+0000 7f2643200640 1 -- 172.21.15.7:0/623433598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26200744b0 msgr2=0x7f26200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.700+0000 7f2643200640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26200744b0 0x7f26200769c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f262c034000 tx=0x7f262c02e000 comp rx=0 tx=0).stop 2024-04-25T11:14:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.700+0000 7f2643200640 1 -- 172.21.15.7:0/623433598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26440d09c0 msgr2=0x7f2644160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.700+0000 7f2643200640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26440d09c0 0x7f2644160600 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f263400e840 tx=0x7f263400ed00 comp rx=0 tx=0).stop 2024-04-25T11:14:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.701+0000 7f2643200640 1 -- 172.21.15.7:0/623433598 shutdown_connections 2024-04-25T11:14:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.701+0000 7f2643200640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26200744b0 0x7f26200769c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.701+0000 7f2643200640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26441583c0 0x7f2644160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.701+0000 7f2643200640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26440d09c0 0x7f2644160600 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.701+0000 7f2643200640 1 --2- 172.21.15.7:0/623433598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26440cfb50 0x7f264413feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.701+0000 7f2643200640 1 -- 172.21.15.7:0/623433598 >> 172.21.15.7:0/623433598 conn(0x7f26440740f0 msgr2=0x7f264406d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:01.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.701+0000 7f2643200640 1 -- 172.21.15.7:0/623433598 shutdown_connections 2024-04-25T11:14:01.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:01.701+0000 7f2643200640 1 -- 172.21.15.7:0/623433598 wait complete. 2024-04-25T11:14:01.727 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:14:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.005+0000 7f648b400640 1 Processor -- start 2024-04-25T11:14:02.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.005+0000 7f648b400640 1 -- start start 2024-04-25T11:14:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f648b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f648415aaa0 0x7f64840affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f648b400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f648406f5b0 con 0x7f64840d0ee0 2024-04-25T11:14:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f648b400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64840d14a0 con 0x7f64840b5020 2024-04-25T11:14:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f648b400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f64840af290 con 0x7f648415aaa0 2024-04-25T11:14:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f64840d0ee0 0x7f64840b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41196/0 (socket says 172.21.15.7:41196) 2024-04-25T11:14:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483e00640 1 -- 172.21.15.7:0/3867470459 learned_addr learned my addr 172.21.15.7:0/3867470459 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6488a00640 1 --2- 172.21.15.7:0/3867470459 >> v2:172.21.15.87:3300/0 conn(0x7f648415aaa0 0x7f64840affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122313751 0 0) 0x7f648406f5b0 con 0x7f64840d0ee0 2024-04-25T11:14:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f646c003520 con 0x7f64840d0ee0 2024-04-25T11:14:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1018994118 0 0) 0x7f646c003520 con 0x7f64840d0ee0 2024-04-25T11:14:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 >> v2:172.21.15.87:3300/0 conn(0x7f648415aaa0 msgr2=0x7f64840affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 --2- 172.21.15.7:0/3867470459 >> v2:172.21.15.87:3300/0 conn(0x7f648415aaa0 0x7f64840affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 >> v1:172.21.15.87:6789/0 conn(0x7f64840b5020 legacy=0x7f6484160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64840af4e0 con 0x7f64840d0ee0 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f6470001670 con 0x7f64840d0ee0 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 >> v1:172.21.15.7:6789/0 conn(0x7f64840d0ee0 legacy=0x7f64840b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 0x7f646c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f646c004800 0x7f646c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f646c0051c0 0x7f646c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.009+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f64840af290 con 0x7f646c0039b0 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.010+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f646c00e370 con 0x7f646c0051c0 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.010+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f646c00e570 con 0x7f646c004800 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.010+0000 7f6488a00640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 0x7f646c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.010+0000 7f6483e00640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f646c004800 0x7f646c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.010+0000 7f6488a00640 1 -- 172.21.15.7:0/3867470459 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f646c004800 msgr2=0x7f646c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:02.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.010+0000 7f6488a00640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f646c004800 0x7f646c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.010+0000 7f6489400640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f646c0051c0 0x7f646c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.010+0000 7f6488a00640 1 -- 172.21.15.7:0/3867470459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f646c0051c0 msgr2=0x7f646c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:02.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.011+0000 7f6488a00640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f646c0051c0 0x7f646c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.011+0000 7f6488a00640 1 -- 172.21.15.7:0/3867470459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c003520 con 0x7f646c0039b0 2024-04-25T11:14:02.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.011+0000 7f6489400640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f646c0051c0 0x7f646c00dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:02.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.011+0000 7f6489400640 1 -- 172.21.15.7:0/3867470459 reap_dead start 2024-04-25T11:14:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.013+0000 7f6488a00640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 0x7f646c003e20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6478009ac0 tx=0x7f646c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4606a4782416411d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:02.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.013+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478036070 con 0x7f646c0039b0 2024-04-25T11:14:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.013+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6478031400 con 0x7f646c0039b0 2024-04-25T11:14:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.013+0000 7f6483400640 1 -- 172.21.15.7:0/3867470459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478009690 con 0x7f646c0039b0 2024-04-25T11:14:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.013+0000 7f648b400640 1 -- 172.21.15.7:0/3867470459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 msgr2=0x7f646c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.013+0000 7f648b400640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 0x7f646c003e20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6478009ac0 tx=0x7f646c005830 comp rx=0 tx=0).stop 2024-04-25T11:14:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.013+0000 7f648b400640 1 -- 172.21.15.7:0/3867470459 shutdown_connections 2024-04-25T11:14:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.013+0000 7f648b400640 1 --2- 172.21.15.7:0/3867470459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 0x7f646c003e20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.013+0000 7f648b400640 1 -- 172.21.15.7:0/3867470459 >> 172.21.15.7:0/3867470459 conn(0x7f64840742c0 msgr2=0x7f648406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.013+0000 7f648b400640 1 -- 172.21.15.7:0/3867470459 shutdown_connections 2024-04-25T11:14:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.013+0000 7f648b400640 1 -- 172.21.15.7:0/3867470459 wait complete. 2024-04-25T11:14:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.016+0000 7f648b400640 1 Processor -- start 2024-04-25T11:14:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.016+0000 7f648b400640 1 -- start start 2024-04-25T11:14:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f648b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 0x7f648414a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f648b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64841e5220 0x7f64841e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f648b400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64841f5910 0x7f64841f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f648b400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f64840b5430 con 0x7f646c0039b0 2024-04-25T11:14:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f648b400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f64840afaa0 con 0x7f64841e5220 2024-04-25T11:14:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f648b400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f64840b55b0 con 0x7f64841f5910 2024-04-25T11:14:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f6488a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 0x7f648414a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f6488a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 0x7f648414a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37862/0 (socket says 172.21.15.7:37862) 2024-04-25T11:14:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f6488a00640 1 -- 172.21.15.7:0/2497057250 learned_addr learned my addr 172.21.15.7:0/2497057250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:02.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f6489400640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64841f5910 0x7f64841f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f6483e00640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64841e5220 0x7f64841e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f6488a00640 1 -- 172.21.15.7:0/2497057250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64841f5910 msgr2=0x7f64841f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f6488a00640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64841f5910 0x7f64841f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f6488a00640 1 -- 172.21.15.7:0/2497057250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64841e5220 msgr2=0x7f64841e6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f6488a00640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64841e5220 0x7f64841e6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f6488a00640 1 -- 172.21.15.7:0/2497057250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64841e5630 con 0x7f646c0039b0 2024-04-25T11:14:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.018+0000 7f6483e00640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64841e5220 0x7f64841e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.019+0000 7f6488a00640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 0x7f648414a4c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6478034040 tx=0x7f6478004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:02.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.019+0000 7f6480c00640 1 -- 172.21.15.7:0/2497057250 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647803f020 con 0x7f646c0039b0 2024-04-25T11:14:02.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.019+0000 7f6480c00640 1 -- 172.21.15.7:0/2497057250 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6478037040 con 0x7f646c0039b0 2024-04-25T11:14:02.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.019+0000 7f6480c00640 1 -- 172.21.15.7:0/2497057250 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478036070 con 0x7f646c0039b0 2024-04-25T11:14:02.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.019+0000 7f648b400640 1 -- 172.21.15.7:0/2497057250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64841e7040 con 0x7f646c0039b0 2024-04-25T11:14:02.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.019+0000 7f648b400640 1 -- 172.21.15.7:0/2497057250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648415a140 con 0x7f646c0039b0 2024-04-25T11:14:02.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.021+0000 7f648b400640 1 -- 172.21.15.7:0/2497057250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6484050740 con 0x7f646c0039b0 2024-04-25T11:14:02.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.022+0000 7f6480c00640 1 -- 172.21.15.7:0/2497057250 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6478042020 con 0x7f646c0039b0 2024-04-25T11:14:02.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.024+0000 7f6480c00640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64640744b0 0x7f64640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.024+0000 7f6480c00640 1 -- 172.21.15.7:0/2497057250 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f64780b55f0 con 0x7f646c0039b0 2024-04-25T11:14:02.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.024+0000 7f6483e00640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64640744b0 0x7f64640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.024+0000 7f6483e00640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64640744b0 0x7f64640769c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6470001200 tx=0x7f6470001150 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.028+0000 7f6480c00640 1 -- 172.21.15.7:0/2497057250 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6478049030 con 0x7f646c0039b0 2024-04-25T11:14:02.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.239+0000 7f648b400640 1 -- 172.21.15.7:0/2497057250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64841e7270 con 0x7f646c0039b0 2024-04-25T11:14:02.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.240+0000 7f6480c00640 1 -- 172.21.15.7:0/2497057250 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) ==== 104+0+11 (secure 0 0 0) 0x7f647807fc10 con 0x7f646c0039b0 2024-04-25T11:14:02.241 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:14:02.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 -- 172.21.15.7:0/2497057250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64640744b0 msgr2=0x7f64640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64640744b0 0x7f64640769c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6470001200 tx=0x7f6470001150 comp rx=0 tx=0).stop 2024-04-25T11:14:02.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 -- 172.21.15.7:0/2497057250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 msgr2=0x7f648414a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 0x7f648414a4c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6478034040 tx=0x7f6478004730 comp rx=0 tx=0).stop 2024-04-25T11:14:02.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 -- 172.21.15.7:0/2497057250 shutdown_connections 2024-04-25T11:14:02.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64640744b0 0x7f64640769c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64841f5910 0x7f64841f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64841e5220 0x7f64841e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 --2- 172.21.15.7:0/2497057250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646c0039b0 0x7f648414a4c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 -- 172.21.15.7:0/2497057250 >> 172.21.15.7:0/2497057250 conn(0x7f64840742c0 msgr2=0x7f648415bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:02.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 -- 172.21.15.7:0/2497057250 shutdown_connections 2024-04-25T11:14:02.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.243+0000 7f648b400640 1 -- 172.21.15.7:0/2497057250 wait complete. 2024-04-25T11:14:02.274 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:14:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.557+0000 7fa1f6c00640 1 Processor -- start 2024-04-25T11:14:02.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.557+0000 7fa1f6c00640 1 -- start start 2024-04-25T11:14:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.559+0000 7fa1f6c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa1f00b5580 0x7fa1f00cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.559+0000 7fa1f6c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1f00cf560 0x7fa1f0144220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1f6c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f00b5990 con 0x7fa1f00b4690 2024-04-25T11:14:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1f6c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f00cf970 con 0x7fa1f00cf560 2024-04-25T11:14:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1f6c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f00b5c40 con 0x7fa1f00b5580 2024-04-25T11:14:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1f6200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1f00cf560 0x7fa1f0144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1f5800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa1f00b4690 0x7fa1f0143b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41206/0 (socket says 172.21.15.7:41206) 2024-04-25T11:14:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1f5800640 1 -- 172.21.15.7:0/1542196422 learned_addr learned my addr 172.21.15.7:0/1542196422 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1f4e00640 1 --2- 172.21.15.7:0/1542196422 >> v2:172.21.15.87:3300/0 conn(0x7fa1f00b5580 0x7fa1f00cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1f6200640 1 -- 172.21.15.7:0/1542196422 >> v2:172.21.15.87:3300/0 conn(0x7fa1f00b5580 msgr2=0x7fa1f00cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1f6200640 1 --2- 172.21.15.7:0/1542196422 >> v2:172.21.15.87:3300/0 conn(0x7fa1f00b5580 0x7fa1f00cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1f6200640 1 -- 172.21.15.7:0/1542196422 >> v1:172.21.15.7:6789/0 conn(0x7fa1f00b4690 legacy=0x7fa1f0143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1f6200640 1 -- 172.21.15.7:0/1542196422 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00b5f20 con 0x7fa1f00cf560 2024-04-25T11:14:02.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.560+0000 7fa1efe00640 1 -- 172.21.15.7:0/1542196422 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3459931902 0 0) 0x7fa1f00b5990 con 0x7fa1f00b4690 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.562+0000 7fa1f6200640 1 --2- 172.21.15.7:0/1542196422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00cf560 0x7fa1f0144220 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa1f006e9a0 tx=0x7fa1e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8868939cf1f754c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.562+0000 7fa1efe00640 1 -- 172.21.15.7:0/1542196422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8037070 con 0x7fa1f00cf560 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.562+0000 7fa1efe00640 1 -- 172.21.15.7:0/1542196422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e8032440 con 0x7fa1f00cf560 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.562+0000 7fa1efe00640 1 -- 172.21.15.7:0/1542196422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8032810 con 0x7fa1f00cf560 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.564+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1542196422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00cf560 msgr2=0x7fa1f0144220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.564+0000 7fa1f6c00640 1 --2- 172.21.15.7:0/1542196422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00cf560 0x7fa1f0144220 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa1f006e9a0 tx=0x7fa1e802a230 comp rx=0 tx=0).stop 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.564+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1542196422 shutdown_connections 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.564+0000 7fa1f6c00640 1 --2- 172.21.15.7:0/1542196422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00cf560 0x7fa1f0144220 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.564+0000 7fa1f6c00640 1 --2- 172.21.15.7:0/1542196422 >> v2:172.21.15.87:3300/0 conn(0x7fa1f00b5580 0x7fa1f00cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.564+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1542196422 >> 172.21.15.7:0/1542196422 conn(0x7fa1f0074140 msgr2=0x7fa1f006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.564+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1542196422 shutdown_connections 2024-04-25T11:14:02.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.564+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1542196422 wait complete. 2024-04-25T11:14:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.566+0000 7fa1f6c00640 1 Processor -- start 2024-04-25T11:14:02.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.566+0000 7fa1f6c00640 1 -- start start 2024-04-25T11:14:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1f00b4690 0x7fa1f00cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1f00b5580 0x7fa1f00b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00cf560 0x7fa1f00caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00b67c0 con 0x7fa1f00cf560 2024-04-25T11:14:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00b64e0 con 0x7fa1f00b4690 2024-04-25T11:14:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f0070fa0 con 0x7fa1f00b5580 2024-04-25T11:14:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00cf560 0x7fa1f00caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00cf560 0x7fa1f00caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37882/0 (socket says 172.21.15.7:37882) 2024-04-25T11:14:02.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6200640 1 -- 172.21.15.7:0/1717354478 learned_addr learned my addr 172.21.15.7:0/1717354478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f5800640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1f00b4690 0x7fa1f00cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f4e00640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1f00b5580 0x7fa1f00b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6200640 1 -- 172.21.15.7:0/1717354478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1f00b5580 msgr2=0x7fa1f00b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6200640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1f00b5580 0x7fa1f00b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6200640 1 -- 172.21.15.7:0/1717354478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1f00b4690 msgr2=0x7fa1f00cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6200640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1f00b4690 0x7fa1f00cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6200640 1 -- 172.21.15.7:0/1717354478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00ccf60 con 0x7fa1f00cf560 2024-04-25T11:14:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.568+0000 7fa1f6200640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00cf560 0x7fa1f00caa10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa1e802a270 tx=0x7fa1e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.569+0000 7fa1ed600640 1 -- 172.21.15.7:0/1717354478 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8032e60 con 0x7fa1f00cf560 2024-04-25T11:14:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.569+0000 7fa1ed600640 1 -- 172.21.15.7:0/1717354478 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e8042070 con 0x7fa1f00cf560 2024-04-25T11:14:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.569+0000 7fa1ed600640 1 -- 172.21.15.7:0/1717354478 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8037040 con 0x7fa1f00cf560 2024-04-25T11:14:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.569+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1717354478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f00cda30 con 0x7fa1f00cf560 2024-04-25T11:14:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.569+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1717354478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f00cd750 con 0x7fa1f00cf560 2024-04-25T11:14:02.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.570+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1717354478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f00cd1e0 con 0x7fa1f00cf560 2024-04-25T11:14:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.571+0000 7fa1ed600640 1 -- 172.21.15.7:0/1717354478 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa1e8030b50 con 0x7fa1f00cf560 2024-04-25T11:14:02.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.572+0000 7fa1ed600640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1d00744e0 0x7fa1d00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:02.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.573+0000 7fa1ed600640 1 -- 172.21.15.7:0/1717354478 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fa1e80b5f40 con 0x7fa1f00cf560 2024-04-25T11:14:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.573+0000 7fa1f5800640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1d00744e0 0x7fa1d00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:02.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.574+0000 7fa1f5800640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1d00744e0 0x7fa1d00769f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4007ed0 tx=0x7fa1e4007e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:02.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.578+0000 7fa1ed600640 1 -- 172.21.15.7:0/1717354478 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa1e80804a0 con 0x7fa1f00cf560 2024-04-25T11:14:02.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.787+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1717354478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1f015cd60 con 0x7fa1f00cf560 2024-04-25T11:14:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.789+0000 7fa1ed600640 1 -- 172.21.15.7:0/1717354478 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) ==== 100+0+11 (secure 0 0 0) 0x7fa1e807fda0 con 0x7fa1f00cf560 2024-04-25T11:14:02.789 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:14:02.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1717354478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1d00744e0 msgr2=0x7fa1d00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1d00744e0 0x7fa1d00769f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4007ed0 tx=0x7fa1e4007e60 comp rx=0 tx=0).stop 2024-04-25T11:14:02.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1717354478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00cf560 msgr2=0x7fa1f00caa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:02.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00cf560 0x7fa1f00caa10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa1e802a270 tx=0x7fa1e80046c0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1717354478 shutdown_connections 2024-04-25T11:14:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1d00744e0 0x7fa1d00769f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1f00cf560 0x7fa1f00caa10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1f00b5580 0x7fa1f00b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 --2- 172.21.15.7:0/1717354478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1f00b4690 0x7fa1f00cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1717354478 >> 172.21.15.7:0/1717354478 conn(0x7fa1f0074140 msgr2=0x7fa1f00cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1717354478 shutdown_connections 2024-04-25T11:14:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:02.792+0000 7fa1f6c00640 1 -- 172.21.15.7:0/1717354478 wait complete. 2024-04-25T11:14:02.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.105+0000 7fe16ca00640 1 Processor -- start 2024-04-25T11:14:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.105+0000 7fe16ca00640 1 -- start start 2024-04-25T11:14:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.107+0000 7fe16ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe168070a70 0x7fe1680ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe16ca00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1680b0be0 0x7fe1680b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe16ca00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe16815e190 con 0x7fe1680aedc0 2024-04-25T11:14:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe16ca00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe16815d610 con 0x7fe168070a70 2024-04-25T11:14:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe16ca00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe16815d8f0 con 0x7fe1680b0be0 2024-04-25T11:14:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe166800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe168070a70 0x7fe1680ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe165e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe1680aedc0 0x7fe1680b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41212/0 (socket says 172.21.15.7:41212) 2024-04-25T11:14:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe165e00640 1 -- 172.21.15.7:0/1663431728 learned_addr learned my addr 172.21.15.7:0/1663431728 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe166800640 1 -- 172.21.15.7:0/1663431728 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1680b0be0 msgr2=0x7fe1680b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe167200640 1 --2- 172.21.15.7:0/1663431728 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1680b0be0 0x7fe1680b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe166800640 1 --2- 172.21.15.7:0/1663431728 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1680b0be0 0x7fe1680b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe165400640 1 -- 172.21.15.7:0/1663431728 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4220561402 0 0) 0x7fe16815e190 con 0x7fe1680aedc0 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe166800640 1 -- 172.21.15.7:0/1663431728 >> v1:172.21.15.7:6789/0 conn(0x7fe1680aedc0 legacy=0x7fe1680b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.108+0000 7fe166800640 1 -- 172.21.15.7:0/1663431728 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16815dbd0 con 0x7fe168070a70 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.110+0000 7fe166800640 1 --2- 172.21.15.7:0/1663431728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe168070a70 0x7fe1680ae880 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe158009d90 tx=0x7fe15802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af17c3cfc62d693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.110+0000 7fe165400640 1 -- 172.21.15.7:0/1663431728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158007740 con 0x7fe168070a70 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.110+0000 7fe165400640 1 -- 172.21.15.7:0/1663431728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1580078e0 con 0x7fe168070a70 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.110+0000 7fe165400640 1 -- 172.21.15.7:0/1663431728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158007cb0 con 0x7fe168070a70 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.111+0000 7fe16ca00640 1 -- 172.21.15.7:0/1663431728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe168070a70 msgr2=0x7fe1680ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.111+0000 7fe16ca00640 1 --2- 172.21.15.7:0/1663431728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe168070a70 0x7fe1680ae880 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe158009d90 tx=0x7fe15802a660 comp rx=0 tx=0).stop 2024-04-25T11:14:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.111+0000 7fe16ca00640 1 -- 172.21.15.7:0/1663431728 shutdown_connections 2024-04-25T11:14:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.111+0000 7fe16ca00640 1 --2- 172.21.15.7:0/1663431728 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1680b0be0 0x7fe1680b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.111+0000 7fe16ca00640 1 --2- 172.21.15.7:0/1663431728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe168070a70 0x7fe1680ae880 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.111+0000 7fe16ca00640 1 -- 172.21.15.7:0/1663431728 >> 172.21.15.7:0/1663431728 conn(0x7fe1680b3820 msgr2=0x7fe1680b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.111+0000 7fe16ca00640 1 -- 172.21.15.7:0/1663431728 shutdown_connections 2024-04-25T11:14:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.111+0000 7fe16ca00640 1 -- 172.21.15.7:0/1663431728 wait complete. 2024-04-25T11:14:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.113+0000 7fe16ca00640 1 Processor -- start 2024-04-25T11:14:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.113+0000 7fe16ca00640 1 -- start start 2024-04-25T11:14:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.115+0000 7fe16ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168070a70 0x7fe16809c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.115+0000 7fe16ca00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1680aedc0 0x7fe16815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.115+0000 7fe16ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1680b0be0 0x7fe1680b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.115+0000 7fe16ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe16815d2f0 con 0x7fe1680b0be0 2024-04-25T11:14:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.115+0000 7fe16ca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe16815d170 con 0x7fe168070a70 2024-04-25T11:14:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.115+0000 7fe16ca00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe1680b4fc0 con 0x7fe1680aedc0 2024-04-25T11:14:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.115+0000 7fe167200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1680b0be0 0x7fe1680b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.115+0000 7fe167200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1680b0be0 0x7fe1680b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37896/0 (socket says 172.21.15.7:37896) 2024-04-25T11:14:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.115+0000 7fe167200640 1 -- 172.21.15.7:0/2638887295 learned_addr learned my addr 172.21.15.7:0/2638887295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.116+0000 7fe166800640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168070a70 0x7fe16809c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.116+0000 7fe165e00640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1680aedc0 0x7fe16815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.116+0000 7fe167200640 1 -- 172.21.15.7:0/2638887295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1680aedc0 msgr2=0x7fe16815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.116+0000 7fe167200640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1680aedc0 0x7fe16815eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.116+0000 7fe167200640 1 -- 172.21.15.7:0/2638887295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168070a70 msgr2=0x7fe16809c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.116+0000 7fe167200640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168070a70 0x7fe16809c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.116+0000 7fe167200640 1 -- 172.21.15.7:0/2638887295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16809d7e0 con 0x7fe1680b0be0 2024-04-25T11:14:03.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.116+0000 7fe167200640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1680b0be0 0x7fe1680b2000 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe16000c5c0 tx=0x7fe16000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:03.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.117+0000 7fe15ea00640 1 -- 172.21.15.7:0/2638887295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160015070 con 0x7fe1680b0be0 2024-04-25T11:14:03.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.117+0000 7fe15ea00640 1 -- 172.21.15.7:0/2638887295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe160004220 con 0x7fe1680b0be0 2024-04-25T11:14:03.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.117+0000 7fe16ca00640 1 -- 172.21.15.7:0/2638887295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16815f430 con 0x7fe1680b0be0 2024-04-25T11:14:03.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.117+0000 7fe15ea00640 1 -- 172.21.15.7:0/2638887295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160003610 con 0x7fe1680b0be0 2024-04-25T11:14:03.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.117+0000 7fe16ca00640 1 -- 172.21.15.7:0/2638887295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16815fca0 con 0x7fe1680b0be0 2024-04-25T11:14:03.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.118+0000 7fe16ca00640 1 -- 172.21.15.7:0/2638887295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe168050740 con 0x7fe1680b0be0 2024-04-25T11:14:03.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.119+0000 7fe15ea00640 1 -- 172.21.15.7:0/2638887295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe1600037b0 con 0x7fe1680b0be0 2024-04-25T11:14:03.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.120+0000 7fe15ea00640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 0x7fe1480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.120+0000 7fe15ea00640 1 -- 172.21.15.7:0/2638887295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7fe160095150 con 0x7fe1680b0be0 2024-04-25T11:14:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.121+0000 7fe166800640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 0x7fe1480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:03.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.122+0000 7fe166800640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 0x7fe1480769c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe158034000 tx=0x7fe158033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:03.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.125+0000 7fe15ea00640 1 -- 172.21.15.7:0/2638887295 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe16005f7e0 con 0x7fe1680b0be0 2024-04-25T11:14:03.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.367+0000 7fe16ca00640 1 -- 172.21.15.7:0/2638887295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe16806fc50 con 0x7fe1680b0be0 2024-04-25T11:14:03.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.368+0000 7fe15ea00640 1 -- 172.21.15.7:0/2638887295 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5172 (secure 0 0 0) 0x7fe16005f0e0 con 0x7fe1680b0be0 2024-04-25T11:14:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:03.369 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2225639104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2225639104},{"type":"v1","addr":"172.21.15.87:6834","nonce":2225639104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2625900548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2625900548},{"type":"v1","addr":"172.21.15.7:6837","nonce":2625900548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/717352569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":717352569},{"type":"v1","addr":"172.21.15.87:6835","nonce":717352569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:13:42.310229+0000","modified":"2024-04-25T11:13:45.341610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6836/3503942462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3503942462},{"type":"v1","addr":"172.21.15.7:6836","nonce":3503942462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-25T11:14:03.369 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-25T11:14:03.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 -- 172.21.15.7:0/2638887295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 msgr2=0x7fe1480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 0x7fe1480769c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe158034000 tx=0x7fe158033040 comp rx=0 tx=0).stop 2024-04-25T11:14:03.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 -- 172.21.15.7:0/2638887295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1680b0be0 msgr2=0x7fe1680b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1680b0be0 0x7fe1680b2000 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe16000c5c0 tx=0x7fe16000ca80 comp rx=0 tx=0).stop 2024-04-25T11:14:03.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 -- 172.21.15.7:0/2638887295 shutdown_connections 2024-04-25T11:14:03.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 0x7fe1480769c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1680b0be0 0x7fe1680b2000 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1680aedc0 0x7fe16815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 --2- 172.21.15.7:0/2638887295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168070a70 0x7fe16809c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 -- 172.21.15.7:0/2638887295 >> 172.21.15.7:0/2638887295 conn(0x7fe1680b3820 msgr2=0x7fe1680b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:03.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 -- 172.21.15.7:0/2638887295 shutdown_connections 2024-04-25T11:14:03.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.371+0000 7fe16ca00640 1 -- 172.21.15.7:0/2638887295 wait complete. 2024-04-25T11:14:03.399 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:03.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.675+0000 7f7d5f200640 1 Processor -- start 2024-04-25T11:14:03.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.675+0000 7f7d5f200640 1 -- start start 2024-04-25T11:14:03.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.677+0000 7f7d5f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7d6006e950 0x7f7d600d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:03.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.677+0000 7f7d5f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d6006ed60 con 0x7f7d600b7f90 2024-04-25T11:14:03.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.677+0000 7f7d5f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d600d12d0 con 0x7f7d600d0810 2024-04-25T11:14:03.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.677+0000 7f7d5f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7d600d15b0 con 0x7f7d6006e950 2024-04-25T11:14:03.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.677+0000 7f7d5de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7d6006e950 0x7f7d600d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:03.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.677+0000 7f7d5de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7d6006e950 0x7f7d600d4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37898/0 (socket says 172.21.15.7:37898) 2024-04-25T11:14:03.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.677+0000 7f7d5e800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7d600b7f90 0x7f7d600b8ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41214/0 (socket says 172.21.15.7:41214) 2024-04-25T11:14:03.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.677+0000 7f7d5de00640 1 -- 172.21.15.7:0/1027541207 learned_addr learned my addr 172.21.15.7:0/1027541207 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:03.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.678+0000 7f7d5de00640 1 -- 172.21.15.7:0/1027541207 >> v1:172.21.15.87:6789/0 conn(0x7f7d600d0810 legacy=0x7f7d600b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:03.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.678+0000 7f7d5de00640 1 -- 172.21.15.7:0/1027541207 >> v1:172.21.15.7:6789/0 conn(0x7f7d600b7f90 legacy=0x7f7d600b8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.678+0000 7f7d5de00640 1 -- 172.21.15.7:0/1027541207 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6006de70 con 0x7f7d6006e950 2024-04-25T11:14:03.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.678+0000 7f7d5ca00640 1 -- 172.21.15.7:0/1027541207 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446829590 0 0) 0x7f7d6006ed60 con 0x7f7d600b7f90 2024-04-25T11:14:03.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.679+0000 7f7d5de00640 1 --2- 172.21.15.7:0/1027541207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6006e950 0x7f7d600d4c50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009870 tx=0x7f7d5002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb08763f4521fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:03.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.679+0000 7f7d5ca00640 1 -- 172.21.15.7:0/1027541207 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50037070 con 0x7f7d6006e950 2024-04-25T11:14:03.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.679+0000 7f7d5ca00640 1 -- 172.21.15.7:0/1027541207 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d50032440 con 0x7f7d6006e950 2024-04-25T11:14:03.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.679+0000 7f7d5ca00640 1 -- 172.21.15.7:0/1027541207 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50032810 con 0x7f7d6006e950 2024-04-25T11:14:03.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.680+0000 7f7d5f200640 1 -- 172.21.15.7:0/1027541207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6006e950 msgr2=0x7f7d600d4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.680+0000 7f7d5f200640 1 --2- 172.21.15.7:0/1027541207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6006e950 0x7f7d600d4c50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009870 tx=0x7f7d5002a2a0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.680+0000 7f7d5f200640 1 -- 172.21.15.7:0/1027541207 shutdown_connections 2024-04-25T11:14:03.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.680+0000 7f7d5f200640 1 --2- 172.21.15.7:0/1027541207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d6006e950 0x7f7d600d4c50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.680+0000 7f7d5f200640 1 -- 172.21.15.7:0/1027541207 >> 172.21.15.7:0/1027541207 conn(0x7f7d600d2840 msgr2=0x7f7d600d2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:03.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.680+0000 7f7d5f200640 1 -- 172.21.15.7:0/1027541207 shutdown_connections 2024-04-25T11:14:03.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.680+0000 7f7d5f200640 1 -- 172.21.15.7:0/1027541207 wait complete. 2024-04-25T11:14:03.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.683+0000 7f7d5f200640 1 Processor -- start 2024-04-25T11:14:03.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.683+0000 7f7d5f200640 1 -- start start 2024-04-25T11:14:03.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.684+0000 7f7d5f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6006e950 0x7f7d601e9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:03.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.684+0000 7f7d5f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d600b7f90 0x7f7d600ce640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.684+0000 7f7d5f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d600d0810 0x7f7d601ead40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.684+0000 7f7d5f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d600d5190 con 0x7f7d600d0810 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.684+0000 7f7d5f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6006e430 con 0x7f7d6006e950 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.684+0000 7f7d5f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7d600d5310 con 0x7f7d600b7f90 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.685+0000 7f7d5e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d600d0810 0x7f7d601ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.685+0000 7f7d5e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d600d0810 0x7f7d601ead40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37906/0 (socket says 172.21.15.7:37906) 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.685+0000 7f7d5e800640 1 -- 172.21.15.7:0/441165772 learned_addr learned my addr 172.21.15.7:0/441165772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.685+0000 7f7d5de00640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6006e950 0x7f7d601e9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.685+0000 7f7d5d400640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d600b7f90 0x7f7d600ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.685+0000 7f7d5e800640 1 -- 172.21.15.7:0/441165772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d600b7f90 msgr2=0x7f7d600ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.685+0000 7f7d5e800640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d600b7f90 0x7f7d600ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.685+0000 7f7d5e800640 1 -- 172.21.15.7:0/441165772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6006e950 msgr2=0x7f7d601e9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.685+0000 7f7d5e800640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6006e950 0x7f7d601e9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.685+0000 7f7d5e800640 1 -- 172.21.15.7:0/441165772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d601fa360 con 0x7f7d600d0810 2024-04-25T11:14:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.685+0000 7f7d5de00640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6006e950 0x7f7d601e9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:14:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.686+0000 7f7d5e800640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d600d0810 0x7f7d601ead40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c3e0 tx=0x7f7d5800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.686+0000 7f7d56000640 1 -- 172.21.15.7:0/441165772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d580074e0 con 0x7f7d600d0810 2024-04-25T11:14:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.686+0000 7f7d56000640 1 -- 172.21.15.7:0/441165772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d580047e0 con 0x7f7d600d0810 2024-04-25T11:14:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.686+0000 7f7d56000640 1 -- 172.21.15.7:0/441165772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58004bd0 con 0x7f7d600d0810 2024-04-25T11:14:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.686+0000 7f7d5f200640 1 -- 172.21.15.7:0/441165772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d601ea240 con 0x7f7d600d0810 2024-04-25T11:14:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.686+0000 7f7d5f200640 1 -- 172.21.15.7:0/441165772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d601ea800 con 0x7f7d600d0810 2024-04-25T11:14:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.687+0000 7f7d5f200640 1 -- 172.21.15.7:0/441165772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d600b85f0 con 0x7f7d600d0810 2024-04-25T11:14:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.689+0000 7f7d56000640 1 -- 172.21.15.7:0/441165772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7d58011070 con 0x7f7d600d0810 2024-04-25T11:14:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.690+0000 7f7d56000640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d3c0744b0 0x7f7d3c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.690+0000 7f7d56000640 1 -- 172.21.15.7:0/441165772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f7d58094fe0 con 0x7f7d600d0810 2024-04-25T11:14:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.691+0000 7f7d5de00640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d3c0744b0 0x7f7d3c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.691+0000 7f7d5de00640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d3c0744b0 0x7f7d3c0769c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7d50034000 tx=0x7f7d5002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:03.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.695+0000 7f7d56000640 1 -- 172.21.15.7:0/441165772 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d600b85f0 con 0x7f7d600d0810 2024-04-25T11:14:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.902+0000 7f7d5f200640 1 -- 172.21.15.7:0/441165772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d600d1750 con 0x7f7d600d0810 2024-04-25T11:14:03.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.904+0000 7f7d56000640 1 -- 172.21.15.7:0/441165772 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16941 (secure 0 0 0) 0x7f7d600d1750 con 0x7f7d600d0810 2024-04-25T11:14:03.904 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:03.905 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:02.984608+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:13:40.446828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T11:13:41.263101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.e","mappings":[{"from":0,"to":6}]},{"pgid":"8.32","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:03.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 -- 172.21.15.7:0/441165772 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d3c0744b0 msgr2=0x7f7d3c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d3c0744b0 0x7f7d3c0769c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7d50034000 tx=0x7f7d5002e000 comp rx=0 tx=0).stop 2024-04-25T11:14:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 -- 172.21.15.7:0/441165772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d600d0810 msgr2=0x7f7d601ead40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d600d0810 0x7f7d601ead40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c3e0 tx=0x7f7d5800c8a0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 -- 172.21.15.7:0/441165772 shutdown_connections 2024-04-25T11:14:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d3c0744b0 0x7f7d3c0769c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d600d0810 0x7f7d601ead40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d600b7f90 0x7f7d600ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 --2- 172.21.15.7:0/441165772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d6006e950 0x7f7d601e9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 -- 172.21.15.7:0/441165772 >> 172.21.15.7:0/441165772 conn(0x7f7d600d2840 msgr2=0x7f7d6015ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 -- 172.21.15.7:0/441165772 shutdown_connections 2024-04-25T11:14:03.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:03.907+0000 7f7d5f200640 1 -- 172.21.15.7:0/441165772 wait complete. 2024-04-25T11:14:03.934 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:14:03.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.219+0000 7fd36ae00640 1 Processor -- start 2024-04-25T11:14:04.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.219+0000 7fd36ae00640 1 -- start start 2024-04-25T11:14:04.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.222+0000 7fd36ae00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3640b14c0 0x7fd36415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.222+0000 7fd36ae00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd3640d2650 0x7fd3640b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.222+0000 7fd36ae00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd36406d720 con 0x7fd3640b44d0 2024-04-25T11:14:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.222+0000 7fd36ae00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd3640d2df0 con 0x7fd3640b14c0 2024-04-25T11:14:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.222+0000 7fd36ae00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd3640d30d0 con 0x7fd3640d2650 2024-04-25T11:14:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.222+0000 7fd363e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3640b14c0 0x7fd36415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.222+0000 7fd368e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd3640b44d0 0x7fd3640b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41216/0 (socket says 172.21.15.7:41216) 2024-04-25T11:14:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.222+0000 7fd363e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3640b14c0 0x7fd36415dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37918/0 (socket says 172.21.15.7:37918) 2024-04-25T11:14:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.222+0000 7fd368e00640 1 -- 172.21.15.7:0/2177614936 learned_addr learned my addr 172.21.15.7:0/2177614936 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:04.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.222+0000 7fd363400640 1 --2- 172.21.15.7:0/2177614936 >> v2:172.21.15.87:3300/0 conn(0x7fd3640d2650 0x7fd3640b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.223+0000 7fd363e00640 1 -- 172.21.15.7:0/2177614936 >> v2:172.21.15.87:3300/0 conn(0x7fd3640d2650 msgr2=0x7fd3640b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.223+0000 7fd363e00640 1 --2- 172.21.15.7:0/2177614936 >> v2:172.21.15.87:3300/0 conn(0x7fd3640d2650 0x7fd3640b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.223+0000 7fd363e00640 1 -- 172.21.15.7:0/2177614936 >> v1:172.21.15.7:6789/0 conn(0x7fd3640b44d0 legacy=0x7fd3640b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:04.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.223+0000 7fd363e00640 1 -- 172.21.15.7:0/2177614936 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3640d33b0 con 0x7fd3640b14c0 2024-04-25T11:14:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.223+0000 7fd362a00640 1 -- 172.21.15.7:0/2177614936 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4015783355 0 0) 0x7fd36406d720 con 0x7fd3640b44d0 2024-04-25T11:14:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.224+0000 7fd363e00640 1 --2- 172.21.15.7:0/2177614936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3640b14c0 0x7fd36415dee0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd36406d8f0 tx=0x7fd35402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e430f26d0252008 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.224+0000 7fd362a00640 1 -- 172.21.15.7:0/2177614936 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354037070 con 0x7fd3640b14c0 2024-04-25T11:14:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.224+0000 7fd362a00640 1 -- 172.21.15.7:0/2177614936 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd354032440 con 0x7fd3640b14c0 2024-04-25T11:14:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.224+0000 7fd362a00640 1 -- 172.21.15.7:0/2177614936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35402ae60 con 0x7fd3640b14c0 2024-04-25T11:14:04.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.225+0000 7fd36ae00640 1 -- 172.21.15.7:0/2177614936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3640b14c0 msgr2=0x7fd36415dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.225+0000 7fd36ae00640 1 --2- 172.21.15.7:0/2177614936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3640b14c0 0x7fd36415dee0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd36406d8f0 tx=0x7fd35402a110 comp rx=0 tx=0).stop 2024-04-25T11:14:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.225+0000 7fd36ae00640 1 -- 172.21.15.7:0/2177614936 shutdown_connections 2024-04-25T11:14:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.225+0000 7fd36ae00640 1 --2- 172.21.15.7:0/2177614936 >> v2:172.21.15.87:3300/0 conn(0x7fd3640d2650 0x7fd3640b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.225+0000 7fd36ae00640 1 --2- 172.21.15.7:0/2177614936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3640b14c0 0x7fd36415dee0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.225+0000 7fd36ae00640 1 -- 172.21.15.7:0/2177614936 >> 172.21.15.7:0/2177614936 conn(0x7fd3640b7f50 msgr2=0x7fd3640b76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.225+0000 7fd36ae00640 1 -- 172.21.15.7:0/2177614936 shutdown_connections 2024-04-25T11:14:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.225+0000 7fd36ae00640 1 -- 172.21.15.7:0/2177614936 wait complete. 2024-04-25T11:14:04.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.227+0000 7fd36ae00640 1 Processor -- start 2024-04-25T11:14:04.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.228+0000 7fd36ae00640 1 -- start start 2024-04-25T11:14:04.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.229+0000 7fd36ae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3640b14c0 0x7fd3641f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.229+0000 7fd36ae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3640b44d0 0x7fd364155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.229+0000 7fd36ae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3640d2650 0x7fd364143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.229+0000 7fd36ae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3640b0840 con 0x7fd3640d2650 2024-04-25T11:14:04.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.229+0000 7fd36ae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd3640d2a60 con 0x7fd3640b14c0 2024-04-25T11:14:04.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.229+0000 7fd36ae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd3640b09c0 con 0x7fd3640b44d0 2024-04-25T11:14:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.230+0000 7fd368e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3640d2650 0x7fd364143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:04.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.230+0000 7fd368e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3640d2650 0x7fd364143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37934/0 (socket says 172.21.15.7:37934) 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.230+0000 7fd368e00640 1 -- 172.21.15.7:0/2145157566 learned_addr learned my addr 172.21.15.7:0/2145157566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.230+0000 7fd363e00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3640b14c0 0x7fd3641f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.230+0000 7fd368e00640 1 -- 172.21.15.7:0/2145157566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3640b44d0 msgr2=0x7fd364155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.230+0000 7fd368e00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3640b44d0 0x7fd364155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.230+0000 7fd368e00640 1 -- 172.21.15.7:0/2145157566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3640b14c0 msgr2=0x7fd3641f58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.230+0000 7fd368e00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3640b14c0 0x7fd3641f58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.230+0000 7fd368e00640 1 -- 172.21.15.7:0/2145157566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3641f64a0 con 0x7fd3640d2650 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.230+0000 7fd363e00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3640b14c0 0x7fd3641f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.231+0000 7fd368e00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3640d2650 0x7fd364143f00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00c700 tx=0x7fd35c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.231+0000 7fd35be00640 1 -- 172.21.15.7:0/2145157566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c018070 con 0x7fd3640d2650 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.231+0000 7fd35be00640 1 -- 172.21.15.7:0/2145157566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd35c00ce10 con 0x7fd3640d2650 2024-04-25T11:14:04.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.231+0000 7fd35be00640 1 -- 172.21.15.7:0/2145157566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00a860 con 0x7fd3640d2650 2024-04-25T11:14:04.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.231+0000 7fd36ae00640 1 -- 172.21.15.7:0/2145157566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3641f6780 con 0x7fd3640d2650 2024-04-25T11:14:04.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.231+0000 7fd36ae00640 1 -- 172.21.15.7:0/2145157566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36415f060 con 0x7fd3640d2650 2024-04-25T11:14:04.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.232+0000 7fd36ae00640 1 -- 172.21.15.7:0/2145157566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3641f6210 con 0x7fd3640d2650 2024-04-25T11:14:04.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.234+0000 7fd35be00640 1 -- 172.21.15.7:0/2145157566 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd35c00aa00 con 0x7fd3640d2650 2024-04-25T11:14:04.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.235+0000 7fd35be00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3440744b0 0x7fd3440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.235+0000 7fd35be00640 1 -- 172.21.15.7:0/2145157566 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6159+0+0 (secure 0 0 0) 0x7fd35c094ad0 con 0x7fd3640d2650 2024-04-25T11:14:04.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.235+0000 7fd363e00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3440744b0 0x7fd3440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:04.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.236+0000 7fd363e00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3440744b0 0x7fd3440769c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd354034000 tx=0x7fd35402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:04.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.239+0000 7fd35be00640 1 -- 172.21.15.7:0/2145157566 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd35c05f000 con 0x7fd3640d2650 2024-04-25T11:14:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.485+0000 7fd36ae00640 1 -- 172.21.15.7:0/2145157566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd36406a100 con 0x7fd3640d2650 2024-04-25T11:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.487+0000 7fd35be00640 1 -- 172.21.15.7:0/2145157566 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5172 (secure 0 0 0) 0x7fd35c05e900 con 0x7fd3640d2650 2024-04-25T11:14:04.487 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:04.487 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2225639104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2225639104},{"type":"v1","addr":"172.21.15.87:6834","nonce":2225639104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2625900548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2625900548},{"type":"v1","addr":"172.21.15.7:6837","nonce":2625900548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/717352569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":717352569},{"type":"v1","addr":"172.21.15.87:6835","nonce":717352569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:13:42.310229+0000","modified":"2024-04-25T11:13:45.341610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6836/3503942462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3503942462},{"type":"v1","addr":"172.21.15.7:6836","nonce":3503942462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-25T11:14:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-25T11:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.489+0000 7fd36ae00640 1 -- 172.21.15.7:0/2145157566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3440744b0 msgr2=0x7fd3440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.490+0000 7fd36ae00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3440744b0 0x7fd3440769c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd354034000 tx=0x7fd35402e000 comp rx=0 tx=0).stop 2024-04-25T11:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.490+0000 7fd36ae00640 1 -- 172.21.15.7:0/2145157566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3640d2650 msgr2=0x7fd364143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.490+0000 7fd36ae00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3640d2650 0x7fd364143f00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00c700 tx=0x7fd35c00cbc0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.490+0000 7fd36ae00640 1 -- 172.21.15.7:0/2145157566 shutdown_connections 2024-04-25T11:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.490+0000 7fd36ae00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3440744b0 0x7fd3440769c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.490+0000 7fd36ae00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3640d2650 0x7fd364143f00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.490+0000 7fd36ae00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3640b44d0 0x7fd364155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.490+0000 7fd36ae00640 1 --2- 172.21.15.7:0/2145157566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3640b14c0 0x7fd3641f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.490+0000 7fd36ae00640 1 -- 172.21.15.7:0/2145157566 >> 172.21.15.7:0/2145157566 conn(0x7fd3640b7f50 msgr2=0x7fd36406cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.490+0000 7fd36ae00640 1 -- 172.21.15.7:0/2145157566 shutdown_connections 2024-04-25T11:14:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.490+0000 7fd36ae00640 1 -- 172.21.15.7:0/2145157566 wait complete. 2024-04-25T11:14:04.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.807+0000 7ffb26400640 1 Processor -- start 2024-04-25T11:14:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.807+0000 7ffb26400640 1 -- start start 2024-04-25T11:14:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.809+0000 7ffb26400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb2006dcd0 0x7ffb2006e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.809+0000 7ffb26400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffb2015f580 0x7ffb2015f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.809+0000 7ffb26400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb200d35a0 con 0x7ffb20163ac0 2024-04-25T11:14:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.810+0000 7ffb26400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ffb2006e620 con 0x7ffb2006dcd0 2024-04-25T11:14:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.810+0000 7ffb26400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ffb20164310 con 0x7ffb2015f580 2024-04-25T11:14:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.810+0000 7ffb1fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb2006dcd0 0x7ffb2006e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.810+0000 7ffb1fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb2006dcd0 0x7ffb2006e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37948/0 (socket says 172.21.15.7:37948) 2024-04-25T11:14:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.810+0000 7ffb1fe00640 1 -- 172.21.15.7:0/404767226 learned_addr learned my addr 172.21.15.7:0/404767226 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.810+0000 7ffb25a00640 1 --2- 172.21.15.7:0/404767226 >> v2:172.21.15.87:3300/0 conn(0x7ffb2015f580 0x7ffb2015f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.810+0000 7ffb1fe00640 1 -- 172.21.15.7:0/404767226 >> v2:172.21.15.87:3300/0 conn(0x7ffb2015f580 msgr2=0x7ffb2015f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.810+0000 7ffb1fe00640 1 --2- 172.21.15.7:0/404767226 >> v2:172.21.15.87:3300/0 conn(0x7ffb2015f580 0x7ffb2015f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.810+0000 7ffb1fe00640 1 -- 172.21.15.7:0/404767226 >> v1:172.21.15.87:6789/0 conn(0x7ffb20163ac0 legacy=0x7ffb200b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.810+0000 7ffb1fe00640 1 -- 172.21.15.7:0/404767226 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb201645f0 con 0x7ffb2006dcd0 2024-04-25T11:14:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.810+0000 7ffb25a00640 1 --2- 172.21.15.7:0/404767226 >> v2:172.21.15.87:3300/0 conn(0x7ffb2015f580 0x7ffb2015f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.812+0000 7ffb1fe00640 1 --2- 172.21.15.7:0/404767226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb2006dcd0 0x7ffb2006e0e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ffb2006eba0 tx=0x7ffb0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f488b69f24fbf3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.812+0000 7ffb1f400640 1 -- 172.21.15.7:0/404767226 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08037070 con 0x7ffb2006dcd0 2024-04-25T11:14:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.812+0000 7ffb1f400640 1 -- 172.21.15.7:0/404767226 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb08032440 con 0x7ffb2006dcd0 2024-04-25T11:14:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.812+0000 7ffb1f400640 1 -- 172.21.15.7:0/404767226 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08032810 con 0x7ffb2006dcd0 2024-04-25T11:14:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.813+0000 7ffb26400640 1 -- 172.21.15.7:0/404767226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb2006dcd0 msgr2=0x7ffb2006e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.813+0000 7ffb26400640 1 --2- 172.21.15.7:0/404767226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb2006dcd0 0x7ffb2006e0e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ffb2006eba0 tx=0x7ffb0802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.813+0000 7ffb26400640 1 -- 172.21.15.7:0/404767226 shutdown_connections 2024-04-25T11:14:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.813+0000 7ffb26400640 1 --2- 172.21.15.7:0/404767226 >> v2:172.21.15.87:3300/0 conn(0x7ffb2015f580 0x7ffb2015f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.813+0000 7ffb26400640 1 --2- 172.21.15.7:0/404767226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb2006dcd0 0x7ffb2006e0e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.813+0000 7ffb26400640 1 -- 172.21.15.7:0/404767226 >> 172.21.15.7:0/404767226 conn(0x7ffb200d29e0 msgr2=0x7ffb200d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.813+0000 7ffb26400640 1 -- 172.21.15.7:0/404767226 shutdown_connections 2024-04-25T11:14:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.813+0000 7ffb26400640 1 -- 172.21.15.7:0/404767226 wait complete. 2024-04-25T11:14:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.816+0000 7ffb26400640 1 Processor -- start 2024-04-25T11:14:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.817+0000 7ffb26400640 1 -- start start 2024-04-25T11:14:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb26400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb2006dcd0 0x7ffb201f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb26400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb2015f580 0x7ffb20154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb26400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb20163ac0 0x7ffb20158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb26400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffb20164050 con 0x7ffb20163ac0 2024-04-25T11:14:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb26400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffb20163ed0 con 0x7ffb2006dcd0 2024-04-25T11:14:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb26400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2015fed0 con 0x7ffb2015f580 2024-04-25T11:14:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb25a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb20163ac0 0x7ffb20158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb25a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb20163ac0 0x7ffb20158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37950/0 (socket says 172.21.15.7:37950) 2024-04-25T11:14:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb25a00640 1 -- 172.21.15.7:0/988088397 learned_addr learned my addr 172.21.15.7:0/988088397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb1fe00640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb2015f580 0x7ffb20154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb25000640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb2006dcd0 0x7ffb201f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb25a00640 1 -- 172.21.15.7:0/988088397 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb2015f580 msgr2=0x7ffb20154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb25a00640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb2015f580 0x7ffb20154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb25a00640 1 -- 172.21.15.7:0/988088397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb2006dcd0 msgr2=0x7ffb201f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb25a00640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb2006dcd0 0x7ffb201f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.819+0000 7ffb25a00640 1 -- 172.21.15.7:0/988088397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2015a9a0 con 0x7ffb20163ac0 2024-04-25T11:14:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.820+0000 7ffb25a00640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb20163ac0 0x7ffb20158040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c7c0 tx=0x7ffb1000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.820+0000 7ffb1cc00640 1 -- 172.21.15.7:0/988088397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10015020 con 0x7ffb20163ac0 2024-04-25T11:14:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.820+0000 7ffb1cc00640 1 -- 172.21.15.7:0/988088397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb10004230 con 0x7ffb20163ac0 2024-04-25T11:14:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.820+0000 7ffb1cc00640 1 -- 172.21.15.7:0/988088397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb10003610 con 0x7ffb20163ac0 2024-04-25T11:14:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.820+0000 7ffb26400640 1 -- 172.21.15.7:0/988088397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2015ac80 con 0x7ffb20163ac0 2024-04-25T11:14:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.820+0000 7ffb26400640 1 -- 172.21.15.7:0/988088397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb20158580 con 0x7ffb20163ac0 2024-04-25T11:14:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.822+0000 7ffb26400640 1 -- 172.21.15.7:0/988088397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb2015b1a0 con 0x7ffb20163ac0 2024-04-25T11:14:04.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.823+0000 7ffb1cc00640 1 -- 172.21.15.7:0/988088397 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffb10018020 con 0x7ffb20163ac0 2024-04-25T11:14:04.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.824+0000 7ffb1cc00640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffafc0744b0 0x7ffafc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:04.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.824+0000 7ffb1cc00640 1 -- 172.21.15.7:0/988088397 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6159+0+0 (secure 0 0 0) 0x7ffb100954d0 con 0x7ffb20163ac0 2024-04-25T11:14:04.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.824+0000 7ffb25000640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffafc0744b0 0x7ffafc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:04.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.825+0000 7ffb25000640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffafc0744b0 0x7ffafc0769c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009f60 tx=0x7ffb180094f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:04.829+0000 7ffb1cc00640 1 -- 172.21.15.7:0/988088397 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffb1001f020 con 0x7ffb20163ac0 2024-04-25T11:14:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.075+0000 7ffb26400640 1 -- 172.21.15.7:0/988088397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb20050740 con 0x7ffb20163ac0 2024-04-25T11:14:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.076+0000 7ffb1cc00640 1 -- 172.21.15.7:0/988088397 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5172 (secure 0 0 0) 0x7ffb1005fb50 con 0x7ffb20163ac0 2024-04-25T11:14:05.077 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:05.077 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2225639104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2225639104},{"type":"v1","addr":"172.21.15.87:6834","nonce":2225639104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2625900548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2625900548},{"type":"v1","addr":"172.21.15.7:6837","nonce":2625900548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/717352569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":717352569},{"type":"v1","addr":"172.21.15.87:6835","nonce":717352569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:13:42.310229+0000","modified":"2024-04-25T11:13:45.341610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5742":{"gid":5742,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6836/3503942462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3503942462},{"type":"v1","addr":"172.21.15.7:6836","nonce":3503942462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5742,"qdb_cluster":[5742]},"id":3}]} 2024-04-25T11:14:05.077 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2024-04-25T11:14:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 -- 172.21.15.7:0/988088397 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffafc0744b0 msgr2=0x7ffafc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffafc0744b0 0x7ffafc0769c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009f60 tx=0x7ffb180094f0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 -- 172.21.15.7:0/988088397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb20163ac0 msgr2=0x7ffb20158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb20163ac0 0x7ffb20158040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c7c0 tx=0x7ffb1000cc80 comp rx=0 tx=0).stop 2024-04-25T11:14:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 -- 172.21.15.7:0/988088397 shutdown_connections 2024-04-25T11:14:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffafc0744b0 0x7ffafc0769c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb20163ac0 0x7ffb20158040 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb2015f580 0x7ffb20154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 --2- 172.21.15.7:0/988088397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb2006dcd0 0x7ffb201f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 -- 172.21.15.7:0/988088397 >> 172.21.15.7:0/988088397 conn(0x7ffb200d29e0 msgr2=0x7ffb20161600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 -- 172.21.15.7:0/988088397 shutdown_connections 2024-04-25T11:14:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.079+0000 7ffb26400640 1 -- 172.21.15.7:0/988088397 wait complete. 2024-04-25T11:14:05.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.372+0000 7f3ee3400640 1 Processor -- start 2024-04-25T11:14:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.373+0000 7f3ee3400640 1 -- start start 2024-04-25T11:14:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.375+0000 7f3ee3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3edc0af540 0x7f3edc09f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.375+0000 7f3ee3400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3edc15f4e0 con 0x7f3edc15ef90 2024-04-25T11:14:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.375+0000 7f3ee3400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3edc0b0560 con 0x7f3edc15bc40 2024-04-25T11:14:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.375+0000 7f3ee3400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3edc071190 con 0x7f3edc0af540 2024-04-25T11:14:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.376+0000 7f3ee0a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3edc15ef90 0x7f3edc0aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41228/0 (socket says 172.21.15.7:41228) 2024-04-25T11:14:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.376+0000 7f3ee0a00640 1 -- 172.21.15.7:0/2657553421 learned_addr learned my addr 172.21.15.7:0/2657553421 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:05.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.376+0000 7f3edbe00640 1 --2- 172.21.15.7:0/2657553421 >> v2:172.21.15.87:3300/0 conn(0x7f3edc0af540 0x7f3edc09f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.376+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1466657537 0 0) 0x7f3edc15f4e0 con 0x7f3edc15ef90 2024-04-25T11:14:05.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.376+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ec4003520 con 0x7f3edc15ef90 2024-04-25T11:14:05.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 775608683 0 0) 0x7f3ec4003520 con 0x7f3edc15ef90 2024-04-25T11:14:05.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 >> v2:172.21.15.87:3300/0 conn(0x7f3edc0af540 msgr2=0x7f3edc09f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 --2- 172.21.15.7:0/2657553421 >> v2:172.21.15.87:3300/0 conn(0x7f3edc0af540 0x7f3edc09f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 >> v1:172.21.15.87:6789/0 conn(0x7f3edc15bc40 legacy=0x7f3edc0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc0b0840 con 0x7f3edc15ef90 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f3ed0001670 con 0x7f3edc15ef90 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 >> v1:172.21.15.7:6789/0 conn(0x7f3edc15ef90 legacy=0x7f3edc0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ec4003a00 0x7f3ec4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ec4004800 0x7f3ec4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ec40052a0 0x7f3ec400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc071190 con 0x7f3ec4003a00 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec400c230 con 0x7f3ec40052a0 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec400c430 con 0x7f3ec4004800 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.377+0000 7f3ee0a00640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ec4003a00 0x7f3ec4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.378+0000 7f3ee1400640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ec40052a0 0x7f3ec400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.378+0000 7f3edbe00640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ec4004800 0x7f3ec4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.378+0000 7f3ee0a00640 1 -- 172.21.15.7:0/2657553421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ec4004800 msgr2=0x7f3ec4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.378+0000 7f3ee0a00640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ec4004800 0x7f3ec4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.378+0000 7f3ee0a00640 1 -- 172.21.15.7:0/2657553421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ec40052a0 msgr2=0x7f3ec400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.378+0000 7f3ee0a00640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ec40052a0 0x7f3ec400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.378+0000 7f3ee0a00640 1 -- 172.21.15.7:0/2657553421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec4003520 con 0x7f3ec4003a00 2024-04-25T11:14:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.378+0000 7f3ee1400640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ec40052a0 0x7f3ec400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:14:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.378+0000 7f3ee1400640 1 -- 172.21.15.7:0/2657553421 reap_dead start 2024-04-25T11:14:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.379+0000 7f3ee0a00640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ec4003a00 0x7f3ec4003e70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c110 tx=0x7f3ec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23476eaac047e445 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.380+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0007c40 con 0x7f3ec4003a00 2024-04-25T11:14:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.380+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed0005af0 con 0x7f3ec4003a00 2024-04-25T11:14:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.380+0000 7f3edb400640 1 -- 172.21.15.7:0/2657553421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0007070 con 0x7f3ec4003a00 2024-04-25T11:14:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.380+0000 7f3ee3400640 1 -- 172.21.15.7:0/2657553421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ec4003a00 msgr2=0x7f3ec4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.380+0000 7f3ee3400640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ec4003a00 0x7f3ec4003e70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000c110 tx=0x7f3ec4005830 comp rx=0 tx=0).stop 2024-04-25T11:14:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.380+0000 7f3ee3400640 1 -- 172.21.15.7:0/2657553421 shutdown_connections 2024-04-25T11:14:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.380+0000 7f3ee3400640 1 --2- 172.21.15.7:0/2657553421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ec4003a00 0x7f3ec4003e70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.380+0000 7f3ee3400640 1 -- 172.21.15.7:0/2657553421 >> 172.21.15.7:0/2657553421 conn(0x7f3edc074200 msgr2=0x7f3edc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.381+0000 7f3ee3400640 1 -- 172.21.15.7:0/2657553421 shutdown_connections 2024-04-25T11:14:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.381+0000 7f3ee3400640 1 -- 172.21.15.7:0/2657553421 wait complete. 2024-04-25T11:14:05.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.383+0000 7f3ee3400640 1 Processor -- start 2024-04-25T11:14:05.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.383+0000 7f3ee3400640 1 -- start start 2024-04-25T11:14:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.384+0000 7f3ee3400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ec4003a00 0x7f3edc15ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.384+0000 7f3ee3400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3edc152090 0x7f3edc147270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.384+0000 7f3ee3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3edc156070 0x7f3edc156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.384+0000 7f3ee3400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc0b1320 con 0x7f3edc156070 2024-04-25T11:14:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.384+0000 7f3ee3400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc0b0e00 con 0x7f3ec4003a00 2024-04-25T11:14:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.384+0000 7f3ee3400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3edc0afb80 con 0x7f3edc152090 2024-04-25T11:14:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3ee1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3edc156070 0x7f3edc156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3ee1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3edc156070 0x7f3edc156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37976/0 (socket says 172.21.15.7:37976) 2024-04-25T11:14:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3ee1400640 1 -- 172.21.15.7:0/2892109164 learned_addr learned my addr 172.21.15.7:0/2892109164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:05.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3ee0a00640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ec4003a00 0x7f3edc15ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3ee1400640 1 -- 172.21.15.7:0/2892109164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3edc152090 msgr2=0x7f3edc147270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3edbe00640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3edc152090 0x7f3edc147270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3ee1400640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3edc152090 0x7f3edc147270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3ee1400640 1 -- 172.21.15.7:0/2892109164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ec4003a00 msgr2=0x7f3edc15ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3ee1400640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ec4003a00 0x7f3edc15ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3ee1400640 1 -- 172.21.15.7:0/2892109164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc1569c0 con 0x7f3edc156070 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3edbe00640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3edc152090 0x7f3edc147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.385+0000 7f3ee0a00640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ec4003a00 0x7f3edc15ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.386+0000 7f3ee1400640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3edc156070 0x7f3edc156480 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0cf1e0 tx=0x7f3ed400e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.386+0000 7f3ed8c00640 1 -- 172.21.15.7:0/2892109164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4003b80 con 0x7f3edc156070 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.386+0000 7f3ed8c00640 1 -- 172.21.15.7:0/2892109164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed4018070 con 0x7f3edc156070 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.386+0000 7f3ed8c00640 1 -- 172.21.15.7:0/2892109164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed40050d0 con 0x7f3edc156070 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.386+0000 7f3ee3400640 1 -- 172.21.15.7:0/2892109164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3edc1595f0 con 0x7f3edc156070 2024-04-25T11:14:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.386+0000 7f3ee3400640 1 -- 172.21.15.7:0/2892109164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3edc159dd0 con 0x7f3edc156070 2024-04-25T11:14:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.387+0000 7f3ee3400640 1 -- 172.21.15.7:0/2892109164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc159860 con 0x7f3edc156070 2024-04-25T11:14:05.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.388+0000 7f3ed8c00640 1 -- 172.21.15.7:0/2892109164 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3ed4005270 con 0x7f3edc156070 2024-04-25T11:14:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.390+0000 7f3ed8c00640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ebc0744b0 0x7f3ebc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.391+0000 7f3ed8c00640 1 -- 172.21.15.7:0/2892109164 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6159+0+0 (secure 0 0 0) 0x7f3ed4094fe0 con 0x7f3edc156070 2024-04-25T11:14:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.391+0000 7f3ee0a00640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ebc0744b0 0x7f3ebc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.391+0000 7f3ee0a00640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ebc0744b0 0x7f3ebc0769c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0032000 tx=0x7f3ed0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:05.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.395+0000 7f3ed8c00640 1 -- 172.21.15.7:0/2892109164 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3ed405f630 con 0x7f3edc156070 2024-04-25T11:14:05.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.601+0000 7f3ee3400640 1 -- 172.21.15.7:0/2892109164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3edc0b7c50 con 0x7f3edc156070 2024-04-25T11:14:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.604+0000 7f3ed8c00640 1 -- 172.21.15.7:0/2892109164 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16941 (secure 0 0 0) 0x7f3edc0b7c50 con 0x7f3edc156070 2024-04-25T11:14:05.604 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:05.604 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":56,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:03.998283+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:13:40.446828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T11:13:41.263101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.e","mappings":[{"from":0,"to":6}]},{"pgid":"8.32","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.606+0000 7f3ee3400640 1 -- 172.21.15.7:0/2892109164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ebc0744b0 msgr2=0x7f3ebc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.606+0000 7f3ee3400640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ebc0744b0 0x7f3ebc0769c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0032000 tx=0x7f3ed0009030 comp rx=0 tx=0).stop 2024-04-25T11:14:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.606+0000 7f3ee3400640 1 -- 172.21.15.7:0/2892109164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3edc156070 msgr2=0x7f3edc156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.606+0000 7f3ee3400640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3edc156070 0x7f3edc156480 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0cf1e0 tx=0x7f3ed400e980 comp rx=0 tx=0).stop 2024-04-25T11:14:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.607+0000 7f3ee3400640 1 -- 172.21.15.7:0/2892109164 shutdown_connections 2024-04-25T11:14:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.607+0000 7f3ee3400640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ebc0744b0 0x7f3ebc0769c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.607+0000 7f3ee3400640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3edc156070 0x7f3edc156480 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.607+0000 7f3ee3400640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3edc152090 0x7f3edc147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.607+0000 7f3ee3400640 1 --2- 172.21.15.7:0/2892109164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ec4003a00 0x7f3edc15ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.607+0000 7f3ee3400640 1 -- 172.21.15.7:0/2892109164 >> 172.21.15.7:0/2892109164 conn(0x7f3edc074200 msgr2=0x7f3edc06d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.607+0000 7f3ee3400640 1 -- 172.21.15.7:0/2892109164 shutdown_connections 2024-04-25T11:14:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.607+0000 7f3ee3400640 1 -- 172.21.15.7:0/2892109164 wait complete. 2024-04-25T11:14:05.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:14:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.911+0000 7ff3a7200640 1 Processor -- start 2024-04-25T11:14:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.911+0000 7ff3a7200640 1 -- start start 2024-04-25T11:14:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.914+0000 7ff3a7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a81416a0 0x7ff3a80b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.914+0000 7ff3a7200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a806c570 con 0x7ff3a8141290 2024-04-25T11:14:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.914+0000 7ff3a7200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a806c850 con 0x7ff3a815eed0 2024-04-25T11:14:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.914+0000 7ff3a7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a806cb30 con 0x7ff3a81416a0 2024-04-25T11:14:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.914+0000 7ff39d400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff3a8141290 0x7ff3a806d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41242/0 (socket says 172.21.15.7:41242) 2024-04-25T11:14:05.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.914+0000 7ff39d400640 1 -- 172.21.15.7:0/1336935663 learned_addr learned my addr 172.21.15.7:0/1336935663 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:05.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.914+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763270885 0 0) 0x7ff3a806c570 con 0x7ff3a8141290 2024-04-25T11:14:05.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.914+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff388003520 con 0x7ff3a8141290 2024-04-25T11:14:05.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.914+0000 7ff3a6800640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a81416a0 0x7ff3a80b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 6985614 0 0) 0x7ff388003520 con 0x7ff3a8141290 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a81416a0 msgr2=0x7ff3a80b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a81416a0 0x7ff3a80b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 >> v1:172.21.15.87:6789/0 conn(0x7ff3a815eed0 legacy=0x7ff3a80cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a806ce10 con 0x7ff3a8141290 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7ff390001690 con 0x7ff3a8141290 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 >> v1:172.21.15.7:6789/0 conn(0x7ff3a8141290 legacy=0x7ff3a806d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3880039e0 0x7ff388003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 0x7ff388004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a81416a0 0x7ff3880052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a806cb30 con 0x7ff388004800 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff38800bbd0 con 0x7ff3880039e0 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff38800bdd0 con 0x7ff3a81416a0 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.915+0000 7ff39d400640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 0x7ff388004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.916+0000 7ff3a5e00640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3880039e0 0x7ff388003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.916+0000 7ff3a6800640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a81416a0 0x7ff3880052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.916+0000 7ff39d400640 1 -- 172.21.15.7:0/1336935663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a81416a0 msgr2=0x7ff3880052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.916+0000 7ff39d400640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a81416a0 0x7ff3880052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.916+0000 7ff39d400640 1 -- 172.21.15.7:0/1336935663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3880039e0 msgr2=0x7ff388003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.916+0000 7ff39d400640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3880039e0 0x7ff388003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.916+0000 7ff39d400640 1 -- 172.21.15.7:0/1336935663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff388003520 con 0x7ff388004800 2024-04-25T11:14:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.917+0000 7ff39d400640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 0x7ff388004c10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff39000c210 tx=0x7ff388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbffc760db20a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.917+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390007d30 con 0x7ff388004800 2024-04-25T11:14:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.917+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3900058d0 con 0x7ff388004800 2024-04-25T11:14:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.917+0000 7ff3a5400640 1 -- 172.21.15.7:0/1336935663 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390005ca0 con 0x7ff388004800 2024-04-25T11:14:05.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.918+0000 7ff3a7200640 1 -- 172.21.15.7:0/1336935663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 msgr2=0x7ff388004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.918+0000 7ff3a7200640 1 --2- 172.21.15.7:0/1336935663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 0x7ff388004c10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff39000c210 tx=0x7ff388005830 comp rx=0 tx=0).stop 2024-04-25T11:14:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.918+0000 7ff3a6800640 1 -- 172.21.15.7:0/1336935663 reap_dead start 2024-04-25T11:14:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.918+0000 7ff3a7200640 1 -- 172.21.15.7:0/1336935663 shutdown_connections 2024-04-25T11:14:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.918+0000 7ff3a7200640 1 -- 172.21.15.7:0/1336935663 >> 172.21.15.7:0/1336935663 conn(0x7ff3a80d1540 msgr2=0x7ff3a80d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.918+0000 7ff3a7200640 1 -- 172.21.15.7:0/1336935663 shutdown_connections 2024-04-25T11:14:05.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.918+0000 7ff3a7200640 1 -- 172.21.15.7:0/1336935663 wait complete. 2024-04-25T11:14:05.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.921+0000 7ff3a7200640 1 Processor -- start 2024-04-25T11:14:05.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.921+0000 7ff3a7200640 1 -- start start 2024-04-25T11:14:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.922+0000 7ff3a7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 0x7ff3a81f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a7200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3a81399e0 0x7ff3a815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a815d230 0x7ff3a80d0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a806dd20 con 0x7ff388004800 2024-04-25T11:14:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a7200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a8141ab0 con 0x7ff3a81399e0 2024-04-25T11:14:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a806dea0 con 0x7ff3a815d230 2024-04-25T11:14:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 0x7ff3a81f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 0x7ff3a81f59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37998/0 (socket says 172.21.15.7:37998) 2024-04-25T11:14:05.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a5e00640 1 -- 172.21.15.7:0/2374919586 learned_addr learned my addr 172.21.15.7:0/2374919586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:05.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff39d400640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3a81399e0 0x7ff3a815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a5e00640 1 -- 172.21.15.7:0/2374919586 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a815d230 msgr2=0x7ff3a80d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a5e00640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a815d230 0x7ff3a80d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a5e00640 1 -- 172.21.15.7:0/2374919586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3a81399e0 msgr2=0x7ff3a815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a5e00640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3a81399e0 0x7ff3a815ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.923+0000 7ff3a5e00640 1 -- 172.21.15.7:0/2374919586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a815d640 con 0x7ff388004800 2024-04-25T11:14:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.924+0000 7ff3a5e00640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 0x7ff3a81f59b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff39800e580 tx=0x7ff39800ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.924+0000 7ff39ea00640 1 -- 172.21.15.7:0/2374919586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3980052e0 con 0x7ff388004800 2024-04-25T11:14:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.924+0000 7ff39ea00640 1 -- 172.21.15.7:0/2374919586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff398018070 con 0x7ff388004800 2024-04-25T11:14:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.924+0000 7ff39ea00640 1 -- 172.21.15.7:0/2374919586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398003810 con 0x7ff388004800 2024-04-25T11:14:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.924+0000 7ff3a7200640 1 -- 172.21.15.7:0/2374919586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a80cf460 con 0x7ff388004800 2024-04-25T11:14:05.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.924+0000 7ff3a7200640 1 -- 172.21.15.7:0/2374919586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a80cfa20 con 0x7ff388004800 2024-04-25T11:14:05.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.925+0000 7ff3a7200640 1 -- 172.21.15.7:0/2374919586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a8050740 con 0x7ff388004800 2024-04-25T11:14:05.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.927+0000 7ff39ea00640 1 -- 172.21.15.7:0/2374919586 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff3980039b0 con 0x7ff388004800 2024-04-25T11:14:05.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.928+0000 7ff39ea00640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3840744b0 0x7ff3840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.928+0000 7ff39ea00640 1 -- 172.21.15.7:0/2374919586 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6159+0+0 (secure 0 0 0) 0x7ff3980950c0 con 0x7ff388004800 2024-04-25T11:14:05.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.928+0000 7ff39d400640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3840744b0 0x7ff3840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:05.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.929+0000 7ff39d400640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3840744b0 0x7ff3840769c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff390033000 tx=0x7ff39002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:05.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:05.933+0000 7ff39ea00640 1 -- 172.21.15.7:0/2374919586 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3a815d640 con 0x7ff388004800 2024-04-25T11:14:06.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.178+0000 7ff3a7200640 1 -- 172.21.15.7:0/2374919586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff3a8140680 con 0x7ff388004800 2024-04-25T11:14:06.179 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-04-25T11:14:06.179+0000 7fac34e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:14:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.182+0000 7ff39ea00640 1 -- 172.21.15.7:0/2374919586 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7ff39805f730 con 0x7ff388004800 2024-04-25T11:14:06.182 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:14:06.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.186+0000 7ff3a7200640 1 -- 172.21.15.7:0/2374919586 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3840744b0 msgr2=0x7ff3840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:06.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.186+0000 7ff3a7200640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3840744b0 0x7ff3840769c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff390033000 tx=0x7ff39002e000 comp rx=0 tx=0).stop 2024-04-25T11:14:06.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.186+0000 7ff3a7200640 1 -- 172.21.15.7:0/2374919586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 msgr2=0x7ff3a81f59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:06.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.186+0000 7ff3a7200640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 0x7ff3a81f59b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff39800e580 tx=0x7ff39800ea40 comp rx=0 tx=0).stop 2024-04-25T11:14:06.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.187+0000 7ff3a7200640 1 -- 172.21.15.7:0/2374919586 shutdown_connections 2024-04-25T11:14:06.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.187+0000 7ff3a7200640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3840744b0 0x7ff3840769c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:06.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.187+0000 7ff3a7200640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3a815d230 0x7ff3a80d0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:06.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.187+0000 7ff3a7200640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3a81399e0 0x7ff3a815ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:06.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.187+0000 7ff3a7200640 1 --2- 172.21.15.7:0/2374919586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff388004800 0x7ff3a81f59b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:06.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.187+0000 7ff3a7200640 1 -- 172.21.15.7:0/2374919586 >> 172.21.15.7:0/2374919586 conn(0x7ff3a80d1540 msgr2=0x7ff3a8143200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:06.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.188+0000 7ff3a7200640 1 -- 172.21.15.7:0/2374919586 shutdown_connections 2024-04-25T11:14:06.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.188+0000 7ff3a7200640 1 -- 172.21.15.7:0/2374919586 wait complete. 2024-04-25T11:14:06.194 INFO:tasks.ceph.mds.a.smithi007.stderr: -2432> 2024-04-25T11:13:47.653+0000 7f673e800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:14:06.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:14:06.321 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:14:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.486+0000 7f533ce00640 1 Processor -- start 2024-04-25T11:14:06.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.486+0000 7f533ce00640 1 -- start start 2024-04-25T11:14:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.489+0000 7f533ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f53380ce8d0 0x7f53380cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.489+0000 7f533ce00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f533806fe40 0x7f533815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.489+0000 7f533ce00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53380b4830 con 0x7f533815c2e0 2024-04-25T11:14:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.489+0000 7f533ce00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f53380b6600 con 0x7f53380ce8d0 2024-04-25T11:14:06.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.489+0000 7f533ce00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f53380b4b10 con 0x7f533806fe40 2024-04-25T11:14:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.489+0000 7f5337400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f53380ce8d0 0x7f53380cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.489+0000 7f5337e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f533806fe40 0x7f533815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.490+0000 7f5337400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f53380ce8d0 0x7f53380cece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50122/0 (socket says 172.21.15.7:50122) 2024-04-25T11:14:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.490+0000 7f5337400640 1 -- 172.21.15.7:0/1622504674 learned_addr learned my addr 172.21.15.7:0/1622504674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:06.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.490+0000 7f5337e00640 1 -- 172.21.15.7:0/1622504674 >> v2:172.21.15.87:3300/0 conn(0x7f53380ce8d0 msgr2=0x7f53380cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.490+0000 7f5337e00640 1 --2- 172.21.15.7:0/1622504674 >> v2:172.21.15.87:3300/0 conn(0x7f53380ce8d0 0x7f53380cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.490+0000 7f5337e00640 1 -- 172.21.15.7:0/1622504674 >> v1:172.21.15.87:6789/0 conn(0x7f533815c2e0 legacy=0x7f533806f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.490+0000 7f5337e00640 1 -- 172.21.15.7:0/1622504674 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53380b4df0 con 0x7f533806fe40 2024-04-25T11:14:06.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.490+0000 7f5337400640 1 --2- 172.21.15.7:0/1622504674 >> v2:172.21.15.87:3300/0 conn(0x7f53380ce8d0 0x7f53380cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:14:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.492+0000 7f5337e00640 1 --2- 172.21.15.7:0/1622504674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f533806fe40 0x7f533815c6f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5330009d90 tx=0x7f533002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17532ba44d6024a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:06.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.492+0000 7f5336000640 1 -- 172.21.15.7:0/1622504674 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330007740 con 0x7f533806fe40 2024-04-25T11:14:06.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.493+0000 7f5336000640 1 -- 172.21.15.7:0/1622504674 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53300078e0 con 0x7f533806fe40 2024-04-25T11:14:06.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.493+0000 7f5336000640 1 -- 172.21.15.7:0/1622504674 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330007cb0 con 0x7f533806fe40 2024-04-25T11:14:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.494+0000 7f533ce00640 1 -- 172.21.15.7:0/1622504674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f533806fe40 msgr2=0x7f533815c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.494+0000 7f533ce00640 1 --2- 172.21.15.7:0/1622504674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f533806fe40 0x7f533815c6f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5330009d90 tx=0x7f533002a660 comp rx=0 tx=0).stop 2024-04-25T11:14:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.494+0000 7f533ce00640 1 -- 172.21.15.7:0/1622504674 shutdown_connections 2024-04-25T11:14:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.494+0000 7f533ce00640 1 --2- 172.21.15.7:0/1622504674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f533806fe40 0x7f533815c6f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.494+0000 7f533ce00640 1 --2- 172.21.15.7:0/1622504674 >> v2:172.21.15.87:3300/0 conn(0x7f53380ce8d0 0x7f53380cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.494+0000 7f533ce00640 1 -- 172.21.15.7:0/1622504674 >> 172.21.15.7:0/1622504674 conn(0x7f53380740f0 msgr2=0x7f533806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.494+0000 7f533ce00640 1 -- 172.21.15.7:0/1622504674 shutdown_connections 2024-04-25T11:14:06.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.494+0000 7f533ce00640 1 -- 172.21.15.7:0/1622504674 wait complete. 2024-04-25T11:14:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.497+0000 7f533ce00640 1 Processor -- start 2024-04-25T11:14:06.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.497+0000 7f533ce00640 1 -- start start 2024-04-25T11:14:06.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.499+0000 7f533ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53380ce8d0 0x7f5338155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.499+0000 7f533ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f533815c2e0 0x7f53381484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.499+0000 7f533ce00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5338145b50 0x7f5338145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.499+0000 7f533ce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5338070ff0 con 0x7f53380ce8d0 2024-04-25T11:14:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.499+0000 7f533ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5338070e70 con 0x7f533815c2e0 2024-04-25T11:14:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.499+0000 7f533ce00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f533806eea0 con 0x7f5338145b50 2024-04-25T11:14:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.500+0000 7f5337400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53380ce8d0 0x7f5338155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.500+0000 7f5337400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53380ce8d0 0x7f5338155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38020/0 (socket says 172.21.15.7:38020) 2024-04-25T11:14:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.500+0000 7f5337400640 1 -- 172.21.15.7:0/3541326826 learned_addr learned my addr 172.21.15.7:0/3541326826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.500+0000 7f5337e00640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5338145b50 0x7f5338145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:06.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.500+0000 7f5336a00640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f533815c2e0 0x7f53381484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.500+0000 7f5337400640 1 -- 172.21.15.7:0/3541326826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5338145b50 msgr2=0x7f5338145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.500+0000 7f5337400640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5338145b50 0x7f5338145f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.500+0000 7f5337400640 1 -- 172.21.15.7:0/3541326826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f533815c2e0 msgr2=0x7f53381484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.500+0000 7f5337400640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f533815c2e0 0x7f53381484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.500+0000 7f5337400640 1 -- 172.21.15.7:0/3541326826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5338154700 con 0x7f53380ce8d0 2024-04-25T11:14:06.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.500+0000 7f5337400640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53380ce8d0 0x7f5338155d40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f532800ed50 tx=0x7f532800d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.501+0000 7f532f400640 1 -- 172.21.15.7:0/3541326826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328010440 con 0x7f53380ce8d0 2024-04-25T11:14:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.501+0000 7f532f400640 1 -- 172.21.15.7:0/3541326826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53280105e0 con 0x7f53380ce8d0 2024-04-25T11:14:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.501+0000 7f533ce00640 1 -- 172.21.15.7:0/3541326826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5338159510 con 0x7f53380ce8d0 2024-04-25T11:14:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.501+0000 7f532f400640 1 -- 172.21.15.7:0/3541326826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53280109b0 con 0x7f53380ce8d0 2024-04-25T11:14:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.501+0000 7f533ce00640 1 -- 172.21.15.7:0/3541326826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5338154ee0 con 0x7f53380ce8d0 2024-04-25T11:14:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.502+0000 7f533ce00640 1 -- 172.21.15.7:0/3541326826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5338050740 con 0x7f53380ce8d0 2024-04-25T11:14:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.504+0000 7f532f400640 1 -- 172.21.15.7:0/3541326826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5328010b50 con 0x7f53380ce8d0 2024-04-25T11:14:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.505+0000 7f532f400640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53180744e0 0x7f53180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.505+0000 7f532f400640 1 -- 172.21.15.7:0/3541326826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6245+0+0 (secure 0 0 0) 0x7f5328095dd0 con 0x7f53380ce8d0 2024-04-25T11:14:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.505+0000 7f5336a00640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53180744e0 0x7f53180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.506+0000 7f5336a00640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53180744e0 0x7f53180769f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5324007fd0 tx=0x7f5324009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:06.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.512+0000 7f532f400640 1 -- 172.21.15.7:0/3541326826 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5328014030 con 0x7f53380ce8d0 2024-04-25T11:14:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:06.765+0000 7f533ce00640 1 -- 172.21.15.7:0/3541326826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f533806cc80 con 0x7f53380ce8d0 2024-04-25T11:14:07.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.186+0000 7f532f400640 1 -- 172.21.15.7:0/3541326826 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f5328060330 con 0x7f53380ce8d0 2024-04-25T11:14:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 -- 172.21.15.7:0/3541326826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53180744e0 msgr2=0x7f53180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53180744e0 0x7f53180769f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5324007fd0 tx=0x7f5324009040 comp rx=0 tx=0).stop 2024-04-25T11:14:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 -- 172.21.15.7:0/3541326826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53380ce8d0 msgr2=0x7f5338155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53380ce8d0 0x7f5338155d40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f532800ed50 tx=0x7f532800d720 comp rx=0 tx=0).stop 2024-04-25T11:14:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 -- 172.21.15.7:0/3541326826 shutdown_connections 2024-04-25T11:14:07.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53180744e0 0x7f53180769f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:07.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5338145b50 0x7f5338145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:07.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f533815c2e0 0x7f53381484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:07.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 --2- 172.21.15.7:0/3541326826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53380ce8d0 0x7f5338155d40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:07.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 -- 172.21.15.7:0/3541326826 >> 172.21.15.7:0/3541326826 conn(0x7f53380740f0 msgr2=0x7f53380b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:07.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 -- 172.21.15.7:0/3541326826 shutdown_connections 2024-04-25T11:14:07.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.190+0000 7f533ce00640 1 -- 172.21.15.7:0/3541326826 wait complete. 2024-04-25T11:14:07.224 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:14:07.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.515+0000 7fa9a4c00640 1 Processor -- start 2024-04-25T11:14:07.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.515+0000 7fa9a4c00640 1 -- start start 2024-04-25T11:14:07.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.518+0000 7fa9a4c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0162d30 0x7fa9a0169870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.518+0000 7fa9a4c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a00cf800 con 0x7fa9a008cbb0 2024-04-25T11:14:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.518+0000 7fa9a4c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a009d170 con 0x7fa9a00b8eb0 2024-04-25T11:14:07.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.518+0000 7fa9a4c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a00b43b0 con 0x7fa9a0162d30 2024-04-25T11:14:07.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.518+0000 7fa99d400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa9a008cbb0 0x7fa9a0162620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41270/0 (socket says 172.21.15.7:41270) 2024-04-25T11:14:07.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.518+0000 7fa99d400640 1 -- 172.21.15.7:0/3130558902 learned_addr learned my addr 172.21.15.7:0/3130558902 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:07.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.518+0000 7fa99e800640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0162d30 0x7fa9a0169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:07.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.518+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1872025664 0 0) 0x7fa9a00cf800 con 0x7fa9a008cbb0 2024-04-25T11:14:07.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa984003520 con 0x7fa9a008cbb0 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1408574489 0 0) 0x7fa984003520 con 0x7fa9a008cbb0 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0162d30 msgr2=0x7fa9a0169870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0162d30 0x7fa9a0169870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 >> v1:172.21.15.87:6789/0 conn(0x7fa9a00b8eb0 legacy=0x7fa9a009bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a00b4e00 con 0x7fa9a008cbb0 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fa98c001670 con 0x7fa9a008cbb0 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 >> v1:172.21.15.7:6789/0 conn(0x7fa9a008cbb0 legacy=0x7fa9a0162620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9840039e0 0x7fa984003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa984004800 0x7fa984004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0162d30 0x7fa9840052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a00b43b0 con 0x7fa984004800 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa98400bb50 con 0x7fa9840039e0 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.519+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa98400bd50 con 0x7fa9a0162d30 2024-04-25T11:14:07.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.520+0000 7fa99d400640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa984004800 0x7fa984004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.520+0000 7fa99de00640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9840039e0 0x7fa984003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.520+0000 7fa99e800640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0162d30 0x7fa9840052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.520+0000 7fa99d400640 1 -- 172.21.15.7:0/3130558902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0162d30 msgr2=0x7fa9840052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.520+0000 7fa99d400640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0162d30 0x7fa9840052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.520+0000 7fa99d400640 1 -- 172.21.15.7:0/3130558902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9840039e0 msgr2=0x7fa984003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.520+0000 7fa99d400640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9840039e0 0x7fa984003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:07.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.520+0000 7fa99d400640 1 -- 172.21.15.7:0/3130558902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa984003520 con 0x7fa984004800 2024-04-25T11:14:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.521+0000 7fa99d400640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa984004800 0x7fa984004c10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00caa00 tx=0x7fa984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719026659752c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.522+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c007c40 con 0x7fa984004800 2024-04-25T11:14:07.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.522+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98c005af0 con 0x7fa984004800 2024-04-25T11:14:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.522+0000 7fa99ca00640 1 -- 172.21.15.7:0/3130558902 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c007070 con 0x7fa984004800 2024-04-25T11:14:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.522+0000 7fa9a4c00640 1 -- 172.21.15.7:0/3130558902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa984004800 msgr2=0x7fa984004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.522+0000 7fa9a4c00640 1 --2- 172.21.15.7:0/3130558902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa984004800 0x7fa984004c10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00caa00 tx=0x7fa984005830 comp rx=0 tx=0).stop 2024-04-25T11:14:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.522+0000 7fa99e800640 1 -- 172.21.15.7:0/3130558902 reap_dead start 2024-04-25T11:14:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.523+0000 7fa9a4c00640 1 -- 172.21.15.7:0/3130558902 shutdown_connections 2024-04-25T11:14:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.523+0000 7fa9a4c00640 1 -- 172.21.15.7:0/3130558902 >> 172.21.15.7:0/3130558902 conn(0x7fa9a00b5470 msgr2=0x7fa9a00b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.523+0000 7fa9a4c00640 1 -- 172.21.15.7:0/3130558902 shutdown_connections 2024-04-25T11:14:07.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.523+0000 7fa9a4c00640 1 -- 172.21.15.7:0/3130558902 wait complete. 2024-04-25T11:14:07.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.525+0000 7fa9a4c00640 1 Processor -- start 2024-04-25T11:14:07.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.525+0000 7fa9a4c00640 1 -- start start 2024-04-25T11:14:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.527+0000 7fa9a4c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0159330 0x7fa9a00b8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.527+0000 7fa9a4c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9a00b88a0 0x7fa9a015d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.527+0000 7fa9a4c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9a015da70 0x7fa9a00b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.527+0000 7fa9a4c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a00cf2e0 con 0x7fa9a00b88a0 2024-04-25T11:14:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.527+0000 7fa9a4c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a009cf20 con 0x7fa9a015da70 2024-04-25T11:14:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.527+0000 7fa9a4c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a009c440 con 0x7fa9a0159330 2024-04-25T11:14:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.527+0000 7fa99d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9a00b88a0 0x7fa9a015d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:07.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.527+0000 7fa99d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9a00b88a0 0x7fa9a015d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38032/0 (socket says 172.21.15.7:38032) 2024-04-25T11:14:07.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.527+0000 7fa99d400640 1 -- 172.21.15.7:0/1681705692 learned_addr learned my addr 172.21.15.7:0/1681705692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.527+0000 7fa99de00640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0159330 0x7fa9a00b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.528+0000 7fa99e800640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9a015da70 0x7fa9a00b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.528+0000 7fa99d400640 1 -- 172.21.15.7:0/1681705692 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0159330 msgr2=0x7fa9a00b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.528+0000 7fa99d400640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0159330 0x7fa9a00b8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.528+0000 7fa99d400640 1 -- 172.21.15.7:0/1681705692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9a015da70 msgr2=0x7fa9a00b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.528+0000 7fa99d400640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9a015da70 0x7fa9a00b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.528+0000 7fa99d400640 1 -- 172.21.15.7:0/1681705692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a00b8cb0 con 0x7fa9a00b88a0 2024-04-25T11:14:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.528+0000 7fa99d400640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9a00b88a0 0x7fa9a015d530 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa98c02acf0 tx=0x7fa9a00b4fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.528+0000 7fa996000640 1 -- 172.21.15.7:0/1681705692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c043070 con 0x7fa9a00b88a0 2024-04-25T11:14:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.529+0000 7fa996000640 1 -- 172.21.15.7:0/1681705692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98c038040 con 0x7fa9a00b88a0 2024-04-25T11:14:07.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.529+0000 7fa996000640 1 -- 172.21.15.7:0/1681705692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0072b0 con 0x7fa9a00b88a0 2024-04-25T11:14:07.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.529+0000 7fa9a4c00640 1 -- 172.21.15.7:0/1681705692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a015de80 con 0x7fa9a00b88a0 2024-04-25T11:14:07.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.529+0000 7fa9a4c00640 1 -- 172.21.15.7:0/1681705692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a00cae80 con 0x7fa9a00b88a0 2024-04-25T11:14:07.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.530+0000 7fa9a4c00640 1 -- 172.21.15.7:0/1681705692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a006a100 con 0x7fa9a00b88a0 2024-04-25T11:14:07.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.531+0000 7fa996000640 1 -- 172.21.15.7:0/1681705692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa98c007450 con 0x7fa9a00b88a0 2024-04-25T11:14:07.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.532+0000 7fa996000640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9800744b0 0x7fa9800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:07.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.532+0000 7fa996000640 1 -- 172.21.15.7:0/1681705692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6245+0+0 (secure 0 0 0) 0x7fa98c0b4fe0 con 0x7fa9a00b88a0 2024-04-25T11:14:07.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.533+0000 7fa99de00640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9800744b0 0x7fa9800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:07.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.533+0000 7fa99de00640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9800744b0 0x7fa9800769c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00b0f80 tx=0x7fa990009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:07.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.537+0000 7fa996000640 1 -- 172.21.15.7:0/1681705692 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa9a015de80 con 0x7fa9a00b88a0 2024-04-25T11:14:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:07.749+0000 7fa9a4c00640 1 -- 172.21.15.7:0/1681705692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa9a009c7f0 con 0x7fa9a00b88a0 2024-04-25T11:14:08.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.190+0000 7fa996000640 1 -- 172.21.15.7:0/1681705692 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) ==== 181+0+0 (secure 0 0 0) 0x7fa98c001630 con 0x7fa9a00b88a0 2024-04-25T11:14:08.191 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:14:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.194+0000 7fa9a4c00640 1 -- 172.21.15.7:0/1681705692 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9800744b0 msgr2=0x7fa9800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.194+0000 7fa9a4c00640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9800744b0 0x7fa9800769c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00b0f80 tx=0x7fa990009040 comp rx=0 tx=0).stop 2024-04-25T11:14:08.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.194+0000 7fa9a4c00640 1 -- 172.21.15.7:0/1681705692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9a00b88a0 msgr2=0x7fa9a015d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.194+0000 7fa9a4c00640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9a00b88a0 0x7fa9a015d530 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa98c02acf0 tx=0x7fa9a00b4fd0 comp rx=0 tx=0).stop 2024-04-25T11:14:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.194+0000 7fa9a4c00640 1 -- 172.21.15.7:0/1681705692 shutdown_connections 2024-04-25T11:14:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.194+0000 7fa9a4c00640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9800744b0 0x7fa9800769c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.194+0000 7fa9a4c00640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9a015da70 0x7fa9a00b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.194+0000 7fa9a4c00640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9a00b88a0 0x7fa9a015d530 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.194+0000 7fa9a4c00640 1 --2- 172.21.15.7:0/1681705692 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9a0159330 0x7fa9a00b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:08.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.194+0000 7fa9a4c00640 1 -- 172.21.15.7:0/1681705692 >> 172.21.15.7:0/1681705692 conn(0x7fa9a00b5470 msgr2=0x7fa9a006eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:08.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.195+0000 7fa9a4c00640 1 -- 172.21.15.7:0/1681705692 shutdown_connections 2024-04-25T11:14:08.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.196+0000 7fa9a4c00640 1 -- 172.21.15.7:0/1681705692 wait complete. 2024-04-25T11:14:08.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:14:08.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.501+0000 7f2d4ac00640 1 Processor -- start 2024-04-25T11:14:08.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.501+0000 7f2d4ac00640 1 -- start start 2024-04-25T11:14:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.503+0000 7f2d4ac00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2d440b2a50 0x7f2d440b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.503+0000 7f2d4ac00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d4406ebc0 con 0x7f2d4406dd90 2024-04-25T11:14:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.504+0000 7f2d4ac00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d440d2db0 con 0x7f2d4408cad0 2024-04-25T11:14:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.504+0000 7f2d4ac00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2d440d0cf0 con 0x7f2d440b2a50 2024-04-25T11:14:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.504+0000 7f2d4a200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2d4406dd90 0x7f2d4416a440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41274/0 (socket says 172.21.15.7:41274) 2024-04-25T11:14:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.504+0000 7f2d4a200640 1 -- 172.21.15.7:0/599186164 learned_addr learned my addr 172.21.15.7:0/599186164 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.504+0000 7f2d49800640 1 --2- 172.21.15.7:0/599186164 >> v2:172.21.15.87:3300/0 conn(0x7f2d440b2a50 0x7f2d440b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.504+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2428532740 0 0) 0x7f2d4406ebc0 con 0x7f2d4406dd90 2024-04-25T11:14:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d2c003520 con 0x7f2d4406dd90 2024-04-25T11:14:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1093948019 0 0) 0x7f2d2c003520 con 0x7f2d4406dd90 2024-04-25T11:14:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 >> v2:172.21.15.87:3300/0 conn(0x7f2d440b2a50 msgr2=0x7f2d440b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 --2- 172.21.15.7:0/599186164 >> v2:172.21.15.87:3300/0 conn(0x7f2d440b2a50 0x7f2d440b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 >> v1:172.21.15.87:6789/0 conn(0x7f2d4408cad0 legacy=0x7f2d4406d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d440d0f40 con 0x7f2d4406dd90 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2d3c0019f0 con 0x7f2d4406dd90 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 >> v1:172.21.15.7:6789/0 conn(0x7f2d4406dd90 legacy=0x7f2d4416a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 --2- 172.21.15.7:0/599186164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2c003a00 0x7f2d2c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 --2- 172.21.15.7:0/599186164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2c004800 0x7f2d2c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 --2- 172.21.15.7:0/599186164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d2c005270 0x7f2d2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d440d0cf0 con 0x7f2d2c005270 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c00c190 con 0x7f2d2c004800 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.505+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2c00c390 con 0x7f2d2c003a00 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.506+0000 7f2d4a200640 1 --2- 172.21.15.7:0/599186164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d2c005270 0x7f2d2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.506+0000 7f2d49800640 1 --2- 172.21.15.7:0/599186164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2c003a00 0x7f2d2c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.506+0000 7f2d48e00640 1 --2- 172.21.15.7:0/599186164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2c004800 0x7f2d2c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.506+0000 7f2d4a200640 1 -- 172.21.15.7:0/599186164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2c003a00 msgr2=0x7f2d2c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.506+0000 7f2d4a200640 1 --2- 172.21.15.7:0/599186164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2c003a00 0x7f2d2c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.506+0000 7f2d4a200640 1 -- 172.21.15.7:0/599186164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2c004800 msgr2=0x7f2d2c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.506+0000 7f2d4a200640 1 --2- 172.21.15.7:0/599186164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2c004800 0x7f2d2c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.506+0000 7f2d4a200640 1 -- 172.21.15.7:0/599186164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c003520 con 0x7f2d2c005270 2024-04-25T11:14:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.507+0000 7f2d4a200640 1 -- 172.21.15.7:0/599186164 reap_dead start 2024-04-25T11:14:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.507+0000 7f2d4a200640 1 --2- 172.21.15.7:0/599186164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d2c005270 0x7f2d2c00b9c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00a7b0 tx=0x7f2d2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d994b866ff5d7272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.507+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c00a3d0 con 0x7f2d2c005270 2024-04-25T11:14:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.508+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d3c02f070 con 0x7f2d2c005270 2024-04-25T11:14:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.508+0000 7f2d43e00640 1 -- 172.21.15.7:0/599186164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c02bc40 con 0x7f2d2c005270 2024-04-25T11:14:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.508+0000 7f2d4ac00640 1 -- 172.21.15.7:0/599186164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d2c005270 msgr2=0x7f2d2c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.508+0000 7f2d4ac00640 1 --2- 172.21.15.7:0/599186164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d2c005270 0x7f2d2c00b9c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00a7b0 tx=0x7f2d2c005830 comp rx=0 tx=0).stop 2024-04-25T11:14:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.508+0000 7f2d4ac00640 1 -- 172.21.15.7:0/599186164 shutdown_connections 2024-04-25T11:14:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.508+0000 7f2d4ac00640 1 --2- 172.21.15.7:0/599186164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d2c005270 0x7f2d2c00b9c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:08.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.508+0000 7f2d4ac00640 1 -- 172.21.15.7:0/599186164 >> 172.21.15.7:0/599186164 conn(0x7f2d440d01d0 msgr2=0x7f2d440d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:08.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.508+0000 7f2d4ac00640 1 -- 172.21.15.7:0/599186164 shutdown_connections 2024-04-25T11:14:08.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.509+0000 7f2d4ac00640 1 -- 172.21.15.7:0/599186164 wait complete. 2024-04-25T11:14:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.511+0000 7f2d4ac00640 1 Processor -- start 2024-04-25T11:14:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.511+0000 7f2d4ac00640 1 -- start start 2024-04-25T11:14:08.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.513+0000 7f2d4ac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2c005270 0x7f2d441df490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:08.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.513+0000 7f2d4ac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d441dfce0 0x7f2d440b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:08.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.513+0000 7f2d4ac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d441e00f0 0x7f2d440b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.513+0000 7f2d4ac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4409d180 con 0x7f2d441e00f0 2024-04-25T11:14:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.513+0000 7f2d4ac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4409d000 con 0x7f2d2c005270 2024-04-25T11:14:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.513+0000 7f2d4ac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2d440d3310 con 0x7f2d441dfce0 2024-04-25T11:14:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.513+0000 7f2d4a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d441e00f0 0x7f2d440b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.513+0000 7f2d4a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d441e00f0 0x7f2d440b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38042/0 (socket says 172.21.15.7:38042) 2024-04-25T11:14:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.513+0000 7f2d4a200640 1 -- 172.21.15.7:0/1784845908 learned_addr learned my addr 172.21.15.7:0/1784845908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.514+0000 7f2d49800640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2c005270 0x7f2d441df490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.514+0000 7f2d48e00640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d441dfce0 0x7f2d440b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:08.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.514+0000 7f2d4a200640 1 -- 172.21.15.7:0/1784845908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d441dfce0 msgr2=0x7f2d440b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:08.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.514+0000 7f2d4a200640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d441dfce0 0x7f2d440b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:08.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.514+0000 7f2d4a200640 1 -- 172.21.15.7:0/1784845908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2c005270 msgr2=0x7f2d441df490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:08.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.514+0000 7f2d4a200640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2c005270 0x7f2d441df490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:08.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.514+0000 7f2d4a200640 1 -- 172.21.15.7:0/1784845908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d440b6790 con 0x7f2d441e00f0 2024-04-25T11:14:08.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.514+0000 7f2d4a200640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d441e00f0 0x7f2d440b19d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00a8a0 tx=0x7f2d3c02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:08.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.515+0000 7f2d41600640 1 -- 172.21.15.7:0/1784845908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c02bbf0 con 0x7f2d441e00f0 2024-04-25T11:14:08.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.515+0000 7f2d41600640 1 -- 172.21.15.7:0/1784845908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d3c02bd90 con 0x7f2d441e00f0 2024-04-25T11:14:08.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.515+0000 7f2d41600640 1 -- 172.21.15.7:0/1784845908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c034670 con 0x7f2d441e00f0 2024-04-25T11:14:08.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.515+0000 7f2d4ac00640 1 -- 172.21.15.7:0/1784845908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d440b6a70 con 0x7f2d441e00f0 2024-04-25T11:14:08.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.515+0000 7f2d4ac00640 1 -- 172.21.15.7:0/1784845908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d440b7030 con 0x7f2d441e00f0 2024-04-25T11:14:08.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.516+0000 7f2d4ac00640 1 -- 172.21.15.7:0/1784845908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d44050740 con 0x7f2d441e00f0 2024-04-25T11:14:08.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.517+0000 7f2d41600640 1 -- 172.21.15.7:0/1784845908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d3c034810 con 0x7f2d441e00f0 2024-04-25T11:14:08.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.518+0000 7f2d41600640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d240744e0 0x7f2d240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:08.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.518+0000 7f2d41600640 1 -- 172.21.15.7:0/1784845908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5807+0+0 (secure 0 0 0) 0x7f2d3c0b5490 con 0x7f2d441e00f0 2024-04-25T11:14:08.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.519+0000 7f2d49800640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d240744e0 0x7f2d240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:08.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.519+0000 7f2d49800640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d240744e0 0x7f2d240769f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2d440713b0 tx=0x7f2d38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:08.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.523+0000 7f2d41600640 1 -- 172.21.15.7:0/1784845908 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d3c07fbb0 con 0x7f2d441e00f0 2024-04-25T11:14:08.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:08.730+0000 7f2d4ac00640 1 -- 172.21.15.7:0/1784845908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2d440b64e0 con 0x7f2d441e00f0 2024-04-25T11:14:09.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.206+0000 7f2d41600640 1 -- 172.21.15.7:0/1784845908 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) ==== 169+0+0 (secure 0 0 0) 0x7f2d3c07f4b0 con 0x7f2d441e00f0 2024-04-25T11:14:09.207 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-25T11:14:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.211+0000 7f2d4ac00640 1 -- 172.21.15.7:0/1784845908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d240744e0 msgr2=0x7f2d240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.211+0000 7f2d4ac00640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d240744e0 0x7f2d240769f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2d440713b0 tx=0x7f2d38008040 comp rx=0 tx=0).stop 2024-04-25T11:14:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.211+0000 7f2d4ac00640 1 -- 172.21.15.7:0/1784845908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d441e00f0 msgr2=0x7f2d440b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.211+0000 7f2d4ac00640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d441e00f0 0x7f2d440b19d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00a8a0 tx=0x7f2d3c02d070 comp rx=0 tx=0).stop 2024-04-25T11:14:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.212+0000 7f2d4ac00640 1 -- 172.21.15.7:0/1784845908 shutdown_connections 2024-04-25T11:14:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.212+0000 7f2d4ac00640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d240744e0 0x7f2d240769f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.212+0000 7f2d4ac00640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d441e00f0 0x7f2d440b19d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.212+0000 7f2d4ac00640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d441dfce0 0x7f2d440b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.212+0000 7f2d4ac00640 1 --2- 172.21.15.7:0/1784845908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2c005270 0x7f2d441df490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.212+0000 7f2d4ac00640 1 -- 172.21.15.7:0/1784845908 >> 172.21.15.7:0/1784845908 conn(0x7f2d440d01d0 msgr2=0x7f2d440b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:09.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.212+0000 7f2d4ac00640 1 -- 172.21.15.7:0/1784845908 shutdown_connections 2024-04-25T11:14:09.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.212+0000 7f2d4ac00640 1 -- 172.21.15.7:0/1784845908 wait complete. 2024-04-25T11:14:09.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:14:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.535+0000 7f8a14a00640 1 Processor -- start 2024-04-25T11:14:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.535+0000 7f8a14a00640 1 -- start start 2024-04-25T11:14:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.537+0000 7f8a14a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1006d860 0x7f8a1006dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.537+0000 7f8a14a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a101645e0 0x7f8a101649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.537+0000 7f8a14a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a100b2ce0 con 0x7f8a10164f30 2024-04-25T11:14:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.537+0000 7f8a14a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8a100d0f80 con 0x7f8a101645e0 2024-04-25T11:14:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.537+0000 7f8a14a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1009cb10 con 0x7f8a1006d860 2024-04-25T11:14:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.538+0000 7f8a0f200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8a10164f30 0x7f8a100cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41280/0 (socket says 172.21.15.7:41280) 2024-04-25T11:14:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.538+0000 7f8a0f200640 1 -- 172.21.15.7:0/997900612 learned_addr learned my addr 172.21.15.7:0/997900612 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.538+0000 7f8a0de00640 1 --2- 172.21.15.7:0/997900612 >> v2:172.21.15.7:3300/0 conn(0x7f8a101645e0 0x7f8a101649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.538+0000 7f8a0e800640 1 --2- 172.21.15.7:0/997900612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1006d860 0x7f8a1006dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.538+0000 7f8a0d400640 1 -- 172.21.15.7:0/997900612 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 440468898 0 0) 0x7f8a100b2ce0 con 0x7f8a10164f30 2024-04-25T11:14:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.538+0000 7f8a0d400640 1 -- 172.21.15.7:0/997900612 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89f4003520 con 0x7f8a10164f30 2024-04-25T11:14:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.538+0000 7f8a0de00640 1 -- 172.21.15.7:0/997900612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1006d860 msgr2=0x7f8a1006dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.538+0000 7f8a0de00640 1 --2- 172.21.15.7:0/997900612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1006d860 0x7f8a1006dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.538+0000 7f8a0de00640 1 -- 172.21.15.7:0/997900612 >> v1:172.21.15.7:6789/0 conn(0x7f8a10164f30 legacy=0x7f8a100cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.538+0000 7f8a0de00640 1 -- 172.21.15.7:0/997900612 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1009d030 con 0x7f8a101645e0 2024-04-25T11:14:09.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.539+0000 7f8a0de00640 1 --2- 172.21.15.7:0/997900612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a101645e0 0x7f8a101649f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0097a0 tx=0x7f89fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8154ffb2616c4fa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:09.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.540+0000 7f8a0d400640 1 -- 172.21.15.7:0/997900612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc037070 con 0x7f8a101645e0 2024-04-25T11:14:09.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.540+0000 7f8a0d400640 1 -- 172.21.15.7:0/997900612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89fc032440 con 0x7f8a101645e0 2024-04-25T11:14:09.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.540+0000 7f8a0d400640 1 -- 172.21.15.7:0/997900612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc032810 con 0x7f8a101645e0 2024-04-25T11:14:09.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.540+0000 7f8a14a00640 1 -- 172.21.15.7:0/997900612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a101645e0 msgr2=0x7f8a101649f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:09.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.540+0000 7f8a14a00640 1 --2- 172.21.15.7:0/997900612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a101645e0 0x7f8a101649f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0097a0 tx=0x7f89fc02a180 comp rx=0 tx=0).stop 2024-04-25T11:14:09.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.541+0000 7f8a14a00640 1 -- 172.21.15.7:0/997900612 shutdown_connections 2024-04-25T11:14:09.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.541+0000 7f8a14a00640 1 --2- 172.21.15.7:0/997900612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a101645e0 0x7f8a101649f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.541+0000 7f8a14a00640 1 --2- 172.21.15.7:0/997900612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1006d860 0x7f8a1006dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.541+0000 7f8a14a00640 1 -- 172.21.15.7:0/997900612 >> 172.21.15.7:0/997900612 conn(0x7f8a100d02c0 msgr2=0x7f8a100d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:09.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.541+0000 7f8a14a00640 1 -- 172.21.15.7:0/997900612 shutdown_connections 2024-04-25T11:14:09.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.541+0000 7f8a14a00640 1 -- 172.21.15.7:0/997900612 wait complete. 2024-04-25T11:14:09.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.543+0000 7f8a14a00640 1 Processor -- start 2024-04-25T11:14:09.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.543+0000 7f8a14a00640 1 -- start start 2024-04-25T11:14:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.545+0000 7f8a14a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1006d860 0x7f8a1015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.545+0000 7f8a14a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a101645e0 0x7f8a10150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.545+0000 7f8a14a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a10164f30 0x7f8a100b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.545+0000 7f8a14a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a100b8c90 con 0x7f8a10164f30 2024-04-25T11:14:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.545+0000 7f8a14a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8a10165340 con 0x7f8a101645e0 2024-04-25T11:14:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.545+0000 7f8a14a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8a100b8e10 con 0x7f8a1006d860 2024-04-25T11:14:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.545+0000 7f8a0f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a10164f30 0x7f8a100b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.545+0000 7f8a0f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a10164f30 0x7f8a100b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38068/0 (socket says 172.21.15.7:38068) 2024-04-25T11:14:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.545+0000 7f8a0f200640 1 -- 172.21.15.7:0/2490315639 learned_addr learned my addr 172.21.15.7:0/2490315639 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.545+0000 7f8a0e800640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1006d860 0x7f8a1015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.545+0000 7f8a0de00640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a101645e0 0x7f8a10150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.546+0000 7f8a0f200640 1 -- 172.21.15.7:0/2490315639 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1006d860 msgr2=0x7f8a1015f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.546+0000 7f8a0f200640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1006d860 0x7f8a1015f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.546+0000 7f8a0f200640 1 -- 172.21.15.7:0/2490315639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a101645e0 msgr2=0x7f8a10150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.546+0000 7f8a0f200640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a101645e0 0x7f8a10150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.546+0000 7f8a0f200640 1 -- 172.21.15.7:0/2490315639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1015c640 con 0x7f8a10164f30 2024-04-25T11:14:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.546+0000 7f8a0f200640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a10164f30 0x7f8a100b1500 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800caa0 tx=0x7f8a0800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.546+0000 7f8a06a00640 1 -- 172.21.15.7:0/2490315639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08017070 con 0x7f8a10164f30 2024-04-25T11:14:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.546+0000 7f8a06a00640 1 -- 172.21.15.7:0/2490315639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0800aba0 con 0x7f8a10164f30 2024-04-25T11:14:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.546+0000 7f8a06a00640 1 -- 172.21.15.7:0/2490315639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08012400 con 0x7f8a10164f30 2024-04-25T11:14:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.546+0000 7f8a14a00640 1 -- 172.21.15.7:0/2490315639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1015c8e0 con 0x7f8a10164f30 2024-04-25T11:14:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.547+0000 7f8a14a00640 1 -- 172.21.15.7:0/2490315639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1015cea0 con 0x7f8a10164f30 2024-04-25T11:14:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.548+0000 7f8a14a00640 1 -- 172.21.15.7:0/2490315639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a10050740 con 0x7f8a10164f30 2024-04-25T11:14:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.549+0000 7f8a06a00640 1 -- 172.21.15.7:0/2490315639 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8a0801a070 con 0x7f8a10164f30 2024-04-25T11:14:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.550+0000 7f8a06a00640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f00744b0 0x7f89f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.550+0000 7f8a06a00640 1 -- 172.21.15.7:0/2490315639 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7f8a08094ed0 con 0x7f8a10164f30 2024-04-25T11:14:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.551+0000 7f8a0e800640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f00744b0 0x7f89f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.551+0000 7f8a0e800640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f00744b0 0x7f89f00769c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8a00006fd0 tx=0x7f8a00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.555+0000 7f8a06a00640 1 -- 172.21.15.7:0/2490315639 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a0805f7c0 con 0x7f8a10164f30 2024-04-25T11:14:09.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.762+0000 7f8a14a00640 1 -- 172.21.15.7:0/2490315639 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f8a100b47c0 con 0x7f8a10164f30 2024-04-25T11:14:09.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.768+0000 7f8a06a00640 1 -- 172.21.15.7:0/2490315639 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f8a0805f0c0 con 0x7f8a10164f30 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 -- 172.21.15.7:0/2490315639 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f00744b0 msgr2=0x7f89f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f00744b0 0x7f89f00769c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8a00006fd0 tx=0x7f8a00008040 comp rx=0 tx=0).stop 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 -- 172.21.15.7:0/2490315639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a10164f30 msgr2=0x7f8a100b1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a10164f30 0x7f8a100b1500 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800caa0 tx=0x7f8a0800cf60 comp rx=0 tx=0).stop 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 -- 172.21.15.7:0/2490315639 shutdown_connections 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f00744b0 0x7f89f00769c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a10164f30 0x7f8a100b1500 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a101645e0 0x7f8a10150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 --2- 172.21.15.7:0/2490315639 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1006d860 0x7f8a1015f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 -- 172.21.15.7:0/2490315639 >> 172.21.15.7:0/2490315639 conn(0x7f8a100d02c0 msgr2=0x7f8a100d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 -- 172.21.15.7:0/2490315639 shutdown_connections 2024-04-25T11:14:09.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:09.771+0000 7f8a14a00640 1 -- 172.21.15.7:0/2490315639 wait complete. 2024-04-25T11:14:09.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid' 2024-04-25T11:14:10.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.083+0000 7f06a9a00640 1 Processor -- start 2024-04-25T11:14:10.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.083+0000 7f06a9a00640 1 -- start start 2024-04-25T11:14:10.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.085+0000 7f06a9a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f06a415e2f0 0x7f06a4161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:10.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.086+0000 7f06a9a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a415e700 con 0x7f06a415baa0 2024-04-25T11:14:10.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.086+0000 7f06a9a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a415e980 con 0x7f06a4162420 2024-04-25T11:14:10.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.086+0000 7f06a9a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f06a4162830 con 0x7f06a415e2f0 2024-04-25T11:14:10.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.086+0000 7f06a3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f06a415baa0 0x7f06a4070640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43466/0 (socket says 172.21.15.7:43466) 2024-04-25T11:14:10.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.086+0000 7f06a3e00640 1 -- 172.21.15.7:0/391422206 learned_addr learned my addr 172.21.15.7:0/391422206 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:10.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.086+0000 7f06a3400640 1 --2- 172.21.15.7:0/391422206 >> v2:172.21.15.87:3300/0 conn(0x7f06a415e2f0 0x7f06a4161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:10.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.086+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1755463642 0 0) 0x7f0694001110 con 0x7f06a415baa0 2024-04-25T11:14:10.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0688003520 con 0x7f06a415baa0 2024-04-25T11:14:10.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3331452273 0 0) 0x7f0688003520 con 0x7f06a415baa0 2024-04-25T11:14:10.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 >> v2:172.21.15.87:3300/0 conn(0x7f06a415e2f0 msgr2=0x7f06a4161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:10.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 --2- 172.21.15.7:0/391422206 >> v2:172.21.15.87:3300/0 conn(0x7f06a415e2f0 0x7f06a4161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:10.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 >> v1:172.21.15.87:6789/0 conn(0x7f06a4162420 legacy=0x7f06a406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a415ec60 con 0x7f06a415baa0 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0694001840 con 0x7f06a415baa0 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 >> v1:172.21.15.7:6789/0 conn(0x7f06a415baa0 legacy=0x7f06a4070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 --2- 172.21.15.7:0/391422206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 0x7f0688003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 --2- 172.21.15.7:0/391422206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0688004800 0x7f0688004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 --2- 172.21.15.7:0/391422206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0688005270 0x7f068800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06a4162830 con 0x7f0688003a00 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f068800c190 con 0x7f0688005270 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f068800c390 con 0x7f0688004800 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.087+0000 7f06a3e00640 1 --2- 172.21.15.7:0/391422206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 0x7f0688003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.088+0000 7f06a3400640 1 --2- 172.21.15.7:0/391422206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0688004800 0x7f0688004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.088+0000 7f06a9000640 1 --2- 172.21.15.7:0/391422206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0688005270 0x7f068800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.088+0000 7f06a3e00640 1 -- 172.21.15.7:0/391422206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0688004800 msgr2=0x7f0688004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.088+0000 7f06a3e00640 1 --2- 172.21.15.7:0/391422206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0688004800 0x7f0688004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:10.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.088+0000 7f06a3e00640 1 -- 172.21.15.7:0/391422206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0688005270 msgr2=0x7f068800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:10.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.088+0000 7f06a3e00640 1 --2- 172.21.15.7:0/391422206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0688005270 0x7f068800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:10.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.088+0000 7f06a9000640 1 -- 172.21.15.7:0/391422206 reap_dead start 2024-04-25T11:14:10.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.088+0000 7f06a3e00640 1 -- 172.21.15.7:0/391422206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0688003520 con 0x7f0688003a00 2024-04-25T11:14:10.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.089+0000 7f06a3e00640 1 --2- 172.21.15.7:0/391422206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 0x7f0688003e70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f069400c320 tx=0x7f0688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37273cdd057d6ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:10.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.089+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694007e10 con 0x7f0688003a00 2024-04-25T11:14:10.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.089+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0694005ad0 con 0x7f0688003a00 2024-04-25T11:14:10.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.089+0000 7f06a2a00640 1 -- 172.21.15.7:0/391422206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694007070 con 0x7f0688003a00 2024-04-25T11:14:10.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.090+0000 7f06a9a00640 1 -- 172.21.15.7:0/391422206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 msgr2=0x7f0688003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:10.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.090+0000 7f06a9a00640 1 --2- 172.21.15.7:0/391422206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 0x7f0688003e70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f069400c320 tx=0x7f0688005830 comp rx=0 tx=0).stop 2024-04-25T11:14:10.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.090+0000 7f06a9a00640 1 -- 172.21.15.7:0/391422206 shutdown_connections 2024-04-25T11:14:10.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.090+0000 7f06a9a00640 1 --2- 172.21.15.7:0/391422206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 0x7f0688003e70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:10.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.090+0000 7f06a9a00640 1 -- 172.21.15.7:0/391422206 >> 172.21.15.7:0/391422206 conn(0x7f06a4073090 msgr2=0x7f06a40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:10.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.090+0000 7f06a9a00640 1 -- 172.21.15.7:0/391422206 shutdown_connections 2024-04-25T11:14:10.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.090+0000 7f06a9a00640 1 -- 172.21.15.7:0/391422206 wait complete. 2024-04-25T11:14:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.093+0000 7f06a9a00640 1 Processor -- start 2024-04-25T11:14:10.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.093+0000 7f06a9a00640 1 -- start start 2024-04-25T11:14:10.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.094+0000 7f06a9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 0x7f06a415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.094+0000 7f06a9a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06a4143550 0x7f06a415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.094+0000 7f06a9a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a415d410 0x7f06a413e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.094+0000 7f06a9a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06a415baa0 con 0x7f0688003a00 2024-04-25T11:14:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.094+0000 7f06a9a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f06a41619c0 con 0x7f06a415d410 2024-04-25T11:14:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.094+0000 7f06a9a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f06a415bc20 con 0x7f06a4143550 2024-04-25T11:14:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.095+0000 7f06a3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 0x7f06a415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.095+0000 7f06a3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 0x7f06a415a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53528/0 (socket says 172.21.15.7:53528) 2024-04-25T11:14:10.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.095+0000 7f06a3e00640 1 -- 172.21.15.7:0/646165070 learned_addr learned my addr 172.21.15.7:0/646165070 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.095+0000 7f06a3e00640 1 -- 172.21.15.7:0/646165070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06a4143550 msgr2=0x7f06a415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.095+0000 7f06a3e00640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06a4143550 0x7f06a415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.095+0000 7f06a3e00640 1 -- 172.21.15.7:0/646165070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a415d410 msgr2=0x7f06a413e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.095+0000 7f06a3e00640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a415d410 0x7f06a413e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.095+0000 7f06a3e00640 1 -- 172.21.15.7:0/646165070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a415aba0 con 0x7f0688003a00 2024-04-25T11:14:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.095+0000 7f06a3e00640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 0x7f06a415a660 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f069402af20 tx=0x7f06a415eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.096+0000 7f069be00640 1 -- 172.21.15.7:0/646165070 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694001840 con 0x7f0688003a00 2024-04-25T11:14:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.096+0000 7f069be00640 1 -- 172.21.15.7:0/646165070 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0694044070 con 0x7f0688003a00 2024-04-25T11:14:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.096+0000 7f069be00640 1 -- 172.21.15.7:0/646165070 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069403c690 con 0x7f0688003a00 2024-04-25T11:14:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.096+0000 7f06a9a00640 1 -- 172.21.15.7:0/646165070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a415d820 con 0x7f0688003a00 2024-04-25T11:14:10.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.096+0000 7f06a9a00640 1 -- 172.21.15.7:0/646165070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a4142250 con 0x7f0688003a00 2024-04-25T11:14:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.097+0000 7f06a9a00640 1 -- 172.21.15.7:0/646165070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a40b67a0 con 0x7f0688003a00 2024-04-25T11:14:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.098+0000 7f069be00640 1 -- 172.21.15.7:0/646165070 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f069403c830 con 0x7f0688003a00 2024-04-25T11:14:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.099+0000 7f069be00640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06840744e0 0x7f06840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.100+0000 7f069be00640 1 -- 172.21.15.7:0/646165070 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7f06940b59a0 con 0x7f0688003a00 2024-04-25T11:14:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.100+0000 7f06a3400640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06840744e0 0x7f06840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:10.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.101+0000 7f06a3400640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06840744e0 0x7f06840769f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f06900090e0 tx=0x7f0690008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:10.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.105+0000 7f069be00640 1 -- 172.21.15.7:0/646165070 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06a40b67a0 con 0x7f0688003a00 2024-04-25T11:14:10.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:10.391+0000 7f06a9a00640 1 -- 172.21.15.7:0/646165070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]} v 0) -- 0x7f06a4050740 con 0x7f0688003a00 2024-04-25T11:14:11.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.212+0000 7f069be00640 1 -- 172.21.15.7:0/646165070 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_blocklisted_oldest_tid"]}]=0 v215) ==== 162+0+0 (secure 0 0 0) 0x7f0694088020 con 0x7f0688003a00 2024-04-25T11:14:11.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.217+0000 7f06a9a00640 1 -- 172.21.15.7:0/646165070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06840744e0 msgr2=0x7f06840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.217+0000 7f06a9a00640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06840744e0 0x7f06840769f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f06900090e0 tx=0x7f0690008040 comp rx=0 tx=0).stop 2024-04-25T11:14:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.217+0000 7f06a9a00640 1 -- 172.21.15.7:0/646165070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 msgr2=0x7f06a415a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.217+0000 7f06a9a00640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 0x7f06a415a660 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f069402af20 tx=0x7f06a415eb50 comp rx=0 tx=0).stop 2024-04-25T11:14:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.217+0000 7f06a9a00640 1 -- 172.21.15.7:0/646165070 shutdown_connections 2024-04-25T11:14:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.217+0000 7f06a9a00640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06840744e0 0x7f06840769f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.217+0000 7f06a9a00640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06a415d410 0x7f06a413e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.217+0000 7f06a9a00640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06a4143550 0x7f06a415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.217+0000 7f06a9a00640 1 --2- 172.21.15.7:0/646165070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0688003a00 0x7f06a415a660 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.217+0000 7f06a9a00640 1 -- 172.21.15.7:0/646165070 >> 172.21.15.7:0/646165070 conn(0x7f06a4073090 msgr2=0x7f06a40684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.218+0000 7f06a9a00640 1 -- 172.21.15.7:0/646165070 shutdown_connections 2024-04-25T11:14:11.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.218+0000 7f06a9a00640 1 -- 172.21.15.7:0/646165070 wait complete. 2024-04-25T11:14:11.250 INFO:tasks.cephfs_test_runner:that a client is blocklisted when its encoded session metadata exceeds the ... skipped 'Require FUSE client to inject client release failure' 2024-04-25T11:14:11.250 INFO:tasks.cephfs_test_runner:Starting test: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-25T11:14:11.250 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:11.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.546+0000 7ff790a00640 1 Processor -- start 2024-04-25T11:14:11.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.547+0000 7ff790a00640 1 -- start start 2024-04-25T11:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.549+0000 7ff790a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff78c164380 0x7ff78c0b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.549+0000 7ff790a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff78c0b5d40 0x7ff78c167f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.549+0000 7ff790a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff78c0b47d0 con 0x7ff78c06ffd0 2024-04-25T11:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.549+0000 7ff790a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff78c0b4a40 con 0x7ff78c0b5d40 2024-04-25T11:14:11.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.549+0000 7ff790a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff78c0b4d20 con 0x7ff78c164380 2024-04-25T11:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.549+0000 7ff78b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff78c0b5d40 0x7ff78c167f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.549+0000 7ff78b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff78c0b5d40 0x7ff78c167f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53534/0 (socket says 172.21.15.7:53534) 2024-04-25T11:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.549+0000 7ff78b200640 1 -- 172.21.15.7:0/600465393 learned_addr learned my addr 172.21.15.7:0/600465393 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.549+0000 7ff781e00640 1 --2- 172.21.15.7:0/600465393 >> v2:172.21.15.87:3300/0 conn(0x7ff78c164380 0x7ff78c0b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.550+0000 7ff78b200640 1 -- 172.21.15.7:0/600465393 >> v2:172.21.15.87:3300/0 conn(0x7ff78c164380 msgr2=0x7ff78c0b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.550+0000 7ff78b200640 1 --2- 172.21.15.7:0/600465393 >> v2:172.21.15.87:3300/0 conn(0x7ff78c164380 0x7ff78c0b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.550+0000 7ff78b200640 1 -- 172.21.15.7:0/600465393 >> v1:172.21.15.87:6789/0 conn(0x7ff78c06ffd0 legacy=0x7ff78c0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:11.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.550+0000 7ff78b200640 1 -- 172.21.15.7:0/600465393 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c0b5000 con 0x7ff78c0b5d40 2024-04-25T11:14:11.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.551+0000 7ff78b200640 1 --2- 172.21.15.7:0/600465393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c0b5d40 0x7ff78c167f50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff78c06f8b0 tx=0x7ff78402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d230630b854a95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:11.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.551+0000 7ff789e00640 1 -- 172.21.15.7:0/600465393 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784036070 con 0x7ff78c0b5d40 2024-04-25T11:14:11.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.551+0000 7ff789e00640 1 -- 172.21.15.7:0/600465393 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff784031600 con 0x7ff78c0b5d40 2024-04-25T11:14:11.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.551+0000 7ff789e00640 1 -- 172.21.15.7:0/600465393 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7840319d0 con 0x7ff78c0b5d40 2024-04-25T11:14:11.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.552+0000 7ff790a00640 1 -- 172.21.15.7:0/600465393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c0b5d40 msgr2=0x7ff78c167f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:11.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.552+0000 7ff790a00640 1 --2- 172.21.15.7:0/600465393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c0b5d40 0x7ff78c167f50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff78c06f8b0 tx=0x7ff78402a6f0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.552+0000 7ff790a00640 1 -- 172.21.15.7:0/600465393 shutdown_connections 2024-04-25T11:14:11.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.552+0000 7ff790a00640 1 --2- 172.21.15.7:0/600465393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c0b5d40 0x7ff78c167f50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.552+0000 7ff790a00640 1 --2- 172.21.15.7:0/600465393 >> v2:172.21.15.87:3300/0 conn(0x7ff78c164380 0x7ff78c0b5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.552+0000 7ff790a00640 1 -- 172.21.15.7:0/600465393 >> 172.21.15.7:0/600465393 conn(0x7ff78c0b3820 msgr2=0x7ff78c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:11.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.552+0000 7ff790a00640 1 -- 172.21.15.7:0/600465393 shutdown_connections 2024-04-25T11:14:11.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.552+0000 7ff790a00640 1 -- 172.21.15.7:0/600465393 wait complete. 2024-04-25T11:14:11.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.554+0000 7ff790a00640 1 Processor -- start 2024-04-25T11:14:11.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.555+0000 7ff790a00640 1 -- start start 2024-04-25T11:14:11.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.556+0000 7ff790a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c06ffd0 0x7ff78c14da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:11.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.556+0000 7ff790a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff78c0b5d40 0x7ff78c1edd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:11.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.556+0000 7ff790a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff78c164380 0x7ff78c0cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:11.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.556+0000 7ff790a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c0afaa0 con 0x7ff78c06ffd0 2024-04-25T11:14:11.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.556+0000 7ff790a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff78c164790 con 0x7ff78c164380 2024-04-25T11:14:11.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.556+0000 7ff790a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff78c06fa70 con 0x7ff78c0b5d40 2024-04-25T11:14:11.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.556+0000 7ff78a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c06ffd0 0x7ff78c14da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:11.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.556+0000 7ff78a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c06ffd0 0x7ff78c14da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53538/0 (socket says 172.21.15.7:53538) 2024-04-25T11:14:11.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.556+0000 7ff78a800640 1 -- 172.21.15.7:0/453334486 learned_addr learned my addr 172.21.15.7:0/453334486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.557+0000 7ff781e00640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff78c0b5d40 0x7ff78c1edd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.557+0000 7ff78b200640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff78c164380 0x7ff78c0cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.557+0000 7ff78a800640 1 -- 172.21.15.7:0/453334486 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff78c0b5d40 msgr2=0x7ff78c1edd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:11.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.557+0000 7ff78a800640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff78c0b5d40 0x7ff78c1edd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.557+0000 7ff78a800640 1 -- 172.21.15.7:0/453334486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff78c164380 msgr2=0x7ff78c0cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.557+0000 7ff78a800640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff78c164380 0x7ff78c0cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.557+0000 7ff78a800640 1 -- 172.21.15.7:0/453334486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff78c1ee2b0 con 0x7ff78c06ffd0 2024-04-25T11:14:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.557+0000 7ff781e00640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff78c0b5d40 0x7ff78c1edd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.557+0000 7ff78a800640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c06ffd0 0x7ff78c14da30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00b3e0 tx=0x7ff77c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.558+0000 7ff783400640 1 -- 172.21.15.7:0/453334486 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c015020 con 0x7ff78c06ffd0 2024-04-25T11:14:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.558+0000 7ff783400640 1 -- 172.21.15.7:0/453334486 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77c011830 con 0x7ff78c06ffd0 2024-04-25T11:14:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.558+0000 7ff783400640 1 -- 172.21.15.7:0/453334486 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c011c20 con 0x7ff78c06ffd0 2024-04-25T11:14:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.558+0000 7ff790a00640 1 -- 172.21.15.7:0/453334486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff78c1ee6f0 con 0x7ff78c06ffd0 2024-04-25T11:14:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.558+0000 7ff790a00640 1 -- 172.21.15.7:0/453334486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff78c1eecb0 con 0x7ff78c06ffd0 2024-04-25T11:14:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.559+0000 7ff790a00640 1 -- 172.21.15.7:0/453334486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff78c050740 con 0x7ff78c06ffd0 2024-04-25T11:14:11.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.560+0000 7ff783400640 1 -- 172.21.15.7:0/453334486 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff77c018020 con 0x7ff78c06ffd0 2024-04-25T11:14:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.561+0000 7ff783400640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7680744b0 0x7ff7680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:11.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.561+0000 7ff783400640 1 -- 172.21.15.7:0/453334486 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7ff77c094fe0 con 0x7ff78c06ffd0 2024-04-25T11:14:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.562+0000 7ff781e00640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7680744b0 0x7ff7680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:11.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.563+0000 7ff781e00640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7680744b0 0x7ff7680769c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff774006fd0 tx=0x7ff774008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.567+0000 7ff783400640 1 -- 172.21.15.7:0/453334486 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff77c01f020 con 0x7ff78c06ffd0 2024-04-25T11:14:11.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:14:11.734 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:14:11.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.772+0000 7ff790a00640 1 -- 172.21.15.7:0/453334486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff78c0cde20 con 0x7ff78c06ffd0 2024-04-25T11:14:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.773+0000 7ff783400640 1 -- 172.21.15.7:0/453334486 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+13195 (secure 0 0 0) 0x7ff77c05f8f0 con 0x7ff78c06ffd0 2024-04-25T11:14:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:11.774 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":59,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:09.202854+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6836/3503942462":"2024-04-26T11:14:06.179527+0000","172.21.15.7:6834/3503942462":"2024-04-26T11:14:06.179527+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 -- 172.21.15.7:0/453334486 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7680744b0 msgr2=0x7ff7680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7680744b0 0x7ff7680769c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff774006fd0 tx=0x7ff774008040 comp rx=0 tx=0).stop 2024-04-25T11:14:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 -- 172.21.15.7:0/453334486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c06ffd0 msgr2=0x7ff78c14da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c06ffd0 0x7ff78c14da30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00b3e0 tx=0x7ff77c00b8a0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 -- 172.21.15.7:0/453334486 shutdown_connections 2024-04-25T11:14:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff7680744b0 0x7ff7680769c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff78c164380 0x7ff78c0cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff78c0b5d40 0x7ff78c1edd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 --2- 172.21.15.7:0/453334486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff78c06ffd0 0x7ff78c14da30 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 -- 172.21.15.7:0/453334486 >> 172.21.15.7:0/453334486 conn(0x7ff78c0b3820 msgr2=0x7ff78c0af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 -- 172.21.15.7:0/453334486 shutdown_connections 2024-04-25T11:14:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:11.776+0000 7ff790a00640 1 -- 172.21.15.7:0/453334486 wait complete. 2024-04-25T11:14:11.805 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:14:11.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:14:12.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.159+0000 7f6a87e00640 1 Processor -- start 2024-04-25T11:14:12.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.159+0000 7f6a87e00640 1 -- start start 2024-04-25T11:14:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.161+0000 7f6a87e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6a880d5190 0x7f6a881601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.161+0000 7f6a87e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a88161030 con 0x7f6a880d2e90 2024-04-25T11:14:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.161+0000 7f6a87e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a88161310 con 0x7f6a880d4d80 2024-04-25T11:14:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.161+0000 7f6a87e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6a880d4580 con 0x7f6a880d5190 2024-04-25T11:14:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.161+0000 7f6a86000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6a880d2e90 0x7f6a881616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43476/0 (socket says 172.21.15.7:43476) 2024-04-25T11:14:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.162+0000 7f6a86000640 1 -- 172.21.15.7:0/2667380571 learned_addr learned my addr 172.21.15.7:0/2667380571 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.162+0000 7f6a87400640 1 --2- 172.21.15.7:0/2667380571 >> v2:172.21.15.87:3300/0 conn(0x7f6a880d5190 0x7f6a881601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.162+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2429950742 0 0) 0x7f6a88161030 con 0x7f6a880d2e90 2024-04-25T11:14:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.162+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a6c003520 con 0x7f6a880d2e90 2024-04-25T11:14:12.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.162+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2645225874 0 0) 0x7f6a6c003520 con 0x7f6a880d2e90 2024-04-25T11:14:12.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.162+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 >> v2:172.21.15.87:3300/0 conn(0x7f6a880d5190 msgr2=0x7f6a881601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:12.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.162+0000 7f6a85600640 1 --2- 172.21.15.7:0/2667380571 >> v2:172.21.15.87:3300/0 conn(0x7f6a880d5190 0x7f6a881601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.162+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 >> v1:172.21.15.87:6789/0 conn(0x7f6a880d4d80 legacy=0x7f6a880d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.162+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a880d4860 con 0x7f6a880d2e90 2024-04-25T11:14:12.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.162+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f6a74001670 con 0x7f6a880d2e90 2024-04-25T11:14:12.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 >> v1:172.21.15.7:6789/0 conn(0x7f6a880d2e90 legacy=0x7f6a881616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a85600640 1 --2- 172.21.15.7:0/2667380571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a6c003a00 0x7f6a6c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a85600640 1 --2- 172.21.15.7:0/2667380571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a6c004800 0x7f6a6c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a85600640 1 --2- 172.21.15.7:0/2667380571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c005270 0x7f6a6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880d4580 con 0x7f6a6c005270 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c00c190 con 0x7f6a6c003a00 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6a6c00c390 con 0x7f6a6c004800 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a87400640 1 --2- 172.21.15.7:0/2667380571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c005270 0x7f6a6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a86a00640 1 --2- 172.21.15.7:0/2667380571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a6c003a00 0x7f6a6c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a86000640 1 --2- 172.21.15.7:0/2667380571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a6c004800 0x7f6a6c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a87400640 1 -- 172.21.15.7:0/2667380571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a6c004800 msgr2=0x7f6a6c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a87400640 1 --2- 172.21.15.7:0/2667380571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a6c004800 0x7f6a6c004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a87400640 1 -- 172.21.15.7:0/2667380571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a6c003a00 msgr2=0x7f6a6c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a87400640 1 --2- 172.21.15.7:0/2667380571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a6c003a00 0x7f6a6c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.163+0000 7f6a87400640 1 -- 172.21.15.7:0/2667380571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6c003520 con 0x7f6a6c005270 2024-04-25T11:14:12.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.164+0000 7f6a87400640 1 -- 172.21.15.7:0/2667380571 reap_dead start 2024-04-25T11:14:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.164+0000 7f6a87400640 1 --2- 172.21.15.7:0/2667380571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c005270 0x7f6a6c00b9c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009ac0 tx=0x7f6a6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc84149614cd014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.165+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80009690 con 0x7f6a6c005270 2024-04-25T11:14:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.165+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a80004020 con 0x7f6a6c005270 2024-04-25T11:14:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.165+0000 7f6a85600640 1 -- 172.21.15.7:0/2667380571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a800043d0 con 0x7f6a6c005270 2024-04-25T11:14:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.166+0000 7f6a87e00640 1 -- 172.21.15.7:0/2667380571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c005270 msgr2=0x7f6a6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.166+0000 7f6a87e00640 1 --2- 172.21.15.7:0/2667380571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c005270 0x7f6a6c00b9c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009ac0 tx=0x7f6a6c005830 comp rx=0 tx=0).stop 2024-04-25T11:14:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.166+0000 7f6a87e00640 1 -- 172.21.15.7:0/2667380571 shutdown_connections 2024-04-25T11:14:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.166+0000 7f6a87e00640 1 --2- 172.21.15.7:0/2667380571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c005270 0x7f6a6c00b9c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.166+0000 7f6a87e00640 1 -- 172.21.15.7:0/2667380571 >> 172.21.15.7:0/2667380571 conn(0x7f6a8806c750 msgr2=0x7f6a8806cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.166+0000 7f6a87e00640 1 -- 172.21.15.7:0/2667380571 shutdown_connections 2024-04-25T11:14:12.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.166+0000 7f6a87e00640 1 -- 172.21.15.7:0/2667380571 wait complete. 2024-04-25T11:14:12.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.168+0000 7f6a87e00640 1 Processor -- start 2024-04-25T11:14:12.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.168+0000 7f6a87e00640 1 -- start start 2024-04-25T11:14:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a87e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a6c005270 0x7f6a88155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a87e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a880b7080 0x7f6a881fa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a87e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a881fa560 0x7f6a880cef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a87e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880d2900 con 0x7f6a880b7080 2024-04-25T11:14:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a87e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880d2780 con 0x7f6a6c005270 2024-04-25T11:14:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a87e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6a880d2a80 con 0x7f6a881fa560 2024-04-25T11:14:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a86000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a880b7080 0x7f6a881fa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a86000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a880b7080 0x7f6a881fa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53564/0 (socket says 172.21.15.7:53564) 2024-04-25T11:14:12.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a86000640 1 -- 172.21.15.7:0/2649124609 learned_addr learned my addr 172.21.15.7:0/2649124609 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:12.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a87400640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a881fa560 0x7f6a880cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a86000640 1 -- 172.21.15.7:0/2649124609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a881fa560 msgr2=0x7f6a880cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a86000640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a881fa560 0x7f6a880cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a86000640 1 -- 172.21.15.7:0/2649124609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a6c005270 msgr2=0x7f6a88155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a86000640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a6c005270 0x7f6a88155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.170+0000 7f6a86000640 1 -- 172.21.15.7:0/2649124609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a881fadb0 con 0x7f6a880b7080 2024-04-25T11:14:12.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.171+0000 7f6a86000640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a880b7080 0x7f6a881fa020 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400d7d0 tx=0x7f6a7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:12.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.171+0000 7f6a7ea00640 1 -- 172.21.15.7:0/2649124609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74019070 con 0x7f6a880b7080 2024-04-25T11:14:12.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.171+0000 7f6a7ea00640 1 -- 172.21.15.7:0/2649124609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a740072d0 con 0x7f6a880b7080 2024-04-25T11:14:12.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.171+0000 7f6a7ea00640 1 -- 172.21.15.7:0/2649124609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a740047e0 con 0x7f6a880b7080 2024-04-25T11:14:12.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.171+0000 7f6a87e00640 1 -- 172.21.15.7:0/2649124609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a881564d0 con 0x7f6a880b7080 2024-04-25T11:14:12.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.171+0000 7f6a87e00640 1 -- 172.21.15.7:0/2649124609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a880b65a0 con 0x7f6a880b7080 2024-04-25T11:14:12.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.172+0000 7f6a87e00640 1 -- 172.21.15.7:0/2649124609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a88050740 con 0x7f6a880b7080 2024-04-25T11:14:12.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.173+0000 7f6a7ea00640 1 -- 172.21.15.7:0/2649124609 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6a7401c070 con 0x7f6a880b7080 2024-04-25T11:14:12.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.174+0000 7f6a7ea00640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a680744e0 0x7f6a680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.174+0000 7f6a7ea00640 1 -- 172.21.15.7:0/2649124609 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7f6a74094ab0 con 0x7f6a880b7080 2024-04-25T11:14:12.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.175+0000 7f6a86a00640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a680744e0 0x7f6a680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.175+0000 7f6a86a00640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a680744e0 0x7f6a680769f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6a780013d0 tx=0x7f6a78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:12.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.180+0000 7f6a7ea00640 1 -- 172.21.15.7:0/2649124609 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a881564d0 con 0x7f6a880b7080 2024-04-25T11:14:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.384+0000 7f6a87e00640 1 -- 172.21.15.7:0/2649124609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6a880c8390 con 0x7f6a880b7080 2024-04-25T11:14:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.385+0000 7f6a7ea00640 1 -- 172.21.15.7:0/2649124609 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f6a7405f310 con 0x7f6a880b7080 2024-04-25T11:14:12.385 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:14:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 -- 172.21.15.7:0/2649124609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a680744e0 msgr2=0x7f6a680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a680744e0 0x7f6a680769f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6a780013d0 tx=0x7f6a78009040 comp rx=0 tx=0).stop 2024-04-25T11:14:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 -- 172.21.15.7:0/2649124609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a880b7080 msgr2=0x7f6a881fa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a880b7080 0x7f6a881fa020 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400d7d0 tx=0x7f6a7400dc90 comp rx=0 tx=0).stop 2024-04-25T11:14:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 -- 172.21.15.7:0/2649124609 shutdown_connections 2024-04-25T11:14:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a680744e0 0x7f6a680769f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a881fa560 0x7f6a880cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a880b7080 0x7f6a881fa020 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 --2- 172.21.15.7:0/2649124609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a6c005270 0x7f6a88155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 -- 172.21.15.7:0/2649124609 >> 172.21.15.7:0/2649124609 conn(0x7f6a8806c750 msgr2=0x7f6a8815e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 -- 172.21.15.7:0/2649124609 shutdown_connections 2024-04-25T11:14:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.388+0000 7f6a87e00640 1 -- 172.21.15.7:0/2649124609 wait complete. 2024-04-25T11:14:12.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2024-04-25T11:14:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.702+0000 7f7930a00640 1 Processor -- start 2024-04-25T11:14:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.702+0000 7f7930a00640 1 -- start start 2024-04-25T11:14:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.705+0000 7f7930a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c15f180 0x7f792c161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.705+0000 7f7930a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f792c162700 con 0x7f792c06dc90 2024-04-25T11:14:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.705+0000 7f7930a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f792c1629e0 con 0x7f792c1621a0 2024-04-25T11:14:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.705+0000 7f7930a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f792c15f590 con 0x7f792c15f180 2024-04-25T11:14:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.705+0000 7f792a800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f792c06dc90 0x7f792c06f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43488/0 (socket says 172.21.15.7:43488) 2024-04-25T11:14:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.705+0000 7f792a800640 1 -- 172.21.15.7:0/829538603 learned_addr learned my addr 172.21.15.7:0/829538603 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:12.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7929e00640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c15f180 0x7f792c161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 241861946 0 0) 0x7f792c162700 con 0x7f792c06dc90 2024-04-25T11:14:12.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7910003520 con 0x7f792c06dc90 2024-04-25T11:14:12.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1092304361 0 0) 0x7f7910003520 con 0x7f792c06dc90 2024-04-25T11:14:12.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c15f180 msgr2=0x7f792c161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:12.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c15f180 0x7f792c161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 >> v1:172.21.15.87:6789/0 conn(0x7f792c1621a0 legacy=0x7f792c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c15e980 con 0x7f792c06dc90 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f79240019f0 con 0x7f792c06dc90 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 >> v1:172.21.15.7:6789/0 conn(0x7f792c06dc90 legacy=0x7f792c06f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79100039e0 0x7f7910003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 0x7f7910004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c15f180 0x7f791000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f792c15f590 con 0x7f7910004800 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f79100054e0 con 0x7f79100039e0 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.706+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f791000bf00 con 0x7f792c15f180 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.707+0000 7f7929400640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 0x7f7910004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.707+0000 7f7929e00640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79100039e0 0x7f7910003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.707+0000 7f792a800640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c15f180 0x7f791000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.707+0000 7f7929400640 1 -- 172.21.15.7:0/829538603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c15f180 msgr2=0x7f791000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.707+0000 7f7929400640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c15f180 0x7f791000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.707+0000 7f7929400640 1 -- 172.21.15.7:0/829538603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79100039e0 msgr2=0x7f7910003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.707+0000 7f7929400640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79100039e0 0x7f7910003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.707+0000 7f7929400640 1 -- 172.21.15.7:0/829538603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7910003520 con 0x7f7910004800 2024-04-25T11:14:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.708+0000 7f7929400640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 0x7f7910004c10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f792c071740 tx=0x7f7910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed093ef9bfe9a5e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.708+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918001370 con 0x7f7910004800 2024-04-25T11:14:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.708+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79180052d0 con 0x7f7910004800 2024-04-25T11:14:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.708+0000 7f7928a00640 1 -- 172.21.15.7:0/829538603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791802ab90 con 0x7f7910004800 2024-04-25T11:14:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.709+0000 7f7930a00640 1 -- 172.21.15.7:0/829538603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 msgr2=0x7f7910004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.709+0000 7f7930a00640 1 --2- 172.21.15.7:0/829538603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 0x7f7910004c10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f792c071740 tx=0x7f7910005830 comp rx=0 tx=0).stop 2024-04-25T11:14:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.709+0000 7f792a800640 1 -- 172.21.15.7:0/829538603 reap_dead start 2024-04-25T11:14:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.709+0000 7f7930a00640 1 -- 172.21.15.7:0/829538603 shutdown_connections 2024-04-25T11:14:12.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.709+0000 7f7930a00640 1 -- 172.21.15.7:0/829538603 >> 172.21.15.7:0/829538603 conn(0x7f792c0732b0 msgr2=0x7f792c0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:12.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.709+0000 7f7930a00640 1 -- 172.21.15.7:0/829538603 shutdown_connections 2024-04-25T11:14:12.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.709+0000 7f7930a00640 1 -- 172.21.15.7:0/829538603 wait complete. 2024-04-25T11:14:12.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.712+0000 7f7930a00640 1 Processor -- start 2024-04-25T11:14:12.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.712+0000 7f7930a00640 1 -- start start 2024-04-25T11:14:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.713+0000 7f7930a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 0x7f792c143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.713+0000 7f7930a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f792c1e7b60 0x7f792c1e7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.713+0000 7f7930a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c06f1e0 0x7f792c06f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.713+0000 7f7930a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f792c15e670 con 0x7f7910004800 2024-04-25T11:14:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.713+0000 7f7930a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f792c158ae0 con 0x7f792c1e7b60 2024-04-25T11:14:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.713+0000 7f7930a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f792c0d1310 con 0x7f792c06f1e0 2024-04-25T11:14:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.714+0000 7f7929e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 0x7f792c143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.714+0000 7f7929e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 0x7f792c143730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53584/0 (socket says 172.21.15.7:53584) 2024-04-25T11:14:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.714+0000 7f7929e00640 1 -- 172.21.15.7:0/1893698568 learned_addr learned my addr 172.21.15.7:0/1893698568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:12.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.714+0000 7f7929400640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f792c1e7b60 0x7f792c1e7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.714+0000 7f7929e00640 1 -- 172.21.15.7:0/1893698568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c06f1e0 msgr2=0x7f792c06f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.714+0000 7f7929e00640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c06f1e0 0x7f792c06f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.714+0000 7f7929e00640 1 -- 172.21.15.7:0/1893698568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f792c1e7b60 msgr2=0x7f792c1e7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.714+0000 7f7929e00640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f792c1e7b60 0x7f792c1e7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.714+0000 7f7929e00640 1 -- 172.21.15.7:0/1893698568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c06fb30 con 0x7f7910004800 2024-04-25T11:14:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.715+0000 7f7929e00640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 0x7f792c143730 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f791c0115d0 tx=0x7f791c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.715+0000 7f7922000640 1 -- 172.21.15.7:0/1893698568 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c015870 con 0x7f7910004800 2024-04-25T11:14:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.715+0000 7f7922000640 1 -- 172.21.15.7:0/1893698568 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f791c015a10 con 0x7f7910004800 2024-04-25T11:14:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.715+0000 7f7930a00640 1 -- 172.21.15.7:0/1893698568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c1e8900 con 0x7f7910004800 2024-04-25T11:14:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.715+0000 7f7922000640 1 -- 172.21.15.7:0/1893698568 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c015e00 con 0x7f7910004800 2024-04-25T11:14:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.715+0000 7f7930a00640 1 -- 172.21.15.7:0/1893698568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c144260 con 0x7f7910004800 2024-04-25T11:14:12.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.716+0000 7f7930a00640 1 -- 172.21.15.7:0/1893698568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c074250 con 0x7f7910004800 2024-04-25T11:14:12.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.718+0000 7f7922000640 1 -- 172.21.15.7:0/1893698568 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f791c00f480 con 0x7f7910004800 2024-04-25T11:14:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.719+0000 7f7922000640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f790c0744b0 0x7f790c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.719+0000 7f7922000640 1 -- 172.21.15.7:0/1893698568 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7f791c095b10 con 0x7f7910004800 2024-04-25T11:14:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.720+0000 7f7929400640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f790c0744b0 0x7f790c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.720+0000 7f7929400640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f790c0744b0 0x7f790c0769c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7918032000 tx=0x7f791802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:12.724+0000 7f7922000640 1 -- 172.21.15.7:0/1893698568 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f792c06fb30 con 0x7f7910004800 2024-04-25T11:14:13.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.007+0000 7f7930a00640 1 -- 172.21.15.7:0/1893698568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) -- 0x7f792c06e2b0 con 0x7f7910004800 2024-04-25T11:14:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.226+0000 7f7922000640 1 -- 172.21.15.7:0/1893698568 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v217) ==== 153+0+0 (secure 0 0 0) 0x7f791c019030 con 0x7f7910004800 2024-04-25T11:14:13.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.230+0000 7f7930a00640 1 -- 172.21.15.7:0/1893698568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f790c0744b0 msgr2=0x7f790c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.230+0000 7f7930a00640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f790c0744b0 0x7f790c0769c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7918032000 tx=0x7f791802e000 comp rx=0 tx=0).stop 2024-04-25T11:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.230+0000 7f7930a00640 1 -- 172.21.15.7:0/1893698568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 msgr2=0x7f792c143730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.230+0000 7f7930a00640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 0x7f792c143730 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f791c0115d0 tx=0x7f791c011a90 comp rx=0 tx=0).stop 2024-04-25T11:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.230+0000 7f7930a00640 1 -- 172.21.15.7:0/1893698568 shutdown_connections 2024-04-25T11:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.230+0000 7f7930a00640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f790c0744b0 0x7f790c0769c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.230+0000 7f7930a00640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f792c06f1e0 0x7f792c06f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.230+0000 7f7930a00640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f792c1e7b60 0x7f792c1e7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.230+0000 7f7930a00640 1 --2- 172.21.15.7:0/1893698568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7910004800 0x7f792c143730 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.230+0000 7f7930a00640 1 -- 172.21.15.7:0/1893698568 >> 172.21.15.7:0/1893698568 conn(0x7f792c0732b0 msgr2=0x7f792c1617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.231+0000 7f7930a00640 1 -- 172.21.15.7:0/1893698568 shutdown_connections 2024-04-25T11:14:13.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.231+0000 7f7930a00640 1 -- 172.21.15.7:0/1893698568 wait complete. 2024-04-25T11:14:13.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:14:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.553+0000 7fb158a00640 1 Processor -- start 2024-04-25T11:14:13.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.554+0000 7fb158a00640 1 -- start start 2024-04-25T11:14:13.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.556+0000 7fb158a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb15415dfb0 0x7fb1540b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.556+0000 7fb158a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1540d0b40 0x7fb15406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.556+0000 7fb158a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb15415e3c0 con 0x7fb1540d0f50 2024-04-25T11:14:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.556+0000 7fb158a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb1540b09b0 con 0x7fb15415dfb0 2024-04-25T11:14:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.556+0000 7fb158a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb1540cf800 con 0x7fb1540d0b40 2024-04-25T11:14:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.557+0000 7fb152800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb1540d0f50 0x7fb154161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43498/0 (socket says 172.21.15.7:43498) 2024-04-25T11:14:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.557+0000 7fb152800640 1 -- 172.21.15.7:0/732817591 learned_addr learned my addr 172.21.15.7:0/732817591 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.557+0000 7fb151e00640 1 --2- 172.21.15.7:0/732817591 >> v2:172.21.15.7:3300/0 conn(0x7fb15415dfb0 0x7fb1540b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:13.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.557+0000 7fb151400640 1 --2- 172.21.15.7:0/732817591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1540d0b40 0x7fb15406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:13.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.557+0000 7fb150a00640 1 -- 172.21.15.7:0/732817591 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678245406 0 0) 0x7fb15415e3c0 con 0x7fb1540d0f50 2024-04-25T11:14:13.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.557+0000 7fb150a00640 1 -- 172.21.15.7:0/732817591 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb138003520 con 0x7fb1540d0f50 2024-04-25T11:14:13.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.557+0000 7fb151e00640 1 -- 172.21.15.7:0/732817591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1540d0b40 msgr2=0x7fb15406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:13.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.557+0000 7fb151e00640 1 --2- 172.21.15.7:0/732817591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1540d0b40 0x7fb15406f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.557+0000 7fb151e00640 1 -- 172.21.15.7:0/732817591 >> v1:172.21.15.7:6789/0 conn(0x7fb1540d0f50 legacy=0x7fb154161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:13.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.557+0000 7fb151e00640 1 -- 172.21.15.7:0/732817591 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1540cfaa0 con 0x7fb15415dfb0 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.558+0000 7fb151e00640 1 --2- 172.21.15.7:0/732817591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb15415dfb0 0x7fb1540b0470 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb1440097a0 tx=0x7fb14402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cfa3c0acd1d65d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.559+0000 7fb150a00640 1 -- 172.21.15.7:0/732817591 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144037070 con 0x7fb15415dfb0 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.559+0000 7fb150a00640 1 -- 172.21.15.7:0/732817591 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb144032440 con 0x7fb15415dfb0 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.559+0000 7fb150a00640 1 -- 172.21.15.7:0/732817591 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1440327f0 con 0x7fb15415dfb0 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.559+0000 7fb158a00640 1 -- 172.21.15.7:0/732817591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb15415dfb0 msgr2=0x7fb1540b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.559+0000 7fb158a00640 1 --2- 172.21.15.7:0/732817591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb15415dfb0 0x7fb1540b0470 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb1440097a0 tx=0x7fb14402a180 comp rx=0 tx=0).stop 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.560+0000 7fb158a00640 1 -- 172.21.15.7:0/732817591 shutdown_connections 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.560+0000 7fb158a00640 1 --2- 172.21.15.7:0/732817591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1540d0b40 0x7fb15406f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.560+0000 7fb158a00640 1 --2- 172.21.15.7:0/732817591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb15415dfb0 0x7fb1540b0470 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.560+0000 7fb158a00640 1 -- 172.21.15.7:0/732817591 >> 172.21.15.7:0/732817591 conn(0x7fb1540742c0 msgr2=0x7fb15406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.560+0000 7fb158a00640 1 -- 172.21.15.7:0/732817591 shutdown_connections 2024-04-25T11:14:13.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.560+0000 7fb158a00640 1 -- 172.21.15.7:0/732817591 wait complete. 2024-04-25T11:14:13.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.562+0000 7fb158a00640 1 Processor -- start 2024-04-25T11:14:13.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.562+0000 7fb158a00640 1 -- start start 2024-04-25T11:14:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.564+0000 7fb158a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1540d0b40 0x7fb154154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.564+0000 7fb158a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1540d0f50 0x7fb1541551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.564+0000 7fb158a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb15415dfb0 0x7fb1540b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.564+0000 7fb158a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb1540d0340 con 0x7fb15415dfb0 2024-04-25T11:14:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.564+0000 7fb158a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb1540d0060 con 0x7fb1540d0f50 2024-04-25T11:14:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.564+0000 7fb158a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb1540d0620 con 0x7fb1540d0b40 2024-04-25T11:14:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.564+0000 7fb152800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb15415dfb0 0x7fb1540b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:13.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.564+0000 7fb152800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb15415dfb0 0x7fb1540b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53602/0 (socket says 172.21.15.7:53602) 2024-04-25T11:14:13.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.564+0000 7fb152800640 1 -- 172.21.15.7:0/3486790282 learned_addr learned my addr 172.21.15.7:0/3486790282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:13.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb151400640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1540d0f50 0x7fb1541551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:13.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb151e00640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1540d0b40 0x7fb154154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:13.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb152800640 1 -- 172.21.15.7:0/3486790282 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1540d0b40 msgr2=0x7fb154154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:13.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb152800640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1540d0b40 0x7fb154154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb152800640 1 -- 172.21.15.7:0/3486790282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1540d0f50 msgr2=0x7fb1541551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:13.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb152800640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1540d0f50 0x7fb1541551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb152800640 1 -- 172.21.15.7:0/3486790282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1541559e0 con 0x7fb15415dfb0 2024-04-25T11:14:13.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb151400640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1540d0f50 0x7fb1541551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:13.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb152800640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb15415dfb0 0x7fb1540b5a90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00ca70 tx=0x7fb14c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:13.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb14a000640 1 -- 172.21.15.7:0/3486790282 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c017070 con 0x7fb15415dfb0 2024-04-25T11:14:13.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb14a000640 1 -- 172.21.15.7:0/3486790282 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb14c00ab70 con 0x7fb15415dfb0 2024-04-25T11:14:13.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb14a000640 1 -- 172.21.15.7:0/3486790282 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c012400 con 0x7fb15415dfb0 2024-04-25T11:14:13.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.565+0000 7fb158a00640 1 -- 172.21.15.7:0/3486790282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15415b880 con 0x7fb15415dfb0 2024-04-25T11:14:13.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.566+0000 7fb158a00640 1 -- 172.21.15.7:0/3486790282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb15415be40 con 0x7fb15415dfb0 2024-04-25T11:14:13.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.567+0000 7fb158a00640 1 -- 172.21.15.7:0/3486790282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1540b0770 con 0x7fb15415dfb0 2024-04-25T11:14:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.568+0000 7fb14a000640 1 -- 172.21.15.7:0/3486790282 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb14c01a070 con 0x7fb15415dfb0 2024-04-25T11:14:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.569+0000 7fb14a000640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb1340744e0 0x7fb1340769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.569+0000 7fb14a000640 1 -- 172.21.15.7:0/3486790282 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7fb14c094d50 con 0x7fb15415dfb0 2024-04-25T11:14:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.569+0000 7fb151e00640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb1340744e0 0x7fb1340769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.570+0000 7fb151e00640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb1340744e0 0x7fb1340769f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb144034000 tx=0x7fb14402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.574+0000 7fb14a000640 1 -- 172.21.15.7:0/3486790282 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb1540b0770 con 0x7fb15415dfb0 2024-04-25T11:14:13.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.778+0000 7fb158a00640 1 -- 172.21.15.7:0/3486790282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb154050740 con 0x7fb15415dfb0 2024-04-25T11:14:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.784+0000 7fb14a000640 1 -- 172.21.15.7:0/3486790282 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fb14c05ef60 con 0x7fb15415dfb0 2024-04-25T11:14:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 -- 172.21.15.7:0/3486790282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb1340744e0 msgr2=0x7fb1340769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb1340744e0 0x7fb1340769f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb144034000 tx=0x7fb14402e000 comp rx=0 tx=0).stop 2024-04-25T11:14:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 -- 172.21.15.7:0/3486790282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb15415dfb0 msgr2=0x7fb1540b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:13.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb15415dfb0 0x7fb1540b5a90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00ca70 tx=0x7fb14c00cf30 comp rx=0 tx=0).stop 2024-04-25T11:14:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 -- 172.21.15.7:0/3486790282 shutdown_connections 2024-04-25T11:14:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb1340744e0 0x7fb1340769f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb15415dfb0 0x7fb1540b5a90 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1540d0f50 0x7fb1541551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 --2- 172.21.15.7:0/3486790282 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1540d0b40 0x7fb154154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 -- 172.21.15.7:0/3486790282 >> 172.21.15.7:0/3486790282 conn(0x7fb1540742c0 msgr2=0x7fb154070080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 -- 172.21.15.7:0/3486790282 shutdown_connections 2024-04-25T11:14:13.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:13.787+0000 7fb158a00640 1 -- 172.21.15.7:0/3486790282 wait complete. 2024-04-25T11:14:13.813 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:14:13.814 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:14:13.925 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:14:14.041 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:14:14.041 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:14:14.041 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:14.102 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:14:14.103 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:14:14.103 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:14.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:14:14.160 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:14:14.160 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:14.223 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:14:14.223 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:14:14.224 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:14.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:14:14.281 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:14:14.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.571+0000 7fe8c7000640 1 Processor -- start 2024-04-25T11:14:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.571+0000 7fe8c7000640 1 -- start start 2024-04-25T11:14:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.574+0000 7fe8c7000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c00b4aa0 0x7fe8c00ae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.574+0000 7fe8c7000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe8c0164710 0x7fe8c00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.574+0000 7fe8c7000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8c0164c80 con 0x7fe8c00b0da0 2024-04-25T11:14:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.574+0000 7fe8c7000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c01655b0 con 0x7fe8c0164710 2024-04-25T11:14:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.574+0000 7fe8c7000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c0165a40 con 0x7fe8c00b4aa0 2024-04-25T11:14:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.575+0000 7fe8c5000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe8c00b0da0 0x7fe8c00b4f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43512/0 (socket says 172.21.15.7:43512) 2024-04-25T11:14:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.575+0000 7fe8c5000640 1 -- 172.21.15.7:0/3670801091 learned_addr learned my addr 172.21.15.7:0/3670801091 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.575+0000 7fe8bfe00640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c00b4aa0 0x7fe8c00ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.575+0000 7fe8bf400640 1 --2- 172.21.15.7:0/3670801091 >> v2:172.21.15.87:3300/0 conn(0x7fe8c0164710 0x7fe8c00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.575+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1360298555 0 0) 0x7fe8c0164c80 con 0x7fe8c00b0da0 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.575+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8a4003520 con 0x7fe8c00b0da0 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 304239581 0 0) 0x7fe8a4003520 con 0x7fe8c00b0da0 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c00b4aa0 msgr2=0x7fe8c00ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c00b4aa0 0x7fe8c00ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 >> v2:172.21.15.87:3300/0 conn(0x7fe8c0164710 msgr2=0x7fe8c00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 --2- 172.21.15.7:0/3670801091 >> v2:172.21.15.87:3300/0 conn(0x7fe8c0164710 0x7fe8c00b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c0164f60 con 0x7fe8c00b0da0 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fe8b80019f0 con 0x7fe8c00b0da0 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 >> v1:172.21.15.7:6789/0 conn(0x7fe8c00b0da0 legacy=0x7fe8c00b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a40039b0 0x7fe8a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c00b4aa0 0x7fe8a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8a4004d40 0x7fe8a4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c01655b0 con 0x7fe8a4004d40 2024-04-25T11:14:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0165a40 con 0x7fe8a40039b0 2024-04-25T11:14:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a400b9c0 con 0x7fe8c00b4aa0 2024-04-25T11:14:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.576+0000 7fe8c5000640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8a4004d40 0x7fe8a4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.577+0000 7fe8bf400640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c00b4aa0 0x7fe8a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.577+0000 7fe8bfe00640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a40039b0 0x7fe8a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.577+0000 7fe8c5000640 1 -- 172.21.15.7:0/3670801091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c00b4aa0 msgr2=0x7fe8a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.577+0000 7fe8c5000640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c00b4aa0 0x7fe8a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.577+0000 7fe8c5000640 1 -- 172.21.15.7:0/3670801091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a40039b0 msgr2=0x7fe8a4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.577+0000 7fe8c5000640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8a40039b0 0x7fe8a4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.577+0000 7fe8c5000640 1 -- 172.21.15.7:0/3670801091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a4003520 con 0x7fe8a4004d40 2024-04-25T11:14:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.578+0000 7fe8c5000640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8a4004d40 0x7fe8a4005150 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0073280 tx=0x7fe8a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49d4fa1f53cd8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.578+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b800a4c0 con 0x7fe8a4004d40 2024-04-25T11:14:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.578+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b802f070 con 0x7fe8a4004d40 2024-04-25T11:14:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.578+0000 7fe8bea00640 1 -- 172.21.15.7:0/3670801091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b802bc70 con 0x7fe8a4004d40 2024-04-25T11:14:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.579+0000 7fe8c7000640 1 -- 172.21.15.7:0/3670801091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8a4004d40 msgr2=0x7fe8a4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.579+0000 7fe8c7000640 1 --2- 172.21.15.7:0/3670801091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8a4004d40 0x7fe8a4005150 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0073280 tx=0x7fe8a4005830 comp rx=0 tx=0).stop 2024-04-25T11:14:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.579+0000 7fe8c5000640 1 -- 172.21.15.7:0/3670801091 reap_dead start 2024-04-25T11:14:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.579+0000 7fe8c7000640 1 -- 172.21.15.7:0/3670801091 shutdown_connections 2024-04-25T11:14:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.579+0000 7fe8c7000640 1 -- 172.21.15.7:0/3670801091 >> 172.21.15.7:0/3670801091 conn(0x7fe8c0074200 msgr2=0x7fe8c006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.579+0000 7fe8c7000640 1 -- 172.21.15.7:0/3670801091 shutdown_connections 2024-04-25T11:14:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.579+0000 7fe8c7000640 1 -- 172.21.15.7:0/3670801091 wait complete. 2024-04-25T11:14:14.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.582+0000 7fe8c7000640 1 Processor -- start 2024-04-25T11:14:14.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.582+0000 7fe8c7000640 1 -- start start 2024-04-25T11:14:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.583+0000 7fe8c7000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8a4004d40 0x7fe8c014cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.583+0000 7fe8c7000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8c015c9b0 0x7fe8c015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.584+0000 7fe8c7000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8c015dae0 0x7fe8c01ede60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.584+0000 7fe8c7000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c006f300 con 0x7fe8c015dae0 2024-04-25T11:14:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.584+0000 7fe8c7000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c00af340 con 0x7fe8c015c9b0 2024-04-25T11:14:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.584+0000 7fe8c7000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c00b11b0 con 0x7fe8a4004d40 2024-04-25T11:14:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.584+0000 7fe8c5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8c015dae0 0x7fe8c01ede60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.584+0000 7fe8c5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8c015dae0 0x7fe8c01ede60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53622/0 (socket says 172.21.15.7:53622) 2024-04-25T11:14:14.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.584+0000 7fe8c5000640 1 -- 172.21.15.7:0/3891494887 learned_addr learned my addr 172.21.15.7:0/3891494887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:14.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.584+0000 7fe8bfe00640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8a4004d40 0x7fe8c014cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:14.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.584+0000 7fe8bf400640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8c015c9b0 0x7fe8c015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8c5000640 1 -- 172.21.15.7:0/3891494887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8a4004d40 msgr2=0x7fe8c014cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8c5000640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8a4004d40 0x7fe8c014cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8c5000640 1 -- 172.21.15.7:0/3891494887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8c015c9b0 msgr2=0x7fe8c015d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8c5000640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8c015c9b0 0x7fe8c015d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8c5000640 1 -- 172.21.15.7:0/3891494887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c015def0 con 0x7fe8c015dae0 2024-04-25T11:14:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8bf400640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8c015c9b0 0x7fe8c015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:14:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8c5000640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8c015dae0 0x7fe8c01ede60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe8b802b500 tx=0x7fe8c0165520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8b7e00640 1 -- 172.21.15.7:0/3891494887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8044070 con 0x7fe8c015dae0 2024-04-25T11:14:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8b7e00640 1 -- 172.21.15.7:0/3891494887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8b8034470 con 0x7fe8c015dae0 2024-04-25T11:14:14.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8b7e00640 1 -- 172.21.15.7:0/3891494887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b800a4c0 con 0x7fe8c015dae0 2024-04-25T11:14:14.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8c7000640 1 -- 172.21.15.7:0/3891494887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c015fc10 con 0x7fe8c015dae0 2024-04-25T11:14:14.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.585+0000 7fe8c7000640 1 -- 172.21.15.7:0/3891494887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c01593f0 con 0x7fe8c015dae0 2024-04-25T11:14:14.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.587+0000 7fe8c7000640 1 -- 172.21.15.7:0/3891494887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c006a100 con 0x7fe8c015dae0 2024-04-25T11:14:14.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.588+0000 7fe8b7e00640 1 -- 172.21.15.7:0/3891494887 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe8b8047070 con 0x7fe8c015dae0 2024-04-25T11:14:14.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.589+0000 7fe8b7e00640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8a00744b0 0x7fe8a00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:14.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.589+0000 7fe8b7e00640 1 -- 172.21.15.7:0/3891494887 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5335+0+0 (secure 0 0 0) 0x7fe8b80b5470 con 0x7fe8c015dae0 2024-04-25T11:14:14.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.589+0000 7fe8bfe00640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8a00744b0 0x7fe8a00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.590+0000 7fe8bfe00640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8a00744b0 0x7fe8a00769c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0072200 tx=0x7fe8b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:14.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.594+0000 7fe8b7e00640 1 -- 172.21.15.7:0/3891494887 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe8c015def0 con 0x7fe8c015dae0 2024-04-25T11:14:14.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.841+0000 7fe8c7000640 1 -- 172.21.15.7:0/3891494887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8c00d0f30 con 0x7fe8c015dae0 2024-04-25T11:14:14.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.841+0000 7fe8b7e00640 1 -- 172.21.15.7:0/3891494887 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3860 (secure 0 0 0) 0x7fe8b807fcd0 con 0x7fe8c015dae0 2024-04-25T11:14:14.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:14.842 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5006,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2225639104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2225639104},{"type":"v1","addr":"172.21.15.87:6834","nonce":2225639104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2625900548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2625900548},{"type":"v1","addr":"172.21.15.7:6837","nonce":2625900548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5320,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/717352569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":717352569},{"type":"v1","addr":"172.21.15.87:6835","nonce":717352569}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":6045,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1284645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1284645906},{"type":"v1","addr":"172.21.15.7:6836","nonce":1284645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-04-25T11:14:14.842 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 29 2024-04-25T11:14:14.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.844+0000 7fe8c7000640 1 -- 172.21.15.7:0/3891494887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8a00744b0 msgr2=0x7fe8a00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:14.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.844+0000 7fe8c7000640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8a00744b0 0x7fe8a00769c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0072200 tx=0x7fe8b0008040 comp rx=0 tx=0).stop 2024-04-25T11:14:14.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.844+0000 7fe8c7000640 1 -- 172.21.15.7:0/3891494887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8c015dae0 msgr2=0x7fe8c01ede60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:14.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.844+0000 7fe8c7000640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8c015dae0 0x7fe8c01ede60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe8b802b500 tx=0x7fe8c0165520 comp rx=0 tx=0).stop 2024-04-25T11:14:14.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.844+0000 7fe8c7000640 1 -- 172.21.15.7:0/3891494887 shutdown_connections 2024-04-25T11:14:14.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.844+0000 7fe8c7000640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8a00744b0 0x7fe8a00769c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:14.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.844+0000 7fe8c7000640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8c015dae0 0x7fe8c01ede60 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:14.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.844+0000 7fe8c7000640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8c015c9b0 0x7fe8c015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:14.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.844+0000 7fe8c7000640 1 --2- 172.21.15.7:0/3891494887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8a4004d40 0x7fe8c014cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:14.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.844+0000 7fe8c7000640 1 -- 172.21.15.7:0/3891494887 >> 172.21.15.7:0/3891494887 conn(0x7fe8c0074200 msgr2=0x7fe8c00b6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:14.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.844+0000 7fe8c7000640 1 -- 172.21.15.7:0/3891494887 shutdown_connections 2024-04-25T11:14:14.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:14.845+0000 7fe8c7000640 1 -- 172.21.15.7:0/3891494887 wait complete. 2024-04-25T11:14:14.871 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:14:14.872 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:14:14.872 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:14:14.872 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:14:14.872 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:14:14.872 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:14:14.872 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:14:14.873 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:14:14.873 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:14:14.873 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:14:14.873 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:14:14.873 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:14:14.873 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:14:14.873 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:14:14.873 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:14:14.874 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:14:14.901 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:14:14.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:14:14.912 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:14:14.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:14:14.913 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:14:14.913 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:14:14.922 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:14:14.922 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:14:14.959 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:14:14.960 DEBUG:teuthology.parallel:result is None 2024-04-25T11:14:14.966 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:14:14.966 DEBUG:teuthology.parallel:result is None 2024-04-25T11:14:14.969 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:14:14.970 DEBUG:teuthology.parallel:result is None 2024-04-25T11:14:14.976 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:14:14.977 DEBUG:teuthology.parallel:result is None 2024-04-25T11:14:14.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:14:15.195 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:14:15.195 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:14:15.208 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:14:15.208 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:14:15.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.235+0000 7fbab3c00640 1 Processor -- start 2024-04-25T11:14:15.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.235+0000 7fbab3c00640 1 -- start start 2024-04-25T11:14:15.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.238+0000 7fbab3c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbaac0b4750 0x7fbaac06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:15.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.238+0000 7fbab3c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0ce950 0x7fbaac0ced60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:15.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.238+0000 7fbab3c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaac0cf2a0 con 0x7fbaac0b76d0 2024-04-25T11:14:15.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.238+0000 7fbab3c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbaac0c89a0 con 0x7fbaac0b4750 2024-04-25T11:14:15.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.238+0000 7fbab3c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbaac0c8c40 con 0x7fbaac0ce950 2024-04-25T11:14:15.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.238+0000 7fbaabe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbaac0b4750 0x7fbaac06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:15.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.238+0000 7fbab1c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0ce950 0x7fbaac0ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:15.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.238+0000 7fbaabe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbaac0b4750 0x7fbaac06c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:43602/0 (socket says 172.21.15.7:43602) 2024-04-25T11:14:15.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.238+0000 7fbab1c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0ce950 0x7fbaac0ced60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:41126/0 (socket says 172.21.15.7:41126) 2024-04-25T11:14:15.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.238+0000 7fbaabe00640 1 -- 172.21.15.7:0/2877834195 learned_addr learned my addr 172.21.15.7:0/2877834195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:15.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.239+0000 7fbaabe00640 1 -- 172.21.15.7:0/2877834195 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0ce950 msgr2=0x7fbaac0ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:15.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.239+0000 7fbaabe00640 1 --2- 172.21.15.7:0/2877834195 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0ce950 0x7fbaac0ced60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:15.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.239+0000 7fbaabe00640 1 -- 172.21.15.7:0/2877834195 >> v1:172.21.15.87:6789/0 conn(0x7fbaac0b76d0 legacy=0x7fbaac161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:15.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.239+0000 7fbaabe00640 1 -- 172.21.15.7:0/2877834195 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac0c6fb0 con 0x7fbaac0b4750 2024-04-25T11:14:15.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.240+0000 7fbaabe00640 1 --2- 172.21.15.7:0/2877834195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac0b4750 0x7fbaac06c570 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009870 tx=0x7fba9c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ecfa02446f8b1f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.242+0000 7fbaab400640 1 -- 172.21.15.7:0/2877834195 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c037070 con 0x7fbaac0b4750 2024-04-25T11:14:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.242+0000 7fbaab400640 1 -- 172.21.15.7:0/2877834195 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9c032440 con 0x7fbaac0b4750 2024-04-25T11:14:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.242+0000 7fbaab400640 1 -- 172.21.15.7:0/2877834195 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c032810 con 0x7fbaac0b4750 2024-04-25T11:14:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.243+0000 7fbab3c00640 1 -- 172.21.15.7:0/2877834195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac0b4750 msgr2=0x7fbaac06c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.243+0000 7fbab3c00640 1 --2- 172.21.15.7:0/2877834195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac0b4750 0x7fbaac06c570 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009870 tx=0x7fba9c02a250 comp rx=0 tx=0).stop 2024-04-25T11:14:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.243+0000 7fbab3c00640 1 -- 172.21.15.7:0/2877834195 shutdown_connections 2024-04-25T11:14:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.243+0000 7fbab3c00640 1 --2- 172.21.15.7:0/2877834195 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0ce950 0x7fbaac0ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.243+0000 7fbab3c00640 1 --2- 172.21.15.7:0/2877834195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac0b4750 0x7fbaac06c570 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.243+0000 7fbab3c00640 1 -- 172.21.15.7:0/2877834195 >> 172.21.15.7:0/2877834195 conn(0x7fbaac09b540 msgr2=0x7fbaac0b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.243+0000 7fbab3c00640 1 -- 172.21.15.7:0/2877834195 shutdown_connections 2024-04-25T11:14:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.243+0000 7fbab3c00640 1 -- 172.21.15.7:0/2877834195 wait complete. 2024-04-25T11:14:15.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.246+0000 7fbab3c00640 1 Processor -- start 2024-04-25T11:14:15.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.246+0000 7fbab3c00640 1 -- start start 2024-04-25T11:14:15.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.248+0000 7fbab3c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0b4750 0x7fbaac15d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:15.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.248+0000 7fbab3c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac0b76d0 0x7fbaac159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:15.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.248+0000 7fbab3c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaac0ce950 0x7fbaac160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:15.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.248+0000 7fbab3c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac0c7b80 con 0x7fbaac0ce950 2024-04-25T11:14:15.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.248+0000 7fbab3c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac0c7a00 con 0x7fbaac0b76d0 2024-04-25T11:14:15.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.248+0000 7fbab3c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbaac0c7d00 con 0x7fbaac0b4750 2024-04-25T11:14:15.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.249+0000 7fbab1200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0b4750 0x7fbaac15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:15.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.249+0000 7fbaabe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac0b76d0 0x7fbaac159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:15.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.249+0000 7fbab1200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0b4750 0x7fbaac15d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:41130/0 (socket says 172.21.15.7:41130) 2024-04-25T11:14:15.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.249+0000 7fbab1200640 1 -- 172.21.15.7:0/1015226159 learned_addr learned my addr 172.21.15.7:0/1015226159 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:15.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.249+0000 7fbab1c00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaac0ce950 0x7fbaac160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:15.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.249+0000 7fbab1200640 1 -- 172.21.15.7:0/1015226159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac0b76d0 msgr2=0x7fbaac159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:15.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.249+0000 7fbab1200640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac0b76d0 0x7fbaac159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:15.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.250+0000 7fbab1200640 1 -- 172.21.15.7:0/1015226159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaac0ce950 msgr2=0x7fbaac160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:15.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.250+0000 7fbab1200640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaac0ce950 0x7fbaac160ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:15.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.250+0000 7fbab1200640 1 -- 172.21.15.7:0/1015226159 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac15e190 con 0x7fbaac0b4750 2024-04-25T11:14:15.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.250+0000 7fbaabe00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac0b76d0 0x7fbaac159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:14:15.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.250+0000 7fbab1c00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaac0ce950 0x7fbaac160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:14:15.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.250+0000 7fbab1200640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0b4750 0x7fbaac15d890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000c3f0 tx=0x7fbaa000c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:15.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.252+0000 7fbaa8c00640 1 -- 172.21.15.7:0/1015226159 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00052d0 con 0x7fbaac0b4750 2024-04-25T11:14:15.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.252+0000 7fbaa8c00640 1 -- 172.21.15.7:0/1015226159 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaa0005490 con 0x7fbaac0b4750 2024-04-25T11:14:15.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.252+0000 7fbaa8c00640 1 -- 172.21.15.7:0/1015226159 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0007260 con 0x7fbaac0b4750 2024-04-25T11:14:15.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.252+0000 7fbab3c00640 1 -- 172.21.15.7:0/1015226159 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaac15e470 con 0x7fbaac0b4750 2024-04-25T11:14:15.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.252+0000 7fbab3c00640 1 -- 172.21.15.7:0/1015226159 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaac15ea30 con 0x7fbaac0b4750 2024-04-25T11:14:15.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.253+0000 7fbab3c00640 1 -- 172.21.15.7:0/1015226159 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaac0b7ae0 con 0x7fbaac0b4750 2024-04-25T11:14:15.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.255+0000 7fbaa8c00640 1 -- 172.21.15.7:0/1015226159 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbaa0007400 con 0x7fbaac0b4750 2024-04-25T11:14:15.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.256+0000 7fbaa8c00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fba900744b0 0x7fba900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:15.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.256+0000 7fbaa8c00640 1 -- 172.21.15.7:0/1015226159 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 5335+0+0 (secure 0 0 0) 0x7fbaa0094a30 con 0x7fbaac0b4750 2024-04-25T11:14:15.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.256+0000 7fbaabe00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fba900744b0 0x7fba900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:15.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.257+0000 7fbaabe00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fba900744b0 0x7fba900769c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fba9c034000 tx=0x7fba9c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.260+0000 7fbaa8c00640 1 -- 172.21.15.7:0/1015226159 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbaac0b7ae0 con 0x7fbaac0b4750 2024-04-25T11:14:15.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:15.480+0000 7fbab3c00640 1 -- 172.21.15.7:0/1015226159 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbaac050740 con 0x7fbaac0b4750 2024-04-25T11:14:16.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.263+0000 7fbaa8c00640 1 -- 172.21.15.7:0/1015226159 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) ==== 97+0+0 (secure 0 0 0) 0x7fbaac0cf2a0 con 0x7fbaac0b4750 2024-04-25T11:14:16.263 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-04-25T11:14:16.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 -- 172.21.15.7:0/1015226159 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fba900744b0 msgr2=0x7fba900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:16.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fba900744b0 0x7fba900769c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fba9c034000 tx=0x7fba9c02e000 comp rx=0 tx=0).stop 2024-04-25T11:14:16.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 -- 172.21.15.7:0/1015226159 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0b4750 msgr2=0x7fbaac15d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:16.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0b4750 0x7fbaac15d890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000c3f0 tx=0x7fbaa000c8b0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 -- 172.21.15.7:0/1015226159 shutdown_connections 2024-04-25T11:14:16.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fba900744b0 0x7fba900769c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbaac0ce950 0x7fbaac160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbaac0b76d0 0x7fbaac159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 --2- 172.21.15.7:0/1015226159 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbaac0b4750 0x7fbaac15d890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 -- 172.21.15.7:0/1015226159 >> 172.21.15.7:0/1015226159 conn(0x7fbaac09b540 msgr2=0x7fbaac06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:16.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 -- 172.21.15.7:0/1015226159 shutdown_connections 2024-04-25T11:14:16.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.267+0000 7fbab3c00640 1 -- 172.21.15.7:0/1015226159 wait complete. 2024-04-25T11:14:16.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:14:16.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.581+0000 7f5687200640 1 Processor -- start 2024-04-25T11:14:16.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.581+0000 7f5687200640 1 -- start start 2024-04-25T11:14:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.583+0000 7f5687200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f568814ab20 0x7f568806f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.583+0000 7f5687200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f568806dc90 0x7f56880b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.583+0000 7f5687200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f568806f920 con 0x7f5688168ac0 2024-04-25T11:14:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.583+0000 7f5687200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f568806fc00 con 0x7f568814ab20 2024-04-25T11:14:16.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.583+0000 7f5687200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f568806fee0 con 0x7f568806dc90 2024-04-25T11:14:16.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.584+0000 7f5685e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f568814ab20 0x7f568806f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:16.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.584+0000 7f5685400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5688168ac0 0x7f568806d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43546/0 (socket says 172.21.15.7:43546) 2024-04-25T11:14:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.584+0000 7f5685400640 1 -- 172.21.15.7:0/1335270759 learned_addr learned my addr 172.21.15.7:0/1335270759 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.584+0000 7f5686800640 1 --2- 172.21.15.7:0/1335270759 >> v2:172.21.15.87:3300/0 conn(0x7f568806dc90 0x7f56880b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.584+0000 7f5685e00640 1 -- 172.21.15.7:0/1335270759 >> v2:172.21.15.87:3300/0 conn(0x7f568806dc90 msgr2=0x7f56880b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.584+0000 7f5685e00640 1 --2- 172.21.15.7:0/1335270759 >> v2:172.21.15.87:3300/0 conn(0x7f568806dc90 0x7f56880b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.584+0000 7f5685e00640 1 -- 172.21.15.7:0/1335270759 >> v1:172.21.15.7:6789/0 conn(0x7f5688168ac0 legacy=0x7f568806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.584+0000 7f5684a00640 1 -- 172.21.15.7:0/1335270759 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3257572580 0 0) 0x7f568806f920 con 0x7f5688168ac0 2024-04-25T11:14:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.585+0000 7f5685e00640 1 -- 172.21.15.7:0/1335270759 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56880b4690 con 0x7f568814ab20 2024-04-25T11:14:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.586+0000 7f5685e00640 1 --2- 172.21.15.7:0/1335270759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f568814ab20 0x7f568806f0f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f56881659f0 tx=0x7f567802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d756e42bb81fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:16.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.586+0000 7f5684a00640 1 -- 172.21.15.7:0/1335270759 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678037070 con 0x7f568814ab20 2024-04-25T11:14:16.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.586+0000 7f5684a00640 1 -- 172.21.15.7:0/1335270759 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5678032440 con 0x7f568814ab20 2024-04-25T11:14:16.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.586+0000 7f5684a00640 1 -- 172.21.15.7:0/1335270759 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567802ae60 con 0x7f568814ab20 2024-04-25T11:14:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.587+0000 7f5687200640 1 -- 172.21.15.7:0/1335270759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f568814ab20 msgr2=0x7f568806f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.587+0000 7f5687200640 1 --2- 172.21.15.7:0/1335270759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f568814ab20 0x7f568806f0f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f56881659f0 tx=0x7f567802a110 comp rx=0 tx=0).stop 2024-04-25T11:14:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.587+0000 7f5687200640 1 -- 172.21.15.7:0/1335270759 shutdown_connections 2024-04-25T11:14:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.587+0000 7f5687200640 1 --2- 172.21.15.7:0/1335270759 >> v2:172.21.15.87:3300/0 conn(0x7f568806dc90 0x7f56880b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.587+0000 7f5687200640 1 --2- 172.21.15.7:0/1335270759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f568814ab20 0x7f568806f0f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.587+0000 7f5687200640 1 -- 172.21.15.7:0/1335270759 >> 172.21.15.7:0/1335270759 conn(0x7f5688073090 msgr2=0x7f56880734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.587+0000 7f5687200640 1 -- 172.21.15.7:0/1335270759 shutdown_connections 2024-04-25T11:14:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.587+0000 7f5687200640 1 -- 172.21.15.7:0/1335270759 wait complete. 2024-04-25T11:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.589+0000 7f5687200640 1 Processor -- start 2024-04-25T11:14:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.589+0000 7f5687200640 1 -- start start 2024-04-25T11:14:16.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.591+0000 7f5687200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5688168ac0 0x7f56881fdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:16.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.591+0000 7f5687200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5688163f90 0x7f56881f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:16.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.591+0000 7f5687200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56881497a0 0x7f5688149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:16.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.591+0000 7f5687200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f56880b53d0 con 0x7f5688168ac0 2024-04-25T11:14:16.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.591+0000 7f5687200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f568806f630 con 0x7f56881497a0 2024-04-25T11:14:16.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.591+0000 7f5687200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f56880b5550 con 0x7f5688163f90 2024-04-25T11:14:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.591+0000 7f5685e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5688168ac0 0x7f56881fdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.591+0000 7f5685e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5688168ac0 0x7f56881fdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53692/0 (socket says 172.21.15.7:53692) 2024-04-25T11:14:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.591+0000 7f5685e00640 1 -- 172.21.15.7:0/2682061859 learned_addr learned my addr 172.21.15.7:0/2682061859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.592+0000 7f5686800640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56881497a0 0x7f5688149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.592+0000 7f5685e00640 1 -- 172.21.15.7:0/2682061859 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5688163f90 msgr2=0x7f56881f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.592+0000 7f5685e00640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5688163f90 0x7f56881f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.592+0000 7f5685e00640 1 -- 172.21.15.7:0/2682061859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56881497a0 msgr2=0x7f5688149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.592+0000 7f5685e00640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56881497a0 0x7f5688149bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.592+0000 7f5685e00640 1 -- 172.21.15.7:0/2682061859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568814a0f0 con 0x7f5688168ac0 2024-04-25T11:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.592+0000 7f5685e00640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5688168ac0 0x7f56881fdb10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5678009890 tx=0x7f56780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.592+0000 7f567e000640 1 -- 172.21.15.7:0/2682061859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678037040 con 0x7f5688168ac0 2024-04-25T11:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.593+0000 7f567e000640 1 -- 172.21.15.7:0/2682061859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5678042070 con 0x7f5688168ac0 2024-04-25T11:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.593+0000 7f567e000640 1 -- 172.21.15.7:0/2682061859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678030bd0 con 0x7f5688168ac0 2024-04-25T11:14:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.593+0000 7f5687200640 1 -- 172.21.15.7:0/2682061859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56881f99e0 con 0x7f5688168ac0 2024-04-25T11:14:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.593+0000 7f5687200640 1 -- 172.21.15.7:0/2682061859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5688163300 con 0x7f5688168ac0 2024-04-25T11:14:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.594+0000 7f5687200640 1 -- 172.21.15.7:0/2682061859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5688050740 con 0x7f5688168ac0 2024-04-25T11:14:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.595+0000 7f567e000640 1 -- 172.21.15.7:0/2682061859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5678047460 con 0x7f5688168ac0 2024-04-25T11:14:16.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.596+0000 7f567e000640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f56640744e0 0x7f56640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:16.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.596+0000 7f567e000640 1 -- 172.21.15.7:0/2682061859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5249+0+0 (secure 0 0 0) 0x7f56780b5a90 con 0x7f5688168ac0 2024-04-25T11:14:16.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.597+0000 7f5685400640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f56640744e0 0x7f56640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:16.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.597+0000 7f5685400640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f56640744e0 0x7f56640769f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5670007fd0 tx=0x7f5670007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.602+0000 7f567e000640 1 -- 172.21.15.7:0/2682061859 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f567804c020 con 0x7f5688168ac0 2024-04-25T11:14:16.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.884+0000 7f5687200640 1 -- 172.21.15.7:0/2682061859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f568806d720 con 0x7f5688168ac0 2024-04-25T11:14:16.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.885+0000 7f567e000640 1 -- 172.21.15.7:0/2682061859 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f568814a0f0 con 0x7f5688168ac0 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w==", 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw==", 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw==", 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A==", 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA==", 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg==", 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:14:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw==", 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ==", 2024-04-25T11:14:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw==", 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng==", 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ==", 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:14:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A==", 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg==", 2024-04-25T11:14:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA==", 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:14:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg==", 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ==", 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDZOSpmS8ObHxAAkXXXg1OI1YFC2/OZER3woA==", 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw==", 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w==", 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:14:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:14:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:14:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:14:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-25T11:14:16.898 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:14:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 -- 172.21.15.7:0/2682061859 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f56640744e0 msgr2=0x7f56640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f56640744e0 0x7f56640769f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5670007fd0 tx=0x7f5670007960 comp rx=0 tx=0).stop 2024-04-25T11:14:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 -- 172.21.15.7:0/2682061859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5688168ac0 msgr2=0x7f56881fdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5688168ac0 0x7f56881fdb10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5678009890 tx=0x7f56780046c0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 -- 172.21.15.7:0/2682061859 shutdown_connections 2024-04-25T11:14:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f56640744e0 0x7f56640769f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56881497a0 0x7f5688149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5688163f90 0x7f56881f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 --2- 172.21.15.7:0/2682061859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5688168ac0 0x7f56881fdb10 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:16.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 -- 172.21.15.7:0/2682061859 >> 172.21.15.7:0/2682061859 conn(0x7f5688073090 msgr2=0x7f5688166f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:16.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 -- 172.21.15.7:0/2682061859 shutdown_connections 2024-04-25T11:14:16.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:16.888+0000 7f5687200640 1 -- 172.21.15.7:0/2682061859 wait complete. 2024-04-25T11:14:16.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:17.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.208+0000 7f0e75200640 1 Processor -- start 2024-04-25T11:14:17.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.208+0000 7f0e75200640 1 -- start start 2024-04-25T11:14:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.210+0000 7f0e75200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0e7015bc40 0x7f0e7009f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.210+0000 7f0e75200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e7015f4e0 con 0x7f0e700af540 2024-04-25T11:14:17.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.210+0000 7f0e75200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e700b0560 con 0x7f0e7015ef90 2024-04-25T11:14:17.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.210+0000 7f0e75200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0e70071190 con 0x7f0e7015bc40 2024-04-25T11:14:17.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.211+0000 7f0e6d400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0e700af540 0x7f0e700cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43554/0 (socket says 172.21.15.7:43554) 2024-04-25T11:14:17.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.211+0000 7f0e6d400640 1 -- 172.21.15.7:0/3982665715 learned_addr learned my addr 172.21.15.7:0/3982665715 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.211+0000 7f0e6e800640 1 --2- 172.21.15.7:0/3982665715 >> v2:172.21.15.87:3300/0 conn(0x7f0e7015bc40 0x7f0e7009f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.211+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1032095563 0 0) 0x7f0e7015f4e0 con 0x7f0e700af540 2024-04-25T11:14:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.211+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e54003520 con 0x7f0e700af540 2024-04-25T11:14:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 612920708 0 0) 0x7f0e54003520 con 0x7f0e700af540 2024-04-25T11:14:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 >> v2:172.21.15.87:3300/0 conn(0x7f0e7015bc40 msgr2=0x7f0e7009f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 --2- 172.21.15.7:0/3982665715 >> v2:172.21.15.87:3300/0 conn(0x7f0e7015bc40 0x7f0e7009f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 >> v1:172.21.15.87:6789/0 conn(0x7f0e7015ef90 legacy=0x7f0e700aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e700b0840 con 0x7f0e700af540 2024-04-25T11:14:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0e5c001670 con 0x7f0e700af540 2024-04-25T11:14:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 >> v1:172.21.15.7:6789/0 conn(0x7f0e700af540 legacy=0x7f0e700cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e540039b0 0x7f0e54003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e54004800 0x7f0e54004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e54005250 0x7f0e5400fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e70071190 con 0x7f0e540039b0 2024-04-25T11:14:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0e540105f0 con 0x7f0e54005250 2024-04-25T11:14:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0e540107f0 con 0x7f0e54004800 2024-04-25T11:14:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6de00640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e540039b0 0x7f0e54003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6e800640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e54005250 0x7f0e5400fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.212+0000 7f0e6d400640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e54004800 0x7f0e54004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.213+0000 7f0e6de00640 1 -- 172.21.15.7:0/3982665715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e54004800 msgr2=0x7f0e54004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.213+0000 7f0e6de00640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e54004800 0x7f0e54004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.213+0000 7f0e6de00640 1 -- 172.21.15.7:0/3982665715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e54005250 msgr2=0x7f0e5400fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.213+0000 7f0e6de00640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e54005250 0x7f0e5400fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.213+0000 7f0e6de00640 1 -- 172.21.15.7:0/3982665715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e54003520 con 0x7f0e540039b0 2024-04-25T11:14:17.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.213+0000 7f0e6e800640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e54005250 0x7f0e5400fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:17.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.213+0000 7f0e6e800640 1 -- 172.21.15.7:0/3982665715 reap_dead start 2024-04-25T11:14:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.214+0000 7f0e6de00640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e540039b0 0x7f0e54003e20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0e60001750 tx=0x7f0e54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820a0eb73c7236d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.214+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60001370 con 0x7f0e540039b0 2024-04-25T11:14:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.214+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e600052d0 con 0x7f0e540039b0 2024-04-25T11:14:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.214+0000 7f0e6ca00640 1 -- 172.21.15.7:0/3982665715 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6002abc0 con 0x7f0e540039b0 2024-04-25T11:14:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.215+0000 7f0e75200640 1 -- 172.21.15.7:0/3982665715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e540039b0 msgr2=0x7f0e54003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.215+0000 7f0e75200640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e540039b0 0x7f0e54003e20 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0e60001750 tx=0x7f0e54005830 comp rx=0 tx=0).stop 2024-04-25T11:14:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.215+0000 7f0e75200640 1 -- 172.21.15.7:0/3982665715 shutdown_connections 2024-04-25T11:14:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.215+0000 7f0e75200640 1 --2- 172.21.15.7:0/3982665715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e540039b0 0x7f0e54003e20 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.215+0000 7f0e75200640 1 -- 172.21.15.7:0/3982665715 >> 172.21.15.7:0/3982665715 conn(0x7f0e70074200 msgr2=0x7f0e7006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.215+0000 7f0e75200640 1 -- 172.21.15.7:0/3982665715 shutdown_connections 2024-04-25T11:14:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.215+0000 7f0e75200640 1 -- 172.21.15.7:0/3982665715 wait complete. 2024-04-25T11:14:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.217+0000 7f0e75200640 1 Processor -- start 2024-04-25T11:14:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.218+0000 7f0e75200640 1 -- start start 2024-04-25T11:14:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.219+0000 7f0e75200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e540039b0 0x7f0e7006d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.219+0000 7f0e75200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e7006d860 0x7f0e701602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.219+0000 7f0e75200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e70160820 0x7f0e70144f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.219+0000 7f0e75200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e700b1320 con 0x7f0e70160820 2024-04-25T11:14:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.219+0000 7f0e75200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0e700b0e00 con 0x7f0e7006d860 2024-04-25T11:14:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.219+0000 7f0e75200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0e700afb80 con 0x7f0e540039b0 2024-04-25T11:14:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.219+0000 7f0e6e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e70160820 0x7f0e70144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.219+0000 7f0e6e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e70160820 0x7f0e70144f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53702/0 (socket says 172.21.15.7:53702) 2024-04-25T11:14:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.220+0000 7f0e6e800640 1 -- 172.21.15.7:0/1048938658 learned_addr learned my addr 172.21.15.7:0/1048938658 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.220+0000 7f0e6de00640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e540039b0 0x7f0e7006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.220+0000 7f0e6d400640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e7006d860 0x7f0e701602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.220+0000 7f0e6e800640 1 -- 172.21.15.7:0/1048938658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e540039b0 msgr2=0x7f0e7006d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.220+0000 7f0e6e800640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e540039b0 0x7f0e7006d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.220+0000 7f0e6e800640 1 -- 172.21.15.7:0/1048938658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e7006d860 msgr2=0x7f0e701602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.220+0000 7f0e6e800640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e7006d860 0x7f0e701602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.220+0000 7f0e6e800640 1 -- 172.21.15.7:0/1048938658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e70156270 con 0x7f0e70160820 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.220+0000 7f0e6e800640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e70160820 0x7f0e70144f00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0e700ce140 tx=0x7f0e6800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.221+0000 7f0e66000640 1 -- 172.21.15.7:0/1048938658 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68016620 con 0x7f0e70160820 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.221+0000 7f0e66000640 1 -- 172.21.15.7:0/1048938658 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6801b070 con 0x7f0e70160820 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.221+0000 7f0e66000640 1 -- 172.21.15.7:0/1048938658 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e680169c0 con 0x7f0e70160820 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.221+0000 7f0e75200640 1 -- 172.21.15.7:0/1048938658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e701554b0 con 0x7f0e70160820 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.221+0000 7f0e75200640 1 -- 172.21.15.7:0/1048938658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e70155a70 con 0x7f0e70160820 2024-04-25T11:14:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.222+0000 7f0e75200640 1 -- 172.21.15.7:0/1048938658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e70160e10 con 0x7f0e70160820 2024-04-25T11:14:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.223+0000 7f0e66000640 1 -- 172.21.15.7:0/1048938658 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0e68005d10 con 0x7f0e70160820 2024-04-25T11:14:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.225+0000 7f0e66000640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e500744b0 0x7f0e500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.225+0000 7f0e66000640 1 -- 172.21.15.7:0/1048938658 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5249+0+0 (secure 0 0 0) 0x7f0e68095c50 con 0x7f0e70160820 2024-04-25T11:14:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.225+0000 7f0e6de00640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e500744b0 0x7f0e500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.226+0000 7f0e6de00640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e500744b0 0x7f0e500769c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0e60032000 tx=0x7f0e6002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:17.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.229+0000 7f0e66000640 1 -- 172.21.15.7:0/1048938658 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e68025020 con 0x7f0e70160820 2024-04-25T11:14:17.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.442+0000 7f0e75200640 1 -- 172.21.15.7:0/1048938658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e7006a100 con 0x7f0e70160820 2024-04-25T11:14:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.444+0000 7f0e66000640 1 -- 172.21.15.7:0/1048938658 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+13068 (secure 0 0 0) 0x7f0e68060590 con 0x7f0e70160820 2024-04-25T11:14:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:17.445 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":64,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:16.249577+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:17.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 -- 172.21.15.7:0/1048938658 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e500744b0 msgr2=0x7f0e500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e500744b0 0x7f0e500769c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0e60032000 tx=0x7f0e6002e000 comp rx=0 tx=0).stop 2024-04-25T11:14:17.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 -- 172.21.15.7:0/1048938658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e70160820 msgr2=0x7f0e70144f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e70160820 0x7f0e70144f00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0e700ce140 tx=0x7f0e6800f520 comp rx=0 tx=0).stop 2024-04-25T11:14:17.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 -- 172.21.15.7:0/1048938658 shutdown_connections 2024-04-25T11:14:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e500744b0 0x7f0e500769c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e70160820 0x7f0e70144f00 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e7006d860 0x7f0e701602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 --2- 172.21.15.7:0/1048938658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e540039b0 0x7f0e7006d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 -- 172.21.15.7:0/1048938658 >> 172.21.15.7:0/1048938658 conn(0x7f0e70074200 msgr2=0x7f0e700b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 -- 172.21.15.7:0/1048938658 shutdown_connections 2024-04-25T11:14:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.447+0000 7f0e75200640 1 -- 172.21.15.7:0/1048938658 wait complete. 2024-04-25T11:14:17.475 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:14:17.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:14:17.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.764+0000 7f0100000640 1 Processor -- start 2024-04-25T11:14:17.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.765+0000 7f0100000640 1 -- start start 2024-04-25T11:14:17.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.767+0000 7f0100000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00f806fab0 0x7f00f8168e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.767+0000 7f0100000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f815e3c0 con 0x7f00f815dfb0 2024-04-25T11:14:17.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.767+0000 7f0100000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f80cf800 con 0x7f00f80b0470 2024-04-25T11:14:17.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.767+0000 7f0100000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f00f80d1250 con 0x7f00f806fab0 2024-04-25T11:14:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.767+0000 7f00fd600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f00f815dfb0 0x7f00f80d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43568/0 (socket says 172.21.15.7:43568) 2024-04-25T11:14:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.767+0000 7f00fd600640 1 -- 172.21.15.7:0/977761441 learned_addr learned my addr 172.21.15.7:0/977761441 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.768+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 396873645 0 0) 0x7f00f815e3c0 con 0x7f00f815dfb0 2024-04-25T11:14:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.768+0000 7f00fe000640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00f806fab0 0x7f00f8168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.768+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00e0003520 con 0x7f00f815dfb0 2024-04-25T11:14:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.768+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2315434235 0 0) 0x7f00e0003520 con 0x7f00f815dfb0 2024-04-25T11:14:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.768+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00f806fab0 msgr2=0x7f00f8168e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.768+0000 7f00f7e00640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00f806fab0 0x7f00f8168e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.768+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 >> v1:172.21.15.87:6789/0 conn(0x7f00f80b0470 legacy=0x7f00f806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.768+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f80cfaa0 con 0x7f00f815dfb0 2024-04-25T11:14:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.768+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f00ec001670 con 0x7f00f815dfb0 2024-04-25T11:14:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.768+0000 7f00fe000640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00f806fab0 0x7f00f8168e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.768+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 >> v1:172.21.15.7:6789/0 conn(0x7f00f815dfb0 legacy=0x7f00f80d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00f7e00640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00f806fab0 0x7f00e00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00f7e00640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e0003ff0 0x7f00e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00f7e00640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e0004d40 0x7f00e0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00f80d1250 con 0x7f00e0003ff0 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f00e000bbd0 con 0x7f00e0004d40 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f00e000bdd0 con 0x7f00f806fab0 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00fcc00640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e0003ff0 0x7f00e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00fd600640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00f806fab0 0x7f00e00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00fe000640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e0004d40 0x7f00e0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00fcc00640 1 -- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00f806fab0 msgr2=0x7f00e00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00fcc00640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00f806fab0 0x7f00e00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00fcc00640 1 -- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e0004d40 msgr2=0x7f00e0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:17.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00fcc00640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e0004d40 0x7f00e0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00fcc00640 1 -- 172.21.15.7:0/977761441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e0003520 con 0x7f00e0003ff0 2024-04-25T11:14:17.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.769+0000 7f00fe000640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e0004d40 0x7f00e0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:17.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.770+0000 7f00fcc00640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e0003ff0 0x7f00e0004800 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f00f80b6c90 tx=0x7f00e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b318b5496c6d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:17.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.771+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4001370 con 0x7f00e0003ff0 2024-04-25T11:14:17.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.771+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e40052d0 con 0x7f00e0003ff0 2024-04-25T11:14:17.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.771+0000 7f00f7e00640 1 -- 172.21.15.7:0/977761441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e402ab90 con 0x7f00e0003ff0 2024-04-25T11:14:17.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.771+0000 7f0100000640 1 -- 172.21.15.7:0/977761441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e0003ff0 msgr2=0x7f00e0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.771+0000 7f0100000640 1 --2- 172.21.15.7:0/977761441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00e0003ff0 0x7f00e0004800 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f00f80b6c90 tx=0x7f00e0005830 comp rx=0 tx=0).stop 2024-04-25T11:14:17.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.771+0000 7f00fe000640 1 -- 172.21.15.7:0/977761441 reap_dead start 2024-04-25T11:14:17.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.772+0000 7f0100000640 1 -- 172.21.15.7:0/977761441 shutdown_connections 2024-04-25T11:14:17.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.772+0000 7f0100000640 1 -- 172.21.15.7:0/977761441 >> 172.21.15.7:0/977761441 conn(0x7f00f80742c0 msgr2=0x7f00f806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:17.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.772+0000 7f0100000640 1 -- 172.21.15.7:0/977761441 shutdown_connections 2024-04-25T11:14:17.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.772+0000 7f0100000640 1 -- 172.21.15.7:0/977761441 wait complete. 2024-04-25T11:14:17.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.774+0000 7f0100000640 1 Processor -- start 2024-04-25T11:14:17.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.774+0000 7f0100000640 1 -- start start 2024-04-25T11:14:17.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.776+0000 7f0100000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0003ff0 0x7f00f8149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.776+0000 7f0100000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81e97a0 0x7f00f81604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.776+0000 7f0100000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00f809b9d0 0x7f00f809bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.776+0000 7f0100000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00f80d0340 con 0x7f00f81e97a0 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.776+0000 7f0100000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f00f80d0060 con 0x7f00f809b9d0 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.776+0000 7f0100000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f00f80d0620 con 0x7f00e0003ff0 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.776+0000 7f00fcc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81e97a0 0x7f00f81604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.776+0000 7f00fcc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81e97a0 0x7f00f81604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53722/0 (socket says 172.21.15.7:53722) 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.776+0000 7f00fcc00640 1 -- 172.21.15.7:0/1935053818 learned_addr learned my addr 172.21.15.7:0/1935053818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.777+0000 7f00fe000640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00f809b9d0 0x7f00f809bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.777+0000 7f00fd600640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0003ff0 0x7f00f8149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.777+0000 7f00fcc00640 1 -- 172.21.15.7:0/1935053818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0003ff0 msgr2=0x7f00f8149ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.777+0000 7f00fcc00640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0003ff0 0x7f00f8149ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.777+0000 7f00fcc00640 1 -- 172.21.15.7:0/1935053818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00f809b9d0 msgr2=0x7f00f809bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:17.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.777+0000 7f00fcc00640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00f809b9d0 0x7f00f809bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:17.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.777+0000 7f00fcc00640 1 -- 172.21.15.7:0/1935053818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f81e8700 con 0x7f00f81e97a0 2024-04-25T11:14:17.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.777+0000 7f00fd600640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0003ff0 0x7f00f8149ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.777+0000 7f00fcc00640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81e97a0 0x7f00f81604f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f00e402a540 tx=0x7f00e4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.777+0000 7f00f5600640 1 -- 172.21.15.7:0/1935053818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4037040 con 0x7f00f81e97a0 2024-04-25T11:14:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.777+0000 7f00f5600640 1 -- 172.21.15.7:0/1935053818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e4042070 con 0x7f00f81e97a0 2024-04-25T11:14:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.778+0000 7f00f5600640 1 -- 172.21.15.7:0/1935053818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4007a30 con 0x7f00f81e97a0 2024-04-25T11:14:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.778+0000 7f0100000640 1 -- 172.21.15.7:0/1935053818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f81e89e0 con 0x7f00f81e97a0 2024-04-25T11:14:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.778+0000 7f0100000640 1 -- 172.21.15.7:0/1935053818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f81e8fa0 con 0x7f00f81e97a0 2024-04-25T11:14:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.779+0000 7f0100000640 1 -- 172.21.15.7:0/1935053818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f8050740 con 0x7f00f81e97a0 2024-04-25T11:14:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.780+0000 7f00f5600640 1 -- 172.21.15.7:0/1935053818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f00e40052d0 con 0x7f00f81e97a0 2024-04-25T11:14:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.782+0000 7f00f5600640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00d80744b0 0x7f00d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.782+0000 7f00f5600640 1 -- 172.21.15.7:0/1935053818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5249+0+0 (secure 0 0 0) 0x7f00e40b53d0 con 0x7f00f81e97a0 2024-04-25T11:14:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.782+0000 7f00fd600640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00d80744b0 0x7f00d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.783+0000 7f00fd600640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00d80744b0 0x7f00d80769c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f00ec001cc0 tx=0x7f00ec001d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.787+0000 7f00f5600640 1 -- 172.21.15.7:0/1935053818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f00e407fda0 con 0x7f00f81e97a0 2024-04-25T11:14:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.996+0000 7f0100000640 1 -- 172.21.15.7:0/1935053818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f00f8070fb0 con 0x7f00f81e97a0 2024-04-25T11:14:17.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.997+0000 7f00f5600640 1 -- 172.21.15.7:0/1935053818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) ==== 93+0+10 (secure 0 0 0) 0x7f00e407f6a0 con 0x7f00f81e97a0 2024-04-25T11:14:17.998 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:14:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.999+0000 7f0100000640 1 -- 172.21.15.7:0/1935053818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00d80744b0 msgr2=0x7f00d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.999+0000 7f0100000640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00d80744b0 0x7f00d80769c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f00ec001cc0 tx=0x7f00ec001d70 comp rx=0 tx=0).stop 2024-04-25T11:14:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.999+0000 7f0100000640 1 -- 172.21.15.7:0/1935053818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81e97a0 msgr2=0x7f00f81604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.999+0000 7f0100000640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81e97a0 0x7f00f81604f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f00e402a540 tx=0x7f00e4001120 comp rx=0 tx=0).stop 2024-04-25T11:14:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.999+0000 7f0100000640 1 -- 172.21.15.7:0/1935053818 shutdown_connections 2024-04-25T11:14:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.999+0000 7f0100000640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f00d80744b0 0x7f00d80769c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.999+0000 7f0100000640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00f809b9d0 0x7f00f809bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.999+0000 7f0100000640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00f81e97a0 0x7f00f81604f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.999+0000 7f0100000640 1 --2- 172.21.15.7:0/1935053818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0003ff0 0x7f00f8149ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:17.999+0000 7f0100000640 1 -- 172.21.15.7:0/1935053818 >> 172.21.15.7:0/1935053818 conn(0x7f00f80742c0 msgr2=0x7f00f806d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.000+0000 7f0100000640 1 -- 172.21.15.7:0/1935053818 shutdown_connections 2024-04-25T11:14:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.000+0000 7f0100000640 1 -- 172.21.15.7:0/1935053818 wait complete. 2024-04-25T11:14:18.028 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:14:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.314+0000 7fa7b8a00640 1 Processor -- start 2024-04-25T11:14:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.314+0000 7fa7b8a00640 1 -- start start 2024-04-25T11:14:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.316+0000 7fa7b8a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b4160a10 0x7fa7b40d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.316+0000 7fa7b8a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b4160e20 con 0x7fa7b40cf710 2024-04-25T11:14:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.316+0000 7fa7b8a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b40b4810 con 0x7fa7b406d580 2024-04-25T11:14:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.316+0000 7fa7b8a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b40d12b0 con 0x7fa7b4160a10 2024-04-25T11:14:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.317+0000 7fa7b3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa7b40cf710 0x7fa7b40cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43584/0 (socket says 172.21.15.7:43584) 2024-04-25T11:14:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.317+0000 7fa7b3e00640 1 -- 172.21.15.7:0/2534417903 learned_addr learned my addr 172.21.15.7:0/2534417903 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.317+0000 7fa7b3400640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b4160a10 0x7fa7b40d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.317+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3734111491 0 0) 0x7fa7b4160e20 con 0x7fa7b40cf710 2024-04-25T11:14:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.317+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa798003520 con 0x7fa7b40cf710 2024-04-25T11:14:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3725452782 0 0) 0x7fa798003520 con 0x7fa7b40cf710 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b4160a10 msgr2=0x7fa7b40d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b4160a10 0x7fa7b40d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 >> v1:172.21.15.87:6789/0 conn(0x7fa7b406d580 legacy=0x7fa7b406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b40b4af0 con 0x7fa7b40cf710 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fa7ac001670 con 0x7fa7b40cf710 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 >> v1:172.21.15.7:6789/0 conn(0x7fa7b40cf710 legacy=0x7fa7b40cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b4160a10 0x7fa7980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa798003fb0 0x7fa798004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa798004d40 0x7fa7980051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa79800bb80 con 0x7fa798003fb0 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa79800bd30 con 0x7fa798004d40 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa79800bf30 con 0x7fa7b4160a10 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.318+0000 7fa7b2a00640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa798003fb0 0x7fa798004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.319+0000 7fa7b3e00640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa798004d40 0x7fa7980051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.319+0000 7fa7b3400640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b4160a10 0x7fa7980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.319+0000 7fa7b2a00640 1 -- 172.21.15.7:0/2534417903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b4160a10 msgr2=0x7fa7980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.319+0000 7fa7b2a00640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b4160a10 0x7fa7980039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.319+0000 7fa7b2a00640 1 -- 172.21.15.7:0/2534417903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa798004d40 msgr2=0x7fa7980051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.319+0000 7fa7b2a00640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa798004d40 0x7fa7980051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.319+0000 7fa7b2a00640 1 -- 172.21.15.7:0/2534417903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa798003520 con 0x7fa798003fb0 2024-04-25T11:14:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.320+0000 7fa7b2a00640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa798003fb0 0x7fa798004800 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40715a0 tx=0x7fa798005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818158bd1fceae04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.320+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0001390 con 0x7fa798003fb0 2024-04-25T11:14:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.320+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a00052d0 con 0x7fa798003fb0 2024-04-25T11:14:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.320+0000 7fa7b2000640 1 -- 172.21.15.7:0/2534417903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a002a9d0 con 0x7fa798003fb0 2024-04-25T11:14:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.321+0000 7fa7b8a00640 1 -- 172.21.15.7:0/2534417903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa798003fb0 msgr2=0x7fa798004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.321+0000 7fa7b8a00640 1 --2- 172.21.15.7:0/2534417903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa798003fb0 0x7fa798004800 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa7b40715a0 tx=0x7fa798005830 comp rx=0 tx=0).stop 2024-04-25T11:14:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.321+0000 7fa7b3e00640 1 -- 172.21.15.7:0/2534417903 reap_dead start 2024-04-25T11:14:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.321+0000 7fa7b8a00640 1 -- 172.21.15.7:0/2534417903 shutdown_connections 2024-04-25T11:14:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.321+0000 7fa7b8a00640 1 -- 172.21.15.7:0/2534417903 >> 172.21.15.7:0/2534417903 conn(0x7fa7b4073090 msgr2=0x7fa7b40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.321+0000 7fa7b8a00640 1 -- 172.21.15.7:0/2534417903 shutdown_connections 2024-04-25T11:14:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.321+0000 7fa7b8a00640 1 -- 172.21.15.7:0/2534417903 wait complete. 2024-04-25T11:14:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.324+0000 7fa7b8a00640 1 Processor -- start 2024-04-25T11:14:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.324+0000 7fa7b8a00640 1 -- start start 2024-04-25T11:14:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.325+0000 7fa7b8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b414d460 0x7fa7b414ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.325+0000 7fa7b8a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b413ff60 0x7fa7b41e9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.325+0000 7fa7b8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7b41ea410 0x7fa7b414ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.325+0000 7fa7b8a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b415c650 con 0x7fa7b414d460 2024-04-25T11:14:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.325+0000 7fa7b8a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b40b50b0 con 0x7fa7b41ea410 2024-04-25T11:14:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.325+0000 7fa7b8a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b415c7d0 con 0x7fa7b413ff60 2024-04-25T11:14:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.326+0000 7fa7b3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b414d460 0x7fa7b414ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.326+0000 7fa7b3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b414d460 0x7fa7b414ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53734/0 (socket says 172.21.15.7:53734) 2024-04-25T11:14:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.326+0000 7fa7b3400640 1 -- 172.21.15.7:0/4093875860 learned_addr learned my addr 172.21.15.7:0/4093875860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.326+0000 7fa7b3400640 1 -- 172.21.15.7:0/4093875860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b413ff60 msgr2=0x7fa7b41e9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.326+0000 7fa7b3400640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b413ff60 0x7fa7b41e9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.326+0000 7fa7b3400640 1 -- 172.21.15.7:0/4093875860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7b41ea410 msgr2=0x7fa7b414ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.326+0000 7fa7b3400640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7b41ea410 0x7fa7b414ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.326+0000 7fa7b3400640 1 -- 172.21.15.7:0/4093875860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b41ea820 con 0x7fa7b414d460 2024-04-25T11:14:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.327+0000 7fa7b3400640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b414d460 0x7fa7b414ed60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400ae10 tx=0x7fa7a4010580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.327+0000 7fa7ab400640 1 -- 172.21.15.7:0/4093875860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4019030 con 0x7fa7b414d460 2024-04-25T11:14:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.327+0000 7fa7ab400640 1 -- 172.21.15.7:0/4093875860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a400a0d0 con 0x7fa7b414d460 2024-04-25T11:14:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.327+0000 7fa7ab400640 1 -- 172.21.15.7:0/4093875860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4003bd0 con 0x7fa7b414d460 2024-04-25T11:14:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.327+0000 7fa7b8a00640 1 -- 172.21.15.7:0/4093875860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b414d870 con 0x7fa7b414d460 2024-04-25T11:14:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.327+0000 7fa7b8a00640 1 -- 172.21.15.7:0/4093875860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b414e0e0 con 0x7fa7b414d460 2024-04-25T11:14:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.328+0000 7fa7b8a00640 1 -- 172.21.15.7:0/4093875860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b4050740 con 0x7fa7b414d460 2024-04-25T11:14:18.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.329+0000 7fa7ab400640 1 -- 172.21.15.7:0/4093875860 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa7a4005d10 con 0x7fa7b414d460 2024-04-25T11:14:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.330+0000 7fa7ab400640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7940744b0 0x7fa7940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.330+0000 7fa7ab400640 1 -- 172.21.15.7:0/4093875860 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5249+0+0 (secure 0 0 0) 0x7fa7a4095770 con 0x7fa7b414d460 2024-04-25T11:14:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.331+0000 7fa7b2a00640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7940744b0 0x7fa7940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.332+0000 7fa7b2a00640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7940744b0 0x7fa7940769c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0032000 tx=0x7fa7a002c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.335+0000 7fa7ab400640 1 -- 172.21.15.7:0/4093875860 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa7b41ea820 con 0x7fa7b414d460 2024-04-25T11:14:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.540+0000 7fa7b8a00640 1 -- 172.21.15.7:0/4093875860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa7b4070900 con 0x7fa7b414d460 2024-04-25T11:14:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.541+0000 7fa7ab400640 1 -- 172.21.15.7:0/4093875860 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) ==== 84+0+65 (secure 0 0 0) 0x7fa7a4011040 con 0x7fa7b414d460 2024-04-25T11:14:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:18.541 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-25T11:14:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:14:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-25T11:14:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-25T11:14:18.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:14:18.541 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-25T11:14:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.543+0000 7fa7b8a00640 1 -- 172.21.15.7:0/4093875860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7940744b0 msgr2=0x7fa7940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.543+0000 7fa7b8a00640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7940744b0 0x7fa7940769c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0032000 tx=0x7fa7a002c000 comp rx=0 tx=0).stop 2024-04-25T11:14:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.543+0000 7fa7b8a00640 1 -- 172.21.15.7:0/4093875860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b414d460 msgr2=0x7fa7b414ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.543+0000 7fa7b8a00640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b414d460 0x7fa7b414ed60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400ae10 tx=0x7fa7a4010580 comp rx=0 tx=0).stop 2024-04-25T11:14:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.543+0000 7fa7b8a00640 1 -- 172.21.15.7:0/4093875860 shutdown_connections 2024-04-25T11:14:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.543+0000 7fa7b8a00640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7940744b0 0x7fa7940769c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.543+0000 7fa7b8a00640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7b41ea410 0x7fa7b414ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.543+0000 7fa7b8a00640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b413ff60 0x7fa7b41e9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.543+0000 7fa7b8a00640 1 --2- 172.21.15.7:0/4093875860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b414d460 0x7fa7b414ed60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.543+0000 7fa7b8a00640 1 -- 172.21.15.7:0/4093875860 >> 172.21.15.7:0/4093875860 conn(0x7fa7b4073090 msgr2=0x7fa7b414bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.544+0000 7fa7b8a00640 1 -- 172.21.15.7:0/4093875860 shutdown_connections 2024-04-25T11:14:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.544+0000 7fa7b8a00640 1 -- 172.21.15.7:0/4093875860 wait complete. 2024-04-25T11:14:18.568 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:14:18.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:14:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.858+0000 7f87cf600640 1 Processor -- start 2024-04-25T11:14:18.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.859+0000 7f87cf600640 1 -- start start 2024-04-25T11:14:18.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.861+0000 7f87cf600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87c80b48a0 0x7f87c81620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.861+0000 7f87cf600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f87c8070db0 0x7f87c80711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.861+0000 7f87cf600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87c80b06a0 0x7f87c80b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.861+0000 7f87cf600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f87c8168ed0 con 0x7f87c80b48a0 2024-04-25T11:14:18.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.861+0000 7f87cf600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f87c809f9c0 con 0x7f87c8070db0 2024-04-25T11:14:18.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.861+0000 7f87cf600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87c81682c0 con 0x7f87c80b06a0 2024-04-25T11:14:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.862+0000 7f87ccc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87c80b48a0 0x7f87c81620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.862+0000 7f87ccc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87c80b48a0 0x7f87c81620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53744/0 (socket says 172.21.15.7:53744) 2024-04-25T11:14:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.862+0000 7f87ccc00640 1 -- 172.21.15.7:0/856640546 learned_addr learned my addr 172.21.15.7:0/856640546 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.862+0000 7f87c7e00640 1 --2- 172.21.15.7:0/856640546 >> v2:172.21.15.87:3300/0 conn(0x7f87c8070db0 0x7f87c80711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.862+0000 7f87cd600640 1 --2- 172.21.15.7:0/856640546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87c80b06a0 0x7f87c80b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.862+0000 7f87ccc00640 1 -- 172.21.15.7:0/856640546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87c80b06a0 msgr2=0x7f87c80b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.862+0000 7f87ccc00640 1 --2- 172.21.15.7:0/856640546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87c80b06a0 0x7f87c80b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.862+0000 7f87ccc00640 1 -- 172.21.15.7:0/856640546 >> v2:172.21.15.87:3300/0 conn(0x7f87c8070db0 msgr2=0x7f87c80711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.862+0000 7f87ccc00640 1 --2- 172.21.15.7:0/856640546 >> v2:172.21.15.87:3300/0 conn(0x7f87c8070db0 0x7f87c80711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.862+0000 7f87ccc00640 1 -- 172.21.15.7:0/856640546 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c81685a0 con 0x7f87c80b48a0 2024-04-25T11:14:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.863+0000 7f87ccc00640 1 --2- 172.21.15.7:0/856640546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c80b48a0 0x7f87c81620a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009870 tx=0x7f87bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa8f6397cd7f16e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.864+0000 7f87c7400640 1 -- 172.21.15.7:0/856640546 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc037070 con 0x7f87c80b48a0 2024-04-25T11:14:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.864+0000 7f87c7400640 1 -- 172.21.15.7:0/856640546 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87bc032440 con 0x7f87c80b48a0 2024-04-25T11:14:18.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.864+0000 7f87c7400640 1 -- 172.21.15.7:0/856640546 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc032810 con 0x7f87c80b48a0 2024-04-25T11:14:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.864+0000 7f87cf600640 1 -- 172.21.15.7:0/856640546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c80b48a0 msgr2=0x7f87c81620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.864+0000 7f87cf600640 1 --2- 172.21.15.7:0/856640546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c80b48a0 0x7f87c81620a0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f87bc009870 tx=0x7f87bc02a180 comp rx=0 tx=0).stop 2024-04-25T11:14:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.865+0000 7f87cf600640 1 -- 172.21.15.7:0/856640546 shutdown_connections 2024-04-25T11:14:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.865+0000 7f87cf600640 1 --2- 172.21.15.7:0/856640546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87c80b06a0 0x7f87c80b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.865+0000 7f87cf600640 1 --2- 172.21.15.7:0/856640546 >> v2:172.21.15.87:3300/0 conn(0x7f87c8070db0 0x7f87c80711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.865+0000 7f87cf600640 1 --2- 172.21.15.7:0/856640546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c80b48a0 0x7f87c81620a0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.865+0000 7f87cf600640 1 -- 172.21.15.7:0/856640546 >> 172.21.15.7:0/856640546 conn(0x7f87c8073920 msgr2=0x7f87c8073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.865+0000 7f87cf600640 1 -- 172.21.15.7:0/856640546 shutdown_connections 2024-04-25T11:14:18.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.865+0000 7f87cf600640 1 -- 172.21.15.7:0/856640546 wait complete. 2024-04-25T11:14:18.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.867+0000 7f87cf600640 1 Processor -- start 2024-04-25T11:14:18.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.867+0000 7f87cf600640 1 -- start start 2024-04-25T11:14:18.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.869+0000 7f87cf600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c8070db0 0x7f87c81598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.869+0000 7f87cf600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87c80b06a0 0x7f87c8166690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.869+0000 7f87cf600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87c80b48a0 0x7f87c815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.869+0000 7f87cf600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87c80b0ff0 con 0x7f87c8070db0 2024-04-25T11:14:18.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.869+0000 7f87cf600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f87c8168b60 con 0x7f87c80b06a0 2024-04-25T11:14:18.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.869+0000 7f87cf600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87c806f8e0 con 0x7f87c80b48a0 2024-04-25T11:14:18.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.869+0000 7f87ccc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c8070db0 0x7f87c81598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.869+0000 7f87ccc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c8070db0 0x7f87c81598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53758/0 (socket says 172.21.15.7:53758) 2024-04-25T11:14:18.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.869+0000 7f87ccc00640 1 -- 172.21.15.7:0/2549459682 learned_addr learned my addr 172.21.15.7:0/2549459682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.869+0000 7f87c7e00640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87c80b06a0 0x7f87c8166690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87cd600640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87c80b48a0 0x7f87c815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87ccc00640 1 -- 172.21.15.7:0/2549459682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87c80b48a0 msgr2=0x7f87c815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87ccc00640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87c80b48a0 0x7f87c815dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87ccc00640 1 -- 172.21.15.7:0/2549459682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87c80b06a0 msgr2=0x7f87c8166690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87ccc00640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87c80b06a0 0x7f87c8166690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87ccc00640 1 -- 172.21.15.7:0/2549459682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c81650d0 con 0x7f87c8070db0 2024-04-25T11:14:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87c7e00640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87c80b06a0 0x7f87c8166690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87ccc00640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c8070db0 0x7f87c81598d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f87bc02a1c0 tx=0x7f87bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87c4c00640 1 -- 172.21.15.7:0/2549459682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc041020 con 0x7f87c8070db0 2024-04-25T11:14:18.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87c4c00640 1 -- 172.21.15.7:0/2549459682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87bc0306c0 con 0x7f87c8070db0 2024-04-25T11:14:18.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87c4c00640 1 -- 172.21.15.7:0/2549459682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87bc037040 con 0x7f87c8070db0 2024-04-25T11:14:18.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.870+0000 7f87cf600640 1 -- 172.21.15.7:0/2549459682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c81653b0 con 0x7f87c8070db0 2024-04-25T11:14:18.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.871+0000 7f87cf600640 1 -- 172.21.15.7:0/2549459682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c8165970 con 0x7f87c8070db0 2024-04-25T11:14:18.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.872+0000 7f87cf600640 1 -- 172.21.15.7:0/2549459682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c806a100 con 0x7f87c8070db0 2024-04-25T11:14:18.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.873+0000 7f87c4c00640 1 -- 172.21.15.7:0/2549459682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f87bc044020 con 0x7f87c8070db0 2024-04-25T11:14:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.874+0000 7f87c4c00640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87a80744b0 0x7f87a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:18.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.875+0000 7f87c4c00640 1 -- 172.21.15.7:0/2549459682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5249+0+0 (secure 0 0 0) 0x7f87bc0b59a0 con 0x7f87c8070db0 2024-04-25T11:14:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.875+0000 7f87c7e00640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87a80744b0 0x7f87a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:18.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.876+0000 7f87c7e00640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87a80744b0 0x7f87a80769c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f87b4006fd0 tx=0x7f87b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:18.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:18.880+0000 7f87c4c00640 1 -- 172.21.15.7:0/2549459682 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f87bc04b030 con 0x7f87c8070db0 2024-04-25T11:14:19.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.094+0000 7f87cf600640 1 -- 172.21.15.7:0/2549459682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f87c80699d0 con 0x7f87c8070db0 2024-04-25T11:14:19.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.290+0000 7f87c4c00640 1 -- 172.21.15.7:0/2549459682 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v65) ==== 138+0+0 (secure 0 0 0) 0x7f87bc080250 con 0x7f87c8070db0 2024-04-25T11:14:19.291 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-25T11:14:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.294+0000 7f87cf600640 1 -- 172.21.15.7:0/2549459682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87a80744b0 msgr2=0x7f87a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.294+0000 7f87cf600640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87a80744b0 0x7f87a80769c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f87b4006fd0 tx=0x7f87b4008040 comp rx=0 tx=0).stop 2024-04-25T11:14:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.294+0000 7f87cf600640 1 -- 172.21.15.7:0/2549459682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c8070db0 msgr2=0x7f87c81598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.294+0000 7f87cf600640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c8070db0 0x7f87c81598d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f87bc02a1c0 tx=0x7f87bc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:14:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.295+0000 7f87cf600640 1 -- 172.21.15.7:0/2549459682 shutdown_connections 2024-04-25T11:14:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.295+0000 7f87cf600640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87a80744b0 0x7f87a80769c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.295+0000 7f87cf600640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87c80b48a0 0x7f87c815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.295+0000 7f87cf600640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87c80b06a0 0x7f87c8166690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:19.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.295+0000 7f87cf600640 1 --2- 172.21.15.7:0/2549459682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87c8070db0 0x7f87c81598d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:19.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.295+0000 7f87cf600640 1 -- 172.21.15.7:0/2549459682 >> 172.21.15.7:0/2549459682 conn(0x7f87c8073920 msgr2=0x7f87c80b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:19.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.295+0000 7f87cf600640 1 -- 172.21.15.7:0/2549459682 shutdown_connections 2024-04-25T11:14:19.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.295+0000 7f87cf600640 1 -- 172.21.15.7:0/2549459682 wait complete. 2024-04-25T11:14:19.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.600+0000 7efddda00640 1 Processor -- start 2024-04-25T11:14:19.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.600+0000 7efddda00640 1 -- start start 2024-04-25T11:14:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.602+0000 7efddda00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efdd8144a80 0x7efdd8146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.602+0000 7efddda00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd80b66b0 con 0x7efdd80b4f80 2024-04-25T11:14:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.602+0000 7efddda00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd80b5b30 con 0x7efdd8145e40 2024-04-25T11:14:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.602+0000 7efddda00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efdd8145b70 con 0x7efdd8144a80 2024-04-25T11:14:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.603+0000 7efdd7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efdd8144a80 0x7efdd8146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.603+0000 7efddd000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efdd80b4f80 0x7efdd81606f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43598/0 (socket says 172.21.15.7:43598) 2024-04-25T11:14:19.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.603+0000 7efddd000640 1 -- 172.21.15.7:0/550485387 learned_addr learned my addr 172.21.15.7:0/550485387 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:19.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.603+0000 7efdd6a00640 1 -- 172.21.15.7:0/550485387 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819403986 0 0) 0x7efdd80b66b0 con 0x7efdd80b4f80 2024-04-25T11:14:19.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.603+0000 7efdd6a00640 1 -- 172.21.15.7:0/550485387 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdbc003520 con 0x7efdd80b4f80 2024-04-25T11:14:19.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.603+0000 7efdd7400640 1 -- 172.21.15.7:0/550485387 >> v1:172.21.15.87:6789/0 conn(0x7efdd8145e40 legacy=0x7efdd80b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:19.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.603+0000 7efdd7400640 1 -- 172.21.15.7:0/550485387 >> v1:172.21.15.7:6789/0 conn(0x7efdd80b4f80 legacy=0x7efdd81606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:19.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.603+0000 7efdd7400640 1 -- 172.21.15.7:0/550485387 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd80b5e10 con 0x7efdd8144a80 2024-04-25T11:14:19.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.604+0000 7efdd7400640 1 --2- 172.21.15.7:0/550485387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd8144a80 0x7efdd8146250 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009d90 tx=0x7efdc002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a2034494815517 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:19.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.605+0000 7efdd6a00640 1 -- 172.21.15.7:0/550485387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0007740 con 0x7efdd8144a80 2024-04-25T11:14:19.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.605+0000 7efdd6a00640 1 -- 172.21.15.7:0/550485387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc00078e0 con 0x7efdd8144a80 2024-04-25T11:14:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.605+0000 7efdd6a00640 1 -- 172.21.15.7:0/550485387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0007cb0 con 0x7efdd8144a80 2024-04-25T11:14:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.605+0000 7efddda00640 1 -- 172.21.15.7:0/550485387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd8144a80 msgr2=0x7efdd8146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.605+0000 7efddda00640 1 --2- 172.21.15.7:0/550485387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd8144a80 0x7efdd8146250 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009d90 tx=0x7efdc002a660 comp rx=0 tx=0).stop 2024-04-25T11:14:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.606+0000 7efddda00640 1 -- 172.21.15.7:0/550485387 shutdown_connections 2024-04-25T11:14:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.606+0000 7efddda00640 1 --2- 172.21.15.7:0/550485387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd8144a80 0x7efdd8146250 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.606+0000 7efddda00640 1 -- 172.21.15.7:0/550485387 >> 172.21.15.7:0/550485387 conn(0x7efdd80740f0 msgr2=0x7efdd806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.606+0000 7efddda00640 1 -- 172.21.15.7:0/550485387 shutdown_connections 2024-04-25T11:14:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.606+0000 7efddda00640 1 -- 172.21.15.7:0/550485387 wait complete. 2024-04-25T11:14:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.608+0000 7efddda00640 1 Processor -- start 2024-04-25T11:14:19.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.608+0000 7efddda00640 1 -- start start 2024-04-25T11:14:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.610+0000 7efddda00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efdd80b4f80 0x7efdd815a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.610+0000 7efddda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdd8144a80 0x7efdd813f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.610+0000 7efddda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd8145e40 0x7efdd8161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.610+0000 7efddda00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efdd8161e10 con 0x7efdd8145e40 2024-04-25T11:14:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.610+0000 7efddda00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efdd80b63d0 con 0x7efdd8144a80 2024-04-25T11:14:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.610+0000 7efddda00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efdd8161f90 con 0x7efdd80b4f80 2024-04-25T11:14:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.610+0000 7efddd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd8145e40 0x7efdd8161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.610+0000 7efddd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd8145e40 0x7efdd8161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53768/0 (socket says 172.21.15.7:53768) 2024-04-25T11:14:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.610+0000 7efddd000640 1 -- 172.21.15.7:0/3643381917 learned_addr learned my addr 172.21.15.7:0/3643381917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.610+0000 7efdd7e00640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efdd80b4f80 0x7efdd815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.610+0000 7efdd7400640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdd8144a80 0x7efdd813f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.611+0000 7efddd000640 1 -- 172.21.15.7:0/3643381917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efdd80b4f80 msgr2=0x7efdd815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.611+0000 7efddd000640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efdd80b4f80 0x7efdd815a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.611+0000 7efddd000640 1 -- 172.21.15.7:0/3643381917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdd8144a80 msgr2=0x7efdd813f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.611+0000 7efddd000640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdd8144a80 0x7efdd813f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.611+0000 7efddd000640 1 -- 172.21.15.7:0/3643381917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd813f920 con 0x7efdd8145e40 2024-04-25T11:14:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.611+0000 7efddd000640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd8145e40 0x7efdd8161120 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7efdd000c3c0 tx=0x7efdd000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.611+0000 7efdcfe00640 1 -- 172.21.15.7:0/3643381917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0017070 con 0x7efdd8145e40 2024-04-25T11:14:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.611+0000 7efdcfe00640 1 -- 172.21.15.7:0/3643381917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd0005ce0 con 0x7efdd8145e40 2024-04-25T11:14:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.611+0000 7efdcfe00640 1 -- 172.21.15.7:0/3643381917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00072d0 con 0x7efdd8145e40 2024-04-25T11:14:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.611+0000 7efddda00640 1 -- 172.21.15.7:0/3643381917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd813d9c0 con 0x7efdd8145e40 2024-04-25T11:14:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.612+0000 7efddda00640 1 -- 172.21.15.7:0/3643381917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd813df80 con 0x7efdd8145e40 2024-04-25T11:14:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.613+0000 7efddda00640 1 -- 172.21.15.7:0/3643381917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd8144870 con 0x7efdd8145e40 2024-04-25T11:14:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.614+0000 7efdcfe00640 1 -- 172.21.15.7:0/3643381917 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efdd001a070 con 0x7efdd8145e40 2024-04-25T11:14:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.615+0000 7efdcfe00640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdb40744b0 0x7efdb40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.615+0000 7efdcfe00640 1 -- 172.21.15.7:0/3643381917 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5619+0+0 (secure 0 0 0) 0x7efdd0094350 con 0x7efdd8145e40 2024-04-25T11:14:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.616+0000 7efdd7e00640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdb40744b0 0x7efdb40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.616+0000 7efdd7e00640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdb40744b0 0x7efdb40769c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efdc8000f60 tx=0x7efdc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:19.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.621+0000 7efdcfe00640 1 -- 172.21.15.7:0/3643381917 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efdd8144870 con 0x7efdd8145e40 2024-04-25T11:14:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:19.827+0000 7efddda00640 1 -- 172.21.15.7:0/3643381917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efdd8050740 con 0x7efdd8145e40 2024-04-25T11:14:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.297+0000 7efdcfe00640 1 -- 172.21.15.7:0/3643381917 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v66) ==== 170+0+0 (secure 0 0 0) 0x7efdd005e600 con 0x7efdd8145e40 2024-04-25T11:14:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-25T11:14:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.301+0000 7efddda00640 1 -- 172.21.15.7:0/3643381917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdb40744b0 msgr2=0x7efdb40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.301+0000 7efddda00640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdb40744b0 0x7efdb40769c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7efdc8000f60 tx=0x7efdc8009040 comp rx=0 tx=0).stop 2024-04-25T11:14:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.301+0000 7efddda00640 1 -- 172.21.15.7:0/3643381917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd8145e40 msgr2=0x7efdd8161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.301+0000 7efddda00640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd8145e40 0x7efdd8161120 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7efdd000c3c0 tx=0x7efdd000c880 comp rx=0 tx=0).stop 2024-04-25T11:14:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.301+0000 7efddda00640 1 -- 172.21.15.7:0/3643381917 shutdown_connections 2024-04-25T11:14:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.301+0000 7efddda00640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdb40744b0 0x7efdb40769c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.301+0000 7efddda00640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd8145e40 0x7efdd8161120 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.301+0000 7efddda00640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efdd8144a80 0x7efdd813f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.301+0000 7efddda00640 1 --2- 172.21.15.7:0/3643381917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efdd80b4f80 0x7efdd815a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.301+0000 7efddda00640 1 -- 172.21.15.7:0/3643381917 >> 172.21.15.7:0/3643381917 conn(0x7efdd80740f0 msgr2=0x7efdd815c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.302+0000 7efddda00640 1 -- 172.21.15.7:0/3643381917 shutdown_connections 2024-04-25T11:14:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.302+0000 7efddda00640 1 -- 172.21.15.7:0/3643381917 wait complete. 2024-04-25T11:14:20.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:14:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.606+0000 7f857be00640 1 Processor -- start 2024-04-25T11:14:20.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.606+0000 7f857be00640 1 -- start start 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.609+0000 7f857be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c145e40 0x7f857c146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.609+0000 7f857be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f857c144a80 0x7f857c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.609+0000 7f857be00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f857c0b66b0 con 0x7f857c0b4870 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.609+0000 7f857be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f857c145b70 con 0x7f857c144a80 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.609+0000 7f857be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f857c0b58d0 con 0x7f857c145e40 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.609+0000 7f857a000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f857c0b4870 0x7f857c0b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52918/0 (socket says 172.21.15.7:52918) 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.609+0000 7f857a000640 1 -- 172.21.15.7:0/2716150995 learned_addr learned my addr 172.21.15.7:0/2716150995 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.609+0000 7f857aa00640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c145e40 0x7f857c146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.609+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1966065317 0 0) 0x7f857c0b66b0 con 0x7f857c0b4870 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.609+0000 7f857b400640 1 --2- 172.21.15.7:0/2716150995 >> v2:172.21.15.87:3300/0 conn(0x7f857c144a80 0x7f857c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8560003520 con 0x7f857c0b4870 2024-04-25T11:14:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2847663210 0 0) 0x7f8560003520 con 0x7f857c0b4870 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c145e40 msgr2=0x7f857c146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c145e40 0x7f857c146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 >> v2:172.21.15.87:3300/0 conn(0x7f857c144a80 msgr2=0x7f857c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 --2- 172.21.15.7:0/2716150995 >> v2:172.21.15.87:3300/0 conn(0x7f857c144a80 0x7f857c0b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857c0b5b30 con 0x7f857c0b4870 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f8568001670 con 0x7f857c0b4870 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 >> v1:172.21.15.7:6789/0 conn(0x7f857c0b4870 legacy=0x7f857c0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85600039b0 0x7f8560003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8560004800 0x7f8560004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c145e40 0x7f8560005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f857c0b58d0 con 0x7f8560004800 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f856000bad0 con 0x7f85600039b0 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.610+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f856000bcd0 con 0x7f857c145e40 2024-04-25T11:14:20.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.611+0000 7f857a000640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8560004800 0x7f8560004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:20.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.611+0000 7f857aa00640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85600039b0 0x7f8560003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:20.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.611+0000 7f857b400640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c145e40 0x7f8560005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.611+0000 7f857a000640 1 -- 172.21.15.7:0/2716150995 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c145e40 msgr2=0x7f8560005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.611+0000 7f857a000640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c145e40 0x7f8560005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.611+0000 7f857a000640 1 -- 172.21.15.7:0/2716150995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85600039b0 msgr2=0x7f8560003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.611+0000 7f857a000640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85600039b0 0x7f8560003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.611+0000 7f857a000640 1 -- 172.21.15.7:0/2716150995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8560003520 con 0x7f8560004800 2024-04-25T11:14:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.612+0000 7f857a000640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8560004800 0x7f8560004c10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f856800c160 tx=0x7f8560005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275d6ef030cb243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.613+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568007c80 con 0x7f8560004800 2024-04-25T11:14:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.613+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8568005ad0 con 0x7f8560004800 2024-04-25T11:14:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.613+0000 7f8579600640 1 -- 172.21.15.7:0/2716150995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568007070 con 0x7f8560004800 2024-04-25T11:14:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.613+0000 7f857be00640 1 -- 172.21.15.7:0/2716150995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8560004800 msgr2=0x7f8560004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.613+0000 7f857be00640 1 --2- 172.21.15.7:0/2716150995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8560004800 0x7f8560004c10 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f856800c160 tx=0x7f8560005830 comp rx=0 tx=0).stop 2024-04-25T11:14:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.613+0000 7f857b400640 1 -- 172.21.15.7:0/2716150995 reap_dead start 2024-04-25T11:14:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.613+0000 7f857be00640 1 -- 172.21.15.7:0/2716150995 shutdown_connections 2024-04-25T11:14:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.613+0000 7f857be00640 1 -- 172.21.15.7:0/2716150995 >> 172.21.15.7:0/2716150995 conn(0x7f857c0740f0 msgr2=0x7f857c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.614+0000 7f857be00640 1 -- 172.21.15.7:0/2716150995 shutdown_connections 2024-04-25T11:14:20.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.614+0000 7f857be00640 1 -- 172.21.15.7:0/2716150995 wait complete. 2024-04-25T11:14:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.616+0000 7f857be00640 1 Processor -- start 2024-04-25T11:14:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.616+0000 7f857be00640 1 -- start start 2024-04-25T11:14:20.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.618+0000 7f857be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8560004800 0x7f857c141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:20.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.618+0000 7f857be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c1424b0 0x7f857c160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.618+0000 7f857be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857c15be90 0x7f857c160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.618+0000 7f857be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f857c0b63d0 con 0x7f857c15be90 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.618+0000 7f857be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f857c0b60f0 con 0x7f8560004800 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.618+0000 7f857be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f857c146250 con 0x7f857c1424b0 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.618+0000 7f857b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857c15be90 0x7f857c160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.618+0000 7f857b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857c15be90 0x7f857c160bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35308/0 (socket says 172.21.15.7:35308) 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.618+0000 7f857b400640 1 -- 172.21.15.7:0/3045341907 learned_addr learned my addr 172.21.15.7:0/3045341907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.618+0000 7f857a000640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c1424b0 0x7f857c160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.618+0000 7f857aa00640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8560004800 0x7f857c141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.619+0000 7f857b400640 1 -- 172.21.15.7:0/3045341907 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c1424b0 msgr2=0x7f857c160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.619+0000 7f857b400640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c1424b0 0x7f857c160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.619+0000 7f857b400640 1 -- 172.21.15.7:0/3045341907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8560004800 msgr2=0x7f857c141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.619+0000 7f857b400640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8560004800 0x7f857c141f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:20.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.619+0000 7f857b400640 1 -- 172.21.15.7:0/3045341907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857c13b0a0 con 0x7f857c15be90 2024-04-25T11:14:20.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.619+0000 7f857b400640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857c15be90 0x7f857c160bc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f857c06e0f0 tx=0x7f8574007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:20.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.619+0000 7f8573400640 1 -- 172.21.15.7:0/3045341907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574014410 con 0x7f857c15be90 2024-04-25T11:14:20.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.619+0000 7f8573400640 1 -- 172.21.15.7:0/3045341907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8574019070 con 0x7f857c15be90 2024-04-25T11:14:20.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.619+0000 7f8573400640 1 -- 172.21.15.7:0/3045341907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85740147f0 con 0x7f857c15be90 2024-04-25T11:14:20.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.619+0000 7f857be00640 1 -- 172.21.15.7:0/3045341907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857c1429f0 con 0x7f857c15be90 2024-04-25T11:14:20.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.620+0000 7f857be00640 1 -- 172.21.15.7:0/3045341907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857c142fb0 con 0x7f857c15be90 2024-04-25T11:14:20.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.621+0000 7f857be00640 1 -- 172.21.15.7:0/3045341907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f857c13ae10 con 0x7f857c15be90 2024-04-25T11:14:20.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.622+0000 7f8573400640 1 -- 172.21.15.7:0/3045341907 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8574014990 con 0x7f857c15be90 2024-04-25T11:14:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.623+0000 7f8573400640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f855c0744b0 0x7f855c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.623+0000 7f8573400640 1 -- 172.21.15.7:0/3045341907 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6001+0+0 (secure 0 0 0) 0x7f8574095980 con 0x7f857c15be90 2024-04-25T11:14:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.623+0000 7f857aa00640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f855c0744b0 0x7f855c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.624+0000 7f857aa00640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f855c0744b0 0x7f855c0769c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f856c009a10 tx=0x7f856c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:20.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.629+0000 7f8573400640 1 -- 172.21.15.7:0/3045341907 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f857405ffd0 con 0x7f857c15be90 2024-04-25T11:14:20.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:20.876+0000 7f857be00640 1 -- 172.21.15.7:0/3045341907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f857c050740 con 0x7f857c15be90 2024-04-25T11:14:21.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.326+0000 7f8573400640 1 -- 172.21.15.7:0/3045341907 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) ==== 173+0+0 (secure 0 0 0) 0x7f857405f8d0 con 0x7f857c15be90 2024-04-25T11:14:21.326 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T11:14:21.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.330+0000 7f857be00640 1 -- 172.21.15.7:0/3045341907 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f855c0744b0 msgr2=0x7f855c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.330+0000 7f857be00640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f855c0744b0 0x7f855c0769c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f856c009a10 tx=0x7f856c008040 comp rx=0 tx=0).stop 2024-04-25T11:14:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.330+0000 7f857be00640 1 -- 172.21.15.7:0/3045341907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857c15be90 msgr2=0x7f857c160bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.330+0000 7f857be00640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857c15be90 0x7f857c160bc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f857c06e0f0 tx=0x7f8574007700 comp rx=0 tx=0).stop 2024-04-25T11:14:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.330+0000 7f857be00640 1 -- 172.21.15.7:0/3045341907 shutdown_connections 2024-04-25T11:14:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.330+0000 7f857be00640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f855c0744b0 0x7f855c0769c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.330+0000 7f857be00640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857c15be90 0x7f857c160bc0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.330+0000 7f857be00640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c1424b0 0x7f857c160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.330+0000 7f857be00640 1 --2- 172.21.15.7:0/3045341907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8560004800 0x7f857c141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.330+0000 7f857be00640 1 -- 172.21.15.7:0/3045341907 >> 172.21.15.7:0/3045341907 conn(0x7f857c0740f0 msgr2=0x7f857c06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:21.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.331+0000 7f857be00640 1 -- 172.21.15.7:0/3045341907 shutdown_connections 2024-04-25T11:14:21.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.331+0000 7f857be00640 1 -- 172.21.15.7:0/3045341907 wait complete. 2024-04-25T11:14:21.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:21.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.630+0000 7fd6be000640 1 Processor -- start 2024-04-25T11:14:21.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.631+0000 7fd6be000640 1 -- start start 2024-04-25T11:14:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.633+0000 7fd6be000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd6b8164690 0x7fd6b8070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.633+0000 7fd6be000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd6b8156cd0 0x7fd6b8163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:21.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.633+0000 7fd6be000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6b81570e0 0x7fd6b8164010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.633+0000 7fd6be000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b80b5080 con 0x7fd6b8164690 2024-04-25T11:14:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.633+0000 7fd6be000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd6b80b5360 con 0x7fd6b8156cd0 2024-04-25T11:14:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.633+0000 7fd6be000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b80b5640 con 0x7fd6b81570e0 2024-04-25T11:14:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.633+0000 7fd6bcc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd6b8164690 0x7fd6b8070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.633+0000 7fd6bcc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd6b8164690 0x7fd6b8070a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35320/0 (socket says 172.21.15.7:35320) 2024-04-25T11:14:21.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.633+0000 7fd6bcc00640 1 -- 172.21.15.7:0/3984925496 learned_addr learned my addr 172.21.15.7:0/3984925496 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.634+0000 7fd6b7e00640 1 --2- 172.21.15.7:0/3984925496 >> v2:172.21.15.87:3300/0 conn(0x7fd6b8156cd0 0x7fd6b8163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.634+0000 7fd6bd600640 1 --2- 172.21.15.7:0/3984925496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6b81570e0 0x7fd6b8164010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.634+0000 7fd6bcc00640 1 -- 172.21.15.7:0/3984925496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6b81570e0 msgr2=0x7fd6b8164010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.634+0000 7fd6bcc00640 1 --2- 172.21.15.7:0/3984925496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6b81570e0 0x7fd6b8164010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.634+0000 7fd6bcc00640 1 -- 172.21.15.7:0/3984925496 >> v2:172.21.15.87:3300/0 conn(0x7fd6b8156cd0 msgr2=0x7fd6b8163ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.634+0000 7fd6bcc00640 1 --2- 172.21.15.7:0/3984925496 >> v2:172.21.15.87:3300/0 conn(0x7fd6b8156cd0 0x7fd6b8163ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.634+0000 7fd6bcc00640 1 -- 172.21.15.7:0/3984925496 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b80b5920 con 0x7fd6b8164690 2024-04-25T11:14:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.635+0000 7fd6bcc00640 1 --2- 172.21.15.7:0/3984925496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b8164690 0x7fd6b8070a20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80722d0 tx=0x7fd6ac02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfbc0ab6a5ed906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.636+0000 7fd6b7400640 1 -- 172.21.15.7:0/3984925496 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac037070 con 0x7fd6b8164690 2024-04-25T11:14:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.636+0000 7fd6b7400640 1 -- 172.21.15.7:0/3984925496 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6ac032440 con 0x7fd6b8164690 2024-04-25T11:14:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.636+0000 7fd6b7400640 1 -- 172.21.15.7:0/3984925496 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac032810 con 0x7fd6b8164690 2024-04-25T11:14:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.637+0000 7fd6be000640 1 -- 172.21.15.7:0/3984925496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b8164690 msgr2=0x7fd6b8070a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.637+0000 7fd6be000640 1 --2- 172.21.15.7:0/3984925496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b8164690 0x7fd6b8070a20 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80722d0 tx=0x7fd6ac02a140 comp rx=0 tx=0).stop 2024-04-25T11:14:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.637+0000 7fd6be000640 1 -- 172.21.15.7:0/3984925496 shutdown_connections 2024-04-25T11:14:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.637+0000 7fd6be000640 1 --2- 172.21.15.7:0/3984925496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6b81570e0 0x7fd6b8164010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.637+0000 7fd6be000640 1 --2- 172.21.15.7:0/3984925496 >> v2:172.21.15.87:3300/0 conn(0x7fd6b8156cd0 0x7fd6b8163ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.637+0000 7fd6be000640 1 --2- 172.21.15.7:0/3984925496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b8164690 0x7fd6b8070a20 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.637+0000 7fd6be000640 1 -- 172.21.15.7:0/3984925496 >> 172.21.15.7:0/3984925496 conn(0x7fd6b80b3820 msgr2=0x7fd6b80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.637+0000 7fd6be000640 1 -- 172.21.15.7:0/3984925496 shutdown_connections 2024-04-25T11:14:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.637+0000 7fd6be000640 1 -- 172.21.15.7:0/3984925496 wait complete. 2024-04-25T11:14:21.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.639+0000 7fd6be000640 1 Processor -- start 2024-04-25T11:14:21.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.639+0000 7fd6be000640 1 -- start start 2024-04-25T11:14:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.641+0000 7fd6be000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6b8156cd0 0x7fd6b80b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.641+0000 7fd6be000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6b81570e0 0x7fd6b815d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.641+0000 7fd6be000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b8164690 0x7fd6b815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.641+0000 7fd6be000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b80b0210 con 0x7fd6b8164690 2024-04-25T11:14:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.641+0000 7fd6be000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8070f60 con 0x7fd6b8156cd0 2024-04-25T11:14:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.641+0000 7fd6be000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b80b0390 con 0x7fd6b81570e0 2024-04-25T11:14:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.641+0000 7fd6bd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b8164690 0x7fd6b815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.641+0000 7fd6bd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b8164690 0x7fd6b815b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35334/0 (socket says 172.21.15.7:35334) 2024-04-25T11:14:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.641+0000 7fd6bd600640 1 -- 172.21.15.7:0/608682797 learned_addr learned my addr 172.21.15.7:0/608682797 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.641+0000 7fd6bcc00640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6b8156cd0 0x7fd6b80b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.641+0000 7fd6b7e00640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6b81570e0 0x7fd6b815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:21.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6bd600640 1 -- 172.21.15.7:0/608682797 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6b81570e0 msgr2=0x7fd6b815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:21.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6bd600640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6b81570e0 0x7fd6b815d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6bd600640 1 -- 172.21.15.7:0/608682797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6b8156cd0 msgr2=0x7fd6b80b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:21.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6bd600640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6b8156cd0 0x7fd6b80b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6bd600640 1 -- 172.21.15.7:0/608682797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b81ebd90 con 0x7fd6b8164690 2024-04-25T11:14:21.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6b7e00640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6b81570e0 0x7fd6b815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:21.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6bd600640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b8164690 0x7fd6b815b900 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0007e60 tx=0x7fd6b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:21.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6b4c00640 1 -- 172.21.15.7:0/608682797 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0014420 con 0x7fd6b8164690 2024-04-25T11:14:21.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6b4c00640 1 -- 172.21.15.7:0/608682797 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0019070 con 0x7fd6b8164690 2024-04-25T11:14:21.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6be000640 1 -- 172.21.15.7:0/608682797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b81ec070 con 0x7fd6b8164690 2024-04-25T11:14:21.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6b4c00640 1 -- 172.21.15.7:0/608682797 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0014820 con 0x7fd6b8164690 2024-04-25T11:14:21.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.642+0000 7fd6be000640 1 -- 172.21.15.7:0/608682797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b81ec630 con 0x7fd6b8164690 2024-04-25T11:14:21.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.644+0000 7fd6be000640 1 -- 172.21.15.7:0/608682797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b8050740 con 0x7fd6b8164690 2024-04-25T11:14:21.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.645+0000 7fd6b4c00640 1 -- 172.21.15.7:0/608682797 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd6b00149c0 con 0x7fd6b8164690 2024-04-25T11:14:21.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.646+0000 7fd6b4c00640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 0x7fd6980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:21.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.647+0000 7fd6b4c00640 1 -- 172.21.15.7:0/608682797 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fd6b0095cd0 con 0x7fd6b8164690 2024-04-25T11:14:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.647+0000 7fd6bcc00640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 0x7fd6980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.647+0000 7fd6bcc00640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 0x7fd6980769c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac034000 tx=0x7fd6ac02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.651+0000 7fd6b4c00640 1 -- 172.21.15.7:0/608682797 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6b0060230 con 0x7fd6b8164690 2024-04-25T11:14:21.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.849+0000 7fd6be000640 1 -- 172.21.15.7:0/608682797 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6b80b1140 con 0x7fd6b8164690 2024-04-25T11:14:21.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.852+0000 7fd6b4c00640 1 -- 172.21.15.7:0/608682797 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16801 (secure 0 0 0) 0x7fd6b005fb30 con 0x7fd6b8164690 2024-04-25T11:14:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:21.853 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:20.877482+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:14:19.115741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T11:14:19.849856+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:21.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.855+0000 7fd6be000640 1 -- 172.21.15.7:0/608682797 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 msgr2=0x7fd6980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:21.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.855+0000 7fd6be000640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 0x7fd6980769c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac034000 tx=0x7fd6ac02e000 comp rx=0 tx=0).stop 2024-04-25T11:14:21.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.855+0000 7fd6be000640 1 -- 172.21.15.7:0/608682797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b8164690 msgr2=0x7fd6b815b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:21.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.855+0000 7fd6be000640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b8164690 0x7fd6b815b900 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0007e60 tx=0x7fd6b000e4b0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.855+0000 7fd6be000640 1 -- 172.21.15.7:0/608682797 shutdown_connections 2024-04-25T11:14:21.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.855+0000 7fd6be000640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd6980744b0 0x7fd6980769c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.855+0000 7fd6be000640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b8164690 0x7fd6b815b900 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.855+0000 7fd6be000640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6b81570e0 0x7fd6b815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.855+0000 7fd6be000640 1 --2- 172.21.15.7:0/608682797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6b8156cd0 0x7fd6b80b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:21.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.855+0000 7fd6be000640 1 -- 172.21.15.7:0/608682797 >> 172.21.15.7:0/608682797 conn(0x7fd6b80b3820 msgr2=0x7fd6b80b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:21.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.855+0000 7fd6be000640 1 -- 172.21.15.7:0/608682797 shutdown_connections 2024-04-25T11:14:21.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:21.856+0000 7fd6be000640 1 -- 172.21.15.7:0/608682797 wait complete. 2024-04-25T11:14:21.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:22.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.161+0000 7fd9db200640 1 Processor -- start 2024-04-25T11:14:22.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.161+0000 7fd9db200640 1 -- start start 2024-04-25T11:14:22.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.163+0000 7fd9db200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9dc160590 0x7fd9dc0d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:22.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.163+0000 7fd9db200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc0d35a0 con 0x7fd9dc06dcd0 2024-04-25T11:14:22.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.163+0000 7fd9db200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9dc164310 con 0x7fd9dc163ac0 2024-04-25T11:14:22.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.163+0000 7fd9db200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd9dc1645f0 con 0x7fd9dc160590 2024-04-25T11:14:22.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.164+0000 7fd9da800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9dc160590 0x7fd9dc0d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:22.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.164+0000 7fd9d9400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd9dc06dcd0 0x7fd9dc06e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52922/0 (socket says 172.21.15.7:52922) 2024-04-25T11:14:22.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.164+0000 7fd9d9400640 1 -- 172.21.15.7:0/374595002 learned_addr learned my addr 172.21.15.7:0/374595002 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:22.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.164+0000 7fd9da800640 1 -- 172.21.15.7:0/374595002 >> v1:172.21.15.87:6789/0 conn(0x7fd9dc163ac0 legacy=0x7fd9dc0b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:22.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.164+0000 7fd9da800640 1 -- 172.21.15.7:0/374595002 >> v1:172.21.15.7:6789/0 conn(0x7fd9dc06dcd0 legacy=0x7fd9dc06e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:22.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.164+0000 7fd9da800640 1 -- 172.21.15.7:0/374595002 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc1648d0 con 0x7fd9dc160590 2024-04-25T11:14:22.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.165+0000 7fd9da800640 1 --2- 172.21.15.7:0/374595002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9dc160590 0x7fd9dc0d3d30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009d90 tx=0x7fd9d402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63435d162a5d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:22.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.166+0000 7fd9d8a00640 1 -- 172.21.15.7:0/374595002 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4031600 con 0x7fd9dc160590 2024-04-25T11:14:22.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.166+0000 7fd9d8a00640 1 -- 172.21.15.7:0/374595002 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d40317a0 con 0x7fd9dc160590 2024-04-25T11:14:22.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.166+0000 7fd9d8a00640 1 -- 172.21.15.7:0/374595002 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4031b70 con 0x7fd9dc160590 2024-04-25T11:14:22.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.167+0000 7fd9db200640 1 -- 172.21.15.7:0/374595002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9dc160590 msgr2=0x7fd9dc0d3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:22.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.167+0000 7fd9db200640 1 --2- 172.21.15.7:0/374595002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9dc160590 0x7fd9dc0d3d30 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009d90 tx=0x7fd9d402a610 comp rx=0 tx=0).stop 2024-04-25T11:14:22.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.167+0000 7fd9db200640 1 -- 172.21.15.7:0/374595002 shutdown_connections 2024-04-25T11:14:22.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.167+0000 7fd9db200640 1 --2- 172.21.15.7:0/374595002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9dc160590 0x7fd9dc0d3d30 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.167+0000 7fd9db200640 1 -- 172.21.15.7:0/374595002 >> 172.21.15.7:0/374595002 conn(0x7fd9dc0d29e0 msgr2=0x7fd9dc0d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:22.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.167+0000 7fd9db200640 1 -- 172.21.15.7:0/374595002 shutdown_connections 2024-04-25T11:14:22.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.167+0000 7fd9db200640 1 -- 172.21.15.7:0/374595002 wait complete. 2024-04-25T11:14:22.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.169+0000 7fd9db200640 1 Processor -- start 2024-04-25T11:14:22.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.169+0000 7fd9db200640 1 -- start start 2024-04-25T11:14:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9db200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9dc06dcd0 0x7fd9dc158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9db200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9dc160590 0x7fd9dc159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9db200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9dc163ac0 0x7fd9dc1ea050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9db200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc0d43f0 con 0x7fd9dc163ac0 2024-04-25T11:14:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9db200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd9dc0d4270 con 0x7fd9dc160590 2024-04-25T11:14:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9db200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9dc0d4570 con 0x7fd9dc06dcd0 2024-04-25T11:14:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9da800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9dc163ac0 0x7fd9dc1ea050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9da800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9dc163ac0 0x7fd9dc1ea050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35348/0 (socket says 172.21.15.7:35348) 2024-04-25T11:14:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9da800640 1 -- 172.21.15.7:0/3369489618 learned_addr learned my addr 172.21.15.7:0/3369489618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9d9400640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9dc160590 0x7fd9dc159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9d9e00640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9dc06dcd0 0x7fd9dc158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9da800640 1 -- 172.21.15.7:0/3369489618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9dc06dcd0 msgr2=0x7fd9dc158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9da800640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9dc06dcd0 0x7fd9dc158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9da800640 1 -- 172.21.15.7:0/3369489618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9dc160590 msgr2=0x7fd9dc159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9da800640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9dc160590 0x7fd9dc159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9da800640 1 -- 172.21.15.7:0/3369489618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9dc159c90 con 0x7fd9dc163ac0 2024-04-25T11:14:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.171+0000 7fd9d9400640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9dc160590 0x7fd9dc159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.172+0000 7fd9da800640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9dc163ac0 0x7fd9dc1ea050 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009eb0 tx=0x7fd9d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.172+0000 7fd9d2000640 1 -- 172.21.15.7:0/3369489618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4040070 con 0x7fd9dc163ac0 2024-04-25T11:14:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.172+0000 7fd9d2000640 1 -- 172.21.15.7:0/3369489618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d4030b60 con 0x7fd9dc163ac0 2024-04-25T11:14:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.172+0000 7fd9d2000640 1 -- 172.21.15.7:0/3369489618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4045460 con 0x7fd9dc163ac0 2024-04-25T11:14:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.172+0000 7fd9db200640 1 -- 172.21.15.7:0/3369489618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9dc159f70 con 0x7fd9dc163ac0 2024-04-25T11:14:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.172+0000 7fd9db200640 1 -- 172.21.15.7:0/3369489618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9dc1e9b30 con 0x7fd9dc163ac0 2024-04-25T11:14:22.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.174+0000 7fd9db200640 1 -- 172.21.15.7:0/3369489618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9dc0747d0 con 0x7fd9dc163ac0 2024-04-25T11:14:22.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.175+0000 7fd9d2000640 1 -- 172.21.15.7:0/3369489618 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd9d4039040 con 0x7fd9dc163ac0 2024-04-25T11:14:22.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.176+0000 7fd9d2000640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9b80744b0 0x7fd9b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:22.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.176+0000 7fd9d2000640 1 -- 172.21.15.7:0/3369489618 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fd9d40b6c80 con 0x7fd9dc163ac0 2024-04-25T11:14:22.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.176+0000 7fd9d9e00640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9b80744b0 0x7fd9b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:22.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.177+0000 7fd9d9e00640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9b80744b0 0x7fd9b80769c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00a750 tx=0x7fd9cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:22.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.180+0000 7fd9d2000640 1 -- 172.21.15.7:0/3369489618 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9dc0747d0 con 0x7fd9dc163ac0 2024-04-25T11:14:22.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.383+0000 7fd9db200640 1 -- 172.21.15.7:0/3369489618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9dc1599f0 con 0x7fd9dc163ac0 2024-04-25T11:14:22.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.385+0000 7fd9d2000640 1 -- 172.21.15.7:0/3369489618 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16788 (secure 0 0 0) 0x7fd9dc1599f0 con 0x7fd9dc163ac0 2024-04-25T11:14:22.385 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:22.385 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:22.341351+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:14:19.115741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T11:14:19.849856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:22.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 -- 172.21.15.7:0/3369489618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9b80744b0 msgr2=0x7fd9b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:22.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9b80744b0 0x7fd9b80769c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00a750 tx=0x7fd9cc008040 comp rx=0 tx=0).stop 2024-04-25T11:14:22.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 -- 172.21.15.7:0/3369489618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9dc163ac0 msgr2=0x7fd9dc1ea050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:22.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9dc163ac0 0x7fd9dc1ea050 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009eb0 tx=0x7fd9d4004a40 comp rx=0 tx=0).stop 2024-04-25T11:14:22.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 -- 172.21.15.7:0/3369489618 shutdown_connections 2024-04-25T11:14:22.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9b80744b0 0x7fd9b80769c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9dc163ac0 0x7fd9dc1ea050 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9dc160590 0x7fd9dc159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 --2- 172.21.15.7:0/3369489618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9dc06dcd0 0x7fd9dc158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 -- 172.21.15.7:0/3369489618 >> 172.21.15.7:0/3369489618 conn(0x7fd9dc0d29e0 msgr2=0x7fd9dc163080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 -- 172.21.15.7:0/3369489618 shutdown_connections 2024-04-25T11:14:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.388+0000 7fd9db200640 1 -- 172.21.15.7:0/3369489618 wait complete. 2024-04-25T11:14:22.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:14:22.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.680+0000 7f91ba400640 1 Processor -- start 2024-04-25T11:14:22.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.680+0000 7f91ba400640 1 -- start start 2024-04-25T11:14:22.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.682+0000 7f91ba400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91b41583c0 0x7f91b40cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:22.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.682+0000 7f91ba400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91b40cfb50 0x7f91b40d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:22.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.682+0000 7f91ba400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91b40ce570 con 0x7f91b40d09c0 2024-04-25T11:14:22.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.682+0000 7f91ba400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f91b415f190 con 0x7f91b41583c0 2024-04-25T11:14:22.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.682+0000 7f91ba400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f91b40ce850 con 0x7f91b40cfb50 2024-04-25T11:14:22.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.682+0000 7f91b9000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91b41583c0 0x7f91b40cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:22.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.682+0000 7f91b9a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91b40d09c0 0x7f91b406eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52926/0 (socket says 172.21.15.7:52926) 2024-04-25T11:14:22.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.682+0000 7f91b9a00640 1 -- 172.21.15.7:0/1157612830 learned_addr learned my addr 172.21.15.7:0/1157612830 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:22.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.683+0000 7f91b9000640 1 -- 172.21.15.7:0/1157612830 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91b40cfb50 msgr2=0x7f91b40d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:22.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.683+0000 7f91b3e00640 1 --2- 172.21.15.7:0/1157612830 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91b40cfb50 0x7f91b40d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:22.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.683+0000 7f91b9000640 1 --2- 172.21.15.7:0/1157612830 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91b40cfb50 0x7f91b40d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.683+0000 7f91b9000640 1 -- 172.21.15.7:0/1157612830 >> v1:172.21.15.7:6789/0 conn(0x7f91b40d09c0 legacy=0x7f91b406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.683+0000 7f91b3400640 1 -- 172.21.15.7:0/1157612830 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46205205 0 0) 0x7f91b40ce570 con 0x7f91b40d09c0 2024-04-25T11:14:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.683+0000 7f91b9000640 1 -- 172.21.15.7:0/1157612830 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b40ceb30 con 0x7f91b41583c0 2024-04-25T11:14:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.684+0000 7f91b9000640 1 --2- 172.21.15.7:0/1157612830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b41583c0 0x7f91b40cf610 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f91a80097a0 tx=0x7f91a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b08461a2c4e414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.684+0000 7f91b3400640 1 -- 172.21.15.7:0/1157612830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8037070 con 0x7f91b41583c0 2024-04-25T11:14:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.684+0000 7f91b3400640 1 -- 172.21.15.7:0/1157612830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91a8032440 con 0x7f91b41583c0 2024-04-25T11:14:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.684+0000 7f91b3400640 1 -- 172.21.15.7:0/1157612830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a8032810 con 0x7f91b41583c0 2024-04-25T11:14:22.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.685+0000 7f91ba400640 1 -- 172.21.15.7:0/1157612830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b41583c0 msgr2=0x7f91b40cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:22.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.685+0000 7f91ba400640 1 --2- 172.21.15.7:0/1157612830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b41583c0 0x7f91b40cf610 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f91a80097a0 tx=0x7f91a802a180 comp rx=0 tx=0).stop 2024-04-25T11:14:22.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.685+0000 7f91ba400640 1 -- 172.21.15.7:0/1157612830 shutdown_connections 2024-04-25T11:14:22.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.685+0000 7f91ba400640 1 --2- 172.21.15.7:0/1157612830 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91b40cfb50 0x7f91b40d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.685+0000 7f91ba400640 1 --2- 172.21.15.7:0/1157612830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b41583c0 0x7f91b40cf610 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.685+0000 7f91ba400640 1 -- 172.21.15.7:0/1157612830 >> 172.21.15.7:0/1157612830 conn(0x7f91b40740f0 msgr2=0x7f91b406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.685+0000 7f91ba400640 1 -- 172.21.15.7:0/1157612830 shutdown_connections 2024-04-25T11:14:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.685+0000 7f91ba400640 1 -- 172.21.15.7:0/1157612830 wait complete. 2024-04-25T11:14:22.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.687+0000 7f91ba400640 1 Processor -- start 2024-04-25T11:14:22.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.688+0000 7f91ba400640 1 -- start start 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.689+0000 7f91ba400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91b40cfb50 0x7f91b4143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.689+0000 7f91ba400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91b40d09c0 0x7f91b41e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.689+0000 7f91ba400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b41583c0 0x7f91b41518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.689+0000 7f91ba400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f91b406f5b0 con 0x7f91b41583c0 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.689+0000 7f91ba400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f91b40cf0f0 con 0x7f91b40cfb50 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.689+0000 7f91ba400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f91b406f730 con 0x7f91b40d09c0 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.689+0000 7f91b9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b41583c0 0x7f91b41518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.689+0000 7f91b9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b41583c0 0x7f91b41518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35366/0 (socket says 172.21.15.7:35366) 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.689+0000 7f91b9a00640 1 -- 172.21.15.7:0/2265969756 learned_addr learned my addr 172.21.15.7:0/2265969756 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b9000640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91b40cfb50 0x7f91b4143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b3e00640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91b40d09c0 0x7f91b41e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:22.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b9a00640 1 -- 172.21.15.7:0/2265969756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91b40d09c0 msgr2=0x7f91b41e8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b9a00640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91b40d09c0 0x7f91b41e8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b9a00640 1 -- 172.21.15.7:0/2265969756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91b40cfb50 msgr2=0x7f91b4143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b9a00640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91b40cfb50 0x7f91b4143f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b9a00640 1 -- 172.21.15.7:0/2265969756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b41e77b0 con 0x7f91b41583c0 2024-04-25T11:14:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b9000640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91b40cfb50 0x7f91b4143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b9a00640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b41583c0 0x7f91b41518b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00c700 tx=0x7f91ac00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b0c00640 1 -- 172.21.15.7:0/2265969756 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac018070 con 0x7f91b41583c0 2024-04-25T11:14:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b0c00640 1 -- 172.21.15.7:0/2265969756 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91ac00ce10 con 0x7f91b41583c0 2024-04-25T11:14:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91b0c00640 1 -- 172.21.15.7:0/2265969756 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac00a880 con 0x7f91b41583c0 2024-04-25T11:14:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.690+0000 7f91ba400640 1 -- 172.21.15.7:0/2265969756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b41e7a90 con 0x7f91b41583c0 2024-04-25T11:14:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.691+0000 7f91ba400640 1 -- 172.21.15.7:0/2265969756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b41e8050 con 0x7f91b41583c0 2024-04-25T11:14:22.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.692+0000 7f91ba400640 1 -- 172.21.15.7:0/2265969756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b4050740 con 0x7f91b41583c0 2024-04-25T11:14:22.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.693+0000 7f91b0c00640 1 -- 172.21.15.7:0/2265969756 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f91ac01b070 con 0x7f91b41583c0 2024-04-25T11:14:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.695+0000 7f91b0c00640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9194074530 0x7f9194076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.695+0000 7f91b0c00640 1 -- 172.21.15.7:0/2265969756 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f91ac0949d0 con 0x7f91b41583c0 2024-04-25T11:14:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.695+0000 7f91b9000640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9194074530 0x7f9194076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:22.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.696+0000 7f91b9000640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9194074530 0x7f9194076a40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f91a8034000 tx=0x7f91a802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.700+0000 7f91b0c00640 1 -- 172.21.15.7:0/2265969756 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f91ac05f040 con 0x7f91b41583c0 2024-04-25T11:14:22.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:22.942+0000 7f91ba400640 1 -- 172.21.15.7:0/2265969756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f91b4070dd0 con 0x7f91b41583c0 2024-04-25T11:14:23.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.373+0000 7f91b0c00640 1 -- 172.21.15.7:0/2265969756 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7f91ac05e940 con 0x7f91b41583c0 2024-04-25T11:14:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.376+0000 7f91ba400640 1 -- 172.21.15.7:0/2265969756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9194074530 msgr2=0x7f9194076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.376+0000 7f91ba400640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9194074530 0x7f9194076a40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f91a8034000 tx=0x7f91a802e000 comp rx=0 tx=0).stop 2024-04-25T11:14:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.376+0000 7f91ba400640 1 -- 172.21.15.7:0/2265969756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b41583c0 msgr2=0x7f91b41518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.376+0000 7f91ba400640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b41583c0 0x7f91b41518b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00c700 tx=0x7f91ac00cbc0 comp rx=0 tx=0).stop 2024-04-25T11:14:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.376+0000 7f91ba400640 1 -- 172.21.15.7:0/2265969756 shutdown_connections 2024-04-25T11:14:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.376+0000 7f91ba400640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9194074530 0x7f9194076a40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.376+0000 7f91ba400640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91b41583c0 0x7f91b41518b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.376+0000 7f91ba400640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91b40d09c0 0x7f91b41e8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.376+0000 7f91ba400640 1 --2- 172.21.15.7:0/2265969756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91b40cfb50 0x7f91b4143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.376+0000 7f91ba400640 1 -- 172.21.15.7:0/2265969756 >> 172.21.15.7:0/2265969756 conn(0x7f91b40740f0 msgr2=0x7f91b415cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.377+0000 7f91ba400640 1 -- 172.21.15.7:0/2265969756 shutdown_connections 2024-04-25T11:14:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.377+0000 7f91ba400640 1 -- 172.21.15.7:0/2265969756 wait complete. 2024-04-25T11:14:23.410 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:14:23.410 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:14:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.701+0000 7ff023200640 1 Processor -- start 2024-04-25T11:14:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.701+0000 7ff023200640 1 -- start start 2024-04-25T11:14:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.703+0000 7ff023200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff01c0b2550 0x7ff01c0b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.703+0000 7ff023200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff01c0b2a80 0x7ff01c0d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.703+0000 7ff023200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff01c160770 con 0x7ff01c0d2df0 2024-04-25T11:14:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.703+0000 7ff023200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff01c06e960 con 0x7ff01c0b2a80 2024-04-25T11:14:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.703+0000 7ff023200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff01c0d1dd0 con 0x7ff01c0b2550 2024-04-25T11:14:23.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.703+0000 7ff021e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff01c0b2550 0x7ff01c0b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:23.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.703+0000 7ff021400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff01c0b2a80 0x7ff01c0d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:23.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.704+0000 7ff022800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff01c0d2df0 0x7ff01c0b2ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:57362/0 (socket says 172.21.15.7:57362) 2024-04-25T11:14:23.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.704+0000 7ff022800640 1 -- 172.21.15.7:0/2171777370 learned_addr learned my addr 172.21.15.7:0/2171777370 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:23.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.705+0000 7ff021e00640 1 -- 172.21.15.7:0/2171777370 >> v2:172.21.15.87:3300/0 conn(0x7ff01c0b2a80 msgr2=0x7ff01c0d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.705+0000 7ff021e00640 1 --2- 172.21.15.7:0/2171777370 >> v2:172.21.15.87:3300/0 conn(0x7ff01c0b2a80 0x7ff01c0d28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.705+0000 7ff021e00640 1 -- 172.21.15.7:0/2171777370 >> v1:172.21.15.87:6789/0 conn(0x7ff01c0d2df0 legacy=0x7ff01c0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.705+0000 7ff021e00640 1 -- 172.21.15.7:0/2171777370 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c0d20b0 con 0x7ff01c0b2550 2024-04-25T11:14:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.705+0000 7ff021400640 1 --2- 172.21.15.7:0/2171777370 >> v2:172.21.15.87:3300/0 conn(0x7ff01c0b2a80 0x7ff01c0d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:14:23.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.706+0000 7ff021e00640 1 --2- 172.21.15.7:0/2171777370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff01c0b2550 0x7ff01c0b8eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff0080097a0 tx=0x7ff00802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc4069c1a0c04a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:23.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.707+0000 7ff020a00640 1 -- 172.21.15.7:0/2171777370 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008037070 con 0x7ff01c0b2550 2024-04-25T11:14:23.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.707+0000 7ff020a00640 1 -- 172.21.15.7:0/2171777370 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff00802ae40 con 0x7ff01c0b2550 2024-04-25T11:14:23.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.707+0000 7ff020a00640 1 -- 172.21.15.7:0/2171777370 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008032660 con 0x7ff01c0b2550 2024-04-25T11:14:23.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.707+0000 7ff023200640 1 -- 172.21.15.7:0/2171777370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff01c0b2550 msgr2=0x7ff01c0b8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:23.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.707+0000 7ff023200640 1 --2- 172.21.15.7:0/2171777370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff01c0b2550 0x7ff01c0b8eb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff0080097a0 tx=0x7ff00802a0f0 comp rx=0 tx=0).stop 2024-04-25T11:14:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.707+0000 7ff023200640 1 -- 172.21.15.7:0/2171777370 shutdown_connections 2024-04-25T11:14:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.707+0000 7ff023200640 1 --2- 172.21.15.7:0/2171777370 >> v2:172.21.15.87:3300/0 conn(0x7ff01c0b2a80 0x7ff01c0d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.707+0000 7ff023200640 1 --2- 172.21.15.7:0/2171777370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff01c0b2550 0x7ff01c0b8eb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.707+0000 7ff023200640 1 -- 172.21.15.7:0/2171777370 >> 172.21.15.7:0/2171777370 conn(0x7ff01c0b5240 msgr2=0x7ff01c0b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.708+0000 7ff023200640 1 -- 172.21.15.7:0/2171777370 shutdown_connections 2024-04-25T11:14:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.708+0000 7ff023200640 1 -- 172.21.15.7:0/2171777370 wait complete. 2024-04-25T11:14:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.710+0000 7ff023200640 1 Processor -- start 2024-04-25T11:14:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.710+0000 7ff023200640 1 -- start start 2024-04-25T11:14:23.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.712+0000 7ff023200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff01c0b2550 0x7ff01c15d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:23.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.712+0000 7ff023200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff01c0b2a80 0x7ff01c163170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:23.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.712+0000 7ff023200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff01c0d2df0 0x7ff01c1636b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:23.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.712+0000 7ff023200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c06e5f0 con 0x7ff01c0b2550 2024-04-25T11:14:23.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.712+0000 7ff023200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c0d1ac0 con 0x7ff01c0b2a80 2024-04-25T11:14:23.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.712+0000 7ff023200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff01c0b3700 con 0x7ff01c0d2df0 2024-04-25T11:14:23.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.712+0000 7ff021e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff01c0b2550 0x7ff01c15d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:23.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.712+0000 7ff021e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff01c0b2550 0x7ff01c15d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35372/0 (socket says 172.21.15.7:35372) 2024-04-25T11:14:23.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.712+0000 7ff021e00640 1 -- 172.21.15.7:0/3900383034 learned_addr learned my addr 172.21.15.7:0/3900383034 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.713+0000 7ff022800640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff01c0d2df0 0x7ff01c1636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.713+0000 7ff021400640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff01c0b2a80 0x7ff01c163170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.713+0000 7ff021e00640 1 -- 172.21.15.7:0/3900383034 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff01c0d2df0 msgr2=0x7ff01c1636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.713+0000 7ff021e00640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff01c0d2df0 0x7ff01c1636b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.713+0000 7ff021e00640 1 -- 172.21.15.7:0/3900383034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff01c0b2a80 msgr2=0x7ff01c163170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.713+0000 7ff021e00640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff01c0b2a80 0x7ff01c163170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.713+0000 7ff021e00640 1 -- 172.21.15.7:0/3900383034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c15dd40 con 0x7ff01c0b2550 2024-04-25T11:14:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.713+0000 7ff021400640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff01c0b2a80 0x7ff01c163170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.713+0000 7ff021e00640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff01c0b2550 0x7ff01c15d800 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff008035040 tx=0x7ff0080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.714+0000 7ff016000640 1 -- 172.21.15.7:0/3900383034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008041070 con 0x7ff01c0b2550 2024-04-25T11:14:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.714+0000 7ff016000640 1 -- 172.21.15.7:0/3900383034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff00803c460 con 0x7ff01c0b2550 2024-04-25T11:14:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.714+0000 7ff016000640 1 -- 172.21.15.7:0/3900383034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008037040 con 0x7ff01c0b2550 2024-04-25T11:14:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.714+0000 7ff023200640 1 -- 172.21.15.7:0/3900383034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01c159ea0 con 0x7ff01c0b2550 2024-04-25T11:14:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.714+0000 7ff023200640 1 -- 172.21.15.7:0/3900383034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01c159bc0 con 0x7ff01c0b2550 2024-04-25T11:14:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.715+0000 7ff023200640 1 -- 172.21.15.7:0/3900383034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff01c161560 con 0x7ff01c0b2550 2024-04-25T11:14:23.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.716+0000 7ff016000640 1 -- 172.21.15.7:0/3900383034 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff00803a070 con 0x7ff01c0b2550 2024-04-25T11:14:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.718+0000 7ff016000640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feffc0744e0 0x7feffc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.718+0000 7ff016000640 1 -- 172.21.15.7:0/3900383034 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7ff0080b5700 con 0x7ff01c0b2550 2024-04-25T11:14:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.718+0000 7ff021400640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feffc0744e0 0x7feffc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:23.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.719+0000 7ff021400640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feffc0744e0 0x7feffc0769f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0cf5d0 tx=0x7ff010008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:23.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.723+0000 7ff016000640 1 -- 172.21.15.7:0/3900383034 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff00807fcf0 con 0x7ff01c0b2550 2024-04-25T11:14:23.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:23.976+0000 7ff023200640 1 -- 172.21.15.7:0/3900383034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff01c050740 con 0x7ff01c0b2550 2024-04-25T11:14:24.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.376+0000 7ff016000640 1 -- 172.21.15.7:0/3900383034 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7ff0080b3e20 con 0x7ff01c0b2550 2024-04-25T11:14:24.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.380+0000 7ff023200640 1 -- 172.21.15.7:0/3900383034 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feffc0744e0 msgr2=0x7feffc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:24.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.380+0000 7ff023200640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feffc0744e0 0x7feffc0769f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0cf5d0 tx=0x7ff010008040 comp rx=0 tx=0).stop 2024-04-25T11:14:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.380+0000 7ff023200640 1 -- 172.21.15.7:0/3900383034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff01c0b2550 msgr2=0x7ff01c15d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.380+0000 7ff023200640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff01c0b2550 0x7ff01c15d800 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff008035040 tx=0x7ff0080046c0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.380+0000 7ff023200640 1 -- 172.21.15.7:0/3900383034 shutdown_connections 2024-04-25T11:14:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.380+0000 7ff023200640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feffc0744e0 0x7feffc0769f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.380+0000 7ff023200640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff01c0d2df0 0x7ff01c1636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.380+0000 7ff023200640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff01c0b2a80 0x7ff01c163170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.380+0000 7ff023200640 1 --2- 172.21.15.7:0/3900383034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff01c0b2550 0x7ff01c15d800 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.380+0000 7ff023200640 1 -- 172.21.15.7:0/3900383034 >> 172.21.15.7:0/3900383034 conn(0x7ff01c0b5240 msgr2=0x7ff01c0cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.380+0000 7ff023200640 1 -- 172.21.15.7:0/3900383034 shutdown_connections 2024-04-25T11:14:24.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.381+0000 7ff023200640 1 -- 172.21.15.7:0/3900383034 wait complete. 2024-04-25T11:14:24.411 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:14:24.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:24.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.700+0000 7f48fe600640 1 Processor -- start 2024-04-25T11:14:24.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.701+0000 7f48fe600640 1 -- start start 2024-04-25T11:14:24.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.703+0000 7f48fe600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f48f80aed40 0x7f48f815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:24.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.703+0000 7f48fe600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f80cebb0 con 0x7f48f80af420 2024-04-25T11:14:24.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.703+0000 7f48fe600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f80b64b0 con 0x7f48f80b5220 2024-04-25T11:14:24.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.703+0000 7f48fe600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f48f815e1f0 con 0x7f48f80aed40 2024-04-25T11:14:24.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.704+0000 7f48fdc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f48f80af420 0x7f48f80af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52940/0 (socket says 172.21.15.7:52940) 2024-04-25T11:14:24.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.704+0000 7f48fdc00640 1 -- 172.21.15.7:0/4041907588 learned_addr learned my addr 172.21.15.7:0/4041907588 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:24.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.704+0000 7f48fd200640 1 --2- 172.21.15.7:0/4041907588 >> v2:172.21.15.87:3300/0 conn(0x7f48f80aed40 0x7f48f815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:24.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.704+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1670731347 0 0) 0x7f48f0001490 con 0x7f48f80af420 2024-04-25T11:14:24.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.704+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48e0003520 con 0x7f48f80af420 2024-04-25T11:14:24.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 698104778 0 0) 0x7f48e0003520 con 0x7f48f80af420 2024-04-25T11:14:24.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 >> v2:172.21.15.87:3300/0 conn(0x7f48f80aed40 msgr2=0x7f48f815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 --2- 172.21.15.7:0/4041907588 >> v2:172.21.15.87:3300/0 conn(0x7f48f80aed40 0x7f48f815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 >> v1:172.21.15.87:6789/0 conn(0x7f48f80b5220 legacy=0x7f48f80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f806c610 con 0x7f48f80af420 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f48f0001bc0 con 0x7f48f80af420 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 >> v1:172.21.15.7:6789/0 conn(0x7f48f80af420 legacy=0x7f48f80af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e00039b0 0x7f48e0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e0004800 0x7f48e0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48e00051c0 0x7f48e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48f815e1f0 con 0x7f48e00051c0 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48e000c160 con 0x7f48e00039b0 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48e000c360 con 0x7f48e0004800 2024-04-25T11:14:24.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.705+0000 7f48fdc00640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48e00051c0 0x7f48e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:24.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.706+0000 7f48fd200640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e00039b0 0x7f48e0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:24.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.706+0000 7f48f7e00640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e0004800 0x7f48e0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:24.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.706+0000 7f48fdc00640 1 -- 172.21.15.7:0/4041907588 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e0004800 msgr2=0x7f48e0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:24.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.706+0000 7f48fdc00640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48e0004800 0x7f48e0004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.706+0000 7f48fdc00640 1 -- 172.21.15.7:0/4041907588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e00039b0 msgr2=0x7f48e0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:24.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.706+0000 7f48fdc00640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e00039b0 0x7f48e0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.706+0000 7f48fdc00640 1 -- 172.21.15.7:0/4041907588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e0003520 con 0x7f48e00051c0 2024-04-25T11:14:24.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.706+0000 7f48fd200640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e00039b0 0x7f48e0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:24.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.707+0000 7f48fdc00640 1 -- 172.21.15.7:0/4041907588 reap_dead start 2024-04-25T11:14:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.707+0000 7f48fdc00640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48e00051c0 0x7f48e000b9c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f48f000aa30 tx=0x7f48e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418c117f1f72558b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.707+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f000a650 con 0x7f48e00051c0 2024-04-25T11:14:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.707+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48f002f070 con 0x7f48e00051c0 2024-04-25T11:14:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.707+0000 7f48f7400640 1 -- 172.21.15.7:0/4041907588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f002bde0 con 0x7f48e00051c0 2024-04-25T11:14:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.708+0000 7f48fe600640 1 -- 172.21.15.7:0/4041907588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48e00051c0 msgr2=0x7f48e000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.708+0000 7f48fe600640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48e00051c0 0x7f48e000b9c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f48f000aa30 tx=0x7f48e0005830 comp rx=0 tx=0).stop 2024-04-25T11:14:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.708+0000 7f48fe600640 1 -- 172.21.15.7:0/4041907588 shutdown_connections 2024-04-25T11:14:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.708+0000 7f48fe600640 1 --2- 172.21.15.7:0/4041907588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48e00051c0 0x7f48e000b9c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.708+0000 7f48fe600640 1 -- 172.21.15.7:0/4041907588 >> 172.21.15.7:0/4041907588 conn(0x7f48f80b3820 msgr2=0x7f48f80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.708+0000 7f48fe600640 1 -- 172.21.15.7:0/4041907588 shutdown_connections 2024-04-25T11:14:24.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.708+0000 7f48fe600640 1 -- 172.21.15.7:0/4041907588 wait complete. 2024-04-25T11:14:24.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.710+0000 7f48fe600640 1 Processor -- start 2024-04-25T11:14:24.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.711+0000 7f48fe600640 1 -- start start 2024-04-25T11:14:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.712+0000 7f48fe600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e00051c0 0x7f48f81f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.712+0000 7f48fe600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f806ee70 0x7f48f806f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.712+0000 7f48fe600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48f815c220 0x7f48f815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:24.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.712+0000 7f48fe600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48f806ceb0 con 0x7f48f806ee70 2024-04-25T11:14:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.712+0000 7f48fe600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48f806cbd0 con 0x7f48e00051c0 2024-04-25T11:14:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.712+0000 7f48fe600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48f80b6110 con 0x7f48f815c220 2024-04-25T11:14:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.712+0000 7f48f7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f806ee70 0x7f48f806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.712+0000 7f48f7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f806ee70 0x7f48f806f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35390/0 (socket says 172.21.15.7:35390) 2024-04-25T11:14:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.712+0000 7f48f7e00640 1 -- 172.21.15.7:0/2449929687 learned_addr learned my addr 172.21.15.7:0/2449929687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.713+0000 7f48fdc00640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48f815c220 0x7f48f815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.713+0000 7f48fd200640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e00051c0 0x7f48f81f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.713+0000 7f48f7e00640 1 -- 172.21.15.7:0/2449929687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48f815c220 msgr2=0x7f48f815c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.713+0000 7f48f7e00640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48f815c220 0x7f48f815c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.713+0000 7f48f7e00640 1 -- 172.21.15.7:0/2449929687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e00051c0 msgr2=0x7f48f81f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:24.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.713+0000 7f48f7e00640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e00051c0 0x7f48f81f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.713+0000 7f48f7e00640 1 -- 172.21.15.7:0/2449929687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f815cb70 con 0x7f48f806ee70 2024-04-25T11:14:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.713+0000 7f48f7e00640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f806ee70 0x7f48f806f280 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f48e400d4c0 tx=0x7f48e400d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:24.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.714+0000 7f48f4c00640 1 -- 172.21.15.7:0/2449929687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4005480 con 0x7f48f806ee70 2024-04-25T11:14:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.714+0000 7f48f4c00640 1 -- 172.21.15.7:0/2449929687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e4017070 con 0x7f48f806ee70 2024-04-25T11:14:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.714+0000 7f48f4c00640 1 -- 172.21.15.7:0/2449929687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4003830 con 0x7f48f806ee70 2024-04-25T11:14:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.714+0000 7f48fe600640 1 -- 172.21.15.7:0/2449929687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f81f6180 con 0x7f48f806ee70 2024-04-25T11:14:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.714+0000 7f48fe600640 1 -- 172.21.15.7:0/2449929687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f80cb0e0 con 0x7f48f806ee70 2024-04-25T11:14:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.715+0000 7f48fe600640 1 -- 172.21.15.7:0/2449929687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f814f9c0 con 0x7f48f806ee70 2024-04-25T11:14:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.716+0000 7f48f4c00640 1 -- 172.21.15.7:0/2449929687 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f48e40039d0 con 0x7f48f806ee70 2024-04-25T11:14:24.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.717+0000 7f48f4c00640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d80744e0 0x7f48d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:24.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.717+0000 7f48f4c00640 1 -- 172.21.15.7:0/2449929687 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f48e40952e0 con 0x7f48f806ee70 2024-04-25T11:14:24.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.718+0000 7f48fd200640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d80744e0 0x7f48d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:24.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.718+0000 7f48fd200640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d80744e0 0x7f48d80769f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f48f8071200 tx=0x7f48ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:24.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.722+0000 7f48f4c00640 1 -- 172.21.15.7:0/2449929687 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48f814f9c0 con 0x7f48f806ee70 2024-04-25T11:14:24.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.966+0000 7f48fe600640 1 -- 172.21.15.7:0/2449929687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48f8050740 con 0x7f48f806ee70 2024-04-25T11:14:24.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.968+0000 7f48f4c00640 1 -- 172.21.15.7:0/2449929687 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7f48f815cb70 con 0x7f48f806ee70 2024-04-25T11:14:24.968 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:24.968 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2941354720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2941354720},{"type":"v1","addr":"172.21.15.87:6834","nonce":2941354720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3135576784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3135576784},{"type":"v1","addr":"172.21.15.87:6835","nonce":3135576784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2590173274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2590173274},{"type":"v1","addr":"172.21.15.7:6836","nonce":2590173274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:14:20.877428+0000","modified":"2024-04-25T11:14:23.977883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6120":{"gid":6120,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2403088100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2403088100},{"type":"v1","addr":"172.21.15.7:6837","nonce":2403088100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6120,"qdb_cluster":[6120]},"id":4}]} 2024-04-25T11:14:24.968 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-04-25T11:14:24.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.970+0000 7f48fe600640 1 -- 172.21.15.7:0/2449929687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d80744e0 msgr2=0x7f48d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:24.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.970+0000 7f48fe600640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d80744e0 0x7f48d80769f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f48f8071200 tx=0x7f48ec008040 comp rx=0 tx=0).stop 2024-04-25T11:14:24.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.971+0000 7f48fe600640 1 -- 172.21.15.7:0/2449929687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f806ee70 msgr2=0x7f48f806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:24.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.971+0000 7f48fe600640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f806ee70 0x7f48f806f280 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f48e400d4c0 tx=0x7f48e400d980 comp rx=0 tx=0).stop 2024-04-25T11:14:24.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.971+0000 7f48fe600640 1 -- 172.21.15.7:0/2449929687 shutdown_connections 2024-04-25T11:14:24.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.971+0000 7f48fe600640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48d80744e0 0x7f48d80769f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.971+0000 7f48fe600640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48f815c220 0x7f48f815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.971+0000 7f48fe600640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f806ee70 0x7f48f806f280 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.971+0000 7f48fe600640 1 --2- 172.21.15.7:0/2449929687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48e00051c0 0x7f48f81f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.971+0000 7f48fe600640 1 -- 172.21.15.7:0/2449929687 >> 172.21.15.7:0/2449929687 conn(0x7f48f80b3820 msgr2=0x7f48f806d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.971+0000 7f48fe600640 1 -- 172.21.15.7:0/2449929687 shutdown_connections 2024-04-25T11:14:24.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:24.971+0000 7f48fe600640 1 -- 172.21.15.7:0/2449929687 wait complete. 2024-04-25T11:14:24.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:25.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.284+0000 7fe3bd600640 1 Processor -- start 2024-04-25T11:14:25.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.285+0000 7fe3bd600640 1 -- start start 2024-04-25T11:14:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.287+0000 7fe3bd600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3b80aed40 0x7fe3b815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.287+0000 7fe3bd600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 0x7fe3b80ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.287+0000 7fe3bd600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b80b64b0 con 0x7fe3b80af420 2024-04-25T11:14:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.287+0000 7fe3bd600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b815e1f0 con 0x7fe3b80aed40 2024-04-25T11:14:25.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.287+0000 7fe3bd600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b806c610 con 0x7fe3b80b5220 2024-04-25T11:14:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.288+0000 7fe3b7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3b80aed40 0x7fe3b815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.288+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 0x7fe3b80ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.288+0000 7fe3b7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3b80aed40 0x7fe3b815dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:33844/0 (socket says 172.21.15.7:33844) 2024-04-25T11:14:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.288+0000 7fe3b7e00640 1 -- 172.21.15.7:0/1191151135 learned_addr learned my addr 172.21.15.7:0/1191151135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:25.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.288+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 0x7fe3b80ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:46514/0 (socket says 172.21.15.7:46514) 2024-04-25T11:14:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.289+0000 7fe3b7e00640 1 -- 172.21.15.7:0/1191151135 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 msgr2=0x7fe3b80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.289+0000 7fe3b7e00640 1 --2- 172.21.15.7:0/1191151135 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 0x7fe3b80ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.289+0000 7fe3b7e00640 1 -- 172.21.15.7:0/1191151135 >> v1:172.21.15.87:6789/0 conn(0x7fe3b80af420 legacy=0x7fe3b80af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.289+0000 7fe3b7e00640 1 -- 172.21.15.7:0/1191151135 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b806c8f0 con 0x7fe3b80aed40 2024-04-25T11:14:25.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.289+0000 7fe3b7400640 1 --2- 172.21.15.7:0/1191151135 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 0x7fe3b80ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:14:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.290+0000 7fe3b7e00640 1 --2- 172.21.15.7:0/1191151135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b80aed40 0x7fe3b815dcb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe3b815e370 tx=0x7fe3a802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb1350bd95e9b012 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.291+0000 7fe3b6a00640 1 -- 172.21.15.7:0/1191151135 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8036070 con 0x7fe3b80aed40 2024-04-25T11:14:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.291+0000 7fe3b6a00640 1 -- 172.21.15.7:0/1191151135 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a8031600 con 0x7fe3b80aed40 2024-04-25T11:14:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.291+0000 7fe3b6a00640 1 -- 172.21.15.7:0/1191151135 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a80319d0 con 0x7fe3b80aed40 2024-04-25T11:14:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.292+0000 7fe3bd600640 1 -- 172.21.15.7:0/1191151135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b80aed40 msgr2=0x7fe3b815dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.292+0000 7fe3bd600640 1 --2- 172.21.15.7:0/1191151135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b80aed40 0x7fe3b815dcb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe3b815e370 tx=0x7fe3a802a710 comp rx=0 tx=0).stop 2024-04-25T11:14:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.292+0000 7fe3bd600640 1 -- 172.21.15.7:0/1191151135 shutdown_connections 2024-04-25T11:14:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.292+0000 7fe3bd600640 1 --2- 172.21.15.7:0/1191151135 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 0x7fe3b80ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.292+0000 7fe3bd600640 1 --2- 172.21.15.7:0/1191151135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b80aed40 0x7fe3b815dcb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.292+0000 7fe3bd600640 1 -- 172.21.15.7:0/1191151135 >> 172.21.15.7:0/1191151135 conn(0x7fe3b80b3820 msgr2=0x7fe3b80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.292+0000 7fe3bd600640 1 -- 172.21.15.7:0/1191151135 shutdown_connections 2024-04-25T11:14:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.293+0000 7fe3bd600640 1 -- 172.21.15.7:0/1191151135 wait complete. 2024-04-25T11:14:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.295+0000 7fe3bd600640 1 Processor -- start 2024-04-25T11:14:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.295+0000 7fe3bd600640 1 -- start start 2024-04-25T11:14:25.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.297+0000 7fe3bd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b80af420 0x7fe3b809c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.297+0000 7fe3bd600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 0x7fe3b81597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.297+0000 7fe3bd600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b809c640 0x7fe3b8159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.297+0000 7fe3bd600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b80b6110 con 0x7fe3b80af420 2024-04-25T11:14:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.297+0000 7fe3bd600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b806ceb0 con 0x7fe3b809c640 2024-04-25T11:14:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.297+0000 7fe3bd600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b80b6290 con 0x7fe3b80b5220 2024-04-25T11:14:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.297+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b80af420 0x7fe3b809c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.297+0000 7fe3b7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b80af420 0x7fe3b809c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35396/0 (socket says 172.21.15.7:35396) 2024-04-25T11:14:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.297+0000 7fe3b7e00640 1 -- 172.21.15.7:0/3459644672 learned_addr learned my addr 172.21.15.7:0/3459644672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.298+0000 7fe3b7400640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 0x7fe3b81597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.298+0000 7fe3bcc00640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b809c640 0x7fe3b8159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.298+0000 7fe3b7e00640 1 -- 172.21.15.7:0/3459644672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 msgr2=0x7fe3b81597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.298+0000 7fe3b7e00640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 0x7fe3b81597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.298+0000 7fe3b7e00640 1 -- 172.21.15.7:0/3459644672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b809c640 msgr2=0x7fe3b8159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.298+0000 7fe3b7e00640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b809c640 0x7fe3b8159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.298+0000 7fe3b7e00640 1 -- 172.21.15.7:0/3459644672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b809e180 con 0x7fe3b80af420 2024-04-25T11:14:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.298+0000 7fe3b7e00640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b80af420 0x7fe3b809c100 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe3a802ac80 tx=0x7fe3a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.299+0000 7fe3afe00640 1 -- 172.21.15.7:0/3459644672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8036050 con 0x7fe3b80af420 2024-04-25T11:14:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.299+0000 7fe3afe00640 1 -- 172.21.15.7:0/3459644672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a8042070 con 0x7fe3b80af420 2024-04-25T11:14:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.299+0000 7fe3afe00640 1 -- 172.21.15.7:0/3459644672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8009510 con 0x7fe3b80af420 2024-04-25T11:14:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.299+0000 7fe3bd600640 1 -- 172.21.15.7:0/3459644672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b809e460 con 0x7fe3b80af420 2024-04-25T11:14:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.299+0000 7fe3bd600640 1 -- 172.21.15.7:0/3459644672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b809ea20 con 0x7fe3b80af420 2024-04-25T11:14:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.300+0000 7fe3bd600640 1 -- 172.21.15.7:0/3459644672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b8050740 con 0x7fe3b80af420 2024-04-25T11:14:25.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.302+0000 7fe3afe00640 1 -- 172.21.15.7:0/3459644672 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe3a8007c70 con 0x7fe3b80af420 2024-04-25T11:14:25.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.303+0000 7fe3afe00640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3980744b0 0x7fe3980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.303+0000 7fe3afe00640 1 -- 172.21.15.7:0/3459644672 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7fe3a80b6200 con 0x7fe3b80af420 2024-04-25T11:14:25.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.304+0000 7fe3b7400640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3980744b0 0x7fe3980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.305+0000 7fe3b7400640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3980744b0 0x7fe3980769c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe3b815ad90 tx=0x7fe3a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:25.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.309+0000 7fe3afe00640 1 -- 172.21.15.7:0/3459644672 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe3a8080760 con 0x7fe3b80af420 2024-04-25T11:14:25.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.521+0000 7fe3bd600640 1 -- 172.21.15.7:0/3459644672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3b80b3dd0 con 0x7fe3b80af420 2024-04-25T11:14:25.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.523+0000 7fe3afe00640 1 -- 172.21.15.7:0/3459644672 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16788 (secure 0 0 0) 0x7fe3a8080060 con 0x7fe3b80af420 2024-04-25T11:14:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:25.524 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:22.341351+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:14:19.115741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T11:14:19.849856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:25.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.527+0000 7fe3bd600640 1 -- 172.21.15.7:0/3459644672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3980744b0 msgr2=0x7fe3980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:25.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.527+0000 7fe3bd600640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3980744b0 0x7fe3980769c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe3b815ad90 tx=0x7fe3a4008040 comp rx=0 tx=0).stop 2024-04-25T11:14:25.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.527+0000 7fe3bd600640 1 -- 172.21.15.7:0/3459644672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b80af420 msgr2=0x7fe3b809c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:25.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.527+0000 7fe3bd600640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b80af420 0x7fe3b809c100 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe3a802ac80 tx=0x7fe3a8004a40 comp rx=0 tx=0).stop 2024-04-25T11:14:25.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.527+0000 7fe3bd600640 1 -- 172.21.15.7:0/3459644672 shutdown_connections 2024-04-25T11:14:25.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.527+0000 7fe3bd600640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3980744b0 0x7fe3980769c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.527+0000 7fe3bd600640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b809c640 0x7fe3b8159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.527+0000 7fe3bd600640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b80b5220 0x7fe3b81597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.527+0000 7fe3bd600640 1 --2- 172.21.15.7:0/3459644672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b80af420 0x7fe3b809c100 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.527+0000 7fe3bd600640 1 -- 172.21.15.7:0/3459644672 >> 172.21.15.7:0/3459644672 conn(0x7fe3b80b3820 msgr2=0x7fe3b80af830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:25.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.528+0000 7fe3bd600640 1 -- 172.21.15.7:0/3459644672 shutdown_connections 2024-04-25T11:14:25.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.528+0000 7fe3bd600640 1 -- 172.21.15.7:0/3459644672 wait complete. 2024-04-25T11:14:25.553 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:14:25.553 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:14:25.807 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-04-25T11:14:25.807+0000 7f5efcc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:14:25.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.833+0000 7f5a3be00640 1 Processor -- start 2024-04-25T11:14:25.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.833+0000 7f5a3be00640 1 -- start start 2024-04-25T11:14:25.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.836+0000 7f5a3be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a3c06dc70 0x7f5a3c0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.836+0000 7f5a3be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a3c0d24f0 0x7f5a3c0aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.836+0000 7f5a3be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a3c0b2820 con 0x7f5a3c0d2900 2024-04-25T11:14:25.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.836+0000 7f5a3be00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5a3c0b2260 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.836+0000 7f5a3be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3c0b2540 con 0x7f5a3c06dc70 2024-04-25T11:14:25.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.836+0000 7f5a33e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a3c0d24f0 0x7f5a3c0aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.836+0000 7f5a33e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a3c0d24f0 0x7f5a3c0aefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35404/0 (socket says 172.21.15.7:35404) 2024-04-25T11:14:25.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.836+0000 7f5a33e00640 1 -- 172.21.15.7:0/2344356637 learned_addr learned my addr 172.21.15.7:0/2344356637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:25.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.837+0000 7f5a3aa00640 1 --2- 172.21.15.7:0/2344356637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a3c06dc70 0x7f5a3c0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.837+0000 7f5a33e00640 1 -- 172.21.15.7:0/2344356637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a3c06dc70 msgr2=0x7f5a3c0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:25.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.837+0000 7f5a33e00640 1 --2- 172.21.15.7:0/2344356637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a3c06dc70 0x7f5a3c0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.837+0000 7f5a33e00640 1 -- 172.21.15.7:0/2344356637 >> v1:172.21.15.87:6789/0 conn(0x7f5a3c0d2900 legacy=0x7f5a3c0af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:25.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.837+0000 7f5a33e00640 1 -- 172.21.15.7:0/2344356637 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c06d120 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.838+0000 7f5a33e00640 1 --2- 172.21.15.7:0/2344356637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a3c0d24f0 0x7f5a3c0aefa0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009870 tx=0x7f5a2402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bea5fde20eb778 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:25.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.838+0000 7f5a3a000640 1 -- 172.21.15.7:0/2344356637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24037070 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.838+0000 7f5a3a000640 1 -- 172.21.15.7:0/2344356637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a24032440 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.838+0000 7f5a3a000640 1 -- 172.21.15.7:0/2344356637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24032810 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.839+0000 7f5a3be00640 1 -- 172.21.15.7:0/2344356637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a3c0d24f0 msgr2=0x7f5a3c0aefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:25.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.839+0000 7f5a3be00640 1 --2- 172.21.15.7:0/2344356637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a3c0d24f0 0x7f5a3c0aefa0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009870 tx=0x7f5a2402a210 comp rx=0 tx=0).stop 2024-04-25T11:14:25.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.839+0000 7f5a3be00640 1 -- 172.21.15.7:0/2344356637 shutdown_connections 2024-04-25T11:14:25.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.839+0000 7f5a3be00640 1 --2- 172.21.15.7:0/2344356637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a3c0d24f0 0x7f5a3c0aefa0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.839+0000 7f5a3be00640 1 --2- 172.21.15.7:0/2344356637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a3c06dc70 0x7f5a3c0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.839+0000 7f5a3be00640 1 -- 172.21.15.7:0/2344356637 >> 172.21.15.7:0/2344356637 conn(0x7f5a3c073b60 msgr2=0x7f5a3c0b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:25.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.839+0000 7f5a3be00640 1 -- 172.21.15.7:0/2344356637 shutdown_connections 2024-04-25T11:14:25.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.839+0000 7f5a3be00640 1 -- 172.21.15.7:0/2344356637 wait complete. 2024-04-25T11:14:25.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.842+0000 7f5a3be00640 1 Processor -- start 2024-04-25T11:14:25.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.842+0000 7f5a3be00640 1 -- start start 2024-04-25T11:14:25.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.843+0000 7f5a3be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a3c06dc70 0x7f5a3c14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.843+0000 7f5a3be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a3c0d24f0 0x7f5a3c0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.843+0000 7f5a3be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a3c0d2900 0x7f5a3c0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a3be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c0b1ea0 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a3be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c09b990 con 0x7f5a3c06dc70 2024-04-25T11:14:25.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a3be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3c09b540 con 0x7f5a3c0d2900 2024-04-25T11:14:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a3c0d24f0 0x7f5a3c0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a3c0d24f0 0x7f5a3c0ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35414/0 (socket says 172.21.15.7:35414) 2024-04-25T11:14:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a33e00640 1 -- 172.21.15.7:0/3712550431 learned_addr learned my addr 172.21.15.7:0/3712550431 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a3aa00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a3c06dc70 0x7f5a3c14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a33e00640 1 -- 172.21.15.7:0/3712550431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a3c0d2900 msgr2=0x7f5a3c0ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:25.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a3b400640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a3c0d2900 0x7f5a3c0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a33e00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a3c0d2900 0x7f5a3c0ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a33e00640 1 -- 172.21.15.7:0/3712550431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a3c06dc70 msgr2=0x7f5a3c14f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a33e00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a3c06dc70 0x7f5a3c14f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a33e00640 1 -- 172.21.15.7:0/3712550431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c14fe50 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.844+0000 7f5a3b400640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a3c0d2900 0x7f5a3c0ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.845+0000 7f5a33e00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a3c0d24f0 0x7f5a3c0ce770 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5a24000c00 tx=0x7f5a240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.845+0000 7f5a32a00640 1 -- 172.21.15.7:0/3712550431 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24041020 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.845+0000 7f5a32a00640 1 -- 172.21.15.7:0/3712550431 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a240306c0 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.845+0000 7f5a3be00640 1 -- 172.21.15.7:0/3712550431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3c0cecb0 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.845+0000 7f5a32a00640 1 -- 172.21.15.7:0/3712550431 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24037040 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.845+0000 7f5a3be00640 1 -- 172.21.15.7:0/3712550431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3c0cf4a0 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.847+0000 7f5a3be00640 1 -- 172.21.15.7:0/3712550431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3c0cef30 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.848+0000 7f5a32a00640 1 -- 172.21.15.7:0/3712550431 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5a24044020 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.849+0000 7f5a32a00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a180744e0 0x7f5a180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:25.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.850+0000 7f5a32a00640 1 -- 172.21.15.7:0/3712550431 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f5a240b5cf0 con 0x7f5a3c0d24f0 2024-04-25T11:14:25.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.850+0000 7f5a3aa00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a180744e0 0x7f5a180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:25.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.851+0000 7f5a3aa00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a180744e0 0x7f5a180769f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c006fd0 tx=0x7f5a2c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:25.855+0000 7f5a32a00640 1 -- 172.21.15.7:0/3712550431 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a2404b030 con 0x7f5a3c0d24f0 2024-04-25T11:14:26.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.056+0000 7f5a3be00640 1 -- 172.21.15.7:0/3712550431 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a3c050740 con 0x7f5a180744e0 2024-04-25T11:14:26.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.076+0000 7f5a32a00640 1 -- 172.21.15.7:0/3712550431 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248630 (secure 0 0 0) 0x7f5a3c050740 con 0x7f5a180744e0 2024-04-25T11:14:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:26.084 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:14:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 -- 172.21.15.7:0/3712550431 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a180744e0 msgr2=0x7f5a180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a180744e0 0x7f5a180769f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c006fd0 tx=0x7f5a2c008040 comp rx=0 tx=0).stop 2024-04-25T11:14:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 -- 172.21.15.7:0/3712550431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a3c0d24f0 msgr2=0x7f5a3c0ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a3c0d24f0 0x7f5a3c0ce770 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5a24000c00 tx=0x7f5a240046c0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 -- 172.21.15.7:0/3712550431 shutdown_connections 2024-04-25T11:14:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a180744e0 0x7f5a180769f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a3c0d2900 0x7f5a3c0ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a3c0d24f0 0x7f5a3c0ce770 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 --2- 172.21.15.7:0/3712550431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a3c06dc70 0x7f5a3c14f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 -- 172.21.15.7:0/3712550431 >> 172.21.15.7:0/3712550431 conn(0x7f5a3c073b60 msgr2=0x7f5a3c15cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:26.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 -- 172.21.15.7:0/3712550431 shutdown_connections 2024-04-25T11:14:26.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.083+0000 7f5a3be00640 1 -- 172.21.15.7:0/3712550431 wait complete. 2024-04-25T11:14:26.113 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-04-25T11:14:25.875776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592562,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":118,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248556,"kb_used_data":7416,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726164,"statfs":{"total":773094113280,"available":772839591936,"internally_reserved":0,"allocated":7593984,"data_stored":4052297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001944"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045327+0000","last_change":"2024-04-25T11:14:21.374239+0000","last_active":"2024-04-25T11:14:24.045327+0000","last_peered":"2024-04-25T11:14:24.045327+0000","last_clean":"2024-04-25T11:14:24.045327+0000","last_became_active":"2024-04-25T11:14:21.373912+0000","last_became_peered":"2024-04-25T11:14:21.373912+0000","last_unstale":"2024-04-25T11:14:24.045327+0000","last_undegraded":"2024-04-25T11:14:24.045327+0000","last_fullsized":"2024-04-25T11:14:24.045327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:02.752329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045090+0000","last_change":"2024-04-25T11:14:21.358156+0000","last_active":"2024-04-25T11:14:24.045090+0000","last_peered":"2024-04-25T11:14:24.045090+0000","last_clean":"2024-04-25T11:14:24.045090+0000","last_became_active":"2024-04-25T11:14:21.357825+0000","last_became_peered":"2024-04-25T11:14:21.357825+0000","last_unstale":"2024-04-25T11:14:24.045090+0000","last_undegraded":"2024-04-25T11:14:24.045090+0000","last_fullsized":"2024-04-25T11:14:24.045090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.581113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.306713+0000","last_change":"2024-04-25T11:14:20.306713+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.306713+0000","last_undegraded":"2024-04-25T11:14:20.306713+0000","last_fullsized":"2024-04-25T11:14:20.306713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.323487+0000","last_change":"2024-04-25T11:14:20.323487+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.323487+0000","last_undegraded":"2024-04-25T11:14:20.323487+0000","last_fullsized":"2024-04-25T11:14:20.323487+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.334553+0000","last_change":"2024-04-25T11:14:20.334553+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.334553+0000","last_undegraded":"2024-04-25T11:14:20.334553+0000","last_fullsized":"2024-04-25T11:14:20.334553+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962720+0000","last_change":"2024-04-25T11:14:21.381797+0000","last_active":"2024-04-25T11:14:22.962720+0000","last_peered":"2024-04-25T11:14:22.962720+0000","last_clean":"2024-04-25T11:14:22.962720+0000","last_became_active":"2024-04-25T11:14:21.381332+0000","last_became_peered":"2024-04-25T11:14:21.381332+0000","last_unstale":"2024-04-25T11:14:22.962720+0000","last_undegraded":"2024-04-25T11:14:22.962720+0000","last_fullsized":"2024-04-25T11:14:22.962720+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.347162+0000","last_change":"2024-04-25T11:14:21.367266+0000","last_active":"2024-04-25T11:14:23.347162+0000","last_peered":"2024-04-25T11:14:23.347162+0000","last_clean":"2024-04-25T11:14:23.347162+0000","last_became_active":"2024-04-25T11:14:21.366376+0000","last_became_peered":"2024-04-25T11:14:21.366376+0000","last_unstale":"2024-04-25T11:14:23.347162+0000","last_undegraded":"2024-04-25T11:14:23.347162+0000","last_fullsized":"2024-04-25T11:14:23.347162+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:26.828975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328195+0000","last_change":"2024-04-25T11:14:20.328195+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328195+0000","last_undegraded":"2024-04-25T11:14:20.328195+0000","last_fullsized":"2024-04-25T11:14:20.328195+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.307139+0000","last_change":"2024-04-25T11:14:20.307139+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.307139+0000","last_undegraded":"2024-04-25T11:14:20.307139+0000","last_fullsized":"2024-04-25T11:14:20.307139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.307340+0000","last_change":"2024-04-25T11:14:20.307340+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.307340+0000","last_undegraded":"2024-04-25T11:14:20.307340+0000","last_fullsized":"2024-04-25T11:14:20.307340+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.320107+0000","last_change":"2024-04-25T11:14:20.320107+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.320107+0000","last_undegraded":"2024-04-25T11:14:20.320107+0000","last_fullsized":"2024-04-25T11:14:20.320107+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.327261+0000","last_change":"2024-04-25T11:14:20.327261+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.327261+0000","last_undegraded":"2024-04-25T11:14:20.327261+0000","last_fullsized":"2024-04-25T11:14:20.327261+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328318+0000","last_change":"2024-04-25T11:14:20.328318+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328318+0000","last_undegraded":"2024-04-25T11:14:20.328318+0000","last_fullsized":"2024-04-25T11:14:20.328318+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.309144+0000","last_change":"2024-04-25T11:14:20.309144+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.309144+0000","last_undegraded":"2024-04-25T11:14:20.309144+0000","last_fullsized":"2024-04-25T11:14:20.309144+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.307698+0000","last_change":"2024-04-25T11:14:20.307698+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.307698+0000","last_undegraded":"2024-04-25T11:14:20.307698+0000","last_fullsized":"2024-04-25T11:14:20.307698+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346754+0000","last_change":"2024-04-25T11:14:21.374629+0000","last_active":"2024-04-25T11:14:23.346754+0000","last_peered":"2024-04-25T11:14:23.346754+0000","last_clean":"2024-04-25T11:14:23.346754+0000","last_became_active":"2024-04-25T11:14:21.373757+0000","last_became_peered":"2024-04-25T11:14:21.373757+0000","last_unstale":"2024-04-25T11:14:23.346754+0000","last_undegraded":"2024-04-25T11:14:23.346754+0000","last_fullsized":"2024-04-25T11:14:23.346754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:46.489832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045260+0000","last_change":"2024-04-25T11:14:21.384370+0000","last_active":"2024-04-25T11:14:24.045260+0000","last_peered":"2024-04-25T11:14:24.045260+0000","last_clean":"2024-04-25T11:14:24.045260+0000","last_became_active":"2024-04-25T11:14:21.383916+0000","last_became_peered":"2024-04-25T11:14:21.383916+0000","last_unstale":"2024-04-25T11:14:24.045260+0000","last_undegraded":"2024-04-25T11:14:24.045260+0000","last_fullsized":"2024-04-25T11:14:24.045260+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.182058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.323323+0000","last_change":"2024-04-25T11:14:20.323323+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.323323+0000","last_undegraded":"2024-04-25T11:14:20.323323+0000","last_fullsized":"2024-04-25T11:14:20.323323+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.325689+0000","last_change":"2024-04-25T11:14:20.325689+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.325689+0000","last_undegraded":"2024-04-25T11:14:20.325689+0000","last_fullsized":"2024-04-25T11:14:20.325689+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362235+0000","last_change":"2024-04-25T11:14:21.370707+0000","last_active":"2024-04-25T11:14:22.362235+0000","last_peered":"2024-04-25T11:14:22.362235+0000","last_clean":"2024-04-25T11:14:22.362235+0000","last_became_active":"2024-04-25T11:14:21.370409+0000","last_became_peered":"2024-04-25T11:14:21.370409+0000","last_unstale":"2024-04-25T11:14:22.362235+0000","last_undegraded":"2024-04-25T11:14:22.362235+0000","last_fullsized":"2024-04-25T11:14:22.362235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.327665+0000","last_change":"2024-04-25T11:14:20.327665+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.327665+0000","last_undegraded":"2024-04-25T11:14:20.327665+0000","last_fullsized":"2024-04-25T11:14:20.327665+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362273+0000","last_change":"2024-04-25T11:14:21.380852+0000","last_active":"2024-04-25T11:14:22.362273+0000","last_peered":"2024-04-25T11:14:22.362273+0000","last_clean":"2024-04-25T11:14:22.362273+0000","last_became_active":"2024-04-25T11:14:21.380226+0000","last_became_peered":"2024-04-25T11:14:21.380226+0000","last_unstale":"2024-04-25T11:14:22.362273+0000","last_undegraded":"2024-04-25T11:14:22.362273+0000","last_fullsized":"2024-04-25T11:14:22.362273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:12.837877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361853+0000","last_change":"2024-04-25T11:14:21.365901+0000","last_active":"2024-04-25T11:14:22.361853+0000","last_peered":"2024-04-25T11:14:22.361853+0000","last_clean":"2024-04-25T11:14:22.361853+0000","last_became_active":"2024-04-25T11:14:21.365551+0000","last_became_peered":"2024-04-25T11:14:21.365551+0000","last_unstale":"2024-04-25T11:14:22.361853+0000","last_undegraded":"2024-04-25T11:14:22.361853+0000","last_fullsized":"2024-04-25T11:14:22.361853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:42.952154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.305880+0000","last_change":"2024-04-25T11:14:20.305880+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.305880+0000","last_undegraded":"2024-04-25T11:14:20.305880+0000","last_fullsized":"2024-04-25T11:14:20.305880+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328612+0000","last_change":"2024-04-25T11:14:20.328612+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328612+0000","last_undegraded":"2024-04-25T11:14:20.328612+0000","last_fullsized":"2024-04-25T11:14:20.328612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328574+0000","last_change":"2024-04-25T11:14:20.328574+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328574+0000","last_undegraded":"2024-04-25T11:14:20.328574+0000","last_fullsized":"2024-04-25T11:14:20.328574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.318869+0000","last_change":"2024-04-25T11:14:20.318869+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.318869+0000","last_undegraded":"2024-04-25T11:14:20.318869+0000","last_fullsized":"2024-04-25T11:14:20.318869+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044519+0000","last_change":"2024-04-25T11:14:21.371250+0000","last_active":"2024-04-25T11:14:24.044519+0000","last_peered":"2024-04-25T11:14:24.044519+0000","last_clean":"2024-04-25T11:14:24.044519+0000","last_became_active":"2024-04-25T11:14:21.370864+0000","last_became_peered":"2024-04-25T11:14:21.370864+0000","last_unstale":"2024-04-25T11:14:24.044519+0000","last_undegraded":"2024-04-25T11:14:24.044519+0000","last_fullsized":"2024-04-25T11:14:24.044519+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:09.063896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.305995+0000","last_change":"2024-04-25T11:14:20.305995+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.305995+0000","last_undegraded":"2024-04-25T11:14:20.305995+0000","last_fullsized":"2024-04-25T11:14:20.305995+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328804+0000","last_change":"2024-04-25T11:14:20.328804+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328804+0000","last_undegraded":"2024-04-25T11:14:20.328804+0000","last_fullsized":"2024-04-25T11:14:20.328804+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.318482+0000","last_change":"2024-04-25T11:14:20.318482+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.318482+0000","last_undegraded":"2024-04-25T11:14:20.318482+0000","last_fullsized":"2024-04-25T11:14:20.318482+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961814+0000","last_change":"2024-04-25T11:14:21.360674+0000","last_active":"2024-04-25T11:14:22.961814+0000","last_peered":"2024-04-25T11:14:22.961814+0000","last_clean":"2024-04-25T11:14:22.961814+0000","last_became_active":"2024-04-25T11:14:21.359863+0000","last_became_peered":"2024-04-25T11:14:21.359863+0000","last_unstale":"2024-04-25T11:14:22.961814+0000","last_undegraded":"2024-04-25T11:14:22.961814+0000","last_fullsized":"2024-04-25T11:14:22.961814+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:49.430307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.323584+0000","last_change":"2024-04-25T11:14:20.323584+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.323584+0000","last_undegraded":"2024-04-25T11:14:20.323584+0000","last_fullsized":"2024-04-25T11:14:20.323584+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"67'4","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045418+0000","last_change":"2024-04-25T11:14:20.334352+0000","last_active":"2024-04-25T11:14:24.045418+0000","last_peered":"2024-04-25T11:14:24.045418+0000","last_clean":"2024-04-25T11:14:24.045418+0000","last_became_active":"2024-04-25T11:14:20.329783+0000","last_became_peered":"2024-04-25T11:14:20.329783+0000","last_unstale":"2024-04-25T11:14:24.045418+0000","last_undegraded":"2024-04-25T11:14:24.045418+0000","last_fullsized":"2024-04-25T11:14:24.045418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:55.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.317695+0000","last_change":"2024-04-25T11:14:20.317695+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.317695+0000","last_undegraded":"2024-04-25T11:14:20.317695+0000","last_fullsized":"2024-04-25T11:14:20.317695+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962745+0000","last_change":"2024-04-25T11:14:20.346713+0000","last_active":"2024-04-25T11:14:22.962745+0000","last_peered":"2024-04-25T11:14:22.962745+0000","last_clean":"2024-04-25T11:14:22.962745+0000","last_became_active":"2024-04-25T11:14:20.346320+0000","last_became_peered":"2024-04-25T11:14:20.346320+0000","last_unstale":"2024-04-25T11:14:22.962745+0000","last_undegraded":"2024-04-25T11:14:22.962745+0000","last_fullsized":"2024-04-25T11:14:22.962745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.559248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962535+0000","last_change":"2024-04-25T11:14:21.356785+0000","last_active":"2024-04-25T11:14:22.962535+0000","last_peered":"2024-04-25T11:14:22.962535+0000","last_clean":"2024-04-25T11:14:22.962535+0000","last_became_active":"2024-04-25T11:14:21.356335+0000","last_became_peered":"2024-04-25T11:14:21.356335+0000","last_unstale":"2024-04-25T11:14:22.962535+0000","last_undegraded":"2024-04-25T11:14:22.962535+0000","last_fullsized":"2024-04-25T11:14:22.962535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.363496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962440+0000","last_change":"2024-04-25T11:14:20.346384+0000","last_active":"2024-04-25T11:14:22.962440+0000","last_peered":"2024-04-25T11:14:22.962440+0000","last_clean":"2024-04-25T11:14:22.962440+0000","last_became_active":"2024-04-25T11:14:20.346031+0000","last_became_peered":"2024-04-25T11:14:20.346031+0000","last_unstale":"2024-04-25T11:14:22.962440+0000","last_undegraded":"2024-04-25T11:14:22.962440+0000","last_fullsized":"2024-04-25T11:14:22.962440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:22.963995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.308498+0000","last_change":"2024-04-25T11:14:20.308498+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.308498+0000","last_undegraded":"2024-04-25T11:14:20.308498+0000","last_fullsized":"2024-04-25T11:14:20.308498+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962159+0000","last_change":"2024-04-25T11:14:20.337082+0000","last_active":"2024-04-25T11:14:22.962159+0000","last_peered":"2024-04-25T11:14:22.962159+0000","last_clean":"2024-04-25T11:14:22.962159+0000","last_became_active":"2024-04-25T11:14:20.336766+0000","last_became_peered":"2024-04-25T11:14:20.336766+0000","last_unstale":"2024-04-25T11:14:22.962159+0000","last_undegraded":"2024-04-25T11:14:22.962159+0000","last_fullsized":"2024-04-25T11:14:22.962159+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:12.462630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346829+0000","last_change":"2024-04-25T11:14:21.346395+0000","last_active":"2024-04-25T11:14:23.346829+0000","last_peered":"2024-04-25T11:14:23.346829+0000","last_clean":"2024-04-25T11:14:23.346829+0000","last_became_active":"2024-04-25T11:14:21.345933+0000","last_became_peered":"2024-04-25T11:14:21.345933+0000","last_unstale":"2024-04-25T11:14:23.346829+0000","last_undegraded":"2024-04-25T11:14:23.346829+0000","last_fullsized":"2024-04-25T11:14:23.346829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.002374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362073+0000","last_change":"2024-04-25T11:14:20.335533+0000","last_active":"2024-04-25T11:14:22.362073+0000","last_peered":"2024-04-25T11:14:22.362073+0000","last_clean":"2024-04-25T11:14:22.362073+0000","last_became_active":"2024-04-25T11:14:20.335228+0000","last_became_peered":"2024-04-25T11:14:20.335228+0000","last_unstale":"2024-04-25T11:14:22.362073+0000","last_undegraded":"2024-04-25T11:14:22.362073+0000","last_fullsized":"2024-04-25T11:14:22.362073+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.347332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044109+0000","last_change":"2024-04-25T11:14:21.384939+0000","last_active":"2024-04-25T11:14:24.044109+0000","last_peered":"2024-04-25T11:14:24.044109+0000","last_clean":"2024-04-25T11:14:24.044109+0000","last_became_active":"2024-04-25T11:14:21.384525+0000","last_became_peered":"2024-04-25T11:14:21.384525+0000","last_unstale":"2024-04-25T11:14:24.044109+0000","last_undegraded":"2024-04-25T11:14:24.044109+0000","last_fullsized":"2024-04-25T11:14:24.044109+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:19.066276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361973+0000","last_change":"2024-04-25T11:14:20.317044+0000","last_active":"2024-04-25T11:14:22.361973+0000","last_peered":"2024-04-25T11:14:22.361973+0000","last_clean":"2024-04-25T11:14:22.361973+0000","last_became_active":"2024-04-25T11:14:20.316634+0000","last_became_peered":"2024-04-25T11:14:20.316634+0000","last_unstale":"2024-04-25T11:14:22.361973+0000","last_undegraded":"2024-04-25T11:14:22.361973+0000","last_fullsized":"2024-04-25T11:14:22.361973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:12.462630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.325823+0000","last_change":"2024-04-25T11:14:20.325823+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.325823+0000","last_undegraded":"2024-04-25T11:14:20.325823+0000","last_fullsized":"2024-04-25T11:14:20.325823+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361960+0000","last_change":"2024-04-25T11:14:20.339374+0000","last_active":"2024-04-25T11:14:22.361960+0000","last_peered":"2024-04-25T11:14:22.361960+0000","last_clean":"2024-04-25T11:14:22.361960+0000","last_became_active":"2024-04-25T11:14:20.339026+0000","last_became_peered":"2024-04-25T11:14:20.339026+0000","last_unstale":"2024-04-25T11:14:22.361960+0000","last_undegraded":"2024-04-25T11:14:22.361960+0000","last_fullsized":"2024-04-25T11:14:22.361960+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:09.182038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362009+0000","last_change":"2024-04-25T11:14:21.380186+0000","last_active":"2024-04-25T11:14:22.362009+0000","last_peered":"2024-04-25T11:14:22.362009+0000","last_clean":"2024-04-25T11:14:22.362009+0000","last_became_active":"2024-04-25T11:14:21.379835+0000","last_became_peered":"2024-04-25T11:14:21.379835+0000","last_unstale":"2024-04-25T11:14:22.362009+0000","last_undegraded":"2024-04-25T11:14:22.362009+0000","last_fullsized":"2024-04-25T11:14:22.362009+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:25.039608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350495+0000","last_change":"2024-04-25T11:14:20.344908+0000","last_active":"2024-04-25T11:14:20.350495+0000","last_peered":"2024-04-25T11:14:20.350495+0000","last_clean":"2024-04-25T11:14:20.350495+0000","last_became_active":"2024-04-25T11:14:20.344554+0000","last_became_peered":"2024-04-25T11:14:20.344554+0000","last_unstale":"2024-04-25T11:14:20.350495+0000","last_undegraded":"2024-04-25T11:14:20.350495+0000","last_fullsized":"2024-04-25T11:14:20.350495+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:55.261892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044489+0000","last_change":"2024-04-25T11:14:21.352169+0000","last_active":"2024-04-25T11:14:24.044489+0000","last_peered":"2024-04-25T11:14:24.044489+0000","last_clean":"2024-04-25T11:14:24.044489+0000","last_became_active":"2024-04-25T11:14:21.351353+0000","last_became_peered":"2024-04-25T11:14:21.351353+0000","last_unstale":"2024-04-25T11:14:24.044489+0000","last_undegraded":"2024-04-25T11:14:24.044489+0000","last_fullsized":"2024-04-25T11:14:24.044489+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:40.169340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.342416+0000","last_change":"2024-04-25T11:14:20.330643+0000","last_active":"2024-04-25T11:14:20.342416+0000","last_peered":"2024-04-25T11:14:20.342416+0000","last_clean":"2024-04-25T11:14:20.342416+0000","last_became_active":"2024-04-25T11:14:20.330199+0000","last_became_peered":"2024-04-25T11:14:20.330199+0000","last_unstale":"2024-04-25T11:14:20.342416+0000","last_undegraded":"2024-04-25T11:14:20.342416+0000","last_fullsized":"2024-04-25T11:14:20.342416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:03.000623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362035+0000","last_change":"2024-04-25T11:14:21.358807+0000","last_active":"2024-04-25T11:14:22.362035+0000","last_peered":"2024-04-25T11:14:22.362035+0000","last_clean":"2024-04-25T11:14:22.362035+0000","last_became_active":"2024-04-25T11:14:21.358476+0000","last_became_peered":"2024-04-25T11:14:21.358476+0000","last_unstale":"2024-04-25T11:14:22.362035+0000","last_undegraded":"2024-04-25T11:14:22.362035+0000","last_fullsized":"2024-04-25T11:14:22.362035+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.363496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.351029+0000","last_change":"2024-04-25T11:14:20.344529+0000","last_active":"2024-04-25T11:14:20.351029+0000","last_peered":"2024-04-25T11:14:20.351029+0000","last_clean":"2024-04-25T11:14:20.351029+0000","last_became_active":"2024-04-25T11:14:20.344184+0000","last_became_peered":"2024-04-25T11:14:20.344184+0000","last_unstale":"2024-04-25T11:14:20.351029+0000","last_undegraded":"2024-04-25T11:14:20.351029+0000","last_fullsized":"2024-04-25T11:14:20.351029+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:34.633280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.335208+0000","last_change":"2024-04-25T11:14:20.335208+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.335208+0000","last_undegraded":"2024-04-25T11:14:20.335208+0000","last_fullsized":"2024-04-25T11:14:20.335208+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344972+0000","last_change":"2024-04-25T11:14:20.337503+0000","last_active":"2024-04-25T11:14:20.344972+0000","last_peered":"2024-04-25T11:14:20.344972+0000","last_clean":"2024-04-25T11:14:20.344972+0000","last_became_active":"2024-04-25T11:14:20.337171+0000","last_became_peered":"2024-04-25T11:14:20.337171+0000","last_unstale":"2024-04-25T11:14:20.344972+0000","last_undegraded":"2024-04-25T11:14:20.344972+0000","last_fullsized":"2024-04-25T11:14:20.344972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:08.289759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321558+0000","last_change":"2024-04-25T11:14:20.321558+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321558+0000","last_undegraded":"2024-04-25T11:14:20.321558+0000","last_fullsized":"2024-04-25T11:14:20.321558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"67'6","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044599+0000","last_change":"2024-04-25T11:14:20.343544+0000","last_active":"2024-04-25T11:14:24.044599+0000","last_peered":"2024-04-25T11:14:24.044599+0000","last_clean":"2024-04-25T11:14:24.044599+0000","last_became_active":"2024-04-25T11:14:20.343205+0000","last_became_peered":"2024-04-25T11:14:20.343205+0000","last_unstale":"2024-04-25T11:14:24.044599+0000","last_undegraded":"2024-04-25T11:14:24.044599+0000","last_fullsized":"2024-04-25T11:14:24.044599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:03.224076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.316535+0000","last_change":"2024-04-25T11:14:20.316535+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.316535+0000","last_undegraded":"2024-04-25T11:14:20.316535+0000","last_fullsized":"2024-04-25T11:14:20.316535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.346424+0000","last_change":"2024-04-25T11:14:20.339359+0000","last_active":"2024-04-25T11:14:20.346424+0000","last_peered":"2024-04-25T11:14:20.346424+0000","last_clean":"2024-04-25T11:14:20.346424+0000","last_became_active":"2024-04-25T11:14:20.338755+0000","last_became_peered":"2024-04-25T11:14:20.338755+0000","last_unstale":"2024-04-25T11:14:20.346424+0000","last_undegraded":"2024-04-25T11:14:20.346424+0000","last_fullsized":"2024-04-25T11:14:20.346424+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:22.063307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.324363+0000","last_change":"2024-04-25T11:14:20.324363+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.324363+0000","last_undegraded":"2024-04-25T11:14:20.324363+0000","last_fullsized":"2024-04-25T11:14:20.324363+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.342888+0000","last_change":"2024-04-25T11:14:20.333436+0000","last_active":"2024-04-25T11:14:20.342888+0000","last_peered":"2024-04-25T11:14:20.342888+0000","last_clean":"2024-04-25T11:14:20.342888+0000","last_became_active":"2024-04-25T11:14:20.333048+0000","last_became_peered":"2024-04-25T11:14:20.333048+0000","last_unstale":"2024-04-25T11:14:20.342888+0000","last_undegraded":"2024-04-25T11:14:20.342888+0000","last_fullsized":"2024-04-25T11:14:20.342888+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:15.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.324999+0000","last_change":"2024-04-25T11:14:20.324999+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.324999+0000","last_undegraded":"2024-04-25T11:14:20.324999+0000","last_fullsized":"2024-04-25T11:14:20.324999+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344030+0000","last_change":"2024-04-25T11:14:20.331830+0000","last_active":"2024-04-25T11:14:20.344030+0000","last_peered":"2024-04-25T11:14:20.344030+0000","last_clean":"2024-04-25T11:14:20.344030+0000","last_became_active":"2024-04-25T11:14:20.331453+0000","last_became_peered":"2024-04-25T11:14:20.331453+0000","last_unstale":"2024-04-25T11:14:20.344030+0000","last_undegraded":"2024-04-25T11:14:20.344030+0000","last_fullsized":"2024-04-25T11:14:20.344030+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:52.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044411+0000","last_change":"2024-04-25T11:14:21.365796+0000","last_active":"2024-04-25T11:14:24.044411+0000","last_peered":"2024-04-25T11:14:24.044411+0000","last_clean":"2024-04-25T11:14:24.044411+0000","last_became_active":"2024-04-25T11:14:21.364748+0000","last_became_peered":"2024-04-25T11:14:21.364748+0000","last_unstale":"2024-04-25T11:14:24.044411+0000","last_undegraded":"2024-04-25T11:14:24.044411+0000","last_fullsized":"2024-04-25T11:14:24.044411+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:20.532234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.342714+0000","last_change":"2024-04-25T11:14:20.339581+0000","last_active":"2024-04-25T11:14:20.342714+0000","last_peered":"2024-04-25T11:14:20.342714+0000","last_clean":"2024-04-25T11:14:20.342714+0000","last_became_active":"2024-04-25T11:14:20.339208+0000","last_became_peered":"2024-04-25T11:14:20.339208+0000","last_unstale":"2024-04-25T11:14:20.342714+0000","last_undegraded":"2024-04-25T11:14:20.342714+0000","last_fullsized":"2024-04-25T11:14:20.342714+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:30.250638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.306095+0000","last_change":"2024-04-25T11:14:20.306095+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.306095+0000","last_undegraded":"2024-04-25T11:14:20.306095+0000","last_fullsized":"2024-04-25T11:14:20.306095+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350366+0000","last_change":"2024-04-25T11:14:20.344787+0000","last_active":"2024-04-25T11:14:20.350366+0000","last_peered":"2024-04-25T11:14:20.350366+0000","last_clean":"2024-04-25T11:14:20.350366+0000","last_became_active":"2024-04-25T11:14:20.343403+0000","last_became_peered":"2024-04-25T11:14:20.343403+0000","last_unstale":"2024-04-25T11:14:20.350366+0000","last_undegraded":"2024-04-25T11:14:20.350366+0000","last_fullsized":"2024-04-25T11:14:20.350366+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:52.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.324658+0000","last_change":"2024-04-25T11:14:20.324658+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.324658+0000","last_undegraded":"2024-04-25T11:14:20.324658+0000","last_fullsized":"2024-04-25T11:14:20.324658+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350521+0000","last_change":"2024-04-25T11:14:20.342160+0000","last_active":"2024-04-25T11:14:20.350521+0000","last_peered":"2024-04-25T11:14:20.350521+0000","last_clean":"2024-04-25T11:14:20.350521+0000","last_became_active":"2024-04-25T11:14:20.341788+0000","last_became_peered":"2024-04-25T11:14:20.341788+0000","last_unstale":"2024-04-25T11:14:20.350521+0000","last_undegraded":"2024-04-25T11:14:20.350521+0000","last_fullsized":"2024-04-25T11:14:20.350521+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:32.205173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.322564+0000","last_change":"2024-04-25T11:14:20.322564+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.322564+0000","last_undegraded":"2024-04-25T11:14:20.322564+0000","last_fullsized":"2024-04-25T11:14:20.322564+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.346096+0000","last_change":"2024-04-25T11:14:20.341298+0000","last_active":"2024-04-25T11:14:20.346096+0000","last_peered":"2024-04-25T11:14:20.346096+0000","last_clean":"2024-04-25T11:14:20.346096+0000","last_became_active":"2024-04-25T11:14:20.340803+0000","last_became_peered":"2024-04-25T11:14:20.340803+0000","last_unstale":"2024-04-25T11:14:20.346096+0000","last_undegraded":"2024-04-25T11:14:20.346096+0000","last_fullsized":"2024-04-25T11:14:20.346096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:21.473511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.327161+0000","last_change":"2024-04-25T11:14:20.327161+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.327161+0000","last_undegraded":"2024-04-25T11:14:20.327161+0000","last_fullsized":"2024-04-25T11:14:20.327161+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362338+0000","last_change":"2024-04-25T11:14:20.341606+0000","last_active":"2024-04-25T11:14:22.362338+0000","last_peered":"2024-04-25T11:14:22.362338+0000","last_clean":"2024-04-25T11:14:22.362338+0000","last_became_active":"2024-04-25T11:14:20.341220+0000","last_became_peered":"2024-04-25T11:14:20.341220+0000","last_unstale":"2024-04-25T11:14:22.362338+0000","last_undegraded":"2024-04-25T11:14:22.362338+0000","last_fullsized":"2024-04-25T11:14:22.362338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:13.222820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961827+0000","last_change":"2024-04-25T11:14:21.359973+0000","last_active":"2024-04-25T11:14:22.961827+0000","last_peered":"2024-04-25T11:14:22.961827+0000","last_clean":"2024-04-25T11:14:22.961827+0000","last_became_active":"2024-04-25T11:14:21.359599+0000","last_became_peered":"2024-04-25T11:14:21.359599+0000","last_unstale":"2024-04-25T11:14:22.961827+0000","last_undegraded":"2024-04-25T11:14:22.961827+0000","last_fullsized":"2024-04-25T11:14:22.961827+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:42.952154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961999+0000","last_change":"2024-04-25T11:14:20.341545+0000","last_active":"2024-04-25T11:14:22.961999+0000","last_peered":"2024-04-25T11:14:22.961999+0000","last_clean":"2024-04-25T11:14:22.961999+0000","last_became_active":"2024-04-25T11:14:20.341206+0000","last_became_peered":"2024-04-25T11:14:20.341206+0000","last_unstale":"2024-04-25T11:14:22.961999+0000","last_undegraded":"2024-04-25T11:14:22.961999+0000","last_fullsized":"2024-04-25T11:14:22.961999+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:13.222820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321204+0000","last_change":"2024-04-25T11:14:20.321204+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321204+0000","last_undegraded":"2024-04-25T11:14:20.321204+0000","last_fullsized":"2024-04-25T11:14:20.321204+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.343502+0000","last_change":"2024-04-25T11:14:20.341156+0000","last_active":"2024-04-25T11:14:20.343502+0000","last_peered":"2024-04-25T11:14:20.343502+0000","last_clean":"2024-04-25T11:14:20.343502+0000","last_became_active":"2024-04-25T11:14:20.340815+0000","last_became_peered":"2024-04-25T11:14:20.340815+0000","last_unstale":"2024-04-25T11:14:20.343502+0000","last_undegraded":"2024-04-25T11:14:20.343502+0000","last_fullsized":"2024-04-25T11:14:20.343502+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:03.000623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321752+0000","last_change":"2024-04-25T11:14:20.321752+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321752+0000","last_undegraded":"2024-04-25T11:14:20.321752+0000","last_fullsized":"2024-04-25T11:14:20.321752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961780+0000","last_change":"2024-04-25T11:14:20.345177+0000","last_active":"2024-04-25T11:14:22.961780+0000","last_peered":"2024-04-25T11:14:22.961780+0000","last_clean":"2024-04-25T11:14:22.961780+0000","last_became_active":"2024-04-25T11:14:20.344632+0000","last_became_peered":"2024-04-25T11:14:20.344632+0000","last_unstale":"2024-04-25T11:14:22.961780+0000","last_undegraded":"2024-04-25T11:14:22.961780+0000","last_fullsized":"2024-04-25T11:14:22.961780+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:58.724816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962522+0000","last_change":"2024-04-25T11:14:21.380957+0000","last_active":"2024-04-25T11:14:22.962522+0000","last_peered":"2024-04-25T11:14:22.962522+0000","last_clean":"2024-04-25T11:14:22.962522+0000","last_became_active":"2024-04-25T11:14:21.380702+0000","last_became_peered":"2024-04-25T11:14:21.380702+0000","last_unstale":"2024-04-25T11:14:22.962522+0000","last_undegraded":"2024-04-25T11:14:22.962522+0000","last_fullsized":"2024-04-25T11:14:22.962522+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:25.039608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350343+0000","last_change":"2024-04-25T11:14:20.343699+0000","last_active":"2024-04-25T11:14:20.350343+0000","last_peered":"2024-04-25T11:14:20.350343+0000","last_clean":"2024-04-25T11:14:20.350343+0000","last_became_active":"2024-04-25T11:14:20.343351+0000","last_became_peered":"2024-04-25T11:14:20.343351+0000","last_unstale":"2024-04-25T11:14:20.350343+0000","last_undegraded":"2024-04-25T11:14:20.350343+0000","last_fullsized":"2024-04-25T11:14:20.350343+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:55.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.347275+0000","last_change":"2024-04-25T11:14:21.355386+0000","last_active":"2024-04-25T11:14:23.347275+0000","last_peered":"2024-04-25T11:14:23.347275+0000","last_clean":"2024-04-25T11:14:23.347275+0000","last_became_active":"2024-04-25T11:14:21.355092+0000","last_became_peered":"2024-04-25T11:14:21.355092+0000","last_unstale":"2024-04-25T11:14:23.347275+0000","last_undegraded":"2024-04-25T11:14:23.347275+0000","last_fullsized":"2024-04-25T11:14:23.347275+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:55.946442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344329+0000","last_change":"2024-04-25T11:14:20.336075+0000","last_active":"2024-04-25T11:14:20.344329+0000","last_peered":"2024-04-25T11:14:20.344329+0000","last_clean":"2024-04-25T11:14:20.344329+0000","last_became_active":"2024-04-25T11:14:20.335712+0000","last_became_peered":"2024-04-25T11:14:20.335712+0000","last_unstale":"2024-04-25T11:14:20.344329+0000","last_undegraded":"2024-04-25T11:14:20.344329+0000","last_fullsized":"2024-04-25T11:14:20.344329+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:03.224076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321323+0000","last_change":"2024-04-25T11:14:20.321323+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321323+0000","last_undegraded":"2024-04-25T11:14:20.321323+0000","last_fullsized":"2024-04-25T11:14:20.321323+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962444+0000","last_change":"2024-04-25T11:14:20.346848+0000","last_active":"2024-04-25T11:14:22.962444+0000","last_peered":"2024-04-25T11:14:22.962444+0000","last_clean":"2024-04-25T11:14:22.962444+0000","last_became_active":"2024-04-25T11:14:20.346439+0000","last_became_peered":"2024-04-25T11:14:20.346439+0000","last_unstale":"2024-04-25T11:14:22.962444+0000","last_undegraded":"2024-04-25T11:14:22.962444+0000","last_fullsized":"2024-04-25T11:14:22.962444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.347332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.326463+0000","last_change":"2024-04-25T11:14:20.326463+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.326463+0000","last_undegraded":"2024-04-25T11:14:20.326463+0000","last_fullsized":"2024-04-25T11:14:20.326463+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044726+0000","last_change":"2024-04-25T11:14:20.344137+0000","last_active":"2024-04-25T11:14:24.044726+0000","last_peered":"2024-04-25T11:14:24.044726+0000","last_clean":"2024-04-25T11:14:24.044726+0000","last_became_active":"2024-04-25T11:14:20.343813+0000","last_became_peered":"2024-04-25T11:14:20.343813+0000","last_unstale":"2024-04-25T11:14:24.044726+0000","last_undegraded":"2024-04-25T11:14:24.044726+0000","last_fullsized":"2024-04-25T11:14:24.044726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:03.000623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.326037+0000","last_change":"2024-04-25T11:14:20.326037+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.326037+0000","last_undegraded":"2024-04-25T11:14:20.326037+0000","last_fullsized":"2024-04-25T11:14:20.326037+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344257+0000","last_change":"2024-04-25T11:14:20.331243+0000","last_active":"2024-04-25T11:14:20.344257+0000","last_peered":"2024-04-25T11:14:20.344257+0000","last_clean":"2024-04-25T11:14:20.344257+0000","last_became_active":"2024-04-25T11:14:20.330860+0000","last_became_peered":"2024-04-25T11:14:20.330860+0000","last_unstale":"2024-04-25T11:14:20.344257+0000","last_undegraded":"2024-04-25T11:14:20.344257+0000","last_fullsized":"2024-04-25T11:14:20.344257+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:55.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.317213+0000","last_change":"2024-04-25T11:14:20.317213+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.317213+0000","last_undegraded":"2024-04-25T11:14:20.317213+0000","last_fullsized":"2024-04-25T11:14:20.317213+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362445+0000","last_change":"2024-04-25T11:14:20.325726+0000","last_active":"2024-04-25T11:14:22.362445+0000","last_peered":"2024-04-25T11:14:22.362445+0000","last_clean":"2024-04-25T11:14:22.362445+0000","last_became_active":"2024-04-25T11:14:20.325199+0000","last_became_peered":"2024-04-25T11:14:20.325199+0000","last_unstale":"2024-04-25T11:14:22.362445+0000","last_undegraded":"2024-04-25T11:14:22.362445+0000","last_fullsized":"2024-04-25T11:14:22.362445+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:22.963995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044144+0000","last_change":"2024-04-25T11:14:21.377172+0000","last_active":"2024-04-25T11:14:24.044144+0000","last_peered":"2024-04-25T11:14:24.044144+0000","last_clean":"2024-04-25T11:14:24.044144+0000","last_became_active":"2024-04-25T11:14:21.376648+0000","last_became_peered":"2024-04-25T11:14:21.376648+0000","last_unstale":"2024-04-25T11:14:24.044144+0000","last_undegraded":"2024-04-25T11:14:24.044144+0000","last_fullsized":"2024-04-25T11:14:24.044144+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:49.106485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361947+0000","last_change":"2024-04-25T11:14:20.339034+0000","last_active":"2024-04-25T11:14:22.361947+0000","last_peered":"2024-04-25T11:14:22.361947+0000","last_clean":"2024-04-25T11:14:22.361947+0000","last_became_active":"2024-04-25T11:14:20.338713+0000","last_became_peered":"2024-04-25T11:14:20.338713+0000","last_unstale":"2024-04-25T11:14:22.361947+0000","last_undegraded":"2024-04-25T11:14:22.361947+0000","last_fullsized":"2024-04-25T11:14:22.361947+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:06.914535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346525+0000","last_change":"2024-04-25T11:14:21.373882+0000","last_active":"2024-04-25T11:14:23.346525+0000","last_peered":"2024-04-25T11:14:23.346525+0000","last_clean":"2024-04-25T11:14:23.346525+0000","last_became_active":"2024-04-25T11:14:21.373533+0000","last_became_peered":"2024-04-25T11:14:21.373533+0000","last_unstale":"2024-04-25T11:14:23.346525+0000","last_undegraded":"2024-04-25T11:14:23.346525+0000","last_fullsized":"2024-04-25T11:14:23.346525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:01.628406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.043503+0000","last_change":"2024-04-25T11:14:20.339826+0000","last_active":"2024-04-25T11:14:24.043503+0000","last_peered":"2024-04-25T11:14:24.043503+0000","last_clean":"2024-04-25T11:14:24.043503+0000","last_became_active":"2024-04-25T11:14:20.339366+0000","last_became_peered":"2024-04-25T11:14:20.339366+0000","last_unstale":"2024-04-25T11:14:24.043503+0000","last_undegraded":"2024-04-25T11:14:24.043503+0000","last_fullsized":"2024-04-25T11:14:24.043503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:15.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.326984+0000","last_change":"2024-04-25T11:14:20.326984+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.326984+0000","last_undegraded":"2024-04-25T11:14:20.326984+0000","last_fullsized":"2024-04-25T11:14:20.326984+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361619+0000","last_change":"2024-04-25T11:14:20.334793+0000","last_active":"2024-04-25T11:14:22.361619+0000","last_peered":"2024-04-25T11:14:22.361619+0000","last_clean":"2024-04-25T11:14:22.361619+0000","last_became_active":"2024-04-25T11:14:20.333198+0000","last_became_peered":"2024-04-25T11:14:20.333198+0000","last_unstale":"2024-04-25T11:14:22.361619+0000","last_undegraded":"2024-04-25T11:14:22.361619+0000","last_fullsized":"2024-04-25T11:14:22.361619+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.559248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":199,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.310353+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:14:20.310353+0000","last_peered":"2024-04-25T11:14:20.310353+0000","last_clean":"2024-04-25T11:14:20.310353+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:14:20.310353+0000","last_undegraded":"2024-04-25T11:14:20.310353+0000","last_fullsized":"2024-04-25T11:14:20.310353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476788,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1380352,"data_stored":942919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476789,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":786432,"data_stored":352551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":774144,"data_stored":345892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476789,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":352551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":366410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":368786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1404928,"data_stored":956778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":366410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:14:26.118 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:14:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.448+0000 7f3976400640 1 Processor -- start 2024-04-25T11:14:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.449+0000 7f3976400640 1 -- start start 2024-04-25T11:14:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.451+0000 7f3976400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39700b5f10 0x7f397016f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.451+0000 7f3976400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39700b47d0 con 0x7f3970164380 2024-04-25T11:14:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.451+0000 7f3976400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39700b4a40 con 0x7f397006ffd0 2024-04-25T11:14:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.451+0000 7f3976400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f39700b6320 con 0x7f39700b5f10 2024-04-25T11:14:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.452+0000 7f396fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3970164380 0x7f39700b5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52942/0 (socket says 172.21.15.7:52942) 2024-04-25T11:14:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.452+0000 7f396fe00640 1 -- 172.21.15.7:0/2095148006 learned_addr learned my addr 172.21.15.7:0/2095148006 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:26.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.452+0000 7f3975a00640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39700b5f10 0x7f397016f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:26.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.452+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 996250889 0 0) 0x7f39700b47d0 con 0x7f3970164380 2024-04-25T11:14:26.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.452+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3958003520 con 0x7f3970164380 2024-04-25T11:14:26.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2088035006 0 0) 0x7f3958003520 con 0x7f3970164380 2024-04-25T11:14:26.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39700b5f10 msgr2=0x7f397016f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:26.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39700b5f10 0x7f397016f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 >> v1:172.21.15.87:6789/0 conn(0x7f397006ffd0 legacy=0x7f39700af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39700b4d20 con 0x7f3970164380 2024-04-25T11:14:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f395c001670 con 0x7f3970164380 2024-04-25T11:14:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 >> v1:172.21.15.7:6789/0 conn(0x7f3970164380 legacy=0x7f39700b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39580039e0 0x7f3958003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3958004800 0x7f3958004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39700b5f10 0x7f395800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:26.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39580054b0 con 0x7f39580039e0 2024-04-25T11:14:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3958005660 con 0x7f3958004800 2024-04-25T11:14:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f395800bf50 con 0x7f39700b5f10 2024-04-25T11:14:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.453+0000 7f3975000640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39580039e0 0x7f3958003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.454+0000 7f3975a00640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39700b5f10 0x7f395800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.454+0000 7f396fe00640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3958004800 0x7f3958004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.454+0000 7f3975000640 1 -- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39700b5f10 msgr2=0x7f395800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.454+0000 7f3975000640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39700b5f10 0x7f395800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.454+0000 7f3975000640 1 -- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3958004800 msgr2=0x7f3958004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.454+0000 7f3975000640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3958004800 0x7f3958004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.454+0000 7f3975000640 1 -- 172.21.15.7:0/2095148006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3958003520 con 0x7f39580039e0 2024-04-25T11:14:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.454+0000 7f396fe00640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3958004800 0x7f3958004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.455+0000 7f3975000640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39580039e0 0x7f3958003e50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f39700cda30 tx=0x7f3958005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45336e19525f76e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.455+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964001370 con 0x7f39580039e0 2024-04-25T11:14:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.455+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39640052d0 con 0x7f39580039e0 2024-04-25T11:14:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.455+0000 7f396f400640 1 -- 172.21.15.7:0/2095148006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396402ab90 con 0x7f39580039e0 2024-04-25T11:14:26.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.456+0000 7f3976400640 1 -- 172.21.15.7:0/2095148006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39580039e0 msgr2=0x7f3958003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:26.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.456+0000 7f3976400640 1 --2- 172.21.15.7:0/2095148006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39580039e0 0x7f3958003e50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f39700cda30 tx=0x7f3958005830 comp rx=0 tx=0).stop 2024-04-25T11:14:26.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.456+0000 7f3975a00640 1 -- 172.21.15.7:0/2095148006 reap_dead start 2024-04-25T11:14:26.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.456+0000 7f3976400640 1 -- 172.21.15.7:0/2095148006 shutdown_connections 2024-04-25T11:14:26.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.456+0000 7f3976400640 1 -- 172.21.15.7:0/2095148006 >> 172.21.15.7:0/2095148006 conn(0x7f39700b3820 msgr2=0x7f39700b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:26.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.456+0000 7f3976400640 1 -- 172.21.15.7:0/2095148006 shutdown_connections 2024-04-25T11:14:26.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.456+0000 7f3976400640 1 -- 172.21.15.7:0/2095148006 wait complete. 2024-04-25T11:14:26.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.459+0000 7f3976400640 1 Processor -- start 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.459+0000 7f3976400640 1 -- start start 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.460+0000 7f3976400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39580039e0 0x7f3970164960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.460+0000 7f3976400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397014b7f0 0x7f39701663f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.460+0000 7f3976400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3970166930 0x7f397015f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.460+0000 7f3976400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3970164790 con 0x7f397014b7f0 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.460+0000 7f3976400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f39700b52e0 con 0x7f3970166930 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.460+0000 7f3976400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f39700afaa0 con 0x7f39580039e0 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.461+0000 7f396fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397014b7f0 0x7f39701663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.461+0000 7f396fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397014b7f0 0x7f39701663f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35436/0 (socket says 172.21.15.7:35436) 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.461+0000 7f396fe00640 1 -- 172.21.15.7:0/2999743208 learned_addr learned my addr 172.21.15.7:0/2999743208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.461+0000 7f3975000640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39580039e0 0x7f3970164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.461+0000 7f396fe00640 1 -- 172.21.15.7:0/2999743208 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39580039e0 msgr2=0x7f3970164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:26.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.461+0000 7f396fe00640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39580039e0 0x7f3970164960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.461+0000 7f396fe00640 1 -- 172.21.15.7:0/2999743208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3970166930 msgr2=0x7f397015f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.461+0000 7f3975a00640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3970166930 0x7f397015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.461+0000 7f396fe00640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3970166930 0x7f397015f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.461+0000 7f396fe00640 1 -- 172.21.15.7:0/2999743208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3970164ea0 con 0x7f397014b7f0 2024-04-25T11:14:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.462+0000 7f396fe00640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397014b7f0 0x7f39701663f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f395c00d7a0 tx=0x7f395c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.462+0000 7f396cc00640 1 -- 172.21.15.7:0/2999743208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c019070 con 0x7f397014b7f0 2024-04-25T11:14:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.462+0000 7f396cc00640 1 -- 172.21.15.7:0/2999743208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395c00de50 con 0x7f397014b7f0 2024-04-25T11:14:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.462+0000 7f3976400640 1 -- 172.21.15.7:0/2999743208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3970166d40 con 0x7f397014b7f0 2024-04-25T11:14:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.462+0000 7f396cc00640 1 -- 172.21.15.7:0/2999743208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c0047e0 con 0x7f397014b7f0 2024-04-25T11:14:26.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.462+0000 7f3976400640 1 -- 172.21.15.7:0/2999743208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397015ae70 con 0x7f397014b7f0 2024-04-25T11:14:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.463+0000 7f3976400640 1 -- 172.21.15.7:0/2999743208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3970050740 con 0x7f397014b7f0 2024-04-25T11:14:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.464+0000 7f396cc00640 1 -- 172.21.15.7:0/2999743208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f395c01c070 con 0x7f397014b7f0 2024-04-25T11:14:26.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.465+0000 7f396cc00640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39500744e0 0x7f39500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.466+0000 7f396cc00640 1 -- 172.21.15.7:0/2999743208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f395c094ec0 con 0x7f397014b7f0 2024-04-25T11:14:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.466+0000 7f3975000640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39500744e0 0x7f39500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:26.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.467+0000 7f3975000640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39500744e0 0x7f39500769f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3964032000 tx=0x7f396402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:26.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.471+0000 7f396cc00640 1 -- 172.21.15.7:0/2999743208 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3970166d40 con 0x7f397014b7f0 2024-04-25T11:14:26.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.676+0000 7f3976400640 1 -- 172.21.15.7:0/2999743208 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f397006c8f0 con 0x7f39500744e0 2024-04-25T11:14:26.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.687+0000 7f396cc00640 1 -- 172.21.15.7:0/2999743208 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248630 (secure 0 0 0) 0x7f396407eb50 con 0x7f39500744e0 2024-04-25T11:14:26.691 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:26.692 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:14:26.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.693+0000 7f3976400640 1 -- 172.21.15.7:0/2999743208 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39500744e0 msgr2=0x7f39500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:26.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.693+0000 7f3976400640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39500744e0 0x7f39500769f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3964032000 tx=0x7f396402e000 comp rx=0 tx=0).stop 2024-04-25T11:14:26.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.693+0000 7f3976400640 1 -- 172.21.15.7:0/2999743208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397014b7f0 msgr2=0x7f39701663f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:26.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.693+0000 7f3976400640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397014b7f0 0x7f39701663f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f395c00d7a0 tx=0x7f395c00dc60 comp rx=0 tx=0).stop 2024-04-25T11:14:26.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.693+0000 7f3976400640 1 -- 172.21.15.7:0/2999743208 shutdown_connections 2024-04-25T11:14:26.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.694+0000 7f3976400640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f39500744e0 0x7f39500769f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.694+0000 7f3976400640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3970166930 0x7f397015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.694+0000 7f3976400640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f397014b7f0 0x7f39701663f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.694+0000 7f3976400640 1 --2- 172.21.15.7:0/2999743208 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39580039e0 0x7f3970164960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:26.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.694+0000 7f3976400640 1 -- 172.21.15.7:0/2999743208 >> 172.21.15.7:0/2999743208 conn(0x7f39700b3820 msgr2=0x7f397006ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:26.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.694+0000 7f3976400640 1 -- 172.21.15.7:0/2999743208 shutdown_connections 2024-04-25T11:14:26.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:26.694+0000 7f3976400640 1 -- 172.21.15.7:0/2999743208 wait complete. 2024-04-25T11:14:26.722 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-04-25T11:14:25.875776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592562,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":118,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248556,"kb_used_data":7416,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726164,"statfs":{"total":773094113280,"available":772839591936,"internally_reserved":0,"allocated":7593984,"data_stored":4052297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001944"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045327+0000","last_change":"2024-04-25T11:14:21.374239+0000","last_active":"2024-04-25T11:14:24.045327+0000","last_peered":"2024-04-25T11:14:24.045327+0000","last_clean":"2024-04-25T11:14:24.045327+0000","last_became_active":"2024-04-25T11:14:21.373912+0000","last_became_peered":"2024-04-25T11:14:21.373912+0000","last_unstale":"2024-04-25T11:14:24.045327+0000","last_undegraded":"2024-04-25T11:14:24.045327+0000","last_fullsized":"2024-04-25T11:14:24.045327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:02.752329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045090+0000","last_change":"2024-04-25T11:14:21.358156+0000","last_active":"2024-04-25T11:14:24.045090+0000","last_peered":"2024-04-25T11:14:24.045090+0000","last_clean":"2024-04-25T11:14:24.045090+0000","last_became_active":"2024-04-25T11:14:21.357825+0000","last_became_peered":"2024-04-25T11:14:21.357825+0000","last_unstale":"2024-04-25T11:14:24.045090+0000","last_undegraded":"2024-04-25T11:14:24.045090+0000","last_fullsized":"2024-04-25T11:14:24.045090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.581113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.306713+0000","last_change":"2024-04-25T11:14:20.306713+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.306713+0000","last_undegraded":"2024-04-25T11:14:20.306713+0000","last_fullsized":"2024-04-25T11:14:20.306713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.323487+0000","last_change":"2024-04-25T11:14:20.323487+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.323487+0000","last_undegraded":"2024-04-25T11:14:20.323487+0000","last_fullsized":"2024-04-25T11:14:20.323487+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.334553+0000","last_change":"2024-04-25T11:14:20.334553+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.334553+0000","last_undegraded":"2024-04-25T11:14:20.334553+0000","last_fullsized":"2024-04-25T11:14:20.334553+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962720+0000","last_change":"2024-04-25T11:14:21.381797+0000","last_active":"2024-04-25T11:14:22.962720+0000","last_peered":"2024-04-25T11:14:22.962720+0000","last_clean":"2024-04-25T11:14:22.962720+0000","last_became_active":"2024-04-25T11:14:21.381332+0000","last_became_peered":"2024-04-25T11:14:21.381332+0000","last_unstale":"2024-04-25T11:14:22.962720+0000","last_undegraded":"2024-04-25T11:14:22.962720+0000","last_fullsized":"2024-04-25T11:14:22.962720+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.347162+0000","last_change":"2024-04-25T11:14:21.367266+0000","last_active":"2024-04-25T11:14:23.347162+0000","last_peered":"2024-04-25T11:14:23.347162+0000","last_clean":"2024-04-25T11:14:23.347162+0000","last_became_active":"2024-04-25T11:14:21.366376+0000","last_became_peered":"2024-04-25T11:14:21.366376+0000","last_unstale":"2024-04-25T11:14:23.347162+0000","last_undegraded":"2024-04-25T11:14:23.347162+0000","last_fullsized":"2024-04-25T11:14:23.347162+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:26.828975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328195+0000","last_change":"2024-04-25T11:14:20.328195+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328195+0000","last_undegraded":"2024-04-25T11:14:20.328195+0000","last_fullsized":"2024-04-25T11:14:20.328195+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.307139+0000","last_change":"2024-04-25T11:14:20.307139+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.307139+0000","last_undegraded":"2024-04-25T11:14:20.307139+0000","last_fullsized":"2024-04-25T11:14:20.307139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.307340+0000","last_change":"2024-04-25T11:14:20.307340+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.307340+0000","last_undegraded":"2024-04-25T11:14:20.307340+0000","last_fullsized":"2024-04-25T11:14:20.307340+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.320107+0000","last_change":"2024-04-25T11:14:20.320107+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.320107+0000","last_undegraded":"2024-04-25T11:14:20.320107+0000","last_fullsized":"2024-04-25T11:14:20.320107+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.327261+0000","last_change":"2024-04-25T11:14:20.327261+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.327261+0000","last_undegraded":"2024-04-25T11:14:20.327261+0000","last_fullsized":"2024-04-25T11:14:20.327261+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328318+0000","last_change":"2024-04-25T11:14:20.328318+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328318+0000","last_undegraded":"2024-04-25T11:14:20.328318+0000","last_fullsized":"2024-04-25T11:14:20.328318+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.309144+0000","last_change":"2024-04-25T11:14:20.309144+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.309144+0000","last_undegraded":"2024-04-25T11:14:20.309144+0000","last_fullsized":"2024-04-25T11:14:20.309144+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.307698+0000","last_change":"2024-04-25T11:14:20.307698+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.307698+0000","last_undegraded":"2024-04-25T11:14:20.307698+0000","last_fullsized":"2024-04-25T11:14:20.307698+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346754+0000","last_change":"2024-04-25T11:14:21.374629+0000","last_active":"2024-04-25T11:14:23.346754+0000","last_peered":"2024-04-25T11:14:23.346754+0000","last_clean":"2024-04-25T11:14:23.346754+0000","last_became_active":"2024-04-25T11:14:21.373757+0000","last_became_peered":"2024-04-25T11:14:21.373757+0000","last_unstale":"2024-04-25T11:14:23.346754+0000","last_undegraded":"2024-04-25T11:14:23.346754+0000","last_fullsized":"2024-04-25T11:14:23.346754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:46.489832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045260+0000","last_change":"2024-04-25T11:14:21.384370+0000","last_active":"2024-04-25T11:14:24.045260+0000","last_peered":"2024-04-25T11:14:24.045260+0000","last_clean":"2024-04-25T11:14:24.045260+0000","last_became_active":"2024-04-25T11:14:21.383916+0000","last_became_peered":"2024-04-25T11:14:21.383916+0000","last_unstale":"2024-04-25T11:14:24.045260+0000","last_undegraded":"2024-04-25T11:14:24.045260+0000","last_fullsized":"2024-04-25T11:14:24.045260+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.182058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.323323+0000","last_change":"2024-04-25T11:14:20.323323+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.323323+0000","last_undegraded":"2024-04-25T11:14:20.323323+0000","last_fullsized":"2024-04-25T11:14:20.323323+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.325689+0000","last_change":"2024-04-25T11:14:20.325689+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.325689+0000","last_undegraded":"2024-04-25T11:14:20.325689+0000","last_fullsized":"2024-04-25T11:14:20.325689+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362235+0000","last_change":"2024-04-25T11:14:21.370707+0000","last_active":"2024-04-25T11:14:22.362235+0000","last_peered":"2024-04-25T11:14:22.362235+0000","last_clean":"2024-04-25T11:14:22.362235+0000","last_became_active":"2024-04-25T11:14:21.370409+0000","last_became_peered":"2024-04-25T11:14:21.370409+0000","last_unstale":"2024-04-25T11:14:22.362235+0000","last_undegraded":"2024-04-25T11:14:22.362235+0000","last_fullsized":"2024-04-25T11:14:22.362235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.327665+0000","last_change":"2024-04-25T11:14:20.327665+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.327665+0000","last_undegraded":"2024-04-25T11:14:20.327665+0000","last_fullsized":"2024-04-25T11:14:20.327665+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362273+0000","last_change":"2024-04-25T11:14:21.380852+0000","last_active":"2024-04-25T11:14:22.362273+0000","last_peered":"2024-04-25T11:14:22.362273+0000","last_clean":"2024-04-25T11:14:22.362273+0000","last_became_active":"2024-04-25T11:14:21.380226+0000","last_became_peered":"2024-04-25T11:14:21.380226+0000","last_unstale":"2024-04-25T11:14:22.362273+0000","last_undegraded":"2024-04-25T11:14:22.362273+0000","last_fullsized":"2024-04-25T11:14:22.362273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:12.837877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361853+0000","last_change":"2024-04-25T11:14:21.365901+0000","last_active":"2024-04-25T11:14:22.361853+0000","last_peered":"2024-04-25T11:14:22.361853+0000","last_clean":"2024-04-25T11:14:22.361853+0000","last_became_active":"2024-04-25T11:14:21.365551+0000","last_became_peered":"2024-04-25T11:14:21.365551+0000","last_unstale":"2024-04-25T11:14:22.361853+0000","last_undegraded":"2024-04-25T11:14:22.361853+0000","last_fullsized":"2024-04-25T11:14:22.361853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:42.952154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.305880+0000","last_change":"2024-04-25T11:14:20.305880+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.305880+0000","last_undegraded":"2024-04-25T11:14:20.305880+0000","last_fullsized":"2024-04-25T11:14:20.305880+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328612+0000","last_change":"2024-04-25T11:14:20.328612+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328612+0000","last_undegraded":"2024-04-25T11:14:20.328612+0000","last_fullsized":"2024-04-25T11:14:20.328612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328574+0000","last_change":"2024-04-25T11:14:20.328574+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328574+0000","last_undegraded":"2024-04-25T11:14:20.328574+0000","last_fullsized":"2024-04-25T11:14:20.328574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.318869+0000","last_change":"2024-04-25T11:14:20.318869+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.318869+0000","last_undegraded":"2024-04-25T11:14:20.318869+0000","last_fullsized":"2024-04-25T11:14:20.318869+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044519+0000","last_change":"2024-04-25T11:14:21.371250+0000","last_active":"2024-04-25T11:14:24.044519+0000","last_peered":"2024-04-25T11:14:24.044519+0000","last_clean":"2024-04-25T11:14:24.044519+0000","last_became_active":"2024-04-25T11:14:21.370864+0000","last_became_peered":"2024-04-25T11:14:21.370864+0000","last_unstale":"2024-04-25T11:14:24.044519+0000","last_undegraded":"2024-04-25T11:14:24.044519+0000","last_fullsized":"2024-04-25T11:14:24.044519+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:09.063896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.305995+0000","last_change":"2024-04-25T11:14:20.305995+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.305995+0000","last_undegraded":"2024-04-25T11:14:20.305995+0000","last_fullsized":"2024-04-25T11:14:20.305995+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328804+0000","last_change":"2024-04-25T11:14:20.328804+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328804+0000","last_undegraded":"2024-04-25T11:14:20.328804+0000","last_fullsized":"2024-04-25T11:14:20.328804+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.318482+0000","last_change":"2024-04-25T11:14:20.318482+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.318482+0000","last_undegraded":"2024-04-25T11:14:20.318482+0000","last_fullsized":"2024-04-25T11:14:20.318482+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961814+0000","last_change":"2024-04-25T11:14:21.360674+0000","last_active":"2024-04-25T11:14:22.961814+0000","last_peered":"2024-04-25T11:14:22.961814+0000","last_clean":"2024-04-25T11:14:22.961814+0000","last_became_active":"2024-04-25T11:14:21.359863+0000","last_became_peered":"2024-04-25T11:14:21.359863+0000","last_unstale":"2024-04-25T11:14:22.961814+0000","last_undegraded":"2024-04-25T11:14:22.961814+0000","last_fullsized":"2024-04-25T11:14:22.961814+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:49.430307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.323584+0000","last_change":"2024-04-25T11:14:20.323584+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.323584+0000","last_undegraded":"2024-04-25T11:14:20.323584+0000","last_fullsized":"2024-04-25T11:14:20.323584+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"67'4","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045418+0000","last_change":"2024-04-25T11:14:20.334352+0000","last_active":"2024-04-25T11:14:24.045418+0000","last_peered":"2024-04-25T11:14:24.045418+0000","last_clean":"2024-04-25T11:14:24.045418+0000","last_became_active":"2024-04-25T11:14:20.329783+0000","last_became_peered":"2024-04-25T11:14:20.329783+0000","last_unstale":"2024-04-25T11:14:24.045418+0000","last_undegraded":"2024-04-25T11:14:24.045418+0000","last_fullsized":"2024-04-25T11:14:24.045418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:55.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.317695+0000","last_change":"2024-04-25T11:14:20.317695+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.317695+0000","last_undegraded":"2024-04-25T11:14:20.317695+0000","last_fullsized":"2024-04-25T11:14:20.317695+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962745+0000","last_change":"2024-04-25T11:14:20.346713+0000","last_active":"2024-04-25T11:14:22.962745+0000","last_peered":"2024-04-25T11:14:22.962745+0000","last_clean":"2024-04-25T11:14:22.962745+0000","last_became_active":"2024-04-25T11:14:20.346320+0000","last_became_peered":"2024-04-25T11:14:20.346320+0000","last_unstale":"2024-04-25T11:14:22.962745+0000","last_undegraded":"2024-04-25T11:14:22.962745+0000","last_fullsized":"2024-04-25T11:14:22.962745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.559248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962535+0000","last_change":"2024-04-25T11:14:21.356785+0000","last_active":"2024-04-25T11:14:22.962535+0000","last_peered":"2024-04-25T11:14:22.962535+0000","last_clean":"2024-04-25T11:14:22.962535+0000","last_became_active":"2024-04-25T11:14:21.356335+0000","last_became_peered":"2024-04-25T11:14:21.356335+0000","last_unstale":"2024-04-25T11:14:22.962535+0000","last_undegraded":"2024-04-25T11:14:22.962535+0000","last_fullsized":"2024-04-25T11:14:22.962535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.363496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962440+0000","last_change":"2024-04-25T11:14:20.346384+0000","last_active":"2024-04-25T11:14:22.962440+0000","last_peered":"2024-04-25T11:14:22.962440+0000","last_clean":"2024-04-25T11:14:22.962440+0000","last_became_active":"2024-04-25T11:14:20.346031+0000","last_became_peered":"2024-04-25T11:14:20.346031+0000","last_unstale":"2024-04-25T11:14:22.962440+0000","last_undegraded":"2024-04-25T11:14:22.962440+0000","last_fullsized":"2024-04-25T11:14:22.962440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:22.963995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.308498+0000","last_change":"2024-04-25T11:14:20.308498+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.308498+0000","last_undegraded":"2024-04-25T11:14:20.308498+0000","last_fullsized":"2024-04-25T11:14:20.308498+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962159+0000","last_change":"2024-04-25T11:14:20.337082+0000","last_active":"2024-04-25T11:14:22.962159+0000","last_peered":"2024-04-25T11:14:22.962159+0000","last_clean":"2024-04-25T11:14:22.962159+0000","last_became_active":"2024-04-25T11:14:20.336766+0000","last_became_peered":"2024-04-25T11:14:20.336766+0000","last_unstale":"2024-04-25T11:14:22.962159+0000","last_undegraded":"2024-04-25T11:14:22.962159+0000","last_fullsized":"2024-04-25T11:14:22.962159+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:12.462630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346829+0000","last_change":"2024-04-25T11:14:21.346395+0000","last_active":"2024-04-25T11:14:23.346829+0000","last_peered":"2024-04-25T11:14:23.346829+0000","last_clean":"2024-04-25T11:14:23.346829+0000","last_became_active":"2024-04-25T11:14:21.345933+0000","last_became_peered":"2024-04-25T11:14:21.345933+0000","last_unstale":"2024-04-25T11:14:23.346829+0000","last_undegraded":"2024-04-25T11:14:23.346829+0000","last_fullsized":"2024-04-25T11:14:23.346829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.002374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362073+0000","last_change":"2024-04-25T11:14:20.335533+0000","last_active":"2024-04-25T11:14:22.362073+0000","last_peered":"2024-04-25T11:14:22.362073+0000","last_clean":"2024-04-25T11:14:22.362073+0000","last_became_active":"2024-04-25T11:14:20.335228+0000","last_became_peered":"2024-04-25T11:14:20.335228+0000","last_unstale":"2024-04-25T11:14:22.362073+0000","last_undegraded":"2024-04-25T11:14:22.362073+0000","last_fullsized":"2024-04-25T11:14:22.362073+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.347332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044109+0000","last_change":"2024-04-25T11:14:21.384939+0000","last_active":"2024-04-25T11:14:24.044109+0000","last_peered":"2024-04-25T11:14:24.044109+0000","last_clean":"2024-04-25T11:14:24.044109+0000","last_became_active":"2024-04-25T11:14:21.384525+0000","last_became_peered":"2024-04-25T11:14:21.384525+0000","last_unstale":"2024-04-25T11:14:24.044109+0000","last_undegraded":"2024-04-25T11:14:24.044109+0000","last_fullsized":"2024-04-25T11:14:24.044109+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:19.066276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361973+0000","last_change":"2024-04-25T11:14:20.317044+0000","last_active":"2024-04-25T11:14:22.361973+0000","last_peered":"2024-04-25T11:14:22.361973+0000","last_clean":"2024-04-25T11:14:22.361973+0000","last_became_active":"2024-04-25T11:14:20.316634+0000","last_became_peered":"2024-04-25T11:14:20.316634+0000","last_unstale":"2024-04-25T11:14:22.361973+0000","last_undegraded":"2024-04-25T11:14:22.361973+0000","last_fullsized":"2024-04-25T11:14:22.361973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:12.462630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.325823+0000","last_change":"2024-04-25T11:14:20.325823+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.325823+0000","last_undegraded":"2024-04-25T11:14:20.325823+0000","last_fullsized":"2024-04-25T11:14:20.325823+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361960+0000","last_change":"2024-04-25T11:14:20.339374+0000","last_active":"2024-04-25T11:14:22.361960+0000","last_peered":"2024-04-25T11:14:22.361960+0000","last_clean":"2024-04-25T11:14:22.361960+0000","last_became_active":"2024-04-25T11:14:20.339026+0000","last_became_peered":"2024-04-25T11:14:20.339026+0000","last_unstale":"2024-04-25T11:14:22.361960+0000","last_undegraded":"2024-04-25T11:14:22.361960+0000","last_fullsized":"2024-04-25T11:14:22.361960+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:09.182038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362009+0000","last_change":"2024-04-25T11:14:21.380186+0000","last_active":"2024-04-25T11:14:22.362009+0000","last_peered":"2024-04-25T11:14:22.362009+0000","last_clean":"2024-04-25T11:14:22.362009+0000","last_became_active":"2024-04-25T11:14:21.379835+0000","last_became_peered":"2024-04-25T11:14:21.379835+0000","last_unstale":"2024-04-25T11:14:22.362009+0000","last_undegraded":"2024-04-25T11:14:22.362009+0000","last_fullsized":"2024-04-25T11:14:22.362009+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:25.039608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350495+0000","last_change":"2024-04-25T11:14:20.344908+0000","last_active":"2024-04-25T11:14:20.350495+0000","last_peered":"2024-04-25T11:14:20.350495+0000","last_clean":"2024-04-25T11:14:20.350495+0000","last_became_active":"2024-04-25T11:14:20.344554+0000","last_became_peered":"2024-04-25T11:14:20.344554+0000","last_unstale":"2024-04-25T11:14:20.350495+0000","last_undegraded":"2024-04-25T11:14:20.350495+0000","last_fullsized":"2024-04-25T11:14:20.350495+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:55.261892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044489+0000","last_change":"2024-04-25T11:14:21.352169+0000","last_active":"2024-04-25T11:14:24.044489+0000","last_peered":"2024-04-25T11:14:24.044489+0000","last_clean":"2024-04-25T11:14:24.044489+0000","last_became_active":"2024-04-25T11:14:21.351353+0000","last_became_peered":"2024-04-25T11:14:21.351353+0000","last_unstale":"2024-04-25T11:14:24.044489+0000","last_undegraded":"2024-04-25T11:14:24.044489+0000","last_fullsized":"2024-04-25T11:14:24.044489+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:40.169340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.342416+0000","last_change":"2024-04-25T11:14:20.330643+0000","last_active":"2024-04-25T11:14:20.342416+0000","last_peered":"2024-04-25T11:14:20.342416+0000","last_clean":"2024-04-25T11:14:20.342416+0000","last_became_active":"2024-04-25T11:14:20.330199+0000","last_became_peered":"2024-04-25T11:14:20.330199+0000","last_unstale":"2024-04-25T11:14:20.342416+0000","last_undegraded":"2024-04-25T11:14:20.342416+0000","last_fullsized":"2024-04-25T11:14:20.342416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:03.000623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362035+0000","last_change":"2024-04-25T11:14:21.358807+0000","last_active":"2024-04-25T11:14:22.362035+0000","last_peered":"2024-04-25T11:14:22.362035+0000","last_clean":"2024-04-25T11:14:22.362035+0000","last_became_active":"2024-04-25T11:14:21.358476+0000","last_became_peered":"2024-04-25T11:14:21.358476+0000","last_unstale":"2024-04-25T11:14:22.362035+0000","last_undegraded":"2024-04-25T11:14:22.362035+0000","last_fullsized":"2024-04-25T11:14:22.362035+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.363496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.351029+0000","last_change":"2024-04-25T11:14:20.344529+0000","last_active":"2024-04-25T11:14:20.351029+0000","last_peered":"2024-04-25T11:14:20.351029+0000","last_clean":"2024-04-25T11:14:20.351029+0000","last_became_active":"2024-04-25T11:14:20.344184+0000","last_became_peered":"2024-04-25T11:14:20.344184+0000","last_unstale":"2024-04-25T11:14:20.351029+0000","last_undegraded":"2024-04-25T11:14:20.351029+0000","last_fullsized":"2024-04-25T11:14:20.351029+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:34.633280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.335208+0000","last_change":"2024-04-25T11:14:20.335208+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.335208+0000","last_undegraded":"2024-04-25T11:14:20.335208+0000","last_fullsized":"2024-04-25T11:14:20.335208+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344972+0000","last_change":"2024-04-25T11:14:20.337503+0000","last_active":"2024-04-25T11:14:20.344972+0000","last_peered":"2024-04-25T11:14:20.344972+0000","last_clean":"2024-04-25T11:14:20.344972+0000","last_became_active":"2024-04-25T11:14:20.337171+0000","last_became_peered":"2024-04-25T11:14:20.337171+0000","last_unstale":"2024-04-25T11:14:20.344972+0000","last_undegraded":"2024-04-25T11:14:20.344972+0000","last_fullsized":"2024-04-25T11:14:20.344972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:08.289759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321558+0000","last_change":"2024-04-25T11:14:20.321558+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321558+0000","last_undegraded":"2024-04-25T11:14:20.321558+0000","last_fullsized":"2024-04-25T11:14:20.321558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"67'6","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044599+0000","last_change":"2024-04-25T11:14:20.343544+0000","last_active":"2024-04-25T11:14:24.044599+0000","last_peered":"2024-04-25T11:14:24.044599+0000","last_clean":"2024-04-25T11:14:24.044599+0000","last_became_active":"2024-04-25T11:14:20.343205+0000","last_became_peered":"2024-04-25T11:14:20.343205+0000","last_unstale":"2024-04-25T11:14:24.044599+0000","last_undegraded":"2024-04-25T11:14:24.044599+0000","last_fullsized":"2024-04-25T11:14:24.044599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:03.224076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.316535+0000","last_change":"2024-04-25T11:14:20.316535+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.316535+0000","last_undegraded":"2024-04-25T11:14:20.316535+0000","last_fullsized":"2024-04-25T11:14:20.316535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.346424+0000","last_change":"2024-04-25T11:14:20.339359+0000","last_active":"2024-04-25T11:14:20.346424+0000","last_peered":"2024-04-25T11:14:20.346424+0000","last_clean":"2024-04-25T11:14:20.346424+0000","last_became_active":"2024-04-25T11:14:20.338755+0000","last_became_peered":"2024-04-25T11:14:20.338755+0000","last_unstale":"2024-04-25T11:14:20.346424+0000","last_undegraded":"2024-04-25T11:14:20.346424+0000","last_fullsized":"2024-04-25T11:14:20.346424+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:22.063307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.324363+0000","last_change":"2024-04-25T11:14:20.324363+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.324363+0000","last_undegraded":"2024-04-25T11:14:20.324363+0000","last_fullsized":"2024-04-25T11:14:20.324363+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.342888+0000","last_change":"2024-04-25T11:14:20.333436+0000","last_active":"2024-04-25T11:14:20.342888+0000","last_peered":"2024-04-25T11:14:20.342888+0000","last_clean":"2024-04-25T11:14:20.342888+0000","last_became_active":"2024-04-25T11:14:20.333048+0000","last_became_peered":"2024-04-25T11:14:20.333048+0000","last_unstale":"2024-04-25T11:14:20.342888+0000","last_undegraded":"2024-04-25T11:14:20.342888+0000","last_fullsized":"2024-04-25T11:14:20.342888+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:15.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.324999+0000","last_change":"2024-04-25T11:14:20.324999+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.324999+0000","last_undegraded":"2024-04-25T11:14:20.324999+0000","last_fullsized":"2024-04-25T11:14:20.324999+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344030+0000","last_change":"2024-04-25T11:14:20.331830+0000","last_active":"2024-04-25T11:14:20.344030+0000","last_peered":"2024-04-25T11:14:20.344030+0000","last_clean":"2024-04-25T11:14:20.344030+0000","last_became_active":"2024-04-25T11:14:20.331453+0000","last_became_peered":"2024-04-25T11:14:20.331453+0000","last_unstale":"2024-04-25T11:14:20.344030+0000","last_undegraded":"2024-04-25T11:14:20.344030+0000","last_fullsized":"2024-04-25T11:14:20.344030+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:52.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044411+0000","last_change":"2024-04-25T11:14:21.365796+0000","last_active":"2024-04-25T11:14:24.044411+0000","last_peered":"2024-04-25T11:14:24.044411+0000","last_clean":"2024-04-25T11:14:24.044411+0000","last_became_active":"2024-04-25T11:14:21.364748+0000","last_became_peered":"2024-04-25T11:14:21.364748+0000","last_unstale":"2024-04-25T11:14:24.044411+0000","last_undegraded":"2024-04-25T11:14:24.044411+0000","last_fullsized":"2024-04-25T11:14:24.044411+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:20.532234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.342714+0000","last_change":"2024-04-25T11:14:20.339581+0000","last_active":"2024-04-25T11:14:20.342714+0000","last_peered":"2024-04-25T11:14:20.342714+0000","last_clean":"2024-04-25T11:14:20.342714+0000","last_became_active":"2024-04-25T11:14:20.339208+0000","last_became_peered":"2024-04-25T11:14:20.339208+0000","last_unstale":"2024-04-25T11:14:20.342714+0000","last_undegraded":"2024-04-25T11:14:20.342714+0000","last_fullsized":"2024-04-25T11:14:20.342714+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:30.250638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.306095+0000","last_change":"2024-04-25T11:14:20.306095+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.306095+0000","last_undegraded":"2024-04-25T11:14:20.306095+0000","last_fullsized":"2024-04-25T11:14:20.306095+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350366+0000","last_change":"2024-04-25T11:14:20.344787+0000","last_active":"2024-04-25T11:14:20.350366+0000","last_peered":"2024-04-25T11:14:20.350366+0000","last_clean":"2024-04-25T11:14:20.350366+0000","last_became_active":"2024-04-25T11:14:20.343403+0000","last_became_peered":"2024-04-25T11:14:20.343403+0000","last_unstale":"2024-04-25T11:14:20.350366+0000","last_undegraded":"2024-04-25T11:14:20.350366+0000","last_fullsized":"2024-04-25T11:14:20.350366+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:52.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.324658+0000","last_change":"2024-04-25T11:14:20.324658+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.324658+0000","last_undegraded":"2024-04-25T11:14:20.324658+0000","last_fullsized":"2024-04-25T11:14:20.324658+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350521+0000","last_change":"2024-04-25T11:14:20.342160+0000","last_active":"2024-04-25T11:14:20.350521+0000","last_peered":"2024-04-25T11:14:20.350521+0000","last_clean":"2024-04-25T11:14:20.350521+0000","last_became_active":"2024-04-25T11:14:20.341788+0000","last_became_peered":"2024-04-25T11:14:20.341788+0000","last_unstale":"2024-04-25T11:14:20.350521+0000","last_undegraded":"2024-04-25T11:14:20.350521+0000","last_fullsized":"2024-04-25T11:14:20.350521+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:32.205173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.322564+0000","last_change":"2024-04-25T11:14:20.322564+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.322564+0000","last_undegraded":"2024-04-25T11:14:20.322564+0000","last_fullsized":"2024-04-25T11:14:20.322564+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.346096+0000","last_change":"2024-04-25T11:14:20.341298+0000","last_active":"2024-04-25T11:14:20.346096+0000","last_peered":"2024-04-25T11:14:20.346096+0000","last_clean":"2024-04-25T11:14:20.346096+0000","last_became_active":"2024-04-25T11:14:20.340803+0000","last_became_peered":"2024-04-25T11:14:20.340803+0000","last_unstale":"2024-04-25T11:14:20.346096+0000","last_undegraded":"2024-04-25T11:14:20.346096+0000","last_fullsized":"2024-04-25T11:14:20.346096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:21.473511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.327161+0000","last_change":"2024-04-25T11:14:20.327161+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.327161+0000","last_undegraded":"2024-04-25T11:14:20.327161+0000","last_fullsized":"2024-04-25T11:14:20.327161+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362338+0000","last_change":"2024-04-25T11:14:20.341606+0000","last_active":"2024-04-25T11:14:22.362338+0000","last_peered":"2024-04-25T11:14:22.362338+0000","last_clean":"2024-04-25T11:14:22.362338+0000","last_became_active":"2024-04-25T11:14:20.341220+0000","last_became_peered":"2024-04-25T11:14:20.341220+0000","last_unstale":"2024-04-25T11:14:22.362338+0000","last_undegraded":"2024-04-25T11:14:22.362338+0000","last_fullsized":"2024-04-25T11:14:22.362338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:13.222820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961827+0000","last_change":"2024-04-25T11:14:21.359973+0000","last_active":"2024-04-25T11:14:22.961827+0000","last_peered":"2024-04-25T11:14:22.961827+0000","last_clean":"2024-04-25T11:14:22.961827+0000","last_became_active":"2024-04-25T11:14:21.359599+0000","last_became_peered":"2024-04-25T11:14:21.359599+0000","last_unstale":"2024-04-25T11:14:22.961827+0000","last_undegraded":"2024-04-25T11:14:22.961827+0000","last_fullsized":"2024-04-25T11:14:22.961827+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:42.952154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961999+0000","last_change":"2024-04-25T11:14:20.341545+0000","last_active":"2024-04-25T11:14:22.961999+0000","last_peered":"2024-04-25T11:14:22.961999+0000","last_clean":"2024-04-25T11:14:22.961999+0000","last_became_active":"2024-04-25T11:14:20.341206+0000","last_became_peered":"2024-04-25T11:14:20.341206+0000","last_unstale":"2024-04-25T11:14:22.961999+0000","last_undegraded":"2024-04-25T11:14:22.961999+0000","last_fullsized":"2024-04-25T11:14:22.961999+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:13.222820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321204+0000","last_change":"2024-04-25T11:14:20.321204+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321204+0000","last_undegraded":"2024-04-25T11:14:20.321204+0000","last_fullsized":"2024-04-25T11:14:20.321204+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.343502+0000","last_change":"2024-04-25T11:14:20.341156+0000","last_active":"2024-04-25T11:14:20.343502+0000","last_peered":"2024-04-25T11:14:20.343502+0000","last_clean":"2024-04-25T11:14:20.343502+0000","last_became_active":"2024-04-25T11:14:20.340815+0000","last_became_peered":"2024-04-25T11:14:20.340815+0000","last_unstale":"2024-04-25T11:14:20.343502+0000","last_undegraded":"2024-04-25T11:14:20.343502+0000","last_fullsized":"2024-04-25T11:14:20.343502+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:03.000623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321752+0000","last_change":"2024-04-25T11:14:20.321752+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321752+0000","last_undegraded":"2024-04-25T11:14:20.321752+0000","last_fullsized":"2024-04-25T11:14:20.321752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961780+0000","last_change":"2024-04-25T11:14:20.345177+0000","last_active":"2024-04-25T11:14:22.961780+0000","last_peered":"2024-04-25T11:14:22.961780+0000","last_clean":"2024-04-25T11:14:22.961780+0000","last_became_active":"2024-04-25T11:14:20.344632+0000","last_became_peered":"2024-04-25T11:14:20.344632+0000","last_unstale":"2024-04-25T11:14:22.961780+0000","last_undegraded":"2024-04-25T11:14:22.961780+0000","last_fullsized":"2024-04-25T11:14:22.961780+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:58.724816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962522+0000","last_change":"2024-04-25T11:14:21.380957+0000","last_active":"2024-04-25T11:14:22.962522+0000","last_peered":"2024-04-25T11:14:22.962522+0000","last_clean":"2024-04-25T11:14:22.962522+0000","last_became_active":"2024-04-25T11:14:21.380702+0000","last_became_peered":"2024-04-25T11:14:21.380702+0000","last_unstale":"2024-04-25T11:14:22.962522+0000","last_undegraded":"2024-04-25T11:14:22.962522+0000","last_fullsized":"2024-04-25T11:14:22.962522+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:25.039608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350343+0000","last_change":"2024-04-25T11:14:20.343699+0000","last_active":"2024-04-25T11:14:20.350343+0000","last_peered":"2024-04-25T11:14:20.350343+0000","last_clean":"2024-04-25T11:14:20.350343+0000","last_became_active":"2024-04-25T11:14:20.343351+0000","last_became_peered":"2024-04-25T11:14:20.343351+0000","last_unstale":"2024-04-25T11:14:20.350343+0000","last_undegraded":"2024-04-25T11:14:20.350343+0000","last_fullsized":"2024-04-25T11:14:20.350343+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:55.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.347275+0000","last_change":"2024-04-25T11:14:21.355386+0000","last_active":"2024-04-25T11:14:23.347275+0000","last_peered":"2024-04-25T11:14:23.347275+0000","last_clean":"2024-04-25T11:14:23.347275+0000","last_became_active":"2024-04-25T11:14:21.355092+0000","last_became_peered":"2024-04-25T11:14:21.355092+0000","last_unstale":"2024-04-25T11:14:23.347275+0000","last_undegraded":"2024-04-25T11:14:23.347275+0000","last_fullsized":"2024-04-25T11:14:23.347275+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:55.946442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344329+0000","last_change":"2024-04-25T11:14:20.336075+0000","last_active":"2024-04-25T11:14:20.344329+0000","last_peered":"2024-04-25T11:14:20.344329+0000","last_clean":"2024-04-25T11:14:20.344329+0000","last_became_active":"2024-04-25T11:14:20.335712+0000","last_became_peered":"2024-04-25T11:14:20.335712+0000","last_unstale":"2024-04-25T11:14:20.344329+0000","last_undegraded":"2024-04-25T11:14:20.344329+0000","last_fullsized":"2024-04-25T11:14:20.344329+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:03.224076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321323+0000","last_change":"2024-04-25T11:14:20.321323+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321323+0000","last_undegraded":"2024-04-25T11:14:20.321323+0000","last_fullsized":"2024-04-25T11:14:20.321323+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962444+0000","last_change":"2024-04-25T11:14:20.346848+0000","last_active":"2024-04-25T11:14:22.962444+0000","last_peered":"2024-04-25T11:14:22.962444+0000","last_clean":"2024-04-25T11:14:22.962444+0000","last_became_active":"2024-04-25T11:14:20.346439+0000","last_became_peered":"2024-04-25T11:14:20.346439+0000","last_unstale":"2024-04-25T11:14:22.962444+0000","last_undegraded":"2024-04-25T11:14:22.962444+0000","last_fullsized":"2024-04-25T11:14:22.962444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.347332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.326463+0000","last_change":"2024-04-25T11:14:20.326463+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.326463+0000","last_undegraded":"2024-04-25T11:14:20.326463+0000","last_fullsized":"2024-04-25T11:14:20.326463+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044726+0000","last_change":"2024-04-25T11:14:20.344137+0000","last_active":"2024-04-25T11:14:24.044726+0000","last_peered":"2024-04-25T11:14:24.044726+0000","last_clean":"2024-04-25T11:14:24.044726+0000","last_became_active":"2024-04-25T11:14:20.343813+0000","last_became_peered":"2024-04-25T11:14:20.343813+0000","last_unstale":"2024-04-25T11:14:24.044726+0000","last_undegraded":"2024-04-25T11:14:24.044726+0000","last_fullsized":"2024-04-25T11:14:24.044726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:03.000623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.326037+0000","last_change":"2024-04-25T11:14:20.326037+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.326037+0000","last_undegraded":"2024-04-25T11:14:20.326037+0000","last_fullsized":"2024-04-25T11:14:20.326037+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344257+0000","last_change":"2024-04-25T11:14:20.331243+0000","last_active":"2024-04-25T11:14:20.344257+0000","last_peered":"2024-04-25T11:14:20.344257+0000","last_clean":"2024-04-25T11:14:20.344257+0000","last_became_active":"2024-04-25T11:14:20.330860+0000","last_became_peered":"2024-04-25T11:14:20.330860+0000","last_unstale":"2024-04-25T11:14:20.344257+0000","last_undegraded":"2024-04-25T11:14:20.344257+0000","last_fullsized":"2024-04-25T11:14:20.344257+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:55.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.317213+0000","last_change":"2024-04-25T11:14:20.317213+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.317213+0000","last_undegraded":"2024-04-25T11:14:20.317213+0000","last_fullsized":"2024-04-25T11:14:20.317213+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362445+0000","last_change":"2024-04-25T11:14:20.325726+0000","last_active":"2024-04-25T11:14:22.362445+0000","last_peered":"2024-04-25T11:14:22.362445+0000","last_clean":"2024-04-25T11:14:22.362445+0000","last_became_active":"2024-04-25T11:14:20.325199+0000","last_became_peered":"2024-04-25T11:14:20.325199+0000","last_unstale":"2024-04-25T11:14:22.362445+0000","last_undegraded":"2024-04-25T11:14:22.362445+0000","last_fullsized":"2024-04-25T11:14:22.362445+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:22.963995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044144+0000","last_change":"2024-04-25T11:14:21.377172+0000","last_active":"2024-04-25T11:14:24.044144+0000","last_peered":"2024-04-25T11:14:24.044144+0000","last_clean":"2024-04-25T11:14:24.044144+0000","last_became_active":"2024-04-25T11:14:21.376648+0000","last_became_peered":"2024-04-25T11:14:21.376648+0000","last_unstale":"2024-04-25T11:14:24.044144+0000","last_undegraded":"2024-04-25T11:14:24.044144+0000","last_fullsized":"2024-04-25T11:14:24.044144+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:49.106485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361947+0000","last_change":"2024-04-25T11:14:20.339034+0000","last_active":"2024-04-25T11:14:22.361947+0000","last_peered":"2024-04-25T11:14:22.361947+0000","last_clean":"2024-04-25T11:14:22.361947+0000","last_became_active":"2024-04-25T11:14:20.338713+0000","last_became_peered":"2024-04-25T11:14:20.338713+0000","last_unstale":"2024-04-25T11:14:22.361947+0000","last_undegraded":"2024-04-25T11:14:22.361947+0000","last_fullsized":"2024-04-25T11:14:22.361947+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:06.914535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346525+0000","last_change":"2024-04-25T11:14:21.373882+0000","last_active":"2024-04-25T11:14:23.346525+0000","last_peered":"2024-04-25T11:14:23.346525+0000","last_clean":"2024-04-25T11:14:23.346525+0000","last_became_active":"2024-04-25T11:14:21.373533+0000","last_became_peered":"2024-04-25T11:14:21.373533+0000","last_unstale":"2024-04-25T11:14:23.346525+0000","last_undegraded":"2024-04-25T11:14:23.346525+0000","last_fullsized":"2024-04-25T11:14:23.346525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:01.628406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.043503+0000","last_change":"2024-04-25T11:14:20.339826+0000","last_active":"2024-04-25T11:14:24.043503+0000","last_peered":"2024-04-25T11:14:24.043503+0000","last_clean":"2024-04-25T11:14:24.043503+0000","last_became_active":"2024-04-25T11:14:20.339366+0000","last_became_peered":"2024-04-25T11:14:20.339366+0000","last_unstale":"2024-04-25T11:14:24.043503+0000","last_undegraded":"2024-04-25T11:14:24.043503+0000","last_fullsized":"2024-04-25T11:14:24.043503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:15.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.326984+0000","last_change":"2024-04-25T11:14:20.326984+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.326984+0000","last_undegraded":"2024-04-25T11:14:20.326984+0000","last_fullsized":"2024-04-25T11:14:20.326984+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361619+0000","last_change":"2024-04-25T11:14:20.334793+0000","last_active":"2024-04-25T11:14:22.361619+0000","last_peered":"2024-04-25T11:14:22.361619+0000","last_clean":"2024-04-25T11:14:22.361619+0000","last_became_active":"2024-04-25T11:14:20.333198+0000","last_became_peered":"2024-04-25T11:14:20.333198+0000","last_unstale":"2024-04-25T11:14:22.361619+0000","last_undegraded":"2024-04-25T11:14:22.361619+0000","last_fullsized":"2024-04-25T11:14:22.361619+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.559248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":199,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.310353+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:14:20.310353+0000","last_peered":"2024-04-25T11:14:20.310353+0000","last_clean":"2024-04-25T11:14:20.310353+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:14:20.310353+0000","last_undegraded":"2024-04-25T11:14:20.310353+0000","last_fullsized":"2024-04-25T11:14:20.310353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476788,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1380352,"data_stored":942919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476789,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":786432,"data_stored":352551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":774144,"data_stored":345892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476789,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":352551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":366410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":368786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1404928,"data_stored":956778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":366410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:14:26.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:14:27.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.051+0000 7f2fcbc00640 1 Processor -- start 2024-04-25T11:14:27.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.051+0000 7f2fcbc00640 1 -- start start 2024-04-25T11:14:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.053+0000 7f2fcbc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2fc406dc90 0x7f2fc40afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.053+0000 7f2fcbc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160a30 0x7f2fc40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.053+0000 7f2fcbc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc40b4980 con 0x7f2fc406f300 2024-04-25T11:14:27.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.053+0000 7f2fcbc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc40b4c60 con 0x7f2fc406dc90 2024-04-25T11:14:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.053+0000 7f2fcbc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc40b4f40 con 0x7f2fc4160a30 2024-04-25T11:14:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.054+0000 7f2fc3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2fc406f300 0x7f2fc40ae960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52956/0 (socket says 172.21.15.7:52956) 2024-04-25T11:14:27.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.054+0000 7f2fc3e00640 1 -- 172.21.15.7:0/3168660554 learned_addr learned my addr 172.21.15.7:0/3168660554 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.054+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1370354159 0 0) 0x7f2fc40b4980 con 0x7f2fc406f300 2024-04-25T11:14:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.054+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fa8003520 con 0x7f2fc406f300 2024-04-25T11:14:27.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.054+0000 7f2fc9200640 1 --2- 172.21.15.7:0/3168660554 >> v2:172.21.15.87:3300/0 conn(0x7f2fc406dc90 0x7f2fc40afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.054+0000 7f2fc9c00640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160a30 0x7f2fc40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.054+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2065720257 0 0) 0x7f2fa8003520 con 0x7f2fc406f300 2024-04-25T11:14:27.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160a30 msgr2=0x7f2fc40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:27.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160a30 0x7f2fc40b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 >> v2:172.21.15.87:3300/0 conn(0x7f2fc406dc90 msgr2=0x7f2fc40afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:27.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 --2- 172.21.15.7:0/3168660554 >> v2:172.21.15.87:3300/0 conn(0x7f2fc406dc90 0x7f2fc40afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc40b5220 con 0x7f2fc406f300 2024-04-25T11:14:27.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2fb0001670 con 0x7f2fc406f300 2024-04-25T11:14:27.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 >> v1:172.21.15.7:6789/0 conn(0x7f2fc406f300 legacy=0x7f2fc40ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039b0 0x7f2fa8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8004800 0x7f2fa8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160a30 0x7f2fa8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc40b4c60 con 0x7f2fa80039b0 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc40b4f40 con 0x7f2fa8004800 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa800b9f0 con 0x7f2fc4160a30 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc9200640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039b0 0x7f2fa8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc9c00640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160a30 0x7f2fa8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.055+0000 7f2fc3e00640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8004800 0x7f2fa8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.056+0000 7f2fc9200640 1 -- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160a30 msgr2=0x7f2fa8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.056+0000 7f2fc9200640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160a30 0x7f2fa8005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.056+0000 7f2fc9200640 1 -- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8004800 msgr2=0x7f2fa8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:27.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.056+0000 7f2fc9200640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8004800 0x7f2fa8004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.056+0000 7f2fc9200640 1 -- 172.21.15.7:0/3168660554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa8003520 con 0x7f2fa80039b0 2024-04-25T11:14:27.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.056+0000 7f2fc9c00640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160a30 0x7f2fa8005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:27.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.056+0000 7f2fc3e00640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8004800 0x7f2fa8004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.057+0000 7f2fc9200640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039b0 0x7f2fa8003e20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009a90 tx=0x7f2fa8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b571b221a11f42d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.057+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4036070 con 0x7f2fa80039b0 2024-04-25T11:14:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.057+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb4031400 con 0x7f2fa80039b0 2024-04-25T11:14:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.057+0000 7f2fc3400640 1 -- 172.21.15.7:0/3168660554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4009690 con 0x7f2fa80039b0 2024-04-25T11:14:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.058+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3168660554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039b0 msgr2=0x7f2fa8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.058+0000 7f2fcbc00640 1 --2- 172.21.15.7:0/3168660554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039b0 0x7f2fa8003e20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009a90 tx=0x7f2fa8005830 comp rx=0 tx=0).stop 2024-04-25T11:14:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.058+0000 7f2fc9c00640 1 -- 172.21.15.7:0/3168660554 reap_dead start 2024-04-25T11:14:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.058+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3168660554 shutdown_connections 2024-04-25T11:14:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.058+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3168660554 >> 172.21.15.7:0/3168660554 conn(0x7f2fc4074200 msgr2=0x7f2fc406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.058+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3168660554 shutdown_connections 2024-04-25T11:14:27.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.058+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3168660554 wait complete. 2024-04-25T11:14:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.060+0000 7f2fcbc00640 1 Processor -- start 2024-04-25T11:14:27.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.061+0000 7f2fcbc00640 1 -- start start 2024-04-25T11:14:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.062+0000 7f2fcbc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fa80039b0 0x7f2fc414b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.062+0000 7f2fcbc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc40b2d90 0x7f2fc41f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.062+0000 7f2fcbc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc40b1990 0x7f2fc40b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.062+0000 7f2fcbc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc40b6a30 con 0x7f2fc40b1990 2024-04-25T11:14:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.062+0000 7f2fcbc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc415d220 con 0x7f2fc40b2d90 2024-04-25T11:14:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.062+0000 7f2fcbc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc40ce920 con 0x7f2fa80039b0 2024-04-25T11:14:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.062+0000 7f2fc9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc40b1990 0x7f2fc40b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.063+0000 7f2fc9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc40b1990 0x7f2fc40b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35446/0 (socket says 172.21.15.7:35446) 2024-04-25T11:14:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.063+0000 7f2fc9c00640 1 -- 172.21.15.7:0/3312762454 learned_addr learned my addr 172.21.15.7:0/3312762454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.063+0000 7f2fc9200640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fa80039b0 0x7f2fc414b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.063+0000 7f2fc3e00640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc40b2d90 0x7f2fc41f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.063+0000 7f2fc9c00640 1 -- 172.21.15.7:0/3312762454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fa80039b0 msgr2=0x7f2fc414b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.063+0000 7f2fc9c00640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fa80039b0 0x7f2fc414b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.063+0000 7f2fc9c00640 1 -- 172.21.15.7:0/3312762454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc40b2d90 msgr2=0x7f2fc41f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:27.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.063+0000 7f2fc9c00640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc40b2d90 0x7f2fc41f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.063+0000 7f2fc9c00640 1 -- 172.21.15.7:0/3312762454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc414b6c0 con 0x7f2fc40b1990 2024-04-25T11:14:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.064+0000 7f2fc9c00640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc40b1990 0x7f2fc40b1da0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40d18a0 tx=0x7f2fbc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.064+0000 7f2fc0c00640 1 -- 172.21.15.7:0/3312762454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc01c070 con 0x7f2fc40b1990 2024-04-25T11:14:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.064+0000 7f2fc0c00640 1 -- 172.21.15.7:0/3312762454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fbc017400 con 0x7f2fc40b1990 2024-04-25T11:14:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.064+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3312762454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc414a0e0 con 0x7f2fc40b1990 2024-04-25T11:14:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.064+0000 7f2fc0c00640 1 -- 172.21.15.7:0/3312762454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc0177d0 con 0x7f2fc40b1990 2024-04-25T11:14:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.064+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3312762454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc414a6a0 con 0x7f2fc40b1990 2024-04-25T11:14:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.065+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3312762454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc406a100 con 0x7f2fc40b1990 2024-04-25T11:14:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.066+0000 7f2fc0c00640 1 -- 172.21.15.7:0/3312762454 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2fbc01f070 con 0x7f2fc40b1990 2024-04-25T11:14:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.067+0000 7f2fc0c00640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.068+0000 7f2fc0c00640 1 -- 172.21.15.7:0/3312762454 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f2fbc0953e0 con 0x7f2fc40b1990 2024-04-25T11:14:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.068+0000 7f2fc9200640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.068+0000 7f2fc9200640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4033000 tx=0x7f2fb4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:27.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.072+0000 7f2fc0c00640 1 -- 172.21.15.7:0/3312762454 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fbc012070 con 0x7f2fc40b1990 2024-04-25T11:14:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.265+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3312762454 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2fc41605e0 con 0x7f2fa40744b0 2024-04-25T11:14:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.272+0000 7f2fc0c00640 1 -- 172.21.15.7:0/3312762454 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248630 (secure 0 0 0) 0x7f2fc41605e0 con 0x7f2fa40744b0 2024-04-25T11:14:27.275 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:14:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.277+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3312762454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 msgr2=0x7f2fa40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.277+0000 7f2fcbc00640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4033000 tx=0x7f2fb4008030 comp rx=0 tx=0).stop 2024-04-25T11:14:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.277+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3312762454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc40b1990 msgr2=0x7f2fc40b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.277+0000 7f2fcbc00640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc40b1990 0x7f2fc40b1da0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2fc40d18a0 tx=0x7f2fbc00f4a0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.278+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3312762454 shutdown_connections 2024-04-25T11:14:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.278+0000 7f2fcbc00640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.278+0000 7f2fcbc00640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc40b1990 0x7f2fc40b1da0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.278+0000 7f2fcbc00640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc40b2d90 0x7f2fc41f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.278+0000 7f2fcbc00640 1 --2- 172.21.15.7:0/3312762454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fa80039b0 0x7f2fc414b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.278+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3312762454 >> 172.21.15.7:0/3312762454 conn(0x7f2fc4074200 msgr2=0x7f2fc406fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.278+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3312762454 shutdown_connections 2024-04-25T11:14:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.278+0000 7f2fcbc00640 1 -- 172.21.15.7:0/3312762454 wait complete. 2024-04-25T11:14:27.304 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-04-25T11:14:25.875776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592562,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":118,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248556,"kb_used_data":7416,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726164,"statfs":{"total":773094113280,"available":772839591936,"internally_reserved":0,"allocated":7593984,"data_stored":4052297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001944"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045327+0000","last_change":"2024-04-25T11:14:21.374239+0000","last_active":"2024-04-25T11:14:24.045327+0000","last_peered":"2024-04-25T11:14:24.045327+0000","last_clean":"2024-04-25T11:14:24.045327+0000","last_became_active":"2024-04-25T11:14:21.373912+0000","last_became_peered":"2024-04-25T11:14:21.373912+0000","last_unstale":"2024-04-25T11:14:24.045327+0000","last_undegraded":"2024-04-25T11:14:24.045327+0000","last_fullsized":"2024-04-25T11:14:24.045327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:02.752329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045090+0000","last_change":"2024-04-25T11:14:21.358156+0000","last_active":"2024-04-25T11:14:24.045090+0000","last_peered":"2024-04-25T11:14:24.045090+0000","last_clean":"2024-04-25T11:14:24.045090+0000","last_became_active":"2024-04-25T11:14:21.357825+0000","last_became_peered":"2024-04-25T11:14:21.357825+0000","last_unstale":"2024-04-25T11:14:24.045090+0000","last_undegraded":"2024-04-25T11:14:24.045090+0000","last_fullsized":"2024-04-25T11:14:24.045090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.581113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.306713+0000","last_change":"2024-04-25T11:14:20.306713+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.306713+0000","last_undegraded":"2024-04-25T11:14:20.306713+0000","last_fullsized":"2024-04-25T11:14:20.306713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.323487+0000","last_change":"2024-04-25T11:14:20.323487+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.323487+0000","last_undegraded":"2024-04-25T11:14:20.323487+0000","last_fullsized":"2024-04-25T11:14:20.323487+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.334553+0000","last_change":"2024-04-25T11:14:20.334553+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.334553+0000","last_undegraded":"2024-04-25T11:14:20.334553+0000","last_fullsized":"2024-04-25T11:14:20.334553+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962720+0000","last_change":"2024-04-25T11:14:21.381797+0000","last_active":"2024-04-25T11:14:22.962720+0000","last_peered":"2024-04-25T11:14:22.962720+0000","last_clean":"2024-04-25T11:14:22.962720+0000","last_became_active":"2024-04-25T11:14:21.381332+0000","last_became_peered":"2024-04-25T11:14:21.381332+0000","last_unstale":"2024-04-25T11:14:22.962720+0000","last_undegraded":"2024-04-25T11:14:22.962720+0000","last_fullsized":"2024-04-25T11:14:22.962720+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.347162+0000","last_change":"2024-04-25T11:14:21.367266+0000","last_active":"2024-04-25T11:14:23.347162+0000","last_peered":"2024-04-25T11:14:23.347162+0000","last_clean":"2024-04-25T11:14:23.347162+0000","last_became_active":"2024-04-25T11:14:21.366376+0000","last_became_peered":"2024-04-25T11:14:21.366376+0000","last_unstale":"2024-04-25T11:14:23.347162+0000","last_undegraded":"2024-04-25T11:14:23.347162+0000","last_fullsized":"2024-04-25T11:14:23.347162+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:26.828975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328195+0000","last_change":"2024-04-25T11:14:20.328195+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328195+0000","last_undegraded":"2024-04-25T11:14:20.328195+0000","last_fullsized":"2024-04-25T11:14:20.328195+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.307139+0000","last_change":"2024-04-25T11:14:20.307139+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.307139+0000","last_undegraded":"2024-04-25T11:14:20.307139+0000","last_fullsized":"2024-04-25T11:14:20.307139+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.307340+0000","last_change":"2024-04-25T11:14:20.307340+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.307340+0000","last_undegraded":"2024-04-25T11:14:20.307340+0000","last_fullsized":"2024-04-25T11:14:20.307340+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.320107+0000","last_change":"2024-04-25T11:14:20.320107+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.320107+0000","last_undegraded":"2024-04-25T11:14:20.320107+0000","last_fullsized":"2024-04-25T11:14:20.320107+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.327261+0000","last_change":"2024-04-25T11:14:20.327261+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.327261+0000","last_undegraded":"2024-04-25T11:14:20.327261+0000","last_fullsized":"2024-04-25T11:14:20.327261+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328318+0000","last_change":"2024-04-25T11:14:20.328318+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328318+0000","last_undegraded":"2024-04-25T11:14:20.328318+0000","last_fullsized":"2024-04-25T11:14:20.328318+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.309144+0000","last_change":"2024-04-25T11:14:20.309144+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.309144+0000","last_undegraded":"2024-04-25T11:14:20.309144+0000","last_fullsized":"2024-04-25T11:14:20.309144+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.307698+0000","last_change":"2024-04-25T11:14:20.307698+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.307698+0000","last_undegraded":"2024-04-25T11:14:20.307698+0000","last_fullsized":"2024-04-25T11:14:20.307698+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346754+0000","last_change":"2024-04-25T11:14:21.374629+0000","last_active":"2024-04-25T11:14:23.346754+0000","last_peered":"2024-04-25T11:14:23.346754+0000","last_clean":"2024-04-25T11:14:23.346754+0000","last_became_active":"2024-04-25T11:14:21.373757+0000","last_became_peered":"2024-04-25T11:14:21.373757+0000","last_unstale":"2024-04-25T11:14:23.346754+0000","last_undegraded":"2024-04-25T11:14:23.346754+0000","last_fullsized":"2024-04-25T11:14:23.346754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:46.489832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045260+0000","last_change":"2024-04-25T11:14:21.384370+0000","last_active":"2024-04-25T11:14:24.045260+0000","last_peered":"2024-04-25T11:14:24.045260+0000","last_clean":"2024-04-25T11:14:24.045260+0000","last_became_active":"2024-04-25T11:14:21.383916+0000","last_became_peered":"2024-04-25T11:14:21.383916+0000","last_unstale":"2024-04-25T11:14:24.045260+0000","last_undegraded":"2024-04-25T11:14:24.045260+0000","last_fullsized":"2024-04-25T11:14:24.045260+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.182058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.323323+0000","last_change":"2024-04-25T11:14:20.323323+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.323323+0000","last_undegraded":"2024-04-25T11:14:20.323323+0000","last_fullsized":"2024-04-25T11:14:20.323323+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.325689+0000","last_change":"2024-04-25T11:14:20.325689+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.325689+0000","last_undegraded":"2024-04-25T11:14:20.325689+0000","last_fullsized":"2024-04-25T11:14:20.325689+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362235+0000","last_change":"2024-04-25T11:14:21.370707+0000","last_active":"2024-04-25T11:14:22.362235+0000","last_peered":"2024-04-25T11:14:22.362235+0000","last_clean":"2024-04-25T11:14:22.362235+0000","last_became_active":"2024-04-25T11:14:21.370409+0000","last_became_peered":"2024-04-25T11:14:21.370409+0000","last_unstale":"2024-04-25T11:14:22.362235+0000","last_undegraded":"2024-04-25T11:14:22.362235+0000","last_fullsized":"2024-04-25T11:14:22.362235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.327665+0000","last_change":"2024-04-25T11:14:20.327665+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.327665+0000","last_undegraded":"2024-04-25T11:14:20.327665+0000","last_fullsized":"2024-04-25T11:14:20.327665+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362273+0000","last_change":"2024-04-25T11:14:21.380852+0000","last_active":"2024-04-25T11:14:22.362273+0000","last_peered":"2024-04-25T11:14:22.362273+0000","last_clean":"2024-04-25T11:14:22.362273+0000","last_became_active":"2024-04-25T11:14:21.380226+0000","last_became_peered":"2024-04-25T11:14:21.380226+0000","last_unstale":"2024-04-25T11:14:22.362273+0000","last_undegraded":"2024-04-25T11:14:22.362273+0000","last_fullsized":"2024-04-25T11:14:22.362273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:12.837877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361853+0000","last_change":"2024-04-25T11:14:21.365901+0000","last_active":"2024-04-25T11:14:22.361853+0000","last_peered":"2024-04-25T11:14:22.361853+0000","last_clean":"2024-04-25T11:14:22.361853+0000","last_became_active":"2024-04-25T11:14:21.365551+0000","last_became_peered":"2024-04-25T11:14:21.365551+0000","last_unstale":"2024-04-25T11:14:22.361853+0000","last_undegraded":"2024-04-25T11:14:22.361853+0000","last_fullsized":"2024-04-25T11:14:22.361853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:42.952154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.305880+0000","last_change":"2024-04-25T11:14:20.305880+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.305880+0000","last_undegraded":"2024-04-25T11:14:20.305880+0000","last_fullsized":"2024-04-25T11:14:20.305880+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328612+0000","last_change":"2024-04-25T11:14:20.328612+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328612+0000","last_undegraded":"2024-04-25T11:14:20.328612+0000","last_fullsized":"2024-04-25T11:14:20.328612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328574+0000","last_change":"2024-04-25T11:14:20.328574+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328574+0000","last_undegraded":"2024-04-25T11:14:20.328574+0000","last_fullsized":"2024-04-25T11:14:20.328574+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.318869+0000","last_change":"2024-04-25T11:14:20.318869+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.318869+0000","last_undegraded":"2024-04-25T11:14:20.318869+0000","last_fullsized":"2024-04-25T11:14:20.318869+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044519+0000","last_change":"2024-04-25T11:14:21.371250+0000","last_active":"2024-04-25T11:14:24.044519+0000","last_peered":"2024-04-25T11:14:24.044519+0000","last_clean":"2024-04-25T11:14:24.044519+0000","last_became_active":"2024-04-25T11:14:21.370864+0000","last_became_peered":"2024-04-25T11:14:21.370864+0000","last_unstale":"2024-04-25T11:14:24.044519+0000","last_undegraded":"2024-04-25T11:14:24.044519+0000","last_fullsized":"2024-04-25T11:14:24.044519+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:09.063896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.305995+0000","last_change":"2024-04-25T11:14:20.305995+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.305995+0000","last_undegraded":"2024-04-25T11:14:20.305995+0000","last_fullsized":"2024-04-25T11:14:20.305995+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.328804+0000","last_change":"2024-04-25T11:14:20.328804+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.328804+0000","last_undegraded":"2024-04-25T11:14:20.328804+0000","last_fullsized":"2024-04-25T11:14:20.328804+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.318482+0000","last_change":"2024-04-25T11:14:20.318482+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.318482+0000","last_undegraded":"2024-04-25T11:14:20.318482+0000","last_fullsized":"2024-04-25T11:14:20.318482+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961814+0000","last_change":"2024-04-25T11:14:21.360674+0000","last_active":"2024-04-25T11:14:22.961814+0000","last_peered":"2024-04-25T11:14:22.961814+0000","last_clean":"2024-04-25T11:14:22.961814+0000","last_became_active":"2024-04-25T11:14:21.359863+0000","last_became_peered":"2024-04-25T11:14:21.359863+0000","last_unstale":"2024-04-25T11:14:22.961814+0000","last_undegraded":"2024-04-25T11:14:22.961814+0000","last_fullsized":"2024-04-25T11:14:22.961814+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:49.430307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.323584+0000","last_change":"2024-04-25T11:14:20.323584+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.323584+0000","last_undegraded":"2024-04-25T11:14:20.323584+0000","last_fullsized":"2024-04-25T11:14:20.323584+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"67'4","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045418+0000","last_change":"2024-04-25T11:14:20.334352+0000","last_active":"2024-04-25T11:14:24.045418+0000","last_peered":"2024-04-25T11:14:24.045418+0000","last_clean":"2024-04-25T11:14:24.045418+0000","last_became_active":"2024-04-25T11:14:20.329783+0000","last_became_peered":"2024-04-25T11:14:20.329783+0000","last_unstale":"2024-04-25T11:14:24.045418+0000","last_undegraded":"2024-04-25T11:14:24.045418+0000","last_fullsized":"2024-04-25T11:14:24.045418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:55.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.317695+0000","last_change":"2024-04-25T11:14:20.317695+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.317695+0000","last_undegraded":"2024-04-25T11:14:20.317695+0000","last_fullsized":"2024-04-25T11:14:20.317695+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962745+0000","last_change":"2024-04-25T11:14:20.346713+0000","last_active":"2024-04-25T11:14:22.962745+0000","last_peered":"2024-04-25T11:14:22.962745+0000","last_clean":"2024-04-25T11:14:22.962745+0000","last_became_active":"2024-04-25T11:14:20.346320+0000","last_became_peered":"2024-04-25T11:14:20.346320+0000","last_unstale":"2024-04-25T11:14:22.962745+0000","last_undegraded":"2024-04-25T11:14:22.962745+0000","last_fullsized":"2024-04-25T11:14:22.962745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.559248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962535+0000","last_change":"2024-04-25T11:14:21.356785+0000","last_active":"2024-04-25T11:14:22.962535+0000","last_peered":"2024-04-25T11:14:22.962535+0000","last_clean":"2024-04-25T11:14:22.962535+0000","last_became_active":"2024-04-25T11:14:21.356335+0000","last_became_peered":"2024-04-25T11:14:21.356335+0000","last_unstale":"2024-04-25T11:14:22.962535+0000","last_undegraded":"2024-04-25T11:14:22.962535+0000","last_fullsized":"2024-04-25T11:14:22.962535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.363496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962440+0000","last_change":"2024-04-25T11:14:20.346384+0000","last_active":"2024-04-25T11:14:22.962440+0000","last_peered":"2024-04-25T11:14:22.962440+0000","last_clean":"2024-04-25T11:14:22.962440+0000","last_became_active":"2024-04-25T11:14:20.346031+0000","last_became_peered":"2024-04-25T11:14:20.346031+0000","last_unstale":"2024-04-25T11:14:22.962440+0000","last_undegraded":"2024-04-25T11:14:22.962440+0000","last_fullsized":"2024-04-25T11:14:22.962440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:22.963995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.308498+0000","last_change":"2024-04-25T11:14:20.308498+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.308498+0000","last_undegraded":"2024-04-25T11:14:20.308498+0000","last_fullsized":"2024-04-25T11:14:20.308498+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962159+0000","last_change":"2024-04-25T11:14:20.337082+0000","last_active":"2024-04-25T11:14:22.962159+0000","last_peered":"2024-04-25T11:14:22.962159+0000","last_clean":"2024-04-25T11:14:22.962159+0000","last_became_active":"2024-04-25T11:14:20.336766+0000","last_became_peered":"2024-04-25T11:14:20.336766+0000","last_unstale":"2024-04-25T11:14:22.962159+0000","last_undegraded":"2024-04-25T11:14:22.962159+0000","last_fullsized":"2024-04-25T11:14:22.962159+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:12.462630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346829+0000","last_change":"2024-04-25T11:14:21.346395+0000","last_active":"2024-04-25T11:14:23.346829+0000","last_peered":"2024-04-25T11:14:23.346829+0000","last_clean":"2024-04-25T11:14:23.346829+0000","last_became_active":"2024-04-25T11:14:21.345933+0000","last_became_peered":"2024-04-25T11:14:21.345933+0000","last_unstale":"2024-04-25T11:14:23.346829+0000","last_undegraded":"2024-04-25T11:14:23.346829+0000","last_fullsized":"2024-04-25T11:14:23.346829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.002374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362073+0000","last_change":"2024-04-25T11:14:20.335533+0000","last_active":"2024-04-25T11:14:22.362073+0000","last_peered":"2024-04-25T11:14:22.362073+0000","last_clean":"2024-04-25T11:14:22.362073+0000","last_became_active":"2024-04-25T11:14:20.335228+0000","last_became_peered":"2024-04-25T11:14:20.335228+0000","last_unstale":"2024-04-25T11:14:22.362073+0000","last_undegraded":"2024-04-25T11:14:22.362073+0000","last_fullsized":"2024-04-25T11:14:22.362073+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.347332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044109+0000","last_change":"2024-04-25T11:14:21.384939+0000","last_active":"2024-04-25T11:14:24.044109+0000","last_peered":"2024-04-25T11:14:24.044109+0000","last_clean":"2024-04-25T11:14:24.044109+0000","last_became_active":"2024-04-25T11:14:21.384525+0000","last_became_peered":"2024-04-25T11:14:21.384525+0000","last_unstale":"2024-04-25T11:14:24.044109+0000","last_undegraded":"2024-04-25T11:14:24.044109+0000","last_fullsized":"2024-04-25T11:14:24.044109+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:19.066276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361973+0000","last_change":"2024-04-25T11:14:20.317044+0000","last_active":"2024-04-25T11:14:22.361973+0000","last_peered":"2024-04-25T11:14:22.361973+0000","last_clean":"2024-04-25T11:14:22.361973+0000","last_became_active":"2024-04-25T11:14:20.316634+0000","last_became_peered":"2024-04-25T11:14:20.316634+0000","last_unstale":"2024-04-25T11:14:22.361973+0000","last_undegraded":"2024-04-25T11:14:22.361973+0000","last_fullsized":"2024-04-25T11:14:22.361973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:12.462630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.325823+0000","last_change":"2024-04-25T11:14:20.325823+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.325823+0000","last_undegraded":"2024-04-25T11:14:20.325823+0000","last_fullsized":"2024-04-25T11:14:20.325823+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361960+0000","last_change":"2024-04-25T11:14:20.339374+0000","last_active":"2024-04-25T11:14:22.361960+0000","last_peered":"2024-04-25T11:14:22.361960+0000","last_clean":"2024-04-25T11:14:22.361960+0000","last_became_active":"2024-04-25T11:14:20.339026+0000","last_became_peered":"2024-04-25T11:14:20.339026+0000","last_unstale":"2024-04-25T11:14:22.361960+0000","last_undegraded":"2024-04-25T11:14:22.361960+0000","last_fullsized":"2024-04-25T11:14:22.361960+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:09.182038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362009+0000","last_change":"2024-04-25T11:14:21.380186+0000","last_active":"2024-04-25T11:14:22.362009+0000","last_peered":"2024-04-25T11:14:22.362009+0000","last_clean":"2024-04-25T11:14:22.362009+0000","last_became_active":"2024-04-25T11:14:21.379835+0000","last_became_peered":"2024-04-25T11:14:21.379835+0000","last_unstale":"2024-04-25T11:14:22.362009+0000","last_undegraded":"2024-04-25T11:14:22.362009+0000","last_fullsized":"2024-04-25T11:14:22.362009+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:25.039608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350495+0000","last_change":"2024-04-25T11:14:20.344908+0000","last_active":"2024-04-25T11:14:20.350495+0000","last_peered":"2024-04-25T11:14:20.350495+0000","last_clean":"2024-04-25T11:14:20.350495+0000","last_became_active":"2024-04-25T11:14:20.344554+0000","last_became_peered":"2024-04-25T11:14:20.344554+0000","last_unstale":"2024-04-25T11:14:20.350495+0000","last_undegraded":"2024-04-25T11:14:20.350495+0000","last_fullsized":"2024-04-25T11:14:20.350495+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:55.261892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044489+0000","last_change":"2024-04-25T11:14:21.352169+0000","last_active":"2024-04-25T11:14:24.044489+0000","last_peered":"2024-04-25T11:14:24.044489+0000","last_clean":"2024-04-25T11:14:24.044489+0000","last_became_active":"2024-04-25T11:14:21.351353+0000","last_became_peered":"2024-04-25T11:14:21.351353+0000","last_unstale":"2024-04-25T11:14:24.044489+0000","last_undegraded":"2024-04-25T11:14:24.044489+0000","last_fullsized":"2024-04-25T11:14:24.044489+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:40.169340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.342416+0000","last_change":"2024-04-25T11:14:20.330643+0000","last_active":"2024-04-25T11:14:20.342416+0000","last_peered":"2024-04-25T11:14:20.342416+0000","last_clean":"2024-04-25T11:14:20.342416+0000","last_became_active":"2024-04-25T11:14:20.330199+0000","last_became_peered":"2024-04-25T11:14:20.330199+0000","last_unstale":"2024-04-25T11:14:20.342416+0000","last_undegraded":"2024-04-25T11:14:20.342416+0000","last_fullsized":"2024-04-25T11:14:20.342416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:03.000623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362035+0000","last_change":"2024-04-25T11:14:21.358807+0000","last_active":"2024-04-25T11:14:22.362035+0000","last_peered":"2024-04-25T11:14:22.362035+0000","last_clean":"2024-04-25T11:14:22.362035+0000","last_became_active":"2024-04-25T11:14:21.358476+0000","last_became_peered":"2024-04-25T11:14:21.358476+0000","last_unstale":"2024-04-25T11:14:22.362035+0000","last_undegraded":"2024-04-25T11:14:22.362035+0000","last_fullsized":"2024-04-25T11:14:22.362035+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.363496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.351029+0000","last_change":"2024-04-25T11:14:20.344529+0000","last_active":"2024-04-25T11:14:20.351029+0000","last_peered":"2024-04-25T11:14:20.351029+0000","last_clean":"2024-04-25T11:14:20.351029+0000","last_became_active":"2024-04-25T11:14:20.344184+0000","last_became_peered":"2024-04-25T11:14:20.344184+0000","last_unstale":"2024-04-25T11:14:20.351029+0000","last_undegraded":"2024-04-25T11:14:20.351029+0000","last_fullsized":"2024-04-25T11:14:20.351029+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:34.633280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.335208+0000","last_change":"2024-04-25T11:14:20.335208+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.335208+0000","last_undegraded":"2024-04-25T11:14:20.335208+0000","last_fullsized":"2024-04-25T11:14:20.335208+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344972+0000","last_change":"2024-04-25T11:14:20.337503+0000","last_active":"2024-04-25T11:14:20.344972+0000","last_peered":"2024-04-25T11:14:20.344972+0000","last_clean":"2024-04-25T11:14:20.344972+0000","last_became_active":"2024-04-25T11:14:20.337171+0000","last_became_peered":"2024-04-25T11:14:20.337171+0000","last_unstale":"2024-04-25T11:14:20.344972+0000","last_undegraded":"2024-04-25T11:14:20.344972+0000","last_fullsized":"2024-04-25T11:14:20.344972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:08.289759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321558+0000","last_change":"2024-04-25T11:14:20.321558+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321558+0000","last_undegraded":"2024-04-25T11:14:20.321558+0000","last_fullsized":"2024-04-25T11:14:20.321558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"67'6","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044599+0000","last_change":"2024-04-25T11:14:20.343544+0000","last_active":"2024-04-25T11:14:24.044599+0000","last_peered":"2024-04-25T11:14:24.044599+0000","last_clean":"2024-04-25T11:14:24.044599+0000","last_became_active":"2024-04-25T11:14:20.343205+0000","last_became_peered":"2024-04-25T11:14:20.343205+0000","last_unstale":"2024-04-25T11:14:24.044599+0000","last_undegraded":"2024-04-25T11:14:24.044599+0000","last_fullsized":"2024-04-25T11:14:24.044599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:03.224076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.316535+0000","last_change":"2024-04-25T11:14:20.316535+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.316535+0000","last_undegraded":"2024-04-25T11:14:20.316535+0000","last_fullsized":"2024-04-25T11:14:20.316535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.346424+0000","last_change":"2024-04-25T11:14:20.339359+0000","last_active":"2024-04-25T11:14:20.346424+0000","last_peered":"2024-04-25T11:14:20.346424+0000","last_clean":"2024-04-25T11:14:20.346424+0000","last_became_active":"2024-04-25T11:14:20.338755+0000","last_became_peered":"2024-04-25T11:14:20.338755+0000","last_unstale":"2024-04-25T11:14:20.346424+0000","last_undegraded":"2024-04-25T11:14:20.346424+0000","last_fullsized":"2024-04-25T11:14:20.346424+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:22.063307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.324363+0000","last_change":"2024-04-25T11:14:20.324363+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.324363+0000","last_undegraded":"2024-04-25T11:14:20.324363+0000","last_fullsized":"2024-04-25T11:14:20.324363+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.342888+0000","last_change":"2024-04-25T11:14:20.333436+0000","last_active":"2024-04-25T11:14:20.342888+0000","last_peered":"2024-04-25T11:14:20.342888+0000","last_clean":"2024-04-25T11:14:20.342888+0000","last_became_active":"2024-04-25T11:14:20.333048+0000","last_became_peered":"2024-04-25T11:14:20.333048+0000","last_unstale":"2024-04-25T11:14:20.342888+0000","last_undegraded":"2024-04-25T11:14:20.342888+0000","last_fullsized":"2024-04-25T11:14:20.342888+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:15.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.324999+0000","last_change":"2024-04-25T11:14:20.324999+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.324999+0000","last_undegraded":"2024-04-25T11:14:20.324999+0000","last_fullsized":"2024-04-25T11:14:20.324999+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344030+0000","last_change":"2024-04-25T11:14:20.331830+0000","last_active":"2024-04-25T11:14:20.344030+0000","last_peered":"2024-04-25T11:14:20.344030+0000","last_clean":"2024-04-25T11:14:20.344030+0000","last_became_active":"2024-04-25T11:14:20.331453+0000","last_became_peered":"2024-04-25T11:14:20.331453+0000","last_unstale":"2024-04-25T11:14:20.344030+0000","last_undegraded":"2024-04-25T11:14:20.344030+0000","last_fullsized":"2024-04-25T11:14:20.344030+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:52.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044411+0000","last_change":"2024-04-25T11:14:21.365796+0000","last_active":"2024-04-25T11:14:24.044411+0000","last_peered":"2024-04-25T11:14:24.044411+0000","last_clean":"2024-04-25T11:14:24.044411+0000","last_became_active":"2024-04-25T11:14:21.364748+0000","last_became_peered":"2024-04-25T11:14:21.364748+0000","last_unstale":"2024-04-25T11:14:24.044411+0000","last_undegraded":"2024-04-25T11:14:24.044411+0000","last_fullsized":"2024-04-25T11:14:24.044411+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:20.532234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.342714+0000","last_change":"2024-04-25T11:14:20.339581+0000","last_active":"2024-04-25T11:14:20.342714+0000","last_peered":"2024-04-25T11:14:20.342714+0000","last_clean":"2024-04-25T11:14:20.342714+0000","last_became_active":"2024-04-25T11:14:20.339208+0000","last_became_peered":"2024-04-25T11:14:20.339208+0000","last_unstale":"2024-04-25T11:14:20.342714+0000","last_undegraded":"2024-04-25T11:14:20.342714+0000","last_fullsized":"2024-04-25T11:14:20.342714+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:30.250638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.306095+0000","last_change":"2024-04-25T11:14:20.306095+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.306095+0000","last_undegraded":"2024-04-25T11:14:20.306095+0000","last_fullsized":"2024-04-25T11:14:20.306095+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350366+0000","last_change":"2024-04-25T11:14:20.344787+0000","last_active":"2024-04-25T11:14:20.350366+0000","last_peered":"2024-04-25T11:14:20.350366+0000","last_clean":"2024-04-25T11:14:20.350366+0000","last_became_active":"2024-04-25T11:14:20.343403+0000","last_became_peered":"2024-04-25T11:14:20.343403+0000","last_unstale":"2024-04-25T11:14:20.350366+0000","last_undegraded":"2024-04-25T11:14:20.350366+0000","last_fullsized":"2024-04-25T11:14:20.350366+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:52.093850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.324658+0000","last_change":"2024-04-25T11:14:20.324658+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.324658+0000","last_undegraded":"2024-04-25T11:14:20.324658+0000","last_fullsized":"2024-04-25T11:14:20.324658+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350521+0000","last_change":"2024-04-25T11:14:20.342160+0000","last_active":"2024-04-25T11:14:20.350521+0000","last_peered":"2024-04-25T11:14:20.350521+0000","last_clean":"2024-04-25T11:14:20.350521+0000","last_became_active":"2024-04-25T11:14:20.341788+0000","last_became_peered":"2024-04-25T11:14:20.341788+0000","last_unstale":"2024-04-25T11:14:20.350521+0000","last_undegraded":"2024-04-25T11:14:20.350521+0000","last_fullsized":"2024-04-25T11:14:20.350521+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:32.205173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.322564+0000","last_change":"2024-04-25T11:14:20.322564+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.322564+0000","last_undegraded":"2024-04-25T11:14:20.322564+0000","last_fullsized":"2024-04-25T11:14:20.322564+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.346096+0000","last_change":"2024-04-25T11:14:20.341298+0000","last_active":"2024-04-25T11:14:20.346096+0000","last_peered":"2024-04-25T11:14:20.346096+0000","last_clean":"2024-04-25T11:14:20.346096+0000","last_became_active":"2024-04-25T11:14:20.340803+0000","last_became_peered":"2024-04-25T11:14:20.340803+0000","last_unstale":"2024-04-25T11:14:20.346096+0000","last_undegraded":"2024-04-25T11:14:20.346096+0000","last_fullsized":"2024-04-25T11:14:20.346096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:21.473511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.327161+0000","last_change":"2024-04-25T11:14:20.327161+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.327161+0000","last_undegraded":"2024-04-25T11:14:20.327161+0000","last_fullsized":"2024-04-25T11:14:20.327161+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362338+0000","last_change":"2024-04-25T11:14:20.341606+0000","last_active":"2024-04-25T11:14:22.362338+0000","last_peered":"2024-04-25T11:14:22.362338+0000","last_clean":"2024-04-25T11:14:22.362338+0000","last_became_active":"2024-04-25T11:14:20.341220+0000","last_became_peered":"2024-04-25T11:14:20.341220+0000","last_unstale":"2024-04-25T11:14:22.362338+0000","last_undegraded":"2024-04-25T11:14:22.362338+0000","last_fullsized":"2024-04-25T11:14:22.362338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:13.222820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961827+0000","last_change":"2024-04-25T11:14:21.359973+0000","last_active":"2024-04-25T11:14:22.961827+0000","last_peered":"2024-04-25T11:14:22.961827+0000","last_clean":"2024-04-25T11:14:22.961827+0000","last_became_active":"2024-04-25T11:14:21.359599+0000","last_became_peered":"2024-04-25T11:14:21.359599+0000","last_unstale":"2024-04-25T11:14:22.961827+0000","last_undegraded":"2024-04-25T11:14:22.961827+0000","last_fullsized":"2024-04-25T11:14:22.961827+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:42.952154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961999+0000","last_change":"2024-04-25T11:14:20.341545+0000","last_active":"2024-04-25T11:14:22.961999+0000","last_peered":"2024-04-25T11:14:22.961999+0000","last_clean":"2024-04-25T11:14:22.961999+0000","last_became_active":"2024-04-25T11:14:20.341206+0000","last_became_peered":"2024-04-25T11:14:20.341206+0000","last_unstale":"2024-04-25T11:14:22.961999+0000","last_undegraded":"2024-04-25T11:14:22.961999+0000","last_fullsized":"2024-04-25T11:14:22.961999+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:13.222820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321204+0000","last_change":"2024-04-25T11:14:20.321204+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321204+0000","last_undegraded":"2024-04-25T11:14:20.321204+0000","last_fullsized":"2024-04-25T11:14:20.321204+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.343502+0000","last_change":"2024-04-25T11:14:20.341156+0000","last_active":"2024-04-25T11:14:20.343502+0000","last_peered":"2024-04-25T11:14:20.343502+0000","last_clean":"2024-04-25T11:14:20.343502+0000","last_became_active":"2024-04-25T11:14:20.340815+0000","last_became_peered":"2024-04-25T11:14:20.340815+0000","last_unstale":"2024-04-25T11:14:20.343502+0000","last_undegraded":"2024-04-25T11:14:20.343502+0000","last_fullsized":"2024-04-25T11:14:20.343502+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:03.000623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321752+0000","last_change":"2024-04-25T11:14:20.321752+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321752+0000","last_undegraded":"2024-04-25T11:14:20.321752+0000","last_fullsized":"2024-04-25T11:14:20.321752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961780+0000","last_change":"2024-04-25T11:14:20.345177+0000","last_active":"2024-04-25T11:14:22.961780+0000","last_peered":"2024-04-25T11:14:22.961780+0000","last_clean":"2024-04-25T11:14:22.961780+0000","last_became_active":"2024-04-25T11:14:20.344632+0000","last_became_peered":"2024-04-25T11:14:20.344632+0000","last_unstale":"2024-04-25T11:14:22.961780+0000","last_undegraded":"2024-04-25T11:14:22.961780+0000","last_fullsized":"2024-04-25T11:14:22.961780+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:58.724816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962522+0000","last_change":"2024-04-25T11:14:21.380957+0000","last_active":"2024-04-25T11:14:22.962522+0000","last_peered":"2024-04-25T11:14:22.962522+0000","last_clean":"2024-04-25T11:14:22.962522+0000","last_became_active":"2024-04-25T11:14:21.380702+0000","last_became_peered":"2024-04-25T11:14:21.380702+0000","last_unstale":"2024-04-25T11:14:22.962522+0000","last_undegraded":"2024-04-25T11:14:22.962522+0000","last_fullsized":"2024-04-25T11:14:22.962522+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:25.039608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.350343+0000","last_change":"2024-04-25T11:14:20.343699+0000","last_active":"2024-04-25T11:14:20.350343+0000","last_peered":"2024-04-25T11:14:20.350343+0000","last_clean":"2024-04-25T11:14:20.350343+0000","last_became_active":"2024-04-25T11:14:20.343351+0000","last_became_peered":"2024-04-25T11:14:20.343351+0000","last_unstale":"2024-04-25T11:14:20.350343+0000","last_undegraded":"2024-04-25T11:14:20.350343+0000","last_fullsized":"2024-04-25T11:14:20.350343+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:55.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.347275+0000","last_change":"2024-04-25T11:14:21.355386+0000","last_active":"2024-04-25T11:14:23.347275+0000","last_peered":"2024-04-25T11:14:23.347275+0000","last_clean":"2024-04-25T11:14:23.347275+0000","last_became_active":"2024-04-25T11:14:21.355092+0000","last_became_peered":"2024-04-25T11:14:21.355092+0000","last_unstale":"2024-04-25T11:14:23.347275+0000","last_undegraded":"2024-04-25T11:14:23.347275+0000","last_fullsized":"2024-04-25T11:14:23.347275+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:55.946442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344329+0000","last_change":"2024-04-25T11:14:20.336075+0000","last_active":"2024-04-25T11:14:20.344329+0000","last_peered":"2024-04-25T11:14:20.344329+0000","last_clean":"2024-04-25T11:14:20.344329+0000","last_became_active":"2024-04-25T11:14:20.335712+0000","last_became_peered":"2024-04-25T11:14:20.335712+0000","last_unstale":"2024-04-25T11:14:20.344329+0000","last_undegraded":"2024-04-25T11:14:20.344329+0000","last_fullsized":"2024-04-25T11:14:20.344329+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:03.224076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.321323+0000","last_change":"2024-04-25T11:14:20.321323+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.321323+0000","last_undegraded":"2024-04-25T11:14:20.321323+0000","last_fullsized":"2024-04-25T11:14:20.321323+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962444+0000","last_change":"2024-04-25T11:14:20.346848+0000","last_active":"2024-04-25T11:14:22.962444+0000","last_peered":"2024-04-25T11:14:22.962444+0000","last_clean":"2024-04-25T11:14:22.962444+0000","last_became_active":"2024-04-25T11:14:20.346439+0000","last_became_peered":"2024-04-25T11:14:20.346439+0000","last_unstale":"2024-04-25T11:14:22.962444+0000","last_undegraded":"2024-04-25T11:14:22.962444+0000","last_fullsized":"2024-04-25T11:14:22.962444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.347332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.326463+0000","last_change":"2024-04-25T11:14:20.326463+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.326463+0000","last_undegraded":"2024-04-25T11:14:20.326463+0000","last_fullsized":"2024-04-25T11:14:20.326463+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044726+0000","last_change":"2024-04-25T11:14:20.344137+0000","last_active":"2024-04-25T11:14:24.044726+0000","last_peered":"2024-04-25T11:14:24.044726+0000","last_clean":"2024-04-25T11:14:24.044726+0000","last_became_active":"2024-04-25T11:14:20.343813+0000","last_became_peered":"2024-04-25T11:14:20.343813+0000","last_unstale":"2024-04-25T11:14:24.044726+0000","last_undegraded":"2024-04-25T11:14:24.044726+0000","last_fullsized":"2024-04-25T11:14:24.044726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:03.000623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.326037+0000","last_change":"2024-04-25T11:14:20.326037+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.326037+0000","last_undegraded":"2024-04-25T11:14:20.326037+0000","last_fullsized":"2024-04-25T11:14:20.326037+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.344257+0000","last_change":"2024-04-25T11:14:20.331243+0000","last_active":"2024-04-25T11:14:20.344257+0000","last_peered":"2024-04-25T11:14:20.344257+0000","last_clean":"2024-04-25T11:14:20.344257+0000","last_became_active":"2024-04-25T11:14:20.330860+0000","last_became_peered":"2024-04-25T11:14:20.330860+0000","last_unstale":"2024-04-25T11:14:20.344257+0000","last_undegraded":"2024-04-25T11:14:20.344257+0000","last_fullsized":"2024-04-25T11:14:20.344257+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:55.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.317213+0000","last_change":"2024-04-25T11:14:20.317213+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.317213+0000","last_undegraded":"2024-04-25T11:14:20.317213+0000","last_fullsized":"2024-04-25T11:14:20.317213+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362445+0000","last_change":"2024-04-25T11:14:20.325726+0000","last_active":"2024-04-25T11:14:22.362445+0000","last_peered":"2024-04-25T11:14:22.362445+0000","last_clean":"2024-04-25T11:14:22.362445+0000","last_became_active":"2024-04-25T11:14:20.325199+0000","last_became_peered":"2024-04-25T11:14:20.325199+0000","last_unstale":"2024-04-25T11:14:22.362445+0000","last_undegraded":"2024-04-25T11:14:22.362445+0000","last_fullsized":"2024-04-25T11:14:22.362445+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:22.963995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044144+0000","last_change":"2024-04-25T11:14:21.377172+0000","last_active":"2024-04-25T11:14:24.044144+0000","last_peered":"2024-04-25T11:14:24.044144+0000","last_clean":"2024-04-25T11:14:24.044144+0000","last_became_active":"2024-04-25T11:14:21.376648+0000","last_became_peered":"2024-04-25T11:14:21.376648+0000","last_unstale":"2024-04-25T11:14:24.044144+0000","last_undegraded":"2024-04-25T11:14:24.044144+0000","last_fullsized":"2024-04-25T11:14:24.044144+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:49.106485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361947+0000","last_change":"2024-04-25T11:14:20.339034+0000","last_active":"2024-04-25T11:14:22.361947+0000","last_peered":"2024-04-25T11:14:22.361947+0000","last_clean":"2024-04-25T11:14:22.361947+0000","last_became_active":"2024-04-25T11:14:20.338713+0000","last_became_peered":"2024-04-25T11:14:20.338713+0000","last_unstale":"2024-04-25T11:14:22.361947+0000","last_undegraded":"2024-04-25T11:14:22.361947+0000","last_fullsized":"2024-04-25T11:14:22.361947+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:06.914535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346525+0000","last_change":"2024-04-25T11:14:21.373882+0000","last_active":"2024-04-25T11:14:23.346525+0000","last_peered":"2024-04-25T11:14:23.346525+0000","last_clean":"2024-04-25T11:14:23.346525+0000","last_became_active":"2024-04-25T11:14:21.373533+0000","last_became_peered":"2024-04-25T11:14:21.373533+0000","last_unstale":"2024-04-25T11:14:23.346525+0000","last_undegraded":"2024-04-25T11:14:23.346525+0000","last_fullsized":"2024-04-25T11:14:23.346525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:01.628406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.043503+0000","last_change":"2024-04-25T11:14:20.339826+0000","last_active":"2024-04-25T11:14:24.043503+0000","last_peered":"2024-04-25T11:14:24.043503+0000","last_clean":"2024-04-25T11:14:24.043503+0000","last_became_active":"2024-04-25T11:14:20.339366+0000","last_became_peered":"2024-04-25T11:14:20.339366+0000","last_unstale":"2024-04-25T11:14:24.043503+0000","last_undegraded":"2024-04-25T11:14:24.043503+0000","last_fullsized":"2024-04-25T11:14:24.043503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:15.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":66,"state":"creating+peering","last_fresh":"2024-04-25T11:14:20.326984+0000","last_change":"2024-04-25T11:14:20.326984+0000","last_active":"2024-04-25T11:14:20.290926+0000","last_peered":"2024-04-25T11:14:20.290926+0000","last_clean":"2024-04-25T11:14:20.290926+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T11:14:20.326984+0000","last_undegraded":"2024-04-25T11:14:20.326984+0000","last_fullsized":"2024-04-25T11:14:20.326984+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361619+0000","last_change":"2024-04-25T11:14:20.334793+0000","last_active":"2024-04-25T11:14:22.361619+0000","last_peered":"2024-04-25T11:14:22.361619+0000","last_clean":"2024-04-25T11:14:22.361619+0000","last_became_active":"2024-04-25T11:14:20.333198+0000","last_became_peered":"2024-04-25T11:14:20.333198+0000","last_unstale":"2024-04-25T11:14:22.361619+0000","last_undegraded":"2024-04-25T11:14:22.361619+0000","last_fullsized":"2024-04-25T11:14:22.361619+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.559248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":199,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T11:14:20.310353+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:14:20.310353+0000","last_peered":"2024-04-25T11:14:20.310353+0000","last_clean":"2024-04-25T11:14:20.310353+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:14:20.310353+0000","last_undegraded":"2024-04-25T11:14:20.310353+0000","last_fullsized":"2024-04-25T11:14:20.310353+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2194,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476788,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1380352,"data_stored":942919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476789,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":786432,"data_stored":352551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476788,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":774144,"data_stored":345892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476789,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":786432,"data_stored":352551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":366410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":368786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1404928,"data_stored":956778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":366410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:14:27.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2024-04-25T11:14:27.308 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.306713+0000', 'last_change': '2024-04-25T11:14:20.306713+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.306713+0000', 'last_undegraded': '2024-04-25T11:14:20.306713+0000', 'last_fullsized': '2024-04-25T11:14:20.306713+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:14:27.308 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2024-04-25T11:14:27.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.323487+0000', 'last_change': '2024-04-25T11:14:20.323487+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.323487+0000', 'last_undegraded': '2024-04-25T11:14:20.323487+0000', 'last_fullsized': '2024-04-25T11:14:20.323487+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:14:27.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2024-04-25T11:14:27.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.334553+0000', 'last_change': '2024-04-25T11:14:20.334553+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.334553+0000', 'last_undegraded': '2024-04-25T11:14:20.334553+0000', 'last_fullsized': '2024-04-25T11:14:20.334553+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2024-04-25T11:14:27.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.328195+0000', 'last_change': '2024-04-25T11:14:20.328195+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.328195+0000', 'last_undegraded': '2024-04-25T11:14:20.328195+0000', 'last_fullsized': '2024-04-25T11:14:20.328195+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:14:27.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2024-04-25T11:14:27.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.307139+0000', 'last_change': '2024-04-25T11:14:20.307139+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.307139+0000', 'last_undegraded': '2024-04-25T11:14:20.307139+0000', 'last_fullsized': '2024-04-25T11:14:20.307139+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:14:27.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2024-04-25T11:14:27.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.307340+0000', 'last_change': '2024-04-25T11:14:20.307340+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.307340+0000', 'last_undegraded': '2024-04-25T11:14:20.307340+0000', 'last_fullsized': '2024-04-25T11:14:20.307340+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:14:27.309 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2024-04-25T11:14:27.309 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.320107+0000', 'last_change': '2024-04-25T11:14:20.320107+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.320107+0000', 'last_undegraded': '2024-04-25T11:14:20.320107+0000', 'last_fullsized': '2024-04-25T11:14:20.320107+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:14:27.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2024-04-25T11:14:27.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.327261+0000', 'last_change': '2024-04-25T11:14:20.327261+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.327261+0000', 'last_undegraded': '2024-04-25T11:14:20.327261+0000', 'last_fullsized': '2024-04-25T11:14:20.327261+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:14:27.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2024-04-25T11:14:27.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.328318+0000', 'last_change': '2024-04-25T11:14:20.328318+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.328318+0000', 'last_undegraded': '2024-04-25T11:14:20.328318+0000', 'last_fullsized': '2024-04-25T11:14:20.328318+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:14:27.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2024-04-25T11:14:27.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.309144+0000', 'last_change': '2024-04-25T11:14:20.309144+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.309144+0000', 'last_undegraded': '2024-04-25T11:14:20.309144+0000', 'last_fullsized': '2024-04-25T11:14:20.309144+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:14:27.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2024-04-25T11:14:27.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.307698+0000', 'last_change': '2024-04-25T11:14:20.307698+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.307698+0000', 'last_undegraded': '2024-04-25T11:14:20.307698+0000', 'last_fullsized': '2024-04-25T11:14:20.307698+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:14:27.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2024-04-25T11:14:27.310 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.323323+0000', 'last_change': '2024-04-25T11:14:20.323323+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.323323+0000', 'last_undegraded': '2024-04-25T11:14:20.323323+0000', 'last_fullsized': '2024-04-25T11:14:20.323323+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:14:27.310 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.325689+0000', 'last_change': '2024-04-25T11:14:20.325689+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.325689+0000', 'last_undegraded': '2024-04-25T11:14:20.325689+0000', 'last_fullsized': '2024-04-25T11:14:20.325689+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.327665+0000', 'last_change': '2024-04-25T11:14:20.327665+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.327665+0000', 'last_undegraded': '2024-04-25T11:14:20.327665+0000', 'last_fullsized': '2024-04-25T11:14:20.327665+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.305880+0000', 'last_change': '2024-04-25T11:14:20.305880+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.305880+0000', 'last_undegraded': '2024-04-25T11:14:20.305880+0000', 'last_fullsized': '2024-04-25T11:14:20.305880+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.328612+0000', 'last_change': '2024-04-25T11:14:20.328612+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.328612+0000', 'last_undegraded': '2024-04-25T11:14:20.328612+0000', 'last_fullsized': '2024-04-25T11:14:20.328612+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.328574+0000', 'last_change': '2024-04-25T11:14:20.328574+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.328574+0000', 'last_undegraded': '2024-04-25T11:14:20.328574+0000', 'last_fullsized': '2024-04-25T11:14:20.328574+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.318869+0000', 'last_change': '2024-04-25T11:14:20.318869+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.318869+0000', 'last_undegraded': '2024-04-25T11:14:20.318869+0000', 'last_fullsized': '2024-04-25T11:14:20.318869+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:14:27.311 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2024-04-25T11:14:27.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.305995+0000', 'last_change': '2024-04-25T11:14:20.305995+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.305995+0000', 'last_undegraded': '2024-04-25T11:14:20.305995+0000', 'last_fullsized': '2024-04-25T11:14:20.305995+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:14:27.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2024-04-25T11:14:27.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.328804+0000', 'last_change': '2024-04-25T11:14:20.328804+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.328804+0000', 'last_undegraded': '2024-04-25T11:14:20.328804+0000', 'last_fullsized': '2024-04-25T11:14:20.328804+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:14:27.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2024-04-25T11:14:27.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.318482+0000', 'last_change': '2024-04-25T11:14:20.318482+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.318482+0000', 'last_undegraded': '2024-04-25T11:14:20.318482+0000', 'last_fullsized': '2024-04-25T11:14:20.318482+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:14:27.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-04-25T11:14:27.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.323584+0000', 'last_change': '2024-04-25T11:14:20.323584+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.323584+0000', 'last_undegraded': '2024-04-25T11:14:20.323584+0000', 'last_fullsized': '2024-04-25T11:14:20.323584+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2024-04-25T11:14:27.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.317695+0000', 'last_change': '2024-04-25T11:14:20.317695+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.317695+0000', 'last_undegraded': '2024-04-25T11:14:20.317695+0000', 'last_fullsized': '2024-04-25T11:14:20.317695+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:14:27.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2024-04-25T11:14:27.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.308498+0000', 'last_change': '2024-04-25T11:14:20.308498+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.308498+0000', 'last_undegraded': '2024-04-25T11:14:20.308498+0000', 'last_fullsized': '2024-04-25T11:14:20.308498+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:14:27.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-04-25T11:14:27.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.325823+0000', 'last_change': '2024-04-25T11:14:20.325823+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.325823+0000', 'last_undegraded': '2024-04-25T11:14:20.325823+0000', 'last_fullsized': '2024-04-25T11:14:20.325823+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-04-25T11:14:27.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.335208+0000', 'last_change': '2024-04-25T11:14:20.335208+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.335208+0000', 'last_undegraded': '2024-04-25T11:14:20.335208+0000', 'last_fullsized': '2024-04-25T11:14:20.335208+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2024-04-25T11:14:27.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.321558+0000', 'last_change': '2024-04-25T11:14:20.321558+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.321558+0000', 'last_undegraded': '2024-04-25T11:14:20.321558+0000', 'last_fullsized': '2024-04-25T11:14:20.321558+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:14:27.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2024-04-25T11:14:27.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.316535+0000', 'last_change': '2024-04-25T11:14:20.316535+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.316535+0000', 'last_undegraded': '2024-04-25T11:14:20.316535+0000', 'last_fullsized': '2024-04-25T11:14:20.316535+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:14:27.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2024-04-25T11:14:27.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.324363+0000', 'last_change': '2024-04-25T11:14:20.324363+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.324363+0000', 'last_undegraded': '2024-04-25T11:14:20.324363+0000', 'last_fullsized': '2024-04-25T11:14:20.324363+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:14:27.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-04-25T11:14:27.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.324999+0000', 'last_change': '2024-04-25T11:14:20.324999+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.324999+0000', 'last_undegraded': '2024-04-25T11:14:20.324999+0000', 'last_fullsized': '2024-04-25T11:14:20.324999+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2024-04-25T11:14:27.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.306095+0000', 'last_change': '2024-04-25T11:14:20.306095+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.306095+0000', 'last_undegraded': '2024-04-25T11:14:20.306095+0000', 'last_fullsized': '2024-04-25T11:14:20.306095+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T11:14:27.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-04-25T11:14:27.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.324658+0000', 'last_change': '2024-04-25T11:14:20.324658+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.324658+0000', 'last_undegraded': '2024-04-25T11:14:20.324658+0000', 'last_fullsized': '2024-04-25T11:14:20.324658+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2024-04-25T11:14:27.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.322564+0000', 'last_change': '2024-04-25T11:14:20.322564+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.322564+0000', 'last_undegraded': '2024-04-25T11:14:20.322564+0000', 'last_fullsized': '2024-04-25T11:14:20.322564+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:14:27.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-04-25T11:14:27.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.327161+0000', 'last_change': '2024-04-25T11:14:20.327161+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.327161+0000', 'last_undegraded': '2024-04-25T11:14:20.327161+0000', 'last_fullsized': '2024-04-25T11:14:20.327161+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2024-04-25T11:14:27.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.321204+0000', 'last_change': '2024-04-25T11:14:20.321204+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.321204+0000', 'last_undegraded': '2024-04-25T11:14:20.321204+0000', 'last_fullsized': '2024-04-25T11:14:20.321204+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:14:27.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-04-25T11:14:27.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.321752+0000', 'last_change': '2024-04-25T11:14:20.321752+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.321752+0000', 'last_undegraded': '2024-04-25T11:14:20.321752+0000', 'last_fullsized': '2024-04-25T11:14:20.321752+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2024-04-25T11:14:27.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.321323+0000', 'last_change': '2024-04-25T11:14:20.321323+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.321323+0000', 'last_undegraded': '2024-04-25T11:14:20.321323+0000', 'last_fullsized': '2024-04-25T11:14:20.321323+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T11:14:27.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-04-25T11:14:27.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.326463+0000', 'last_change': '2024-04-25T11:14:20.326463+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.326463+0000', 'last_undegraded': '2024-04-25T11:14:20.326463+0000', 'last_fullsized': '2024-04-25T11:14:20.326463+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-04-25T11:14:27.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.326037+0000', 'last_change': '2024-04-25T11:14:20.326037+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.326037+0000', 'last_undegraded': '2024-04-25T11:14:20.326037+0000', 'last_fullsized': '2024-04-25T11:14:20.326037+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2024-04-25T11:14:27.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.317213+0000', 'last_change': '2024-04-25T11:14:20.317213+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.317213+0000', 'last_undegraded': '2024-04-25T11:14:20.317213+0000', 'last_fullsized': '2024-04-25T11:14:20.317213+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T11:14:27.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-04-25T11:14:27.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 66, 'state': 'creating+peering', 'last_fresh': '2024-04-25T11:14:20.326984+0000', 'last_change': '2024-04-25T11:14:20.326984+0000', 'last_active': '2024-04-25T11:14:20.290926+0000', 'last_peered': '2024-04-25T11:14:20.290926+0000', 'last_clean': '2024-04-25T11:14:20.290926+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T11:14:20.326984+0000', 'last_undegraded': '2024-04-25T11:14:20.326984+0000', 'last_fullsized': '2024-04-25T11:14:20.326984+0000', 'mapping_epoch': 66, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 66, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'last_clean_scrub_stamp': '2024-04-25T11:14:20.290926+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T11:14:27.320 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T11:14:27.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.629+0000 7ff17b200640 1 Processor -- start 2024-04-25T11:14:27.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.629+0000 7ff17b200640 1 -- start start 2024-04-25T11:14:27.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.631+0000 7ff17b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff17415e2f0 0x7ff174161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.631+0000 7ff17b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff174162420 0x7ff17406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.631+0000 7ff17b200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff17415e700 con 0x7ff17415baa0 2024-04-25T11:14:27.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.631+0000 7ff17b200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff174162830 con 0x7ff174162420 2024-04-25T11:14:27.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.631+0000 7ff17b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff17415e980 con 0x7ff17415e2f0 2024-04-25T11:14:27.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.632+0000 7ff17a800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff174162420 0x7ff17406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.632+0000 7ff17a800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff174162420 0x7ff17406d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35454/0 (socket says 172.21.15.7:35454) 2024-04-25T11:14:27.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.632+0000 7ff17a800640 1 -- 172.21.15.7:0/1327512845 learned_addr learned my addr 172.21.15.7:0/1327512845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:27.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.632+0000 7ff179400640 1 --2- 172.21.15.7:0/1327512845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff17415e2f0 0x7ff174161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.632+0000 7ff17a800640 1 -- 172.21.15.7:0/1327512845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff17415e2f0 msgr2=0x7ff174161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:27.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.632+0000 7ff17a800640 1 --2- 172.21.15.7:0/1327512845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff17415e2f0 0x7ff174161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.632+0000 7ff17a800640 1 -- 172.21.15.7:0/1327512845 >> v1:172.21.15.87:6789/0 conn(0x7ff17415baa0 legacy=0x7ff174070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:27.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.632+0000 7ff17a800640 1 -- 172.21.15.7:0/1327512845 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17415ec60 con 0x7ff174162420 2024-04-25T11:14:27.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.633+0000 7ff17a800640 1 --2- 172.21.15.7:0/1327512845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff174162420 0x7ff17406d580 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff170009e50 tx=0x7ff17002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459fd268e8df99b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:27.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.634+0000 7ff178a00640 1 -- 172.21.15.7:0/1327512845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170036070 con 0x7ff174162420 2024-04-25T11:14:27.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.634+0000 7ff178a00640 1 -- 172.21.15.7:0/1327512845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff170031600 con 0x7ff174162420 2024-04-25T11:14:27.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.634+0000 7ff178a00640 1 -- 172.21.15.7:0/1327512845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1700319d0 con 0x7ff174162420 2024-04-25T11:14:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.634+0000 7ff17b200640 1 -- 172.21.15.7:0/1327512845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff174162420 msgr2=0x7ff17406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.634+0000 7ff17b200640 1 --2- 172.21.15.7:0/1327512845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff174162420 0x7ff17406d580 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff170009e50 tx=0x7ff17002a6c0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.635+0000 7ff17b200640 1 -- 172.21.15.7:0/1327512845 shutdown_connections 2024-04-25T11:14:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.635+0000 7ff17b200640 1 --2- 172.21.15.7:0/1327512845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff174162420 0x7ff17406d580 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.635+0000 7ff17b200640 1 --2- 172.21.15.7:0/1327512845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff17415e2f0 0x7ff174161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.635+0000 7ff17b200640 1 -- 172.21.15.7:0/1327512845 >> 172.21.15.7:0/1327512845 conn(0x7ff174073090 msgr2=0x7ff1740734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.635+0000 7ff17b200640 1 -- 172.21.15.7:0/1327512845 shutdown_connections 2024-04-25T11:14:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.635+0000 7ff17b200640 1 -- 172.21.15.7:0/1327512845 wait complete. 2024-04-25T11:14:27.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.637+0000 7ff17b200640 1 Processor -- start 2024-04-25T11:14:27.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.637+0000 7ff17b200640 1 -- start start 2024-04-25T11:14:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff17b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff17415e2f0 0x7ff1741e7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff17b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff174162420 0x7ff1741e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff17b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1741ea2f0 0x7ff1741367f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff17b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff17406dac0 con 0x7ff174162420 2024-04-25T11:14:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff17b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff1741619c0 con 0x7ff1741ea2f0 2024-04-25T11:14:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff17b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff17406dc40 con 0x7ff17415e2f0 2024-04-25T11:14:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff179400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff174162420 0x7ff1741e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff179400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff174162420 0x7ff1741e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35464/0 (socket says 172.21.15.7:35464) 2024-04-25T11:14:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff179400640 1 -- 172.21.15.7:0/1401472870 learned_addr learned my addr 172.21.15.7:0/1401472870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff179e00640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff17415e2f0 0x7ff1741e7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff17a800640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1741ea2f0 0x7ff1741367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.639+0000 7ff179400640 1 -- 172.21.15.7:0/1401472870 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff17415e2f0 msgr2=0x7ff1741e7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff179400640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff17415e2f0 0x7ff1741e7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff179400640 1 -- 172.21.15.7:0/1401472870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1741ea2f0 msgr2=0x7ff1741367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff179400640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1741ea2f0 0x7ff1741367f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff179400640 1 -- 172.21.15.7:0/1401472870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff174134230 con 0x7ff174162420 2024-04-25T11:14:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff179e00640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff17415e2f0 0x7ff1741e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff17a800640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1741ea2f0 0x7ff1741367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff179400640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff174162420 0x7ff1741e9db0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff16400d5f0 tx=0x7ff16400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff16e000640 1 -- 172.21.15.7:0/1401472870 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1640125f0 con 0x7ff174162420 2024-04-25T11:14:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff16e000640 1 -- 172.21.15.7:0/1401472870 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff164019070 con 0x7ff174162420 2024-04-25T11:14:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff16e000640 1 -- 172.21.15.7:0/1401472870 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1640129d0 con 0x7ff174162420 2024-04-25T11:14:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff17b200640 1 -- 172.21.15.7:0/1401472870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff174134510 con 0x7ff174162420 2024-04-25T11:14:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.640+0000 7ff17b200640 1 -- 172.21.15.7:0/1401472870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff174134ad0 con 0x7ff174162420 2024-04-25T11:14:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.641+0000 7ff17b200640 1 -- 172.21.15.7:0/1401472870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff174074020 con 0x7ff174162420 2024-04-25T11:14:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.643+0000 7ff16e000640 1 -- 172.21.15.7:0/1401472870 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff164012b70 con 0x7ff174162420 2024-04-25T11:14:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.644+0000 7ff16e000640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1580744e0 0x7ff1580769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.644+0000 7ff16e000640 1 -- 172.21.15.7:0/1401472870 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7ff1640958a0 con 0x7ff174162420 2024-04-25T11:14:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.644+0000 7ff179e00640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1580744e0 0x7ff1580769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.645+0000 7ff179e00640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1580744e0 0x7ff1580769f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff16800a830 tx=0x7ff168008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:27.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.649+0000 7ff16e000640 1 -- 172.21.15.7:0/1401472870 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff16405fe00 con 0x7ff174162420 2024-04-25T11:14:27.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.921+0000 7ff17b200640 1 -- 172.21.15.7:0/1401472870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff174161f80 con 0x7ff174162420 2024-04-25T11:14:27.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.922+0000 7ff16e000640 1 -- 172.21.15.7:0/1401472870 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1098 (secure 0 0 0) 0x7ff16405f700 con 0x7ff174162420 2024-04-25T11:14:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:27.931 INFO:teuthology.orchestra.run.smithi007.stdout:{"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":271,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":68,"num_osds":8,"num_up_osds":8,"osd_up_since":1714043410,"num_in_osds":8,"osd_in_since":1714043400,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":56},{"state_name":"creating+peering","count":41}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":592562,"bytes_used":254521344,"bytes_avail":772839591936,"bytes_total":773094113280,"inactive_pgs_ratio":0.42268040776252747,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":35,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6120}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-04-25T11:13:25.854844+0000","services":{}},"progress_events":{}} 2024-04-25T11:14:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.925+0000 7ff17b200640 1 -- 172.21.15.7:0/1401472870 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1580744e0 msgr2=0x7ff1580769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.925+0000 7ff17b200640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1580744e0 0x7ff1580769f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff16800a830 tx=0x7ff168008040 comp rx=0 tx=0).stop 2024-04-25T11:14:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.925+0000 7ff17b200640 1 -- 172.21.15.7:0/1401472870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff174162420 msgr2=0x7ff1741e9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.925+0000 7ff17b200640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff174162420 0x7ff1741e9db0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff16400d5f0 tx=0x7ff16400dab0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.925+0000 7ff17b200640 1 -- 172.21.15.7:0/1401472870 shutdown_connections 2024-04-25T11:14:27.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.925+0000 7ff17b200640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1580744e0 0x7ff1580769f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.925+0000 7ff17b200640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1741ea2f0 0x7ff1741367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.925+0000 7ff17b200640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff174162420 0x7ff1741e9db0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.925+0000 7ff17b200640 1 --2- 172.21.15.7:0/1401472870 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff17415e2f0 0x7ff1741e7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:27.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.925+0000 7ff17b200640 1 -- 172.21.15.7:0/1401472870 >> 172.21.15.7:0/1401472870 conn(0x7ff174073090 msgr2=0x7ff17406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:27.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.926+0000 7ff17b200640 1 -- 172.21.15.7:0/1401472870 shutdown_connections 2024-04-25T11:14:27.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:27.926+0000 7ff17b200640 1 -- 172.21.15.7:0/1401472870 wait complete. 2024-04-25T11:14:27.953 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T11:14:27.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:14:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.215+0000 7f882d600640 1 Processor -- start 2024-04-25T11:14:28.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.215+0000 7f882d600640 1 -- start start 2024-04-25T11:14:28.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.218+0000 7f882d600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8828163bb0 0x7f88280d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.218+0000 7f882d600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88280bae90 0x7f88280b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.218+0000 7f882d600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f882806c570 0x7f882806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.218+0000 7f882d600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f88280d08e0 con 0x7f8828163bb0 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.218+0000 7f882d600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f88280b51c0 con 0x7f882806c570 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.218+0000 7f882d600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f882806cec0 con 0x7f88280bae90 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.218+0000 7f8827e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8828163bb0 0x7f88280d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.218+0000 7f8827e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8828163bb0 0x7f88280d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35480/0 (socket says 172.21.15.7:35480) 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.218+0000 7f8827e00640 1 -- 172.21.15.7:0/3271232801 learned_addr learned my addr 172.21.15.7:0/3271232801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.218+0000 7f8827400640 1 --2- 172.21.15.7:0/3271232801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88280bae90 0x7f88280b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.218+0000 7f882cc00640 1 --2- 172.21.15.7:0/3271232801 >> v2:172.21.15.87:3300/0 conn(0x7f882806c570 0x7f882806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.219+0000 7f8827e00640 1 -- 172.21.15.7:0/3271232801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88280bae90 msgr2=0x7f88280b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.219+0000 7f8827e00640 1 --2- 172.21.15.7:0/3271232801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88280bae90 0x7f88280b8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.219+0000 7f8827e00640 1 -- 172.21.15.7:0/3271232801 >> v2:172.21.15.87:3300/0 conn(0x7f882806c570 msgr2=0x7f882806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:28.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.219+0000 7f8827e00640 1 --2- 172.21.15.7:0/3271232801 >> v2:172.21.15.87:3300/0 conn(0x7f882806c570 0x7f882806c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.219+0000 7f8827e00640 1 -- 172.21.15.7:0/3271232801 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8828164120 con 0x7f8828163bb0 2024-04-25T11:14:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.219+0000 7f882cc00640 1 --2- 172.21.15.7:0/3271232801 >> v2:172.21.15.87:3300/0 conn(0x7f882806c570 0x7f882806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.220+0000 7f8827e00640 1 --2- 172.21.15.7:0/3271232801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8828163bb0 0x7f88280d03a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f88280bb720 tx=0x7f881802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c689ff9a040681 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.220+0000 7f8826a00640 1 -- 172.21.15.7:0/3271232801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818037070 con 0x7f8828163bb0 2024-04-25T11:14:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.220+0000 7f8826a00640 1 -- 172.21.15.7:0/3271232801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8818032440 con 0x7f8828163bb0 2024-04-25T11:14:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.220+0000 7f8826a00640 1 -- 172.21.15.7:0/3271232801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818032810 con 0x7f8828163bb0 2024-04-25T11:14:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.221+0000 7f882d600640 1 -- 172.21.15.7:0/3271232801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8828163bb0 msgr2=0x7f88280d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.221+0000 7f882d600640 1 --2- 172.21.15.7:0/3271232801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8828163bb0 0x7f88280d03a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f88280bb720 tx=0x7f881802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.221+0000 7f882d600640 1 -- 172.21.15.7:0/3271232801 shutdown_connections 2024-04-25T11:14:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.221+0000 7f882d600640 1 --2- 172.21.15.7:0/3271232801 >> v2:172.21.15.87:3300/0 conn(0x7f882806c570 0x7f882806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.221+0000 7f882d600640 1 --2- 172.21.15.7:0/3271232801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88280bae90 0x7f88280b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.221+0000 7f882d600640 1 --2- 172.21.15.7:0/3271232801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8828163bb0 0x7f88280d03a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.221+0000 7f882d600640 1 -- 172.21.15.7:0/3271232801 >> 172.21.15.7:0/3271232801 conn(0x7f882806eb00 msgr2=0x7f88280d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:28.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.221+0000 7f882d600640 1 -- 172.21.15.7:0/3271232801 shutdown_connections 2024-04-25T11:14:28.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.221+0000 7f882d600640 1 -- 172.21.15.7:0/3271232801 wait complete. 2024-04-25T11:14:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.223+0000 7f882d600640 1 Processor -- start 2024-04-25T11:14:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.224+0000 7f882d600640 1 -- start start 2024-04-25T11:14:28.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.225+0000 7f882d600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f882806c570 0x7f882814ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:28.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.225+0000 7f882d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88280bae90 0x7f88280caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:28.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.225+0000 7f882d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8828163bb0 0x7f88280cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:28.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.225+0000 7f882d600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88281649c0 con 0x7f8828163bb0 2024-04-25T11:14:28.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.225+0000 7f882d600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f88281646e0 con 0x7f88280bae90 2024-04-25T11:14:28.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.225+0000 7f882d600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f88280d4170 con 0x7f882806c570 2024-04-25T11:14:28.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.225+0000 7f882cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8828163bb0 0x7f88280cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:28.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.225+0000 7f882cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8828163bb0 0x7f88280cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35496/0 (socket says 172.21.15.7:35496) 2024-04-25T11:14:28.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.226+0000 7f882cc00640 1 -- 172.21.15.7:0/3333205417 learned_addr learned my addr 172.21.15.7:0/3333205417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:28.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.226+0000 7f8827400640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88280bae90 0x7f88280caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.226+0000 7f8827e00640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f882806c570 0x7f882814ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.226+0000 7f882cc00640 1 -- 172.21.15.7:0/3333205417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f882806c570 msgr2=0x7f882814ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.226+0000 7f882cc00640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f882806c570 0x7f882814ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.226+0000 7f882cc00640 1 -- 172.21.15.7:0/3333205417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88280bae90 msgr2=0x7f88280caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.226+0000 7f882cc00640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88280bae90 0x7f88280caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.226+0000 7f882cc00640 1 -- 172.21.15.7:0/3333205417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88280cb490 con 0x7f8828163bb0 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.226+0000 7f8827e00640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f882806c570 0x7f882814ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.226+0000 7f8827400640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88280bae90 0x7f88280caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.226+0000 7f882cc00640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8828163bb0 0x7f88280cd670 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f882000cc90 tx=0x7f882000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.227+0000 7f881fe00640 1 -- 172.21.15.7:0/3333205417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882000f680 con 0x7f8828163bb0 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.227+0000 7f881fe00640 1 -- 172.21.15.7:0/3333205417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882000f820 con 0x7f8828163bb0 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.227+0000 7f881fe00640 1 -- 172.21.15.7:0/3333205417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882000fc10 con 0x7f8828163bb0 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.227+0000 7f882d600640 1 -- 172.21.15.7:0/3333205417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88280cb910 con 0x7f8828163bb0 2024-04-25T11:14:28.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.227+0000 7f882d600640 1 -- 172.21.15.7:0/3333205417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88280cce70 con 0x7f8828163bb0 2024-04-25T11:14:28.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.228+0000 7f882d600640 1 -- 172.21.15.7:0/3333205417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88280cb660 con 0x7f8828163bb0 2024-04-25T11:14:28.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.229+0000 7f881fe00640 1 -- 172.21.15.7:0/3333205417 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f88200213f0 con 0x7f8828163bb0 2024-04-25T11:14:28.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.230+0000 7f881fe00640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88080744b0 0x7f88080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:28.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.230+0000 7f881fe00640 1 -- 172.21.15.7:0/3333205417 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f8820096130 con 0x7f8828163bb0 2024-04-25T11:14:28.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.231+0000 7f8827e00640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88080744b0 0x7f88080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:28.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.231+0000 7f8827e00640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88080744b0 0x7f88080769c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8818034000 tx=0x7f881802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:28.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.236+0000 7f881fe00640 1 -- 172.21.15.7:0/3333205417 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8820014030 con 0x7f8828163bb0 2024-04-25T11:14:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.433+0000 7f882d600640 1 -- 172.21.15.7:0/3333205417 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f882814f480 con 0x7f88080744b0 2024-04-25T11:14:28.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.443+0000 7f881fe00640 1 -- 172.21.15.7:0/3333205417 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252009 (secure 0 0 0) 0x7f882814f480 con 0x7f88080744b0 2024-04-25T11:14:28.444 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:28.448 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:14:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.449+0000 7f882d600640 1 -- 172.21.15.7:0/3333205417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88080744b0 msgr2=0x7f88080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.449+0000 7f882d600640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88080744b0 0x7f88080769c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8818034000 tx=0x7f881802e000 comp rx=0 tx=0).stop 2024-04-25T11:14:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.449+0000 7f882d600640 1 -- 172.21.15.7:0/3333205417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8828163bb0 msgr2=0x7f88280cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.449+0000 7f882d600640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8828163bb0 0x7f88280cd670 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f882000cc90 tx=0x7f882000a650 comp rx=0 tx=0).stop 2024-04-25T11:14:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.449+0000 7f882d600640 1 -- 172.21.15.7:0/3333205417 shutdown_connections 2024-04-25T11:14:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.449+0000 7f882d600640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88080744b0 0x7f88080769c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.449+0000 7f882d600640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8828163bb0 0x7f88280cd670 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.449+0000 7f882d600640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88280bae90 0x7f88280caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.449+0000 7f882d600640 1 --2- 172.21.15.7:0/3333205417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f882806c570 0x7f882814ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.449+0000 7f882d600640 1 -- 172.21.15.7:0/3333205417 >> 172.21.15.7:0/3333205417 conn(0x7f882806eb00 msgr2=0x7f88280b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.449+0000 7f882d600640 1 -- 172.21.15.7:0/3333205417 shutdown_connections 2024-04-25T11:14:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:28.450+0000 7f882d600640 1 -- 172.21.15.7:0/3333205417 wait complete. 2024-04-25T11:14:28.475 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-04-25T11:14:27.876603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252864,"kb_used_data":7560,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721856,"statfs":{"total":773094113280,"available":772835180544,"internally_reserved":0,"allocated":7741440,"data_stored":4116836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.577111"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045327+0000","last_change":"2024-04-25T11:14:21.374239+0000","last_active":"2024-04-25T11:14:24.045327+0000","last_peered":"2024-04-25T11:14:24.045327+0000","last_clean":"2024-04-25T11:14:24.045327+0000","last_became_active":"2024-04-25T11:14:21.373912+0000","last_became_peered":"2024-04-25T11:14:21.373912+0000","last_unstale":"2024-04-25T11:14:24.045327+0000","last_undegraded":"2024-04-25T11:14:24.045327+0000","last_fullsized":"2024-04-25T11:14:24.045327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:02.752329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045090+0000","last_change":"2024-04-25T11:14:21.358156+0000","last_active":"2024-04-25T11:14:24.045090+0000","last_peered":"2024-04-25T11:14:24.045090+0000","last_clean":"2024-04-25T11:14:24.045090+0000","last_became_active":"2024-04-25T11:14:21.357825+0000","last_became_peered":"2024-04-25T11:14:21.357825+0000","last_unstale":"2024-04-25T11:14:24.045090+0000","last_undegraded":"2024-04-25T11:14:24.045090+0000","last_fullsized":"2024-04-25T11:14:24.045090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.581113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366418+0000","last_change":"2024-04-25T11:14:21.362112+0000","last_active":"2024-04-25T11:14:22.366418+0000","last_peered":"2024-04-25T11:14:22.366418+0000","last_clean":"2024-04-25T11:14:22.366418+0000","last_became_active":"2024-04-25T11:14:21.361509+0000","last_became_peered":"2024-04-25T11:14:21.361509+0000","last_unstale":"2024-04-25T11:14:22.366418+0000","last_undegraded":"2024-04-25T11:14:22.366418+0000","last_fullsized":"2024-04-25T11:14:22.366418+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:10.199408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.539979+0000","last_change":"2024-04-25T11:14:21.379397+0000","last_active":"2024-04-25T11:14:23.539979+0000","last_peered":"2024-04-25T11:14:23.539979+0000","last_clean":"2024-04-25T11:14:23.539979+0000","last_became_active":"2024-04-25T11:14:21.378997+0000","last_became_peered":"2024-04-25T11:14:21.378997+0000","last_unstale":"2024-04-25T11:14:23.539979+0000","last_undegraded":"2024-04-25T11:14:23.539979+0000","last_fullsized":"2024-04-25T11:14:23.539979+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:56.732074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490210+0000","last_change":"2024-04-25T11:14:21.384178+0000","last_active":"2024-04-25T11:14:23.490210+0000","last_peered":"2024-04-25T11:14:23.490210+0000","last_clean":"2024-04-25T11:14:23.490210+0000","last_became_active":"2024-04-25T11:14:21.383871+0000","last_became_peered":"2024-04-25T11:14:21.383871+0000","last_unstale":"2024-04-25T11:14:23.490210+0000","last_undegraded":"2024-04-25T11:14:23.490210+0000","last_fullsized":"2024-04-25T11:14:23.490210+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:17.347053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962720+0000","last_change":"2024-04-25T11:14:21.381797+0000","last_active":"2024-04-25T11:14:22.962720+0000","last_peered":"2024-04-25T11:14:22.962720+0000","last_clean":"2024-04-25T11:14:22.962720+0000","last_became_active":"2024-04-25T11:14:21.381332+0000","last_became_peered":"2024-04-25T11:14:21.381332+0000","last_unstale":"2024-04-25T11:14:22.962720+0000","last_undegraded":"2024-04-25T11:14:22.962720+0000","last_fullsized":"2024-04-25T11:14:22.962720+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.347162+0000","last_change":"2024-04-25T11:14:21.367266+0000","last_active":"2024-04-25T11:14:23.347162+0000","last_peered":"2024-04-25T11:14:23.347162+0000","last_clean":"2024-04-25T11:14:23.347162+0000","last_became_active":"2024-04-25T11:14:21.366376+0000","last_became_peered":"2024-04-25T11:14:21.366376+0000","last_unstale":"2024-04-25T11:14:23.347162+0000","last_undegraded":"2024-04-25T11:14:23.347162+0000","last_fullsized":"2024-04-25T11:14:23.347162+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:26.828975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.360817+0000","last_change":"2024-04-25T11:14:21.388651+0000","last_active":"2024-04-25T11:14:22.360817+0000","last_peered":"2024-04-25T11:14:22.360817+0000","last_clean":"2024-04-25T11:14:22.360817+0000","last_became_active":"2024-04-25T11:14:21.388262+0000","last_became_peered":"2024-04-25T11:14:21.388262+0000","last_unstale":"2024-04-25T11:14:22.360817+0000","last_undegraded":"2024-04-25T11:14:22.360817+0000","last_fullsized":"2024-04-25T11:14:22.360817+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:02.752329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367959+0000","last_change":"2024-04-25T11:14:21.383853+0000","last_active":"2024-04-25T11:14:22.367959+0000","last_peered":"2024-04-25T11:14:22.367959+0000","last_clean":"2024-04-25T11:14:22.367959+0000","last_became_active":"2024-04-25T11:14:21.383514+0000","last_became_peered":"2024-04-25T11:14:21.383514+0000","last_unstale":"2024-04-25T11:14:22.367959+0000","last_undegraded":"2024-04-25T11:14:22.367959+0000","last_fullsized":"2024-04-25T11:14:22.367959+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:40.752199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367177+0000","last_change":"2024-04-25T11:14:21.375219+0000","last_active":"2024-04-25T11:14:22.367177+0000","last_peered":"2024-04-25T11:14:22.367177+0000","last_clean":"2024-04-25T11:14:22.367177+0000","last_became_active":"2024-04-25T11:14:21.374869+0000","last_became_peered":"2024-04-25T11:14:21.374869+0000","last_unstale":"2024-04-25T11:14:22.367177+0000","last_undegraded":"2024-04-25T11:14:22.367177+0000","last_fullsized":"2024-04-25T11:14:22.367177+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:23.091769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.539810+0000","last_change":"2024-04-25T11:14:21.381440+0000","last_active":"2024-04-25T11:14:23.539810+0000","last_peered":"2024-04-25T11:14:23.539810+0000","last_clean":"2024-04-25T11:14:23.539810+0000","last_became_active":"2024-04-25T11:14:21.381002+0000","last_became_peered":"2024-04-25T11:14:21.381002+0000","last_unstale":"2024-04-25T11:14:23.539810+0000","last_undegraded":"2024-04-25T11:14:23.539810+0000","last_fullsized":"2024-04-25T11:14:23.539810+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:04.017992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361966+0000","last_change":"2024-04-25T11:14:21.374126+0000","last_active":"2024-04-25T11:14:22.361966+0000","last_peered":"2024-04-25T11:14:22.361966+0000","last_clean":"2024-04-25T11:14:22.361966+0000","last_became_active":"2024-04-25T11:14:21.373771+0000","last_became_peered":"2024-04-25T11:14:21.373771+0000","last_unstale":"2024-04-25T11:14:22.361966+0000","last_undegraded":"2024-04-25T11:14:22.361966+0000","last_fullsized":"2024-04-25T11:14:22.361966+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:09.576618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361433+0000","last_change":"2024-04-25T11:14:21.382011+0000","last_active":"2024-04-25T11:14:22.361433+0000","last_peered":"2024-04-25T11:14:22.361433+0000","last_clean":"2024-04-25T11:14:22.361433+0000","last_became_active":"2024-04-25T11:14:21.381691+0000","last_became_peered":"2024-04-25T11:14:21.381691+0000","last_unstale":"2024-04-25T11:14:22.361433+0000","last_undegraded":"2024-04-25T11:14:22.361433+0000","last_fullsized":"2024-04-25T11:14:22.361433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:21.343521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366388+0000","last_change":"2024-04-25T11:14:21.374528+0000","last_active":"2024-04-25T11:14:22.366388+0000","last_peered":"2024-04-25T11:14:22.366388+0000","last_clean":"2024-04-25T11:14:22.366388+0000","last_became_active":"2024-04-25T11:14:21.372240+0000","last_became_peered":"2024-04-25T11:14:21.372240+0000","last_unstale":"2024-04-25T11:14:22.366388+0000","last_undegraded":"2024-04-25T11:14:22.366388+0000","last_fullsized":"2024-04-25T11:14:22.366388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:39.364702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.365670+0000","last_change":"2024-04-25T11:14:21.381099+0000","last_active":"2024-04-25T11:14:22.365670+0000","last_peered":"2024-04-25T11:14:22.365670+0000","last_clean":"2024-04-25T11:14:22.365670+0000","last_became_active":"2024-04-25T11:14:21.380778+0000","last_became_peered":"2024-04-25T11:14:21.380778+0000","last_unstale":"2024-04-25T11:14:22.365670+0000","last_undegraded":"2024-04-25T11:14:22.365670+0000","last_fullsized":"2024-04-25T11:14:22.365670+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:13.479999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346754+0000","last_change":"2024-04-25T11:14:21.374629+0000","last_active":"2024-04-25T11:14:23.346754+0000","last_peered":"2024-04-25T11:14:23.346754+0000","last_clean":"2024-04-25T11:14:23.346754+0000","last_became_active":"2024-04-25T11:14:21.373757+0000","last_became_peered":"2024-04-25T11:14:21.373757+0000","last_unstale":"2024-04-25T11:14:23.346754+0000","last_undegraded":"2024-04-25T11:14:23.346754+0000","last_fullsized":"2024-04-25T11:14:23.346754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:46.489832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045260+0000","last_change":"2024-04-25T11:14:21.384370+0000","last_active":"2024-04-25T11:14:24.045260+0000","last_peered":"2024-04-25T11:14:24.045260+0000","last_clean":"2024-04-25T11:14:24.045260+0000","last_became_active":"2024-04-25T11:14:21.383916+0000","last_became_peered":"2024-04-25T11:14:21.383916+0000","last_unstale":"2024-04-25T11:14:24.045260+0000","last_undegraded":"2024-04-25T11:14:24.045260+0000","last_fullsized":"2024-04-25T11:14:24.045260+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.182058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.539957+0000","last_change":"2024-04-25T11:14:21.380728+0000","last_active":"2024-04-25T11:14:23.539957+0000","last_peered":"2024-04-25T11:14:23.539957+0000","last_clean":"2024-04-25T11:14:23.539957+0000","last_became_active":"2024-04-25T11:14:21.380128+0000","last_became_peered":"2024-04-25T11:14:21.380128+0000","last_unstale":"2024-04-25T11:14:23.539957+0000","last_undegraded":"2024-04-25T11:14:23.539957+0000","last_fullsized":"2024-04-25T11:14:23.539957+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:09.307128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490690+0000","last_change":"2024-04-25T11:14:21.363932+0000","last_active":"2024-04-25T11:14:23.490690+0000","last_peered":"2024-04-25T11:14:23.490690+0000","last_clean":"2024-04-25T11:14:23.490690+0000","last_became_active":"2024-04-25T11:14:21.363589+0000","last_became_peered":"2024-04-25T11:14:21.363589+0000","last_unstale":"2024-04-25T11:14:23.490690+0000","last_undegraded":"2024-04-25T11:14:23.490690+0000","last_fullsized":"2024-04-25T11:14:23.490690+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:19.066276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362235+0000","last_change":"2024-04-25T11:14:21.370707+0000","last_active":"2024-04-25T11:14:22.362235+0000","last_peered":"2024-04-25T11:14:22.362235+0000","last_clean":"2024-04-25T11:14:22.362235+0000","last_became_active":"2024-04-25T11:14:21.370409+0000","last_became_peered":"2024-04-25T11:14:21.370409+0000","last_unstale":"2024-04-25T11:14:22.362235+0000","last_undegraded":"2024-04-25T11:14:22.362235+0000","last_fullsized":"2024-04-25T11:14:22.362235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361908+0000","last_change":"2024-04-25T11:14:21.363401+0000","last_active":"2024-04-25T11:14:22.361908+0000","last_peered":"2024-04-25T11:14:22.361908+0000","last_clean":"2024-04-25T11:14:22.361908+0000","last_became_active":"2024-04-25T11:14:21.363014+0000","last_became_peered":"2024-04-25T11:14:21.363014+0000","last_unstale":"2024-04-25T11:14:22.361908+0000","last_undegraded":"2024-04-25T11:14:22.361908+0000","last_fullsized":"2024-04-25T11:14:22.361908+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:14.240189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362273+0000","last_change":"2024-04-25T11:14:21.380852+0000","last_active":"2024-04-25T11:14:22.362273+0000","last_peered":"2024-04-25T11:14:22.362273+0000","last_clean":"2024-04-25T11:14:22.362273+0000","last_became_active":"2024-04-25T11:14:21.380226+0000","last_became_peered":"2024-04-25T11:14:21.380226+0000","last_unstale":"2024-04-25T11:14:22.362273+0000","last_undegraded":"2024-04-25T11:14:22.362273+0000","last_fullsized":"2024-04-25T11:14:22.362273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:12.837877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361853+0000","last_change":"2024-04-25T11:14:21.365901+0000","last_active":"2024-04-25T11:14:22.361853+0000","last_peered":"2024-04-25T11:14:22.361853+0000","last_clean":"2024-04-25T11:14:22.361853+0000","last_became_active":"2024-04-25T11:14:21.365551+0000","last_became_peered":"2024-04-25T11:14:21.365551+0000","last_unstale":"2024-04-25T11:14:22.361853+0000","last_undegraded":"2024-04-25T11:14:22.361853+0000","last_fullsized":"2024-04-25T11:14:22.361853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:42.952154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.365456+0000","last_change":"2024-04-25T11:14:21.364597+0000","last_active":"2024-04-25T11:14:22.365456+0000","last_peered":"2024-04-25T11:14:22.365456+0000","last_clean":"2024-04-25T11:14:22.365456+0000","last_became_active":"2024-04-25T11:14:21.363451+0000","last_became_peered":"2024-04-25T11:14:21.363451+0000","last_unstale":"2024-04-25T11:14:22.365456+0000","last_undegraded":"2024-04-25T11:14:22.365456+0000","last_fullsized":"2024-04-25T11:14:22.365456+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:09.576618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362664+0000","last_change":"2024-04-25T11:14:21.368883+0000","last_active":"2024-04-25T11:14:22.362664+0000","last_peered":"2024-04-25T11:14:22.362664+0000","last_clean":"2024-04-25T11:14:22.362664+0000","last_became_active":"2024-04-25T11:14:21.368181+0000","last_became_peered":"2024-04-25T11:14:21.368181+0000","last_unstale":"2024-04-25T11:14:22.362664+0000","last_undegraded":"2024-04-25T11:14:22.362664+0000","last_fullsized":"2024-04-25T11:14:22.362664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:23.981365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362255+0000","last_change":"2024-04-25T11:14:21.373674+0000","last_active":"2024-04-25T11:14:22.362255+0000","last_peered":"2024-04-25T11:14:22.362255+0000","last_clean":"2024-04-25T11:14:22.362255+0000","last_became_active":"2024-04-25T11:14:21.373331+0000","last_became_peered":"2024-04-25T11:14:21.373331+0000","last_unstale":"2024-04-25T11:14:22.362255+0000","last_undegraded":"2024-04-25T11:14:22.362255+0000","last_fullsized":"2024-04-25T11:14:22.362255+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:13.479999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.539982+0000","last_change":"2024-04-25T11:14:21.377004+0000","last_active":"2024-04-25T11:14:23.539982+0000","last_peered":"2024-04-25T11:14:23.539982+0000","last_clean":"2024-04-25T11:14:23.539982+0000","last_became_active":"2024-04-25T11:14:21.376617+0000","last_became_peered":"2024-04-25T11:14:21.376617+0000","last_unstale":"2024-04-25T11:14:23.539982+0000","last_undegraded":"2024-04-25T11:14:23.539982+0000","last_fullsized":"2024-04-25T11:14:23.539982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:53.111220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044519+0000","last_change":"2024-04-25T11:14:21.371250+0000","last_active":"2024-04-25T11:14:24.044519+0000","last_peered":"2024-04-25T11:14:24.044519+0000","last_clean":"2024-04-25T11:14:24.044519+0000","last_became_active":"2024-04-25T11:14:21.370864+0000","last_became_peered":"2024-04-25T11:14:21.370864+0000","last_unstale":"2024-04-25T11:14:24.044519+0000","last_undegraded":"2024-04-25T11:14:24.044519+0000","last_fullsized":"2024-04-25T11:14:24.044519+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:09.063896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366579+0000","last_change":"2024-04-25T11:14:21.384753+0000","last_active":"2024-04-25T11:14:22.366579+0000","last_peered":"2024-04-25T11:14:22.366579+0000","last_clean":"2024-04-25T11:14:22.366579+0000","last_became_active":"2024-04-25T11:14:21.384449+0000","last_became_peered":"2024-04-25T11:14:21.384449+0000","last_unstale":"2024-04-25T11:14:22.366579+0000","last_undegraded":"2024-04-25T11:14:22.366579+0000","last_fullsized":"2024-04-25T11:14:22.366579+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:07.931905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361209+0000","last_change":"2024-04-25T11:14:21.383853+0000","last_active":"2024-04-25T11:14:22.361209+0000","last_peered":"2024-04-25T11:14:22.361209+0000","last_clean":"2024-04-25T11:14:22.361209+0000","last_became_active":"2024-04-25T11:14:21.383527+0000","last_became_peered":"2024-04-25T11:14:21.383527+0000","last_unstale":"2024-04-25T11:14:22.361209+0000","last_undegraded":"2024-04-25T11:14:22.361209+0000","last_fullsized":"2024-04-25T11:14:22.361209+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.742186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.538857+0000","last_change":"2024-04-25T11:14:21.370126+0000","last_active":"2024-04-25T11:14:23.538857+0000","last_peered":"2024-04-25T11:14:23.538857+0000","last_clean":"2024-04-25T11:14:23.538857+0000","last_became_active":"2024-04-25T11:14:21.369727+0000","last_became_peered":"2024-04-25T11:14:21.369727+0000","last_unstale":"2024-04-25T11:14:23.538857+0000","last_undegraded":"2024-04-25T11:14:23.538857+0000","last_fullsized":"2024-04-25T11:14:23.538857+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:17.010334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961814+0000","last_change":"2024-04-25T11:14:21.360674+0000","last_active":"2024-04-25T11:14:22.961814+0000","last_peered":"2024-04-25T11:14:22.961814+0000","last_clean":"2024-04-25T11:14:22.961814+0000","last_became_active":"2024-04-25T11:14:21.359863+0000","last_became_peered":"2024-04-25T11:14:21.359863+0000","last_unstale":"2024-04-25T11:14:22.961814+0000","last_undegraded":"2024-04-25T11:14:22.961814+0000","last_fullsized":"2024-04-25T11:14:22.961814+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:49.430307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490947+0000","last_change":"2024-04-25T11:14:21.360138+0000","last_active":"2024-04-25T11:14:23.490947+0000","last_peered":"2024-04-25T11:14:23.490947+0000","last_clean":"2024-04-25T11:14:23.490947+0000","last_became_active":"2024-04-25T11:14:21.359803+0000","last_became_peered":"2024-04-25T11:14:21.359803+0000","last_unstale":"2024-04-25T11:14:23.490947+0000","last_undegraded":"2024-04-25T11:14:23.490947+0000","last_fullsized":"2024-04-25T11:14:23.490947+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:20.532234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"67'4","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045418+0000","last_change":"2024-04-25T11:14:20.334352+0000","last_active":"2024-04-25T11:14:24.045418+0000","last_peered":"2024-04-25T11:14:24.045418+0000","last_clean":"2024-04-25T11:14:24.045418+0000","last_became_active":"2024-04-25T11:14:20.329783+0000","last_became_peered":"2024-04-25T11:14:20.329783+0000","last_unstale":"2024-04-25T11:14:24.045418+0000","last_undegraded":"2024-04-25T11:14:24.045418+0000","last_fullsized":"2024-04-25T11:14:24.045418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:55.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.540403+0000","last_change":"2024-04-25T11:14:21.378061+0000","last_active":"2024-04-25T11:14:23.540403+0000","last_peered":"2024-04-25T11:14:23.540403+0000","last_clean":"2024-04-25T11:14:23.540403+0000","last_became_active":"2024-04-25T11:14:21.377684+0000","last_became_peered":"2024-04-25T11:14:21.377684+0000","last_unstale":"2024-04-25T11:14:23.540403+0000","last_undegraded":"2024-04-25T11:14:23.540403+0000","last_fullsized":"2024-04-25T11:14:23.540403+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:22.490881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962745+0000","last_change":"2024-04-25T11:14:20.346713+0000","last_active":"2024-04-25T11:14:22.962745+0000","last_peered":"2024-04-25T11:14:22.962745+0000","last_clean":"2024-04-25T11:14:22.962745+0000","last_became_active":"2024-04-25T11:14:20.346320+0000","last_became_peered":"2024-04-25T11:14:20.346320+0000","last_unstale":"2024-04-25T11:14:22.962745+0000","last_undegraded":"2024-04-25T11:14:22.962745+0000","last_fullsized":"2024-04-25T11:14:22.962745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.559248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962535+0000","last_change":"2024-04-25T11:14:21.356785+0000","last_active":"2024-04-25T11:14:22.962535+0000","last_peered":"2024-04-25T11:14:22.962535+0000","last_clean":"2024-04-25T11:14:22.962535+0000","last_became_active":"2024-04-25T11:14:21.356335+0000","last_became_peered":"2024-04-25T11:14:21.356335+0000","last_unstale":"2024-04-25T11:14:22.962535+0000","last_undegraded":"2024-04-25T11:14:22.962535+0000","last_fullsized":"2024-04-25T11:14:22.962535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.363496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962440+0000","last_change":"2024-04-25T11:14:20.346384+0000","last_active":"2024-04-25T11:14:22.962440+0000","last_peered":"2024-04-25T11:14:22.962440+0000","last_clean":"2024-04-25T11:14:22.962440+0000","last_became_active":"2024-04-25T11:14:20.346031+0000","last_became_peered":"2024-04-25T11:14:20.346031+0000","last_unstale":"2024-04-25T11:14:22.962440+0000","last_undegraded":"2024-04-25T11:14:22.962440+0000","last_fullsized":"2024-04-25T11:14:22.962440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:22.963995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367664+0000","last_change":"2024-04-25T11:14:21.375789+0000","last_active":"2024-04-25T11:14:22.367664+0000","last_peered":"2024-04-25T11:14:22.367664+0000","last_clean":"2024-04-25T11:14:22.367664+0000","last_became_active":"2024-04-25T11:14:21.375492+0000","last_became_peered":"2024-04-25T11:14:21.375492+0000","last_unstale":"2024-04-25T11:14:22.367664+0000","last_undegraded":"2024-04-25T11:14:22.367664+0000","last_fullsized":"2024-04-25T11:14:22.367664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:46.640556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962159+0000","last_change":"2024-04-25T11:14:20.337082+0000","last_active":"2024-04-25T11:14:22.962159+0000","last_peered":"2024-04-25T11:14:22.962159+0000","last_clean":"2024-04-25T11:14:22.962159+0000","last_became_active":"2024-04-25T11:14:20.336766+0000","last_became_peered":"2024-04-25T11:14:20.336766+0000","last_unstale":"2024-04-25T11:14:22.962159+0000","last_undegraded":"2024-04-25T11:14:22.962159+0000","last_fullsized":"2024-04-25T11:14:22.962159+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:12.462630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346829+0000","last_change":"2024-04-25T11:14:21.346395+0000","last_active":"2024-04-25T11:14:23.346829+0000","last_peered":"2024-04-25T11:14:23.346829+0000","last_clean":"2024-04-25T11:14:23.346829+0000","last_became_active":"2024-04-25T11:14:21.345933+0000","last_became_peered":"2024-04-25T11:14:21.345933+0000","last_unstale":"2024-04-25T11:14:23.346829+0000","last_undegraded":"2024-04-25T11:14:23.346829+0000","last_fullsized":"2024-04-25T11:14:23.346829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.002374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362073+0000","last_change":"2024-04-25T11:14:20.335533+0000","last_active":"2024-04-25T11:14:22.362073+0000","last_peered":"2024-04-25T11:14:22.362073+0000","last_clean":"2024-04-25T11:14:22.362073+0000","last_became_active":"2024-04-25T11:14:20.335228+0000","last_became_peered":"2024-04-25T11:14:20.335228+0000","last_unstale":"2024-04-25T11:14:22.362073+0000","last_undegraded":"2024-04-25T11:14:22.362073+0000","last_fullsized":"2024-04-25T11:14:22.362073+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.347332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044109+0000","last_change":"2024-04-25T11:14:21.384939+0000","last_active":"2024-04-25T11:14:24.044109+0000","last_peered":"2024-04-25T11:14:24.044109+0000","last_clean":"2024-04-25T11:14:24.044109+0000","last_became_active":"2024-04-25T11:14:21.384525+0000","last_became_peered":"2024-04-25T11:14:21.384525+0000","last_unstale":"2024-04-25T11:14:24.044109+0000","last_undegraded":"2024-04-25T11:14:24.044109+0000","last_fullsized":"2024-04-25T11:14:24.044109+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:19.066276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361973+0000","last_change":"2024-04-25T11:14:20.317044+0000","last_active":"2024-04-25T11:14:22.361973+0000","last_peered":"2024-04-25T11:14:22.361973+0000","last_clean":"2024-04-25T11:14:22.361973+0000","last_became_active":"2024-04-25T11:14:20.316634+0000","last_became_peered":"2024-04-25T11:14:20.316634+0000","last_unstale":"2024-04-25T11:14:22.361973+0000","last_undegraded":"2024-04-25T11:14:22.361973+0000","last_fullsized":"2024-04-25T11:14:22.361973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:12.462630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490055+0000","last_change":"2024-04-25T11:14:21.372187+0000","last_active":"2024-04-25T11:14:23.490055+0000","last_peered":"2024-04-25T11:14:23.490055+0000","last_clean":"2024-04-25T11:14:23.490055+0000","last_became_active":"2024-04-25T11:14:21.370797+0000","last_became_peered":"2024-04-25T11:14:21.370797+0000","last_unstale":"2024-04-25T11:14:23.490055+0000","last_undegraded":"2024-04-25T11:14:23.490055+0000","last_fullsized":"2024-04-25T11:14:23.490055+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:11.224114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361960+0000","last_change":"2024-04-25T11:14:20.339374+0000","last_active":"2024-04-25T11:14:22.361960+0000","last_peered":"2024-04-25T11:14:22.361960+0000","last_clean":"2024-04-25T11:14:22.361960+0000","last_became_active":"2024-04-25T11:14:20.339026+0000","last_became_peered":"2024-04-25T11:14:20.339026+0000","last_unstale":"2024-04-25T11:14:22.361960+0000","last_undegraded":"2024-04-25T11:14:22.361960+0000","last_fullsized":"2024-04-25T11:14:22.361960+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:09.182038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362009+0000","last_change":"2024-04-25T11:14:21.380186+0000","last_active":"2024-04-25T11:14:22.362009+0000","last_peered":"2024-04-25T11:14:22.362009+0000","last_clean":"2024-04-25T11:14:22.362009+0000","last_became_active":"2024-04-25T11:14:21.379835+0000","last_became_peered":"2024-04-25T11:14:21.379835+0000","last_unstale":"2024-04-25T11:14:22.362009+0000","last_undegraded":"2024-04-25T11:14:22.362009+0000","last_fullsized":"2024-04-25T11:14:22.362009+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:25.039608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490006+0000","last_change":"2024-04-25T11:14:20.344908+0000","last_active":"2024-04-25T11:14:23.490006+0000","last_peered":"2024-04-25T11:14:23.490006+0000","last_clean":"2024-04-25T11:14:23.490006+0000","last_became_active":"2024-04-25T11:14:20.344554+0000","last_became_peered":"2024-04-25T11:14:20.344554+0000","last_unstale":"2024-04-25T11:14:23.490006+0000","last_undegraded":"2024-04-25T11:14:23.490006+0000","last_fullsized":"2024-04-25T11:14:23.490006+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.170244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044489+0000","last_change":"2024-04-25T11:14:21.352169+0000","last_active":"2024-04-25T11:14:24.044489+0000","last_peered":"2024-04-25T11:14:24.044489+0000","last_clean":"2024-04-25T11:14:24.044489+0000","last_became_active":"2024-04-25T11:14:21.351353+0000","last_became_peered":"2024-04-25T11:14:21.351353+0000","last_unstale":"2024-04-25T11:14:24.044489+0000","last_undegraded":"2024-04-25T11:14:24.044489+0000","last_fullsized":"2024-04-25T11:14:24.044489+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:40.169340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367869+0000","last_change":"2024-04-25T11:14:20.330643+0000","last_active":"2024-04-25T11:14:22.367869+0000","last_peered":"2024-04-25T11:14:22.367869+0000","last_clean":"2024-04-25T11:14:22.367869+0000","last_became_active":"2024-04-25T11:14:20.330199+0000","last_became_peered":"2024-04-25T11:14:20.330199+0000","last_unstale":"2024-04-25T11:14:22.367869+0000","last_undegraded":"2024-04-25T11:14:22.367869+0000","last_fullsized":"2024-04-25T11:14:22.367869+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:48.089116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362035+0000","last_change":"2024-04-25T11:14:21.358807+0000","last_active":"2024-04-25T11:14:22.362035+0000","last_peered":"2024-04-25T11:14:22.362035+0000","last_clean":"2024-04-25T11:14:22.362035+0000","last_became_active":"2024-04-25T11:14:21.358476+0000","last_became_peered":"2024-04-25T11:14:21.358476+0000","last_unstale":"2024-04-25T11:14:22.362035+0000","last_undegraded":"2024-04-25T11:14:22.362035+0000","last_fullsized":"2024-04-25T11:14:22.362035+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.363496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.540476+0000","last_change":"2024-04-25T11:14:20.344529+0000","last_active":"2024-04-25T11:14:23.540476+0000","last_peered":"2024-04-25T11:14:23.540476+0000","last_clean":"2024-04-25T11:14:23.540476+0000","last_became_active":"2024-04-25T11:14:20.344184+0000","last_became_peered":"2024-04-25T11:14:20.344184+0000","last_unstale":"2024-04-25T11:14:23.540476+0000","last_undegraded":"2024-04-25T11:14:23.540476+0000","last_fullsized":"2024-04-25T11:14:23.540476+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:12.962009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.491302+0000","last_change":"2024-04-25T11:14:21.372160+0000","last_active":"2024-04-25T11:14:23.491302+0000","last_peered":"2024-04-25T11:14:23.491302+0000","last_clean":"2024-04-25T11:14:23.491302+0000","last_became_active":"2024-04-25T11:14:21.371743+0000","last_became_peered":"2024-04-25T11:14:21.371743+0000","last_unstale":"2024-04-25T11:14:23.491302+0000","last_undegraded":"2024-04-25T11:14:23.491302+0000","last_fullsized":"2024-04-25T11:14:23.491302+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.581113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"67'3","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366722+0000","last_change":"2024-04-25T11:14:20.337503+0000","last_active":"2024-04-25T11:14:22.366722+0000","last_peered":"2024-04-25T11:14:22.366722+0000","last_clean":"2024-04-25T11:14:22.366722+0000","last_became_active":"2024-04-25T11:14:20.337171+0000","last_became_peered":"2024-04-25T11:14:20.337171+0000","last_unstale":"2024-04-25T11:14:22.366722+0000","last_undegraded":"2024-04-25T11:14:22.366722+0000","last_fullsized":"2024-04-25T11:14:22.366722+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:07.071825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.540457+0000","last_change":"2024-04-25T11:14:21.370932+0000","last_active":"2024-04-25T11:14:23.540457+0000","last_peered":"2024-04-25T11:14:23.540457+0000","last_clean":"2024-04-25T11:14:23.540457+0000","last_became_active":"2024-04-25T11:14:21.370600+0000","last_became_peered":"2024-04-25T11:14:21.370600+0000","last_unstale":"2024-04-25T11:14:23.540457+0000","last_undegraded":"2024-04-25T11:14:23.540457+0000","last_fullsized":"2024-04-25T11:14:23.540457+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:33.222542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"67'6","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044599+0000","last_change":"2024-04-25T11:14:20.343544+0000","last_active":"2024-04-25T11:14:24.044599+0000","last_peered":"2024-04-25T11:14:24.044599+0000","last_clean":"2024-04-25T11:14:24.044599+0000","last_became_active":"2024-04-25T11:14:20.343205+0000","last_became_peered":"2024-04-25T11:14:20.343205+0000","last_unstale":"2024-04-25T11:14:24.044599+0000","last_undegraded":"2024-04-25T11:14:24.044599+0000","last_fullsized":"2024-04-25T11:14:24.044599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:03.224076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.538721+0000","last_change":"2024-04-25T11:14:21.390400+0000","last_active":"2024-04-25T11:14:23.538721+0000","last_peered":"2024-04-25T11:14:23.538721+0000","last_clean":"2024-04-25T11:14:23.538721+0000","last_became_active":"2024-04-25T11:14:21.389956+0000","last_became_peered":"2024-04-25T11:14:21.389956+0000","last_unstale":"2024-04-25T11:14:23.538721+0000","last_undegraded":"2024-04-25T11:14:23.538721+0000","last_fullsized":"2024-04-25T11:14:23.538721+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:50.079030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366706+0000","last_change":"2024-04-25T11:14:20.339359+0000","last_active":"2024-04-25T11:14:22.366706+0000","last_peered":"2024-04-25T11:14:22.366706+0000","last_clean":"2024-04-25T11:14:22.366706+0000","last_became_active":"2024-04-25T11:14:20.338755+0000","last_became_peered":"2024-04-25T11:14:20.338755+0000","last_unstale":"2024-04-25T11:14:22.366706+0000","last_undegraded":"2024-04-25T11:14:22.366706+0000","last_fullsized":"2024-04-25T11:14:22.366706+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:16.329683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361579+0000","last_change":"2024-04-25T11:14:21.357141+0000","last_active":"2024-04-25T11:14:22.361579+0000","last_peered":"2024-04-25T11:14:22.361579+0000","last_clean":"2024-04-25T11:14:22.361579+0000","last_became_active":"2024-04-25T11:14:21.356725+0000","last_became_peered":"2024-04-25T11:14:21.356725+0000","last_unstale":"2024-04-25T11:14:22.361579+0000","last_undegraded":"2024-04-25T11:14:22.361579+0000","last_fullsized":"2024-04-25T11:14:22.361579+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.740665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361050+0000","last_change":"2024-04-25T11:14:20.333436+0000","last_active":"2024-04-25T11:14:22.361050+0000","last_peered":"2024-04-25T11:14:22.361050+0000","last_clean":"2024-04-25T11:14:22.361050+0000","last_became_active":"2024-04-25T11:14:20.333048+0000","last_became_peered":"2024-04-25T11:14:20.333048+0000","last_unstale":"2024-04-25T11:14:22.361050+0000","last_undegraded":"2024-04-25T11:14:22.361050+0000","last_fullsized":"2024-04-25T11:14:22.361050+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:08.289759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490394+0000","last_change":"2024-04-25T11:14:21.370711+0000","last_active":"2024-04-25T11:14:23.490394+0000","last_peered":"2024-04-25T11:14:23.490394+0000","last_clean":"2024-04-25T11:14:23.490394+0000","last_became_active":"2024-04-25T11:14:21.369950+0000","last_became_peered":"2024-04-25T11:14:21.369950+0000","last_unstale":"2024-04-25T11:14:23.490394+0000","last_undegraded":"2024-04-25T11:14:23.490394+0000","last_fullsized":"2024-04-25T11:14:23.490394+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:08.089195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.365980+0000","last_change":"2024-04-25T11:14:20.331830+0000","last_active":"2024-04-25T11:14:22.365980+0000","last_peered":"2024-04-25T11:14:22.365980+0000","last_clean":"2024-04-25T11:14:22.365980+0000","last_became_active":"2024-04-25T11:14:20.331453+0000","last_became_peered":"2024-04-25T11:14:20.331453+0000","last_unstale":"2024-04-25T11:14:22.365980+0000","last_undegraded":"2024-04-25T11:14:22.365980+0000","last_fullsized":"2024-04-25T11:14:22.365980+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:19.514865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044411+0000","last_change":"2024-04-25T11:14:21.365796+0000","last_active":"2024-04-25T11:14:24.044411+0000","last_peered":"2024-04-25T11:14:24.044411+0000","last_clean":"2024-04-25T11:14:24.044411+0000","last_became_active":"2024-04-25T11:14:21.364748+0000","last_became_peered":"2024-04-25T11:14:21.364748+0000","last_unstale":"2024-04-25T11:14:24.044411+0000","last_undegraded":"2024-04-25T11:14:24.044411+0000","last_fullsized":"2024-04-25T11:14:24.044411+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:20.532234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"67'8","reported_seq":22,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.538655+0000","last_change":"2024-04-25T11:14:20.339581+0000","last_active":"2024-04-25T11:14:23.538655+0000","last_peered":"2024-04-25T11:14:23.538655+0000","last_clean":"2024-04-25T11:14:23.538655+0000","last_became_active":"2024-04-25T11:14:20.339208+0000","last_became_peered":"2024-04-25T11:14:20.339208+0000","last_unstale":"2024-04-25T11:14:23.538655+0000","last_undegraded":"2024-04-25T11:14:23.538655+0000","last_fullsized":"2024-04-25T11:14:23.538655+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:40.376640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366146+0000","last_change":"2024-04-25T11:14:21.384873+0000","last_active":"2024-04-25T11:14:22.366146+0000","last_peered":"2024-04-25T11:14:22.366146+0000","last_clean":"2024-04-25T11:14:22.366146+0000","last_became_active":"2024-04-25T11:14:21.383780+0000","last_became_peered":"2024-04-25T11:14:21.383780+0000","last_unstale":"2024-04-25T11:14:22.366146+0000","last_undegraded":"2024-04-25T11:14:22.366146+0000","last_fullsized":"2024-04-25T11:14:22.366146+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:23.981365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362950+0000","last_change":"2024-04-25T11:14:20.344787+0000","last_active":"2024-04-25T11:14:22.362950+0000","last_peered":"2024-04-25T11:14:22.362950+0000","last_clean":"2024-04-25T11:14:22.362950+0000","last_became_active":"2024-04-25T11:14:20.343403+0000","last_became_peered":"2024-04-25T11:14:20.343403+0000","last_unstale":"2024-04-25T11:14:22.362950+0000","last_undegraded":"2024-04-25T11:14:22.362950+0000","last_fullsized":"2024-04-25T11:14:22.362950+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:32.205173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490888+0000","last_change":"2024-04-25T11:14:21.383372+0000","last_active":"2024-04-25T11:14:23.490888+0000","last_peered":"2024-04-25T11:14:23.490888+0000","last_clean":"2024-04-25T11:14:23.490888+0000","last_became_active":"2024-04-25T11:14:21.383058+0000","last_became_peered":"2024-04-25T11:14:21.383058+0000","last_unstale":"2024-04-25T11:14:23.490888+0000","last_undegraded":"2024-04-25T11:14:23.490888+0000","last_fullsized":"2024-04-25T11:14:23.490888+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:49.106485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367149+0000","last_change":"2024-04-25T11:14:20.342160+0000","last_active":"2024-04-25T11:14:22.367149+0000","last_peered":"2024-04-25T11:14:22.367149+0000","last_clean":"2024-04-25T11:14:22.367149+0000","last_became_active":"2024-04-25T11:14:20.341788+0000","last_became_peered":"2024-04-25T11:14:20.341788+0000","last_unstale":"2024-04-25T11:14:22.367149+0000","last_undegraded":"2024-04-25T11:14:22.367149+0000","last_fullsized":"2024-04-25T11:14:22.367149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:18.048907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.540266+0000","last_change":"2024-04-25T11:14:21.380455+0000","last_active":"2024-04-25T11:14:23.540266+0000","last_peered":"2024-04-25T11:14:23.540266+0000","last_clean":"2024-04-25T11:14:23.540266+0000","last_became_active":"2024-04-25T11:14:21.379717+0000","last_became_peered":"2024-04-25T11:14:21.379717+0000","last_unstale":"2024-04-25T11:14:23.540266+0000","last_undegraded":"2024-04-25T11:14:23.540266+0000","last_fullsized":"2024-04-25T11:14:23.540266+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:23.080677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367001+0000","last_change":"2024-04-25T11:14:20.341298+0000","last_active":"2024-04-25T11:14:22.367001+0000","last_peered":"2024-04-25T11:14:22.367001+0000","last_clean":"2024-04-25T11:14:22.367001+0000","last_became_active":"2024-04-25T11:14:20.340803+0000","last_became_peered":"2024-04-25T11:14:20.340803+0000","last_unstale":"2024-04-25T11:14:22.367001+0000","last_undegraded":"2024-04-25T11:14:22.367001+0000","last_fullsized":"2024-04-25T11:14:22.367001+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:24.571848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.491062+0000","last_change":"2024-04-25T11:14:21.367727+0000","last_active":"2024-04-25T11:14:23.491062+0000","last_peered":"2024-04-25T11:14:23.491062+0000","last_clean":"2024-04-25T11:14:23.491062+0000","last_became_active":"2024-04-25T11:14:21.367327+0000","last_became_peered":"2024-04-25T11:14:21.367327+0000","last_unstale":"2024-04-25T11:14:23.491062+0000","last_undegraded":"2024-04-25T11:14:23.491062+0000","last_fullsized":"2024-04-25T11:14:23.491062+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:40.169340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362338+0000","last_change":"2024-04-25T11:14:20.341606+0000","last_active":"2024-04-25T11:14:22.362338+0000","last_peered":"2024-04-25T11:14:22.362338+0000","last_clean":"2024-04-25T11:14:22.362338+0000","last_became_active":"2024-04-25T11:14:20.341220+0000","last_became_peered":"2024-04-25T11:14:20.341220+0000","last_unstale":"2024-04-25T11:14:22.362338+0000","last_undegraded":"2024-04-25T11:14:22.362338+0000","last_fullsized":"2024-04-25T11:14:22.362338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:13.222820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961827+0000","last_change":"2024-04-25T11:14:21.359973+0000","last_active":"2024-04-25T11:14:22.961827+0000","last_peered":"2024-04-25T11:14:22.961827+0000","last_clean":"2024-04-25T11:14:22.961827+0000","last_became_active":"2024-04-25T11:14:21.359599+0000","last_became_peered":"2024-04-25T11:14:21.359599+0000","last_unstale":"2024-04-25T11:14:22.961827+0000","last_undegraded":"2024-04-25T11:14:22.961827+0000","last_fullsized":"2024-04-25T11:14:22.961827+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:42.952154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961999+0000","last_change":"2024-04-25T11:14:20.341545+0000","last_active":"2024-04-25T11:14:22.961999+0000","last_peered":"2024-04-25T11:14:22.961999+0000","last_clean":"2024-04-25T11:14:22.961999+0000","last_became_active":"2024-04-25T11:14:20.341206+0000","last_became_peered":"2024-04-25T11:14:20.341206+0000","last_unstale":"2024-04-25T11:14:22.961999+0000","last_undegraded":"2024-04-25T11:14:22.961999+0000","last_fullsized":"2024-04-25T11:14:22.961999+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:13.222820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361215+0000","last_change":"2024-04-25T11:14:21.367533+0000","last_active":"2024-04-25T11:14:22.361215+0000","last_peered":"2024-04-25T11:14:22.361215+0000","last_clean":"2024-04-25T11:14:22.361215+0000","last_became_active":"2024-04-25T11:14:21.367177+0000","last_became_peered":"2024-04-25T11:14:21.367177+0000","last_unstale":"2024-04-25T11:14:22.361215+0000","last_undegraded":"2024-04-25T11:14:22.361215+0000","last_fullsized":"2024-04-25T11:14:22.361215+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:57.447950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361409+0000","last_change":"2024-04-25T11:14:20.341156+0000","last_active":"2024-04-25T11:14:22.361409+0000","last_peered":"2024-04-25T11:14:22.361409+0000","last_clean":"2024-04-25T11:14:22.361409+0000","last_became_active":"2024-04-25T11:14:20.340815+0000","last_became_peered":"2024-04-25T11:14:20.340815+0000","last_unstale":"2024-04-25T11:14:22.361409+0000","last_undegraded":"2024-04-25T11:14:22.361409+0000","last_fullsized":"2024-04-25T11:14:22.361409+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:22.063307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490616+0000","last_change":"2024-04-25T11:14:21.370991+0000","last_active":"2024-04-25T11:14:23.490616+0000","last_peered":"2024-04-25T11:14:23.490616+0000","last_clean":"2024-04-25T11:14:23.490616+0000","last_became_active":"2024-04-25T11:14:21.370401+0000","last_became_peered":"2024-04-25T11:14:21.370401+0000","last_unstale":"2024-04-25T11:14:23.490616+0000","last_undegraded":"2024-04-25T11:14:23.490616+0000","last_fullsized":"2024-04-25T11:14:23.490616+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:33.746782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961780+0000","last_change":"2024-04-25T11:14:20.345177+0000","last_active":"2024-04-25T11:14:22.961780+0000","last_peered":"2024-04-25T11:14:22.961780+0000","last_clean":"2024-04-25T11:14:22.961780+0000","last_became_active":"2024-04-25T11:14:20.344632+0000","last_became_peered":"2024-04-25T11:14:20.344632+0000","last_unstale":"2024-04-25T11:14:22.961780+0000","last_undegraded":"2024-04-25T11:14:22.961780+0000","last_fullsized":"2024-04-25T11:14:22.961780+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:58.724816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962522+0000","last_change":"2024-04-25T11:14:21.380957+0000","last_active":"2024-04-25T11:14:22.962522+0000","last_peered":"2024-04-25T11:14:22.962522+0000","last_clean":"2024-04-25T11:14:22.962522+0000","last_became_active":"2024-04-25T11:14:21.380702+0000","last_became_peered":"2024-04-25T11:14:21.380702+0000","last_unstale":"2024-04-25T11:14:22.962522+0000","last_undegraded":"2024-04-25T11:14:22.962522+0000","last_fullsized":"2024-04-25T11:14:22.962522+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:25.039608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.363401+0000","last_change":"2024-04-25T11:14:20.343699+0000","last_active":"2024-04-25T11:14:22.363401+0000","last_peered":"2024-04-25T11:14:22.363401+0000","last_clean":"2024-04-25T11:14:22.363401+0000","last_became_active":"2024-04-25T11:14:20.343351+0000","last_became_peered":"2024-04-25T11:14:20.343351+0000","last_unstale":"2024-04-25T11:14:22.363401+0000","last_undegraded":"2024-04-25T11:14:22.363401+0000","last_fullsized":"2024-04-25T11:14:22.363401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:10.206744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.347275+0000","last_change":"2024-04-25T11:14:21.355386+0000","last_active":"2024-04-25T11:14:23.347275+0000","last_peered":"2024-04-25T11:14:23.347275+0000","last_clean":"2024-04-25T11:14:23.347275+0000","last_became_active":"2024-04-25T11:14:21.355092+0000","last_became_peered":"2024-04-25T11:14:21.355092+0000","last_unstale":"2024-04-25T11:14:23.347275+0000","last_undegraded":"2024-04-25T11:14:23.347275+0000","last_fullsized":"2024-04-25T11:14:23.347275+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:55.946442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"68'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.363188+0000","last_change":"2024-04-25T11:14:20.336075+0000","last_active":"2024-04-25T11:14:22.363188+0000","last_peered":"2024-04-25T11:14:22.363188+0000","last_clean":"2024-04-25T11:14:22.363188+0000","last_became_active":"2024-04-25T11:14:20.335712+0000","last_became_peered":"2024-04-25T11:14:20.335712+0000","last_unstale":"2024-04-25T11:14:22.363188+0000","last_undegraded":"2024-04-25T11:14:22.363188+0000","last_fullsized":"2024-04-25T11:14:22.363188+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:21.473511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362716+0000","last_change":"2024-04-25T11:14:21.364681+0000","last_active":"2024-04-25T11:14:22.362716+0000","last_peered":"2024-04-25T11:14:22.362716+0000","last_clean":"2024-04-25T11:14:22.362716+0000","last_became_active":"2024-04-25T11:14:21.364334+0000","last_became_peered":"2024-04-25T11:14:21.364334+0000","last_unstale":"2024-04-25T11:14:22.362716+0000","last_undegraded":"2024-04-25T11:14:22.362716+0000","last_fullsized":"2024-04-25T11:14:22.362716+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:39.364702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962444+0000","last_change":"2024-04-25T11:14:20.346848+0000","last_active":"2024-04-25T11:14:22.962444+0000","last_peered":"2024-04-25T11:14:22.962444+0000","last_clean":"2024-04-25T11:14:22.962444+0000","last_became_active":"2024-04-25T11:14:20.346439+0000","last_became_peered":"2024-04-25T11:14:20.346439+0000","last_unstale":"2024-04-25T11:14:22.962444+0000","last_undegraded":"2024-04-25T11:14:22.962444+0000","last_fullsized":"2024-04-25T11:14:22.962444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.347332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.491183+0000","last_change":"2024-04-25T11:14:21.368766+0000","last_active":"2024-04-25T11:14:23.491183+0000","last_peered":"2024-04-25T11:14:23.491183+0000","last_clean":"2024-04-25T11:14:23.491183+0000","last_became_active":"2024-04-25T11:14:21.368466+0000","last_became_peered":"2024-04-25T11:14:21.368466+0000","last_unstale":"2024-04-25T11:14:23.491183+0000","last_undegraded":"2024-04-25T11:14:23.491183+0000","last_fullsized":"2024-04-25T11:14:23.491183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:09.063896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044726+0000","last_change":"2024-04-25T11:14:20.344137+0000","last_active":"2024-04-25T11:14:24.044726+0000","last_peered":"2024-04-25T11:14:24.044726+0000","last_clean":"2024-04-25T11:14:24.044726+0000","last_became_active":"2024-04-25T11:14:20.343813+0000","last_became_peered":"2024-04-25T11:14:20.343813+0000","last_unstale":"2024-04-25T11:14:24.044726+0000","last_undegraded":"2024-04-25T11:14:24.044726+0000","last_fullsized":"2024-04-25T11:14:24.044726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:03.000623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490040+0000","last_change":"2024-04-25T11:14:21.361869+0000","last_active":"2024-04-25T11:14:23.490040+0000","last_peered":"2024-04-25T11:14:23.490040+0000","last_clean":"2024-04-25T11:14:23.490040+0000","last_became_active":"2024-04-25T11:14:21.358307+0000","last_became_peered":"2024-04-25T11:14:21.358307+0000","last_unstale":"2024-04-25T11:14:23.490040+0000","last_undegraded":"2024-04-25T11:14:23.490040+0000","last_fullsized":"2024-04-25T11:14:23.490040+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:33.222542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366878+0000","last_change":"2024-04-25T11:14:20.331243+0000","last_active":"2024-04-25T11:14:22.366878+0000","last_peered":"2024-04-25T11:14:22.366878+0000","last_clean":"2024-04-25T11:14:22.366878+0000","last_became_active":"2024-04-25T11:14:20.330860+0000","last_became_peered":"2024-04-25T11:14:20.330860+0000","last_unstale":"2024-04-25T11:14:22.366878+0000","last_undegraded":"2024-04-25T11:14:22.366878+0000","last_fullsized":"2024-04-25T11:14:22.366878+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:32.729412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.539148+0000","last_change":"2024-04-25T11:14:21.390108+0000","last_active":"2024-04-25T11:14:23.539148+0000","last_peered":"2024-04-25T11:14:23.539148+0000","last_clean":"2024-04-25T11:14:23.539148+0000","last_became_active":"2024-04-25T11:14:21.389767+0000","last_became_peered":"2024-04-25T11:14:21.389767+0000","last_unstale":"2024-04-25T11:14:23.539148+0000","last_undegraded":"2024-04-25T11:14:23.539148+0000","last_fullsized":"2024-04-25T11:14:23.539148+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:04.241445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362445+0000","last_change":"2024-04-25T11:14:20.325726+0000","last_active":"2024-04-25T11:14:22.362445+0000","last_peered":"2024-04-25T11:14:22.362445+0000","last_clean":"2024-04-25T11:14:22.362445+0000","last_became_active":"2024-04-25T11:14:20.325199+0000","last_became_peered":"2024-04-25T11:14:20.325199+0000","last_unstale":"2024-04-25T11:14:22.362445+0000","last_undegraded":"2024-04-25T11:14:22.362445+0000","last_fullsized":"2024-04-25T11:14:22.362445+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:22.963995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044144+0000","last_change":"2024-04-25T11:14:21.377172+0000","last_active":"2024-04-25T11:14:24.044144+0000","last_peered":"2024-04-25T11:14:24.044144+0000","last_clean":"2024-04-25T11:14:24.044144+0000","last_became_active":"2024-04-25T11:14:21.376648+0000","last_became_peered":"2024-04-25T11:14:21.376648+0000","last_unstale":"2024-04-25T11:14:24.044144+0000","last_undegraded":"2024-04-25T11:14:24.044144+0000","last_fullsized":"2024-04-25T11:14:24.044144+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:49.106485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361947+0000","last_change":"2024-04-25T11:14:20.339034+0000","last_active":"2024-04-25T11:14:22.361947+0000","last_peered":"2024-04-25T11:14:22.361947+0000","last_clean":"2024-04-25T11:14:22.361947+0000","last_became_active":"2024-04-25T11:14:20.338713+0000","last_became_peered":"2024-04-25T11:14:20.338713+0000","last_unstale":"2024-04-25T11:14:22.361947+0000","last_undegraded":"2024-04-25T11:14:22.361947+0000","last_fullsized":"2024-04-25T11:14:22.361947+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:06.914535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346525+0000","last_change":"2024-04-25T11:14:21.373882+0000","last_active":"2024-04-25T11:14:23.346525+0000","last_peered":"2024-04-25T11:14:23.346525+0000","last_clean":"2024-04-25T11:14:23.346525+0000","last_became_active":"2024-04-25T11:14:21.373533+0000","last_became_peered":"2024-04-25T11:14:21.373533+0000","last_unstale":"2024-04-25T11:14:23.346525+0000","last_undegraded":"2024-04-25T11:14:23.346525+0000","last_fullsized":"2024-04-25T11:14:23.346525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:01.628406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.043503+0000","last_change":"2024-04-25T11:14:20.339826+0000","last_active":"2024-04-25T11:14:24.043503+0000","last_peered":"2024-04-25T11:14:24.043503+0000","last_clean":"2024-04-25T11:14:24.043503+0000","last_became_active":"2024-04-25T11:14:20.339366+0000","last_became_peered":"2024-04-25T11:14:20.339366+0000","last_unstale":"2024-04-25T11:14:24.043503+0000","last_undegraded":"2024-04-25T11:14:24.043503+0000","last_fullsized":"2024-04-25T11:14:24.043503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:15.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490643+0000","last_change":"2024-04-25T11:14:21.363893+0000","last_active":"2024-04-25T11:14:23.490643+0000","last_peered":"2024-04-25T11:14:23.490643+0000","last_clean":"2024-04-25T11:14:23.490643+0000","last_became_active":"2024-04-25T11:14:21.363555+0000","last_became_peered":"2024-04-25T11:14:21.363555+0000","last_unstale":"2024-04-25T11:14:23.490643+0000","last_undegraded":"2024-04-25T11:14:23.490643+0000","last_fullsized":"2024-04-25T11:14:23.490643+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:25.589218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361619+0000","last_change":"2024-04-25T11:14:20.334793+0000","last_active":"2024-04-25T11:14:22.361619+0000","last_peered":"2024-04-25T11:14:22.361619+0000","last_clean":"2024-04-25T11:14:22.361619+0000","last_became_active":"2024-04-25T11:14:20.333198+0000","last_became_peered":"2024-04-25T11:14:20.333198+0000","last_unstale":"2024-04-25T11:14:22.361619+0000","last_undegraded":"2024-04-25T11:14:22.361619+0000","last_fullsized":"2024-04-25T11:14:22.361619+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.559248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":203,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.365324+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:14:22.365324+0000","last_peered":"2024-04-25T11:14:22.365324+0000","last_clean":"2024-04-25T11:14:22.365324+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:14:22.365324+0000","last_undegraded":"2024-04-25T11:14:22.365324+0000","last_fullsized":"2024-04-25T11:14:22.365324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476789,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1421312,"data_stored":957554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476790,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":366444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476789,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":366432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476790,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":368022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509493,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":366410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":368786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1404928,"data_stored":956778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":366410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:14:31.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:14:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.778+0000 7f20b3e00640 1 Processor -- start 2024-04-25T11:14:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.778+0000 7f20b3e00640 1 -- start start 2024-04-25T11:14:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.781+0000 7f20b3e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b406dab0 0x7f20b4161400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.781+0000 7f20b3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20b406cf50 0x7f20b4075b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.781+0000 7f20b3e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b409b670 con 0x7f20b40b1320 2024-04-25T11:14:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.781+0000 7f20b3e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f20b40d28b0 con 0x7f20b406cf50 2024-04-25T11:14:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.781+0000 7f20b3e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f20b409b950 con 0x7f20b406dab0 2024-04-25T11:14:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.782+0000 7f20b3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20b406cf50 0x7f20b4075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.782+0000 7f20aa000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b406dab0 0x7f20b4161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.782+0000 7f20b2a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f20b40b1320 0x7f20b40b3400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:52700/0 (socket says 172.21.15.7:52700) 2024-04-25T11:14:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.782+0000 7f20b3400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20b406cf50 0x7f20b4075b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:58948/0 (socket says 172.21.15.7:58948) 2024-04-25T11:14:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.782+0000 7f20b2a00640 1 -- 172.21.15.7:0/2522277114 learned_addr learned my addr 172.21.15.7:0/2522277114 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.783+0000 7f20b3400640 1 -- 172.21.15.7:0/2522277114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b406dab0 msgr2=0x7f20b4161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.783+0000 7f20b3400640 1 --2- 172.21.15.7:0/2522277114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b406dab0 0x7f20b4161400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.783+0000 7f20b3400640 1 -- 172.21.15.7:0/2522277114 >> v1:172.21.15.87:6789/0 conn(0x7f20b40b1320 legacy=0x7f20b40b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.783+0000 7f20b3400640 1 -- 172.21.15.7:0/2522277114 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b40b1950 con 0x7f20b406cf50 2024-04-25T11:14:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.784+0000 7f20b3400640 1 --2- 172.21.15.7:0/2522277114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b406cf50 0x7f20b4075b10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009d90 tx=0x7f20ac02a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b812ccc0d5c94c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.784+0000 7f20b2000640 1 -- 172.21.15.7:0/2522277114 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac007740 con 0x7f20b406cf50 2024-04-25T11:14:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.784+0000 7f20b2000640 1 -- 172.21.15.7:0/2522277114 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ac0078e0 con 0x7f20b406cf50 2024-04-25T11:14:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.785+0000 7f20b2000640 1 -- 172.21.15.7:0/2522277114 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac007cb0 con 0x7f20b406cf50 2024-04-25T11:14:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.785+0000 7f20b3e00640 1 -- 172.21.15.7:0/2522277114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b406cf50 msgr2=0x7f20b4075b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.785+0000 7f20b3e00640 1 --2- 172.21.15.7:0/2522277114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b406cf50 0x7f20b4075b10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009d90 tx=0x7f20ac02a530 comp rx=0 tx=0).stop 2024-04-25T11:14:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.786+0000 7f20b3e00640 1 -- 172.21.15.7:0/2522277114 shutdown_connections 2024-04-25T11:14:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.786+0000 7f20b3e00640 1 --2- 172.21.15.7:0/2522277114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b406cf50 0x7f20b4075b10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.786+0000 7f20b3e00640 1 --2- 172.21.15.7:0/2522277114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b406dab0 0x7f20b4161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.786+0000 7f20b3e00640 1 -- 172.21.15.7:0/2522277114 >> 172.21.15.7:0/2522277114 conn(0x7f20b4073ba0 msgr2=0x7f20b40b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.786+0000 7f20b3e00640 1 -- 172.21.15.7:0/2522277114 shutdown_connections 2024-04-25T11:14:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.786+0000 7f20b3e00640 1 -- 172.21.15.7:0/2522277114 wait complete. 2024-04-25T11:14:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.789+0000 7f20b3e00640 1 Processor -- start 2024-04-25T11:14:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.789+0000 7f20b3e00640 1 -- start start 2024-04-25T11:14:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.791+0000 7f20b3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b406cf50 0x7f20b40ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.791+0000 7f20b3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20b406dab0 0x7f20b40cff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.791+0000 7f20b3e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b40b1320 0x7f20b41457d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.791+0000 7f20b3e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f20b40761d0 con 0x7f20b406dab0 2024-04-25T11:14:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.791+0000 7f20b3e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4076050 con 0x7f20b406cf50 2024-04-25T11:14:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.791+0000 7f20b3e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f20b406d360 con 0x7f20b40b1320 2024-04-25T11:14:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.792+0000 7f20aa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20b406dab0 0x7f20b40cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.792+0000 7f20aa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20b406dab0 0x7f20b40cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37940/0 (socket says 172.21.15.7:37940) 2024-04-25T11:14:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.792+0000 7f20aa000640 1 -- 172.21.15.7:0/1740866301 learned_addr learned my addr 172.21.15.7:0/1740866301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.792+0000 7f20b3400640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b40b1320 0x7f20b41457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.792+0000 7f20b2a00640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b406cf50 0x7f20b40ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.792+0000 7f20aa000640 1 -- 172.21.15.7:0/1740866301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b40b1320 msgr2=0x7f20b41457d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.792+0000 7f20aa000640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b40b1320 0x7f20b41457d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.792+0000 7f20aa000640 1 -- 172.21.15.7:0/1740866301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b406cf50 msgr2=0x7f20b40ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.792+0000 7f20aa000640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b406cf50 0x7f20b40ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.792+0000 7f20aa000640 1 -- 172.21.15.7:0/1740866301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b40cee70 con 0x7f20b406dab0 2024-04-25T11:14:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.793+0000 7f20aa000640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20b406dab0 0x7f20b40cff10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f209c00e8a0 tx=0x7f209c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.793+0000 7f20ab400640 1 -- 172.21.15.7:0/1740866301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c017070 con 0x7f20b406dab0 2024-04-25T11:14:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.793+0000 7f20ab400640 1 -- 172.21.15.7:0/1740866301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f209c00c940 con 0x7f20b406dab0 2024-04-25T11:14:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.793+0000 7f20ab400640 1 -- 172.21.15.7:0/1740866301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c00cd30 con 0x7f20b406dab0 2024-04-25T11:14:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.793+0000 7f20b3e00640 1 -- 172.21.15.7:0/1740866301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b40cf150 con 0x7f20b406dab0 2024-04-25T11:14:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.794+0000 7f20b3e00640 1 -- 172.21.15.7:0/1740866301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b40cf710 con 0x7f20b406dab0 2024-04-25T11:14:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.795+0000 7f20b3e00640 1 -- 172.21.15.7:0/1740866301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b40d2a80 con 0x7f20b406dab0 2024-04-25T11:14:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.796+0000 7f20ab400640 1 -- 172.21.15.7:0/1740866301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f209c01a070 con 0x7f20b406dab0 2024-04-25T11:14:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.797+0000 7f20ab400640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20900744b0 0x7f20900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.797+0000 7f20ab400640 1 -- 172.21.15.7:0/1740866301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f209c0956d0 con 0x7f20b406dab0 2024-04-25T11:14:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.798+0000 7f20b2a00640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20900744b0 0x7f20900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.798+0000 7f20b2a00640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20900744b0 0x7f20900769c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f20a4005880 tx=0x7f20a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:31.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:31.802+0000 7f20ab400640 1 -- 172.21.15.7:0/1740866301 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f209c05fc30 con 0x7f20b406dab0 2024-04-25T11:14:32.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.008+0000 7f20b3e00640 1 -- 172.21.15.7:0/1740866301 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f20b406f820 con 0x7f20900744b0 2024-04-25T11:14:32.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.024+0000 7f20ab400640 1 -- 172.21.15.7:0/1740866301 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251898 (secure 0 0 0) 0x7f20b406f820 con 0x7f20900744b0 2024-04-25T11:14:32.026 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:32.029 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:14:32.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.031+0000 7f20b3e00640 1 -- 172.21.15.7:0/1740866301 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20900744b0 msgr2=0x7f20900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:32.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.031+0000 7f20b3e00640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20900744b0 0x7f20900769c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f20a4005880 tx=0x7f20a4009040 comp rx=0 tx=0).stop 2024-04-25T11:14:32.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.031+0000 7f20b3e00640 1 -- 172.21.15.7:0/1740866301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20b406dab0 msgr2=0x7f20b40cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:32.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.031+0000 7f20b3e00640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20b406dab0 0x7f20b40cff10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f209c00e8a0 tx=0x7f209c00ed60 comp rx=0 tx=0).stop 2024-04-25T11:14:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.031+0000 7f20b3e00640 1 -- 172.21.15.7:0/1740866301 shutdown_connections 2024-04-25T11:14:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.031+0000 7f20b3e00640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20900744b0 0x7f20900769c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.031+0000 7f20b3e00640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b40b1320 0x7f20b41457d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.031+0000 7f20b3e00640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20b406dab0 0x7f20b40cff10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.031+0000 7f20b3e00640 1 --2- 172.21.15.7:0/1740866301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b406cf50 0x7f20b40ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.031+0000 7f20b3e00640 1 -- 172.21.15.7:0/1740866301 >> 172.21.15.7:0/1740866301 conn(0x7f20b4073ba0 msgr2=0x7f20b4160530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.031+0000 7f20b3e00640 1 -- 172.21.15.7:0/1740866301 shutdown_connections 2024-04-25T11:14:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.032+0000 7f20b3e00640 1 -- 172.21.15.7:0/1740866301 wait complete. 2024-04-25T11:14:32.058 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-04-25T11:14:31.878427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252864,"kb_used_data":7560,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721856,"statfs":{"total":773094113280,"available":772835180544,"internally_reserved":0,"allocated":7741440,"data_stored":4116836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004088"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045327+0000","last_change":"2024-04-25T11:14:21.374239+0000","last_active":"2024-04-25T11:14:24.045327+0000","last_peered":"2024-04-25T11:14:24.045327+0000","last_clean":"2024-04-25T11:14:24.045327+0000","last_became_active":"2024-04-25T11:14:21.373912+0000","last_became_peered":"2024-04-25T11:14:21.373912+0000","last_unstale":"2024-04-25T11:14:24.045327+0000","last_undegraded":"2024-04-25T11:14:24.045327+0000","last_fullsized":"2024-04-25T11:14:24.045327+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:02.752329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045090+0000","last_change":"2024-04-25T11:14:21.358156+0000","last_active":"2024-04-25T11:14:24.045090+0000","last_peered":"2024-04-25T11:14:24.045090+0000","last_clean":"2024-04-25T11:14:24.045090+0000","last_became_active":"2024-04-25T11:14:21.357825+0000","last_became_peered":"2024-04-25T11:14:21.357825+0000","last_unstale":"2024-04-25T11:14:24.045090+0000","last_undegraded":"2024-04-25T11:14:24.045090+0000","last_fullsized":"2024-04-25T11:14:24.045090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.581113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366418+0000","last_change":"2024-04-25T11:14:21.362112+0000","last_active":"2024-04-25T11:14:22.366418+0000","last_peered":"2024-04-25T11:14:22.366418+0000","last_clean":"2024-04-25T11:14:22.366418+0000","last_became_active":"2024-04-25T11:14:21.361509+0000","last_became_peered":"2024-04-25T11:14:21.361509+0000","last_unstale":"2024-04-25T11:14:22.366418+0000","last_undegraded":"2024-04-25T11:14:22.366418+0000","last_fullsized":"2024-04-25T11:14:22.366418+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:10.199408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.539979+0000","last_change":"2024-04-25T11:14:21.379397+0000","last_active":"2024-04-25T11:14:23.539979+0000","last_peered":"2024-04-25T11:14:23.539979+0000","last_clean":"2024-04-25T11:14:23.539979+0000","last_became_active":"2024-04-25T11:14:21.378997+0000","last_became_peered":"2024-04-25T11:14:21.378997+0000","last_unstale":"2024-04-25T11:14:23.539979+0000","last_undegraded":"2024-04-25T11:14:23.539979+0000","last_fullsized":"2024-04-25T11:14:23.539979+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:56.732074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490210+0000","last_change":"2024-04-25T11:14:21.384178+0000","last_active":"2024-04-25T11:14:23.490210+0000","last_peered":"2024-04-25T11:14:23.490210+0000","last_clean":"2024-04-25T11:14:23.490210+0000","last_became_active":"2024-04-25T11:14:21.383871+0000","last_became_peered":"2024-04-25T11:14:21.383871+0000","last_unstale":"2024-04-25T11:14:23.490210+0000","last_undegraded":"2024-04-25T11:14:23.490210+0000","last_fullsized":"2024-04-25T11:14:23.490210+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:17.347053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962720+0000","last_change":"2024-04-25T11:14:21.381797+0000","last_active":"2024-04-25T11:14:22.962720+0000","last_peered":"2024-04-25T11:14:22.962720+0000","last_clean":"2024-04-25T11:14:22.962720+0000","last_became_active":"2024-04-25T11:14:21.381332+0000","last_became_peered":"2024-04-25T11:14:21.381332+0000","last_unstale":"2024-04-25T11:14:22.962720+0000","last_undegraded":"2024-04-25T11:14:22.962720+0000","last_fullsized":"2024-04-25T11:14:22.962720+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.347162+0000","last_change":"2024-04-25T11:14:21.367266+0000","last_active":"2024-04-25T11:14:23.347162+0000","last_peered":"2024-04-25T11:14:23.347162+0000","last_clean":"2024-04-25T11:14:23.347162+0000","last_became_active":"2024-04-25T11:14:21.366376+0000","last_became_peered":"2024-04-25T11:14:21.366376+0000","last_unstale":"2024-04-25T11:14:23.347162+0000","last_undegraded":"2024-04-25T11:14:23.347162+0000","last_fullsized":"2024-04-25T11:14:23.347162+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:26.828975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.360817+0000","last_change":"2024-04-25T11:14:21.388651+0000","last_active":"2024-04-25T11:14:22.360817+0000","last_peered":"2024-04-25T11:14:22.360817+0000","last_clean":"2024-04-25T11:14:22.360817+0000","last_became_active":"2024-04-25T11:14:21.388262+0000","last_became_peered":"2024-04-25T11:14:21.388262+0000","last_unstale":"2024-04-25T11:14:22.360817+0000","last_undegraded":"2024-04-25T11:14:22.360817+0000","last_fullsized":"2024-04-25T11:14:22.360817+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:02.752329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367959+0000","last_change":"2024-04-25T11:14:21.383853+0000","last_active":"2024-04-25T11:14:22.367959+0000","last_peered":"2024-04-25T11:14:22.367959+0000","last_clean":"2024-04-25T11:14:22.367959+0000","last_became_active":"2024-04-25T11:14:21.383514+0000","last_became_peered":"2024-04-25T11:14:21.383514+0000","last_unstale":"2024-04-25T11:14:22.367959+0000","last_undegraded":"2024-04-25T11:14:22.367959+0000","last_fullsized":"2024-04-25T11:14:22.367959+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:40.752199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367177+0000","last_change":"2024-04-25T11:14:21.375219+0000","last_active":"2024-04-25T11:14:22.367177+0000","last_peered":"2024-04-25T11:14:22.367177+0000","last_clean":"2024-04-25T11:14:22.367177+0000","last_became_active":"2024-04-25T11:14:21.374869+0000","last_became_peered":"2024-04-25T11:14:21.374869+0000","last_unstale":"2024-04-25T11:14:22.367177+0000","last_undegraded":"2024-04-25T11:14:22.367177+0000","last_fullsized":"2024-04-25T11:14:22.367177+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:23.091769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.539810+0000","last_change":"2024-04-25T11:14:21.381440+0000","last_active":"2024-04-25T11:14:23.539810+0000","last_peered":"2024-04-25T11:14:23.539810+0000","last_clean":"2024-04-25T11:14:23.539810+0000","last_became_active":"2024-04-25T11:14:21.381002+0000","last_became_peered":"2024-04-25T11:14:21.381002+0000","last_unstale":"2024-04-25T11:14:23.539810+0000","last_undegraded":"2024-04-25T11:14:23.539810+0000","last_fullsized":"2024-04-25T11:14:23.539810+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:04.017992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361966+0000","last_change":"2024-04-25T11:14:21.374126+0000","last_active":"2024-04-25T11:14:22.361966+0000","last_peered":"2024-04-25T11:14:22.361966+0000","last_clean":"2024-04-25T11:14:22.361966+0000","last_became_active":"2024-04-25T11:14:21.373771+0000","last_became_peered":"2024-04-25T11:14:21.373771+0000","last_unstale":"2024-04-25T11:14:22.361966+0000","last_undegraded":"2024-04-25T11:14:22.361966+0000","last_fullsized":"2024-04-25T11:14:22.361966+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:09.576618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361433+0000","last_change":"2024-04-25T11:14:21.382011+0000","last_active":"2024-04-25T11:14:22.361433+0000","last_peered":"2024-04-25T11:14:22.361433+0000","last_clean":"2024-04-25T11:14:22.361433+0000","last_became_active":"2024-04-25T11:14:21.381691+0000","last_became_peered":"2024-04-25T11:14:21.381691+0000","last_unstale":"2024-04-25T11:14:22.361433+0000","last_undegraded":"2024-04-25T11:14:22.361433+0000","last_fullsized":"2024-04-25T11:14:22.361433+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:21.343521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366388+0000","last_change":"2024-04-25T11:14:21.374528+0000","last_active":"2024-04-25T11:14:22.366388+0000","last_peered":"2024-04-25T11:14:22.366388+0000","last_clean":"2024-04-25T11:14:22.366388+0000","last_became_active":"2024-04-25T11:14:21.372240+0000","last_became_peered":"2024-04-25T11:14:21.372240+0000","last_unstale":"2024-04-25T11:14:22.366388+0000","last_undegraded":"2024-04-25T11:14:22.366388+0000","last_fullsized":"2024-04-25T11:14:22.366388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:39.364702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.365670+0000","last_change":"2024-04-25T11:14:21.381099+0000","last_active":"2024-04-25T11:14:22.365670+0000","last_peered":"2024-04-25T11:14:22.365670+0000","last_clean":"2024-04-25T11:14:22.365670+0000","last_became_active":"2024-04-25T11:14:21.380778+0000","last_became_peered":"2024-04-25T11:14:21.380778+0000","last_unstale":"2024-04-25T11:14:22.365670+0000","last_undegraded":"2024-04-25T11:14:22.365670+0000","last_fullsized":"2024-04-25T11:14:22.365670+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:13.479999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346754+0000","last_change":"2024-04-25T11:14:21.374629+0000","last_active":"2024-04-25T11:14:23.346754+0000","last_peered":"2024-04-25T11:14:23.346754+0000","last_clean":"2024-04-25T11:14:23.346754+0000","last_became_active":"2024-04-25T11:14:21.373757+0000","last_became_peered":"2024-04-25T11:14:21.373757+0000","last_unstale":"2024-04-25T11:14:23.346754+0000","last_undegraded":"2024-04-25T11:14:23.346754+0000","last_fullsized":"2024-04-25T11:14:23.346754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:46.489832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045260+0000","last_change":"2024-04-25T11:14:21.384370+0000","last_active":"2024-04-25T11:14:24.045260+0000","last_peered":"2024-04-25T11:14:24.045260+0000","last_clean":"2024-04-25T11:14:24.045260+0000","last_became_active":"2024-04-25T11:14:21.383916+0000","last_became_peered":"2024-04-25T11:14:21.383916+0000","last_unstale":"2024-04-25T11:14:24.045260+0000","last_undegraded":"2024-04-25T11:14:24.045260+0000","last_fullsized":"2024-04-25T11:14:24.045260+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.182058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.539957+0000","last_change":"2024-04-25T11:14:21.380728+0000","last_active":"2024-04-25T11:14:23.539957+0000","last_peered":"2024-04-25T11:14:23.539957+0000","last_clean":"2024-04-25T11:14:23.539957+0000","last_became_active":"2024-04-25T11:14:21.380128+0000","last_became_peered":"2024-04-25T11:14:21.380128+0000","last_unstale":"2024-04-25T11:14:23.539957+0000","last_undegraded":"2024-04-25T11:14:23.539957+0000","last_fullsized":"2024-04-25T11:14:23.539957+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:09.307128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490690+0000","last_change":"2024-04-25T11:14:21.363932+0000","last_active":"2024-04-25T11:14:23.490690+0000","last_peered":"2024-04-25T11:14:23.490690+0000","last_clean":"2024-04-25T11:14:23.490690+0000","last_became_active":"2024-04-25T11:14:21.363589+0000","last_became_peered":"2024-04-25T11:14:21.363589+0000","last_unstale":"2024-04-25T11:14:23.490690+0000","last_undegraded":"2024-04-25T11:14:23.490690+0000","last_fullsized":"2024-04-25T11:14:23.490690+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:19.066276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362235+0000","last_change":"2024-04-25T11:14:21.370707+0000","last_active":"2024-04-25T11:14:22.362235+0000","last_peered":"2024-04-25T11:14:22.362235+0000","last_clean":"2024-04-25T11:14:22.362235+0000","last_became_active":"2024-04-25T11:14:21.370409+0000","last_became_peered":"2024-04-25T11:14:21.370409+0000","last_unstale":"2024-04-25T11:14:22.362235+0000","last_undegraded":"2024-04-25T11:14:22.362235+0000","last_fullsized":"2024-04-25T11:14:22.362235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:40.008261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361908+0000","last_change":"2024-04-25T11:14:21.363401+0000","last_active":"2024-04-25T11:14:22.361908+0000","last_peered":"2024-04-25T11:14:22.361908+0000","last_clean":"2024-04-25T11:14:22.361908+0000","last_became_active":"2024-04-25T11:14:21.363014+0000","last_became_peered":"2024-04-25T11:14:21.363014+0000","last_unstale":"2024-04-25T11:14:22.361908+0000","last_undegraded":"2024-04-25T11:14:22.361908+0000","last_fullsized":"2024-04-25T11:14:22.361908+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:14.240189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362273+0000","last_change":"2024-04-25T11:14:21.380852+0000","last_active":"2024-04-25T11:14:22.362273+0000","last_peered":"2024-04-25T11:14:22.362273+0000","last_clean":"2024-04-25T11:14:22.362273+0000","last_became_active":"2024-04-25T11:14:21.380226+0000","last_became_peered":"2024-04-25T11:14:21.380226+0000","last_unstale":"2024-04-25T11:14:22.362273+0000","last_undegraded":"2024-04-25T11:14:22.362273+0000","last_fullsized":"2024-04-25T11:14:22.362273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:12.837877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361853+0000","last_change":"2024-04-25T11:14:21.365901+0000","last_active":"2024-04-25T11:14:22.361853+0000","last_peered":"2024-04-25T11:14:22.361853+0000","last_clean":"2024-04-25T11:14:22.361853+0000","last_became_active":"2024-04-25T11:14:21.365551+0000","last_became_peered":"2024-04-25T11:14:21.365551+0000","last_unstale":"2024-04-25T11:14:22.361853+0000","last_undegraded":"2024-04-25T11:14:22.361853+0000","last_fullsized":"2024-04-25T11:14:22.361853+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:42.952154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.365456+0000","last_change":"2024-04-25T11:14:21.364597+0000","last_active":"2024-04-25T11:14:22.365456+0000","last_peered":"2024-04-25T11:14:22.365456+0000","last_clean":"2024-04-25T11:14:22.365456+0000","last_became_active":"2024-04-25T11:14:21.363451+0000","last_became_peered":"2024-04-25T11:14:21.363451+0000","last_unstale":"2024-04-25T11:14:22.365456+0000","last_undegraded":"2024-04-25T11:14:22.365456+0000","last_fullsized":"2024-04-25T11:14:22.365456+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:09.576618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362664+0000","last_change":"2024-04-25T11:14:21.368883+0000","last_active":"2024-04-25T11:14:22.362664+0000","last_peered":"2024-04-25T11:14:22.362664+0000","last_clean":"2024-04-25T11:14:22.362664+0000","last_became_active":"2024-04-25T11:14:21.368181+0000","last_became_peered":"2024-04-25T11:14:21.368181+0000","last_unstale":"2024-04-25T11:14:22.362664+0000","last_undegraded":"2024-04-25T11:14:22.362664+0000","last_fullsized":"2024-04-25T11:14:22.362664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:23.981365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362255+0000","last_change":"2024-04-25T11:14:21.373674+0000","last_active":"2024-04-25T11:14:22.362255+0000","last_peered":"2024-04-25T11:14:22.362255+0000","last_clean":"2024-04-25T11:14:22.362255+0000","last_became_active":"2024-04-25T11:14:21.373331+0000","last_became_peered":"2024-04-25T11:14:21.373331+0000","last_unstale":"2024-04-25T11:14:22.362255+0000","last_undegraded":"2024-04-25T11:14:22.362255+0000","last_fullsized":"2024-04-25T11:14:22.362255+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:13.479999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.539982+0000","last_change":"2024-04-25T11:14:21.377004+0000","last_active":"2024-04-25T11:14:23.539982+0000","last_peered":"2024-04-25T11:14:23.539982+0000","last_clean":"2024-04-25T11:14:23.539982+0000","last_became_active":"2024-04-25T11:14:21.376617+0000","last_became_peered":"2024-04-25T11:14:21.376617+0000","last_unstale":"2024-04-25T11:14:23.539982+0000","last_undegraded":"2024-04-25T11:14:23.539982+0000","last_fullsized":"2024-04-25T11:14:23.539982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:53.111220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044519+0000","last_change":"2024-04-25T11:14:21.371250+0000","last_active":"2024-04-25T11:14:24.044519+0000","last_peered":"2024-04-25T11:14:24.044519+0000","last_clean":"2024-04-25T11:14:24.044519+0000","last_became_active":"2024-04-25T11:14:21.370864+0000","last_became_peered":"2024-04-25T11:14:21.370864+0000","last_unstale":"2024-04-25T11:14:24.044519+0000","last_undegraded":"2024-04-25T11:14:24.044519+0000","last_fullsized":"2024-04-25T11:14:24.044519+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:09.063896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366579+0000","last_change":"2024-04-25T11:14:21.384753+0000","last_active":"2024-04-25T11:14:22.366579+0000","last_peered":"2024-04-25T11:14:22.366579+0000","last_clean":"2024-04-25T11:14:22.366579+0000","last_became_active":"2024-04-25T11:14:21.384449+0000","last_became_peered":"2024-04-25T11:14:21.384449+0000","last_unstale":"2024-04-25T11:14:22.366579+0000","last_undegraded":"2024-04-25T11:14:22.366579+0000","last_fullsized":"2024-04-25T11:14:22.366579+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:07.931905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361209+0000","last_change":"2024-04-25T11:14:21.383853+0000","last_active":"2024-04-25T11:14:22.361209+0000","last_peered":"2024-04-25T11:14:22.361209+0000","last_clean":"2024-04-25T11:14:22.361209+0000","last_became_active":"2024-04-25T11:14:21.383527+0000","last_became_peered":"2024-04-25T11:14:21.383527+0000","last_unstale":"2024-04-25T11:14:22.361209+0000","last_undegraded":"2024-04-25T11:14:22.361209+0000","last_fullsized":"2024-04-25T11:14:22.361209+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:59.742186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.538857+0000","last_change":"2024-04-25T11:14:21.370126+0000","last_active":"2024-04-25T11:14:23.538857+0000","last_peered":"2024-04-25T11:14:23.538857+0000","last_clean":"2024-04-25T11:14:23.538857+0000","last_became_active":"2024-04-25T11:14:21.369727+0000","last_became_peered":"2024-04-25T11:14:21.369727+0000","last_unstale":"2024-04-25T11:14:23.538857+0000","last_undegraded":"2024-04-25T11:14:23.538857+0000","last_fullsized":"2024-04-25T11:14:23.538857+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:17.010334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961814+0000","last_change":"2024-04-25T11:14:21.360674+0000","last_active":"2024-04-25T11:14:22.961814+0000","last_peered":"2024-04-25T11:14:22.961814+0000","last_clean":"2024-04-25T11:14:22.961814+0000","last_became_active":"2024-04-25T11:14:21.359863+0000","last_became_peered":"2024-04-25T11:14:21.359863+0000","last_unstale":"2024-04-25T11:14:22.961814+0000","last_undegraded":"2024-04-25T11:14:22.961814+0000","last_fullsized":"2024-04-25T11:14:22.961814+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:49.430307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490947+0000","last_change":"2024-04-25T11:14:21.360138+0000","last_active":"2024-04-25T11:14:23.490947+0000","last_peered":"2024-04-25T11:14:23.490947+0000","last_clean":"2024-04-25T11:14:23.490947+0000","last_became_active":"2024-04-25T11:14:21.359803+0000","last_became_peered":"2024-04-25T11:14:21.359803+0000","last_unstale":"2024-04-25T11:14:23.490947+0000","last_undegraded":"2024-04-25T11:14:23.490947+0000","last_fullsized":"2024-04-25T11:14:23.490947+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:20.532234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"67'4","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.045418+0000","last_change":"2024-04-25T11:14:20.334352+0000","last_active":"2024-04-25T11:14:24.045418+0000","last_peered":"2024-04-25T11:14:24.045418+0000","last_clean":"2024-04-25T11:14:24.045418+0000","last_became_active":"2024-04-25T11:14:20.329783+0000","last_became_peered":"2024-04-25T11:14:20.329783+0000","last_unstale":"2024-04-25T11:14:24.045418+0000","last_undegraded":"2024-04-25T11:14:24.045418+0000","last_fullsized":"2024-04-25T11:14:24.045418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:55.714704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.540403+0000","last_change":"2024-04-25T11:14:21.378061+0000","last_active":"2024-04-25T11:14:23.540403+0000","last_peered":"2024-04-25T11:14:23.540403+0000","last_clean":"2024-04-25T11:14:23.540403+0000","last_became_active":"2024-04-25T11:14:21.377684+0000","last_became_peered":"2024-04-25T11:14:21.377684+0000","last_unstale":"2024-04-25T11:14:23.540403+0000","last_undegraded":"2024-04-25T11:14:23.540403+0000","last_fullsized":"2024-04-25T11:14:23.540403+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:22.490881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962745+0000","last_change":"2024-04-25T11:14:20.346713+0000","last_active":"2024-04-25T11:14:22.962745+0000","last_peered":"2024-04-25T11:14:22.962745+0000","last_clean":"2024-04-25T11:14:22.962745+0000","last_became_active":"2024-04-25T11:14:20.346320+0000","last_became_peered":"2024-04-25T11:14:20.346320+0000","last_unstale":"2024-04-25T11:14:22.962745+0000","last_undegraded":"2024-04-25T11:14:22.962745+0000","last_fullsized":"2024-04-25T11:14:22.962745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.559248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962535+0000","last_change":"2024-04-25T11:14:21.356785+0000","last_active":"2024-04-25T11:14:22.962535+0000","last_peered":"2024-04-25T11:14:22.962535+0000","last_clean":"2024-04-25T11:14:22.962535+0000","last_became_active":"2024-04-25T11:14:21.356335+0000","last_became_peered":"2024-04-25T11:14:21.356335+0000","last_unstale":"2024-04-25T11:14:22.962535+0000","last_undegraded":"2024-04-25T11:14:22.962535+0000","last_fullsized":"2024-04-25T11:14:22.962535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.363496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962440+0000","last_change":"2024-04-25T11:14:20.346384+0000","last_active":"2024-04-25T11:14:22.962440+0000","last_peered":"2024-04-25T11:14:22.962440+0000","last_clean":"2024-04-25T11:14:22.962440+0000","last_became_active":"2024-04-25T11:14:20.346031+0000","last_became_peered":"2024-04-25T11:14:20.346031+0000","last_unstale":"2024-04-25T11:14:22.962440+0000","last_undegraded":"2024-04-25T11:14:22.962440+0000","last_fullsized":"2024-04-25T11:14:22.962440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:22.963995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367664+0000","last_change":"2024-04-25T11:14:21.375789+0000","last_active":"2024-04-25T11:14:22.367664+0000","last_peered":"2024-04-25T11:14:22.367664+0000","last_clean":"2024-04-25T11:14:22.367664+0000","last_became_active":"2024-04-25T11:14:21.375492+0000","last_became_peered":"2024-04-25T11:14:21.375492+0000","last_unstale":"2024-04-25T11:14:22.367664+0000","last_undegraded":"2024-04-25T11:14:22.367664+0000","last_fullsized":"2024-04-25T11:14:22.367664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:46.640556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962159+0000","last_change":"2024-04-25T11:14:20.337082+0000","last_active":"2024-04-25T11:14:22.962159+0000","last_peered":"2024-04-25T11:14:22.962159+0000","last_clean":"2024-04-25T11:14:22.962159+0000","last_became_active":"2024-04-25T11:14:20.336766+0000","last_became_peered":"2024-04-25T11:14:20.336766+0000","last_unstale":"2024-04-25T11:14:22.962159+0000","last_undegraded":"2024-04-25T11:14:22.962159+0000","last_fullsized":"2024-04-25T11:14:22.962159+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:12.462630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346829+0000","last_change":"2024-04-25T11:14:21.346395+0000","last_active":"2024-04-25T11:14:23.346829+0000","last_peered":"2024-04-25T11:14:23.346829+0000","last_clean":"2024-04-25T11:14:23.346829+0000","last_became_active":"2024-04-25T11:14:21.345933+0000","last_became_peered":"2024-04-25T11:14:21.345933+0000","last_unstale":"2024-04-25T11:14:23.346829+0000","last_undegraded":"2024-04-25T11:14:23.346829+0000","last_fullsized":"2024-04-25T11:14:23.346829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.002374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362073+0000","last_change":"2024-04-25T11:14:20.335533+0000","last_active":"2024-04-25T11:14:22.362073+0000","last_peered":"2024-04-25T11:14:22.362073+0000","last_clean":"2024-04-25T11:14:22.362073+0000","last_became_active":"2024-04-25T11:14:20.335228+0000","last_became_peered":"2024-04-25T11:14:20.335228+0000","last_unstale":"2024-04-25T11:14:22.362073+0000","last_undegraded":"2024-04-25T11:14:22.362073+0000","last_fullsized":"2024-04-25T11:14:22.362073+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.347332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044109+0000","last_change":"2024-04-25T11:14:21.384939+0000","last_active":"2024-04-25T11:14:24.044109+0000","last_peered":"2024-04-25T11:14:24.044109+0000","last_clean":"2024-04-25T11:14:24.044109+0000","last_became_active":"2024-04-25T11:14:21.384525+0000","last_became_peered":"2024-04-25T11:14:21.384525+0000","last_unstale":"2024-04-25T11:14:24.044109+0000","last_undegraded":"2024-04-25T11:14:24.044109+0000","last_fullsized":"2024-04-25T11:14:24.044109+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:19.066276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361973+0000","last_change":"2024-04-25T11:14:20.317044+0000","last_active":"2024-04-25T11:14:22.361973+0000","last_peered":"2024-04-25T11:14:22.361973+0000","last_clean":"2024-04-25T11:14:22.361973+0000","last_became_active":"2024-04-25T11:14:20.316634+0000","last_became_peered":"2024-04-25T11:14:20.316634+0000","last_unstale":"2024-04-25T11:14:22.361973+0000","last_undegraded":"2024-04-25T11:14:22.361973+0000","last_fullsized":"2024-04-25T11:14:22.361973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:12.462630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490055+0000","last_change":"2024-04-25T11:14:21.372187+0000","last_active":"2024-04-25T11:14:23.490055+0000","last_peered":"2024-04-25T11:14:23.490055+0000","last_clean":"2024-04-25T11:14:23.490055+0000","last_became_active":"2024-04-25T11:14:21.370797+0000","last_became_peered":"2024-04-25T11:14:21.370797+0000","last_unstale":"2024-04-25T11:14:23.490055+0000","last_undegraded":"2024-04-25T11:14:23.490055+0000","last_fullsized":"2024-04-25T11:14:23.490055+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:11.224114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361960+0000","last_change":"2024-04-25T11:14:20.339374+0000","last_active":"2024-04-25T11:14:22.361960+0000","last_peered":"2024-04-25T11:14:22.361960+0000","last_clean":"2024-04-25T11:14:22.361960+0000","last_became_active":"2024-04-25T11:14:20.339026+0000","last_became_peered":"2024-04-25T11:14:20.339026+0000","last_unstale":"2024-04-25T11:14:22.361960+0000","last_undegraded":"2024-04-25T11:14:22.361960+0000","last_fullsized":"2024-04-25T11:14:22.361960+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:09.182038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362009+0000","last_change":"2024-04-25T11:14:21.380186+0000","last_active":"2024-04-25T11:14:22.362009+0000","last_peered":"2024-04-25T11:14:22.362009+0000","last_clean":"2024-04-25T11:14:22.362009+0000","last_became_active":"2024-04-25T11:14:21.379835+0000","last_became_peered":"2024-04-25T11:14:21.379835+0000","last_unstale":"2024-04-25T11:14:22.362009+0000","last_undegraded":"2024-04-25T11:14:22.362009+0000","last_fullsized":"2024-04-25T11:14:22.362009+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:25.039608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490006+0000","last_change":"2024-04-25T11:14:20.344908+0000","last_active":"2024-04-25T11:14:23.490006+0000","last_peered":"2024-04-25T11:14:23.490006+0000","last_clean":"2024-04-25T11:14:23.490006+0000","last_became_active":"2024-04-25T11:14:20.344554+0000","last_became_peered":"2024-04-25T11:14:20.344554+0000","last_unstale":"2024-04-25T11:14:23.490006+0000","last_undegraded":"2024-04-25T11:14:23.490006+0000","last_fullsized":"2024-04-25T11:14:23.490006+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:12.170244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044489+0000","last_change":"2024-04-25T11:14:21.352169+0000","last_active":"2024-04-25T11:14:24.044489+0000","last_peered":"2024-04-25T11:14:24.044489+0000","last_clean":"2024-04-25T11:14:24.044489+0000","last_became_active":"2024-04-25T11:14:21.351353+0000","last_became_peered":"2024-04-25T11:14:21.351353+0000","last_unstale":"2024-04-25T11:14:24.044489+0000","last_undegraded":"2024-04-25T11:14:24.044489+0000","last_fullsized":"2024-04-25T11:14:24.044489+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:40.169340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367869+0000","last_change":"2024-04-25T11:14:20.330643+0000","last_active":"2024-04-25T11:14:22.367869+0000","last_peered":"2024-04-25T11:14:22.367869+0000","last_clean":"2024-04-25T11:14:22.367869+0000","last_became_active":"2024-04-25T11:14:20.330199+0000","last_became_peered":"2024-04-25T11:14:20.330199+0000","last_unstale":"2024-04-25T11:14:22.367869+0000","last_undegraded":"2024-04-25T11:14:22.367869+0000","last_fullsized":"2024-04-25T11:14:22.367869+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:48.089116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362035+0000","last_change":"2024-04-25T11:14:21.358807+0000","last_active":"2024-04-25T11:14:22.362035+0000","last_peered":"2024-04-25T11:14:22.362035+0000","last_clean":"2024-04-25T11:14:22.362035+0000","last_became_active":"2024-04-25T11:14:21.358476+0000","last_became_peered":"2024-04-25T11:14:21.358476+0000","last_unstale":"2024-04-25T11:14:22.362035+0000","last_undegraded":"2024-04-25T11:14:22.362035+0000","last_fullsized":"2024-04-25T11:14:22.362035+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:40.363496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.540476+0000","last_change":"2024-04-25T11:14:20.344529+0000","last_active":"2024-04-25T11:14:23.540476+0000","last_peered":"2024-04-25T11:14:23.540476+0000","last_clean":"2024-04-25T11:14:23.540476+0000","last_became_active":"2024-04-25T11:14:20.344184+0000","last_became_peered":"2024-04-25T11:14:20.344184+0000","last_unstale":"2024-04-25T11:14:23.540476+0000","last_undegraded":"2024-04-25T11:14:23.540476+0000","last_fullsized":"2024-04-25T11:14:23.540476+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:12.962009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.491302+0000","last_change":"2024-04-25T11:14:21.372160+0000","last_active":"2024-04-25T11:14:23.491302+0000","last_peered":"2024-04-25T11:14:23.491302+0000","last_clean":"2024-04-25T11:14:23.491302+0000","last_became_active":"2024-04-25T11:14:21.371743+0000","last_became_peered":"2024-04-25T11:14:21.371743+0000","last_unstale":"2024-04-25T11:14:23.491302+0000","last_undegraded":"2024-04-25T11:14:23.491302+0000","last_fullsized":"2024-04-25T11:14:23.491302+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.581113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"67'3","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366722+0000","last_change":"2024-04-25T11:14:20.337503+0000","last_active":"2024-04-25T11:14:22.366722+0000","last_peered":"2024-04-25T11:14:22.366722+0000","last_clean":"2024-04-25T11:14:22.366722+0000","last_became_active":"2024-04-25T11:14:20.337171+0000","last_became_peered":"2024-04-25T11:14:20.337171+0000","last_unstale":"2024-04-25T11:14:22.366722+0000","last_undegraded":"2024-04-25T11:14:22.366722+0000","last_fullsized":"2024-04-25T11:14:22.366722+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:07.071825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.540457+0000","last_change":"2024-04-25T11:14:21.370932+0000","last_active":"2024-04-25T11:14:23.540457+0000","last_peered":"2024-04-25T11:14:23.540457+0000","last_clean":"2024-04-25T11:14:23.540457+0000","last_became_active":"2024-04-25T11:14:21.370600+0000","last_became_peered":"2024-04-25T11:14:21.370600+0000","last_unstale":"2024-04-25T11:14:23.540457+0000","last_undegraded":"2024-04-25T11:14:23.540457+0000","last_fullsized":"2024-04-25T11:14:23.540457+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:33.222542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"67'6","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044599+0000","last_change":"2024-04-25T11:14:20.343544+0000","last_active":"2024-04-25T11:14:24.044599+0000","last_peered":"2024-04-25T11:14:24.044599+0000","last_clean":"2024-04-25T11:14:24.044599+0000","last_became_active":"2024-04-25T11:14:20.343205+0000","last_became_peered":"2024-04-25T11:14:20.343205+0000","last_unstale":"2024-04-25T11:14:24.044599+0000","last_undegraded":"2024-04-25T11:14:24.044599+0000","last_fullsized":"2024-04-25T11:14:24.044599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:03.224076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.538721+0000","last_change":"2024-04-25T11:14:21.390400+0000","last_active":"2024-04-25T11:14:23.538721+0000","last_peered":"2024-04-25T11:14:23.538721+0000","last_clean":"2024-04-25T11:14:23.538721+0000","last_became_active":"2024-04-25T11:14:21.389956+0000","last_became_peered":"2024-04-25T11:14:21.389956+0000","last_unstale":"2024-04-25T11:14:23.538721+0000","last_undegraded":"2024-04-25T11:14:23.538721+0000","last_fullsized":"2024-04-25T11:14:23.538721+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:50.079030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366706+0000","last_change":"2024-04-25T11:14:20.339359+0000","last_active":"2024-04-25T11:14:22.366706+0000","last_peered":"2024-04-25T11:14:22.366706+0000","last_clean":"2024-04-25T11:14:22.366706+0000","last_became_active":"2024-04-25T11:14:20.338755+0000","last_became_peered":"2024-04-25T11:14:20.338755+0000","last_unstale":"2024-04-25T11:14:22.366706+0000","last_undegraded":"2024-04-25T11:14:22.366706+0000","last_fullsized":"2024-04-25T11:14:22.366706+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:16.329683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361579+0000","last_change":"2024-04-25T11:14:21.357141+0000","last_active":"2024-04-25T11:14:22.361579+0000","last_peered":"2024-04-25T11:14:22.361579+0000","last_clean":"2024-04-25T11:14:22.361579+0000","last_became_active":"2024-04-25T11:14:21.356725+0000","last_became_peered":"2024-04-25T11:14:21.356725+0000","last_unstale":"2024-04-25T11:14:22.361579+0000","last_undegraded":"2024-04-25T11:14:22.361579+0000","last_fullsized":"2024-04-25T11:14:22.361579+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:38.740665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361050+0000","last_change":"2024-04-25T11:14:20.333436+0000","last_active":"2024-04-25T11:14:22.361050+0000","last_peered":"2024-04-25T11:14:22.361050+0000","last_clean":"2024-04-25T11:14:22.361050+0000","last_became_active":"2024-04-25T11:14:20.333048+0000","last_became_peered":"2024-04-25T11:14:20.333048+0000","last_unstale":"2024-04-25T11:14:22.361050+0000","last_undegraded":"2024-04-25T11:14:22.361050+0000","last_fullsized":"2024-04-25T11:14:22.361050+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:08.289759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490394+0000","last_change":"2024-04-25T11:14:21.370711+0000","last_active":"2024-04-25T11:14:23.490394+0000","last_peered":"2024-04-25T11:14:23.490394+0000","last_clean":"2024-04-25T11:14:23.490394+0000","last_became_active":"2024-04-25T11:14:21.369950+0000","last_became_peered":"2024-04-25T11:14:21.369950+0000","last_unstale":"2024-04-25T11:14:23.490394+0000","last_undegraded":"2024-04-25T11:14:23.490394+0000","last_fullsized":"2024-04-25T11:14:23.490394+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:08.089195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.365980+0000","last_change":"2024-04-25T11:14:20.331830+0000","last_active":"2024-04-25T11:14:22.365980+0000","last_peered":"2024-04-25T11:14:22.365980+0000","last_clean":"2024-04-25T11:14:22.365980+0000","last_became_active":"2024-04-25T11:14:20.331453+0000","last_became_peered":"2024-04-25T11:14:20.331453+0000","last_unstale":"2024-04-25T11:14:22.365980+0000","last_undegraded":"2024-04-25T11:14:22.365980+0000","last_fullsized":"2024-04-25T11:14:22.365980+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:19.514865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044411+0000","last_change":"2024-04-25T11:14:21.365796+0000","last_active":"2024-04-25T11:14:24.044411+0000","last_peered":"2024-04-25T11:14:24.044411+0000","last_clean":"2024-04-25T11:14:24.044411+0000","last_became_active":"2024-04-25T11:14:21.364748+0000","last_became_peered":"2024-04-25T11:14:21.364748+0000","last_unstale":"2024-04-25T11:14:24.044411+0000","last_undegraded":"2024-04-25T11:14:24.044411+0000","last_fullsized":"2024-04-25T11:14:24.044411+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:20.532234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"67'8","reported_seq":22,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.538655+0000","last_change":"2024-04-25T11:14:20.339581+0000","last_active":"2024-04-25T11:14:23.538655+0000","last_peered":"2024-04-25T11:14:23.538655+0000","last_clean":"2024-04-25T11:14:23.538655+0000","last_became_active":"2024-04-25T11:14:20.339208+0000","last_became_peered":"2024-04-25T11:14:20.339208+0000","last_unstale":"2024-04-25T11:14:23.538655+0000","last_undegraded":"2024-04-25T11:14:23.538655+0000","last_fullsized":"2024-04-25T11:14:23.538655+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:40.376640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366146+0000","last_change":"2024-04-25T11:14:21.384873+0000","last_active":"2024-04-25T11:14:22.366146+0000","last_peered":"2024-04-25T11:14:22.366146+0000","last_clean":"2024-04-25T11:14:22.366146+0000","last_became_active":"2024-04-25T11:14:21.383780+0000","last_became_peered":"2024-04-25T11:14:21.383780+0000","last_unstale":"2024-04-25T11:14:22.366146+0000","last_undegraded":"2024-04-25T11:14:22.366146+0000","last_fullsized":"2024-04-25T11:14:22.366146+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:23.981365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362950+0000","last_change":"2024-04-25T11:14:20.344787+0000","last_active":"2024-04-25T11:14:22.362950+0000","last_peered":"2024-04-25T11:14:22.362950+0000","last_clean":"2024-04-25T11:14:22.362950+0000","last_became_active":"2024-04-25T11:14:20.343403+0000","last_became_peered":"2024-04-25T11:14:20.343403+0000","last_unstale":"2024-04-25T11:14:22.362950+0000","last_undegraded":"2024-04-25T11:14:22.362950+0000","last_fullsized":"2024-04-25T11:14:22.362950+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:32.205173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490888+0000","last_change":"2024-04-25T11:14:21.383372+0000","last_active":"2024-04-25T11:14:23.490888+0000","last_peered":"2024-04-25T11:14:23.490888+0000","last_clean":"2024-04-25T11:14:23.490888+0000","last_became_active":"2024-04-25T11:14:21.383058+0000","last_became_peered":"2024-04-25T11:14:21.383058+0000","last_unstale":"2024-04-25T11:14:23.490888+0000","last_undegraded":"2024-04-25T11:14:23.490888+0000","last_fullsized":"2024-04-25T11:14:23.490888+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:49.106485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367149+0000","last_change":"2024-04-25T11:14:20.342160+0000","last_active":"2024-04-25T11:14:22.367149+0000","last_peered":"2024-04-25T11:14:22.367149+0000","last_clean":"2024-04-25T11:14:22.367149+0000","last_became_active":"2024-04-25T11:14:20.341788+0000","last_became_peered":"2024-04-25T11:14:20.341788+0000","last_unstale":"2024-04-25T11:14:22.367149+0000","last_undegraded":"2024-04-25T11:14:22.367149+0000","last_fullsized":"2024-04-25T11:14:22.367149+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:30:18.048907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.540266+0000","last_change":"2024-04-25T11:14:21.380455+0000","last_active":"2024-04-25T11:14:23.540266+0000","last_peered":"2024-04-25T11:14:23.540266+0000","last_clean":"2024-04-25T11:14:23.540266+0000","last_became_active":"2024-04-25T11:14:21.379717+0000","last_became_peered":"2024-04-25T11:14:21.379717+0000","last_unstale":"2024-04-25T11:14:23.540266+0000","last_undegraded":"2024-04-25T11:14:23.540266+0000","last_fullsized":"2024-04-25T11:14:23.540266+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:23.080677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.367001+0000","last_change":"2024-04-25T11:14:20.341298+0000","last_active":"2024-04-25T11:14:22.367001+0000","last_peered":"2024-04-25T11:14:22.367001+0000","last_clean":"2024-04-25T11:14:22.367001+0000","last_became_active":"2024-04-25T11:14:20.340803+0000","last_became_peered":"2024-04-25T11:14:20.340803+0000","last_unstale":"2024-04-25T11:14:22.367001+0000","last_undegraded":"2024-04-25T11:14:22.367001+0000","last_fullsized":"2024-04-25T11:14:22.367001+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:24.571848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.491062+0000","last_change":"2024-04-25T11:14:21.367727+0000","last_active":"2024-04-25T11:14:23.491062+0000","last_peered":"2024-04-25T11:14:23.491062+0000","last_clean":"2024-04-25T11:14:23.491062+0000","last_became_active":"2024-04-25T11:14:21.367327+0000","last_became_peered":"2024-04-25T11:14:21.367327+0000","last_unstale":"2024-04-25T11:14:23.491062+0000","last_undegraded":"2024-04-25T11:14:23.491062+0000","last_fullsized":"2024-04-25T11:14:23.491062+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:40.169340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362338+0000","last_change":"2024-04-25T11:14:20.341606+0000","last_active":"2024-04-25T11:14:22.362338+0000","last_peered":"2024-04-25T11:14:22.362338+0000","last_clean":"2024-04-25T11:14:22.362338+0000","last_became_active":"2024-04-25T11:14:20.341220+0000","last_became_peered":"2024-04-25T11:14:20.341220+0000","last_unstale":"2024-04-25T11:14:22.362338+0000","last_undegraded":"2024-04-25T11:14:22.362338+0000","last_fullsized":"2024-04-25T11:14:22.362338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:13.222820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961827+0000","last_change":"2024-04-25T11:14:21.359973+0000","last_active":"2024-04-25T11:14:22.961827+0000","last_peered":"2024-04-25T11:14:22.961827+0000","last_clean":"2024-04-25T11:14:22.961827+0000","last_became_active":"2024-04-25T11:14:21.359599+0000","last_became_peered":"2024-04-25T11:14:21.359599+0000","last_unstale":"2024-04-25T11:14:22.961827+0000","last_undegraded":"2024-04-25T11:14:22.961827+0000","last_fullsized":"2024-04-25T11:14:22.961827+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:42.952154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961999+0000","last_change":"2024-04-25T11:14:20.341545+0000","last_active":"2024-04-25T11:14:22.961999+0000","last_peered":"2024-04-25T11:14:22.961999+0000","last_clean":"2024-04-25T11:14:22.961999+0000","last_became_active":"2024-04-25T11:14:20.341206+0000","last_became_peered":"2024-04-25T11:14:20.341206+0000","last_unstale":"2024-04-25T11:14:22.961999+0000","last_undegraded":"2024-04-25T11:14:22.961999+0000","last_fullsized":"2024-04-25T11:14:22.961999+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:13.222820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361215+0000","last_change":"2024-04-25T11:14:21.367533+0000","last_active":"2024-04-25T11:14:22.361215+0000","last_peered":"2024-04-25T11:14:22.361215+0000","last_clean":"2024-04-25T11:14:22.361215+0000","last_became_active":"2024-04-25T11:14:21.367177+0000","last_became_peered":"2024-04-25T11:14:21.367177+0000","last_unstale":"2024-04-25T11:14:22.361215+0000","last_undegraded":"2024-04-25T11:14:22.361215+0000","last_fullsized":"2024-04-25T11:14:22.361215+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:57.447950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361409+0000","last_change":"2024-04-25T11:14:20.341156+0000","last_active":"2024-04-25T11:14:22.361409+0000","last_peered":"2024-04-25T11:14:22.361409+0000","last_clean":"2024-04-25T11:14:22.361409+0000","last_became_active":"2024-04-25T11:14:20.340815+0000","last_became_peered":"2024-04-25T11:14:20.340815+0000","last_unstale":"2024-04-25T11:14:22.361409+0000","last_undegraded":"2024-04-25T11:14:22.361409+0000","last_fullsized":"2024-04-25T11:14:22.361409+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:22.063307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490616+0000","last_change":"2024-04-25T11:14:21.370991+0000","last_active":"2024-04-25T11:14:23.490616+0000","last_peered":"2024-04-25T11:14:23.490616+0000","last_clean":"2024-04-25T11:14:23.490616+0000","last_became_active":"2024-04-25T11:14:21.370401+0000","last_became_peered":"2024-04-25T11:14:21.370401+0000","last_unstale":"2024-04-25T11:14:23.490616+0000","last_undegraded":"2024-04-25T11:14:23.490616+0000","last_fullsized":"2024-04-25T11:14:23.490616+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:33.746782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.961780+0000","last_change":"2024-04-25T11:14:20.345177+0000","last_active":"2024-04-25T11:14:22.961780+0000","last_peered":"2024-04-25T11:14:22.961780+0000","last_clean":"2024-04-25T11:14:22.961780+0000","last_became_active":"2024-04-25T11:14:20.344632+0000","last_became_peered":"2024-04-25T11:14:20.344632+0000","last_unstale":"2024-04-25T11:14:22.961780+0000","last_undegraded":"2024-04-25T11:14:22.961780+0000","last_fullsized":"2024-04-25T11:14:22.961780+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:58.724816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962522+0000","last_change":"2024-04-25T11:14:21.380957+0000","last_active":"2024-04-25T11:14:22.962522+0000","last_peered":"2024-04-25T11:14:22.962522+0000","last_clean":"2024-04-25T11:14:22.962522+0000","last_became_active":"2024-04-25T11:14:21.380702+0000","last_became_peered":"2024-04-25T11:14:21.380702+0000","last_unstale":"2024-04-25T11:14:22.962522+0000","last_undegraded":"2024-04-25T11:14:22.962522+0000","last_fullsized":"2024-04-25T11:14:22.962522+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:25.039608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.363401+0000","last_change":"2024-04-25T11:14:20.343699+0000","last_active":"2024-04-25T11:14:22.363401+0000","last_peered":"2024-04-25T11:14:22.363401+0000","last_clean":"2024-04-25T11:14:22.363401+0000","last_became_active":"2024-04-25T11:14:20.343351+0000","last_became_peered":"2024-04-25T11:14:20.343351+0000","last_unstale":"2024-04-25T11:14:22.363401+0000","last_undegraded":"2024-04-25T11:14:22.363401+0000","last_fullsized":"2024-04-25T11:14:22.363401+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:10.206744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.347275+0000","last_change":"2024-04-25T11:14:21.355386+0000","last_active":"2024-04-25T11:14:23.347275+0000","last_peered":"2024-04-25T11:14:23.347275+0000","last_clean":"2024-04-25T11:14:23.347275+0000","last_became_active":"2024-04-25T11:14:21.355092+0000","last_became_peered":"2024-04-25T11:14:21.355092+0000","last_unstale":"2024-04-25T11:14:23.347275+0000","last_undegraded":"2024-04-25T11:14:23.347275+0000","last_fullsized":"2024-04-25T11:14:23.347275+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:55.946442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"68'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.363188+0000","last_change":"2024-04-25T11:14:20.336075+0000","last_active":"2024-04-25T11:14:22.363188+0000","last_peered":"2024-04-25T11:14:22.363188+0000","last_clean":"2024-04-25T11:14:22.363188+0000","last_became_active":"2024-04-25T11:14:20.335712+0000","last_became_peered":"2024-04-25T11:14:20.335712+0000","last_unstale":"2024-04-25T11:14:22.363188+0000","last_undegraded":"2024-04-25T11:14:22.363188+0000","last_fullsized":"2024-04-25T11:14:22.363188+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:21.473511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362716+0000","last_change":"2024-04-25T11:14:21.364681+0000","last_active":"2024-04-25T11:14:22.362716+0000","last_peered":"2024-04-25T11:14:22.362716+0000","last_clean":"2024-04-25T11:14:22.362716+0000","last_became_active":"2024-04-25T11:14:21.364334+0000","last_became_peered":"2024-04-25T11:14:21.364334+0000","last_unstale":"2024-04-25T11:14:22.362716+0000","last_undegraded":"2024-04-25T11:14:22.362716+0000","last_fullsized":"2024-04-25T11:14:22.362716+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:39.364702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.962444+0000","last_change":"2024-04-25T11:14:20.346848+0000","last_active":"2024-04-25T11:14:22.962444+0000","last_peered":"2024-04-25T11:14:22.962444+0000","last_clean":"2024-04-25T11:14:22.962444+0000","last_became_active":"2024-04-25T11:14:20.346439+0000","last_became_peered":"2024-04-25T11:14:20.346439+0000","last_unstale":"2024-04-25T11:14:22.962444+0000","last_undegraded":"2024-04-25T11:14:22.962444+0000","last_fullsized":"2024-04-25T11:14:22.962444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:38.347332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.491183+0000","last_change":"2024-04-25T11:14:21.368766+0000","last_active":"2024-04-25T11:14:23.491183+0000","last_peered":"2024-04-25T11:14:23.491183+0000","last_clean":"2024-04-25T11:14:23.491183+0000","last_became_active":"2024-04-25T11:14:21.368466+0000","last_became_peered":"2024-04-25T11:14:21.368466+0000","last_unstale":"2024-04-25T11:14:23.491183+0000","last_undegraded":"2024-04-25T11:14:23.491183+0000","last_fullsized":"2024-04-25T11:14:23.491183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:09.063896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044726+0000","last_change":"2024-04-25T11:14:20.344137+0000","last_active":"2024-04-25T11:14:24.044726+0000","last_peered":"2024-04-25T11:14:24.044726+0000","last_clean":"2024-04-25T11:14:24.044726+0000","last_became_active":"2024-04-25T11:14:20.343813+0000","last_became_peered":"2024-04-25T11:14:20.343813+0000","last_unstale":"2024-04-25T11:14:24.044726+0000","last_undegraded":"2024-04-25T11:14:24.044726+0000","last_fullsized":"2024-04-25T11:14:24.044726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:03.000623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490040+0000","last_change":"2024-04-25T11:14:21.361869+0000","last_active":"2024-04-25T11:14:23.490040+0000","last_peered":"2024-04-25T11:14:23.490040+0000","last_clean":"2024-04-25T11:14:23.490040+0000","last_became_active":"2024-04-25T11:14:21.358307+0000","last_became_peered":"2024-04-25T11:14:21.358307+0000","last_unstale":"2024-04-25T11:14:23.490040+0000","last_undegraded":"2024-04-25T11:14:23.490040+0000","last_fullsized":"2024-04-25T11:14:23.490040+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:33.222542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.366878+0000","last_change":"2024-04-25T11:14:20.331243+0000","last_active":"2024-04-25T11:14:22.366878+0000","last_peered":"2024-04-25T11:14:22.366878+0000","last_clean":"2024-04-25T11:14:22.366878+0000","last_became_active":"2024-04-25T11:14:20.330860+0000","last_became_peered":"2024-04-25T11:14:20.330860+0000","last_unstale":"2024-04-25T11:14:22.366878+0000","last_undegraded":"2024-04-25T11:14:22.366878+0000","last_fullsized":"2024-04-25T11:14:22.366878+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:32.729412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.539148+0000","last_change":"2024-04-25T11:14:21.390108+0000","last_active":"2024-04-25T11:14:23.539148+0000","last_peered":"2024-04-25T11:14:23.539148+0000","last_clean":"2024-04-25T11:14:23.539148+0000","last_became_active":"2024-04-25T11:14:21.389767+0000","last_became_peered":"2024-04-25T11:14:21.389767+0000","last_unstale":"2024-04-25T11:14:23.539148+0000","last_undegraded":"2024-04-25T11:14:23.539148+0000","last_fullsized":"2024-04-25T11:14:23.539148+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:04.241445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.362445+0000","last_change":"2024-04-25T11:14:20.325726+0000","last_active":"2024-04-25T11:14:22.362445+0000","last_peered":"2024-04-25T11:14:22.362445+0000","last_clean":"2024-04-25T11:14:22.362445+0000","last_became_active":"2024-04-25T11:14:20.325199+0000","last_became_peered":"2024-04-25T11:14:20.325199+0000","last_unstale":"2024-04-25T11:14:22.362445+0000","last_undegraded":"2024-04-25T11:14:22.362445+0000","last_fullsized":"2024-04-25T11:14:22.362445+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:22.963995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.044144+0000","last_change":"2024-04-25T11:14:21.377172+0000","last_active":"2024-04-25T11:14:24.044144+0000","last_peered":"2024-04-25T11:14:24.044144+0000","last_clean":"2024-04-25T11:14:24.044144+0000","last_became_active":"2024-04-25T11:14:21.376648+0000","last_became_peered":"2024-04-25T11:14:21.376648+0000","last_unstale":"2024-04-25T11:14:24.044144+0000","last_undegraded":"2024-04-25T11:14:24.044144+0000","last_fullsized":"2024-04-25T11:14:24.044144+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:49.106485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361947+0000","last_change":"2024-04-25T11:14:20.339034+0000","last_active":"2024-04-25T11:14:22.361947+0000","last_peered":"2024-04-25T11:14:22.361947+0000","last_clean":"2024-04-25T11:14:22.361947+0000","last_became_active":"2024-04-25T11:14:20.338713+0000","last_became_peered":"2024-04-25T11:14:20.338713+0000","last_unstale":"2024-04-25T11:14:22.361947+0000","last_undegraded":"2024-04-25T11:14:22.361947+0000","last_fullsized":"2024-04-25T11:14:22.361947+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:06.914535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.346525+0000","last_change":"2024-04-25T11:14:21.373882+0000","last_active":"2024-04-25T11:14:23.346525+0000","last_peered":"2024-04-25T11:14:23.346525+0000","last_clean":"2024-04-25T11:14:23.346525+0000","last_became_active":"2024-04-25T11:14:21.373533+0000","last_became_peered":"2024-04-25T11:14:21.373533+0000","last_unstale":"2024-04-25T11:14:23.346525+0000","last_undegraded":"2024-04-25T11:14:23.346525+0000","last_fullsized":"2024-04-25T11:14:23.346525+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:01.628406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:24.043503+0000","last_change":"2024-04-25T11:14:20.339826+0000","last_active":"2024-04-25T11:14:24.043503+0000","last_peered":"2024-04-25T11:14:24.043503+0000","last_clean":"2024-04-25T11:14:24.043503+0000","last_became_active":"2024-04-25T11:14:20.339366+0000","last_became_peered":"2024-04-25T11:14:20.339366+0000","last_unstale":"2024-04-25T11:14:24.043503+0000","last_undegraded":"2024-04-25T11:14:24.043503+0000","last_fullsized":"2024-04-25T11:14:24.043503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:15.992964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:23.490643+0000","last_change":"2024-04-25T11:14:21.363893+0000","last_active":"2024-04-25T11:14:23.490643+0000","last_peered":"2024-04-25T11:14:23.490643+0000","last_clean":"2024-04-25T11:14:23.490643+0000","last_became_active":"2024-04-25T11:14:21.363555+0000","last_became_peered":"2024-04-25T11:14:21.363555+0000","last_unstale":"2024-04-25T11:14:23.490643+0000","last_undegraded":"2024-04-25T11:14:23.490643+0000","last_fullsized":"2024-04-25T11:14:23.490643+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:20.290926+0000","last_clean_scrub_stamp":"2024-04-25T11:14:20.290926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:25.589218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.361619+0000","last_change":"2024-04-25T11:14:20.334793+0000","last_active":"2024-04-25T11:14:22.361619+0000","last_peered":"2024-04-25T11:14:22.361619+0000","last_clean":"2024-04-25T11:14:22.361619+0000","last_became_active":"2024-04-25T11:14:20.333198+0000","last_became_peered":"2024-04-25T11:14:20.333198+0000","last_unstale":"2024-04-25T11:14:22.361619+0000","last_undegraded":"2024-04-25T11:14:22.361619+0000","last_fullsized":"2024-04-25T11:14:22.361619+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:14:19.273556+0000","last_clean_scrub_stamp":"2024-04-25T11:14:19.273556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:08.559248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":203,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T11:14:22.365324+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:14:22.365324+0000","last_peered":"2024-04-25T11:14:22.365324+0000","last_clean":"2024-04-25T11:14:22.365324+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:14:22.365324+0000","last_undegraded":"2024-04-25T11:14:22.365324+0000","last_fullsized":"2024-04-25T11:14:22.365324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476790,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1421312,"data_stored":957554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476791,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":815104,"data_stored":366444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476790,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":366432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476791,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":368022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":366410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509495,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":368786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32560,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339280,"statfs":{"total":96636764160,"available":96603422720,"internally_reserved":0,"allocated":1404928,"data_stored":956778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":811008,"data_stored":366410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:14:32.062 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:14:32.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:14:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.399+0000 7f81b6400640 1 Processor -- start 2024-04-25T11:14:32.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.399+0000 7f81b6400640 1 -- start start 2024-04-25T11:14:32.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.402+0000 7f81b6400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81b00ce8d0 0x7f81b00cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:32.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.402+0000 7f81b6400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f81b006fe40 0x7f81b015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:32.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.402+0000 7f81b6400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81b00b4830 con 0x7f81b015c2e0 2024-04-25T11:14:32.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.402+0000 7f81b6400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f81b00b6600 con 0x7f81b006fe40 2024-04-25T11:14:32.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.402+0000 7f81b6400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f81b00b4b10 con 0x7f81b00ce8d0 2024-04-25T11:14:32.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.402+0000 7f81b5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f81b006fe40 0x7f81b015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:32.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.402+0000 7f81afe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f81b015c2e0 0x7f81b006f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43954/0 (socket says 172.21.15.7:43954) 2024-04-25T11:14:32.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.402+0000 7f81afe00640 1 -- 172.21.15.7:0/3044716690 learned_addr learned my addr 172.21.15.7:0/3044716690 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:32.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.403+0000 7f81b5000640 1 --2- 172.21.15.7:0/3044716690 >> v2:172.21.15.87:3300/0 conn(0x7f81b00ce8d0 0x7f81b00cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:32.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.403+0000 7f81b5a00640 1 -- 172.21.15.7:0/3044716690 >> v2:172.21.15.87:3300/0 conn(0x7f81b00ce8d0 msgr2=0x7f81b00cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:32.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.403+0000 7f81b5a00640 1 --2- 172.21.15.7:0/3044716690 >> v2:172.21.15.87:3300/0 conn(0x7f81b00ce8d0 0x7f81b00cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.403+0000 7f81b5a00640 1 -- 172.21.15.7:0/3044716690 >> v1:172.21.15.7:6789/0 conn(0x7f81b015c2e0 legacy=0x7f81b006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:32.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.403+0000 7f81b5a00640 1 -- 172.21.15.7:0/3044716690 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b00b4df0 con 0x7f81b006fe40 2024-04-25T11:14:32.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.403+0000 7f81b5000640 1 --2- 172.21.15.7:0/3044716690 >> v2:172.21.15.87:3300/0 conn(0x7f81b00ce8d0 0x7f81b00cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:14:32.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.403+0000 7f81af400640 1 -- 172.21.15.7:0/3044716690 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2340261150 0 0) 0x7f81b00b4830 con 0x7f81b015c2e0 2024-04-25T11:14:32.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.404+0000 7f81b5a00640 1 --2- 172.21.15.7:0/3044716690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b006fe40 0x7f81b015c6f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f81b0072330 tx=0x7f81a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4165c9a3fdd3e33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:32.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.404+0000 7f81af400640 1 -- 172.21.15.7:0/3044716690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8037070 con 0x7f81b006fe40 2024-04-25T11:14:32.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.404+0000 7f81af400640 1 -- 172.21.15.7:0/3044716690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a8032440 con 0x7f81b006fe40 2024-04-25T11:14:32.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.404+0000 7f81af400640 1 -- 172.21.15.7:0/3044716690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8032810 con 0x7f81b006fe40 2024-04-25T11:14:32.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.405+0000 7f81b6400640 1 -- 172.21.15.7:0/3044716690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b006fe40 msgr2=0x7f81b015c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:32.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.405+0000 7f81b6400640 1 --2- 172.21.15.7:0/3044716690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b006fe40 0x7f81b015c6f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f81b0072330 tx=0x7f81a802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.405+0000 7f81b6400640 1 -- 172.21.15.7:0/3044716690 shutdown_connections 2024-04-25T11:14:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.405+0000 7f81b6400640 1 --2- 172.21.15.7:0/3044716690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b006fe40 0x7f81b015c6f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.405+0000 7f81b6400640 1 --2- 172.21.15.7:0/3044716690 >> v2:172.21.15.87:3300/0 conn(0x7f81b00ce8d0 0x7f81b00cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.405+0000 7f81b6400640 1 -- 172.21.15.7:0/3044716690 >> 172.21.15.7:0/3044716690 conn(0x7f81b00740f0 msgr2=0x7f81b006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.405+0000 7f81b6400640 1 -- 172.21.15.7:0/3044716690 shutdown_connections 2024-04-25T11:14:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.406+0000 7f81b6400640 1 -- 172.21.15.7:0/3044716690 wait complete. 2024-04-25T11:14:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.408+0000 7f81b6400640 1 Processor -- start 2024-04-25T11:14:32.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.408+0000 7f81b6400640 1 -- start start 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.409+0000 7f81b6400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81b00ce8d0 0x7f81b015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81b6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015c2e0 0x7f81b0149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81b6400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0156490 0x7f81b014ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81b6400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0070ff0 con 0x7f81b015c2e0 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81b6400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0070e70 con 0x7f81b0156490 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81b6400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f81b006eea0 con 0x7f81b00ce8d0 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015c2e0 0x7f81b0149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015c2e0 0x7f81b0149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37956/0 (socket says 172.21.15.7:37956) 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81afe00640 1 -- 172.21.15.7:0/3808743560 learned_addr learned my addr 172.21.15.7:0/3808743560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81b5000640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81b00ce8d0 0x7f81b015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81b5a00640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0156490 0x7f81b014ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81afe00640 1 -- 172.21.15.7:0/3808743560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81b00ce8d0 msgr2=0x7f81b015aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:32.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81afe00640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81b00ce8d0 0x7f81b015aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81afe00640 1 -- 172.21.15.7:0/3808743560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0156490 msgr2=0x7f81b014ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:32.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81afe00640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0156490 0x7f81b014ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81afe00640 1 -- 172.21.15.7:0/3808743560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b015b440 con 0x7f81b015c2e0 2024-04-25T11:14:32.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.410+0000 7f81b5000640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81b00ce8d0 0x7f81b015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:32.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.411+0000 7f81afe00640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015c2e0 0x7f81b0149240 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f819800c7b0 tx=0x7f819800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.411+0000 7f81acc00640 1 -- 172.21.15.7:0/3808743560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198018070 con 0x7f81b015c2e0 2024-04-25T11:14:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.411+0000 7f81acc00640 1 -- 172.21.15.7:0/3808743560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819800a640 con 0x7f81b015c2e0 2024-04-25T11:14:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.411+0000 7f81acc00640 1 -- 172.21.15.7:0/3808743560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819800aa30 con 0x7f81b015c2e0 2024-04-25T11:14:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.411+0000 7f81b6400640 1 -- 172.21.15.7:0/3808743560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b015b720 con 0x7f81b015c2e0 2024-04-25T11:14:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.411+0000 7f81b6400640 1 -- 172.21.15.7:0/3808743560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b015ee10 con 0x7f81b015c2e0 2024-04-25T11:14:32.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.413+0000 7f81b6400640 1 -- 172.21.15.7:0/3808743560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b0050740 con 0x7f81b015c2e0 2024-04-25T11:14:32.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.414+0000 7f81acc00640 1 -- 172.21.15.7:0/3808743560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f819800abd0 con 0x7f81b015c2e0 2024-04-25T11:14:32.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.415+0000 7f81acc00640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f818c0744e0 0x7f818c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:32.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.415+0000 7f81acc00640 1 -- 172.21.15.7:0/3808743560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f8198094950 con 0x7f81b015c2e0 2024-04-25T11:14:32.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.415+0000 7f81b5000640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f818c0744e0 0x7f818c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:32.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.416+0000 7f81b5000640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f818c0744e0 0x7f818c0769f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f81a0006fd0 tx=0x7f81a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:32.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.420+0000 7f81acc00640 1 -- 172.21.15.7:0/3808743560 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f819805ef40 con 0x7f81b015c2e0 2024-04-25T11:14:32.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.699+0000 7f81b6400640 1 -- 172.21.15.7:0/3808743560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f81b00709b0 con 0x7f81b015c2e0 2024-04-25T11:14:32.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.704+0000 7f81acc00640 1 -- 172.21.15.7:0/3808743560 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f819805e840 con 0x7f81b015c2e0 2024-04-25T11:14:32.705 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-25T11:14:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:14:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-04-25T11:14:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-04-25T11:14:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:14:32.705 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.707+0000 7f81b6400640 1 -- 172.21.15.7:0/3808743560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f818c0744e0 msgr2=0x7f818c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.707+0000 7f81b6400640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f818c0744e0 0x7f818c0769f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f81a0006fd0 tx=0x7f81a0008040 comp rx=0 tx=0).stop 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.707+0000 7f81b6400640 1 -- 172.21.15.7:0/3808743560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015c2e0 msgr2=0x7f81b0149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.707+0000 7f81b6400640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015c2e0 0x7f81b0149240 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f819800c7b0 tx=0x7f819800cc70 comp rx=0 tx=0).stop 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.707+0000 7f81b6400640 1 -- 172.21.15.7:0/3808743560 shutdown_connections 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.707+0000 7f81b6400640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f818c0744e0 0x7f818c0769f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.707+0000 7f81b6400640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b0156490 0x7f81b014ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.707+0000 7f81b6400640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015c2e0 0x7f81b0149240 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.707+0000 7f81b6400640 1 --2- 172.21.15.7:0/3808743560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81b00ce8d0 0x7f81b015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.707+0000 7f81b6400640 1 -- 172.21.15.7:0/3808743560 >> 172.21.15.7:0/3808743560 conn(0x7f81b00740f0 msgr2=0x7f81b006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.708+0000 7f81b6400640 1 -- 172.21.15.7:0/3808743560 shutdown_connections 2024-04-25T11:14:32.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:32.708+0000 7f81b6400640 1 -- 172.21.15.7:0/3808743560 wait complete. 2024-04-25T11:14:32.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:33.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.027+0000 7f2473200640 1 Processor -- start 2024-04-25T11:14:33.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.027+0000 7f2473200640 1 -- start start 2024-04-25T11:14:33.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.029+0000 7f2473200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24740b9010 0x7f2474161340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:33.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.029+0000 7f2473200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f24740b2ae0 0x7f247409b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:33.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.029+0000 7f2473200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f247409c240 con 0x7f247409be30 2024-04-25T11:14:33.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.029+0000 7f2473200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2474161880 con 0x7f24740b9010 2024-04-25T11:14:33.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.029+0000 7f2473200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2474160740 con 0x7f24740b2ae0 2024-04-25T11:14:33.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.029+0000 7f2471e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24740b9010 0x7f2474161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:33.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.029+0000 7f2472800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f247409be30 0x7f2474163e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43968/0 (socket says 172.21.15.7:43968) 2024-04-25T11:14:33.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.029+0000 7f2472800640 1 -- 172.21.15.7:0/597568868 learned_addr learned my addr 172.21.15.7:0/597568868 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:33.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.029+0000 7f246be00640 1 --2- 172.21.15.7:0/597568868 >> v2:172.21.15.87:3300/0 conn(0x7f24740b2ae0 0x7f247409b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.030+0000 7f2471e00640 1 -- 172.21.15.7:0/597568868 >> v2:172.21.15.87:3300/0 conn(0x7f24740b2ae0 msgr2=0x7f247409b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.030+0000 7f2471e00640 1 --2- 172.21.15.7:0/597568868 >> v2:172.21.15.87:3300/0 conn(0x7f24740b2ae0 0x7f247409b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.030+0000 7f2471e00640 1 -- 172.21.15.7:0/597568868 >> v1:172.21.15.7:6789/0 conn(0x7f247409be30 legacy=0x7f2474163e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.030+0000 7f2471e00640 1 -- 172.21.15.7:0/597568868 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247406dc20 con 0x7f24740b9010 2024-04-25T11:14:33.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.030+0000 7f2471400640 1 -- 172.21.15.7:0/597568868 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1397836708 0 0) 0x7f246c001490 con 0x7f247409be30 2024-04-25T11:14:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.031+0000 7f2471e00640 1 --2- 172.21.15.7:0/597568868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24740b9010 0x7f2474161340 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f24740d47d0 tx=0x7f246402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9508cebc16585ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.031+0000 7f2471400640 1 -- 172.21.15.7:0/597568868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464037070 con 0x7f24740b9010 2024-04-25T11:14:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.031+0000 7f2471400640 1 -- 172.21.15.7:0/597568868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2464032440 con 0x7f24740b9010 2024-04-25T11:14:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.031+0000 7f2471400640 1 -- 172.21.15.7:0/597568868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246402ae60 con 0x7f24740b9010 2024-04-25T11:14:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.032+0000 7f2473200640 1 -- 172.21.15.7:0/597568868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24740b9010 msgr2=0x7f2474161340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:33.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.032+0000 7f2473200640 1 --2- 172.21.15.7:0/597568868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24740b9010 0x7f2474161340 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f24740d47d0 tx=0x7f246402a110 comp rx=0 tx=0).stop 2024-04-25T11:14:33.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.032+0000 7f2473200640 1 -- 172.21.15.7:0/597568868 shutdown_connections 2024-04-25T11:14:33.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.032+0000 7f2473200640 1 --2- 172.21.15.7:0/597568868 >> v2:172.21.15.87:3300/0 conn(0x7f24740b2ae0 0x7f247409b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:33.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.032+0000 7f2473200640 1 --2- 172.21.15.7:0/597568868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24740b9010 0x7f2474161340 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:33.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.032+0000 7f2473200640 1 -- 172.21.15.7:0/597568868 >> 172.21.15.7:0/597568868 conn(0x7f24740b5210 msgr2=0x7f24740b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:33.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.032+0000 7f2473200640 1 -- 172.21.15.7:0/597568868 shutdown_connections 2024-04-25T11:14:33.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.032+0000 7f2473200640 1 -- 172.21.15.7:0/597568868 wait complete. 2024-04-25T11:14:33.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.035+0000 7f2473200640 1 Processor -- start 2024-04-25T11:14:33.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.035+0000 7f2473200640 1 -- start start 2024-04-25T11:14:33.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.036+0000 7f2473200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f247409be30 0x7f24741e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:33.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.036+0000 7f2473200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24740b2ae0 0x7f24741e7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:33.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.036+0000 7f2473200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24740b9010 0x7f247414fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:33.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.036+0000 7f2473200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f247406e4c0 con 0x7f24740b9010 2024-04-25T11:14:33.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.036+0000 7f2473200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f247406e1e0 con 0x7f247409be30 2024-04-25T11:14:33.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.036+0000 7f2473200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f247406e7a0 con 0x7f24740b2ae0 2024-04-25T11:14:33.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.037+0000 7f2472800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24740b9010 0x7f247414fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:33.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.037+0000 7f2472800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24740b9010 0x7f247414fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37970/0 (socket says 172.21.15.7:37970) 2024-04-25T11:14:33.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.037+0000 7f2472800640 1 -- 172.21.15.7:0/1663263075 learned_addr learned my addr 172.21.15.7:0/1663263075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:33.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.037+0000 7f2471e00640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f247409be30 0x7f24741e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:33.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.037+0000 7f246be00640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24740b2ae0 0x7f24741e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:33.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.037+0000 7f2472800640 1 -- 172.21.15.7:0/1663263075 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24740b2ae0 msgr2=0x7f24741e7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:33.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.037+0000 7f2472800640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24740b2ae0 0x7f24741e7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:33.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.037+0000 7f2472800640 1 -- 172.21.15.7:0/1663263075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f247409be30 msgr2=0x7f24741e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:33.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.037+0000 7f2472800640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f247409be30 0x7f24741e7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:33.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.037+0000 7f2472800640 1 -- 172.21.15.7:0/1663263075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24741e83b0 con 0x7f24740b9010 2024-04-25T11:14:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.038+0000 7f2472800640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24740b9010 0x7f247414fd70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f246c00c6d0 tx=0x7f246c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.038+0000 7f246a000640 1 -- 172.21.15.7:0/1663263075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c018070 con 0x7f24740b9010 2024-04-25T11:14:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.038+0000 7f246a000640 1 -- 172.21.15.7:0/1663263075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c004d60 con 0x7f24740b9010 2024-04-25T11:14:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.038+0000 7f246a000640 1 -- 172.21.15.7:0/1663263075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c00a880 con 0x7f24740b9010 2024-04-25T11:14:33.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.038+0000 7f2473200640 1 -- 172.21.15.7:0/1663263075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24741e8690 con 0x7f24740b9010 2024-04-25T11:14:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.038+0000 7f2473200640 1 -- 172.21.15.7:0/1663263075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24741e8c50 con 0x7f24740b9010 2024-04-25T11:14:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.039+0000 7f2473200640 1 -- 172.21.15.7:0/1663263075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2474050740 con 0x7f24740b9010 2024-04-25T11:14:33.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.041+0000 7f246a000640 1 -- 172.21.15.7:0/1663263075 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f246c00ac30 con 0x7f24740b9010 2024-04-25T11:14:33.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.042+0000 7f246a000640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24500744b0 0x7f24500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:33.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.042+0000 7f246a000640 1 -- 172.21.15.7:0/1663263075 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f246c094b40 con 0x7f24740b9010 2024-04-25T11:14:33.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.042+0000 7f2471e00640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24500744b0 0x7f24500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:33.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.043+0000 7f2471e00640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24500744b0 0x7f24500769c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2464034000 tx=0x7f246402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:33.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.047+0000 7f246a000640 1 -- 172.21.15.7:0/1663263075 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f247409c240 con 0x7f24740b9010 2024-04-25T11:14:33.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.287+0000 7f2473200640 1 -- 172.21.15.7:0/1663263075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f247409c730 con 0x7f24740b9010 2024-04-25T11:14:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.289+0000 7f246a000640 1 -- 172.21.15.7:0/1663263075 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7f246c05f1d0 con 0x7f24740b9010 2024-04-25T11:14:33.289 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:33.289 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2941354720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2941354720},{"type":"v1","addr":"172.21.15.87:6834","nonce":2941354720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3135576784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3135576784},{"type":"v1","addr":"172.21.15.87:6835","nonce":3135576784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2590173274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2590173274},{"type":"v1","addr":"172.21.15.7:6836","nonce":2590173274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:14:20.877428+0000","modified":"2024-04-25T11:14:23.977883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6120":{"gid":6120,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2403088100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2403088100},{"type":"v1","addr":"172.21.15.7:6837","nonce":2403088100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6120,"qdb_cluster":[6120]},"id":4}]} 2024-04-25T11:14:33.289 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-04-25T11:14:33.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.291+0000 7f2473200640 1 -- 172.21.15.7:0/1663263075 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24500744b0 msgr2=0x7f24500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:33.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.291+0000 7f2473200640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24500744b0 0x7f24500769c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2464034000 tx=0x7f246402e000 comp rx=0 tx=0).stop 2024-04-25T11:14:33.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.291+0000 7f2473200640 1 -- 172.21.15.7:0/1663263075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24740b9010 msgr2=0x7f247414fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:33.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.291+0000 7f2473200640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24740b9010 0x7f247414fd70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f246c00c6d0 tx=0x7f246c00cb90 comp rx=0 tx=0).stop 2024-04-25T11:14:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.291+0000 7f2473200640 1 -- 172.21.15.7:0/1663263075 shutdown_connections 2024-04-25T11:14:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.291+0000 7f2473200640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24500744b0 0x7f24500769c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.291+0000 7f2473200640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24740b9010 0x7f247414fd70 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.291+0000 7f2473200640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24740b2ae0 0x7f24741e7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.291+0000 7f2473200640 1 --2- 172.21.15.7:0/1663263075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f247409be30 0x7f24741e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.291+0000 7f2473200640 1 -- 172.21.15.7:0/1663263075 >> 172.21.15.7:0/1663263075 conn(0x7f24740b5210 msgr2=0x7f24740b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.291+0000 7f2473200640 1 -- 172.21.15.7:0/1663263075 shutdown_connections 2024-04-25T11:14:33.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:33.292+0000 7f2473200640 1 -- 172.21.15.7:0/1663263075 wait complete. 2024-04-25T11:14:33.720 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:14:20.877428+0000', 'modified': '2024-04-25T11:14:23.977883+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6120}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6120': {'gid': 6120, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6837/2403088100', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6835', 'nonce': 2403088100}, {'type': 'v1', 'addr': '172.21.15.7:6837', 'nonce': 2403088100}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6120, 'qdb_cluster': [6120]} 2024-04-25T11:14:33.720 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:14:33.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:14:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.008+0000 7f07db200640 1 Processor -- start 2024-04-25T11:14:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.008+0000 7f07db200640 1 -- start start 2024-04-25T11:14:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.010+0000 7f07db200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f07d406c570 0x7f07d40b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.011+0000 7f07db200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07d40b5c20 con 0x7f07d406f3f0 2024-04-25T11:14:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.011+0000 7f07db200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07d40b5f00 con 0x7f07d40d0330 2024-04-25T11:14:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.011+0000 7f07db200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f07d40b61e0 con 0x7f07d406c570 2024-04-25T11:14:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.011+0000 7f07da800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f07d406f3f0 0x7f07d406f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43976/0 (socket says 172.21.15.7:43976) 2024-04-25T11:14:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.011+0000 7f07da800640 1 -- 172.21.15.7:0/1351296350 learned_addr learned my addr 172.21.15.7:0/1351296350 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.011+0000 7f07d9e00640 1 --2- 172.21.15.7:0/1351296350 >> v2:172.21.15.87:3300/0 conn(0x7f07d406c570 0x7f07d40b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.011+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911270884 0 0) 0x7f07d40b5c20 con 0x7f07d406f3f0 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07bc003520 con 0x7f07d406f3f0 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1865139846 0 0) 0x7f07bc003520 con 0x7f07d406f3f0 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 >> v2:172.21.15.87:3300/0 conn(0x7f07d406c570 msgr2=0x7f07d40b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 --2- 172.21.15.7:0/1351296350 >> v2:172.21.15.87:3300/0 conn(0x7f07d406c570 0x7f07d40b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 >> v1:172.21.15.87:6789/0 conn(0x7f07d40d0330 legacy=0x7f07d40d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d40b64c0 con 0x7f07d406f3f0 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f07d00019f0 con 0x7f07d406f3f0 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 >> v1:172.21.15.7:6789/0 conn(0x7f07d406f3f0 legacy=0x7f07d406f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 --2- 172.21.15.7:0/1351296350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07bc0039b0 0x7f07bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 --2- 172.21.15.7:0/1351296350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 0x7f07bc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 --2- 172.21.15.7:0/1351296350 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07bc0051a0 0x7f07bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07d40b61e0 con 0x7f07bc004800 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc00c140 con 0x7f07bc0039b0 2024-04-25T11:14:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f07bc00c340 con 0x7f07bc0051a0 2024-04-25T11:14:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.012+0000 7f07d9400640 1 --2- 172.21.15.7:0/1351296350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 0x7f07bc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.013+0000 7f07d9e00640 1 --2- 172.21.15.7:0/1351296350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07bc0039b0 0x7f07bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.013+0000 7f07da800640 1 --2- 172.21.15.7:0/1351296350 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07bc0051a0 0x7f07bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.013+0000 7f07d9400640 1 -- 172.21.15.7:0/1351296350 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07bc0051a0 msgr2=0x7f07bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.013+0000 7f07d9400640 1 --2- 172.21.15.7:0/1351296350 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07bc0051a0 0x7f07bc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.013+0000 7f07d9400640 1 -- 172.21.15.7:0/1351296350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07bc0039b0 msgr2=0x7f07bc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.013+0000 7f07d9400640 1 --2- 172.21.15.7:0/1351296350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07bc0039b0 0x7f07bc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.013+0000 7f07d9400640 1 -- 172.21.15.7:0/1351296350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc003520 con 0x7f07bc004800 2024-04-25T11:14:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.013+0000 7f07da800640 1 -- 172.21.15.7:0/1351296350 reap_dead start 2024-04-25T11:14:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.014+0000 7f07d9400640 1 --2- 172.21.15.7:0/1351296350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 0x7f07bc004c10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f07bc004780 tx=0x7f07bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b19855f44a284af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.014+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4037070 con 0x7f07bc004800 2024-04-25T11:14:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.014+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c402a940 con 0x7f07bc004800 2024-04-25T11:14:34.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.014+0000 7f07d8a00640 1 -- 172.21.15.7:0/1351296350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c40039a0 con 0x7f07bc004800 2024-04-25T11:14:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.015+0000 7f07db200640 1 -- 172.21.15.7:0/1351296350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 msgr2=0x7f07bc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.015+0000 7f07db200640 1 --2- 172.21.15.7:0/1351296350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 0x7f07bc004c10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f07bc004780 tx=0x7f07bc005830 comp rx=0 tx=0).stop 2024-04-25T11:14:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.015+0000 7f07db200640 1 -- 172.21.15.7:0/1351296350 shutdown_connections 2024-04-25T11:14:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.015+0000 7f07db200640 1 --2- 172.21.15.7:0/1351296350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 0x7f07bc004c10 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.015+0000 7f07db200640 1 -- 172.21.15.7:0/1351296350 >> 172.21.15.7:0/1351296350 conn(0x7f07d40d1540 msgr2=0x7f07d40d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.015+0000 7f07db200640 1 -- 172.21.15.7:0/1351296350 shutdown_connections 2024-04-25T11:14:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.015+0000 7f07db200640 1 -- 172.21.15.7:0/1351296350 wait complete. 2024-04-25T11:14:34.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.018+0000 7f07db200640 1 Processor -- start 2024-04-25T11:14:34.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.018+0000 7f07db200640 1 -- start start 2024-04-25T11:14:34.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.020+0000 7f07db200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 0x7f07d41d5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.020+0000 7f07db200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07d41e90f0 0x7f07d41e9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.020+0000 7f07db200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07d41d7c10 0x7f07d41d8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.020+0000 7f07db200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07d4143420 con 0x7f07bc004800 2024-04-25T11:14:34.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.020+0000 7f07db200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07d41432a0 con 0x7f07d41e90f0 2024-04-25T11:14:34.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.020+0000 7f07db200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f07d40d0740 con 0x7f07d41d7c10 2024-04-25T11:14:34.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.020+0000 7f07d9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 0x7f07d41d5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.020+0000 7f07d9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 0x7f07d41d5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37994/0 (socket says 172.21.15.7:37994) 2024-04-25T11:14:34.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.020+0000 7f07d9e00640 1 -- 172.21.15.7:0/1312249627 learned_addr learned my addr 172.21.15.7:0/1312249627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.021+0000 7f07d9400640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07d41e90f0 0x7f07d41e9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.021+0000 7f07d9e00640 1 -- 172.21.15.7:0/1312249627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07d41d7c10 msgr2=0x7f07d41d8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.021+0000 7f07da800640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07d41d7c10 0x7f07d41d8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.021+0000 7f07d9e00640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07d41d7c10 0x7f07d41d8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.021+0000 7f07d9e00640 1 -- 172.21.15.7:0/1312249627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07d41e90f0 msgr2=0x7f07d41e9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.021+0000 7f07d9e00640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07d41e90f0 0x7f07d41e9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.021+0000 7f07d9e00640 1 -- 172.21.15.7:0/1312249627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d41d8560 con 0x7f07bc004800 2024-04-25T11:14:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.021+0000 7f07d9400640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07d41e90f0 0x7f07d41e9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.021+0000 7f07d9e00640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 0x7f07d41d5420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f07c8011620 tx=0x7f07c8011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.021+0000 7f07ce000640 1 -- 172.21.15.7:0/1312249627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c800f920 con 0x7f07bc004800 2024-04-25T11:14:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.021+0000 7f07ce000640 1 -- 172.21.15.7:0/1312249627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c800fac0 con 0x7f07bc004800 2024-04-25T11:14:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.022+0000 7f07db200640 1 -- 172.21.15.7:0/1312249627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d41d5dd0 con 0x7f07bc004800 2024-04-25T11:14:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.022+0000 7f07ce000640 1 -- 172.21.15.7:0/1312249627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c800fe90 con 0x7f07bc004800 2024-04-25T11:14:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.022+0000 7f07db200640 1 -- 172.21.15.7:0/1312249627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d41d6390 con 0x7f07bc004800 2024-04-25T11:14:34.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.024+0000 7f07ce000640 1 -- 172.21.15.7:0/1312249627 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f07c8015460 con 0x7f07bc004800 2024-04-25T11:14:34.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.026+0000 7f07ce000640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07b8074650 0x7f07b8076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.026+0000 7f07ce000640 1 -- 172.21.15.7:0/1312249627 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f07c8095df0 con 0x7f07bc004800 2024-04-25T11:14:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.026+0000 7f07d9400640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07b8074650 0x7f07b8076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.026+0000 7f07e37f2740 1 Processor -- start 2024-04-25T11:14:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.027+0000 7f07d9400640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07b8074650 0x7f07b8076b60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f07c4034000 tx=0x7f07c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.027+0000 7f07e37f2740 1 -- start start 2024-04-25T11:14:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.028+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fdb270 0x55f0353a8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.028+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fb23b0 0x55f0353c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.028+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fba590 0x55f0353c3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.028+0000 7f07e37f2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f0352e1600 con 0x55f034fb23b0 2024-04-25T11:14:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.028+0000 7f07e37f2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f03527e100 con 0x55f034fdb270 2024-04-25T11:14:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.028+0000 7f07e37f2740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55f0352dd5c0 con 0x55f034fba590 2024-04-25T11:14:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.028+0000 7f07d9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fb23b0 0x55f0353c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.028+0000 7f07d9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fb23b0 0x55f0353c2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38000/0 (socket says 172.21.15.7:38000) 2024-04-25T11:14:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.028+0000 7f07d9400640 1 -- 172.21.15.7:0/4197660563 learned_addr learned my addr 172.21.15.7:0/4197660563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07da800640 1 --2- 172.21.15.7:0/4197660563 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fba590 0x55f0353c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07d9e00640 1 --2- 172.21.15.7:0/4197660563 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fdb270 0x55f0353a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07d9400640 1 -- 172.21.15.7:0/4197660563 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fba590 msgr2=0x55f0353c3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07d9400640 1 --2- 172.21.15.7:0/4197660563 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fba590 0x55f0353c3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07d9400640 1 -- 172.21.15.7:0/4197660563 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fdb270 msgr2=0x55f0353a8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07d9400640 1 --2- 172.21.15.7:0/4197660563 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fdb270 0x55f0353a8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07d9400640 1 -- 172.21.15.7:0/4197660563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f034ff32d0 con 0x55f034fb23b0 2024-04-25T11:14:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07d9e00640 1 --2- 172.21.15.7:0/4197660563 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fdb270 0x55f0353a8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07d9400640 1 --2- 172.21.15.7:0/4197660563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fb23b0 0x55f0353c2f40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f07c402ac50 tx=0x7f07c402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb609ffa9bcf838b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07c2600640 1 -- 172.21.15.7:0/4197660563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4037040 con 0x55f034fb23b0 2024-04-25T11:14:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07c2600640 1 -- 172.21.15.7:0/4197660563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c4007530 con 0x55f034fb23b0 2024-04-25T11:14:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.029+0000 7f07c2600640 1 -- 172.21.15.7:0/4197660563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c40069d0 con 0x55f034fb23b0 2024-04-25T11:14:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.030+0000 7f07e37f2740 1 -- 172.21.15.7:0/4197660563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fb23b0 msgr2=0x55f0353c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.030+0000 7f07e37f2740 1 --2- 172.21.15.7:0/4197660563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fb23b0 0x55f0353c2f40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f07c402ac50 tx=0x7f07c402c000 comp rx=0 tx=0).stop 2024-04-25T11:14:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.030+0000 7f07e37f2740 1 -- 172.21.15.7:0/4197660563 shutdown_connections 2024-04-25T11:14:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.030+0000 7f07e37f2740 1 --2- 172.21.15.7:0/4197660563 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fba590 0x55f0353c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.030+0000 7f07e37f2740 1 --2- 172.21.15.7:0/4197660563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fb23b0 0x55f0353c2f40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.030+0000 7f07e37f2740 1 --2- 172.21.15.7:0/4197660563 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fdb270 0x55f0353a8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.030+0000 7f07e37f2740 1 -- 172.21.15.7:0/4197660563 >> 172.21.15.7:0/4197660563 conn(0x55f034fcf3c0 msgr2=0x55f03537d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.030+0000 7f07e37f2740 1 -- 172.21.15.7:0/4197660563 shutdown_connections 2024-04-25T11:14:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.030+0000 7f07e37f2740 1 -- 172.21.15.7:0/4197660563 wait complete. 2024-04-25T11:14:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.031+0000 7f07e37f2740 1 -- start start 2024-04-25T11:14:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.032+0000 7f07e37f2740 1 Processor -- start 2024-04-25T11:14:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.033+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fb23b0 0x55f0353c2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.033+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fba590 0x55f0353bfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.033+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f0353ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.033+0000 7f07e37f2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f035268f70 con 0x55f034fdb270 2024-04-25T11:14:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.033+0000 7f07e37f2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f035399250 con 0x55f034fba590 2024-04-25T11:14:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.033+0000 7f07e37f2740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55f0353c0200 con 0x55f034fb23b0 2024-04-25T11:14:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.033+0000 7f07e37f2740 10 client.0 _ll_register_callbacks cb 0x55f0352d7480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:14:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07da800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f0353ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07da800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f0353ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38008/0 (socket says 172.21.15.7:38008) 2024-04-25T11:14:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07da800640 1 -- 172.21.15.7:0/2177871087 learned_addr learned my addr 172.21.15.7:0/2177871087 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07d9400640 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fba590 0x55f0353bfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07d9e00640 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fb23b0 0x55f0353c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07da800640 1 -- 172.21.15.7:0/2177871087 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fb23b0 msgr2=0x55f0353c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07da800640 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fb23b0 0x55f0353c2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07da800640 1 -- 172.21.15.7:0/2177871087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fba590 msgr2=0x55f0353bfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07da800640 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fba590 0x55f0353bfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07da800640 1 -- 172.21.15.7:0/2177871087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0350bd6b0 con 0x55f034fdb270 2024-04-25T11:14:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07d9400640 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fba590 0x55f0353bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07da800640 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f0353ab140 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f07d4073040 tx=0x7f07d000fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6944ebc4b2936886 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.034+0000 7f07b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:14:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07b6a00640 1 -- 172.21.15.7:0/2177871087 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0004d70 con 0x55f034fdb270 2024-04-25T11:14:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07b6a00640 1 -- 172.21.15.7:0/2177871087 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f07d00158b0 con 0x55f034fdb270 2024-04-25T11:14:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07e37f2740 1 -- 172.21.15.7:0/2177871087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07c4007530 con 0x55f034fdb270 2024-04-25T11:14:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07b6a00640 1 -- 172.21.15.7:0/2177871087 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f07d0015ab0 con 0x55f034fdb270 2024-04-25T11:14:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07e37f2740 10 client.6285 fetch_fsmap learned FSMap version 35 2024-04-25T11:14:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07e37f2740 1 -- 172.21.15.7:0/2177871087 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55f0353c14f0 con 0x55f034fdb270 2024-04-25T11:14:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07b6a00640 1 -- 172.21.15.7:0/2177871087 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c4007530 con 0x55f034fdb270 2024-04-25T11:14:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07b6a00640 1 -- 172.21.15.7:0/2177871087 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f07d0015c50 con 0x55f034fdb270 2024-04-25T11:14:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07e37f2740 10 client.6285 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T11:14:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07e37f2740 10 client.6285 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:14:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07e37f2740 20 client.6285 populate_metadata read hostname 'smithi007' 2024-04-25T11:14:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07e37f2740 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.7:6835/2403088100,v1:172.21.15.7:6837/2403088100] conn(0x55f034f99e40 0x55f0353bdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07e37f2740 4 client.6285 mds_command: new command op to 6120 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:14:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.035+0000 7f07e37f2740 1 -- 172.21.15.7:0/2177871087 --> [v2:172.21.15.7:6835/2403088100,v1:172.21.15.7:6837/2403088100] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0352ef210 con 0x55f034f99e40 2024-04-25T11:14:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.036+0000 7f07d9e00640 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.7:6835/2403088100,v1:172.21.15.7:6837/2403088100] conn(0x55f034f99e40 0x55f0353bdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.036+0000 7f07d9e00640 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.7:6835/2403088100,v1:172.21.15.7:6837/2403088100] conn(0x55f034f99e40 0x55f0353bdb50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f962cb2da4ae392 server_cookie=db3a3a9aaad5b32b in_seq=0 out_seq=0 2024-04-25T11:14:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.036+0000 7f07b6a00640 10 client.6285 ms_handle_connect on v2:172.21.15.7:6835/2403088100 2024-04-25T11:14:34.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.037+0000 7f07b6a00640 1 -- 172.21.15.7:0/2177871087 <== mds.0 v2:172.21.15.7:6835/2403088100 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f0352ef210 con 0x55f034f99e40 2024-04-25T11:14:34.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.037+0000 7f07b6a00640 10 client.6285 handle_command_reply: tid=0 2024-04-25T11:14:34.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.037+0000 7f07e37f2740 1 client.6285 shutdown 2024-04-25T11:14:34.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.037+0000 7f07e37f2740 1 -- 172.21.15.7:0/2177871087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 msgr2=0x55f0353ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.037+0000 7f07e37f2740 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f0353ab140 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f07d4073040 tx=0x7f07d000fce0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.038+0000 7f07e37f2740 1 -- 172.21.15.7:0/2177871087 shutdown_connections 2024-04-25T11:14:34.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.038+0000 7f07e37f2740 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.7:6835/2403088100,v1:172.21.15.7:6837/2403088100] conn(0x55f034f99e40 0x55f0353bdb50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.038+0000 7f07e37f2740 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f0353ab140 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.038+0000 7f07e37f2740 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fba590 0x55f0353bfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.038+0000 7f07e37f2740 1 --2- 172.21.15.7:0/2177871087 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fb23b0 0x55f0353c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.038+0000 7f07e37f2740 1 -- 172.21.15.7:0/2177871087 >> 172.21.15.7:0/2177871087 conn(0x55f034fcf3c0 msgr2=0x55f0353a4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:34.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.038+0000 7f07b6a00640 0 client.6285 ms_handle_reset on v2:172.21.15.7:6835/2403088100 2024-04-25T11:14:34.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.038+0000 7f07e37f2740 1 -- 172.21.15.7:0/2177871087 shutdown_connections 2024-04-25T11:14:34.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.038+0000 7f07e37f2740 1 -- 172.21.15.7:0/2177871087 wait complete. 2024-04-25T11:14:34.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.038+0000 7f07e37f2740 20 client.6285 trim_cache size 0 max 16384 2024-04-25T11:14:34.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.058+0000 7f07e37f2740 1 Processor -- start 2024-04-25T11:14:34.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.058+0000 7f07e37f2740 1 -- start start 2024-04-25T11:14:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.060+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fb23b0 0x55f03543a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.060+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fba590 0x55f0353bf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.060+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fdb270 0x55f0353bf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.060+0000 7f07e37f2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07d4202650 con 0x55f034fba590 2024-04-25T11:14:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.060+0000 7f07e37f2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07d41e8c40 con 0x55f034fb23b0 2024-04-25T11:14:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.060+0000 7f07e37f2740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f07d4202000 con 0x55f034fdb270 2024-04-25T11:14:34.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.060+0000 7f07d9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fba590 0x55f0353bf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.060+0000 7f07d9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fba590 0x55f0353bf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38012/0 (socket says 172.21.15.7:38012) 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.060+0000 7f07d9400640 1 -- 172.21.15.7:0/365658554 learned_addr learned my addr 172.21.15.7:0/365658554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.061+0000 7f07d9e00640 1 --2- 172.21.15.7:0/365658554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fb23b0 0x55f03543a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.061+0000 7f07da800640 1 --2- 172.21.15.7:0/365658554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fdb270 0x55f0353bf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.061+0000 7f07d9400640 1 -- 172.21.15.7:0/365658554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fdb270 msgr2=0x55f0353bf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.061+0000 7f07d9400640 1 --2- 172.21.15.7:0/365658554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fdb270 0x55f0353bf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.061+0000 7f07d9400640 1 -- 172.21.15.7:0/365658554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fb23b0 msgr2=0x55f03543a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.061+0000 7f07d9400640 1 --2- 172.21.15.7:0/365658554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fb23b0 0x55f03543a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.061+0000 7f07d9400640 1 -- 172.21.15.7:0/365658554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0353c0540 con 0x55f034fba590 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.061+0000 7f07d9e00640 1 --2- 172.21.15.7:0/365658554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fb23b0 0x55f03543a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.061+0000 7f07d9400640 1 --2- 172.21.15.7:0/365658554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fba590 0x55f0353bf100 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f07c4007400 tx=0x7f07c403a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40650cc0d2bd79cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.062+0000 7f07c1c00640 1 -- 172.21.15.7:0/365658554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4006910 con 0x55f034fba590 2024-04-25T11:14:34.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.062+0000 7f07c1c00640 1 -- 172.21.15.7:0/365658554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c4006ab0 con 0x55f034fba590 2024-04-25T11:14:34.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.062+0000 7f07c1c00640 1 -- 172.21.15.7:0/365658554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c4032440 con 0x55f034fba590 2024-04-25T11:14:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.062+0000 7f07e37f2740 1 -- 172.21.15.7:0/365658554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fba590 msgr2=0x55f0353bf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.062+0000 7f07e37f2740 1 --2- 172.21.15.7:0/365658554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fba590 0x55f0353bf100 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f07c4007400 tx=0x7f07c403a000 comp rx=0 tx=0).stop 2024-04-25T11:14:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.062+0000 7f07e37f2740 1 -- 172.21.15.7:0/365658554 shutdown_connections 2024-04-25T11:14:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.062+0000 7f07e37f2740 1 --2- 172.21.15.7:0/365658554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fdb270 0x55f0353bf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.062+0000 7f07e37f2740 1 --2- 172.21.15.7:0/365658554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fba590 0x55f0353bf100 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.062+0000 7f07e37f2740 1 --2- 172.21.15.7:0/365658554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fb23b0 0x55f03543a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.062+0000 7f07e37f2740 1 -- 172.21.15.7:0/365658554 >> 172.21.15.7:0/365658554 conn(0x55f034fcf3c0 msgr2=0x55f0353e5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.063+0000 7f07e37f2740 1 -- 172.21.15.7:0/365658554 shutdown_connections 2024-04-25T11:14:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.063+0000 7f07e37f2740 1 -- 172.21.15.7:0/365658554 wait complete. 2024-04-25T11:14:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.064+0000 7f07e37f2740 1 -- start start 2024-04-25T11:14:34.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.064+0000 7f07e37f2740 1 Processor -- start 2024-04-25T11:14:34.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.065+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fb23b0 0x55f0353d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.065+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fba590 0x55f0353d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.065+0000 7f07e37f2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f03537d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.065+0000 7f07e37f2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f035398330 con 0x55f034fdb270 2024-04-25T11:14:34.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.065+0000 7f07e37f2740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f035397b70 con 0x55f034fba590 2024-04-25T11:14:34.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.065+0000 7f07e37f2740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55f035398920 con 0x55f034fb23b0 2024-04-25T11:14:34.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.065+0000 7f07e37f2740 10 client.0 _ll_register_callbacks cb 0x7f07d0001870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:14:34.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.066+0000 7f07da800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f03537d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.066+0000 7f07da800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f03537d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38024/0 (socket says 172.21.15.7:38024) 2024-04-25T11:14:34.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.066+0000 7f07da800640 1 -- 172.21.15.7:0/1472466771 learned_addr learned my addr 172.21.15.7:0/1472466771 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:34.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.066+0000 7f07d9400640 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fba590 0x55f0353d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.066+0000 7f07da800640 1 -- 172.21.15.7:0/1472466771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fb23b0 msgr2=0x55f0353d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.066+0000 7f07da800640 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fb23b0 0x55f0353d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.066+0000 7f07da800640 1 -- 172.21.15.7:0/1472466771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fba590 msgr2=0x55f0353d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.066+0000 7f07da800640 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fba590 0x55f0353d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.066+0000 7f07da800640 1 -- 172.21.15.7:0/1472466771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f035397050 con 0x55f034fdb270 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.066+0000 7f07da800640 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f03537d840 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55f035379e60 tx=0x7f07d0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d056602dec32bd7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.066+0000 7f07b6a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.067+0000 7f07b6a00640 1 -- 172.21.15.7:0/1472466771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0004d70 con 0x55f034fdb270 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.067+0000 7f07b6a00640 1 -- 172.21.15.7:0/1472466771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f07d0019040 con 0x55f034fdb270 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.067+0000 7f07e37f2740 1 -- 172.21.15.7:0/1472466771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07c4006ab0 con 0x55f034fdb270 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.067+0000 7f07b6a00640 1 -- 172.21.15.7:0/1472466771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f07d0033050 con 0x55f034fdb270 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.067+0000 7f07e37f2740 10 client.6291 fetch_fsmap learned FSMap version 35 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.067+0000 7f07e37f2740 1 -- 172.21.15.7:0/1472466771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55f0353a49f0 con 0x55f034fdb270 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.067+0000 7f07b6a00640 1 -- 172.21.15.7:0/1472466771 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c4006ab0 con 0x55f034fdb270 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.067+0000 7f07b6a00640 1 -- 172.21.15.7:0/1472466771 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f07d0036020 con 0x55f034fdb270 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.068+0000 7f07e37f2740 10 client.6291 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T11:14:34.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.068+0000 7f07e37f2740 10 client.6291 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.068+0000 7f07e37f2740 20 client.6291 populate_metadata read hostname 'smithi007' 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.068+0000 7f07e37f2740 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.7:6835/2403088100,v1:172.21.15.7:6837/2403088100] conn(0x55f035377270 0x55f035377680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.068+0000 7f07e37f2740 4 client.6291 mds_command: new command op to 6120 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.068+0000 7f07e37f2740 1 -- 172.21.15.7:0/1472466771 --> [v2:172.21.15.7:6835/2403088100,v1:172.21.15.7:6837/2403088100] -- command(tid 0: {"prefix": "status"}) -- 0x55f0352e9080 con 0x55f035377270 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.068+0000 7f07d9e00640 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.7:6835/2403088100,v1:172.21.15.7:6837/2403088100] conn(0x55f035377270 0x55f035377680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.068+0000 7f07d9e00640 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.7:6835/2403088100,v1:172.21.15.7:6837/2403088100] conn(0x55f035377270 0x55f035377680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f4ab0d1e4144d1b server_cookie=c1c572dc8cdd607d in_seq=0 out_seq=0 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.068+0000 7f07b6a00640 10 client.6291 ms_handle_connect on v2:172.21.15.7:6835/2403088100 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07b6a00640 1 -- 172.21.15.7:0/1472466771 <== mds.0 v2:172.21.15.7:6835/2403088100 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55f0352e9080 con 0x55f035377270 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07b6a00640 10 client.6291 handle_command_reply: tid=0 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 1 client.6291 shutdown 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 1 -- 172.21.15.7:0/1472466771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 msgr2=0x55f03537d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f03537d840 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x55f035379e60 tx=0x7f07d0001250 comp rx=0 tx=0).stop 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 1 -- 172.21.15.7:0/1472466771 shutdown_connections 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.7:6835/2403088100,v1:172.21.15.7:6837/2403088100] conn(0x55f035377270 0x55f035377680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f034fdb270 0x55f03537d840 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f034fba590 0x55f0353d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 1 --2- 172.21.15.7:0/1472466771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55f034fb23b0 0x55f0353d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07b6a00640 0 client.6291 ms_handle_reset on v2:172.21.15.7:6835/2403088100 2024-04-25T11:14:34.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 1 -- 172.21.15.7:0/1472466771 >> 172.21.15.7:0/1472466771 conn(0x55f034fcf3c0 msgr2=0x55f0353a3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:34.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 1 -- 172.21.15.7:0/1472466771 shutdown_connections 2024-04-25T11:14:34.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 1 -- 172.21.15.7:0/1472466771 wait complete. 2024-04-25T11:14:34.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.069+0000 7f07e37f2740 20 client.6291 trim_cache size 0 max 16384 2024-04-25T11:14:34.072 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:14:34.072 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:14:34.072 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:14:34.072 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 6120, 2024-04-25T11:14:34.073 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:14:34.073 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:14:34.073 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:14:34.073 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 12.741315258, 2024-04-25T11:14:34.073 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 35, 2024-04-25T11:14:34.073 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 68, 2024-04-25T11:14:34.073 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 68, 2024-04-25T11:14:34.073 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 18.869273609 2024-04-25T11:14:34.073 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:14:34.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 -- 172.21.15.7:0/1312249627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07b8074650 msgr2=0x7f07b8076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07b8074650 0x7f07b8076b60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f07c4034000 tx=0x7f07c402e000 comp rx=0 tx=0).stop 2024-04-25T11:14:34.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 -- 172.21.15.7:0/1312249627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 msgr2=0x7f07d41d5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 0x7f07d41d5420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f07c8011620 tx=0x7f07c8011ae0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 -- 172.21.15.7:0/1312249627 shutdown_connections 2024-04-25T11:14:34.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07b8074650 0x7f07b8076b60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07d41d7c10 0x7f07d41d8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07d41e90f0 0x7f07d41e9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 --2- 172.21.15.7:0/1312249627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07bc004800 0x7f07d41d5420 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 -- 172.21.15.7:0/1312249627 >> 172.21.15.7:0/1312249627 conn(0x7f07d40d1540 msgr2=0x7f07d40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:34.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 -- 172.21.15.7:0/1312249627 shutdown_connections 2024-04-25T11:14:34.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.070+0000 7f07db200640 1 -- 172.21.15.7:0/1312249627 wait complete. 2024-04-25T11:14:34.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:14:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.362+0000 7f89ff000640 1 Processor -- start 2024-04-25T11:14:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.362+0000 7f89ff000640 1 -- start start 2024-04-25T11:14:34.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.364+0000 7f89ff000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f89f806d580 0x7f89f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.364+0000 7f89ff000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f80b55a0 con 0x7f89f806f390 2024-04-25T11:14:34.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.364+0000 7f89ff000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f806ded0 con 0x7f89f80b5190 2024-04-25T11:14:34.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.364+0000 7f89ff000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f89f815d560 con 0x7f89f806d580 2024-04-25T11:14:34.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.364+0000 7f89fd000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f89f806f390 0x7f89f806f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43978/0 (socket says 172.21.15.7:43978) 2024-04-25T11:14:34.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.364+0000 7f89fd000640 1 -- 172.21.15.7:0/65232304 learned_addr learned my addr 172.21.15.7:0/65232304 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.364+0000 7f89efe00640 1 --2- 172.21.15.7:0/65232304 >> v2:172.21.15.87:3300/0 conn(0x7f89f806d580 0x7f89f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.364+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1101053956 0 0) 0x7f89f80b55a0 con 0x7f89f806f390 2024-04-25T11:14:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89dc003520 con 0x7f89f806f390 2024-04-25T11:14:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1739531490 0 0) 0x7f89dc003520 con 0x7f89f806f390 2024-04-25T11:14:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 >> v2:172.21.15.87:3300/0 conn(0x7f89f806d580 msgr2=0x7f89f806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 --2- 172.21.15.7:0/65232304 >> v2:172.21.15.87:3300/0 conn(0x7f89f806d580 0x7f89f806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 >> v1:172.21.15.87:6789/0 conn(0x7f89f80b5190 legacy=0x7f89f80d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f815d820 con 0x7f89f806f390 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f89f00019f0 con 0x7f89f806f390 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 >> v1:172.21.15.7:6789/0 conn(0x7f89f806f390 legacy=0x7f89f806f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 --2- 172.21.15.7:0/65232304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89dc0039b0 0x7f89dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 --2- 172.21.15.7:0/65232304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc004800 0x7f89dc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 --2- 172.21.15.7:0/65232304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 0x7f89dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89f815d560 con 0x7f89dc005250 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89dc00c1e0 con 0x7f89dc004800 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89dc00e5f0 con 0x7f89dc0039b0 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.365+0000 7f89fd000640 1 --2- 172.21.15.7:0/65232304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 0x7f89dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.366+0000 7f89efe00640 1 --2- 172.21.15.7:0/65232304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc004800 0x7f89dc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.366+0000 7f89f7e00640 1 --2- 172.21.15.7:0/65232304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89dc0039b0 0x7f89dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.366+0000 7f89fd000640 1 -- 172.21.15.7:0/65232304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89dc0039b0 msgr2=0x7f89dc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.366+0000 7f89fd000640 1 --2- 172.21.15.7:0/65232304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89dc0039b0 0x7f89dc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.366+0000 7f89fd000640 1 -- 172.21.15.7:0/65232304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc004800 msgr2=0x7f89dc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:34.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.366+0000 7f89fd000640 1 --2- 172.21.15.7:0/65232304 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89dc004800 0x7f89dc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.366+0000 7f89fd000640 1 -- 172.21.15.7:0/65232304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89dc003520 con 0x7f89dc005250 2024-04-25T11:14:34.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.367+0000 7f89fd000640 1 -- 172.21.15.7:0/65232304 reap_dead start 2024-04-25T11:14:34.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.367+0000 7f89fd000640 1 --2- 172.21.15.7:0/65232304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 0x7f89dc00b9c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f89f8071770 tx=0x7f89dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571bd0ea69f3c33c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:34.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.367+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f000a460 con 0x7f89dc005250 2024-04-25T11:14:34.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.367+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f002f070 con 0x7f89dc005250 2024-04-25T11:14:34.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.367+0000 7f89f7400640 1 -- 172.21.15.7:0/65232304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f002bc50 con 0x7f89dc005250 2024-04-25T11:14:34.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.368+0000 7f89ff000640 1 -- 172.21.15.7:0/65232304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 msgr2=0x7f89dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.368+0000 7f89ff000640 1 --2- 172.21.15.7:0/65232304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 0x7f89dc00b9c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f89f8071770 tx=0x7f89dc005830 comp rx=0 tx=0).stop 2024-04-25T11:14:34.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.368+0000 7f89ff000640 1 -- 172.21.15.7:0/65232304 shutdown_connections 2024-04-25T11:14:34.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.368+0000 7f89ff000640 1 --2- 172.21.15.7:0/65232304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 0x7f89dc00b9c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.368+0000 7f89ff000640 1 -- 172.21.15.7:0/65232304 >> 172.21.15.7:0/65232304 conn(0x7f89f80732b0 msgr2=0x7f89f80736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:34.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.368+0000 7f89ff000640 1 -- 172.21.15.7:0/65232304 shutdown_connections 2024-04-25T11:14:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.368+0000 7f89ff000640 1 -- 172.21.15.7:0/65232304 wait complete. 2024-04-25T11:14:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.370+0000 7f89ff000640 1 Processor -- start 2024-04-25T11:14:34.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.371+0000 7f89ff000640 1 -- start start 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.372+0000 7f89ff000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 0x7f89f8153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.372+0000 7f89ff000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89f81541f0 0x7f89f8155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.372+0000 7f89ff000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f8155fe0 0x7f89f80af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.372+0000 7f89ff000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89f815e0c0 con 0x7f89dc005250 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.372+0000 7f89ff000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89f815dde0 con 0x7f89f8155fe0 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.372+0000 7f89ff000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89f809f8e0 con 0x7f89f81541f0 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.372+0000 7f89f7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 0x7f89f8153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.372+0000 7f89f7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 0x7f89f8153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38050/0 (socket says 172.21.15.7:38050) 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.372+0000 7f89f7e00640 1 -- 172.21.15.7:0/3429973038 learned_addr learned my addr 172.21.15.7:0/3429973038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89efe00640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89f81541f0 0x7f89f8155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89f7e00640 1 -- 172.21.15.7:0/3429973038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89f81541f0 msgr2=0x7f89f8155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89f7e00640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89f81541f0 0x7f89f8155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89f7e00640 1 -- 172.21.15.7:0/3429973038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f8155fe0 msgr2=0x7f89f80af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89f7e00640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f8155fe0 0x7f89f80af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89f7e00640 1 -- 172.21.15.7:0/3429973038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f8150dc0 con 0x7f89dc005250 2024-04-25T11:14:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89f7e00640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 0x7f89f8153cb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f89e800e4b0 tx=0x7f89e800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89f4c00640 1 -- 172.21.15.7:0/3429973038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e80052e0 con 0x7f89dc005250 2024-04-25T11:14:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89f4c00640 1 -- 172.21.15.7:0/3429973038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e8018070 con 0x7f89dc005250 2024-04-25T11:14:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89f4c00640 1 -- 172.21.15.7:0/3429973038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8003810 con 0x7f89dc005250 2024-04-25T11:14:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89ff000640 1 -- 172.21.15.7:0/3429973038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f8150000 con 0x7f89dc005250 2024-04-25T11:14:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.373+0000 7f89ff000640 1 -- 172.21.15.7:0/3429973038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f81505c0 con 0x7f89dc005250 2024-04-25T11:14:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.375+0000 7f89ff000640 1 -- 172.21.15.7:0/3429973038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f8152510 con 0x7f89dc005250 2024-04-25T11:14:34.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.376+0000 7f89f4c00640 1 -- 172.21.15.7:0/3429973038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f89e80039b0 con 0x7f89dc005250 2024-04-25T11:14:34.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.377+0000 7f89f4c00640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89d40744b0 0x7f89d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:34.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.377+0000 7f89f4c00640 1 -- 172.21.15.7:0/3429973038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f89e8094fe0 con 0x7f89dc005250 2024-04-25T11:14:34.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.377+0000 7f89efe00640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89d40744b0 0x7f89d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:34.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.378+0000 7f89efe00640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89d40744b0 0x7f89d40769c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f89f81f9b70 tx=0x7f89e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:34.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.382+0000 7f89f4c00640 1 -- 172.21.15.7:0/3429973038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89f806ded0 con 0x7f89dc005250 2024-04-25T11:14:34.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.624+0000 7f89ff000640 1 -- 172.21.15.7:0/3429973038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f89f8074430 con 0x7f89dc005250 2024-04-25T11:14:34.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.625+0000 7f89f4c00640 1 -- 172.21.15.7:0/3429973038 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f89e805f6f0 con 0x7f89dc005250 2024-04-25T11:14:34.626 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:14:34.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 -- 172.21.15.7:0/3429973038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89d40744b0 msgr2=0x7f89d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89d40744b0 0x7f89d40769c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f89f81f9b70 tx=0x7f89e0008040 comp rx=0 tx=0).stop 2024-04-25T11:14:34.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 -- 172.21.15.7:0/3429973038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 msgr2=0x7f89f8153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:34.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 0x7f89f8153cb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f89e800e4b0 tx=0x7f89e800e970 comp rx=0 tx=0).stop 2024-04-25T11:14:34.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 -- 172.21.15.7:0/3429973038 shutdown_connections 2024-04-25T11:14:34.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89d40744b0 0x7f89d40769c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89f8155fe0 0x7f89f80af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89f81541f0 0x7f89f8155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 --2- 172.21.15.7:0/3429973038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89dc005250 0x7f89f8153cb0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:34.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 -- 172.21.15.7:0/3429973038 >> 172.21.15.7:0/3429973038 conn(0x7f89f80732b0 msgr2=0x7f89f806f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:34.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 -- 172.21.15.7:0/3429973038 shutdown_connections 2024-04-25T11:14:34.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34.628+0000 7f89ff000640 1 -- 172.21.15.7:0/3429973038 wait complete. 2024-04-25T11:14:34.654 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:14:34.654 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:14:34.654 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:14:34.654 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:14:34.655 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:14:34.655 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:14:34.655 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:14:34.655 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:14:34.655 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:14:34.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:34.655 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:14:34.706 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:14:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:14:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:14:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:14:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:34.706 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:14:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:14:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:14:34.706 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:14:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42300sec preferred_lft 42300sec 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:14:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6098:c7ff:fe8a:4a30/64 scope link 2024-04-25T11:14:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:34.708 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:14:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:14:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::646a:35ff:fe7b:ad0/64 scope link 2024-04-25T11:14:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:34.708 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:14:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:14:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4465:c4ff:fe62:f75f/64 scope link 2024-04-25T11:14:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:34.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:34.709 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:14:34.784 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:14:34.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:34.788 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:14:34.847 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:14:34.847 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:14:34.848 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:14:34.848 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:14:34.848 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:14:34.848 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:14:34.848 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:14:34.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:14:34.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:14:34.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:14:34.848 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:14:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:14:35.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:14:35.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:35.073 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:14:35.124 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:14:35.124 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:14:35.125 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:14:35.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:35.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:14:35.210 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:14:35.211 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:14:35.266 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:14:35.266 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:14:35.266 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:14:35.319 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:14:35.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:35.319 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:14:35.532 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:14:35.532 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:14:35.574 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:14:35.574 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:14:35.574 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:14:35.574 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:14:35.574 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:14:35.576 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:14:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:14:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:14:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:14:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:14:35.581 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:14:35.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:35.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:14:35.669 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:14:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:14:35.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:14:35.820 INFO:teuthology.orchestra.run.smithi007.stdout:client6300 2024-04-25T11:14:35.820 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:14:35.820 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:35.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:14:36.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.162+0000 7f041dc00640 1 Processor -- start 2024-04-25T11:14:36.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.162+0000 7f041dc00640 1 -- start start 2024-04-25T11:14:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.165+0000 7f041dc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f041815baa0 0x7f041815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.165+0000 7f041dc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f041806d580 0x7f041806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.165+0000 7f041dc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f041806ded0 con 0x7f0418070640 2024-04-25T11:14:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.165+0000 7f041dc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f041815e980 con 0x7f041815baa0 2024-04-25T11:14:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.165+0000 7f041dc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f041815ec60 con 0x7f041806d580 2024-04-25T11:14:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.165+0000 7f040f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0418070640 0x7f0418161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43982/0 (socket says 172.21.15.7:43982) 2024-04-25T11:14:36.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.165+0000 7f040f400640 1 -- 172.21.15.7:0/978855222 learned_addr learned my addr 172.21.15.7:0/978855222 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417e00640 1 --2- 172.21.15.7:0/978855222 >> v2:172.21.15.87:3300/0 conn(0x7f041815baa0 0x7f041815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f041d200640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f041806d580 0x7f041806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2392297626 0 0) 0x7f0400001110 con 0x7f0418070640 2024-04-25T11:14:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03fc003520 con 0x7f0418070640 2024-04-25T11:14:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3097586987 0 0) 0x7f03fc003520 con 0x7f0418070640 2024-04-25T11:14:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f041806d580 msgr2=0x7f041806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:36.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f041806d580 0x7f041806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 >> v2:172.21.15.87:3300/0 conn(0x7f041815baa0 msgr2=0x7f041815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 --2- 172.21.15.7:0/978855222 >> v2:172.21.15.87:3300/0 conn(0x7f041815baa0 0x7f041815e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041815ef40 con 0x7f0418070640 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0400001860 con 0x7f0418070640 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 >> v1:172.21.15.7:6789/0 conn(0x7f0418070640 legacy=0x7f0418161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f041806d580 0x7f03fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03fc003f30 0x7f03fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc004d40 0x7f03fc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f041815e980 con 0x7f03fc004d40 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f041815ec60 con 0x7f03fc003f30 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.166+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc00ba10 con 0x7f041806d580 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.167+0000 7f041d200640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc004d40 0x7f03fc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.167+0000 7f0417e00640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f041806d580 0x7f03fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.167+0000 7f040f400640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03fc003f30 0x7f03fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:36.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.167+0000 7f041d200640 1 -- 172.21.15.7:0/978855222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f041806d580 msgr2=0x7f03fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:36.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.167+0000 7f041d200640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f041806d580 0x7f03fc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:36.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.167+0000 7f041d200640 1 -- 172.21.15.7:0/978855222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03fc003f30 msgr2=0x7f03fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:36.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.167+0000 7f041d200640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03fc003f30 0x7f03fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:36.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.167+0000 7f041d200640 1 -- 172.21.15.7:0/978855222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc003520 con 0x7f03fc004d40 2024-04-25T11:14:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.168+0000 7f041d200640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc004d40 0x7f03fc005150 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0418073040 tx=0x7f03fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1042b012d360f0d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.169+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410036070 con 0x7f03fc004d40 2024-04-25T11:14:36.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.169+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0410031400 con 0x7f03fc004d40 2024-04-25T11:14:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.169+0000 7f0417400640 1 -- 172.21.15.7:0/978855222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410009690 con 0x7f03fc004d40 2024-04-25T11:14:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.169+0000 7f041dc00640 1 -- 172.21.15.7:0/978855222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc004d40 msgr2=0x7f03fc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.169+0000 7f041dc00640 1 --2- 172.21.15.7:0/978855222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03fc004d40 0x7f03fc005150 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0418073040 tx=0x7f03fc005830 comp rx=0 tx=0).stop 2024-04-25T11:14:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.169+0000 7f041d200640 1 -- 172.21.15.7:0/978855222 reap_dead start 2024-04-25T11:14:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.170+0000 7f041dc00640 1 -- 172.21.15.7:0/978855222 shutdown_connections 2024-04-25T11:14:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.170+0000 7f041dc00640 1 -- 172.21.15.7:0/978855222 >> 172.21.15.7:0/978855222 conn(0x7f0418073090 msgr2=0x7f04180734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.170+0000 7f041dc00640 1 -- 172.21.15.7:0/978855222 shutdown_connections 2024-04-25T11:14:36.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.170+0000 7f041dc00640 1 -- 172.21.15.7:0/978855222 wait complete. 2024-04-25T11:14:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.172+0000 7f041dc00640 1 Processor -- start 2024-04-25T11:14:36.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.172+0000 7f041dc00640 1 -- start start 2024-04-25T11:14:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.174+0000 7f041dc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03fc004d40 0x7f041815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.174+0000 7f041dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418140c40 0x7f04181429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.174+0000 7f041dc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04180b0800 0x7f0418142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.174+0000 7f041dc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0418070bd0 con 0x7f0418140c40 2024-04-25T11:14:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.174+0000 7f041dc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0418070a50 con 0x7f04180b0800 2024-04-25T11:14:36.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.174+0000 7f041dc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f04181625f0 con 0x7f03fc004d40 2024-04-25T11:14:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.174+0000 7f040f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418140c40 0x7f04181429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.174+0000 7f040f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418140c40 0x7f04181429d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38074/0 (socket says 172.21.15.7:38074) 2024-04-25T11:14:36.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.174+0000 7f040f400640 1 -- 172.21.15.7:0/2436931833 learned_addr learned my addr 172.21.15.7:0/2436931833 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.175+0000 7f0417e00640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03fc004d40 0x7f041815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.175+0000 7f041d200640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04180b0800 0x7f0418142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.175+0000 7f040f400640 1 -- 172.21.15.7:0/2436931833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03fc004d40 msgr2=0x7f041815aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.175+0000 7f040f400640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03fc004d40 0x7f041815aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:36.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.175+0000 7f040f400640 1 -- 172.21.15.7:0/2436931833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04180b0800 msgr2=0x7f0418142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.175+0000 7f040f400640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04180b0800 0x7f0418142f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.175+0000 7f040f400640 1 -- 172.21.15.7:0/2436931833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041815d360 con 0x7f0418140c40 2024-04-25T11:14:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.175+0000 7f040f400640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418140c40 0x7f04181429d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f040000e870 tx=0x7f040000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.175+0000 7f0414c00640 1 -- 172.21.15.7:0/2436931833 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040000c8d0 con 0x7f0418140c40 2024-04-25T11:14:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.176+0000 7f0414c00640 1 -- 172.21.15.7:0/2436931833 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0400018070 con 0x7f0418140c40 2024-04-25T11:14:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.176+0000 7f0414c00640 1 -- 172.21.15.7:0/2436931833 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040000ccd0 con 0x7f0418140c40 2024-04-25T11:14:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.176+0000 7f041dc00640 1 -- 172.21.15.7:0/2436931833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041815d640 con 0x7f0418140c40 2024-04-25T11:14:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.176+0000 7f041dc00640 1 -- 172.21.15.7:0/2436931833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041813e950 con 0x7f0418140c40 2024-04-25T11:14:36.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.177+0000 7f041dc00640 1 -- 172.21.15.7:0/2436931833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0418141100 con 0x7f0418140c40 2024-04-25T11:14:36.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.178+0000 7f0414c00640 1 -- 172.21.15.7:0/2436931833 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0400013420 con 0x7f0418140c40 2024-04-25T11:14:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.180+0000 7f0414c00640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03f40744e0 0x7f03f40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.180+0000 7f0414c00640 1 -- 172.21.15.7:0/2436931833 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f0400095090 con 0x7f0418140c40 2024-04-25T11:14:36.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.180+0000 7f0417e00640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03f40744e0 0x7f03f40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:36.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.181+0000 7f0417e00640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03f40744e0 0x7f03f40769f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0408007520 tx=0x7f040800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:36.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.185+0000 7f0414c00640 1 -- 172.21.15.7:0/2436931833 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f041806ded0 con 0x7f0418140c40 2024-04-25T11:14:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.434+0000 7f041dc00640 1 -- 172.21.15.7:0/2436931833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0418074020 con 0x7f0418140c40 2024-04-25T11:14:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.434+0000 7f0414c00640 1 -- 172.21.15.7:0/2436931833 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f040005f740 con 0x7f0418140c40 2024-04-25T11:14:36.435 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.437+0000 7f041dc00640 1 -- 172.21.15.7:0/2436931833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03f40744e0 msgr2=0x7f03f40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.437+0000 7f041dc00640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03f40744e0 0x7f03f40769f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0408007520 tx=0x7f040800b040 comp rx=0 tx=0).stop 2024-04-25T11:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.437+0000 7f041dc00640 1 -- 172.21.15.7:0/2436931833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418140c40 msgr2=0x7f04181429d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.437+0000 7f041dc00640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418140c40 0x7f04181429d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f040000e870 tx=0x7f040000ed30 comp rx=0 tx=0).stop 2024-04-25T11:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.437+0000 7f041dc00640 1 -- 172.21.15.7:0/2436931833 shutdown_connections 2024-04-25T11:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.437+0000 7f041dc00640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03f40744e0 0x7f03f40769f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.438+0000 7f041dc00640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04180b0800 0x7f0418142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.438+0000 7f041dc00640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0418140c40 0x7f04181429d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.438+0000 7f041dc00640 1 --2- 172.21.15.7:0/2436931833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03fc004d40 0x7f041815aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:36.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.438+0000 7f041dc00640 1 -- 172.21.15.7:0/2436931833 >> 172.21.15.7:0/2436931833 conn(0x7f0418073090 msgr2=0x7f041815fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:36.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.438+0000 7f041dc00640 1 -- 172.21.15.7:0/2436931833 shutdown_connections 2024-04-25T11:14:36.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:36.438+0000 7f041dc00640 1 -- 172.21.15.7:0/2436931833 wait complete. 2024-04-25T11:14:36.465 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:14:36.465 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:14:36.465 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:14:36.465 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:14:36.466 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:14:36.466 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:14:36.466 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:14:36.466 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:14:36.466 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:14:36.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:36.466 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:14:36.525 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:14:36.526 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:14:36.526 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:14:36.526 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:36.526 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:14:36.526 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:36.526 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:14:36.526 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:14:36.526 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:14:36.526 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:14:36.526 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:14:36.527 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:14:36.527 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:14:36.527 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:14:36.527 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:14:36.527 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:14:36.527 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:14:36.527 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42299sec preferred_lft 42299sec 2024-04-25T11:14:36.527 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:14:36.527 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:36.527 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:14:36.527 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::58f2:6bff:fe8b:8b4c/64 scope link 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::98fe:1bff:fe9f:d71c/64 scope link 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::3c38:a8ff:fec7:ea00/64 scope link 2024-04-25T11:14:36.528 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:14:36.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:36.529 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:14:36.606 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:14:36.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:36.611 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:14:36.661 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:14:36.662 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:14:36.662 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:14:36.662 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:14:36.662 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:14:36.662 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:14:36.662 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:14:36.662 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:14:36.662 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:14:36.662 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:14:36.663 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:14:36.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:14:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:14:36.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:14:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:14:36.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:36.868 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:14:36.914 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:14:36.914 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:14:36.914 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:14:36.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:36.914 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T11:14:37.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:14:37.042 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:14:37.095 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:14:37.095 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:14:37.095 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:14:37.145 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:14:37.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:37.145 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:14:37.336 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:14:37.337 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:14:37.382 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:14:37.382 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:14:37.382 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:14:37.382 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 1@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:14:37.382 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:14:37.385 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:14:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:14:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:14:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:14:37.385 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:14:37.388 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:14:37.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:37.388 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:14:37.454 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:14:37.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:14:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:14:37.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:14:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:14:37.562 INFO:teuthology.orchestra.run.smithi087.stdout:client5267 2024-04-25T11:14:37.562 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:14:37.562 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:37.612 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:14:37.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:14:37.737 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:14:37.836 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:14:37.836 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:14:37.836 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:14:37.837 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:37.895 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:14:37.895 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:37.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:38.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.239+0000 7f823e400640 1 Processor -- start 2024-04-25T11:14:38.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.239+0000 7f823e400640 1 -- start start 2024-04-25T11:14:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.241+0000 7f823e400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f82380aed40 0x7f823815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.241+0000 7f823e400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82380af420 0x7f82380af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.242+0000 7f823e400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82380cebb0 con 0x7f82380b5220 2024-04-25T11:14:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.242+0000 7f823e400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f823815e1f0 con 0x7f82380af420 2024-04-25T11:14:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.242+0000 7f823e400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f82380afd70 con 0x7f82380aed40 2024-04-25T11:14:38.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.242+0000 7f8237e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f82380b5220 0x7f82380ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43988/0 (socket says 172.21.15.7:43988) 2024-04-25T11:14:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.242+0000 7f8237e00640 1 -- 172.21.15.7:0/1261834656 learned_addr learned my addr 172.21.15.7:0/1261834656 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.242+0000 7f823da00640 1 --2- 172.21.15.7:0/1261834656 >> v2:172.21.15.7:3300/0 conn(0x7f82380af420 0x7f82380af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.242+0000 7f823d000640 1 --2- 172.21.15.7:0/1261834656 >> v2:172.21.15.87:3300/0 conn(0x7f82380aed40 0x7f823815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.242+0000 7f8237400640 1 -- 172.21.15.7:0/1261834656 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 60012536 0 0) 0x7f8224001110 con 0x7f82380b5220 2024-04-25T11:14:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.242+0000 7f8237400640 1 -- 172.21.15.7:0/1261834656 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8220003520 con 0x7f82380b5220 2024-04-25T11:14:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.243+0000 7f823da00640 1 -- 172.21.15.7:0/1261834656 >> v2:172.21.15.87:3300/0 conn(0x7f82380aed40 msgr2=0x7f823815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.243+0000 7f823da00640 1 --2- 172.21.15.7:0/1261834656 >> v2:172.21.15.87:3300/0 conn(0x7f82380aed40 0x7f823815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.243+0000 7f823da00640 1 -- 172.21.15.7:0/1261834656 >> v1:172.21.15.7:6789/0 conn(0x7f82380b5220 legacy=0x7f82380ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.243+0000 7f823da00640 1 -- 172.21.15.7:0/1261834656 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82380b64b0 con 0x7f82380af420 2024-04-25T11:14:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.244+0000 7f823da00640 1 --2- 172.21.15.7:0/1261834656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82380af420 0x7f82380af830 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f82380725e0 tx=0x7f823002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7886ee55e2f8aabc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.244+0000 7f8237400640 1 -- 172.21.15.7:0/1261834656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230007740 con 0x7f82380af420 2024-04-25T11:14:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.244+0000 7f8237400640 1 -- 172.21.15.7:0/1261834656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82300078e0 con 0x7f82380af420 2024-04-25T11:14:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.244+0000 7f8237400640 1 -- 172.21.15.7:0/1261834656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230007cb0 con 0x7f82380af420 2024-04-25T11:14:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.245+0000 7f823e400640 1 -- 172.21.15.7:0/1261834656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82380af420 msgr2=0x7f82380af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.245+0000 7f823e400640 1 --2- 172.21.15.7:0/1261834656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82380af420 0x7f82380af830 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f82380725e0 tx=0x7f823002a680 comp rx=0 tx=0).stop 2024-04-25T11:14:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.245+0000 7f823e400640 1 -- 172.21.15.7:0/1261834656 shutdown_connections 2024-04-25T11:14:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.245+0000 7f823e400640 1 --2- 172.21.15.7:0/1261834656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82380af420 0x7f82380af830 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.245+0000 7f823e400640 1 --2- 172.21.15.7:0/1261834656 >> v2:172.21.15.87:3300/0 conn(0x7f82380aed40 0x7f823815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.245+0000 7f823e400640 1 -- 172.21.15.7:0/1261834656 >> 172.21.15.7:0/1261834656 conn(0x7f82380b3820 msgr2=0x7f82380b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.245+0000 7f823e400640 1 -- 172.21.15.7:0/1261834656 shutdown_connections 2024-04-25T11:14:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.245+0000 7f823e400640 1 -- 172.21.15.7:0/1261834656 wait complete. 2024-04-25T11:14:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.248+0000 7f823e400640 1 Processor -- start 2024-04-25T11:14:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.248+0000 7f823e400640 1 -- start start 2024-04-25T11:14:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.249+0000 7f823e400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82380aed40 0x7f8238151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.249+0000 7f823e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82380af420 0x7f8238155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.249+0000 7f823e400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82380b5220 0x7f82381561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.249+0000 7f823e400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f823806cbd0 con 0x7f82380af420 2024-04-25T11:14:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.249+0000 7f823e400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f823806c8f0 con 0x7f82380b5220 2024-04-25T11:14:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.249+0000 7f823e400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f823806ceb0 con 0x7f82380aed40 2024-04-25T11:14:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.250+0000 7f8237e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82380af420 0x7f8238155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.250+0000 7f8237e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82380af420 0x7f8238155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38080/0 (socket says 172.21.15.7:38080) 2024-04-25T11:14:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.250+0000 7f8237e00640 1 -- 172.21.15.7:0/2631235282 learned_addr learned my addr 172.21.15.7:0/2631235282 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.250+0000 7f823da00640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82380b5220 0x7f82381561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.250+0000 7f823d000640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82380aed40 0x7f8238151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.250+0000 7f8237e00640 1 -- 172.21.15.7:0/2631235282 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82380aed40 msgr2=0x7f8238151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.250+0000 7f8237e00640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82380aed40 0x7f8238151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.250+0000 7f8237e00640 1 -- 172.21.15.7:0/2631235282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82380b5220 msgr2=0x7f82381561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.250+0000 7f8237e00640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82380b5220 0x7f82381561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.250+0000 7f8237e00640 1 -- 172.21.15.7:0/2631235282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82380d0e80 con 0x7f82380af420 2024-04-25T11:14:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.251+0000 7f8237e00640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82380af420 0x7f8238155c90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f822400c3a0 tx=0x7f822400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.251+0000 7f8234c00640 1 -- 172.21.15.7:0/2631235282 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224017070 con 0x7f82380af420 2024-04-25T11:14:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.251+0000 7f8234c00640 1 -- 172.21.15.7:0/2631235282 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8224005ce0 con 0x7f82380af420 2024-04-25T11:14:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.251+0000 7f823e400640 1 -- 172.21.15.7:0/2631235282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82381551b0 con 0x7f82380af420 2024-04-25T11:14:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.251+0000 7f8234c00640 1 -- 172.21.15.7:0/2631235282 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224007310 con 0x7f82380af420 2024-04-25T11:14:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.251+0000 7f823e400640 1 -- 172.21.15.7:0/2631235282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8238155770 con 0x7f82380af420 2024-04-25T11:14:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.252+0000 7f823e400640 1 -- 172.21.15.7:0/2631235282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82380d0bf0 con 0x7f82380af420 2024-04-25T11:14:38.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.254+0000 7f8234c00640 1 -- 172.21.15.7:0/2631235282 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f822401a070 con 0x7f82380af420 2024-04-25T11:14:38.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.255+0000 7f8234c00640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82180744b0 0x7f82180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.255+0000 7f8234c00640 1 -- 172.21.15.7:0/2631235282 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f8224094730 con 0x7f82380af420 2024-04-25T11:14:38.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.255+0000 7f823d000640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82180744b0 0x7f82180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.256+0000 7f823d000640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82180744b0 0x7f82180769c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8238071200 tx=0x7f822c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:38.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.260+0000 7f8234c00640 1 -- 172.21.15.7:0/2631235282 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82380cebb0 con 0x7f82380af420 2024-04-25T11:14:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.465+0000 7f823e400640 1 -- 172.21.15.7:0/2631235282 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8238050740 con 0x7f82380af420 2024-04-25T11:14:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.467+0000 7f8234c00640 1 -- 172.21.15.7:0/2631235282 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16788 (secure 0 0 0) 0x7f822405e6d0 con 0x7f82380af420 2024-04-25T11:14:38.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:38.468 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:22.341351+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:14:19.115741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T11:14:19.849856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.470+0000 7f823e400640 1 -- 172.21.15.7:0/2631235282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82180744b0 msgr2=0x7f82180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.470+0000 7f823e400640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82180744b0 0x7f82180769c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8238071200 tx=0x7f822c008040 comp rx=0 tx=0).stop 2024-04-25T11:14:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.470+0000 7f823e400640 1 -- 172.21.15.7:0/2631235282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82380af420 msgr2=0x7f8238155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.470+0000 7f823e400640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82380af420 0x7f8238155c90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f822400c3a0 tx=0x7f822400c860 comp rx=0 tx=0).stop 2024-04-25T11:14:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.470+0000 7f823e400640 1 -- 172.21.15.7:0/2631235282 shutdown_connections 2024-04-25T11:14:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.470+0000 7f823e400640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82180744b0 0x7f82180769c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.470+0000 7f823e400640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82380b5220 0x7f82381561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.470+0000 7f823e400640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82380af420 0x7f8238155c90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.470+0000 7f823e400640 1 --2- 172.21.15.7:0/2631235282 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82380aed40 0x7f8238151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.470+0000 7f823e400640 1 -- 172.21.15.7:0/2631235282 >> 172.21.15.7:0/2631235282 conn(0x7f82380b3820 msgr2=0x7f82380684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:38.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.471+0000 7f823e400640 1 -- 172.21.15.7:0/2631235282 shutdown_connections 2024-04-25T11:14:38.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.471+0000 7f823e400640 1 -- 172.21.15.7:0/2631235282 wait complete. 2024-04-25T11:14:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:14:19.115741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:14:19.849856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:14:38.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:14:38.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.791+0000 7f4cb2a00640 1 Processor -- start 2024-04-25T11:14:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.791+0000 7f4cb2a00640 1 -- start start 2024-04-25T11:14:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.794+0000 7f4cb2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4cac145e40 0x7f4cac146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.794+0000 7f4cb2a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cac0b4870 0x7f4cac0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.794+0000 7f4cb2a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cac0b66b0 con 0x7f4cac0b51c0 2024-04-25T11:14:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.794+0000 7f4cb2a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4cac145b70 con 0x7f4cac0b4870 2024-04-25T11:14:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.794+0000 7f4cb2a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4cac0b5b30 con 0x7f4cac145e40 2024-04-25T11:14:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.794+0000 7f4cb0c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cac0b4870 0x7f4cac0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.794+0000 7f4cb0c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cac0b4870 0x7f4cac0b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38086/0 (socket says 172.21.15.7:38086) 2024-04-25T11:14:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.794+0000 7f4cb0c00640 1 -- 172.21.15.7:0/1773287769 learned_addr learned my addr 172.21.15.7:0/1773287769 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.794+0000 7f4cb1600640 1 --2- 172.21.15.7:0/1773287769 >> v2:172.21.15.87:3300/0 conn(0x7f4cac145e40 0x7f4cac146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.795+0000 7f4cb0c00640 1 -- 172.21.15.7:0/1773287769 >> v2:172.21.15.87:3300/0 conn(0x7f4cac145e40 msgr2=0x7f4cac146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.795+0000 7f4cb0c00640 1 --2- 172.21.15.7:0/1773287769 >> v2:172.21.15.87:3300/0 conn(0x7f4cac145e40 0x7f4cac146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.795+0000 7f4cb0c00640 1 -- 172.21.15.7:0/1773287769 >> v1:172.21.15.87:6789/0 conn(0x7f4cac0b51c0 legacy=0x7f4cac143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.795+0000 7f4cb0c00640 1 -- 172.21.15.7:0/1773287769 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac0b5e10 con 0x7f4cac0b4870 2024-04-25T11:14:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.796+0000 7f4cb0c00640 1 --2- 172.21.15.7:0/1773287769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0b4870 0x7f4cac0b4c80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4cac06ded0 tx=0x7f4c9802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d9756857f4bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.796+0000 7f4cabe00640 1 -- 172.21.15.7:0/1773287769 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98033640 con 0x7f4cac0b4870 2024-04-25T11:14:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.796+0000 7f4cabe00640 1 -- 172.21.15.7:0/1773287769 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c98038070 con 0x7f4cac0b4870 2024-04-25T11:14:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.796+0000 7f4cabe00640 1 -- 172.21.15.7:0/1773287769 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98033a00 con 0x7f4cac0b4870 2024-04-25T11:14:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.797+0000 7f4cb2a00640 1 -- 172.21.15.7:0/1773287769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0b4870 msgr2=0x7f4cac0b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.797+0000 7f4cb2a00640 1 --2- 172.21.15.7:0/1773287769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0b4870 0x7f4cac0b4c80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4cac06ded0 tx=0x7f4c9802c8b0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.797+0000 7f4cb2a00640 1 -- 172.21.15.7:0/1773287769 shutdown_connections 2024-04-25T11:14:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.797+0000 7f4cb2a00640 1 --2- 172.21.15.7:0/1773287769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0b4870 0x7f4cac0b4c80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.797+0000 7f4cb2a00640 1 --2- 172.21.15.7:0/1773287769 >> v2:172.21.15.87:3300/0 conn(0x7f4cac145e40 0x7f4cac146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.797+0000 7f4cb2a00640 1 -- 172.21.15.7:0/1773287769 >> 172.21.15.7:0/1773287769 conn(0x7f4cac0740f0 msgr2=0x7f4cac06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.797+0000 7f4cb2a00640 1 -- 172.21.15.7:0/1773287769 shutdown_connections 2024-04-25T11:14:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.797+0000 7f4cb2a00640 1 -- 172.21.15.7:0/1773287769 wait complete. 2024-04-25T11:14:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.800+0000 7f4cb2a00640 1 Processor -- start 2024-04-25T11:14:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.800+0000 7f4cb2a00640 1 -- start start 2024-04-25T11:14:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0b4870 0x7f4cac133880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cac0b51c0 0x7f4cac13ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb2a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cac145e40 0x7f4cac144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb2a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac0b55d0 con 0x7f4cac0b4870 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb2a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac0b63d0 con 0x7f4cac0b51c0 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb2a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4cac0b5750 con 0x7f4cac145e40 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0b4870 0x7f4cac133880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0b4870 0x7f4cac133880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38092/0 (socket says 172.21.15.7:38092) 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb1600640 1 -- 172.21.15.7:0/3186228865 learned_addr learned my addr 172.21.15.7:0/3186228865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb0c00640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cac0b51c0 0x7f4cac13ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb1600640 1 -- 172.21.15.7:0/3186228865 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cac145e40 msgr2=0x7f4cac144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb1600640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cac145e40 0x7f4cac144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb1600640 1 -- 172.21.15.7:0/3186228865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cac0b51c0 msgr2=0x7f4cac13ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb1600640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cac0b51c0 0x7f4cac13ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.802+0000 7f4cb1600640 1 -- 172.21.15.7:0/3186228865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac133dc0 con 0x7f4cac0b4870 2024-04-25T11:14:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.803+0000 7f4cb1600640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0b4870 0x7f4cac133880 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000e8a0 tx=0x7f4ca000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.803+0000 7f4ca9600640 1 -- 172.21.15.7:0/3186228865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0017070 con 0x7f4cac0b4870 2024-04-25T11:14:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.803+0000 7f4ca9600640 1 -- 172.21.15.7:0/3186228865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ca000c940 con 0x7f4cac0b4870 2024-04-25T11:14:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.803+0000 7f4ca9600640 1 -- 172.21.15.7:0/3186228865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca000cd30 con 0x7f4cac0b4870 2024-04-25T11:14:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.803+0000 7f4cb2a00640 1 -- 172.21.15.7:0/3186228865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac131180 con 0x7f4cac0b4870 2024-04-25T11:14:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.803+0000 7f4cb2a00640 1 -- 172.21.15.7:0/3186228865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac131740 con 0x7f4cac0b4870 2024-04-25T11:14:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.804+0000 7f4cb2a00640 1 -- 172.21.15.7:0/3186228865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cac050740 con 0x7f4cac0b4870 2024-04-25T11:14:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.806+0000 7f4ca9600640 1 -- 172.21.15.7:0/3186228865 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ca001a070 con 0x7f4cac0b4870 2024-04-25T11:14:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.807+0000 7f4ca9600640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4c8c0744e0 0x7f4c8c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.807+0000 7f4ca9600640 1 -- 172.21.15.7:0/3186228865 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f4ca00955e0 con 0x7f4cac0b4870 2024-04-25T11:14:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.807+0000 7f4cb0c00640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4c8c0744e0 0x7f4c8c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.808+0000 7f4cb0c00640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4c8c0744e0 0x7f4c8c0769f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4c98037000 tx=0x7f4c98032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:38.812+0000 7f4ca9600640 1 -- 172.21.15.7:0/3186228865 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4cac133dc0 con 0x7f4cac0b4870 2024-04-25T11:14:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.025+0000 7f4cb2a00640 1 -- 172.21.15.7:0/3186228865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4cac15cf10 con 0x7f4cac0b4870 2024-04-25T11:14:39.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.026+0000 7f4ca9600640 1 -- 172.21.15.7:0/3186228865 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7f4ca005fbd0 con 0x7f4cac0b4870 2024-04-25T11:14:39.026 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.028+0000 7f4cb2a00640 1 -- 172.21.15.7:0/3186228865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4c8c0744e0 msgr2=0x7f4c8c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.028+0000 7f4cb2a00640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4c8c0744e0 0x7f4c8c0769f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4c98037000 tx=0x7f4c98032000 comp rx=0 tx=0).stop 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.028+0000 7f4cb2a00640 1 -- 172.21.15.7:0/3186228865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0b4870 msgr2=0x7f4cac133880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.028+0000 7f4cb2a00640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0b4870 0x7f4cac133880 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000e8a0 tx=0x7f4ca000ed60 comp rx=0 tx=0).stop 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.028+0000 7f4cb2a00640 1 -- 172.21.15.7:0/3186228865 shutdown_connections 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.028+0000 7f4cb2a00640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4c8c0744e0 0x7f4c8c0769f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.028+0000 7f4cb2a00640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cac145e40 0x7f4cac144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.028+0000 7f4cb2a00640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cac0b51c0 0x7f4cac13ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.028+0000 7f4cb2a00640 1 --2- 172.21.15.7:0/3186228865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0b4870 0x7f4cac133880 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.028+0000 7f4cb2a00640 1 -- 172.21.15.7:0/3186228865 >> 172.21.15.7:0/3186228865 conn(0x7f4cac0740f0 msgr2=0x7f4cac06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.029+0000 7f4cb2a00640 1 -- 172.21.15.7:0/3186228865 shutdown_connections 2024-04-25T11:14:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.029+0000 7f4cb2a00640 1 -- 172.21.15.7:0/3186228865 wait complete. 2024-04-25T11:14:39.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:14:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.347+0000 7f9503200640 1 Processor -- start 2024-04-25T11:14:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.347+0000 7f9503200640 1 -- start start 2024-04-25T11:14:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.350+0000 7f9503200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f95040ce8d0 0x7f95040cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.350+0000 7f9503200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f950406fe40 0x7f950415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.350+0000 7f9503200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95040b4830 con 0x7f950415c2e0 2024-04-25T11:14:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.350+0000 7f9503200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f95040b6600 con 0x7f950406fe40 2024-04-25T11:14:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.350+0000 7f9503200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f95040b4b10 con 0x7f95040ce8d0 2024-04-25T11:14:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.350+0000 7f9501400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f950415c2e0 0x7f950406f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44000/0 (socket says 172.21.15.7:44000) 2024-04-25T11:14:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.350+0000 7f9501400640 1 -- 172.21.15.7:0/1277874187 learned_addr learned my addr 172.21.15.7:0/1277874187 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.350+0000 7f9502800640 1 --2- 172.21.15.7:0/1277874187 >> v2:172.21.15.7:3300/0 conn(0x7f950406fe40 0x7f950415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.350+0000 7f9501e00640 1 --2- 172.21.15.7:0/1277874187 >> v2:172.21.15.87:3300/0 conn(0x7f95040ce8d0 0x7f95040cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.351+0000 7f9502800640 1 -- 172.21.15.7:0/1277874187 >> v2:172.21.15.87:3300/0 conn(0x7f95040ce8d0 msgr2=0x7f95040cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.351+0000 7f9502800640 1 --2- 172.21.15.7:0/1277874187 >> v2:172.21.15.87:3300/0 conn(0x7f95040ce8d0 0x7f95040cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.351+0000 7f9502800640 1 -- 172.21.15.7:0/1277874187 >> v1:172.21.15.7:6789/0 conn(0x7f950415c2e0 legacy=0x7f950406f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.351+0000 7f9502800640 1 -- 172.21.15.7:0/1277874187 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95040b4df0 con 0x7f950406fe40 2024-04-25T11:14:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.351+0000 7f9500a00640 1 -- 172.21.15.7:0/1277874187 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2420815967 0 0) 0x7f95040b4830 con 0x7f950415c2e0 2024-04-25T11:14:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.352+0000 7f9502800640 1 --2- 172.21.15.7:0/1277874187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f950406fe40 0x7f950415c6f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9504072330 tx=0x7f94fc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e585ae09c4692c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.353+0000 7f9500a00640 1 -- 172.21.15.7:0/1277874187 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc007740 con 0x7f950406fe40 2024-04-25T11:14:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.353+0000 7f9500a00640 1 -- 172.21.15.7:0/1277874187 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94fc0078e0 con 0x7f950406fe40 2024-04-25T11:14:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.353+0000 7f9500a00640 1 -- 172.21.15.7:0/1277874187 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc007cb0 con 0x7f950406fe40 2024-04-25T11:14:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.353+0000 7f9503200640 1 -- 172.21.15.7:0/1277874187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f950406fe40 msgr2=0x7f950415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.353+0000 7f9503200640 1 --2- 172.21.15.7:0/1277874187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f950406fe40 0x7f950415c6f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9504072330 tx=0x7f94fc02a680 comp rx=0 tx=0).stop 2024-04-25T11:14:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.354+0000 7f9503200640 1 -- 172.21.15.7:0/1277874187 shutdown_connections 2024-04-25T11:14:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.354+0000 7f9503200640 1 --2- 172.21.15.7:0/1277874187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f950406fe40 0x7f950415c6f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.354+0000 7f9503200640 1 --2- 172.21.15.7:0/1277874187 >> v2:172.21.15.87:3300/0 conn(0x7f95040ce8d0 0x7f95040cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.354+0000 7f9503200640 1 -- 172.21.15.7:0/1277874187 >> 172.21.15.7:0/1277874187 conn(0x7f95040740f0 msgr2=0x7f950406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.354+0000 7f9503200640 1 -- 172.21.15.7:0/1277874187 shutdown_connections 2024-04-25T11:14:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.354+0000 7f9503200640 1 -- 172.21.15.7:0/1277874187 wait complete. 2024-04-25T11:14:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.356+0000 7f9503200640 1 Processor -- start 2024-04-25T11:14:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.357+0000 7f9503200640 1 -- start start 2024-04-25T11:14:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.358+0000 7f9503200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95040ce8d0 0x7f9504157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.358+0000 7f9503200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f950415c2e0 0x7f95041f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.358+0000 7f9503200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95041f5b00 0x7f950415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.358+0000 7f9503200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9504070ff0 con 0x7f95040ce8d0 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.358+0000 7f9503200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9504070e70 con 0x7f95041f5b00 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.358+0000 7f9503200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f950406eea0 con 0x7f950415c2e0 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.358+0000 7f9501e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95040ce8d0 0x7f9504157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.359+0000 7f9501e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95040ce8d0 0x7f9504157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38108/0 (socket says 172.21.15.7:38108) 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.359+0000 7f9501e00640 1 -- 172.21.15.7:0/31835449 learned_addr learned my addr 172.21.15.7:0/31835449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.359+0000 7f9501400640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f950415c2e0 0x7f95041f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.359+0000 7f9502800640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95041f5b00 0x7f950415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.359+0000 7f9501e00640 1 -- 172.21.15.7:0/31835449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f950415c2e0 msgr2=0x7f95041f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.359+0000 7f9501e00640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f950415c2e0 0x7f95041f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.359+0000 7f9501e00640 1 -- 172.21.15.7:0/31835449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95041f5b00 msgr2=0x7f950415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.359+0000 7f9501e00640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95041f5b00 0x7f950415cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.359+0000 7f9501e00640 1 -- 172.21.15.7:0/31835449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95041588c0 con 0x7f95040ce8d0 2024-04-25T11:14:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.359+0000 7f9501400640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f950415c2e0 0x7f95041f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.359+0000 7f9502800640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95041f5b00 0x7f950415cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.360+0000 7f9501e00640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95040ce8d0 0x7f9504157e40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f94f400d5d0 tx=0x7f94f400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.360+0000 7f94fa000640 1 -- 172.21.15.7:0/31835449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4004240 con 0x7f95040ce8d0 2024-04-25T11:14:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.360+0000 7f94fa000640 1 -- 172.21.15.7:0/31835449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94f40043e0 con 0x7f95040ce8d0 2024-04-25T11:14:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.360+0000 7f94fa000640 1 -- 172.21.15.7:0/31835449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f4003830 con 0x7f95040ce8d0 2024-04-25T11:14:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.360+0000 7f9503200640 1 -- 172.21.15.7:0/31835449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9504158ba0 con 0x7f95040ce8d0 2024-04-25T11:14:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.360+0000 7f9503200640 1 -- 172.21.15.7:0/31835449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9504158380 con 0x7f95040ce8d0 2024-04-25T11:14:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.361+0000 7f9503200640 1 -- 172.21.15.7:0/31835449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9504050740 con 0x7f95040ce8d0 2024-04-25T11:14:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.363+0000 7f94fa000640 1 -- 172.21.15.7:0/31835449 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f94f40039d0 con 0x7f95040ce8d0 2024-04-25T11:14:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.363+0000 7f94fa000640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f94e40744b0 0x7f94e40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.364+0000 7f94fa000640 1 -- 172.21.15.7:0/31835449 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f94f4095630 con 0x7f95040ce8d0 2024-04-25T11:14:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.364+0000 7f9501400640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f94e40744b0 0x7f94e40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.365+0000 7f9501400640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f94e40744b0 0x7f94e40769c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f950415d860 tx=0x7f94f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.368+0000 7f94fa000640 1 -- 172.21.15.7:0/31835449 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f94f405fcb0 con 0x7f95040ce8d0 2024-04-25T11:14:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.571+0000 7f9503200640 1 -- 172.21.15.7:0/31835449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95040709b0 con 0x7f95040ce8d0 2024-04-25T11:14:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.572+0000 7f94fa000640 1 -- 172.21.15.7:0/31835449 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) ==== 104+0+11 (secure 0 0 0) 0x7f94f405f5b0 con 0x7f95040ce8d0 2024-04-25T11:14:39.572 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:14:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 -- 172.21.15.7:0/31835449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f94e40744b0 msgr2=0x7f94e40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f94e40744b0 0x7f94e40769c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f950415d860 tx=0x7f94f0009040 comp rx=0 tx=0).stop 2024-04-25T11:14:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 -- 172.21.15.7:0/31835449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95040ce8d0 msgr2=0x7f9504157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95040ce8d0 0x7f9504157e40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f94f400d5d0 tx=0x7f94f400da90 comp rx=0 tx=0).stop 2024-04-25T11:14:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 -- 172.21.15.7:0/31835449 shutdown_connections 2024-04-25T11:14:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f94e40744b0 0x7f94e40769c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95041f5b00 0x7f950415cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f950415c2e0 0x7f95041f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 --2- 172.21.15.7:0/31835449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95040ce8d0 0x7f9504157e40 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 -- 172.21.15.7:0/31835449 >> 172.21.15.7:0/31835449 conn(0x7f95040740f0 msgr2=0x7f950414efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 -- 172.21.15.7:0/31835449 shutdown_connections 2024-04-25T11:14:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.575+0000 7f9503200640 1 -- 172.21.15.7:0/31835449 wait complete. 2024-04-25T11:14:39.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:14:39.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.885+0000 7ffa31600640 1 Processor -- start 2024-04-25T11:14:39.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.885+0000 7ffa31600640 1 -- start start 2024-04-25T11:14:39.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.887+0000 7ffa31600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffa2c0b4690 0x7ffa2c0b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.887+0000 7ffa31600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa2c144630 0x7ffa2c144a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.888+0000 7ffa31600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa2c0b5c40 con 0x7ffa2c143b10 2024-04-25T11:14:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.888+0000 7ffa31600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ffa2c0b5f20 con 0x7ffa2c144630 2024-04-25T11:14:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.888+0000 7ffa31600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ffa2c0b6200 con 0x7ffa2c0b4690 2024-04-25T11:14:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.888+0000 7ffa30c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa2c144630 0x7ffa2c144a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.888+0000 7ffa23e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffa2c143b10 0x7ffa2c143f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53446/0 (socket says 172.21.15.7:53446) 2024-04-25T11:14:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.888+0000 7ffa30c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa2c144630 0x7ffa2c144a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57016/0 (socket says 172.21.15.7:57016) 2024-04-25T11:14:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.888+0000 7ffa23e00640 1 -- 172.21.15.7:0/1912974855 learned_addr learned my addr 172.21.15.7:0/1912974855 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.888+0000 7ffa2be00640 1 --2- 172.21.15.7:0/1912974855 >> v2:172.21.15.87:3300/0 conn(0x7ffa2c0b4690 0x7ffa2c0b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.888+0000 7ffa30c00640 1 -- 172.21.15.7:0/1912974855 >> v2:172.21.15.87:3300/0 conn(0x7ffa2c0b4690 msgr2=0x7ffa2c0b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.888+0000 7ffa30c00640 1 --2- 172.21.15.7:0/1912974855 >> v2:172.21.15.87:3300/0 conn(0x7ffa2c0b4690 0x7ffa2c0b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.889+0000 7ffa30c00640 1 -- 172.21.15.7:0/1912974855 >> v1:172.21.15.7:6789/0 conn(0x7ffa2c143b10 legacy=0x7ffa2c143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.889+0000 7ffa30c00640 1 -- 172.21.15.7:0/1912974855 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c0b64e0 con 0x7ffa2c144630 2024-04-25T11:14:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.889+0000 7ffa2b400640 1 -- 172.21.15.7:0/1912974855 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3252759196 0 0) 0x7ffa2c0b5c40 con 0x7ffa2c143b10 2024-04-25T11:14:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.890+0000 7ffa30c00640 1 --2- 172.21.15.7:0/1912974855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2c144630 0x7ffa2c144a40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c06e8e0 tx=0x7ffa2402c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad94bbc1493d4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.890+0000 7ffa2b400640 1 -- 172.21.15.7:0/1912974855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24009540 con 0x7ffa2c144630 2024-04-25T11:14:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.890+0000 7ffa2b400640 1 -- 172.21.15.7:0/1912974855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa24037070 con 0x7ffa2c144630 2024-04-25T11:14:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.890+0000 7ffa2b400640 1 -- 172.21.15.7:0/1912974855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24009900 con 0x7ffa2c144630 2024-04-25T11:14:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.891+0000 7ffa31600640 1 -- 172.21.15.7:0/1912974855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2c144630 msgr2=0x7ffa2c144a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.891+0000 7ffa31600640 1 --2- 172.21.15.7:0/1912974855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2c144630 0x7ffa2c144a40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c06e8e0 tx=0x7ffa2402c910 comp rx=0 tx=0).stop 2024-04-25T11:14:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.891+0000 7ffa31600640 1 -- 172.21.15.7:0/1912974855 shutdown_connections 2024-04-25T11:14:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.891+0000 7ffa31600640 1 --2- 172.21.15.7:0/1912974855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2c144630 0x7ffa2c144a40 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.891+0000 7ffa31600640 1 --2- 172.21.15.7:0/1912974855 >> v2:172.21.15.87:3300/0 conn(0x7ffa2c0b4690 0x7ffa2c0b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.891+0000 7ffa31600640 1 -- 172.21.15.7:0/1912974855 >> 172.21.15.7:0/1912974855 conn(0x7ffa2c074140 msgr2=0x7ffa2c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.891+0000 7ffa31600640 1 -- 172.21.15.7:0/1912974855 shutdown_connections 2024-04-25T11:14:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.891+0000 7ffa31600640 1 -- 172.21.15.7:0/1912974855 wait complete. 2024-04-25T11:14:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.893+0000 7ffa31600640 1 Processor -- start 2024-04-25T11:14:39.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.894+0000 7ffa31600640 1 -- start start 2024-04-25T11:14:39.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.895+0000 7ffa31600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2c0b4690 0x7ffa2c13e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.895+0000 7ffa31600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa2c143b10 0x7ffa2c13bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.895+0000 7ffa31600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffa2c144630 0x7ffa2c145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.895+0000 7ffa31600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c143200 con 0x7ffa2c0b4690 2024-04-25T11:14:39.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.895+0000 7ffa31600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c070fa0 con 0x7ffa2c143b10 2024-04-25T11:14:39.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.895+0000 7ffa31600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ffa2c143380 con 0x7ffa2c144630 2024-04-25T11:14:39.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.895+0000 7ffa2be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2c0b4690 0x7ffa2c13e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:39.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.895+0000 7ffa2be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2c0b4690 0x7ffa2c13e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57032/0 (socket says 172.21.15.7:57032) 2024-04-25T11:14:39.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.896+0000 7ffa2be00640 1 -- 172.21.15.7:0/182101158 learned_addr learned my addr 172.21.15.7:0/182101158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:39.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.896+0000 7ffa2be00640 1 -- 172.21.15.7:0/182101158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffa2c144630 msgr2=0x7ffa2c145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.896+0000 7ffa2be00640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffa2c144630 0x7ffa2c145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.896+0000 7ffa2be00640 1 -- 172.21.15.7:0/182101158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa2c143b10 msgr2=0x7ffa2c13bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:39.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.896+0000 7ffa2be00640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa2c143b10 0x7ffa2c13bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:39.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.896+0000 7ffa2be00640 1 -- 172.21.15.7:0/182101158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c152000 con 0x7ffa2c0b4690 2024-04-25T11:14:39.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.896+0000 7ffa2be00640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2c0b4690 0x7ffa2c13e5e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00d620 tx=0x7ffa1c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:39.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.897+0000 7ffa28c00640 1 -- 172.21.15.7:0/182101158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c00b620 con 0x7ffa2c0b4690 2024-04-25T11:14:39.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.897+0000 7ffa28c00640 1 -- 172.21.15.7:0/182101158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c019070 con 0x7ffa2c0b4690 2024-04-25T11:14:39.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.897+0000 7ffa28c00640 1 -- 172.21.15.7:0/182101158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c00ba00 con 0x7ffa2c0b4690 2024-04-25T11:14:39.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.897+0000 7ffa31600640 1 -- 172.21.15.7:0/182101158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2c13eb20 con 0x7ffa2c0b4690 2024-04-25T11:14:39.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.897+0000 7ffa31600640 1 -- 172.21.15.7:0/182101158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2c0caf40 con 0x7ffa2c0b4690 2024-04-25T11:14:39.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.898+0000 7ffa31600640 1 -- 172.21.15.7:0/182101158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa2c050740 con 0x7ffa2c0b4690 2024-04-25T11:14:39.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.899+0000 7ffa28c00640 1 -- 172.21.15.7:0/182101158 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffa1c00bba0 con 0x7ffa2c0b4690 2024-04-25T11:14:39.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.901+0000 7ffa28c00640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffa080744e0 0x7ffa080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:39.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.901+0000 7ffa28c00640 1 -- 172.21.15.7:0/182101158 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7ffa1c095810 con 0x7ffa2c0b4690 2024-04-25T11:14:39.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.902+0000 7ffa23e00640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffa080744e0 0x7ffa080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:39.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.902+0000 7ffa23e00640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffa080744e0 0x7ffa080769f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffa14007fd0 tx=0x7ffa14007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:39.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:39.906+0000 7ffa28c00640 1 -- 172.21.15.7:0/182101158 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffa1c05fe00 con 0x7ffa2c0b4690 2024-04-25T11:14:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.114+0000 7ffa31600640 1 -- 172.21.15.7:0/182101158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa2c1521d0 con 0x7ffa2c0b4690 2024-04-25T11:14:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.115+0000 7ffa28c00640 1 -- 172.21.15.7:0/182101158 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) ==== 100+0+11 (secure 0 0 0) 0x7ffa1c05f700 con 0x7ffa2c0b4690 2024-04-25T11:14:40.116 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:14:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 -- 172.21.15.7:0/182101158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffa080744e0 msgr2=0x7ffa080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffa080744e0 0x7ffa080769f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffa14007fd0 tx=0x7ffa14007960 comp rx=0 tx=0).stop 2024-04-25T11:14:40.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 -- 172.21.15.7:0/182101158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2c0b4690 msgr2=0x7ffa2c13e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2c0b4690 0x7ffa2c13e5e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00d620 tx=0x7ffa1c00dae0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 -- 172.21.15.7:0/182101158 shutdown_connections 2024-04-25T11:14:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffa080744e0 0x7ffa080769f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffa2c144630 0x7ffa2c145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa2c143b10 0x7ffa2c13bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 --2- 172.21.15.7:0/182101158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa2c0b4690 0x7ffa2c13e5e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 -- 172.21.15.7:0/182101158 >> 172.21.15.7:0/182101158 conn(0x7ffa2c074140 msgr2=0x7ffa2c144bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 -- 172.21.15.7:0/182101158 shutdown_connections 2024-04-25T11:14:40.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.119+0000 7ffa31600640 1 -- 172.21.15.7:0/182101158 wait complete. 2024-04-25T11:14:40.147 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:14:40.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.429+0000 7fd32fe00640 1 Processor -- start 2024-04-25T11:14:40.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.429+0000 7fd32fe00640 1 -- start start 2024-04-25T11:14:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.431+0000 7fd32fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd330156be0 0x7fd33006ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.431+0000 7fd32fe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3300cf1a0 con 0x7fd33006d340 2024-04-25T11:14:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.431+0000 7fd32fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd33006d750 con 0x7fd3300cfb30 2024-04-25T11:14:40.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.431+0000 7fd32fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd330070fc0 con 0x7fd330156be0 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.431+0000 7fd32f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd33006d340 0x7fd3300b5670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53448/0 (socket says 172.21.15.7:53448) 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.431+0000 7fd32f400640 1 -- 172.21.15.7:0/1027738549 learned_addr learned my addr 172.21.15.7:0/1027738549 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.432+0000 7fd32e000640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd330156be0 0x7fd33006ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.432+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291009310 0 0) 0x7fd328001490 con 0x7fd33006d340 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.432+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd314003440 con 0x7fd33006d340 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.432+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2382570227 0 0) 0x7fd314003440 con 0x7fd33006d340 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.432+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd330156be0 msgr2=0x7fd33006ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.432+0000 7fd32d600640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd330156be0 0x7fd33006ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.432+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 >> v1:172.21.15.87:6789/0 conn(0x7fd3300cfb30 legacy=0x7fd33015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.432+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3300b4c10 con 0x7fd33006d340 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.432+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fd328001bc0 con 0x7fd33006d340 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.432+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 >> v1:172.21.15.7:6789/0 conn(0x7fd33006d340 legacy=0x7fd3300b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32d600640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd314003ad0 0x7fd314003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:40.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32d600640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd330156be0 0x7fd31400a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32d600640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd31400b0a0 0x7fd31400d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd330070fc0 con 0x7fd314003ad0 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd31400dd60 con 0x7fd31400b0a0 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd31400df60 con 0x7fd330156be0 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32ea00640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd314003ad0 0x7fd314003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32e000640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd330156be0 0x7fd31400a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32f400640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd31400b0a0 0x7fd31400d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32ea00640 1 -- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd330156be0 msgr2=0x7fd31400a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32ea00640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd330156be0 0x7fd31400a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32ea00640 1 -- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd31400b0a0 msgr2=0x7fd31400d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32ea00640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd31400b0a0 0x7fd31400d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32ea00640 1 -- 172.21.15.7:0/1027738549 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd314003440 con 0x7fd314003ad0 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.434+0000 7fd32e000640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd330156be0 0x7fd31400a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:40.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.433+0000 7fd32f400640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd31400b0a0 0x7fd31400d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.435+0000 7fd32ea00640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd314003ad0 0x7fd314003f60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd330072e80 tx=0x7fd314003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc8fc1b751b52a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.435+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd320001370 con 0x7fd314003ad0 2024-04-25T11:14:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.435+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3200052d0 con 0x7fd314003ad0 2024-04-25T11:14:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.435+0000 7fd32d600640 1 -- 172.21.15.7:0/1027738549 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32002ab90 con 0x7fd314003ad0 2024-04-25T11:14:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.436+0000 7fd32fe00640 1 -- 172.21.15.7:0/1027738549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd314003ad0 msgr2=0x7fd314003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.436+0000 7fd32fe00640 1 --2- 172.21.15.7:0/1027738549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd314003ad0 0x7fd314003f60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd330072e80 tx=0x7fd314003730 comp rx=0 tx=0).stop 2024-04-25T11:14:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.436+0000 7fd32f400640 1 -- 172.21.15.7:0/1027738549 reap_dead start 2024-04-25T11:14:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.436+0000 7fd32fe00640 1 -- 172.21.15.7:0/1027738549 shutdown_connections 2024-04-25T11:14:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.436+0000 7fd32fe00640 1 -- 172.21.15.7:0/1027738549 >> 172.21.15.7:0/1027738549 conn(0x7fd3300d1540 msgr2=0x7fd3300d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.436+0000 7fd32fe00640 1 -- 172.21.15.7:0/1027738549 shutdown_connections 2024-04-25T11:14:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.436+0000 7fd32fe00640 1 -- 172.21.15.7:0/1027738549 wait complete. 2024-04-25T11:14:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.438+0000 7fd32fe00640 1 Processor -- start 2024-04-25T11:14:40.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.438+0000 7fd32fe00640 1 -- start start 2024-04-25T11:14:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.440+0000 7fd32fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd314003ad0 0x7fd33014be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.440+0000 7fd32fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3300c6570 0x7fd3300c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.440+0000 7fd32fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd33015b400 0x7fd33015b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.440+0000 7fd32fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3300b4eb0 con 0x7fd3300c6570 2024-04-25T11:14:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.440+0000 7fd32fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd3300b51e0 con 0x7fd33015b400 2024-04-25T11:14:40.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.440+0000 7fd32fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd3300cfff0 con 0x7fd314003ad0 2024-04-25T11:14:40.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.440+0000 7fd32e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3300c6570 0x7fd3300c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:40.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.440+0000 7fd32e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3300c6570 0x7fd3300c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57058/0 (socket says 172.21.15.7:57058) 2024-04-25T11:14:40.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.440+0000 7fd32e000640 1 -- 172.21.15.7:0/247650210 learned_addr learned my addr 172.21.15.7:0/247650210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.441+0000 7fd32e000640 1 -- 172.21.15.7:0/247650210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd314003ad0 msgr2=0x7fd33014be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.441+0000 7fd32ea00640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd314003ad0 0x7fd33014be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.441+0000 7fd32e000640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd314003ad0 0x7fd33014be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.441+0000 7fd32e000640 1 -- 172.21.15.7:0/247650210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd33015b400 msgr2=0x7fd33015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.441+0000 7fd32e000640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd33015b400 0x7fd33015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.441+0000 7fd32e000640 1 -- 172.21.15.7:0/247650210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33015bd50 con 0x7fd3300c6570 2024-04-25T11:14:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.441+0000 7fd32e000640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3300c6570 0x7fd3300c6980 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0108b0 tx=0x7fd31c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.442+0000 7fd326a00640 1 -- 172.21.15.7:0/247650210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c00f040 con 0x7fd3300c6570 2024-04-25T11:14:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.442+0000 7fd326a00640 1 -- 172.21.15.7:0/247650210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c00eb60 con 0x7fd3300c6570 2024-04-25T11:14:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.442+0000 7fd326a00640 1 -- 172.21.15.7:0/247650210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c01c400 con 0x7fd3300c6570 2024-04-25T11:14:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.442+0000 7fd32fe00640 1 -- 172.21.15.7:0/247650210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3300c72a0 con 0x7fd3300c6570 2024-04-25T11:14:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.442+0000 7fd32fe00640 1 -- 172.21.15.7:0/247650210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3300c7860 con 0x7fd3300c6570 2024-04-25T11:14:40.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.443+0000 7fd32fe00640 1 -- 172.21.15.7:0/247650210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3300c7010 con 0x7fd3300c6570 2024-04-25T11:14:40.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.444+0000 7fd326a00640 1 -- 172.21.15.7:0/247650210 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd31c01c5a0 con 0x7fd3300c6570 2024-04-25T11:14:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.445+0000 7fd326a00640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3100744b0 0x7fd3100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:40.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.446+0000 7fd326a00640 1 -- 172.21.15.7:0/247650210 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7fd31c095af0 con 0x7fd3300c6570 2024-04-25T11:14:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.446+0000 7fd32ea00640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3100744b0 0x7fd3100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:40.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.447+0000 7fd32ea00640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3100744b0 0x7fd3100769c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd320032000 tx=0x7fd32002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.451+0000 7fd326a00640 1 -- 172.21.15.7:0/247650210 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd33015bd50 con 0x7fd3300c6570 2024-04-25T11:14:40.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.675+0000 7fd32fe00640 1 -- 172.21.15.7:0/247650210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd330050740 con 0x7fd3300c6570 2024-04-25T11:14:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.675+0000 7fd326a00640 1 -- 172.21.15.7:0/247650210 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7fd31c018030 con 0x7fd3300c6570 2024-04-25T11:14:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:14:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.678+0000 7fd32fe00640 1 -- 172.21.15.7:0/247650210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3100744b0 msgr2=0x7fd3100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.678+0000 7fd32fe00640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3100744b0 0x7fd3100769c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd320032000 tx=0x7fd32002e000 comp rx=0 tx=0).stop 2024-04-25T11:14:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.678+0000 7fd32fe00640 1 -- 172.21.15.7:0/247650210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3300c6570 msgr2=0x7fd3300c6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.678+0000 7fd32fe00640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3300c6570 0x7fd3300c6980 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0108b0 tx=0x7fd31c010d70 comp rx=0 tx=0).stop 2024-04-25T11:14:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.679+0000 7fd32fe00640 1 -- 172.21.15.7:0/247650210 shutdown_connections 2024-04-25T11:14:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.679+0000 7fd32fe00640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3100744b0 0x7fd3100769c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.679+0000 7fd32fe00640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd33015b400 0x7fd33015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.679+0000 7fd32fe00640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3300c6570 0x7fd3300c6980 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.679+0000 7fd32fe00640 1 --2- 172.21.15.7:0/247650210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd314003ad0 0x7fd33014be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.679+0000 7fd32fe00640 1 -- 172.21.15.7:0/247650210 >> 172.21.15.7:0/247650210 conn(0x7fd3300d1540 msgr2=0x7fd3300b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.679+0000 7fd32fe00640 1 -- 172.21.15.7:0/247650210 shutdown_connections 2024-04-25T11:14:40.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:40.679+0000 7fd32fe00640 1 -- 172.21.15.7:0/247650210 wait complete. 2024-04-25T11:14:40.707 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:14:40.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:40.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:14:40.808 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:14:40.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:40.808 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:14:40.863 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:14:40.863 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:14:40.863 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:40.925 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:14:40.925 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:40.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:41.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.273+0000 7f1735e00640 1 Processor -- start 2024-04-25T11:14:41.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.274+0000 7f1735e00640 1 -- start start 2024-04-25T11:14:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.276+0000 7f1735e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17300af540 0x7f173009f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.276+0000 7f1735e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f173015bc40 0x7f17300cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.276+0000 7f1735e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f173015f4e0 con 0x7f173015ef90 2024-04-25T11:14:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.276+0000 7f1735e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1730071190 con 0x7f173015bc40 2024-04-25T11:14:41.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.276+0000 7f1735e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f17300b0560 con 0x7f17300af540 2024-04-25T11:14:41.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.277+0000 7f172ea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f173015ef90 0x7f17300aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53454/0 (socket says 172.21.15.7:53454) 2024-04-25T11:14:41.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.277+0000 7f172ea00640 1 -- 172.21.15.7:0/382338856 learned_addr learned my addr 172.21.15.7:0/382338856 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:41.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.277+0000 7f172f400640 1 --2- 172.21.15.7:0/382338856 >> v2:172.21.15.7:3300/0 conn(0x7f173015bc40 0x7f17300cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.277+0000 7f172e000640 1 --2- 172.21.15.7:0/382338856 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17300af540 0x7f173009f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.278+0000 7f172f400640 1 -- 172.21.15.7:0/382338856 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17300af540 msgr2=0x7f173009f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:41.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.278+0000 7f172f400640 1 --2- 172.21.15.7:0/382338856 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17300af540 0x7f173009f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.278+0000 7f172f400640 1 -- 172.21.15.7:0/382338856 >> v1:172.21.15.7:6789/0 conn(0x7f173015ef90 legacy=0x7f17300aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.278+0000 7f172d600640 1 -- 172.21.15.7:0/382338856 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 140388443 0 0) 0x7f173015f4e0 con 0x7f173015ef90 2024-04-25T11:14:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.278+0000 7f172f400640 1 -- 172.21.15.7:0/382338856 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17300b0840 con 0x7f173015bc40 2024-04-25T11:14:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.279+0000 7f172f400640 1 --2- 172.21.15.7:0/382338856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173015bc40 0x7f17300cf980 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1728009d90 tx=0x7f172802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351bad7603c43fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.279+0000 7f172d600640 1 -- 172.21.15.7:0/382338856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728031600 con 0x7f173015bc40 2024-04-25T11:14:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.279+0000 7f172d600640 1 -- 172.21.15.7:0/382338856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17280317a0 con 0x7f173015bc40 2024-04-25T11:14:41.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.279+0000 7f172d600640 1 -- 172.21.15.7:0/382338856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728031b70 con 0x7f173015bc40 2024-04-25T11:14:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.280+0000 7f1735e00640 1 -- 172.21.15.7:0/382338856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173015bc40 msgr2=0x7f17300cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:41.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.280+0000 7f1735e00640 1 --2- 172.21.15.7:0/382338856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173015bc40 0x7f17300cf980 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1728009d90 tx=0x7f172802a5d0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.280+0000 7f1735e00640 1 -- 172.21.15.7:0/382338856 shutdown_connections 2024-04-25T11:14:41.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.280+0000 7f1735e00640 1 --2- 172.21.15.7:0/382338856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173015bc40 0x7f17300cf980 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.280+0000 7f1735e00640 1 --2- 172.21.15.7:0/382338856 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17300af540 0x7f173009f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.280+0000 7f1735e00640 1 -- 172.21.15.7:0/382338856 >> 172.21.15.7:0/382338856 conn(0x7f1730074200 msgr2=0x7f173006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:41.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.280+0000 7f1735e00640 1 -- 172.21.15.7:0/382338856 shutdown_connections 2024-04-25T11:14:41.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.280+0000 7f1735e00640 1 -- 172.21.15.7:0/382338856 wait complete. 2024-04-25T11:14:41.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.283+0000 7f1735e00640 1 Processor -- start 2024-04-25T11:14:41.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.283+0000 7f1735e00640 1 -- start start 2024-04-25T11:14:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.284+0000 7f1735e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173015bc40 0x7f17301e8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.284+0000 7f1735e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f173015ef90 0x7f1730147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.284+0000 7f1735e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f173015da00 0x7f1730147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.284+0000 7f1735e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17300b1320 con 0x7f173015bc40 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.284+0000 7f1735e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f17300b0e00 con 0x7f173015da00 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.284+0000 7f1735e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f17300afb80 con 0x7f173015ef90 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.285+0000 7f172ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173015bc40 0x7f17301e8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.285+0000 7f172ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173015bc40 0x7f17301e8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57074/0 (socket says 172.21.15.7:57074) 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.285+0000 7f172ea00640 1 -- 172.21.15.7:0/2070925368 learned_addr learned my addr 172.21.15.7:0/2070925368 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.285+0000 7f172f400640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f173015da00 0x7f1730147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.285+0000 7f172e000640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f173015ef90 0x7f1730147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.285+0000 7f172ea00640 1 -- 172.21.15.7:0/2070925368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f173015ef90 msgr2=0x7f1730147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.285+0000 7f172ea00640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f173015ef90 0x7f1730147040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.285+0000 7f172ea00640 1 -- 172.21.15.7:0/2070925368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f173015da00 msgr2=0x7f1730147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.285+0000 7f172ea00640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f173015da00 0x7f1730147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.285+0000 7f172ea00640 1 -- 172.21.15.7:0/2070925368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17301e7940 con 0x7f173015bc40 2024-04-25T11:14:41.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.285+0000 7f172f400640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f173015da00 0x7f1730147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.286+0000 7f172ea00640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173015bc40 0x7f17301e8e50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f172000c430 tx=0x7f172000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.286+0000 7f1727400640 1 -- 172.21.15.7:0/2070925368 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17200074e0 con 0x7f173015bc40 2024-04-25T11:14:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.286+0000 7f1727400640 1 -- 172.21.15.7:0/2070925368 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17200047e0 con 0x7f173015bc40 2024-04-25T11:14:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.286+0000 7f1727400640 1 -- 172.21.15.7:0/2070925368 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720004bd0 con 0x7f173015bc40 2024-04-25T11:14:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.286+0000 7f1735e00640 1 -- 172.21.15.7:0/2070925368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17301e7db0 con 0x7f173015bc40 2024-04-25T11:14:41.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.286+0000 7f1735e00640 1 -- 172.21.15.7:0/2070925368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17301e8370 con 0x7f173015bc40 2024-04-25T11:14:41.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.287+0000 7f1735e00640 1 -- 172.21.15.7:0/2070925368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f173006a100 con 0x7f173015bc40 2024-04-25T11:14:41.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.289+0000 7f1727400640 1 -- 172.21.15.7:0/2070925368 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1720011070 con 0x7f173015bc40 2024-04-25T11:14:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.290+0000 7f1727400640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f170c0744b0 0x7f170c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.290+0000 7f1727400640 1 -- 172.21.15.7:0/2070925368 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f1720094f80 con 0x7f173015bc40 2024-04-25T11:14:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.291+0000 7f172e000640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f170c0744b0 0x7f170c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.291+0000 7f172e000640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f170c0744b0 0x7f170c0769c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f17300d08a0 tx=0x7f1718008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:41.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.296+0000 7f1727400640 1 -- 172.21.15.7:0/2070925368 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f172005f570 con 0x7f173015bc40 2024-04-25T11:14:41.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.503+0000 7f1735e00640 1 -- 172.21.15.7:0/2070925368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17300b1f70 con 0x7f173015bc40 2024-04-25T11:14:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.507+0000 7f1727400640 1 -- 172.21.15.7:0/2070925368 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16788 (secure 0 0 0) 0x7f172005ee70 con 0x7f173015bc40 2024-04-25T11:14:41.507 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:22.341351+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:14:19.115741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T11:14:19.849856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.509+0000 7f1735e00640 1 -- 172.21.15.7:0/2070925368 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f170c0744b0 msgr2=0x7f170c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.509+0000 7f1735e00640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f170c0744b0 0x7f170c0769c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f17300d08a0 tx=0x7f1718008040 comp rx=0 tx=0).stop 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.509+0000 7f1735e00640 1 -- 172.21.15.7:0/2070925368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173015bc40 msgr2=0x7f17301e8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.509+0000 7f1735e00640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173015bc40 0x7f17301e8e50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f172000c430 tx=0x7f172000c8f0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.509+0000 7f1735e00640 1 -- 172.21.15.7:0/2070925368 shutdown_connections 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.509+0000 7f1735e00640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f170c0744b0 0x7f170c0769c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.509+0000 7f1735e00640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f173015da00 0x7f1730147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.510+0000 7f1735e00640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f173015ef90 0x7f1730147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.510+0000 7f1735e00640 1 --2- 172.21.15.7:0/2070925368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f173015bc40 0x7f17301e8e50 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.510+0000 7f1735e00640 1 -- 172.21.15.7:0/2070925368 >> 172.21.15.7:0/2070925368 conn(0x7f1730074200 msgr2=0x7f17300b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.510+0000 7f1735e00640 1 -- 172.21.15.7:0/2070925368 shutdown_connections 2024-04-25T11:14:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.510+0000 7f1735e00640 1 -- 172.21.15.7:0/2070925368 wait complete. 2024-04-25T11:14:41.537 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:14:19.115741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:14:19.849856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:14:41.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:14:41.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.821+0000 7f2018800640 1 Processor -- start 2024-04-25T11:14:41.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.821+0000 7f2018800640 1 -- start start 2024-04-25T11:14:41.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.824+0000 7f2018800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20100affc0 0x7f2010167d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.824+0000 7f2018800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20100b5730 con 0x7f201015aaa0 2024-04-25T11:14:41.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.824+0000 7f2018800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f201006f5b0 con 0x7f20100d0ee0 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.824+0000 7f2018800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f20100af290 con 0x7f20100affc0 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.824+0000 7f2015e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f201015aaa0 0x7f20100b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53464/0 (socket says 172.21.15.7:53464) 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.824+0000 7f2015e00640 1 -- 172.21.15.7:0/1870803392 learned_addr learned my addr 172.21.15.7:0/1870803392 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.824+0000 7f2016800640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20100affc0 0x7f2010167d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.824+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1898194060 0 0) 0x7f2004001110 con 0x7f201015aaa0 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ff8005730 con 0x7f201015aaa0 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1948614270 0 0) 0x7f1ff8005730 con 0x7f201015aaa0 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20100affc0 msgr2=0x7f2010167d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20100affc0 0x7f2010167d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 >> v1:172.21.15.87:6789/0 conn(0x7f20100d0ee0 legacy=0x7f20100b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20100d14a0 con 0x7f201015aaa0 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2004001840 con 0x7f201015aaa0 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 >> v1:172.21.15.7:6789/0 conn(0x7f201015aaa0 legacy=0x7f20100b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20100affc0 0x7f1ff8005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ff8006220 0x7f1ff8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff8006f50 0x7f1ff8007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f20100af290 con 0x7f1ff8006f50 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff800df00 con 0x7f1ff8006220 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.825+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff800e100 con 0x7f20100affc0 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.826+0000 7f2016800640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff8006f50 0x7f1ff8007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.826+0000 7f2015e00640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20100affc0 0x7f1ff8005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.826+0000 7f2016800640 1 -- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20100affc0 msgr2=0x7f1ff8005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.826+0000 7f2016800640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20100affc0 0x7f1ff8005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.826+0000 7f2015400640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ff8006220 0x7f1ff8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.826+0000 7f2016800640 1 -- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ff8006220 msgr2=0x7f1ff8006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.826+0000 7f2016800640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ff8006220 0x7f1ff8006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.826+0000 7f2016800640 1 -- 172.21.15.7:0/1870803392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff8005730 con 0x7f1ff8006f50 2024-04-25T11:14:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.826+0000 7f2015400640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ff8006220 0x7f1ff8006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.827+0000 7f2016800640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff8006f50 0x7f1ff8007420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f200c007360 tx=0x7f1ff8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba98e5e72ddab803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.828+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c009be0 con 0x7f1ff8006f50 2024-04-25T11:14:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.828+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f200c036600 con 0x7f1ff8006f50 2024-04-25T11:14:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.828+0000 7f2014a00640 1 -- 172.21.15.7:0/1870803392 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0369d0 con 0x7f1ff8006f50 2024-04-25T11:14:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.828+0000 7f2018800640 1 -- 172.21.15.7:0/1870803392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff8006f50 msgr2=0x7f1ff8007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.828+0000 7f2018800640 1 --2- 172.21.15.7:0/1870803392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ff8006f50 0x7f1ff8007420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f200c007360 tx=0x7f1ff8007a40 comp rx=0 tx=0).stop 2024-04-25T11:14:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.828+0000 7f2016800640 1 -- 172.21.15.7:0/1870803392 reap_dead start 2024-04-25T11:14:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.829+0000 7f2018800640 1 -- 172.21.15.7:0/1870803392 shutdown_connections 2024-04-25T11:14:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.829+0000 7f2018800640 1 -- 172.21.15.7:0/1870803392 >> 172.21.15.7:0/1870803392 conn(0x7f20100742c0 msgr2=0x7f201006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.829+0000 7f2018800640 1 -- 172.21.15.7:0/1870803392 shutdown_connections 2024-04-25T11:14:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.829+0000 7f2018800640 1 -- 172.21.15.7:0/1870803392 wait complete. 2024-04-25T11:14:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.831+0000 7f2018800640 1 Processor -- start 2024-04-25T11:14:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.831+0000 7f2018800640 1 -- start start 2024-04-25T11:14:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.833+0000 7f2018800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ff8006f50 0x7f201014efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.833+0000 7f2018800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20101e6b00 0x7f20101e6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.833+0000 7f2018800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20101e7450 0x7f2010158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.833+0000 7f2018800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f20100afaa0 con 0x7f20101e6b00 2024-04-25T11:14:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.833+0000 7f2018800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f20100af7c0 con 0x7f20101e7450 2024-04-25T11:14:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.833+0000 7f2018800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f201006fc30 con 0x7f1ff8006f50 2024-04-25T11:14:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.833+0000 7f2015400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20101e6b00 0x7f20101e6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.833+0000 7f2015400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20101e6b00 0x7f20101e6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57096/0 (socket says 172.21.15.7:57096) 2024-04-25T11:14:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.833+0000 7f2015400640 1 -- 172.21.15.7:0/659559129 learned_addr learned my addr 172.21.15.7:0/659559129 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.833+0000 7f2016800640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20101e7450 0x7f2010158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.833+0000 7f2015e00640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ff8006f50 0x7f201014efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.834+0000 7f2015400640 1 -- 172.21.15.7:0/659559129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ff8006f50 msgr2=0x7f201014efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.834+0000 7f2015400640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ff8006f50 0x7f201014efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.834+0000 7f2015400640 1 -- 172.21.15.7:0/659559129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20101e7450 msgr2=0x7f2010158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.834+0000 7f2015400640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20101e7450 0x7f2010158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.834+0000 7f2015400640 1 -- 172.21.15.7:0/659559129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20100b20c0 con 0x7f20101e6b00 2024-04-25T11:14:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.834+0000 7f2015400640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20101e6b00 0x7f20101e6f10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f200000d800 tx=0x7f200000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.834+0000 7f200a000640 1 -- 172.21.15.7:0/659559129 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000005070 con 0x7f20101e6b00 2024-04-25T11:14:41.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.834+0000 7f200a000640 1 -- 172.21.15.7:0/659559129 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2000005210 con 0x7f20101e6b00 2024-04-25T11:14:41.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.834+0000 7f200a000640 1 -- 172.21.15.7:0/659559129 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000007030 con 0x7f20101e6b00 2024-04-25T11:14:41.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.834+0000 7f2018800640 1 -- 172.21.15.7:0/659559129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2010154600 con 0x7f20101e6b00 2024-04-25T11:14:41.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.835+0000 7f2018800640 1 -- 172.21.15.7:0/659559129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2010154bc0 con 0x7f20101e6b00 2024-04-25T11:14:41.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.836+0000 7f2018800640 1 -- 172.21.15.7:0/659559129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2010050740 con 0x7f20101e6b00 2024-04-25T11:14:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.838+0000 7f200a000640 1 -- 172.21.15.7:0/659559129 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f200001a030 con 0x7f20101e6b00 2024-04-25T11:14:41.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.839+0000 7f200a000640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ff40744e0 0x7f1ff40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:41.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.840+0000 7f200a000640 1 -- 172.21.15.7:0/659559129 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f2000094960 con 0x7f20101e6b00 2024-04-25T11:14:41.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.840+0000 7f2015e00640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ff40744e0 0x7f1ff40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:41.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.841+0000 7f2015e00640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ff40744e0 0x7f1ff40769f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2004001d50 tx=0x7f2004009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:41.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:41.845+0000 7f200a000640 1 -- 172.21.15.7:0/659559129 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2000021020 con 0x7f20101e6b00 2024-04-25T11:14:41.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:14:41.863 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:14:42.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.060+0000 7f2018800640 1 -- 172.21.15.7:0/659559129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2010070f20 con 0x7f20101e6b00 2024-04-25T11:14:42.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.061+0000 7f200a000640 1 -- 172.21.15.7:0/659559129 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7f200005f080 con 0x7f20101e6b00 2024-04-25T11:14:42.061 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:14:42.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 -- 172.21.15.7:0/659559129 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ff40744e0 msgr2=0x7f1ff40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:42.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ff40744e0 0x7f1ff40769f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2004001d50 tx=0x7f2004009040 comp rx=0 tx=0).stop 2024-04-25T11:14:42.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 -- 172.21.15.7:0/659559129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20101e6b00 msgr2=0x7f20101e6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:42.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20101e6b00 0x7f20101e6f10 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f200000d800 tx=0x7f200000dcc0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 -- 172.21.15.7:0/659559129 shutdown_connections 2024-04-25T11:14:42.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ff40744e0 0x7f1ff40769f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20101e7450 0x7f2010158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20101e6b00 0x7f20101e6f10 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 --2- 172.21.15.7:0/659559129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ff8006f50 0x7f201014efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 -- 172.21.15.7:0/659559129 >> 172.21.15.7:0/659559129 conn(0x7f20100742c0 msgr2=0x7f201006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:42.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 -- 172.21.15.7:0/659559129 shutdown_connections 2024-04-25T11:14:42.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.064+0000 7f2018800640 1 -- 172.21.15.7:0/659559129 wait complete. 2024-04-25T11:14:42.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:14:42.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.387+0000 7ff58ac00640 1 Processor -- start 2024-04-25T11:14:42.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.387+0000 7ff58ac00640 1 -- start start 2024-04-25T11:14:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.389+0000 7ff58ac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff584145e40 0x7ff584146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.389+0000 7ff58ac00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5840b4870 0x7ff5840b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.389+0000 7ff58ac00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5840b51c0 0x7ff584143a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.389+0000 7ff58ac00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff584145b70 con 0x7ff5840b4870 2024-04-25T11:14:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.389+0000 7ff58ac00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff5840b66b0 con 0x7ff5840b51c0 2024-04-25T11:14:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.389+0000 7ff58ac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5840b5b30 con 0x7ff584145e40 2024-04-25T11:14:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.390+0000 7ff588e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5840b4870 0x7ff5840b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.390+0000 7ff588e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5840b4870 0x7ff5840b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57102/0 (socket says 172.21.15.7:57102) 2024-04-25T11:14:42.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.390+0000 7ff588e00640 1 -- 172.21.15.7:0/2696085576 learned_addr learned my addr 172.21.15.7:0/2696085576 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.390+0000 7ff589800640 1 --2- 172.21.15.7:0/2696085576 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff584145e40 0x7ff584146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.390+0000 7ff58a200640 1 --2- 172.21.15.7:0/2696085576 >> v2:172.21.15.87:3300/0 conn(0x7ff5840b51c0 0x7ff584143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.390+0000 7ff588e00640 1 -- 172.21.15.7:0/2696085576 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff584145e40 msgr2=0x7ff584146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.390+0000 7ff588e00640 1 --2- 172.21.15.7:0/2696085576 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff584145e40 0x7ff584146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.391+0000 7ff588e00640 1 -- 172.21.15.7:0/2696085576 >> v2:172.21.15.87:3300/0 conn(0x7ff5840b51c0 msgr2=0x7ff584143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.391+0000 7ff588e00640 1 --2- 172.21.15.7:0/2696085576 >> v2:172.21.15.87:3300/0 conn(0x7ff5840b51c0 0x7ff584143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.391+0000 7ff588e00640 1 -- 172.21.15.7:0/2696085576 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5840b5e10 con 0x7ff5840b4870 2024-04-25T11:14:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.392+0000 7ff588e00640 1 --2- 172.21.15.7:0/2696085576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5840b4870 0x7ff5840b4c80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff584072f70 tx=0x7ff57002c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7bf8617ff07413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.392+0000 7ff583e00640 1 -- 172.21.15.7:0/2696085576 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570039070 con 0x7ff5840b4870 2024-04-25T11:14:42.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.392+0000 7ff583e00640 1 -- 172.21.15.7:0/2696085576 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff570034640 con 0x7ff5840b4870 2024-04-25T11:14:42.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.392+0000 7ff583e00640 1 -- 172.21.15.7:0/2696085576 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570034a10 con 0x7ff5840b4870 2024-04-25T11:14:42.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.393+0000 7ff58ac00640 1 -- 172.21.15.7:0/2696085576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5840b4870 msgr2=0x7ff5840b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:42.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.393+0000 7ff58ac00640 1 --2- 172.21.15.7:0/2696085576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5840b4870 0x7ff5840b4c80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff584072f70 tx=0x7ff57002c350 comp rx=0 tx=0).stop 2024-04-25T11:14:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.393+0000 7ff58ac00640 1 -- 172.21.15.7:0/2696085576 shutdown_connections 2024-04-25T11:14:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.393+0000 7ff58ac00640 1 --2- 172.21.15.7:0/2696085576 >> v2:172.21.15.87:3300/0 conn(0x7ff5840b51c0 0x7ff584143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.393+0000 7ff58ac00640 1 --2- 172.21.15.7:0/2696085576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5840b4870 0x7ff5840b4c80 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.393+0000 7ff58ac00640 1 --2- 172.21.15.7:0/2696085576 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff584145e40 0x7ff584146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.393+0000 7ff58ac00640 1 -- 172.21.15.7:0/2696085576 >> 172.21.15.7:0/2696085576 conn(0x7ff5840740f0 msgr2=0x7ff58406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.393+0000 7ff58ac00640 1 -- 172.21.15.7:0/2696085576 shutdown_connections 2024-04-25T11:14:42.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.393+0000 7ff58ac00640 1 -- 172.21.15.7:0/2696085576 wait complete. 2024-04-25T11:14:42.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.396+0000 7ff58ac00640 1 Processor -- start 2024-04-25T11:14:42.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.396+0000 7ff58ac00640 1 -- start start 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff58ac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5840b4870 0x7ff5841e7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff58ac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5840b51c0 0x7ff5841e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff58ac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff584145e40 0x7ff5841f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff58ac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5840b55d0 con 0x7ff5840b4870 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff58ac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff5840b63d0 con 0x7ff5840b51c0 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff58ac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5840b5750 con 0x7ff584145e40 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff589800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5840b4870 0x7ff5841e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff589800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5840b4870 0x7ff5841e7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57108/0 (socket says 172.21.15.7:57108) 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff589800640 1 -- 172.21.15.7:0/3785602473 learned_addr learned my addr 172.21.15.7:0/3785602473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff588e00640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5840b51c0 0x7ff5841e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff58a200640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff584145e40 0x7ff5841f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff589800640 1 -- 172.21.15.7:0/3785602473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff584145e40 msgr2=0x7ff5841f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff589800640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff584145e40 0x7ff5841f8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.398+0000 7ff589800640 1 -- 172.21.15.7:0/3785602473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5840b51c0 msgr2=0x7ff5841e7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.399+0000 7ff589800640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5840b51c0 0x7ff5841e7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.399+0000 7ff589800640 1 -- 172.21.15.7:0/3785602473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5840aa680 con 0x7ff5840b4870 2024-04-25T11:14:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.399+0000 7ff58a200640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff584145e40 0x7ff5841f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.399+0000 7ff589800640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5840b4870 0x7ff5841e7820 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff57800d620 tx=0x7ff57800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.399+0000 7ff581600640 1 -- 172.21.15.7:0/3785602473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57800b620 con 0x7ff5840b4870 2024-04-25T11:14:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.399+0000 7ff581600640 1 -- 172.21.15.7:0/3785602473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff578019070 con 0x7ff5840b4870 2024-04-25T11:14:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.399+0000 7ff58ac00640 1 -- 172.21.15.7:0/3785602473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5841f76a0 con 0x7ff5840b4870 2024-04-25T11:14:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.399+0000 7ff581600640 1 -- 172.21.15.7:0/3785602473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57800ba00 con 0x7ff5840b4870 2024-04-25T11:14:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.399+0000 7ff58ac00640 1 -- 172.21.15.7:0/3785602473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5841f7c60 con 0x7ff5840b4870 2024-04-25T11:14:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.401+0000 7ff58ac00640 1 -- 172.21.15.7:0/3785602473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff584068f10 con 0x7ff5840b4870 2024-04-25T11:14:42.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.402+0000 7ff581600640 1 -- 172.21.15.7:0/3785602473 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff57800bba0 con 0x7ff5840b4870 2024-04-25T11:14:42.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.403+0000 7ff581600640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5640744e0 0x7ff5640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.403+0000 7ff581600640 1 -- 172.21.15.7:0/3785602473 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7ff578095860 con 0x7ff5840b4870 2024-04-25T11:14:42.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.404+0000 7ff588e00640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5640744e0 0x7ff5640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.404+0000 7ff588e00640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5640744e0 0x7ff5640769f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff570036000 tx=0x7ff570030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.408+0000 7ff581600640 1 -- 172.21.15.7:0/3785602473 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff584068f10 con 0x7ff5840b4870 2024-04-25T11:14:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.618+0000 7ff58ac00640 1 -- 172.21.15.7:0/3785602473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff584050740 con 0x7ff5840b4870 2024-04-25T11:14:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.619+0000 7ff581600640 1 -- 172.21.15.7:0/3785602473 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) ==== 104+0+11 (secure 0 0 0) 0x7ff57805fe50 con 0x7ff5840b4870 2024-04-25T11:14:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:14:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 -- 172.21.15.7:0/3785602473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5640744e0 msgr2=0x7ff5640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5640744e0 0x7ff5640769f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff570036000 tx=0x7ff570030000 comp rx=0 tx=0).stop 2024-04-25T11:14:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 -- 172.21.15.7:0/3785602473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5840b4870 msgr2=0x7ff5841e7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5840b4870 0x7ff5841e7820 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff57800d620 tx=0x7ff57800dae0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 -- 172.21.15.7:0/3785602473 shutdown_connections 2024-04-25T11:14:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5640744e0 0x7ff5640769f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff584145e40 0x7ff5841f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5840b51c0 0x7ff5841e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 --2- 172.21.15.7:0/3785602473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5840b4870 0x7ff5841e7820 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 -- 172.21.15.7:0/3785602473 >> 172.21.15.7:0/3785602473 conn(0x7ff5840740f0 msgr2=0x7ff5840ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 -- 172.21.15.7:0/3785602473 shutdown_connections 2024-04-25T11:14:42.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.622+0000 7ff58ac00640 1 -- 172.21.15.7:0/3785602473 wait complete. 2024-04-25T11:14:42.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:14:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.947+0000 7f5cac800640 1 Processor -- start 2024-04-25T11:14:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.947+0000 7f5cac800640 1 -- start start 2024-04-25T11:14:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.949+0000 7f5cac800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5ca40b6720 0x7f5ca4164710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.949+0000 7f5cac800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ca4157320 0x7f5ca4160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.949+0000 7f5cac800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca40b1390 con 0x7f5ca406f3a0 2024-04-25T11:14:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.949+0000 7f5cac800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca4160a60 con 0x7f5ca4157320 2024-04-25T11:14:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.949+0000 7f5cac800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca40ae880 con 0x7f5ca40b6720 2024-04-25T11:14:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.950+0000 7f5caa800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5ca406f3a0 0x7f5ca406f7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53468/0 (socket says 172.21.15.7:53468) 2024-04-25T11:14:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.950+0000 7f5caa800640 1 -- 172.21.15.7:0/1258353719 learned_addr learned my addr 172.21.15.7:0/1258353719 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.950+0000 7f5ca9400640 1 --2- 172.21.15.7:0/1258353719 >> v2:172.21.15.7:3300/0 conn(0x7f5ca4157320 0x7f5ca4160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.950+0000 7f5ca9e00640 1 --2- 172.21.15.7:0/1258353719 >> v2:172.21.15.87:3300/0 conn(0x7f5ca40b6720 0x7f5ca4164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.950+0000 7f5ca9400640 1 -- 172.21.15.7:0/1258353719 >> v2:172.21.15.87:3300/0 conn(0x7f5ca40b6720 msgr2=0x7f5ca4164710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.950+0000 7f5ca9400640 1 --2- 172.21.15.7:0/1258353719 >> v2:172.21.15.87:3300/0 conn(0x7f5ca40b6720 0x7f5ca4164710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.950+0000 7f5ca9400640 1 -- 172.21.15.7:0/1258353719 >> v1:172.21.15.7:6789/0 conn(0x7f5ca406f3a0 legacy=0x7f5ca406f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.950+0000 7f5ca9400640 1 -- 172.21.15.7:0/1258353719 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca4160710 con 0x7f5ca4157320 2024-04-25T11:14:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.950+0000 7f5ca8a00640 1 -- 172.21.15.7:0/1258353719 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2482818743 0 0) 0x7f5ca40b1390 con 0x7f5ca406f3a0 2024-04-25T11:14:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.951+0000 7f5ca9400640 1 --2- 172.21.15.7:0/1258353719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca4157320 0x7f5ca4160ef0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4072830 tx=0x7f5c9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68769624611dc773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.952+0000 7f5ca8a00640 1 -- 172.21.15.7:0/1258353719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98037070 con 0x7f5ca4157320 2024-04-25T11:14:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.952+0000 7f5ca8a00640 1 -- 172.21.15.7:0/1258353719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c98032440 con 0x7f5ca4157320 2024-04-25T11:14:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.952+0000 7f5ca8a00640 1 -- 172.21.15.7:0/1258353719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98032810 con 0x7f5ca4157320 2024-04-25T11:14:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.952+0000 7f5cac800640 1 -- 172.21.15.7:0/1258353719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca4157320 msgr2=0x7f5ca4160ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.952+0000 7f5cac800640 1 --2- 172.21.15.7:0/1258353719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca4157320 0x7f5ca4160ef0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4072830 tx=0x7f5c9802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.953+0000 7f5cac800640 1 -- 172.21.15.7:0/1258353719 shutdown_connections 2024-04-25T11:14:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.953+0000 7f5cac800640 1 --2- 172.21.15.7:0/1258353719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca4157320 0x7f5ca4160ef0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.953+0000 7f5cac800640 1 --2- 172.21.15.7:0/1258353719 >> v2:172.21.15.87:3300/0 conn(0x7f5ca40b6720 0x7f5ca4164710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.953+0000 7f5cac800640 1 -- 172.21.15.7:0/1258353719 >> 172.21.15.7:0/1258353719 conn(0x7f5ca40742c0 msgr2=0x7f5ca406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.953+0000 7f5cac800640 1 -- 172.21.15.7:0/1258353719 shutdown_connections 2024-04-25T11:14:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.953+0000 7f5cac800640 1 -- 172.21.15.7:0/1258353719 wait complete. 2024-04-25T11:14:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.955+0000 7f5cac800640 1 Processor -- start 2024-04-25T11:14:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.955+0000 7f5cac800640 1 -- start start 2024-04-25T11:14:42.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.956+0000 7f5cac800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ca406f3a0 0x7f5ca414e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.956+0000 7f5cac800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ca40b6720 0x7f5ca41529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.956+0000 7f5cac800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca4157320 0x7f5ca4162810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5cac800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca4161430 con 0x7f5ca4157320 2024-04-25T11:14:42.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5cac800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca40aee00 con 0x7f5ca40b6720 2024-04-25T11:14:42.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5cac800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca41615b0 con 0x7f5ca406f3a0 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5caa800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca4157320 0x7f5ca4162810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5caa800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca4157320 0x7f5ca4162810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57120/0 (socket says 172.21.15.7:57120) 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5caa800640 1 -- 172.21.15.7:0/3788276991 learned_addr learned my addr 172.21.15.7:0/3788276991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5ca9400640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ca40b6720 0x7f5ca41529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5ca9e00640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ca406f3a0 0x7f5ca414e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5caa800640 1 -- 172.21.15.7:0/3788276991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ca406f3a0 msgr2=0x7f5ca414e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5caa800640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ca406f3a0 0x7f5ca414e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5caa800640 1 -- 172.21.15.7:0/3788276991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ca40b6720 msgr2=0x7f5ca41529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5caa800640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ca40b6720 0x7f5ca41529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.957+0000 7f5caa800640 1 -- 172.21.15.7:0/3788276991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca4153d30 con 0x7f5ca4157320 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.958+0000 7f5caa800640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca4157320 0x7f5ca4162810 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000c700 tx=0x7f5ca000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.958+0000 7f5c9e000640 1 -- 172.21.15.7:0/3788276991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0018070 con 0x7f5ca4157320 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.958+0000 7f5c9e000640 1 -- 172.21.15.7:0/3788276991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca000ce10 con 0x7f5ca4157320 2024-04-25T11:14:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.958+0000 7f5c9e000640 1 -- 172.21.15.7:0/3788276991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000a880 con 0x7f5ca4157320 2024-04-25T11:14:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.958+0000 7f5cac800640 1 -- 172.21.15.7:0/3788276991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca4154010 con 0x7f5ca4157320 2024-04-25T11:14:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.958+0000 7f5cac800640 1 -- 172.21.15.7:0/3788276991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca4152f30 con 0x7f5ca4157320 2024-04-25T11:14:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.959+0000 7f5cac800640 1 -- 172.21.15.7:0/3788276991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca4050740 con 0x7f5ca4157320 2024-04-25T11:14:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.961+0000 7f5c9e000640 1 -- 172.21.15.7:0/3788276991 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5ca001b070 con 0x7f5ca4157320 2024-04-25T11:14:42.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.962+0000 7f5c9e000640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c84074500 0x7f5c84076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:42.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.962+0000 7f5c9e000640 1 -- 172.21.15.7:0/3788276991 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f5ca0094a50 con 0x7f5ca4157320 2024-04-25T11:14:42.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.962+0000 7f5ca9e00640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c84074500 0x7f5c84076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:42.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.963+0000 7f5ca9e00640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c84074500 0x7f5c84076a10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5c90006fd0 tx=0x7f5c90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:42.966+0000 7f5c9e000640 1 -- 172.21.15.7:0/3788276991 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5ca005f030 con 0x7f5ca4157320 2024-04-25T11:14:43.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.164+0000 7f5cac800640 1 -- 172.21.15.7:0/3788276991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5ca4153ac0 con 0x7f5ca4157320 2024-04-25T11:14:43.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.165+0000 7f5c9e000640 1 -- 172.21.15.7:0/3788276991 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) ==== 100+0+11 (secure 0 0 0) 0x7f5ca00670b0 con 0x7f5ca4157320 2024-04-25T11:14:43.165 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.168+0000 7f5cac800640 1 -- 172.21.15.7:0/3788276991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c84074500 msgr2=0x7f5c84076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.168+0000 7f5cac800640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c84074500 0x7f5c84076a10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5c90006fd0 tx=0x7f5c90008040 comp rx=0 tx=0).stop 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.168+0000 7f5cac800640 1 -- 172.21.15.7:0/3788276991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca4157320 msgr2=0x7f5ca4162810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.168+0000 7f5cac800640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca4157320 0x7f5ca4162810 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000c700 tx=0x7f5ca000cbc0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.168+0000 7f5cac800640 1 -- 172.21.15.7:0/3788276991 shutdown_connections 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.168+0000 7f5cac800640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c84074500 0x7f5c84076a10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.168+0000 7f5cac800640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca4157320 0x7f5ca4162810 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.168+0000 7f5cac800640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ca40b6720 0x7f5ca41529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.168+0000 7f5cac800640 1 --2- 172.21.15.7:0/3788276991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ca406f3a0 0x7f5ca414e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.168+0000 7f5cac800640 1 -- 172.21.15.7:0/3788276991 >> 172.21.15.7:0/3788276991 conn(0x7f5ca40742c0 msgr2=0x7f5ca406cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.169+0000 7f5cac800640 1 -- 172.21.15.7:0/3788276991 shutdown_connections 2024-04-25T11:14:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.169+0000 7f5cac800640 1 -- 172.21.15.7:0/3788276991 wait complete. 2024-04-25T11:14:43.194 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:14:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.473+0000 7ff885e00640 1 Processor -- start 2024-04-25T11:14:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.473+0000 7ff885e00640 1 -- start start 2024-04-25T11:14:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff885e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff88015eed0 0x7ff8800cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff885e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff880141290 0x7ff8801416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff885e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff88006d610 0x7ff88006da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff885e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff88006c570 con 0x7ff880141290 2024-04-25T11:14:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff885e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff88006c850 con 0x7ff88006d610 2024-04-25T11:14:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff885e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff88006cb30 con 0x7ff88015eed0 2024-04-25T11:14:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff87fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff880141290 0x7ff8801416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff87fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff880141290 0x7ff8801416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57132/0 (socket says 172.21.15.7:57132) 2024-04-25T11:14:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff87fe00640 1 -- 172.21.15.7:0/3336214958 learned_addr learned my addr 172.21.15.7:0/3336214958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff884a00640 1 --2- 172.21.15.7:0/3336214958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff88015eed0 0x7ff8800cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff885400640 1 --2- 172.21.15.7:0/3336214958 >> v2:172.21.15.87:3300/0 conn(0x7ff88006d610 0x7ff88006da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.475+0000 7ff87fe00640 1 -- 172.21.15.7:0/3336214958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff88015eed0 msgr2=0x7ff8800cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.476+0000 7ff87fe00640 1 --2- 172.21.15.7:0/3336214958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff88015eed0 0x7ff8800cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.476+0000 7ff87fe00640 1 -- 172.21.15.7:0/3336214958 >> v2:172.21.15.87:3300/0 conn(0x7ff88006d610 msgr2=0x7ff88006da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.476+0000 7ff87fe00640 1 --2- 172.21.15.7:0/3336214958 >> v2:172.21.15.87:3300/0 conn(0x7ff88006d610 0x7ff88006da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.476+0000 7ff87fe00640 1 -- 172.21.15.7:0/3336214958 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88006ce10 con 0x7ff880141290 2024-04-25T11:14:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.476+0000 7ff885400640 1 --2- 172.21.15.7:0/3336214958 >> v2:172.21.15.87:3300/0 conn(0x7ff88006d610 0x7ff88006da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.476+0000 7ff884a00640 1 --2- 172.21.15.7:0/3336214958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff88015eed0 0x7ff8800cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.477+0000 7ff87fe00640 1 --2- 172.21.15.7:0/3336214958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff880141290 0x7ff8801416a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff88006eed0 tx=0x7ff86c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae91b862077c3295 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.477+0000 7ff87f400640 1 -- 172.21.15.7:0/3336214958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c037070 con 0x7ff880141290 2024-04-25T11:14:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.477+0000 7ff87f400640 1 -- 172.21.15.7:0/3336214958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86c032440 con 0x7ff880141290 2024-04-25T11:14:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.477+0000 7ff87f400640 1 -- 172.21.15.7:0/3336214958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c032810 con 0x7ff880141290 2024-04-25T11:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.478+0000 7ff885e00640 1 -- 172.21.15.7:0/3336214958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff880141290 msgr2=0x7ff8801416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.478+0000 7ff885e00640 1 --2- 172.21.15.7:0/3336214958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff880141290 0x7ff8801416a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff88006eed0 tx=0x7ff86c02a1d0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.478+0000 7ff885e00640 1 -- 172.21.15.7:0/3336214958 shutdown_connections 2024-04-25T11:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.478+0000 7ff885e00640 1 --2- 172.21.15.7:0/3336214958 >> v2:172.21.15.87:3300/0 conn(0x7ff88006d610 0x7ff88006da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.478+0000 7ff885e00640 1 --2- 172.21.15.7:0/3336214958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff880141290 0x7ff8801416a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.478+0000 7ff885e00640 1 --2- 172.21.15.7:0/3336214958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff88015eed0 0x7ff8800cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.478+0000 7ff885e00640 1 -- 172.21.15.7:0/3336214958 >> 172.21.15.7:0/3336214958 conn(0x7ff8800d1540 msgr2=0x7ff8800d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.478+0000 7ff885e00640 1 -- 172.21.15.7:0/3336214958 shutdown_connections 2024-04-25T11:14:43.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.478+0000 7ff885e00640 1 -- 172.21.15.7:0/3336214958 wait complete. 2024-04-25T11:14:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.481+0000 7ff885e00640 1 Processor -- start 2024-04-25T11:14:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.481+0000 7ff885e00640 1 -- start start 2024-04-25T11:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.482+0000 7ff885e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff88006d610 0x7ff8801e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.482+0000 7ff885e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff880141290 0x7ff8801f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.482+0000 7ff885e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88015eed0 0x7ff8801f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff885e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff880140da0 con 0x7ff88015eed0 2024-04-25T11:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff885e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff8800cdcc0 con 0x7ff880141290 2024-04-25T11:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff885e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff880140f20 con 0x7ff88006d610 2024-04-25T11:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff885400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88015eed0 0x7ff8801f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff885400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88015eed0 0x7ff8801f7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57144/0 (socket says 172.21.15.7:57144) 2024-04-25T11:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff885400640 1 -- 172.21.15.7:0/2543083173 learned_addr learned my addr 172.21.15.7:0/2543083173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff884a00640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff88006d610 0x7ff8801e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff87fe00640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff880141290 0x7ff8801f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:43.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff885400640 1 -- 172.21.15.7:0/2543083173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff88006d610 msgr2=0x7ff8801e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:43.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff885400640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff88006d610 0x7ff8801e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff885400640 1 -- 172.21.15.7:0/2543083173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff880141290 msgr2=0x7ff8801f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:43.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff885400640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff880141290 0x7ff8801f75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.483+0000 7ff885400640 1 -- 172.21.15.7:0/2543083173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff880150690 con 0x7ff88015eed0 2024-04-25T11:14:43.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.484+0000 7ff885400640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88015eed0 0x7ff8801f7af0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff87800cd80 tx=0x7ff87800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:43.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.486+0000 7ff87cc00640 1 -- 172.21.15.7:0/2543083173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87800f680 con 0x7ff88015eed0 2024-04-25T11:14:43.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.486+0000 7ff87cc00640 1 -- 172.21.15.7:0/2543083173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87800f840 con 0x7ff88015eed0 2024-04-25T11:14:43.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.486+0000 7ff87cc00640 1 -- 172.21.15.7:0/2543083173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87800fc30 con 0x7ff88015eed0 2024-04-25T11:14:43.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.486+0000 7ff885e00640 1 -- 172.21.15.7:0/2543083173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff880150970 con 0x7ff88015eed0 2024-04-25T11:14:43.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.486+0000 7ff885e00640 1 -- 172.21.15.7:0/2543083173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff880150f30 con 0x7ff88015eed0 2024-04-25T11:14:43.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.487+0000 7ff885e00640 1 -- 172.21.15.7:0/2543083173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff880050740 con 0x7ff88015eed0 2024-04-25T11:14:43.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.488+0000 7ff87cc00640 1 -- 172.21.15.7:0/2543083173 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff8780213f0 con 0x7ff88015eed0 2024-04-25T11:14:43.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.489+0000 7ff87cc00640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff860074500 0x7ff860076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:43.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.490+0000 7ff87cc00640 1 -- 172.21.15.7:0/2543083173 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7ff878096220 con 0x7ff88015eed0 2024-04-25T11:14:43.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.490+0000 7ff884a00640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff860074500 0x7ff860076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:43.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.491+0000 7ff884a00640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff860074500 0x7ff860076a10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff880071fe0 tx=0x7ff874008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:43.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.494+0000 7ff87cc00640 1 -- 172.21.15.7:0/2543083173 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff878014030 con 0x7ff88015eed0 2024-04-25T11:14:43.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.718+0000 7ff885e00640 1 -- 172.21.15.7:0/2543083173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff880158660 con 0x7ff88015eed0 2024-04-25T11:14:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.719+0000 7ff87cc00640 1 -- 172.21.15.7:0/2543083173 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7ff878060800 con 0x7ff88015eed0 2024-04-25T11:14:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.722+0000 7ff885e00640 1 -- 172.21.15.7:0/2543083173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff860074500 msgr2=0x7ff860076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.722+0000 7ff885e00640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff860074500 0x7ff860076a10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff880071fe0 tx=0x7ff874008040 comp rx=0 tx=0).stop 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.722+0000 7ff885e00640 1 -- 172.21.15.7:0/2543083173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88015eed0 msgr2=0x7ff8801f7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.722+0000 7ff885e00640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88015eed0 0x7ff8801f7af0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff87800cd80 tx=0x7ff87800a650 comp rx=0 tx=0).stop 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.722+0000 7ff885e00640 1 -- 172.21.15.7:0/2543083173 shutdown_connections 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.722+0000 7ff885e00640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff860074500 0x7ff860076a10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.722+0000 7ff885e00640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff88015eed0 0x7ff8801f7af0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.722+0000 7ff885e00640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff880141290 0x7ff8801f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.722+0000 7ff885e00640 1 --2- 172.21.15.7:0/2543083173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff88006d610 0x7ff8801e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.722+0000 7ff885e00640 1 -- 172.21.15.7:0/2543083173 >> 172.21.15.7:0/2543083173 conn(0x7ff8800d1540 msgr2=0x7ff88006f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.723+0000 7ff885e00640 1 -- 172.21.15.7:0/2543083173 shutdown_connections 2024-04-25T11:14:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:43.723+0000 7ff885e00640 1 -- 172.21.15.7:0/2543083173 wait complete. 2024-04-25T11:14:43.750 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:14:43.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:43.750 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T11:14:43.852 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:14:43.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:14:43.852 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:14:43.904 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:14:43.904 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:14:43.904 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:43.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:14:43.965 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:14:43.966 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:14:43.966 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:44.024 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:14:44.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.322+0000 7f0cd9000640 1 Processor -- start 2024-04-25T11:14:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.322+0000 7f0cd9000640 1 -- start start 2024-04-25T11:14:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.325+0000 7f0cd9000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cd40b5220 0x7f0cd415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.325+0000 7f0cd9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0cd40af420 0x7f0cd40af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.325+0000 7f0cd9000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cd40cebb0 con 0x7f0cd40aed40 2024-04-25T11:14:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.325+0000 7f0cd9000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd415e1f0 con 0x7f0cd40b5220 2024-04-25T11:14:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.325+0000 7f0cd9000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0cd40afd70 con 0x7f0cd40af420 2024-04-25T11:14:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.325+0000 7f0cd3400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0cd40aed40 0x7f0cd40ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53480/0 (socket says 172.21.15.7:53480) 2024-04-25T11:14:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.325+0000 7f0cd3400640 1 -- 172.21.15.7:0/3456895558 learned_addr learned my addr 172.21.15.7:0/3456895558 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.326+0000 7f0ccbe00640 1 --2- 172.21.15.7:0/3456895558 >> v2:172.21.15.7:3300/0 conn(0x7f0cd40b5220 0x7f0cd415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.326+0000 7f0cd3e00640 1 --2- 172.21.15.7:0/3456895558 >> v2:172.21.15.87:3300/0 conn(0x7f0cd40af420 0x7f0cd40af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.326+0000 7f0ccbe00640 1 -- 172.21.15.7:0/3456895558 >> v2:172.21.15.87:3300/0 conn(0x7f0cd40af420 msgr2=0x7f0cd40af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.326+0000 7f0ccbe00640 1 --2- 172.21.15.7:0/3456895558 >> v2:172.21.15.87:3300/0 conn(0x7f0cd40af420 0x7f0cd40af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.326+0000 7f0ccbe00640 1 -- 172.21.15.7:0/3456895558 >> v1:172.21.15.7:6789/0 conn(0x7f0cd40aed40 legacy=0x7f0cd40ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.326+0000 7f0ccbe00640 1 -- 172.21.15.7:0/3456895558 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd40b64b0 con 0x7f0cd40b5220 2024-04-25T11:14:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.326+0000 7f0cd2a00640 1 -- 172.21.15.7:0/3456895558 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1446148043 0 0) 0x7f0cc4001110 con 0x7f0cd40aed40 2024-04-25T11:14:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.327+0000 7f0ccbe00640 1 --2- 172.21.15.7:0/3456895558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd40b5220 0x7f0cd415dcb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4072ec0 tx=0x7f0cbc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed060d429c92c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:44.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.328+0000 7f0cd2a00640 1 -- 172.21.15.7:0/3456895558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc037070 con 0x7f0cd40b5220 2024-04-25T11:14:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.328+0000 7f0cd2a00640 1 -- 172.21.15.7:0/3456895558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cbc032440 con 0x7f0cd40b5220 2024-04-25T11:14:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.328+0000 7f0cd2a00640 1 -- 172.21.15.7:0/3456895558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc032810 con 0x7f0cd40b5220 2024-04-25T11:14:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.328+0000 7f0cd9000640 1 -- 172.21.15.7:0/3456895558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd40b5220 msgr2=0x7f0cd415dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.328+0000 7f0cd9000640 1 --2- 172.21.15.7:0/3456895558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd40b5220 0x7f0cd415dcb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4072ec0 tx=0x7f0cbc02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.328+0000 7f0cd9000640 1 -- 172.21.15.7:0/3456895558 shutdown_connections 2024-04-25T11:14:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.328+0000 7f0cd9000640 1 --2- 172.21.15.7:0/3456895558 >> v2:172.21.15.87:3300/0 conn(0x7f0cd40af420 0x7f0cd40af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.328+0000 7f0cd9000640 1 --2- 172.21.15.7:0/3456895558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd40b5220 0x7f0cd415dcb0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.328+0000 7f0cd9000640 1 -- 172.21.15.7:0/3456895558 >> 172.21.15.7:0/3456895558 conn(0x7f0cd40b3820 msgr2=0x7f0cd40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.329+0000 7f0cd9000640 1 -- 172.21.15.7:0/3456895558 shutdown_connections 2024-04-25T11:14:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.329+0000 7f0cd9000640 1 -- 172.21.15.7:0/3456895558 wait complete. 2024-04-25T11:14:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.331+0000 7f0cd9000640 1 Processor -- start 2024-04-25T11:14:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.331+0000 7f0cd9000640 1 -- start start 2024-04-25T11:14:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.333+0000 7f0cd9000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cd40aed40 0x7f0cd41499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.333+0000 7f0cd9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd40af420 0x7f0cd4158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.333+0000 7f0cd9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd40b5220 0x7f0cd41592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.333+0000 7f0cd9000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd406cbd0 con 0x7f0cd40af420 2024-04-25T11:14:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.333+0000 7f0cd9000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd406c8f0 con 0x7f0cd40b5220 2024-04-25T11:14:44.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.333+0000 7f0cd9000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd406ceb0 con 0x7f0cd40aed40 2024-04-25T11:14:44.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.333+0000 7f0ccbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd40af420 0x7f0cd4158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.333+0000 7f0ccbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd40af420 0x7f0cd4158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57158/0 (socket says 172.21.15.7:57158) 2024-04-25T11:14:44.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.333+0000 7f0ccbe00640 1 -- 172.21.15.7:0/3404858154 learned_addr learned my addr 172.21.15.7:0/3404858154 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:44.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.334+0000 7f0cd3e00640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd40b5220 0x7f0cd41592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.334+0000 7f0cd3400640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cd40aed40 0x7f0cd41499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.334+0000 7f0ccbe00640 1 -- 172.21.15.7:0/3404858154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cd40aed40 msgr2=0x7f0cd41499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.334+0000 7f0ccbe00640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cd40aed40 0x7f0cd41499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.334+0000 7f0ccbe00640 1 -- 172.21.15.7:0/3404858154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd40b5220 msgr2=0x7f0cd41592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.334+0000 7f0ccbe00640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd40b5220 0x7f0cd41592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.334+0000 7f0ccbe00640 1 -- 172.21.15.7:0/3404858154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd40cd640 con 0x7f0cd40af420 2024-04-25T11:14:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.334+0000 7f0cd3e00640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd40b5220 0x7f0cd41592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:14:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.334+0000 7f0ccbe00640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd40af420 0x7f0cd4158d60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc000c00 tx=0x7f0cbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.335+0000 7f0ccb400640 1 -- 172.21.15.7:0/3404858154 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc037040 con 0x7f0cd40af420 2024-04-25T11:14:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.335+0000 7f0ccb400640 1 -- 172.21.15.7:0/3404858154 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cbc042070 con 0x7f0cd40af420 2024-04-25T11:14:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.335+0000 7f0cd9000640 1 -- 172.21.15.7:0/3404858154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd40d07d0 con 0x7f0cd40af420 2024-04-25T11:14:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.335+0000 7f0ccb400640 1 -- 172.21.15.7:0/3404858154 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc030bd0 con 0x7f0cd40af420 2024-04-25T11:14:44.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.335+0000 7f0cd9000640 1 -- 172.21.15.7:0/3404858154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd40d0d90 con 0x7f0cd40af420 2024-04-25T11:14:44.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.336+0000 7f0cd9000640 1 -- 172.21.15.7:0/3404858154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd40afef0 con 0x7f0cd40af420 2024-04-25T11:14:44.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.338+0000 7f0ccb400640 1 -- 172.21.15.7:0/3404858154 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0cbc047460 con 0x7f0cd40af420 2024-04-25T11:14:44.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.339+0000 7f0ccb400640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0cb00744b0 0x7f0cb00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.339+0000 7f0ccb400640 1 -- 172.21.15.7:0/3404858154 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f0cbc0b60b0 con 0x7f0cd40af420 2024-04-25T11:14:44.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.340+0000 7f0cd3400640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0cb00744b0 0x7f0cb00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.340+0000 7f0cd3400640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0cb00744b0 0x7f0cb00769c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4000f60 tx=0x7f0cc4007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:44.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.345+0000 7f0ccb400640 1 -- 172.21.15.7:0/3404858154 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0cbc032e20 con 0x7f0cd40af420 2024-04-25T11:14:44.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.589+0000 7f0cd9000640 1 -- 172.21.15.7:0/3404858154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cd4050740 con 0x7f0cd40af420 2024-04-25T11:14:44.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.590+0000 7f0ccb400640 1 -- 172.21.15.7:0/3404858154 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7f0cbc080610 con 0x7f0cd40af420 2024-04-25T11:14:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:44.592 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2941354720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2941354720},{"type":"v1","addr":"172.21.15.87:6834","nonce":2941354720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3135576784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3135576784},{"type":"v1","addr":"172.21.15.87:6835","nonce":3135576784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2590173274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2590173274},{"type":"v1","addr":"172.21.15.7:6836","nonce":2590173274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:14:20.877428+0000","modified":"2024-04-25T11:14:23.977883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6120":{"gid":6120,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2403088100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2403088100},{"type":"v1","addr":"172.21.15.7:6837","nonce":2403088100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6120,"qdb_cluster":[6120]},"id":4}]} 2024-04-25T11:14:44.592 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-04-25T11:14:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.592+0000 7f0cd9000640 1 -- 172.21.15.7:0/3404858154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0cb00744b0 msgr2=0x7f0cb00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.592+0000 7f0cd9000640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0cb00744b0 0x7f0cb00769c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4000f60 tx=0x7f0cc4007c00 comp rx=0 tx=0).stop 2024-04-25T11:14:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.592+0000 7f0cd9000640 1 -- 172.21.15.7:0/3404858154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd40af420 msgr2=0x7f0cd4158d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.592+0000 7f0cd9000640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd40af420 0x7f0cd4158d60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc000c00 tx=0x7f0cbc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.593+0000 7f0cd9000640 1 -- 172.21.15.7:0/3404858154 shutdown_connections 2024-04-25T11:14:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.593+0000 7f0cd9000640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0cb00744b0 0x7f0cb00769c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.593+0000 7f0cd9000640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cd40b5220 0x7f0cd41592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.593+0000 7f0cd9000640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cd40af420 0x7f0cd4158d60 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.593+0000 7f0cd9000640 1 --2- 172.21.15.7:0/3404858154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cd40aed40 0x7f0cd41499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.593+0000 7f0cd9000640 1 -- 172.21.15.7:0/3404858154 >> 172.21.15.7:0/3404858154 conn(0x7f0cd40b3820 msgr2=0x7f0cd40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:44.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.593+0000 7f0cd9000640 1 -- 172.21.15.7:0/3404858154 shutdown_connections 2024-04-25T11:14:44.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.593+0000 7f0cd9000640 1 -- 172.21.15.7:0/3404858154 wait complete. 2024-04-25T11:14:44.620 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.903+0000 7feaae200640 1 Processor -- start 2024-04-25T11:14:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.903+0000 7feaae200640 1 -- start start 2024-04-25T11:14:44.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.906+0000 7feaae200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feaa81641e0 0x7feaa80b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.906+0000 7feaae200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaa80b5080 con 0x7feaa8164690 2024-04-25T11:14:44.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.906+0000 7feaae200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaa80b5360 con 0x7feaa8156cd0 2024-04-25T11:14:44.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.906+0000 7feaae200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7feaa80b5640 con 0x7feaa81641e0 2024-04-25T11:14:44.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.906+0000 7feaace00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7feaa8164690 0x7feaa8070a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53484/0 (socket says 172.21.15.7:53484) 2024-04-25T11:14:44.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.906+0000 7feaace00640 1 -- 172.21.15.7:0/2357864790 learned_addr learned my addr 172.21.15.7:0/2357864790 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.906+0000 7feaad800640 1 --2- 172.21.15.7:0/2357864790 >> v2:172.21.15.87:3300/0 conn(0x7feaa81641e0 0x7feaa80b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3670046771 0 0) 0x7feaa80b5080 con 0x7feaa8164690 2024-04-25T11:14:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea8c003520 con 0x7feaa8164690 2024-04-25T11:14:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1263895792 0 0) 0x7fea8c003520 con 0x7feaa8164690 2024-04-25T11:14:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 >> v2:172.21.15.87:3300/0 conn(0x7feaa81641e0 msgr2=0x7feaa80b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 --2- 172.21.15.7:0/2357864790 >> v2:172.21.15.87:3300/0 conn(0x7feaa81641e0 0x7feaa80b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 >> v1:172.21.15.87:6789/0 conn(0x7feaa8156cd0 legacy=0x7feaa8163ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa80b5920 con 0x7feaa8164690 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fea98001670 con 0x7feaa8164690 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 >> v1:172.21.15.7:6789/0 conn(0x7feaa8164690 legacy=0x7feaa8070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 --2- 172.21.15.7:0/2357864790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea8c0039b0 0x7fea8c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 --2- 172.21.15.7:0/2357864790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 0x7fea8c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 --2- 172.21.15.7:0/2357864790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea8c005300 0x7fea8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feaa80b5640 con 0x7fea8c004800 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c00c2a0 con 0x7fea8c0039b0 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fea8c00c4a0 con 0x7fea8c005300 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.907+0000 7feaa7e00640 1 --2- 172.21.15.7:0/2357864790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 0x7fea8c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.908+0000 7feaace00640 1 --2- 172.21.15.7:0/2357864790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea8c0039b0 0x7fea8c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.908+0000 7feaa7e00640 1 -- 172.21.15.7:0/2357864790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea8c005300 msgr2=0x7fea8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.908+0000 7feaa7e00640 1 --2- 172.21.15.7:0/2357864790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea8c005300 0x7fea8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.908+0000 7feaa7e00640 1 -- 172.21.15.7:0/2357864790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea8c0039b0 msgr2=0x7fea8c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.908+0000 7feaa7e00640 1 --2- 172.21.15.7:0/2357864790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea8c0039b0 0x7fea8c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.908+0000 7feaa7e00640 1 -- 172.21.15.7:0/2357864790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c003520 con 0x7fea8c004800 2024-04-25T11:14:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.908+0000 7feaad800640 1 -- 172.21.15.7:0/2357864790 reap_dead start 2024-04-25T11:14:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.909+0000 7feaa7e00640 1 --2- 172.21.15.7:0/2357864790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 0x7fea8c004c10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fea94001750 tx=0x7fea8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587f4dcbb99d4258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.909+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94001370 con 0x7fea8c004800 2024-04-25T11:14:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.909+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea940052d0 con 0x7fea8c004800 2024-04-25T11:14:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.909+0000 7feaa7400640 1 -- 172.21.15.7:0/2357864790 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9402abc0 con 0x7fea8c004800 2024-04-25T11:14:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.910+0000 7feaae200640 1 -- 172.21.15.7:0/2357864790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 msgr2=0x7fea8c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.910+0000 7feaae200640 1 --2- 172.21.15.7:0/2357864790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 0x7fea8c004c10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fea94001750 tx=0x7fea8c005830 comp rx=0 tx=0).stop 2024-04-25T11:14:44.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.910+0000 7feaae200640 1 -- 172.21.15.7:0/2357864790 shutdown_connections 2024-04-25T11:14:44.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.910+0000 7feaae200640 1 --2- 172.21.15.7:0/2357864790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 0x7fea8c004c10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.910+0000 7feaae200640 1 -- 172.21.15.7:0/2357864790 >> 172.21.15.7:0/2357864790 conn(0x7feaa80b3820 msgr2=0x7feaa80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:44.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.910+0000 7feaae200640 1 -- 172.21.15.7:0/2357864790 shutdown_connections 2024-04-25T11:14:44.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.910+0000 7feaae200640 1 -- 172.21.15.7:0/2357864790 wait complete. 2024-04-25T11:14:44.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.913+0000 7feaae200640 1 Processor -- start 2024-04-25T11:14:44.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.913+0000 7feaae200640 1 -- start start 2024-04-25T11:14:44.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaae200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 0x7feaa814fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaae200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feaa814f3d0 0x7feaa8141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaae200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feaa80b0250 0x7feaa80b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaae200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feaa8157260 con 0x7fea8c004800 2024-04-25T11:14:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaae200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feaa81570e0 con 0x7feaa814f3d0 2024-04-25T11:14:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaae200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feaa8142ad0 con 0x7feaa80b0250 2024-04-25T11:14:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaace00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 0x7feaa814fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaace00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 0x7feaa814fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57174/0 (socket says 172.21.15.7:57174) 2024-04-25T11:14:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaace00640 1 -- 172.21.15.7:0/3782626697 learned_addr learned my addr 172.21.15.7:0/3782626697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaa7e00640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feaa814f3d0 0x7feaa8141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaace00640 1 -- 172.21.15.7:0/3782626697 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feaa80b0250 msgr2=0x7feaa80b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaace00640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feaa80b0250 0x7feaa80b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaace00640 1 -- 172.21.15.7:0/3782626697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feaa814f3d0 msgr2=0x7feaa8141ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaace00640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feaa814f3d0 0x7feaa8141ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:44.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.915+0000 7feaace00640 1 -- 172.21.15.7:0/3782626697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa80b0ba0 con 0x7fea8c004800 2024-04-25T11:14:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.916+0000 7feaace00640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 0x7feaa814fca0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fea9800e760 tx=0x7fea9800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.916+0000 7feaa4c00640 1 -- 172.21.15.7:0/3782626697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9800ee70 con 0x7fea8c004800 2024-04-25T11:14:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.916+0000 7feaa4c00640 1 -- 172.21.15.7:0/3782626697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea98018070 con 0x7fea8c004800 2024-04-25T11:14:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.916+0000 7feaa4c00640 1 -- 172.21.15.7:0/3782626697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9800ca70 con 0x7fea8c004800 2024-04-25T11:14:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.916+0000 7feaae200640 1 -- 172.21.15.7:0/3782626697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa8142880 con 0x7fea8c004800 2024-04-25T11:14:44.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.916+0000 7feaae200640 1 -- 172.21.15.7:0/3782626697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa8152e10 con 0x7fea8c004800 2024-04-25T11:14:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.917+0000 7feaae200640 1 -- 172.21.15.7:0/3782626697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa8050740 con 0x7fea8c004800 2024-04-25T11:14:44.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.919+0000 7feaa4c00640 1 -- 172.21.15.7:0/3782626697 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fea9800cc10 con 0x7fea8c004800 2024-04-25T11:14:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.920+0000 7feaa4c00640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea880744e0 0x7fea880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.920+0000 7feaa4c00640 1 -- 172.21.15.7:0/3782626697 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7fea98095210 con 0x7fea8c004800 2024-04-25T11:14:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.920+0000 7feaa7e00640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea880744e0 0x7fea880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:44.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.921+0000 7feaa7e00640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea880744e0 0x7fea880769f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fea94032000 tx=0x7fea9402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:44.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:44.925+0000 7feaa4c00640 1 -- 172.21.15.7:0/3782626697 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feaa80b0ba0 con 0x7fea8c004800 2024-04-25T11:14:45.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.129+0000 7feaae200640 1 -- 172.21.15.7:0/3782626697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feaa8153fd0 con 0x7fea8c004800 2024-04-25T11:14:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.132+0000 7feaa4c00640 1 -- 172.21.15.7:0/3782626697 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16788 (secure 0 0 0) 0x7fea9805f800 con 0x7fea8c004800 2024-04-25T11:14:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:45.134 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:22.341351+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:14:19.115741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T11:14:19.849856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.135+0000 7feaae200640 1 -- 172.21.15.7:0/3782626697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea880744e0 msgr2=0x7fea880769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.135+0000 7feaae200640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea880744e0 0x7fea880769f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fea94032000 tx=0x7fea9402e000 comp rx=0 tx=0).stop 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.135+0000 7feaae200640 1 -- 172.21.15.7:0/3782626697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 msgr2=0x7feaa814fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.135+0000 7feaae200640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 0x7feaa814fca0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fea9800e760 tx=0x7fea9800ec20 comp rx=0 tx=0).stop 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.135+0000 7feaae200640 1 -- 172.21.15.7:0/3782626697 shutdown_connections 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.135+0000 7feaae200640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea880744e0 0x7fea880769f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.135+0000 7feaae200640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feaa80b0250 0x7feaa80b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.135+0000 7feaae200640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feaa814f3d0 0x7feaa8141ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.135+0000 7feaae200640 1 --2- 172.21.15.7:0/3782626697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea8c004800 0x7feaa814fca0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.135+0000 7feaae200640 1 -- 172.21.15.7:0/3782626697 >> 172.21.15.7:0/3782626697 conn(0x7feaa80b3820 msgr2=0x7feaa80af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.136+0000 7feaae200640 1 -- 172.21.15.7:0/3782626697 shutdown_connections 2024-04-25T11:14:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.136+0000 7feaae200640 1 -- 172.21.15.7:0/3782626697 wait complete. 2024-04-25T11:14:45.167 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:14:19.115741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:14:19.849856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:14:45.168 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:14:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.446+0000 7f57f8a00640 1 Processor -- start 2024-04-25T11:14:45.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.447+0000 7f57f8a00640 1 -- start start 2024-04-25T11:14:45.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.448+0000 7f57f8a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f4145e40 0x7f57f4146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.448+0000 7f57f8a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57f4144a80 0x7f57f40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.448+0000 7f57f8a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f40b66b0 con 0x7f57f40b4870 2024-04-25T11:14:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.448+0000 7f57f8a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f57f4145b70 con 0x7f57f4144a80 2024-04-25T11:14:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.449+0000 7f57f8a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57f40b58d0 con 0x7f57f4145e40 2024-04-25T11:14:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.449+0000 7f57f2800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f4145e40 0x7f57f4146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.449+0000 7f57f3200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57f4144a80 0x7f57f40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.449+0000 7f57f1e00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f57f40b4870 0x7f57f40b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:53640/0 (socket says 172.21.15.7:53640) 2024-04-25T11:14:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.449+0000 7f57f1e00640 1 -- 172.21.15.7:0/2273965046 learned_addr learned my addr 172.21.15.7:0/2273965046 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.450+0000 7f57f1400640 1 -- 172.21.15.7:0/2273965046 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1477596203 0 0) 0x7f57f40b66b0 con 0x7f57f40b4870 2024-04-25T11:14:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.450+0000 7f57f1400640 1 -- 172.21.15.7:0/2273965046 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57d8003520 con 0x7f57f40b4870 2024-04-25T11:14:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.450+0000 7f57f2800640 1 -- 172.21.15.7:0/2273965046 >> v2:172.21.15.87:3300/0 conn(0x7f57f4144a80 msgr2=0x7f57f40b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.450+0000 7f57f2800640 1 --2- 172.21.15.7:0/2273965046 >> v2:172.21.15.87:3300/0 conn(0x7f57f4144a80 0x7f57f40b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.450+0000 7f57f2800640 1 -- 172.21.15.7:0/2273965046 >> v1:172.21.15.87:6789/0 conn(0x7f57f40b4870 legacy=0x7f57f40b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.450+0000 7f57f2800640 1 -- 172.21.15.7:0/2273965046 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f40b5b30 con 0x7f57f4145e40 2024-04-25T11:14:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.452+0000 7f57f2800640 1 --2- 172.21.15.7:0/2273965046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f4145e40 0x7f57f4146250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f57e40097a0 tx=0x7f57e402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c81ba76ffed2711 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.452+0000 7f57f1400640 1 -- 172.21.15.7:0/2273965046 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4037070 con 0x7f57f4145e40 2024-04-25T11:14:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.452+0000 7f57f1400640 1 -- 172.21.15.7:0/2273965046 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e402ae50 con 0x7f57f4145e40 2024-04-25T11:14:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.452+0000 7f57f1400640 1 -- 172.21.15.7:0/2273965046 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4032660 con 0x7f57f4145e40 2024-04-25T11:14:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.453+0000 7f57f8a00640 1 -- 172.21.15.7:0/2273965046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f4145e40 msgr2=0x7f57f4146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.453+0000 7f57f8a00640 1 --2- 172.21.15.7:0/2273965046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f4145e40 0x7f57f4146250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f57e40097a0 tx=0x7f57e402a100 comp rx=0 tx=0).stop 2024-04-25T11:14:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.453+0000 7f57f8a00640 1 -- 172.21.15.7:0/2273965046 shutdown_connections 2024-04-25T11:14:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.453+0000 7f57f8a00640 1 --2- 172.21.15.7:0/2273965046 >> v2:172.21.15.87:3300/0 conn(0x7f57f4144a80 0x7f57f40b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.453+0000 7f57f8a00640 1 --2- 172.21.15.7:0/2273965046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f4145e40 0x7f57f4146250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.453+0000 7f57f8a00640 1 -- 172.21.15.7:0/2273965046 >> 172.21.15.7:0/2273965046 conn(0x7f57f40740f0 msgr2=0x7f57f406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.453+0000 7f57f8a00640 1 -- 172.21.15.7:0/2273965046 shutdown_connections 2024-04-25T11:14:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.453+0000 7f57f8a00640 1 -- 172.21.15.7:0/2273965046 wait complete. 2024-04-25T11:14:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.455+0000 7f57f8a00640 1 Processor -- start 2024-04-25T11:14:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.456+0000 7f57f8a00640 1 -- start start 2024-04-25T11:14:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.457+0000 7f57f8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f40b4870 0x7f57f4141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.457+0000 7f57f8a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f4144a80 0x7f57f415e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.457+0000 7f57f8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57f4145e40 0x7f57f413aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.457+0000 7f57f8a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57f40b63d0 con 0x7f57f4145e40 2024-04-25T11:14:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.457+0000 7f57f8a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57f40b60f0 con 0x7f57f40b4870 2024-04-25T11:14:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.457+0000 7f57f8a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57f406f730 con 0x7f57f4144a80 2024-04-25T11:14:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.457+0000 7f57f3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57f4145e40 0x7f57f413aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.457+0000 7f57f3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57f4145e40 0x7f57f413aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57182/0 (socket says 172.21.15.7:57182) 2024-04-25T11:14:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.457+0000 7f57f3200640 1 -- 172.21.15.7:0/2096497217 learned_addr learned my addr 172.21.15.7:0/2096497217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.458+0000 7f57f2800640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f40b4870 0x7f57f4141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.458+0000 7f57f1e00640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f4144a80 0x7f57f415e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.458+0000 7f57f3200640 1 -- 172.21.15.7:0/2096497217 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f4144a80 msgr2=0x7f57f415e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.458+0000 7f57f3200640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f4144a80 0x7f57f415e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.458+0000 7f57f3200640 1 -- 172.21.15.7:0/2096497217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f40b4870 msgr2=0x7f57f4141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.458+0000 7f57f3200640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f40b4870 0x7f57f4141090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.458+0000 7f57f3200640 1 -- 172.21.15.7:0/2096497217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f415f120 con 0x7f57f4145e40 2024-04-25T11:14:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.458+0000 7f57f2800640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f40b4870 0x7f57f4141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:14:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.458+0000 7f57f3200640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57f4145e40 0x7f57f413aff0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00efc0 tx=0x7f57ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.459+0000 7f57eaa00640 1 -- 172.21.15.7:0/2096497217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec018020 con 0x7f57f4145e40 2024-04-25T11:14:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.459+0000 7f57eaa00640 1 -- 172.21.15.7:0/2096497217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec014480 con 0x7f57f4145e40 2024-04-25T11:14:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.459+0000 7f57eaa00640 1 -- 172.21.15.7:0/2096497217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec009e60 con 0x7f57f4145e40 2024-04-25T11:14:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.459+0000 7f57f8a00640 1 -- 172.21.15.7:0/2096497217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f415f400 con 0x7f57f4145e40 2024-04-25T11:14:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.459+0000 7f57f8a00640 1 -- 172.21.15.7:0/2096497217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f415f9c0 con 0x7f57f4145e40 2024-04-25T11:14:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.460+0000 7f57f8a00640 1 -- 172.21.15.7:0/2096497217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f4050740 con 0x7f57f4145e40 2024-04-25T11:14:45.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.461+0000 7f57eaa00640 1 -- 172.21.15.7:0/2096497217 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f57ec005d10 con 0x7f57f4145e40 2024-04-25T11:14:45.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.462+0000 7f57eaa00640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57d40744b0 0x7f57d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.462+0000 7f57eaa00640 1 -- 172.21.15.7:0/2096497217 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f57ec0961b0 con 0x7f57f4145e40 2024-04-25T11:14:45.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.462+0000 7f57f2800640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57d40744b0 0x7f57d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.463+0000 7f57f2800640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57d40744b0 0x7f57d40769c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f57e4034000 tx=0x7f57e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.467+0000 7f57eaa00640 1 -- 172.21.15.7:0/2096497217 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57ec022030 con 0x7f57f4145e40 2024-04-25T11:14:45.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.660+0000 7f57f8a00640 1 -- 172.21.15.7:0/2096497217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57f415ee70 con 0x7f57f4145e40 2024-04-25T11:14:45.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.661+0000 7f57eaa00640 1 -- 172.21.15.7:0/2096497217 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7f57ec010040 con 0x7f57f4145e40 2024-04-25T11:14:45.662 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.664+0000 7f57f8a00640 1 -- 172.21.15.7:0/2096497217 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57d40744b0 msgr2=0x7f57d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.664+0000 7f57f8a00640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57d40744b0 0x7f57d40769c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f57e4034000 tx=0x7f57e402e000 comp rx=0 tx=0).stop 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.664+0000 7f57f8a00640 1 -- 172.21.15.7:0/2096497217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57f4145e40 msgr2=0x7f57f413aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.664+0000 7f57f8a00640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57f4145e40 0x7f57f413aff0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00efc0 tx=0x7f57ec00c490 comp rx=0 tx=0).stop 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.664+0000 7f57f8a00640 1 -- 172.21.15.7:0/2096497217 shutdown_connections 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.664+0000 7f57f8a00640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57d40744b0 0x7f57d40769c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.664+0000 7f57f8a00640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57f4145e40 0x7f57f413aff0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.664+0000 7f57f8a00640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f4144a80 0x7f57f415e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.664+0000 7f57f8a00640 1 --2- 172.21.15.7:0/2096497217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f40b4870 0x7f57f4141090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.664+0000 7f57f8a00640 1 -- 172.21.15.7:0/2096497217 >> 172.21.15.7:0/2096497217 conn(0x7f57f40740f0 msgr2=0x7f57f4161710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.664+0000 7f57f8a00640 1 -- 172.21.15.7:0/2096497217 shutdown_connections 2024-04-25T11:14:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.665+0000 7f57f8a00640 1 -- 172.21.15.7:0/2096497217 wait complete. 2024-04-25T11:14:45.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:14:45.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.978+0000 7f152a600640 1 Processor -- start 2024-04-25T11:14:45.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.979+0000 7f152a600640 1 -- start start 2024-04-25T11:14:45.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.981+0000 7f152a600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15240b6720 0x7f1524164710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.981+0000 7f152a600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15240b1390 con 0x7f1524157320 2024-04-25T11:14:45.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.981+0000 7f152a600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15240ae880 con 0x7f152406f3a0 2024-04-25T11:14:45.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.981+0000 7f152a600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1524160a60 con 0x7f15240b6720 2024-04-25T11:14:45.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.981+0000 7f151be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1524157320 0x7f1524160ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53498/0 (socket says 172.21.15.7:53498) 2024-04-25T11:14:45.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.981+0000 7f151be00640 1 -- 172.21.15.7:0/3694659746 learned_addr learned my addr 172.21.15.7:0/3694659746 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:45.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1523400640 1 --2- 172.21.15.7:0/3694659746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15240b6720 0x7f1524164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2534775488 0 0) 0x7f15240b1390 con 0x7f1524157320 2024-04-25T11:14:45.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1508003520 con 0x7f1524157320 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1471308363 0 0) 0x7f1508003520 con 0x7f1524157320 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15240b6720 msgr2=0x7f1524164710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1522a00640 1 --2- 172.21.15.7:0/3694659746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15240b6720 0x7f1524164710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 >> v1:172.21.15.87:6789/0 conn(0x7f152406f3a0 legacy=0x7f152406f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1524160710 con 0x7f1524157320 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f150c001670 con 0x7f1524157320 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 >> v1:172.21.15.7:6789/0 conn(0x7f1524157320 legacy=0x7f1524160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1522a00640 1 --2- 172.21.15.7:0/3694659746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15080039e0 0x7f1508003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.982+0000 7f1522a00640 1 --2- 172.21.15.7:0/3694659746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1508004800 0x7f1508004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.983+0000 7f1522a00640 1 --2- 172.21.15.7:0/3694659746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15240b6720 0x7f15080052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.983+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1524160a60 con 0x7f15080039e0 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.983+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f150800bb50 con 0x7f1508004800 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.983+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f150800bd50 con 0x7f15240b6720 2024-04-25T11:14:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.983+0000 7f1523400640 1 --2- 172.21.15.7:0/3694659746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15080039e0 0x7f1508003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.983+0000 7f151be00640 1 --2- 172.21.15.7:0/3694659746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1508004800 0x7f1508004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.983+0000 7f1523400640 1 -- 172.21.15.7:0/3694659746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15240b6720 msgr2=0x7f15080052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:45.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.983+0000 7f1523400640 1 --2- 172.21.15.7:0/3694659746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15240b6720 0x7f15080052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.983+0000 7f1523400640 1 -- 172.21.15.7:0/3694659746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1508004800 msgr2=0x7f1508004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:45.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.983+0000 7f1523400640 1 --2- 172.21.15.7:0/3694659746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1508004800 0x7f1508004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.983+0000 7f1523400640 1 -- 172.21.15.7:0/3694659746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1508003520 con 0x7f15080039e0 2024-04-25T11:14:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.984+0000 7f1523400640 1 --2- 172.21.15.7:0/3694659746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15080039e0 0x7f1508003e70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1514009a90 tx=0x7f1508005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e3b2d1a2554b60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.985+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514036070 con 0x7f15080039e0 2024-04-25T11:14:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.985+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1514031400 con 0x7f15080039e0 2024-04-25T11:14:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.985+0000 7f1522a00640 1 -- 172.21.15.7:0/3694659746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514009690 con 0x7f15080039e0 2024-04-25T11:14:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.985+0000 7f152a600640 1 -- 172.21.15.7:0/3694659746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15080039e0 msgr2=0x7f1508003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.985+0000 7f152a600640 1 --2- 172.21.15.7:0/3694659746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15080039e0 0x7f1508003e70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1514009a90 tx=0x7f1508005830 comp rx=0 tx=0).stop 2024-04-25T11:14:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.985+0000 7f1523e00640 1 -- 172.21.15.7:0/3694659746 reap_dead start 2024-04-25T11:14:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.986+0000 7f152a600640 1 -- 172.21.15.7:0/3694659746 shutdown_connections 2024-04-25T11:14:45.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.986+0000 7f152a600640 1 -- 172.21.15.7:0/3694659746 >> 172.21.15.7:0/3694659746 conn(0x7f15240742c0 msgr2=0x7f152406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:45.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.986+0000 7f152a600640 1 -- 172.21.15.7:0/3694659746 shutdown_connections 2024-04-25T11:14:45.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.986+0000 7f152a600640 1 -- 172.21.15.7:0/3694659746 wait complete. 2024-04-25T11:14:45.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.988+0000 7f152a600640 1 Processor -- start 2024-04-25T11:14:45.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.988+0000 7f152a600640 1 -- start start 2024-04-25T11:14:45.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.990+0000 7f152a600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15080039e0 0x7f15241f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.990+0000 7f152a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152415ef70 0x7f152415f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.990+0000 7f152a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15240d01f0 0x7f15240d0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.990+0000 7f152a600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15240aee00 con 0x7f152415ef70 2024-04-25T11:14:45.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.990+0000 7f152a600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f15240aeb20 con 0x7f15240d01f0 2024-04-25T11:14:45.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.990+0000 7f152a600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1524164710 con 0x7f15080039e0 2024-04-25T11:14:45.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.990+0000 7f151be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152415ef70 0x7f152415f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.990+0000 7f151be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152415ef70 0x7f152415f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57200/0 (socket says 172.21.15.7:57200) 2024-04-25T11:14:45.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.990+0000 7f151be00640 1 -- 172.21.15.7:0/697923961 learned_addr learned my addr 172.21.15.7:0/697923961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.990+0000 7f1523400640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15080039e0 0x7f15241f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.990+0000 7f1523e00640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15240d01f0 0x7f15240d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.991+0000 7f151be00640 1 -- 172.21.15.7:0/697923961 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15080039e0 msgr2=0x7f15241f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.991+0000 7f151be00640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15080039e0 0x7f15241f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.991+0000 7f151be00640 1 -- 172.21.15.7:0/697923961 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15240d01f0 msgr2=0x7f15240d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.991+0000 7f151be00640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15240d01f0 0x7f15240d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.991+0000 7f151be00640 1 -- 172.21.15.7:0/697923961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15240d0b40 con 0x7f152415ef70 2024-04-25T11:14:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.991+0000 7f151be00640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152415ef70 0x7f152415f380 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f150c001eb0 tx=0x7f150c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.991+0000 7f151b400640 1 -- 172.21.15.7:0/697923961 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c019070 con 0x7f152415ef70 2024-04-25T11:14:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.991+0000 7f151b400640 1 -- 172.21.15.7:0/697923961 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150c00de50 con 0x7f152415ef70 2024-04-25T11:14:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.991+0000 7f151b400640 1 -- 172.21.15.7:0/697923961 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c0047e0 con 0x7f152415ef70 2024-04-25T11:14:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.991+0000 7f152a600640 1 -- 172.21.15.7:0/697923961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15240cdca0 con 0x7f152415ef70 2024-04-25T11:14:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.992+0000 7f152a600640 1 -- 172.21.15.7:0/697923961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15240ce260 con 0x7f152415ef70 2024-04-25T11:14:45.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.993+0000 7f152a600640 1 -- 172.21.15.7:0/697923961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1524050740 con 0x7f152415ef70 2024-04-25T11:14:45.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.994+0000 7f151b400640 1 -- 172.21.15.7:0/697923961 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f150c01c070 con 0x7f152415ef70 2024-04-25T11:14:45.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.995+0000 7f151b400640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1500074530 0x7f1500076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:45.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.995+0000 7f151b400640 1 -- 172.21.15.7:0/697923961 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f150c094cd0 con 0x7f152415ef70 2024-04-25T11:14:45.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.995+0000 7f1523400640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1500074530 0x7f1500076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:45.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:45.996+0000 7f1523400640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1500074530 0x7f1500076a40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1514033000 tx=0x7f1514008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:46.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.000+0000 7f151b400640 1 -- 172.21.15.7:0/697923961 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15240d0b40 con 0x7f152415ef70 2024-04-25T11:14:46.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.205+0000 7f152a600640 1 -- 172.21.15.7:0/697923961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f152406fb80 con 0x7f152415ef70 2024-04-25T11:14:46.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.207+0000 7f151b400640 1 -- 172.21.15.7:0/697923961 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) ==== 104+0+11 (secure 0 0 0) 0x7f150c05ecf0 con 0x7f152415ef70 2024-04-25T11:14:46.208 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 -- 172.21.15.7:0/697923961 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1500074530 msgr2=0x7f1500076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1500074530 0x7f1500076a40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1514033000 tx=0x7f1514008030 comp rx=0 tx=0).stop 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 -- 172.21.15.7:0/697923961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152415ef70 msgr2=0x7f152415f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152415ef70 0x7f152415f380 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f150c001eb0 tx=0x7f150c00dc00 comp rx=0 tx=0).stop 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 -- 172.21.15.7:0/697923961 shutdown_connections 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1500074530 0x7f1500076a40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15240d01f0 0x7f15240d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f152415ef70 0x7f152415f380 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 --2- 172.21.15.7:0/697923961 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15080039e0 0x7f15241f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 -- 172.21.15.7:0/697923961 >> 172.21.15.7:0/697923961 conn(0x7f15240742c0 msgr2=0x7f1524163c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 -- 172.21.15.7:0/697923961 shutdown_connections 2024-04-25T11:14:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.209+0000 7f152a600640 1 -- 172.21.15.7:0/697923961 wait complete. 2024-04-25T11:14:46.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:14:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.521+0000 7fa58e600640 1 Processor -- start 2024-04-25T11:14:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.521+0000 7fa58e600640 1 -- start start 2024-04-25T11:14:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.523+0000 7fa58e600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5880d3250 0x7fa5880cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.523+0000 7fa58e600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5880cdb70 con 0x7fa58806ed00 2024-04-25T11:14:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.523+0000 7fa58e600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa588161990 con 0x7fa5881604c0 2024-04-25T11:14:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.523+0000 7fa58e600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa5880d2dc0 con 0x7fa5880d3250 2024-04-25T11:14:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.523+0000 7fa587400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa58806ed00 0x7fa5880ccf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53512/0 (socket says 172.21.15.7:53512) 2024-04-25T11:14:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.523+0000 7fa587400640 1 -- 172.21.15.7:0/1620116459 learned_addr learned my addr 172.21.15.7:0/1620116459 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.524+0000 7fa586a00640 1 --2- 172.21.15.7:0/1620116459 >> v2:172.21.15.7:3300/0 conn(0x7fa5880d3250 0x7fa5880cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.524+0000 7fa586000640 1 -- 172.21.15.7:0/1620116459 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3118672181 0 0) 0x7fa5880cdb70 con 0x7fa58806ed00 2024-04-25T11:14:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.524+0000 7fa586000640 1 -- 172.21.15.7:0/1620116459 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa56c003520 con 0x7fa58806ed00 2024-04-25T11:14:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.524+0000 7fa586a00640 1 -- 172.21.15.7:0/1620116459 >> v1:172.21.15.87:6789/0 conn(0x7fa5881604c0 legacy=0x7fa5881608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.524+0000 7fa586a00640 1 -- 172.21.15.7:0/1620116459 >> v1:172.21.15.7:6789/0 conn(0x7fa58806ed00 legacy=0x7fa5880ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.524+0000 7fa586a00640 1 -- 172.21.15.7:0/1620116459 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5880d24b0 con 0x7fa5880d3250 2024-04-25T11:14:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.525+0000 7fa586a00640 1 --2- 172.21.15.7:0/1620116459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5880d3250 0x7fa5880cd630 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa5740097a0 tx=0x7fa57402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92d22ba1fd54b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:46.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.526+0000 7fa586000640 1 -- 172.21.15.7:0/1620116459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574037070 con 0x7fa5880d3250 2024-04-25T11:14:46.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.526+0000 7fa586000640 1 -- 172.21.15.7:0/1620116459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa574032440 con 0x7fa5880d3250 2024-04-25T11:14:46.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.526+0000 7fa586000640 1 -- 172.21.15.7:0/1620116459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574032810 con 0x7fa5880d3250 2024-04-25T11:14:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.526+0000 7fa58e600640 1 -- 172.21.15.7:0/1620116459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5880d3250 msgr2=0x7fa5880cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.526+0000 7fa58e600640 1 --2- 172.21.15.7:0/1620116459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5880d3250 0x7fa5880cd630 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa5740097a0 tx=0x7fa57402a230 comp rx=0 tx=0).stop 2024-04-25T11:14:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.527+0000 7fa58e600640 1 -- 172.21.15.7:0/1620116459 shutdown_connections 2024-04-25T11:14:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.527+0000 7fa58e600640 1 --2- 172.21.15.7:0/1620116459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5880d3250 0x7fa5880cd630 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.527+0000 7fa58e600640 1 -- 172.21.15.7:0/1620116459 >> 172.21.15.7:0/1620116459 conn(0x7fa5880b7640 msgr2=0x7fa5880b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.527+0000 7fa58e600640 1 -- 172.21.15.7:0/1620116459 shutdown_connections 2024-04-25T11:14:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.527+0000 7fa58e600640 1 -- 172.21.15.7:0/1620116459 wait complete. 2024-04-25T11:14:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.529+0000 7fa58e600640 1 Processor -- start 2024-04-25T11:14:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.529+0000 7fa58e600640 1 -- start start 2024-04-25T11:14:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa58e600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa58806ed00 0x7fa588158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa58e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5880d3250 0x7fa588154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa58e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5881604c0 0x7fa58809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa58e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5880d3660 con 0x7fa5880d3250 2024-04-25T11:14:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa58e600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa5880d2a50 con 0x7fa5881604c0 2024-04-25T11:14:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa58e600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa5880cf110 con 0x7fa58806ed00 2024-04-25T11:14:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa586a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5880d3250 0x7fa588154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa586a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5880d3250 0x7fa588154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57210/0 (socket says 172.21.15.7:57210) 2024-04-25T11:14:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa586a00640 1 -- 172.21.15.7:0/1041445164 learned_addr learned my addr 172.21.15.7:0/1041445164 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa586a00640 1 -- 172.21.15.7:0/1041445164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa58806ed00 msgr2=0x7fa588158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa586a00640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa58806ed00 0x7fa588158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa586a00640 1 -- 172.21.15.7:0/1041445164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5881604c0 msgr2=0x7fa58809d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.531+0000 7fa586a00640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5881604c0 0x7fa58809d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.532+0000 7fa586a00640 1 -- 172.21.15.7:0/1041445164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5881572f0 con 0x7fa5880d3250 2024-04-25T11:14:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.532+0000 7fa586a00640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5880d3250 0x7fa588154d90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa5740098c0 tx=0x7fa5740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.532+0000 7fa57f400640 1 -- 172.21.15.7:0/1041445164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574037040 con 0x7fa5880d3250 2024-04-25T11:14:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.532+0000 7fa57f400640 1 -- 172.21.15.7:0/1041445164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa574030500 con 0x7fa5880d3250 2024-04-25T11:14:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.532+0000 7fa58e600640 1 -- 172.21.15.7:0/1041445164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5881575d0 con 0x7fa5880d3250 2024-04-25T11:14:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.532+0000 7fa57f400640 1 -- 172.21.15.7:0/1041445164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740308d0 con 0x7fa5880d3250 2024-04-25T11:14:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.532+0000 7fa58e600640 1 -- 172.21.15.7:0/1041445164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa588157b90 con 0x7fa5880d3250 2024-04-25T11:14:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.533+0000 7fa58e600640 1 -- 172.21.15.7:0/1041445164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa588157060 con 0x7fa5880d3250 2024-04-25T11:14:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.535+0000 7fa57f400640 1 -- 172.21.15.7:0/1041445164 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa574044020 con 0x7fa5880d3250 2024-04-25T11:14:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.536+0000 7fa57f400640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5680744b0 0x7fa5680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.536+0000 7fa57f400640 1 -- 172.21.15.7:0/1041445164 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7fa5740b57a0 con 0x7fa5880d3250 2024-04-25T11:14:46.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.536+0000 7fa587400640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5680744b0 0x7fa5680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.537+0000 7fa587400640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5680744b0 0x7fa5680769c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa578001570 tx=0x7fa578007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.541+0000 7fa57f400640 1 -- 172.21.15.7:0/1041445164 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa57407fd00 con 0x7fa5880d3250 2024-04-25T11:14:46.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.740+0000 7fa58e600640 1 -- 172.21.15.7:0/1041445164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa588050740 con 0x7fa5880d3250 2024-04-25T11:14:46.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.741+0000 7fa57f400640 1 -- 172.21.15.7:0/1041445164 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) ==== 100+0+11 (secure 0 0 0) 0x7fa57407f600 con 0x7fa5880d3250 2024-04-25T11:14:46.742 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:14:46.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.743+0000 7fa58e600640 1 -- 172.21.15.7:0/1041445164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5680744b0 msgr2=0x7fa5680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:46.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.743+0000 7fa58e600640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5680744b0 0x7fa5680769c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa578001570 tx=0x7fa578007930 comp rx=0 tx=0).stop 2024-04-25T11:14:46.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.743+0000 7fa58e600640 1 -- 172.21.15.7:0/1041445164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5880d3250 msgr2=0x7fa588154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:46.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.743+0000 7fa58e600640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5880d3250 0x7fa588154d90 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa5740098c0 tx=0x7fa5740046c0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.744+0000 7fa58e600640 1 -- 172.21.15.7:0/1041445164 shutdown_connections 2024-04-25T11:14:46.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.744+0000 7fa58e600640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5680744b0 0x7fa5680769c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.744+0000 7fa58e600640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5881604c0 0x7fa58809d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.744+0000 7fa58e600640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5880d3250 0x7fa588154d90 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.744+0000 7fa58e600640 1 --2- 172.21.15.7:0/1041445164 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa58806ed00 0x7fa588158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:46.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.744+0000 7fa58e600640 1 -- 172.21.15.7:0/1041445164 >> 172.21.15.7:0/1041445164 conn(0x7fa5880b7640 msgr2=0x7fa5880b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:46.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.744+0000 7fa58e600640 1 -- 172.21.15.7:0/1041445164 shutdown_connections 2024-04-25T11:14:46.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:46.744+0000 7fa58e600640 1 -- 172.21.15.7:0/1041445164 wait complete. 2024-04-25T11:14:46.772 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:47.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.051+0000 7f49c9e00640 1 Processor -- start 2024-04-25T11:14:47.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.051+0000 7f49c9e00640 1 -- start start 2024-04-25T11:14:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.053+0000 7f49c9e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49c415ef90 0x7f49c409f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.053+0000 7f49c9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f49c415bc40 0x7f49c40aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.053+0000 7f49c9e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49c415f4e0 con 0x7f49c40af370 2024-04-25T11:14:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.053+0000 7f49c9e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f49c4071190 con 0x7f49c415ef90 2024-04-25T11:14:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.053+0000 7f49c9e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f49c40af780 con 0x7f49c415bc40 2024-04-25T11:14:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.053+0000 7f49c2a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49c415ef90 0x7f49c409f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.053+0000 7f49c3400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f49c40af370 0x7f49c4161b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53522/0 (socket says 172.21.15.7:53522) 2024-04-25T11:14:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.054+0000 7f49c3400640 1 -- 172.21.15.7:0/1133261225 learned_addr learned my addr 172.21.15.7:0/1133261225 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.054+0000 7f49c2000640 1 --2- 172.21.15.7:0/1133261225 >> v2:172.21.15.87:3300/0 conn(0x7f49c415bc40 0x7f49c40aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.054+0000 7f49c2a00640 1 -- 172.21.15.7:0/1133261225 >> v2:172.21.15.87:3300/0 conn(0x7f49c415bc40 msgr2=0x7f49c40aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.054+0000 7f49c2a00640 1 --2- 172.21.15.7:0/1133261225 >> v2:172.21.15.87:3300/0 conn(0x7f49c415bc40 0x7f49c40aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.054+0000 7f49c2a00640 1 -- 172.21.15.7:0/1133261225 >> v1:172.21.15.7:6789/0 conn(0x7f49c40af370 legacy=0x7f49c4161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.054+0000 7f49c2a00640 1 -- 172.21.15.7:0/1133261225 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c40b0560 con 0x7f49c415ef90 2024-04-25T11:14:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.054+0000 7f49c1600640 1 -- 172.21.15.7:0/1133261225 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 802464173 0 0) 0x7f49c415f4e0 con 0x7f49c40af370 2024-04-25T11:14:47.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.055+0000 7f49c2a00640 1 --2- 172.21.15.7:0/1133261225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c415ef90 0x7f49c409f790 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f49c406f2b0 tx=0x7f49b402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f892085e949b363 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.055+0000 7f49c1600640 1 -- 172.21.15.7:0/1133261225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4007740 con 0x7f49c415ef90 2024-04-25T11:14:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.056+0000 7f49c1600640 1 -- 172.21.15.7:0/1133261225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b40078e0 con 0x7f49c415ef90 2024-04-25T11:14:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.056+0000 7f49c1600640 1 -- 172.21.15.7:0/1133261225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4007cb0 con 0x7f49c415ef90 2024-04-25T11:14:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.056+0000 7f49c9e00640 1 -- 172.21.15.7:0/1133261225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c415ef90 msgr2=0x7f49c409f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.056+0000 7f49c9e00640 1 --2- 172.21.15.7:0/1133261225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c415ef90 0x7f49c409f790 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f49c406f2b0 tx=0x7f49b402a5f0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.056+0000 7f49c9e00640 1 -- 172.21.15.7:0/1133261225 shutdown_connections 2024-04-25T11:14:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.056+0000 7f49c9e00640 1 --2- 172.21.15.7:0/1133261225 >> v2:172.21.15.87:3300/0 conn(0x7f49c415bc40 0x7f49c40aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.056+0000 7f49c9e00640 1 --2- 172.21.15.7:0/1133261225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c415ef90 0x7f49c409f790 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.056+0000 7f49c9e00640 1 -- 172.21.15.7:0/1133261225 >> 172.21.15.7:0/1133261225 conn(0x7f49c4074200 msgr2=0x7f49c406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.056+0000 7f49c9e00640 1 -- 172.21.15.7:0/1133261225 shutdown_connections 2024-04-25T11:14:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.057+0000 7f49c9e00640 1 -- 172.21.15.7:0/1133261225 wait complete. 2024-04-25T11:14:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.059+0000 7f49c9e00640 1 Processor -- start 2024-04-25T11:14:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.059+0000 7f49c9e00640 1 -- start start 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49c40af370 0x7f49c4156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c415bc40 0x7f49c41f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c9e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49c415ef90 0x7f49c41f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c9e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49c40b0e00 con 0x7f49c415bc40 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c9e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49c40b0b20 con 0x7f49c40af370 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c9e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f49c40b1320 con 0x7f49c415ef90 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c415bc40 0x7f49c41f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c415bc40 0x7f49c41f7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57214/0 (socket says 172.21.15.7:57214) 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c2000640 1 -- 172.21.15.7:0/3545150611 learned_addr learned my addr 172.21.15.7:0/3545150611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c2a00640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49c40af370 0x7f49c4156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c3400640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49c415ef90 0x7f49c41f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c2000640 1 -- 172.21.15.7:0/3545150611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49c415ef90 msgr2=0x7f49c41f40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c2000640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49c415ef90 0x7f49c41f40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c2000640 1 -- 172.21.15.7:0/3545150611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49c40af370 msgr2=0x7f49c4156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c2000640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49c40af370 0x7f49c4156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.061+0000 7f49c2000640 1 -- 172.21.15.7:0/3545150611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c4154d10 con 0x7f49c415bc40 2024-04-25T11:14:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.062+0000 7f49c2000640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c415bc40 0x7f49c41f7000 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f49b000d620 tx=0x7f49b000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:47.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.062+0000 7f49bb400640 1 -- 172.21.15.7:0/3545150611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b000b620 con 0x7f49c415bc40 2024-04-25T11:14:47.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.062+0000 7f49bb400640 1 -- 172.21.15.7:0/3545150611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b0019070 con 0x7f49c415bc40 2024-04-25T11:14:47.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.062+0000 7f49bb400640 1 -- 172.21.15.7:0/3545150611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b000ba00 con 0x7f49c415bc40 2024-04-25T11:14:47.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.062+0000 7f49c9e00640 1 -- 172.21.15.7:0/3545150611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c41f49f0 con 0x7f49c415bc40 2024-04-25T11:14:47.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.062+0000 7f49c9e00640 1 -- 172.21.15.7:0/3545150611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c41f4fb0 con 0x7f49c415bc40 2024-04-25T11:14:47.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.063+0000 7f49c9e00640 1 -- 172.21.15.7:0/3545150611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c406a100 con 0x7f49c415bc40 2024-04-25T11:14:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.065+0000 7f49bb400640 1 -- 172.21.15.7:0/3545150611 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f49b000bba0 con 0x7f49c415bc40 2024-04-25T11:14:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.066+0000 7f49bb400640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49a4074500 0x7f49a4076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.066+0000 7f49bb400640 1 -- 172.21.15.7:0/3545150611 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f49b0095680 con 0x7f49c415bc40 2024-04-25T11:14:47.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.066+0000 7f49c2a00640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49a4074500 0x7f49a4076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.067+0000 7f49c2a00640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49a4074500 0x7f49a4076a10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f49b4034000 tx=0x7f49b4033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.071+0000 7f49bb400640 1 -- 172.21.15.7:0/3545150611 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49b005fdf0 con 0x7f49c415bc40 2024-04-25T11:14:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.313+0000 7f49c9e00640 1 -- 172.21.15.7:0/3545150611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49c415e0f0 con 0x7f49c415bc40 2024-04-25T11:14:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.314+0000 7f49bb400640 1 -- 172.21.15.7:0/3545150611 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7f49b005f6f0 con 0x7f49c415bc40 2024-04-25T11:14:47.317 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:47.317 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2941354720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2941354720},{"type":"v1","addr":"172.21.15.87:6834","nonce":2941354720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3135576784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3135576784},{"type":"v1","addr":"172.21.15.87:6835","nonce":3135576784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2590173274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2590173274},{"type":"v1","addr":"172.21.15.7:6836","nonce":2590173274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:14:20.877428+0000","modified":"2024-04-25T11:14:23.977883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6120":{"gid":6120,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2403088100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2403088100},{"type":"v1","addr":"172.21.15.7:6837","nonce":2403088100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6120,"qdb_cluster":[6120]},"id":4}]} 2024-04-25T11:14:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-04-25T11:14:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 -- 172.21.15.7:0/3545150611 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49a4074500 msgr2=0x7f49a4076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49a4074500 0x7f49a4076a10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f49b4034000 tx=0x7f49b4033040 comp rx=0 tx=0).stop 2024-04-25T11:14:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 -- 172.21.15.7:0/3545150611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c415bc40 msgr2=0x7f49c41f7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:47.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c415bc40 0x7f49c41f7000 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f49b000d620 tx=0x7f49b000dae0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 -- 172.21.15.7:0/3545150611 shutdown_connections 2024-04-25T11:14:47.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49a4074500 0x7f49a4076a10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49c415ef90 0x7f49c41f40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c415bc40 0x7f49c41f7000 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 --2- 172.21.15.7:0/3545150611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49c40af370 0x7f49c4156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 -- 172.21.15.7:0/3545150611 >> 172.21.15.7:0/3545150611 conn(0x7f49c4074200 msgr2=0x7f49c41621e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:47.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 -- 172.21.15.7:0/3545150611 shutdown_connections 2024-04-25T11:14:47.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.317+0000 7f49c9e00640 1 -- 172.21.15.7:0/3545150611 wait complete. 2024-04-25T11:14:47.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:47.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.627+0000 7f88f9800640 1 Processor -- start 2024-04-25T11:14:47.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.627+0000 7f88f9800640 1 -- start start 2024-04-25T11:14:47.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.630+0000 7f88f9800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88f40b29f0 0x7f88f40b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.630+0000 7f88f9800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88f40c67a0 0x7f88f40c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.630+0000 7f88f9800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88f40c70f0 con 0x7f88f40b8640 2024-04-25T11:14:47.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.630+0000 7f88f9800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f88f406f680 con 0x7f88f40c67a0 2024-04-25T11:14:47.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.630+0000 7f88f9800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f88f40b5090 con 0x7f88f40b29f0 2024-04-25T11:14:47.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.631+0000 7f88f3400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88f40c67a0 0x7f88f40c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.631+0000 7f88f3400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88f40c67a0 0x7f88f40c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57228/0 (socket says 172.21.15.7:57228) 2024-04-25T11:14:47.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.631+0000 7f88f3400640 1 -- 172.21.15.7:0/3556887069 learned_addr learned my addr 172.21.15.7:0/3556887069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:47.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.631+0000 7f88f3e00640 1 --2- 172.21.15.7:0/3556887069 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88f40b29f0 0x7f88f40b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.631+0000 7f88f3400640 1 -- 172.21.15.7:0/3556887069 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88f40b29f0 msgr2=0x7f88f40b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:47.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.631+0000 7f88f3400640 1 --2- 172.21.15.7:0/3556887069 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88f40b29f0 0x7f88f40b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.631+0000 7f88f3400640 1 -- 172.21.15.7:0/3556887069 >> v1:172.21.15.87:6789/0 conn(0x7f88f40b8640 legacy=0x7f88f40b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:47.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.631+0000 7f88f3400640 1 -- 172.21.15.7:0/3556887069 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f406d7b0 con 0x7f88f40c67a0 2024-04-25T11:14:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.632+0000 7f88f3400640 1 --2- 172.21.15.7:0/3556887069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c67a0 0x7f88f40c6bb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009870 tx=0x7f88dc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47f10dcd25f0f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.633+0000 7f88f2a00640 1 -- 172.21.15.7:0/3556887069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc037070 con 0x7f88f40c67a0 2024-04-25T11:14:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.633+0000 7f88f2a00640 1 -- 172.21.15.7:0/3556887069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88dc02ae10 con 0x7f88f40c67a0 2024-04-25T11:14:47.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.633+0000 7f88f2a00640 1 -- 172.21.15.7:0/3556887069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc032660 con 0x7f88f40c67a0 2024-04-25T11:14:47.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.633+0000 7f88f9800640 1 -- 172.21.15.7:0/3556887069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c67a0 msgr2=0x7f88f40c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:47.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.633+0000 7f88f9800640 1 --2- 172.21.15.7:0/3556887069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c67a0 0x7f88f40c6bb0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009870 tx=0x7f88dc02a100 comp rx=0 tx=0).stop 2024-04-25T11:14:47.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.634+0000 7f88f9800640 1 -- 172.21.15.7:0/3556887069 shutdown_connections 2024-04-25T11:14:47.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.634+0000 7f88f9800640 1 --2- 172.21.15.7:0/3556887069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c67a0 0x7f88f40c6bb0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.634+0000 7f88f9800640 1 --2- 172.21.15.7:0/3556887069 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88f40b29f0 0x7f88f40b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.634+0000 7f88f9800640 1 -- 172.21.15.7:0/3556887069 >> 172.21.15.7:0/3556887069 conn(0x7f88f40d0250 msgr2=0x7f88f40d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:47.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.634+0000 7f88f9800640 1 -- 172.21.15.7:0/3556887069 shutdown_connections 2024-04-25T11:14:47.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.634+0000 7f88f9800640 1 -- 172.21.15.7:0/3556887069 wait complete. 2024-04-25T11:14:47.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.636+0000 7f88f9800640 1 Processor -- start 2024-04-25T11:14:47.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.636+0000 7f88f9800640 1 -- start start 2024-04-25T11:14:47.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.638+0000 7f88f9800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88f40b29f0 0x7f88f4161e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.638+0000 7f88f9800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88f40b8640 0x7f88f4167040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.638+0000 7f88f9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c67a0 0x7f88f40b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.638+0000 7f88f9800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88f40b2e00 con 0x7f88f40c67a0 2024-04-25T11:14:47.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.638+0000 7f88f9800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f88f406dd70 con 0x7f88f40b8640 2024-04-25T11:14:47.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.638+0000 7f88f9800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f88f40d0d70 con 0x7f88f40b29f0 2024-04-25T11:14:47.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.638+0000 7f88f8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c67a0 0x7f88f40b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.638+0000 7f88f8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c67a0 0x7f88f40b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57238/0 (socket says 172.21.15.7:57238) 2024-04-25T11:14:47.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.638+0000 7f88f8e00640 1 -- 172.21.15.7:0/3208029462 learned_addr learned my addr 172.21.15.7:0/3208029462 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.639+0000 7f88f3400640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88f40b8640 0x7f88f4167040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.639+0000 7f88f3e00640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88f40b29f0 0x7f88f4161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.639+0000 7f88f8e00640 1 -- 172.21.15.7:0/3208029462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88f40b29f0 msgr2=0x7f88f4161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.639+0000 7f88f8e00640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88f40b29f0 0x7f88f4161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.639+0000 7f88f8e00640 1 -- 172.21.15.7:0/3208029462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88f40b8640 msgr2=0x7f88f4167040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.639+0000 7f88f8e00640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88f40b8640 0x7f88f4167040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.639+0000 7f88f8e00640 1 -- 172.21.15.7:0/3208029462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f4164fc0 con 0x7f88f40c67a0 2024-04-25T11:14:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.639+0000 7f88f8e00640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c67a0 0x7f88f40b4180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00b9e0 tx=0x7f88ec00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.640+0000 7f88ebe00640 1 -- 172.21.15.7:0/3208029462 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec003b70 con 0x7f88f40c67a0 2024-04-25T11:14:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.640+0000 7f88ebe00640 1 -- 172.21.15.7:0/3208029462 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ec003d10 con 0x7f88f40c67a0 2024-04-25T11:14:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.640+0000 7f88ebe00640 1 -- 172.21.15.7:0/3208029462 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00cb60 con 0x7f88f40c67a0 2024-04-25T11:14:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.640+0000 7f88f9800640 1 -- 172.21.15.7:0/3208029462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f41652a0 con 0x7f88f40c67a0 2024-04-25T11:14:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.640+0000 7f88f9800640 1 -- 172.21.15.7:0/3208029462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f4165860 con 0x7f88f40c67a0 2024-04-25T11:14:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.641+0000 7f88f9800640 1 -- 172.21.15.7:0/3208029462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f4164d30 con 0x7f88f40c67a0 2024-04-25T11:14:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.643+0000 7f88ebe00640 1 -- 172.21.15.7:0/3208029462 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f88ec00cd00 con 0x7f88f40c67a0 2024-04-25T11:14:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.643+0000 7f88ebe00640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88d00744e0 0x7f88d00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.644+0000 7f88ebe00640 1 -- 172.21.15.7:0/3208029462 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f88ec0951a0 con 0x7f88f40c67a0 2024-04-25T11:14:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.644+0000 7f88f3e00640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88d00744e0 0x7f88d00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:47.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.645+0000 7f88f3e00640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88d00744e0 0x7f88d00769f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f88f4168140 tx=0x7f88e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.649+0000 7f88ebe00640 1 -- 172.21.15.7:0/3208029462 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88ec05f140 con 0x7f88f40c67a0 2024-04-25T11:14:47.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.852+0000 7f88f9800640 1 -- 172.21.15.7:0/3208029462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88f40d3080 con 0x7f88f40c67a0 2024-04-25T11:14:47.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.855+0000 7f88ebe00640 1 -- 172.21.15.7:0/3208029462 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16788 (secure 0 0 0) 0x7f88ec063440 con 0x7f88f40c67a0 2024-04-25T11:14:47.856 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:47.857 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:22.341351+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:14:19.115741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T11:14:19.849856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:47.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.857+0000 7f88f9800640 1 -- 172.21.15.7:0/3208029462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88d00744e0 msgr2=0x7f88d00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:47.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.857+0000 7f88f9800640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88d00744e0 0x7f88d00769f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f88f4168140 tx=0x7f88e4008040 comp rx=0 tx=0).stop 2024-04-25T11:14:47.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.858+0000 7f88f9800640 1 -- 172.21.15.7:0/3208029462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c67a0 msgr2=0x7f88f40b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:47.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.858+0000 7f88f9800640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c67a0 0x7f88f40b4180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00b9e0 tx=0x7f88ec00bea0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.858+0000 7f88f9800640 1 -- 172.21.15.7:0/3208029462 shutdown_connections 2024-04-25T11:14:47.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.858+0000 7f88f9800640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88d00744e0 0x7f88d00769f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.858+0000 7f88f9800640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88f40c67a0 0x7f88f40b4180 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.858+0000 7f88f9800640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88f40b8640 0x7f88f4167040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.858+0000 7f88f9800640 1 --2- 172.21.15.7:0/3208029462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88f40b29f0 0x7f88f4161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.858+0000 7f88f9800640 1 -- 172.21.15.7:0/3208029462 >> 172.21.15.7:0/3208029462 conn(0x7f88f40d0250 msgr2=0x7f88f40714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.858+0000 7f88f9800640 1 -- 172.21.15.7:0/3208029462 shutdown_connections 2024-04-25T11:14:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:47.858+0000 7f88f9800640 1 -- 172.21.15.7:0/3208029462 wait complete. 2024-04-25T11:14:47.892 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:14:47.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.182+0000 7ff527200640 1 Processor -- start 2024-04-25T11:14:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.182+0000 7ff527200640 1 -- start start 2024-04-25T11:14:48.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.184+0000 7ff527200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5280aed40 0x7ff52815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.185+0000 7ff527200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5280cebb0 con 0x7ff5280af420 2024-04-25T11:14:48.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.185+0000 7ff527200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5280b64b0 con 0x7ff5280b5220 2024-04-25T11:14:48.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.185+0000 7ff527200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff52815e1f0 con 0x7ff5280aed40 2024-04-25T11:14:48.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.185+0000 7ff526800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff5280af420 0x7ff5280af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53528/0 (socket says 172.21.15.7:53528) 2024-04-25T11:14:48.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.185+0000 7ff526800640 1 -- 172.21.15.7:0/1510589221 learned_addr learned my addr 172.21.15.7:0/1510589221 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:48.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.185+0000 7ff525e00640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5280aed40 0x7ff52815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3730987467 0 0) 0x7ff520001490 con 0x7ff5280af420 2024-04-25T11:14:48.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff50c003520 con 0x7ff5280af420 2024-04-25T11:14:48.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2592723379 0 0) 0x7ff5280b64b0 con 0x7ff5280b5220 2024-04-25T11:14:48.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff520001490 con 0x7ff5280b5220 2024-04-25T11:14:48.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1733663384 0 0) 0x7ff50c003520 con 0x7ff5280af420 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5280aed40 msgr2=0x7ff52815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5280aed40 0x7ff52815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 >> v1:172.21.15.87:6789/0 conn(0x7ff5280b5220 legacy=0x7ff5280ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52806c610 con 0x7ff5280af420 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7ff520001bc0 con 0x7ff5280af420 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 >> v1:172.21.15.7:6789/0 conn(0x7ff5280af420 legacy=0x7ff5280af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff50c003b70 0x7ff50c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff50c004800 0x7ff50c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.186+0000 7ff524a00640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff50c005150 0x7ff50c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.187+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff52815e1f0 con 0x7ff50c005150 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.187+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c00c090 con 0x7ff50c004800 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.187+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c00c290 con 0x7ff50c003b70 2024-04-25T11:14:48.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.187+0000 7ff525e00640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5280aed40 0x7ff52815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:14:48.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.187+0000 7ff526800640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff50c005150 0x7ff50c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.187+0000 7ff525400640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff50c004800 0x7ff50c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.188+0000 7ff526800640 1 -- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff50c003b70 msgr2=0x7ff50c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.187+0000 7ff525e00640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff50c003b70 0x7ff50c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.188+0000 7ff526800640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff50c003b70 0x7ff50c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.188+0000 7ff526800640 1 -- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff50c004800 msgr2=0x7ff50c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.188+0000 7ff526800640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff50c004800 0x7ff50c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.188+0000 7ff526800640 1 -- 172.21.15.7:0/1510589221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c003520 con 0x7ff50c005150 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.188+0000 7ff525e00640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff50c003b70 0x7ff50c003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.189+0000 7ff526800640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff50c005150 0x7ff50c00b9c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff52000a9e0 tx=0x7ff50c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dd419bc889e161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.189+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52000a600 con 0x7ff50c005150 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.189+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff52002f070 con 0x7ff50c005150 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.189+0000 7ff524a00640 1 -- 172.21.15.7:0/1510589221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52002be30 con 0x7ff50c005150 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.190+0000 7ff527200640 1 -- 172.21.15.7:0/1510589221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff50c005150 msgr2=0x7ff50c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.190+0000 7ff527200640 1 --2- 172.21.15.7:0/1510589221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff50c005150 0x7ff50c00b9c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff52000a9e0 tx=0x7ff50c005830 comp rx=0 tx=0).stop 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.190+0000 7ff526800640 1 -- 172.21.15.7:0/1510589221 reap_dead start 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.190+0000 7ff527200640 1 -- 172.21.15.7:0/1510589221 shutdown_connections 2024-04-25T11:14:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.190+0000 7ff527200640 1 -- 172.21.15.7:0/1510589221 >> 172.21.15.7:0/1510589221 conn(0x7ff5280b3820 msgr2=0x7ff5280b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.190+0000 7ff527200640 1 -- 172.21.15.7:0/1510589221 shutdown_connections 2024-04-25T11:14:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.190+0000 7ff527200640 1 -- 172.21.15.7:0/1510589221 wait complete. 2024-04-25T11:14:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.193+0000 7ff527200640 1 Processor -- start 2024-04-25T11:14:48.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.193+0000 7ff527200640 1 -- start start 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.194+0000 7ff527200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff50c005150 0x7ff52814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.194+0000 7ff527200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff52815b7f0 0x7ff528159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.194+0000 7ff527200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff528159f60 0x7ff52815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.194+0000 7ff527200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff52806ceb0 con 0x7ff528159f60 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.194+0000 7ff527200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff52806cbd0 con 0x7ff50c005150 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.194+0000 7ff527200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5280b6110 con 0x7ff52815b7f0 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.194+0000 7ff526800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff528159f60 0x7ff52815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.195+0000 7ff526800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff528159f60 0x7ff52815eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57252/0 (socket says 172.21.15.7:57252) 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.195+0000 7ff526800640 1 -- 172.21.15.7:0/3230711599 learned_addr learned my addr 172.21.15.7:0/3230711599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.195+0000 7ff525400640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff52815b7f0 0x7ff528159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.195+0000 7ff525e00640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff50c005150 0x7ff52814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.195+0000 7ff526800640 1 -- 172.21.15.7:0/3230711599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff52815b7f0 msgr2=0x7ff528159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.195+0000 7ff526800640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff52815b7f0 0x7ff528159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.195+0000 7ff526800640 1 -- 172.21.15.7:0/3230711599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff50c005150 msgr2=0x7ff52814c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.195+0000 7ff526800640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff50c005150 0x7ff52814c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.195+0000 7ff526800640 1 -- 172.21.15.7:0/3230711599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52815a750 con 0x7ff528159f60 2024-04-25T11:14:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.195+0000 7ff525e00640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff50c005150 0x7ff52814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.195+0000 7ff526800640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff528159f60 0x7ff52815eb60 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff52002b680 tx=0x7ff520001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.196+0000 7ff51e000640 1 -- 172.21.15.7:0/3230711599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52000a600 con 0x7ff528159f60 2024-04-25T11:14:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.196+0000 7ff51e000640 1 -- 172.21.15.7:0/3230711599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff520001c30 con 0x7ff528159f60 2024-04-25T11:14:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.196+0000 7ff51e000640 1 -- 172.21.15.7:0/3230711599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520034a80 con 0x7ff528159f60 2024-04-25T11:14:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.196+0000 7ff527200640 1 -- 172.21.15.7:0/3230711599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52815aa30 con 0x7ff528159f60 2024-04-25T11:14:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.196+0000 7ff527200640 1 -- 172.21.15.7:0/3230711599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52815aff0 con 0x7ff528159f60 2024-04-25T11:14:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.197+0000 7ff527200640 1 -- 172.21.15.7:0/3230711599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff528050740 con 0x7ff528159f60 2024-04-25T11:14:48.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.198+0000 7ff51e000640 1 -- 172.21.15.7:0/3230711599 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff520034c20 con 0x7ff528159f60 2024-04-25T11:14:48.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.199+0000 7ff51e000640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5040744b0 0x7ff5040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.199+0000 7ff51e000640 1 -- 172.21.15.7:0/3230711599 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7ff5200b52c0 con 0x7ff528159f60 2024-04-25T11:14:48.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.200+0000 7ff525e00640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5040744b0 0x7ff5040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.200+0000 7ff525e00640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5040744b0 0x7ff5040769c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff518006e80 tx=0x7ff518008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:48.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.204+0000 7ff51e000640 1 -- 172.21.15.7:0/3230711599 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff520047070 con 0x7ff528159f60 2024-04-25T11:14:48.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.452+0000 7ff527200640 1 -- 172.21.15.7:0/3230711599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff52815bc00 con 0x7ff528159f60 2024-04-25T11:14:48.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.454+0000 7ff51e000640 1 -- 172.21.15.7:0/3230711599 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7ff5280cebb0 con 0x7ff528159f60 2024-04-25T11:14:48.456 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:48.456 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2941354720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2941354720},{"type":"v1","addr":"172.21.15.87:6834","nonce":2941354720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3135576784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3135576784},{"type":"v1","addr":"172.21.15.87:6835","nonce":3135576784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2590173274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2590173274},{"type":"v1","addr":"172.21.15.7:6836","nonce":2590173274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:14:20.877428+0000","modified":"2024-04-25T11:14:23.977883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6120":{"gid":6120,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2403088100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2403088100},{"type":"v1","addr":"172.21.15.7:6837","nonce":2403088100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6120,"qdb_cluster":[6120]},"id":4}]} 2024-04-25T11:14:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-04-25T11:14:48.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 -- 172.21.15.7:0/3230711599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5040744b0 msgr2=0x7ff5040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:48.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5040744b0 0x7ff5040769c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff518006e80 tx=0x7ff518008040 comp rx=0 tx=0).stop 2024-04-25T11:14:48.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 -- 172.21.15.7:0/3230711599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff528159f60 msgr2=0x7ff52815eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:48.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff528159f60 0x7ff52815eb60 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff52002b680 tx=0x7ff520001f10 comp rx=0 tx=0).stop 2024-04-25T11:14:48.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 -- 172.21.15.7:0/3230711599 shutdown_connections 2024-04-25T11:14:48.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5040744b0 0x7ff5040769c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff528159f60 0x7ff52815eb60 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff52815b7f0 0x7ff528159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 --2- 172.21.15.7:0/3230711599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff50c005150 0x7ff52814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 -- 172.21.15.7:0/3230711599 >> 172.21.15.7:0/3230711599 conn(0x7ff5280b3820 msgr2=0x7ff5280b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:48.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 -- 172.21.15.7:0/3230711599 shutdown_connections 2024-04-25T11:14:48.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.458+0000 7ff527200640 1 -- 172.21.15.7:0/3230711599 wait complete. 2024-04-25T11:14:48.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:48.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.771+0000 7f7f6c400640 1 Processor -- start 2024-04-25T11:14:48.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.771+0000 7f7f6c400640 1 -- start start 2024-04-25T11:14:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.774+0000 7f7f6c400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f640af540 0x7f7f6409f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.774+0000 7f7f6c400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f6415f4e0 con 0x7f7f6415bc40 2024-04-25T11:14:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.774+0000 7f7f6c400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f640b0560 con 0x7f7f6415ef90 2024-04-25T11:14:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.774+0000 7f7f6c400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7f64071190 con 0x7f7f640af540 2024-04-25T11:14:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.774+0000 7f7f6a400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7f6415bc40 0x7f7f640cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53534/0 (socket says 172.21.15.7:53534) 2024-04-25T11:14:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.774+0000 7f7f6a400640 1 -- 172.21.15.7:0/339994739 learned_addr learned my addr 172.21.15.7:0/339994739 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.774+0000 7f7f69000640 1 --2- 172.21.15.7:0/339994739 >> v2:172.21.15.87:3300/0 conn(0x7f7f640af540 0x7f7f6409f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 76952914 0 0) 0x7f7f6415f4e0 con 0x7f7f6415bc40 2024-04-25T11:14:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f4c003520 con 0x7f7f6415bc40 2024-04-25T11:14:48.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 689861739 0 0) 0x7f7f4c003520 con 0x7f7f6415bc40 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 >> v2:172.21.15.87:3300/0 conn(0x7f7f640af540 msgr2=0x7f7f6409f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 --2- 172.21.15.7:0/339994739 >> v2:172.21.15.87:3300/0 conn(0x7f7f640af540 0x7f7f6409f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 >> v1:172.21.15.87:6789/0 conn(0x7f7f6415ef90 legacy=0x7f7f640aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f640b0840 con 0x7f7f6415bc40 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f7f5c0019f0 con 0x7f7f6415bc40 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 >> v1:172.21.15.7:6789/0 conn(0x7f7f6415bc40 legacy=0x7f7f640cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f4c003a00 0x7f7f4c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f4c004800 0x7f7f4c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 0x7f7f4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f64071190 con 0x7f7f4c005350 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c00c230 con 0x7f7f4c004800 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.775+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4c00c430 con 0x7f7f4c003a00 2024-04-25T11:14:48.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.776+0000 7f7f6a400640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 0x7f7f4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.776+0000 7f7f69000640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f4c004800 0x7f7f4c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.776+0000 7f7f69a00640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f4c003a00 0x7f7f4c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.776+0000 7f7f6a400640 1 -- 172.21.15.7:0/339994739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f4c003a00 msgr2=0x7f7f4c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.776+0000 7f7f6a400640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f4c003a00 0x7f7f4c003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.776+0000 7f7f6a400640 1 -- 172.21.15.7:0/339994739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f4c004800 msgr2=0x7f7f4c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.776+0000 7f7f6a400640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f4c004800 0x7f7f4c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.776+0000 7f7f6a400640 1 -- 172.21.15.7:0/339994739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c003520 con 0x7f7f4c005350 2024-04-25T11:14:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.776+0000 7f7f69000640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f4c004800 0x7f7f4c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.777+0000 7f7f6a400640 1 -- 172.21.15.7:0/339994739 reap_dead start 2024-04-25T11:14:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.777+0000 7f7f6a400640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 0x7f7f4c00b9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00a7b0 tx=0x7f7f4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bb404aa20a1473 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.777+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00a3d0 con 0x7f7f4c005350 2024-04-25T11:14:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.778+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5c02f070 con 0x7f7f4c005350 2024-04-25T11:14:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.778+0000 7f7f63e00640 1 -- 172.21.15.7:0/339994739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c02bc40 con 0x7f7f4c005350 2024-04-25T11:14:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.778+0000 7f7f6c400640 1 -- 172.21.15.7:0/339994739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 msgr2=0x7f7f4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.778+0000 7f7f6c400640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 0x7f7f4c00b9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00a7b0 tx=0x7f7f4c005830 comp rx=0 tx=0).stop 2024-04-25T11:14:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.778+0000 7f7f6c400640 1 -- 172.21.15.7:0/339994739 shutdown_connections 2024-04-25T11:14:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.778+0000 7f7f6c400640 1 --2- 172.21.15.7:0/339994739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 0x7f7f4c00b9c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.778+0000 7f7f6c400640 1 -- 172.21.15.7:0/339994739 >> 172.21.15.7:0/339994739 conn(0x7f7f64074200 msgr2=0x7f7f6406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.779+0000 7f7f6c400640 1 -- 172.21.15.7:0/339994739 shutdown_connections 2024-04-25T11:14:48.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.779+0000 7f7f6c400640 1 -- 172.21.15.7:0/339994739 wait complete. 2024-04-25T11:14:48.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.781+0000 7f7f6c400640 1 Processor -- start 2024-04-25T11:14:48.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.782+0000 7f7f6c400640 1 -- start start 2024-04-25T11:14:48.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.783+0000 7f7f6c400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 0x7f7f64160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.783+0000 7f7f6c400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f640cda60 0x7f7f6414d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.783+0000 7f7f6c400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f641f60c0 0x7f7f641f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.783+0000 7f7f6c400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f640b1320 con 0x7f7f4c005350 2024-04-25T11:14:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.783+0000 7f7f6c400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7f640b0e00 con 0x7f7f640cda60 2024-04-25T11:14:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.783+0000 7f7f6c400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f640afb80 con 0x7f7f641f60c0 2024-04-25T11:14:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.783+0000 7f7f69a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 0x7f7f64160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.784+0000 7f7f69a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 0x7f7f64160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57270/0 (socket says 172.21.15.7:57270) 2024-04-25T11:14:48.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.784+0000 7f7f69a00640 1 -- 172.21.15.7:0/3186526528 learned_addr learned my addr 172.21.15.7:0/3186526528 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:48.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.784+0000 7f7f69a00640 1 -- 172.21.15.7:0/3186526528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f641f60c0 msgr2=0x7f7f641f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:48.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.784+0000 7f7f69a00640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f641f60c0 0x7f7f641f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.784+0000 7f7f69a00640 1 -- 172.21.15.7:0/3186526528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f640cda60 msgr2=0x7f7f6414d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:48.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.784+0000 7f7f69a00640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f640cda60 0x7f7f6414d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:48.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.784+0000 7f7f69a00640 1 -- 172.21.15.7:0/3186526528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6415fb70 con 0x7f7f4c005350 2024-04-25T11:14:48.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.784+0000 7f7f69a00640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 0x7f7f64160c10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800d4e0 tx=0x7f7f5800d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.785+0000 7f7f61600640 1 -- 172.21.15.7:0/3186526528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58016070 con 0x7f7f4c005350 2024-04-25T11:14:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.785+0000 7f7f61600640 1 -- 172.21.15.7:0/3186526528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f58012800 con 0x7f7f4c005350 2024-04-25T11:14:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.785+0000 7f7f61600640 1 -- 172.21.15.7:0/3186526528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f58012c30 con 0x7f7f4c005350 2024-04-25T11:14:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.785+0000 7f7f6c400640 1 -- 172.21.15.7:0/3186526528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6415fe50 con 0x7f7f4c005350 2024-04-25T11:14:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.785+0000 7f7f6c400640 1 -- 172.21.15.7:0/3186526528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f64160410 con 0x7f7f4c005350 2024-04-25T11:14:48.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.786+0000 7f7f6c400640 1 -- 172.21.15.7:0/3186526528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6406a100 con 0x7f7f4c005350 2024-04-25T11:14:48.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.787+0000 7f7f61600640 1 -- 172.21.15.7:0/3186526528 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7f58019070 con 0x7f7f4c005350 2024-04-25T11:14:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.788+0000 7f7f61600640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f440744b0 0x7f7f440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:48.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.788+0000 7f7f61600640 1 -- 172.21.15.7:0/3186526528 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f7f580952d0 con 0x7f7f4c005350 2024-04-25T11:14:48.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.789+0000 7f7f69000640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f440744b0 0x7f7f440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:48.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.790+0000 7f7f69000640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f440744b0 0x7f7f440769c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7f640b6030 tx=0x7f7f50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:48.793+0000 7f7f61600640 1 -- 172.21.15.7:0/3186526528 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7f5805f8c0 con 0x7f7f4c005350 2024-04-25T11:14:49.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.038+0000 7f7f6c400640 1 -- 172.21.15.7:0/3186526528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f6415ed70 con 0x7f7f4c005350 2024-04-25T11:14:49.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.039+0000 7f7f61600640 1 -- 172.21.15.7:0/3186526528 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5176 (secure 0 0 0) 0x7f7f5805f1c0 con 0x7f7f4c005350 2024-04-25T11:14:49.041 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:49.041 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2941354720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2941354720},{"type":"v1","addr":"172.21.15.87:6834","nonce":2941354720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3135576784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3135576784},{"type":"v1","addr":"172.21.15.87:6835","nonce":3135576784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2590173274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2590173274},{"type":"v1","addr":"172.21.15.7:6836","nonce":2590173274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:14:20.877428+0000","modified":"2024-04-25T11:14:23.977883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6120},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6120":{"gid":6120,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2403088100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2403088100},{"type":"v1","addr":"172.21.15.7:6837","nonce":2403088100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6120,"qdb_cluster":[6120]},"id":4}]} 2024-04-25T11:14:49.041 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-04-25T11:14:49.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 -- 172.21.15.7:0/3186526528 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f440744b0 msgr2=0x7f7f440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:49.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f440744b0 0x7f7f440769c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7f640b6030 tx=0x7f7f50008040 comp rx=0 tx=0).stop 2024-04-25T11:14:49.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 -- 172.21.15.7:0/3186526528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 msgr2=0x7f7f64160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:49.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 0x7f7f64160c10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800d4e0 tx=0x7f7f5800d9a0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 -- 172.21.15.7:0/3186526528 shutdown_connections 2024-04-25T11:14:49.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7f440744b0 0x7f7f440769c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f641f60c0 0x7f7f641f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f640cda60 0x7f7f6414d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 --2- 172.21.15.7:0/3186526528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f4c005350 0x7f7f64160c10 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 -- 172.21.15.7:0/3186526528 >> 172.21.15.7:0/3186526528 conn(0x7f7f64074200 msgr2=0x7f7f640b61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:49.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 -- 172.21.15.7:0/3186526528 shutdown_connections 2024-04-25T11:14:49.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.043+0000 7f7f6c400640 1 -- 172.21.15.7:0/3186526528 wait complete. 2024-04-25T11:14:49.070 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.352+0000 7f2d2ae00640 1 Processor -- start 2024-04-25T11:14:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.352+0000 7f2d2ae00640 1 -- start start 2024-04-25T11:14:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.354+0000 7f2d2ae00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2d240b2710 0x7f2d2409bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.355+0000 7f2d2ae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 0x7f2d241625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.355+0000 7f2d2ae00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d24162b10 0x7f2d24162f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.355+0000 7f2d2ae00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2d24163460 con 0x7f2d24162b10 2024-04-25T11:14:49.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.355+0000 7f2d2ae00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2d240ce2c0 con 0x7f2d240b2710 2024-04-25T11:14:49.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.355+0000 7f2d2ae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2d240b3fd0 con 0x7f2d2409c010 2024-04-25T11:14:49.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.355+0000 7f2d2a400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d24162b10 0x7f2d24162f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.355+0000 7f2d2a400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d24162b10 0x7f2d24162f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57272/0 (socket says 172.21.15.7:57272) 2024-04-25T11:14:49.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.355+0000 7f2d2a400640 1 -- 172.21.15.7:0/564109370 learned_addr learned my addr 172.21.15.7:0/564109370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:49.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.355+0000 7f2d29000640 1 --2- 172.21.15.7:0/564109370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 0x7f2d241625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.355+0000 7f2d29a00640 1 --2- 172.21.15.7:0/564109370 >> v2:172.21.15.87:3300/0 conn(0x7f2d240b2710 0x7f2d2409bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.356+0000 7f2d2a400640 1 -- 172.21.15.7:0/564109370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 msgr2=0x7f2d241625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:49.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.356+0000 7f2d2a400640 1 --2- 172.21.15.7:0/564109370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 0x7f2d241625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.356+0000 7f2d2a400640 1 -- 172.21.15.7:0/564109370 >> v2:172.21.15.87:3300/0 conn(0x7f2d240b2710 msgr2=0x7f2d2409bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:49.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.356+0000 7f2d2a400640 1 --2- 172.21.15.7:0/564109370 >> v2:172.21.15.87:3300/0 conn(0x7f2d240b2710 0x7f2d2409bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.356+0000 7f2d2a400640 1 -- 172.21.15.7:0/564109370 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d240b4eb0 con 0x7f2d24162b10 2024-04-25T11:14:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.356+0000 7f2d29000640 1 --2- 172.21.15.7:0/564109370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 0x7f2d241625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:14:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.357+0000 7f2d2a400640 1 --2- 172.21.15.7:0/564109370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d24162b10 0x7f2d24162f20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2d24071010 tx=0x7f2d1c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0952837deefc40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.357+0000 7f2d23e00640 1 -- 172.21.15.7:0/564109370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c036070 con 0x7f2d24162b10 2024-04-25T11:14:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.358+0000 7f2d23e00640 1 -- 172.21.15.7:0/564109370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d1c031600 con 0x7f2d24162b10 2024-04-25T11:14:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.358+0000 7f2d23e00640 1 -- 172.21.15.7:0/564109370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c0319d0 con 0x7f2d24162b10 2024-04-25T11:14:49.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.358+0000 7f2d2ae00640 1 -- 172.21.15.7:0/564109370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d24162b10 msgr2=0x7f2d24162f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:49.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.358+0000 7f2d2ae00640 1 --2- 172.21.15.7:0/564109370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d24162b10 0x7f2d24162f20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2d24071010 tx=0x7f2d1c02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.359+0000 7f2d2ae00640 1 -- 172.21.15.7:0/564109370 shutdown_connections 2024-04-25T11:14:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.359+0000 7f2d2ae00640 1 --2- 172.21.15.7:0/564109370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d24162b10 0x7f2d24162f20 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.359+0000 7f2d2ae00640 1 --2- 172.21.15.7:0/564109370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 0x7f2d241625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.359+0000 7f2d2ae00640 1 --2- 172.21.15.7:0/564109370 >> v2:172.21.15.87:3300/0 conn(0x7f2d240b2710 0x7f2d2409bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.359+0000 7f2d2ae00640 1 -- 172.21.15.7:0/564109370 >> 172.21.15.7:0/564109370 conn(0x7f2d240b5210 msgr2=0x7f2d240b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.359+0000 7f2d2ae00640 1 -- 172.21.15.7:0/564109370 shutdown_connections 2024-04-25T11:14:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.359+0000 7f2d2ae00640 1 -- 172.21.15.7:0/564109370 wait complete. 2024-04-25T11:14:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.361+0000 7f2d2ae00640 1 Processor -- start 2024-04-25T11:14:49.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.361+0000 7f2d2ae00640 1 -- start start 2024-04-25T11:14:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.363+0000 7f2d2ae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 0x7f2d24158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.363+0000 7f2d2ae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d240b2710 0x7f2d240b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.363+0000 7f2d2ae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d24162b10 0x7f2d240b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.363+0000 7f2d2ae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d240b3cf0 con 0x7f2d24162b10 2024-04-25T11:14:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.363+0000 7f2d2ae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2d240b3a10 con 0x7f2d240b2710 2024-04-25T11:14:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.363+0000 7f2d2ae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2d240b4a80 con 0x7f2d2409c010 2024-04-25T11:14:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.363+0000 7f2d2a400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d24162b10 0x7f2d240b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.363+0000 7f2d2a400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d24162b10 0x7f2d240b7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57286/0 (socket says 172.21.15.7:57286) 2024-04-25T11:14:49.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.363+0000 7f2d2a400640 1 -- 172.21.15.7:0/3208470709 learned_addr learned my addr 172.21.15.7:0/3208470709 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.363+0000 7f2d29a00640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 0x7f2d24158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d29000640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d240b2710 0x7f2d240b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d2a400640 1 -- 172.21.15.7:0/3208470709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 msgr2=0x7f2d24158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d2a400640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 0x7f2d24158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d2a400640 1 -- 172.21.15.7:0/3208470709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d240b2710 msgr2=0x7f2d240b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d2a400640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d240b2710 0x7f2d240b72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d2a400640 1 -- 172.21.15.7:0/3208470709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d240a0120 con 0x7f2d24162b10 2024-04-25T11:14:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d29a00640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 0x7f2d24158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d29000640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d240b2710 0x7f2d240b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d2a400640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d24162b10 0x7f2d240b7830 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c034040 tx=0x7f2d1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:49.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d21600640 1 -- 172.21.15.7:0/3208470709 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c036050 con 0x7f2d24162b10 2024-04-25T11:14:49.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d21600640 1 -- 172.21.15.7:0/3208470709 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d1c042070 con 0x7f2d24162b10 2024-04-25T11:14:49.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d2ae00640 1 -- 172.21.15.7:0/3208470709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d240a0400 con 0x7f2d24162b10 2024-04-25T11:14:49.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.364+0000 7f2d21600640 1 -- 172.21.15.7:0/3208470709 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c009510 con 0x7f2d24162b10 2024-04-25T11:14:49.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.365+0000 7f2d2ae00640 1 -- 172.21.15.7:0/3208470709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d240a09c0 con 0x7f2d24162b10 2024-04-25T11:14:49.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.366+0000 7f2d2ae00640 1 -- 172.21.15.7:0/3208470709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d24050740 con 0x7f2d24162b10 2024-04-25T11:14:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.367+0000 7f2d21600640 1 -- 172.21.15.7:0/3208470709 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d1c007bf0 con 0x7f2d24162b10 2024-04-25T11:14:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.368+0000 7f2d21600640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d040744b0 0x7f2d040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.368+0000 7f2d21600640 1 -- 172.21.15.7:0/3208470709 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f2d1c0b61e0 con 0x7f2d24162b10 2024-04-25T11:14:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.368+0000 7f2d29a00640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d040744b0 0x7f2d040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.369+0000 7f2d29a00640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d040744b0 0x7f2d040769c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009a20 tx=0x7f2d18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:49.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.372+0000 7f2d21600640 1 -- 172.21.15.7:0/3208470709 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d1c0807d0 con 0x7f2d24162b10 2024-04-25T11:14:49.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.578+0000 7f2d2ae00640 1 -- 172.21.15.7:0/3208470709 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d2406e690 con 0x7f2d24162b10 2024-04-25T11:14:49.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.581+0000 7f2d21600640 1 -- 172.21.15.7:0/3208470709 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16788 (secure 0 0 0) 0x7f2d1c0800d0 con 0x7f2d24162b10 2024-04-25T11:14:49.583 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:49.583 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:22.341351+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:14:19.115741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T11:14:19.849856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:49.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.584+0000 7f2d2ae00640 1 -- 172.21.15.7:0/3208470709 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d040744b0 msgr2=0x7f2d040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:49.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.584+0000 7f2d2ae00640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d040744b0 0x7f2d040769c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009a20 tx=0x7f2d18008040 comp rx=0 tx=0).stop 2024-04-25T11:14:49.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.584+0000 7f2d2ae00640 1 -- 172.21.15.7:0/3208470709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d24162b10 msgr2=0x7f2d240b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:49.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.584+0000 7f2d2ae00640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d24162b10 0x7f2d240b7830 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c034040 tx=0x7f2d1c004a40 comp rx=0 tx=0).stop 2024-04-25T11:14:49.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.584+0000 7f2d2ae00640 1 -- 172.21.15.7:0/3208470709 shutdown_connections 2024-04-25T11:14:49.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.584+0000 7f2d2ae00640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d040744b0 0x7f2d040769c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.584+0000 7f2d2ae00640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d24162b10 0x7f2d240b7830 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.584+0000 7f2d2ae00640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d240b2710 0x7f2d240b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.584+0000 7f2d2ae00640 1 --2- 172.21.15.7:0/3208470709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2409c010 0x7f2d24158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.584+0000 7f2d2ae00640 1 -- 172.21.15.7:0/3208470709 >> 172.21.15.7:0/3208470709 conn(0x7f2d240b5210 msgr2=0x7f2d240ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:49.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.584+0000 7f2d2ae00640 1 -- 172.21.15.7:0/3208470709 shutdown_connections 2024-04-25T11:14:49.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.585+0000 7f2d2ae00640 1 -- 172.21.15.7:0/3208470709 wait complete. 2024-04-25T11:14:49.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:14:49.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.901+0000 7f5ab0e00640 1 Processor -- start 2024-04-25T11:14:49.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.902+0000 7f5ab0e00640 1 -- start start 2024-04-25T11:14:49.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.904+0000 7f5ab0e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5aac06ed90 0x7f5aac1600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.904+0000 7f5ab0e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5aac0b3960 0x7f5aac0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.904+0000 7f5ab0e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aac0b7d70 con 0x7f5aac0b7960 2024-04-25T11:14:49.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.904+0000 7f5ab0e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5aac160630 con 0x7f5aac0b3960 2024-04-25T11:14:49.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.904+0000 7f5ab0e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5aac0af730 con 0x7f5aac06ed90 2024-04-25T11:14:49.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.905+0000 7f5aa3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5aac0b3960 0x7f5aac0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.905+0000 7f5aa3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5aac0b3960 0x7f5aac0b7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39384/0 (socket says 172.21.15.7:39384) 2024-04-25T11:14:49.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.905+0000 7f5aa3e00640 1 -- 172.21.15.7:0/1975017709 learned_addr learned my addr 172.21.15.7:0/1975017709 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:49.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.905+0000 7f5aab400640 1 --2- 172.21.15.7:0/1975017709 >> v2:172.21.15.87:3300/0 conn(0x7f5aac06ed90 0x7f5aac1600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.905+0000 7f5aa3e00640 1 -- 172.21.15.7:0/1975017709 >> v2:172.21.15.87:3300/0 conn(0x7f5aac06ed90 msgr2=0x7f5aac1600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:49.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.905+0000 7f5aa3e00640 1 --2- 172.21.15.7:0/1975017709 >> v2:172.21.15.87:3300/0 conn(0x7f5aac06ed90 0x7f5aac1600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.905+0000 7f5aa3e00640 1 -- 172.21.15.7:0/1975017709 >> v1:172.21.15.87:6789/0 conn(0x7f5aac0b7960 legacy=0x7f5aac0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:49.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.905+0000 7f5aa3e00640 1 -- 172.21.15.7:0/1975017709 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac0cf850 con 0x7f5aac0b3960 2024-04-25T11:14:49.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.906+0000 7f5aa3e00640 1 --2- 172.21.15.7:0/1975017709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aac0b3960 0x7f5aac0b7420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0b1540 tx=0x7f5a9402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337d22dceb03e2ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.907+0000 7f5aaaa00640 1 -- 172.21.15.7:0/1975017709 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94037070 con 0x7f5aac0b3960 2024-04-25T11:14:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.907+0000 7f5aaaa00640 1 -- 172.21.15.7:0/1975017709 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a94032440 con 0x7f5aac0b3960 2024-04-25T11:14:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.907+0000 7f5aaaa00640 1 -- 172.21.15.7:0/1975017709 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94032810 con 0x7f5aac0b3960 2024-04-25T11:14:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.908+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1975017709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aac0b3960 msgr2=0x7f5aac0b7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:49.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.908+0000 7f5ab0e00640 1 --2- 172.21.15.7:0/1975017709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aac0b3960 0x7f5aac0b7420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0b1540 tx=0x7f5a9402a230 comp rx=0 tx=0).stop 2024-04-25T11:14:49.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.908+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1975017709 shutdown_connections 2024-04-25T11:14:49.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.908+0000 7f5ab0e00640 1 --2- 172.21.15.7:0/1975017709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aac0b3960 0x7f5aac0b7420 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.908+0000 7f5ab0e00640 1 --2- 172.21.15.7:0/1975017709 >> v2:172.21.15.87:3300/0 conn(0x7f5aac06ed90 0x7f5aac1600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.908+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1975017709 >> 172.21.15.7:0/1975017709 conn(0x7f5aac09b540 msgr2=0x7f5aac09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:49.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.908+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1975017709 shutdown_connections 2024-04-25T11:14:49.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.908+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1975017709 wait complete. 2024-04-25T11:14:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.910+0000 7f5ab0e00640 1 Processor -- start 2024-04-25T11:14:49.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.911+0000 7f5ab0e00640 1 -- start start 2024-04-25T11:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.912+0000 7f5ab0e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5aac06ed90 0x7f5aac1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.912+0000 7f5ab0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aac0b3960 0x7f5aac0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.912+0000 7f5ab0e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aac0b7960 0x7f5aac14c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.912+0000 7f5ab0e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac0b3d70 con 0x7f5aac0b3960 2024-04-25T11:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.912+0000 7f5ab0e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac0cfe10 con 0x7f5aac0b7960 2024-04-25T11:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.912+0000 7f5ab0e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5aac0b3ef0 con 0x7f5aac06ed90 2024-04-25T11:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.912+0000 7f5aa3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aac0b3960 0x7f5aac0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.912+0000 7f5aa3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aac0b3960 0x7f5aac0c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39386/0 (socket says 172.21.15.7:39386) 2024-04-25T11:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.912+0000 7f5aa3e00640 1 -- 172.21.15.7:0/1704647094 learned_addr learned my addr 172.21.15.7:0/1704647094 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.913+0000 7f5aa3e00640 1 -- 172.21.15.7:0/1704647094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5aac06ed90 msgr2=0x7f5aac1f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.913+0000 7f5aa3e00640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5aac06ed90 0x7f5aac1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.913+0000 7f5aa3e00640 1 -- 172.21.15.7:0/1704647094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aac0b7960 msgr2=0x7f5aac14c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.913+0000 7f5aabe00640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aac0b7960 0x7f5aac14c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.913+0000 7f5aa3e00640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aac0b7960 0x7f5aac14c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.913+0000 7f5aa3e00640 1 -- 172.21.15.7:0/1704647094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac15b120 con 0x7f5aac0b3960 2024-04-25T11:14:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.913+0000 7f5aa3e00640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aac0b3960 0x7f5aac0c9700 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5a94000c00 tx=0x7f5a940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.914+0000 7f5aa3400640 1 -- 172.21.15.7:0/1704647094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94041020 con 0x7f5aac0b3960 2024-04-25T11:14:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.914+0000 7f5aa3400640 1 -- 172.21.15.7:0/1704647094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a940306c0 con 0x7f5aac0b3960 2024-04-25T11:14:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.914+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1704647094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac15b400 con 0x7f5aac0b3960 2024-04-25T11:14:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.914+0000 7f5aa3400640 1 -- 172.21.15.7:0/1704647094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94037040 con 0x7f5aac0b3960 2024-04-25T11:14:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.914+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1704647094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac15b9c0 con 0x7f5aac0b3960 2024-04-25T11:14:49.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.915+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1704647094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aac050740 con 0x7f5aac0b3960 2024-04-25T11:14:49.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.916+0000 7f5aa3400640 1 -- 172.21.15.7:0/1704647094 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5a94044020 con 0x7f5aac0b3960 2024-04-25T11:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.917+0000 7f5aa3400640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a880744b0 0x7f5a880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.917+0000 7f5aa3400640 1 -- 172.21.15.7:0/1704647094 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6101+0+0 (secure 0 0 0) 0x7f5a940b5cb0 con 0x7f5aac0b3960 2024-04-25T11:14:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.917+0000 7f5aab400640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a880744b0 0x7f5a880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.918+0000 7f5aab400640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a880744b0 0x7f5a880769c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c006fd0 tx=0x7f5a9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:49.922+0000 7f5aa3400640 1 -- 172.21.15.7:0/1704647094 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a9404b030 con 0x7f5aac0b3960 2024-04-25T11:14:50.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.165+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1704647094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5aac071f80 con 0x7f5aac0b3960 2024-04-25T11:14:50.168 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-04-25T11:14:50.167+0000 7fac34e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:14:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.171+0000 7f5aa3400640 1 -- 172.21.15.7:0/1704647094 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7f5aac0b7d70 con 0x7f5aac0b3960 2024-04-25T11:14:50.173 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:14:50.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.174+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1704647094 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a880744b0 msgr2=0x7f5a880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:50.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.175+0000 7f5ab0e00640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a880744b0 0x7f5a880769c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c006fd0 tx=0x7f5a9c008040 comp rx=0 tx=0).stop 2024-04-25T11:14:50.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.175+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1704647094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aac0b3960 msgr2=0x7f5aac0c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.175+0000 7f5ab0e00640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aac0b3960 0x7f5aac0c9700 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5a94000c00 tx=0x7f5a940046c0 comp rx=0 tx=0).stop 2024-04-25T11:14:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.175+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1704647094 shutdown_connections 2024-04-25T11:14:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.175+0000 7f5ab0e00640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a880744b0 0x7f5a880769c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.175+0000 7f5ab0e00640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5aac0b7960 0x7f5aac14c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.175+0000 7f5ab0e00640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5aac0b3960 0x7f5aac0c9700 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.175+0000 7f5ab0e00640 1 --2- 172.21.15.7:0/1704647094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5aac06ed90 0x7f5aac1f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.175+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1704647094 >> 172.21.15.7:0/1704647094 conn(0x7f5aac09b540 msgr2=0x7f5aac071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.176+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1704647094 shutdown_connections 2024-04-25T11:14:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.176+0000 7f5ab0e00640 1 -- 172.21.15.7:0/1704647094 wait complete. 2024-04-25T11:14:50.183 INFO:tasks.ceph.mds.c.smithi007.stderr: -2556> 2024-04-25T11:14:25.807+0000 7f5efcc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:14:50.207 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:14:50.322 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:14:50.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.473+0000 7efd55e00640 1 Processor -- start 2024-04-25T11:14:50.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.473+0000 7efd55e00640 1 -- start start 2024-04-25T11:14:50.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.476+0000 7efd55e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efd5015dfb0 0x7efd500b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:50.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.476+0000 7efd55e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd500d0b40 0x7efd5006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:50.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.476+0000 7efd55e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd5015e3c0 con 0x7efd500d0f50 2024-04-25T11:14:50.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.476+0000 7efd55e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efd500b09b0 con 0x7efd5015dfb0 2024-04-25T11:14:50.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.476+0000 7efd55e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efd500cf800 con 0x7efd500d0b40 2024-04-25T11:14:50.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.476+0000 7efd4ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efd5015dfb0 0x7efd500b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:50.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.476+0000 7efd4e000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd500d0b40 0x7efd5006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:50.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.476+0000 7efd4f400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7efd500d0f50 0x7efd50161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:46036/0 (socket says 172.21.15.7:46036) 2024-04-25T11:14:50.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.476+0000 7efd4f400640 1 -- 172.21.15.7:0/2423662528 learned_addr learned my addr 172.21.15.7:0/2423662528 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:50.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.477+0000 7efd4ea00640 1 -- 172.21.15.7:0/2423662528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd500d0b40 msgr2=0x7efd5006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:50.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.477+0000 7efd4ea00640 1 --2- 172.21.15.7:0/2423662528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd500d0b40 0x7efd5006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:50.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.477+0000 7efd4ea00640 1 -- 172.21.15.7:0/2423662528 >> v1:172.21.15.87:6789/0 conn(0x7efd500d0f50 legacy=0x7efd50161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:50.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.477+0000 7efd4ea00640 1 -- 172.21.15.7:0/2423662528 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd500cfaa0 con 0x7efd5015dfb0 2024-04-25T11:14:50.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.478+0000 7efd4ea00640 1 --2- 172.21.15.7:0/2423662528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd5015dfb0 0x7efd500b0470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efd400097a0 tx=0x7efd4002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d665072dae7403 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:50.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.479+0000 7efd4d600640 1 -- 172.21.15.7:0/2423662528 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40037070 con 0x7efd5015dfb0 2024-04-25T11:14:50.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.479+0000 7efd4d600640 1 -- 172.21.15.7:0/2423662528 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd40032440 con 0x7efd5015dfb0 2024-04-25T11:14:50.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.479+0000 7efd4d600640 1 -- 172.21.15.7:0/2423662528 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40032810 con 0x7efd5015dfb0 2024-04-25T11:14:50.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.480+0000 7efd55e00640 1 -- 172.21.15.7:0/2423662528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd5015dfb0 msgr2=0x7efd500b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:50.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.480+0000 7efd55e00640 1 --2- 172.21.15.7:0/2423662528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd5015dfb0 0x7efd500b0470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efd400097a0 tx=0x7efd4002a180 comp rx=0 tx=0).stop 2024-04-25T11:14:50.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.480+0000 7efd55e00640 1 -- 172.21.15.7:0/2423662528 shutdown_connections 2024-04-25T11:14:50.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.480+0000 7efd55e00640 1 --2- 172.21.15.7:0/2423662528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd500d0b40 0x7efd5006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:50.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.480+0000 7efd55e00640 1 --2- 172.21.15.7:0/2423662528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd5015dfb0 0x7efd500b0470 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:50.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.480+0000 7efd55e00640 1 -- 172.21.15.7:0/2423662528 >> 172.21.15.7:0/2423662528 conn(0x7efd500742c0 msgr2=0x7efd5006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:50.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.480+0000 7efd55e00640 1 -- 172.21.15.7:0/2423662528 shutdown_connections 2024-04-25T11:14:50.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.480+0000 7efd55e00640 1 -- 172.21.15.7:0/2423662528 wait complete. 2024-04-25T11:14:50.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.483+0000 7efd55e00640 1 Processor -- start 2024-04-25T11:14:50.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.483+0000 7efd55e00640 1 -- start start 2024-04-25T11:14:50.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.485+0000 7efd55e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd500d0b40 0x7efd500cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:50.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.485+0000 7efd55e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd500d0f50 0x7efd501f6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:50.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.485+0000 7efd55e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd5015dfb0 0x7efd501566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.485+0000 7efd55e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd500d0340 con 0x7efd500d0f50 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.485+0000 7efd55e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efd500d0060 con 0x7efd5015dfb0 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.485+0000 7efd55e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efd500d0620 con 0x7efd500d0b40 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.485+0000 7efd4e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd500d0f50 0x7efd501f6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.485+0000 7efd4e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd500d0f50 0x7efd501f6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39408/0 (socket says 172.21.15.7:39408) 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.485+0000 7efd4e000640 1 -- 172.21.15.7:0/3720694885 learned_addr learned my addr 172.21.15.7:0/3720694885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.485+0000 7efd4ea00640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd500d0b40 0x7efd500cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.485+0000 7efd4f400640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd5015dfb0 0x7efd501566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.486+0000 7efd4e000640 1 -- 172.21.15.7:0/3720694885 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd500d0b40 msgr2=0x7efd500cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.486+0000 7efd4e000640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd500d0b40 0x7efd500cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.486+0000 7efd4e000640 1 -- 172.21.15.7:0/3720694885 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd5015dfb0 msgr2=0x7efd501566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.486+0000 7efd4e000640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd5015dfb0 0x7efd501566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.486+0000 7efd4e000640 1 -- 172.21.15.7:0/3720694885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd501f6ec0 con 0x7efd500d0f50 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.486+0000 7efd4e000640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd500d0f50 0x7efd501f6400 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00e8a0 tx=0x7efd3c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.486+0000 7efd47400640 1 -- 172.21.15.7:0/3720694885 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c017070 con 0x7efd500d0f50 2024-04-25T11:14:50.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.486+0000 7efd47400640 1 -- 172.21.15.7:0/3720694885 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd3c00c940 con 0x7efd500d0f50 2024-04-25T11:14:50.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.486+0000 7efd47400640 1 -- 172.21.15.7:0/3720694885 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c00cd30 con 0x7efd500d0f50 2024-04-25T11:14:50.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.486+0000 7efd55e00640 1 -- 172.21.15.7:0/3720694885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd501f71a0 con 0x7efd500d0f50 2024-04-25T11:14:50.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.486+0000 7efd55e00640 1 -- 172.21.15.7:0/3720694885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd501f6940 con 0x7efd500d0f50 2024-04-25T11:14:50.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.487+0000 7efd55e00640 1 -- 172.21.15.7:0/3720694885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd50050740 con 0x7efd500d0f50 2024-04-25T11:14:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.489+0000 7efd47400640 1 -- 172.21.15.7:0/3720694885 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efd3c01a070 con 0x7efd500d0f50 2024-04-25T11:14:50.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.491+0000 7efd47400640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efd300744b0 0x7efd300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:50.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.491+0000 7efd47400640 1 -- 172.21.15.7:0/3720694885 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6187+0+0 (secure 0 0 0) 0x7efd3c0956f0 con 0x7efd500d0f50 2024-04-25T11:14:50.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.491+0000 7efd4ea00640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efd300744b0 0x7efd300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:50.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.492+0000 7efd4ea00640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efd300744b0 0x7efd300769c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efd40034000 tx=0x7efd4002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.496+0000 7efd47400640 1 -- 172.21.15.7:0/3720694885 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efd3c05fc90 con 0x7efd500d0f50 2024-04-25T11:14:50.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:50.743+0000 7efd55e00640 1 -- 172.21.15.7:0/3720694885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7efd501f6c50 con 0x7efd500d0f50 2024-04-25T11:14:51.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.175+0000 7efd47400640 1 -- 172.21.15.7:0/3720694885 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7efd3c05f590 con 0x7efd500d0f50 2024-04-25T11:14:51.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 -- 172.21.15.7:0/3720694885 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efd300744b0 msgr2=0x7efd300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:51.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efd300744b0 0x7efd300769c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7efd40034000 tx=0x7efd4002e000 comp rx=0 tx=0).stop 2024-04-25T11:14:51.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 -- 172.21.15.7:0/3720694885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd500d0f50 msgr2=0x7efd501f6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:51.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd500d0f50 0x7efd501f6400 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00e8a0 tx=0x7efd3c00ed60 comp rx=0 tx=0).stop 2024-04-25T11:14:51.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 -- 172.21.15.7:0/3720694885 shutdown_connections 2024-04-25T11:14:51.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efd300744b0 0x7efd300769c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:51.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd5015dfb0 0x7efd501566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:51.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd500d0f50 0x7efd501f6400 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:51.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 --2- 172.21.15.7:0/3720694885 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd500d0b40 0x7efd500cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:51.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 -- 172.21.15.7:0/3720694885 >> 172.21.15.7:0/3720694885 conn(0x7efd500742c0 msgr2=0x7efd500ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:51.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 -- 172.21.15.7:0/3720694885 shutdown_connections 2024-04-25T11:14:51.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.179+0000 7efd55e00640 1 -- 172.21.15.7:0/3720694885 wait complete. 2024-04-25T11:14:51.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:14:51.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.510+0000 7f6b5ea00640 1 Processor -- start 2024-04-25T11:14:51.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.510+0000 7f6b5ea00640 1 -- start start 2024-04-25T11:14:51.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.513+0000 7f6b5ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5806d580 0x7f6b5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:51.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.513+0000 7f6b5ea00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b5806ded0 con 0x7f6b581610d0 2024-04-25T11:14:51.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.513+0000 7f6b5ea00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b580b4630 con 0x7f6b5815f3d0 2024-04-25T11:14:51.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.513+0000 7f6b5ea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6b580b48d0 con 0x7f6b5806d580 2024-04-25T11:14:51.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.513+0000 7f6b5d600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6b581610d0 0x7f6b580b5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43232/0 (socket says 172.21.15.7:43232) 2024-04-25T11:14:51.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.513+0000 7f6b5d600640 1 -- 172.21.15.7:0/1567866860 learned_addr learned my addr 172.21.15.7:0/1567866860 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:51.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.513+0000 7f6b5cc00640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5806d580 0x7f6b5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:51.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 708478059 0 0) 0x7f6b4c001490 con 0x7f6b581610d0 2024-04-25T11:14:51.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b40003520 con 0x7f6b581610d0 2024-04-25T11:14:51.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 26589513 0 0) 0x7f6b40003520 con 0x7f6b581610d0 2024-04-25T11:14:51.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5806d580 msgr2=0x7f6b5806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:51.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5806d580 0x7f6b5806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:51.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 >> v1:172.21.15.87:6789/0 conn(0x7f6b5815f3d0 legacy=0x7f6b5815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:51.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b580b4bb0 con 0x7f6b581610d0 2024-04-25T11:14:51.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f6b4c001bc0 con 0x7f6b581610d0 2024-04-25T11:14:51.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 >> v1:172.21.15.7:6789/0 conn(0x7f6b581610d0 legacy=0x7f6b580b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:51.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5806d580 0x7f6b400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:51.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 0x7f6b40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:51.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b40004d40 0x7f6b400051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:51.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b580b48d0 con 0x7f6b40003fb0 2024-04-25T11:14:51.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4000bbb0 con 0x7f6b40004d40 2024-04-25T11:14:51.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.514+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6b4000bdb0 con 0x7f6b5806d580 2024-04-25T11:14:51.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.515+0000 7f6b5cc00640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 0x7f6b40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:51.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.515+0000 7f6b5e000640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b40004d40 0x7f6b400051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:51.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.515+0000 7f6b5d600640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5806d580 0x7f6b400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:51.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.515+0000 7f6b5cc00640 1 -- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5806d580 msgr2=0x7f6b400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.515+0000 7f6b5cc00640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5806d580 0x7f6b400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.515+0000 7f6b5cc00640 1 -- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b40004d40 msgr2=0x7f6b400051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.515+0000 7f6b5cc00640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b40004d40 0x7f6b400051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.515+0000 7f6b5cc00640 1 -- 172.21.15.7:0/1567866860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b40003520 con 0x7f6b40003fb0 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.515+0000 7f6b5d600640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5806d580 0x7f6b400039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.516+0000 7f6b5cc00640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 0x7f6b40004800 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009a90 tx=0x7f6b40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e5a4c29fb4e9c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.517+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44036070 con 0x7f6b40003fb0 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.517+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b44031400 con 0x7f6b40003fb0 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.517+0000 7f6b57e00640 1 -- 172.21.15.7:0/1567866860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44009690 con 0x7f6b40003fb0 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.517+0000 7f6b5ea00640 1 -- 172.21.15.7:0/1567866860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 msgr2=0x7f6b40004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.517+0000 7f6b5ea00640 1 --2- 172.21.15.7:0/1567866860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 0x7f6b40004800 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009a90 tx=0x7f6b40005830 comp rx=0 tx=0).stop 2024-04-25T11:14:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.517+0000 7f6b5e000640 1 -- 172.21.15.7:0/1567866860 reap_dead start 2024-04-25T11:14:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.518+0000 7f6b5ea00640 1 -- 172.21.15.7:0/1567866860 shutdown_connections 2024-04-25T11:14:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.518+0000 7f6b5ea00640 1 -- 172.21.15.7:0/1567866860 >> 172.21.15.7:0/1567866860 conn(0x7f6b58073160 msgr2=0x7f6b58073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.518+0000 7f6b5ea00640 1 -- 172.21.15.7:0/1567866860 shutdown_connections 2024-04-25T11:14:51.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.518+0000 7f6b5ea00640 1 -- 172.21.15.7:0/1567866860 wait complete. 2024-04-25T11:14:51.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.520+0000 7f6b5ea00640 1 Processor -- start 2024-04-25T11:14:51.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.520+0000 7f6b5ea00640 1 -- start start 2024-04-25T11:14:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 0x7f6b5814fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b58159d90 0x7f6b5814e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b580702d0 0x7f6b580706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b581614e0 con 0x7f6b40003fb0 2024-04-25T11:14:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5ea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6b580b5170 con 0x7f6b580702d0 2024-04-25T11:14:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5ea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6b5815f0d0 con 0x7f6b58159d90 2024-04-25T11:14:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 0x7f6b5814fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 0x7f6b5814fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39416/0 (socket says 172.21.15.7:39416) 2024-04-25T11:14:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5d600640 1 -- 172.21.15.7:0/854707965 learned_addr learned my addr 172.21.15.7:0/854707965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5e000640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b580702d0 0x7f6b580706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5cc00640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b58159d90 0x7f6b5814e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5d600640 1 -- 172.21.15.7:0/854707965 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b58159d90 msgr2=0x7f6b5814e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.522+0000 7f6b5d600640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b58159d90 0x7f6b5814e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.523+0000 7f6b5d600640 1 -- 172.21.15.7:0/854707965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b580702d0 msgr2=0x7f6b580706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.523+0000 7f6b5d600640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b580702d0 0x7f6b580706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.523+0000 7f6b5d600640 1 -- 172.21.15.7:0/854707965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b58070c20 con 0x7f6b40003fb0 2024-04-25T11:14:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.523+0000 7f6b5d600640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 0x7f6b5814fd90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c001cb0 tx=0x7f6b4c00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.523+0000 7f6b55600640 1 -- 172.21.15.7:0/854707965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c019070 con 0x7f6b40003fb0 2024-04-25T11:14:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.523+0000 7f6b55600640 1 -- 172.21.15.7:0/854707965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4c004d60 con 0x7f6b40003fb0 2024-04-25T11:14:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.523+0000 7f6b55600640 1 -- 172.21.15.7:0/854707965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c0072d0 con 0x7f6b40003fb0 2024-04-25T11:14:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.523+0000 7f6b5ea00640 1 -- 172.21.15.7:0/854707965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5815a570 con 0x7f6b40003fb0 2024-04-25T11:14:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.523+0000 7f6b5ea00640 1 -- 172.21.15.7:0/854707965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5815ab30 con 0x7f6b40003fb0 2024-04-25T11:14:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.524+0000 7f6b5ea00640 1 -- 172.21.15.7:0/854707965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b58050740 con 0x7f6b40003fb0 2024-04-25T11:14:51.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.526+0000 7f6b55600640 1 -- 172.21.15.7:0/854707965 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6b4c01c070 con 0x7f6b40003fb0 2024-04-25T11:14:51.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.527+0000 7f6b55600640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 0x7f6b380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.527+0000 7f6b55600640 1 -- 172.21.15.7:0/854707965 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6187+0+0 (secure 0 0 0) 0x7f6b4c095770 con 0x7f6b40003fb0 2024-04-25T11:14:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.527+0000 7f6b5cc00640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 0x7f6b380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.528+0000 7f6b5cc00640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 0x7f6b380769f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6b44033000 tx=0x7f6b44008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.532+0000 7f6b55600640 1 -- 172.21.15.7:0/854707965 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b58070c20 con 0x7f6b40003fb0 2024-04-25T11:14:51.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:51.725+0000 7f6b5ea00640 1 -- 172.21.15.7:0/854707965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6b58160b30 con 0x7f6b40003fb0 2024-04-25T11:14:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.552+0000 7f6b55600640 1 -- 172.21.15.7:0/854707965 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v70) ==== 181+0+0 (secure 0 0 0) 0x7f6b5806ded0 con 0x7f6b40003fb0 2024-04-25T11:14:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:14:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.556+0000 7f6b5ea00640 1 -- 172.21.15.7:0/854707965 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 msgr2=0x7f6b380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.556+0000 7f6b5ea00640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 0x7f6b380769f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6b44033000 tx=0x7f6b44008030 comp rx=0 tx=0).stop 2024-04-25T11:14:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.556+0000 7f6b5ea00640 1 -- 172.21.15.7:0/854707965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 msgr2=0x7f6b5814fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.556+0000 7f6b5ea00640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 0x7f6b5814fd90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c001cb0 tx=0x7f6b4c00b560 comp rx=0 tx=0).stop 2024-04-25T11:14:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.557+0000 7f6b5ea00640 1 -- 172.21.15.7:0/854707965 shutdown_connections 2024-04-25T11:14:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.557+0000 7f6b5ea00640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 0x7f6b380769f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.557+0000 7f6b5ea00640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b580702d0 0x7f6b580706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.557+0000 7f6b5ea00640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b58159d90 0x7f6b5814e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:52.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.557+0000 7f6b5ea00640 1 --2- 172.21.15.7:0/854707965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b40003fb0 0x7f6b5814fd90 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.557+0000 7f6b5ea00640 1 -- 172.21.15.7:0/854707965 >> 172.21.15.7:0/854707965 conn(0x7f6b58073160 msgr2=0x7f6b581553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.558+0000 7f6b5ea00640 1 -- 172.21.15.7:0/854707965 shutdown_connections 2024-04-25T11:14:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.558+0000 7f6b5ea00640 1 -- 172.21.15.7:0/854707965 wait complete. 2024-04-25T11:14:52.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:14:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.881+0000 7f58fd000640 1 Processor -- start 2024-04-25T11:14:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.881+0000 7f58fd000640 1 -- start start 2024-04-25T11:14:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.883+0000 7f58fd000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58f806d580 0x7f58f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.883+0000 7f58fd000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58f815ca60 0x7f58f815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.883+0000 7f58fd000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58f806ded0 con 0x7f58f80b4690 2024-04-25T11:14:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.883+0000 7f58fd000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f58f806fef0 con 0x7f58f806d580 2024-04-25T11:14:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.883+0000 7f58fd000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f58f815d7d0 con 0x7f58f815ca60 2024-04-25T11:14:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.884+0000 7f58f7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58f806d580 0x7f58f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.884+0000 7f58f7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58f806d580 0x7f58f806d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39432/0 (socket says 172.21.15.7:39432) 2024-04-25T11:14:52.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.884+0000 7f58f7400640 1 -- 172.21.15.7:0/3184996330 learned_addr learned my addr 172.21.15.7:0/3184996330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:52.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.884+0000 7f58f6a00640 1 --2- 172.21.15.7:0/3184996330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58f815ca60 0x7f58f815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:52.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.884+0000 7f58f7400640 1 -- 172.21.15.7:0/3184996330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58f815ca60 msgr2=0x7f58f815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:52.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.884+0000 7f58f7400640 1 --2- 172.21.15.7:0/3184996330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58f815ca60 0x7f58f815ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:52.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.884+0000 7f58f7400640 1 -- 172.21.15.7:0/3184996330 >> v1:172.21.15.87:6789/0 conn(0x7f58f80b4690 legacy=0x7f58f80b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:52.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.884+0000 7f58f7400640 1 -- 172.21.15.7:0/3184996330 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f815ff00 con 0x7f58f806d580 2024-04-25T11:14:52.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.884+0000 7f58f6a00640 1 --2- 172.21.15.7:0/3184996330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58f815ca60 0x7f58f815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.885+0000 7f58f7400640 1 --2- 172.21.15.7:0/3184996330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f806d580 0x7f58f806d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f58e800d150 tx=0x7f58e802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0569c8cab4d8a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.886+0000 7f58f6000640 1 -- 172.21.15.7:0/3184996330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8035400 con 0x7f58f806d580 2024-04-25T11:14:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.886+0000 7f58f6000640 1 -- 172.21.15.7:0/3184996330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e80355a0 con 0x7f58f806d580 2024-04-25T11:14:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.886+0000 7f58f6000640 1 -- 172.21.15.7:0/3184996330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8035970 con 0x7f58f806d580 2024-04-25T11:14:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.887+0000 7f58fd000640 1 -- 172.21.15.7:0/3184996330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f806d580 msgr2=0x7f58f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.887+0000 7f58fd000640 1 --2- 172.21.15.7:0/3184996330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f806d580 0x7f58f806d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f58e800d150 tx=0x7f58e802c8e0 comp rx=0 tx=0).stop 2024-04-25T11:14:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.887+0000 7f58fd000640 1 -- 172.21.15.7:0/3184996330 shutdown_connections 2024-04-25T11:14:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.887+0000 7f58fd000640 1 --2- 172.21.15.7:0/3184996330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58f815ca60 0x7f58f815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.887+0000 7f58fd000640 1 --2- 172.21.15.7:0/3184996330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f806d580 0x7f58f806d990 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.887+0000 7f58fd000640 1 -- 172.21.15.7:0/3184996330 >> 172.21.15.7:0/3184996330 conn(0x7f58f8073160 msgr2=0x7f58f8073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.887+0000 7f58fd000640 1 -- 172.21.15.7:0/3184996330 shutdown_connections 2024-04-25T11:14:52.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.887+0000 7f58fd000640 1 -- 172.21.15.7:0/3184996330 wait complete. 2024-04-25T11:14:52.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.890+0000 7f58fd000640 1 Processor -- start 2024-04-25T11:14:52.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.890+0000 7f58fd000640 1 -- start start 2024-04-25T11:14:52.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.892+0000 7f58fd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f806d580 0x7f58f80b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:52.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.892+0000 7f58fd000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58f80b4690 0x7f58f81550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.892+0000 7f58fd000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58f815ca60 0x7f58f81555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.892+0000 7f58fd000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81607a0 con 0x7f58f806d580 2024-04-25T11:14:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.892+0000 7f58fd000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f58f81604c0 con 0x7f58f80b4690 2024-04-25T11:14:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.892+0000 7f58fd000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f58f8160cc0 con 0x7f58f815ca60 2024-04-25T11:14:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.892+0000 7f58f7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f806d580 0x7f58f80b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.893+0000 7f58f7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f806d580 0x7f58f80b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39444/0 (socket says 172.21.15.7:39444) 2024-04-25T11:14:52.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.893+0000 7f58f7400640 1 -- 172.21.15.7:0/3521572036 learned_addr learned my addr 172.21.15.7:0/3521572036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.893+0000 7f58f7400640 1 -- 172.21.15.7:0/3521572036 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58f815ca60 msgr2=0x7f58f81555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.893+0000 7f58f7400640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58f815ca60 0x7f58f81555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.893+0000 7f58f7400640 1 -- 172.21.15.7:0/3521572036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58f80b4690 msgr2=0x7f58f81550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.893+0000 7f58f7400640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58f80b4690 0x7f58f81550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:52.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.893+0000 7f58f7400640 1 -- 172.21.15.7:0/3521572036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f8155ec0 con 0x7f58f806d580 2024-04-25T11:14:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.893+0000 7f58f7400640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f806d580 0x7f58f80b0730 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f58e800d270 tx=0x7f58e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.894+0000 7f58ef400640 1 -- 172.21.15.7:0/3521572036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e800b720 con 0x7f58f806d580 2024-04-25T11:14:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.894+0000 7f58ef400640 1 -- 172.21.15.7:0/3521572036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e8044070 con 0x7f58f806d580 2024-04-25T11:14:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.894+0000 7f58fd000640 1 -- 172.21.15.7:0/3521572036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f8154010 con 0x7f58f806d580 2024-04-25T11:14:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.894+0000 7f58ef400640 1 -- 172.21.15.7:0/3521572036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8008830 con 0x7f58f806d580 2024-04-25T11:14:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.894+0000 7f58fd000640 1 -- 172.21.15.7:0/3521572036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f81545d0 con 0x7f58f806d580 2024-04-25T11:14:52.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.895+0000 7f58fd000640 1 -- 172.21.15.7:0/3521572036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f8050740 con 0x7f58f806d580 2024-04-25T11:14:52.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.897+0000 7f58ef400640 1 -- 172.21.15.7:0/3521572036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f58e803e070 con 0x7f58f806d580 2024-04-25T11:14:52.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.898+0000 7f58ef400640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58d40744b0 0x7f58d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:52.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.898+0000 7f58ef400640 1 -- 172.21.15.7:0/3521572036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5749+0+0 (secure 0 0 0) 0x7f58e80b5df0 con 0x7f58f806d580 2024-04-25T11:14:52.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.898+0000 7f58f6a00640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58d40744b0 0x7f58d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:52.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.899+0000 7f58f6a00640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58d40744b0 0x7f58d40769c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f58e0006fd0 tx=0x7f58e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:52.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:52.903+0000 7f58ef400640 1 -- 172.21.15.7:0/3521572036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f58f806ded0 con 0x7f58f806d580 2024-04-25T11:14:53.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.107+0000 7f58fd000640 1 -- 172.21.15.7:0/3521572036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f58f8070720 con 0x7f58f806d580 2024-04-25T11:14:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.566+0000 7f58ef400640 1 -- 172.21.15.7:0/3521572036 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v71) ==== 169+0+0 (secure 0 0 0) 0x7f58e8080540 con 0x7f58f806d580 2024-04-25T11:14:53.568 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-25T11:14:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.570+0000 7f58fd000640 1 -- 172.21.15.7:0/3521572036 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58d40744b0 msgr2=0x7f58d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.570+0000 7f58fd000640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58d40744b0 0x7f58d40769c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f58e0006fd0 tx=0x7f58e0008040 comp rx=0 tx=0).stop 2024-04-25T11:14:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.570+0000 7f58fd000640 1 -- 172.21.15.7:0/3521572036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f806d580 msgr2=0x7f58f80b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.570+0000 7f58fd000640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f806d580 0x7f58f80b0730 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f58e800d270 tx=0x7f58e8004a40 comp rx=0 tx=0).stop 2024-04-25T11:14:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.571+0000 7f58fd000640 1 -- 172.21.15.7:0/3521572036 shutdown_connections 2024-04-25T11:14:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.571+0000 7f58fd000640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58d40744b0 0x7f58d40769c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.571+0000 7f58fd000640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58f815ca60 0x7f58f81555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.571+0000 7f58fd000640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58f80b4690 0x7f58f81550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.571+0000 7f58fd000640 1 --2- 172.21.15.7:0/3521572036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f806d580 0x7f58f80b0730 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.571+0000 7f58fd000640 1 -- 172.21.15.7:0/3521572036 >> 172.21.15.7:0/3521572036 conn(0x7f58f8073160 msgr2=0x7f58f806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.571+0000 7f58fd000640 1 -- 172.21.15.7:0/3521572036 shutdown_connections 2024-04-25T11:14:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.571+0000 7f58fd000640 1 -- 172.21.15.7:0/3521572036 wait complete. 2024-04-25T11:14:53.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:14:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.894+0000 7fb361400640 1 Processor -- start 2024-04-25T11:14:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.894+0000 7fb361400640 1 -- start start 2024-04-25T11:14:53.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.896+0000 7fb361400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb35c0cfa00 0x7fb35c15e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.896+0000 7fb361400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb35c09b540 0x7fb35c09b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.896+0000 7fb361400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35c09be90 con 0x7fb35c0b21d0 2024-04-25T11:14:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.896+0000 7fb361400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb35c0cffe0 con 0x7fb35c0cfa00 2024-04-25T11:14:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.896+0000 7fb361400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb35c0d02c0 con 0x7fb35c09b540 2024-04-25T11:14:53.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.897+0000 7fb35a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb35c0cfa00 0x7fb35c15e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.897+0000 7fb35b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb35c09b540 0x7fb35c09b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.897+0000 7fb35a800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb35c0cfa00 0x7fb35c15e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50912/0 (socket says 172.21.15.7:50912) 2024-04-25T11:14:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.897+0000 7fb35a800640 1 -- 172.21.15.7:0/3123852763 learned_addr learned my addr 172.21.15.7:0/3123852763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.898+0000 7fb35a800640 1 -- 172.21.15.7:0/3123852763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb35c09b540 msgr2=0x7fb35c09b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.898+0000 7fb35a800640 1 --2- 172.21.15.7:0/3123852763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb35c09b540 0x7fb35c09b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.898+0000 7fb35a800640 1 -- 172.21.15.7:0/3123852763 >> v1:172.21.15.87:6789/0 conn(0x7fb35c0b21d0 legacy=0x7fb35c157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:53.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.898+0000 7fb35a800640 1 -- 172.21.15.7:0/3123852763 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c15da20 con 0x7fb35c0cfa00 2024-04-25T11:14:53.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.899+0000 7fb35a800640 1 --2- 172.21.15.7:0/3123852763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb35c0cfa00 0x7fb35c15e220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009870 tx=0x7fb34c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9580c3b41b6012ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:53.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.900+0000 7fb359400640 1 -- 172.21.15.7:0/3123852763 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c037070 con 0x7fb35c0cfa00 2024-04-25T11:14:53.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.900+0000 7fb359400640 1 -- 172.21.15.7:0/3123852763 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34c032440 con 0x7fb35c0cfa00 2024-04-25T11:14:53.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.900+0000 7fb359400640 1 -- 172.21.15.7:0/3123852763 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c032810 con 0x7fb35c0cfa00 2024-04-25T11:14:53.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.901+0000 7fb361400640 1 -- 172.21.15.7:0/3123852763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb35c0cfa00 msgr2=0x7fb35c15e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:53.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.901+0000 7fb361400640 1 --2- 172.21.15.7:0/3123852763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb35c0cfa00 0x7fb35c15e220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009870 tx=0x7fb34c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:14:53.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.901+0000 7fb361400640 1 -- 172.21.15.7:0/3123852763 shutdown_connections 2024-04-25T11:14:53.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.901+0000 7fb361400640 1 --2- 172.21.15.7:0/3123852763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb35c09b540 0x7fb35c09b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:53.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.901+0000 7fb361400640 1 --2- 172.21.15.7:0/3123852763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb35c0cfa00 0x7fb35c15e220 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:53.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.901+0000 7fb361400640 1 -- 172.21.15.7:0/3123852763 >> 172.21.15.7:0/3123852763 conn(0x7fb35c0b7620 msgr2=0x7fb35c0b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:53.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.901+0000 7fb361400640 1 -- 172.21.15.7:0/3123852763 shutdown_connections 2024-04-25T11:14:53.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.901+0000 7fb361400640 1 -- 172.21.15.7:0/3123852763 wait complete. 2024-04-25T11:14:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.903+0000 7fb361400640 1 Processor -- start 2024-04-25T11:14:53.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.904+0000 7fb361400640 1 -- start start 2024-04-25T11:14:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.905+0000 7fb361400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb35c09b540 0x7fb35c16a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.905+0000 7fb361400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb35c0b21d0 0x7fb35c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.905+0000 7fb361400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb35c0cfa00 0x7fb35c169060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.905+0000 7fb361400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c0b2760 con 0x7fb35c09b540 2024-04-25T11:14:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.905+0000 7fb361400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb35c0b25e0 con 0x7fb35c0cfa00 2024-04-25T11:14:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.905+0000 7fb361400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb35c168050 con 0x7fb35c0b21d0 2024-04-25T11:14:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.905+0000 7fb35a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb35c09b540 0x7fb35c16a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.905+0000 7fb35a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb35c09b540 0x7fb35c16a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39446/0 (socket says 172.21.15.7:39446) 2024-04-25T11:14:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.905+0000 7fb35a800640 1 -- 172.21.15.7:0/2549540941 learned_addr learned my addr 172.21.15.7:0/2549540941 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.906+0000 7fb359e00640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb35c0b21d0 0x7fb35c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.906+0000 7fb35a800640 1 -- 172.21.15.7:0/2549540941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb35c0b21d0 msgr2=0x7fb35c15e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.906+0000 7fb35a800640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb35c0b21d0 0x7fb35c15e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.906+0000 7fb35a800640 1 -- 172.21.15.7:0/2549540941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb35c0cfa00 msgr2=0x7fb35c169060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.906+0000 7fb35a800640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb35c0cfa00 0x7fb35c169060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.906+0000 7fb35a800640 1 -- 172.21.15.7:0/2549540941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35c0b1c00 con 0x7fb35c09b540 2024-04-25T11:14:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.906+0000 7fb35a800640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb35c09b540 0x7fb35c16a490 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb34c000c00 tx=0x7fb34c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.907+0000 7fb352a00640 1 -- 172.21.15.7:0/2549540941 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c037040 con 0x7fb35c09b540 2024-04-25T11:14:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.907+0000 7fb352a00640 1 -- 172.21.15.7:0/2549540941 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34c042070 con 0x7fb35c09b540 2024-04-25T11:14:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.907+0000 7fb361400640 1 -- 172.21.15.7:0/2549540941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35c0b1ee0 con 0x7fb35c09b540 2024-04-25T11:14:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.907+0000 7fb352a00640 1 -- 172.21.15.7:0/2549540941 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c03b870 con 0x7fb35c09b540 2024-04-25T11:14:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.907+0000 7fb361400640 1 -- 172.21.15.7:0/2549540941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb35c0b0360 con 0x7fb35c09b540 2024-04-25T11:14:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.908+0000 7fb361400640 1 -- 172.21.15.7:0/2549540941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35c050740 con 0x7fb35c09b540 2024-04-25T11:14:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.909+0000 7fb352a00640 1 -- 172.21.15.7:0/2549540941 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb34c02ad10 con 0x7fb35c09b540 2024-04-25T11:14:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.910+0000 7fb352a00640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb33c0744e0 0x7fb33c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.910+0000 7fb352a00640 1 -- 172.21.15.7:0/2549540941 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7fb34c0b5660 con 0x7fb35c09b540 2024-04-25T11:14:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.910+0000 7fb359e00640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb33c0744e0 0x7fb33c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:53.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.911+0000 7fb359e00640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb33c0744e0 0x7fb33c0769f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb348007fd0 tx=0x7fb348009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:53.915+0000 7fb352a00640 1 -- 172.21.15.7:0/2549540941 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb34c07fed0 con 0x7fb35c09b540 2024-04-25T11:14:54.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.121+0000 7fb361400640 1 -- 172.21.15.7:0/2549540941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb35c1618b0 con 0x7fb35c09b540 2024-04-25T11:14:54.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.128+0000 7fb352a00640 1 -- 172.21.15.7:0/2549540941 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fb34c07f7d0 con 0x7fb35c09b540 2024-04-25T11:14:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.130+0000 7fb361400640 1 -- 172.21.15.7:0/2549540941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb33c0744e0 msgr2=0x7fb33c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.130+0000 7fb361400640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb33c0744e0 0x7fb33c0769f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb348007fd0 tx=0x7fb348009040 comp rx=0 tx=0).stop 2024-04-25T11:14:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.130+0000 7fb361400640 1 -- 172.21.15.7:0/2549540941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb35c09b540 msgr2=0x7fb35c16a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.130+0000 7fb361400640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb35c09b540 0x7fb35c16a490 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb34c000c00 tx=0x7fb34c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:14:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.131+0000 7fb361400640 1 -- 172.21.15.7:0/2549540941 shutdown_connections 2024-04-25T11:14:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.131+0000 7fb361400640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb33c0744e0 0x7fb33c0769f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.131+0000 7fb361400640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb35c0cfa00 0x7fb35c169060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.131+0000 7fb361400640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb35c0b21d0 0x7fb35c15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.131+0000 7fb361400640 1 --2- 172.21.15.7:0/2549540941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb35c09b540 0x7fb35c16a490 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.131+0000 7fb361400640 1 -- 172.21.15.7:0/2549540941 >> 172.21.15.7:0/2549540941 conn(0x7fb35c0b7620 msgr2=0x7fb35c15c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:54.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.131+0000 7fb361400640 1 -- 172.21.15.7:0/2549540941 shutdown_connections 2024-04-25T11:14:54.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.131+0000 7fb361400640 1 -- 172.21.15.7:0/2549540941 wait complete. 2024-04-25T11:14:54.168 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size' 2024-04-25T11:14:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.425+0000 7f555b800640 1 Processor -- start 2024-04-25T11:14:54.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.425+0000 7f555b800640 1 -- start start 2024-04-25T11:14:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.428+0000 7f555b800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f555415f180 0x7f5554161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.428+0000 7f555b800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f55541621a0 0x7f555406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.428+0000 7f555b800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f555406ded0 con 0x7f555406dac0 2024-04-25T11:14:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.428+0000 7f555b800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f555415f590 con 0x7f55541621a0 2024-04-25T11:14:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.428+0000 7f555b800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5554158ae0 con 0x7f555415f180 2024-04-25T11:14:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.429+0000 7f5553e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f55541621a0 0x7f555406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.429+0000 7f5553e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f55541621a0 0x7f555406d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39450/0 (socket says 172.21.15.7:39450) 2024-04-25T11:14:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.429+0000 7f5553e00640 1 -- 172.21.15.7:0/1061475612 learned_addr learned my addr 172.21.15.7:0/1061475612 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:54.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.429+0000 7f5558e00640 1 --2- 172.21.15.7:0/1061475612 >> v2:172.21.15.87:3300/0 conn(0x7f555415f180 0x7f5554161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.429+0000 7f5553e00640 1 -- 172.21.15.7:0/1061475612 >> v2:172.21.15.87:3300/0 conn(0x7f555415f180 msgr2=0x7f5554161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.429+0000 7f5553e00640 1 --2- 172.21.15.7:0/1061475612 >> v2:172.21.15.87:3300/0 conn(0x7f555415f180 0x7f5554161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.429+0000 7f5553e00640 1 -- 172.21.15.7:0/1061475612 >> v1:172.21.15.87:6789/0 conn(0x7f555406dac0 legacy=0x7f555406f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.429+0000 7f5553e00640 1 -- 172.21.15.7:0/1061475612 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5554162700 con 0x7f55541621a0 2024-04-25T11:14:54.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.430+0000 7f5553e00640 1 --2- 172.21.15.7:0/1061475612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55541621a0 0x7f555406d580 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5554070340 tx=0x7f554402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fef96e295968c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.431+0000 7f5553400640 1 -- 172.21.15.7:0/1061475612 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5544036070 con 0x7f55541621a0 2024-04-25T11:14:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.431+0000 7f5553400640 1 -- 172.21.15.7:0/1061475612 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5544031600 con 0x7f55541621a0 2024-04-25T11:14:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.431+0000 7f5553400640 1 -- 172.21.15.7:0/1061475612 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554402ae60 con 0x7f55541621a0 2024-04-25T11:14:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.431+0000 7f555b800640 1 -- 172.21.15.7:0/1061475612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55541621a0 msgr2=0x7f555406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.431+0000 7f555b800640 1 --2- 172.21.15.7:0/1061475612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55541621a0 0x7f555406d580 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5554070340 tx=0x7f554402a680 comp rx=0 tx=0).stop 2024-04-25T11:14:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.432+0000 7f555b800640 1 -- 172.21.15.7:0/1061475612 shutdown_connections 2024-04-25T11:14:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.432+0000 7f555b800640 1 --2- 172.21.15.7:0/1061475612 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55541621a0 0x7f555406d580 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.432+0000 7f555b800640 1 --2- 172.21.15.7:0/1061475612 >> v2:172.21.15.87:3300/0 conn(0x7f555415f180 0x7f5554161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.432+0000 7f555b800640 1 -- 172.21.15.7:0/1061475612 >> 172.21.15.7:0/1061475612 conn(0x7f55540732b0 msgr2=0x7f55540736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.432+0000 7f555b800640 1 -- 172.21.15.7:0/1061475612 shutdown_connections 2024-04-25T11:14:54.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.432+0000 7f555b800640 1 -- 172.21.15.7:0/1061475612 wait complete. 2024-04-25T11:14:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.434+0000 7f555b800640 1 Processor -- start 2024-04-25T11:14:54.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.434+0000 7f555b800640 1 -- start start 2024-04-25T11:14:54.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.436+0000 7f555b800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f555406dac0 0x7f555409b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:54.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.436+0000 7f555b800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f555415f180 0x7f55540b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:54.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.436+0000 7f555b800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55541621a0 0x7f555409e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:54.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.436+0000 7f555b800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f555415ec60 con 0x7f55541621a0 2024-04-25T11:14:54.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.436+0000 7f555b800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f555415e980 con 0x7f555415f180 2024-04-25T11:14:54.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.436+0000 7f555b800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f555415e670 con 0x7f555406dac0 2024-04-25T11:14:54.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.436+0000 7f5559800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55541621a0 0x7f555409e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:54.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.436+0000 7f5559800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55541621a0 0x7f555409e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39464/0 (socket says 172.21.15.7:39464) 2024-04-25T11:14:54.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.436+0000 7f5559800640 1 -- 172.21.15.7:0/3212144921 learned_addr learned my addr 172.21.15.7:0/3212144921 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:54.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.436+0000 7f5558e00640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f555406dac0 0x7f555409b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:54.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f5553e00640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f555415f180 0x7f55540b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:54.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f5559800640 1 -- 172.21.15.7:0/3212144921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f555406dac0 msgr2=0x7f555409b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:54.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f5559800640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f555406dac0 0x7f555409b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:54.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f5559800640 1 -- 172.21.15.7:0/3212144921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f555415f180 msgr2=0x7f55540b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:54.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f5559800640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f555415f180 0x7f55540b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:54.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f5559800640 1 -- 172.21.15.7:0/3212144921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f555409bd20 con 0x7f55541621a0 2024-04-25T11:14:54.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f5553e00640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f555415f180 0x7f55540b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:54.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f5559800640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55541621a0 0x7f555409e2f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f554c00b470 tx=0x7f554c00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:54.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f5550c00640 1 -- 172.21.15.7:0/3212144921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c015020 con 0x7f55541621a0 2024-04-25T11:14:54.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f5550c00640 1 -- 172.21.15.7:0/3212144921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554c011830 con 0x7f55541621a0 2024-04-25T11:14:54.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f5550c00640 1 -- 172.21.15.7:0/3212144921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c011c20 con 0x7f55541621a0 2024-04-25T11:14:54.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.437+0000 7f555b800640 1 -- 172.21.15.7:0/3212144921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55541e7fb0 con 0x7f55541621a0 2024-04-25T11:14:54.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.438+0000 7f555b800640 1 -- 172.21.15.7:0/3212144921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55541e8570 con 0x7f55541621a0 2024-04-25T11:14:54.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.439+0000 7f555b800640 1 -- 172.21.15.7:0/3212144921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5554050740 con 0x7f55541621a0 2024-04-25T11:14:54.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.440+0000 7f5550c00640 1 -- 172.21.15.7:0/3212144921 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f554c018020 con 0x7f55541621a0 2024-04-25T11:14:54.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.441+0000 7f5550c00640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55300744e0 0x7f55300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:54.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.441+0000 7f5550c00640 1 -- 172.21.15.7:0/3212144921 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7f554c094f00 con 0x7f55541621a0 2024-04-25T11:14:54.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.441+0000 7f5558e00640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55300744e0 0x7f55300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:54.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.442+0000 7f5558e00640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55300744e0 0x7f55300769f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f553c006fd0 tx=0x7f553c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:54.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.446+0000 7f5550c00640 1 -- 172.21.15.7:0/3212144921 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f554c01f020 con 0x7f55541621a0 2024-04-25T11:14:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:54.733+0000 7f555b800640 1 -- 172.21.15.7:0/3212144921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]} v 0) -- 0x7f5554074250 con 0x7f55541621a0 2024-04-25T11:14:55.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.581+0000 7f5550c00640 1 -- 172.21.15.7:0/3212144921 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_cache_size"]}]=0 v255) ==== 150+0+0 (secure 0 0 0) 0x7f555409bd20 con 0x7f55541621a0 2024-04-25T11:14:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.585+0000 7f555b800640 1 -- 172.21.15.7:0/3212144921 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55300744e0 msgr2=0x7f55300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.585+0000 7f555b800640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55300744e0 0x7f55300769f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f553c006fd0 tx=0x7f553c008040 comp rx=0 tx=0).stop 2024-04-25T11:14:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.585+0000 7f555b800640 1 -- 172.21.15.7:0/3212144921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55541621a0 msgr2=0x7f555409e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.585+0000 7f555b800640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55541621a0 0x7f555409e2f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f554c00b470 tx=0x7f554c00b930 comp rx=0 tx=0).stop 2024-04-25T11:14:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.585+0000 7f555b800640 1 -- 172.21.15.7:0/3212144921 shutdown_connections 2024-04-25T11:14:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.585+0000 7f555b800640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f55300744e0 0x7f55300769f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:55.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.585+0000 7f555b800640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55541621a0 0x7f555409e2f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:55.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.585+0000 7f555b800640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f555415f180 0x7f55540b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:55.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.585+0000 7f555b800640 1 --2- 172.21.15.7:0/3212144921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f555406dac0 0x7f555409b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:55.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.585+0000 7f555b800640 1 -- 172.21.15.7:0/3212144921 >> 172.21.15.7:0/3212144921 conn(0x7f55540732b0 msgr2=0x7f5554161790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:55.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.586+0000 7f555b800640 1 -- 172.21.15.7:0/3212144921 shutdown_connections 2024-04-25T11:14:55.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.586+0000 7f555b800640 1 -- 172.21.15.7:0/3212144921 wait complete. 2024-04-25T11:14:55.618 INFO:tasks.cephfs_test_runner:test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits) ... skipped 'Require FUSE client to inject client release failure' 2024-04-25T11:14:55.619 INFO:tasks.cephfs_test_runner:Starting test: test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) That the MDS will not let a client sit above mds_max_caps_per_client caps. 2024-04-25T11:14:55.619 INFO:tasks.cephfs_test_runner:test_client_max_caps (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-25T11:14:55.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:14:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.906+0000 7f521ae00640 1 Processor -- start 2024-04-25T11:14:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.906+0000 7f521ae00640 1 -- start start 2024-04-25T11:14:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.908+0000 7f521ae00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f521415ef90 0x7f521409f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.909+0000 7f521ae00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f521415bc40 0x7f52140aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.909+0000 7f521ae00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f521415f4e0 con 0x7f52140af370 2024-04-25T11:14:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.909+0000 7f521ae00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5214071190 con 0x7f521415ef90 2024-04-25T11:14:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.909+0000 7f521ae00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f52140af780 con 0x7f521415bc40 2024-04-25T11:14:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.909+0000 7f5213e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f521415ef90 0x7f521409f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.909+0000 7f5218e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f52140af370 0x7f5214161b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43236/0 (socket says 172.21.15.7:43236) 2024-04-25T11:14:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.909+0000 7f5218e00640 1 -- 172.21.15.7:0/3481670852 learned_addr learned my addr 172.21.15.7:0/3481670852 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.909+0000 7f5213400640 1 --2- 172.21.15.7:0/3481670852 >> v2:172.21.15.87:3300/0 conn(0x7f521415bc40 0x7f52140aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.910+0000 7f5213e00640 1 -- 172.21.15.7:0/3481670852 >> v2:172.21.15.87:3300/0 conn(0x7f521415bc40 msgr2=0x7f52140aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.910+0000 7f5213e00640 1 --2- 172.21.15.7:0/3481670852 >> v2:172.21.15.87:3300/0 conn(0x7f521415bc40 0x7f52140aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:55.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.910+0000 7f5213e00640 1 -- 172.21.15.7:0/3481670852 >> v1:172.21.15.7:6789/0 conn(0x7f52140af370 legacy=0x7f5214161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:55.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.910+0000 7f5213e00640 1 -- 172.21.15.7:0/3481670852 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52140b0560 con 0x7f521415ef90 2024-04-25T11:14:55.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.910+0000 7f5212a00640 1 -- 172.21.15.7:0/3481670852 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032065707 0 0) 0x7f521415f4e0 con 0x7f52140af370 2024-04-25T11:14:55.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.911+0000 7f5213e00640 1 --2- 172.21.15.7:0/3481670852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521415ef90 0x7f521409f790 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f521406f2b0 tx=0x7f520402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b630df6dafe8c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:55.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.911+0000 7f5212a00640 1 -- 172.21.15.7:0/3481670852 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204037070 con 0x7f521415ef90 2024-04-25T11:14:55.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.911+0000 7f5212a00640 1 -- 172.21.15.7:0/3481670852 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5204032440 con 0x7f521415ef90 2024-04-25T11:14:55.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.911+0000 7f5212a00640 1 -- 172.21.15.7:0/3481670852 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520402ae60 con 0x7f521415ef90 2024-04-25T11:14:55.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.912+0000 7f521ae00640 1 -- 172.21.15.7:0/3481670852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521415ef90 msgr2=0x7f521409f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:55.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.912+0000 7f521ae00640 1 --2- 172.21.15.7:0/3481670852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521415ef90 0x7f521409f790 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f521406f2b0 tx=0x7f520402a110 comp rx=0 tx=0).stop 2024-04-25T11:14:55.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.912+0000 7f521ae00640 1 -- 172.21.15.7:0/3481670852 shutdown_connections 2024-04-25T11:14:55.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.912+0000 7f521ae00640 1 --2- 172.21.15.7:0/3481670852 >> v2:172.21.15.87:3300/0 conn(0x7f521415bc40 0x7f52140aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.912+0000 7f521ae00640 1 --2- 172.21.15.7:0/3481670852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521415ef90 0x7f521409f790 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.912+0000 7f521ae00640 1 -- 172.21.15.7:0/3481670852 >> 172.21.15.7:0/3481670852 conn(0x7f5214074200 msgr2=0x7f521406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.912+0000 7f521ae00640 1 -- 172.21.15.7:0/3481670852 shutdown_connections 2024-04-25T11:14:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.912+0000 7f521ae00640 1 -- 172.21.15.7:0/3481670852 wait complete. 2024-04-25T11:14:55.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.915+0000 7f521ae00640 1 Processor -- start 2024-04-25T11:14:55.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.915+0000 7f521ae00640 1 -- start start 2024-04-25T11:14:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.916+0000 7f521ae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52140af370 0x7f5214147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.916+0000 7f521ae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521415bc40 0x7f5214156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.916+0000 7f521ae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f521415ef90 0x7f5214160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.916+0000 7f521ae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52140b0e00 con 0x7f521415bc40 2024-04-25T11:14:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.916+0000 7f521ae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f52140b0b20 con 0x7f521415ef90 2024-04-25T11:14:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.916+0000 7f521ae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f52140b1320 con 0x7f52140af370 2024-04-25T11:14:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.917+0000 7f5213400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521415bc40 0x7f5214156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.917+0000 7f5213400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521415bc40 0x7f5214156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39470/0 (socket says 172.21.15.7:39470) 2024-04-25T11:14:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.917+0000 7f5213400640 1 -- 172.21.15.7:0/2891506473 learned_addr learned my addr 172.21.15.7:0/2891506473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:55.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.917+0000 7f5218e00640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f521415ef90 0x7f5214160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.917+0000 7f5213400640 1 -- 172.21.15.7:0/2891506473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52140af370 msgr2=0x7f5214147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.917+0000 7f5213400640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52140af370 0x7f5214147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.917+0000 7f5213400640 1 -- 172.21.15.7:0/2891506473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f521415ef90 msgr2=0x7f5214160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.917+0000 7f5213400640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f521415ef90 0x7f5214160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.917+0000 7f5213400640 1 -- 172.21.15.7:0/2891506473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52141f7c30 con 0x7f521415bc40 2024-04-25T11:14:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.918+0000 7f5213400640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521415bc40 0x7f5214156d10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f520000d620 tx=0x7f520000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.918+0000 7f520be00640 1 -- 172.21.15.7:0/2891506473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520000b620 con 0x7f521415bc40 2024-04-25T11:14:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.918+0000 7f520be00640 1 -- 172.21.15.7:0/2891506473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5200019070 con 0x7f521415bc40 2024-04-25T11:14:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.918+0000 7f520be00640 1 -- 172.21.15.7:0/2891506473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520000ba00 con 0x7f521415bc40 2024-04-25T11:14:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.918+0000 7f521ae00640 1 -- 172.21.15.7:0/2891506473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52141f7f10 con 0x7f521415bc40 2024-04-25T11:14:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.918+0000 7f521ae00640 1 -- 172.21.15.7:0/2891506473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52141f84d0 con 0x7f521415bc40 2024-04-25T11:14:55.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.920+0000 7f521ae00640 1 -- 172.21.15.7:0/2891506473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f521406a100 con 0x7f521415bc40 2024-04-25T11:14:55.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.921+0000 7f520be00640 1 -- 172.21.15.7:0/2891506473 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f520000bba0 con 0x7f521415bc40 2024-04-25T11:14:55.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.922+0000 7f520be00640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51f40744b0 0x7f51f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:55.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.923+0000 7f520be00640 1 -- 172.21.15.7:0/2891506473 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7f52000954b0 con 0x7f521415bc40 2024-04-25T11:14:55.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.923+0000 7f5213e00640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51f40744b0 0x7f51f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:55.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.924+0000 7f5213e00640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51f40744b0 0x7f51f40769c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5204034000 tx=0x7f520402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:55.928+0000 7f520be00640 1 -- 172.21.15.7:0/2891506473 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f520005fe40 con 0x7f521415bc40 2024-04-25T11:14:56.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.133+0000 7f521ae00640 1 -- 172.21.15.7:0/2891506473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f521415e0f0 con 0x7f521415bc40 2024-04-25T11:14:56.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.135+0000 7f520be00640 1 -- 172.21.15.7:0/2891506473 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+13196 (secure 0 0 0) 0x7f520005f740 con 0x7f521415bc40 2024-04-25T11:14:56.137 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:56.137 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":71,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:14:53.554645+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6837/2403088100":"2024-04-26T11:14:50.168218+0000","172.21.15.7:6835/2403088100":"2024-04-26T11:14:50.168218+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.138+0000 7f521ae00640 1 -- 172.21.15.7:0/2891506473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51f40744b0 msgr2=0x7f51f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.138+0000 7f521ae00640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51f40744b0 0x7f51f40769c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5204034000 tx=0x7f520402c000 comp rx=0 tx=0).stop 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.138+0000 7f521ae00640 1 -- 172.21.15.7:0/2891506473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521415bc40 msgr2=0x7f5214156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.138+0000 7f521ae00640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521415bc40 0x7f5214156d10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f520000d620 tx=0x7f520000dae0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.138+0000 7f521ae00640 1 -- 172.21.15.7:0/2891506473 shutdown_connections 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.138+0000 7f521ae00640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51f40744b0 0x7f51f40769c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.138+0000 7f521ae00640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f521415ef90 0x7f5214160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.138+0000 7f521ae00640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521415bc40 0x7f5214156d10 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.138+0000 7f521ae00640 1 --2- 172.21.15.7:0/2891506473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52140af370 0x7f5214147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.138+0000 7f521ae00640 1 -- 172.21.15.7:0/2891506473 >> 172.21.15.7:0/2891506473 conn(0x7f5214074200 msgr2=0x7f52141621e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.139+0000 7f521ae00640 1 -- 172.21.15.7:0/2891506473 shutdown_connections 2024-04-25T11:14:56.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.139+0000 7f521ae00640 1 -- 172.21.15.7:0/2891506473 wait complete. 2024-04-25T11:14:56.168 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:14:56.168 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:14:56.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.458+0000 7fb5c7e00640 1 Processor -- start 2024-04-25T11:14:56.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.458+0000 7fb5c7e00640 1 -- start start 2024-04-25T11:14:56.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.460+0000 7fb5c7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5c81583c0 0x7fb5c80cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:56.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.461+0000 7fb5c7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5c806eea0 0x7fb5c806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:56.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.461+0000 7fb5c7e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c80ce570 con 0x7fb5c80cfb50 2024-04-25T11:14:56.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.461+0000 7fb5c7e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c806f7f0 con 0x7fb5c81583c0 2024-04-25T11:14:56.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.461+0000 7fb5c7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c815f190 con 0x7fb5c806eea0 2024-04-25T11:14:56.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.461+0000 7fb5c6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5c81583c0 0x7fb5c80cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.461+0000 7fb5c7400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5c806eea0 0x7fb5c806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.461+0000 7fb5c6a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb5c81583c0 0x7fb5c80cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50968/0 (socket says 172.21.15.7:50968) 2024-04-25T11:14:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.461+0000 7fb5c7400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5c806eea0 0x7fb5c806f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:36158/0 (socket says 172.21.15.7:36158) 2024-04-25T11:14:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.461+0000 7fb5c6a00640 1 -- 172.21.15.7:0/1217738335 learned_addr learned my addr 172.21.15.7:0/1217738335 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.462+0000 7fb5c6a00640 1 -- 172.21.15.7:0/1217738335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5c806eea0 msgr2=0x7fb5c806f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.462+0000 7fb5c6a00640 1 --2- 172.21.15.7:0/1217738335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5c806eea0 0x7fb5c806f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.462+0000 7fb5c6a00640 1 -- 172.21.15.7:0/1217738335 >> v1:172.21.15.87:6789/0 conn(0x7fb5c80cfb50 legacy=0x7fb5c80d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.462+0000 7fb5c6a00640 1 -- 172.21.15.7:0/1217738335 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c80ce850 con 0x7fb5c81583c0 2024-04-25T11:14:56.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.464+0000 7fb5c6a00640 1 --2- 172.21.15.7:0/1217738335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5c81583c0 0x7fb5c80cf610 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009870 tx=0x7fb5b802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=695ab54bdf000048 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.464+0000 7fb5c6000640 1 -- 172.21.15.7:0/1217738335 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8037070 con 0x7fb5c81583c0 2024-04-25T11:14:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.465+0000 7fb5c6000640 1 -- 172.21.15.7:0/1217738335 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b8032440 con 0x7fb5c81583c0 2024-04-25T11:14:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.465+0000 7fb5c6000640 1 -- 172.21.15.7:0/1217738335 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8032810 con 0x7fb5c81583c0 2024-04-25T11:14:56.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.465+0000 7fb5c7e00640 1 -- 172.21.15.7:0/1217738335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5c81583c0 msgr2=0x7fb5c80cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:56.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.465+0000 7fb5c7e00640 1 --2- 172.21.15.7:0/1217738335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5c81583c0 0x7fb5c80cf610 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009870 tx=0x7fb5b802a210 comp rx=0 tx=0).stop 2024-04-25T11:14:56.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.466+0000 7fb5c7e00640 1 -- 172.21.15.7:0/1217738335 shutdown_connections 2024-04-25T11:14:56.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.466+0000 7fb5c7e00640 1 --2- 172.21.15.7:0/1217738335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5c806eea0 0x7fb5c806f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.466+0000 7fb5c7e00640 1 --2- 172.21.15.7:0/1217738335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5c81583c0 0x7fb5c80cf610 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.466+0000 7fb5c7e00640 1 -- 172.21.15.7:0/1217738335 >> 172.21.15.7:0/1217738335 conn(0x7fb5c80740f0 msgr2=0x7fb5c806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:56.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.466+0000 7fb5c7e00640 1 -- 172.21.15.7:0/1217738335 shutdown_connections 2024-04-25T11:14:56.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.466+0000 7fb5c7e00640 1 -- 172.21.15.7:0/1217738335 wait complete. 2024-04-25T11:14:56.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.469+0000 7fb5c7e00640 1 Processor -- start 2024-04-25T11:14:56.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.469+0000 7fb5c7e00640 1 -- start start 2024-04-25T11:14:56.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5c806eea0 0x7fb5c815f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:56.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5c80cfb50 0x7fb5c81e9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:56.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5c81583c0 0x7fb5c81615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:56.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c80cf0f0 con 0x7fb5c806eea0 2024-04-25T11:14:56.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c80cee10 con 0x7fb5c81583c0 2024-04-25T11:14:56.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c80d0b90 con 0x7fb5c80cfb50 2024-04-25T11:14:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5c806eea0 0x7fb5c815f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5c806eea0 0x7fb5c815f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39474/0 (socket says 172.21.15.7:39474) 2024-04-25T11:14:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c6a00640 1 -- 172.21.15.7:0/3554643285 learned_addr learned my addr 172.21.15.7:0/3554643285 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c6a00640 1 -- 172.21.15.7:0/3554643285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5c80cfb50 msgr2=0x7fb5c81e9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c6a00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5c80cfb50 0x7fb5c81e9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c6a00640 1 -- 172.21.15.7:0/3554643285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5c81583c0 msgr2=0x7fb5c81615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c6a00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5c81583c0 0x7fb5c81615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.472+0000 7fb5c6a00640 1 -- 172.21.15.7:0/3554643285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c815fe10 con 0x7fb5c806eea0 2024-04-25T11:14:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.473+0000 7fb5c6a00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5c806eea0 0x7fb5c815f350 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009870 tx=0x7fb5b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.473+0000 7fb5bea00640 1 -- 172.21.15.7:0/3554643285 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8041020 con 0x7fb5c806eea0 2024-04-25T11:14:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.473+0000 7fb5bea00640 1 -- 172.21.15.7:0/3554643285 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b803d620 con 0x7fb5c806eea0 2024-04-25T11:14:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.473+0000 7fb5bea00640 1 -- 172.21.15.7:0/3554643285 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8037040 con 0x7fb5c806eea0 2024-04-25T11:14:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.473+0000 7fb5c7e00640 1 -- 172.21.15.7:0/3554643285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c81600f0 con 0x7fb5c806eea0 2024-04-25T11:14:56.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.473+0000 7fb5c7e00640 1 -- 172.21.15.7:0/3554643285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c814f3d0 con 0x7fb5c806eea0 2024-04-25T11:14:56.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.475+0000 7fb5c7e00640 1 -- 172.21.15.7:0/3554643285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c8050740 con 0x7fb5c806eea0 2024-04-25T11:14:56.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.476+0000 7fb5bea00640 1 -- 172.21.15.7:0/3554643285 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb5b8044020 con 0x7fb5c806eea0 2024-04-25T11:14:56.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.477+0000 7fb5bea00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5a40744e0 0x7fb5a40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:56.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.477+0000 7fb5bea00640 1 -- 172.21.15.7:0/3554643285 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7fb5b80b57d0 con 0x7fb5c806eea0 2024-04-25T11:14:56.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.477+0000 7fb5bfe00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5a40744e0 0x7fb5a40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:56.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.478+0000 7fb5bfe00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5a40744e0 0x7fb5a40769f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0006fd0 tx=0x7fb5b0006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:56.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.483+0000 7fb5bea00640 1 -- 172.21.15.7:0/3554643285 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb5b804b030 con 0x7fb5c806eea0 2024-04-25T11:14:56.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.687+0000 7fb5c7e00640 1 -- 172.21.15.7:0/3554643285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb5c8070dd0 con 0x7fb5c806eea0 2024-04-25T11:14:56.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.688+0000 7fb5bea00640 1 -- 172.21.15.7:0/3554643285 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fb5b8080160 con 0x7fb5c806eea0 2024-04-25T11:14:56.689 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:14:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 -- 172.21.15.7:0/3554643285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5a40744e0 msgr2=0x7fb5a40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5a40744e0 0x7fb5a40769f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0006fd0 tx=0x7fb5b0006bc0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 -- 172.21.15.7:0/3554643285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5c806eea0 msgr2=0x7fb5c815f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:56.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5c806eea0 0x7fb5c815f350 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009870 tx=0x7fb5b80046c0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 -- 172.21.15.7:0/3554643285 shutdown_connections 2024-04-25T11:14:56.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5a40744e0 0x7fb5a40769f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5c81583c0 0x7fb5c81615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5c80cfb50 0x7fb5c81e9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 --2- 172.21.15.7:0/3554643285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5c806eea0 0x7fb5c815f350 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 -- 172.21.15.7:0/3554643285 >> 172.21.15.7:0/3554643285 conn(0x7fb5c80740f0 msgr2=0x7fb5c806c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:56.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 -- 172.21.15.7:0/3554643285 shutdown_connections 2024-04-25T11:14:56.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.693+0000 7fb5c7e00640 1 -- 172.21.15.7:0/3554643285 wait complete. 2024-04-25T11:14:56.719 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2024-04-25T11:14:56.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.988+0000 7fb36f200640 1 Processor -- start 2024-04-25T11:14:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.988+0000 7fb36f200640 1 -- start start 2024-04-25T11:14:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.991+0000 7fb36f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb370070810 0x7fb37006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.991+0000 7fb36f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb37006dac0 0x7fb37015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.991+0000 7fb36f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3700cffa0 con 0x7fb370070c20 2024-04-25T11:14:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.991+0000 7fb36f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb37006ded0 con 0x7fb37006dac0 2024-04-25T11:14:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.991+0000 7fb36f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb370160d90 con 0x7fb370070810 2024-04-25T11:14:56.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.992+0000 7fb36de00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb370070810 0x7fb37006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:56.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.992+0000 7fb36e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb37006dac0 0x7fb37015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:56.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.992+0000 7fb36e800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb37006dac0 0x7fb37015d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:50978/0 (socket says 172.21.15.7:50978) 2024-04-25T11:14:56.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.992+0000 7fb36de00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb370070810 0x7fb37006d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:36176/0 (socket says 172.21.15.7:36176) 2024-04-25T11:14:56.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.992+0000 7fb36e800640 1 -- 172.21.15.7:0/1085133794 learned_addr learned my addr 172.21.15.7:0/1085133794 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:56.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.993+0000 7fb36e800640 1 -- 172.21.15.7:0/1085133794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb370070810 msgr2=0x7fb37006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:56.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.993+0000 7fb36e800640 1 --2- 172.21.15.7:0/1085133794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb370070810 0x7fb37006d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.993+0000 7fb36e800640 1 -- 172.21.15.7:0/1085133794 >> v1:172.21.15.87:6789/0 conn(0x7fb370070c20 legacy=0x7fb37015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:56.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.993+0000 7fb36e800640 1 -- 172.21.15.7:0/1085133794 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb370161070 con 0x7fb37006dac0 2024-04-25T11:14:56.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.994+0000 7fb36e800640 1 --2- 172.21.15.7:0/1085133794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb37006dac0 0x7fb37015d9e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb368009e50 tx=0x7fb36802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa9e503574fcf4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:56.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.994+0000 7fb36ca00640 1 -- 172.21.15.7:0/1085133794 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368036070 con 0x7fb37006dac0 2024-04-25T11:14:56.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.994+0000 7fb36ca00640 1 -- 172.21.15.7:0/1085133794 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb368031600 con 0x7fb37006dac0 2024-04-25T11:14:56.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.994+0000 7fb36ca00640 1 -- 172.21.15.7:0/1085133794 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3680319d0 con 0x7fb37006dac0 2024-04-25T11:14:56.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.995+0000 7fb36f200640 1 -- 172.21.15.7:0/1085133794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb37006dac0 msgr2=0x7fb37015d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:56.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.995+0000 7fb36f200640 1 --2- 172.21.15.7:0/1085133794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb37006dac0 0x7fb37015d9e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb368009e50 tx=0x7fb36802a710 comp rx=0 tx=0).stop 2024-04-25T11:14:56.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.995+0000 7fb36f200640 1 -- 172.21.15.7:0/1085133794 shutdown_connections 2024-04-25T11:14:56.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.995+0000 7fb36f200640 1 --2- 172.21.15.7:0/1085133794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb37006dac0 0x7fb37015d9e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.995+0000 7fb36f200640 1 --2- 172.21.15.7:0/1085133794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb370070810 0x7fb37006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:56.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.995+0000 7fb36f200640 1 -- 172.21.15.7:0/1085133794 >> 172.21.15.7:0/1085133794 conn(0x7fb370073090 msgr2=0x7fb3700734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:56.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.995+0000 7fb36f200640 1 -- 172.21.15.7:0/1085133794 shutdown_connections 2024-04-25T11:14:56.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.995+0000 7fb36f200640 1 -- 172.21.15.7:0/1085133794 wait complete. 2024-04-25T11:14:56.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.998+0000 7fb36f200640 1 Processor -- start 2024-04-25T11:14:56.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:56.999+0000 7fb36f200640 1 -- start start 2024-04-25T11:14:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.000+0000 7fb36f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb37006dac0 0x7fb37015f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.000+0000 7fb36f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb370070810 0x7fb370141850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.000+0000 7fb36f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb370070c20 0x7fb370143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.000+0000 7fb36f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3701433b0 con 0x7fb370070c20 2024-04-25T11:14:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.000+0000 7fb36f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb370161630 con 0x7fb37006dac0 2024-04-25T11:14:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.000+0000 7fb36f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb370143530 con 0x7fb370070810 2024-04-25T11:14:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.001+0000 7fb36e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb370070c20 0x7fb370143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.001+0000 7fb36e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb370070c20 0x7fb370143710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39484/0 (socket says 172.21.15.7:39484) 2024-04-25T11:14:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.001+0000 7fb36e800640 1 -- 172.21.15.7:0/3663626396 learned_addr learned my addr 172.21.15.7:0/3663626396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.001+0000 7fb36de00640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb37006dac0 0x7fb37015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.001+0000 7fb36d400640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb370070810 0x7fb370141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.001+0000 7fb36e800640 1 -- 172.21.15.7:0/3663626396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb370070810 msgr2=0x7fb370141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.001+0000 7fb36e800640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb370070810 0x7fb370141850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.001+0000 7fb36e800640 1 -- 172.21.15.7:0/3663626396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb37006dac0 msgr2=0x7fb37015f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.001+0000 7fb36e800640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb37006dac0 0x7fb37015f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.001+0000 7fb36e800640 1 -- 172.21.15.7:0/3663626396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37015e380 con 0x7fb370070c20 2024-04-25T11:14:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.002+0000 7fb36e800640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb370070c20 0x7fb370143710 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb368009f70 tx=0x7fb368004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.002+0000 7fb366000640 1 -- 172.21.15.7:0/3663626396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36803f020 con 0x7fb370070c20 2024-04-25T11:14:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.002+0000 7fb36f200640 1 -- 172.21.15.7:0/3663626396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37015e660 con 0x7fb370070c20 2024-04-25T11:14:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.002+0000 7fb366000640 1 -- 172.21.15.7:0/3663626396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb368037040 con 0x7fb370070c20 2024-04-25T11:14:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.002+0000 7fb366000640 1 -- 172.21.15.7:0/3663626396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368036070 con 0x7fb370070c20 2024-04-25T11:14:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.002+0000 7fb36f200640 1 -- 172.21.15.7:0/3663626396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb370141d90 con 0x7fb370070c20 2024-04-25T11:14:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.003+0000 7fb36f200640 1 -- 172.21.15.7:0/3663626396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb370050740 con 0x7fb370070c20 2024-04-25T11:14:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.005+0000 7fb366000640 1 -- 172.21.15.7:0/3663626396 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb368042020 con 0x7fb370070c20 2024-04-25T11:14:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.006+0000 7fb366000640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3500744e0 0x7fb3500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.006+0000 7fb366000640 1 -- 172.21.15.7:0/3663626396 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7fb3680b5630 con 0x7fb370070c20 2024-04-25T11:14:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.006+0000 7fb36de00640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3500744e0 0x7fb3500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:57.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.007+0000 7fb36de00640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3500744e0 0x7fb3500769f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb360009a30 tx=0x7fb360008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:57.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.013+0000 7fb366000640 1 -- 172.21.15.7:0/3663626396 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb368049030 con 0x7fb370070c20 2024-04-25T11:14:57.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.283+0000 7fb36f200640 1 -- 172.21.15.7:0/3663626396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) -- 0x7fb370142c60 con 0x7fb370070c20 2024-04-25T11:14:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.596+0000 7fb366000640 1 -- 172.21.15.7:0/3663626396 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v257) ==== 151+0+0 (secure 0 0 0) 0x7fb36807fe10 con 0x7fb370070c20 2024-04-25T11:14:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.600+0000 7fb36f200640 1 -- 172.21.15.7:0/3663626396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3500744e0 msgr2=0x7fb3500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.600+0000 7fb36f200640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3500744e0 0x7fb3500769f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb360009a30 tx=0x7fb360008040 comp rx=0 tx=0).stop 2024-04-25T11:14:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.600+0000 7fb36f200640 1 -- 172.21.15.7:0/3663626396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb370070c20 msgr2=0x7fb370143710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.600+0000 7fb36f200640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb370070c20 0x7fb370143710 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb368009f70 tx=0x7fb368004a40 comp rx=0 tx=0).stop 2024-04-25T11:14:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.600+0000 7fb36f200640 1 -- 172.21.15.7:0/3663626396 shutdown_connections 2024-04-25T11:14:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.600+0000 7fb36f200640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3500744e0 0x7fb3500769f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.600+0000 7fb36f200640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb370070c20 0x7fb370143710 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.600+0000 7fb36f200640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb370070810 0x7fb370141850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.600+0000 7fb36f200640 1 --2- 172.21.15.7:0/3663626396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb37006dac0 0x7fb37015f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.600+0000 7fb36f200640 1 -- 172.21.15.7:0/3663626396 >> 172.21.15.7:0/3663626396 conn(0x7fb370073090 msgr2=0x7fb37006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.601+0000 7fb36f200640 1 -- 172.21.15.7:0/3663626396 shutdown_connections 2024-04-25T11:14:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.601+0000 7fb36f200640 1 -- 172.21.15.7:0/3663626396 wait complete. 2024-04-25T11:14:57.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:14:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.916+0000 7f5131000640 1 Processor -- start 2024-04-25T11:14:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.916+0000 7f5131000640 1 -- start start 2024-04-25T11:14:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.918+0000 7f5131000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f512c0d0fb0 0x7f512c15fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.918+0000 7f5131000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512c0b8e50 0x7f512c0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f5131000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f512c0b7920 0x7f512c0d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f5131000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f512c0b7d30 con 0x7f512c0b7920 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f5131000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f512c0d13c0 con 0x7f512c0d0fb0 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f5131000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f512c163f10 con 0x7f512c0b8e50 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f512be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f512c0b7920 0x7f512c0d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f512be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f512c0b7920 0x7f512c0d45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39496/0 (socket says 172.21.15.7:39496) 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f512be00640 1 -- 172.21.15.7:0/1427177012 learned_addr learned my addr 172.21.15.7:0/1427177012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f512aa00640 1 --2- 172.21.15.7:0/1427177012 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512c0b8e50 0x7f512c0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f512b400640 1 --2- 172.21.15.7:0/1427177012 >> v2:172.21.15.87:3300/0 conn(0x7f512c0d0fb0 0x7f512c15fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f512be00640 1 -- 172.21.15.7:0/1427177012 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512c0b8e50 msgr2=0x7f512c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f512be00640 1 --2- 172.21.15.7:0/1427177012 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512c0b8e50 0x7f512c0b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f512be00640 1 -- 172.21.15.7:0/1427177012 >> v2:172.21.15.87:3300/0 conn(0x7f512c0d0fb0 msgr2=0x7f512c15fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f512be00640 1 --2- 172.21.15.7:0/1427177012 >> v2:172.21.15.87:3300/0 conn(0x7f512c0d0fb0 0x7f512c15fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.919+0000 7f512be00640 1 -- 172.21.15.7:0/1427177012 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c1641f0 con 0x7f512c0b7920 2024-04-25T11:14:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.920+0000 7f512be00640 1 --2- 172.21.15.7:0/1427177012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512c0b7920 0x7f512c0d45e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f512c0d33d0 tx=0x7f512402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2c1e440e40486d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.921+0000 7f512a000640 1 -- 172.21.15.7:0/1427177012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124036070 con 0x7f512c0b7920 2024-04-25T11:14:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.921+0000 7f512a000640 1 -- 172.21.15.7:0/1427177012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5124031600 con 0x7f512c0b7920 2024-04-25T11:14:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.921+0000 7f512a000640 1 -- 172.21.15.7:0/1427177012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51240319d0 con 0x7f512c0b7920 2024-04-25T11:14:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.921+0000 7f5131000640 1 -- 172.21.15.7:0/1427177012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512c0b7920 msgr2=0x7f512c0d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.921+0000 7f5131000640 1 --2- 172.21.15.7:0/1427177012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512c0b7920 0x7f512c0d45e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f512c0d33d0 tx=0x7f512402a610 comp rx=0 tx=0).stop 2024-04-25T11:14:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.922+0000 7f5131000640 1 -- 172.21.15.7:0/1427177012 shutdown_connections 2024-04-25T11:14:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.922+0000 7f5131000640 1 --2- 172.21.15.7:0/1427177012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512c0b7920 0x7f512c0d45e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.922+0000 7f5131000640 1 --2- 172.21.15.7:0/1427177012 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512c0b8e50 0x7f512c0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.922+0000 7f5131000640 1 --2- 172.21.15.7:0/1427177012 >> v2:172.21.15.87:3300/0 conn(0x7f512c0d0fb0 0x7f512c15fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.922+0000 7f5131000640 1 -- 172.21.15.7:0/1427177012 >> 172.21.15.7:0/1427177012 conn(0x7f512c0d23e0 msgr2=0x7f512c0d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:57.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.922+0000 7f5131000640 1 -- 172.21.15.7:0/1427177012 shutdown_connections 2024-04-25T11:14:57.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.922+0000 7f5131000640 1 -- 172.21.15.7:0/1427177012 wait complete. 2024-04-25T11:14:57.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.924+0000 7f5131000640 1 Processor -- start 2024-04-25T11:14:57.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.924+0000 7f5131000640 1 -- start start 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.926+0000 7f5131000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512c0b7920 0x7f512c1f9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.926+0000 7f5131000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512c0b8e50 0x7f512c0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.926+0000 7f5131000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f512c0d0fb0 0x7f512c158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.926+0000 7f5131000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f512c0b4bc0 con 0x7f512c0b8e50 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.926+0000 7f5131000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f512c1647b0 con 0x7f512c0d0fb0 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.926+0000 7f5131000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f512c0b9260 con 0x7f512c0b7920 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.926+0000 7f512aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512c0b8e50 0x7f512c0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.926+0000 7f512aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512c0b8e50 0x7f512c0b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39504/0 (socket says 172.21.15.7:39504) 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.926+0000 7f512aa00640 1 -- 172.21.15.7:0/822685033 learned_addr learned my addr 172.21.15.7:0/822685033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.926+0000 7f512b400640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512c0b7920 0x7f512c1f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.926+0000 7f512be00640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f512c0d0fb0 0x7f512c158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f512aa00640 1 -- 172.21.15.7:0/822685033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512c0b7920 msgr2=0x7f512c1f9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:57.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f512aa00640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512c0b7920 0x7f512c1f9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f512aa00640 1 -- 172.21.15.7:0/822685033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f512c0d0fb0 msgr2=0x7f512c158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f512aa00640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f512c0d0fb0 0x7f512c158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f512aa00640 1 -- 172.21.15.7:0/822685033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c1fa080 con 0x7f512c0b8e50 2024-04-25T11:14:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f512b400640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512c0b7920 0x7f512c1f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:14:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f512aa00640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512c0b8e50 0x7f512c0b6c10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f511800c5a0 tx=0x7f511800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f5123400640 1 -- 172.21.15.7:0/822685033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118015070 con 0x7f512c0b8e50 2024-04-25T11:14:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f5123400640 1 -- 172.21.15.7:0/822685033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5118004220 con 0x7f512c0b8e50 2024-04-25T11:14:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f5131000640 1 -- 172.21.15.7:0/822685033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c1fa360 con 0x7f512c0b8e50 2024-04-25T11:14:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f5123400640 1 -- 172.21.15.7:0/822685033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118003610 con 0x7f512c0b8e50 2024-04-25T11:14:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.927+0000 7f5131000640 1 -- 172.21.15.7:0/822685033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c1602b0 con 0x7f512c0b8e50 2024-04-25T11:14:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.928+0000 7f5131000640 1 -- 172.21.15.7:0/822685033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512c050740 con 0x7f512c0b8e50 2024-04-25T11:14:57.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.930+0000 7f5123400640 1 -- 172.21.15.7:0/822685033 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f51180037b0 con 0x7f512c0b8e50 2024-04-25T11:14:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.931+0000 7f5123400640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f510c074500 0x7f510c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.931+0000 7f5123400640 1 -- 172.21.15.7:0/822685033 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7f5118094d20 con 0x7f512c0b8e50 2024-04-25T11:14:57.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.931+0000 7f512b400640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f510c074500 0x7f510c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:57.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.932+0000 7f512b400640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f510c074500 0x7f510c076a10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f511c006fd0 tx=0x7f511c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:57.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:57.936+0000 7f5123400640 1 -- 172.21.15.7:0/822685033 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f511805f720 con 0x7f512c0b8e50 2024-04-25T11:14:58.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.140+0000 7f5131000640 1 -- 172.21.15.7:0/822685033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f512c1f9e10 con 0x7f512c0b8e50 2024-04-25T11:14:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.147+0000 7f5123400640 1 -- 172.21.15.7:0/822685033 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f511805f020 con 0x7f512c0b8e50 2024-04-25T11:14:58.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.149+0000 7f5131000640 1 -- 172.21.15.7:0/822685033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f510c074500 msgr2=0x7f510c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:58.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.149+0000 7f5131000640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f510c074500 0x7f510c076a10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f511c006fd0 tx=0x7f511c008040 comp rx=0 tx=0).stop 2024-04-25T11:14:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.149+0000 7f5131000640 1 -- 172.21.15.7:0/822685033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512c0b8e50 msgr2=0x7f512c0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.149+0000 7f5131000640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512c0b8e50 0x7f512c0b6c10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f511800c5a0 tx=0x7f511800ca60 comp rx=0 tx=0).stop 2024-04-25T11:14:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.150+0000 7f5131000640 1 -- 172.21.15.7:0/822685033 shutdown_connections 2024-04-25T11:14:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.150+0000 7f5131000640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f510c074500 0x7f510c076a10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.150+0000 7f5131000640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f512c0d0fb0 0x7f512c158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.150+0000 7f5131000640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512c0b8e50 0x7f512c0b6c10 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.150+0000 7f5131000640 1 --2- 172.21.15.7:0/822685033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512c0b7920 0x7f512c1f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.150+0000 7f5131000640 1 -- 172.21.15.7:0/822685033 >> 172.21.15.7:0/822685033 conn(0x7f512c0d23e0 msgr2=0x7f512c15bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.150+0000 7f5131000640 1 -- 172.21.15.7:0/822685033 shutdown_connections 2024-04-25T11:14:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.150+0000 7f5131000640 1 -- 172.21.15.7:0/822685033 wait complete. 2024-04-25T11:14:58.180 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:14:58.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:14:58.284 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:14:58.401 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:14:58.402 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:14:58.402 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:58.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:14:58.462 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:14:58.462 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:58.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:14:58.520 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:14:58.520 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:58.581 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:14:58.581 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:14:58.581 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:14:58.636 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:14:58.636 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:14:58.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:14:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.928+0000 7f5c8be00640 1 Processor -- start 2024-04-25T11:14:58.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.928+0000 7f5c8be00640 1 -- start start 2024-04-25T11:14:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.931+0000 7f5c8be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5c8c0d2d90 0x7f5c8c09ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.931+0000 7f5c8be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5c8c0d31a0 0x7f5c8c09bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.931+0000 7f5c8be00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c8c0d1cf0 con 0x7f5c8c0cddb0 2024-04-25T11:14:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.931+0000 7f5c8be00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5c8c0d1fd0 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.931+0000 7f5c8be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5c8c0d22b0 con 0x7f5c8c0d2d90 2024-04-25T11:14:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.931+0000 7f5c8a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5c8c0d31a0 0x7f5c8c09bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.931+0000 7f5c8b400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5c8c0cddb0 0x7f5c8c0ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43240/0 (socket says 172.21.15.7:43240) 2024-04-25T11:14:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.931+0000 7f5c8b400640 1 -- 172.21.15.7:0/3242865524 learned_addr learned my addr 172.21.15.7:0/3242865524 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:58.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.932+0000 7f5c8aa00640 1 --2- 172.21.15.7:0/3242865524 >> v2:172.21.15.87:3300/0 conn(0x7f5c8c0d2d90 0x7f5c8c09ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.932+0000 7f5c8a000640 1 -- 172.21.15.7:0/3242865524 >> v2:172.21.15.87:3300/0 conn(0x7f5c8c0d2d90 msgr2=0x7f5c8c09ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:14:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.932+0000 7f5c8a000640 1 --2- 172.21.15.7:0/3242865524 >> v2:172.21.15.87:3300/0 conn(0x7f5c8c0d2d90 0x7f5c8c09ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.932+0000 7f5c8a000640 1 -- 172.21.15.7:0/3242865524 >> v1:172.21.15.7:6789/0 conn(0x7f5c8c0cddb0 legacy=0x7f5c8c0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.932+0000 7f5c8a000640 1 -- 172.21.15.7:0/3242865524 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8c0d2590 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.932+0000 7f5c89600640 1 -- 172.21.15.7:0/3242865524 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 45516073 0 0) 0x7f5c8c0d1cf0 con 0x7f5c8c0cddb0 2024-04-25T11:14:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.933+0000 7f5c8a000640 1 --2- 172.21.15.7:0/3242865524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8c0d31a0 0x7f5c8c09bf40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c073dd0 tx=0x7f5c7402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5bcefc39788b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:58.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.933+0000 7f5c89600640 1 -- 172.21.15.7:0/3242865524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74007740 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.933+0000 7f5c89600640 1 -- 172.21.15.7:0/3242865524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c740078e0 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.933+0000 7f5c89600640 1 -- 172.21.15.7:0/3242865524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74007cb0 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.934+0000 7f5c8be00640 1 -- 172.21.15.7:0/3242865524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8c0d31a0 msgr2=0x7f5c8c09bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.934+0000 7f5c8be00640 1 --2- 172.21.15.7:0/3242865524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8c0d31a0 0x7f5c8c09bf40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c073dd0 tx=0x7f5c7402a680 comp rx=0 tx=0).stop 2024-04-25T11:14:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.934+0000 7f5c8be00640 1 -- 172.21.15.7:0/3242865524 shutdown_connections 2024-04-25T11:14:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.934+0000 7f5c8be00640 1 --2- 172.21.15.7:0/3242865524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8c0d31a0 0x7f5c8c09bf40 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.934+0000 7f5c8be00640 1 --2- 172.21.15.7:0/3242865524 >> v2:172.21.15.87:3300/0 conn(0x7f5c8c0d2d90 0x7f5c8c09ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.934+0000 7f5c8be00640 1 -- 172.21.15.7:0/3242865524 >> 172.21.15.7:0/3242865524 conn(0x7f5c8c0b5250 msgr2=0x7f5c8c0b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.934+0000 7f5c8be00640 1 -- 172.21.15.7:0/3242865524 shutdown_connections 2024-04-25T11:14:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.934+0000 7f5c8be00640 1 -- 172.21.15.7:0/3242865524 wait complete. 2024-04-25T11:14:58.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.937+0000 7f5c8be00640 1 Processor -- start 2024-04-25T11:14:58.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.937+0000 7f5c8be00640 1 -- start start 2024-04-25T11:14:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.938+0000 7f5c8be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c8c0cddb0 0x7f5c8c1f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.938+0000 7f5c8be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c8c0d2d90 0x7f5c8c15f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.938+0000 7f5c8be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8c0d31a0 0x7f5c8c0c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.938+0000 7f5c8be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8c0b91e0 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.938+0000 7f5c8be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5c8c0d19a0 con 0x7f5c8c0cddb0 2024-04-25T11:14:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.938+0000 7f5c8be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c8c06dce0 con 0x7f5c8c0d2d90 2024-04-25T11:14:58.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.939+0000 7f5c8b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8c0d31a0 0x7f5c8c0c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.939+0000 7f5c8b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8c0d31a0 0x7f5c8c0c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39528/0 (socket says 172.21.15.7:39528) 2024-04-25T11:14:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.939+0000 7f5c8b400640 1 -- 172.21.15.7:0/3786240068 learned_addr learned my addr 172.21.15.7:0/3786240068 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.939+0000 7f5c8b400640 1 -- 172.21.15.7:0/3786240068 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c8c0d2d90 msgr2=0x7f5c8c15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.939+0000 7f5c8b400640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c8c0d2d90 0x7f5c8c15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.939+0000 7f5c8b400640 1 -- 172.21.15.7:0/3786240068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c8c0cddb0 msgr2=0x7f5c8c1f67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.939+0000 7f5c8b400640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c8c0cddb0 0x7f5c8c1f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:58.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.939+0000 7f5c8b400640 1 -- 172.21.15.7:0/3786240068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8c1f6d00 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.939+0000 7f5c8b400640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8c0d31a0 0x7f5c8c0c94c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400c410 tx=0x7f5c8400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.940+0000 7f5c82a00640 1 -- 172.21.15.7:0/3786240068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840074e0 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.940+0000 7f5c82a00640 1 -- 172.21.15.7:0/3786240068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c840047e0 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.940+0000 7f5c8be00640 1 -- 172.21.15.7:0/3786240068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8c162930 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.940+0000 7f5c82a00640 1 -- 172.21.15.7:0/3786240068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84004bd0 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.940+0000 7f5c8be00640 1 -- 172.21.15.7:0/3786240068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c8c0c7680 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.941+0000 7f5c8be00640 1 -- 172.21.15.7:0/3786240068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c8c050740 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.942+0000 7f5c82a00640 1 -- 172.21.15.7:0/3786240068 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5c84011070 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.943+0000 7f5c82a00640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c680744b0 0x7f5c680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.943+0000 7f5c82a00640 1 -- 172.21.15.7:0/3786240068 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7f5c84094b90 con 0x7f5c8c0d31a0 2024-04-25T11:14:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.944+0000 7f5c8aa00640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c680744b0 0x7f5c680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:58.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.944+0000 7f5c8aa00640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c680744b0 0x7f5c680769c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c006e20 tx=0x7f5c7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:58.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:58.948+0000 7f5c82a00640 1 -- 172.21.15.7:0/3786240068 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c8c1f6d00 con 0x7f5c8c0d31a0 2024-04-25T11:14:59.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.193+0000 7f5c8be00640 1 -- 172.21.15.7:0/3786240068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c8c09c6e0 con 0x7f5c8c0d31a0 2024-04-25T11:14:59.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.194+0000 7f5c82a00640 1 -- 172.21.15.7:0/3786240068 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3863 (secure 0 0 0) 0x7f5c8405f510 con 0x7f5c8c0d31a0 2024-04-25T11:14:59.196 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:14:59.196 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5183,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2941354720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2941354720},{"type":"v1","addr":"172.21.15.87:6834","nonce":2941354720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5186,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3135576784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":3135576784},{"type":"v1","addr":"172.21.15.87:6835","nonce":3135576784}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2590173274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2590173274},{"type":"v1","addr":"172.21.15.7:6836","nonce":2590173274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6459,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2119826934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2119826934},{"type":"v1","addr":"172.21.15.7:6837","nonce":2119826934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[]} 2024-04-25T11:14:59.196 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-04-25T11:14:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.197+0000 7f5c8be00640 1 -- 172.21.15.7:0/3786240068 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c680744b0 msgr2=0x7f5c680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.197+0000 7f5c8be00640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c680744b0 0x7f5c680769c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c006e20 tx=0x7f5c7c008040 comp rx=0 tx=0).stop 2024-04-25T11:14:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.197+0000 7f5c8be00640 1 -- 172.21.15.7:0/3786240068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8c0d31a0 msgr2=0x7f5c8c0c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.197+0000 7f5c8be00640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8c0d31a0 0x7f5c8c0c94c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400c410 tx=0x7f5c8400c8d0 comp rx=0 tx=0).stop 2024-04-25T11:14:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.197+0000 7f5c8be00640 1 -- 172.21.15.7:0/3786240068 shutdown_connections 2024-04-25T11:14:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.197+0000 7f5c8be00640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c680744b0 0x7f5c680769c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.197+0000 7f5c8be00640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8c0d31a0 0x7f5c8c0c94c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.197+0000 7f5c8be00640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c8c0d2d90 0x7f5c8c15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:59.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.197+0000 7f5c8be00640 1 --2- 172.21.15.7:0/3786240068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c8c0cddb0 0x7f5c8c1f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.197+0000 7f5c8be00640 1 -- 172.21.15.7:0/3786240068 >> 172.21.15.7:0/3786240068 conn(0x7f5c8c0b5250 msgr2=0x7f5c8c0b89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.198+0000 7f5c8be00640 1 -- 172.21.15.7:0/3786240068 shutdown_connections 2024-04-25T11:14:59.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.198+0000 7f5c8be00640 1 -- 172.21.15.7:0/3786240068 wait complete. 2024-04-25T11:14:59.225 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:14:59.226 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:14:59.226 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:14:59.226 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:14:59.226 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:14:59.226 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:14:59.226 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:14:59.226 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:14:59.227 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:14:59.227 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:14:59.227 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:14:59.227 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:14:59.227 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:14:59.227 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:14:59.227 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:14:59.227 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:14:59.254 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:14:59.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:14:59.255 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:14:59.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:14:59.256 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:14:59.256 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:14:59.265 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:14:59.265 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:14:59.302 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:14:59.302 DEBUG:teuthology.parallel:result is None 2024-04-25T11:14:59.307 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:14:59.307 DEBUG:teuthology.parallel:result is None 2024-04-25T11:14:59.312 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:14:59.313 DEBUG:teuthology.parallel:result is None 2024-04-25T11:14:59.319 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:14:59.319 DEBUG:teuthology.parallel:result is None 2024-04-25T11:14:59.320 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:14:59.467 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T11:14:59.528 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:14:59.528 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:14:59.542 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:14:59.544 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:14:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.580+0000 7f6fcba00640 1 Processor -- start 2024-04-25T11:14:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.580+0000 7f6fcba00640 1 -- start start 2024-04-25T11:14:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.582+0000 7f6fcba00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fc415f180 0x7f6fc4161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.582+0000 7f6fcba00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fc4162700 con 0x7f6fc406dc90 2024-04-25T11:14:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.582+0000 7f6fcba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fc41629e0 con 0x7f6fc41621a0 2024-04-25T11:14:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.582+0000 7f6fcba00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc415f590 con 0x7f6fc415f180 2024-04-25T11:14:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.582+0000 7f6fc9a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6fc406dc90 0x7f6fc406f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43262/0 (socket says 172.21.15.7:43262) 2024-04-25T11:14:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.582+0000 7f6fc9a00640 1 -- 172.21.15.7:0/3411069116 learned_addr learned my addr 172.21.15.7:0/3411069116 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:14:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.583+0000 7f6fc9000640 1 --2- 172.21.15.7:0/3411069116 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fc415f180 0x7f6fc4161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.583+0000 7f6fc3e00640 1 -- 172.21.15.7:0/3411069116 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1760254522 0 0) 0x7f6fc4162700 con 0x7f6fc406dc90 2024-04-25T11:14:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.583+0000 7f6fc3e00640 1 -- 172.21.15.7:0/3411069116 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fa8003520 con 0x7f6fc406dc90 2024-04-25T11:14:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.583+0000 7f6fc3e00640 1 -- 172.21.15.7:0/3411069116 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069823655 0 0) 0x7f6fc41629e0 con 0x7f6fc41621a0 2024-04-25T11:14:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.583+0000 7f6fc3e00640 1 -- 172.21.15.7:0/3411069116 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fc4162700 con 0x7f6fc41621a0 2024-04-25T11:14:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.583+0000 7f6fc3e00640 1 -- 172.21.15.7:0/3411069116 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2996056352 0 0) 0x7f6fa8003520 con 0x7f6fc406dc90 2024-04-25T11:14:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.583+0000 7f6fc9000640 1 -- 172.21.15.7:0/3411069116 >> v1:172.21.15.87:6789/0 conn(0x7f6fc41621a0 legacy=0x7f6fc406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.583+0000 7f6fc9000640 1 -- 172.21.15.7:0/3411069116 >> v1:172.21.15.7:6789/0 conn(0x7f6fc406dc90 legacy=0x7f6fc406f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.583+0000 7f6fc9000640 1 -- 172.21.15.7:0/3411069116 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc415e980 con 0x7f6fc415f180 2024-04-25T11:14:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.585+0000 7f6fc9000640 1 --2- 172.21.15.7:0/3411069116 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fc415f180 0x7f6fc4161c60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40097d0 tx=0x7f6fb402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c49ff67eb165a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.585+0000 7f6fc3e00640 1 -- 172.21.15.7:0/3411069116 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4037070 con 0x7f6fc415f180 2024-04-25T11:14:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.585+0000 7f6fc3e00640 1 -- 172.21.15.7:0/3411069116 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb402ab90 con 0x7f6fc415f180 2024-04-25T11:14:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.585+0000 7f6fc3e00640 1 -- 172.21.15.7:0/3411069116 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc0019f0 con 0x7f6fc415f180 2024-04-25T11:14:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.586+0000 7f6fcba00640 1 -- 172.21.15.7:0/3411069116 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fc415f180 msgr2=0x7f6fc4161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.586+0000 7f6fcba00640 1 --2- 172.21.15.7:0/3411069116 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fc415f180 0x7f6fc4161c60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40097d0 tx=0x7f6fb402a100 comp rx=0 tx=0).stop 2024-04-25T11:14:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.586+0000 7f6fcba00640 1 -- 172.21.15.7:0/3411069116 shutdown_connections 2024-04-25T11:14:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.586+0000 7f6fcba00640 1 --2- 172.21.15.7:0/3411069116 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fc415f180 0x7f6fc4161c60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.586+0000 7f6fcba00640 1 -- 172.21.15.7:0/3411069116 >> 172.21.15.7:0/3411069116 conn(0x7f6fc40732b0 msgr2=0x7f6fc40736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:14:59.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.586+0000 7f6fcba00640 1 -- 172.21.15.7:0/3411069116 shutdown_connections 2024-04-25T11:14:59.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.586+0000 7f6fcba00640 1 -- 172.21.15.7:0/3411069116 wait complete. 2024-04-25T11:14:59.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.588+0000 7f6fcba00640 1 Processor -- start 2024-04-25T11:14:59.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.588+0000 7f6fcba00640 1 -- start start 2024-04-25T11:14:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.590+0000 7f6fcba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fc415f180 0x7f6fc4160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.590+0000 7f6fcba00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc41621a0 0x7f6fc41523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.590+0000 7f6fcba00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fc414ee00 0x7f6fc41528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.590+0000 7f6fcba00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc415e670 con 0x7f6fc41621a0 2024-04-25T11:14:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.590+0000 7f6fcba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc4158ae0 con 0x7f6fc415f180 2024-04-25T11:14:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.590+0000 7f6fcba00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc40d1310 con 0x7f6fc414ee00 2024-04-25T11:14:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.590+0000 7f6fbbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc41621a0 0x7f6fc41523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.590+0000 7f6fbbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc41621a0 0x7f6fc41523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39578/0 (socket says 172.21.15.7:39578) 2024-04-25T11:14:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.590+0000 7f6fbbe00640 1 -- 172.21.15.7:0/1913569501 learned_addr learned my addr 172.21.15.7:0/1913569501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:14:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.590+0000 7f6fc9a00640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fc414ee00 0x7f6fc41528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.590+0000 7f6fc9000640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fc415f180 0x7f6fc4160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.591+0000 7f6fbbe00640 1 -- 172.21.15.7:0/1913569501 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fc414ee00 msgr2=0x7f6fc41528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.591+0000 7f6fbbe00640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fc414ee00 0x7f6fc41528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.591+0000 7f6fbbe00640 1 -- 172.21.15.7:0/1913569501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fc415f180 msgr2=0x7f6fc4160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:14:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.591+0000 7f6fbbe00640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fc415f180 0x7f6fc4160fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:14:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.591+0000 7f6fbbe00640 1 -- 172.21.15.7:0/1913569501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc4150dc0 con 0x7f6fc41621a0 2024-04-25T11:14:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.591+0000 7f6fbbe00640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc41621a0 0x7f6fc41523a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00c3c0 tx=0x7f6fac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.591+0000 7f6fc1600640 1 -- 172.21.15.7:0/1913569501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac017070 con 0x7f6fc41621a0 2024-04-25T11:14:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.591+0000 7f6fc1600640 1 -- 172.21.15.7:0/1913569501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fac0052f0 con 0x7f6fc41621a0 2024-04-25T11:14:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.591+0000 7f6fc1600640 1 -- 172.21.15.7:0/1913569501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac003830 con 0x7f6fc41621a0 2024-04-25T11:14:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.591+0000 7f6fcba00640 1 -- 172.21.15.7:0/1913569501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc41510a0 con 0x7f6fc41621a0 2024-04-25T11:14:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.592+0000 7f6fcba00640 1 -- 172.21.15.7:0/1913569501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc4151660 con 0x7f6fc41621a0 2024-04-25T11:14:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.593+0000 7f6fcba00640 1 -- 172.21.15.7:0/1913569501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc4150b30 con 0x7f6fc41621a0 2024-04-25T11:14:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.594+0000 7f6fc1600640 1 -- 172.21.15.7:0/1913569501 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6fac01a070 con 0x7f6fc41621a0 2024-04-25T11:14:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.595+0000 7f6fc1600640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fa00744b0 0x7f6fa00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:14:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.595+0000 7f6fc1600640 1 -- 172.21.15.7:0/1913569501 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7f6fac0943e0 con 0x7f6fc41621a0 2024-04-25T11:14:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.595+0000 7f6fc9000640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fa00744b0 0x7f6fa00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:14:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.596+0000 7f6fc9000640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fa00744b0 0x7f6fa00769c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4034000 tx=0x7f6fb402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:14:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.600+0000 7f6fc1600640 1 -- 172.21.15.7:0/1913569501 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6fac05e6c0 con 0x7f6fc41621a0 2024-04-25T11:14:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:14:59.821+0000 7f6fcba00640 1 -- 172.21.15.7:0/1913569501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6fc4050740 con 0x7f6fc41621a0 2024-04-25T11:15:00.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.661+0000 7f6fc1600640 1 -- 172.21.15.7:0/1913569501 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) ==== 97+0+0 (secure 0 0 0) 0x7f6fac0629c0 con 0x7f6fc41621a0 2024-04-25T11:15:00.663 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-04-25T11:15:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 -- 172.21.15.7:0/1913569501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fa00744b0 msgr2=0x7f6fa00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fa00744b0 0x7f6fa00769c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4034000 tx=0x7f6fb402e000 comp rx=0 tx=0).stop 2024-04-25T11:15:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 -- 172.21.15.7:0/1913569501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc41621a0 msgr2=0x7f6fc41523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc41621a0 0x7f6fc41523a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00c3c0 tx=0x7f6fac00c880 comp rx=0 tx=0).stop 2024-04-25T11:15:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 -- 172.21.15.7:0/1913569501 shutdown_connections 2024-04-25T11:15:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6fa00744b0 0x7f6fa00769c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fc414ee00 0x7f6fc41528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:00.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6fc41621a0 0x7f6fc41523a0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:00.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 --2- 172.21.15.7:0/1913569501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fc415f180 0x7f6fc4160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:00.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 -- 172.21.15.7:0/1913569501 >> 172.21.15.7:0/1913569501 conn(0x7f6fc40732b0 msgr2=0x7f6fc406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:00.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 -- 172.21.15.7:0/1913569501 shutdown_connections 2024-04-25T11:15:00.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.665+0000 7f6fcba00640 1 -- 172.21.15.7:0/1913569501 wait complete. 2024-04-25T11:15:00.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:15:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.974+0000 7f8955400640 1 Processor -- start 2024-04-25T11:15:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.974+0000 7f8955400640 1 -- start start 2024-04-25T11:15:00.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.976+0000 7f8955400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89500b2a90 0x7f895006d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:00.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.976+0000 7f8955400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f895006dda0 0x7f8950164ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:00.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.976+0000 7f8955400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89500d0f50 con 0x7f89500b9050 2024-04-25T11:15:00.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.976+0000 7f8955400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f89500d0d00 con 0x7f895006dda0 2024-04-25T11:15:00.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.976+0000 7f8955400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f895009c970 con 0x7f89500b2a90 2024-04-25T11:15:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.976+0000 7f8954a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f895006dda0 0x7f8950164ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.976+0000 7f894fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89500b2a90 0x7f895006d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.977+0000 7f8954a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f895006dda0 0x7f8950164ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:47320/0 (socket says 172.21.15.7:47320) 2024-04-25T11:15:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.977+0000 7f8954a00640 1 -- 172.21.15.7:0/3457566474 learned_addr learned my addr 172.21.15.7:0/3457566474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.977+0000 7f894fe00640 1 -- 172.21.15.7:0/3457566474 >> v2:172.21.15.87:3300/0 conn(0x7f895006dda0 msgr2=0x7f8950164ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.977+0000 7f894fe00640 1 --2- 172.21.15.7:0/3457566474 >> v2:172.21.15.87:3300/0 conn(0x7f895006dda0 0x7f8950164ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:00.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.977+0000 7f894fe00640 1 -- 172.21.15.7:0/3457566474 >> v1:172.21.15.87:6789/0 conn(0x7f89500b9050 legacy=0x7f8950164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:00.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.977+0000 7f894fe00640 1 -- 172.21.15.7:0/3457566474 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895009cbc0 con 0x7f89500b2a90 2024-04-25T11:15:00.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.977+0000 7f8954a00640 1 --2- 172.21.15.7:0/3457566474 >> v2:172.21.15.87:3300/0 conn(0x7f895006dda0 0x7f8950164ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:15:00.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.979+0000 7f894fe00640 1 --2- 172.21.15.7:0/3457566474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89500b2a90 0x7f895006d860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f89400097a0 tx=0x7f894002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a231f3dc72480db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.980+0000 7f894ea00640 1 -- 172.21.15.7:0/3457566474 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940037070 con 0x7f89500b2a90 2024-04-25T11:15:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.980+0000 7f894ea00640 1 -- 172.21.15.7:0/3457566474 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894002ae50 con 0x7f89500b2a90 2024-04-25T11:15:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.980+0000 7f894ea00640 1 -- 172.21.15.7:0/3457566474 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940032660 con 0x7f89500b2a90 2024-04-25T11:15:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.980+0000 7f8955400640 1 -- 172.21.15.7:0/3457566474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89500b2a90 msgr2=0x7f895006d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.980+0000 7f8955400640 1 --2- 172.21.15.7:0/3457566474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89500b2a90 0x7f895006d860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f89400097a0 tx=0x7f894002a100 comp rx=0 tx=0).stop 2024-04-25T11:15:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.981+0000 7f8955400640 1 -- 172.21.15.7:0/3457566474 shutdown_connections 2024-04-25T11:15:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.981+0000 7f8955400640 1 --2- 172.21.15.7:0/3457566474 >> v2:172.21.15.87:3300/0 conn(0x7f895006dda0 0x7f8950164ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.981+0000 7f8955400640 1 --2- 172.21.15.7:0/3457566474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89500b2a90 0x7f895006d860 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.981+0000 7f8955400640 1 -- 172.21.15.7:0/3457566474 >> 172.21.15.7:0/3457566474 conn(0x7f89500d01e0 msgr2=0x7f89500d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.981+0000 7f8955400640 1 -- 172.21.15.7:0/3457566474 shutdown_connections 2024-04-25T11:15:00.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.981+0000 7f8955400640 1 -- 172.21.15.7:0/3457566474 wait complete. 2024-04-25T11:15:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.983+0000 7f8955400640 1 Processor -- start 2024-04-25T11:15:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.983+0000 7f8955400640 1 -- start start 2024-04-25T11:15:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.985+0000 7f8955400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f895006dda0 0x7f895015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.985+0000 7f8955400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89500b2a90 0x7f89500c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.985+0000 7f8955400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89500b9050 0x7f8950163710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.985+0000 7f8955400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89500b4290 con 0x7f89500b9050 2024-04-25T11:15:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.985+0000 7f8955400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89500b4110 con 0x7f895006dda0 2024-04-25T11:15:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.985+0000 7f8955400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89500b4410 con 0x7f89500b2a90 2024-04-25T11:15:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.985+0000 7f8954a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89500b9050 0x7f8950163710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.985+0000 7f8954a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89500b9050 0x7f8950163710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41512/0 (socket says 172.21.15.7:41512) 2024-04-25T11:15:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.985+0000 7f8954a00640 1 -- 172.21.15.7:0/3334338587 learned_addr learned my addr 172.21.15.7:0/3334338587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.985+0000 7f894fe00640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f895006dda0 0x7f895015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.985+0000 7f894f400640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89500b2a90 0x7f89500c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.986+0000 7f8954a00640 1 -- 172.21.15.7:0/3334338587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89500b2a90 msgr2=0x7f89500c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.986+0000 7f8954a00640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89500b2a90 0x7f89500c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.986+0000 7f8954a00640 1 -- 172.21.15.7:0/3334338587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f895006dda0 msgr2=0x7f895015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.986+0000 7f8954a00640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f895006dda0 0x7f895015b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.986+0000 7f8954a00640 1 -- 172.21.15.7:0/3334338587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89500c8010 con 0x7f89500b9050 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.986+0000 7f8954a00640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89500b9050 0x7f8950163710 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8948006e00 tx=0x7f894800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.986+0000 7f8947e00640 1 -- 172.21.15.7:0/3334338587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948004030 con 0x7f89500b9050 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.986+0000 7f8947e00640 1 -- 172.21.15.7:0/3334338587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89480041d0 con 0x7f89500b9050 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.986+0000 7f8947e00640 1 -- 172.21.15.7:0/3334338587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894800c8b0 con 0x7f89500b9050 2024-04-25T11:15:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.986+0000 7f8955400640 1 -- 172.21.15.7:0/3334338587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89500c82f0 con 0x7f89500b9050 2024-04-25T11:15:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.987+0000 7f8955400640 1 -- 172.21.15.7:0/3334338587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89500ccd50 con 0x7f89500b9050 2024-04-25T11:15:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.988+0000 7f8955400640 1 -- 172.21.15.7:0/3334338587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89500ccfb0 con 0x7f89500b9050 2024-04-25T11:15:00.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.989+0000 7f8947e00640 1 -- 172.21.15.7:0/3334338587 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f894800ca50 con 0x7f89500b9050 2024-04-25T11:15:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.990+0000 7f8947e00640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f892c0744b0 0x7f892c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.990+0000 7f8947e00640 1 -- 172.21.15.7:0/3334338587 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f8948095390 con 0x7f89500b9050 2024-04-25T11:15:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.991+0000 7f894fe00640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f892c0744b0 0x7f892c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.991+0000 7f894fe00640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f892c0744b0 0x7f892c0769c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8940034000 tx=0x7f894002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:00.995+0000 7f8947e00640 1 -- 172.21.15.7:0/3334338587 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8948017020 con 0x7f89500b9050 2024-04-25T11:15:01.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.274+0000 7f8955400640 1 -- 172.21.15.7:0/3334338587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f89500c8810 con 0x7f89500b9050 2024-04-25T11:15:01.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.275+0000 7f8947e00640 1 -- 172.21.15.7:0/3334338587 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f894805fcd0 con 0x7f89500b9050 2024-04-25T11:15:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:01.276 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:15:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-04-25T11:15:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-04-25T11:15:01.276 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w==", 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw==", 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:15:01.277 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw==", 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A==", 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA==", 2024-04-25T11:15:01.279 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg==", 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw==", 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ==", 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:01.281 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw==", 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng==", 2024-04-25T11:15:01.282 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ==", 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A==", 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg==", 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA==", 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg==", 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ==", 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:01.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDZOSpmS8ObHxAAkXXXg1OI1YFC2/OZER3woA==", 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw==", 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w==", 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:15:01.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:01.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:01.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-25T11:15:01.289 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:15:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.277+0000 7f8955400640 1 -- 172.21.15.7:0/3334338587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f892c0744b0 msgr2=0x7f892c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.277+0000 7f8955400640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f892c0744b0 0x7f892c0769c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8940034000 tx=0x7f894002e000 comp rx=0 tx=0).stop 2024-04-25T11:15:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.278+0000 7f8955400640 1 -- 172.21.15.7:0/3334338587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89500b9050 msgr2=0x7f8950163710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.278+0000 7f8955400640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89500b9050 0x7f8950163710 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8948006e00 tx=0x7f894800ed40 comp rx=0 tx=0).stop 2024-04-25T11:15:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.278+0000 7f8955400640 1 -- 172.21.15.7:0/3334338587 shutdown_connections 2024-04-25T11:15:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.278+0000 7f8955400640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f892c0744b0 0x7f892c0769c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.278+0000 7f8955400640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89500b9050 0x7f8950163710 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.278+0000 7f8955400640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89500b2a90 0x7f89500c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.278+0000 7f8955400640 1 --2- 172.21.15.7:0/3334338587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f895006dda0 0x7f895015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.278+0000 7f8955400640 1 -- 172.21.15.7:0/3334338587 >> 172.21.15.7:0/3334338587 conn(0x7f89500d01e0 msgr2=0x7f89500d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.278+0000 7f8955400640 1 -- 172.21.15.7:0/3334338587 shutdown_connections 2024-04-25T11:15:01.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.278+0000 7f8955400640 1 -- 172.21.15.7:0/3334338587 wait complete. 2024-04-25T11:15:01.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:15:01.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.575+0000 7f8d97200640 1 Processor -- start 2024-04-25T11:15:01.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.575+0000 7f8d97200640 1 -- start start 2024-04-25T11:15:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.577+0000 7f8d97200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d980aed40 0x7f8d9815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.577+0000 7f8d97200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980af420 0x7f8d980af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:01.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.577+0000 7f8d97200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d980cebb0 con 0x7f8d980b5220 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.577+0000 7f8d97200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8d9815e1f0 con 0x7f8d980aed40 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.577+0000 7f8d97200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8d980afd70 con 0x7f8d980af420 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.578+0000 7f8d95e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d980aed40 0x7f8d9815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.578+0000 7f8d96800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980af420 0x7f8d980af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.578+0000 7f8d95e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d980aed40 0x7f8d9815dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:47334/0 (socket says 172.21.15.7:47334) 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.578+0000 7f8d95e00640 1 -- 172.21.15.7:0/1325822113 learned_addr learned my addr 172.21.15.7:0/1325822113 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.579+0000 7f8d96800640 1 -- 172.21.15.7:0/1325822113 >> v2:172.21.15.87:3300/0 conn(0x7f8d980aed40 msgr2=0x7f8d9815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.579+0000 7f8d96800640 1 --2- 172.21.15.7:0/1325822113 >> v2:172.21.15.87:3300/0 conn(0x7f8d980aed40 0x7f8d9815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.579+0000 7f8d96800640 1 -- 172.21.15.7:0/1325822113 >> v1:172.21.15.87:6789/0 conn(0x7f8d980b5220 legacy=0x7f8d980ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.579+0000 7f8d96800640 1 -- 172.21.15.7:0/1325822113 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d980b64b0 con 0x7f8d980af420 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.580+0000 7f8d96800640 1 --2- 172.21.15.7:0/1325822113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980af420 0x7f8d980af830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009d90 tx=0x7f8d9002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca8d6cc5eb8e237e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.581+0000 7f8d94a00640 1 -- 172.21.15.7:0/1325822113 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90007740 con 0x7f8d980af420 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.581+0000 7f8d94a00640 1 -- 172.21.15.7:0/1325822113 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d900078e0 con 0x7f8d980af420 2024-04-25T11:15:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.581+0000 7f8d94a00640 1 -- 172.21.15.7:0/1325822113 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90007cb0 con 0x7f8d980af420 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.581+0000 7f8d97200640 1 -- 172.21.15.7:0/1325822113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980af420 msgr2=0x7f8d980af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.581+0000 7f8d97200640 1 --2- 172.21.15.7:0/1325822113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980af420 0x7f8d980af830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009d90 tx=0x7f8d9002a660 comp rx=0 tx=0).stop 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.582+0000 7f8d97200640 1 -- 172.21.15.7:0/1325822113 shutdown_connections 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.582+0000 7f8d97200640 1 --2- 172.21.15.7:0/1325822113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980af420 0x7f8d980af830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.582+0000 7f8d97200640 1 --2- 172.21.15.7:0/1325822113 >> v2:172.21.15.87:3300/0 conn(0x7f8d980aed40 0x7f8d9815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.582+0000 7f8d97200640 1 -- 172.21.15.7:0/1325822113 >> 172.21.15.7:0/1325822113 conn(0x7f8d980b3820 msgr2=0x7f8d980b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.582+0000 7f8d97200640 1 -- 172.21.15.7:0/1325822113 shutdown_connections 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.582+0000 7f8d97200640 1 -- 172.21.15.7:0/1325822113 wait complete. 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.584+0000 7f8d97200640 1 Processor -- start 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.584+0000 7f8d97200640 1 -- start start 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.586+0000 7f8d97200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d980aed40 0x7f8d981587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.586+0000 7f8d97200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d980af420 0x7f8d981f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.586+0000 7f8d97200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980b5220 0x7f8d98159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.586+0000 7f8d97200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9806cbd0 con 0x7f8d980af420 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.586+0000 7f8d97200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9806c8f0 con 0x7f8d980aed40 2024-04-25T11:15:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.586+0000 7f8d97200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8d9806ceb0 con 0x7f8d980b5220 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.586+0000 7f8d95400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d980af420 0x7f8d981f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.586+0000 7f8d95400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d980af420 0x7f8d981f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41526/0 (socket says 172.21.15.7:41526) 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.586+0000 7f8d95400640 1 -- 172.21.15.7:0/2289683260 learned_addr learned my addr 172.21.15.7:0/2289683260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.586+0000 7f8d95e00640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d980aed40 0x7f8d981587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d96800640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980b5220 0x7f8d98159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d95400640 1 -- 172.21.15.7:0/2289683260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980b5220 msgr2=0x7f8d98159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d95400640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980b5220 0x7f8d98159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d95400640 1 -- 172.21.15.7:0/2289683260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d980aed40 msgr2=0x7f8d981587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d95400640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d980aed40 0x7f8d981587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d95400640 1 -- 172.21.15.7:0/2289683260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d981592d0 con 0x7f8d980af420 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d96800640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980b5220 0x7f8d98159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d95400640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d980af420 0x7f8d981f1b80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c3c0 tx=0x7f8d8400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d8e000640 1 -- 172.21.15.7:0/2289683260 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840052d0 con 0x7f8d980af420 2024-04-25T11:15:01.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d8e000640 1 -- 172.21.15.7:0/2289683260 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d84005490 con 0x7f8d980af420 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d8e000640 1 -- 172.21.15.7:0/2289683260 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d84007260 con 0x7f8d980af420 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d97200640 1 -- 172.21.15.7:0/2289683260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d98158ff0 con 0x7f8d980af420 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.587+0000 7f8d97200640 1 -- 172.21.15.7:0/2289683260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9815e720 con 0x7f8d980af420 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.588+0000 7f8d97200640 1 -- 172.21.15.7:0/2289683260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d980ce0e0 con 0x7f8d980af420 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.590+0000 7f8d8e000640 1 -- 172.21.15.7:0/2289683260 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8d84007400 con 0x7f8d980af420 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.591+0000 7f8d8e000640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d780744b0 0x7f8d780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.591+0000 7f8d8e000640 1 -- 172.21.15.7:0/2289683260 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f8d840944c0 con 0x7f8d980af420 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.591+0000 7f8d95e00640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d780744b0 0x7f8d780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.592+0000 7f8d95e00640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d780744b0 0x7f8d780769c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8d98071200 tx=0x7f8d88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.596+0000 7f8d8e000640 1 -- 172.21.15.7:0/2289683260 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8d980ce0e0 con 0x7f8d980af420 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.791+0000 7f8d97200640 1 -- 172.21.15.7:0/2289683260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d98158d50 con 0x7f8d980af420 2024-04-25T11:15:01.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.792+0000 7f8d8e000640 1 -- 172.21.15.7:0/2289683260 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+13069 (secure 0 0 0) 0x7f8d980cebb0 con 0x7f8d980af420 2024-04-25T11:15:01.966 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:01.966 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:15:00.650726+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:15:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 -- 172.21.15.7:0/2289683260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d780744b0 msgr2=0x7f8d780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d780744b0 0x7f8d780769c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8d98071200 tx=0x7f8d88008040 comp rx=0 tx=0).stop 2024-04-25T11:15:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 -- 172.21.15.7:0/2289683260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d980af420 msgr2=0x7f8d981f1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d980af420 0x7f8d981f1b80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400c3c0 tx=0x7f8d8400c880 comp rx=0 tx=0).stop 2024-04-25T11:15:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 -- 172.21.15.7:0/2289683260 shutdown_connections 2024-04-25T11:15:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8d780744b0 0x7f8d780769c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d980b5220 0x7f8d98159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d980af420 0x7f8d981f1b80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 --2- 172.21.15.7:0/2289683260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d980aed40 0x7f8d981587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 -- 172.21.15.7:0/2289683260 >> 172.21.15.7:0/2289683260 conn(0x7f8d980b3820 msgr2=0x7f8d980689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 -- 172.21.15.7:0/2289683260 shutdown_connections 2024-04-25T11:15:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:01.795+0000 7f8d97200640 1 -- 172.21.15.7:0/2289683260 wait complete. 2024-04-25T11:15:01.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:15:01.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:15:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.263+0000 7fe53e400640 1 Processor -- start 2024-04-25T11:15:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.264+0000 7fe53e400640 1 -- start start 2024-04-25T11:15:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.266+0000 7fe53e400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe53806fab0 0x7fe538168e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.266+0000 7fe53e400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53815e3c0 con 0x7fe5380b0470 2024-04-25T11:15:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.266+0000 7fe53e400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5380cf800 con 0x7fe53815dfb0 2024-04-25T11:15:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.266+0000 7fe53e400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe5380d1250 con 0x7fe53806fab0 2024-04-25T11:15:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.266+0000 7fe52ea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe5380b0470 0x7fe53806f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58500/0 (socket says 172.21.15.7:58500) 2024-04-25T11:15:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.266+0000 7fe52ea00640 1 -- 172.21.15.7:0/2529592600 learned_addr learned my addr 172.21.15.7:0/2529592600 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe537e00640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe53806fab0 0x7fe538168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2259799618 0 0) 0x7fe53815e3c0 con 0x7fe5380b0470 2024-04-25T11:15:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe51c003520 con 0x7fe5380b0470 2024-04-25T11:15:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 119082424 0 0) 0x7fe51c003520 con 0x7fe5380b0470 2024-04-25T11:15:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe53806fab0 msgr2=0x7fe538168e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe53806fab0 0x7fe538168e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 >> v1:172.21.15.87:6789/0 conn(0x7fe53815dfb0 legacy=0x7fe5380d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5380cfaa0 con 0x7fe5380b0470 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fe520001670 con 0x7fe5380b0470 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 >> v1:172.21.15.7:6789/0 conn(0x7fe5380b0470 legacy=0x7fe53806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe53806fab0 0x7fe51c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe51c003f60 0x7fe51c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe51c004d40 0x7fe51c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5380d1250 con 0x7fe51c004d40 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c00bb50 con 0x7fe51c003f60 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.267+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe51c00bd50 con 0x7fe53806fab0 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.268+0000 7fe537e00640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe51c004d40 0x7fe51c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.268+0000 7fe537400640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe53806fab0 0x7fe51c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.268+0000 7fe52ea00640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe51c003f60 0x7fe51c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.268+0000 7fe537e00640 1 -- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe53806fab0 msgr2=0x7fe51c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.268+0000 7fe537e00640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe53806fab0 0x7fe51c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.268+0000 7fe537e00640 1 -- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe51c003f60 msgr2=0x7fe51c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.268+0000 7fe537e00640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe51c003f60 0x7fe51c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.268+0000 7fe537e00640 1 -- 172.21.15.7:0/2529592600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c003520 con 0x7fe51c004d40 2024-04-25T11:15:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.268+0000 7fe52ea00640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe51c003f60 0x7fe51c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.269+0000 7fe537e00640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe51c004d40 0x7fe51c005150 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe530009fc0 tx=0x7fe51c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0eabf8de3e41dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.270+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530034020 con 0x7fe51c004d40 2024-04-25T11:15:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.270+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5300305f0 con 0x7fe51c004d40 2024-04-25T11:15:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.270+0000 7fe536a00640 1 -- 172.21.15.7:0/2529592600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530009c10 con 0x7fe51c004d40 2024-04-25T11:15:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.271+0000 7fe53e400640 1 -- 172.21.15.7:0/2529592600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe51c004d40 msgr2=0x7fe51c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.271+0000 7fe53e400640 1 --2- 172.21.15.7:0/2529592600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe51c004d40 0x7fe51c005150 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe530009fc0 tx=0x7fe51c005830 comp rx=0 tx=0).stop 2024-04-25T11:15:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.271+0000 7fe537e00640 1 -- 172.21.15.7:0/2529592600 reap_dead start 2024-04-25T11:15:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.271+0000 7fe53e400640 1 -- 172.21.15.7:0/2529592600 shutdown_connections 2024-04-25T11:15:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.271+0000 7fe53e400640 1 -- 172.21.15.7:0/2529592600 >> 172.21.15.7:0/2529592600 conn(0x7fe5380742c0 msgr2=0x7fe53806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.271+0000 7fe53e400640 1 -- 172.21.15.7:0/2529592600 shutdown_connections 2024-04-25T11:15:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.271+0000 7fe53e400640 1 -- 172.21.15.7:0/2529592600 wait complete. 2024-04-25T11:15:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.273+0000 7fe53e400640 1 Processor -- start 2024-04-25T11:15:02.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.273+0000 7fe53e400640 1 -- start start 2024-04-25T11:15:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.275+0000 7fe53e400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5381e91d0 0x7fe5381f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.275+0000 7fe53e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe538150d50 0x7fe5381564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.275+0000 7fe53e400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe538151160 0x7fe538156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.275+0000 7fe53e400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5380d0340 con 0x7fe538150d50 2024-04-25T11:15:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.275+0000 7fe53e400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe5380d0060 con 0x7fe5381e91d0 2024-04-25T11:15:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.275+0000 7fe53e400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe5380d0620 con 0x7fe538151160 2024-04-25T11:15:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.275+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe538150d50 0x7fe5381564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.275+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe538150d50 0x7fe5381564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41544/0 (socket says 172.21.15.7:41544) 2024-04-25T11:15:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.275+0000 7fe52ea00640 1 -- 172.21.15.7:0/4223450297 learned_addr learned my addr 172.21.15.7:0/4223450297 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe537400640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5381e91d0 0x7fe5381f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe537e00640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe538151160 0x7fe538156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe52ea00640 1 -- 172.21.15.7:0/4223450297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe538151160 msgr2=0x7fe538156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe52ea00640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe538151160 0x7fe538156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe52ea00640 1 -- 172.21.15.7:0/4223450297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5381e91d0 msgr2=0x7fe5381f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe52ea00640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5381e91d0 0x7fe5381f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe52ea00640 1 -- 172.21.15.7:0/4223450297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5381e9820 con 0x7fe538150d50 2024-04-25T11:15:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe52ea00640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe538150d50 0x7fe5381564d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe52000db10 tx=0x7fe52000df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe52fe00640 1 -- 172.21.15.7:0/4223450297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520019070 con 0x7fe538150d50 2024-04-25T11:15:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe52fe00640 1 -- 172.21.15.7:0/4223450297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe520004cb0 con 0x7fe538150d50 2024-04-25T11:15:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe52fe00640 1 -- 172.21.15.7:0/4223450297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520004e60 con 0x7fe538150d50 2024-04-25T11:15:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.276+0000 7fe53e400640 1 -- 172.21.15.7:0/4223450297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5381e9b00 con 0x7fe538150d50 2024-04-25T11:15:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.277+0000 7fe53e400640 1 -- 172.21.15.7:0/4223450297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5381ea0c0 con 0x7fe538150d50 2024-04-25T11:15:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.278+0000 7fe53e400640 1 -- 172.21.15.7:0/4223450297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe538050740 con 0x7fe538150d50 2024-04-25T11:15:02.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.279+0000 7fe52fe00640 1 -- 172.21.15.7:0/4223450297 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe52001c070 con 0x7fe538150d50 2024-04-25T11:15:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.280+0000 7fe52fe00640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5140744b0 0x7fe5140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.280+0000 7fe52fe00640 1 -- 172.21.15.7:0/4223450297 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7fe520094cf0 con 0x7fe538150d50 2024-04-25T11:15:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.281+0000 7fe537400640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5140744b0 0x7fe5140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.281+0000 7fe537400640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5140744b0 0x7fe5140769c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe538161180 tx=0x7fe528009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:02.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.286+0000 7fe52fe00640 1 -- 172.21.15.7:0/4223450297 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe52005f5a0 con 0x7fe538150d50 2024-04-25T11:15:02.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.480+0000 7fe53e400640 1 -- 172.21.15.7:0/4223450297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe538157d90 con 0x7fe538150d50 2024-04-25T11:15:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.481+0000 7fe52fe00640 1 -- 172.21.15.7:0/4223450297 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7fe52005eea0 con 0x7fe538150d50 2024-04-25T11:15:02.482 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:15:02.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.483+0000 7fe53e400640 1 -- 172.21.15.7:0/4223450297 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5140744b0 msgr2=0x7fe5140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:02.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.483+0000 7fe53e400640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5140744b0 0x7fe5140769c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe538161180 tx=0x7fe528009040 comp rx=0 tx=0).stop 2024-04-25T11:15:02.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.483+0000 7fe53e400640 1 -- 172.21.15.7:0/4223450297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe538150d50 msgr2=0x7fe5381564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:02.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.483+0000 7fe53e400640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe538150d50 0x7fe5381564d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe52000db10 tx=0x7fe52000df10 comp rx=0 tx=0).stop 2024-04-25T11:15:02.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.484+0000 7fe53e400640 1 -- 172.21.15.7:0/4223450297 shutdown_connections 2024-04-25T11:15:02.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.484+0000 7fe53e400640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe5140744b0 0x7fe5140769c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.484+0000 7fe53e400640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe538151160 0x7fe538156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.484+0000 7fe53e400640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe538150d50 0x7fe5381564d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.484+0000 7fe53e400640 1 --2- 172.21.15.7:0/4223450297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5381e91d0 0x7fe5381f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.484+0000 7fe53e400640 1 -- 172.21.15.7:0/4223450297 >> 172.21.15.7:0/4223450297 conn(0x7fe5380742c0 msgr2=0x7fe53806f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:02.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.484+0000 7fe53e400640 1 -- 172.21.15.7:0/4223450297 shutdown_connections 2024-04-25T11:15:02.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.484+0000 7fe53e400640 1 -- 172.21.15.7:0/4223450297 wait complete. 2024-04-25T11:15:02.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:15:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.788+0000 7f03d7e00640 1 Processor -- start 2024-04-25T11:15:02.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.788+0000 7f03d7e00640 1 -- start start 2024-04-25T11:15:02.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.791+0000 7f03d7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03d815baa0 0x7f03d815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.791+0000 7f03d7e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d815e980 con 0x7f03d8070640 2024-04-25T11:15:02.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.791+0000 7f03d7e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03d815ec60 con 0x7f03d806d580 2024-04-25T11:15:02.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.791+0000 7f03d7e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f03d815ef40 con 0x7f03d815baa0 2024-04-25T11:15:02.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.791+0000 7f03cfe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f03d8070640 0x7f03d8161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58502/0 (socket says 172.21.15.7:58502) 2024-04-25T11:15:02.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.791+0000 7f03cfe00640 1 -- 172.21.15.7:0/3188178521 learned_addr learned my addr 172.21.15.7:0/3188178521 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.791+0000 7f03d6a00640 1 --2- 172.21.15.7:0/3188178521 >> v2:172.21.15.7:3300/0 conn(0x7f03d815baa0 0x7f03d815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.792+0000 7f03d6a00640 1 -- 172.21.15.7:0/3188178521 >> v1:172.21.15.87:6789/0 conn(0x7f03d806d580 legacy=0x7f03d806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.792+0000 7f03d6a00640 1 -- 172.21.15.7:0/3188178521 >> v1:172.21.15.7:6789/0 conn(0x7f03d8070640 legacy=0x7f03d8161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.792+0000 7f03d6a00640 1 -- 172.21.15.7:0/3188178521 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d81619c0 con 0x7f03d815baa0 2024-04-25T11:15:02.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.793+0000 7f03d6a00640 1 --2- 172.21.15.7:0/3188178521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d815baa0 0x7f03d815e2f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f03c80097a0 tx=0x7f03c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5da496924ae764 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:02.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.793+0000 7f03d6000640 1 -- 172.21.15.7:0/3188178521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8037070 con 0x7f03d815baa0 2024-04-25T11:15:02.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.793+0000 7f03d6000640 1 -- 172.21.15.7:0/3188178521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c8032440 con 0x7f03d815baa0 2024-04-25T11:15:02.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.793+0000 7f03d6000640 1 -- 172.21.15.7:0/3188178521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8032810 con 0x7f03d815baa0 2024-04-25T11:15:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.794+0000 7f03d7e00640 1 -- 172.21.15.7:0/3188178521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d815baa0 msgr2=0x7f03d815e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.794+0000 7f03d7e00640 1 --2- 172.21.15.7:0/3188178521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d815baa0 0x7f03d815e2f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f03c80097a0 tx=0x7f03c802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.794+0000 7f03d7e00640 1 -- 172.21.15.7:0/3188178521 shutdown_connections 2024-04-25T11:15:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.794+0000 7f03d7e00640 1 --2- 172.21.15.7:0/3188178521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d815baa0 0x7f03d815e2f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.794+0000 7f03d7e00640 1 -- 172.21.15.7:0/3188178521 >> 172.21.15.7:0/3188178521 conn(0x7f03d8073090 msgr2=0x7f03d80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.794+0000 7f03d7e00640 1 -- 172.21.15.7:0/3188178521 shutdown_connections 2024-04-25T11:15:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.794+0000 7f03d7e00640 1 -- 172.21.15.7:0/3188178521 wait complete. 2024-04-25T11:15:02.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.796+0000 7f03d7e00640 1 Processor -- start 2024-04-25T11:15:02.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.796+0000 7f03d7e00640 1 -- start start 2024-04-25T11:15:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.798+0000 7f03d7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d806d580 0x7f03d8144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.798+0000 7f03d7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03d8070640 0x7f03d8142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.798+0000 7f03d7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d80b0370 0x7f03d80ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.798+0000 7f03d7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8162770 con 0x7f03d806d580 2024-04-25T11:15:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.798+0000 7f03d7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81625f0 con 0x7f03d8070640 2024-04-25T11:15:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.798+0000 7f03d7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f03d815f220 con 0x7f03d80b0370 2024-04-25T11:15:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.798+0000 7f03d6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d806d580 0x7f03d8144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.798+0000 7f03d6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d806d580 0x7f03d8144ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41564/0 (socket says 172.21.15.7:41564) 2024-04-25T11:15:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.798+0000 7f03d6a00640 1 -- 172.21.15.7:0/4288557064 learned_addr learned my addr 172.21.15.7:0/4288557064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03cfe00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03d8070640 0x7f03d8142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03d6a00640 1 -- 172.21.15.7:0/4288557064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d80b0370 msgr2=0x7f03d80ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:02.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03d7400640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d80b0370 0x7f03d80ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03d6a00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d80b0370 0x7f03d80ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03d6a00640 1 -- 172.21.15.7:0/4288557064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03d8070640 msgr2=0x7f03d8142ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:02.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03d6a00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03d8070640 0x7f03d8142ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:02.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03d6a00640 1 -- 172.21.15.7:0/4288557064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d81f3df0 con 0x7f03d806d580 2024-04-25T11:15:02.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03cfe00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03d8070640 0x7f03d8142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:02.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03d6a00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d806d580 0x7f03d8144ef0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f03c8000c00 tx=0x7f03c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:02.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03cea00640 1 -- 172.21.15.7:0/4288557064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8041070 con 0x7f03d806d580 2024-04-25T11:15:02.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03cea00640 1 -- 172.21.15.7:0/4288557064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c803c930 con 0x7f03d806d580 2024-04-25T11:15:02.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03cea00640 1 -- 172.21.15.7:0/4288557064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8037040 con 0x7f03d806d580 2024-04-25T11:15:02.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.799+0000 7f03d7e00640 1 -- 172.21.15.7:0/4288557064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d80b0dc0 con 0x7f03d806d580 2024-04-25T11:15:02.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.800+0000 7f03d7e00640 1 -- 172.21.15.7:0/4288557064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d80ab1e0 con 0x7f03d806d580 2024-04-25T11:15:02.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.801+0000 7f03d7e00640 1 -- 172.21.15.7:0/4288557064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d8050740 con 0x7f03d806d580 2024-04-25T11:15:02.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.802+0000 7f03cea00640 1 -- 172.21.15.7:0/4288557064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f03c8044070 con 0x7f03d806d580 2024-04-25T11:15:02.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.803+0000 7f03cea00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03b40744e0 0x7f03b40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:02.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.803+0000 7f03cea00640 1 -- 172.21.15.7:0/4288557064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f03c80b59f0 con 0x7f03d806d580 2024-04-25T11:15:02.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.803+0000 7f03cfe00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03b40744e0 0x7f03b40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:02.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.804+0000 7f03cfe00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03b40744e0 0x7f03b40769f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f03d81617e0 tx=0x7f03c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:02.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:02.808+0000 7f03cea00640 1 -- 172.21.15.7:0/4288557064 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f03c80802a0 con 0x7f03d806d580 2024-04-25T11:15:03.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.006+0000 7f03d7e00640 1 -- 172.21.15.7:0/4288557064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f03d8074020 con 0x7f03d806d580 2024-04-25T11:15:03.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.007+0000 7f03cea00640 1 -- 172.21.15.7:0/4288557064 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f03c8088320 con 0x7f03d806d580 2024-04-25T11:15:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:03.009 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-25T11:15:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-25T11:15:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-25T11:15:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:03.009 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.010+0000 7f03d7e00640 1 -- 172.21.15.7:0/4288557064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03b40744e0 msgr2=0x7f03b40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.010+0000 7f03d7e00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03b40744e0 0x7f03b40769f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f03d81617e0 tx=0x7f03c0009040 comp rx=0 tx=0).stop 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.010+0000 7f03d7e00640 1 -- 172.21.15.7:0/4288557064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d806d580 msgr2=0x7f03d8144ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.010+0000 7f03d7e00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d806d580 0x7f03d8144ef0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f03c8000c00 tx=0x7f03c80046c0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.010+0000 7f03d7e00640 1 -- 172.21.15.7:0/4288557064 shutdown_connections 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.010+0000 7f03d7e00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f03b40744e0 0x7f03b40769f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.010+0000 7f03d7e00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d80b0370 0x7f03d80ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.010+0000 7f03d7e00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03d8070640 0x7f03d8142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.010+0000 7f03d7e00640 1 --2- 172.21.15.7:0/4288557064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03d806d580 0x7f03d8144ef0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.010+0000 7f03d7e00640 1 -- 172.21.15.7:0/4288557064 >> 172.21.15.7:0/4288557064 conn(0x7f03d8073090 msgr2=0x7f03d806f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.011+0000 7f03d7e00640 1 -- 172.21.15.7:0/4288557064 shutdown_connections 2024-04-25T11:15:03.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.011+0000 7f03d7e00640 1 -- 172.21.15.7:0/4288557064 wait complete. 2024-04-25T11:15:03.036 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:15:03.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.330+0000 7fbe14e00640 1 Processor -- start 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.330+0000 7fbe14e00640 1 -- start start 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.332+0000 7fbe14e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe1016d090 0x7fbe1006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.332+0000 7fbe14e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe1006dc90 0x7fbe100b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.332+0000 7fbe14e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe1016d6e0 con 0x7fbe100b5df0 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.332+0000 7fbe14e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbe1016d4a0 con 0x7fbe1006dc90 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.332+0000 7fbe14e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe100b47c0 con 0x7fbe1016d090 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.332+0000 7fbe0fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe1006dc90 0x7fbe100b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.333+0000 7fbe0fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe1006dc90 0x7fbe100b4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41576/0 (socket says 172.21.15.7:41576) 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.333+0000 7fbe0fe00640 1 -- 172.21.15.7:0/3025116808 learned_addr learned my addr 172.21.15.7:0/3025116808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.334+0000 7fbe0e000640 1 -- 172.21.15.7:0/3025116808 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170517251 0 0) 0x7fbe1016d6e0 con 0x7fbe100b5df0 2024-04-25T11:15:03.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.333+0000 7fbe0f400640 1 --2- 172.21.15.7:0/3025116808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe1016d090 0x7fbe1006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:03.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.339+0000 7fbe0e000640 1 -- 172.21.15.7:0/3025116808 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdf4003500 con 0x7fbe100b5df0 2024-04-25T11:15:03.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.339+0000 7fbe0fe00640 1 -- 172.21.15.7:0/3025116808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe1016d090 msgr2=0x7fbe1006eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:03.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.339+0000 7fbe0fe00640 1 --2- 172.21.15.7:0/3025116808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe1016d090 0x7fbe1006eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.339+0000 7fbe0fe00640 1 -- 172.21.15.7:0/3025116808 >> v1:172.21.15.87:6789/0 conn(0x7fbe100b5df0 legacy=0x7fbe1006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.339+0000 7fbe0fe00640 1 -- 172.21.15.7:0/3025116808 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1016d9c0 con 0x7fbe1006dc90 2024-04-25T11:15:03.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.339+0000 7fbe0f400640 1 --2- 172.21.15.7:0/3025116808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe1016d090 0x7fbe1006eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:15:03.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.340+0000 7fbe0fe00640 1 --2- 172.21.15.7:0/3025116808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe1006dc90 0x7fbe100b4c50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbe080083f0 tx=0x7fbe08032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b7d501632518f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:03.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.341+0000 7fbe0e000640 1 -- 172.21.15.7:0/3025116808 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08033860 con 0x7fbe1006dc90 2024-04-25T11:15:03.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.341+0000 7fbe0e000640 1 -- 172.21.15.7:0/3025116808 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe08033a00 con 0x7fbe1006dc90 2024-04-25T11:15:03.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.341+0000 7fbe0e000640 1 -- 172.21.15.7:0/3025116808 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08033db0 con 0x7fbe1006dc90 2024-04-25T11:15:03.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.342+0000 7fbe14e00640 1 -- 172.21.15.7:0/3025116808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe1006dc90 msgr2=0x7fbe100b4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.342+0000 7fbe14e00640 1 --2- 172.21.15.7:0/3025116808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe1006dc90 0x7fbe100b4c50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbe080083f0 tx=0x7fbe08032f20 comp rx=0 tx=0).stop 2024-04-25T11:15:03.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.342+0000 7fbe14e00640 1 -- 172.21.15.7:0/3025116808 shutdown_connections 2024-04-25T11:15:03.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.342+0000 7fbe14e00640 1 --2- 172.21.15.7:0/3025116808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe1006dc90 0x7fbe100b4c50 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.342+0000 7fbe14e00640 1 --2- 172.21.15.7:0/3025116808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe1016d090 0x7fbe1006eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.342+0000 7fbe14e00640 1 -- 172.21.15.7:0/3025116808 >> 172.21.15.7:0/3025116808 conn(0x7fbe10073160 msgr2=0x7fbe10073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:03.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.342+0000 7fbe14e00640 1 -- 172.21.15.7:0/3025116808 shutdown_connections 2024-04-25T11:15:03.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.342+0000 7fbe14e00640 1 -- 172.21.15.7:0/3025116808 wait complete. 2024-04-25T11:15:03.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.345+0000 7fbe14e00640 1 Processor -- start 2024-04-25T11:15:03.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.345+0000 7fbe14e00640 1 -- start start 2024-04-25T11:15:03.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.346+0000 7fbe14e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe100b5df0 0x7fbe100cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:03.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.346+0000 7fbe14e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe1016d090 0x7fbe102026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:03.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe14e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe10202c30 0x7fbe10205ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:03.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe14e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1006f420 con 0x7fbe100b5df0 2024-04-25T11:15:03.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe14e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1016df80 con 0x7fbe1016d090 2024-04-25T11:15:03.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe14e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe1006f5a0 con 0x7fbe10202c30 2024-04-25T11:15:03.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe0f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe100b5df0 0x7fbe100cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:03.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe0f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe100b5df0 0x7fbe100cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41590/0 (socket says 172.21.15.7:41590) 2024-04-25T11:15:03.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe0f400640 1 -- 172.21.15.7:0/967819588 learned_addr learned my addr 172.21.15.7:0/967819588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:03.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe0f400640 1 -- 172.21.15.7:0/967819588 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe10202c30 msgr2=0x7fbe10205ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe0f400640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe10202c30 0x7fbe10205ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe0f400640 1 -- 172.21.15.7:0/967819588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe1016d090 msgr2=0x7fbe102026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe0f400640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe1016d090 0x7fbe102026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe0f400640 1 -- 172.21.15.7:0/967819588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe10203450 con 0x7fbe100b5df0 2024-04-25T11:15:03.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.347+0000 7fbe0f400640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe100b5df0 0x7fbe100cd060 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000da30 tx=0x7fbe0000db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:03.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.348+0000 7fbe07400640 1 -- 172.21.15.7:0/967819588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00016030 con 0x7fbe100b5df0 2024-04-25T11:15:03.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.348+0000 7fbe07400640 1 -- 172.21.15.7:0/967819588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe00004020 con 0x7fbe100b5df0 2024-04-25T11:15:03.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.348+0000 7fbe14e00640 1 -- 172.21.15.7:0/967819588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe10166b20 con 0x7fbe100b5df0 2024-04-25T11:15:03.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.348+0000 7fbe07400640 1 -- 172.21.15.7:0/967819588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00004410 con 0x7fbe100b5df0 2024-04-25T11:15:03.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.348+0000 7fbe14e00640 1 -- 172.21.15.7:0/967819588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe101670e0 con 0x7fbe100b5df0 2024-04-25T11:15:03.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.349+0000 7fbe14e00640 1 -- 172.21.15.7:0/967819588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe10050740 con 0x7fbe100b5df0 2024-04-25T11:15:03.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.350+0000 7fbe07400640 1 -- 172.21.15.7:0/967819588 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbe00019070 con 0x7fbe100b5df0 2024-04-25T11:15:03.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.351+0000 7fbe07400640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdf00744b0 0x7fbdf00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:03.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.351+0000 7fbe07400640 1 -- 172.21.15.7:0/967819588 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7fbe00095050 con 0x7fbe100b5df0 2024-04-25T11:15:03.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.352+0000 7fbe0ea00640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdf00744b0 0x7fbdf00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:03.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.352+0000 7fbe0ea00640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdf00744b0 0x7fbdf00769c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0011e0 tx=0x7fbdfc0078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:03.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.356+0000 7fbe07400640 1 -- 172.21.15.7:0/967819588 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbe0005fa20 con 0x7fbe100b5df0 2024-04-25T11:15:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.567+0000 7fbe14e00640 1 -- 172.21.15.7:0/967819588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbe100736a0 con 0x7fbe100b5df0 2024-04-25T11:15:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.663+0000 7fbe07400640 1 -- 172.21.15.7:0/967819588 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7fbe0005f320 con 0x7fbe100b5df0 2024-04-25T11:15:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-25T11:15:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.667+0000 7fbe14e00640 1 -- 172.21.15.7:0/967819588 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdf00744b0 msgr2=0x7fbdf00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.667+0000 7fbe14e00640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdf00744b0 0x7fbdf00769c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0011e0 tx=0x7fbdfc0078f0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.667+0000 7fbe14e00640 1 -- 172.21.15.7:0/967819588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe100b5df0 msgr2=0x7fbe100cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.667+0000 7fbe14e00640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe100b5df0 0x7fbe100cd060 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000da30 tx=0x7fbe0000db20 comp rx=0 tx=0).stop 2024-04-25T11:15:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.667+0000 7fbe14e00640 1 -- 172.21.15.7:0/967819588 shutdown_connections 2024-04-25T11:15:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.667+0000 7fbe14e00640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdf00744b0 0x7fbdf00769c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.667+0000 7fbe14e00640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe10202c30 0x7fbe10205ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.667+0000 7fbe14e00640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe1016d090 0x7fbe102026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.667+0000 7fbe14e00640 1 --2- 172.21.15.7:0/967819588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe100b5df0 0x7fbe100cd060 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.667+0000 7fbe14e00640 1 -- 172.21.15.7:0/967819588 >> 172.21.15.7:0/967819588 conn(0x7fbe10073160 msgr2=0x7fbe1016aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.668+0000 7fbe14e00640 1 -- 172.21.15.7:0/967819588 shutdown_connections 2024-04-25T11:15:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.668+0000 7fbe14e00640 1 -- 172.21.15.7:0/967819588 wait complete. 2024-04-25T11:15:03.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:15:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.978+0000 7fe345c00640 1 Processor -- start 2024-04-25T11:15:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.978+0000 7fe345c00640 1 -- start start 2024-04-25T11:15:03.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.981+0000 7fe345c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe340145e40 0x7fe340146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:03.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.981+0000 7fe345c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3400b66b0 con 0x7fe3400b4870 2024-04-25T11:15:03.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.981+0000 7fe345c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3400b5b30 con 0x7fe340144a80 2024-04-25T11:15:03.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.981+0000 7fe345c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe340145b70 con 0x7fe340145e40 2024-04-25T11:15:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.981+0000 7fe33fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe340145e40 0x7fe340146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.981+0000 7fe33f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe3400b4870 0x7fe3400b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58518/0 (socket says 172.21.15.7:58518) 2024-04-25T11:15:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.981+0000 7fe33f400640 1 -- 172.21.15.7:0/1645765973 learned_addr learned my addr 172.21.15.7:0/1645765973 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.982+0000 7fe33fe00640 1 -- 172.21.15.7:0/1645765973 >> v1:172.21.15.87:6789/0 conn(0x7fe340144a80 legacy=0x7fe3401606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.982+0000 7fe33fe00640 1 -- 172.21.15.7:0/1645765973 >> v1:172.21.15.7:6789/0 conn(0x7fe3400b4870 legacy=0x7fe3400b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.982+0000 7fe33fe00640 1 -- 172.21.15.7:0/1645765973 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3400b5e10 con 0x7fe340145e40 2024-04-25T11:15:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.983+0000 7fe33fe00640 1 --2- 172.21.15.7:0/1645765973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe340145e40 0x7fe340146250 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe3300097a0 tx=0x7fe33002c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aabf1a8f25f1bed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.984+0000 7fe33ea00640 1 -- 172.21.15.7:0/1645765973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330039070 con 0x7fe340145e40 2024-04-25T11:15:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.984+0000 7fe33ea00640 1 -- 172.21.15.7:0/1645765973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe330034640 con 0x7fe340145e40 2024-04-25T11:15:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.984+0000 7fe33ea00640 1 -- 172.21.15.7:0/1645765973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330034a10 con 0x7fe340145e40 2024-04-25T11:15:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.984+0000 7fe345c00640 1 -- 172.21.15.7:0/1645765973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe340145e40 msgr2=0x7fe340146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.984+0000 7fe345c00640 1 --2- 172.21.15.7:0/1645765973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe340145e40 0x7fe340146250 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe3300097a0 tx=0x7fe33002c3b0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.984+0000 7fe345c00640 1 -- 172.21.15.7:0/1645765973 shutdown_connections 2024-04-25T11:15:03.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.984+0000 7fe345c00640 1 --2- 172.21.15.7:0/1645765973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe340145e40 0x7fe340146250 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.984+0000 7fe345c00640 1 -- 172.21.15.7:0/1645765973 >> 172.21.15.7:0/1645765973 conn(0x7fe3400740f0 msgr2=0x7fe34006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:03.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.985+0000 7fe345c00640 1 -- 172.21.15.7:0/1645765973 shutdown_connections 2024-04-25T11:15:03.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.985+0000 7fe345c00640 1 -- 172.21.15.7:0/1645765973 wait complete. 2024-04-25T11:15:03.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.987+0000 7fe345c00640 1 Processor -- start 2024-04-25T11:15:03.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.987+0000 7fe345c00640 1 -- start start 2024-04-25T11:15:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.988+0000 7fe345c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3400b4870 0x7fe34013add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe345c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe340144a80 0x7fe34014ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe345c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe340145e40 0x7fe34013dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe345c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe340161e10 con 0x7fe3400b4870 2024-04-25T11:15:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe345c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe3400b63d0 con 0x7fe340145e40 2024-04-25T11:15:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe345c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe340161f90 con 0x7fe340144a80 2024-04-25T11:15:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe33fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3400b4870 0x7fe34013add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe33fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3400b4870 0x7fe34013add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41614/0 (socket says 172.21.15.7:41614) 2024-04-25T11:15:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe33fe00640 1 -- 172.21.15.7:0/3811948222 learned_addr learned my addr 172.21.15.7:0/3811948222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe345200640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe340145e40 0x7fe34013dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe33f400640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe340144a80 0x7fe34014ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe33fe00640 1 -- 172.21.15.7:0/3811948222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe340144a80 msgr2=0x7fe34014ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe33fe00640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe340144a80 0x7fe34014ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe33fe00640 1 -- 172.21.15.7:0/3811948222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe340145e40 msgr2=0x7fe34013dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe33fe00640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe340145e40 0x7fe34013dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.989+0000 7fe33fe00640 1 -- 172.21.15.7:0/3811948222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34013f970 con 0x7fe3400b4870 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.990+0000 7fe33fe00640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3400b4870 0x7fe34013add0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe33002c3f0 tx=0x7fe3300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.990+0000 7fe337e00640 1 -- 172.21.15.7:0/3811948222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330044070 con 0x7fe3400b4870 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.990+0000 7fe337e00640 1 -- 172.21.15.7:0/3811948222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe330032b80 con 0x7fe3400b4870 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.990+0000 7fe337e00640 1 -- 172.21.15.7:0/3811948222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330039070 con 0x7fe3400b4870 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.990+0000 7fe345c00640 1 -- 172.21.15.7:0/3811948222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34013fc50 con 0x7fe3400b4870 2024-04-25T11:15:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.990+0000 7fe345c00640 1 -- 172.21.15.7:0/3811948222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe34013daa0 con 0x7fe3400b4870 2024-04-25T11:15:03.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.992+0000 7fe345c00640 1 -- 172.21.15.7:0/3811948222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe340050740 con 0x7fe3400b4870 2024-04-25T11:15:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.993+0000 7fe337e00640 1 -- 172.21.15.7:0/3811948222 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe33003d070 con 0x7fe3400b4870 2024-04-25T11:15:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.994+0000 7fe337e00640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3200744b0 0x7fe3200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.994+0000 7fe337e00640 1 -- 172.21.15.7:0/3811948222 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5619+0+0 (secure 0 0 0) 0x7fe3300b54e0 con 0x7fe3400b4870 2024-04-25T11:15:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.995+0000 7fe33f400640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3200744b0 0x7fe3200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.995+0000 7fe33f400640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3200744b0 0x7fe3200769c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe3400702b0 tx=0x7fe32c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:03.998+0000 7fe337e00640 1 -- 172.21.15.7:0/3811948222 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe33004d080 con 0x7fe3400b4870 2024-04-25T11:15:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.204+0000 7fe345c00640 1 -- 172.21.15.7:0/3811948222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe3400705c0 con 0x7fe3400b4870 2024-04-25T11:15:04.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.683+0000 7fe337e00640 1 -- 172.21.15.7:0/3811948222 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7fe33007fdc0 con 0x7fe3400b4870 2024-04-25T11:15:04.685 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-25T11:15:04.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.687+0000 7fe345c00640 1 -- 172.21.15.7:0/3811948222 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3200744b0 msgr2=0x7fe3200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:04.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.687+0000 7fe345c00640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3200744b0 0x7fe3200769c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe3400702b0 tx=0x7fe32c009040 comp rx=0 tx=0).stop 2024-04-25T11:15:04.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.687+0000 7fe345c00640 1 -- 172.21.15.7:0/3811948222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3400b4870 msgr2=0x7fe34013add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:04.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.687+0000 7fe345c00640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3400b4870 0x7fe34013add0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe33002c3f0 tx=0x7fe3300046c0 comp rx=0 tx=0).stop 2024-04-25T11:15:04.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.687+0000 7fe345c00640 1 -- 172.21.15.7:0/3811948222 shutdown_connections 2024-04-25T11:15:04.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.687+0000 7fe345c00640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3200744b0 0x7fe3200769c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:04.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.687+0000 7fe345c00640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe340145e40 0x7fe34013dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:04.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.687+0000 7fe345c00640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe340144a80 0x7fe34014ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:04.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.687+0000 7fe345c00640 1 --2- 172.21.15.7:0/3811948222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3400b4870 0x7fe34013add0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:04.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.687+0000 7fe345c00640 1 -- 172.21.15.7:0/3811948222 >> 172.21.15.7:0/3811948222 conn(0x7fe3400740f0 msgr2=0x7fe3401e6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:04.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.688+0000 7fe345c00640 1 -- 172.21.15.7:0/3811948222 shutdown_connections 2024-04-25T11:15:04.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.688+0000 7fe345c00640 1 -- 172.21.15.7:0/3811948222 wait complete. 2024-04-25T11:15:04.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:15:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.989+0000 7facb8c00640 1 Processor -- start 2024-04-25T11:15:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.990+0000 7facb8c00640 1 -- start start 2024-04-25T11:15:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.992+0000 7facb8c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7facb406fab0 0x7facb4168e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.992+0000 7facb8c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facb415e3c0 con 0x7facb415dfb0 2024-04-25T11:15:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.992+0000 7facb8c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facb40cf800 con 0x7facb40b0470 2024-04-25T11:15:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.992+0000 7facb8c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7facb40d1250 con 0x7facb406fab0 2024-04-25T11:15:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.993+0000 7facb2800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7facb406fab0 0x7facb4168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.993+0000 7facb1e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7facb415dfb0 0x7facb40d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58522/0 (socket says 172.21.15.7:58522) 2024-04-25T11:15:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.993+0000 7facb1e00640 1 -- 172.21.15.7:0/2297629846 learned_addr learned my addr 172.21.15.7:0/2297629846 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.993+0000 7facb2800640 1 -- 172.21.15.7:0/2297629846 >> v1:172.21.15.87:6789/0 conn(0x7facb40b0470 legacy=0x7facb406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.993+0000 7facb2800640 1 -- 172.21.15.7:0/2297629846 >> v1:172.21.15.7:6789/0 conn(0x7facb415dfb0 legacy=0x7facb40d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.993+0000 7facb2800640 1 -- 172.21.15.7:0/2297629846 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb40cfaa0 con 0x7facb406fab0 2024-04-25T11:15:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.993+0000 7facb1400640 1 -- 172.21.15.7:0/2297629846 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3734834333 0 0) 0x7facb415e3c0 con 0x7facb415dfb0 2024-04-25T11:15:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.994+0000 7facb2800640 1 --2- 172.21.15.7:0/2297629846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facb406fab0 0x7facb4168e60 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7facb40ceb10 tx=0x7facac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fda2ed5bb0b2950 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.995+0000 7facb1400640 1 -- 172.21.15.7:0/2297629846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac007740 con 0x7facb406fab0 2024-04-25T11:15:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.995+0000 7facb1400640 1 -- 172.21.15.7:0/2297629846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facac0078e0 con 0x7facb406fab0 2024-04-25T11:15:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.995+0000 7facb1400640 1 -- 172.21.15.7:0/2297629846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facac007cb0 con 0x7facb406fab0 2024-04-25T11:15:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.995+0000 7facb8c00640 1 -- 172.21.15.7:0/2297629846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facb406fab0 msgr2=0x7facb4168e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.995+0000 7facb8c00640 1 --2- 172.21.15.7:0/2297629846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facb406fab0 0x7facb4168e60 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7facb40ceb10 tx=0x7facac02a6f0 comp rx=0 tx=0).stop 2024-04-25T11:15:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.996+0000 7facb8c00640 1 -- 172.21.15.7:0/2297629846 shutdown_connections 2024-04-25T11:15:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.996+0000 7facb8c00640 1 --2- 172.21.15.7:0/2297629846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facb406fab0 0x7facb4168e60 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.996+0000 7facb8c00640 1 -- 172.21.15.7:0/2297629846 >> 172.21.15.7:0/2297629846 conn(0x7facb40742c0 msgr2=0x7facb406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.996+0000 7facb8c00640 1 -- 172.21.15.7:0/2297629846 shutdown_connections 2024-04-25T11:15:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.996+0000 7facb8c00640 1 -- 172.21.15.7:0/2297629846 wait complete. 2024-04-25T11:15:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.998+0000 7facb8c00640 1 Processor -- start 2024-04-25T11:15:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:04.998+0000 7facb8c00640 1 -- start start 2024-04-25T11:15:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.000+0000 7facb8c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facb406fab0 0x7facb41e7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.000+0000 7facb8c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facb40b0470 0x7facb41e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.000+0000 7facb8c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb415dfb0 0x7facb41f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.000+0000 7facb8c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7facb40d0340 con 0x7facb40b0470 2024-04-25T11:15:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.000+0000 7facb8c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7facb40d0060 con 0x7facb415dfb0 2024-04-25T11:15:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.000+0000 7facb8c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7facb40d0620 con 0x7facb406fab0 2024-04-25T11:15:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.000+0000 7facabe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facb40b0470 0x7facb41e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.000+0000 7facabe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facb40b0470 0x7facb41e7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41632/0 (socket says 172.21.15.7:41632) 2024-04-25T11:15:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.000+0000 7facabe00640 1 -- 172.21.15.7:0/1584144454 learned_addr learned my addr 172.21.15.7:0/1584144454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.000+0000 7facb2800640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb415dfb0 0x7facb41f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.001+0000 7facb1e00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facb406fab0 0x7facb41e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.001+0000 7facabe00640 1 -- 172.21.15.7:0/1584144454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facb406fab0 msgr2=0x7facb41e7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.001+0000 7facabe00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facb406fab0 0x7facb41e7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.001+0000 7facabe00640 1 -- 172.21.15.7:0/1584144454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb415dfb0 msgr2=0x7facb41f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.001+0000 7facabe00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb415dfb0 0x7facb41f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.001+0000 7facabe00640 1 -- 172.21.15.7:0/1584144454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb40b4b90 con 0x7facb40b0470 2024-04-25T11:15:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.001+0000 7facb1e00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facb406fab0 0x7facb41e7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.001+0000 7facabe00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facb40b0470 0x7facb41e7e50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00c3f0 tx=0x7fac9c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.001+0000 7facaa000640 1 -- 172.21.15.7:0/1584144454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c017070 con 0x7facb40b0470 2024-04-25T11:15:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.001+0000 7facaa000640 1 -- 172.21.15.7:0/1584144454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9c0052d0 con 0x7facb40b0470 2024-04-25T11:15:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.001+0000 7facaa000640 1 -- 172.21.15.7:0/1584144454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c003830 con 0x7facb40b0470 2024-04-25T11:15:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.002+0000 7facb8c00640 1 -- 172.21.15.7:0/1584144454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb41ea0f0 con 0x7facb40b0470 2024-04-25T11:15:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.002+0000 7facb8c00640 1 -- 172.21.15.7:0/1584144454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb41ea6b0 con 0x7facb40b0470 2024-04-25T11:15:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.003+0000 7facb8c00640 1 -- 172.21.15.7:0/1584144454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb40b4d60 con 0x7facb40b0470 2024-04-25T11:15:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.006+0000 7facaa000640 1 -- 172.21.15.7:0/1584144454 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fac9c01a070 con 0x7facb40b0470 2024-04-25T11:15:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.006+0000 7facaa000640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac90074500 0x7fac90076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.006+0000 7facaa000640 1 -- 172.21.15.7:0/1584144454 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6001+0+0 (secure 0 0 0) 0x7fac9c0949b0 con 0x7facb40b0470 2024-04-25T11:15:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.006+0000 7facb1e00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac90074500 0x7fac90076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:05.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.007+0000 7facb1e00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac90074500 0x7fac90076a10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faca4000f60 tx=0x7faca4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:05.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.012+0000 7facaa000640 1 -- 172.21.15.7:0/1584144454 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fac9c021020 con 0x7facb40b0470 2024-04-25T11:15:05.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.256+0000 7facb8c00640 1 -- 172.21.15.7:0/1584144454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7facb4050740 con 0x7facb40b0470 2024-04-25T11:15:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T11:15:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.700+0000 7facaa000640 1 -- 172.21.15.7:0/1584144454 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v40) ==== 174+0+0 (secure 0 0 0) 0x7fac9c05f0f0 con 0x7facb40b0470 2024-04-25T11:15:05.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.712+0000 7facb8c00640 1 -- 172.21.15.7:0/1584144454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac90074500 msgr2=0x7fac90076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:05.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.712+0000 7facb8c00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac90074500 0x7fac90076a10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faca4000f60 tx=0x7faca4009040 comp rx=0 tx=0).stop 2024-04-25T11:15:05.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.712+0000 7facb8c00640 1 -- 172.21.15.7:0/1584144454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facb40b0470 msgr2=0x7facb41e7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:05.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.712+0000 7facb8c00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facb40b0470 0x7facb41e7e50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fac9c00c3f0 tx=0x7fac9c00c8b0 comp rx=0 tx=0).stop 2024-04-25T11:15:05.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.714+0000 7facb8c00640 1 -- 172.21.15.7:0/1584144454 shutdown_connections 2024-04-25T11:15:05.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.714+0000 7facb8c00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fac90074500 0x7fac90076a10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:05.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.714+0000 7facb8c00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facb415dfb0 0x7facb41f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:05.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.714+0000 7facb8c00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facb40b0470 0x7facb41e7e50 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:05.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.714+0000 7facb8c00640 1 --2- 172.21.15.7:0/1584144454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facb406fab0 0x7facb41e7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:05.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.714+0000 7facb8c00640 1 -- 172.21.15.7:0/1584144454 >> 172.21.15.7:0/1584144454 conn(0x7facb40742c0 msgr2=0x7facb40b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:05.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.714+0000 7facb8c00640 1 -- 172.21.15.7:0/1584144454 shutdown_connections 2024-04-25T11:15:05.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:05.715+0000 7facb8c00640 1 -- 172.21.15.7:0/1584144454 wait complete. 2024-04-25T11:15:05.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:15:06.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.005+0000 7f9284400640 1 Processor -- start 2024-04-25T11:15:06.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.005+0000 7f9284400640 1 -- start start 2024-04-25T11:15:06.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.007+0000 7f9284400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f927c06dc90 0x7f927c0afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.008+0000 7f9284400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f927c06f300 0x7f927c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.008+0000 7f9284400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f927c0b4980 con 0x7f927c160a30 2024-04-25T11:15:06.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.008+0000 7f9284400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f927c0b4c60 con 0x7f927c06dc90 2024-04-25T11:15:06.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.008+0000 7f9284400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f927c0b4f40 con 0x7f927c06f300 2024-04-25T11:15:06.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.008+0000 7f9282400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f927c160a30 0x7f927c0ae960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58530/0 (socket says 172.21.15.7:58530) 2024-04-25T11:15:06.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.008+0000 7f9282400640 1 -- 172.21.15.7:0/3943914455 learned_addr learned my addr 172.21.15.7:0/3943914455 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.008+0000 7f9281000640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f927c06f300 0x7f927c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.008+0000 7f9281a00640 1 --2- 172.21.15.7:0/3943914455 >> v2:172.21.15.87:3300/0 conn(0x7f927c06dc90 0x7f927c0afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 621110347 0 0) 0x7f927c0b4980 con 0x7f927c160a30 2024-04-25T11:15:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9264003520 con 0x7f927c160a30 2024-04-25T11:15:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1366190780 0 0) 0x7f9264003520 con 0x7f927c160a30 2024-04-25T11:15:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f927c06f300 msgr2=0x7f927c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f927c06f300 0x7f927c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 >> v2:172.21.15.87:3300/0 conn(0x7f927c06dc90 msgr2=0x7f927c0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 --2- 172.21.15.7:0/3943914455 >> v2:172.21.15.87:3300/0 conn(0x7f927c06dc90 0x7f927c0afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c0b5220 con 0x7f927c160a30 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f92740019f0 con 0x7f927c160a30 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 >> v1:172.21.15.7:6789/0 conn(0x7f927c160a30 legacy=0x7f927c0ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f927c06f300 0x7f92640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9264003f80 0x7f9264004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9264004d40 0x7f9264005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f927c0b4c60 con 0x7f9264004d40 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f927c0b4f40 con 0x7f9264003f80 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.009+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f926400ba70 con 0x7f927c06f300 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.010+0000 7f9282400640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9264004d40 0x7f9264005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.010+0000 7f9281000640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9264003f80 0x7f9264004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.010+0000 7f9281a00640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f927c06f300 0x7f92640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.010+0000 7f9282400640 1 -- 172.21.15.7:0/3943914455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f927c06f300 msgr2=0x7f92640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.010+0000 7f9282400640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f927c06f300 0x7f92640039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.010+0000 7f9282400640 1 -- 172.21.15.7:0/3943914455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9264003f80 msgr2=0x7f9264004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.010+0000 7f9282400640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9264003f80 0x7f9264004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.010+0000 7f9282400640 1 -- 172.21.15.7:0/3943914455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9264003520 con 0x7f9264004d40 2024-04-25T11:15:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.011+0000 7f9282400640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9264004d40 0x7f9264005150 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f927c073280 tx=0x7f9264005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9921b8c0857f9b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.011+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927400a3f0 con 0x7f9264004d40 2024-04-25T11:15:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.011+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f927402f070 con 0x7f9264004d40 2024-04-25T11:15:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.011+0000 7f927be00640 1 -- 172.21.15.7:0/3943914455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927402bc00 con 0x7f9264004d40 2024-04-25T11:15:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.012+0000 7f9284400640 1 -- 172.21.15.7:0/3943914455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9264004d40 msgr2=0x7f9264005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.012+0000 7f9284400640 1 --2- 172.21.15.7:0/3943914455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9264004d40 0x7f9264005150 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f927c073280 tx=0x7f9264005830 comp rx=0 tx=0).stop 2024-04-25T11:15:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.012+0000 7f9282400640 1 -- 172.21.15.7:0/3943914455 reap_dead start 2024-04-25T11:15:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.012+0000 7f9284400640 1 -- 172.21.15.7:0/3943914455 shutdown_connections 2024-04-25T11:15:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.012+0000 7f9284400640 1 -- 172.21.15.7:0/3943914455 >> 172.21.15.7:0/3943914455 conn(0x7f927c074200 msgr2=0x7f927c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.012+0000 7f9284400640 1 -- 172.21.15.7:0/3943914455 shutdown_connections 2024-04-25T11:15:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.012+0000 7f9284400640 1 -- 172.21.15.7:0/3943914455 wait complete. 2024-04-25T11:15:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.015+0000 7f9284400640 1 Processor -- start 2024-04-25T11:15:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.015+0000 7f9284400640 1 -- start start 2024-04-25T11:15:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.016+0000 7f9284400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9264004d40 0x7f927c145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.016+0000 7f9284400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f927c1511c0 0x7f927c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.016+0000 7f9284400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c1515d0 0x7f927c153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.016+0000 7f9284400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f927c0b6a30 con 0x7f927c1515d0 2024-04-25T11:15:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.016+0000 7f9284400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f927c15d220 con 0x7f927c1511c0 2024-04-25T11:15:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.016+0000 7f9284400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f927c0ce920 con 0x7f9264004d40 2024-04-25T11:15:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9282400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c1515d0 0x7f927c153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9282400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c1515d0 0x7f927c153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41652/0 (socket says 172.21.15.7:41652) 2024-04-25T11:15:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9282400640 1 -- 172.21.15.7:0/361347046 learned_addr learned my addr 172.21.15.7:0/361347046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9281000640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f927c1511c0 0x7f927c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9281a00640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9264004d40 0x7f927c145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9282400640 1 -- 172.21.15.7:0/361347046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9264004d40 msgr2=0x7f927c145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9282400640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9264004d40 0x7f927c145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9282400640 1 -- 172.21.15.7:0/361347046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f927c1511c0 msgr2=0x7f927c153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9282400640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f927c1511c0 0x7f927c153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9282400640 1 -- 172.21.15.7:0/361347046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c160140 con 0x7f927c1515d0 2024-04-25T11:15:06.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9281a00640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9264004d40 0x7f927c145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.017+0000 7f9282400640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c1515d0 0x7f927c153890 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f927400a860 tx=0x7f927c0b4b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.018+0000 7f9279600640 1 -- 172.21.15.7:0/361347046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274044070 con 0x7f927c1515d0 2024-04-25T11:15:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.018+0000 7f9279600640 1 -- 172.21.15.7:0/361347046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f927400a3d0 con 0x7f927c1515d0 2024-04-25T11:15:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.018+0000 7f9279600640 1 -- 172.21.15.7:0/361347046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274001f30 con 0x7f927c1515d0 2024-04-25T11:15:06.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.018+0000 7f9284400640 1 -- 172.21.15.7:0/361347046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c15fb70 con 0x7f927c1515d0 2024-04-25T11:15:06.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.018+0000 7f9284400640 1 -- 172.21.15.7:0/361347046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c15f090 con 0x7f927c1515d0 2024-04-25T11:15:06.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.019+0000 7f9284400640 1 -- 172.21.15.7:0/361347046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f927c151b10 con 0x7f927c1515d0 2024-04-25T11:15:06.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.020+0000 7f9279600640 1 -- 172.21.15.7:0/361347046 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9274047070 con 0x7f927c1515d0 2024-04-25T11:15:06.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.021+0000 7f9279600640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f925c0744b0 0x7f925c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.022+0000 7f9279600640 1 -- 172.21.15.7:0/361347046 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f92740b5570 con 0x7f927c1515d0 2024-04-25T11:15:06.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.022+0000 7f9281a00640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f925c0744b0 0x7f925c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.022+0000 7f9281a00640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f925c0744b0 0x7f925c0769c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f927c0730d0 tx=0x7f9270008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:06.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.026+0000 7f9279600640 1 -- 172.21.15.7:0/361347046 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f927c160140 con 0x7f927c1515d0 2024-04-25T11:15:06.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T11:15:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.223+0000 7f9284400640 1 -- 172.21.15.7:0/361347046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f927c06a100 con 0x7f927c1515d0 2024-04-25T11:15:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.224+0000 7f9279600640 1 -- 172.21.15.7:0/361347046 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16862 (secure 0 0 0) 0x7f927407f480 con 0x7f927c1515d0 2024-04-25T11:15:06.225 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:06.225 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":79,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:15:05.259399+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:03.587954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T11:15:04.226794+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:15:06.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 -- 172.21.15.7:0/361347046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f925c0744b0 msgr2=0x7f925c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:06.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f925c0744b0 0x7f925c0769c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f927c0730d0 tx=0x7f9270008040 comp rx=0 tx=0).stop 2024-04-25T11:15:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 -- 172.21.15.7:0/361347046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c1515d0 msgr2=0x7f927c153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c1515d0 0x7f927c153890 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f927400a860 tx=0x7f927c0b4b50 comp rx=0 tx=0).stop 2024-04-25T11:15:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 -- 172.21.15.7:0/361347046 shutdown_connections 2024-04-25T11:15:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f925c0744b0 0x7f925c0769c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f927c1515d0 0x7f927c153890 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f927c1511c0 0x7f927c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 --2- 172.21.15.7:0/361347046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9264004d40 0x7f927c145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 -- 172.21.15.7:0/361347046 >> 172.21.15.7:0/361347046 conn(0x7f927c074200 msgr2=0x7f927c157640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 -- 172.21.15.7:0/361347046 shutdown_connections 2024-04-25T11:15:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.227+0000 7f9284400640 1 -- 172.21.15.7:0/361347046 wait complete. 2024-04-25T11:15:06.252 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:15:06.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.515+0000 7fca7f200640 1 Processor -- start 2024-04-25T11:15:06.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.515+0000 7fca7f200640 1 -- start start 2024-04-25T11:15:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.517+0000 7fca7f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca8006dc70 0x7fca800d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.517+0000 7fca7f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca800d2900 0x7fca800af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.517+0000 7fca7f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca800b2820 con 0x7fca800d24f0 2024-04-25T11:15:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.517+0000 7fca7f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fca800b2260 con 0x7fca800d2900 2024-04-25T11:15:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.517+0000 7fca7f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fca800b2540 con 0x7fca8006dc70 2024-04-25T11:15:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.518+0000 7fca7e800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca800d2900 0x7fca800af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.518+0000 7fca7e800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fca800d2900 0x7fca800af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41656/0 (socket says 172.21.15.7:41656) 2024-04-25T11:15:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.518+0000 7fca7e800640 1 -- 172.21.15.7:0/2652893835 learned_addr learned my addr 172.21.15.7:0/2652893835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:06.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.518+0000 7fca7de00640 1 --2- 172.21.15.7:0/2652893835 >> v2:172.21.15.87:3300/0 conn(0x7fca8006dc70 0x7fca800d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.518+0000 7fca7e800640 1 -- 172.21.15.7:0/2652893835 >> v2:172.21.15.87:3300/0 conn(0x7fca8006dc70 msgr2=0x7fca800d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:06.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.518+0000 7fca7e800640 1 --2- 172.21.15.7:0/2652893835 >> v2:172.21.15.87:3300/0 conn(0x7fca8006dc70 0x7fca800d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.518+0000 7fca7e800640 1 -- 172.21.15.7:0/2652893835 >> v1:172.21.15.87:6789/0 conn(0x7fca800d24f0 legacy=0x7fca800aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:06.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.518+0000 7fca7e800640 1 -- 172.21.15.7:0/2652893835 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8006d120 con 0x7fca800d2900 2024-04-25T11:15:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.520+0000 7fca7e800640 1 --2- 172.21.15.7:0/2652893835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca800d2900 0x7fca800af6b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fca80073f90 tx=0x7fca7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3d760ed32f3186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.520+0000 7fca7ca00640 1 -- 172.21.15.7:0/2652893835 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78037070 con 0x7fca800d2900 2024-04-25T11:15:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.520+0000 7fca7ca00640 1 -- 172.21.15.7:0/2652893835 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca78032440 con 0x7fca800d2900 2024-04-25T11:15:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.520+0000 7fca7ca00640 1 -- 172.21.15.7:0/2652893835 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78032810 con 0x7fca800d2900 2024-04-25T11:15:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.521+0000 7fca7f200640 1 -- 172.21.15.7:0/2652893835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca800d2900 msgr2=0x7fca800af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.521+0000 7fca7f200640 1 --2- 172.21.15.7:0/2652893835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca800d2900 0x7fca800af6b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fca80073f90 tx=0x7fca7802a230 comp rx=0 tx=0).stop 2024-04-25T11:15:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.521+0000 7fca7f200640 1 -- 172.21.15.7:0/2652893835 shutdown_connections 2024-04-25T11:15:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.521+0000 7fca7f200640 1 --2- 172.21.15.7:0/2652893835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca800d2900 0x7fca800af6b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.521+0000 7fca7f200640 1 --2- 172.21.15.7:0/2652893835 >> v2:172.21.15.87:3300/0 conn(0x7fca8006dc70 0x7fca800d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.521+0000 7fca7f200640 1 -- 172.21.15.7:0/2652893835 >> 172.21.15.7:0/2652893835 conn(0x7fca80073b60 msgr2=0x7fca800b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:06.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.521+0000 7fca7f200640 1 -- 172.21.15.7:0/2652893835 shutdown_connections 2024-04-25T11:15:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.521+0000 7fca7f200640 1 -- 172.21.15.7:0/2652893835 wait complete. 2024-04-25T11:15:06.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.524+0000 7fca7f200640 1 Processor -- start 2024-04-25T11:15:06.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.524+0000 7fca7f200640 1 -- start start 2024-04-25T11:15:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.526+0000 7fca7f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca8006dc70 0x7fca80156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.526+0000 7fca7f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca800d24f0 0x7fca80144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.526+0000 7fca7f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca800d2900 0x7fca80158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.526+0000 7fca7f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca800b1ea0 con 0x7fca8006dc70 2024-04-25T11:15:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.526+0000 7fca7f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca8009b990 con 0x7fca800d2900 2024-04-25T11:15:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.526+0000 7fca7f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fca8009b540 con 0x7fca800d24f0 2024-04-25T11:15:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.527+0000 7fca7de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca8006dc70 0x7fca80156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.527+0000 7fca7de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca8006dc70 0x7fca80156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41660/0 (socket says 172.21.15.7:41660) 2024-04-25T11:15:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.527+0000 7fca7de00640 1 -- 172.21.15.7:0/347857467 learned_addr learned my addr 172.21.15.7:0/347857467 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.527+0000 7fca7e800640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca800d2900 0x7fca80158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.527+0000 7fca7de00640 1 -- 172.21.15.7:0/347857467 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca800d24f0 msgr2=0x7fca80144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.527+0000 7fca7d400640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca800d24f0 0x7fca80144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.527+0000 7fca7de00640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca800d24f0 0x7fca80144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.527+0000 7fca7de00640 1 -- 172.21.15.7:0/347857467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca800d2900 msgr2=0x7fca80158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.527+0000 7fca7de00640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca800d2900 0x7fca80158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.527+0000 7fca7de00640 1 -- 172.21.15.7:0/347857467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca80144da0 con 0x7fca8006dc70 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.527+0000 7fca7e800640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca800d2900 0x7fca80158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.528+0000 7fca7de00640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca8006dc70 0x7fca80156590 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fca70007e70 tx=0x7fca7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.528+0000 7fca76000640 1 -- 172.21.15.7:0/347857467 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70014620 con 0x7fca8006dc70 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.528+0000 7fca76000640 1 -- 172.21.15.7:0/347857467 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca70019070 con 0x7fca8006dc70 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.528+0000 7fca7f200640 1 -- 172.21.15.7:0/347857467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca801554f0 con 0x7fca8006dc70 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.528+0000 7fca76000640 1 -- 172.21.15.7:0/347857467 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70014a00 con 0x7fca8006dc70 2024-04-25T11:15:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.528+0000 7fca7f200640 1 -- 172.21.15.7:0/347857467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca80155ab0 con 0x7fca8006dc70 2024-04-25T11:15:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.529+0000 7fca7f200640 1 -- 172.21.15.7:0/347857467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca80050740 con 0x7fca8006dc70 2024-04-25T11:15:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.530+0000 7fca76000640 1 -- 172.21.15.7:0/347857467 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fca70014ba0 con 0x7fca8006dc70 2024-04-25T11:15:06.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.531+0000 7fca76000640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fca5c0744e0 0x7fca5c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.532+0000 7fca76000640 1 -- 172.21.15.7:0/347857467 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7fca70095cf0 con 0x7fca8006dc70 2024-04-25T11:15:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.532+0000 7fca7d400640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fca5c0744e0 0x7fca5c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:06.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.533+0000 7fca7d400640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fca5c0744e0 0x7fca5c0769f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fca68006fd0 tx=0x7fca68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.536+0000 7fca76000640 1 -- 172.21.15.7:0/347857467 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fca70060250 con 0x7fca8006dc70 2024-04-25T11:15:06.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.751+0000 7fca7f200640 1 -- 172.21.15.7:0/347857467 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca801585b0 con 0x7fca8006dc70 2024-04-25T11:15:06.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.753+0000 7fca76000640 1 -- 172.21.15.7:0/347857467 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16849 (secure 0 0 0) 0x7fca7005fb50 con 0x7fca8006dc70 2024-04-25T11:15:06.753 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:06.754 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:15:06.697374+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:03.587954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T11:15:04.226794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:15:06.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.755+0000 7fca7f200640 1 -- 172.21.15.7:0/347857467 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fca5c0744e0 msgr2=0x7fca5c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:06.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.755+0000 7fca7f200640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fca5c0744e0 0x7fca5c0769f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fca68006fd0 tx=0x7fca68008040 comp rx=0 tx=0).stop 2024-04-25T11:15:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.755+0000 7fca7f200640 1 -- 172.21.15.7:0/347857467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca8006dc70 msgr2=0x7fca80156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.755+0000 7fca7f200640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca8006dc70 0x7fca80156590 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fca70007e70 tx=0x7fca7000e4b0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.756+0000 7fca7f200640 1 -- 172.21.15.7:0/347857467 shutdown_connections 2024-04-25T11:15:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.756+0000 7fca7f200640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fca5c0744e0 0x7fca5c0769f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.756+0000 7fca7f200640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca800d2900 0x7fca80158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.756+0000 7fca7f200640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca800d24f0 0x7fca80144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.756+0000 7fca7f200640 1 --2- 172.21.15.7:0/347857467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca8006dc70 0x7fca80156590 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:06.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.756+0000 7fca7f200640 1 -- 172.21.15.7:0/347857467 >> 172.21.15.7:0/347857467 conn(0x7fca80073b60 msgr2=0x7fca8015dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:06.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.756+0000 7fca7f200640 1 -- 172.21.15.7:0/347857467 shutdown_connections 2024-04-25T11:15:06.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:06.756+0000 7fca7f200640 1 -- 172.21.15.7:0/347857467 wait complete. 2024-04-25T11:15:06.783 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:15:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.075+0000 7fda11e00640 1 Processor -- start 2024-04-25T11:15:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.075+0000 7fda11e00640 1 -- start start 2024-04-25T11:15:07.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.077+0000 7fda11e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda0c15dfb0 0x7fda0c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:07.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.077+0000 7fda11e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda0c06fab0 0x7fda0c0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:07.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.077+0000 7fda11e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda0c15e3c0 con 0x7fda0c0d0b40 2024-04-25T11:15:07.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.077+0000 7fda11e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fda0c0b09b0 con 0x7fda0c06fab0 2024-04-25T11:15:07.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.077+0000 7fda11e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fda0c0cf800 con 0x7fda0c15dfb0 2024-04-25T11:15:07.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.078+0000 7fda0b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda0c06fab0 0x7fda0c0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.078+0000 7fda0b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda0c06fab0 0x7fda0c0aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41668/0 (socket says 172.21.15.7:41668) 2024-04-25T11:15:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.078+0000 7fda0b400640 1 -- 172.21.15.7:0/2196618289 learned_addr learned my addr 172.21.15.7:0/2196618289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.078+0000 7fda0aa00640 1 --2- 172.21.15.7:0/2196618289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda0c15dfb0 0x7fda0c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.078+0000 7fda0b400640 1 -- 172.21.15.7:0/2196618289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda0c15dfb0 msgr2=0x7fda0c0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.078+0000 7fda0b400640 1 --2- 172.21.15.7:0/2196618289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda0c15dfb0 0x7fda0c0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.078+0000 7fda0b400640 1 -- 172.21.15.7:0/2196618289 >> v1:172.21.15.87:6789/0 conn(0x7fda0c0d0b40 legacy=0x7fda0c06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.079+0000 7fda0b400640 1 -- 172.21.15.7:0/2196618289 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c0cfaa0 con 0x7fda0c06fab0 2024-04-25T11:15:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.080+0000 7fda0b400640 1 --2- 172.21.15.7:0/2196618289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c06fab0 0x7fda0c0aea90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fda04009e50 tx=0x7fda0402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e58f76f4c5802b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:07.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.080+0000 7fda0a000640 1 -- 172.21.15.7:0/2196618289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda04036070 con 0x7fda0c06fab0 2024-04-25T11:15:07.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.080+0000 7fda0a000640 1 -- 172.21.15.7:0/2196618289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda04031600 con 0x7fda0c06fab0 2024-04-25T11:15:07.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.080+0000 7fda0a000640 1 -- 172.21.15.7:0/2196618289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda040319d0 con 0x7fda0c06fab0 2024-04-25T11:15:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.081+0000 7fda11e00640 1 -- 172.21.15.7:0/2196618289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c06fab0 msgr2=0x7fda0c0aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.081+0000 7fda11e00640 1 --2- 172.21.15.7:0/2196618289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c06fab0 0x7fda0c0aea90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fda04009e50 tx=0x7fda0402a6c0 comp rx=0 tx=0).stop 2024-04-25T11:15:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.081+0000 7fda11e00640 1 -- 172.21.15.7:0/2196618289 shutdown_connections 2024-04-25T11:15:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.081+0000 7fda11e00640 1 --2- 172.21.15.7:0/2196618289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c06fab0 0x7fda0c0aea90 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.081+0000 7fda11e00640 1 --2- 172.21.15.7:0/2196618289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda0c15dfb0 0x7fda0c0b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.081+0000 7fda11e00640 1 -- 172.21.15.7:0/2196618289 >> 172.21.15.7:0/2196618289 conn(0x7fda0c0742c0 msgr2=0x7fda0c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.081+0000 7fda11e00640 1 -- 172.21.15.7:0/2196618289 shutdown_connections 2024-04-25T11:15:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.081+0000 7fda11e00640 1 -- 172.21.15.7:0/2196618289 wait complete. 2024-04-25T11:15:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.083+0000 7fda11e00640 1 Processor -- start 2024-04-25T11:15:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.084+0000 7fda11e00640 1 -- start start 2024-04-25T11:15:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.085+0000 7fda11e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c06fab0 0x7fda0c158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.085+0000 7fda11e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda0c0d0b40 0x7fda0c151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.085+0000 7fda11e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda0c15dfb0 0x7fda0c15c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.085+0000 7fda11e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c0d0340 con 0x7fda0c06fab0 2024-04-25T11:15:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.085+0000 7fda11e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c0d0060 con 0x7fda0c15dfb0 2024-04-25T11:15:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.085+0000 7fda11e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fda0c0d0620 con 0x7fda0c0d0b40 2024-04-25T11:15:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.085+0000 7fda0aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c06fab0 0x7fda0c158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.086+0000 7fda0aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c06fab0 0x7fda0c158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41676/0 (socket says 172.21.15.7:41676) 2024-04-25T11:15:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.086+0000 7fda0aa00640 1 -- 172.21.15.7:0/1892522009 learned_addr learned my addr 172.21.15.7:0/1892522009 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.086+0000 7fda0b400640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda0c15dfb0 0x7fda0c15c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.086+0000 7fda03e00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda0c0d0b40 0x7fda0c151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.086+0000 7fda0aa00640 1 -- 172.21.15.7:0/1892522009 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda0c0d0b40 msgr2=0x7fda0c151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.086+0000 7fda0aa00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda0c0d0b40 0x7fda0c151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.086+0000 7fda0aa00640 1 -- 172.21.15.7:0/1892522009 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda0c15dfb0 msgr2=0x7fda0c15c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.086+0000 7fda0aa00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda0c15dfb0 0x7fda0c15c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.086+0000 7fda0aa00640 1 -- 172.21.15.7:0/1892522009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c1592b0 con 0x7fda0c06fab0 2024-04-25T11:15:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.086+0000 7fda0aa00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c06fab0 0x7fda0c158a70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00d600 tx=0x7fd9fc00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.087+0000 7fda03400640 1 -- 172.21.15.7:0/1892522009 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc004240 con 0x7fda0c06fab0 2024-04-25T11:15:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.087+0000 7fda03400640 1 -- 172.21.15.7:0/1892522009 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9fc0043e0 con 0x7fda0c06fab0 2024-04-25T11:15:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.087+0000 7fda03400640 1 -- 172.21.15.7:0/1892522009 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc003830 con 0x7fda0c06fab0 2024-04-25T11:15:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.087+0000 7fda11e00640 1 -- 172.21.15.7:0/1892522009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c159590 con 0x7fda0c06fab0 2024-04-25T11:15:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.087+0000 7fda11e00640 1 -- 172.21.15.7:0/1892522009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c159b50 con 0x7fda0c06fab0 2024-04-25T11:15:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.088+0000 7fda11e00640 1 -- 172.21.15.7:0/1892522009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c050740 con 0x7fda0c06fab0 2024-04-25T11:15:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.089+0000 7fda03400640 1 -- 172.21.15.7:0/1892522009 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd9fc0039d0 con 0x7fda0c06fab0 2024-04-25T11:15:07.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.090+0000 7fda03400640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9e80744b0 0x7fd9e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:07.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.090+0000 7fda03400640 1 -- 172.21.15.7:0/1892522009 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fd9fc0956f0 con 0x7fda0c06fab0 2024-04-25T11:15:07.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.091+0000 7fda03e00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9e80744b0 0x7fd9e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:07.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.091+0000 7fda03e00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9e80744b0 0x7fd9e80769c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4006fd0 tx=0x7fd9f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.095+0000 7fda03400640 1 -- 172.21.15.7:0/1892522009 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9fc05fce0 con 0x7fda0c06fab0 2024-04-25T11:15:07.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.341+0000 7fda11e00640 1 -- 172.21.15.7:0/1892522009 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fda0c157e70 con 0x7fda0c06fab0 2024-04-25T11:15:07.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.713+0000 7fda03400640 1 -- 172.21.15.7:0/1892522009 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) ==== 118+0+0 (secure 0 0 0) 0x7fd9fc05f5e0 con 0x7fda0c06fab0 2024-04-25T11:15:07.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.717+0000 7fda11e00640 1 -- 172.21.15.7:0/1892522009 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9e80744b0 msgr2=0x7fd9e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.717+0000 7fda11e00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9e80744b0 0x7fd9e80769c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4006fd0 tx=0x7fd9f4008040 comp rx=0 tx=0).stop 2024-04-25T11:15:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.717+0000 7fda11e00640 1 -- 172.21.15.7:0/1892522009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c06fab0 msgr2=0x7fda0c158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.717+0000 7fda11e00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c06fab0 0x7fda0c158a70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc00d600 tx=0x7fd9fc00dac0 comp rx=0 tx=0).stop 2024-04-25T11:15:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.717+0000 7fda11e00640 1 -- 172.21.15.7:0/1892522009 shutdown_connections 2024-04-25T11:15:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.717+0000 7fda11e00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9e80744b0 0x7fd9e80769c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.717+0000 7fda11e00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda0c15dfb0 0x7fda0c15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.717+0000 7fda11e00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda0c0d0b40 0x7fda0c151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.717+0000 7fda11e00640 1 --2- 172.21.15.7:0/1892522009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0c06fab0 0x7fda0c158a70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:07.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.717+0000 7fda11e00640 1 -- 172.21.15.7:0/1892522009 >> 172.21.15.7:0/1892522009 conn(0x7fda0c0742c0 msgr2=0x7fda0c06cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:07.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.718+0000 7fda11e00640 1 -- 172.21.15.7:0/1892522009 shutdown_connections 2024-04-25T11:15:07.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:07.718+0000 7fda11e00640 1 -- 172.21.15.7:0/1892522009 wait complete. 2024-04-25T11:15:07.753 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:15:07.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:15:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.044+0000 7fc996a00640 1 Processor -- start 2024-04-25T11:15:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.044+0000 7fc996a00640 1 -- start start 2024-04-25T11:15:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.046+0000 7fc996a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc99015aaa0 0x7fc9900affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.046+0000 7fc996a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc9900b5020 0x7fc990160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.046+0000 7fc996a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc99006f5b0 con 0x7fc9900d0ee0 2024-04-25T11:15:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.047+0000 7fc996a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc9900af290 con 0x7fc9900b5020 2024-04-25T11:15:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.047+0000 7fc996a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc9900d14a0 con 0x7fc99015aaa0 2024-04-25T11:15:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.047+0000 7fc994a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc9900b5020 0x7fc990160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.047+0000 7fc994a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc9900b5020 0x7fc990160990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41688/0 (socket says 172.21.15.7:41688) 2024-04-25T11:15:08.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.047+0000 7fc994a00640 1 -- 172.21.15.7:0/2272511892 learned_addr learned my addr 172.21.15.7:0/2272511892 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.047+0000 7fc98fe00640 1 --2- 172.21.15.7:0/2272511892 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc99015aaa0 0x7fc9900affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.047+0000 7fc994a00640 1 -- 172.21.15.7:0/2272511892 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc99015aaa0 msgr2=0x7fc9900affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.047+0000 7fc994a00640 1 --2- 172.21.15.7:0/2272511892 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc99015aaa0 0x7fc9900affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.047+0000 7fc994a00640 1 -- 172.21.15.7:0/2272511892 >> v1:172.21.15.87:6789/0 conn(0x7fc9900d0ee0 legacy=0x7fc9900b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.047+0000 7fc994a00640 1 -- 172.21.15.7:0/2272511892 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9900af4e0 con 0x7fc9900b5020 2024-04-25T11:15:08.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.049+0000 7fc994a00640 1 --2- 172.21.15.7:0/2272511892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5020 0x7fc990160990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc988009e50 tx=0x7fc98802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9f8a8e4a0a685 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.049+0000 7fc98f400640 1 -- 172.21.15.7:0/2272511892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988036070 con 0x7fc9900b5020 2024-04-25T11:15:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.049+0000 7fc98f400640 1 -- 172.21.15.7:0/2272511892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc988031600 con 0x7fc9900b5020 2024-04-25T11:15:08.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.049+0000 7fc98f400640 1 -- 172.21.15.7:0/2272511892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9880319d0 con 0x7fc9900b5020 2024-04-25T11:15:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.050+0000 7fc996a00640 1 -- 172.21.15.7:0/2272511892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5020 msgr2=0x7fc990160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.050+0000 7fc996a00640 1 --2- 172.21.15.7:0/2272511892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5020 0x7fc990160990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc988009e50 tx=0x7fc98802a6c0 comp rx=0 tx=0).stop 2024-04-25T11:15:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.050+0000 7fc996a00640 1 -- 172.21.15.7:0/2272511892 shutdown_connections 2024-04-25T11:15:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.050+0000 7fc996a00640 1 --2- 172.21.15.7:0/2272511892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5020 0x7fc990160990 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.050+0000 7fc996a00640 1 --2- 172.21.15.7:0/2272511892 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc99015aaa0 0x7fc9900affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:08.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.050+0000 7fc996a00640 1 -- 172.21.15.7:0/2272511892 >> 172.21.15.7:0/2272511892 conn(0x7fc9900742c0 msgr2=0x7fc99006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.050+0000 7fc996a00640 1 -- 172.21.15.7:0/2272511892 shutdown_connections 2024-04-25T11:15:08.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.050+0000 7fc996a00640 1 -- 172.21.15.7:0/2272511892 wait complete. 2024-04-25T11:15:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.053+0000 7fc996a00640 1 Processor -- start 2024-04-25T11:15:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.053+0000 7fc996a00640 1 -- start start 2024-04-25T11:15:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.054+0000 7fc996a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5020 0x7fc990136420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.054+0000 7fc996a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9900d0ee0 0x7fc990135bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.054+0000 7fc996a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99015aaa0 0x7fc99014cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.054+0000 7fc996a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9900b5430 con 0x7fc9900b5020 2024-04-25T11:15:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.054+0000 7fc996a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc9900afaa0 con 0x7fc99015aaa0 2024-04-25T11:15:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.054+0000 7fc996a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc9900b55b0 con 0x7fc9900d0ee0 2024-04-25T11:15:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.055+0000 7fc98fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5020 0x7fc990136420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.055+0000 7fc98fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5020 0x7fc990136420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41698/0 (socket says 172.21.15.7:41698) 2024-04-25T11:15:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.055+0000 7fc98fe00640 1 -- 172.21.15.7:0/3797245106 learned_addr learned my addr 172.21.15.7:0/3797245106 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.055+0000 7fc994a00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99015aaa0 0x7fc99014cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.055+0000 7fc987400640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9900d0ee0 0x7fc990135bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.055+0000 7fc98fe00640 1 -- 172.21.15.7:0/3797245106 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9900d0ee0 msgr2=0x7fc990135bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.055+0000 7fc98fe00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9900d0ee0 0x7fc990135bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.055+0000 7fc98fe00640 1 -- 172.21.15.7:0/3797245106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99015aaa0 msgr2=0x7fc99014cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.055+0000 7fc98fe00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99015aaa0 0x7fc99014cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.055+0000 7fc98fe00640 1 -- 172.21.15.7:0/3797245106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc990168d30 con 0x7fc9900b5020 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.055+0000 7fc994a00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99015aaa0 0x7fc99014cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.056+0000 7fc98fe00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5020 0x7fc990136420 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d5f0 tx=0x7fc98000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.056+0000 7fc98cc00640 1 -- 172.21.15.7:0/3797245106 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9800125f0 con 0x7fc9900b5020 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.056+0000 7fc98cc00640 1 -- 172.21.15.7:0/3797245106 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc980019070 con 0x7fc9900b5020 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.056+0000 7fc98cc00640 1 -- 172.21.15.7:0/3797245106 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9800129d0 con 0x7fc9900b5020 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.056+0000 7fc996a00640 1 -- 172.21.15.7:0/3797245106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9901fa8d0 con 0x7fc9900b5020 2024-04-25T11:15:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.056+0000 7fc996a00640 1 -- 172.21.15.7:0/3797245106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9901692b0 con 0x7fc9900b5020 2024-04-25T11:15:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.057+0000 7fc996a00640 1 -- 172.21.15.7:0/3797245106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc990157e40 con 0x7fc9900b5020 2024-04-25T11:15:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.059+0000 7fc98cc00640 1 -- 172.21.15.7:0/3797245106 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc980012b70 con 0x7fc9900b5020 2024-04-25T11:15:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.060+0000 7fc98cc00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 0x7fc96c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.060+0000 7fc98cc00640 1 -- 172.21.15.7:0/3797245106 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fc980095810 con 0x7fc9900b5020 2024-04-25T11:15:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.060+0000 7fc987400640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 0x7fc96c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.061+0000 7fc987400640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 0x7fc96c0769c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc978006fd0 tx=0x7fc978008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.065+0000 7fc98cc00640 1 -- 172.21.15.7:0/3797245106 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc990157e40 con 0x7fc9900b5020 2024-04-25T11:15:08.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.312+0000 7fc996a00640 1 -- 172.21.15.7:0/3797245106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc990050740 con 0x7fc9900b5020 2024-04-25T11:15:08.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.728+0000 7fc98cc00640 1 -- 172.21.15.7:0/3797245106 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) ==== 122+0+0 (secure 0 0 0) 0x7fc98005fe00 con 0x7fc9900b5020 2024-04-25T11:15:08.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.732+0000 7fc996a00640 1 -- 172.21.15.7:0/3797245106 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 msgr2=0x7fc96c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:08.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.732+0000 7fc996a00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 0x7fc96c0769c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc978006fd0 tx=0x7fc978008040 comp rx=0 tx=0).stop 2024-04-25T11:15:08.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.732+0000 7fc996a00640 1 -- 172.21.15.7:0/3797245106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5020 msgr2=0x7fc990136420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:08.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.732+0000 7fc996a00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5020 0x7fc990136420 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d5f0 tx=0x7fc98000dab0 comp rx=0 tx=0).stop 2024-04-25T11:15:08.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.732+0000 7fc996a00640 1 -- 172.21.15.7:0/3797245106 shutdown_connections 2024-04-25T11:15:08.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.732+0000 7fc996a00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc96c0744b0 0x7fc96c0769c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:08.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.732+0000 7fc996a00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99015aaa0 0x7fc99014cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:08.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.732+0000 7fc996a00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9900d0ee0 0x7fc990135bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:08.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.732+0000 7fc996a00640 1 --2- 172.21.15.7:0/3797245106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9900b5020 0x7fc990136420 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:08.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.732+0000 7fc996a00640 1 -- 172.21.15.7:0/3797245106 >> 172.21.15.7:0/3797245106 conn(0x7fc9900742c0 msgr2=0x7fc99014ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:08.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.732+0000 7fc996a00640 1 -- 172.21.15.7:0/3797245106 shutdown_connections 2024-04-25T11:15:08.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:08.733+0000 7fc996a00640 1 -- 172.21.15.7:0/3797245106 wait complete. 2024-04-25T11:15:08.767 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:15:08.767 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:15:09.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.053+0000 7f104f600640 1 Processor -- start 2024-04-25T11:15:09.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.053+0000 7f104f600640 1 -- start start 2024-04-25T11:15:09.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.056+0000 7f104f600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10480af540 0x7f104809f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.056+0000 7f104f600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f104815f4e0 con 0x7f104815ef90 2024-04-25T11:15:09.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.056+0000 7f104f600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10480b0560 con 0x7f104815bc40 2024-04-25T11:15:09.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.056+0000 7f104f600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1048071190 con 0x7f10480af540 2024-04-25T11:15:09.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.056+0000 7f104cc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f104815ef90 0x7f10480aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58544/0 (socket says 172.21.15.7:58544) 2024-04-25T11:15:09.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.056+0000 7f104cc00640 1 -- 172.21.15.7:0/1887578259 learned_addr learned my addr 172.21.15.7:0/1887578259 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.057+0000 7f1047e00640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10480af540 0x7f104809f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.057+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1935832971 0 0) 0x7f104815f4e0 con 0x7f104815ef90 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.057+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1030003520 con 0x7f104815ef90 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.057+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 610460391 0 0) 0x7f1030003520 con 0x7f104815ef90 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.057+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10480af540 msgr2=0x7f104809f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.057+0000 7f1047400640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10480af540 0x7f104809f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.057+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 >> v1:172.21.15.87:6789/0 conn(0x7f104815bc40 legacy=0x7f10480cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.057+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10480b0840 con 0x7f104815ef90 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.058+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f103c001670 con 0x7f104815ef90 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.058+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 >> v1:172.21.15.7:6789/0 conn(0x7f104815ef90 legacy=0x7f10480aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.058+0000 7f1047400640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10300039e0 0x7f1030003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.058+0000 7f1047400640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1030004370 0x7f10300048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.058+0000 7f1047400640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10300050c0 0x7f103000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.058+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1048071190 con 0x7f1030004370 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.058+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1030010420 con 0x7f10300050c0 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.058+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1030010620 con 0x7f10300039e0 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.058+0000 7f1047e00640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1030004370 0x7f10300048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.058+0000 7f104cc00640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10300039e0 0x7f1030003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.058+0000 7f104d600640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10300050c0 0x7f103000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.059+0000 7f1047e00640 1 -- 172.21.15.7:0/1887578259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10300039e0 msgr2=0x7f1030003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.059+0000 7f1047e00640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10300039e0 0x7f1030003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.059+0000 7f1047e00640 1 -- 172.21.15.7:0/1887578259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10300050c0 msgr2=0x7f103000fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.059+0000 7f1047e00640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10300050c0 0x7f103000fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.059+0000 7f1047e00640 1 -- 172.21.15.7:0/1887578259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1030003520 con 0x7f1030004370 2024-04-25T11:15:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.060+0000 7f1047e00640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1030004370 0x7f10300048c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1034009ac0 tx=0x7f1030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e62b26fc9ebc92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.060+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034036070 con 0x7f1030004370 2024-04-25T11:15:09.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.060+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1034031400 con 0x7f1030004370 2024-04-25T11:15:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.060+0000 7f1047400640 1 -- 172.21.15.7:0/1887578259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034009690 con 0x7f1030004370 2024-04-25T11:15:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.061+0000 7f104f600640 1 -- 172.21.15.7:0/1887578259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1030004370 msgr2=0x7f10300048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.061+0000 7f104f600640 1 --2- 172.21.15.7:0/1887578259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1030004370 0x7f10300048c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f1034009ac0 tx=0x7f1030005830 comp rx=0 tx=0).stop 2024-04-25T11:15:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.061+0000 7f104d600640 1 -- 172.21.15.7:0/1887578259 reap_dead start 2024-04-25T11:15:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.061+0000 7f104f600640 1 -- 172.21.15.7:0/1887578259 shutdown_connections 2024-04-25T11:15:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.061+0000 7f104f600640 1 -- 172.21.15.7:0/1887578259 >> 172.21.15.7:0/1887578259 conn(0x7f1048074200 msgr2=0x7f104806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.061+0000 7f104f600640 1 -- 172.21.15.7:0/1887578259 shutdown_connections 2024-04-25T11:15:09.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.061+0000 7f104f600640 1 -- 172.21.15.7:0/1887578259 wait complete. 2024-04-25T11:15:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.063+0000 7f104f600640 1 Processor -- start 2024-04-25T11:15:09.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.063+0000 7f104f600640 1 -- start start 2024-04-25T11:15:09.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.065+0000 7f104f600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1030004370 0x7f10481549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.065+0000 7f104f600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f104809ee20 0x7f1048157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.065+0000 7f104f600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1048155bd0 0x7f1048155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.065+0000 7f104f600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f10480b1320 con 0x7f1048155bd0 2024-04-25T11:15:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.065+0000 7f104f600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f10480b0e00 con 0x7f104809ee20 2024-04-25T11:15:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.065+0000 7f104f600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f10480afb80 con 0x7f1030004370 2024-04-25T11:15:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.065+0000 7f104d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1048155bd0 0x7f1048155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.065+0000 7f104d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1048155bd0 0x7f1048155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41706/0 (socket says 172.21.15.7:41706) 2024-04-25T11:15:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.065+0000 7f104d600640 1 -- 172.21.15.7:0/4255368487 learned_addr learned my addr 172.21.15.7:0/4255368487 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.066+0000 7f104cc00640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1030004370 0x7f10481549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.066+0000 7f1047e00640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f104809ee20 0x7f1048157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.066+0000 7f104d600640 1 -- 172.21.15.7:0/4255368487 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1030004370 msgr2=0x7f10481549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.066+0000 7f104d600640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1030004370 0x7f10481549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.066+0000 7f104d600640 1 -- 172.21.15.7:0/4255368487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f104809ee20 msgr2=0x7f1048157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.066+0000 7f104d600640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f104809ee20 0x7f1048157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.066+0000 7f104d600640 1 -- 172.21.15.7:0/4255368487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10480b67e0 con 0x7f1048155bd0 2024-04-25T11:15:09.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.066+0000 7f104d600640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1048155bd0 0x7f1048155fe0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f10480cf130 tx=0x7f104000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:09.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.067+0000 7f1044c00640 1 -- 172.21.15.7:0/4255368487 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040003970 con 0x7f1048155bd0 2024-04-25T11:15:09.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.067+0000 7f1044c00640 1 -- 172.21.15.7:0/4255368487 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1040018070 con 0x7f1048155bd0 2024-04-25T11:15:09.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.067+0000 7f1044c00640 1 -- 172.21.15.7:0/4255368487 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040003d50 con 0x7f1048155bd0 2024-04-25T11:15:09.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.067+0000 7f104f600640 1 -- 172.21.15.7:0/4255368487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10480b6ac0 con 0x7f1048155bd0 2024-04-25T11:15:09.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.067+0000 7f104f600640 1 -- 172.21.15.7:0/4255368487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10481553d0 con 0x7f1048155bd0 2024-04-25T11:15:09.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.068+0000 7f104f600640 1 -- 172.21.15.7:0/4255368487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104806a100 con 0x7f1048155bd0 2024-04-25T11:15:09.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.069+0000 7f1044c00640 1 -- 172.21.15.7:0/4255368487 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f10400050d0 con 0x7f1048155bd0 2024-04-25T11:15:09.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.070+0000 7f1044c00640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 0x7f10280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.071+0000 7f1044c00640 1 -- 172.21.15.7:0/4255368487 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f1040094ef0 con 0x7f1048155bd0 2024-04-25T11:15:09.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.071+0000 7f104cc00640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 0x7f10280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.072+0000 7f104cc00640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 0x7f10280769c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f103c0017e0 tx=0x7f103c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:09.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.075+0000 7f1044c00640 1 -- 172.21.15.7:0/4255368487 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f104005ef30 con 0x7f1048155bd0 2024-04-25T11:15:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.323+0000 7f104f600640 1 -- 172.21.15.7:0/4255368487 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1048072ee0 con 0x7f1048155bd0 2024-04-25T11:15:09.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.324+0000 7f1044c00640 1 -- 172.21.15.7:0/4255368487 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5171 (secure 0 0 0) 0x7f1040062450 con 0x7f1048155bd0 2024-04-25T11:15:09.325 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:09.325 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/779859707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":779859707},{"type":"v1","addr":"172.21.15.87:6834","nonce":779859707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/454388957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":454388957},{"type":"v1","addr":"172.21.15.87:6835","nonce":454388957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2318271315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2318271315},{"type":"v1","addr":"172.21.15.7:6836","nonce":2318271315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:15:05.259317+0000","modified":"2024-04-25T11:15:08.314094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6528},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6528":{"gid":6528,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2955530443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2955530443},{"type":"v1","addr":"172.21.15.7:6837","nonce":2955530443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6528,"qdb_cluster":[6528]},"id":5}]} 2024-04-25T11:15:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 43 2024-04-25T11:15:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.327+0000 7f104f600640 1 -- 172.21.15.7:0/4255368487 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 msgr2=0x7f10280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.327+0000 7f104f600640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 0x7f10280769c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f103c0017e0 tx=0x7f103c007040 comp rx=0 tx=0).stop 2024-04-25T11:15:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.327+0000 7f104f600640 1 -- 172.21.15.7:0/4255368487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1048155bd0 msgr2=0x7f1048155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.327+0000 7f104f600640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1048155bd0 0x7f1048155fe0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f10480cf130 tx=0x7f104000ec10 comp rx=0 tx=0).stop 2024-04-25T11:15:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.327+0000 7f104f600640 1 -- 172.21.15.7:0/4255368487 shutdown_connections 2024-04-25T11:15:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.327+0000 7f104f600640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10280744b0 0x7f10280769c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.327+0000 7f104f600640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1048155bd0 0x7f1048155fe0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.327+0000 7f104f600640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f104809ee20 0x7f1048157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.327+0000 7f104f600640 1 --2- 172.21.15.7:0/4255368487 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1030004370 0x7f10481549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.327+0000 7f104f600640 1 -- 172.21.15.7:0/4255368487 >> 172.21.15.7:0/4255368487 conn(0x7f1048074200 msgr2=0x7f104806d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.327+0000 7f104f600640 1 -- 172.21.15.7:0/4255368487 shutdown_connections 2024-04-25T11:15:09.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.328+0000 7f104f600640 1 -- 172.21.15.7:0/4255368487 wait complete. 2024-04-25T11:15:09.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:15:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.636+0000 7fbb52200640 1 Processor -- start 2024-04-25T11:15:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.636+0000 7fbb52200640 1 -- start start 2024-04-25T11:15:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.638+0000 7fbb52200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c164690 0x7fbb4c070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.638+0000 7fbb52200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbb4c156cd0 0x7fbb4c163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.638+0000 7fbb52200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb4c0b5080 con 0x7fbb4c1570e0 2024-04-25T11:15:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.639+0000 7fbb52200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbb4c0b5360 con 0x7fbb4c156cd0 2024-04-25T11:15:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.639+0000 7fbb52200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbb4c0b5640 con 0x7fbb4c164690 2024-04-25T11:15:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.639+0000 7fbb50e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c164690 0x7fbb4c070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.639+0000 7fbb4be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbb4c156cd0 0x7fbb4c163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.639+0000 7fbb50e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c164690 0x7fbb4c070a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:34294/0 (socket says 172.21.15.7:34294) 2024-04-25T11:15:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.639+0000 7fbb50e00640 1 -- 172.21.15.7:0/845798980 learned_addr learned my addr 172.21.15.7:0/845798980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:09.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.640+0000 7fbb50e00640 1 -- 172.21.15.7:0/845798980 >> v2:172.21.15.87:3300/0 conn(0x7fbb4c156cd0 msgr2=0x7fbb4c163ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:09.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.640+0000 7fbb50e00640 1 --2- 172.21.15.7:0/845798980 >> v2:172.21.15.87:3300/0 conn(0x7fbb4c156cd0 0x7fbb4c163ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.640+0000 7fbb50e00640 1 -- 172.21.15.7:0/845798980 >> v1:172.21.15.87:6789/0 conn(0x7fbb4c1570e0 legacy=0x7fbb4c0b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.640+0000 7fbb50e00640 1 -- 172.21.15.7:0/845798980 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c0b5920 con 0x7fbb4c164690 2024-04-25T11:15:09.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.641+0000 7fbb50e00640 1 --2- 172.21.15.7:0/845798980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c164690 0x7fbb4c070a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009870 tx=0x7fbb4002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2336a5e5250f2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:09.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.642+0000 7fbb4b400640 1 -- 172.21.15.7:0/845798980 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40037070 con 0x7fbb4c164690 2024-04-25T11:15:09.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.642+0000 7fbb4b400640 1 -- 172.21.15.7:0/845798980 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb40032440 con 0x7fbb4c164690 2024-04-25T11:15:09.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.642+0000 7fbb4b400640 1 -- 172.21.15.7:0/845798980 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb40032810 con 0x7fbb4c164690 2024-04-25T11:15:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.643+0000 7fbb52200640 1 -- 172.21.15.7:0/845798980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c164690 msgr2=0x7fbb4c070a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.643+0000 7fbb52200640 1 --2- 172.21.15.7:0/845798980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c164690 0x7fbb4c070a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009870 tx=0x7fbb4002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.643+0000 7fbb52200640 1 -- 172.21.15.7:0/845798980 shutdown_connections 2024-04-25T11:15:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.643+0000 7fbb52200640 1 --2- 172.21.15.7:0/845798980 >> v2:172.21.15.87:3300/0 conn(0x7fbb4c156cd0 0x7fbb4c163ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.643+0000 7fbb52200640 1 --2- 172.21.15.7:0/845798980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c164690 0x7fbb4c070a20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.643+0000 7fbb52200640 1 -- 172.21.15.7:0/845798980 >> 172.21.15.7:0/845798980 conn(0x7fbb4c0b3820 msgr2=0x7fbb4c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.643+0000 7fbb52200640 1 -- 172.21.15.7:0/845798980 shutdown_connections 2024-04-25T11:15:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.643+0000 7fbb52200640 1 -- 172.21.15.7:0/845798980 wait complete. 2024-04-25T11:15:09.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.645+0000 7fbb52200640 1 Processor -- start 2024-04-25T11:15:09.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.645+0000 7fbb52200640 1 -- start start 2024-04-25T11:15:09.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.647+0000 7fbb52200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c156cd0 0x7fbb4c1fa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.647+0000 7fbb52200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb4c1570e0 0x7fbb4c1f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.647+0000 7fbb52200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c153960 0x7fbb4c1f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.647+0000 7fbb52200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c164010 con 0x7fbb4c153960 2024-04-25T11:15:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.647+0000 7fbb52200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4c070f60 con 0x7fbb4c1570e0 2024-04-25T11:15:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.647+0000 7fbb52200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbb4c164190 con 0x7fbb4c156cd0 2024-04-25T11:15:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.647+0000 7fbb51800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c153960 0x7fbb4c1f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.647+0000 7fbb51800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c153960 0x7fbb4c1f9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41720/0 (socket says 172.21.15.7:41720) 2024-04-25T11:15:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.647+0000 7fbb51800640 1 -- 172.21.15.7:0/1683685473 learned_addr learned my addr 172.21.15.7:0/1683685473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.648+0000 7fbb4be00640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb4c1570e0 0x7fbb4c1f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.648+0000 7fbb50e00640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c156cd0 0x7fbb4c1fa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.648+0000 7fbb51800640 1 -- 172.21.15.7:0/1683685473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c156cd0 msgr2=0x7fbb4c1fa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.648+0000 7fbb51800640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c156cd0 0x7fbb4c1fa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.648+0000 7fbb51800640 1 -- 172.21.15.7:0/1683685473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb4c1570e0 msgr2=0x7fbb4c1f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.648+0000 7fbb51800640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb4c1570e0 0x7fbb4c1f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.648+0000 7fbb51800640 1 -- 172.21.15.7:0/1683685473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4c15f2b0 con 0x7fbb4c153960 2024-04-25T11:15:09.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.648+0000 7fbb51800640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c153960 0x7fbb4c1f9e30 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400c730 tx=0x7fbb4400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:09.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.649+0000 7fbb48c00640 1 -- 172.21.15.7:0/1683685473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44016020 con 0x7fbb4c153960 2024-04-25T11:15:09.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.649+0000 7fbb48c00640 1 -- 172.21.15.7:0/1683685473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb44006d50 con 0x7fbb4c153960 2024-04-25T11:15:09.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.649+0000 7fbb48c00640 1 -- 172.21.15.7:0/1683685473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44003bb0 con 0x7fbb4c153960 2024-04-25T11:15:09.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.649+0000 7fbb52200640 1 -- 172.21.15.7:0/1683685473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4c15f590 con 0x7fbb4c153960 2024-04-25T11:15:09.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.649+0000 7fbb52200640 1 -- 172.21.15.7:0/1683685473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4c158a40 con 0x7fbb4c153960 2024-04-25T11:15:09.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.650+0000 7fbb52200640 1 -- 172.21.15.7:0/1683685473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb4c06fd80 con 0x7fbb4c153960 2024-04-25T11:15:09.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.651+0000 7fbb48c00640 1 -- 172.21.15.7:0/1683685473 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbb440050d0 con 0x7fbb4c153960 2024-04-25T11:15:09.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.652+0000 7fbb48c00640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb300744e0 0x7fbb300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:09.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.652+0000 7fbb48c00640 1 -- 172.21.15.7:0/1683685473 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fbb44094da0 con 0x7fbb4c153960 2024-04-25T11:15:09.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.653+0000 7fbb50e00640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb300744e0 0x7fbb300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:09.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.653+0000 7fbb50e00640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb300744e0 0x7fbb300769f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbb40034000 tx=0x7fbb4002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:09.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.657+0000 7fbb48c00640 1 -- 172.21.15.7:0/1683685473 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb4c06fd80 con 0x7fbb4c153960 2024-04-25T11:15:09.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.865+0000 7fbb52200640 1 -- 172.21.15.7:0/1683685473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb4c158ca0 con 0x7fbb4c153960 2024-04-25T11:15:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.868+0000 7fbb48c00640 1 -- 172.21.15.7:0/1683685473 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16849 (secure 0 0 0) 0x7fbb44020020 con 0x7fbb4c153960 2024-04-25T11:15:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:09.869 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:15:06.697374+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:03.587954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T11:15:04.226794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:15:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 -- 172.21.15.7:0/1683685473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb300744e0 msgr2=0x7fbb300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb300744e0 0x7fbb300769f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbb40034000 tx=0x7fbb4002e000 comp rx=0 tx=0).stop 2024-04-25T11:15:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 -- 172.21.15.7:0/1683685473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c153960 msgr2=0x7fbb4c1f9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c153960 0x7fbb4c1f9e30 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400c730 tx=0x7fbb4400cbf0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 -- 172.21.15.7:0/1683685473 shutdown_connections 2024-04-25T11:15:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb300744e0 0x7fbb300769f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb4c153960 0x7fbb4c1f9e30 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb4c1570e0 0x7fbb4c1f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 --2- 172.21.15.7:0/1683685473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb4c156cd0 0x7fbb4c1fa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 -- 172.21.15.7:0/1683685473 >> 172.21.15.7:0/1683685473 conn(0x7fbb4c0b3820 msgr2=0x7fbb4c155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 -- 172.21.15.7:0/1683685473 shutdown_connections 2024-04-25T11:15:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:09.871+0000 7fbb52200640 1 -- 172.21.15.7:0/1683685473 wait complete. 2024-04-25T11:15:09.902 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:15:09.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:15:10.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.184+0000 7f6c5ec00640 1 Processor -- start 2024-04-25T11:15:10.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.184+0000 7f6c5ec00640 1 -- start start 2024-04-25T11:15:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.186+0000 7f6c5ec00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c580b1ae0 0x7f6c580b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.186+0000 7f6c5ec00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6c5809b620 0x7f6c5809ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.186+0000 7f6c5ec00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c5806c670 0x7f6c5806ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.187+0000 7f6c5ec00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6c5815bdf0 con 0x7f6c580b1ae0 2024-04-25T11:15:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.187+0000 7f6c5ec00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6c580b2430 con 0x7f6c5809b620 2024-04-25T11:15:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.187+0000 7f6c5ec00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5815e590 con 0x7f6c5806c670 2024-04-25T11:15:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.187+0000 7f6c5d800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c580b1ae0 0x7f6c580b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.187+0000 7f6c5d800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c580b1ae0 0x7f6c580b1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40662/0 (socket says 172.21.15.7:40662) 2024-04-25T11:15:10.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.187+0000 7f6c5d800640 1 -- 172.21.15.7:0/4094898041 learned_addr learned my addr 172.21.15.7:0/4094898041 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.187+0000 7f6c5ce00640 1 --2- 172.21.15.7:0/4094898041 >> v2:172.21.15.87:3300/0 conn(0x7f6c5809b620 0x7f6c5809ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.187+0000 7f6c5e200640 1 --2- 172.21.15.7:0/4094898041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c5806c670 0x7f6c5806ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.188+0000 7f6c5d800640 1 -- 172.21.15.7:0/4094898041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c5806c670 msgr2=0x7f6c5806ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.188+0000 7f6c5d800640 1 --2- 172.21.15.7:0/4094898041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c5806c670 0x7f6c5806ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.188+0000 7f6c5d800640 1 -- 172.21.15.7:0/4094898041 >> v2:172.21.15.87:3300/0 conn(0x7f6c5809b620 msgr2=0x7f6c5809ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.188+0000 7f6c5d800640 1 --2- 172.21.15.7:0/4094898041 >> v2:172.21.15.87:3300/0 conn(0x7f6c5809b620 0x7f6c5809ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.188+0000 7f6c5d800640 1 -- 172.21.15.7:0/4094898041 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5815e870 con 0x7f6c580b1ae0 2024-04-25T11:15:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.188+0000 7f6c5ce00640 1 --2- 172.21.15.7:0/4094898041 >> v2:172.21.15.87:3300/0 conn(0x7f6c5809b620 0x7f6c5809ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:15:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.188+0000 7f6c5e200640 1 --2- 172.21.15.7:0/4094898041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c5806c670 0x7f6c5806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:15:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.189+0000 7f6c5d800640 1 --2- 172.21.15.7:0/4094898041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c580b1ae0 0x7f6c580b1ef0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6c58074cd0 tx=0x7f6c4802a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdcdaafd9c101dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:10.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.190+0000 7f6c57e00640 1 -- 172.21.15.7:0/4094898041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48036070 con 0x7f6c580b1ae0 2024-04-25T11:15:10.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.190+0000 7f6c57e00640 1 -- 172.21.15.7:0/4094898041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c48031600 con 0x7f6c580b1ae0 2024-04-25T11:15:10.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.190+0000 7f6c57e00640 1 -- 172.21.15.7:0/4094898041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c480319d0 con 0x7f6c580b1ae0 2024-04-25T11:15:10.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.190+0000 7f6c5ec00640 1 -- 172.21.15.7:0/4094898041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c580b1ae0 msgr2=0x7f6c580b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:10.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.190+0000 7f6c5ec00640 1 --2- 172.21.15.7:0/4094898041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c580b1ae0 0x7f6c580b1ef0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6c58074cd0 tx=0x7f6c4802a600 comp rx=0 tx=0).stop 2024-04-25T11:15:10.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.191+0000 7f6c5ec00640 1 -- 172.21.15.7:0/4094898041 shutdown_connections 2024-04-25T11:15:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.191+0000 7f6c5ec00640 1 --2- 172.21.15.7:0/4094898041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c5806c670 0x7f6c5806ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.191+0000 7f6c5ec00640 1 --2- 172.21.15.7:0/4094898041 >> v2:172.21.15.87:3300/0 conn(0x7f6c5809b620 0x7f6c5809ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.191+0000 7f6c5ec00640 1 --2- 172.21.15.7:0/4094898041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c580b1ae0 0x7f6c580b1ef0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.191+0000 7f6c5ec00640 1 -- 172.21.15.7:0/4094898041 >> 172.21.15.7:0/4094898041 conn(0x7f6c58074410 msgr2=0x7f6c580b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.191+0000 7f6c5ec00640 1 -- 172.21.15.7:0/4094898041 shutdown_connections 2024-04-25T11:15:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.191+0000 7f6c5ec00640 1 -- 172.21.15.7:0/4094898041 wait complete. 2024-04-25T11:15:10.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.193+0000 7f6c5ec00640 1 Processor -- start 2024-04-25T11:15:10.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.193+0000 7f6c5ec00640 1 -- start start 2024-04-25T11:15:10.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.195+0000 7f6c5ec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c5806c670 0x7f6c58157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.195+0000 7f6c5ec00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c5809b620 0x7f6c5809cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.195+0000 7f6c5ec00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c580b1ae0 0x7f6c5809d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.195+0000 7f6c5ec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5815f110 con 0x7f6c5806c670 2024-04-25T11:15:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.195+0000 7f6c5ec00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5815ee30 con 0x7f6c5809b620 2024-04-25T11:15:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.195+0000 7f6c5ec00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5809bf70 con 0x7f6c580b1ae0 2024-04-25T11:15:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.195+0000 7f6c5d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c5806c670 0x7f6c58157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.195+0000 7f6c5d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c5806c670 0x7f6c58157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40678/0 (socket says 172.21.15.7:40678) 2024-04-25T11:15:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.195+0000 7f6c5d800640 1 -- 172.21.15.7:0/2773551481 learned_addr learned my addr 172.21.15.7:0/2773551481 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c5ce00640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c5809b620 0x7f6c5809cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c5e200640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c580b1ae0 0x7f6c5809d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c5d800640 1 -- 172.21.15.7:0/2773551481 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c580b1ae0 msgr2=0x7f6c5809d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:10.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c5d800640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c580b1ae0 0x7f6c5809d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c5d800640 1 -- 172.21.15.7:0/2773551481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c5809b620 msgr2=0x7f6c5809cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:10.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c5d800640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c5809b620 0x7f6c5809cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c5d800640 1 -- 172.21.15.7:0/2773551481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5809f7d0 con 0x7f6c5806c670 2024-04-25T11:15:10.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c5ce00640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c5809b620 0x7f6c5809cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:10.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c5d800640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c5806c670 0x7f6c58157380 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6c4802abf0 tx=0x7f6c48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:10.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c55600640 1 -- 172.21.15.7:0/2773551481 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4803f020 con 0x7f6c5806c670 2024-04-25T11:15:10.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c55600640 1 -- 172.21.15.7:0/2773551481 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c48037040 con 0x7f6c5806c670 2024-04-25T11:15:10.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c55600640 1 -- 172.21.15.7:0/2773551481 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48036070 con 0x7f6c5806c670 2024-04-25T11:15:10.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.196+0000 7f6c5ec00640 1 -- 172.21.15.7:0/2773551481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5809fab0 con 0x7f6c5806c670 2024-04-25T11:15:10.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.197+0000 7f6c5ec00640 1 -- 172.21.15.7:0/2773551481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c580a0070 con 0x7f6c5806c670 2024-04-25T11:15:10.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.198+0000 7f6c5ec00640 1 -- 172.21.15.7:0/2773551481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c58050740 con 0x7f6c5806c670 2024-04-25T11:15:10.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.199+0000 7f6c55600640 1 -- 172.21.15.7:0/2773551481 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6c48042020 con 0x7f6c5806c670 2024-04-25T11:15:10.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.200+0000 7f6c55600640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c380744e0 0x7f6c380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.200+0000 7f6c55600640 1 -- 172.21.15.7:0/2773551481 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6c480b5c20 con 0x7f6c5806c670 2024-04-25T11:15:10.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.200+0000 7f6c5ce00640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c380744e0 0x7f6c380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.201+0000 7f6c5ce00640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c380744e0 0x7f6c380769f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6c5815b160 tx=0x7f6c44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:10.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.205+0000 7f6c55600640 1 -- 172.21.15.7:0/2773551481 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c48049020 con 0x7f6c5806c670 2024-04-25T11:15:10.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.404+0000 7f6c5ec00640 1 -- 172.21.15.7:0/2773551481 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c581578c0 con 0x7f6c380744e0 2024-04-25T11:15:10.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.414+0000 7f6c55600640 1 -- 172.21.15.7:0/2773551481 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252030 (secure 0 0 0) 0x7f6c44009d70 con 0x7f6c380744e0 2024-04-25T11:15:10.415 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:10.419 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:15:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 -- 172.21.15.7:0/2773551481 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c380744e0 msgr2=0x7f6c380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c380744e0 0x7f6c380769f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6c5815b160 tx=0x7f6c44008040 comp rx=0 tx=0).stop 2024-04-25T11:15:10.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 -- 172.21.15.7:0/2773551481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c5806c670 msgr2=0x7f6c58157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c5806c670 0x7f6c58157380 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6c4802abf0 tx=0x7f6c48004a40 comp rx=0 tx=0).stop 2024-04-25T11:15:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 -- 172.21.15.7:0/2773551481 shutdown_connections 2024-04-25T11:15:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c380744e0 0x7f6c380769f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c580b1ae0 0x7f6c5809d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c5809b620 0x7f6c5809cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 --2- 172.21.15.7:0/2773551481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c5806c670 0x7f6c58157380 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 -- 172.21.15.7:0/2773551481 >> 172.21.15.7:0/2773551481 conn(0x7f6c58074410 msgr2=0x7f6c5815bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 -- 172.21.15.7:0/2773551481 shutdown_connections 2024-04-25T11:15:10.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.421+0000 7f6c5ec00640 1 -- 172.21.15.7:0/2773551481 wait complete. 2024-04-25T11:15:10.450 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-04-25T11:15:09.891238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258168,"kb_used_data":8700,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716552,"statfs":{"total":773094113280,"available":772829749248,"internally_reserved":0,"allocated":8908800,"data_stored":4667007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001506"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.716472+0000","last_change":"2024-04-25T11:15:05.710923+0000","last_active":"2024-04-25T11:15:05.716472+0000","last_peered":"2024-04-25T11:15:05.716472+0000","last_clean":"2024-04-25T11:15:05.716472+0000","last_became_active":"2024-04-25T11:15:05.708870+0000","last_became_peered":"2024-04-25T11:15:05.708870+0000","last_unstale":"2024-04-25T11:15:05.716472+0000","last_undegraded":"2024-04-25T11:15:05.716472+0000","last_fullsized":"2024-04-25T11:15:05.716472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:07.854228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.724215+0000","last_change":"2024-04-25T11:15:05.743495+0000","last_active":"2024-04-25T11:15:06.724215+0000","last_peered":"2024-04-25T11:15:06.724215+0000","last_clean":"2024-04-25T11:15:06.724215+0000","last_became_active":"2024-04-25T11:15:05.743186+0000","last_became_peered":"2024-04-25T11:15:05.743186+0000","last_unstale":"2024-04-25T11:15:06.724215+0000","last_undegraded":"2024-04-25T11:15:06.724215+0000","last_fullsized":"2024-04-25T11:15:06.724215+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:20.076047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.839086+0000","last_change":"2024-04-25T11:15:05.731046+0000","last_active":"2024-04-25T11:15:06.839086+0000","last_peered":"2024-04-25T11:15:06.839086+0000","last_clean":"2024-04-25T11:15:06.839086+0000","last_became_active":"2024-04-25T11:15:05.730705+0000","last_became_peered":"2024-04-25T11:15:05.730705+0000","last_unstale":"2024-04-25T11:15:06.839086+0000","last_undegraded":"2024-04-25T11:15:06.839086+0000","last_fullsized":"2024-04-25T11:15:06.839086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:07.009824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719802+0000","last_change":"2024-04-25T11:15:05.714141+0000","last_active":"2024-04-25T11:15:05.719802+0000","last_peered":"2024-04-25T11:15:05.719802+0000","last_clean":"2024-04-25T11:15:05.719802+0000","last_became_active":"2024-04-25T11:15:05.713721+0000","last_became_peered":"2024-04-25T11:15:05.713721+0000","last_unstale":"2024-04-25T11:15:05.719802+0000","last_undegraded":"2024-04-25T11:15:05.719802+0000","last_fullsized":"2024-04-25T11:15:05.719802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:49.369927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.730495+0000","last_change":"2024-04-25T11:15:05.708290+0000","last_active":"2024-04-25T11:15:06.730495+0000","last_peered":"2024-04-25T11:15:06.730495+0000","last_clean":"2024-04-25T11:15:06.730495+0000","last_became_active":"2024-04-25T11:15:05.707988+0000","last_became_peered":"2024-04-25T11:15:05.707988+0000","last_unstale":"2024-04-25T11:15:06.730495+0000","last_undegraded":"2024-04-25T11:15:06.730495+0000","last_fullsized":"2024-04-25T11:15:06.730495+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:07.300246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837655+0000","last_change":"2024-04-25T11:15:05.725952+0000","last_active":"2024-04-25T11:15:06.837655+0000","last_peered":"2024-04-25T11:15:06.837655+0000","last_clean":"2024-04-25T11:15:06.837655+0000","last_became_active":"2024-04-25T11:15:05.725015+0000","last_became_peered":"2024-04-25T11:15:05.725015+0000","last_unstale":"2024-04-25T11:15:06.837655+0000","last_undegraded":"2024-04-25T11:15:06.837655+0000","last_fullsized":"2024-04-25T11:15:06.837655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:56.966887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719300+0000","last_change":"2024-04-25T11:15:05.714760+0000","last_active":"2024-04-25T11:15:05.719300+0000","last_peered":"2024-04-25T11:15:05.719300+0000","last_clean":"2024-04-25T11:15:05.719300+0000","last_became_active":"2024-04-25T11:15:05.713302+0000","last_became_peered":"2024-04-25T11:15:05.713302+0000","last_unstale":"2024-04-25T11:15:05.719300+0000","last_undegraded":"2024-04-25T11:15:05.719300+0000","last_fullsized":"2024-04-25T11:15:05.719300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:38.194245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.729846+0000","last_change":"2024-04-25T11:15:05.710077+0000","last_active":"2024-04-25T11:15:06.729846+0000","last_peered":"2024-04-25T11:15:06.729846+0000","last_clean":"2024-04-25T11:15:06.729846+0000","last_became_active":"2024-04-25T11:15:05.709616+0000","last_became_peered":"2024-04-25T11:15:05.709616+0000","last_unstale":"2024-04-25T11:15:06.729846+0000","last_undegraded":"2024-04-25T11:15:06.729846+0000","last_fullsized":"2024-04-25T11:15:06.729846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:53.888139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.723091+0000","last_change":"2024-04-25T11:15:05.719661+0000","last_active":"2024-04-25T11:15:05.723091+0000","last_peered":"2024-04-25T11:15:05.723091+0000","last_clean":"2024-04-25T11:15:05.723091+0000","last_became_active":"2024-04-25T11:15:05.719241+0000","last_became_peered":"2024-04-25T11:15:05.719241+0000","last_unstale":"2024-04-25T11:15:05.723091+0000","last_undegraded":"2024-04-25T11:15:05.723091+0000","last_fullsized":"2024-04-25T11:15:05.723091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:23.739615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719765+0000","last_change":"2024-04-25T11:15:05.711535+0000","last_active":"2024-04-25T11:15:05.719765+0000","last_peered":"2024-04-25T11:15:05.719765+0000","last_clean":"2024-04-25T11:15:05.719765+0000","last_became_active":"2024-04-25T11:15:05.708619+0000","last_became_peered":"2024-04-25T11:15:05.708619+0000","last_unstale":"2024-04-25T11:15:05.719765+0000","last_undegraded":"2024-04-25T11:15:05.719765+0000","last_fullsized":"2024-04-25T11:15:05.719765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:38.244198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.723311+0000","last_change":"2024-04-25T11:15:05.742150+0000","last_active":"2024-04-25T11:15:06.723311+0000","last_peered":"2024-04-25T11:15:06.723311+0000","last_clean":"2024-04-25T11:15:06.723311+0000","last_became_active":"2024-04-25T11:15:05.741608+0000","last_became_peered":"2024-04-25T11:15:05.741608+0000","last_unstale":"2024-04-25T11:15:06.723311+0000","last_undegraded":"2024-04-25T11:15:06.723311+0000","last_fullsized":"2024-04-25T11:15:06.723311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:59.489824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.735593+0000","last_change":"2024-04-25T11:15:05.732537+0000","last_active":"2024-04-25T11:15:05.735593+0000","last_peered":"2024-04-25T11:15:05.735593+0000","last_clean":"2024-04-25T11:15:05.735593+0000","last_became_active":"2024-04-25T11:15:05.732190+0000","last_became_peered":"2024-04-25T11:15:05.732190+0000","last_unstale":"2024-04-25T11:15:05.735593+0000","last_undegraded":"2024-04-25T11:15:05.735593+0000","last_fullsized":"2024-04-25T11:15:05.735593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:56.693917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.730546+0000","last_change":"2024-04-25T11:15:05.729218+0000","last_active":"2024-04-25T11:15:06.730546+0000","last_peered":"2024-04-25T11:15:06.730546+0000","last_clean":"2024-04-25T11:15:06.730546+0000","last_became_active":"2024-04-25T11:15:05.728877+0000","last_became_peered":"2024-04-25T11:15:05.728877+0000","last_unstale":"2024-04-25T11:15:06.730546+0000","last_undegraded":"2024-04-25T11:15:06.730546+0000","last_fullsized":"2024-04-25T11:15:06.730546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:39.558736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837911+0000","last_change":"2024-04-25T11:15:05.726218+0000","last_active":"2024-04-25T11:15:06.837911+0000","last_peered":"2024-04-25T11:15:06.837911+0000","last_clean":"2024-04-25T11:15:06.837911+0000","last_became_active":"2024-04-25T11:15:05.725450+0000","last_became_peered":"2024-04-25T11:15:05.725450+0000","last_unstale":"2024-04-25T11:15:06.837911+0000","last_undegraded":"2024-04-25T11:15:06.837911+0000","last_fullsized":"2024-04-25T11:15:06.837911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:01.008689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722356+0000","last_change":"2024-04-25T11:15:05.731102+0000","last_active":"2024-04-25T11:15:06.722356+0000","last_peered":"2024-04-25T11:15:06.722356+0000","last_clean":"2024-04-25T11:15:06.722356+0000","last_became_active":"2024-04-25T11:15:05.730391+0000","last_became_peered":"2024-04-25T11:15:05.730391+0000","last_unstale":"2024-04-25T11:15:06.722356+0000","last_undegraded":"2024-04-25T11:15:06.722356+0000","last_fullsized":"2024-04-25T11:15:06.722356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:41.946329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.710232+0000","last_change":"2024-04-25T11:15:05.706747+0000","last_active":"2024-04-25T11:15:05.710232+0000","last_peered":"2024-04-25T11:15:05.710232+0000","last_clean":"2024-04-25T11:15:05.710232+0000","last_became_active":"2024-04-25T11:15:05.706369+0000","last_became_peered":"2024-04-25T11:15:05.706369+0000","last_unstale":"2024-04-25T11:15:05.710232+0000","last_undegraded":"2024-04-25T11:15:05.710232+0000","last_fullsized":"2024-04-25T11:15:05.710232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:05.053385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731467+0000","last_change":"2024-04-25T11:15:05.712366+0000","last_active":"2024-04-25T11:15:06.731467+0000","last_peered":"2024-04-25T11:15:06.731467+0000","last_clean":"2024-04-25T11:15:06.731467+0000","last_became_active":"2024-04-25T11:15:05.711969+0000","last_became_peered":"2024-04-25T11:15:05.711969+0000","last_unstale":"2024-04-25T11:15:06.731467+0000","last_undegraded":"2024-04-25T11:15:06.731467+0000","last_fullsized":"2024-04-25T11:15:06.731467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:49.831747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.839229+0000","last_change":"2024-04-25T11:15:05.710484+0000","last_active":"2024-04-25T11:15:06.839229+0000","last_peered":"2024-04-25T11:15:06.839229+0000","last_clean":"2024-04-25T11:15:06.839229+0000","last_became_active":"2024-04-25T11:15:05.710158+0000","last_became_peered":"2024-04-25T11:15:05.710158+0000","last_unstale":"2024-04-25T11:15:06.839229+0000","last_undegraded":"2024-04-25T11:15:06.839229+0000","last_fullsized":"2024-04-25T11:15:06.839229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:15.866515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.836888+0000","last_change":"2024-04-25T11:15:05.720447+0000","last_active":"2024-04-25T11:15:06.836888+0000","last_peered":"2024-04-25T11:15:06.836888+0000","last_clean":"2024-04-25T11:15:06.836888+0000","last_became_active":"2024-04-25T11:15:05.720092+0000","last_became_peered":"2024-04-25T11:15:05.720092+0000","last_unstale":"2024-04-25T11:15:06.836888+0000","last_undegraded":"2024-04-25T11:15:06.836888+0000","last_fullsized":"2024-04-25T11:15:06.836888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:27.480283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731157+0000","last_change":"2024-04-25T11:15:05.713781+0000","last_active":"2024-04-25T11:15:06.731157+0000","last_peered":"2024-04-25T11:15:06.731157+0000","last_clean":"2024-04-25T11:15:06.731157+0000","last_became_active":"2024-04-25T11:15:05.713402+0000","last_became_peered":"2024-04-25T11:15:05.713402+0000","last_unstale":"2024-04-25T11:15:06.731157+0000","last_undegraded":"2024-04-25T11:15:06.731157+0000","last_fullsized":"2024-04-25T11:15:06.731157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:56.709384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.720023+0000","last_change":"2024-04-25T11:15:05.714882+0000","last_active":"2024-04-25T11:15:05.720023+0000","last_peered":"2024-04-25T11:15:05.720023+0000","last_clean":"2024-04-25T11:15:05.720023+0000","last_became_active":"2024-04-25T11:15:05.713946+0000","last_became_peered":"2024-04-25T11:15:05.713946+0000","last_unstale":"2024-04-25T11:15:05.720023+0000","last_undegraded":"2024-04-25T11:15:05.720023+0000","last_fullsized":"2024-04-25T11:15:05.720023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:53.951531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.730187+0000","last_change":"2024-04-25T11:15:05.731825+0000","last_active":"2024-04-25T11:15:06.730187+0000","last_peered":"2024-04-25T11:15:06.730187+0000","last_clean":"2024-04-25T11:15:06.730187+0000","last_became_active":"2024-04-25T11:15:05.731367+0000","last_became_peered":"2024-04-25T11:15:05.731367+0000","last_unstale":"2024-04-25T11:15:06.730187+0000","last_undegraded":"2024-04-25T11:15:06.730187+0000","last_fullsized":"2024-04-25T11:15:06.730187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:14.230825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.718013+0000","last_change":"2024-04-25T11:15:05.711433+0000","last_active":"2024-04-25T11:15:05.718013+0000","last_peered":"2024-04-25T11:15:05.718013+0000","last_clean":"2024-04-25T11:15:05.718013+0000","last_became_active":"2024-04-25T11:15:05.711098+0000","last_became_peered":"2024-04-25T11:15:05.711098+0000","last_unstale":"2024-04-25T11:15:05.718013+0000","last_undegraded":"2024-04-25T11:15:05.718013+0000","last_fullsized":"2024-04-25T11:15:05.718013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:58.615103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722312+0000","last_change":"2024-04-25T11:15:05.740871+0000","last_active":"2024-04-25T11:15:06.722312+0000","last_peered":"2024-04-25T11:15:06.722312+0000","last_clean":"2024-04-25T11:15:06.722312+0000","last_became_active":"2024-04-25T11:15:05.740514+0000","last_became_peered":"2024-04-25T11:15:05.740514+0000","last_unstale":"2024-04-25T11:15:06.722312+0000","last_undegraded":"2024-04-25T11:15:06.722312+0000","last_fullsized":"2024-04-25T11:15:06.722312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:48.289168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.724051+0000","last_change":"2024-04-25T11:15:05.722737+0000","last_active":"2024-04-25T11:15:06.724051+0000","last_peered":"2024-04-25T11:15:06.724051+0000","last_clean":"2024-04-25T11:15:06.724051+0000","last_became_active":"2024-04-25T11:15:05.722295+0000","last_became_peered":"2024-04-25T11:15:05.722295+0000","last_unstale":"2024-04-25T11:15:06.724051+0000","last_undegraded":"2024-04-25T11:15:06.724051+0000","last_fullsized":"2024-04-25T11:15:06.724051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:59.454416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.747316+0000","last_change":"2024-04-25T11:15:05.745824+0000","last_active":"2024-04-25T11:15:05.747316+0000","last_peered":"2024-04-25T11:15:05.747316+0000","last_clean":"2024-04-25T11:15:05.747316+0000","last_became_active":"2024-04-25T11:15:05.745531+0000","last_became_peered":"2024-04-25T11:15:05.745531+0000","last_unstale":"2024-04-25T11:15:05.747316+0000","last_undegraded":"2024-04-25T11:15:05.747316+0000","last_fullsized":"2024-04-25T11:15:05.747316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:28.821187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.710600+0000","last_change":"2024-04-25T11:15:05.706753+0000","last_active":"2024-04-25T11:15:05.710600+0000","last_peered":"2024-04-25T11:15:05.710600+0000","last_clean":"2024-04-25T11:15:05.710600+0000","last_became_active":"2024-04-25T11:15:05.706381+0000","last_became_peered":"2024-04-25T11:15:05.706381+0000","last_unstale":"2024-04-25T11:15:05.710600+0000","last_undegraded":"2024-04-25T11:15:05.710600+0000","last_fullsized":"2024-04-25T11:15:05.710600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:11.918722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731318+0000","last_change":"2024-04-25T11:15:05.709286+0000","last_active":"2024-04-25T11:15:06.731318+0000","last_peered":"2024-04-25T11:15:06.731318+0000","last_clean":"2024-04-25T11:15:06.731318+0000","last_became_active":"2024-04-25T11:15:05.707463+0000","last_became_peered":"2024-04-25T11:15:05.707463+0000","last_unstale":"2024-04-25T11:15:06.731318+0000","last_undegraded":"2024-04-25T11:15:06.731318+0000","last_fullsized":"2024-04-25T11:15:06.731318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:02.203928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.723812+0000","last_change":"2024-04-25T11:15:05.739425+0000","last_active":"2024-04-25T11:15:06.723812+0000","last_peered":"2024-04-25T11:15:06.723812+0000","last_clean":"2024-04-25T11:15:06.723812+0000","last_became_active":"2024-04-25T11:15:05.738816+0000","last_became_peered":"2024-04-25T11:15:05.738816+0000","last_unstale":"2024-04-25T11:15:06.723812+0000","last_undegraded":"2024-04-25T11:15:06.723812+0000","last_fullsized":"2024-04-25T11:15:06.723812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:28.047969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.735029+0000","last_change":"2024-04-25T11:15:05.732603+0000","last_active":"2024-04-25T11:15:05.735029+0000","last_peered":"2024-04-25T11:15:05.735029+0000","last_clean":"2024-04-25T11:15:05.735029+0000","last_became_active":"2024-04-25T11:15:05.732276+0000","last_became_peered":"2024-04-25T11:15:05.732276+0000","last_unstale":"2024-04-25T11:15:05.735029+0000","last_undegraded":"2024-04-25T11:15:05.735029+0000","last_fullsized":"2024-04-25T11:15:05.735029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:34.552177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837370+0000","last_change":"2024-04-25T11:15:05.710200+0000","last_active":"2024-04-25T11:15:06.837370+0000","last_peered":"2024-04-25T11:15:06.837370+0000","last_clean":"2024-04-25T11:15:06.837370+0000","last_became_active":"2024-04-25T11:15:05.709863+0000","last_became_peered":"2024-04-25T11:15:05.709863+0000","last_unstale":"2024-04-25T11:15:06.837370+0000","last_undegraded":"2024-04-25T11:15:06.837370+0000","last_fullsized":"2024-04-25T11:15:06.837370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:29.681447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.711032+0000","last_change":"2024-04-25T11:15:05.709667+0000","last_active":"2024-04-25T11:15:05.711032+0000","last_peered":"2024-04-25T11:15:05.711032+0000","last_clean":"2024-04-25T11:15:05.711032+0000","last_became_active":"2024-04-25T11:15:05.709358+0000","last_became_peered":"2024-04-25T11:15:05.709358+0000","last_unstale":"2024-04-25T11:15:05.711032+0000","last_undegraded":"2024-04-25T11:15:05.711032+0000","last_fullsized":"2024-04-25T11:15:05.711032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:14.090924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837192+0000","last_change":"2024-04-25T11:15:05.711205+0000","last_active":"2024-04-25T11:15:06.837192+0000","last_peered":"2024-04-25T11:15:06.837192+0000","last_clean":"2024-04-25T11:15:06.837192+0000","last_became_active":"2024-04-25T11:15:05.710770+0000","last_became_peered":"2024-04-25T11:15:05.710770+0000","last_unstale":"2024-04-25T11:15:06.837192+0000","last_undegraded":"2024-04-25T11:15:06.837192+0000","last_fullsized":"2024-04-25T11:15:06.837192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:38.194245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"79'4","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731516+0000","last_change":"2024-04-25T11:15:04.723036+0000","last_active":"2024-04-25T11:15:06.731516+0000","last_peered":"2024-04-25T11:15:06.731516+0000","last_clean":"2024-04-25T11:15:06.731516+0000","last_became_active":"2024-04-25T11:15:04.722711+0000","last_became_peered":"2024-04-25T11:15:04.722711+0000","last_unstale":"2024-04-25T11:15:06.731516+0000","last_undegraded":"2024-04-25T11:15:06.731516+0000","last_fullsized":"2024-04-25T11:15:06.731516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:19.031331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722962+0000","last_change":"2024-04-25T11:15:05.722790+0000","last_active":"2024-04-25T11:15:06.722962+0000","last_peered":"2024-04-25T11:15:06.722962+0000","last_clean":"2024-04-25T11:15:06.722962+0000","last_became_active":"2024-04-25T11:15:05.717013+0000","last_became_peered":"2024-04-25T11:15:05.717013+0000","last_unstale":"2024-04-25T11:15:06.722962+0000","last_undegraded":"2024-04-25T11:15:06.722962+0000","last_fullsized":"2024-04-25T11:15:06.722962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:22.581632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.734298+0000","last_change":"2024-04-25T11:15:04.722264+0000","last_active":"2024-04-25T11:15:05.734298+0000","last_peered":"2024-04-25T11:15:05.734298+0000","last_clean":"2024-04-25T11:15:05.734298+0000","last_became_active":"2024-04-25T11:15:04.721918+0000","last_became_peered":"2024-04-25T11:15:04.721918+0000","last_unstale":"2024-04-25T11:15:05.734298+0000","last_undegraded":"2024-04-25T11:15:05.734298+0000","last_fullsized":"2024-04-25T11:15:05.734298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:20.992239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.737790+0000","last_change":"2024-04-25T11:15:05.733503+0000","last_active":"2024-04-25T11:15:05.737790+0000","last_peered":"2024-04-25T11:15:05.737790+0000","last_clean":"2024-04-25T11:15:05.737790+0000","last_became_active":"2024-04-25T11:15:05.732654+0000","last_became_peered":"2024-04-25T11:15:05.732654+0000","last_unstale":"2024-04-25T11:15:05.737790+0000","last_undegraded":"2024-04-25T11:15:05.737790+0000","last_fullsized":"2024-04-25T11:15:05.737790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:54.574321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.745736+0000","last_change":"2024-04-25T11:15:04.722187+0000","last_active":"2024-04-25T11:15:05.745736+0000","last_peered":"2024-04-25T11:15:05.745736+0000","last_clean":"2024-04-25T11:15:05.745736+0000","last_became_active":"2024-04-25T11:15:04.721831+0000","last_became_peered":"2024-04-25T11:15:04.721831+0000","last_unstale":"2024-04-25T11:15:05.745736+0000","last_undegraded":"2024-04-25T11:15:05.745736+0000","last_fullsized":"2024-04-25T11:15:05.745736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:27.803166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.720119+0000","last_change":"2024-04-25T11:15:05.714018+0000","last_active":"2024-04-25T11:15:05.720119+0000","last_peered":"2024-04-25T11:15:05.720119+0000","last_clean":"2024-04-25T11:15:05.720119+0000","last_became_active":"2024-04-25T11:15:05.712906+0000","last_became_peered":"2024-04-25T11:15:05.712906+0000","last_unstale":"2024-04-25T11:15:05.720119+0000","last_undegraded":"2024-04-25T11:15:05.720119+0000","last_fullsized":"2024-04-25T11:15:05.720119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:57.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.735359+0000","last_change":"2024-04-25T11:15:04.730550+0000","last_active":"2024-04-25T11:15:05.735359+0000","last_peered":"2024-04-25T11:15:05.735359+0000","last_clean":"2024-04-25T11:15:05.735359+0000","last_became_active":"2024-04-25T11:15:04.730149+0000","last_became_peered":"2024-04-25T11:15:04.730149+0000","last_unstale":"2024-04-25T11:15:05.735359+0000","last_undegraded":"2024-04-25T11:15:05.735359+0000","last_fullsized":"2024-04-25T11:15:05.735359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:32.787200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.723304+0000","last_change":"2024-04-25T11:15:05.734416+0000","last_active":"2024-04-25T11:15:06.723304+0000","last_peered":"2024-04-25T11:15:06.723304+0000","last_clean":"2024-04-25T11:15:06.723304+0000","last_became_active":"2024-04-25T11:15:05.733860+0000","last_became_peered":"2024-04-25T11:15:05.733860+0000","last_unstale":"2024-04-25T11:15:06.723304+0000","last_undegraded":"2024-04-25T11:15:06.723304+0000","last_fullsized":"2024-04-25T11:15:06.723304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:07.203225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.734667+0000","last_change":"2024-04-25T11:15:04.727194+0000","last_active":"2024-04-25T11:15:05.734667+0000","last_peered":"2024-04-25T11:15:05.734667+0000","last_clean":"2024-04-25T11:15:05.734667+0000","last_became_active":"2024-04-25T11:15:04.726798+0000","last_became_peered":"2024-04-25T11:15:04.726798+0000","last_unstale":"2024-04-25T11:15:05.734667+0000","last_undegraded":"2024-04-25T11:15:05.734667+0000","last_fullsized":"2024-04-25T11:15:05.734667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:04.700414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.723509+0000","last_change":"2024-04-25T11:15:05.732329+0000","last_active":"2024-04-25T11:15:06.723509+0000","last_peered":"2024-04-25T11:15:06.723509+0000","last_clean":"2024-04-25T11:15:06.723509+0000","last_became_active":"2024-04-25T11:15:05.731516+0000","last_became_peered":"2024-04-25T11:15:05.731516+0000","last_unstale":"2024-04-25T11:15:06.723509+0000","last_undegraded":"2024-04-25T11:15:06.723509+0000","last_fullsized":"2024-04-25T11:15:06.723509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:54.469610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.738045+0000","last_change":"2024-04-25T11:15:04.731220+0000","last_active":"2024-04-25T11:15:05.738045+0000","last_peered":"2024-04-25T11:15:05.738045+0000","last_clean":"2024-04-25T11:15:05.738045+0000","last_became_active":"2024-04-25T11:15:04.730506+0000","last_became_peered":"2024-04-25T11:15:04.730506+0000","last_unstale":"2024-04-25T11:15:05.738045+0000","last_undegraded":"2024-04-25T11:15:05.738045+0000","last_fullsized":"2024-04-25T11:15:05.738045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:28.904734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.720096+0000","last_change":"2024-04-25T11:15:05.716373+0000","last_active":"2024-04-25T11:15:05.720096+0000","last_peered":"2024-04-25T11:15:05.720096+0000","last_clean":"2024-04-25T11:15:05.720096+0000","last_became_active":"2024-04-25T11:15:05.715900+0000","last_became_peered":"2024-04-25T11:15:05.715900+0000","last_unstale":"2024-04-25T11:15:05.720096+0000","last_undegraded":"2024-04-25T11:15:05.720096+0000","last_fullsized":"2024-04-25T11:15:05.720096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:08.356278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722704+0000","last_change":"2024-04-25T11:15:04.723351+0000","last_active":"2024-04-25T11:15:06.722704+0000","last_peered":"2024-04-25T11:15:06.722704+0000","last_clean":"2024-04-25T11:15:06.722704+0000","last_became_active":"2024-04-25T11:15:04.722980+0000","last_became_peered":"2024-04-25T11:15:04.722980+0000","last_unstale":"2024-04-25T11:15:06.722704+0000","last_undegraded":"2024-04-25T11:15:06.722704+0000","last_fullsized":"2024-04-25T11:15:06.722704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:21.741485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.721384+0000","last_change":"2024-04-25T11:15:05.718231+0000","last_active":"2024-04-25T11:15:05.721384+0000","last_peered":"2024-04-25T11:15:05.721384+0000","last_clean":"2024-04-25T11:15:05.721384+0000","last_became_active":"2024-04-25T11:15:05.717839+0000","last_became_peered":"2024-04-25T11:15:05.717839+0000","last_unstale":"2024-04-25T11:15:05.721384+0000","last_undegraded":"2024-04-25T11:15:05.721384+0000","last_fullsized":"2024-04-25T11:15:05.721384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:55.988121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.729697+0000","last_change":"2024-04-25T11:15:04.721388+0000","last_active":"2024-04-25T11:15:06.729697+0000","last_peered":"2024-04-25T11:15:06.729697+0000","last_clean":"2024-04-25T11:15:06.729697+0000","last_became_active":"2024-04-25T11:15:04.721030+0000","last_became_peered":"2024-04-25T11:15:04.721030+0000","last_unstale":"2024-04-25T11:15:06.729697+0000","last_undegraded":"2024-04-25T11:15:06.729697+0000","last_fullsized":"2024-04-25T11:15:06.729697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:05.895836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722314+0000","last_change":"2024-04-25T11:15:05.741046+0000","last_active":"2024-04-25T11:15:06.722314+0000","last_peered":"2024-04-25T11:15:06.722314+0000","last_clean":"2024-04-25T11:15:06.722314+0000","last_became_active":"2024-04-25T11:15:05.740545+0000","last_became_peered":"2024-04-25T11:15:05.740545+0000","last_unstale":"2024-04-25T11:15:06.722314+0000","last_undegraded":"2024-04-25T11:15:06.722314+0000","last_fullsized":"2024-04-25T11:15:06.722314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:01.008689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.690353+0000","last_change":"2024-04-25T11:15:04.728429+0000","last_active":"2024-04-25T11:15:05.690353+0000","last_peered":"2024-04-25T11:15:05.690353+0000","last_clean":"2024-04-25T11:15:05.690353+0000","last_became_active":"2024-04-25T11:15:04.728040+0000","last_became_peered":"2024-04-25T11:15:04.728040+0000","last_unstale":"2024-04-25T11:15:05.690353+0000","last_undegraded":"2024-04-25T11:15:05.690353+0000","last_fullsized":"2024-04-25T11:15:05.690353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:44.961588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.717238+0000","last_change":"2024-04-25T11:15:05.708743+0000","last_active":"2024-04-25T11:15:05.717238+0000","last_peered":"2024-04-25T11:15:05.717238+0000","last_clean":"2024-04-25T11:15:05.717238+0000","last_became_active":"2024-04-25T11:15:05.707530+0000","last_became_peered":"2024-04-25T11:15:05.707530+0000","last_unstale":"2024-04-25T11:15:05.717238+0000","last_undegraded":"2024-04-25T11:15:05.717238+0000","last_fullsized":"2024-04-25T11:15:05.717238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:27.757667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.684015+0000","last_change":"2024-04-25T11:15:04.709778+0000","last_active":"2024-04-25T11:15:05.684015+0000","last_peered":"2024-04-25T11:15:05.684015+0000","last_clean":"2024-04-25T11:15:05.684015+0000","last_became_active":"2024-04-25T11:15:04.709330+0000","last_became_peered":"2024-04-25T11:15:04.709330+0000","last_unstale":"2024-04-25T11:15:05.684015+0000","last_undegraded":"2024-04-25T11:15:05.684015+0000","last_fullsized":"2024-04-25T11:15:05.684015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:11.194902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.838136+0000","last_change":"2024-04-25T11:15:05.719161+0000","last_active":"2024-04-25T11:15:06.838136+0000","last_peered":"2024-04-25T11:15:06.838136+0000","last_clean":"2024-04-25T11:15:06.838136+0000","last_became_active":"2024-04-25T11:15:05.718849+0000","last_became_peered":"2024-04-25T11:15:05.718849+0000","last_unstale":"2024-04-25T11:15:06.838136+0000","last_undegraded":"2024-04-25T11:15:06.838136+0000","last_fullsized":"2024-04-25T11:15:06.838136+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:41.946329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"79'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731400+0000","last_change":"2024-04-25T11:15:04.717802+0000","last_active":"2024-04-25T11:15:06.731400+0000","last_peered":"2024-04-25T11:15:06.731400+0000","last_clean":"2024-04-25T11:15:06.731400+0000","last_became_active":"2024-04-25T11:15:04.717425+0000","last_became_peered":"2024-04-25T11:15:04.717425+0000","last_unstale":"2024-04-25T11:15:06.731400+0000","last_undegraded":"2024-04-25T11:15:06.731400+0000","last_fullsized":"2024-04-25T11:15:06.731400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:53.503373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837866+0000","last_change":"2024-04-25T11:15:05.743925+0000","last_active":"2024-04-25T11:15:06.837866+0000","last_peered":"2024-04-25T11:15:06.837866+0000","last_clean":"2024-04-25T11:15:06.837866+0000","last_became_active":"2024-04-25T11:15:05.743562+0000","last_became_peered":"2024-04-25T11:15:05.743562+0000","last_unstale":"2024-04-25T11:15:06.837866+0000","last_undegraded":"2024-04-25T11:15:06.837866+0000","last_fullsized":"2024-04-25T11:15:06.837866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:55.988121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.741444+0000","last_change":"2024-04-25T11:15:04.726433+0000","last_active":"2024-04-25T11:15:05.741444+0000","last_peered":"2024-04-25T11:15:05.741444+0000","last_clean":"2024-04-25T11:15:05.741444+0000","last_became_active":"2024-04-25T11:15:04.726095+0000","last_became_peered":"2024-04-25T11:15:04.726095+0000","last_unstale":"2024-04-25T11:15:05.741444+0000","last_undegraded":"2024-04-25T11:15:05.741444+0000","last_fullsized":"2024-04-25T11:15:05.741444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:26.309047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.722439+0000","last_change":"2024-04-25T11:15:05.717105+0000","last_active":"2024-04-25T11:15:05.722439+0000","last_peered":"2024-04-25T11:15:05.722439+0000","last_clean":"2024-04-25T11:15:05.722439+0000","last_became_active":"2024-04-25T11:15:05.716594+0000","last_became_peered":"2024-04-25T11:15:05.716594+0000","last_unstale":"2024-04-25T11:15:05.722439+0000","last_undegraded":"2024-04-25T11:15:05.722439+0000","last_fullsized":"2024-04-25T11:15:05.722439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:28.810242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.736158+0000","last_change":"2024-04-25T11:15:04.729428+0000","last_active":"2024-04-25T11:15:05.736158+0000","last_peered":"2024-04-25T11:15:05.736158+0000","last_clean":"2024-04-25T11:15:05.736158+0000","last_became_active":"2024-04-25T11:15:04.729036+0000","last_became_peered":"2024-04-25T11:15:04.729036+0000","last_unstale":"2024-04-25T11:15:05.736158+0000","last_undegraded":"2024-04-25T11:15:05.736158+0000","last_fullsized":"2024-04-25T11:15:05.736158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:14.848494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.721342+0000","last_change":"2024-04-25T11:15:05.717256+0000","last_active":"2024-04-25T11:15:05.721342+0000","last_peered":"2024-04-25T11:15:05.721342+0000","last_clean":"2024-04-25T11:15:05.721342+0000","last_became_active":"2024-04-25T11:15:05.715993+0000","last_became_peered":"2024-04-25T11:15:05.715993+0000","last_unstale":"2024-04-25T11:15:05.721342+0000","last_undegraded":"2024-04-25T11:15:05.721342+0000","last_fullsized":"2024-04-25T11:15:05.721342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:29.681447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.685737+0000","last_change":"2024-04-25T11:15:04.711249+0000","last_active":"2024-04-25T11:15:05.685737+0000","last_peered":"2024-04-25T11:15:05.685737+0000","last_clean":"2024-04-25T11:15:05.685737+0000","last_became_active":"2024-04-25T11:15:04.710618+0000","last_became_peered":"2024-04-25T11:15:04.710618+0000","last_unstale":"2024-04-25T11:15:05.685737+0000","last_undegraded":"2024-04-25T11:15:05.685737+0000","last_fullsized":"2024-04-25T11:15:05.685737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:27.792221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.718976+0000","last_change":"2024-04-25T11:15:05.709420+0000","last_active":"2024-04-25T11:15:05.718976+0000","last_peered":"2024-04-25T11:15:05.718976+0000","last_clean":"2024-04-25T11:15:05.718976+0000","last_became_active":"2024-04-25T11:15:05.708866+0000","last_became_peered":"2024-04-25T11:15:05.708866+0000","last_unstale":"2024-04-25T11:15:05.718976+0000","last_undegraded":"2024-04-25T11:15:05.718976+0000","last_fullsized":"2024-04-25T11:15:05.718976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:09.414521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.729735+0000","last_change":"2024-04-25T11:15:04.721687+0000","last_active":"2024-04-25T11:15:06.729735+0000","last_peered":"2024-04-25T11:15:06.729735+0000","last_clean":"2024-04-25T11:15:06.729735+0000","last_became_active":"2024-04-25T11:15:04.720980+0000","last_became_peered":"2024-04-25T11:15:04.720980+0000","last_unstale":"2024-04-25T11:15:06.729735+0000","last_undegraded":"2024-04-25T11:15:06.729735+0000","last_fullsized":"2024-04-25T11:15:06.729735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:55.892571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.724605+0000","last_change":"2024-04-25T11:15:05.723798+0000","last_active":"2024-04-25T11:15:05.724605+0000","last_peered":"2024-04-25T11:15:05.724605+0000","last_clean":"2024-04-25T11:15:05.724605+0000","last_became_active":"2024-04-25T11:15:05.723405+0000","last_became_peered":"2024-04-25T11:15:05.723405+0000","last_unstale":"2024-04-25T11:15:05.724605+0000","last_undegraded":"2024-04-25T11:15:05.724605+0000","last_fullsized":"2024-04-25T11:15:05.724605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:29.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"79'8","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722628+0000","last_change":"2024-04-25T11:15:04.724962+0000","last_active":"2024-04-25T11:15:06.722628+0000","last_peered":"2024-04-25T11:15:06.722628+0000","last_clean":"2024-04-25T11:15:06.722628+0000","last_became_active":"2024-04-25T11:15:04.724620+0000","last_became_peered":"2024-04-25T11:15:04.724620+0000","last_unstale":"2024-04-25T11:15:06.722628+0000","last_undegraded":"2024-04-25T11:15:06.722628+0000","last_fullsized":"2024-04-25T11:15:06.722628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:44.961588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.836476+0000","last_change":"2024-04-25T11:15:05.723591+0000","last_active":"2024-04-25T11:15:06.836476+0000","last_peered":"2024-04-25T11:15:06.836476+0000","last_clean":"2024-04-25T11:15:06.836476+0000","last_became_active":"2024-04-25T11:15:05.723220+0000","last_became_peered":"2024-04-25T11:15:05.723220+0000","last_unstale":"2024-04-25T11:15:06.836476+0000","last_undegraded":"2024-04-25T11:15:06.836476+0000","last_fullsized":"2024-04-25T11:15:06.836476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:04.419956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.732450+0000","last_change":"2024-04-25T11:15:04.727452+0000","last_active":"2024-04-25T11:15:05.732450+0000","last_peered":"2024-04-25T11:15:05.732450+0000","last_clean":"2024-04-25T11:15:05.732450+0000","last_became_active":"2024-04-25T11:15:04.727130+0000","last_became_peered":"2024-04-25T11:15:04.727130+0000","last_unstale":"2024-04-25T11:15:05.732450+0000","last_undegraded":"2024-04-25T11:15:05.732450+0000","last_fullsized":"2024-04-25T11:15:05.732450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:25.431924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.729940+0000","last_change":"2024-04-25T11:15:05.711131+0000","last_active":"2024-04-25T11:15:06.729940+0000","last_peered":"2024-04-25T11:15:06.729940+0000","last_clean":"2024-04-25T11:15:06.729940+0000","last_became_active":"2024-04-25T11:15:05.710828+0000","last_became_peered":"2024-04-25T11:15:05.710828+0000","last_unstale":"2024-04-25T11:15:06.729940+0000","last_undegraded":"2024-04-25T11:15:06.729940+0000","last_fullsized":"2024-04-25T11:15:06.729940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:16.609062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.733942+0000","last_change":"2024-04-25T11:15:04.729489+0000","last_active":"2024-04-25T11:15:05.733942+0000","last_peered":"2024-04-25T11:15:05.733942+0000","last_clean":"2024-04-25T11:15:05.733942+0000","last_became_active":"2024-04-25T11:15:04.729131+0000","last_became_peered":"2024-04-25T11:15:04.729131+0000","last_unstale":"2024-04-25T11:15:05.733942+0000","last_undegraded":"2024-04-25T11:15:05.733942+0000","last_fullsized":"2024-04-25T11:15:05.733942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:22.097558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.729641+0000","last_change":"2024-04-25T11:15:05.732590+0000","last_active":"2024-04-25T11:15:06.729641+0000","last_peered":"2024-04-25T11:15:06.729641+0000","last_clean":"2024-04-25T11:15:06.729641+0000","last_became_active":"2024-04-25T11:15:05.730988+0000","last_became_peered":"2024-04-25T11:15:05.730988+0000","last_unstale":"2024-04-25T11:15:06.729641+0000","last_undegraded":"2024-04-25T11:15:06.729641+0000","last_fullsized":"2024-04-25T11:15:06.729641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:38.822542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731468+0000","last_change":"2024-04-25T11:15:04.720302+0000","last_active":"2024-04-25T11:15:06.731468+0000","last_peered":"2024-04-25T11:15:06.731468+0000","last_clean":"2024-04-25T11:15:06.731468+0000","last_became_active":"2024-04-25T11:15:04.719921+0000","last_became_peered":"2024-04-25T11:15:04.719921+0000","last_unstale":"2024-04-25T11:15:06.731468+0000","last_undegraded":"2024-04-25T11:15:06.731468+0000","last_fullsized":"2024-04-25T11:15:06.731468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:37.693851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.836934+0000","last_change":"2024-04-25T11:15:05.726639+0000","last_active":"2024-04-25T11:15:06.836934+0000","last_peered":"2024-04-25T11:15:06.836934+0000","last_clean":"2024-04-25T11:15:06.836934+0000","last_became_active":"2024-04-25T11:15:05.723378+0000","last_became_peered":"2024-04-25T11:15:05.723378+0000","last_unstale":"2024-04-25T11:15:06.836934+0000","last_undegraded":"2024-04-25T11:15:06.836934+0000","last_fullsized":"2024-04-25T11:15:06.836934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:16.930741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837593+0000","last_change":"2024-04-25T11:15:04.710858+0000","last_active":"2024-04-25T11:15:06.837593+0000","last_peered":"2024-04-25T11:15:06.837593+0000","last_clean":"2024-04-25T11:15:06.837593+0000","last_became_active":"2024-04-25T11:15:04.710448+0000","last_became_peered":"2024-04-25T11:15:04.710448+0000","last_unstale":"2024-04-25T11:15:06.837593+0000","last_undegraded":"2024-04-25T11:15:06.837593+0000","last_fullsized":"2024-04-25T11:15:06.837593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:29.997448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.723542+0000","last_change":"2024-04-25T11:15:05.730079+0000","last_active":"2024-04-25T11:15:06.723542+0000","last_peered":"2024-04-25T11:15:06.723542+0000","last_clean":"2024-04-25T11:15:06.723542+0000","last_became_active":"2024-04-25T11:15:05.723732+0000","last_became_peered":"2024-04-25T11:15:05.723732+0000","last_unstale":"2024-04-25T11:15:06.723542+0000","last_undegraded":"2024-04-25T11:15:06.723542+0000","last_fullsized":"2024-04-25T11:15:06.723542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.750329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837392+0000","last_change":"2024-04-25T11:15:04.728894+0000","last_active":"2024-04-25T11:15:06.837392+0000","last_peered":"2024-04-25T11:15:06.837392+0000","last_clean":"2024-04-25T11:15:06.837392+0000","last_became_active":"2024-04-25T11:15:04.728102+0000","last_became_peered":"2024-04-25T11:15:04.728102+0000","last_unstale":"2024-04-25T11:15:06.837392+0000","last_undegraded":"2024-04-25T11:15:06.837392+0000","last_fullsized":"2024-04-25T11:15:06.837392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:58.653273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.724185+0000","last_change":"2024-04-25T11:15:05.739212+0000","last_active":"2024-04-25T11:15:06.724185+0000","last_peered":"2024-04-25T11:15:06.724185+0000","last_clean":"2024-04-25T11:15:06.724185+0000","last_became_active":"2024-04-25T11:15:05.738822+0000","last_became_peered":"2024-04-25T11:15:05.738822+0000","last_unstale":"2024-04-25T11:15:06.724185+0000","last_undegraded":"2024-04-25T11:15:06.724185+0000","last_fullsized":"2024-04-25T11:15:06.724185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:46.726035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.836549+0000","last_change":"2024-04-25T11:15:04.727438+0000","last_active":"2024-04-25T11:15:06.836549+0000","last_peered":"2024-04-25T11:15:06.836549+0000","last_clean":"2024-04-25T11:15:06.836549+0000","last_became_active":"2024-04-25T11:15:04.727072+0000","last_became_peered":"2024-04-25T11:15:04.727072+0000","last_unstale":"2024-04-25T11:15:06.836549+0000","last_undegraded":"2024-04-25T11:15:06.836549+0000","last_fullsized":"2024-04-25T11:15:06.836549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:33.534156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719032+0000","last_change":"2024-04-25T11:15:05.712026+0000","last_active":"2024-04-25T11:15:05.719032+0000","last_peered":"2024-04-25T11:15:05.719032+0000","last_clean":"2024-04-25T11:15:05.719032+0000","last_became_active":"2024-04-25T11:15:05.711359+0000","last_became_peered":"2024-04-25T11:15:05.711359+0000","last_unstale":"2024-04-25T11:15:05.719032+0000","last_undegraded":"2024-04-25T11:15:05.719032+0000","last_fullsized":"2024-04-25T11:15:05.719032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:24.383175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837098+0000","last_change":"2024-04-25T11:15:04.725067+0000","last_active":"2024-04-25T11:15:06.837098+0000","last_peered":"2024-04-25T11:15:06.837098+0000","last_clean":"2024-04-25T11:15:06.837098+0000","last_became_active":"2024-04-25T11:15:04.724688+0000","last_became_peered":"2024-04-25T11:15:04.724688+0000","last_unstale":"2024-04-25T11:15:06.837098+0000","last_undegraded":"2024-04-25T11:15:06.837098+0000","last_fullsized":"2024-04-25T11:15:06.837098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:10.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.711634+0000","last_change":"2024-04-25T11:15:05.706526+0000","last_active":"2024-04-25T11:15:05.711634+0000","last_peered":"2024-04-25T11:15:05.711634+0000","last_clean":"2024-04-25T11:15:05.711634+0000","last_became_active":"2024-04-25T11:15:05.706195+0000","last_became_peered":"2024-04-25T11:15:05.706195+0000","last_unstale":"2024-04-25T11:15:05.711634+0000","last_undegraded":"2024-04-25T11:15:05.711634+0000","last_fullsized":"2024-04-25T11:15:05.711634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:44.117099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.735586+0000","last_change":"2024-04-25T11:15:04.711192+0000","last_active":"2024-04-25T11:15:05.735586+0000","last_peered":"2024-04-25T11:15:05.735586+0000","last_clean":"2024-04-25T11:15:05.735586+0000","last_became_active":"2024-04-25T11:15:04.710723+0000","last_became_peered":"2024-04-25T11:15:04.710723+0000","last_unstale":"2024-04-25T11:15:05.735586+0000","last_undegraded":"2024-04-25T11:15:05.735586+0000","last_fullsized":"2024-04-25T11:15:05.735586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:54.764754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.718949+0000","last_change":"2024-04-25T11:15:05.710287+0000","last_active":"2024-04-25T11:15:05.718949+0000","last_peered":"2024-04-25T11:15:05.718949+0000","last_clean":"2024-04-25T11:15:05.718949+0000","last_became_active":"2024-04-25T11:15:05.709953+0000","last_became_peered":"2024-04-25T11:15:05.709953+0000","last_unstale":"2024-04-25T11:15:05.718949+0000","last_undegraded":"2024-04-25T11:15:05.718949+0000","last_fullsized":"2024-04-25T11:15:05.718949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:24.738409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"80'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.724420+0000","last_change":"2024-04-25T11:15:04.716460+0000","last_active":"2024-04-25T11:15:06.724420+0000","last_peered":"2024-04-25T11:15:06.724420+0000","last_clean":"2024-04-25T11:15:06.724420+0000","last_became_active":"2024-04-25T11:15:04.716106+0000","last_became_peered":"2024-04-25T11:15:04.716106+0000","last_unstale":"2024-04-25T11:15:06.724420+0000","last_undegraded":"2024-04-25T11:15:06.724420+0000","last_fullsized":"2024-04-25T11:15:06.724420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:04.035365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.737809+0000","last_change":"2024-04-25T11:15:05.733939+0000","last_active":"2024-04-25T11:15:05.737809+0000","last_peered":"2024-04-25T11:15:05.737809+0000","last_clean":"2024-04-25T11:15:05.737809+0000","last_became_active":"2024-04-25T11:15:05.732073+0000","last_became_peered":"2024-04-25T11:15:05.732073+0000","last_unstale":"2024-04-25T11:15:05.737809+0000","last_undegraded":"2024-04-25T11:15:05.737809+0000","last_fullsized":"2024-04-25T11:15:05.737809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:52.306818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.689130+0000","last_change":"2024-04-25T11:15:04.715659+0000","last_active":"2024-04-25T11:15:05.689130+0000","last_peered":"2024-04-25T11:15:05.689130+0000","last_clean":"2024-04-25T11:15:05.689130+0000","last_became_active":"2024-04-25T11:15:04.715297+0000","last_became_peered":"2024-04-25T11:15:04.715297+0000","last_unstale":"2024-04-25T11:15:05.689130+0000","last_undegraded":"2024-04-25T11:15:05.689130+0000","last_fullsized":"2024-04-25T11:15:05.689130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:21.741485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722911+0000","last_change":"2024-04-25T11:15:05.736431+0000","last_active":"2024-04-25T11:15:06.722911+0000","last_peered":"2024-04-25T11:15:06.722911+0000","last_clean":"2024-04-25T11:15:06.722911+0000","last_became_active":"2024-04-25T11:15:05.732065+0000","last_became_peered":"2024-04-25T11:15:05.732065+0000","last_unstale":"2024-04-25T11:15:06.722911+0000","last_undegraded":"2024-04-25T11:15:06.722911+0000","last_fullsized":"2024-04-25T11:15:06.722911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:03.352675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.687791+0000","last_change":"2024-04-25T11:15:04.723594+0000","last_active":"2024-04-25T11:15:05.687791+0000","last_peered":"2024-04-25T11:15:05.687791+0000","last_clean":"2024-04-25T11:15:05.687791+0000","last_became_active":"2024-04-25T11:15:04.722208+0000","last_became_peered":"2024-04-25T11:15:04.722208+0000","last_unstale":"2024-04-25T11:15:05.687791+0000","last_undegraded":"2024-04-25T11:15:05.687791+0000","last_fullsized":"2024-04-25T11:15:05.687791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:55.675896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.722436+0000","last_change":"2024-04-25T11:15:05.718712+0000","last_active":"2024-04-25T11:15:05.722436+0000","last_peered":"2024-04-25T11:15:05.722436+0000","last_clean":"2024-04-25T11:15:05.722436+0000","last_became_active":"2024-04-25T11:15:05.718393+0000","last_became_peered":"2024-04-25T11:15:05.718393+0000","last_unstale":"2024-04-25T11:15:05.722436+0000","last_undegraded":"2024-04-25T11:15:05.722436+0000","last_fullsized":"2024-04-25T11:15:05.722436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:07.009824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.693833+0000","last_change":"2024-04-25T11:15:04.725398+0000","last_active":"2024-04-25T11:15:05.693833+0000","last_peered":"2024-04-25T11:15:05.693833+0000","last_clean":"2024-04-25T11:15:05.693833+0000","last_became_active":"2024-04-25T11:15:04.724971+0000","last_became_peered":"2024-04-25T11:15:04.724971+0000","last_unstale":"2024-04-25T11:15:05.693833+0000","last_undegraded":"2024-04-25T11:15:05.693833+0000","last_fullsized":"2024-04-25T11:15:05.693833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:55.464864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719121+0000","last_change":"2024-04-25T11:15:05.713626+0000","last_active":"2024-04-25T11:15:05.719121+0000","last_peered":"2024-04-25T11:15:05.719121+0000","last_clean":"2024-04-25T11:15:05.719121+0000","last_became_active":"2024-04-25T11:15:05.713202+0000","last_became_peered":"2024-04-25T11:15:05.713202+0000","last_unstale":"2024-04-25T11:15:05.719121+0000","last_undegraded":"2024-04-25T11:15:05.719121+0000","last_fullsized":"2024-04-25T11:15:05.719121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:57.212790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.730271+0000","last_change":"2024-04-25T11:15:04.728786+0000","last_active":"2024-04-25T11:15:06.730271+0000","last_peered":"2024-04-25T11:15:06.730271+0000","last_clean":"2024-04-25T11:15:06.730271+0000","last_became_active":"2024-04-25T11:15:04.727792+0000","last_became_peered":"2024-04-25T11:15:04.727792+0000","last_unstale":"2024-04-25T11:15:06.730271+0000","last_undegraded":"2024-04-25T11:15:06.730271+0000","last_fullsized":"2024-04-25T11:15:06.730271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:28.157302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.839194+0000","last_change":"2024-04-25T11:15:05.744353+0000","last_active":"2024-04-25T11:15:06.839194+0000","last_peered":"2024-04-25T11:15:06.839194+0000","last_clean":"2024-04-25T11:15:06.839194+0000","last_became_active":"2024-04-25T11:15:05.743591+0000","last_became_peered":"2024-04-25T11:15:05.743591+0000","last_unstale":"2024-04-25T11:15:06.839194+0000","last_undegraded":"2024-04-25T11:15:06.839194+0000","last_fullsized":"2024-04-25T11:15:06.839194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:29.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.693695+0000","last_change":"2024-04-25T11:15:04.730935+0000","last_active":"2024-04-25T11:15:05.693695+0000","last_peered":"2024-04-25T11:15:05.693695+0000","last_clean":"2024-04-25T11:15:05.693695+0000","last_became_active":"2024-04-25T11:15:04.728369+0000","last_became_peered":"2024-04-25T11:15:04.728369+0000","last_unstale":"2024-04-25T11:15:05.693695+0000","last_undegraded":"2024-04-25T11:15:05.693695+0000","last_fullsized":"2024-04-25T11:15:05.693695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:24.109091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.839450+0000","last_change":"2024-04-25T11:15:05.709270+0000","last_active":"2024-04-25T11:15:06.839450+0000","last_peered":"2024-04-25T11:15:06.839450+0000","last_clean":"2024-04-25T11:15:06.839450+0000","last_became_active":"2024-04-25T11:15:05.708846+0000","last_became_peered":"2024-04-25T11:15:05.708846+0000","last_unstale":"2024-04-25T11:15:06.839450+0000","last_undegraded":"2024-04-25T11:15:06.839450+0000","last_fullsized":"2024-04-25T11:15:06.839450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:55.782775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837374+0000","last_change":"2024-04-25T11:15:04.723617+0000","last_active":"2024-04-25T11:15:06.837374+0000","last_peered":"2024-04-25T11:15:06.837374+0000","last_clean":"2024-04-25T11:15:06.837374+0000","last_became_active":"2024-04-25T11:15:04.722528+0000","last_became_peered":"2024-04-25T11:15:04.722528+0000","last_unstale":"2024-04-25T11:15:06.837374+0000","last_undegraded":"2024-04-25T11:15:06.837374+0000","last_fullsized":"2024-04-25T11:15:06.837374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:24.109091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719849+0000","last_change":"2024-04-25T11:15:05.716535+0000","last_active":"2024-04-25T11:15:05.719849+0000","last_peered":"2024-04-25T11:15:05.719849+0000","last_clean":"2024-04-25T11:15:05.719849+0000","last_became_active":"2024-04-25T11:15:05.714334+0000","last_became_peered":"2024-04-25T11:15:05.714334+0000","last_unstale":"2024-04-25T11:15:05.719849+0000","last_undegraded":"2024-04-25T11:15:05.719849+0000","last_fullsized":"2024-04-25T11:15:05.719849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:38.268352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.838711+0000","last_change":"2024-04-25T11:15:04.725224+0000","last_active":"2024-04-25T11:15:06.838711+0000","last_peered":"2024-04-25T11:15:06.838711+0000","last_clean":"2024-04-25T11:15:06.838711+0000","last_became_active":"2024-04-25T11:15:04.724784+0000","last_became_peered":"2024-04-25T11:15:04.724784+0000","last_unstale":"2024-04-25T11:15:06.838711+0000","last_undegraded":"2024-04-25T11:15:06.838711+0000","last_fullsized":"2024-04-25T11:15:06.838711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:06.389713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":225,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.685514+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:15:05.685514+0000","last_peered":"2024-04-25T11:15:05.685514+0000","last_clean":"2024-04-25T11:15:05.685514+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:15:05.685514+0000","last_undegraded":"2024-04-25T11:15:05.685514+0000","last_fullsized":"2024-04-25T11:15:05.685514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476797,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":1023327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476798,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":954368,"data_stored":432937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476797,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":425770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476798,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":974848,"data_stored":434653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":440211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509503,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":974848,"data_stored":439741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":1029997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542207,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":440371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:15:10.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:15:10.638 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-04-25T11:15:10.638+0000 7f3c23a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:15:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.784+0000 7fa7d0e00640 1 Processor -- start 2024-04-25T11:15:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.784+0000 7fa7d0e00640 1 -- start start 2024-04-25T11:15:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.787+0000 7fa7d0e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa7cc0b5f10 0x7fa7cc16f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.787+0000 7fa7d0e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7cc0b47d0 con 0x7fa7cc06ffd0 2024-04-25T11:15:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.787+0000 7fa7d0e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7cc0b4a40 con 0x7fa7cc164380 2024-04-25T11:15:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.787+0000 7fa7d0e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa7cc0b6320 con 0x7fa7cc0b5f10 2024-04-25T11:15:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.787+0000 7fa7cb400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa7cc06ffd0 0x7fa7cc0af390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39190/0 (socket says 172.21.15.7:39190) 2024-04-25T11:15:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.787+0000 7fa7cb400640 1 -- 172.21.15.7:0/3258312504 learned_addr learned my addr 172.21.15.7:0/3258312504 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.787+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/3258312504 >> v2:172.21.15.87:3300/0 conn(0x7fa7cc0b5f10 0x7fa7cc16f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2271177040 0 0) 0x7fa7cc0b47d0 con 0x7fa7cc06ffd0 2024-04-25T11:15:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7b0003520 con 0x7fa7cc06ffd0 2024-04-25T11:15:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 160785907 0 0) 0x7fa7b0003520 con 0x7fa7cc06ffd0 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 >> v2:172.21.15.87:3300/0 conn(0x7fa7cc0b5f10 msgr2=0x7fa7cc16f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 --2- 172.21.15.7:0/3258312504 >> v2:172.21.15.87:3300/0 conn(0x7fa7cc0b5f10 0x7fa7cc16f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 >> v1:172.21.15.87:6789/0 conn(0x7fa7cc164380 legacy=0x7fa7cc0b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc0b4d20 con 0x7fa7cc06ffd0 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fa7bc0019f0 con 0x7fa7cc06ffd0 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 >> v1:172.21.15.7:6789/0 conn(0x7fa7cc06ffd0 legacy=0x7fa7cc0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 --2- 172.21.15.7:0/3258312504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7b0003a00 0x7fa7b0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 --2- 172.21.15.7:0/3258312504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 0x7fa7b0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 --2- 172.21.15.7:0/3258312504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b00052a0 0x7fa7b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.788+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b000c200 con 0x7fa7b0004800 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.789+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b000c3b0 con 0x7fa7b0003a00 2024-04-25T11:15:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.789+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b000c5b0 con 0x7fa7b00052a0 2024-04-25T11:15:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.789+0000 7fa7caa00640 1 --2- 172.21.15.7:0/3258312504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 0x7fa7b0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.789+0000 7fa7cb400640 1 --2- 172.21.15.7:0/3258312504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7b0003a00 0x7fa7b0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.789+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/3258312504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b00052a0 0x7fa7b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.789+0000 7fa7caa00640 1 -- 172.21.15.7:0/3258312504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b00052a0 msgr2=0x7fa7b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.789+0000 7fa7caa00640 1 --2- 172.21.15.7:0/3258312504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7b00052a0 0x7fa7b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.789+0000 7fa7caa00640 1 -- 172.21.15.7:0/3258312504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7b0003a00 msgr2=0x7fa7b0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.789+0000 7fa7caa00640 1 --2- 172.21.15.7:0/3258312504 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7b0003a00 0x7fa7b0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.789+0000 7fa7caa00640 1 -- 172.21.15.7:0/3258312504 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b0003520 con 0x7fa7b0004800 2024-04-25T11:15:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.789+0000 7fa7cbe00640 1 -- 172.21.15.7:0/3258312504 reap_dead start 2024-04-25T11:15:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.790+0000 7fa7caa00640 1 --2- 172.21.15.7:0/3258312504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 0x7fa7b0004c10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4001750 tx=0x7fa7b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d9187450456dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.791+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4001370 con 0x7fa7b0004800 2024-04-25T11:15:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.791+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b40052d0 con 0x7fa7b0004800 2024-04-25T11:15:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.791+0000 7fa7ca000640 1 -- 172.21.15.7:0/3258312504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b402abe0 con 0x7fa7b0004800 2024-04-25T11:15:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.791+0000 7fa7d0e00640 1 -- 172.21.15.7:0/3258312504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 msgr2=0x7fa7b0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.791+0000 7fa7d0e00640 1 --2- 172.21.15.7:0/3258312504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 0x7fa7b0004c10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4001750 tx=0x7fa7b0005830 comp rx=0 tx=0).stop 2024-04-25T11:15:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.792+0000 7fa7d0e00640 1 -- 172.21.15.7:0/3258312504 shutdown_connections 2024-04-25T11:15:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.792+0000 7fa7d0e00640 1 --2- 172.21.15.7:0/3258312504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 0x7fa7b0004c10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.792+0000 7fa7d0e00640 1 -- 172.21.15.7:0/3258312504 >> 172.21.15.7:0/3258312504 conn(0x7fa7cc0b3820 msgr2=0x7fa7cc0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.792+0000 7fa7d0e00640 1 -- 172.21.15.7:0/3258312504 shutdown_connections 2024-04-25T11:15:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.792+0000 7fa7d0e00640 1 -- 172.21.15.7:0/3258312504 wait complete. 2024-04-25T11:15:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.794+0000 7fa7d0e00640 1 Processor -- start 2024-04-25T11:15:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.794+0000 7fa7d0e00640 1 -- start start 2024-04-25T11:15:10.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.796+0000 7fa7d0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 0x7fa7cc09e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.796+0000 7fa7d0e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7cc1634f0 0x7fa7cc09c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.796+0000 7fa7d0e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc166440 0x7fa7cc166850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.796+0000 7fa7d0e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc164790 con 0x7fa7b0004800 2024-04-25T11:15:10.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.796+0000 7fa7d0e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc0b52e0 con 0x7fa7cc1634f0 2024-04-25T11:15:10.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.796+0000 7fa7d0e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa7cc06fa70 con 0x7fa7cc166440 2024-04-25T11:15:10.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.796+0000 7fa7cb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 0x7fa7cc09e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.796+0000 7fa7cb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 0x7fa7cc09e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40694/0 (socket says 172.21.15.7:40694) 2024-04-25T11:15:10.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.796+0000 7fa7cb400640 1 -- 172.21.15.7:0/2855641351 learned_addr learned my addr 172.21.15.7:0/2855641351 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:10.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7caa00640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7cc1634f0 0x7fa7cc09c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7cb400640 1 -- 172.21.15.7:0/2855641351 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc166440 msgr2=0x7fa7cc166850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:10.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7cbe00640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc166440 0x7fa7cc166850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7cb400640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc166440 0x7fa7cc166850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7cb400640 1 -- 172.21.15.7:0/2855641351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7cc1634f0 msgr2=0x7fa7cc09c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:10.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7cb400640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7cc1634f0 0x7fa7cc09c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:10.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7cb400640 1 -- 172.21.15.7:0/2855641351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc09f000 con 0x7fa7b0004800 2024-04-25T11:15:10.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7caa00640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7cc1634f0 0x7fa7cc09c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:10.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7cb400640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 0x7fa7cc09e7b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc001370 tx=0x7fa7bc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:10.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7c3400640 1 -- 172.21.15.7:0/2855641351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc00edf0 con 0x7fa7b0004800 2024-04-25T11:15:10.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7c3400640 1 -- 172.21.15.7:0/2855641351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7bc018070 con 0x7fa7b0004800 2024-04-25T11:15:10.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7c3400640 1 -- 172.21.15.7:0/2855641351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc013660 con 0x7fa7b0004800 2024-04-25T11:15:10.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.797+0000 7fa7d0e00640 1 -- 172.21.15.7:0/2855641351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7cc09f2e0 con 0x7fa7b0004800 2024-04-25T11:15:10.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.798+0000 7fa7d0e00640 1 -- 172.21.15.7:0/2855641351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7cc09b820 con 0x7fa7b0004800 2024-04-25T11:15:10.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.799+0000 7fa7d0e00640 1 -- 172.21.15.7:0/2855641351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7cc050740 con 0x7fa7b0004800 2024-04-25T11:15:10.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.800+0000 7fa7c3400640 1 -- 172.21.15.7:0/2855641351 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa7bc013800 con 0x7fa7b0004800 2024-04-25T11:15:10.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.801+0000 7fa7c3400640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 0x7fa7a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:10.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.802+0000 7fa7c3400640 1 -- 172.21.15.7:0/2855641351 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fa7bc096470 con 0x7fa7b0004800 2024-04-25T11:15:10.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.802+0000 7fa7caa00640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 0x7fa7a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:10.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.803+0000 7fa7caa00640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 0x7fa7a80769c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4032000 tx=0x7fa7b402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:10.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:10.805+0000 7fa7c3400640 1 -- 172.21.15.7:0/2855641351 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa7bc060b50 con 0x7fa7b0004800 2024-04-25T11:15:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.008+0000 7fa7d0e00640 1 -- 172.21.15.7:0/2855641351 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7cc0af3f0 con 0x7fa7a80744b0 2024-04-25T11:15:11.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.020+0000 7fa7c3400640 1 -- 172.21.15.7:0/2855641351 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252030 (secure 0 0 0) 0x7fa7cc0af3f0 con 0x7fa7a80744b0 2024-04-25T11:15:11.025 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:11.026 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.024+0000 7fa7d0e00640 1 -- 172.21.15.7:0/2855641351 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 msgr2=0x7fa7a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.024+0000 7fa7d0e00640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 0x7fa7a80769c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4032000 tx=0x7fa7b402e000 comp rx=0 tx=0).stop 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.024+0000 7fa7d0e00640 1 -- 172.21.15.7:0/2855641351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 msgr2=0x7fa7cc09e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.024+0000 7fa7d0e00640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 0x7fa7cc09e7b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc001370 tx=0x7fa7bc00e530 comp rx=0 tx=0).stop 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.025+0000 7fa7d0e00640 1 -- 172.21.15.7:0/2855641351 shutdown_connections 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.025+0000 7fa7d0e00640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa7a80744b0 0x7fa7a80769c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.025+0000 7fa7d0e00640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7cc166440 0x7fa7cc166850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.025+0000 7fa7d0e00640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7cc1634f0 0x7fa7cc09c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.025+0000 7fa7d0e00640 1 --2- 172.21.15.7:0/2855641351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7b0004800 0x7fa7cc09e7b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.025+0000 7fa7d0e00640 1 -- 172.21.15.7:0/2855641351 >> 172.21.15.7:0/2855641351 conn(0x7fa7cc0b3820 msgr2=0x7fa7cc0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.025+0000 7fa7d0e00640 1 -- 172.21.15.7:0/2855641351 shutdown_connections 2024-04-25T11:15:11.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.025+0000 7fa7d0e00640 1 -- 172.21.15.7:0/2855641351 wait complete. 2024-04-25T11:15:11.053 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-04-25T11:15:09.891238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258168,"kb_used_data":8700,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716552,"statfs":{"total":773094113280,"available":772829749248,"internally_reserved":0,"allocated":8908800,"data_stored":4667007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001506"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.716472+0000","last_change":"2024-04-25T11:15:05.710923+0000","last_active":"2024-04-25T11:15:05.716472+0000","last_peered":"2024-04-25T11:15:05.716472+0000","last_clean":"2024-04-25T11:15:05.716472+0000","last_became_active":"2024-04-25T11:15:05.708870+0000","last_became_peered":"2024-04-25T11:15:05.708870+0000","last_unstale":"2024-04-25T11:15:05.716472+0000","last_undegraded":"2024-04-25T11:15:05.716472+0000","last_fullsized":"2024-04-25T11:15:05.716472+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:07.854228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.724215+0000","last_change":"2024-04-25T11:15:05.743495+0000","last_active":"2024-04-25T11:15:06.724215+0000","last_peered":"2024-04-25T11:15:06.724215+0000","last_clean":"2024-04-25T11:15:06.724215+0000","last_became_active":"2024-04-25T11:15:05.743186+0000","last_became_peered":"2024-04-25T11:15:05.743186+0000","last_unstale":"2024-04-25T11:15:06.724215+0000","last_undegraded":"2024-04-25T11:15:06.724215+0000","last_fullsized":"2024-04-25T11:15:06.724215+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:20.076047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.839086+0000","last_change":"2024-04-25T11:15:05.731046+0000","last_active":"2024-04-25T11:15:06.839086+0000","last_peered":"2024-04-25T11:15:06.839086+0000","last_clean":"2024-04-25T11:15:06.839086+0000","last_became_active":"2024-04-25T11:15:05.730705+0000","last_became_peered":"2024-04-25T11:15:05.730705+0000","last_unstale":"2024-04-25T11:15:06.839086+0000","last_undegraded":"2024-04-25T11:15:06.839086+0000","last_fullsized":"2024-04-25T11:15:06.839086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:07.009824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719802+0000","last_change":"2024-04-25T11:15:05.714141+0000","last_active":"2024-04-25T11:15:05.719802+0000","last_peered":"2024-04-25T11:15:05.719802+0000","last_clean":"2024-04-25T11:15:05.719802+0000","last_became_active":"2024-04-25T11:15:05.713721+0000","last_became_peered":"2024-04-25T11:15:05.713721+0000","last_unstale":"2024-04-25T11:15:05.719802+0000","last_undegraded":"2024-04-25T11:15:05.719802+0000","last_fullsized":"2024-04-25T11:15:05.719802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:49.369927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.730495+0000","last_change":"2024-04-25T11:15:05.708290+0000","last_active":"2024-04-25T11:15:06.730495+0000","last_peered":"2024-04-25T11:15:06.730495+0000","last_clean":"2024-04-25T11:15:06.730495+0000","last_became_active":"2024-04-25T11:15:05.707988+0000","last_became_peered":"2024-04-25T11:15:05.707988+0000","last_unstale":"2024-04-25T11:15:06.730495+0000","last_undegraded":"2024-04-25T11:15:06.730495+0000","last_fullsized":"2024-04-25T11:15:06.730495+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:07.300246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837655+0000","last_change":"2024-04-25T11:15:05.725952+0000","last_active":"2024-04-25T11:15:06.837655+0000","last_peered":"2024-04-25T11:15:06.837655+0000","last_clean":"2024-04-25T11:15:06.837655+0000","last_became_active":"2024-04-25T11:15:05.725015+0000","last_became_peered":"2024-04-25T11:15:05.725015+0000","last_unstale":"2024-04-25T11:15:06.837655+0000","last_undegraded":"2024-04-25T11:15:06.837655+0000","last_fullsized":"2024-04-25T11:15:06.837655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:56.966887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719300+0000","last_change":"2024-04-25T11:15:05.714760+0000","last_active":"2024-04-25T11:15:05.719300+0000","last_peered":"2024-04-25T11:15:05.719300+0000","last_clean":"2024-04-25T11:15:05.719300+0000","last_became_active":"2024-04-25T11:15:05.713302+0000","last_became_peered":"2024-04-25T11:15:05.713302+0000","last_unstale":"2024-04-25T11:15:05.719300+0000","last_undegraded":"2024-04-25T11:15:05.719300+0000","last_fullsized":"2024-04-25T11:15:05.719300+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:38.194245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.729846+0000","last_change":"2024-04-25T11:15:05.710077+0000","last_active":"2024-04-25T11:15:06.729846+0000","last_peered":"2024-04-25T11:15:06.729846+0000","last_clean":"2024-04-25T11:15:06.729846+0000","last_became_active":"2024-04-25T11:15:05.709616+0000","last_became_peered":"2024-04-25T11:15:05.709616+0000","last_unstale":"2024-04-25T11:15:06.729846+0000","last_undegraded":"2024-04-25T11:15:06.729846+0000","last_fullsized":"2024-04-25T11:15:06.729846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:53.888139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.723091+0000","last_change":"2024-04-25T11:15:05.719661+0000","last_active":"2024-04-25T11:15:05.723091+0000","last_peered":"2024-04-25T11:15:05.723091+0000","last_clean":"2024-04-25T11:15:05.723091+0000","last_became_active":"2024-04-25T11:15:05.719241+0000","last_became_peered":"2024-04-25T11:15:05.719241+0000","last_unstale":"2024-04-25T11:15:05.723091+0000","last_undegraded":"2024-04-25T11:15:05.723091+0000","last_fullsized":"2024-04-25T11:15:05.723091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:23.739615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719765+0000","last_change":"2024-04-25T11:15:05.711535+0000","last_active":"2024-04-25T11:15:05.719765+0000","last_peered":"2024-04-25T11:15:05.719765+0000","last_clean":"2024-04-25T11:15:05.719765+0000","last_became_active":"2024-04-25T11:15:05.708619+0000","last_became_peered":"2024-04-25T11:15:05.708619+0000","last_unstale":"2024-04-25T11:15:05.719765+0000","last_undegraded":"2024-04-25T11:15:05.719765+0000","last_fullsized":"2024-04-25T11:15:05.719765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:38.244198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.723311+0000","last_change":"2024-04-25T11:15:05.742150+0000","last_active":"2024-04-25T11:15:06.723311+0000","last_peered":"2024-04-25T11:15:06.723311+0000","last_clean":"2024-04-25T11:15:06.723311+0000","last_became_active":"2024-04-25T11:15:05.741608+0000","last_became_peered":"2024-04-25T11:15:05.741608+0000","last_unstale":"2024-04-25T11:15:06.723311+0000","last_undegraded":"2024-04-25T11:15:06.723311+0000","last_fullsized":"2024-04-25T11:15:06.723311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:59.489824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.735593+0000","last_change":"2024-04-25T11:15:05.732537+0000","last_active":"2024-04-25T11:15:05.735593+0000","last_peered":"2024-04-25T11:15:05.735593+0000","last_clean":"2024-04-25T11:15:05.735593+0000","last_became_active":"2024-04-25T11:15:05.732190+0000","last_became_peered":"2024-04-25T11:15:05.732190+0000","last_unstale":"2024-04-25T11:15:05.735593+0000","last_undegraded":"2024-04-25T11:15:05.735593+0000","last_fullsized":"2024-04-25T11:15:05.735593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:56.693917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.730546+0000","last_change":"2024-04-25T11:15:05.729218+0000","last_active":"2024-04-25T11:15:06.730546+0000","last_peered":"2024-04-25T11:15:06.730546+0000","last_clean":"2024-04-25T11:15:06.730546+0000","last_became_active":"2024-04-25T11:15:05.728877+0000","last_became_peered":"2024-04-25T11:15:05.728877+0000","last_unstale":"2024-04-25T11:15:06.730546+0000","last_undegraded":"2024-04-25T11:15:06.730546+0000","last_fullsized":"2024-04-25T11:15:06.730546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:39.558736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837911+0000","last_change":"2024-04-25T11:15:05.726218+0000","last_active":"2024-04-25T11:15:06.837911+0000","last_peered":"2024-04-25T11:15:06.837911+0000","last_clean":"2024-04-25T11:15:06.837911+0000","last_became_active":"2024-04-25T11:15:05.725450+0000","last_became_peered":"2024-04-25T11:15:05.725450+0000","last_unstale":"2024-04-25T11:15:06.837911+0000","last_undegraded":"2024-04-25T11:15:06.837911+0000","last_fullsized":"2024-04-25T11:15:06.837911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:01.008689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722356+0000","last_change":"2024-04-25T11:15:05.731102+0000","last_active":"2024-04-25T11:15:06.722356+0000","last_peered":"2024-04-25T11:15:06.722356+0000","last_clean":"2024-04-25T11:15:06.722356+0000","last_became_active":"2024-04-25T11:15:05.730391+0000","last_became_peered":"2024-04-25T11:15:05.730391+0000","last_unstale":"2024-04-25T11:15:06.722356+0000","last_undegraded":"2024-04-25T11:15:06.722356+0000","last_fullsized":"2024-04-25T11:15:06.722356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:41.946329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.710232+0000","last_change":"2024-04-25T11:15:05.706747+0000","last_active":"2024-04-25T11:15:05.710232+0000","last_peered":"2024-04-25T11:15:05.710232+0000","last_clean":"2024-04-25T11:15:05.710232+0000","last_became_active":"2024-04-25T11:15:05.706369+0000","last_became_peered":"2024-04-25T11:15:05.706369+0000","last_unstale":"2024-04-25T11:15:05.710232+0000","last_undegraded":"2024-04-25T11:15:05.710232+0000","last_fullsized":"2024-04-25T11:15:05.710232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:05.053385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731467+0000","last_change":"2024-04-25T11:15:05.712366+0000","last_active":"2024-04-25T11:15:06.731467+0000","last_peered":"2024-04-25T11:15:06.731467+0000","last_clean":"2024-04-25T11:15:06.731467+0000","last_became_active":"2024-04-25T11:15:05.711969+0000","last_became_peered":"2024-04-25T11:15:05.711969+0000","last_unstale":"2024-04-25T11:15:06.731467+0000","last_undegraded":"2024-04-25T11:15:06.731467+0000","last_fullsized":"2024-04-25T11:15:06.731467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:49.831747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.839229+0000","last_change":"2024-04-25T11:15:05.710484+0000","last_active":"2024-04-25T11:15:06.839229+0000","last_peered":"2024-04-25T11:15:06.839229+0000","last_clean":"2024-04-25T11:15:06.839229+0000","last_became_active":"2024-04-25T11:15:05.710158+0000","last_became_peered":"2024-04-25T11:15:05.710158+0000","last_unstale":"2024-04-25T11:15:06.839229+0000","last_undegraded":"2024-04-25T11:15:06.839229+0000","last_fullsized":"2024-04-25T11:15:06.839229+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:15.866515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.836888+0000","last_change":"2024-04-25T11:15:05.720447+0000","last_active":"2024-04-25T11:15:06.836888+0000","last_peered":"2024-04-25T11:15:06.836888+0000","last_clean":"2024-04-25T11:15:06.836888+0000","last_became_active":"2024-04-25T11:15:05.720092+0000","last_became_peered":"2024-04-25T11:15:05.720092+0000","last_unstale":"2024-04-25T11:15:06.836888+0000","last_undegraded":"2024-04-25T11:15:06.836888+0000","last_fullsized":"2024-04-25T11:15:06.836888+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:27.480283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731157+0000","last_change":"2024-04-25T11:15:05.713781+0000","last_active":"2024-04-25T11:15:06.731157+0000","last_peered":"2024-04-25T11:15:06.731157+0000","last_clean":"2024-04-25T11:15:06.731157+0000","last_became_active":"2024-04-25T11:15:05.713402+0000","last_became_peered":"2024-04-25T11:15:05.713402+0000","last_unstale":"2024-04-25T11:15:06.731157+0000","last_undegraded":"2024-04-25T11:15:06.731157+0000","last_fullsized":"2024-04-25T11:15:06.731157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:56.709384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.720023+0000","last_change":"2024-04-25T11:15:05.714882+0000","last_active":"2024-04-25T11:15:05.720023+0000","last_peered":"2024-04-25T11:15:05.720023+0000","last_clean":"2024-04-25T11:15:05.720023+0000","last_became_active":"2024-04-25T11:15:05.713946+0000","last_became_peered":"2024-04-25T11:15:05.713946+0000","last_unstale":"2024-04-25T11:15:05.720023+0000","last_undegraded":"2024-04-25T11:15:05.720023+0000","last_fullsized":"2024-04-25T11:15:05.720023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:53.951531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.730187+0000","last_change":"2024-04-25T11:15:05.731825+0000","last_active":"2024-04-25T11:15:06.730187+0000","last_peered":"2024-04-25T11:15:06.730187+0000","last_clean":"2024-04-25T11:15:06.730187+0000","last_became_active":"2024-04-25T11:15:05.731367+0000","last_became_peered":"2024-04-25T11:15:05.731367+0000","last_unstale":"2024-04-25T11:15:06.730187+0000","last_undegraded":"2024-04-25T11:15:06.730187+0000","last_fullsized":"2024-04-25T11:15:06.730187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:14.230825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.718013+0000","last_change":"2024-04-25T11:15:05.711433+0000","last_active":"2024-04-25T11:15:05.718013+0000","last_peered":"2024-04-25T11:15:05.718013+0000","last_clean":"2024-04-25T11:15:05.718013+0000","last_became_active":"2024-04-25T11:15:05.711098+0000","last_became_peered":"2024-04-25T11:15:05.711098+0000","last_unstale":"2024-04-25T11:15:05.718013+0000","last_undegraded":"2024-04-25T11:15:05.718013+0000","last_fullsized":"2024-04-25T11:15:05.718013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:58.615103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722312+0000","last_change":"2024-04-25T11:15:05.740871+0000","last_active":"2024-04-25T11:15:06.722312+0000","last_peered":"2024-04-25T11:15:06.722312+0000","last_clean":"2024-04-25T11:15:06.722312+0000","last_became_active":"2024-04-25T11:15:05.740514+0000","last_became_peered":"2024-04-25T11:15:05.740514+0000","last_unstale":"2024-04-25T11:15:06.722312+0000","last_undegraded":"2024-04-25T11:15:06.722312+0000","last_fullsized":"2024-04-25T11:15:06.722312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:48.289168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.724051+0000","last_change":"2024-04-25T11:15:05.722737+0000","last_active":"2024-04-25T11:15:06.724051+0000","last_peered":"2024-04-25T11:15:06.724051+0000","last_clean":"2024-04-25T11:15:06.724051+0000","last_became_active":"2024-04-25T11:15:05.722295+0000","last_became_peered":"2024-04-25T11:15:05.722295+0000","last_unstale":"2024-04-25T11:15:06.724051+0000","last_undegraded":"2024-04-25T11:15:06.724051+0000","last_fullsized":"2024-04-25T11:15:06.724051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:59.454416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.747316+0000","last_change":"2024-04-25T11:15:05.745824+0000","last_active":"2024-04-25T11:15:05.747316+0000","last_peered":"2024-04-25T11:15:05.747316+0000","last_clean":"2024-04-25T11:15:05.747316+0000","last_became_active":"2024-04-25T11:15:05.745531+0000","last_became_peered":"2024-04-25T11:15:05.745531+0000","last_unstale":"2024-04-25T11:15:05.747316+0000","last_undegraded":"2024-04-25T11:15:05.747316+0000","last_fullsized":"2024-04-25T11:15:05.747316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:28.821187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.710600+0000","last_change":"2024-04-25T11:15:05.706753+0000","last_active":"2024-04-25T11:15:05.710600+0000","last_peered":"2024-04-25T11:15:05.710600+0000","last_clean":"2024-04-25T11:15:05.710600+0000","last_became_active":"2024-04-25T11:15:05.706381+0000","last_became_peered":"2024-04-25T11:15:05.706381+0000","last_unstale":"2024-04-25T11:15:05.710600+0000","last_undegraded":"2024-04-25T11:15:05.710600+0000","last_fullsized":"2024-04-25T11:15:05.710600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:11.918722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731318+0000","last_change":"2024-04-25T11:15:05.709286+0000","last_active":"2024-04-25T11:15:06.731318+0000","last_peered":"2024-04-25T11:15:06.731318+0000","last_clean":"2024-04-25T11:15:06.731318+0000","last_became_active":"2024-04-25T11:15:05.707463+0000","last_became_peered":"2024-04-25T11:15:05.707463+0000","last_unstale":"2024-04-25T11:15:06.731318+0000","last_undegraded":"2024-04-25T11:15:06.731318+0000","last_fullsized":"2024-04-25T11:15:06.731318+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:02.203928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.723812+0000","last_change":"2024-04-25T11:15:05.739425+0000","last_active":"2024-04-25T11:15:06.723812+0000","last_peered":"2024-04-25T11:15:06.723812+0000","last_clean":"2024-04-25T11:15:06.723812+0000","last_became_active":"2024-04-25T11:15:05.738816+0000","last_became_peered":"2024-04-25T11:15:05.738816+0000","last_unstale":"2024-04-25T11:15:06.723812+0000","last_undegraded":"2024-04-25T11:15:06.723812+0000","last_fullsized":"2024-04-25T11:15:06.723812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:28.047969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.735029+0000","last_change":"2024-04-25T11:15:05.732603+0000","last_active":"2024-04-25T11:15:05.735029+0000","last_peered":"2024-04-25T11:15:05.735029+0000","last_clean":"2024-04-25T11:15:05.735029+0000","last_became_active":"2024-04-25T11:15:05.732276+0000","last_became_peered":"2024-04-25T11:15:05.732276+0000","last_unstale":"2024-04-25T11:15:05.735029+0000","last_undegraded":"2024-04-25T11:15:05.735029+0000","last_fullsized":"2024-04-25T11:15:05.735029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:34.552177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837370+0000","last_change":"2024-04-25T11:15:05.710200+0000","last_active":"2024-04-25T11:15:06.837370+0000","last_peered":"2024-04-25T11:15:06.837370+0000","last_clean":"2024-04-25T11:15:06.837370+0000","last_became_active":"2024-04-25T11:15:05.709863+0000","last_became_peered":"2024-04-25T11:15:05.709863+0000","last_unstale":"2024-04-25T11:15:06.837370+0000","last_undegraded":"2024-04-25T11:15:06.837370+0000","last_fullsized":"2024-04-25T11:15:06.837370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:29.681447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.711032+0000","last_change":"2024-04-25T11:15:05.709667+0000","last_active":"2024-04-25T11:15:05.711032+0000","last_peered":"2024-04-25T11:15:05.711032+0000","last_clean":"2024-04-25T11:15:05.711032+0000","last_became_active":"2024-04-25T11:15:05.709358+0000","last_became_peered":"2024-04-25T11:15:05.709358+0000","last_unstale":"2024-04-25T11:15:05.711032+0000","last_undegraded":"2024-04-25T11:15:05.711032+0000","last_fullsized":"2024-04-25T11:15:05.711032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:14.090924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837192+0000","last_change":"2024-04-25T11:15:05.711205+0000","last_active":"2024-04-25T11:15:06.837192+0000","last_peered":"2024-04-25T11:15:06.837192+0000","last_clean":"2024-04-25T11:15:06.837192+0000","last_became_active":"2024-04-25T11:15:05.710770+0000","last_became_peered":"2024-04-25T11:15:05.710770+0000","last_unstale":"2024-04-25T11:15:06.837192+0000","last_undegraded":"2024-04-25T11:15:06.837192+0000","last_fullsized":"2024-04-25T11:15:06.837192+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:38.194245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"79'4","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731516+0000","last_change":"2024-04-25T11:15:04.723036+0000","last_active":"2024-04-25T11:15:06.731516+0000","last_peered":"2024-04-25T11:15:06.731516+0000","last_clean":"2024-04-25T11:15:06.731516+0000","last_became_active":"2024-04-25T11:15:04.722711+0000","last_became_peered":"2024-04-25T11:15:04.722711+0000","last_unstale":"2024-04-25T11:15:06.731516+0000","last_undegraded":"2024-04-25T11:15:06.731516+0000","last_fullsized":"2024-04-25T11:15:06.731516+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:19.031331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722962+0000","last_change":"2024-04-25T11:15:05.722790+0000","last_active":"2024-04-25T11:15:06.722962+0000","last_peered":"2024-04-25T11:15:06.722962+0000","last_clean":"2024-04-25T11:15:06.722962+0000","last_became_active":"2024-04-25T11:15:05.717013+0000","last_became_peered":"2024-04-25T11:15:05.717013+0000","last_unstale":"2024-04-25T11:15:06.722962+0000","last_undegraded":"2024-04-25T11:15:06.722962+0000","last_fullsized":"2024-04-25T11:15:06.722962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:22.581632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.734298+0000","last_change":"2024-04-25T11:15:04.722264+0000","last_active":"2024-04-25T11:15:05.734298+0000","last_peered":"2024-04-25T11:15:05.734298+0000","last_clean":"2024-04-25T11:15:05.734298+0000","last_became_active":"2024-04-25T11:15:04.721918+0000","last_became_peered":"2024-04-25T11:15:04.721918+0000","last_unstale":"2024-04-25T11:15:05.734298+0000","last_undegraded":"2024-04-25T11:15:05.734298+0000","last_fullsized":"2024-04-25T11:15:05.734298+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:20.992239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.737790+0000","last_change":"2024-04-25T11:15:05.733503+0000","last_active":"2024-04-25T11:15:05.737790+0000","last_peered":"2024-04-25T11:15:05.737790+0000","last_clean":"2024-04-25T11:15:05.737790+0000","last_became_active":"2024-04-25T11:15:05.732654+0000","last_became_peered":"2024-04-25T11:15:05.732654+0000","last_unstale":"2024-04-25T11:15:05.737790+0000","last_undegraded":"2024-04-25T11:15:05.737790+0000","last_fullsized":"2024-04-25T11:15:05.737790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:54.574321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.745736+0000","last_change":"2024-04-25T11:15:04.722187+0000","last_active":"2024-04-25T11:15:05.745736+0000","last_peered":"2024-04-25T11:15:05.745736+0000","last_clean":"2024-04-25T11:15:05.745736+0000","last_became_active":"2024-04-25T11:15:04.721831+0000","last_became_peered":"2024-04-25T11:15:04.721831+0000","last_unstale":"2024-04-25T11:15:05.745736+0000","last_undegraded":"2024-04-25T11:15:05.745736+0000","last_fullsized":"2024-04-25T11:15:05.745736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:27.803166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.720119+0000","last_change":"2024-04-25T11:15:05.714018+0000","last_active":"2024-04-25T11:15:05.720119+0000","last_peered":"2024-04-25T11:15:05.720119+0000","last_clean":"2024-04-25T11:15:05.720119+0000","last_became_active":"2024-04-25T11:15:05.712906+0000","last_became_peered":"2024-04-25T11:15:05.712906+0000","last_unstale":"2024-04-25T11:15:05.720119+0000","last_undegraded":"2024-04-25T11:15:05.720119+0000","last_fullsized":"2024-04-25T11:15:05.720119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:57.854913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.735359+0000","last_change":"2024-04-25T11:15:04.730550+0000","last_active":"2024-04-25T11:15:05.735359+0000","last_peered":"2024-04-25T11:15:05.735359+0000","last_clean":"2024-04-25T11:15:05.735359+0000","last_became_active":"2024-04-25T11:15:04.730149+0000","last_became_peered":"2024-04-25T11:15:04.730149+0000","last_unstale":"2024-04-25T11:15:05.735359+0000","last_undegraded":"2024-04-25T11:15:05.735359+0000","last_fullsized":"2024-04-25T11:15:05.735359+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:32.787200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.723304+0000","last_change":"2024-04-25T11:15:05.734416+0000","last_active":"2024-04-25T11:15:06.723304+0000","last_peered":"2024-04-25T11:15:06.723304+0000","last_clean":"2024-04-25T11:15:06.723304+0000","last_became_active":"2024-04-25T11:15:05.733860+0000","last_became_peered":"2024-04-25T11:15:05.733860+0000","last_unstale":"2024-04-25T11:15:06.723304+0000","last_undegraded":"2024-04-25T11:15:06.723304+0000","last_fullsized":"2024-04-25T11:15:06.723304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:07.203225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.734667+0000","last_change":"2024-04-25T11:15:04.727194+0000","last_active":"2024-04-25T11:15:05.734667+0000","last_peered":"2024-04-25T11:15:05.734667+0000","last_clean":"2024-04-25T11:15:05.734667+0000","last_became_active":"2024-04-25T11:15:04.726798+0000","last_became_peered":"2024-04-25T11:15:04.726798+0000","last_unstale":"2024-04-25T11:15:05.734667+0000","last_undegraded":"2024-04-25T11:15:05.734667+0000","last_fullsized":"2024-04-25T11:15:05.734667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:04.700414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.723509+0000","last_change":"2024-04-25T11:15:05.732329+0000","last_active":"2024-04-25T11:15:06.723509+0000","last_peered":"2024-04-25T11:15:06.723509+0000","last_clean":"2024-04-25T11:15:06.723509+0000","last_became_active":"2024-04-25T11:15:05.731516+0000","last_became_peered":"2024-04-25T11:15:05.731516+0000","last_unstale":"2024-04-25T11:15:06.723509+0000","last_undegraded":"2024-04-25T11:15:06.723509+0000","last_fullsized":"2024-04-25T11:15:06.723509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:54.469610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.738045+0000","last_change":"2024-04-25T11:15:04.731220+0000","last_active":"2024-04-25T11:15:05.738045+0000","last_peered":"2024-04-25T11:15:05.738045+0000","last_clean":"2024-04-25T11:15:05.738045+0000","last_became_active":"2024-04-25T11:15:04.730506+0000","last_became_peered":"2024-04-25T11:15:04.730506+0000","last_unstale":"2024-04-25T11:15:05.738045+0000","last_undegraded":"2024-04-25T11:15:05.738045+0000","last_fullsized":"2024-04-25T11:15:05.738045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:28.904734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.720096+0000","last_change":"2024-04-25T11:15:05.716373+0000","last_active":"2024-04-25T11:15:05.720096+0000","last_peered":"2024-04-25T11:15:05.720096+0000","last_clean":"2024-04-25T11:15:05.720096+0000","last_became_active":"2024-04-25T11:15:05.715900+0000","last_became_peered":"2024-04-25T11:15:05.715900+0000","last_unstale":"2024-04-25T11:15:05.720096+0000","last_undegraded":"2024-04-25T11:15:05.720096+0000","last_fullsized":"2024-04-25T11:15:05.720096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:08.356278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722704+0000","last_change":"2024-04-25T11:15:04.723351+0000","last_active":"2024-04-25T11:15:06.722704+0000","last_peered":"2024-04-25T11:15:06.722704+0000","last_clean":"2024-04-25T11:15:06.722704+0000","last_became_active":"2024-04-25T11:15:04.722980+0000","last_became_peered":"2024-04-25T11:15:04.722980+0000","last_unstale":"2024-04-25T11:15:06.722704+0000","last_undegraded":"2024-04-25T11:15:06.722704+0000","last_fullsized":"2024-04-25T11:15:06.722704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:21.741485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.721384+0000","last_change":"2024-04-25T11:15:05.718231+0000","last_active":"2024-04-25T11:15:05.721384+0000","last_peered":"2024-04-25T11:15:05.721384+0000","last_clean":"2024-04-25T11:15:05.721384+0000","last_became_active":"2024-04-25T11:15:05.717839+0000","last_became_peered":"2024-04-25T11:15:05.717839+0000","last_unstale":"2024-04-25T11:15:05.721384+0000","last_undegraded":"2024-04-25T11:15:05.721384+0000","last_fullsized":"2024-04-25T11:15:05.721384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:55.988121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.729697+0000","last_change":"2024-04-25T11:15:04.721388+0000","last_active":"2024-04-25T11:15:06.729697+0000","last_peered":"2024-04-25T11:15:06.729697+0000","last_clean":"2024-04-25T11:15:06.729697+0000","last_became_active":"2024-04-25T11:15:04.721030+0000","last_became_peered":"2024-04-25T11:15:04.721030+0000","last_unstale":"2024-04-25T11:15:06.729697+0000","last_undegraded":"2024-04-25T11:15:06.729697+0000","last_fullsized":"2024-04-25T11:15:06.729697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:05.895836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722314+0000","last_change":"2024-04-25T11:15:05.741046+0000","last_active":"2024-04-25T11:15:06.722314+0000","last_peered":"2024-04-25T11:15:06.722314+0000","last_clean":"2024-04-25T11:15:06.722314+0000","last_became_active":"2024-04-25T11:15:05.740545+0000","last_became_peered":"2024-04-25T11:15:05.740545+0000","last_unstale":"2024-04-25T11:15:06.722314+0000","last_undegraded":"2024-04-25T11:15:06.722314+0000","last_fullsized":"2024-04-25T11:15:06.722314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:01.008689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.690353+0000","last_change":"2024-04-25T11:15:04.728429+0000","last_active":"2024-04-25T11:15:05.690353+0000","last_peered":"2024-04-25T11:15:05.690353+0000","last_clean":"2024-04-25T11:15:05.690353+0000","last_became_active":"2024-04-25T11:15:04.728040+0000","last_became_peered":"2024-04-25T11:15:04.728040+0000","last_unstale":"2024-04-25T11:15:05.690353+0000","last_undegraded":"2024-04-25T11:15:05.690353+0000","last_fullsized":"2024-04-25T11:15:05.690353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:44.961588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.717238+0000","last_change":"2024-04-25T11:15:05.708743+0000","last_active":"2024-04-25T11:15:05.717238+0000","last_peered":"2024-04-25T11:15:05.717238+0000","last_clean":"2024-04-25T11:15:05.717238+0000","last_became_active":"2024-04-25T11:15:05.707530+0000","last_became_peered":"2024-04-25T11:15:05.707530+0000","last_unstale":"2024-04-25T11:15:05.717238+0000","last_undegraded":"2024-04-25T11:15:05.717238+0000","last_fullsized":"2024-04-25T11:15:05.717238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:27.757667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.684015+0000","last_change":"2024-04-25T11:15:04.709778+0000","last_active":"2024-04-25T11:15:05.684015+0000","last_peered":"2024-04-25T11:15:05.684015+0000","last_clean":"2024-04-25T11:15:05.684015+0000","last_became_active":"2024-04-25T11:15:04.709330+0000","last_became_peered":"2024-04-25T11:15:04.709330+0000","last_unstale":"2024-04-25T11:15:05.684015+0000","last_undegraded":"2024-04-25T11:15:05.684015+0000","last_fullsized":"2024-04-25T11:15:05.684015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:11.194902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.838136+0000","last_change":"2024-04-25T11:15:05.719161+0000","last_active":"2024-04-25T11:15:06.838136+0000","last_peered":"2024-04-25T11:15:06.838136+0000","last_clean":"2024-04-25T11:15:06.838136+0000","last_became_active":"2024-04-25T11:15:05.718849+0000","last_became_peered":"2024-04-25T11:15:05.718849+0000","last_unstale":"2024-04-25T11:15:06.838136+0000","last_undegraded":"2024-04-25T11:15:06.838136+0000","last_fullsized":"2024-04-25T11:15:06.838136+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:41.946329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"79'3","reported_seq":17,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731400+0000","last_change":"2024-04-25T11:15:04.717802+0000","last_active":"2024-04-25T11:15:06.731400+0000","last_peered":"2024-04-25T11:15:06.731400+0000","last_clean":"2024-04-25T11:15:06.731400+0000","last_became_active":"2024-04-25T11:15:04.717425+0000","last_became_peered":"2024-04-25T11:15:04.717425+0000","last_unstale":"2024-04-25T11:15:06.731400+0000","last_undegraded":"2024-04-25T11:15:06.731400+0000","last_fullsized":"2024-04-25T11:15:06.731400+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:53.503373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837866+0000","last_change":"2024-04-25T11:15:05.743925+0000","last_active":"2024-04-25T11:15:06.837866+0000","last_peered":"2024-04-25T11:15:06.837866+0000","last_clean":"2024-04-25T11:15:06.837866+0000","last_became_active":"2024-04-25T11:15:05.743562+0000","last_became_peered":"2024-04-25T11:15:05.743562+0000","last_unstale":"2024-04-25T11:15:06.837866+0000","last_undegraded":"2024-04-25T11:15:06.837866+0000","last_fullsized":"2024-04-25T11:15:06.837866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:55.988121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.741444+0000","last_change":"2024-04-25T11:15:04.726433+0000","last_active":"2024-04-25T11:15:05.741444+0000","last_peered":"2024-04-25T11:15:05.741444+0000","last_clean":"2024-04-25T11:15:05.741444+0000","last_became_active":"2024-04-25T11:15:04.726095+0000","last_became_peered":"2024-04-25T11:15:04.726095+0000","last_unstale":"2024-04-25T11:15:05.741444+0000","last_undegraded":"2024-04-25T11:15:05.741444+0000","last_fullsized":"2024-04-25T11:15:05.741444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:26.309047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.722439+0000","last_change":"2024-04-25T11:15:05.717105+0000","last_active":"2024-04-25T11:15:05.722439+0000","last_peered":"2024-04-25T11:15:05.722439+0000","last_clean":"2024-04-25T11:15:05.722439+0000","last_became_active":"2024-04-25T11:15:05.716594+0000","last_became_peered":"2024-04-25T11:15:05.716594+0000","last_unstale":"2024-04-25T11:15:05.722439+0000","last_undegraded":"2024-04-25T11:15:05.722439+0000","last_fullsized":"2024-04-25T11:15:05.722439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:28.810242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.736158+0000","last_change":"2024-04-25T11:15:04.729428+0000","last_active":"2024-04-25T11:15:05.736158+0000","last_peered":"2024-04-25T11:15:05.736158+0000","last_clean":"2024-04-25T11:15:05.736158+0000","last_became_active":"2024-04-25T11:15:04.729036+0000","last_became_peered":"2024-04-25T11:15:04.729036+0000","last_unstale":"2024-04-25T11:15:05.736158+0000","last_undegraded":"2024-04-25T11:15:05.736158+0000","last_fullsized":"2024-04-25T11:15:05.736158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:14.848494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.721342+0000","last_change":"2024-04-25T11:15:05.717256+0000","last_active":"2024-04-25T11:15:05.721342+0000","last_peered":"2024-04-25T11:15:05.721342+0000","last_clean":"2024-04-25T11:15:05.721342+0000","last_became_active":"2024-04-25T11:15:05.715993+0000","last_became_peered":"2024-04-25T11:15:05.715993+0000","last_unstale":"2024-04-25T11:15:05.721342+0000","last_undegraded":"2024-04-25T11:15:05.721342+0000","last_fullsized":"2024-04-25T11:15:05.721342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:29.681447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.685737+0000","last_change":"2024-04-25T11:15:04.711249+0000","last_active":"2024-04-25T11:15:05.685737+0000","last_peered":"2024-04-25T11:15:05.685737+0000","last_clean":"2024-04-25T11:15:05.685737+0000","last_became_active":"2024-04-25T11:15:04.710618+0000","last_became_peered":"2024-04-25T11:15:04.710618+0000","last_unstale":"2024-04-25T11:15:05.685737+0000","last_undegraded":"2024-04-25T11:15:05.685737+0000","last_fullsized":"2024-04-25T11:15:05.685737+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:27.792221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.718976+0000","last_change":"2024-04-25T11:15:05.709420+0000","last_active":"2024-04-25T11:15:05.718976+0000","last_peered":"2024-04-25T11:15:05.718976+0000","last_clean":"2024-04-25T11:15:05.718976+0000","last_became_active":"2024-04-25T11:15:05.708866+0000","last_became_peered":"2024-04-25T11:15:05.708866+0000","last_unstale":"2024-04-25T11:15:05.718976+0000","last_undegraded":"2024-04-25T11:15:05.718976+0000","last_fullsized":"2024-04-25T11:15:05.718976+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:09.414521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.729735+0000","last_change":"2024-04-25T11:15:04.721687+0000","last_active":"2024-04-25T11:15:06.729735+0000","last_peered":"2024-04-25T11:15:06.729735+0000","last_clean":"2024-04-25T11:15:06.729735+0000","last_became_active":"2024-04-25T11:15:04.720980+0000","last_became_peered":"2024-04-25T11:15:04.720980+0000","last_unstale":"2024-04-25T11:15:06.729735+0000","last_undegraded":"2024-04-25T11:15:06.729735+0000","last_fullsized":"2024-04-25T11:15:06.729735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:55.892571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.724605+0000","last_change":"2024-04-25T11:15:05.723798+0000","last_active":"2024-04-25T11:15:05.724605+0000","last_peered":"2024-04-25T11:15:05.724605+0000","last_clean":"2024-04-25T11:15:05.724605+0000","last_became_active":"2024-04-25T11:15:05.723405+0000","last_became_peered":"2024-04-25T11:15:05.723405+0000","last_unstale":"2024-04-25T11:15:05.724605+0000","last_undegraded":"2024-04-25T11:15:05.724605+0000","last_fullsized":"2024-04-25T11:15:05.724605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:29.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"79'8","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722628+0000","last_change":"2024-04-25T11:15:04.724962+0000","last_active":"2024-04-25T11:15:06.722628+0000","last_peered":"2024-04-25T11:15:06.722628+0000","last_clean":"2024-04-25T11:15:06.722628+0000","last_became_active":"2024-04-25T11:15:04.724620+0000","last_became_peered":"2024-04-25T11:15:04.724620+0000","last_unstale":"2024-04-25T11:15:06.722628+0000","last_undegraded":"2024-04-25T11:15:06.722628+0000","last_fullsized":"2024-04-25T11:15:06.722628+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:44.961588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.836476+0000","last_change":"2024-04-25T11:15:05.723591+0000","last_active":"2024-04-25T11:15:06.836476+0000","last_peered":"2024-04-25T11:15:06.836476+0000","last_clean":"2024-04-25T11:15:06.836476+0000","last_became_active":"2024-04-25T11:15:05.723220+0000","last_became_peered":"2024-04-25T11:15:05.723220+0000","last_unstale":"2024-04-25T11:15:06.836476+0000","last_undegraded":"2024-04-25T11:15:06.836476+0000","last_fullsized":"2024-04-25T11:15:06.836476+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:04.419956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.732450+0000","last_change":"2024-04-25T11:15:04.727452+0000","last_active":"2024-04-25T11:15:05.732450+0000","last_peered":"2024-04-25T11:15:05.732450+0000","last_clean":"2024-04-25T11:15:05.732450+0000","last_became_active":"2024-04-25T11:15:04.727130+0000","last_became_peered":"2024-04-25T11:15:04.727130+0000","last_unstale":"2024-04-25T11:15:05.732450+0000","last_undegraded":"2024-04-25T11:15:05.732450+0000","last_fullsized":"2024-04-25T11:15:05.732450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:25.431924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.729940+0000","last_change":"2024-04-25T11:15:05.711131+0000","last_active":"2024-04-25T11:15:06.729940+0000","last_peered":"2024-04-25T11:15:06.729940+0000","last_clean":"2024-04-25T11:15:06.729940+0000","last_became_active":"2024-04-25T11:15:05.710828+0000","last_became_peered":"2024-04-25T11:15:05.710828+0000","last_unstale":"2024-04-25T11:15:06.729940+0000","last_undegraded":"2024-04-25T11:15:06.729940+0000","last_fullsized":"2024-04-25T11:15:06.729940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:16.609062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.733942+0000","last_change":"2024-04-25T11:15:04.729489+0000","last_active":"2024-04-25T11:15:05.733942+0000","last_peered":"2024-04-25T11:15:05.733942+0000","last_clean":"2024-04-25T11:15:05.733942+0000","last_became_active":"2024-04-25T11:15:04.729131+0000","last_became_peered":"2024-04-25T11:15:04.729131+0000","last_unstale":"2024-04-25T11:15:05.733942+0000","last_undegraded":"2024-04-25T11:15:05.733942+0000","last_fullsized":"2024-04-25T11:15:05.733942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:22.097558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.729641+0000","last_change":"2024-04-25T11:15:05.732590+0000","last_active":"2024-04-25T11:15:06.729641+0000","last_peered":"2024-04-25T11:15:06.729641+0000","last_clean":"2024-04-25T11:15:06.729641+0000","last_became_active":"2024-04-25T11:15:05.730988+0000","last_became_peered":"2024-04-25T11:15:05.730988+0000","last_unstale":"2024-04-25T11:15:06.729641+0000","last_undegraded":"2024-04-25T11:15:06.729641+0000","last_fullsized":"2024-04-25T11:15:06.729641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:38.822542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"79'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.731468+0000","last_change":"2024-04-25T11:15:04.720302+0000","last_active":"2024-04-25T11:15:06.731468+0000","last_peered":"2024-04-25T11:15:06.731468+0000","last_clean":"2024-04-25T11:15:06.731468+0000","last_became_active":"2024-04-25T11:15:04.719921+0000","last_became_peered":"2024-04-25T11:15:04.719921+0000","last_unstale":"2024-04-25T11:15:06.731468+0000","last_undegraded":"2024-04-25T11:15:06.731468+0000","last_fullsized":"2024-04-25T11:15:06.731468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:37.693851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.836934+0000","last_change":"2024-04-25T11:15:05.726639+0000","last_active":"2024-04-25T11:15:06.836934+0000","last_peered":"2024-04-25T11:15:06.836934+0000","last_clean":"2024-04-25T11:15:06.836934+0000","last_became_active":"2024-04-25T11:15:05.723378+0000","last_became_peered":"2024-04-25T11:15:05.723378+0000","last_unstale":"2024-04-25T11:15:06.836934+0000","last_undegraded":"2024-04-25T11:15:06.836934+0000","last_fullsized":"2024-04-25T11:15:06.836934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:16.930741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837593+0000","last_change":"2024-04-25T11:15:04.710858+0000","last_active":"2024-04-25T11:15:06.837593+0000","last_peered":"2024-04-25T11:15:06.837593+0000","last_clean":"2024-04-25T11:15:06.837593+0000","last_became_active":"2024-04-25T11:15:04.710448+0000","last_became_peered":"2024-04-25T11:15:04.710448+0000","last_unstale":"2024-04-25T11:15:06.837593+0000","last_undegraded":"2024-04-25T11:15:06.837593+0000","last_fullsized":"2024-04-25T11:15:06.837593+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:29.997448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.723542+0000","last_change":"2024-04-25T11:15:05.730079+0000","last_active":"2024-04-25T11:15:06.723542+0000","last_peered":"2024-04-25T11:15:06.723542+0000","last_clean":"2024-04-25T11:15:06.723542+0000","last_became_active":"2024-04-25T11:15:05.723732+0000","last_became_peered":"2024-04-25T11:15:05.723732+0000","last_unstale":"2024-04-25T11:15:06.723542+0000","last_undegraded":"2024-04-25T11:15:06.723542+0000","last_fullsized":"2024-04-25T11:15:06.723542+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.750329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"79'2","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837392+0000","last_change":"2024-04-25T11:15:04.728894+0000","last_active":"2024-04-25T11:15:06.837392+0000","last_peered":"2024-04-25T11:15:06.837392+0000","last_clean":"2024-04-25T11:15:06.837392+0000","last_became_active":"2024-04-25T11:15:04.728102+0000","last_became_peered":"2024-04-25T11:15:04.728102+0000","last_unstale":"2024-04-25T11:15:06.837392+0000","last_undegraded":"2024-04-25T11:15:06.837392+0000","last_fullsized":"2024-04-25T11:15:06.837392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:58.653273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.724185+0000","last_change":"2024-04-25T11:15:05.739212+0000","last_active":"2024-04-25T11:15:06.724185+0000","last_peered":"2024-04-25T11:15:06.724185+0000","last_clean":"2024-04-25T11:15:06.724185+0000","last_became_active":"2024-04-25T11:15:05.738822+0000","last_became_peered":"2024-04-25T11:15:05.738822+0000","last_unstale":"2024-04-25T11:15:06.724185+0000","last_undegraded":"2024-04-25T11:15:06.724185+0000","last_fullsized":"2024-04-25T11:15:06.724185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:46.726035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.836549+0000","last_change":"2024-04-25T11:15:04.727438+0000","last_active":"2024-04-25T11:15:06.836549+0000","last_peered":"2024-04-25T11:15:06.836549+0000","last_clean":"2024-04-25T11:15:06.836549+0000","last_became_active":"2024-04-25T11:15:04.727072+0000","last_became_peered":"2024-04-25T11:15:04.727072+0000","last_unstale":"2024-04-25T11:15:06.836549+0000","last_undegraded":"2024-04-25T11:15:06.836549+0000","last_fullsized":"2024-04-25T11:15:06.836549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:33.534156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719032+0000","last_change":"2024-04-25T11:15:05.712026+0000","last_active":"2024-04-25T11:15:05.719032+0000","last_peered":"2024-04-25T11:15:05.719032+0000","last_clean":"2024-04-25T11:15:05.719032+0000","last_became_active":"2024-04-25T11:15:05.711359+0000","last_became_peered":"2024-04-25T11:15:05.711359+0000","last_unstale":"2024-04-25T11:15:05.719032+0000","last_undegraded":"2024-04-25T11:15:05.719032+0000","last_fullsized":"2024-04-25T11:15:05.719032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:24.383175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837098+0000","last_change":"2024-04-25T11:15:04.725067+0000","last_active":"2024-04-25T11:15:06.837098+0000","last_peered":"2024-04-25T11:15:06.837098+0000","last_clean":"2024-04-25T11:15:06.837098+0000","last_became_active":"2024-04-25T11:15:04.724688+0000","last_became_peered":"2024-04-25T11:15:04.724688+0000","last_unstale":"2024-04-25T11:15:06.837098+0000","last_undegraded":"2024-04-25T11:15:06.837098+0000","last_fullsized":"2024-04-25T11:15:06.837098+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:10.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.711634+0000","last_change":"2024-04-25T11:15:05.706526+0000","last_active":"2024-04-25T11:15:05.711634+0000","last_peered":"2024-04-25T11:15:05.711634+0000","last_clean":"2024-04-25T11:15:05.711634+0000","last_became_active":"2024-04-25T11:15:05.706195+0000","last_became_peered":"2024-04-25T11:15:05.706195+0000","last_unstale":"2024-04-25T11:15:05.711634+0000","last_undegraded":"2024-04-25T11:15:05.711634+0000","last_fullsized":"2024-04-25T11:15:05.711634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:44.117099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.735586+0000","last_change":"2024-04-25T11:15:04.711192+0000","last_active":"2024-04-25T11:15:05.735586+0000","last_peered":"2024-04-25T11:15:05.735586+0000","last_clean":"2024-04-25T11:15:05.735586+0000","last_became_active":"2024-04-25T11:15:04.710723+0000","last_became_peered":"2024-04-25T11:15:04.710723+0000","last_unstale":"2024-04-25T11:15:05.735586+0000","last_undegraded":"2024-04-25T11:15:05.735586+0000","last_fullsized":"2024-04-25T11:15:05.735586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:54.764754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.718949+0000","last_change":"2024-04-25T11:15:05.710287+0000","last_active":"2024-04-25T11:15:05.718949+0000","last_peered":"2024-04-25T11:15:05.718949+0000","last_clean":"2024-04-25T11:15:05.718949+0000","last_became_active":"2024-04-25T11:15:05.709953+0000","last_became_peered":"2024-04-25T11:15:05.709953+0000","last_unstale":"2024-04-25T11:15:05.718949+0000","last_undegraded":"2024-04-25T11:15:05.718949+0000","last_fullsized":"2024-04-25T11:15:05.718949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:24.738409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"80'1","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.724420+0000","last_change":"2024-04-25T11:15:04.716460+0000","last_active":"2024-04-25T11:15:06.724420+0000","last_peered":"2024-04-25T11:15:06.724420+0000","last_clean":"2024-04-25T11:15:06.724420+0000","last_became_active":"2024-04-25T11:15:04.716106+0000","last_became_peered":"2024-04-25T11:15:04.716106+0000","last_unstale":"2024-04-25T11:15:06.724420+0000","last_undegraded":"2024-04-25T11:15:06.724420+0000","last_fullsized":"2024-04-25T11:15:06.724420+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:04.035365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.737809+0000","last_change":"2024-04-25T11:15:05.733939+0000","last_active":"2024-04-25T11:15:05.737809+0000","last_peered":"2024-04-25T11:15:05.737809+0000","last_clean":"2024-04-25T11:15:05.737809+0000","last_became_active":"2024-04-25T11:15:05.732073+0000","last_became_peered":"2024-04-25T11:15:05.732073+0000","last_unstale":"2024-04-25T11:15:05.737809+0000","last_undegraded":"2024-04-25T11:15:05.737809+0000","last_fullsized":"2024-04-25T11:15:05.737809+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:52.306818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.689130+0000","last_change":"2024-04-25T11:15:04.715659+0000","last_active":"2024-04-25T11:15:05.689130+0000","last_peered":"2024-04-25T11:15:05.689130+0000","last_clean":"2024-04-25T11:15:05.689130+0000","last_became_active":"2024-04-25T11:15:04.715297+0000","last_became_peered":"2024-04-25T11:15:04.715297+0000","last_unstale":"2024-04-25T11:15:05.689130+0000","last_undegraded":"2024-04-25T11:15:05.689130+0000","last_fullsized":"2024-04-25T11:15:05.689130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:21.741485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.722911+0000","last_change":"2024-04-25T11:15:05.736431+0000","last_active":"2024-04-25T11:15:06.722911+0000","last_peered":"2024-04-25T11:15:06.722911+0000","last_clean":"2024-04-25T11:15:06.722911+0000","last_became_active":"2024-04-25T11:15:05.732065+0000","last_became_peered":"2024-04-25T11:15:05.732065+0000","last_unstale":"2024-04-25T11:15:06.722911+0000","last_undegraded":"2024-04-25T11:15:06.722911+0000","last_fullsized":"2024-04-25T11:15:06.722911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:08:03.352675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.687791+0000","last_change":"2024-04-25T11:15:04.723594+0000","last_active":"2024-04-25T11:15:05.687791+0000","last_peered":"2024-04-25T11:15:05.687791+0000","last_clean":"2024-04-25T11:15:05.687791+0000","last_became_active":"2024-04-25T11:15:04.722208+0000","last_became_peered":"2024-04-25T11:15:04.722208+0000","last_unstale":"2024-04-25T11:15:05.687791+0000","last_undegraded":"2024-04-25T11:15:05.687791+0000","last_fullsized":"2024-04-25T11:15:05.687791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:55.675896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.722436+0000","last_change":"2024-04-25T11:15:05.718712+0000","last_active":"2024-04-25T11:15:05.722436+0000","last_peered":"2024-04-25T11:15:05.722436+0000","last_clean":"2024-04-25T11:15:05.722436+0000","last_became_active":"2024-04-25T11:15:05.718393+0000","last_became_peered":"2024-04-25T11:15:05.718393+0000","last_unstale":"2024-04-25T11:15:05.722436+0000","last_undegraded":"2024-04-25T11:15:05.722436+0000","last_fullsized":"2024-04-25T11:15:05.722436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:07.009824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.693833+0000","last_change":"2024-04-25T11:15:04.725398+0000","last_active":"2024-04-25T11:15:05.693833+0000","last_peered":"2024-04-25T11:15:05.693833+0000","last_clean":"2024-04-25T11:15:05.693833+0000","last_became_active":"2024-04-25T11:15:04.724971+0000","last_became_peered":"2024-04-25T11:15:04.724971+0000","last_unstale":"2024-04-25T11:15:05.693833+0000","last_undegraded":"2024-04-25T11:15:05.693833+0000","last_fullsized":"2024-04-25T11:15:05.693833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:55.464864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719121+0000","last_change":"2024-04-25T11:15:05.713626+0000","last_active":"2024-04-25T11:15:05.719121+0000","last_peered":"2024-04-25T11:15:05.719121+0000","last_clean":"2024-04-25T11:15:05.719121+0000","last_became_active":"2024-04-25T11:15:05.713202+0000","last_became_peered":"2024-04-25T11:15:05.713202+0000","last_unstale":"2024-04-25T11:15:05.719121+0000","last_undegraded":"2024-04-25T11:15:05.719121+0000","last_fullsized":"2024-04-25T11:15:05.719121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:57.212790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.730271+0000","last_change":"2024-04-25T11:15:04.728786+0000","last_active":"2024-04-25T11:15:06.730271+0000","last_peered":"2024-04-25T11:15:06.730271+0000","last_clean":"2024-04-25T11:15:06.730271+0000","last_became_active":"2024-04-25T11:15:04.727792+0000","last_became_peered":"2024-04-25T11:15:04.727792+0000","last_unstale":"2024-04-25T11:15:06.730271+0000","last_undegraded":"2024-04-25T11:15:06.730271+0000","last_fullsized":"2024-04-25T11:15:06.730271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:28.157302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.839194+0000","last_change":"2024-04-25T11:15:05.744353+0000","last_active":"2024-04-25T11:15:06.839194+0000","last_peered":"2024-04-25T11:15:06.839194+0000","last_clean":"2024-04-25T11:15:06.839194+0000","last_became_active":"2024-04-25T11:15:05.743591+0000","last_became_peered":"2024-04-25T11:15:05.743591+0000","last_unstale":"2024-04-25T11:15:06.839194+0000","last_undegraded":"2024-04-25T11:15:06.839194+0000","last_fullsized":"2024-04-25T11:15:06.839194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:29.922755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.693695+0000","last_change":"2024-04-25T11:15:04.730935+0000","last_active":"2024-04-25T11:15:05.693695+0000","last_peered":"2024-04-25T11:15:05.693695+0000","last_clean":"2024-04-25T11:15:05.693695+0000","last_became_active":"2024-04-25T11:15:04.728369+0000","last_became_peered":"2024-04-25T11:15:04.728369+0000","last_unstale":"2024-04-25T11:15:05.693695+0000","last_undegraded":"2024-04-25T11:15:05.693695+0000","last_fullsized":"2024-04-25T11:15:05.693695+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:24.109091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.839450+0000","last_change":"2024-04-25T11:15:05.709270+0000","last_active":"2024-04-25T11:15:06.839450+0000","last_peered":"2024-04-25T11:15:06.839450+0000","last_clean":"2024-04-25T11:15:06.839450+0000","last_became_active":"2024-04-25T11:15:05.708846+0000","last_became_peered":"2024-04-25T11:15:05.708846+0000","last_unstale":"2024-04-25T11:15:06.839450+0000","last_undegraded":"2024-04-25T11:15:06.839450+0000","last_fullsized":"2024-04-25T11:15:06.839450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:55.782775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.837374+0000","last_change":"2024-04-25T11:15:04.723617+0000","last_active":"2024-04-25T11:15:06.837374+0000","last_peered":"2024-04-25T11:15:06.837374+0000","last_clean":"2024-04-25T11:15:06.837374+0000","last_became_active":"2024-04-25T11:15:04.722528+0000","last_became_peered":"2024-04-25T11:15:04.722528+0000","last_unstale":"2024-04-25T11:15:06.837374+0000","last_undegraded":"2024-04-25T11:15:06.837374+0000","last_fullsized":"2024-04-25T11:15:06.837374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:24.109091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.719849+0000","last_change":"2024-04-25T11:15:05.716535+0000","last_active":"2024-04-25T11:15:05.719849+0000","last_peered":"2024-04-25T11:15:05.719849+0000","last_clean":"2024-04-25T11:15:05.719849+0000","last_became_active":"2024-04-25T11:15:05.714334+0000","last_became_peered":"2024-04-25T11:15:05.714334+0000","last_unstale":"2024-04-25T11:15:05.719849+0000","last_undegraded":"2024-04-25T11:15:05.719849+0000","last_fullsized":"2024-04-25T11:15:05.719849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:04.665839+0000","last_clean_scrub_stamp":"2024-04-25T11:15:04.665839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:38.268352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T11:15:06.838711+0000","last_change":"2024-04-25T11:15:04.725224+0000","last_active":"2024-04-25T11:15:06.838711+0000","last_peered":"2024-04-25T11:15:06.838711+0000","last_clean":"2024-04-25T11:15:06.838711+0000","last_became_active":"2024-04-25T11:15:04.724784+0000","last_became_peered":"2024-04-25T11:15:04.724784+0000","last_unstale":"2024-04-25T11:15:06.838711+0000","last_undegraded":"2024-04-25T11:15:06.838711+0000","last_fullsized":"2024-04-25T11:15:06.838711+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:15:03.647818+0000","last_clean_scrub_stamp":"2024-04-25T11:15:03.647818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:06.389713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":225,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T11:15:05.685514+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:15:05.685514+0000","last_peered":"2024-04-25T11:15:05.685514+0000","last_clean":"2024-04-25T11:15:05.685514+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:15:05.685514+0000","last_undegraded":"2024-04-25T11:15:05.685514+0000","last_fullsized":"2024-04-25T11:15:05.685514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476797,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1552384,"data_stored":1023327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476798,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":954368,"data_stored":432937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476797,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":425770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476798,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":974848,"data_stored":434653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":970752,"data_stored":440211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509503,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":974848,"data_stored":439741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":1029997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542207,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":440371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:15:11.057 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:15:11.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:15:11.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.394+0000 7fb4a6e00640 1 Processor -- start 2024-04-25T11:15:11.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.394+0000 7fb4a6e00640 1 -- start start 2024-04-25T11:15:11.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.396+0000 7fb4a6e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a00b90f0 0x7fb4a006db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.396+0000 7fb4a6e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4a00d29a0 0x7fb4a00d1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.396+0000 7fb4a6e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a00b8df0 con 0x7fb4a00d2490 2024-04-25T11:15:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.396+0000 7fb4a6e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a009ccc0 con 0x7fb4a00d29a0 2024-04-25T11:15:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.396+0000 7fb4a6e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a006d820 con 0x7fb4a00b90f0 2024-04-25T11:15:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.397+0000 7fb4a5000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4a00d29a0 0x7fb4a00d1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.397+0000 7fb4a6400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb4a00d2490 0x7fb4a01644c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39196/0 (socket says 172.21.15.7:39196) 2024-04-25T11:15:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.397+0000 7fb4a6400640 1 -- 172.21.15.7:0/4170174850 learned_addr learned my addr 172.21.15.7:0/4170174850 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.397+0000 7fb4a5a00640 1 --2- 172.21.15.7:0/4170174850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a00b90f0 0x7fb4a006db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.397+0000 7fb4a5000640 1 -- 172.21.15.7:0/4170174850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a00b90f0 msgr2=0x7fb4a006db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.398+0000 7fb4a5000640 1 --2- 172.21.15.7:0/4170174850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a00b90f0 0x7fb4a006db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.398+0000 7fb4a5000640 1 -- 172.21.15.7:0/4170174850 >> v1:172.21.15.7:6789/0 conn(0x7fb4a00d2490 legacy=0x7fb4a01644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.398+0000 7fb4a5000640 1 -- 172.21.15.7:0/4170174850 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a00d0da0 con 0x7fb4a00d29a0 2024-04-25T11:15:11.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.398+0000 7fb49fe00640 1 -- 172.21.15.7:0/4170174850 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577213495 0 0) 0x7fb4a00b8df0 con 0x7fb4a00d2490 2024-04-25T11:15:11.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.399+0000 7fb4a5000640 1 --2- 172.21.15.7:0/4170174850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a00d29a0 0x7fb4a00d1f50 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009d90 tx=0x7fb48c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d6ef334e8a8f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:11.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.399+0000 7fb49fe00640 1 -- 172.21.15.7:0/4170174850 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c007740 con 0x7fb4a00d29a0 2024-04-25T11:15:11.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.399+0000 7fb49fe00640 1 -- 172.21.15.7:0/4170174850 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb48c0078e0 con 0x7fb4a00d29a0 2024-04-25T11:15:11.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.399+0000 7fb49fe00640 1 -- 172.21.15.7:0/4170174850 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c007cb0 con 0x7fb4a00d29a0 2024-04-25T11:15:11.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.400+0000 7fb4a6e00640 1 -- 172.21.15.7:0/4170174850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a00d29a0 msgr2=0x7fb4a00d1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:11.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.400+0000 7fb4a6e00640 1 --2- 172.21.15.7:0/4170174850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a00d29a0 0x7fb4a00d1f50 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009d90 tx=0x7fb48c02a660 comp rx=0 tx=0).stop 2024-04-25T11:15:11.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.400+0000 7fb4a6e00640 1 -- 172.21.15.7:0/4170174850 shutdown_connections 2024-04-25T11:15:11.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.400+0000 7fb4a6e00640 1 --2- 172.21.15.7:0/4170174850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a00d29a0 0x7fb4a00d1f50 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.400+0000 7fb4a6e00640 1 --2- 172.21.15.7:0/4170174850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a00b90f0 0x7fb4a006db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.400+0000 7fb4a6e00640 1 -- 172.21.15.7:0/4170174850 >> 172.21.15.7:0/4170174850 conn(0x7fb4a00d0280 msgr2=0x7fb4a00d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.400+0000 7fb4a6e00640 1 -- 172.21.15.7:0/4170174850 shutdown_connections 2024-04-25T11:15:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.400+0000 7fb4a6e00640 1 -- 172.21.15.7:0/4170174850 wait complete. 2024-04-25T11:15:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.402+0000 7fb4a6e00640 1 Processor -- start 2024-04-25T11:15:11.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.403+0000 7fb4a6e00640 1 -- start start 2024-04-25T11:15:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.404+0000 7fb4a6e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a00b90f0 0x7fb4a00b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.404+0000 7fb4a6e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a00d2490 0x7fb4a00b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.404+0000 7fb4a6e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a00d29a0 0x7fb4a00b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.404+0000 7fb4a6e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a0164bd0 con 0x7fb4a00b90f0 2024-04-25T11:15:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.404+0000 7fb4a6e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a00d3040 con 0x7fb4a00d2490 2024-04-25T11:15:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.404+0000 7fb4a6e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a0164d50 con 0x7fb4a00d29a0 2024-04-25T11:15:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.404+0000 7fb4a5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a00b90f0 0x7fb4a00b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.405+0000 7fb4a5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a00b90f0 0x7fb4a00b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40708/0 (socket says 172.21.15.7:40708) 2024-04-25T11:15:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.405+0000 7fb4a5a00640 1 -- 172.21.15.7:0/3220726313 learned_addr learned my addr 172.21.15.7:0/3220726313 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.405+0000 7fb4a6400640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a00d29a0 0x7fb4a00b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.405+0000 7fb4a5000640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a00d2490 0x7fb4a00b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.405+0000 7fb4a5a00640 1 -- 172.21.15.7:0/3220726313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a00d29a0 msgr2=0x7fb4a00b79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.405+0000 7fb4a5a00640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a00d29a0 0x7fb4a00b79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.405+0000 7fb4a5a00640 1 -- 172.21.15.7:0/3220726313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a00d2490 msgr2=0x7fb4a00b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.405+0000 7fb4a5a00640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a00d2490 0x7fb4a00b7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.405+0000 7fb4a5a00640 1 -- 172.21.15.7:0/3220726313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a01515a0 con 0x7fb4a00b90f0 2024-04-25T11:15:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.405+0000 7fb4a5a00640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a00b90f0 0x7fb4a00b4530 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb49000d830 tx=0x7fb49000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:11.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.406+0000 7fb49d600640 1 -- 172.21.15.7:0/3220726313 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49000be10 con 0x7fb4a00b90f0 2024-04-25T11:15:11.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.406+0000 7fb49d600640 1 -- 172.21.15.7:0/3220726313 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb490005990 con 0x7fb4a00b90f0 2024-04-25T11:15:11.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.406+0000 7fb49d600640 1 -- 172.21.15.7:0/3220726313 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490005b80 con 0x7fb4a00b90f0 2024-04-25T11:15:11.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.406+0000 7fb4a6e00640 1 -- 172.21.15.7:0/3220726313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a0151880 con 0x7fb4a00b90f0 2024-04-25T11:15:11.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.406+0000 7fb4a6e00640 1 -- 172.21.15.7:0/3220726313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a014ece0 con 0x7fb4a00b90f0 2024-04-25T11:15:11.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.407+0000 7fb4a6e00640 1 -- 172.21.15.7:0/3220726313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a0163290 con 0x7fb4a00b90f0 2024-04-25T11:15:11.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.408+0000 7fb49d600640 1 -- 172.21.15.7:0/3220726313 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb49001b020 con 0x7fb4a00b90f0 2024-04-25T11:15:11.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.409+0000 7fb49d600640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4800744e0 0x7fb4800769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:11.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.409+0000 7fb49d600640 1 -- 172.21.15.7:0/3220726313 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fb4900951e0 con 0x7fb4a00b90f0 2024-04-25T11:15:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.410+0000 7fb4a5000640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4800744e0 0x7fb4800769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:11.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.410+0000 7fb4a5000640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4800744e0 0x7fb4800769f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb48c034000 tx=0x7fb48c033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:11.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.415+0000 7fb49d600640 1 -- 172.21.15.7:0/3220726313 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb49005f740 con 0x7fb4a00b90f0 2024-04-25T11:15:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.700+0000 7fb4a6e00640 1 -- 172.21.15.7:0/3220726313 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb4a006dd40 con 0x7fb4a00b90f0 2024-04-25T11:15:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.707+0000 7fb49d600640 1 -- 172.21.15.7:0/3220726313 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fb49005f040 con 0x7fb4a00b90f0 2024-04-25T11:15:11.707 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-25T11:15:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:15:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-04-25T11:15:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-04-25T11:15:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:15:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-04-25T11:15:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.709+0000 7fb4a6e00640 1 -- 172.21.15.7:0/3220726313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4800744e0 msgr2=0x7fb4800769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.709+0000 7fb4a6e00640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4800744e0 0x7fb4800769f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb48c034000 tx=0x7fb48c033040 comp rx=0 tx=0).stop 2024-04-25T11:15:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.709+0000 7fb4a6e00640 1 -- 172.21.15.7:0/3220726313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a00b90f0 msgr2=0x7fb4a00b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.709+0000 7fb4a6e00640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a00b90f0 0x7fb4a00b4530 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb49000d830 tx=0x7fb49000dcf0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.710+0000 7fb4a6e00640 1 -- 172.21.15.7:0/3220726313 shutdown_connections 2024-04-25T11:15:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.710+0000 7fb4a6e00640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4800744e0 0x7fb4800769f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.710+0000 7fb4a6e00640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a00d29a0 0x7fb4a00b79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.710+0000 7fb4a6e00640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a00d2490 0x7fb4a00b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.710+0000 7fb4a6e00640 1 --2- 172.21.15.7:0/3220726313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a00b90f0 0x7fb4a00b4530 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.710+0000 7fb4a6e00640 1 -- 172.21.15.7:0/3220726313 >> 172.21.15.7:0/3220726313 conn(0x7fb4a00d0280 msgr2=0x7fb4a00cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.710+0000 7fb4a6e00640 1 -- 172.21.15.7:0/3220726313 shutdown_connections 2024-04-25T11:15:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:11.710+0000 7fb4a6e00640 1 -- 172.21.15.7:0/3220726313 wait complete. 2024-04-25T11:15:11.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:15:11.994 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:15:12.004 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:15:12.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.022+0000 7f85fbe00640 1 Processor -- start 2024-04-25T11:15:12.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.023+0000 7f85fbe00640 1 -- start start 2024-04-25T11:15:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.025+0000 7f85fbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85fc163bb0 0x7f85fc0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.025+0000 7f85fbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f85fc0bae90 0x7f85fc0b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.025+0000 7f85fbe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85fc164120 con 0x7f85fc06c570 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.025+0000 7f85fbe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f85fc0d08e0 con 0x7f85fc0bae90 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.025+0000 7f85fbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f85fc0b51c0 con 0x7f85fc163bb0 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.026+0000 7f85fb400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f85fc06c570 0x7f85fc06c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39208/0 (socket says 172.21.15.7:39208) 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.026+0000 7f85fb400640 1 -- 172.21.15.7:0/2524386942 learned_addr learned my addr 172.21.15.7:0/2524386942 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.026+0000 7f85f2000640 1 --2- 172.21.15.7:0/2524386942 >> v2:172.21.15.87:3300/0 conn(0x7f85fc0bae90 0x7f85fc0b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.026+0000 7f85faa00640 1 --2- 172.21.15.7:0/2524386942 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85fc163bb0 0x7f85fc0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.026+0000 7f85fa000640 1 -- 172.21.15.7:0/2524386942 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1222448510 0 0) 0x7f85fc164120 con 0x7f85fc06c570 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.026+0000 7f85fa000640 1 -- 172.21.15.7:0/2524386942 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85e0003520 con 0x7f85fc06c570 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.026+0000 7f85f2000640 1 -- 172.21.15.7:0/2524386942 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85fc163bb0 msgr2=0x7f85fc0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.026+0000 7f85f2000640 1 --2- 172.21.15.7:0/2524386942 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85fc163bb0 0x7f85fc0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.026+0000 7f85f2000640 1 -- 172.21.15.7:0/2524386942 >> v1:172.21.15.7:6789/0 conn(0x7f85fc06c570 legacy=0x7f85fc06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:12.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.026+0000 7f85f2000640 1 -- 172.21.15.7:0/2524386942 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc164400 con 0x7f85fc0bae90 2024-04-25T11:15:12.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.026+0000 7f85faa00640 1 --2- 172.21.15.7:0/2524386942 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85fc163bb0 0x7f85fc0d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:15:12.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.028+0000 7f85f2000640 1 --2- 172.21.15.7:0/2524386942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85fc0bae90 0x7f85fc0b8d80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f85e40097a0 tx=0x7f85e402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc254787366475f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:12.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.028+0000 7f85fa000640 1 -- 172.21.15.7:0/2524386942 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4037070 con 0x7f85fc0bae90 2024-04-25T11:15:12.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.028+0000 7f85fa000640 1 -- 172.21.15.7:0/2524386942 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e4032440 con 0x7f85fc0bae90 2024-04-25T11:15:12.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.028+0000 7f85fa000640 1 -- 172.21.15.7:0/2524386942 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e40327f0 con 0x7f85fc0bae90 2024-04-25T11:15:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.029+0000 7f85fbe00640 1 -- 172.21.15.7:0/2524386942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85fc0bae90 msgr2=0x7f85fc0b8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.029+0000 7f85fbe00640 1 --2- 172.21.15.7:0/2524386942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85fc0bae90 0x7f85fc0b8d80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f85e40097a0 tx=0x7f85e402a180 comp rx=0 tx=0).stop 2024-04-25T11:15:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.029+0000 7f85fbe00640 1 -- 172.21.15.7:0/2524386942 shutdown_connections 2024-04-25T11:15:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.029+0000 7f85fbe00640 1 --2- 172.21.15.7:0/2524386942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85fc0bae90 0x7f85fc0b8d80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.029+0000 7f85fbe00640 1 --2- 172.21.15.7:0/2524386942 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85fc163bb0 0x7f85fc0d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.029+0000 7f85fbe00640 1 -- 172.21.15.7:0/2524386942 >> 172.21.15.7:0/2524386942 conn(0x7f85fc06eb00 msgr2=0x7f85fc0d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.029+0000 7f85fbe00640 1 -- 172.21.15.7:0/2524386942 shutdown_connections 2024-04-25T11:15:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.029+0000 7f85fbe00640 1 -- 172.21.15.7:0/2524386942 wait complete. 2024-04-25T11:15:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.032+0000 7f85fbe00640 1 Processor -- start 2024-04-25T11:15:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.032+0000 7f85fbe00640 1 -- start start 2024-04-25T11:15:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.033+0000 7f85fbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85fc06c570 0x7f85fc0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.033+0000 7f85fbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85fc0bae90 0x7f85fc0b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.033+0000 7f85fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc163bb0 0x7f85fc153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.033+0000 7f85fbe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc0d4170 con 0x7f85fc163bb0 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.033+0000 7f85fbe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc1649c0 con 0x7f85fc06c570 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.033+0000 7f85fbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f85fc0b92c0 con 0x7f85fc0bae90 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.034+0000 7f85fb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc163bb0 0x7f85fc153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.034+0000 7f85faa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85fc06c570 0x7f85fc0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.034+0000 7f85fb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc163bb0 0x7f85fc153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40710/0 (socket says 172.21.15.7:40710) 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.034+0000 7f85fb400640 1 -- 172.21.15.7:0/785135925 learned_addr learned my addr 172.21.15.7:0/785135925 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.034+0000 7f85f2000640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85fc0bae90 0x7f85fc0b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.034+0000 7f85fb400640 1 -- 172.21.15.7:0/785135925 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85fc0bae90 msgr2=0x7f85fc0b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.034+0000 7f85fb400640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85fc0bae90 0x7f85fc0b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:12.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.034+0000 7f85fb400640 1 -- 172.21.15.7:0/785135925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85fc06c570 msgr2=0x7f85fc0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.034+0000 7f85fb400640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85fc06c570 0x7f85fc0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.034+0000 7f85fb400640 1 -- 172.21.15.7:0/785135925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc0af3b0 con 0x7f85fc163bb0 2024-04-25T11:15:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.035+0000 7f85fb400640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc163bb0 0x7f85fc153dd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f85f400c390 tx=0x7f85f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.035+0000 7f85f3e00640 1 -- 172.21.15.7:0/785135925 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4017070 con 0x7f85fc163bb0 2024-04-25T11:15:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.035+0000 7f85f3e00640 1 -- 172.21.15.7:0/785135925 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f4005ce0 con 0x7f85fc163bb0 2024-04-25T11:15:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.035+0000 7f85fbe00640 1 -- 172.21.15.7:0/785135925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85fc0ae310 con 0x7f85fc163bb0 2024-04-25T11:15:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.035+0000 7f85f3e00640 1 -- 172.21.15.7:0/785135925 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f40072d0 con 0x7f85fc163bb0 2024-04-25T11:15:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.035+0000 7f85fbe00640 1 -- 172.21.15.7:0/785135925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85fc0ae8d0 con 0x7f85fc163bb0 2024-04-25T11:15:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.037+0000 7f85fbe00640 1 -- 172.21.15.7:0/785135925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85fc050740 con 0x7f85fc163bb0 2024-04-25T11:15:12.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.039+0000 7f85f3e00640 1 -- 172.21.15.7:0/785135925 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f85f401a070 con 0x7f85fc163bb0 2024-04-25T11:15:12.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.039+0000 7f85f3e00640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85d80744b0 0x7f85d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:12.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.040+0000 7f85f3e00640 1 -- 172.21.15.7:0/785135925 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f85f4094680 con 0x7f85fc163bb0 2024-04-25T11:15:12.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.040+0000 7f85faa00640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85d80744b0 0x7f85d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:12.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.041+0000 7f85faa00640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85d80744b0 0x7f85d80769c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009bd0 tx=0x7f85ec00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.044+0000 7f85f3e00640 1 -- 172.21.15.7:0/785135925 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85f405e590 con 0x7f85fc163bb0 2024-04-25T11:15:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.292+0000 7f85fbe00640 1 -- 172.21.15.7:0/785135925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85fc1632f0 con 0x7f85fc163bb0 2024-04-25T11:15:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.293+0000 7f85f3e00640 1 -- 172.21.15.7:0/785135925 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5171 (secure 0 0 0) 0x7f85f4062890 con 0x7f85fc163bb0 2024-04-25T11:15:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:12.294 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/779859707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":779859707},{"type":"v1","addr":"172.21.15.87:6834","nonce":779859707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/454388957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":454388957},{"type":"v1","addr":"172.21.15.87:6835","nonce":454388957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2318271315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2318271315},{"type":"v1","addr":"172.21.15.7:6836","nonce":2318271315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:15:05.259317+0000","modified":"2024-04-25T11:15:08.314094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6528},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6528":{"gid":6528,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2955530443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2955530443},{"type":"v1","addr":"172.21.15.7:6837","nonce":2955530443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6528,"qdb_cluster":[6528]},"id":5}]} 2024-04-25T11:15:12.295 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 43 2024-04-25T11:15:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.296+0000 7f85fbe00640 1 -- 172.21.15.7:0/785135925 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85d80744b0 msgr2=0x7f85d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.296+0000 7f85fbe00640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85d80744b0 0x7f85d80769c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009bd0 tx=0x7f85ec00b040 comp rx=0 tx=0).stop 2024-04-25T11:15:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.296+0000 7f85fbe00640 1 -- 172.21.15.7:0/785135925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc163bb0 msgr2=0x7f85fc153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.296+0000 7f85fbe00640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc163bb0 0x7f85fc153dd0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f85f400c390 tx=0x7f85f400c850 comp rx=0 tx=0).stop 2024-04-25T11:15:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.296+0000 7f85fbe00640 1 -- 172.21.15.7:0/785135925 shutdown_connections 2024-04-25T11:15:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.296+0000 7f85fbe00640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85d80744b0 0x7f85d80769c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.296+0000 7f85fbe00640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85fc163bb0 0x7f85fc153dd0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.296+0000 7f85fbe00640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85fc0bae90 0x7f85fc0b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.296+0000 7f85fbe00640 1 --2- 172.21.15.7:0/785135925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85fc06c570 0x7f85fc0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.296+0000 7f85fbe00640 1 -- 172.21.15.7:0/785135925 >> 172.21.15.7:0/785135925 conn(0x7f85fc06eb00 msgr2=0x7f85fc0d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.297+0000 7f85fbe00640 1 -- 172.21.15.7:0/785135925 shutdown_connections 2024-04-25T11:15:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:12.297+0000 7f85fbe00640 1 -- 172.21.15.7:0/785135925 wait complete. 2024-04-25T11:15:12.727 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:15:05.259317+0000', 'modified': '2024-04-25T11:15:08.314094+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6528}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6528': {'gid': 6528, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6837/2955530443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6835', 'nonce': 2955530443}, {'type': 'v1', 'addr': '172.21.15.7:6837', 'nonce': 2955530443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6528, 'qdb_cluster': [6528]} 2024-04-25T11:15:12.728 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:15:12.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:15:13.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.018+0000 7fcf86e00640 1 Processor -- start 2024-04-25T11:15:13.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.018+0000 7fcf86e00640 1 -- start start 2024-04-25T11:15:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.020+0000 7fcf86e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcf8014d980 0x7fcf8006cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.020+0000 7fcf86e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcf8006f7b0 0x7fcf8006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.020+0000 7fcf86e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf8006d470 con 0x7fcf800cf320 2024-04-25T11:15:13.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.020+0000 7fcf86e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8006d6f0 con 0x7fcf8006f7b0 2024-04-25T11:15:13.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.021+0000 7fcf86e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8006d9d0 con 0x7fcf8014d980 2024-04-25T11:15:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.021+0000 7fcf86400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcf8006f7b0 0x7fcf8006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.021+0000 7fcf86400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcf8006f7b0 0x7fcf8006fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40714/0 (socket says 172.21.15.7:40714) 2024-04-25T11:15:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.021+0000 7fcf86400640 1 -- 172.21.15.7:0/1472572866 learned_addr learned my addr 172.21.15.7:0/1472572866 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.021+0000 7fcf85a00640 1 --2- 172.21.15.7:0/1472572866 >> v2:172.21.15.87:3300/0 conn(0x7fcf8014d980 0x7fcf8006cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.021+0000 7fcf86400640 1 -- 172.21.15.7:0/1472572866 >> v2:172.21.15.87:3300/0 conn(0x7fcf8014d980 msgr2=0x7fcf8006cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.022+0000 7fcf86400640 1 --2- 172.21.15.7:0/1472572866 >> v2:172.21.15.87:3300/0 conn(0x7fcf8014d980 0x7fcf8006cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.022+0000 7fcf86400640 1 -- 172.21.15.7:0/1472572866 >> v1:172.21.15.87:6789/0 conn(0x7fcf800cf320 legacy=0x7fcf800cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.022+0000 7fcf86400640 1 -- 172.21.15.7:0/1472572866 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8006dcb0 con 0x7fcf8006f7b0 2024-04-25T11:15:13.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.023+0000 7fcf86400640 1 --2- 172.21.15.7:0/1472572866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf8006f7b0 0x7fcf8006fbc0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcf80073040 tx=0x7fcf7802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ede61a718574fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:13.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.023+0000 7fcf7fe00640 1 -- 172.21.15.7:0/1472572866 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78036070 con 0x7fcf8006f7b0 2024-04-25T11:15:13.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.023+0000 7fcf7fe00640 1 -- 172.21.15.7:0/1472572866 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf78031600 con 0x7fcf8006f7b0 2024-04-25T11:15:13.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.023+0000 7fcf7fe00640 1 -- 172.21.15.7:0/1472572866 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf780319d0 con 0x7fcf8006f7b0 2024-04-25T11:15:13.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.024+0000 7fcf86e00640 1 -- 172.21.15.7:0/1472572866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf8006f7b0 msgr2=0x7fcf8006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.024+0000 7fcf86e00640 1 --2- 172.21.15.7:0/1472572866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf8006f7b0 0x7fcf8006fbc0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fcf80073040 tx=0x7fcf7802a6c0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.024+0000 7fcf86e00640 1 -- 172.21.15.7:0/1472572866 shutdown_connections 2024-04-25T11:15:13.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.024+0000 7fcf86e00640 1 --2- 172.21.15.7:0/1472572866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf8006f7b0 0x7fcf8006fbc0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.024+0000 7fcf86e00640 1 --2- 172.21.15.7:0/1472572866 >> v2:172.21.15.87:3300/0 conn(0x7fcf8014d980 0x7fcf8006cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.024+0000 7fcf86e00640 1 -- 172.21.15.7:0/1472572866 >> 172.21.15.7:0/1472572866 conn(0x7fcf800d1540 msgr2=0x7fcf800d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:13.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.024+0000 7fcf86e00640 1 -- 172.21.15.7:0/1472572866 shutdown_connections 2024-04-25T11:15:13.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.024+0000 7fcf86e00640 1 -- 172.21.15.7:0/1472572866 wait complete. 2024-04-25T11:15:13.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.027+0000 7fcf86e00640 1 Processor -- start 2024-04-25T11:15:13.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.027+0000 7fcf86e00640 1 -- start start 2024-04-25T11:15:13.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.028+0000 7fcf86e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf8006f7b0 0x7fcf80166b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.028+0000 7fcf86e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf800cf320 0x7fcf800aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf86e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf8014d980 0x7fcf8015a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf86e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8014ab10 con 0x7fcf8006f7b0 2024-04-25T11:15:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf86e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8014dd90 con 0x7fcf8014d980 2024-04-25T11:15:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf86e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf8014ac90 con 0x7fcf800cf320 2024-04-25T11:15:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf85a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf8006f7b0 0x7fcf80166b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf85a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf8006f7b0 0x7fcf80166b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40716/0 (socket says 172.21.15.7:40716) 2024-04-25T11:15:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf85a00640 1 -- 172.21.15.7:0/769621311 learned_addr learned my addr 172.21.15.7:0/769621311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf85a00640 1 -- 172.21.15.7:0/769621311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf800cf320 msgr2=0x7fcf800aac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf85a00640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf800cf320 0x7fcf800aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf85a00640 1 -- 172.21.15.7:0/769621311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf8014d980 msgr2=0x7fcf8015a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf85a00640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf8014d980 0x7fcf8015a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.029+0000 7fcf85a00640 1 -- 172.21.15.7:0/769621311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf80167410 con 0x7fcf8006f7b0 2024-04-25T11:15:13.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.030+0000 7fcf85a00640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf8006f7b0 0x7fcf80166b10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400e8a0 tx=0x7fcf7400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:13.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.030+0000 7fcf7d600640 1 -- 172.21.15.7:0/769621311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74017070 con 0x7fcf8006f7b0 2024-04-25T11:15:13.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.030+0000 7fcf7d600640 1 -- 172.21.15.7:0/769621311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7400c940 con 0x7fcf8006f7b0 2024-04-25T11:15:13.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.030+0000 7fcf86e00640 1 -- 172.21.15.7:0/769621311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf800aa140 con 0x7fcf8006f7b0 2024-04-25T11:15:13.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.030+0000 7fcf7d600640 1 -- 172.21.15.7:0/769621311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400cd30 con 0x7fcf8006f7b0 2024-04-25T11:15:13.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.030+0000 7fcf86e00640 1 -- 172.21.15.7:0/769621311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf801fba20 con 0x7fcf8006f7b0 2024-04-25T11:15:13.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.033+0000 7fcf7d600640 1 -- 172.21.15.7:0/769621311 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf7401a070 con 0x7fcf8006f7b0 2024-04-25T11:15:13.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.034+0000 7fcf7d600640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf600744b0 0x7fcf600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.034+0000 7fcf7d600640 1 -- 172.21.15.7:0/769621311 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fcf74095300 con 0x7fcf8006f7b0 2024-04-25T11:15:13.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.035+0000 7fcf85000640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf600744b0 0x7fcf600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.035+0000 7fcf8f3f1740 1 Processor -- start 2024-04-25T11:15:13.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.035+0000 7fcf8f3f1740 1 -- start start 2024-04-25T11:15:13.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.035+0000 7fcf85000640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf600744b0 0x7fcf600769c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c006fd0 tx=0x7fcf6c006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.036+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f0356210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.036+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff603b0 0x5620f0370f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.036+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff68590 0x5620f0371480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.036+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620f028f600 con 0x5620eff89270 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.036+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5620f022c100 con 0x5620eff603b0 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.036+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5620f028b5c0 con 0x5620eff68590 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.037+0000 7fcf85a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f0356210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.037+0000 7fcf85a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f0356210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40724/0 (socket says 172.21.15.7:40724) 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.037+0000 7fcf85a00640 1 -- 172.21.15.7:0/1956117345 learned_addr learned my addr 172.21.15.7:0/1956117345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.037+0000 7fcf85a00640 1 -- 172.21.15.7:0/1956117345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff68590 msgr2=0x5620f0371480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.037+0000 7fcf85a00640 1 --2- 172.21.15.7:0/1956117345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff68590 0x5620f0371480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.037+0000 7fcf85a00640 1 -- 172.21.15.7:0/1956117345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff603b0 msgr2=0x5620f0370f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.037+0000 7fcf85a00640 1 --2- 172.21.15.7:0/1956117345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff603b0 0x5620f0370f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.037+0000 7fcf85a00640 1 -- 172.21.15.7:0/1956117345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620effa12d0 con 0x5620eff89270 2024-04-25T11:15:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.037+0000 7fcf85a00640 1 --2- 172.21.15.7:0/1956117345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f0356210 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fcf7405b270 tx=0x7fcf7405ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97db02adf7b0379b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.038+0000 7fcf72000640 1 -- 172.21.15.7:0/1956117345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74021040 con 0x5620eff89270 2024-04-25T11:15:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.038+0000 7fcf72000640 1 -- 172.21.15.7:0/1956117345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7405f1f0 con 0x5620eff89270 2024-04-25T11:15:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.038+0000 7fcf72000640 1 -- 172.21.15.7:0/1956117345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf740632d0 con 0x5620eff89270 2024-04-25T11:15:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.038+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/1956117345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 msgr2=0x5620f0356210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.038+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/1956117345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f0356210 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fcf7405b270 tx=0x7fcf7405ed00 comp rx=0 tx=0).stop 2024-04-25T11:15:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.039+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/1956117345 shutdown_connections 2024-04-25T11:15:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.039+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/1956117345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff68590 0x5620f0371480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.039+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/1956117345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff603b0 0x5620f0370f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.039+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/1956117345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f0356210 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.039+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/1956117345 >> 172.21.15.7:0/1956117345 conn(0x5620eff7d3c0 msgr2=0x5620f032b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.039+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/1956117345 shutdown_connections 2024-04-25T11:15:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.039+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/1956117345 wait complete. 2024-04-25T11:15:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.040+0000 7fcf8f3f1740 1 -- start start 2024-04-25T11:15:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.040+0000 7fcf8f3f1740 1 Processor -- start 2024-04-25T11:15:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff603b0 0x5620f0380a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 0x5620f03593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff89270 0x5620f03598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620f0216f70 con 0x5620eff603b0 2024-04-25T11:15:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5620f035a1b0 con 0x5620eff68590 2024-04-25T11:15:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5620f03814e0 con 0x5620eff89270 2024-04-25T11:15:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf8f3f1740 10 client.0 _ll_register_callbacks cb 0x5620f0285480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:15:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf85a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff603b0 0x5620f0380a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf85a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff603b0 0x5620f0380a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40726/0 (socket says 172.21.15.7:40726) 2024-04-25T11:15:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf85a00640 1 -- 172.21.15.7:0/2499595274 learned_addr learned my addr 172.21.15.7:0/2499595274 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf85000640 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 0x5620f03593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf85a00640 1 -- 172.21.15.7:0/2499595274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff89270 msgr2=0x5620f03598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf85a00640 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff89270 0x5620f03598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf85a00640 1 -- 172.21.15.7:0/2499595274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 msgr2=0x5620f03593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf85a00640 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 0x5620f03593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf85a00640 1 -- 172.21.15.7:0/2499595274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f006b6b0 con 0x5620eff603b0 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.042+0000 7fcf85000640 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 0x5620f03593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf85a00640 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff603b0 0x5620f0380a00 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fcf7405c460 tx=0x7fcf74000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fece4c0db890fd8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf66a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf66a00640 1 -- 172.21.15.7:0/2499595274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74090030 con 0x5620eff603b0 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf66a00640 1 -- 172.21.15.7:0/2499595274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fcf7405e660 con 0x5620eff603b0 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2499595274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf7405f1f0 con 0x5620eff603b0 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf66a00640 1 -- 172.21.15.7:0/2499595274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcf74021070 con 0x5620eff603b0 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf8f3f1740 10 client.6669 fetch_fsmap learned FSMap version 43 2024-04-25T11:15:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2499595274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5620f034fc60 con 0x5620eff603b0 2024-04-25T11:15:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf66a00640 1 -- 172.21.15.7:0/2499595274 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7405f1f0 con 0x5620eff603b0 2024-04-25T11:15:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf66a00640 1 -- 172.21.15.7:0/2499595274 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7fcf74024060 con 0x5620eff603b0 2024-04-25T11:15:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf8f3f1740 10 client.6669 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T11:15:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.043+0000 7fcf8f3f1740 10 client.6669 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:15:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.044+0000 7fcf8f3f1740 20 client.6669 populate_metadata read hostname 'smithi007' 2024-04-25T11:15:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.044+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.7:6835/2955530443,v1:172.21.15.7:6837/2955530443] conn(0x5620eff47e40 0x5620f0357780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.044+0000 7fcf8f3f1740 4 client.6669 mds_command: new command op to 6528 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:15:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.044+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2499595274 --> [v2:172.21.15.7:6835/2955530443,v1:172.21.15.7:6837/2955530443] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620f029d210 con 0x5620eff47e40 2024-04-25T11:15:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.044+0000 7fcf85000640 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.7:6835/2955530443,v1:172.21.15.7:6837/2955530443] conn(0x5620eff47e40 0x5620f0357780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.044+0000 7fcf85000640 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.7:6835/2955530443,v1:172.21.15.7:6837/2955530443] conn(0x5620eff47e40 0x5620f0357780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d39181fea0ff59 server_cookie=2f5e55bc7724f1e0 in_seq=0 out_seq=0 2024-04-25T11:15:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.044+0000 7fcf66a00640 10 client.6669 ms_handle_connect on v2:172.21.15.7:6835/2955530443 2024-04-25T11:15:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf66a00640 1 -- 172.21.15.7:0/2499595274 <== mds.0 v2:172.21.15.7:6835/2955530443 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5620f029d210 con 0x5620eff47e40 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf66a00640 10 client.6669 handle_command_reply: tid=0 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 1 client.6669 shutdown 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2499595274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff603b0 msgr2=0x5620f0380a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff603b0 0x5620f0380a00 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fcf7405c460 tx=0x7fcf74000bc0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2499595274 shutdown_connections 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.7:6835/2955530443,v1:172.21.15.7:6837/2955530443] conn(0x5620eff47e40 0x5620f0357780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff89270 0x5620f03598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 0x5620f03593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/2499595274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff603b0 0x5620f0380a00 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2499595274 >> 172.21.15.7:0/2499595274 conn(0x5620eff7d3c0 msgr2=0x5620f032b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf66a00640 0 client.6669 ms_handle_reset on v2:172.21.15.7:6835/2955530443 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2499595274 shutdown_connections 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2499595274 wait complete. 2024-04-25T11:15:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.046+0000 7fcf8f3f1740 20 client.6669 trim_cache size 0 max 16384 2024-04-25T11:15:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.067+0000 7fcf8f3f1740 1 Processor -- start 2024-04-25T11:15:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.067+0000 7fcf8f3f1740 1 -- start start 2024-04-25T11:15:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff603b0 0x5620f03568f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff68590 0x5620f0371040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff89270 0x5620f0372f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcf80206710 con 0x5620eff68590 2024-04-25T11:15:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf80164ba0 con 0x5620eff603b0 2024-04-25T11:15:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf801651f0 con 0x5620eff89270 2024-04-25T11:15:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf85000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff68590 0x5620f0371040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf85000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff68590 0x5620f0371040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40730/0 (socket says 172.21.15.7:40730) 2024-04-25T11:15:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf85000640 1 -- 172.21.15.7:0/2220224362 learned_addr learned my addr 172.21.15.7:0/2220224362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf85000640 1 -- 172.21.15.7:0/2220224362 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff89270 msgr2=0x5620f0372f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf85a00640 1 --2- 172.21.15.7:0/2220224362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff603b0 0x5620f03568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf85000640 1 --2- 172.21.15.7:0/2220224362 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff89270 0x5620f0372f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf85000640 1 -- 172.21.15.7:0/2220224362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff603b0 msgr2=0x5620f03568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.069+0000 7fcf85000640 1 --2- 172.21.15.7:0/2220224362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff603b0 0x5620f03568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.070+0000 7fcf85000640 1 -- 172.21.15.7:0/2220224362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f0255b70 con 0x5620eff68590 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.070+0000 7fcf85000640 1 --2- 172.21.15.7:0/2220224362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff68590 0x5620f0371040 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcf800cdb20 tx=0x7fcf6c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39851bfdbdfa6e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.070+0000 7fcf71600640 1 -- 172.21.15.7:0/2220224362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c027030 con 0x5620eff68590 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.070+0000 7fcf71600640 1 -- 172.21.15.7:0/2220224362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf6c01f890 con 0x5620eff68590 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.070+0000 7fcf71600640 1 -- 172.21.15.7:0/2220224362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c01fc60 con 0x5620eff68590 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.071+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2220224362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff68590 msgr2=0x5620f0371040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.071+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/2220224362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff68590 0x5620f0371040 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcf800cdb20 tx=0x7fcf6c00e000 comp rx=0 tx=0).stop 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.071+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2220224362 shutdown_connections 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.071+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/2220224362 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff89270 0x5620f0372f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.071+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/2220224362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff68590 0x5620f0371040 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.071+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/2220224362 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff603b0 0x5620f03568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.071+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2220224362 >> 172.21.15.7:0/2220224362 conn(0x5620eff7d3c0 msgr2=0x5620f036a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.071+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2220224362 shutdown_connections 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.071+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/2220224362 wait complete. 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.072+0000 7fcf8f3f1740 1 -- start start 2024-04-25T11:15:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.072+0000 7fcf8f3f1740 1 Processor -- start 2024-04-25T11:15:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.074+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff603b0 0x5620f0394840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.074+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 0x5620f03807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.074+0000 7fcf8f3f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f036a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.074+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620f036f7f0 con 0x5620eff89270 2024-04-25T11:15:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.074+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5620f0216f70 con 0x5620eff68590 2024-04-25T11:15:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.074+0000 7fcf8f3f1740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5620f03519a0 con 0x5620eff603b0 2024-04-25T11:15:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.074+0000 7fcf8f3f1740 10 client.0 _ll_register_callbacks cb 0x7fcf7405ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:15:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.074+0000 7fcf86400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f036a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.074+0000 7fcf86400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f036a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40744/0 (socket says 172.21.15.7:40744) 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.074+0000 7fcf86400640 1 -- 172.21.15.7:0/129261881 learned_addr learned my addr 172.21.15.7:0/129261881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.074+0000 7fcf85000640 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 0x5620f03807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.075+0000 7fcf86400640 1 -- 172.21.15.7:0/129261881 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff603b0 msgr2=0x5620f0394840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.075+0000 7fcf86400640 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff603b0 0x5620f0394840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.075+0000 7fcf86400640 1 -- 172.21.15.7:0/129261881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 msgr2=0x5620f03807f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.075+0000 7fcf86400640 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 0x5620f03807f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.075+0000 7fcf86400640 1 -- 172.21.15.7:0/129261881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f038aa50 con 0x5620eff89270 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.075+0000 7fcf85000640 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 0x5620f03807f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.075+0000 7fcf86400640 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f036a9d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009f10 tx=0x7fcf78031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f3d8c0406791ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.075+0000 7fcf66a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.075+0000 7fcf66a00640 1 -- 172.21.15.7:0/129261881 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78040020 con 0x5620eff89270 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.075+0000 7fcf66a00640 1 -- 172.21.15.7:0/129261881 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fcf78031c60 con 0x5620eff89270 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.075+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/129261881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf6c01f890 con 0x5620eff89270 2024-04-25T11:15:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf66a00640 1 -- 172.21.15.7:0/129261881 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcf7804a050 con 0x5620eff89270 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf8f3f1740 10 client.6675 fetch_fsmap learned FSMap version 43 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/129261881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) -- 0x5620f0370200 con 0x5620eff89270 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf66a00640 1 -- 172.21.15.7:0/129261881 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf6c01f890 con 0x5620eff89270 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf66a00640 1 -- 172.21.15.7:0/129261881 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 43) ==== 3494+0+0 (secure 0 0 0) 0x7fcf7804d020 con 0x5620eff89270 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf8f3f1740 10 client.6675 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf8f3f1740 10 client.6675 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf8f3f1740 20 client.6675 populate_metadata read hostname 'smithi007' 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.7:6835/2955530443,v1:172.21.15.7:6837/2955530443] conn(0x5620f034f200 0x5620f034fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf8f3f1740 4 client.6675 mds_command: new command op to 6528 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/129261881 --> [v2:172.21.15.7:6835/2955530443,v1:172.21.15.7:6837/2955530443] -- command(tid 0: {"prefix": "status"}) -- 0x5620f0297080 con 0x5620f034f200 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.076+0000 7fcf85a00640 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.7:6835/2955530443,v1:172.21.15.7:6837/2955530443] conn(0x5620f034f200 0x5620f034fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf85a00640 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.7:6835/2955530443,v1:172.21.15.7:6837/2955530443] conn(0x5620f034f200 0x5620f034fef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a74729b1c4fc4c6 server_cookie=6d338c50f1b8099 in_seq=0 out_seq=0 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf66a00640 10 client.6675 ms_handle_connect on v2:172.21.15.7:6835/2955530443 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf66a00640 1 -- 172.21.15.7:0/129261881 <== mds.0 v2:172.21.15.7:6835/2955530443 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5620f0297080 con 0x5620f034f200 2024-04-25T11:15:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf66a00640 10 client.6675 handle_command_reply: tid=0 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf8f3f1740 1 client.6675 shutdown 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/129261881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 msgr2=0x5620f036a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f036a9d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009f10 tx=0x7fcf78031af0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/129261881 shutdown_connections 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.7:6835/2955530443,v1:172.21.15.7:6837/2955530443] conn(0x5620f034f200 0x5620f034fef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620eff89270 0x5620f036a9d0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5620eff68590 0x5620f03807f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf8f3f1740 1 --2- 172.21.15.7:0/129261881 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5620eff603b0 0x5620f0394840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf66a00640 0 client.6675 ms_handle_reset on v2:172.21.15.7:6835/2955530443 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.077+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/129261881 >> 172.21.15.7:0/129261881 conn(0x5620eff7d3c0 msgr2=0x5620f0329320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/129261881 shutdown_connections 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf8f3f1740 1 -- 172.21.15.7:0/129261881 wait complete. 2024-04-25T11:15:13.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf8f3f1740 20 client.6675 trim_cache size 0 max 16384 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 6528, 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.3726440929999999, 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 43, 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 80, 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T11:15:13.083 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.544391471999999 2024-04-25T11:15:13.084 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:15:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf86e00640 1 -- 172.21.15.7:0/769621311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf600744b0 msgr2=0x7fcf600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf86e00640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf600744b0 0x7fcf600769c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c006fd0 tx=0x7fcf6c006bc0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf86e00640 1 -- 172.21.15.7:0/769621311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf8006f7b0 msgr2=0x7fcf80166b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf86e00640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf8006f7b0 0x7fcf80166b10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400e8a0 tx=0x7fcf7400ed60 comp rx=0 tx=0).stop 2024-04-25T11:15:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf86e00640 1 -- 172.21.15.7:0/769621311 shutdown_connections 2024-04-25T11:15:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf86e00640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf600744b0 0x7fcf600769c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf86e00640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf8014d980 0x7fcf8015a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf86e00640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf800cf320 0x7fcf800aac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf86e00640 1 --2- 172.21.15.7:0/769621311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf8006f7b0 0x7fcf80166b10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.078+0000 7fcf86e00640 1 -- 172.21.15.7:0/769621311 >> 172.21.15.7:0/769621311 conn(0x7fcf800d1540 msgr2=0x7fcf800b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.079+0000 7fcf86e00640 1 -- 172.21.15.7:0/769621311 shutdown_connections 2024-04-25T11:15:13.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.079+0000 7fcf86e00640 1 -- 172.21.15.7:0/769621311 wait complete. 2024-04-25T11:15:13.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:15:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.382+0000 7f164b200640 1 Processor -- start 2024-04-25T11:15:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.382+0000 7f164b200640 1 -- start start 2024-04-25T11:15:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.384+0000 7f164b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f164c0b2a90 0x7f164c06d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.384+0000 7f164b200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f164c1652b0 con 0x7f164c06dda0 2024-04-25T11:15:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.384+0000 7f164b200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f164c0d0f50 con 0x7f164c0b9050 2024-04-25T11:15:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.384+0000 7f164b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f164c0d0d00 con 0x7f164c0b2a90 2024-04-25T11:15:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.385+0000 7f164a800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f164c06dda0 0x7f164c164ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39210/0 (socket says 172.21.15.7:39210) 2024-04-25T11:15:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.385+0000 7f164a800640 1 -- 172.21.15.7:0/2385581335 learned_addr learned my addr 172.21.15.7:0/2385581335 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.385+0000 7f1649e00640 1 --2- 172.21.15.7:0/2385581335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f164c0b2a90 0x7f164c06d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.385+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089344407 0 0) 0x7f164c1652b0 con 0x7f164c06dda0 2024-04-25T11:15:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.385+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1630003520 con 0x7f164c06dda0 2024-04-25T11:15:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2513749763 0 0) 0x7f1630003520 con 0x7f164c06dda0 2024-04-25T11:15:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f164c0b2a90 msgr2=0x7f164c06d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 --2- 172.21.15.7:0/2385581335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f164c0b2a90 0x7f164c06d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 >> v1:172.21.15.87:6789/0 conn(0x7f164c0b9050 legacy=0x7f164c164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164c09cbc0 con 0x7f164c06dda0 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1644001670 con 0x7f164c06dda0 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 >> v1:172.21.15.7:6789/0 conn(0x7f164c06dda0 legacy=0x7f164c164ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 --2- 172.21.15.7:0/2385581335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16300039e0 0x7f1630003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 --2- 172.21.15.7:0/2385581335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1630004800 0x7f1630004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 --2- 172.21.15.7:0/2385581335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f164c0b2a90 0x7f16300052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f164c0d0d00 con 0x7f16300039e0 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f163000bb50 con 0x7f1630004800 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f163000bd50 con 0x7f164c0b2a90 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.386+0000 7f1649e00640 1 --2- 172.21.15.7:0/2385581335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16300039e0 0x7f1630003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.387+0000 7f164a800640 1 --2- 172.21.15.7:0/2385581335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f164c0b2a90 0x7f16300052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.387+0000 7f1649e00640 1 -- 172.21.15.7:0/2385581335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f164c0b2a90 msgr2=0x7f16300052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.387+0000 7f1649e00640 1 --2- 172.21.15.7:0/2385581335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f164c0b2a90 0x7f16300052d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.387+0000 7f1649e00640 1 -- 172.21.15.7:0/2385581335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1630004800 msgr2=0x7f1630004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.387+0000 7f1649e00640 1 --2- 172.21.15.7:0/2385581335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1630004800 0x7f1630004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.387+0000 7f1649e00640 1 -- 172.21.15.7:0/2385581335 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1630003520 con 0x7f16300039e0 2024-04-25T11:15:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.388+0000 7f1649e00640 1 --2- 172.21.15.7:0/2385581335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16300039e0 0x7f1630003e10 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1634009a90 tx=0x7f1630005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9fcdb5aa7763c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.388+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634036070 con 0x7f16300039e0 2024-04-25T11:15:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.388+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1634031400 con 0x7f16300039e0 2024-04-25T11:15:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.388+0000 7f1648a00640 1 -- 172.21.15.7:0/2385581335 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634009690 con 0x7f16300039e0 2024-04-25T11:15:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.389+0000 7f164b200640 1 -- 172.21.15.7:0/2385581335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16300039e0 msgr2=0x7f1630003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.389+0000 7f164b200640 1 --2- 172.21.15.7:0/2385581335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16300039e0 0x7f1630003e10 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f1634009a90 tx=0x7f1630005830 comp rx=0 tx=0).stop 2024-04-25T11:15:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.389+0000 7f164a800640 1 -- 172.21.15.7:0/2385581335 reap_dead start 2024-04-25T11:15:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.389+0000 7f164b200640 1 -- 172.21.15.7:0/2385581335 shutdown_connections 2024-04-25T11:15:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.389+0000 7f164b200640 1 -- 172.21.15.7:0/2385581335 >> 172.21.15.7:0/2385581335 conn(0x7f164c0d01e0 msgr2=0x7f164c0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.389+0000 7f164b200640 1 -- 172.21.15.7:0/2385581335 shutdown_connections 2024-04-25T11:15:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.389+0000 7f164b200640 1 -- 172.21.15.7:0/2385581335 wait complete. 2024-04-25T11:15:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.391+0000 7f164b200640 1 Processor -- start 2024-04-25T11:15:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.391+0000 7f164b200640 1 -- start start 2024-04-25T11:15:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.393+0000 7f164b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16300039e0 0x7f164c162ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.393+0000 7f164b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c163680 0x7f164c0b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.393+0000 7f164b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164c0b8430 0x7f164c0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.393+0000 7f164b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f164c09c970 con 0x7f164c163680 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.393+0000 7f164b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f164c0b43a0 con 0x7f164c0b8430 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.393+0000 7f164b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f164c06d860 con 0x7f16300039e0 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.393+0000 7f1649400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c163680 0x7f164c0b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.393+0000 7f1649400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c163680 0x7f164c0b7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40766/0 (socket says 172.21.15.7:40766) 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.393+0000 7f1649400640 1 -- 172.21.15.7:0/4269877248 learned_addr learned my addr 172.21.15.7:0/4269877248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.393+0000 7f1649e00640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16300039e0 0x7f164c162ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f164a800640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164c0b8430 0x7f164c0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f1649400640 1 -- 172.21.15.7:0/4269877248 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16300039e0 msgr2=0x7f164c162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f1649400640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16300039e0 0x7f164c162ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f1649400640 1 -- 172.21.15.7:0/4269877248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164c0b8430 msgr2=0x7f164c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f1649400640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164c0b8430 0x7f164c0cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f1649400640 1 -- 172.21.15.7:0/4269877248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164c0b8840 con 0x7f164c163680 2024-04-25T11:15:13.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f164a800640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164c0b8430 0x7f164c0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:13.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f1649400640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c163680 0x7f164c0b7ef0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f163c00cb20 tx=0x7f163c0020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:13.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f1642000640 1 -- 172.21.15.7:0/4269877248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c017070 con 0x7f164c163680 2024-04-25T11:15:13.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f1642000640 1 -- 172.21.15.7:0/4269877248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163c003da0 con 0x7f164c163680 2024-04-25T11:15:13.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f164b200640 1 -- 172.21.15.7:0/4269877248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164c0c71c0 con 0x7f164c163680 2024-04-25T11:15:13.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f1642000640 1 -- 172.21.15.7:0/4269877248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c00a900 con 0x7f164c163680 2024-04-25T11:15:13.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.394+0000 7f164b200640 1 -- 172.21.15.7:0/4269877248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f164c0b20b0 con 0x7f164c163680 2024-04-25T11:15:13.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.396+0000 7f164b200640 1 -- 172.21.15.7:0/4269877248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f164c050740 con 0x7f164c163680 2024-04-25T11:15:13.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.397+0000 7f1642000640 1 -- 172.21.15.7:0/4269877248 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f163c01a070 con 0x7f164c163680 2024-04-25T11:15:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.398+0000 7f1642000640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16280744b0 0x7f16280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.398+0000 7f1642000640 1 -- 172.21.15.7:0/4269877248 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f163c094bd0 con 0x7f164c163680 2024-04-25T11:15:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.398+0000 7f1649e00640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16280744b0 0x7f16280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.399+0000 7f1649e00640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16280744b0 0x7f16280769c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1634009a90 tx=0x7f1634008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.403+0000 7f1642000640 1 -- 172.21.15.7:0/4269877248 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f164c0b8840 con 0x7f164c163680 2024-04-25T11:15:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.647+0000 7f164b200640 1 -- 172.21.15.7:0/4269877248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f164c0cfca0 con 0x7f164c163680 2024-04-25T11:15:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.647+0000 7f1642000640 1 -- 172.21.15.7:0/4269877248 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7f163c0671b0 con 0x7f164c163680 2024-04-25T11:15:13.648 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:15:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.650+0000 7f164b200640 1 -- 172.21.15.7:0/4269877248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16280744b0 msgr2=0x7f16280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.650+0000 7f164b200640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16280744b0 0x7f16280769c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1634009a90 tx=0x7f1634008030 comp rx=0 tx=0).stop 2024-04-25T11:15:13.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.650+0000 7f164b200640 1 -- 172.21.15.7:0/4269877248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c163680 msgr2=0x7f164c0b7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:13.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.650+0000 7f164b200640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c163680 0x7f164c0b7ef0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f163c00cb20 tx=0x7f163c0020a0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.650+0000 7f164b200640 1 -- 172.21.15.7:0/4269877248 shutdown_connections 2024-04-25T11:15:13.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.650+0000 7f164b200640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16280744b0 0x7f16280769c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.650+0000 7f164b200640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f164c0b8430 0x7f164c0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.650+0000 7f164b200640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f164c163680 0x7f164c0b7ef0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.650+0000 7f164b200640 1 --2- 172.21.15.7:0/4269877248 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16300039e0 0x7f164c162ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:13.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.650+0000 7f164b200640 1 -- 172.21.15.7:0/4269877248 >> 172.21.15.7:0/4269877248 conn(0x7f164c0d01e0 msgr2=0x7f164c0d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:13.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.651+0000 7f164b200640 1 -- 172.21.15.7:0/4269877248 shutdown_connections 2024-04-25T11:15:13.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:13.651+0000 7f164b200640 1 -- 172.21.15.7:0/4269877248 wait complete. 2024-04-25T11:15:13.678 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:15:13.678 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:15:13.682 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:15:13.682 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:15:13.682 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:15:13.682 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:15:13.682 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:15:13.682 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:15:13.682 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:15:13.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:13.682 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:15:13.740 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:15:13.740 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:15:13.740 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:15:13.740 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:13.740 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:15:13.741 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42261sec preferred_lft 42261sec 2024-04-25T11:15:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:15:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:13.742 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:15:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:15:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:15:13.742 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:15:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:15:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:15:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6098:c7ff:fe8a:4a30/64 scope link 2024-04-25T11:15:13.742 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:13.743 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:15:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:15:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::646a:35ff:fe7b:ad0/64 scope link 2024-04-25T11:15:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:13.743 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:15:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:15:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4465:c4ff:fe62:f75f/64 scope link 2024-04-25T11:15:13.743 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:13.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:13.744 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:15:13.823 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:15:13.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:13.827 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:15:13.885 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:15:13.885 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:15:13.885 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:15:13.885 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:15:13.885 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:15:13.886 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:15:13.886 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:15:13.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:15:13.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:15:13.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:15:13.887 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:15:14.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:15:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:15:14.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:14.116 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:15:14.164 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:15:14.164 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:15:14.165 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:15:14.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:14.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:15:14.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:15:14.245 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:15:14.300 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:15:14.300 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:15:14.300 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:15:14.355 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:15:14.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:14.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:15:14.564 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:15:14.564 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:15:14.609 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:15:14.609 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:15:14.609 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:15:14.609 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:15:14.609 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:15:14.611 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:15:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:15:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:15:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:15:14.611 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:15:14.616 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:15:14.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:14.616 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:15:14.691 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:15:14.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:15:14.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:15:14.833 INFO:teuthology.orchestra.run.smithi007.stdout:client5429 2024-04-25T11:15:14.833 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:15:14.833 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:14.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:15:15.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.163+0000 7fa86a000640 1 Processor -- start 2024-04-25T11:15:15.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.163+0000 7fa86a000640 1 -- start start 2024-04-25T11:15:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.166+0000 7fa86a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa86409cb20 0x7fa8640b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.166+0000 7fa86a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa864163850 0x7fa8640ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.166+0000 7fa86a000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8640d1d30 con 0x7fa8640ce550 2024-04-25T11:15:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.166+0000 7fa86a000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa8640d2620 con 0x7fa86409cb20 2024-04-25T11:15:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.166+0000 7fa86a000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa86406ebb0 con 0x7fa864163850 2024-04-25T11:15:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.166+0000 7fa863400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa86409cb20 0x7fa8640b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.166+0000 7fa863400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa86409cb20 0x7fa8640b91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40778/0 (socket says 172.21.15.7:40778) 2024-04-25T11:15:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.166+0000 7fa863400640 1 -- 172.21.15.7:0/4177582220 learned_addr learned my addr 172.21.15.7:0/4177582220 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.167+0000 7fa862a00640 1 --2- 172.21.15.7:0/4177582220 >> v2:172.21.15.87:3300/0 conn(0x7fa864163850 0x7fa8640ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.167+0000 7fa863400640 1 -- 172.21.15.7:0/4177582220 >> v2:172.21.15.87:3300/0 conn(0x7fa864163850 msgr2=0x7fa8640ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.167+0000 7fa863400640 1 --2- 172.21.15.7:0/4177582220 >> v2:172.21.15.87:3300/0 conn(0x7fa864163850 0x7fa8640ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.167+0000 7fa863400640 1 -- 172.21.15.7:0/4177582220 >> v1:172.21.15.87:6789/0 conn(0x7fa8640ce550 legacy=0x7fa8640d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:15.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.167+0000 7fa863400640 1 -- 172.21.15.7:0/4177582220 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86406e5f0 con 0x7fa86409cb20 2024-04-25T11:15:15.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.168+0000 7fa863400640 1 --2- 172.21.15.7:0/4177582220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa86409cb20 0x7fa8640b91a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa8640d1b20 tx=0x7fa85402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561c203dc8a26ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:15.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.168+0000 7fa862000640 1 -- 172.21.15.7:0/4177582220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa854037070 con 0x7fa86409cb20 2024-04-25T11:15:15.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.168+0000 7fa862000640 1 -- 172.21.15.7:0/4177582220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa854032440 con 0x7fa86409cb20 2024-04-25T11:15:15.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.168+0000 7fa862000640 1 -- 172.21.15.7:0/4177582220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa854032810 con 0x7fa86409cb20 2024-04-25T11:15:15.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.169+0000 7fa86a000640 1 -- 172.21.15.7:0/4177582220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa86409cb20 msgr2=0x7fa8640b91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.169+0000 7fa86a000640 1 --2- 172.21.15.7:0/4177582220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa86409cb20 0x7fa8640b91a0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa8640d1b20 tx=0x7fa85402a230 comp rx=0 tx=0).stop 2024-04-25T11:15:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.169+0000 7fa86a000640 1 -- 172.21.15.7:0/4177582220 shutdown_connections 2024-04-25T11:15:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.169+0000 7fa86a000640 1 --2- 172.21.15.7:0/4177582220 >> v2:172.21.15.87:3300/0 conn(0x7fa864163850 0x7fa8640ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.169+0000 7fa86a000640 1 --2- 172.21.15.7:0/4177582220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa86409cb20 0x7fa8640b91a0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.169+0000 7fa86a000640 1 -- 172.21.15.7:0/4177582220 >> 172.21.15.7:0/4177582220 conn(0x7fa8640b54b0 msgr2=0x7fa8640b58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.169+0000 7fa86a000640 1 -- 172.21.15.7:0/4177582220 shutdown_connections 2024-04-25T11:15:15.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.169+0000 7fa86a000640 1 -- 172.21.15.7:0/4177582220 wait complete. 2024-04-25T11:15:15.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.172+0000 7fa86a000640 1 Processor -- start 2024-04-25T11:15:15.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.172+0000 7fa86a000640 1 -- start start 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa86a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86409cb20 0x7fa864155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa86a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8640ce550 0x7fa8640b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa86a000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa864163850 0x7fa8640a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa86a000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8640d22b0 con 0x7fa8640ce550 2024-04-25T11:15:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa86a000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa8640d1fd0 con 0x7fa86409cb20 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa86a000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa864163c60 con 0x7fa864163850 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa862a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8640ce550 0x7fa8640b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa862a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8640ce550 0x7fa8640b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40784/0 (socket says 172.21.15.7:40784) 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa862a00640 1 -- 172.21.15.7:0/584283741 learned_addr learned my addr 172.21.15.7:0/584283741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa863400640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86409cb20 0x7fa864155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa862a00640 1 -- 172.21.15.7:0/584283741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa864163850 msgr2=0x7fa8640a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa862a00640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa864163850 0x7fa8640a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa862a00640 1 -- 172.21.15.7:0/584283741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86409cb20 msgr2=0x7fa864155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa862a00640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86409cb20 0x7fa864155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.174+0000 7fa862a00640 1 -- 172.21.15.7:0/584283741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8640b1dd0 con 0x7fa8640ce550 2024-04-25T11:15:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.175+0000 7fa862a00640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8640ce550 0x7fa8640b4570 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa85000d620 tx=0x7fa85000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.175+0000 7fa85b400640 1 -- 172.21.15.7:0/584283741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85000b620 con 0x7fa8640ce550 2024-04-25T11:15:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.175+0000 7fa85b400640 1 -- 172.21.15.7:0/584283741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa850019070 con 0x7fa8640ce550 2024-04-25T11:15:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.175+0000 7fa85b400640 1 -- 172.21.15.7:0/584283741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85000ba00 con 0x7fa8640ce550 2024-04-25T11:15:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.175+0000 7fa86a000640 1 -- 172.21.15.7:0/584283741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8641f9650 con 0x7fa8640ce550 2024-04-25T11:15:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.175+0000 7fa86a000640 1 -- 172.21.15.7:0/584283741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8641f7810 con 0x7fa8640ce550 2024-04-25T11:15:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.176+0000 7fa86a000640 1 -- 172.21.15.7:0/584283741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa864050740 con 0x7fa8640ce550 2024-04-25T11:15:15.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.178+0000 7fa85b400640 1 -- 172.21.15.7:0/584283741 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa85000bba0 con 0x7fa8640ce550 2024-04-25T11:15:15.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.179+0000 7fa85b400640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8440744b0 0x7fa8440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:15.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.180+0000 7fa85b400640 1 -- 172.21.15.7:0/584283741 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fa8500958a0 con 0x7fa8640ce550 2024-04-25T11:15:15.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.180+0000 7fa863400640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8440744b0 0x7fa8440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.180+0000 7fa863400640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8440744b0 0x7fa8440769c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa854034000 tx=0x7fa85402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.184+0000 7fa85b400640 1 -- 172.21.15.7:0/584283741 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa85005fe00 con 0x7fa8640ce550 2024-04-25T11:15:15.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.428+0000 7fa86a000640 1 -- 172.21.15.7:0/584283741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa864163230 con 0x7fa8640ce550 2024-04-25T11:15:15.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.429+0000 7fa85b400640 1 -- 172.21.15.7:0/584283741 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) ==== 53+0+73 (secure 0 0 0) 0x7fa85005f700 con 0x7fa8640ce550 2024-04-25T11:15:15.429 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:15:15.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 -- 172.21.15.7:0/584283741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8440744b0 msgr2=0x7fa8440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:15.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8440744b0 0x7fa8440769c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa854034000 tx=0x7fa85402e000 comp rx=0 tx=0).stop 2024-04-25T11:15:15.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 -- 172.21.15.7:0/584283741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8640ce550 msgr2=0x7fa8640b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:15.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8640ce550 0x7fa8640b4570 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa85000d620 tx=0x7fa85000dae0 comp rx=0 tx=0).stop 2024-04-25T11:15:15.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 -- 172.21.15.7:0/584283741 shutdown_connections 2024-04-25T11:15:15.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8440744b0 0x7fa8440769c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:15.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa864163850 0x7fa8640a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:15.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8640ce550 0x7fa8640b4570 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:15.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 --2- 172.21.15.7:0/584283741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86409cb20 0x7fa864155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:15.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 -- 172.21.15.7:0/584283741 >> 172.21.15.7:0/584283741 conn(0x7fa8640b54b0 msgr2=0x7fa864071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:15.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 -- 172.21.15.7:0/584283741 shutdown_connections 2024-04-25T11:15:15.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:15.432+0000 7fa86a000640 1 -- 172.21.15.7:0/584283741 wait complete. 2024-04-25T11:15:15.463 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:15:15.463 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:15:15.463 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:15:15.463 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:15:15.463 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:15:15.463 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:15:15.463 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:15:15.463 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:15:15.464 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:15:15.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:15.464 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:15:15.523 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:15:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:15:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:15:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:15.523 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42260sec preferred_lft 42260sec 2024-04-25T11:15:15.524 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::58f2:6bff:fe8b:8b4c/64 scope link 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::98fe:1bff:fe9f:d71c/64 scope link 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:15.525 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:15:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:15:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::3c38:a8ff:fec7:ea00/64 scope link 2024-04-25T11:15:15.526 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:15:15.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:15.527 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:15:15.606 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:15:15.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:15.611 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:15:15.666 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:15:15.666 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:15:15.666 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:15:15.667 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:15:15.667 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:15:15.667 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:15:15.667 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:15:15.667 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:15:15.667 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:15:15.667 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:15:15.667 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:15:15.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:15:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:15:15.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:15:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:15:15.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:15.876 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:15:15.927 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:15:15.927 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:15:15.927 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:15:15.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:15.927 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T11:15:16.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:15:16.007 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:15:16.059 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:15:16.060 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:15:16.060 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:15:16.111 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:15:16.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:16.111 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:15:16.315 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:15:16.315 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:15:16.364 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:15:16.364 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:15:16.364 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:15:16.365 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 1@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:15:16.365 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:15:16.366 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:15:16.366 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:15:16.366 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:15:16.366 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:15:16.366 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:15:16.371 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:15:16.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:16.371 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:15:16.451 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:15:16.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:15:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:15:16.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:15:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:15:16.569 INFO:teuthology.orchestra.run.smithi087.stdout:client6690 2024-04-25T11:15:16.570 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:15:16.570 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:16.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-04-25T11:15:16.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.920+0000 7f5ae0a00640 1 Processor -- start 2024-04-25T11:15:16.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.920+0000 7f5ae0a00640 1 -- start start 2024-04-25T11:15:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.923+0000 7f5ae0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5adc145e40 0x7f5adc146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.923+0000 7f5ae0a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5adc144a80 0x7f5adc0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.923+0000 7f5ae0a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5adc0b66b0 con 0x7f5adc0b4870 2024-04-25T11:15:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.923+0000 7f5ae0a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5adc145b70 con 0x7f5adc144a80 2024-04-25T11:15:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.923+0000 7f5ae0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5adc0b58d0 con 0x7f5adc145e40 2024-04-25T11:15:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.923+0000 7f5adbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5adc144a80 0x7f5adc0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.923+0000 7f5adbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5adc144a80 0x7f5adc0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40792/0 (socket says 172.21.15.7:40792) 2024-04-25T11:15:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.923+0000 7f5adaa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5adc0b4870 0x7f5adc0b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39220/0 (socket says 172.21.15.7:39220) 2024-04-25T11:15:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.923+0000 7f5adbe00640 1 -- 172.21.15.7:0/4014675726 learned_addr learned my addr 172.21.15.7:0/4014675726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.923+0000 7f5adb400640 1 --2- 172.21.15.7:0/4014675726 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5adc145e40 0x7f5adc146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.924+0000 7f5adbe00640 1 -- 172.21.15.7:0/4014675726 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5adc145e40 msgr2=0x7f5adc146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.924+0000 7f5adbe00640 1 --2- 172.21.15.7:0/4014675726 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5adc145e40 0x7f5adc146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.924+0000 7f5adbe00640 1 -- 172.21.15.7:0/4014675726 >> v1:172.21.15.7:6789/0 conn(0x7f5adc0b4870 legacy=0x7f5adc0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.924+0000 7f5adbe00640 1 -- 172.21.15.7:0/4014675726 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc0b5b30 con 0x7f5adc144a80 2024-04-25T11:15:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.925+0000 7f5adbe00640 1 --2- 172.21.15.7:0/4014675726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5adc144a80 0x7f5adc0b5390 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009870 tx=0x7f5ad402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3568e3ba1e141b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.925+0000 7f5ada000640 1 -- 172.21.15.7:0/4014675726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4037070 con 0x7f5adc144a80 2024-04-25T11:15:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.925+0000 7f5ada000640 1 -- 172.21.15.7:0/4014675726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad4032440 con 0x7f5adc144a80 2024-04-25T11:15:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.925+0000 7f5ada000640 1 -- 172.21.15.7:0/4014675726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4032810 con 0x7f5adc144a80 2024-04-25T11:15:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.926+0000 7f5ae0a00640 1 -- 172.21.15.7:0/4014675726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5adc144a80 msgr2=0x7f5adc0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.926+0000 7f5ae0a00640 1 --2- 172.21.15.7:0/4014675726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5adc144a80 0x7f5adc0b5390 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009870 tx=0x7f5ad402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:15:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.926+0000 7f5ae0a00640 1 -- 172.21.15.7:0/4014675726 shutdown_connections 2024-04-25T11:15:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.926+0000 7f5ae0a00640 1 --2- 172.21.15.7:0/4014675726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5adc144a80 0x7f5adc0b5390 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.926+0000 7f5ae0a00640 1 --2- 172.21.15.7:0/4014675726 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5adc145e40 0x7f5adc146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.926+0000 7f5ae0a00640 1 -- 172.21.15.7:0/4014675726 >> 172.21.15.7:0/4014675726 conn(0x7f5adc0740f0 msgr2=0x7f5adc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.926+0000 7f5ae0a00640 1 -- 172.21.15.7:0/4014675726 shutdown_connections 2024-04-25T11:15:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.926+0000 7f5ae0a00640 1 -- 172.21.15.7:0/4014675726 wait complete. 2024-04-25T11:15:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.929+0000 7f5ae0a00640 1 Processor -- start 2024-04-25T11:15:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.929+0000 7f5ae0a00640 1 -- start start 2024-04-25T11:15:16.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.930+0000 7f5ae0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5adc0b4870 0x7f5adc1e6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:16.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.930+0000 7f5ae0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5adc144a80 0x7f5adc1e8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:16.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.930+0000 7f5ae0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5adc145e40 0x7f5adc1f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:16.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.930+0000 7f5ae0a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc0b63d0 con 0x7f5adc0b4870 2024-04-25T11:15:16.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.930+0000 7f5ae0a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc0b60f0 con 0x7f5adc144a80 2024-04-25T11:15:16.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.930+0000 7f5ae0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5adc06f730 con 0x7f5adc145e40 2024-04-25T11:15:16.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.931+0000 7f5adb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5adc0b4870 0x7f5adc1e6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:16.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.931+0000 7f5adb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5adc0b4870 0x7f5adc1e6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40804/0 (socket says 172.21.15.7:40804) 2024-04-25T11:15:16.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.931+0000 7f5adb400640 1 -- 172.21.15.7:0/2546900508 learned_addr learned my addr 172.21.15.7:0/2546900508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:16.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.931+0000 7f5adaa00640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5adc144a80 0x7f5adc1e8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.931+0000 7f5adb400640 1 -- 172.21.15.7:0/2546900508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5adc145e40 msgr2=0x7f5adc1f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.931+0000 7f5adb400640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5adc145e40 0x7f5adc1f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.931+0000 7f5adb400640 1 -- 172.21.15.7:0/2546900508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5adc144a80 msgr2=0x7f5adc1e8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.931+0000 7f5adb400640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5adc144a80 0x7f5adc1e8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.931+0000 7f5adb400640 1 -- 172.21.15.7:0/2546900508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc1f7ee0 con 0x7f5adc0b4870 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.931+0000 7f5adaa00640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5adc144a80 0x7f5adc1e8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.932+0000 7f5adb400640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5adc0b4870 0x7f5adc1e6700 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5acc007e70 tx=0x7f5acc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.932+0000 7f5ad3400640 1 -- 172.21.15.7:0/2546900508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc014620 con 0x7f5adc0b4870 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.932+0000 7f5ad3400640 1 -- 172.21.15.7:0/2546900508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5acc019070 con 0x7f5adc0b4870 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.932+0000 7f5ad3400640 1 -- 172.21.15.7:0/2546900508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc014a00 con 0x7f5adc0b4870 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.932+0000 7f5ae0a00640 1 -- 172.21.15.7:0/2546900508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5adc1f81c0 con 0x7f5adc0b4870 2024-04-25T11:15:16.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.932+0000 7f5ae0a00640 1 -- 172.21.15.7:0/2546900508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5adc1f8780 con 0x7f5adc0b4870 2024-04-25T11:15:16.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.933+0000 7f5ae0a00640 1 -- 172.21.15.7:0/2546900508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5adc050740 con 0x7f5adc0b4870 2024-04-25T11:15:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.935+0000 7f5ad3400640 1 -- 172.21.15.7:0/2546900508 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5acc014ba0 con 0x7f5adc0b4870 2024-04-25T11:15:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.936+0000 7f5ad3400640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5abc0744b0 0x7f5abc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.936+0000 7f5ad3400640 1 -- 172.21.15.7:0/2546900508 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f5acc095da0 con 0x7f5adc0b4870 2024-04-25T11:15:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.936+0000 7f5adaa00640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5abc0744b0 0x7f5abc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.937+0000 7f5adaa00640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5abc0744b0 0x7f5abc0769c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0708f0 tx=0x7f5ac8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:16.942+0000 7f5ad3400640 1 -- 172.21.15.7:0/2546900508 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5acc060300 con 0x7f5adc0b4870 2024-04-25T11:15:17.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.146+0000 7f5ae0a00640 1 -- 172.21.15.7:0/2546900508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f5adc06fb80 con 0x7f5adc0b4870 2024-04-25T11:15:17.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.147+0000 7f5ad3400640 1 -- 172.21.15.7:0/2546900508 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v20) ==== 106+0+4 (secure 0 0 0) 0x7f5acc05fc00 con 0x7f5adc0b4870 2024-04-25T11:15:17.147 INFO:teuthology.orchestra.run.smithi007.stdout:100 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.149+0000 7f5ae0a00640 1 -- 172.21.15.7:0/2546900508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5abc0744b0 msgr2=0x7f5abc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.149+0000 7f5ae0a00640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5abc0744b0 0x7f5abc0769c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0708f0 tx=0x7f5ac8009040 comp rx=0 tx=0).stop 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.149+0000 7f5ae0a00640 1 -- 172.21.15.7:0/2546900508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5adc0b4870 msgr2=0x7f5adc1e6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.149+0000 7f5ae0a00640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5adc0b4870 0x7f5adc1e6700 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5acc007e70 tx=0x7f5acc00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.149+0000 7f5ae0a00640 1 -- 172.21.15.7:0/2546900508 shutdown_connections 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.149+0000 7f5ae0a00640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5abc0744b0 0x7f5abc0769c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.149+0000 7f5ae0a00640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5adc145e40 0x7f5adc1f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.149+0000 7f5ae0a00640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5adc144a80 0x7f5adc1e8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.149+0000 7f5ae0a00640 1 --2- 172.21.15.7:0/2546900508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5adc0b4870 0x7f5adc1e6700 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.149+0000 7f5ae0a00640 1 -- 172.21.15.7:0/2546900508 >> 172.21.15.7:0/2546900508 conn(0x7f5adc0740f0 msgr2=0x7f5adc0ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.150+0000 7f5ae0a00640 1 -- 172.21.15.7:0/2546900508 shutdown_connections 2024-04-25T11:15:17.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.150+0000 7f5ae0a00640 1 -- 172.21.15.7:0/2546900508 wait complete. 2024-04-25T11:15:17.177 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_max_caps_per_client 200 2024-04-25T11:15:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.444+0000 7f2552400640 1 Processor -- start 2024-04-25T11:15:17.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.444+0000 7f2552400640 1 -- start start 2024-04-25T11:15:17.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.446+0000 7f2552400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f254c0d0480 0x7f254c0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:17.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.446+0000 7f2552400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f254c06eea0 0x7f254c06f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:17.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.446+0000 7f2552400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f254c0ce570 con 0x7f254c1583c0 2024-04-25T11:15:17.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.446+0000 7f2552400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f254c06f7f0 con 0x7f254c06eea0 2024-04-25T11:15:17.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.446+0000 7f2552400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f254c15f190 con 0x7f254c0d0480 2024-04-25T11:15:17.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.446+0000 7f2551000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f254c1583c0 0x7f254c0cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39232/0 (socket says 172.21.15.7:39232) 2024-04-25T11:15:17.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.446+0000 7f2551000640 1 -- 172.21.15.7:0/2857047303 learned_addr learned my addr 172.21.15.7:0/2857047303 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:17.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.446+0000 7f2551a00640 1 --2- 172.21.15.7:0/2857047303 >> v2:172.21.15.7:3300/0 conn(0x7f254c06eea0 0x7f254c06f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.447+0000 7f254b400640 1 -- 172.21.15.7:0/2857047303 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3706148109 0 0) 0x7f254c0ce570 con 0x7f254c1583c0 2024-04-25T11:15:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.447+0000 7f254b400640 1 -- 172.21.15.7:0/2857047303 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2534003520 con 0x7f254c1583c0 2024-04-25T11:15:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.447+0000 7f254be00640 1 --2- 172.21.15.7:0/2857047303 >> v2:172.21.15.87:3300/0 conn(0x7f254c0d0480 0x7f254c0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.447+0000 7f2551a00640 1 -- 172.21.15.7:0/2857047303 >> v2:172.21.15.87:3300/0 conn(0x7f254c0d0480 msgr2=0x7f254c0d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.447+0000 7f2551a00640 1 --2- 172.21.15.7:0/2857047303 >> v2:172.21.15.87:3300/0 conn(0x7f254c0d0480 0x7f254c0d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.447+0000 7f2551a00640 1 -- 172.21.15.7:0/2857047303 >> v1:172.21.15.7:6789/0 conn(0x7f254c1583c0 legacy=0x7f254c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:17.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.447+0000 7f2551a00640 1 -- 172.21.15.7:0/2857047303 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c0ce850 con 0x7f254c06eea0 2024-04-25T11:15:17.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.447+0000 7f254be00640 1 --2- 172.21.15.7:0/2857047303 >> v2:172.21.15.87:3300/0 conn(0x7f254c0d0480 0x7f254c0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:17.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.448+0000 7f2551a00640 1 --2- 172.21.15.7:0/2857047303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f254c06eea0 0x7f254c06f2b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f254c070ad0 tx=0x7f254402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa8d2a7b89949f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:17.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.448+0000 7f254b400640 1 -- 172.21.15.7:0/2857047303 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544007740 con 0x7f254c06eea0 2024-04-25T11:15:17.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.448+0000 7f254b400640 1 -- 172.21.15.7:0/2857047303 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25440078e0 con 0x7f254c06eea0 2024-04-25T11:15:17.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.448+0000 7f254b400640 1 -- 172.21.15.7:0/2857047303 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544007cb0 con 0x7f254c06eea0 2024-04-25T11:15:17.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.449+0000 7f2552400640 1 -- 172.21.15.7:0/2857047303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f254c06eea0 msgr2=0x7f254c06f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:17.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.449+0000 7f2552400640 1 --2- 172.21.15.7:0/2857047303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f254c06eea0 0x7f254c06f2b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f254c070ad0 tx=0x7f254402a5f0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.449+0000 7f2552400640 1 -- 172.21.15.7:0/2857047303 shutdown_connections 2024-04-25T11:15:17.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.449+0000 7f2552400640 1 --2- 172.21.15.7:0/2857047303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f254c06eea0 0x7f254c06f2b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.449+0000 7f2552400640 1 --2- 172.21.15.7:0/2857047303 >> v2:172.21.15.87:3300/0 conn(0x7f254c0d0480 0x7f254c0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.449+0000 7f2552400640 1 -- 172.21.15.7:0/2857047303 >> 172.21.15.7:0/2857047303 conn(0x7f254c0740f0 msgr2=0x7f254c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:17.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.449+0000 7f2552400640 1 -- 172.21.15.7:0/2857047303 shutdown_connections 2024-04-25T11:15:17.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.449+0000 7f2552400640 1 -- 172.21.15.7:0/2857047303 wait complete. 2024-04-25T11:15:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.451+0000 7f2552400640 1 Processor -- start 2024-04-25T11:15:17.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.451+0000 7f2552400640 1 -- start start 2024-04-25T11:15:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2552400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f254c06eea0 0x7f254c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2552400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f254c0d0480 0x7f254c144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2552400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f254c1583c0 0x7f254c15cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:17.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2552400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f254c0cf0f0 con 0x7f254c1583c0 2024-04-25T11:15:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2552400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f254c0cee10 con 0x7f254c06eea0 2024-04-25T11:15:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2552400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f254c0cfd20 con 0x7f254c0d0480 2024-04-25T11:15:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2551a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f254c1583c0 0x7f254c15cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2551a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f254c1583c0 0x7f254c15cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40822/0 (socket says 172.21.15.7:40822) 2024-04-25T11:15:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2551a00640 1 -- 172.21.15.7:0/3908593265 learned_addr learned my addr 172.21.15.7:0/3908593265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2551000640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f254c06eea0 0x7f254c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2551a00640 1 -- 172.21.15.7:0/3908593265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f254c0d0480 msgr2=0x7f254c144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2551a00640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f254c0d0480 0x7f254c144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.453+0000 7f2551a00640 1 -- 172.21.15.7:0/3908593265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f254c06eea0 msgr2=0x7f254c0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.454+0000 7f2551a00640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f254c06eea0 0x7f254c0b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.454+0000 7f2551a00640 1 -- 172.21.15.7:0/3908593265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c0b5690 con 0x7f254c1583c0 2024-04-25T11:15:17.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.454+0000 7f2551a00640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f254c1583c0 0x7f254c15cc90 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f254402abe0 tx=0x7f2544004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:17.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.454+0000 7f2548c00640 1 -- 172.21.15.7:0/3908593265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254403f070 con 0x7f254c1583c0 2024-04-25T11:15:17.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.454+0000 7f2548c00640 1 -- 172.21.15.7:0/3908593265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2544031bc0 con 0x7f254c1583c0 2024-04-25T11:15:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.454+0000 7f2552400640 1 -- 172.21.15.7:0/3908593265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254c0b5970 con 0x7f254c1583c0 2024-04-25T11:15:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.454+0000 7f2548c00640 1 -- 172.21.15.7:0/3908593265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544009450 con 0x7f254c1583c0 2024-04-25T11:15:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.454+0000 7f2552400640 1 -- 172.21.15.7:0/3908593265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254c0b5f30 con 0x7f254c1583c0 2024-04-25T11:15:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.456+0000 7f2552400640 1 -- 172.21.15.7:0/3908593265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254c0b53d0 con 0x7f254c1583c0 2024-04-25T11:15:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.457+0000 7f2548c00640 1 -- 172.21.15.7:0/3908593265 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f254403b410 con 0x7f254c1583c0 2024-04-25T11:15:17.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.458+0000 7f2548c00640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f252c0744b0 0x7f252c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:17.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.458+0000 7f2548c00640 1 -- 172.21.15.7:0/3908593265 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f25440b6cf0 con 0x7f254c1583c0 2024-04-25T11:15:17.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.458+0000 7f2551000640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f252c0744b0 0x7f252c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:17.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.459+0000 7f2551000640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f252c0744b0 0x7f252c0769c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2540000f60 tx=0x7f2540009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:17.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.463+0000 7f2548c00640 1 -- 172.21.15.7:0/3908593265 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2544080c90 con 0x7f254c1583c0 2024-04-25T11:15:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.659+0000 7f2552400640 1 -- 172.21.15.7:0/3908593265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_max_caps_per_client}] v 0) -- 0x7f254c050740 con 0x7f254c1583c0 2024-04-25T11:15:17.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.667+0000 7f2548c00640 1 -- 172.21.15.7:0/3908593265 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_max_caps_per_client}]=0 v21)=0 v21) ==== 123+0+0 (secure 0 0 0) 0x7f2544084f90 con 0x7f254c1583c0 2024-04-25T11:15:17.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.670+0000 7f2552400640 1 -- 172.21.15.7:0/3908593265 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f252c0744b0 msgr2=0x7f252c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:17.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.670+0000 7f2552400640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f252c0744b0 0x7f252c0769c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2540000f60 tx=0x7f2540009040 comp rx=0 tx=0).stop 2024-04-25T11:15:17.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.670+0000 7f2552400640 1 -- 172.21.15.7:0/3908593265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f254c1583c0 msgr2=0x7f254c15cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:17.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.670+0000 7f2552400640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f254c1583c0 0x7f254c15cc90 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f254402abe0 tx=0x7f2544004a40 comp rx=0 tx=0).stop 2024-04-25T11:15:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.671+0000 7f2552400640 1 -- 172.21.15.7:0/3908593265 shutdown_connections 2024-04-25T11:15:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.671+0000 7f2552400640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f252c0744b0 0x7f252c0769c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.671+0000 7f2552400640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f254c1583c0 0x7f254c15cc90 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.671+0000 7f2552400640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f254c0d0480 0x7f254c144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.671+0000 7f2552400640 1 --2- 172.21.15.7:0/3908593265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f254c06eea0 0x7f254c0b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.671+0000 7f2552400640 1 -- 172.21.15.7:0/3908593265 >> 172.21.15.7:0/3908593265 conn(0x7f254c0740f0 msgr2=0x7f254c15f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.671+0000 7f2552400640 1 -- 172.21.15.7:0/3908593265 shutdown_connections 2024-04-25T11:15:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:17.671+0000 7f2552400640 1 -- 172.21.15.7:0/3908593265 wait complete. 2024-04-25T11:15:17.698 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:15:17.698 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:17.744 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> import uuid 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> n = 600 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = False 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> if 0 > 0: 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> create_hard_links = True 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> path = "/home/ubuntu/cephtest/mnt.0/foo/" 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> dpath = os.path.dirname(path) 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> fnameprefix = os.path.basename(path) 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(dpath, exist_ok=True) 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T11:15:17.745 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(n): 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> f.write(f"{i}") 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> if True: 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(f.fileno()) 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> os.unlink(fpath) 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> if create_hard_links: 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> for j in range(0): 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T11:15:17.746 DEBUG:teuthology.orchestra.run.smithi007:> if False: 2024-04-25T11:15:17.747 DEBUG:teuthology.orchestra.run.smithi007:> os.fsync(dirfd) 2024-04-25T11:15:17.747 DEBUG:teuthology.orchestra.run.smithi007:> finally: 2024-04-25T11:15:17.747 DEBUG:teuthology.orchestra.run.smithi007:> os.close(dirfd) 2024-04-25T11:15:17.747 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-04-25T11:15:27.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:15:27.481 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:15:27.481 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:27.587 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:15:27.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:15:27.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:15:27.721 INFO:teuthology.orchestra.run.smithi007.stdout:client5429 2024-04-25T11:15:27.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:15:27.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:27.998+0000 7fb07c600640 1 Processor -- start 2024-04-25T11:15:27.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:27.998+0000 7fb07c600640 1 -- start start 2024-04-25T11:15:28.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.001+0000 7fb07c600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb07415de70 0x7fb074164640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:28.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.001+0000 7fb07c600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0740b6790 0x7fb0740b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:28.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.001+0000 7fb07c600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0740aea00 con 0x7fb0740b0a50 2024-04-25T11:15:28.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.001+0000 7fb07c600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb0741616b0 con 0x7fb07415de70 2024-04-25T11:15:28.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.001+0000 7fb07c600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb0740cdb70 con 0x7fb0740b6790 2024-04-25T11:15:28.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.001+0000 7fb079c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb07415de70 0x7fb074164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:28.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.001+0000 7fb07a600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb0740b0a50 0x7fb0740b4f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50386/0 (socket says 172.21.15.7:50386) 2024-04-25T11:15:28.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.001+0000 7fb07a600640 1 -- 172.21.15.7:0/265859140 learned_addr learned my addr 172.21.15.7:0/265859140 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:28.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.002+0000 7fb079200640 1 --2- 172.21.15.7:0/265859140 >> v2:172.21.15.87:3300/0 conn(0x7fb0740b6790 0x7fb0740b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:28.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.002+0000 7fb079c00640 1 -- 172.21.15.7:0/265859140 >> v2:172.21.15.87:3300/0 conn(0x7fb0740b6790 msgr2=0x7fb0740b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:28.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.002+0000 7fb079c00640 1 --2- 172.21.15.7:0/265859140 >> v2:172.21.15.87:3300/0 conn(0x7fb0740b6790 0x7fb0740b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:28.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.002+0000 7fb079c00640 1 -- 172.21.15.7:0/265859140 >> v1:172.21.15.7:6789/0 conn(0x7fb0740b0a50 legacy=0x7fb0740b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.002+0000 7fb079c00640 1 -- 172.21.15.7:0/265859140 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0740b13e0 con 0x7fb07415de70 2024-04-25T11:15:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.002+0000 7fb079200640 1 --2- 172.21.15.7:0/265859140 >> v2:172.21.15.87:3300/0 conn(0x7fb0740b6790 0x7fb0740b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.003+0000 7fb079c00640 1 --2- 172.21.15.7:0/265859140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07415de70 0x7fb074164640 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb0740cdcf0 tx=0x7fb06802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4acf3aa3ab27be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.003+0000 7fb073e00640 1 -- 172.21.15.7:0/265859140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068037070 con 0x7fb07415de70 2024-04-25T11:15:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.003+0000 7fb073e00640 1 -- 172.21.15.7:0/265859140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb06802ae50 con 0x7fb07415de70 2024-04-25T11:15:28.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.003+0000 7fb073e00640 1 -- 172.21.15.7:0/265859140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068032660 con 0x7fb07415de70 2024-04-25T11:15:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.004+0000 7fb07c600640 1 -- 172.21.15.7:0/265859140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07415de70 msgr2=0x7fb074164640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.004+0000 7fb07c600640 1 --2- 172.21.15.7:0/265859140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07415de70 0x7fb074164640 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb0740cdcf0 tx=0x7fb06802a100 comp rx=0 tx=0).stop 2024-04-25T11:15:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.004+0000 7fb07c600640 1 -- 172.21.15.7:0/265859140 shutdown_connections 2024-04-25T11:15:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.004+0000 7fb07c600640 1 --2- 172.21.15.7:0/265859140 >> v2:172.21.15.87:3300/0 conn(0x7fb0740b6790 0x7fb0740b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:28.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.004+0000 7fb07c600640 1 --2- 172.21.15.7:0/265859140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb07415de70 0x7fb074164640 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:28.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.004+0000 7fb07c600640 1 -- 172.21.15.7:0/265859140 >> 172.21.15.7:0/265859140 conn(0x7fb074074200 msgr2=0x7fb07406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:28.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.004+0000 7fb07c600640 1 -- 172.21.15.7:0/265859140 shutdown_connections 2024-04-25T11:15:28.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.004+0000 7fb07c600640 1 -- 172.21.15.7:0/265859140 wait complete. 2024-04-25T11:15:28.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.007+0000 7fb07c600640 1 Processor -- start 2024-04-25T11:15:28.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.007+0000 7fb07c600640 1 -- start start 2024-04-25T11:15:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.008+0000 7fb07c600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0740b0a50 0x7fb07414a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.008+0000 7fb07c600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0740b6790 0x7fb0740ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.008+0000 7fb07c600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb07415de70 0x7fb0740ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.008+0000 7fb07c600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb0740ce0a0 con 0x7fb0740b6790 2024-04-25T11:15:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.008+0000 7fb07c600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb0740aed30 con 0x7fb07415de70 2024-04-25T11:15:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.008+0000 7fb07c600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb0740b0e60 con 0x7fb0740b0a50 2024-04-25T11:15:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.009+0000 7fb079200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0740b6790 0x7fb0740ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.009+0000 7fb079200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0740b6790 0x7fb0740ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50096/0 (socket says 172.21.15.7:50096) 2024-04-25T11:15:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.009+0000 7fb079200640 1 -- 172.21.15.7:0/3397622421 learned_addr learned my addr 172.21.15.7:0/3397622421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:28.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.009+0000 7fb07a600640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb07415de70 0x7fb0740ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:28.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.009+0000 7fb079200640 1 -- 172.21.15.7:0/3397622421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0740b0a50 msgr2=0x7fb07414a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:28.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.009+0000 7fb079200640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0740b0a50 0x7fb07414a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:28.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.009+0000 7fb079200640 1 -- 172.21.15.7:0/3397622421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb07415de70 msgr2=0x7fb0740ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:28.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.009+0000 7fb079200640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb07415de70 0x7fb0740ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:28.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.009+0000 7fb079200640 1 -- 172.21.15.7:0/3397622421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07414c670 con 0x7fb0740b6790 2024-04-25T11:15:28.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.009+0000 7fb079200640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0740b6790 0x7fb0740ce220 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb06000c820 tx=0x7fb06000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:28.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.010+0000 7fb071600640 1 -- 172.21.15.7:0/3397622421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060015070 con 0x7fb0740b6790 2024-04-25T11:15:28.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.010+0000 7fb071600640 1 -- 172.21.15.7:0/3397622421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb060007a30 con 0x7fb0740b6790 2024-04-25T11:15:28.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.010+0000 7fb071600640 1 -- 172.21.15.7:0/3397622421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060007de0 con 0x7fb0740b6790 2024-04-25T11:15:28.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.010+0000 7fb07c600640 1 -- 172.21.15.7:0/3397622421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb074153750 con 0x7fb0740b6790 2024-04-25T11:15:28.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.010+0000 7fb07c600640 1 -- 172.21.15.7:0/3397622421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07414c100 con 0x7fb0740b6790 2024-04-25T11:15:28.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.011+0000 7fb07c600640 1 -- 172.21.15.7:0/3397622421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07406a100 con 0x7fb0740b6790 2024-04-25T11:15:28.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.012+0000 7fb071600640 1 -- 172.21.15.7:0/3397622421 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb060018070 con 0x7fb0740b6790 2024-04-25T11:15:28.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.013+0000 7fb071600640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0540744b0 0x7fb0540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:28.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.014+0000 7fb071600640 1 -- 172.21.15.7:0/3397622421 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fb060095350 con 0x7fb0740b6790 2024-04-25T11:15:28.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.014+0000 7fb079c00640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0540744b0 0x7fb0540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.014+0000 7fb079c00640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0540744b0 0x7fb0540769c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb068034000 tx=0x7fb06802c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:28.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.019+0000 7fb071600640 1 -- 172.21.15.7:0/3397622421 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb06005f940 con 0x7fb0740b6790 2024-04-25T11:15:28.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.267+0000 7fb07c600640 1 -- 172.21.15.7:0/3397622421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb07414c3c0 con 0x7fb0740b6790 2024-04-25T11:15:28.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.268+0000 7fb071600640 1 -- 172.21.15.7:0/3397622421 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5171 (secure 0 0 0) 0x7fb06005f240 con 0x7fb0740b6790 2024-04-25T11:15:28.268 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:28.268 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/779859707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":779859707},{"type":"v1","addr":"172.21.15.87:6834","nonce":779859707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/454388957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":454388957},{"type":"v1","addr":"172.21.15.87:6835","nonce":454388957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2318271315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2318271315},{"type":"v1","addr":"172.21.15.7:6836","nonce":2318271315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:15:05.259317+0000","modified":"2024-04-25T11:15:08.314094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6528},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6528":{"gid":6528,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2955530443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2955530443},{"type":"v1","addr":"172.21.15.7:6837","nonce":2955530443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6528,"qdb_cluster":[6528]},"id":5}]} 2024-04-25T11:15:28.269 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 43 2024-04-25T11:15:28.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.271+0000 7fb07c600640 1 -- 172.21.15.7:0/3397622421 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0540744b0 msgr2=0x7fb0540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:28.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.271+0000 7fb07c600640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0540744b0 0x7fb0540769c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb068034000 tx=0x7fb06802c000 comp rx=0 tx=0).stop 2024-04-25T11:15:28.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.271+0000 7fb07c600640 1 -- 172.21.15.7:0/3397622421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0740b6790 msgr2=0x7fb0740ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:28.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.271+0000 7fb07c600640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0740b6790 0x7fb0740ce220 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb06000c820 tx=0x7fb06000cce0 comp rx=0 tx=0).stop 2024-04-25T11:15:28.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.271+0000 7fb07c600640 1 -- 172.21.15.7:0/3397622421 shutdown_connections 2024-04-25T11:15:28.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.271+0000 7fb07c600640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb0540744b0 0x7fb0540769c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:28.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.271+0000 7fb07c600640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb07415de70 0x7fb0740ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:28.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.271+0000 7fb07c600640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0740b6790 0x7fb0740ce220 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:28.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.271+0000 7fb07c600640 1 --2- 172.21.15.7:0/3397622421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0740b0a50 0x7fb07414a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:28.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.271+0000 7fb07c600640 1 -- 172.21.15.7:0/3397622421 >> 172.21.15.7:0/3397622421 conn(0x7fb074074200 msgr2=0x7fb07415ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:28.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.271+0000 7fb07c600640 1 -- 172.21.15.7:0/3397622421 shutdown_connections 2024-04-25T11:15:28.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:28.272+0000 7fb07c600640 1 -- 172.21.15.7:0/3397622421 wait complete. 2024-04-25T11:15:28.297 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T11:15:28.614 INFO:teuthology.orchestra.run.smithi007.stdout:[{"id":6690,"entity":{"name":{"type":"client","num":6690},"addr":{"type":"any","addr":"172.21.15.87:0","nonce":2051151439}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":12.240804625000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9235243084592772,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6690 172.21.15.87:0/2051151439","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi087","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":5429,"entity":{"name":{"type":"client","num":5429},"addr":{"type":"any","addr":"172.21.15.7:0","nonce":2406792213}},"state":"open","num_leases":1,"num_caps":289,"request_load_avg":586,"uptime":13.995776615,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":1379.6999403074387,"halflife":60},"release_caps":{"value":303.43150874251791,"halflife":60},"recall_caps_throttle":{"value":468.029979737112,"halflife":1.5},"recall_caps_throttle2o":{"value":108.51585277649593,"halflife":0.5},"session_cache_liveness":{"value":1188.4162531314473,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":630,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5429 172.21.15.7:0/2406792213","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi007","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T11:15:28.615 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi087", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.87:0", "nonce": 2051151439, "type": "any" }, "name": { "num": 6690, "type": "client" } }, "id": 6690, "inst": "client.6690 172.21.15.87:0/2051151439", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9235243084592772 }, "state": "open", "uptime": 12.240804625 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi007", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.7:0", "nonce": 2406792213, "type": "any" }, "name": { "num": 5429, "type": "client" } }, "id": 5429, "inst": "client.5429 172.21.15.7:0/2406792213", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 630, "num_caps": 289, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 1379.6999403074387 }, "recall_caps_throttle": { "halflife": 1.5, "value": 468.029979737112 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 108.51585277649593 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 303.4315087425179 }, "request_load_avg": 586, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1188.4162531314473 }, "state": "open", "uptime": 13.995776615 } ] 2024-04-25T11:15:28.615 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-04-25T11:15:33.617 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:15:33.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.912+0000 7f2bcf200640 1 Processor -- start 2024-04-25T11:15:33.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.912+0000 7f2bcf200640 1 -- start start 2024-04-25T11:15:33.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.914+0000 7f2bcf200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bd00b7ae0 0x7f2bd00af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:33.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.914+0000 7f2bcf200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd00b3d70 0x7f2bd00afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:33.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.914+0000 7f2bcf200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bd00d2990 con 0x7f2bd00b3960 2024-04-25T11:15:33.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.914+0000 7f2bcf200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2bd00d2c50 con 0x7f2bd00b7ae0 2024-04-25T11:15:33.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.914+0000 7f2bcf200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd00d2f30 con 0x7f2bd00b3d70 2024-04-25T11:15:33.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.914+0000 7f2bcd400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bd00b7ae0 0x7f2bd00af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:33.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.914+0000 7f2bcde00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2bd00b3960 0x7f2bd00b73d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60010/0 (socket says 172.21.15.7:60010) 2024-04-25T11:15:33.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.914+0000 7f2bcd400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bd00b7ae0 0x7f2bd00af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53280/0 (socket says 172.21.15.7:53280) 2024-04-25T11:15:33.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.914+0000 7f2bcde00640 1 -- 172.21.15.7:0/210918669 learned_addr learned my addr 172.21.15.7:0/210918669 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:33.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.915+0000 7f2bce800640 1 --2- 172.21.15.7:0/210918669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd00b3d70 0x7f2bd00afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:33.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.915+0000 7f2bcd400640 1 -- 172.21.15.7:0/210918669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd00b3d70 msgr2=0x7f2bd00afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:33.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.915+0000 7f2bcd400640 1 --2- 172.21.15.7:0/210918669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd00b3d70 0x7f2bd00afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:33.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.915+0000 7f2bcd400640 1 -- 172.21.15.7:0/210918669 >> v1:172.21.15.7:6789/0 conn(0x7f2bd00b3960 legacy=0x7f2bd00b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:33.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.915+0000 7f2bcd400640 1 -- 172.21.15.7:0/210918669 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd00d3210 con 0x7f2bd00b7ae0 2024-04-25T11:15:33.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.915+0000 7f2bcca00640 1 -- 172.21.15.7:0/210918669 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 893239054 0 0) 0x7f2bd00d2990 con 0x7f2bd00b3960 2024-04-25T11:15:33.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.915+0000 7f2bce800640 1 --2- 172.21.15.7:0/210918669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd00b3d70 0x7f2bd00afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:33.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.916+0000 7f2bcd400640 1 --2- 172.21.15.7:0/210918669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd00b7ae0 0x7f2bd00af8a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009d90 tx=0x7f2bc002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837e8badc4107b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:33.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.916+0000 7f2bcca00640 1 -- 172.21.15.7:0/210918669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0007740 con 0x7f2bd00b7ae0 2024-04-25T11:15:33.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.916+0000 7f2bcca00640 1 -- 172.21.15.7:0/210918669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc00078e0 con 0x7f2bd00b7ae0 2024-04-25T11:15:33.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.916+0000 7f2bcca00640 1 -- 172.21.15.7:0/210918669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0007cb0 con 0x7f2bd00b7ae0 2024-04-25T11:15:33.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.917+0000 7f2bcf200640 1 -- 172.21.15.7:0/210918669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd00b7ae0 msgr2=0x7f2bd00af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:33.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.917+0000 7f2bcf200640 1 --2- 172.21.15.7:0/210918669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd00b7ae0 0x7f2bd00af8a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009d90 tx=0x7f2bc002a660 comp rx=0 tx=0).stop 2024-04-25T11:15:33.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.917+0000 7f2bcf200640 1 -- 172.21.15.7:0/210918669 shutdown_connections 2024-04-25T11:15:33.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.917+0000 7f2bcf200640 1 --2- 172.21.15.7:0/210918669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd00b3d70 0x7f2bd00afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:33.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.917+0000 7f2bcf200640 1 --2- 172.21.15.7:0/210918669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd00b7ae0 0x7f2bd00af8a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:33.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.917+0000 7f2bcf200640 1 -- 172.21.15.7:0/210918669 >> 172.21.15.7:0/210918669 conn(0x7f2bd009b540 msgr2=0x7f2bd009b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:33.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.917+0000 7f2bcf200640 1 -- 172.21.15.7:0/210918669 shutdown_connections 2024-04-25T11:15:33.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.917+0000 7f2bcf200640 1 -- 172.21.15.7:0/210918669 wait complete. 2024-04-25T11:15:33.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.919+0000 7f2bcf200640 1 Processor -- start 2024-04-25T11:15:33.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.920+0000 7f2bcf200640 1 -- start start 2024-04-25T11:15:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.921+0000 7f2bcf200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd00b3960 0x7f2bd014aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.921+0000 7f2bcf200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd00b3d70 0x7f2bd006c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.921+0000 7f2bcf200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bd006cab0 0x7f2bd006cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.921+0000 7f2bcf200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd006eb10 con 0x7f2bd00b3d70 2024-04-25T11:15:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.921+0000 7f2bcf200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd006e990 con 0x7f2bd006cab0 2024-04-25T11:15:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.921+0000 7f2bcf200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd00b0320 con 0x7f2bd00b3960 2024-04-25T11:15:33.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.921+0000 7f2bcd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd00b3d70 0x7f2bd006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.922+0000 7f2bcd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd00b3d70 0x7f2bd006c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53294/0 (socket says 172.21.15.7:53294) 2024-04-25T11:15:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.922+0000 7f2bcd400640 1 -- 172.21.15.7:0/2819776248 learned_addr learned my addr 172.21.15.7:0/2819776248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.922+0000 7f2bce800640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bd006cab0 0x7f2bd006cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.922+0000 7f2bcde00640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd00b3960 0x7f2bd014aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.922+0000 7f2bcd400640 1 -- 172.21.15.7:0/2819776248 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd00b3960 msgr2=0x7f2bd014aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.922+0000 7f2bcd400640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd00b3960 0x7f2bd014aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.922+0000 7f2bcd400640 1 -- 172.21.15.7:0/2819776248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bd006cab0 msgr2=0x7f2bd006cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.922+0000 7f2bcd400640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bd006cab0 0x7f2bd006cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:33.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.922+0000 7f2bcd400640 1 -- 172.21.15.7:0/2819776248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd01595b0 con 0x7f2bd00b3d70 2024-04-25T11:15:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.922+0000 7f2bcd400640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd00b3d70 0x7f2bd006c570 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2bc002ac50 tx=0x7f2bc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.923+0000 7f2bc6000640 1 -- 172.21.15.7:0/2819776248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc003f070 con 0x7f2bd00b3d70 2024-04-25T11:15:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.923+0000 7f2bc6000640 1 -- 172.21.15.7:0/2819776248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc0031d00 con 0x7f2bd00b3d70 2024-04-25T11:15:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.923+0000 7f2bcf200640 1 -- 172.21.15.7:0/2819776248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd015c2f0 con 0x7f2bd00b3d70 2024-04-25T11:15:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.923+0000 7f2bc6000640 1 -- 172.21.15.7:0/2819776248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0044680 con 0x7f2bd00b3d70 2024-04-25T11:15:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.923+0000 7f2bcf200640 1 -- 172.21.15.7:0/2819776248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd015c8b0 con 0x7f2bd00b3d70 2024-04-25T11:15:33.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.924+0000 7f2bcf200640 1 -- 172.21.15.7:0/2819776248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd015d760 con 0x7f2bd00b3d70 2024-04-25T11:15:33.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.925+0000 7f2bc6000640 1 -- 172.21.15.7:0/2819776248 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2bc0003990 con 0x7f2bd00b3d70 2024-04-25T11:15:33.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.926+0000 7f2bc6000640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bac0744e0 0x7f2bac0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:33.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.926+0000 7f2bc6000640 1 -- 172.21.15.7:0/2819776248 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f2bc00b6e70 con 0x7f2bd00b3d70 2024-04-25T11:15:33.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.927+0000 7f2bcde00640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bac0744e0 0x7f2bac0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:33.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.927+0000 7f2bcde00640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bac0744e0 0x7f2bac0769f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8000f60 tx=0x7f2bb8007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:33.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:33.931+0000 7f2bc6000640 1 -- 172.21.15.7:0/2819776248 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2bd015d760 con 0x7f2bd00b3d70 2024-04-25T11:15:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.183+0000 7f2bcf200640 1 -- 172.21.15.7:0/2819776248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bd0050740 con 0x7f2bd00b3d70 2024-04-25T11:15:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.184+0000 7f2bc6000640 1 -- 172.21.15.7:0/2819776248 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5171 (secure 0 0 0) 0x7f2bc0049060 con 0x7f2bd00b3d70 2024-04-25T11:15:34.185 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:34.185 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/779859707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":779859707},{"type":"v1","addr":"172.21.15.87:6834","nonce":779859707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/454388957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":454388957},{"type":"v1","addr":"172.21.15.87:6835","nonce":454388957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2318271315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2318271315},{"type":"v1","addr":"172.21.15.7:6836","nonce":2318271315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:15:05.259317+0000","modified":"2024-04-25T11:15:08.314094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6528},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6528":{"gid":6528,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2955530443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2955530443},{"type":"v1","addr":"172.21.15.7:6837","nonce":2955530443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6528,"qdb_cluster":[6528]},"id":5}]} 2024-04-25T11:15:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 43 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.187+0000 7f2bcf200640 1 -- 172.21.15.7:0/2819776248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bac0744e0 msgr2=0x7f2bac0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.187+0000 7f2bcf200640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bac0744e0 0x7f2bac0769f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8000f60 tx=0x7f2bb8007c00 comp rx=0 tx=0).stop 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.187+0000 7f2bcf200640 1 -- 172.21.15.7:0/2819776248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd00b3d70 msgr2=0x7f2bd006c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.187+0000 7f2bcf200640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd00b3d70 0x7f2bd006c570 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2bc002ac50 tx=0x7f2bc0004a40 comp rx=0 tx=0).stop 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.187+0000 7f2bcf200640 1 -- 172.21.15.7:0/2819776248 shutdown_connections 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.187+0000 7f2bcf200640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bac0744e0 0x7f2bac0769f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.187+0000 7f2bcf200640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bd006cab0 0x7f2bd006cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.187+0000 7f2bcf200640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd00b3d70 0x7f2bd006c570 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.187+0000 7f2bcf200640 1 --2- 172.21.15.7:0/2819776248 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd00b3960 0x7f2bd014aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.188+0000 7f2bcf200640 1 -- 172.21.15.7:0/2819776248 >> 172.21.15.7:0/2819776248 conn(0x7f2bd009b540 msgr2=0x7f2bd00d20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.188+0000 7f2bcf200640 1 -- 172.21.15.7:0/2819776248 shutdown_connections 2024-04-25T11:15:34.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:34.188+0000 7f2bcf200640 1 -- 172.21.15.7:0/2819776248 wait complete. 2024-04-25T11:15:34.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T11:15:34.526 INFO:teuthology.orchestra.run.smithi007.stdout:[{"id":6690,"entity":{"name":{"type":"client","num":6690},"addr":{"type":"any","addr":"172.21.15.87:0","nonce":2051151439}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":18.154710231999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.8838488843763139,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6690 172.21.15.87:0/2051151439","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi087","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":5429,"entity":{"name":{"type":"client","num":5429},"addr":{"type":"any","addr":"172.21.15.7:0","nonce":2406792213}},"state":"open","num_leases":1,"num_caps":162,"request_load_avg":547,"uptime":19.909682222000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":1462.2819103279621,"halflife":60},"release_caps":{"value":402.98316428105858,"halflife":60},"recall_caps_throttle":{"value":57.396989936734343,"halflife":1.5},"recall_caps_throttle2o":{"value":0.2651498118862366,"halflife":0.5},"session_cache_liveness":{"value":1172.2881440907104,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":601,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.5429 172.21.15.7:0/2406792213","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x1000000044d","length":903}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi007","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T11:15:34.527 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi087", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.87:0", "nonce": 2051151439, "type": "any" }, "name": { "num": 6690, "type": "client" } }, "id": 6690, "inst": "client.6690 172.21.15.87:0/2051151439", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.883848884376314 }, "state": "open", "uptime": 18.154710232 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi007", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.7:0", "nonce": 2406792213, "type": "any" }, "name": { "num": 5429, "type": "client" } }, "id": 5429, "inst": "client.5429 172.21.15.7:0/2406792213", "last_trim_completed_flushes_tid": 601, "last_trim_completed_requests_tid": 0, "num_caps": 162, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 903, "start": "0x1000000044d" } ], "recall_caps": { "halflife": 60, "value": 1462.2819103279621 }, "recall_caps_throttle": { "halflife": 1.5, "value": 57.39698993673434 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0.2651498118862366 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 402.9831642810586 }, "request_load_avg": 547, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1172.2881440907104 }, "state": "open", "uptime": 19.909682222 } ] 2024-04-25T11:15:34.527 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:15:34.527 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:15:34.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:15:34.639 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:15:34.755 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:15:34.755 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:15:34.755 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:15:34.755 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:34.815 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:15:34.815 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:34.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.153+0000 7f64a3200640 1 Processor -- start 2024-04-25T11:15:35.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.153+0000 7f64a3200640 1 -- start start 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.155+0000 7f64a3200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64a40aed40 0x7f64a415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.155+0000 7f64a3200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f64a40af420 0x7f64a40af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.155+0000 7f64a3200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a40cebb0 con 0x7f64a40b5220 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.155+0000 7f64a3200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f64a415e1f0 con 0x7f64a40af420 2024-04-25T11:15:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.155+0000 7f64a3200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f64a40afd70 con 0x7f64a40aed40 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.156+0000 7f64a2800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f64a40af420 0x7f64a40af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.156+0000 7f64a1e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64a40aed40 0x7f64a415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.156+0000 7f64a1400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f64a40b5220 0x7f64a40ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:48874/0 (socket says 172.21.15.7:48874) 2024-04-25T11:15:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.156+0000 7f64a1400640 1 -- 172.21.15.7:0/3152583516 learned_addr learned my addr 172.21.15.7:0/3152583516 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.157+0000 7f64a1e00640 1 -- 172.21.15.7:0/3152583516 >> v2:172.21.15.87:3300/0 conn(0x7f64a40af420 msgr2=0x7f64a40af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.157+0000 7f64a1e00640 1 --2- 172.21.15.7:0/3152583516 >> v2:172.21.15.87:3300/0 conn(0x7f64a40af420 0x7f64a40af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.157+0000 7f64a1e00640 1 -- 172.21.15.7:0/3152583516 >> v1:172.21.15.87:6789/0 conn(0x7f64a40b5220 legacy=0x7f64a40ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.157+0000 7f64a1e00640 1 -- 172.21.15.7:0/3152583516 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a40b64b0 con 0x7f64a40aed40 2024-04-25T11:15:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.157+0000 7f64a2800640 1 --2- 172.21.15.7:0/3152583516 >> v2:172.21.15.87:3300/0 conn(0x7f64a40af420 0x7f64a40af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:15:35.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.158+0000 7f64a1e00640 1 --2- 172.21.15.7:0/3152583516 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64a40aed40 0x7f64a415dcb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6494009d90 tx=0x7f649402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ee4209a6aa13ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.159+0000 7f64a0a00640 1 -- 172.21.15.7:0/3152583516 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494007740 con 0x7f64a40aed40 2024-04-25T11:15:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.159+0000 7f64a0a00640 1 -- 172.21.15.7:0/3152583516 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64940078e0 con 0x7f64a40aed40 2024-04-25T11:15:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.159+0000 7f64a0a00640 1 -- 172.21.15.7:0/3152583516 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494007cb0 con 0x7f64a40aed40 2024-04-25T11:15:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.160+0000 7f64a3200640 1 -- 172.21.15.7:0/3152583516 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64a40aed40 msgr2=0x7f64a415dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.160+0000 7f64a3200640 1 --2- 172.21.15.7:0/3152583516 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64a40aed40 0x7f64a415dcb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6494009d90 tx=0x7f649402a660 comp rx=0 tx=0).stop 2024-04-25T11:15:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.160+0000 7f64a3200640 1 -- 172.21.15.7:0/3152583516 shutdown_connections 2024-04-25T11:15:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.160+0000 7f64a3200640 1 --2- 172.21.15.7:0/3152583516 >> v2:172.21.15.87:3300/0 conn(0x7f64a40af420 0x7f64a40af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.160+0000 7f64a3200640 1 --2- 172.21.15.7:0/3152583516 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64a40aed40 0x7f64a415dcb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.160+0000 7f64a3200640 1 -- 172.21.15.7:0/3152583516 >> 172.21.15.7:0/3152583516 conn(0x7f64a40b3820 msgr2=0x7f64a40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.160+0000 7f64a3200640 1 -- 172.21.15.7:0/3152583516 shutdown_connections 2024-04-25T11:15:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.160+0000 7f64a3200640 1 -- 172.21.15.7:0/3152583516 wait complete. 2024-04-25T11:15:35.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.163+0000 7f64a3200640 1 Processor -- start 2024-04-25T11:15:35.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.163+0000 7f64a3200640 1 -- start start 2024-04-25T11:15:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.164+0000 7f64a3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a40aed40 0x7f64a41587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.164+0000 7f64a3200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64a40af420 0x7f64a41f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.164+0000 7f64a3200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a40b5220 0x7f64a41f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.164+0000 7f64a3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f64a406cbd0 con 0x7f64a40aed40 2024-04-25T11:15:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.164+0000 7f64a3200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f64a406c8f0 con 0x7f64a40b5220 2024-04-25T11:15:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.164+0000 7f64a3200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f64a406ceb0 con 0x7f64a40af420 2024-04-25T11:15:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.165+0000 7f64a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a40aed40 0x7f64a41587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.165+0000 7f64a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a40aed40 0x7f64a41587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53310/0 (socket says 172.21.15.7:53310) 2024-04-25T11:15:35.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.165+0000 7f64a1e00640 1 -- 172.21.15.7:0/2763095964 learned_addr learned my addr 172.21.15.7:0/2763095964 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.165+0000 7f64a2800640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a40b5220 0x7f64a41f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.165+0000 7f64a1400640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64a40af420 0x7f64a41f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.166+0000 7f64a1e00640 1 -- 172.21.15.7:0/2763095964 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64a40af420 msgr2=0x7f64a41f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.166+0000 7f64a1e00640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64a40af420 0x7f64a41f7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.166+0000 7f64a1e00640 1 -- 172.21.15.7:0/2763095964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a40b5220 msgr2=0x7f64a41f8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.166+0000 7f64a1e00640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a40b5220 0x7f64a41f8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.166+0000 7f64a1e00640 1 -- 172.21.15.7:0/2763095964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a4159410 con 0x7f64a40aed40 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.166+0000 7f64a1e00640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a40aed40 0x7f64a41587a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6494009d90 tx=0x7f6494004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.166+0000 7f649a000640 1 -- 172.21.15.7:0/2763095964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649403f070 con 0x7f64a40aed40 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.166+0000 7f649a000640 1 -- 172.21.15.7:0/2763095964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6494031c40 con 0x7f64a40aed40 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.166+0000 7f649a000640 1 -- 172.21.15.7:0/2763095964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6494009450 con 0x7f64a40aed40 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.166+0000 7f64a3200640 1 -- 172.21.15.7:0/2763095964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a41596f0 con 0x7f64a40aed40 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.167+0000 7f64a3200640 1 -- 172.21.15.7:0/2763095964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a41f7920 con 0x7f64a40aed40 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.168+0000 7f64a3200640 1 -- 172.21.15.7:0/2763095964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a4050740 con 0x7f64a40aed40 2024-04-25T11:15:35.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.169+0000 7f649a000640 1 -- 172.21.15.7:0/2763095964 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6494044460 con 0x7f64a40aed40 2024-04-25T11:15:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.170+0000 7f649a000640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64800744b0 0x7f64800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.170+0000 7f649a000640 1 -- 172.21.15.7:0/2763095964 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f64940b6c30 con 0x7f64a40aed40 2024-04-25T11:15:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.170+0000 7f64a1400640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64800744b0 0x7f64800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.172+0000 7f64a1400640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64800744b0 0x7f64800769c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f64a4156a00 tx=0x7f648c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:35.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.175+0000 7f649a000640 1 -- 172.21.15.7:0/2763095964 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6494080bd0 con 0x7f64a40aed40 2024-04-25T11:15:35.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.379+0000 7f64a3200640 1 -- 172.21.15.7:0/2763095964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64a40b2dd0 con 0x7f64a40aed40 2024-04-25T11:15:35.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.381+0000 7f649a000640 1 -- 172.21.15.7:0/2763095964 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16849 (secure 0 0 0) 0x7f6494084ed0 con 0x7f64a40aed40 2024-04-25T11:15:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:35.381 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:15:06.697374+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:03.587954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T11:15:04.226794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:15:35.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 -- 172.21.15.7:0/2763095964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64800744b0 msgr2=0x7f64800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64800744b0 0x7f64800769c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f64a4156a00 tx=0x7f648c009040 comp rx=0 tx=0).stop 2024-04-25T11:15:35.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 -- 172.21.15.7:0/2763095964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a40aed40 msgr2=0x7f64a41587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a40aed40 0x7f64a41587a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6494009d90 tx=0x7f6494004a40 comp rx=0 tx=0).stop 2024-04-25T11:15:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 -- 172.21.15.7:0/2763095964 shutdown_connections 2024-04-25T11:15:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64800744b0 0x7f64800769c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64a40b5220 0x7f64a41f8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64a40af420 0x7f64a41f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 --2- 172.21.15.7:0/2763095964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64a40aed40 0x7f64a41587a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 -- 172.21.15.7:0/2763095964 >> 172.21.15.7:0/2763095964 conn(0x7f64a40b3820 msgr2=0x7f64a40ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 -- 172.21.15.7:0/2763095964 shutdown_connections 2024-04-25T11:15:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.384+0000 7f64a3200640 1 -- 172.21.15.7:0/2763095964 wait complete. 2024-04-25T11:15:35.413 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:15:03.587954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:15:04.226794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:15:35.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:15:35.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.691+0000 7f805b200640 1 Processor -- start 2024-04-25T11:15:35.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.691+0000 7f805b200640 1 -- start start 2024-04-25T11:15:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.694+0000 7f805b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f805c0ce8d0 0x7f805c0cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.694+0000 7f805b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f805c15c2e0 0x7f805c15c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.694+0000 7f805b200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f805c0b4830 con 0x7f805c15fef0 2024-04-25T11:15:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.694+0000 7f805b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f805c0b4b10 con 0x7f805c0ce8d0 2024-04-25T11:15:35.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.694+0000 7f805b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f805c0b4df0 con 0x7f805c15c2e0 2024-04-25T11:15:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.695+0000 7f8059e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f805c0ce8d0 0x7f805c0cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.695+0000 7f8059400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f805c15c2e0 0x7f805c15c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.695+0000 7f8059e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f805c0ce8d0 0x7f805c0cece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:35286/0 (socket says 172.21.15.7:35286) 2024-04-25T11:15:35.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.695+0000 7f8059e00640 1 -- 172.21.15.7:0/1039447146 learned_addr learned my addr 172.21.15.7:0/1039447146 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.696+0000 7f8059e00640 1 -- 172.21.15.7:0/1039447146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f805c15c2e0 msgr2=0x7f805c15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.696+0000 7f8059e00640 1 --2- 172.21.15.7:0/1039447146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f805c15c2e0 0x7f805c15c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.696+0000 7f8058a00640 1 -- 172.21.15.7:0/1039447146 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1790305097 0 0) 0x7f805c0b4830 con 0x7f805c15fef0 2024-04-25T11:15:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.696+0000 7f8059e00640 1 -- 172.21.15.7:0/1039447146 >> v1:172.21.15.87:6789/0 conn(0x7f805c15fef0 legacy=0x7f805c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.696+0000 7f8059e00640 1 -- 172.21.15.7:0/1039447146 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c0b50d0 con 0x7f805c0ce8d0 2024-04-25T11:15:35.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.696+0000 7f8059400640 1 --2- 172.21.15.7:0/1039447146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f805c15c2e0 0x7f805c15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:15:35.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.697+0000 7f8059e00640 1 --2- 172.21.15.7:0/1039447146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f805c0ce8d0 0x7f805c0cece0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f804c00bb10 tx=0x7f804c02c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f0093574192cc6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.698+0000 7f8058a00640 1 -- 172.21.15.7:0/1039447146 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c036020 con 0x7f805c0ce8d0 2024-04-25T11:15:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.698+0000 7f8058a00640 1 -- 172.21.15.7:0/1039447146 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f804c032820 con 0x7f805c0ce8d0 2024-04-25T11:15:35.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.698+0000 7f8058a00640 1 -- 172.21.15.7:0/1039447146 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804c032bf0 con 0x7f805c0ce8d0 2024-04-25T11:15:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.698+0000 7f805b200640 1 -- 172.21.15.7:0/1039447146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f805c0ce8d0 msgr2=0x7f805c0cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.698+0000 7f805b200640 1 --2- 172.21.15.7:0/1039447146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f805c0ce8d0 0x7f805c0cece0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f804c00bb10 tx=0x7f804c02c920 comp rx=0 tx=0).stop 2024-04-25T11:15:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.699+0000 7f805b200640 1 -- 172.21.15.7:0/1039447146 shutdown_connections 2024-04-25T11:15:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.699+0000 7f805b200640 1 --2- 172.21.15.7:0/1039447146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f805c15c2e0 0x7f805c15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.699+0000 7f805b200640 1 --2- 172.21.15.7:0/1039447146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f805c0ce8d0 0x7f805c0cece0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.699+0000 7f805b200640 1 -- 172.21.15.7:0/1039447146 >> 172.21.15.7:0/1039447146 conn(0x7f805c0740f0 msgr2=0x7f805c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.699+0000 7f805b200640 1 -- 172.21.15.7:0/1039447146 shutdown_connections 2024-04-25T11:15:35.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.699+0000 7f805b200640 1 -- 172.21.15.7:0/1039447146 wait complete. 2024-04-25T11:15:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.701+0000 7f805b200640 1 Processor -- start 2024-04-25T11:15:35.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.701+0000 7f805b200640 1 -- start start 2024-04-25T11:15:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.703+0000 7f805b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f805c0ce8d0 0x7f805c1f0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.703+0000 7f805b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805c15c2e0 0x7f805c14d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.703+0000 7f805b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f805c15fef0 0x7f805c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.703+0000 7f805b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f805c070030 con 0x7f805c15c2e0 2024-04-25T11:15:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.703+0000 7f805b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f805c06feb0 con 0x7f805c0ce8d0 2024-04-25T11:15:35.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.703+0000 7f805b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f805c160a10 con 0x7f805c15fef0 2024-04-25T11:15:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.703+0000 7f8059400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805c15c2e0 0x7f805c14d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.703+0000 7f8059400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805c15c2e0 0x7f805c14d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53312/0 (socket says 172.21.15.7:53312) 2024-04-25T11:15:35.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.703+0000 7f8059400640 1 -- 172.21.15.7:0/2888041769 learned_addr learned my addr 172.21.15.7:0/2888041769 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.704+0000 7f8059e00640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f805c0ce8d0 0x7f805c1f0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.704+0000 7f805a800640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f805c15fef0 0x7f805c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.704+0000 7f8059400640 1 -- 172.21.15.7:0/2888041769 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f805c15fef0 msgr2=0x7f805c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.704+0000 7f8059400640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f805c15fef0 0x7f805c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.704+0000 7f8059400640 1 -- 172.21.15.7:0/2888041769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f805c0ce8d0 msgr2=0x7f805c1f0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.704+0000 7f8059400640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f805c0ce8d0 0x7f805c1f0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.704+0000 7f8059400640 1 -- 172.21.15.7:0/2888041769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c1fa170 con 0x7f805c15c2e0 2024-04-25T11:15:35.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.704+0000 7f8059400640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805c15c2e0 0x7f805c14d4e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f804800e990 tx=0x7f804800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:35.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.704+0000 7f8052000640 1 -- 172.21.15.7:0/2888041769 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048017070 con 0x7f805c15c2e0 2024-04-25T11:15:35.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.705+0000 7f805b200640 1 -- 172.21.15.7:0/2888041769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805c1fa450 con 0x7f805c15c2e0 2024-04-25T11:15:35.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.705+0000 7f8052000640 1 -- 172.21.15.7:0/2888041769 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f804800c980 con 0x7f805c15c2e0 2024-04-25T11:15:35.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.705+0000 7f8052000640 1 -- 172.21.15.7:0/2888041769 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804800cd50 con 0x7f805c15c2e0 2024-04-25T11:15:35.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.705+0000 7f805b200640 1 -- 172.21.15.7:0/2888041769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805c1faa10 con 0x7f805c15c2e0 2024-04-25T11:15:35.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.706+0000 7f805b200640 1 -- 172.21.15.7:0/2888041769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805c050740 con 0x7f805c15c2e0 2024-04-25T11:15:35.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.707+0000 7f8052000640 1 -- 172.21.15.7:0/2888041769 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f804801a070 con 0x7f805c15c2e0 2024-04-25T11:15:35.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.709+0000 7f8052000640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 0x7f803c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:35.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.709+0000 7f8052000640 1 -- 172.21.15.7:0/2888041769 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f8048095610 con 0x7f805c15c2e0 2024-04-25T11:15:35.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.709+0000 7f8059e00640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 0x7f803c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:35.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.710+0000 7f8059e00640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 0x7f803c0769f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f804c037000 tx=0x7f804c032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:35.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.713+0000 7f8052000640 1 -- 172.21.15.7:0/2888041769 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f804805fb90 con 0x7f805c15c2e0 2024-04-25T11:15:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.920+0000 7f805b200640 1 -- 172.21.15.7:0/2888041769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f805c1f9ec0 con 0x7f805c15c2e0 2024-04-25T11:15:35.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.921+0000 7f8052000640 1 -- 172.21.15.7:0/2888041769 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f804805f490 con 0x7f805c15c2e0 2024-04-25T11:15:35.922 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:15:35.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 -- 172.21.15.7:0/2888041769 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 msgr2=0x7f803c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 0x7f803c0769f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f804c037000 tx=0x7f804c032000 comp rx=0 tx=0).stop 2024-04-25T11:15:35.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 -- 172.21.15.7:0/2888041769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805c15c2e0 msgr2=0x7f805c14d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:35.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805c15c2e0 0x7f805c14d4e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f804800e990 tx=0x7f804800ee50 comp rx=0 tx=0).stop 2024-04-25T11:15:35.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 -- 172.21.15.7:0/2888041769 shutdown_connections 2024-04-25T11:15:35.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 0x7f803c0769f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f805c15fef0 0x7f805c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f805c15c2e0 0x7f805c14d4e0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 --2- 172.21.15.7:0/2888041769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f805c0ce8d0 0x7f805c1f0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:35.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 -- 172.21.15.7:0/2888041769 >> 172.21.15.7:0/2888041769 conn(0x7f805c0740f0 msgr2=0x7f805c160300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:35.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 -- 172.21.15.7:0/2888041769 shutdown_connections 2024-04-25T11:15:35.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:35.924+0000 7f805b200640 1 -- 172.21.15.7:0/2888041769 wait complete. 2024-04-25T11:15:35.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:15:36.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.239+0000 7fe053e00640 1 Processor -- start 2024-04-25T11:15:36.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.239+0000 7fe053e00640 1 -- start start 2024-04-25T11:15:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.242+0000 7fe053e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe0540d0fb0 0x7fe05415fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.242+0000 7fe053e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0540b8e50 0x7fe0540b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.242+0000 7fe053e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe0540b7920 0x7fe0540d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.242+0000 7fe053e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe0540b7d30 con 0x7fe0540d0fb0 2024-04-25T11:15:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.242+0000 7fe053e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe0540d13c0 con 0x7fe0540b7920 2024-04-25T11:15:36.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.242+0000 7fe053e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe054163f10 con 0x7fe0540b8e50 2024-04-25T11:15:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.242+0000 7fe052a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe0540d0fb0 0x7fe05415fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.242+0000 7fe052a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe0540d0fb0 0x7fe05415fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53326/0 (socket says 172.21.15.7:53326) 2024-04-25T11:15:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.242+0000 7fe052a00640 1 -- 172.21.15.7:0/3258952952 learned_addr learned my addr 172.21.15.7:0/3258952952 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.242+0000 7fe052000640 1 --2- 172.21.15.7:0/3258952952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0540b8e50 0x7fe0540b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.242+0000 7fe053400640 1 --2- 172.21.15.7:0/3258952952 >> v2:172.21.15.87:3300/0 conn(0x7fe0540b7920 0x7fe0540d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.243+0000 7fe052a00640 1 -- 172.21.15.7:0/3258952952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0540b8e50 msgr2=0x7fe0540b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.243+0000 7fe052a00640 1 --2- 172.21.15.7:0/3258952952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0540b8e50 0x7fe0540b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.243+0000 7fe052a00640 1 -- 172.21.15.7:0/3258952952 >> v2:172.21.15.87:3300/0 conn(0x7fe0540b7920 msgr2=0x7fe0540d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.243+0000 7fe052a00640 1 --2- 172.21.15.7:0/3258952952 >> v2:172.21.15.87:3300/0 conn(0x7fe0540b7920 0x7fe0540d45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.243+0000 7fe052a00640 1 -- 172.21.15.7:0/3258952952 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0541641f0 con 0x7fe0540d0fb0 2024-04-25T11:15:36.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.244+0000 7fe052a00640 1 --2- 172.21.15.7:0/3258952952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0540d0fb0 0x7fe05415fea0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe05406ef70 tx=0x7fe04402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd46e83f7f5b76ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.244+0000 7fe051600640 1 -- 172.21.15.7:0/3258952952 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044037070 con 0x7fe0540d0fb0 2024-04-25T11:15:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.244+0000 7fe051600640 1 -- 172.21.15.7:0/3258952952 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe044032440 con 0x7fe0540d0fb0 2024-04-25T11:15:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.244+0000 7fe051600640 1 -- 172.21.15.7:0/3258952952 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044032810 con 0x7fe0540d0fb0 2024-04-25T11:15:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.245+0000 7fe053e00640 1 -- 172.21.15.7:0/3258952952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0540d0fb0 msgr2=0x7fe05415fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:36.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.245+0000 7fe053e00640 1 --2- 172.21.15.7:0/3258952952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0540d0fb0 0x7fe05415fea0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fe05406ef70 tx=0x7fe04402a140 comp rx=0 tx=0).stop 2024-04-25T11:15:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.245+0000 7fe053e00640 1 -- 172.21.15.7:0/3258952952 shutdown_connections 2024-04-25T11:15:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.245+0000 7fe053e00640 1 --2- 172.21.15.7:0/3258952952 >> v2:172.21.15.87:3300/0 conn(0x7fe0540b7920 0x7fe0540d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.245+0000 7fe053e00640 1 --2- 172.21.15.7:0/3258952952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0540b8e50 0x7fe0540b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.245+0000 7fe053e00640 1 --2- 172.21.15.7:0/3258952952 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0540d0fb0 0x7fe05415fea0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.245+0000 7fe053e00640 1 -- 172.21.15.7:0/3258952952 >> 172.21.15.7:0/3258952952 conn(0x7fe0540d23e0 msgr2=0x7fe0540d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.245+0000 7fe053e00640 1 -- 172.21.15.7:0/3258952952 shutdown_connections 2024-04-25T11:15:36.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.245+0000 7fe053e00640 1 -- 172.21.15.7:0/3258952952 wait complete. 2024-04-25T11:15:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.248+0000 7fe053e00640 1 Processor -- start 2024-04-25T11:15:36.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.249+0000 7fe053e00640 1 -- start start 2024-04-25T11:15:36.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.251+0000 7fe053e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0540b7920 0x7fe0541f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.251+0000 7fe053e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0540b8e50 0x7fe0541f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.251+0000 7fe053e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0540d0fb0 0x7fe0541f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.251+0000 7fe053e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe0540b4bc0 con 0x7fe0540d0fb0 2024-04-25T11:15:36.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.251+0000 7fe053e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe0541647b0 con 0x7fe0540b7920 2024-04-25T11:15:36.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.251+0000 7fe053e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe0540b9260 con 0x7fe0540b8e50 2024-04-25T11:15:36.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.251+0000 7fe053400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0540d0fb0 0x7fe0541f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.251+0000 7fe053400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0540d0fb0 0x7fe0541f9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53340/0 (socket says 172.21.15.7:53340) 2024-04-25T11:15:36.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.251+0000 7fe053400640 1 -- 172.21.15.7:0/1362647963 learned_addr learned my addr 172.21.15.7:0/1362647963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:36.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.252+0000 7fe052a00640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0540b7920 0x7fe0541f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.252+0000 7fe052000640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0540b8e50 0x7fe0541f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.252+0000 7fe053400640 1 -- 172.21.15.7:0/1362647963 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0540b8e50 msgr2=0x7fe0541f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:36.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.252+0000 7fe053400640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0540b8e50 0x7fe0541f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.252+0000 7fe053400640 1 -- 172.21.15.7:0/1362647963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0540b7920 msgr2=0x7fe0541f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.252+0000 7fe053400640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0540b7920 0x7fe0541f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.252+0000 7fe053400640 1 -- 172.21.15.7:0/1362647963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe054160f90 con 0x7fe0540d0fb0 2024-04-25T11:15:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.252+0000 7fe052a00640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0540b7920 0x7fe0541f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:15:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.252+0000 7fe053400640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0540d0fb0 0x7fe0541f9a50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe04c007cc0 tx=0x7fe04c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.253+0000 7fe04aa00640 1 -- 172.21.15.7:0/1362647963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c014420 con 0x7fe0540d0fb0 2024-04-25T11:15:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.253+0000 7fe04aa00640 1 -- 172.21.15.7:0/1362647963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe04c019070 con 0x7fe0540d0fb0 2024-04-25T11:15:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.253+0000 7fe04aa00640 1 -- 172.21.15.7:0/1362647963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c014800 con 0x7fe0540d0fb0 2024-04-25T11:15:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.253+0000 7fe053e00640 1 -- 172.21.15.7:0/1362647963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe054161270 con 0x7fe0540d0fb0 2024-04-25T11:15:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.253+0000 7fe053e00640 1 -- 172.21.15.7:0/1362647963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe054161830 con 0x7fe0540d0fb0 2024-04-25T11:15:36.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.254+0000 7fe053e00640 1 -- 172.21.15.7:0/1362647963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe054050740 con 0x7fe0540d0fb0 2024-04-25T11:15:36.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.256+0000 7fe04aa00640 1 -- 172.21.15.7:0/1362647963 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe04c0149a0 con 0x7fe0540d0fb0 2024-04-25T11:15:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.257+0000 7fe04aa00640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe0340744b0 0x7fe0340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.257+0000 7fe04aa00640 1 -- 172.21.15.7:0/1362647963 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fe04c095ac0 con 0x7fe0540d0fb0 2024-04-25T11:15:36.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.257+0000 7fe052a00640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe0340744b0 0x7fe0340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.258+0000 7fe052a00640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe0340744b0 0x7fe0340769c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe044034000 tx=0x7fe04402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:36.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.262+0000 7fe04aa00640 1 -- 172.21.15.7:0/1362647963 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe04c05fa60 con 0x7fe0540d0fb0 2024-04-25T11:15:36.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.465+0000 7fe053e00640 1 -- 172.21.15.7:0/1362647963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe0540d3d40 con 0x7fe0540d0fb0 2024-04-25T11:15:36.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.466+0000 7fe04aa00640 1 -- 172.21.15.7:0/1362647963 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7fe04c063d60 con 0x7fe0540d0fb0 2024-04-25T11:15:36.467 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:15:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 -- 172.21.15.7:0/1362647963 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe0340744b0 msgr2=0x7fe0340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe0340744b0 0x7fe0340769c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe044034000 tx=0x7fe04402e000 comp rx=0 tx=0).stop 2024-04-25T11:15:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 -- 172.21.15.7:0/1362647963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0540d0fb0 msgr2=0x7fe0541f9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0540d0fb0 0x7fe0541f9a50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fe04c007cc0 tx=0x7fe04c00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 -- 172.21.15.7:0/1362647963 shutdown_connections 2024-04-25T11:15:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe0340744b0 0x7fe0340769c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0540d0fb0 0x7fe0541f9a50 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0540b8e50 0x7fe0541f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 --2- 172.21.15.7:0/1362647963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0540b7920 0x7fe0541f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 -- 172.21.15.7:0/1362647963 >> 172.21.15.7:0/1362647963 conn(0x7fe0540d23e0 msgr2=0x7fe0540cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 -- 172.21.15.7:0/1362647963 shutdown_connections 2024-04-25T11:15:36.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.469+0000 7fe053e00640 1 -- 172.21.15.7:0/1362647963 wait complete. 2024-04-25T11:15:36.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:15:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.787+0000 7f4961800640 1 Processor -- start 2024-04-25T11:15:36.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.787+0000 7f4961800640 1 -- start start 2024-04-25T11:15:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.790+0000 7f4961800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f495c06edb0 0x7f495c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.790+0000 7f4961800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c160c00 con 0x7f495c0d1b30 2024-04-25T11:15:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.790+0000 7f4961800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c09beb0 con 0x7f495c1616a0 2024-04-25T11:15:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.790+0000 7f4961800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f495c09c190 con 0x7f495c06edb0 2024-04-25T11:15:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.790+0000 7f495aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f495c06edb0 0x7f495c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.790+0000 7f495b400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f495c0d1b30 0x7f495c0d1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60014/0 (socket says 172.21.15.7:60014) 2024-04-25T11:15:36.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.790+0000 7f495b400640 1 -- 172.21.15.7:0/375438062 learned_addr learned my addr 172.21.15.7:0/375438062 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:36.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.791+0000 7f495aa00640 1 -- 172.21.15.7:0/375438062 >> v1:172.21.15.87:6789/0 conn(0x7f495c1616a0 legacy=0x7f495c09b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:36.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.791+0000 7f495aa00640 1 -- 172.21.15.7:0/375438062 >> v1:172.21.15.7:6789/0 conn(0x7f495c0d1b30 legacy=0x7f495c0d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:36.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.791+0000 7f495aa00640 1 -- 172.21.15.7:0/375438062 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c160ea0 con 0x7f495c06edb0 2024-04-25T11:15:36.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.791+0000 7f4959600640 1 -- 172.21.15.7:0/375438062 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1776764151 0 0) 0x7f495c160c00 con 0x7f495c0d1b30 2024-04-25T11:15:36.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.792+0000 7f495aa00640 1 --2- 172.21.15.7:0/375438062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495c06edb0 0x7f495c0b3c40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f494c0097d0 tx=0x7f494c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85534ab9a952da6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.793+0000 7f4959600640 1 -- 172.21.15.7:0/375438062 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c037070 con 0x7f495c06edb0 2024-04-25T11:15:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.793+0000 7f4959600640 1 -- 172.21.15.7:0/375438062 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494c032440 con 0x7f495c06edb0 2024-04-25T11:15:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.793+0000 7f4959600640 1 -- 172.21.15.7:0/375438062 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c032810 con 0x7f495c06edb0 2024-04-25T11:15:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.794+0000 7f4961800640 1 -- 172.21.15.7:0/375438062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495c06edb0 msgr2=0x7f495c0b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.794+0000 7f4961800640 1 --2- 172.21.15.7:0/375438062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495c06edb0 0x7f495c0b3c40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f494c0097d0 tx=0x7f494c02a270 comp rx=0 tx=0).stop 2024-04-25T11:15:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.794+0000 7f4961800640 1 -- 172.21.15.7:0/375438062 shutdown_connections 2024-04-25T11:15:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.794+0000 7f4961800640 1 --2- 172.21.15.7:0/375438062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495c06edb0 0x7f495c0b3c40 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.794+0000 7f4961800640 1 -- 172.21.15.7:0/375438062 >> 172.21.15.7:0/375438062 conn(0x7f495c0b7620 msgr2=0x7f495c0b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.794+0000 7f4961800640 1 -- 172.21.15.7:0/375438062 shutdown_connections 2024-04-25T11:15:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.794+0000 7f4961800640 1 -- 172.21.15.7:0/375438062 wait complete. 2024-04-25T11:15:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.797+0000 7f4961800640 1 Processor -- start 2024-04-25T11:15:36.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.797+0000 7f4961800640 1 -- start start 2024-04-25T11:15:36.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.798+0000 7f4961800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f495c06edb0 0x7f495c0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.798+0000 7f4961800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f495c0d1b30 0x7f495c157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.798+0000 7f4961800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495c1616a0 0x7f495c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.798+0000 7f4961800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f495c0b4180 con 0x7f495c1616a0 2024-04-25T11:15:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.798+0000 7f4961800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f495c09bc50 con 0x7f495c06edb0 2024-04-25T11:15:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.798+0000 7f4961800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f495c16b370 con 0x7f495c0d1b30 2024-04-25T11:15:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.799+0000 7f495b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495c1616a0 0x7f495c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.799+0000 7f495b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495c1616a0 0x7f495c06d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53360/0 (socket says 172.21.15.7:53360) 2024-04-25T11:15:36.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.799+0000 7f495b400640 1 -- 172.21.15.7:0/4249697650 learned_addr learned my addr 172.21.15.7:0/4249697650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:36.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.799+0000 7f495aa00640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f495c06edb0 0x7f495c0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.799+0000 7f495a000640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f495c0d1b30 0x7f495c157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.799+0000 7f495b400640 1 -- 172.21.15.7:0/4249697650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f495c0d1b30 msgr2=0x7f495c157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.799+0000 7f495b400640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f495c0d1b30 0x7f495c157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.799+0000 7f495b400640 1 -- 172.21.15.7:0/4249697650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f495c06edb0 msgr2=0x7f495c0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.799+0000 7f495b400640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f495c06edb0 0x7f495c0ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.799+0000 7f495b400640 1 -- 172.21.15.7:0/4249697650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c0ce9d0 con 0x7f495c1616a0 2024-04-25T11:15:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.800+0000 7f495b400640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495c1616a0 0x7f495c06d5c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f495400caf0 tx=0x7f4954002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.800+0000 7f4952a00640 1 -- 172.21.15.7:0/4249697650 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495400abd0 con 0x7f495c1616a0 2024-04-25T11:15:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.800+0000 7f4952a00640 1 -- 172.21.15.7:0/4249697650 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f495400ad70 con 0x7f495c1616a0 2024-04-25T11:15:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.800+0000 7f4952a00640 1 -- 172.21.15.7:0/4249697650 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954012640 con 0x7f495c1616a0 2024-04-25T11:15:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.800+0000 7f4961800640 1 -- 172.21.15.7:0/4249697650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495c157e90 con 0x7f495c1616a0 2024-04-25T11:15:36.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.800+0000 7f4961800640 1 -- 172.21.15.7:0/4249697650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495c0cd6d0 con 0x7f495c1616a0 2024-04-25T11:15:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.801+0000 7f4961800640 1 -- 172.21.15.7:0/4249697650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495c050740 con 0x7f495c1616a0 2024-04-25T11:15:36.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.803+0000 7f4952a00640 1 -- 172.21.15.7:0/4249697650 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f49540127e0 con 0x7f495c1616a0 2024-04-25T11:15:36.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.804+0000 7f4952a00640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f493c0744b0 0x7f493c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:36.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.804+0000 7f4952a00640 1 -- 172.21.15.7:0/4249697650 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f49540959a0 con 0x7f495c1616a0 2024-04-25T11:15:36.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.804+0000 7f495aa00640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f493c0744b0 0x7f493c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:36.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.805+0000 7f495aa00640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f493c0744b0 0x7f493c0769c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f494c034000 tx=0x7f494c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:36.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:36.809+0000 7f4952a00640 1 -- 172.21.15.7:0/4249697650 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f495405ff90 con 0x7f495c1616a0 2024-04-25T11:15:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.009+0000 7f4961800640 1 -- 172.21.15.7:0/4249697650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f495c0b5510 con 0x7f495c1616a0 2024-04-25T11:15:37.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.009+0000 7f4952a00640 1 -- 172.21.15.7:0/4249697650 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f495405f890 con 0x7f495c1616a0 2024-04-25T11:15:37.010 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:15:37.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.011+0000 7f4961800640 1 -- 172.21.15.7:0/4249697650 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f493c0744b0 msgr2=0x7f493c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.011+0000 7f4961800640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f493c0744b0 0x7f493c0769c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f494c034000 tx=0x7f494c02e000 comp rx=0 tx=0).stop 2024-04-25T11:15:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.011+0000 7f4961800640 1 -- 172.21.15.7:0/4249697650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495c1616a0 msgr2=0x7f495c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.011+0000 7f4961800640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495c1616a0 0x7f495c06d5c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f495400caf0 tx=0x7f4954002190 comp rx=0 tx=0).stop 2024-04-25T11:15:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.012+0000 7f4961800640 1 -- 172.21.15.7:0/4249697650 shutdown_connections 2024-04-25T11:15:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.012+0000 7f4961800640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f493c0744b0 0x7f493c0769c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.012+0000 7f4961800640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495c1616a0 0x7f495c06d5c0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.012+0000 7f4961800640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f495c0d1b30 0x7f495c157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.012+0000 7f4961800640 1 --2- 172.21.15.7:0/4249697650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f495c06edb0 0x7f495c0ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.012+0000 7f4961800640 1 -- 172.21.15.7:0/4249697650 >> 172.21.15.7:0/4249697650 conn(0x7f495c0b7620 msgr2=0x7f495c0b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:37.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.012+0000 7f4961800640 1 -- 172.21.15.7:0/4249697650 shutdown_connections 2024-04-25T11:15:37.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.012+0000 7f4961800640 1 -- 172.21.15.7:0/4249697650 wait complete. 2024-04-25T11:15:37.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:15:37.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.305+0000 7f49f2600640 1 Processor -- start 2024-04-25T11:15:37.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.305+0000 7f49f2600640 1 -- start start 2024-04-25T11:15:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.307+0000 7f49f2600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f49ec0cfb30 0x7f49ec15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.307+0000 7f49f2600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec06d510 0x7f49ec0b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.307+0000 7f49f2600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49ec0cf1a0 con 0x7f49ec156be0 2024-04-25T11:15:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.307+0000 7f49f2600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f49ec070fc0 con 0x7f49ec0cfb30 2024-04-25T11:15:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.307+0000 7f49f2600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f49ec0b51e0 con 0x7f49ec06d510 2024-04-25T11:15:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.307+0000 7f49e3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f49ec156be0 0x7f49ec06ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60020/0 (socket says 172.21.15.7:60020) 2024-04-25T11:15:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.307+0000 7f49e3e00640 1 -- 172.21.15.7:0/150416972 learned_addr learned my addr 172.21.15.7:0/150416972 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:37.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.307+0000 7f49f1c00640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec06d510 0x7f49ec0b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:37.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.307+0000 7f49f1200640 1 --2- 172.21.15.7:0/150416972 >> v2:172.21.15.87:3300/0 conn(0x7f49ec0cfb30 0x7f49ec15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:37.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.307+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3293941139 0 0) 0x7f49d4001110 con 0x7f49ec156be0 2024-04-25T11:15:37.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49d0003520 con 0x7f49ec156be0 2024-04-25T11:15:37.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2516677600 0 0) 0x7f49d0003520 con 0x7f49ec156be0 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec06d510 msgr2=0x7f49ec0b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec06d510 0x7f49ec0b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 >> v2:172.21.15.87:3300/0 conn(0x7f49ec0cfb30 msgr2=0x7f49ec15d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 --2- 172.21.15.7:0/150416972 >> v2:172.21.15.87:3300/0 conn(0x7f49ec0cfb30 0x7f49ec15d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ec15d910 con 0x7f49ec156be0 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f49d4001840 con 0x7f49ec156be0 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 >> v1:172.21.15.7:6789/0 conn(0x7f49ec156be0 legacy=0x7f49ec06ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 0x7f49d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49d0004800 0x7f49d0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49d0005150 0x7f49d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec070fc0 con 0x7f49d00039b0 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec0b51e0 con 0x7f49d0005150 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.308+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f49d000bf00 con 0x7f49d0004800 2024-04-25T11:15:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.309+0000 7f49f1200640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 0x7f49d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.309+0000 7f49e3e00640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49d0004800 0x7f49d0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.309+0000 7f49f1200640 1 -- 172.21.15.7:0/150416972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49d0004800 msgr2=0x7f49d0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.309+0000 7f49f1200640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49d0004800 0x7f49d0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.309+0000 7f49f1200640 1 -- 172.21.15.7:0/150416972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49d0005150 msgr2=0x7f49d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.309+0000 7f49f1200640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49d0005150 0x7f49d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.309+0000 7f49f1200640 1 -- 172.21.15.7:0/150416972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d0003520 con 0x7f49d00039b0 2024-04-25T11:15:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.309+0000 7f49e3e00640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49d0004800 0x7f49d0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:37.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.310+0000 7f49f1200640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 0x7f49d0003e20 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f49ec072e80 tx=0x7f49d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a621df285afeb129 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:37.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.310+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc00b040 con 0x7f49d00039b0 2024-04-25T11:15:37.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.310+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49dc02d6d0 con 0x7f49d00039b0 2024-04-25T11:15:37.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.310+0000 7f49ebe00640 1 -- 172.21.15.7:0/150416972 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc02dac0 con 0x7f49d00039b0 2024-04-25T11:15:37.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.311+0000 7f49f2600640 1 -- 172.21.15.7:0/150416972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 msgr2=0x7f49d0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:37.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.311+0000 7f49f2600640 1 --2- 172.21.15.7:0/150416972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 0x7f49d0003e20 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f49ec072e80 tx=0x7f49d0005830 comp rx=0 tx=0).stop 2024-04-25T11:15:37.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.311+0000 7f49f1c00640 1 -- 172.21.15.7:0/150416972 reap_dead start 2024-04-25T11:15:37.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.311+0000 7f49f2600640 1 -- 172.21.15.7:0/150416972 shutdown_connections 2024-04-25T11:15:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.311+0000 7f49f2600640 1 -- 172.21.15.7:0/150416972 >> 172.21.15.7:0/150416972 conn(0x7f49ec0d1540 msgr2=0x7f49ec0d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.311+0000 7f49f2600640 1 -- 172.21.15.7:0/150416972 shutdown_connections 2024-04-25T11:15:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.311+0000 7f49f2600640 1 -- 172.21.15.7:0/150416972 wait complete. 2024-04-25T11:15:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.314+0000 7f49f2600640 1 Processor -- start 2024-04-25T11:15:37.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.314+0000 7f49f2600640 1 -- start start 2024-04-25T11:15:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.315+0000 7f49f2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 0x7f49ec1482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.315+0000 7f49f2600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49ec1521a0 0x7f49ec14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.315+0000 7f49f2600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec14b940 0x7f49ec14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.315+0000 7f49f2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec0b4eb0 con 0x7f49d00039b0 2024-04-25T11:15:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.315+0000 7f49f2600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec0b4c10 con 0x7f49ec1521a0 2024-04-25T11:15:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.315+0000 7f49f2600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f49ec0cfff0 con 0x7f49ec14b940 2024-04-25T11:15:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.315+0000 7f49f1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 0x7f49ec1482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.316+0000 7f49f1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 0x7f49ec1482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53376/0 (socket says 172.21.15.7:53376) 2024-04-25T11:15:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.316+0000 7f49f1200640 1 -- 172.21.15.7:0/3846601479 learned_addr learned my addr 172.21.15.7:0/3846601479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.316+0000 7f49e3e00640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49ec1521a0 0x7f49ec14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.316+0000 7f49f1200640 1 -- 172.21.15.7:0/3846601479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec14b940 msgr2=0x7f49ec14f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.316+0000 7f49f1200640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec14b940 0x7f49ec14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.316+0000 7f49f1200640 1 -- 172.21.15.7:0/3846601479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49ec1521a0 msgr2=0x7f49ec14f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.316+0000 7f49f1200640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49ec1521a0 0x7f49ec14f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.316+0000 7f49f1200640 1 -- 172.21.15.7:0/3846601479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ec1f3bd0 con 0x7f49d00039b0 2024-04-25T11:15:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.316+0000 7f49f1200640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 0x7f49ec1482f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f49dc037040 tx=0x7f49dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.317+0000 7f49e9600640 1 -- 172.21.15.7:0/3846601479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc042070 con 0x7f49d00039b0 2024-04-25T11:15:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.317+0000 7f49e9600640 1 -- 172.21.15.7:0/3846601479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49dc02de40 con 0x7f49d00039b0 2024-04-25T11:15:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.317+0000 7f49e9600640 1 -- 172.21.15.7:0/3846601479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc03e630 con 0x7f49d00039b0 2024-04-25T11:15:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.317+0000 7f49f2600640 1 -- 172.21.15.7:0/3846601479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ec150850 con 0x7f49d00039b0 2024-04-25T11:15:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.317+0000 7f49f2600640 1 -- 172.21.15.7:0/3846601479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ec150e10 con 0x7f49d00039b0 2024-04-25T11:15:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.318+0000 7f49f2600640 1 -- 172.21.15.7:0/3846601479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ec050740 con 0x7f49d00039b0 2024-04-25T11:15:37.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.319+0000 7f49e9600640 1 -- 172.21.15.7:0/3846601479 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f49dc045070 con 0x7f49d00039b0 2024-04-25T11:15:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.320+0000 7f49e9600640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49c80744b0 0x7f49c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.321+0000 7f49e9600640 1 -- 172.21.15.7:0/3846601479 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f49dc0b5fa0 con 0x7f49d00039b0 2024-04-25T11:15:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.321+0000 7f49e3e00640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49c80744b0 0x7f49c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.321+0000 7f49e3e00640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49c80744b0 0x7f49c80769c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f49d40012f0 tx=0x7f49d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:37.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.326+0000 7f49e9600640 1 -- 172.21.15.7:0/3846601479 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49dc04c070 con 0x7f49d00039b0 2024-04-25T11:15:37.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.545+0000 7f49f2600640 1 -- 172.21.15.7:0/3846601479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f49ec0b6260 con 0x7f49d00039b0 2024-04-25T11:15:37.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.546+0000 7f49e9600640 1 -- 172.21.15.7:0/3846601479 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) ==== 80+0+0 (secure 0 0 0) 0x7f49dc00a020 con 0x7f49d00039b0 2024-04-25T11:15:37.546 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:15:37.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.548+0000 7f49f2600640 1 -- 172.21.15.7:0/3846601479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49c80744b0 msgr2=0x7f49c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:37.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.548+0000 7f49f2600640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49c80744b0 0x7f49c80769c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f49d40012f0 tx=0x7f49d4009040 comp rx=0 tx=0).stop 2024-04-25T11:15:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.548+0000 7f49f2600640 1 -- 172.21.15.7:0/3846601479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 msgr2=0x7f49ec1482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.548+0000 7f49f2600640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 0x7f49ec1482f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f49dc037040 tx=0x7f49dc004730 comp rx=0 tx=0).stop 2024-04-25T11:15:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.548+0000 7f49f2600640 1 -- 172.21.15.7:0/3846601479 shutdown_connections 2024-04-25T11:15:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.548+0000 7f49f2600640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49c80744b0 0x7f49c80769c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.548+0000 7f49f2600640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec14b940 0x7f49ec14f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.548+0000 7f49f2600640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49ec1521a0 0x7f49ec14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.548+0000 7f49f2600640 1 --2- 172.21.15.7:0/3846601479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49d00039b0 0x7f49ec1482f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.548+0000 7f49f2600640 1 -- 172.21.15.7:0/3846601479 >> 172.21.15.7:0/3846601479 conn(0x7f49ec0d1540 msgr2=0x7f49ec0b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.549+0000 7f49f2600640 1 -- 172.21.15.7:0/3846601479 shutdown_connections 2024-04-25T11:15:37.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:37.549+0000 7f49f2600640 1 -- 172.21.15.7:0/3846601479 wait complete. 2024-04-25T11:15:37.577 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:15:37.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:37.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:15:37.796 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:15:37.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:37.796 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:15:37.835 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:15:37.835 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:15:37.835 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:37.880 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:15:37.880 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:37.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:15:38.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.253+0000 7f0266800640 1 Processor -- start 2024-04-25T11:15:38.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.253+0000 7f0266800640 1 -- start start 2024-04-25T11:15:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.255+0000 7f0266800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02600af540 0x7f026009f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.255+0000 7f0266800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f026015f4e0 con 0x7f026015bc40 2024-04-25T11:15:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.255+0000 7f0266800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02600b0560 con 0x7f026015ef90 2024-04-25T11:15:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.255+0000 7f0266800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0260071190 con 0x7f02600af540 2024-04-25T11:15:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.256+0000 7f025fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f026015bc40 0x7f02600cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60022/0 (socket says 172.21.15.7:60022) 2024-04-25T11:15:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.256+0000 7f025fe00640 1 -- 172.21.15.7:0/2686247006 learned_addr learned my addr 172.21.15.7:0/2686247006 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:38.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.256+0000 7f025ea00640 1 --2- 172.21.15.7:0/2686247006 >> v2:172.21.15.7:3300/0 conn(0x7f02600af540 0x7f026009f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:38.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.256+0000 7f025e000640 1 -- 172.21.15.7:0/2686247006 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3223002266 0 0) 0x7f026015f4e0 con 0x7f026015bc40 2024-04-25T11:15:38.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.256+0000 7f025e000640 1 -- 172.21.15.7:0/2686247006 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0240003520 con 0x7f026015bc40 2024-04-25T11:15:38.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.256+0000 7f025ea00640 1 -- 172.21.15.7:0/2686247006 >> v1:172.21.15.87:6789/0 conn(0x7f026015ef90 legacy=0x7f02600aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:38.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.256+0000 7f025ea00640 1 -- 172.21.15.7:0/2686247006 >> v1:172.21.15.7:6789/0 conn(0x7f026015bc40 legacy=0x7f02600cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:38.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.257+0000 7f025ea00640 1 -- 172.21.15.7:0/2686247006 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02600b0840 con 0x7f02600af540 2024-04-25T11:15:38.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.258+0000 7f025ea00640 1 --2- 172.21.15.7:0/2686247006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02600af540 0x7f026009f790 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0248009d90 tx=0x7f024802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b258029e96caa171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:38.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.258+0000 7f025e000640 1 -- 172.21.15.7:0/2686247006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248007740 con 0x7f02600af540 2024-04-25T11:15:38.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.258+0000 7f025e000640 1 -- 172.21.15.7:0/2686247006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02480078e0 con 0x7f02600af540 2024-04-25T11:15:38.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.258+0000 7f025e000640 1 -- 172.21.15.7:0/2686247006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248007c90 con 0x7f02600af540 2024-04-25T11:15:38.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.259+0000 7f0266800640 1 -- 172.21.15.7:0/2686247006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02600af540 msgr2=0x7f026009f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:38.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.259+0000 7f0266800640 1 --2- 172.21.15.7:0/2686247006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02600af540 0x7f026009f790 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f0248009d90 tx=0x7f024802a660 comp rx=0 tx=0).stop 2024-04-25T11:15:38.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.259+0000 7f0266800640 1 -- 172.21.15.7:0/2686247006 shutdown_connections 2024-04-25T11:15:38.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.259+0000 7f0266800640 1 --2- 172.21.15.7:0/2686247006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02600af540 0x7f026009f790 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.259+0000 7f0266800640 1 -- 172.21.15.7:0/2686247006 >> 172.21.15.7:0/2686247006 conn(0x7f0260074200 msgr2=0x7f026006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:38.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.259+0000 7f0266800640 1 -- 172.21.15.7:0/2686247006 shutdown_connections 2024-04-25T11:15:38.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.259+0000 7f0266800640 1 -- 172.21.15.7:0/2686247006 wait complete. 2024-04-25T11:15:38.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.261+0000 7f0266800640 1 Processor -- start 2024-04-25T11:15:38.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.262+0000 7f0266800640 1 -- start start 2024-04-25T11:15:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.263+0000 7f0266800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f026015bc40 0x7f02600b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.263+0000 7f0266800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f026015ef90 0x7f026014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.263+0000 7f0266800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f026015d130 0x7f026015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.263+0000 7f0266800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02600b1320 con 0x7f026015ef90 2024-04-25T11:15:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.263+0000 7f0266800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f02600b0e00 con 0x7f026015d130 2024-04-25T11:15:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.263+0000 7f0266800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f02600afb80 con 0x7f026015bc40 2024-04-25T11:15:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.263+0000 7f025ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f026015ef90 0x7f026014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.263+0000 7f025ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f026015ef90 0x7f026014e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53390/0 (socket says 172.21.15.7:53390) 2024-04-25T11:15:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.263+0000 7f025ea00640 1 -- 172.21.15.7:0/2426642620 learned_addr learned my addr 172.21.15.7:0/2426642620 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:38.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.264+0000 7f025fe00640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f026015d130 0x7f026015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.264+0000 7f025f400640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f026015bc40 0x7f02600b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.264+0000 7f025ea00640 1 -- 172.21.15.7:0/2426642620 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f026015bc40 msgr2=0x7f02600b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.264+0000 7f025ea00640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f026015bc40 0x7f02600b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.264+0000 7f025ea00640 1 -- 172.21.15.7:0/2426642620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f026015d130 msgr2=0x7f026015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.264+0000 7f025ea00640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f026015d130 0x7f026015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.264+0000 7f025ea00640 1 -- 172.21.15.7:0/2426642620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026015da80 con 0x7f026015ef90 2024-04-25T11:15:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.264+0000 7f025ea00640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f026015ef90 0x7f026014e0b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f0248009d90 tx=0x7f0248004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.265+0000 7f0257400640 1 -- 172.21.15.7:0/2426642620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024803f070 con 0x7f026015ef90 2024-04-25T11:15:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.265+0000 7f0257400640 1 -- 172.21.15.7:0/2426642620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0248031c40 con 0x7f026015ef90 2024-04-25T11:15:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.265+0000 7f0266800640 1 -- 172.21.15.7:0/2426642620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02600b6540 con 0x7f026015ef90 2024-04-25T11:15:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.265+0000 7f0257400640 1 -- 172.21.15.7:0/2426642620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248009450 con 0x7f026015ef90 2024-04-25T11:15:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.265+0000 7f0266800640 1 -- 172.21.15.7:0/2426642620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02601e7940 con 0x7f026015ef90 2024-04-25T11:15:38.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.266+0000 7f0266800640 1 -- 172.21.15.7:0/2426642620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02600b62b0 con 0x7f026015ef90 2024-04-25T11:15:38.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.267+0000 7f0257400640 1 -- 172.21.15.7:0/2426642620 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0248003990 con 0x7f026015ef90 2024-04-25T11:15:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.268+0000 7f0257400640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f023c0744b0 0x7f023c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.268+0000 7f0257400640 1 -- 172.21.15.7:0/2426642620 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f02480b6560 con 0x7f026015ef90 2024-04-25T11:15:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.269+0000 7f025f400640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f023c0744b0 0x7f023c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:38.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.270+0000 7f025f400640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f023c0744b0 0x7f023c0769c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0250007fd0 tx=0x7f0250009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:38.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.274+0000 7f0257400640 1 -- 172.21.15.7:0/2426642620 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f026015da80 con 0x7f026015ef90 2024-04-25T11:15:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.470+0000 7f0266800640 1 -- 172.21.15.7:0/2426642620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f026006a100 con 0x7f026015ef90 2024-04-25T11:15:38.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.473+0000 7f0257400640 1 -- 172.21.15.7:0/2426642620 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16849 (secure 0 0 0) 0x7f0248080ac0 con 0x7f026015ef90 2024-04-25T11:15:38.473 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:38.474 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:15:06.697374+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:03.587954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T11:15:04.226794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:15:38.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.475+0000 7f0266800640 1 -- 172.21.15.7:0/2426642620 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f023c0744b0 msgr2=0x7f023c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:38.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.475+0000 7f0266800640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f023c0744b0 0x7f023c0769c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0250007fd0 tx=0x7f0250009040 comp rx=0 tx=0).stop 2024-04-25T11:15:38.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.475+0000 7f0266800640 1 -- 172.21.15.7:0/2426642620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f026015ef90 msgr2=0x7f026014e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:38.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.475+0000 7f0266800640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f026015ef90 0x7f026014e0b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f0248009d90 tx=0x7f0248004a40 comp rx=0 tx=0).stop 2024-04-25T11:15:38.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.476+0000 7f0266800640 1 -- 172.21.15.7:0/2426642620 shutdown_connections 2024-04-25T11:15:38.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.476+0000 7f0266800640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f023c0744b0 0x7f023c0769c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.476+0000 7f0266800640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f026015d130 0x7f026015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.476+0000 7f0266800640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f026015ef90 0x7f026014e0b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.476+0000 7f0266800640 1 --2- 172.21.15.7:0/2426642620 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f026015bc40 0x7f02600b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.476+0000 7f0266800640 1 -- 172.21.15.7:0/2426642620 >> 172.21.15.7:0/2426642620 conn(0x7f0260074200 msgr2=0x7f02600d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:38.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.476+0000 7f0266800640 1 -- 172.21.15.7:0/2426642620 shutdown_connections 2024-04-25T11:15:38.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.476+0000 7f0266800640 1 -- 172.21.15.7:0/2426642620 wait complete. 2024-04-25T11:15:38.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:15:03.587954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:15:04.226794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:15:38.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:15:38.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.771+0000 7fecda600640 1 Processor -- start 2024-04-25T11:15:38.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.772+0000 7fecda600640 1 -- start start 2024-04-25T11:15:38.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.774+0000 7fecda600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fecd415dfb0 0x7fecd40b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.774+0000 7fecda600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecd40d0b40 0x7fecd406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.774+0000 7fecda600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecd40d0f50 0x7fecd406f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.774+0000 7fecda600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fecd40b09b0 con 0x7fecd40d0f50 2024-04-25T11:15:38.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.774+0000 7fecda600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fecd415e3c0 con 0x7fecd415dfb0 2024-04-25T11:15:38.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.774+0000 7fecda600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fecd40cf800 con 0x7fecd40d0b40 2024-04-25T11:15:38.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.774+0000 7fecd3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecd40d0f50 0x7fecd406f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:38.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.775+0000 7fecd3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fecd40d0f50 0x7fecd406f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53406/0 (socket says 172.21.15.7:53406) 2024-04-25T11:15:38.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.775+0000 7fecd3e00640 1 -- 172.21.15.7:0/3109283479 learned_addr learned my addr 172.21.15.7:0/3109283479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:38.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.775+0000 7feccbe00640 1 --2- 172.21.15.7:0/3109283479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecd40d0b40 0x7fecd406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:38.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.775+0000 7fecd3400640 1 --2- 172.21.15.7:0/3109283479 >> v2:172.21.15.87:3300/0 conn(0x7fecd415dfb0 0x7fecd40b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:38.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.775+0000 7fecd3e00640 1 -- 172.21.15.7:0/3109283479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecd40d0b40 msgr2=0x7fecd406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:38.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.775+0000 7fecd3e00640 1 --2- 172.21.15.7:0/3109283479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecd40d0b40 0x7fecd406f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.775+0000 7fecd3e00640 1 -- 172.21.15.7:0/3109283479 >> v2:172.21.15.87:3300/0 conn(0x7fecd415dfb0 msgr2=0x7fecd40b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:38.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.775+0000 7fecd3e00640 1 --2- 172.21.15.7:0/3109283479 >> v2:172.21.15.87:3300/0 conn(0x7fecd415dfb0 0x7fecd40b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.775+0000 7fecd3e00640 1 -- 172.21.15.7:0/3109283479 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd40cfaa0 con 0x7fecd40d0f50 2024-04-25T11:15:38.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.776+0000 7fecd3e00640 1 --2- 172.21.15.7:0/3109283479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd40d0f50 0x7fecd406f8e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fecd4072b10 tx=0x7feccc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677589dd0d87a340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:38.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.776+0000 7fecd2a00640 1 -- 172.21.15.7:0/3109283479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc036070 con 0x7fecd40d0f50 2024-04-25T11:15:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.776+0000 7fecd2a00640 1 -- 172.21.15.7:0/3109283479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feccc031600 con 0x7fecd40d0f50 2024-04-25T11:15:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.776+0000 7fecd2a00640 1 -- 172.21.15.7:0/3109283479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feccc0319d0 con 0x7fecd40d0f50 2024-04-25T11:15:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.777+0000 7fecda600640 1 -- 172.21.15.7:0/3109283479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd40d0f50 msgr2=0x7fecd406f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.777+0000 7fecda600640 1 --2- 172.21.15.7:0/3109283479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd40d0f50 0x7fecd406f8e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fecd4072b10 tx=0x7feccc02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.777+0000 7fecda600640 1 -- 172.21.15.7:0/3109283479 shutdown_connections 2024-04-25T11:15:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.777+0000 7fecda600640 1 --2- 172.21.15.7:0/3109283479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd40d0f50 0x7fecd406f8e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.777+0000 7fecda600640 1 --2- 172.21.15.7:0/3109283479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecd40d0b40 0x7fecd406f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.777+0000 7fecda600640 1 --2- 172.21.15.7:0/3109283479 >> v2:172.21.15.87:3300/0 conn(0x7fecd415dfb0 0x7fecd40b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.777+0000 7fecda600640 1 -- 172.21.15.7:0/3109283479 >> 172.21.15.7:0/3109283479 conn(0x7fecd40742c0 msgr2=0x7fecd406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.777+0000 7fecda600640 1 -- 172.21.15.7:0/3109283479 shutdown_connections 2024-04-25T11:15:38.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.777+0000 7fecda600640 1 -- 172.21.15.7:0/3109283479 wait complete. 2024-04-25T11:15:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.779+0000 7fecda600640 1 Processor -- start 2024-04-25T11:15:38.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.780+0000 7fecda600640 1 -- start start 2024-04-25T11:15:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.781+0000 7fecda600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd40d0b40 0x7fecd4152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.781+0000 7fecda600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecd40d0f50 0x7fecd40b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.781+0000 7fecda600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecd415dfb0 0x7fecd415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.781+0000 7fecda600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fecd40d0340 con 0x7fecd40d0b40 2024-04-25T11:15:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.781+0000 7fecda600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fecd40d0060 con 0x7fecd40d0f50 2024-04-25T11:15:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.781+0000 7fecda600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fecd40d0620 con 0x7fecd415dfb0 2024-04-25T11:15:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.781+0000 7fecd3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd40d0b40 0x7fecd4152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.781+0000 7fecd3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd40d0b40 0x7fecd4152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53408/0 (socket says 172.21.15.7:53408) 2024-04-25T11:15:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.781+0000 7fecd3400640 1 -- 172.21.15.7:0/3388061033 learned_addr learned my addr 172.21.15.7:0/3388061033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.782+0000 7fecd3400640 1 -- 172.21.15.7:0/3388061033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecd415dfb0 msgr2=0x7fecd415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.782+0000 7fecd3400640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecd415dfb0 0x7fecd415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.782+0000 7feccbe00640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecd40d0f50 0x7fecd40b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.782+0000 7fecd3400640 1 -- 172.21.15.7:0/3388061033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecd40d0f50 msgr2=0x7fecd40b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.782+0000 7fecd3400640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecd40d0f50 0x7fecd40b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.782+0000 7fecd3400640 1 -- 172.21.15.7:0/3388061033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd415e820 con 0x7fecd40d0b40 2024-04-25T11:15:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.782+0000 7fecd3400640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd40d0b40 0x7fecd4152560 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fecc400c820 tx=0x7fecc400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.782+0000 7feccb400640 1 -- 172.21.15.7:0/3388061033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4004030 con 0x7fecd40d0b40 2024-04-25T11:15:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.782+0000 7feccb400640 1 -- 172.21.15.7:0/3388061033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc40041d0 con 0x7fecd40d0b40 2024-04-25T11:15:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.782+0000 7feccb400640 1 -- 172.21.15.7:0/3388061033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4007830 con 0x7fecd40d0b40 2024-04-25T11:15:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.783+0000 7fecda600640 1 -- 172.21.15.7:0/3388061033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd415eb00 con 0x7fecd40d0b40 2024-04-25T11:15:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.783+0000 7fecda600640 1 -- 172.21.15.7:0/3388061033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd415f0c0 con 0x7fecd40d0b40 2024-04-25T11:15:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.784+0000 7fecda600640 1 -- 172.21.15.7:0/3388061033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd4050740 con 0x7fecd40d0b40 2024-04-25T11:15:38.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.785+0000 7feccb400640 1 -- 172.21.15.7:0/3388061033 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fecc40079d0 con 0x7fecd40d0b40 2024-04-25T11:15:38.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.786+0000 7feccb400640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fecb00744e0 0x7fecb00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:38.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.786+0000 7feccb400640 1 -- 172.21.15.7:0/3388061033 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fecc40952d0 con 0x7fecd40d0b40 2024-04-25T11:15:38.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.786+0000 7feccbe00640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fecb00744e0 0x7fecb00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:38.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.787+0000 7feccbe00640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fecb00744e0 0x7fecb00769f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fecd40ce790 tx=0x7fecbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:38.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.791+0000 7feccb400640 1 -- 172.21.15.7:0/3388061033 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fecc4015020 con 0x7fecd40d0b40 2024-04-25T11:15:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.998+0000 7fecda600640 1 -- 172.21.15.7:0/3388061033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fecd40b3380 con 0x7fecd40d0b40 2024-04-25T11:15:38.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:38.999+0000 7feccb400640 1 -- 172.21.15.7:0/3388061033 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fecc405f8c0 con 0x7fecd40d0b40 2024-04-25T11:15:38.999 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:15:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.001+0000 7fecda600640 1 -- 172.21.15.7:0/3388061033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fecb00744e0 msgr2=0x7fecb00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.001+0000 7fecda600640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fecb00744e0 0x7fecb00769f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fecd40ce790 tx=0x7fecbc008040 comp rx=0 tx=0).stop 2024-04-25T11:15:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.001+0000 7fecda600640 1 -- 172.21.15.7:0/3388061033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd40d0b40 msgr2=0x7fecd4152560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.001+0000 7fecda600640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd40d0b40 0x7fecd4152560 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fecc400c820 tx=0x7fecc400cce0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.001+0000 7fecda600640 1 -- 172.21.15.7:0/3388061033 shutdown_connections 2024-04-25T11:15:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.001+0000 7fecda600640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fecb00744e0 0x7fecb00769f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.001+0000 7fecda600640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecd415dfb0 0x7fecd415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.002+0000 7fecda600640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecd40d0f50 0x7fecd40b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.002+0000 7fecda600640 1 --2- 172.21.15.7:0/3388061033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fecd40d0b40 0x7fecd4152560 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.002+0000 7fecda600640 1 -- 172.21.15.7:0/3388061033 >> 172.21.15.7:0/3388061033 conn(0x7fecd40742c0 msgr2=0x7fecd40ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.002+0000 7fecda600640 1 -- 172.21.15.7:0/3388061033 shutdown_connections 2024-04-25T11:15:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.002+0000 7fecda600640 1 -- 172.21.15.7:0/3388061033 wait complete. 2024-04-25T11:15:39.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:15:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.312+0000 7efde7e00640 1 Processor -- start 2024-04-25T11:15:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.313+0000 7efde7e00640 1 -- start start 2024-04-25T11:15:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.315+0000 7efde7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efde81583c0 0x7efde80cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.315+0000 7efde7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efde80cfb50 0x7efde80d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.315+0000 7efde7e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde80ce570 con 0x7efde80d09c0 2024-04-25T11:15:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.315+0000 7efde7e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efde815f190 con 0x7efde80cfb50 2024-04-25T11:15:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.315+0000 7efde7e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efde80ce850 con 0x7efde81583c0 2024-04-25T11:15:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.315+0000 7efddfe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efde80cfb50 0x7efde80d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.315+0000 7efde7400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efde80d09c0 0x7efde806eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60026/0 (socket says 172.21.15.7:60026) 2024-04-25T11:15:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.315+0000 7efde7400640 1 -- 172.21.15.7:0/1004747665 learned_addr learned my addr 172.21.15.7:0/1004747665 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.316+0000 7efde6a00640 1 --2- 172.21.15.7:0/1004747665 >> v2:172.21.15.87:3300/0 conn(0x7efde81583c0 0x7efde80cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.316+0000 7efddfe00640 1 -- 172.21.15.7:0/1004747665 >> v2:172.21.15.87:3300/0 conn(0x7efde81583c0 msgr2=0x7efde80cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.316+0000 7efddfe00640 1 --2- 172.21.15.7:0/1004747665 >> v2:172.21.15.87:3300/0 conn(0x7efde81583c0 0x7efde80cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.316+0000 7efddfe00640 1 -- 172.21.15.7:0/1004747665 >> v1:172.21.15.7:6789/0 conn(0x7efde80d09c0 legacy=0x7efde806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.316+0000 7efde6000640 1 -- 172.21.15.7:0/1004747665 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2562197448 0 0) 0x7efde80ce570 con 0x7efde80d09c0 2024-04-25T11:15:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.316+0000 7efddfe00640 1 -- 172.21.15.7:0/1004747665 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde80ceb30 con 0x7efde80cfb50 2024-04-25T11:15:39.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.317+0000 7efddfe00640 1 --2- 172.21.15.7:0/1004747665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde80cfb50 0x7efde80d0480 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7efde8072330 tx=0x7efdd002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f26fb6193e84259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.317+0000 7efde6000640 1 -- 172.21.15.7:0/1004747665 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0037070 con 0x7efde80cfb50 2024-04-25T11:15:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.317+0000 7efde6000640 1 -- 172.21.15.7:0/1004747665 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd0032440 con 0x7efde80cfb50 2024-04-25T11:15:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.317+0000 7efde6000640 1 -- 172.21.15.7:0/1004747665 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd002ae60 con 0x7efde80cfb50 2024-04-25T11:15:39.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.318+0000 7efde7e00640 1 -- 172.21.15.7:0/1004747665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde80cfb50 msgr2=0x7efde80d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.318+0000 7efde7e00640 1 --2- 172.21.15.7:0/1004747665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde80cfb50 0x7efde80d0480 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7efde8072330 tx=0x7efdd002a110 comp rx=0 tx=0).stop 2024-04-25T11:15:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.318+0000 7efde7e00640 1 -- 172.21.15.7:0/1004747665 shutdown_connections 2024-04-25T11:15:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.318+0000 7efde7e00640 1 --2- 172.21.15.7:0/1004747665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde80cfb50 0x7efde80d0480 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.318+0000 7efde7e00640 1 --2- 172.21.15.7:0/1004747665 >> v2:172.21.15.87:3300/0 conn(0x7efde81583c0 0x7efde80cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.318+0000 7efde7e00640 1 -- 172.21.15.7:0/1004747665 >> 172.21.15.7:0/1004747665 conn(0x7efde80740f0 msgr2=0x7efde806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.318+0000 7efde7e00640 1 -- 172.21.15.7:0/1004747665 shutdown_connections 2024-04-25T11:15:39.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.318+0000 7efde7e00640 1 -- 172.21.15.7:0/1004747665 wait complete. 2024-04-25T11:15:39.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.320+0000 7efde7e00640 1 Processor -- start 2024-04-25T11:15:39.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.321+0000 7efde7e00640 1 -- start start 2024-04-25T11:15:39.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.322+0000 7efde7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efde80cfb50 0x7efde81f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.322+0000 7efde7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d09c0 0x7efde815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.322+0000 7efde7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde81583c0 0x7efde8143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.322+0000 7efde7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efde806f5b0 con 0x7efde81583c0 2024-04-25T11:15:39.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.322+0000 7efde7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efde80cf0f0 con 0x7efde80cfb50 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.322+0000 7efde7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efde806f730 con 0x7efde80d09c0 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.322+0000 7efde7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde81583c0 0x7efde8143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.323+0000 7efde7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde81583c0 0x7efde8143a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53422/0 (socket says 172.21.15.7:53422) 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.323+0000 7efde7400640 1 -- 172.21.15.7:0/967882251 learned_addr learned my addr 172.21.15.7:0/967882251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.323+0000 7efde6a00640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efde80cfb50 0x7efde81f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.323+0000 7efddfe00640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d09c0 0x7efde815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.323+0000 7efde7400640 1 -- 172.21.15.7:0/967882251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d09c0 msgr2=0x7efde815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.323+0000 7efde7400640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d09c0 0x7efde815cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.323+0000 7efde7400640 1 -- 172.21.15.7:0/967882251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efde80cfb50 msgr2=0x7efde81f98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.323+0000 7efde7400640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efde80cfb50 0x7efde81f98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.323+0000 7efde7400640 1 -- 172.21.15.7:0/967882251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde81fa780 con 0x7efde81583c0 2024-04-25T11:15:39.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.323+0000 7efde7400640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde81583c0 0x7efde8143a50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7efde000c700 tx=0x7efde000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:39.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.324+0000 7efddf400640 1 -- 172.21.15.7:0/967882251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0018070 con 0x7efde81583c0 2024-04-25T11:15:39.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.324+0000 7efddf400640 1 -- 172.21.15.7:0/967882251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde000ce10 con 0x7efde81583c0 2024-04-25T11:15:39.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.324+0000 7efddf400640 1 -- 172.21.15.7:0/967882251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde000a880 con 0x7efde81583c0 2024-04-25T11:15:39.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.324+0000 7efde7e00640 1 -- 172.21.15.7:0/967882251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde81fa1c0 con 0x7efde81583c0 2024-04-25T11:15:39.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.324+0000 7efde7e00640 1 -- 172.21.15.7:0/967882251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde81e76b0 con 0x7efde81583c0 2024-04-25T11:15:39.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.325+0000 7efde7e00640 1 -- 172.21.15.7:0/967882251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde81f9f30 con 0x7efde81583c0 2024-04-25T11:15:39.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.326+0000 7efddf400640 1 -- 172.21.15.7:0/967882251 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efde000aa20 con 0x7efde81583c0 2024-04-25T11:15:39.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.327+0000 7efddf400640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 0x7efdc40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.327+0000 7efddf400640 1 -- 172.21.15.7:0/967882251 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7efde0094a10 con 0x7efde81583c0 2024-04-25T11:15:39.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.327+0000 7efde6a00640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 0x7efdc40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.328+0000 7efde6a00640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 0x7efdc40769c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efdd8006fd0 tx=0x7efdd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:39.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.332+0000 7efddf400640 1 -- 172.21.15.7:0/967882251 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efde005ef70 con 0x7efde81583c0 2024-04-25T11:15:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.526+0000 7efde7e00640 1 -- 172.21.15.7:0/967882251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efde80d07e0 con 0x7efde81583c0 2024-04-25T11:15:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.528+0000 7efddf400640 1 -- 172.21.15.7:0/967882251 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7efde80d07e0 con 0x7efde81583c0 2024-04-25T11:15:39.528 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.530+0000 7efde7e00640 1 -- 172.21.15.7:0/967882251 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 msgr2=0x7efdc40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.530+0000 7efde7e00640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 0x7efdc40769c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7efdd8006fd0 tx=0x7efdd8008040 comp rx=0 tx=0).stop 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.530+0000 7efde7e00640 1 -- 172.21.15.7:0/967882251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde81583c0 msgr2=0x7efde8143a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.530+0000 7efde7e00640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde81583c0 0x7efde8143a50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7efde000c700 tx=0x7efde000cbc0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.530+0000 7efde7e00640 1 -- 172.21.15.7:0/967882251 shutdown_connections 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.530+0000 7efde7e00640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 0x7efdc40769c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.530+0000 7efde7e00640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde81583c0 0x7efde8143a50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.530+0000 7efde7e00640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d09c0 0x7efde815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.530+0000 7efde7e00640 1 --2- 172.21.15.7:0/967882251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efde80cfb50 0x7efde81f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.531+0000 7efde7e00640 1 -- 172.21.15.7:0/967882251 >> 172.21.15.7:0/967882251 conn(0x7efde80740f0 msgr2=0x7efde806eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.531+0000 7efde7e00640 1 -- 172.21.15.7:0/967882251 shutdown_connections 2024-04-25T11:15:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.531+0000 7efde7e00640 1 -- 172.21.15.7:0/967882251 wait complete. 2024-04-25T11:15:39.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:15:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.834+0000 7f6c39000640 1 Processor -- start 2024-04-25T11:15:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.834+0000 7f6c39000640 1 -- start start 2024-04-25T11:15:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.837+0000 7f6c39000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6c340cfa80 0x7f6c340d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.837+0000 7f6c39000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c340d0800 0x7f6c3406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.837+0000 7f6c39000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c340d0c10 con 0x7f6c34070630 2024-04-25T11:15:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.837+0000 7f6c39000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6c340b0be0 con 0x7f6c340cfa80 2024-04-25T11:15:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.837+0000 7f6c39000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6c340cfe90 con 0x7f6c340d0800 2024-04-25T11:15:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.838+0000 7f6c32a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6c340cfa80 0x7f6c340d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.838+0000 7f6c33e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c340d0800 0x7f6c3406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.838+0000 7f6c32a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6c340cfa80 0x7f6c340d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:35388/0 (socket says 172.21.15.7:35388) 2024-04-25T11:15:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.838+0000 7f6c32a00640 1 -- 172.21.15.7:0/3582432645 learned_addr learned my addr 172.21.15.7:0/3582432645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.838+0000 7f6c33e00640 1 --2- 172.21.15.7:0/3582432645 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c340d0800 0x7f6c3406eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:49204/0 (socket says 172.21.15.7:49204) 2024-04-25T11:15:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.839+0000 7f6c32a00640 1 -- 172.21.15.7:0/3582432645 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c340d0800 msgr2=0x7f6c3406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.839+0000 7f6c32a00640 1 --2- 172.21.15.7:0/3582432645 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c340d0800 0x7f6c3406eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.839+0000 7f6c32a00640 1 -- 172.21.15.7:0/3582432645 >> v1:172.21.15.87:6789/0 conn(0x7f6c34070630 legacy=0x7f6c3415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.839+0000 7f6c32a00640 1 -- 172.21.15.7:0/3582432645 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c340ce9e0 con 0x7f6c340cfa80 2024-04-25T11:15:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.840+0000 7f6c32a00640 1 --2- 172.21.15.7:0/3582432645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c340cfa80 0x7f6c340d02c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009870 tx=0x7f6c2002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5803ca4161995c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.840+0000 7f6c32000640 1 -- 172.21.15.7:0/3582432645 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20037070 con 0x7f6c340cfa80 2024-04-25T11:15:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.841+0000 7f6c32000640 1 -- 172.21.15.7:0/3582432645 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c20032440 con 0x7f6c340cfa80 2024-04-25T11:15:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.841+0000 7f6c32000640 1 -- 172.21.15.7:0/3582432645 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20032810 con 0x7f6c340cfa80 2024-04-25T11:15:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.841+0000 7f6c39000640 1 -- 172.21.15.7:0/3582432645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c340cfa80 msgr2=0x7f6c340d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.841+0000 7f6c39000640 1 --2- 172.21.15.7:0/3582432645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c340cfa80 0x7f6c340d02c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009870 tx=0x7f6c2002a250 comp rx=0 tx=0).stop 2024-04-25T11:15:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.841+0000 7f6c39000640 1 -- 172.21.15.7:0/3582432645 shutdown_connections 2024-04-25T11:15:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.841+0000 7f6c39000640 1 --2- 172.21.15.7:0/3582432645 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c340d0800 0x7f6c3406eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.841+0000 7f6c39000640 1 --2- 172.21.15.7:0/3582432645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c340cfa80 0x7f6c340d02c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.841+0000 7f6c39000640 1 -- 172.21.15.7:0/3582432645 >> 172.21.15.7:0/3582432645 conn(0x7f6c34074140 msgr2=0x7f6c3406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.842+0000 7f6c39000640 1 -- 172.21.15.7:0/3582432645 shutdown_connections 2024-04-25T11:15:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.842+0000 7f6c39000640 1 -- 172.21.15.7:0/3582432645 wait complete. 2024-04-25T11:15:39.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.844+0000 7f6c39000640 1 Processor -- start 2024-04-25T11:15:39.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.844+0000 7f6c39000640 1 -- start start 2024-04-25T11:15:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.846+0000 7f6c39000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c34070630 0x7f6c3414df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.846+0000 7f6c39000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c340cfa80 0x7f6c340abf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.846+0000 7f6c39000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c340d0800 0x7f6c340ac450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.846+0000 7f6c39000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c340cf280 con 0x7f6c340d0800 2024-04-25T11:15:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.846+0000 7f6c39000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6c340cefa0 con 0x7f6c340cfa80 2024-04-25T11:15:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.846+0000 7f6c39000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6c340cf560 con 0x7f6c34070630 2024-04-25T11:15:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.846+0000 7f6c33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c340d0800 0x7f6c340ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.846+0000 7f6c33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c340d0800 0x7f6c340ac450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53424/0 (socket says 172.21.15.7:53424) 2024-04-25T11:15:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.847+0000 7f6c33e00640 1 -- 172.21.15.7:0/1359679825 learned_addr learned my addr 172.21.15.7:0/1359679825 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.847+0000 7f6c32a00640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c340cfa80 0x7f6c340abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.847+0000 7f6c33400640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c34070630 0x7f6c3414df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.847+0000 7f6c33e00640 1 -- 172.21.15.7:0/1359679825 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c34070630 msgr2=0x7f6c3414df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.847+0000 7f6c33e00640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c34070630 0x7f6c3414df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.847+0000 7f6c33e00640 1 -- 172.21.15.7:0/1359679825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c340cfa80 msgr2=0x7f6c340abf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.847+0000 7f6c33e00640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c340cfa80 0x7f6c340abf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.847+0000 7f6c33e00640 1 -- 172.21.15.7:0/1359679825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3414e4b0 con 0x7f6c340d0800 2024-04-25T11:15:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.848+0000 7f6c33e00640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c340d0800 0x7f6c340ac450 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00ef90 tx=0x7f6c2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:39.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.848+0000 7f6c2b400640 1 -- 172.21.15.7:0/1359679825 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c018020 con 0x7f6c340d0800 2024-04-25T11:15:39.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.848+0000 7f6c2b400640 1 -- 172.21.15.7:0/1359679825 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c2c014480 con 0x7f6c340d0800 2024-04-25T11:15:39.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.848+0000 7f6c39000640 1 -- 172.21.15.7:0/1359679825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c340accc0 con 0x7f6c340d0800 2024-04-25T11:15:39.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.848+0000 7f6c2b400640 1 -- 172.21.15.7:0/1359679825 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c009e60 con 0x7f6c340d0800 2024-04-25T11:15:39.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.848+0000 7f6c39000640 1 -- 172.21.15.7:0/1359679825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c340ad280 con 0x7f6c340d0800 2024-04-25T11:15:39.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.849+0000 7f6c39000640 1 -- 172.21.15.7:0/1359679825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c340aca30 con 0x7f6c340d0800 2024-04-25T11:15:39.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.851+0000 7f6c2b400640 1 -- 172.21.15.7:0/1359679825 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6c2c005d10 con 0x7f6c340d0800 2024-04-25T11:15:39.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.852+0000 7f6c2b400640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c140744e0 0x7f6c140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:39.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.852+0000 7f6c2b400640 1 -- 172.21.15.7:0/1359679825 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6c2c095ec0 con 0x7f6c340d0800 2024-04-25T11:15:39.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.852+0000 7f6c33400640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c140744e0 0x7f6c140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:39.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.853+0000 7f6c33400640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c140744e0 0x7f6c140769f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c240016e0 tx=0x7f6c24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:39.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:39.857+0000 7f6c2b400640 1 -- 172.21.15.7:0/1359679825 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c2c022030 con 0x7f6c340d0800 2024-04-25T11:15:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.064+0000 7f6c39000640 1 -- 172.21.15.7:0/1359679825 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6c34050740 con 0x7f6c340d0800 2024-04-25T11:15:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.065+0000 7f6c2b400640 1 -- 172.21.15.7:0/1359679825 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f6c2c010040 con 0x7f6c340d0800 2024-04-25T11:15:40.065 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:15:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.068+0000 7f6c39000640 1 -- 172.21.15.7:0/1359679825 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c140744e0 msgr2=0x7f6c140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.068+0000 7f6c39000640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c140744e0 0x7f6c140769f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6c240016e0 tx=0x7f6c24009040 comp rx=0 tx=0).stop 2024-04-25T11:15:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.068+0000 7f6c39000640 1 -- 172.21.15.7:0/1359679825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c340d0800 msgr2=0x7f6c340ac450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.068+0000 7f6c39000640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c340d0800 0x7f6c340ac450 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00ef90 tx=0x7f6c2c00c490 comp rx=0 tx=0).stop 2024-04-25T11:15:40.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.068+0000 7f6c39000640 1 -- 172.21.15.7:0/1359679825 shutdown_connections 2024-04-25T11:15:40.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.068+0000 7f6c39000640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6c140744e0 0x7f6c140769f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.068+0000 7f6c39000640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c340d0800 0x7f6c340ac450 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.068+0000 7f6c39000640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c340cfa80 0x7f6c340abf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.068+0000 7f6c39000640 1 --2- 172.21.15.7:0/1359679825 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c34070630 0x7f6c3414df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.068+0000 7f6c39000640 1 -- 172.21.15.7:0/1359679825 >> 172.21.15.7:0/1359679825 conn(0x7f6c34074140 msgr2=0x7f6c3415eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:40.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.069+0000 7f6c39000640 1 -- 172.21.15.7:0/1359679825 shutdown_connections 2024-04-25T11:15:40.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.069+0000 7f6c39000640 1 -- 172.21.15.7:0/1359679825 wait complete. 2024-04-25T11:15:40.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:15:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.376+0000 7f0e43200640 1 Processor -- start 2024-04-25T11:15:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.376+0000 7f0e43200640 1 -- start start 2024-04-25T11:15:40.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.378+0000 7f0e43200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e3c15eed0 0x7f0e3c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.378+0000 7f0e43200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0e3c141290 0x7f0e3c1416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.378+0000 7f0e43200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e3c06d610 0x7f0e3c06da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.378+0000 7f0e43200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0e3c06c570 con 0x7f0e3c06d610 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.378+0000 7f0e43200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0e3c06c850 con 0x7f0e3c141290 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.378+0000 7f0e43200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3c06cb30 con 0x7f0e3c15eed0 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.379+0000 7f0e42800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e3c06d610 0x7f0e3c06da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.379+0000 7f0e42800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e3c06d610 0x7f0e3c06da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54404/0 (socket says 172.21.15.7:54404) 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.379+0000 7f0e42800640 1 -- 172.21.15.7:0/1768913342 learned_addr learned my addr 172.21.15.7:0/1768913342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.379+0000 7f0e41e00640 1 --2- 172.21.15.7:0/1768913342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e3c15eed0 0x7f0e3c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.379+0000 7f0e41400640 1 --2- 172.21.15.7:0/1768913342 >> v2:172.21.15.87:3300/0 conn(0x7f0e3c141290 0x7f0e3c1416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.379+0000 7f0e42800640 1 -- 172.21.15.7:0/1768913342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e3c15eed0 msgr2=0x7f0e3c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.379+0000 7f0e42800640 1 --2- 172.21.15.7:0/1768913342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e3c15eed0 0x7f0e3c0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.379+0000 7f0e42800640 1 -- 172.21.15.7:0/1768913342 >> v2:172.21.15.87:3300/0 conn(0x7f0e3c141290 msgr2=0x7f0e3c1416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.379+0000 7f0e42800640 1 --2- 172.21.15.7:0/1768913342 >> v2:172.21.15.87:3300/0 conn(0x7f0e3c141290 0x7f0e3c1416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.379+0000 7f0e42800640 1 -- 172.21.15.7:0/1768913342 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c06ce10 con 0x7f0e3c06d610 2024-04-25T11:15:40.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.381+0000 7f0e42800640 1 --2- 172.21.15.7:0/1768913342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c06d610 0x7f0e3c06da20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0cefc0 tx=0x7f0e3802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed90c823c262f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:40.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.381+0000 7f0e40a00640 1 -- 172.21.15.7:0/1768913342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38036070 con 0x7f0e3c06d610 2024-04-25T11:15:40.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.381+0000 7f0e40a00640 1 -- 172.21.15.7:0/1768913342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e38031600 con 0x7f0e3c06d610 2024-04-25T11:15:40.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.381+0000 7f0e40a00640 1 -- 172.21.15.7:0/1768913342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e380319d0 con 0x7f0e3c06d610 2024-04-25T11:15:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.382+0000 7f0e43200640 1 -- 172.21.15.7:0/1768913342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c06d610 msgr2=0x7f0e3c06da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.382+0000 7f0e43200640 1 --2- 172.21.15.7:0/1768913342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c06d610 0x7f0e3c06da20 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c0cefc0 tx=0x7f0e3802a6a0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.382+0000 7f0e43200640 1 -- 172.21.15.7:0/1768913342 shutdown_connections 2024-04-25T11:15:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.382+0000 7f0e43200640 1 --2- 172.21.15.7:0/1768913342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c06d610 0x7f0e3c06da20 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.382+0000 7f0e43200640 1 --2- 172.21.15.7:0/1768913342 >> v2:172.21.15.87:3300/0 conn(0x7f0e3c141290 0x7f0e3c1416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.382+0000 7f0e43200640 1 --2- 172.21.15.7:0/1768913342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e3c15eed0 0x7f0e3c0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.382+0000 7f0e43200640 1 -- 172.21.15.7:0/1768913342 >> 172.21.15.7:0/1768913342 conn(0x7f0e3c0d1540 msgr2=0x7f0e3c0d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.382+0000 7f0e43200640 1 -- 172.21.15.7:0/1768913342 shutdown_connections 2024-04-25T11:15:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.382+0000 7f0e43200640 1 -- 172.21.15.7:0/1768913342 wait complete. 2024-04-25T11:15:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.384+0000 7f0e43200640 1 Processor -- start 2024-04-25T11:15:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.384+0000 7f0e43200640 1 -- start start 2024-04-25T11:15:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.386+0000 7f0e43200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e3c06d610 0x7f0e3c138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.386+0000 7f0e43200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e3c141290 0x7f0e3c1f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.386+0000 7f0e43200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c15eed0 0x7f0e3c152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.386+0000 7f0e43200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c140da0 con 0x7f0e3c15eed0 2024-04-25T11:15:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.386+0000 7f0e43200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3c0cdcc0 con 0x7f0e3c06d610 2024-04-25T11:15:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.386+0000 7f0e43200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3c140f20 con 0x7f0e3c141290 2024-04-25T11:15:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.386+0000 7f0e42800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c15eed0 0x7f0e3c152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.386+0000 7f0e42800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c15eed0 0x7f0e3c152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54414/0 (socket says 172.21.15.7:54414) 2024-04-25T11:15:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.386+0000 7f0e42800640 1 -- 172.21.15.7:0/870470607 learned_addr learned my addr 172.21.15.7:0/870470607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.387+0000 7f0e41400640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e3c141290 0x7f0e3c1f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.387+0000 7f0e41e00640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e3c06d610 0x7f0e3c138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.387+0000 7f0e42800640 1 -- 172.21.15.7:0/870470607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e3c141290 msgr2=0x7f0e3c1f9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.387+0000 7f0e42800640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e3c141290 0x7f0e3c1f9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.387+0000 7f0e42800640 1 -- 172.21.15.7:0/870470607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e3c06d610 msgr2=0x7f0e3c138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.387+0000 7f0e42800640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e3c06d610 0x7f0e3c138110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.387+0000 7f0e42800640 1 -- 172.21.15.7:0/870470607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3c138af0 con 0x7f0e3c15eed0 2024-04-25T11:15:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.387+0000 7f0e42800640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c15eed0 0x7f0e3c152540 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009f40 tx=0x7f0e38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.388+0000 7f0e36000640 1 -- 172.21.15.7:0/870470607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38041030 con 0x7f0e3c15eed0 2024-04-25T11:15:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.388+0000 7f0e36000640 1 -- 172.21.15.7:0/870470607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e38031400 con 0x7f0e3c15eed0 2024-04-25T11:15:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.388+0000 7f0e43200640 1 -- 172.21.15.7:0/870470607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3c138dd0 con 0x7f0e3c15eed0 2024-04-25T11:15:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.388+0000 7f0e36000640 1 -- 172.21.15.7:0/870470607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e38036040 con 0x7f0e3c15eed0 2024-04-25T11:15:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.388+0000 7f0e43200640 1 -- 172.21.15.7:0/870470607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e3c139390 con 0x7f0e3c15eed0 2024-04-25T11:15:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.389+0000 7f0e43200640 1 -- 172.21.15.7:0/870470607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e3c050740 con 0x7f0e3c15eed0 2024-04-25T11:15:40.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.391+0000 7f0e36000640 1 -- 172.21.15.7:0/870470607 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0e38044070 con 0x7f0e3c15eed0 2024-04-25T11:15:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.392+0000 7f0e36000640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e200744e0 0x7f0e200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.392+0000 7f0e36000640 1 -- 172.21.15.7:0/870470607 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f0e380b5d40 con 0x7f0e3c15eed0 2024-04-25T11:15:40.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.392+0000 7f0e41e00640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e200744e0 0x7f0e200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:40.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.393+0000 7f0e41e00640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e200744e0 0x7f0e200769f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0e30006fd0 tx=0x7f0e30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:40.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.397+0000 7f0e36000640 1 -- 172.21.15.7:0/870470607 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e3804b070 con 0x7f0e3c15eed0 2024-04-25T11:15:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.617+0000 7f0e43200640 1 -- 172.21.15.7:0/870470607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0e3c138840 con 0x7f0e3c15eed0 2024-04-25T11:15:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.617+0000 7f0e36000640 1 -- 172.21.15.7:0/870470607 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) ==== 80+0+0 (secure 0 0 0) 0x7f0e38080330 con 0x7f0e3c15eed0 2024-04-25T11:15:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:15:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.620+0000 7f0e43200640 1 -- 172.21.15.7:0/870470607 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e200744e0 msgr2=0x7f0e200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.620+0000 7f0e43200640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e200744e0 0x7f0e200769f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0e30006fd0 tx=0x7f0e30008040 comp rx=0 tx=0).stop 2024-04-25T11:15:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.620+0000 7f0e43200640 1 -- 172.21.15.7:0/870470607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c15eed0 msgr2=0x7f0e3c152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.620+0000 7f0e43200640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c15eed0 0x7f0e3c152540 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009f40 tx=0x7f0e38004a40 comp rx=0 tx=0).stop 2024-04-25T11:15:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.620+0000 7f0e43200640 1 -- 172.21.15.7:0/870470607 shutdown_connections 2024-04-25T11:15:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.621+0000 7f0e43200640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e200744e0 0x7f0e200769f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.621+0000 7f0e43200640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e3c15eed0 0x7f0e3c152540 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.621+0000 7f0e43200640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e3c141290 0x7f0e3c1f9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.621+0000 7f0e43200640 1 --2- 172.21.15.7:0/870470607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e3c06d610 0x7f0e3c138110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.621+0000 7f0e43200640 1 -- 172.21.15.7:0/870470607 >> 172.21.15.7:0/870470607 conn(0x7f0e3c0d1540 msgr2=0x7f0e3c15ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.621+0000 7f0e43200640 1 -- 172.21.15.7:0/870470607 shutdown_connections 2024-04-25T11:15:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:40.621+0000 7f0e43200640 1 -- 172.21.15.7:0/870470607 wait complete. 2024-04-25T11:15:40.646 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:15:40.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:40.646 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T11:15:40.747 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:15:40.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:15:40.747 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:15:40.798 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:15:40.798 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:15:40.798 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:40.859 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:15:40.859 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:15:40.859 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:15:40.859 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:40.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:15:40.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:15:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.213+0000 7fc522e00640 1 Processor -- start 2024-04-25T11:15:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.213+0000 7fc522e00640 1 -- start start 2024-04-25T11:15:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.215+0000 7fc522e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc51c0b8890 0x7fc51c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.215+0000 7fc522e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc51c0d1ac0 0x7fc51c0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.215+0000 7fc522e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 0x7fc51c0b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.215+0000 7fc522e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc51c0b8ca0 con 0x7fc51c0b8890 2024-04-25T11:15:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.215+0000 7fc522e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc51c0b92c0 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.215+0000 7fc522e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc51c09cb40 con 0x7fc51c0b7120 2024-04-25T11:15:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.216+0000 7fc521a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc51c0b8890 0x7fc51c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.216+0000 7fc521000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc51c0d1ac0 0x7fc51c0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.216+0000 7fc522400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 0x7fc51c0b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.216+0000 7fc521000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc51c0d1ac0 0x7fc51c0b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:52198/0 (socket says 172.21.15.7:52198) 2024-04-25T11:15:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.216+0000 7fc521000640 1 -- 172.21.15.7:0/2535845581 learned_addr learned my addr 172.21.15.7:0/2535845581 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.217+0000 7fc521000640 1 -- 172.21.15.7:0/2535845581 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 msgr2=0x7fc51c0b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.217+0000 7fc521000640 1 --2- 172.21.15.7:0/2535845581 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 0x7fc51c0b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.217+0000 7fc521000640 1 -- 172.21.15.7:0/2535845581 >> v2:172.21.15.7:3300/0 conn(0x7fc51c0b8890 msgr2=0x7fc51c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.217+0000 7fc521000640 1 --2- 172.21.15.7:0/2535845581 >> v2:172.21.15.7:3300/0 conn(0x7fc51c0b8890 0x7fc51c0d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.217+0000 7fc521000640 1 -- 172.21.15.7:0/2535845581 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c09ce20 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.217+0000 7fc521a00640 1 --2- 172.21.15.7:0/2535845581 >> v2:172.21.15.7:3300/0 conn(0x7fc51c0b8890 0x7fc51c0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:15:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.217+0000 7fc522400640 1 --2- 172.21.15.7:0/2535845581 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 0x7fc51c0b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:15:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.219+0000 7fc521000640 1 --2- 172.21.15.7:0/2535845581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc51c0d1ac0 0x7fc51c0b6be0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc51c071310 tx=0x7fc50802a140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd51bf5b0e90f8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.219+0000 7fc51be00640 1 -- 172.21.15.7:0/2535845581 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508037070 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.220+0000 7fc51be00640 1 -- 172.21.15.7:0/2535845581 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc508032440 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.220+0000 7fc51be00640 1 -- 172.21.15.7:0/2535845581 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508032810 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.221+0000 7fc522e00640 1 -- 172.21.15.7:0/2535845581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc51c0d1ac0 msgr2=0x7fc51c0b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.221+0000 7fc522e00640 1 --2- 172.21.15.7:0/2535845581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc51c0d1ac0 0x7fc51c0b6be0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc51c071310 tx=0x7fc50802a140 comp rx=0 tx=0).stop 2024-04-25T11:15:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.221+0000 7fc522e00640 1 -- 172.21.15.7:0/2535845581 shutdown_connections 2024-04-25T11:15:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.221+0000 7fc522e00640 1 --2- 172.21.15.7:0/2535845581 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 0x7fc51c0b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.221+0000 7fc522e00640 1 --2- 172.21.15.7:0/2535845581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc51c0d1ac0 0x7fc51c0b6be0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.221+0000 7fc522e00640 1 --2- 172.21.15.7:0/2535845581 >> v2:172.21.15.7:3300/0 conn(0x7fc51c0b8890 0x7fc51c0d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.221+0000 7fc522e00640 1 -- 172.21.15.7:0/2535845581 >> 172.21.15.7:0/2535845581 conn(0x7fc51c0d0250 msgr2=0x7fc51c0d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.221+0000 7fc522e00640 1 -- 172.21.15.7:0/2535845581 shutdown_connections 2024-04-25T11:15:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.221+0000 7fc522e00640 1 -- 172.21.15.7:0/2535845581 wait complete. 2024-04-25T11:15:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.225+0000 7fc522e00640 1 Processor -- start 2024-04-25T11:15:41.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.226+0000 7fc522e00640 1 -- start start 2024-04-25T11:15:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.228+0000 7fc522e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 0x7fc51c1e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.228+0000 7fc522e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc51c0b8890 0x7fc51c0a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c0d1ac0 0x7fc51c0a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c0b8f70 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c09d100 con 0x7fc51c0b8890 2024-04-25T11:15:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc51c0b7a70 con 0x7fc51c0b7120 2024-04-25T11:15:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c0d1ac0 0x7fc51c0a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c0d1ac0 0x7fc51c0a0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54418/0 (socket says 172.21.15.7:54418) 2024-04-25T11:15:41.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522400640 1 -- 172.21.15.7:0/2102641988 learned_addr learned my addr 172.21.15.7:0/2102641988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:41.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc521a00640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 0x7fc51c1e5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522400640 1 -- 172.21.15.7:0/2102641988 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 msgr2=0x7fc51c1e5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522400640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 0x7fc51c1e5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522400640 1 -- 172.21.15.7:0/2102641988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc51c0b8890 msgr2=0x7fc51c0a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522400640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc51c0b8890 0x7fc51c0a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc522400640 1 -- 172.21.15.7:0/2102641988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c1e8a00 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.229+0000 7fc521a00640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 0x7fc51c1e5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.230+0000 7fc522400640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c0d1ac0 0x7fc51c0a0db0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc51400eed0 tx=0x7fc51400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.230+0000 7fc519600640 1 -- 172.21.15.7:0/2102641988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514018020 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.230+0000 7fc519600640 1 -- 172.21.15.7:0/2102641988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc514014400 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.230+0000 7fc519600640 1 -- 172.21.15.7:0/2102641988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5140147f0 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.230+0000 7fc522e00640 1 -- 172.21.15.7:0/2102641988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51c1e8ce0 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.230+0000 7fc522e00640 1 -- 172.21.15.7:0/2102641988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51c1e92a0 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.232+0000 7fc522e00640 1 -- 172.21.15.7:0/2102641988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc51c050740 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.233+0000 7fc519600640 1 -- 172.21.15.7:0/2102641988 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc514005d10 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.235+0000 7fc519600640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4fc0744b0 0x7fc4fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.235+0000 7fc519600640 1 -- 172.21.15.7:0/2102641988 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fc514095cc0 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.235+0000 7fc521a00640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4fc0744b0 0x7fc4fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.236+0000 7fc521a00640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4fc0744b0 0x7fc4fc0769c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc510009a20 tx=0x7fc510008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:41.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.239+0000 7fc519600640 1 -- 172.21.15.7:0/2102641988 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc51409c080 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.484+0000 7fc522e00640 1 -- 172.21.15.7:0/2102641988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc51c1e6650 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.485+0000 7fc519600640 1 -- 172.21.15.7:0/2102641988 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5171 (secure 0 0 0) 0x7fc514022030 con 0x7fc51c0d1ac0 2024-04-25T11:15:41.485 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/779859707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":779859707},{"type":"v1","addr":"172.21.15.87:6834","nonce":779859707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/454388957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":454388957},{"type":"v1","addr":"172.21.15.87:6835","nonce":454388957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2318271315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2318271315},{"type":"v1","addr":"172.21.15.7:6836","nonce":2318271315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:15:05.259317+0000","modified":"2024-04-25T11:15:08.314094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6528},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6528":{"gid":6528,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2955530443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2955530443},{"type":"v1","addr":"172.21.15.7:6837","nonce":2955530443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6528,"qdb_cluster":[6528]},"id":5}]} 2024-04-25T11:15:41.485 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 43 2024-04-25T11:15:41.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 -- 172.21.15.7:0/2102641988 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4fc0744b0 msgr2=0x7fc4fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4fc0744b0 0x7fc4fc0769c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc510009a20 tx=0x7fc510008040 comp rx=0 tx=0).stop 2024-04-25T11:15:41.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 -- 172.21.15.7:0/2102641988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c0d1ac0 msgr2=0x7fc51c0a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c0d1ac0 0x7fc51c0a0db0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc51400eed0 tx=0x7fc51400b480 comp rx=0 tx=0).stop 2024-04-25T11:15:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 -- 172.21.15.7:0/2102641988 shutdown_connections 2024-04-25T11:15:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4fc0744b0 0x7fc4fc0769c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc51c0d1ac0 0x7fc51c0a0db0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc51c0b8890 0x7fc51c0a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 --2- 172.21.15.7:0/2102641988 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc51c0b7120 0x7fc51c1e5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 -- 172.21.15.7:0/2102641988 >> 172.21.15.7:0/2102641988 conn(0x7fc51c0d0250 msgr2=0x7fc51c072580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 -- 172.21.15.7:0/2102641988 shutdown_connections 2024-04-25T11:15:41.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.488+0000 7fc522e00640 1 -- 172.21.15.7:0/2102641988 wait complete. 2024-04-25T11:15:41.514 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:15:41.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.785+0000 7f17f7e00640 1 Processor -- start 2024-04-25T11:15:41.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.785+0000 7f17f7e00640 1 -- start start 2024-04-25T11:15:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.787+0000 7f17f7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17f8070890 0x7f17f815c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.787+0000 7f17f7e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17f806ef10 con 0x7f17f80afca0 2024-04-25T11:15:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.787+0000 7f17f7e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17f80b6390 con 0x7f17f80b5140 2024-04-25T11:15:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.787+0000 7f17f7e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f17f80b07c0 con 0x7f17f8070890 2024-04-25T11:15:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.788+0000 7f17f7400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f17f80afca0 0x7f17f80b00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50490/0 (socket says 172.21.15.7:50490) 2024-04-25T11:15:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.788+0000 7f17f7400640 1 -- 172.21.15.7:0/1064916750 learned_addr learned my addr 172.21.15.7:0/1064916750 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.788+0000 7f17f6a00640 1 --2- 172.21.15.7:0/1064916750 >> v2:172.21.15.87:3300/0 conn(0x7f17f8070890 0x7f17f815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.788+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 967463646 0 0) 0x7f17f806ef10 con 0x7f17f80afca0 2024-04-25T11:15:41.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.788+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17dc005730 con 0x7f17f80afca0 2024-04-25T11:15:41.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 344314191 0 0) 0x7f17dc005730 con 0x7f17f80afca0 2024-04-25T11:15:41.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 >> v2:172.21.15.87:3300/0 conn(0x7f17f8070890 msgr2=0x7f17f815c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:41.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 --2- 172.21.15.7:0/1064916750 >> v2:172.21.15.87:3300/0 conn(0x7f17f8070890 0x7f17f815c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 >> v1:172.21.15.87:6789/0 conn(0x7f17f80b5140 legacy=0x7f17f806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f80b6670 con 0x7f17f80afca0 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f17f00019f0 con 0x7f17f80afca0 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 >> v1:172.21.15.7:6789/0 conn(0x7f17f80afca0 legacy=0x7f17f80b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17dc005bc0 0x7f17dc006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17dc006a10 0x7f17dc006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 0x7f17dc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17f80b07c0 con 0x7f17dc007460 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f17dc00e3f0 con 0x7f17dc005bc0 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f17dc00e5f0 con 0x7f17dc006a10 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f7400640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 0x7f17dc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.789+0000 7f17f6a00640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17dc005bc0 0x7f17dc006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.790+0000 7f17f6000640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17dc006a10 0x7f17dc006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.790+0000 7f17f7400640 1 -- 172.21.15.7:0/1064916750 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17dc006a10 msgr2=0x7f17dc006e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.790+0000 7f17f7400640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17dc006a10 0x7f17dc006e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.790+0000 7f17f7400640 1 -- 172.21.15.7:0/1064916750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17dc005bc0 msgr2=0x7f17dc006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.790+0000 7f17f7400640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17dc005bc0 0x7f17dc006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.790+0000 7f17f7400640 1 -- 172.21.15.7:0/1064916750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17dc005730 con 0x7f17dc007460 2024-04-25T11:15:41.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.790+0000 7f17f6a00640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17dc005bc0 0x7f17dc006030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:15:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.791+0000 7f17f7400640 1 -- 172.21.15.7:0/1064916750 reap_dead start 2024-04-25T11:15:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.791+0000 7f17f7400640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 0x7f17dc00dbd0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f17f000a860 tx=0x7f17dc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad701c8a8105dea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.791+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000a480 con 0x7f17dc007460 2024-04-25T11:15:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.791+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f002f070 con 0x7f17dc007460 2024-04-25T11:15:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.791+0000 7f17f5600640 1 -- 172.21.15.7:0/1064916750 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f002bca0 con 0x7f17dc007460 2024-04-25T11:15:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.792+0000 7f17f7e00640 1 -- 172.21.15.7:0/1064916750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 msgr2=0x7f17dc00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.792+0000 7f17f7e00640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 0x7f17dc00dbd0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f17f000a860 tx=0x7f17dc007a40 comp rx=0 tx=0).stop 2024-04-25T11:15:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.792+0000 7f17f7e00640 1 -- 172.21.15.7:0/1064916750 shutdown_connections 2024-04-25T11:15:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.792+0000 7f17f7e00640 1 --2- 172.21.15.7:0/1064916750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 0x7f17dc00dbd0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.792+0000 7f17f7e00640 1 -- 172.21.15.7:0/1064916750 >> 172.21.15.7:0/1064916750 conn(0x7f17f80b3820 msgr2=0x7f17f80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.792+0000 7f17f7e00640 1 -- 172.21.15.7:0/1064916750 shutdown_connections 2024-04-25T11:15:41.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.792+0000 7f17f7e00640 1 -- 172.21.15.7:0/1064916750 wait complete. 2024-04-25T11:15:41.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.795+0000 7f17f7e00640 1 Processor -- start 2024-04-25T11:15:41.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.795+0000 7f17f7e00640 1 -- start start 2024-04-25T11:15:41.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.796+0000 7f17f7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 0x7f17f8159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17f81f92a0 0x7f17f81f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17f81e9d30 0x7f17f81ea140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17f80b6950 con 0x7f17dc007460 2024-04-25T11:15:41.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f17f806f210 con 0x7f17f81e9d30 2024-04-25T11:15:41.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f17f815c390 con 0x7f17f81f92a0 2024-04-25T11:15:41.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 0x7f17f8159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 0x7f17f8159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54444/0 (socket says 172.21.15.7:54444) 2024-04-25T11:15:41.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f6a00640 1 -- 172.21.15.7:0/1101024174 learned_addr learned my addr 172.21.15.7:0/1101024174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:41.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f6a00640 1 -- 172.21.15.7:0/1101024174 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17f81f92a0 msgr2=0x7f17f81f96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f6a00640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17f81f92a0 0x7f17f81f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f6a00640 1 -- 172.21.15.7:0/1101024174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17f81e9d30 msgr2=0x7f17f81ea140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:41.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f6a00640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17f81e9d30 0x7f17f81ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:41.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.797+0000 7f17f6a00640 1 -- 172.21.15.7:0/1101024174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f815d6e0 con 0x7f17dc007460 2024-04-25T11:15:41.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.798+0000 7f17f6a00640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 0x7f17f8159790 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f17e80114e0 tx=0x7f17e80119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:41.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.798+0000 7f17eea00640 1 -- 172.21.15.7:0/1101024174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8015870 con 0x7f17dc007460 2024-04-25T11:15:41.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.798+0000 7f17eea00640 1 -- 172.21.15.7:0/1101024174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e8015a10 con 0x7f17dc007460 2024-04-25T11:15:41.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.798+0000 7f17f7e00640 1 -- 172.21.15.7:0/1101024174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f81f9ff0 con 0x7f17dc007460 2024-04-25T11:15:41.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.798+0000 7f17eea00640 1 -- 172.21.15.7:0/1101024174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8015dc0 con 0x7f17dc007460 2024-04-25T11:15:41.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.798+0000 7f17f7e00640 1 -- 172.21.15.7:0/1101024174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f815dc80 con 0x7f17dc007460 2024-04-25T11:15:41.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.799+0000 7f17f7e00640 1 -- 172.21.15.7:0/1101024174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f8050740 con 0x7f17dc007460 2024-04-25T11:15:41.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.801+0000 7f17eea00640 1 -- 172.21.15.7:0/1101024174 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f17e8013400 con 0x7f17dc007460 2024-04-25T11:15:41.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.802+0000 7f17eea00640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17d80744b0 0x7f17d80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:41.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.802+0000 7f17eea00640 1 -- 172.21.15.7:0/1101024174 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f17e8095d30 con 0x7f17dc007460 2024-04-25T11:15:41.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.802+0000 7f17f6000640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17d80744b0 0x7f17d80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:41.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.803+0000 7f17f6000640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17d80744b0 0x7f17d80769c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f17e40014f0 tx=0x7f17e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:41.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:41.807+0000 7f17eea00640 1 -- 172.21.15.7:0/1101024174 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f17e8019030 con 0x7f17dc007460 2024-04-25T11:15:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.011+0000 7f17f7e00640 1 -- 172.21.15.7:0/1101024174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17f8070440 con 0x7f17dc007460 2024-04-25T11:15:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.014+0000 7f17eea00640 1 -- 172.21.15.7:0/1101024174 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16849 (secure 0 0 0) 0x7f17e8060320 con 0x7f17dc007460 2024-04-25T11:15:42.014 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:42.014 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:15:06.697374+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:03.587954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T11:15:04.226794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:15:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.016+0000 7f17f7e00640 1 -- 172.21.15.7:0/1101024174 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17d80744b0 msgr2=0x7f17d80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.016+0000 7f17f7e00640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17d80744b0 0x7f17d80769c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f17e40014f0 tx=0x7f17e4009040 comp rx=0 tx=0).stop 2024-04-25T11:15:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.016+0000 7f17f7e00640 1 -- 172.21.15.7:0/1101024174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 msgr2=0x7f17f8159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.016+0000 7f17f7e00640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 0x7f17f8159790 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f17e80114e0 tx=0x7f17e80119a0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.016+0000 7f17f7e00640 1 -- 172.21.15.7:0/1101024174 shutdown_connections 2024-04-25T11:15:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.016+0000 7f17f7e00640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17d80744b0 0x7f17d80769c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.016+0000 7f17f7e00640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17f81e9d30 0x7f17f81ea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.016+0000 7f17f7e00640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17f81f92a0 0x7f17f81f96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.016+0000 7f17f7e00640 1 --2- 172.21.15.7:0/1101024174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17dc007460 0x7f17f8159790 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.016+0000 7f17f7e00640 1 -- 172.21.15.7:0/1101024174 >> 172.21.15.7:0/1101024174 conn(0x7f17f80b3820 msgr2=0x7f17f80b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.017+0000 7f17f7e00640 1 -- 172.21.15.7:0/1101024174 shutdown_connections 2024-04-25T11:15:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.017+0000 7f17f7e00640 1 -- 172.21.15.7:0/1101024174 wait complete. 2024-04-25T11:15:42.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:15:03.587954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:15:04.226794+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:15:42.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:15:42.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:15:42.132 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:15:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.328+0000 7f6f7cc00640 1 Processor -- start 2024-04-25T11:15:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.328+0000 7f6f7cc00640 1 -- start start 2024-04-25T11:15:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.330+0000 7f6f7cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f7815dfb0 0x7f6f780b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.330+0000 7f6f7cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6f780d0b40 0x7f6f7806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.330+0000 7f6f7cc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f780d0f50 0x7f6f7806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.330+0000 7f6f7cc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6f780b09b0 con 0x7f6f780d0f50 2024-04-25T11:15:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.330+0000 7f6f7cc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6f7815e3c0 con 0x7f6f780d0b40 2024-04-25T11:15:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.330+0000 7f6f7cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6f780cf800 con 0x7f6f7815dfb0 2024-04-25T11:15:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.331+0000 7f6f76800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f780d0f50 0x7f6f7806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.331+0000 7f6f76800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f780d0f50 0x7f6f7806f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54446/0 (socket says 172.21.15.7:54446) 2024-04-25T11:15:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.331+0000 7f6f76800640 1 -- 172.21.15.7:0/3530392573 learned_addr learned my addr 172.21.15.7:0/3530392573 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.331+0000 7f6f6fe00640 1 --2- 172.21.15.7:0/3530392573 >> v2:172.21.15.87:3300/0 conn(0x7f6f780d0b40 0x7f6f7806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.331+0000 7f6f75e00640 1 --2- 172.21.15.7:0/3530392573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f7815dfb0 0x7f6f780b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.331+0000 7f6f76800640 1 -- 172.21.15.7:0/3530392573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f7815dfb0 msgr2=0x7f6f780b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.331+0000 7f6f76800640 1 --2- 172.21.15.7:0/3530392573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f7815dfb0 0x7f6f780b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.331+0000 7f6f76800640 1 -- 172.21.15.7:0/3530392573 >> v2:172.21.15.87:3300/0 conn(0x7f6f780d0b40 msgr2=0x7f6f7806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.331+0000 7f6f76800640 1 --2- 172.21.15.7:0/3530392573 >> v2:172.21.15.87:3300/0 conn(0x7f6f780d0b40 0x7f6f7806f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.331+0000 7f6f76800640 1 -- 172.21.15.7:0/3530392573 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f780cfaa0 con 0x7f6f780d0f50 2024-04-25T11:15:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.333+0000 7f6f76800640 1 --2- 172.21.15.7:0/3530392573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f780d0f50 0x7f6f7806f8e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6f7806e110 tx=0x7f6f7002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb42ab5abef8ed2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.333+0000 7f6f75400640 1 -- 172.21.15.7:0/3530392573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70036070 con 0x7f6f780d0f50 2024-04-25T11:15:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.333+0000 7f6f75400640 1 -- 172.21.15.7:0/3530392573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f70031600 con 0x7f6f780d0f50 2024-04-25T11:15:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.333+0000 7f6f75400640 1 -- 172.21.15.7:0/3530392573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f700319d0 con 0x7f6f780d0f50 2024-04-25T11:15:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.334+0000 7f6f7cc00640 1 -- 172.21.15.7:0/3530392573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f780d0f50 msgr2=0x7f6f7806f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.334+0000 7f6f7cc00640 1 --2- 172.21.15.7:0/3530392573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f780d0f50 0x7f6f7806f8e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6f7806e110 tx=0x7f6f7002a6a0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.334+0000 7f6f7cc00640 1 -- 172.21.15.7:0/3530392573 shutdown_connections 2024-04-25T11:15:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.334+0000 7f6f7cc00640 1 --2- 172.21.15.7:0/3530392573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f780d0f50 0x7f6f7806f8e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.334+0000 7f6f7cc00640 1 --2- 172.21.15.7:0/3530392573 >> v2:172.21.15.87:3300/0 conn(0x7f6f780d0b40 0x7f6f7806f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.334+0000 7f6f7cc00640 1 --2- 172.21.15.7:0/3530392573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f7815dfb0 0x7f6f780b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.334+0000 7f6f7cc00640 1 -- 172.21.15.7:0/3530392573 >> 172.21.15.7:0/3530392573 conn(0x7f6f780742c0 msgr2=0x7f6f7806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.334+0000 7f6f7cc00640 1 -- 172.21.15.7:0/3530392573 shutdown_connections 2024-04-25T11:15:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.334+0000 7f6f7cc00640 1 -- 172.21.15.7:0/3530392573 wait complete. 2024-04-25T11:15:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.336+0000 7f6f7cc00640 1 Processor -- start 2024-04-25T11:15:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.337+0000 7f6f7cc00640 1 -- start start 2024-04-25T11:15:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.338+0000 7f6f7cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f780d0b40 0x7f6f7814daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.338+0000 7f6f7cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f780d0f50 0x7f6f780cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.338+0000 7f6f7cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7815dfb0 0x7f6f781e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.338+0000 7f6f7cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f780d0340 con 0x7f6f7815dfb0 2024-04-25T11:15:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.338+0000 7f6f7cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6f780d0060 con 0x7f6f780d0b40 2024-04-25T11:15:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.338+0000 7f6f7cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6f780d0620 con 0x7f6f780d0f50 2024-04-25T11:15:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.339+0000 7f6f76800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7815dfb0 0x7f6f781e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.339+0000 7f6f76800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7815dfb0 0x7f6f781e9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54460/0 (socket says 172.21.15.7:54460) 2024-04-25T11:15:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.339+0000 7f6f76800640 1 -- 172.21.15.7:0/674275196 learned_addr learned my addr 172.21.15.7:0/674275196 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:42.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.339+0000 7f6f6fe00640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f780d0f50 0x7f6f780cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.339+0000 7f6f75e00640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f780d0b40 0x7f6f7814daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.339+0000 7f6f76800640 1 -- 172.21.15.7:0/674275196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f780d0f50 msgr2=0x7f6f780cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.339+0000 7f6f76800640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f780d0f50 0x7f6f780cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.339+0000 7f6f76800640 1 -- 172.21.15.7:0/674275196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f780d0b40 msgr2=0x7f6f7814daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.339+0000 7f6f76800640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f780d0b40 0x7f6f7814daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.339+0000 7f6f76800640 1 -- 172.21.15.7:0/674275196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7814e030 con 0x7f6f7815dfb0 2024-04-25T11:15:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.339+0000 7f6f6fe00640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f780d0f50 0x7f6f780cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:15:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.340+0000 7f6f76800640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7815dfb0 0x7f6f781e9ae0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6f7002ac90 tx=0x7f6f70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.340+0000 7f6f6e000640 1 -- 172.21.15.7:0/674275196 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7003f020 con 0x7f6f7815dfb0 2024-04-25T11:15:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.340+0000 7f6f6e000640 1 -- 172.21.15.7:0/674275196 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f70037040 con 0x7f6f7815dfb0 2024-04-25T11:15:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.340+0000 7f6f7cc00640 1 -- 172.21.15.7:0/674275196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f781f9d70 con 0x7f6f7815dfb0 2024-04-25T11:15:42.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.340+0000 7f6f6e000640 1 -- 172.21.15.7:0/674275196 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70036070 con 0x7f6f7815dfb0 2024-04-25T11:15:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.340+0000 7f6f7cc00640 1 -- 172.21.15.7:0/674275196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f781fa330 con 0x7f6f7815dfb0 2024-04-25T11:15:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.341+0000 7f6f7cc00640 1 -- 172.21.15.7:0/674275196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7806fb40 con 0x7f6f7815dfb0 2024-04-25T11:15:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.343+0000 7f6f6e000640 1 -- 172.21.15.7:0/674275196 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6f70042020 con 0x7f6f7815dfb0 2024-04-25T11:15:42.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.344+0000 7f6f6e000640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f540744e0 0x7f6f540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.344+0000 7f6f6e000640 1 -- 172.21.15.7:0/674275196 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6f700b5980 con 0x7f6f7815dfb0 2024-04-25T11:15:42.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.344+0000 7f6f75e00640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f540744e0 0x7f6f540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.345+0000 7f6f75e00640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f540744e0 0x7f6f540769f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6f68006fd0 tx=0x7f6f68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:42.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.349+0000 7f6f6e000640 1 -- 172.21.15.7:0/674275196 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f7806fb40 con 0x7f6f7815dfb0 2024-04-25T11:15:42.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.552+0000 7f6f7cc00640 1 -- 172.21.15.7:0/674275196 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f780cd270 con 0x7f6f7815dfb0 2024-04-25T11:15:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.554+0000 7f6f6e000640 1 -- 172.21.15.7:0/674275196 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7f6f70049030 con 0x7f6f7815dfb0 2024-04-25T11:15:42.554 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:15:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.556+0000 7f6f7cc00640 1 -- 172.21.15.7:0/674275196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f540744e0 msgr2=0x7f6f540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.556+0000 7f6f7cc00640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f540744e0 0x7f6f540769f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6f68006fd0 tx=0x7f6f68008040 comp rx=0 tx=0).stop 2024-04-25T11:15:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.556+0000 7f6f7cc00640 1 -- 172.21.15.7:0/674275196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7815dfb0 msgr2=0x7f6f781e9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.556+0000 7f6f7cc00640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7815dfb0 0x7f6f781e9ae0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f6f7002ac90 tx=0x7f6f70004a40 comp rx=0 tx=0).stop 2024-04-25T11:15:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.557+0000 7f6f7cc00640 1 -- 172.21.15.7:0/674275196 shutdown_connections 2024-04-25T11:15:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.557+0000 7f6f7cc00640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f540744e0 0x7f6f540769f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.557+0000 7f6f7cc00640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7815dfb0 0x7f6f781e9ae0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.557+0000 7f6f7cc00640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f780d0f50 0x7f6f780cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.557+0000 7f6f7cc00640 1 --2- 172.21.15.7:0/674275196 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f780d0b40 0x7f6f7814daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.557+0000 7f6f7cc00640 1 -- 172.21.15.7:0/674275196 >> 172.21.15.7:0/674275196 conn(0x7f6f780742c0 msgr2=0x7f6f7806f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:42.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.557+0000 7f6f7cc00640 1 -- 172.21.15.7:0/674275196 shutdown_connections 2024-04-25T11:15:42.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.557+0000 7f6f7cc00640 1 -- 172.21.15.7:0/674275196 wait complete. 2024-04-25T11:15:42.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:15:42.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.864+0000 7f9649400640 1 Processor -- start 2024-04-25T11:15:42.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.864+0000 7f9649400640 1 -- start start 2024-04-25T11:15:42.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.867+0000 7f9649400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f96440d1ea0 0x7f96440b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.867+0000 7f9649400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96441603d0 con 0x7f96440b7be0 2024-04-25T11:15:42.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.867+0000 7f9649400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f964406d090 con 0x7f964406de10 2024-04-25T11:15:42.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.867+0000 7f9649400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f96440d3490 con 0x7f96440d1ea0 2024-04-25T11:15:42.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.867+0000 7f9643400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f96440b7be0 0x7f96440d1790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50494/0 (socket says 172.21.15.7:50494) 2024-04-25T11:15:42.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.867+0000 7f9643400640 1 -- 172.21.15.7:0/1521272331 learned_addr learned my addr 172.21.15.7:0/1521272331 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:42.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9648a00640 1 --2- 172.21.15.7:0/1521272331 >> v2:172.21.15.87:3300/0 conn(0x7f96440d1ea0 0x7f96440b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 97994382 0 0) 0x7f96441603d0 con 0x7f96440b7be0 2024-04-25T11:15:42.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9628003520 con 0x7f96440b7be0 2024-04-25T11:15:42.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 397177611 0 0) 0x7f9628003520 con 0x7f96440b7be0 2024-04-25T11:15:42.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 >> v2:172.21.15.87:3300/0 conn(0x7f96440d1ea0 msgr2=0x7f96440b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:42.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 --2- 172.21.15.7:0/1521272331 >> v2:172.21.15.87:3300/0 conn(0x7f96440d1ea0 0x7f96440b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 >> v1:172.21.15.87:6789/0 conn(0x7f964406de10 legacy=0x7f96440b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964406d330 con 0x7f96440b7be0 2024-04-25T11:15:42.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f962c001670 con 0x7f96440b7be0 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 >> v1:172.21.15.7:6789/0 conn(0x7f96440b7be0 legacy=0x7f96440d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 --2- 172.21.15.7:0/1521272331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96280039b0 0x7f9628003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 --2- 172.21.15.7:0/1521272331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9628004800 0x7f9628004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 --2- 172.21.15.7:0/1521272331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96280051c0 0x7f962800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96440d3490 con 0x7f96280051c0 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f962800c160 con 0x7f9628004800 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.868+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f962800c360 con 0x7f96280039b0 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.869+0000 7f9648a00640 1 --2- 172.21.15.7:0/1521272331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96280051c0 0x7f962800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.869+0000 7f9643400640 1 --2- 172.21.15.7:0/1521272331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9628004800 0x7f9628004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.869+0000 7f9648a00640 1 -- 172.21.15.7:0/1521272331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96280039b0 msgr2=0x7f9628003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.869+0000 7f9648a00640 1 --2- 172.21.15.7:0/1521272331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96280039b0 0x7f9628003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.869+0000 7f9648a00640 1 -- 172.21.15.7:0/1521272331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9628004800 msgr2=0x7f9628004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.869+0000 7f9648a00640 1 --2- 172.21.15.7:0/1521272331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9628004800 0x7f9628004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.869+0000 7f9648a00640 1 -- 172.21.15.7:0/1521272331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9628003520 con 0x7f96280051c0 2024-04-25T11:15:42.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.870+0000 7f9648a00640 1 -- 172.21.15.7:0/1521272331 reap_dead start 2024-04-25T11:15:42.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.870+0000 7f9648a00640 1 --2- 172.21.15.7:0/1521272331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96280051c0 0x7f962800b9c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f963c009f90 tx=0x7f9628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203080f5a0038bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:42.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.871+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c037070 con 0x7f96280051c0 2024-04-25T11:15:42.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.871+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f963c032620 con 0x7f96280051c0 2024-04-25T11:15:42.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.871+0000 7f9642a00640 1 -- 172.21.15.7:0/1521272331 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c009c10 con 0x7f96280051c0 2024-04-25T11:15:42.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.871+0000 7f9649400640 1 -- 172.21.15.7:0/1521272331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96280051c0 msgr2=0x7f962800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.871+0000 7f9649400640 1 --2- 172.21.15.7:0/1521272331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96280051c0 0x7f962800b9c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f963c009f90 tx=0x7f9628005830 comp rx=0 tx=0).stop 2024-04-25T11:15:42.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.871+0000 7f9649400640 1 -- 172.21.15.7:0/1521272331 shutdown_connections 2024-04-25T11:15:42.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.871+0000 7f9649400640 1 --2- 172.21.15.7:0/1521272331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96280051c0 0x7f962800b9c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.871+0000 7f9649400640 1 -- 172.21.15.7:0/1521272331 >> 172.21.15.7:0/1521272331 conn(0x7f964409b540 msgr2=0x7f964409b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:42.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.872+0000 7f9649400640 1 -- 172.21.15.7:0/1521272331 shutdown_connections 2024-04-25T11:15:42.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.872+0000 7f9649400640 1 -- 172.21.15.7:0/1521272331 wait complete. 2024-04-25T11:15:42.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.874+0000 7f9649400640 1 Processor -- start 2024-04-25T11:15:42.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.874+0000 7f9649400640 1 -- start start 2024-04-25T11:15:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9649400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96280051c0 0x7f96440b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9649400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9644150b70 0x7f96440b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9649400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96440b0ae0 0x7f96440ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9649400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f964406e220 con 0x7f9644150b70 2024-04-25T11:15:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9649400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f964406d8f0 con 0x7f96440b0ae0 2024-04-25T11:15:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9649400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f964406e3a0 con 0x7f96280051c0 2024-04-25T11:15:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9643400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9644150b70 0x7f96440b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9643400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9644150b70 0x7f96440b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54482/0 (socket says 172.21.15.7:54482) 2024-04-25T11:15:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9643400640 1 -- 172.21.15.7:0/2171527603 learned_addr learned my addr 172.21.15.7:0/2171527603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9643e00640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96280051c0 0x7f96440b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9648a00640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96440b0ae0 0x7f96440ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9643400640 1 -- 172.21.15.7:0/2171527603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96280051c0 msgr2=0x7f96440b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9643400640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96280051c0 0x7f96440b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9643400640 1 -- 172.21.15.7:0/2171527603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96440b0ae0 msgr2=0x7f96440ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.876+0000 7f9643400640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96440b0ae0 0x7f96440ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.877+0000 7f9643400640 1 -- 172.21.15.7:0/2171527603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96440c8f60 con 0x7f9644150b70 2024-04-25T11:15:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.877+0000 7f9648a00640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96440b0ae0 0x7f96440ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.877+0000 7f9643400640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9644150b70 0x7f96440b05a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f962c00e730 tx=0x7f962c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.877+0000 7f963be00640 1 -- 172.21.15.7:0/2171527603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c00ee40 con 0x7f9644150b70 2024-04-25T11:15:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.877+0000 7f963be00640 1 -- 172.21.15.7:0/2171527603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f962c018070 con 0x7f9644150b70 2024-04-25T11:15:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.877+0000 7f9649400640 1 -- 172.21.15.7:0/2171527603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96440b1270 con 0x7f9644150b70 2024-04-25T11:15:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.877+0000 7f963be00640 1 -- 172.21.15.7:0/2171527603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c00cb10 con 0x7f9644150b70 2024-04-25T11:15:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.877+0000 7f9649400640 1 -- 172.21.15.7:0/2171527603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96440caf20 con 0x7f9644150b70 2024-04-25T11:15:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.879+0000 7f9649400640 1 -- 172.21.15.7:0/2171527603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9644050740 con 0x7f9644150b70 2024-04-25T11:15:42.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.880+0000 7f963be00640 1 -- 172.21.15.7:0/2171527603 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f962c00ccb0 con 0x7f9644150b70 2024-04-25T11:15:42.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.881+0000 7f963be00640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96200744b0 0x7f96200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:42.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.881+0000 7f963be00640 1 -- 172.21.15.7:0/2171527603 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f962c095350 con 0x7f9644150b70 2024-04-25T11:15:42.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.881+0000 7f9643e00640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96200744b0 0x7f96200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:42.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.882+0000 7f9643e00640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96200744b0 0x7f96200769c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f96340016f0 tx=0x7f9634009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:42.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:42.886+0000 7f963be00640 1 -- 172.21.15.7:0/2171527603 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f962c05f8b0 con 0x7f9644150b70 2024-04-25T11:15:43.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.091+0000 7f9649400640 1 -- 172.21.15.7:0/2171527603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96440b4d90 con 0x7f9644150b70 2024-04-25T11:15:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.092+0000 7f963be00640 1 -- 172.21.15.7:0/2171527603 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f962c05f1b0 con 0x7f9644150b70 2024-04-25T11:15:43.093 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:15:43.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.095+0000 7f9649400640 1 -- 172.21.15.7:0/2171527603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96200744b0 msgr2=0x7f96200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:43.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.095+0000 7f9649400640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96200744b0 0x7f96200769c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f96340016f0 tx=0x7f9634009040 comp rx=0 tx=0).stop 2024-04-25T11:15:43.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.095+0000 7f9649400640 1 -- 172.21.15.7:0/2171527603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9644150b70 msgr2=0x7f96440b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:43.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.095+0000 7f9649400640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9644150b70 0x7f96440b05a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f962c00e730 tx=0x7f962c00ebf0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.095+0000 7f9649400640 1 -- 172.21.15.7:0/2171527603 shutdown_connections 2024-04-25T11:15:43.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.095+0000 7f9649400640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96200744b0 0x7f96200769c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.095+0000 7f9649400640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96440b0ae0 0x7f96440ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.095+0000 7f9649400640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9644150b70 0x7f96440b05a0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.095+0000 7f9649400640 1 --2- 172.21.15.7:0/2171527603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96280051c0 0x7f96440b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.095+0000 7f9649400640 1 -- 172.21.15.7:0/2171527603 >> 172.21.15.7:0/2171527603 conn(0x7f964409b540 msgr2=0x7f9644070660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:43.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.095+0000 7f9649400640 1 -- 172.21.15.7:0/2171527603 shutdown_connections 2024-04-25T11:15:43.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.096+0000 7f9649400640 1 -- 172.21.15.7:0/2171527603 wait complete. 2024-04-25T11:15:43.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:15:43.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.416+0000 7fe719400640 1 Processor -- start 2024-04-25T11:15:43.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.416+0000 7fe719400640 1 -- start start 2024-04-25T11:15:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.418+0000 7fe719400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe71415db10 0x7fe714160470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.419+0000 7fe719400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe71409b540 0x7fe71409b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.419+0000 7fe719400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe71409be90 con 0x7fe71406eb50 2024-04-25T11:15:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.419+0000 7fe719400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe7140b2da0 con 0x7fe71409b540 2024-04-25T11:15:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.419+0000 7fe719400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe714160b70 con 0x7fe71415db10 2024-04-25T11:15:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.419+0000 7fe718a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe71409b540 0x7fe71409b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.419+0000 7fe70be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe71406eb50 0x7fe7140b2690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50510/0 (socket says 172.21.15.7:50510) 2024-04-25T11:15:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.419+0000 7fe70be00640 1 -- 172.21.15.7:0/3167088932 learned_addr learned my addr 172.21.15.7:0/3167088932 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.419+0000 7fe713e00640 1 --2- 172.21.15.7:0/3167088932 >> v2:172.21.15.87:3300/0 conn(0x7fe71415db10 0x7fe714160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.419+0000 7fe713400640 1 -- 172.21.15.7:0/3167088932 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2621926773 0 0) 0x7fe6fc001110 con 0x7fe71406eb50 2024-04-25T11:15:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.420+0000 7fe713400640 1 -- 172.21.15.7:0/3167088932 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6f8003520 con 0x7fe71406eb50 2024-04-25T11:15:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.420+0000 7fe718a00640 1 -- 172.21.15.7:0/3167088932 >> v2:172.21.15.87:3300/0 conn(0x7fe71415db10 msgr2=0x7fe714160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.420+0000 7fe718a00640 1 --2- 172.21.15.7:0/3167088932 >> v2:172.21.15.87:3300/0 conn(0x7fe71415db10 0x7fe714160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.420+0000 7fe718a00640 1 -- 172.21.15.7:0/3167088932 >> v1:172.21.15.7:6789/0 conn(0x7fe71406eb50 legacy=0x7fe7140b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.420+0000 7fe718a00640 1 -- 172.21.15.7:0/3167088932 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe714160e50 con 0x7fe71409b540 2024-04-25T11:15:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.420+0000 7fe713e00640 1 --2- 172.21.15.7:0/3167088932 >> v2:172.21.15.87:3300/0 conn(0x7fe71415db10 0x7fe714160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:15:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.421+0000 7fe718a00640 1 --2- 172.21.15.7:0/3167088932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71409b540 0x7fe71409b950 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009d90 tx=0x7fe70c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261818d457ac53a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.421+0000 7fe713400640 1 -- 172.21.15.7:0/3167088932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c007740 con 0x7fe71409b540 2024-04-25T11:15:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.421+0000 7fe713400640 1 -- 172.21.15.7:0/3167088932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70c0078e0 con 0x7fe71409b540 2024-04-25T11:15:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.421+0000 7fe713400640 1 -- 172.21.15.7:0/3167088932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c007c90 con 0x7fe71409b540 2024-04-25T11:15:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.422+0000 7fe719400640 1 -- 172.21.15.7:0/3167088932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71409b540 msgr2=0x7fe71409b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.422+0000 7fe719400640 1 --2- 172.21.15.7:0/3167088932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71409b540 0x7fe71409b950 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009d90 tx=0x7fe70c02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.422+0000 7fe719400640 1 -- 172.21.15.7:0/3167088932 shutdown_connections 2024-04-25T11:15:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.422+0000 7fe719400640 1 --2- 172.21.15.7:0/3167088932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71409b540 0x7fe71409b950 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.422+0000 7fe719400640 1 --2- 172.21.15.7:0/3167088932 >> v2:172.21.15.87:3300/0 conn(0x7fe71415db10 0x7fe714160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.422+0000 7fe719400640 1 -- 172.21.15.7:0/3167088932 >> 172.21.15.7:0/3167088932 conn(0x7fe714074460 msgr2=0x7fe7140b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.422+0000 7fe719400640 1 -- 172.21.15.7:0/3167088932 shutdown_connections 2024-04-25T11:15:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.422+0000 7fe719400640 1 -- 172.21.15.7:0/3167088932 wait complete. 2024-04-25T11:15:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.425+0000 7fe719400640 1 Processor -- start 2024-04-25T11:15:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.425+0000 7fe719400640 1 -- start start 2024-04-25T11:15:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.426+0000 7fe719400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe71406eb50 0x7fe71414ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.426+0000 7fe719400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71409b540 0x7fe7141fa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.426+0000 7fe719400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe71415db10 0x7fe71414d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.426+0000 7fe719400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe71406ef60 con 0x7fe71409b540 2024-04-25T11:15:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.426+0000 7fe719400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe7140d24d0 con 0x7fe71406eb50 2024-04-25T11:15:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.426+0000 7fe719400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe7140d29f0 con 0x7fe71415db10 2024-04-25T11:15:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.427+0000 7fe70be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71409b540 0x7fe7141fa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.427+0000 7fe70be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71409b540 0x7fe7141fa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54500/0 (socket says 172.21.15.7:54500) 2024-04-25T11:15:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.427+0000 7fe70be00640 1 -- 172.21.15.7:0/3328822676 learned_addr learned my addr 172.21.15.7:0/3328822676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.427+0000 7fe70be00640 1 -- 172.21.15.7:0/3328822676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe71415db10 msgr2=0x7fe71414d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.427+0000 7fe70be00640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe71415db10 0x7fe71414d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.427+0000 7fe713e00640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe71406eb50 0x7fe71414ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.427+0000 7fe70be00640 1 -- 172.21.15.7:0/3328822676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe71406eb50 msgr2=0x7fe71414ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.427+0000 7fe70be00640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe71406eb50 0x7fe71414ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.427+0000 7fe70be00640 1 -- 172.21.15.7:0/3328822676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7141509b0 con 0x7fe71409b540 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.428+0000 7fe70be00640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71409b540 0x7fe7141fa170 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00c3a0 tx=0x7fe6fc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.428+0000 7fe710c00640 1 -- 172.21.15.7:0/3328822676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc017070 con 0x7fe71409b540 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.428+0000 7fe710c00640 1 -- 172.21.15.7:0/3328822676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6fc005ce0 con 0x7fe71409b540 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.428+0000 7fe710c00640 1 -- 172.21.15.7:0/3328822676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc007310 con 0x7fe71409b540 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.428+0000 7fe719400640 1 -- 172.21.15.7:0/3328822676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe714150c90 con 0x7fe71409b540 2024-04-25T11:15:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.428+0000 7fe719400640 1 -- 172.21.15.7:0/3328822676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7141504c0 con 0x7fe71409b540 2024-04-25T11:15:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.429+0000 7fe719400640 1 -- 172.21.15.7:0/3328822676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7141506f0 con 0x7fe71409b540 2024-04-25T11:15:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.431+0000 7fe710c00640 1 -- 172.21.15.7:0/3328822676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe6fc01a070 con 0x7fe71409b540 2024-04-25T11:15:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.432+0000 7fe710c00640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6f00744b0 0x7fe6f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.432+0000 7fe710c00640 1 -- 172.21.15.7:0/3328822676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fe6fc094770 con 0x7fe71409b540 2024-04-25T11:15:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.432+0000 7fe713e00640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6f00744b0 0x7fe6f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.433+0000 7fe713e00640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6f00744b0 0x7fe6f00769c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe704006fd0 tx=0x7fe704008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:43.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.437+0000 7fe710c00640 1 -- 172.21.15.7:0/3328822676 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe71409be90 con 0x7fe71409b540 2024-04-25T11:15:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.638+0000 7fe719400640 1 -- 172.21.15.7:0/3328822676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe714050740 con 0x7fe71409b540 2024-04-25T11:15:43.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.640+0000 7fe710c00640 1 -- 172.21.15.7:0/3328822676 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7fe6fc05edf0 con 0x7fe71409b540 2024-04-25T11:15:43.640 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.642+0000 7fe719400640 1 -- 172.21.15.7:0/3328822676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6f00744b0 msgr2=0x7fe6f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.642+0000 7fe719400640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6f00744b0 0x7fe6f00769c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe704006fd0 tx=0x7fe704008040 comp rx=0 tx=0).stop 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.642+0000 7fe719400640 1 -- 172.21.15.7:0/3328822676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71409b540 msgr2=0x7fe7141fa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.642+0000 7fe719400640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71409b540 0x7fe7141fa170 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00c3a0 tx=0x7fe6fc00c860 comp rx=0 tx=0).stop 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.642+0000 7fe719400640 1 -- 172.21.15.7:0/3328822676 shutdown_connections 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.642+0000 7fe719400640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6f00744b0 0x7fe6f00769c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.643+0000 7fe719400640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe71415db10 0x7fe71414d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.643+0000 7fe719400640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71409b540 0x7fe7141fa170 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.643+0000 7fe719400640 1 --2- 172.21.15.7:0/3328822676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe71406eb50 0x7fe71414ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.643+0000 7fe719400640 1 -- 172.21.15.7:0/3328822676 >> 172.21.15.7:0/3328822676 conn(0x7fe714074460 msgr2=0x7fe71415f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.643+0000 7fe719400640 1 -- 172.21.15.7:0/3328822676 shutdown_connections 2024-04-25T11:15:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.643+0000 7fe719400640 1 -- 172.21.15.7:0/3328822676 wait complete. 2024-04-25T11:15:43.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:15:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.946+0000 7f3c3e000640 1 Processor -- start 2024-04-25T11:15:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.946+0000 7f3c3e000640 1 -- start start 2024-04-25T11:15:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.948+0000 7f3c3e000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c380aed40 0x7f3c3815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.948+0000 7f3c3e000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c380b5220 0x7f3c380ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.948+0000 7f3c3e000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c380b64b0 con 0x7f3c380af420 2024-04-25T11:15:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.948+0000 7f3c3e000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3c3815e1f0 con 0x7f3c380aed40 2024-04-25T11:15:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.948+0000 7f3c3e000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3c3806c610 con 0x7f3c380b5220 2024-04-25T11:15:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.949+0000 7f3c3cc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c380aed40 0x7f3c3815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.949+0000 7f3c3cc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c380aed40 0x7f3c3815dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54508/0 (socket says 172.21.15.7:54508) 2024-04-25T11:15:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.949+0000 7f3c3cc00640 1 -- 172.21.15.7:0/1578250975 learned_addr learned my addr 172.21.15.7:0/1578250975 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.949+0000 7f3c3cc00640 1 -- 172.21.15.7:0/1578250975 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c380b5220 msgr2=0x7f3c380ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.949+0000 7f3c3cc00640 1 --2- 172.21.15.7:0/1578250975 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c380b5220 0x7f3c380ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.949+0000 7f3c3cc00640 1 -- 172.21.15.7:0/1578250975 >> v1:172.21.15.87:6789/0 conn(0x7f3c380af420 legacy=0x7f3c380af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.949+0000 7f3c3cc00640 1 -- 172.21.15.7:0/1578250975 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3806c8f0 con 0x7f3c380aed40 2024-04-25T11:15:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.950+0000 7f3c3cc00640 1 --2- 172.21.15.7:0/1578250975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c380aed40 0x7f3c3815dcb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3c3815e370 tx=0x7f3c2c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25a8e3944f801cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.951+0000 7f3c37400640 1 -- 172.21.15.7:0/1578250975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c037070 con 0x7f3c380aed40 2024-04-25T11:15:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.951+0000 7f3c37400640 1 -- 172.21.15.7:0/1578250975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c2c032440 con 0x7f3c380aed40 2024-04-25T11:15:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.951+0000 7f3c37400640 1 -- 172.21.15.7:0/1578250975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c032810 con 0x7f3c380aed40 2024-04-25T11:15:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.951+0000 7f3c3e000640 1 -- 172.21.15.7:0/1578250975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c380aed40 msgr2=0x7f3c3815dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.951+0000 7f3c3e000640 1 --2- 172.21.15.7:0/1578250975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c380aed40 0x7f3c3815dcb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3c3815e370 tx=0x7f3c2c02a200 comp rx=0 tx=0).stop 2024-04-25T11:15:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.952+0000 7f3c3e000640 1 -- 172.21.15.7:0/1578250975 shutdown_connections 2024-04-25T11:15:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.952+0000 7f3c3e000640 1 --2- 172.21.15.7:0/1578250975 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c380b5220 0x7f3c380ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.952+0000 7f3c3e000640 1 --2- 172.21.15.7:0/1578250975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c380aed40 0x7f3c3815dcb0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.952+0000 7f3c3e000640 1 -- 172.21.15.7:0/1578250975 >> 172.21.15.7:0/1578250975 conn(0x7f3c380b3820 msgr2=0x7f3c380b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.952+0000 7f3c3e000640 1 -- 172.21.15.7:0/1578250975 shutdown_connections 2024-04-25T11:15:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.952+0000 7f3c3e000640 1 -- 172.21.15.7:0/1578250975 wait complete. 2024-04-25T11:15:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.954+0000 7f3c3e000640 1 Processor -- start 2024-04-25T11:15:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.954+0000 7f3c3e000640 1 -- start start 2024-04-25T11:15:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c380aed40 0x7f3c38150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c380af420 0x7f3c38155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3e000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c380b5220 0x7f3c38155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3e000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3c380b6110 con 0x7f3c380aed40 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3e000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3806ceb0 con 0x7f3c380af420 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3e000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3c380b6290 con 0x7f3c380b5220 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c380aed40 0x7f3c38150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c380aed40 0x7f3c38150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54520/0 (socket says 172.21.15.7:54520) 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3cc00640 1 -- 172.21.15.7:0/1436779165 learned_addr learned my addr 172.21.15.7:0/1436779165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3cc00640 1 -- 172.21.15.7:0/1436779165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c380b5220 msgr2=0x7f3c38155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3cc00640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c380b5220 0x7f3c38155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c37e00640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c380af420 0x7f3c38155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3cc00640 1 -- 172.21.15.7:0/1436779165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c380af420 msgr2=0x7f3c38155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3cc00640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c380af420 0x7f3c38155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:43.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.956+0000 7f3c3cc00640 1 -- 172.21.15.7:0/1436779165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c38150da0 con 0x7f3c380aed40 2024-04-25T11:15:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.957+0000 7f3c3cc00640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c380aed40 0x7f3c38150860 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c000c00 tx=0x7f3c2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.957+0000 7f3c34c00640 1 -- 172.21.15.7:0/1436779165 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c041070 con 0x7f3c380aed40 2024-04-25T11:15:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.957+0000 7f3c34c00640 1 -- 172.21.15.7:0/1436779165 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c2c03c890 con 0x7f3c380aed40 2024-04-25T11:15:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.957+0000 7f3c3e000640 1 -- 172.21.15.7:0/1436779165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c38153820 con 0x7f3c380aed40 2024-04-25T11:15:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.957+0000 7f3c34c00640 1 -- 172.21.15.7:0/1436779165 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c037040 con 0x7f3c380aed40 2024-04-25T11:15:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.957+0000 7f3c3e000640 1 -- 172.21.15.7:0/1436779165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c381519e0 con 0x7f3c380aed40 2024-04-25T11:15:43.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.958+0000 7f3c3e000640 1 -- 172.21.15.7:0/1436779165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c38050740 con 0x7f3c380aed40 2024-04-25T11:15:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.960+0000 7f3c34c00640 1 -- 172.21.15.7:0/1436779165 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3c2c044070 con 0x7f3c380aed40 2024-04-25T11:15:43.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.961+0000 7f3c34c00640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c180744e0 0x7f3c180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.961+0000 7f3c34c00640 1 -- 172.21.15.7:0/1436779165 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f3c2c0b5fa0 con 0x7f3c380aed40 2024-04-25T11:15:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.961+0000 7f3c37e00640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c180744e0 0x7f3c180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:43.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.962+0000 7f3c37e00640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c180744e0 0x7f3c180769f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3c38157cb0 tx=0x7f3c24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:43.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:43.966+0000 7f3c34c00640 1 -- 172.21.15.7:0/1436779165 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c2c080590 con 0x7f3c380aed40 2024-04-25T11:15:44.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.210+0000 7f3c3e000640 1 -- 172.21.15.7:0/1436779165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c3815d010 con 0x7f3c380aed40 2024-04-25T11:15:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.211+0000 7f3c34c00640 1 -- 172.21.15.7:0/1436779165 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5171 (secure 0 0 0) 0x7f3c2c07fe90 con 0x7f3c380aed40 2024-04-25T11:15:44.211 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:44.212 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/779859707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":779859707},{"type":"v1","addr":"172.21.15.87:6834","nonce":779859707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/454388957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":454388957},{"type":"v1","addr":"172.21.15.87:6835","nonce":454388957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2318271315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2318271315},{"type":"v1","addr":"172.21.15.7:6836","nonce":2318271315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:15:05.259317+0000","modified":"2024-04-25T11:15:08.314094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6528},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6528":{"gid":6528,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2955530443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2955530443},{"type":"v1","addr":"172.21.15.7:6837","nonce":2955530443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6528,"qdb_cluster":[6528]},"id":5}]} 2024-04-25T11:15:44.212 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 43 2024-04-25T11:15:44.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 -- 172.21.15.7:0/1436779165 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c180744e0 msgr2=0x7f3c180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:44.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c180744e0 0x7f3c180769f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3c38157cb0 tx=0x7f3c24008040 comp rx=0 tx=0).stop 2024-04-25T11:15:44.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 -- 172.21.15.7:0/1436779165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c380aed40 msgr2=0x7f3c38150860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:44.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c380aed40 0x7f3c38150860 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c000c00 tx=0x7f3c2c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 -- 172.21.15.7:0/1436779165 shutdown_connections 2024-04-25T11:15:44.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c180744e0 0x7f3c180769f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c380b5220 0x7f3c38155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c380af420 0x7f3c38155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 --2- 172.21.15.7:0/1436779165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c380aed40 0x7f3c38150860 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 -- 172.21.15.7:0/1436779165 >> 172.21.15.7:0/1436779165 conn(0x7f3c380b3820 msgr2=0x7f3c380af930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:44.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 -- 172.21.15.7:0/1436779165 shutdown_connections 2024-04-25T11:15:44.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.214+0000 7f3c3e000640 1 -- 172.21.15.7:0/1436779165 wait complete. 2024-04-25T11:15:44.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:15:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.516+0000 7f04bfe00640 1 Processor -- start 2024-04-25T11:15:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.516+0000 7f04bfe00640 1 -- start start 2024-04-25T11:15:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.519+0000 7f04bfe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c0070a70 0x7f04c00ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.519+0000 7f04bfe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f04c00aedc0 0x7f04c00b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.519+0000 7f04bfe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f04c00b0a10 0x7f04c01607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.519+0000 7f04bfe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f04c015e190 con 0x7f04c00aedc0 2024-04-25T11:15:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.519+0000 7f04bfe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f04c015d610 con 0x7f04c00b0a10 2024-04-25T11:15:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.519+0000 7f04bfe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f04c015d8f0 con 0x7f04c0070a70 2024-04-25T11:15:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.519+0000 7f04b7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f04c00aedc0 0x7f04c00b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:44.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.519+0000 7f04b7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f04c00aedc0 0x7f04c00b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54524/0 (socket says 172.21.15.7:54524) 2024-04-25T11:15:44.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.519+0000 7f04b7e00640 1 -- 172.21.15.7:0/3851167561 learned_addr learned my addr 172.21.15.7:0/3851167561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:44.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.519+0000 7f04bea00640 1 --2- 172.21.15.7:0/3851167561 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c0070a70 0x7f04c00ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:44.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.519+0000 7f04bf400640 1 --2- 172.21.15.7:0/3851167561 >> v2:172.21.15.87:3300/0 conn(0x7f04c00b0a10 0x7f04c01607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:44.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.520+0000 7f04b7e00640 1 -- 172.21.15.7:0/3851167561 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c0070a70 msgr2=0x7f04c00ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.520+0000 7f04b7e00640 1 --2- 172.21.15.7:0/3851167561 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c0070a70 0x7f04c00ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.520+0000 7f04b7e00640 1 -- 172.21.15.7:0/3851167561 >> v2:172.21.15.87:3300/0 conn(0x7f04c00b0a10 msgr2=0x7f04c01607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.520+0000 7f04b7e00640 1 --2- 172.21.15.7:0/3851167561 >> v2:172.21.15.87:3300/0 conn(0x7f04c00b0a10 0x7f04c01607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.520+0000 7f04b7e00640 1 -- 172.21.15.7:0/3851167561 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c015dbd0 con 0x7f04c00aedc0 2024-04-25T11:15:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.521+0000 7f04b7e00640 1 --2- 172.21.15.7:0/3851167561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c00aedc0 0x7f04c00b04d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f04c0072ea0 tx=0x7f04a802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82afe0c5c6ddd7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.521+0000 7f04be000640 1 -- 172.21.15.7:0/3851167561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8032790 con 0x7f04c00aedc0 2024-04-25T11:15:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.522+0000 7f04be000640 1 -- 172.21.15.7:0/3851167561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a8038070 con 0x7f04c00aedc0 2024-04-25T11:15:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.522+0000 7f04be000640 1 -- 172.21.15.7:0/3851167561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8032b50 con 0x7f04c00aedc0 2024-04-25T11:15:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.522+0000 7f04bfe00640 1 -- 172.21.15.7:0/3851167561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c00aedc0 msgr2=0x7f04c00b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.522+0000 7f04bfe00640 1 --2- 172.21.15.7:0/3851167561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c00aedc0 0x7f04c00b04d0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f04c0072ea0 tx=0x7f04a802c890 comp rx=0 tx=0).stop 2024-04-25T11:15:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.523+0000 7f04bfe00640 1 -- 172.21.15.7:0/3851167561 shutdown_connections 2024-04-25T11:15:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.523+0000 7f04bfe00640 1 --2- 172.21.15.7:0/3851167561 >> v2:172.21.15.87:3300/0 conn(0x7f04c00b0a10 0x7f04c01607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.523+0000 7f04bfe00640 1 --2- 172.21.15.7:0/3851167561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c00aedc0 0x7f04c00b04d0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.523+0000 7f04bfe00640 1 --2- 172.21.15.7:0/3851167561 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c0070a70 0x7f04c00ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.523+0000 7f04bfe00640 1 -- 172.21.15.7:0/3851167561 >> 172.21.15.7:0/3851167561 conn(0x7f04c00b3820 msgr2=0x7f04c00b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.523+0000 7f04bfe00640 1 -- 172.21.15.7:0/3851167561 shutdown_connections 2024-04-25T11:15:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.523+0000 7f04bfe00640 1 -- 172.21.15.7:0/3851167561 wait complete. 2024-04-25T11:15:44.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.525+0000 7f04bfe00640 1 Processor -- start 2024-04-25T11:15:44.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.526+0000 7f04bfe00640 1 -- start start 2024-04-25T11:15:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.527+0000 7f04bfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c0070a70 0x7f04c01f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.527+0000 7f04bfe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c00aedc0 0x7f04c00d0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.527+0000 7f04bfe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c00b0a10 0x7f04c0158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.527+0000 7f04bfe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04c015d170 con 0x7f04c0070a70 2024-04-25T11:15:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.527+0000 7f04bfe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04c00b0e20 con 0x7f04c00b0a10 2024-04-25T11:15:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.527+0000 7f04bfe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f04c015d2f0 con 0x7f04c00aedc0 2024-04-25T11:15:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.527+0000 7f04bea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c0070a70 0x7f04c01f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:44.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.528+0000 7f04bea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c0070a70 0x7f04c01f6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54540/0 (socket says 172.21.15.7:54540) 2024-04-25T11:15:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.528+0000 7f04bea00640 1 -- 172.21.15.7:0/1597276310 learned_addr learned my addr 172.21.15.7:0/1597276310 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.528+0000 7f04b7e00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c00aedc0 0x7f04c00d0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.528+0000 7f04bea00640 1 -- 172.21.15.7:0/1597276310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c00aedc0 msgr2=0x7f04c00d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.528+0000 7f04bea00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c00aedc0 0x7f04c00d0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.528+0000 7f04bea00640 1 -- 172.21.15.7:0/1597276310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c00b0a10 msgr2=0x7f04c0158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.528+0000 7f04bf400640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c00b0a10 0x7f04c0158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.528+0000 7f04bea00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c00b0a10 0x7f04c0158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.528+0000 7f04bea00640 1 -- 172.21.15.7:0/1597276310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c00cca30 con 0x7f04c0070a70 2024-04-25T11:15:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.528+0000 7f04bea00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c0070a70 0x7f04c01f6b10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f04b000e8a0 tx=0x7f04b000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.529+0000 7f04b6a00640 1 -- 172.21.15.7:0/1597276310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0017070 con 0x7f04c0070a70 2024-04-25T11:15:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.529+0000 7f04b6a00640 1 -- 172.21.15.7:0/1597276310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b000ee90 con 0x7f04c0070a70 2024-04-25T11:15:44.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.529+0000 7f04bfe00640 1 -- 172.21.15.7:0/1597276310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c00ccd10 con 0x7f04c0070a70 2024-04-25T11:15:44.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.529+0000 7f04b6a00640 1 -- 172.21.15.7:0/1597276310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b000cb80 con 0x7f04c0070a70 2024-04-25T11:15:44.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.529+0000 7f04bfe00640 1 -- 172.21.15.7:0/1597276310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c0157d30 con 0x7f04c0070a70 2024-04-25T11:15:44.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.530+0000 7f04bfe00640 1 -- 172.21.15.7:0/1597276310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c0050740 con 0x7f04c0070a70 2024-04-25T11:15:44.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.531+0000 7f04b6a00640 1 -- 172.21.15.7:0/1597276310 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f04b001a070 con 0x7f04c0070a70 2024-04-25T11:15:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.532+0000 7f04b6a00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f049c0744e0 0x7f049c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:44.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.533+0000 7f04b6a00640 1 -- 172.21.15.7:0/1597276310 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f04b0095650 con 0x7f04c0070a70 2024-04-25T11:15:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.533+0000 7f04b7e00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f049c0744e0 0x7f049c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:44.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.533+0000 7f04b7e00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f049c0744e0 0x7f049c0769f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f04a8037000 tx=0x7f04a8032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:44.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.537+0000 7f04b6a00640 1 -- 172.21.15.7:0/1597276310 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f04b005fbb0 con 0x7f04c0070a70 2024-04-25T11:15:44.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.746+0000 7f04bfe00640 1 -- 172.21.15.7:0/1597276310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04c015b360 con 0x7f04c0070a70 2024-04-25T11:15:44.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.749+0000 7f04b6a00640 1 -- 172.21.15.7:0/1597276310 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16849 (secure 0 0 0) 0x7f04b005f4b0 con 0x7f04c0070a70 2024-04-25T11:15:44.750 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:44.750 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:15:06.697374+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:03.587954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T11:15:04.226794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.752+0000 7f04bfe00640 1 -- 172.21.15.7:0/1597276310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f049c0744e0 msgr2=0x7f049c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.752+0000 7f04bfe00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f049c0744e0 0x7f049c0769f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f04a8037000 tx=0x7f04a8032000 comp rx=0 tx=0).stop 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.752+0000 7f04bfe00640 1 -- 172.21.15.7:0/1597276310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c0070a70 msgr2=0x7f04c01f6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.752+0000 7f04bfe00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c0070a70 0x7f04c01f6b10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f04b000e8a0 tx=0x7f04b000ed60 comp rx=0 tx=0).stop 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.752+0000 7f04bfe00640 1 -- 172.21.15.7:0/1597276310 shutdown_connections 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.752+0000 7f04bfe00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f049c0744e0 0x7f049c0769f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.752+0000 7f04bfe00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c00b0a10 0x7f04c0158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.752+0000 7f04bfe00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04c00aedc0 0x7f04c00d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.752+0000 7f04bfe00640 1 --2- 172.21.15.7:0/1597276310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04c0070a70 0x7f04c01f6b10 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.752+0000 7f04bfe00640 1 -- 172.21.15.7:0/1597276310 >> 172.21.15.7:0/1597276310 conn(0x7f04c00b3820 msgr2=0x7f04c00b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.752+0000 7f04bfe00640 1 -- 172.21.15.7:0/1597276310 shutdown_connections 2024-04-25T11:15:44.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:44.753+0000 7f04bfe00640 1 -- 172.21.15.7:0/1597276310 wait complete. 2024-04-25T11:15:44.781 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:15:44.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:15:45.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.067+0000 7efcd0400640 1 Processor -- start 2024-04-25T11:15:45.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.067+0000 7efcd0400640 1 -- start start 2024-04-25T11:15:45.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.069+0000 7efcd0400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efcc80b0290 0x7efcc80b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.069+0000 7efcd0400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc80b4980 con 0x7efcc806dc90 2024-04-25T11:15:45.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.070+0000 7efcd0400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc80b4c60 con 0x7efcc806f300 2024-04-25T11:15:45.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.070+0000 7efcd0400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efcc80b4f40 con 0x7efcc80b0290 2024-04-25T11:15:45.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.070+0000 7efccda00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efcc806dc90 0x7efcc80afb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50526/0 (socket says 172.21.15.7:50526) 2024-04-25T11:15:45.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.070+0000 7efccda00640 1 -- 172.21.15.7:0/2612244302 learned_addr learned my addr 172.21.15.7:0/2612244302 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:45.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.070+0000 7efccd000640 1 --2- 172.21.15.7:0/2612244302 >> v2:172.21.15.87:3300/0 conn(0x7efcc80b0290 0x7efcc80b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.071+0000 7efccd000640 1 -- 172.21.15.7:0/2612244302 >> v1:172.21.15.87:6789/0 conn(0x7efcc806f300 legacy=0x7efcc80b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.071+0000 7efccd000640 1 -- 172.21.15.7:0/2612244302 >> v1:172.21.15.7:6789/0 conn(0x7efcc806dc90 legacy=0x7efcc80afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.071+0000 7efccd000640 1 -- 172.21.15.7:0/2612244302 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc80b5220 con 0x7efcc80b0290 2024-04-25T11:15:45.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.073+0000 7efccd000640 1 --2- 172.21.15.7:0/2612244302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcc80b0290 0x7efcc80b5a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efcb40097a0 tx=0x7efcb402c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf51a22373ca68f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.078+0000 7efcc7e00640 1 -- 172.21.15.7:0/2612244302 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4038070 con 0x7efcc80b0290 2024-04-25T11:15:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.078+0000 7efcc7e00640 1 -- 172.21.15.7:0/2612244302 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb4031710 con 0x7efcc80b0290 2024-04-25T11:15:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.078+0000 7efcc7e00640 1 -- 172.21.15.7:0/2612244302 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4031ae0 con 0x7efcc80b0290 2024-04-25T11:15:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.079+0000 7efcd0400640 1 -- 172.21.15.7:0/2612244302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcc80b0290 msgr2=0x7efcc80b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.079+0000 7efcd0400640 1 --2- 172.21.15.7:0/2612244302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcc80b0290 0x7efcc80b5a20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efcb40097a0 tx=0x7efcb402c440 comp rx=0 tx=0).stop 2024-04-25T11:15:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.079+0000 7efcd0400640 1 -- 172.21.15.7:0/2612244302 shutdown_connections 2024-04-25T11:15:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.079+0000 7efcd0400640 1 --2- 172.21.15.7:0/2612244302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcc80b0290 0x7efcc80b5a20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.079+0000 7efcd0400640 1 -- 172.21.15.7:0/2612244302 >> 172.21.15.7:0/2612244302 conn(0x7efcc8074200 msgr2=0x7efcc806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.079+0000 7efcd0400640 1 -- 172.21.15.7:0/2612244302 shutdown_connections 2024-04-25T11:15:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.079+0000 7efcd0400640 1 -- 172.21.15.7:0/2612244302 wait complete. 2024-04-25T11:15:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.082+0000 7efcd0400640 1 Processor -- start 2024-04-25T11:15:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.082+0000 7efcd0400640 1 -- start start 2024-04-25T11:15:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.083+0000 7efcd0400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcc806f300 0x7efcc81fa070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.083+0000 7efcd0400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcc81fa5b0 0x7efcc80b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.083+0000 7efcd0400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc80b2a80 0x7efcc815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.083+0000 7efcd0400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efcc80ce920 con 0x7efcc80b2a80 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.083+0000 7efcd0400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efcc815d220 con 0x7efcc806f300 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.083+0000 7efcd0400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efcc80ceaa0 con 0x7efcc81fa5b0 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.084+0000 7efcce400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc80b2a80 0x7efcc815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.084+0000 7efcce400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc80b2a80 0x7efcc815d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54546/0 (socket says 172.21.15.7:54546) 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.084+0000 7efcce400640 1 -- 172.21.15.7:0/1999774895 learned_addr learned my addr 172.21.15.7:0/1999774895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.084+0000 7efccda00640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcc806f300 0x7efcc81fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.084+0000 7efccd000640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcc81fa5b0 0x7efcc80b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.084+0000 7efcce400640 1 -- 172.21.15.7:0/1999774895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcc81fa5b0 msgr2=0x7efcc80b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.084+0000 7efcce400640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcc81fa5b0 0x7efcc80b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.084+0000 7efcce400640 1 -- 172.21.15.7:0/1999774895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcc806f300 msgr2=0x7efcc81fa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.084+0000 7efcce400640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcc806f300 0x7efcc81fa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.084+0000 7efcce400640 1 -- 172.21.15.7:0/1999774895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc80b2e90 con 0x7efcc80b2a80 2024-04-25T11:15:45.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.085+0000 7efcce400640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc80b2a80 0x7efcc815d430 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7efcc000c760 tx=0x7efcc000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.085+0000 7efcc5600640 1 -- 172.21.15.7:0/1999774895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0003b70 con 0x7efcc80b2a80 2024-04-25T11:15:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.085+0000 7efcc5600640 1 -- 172.21.15.7:0/1999774895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc0003d10 con 0x7efcc80b2a80 2024-04-25T11:15:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.085+0000 7efcc5600640 1 -- 172.21.15.7:0/1999774895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc000aa30 con 0x7efcc80b2a80 2024-04-25T11:15:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.085+0000 7efcd0400640 1 -- 172.21.15.7:0/1999774895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc815aa20 con 0x7efcc80b2a80 2024-04-25T11:15:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.085+0000 7efcd0400640 1 -- 172.21.15.7:0/1999774895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc815afc0 con 0x7efcc80b2a80 2024-04-25T11:15:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.086+0000 7efcd0400640 1 -- 172.21.15.7:0/1999774895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc81fabb0 con 0x7efcc80b2a80 2024-04-25T11:15:45.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.088+0000 7efcc5600640 1 -- 172.21.15.7:0/1999774895 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efcc000abd0 con 0x7efcc80b2a80 2024-04-25T11:15:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.089+0000 7efcc5600640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efca80744b0 0x7efca80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.089+0000 7efcc5600640 1 -- 172.21.15.7:0/1999774895 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7efcc0095180 con 0x7efcc80b2a80 2024-04-25T11:15:45.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.089+0000 7efccda00640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efca80744b0 0x7efca80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.090+0000 7efccda00640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efca80744b0 0x7efca80769c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efcbc000f60 tx=0x7efcbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.094+0000 7efcc5600640 1 -- 172.21.15.7:0/1999774895 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efcc80b2e90 con 0x7efcc80b2a80 2024-04-25T11:15:45.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.340+0000 7efcd0400640 1 -- 172.21.15.7:0/1999774895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcc80ce500 con 0x7efcc80b2a80 2024-04-25T11:15:45.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.342+0000 7efcc5600640 1 -- 172.21.15.7:0/1999774895 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5171 (secure 0 0 0) 0x7efcc80ce500 con 0x7efcc80b2a80 2024-04-25T11:15:45.342 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:45.342 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/779859707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":779859707},{"type":"v1","addr":"172.21.15.87:6834","nonce":779859707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/454388957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":454388957},{"type":"v1","addr":"172.21.15.87:6835","nonce":454388957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2318271315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2318271315},{"type":"v1","addr":"172.21.15.7:6836","nonce":2318271315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:15:05.259317+0000","modified":"2024-04-25T11:15:08.314094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6528},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6528":{"gid":6528,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2955530443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2955530443},{"type":"v1","addr":"172.21.15.7:6837","nonce":2955530443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6528,"qdb_cluster":[6528]},"id":5}]} 2024-04-25T11:15:45.342 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 43 2024-04-25T11:15:45.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 -- 172.21.15.7:0/1999774895 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efca80744b0 msgr2=0x7efca80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efca80744b0 0x7efca80769c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efcbc000f60 tx=0x7efcbc009040 comp rx=0 tx=0).stop 2024-04-25T11:15:45.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 -- 172.21.15.7:0/1999774895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc80b2a80 msgr2=0x7efcc815d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc80b2a80 0x7efcc815d430 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7efcc000c760 tx=0x7efcc000cc20 comp rx=0 tx=0).stop 2024-04-25T11:15:45.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 -- 172.21.15.7:0/1999774895 shutdown_connections 2024-04-25T11:15:45.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efca80744b0 0x7efca80769c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efcc80b2a80 0x7efcc815d430 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcc81fa5b0 0x7efcc80b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 --2- 172.21.15.7:0/1999774895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcc806f300 0x7efcc81fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 -- 172.21.15.7:0/1999774895 >> 172.21.15.7:0/1999774895 conn(0x7efcc8074200 msgr2=0x7efcc80aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:45.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 -- 172.21.15.7:0/1999774895 shutdown_connections 2024-04-25T11:15:45.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.345+0000 7efcd0400640 1 -- 172.21.15.7:0/1999774895 wait complete. 2024-04-25T11:15:45.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:15:45.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.653+0000 7fc366400640 1 Processor -- start 2024-04-25T11:15:45.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.654+0000 7fc366400640 1 -- start start 2024-04-25T11:15:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.656+0000 7fc366400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc36006ffd0 0x7fc360164380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.656+0000 7fc366400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc3600af390 0x7fc3600af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.656+0000 7fc366400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3600b5800 0x7fc3600b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.656+0000 7fc366400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc3600b47d0 con 0x7fc36006ffd0 2024-04-25T11:15:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.656+0000 7fc366400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc3600b4a40 con 0x7fc3600af390 2024-04-25T11:15:45.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.656+0000 7fc366400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc3600b4d20 con 0x7fc3600b5800 2024-04-25T11:15:45.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.657+0000 7fc365000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc36006ffd0 0x7fc360164380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.657+0000 7fc365000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc36006ffd0 0x7fc360164380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54556/0 (socket says 172.21.15.7:54556) 2024-04-25T11:15:45.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.657+0000 7fc365000640 1 -- 172.21.15.7:0/1595164580 learned_addr learned my addr 172.21.15.7:0/1595164580 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:45.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.657+0000 7fc35fe00640 1 --2- 172.21.15.7:0/1595164580 >> v2:172.21.15.87:3300/0 conn(0x7fc3600af390 0x7fc3600af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.657+0000 7fc365a00640 1 --2- 172.21.15.7:0/1595164580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3600b5800 0x7fc3600b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.657+0000 7fc365000640 1 -- 172.21.15.7:0/1595164580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3600b5800 msgr2=0x7fc3600b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:45.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.657+0000 7fc365000640 1 --2- 172.21.15.7:0/1595164580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3600b5800 0x7fc3600b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.657+0000 7fc365000640 1 -- 172.21.15.7:0/1595164580 >> v2:172.21.15.87:3300/0 conn(0x7fc3600af390 msgr2=0x7fc3600af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:45.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.657+0000 7fc365000640 1 --2- 172.21.15.7:0/1595164580 >> v2:172.21.15.87:3300/0 conn(0x7fc3600af390 0x7fc3600af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.657+0000 7fc365000640 1 -- 172.21.15.7:0/1595164580 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3600b5000 con 0x7fc36006ffd0 2024-04-25T11:15:45.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.658+0000 7fc365000640 1 --2- 172.21.15.7:0/1595164580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc36006ffd0 0x7fc360164380 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc36006f8b0 tx=0x7fc35402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed6b12366b71f4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:45.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.659+0000 7fc35f400640 1 -- 172.21.15.7:0/1595164580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354037070 con 0x7fc36006ffd0 2024-04-25T11:15:45.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.659+0000 7fc35f400640 1 -- 172.21.15.7:0/1595164580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc354032440 con 0x7fc36006ffd0 2024-04-25T11:15:45.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.659+0000 7fc35f400640 1 -- 172.21.15.7:0/1595164580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc354032810 con 0x7fc36006ffd0 2024-04-25T11:15:45.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.659+0000 7fc366400640 1 -- 172.21.15.7:0/1595164580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc36006ffd0 msgr2=0x7fc360164380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.659+0000 7fc366400640 1 --2- 172.21.15.7:0/1595164580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc36006ffd0 0x7fc360164380 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc36006f8b0 tx=0x7fc35402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.660+0000 7fc366400640 1 -- 172.21.15.7:0/1595164580 shutdown_connections 2024-04-25T11:15:45.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.660+0000 7fc366400640 1 --2- 172.21.15.7:0/1595164580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3600b5800 0x7fc3600b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.660+0000 7fc366400640 1 --2- 172.21.15.7:0/1595164580 >> v2:172.21.15.87:3300/0 conn(0x7fc3600af390 0x7fc3600af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.660+0000 7fc366400640 1 --2- 172.21.15.7:0/1595164580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc36006ffd0 0x7fc360164380 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.660+0000 7fc366400640 1 -- 172.21.15.7:0/1595164580 >> 172.21.15.7:0/1595164580 conn(0x7fc3600b3820 msgr2=0x7fc3600b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:45.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.660+0000 7fc366400640 1 -- 172.21.15.7:0/1595164580 shutdown_connections 2024-04-25T11:15:45.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.660+0000 7fc366400640 1 -- 172.21.15.7:0/1595164580 wait complete. 2024-04-25T11:15:45.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.662+0000 7fc366400640 1 Processor -- start 2024-04-25T11:15:45.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.662+0000 7fc366400640 1 -- start start 2024-04-25T11:15:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.664+0000 7fc366400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc36006ffd0 0x7fc36015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.664+0000 7fc366400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3600af390 0x7fc360166830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.664+0000 7fc366400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3600b5800 0x7fc360164b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.664+0000 7fc366400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc36006fbf0 con 0x7fc3600af390 2024-04-25T11:15:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.664+0000 7fc366400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc36006fa70 con 0x7fc3600b5800 2024-04-25T11:15:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.664+0000 7fc366400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc3600b6150 con 0x7fc36006ffd0 2024-04-25T11:15:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.664+0000 7fc35fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3600af390 0x7fc360166830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.664+0000 7fc35fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3600af390 0x7fc360166830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54564/0 (socket says 172.21.15.7:54564) 2024-04-25T11:15:45.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.664+0000 7fc35fe00640 1 -- 172.21.15.7:0/291558707 learned_addr learned my addr 172.21.15.7:0/291558707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:45.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.665+0000 7fc365000640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc36006ffd0 0x7fc36015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.665+0000 7fc365a00640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3600b5800 0x7fc360164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.665+0000 7fc35fe00640 1 -- 172.21.15.7:0/291558707 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc36006ffd0 msgr2=0x7fc36015a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.665+0000 7fc35fe00640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc36006ffd0 0x7fc36015a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.665+0000 7fc35fe00640 1 -- 172.21.15.7:0/291558707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3600b5800 msgr2=0x7fc360164b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.665+0000 7fc35fe00640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3600b5800 0x7fc360164b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.665+0000 7fc35fe00640 1 -- 172.21.15.7:0/291558707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36015aba0 con 0x7fc3600af390 2024-04-25T11:15:45.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.665+0000 7fc365a00640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3600b5800 0x7fc360164b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:45.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.665+0000 7fc35fe00640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3600af390 0x7fc360166830 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00d800 tx=0x7fc34c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:45.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.666+0000 7fc35cc00640 1 -- 172.21.15.7:0/291558707 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c018030 con 0x7fc3600af390 2024-04-25T11:15:45.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.666+0000 7fc35cc00640 1 -- 172.21.15.7:0/291558707 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34c0096a0 con 0x7fc3600af390 2024-04-25T11:15:45.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.666+0000 7fc35cc00640 1 -- 172.21.15.7:0/291558707 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c009870 con 0x7fc3600af390 2024-04-25T11:15:45.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.666+0000 7fc366400640 1 -- 172.21.15.7:0/291558707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36015ae80 con 0x7fc3600af390 2024-04-25T11:15:45.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.666+0000 7fc366400640 1 -- 172.21.15.7:0/291558707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36015b440 con 0x7fc3600af390 2024-04-25T11:15:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.667+0000 7fc366400640 1 -- 172.21.15.7:0/291558707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc360050740 con 0x7fc3600af390 2024-04-25T11:15:45.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.668+0000 7fc35cc00640 1 -- 172.21.15.7:0/291558707 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc34c01b070 con 0x7fc3600af390 2024-04-25T11:15:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.670+0000 7fc35cc00640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3400744b0 0x7fc3400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.670+0000 7fc35cc00640 1 -- 172.21.15.7:0/291558707 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fc34c095750 con 0x7fc3600af390 2024-04-25T11:15:45.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.670+0000 7fc365000640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3400744b0 0x7fc3400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:45.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.671+0000 7fc365000640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3400744b0 0x7fc3400769c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc354034000 tx=0x7fc35402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:45.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.675+0000 7fc35cc00640 1 -- 172.21.15.7:0/291558707 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc34c05fcb0 con 0x7fc3600af390 2024-04-25T11:15:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.926+0000 7fc366400640 1 -- 172.21.15.7:0/291558707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc36006dd50 con 0x7fc3600af390 2024-04-25T11:15:45.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.927+0000 7fc35cc00640 1 -- 172.21.15.7:0/291558707 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+5171 (secure 0 0 0) 0x7fc34c05f5b0 con 0x7fc3600af390 2024-04-25T11:15:45.928 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:45.928 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/779859707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":779859707},{"type":"v1","addr":"172.21.15.87:6834","nonce":779859707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/454388957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":454388957},{"type":"v1","addr":"172.21.15.87:6835","nonce":454388957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2318271315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2318271315},{"type":"v1","addr":"172.21.15.7:6836","nonce":2318271315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:15:05.259317+0000","modified":"2024-04-25T11:15:08.314094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6528},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6528":{"gid":6528,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2955530443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2955530443},{"type":"v1","addr":"172.21.15.7:6837","nonce":2955530443}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6528,"qdb_cluster":[6528]},"id":5}]} 2024-04-25T11:15:45.928 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 43 2024-04-25T11:15:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.930+0000 7fc366400640 1 -- 172.21.15.7:0/291558707 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3400744b0 msgr2=0x7fc3400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.930+0000 7fc366400640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3400744b0 0x7fc3400769c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc354034000 tx=0x7fc35402e000 comp rx=0 tx=0).stop 2024-04-25T11:15:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.930+0000 7fc366400640 1 -- 172.21.15.7:0/291558707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3600af390 msgr2=0x7fc360166830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.930+0000 7fc366400640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3600af390 0x7fc360166830 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00d800 tx=0x7fc34c00dcc0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.930+0000 7fc366400640 1 -- 172.21.15.7:0/291558707 shutdown_connections 2024-04-25T11:15:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.930+0000 7fc366400640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc3400744b0 0x7fc3400769c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.930+0000 7fc366400640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3600b5800 0x7fc360164b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.930+0000 7fc366400640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3600af390 0x7fc360166830 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.930+0000 7fc366400640 1 --2- 172.21.15.7:0/291558707 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc36006ffd0 0x7fc36015a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.930+0000 7fc366400640 1 -- 172.21.15.7:0/291558707 >> 172.21.15.7:0/291558707 conn(0x7fc3600b3820 msgr2=0x7fc3600b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:45.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.931+0000 7fc366400640 1 -- 172.21.15.7:0/291558707 shutdown_connections 2024-04-25T11:15:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:45.931+0000 7fc366400640 1 -- 172.21.15.7:0/291558707 wait complete. 2024-04-25T11:15:45.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:15:46.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.237+0000 7fb497e00640 1 Processor -- start 2024-04-25T11:15:46.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.237+0000 7fb497e00640 1 -- start start 2024-04-25T11:15:46.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.239+0000 7fb497e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb498164690 0x7fb498070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.240+0000 7fb497e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb4981641e0 0x7fb4980b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.240+0000 7fb497e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4980b5080 con 0x7fb498156cd0 2024-04-25T11:15:46.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.240+0000 7fb497e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb4980b5360 con 0x7fb4981641e0 2024-04-25T11:15:46.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.240+0000 7fb497e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb4980b5640 con 0x7fb498164690 2024-04-25T11:15:46.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.240+0000 7fb496000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb498156cd0 0x7fb498163ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50540/0 (socket says 172.21.15.7:50540) 2024-04-25T11:15:46.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.240+0000 7fb496000640 1 -- 172.21.15.7:0/1946607701 learned_addr learned my addr 172.21.15.7:0/1946607701 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.240+0000 7fb497400640 1 --2- 172.21.15.7:0/1946607701 >> v2:172.21.15.87:3300/0 conn(0x7fb4981641e0 0x7fb4980b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.240+0000 7fb496a00640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb498164690 0x7fb498070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.240+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1374337049 0 0) 0x7fb4980b5080 con 0x7fb498156cd0 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.240+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb47c003520 con 0x7fb498156cd0 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1060039828 0 0) 0x7fb47c003520 con 0x7fb498156cd0 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb498164690 msgr2=0x7fb498070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb498164690 0x7fb498070a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 >> v2:172.21.15.87:3300/0 conn(0x7fb4981641e0 msgr2=0x7fb4980b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 --2- 172.21.15.7:0/1946607701 >> v2:172.21.15.87:3300/0 conn(0x7fb4981641e0 0x7fb4980b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4980b5920 con 0x7fb498156cd0 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fb4880019f0 con 0x7fb498156cd0 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 >> v1:172.21.15.7:6789/0 conn(0x7fb498156cd0 legacy=0x7fb498163ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb498164690 0x7fb47c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb47c003f80 0x7fb47c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47c004d40 0x7fb47c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4980b5360 con 0x7fb47c004d40 2024-04-25T11:15:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb4980b5640 con 0x7fb47c003f80 2024-04-25T11:15:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb47c00bb20 con 0x7fb498164690 2024-04-25T11:15:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.241+0000 7fb497400640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47c004d40 0x7fb47c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.242+0000 7fb496000640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb47c003f80 0x7fb47c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.242+0000 7fb496a00640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb498164690 0x7fb47c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.242+0000 7fb497400640 1 -- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb498164690 msgr2=0x7fb47c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.242+0000 7fb497400640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb498164690 0x7fb47c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.242+0000 7fb497400640 1 -- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb47c003f80 msgr2=0x7fb47c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.242+0000 7fb497400640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb47c003f80 0x7fb47c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.242+0000 7fb497400640 1 -- 172.21.15.7:0/1946607701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c003520 con 0x7fb47c004d40 2024-04-25T11:15:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.242+0000 7fb496000640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb47c003f80 0x7fb47c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:15:46.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.243+0000 7fb497400640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47c004d40 0x7fb47c005150 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb498072300 tx=0x7fb47c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1c5ebcbab03239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:46.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.243+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490036070 con 0x7fb47c004d40 2024-04-25T11:15:46.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.243+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb490031400 con 0x7fb47c004d40 2024-04-25T11:15:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.243+0000 7fb495600640 1 -- 172.21.15.7:0/1946607701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490009690 con 0x7fb47c004d40 2024-04-25T11:15:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.244+0000 7fb497e00640 1 -- 172.21.15.7:0/1946607701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47c004d40 msgr2=0x7fb47c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.244+0000 7fb497e00640 1 --2- 172.21.15.7:0/1946607701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb47c004d40 0x7fb47c005150 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb498072300 tx=0x7fb47c005830 comp rx=0 tx=0).stop 2024-04-25T11:15:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.244+0000 7fb497400640 1 -- 172.21.15.7:0/1946607701 reap_dead start 2024-04-25T11:15:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.244+0000 7fb497e00640 1 -- 172.21.15.7:0/1946607701 shutdown_connections 2024-04-25T11:15:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.244+0000 7fb497e00640 1 -- 172.21.15.7:0/1946607701 >> 172.21.15.7:0/1946607701 conn(0x7fb4980b3820 msgr2=0x7fb4980b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.244+0000 7fb497e00640 1 -- 172.21.15.7:0/1946607701 shutdown_connections 2024-04-25T11:15:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.244+0000 7fb497e00640 1 -- 172.21.15.7:0/1946607701 wait complete. 2024-04-25T11:15:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.247+0000 7fb497e00640 1 Processor -- start 2024-04-25T11:15:46.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.247+0000 7fb497e00640 1 -- start start 2024-04-25T11:15:46.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.249+0000 7fb497e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb47c004d40 0x7fb4981595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.249+0000 7fb497e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb49814e2e0 0x7fb498144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.249+0000 7fb497e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4980b0210 0x7fb4980b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.249+0000 7fb497e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb498157260 con 0x7fb49814e2e0 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.249+0000 7fb497e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb4981570e0 con 0x7fb47c004d40 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.249+0000 7fb497e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb498153a90 con 0x7fb4980b0210 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.249+0000 7fb496000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb49814e2e0 0x7fb498144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.249+0000 7fb496000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb49814e2e0 0x7fb498144060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54586/0 (socket says 172.21.15.7:54586) 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.249+0000 7fb496000640 1 -- 172.21.15.7:0/2793854774 learned_addr learned my addr 172.21.15.7:0/2793854774 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.249+0000 7fb496a00640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb47c004d40 0x7fb4981595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb497400640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4980b0210 0x7fb4980b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb496000640 1 -- 172.21.15.7:0/2793854774 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4980b0210 msgr2=0x7fb4980b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb496000640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4980b0210 0x7fb4980b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb496000640 1 -- 172.21.15.7:0/2793854774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb47c004d40 msgr2=0x7fb4981595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb496000640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb47c004d40 0x7fb4981595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb496000640 1 -- 172.21.15.7:0/2793854774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4980b0b60 con 0x7fb49814e2e0 2024-04-25T11:15:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb496a00640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb47c004d40 0x7fb4981595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb496000640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb49814e2e0 0x7fb498144060 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb488007a60 tx=0x7fb48800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb48f400640 1 -- 172.21.15.7:0/2793854774 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48800ee50 con 0x7fb49814e2e0 2024-04-25T11:15:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb48f400640 1 -- 172.21.15.7:0/2793854774 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb488018070 con 0x7fb49814e2e0 2024-04-25T11:15:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb48f400640 1 -- 172.21.15.7:0/2793854774 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb488013620 con 0x7fb49814e2e0 2024-04-25T11:15:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb497e00640 1 -- 172.21.15.7:0/2793854774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb498159df0 con 0x7fb49814e2e0 2024-04-25T11:15:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.250+0000 7fb497e00640 1 -- 172.21.15.7:0/2793854774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49815a3b0 con 0x7fb49814e2e0 2024-04-25T11:15:46.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.252+0000 7fb497e00640 1 -- 172.21.15.7:0/2793854774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb498159b60 con 0x7fb49814e2e0 2024-04-25T11:15:46.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.253+0000 7fb48f400640 1 -- 172.21.15.7:0/2793854774 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb4880137c0 con 0x7fb49814e2e0 2024-04-25T11:15:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.254+0000 7fb48f400640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4740744e0 0x7fb4740769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.254+0000 7fb48f400640 1 -- 172.21.15.7:0/2793854774 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fb488095ab0 con 0x7fb49814e2e0 2024-04-25T11:15:46.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.254+0000 7fb496a00640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4740744e0 0x7fb4740769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.255+0000 7fb496a00640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4740744e0 0x7fb4740769f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb480009a00 tx=0x7fb480008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:46.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.259+0000 7fb48f400640 1 -- 172.21.15.7:0/2793854774 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb4980b0b60 con 0x7fb49814e2e0 2024-04-25T11:15:46.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.460+0000 7fb497e00640 1 -- 172.21.15.7:0/2793854774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb498050740 con 0x7fb49814e2e0 2024-04-25T11:15:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.463+0000 7fb48f400640 1 -- 172.21.15.7:0/2793854774 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16849 (secure 0 0 0) 0x7fb4880600a0 con 0x7fb49814e2e0 2024-04-25T11:15:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:15:06.697374+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:03.587954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T11:15:04.226794+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:15:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 -- 172.21.15.7:0/2793854774 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4740744e0 msgr2=0x7fb4740769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4740744e0 0x7fb4740769f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb480009a00 tx=0x7fb480008040 comp rx=0 tx=0).stop 2024-04-25T11:15:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 -- 172.21.15.7:0/2793854774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb49814e2e0 msgr2=0x7fb498144060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb49814e2e0 0x7fb498144060 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb488007a60 tx=0x7fb48800e530 comp rx=0 tx=0).stop 2024-04-25T11:15:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 -- 172.21.15.7:0/2793854774 shutdown_connections 2024-04-25T11:15:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4740744e0 0x7fb4740769f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4980b0210 0x7fb4980b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb49814e2e0 0x7fb498144060 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 --2- 172.21.15.7:0/2793854774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb47c004d40 0x7fb4981595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 -- 172.21.15.7:0/2793854774 >> 172.21.15.7:0/2793854774 conn(0x7fb4980b3820 msgr2=0x7fb498070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 -- 172.21.15.7:0/2793854774 shutdown_connections 2024-04-25T11:15:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.466+0000 7fb497e00640 1 -- 172.21.15.7:0/2793854774 wait complete. 2024-04-25T11:15:46.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:15:46.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.776+0000 7f7eee000640 1 Processor -- start 2024-04-25T11:15:46.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.776+0000 7f7eee000640 1 -- start start 2024-04-25T11:15:46.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.779+0000 7f7eee000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ee806d580 0x7f7ee806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.779+0000 7f7eee000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ee806ded0 con 0x7f7ee815f180 2024-04-25T11:15:46.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.779+0000 7f7eee000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ee8162700 con 0x7f7ee806f1e0 2024-04-25T11:15:46.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.779+0000 7f7eee000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee815f590 con 0x7f7ee806d580 2024-04-25T11:15:46.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.779+0000 7f7ee7400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7ee815f180 0x7f7ee8161c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50552/0 (socket says 172.21.15.7:50552) 2024-04-25T11:15:46.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.779+0000 7f7ee7400640 1 -- 172.21.15.7:0/2238882745 learned_addr learned my addr 172.21.15.7:0/2238882745 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:46.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.779+0000 7f7edfe00640 1 --2- 172.21.15.7:0/2238882745 >> v2:172.21.15.7:3300/0 conn(0x7f7ee806d580 0x7f7ee806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.780+0000 7f7ee6a00640 1 -- 172.21.15.7:0/2238882745 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 721388879 0 0) 0x7f7ed8001110 con 0x7f7ee815f180 2024-04-25T11:15:46.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.780+0000 7f7ee6a00640 1 -- 172.21.15.7:0/2238882745 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ecc003520 con 0x7f7ee815f180 2024-04-25T11:15:46.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.780+0000 7f7edfe00640 1 -- 172.21.15.7:0/2238882745 >> v1:172.21.15.87:6789/0 conn(0x7f7ee806f1e0 legacy=0x7f7ee806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.780+0000 7f7edfe00640 1 -- 172.21.15.7:0/2238882745 >> v1:172.21.15.7:6789/0 conn(0x7f7ee815f180 legacy=0x7f7ee8161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.780+0000 7f7edfe00640 1 -- 172.21.15.7:0/2238882745 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee81629e0 con 0x7f7ee806d580 2024-04-25T11:15:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.781+0000 7f7edfe00640 1 --2- 172.21.15.7:0/2238882745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee806d580 0x7f7ee806d990 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00097a0 tx=0x7f7ed002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60183e7a2efc2814 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.781+0000 7f7ee6a00640 1 -- 172.21.15.7:0/2238882745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0037070 con 0x7f7ee806d580 2024-04-25T11:15:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.781+0000 7f7ee6a00640 1 -- 172.21.15.7:0/2238882745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed0032440 con 0x7f7ee806d580 2024-04-25T11:15:46.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.781+0000 7f7ee6a00640 1 -- 172.21.15.7:0/2238882745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0032810 con 0x7f7ee806d580 2024-04-25T11:15:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.782+0000 7f7eee000640 1 -- 172.21.15.7:0/2238882745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee806d580 msgr2=0x7f7ee806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.782+0000 7f7eee000640 1 --2- 172.21.15.7:0/2238882745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee806d580 0x7f7ee806d990 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00097a0 tx=0x7f7ed002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.782+0000 7f7eee000640 1 -- 172.21.15.7:0/2238882745 shutdown_connections 2024-04-25T11:15:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.782+0000 7f7eee000640 1 --2- 172.21.15.7:0/2238882745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee806d580 0x7f7ee806d990 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.782+0000 7f7eee000640 1 -- 172.21.15.7:0/2238882745 >> 172.21.15.7:0/2238882745 conn(0x7f7ee80732b0 msgr2=0x7f7ee80736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:46.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.782+0000 7f7eee000640 1 -- 172.21.15.7:0/2238882745 shutdown_connections 2024-04-25T11:15:46.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.782+0000 7f7eee000640 1 -- 172.21.15.7:0/2238882745 wait complete. 2024-04-25T11:15:46.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.785+0000 7f7eee000640 1 Processor -- start 2024-04-25T11:15:46.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.785+0000 7f7eee000640 1 -- start start 2024-04-25T11:15:46.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.786+0000 7f7eee000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee806d580 0x7f7ee8161630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.786+0000 7f7eee000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee806f1e0 0x7f7ee81460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.786+0000 7f7eee000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ee815f180 0x7f7ee8146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.786+0000 7f7eee000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee815ec60 con 0x7f7ee806f1e0 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.786+0000 7f7eee000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee8158ae0 con 0x7f7ee806d580 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.786+0000 7f7eee000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee815e670 con 0x7f7ee815f180 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.787+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee806f1e0 0x7f7ee81460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.787+0000 7f7edfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee806f1e0 0x7f7ee81460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54608/0 (socket says 172.21.15.7:54608) 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.787+0000 7f7edfe00640 1 -- 172.21.15.7:0/3634250689 learned_addr learned my addr 172.21.15.7:0/3634250689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.787+0000 7f7ee7400640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee806d580 0x7f7ee8161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.787+0000 7f7ee7e00640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ee815f180 0x7f7ee8146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.787+0000 7f7edfe00640 1 -- 172.21.15.7:0/3634250689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ee815f180 msgr2=0x7f7ee8146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.787+0000 7f7edfe00640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ee815f180 0x7f7ee8146bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.787+0000 7f7edfe00640 1 -- 172.21.15.7:0/3634250689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee806d580 msgr2=0x7f7ee8161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.787+0000 7f7edfe00640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee806d580 0x7f7ee8161630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.787+0000 7f7edfe00640 1 -- 172.21.15.7:0/3634250689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee8160590 con 0x7f7ee806f1e0 2024-04-25T11:15:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.788+0000 7f7edfe00640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee806f1e0 0x7f7ee81460f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00098c0 tx=0x7f7ed00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.788+0000 7f7edf400640 1 -- 172.21.15.7:0/3634250689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0037040 con 0x7f7ee806f1e0 2024-04-25T11:15:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.788+0000 7f7edf400640 1 -- 172.21.15.7:0/3634250689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed0042070 con 0x7f7ee806f1e0 2024-04-25T11:15:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.788+0000 7f7edf400640 1 -- 172.21.15.7:0/3634250689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0030940 con 0x7f7ee806f1e0 2024-04-25T11:15:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.788+0000 7f7eee000640 1 -- 172.21.15.7:0/3634250689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee8160870 con 0x7f7ee806f1e0 2024-04-25T11:15:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.788+0000 7f7eee000640 1 -- 172.21.15.7:0/3634250689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee8160e30 con 0x7f7ee806f1e0 2024-04-25T11:15:46.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.790+0000 7f7eee000640 1 -- 172.21.15.7:0/3634250689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee8050740 con 0x7f7ee806f1e0 2024-04-25T11:15:46.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.791+0000 7f7edf400640 1 -- 172.21.15.7:0/3634250689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7ed0030ae0 con 0x7f7ee806f1e0 2024-04-25T11:15:46.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.792+0000 7f7edf400640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ec40744e0 0x7f7ec40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:46.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.792+0000 7f7edf400640 1 -- 172.21.15.7:0/3634250689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7ed00b68f0 con 0x7f7ee806f1e0 2024-04-25T11:15:46.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.793+0000 7f7ee7400640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ec40744e0 0x7f7ec40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:46.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.794+0000 7f7ee7400640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ec40744e0 0x7f7ec40769f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8007fd0 tx=0x7f7ed8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:46.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:46.797+0000 7f7edf400640 1 -- 172.21.15.7:0/3634250689 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7ed0080ed0 con 0x7f7ee806f1e0 2024-04-25T11:15:47.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.043+0000 7f7eee000640 1 -- 172.21.15.7:0/3634250689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7ee8160320 con 0x7f7ee806f1e0 2024-04-25T11:15:47.046 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-04-25T11:15:47.045+0000 7fac34e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:15:47.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.048+0000 7f7edf400640 1 -- 172.21.15.7:0/3634250689 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) ==== 161+0+0 (secure 0 0 0) 0x7f7ed00807d0 con 0x7f7ee806f1e0 2024-04-25T11:15:47.049 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:15:47.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.051+0000 7f7eee000640 1 -- 172.21.15.7:0/3634250689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ec40744e0 msgr2=0x7f7ec40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:47.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.051+0000 7f7eee000640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ec40744e0 0x7f7ec40769f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8007fd0 tx=0x7f7ed8009040 comp rx=0 tx=0).stop 2024-04-25T11:15:47.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.051+0000 7f7eee000640 1 -- 172.21.15.7:0/3634250689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee806f1e0 msgr2=0x7f7ee81460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:47.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.051+0000 7f7eee000640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee806f1e0 0x7f7ee81460f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00098c0 tx=0x7f7ed00046c0 comp rx=0 tx=0).stop 2024-04-25T11:15:47.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.051+0000 7f7eee000640 1 -- 172.21.15.7:0/3634250689 shutdown_connections 2024-04-25T11:15:47.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.051+0000 7f7eee000640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ec40744e0 0x7f7ec40769f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:47.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.051+0000 7f7eee000640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ee815f180 0x7f7ee8146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:47.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.051+0000 7f7eee000640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ee806f1e0 0x7f7ee81460f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:47.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.051+0000 7f7eee000640 1 --2- 172.21.15.7:0/3634250689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee806d580 0x7f7ee8161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:47.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.051+0000 7f7eee000640 1 -- 172.21.15.7:0/3634250689 >> 172.21.15.7:0/3634250689 conn(0x7f7ee80732b0 msgr2=0x7f7ee80b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:47.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.052+0000 7f7eee000640 1 -- 172.21.15.7:0/3634250689 shutdown_connections 2024-04-25T11:15:47.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.052+0000 7f7eee000640 1 -- 172.21.15.7:0/3634250689 wait complete. 2024-04-25T11:15:47.100 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:15:47.232 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:15:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.361+0000 7f8172c00640 1 Processor -- start 2024-04-25T11:15:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.362+0000 7f8172c00640 1 -- start start 2024-04-25T11:15:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.364+0000 7f8172c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f816c0d2e90 0x7f816c1616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.364+0000 7f8172c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f816c0d5190 0x7f816c161c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.364+0000 7f8172c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f816c161030 con 0x7f816c0d4d80 2024-04-25T11:15:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.364+0000 7f8172c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f816c162160 con 0x7f816c0d2e90 2024-04-25T11:15:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.364+0000 7f8172c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f816c161310 con 0x7f816c0d5190 2024-04-25T11:15:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.365+0000 7f8170e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f816c0d2e90 0x7f816c1616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.365+0000 7f8172200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f816c0d5190 0x7f816c161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.365+0000 7f8172200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f816c0d5190 0x7f816c161c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:41778/0 (socket says 172.21.15.7:41778) 2024-04-25T11:15:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.365+0000 7f8172200640 1 -- 172.21.15.7:0/2307196670 learned_addr learned my addr 172.21.15.7:0/2307196670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.366+0000 7f8170e00640 1 -- 172.21.15.7:0/2307196670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f816c0d5190 msgr2=0x7f816c161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.366+0000 7f8170e00640 1 --2- 172.21.15.7:0/2307196670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f816c0d5190 0x7f816c161c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.366+0000 7f8170e00640 1 -- 172.21.15.7:0/2307196670 >> v1:172.21.15.87:6789/0 conn(0x7f816c0d4d80 legacy=0x7f816c0d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.366+0000 7f8170e00640 1 -- 172.21.15.7:0/2307196670 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c0d4580 con 0x7f816c0d2e90 2024-04-25T11:15:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.367+0000 7f8170e00640 1 --2- 172.21.15.7:0/2307196670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f816c0d2e90 0x7f816c1616e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f81580097a0 tx=0x7f815802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad792909a5087860 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.367+0000 7f816be00640 1 -- 172.21.15.7:0/2307196670 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158037070 con 0x7f816c0d2e90 2024-04-25T11:15:47.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.368+0000 7f816be00640 1 -- 172.21.15.7:0/2307196670 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8158032440 con 0x7f816c0d2e90 2024-04-25T11:15:47.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.368+0000 7f816be00640 1 -- 172.21.15.7:0/2307196670 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158032810 con 0x7f816c0d2e90 2024-04-25T11:15:47.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.368+0000 7f8172c00640 1 -- 172.21.15.7:0/2307196670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f816c0d2e90 msgr2=0x7f816c1616e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:47.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.368+0000 7f8172c00640 1 --2- 172.21.15.7:0/2307196670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f816c0d2e90 0x7f816c1616e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f81580097a0 tx=0x7f815802a180 comp rx=0 tx=0).stop 2024-04-25T11:15:47.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.368+0000 7f8172c00640 1 -- 172.21.15.7:0/2307196670 shutdown_connections 2024-04-25T11:15:47.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.368+0000 7f8172c00640 1 --2- 172.21.15.7:0/2307196670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f816c0d5190 0x7f816c161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:47.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.368+0000 7f8172c00640 1 --2- 172.21.15.7:0/2307196670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f816c0d2e90 0x7f816c1616e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:47.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.368+0000 7f8172c00640 1 -- 172.21.15.7:0/2307196670 >> 172.21.15.7:0/2307196670 conn(0x7f816c06c750 msgr2=0x7f816c06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:47.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.368+0000 7f8172c00640 1 -- 172.21.15.7:0/2307196670 shutdown_connections 2024-04-25T11:15:47.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.368+0000 7f8172c00640 1 -- 172.21.15.7:0/2307196670 wait complete. 2024-04-25T11:15:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.371+0000 7f8172c00640 1 Processor -- start 2024-04-25T11:15:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.371+0000 7f8172c00640 1 -- start start 2024-04-25T11:15:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.372+0000 7f8172c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f816c0d4d80 0x7f816c14fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:47.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.372+0000 7f8172c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f816c0d5190 0x7f816c0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:47.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.372+0000 7f8172c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f816c1f6780 0x7f816c1f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:47.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.372+0000 7f8172c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f816c1601a0 con 0x7f816c0d4d80 2024-04-25T11:15:47.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.372+0000 7f8172c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f816c0d3e70 con 0x7f816c1f6780 2024-04-25T11:15:47.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.372+0000 7f8172c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f816c160320 con 0x7f816c0d5190 2024-04-25T11:15:47.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.372+0000 7f8171800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f816c0d4d80 0x7f816c14fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:47.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.373+0000 7f8171800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f816c0d4d80 0x7f816c14fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54624/0 (socket says 172.21.15.7:54624) 2024-04-25T11:15:47.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.373+0000 7f8171800640 1 -- 172.21.15.7:0/1534986672 learned_addr learned my addr 172.21.15.7:0/1534986672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.373+0000 7f8170e00640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f816c0d5190 0x7f816c0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.373+0000 7f8172200640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f816c1f6780 0x7f816c1f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.373+0000 7f8171800640 1 -- 172.21.15.7:0/1534986672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f816c0d5190 msgr2=0x7f816c0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.373+0000 7f8171800640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f816c0d5190 0x7f816c0cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.373+0000 7f8171800640 1 -- 172.21.15.7:0/1534986672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f816c1f6780 msgr2=0x7f816c1f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.373+0000 7f8171800640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f816c1f6780 0x7f816c1f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.373+0000 7f8171800640 1 -- 172.21.15.7:0/1534986672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c0cec30 con 0x7f816c0d4d80 2024-04-25T11:15:47.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.374+0000 7f8171800640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f816c0d4d80 0x7f816c14fd30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f816000c430 tx=0x7f816000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:47.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.374+0000 7f8169600640 1 -- 172.21.15.7:0/1534986672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160017070 con 0x7f816c0d4d80 2024-04-25T11:15:47.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.374+0000 7f8169600640 1 -- 172.21.15.7:0/1534986672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81600052d0 con 0x7f816c0d4d80 2024-04-25T11:15:47.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.374+0000 7f8169600640 1 -- 172.21.15.7:0/1534986672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160003830 con 0x7f816c0d4d80 2024-04-25T11:15:47.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.374+0000 7f8172c00640 1 -- 172.21.15.7:0/1534986672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c0b7460 con 0x7f816c0d4d80 2024-04-25T11:15:47.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.374+0000 7f8172c00640 1 -- 172.21.15.7:0/1534986672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c0ce020 con 0x7f816c0d4d80 2024-04-25T11:15:47.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.375+0000 7f8172c00640 1 -- 172.21.15.7:0/1534986672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816c160d00 con 0x7f816c0d4d80 2024-04-25T11:15:47.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.376+0000 7f8169600640 1 -- 172.21.15.7:0/1534986672 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f816001a070 con 0x7f816c0d4d80 2024-04-25T11:15:47.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.377+0000 7f8169600640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f814c0744e0 0x7f814c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:47.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.377+0000 7f8169600640 1 -- 172.21.15.7:0/1534986672 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f8160094710 con 0x7f816c0d4d80 2024-04-25T11:15:47.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.378+0000 7f8170e00640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f814c0744e0 0x7f814c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:47.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.378+0000 7f8170e00640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f814c0744e0 0x7f814c0769f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8158034000 tx=0x7f815802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.382+0000 7f8169600640 1 -- 172.21.15.7:0/1534986672 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f816c160d00 con 0x7f816c0d4d80 2024-04-25T11:15:47.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:47.625+0000 7f8172c00640 1 -- 172.21.15.7:0/1534986672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f816c050740 con 0x7f816c0d4d80 2024-04-25T11:15:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.091+0000 7f8169600640 1 -- 172.21.15.7:0/1534986672 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) ==== 104+0+0 (secure 0 0 0) 0x7f816c0b7460 con 0x7f816c0d4d80 2024-04-25T11:15:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 -- 172.21.15.7:0/1534986672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f814c0744e0 msgr2=0x7f814c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f814c0744e0 0x7f814c0769f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8158034000 tx=0x7f815802e000 comp rx=0 tx=0).stop 2024-04-25T11:15:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 -- 172.21.15.7:0/1534986672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f816c0d4d80 msgr2=0x7f816c14fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f816c0d4d80 0x7f816c14fd30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f816000c430 tx=0x7f816000c8f0 comp rx=0 tx=0).stop 2024-04-25T11:15:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 -- 172.21.15.7:0/1534986672 shutdown_connections 2024-04-25T11:15:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f814c0744e0 0x7f814c0769f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f816c1f6780 0x7f816c1f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f816c0d5190 0x7f816c0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 --2- 172.21.15.7:0/1534986672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f816c0d4d80 0x7f816c14fd30 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 -- 172.21.15.7:0/1534986672 >> 172.21.15.7:0/1534986672 conn(0x7f816c06c750 msgr2=0x7f816c0d5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 -- 172.21.15.7:0/1534986672 shutdown_connections 2024-04-25T11:15:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.095+0000 7f8172c00640 1 -- 172.21.15.7:0/1534986672 wait complete. 2024-04-25T11:15:48.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:15:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.408+0000 7f082aa00640 1 Processor -- start 2024-04-25T11:15:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.408+0000 7f082aa00640 1 -- start start 2024-04-25T11:15:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.411+0000 7f082aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f082406dc90 0x7f082415d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.411+0000 7f082aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f082415d9a0 0x7f08240cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:48.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.411+0000 7f082aa00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08241606c0 con 0x7f082413fae0 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.411+0000 7f082aa00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f08241609a0 con 0x7f082406dc90 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.411+0000 7f082aa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0824160c80 con 0x7f082415d9a0 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.411+0000 7f0828c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f082406dc90 0x7f082415d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.411+0000 7f0829600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f082413fae0 0x7f082406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50568/0 (socket says 172.21.15.7:50568) 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.411+0000 7f0829600640 1 -- 172.21.15.7:0/2020404643 learned_addr learned my addr 172.21.15.7:0/2020404643 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.412+0000 7f082a000640 1 --2- 172.21.15.7:0/2020404643 >> v2:172.21.15.87:3300/0 conn(0x7f082415d9a0 0x7f08240cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.412+0000 7f0828c00640 1 -- 172.21.15.7:0/2020404643 >> v2:172.21.15.87:3300/0 conn(0x7f082415d9a0 msgr2=0x7f08240cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.412+0000 7f0828c00640 1 --2- 172.21.15.7:0/2020404643 >> v2:172.21.15.87:3300/0 conn(0x7f082415d9a0 0x7f08240cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.412+0000 7f0828c00640 1 -- 172.21.15.7:0/2020404643 >> v1:172.21.15.7:6789/0 conn(0x7f082413fae0 legacy=0x7f082406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.412+0000 7f0828c00640 1 -- 172.21.15.7:0/2020404643 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0824160f60 con 0x7f082406dc90 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.413+0000 7f0828c00640 1 --2- 172.21.15.7:0/2020404643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f082406dc90 0x7f082415d460 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f082406e0a0 tx=0x7f081002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b579f143df922 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:48.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.413+0000 7f0823e00640 1 -- 172.21.15.7:0/2020404643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810037070 con 0x7f082406dc90 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.413+0000 7f0823e00640 1 -- 172.21.15.7:0/2020404643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0810032440 con 0x7f082406dc90 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.413+0000 7f0823e00640 1 -- 172.21.15.7:0/2020404643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081002ae60 con 0x7f082406dc90 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.414+0000 7f082aa00640 1 -- 172.21.15.7:0/2020404643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f082406dc90 msgr2=0x7f082415d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.414+0000 7f082aa00640 1 --2- 172.21.15.7:0/2020404643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f082406dc90 0x7f082415d460 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f082406e0a0 tx=0x7f081002a110 comp rx=0 tx=0).stop 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.414+0000 7f082aa00640 1 -- 172.21.15.7:0/2020404643 shutdown_connections 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.414+0000 7f082aa00640 1 --2- 172.21.15.7:0/2020404643 >> v2:172.21.15.87:3300/0 conn(0x7f082415d9a0 0x7f08240cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.414+0000 7f082aa00640 1 --2- 172.21.15.7:0/2020404643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f082406dc90 0x7f082415d460 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.414+0000 7f082aa00640 1 -- 172.21.15.7:0/2020404643 >> 172.21.15.7:0/2020404643 conn(0x7f0824073160 msgr2=0x7f0824073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.414+0000 7f082aa00640 1 -- 172.21.15.7:0/2020404643 shutdown_connections 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.414+0000 7f082aa00640 1 -- 172.21.15.7:0/2020404643 wait complete. 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.417+0000 7f082aa00640 1 Processor -- start 2024-04-25T11:15:48.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.417+0000 7f082aa00640 1 -- start start 2024-04-25T11:15:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.418+0000 7f082aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f082406dc90 0x7f08241552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.418+0000 7f082aa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f082413fae0 0x7f08241e9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f082415d9a0 0x7f08240b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082aa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f082415df30 con 0x7f082415d9a0 2024-04-25T11:15:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082aa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f082415ddb0 con 0x7f082406dc90 2024-04-25T11:15:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082aa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f08240d02d0 con 0x7f082413fae0 2024-04-25T11:15:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f082415d9a0 0x7f08240b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:48.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f082415d9a0 0x7f08240b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54636/0 (socket says 172.21.15.7:54636) 2024-04-25T11:15:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082a000640 1 -- 172.21.15.7:0/2038217177 learned_addr learned my addr 172.21.15.7:0/2038217177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f0829600640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f082406dc90 0x7f08241552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f0828c00640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f082413fae0 0x7f08241e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082a000640 1 -- 172.21.15.7:0/2038217177 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f082413fae0 msgr2=0x7f08241e9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082a000640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f082413fae0 0x7f08241e9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082a000640 1 -- 172.21.15.7:0/2038217177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f082406dc90 msgr2=0x7f08241552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082a000640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f082406dc90 0x7f08241552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f082a000640 1 -- 172.21.15.7:0/2038217177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08241ea280 con 0x7f082415d9a0 2024-04-25T11:15:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.419+0000 7f0829600640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f082406dc90 0x7f08241552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.420+0000 7f082a000640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f082415d9a0 0x7f08240b4d80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f081c007e60 tx=0x7f081c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:48.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.420+0000 7f0821600640 1 -- 172.21.15.7:0/2038217177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c014420 con 0x7f082415d9a0 2024-04-25T11:15:48.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.420+0000 7f0821600640 1 -- 172.21.15.7:0/2038217177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081c019070 con 0x7f082415d9a0 2024-04-25T11:15:48.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.420+0000 7f082aa00640 1 -- 172.21.15.7:0/2038217177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08241ea560 con 0x7f082415d9a0 2024-04-25T11:15:48.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.420+0000 7f0821600640 1 -- 172.21.15.7:0/2038217177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081c014820 con 0x7f082415d9a0 2024-04-25T11:15:48.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.420+0000 7f082aa00640 1 -- 172.21.15.7:0/2038217177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08241eab20 con 0x7f082415d9a0 2024-04-25T11:15:48.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.422+0000 7f082aa00640 1 -- 172.21.15.7:0/2038217177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08241e9ff0 con 0x7f082415d9a0 2024-04-25T11:15:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.423+0000 7f0821600640 1 -- 172.21.15.7:0/2038217177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f081c0149c0 con 0x7f082415d9a0 2024-04-25T11:15:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.424+0000 7f0821600640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08040744b0 0x7f08040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.424+0000 7f0821600640 1 -- 172.21.15.7:0/2038217177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6187+0+0 (secure 0 0 0) 0x7f081c095d10 con 0x7f082415d9a0 2024-04-25T11:15:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.424+0000 7f0829600640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08040744b0 0x7f08040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:48.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.425+0000 7f0829600640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08040744b0 0x7f08040769c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0818000f60 tx=0x7f0818009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:48.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.429+0000 7f0821600640 1 -- 172.21.15.7:0/2038217177 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f081c09b020 con 0x7f082415d9a0 2024-04-25T11:15:48.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:48.633+0000 7f082aa00640 1 -- 172.21.15.7:0/2038217177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0824050740 con 0x7f082415d9a0 2024-04-25T11:15:49.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.105+0000 7f0821600640 1 -- 172.21.15.7:0/2038217177 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) ==== 181+0+0 (secure 0 0 0) 0x7f081c060220 con 0x7f082415d9a0 2024-04-25T11:15:49.107 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:15:49.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.109+0000 7f082aa00640 1 -- 172.21.15.7:0/2038217177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08040744b0 msgr2=0x7f08040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.109+0000 7f082aa00640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08040744b0 0x7f08040769c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0818000f60 tx=0x7f0818009040 comp rx=0 tx=0).stop 2024-04-25T11:15:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.109+0000 7f082aa00640 1 -- 172.21.15.7:0/2038217177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f082415d9a0 msgr2=0x7f08240b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.109+0000 7f082aa00640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f082415d9a0 0x7f08240b4d80 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f081c007e60 tx=0x7f081c00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.110+0000 7f082aa00640 1 -- 172.21.15.7:0/2038217177 shutdown_connections 2024-04-25T11:15:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.110+0000 7f082aa00640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f08040744b0 0x7f08040769c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.110+0000 7f082aa00640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f082415d9a0 0x7f08240b4d80 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.110+0000 7f082aa00640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f082413fae0 0x7f08241e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.110+0000 7f082aa00640 1 --2- 172.21.15.7:0/2038217177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f082406dc90 0x7f08241552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.110+0000 7f082aa00640 1 -- 172.21.15.7:0/2038217177 >> 172.21.15.7:0/2038217177 conn(0x7f0824073160 msgr2=0x7f08240738a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.111+0000 7f082aa00640 1 -- 172.21.15.7:0/2038217177 shutdown_connections 2024-04-25T11:15:49.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.111+0000 7f082aa00640 1 -- 172.21.15.7:0/2038217177 wait complete. 2024-04-25T11:15:49.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:15:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.431+0000 7f254d800640 1 Processor -- start 2024-04-25T11:15:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.431+0000 7f254d800640 1 -- start start 2024-04-25T11:15:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.433+0000 7f254d800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481440a0 0x7f254815f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.433+0000 7f254d800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f254806dc90 0x7f25480cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.433+0000 7f254d800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f254815e620 con 0x7f25480cf370 2024-04-25T11:15:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.434+0000 7f254d800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f254815e900 con 0x7f254806dc90 2024-04-25T11:15:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.434+0000 7f254d800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f254815ebe0 con 0x7f25481440a0 2024-04-25T11:15:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.434+0000 7f2547400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f25480cf370 0x7f254806d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50576/0 (socket says 172.21.15.7:50576) 2024-04-25T11:15:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.434+0000 7f2547400640 1 -- 172.21.15.7:0/2909270604 learned_addr learned my addr 172.21.15.7:0/2909270604 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.434+0000 7f2547e00640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481440a0 0x7f254815f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.434+0000 7f254ce00640 1 --2- 172.21.15.7:0/2909270604 >> v2:172.21.15.87:3300/0 conn(0x7f254806dc90 0x7f25480cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3988505304 0 0) 0x7f254815e620 con 0x7f25480cf370 2024-04-25T11:15:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f252c003520 con 0x7f25480cf370 2024-04-25T11:15:49.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3836744631 0 0) 0x7f252c003520 con 0x7f25480cf370 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481440a0 msgr2=0x7f254815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481440a0 0x7f254815f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 >> v2:172.21.15.87:3300/0 conn(0x7f254806dc90 msgr2=0x7f25480cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 --2- 172.21.15.7:0/2909270604 >> v2:172.21.15.87:3300/0 conn(0x7f254806dc90 0x7f25480cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254815eec0 con 0x7f25480cf370 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2530001670 con 0x7f25480cf370 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 >> v1:172.21.15.7:6789/0 conn(0x7f25480cf370 legacy=0x7f254806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f252c0039b0 0x7f252c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c004800 0x7f252c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481440a0 0x7f252c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f254815e900 con 0x7f252c004800 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f254815ebe0 con 0x7f252c0039b0 2024-04-25T11:15:49.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f252c00baa0 con 0x7f25481440a0 2024-04-25T11:15:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.435+0000 7f2547400640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c004800 0x7f252c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.436+0000 7f2547e00640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f252c0039b0 0x7f252c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.436+0000 7f254ce00640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481440a0 0x7f252c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.436+0000 7f2547400640 1 -- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481440a0 msgr2=0x7f252c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.436+0000 7f2547400640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481440a0 0x7f252c005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.436+0000 7f2547400640 1 -- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f252c0039b0 msgr2=0x7f252c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.436+0000 7f2547400640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f252c0039b0 0x7f252c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.436+0000 7f2547400640 1 -- 172.21.15.7:0/2909270604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252c003520 con 0x7f252c004800 2024-04-25T11:15:49.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.436+0000 7f2547e00640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f252c0039b0 0x7f252c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:49.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.437+0000 7f2547400640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c004800 0x7f252c004c10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f253000c160 tx=0x7f252c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e915f7f09e3d7e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:49.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.437+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530007c80 con 0x7f252c004800 2024-04-25T11:15:49.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.437+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2530005ad0 con 0x7f252c004800 2024-04-25T11:15:49.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.437+0000 7f2546a00640 1 -- 172.21.15.7:0/2909270604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530007070 con 0x7f252c004800 2024-04-25T11:15:49.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.438+0000 7f254d800640 1 -- 172.21.15.7:0/2909270604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c004800 msgr2=0x7f252c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:49.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.438+0000 7f254d800640 1 --2- 172.21.15.7:0/2909270604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f252c004800 0x7f252c004c10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f253000c160 tx=0x7f252c005830 comp rx=0 tx=0).stop 2024-04-25T11:15:49.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.438+0000 7f254ce00640 1 -- 172.21.15.7:0/2909270604 reap_dead start 2024-04-25T11:15:49.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.438+0000 7f254d800640 1 -- 172.21.15.7:0/2909270604 shutdown_connections 2024-04-25T11:15:49.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.438+0000 7f254d800640 1 -- 172.21.15.7:0/2909270604 >> 172.21.15.7:0/2909270604 conn(0x7f2548073160 msgr2=0x7f2548073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:49.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.438+0000 7f254d800640 1 -- 172.21.15.7:0/2909270604 shutdown_connections 2024-04-25T11:15:49.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.438+0000 7f254d800640 1 -- 172.21.15.7:0/2909270604 wait complete. 2024-04-25T11:15:49.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.441+0000 7f254d800640 1 Processor -- start 2024-04-25T11:15:49.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.441+0000 7f254d800640 1 -- start start 2024-04-25T11:15:49.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.442+0000 7f254d800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f252c004800 0x7f25480cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:49.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.442+0000 7f254d800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2548136d10 0x7f2548137120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:49.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.442+0000 7f254d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2548137660 0x7f25481550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.442+0000 7f254d800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f254815e380 con 0x7f2548137660 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.442+0000 7f254d800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f254815e200 con 0x7f2548136d10 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.442+0000 7f254d800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f254815f230 con 0x7f252c004800 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.443+0000 7f254ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2548137660 0x7f25481550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.443+0000 7f254ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2548137660 0x7f25481550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54652/0 (socket says 172.21.15.7:54652) 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.443+0000 7f254ce00640 1 -- 172.21.15.7:0/3814576030 learned_addr learned my addr 172.21.15.7:0/3814576030 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.443+0000 7f2547e00640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f252c004800 0x7f25480cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.443+0000 7f2547400640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2548136d10 0x7f2548137120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.443+0000 7f254ce00640 1 -- 172.21.15.7:0/3814576030 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f252c004800 msgr2=0x7f25480cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.443+0000 7f254ce00640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f252c004800 0x7f25480cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.443+0000 7f254ce00640 1 -- 172.21.15.7:0/3814576030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2548136d10 msgr2=0x7f2548137120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.443+0000 7f254ce00640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2548136d10 0x7f2548137120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.443+0000 7f254ce00640 1 -- 172.21.15.7:0/3814576030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25480cca50 con 0x7f2548137660 2024-04-25T11:15:49.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.444+0000 7f254ce00640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2548137660 0x7f25481550e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2548073f70 tx=0x7f254000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.444+0000 7f253fe00640 1 -- 172.21.15.7:0/3814576030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254001c070 con 0x7f2548137660 2024-04-25T11:15:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.444+0000 7f253fe00640 1 -- 172.21.15.7:0/3814576030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2540017400 con 0x7f2548137660 2024-04-25T11:15:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.444+0000 7f254d800640 1 -- 172.21.15.7:0/3814576030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25480cced0 con 0x7f2548137660 2024-04-25T11:15:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.444+0000 7f253fe00640 1 -- 172.21.15.7:0/3814576030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25400177f0 con 0x7f2548137660 2024-04-25T11:15:49.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.444+0000 7f254d800640 1 -- 172.21.15.7:0/3814576030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25480cd490 con 0x7f2548137660 2024-04-25T11:15:49.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.445+0000 7f254d800640 1 -- 172.21.15.7:0/3814576030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25480ccc20 con 0x7f2548137660 2024-04-25T11:15:49.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.447+0000 7f253fe00640 1 -- 172.21.15.7:0/3814576030 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2540005d10 con 0x7f2548137660 2024-04-25T11:15:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.448+0000 7f253fe00640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25240744b0 0x7f25240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.448+0000 7f253fe00640 1 -- 172.21.15.7:0/3814576030 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5749+0+0 (secure 0 0 0) 0x7f2540095460 con 0x7f2548137660 2024-04-25T11:15:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.448+0000 7f2547e00640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25240744b0 0x7f25240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.449+0000 7f2547e00640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25240744b0 0x7f25240769c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f25380090e0 tx=0x7f2538008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:49.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.453+0000 7f253fe00640 1 -- 172.21.15.7:0/3814576030 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2540012070 con 0x7f2548137660 2024-04-25T11:15:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:49.661+0000 7f254d800640 1 -- 172.21.15.7:0/3814576030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2548073c00 con 0x7f2548137660 2024-04-25T11:15:50.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.135+0000 7f253fe00640 1 -- 172.21.15.7:0/3814576030 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) ==== 169+0+0 (secure 0 0 0) 0x7f254005fce0 con 0x7f2548137660 2024-04-25T11:15:50.137 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-25T11:15:50.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.139+0000 7f254d800640 1 -- 172.21.15.7:0/3814576030 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25240744b0 msgr2=0x7f25240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:50.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.139+0000 7f254d800640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25240744b0 0x7f25240769c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f25380090e0 tx=0x7f2538008040 comp rx=0 tx=0).stop 2024-04-25T11:15:50.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.139+0000 7f254d800640 1 -- 172.21.15.7:0/3814576030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2548137660 msgr2=0x7f25481550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:50.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.139+0000 7f254d800640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2548137660 0x7f25481550e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2548073f70 tx=0x7f254000f4a0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.139+0000 7f254d800640 1 -- 172.21.15.7:0/3814576030 shutdown_connections 2024-04-25T11:15:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.139+0000 7f254d800640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25240744b0 0x7f25240769c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.139+0000 7f254d800640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2548137660 0x7f25481550e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.139+0000 7f254d800640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2548136d10 0x7f2548137120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.139+0000 7f254d800640 1 --2- 172.21.15.7:0/3814576030 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f252c004800 0x7f25480cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.139+0000 7f254d800640 1 -- 172.21.15.7:0/3814576030 >> 172.21.15.7:0/3814576030 conn(0x7f2548073160 msgr2=0x7f25480ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.140+0000 7f254d800640 1 -- 172.21.15.7:0/3814576030 shutdown_connections 2024-04-25T11:15:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.140+0000 7f254d800640 1 -- 172.21.15.7:0/3814576030 wait complete. 2024-04-25T11:15:50.196 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_max_caps_per_client 2024-04-25T11:15:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.456+0000 7f93ba200640 1 Processor -- start 2024-04-25T11:15:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.456+0000 7f93ba200640 1 -- start start 2024-04-25T11:15:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.458+0000 7f93ba200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f93b41583c0 0x7f93b40cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.458+0000 7f93ba200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f93b40cfb50 0x7f93b40d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.458+0000 7f93ba200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93b40ce570 con 0x7f93b40d09c0 2024-04-25T11:15:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.458+0000 7f93ba200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f93b415f190 con 0x7f93b41583c0 2024-04-25T11:15:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.458+0000 7f93ba200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f93b40ce850 con 0x7f93b40cfb50 2024-04-25T11:15:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.459+0000 7f93b8e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f93b41583c0 0x7f93b40cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.459+0000 7f93b8e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f93b41583c0 0x7f93b40cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41514/0 (socket says 172.21.15.7:41514) 2024-04-25T11:15:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.459+0000 7f93b8e00640 1 -- 172.21.15.7:0/3406984587 learned_addr learned my addr 172.21.15.7:0/3406984587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.459+0000 7f93b8e00640 1 -- 172.21.15.7:0/3406984587 >> v2:172.21.15.87:3300/0 conn(0x7f93b40cfb50 msgr2=0x7f93b40d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.459+0000 7f93b3e00640 1 --2- 172.21.15.7:0/3406984587 >> v2:172.21.15.87:3300/0 conn(0x7f93b40cfb50 0x7f93b40d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.459+0000 7f93b8e00640 1 --2- 172.21.15.7:0/3406984587 >> v2:172.21.15.87:3300/0 conn(0x7f93b40cfb50 0x7f93b40d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.459+0000 7f93b8e00640 1 -- 172.21.15.7:0/3406984587 >> v1:172.21.15.87:6789/0 conn(0x7f93b40d09c0 legacy=0x7f93b406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.459+0000 7f93b8e00640 1 -- 172.21.15.7:0/3406984587 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b40ceb30 con 0x7f93b41583c0 2024-04-25T11:15:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.459+0000 7f93b3e00640 1 --2- 172.21.15.7:0/3406984587 >> v2:172.21.15.87:3300/0 conn(0x7f93b40cfb50 0x7f93b40d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.461+0000 7f93b8e00640 1 --2- 172.21.15.7:0/3406984587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93b41583c0 0x7f93b40cf610 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f93b4070ad0 tx=0x7f93a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e895c6955feda86d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.461+0000 7f93b3400640 1 -- 172.21.15.7:0/3406984587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8037070 con 0x7f93b41583c0 2024-04-25T11:15:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.461+0000 7f93b3400640 1 -- 172.21.15.7:0/3406984587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a8032440 con 0x7f93b41583c0 2024-04-25T11:15:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.461+0000 7f93b3400640 1 -- 172.21.15.7:0/3406984587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8032810 con 0x7f93b41583c0 2024-04-25T11:15:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.462+0000 7f93ba200640 1 -- 172.21.15.7:0/3406984587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93b41583c0 msgr2=0x7f93b40cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.462+0000 7f93ba200640 1 --2- 172.21.15.7:0/3406984587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93b41583c0 0x7f93b40cf610 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f93b4070ad0 tx=0x7f93a802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.462+0000 7f93ba200640 1 -- 172.21.15.7:0/3406984587 shutdown_connections 2024-04-25T11:15:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.462+0000 7f93ba200640 1 --2- 172.21.15.7:0/3406984587 >> v2:172.21.15.87:3300/0 conn(0x7f93b40cfb50 0x7f93b40d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.462+0000 7f93ba200640 1 --2- 172.21.15.7:0/3406984587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93b41583c0 0x7f93b40cf610 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.462+0000 7f93ba200640 1 -- 172.21.15.7:0/3406984587 >> 172.21.15.7:0/3406984587 conn(0x7f93b40740f0 msgr2=0x7f93b406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.462+0000 7f93ba200640 1 -- 172.21.15.7:0/3406984587 shutdown_connections 2024-04-25T11:15:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.462+0000 7f93ba200640 1 -- 172.21.15.7:0/3406984587 wait complete. 2024-04-25T11:15:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.465+0000 7f93ba200640 1 Processor -- start 2024-04-25T11:15:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.465+0000 7f93ba200640 1 -- start start 2024-04-25T11:15:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.466+0000 7f93ba200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93b40cfb50 0x7f93b4157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.466+0000 7f93ba200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93b40d09c0 0x7f93b41f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.466+0000 7f93ba200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93b41583c0 0x7f93b41416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.466+0000 7f93ba200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f93b406f5b0 con 0x7f93b41583c0 2024-04-25T11:15:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.466+0000 7f93ba200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40cf0f0 con 0x7f93b40d09c0 2024-04-25T11:15:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.466+0000 7f93ba200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f93b406f730 con 0x7f93b40cfb50 2024-04-25T11:15:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.467+0000 7f93b9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93b41583c0 0x7f93b41416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.467+0000 7f93b9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93b41583c0 0x7f93b41416e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41520/0 (socket says 172.21.15.7:41520) 2024-04-25T11:15:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.467+0000 7f93b9800640 1 -- 172.21.15.7:0/2661950682 learned_addr learned my addr 172.21.15.7:0/2661950682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:50.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.467+0000 7f93b8e00640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93b40cfb50 0x7f93b4157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.467+0000 7f93b3e00640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93b40d09c0 0x7f93b41f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.467+0000 7f93b9800640 1 -- 172.21.15.7:0/2661950682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93b40cfb50 msgr2=0x7f93b4157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.467+0000 7f93b9800640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93b40cfb50 0x7f93b4157ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.467+0000 7f93b9800640 1 -- 172.21.15.7:0/2661950682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93b40d09c0 msgr2=0x7f93b41f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.467+0000 7f93b9800640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93b40d09c0 0x7f93b41f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.467+0000 7f93b9800640 1 -- 172.21.15.7:0/2661950682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b4158010 con 0x7f93b41583c0 2024-04-25T11:15:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.467+0000 7f93b8e00640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93b40cfb50 0x7f93b4157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.468+0000 7f93b9800640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93b41583c0 0x7f93b41416e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00b9e0 tx=0x7f93ac00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.468+0000 7f93b0c00640 1 -- 172.21.15.7:0/2661950682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac019070 con 0x7f93b41583c0 2024-04-25T11:15:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.468+0000 7f93b0c00640 1 -- 172.21.15.7:0/2661950682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac00cb20 con 0x7f93b41583c0 2024-04-25T11:15:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.468+0000 7f93b0c00640 1 -- 172.21.15.7:0/2661950682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac014440 con 0x7f93b41583c0 2024-04-25T11:15:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.468+0000 7f93ba200640 1 -- 172.21.15.7:0/2661950682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b41f61a0 con 0x7f93b41583c0 2024-04-25T11:15:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.468+0000 7f93ba200640 1 -- 172.21.15.7:0/2661950682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b4156d10 con 0x7f93b41583c0 2024-04-25T11:15:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.469+0000 7f93ba200640 1 -- 172.21.15.7:0/2661950682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b4050740 con 0x7f93b41583c0 2024-04-25T11:15:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.471+0000 7f93b0c00640 1 -- 172.21.15.7:0/2661950682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f93ac01c070 con 0x7f93b41583c0 2024-04-25T11:15:50.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.472+0000 7f93b0c00640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93940744b0 0x7f93940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:50.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.472+0000 7f93b0c00640 1 -- 172.21.15.7:0/2661950682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5335+0+0 (secure 0 0 0) 0x7f93ac094de0 con 0x7f93b41583c0 2024-04-25T11:15:50.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.472+0000 7f93b8e00640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93940744b0 0x7f93940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:50.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.473+0000 7f93b8e00640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93940744b0 0x7f93940769c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f93a8034000 tx=0x7f93a802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:50.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.476+0000 7f93b0c00640 1 -- 172.21.15.7:0/2661950682 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f93ac05f6d0 con 0x7f93b41583c0 2024-04-25T11:15:50.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.681+0000 7f93ba200640 1 -- 172.21.15.7:0/2661950682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"} v 0) -- 0x7f93b4070dd0 con 0x7f93b41583c0 2024-04-25T11:15:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.687+0000 7f93b0c00640 1 -- 172.21.15.7:0/2661950682 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_max_caps_per_client"}]=0 v22) ==== 106+0+0 (secure 0 0 0) 0x7f93ac05efd0 con 0x7f93b41583c0 2024-04-25T11:15:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 -- 172.21.15.7:0/2661950682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93940744b0 msgr2=0x7f93940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93940744b0 0x7f93940769c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f93a8034000 tx=0x7f93a802e000 comp rx=0 tx=0).stop 2024-04-25T11:15:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 -- 172.21.15.7:0/2661950682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93b41583c0 msgr2=0x7f93b41416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93b41583c0 0x7f93b41416e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00b9e0 tx=0x7f93ac00bea0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 -- 172.21.15.7:0/2661950682 shutdown_connections 2024-04-25T11:15:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93940744b0 0x7f93940769c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93b41583c0 0x7f93b41416e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93b40d09c0 0x7f93b41f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 --2- 172.21.15.7:0/2661950682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93b40cfb50 0x7f93b4157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 -- 172.21.15.7:0/2661950682 >> 172.21.15.7:0/2661950682 conn(0x7f93b40740f0 msgr2=0x7f93b4070110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 -- 172.21.15.7:0/2661950682 shutdown_connections 2024-04-25T11:15:50.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.690+0000 7f93ba200640 1 -- 172.21.15.7:0/2661950682 wait complete. 2024-04-25T11:15:50.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:15:50.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.997+0000 7f831ca00640 1 Processor -- start 2024-04-25T11:15:50.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.997+0000 7f831ca00640 1 -- start start 2024-04-25T11:15:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.999+0000 7f831ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f83180b5580 0x7f83180cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:50.999+0000 7f831ca00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83180b5990 con 0x7f83180cf560 2024-04-25T11:15:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.000+0000 7f831ca00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83180cf970 con 0x7f83180b4690 2024-04-25T11:15:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.000+0000 7f831ca00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f83180b5c40 con 0x7f83180b5580 2024-04-25T11:15:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.000+0000 7f830fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f83180b5580 0x7f83180cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.000+0000 7f8317e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f83180cf560 0x7f8318144220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53582/0 (socket says 172.21.15.7:53582) 2024-04-25T11:15:51.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.000+0000 7f8317e00640 1 -- 172.21.15.7:0/2368416694 learned_addr learned my addr 172.21.15.7:0/2368416694 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.001+0000 7f830fe00640 1 -- 172.21.15.7:0/2368416694 >> v1:172.21.15.87:6789/0 conn(0x7f83180b4690 legacy=0x7f8318143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.001+0000 7f830fe00640 1 -- 172.21.15.7:0/2368416694 >> v1:172.21.15.7:6789/0 conn(0x7f83180cf560 legacy=0x7f8318144220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.001+0000 7f830fe00640 1 -- 172.21.15.7:0/2368416694 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83180b5f20 con 0x7f83180b5580 2024-04-25T11:15:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.001+0000 7f8316a00640 1 -- 172.21.15.7:0/2368416694 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 110970808 0 0) 0x7f83180b5990 con 0x7f83180cf560 2024-04-25T11:15:51.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.002+0000 7f830fe00640 1 --2- 172.21.15.7:0/2368416694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83180b5580 0x7f83180cf020 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f83000097a0 tx=0x7f830002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c76bbce7c6d540a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:51.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.002+0000 7f8316a00640 1 -- 172.21.15.7:0/2368416694 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300037070 con 0x7f83180b5580 2024-04-25T11:15:51.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.002+0000 7f8316a00640 1 -- 172.21.15.7:0/2368416694 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8300032640 con 0x7f83180b5580 2024-04-25T11:15:51.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.002+0000 7f8316a00640 1 -- 172.21.15.7:0/2368416694 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300032a10 con 0x7f83180b5580 2024-04-25T11:15:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.003+0000 7f831ca00640 1 -- 172.21.15.7:0/2368416694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83180b5580 msgr2=0x7f83180cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.003+0000 7f831ca00640 1 --2- 172.21.15.7:0/2368416694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83180b5580 0x7f83180cf020 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f83000097a0 tx=0x7f830002a2c0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.003+0000 7f831ca00640 1 -- 172.21.15.7:0/2368416694 shutdown_connections 2024-04-25T11:15:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.003+0000 7f831ca00640 1 --2- 172.21.15.7:0/2368416694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83180b5580 0x7f83180cf020 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.003+0000 7f831ca00640 1 -- 172.21.15.7:0/2368416694 >> 172.21.15.7:0/2368416694 conn(0x7f8318074140 msgr2=0x7f831806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.003+0000 7f831ca00640 1 -- 172.21.15.7:0/2368416694 shutdown_connections 2024-04-25T11:15:51.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.003+0000 7f831ca00640 1 -- 172.21.15.7:0/2368416694 wait complete. 2024-04-25T11:15:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.006+0000 7f831ca00640 1 Processor -- start 2024-04-25T11:15:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.006+0000 7f831ca00640 1 -- start start 2024-04-25T11:15:51.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.008+0000 7f831ca00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83180b4690 0x7f8318136f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:51.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.008+0000 7f831ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83180b5580 0x7f831813a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:51.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.008+0000 7f831ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83180cf560 0x7f8318145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.008+0000 7f831ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f83180b67c0 con 0x7f83180b5580 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.008+0000 7f831ca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f83180b64e0 con 0x7f83180cf560 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.008+0000 7f831ca00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8318070fa0 con 0x7f83180b4690 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.009+0000 7f830fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83180b5580 0x7f831813a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.009+0000 7f830fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83180b5580 0x7f831813a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41536/0 (socket says 172.21.15.7:41536) 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.009+0000 7f830fe00640 1 -- 172.21.15.7:0/4021653123 learned_addr learned my addr 172.21.15.7:0/4021653123 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.009+0000 7f8317400640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83180b4690 0x7f8318136f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.009+0000 7f8317e00640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83180cf560 0x7f8318145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.009+0000 7f830fe00640 1 -- 172.21.15.7:0/4021653123 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83180b4690 msgr2=0x7f8318136f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.009+0000 7f830fe00640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83180b4690 0x7f8318136f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.009+0000 7f830fe00640 1 -- 172.21.15.7:0/4021653123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83180cf560 msgr2=0x7f8318145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.009+0000 7f830fe00640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83180cf560 0x7f8318145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.009+0000 7f830fe00640 1 -- 172.21.15.7:0/4021653123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8318137440 con 0x7f83180b5580 2024-04-25T11:15:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.010+0000 7f830fe00640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83180b5580 0x7f831813a3a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f8300000c00 tx=0x7f83000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.010+0000 7f830f400640 1 -- 172.21.15.7:0/4021653123 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300041020 con 0x7f83180b5580 2024-04-25T11:15:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.010+0000 7f830f400640 1 -- 172.21.15.7:0/4021653123 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8300030970 con 0x7f83180b5580 2024-04-25T11:15:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.010+0000 7f830f400640 1 -- 172.21.15.7:0/4021653123 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300037040 con 0x7f83180b5580 2024-04-25T11:15:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.010+0000 7f831ca00640 1 -- 172.21.15.7:0/4021653123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83181f5e90 con 0x7f83180b5580 2024-04-25T11:15:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.010+0000 7f831ca00640 1 -- 172.21.15.7:0/4021653123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83181f6450 con 0x7f83180b5580 2024-04-25T11:15:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.012+0000 7f831ca00640 1 -- 172.21.15.7:0/4021653123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83180cfe10 con 0x7f83180b5580 2024-04-25T11:15:51.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.013+0000 7f830f400640 1 -- 172.21.15.7:0/4021653123 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8300044020 con 0x7f83180b5580 2024-04-25T11:15:51.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.015+0000 7f830f400640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82f40744e0 0x7f82f40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:51.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.015+0000 7f830f400640 1 -- 172.21.15.7:0/4021653123 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5335+0+0 (secure 0 0 0) 0x7f83000b5880 con 0x7f83180b5580 2024-04-25T11:15:51.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.015+0000 7f8317400640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82f40744e0 0x7f82f40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:51.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.016+0000 7f8317400640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82f40744e0 0x7f82f40769f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8308000f60 tx=0x7f8308009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:51.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.021+0000 7f830f400640 1 -- 172.21.15.7:0/4021653123 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83180cfe10 con 0x7f83180b5580 2024-04-25T11:15:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.229+0000 7f831ca00640 1 -- 172.21.15.7:0/4021653123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f8318050740 con 0x7f83180b5580 2024-04-25T11:15:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.237+0000 7f830f400640 1 -- 172.21.15.7:0/4021653123 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f830004b070 con 0x7f83180b5580 2024-04-25T11:15:51.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.240+0000 7f831ca00640 1 -- 172.21.15.7:0/4021653123 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82f40744e0 msgr2=0x7f82f40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.240+0000 7f831ca00640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82f40744e0 0x7f82f40769f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8308000f60 tx=0x7f8308009040 comp rx=0 tx=0).stop 2024-04-25T11:15:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.240+0000 7f831ca00640 1 -- 172.21.15.7:0/4021653123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83180b5580 msgr2=0x7f831813a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.240+0000 7f831ca00640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83180b5580 0x7f831813a3a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f8300000c00 tx=0x7f83000046c0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.240+0000 7f831ca00640 1 -- 172.21.15.7:0/4021653123 shutdown_connections 2024-04-25T11:15:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.240+0000 7f831ca00640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82f40744e0 0x7f82f40769f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.240+0000 7f831ca00640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83180cf560 0x7f8318145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.240+0000 7f831ca00640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83180b5580 0x7f831813a3a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.240+0000 7f831ca00640 1 --2- 172.21.15.7:0/4021653123 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83180b4690 0x7f8318136f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.240+0000 7f831ca00640 1 -- 172.21.15.7:0/4021653123 >> 172.21.15.7:0/4021653123 conn(0x7f8318074140 msgr2=0x7f831813fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.241+0000 7f831ca00640 1 -- 172.21.15.7:0/4021653123 shutdown_connections 2024-04-25T11:15:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.241+0000 7f831ca00640 1 -- 172.21.15.7:0/4021653123 wait complete. 2024-04-25T11:15:51.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps' 2024-04-25T11:15:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.554+0000 7feab3200640 1 Processor -- start 2024-04-25T11:15:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.554+0000 7feab3200640 1 -- start start 2024-04-25T11:15:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.557+0000 7feab3200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feab4168ac0 0x7feab406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.557+0000 7feab3200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feab406dac0 0x7feab40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.557+0000 7feab3200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feab406ded0 con 0x7feab414ab20 2024-04-25T11:15:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.557+0000 7feab3200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7feab406f920 con 0x7feab4168ac0 2024-04-25T11:15:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.557+0000 7feab3200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feab406fc00 con 0x7feab406dac0 2024-04-25T11:15:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.557+0000 7feab1e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7feab414ab20 0x7feab406f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53594/0 (socket says 172.21.15.7:53594) 2024-04-25T11:15:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.557+0000 7feab1e00640 1 -- 172.21.15.7:0/1725340732 learned_addr learned my addr 172.21.15.7:0/1725340732 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.557+0000 7feab1400640 1 --2- 172.21.15.7:0/1725340732 >> v2:172.21.15.7:3300/0 conn(0x7feab4168ac0 0x7feab406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.557+0000 7feab2800640 1 --2- 172.21.15.7:0/1725340732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feab406dac0 0x7feab40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:51.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.558+0000 7feab1400640 1 -- 172.21.15.7:0/1725340732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feab406dac0 msgr2=0x7feab40b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:51.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.558+0000 7feab1400640 1 --2- 172.21.15.7:0/1725340732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feab406dac0 0x7feab40b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.558+0000 7feab0a00640 1 -- 172.21.15.7:0/1725340732 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3660295831 0 0) 0x7feaa4001490 con 0x7feab414ab20 2024-04-25T11:15:51.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.558+0000 7feab1400640 1 -- 172.21.15.7:0/1725340732 >> v1:172.21.15.7:6789/0 conn(0x7feab414ab20 legacy=0x7feab406f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:51.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.558+0000 7feab1400640 1 -- 172.21.15.7:0/1725340732 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab406fee0 con 0x7feab4168ac0 2024-04-25T11:15:51.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.559+0000 7feab1400640 1 --2- 172.21.15.7:0/1725340732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4168ac0 0x7feab406d580 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7feaa00097a0 tx=0x7feaa002a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11541363083be5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:51.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.559+0000 7feab0a00640 1 -- 172.21.15.7:0/1725340732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0037070 con 0x7feab4168ac0 2024-04-25T11:15:51.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.559+0000 7feab0a00640 1 -- 172.21.15.7:0/1725340732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa002ae40 con 0x7feab4168ac0 2024-04-25T11:15:51.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.559+0000 7feab0a00640 1 -- 172.21.15.7:0/1725340732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0032660 con 0x7feab4168ac0 2024-04-25T11:15:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.560+0000 7feab3200640 1 -- 172.21.15.7:0/1725340732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4168ac0 msgr2=0x7feab406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.560+0000 7feab3200640 1 --2- 172.21.15.7:0/1725340732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4168ac0 0x7feab406d580 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7feaa00097a0 tx=0x7feaa002a0f0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.560+0000 7feab3200640 1 -- 172.21.15.7:0/1725340732 shutdown_connections 2024-04-25T11:15:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.560+0000 7feab3200640 1 --2- 172.21.15.7:0/1725340732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feab406dac0 0x7feab40b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.560+0000 7feab3200640 1 --2- 172.21.15.7:0/1725340732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4168ac0 0x7feab406d580 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.560+0000 7feab3200640 1 -- 172.21.15.7:0/1725340732 >> 172.21.15.7:0/1725340732 conn(0x7feab4073090 msgr2=0x7feab40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.560+0000 7feab3200640 1 -- 172.21.15.7:0/1725340732 shutdown_connections 2024-04-25T11:15:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.560+0000 7feab3200640 1 -- 172.21.15.7:0/1725340732 wait complete. 2024-04-25T11:15:51.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.562+0000 7feab3200640 1 Processor -- start 2024-04-25T11:15:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.563+0000 7feab3200640 1 -- start start 2024-04-25T11:15:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.564+0000 7feab3200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feab406dac0 0x7feab414c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.564+0000 7feab3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4168ac0 0x7feab4163ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.564+0000 7feab3200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feab414d920 0x7feab40ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.564+0000 7feab3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feab40b53d0 con 0x7feab4168ac0 2024-04-25T11:15:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.564+0000 7feab3200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feab40b4970 con 0x7feab414d920 2024-04-25T11:15:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.564+0000 7feab3200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feab40b5550 con 0x7feab406dac0 2024-04-25T11:15:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.565+0000 7feab1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4168ac0 0x7feab4163ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.565+0000 7feab1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4168ac0 0x7feab4163ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41546/0 (socket says 172.21.15.7:41546) 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.565+0000 7feab1400640 1 -- 172.21.15.7:0/258986464 learned_addr learned my addr 172.21.15.7:0/258986464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.565+0000 7feab1e00640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feab406dac0 0x7feab414c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.565+0000 7feab2800640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feab414d920 0x7feab40ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.565+0000 7feab1400640 1 -- 172.21.15.7:0/258986464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feab406dac0 msgr2=0x7feab414c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.565+0000 7feab1400640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feab406dac0 0x7feab414c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.565+0000 7feab1400640 1 -- 172.21.15.7:0/258986464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feab414d920 msgr2=0x7feab40ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.565+0000 7feab1400640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feab414d920 0x7feab40ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.565+0000 7feab1400640 1 -- 172.21.15.7:0/258986464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab40c8c80 con 0x7feab4168ac0 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.565+0000 7feab1e00640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feab406dac0 0x7feab414c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.566+0000 7feab1400640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4168ac0 0x7feab4163ea0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7feaa0000c00 tx=0x7feaa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.566+0000 7feaaa000640 1 -- 172.21.15.7:0/258986464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0041070 con 0x7feab4168ac0 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.566+0000 7feaaa000640 1 -- 172.21.15.7:0/258986464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa003c670 con 0x7feab4168ac0 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.566+0000 7feaaa000640 1 -- 172.21.15.7:0/258986464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0037040 con 0x7feab4168ac0 2024-04-25T11:15:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.566+0000 7feab3200640 1 -- 172.21.15.7:0/258986464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab414b6d0 con 0x7feab4168ac0 2024-04-25T11:15:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.566+0000 7feab3200640 1 -- 172.21.15.7:0/258986464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab414bc90 con 0x7feab4168ac0 2024-04-25T11:15:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.567+0000 7feab3200640 1 -- 172.21.15.7:0/258986464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab40c89f0 con 0x7feab4168ac0 2024-04-25T11:15:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.568+0000 7feaaa000640 1 -- 172.21.15.7:0/258986464 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feaa0044070 con 0x7feab4168ac0 2024-04-25T11:15:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.569+0000 7feaaa000640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea940744b0 0x7fea940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.570+0000 7feaaa000640 1 -- 172.21.15.7:0/258986464 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5335+0+0 (secure 0 0 0) 0x7feaa00b53c0 con 0x7feab4168ac0 2024-04-25T11:15:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.570+0000 7feab1e00640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea940744b0 0x7fea940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.571+0000 7feab1e00640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea940744b0 0x7fea940769c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7feaa4007fd0 tx=0x7feaa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:51.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.575+0000 7feaaa000640 1 -- 172.21.15.7:0/258986464 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feaa007fcb0 con 0x7feab4168ac0 2024-04-25T11:15:51.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:51.860+0000 7feab3200640 1 -- 172.21.15.7:0/258986464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]} v 0) -- 0x7feab4050740 con 0x7feab4168ac0 2024-04-25T11:15:52.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.141+0000 7feaaa000640 1 -- 172.21.15.7:0/258986464 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_max_caps"]}]=0 v303) ==== 148+0+0 (secure 0 0 0) 0x7feaa007f5b0 con 0x7feab4168ac0 2024-04-25T11:15:52.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.145+0000 7feab3200640 1 -- 172.21.15.7:0/258986464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea940744b0 msgr2=0x7fea940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:52.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.145+0000 7feab3200640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea940744b0 0x7fea940769c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7feaa4007fd0 tx=0x7feaa4009040 comp rx=0 tx=0).stop 2024-04-25T11:15:52.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.145+0000 7feab3200640 1 -- 172.21.15.7:0/258986464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4168ac0 msgr2=0x7feab4163ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:52.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.145+0000 7feab3200640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4168ac0 0x7feab4163ea0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7feaa0000c00 tx=0x7feaa00046c0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.146+0000 7feab3200640 1 -- 172.21.15.7:0/258986464 shutdown_connections 2024-04-25T11:15:52.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.146+0000 7feab3200640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fea940744b0 0x7fea940769c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.146+0000 7feab3200640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feab414d920 0x7feab40ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.146+0000 7feab3200640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feab4168ac0 0x7feab4163ea0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.146+0000 7feab3200640 1 --2- 172.21.15.7:0/258986464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feab406dac0 0x7feab414c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.146+0000 7feab3200640 1 -- 172.21.15.7:0/258986464 >> 172.21.15.7:0/258986464 conn(0x7feab4073090 msgr2=0x7feab40b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:52.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.146+0000 7feab3200640 1 -- 172.21.15.7:0/258986464 shutdown_connections 2024-04-25T11:15:52.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.146+0000 7feab3200640 1 -- 172.21.15.7:0/258986464 wait complete. 2024-04-25T11:15:52.181 INFO:tasks.cephfs_test_runner:That the MDS will not let a client sit above mds_max_caps_per_client caps. ... ok 2024-04-25T11:15:52.181 INFO:tasks.cephfs_test_runner:Starting test: test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) When a client has inodes pinned in its cache (open files), that the MDS 2024-04-25T11:15:52.181 INFO:tasks.cephfs_test_runner:test_client_min_caps_working_set (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-25T11:15:52.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:15:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.467+0000 7ff573e00640 1 Processor -- start 2024-04-25T11:15:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.467+0000 7ff573e00640 1 -- start start 2024-04-25T11:15:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.469+0000 7ff573e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff574163ac0 0x7ff5740b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.469+0000 7ff573e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff57406dcd0 0x7ff57406e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.469+0000 7ff573e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5740d0570 0x7ff5740d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.469+0000 7ff573e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff5740d35a0 con 0x7ff57406dcd0 2024-04-25T11:15:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.469+0000 7ff573e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff57406e620 con 0x7ff574163ac0 2024-04-25T11:15:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.469+0000 7ff573e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff574164310 con 0x7ff5740d0570 2024-04-25T11:15:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.470+0000 7ff572000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff57406dcd0 0x7ff57406e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.470+0000 7ff572000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff57406dcd0 0x7ff57406e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41552/0 (socket says 172.21.15.7:41552) 2024-04-25T11:15:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.470+0000 7ff572000640 1 -- 172.21.15.7:0/1561034611 learned_addr learned my addr 172.21.15.7:0/1561034611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.470+0000 7ff572a00640 1 --2- 172.21.15.7:0/1561034611 >> v2:172.21.15.87:3300/0 conn(0x7ff574163ac0 0x7ff5740b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.470+0000 7ff573400640 1 --2- 172.21.15.7:0/1561034611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5740d0570 0x7ff5740d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.470+0000 7ff572000640 1 -- 172.21.15.7:0/1561034611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5740d0570 msgr2=0x7ff5740d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.470+0000 7ff572000640 1 --2- 172.21.15.7:0/1561034611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5740d0570 0x7ff5740d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.470+0000 7ff572000640 1 -- 172.21.15.7:0/1561034611 >> v2:172.21.15.87:3300/0 conn(0x7ff574163ac0 msgr2=0x7ff5740b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.470+0000 7ff572000640 1 --2- 172.21.15.7:0/1561034611 >> v2:172.21.15.87:3300/0 conn(0x7ff574163ac0 0x7ff5740b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.470+0000 7ff572000640 1 -- 172.21.15.7:0/1561034611 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5741645f0 con 0x7ff57406dcd0 2024-04-25T11:15:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.472+0000 7ff572000640 1 --2- 172.21.15.7:0/1561034611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff57406dcd0 0x7ff57406e0e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff57408e560 tx=0x7ff55c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cc811cf3318e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.472+0000 7ff571600640 1 -- 172.21.15.7:0/1561034611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c036070 con 0x7ff57406dcd0 2024-04-25T11:15:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.472+0000 7ff571600640 1 -- 172.21.15.7:0/1561034611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c031600 con 0x7ff57406dcd0 2024-04-25T11:15:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.472+0000 7ff571600640 1 -- 172.21.15.7:0/1561034611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0319d0 con 0x7ff57406dcd0 2024-04-25T11:15:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.473+0000 7ff573e00640 1 -- 172.21.15.7:0/1561034611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff57406dcd0 msgr2=0x7ff57406e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.473+0000 7ff573e00640 1 --2- 172.21.15.7:0/1561034611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff57406dcd0 0x7ff57406e0e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff57408e560 tx=0x7ff55c02a610 comp rx=0 tx=0).stop 2024-04-25T11:15:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.473+0000 7ff573e00640 1 -- 172.21.15.7:0/1561034611 shutdown_connections 2024-04-25T11:15:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.473+0000 7ff573e00640 1 --2- 172.21.15.7:0/1561034611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5740d0570 0x7ff5740d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.473+0000 7ff573e00640 1 --2- 172.21.15.7:0/1561034611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff57406dcd0 0x7ff57406e0e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.473+0000 7ff573e00640 1 --2- 172.21.15.7:0/1561034611 >> v2:172.21.15.87:3300/0 conn(0x7ff574163ac0 0x7ff5740b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.473+0000 7ff573e00640 1 -- 172.21.15.7:0/1561034611 >> 172.21.15.7:0/1561034611 conn(0x7ff5740d29e0 msgr2=0x7ff5740d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:52.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.473+0000 7ff573e00640 1 -- 172.21.15.7:0/1561034611 shutdown_connections 2024-04-25T11:15:52.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.473+0000 7ff573e00640 1 -- 172.21.15.7:0/1561034611 wait complete. 2024-04-25T11:15:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.476+0000 7ff573e00640 1 Processor -- start 2024-04-25T11:15:52.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.476+0000 7ff573e00640 1 -- start start 2024-04-25T11:15:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.478+0000 7ff573e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff57406dcd0 0x7ff57415c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.478+0000 7ff573e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5740d0570 0x7ff574154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff573e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff574163ac0 0x7ff5741587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff573e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff574163ed0 con 0x7ff57406dcd0 2024-04-25T11:15:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff573e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff5740b50d0 con 0x7ff5740d0570 2024-04-25T11:15:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff573e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff574164050 con 0x7ff574163ac0 2024-04-25T11:15:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff572a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff57406dcd0 0x7ff57415c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff572a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff57406dcd0 0x7ff57415c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41568/0 (socket says 172.21.15.7:41568) 2024-04-25T11:15:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff572a00640 1 -- 172.21.15.7:0/1960687168 learned_addr learned my addr 172.21.15.7:0/1960687168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff572a00640 1 -- 172.21.15.7:0/1960687168 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff574163ac0 msgr2=0x7ff5741587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff572a00640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff574163ac0 0x7ff5741587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff572a00640 1 -- 172.21.15.7:0/1960687168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5740d0570 msgr2=0x7ff574154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff572a00640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5740d0570 0x7ff574154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.479+0000 7ff572a00640 1 -- 172.21.15.7:0/1960687168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff574155930 con 0x7ff57406dcd0 2024-04-25T11:15:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.480+0000 7ff572a00640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff57406dcd0 0x7ff57415c4e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff56400d5f0 tx=0x7ff56400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.480+0000 7ff56aa00640 1 -- 172.21.15.7:0/1960687168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5640125f0 con 0x7ff57406dcd0 2024-04-25T11:15:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.480+0000 7ff56aa00640 1 -- 172.21.15.7:0/1960687168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff564019070 con 0x7ff57406dcd0 2024-04-25T11:15:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.480+0000 7ff573e00640 1 -- 172.21.15.7:0/1960687168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5741565f0 con 0x7ff57406dcd0 2024-04-25T11:15:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.480+0000 7ff56aa00640 1 -- 172.21.15.7:0/1960687168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5640129d0 con 0x7ff57406dcd0 2024-04-25T11:15:52.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.480+0000 7ff573e00640 1 -- 172.21.15.7:0/1960687168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5741eaf50 con 0x7ff57406dcd0 2024-04-25T11:15:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.481+0000 7ff573e00640 1 -- 172.21.15.7:0/1960687168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff574162d90 con 0x7ff57406dcd0 2024-04-25T11:15:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.483+0000 7ff56aa00640 1 -- 172.21.15.7:0/1960687168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff564012b70 con 0x7ff57406dcd0 2024-04-25T11:15:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.484+0000 7ff56aa00640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5500744e0 0x7ff5500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.484+0000 7ff56aa00640 1 -- 172.21.15.7:0/1960687168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5335+0+0 (secure 0 0 0) 0x7ff564095590 con 0x7ff57406dcd0 2024-04-25T11:15:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.484+0000 7ff572000640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5500744e0 0x7ff5500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.485+0000 7ff572000640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5500744e0 0x7ff5500769f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff55c033000 tx=0x7ff55c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:52.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.489+0000 7ff56aa00640 1 -- 172.21.15.7:0/1960687168 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff574162d90 con 0x7ff57406dcd0 2024-04-25T11:15:52.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.694+0000 7ff573e00640 1 -- 172.21.15.7:0/1960687168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff574050740 con 0x7ff57406dcd0 2024-04-25T11:15:52.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.695+0000 7ff56aa00640 1 -- 172.21.15.7:0/1960687168 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+13196 (secure 0 0 0) 0x7ff564066e20 con 0x7ff57406dcd0 2024-04-25T11:15:52.697 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:52.697 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":83,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:15:50.123095+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6837/2955530443":"2024-04-26T11:15:47.046083+0000","172.21.15.7:6835/2955530443":"2024-04-26T11:15:47.046083+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:15:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.698+0000 7ff573e00640 1 -- 172.21.15.7:0/1960687168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5500744e0 msgr2=0x7ff5500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.698+0000 7ff573e00640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5500744e0 0x7ff5500769f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff55c033000 tx=0x7ff55c02e000 comp rx=0 tx=0).stop 2024-04-25T11:15:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.698+0000 7ff573e00640 1 -- 172.21.15.7:0/1960687168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff57406dcd0 msgr2=0x7ff57415c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.698+0000 7ff573e00640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff57406dcd0 0x7ff57415c4e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff56400d5f0 tx=0x7ff56400dab0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.698+0000 7ff573e00640 1 -- 172.21.15.7:0/1960687168 shutdown_connections 2024-04-25T11:15:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.698+0000 7ff573e00640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5500744e0 0x7ff5500769f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.698+0000 7ff573e00640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff574163ac0 0x7ff5741587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.698+0000 7ff573e00640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5740d0570 0x7ff574154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.698+0000 7ff573e00640 1 --2- 172.21.15.7:0/1960687168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff57406dcd0 0x7ff57415c4e0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.698+0000 7ff573e00640 1 -- 172.21.15.7:0/1960687168 >> 172.21.15.7:0/1960687168 conn(0x7ff5740d29e0 msgr2=0x7ff574162680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.699+0000 7ff573e00640 1 -- 172.21.15.7:0/1960687168 shutdown_connections 2024-04-25T11:15:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:52.699+0000 7ff573e00640 1 -- 172.21.15.7:0/1960687168 wait complete. 2024-04-25T11:15:52.726 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:15:52.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:15:53.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.013+0000 7fcf25400640 1 Processor -- start 2024-04-25T11:15:53.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.013+0000 7fcf25400640 1 -- start start 2024-04-25T11:15:53.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.015+0000 7fcf25400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2006de10 0x7fcf200b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.015+0000 7fcf25400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcf200b7a10 0x7fcf200d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.015+0000 7fcf25400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf200b7e20 con 0x7fcf200d1cd0 2024-04-25T11:15:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.015+0000 7fcf25400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcf201603d0 con 0x7fcf200b7a10 2024-04-25T11:15:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.015+0000 7fcf25400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2006d090 con 0x7fcf2006de10 2024-04-25T11:15:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.015+0000 7fcf24a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcf200d1cd0 0x7fcf200b2150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53610/0 (socket says 172.21.15.7:53610) 2024-04-25T11:15:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.015+0000 7fcf24a00640 1 -- 172.21.15.7:0/1121191223 learned_addr learned my addr 172.21.15.7:0/1121191223 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1fe00640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2006de10 0x7fcf200b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1f400640 1 --2- 172.21.15.7:0/1121191223 >> v2:172.21.15.87:3300/0 conn(0x7fcf200b7a10 0x7fcf200d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1137860040 0 0) 0x7fcf18001110 con 0x7fcf200d1cd0 2024-04-25T11:15:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf04003520 con 0x7fcf200d1cd0 2024-04-25T11:15:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2356973963 0 0) 0x7fcf04003520 con 0x7fcf200d1cd0 2024-04-25T11:15:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2006de10 msgr2=0x7fcf200b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1ea00640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2006de10 0x7fcf200b74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 >> v2:172.21.15.87:3300/0 conn(0x7fcf200b7a10 msgr2=0x7fcf200d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1ea00640 1 --2- 172.21.15.7:0/1121191223 >> v2:172.21.15.87:3300/0 conn(0x7fcf200b7a10 0x7fcf200d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf200d3490 con 0x7fcf200d1cd0 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fcf18001840 con 0x7fcf200d1cd0 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 >> v1:172.21.15.7:6789/0 conn(0x7fcf200d1cd0 legacy=0x7fcf200b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.016+0000 7fcf1ea00640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf040039b0 0x7fcf04003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.017+0000 7fcf1ea00640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2006de10 0x7fcf04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.017+0000 7fcf1ea00640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf04004d40 0x7fcf04005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.017+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcf201603d0 con 0x7fcf04004d40 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.017+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2006d090 con 0x7fcf040039b0 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.017+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0400b9f0 con 0x7fcf2006de10 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.017+0000 7fcf24a00640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf04004d40 0x7fcf04005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.017+0000 7fcf1fe00640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf040039b0 0x7fcf04003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.017+0000 7fcf1f400640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2006de10 0x7fcf04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.017+0000 7fcf24a00640 1 -- 172.21.15.7:0/1121191223 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2006de10 msgr2=0x7fcf04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:53.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.017+0000 7fcf24a00640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2006de10 0x7fcf04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.018+0000 7fcf24a00640 1 -- 172.21.15.7:0/1121191223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf040039b0 msgr2=0x7fcf04003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.018+0000 7fcf24a00640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf040039b0 0x7fcf04003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.018+0000 7fcf24a00640 1 -- 172.21.15.7:0/1121191223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf04003520 con 0x7fcf04004d40 2024-04-25T11:15:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.019+0000 7fcf24a00640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf04004d40 0x7fcf04005150 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800c2f0 tx=0x7fcf04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371f8b92138aca0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.019+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18035070 con 0x7fcf04004d40 2024-04-25T11:15:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.019+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf18007e50 con 0x7fcf04004d40 2024-04-25T11:15:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.019+0000 7fcf1ea00640 1 -- 172.21.15.7:0/1121191223 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18005ad0 con 0x7fcf04004d40 2024-04-25T11:15:53.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.020+0000 7fcf25400640 1 -- 172.21.15.7:0/1121191223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf04004d40 msgr2=0x7fcf04005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.020+0000 7fcf25400640 1 --2- 172.21.15.7:0/1121191223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf04004d40 0x7fcf04005150 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800c2f0 tx=0x7fcf04005830 comp rx=0 tx=0).stop 2024-04-25T11:15:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.020+0000 7fcf24a00640 1 -- 172.21.15.7:0/1121191223 reap_dead start 2024-04-25T11:15:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.020+0000 7fcf25400640 1 -- 172.21.15.7:0/1121191223 shutdown_connections 2024-04-25T11:15:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.020+0000 7fcf25400640 1 -- 172.21.15.7:0/1121191223 >> 172.21.15.7:0/1121191223 conn(0x7fcf2009b540 msgr2=0x7fcf2009b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:53.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.020+0000 7fcf25400640 1 -- 172.21.15.7:0/1121191223 shutdown_connections 2024-04-25T11:15:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.020+0000 7fcf25400640 1 -- 172.21.15.7:0/1121191223 wait complete. 2024-04-25T11:15:53.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.022+0000 7fcf25400640 1 Processor -- start 2024-04-25T11:15:53.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.023+0000 7fcf25400640 1 -- start start 2024-04-25T11:15:53.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.024+0000 7fcf25400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf04004d40 0x7fcf20151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.024+0000 7fcf25400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf200ce440 0x7fcf2015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.024+0000 7fcf25400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2015c220 0x7fcf200cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.024+0000 7fcf25400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2006d8f0 con 0x7fcf200ce440 2024-04-25T11:15:53.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.024+0000 7fcf25400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2006d610 con 0x7fcf04004d40 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.024+0000 7fcf25400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf200d20e0 con 0x7fcf2015c220 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.024+0000 7fcf1f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf200ce440 0x7fcf2015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.025+0000 7fcf1f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf200ce440 0x7fcf2015bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41582/0 (socket says 172.21.15.7:41582) 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.025+0000 7fcf1f400640 1 -- 172.21.15.7:0/2164494079 learned_addr learned my addr 172.21.15.7:0/2164494079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.025+0000 7fcf1fe00640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf04004d40 0x7fcf20151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.025+0000 7fcf24a00640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2015c220 0x7fcf200cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.025+0000 7fcf1f400640 1 -- 172.21.15.7:0/2164494079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2015c220 msgr2=0x7fcf200cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.025+0000 7fcf1f400640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2015c220 0x7fcf200cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.025+0000 7fcf1f400640 1 -- 172.21.15.7:0/2164494079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf04004d40 msgr2=0x7fcf20151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.025+0000 7fcf1f400640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf04004d40 0x7fcf20151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.025+0000 7fcf1f400640 1 -- 172.21.15.7:0/2164494079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2015c630 con 0x7fcf200ce440 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.025+0000 7fcf1f400640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf200ce440 0x7fcf2015bce0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcf08011700 tx=0x7fcf08011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.026+0000 7fcf17e00640 1 -- 172.21.15.7:0/2164494079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800f9c0 con 0x7fcf200ce440 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.026+0000 7fcf17e00640 1 -- 172.21.15.7:0/2164494079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0800fb60 con 0x7fcf200ce440 2024-04-25T11:15:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.026+0000 7fcf17e00640 1 -- 172.21.15.7:0/2164494079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08015460 con 0x7fcf200ce440 2024-04-25T11:15:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.026+0000 7fcf25400640 1 -- 172.21.15.7:0/2164494079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf200c9840 con 0x7fcf200ce440 2024-04-25T11:15:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.026+0000 7fcf25400640 1 -- 172.21.15.7:0/2164494079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf200c9e00 con 0x7fcf200ce440 2024-04-25T11:15:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.027+0000 7fcf25400640 1 -- 172.21.15.7:0/2164494079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf20050740 con 0x7fcf200ce440 2024-04-25T11:15:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.028+0000 7fcf17e00640 1 -- 172.21.15.7:0/2164494079 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf08015600 con 0x7fcf200ce440 2024-04-25T11:15:53.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.029+0000 7fcf17e00640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcefc0744b0 0x7fcefc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.029+0000 7fcf17e00640 1 -- 172.21.15.7:0/2164494079 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5335+0+0 (secure 0 0 0) 0x7fcf08095bb0 con 0x7fcf200ce440 2024-04-25T11:15:53.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.030+0000 7fcf1fe00640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcefc0744b0 0x7fcefc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.031+0000 7fcf1fe00640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcefc0744b0 0x7fcefc0769c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009fd0 tx=0x7fcf1000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:53.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.034+0000 7fcf17e00640 1 -- 172.21.15.7:0/2164494079 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf2015c630 con 0x7fcf200ce440 2024-04-25T11:15:53.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.240+0000 7fcf25400640 1 -- 172.21.15.7:0/2164494079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf2015fb90 con 0x7fcf200ce440 2024-04-25T11:15:53.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.241+0000 7fcf17e00640 1 -- 172.21.15.7:0/2164494079 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fcf08019030 con 0x7fcf200ce440 2024-04-25T11:15:53.243 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:15:53.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 -- 172.21.15.7:0/2164494079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcefc0744b0 msgr2=0x7fcefc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:53.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcefc0744b0 0x7fcefc0769c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009fd0 tx=0x7fcf1000b040 comp rx=0 tx=0).stop 2024-04-25T11:15:53.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 -- 172.21.15.7:0/2164494079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf200ce440 msgr2=0x7fcf2015bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf200ce440 0x7fcf2015bce0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcf08011700 tx=0x7fcf08011bc0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 -- 172.21.15.7:0/2164494079 shutdown_connections 2024-04-25T11:15:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcefc0744b0 0x7fcefc0769c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf2015c220 0x7fcf200cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf200ce440 0x7fcf2015bce0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 --2- 172.21.15.7:0/2164494079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf04004d40 0x7fcf20151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 -- 172.21.15.7:0/2164494079 >> 172.21.15.7:0/2164494079 conn(0x7fcf2009b540 msgr2=0x7fcf2015cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 -- 172.21.15.7:0/2164494079 shutdown_connections 2024-04-25T11:15:53.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.244+0000 7fcf25400640 1 -- 172.21.15.7:0/2164494079 wait complete. 2024-04-25T11:15:53.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2024-04-25T11:15:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.555+0000 7fad69a00640 1 Processor -- start 2024-04-25T11:15:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.555+0000 7fad69a00640 1 -- start start 2024-04-25T11:15:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.557+0000 7fad69a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fad640d0d30 0x7fad640b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.558+0000 7fad69a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad640ce7d0 0x7fad640cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.558+0000 7fad69a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad6409cc20 con 0x7fad640d2c50 2024-04-25T11:15:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.558+0000 7fad69a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fad6409cf00 con 0x7fad640ce7d0 2024-04-25T11:15:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.558+0000 7fad69a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fad6406e760 con 0x7fad640d0d30 2024-04-25T11:15:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.558+0000 7fad63400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fad640d2c50 0x7fad640d1e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53618/0 (socket says 172.21.15.7:53618) 2024-04-25T11:15:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.558+0000 7fad63400640 1 -- 172.21.15.7:0/845584902 learned_addr learned my addr 172.21.15.7:0/845584902 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.558+0000 7fad69000640 1 --2- 172.21.15.7:0/845584902 >> v2:172.21.15.7:3300/0 conn(0x7fad640ce7d0 0x7fad640cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.558+0000 7fad63e00640 1 --2- 172.21.15.7:0/845584902 >> v2:172.21.15.87:3300/0 conn(0x7fad640d0d30 0x7fad640b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.559+0000 7fad62a00640 1 -- 172.21.15.7:0/845584902 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1116026500 0 0) 0x7fad6409cc20 con 0x7fad640d2c50 2024-04-25T11:15:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.559+0000 7fad62a00640 1 -- 172.21.15.7:0/845584902 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad48003520 con 0x7fad640d2c50 2024-04-25T11:15:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.559+0000 7fad69000640 1 -- 172.21.15.7:0/845584902 >> v2:172.21.15.87:3300/0 conn(0x7fad640d0d30 msgr2=0x7fad640b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.559+0000 7fad69000640 1 --2- 172.21.15.7:0/845584902 >> v2:172.21.15.87:3300/0 conn(0x7fad640d0d30 0x7fad640b89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.559+0000 7fad69000640 1 -- 172.21.15.7:0/845584902 >> v1:172.21.15.7:6789/0 conn(0x7fad640d2c50 legacy=0x7fad640d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.559+0000 7fad69000640 1 -- 172.21.15.7:0/845584902 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6406ea40 con 0x7fad640ce7d0 2024-04-25T11:15:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.560+0000 7fad69000640 1 --2- 172.21.15.7:0/845584902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad640ce7d0 0x7fad640cebe0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fad64074ab0 tx=0x7fad5c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3be150f39bcd123 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:53.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.560+0000 7fad62a00640 1 -- 172.21.15.7:0/845584902 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c037070 con 0x7fad640ce7d0 2024-04-25T11:15:53.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.560+0000 7fad62a00640 1 -- 172.21.15.7:0/845584902 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad5c032440 con 0x7fad640ce7d0 2024-04-25T11:15:53.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.560+0000 7fad62a00640 1 -- 172.21.15.7:0/845584902 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0327f0 con 0x7fad640ce7d0 2024-04-25T11:15:53.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.561+0000 7fad69a00640 1 -- 172.21.15.7:0/845584902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad640ce7d0 msgr2=0x7fad640cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:53.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.561+0000 7fad69a00640 1 --2- 172.21.15.7:0/845584902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad640ce7d0 0x7fad640cebe0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fad64074ab0 tx=0x7fad5c02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.562+0000 7fad69a00640 1 -- 172.21.15.7:0/845584902 shutdown_connections 2024-04-25T11:15:53.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.562+0000 7fad69a00640 1 --2- 172.21.15.7:0/845584902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad640ce7d0 0x7fad640cebe0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.562+0000 7fad69a00640 1 --2- 172.21.15.7:0/845584902 >> v2:172.21.15.87:3300/0 conn(0x7fad640d0d30 0x7fad640b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.562+0000 7fad69a00640 1 -- 172.21.15.7:0/845584902 >> 172.21.15.7:0/845584902 conn(0x7fad640d0210 msgr2=0x7fad640d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:53.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.562+0000 7fad69a00640 1 -- 172.21.15.7:0/845584902 shutdown_connections 2024-04-25T11:15:53.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.562+0000 7fad69a00640 1 -- 172.21.15.7:0/845584902 wait complete. 2024-04-25T11:15:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.564+0000 7fad69a00640 1 Processor -- start 2024-04-25T11:15:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.565+0000 7fad69a00640 1 -- start start 2024-04-25T11:15:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.566+0000 7fad69a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad640ce7d0 0x7fad641df150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.566+0000 7fad69a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad640d0d30 0x7fad641de960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.566+0000 7fad69a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad640d2c50 0x7fad6409ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.566+0000 7fad69a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad640cf120 con 0x7fad640ce7d0 2024-04-25T11:15:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.566+0000 7fad69a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fad640d2790 con 0x7fad640d2c50 2024-04-25T11:15:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.566+0000 7fad69a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad640cf2a0 con 0x7fad640d0d30 2024-04-25T11:15:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.567+0000 7fad63e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad640ce7d0 0x7fad641df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.567+0000 7fad63e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad640ce7d0 0x7fad641df150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41592/0 (socket says 172.21.15.7:41592) 2024-04-25T11:15:53.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.567+0000 7fad63e00640 1 -- 172.21.15.7:0/1638341850 learned_addr learned my addr 172.21.15.7:0/1638341850 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.567+0000 7fad63e00640 1 -- 172.21.15.7:0/1638341850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad640d0d30 msgr2=0x7fad641de960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.567+0000 7fad63400640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad640d0d30 0x7fad641de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.567+0000 7fad69000640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad640d2c50 0x7fad6409ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.567+0000 7fad63e00640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad640d0d30 0x7fad641de960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.567+0000 7fad63e00640 1 -- 172.21.15.7:0/1638341850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad640d2c50 msgr2=0x7fad6409ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.567+0000 7fad63e00640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad640d2c50 0x7fad6409ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.567+0000 7fad63e00640 1 -- 172.21.15.7:0/1638341850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad641dfbd0 con 0x7fad640ce7d0 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.567+0000 7fad69000640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad640d2c50 0x7fad6409ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.568+0000 7fad63e00640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad640ce7d0 0x7fad641df150 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fad54009cd0 tx=0x7fad5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.568+0000 7fad5be00640 1 -- 172.21.15.7:0/1638341850 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54018020 con 0x7fad640ce7d0 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.568+0000 7fad5be00640 1 -- 172.21.15.7:0/1638341850 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad54014480 con 0x7fad640ce7d0 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.568+0000 7fad69a00640 1 -- 172.21.15.7:0/1638341850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad641dfeb0 con 0x7fad640ce7d0 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.568+0000 7fad5be00640 1 -- 172.21.15.7:0/1638341850 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54014850 con 0x7fad640ce7d0 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.568+0000 7fad69a00640 1 -- 172.21.15.7:0/1638341850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad641e0470 con 0x7fad640ce7d0 2024-04-25T11:15:53.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.570+0000 7fad69a00640 1 -- 172.21.15.7:0/1638341850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad64050740 con 0x7fad640ce7d0 2024-04-25T11:15:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.571+0000 7fad5be00640 1 -- 172.21.15.7:0/1638341850 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad54005d10 con 0x7fad640ce7d0 2024-04-25T11:15:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.572+0000 7fad5be00640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad400744b0 0x7fad400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.572+0000 7fad5be00640 1 -- 172.21.15.7:0/1638341850 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5335+0+0 (secure 0 0 0) 0x7fad54095db0 con 0x7fad640ce7d0 2024-04-25T11:15:53.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.572+0000 7fad63400640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad400744b0 0x7fad400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:53.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.573+0000 7fad63400640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad400744b0 0x7fad400769c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fad64071690 tx=0x7fad4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.577+0000 7fad5be00640 1 -- 172.21.15.7:0/1638341850 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad54022030 con 0x7fad640ce7d0 2024-04-25T11:15:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:53.846+0000 7fad69a00640 1 -- 172.21.15.7:0/1638341850 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) -- 0x7fad641df690 con 0x7fad640ce7d0 2024-04-25T11:15:54.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.156+0000 7fad5be00640 1 -- 172.21.15.7:0/1638341850 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v305) ==== 163+0+0 (secure 0 0 0) 0x7fad54010040 con 0x7fad640ce7d0 2024-04-25T11:15:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.159+0000 7fad69a00640 1 -- 172.21.15.7:0/1638341850 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad400744b0 msgr2=0x7fad400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.159+0000 7fad69a00640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad400744b0 0x7fad400769c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fad64071690 tx=0x7fad4c009040 comp rx=0 tx=0).stop 2024-04-25T11:15:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.159+0000 7fad69a00640 1 -- 172.21.15.7:0/1638341850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad640ce7d0 msgr2=0x7fad641df150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.159+0000 7fad69a00640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad640ce7d0 0x7fad641df150 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fad54009cd0 tx=0x7fad5400c490 comp rx=0 tx=0).stop 2024-04-25T11:15:54.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.159+0000 7fad69a00640 1 -- 172.21.15.7:0/1638341850 shutdown_connections 2024-04-25T11:15:54.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.159+0000 7fad69a00640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad400744b0 0x7fad400769c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.159+0000 7fad69a00640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad640d2c50 0x7fad6409ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.159+0000 7fad69a00640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad640d0d30 0x7fad641de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.159+0000 7fad69a00640 1 --2- 172.21.15.7:0/1638341850 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad640ce7d0 0x7fad641df150 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.159+0000 7fad69a00640 1 -- 172.21.15.7:0/1638341850 >> 172.21.15.7:0/1638341850 conn(0x7fad640d0210 msgr2=0x7fad64072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:54.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.159+0000 7fad69a00640 1 -- 172.21.15.7:0/1638341850 shutdown_connections 2024-04-25T11:15:54.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.160+0000 7fad69a00640 1 -- 172.21.15.7:0/1638341850 wait complete. 2024-04-25T11:15:54.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:15:54.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.471+0000 7f1ed2000640 1 Processor -- start 2024-04-25T11:15:54.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.471+0000 7f1ed2000640 1 -- start start 2024-04-25T11:15:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.473+0000 7f1ed2000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ecc1583c0 0x7f1ecc0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.474+0000 7f1ed2000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ecc0ce570 con 0x7f1ecc06eea0 2024-04-25T11:15:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.474+0000 7f1ed2000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ecc0ce850 con 0x7f1ecc0cfb50 2024-04-25T11:15:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.474+0000 7f1ed2000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1ecc15f190 con 0x7f1ecc1583c0 2024-04-25T11:15:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.474+0000 7f1ed1600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ecc06eea0 0x7f1ecc0b0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53630/0 (socket says 172.21.15.7:53630) 2024-04-25T11:15:54.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.474+0000 7f1ed1600640 1 -- 172.21.15.7:0/1098063628 learned_addr learned my addr 172.21.15.7:0/1098063628 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.474+0000 7f1ed0c00640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ecc1583c0 0x7f1ecc0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3125152507 0 0) 0x7f1ecc0ce570 con 0x7f1ecc06eea0 2024-04-25T11:15:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1eb0003520 con 0x7f1ecc06eea0 2024-04-25T11:15:54.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1882530833 0 0) 0x7f1eb0003520 con 0x7f1ecc06eea0 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ecc1583c0 msgr2=0x7f1ecc0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ecc1583c0 0x7f1ecc0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 >> v1:172.21.15.87:6789/0 conn(0x7f1ecc0cfb50 legacy=0x7f1ecc0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc0ceb30 con 0x7f1ecc06eea0 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1ec40019f0 con 0x7f1ecc06eea0 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 >> v1:172.21.15.7:6789/0 conn(0x7f1ecc06eea0 legacy=0x7f1ecc0b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb00039e0 0x7f1eb0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ecc1583c0 0x7f1eb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1eb0004d40 0x7f1eb0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc15f190 con 0x7f1eb00039e0 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb000baf0 con 0x7f1eb0004d40 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb000bcf0 con 0x7f1ecc1583c0 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.475+0000 7f1ed0c00640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb00039e0 0x7f1eb0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.476+0000 7f1ecbe00640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ecc1583c0 0x7f1eb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.476+0000 7f1ed0c00640 1 -- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ecc1583c0 msgr2=0x7f1eb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.476+0000 7f1ed1600640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1eb0004d40 0x7f1eb0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.476+0000 7f1ed0c00640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ecc1583c0 0x7f1eb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.476+0000 7f1ed0c00640 1 -- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1eb0004d40 msgr2=0x7f1eb0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.476+0000 7f1ed0c00640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1eb0004d40 0x7f1eb0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.476+0000 7f1ed0c00640 1 -- 172.21.15.7:0/1098063628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb0003520 con 0x7f1eb00039e0 2024-04-25T11:15:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.476+0000 7f1ecbe00640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ecc1583c0 0x7f1eb0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.476+0000 7f1ed1600640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1eb0004d40 0x7f1eb0005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:54.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.477+0000 7f1ed0c00640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb00039e0 0x7f1eb0003e50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0b43c0 tx=0x7f1eb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deece55029d5efed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:54.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.477+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc036070 con 0x7f1eb00039e0 2024-04-25T11:15:54.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.477+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ebc031400 con 0x7f1eb00039e0 2024-04-25T11:15:54.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.477+0000 7f1ecb400640 1 -- 172.21.15.7:0/1098063628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc009690 con 0x7f1eb00039e0 2024-04-25T11:15:54.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.478+0000 7f1ed2000640 1 -- 172.21.15.7:0/1098063628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb00039e0 msgr2=0x7f1eb0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:54.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.478+0000 7f1ed2000640 1 --2- 172.21.15.7:0/1098063628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb00039e0 0x7f1eb0003e50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc0b43c0 tx=0x7f1eb0005830 comp rx=0 tx=0).stop 2024-04-25T11:15:54.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.478+0000 7f1ed1600640 1 -- 172.21.15.7:0/1098063628 reap_dead start 2024-04-25T11:15:54.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.478+0000 7f1ed2000640 1 -- 172.21.15.7:0/1098063628 shutdown_connections 2024-04-25T11:15:54.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.478+0000 7f1ed2000640 1 -- 172.21.15.7:0/1098063628 >> 172.21.15.7:0/1098063628 conn(0x7f1ecc0740f0 msgr2=0x7f1ecc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:54.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.478+0000 7f1ed2000640 1 -- 172.21.15.7:0/1098063628 shutdown_connections 2024-04-25T11:15:54.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.478+0000 7f1ed2000640 1 -- 172.21.15.7:0/1098063628 wait complete. 2024-04-25T11:15:54.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.481+0000 7f1ed2000640 1 Processor -- start 2024-04-25T11:15:54.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.481+0000 7f1ed2000640 1 -- start start 2024-04-25T11:15:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.482+0000 7f1ed2000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1eb00039e0 0x7f1ecc13ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.482+0000 7f1ed2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc140760 0x7f1ecc0b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.482+0000 7f1ed2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ecc1462b0 0x7f1ecc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:54.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.482+0000 7f1ed2000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc0d0b90 con 0x7f1ecc140760 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.482+0000 7f1ed2000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc0cf0f0 con 0x7f1ecc1462b0 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.482+0000 7f1ed2000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1ecc0b0e70 con 0x7f1eb00039e0 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.483+0000 7f1ecbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc140760 0x7f1ecc0b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.483+0000 7f1ecbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc140760 0x7f1ecc0b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41616/0 (socket says 172.21.15.7:41616) 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.483+0000 7f1ecbe00640 1 -- 172.21.15.7:0/3586258590 learned_addr learned my addr 172.21.15.7:0/3586258590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.483+0000 7f1ed0c00640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1eb00039e0 0x7f1ecc13ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.483+0000 7f1ed1600640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ecc1462b0 0x7f1ecc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.483+0000 7f1ecbe00640 1 -- 172.21.15.7:0/3586258590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1eb00039e0 msgr2=0x7f1ecc13ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.483+0000 7f1ecbe00640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1eb00039e0 0x7f1ecc13ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.483+0000 7f1ecbe00640 1 -- 172.21.15.7:0/3586258590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ecc1462b0 msgr2=0x7f1ecc0b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.483+0000 7f1ecbe00640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ecc1462b0 0x7f1ecc0b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.483+0000 7f1ecbe00640 1 -- 172.21.15.7:0/3586258590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc140480 con 0x7f1ecc140760 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.483+0000 7f1ed1600640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ecc1462b0 0x7f1ecc0b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.484+0000 7f1ecbe00640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc140760 0x7f1ecc0b5c20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d7a0 tx=0x7f1eb400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:54.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.484+0000 7f1ec8c00640 1 -- 172.21.15.7:0/3586258590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4005070 con 0x7f1ecc140760 2024-04-25T11:15:54.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.484+0000 7f1ec8c00640 1 -- 172.21.15.7:0/3586258590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb4005210 con 0x7f1ecc140760 2024-04-25T11:15:54.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.484+0000 7f1ed2000640 1 -- 172.21.15.7:0/3586258590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc1fa190 con 0x7f1ecc140760 2024-04-25T11:15:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.484+0000 7f1ec8c00640 1 -- 172.21.15.7:0/3586258590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4007030 con 0x7f1ecc140760 2024-04-25T11:15:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.484+0000 7f1ed2000640 1 -- 172.21.15.7:0/3586258590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc1fa750 con 0x7f1ecc140760 2024-04-25T11:15:54.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.485+0000 7f1ed2000640 1 -- 172.21.15.7:0/3586258590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ecc050740 con 0x7f1ecc140760 2024-04-25T11:15:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.486+0000 7f1ec8c00640 1 -- 172.21.15.7:0/3586258590 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1eb401a020 con 0x7f1ecc140760 2024-04-25T11:15:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.487+0000 7f1ec8c00640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ea80744e0 0x7f1ea80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.487+0000 7f1ec8c00640 1 -- 172.21.15.7:0/3586258590 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5335+0+0 (secure 0 0 0) 0x7f1eb4094920 con 0x7f1ecc140760 2024-04-25T11:15:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.488+0000 7f1ed0c00640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ea80744e0 0x7f1ea80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.488+0000 7f1ed0c00640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ea80744e0 0x7f1ea80769f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc033000 tx=0x7f1ebc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:54.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.492+0000 7f1ec8c00640 1 -- 172.21.15.7:0/3586258590 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ecc140480 con 0x7f1ecc140760 2024-04-25T11:15:54.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.702+0000 7f1ed2000640 1 -- 172.21.15.7:0/3586258590 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1ecc070dd0 con 0x7f1ecc140760 2024-04-25T11:15:54.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.709+0000 7f1ec8c00640 1 -- 172.21.15.7:0/3586258590 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f1eb4067020 con 0x7f1ecc140760 2024-04-25T11:15:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 -- 172.21.15.7:0/3586258590 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ea80744e0 msgr2=0x7f1ea80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ea80744e0 0x7f1ea80769f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc033000 tx=0x7f1ebc008030 comp rx=0 tx=0).stop 2024-04-25T11:15:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 -- 172.21.15.7:0/3586258590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc140760 msgr2=0x7f1ecc0b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc140760 0x7f1ecc0b5c20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d7a0 tx=0x7f1eb400dc60 comp rx=0 tx=0).stop 2024-04-25T11:15:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 -- 172.21.15.7:0/3586258590 shutdown_connections 2024-04-25T11:15:54.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ea80744e0 0x7f1ea80769f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ecc1462b0 0x7f1ecc0b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ecc140760 0x7f1ecc0b5c20 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 --2- 172.21.15.7:0/3586258590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1eb00039e0 0x7f1ecc13ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:54.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 -- 172.21.15.7:0/3586258590 >> 172.21.15.7:0/3586258590 conn(0x7f1ecc0740f0 msgr2=0x7f1ecc0b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:54.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 -- 172.21.15.7:0/3586258590 shutdown_connections 2024-04-25T11:15:54.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:54.712+0000 7f1ed2000640 1 -- 172.21.15.7:0/3586258590 wait complete. 2024-04-25T11:15:54.738 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:15:54.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:15:54.846 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:15:54.961 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:15:54.961 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:15:54.961 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:55.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:15:55.020 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:15:55.020 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:55.079 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:15:55.079 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:15:55.079 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:55.139 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:15:55.139 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:15:55.139 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:15:55.195 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:15:55.195 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:15:55.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:15:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.482+0000 7feea9000640 1 Processor -- start 2024-04-25T11:15:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.482+0000 7feea9000640 1 -- start start 2024-04-25T11:15:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.484+0000 7feea9000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feea40b6790 0x7feea4164640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:55.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.484+0000 7feea9000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea40aea00 con 0x7feea415de70 2024-04-25T11:15:55.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.484+0000 7feea9000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea40b13e0 con 0x7feea40b0c20 2024-04-25T11:15:55.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.484+0000 7feea9000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7feea40cdb70 con 0x7feea40b6790 2024-04-25T11:15:55.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.485+0000 7feea1e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7feea415de70 0x7feea40b0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53632/0 (socket says 172.21.15.7:53632) 2024-04-25T11:15:55.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.485+0000 7feea1e00640 1 -- 172.21.15.7:0/2824938604 learned_addr learned my addr 172.21.15.7:0/2824938604 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.485+0000 7feea1400640 1 --2- 172.21.15.7:0/2824938604 >> v2:172.21.15.87:3300/0 conn(0x7feea40b6790 0x7feea4164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.485+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3394834888 0 0) 0x7feea40aea00 con 0x7feea415de70 2024-04-25T11:15:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.485+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee88003520 con 0x7feea415de70 2024-04-25T11:15:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1211663371 0 0) 0x7fee88003520 con 0x7feea415de70 2024-04-25T11:15:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 >> v2:172.21.15.87:3300/0 conn(0x7feea40b6790 msgr2=0x7feea4164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 --2- 172.21.15.7:0/2824938604 >> v2:172.21.15.87:3300/0 conn(0x7feea40b6790 0x7feea4164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 >> v1:172.21.15.87:6789/0 conn(0x7feea40b0c20 legacy=0x7feea41606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea40cddc0 con 0x7feea415de70 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fee94001670 con 0x7feea415de70 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 >> v1:172.21.15.7:6789/0 conn(0x7feea415de70 legacy=0x7feea40b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee880039b0 0x7fee88003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee88004800 0x7fee88004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee88005250 0x7fee8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feea40cdb70 con 0x7fee880039b0 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fee8800c1e0 con 0x7fee88005250 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fee8800c3e0 con 0x7fee88004800 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.486+0000 7feea1e00640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee880039b0 0x7fee88003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.487+0000 7feea1400640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee88004800 0x7fee88004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.487+0000 7feea2800640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee88005250 0x7fee8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.487+0000 7feea1e00640 1 -- 172.21.15.7:0/2824938604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee88004800 msgr2=0x7fee88004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.487+0000 7feea1e00640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee88004800 0x7fee88004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.487+0000 7feea1e00640 1 -- 172.21.15.7:0/2824938604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee88005250 msgr2=0x7fee8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.487+0000 7feea1e00640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee88005250 0x7fee8800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.487+0000 7feea1e00640 1 -- 172.21.15.7:0/2824938604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee88003520 con 0x7fee880039b0 2024-04-25T11:15:55.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.487+0000 7feea2800640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee88005250 0x7fee8800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:55.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.487+0000 7feea2800640 1 -- 172.21.15.7:0/2824938604 reap_dead start 2024-04-25T11:15:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.488+0000 7feea1e00640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee880039b0 0x7fee88003e20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fee9400c210 tx=0x7fee88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6dfc00b3bfcc91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.488+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94007d10 con 0x7fee880039b0 2024-04-25T11:15:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.488+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee94005ad0 con 0x7fee880039b0 2024-04-25T11:15:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.488+0000 7feea0a00640 1 -- 172.21.15.7:0/2824938604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94007070 con 0x7fee880039b0 2024-04-25T11:15:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.489+0000 7feea9000640 1 -- 172.21.15.7:0/2824938604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee880039b0 msgr2=0x7fee88003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.489+0000 7feea9000640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee880039b0 0x7fee88003e20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fee9400c210 tx=0x7fee88005830 comp rx=0 tx=0).stop 2024-04-25T11:15:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.489+0000 7feea9000640 1 -- 172.21.15.7:0/2824938604 shutdown_connections 2024-04-25T11:15:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.489+0000 7feea9000640 1 --2- 172.21.15.7:0/2824938604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee880039b0 0x7fee88003e20 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.489+0000 7feea9000640 1 -- 172.21.15.7:0/2824938604 >> 172.21.15.7:0/2824938604 conn(0x7feea4074200 msgr2=0x7feea406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.489+0000 7feea9000640 1 -- 172.21.15.7:0/2824938604 shutdown_connections 2024-04-25T11:15:55.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.489+0000 7feea9000640 1 -- 172.21.15.7:0/2824938604 wait complete. 2024-04-25T11:15:55.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.492+0000 7feea9000640 1 Processor -- start 2024-04-25T11:15:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.492+0000 7feea9000640 1 -- start start 2024-04-25T11:15:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.493+0000 7feea9000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee880039b0 0x7feea4159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.493+0000 7feea9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feea4155380 0x7feea41451e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.493+0000 7feea9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feea40cf2d0 0x7feea40cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.493+0000 7feea9000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feea406f300 con 0x7feea40cf2d0 2024-04-25T11:15:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.493+0000 7feea9000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feea40aed30 con 0x7feea4155380 2024-04-25T11:15:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.493+0000 7feea9000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feea406f480 con 0x7fee880039b0 2024-04-25T11:15:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.494+0000 7feea2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feea40cf2d0 0x7feea40cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.494+0000 7feea2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feea40cf2d0 0x7feea40cf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41630/0 (socket says 172.21.15.7:41630) 2024-04-25T11:15:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.494+0000 7feea2800640 1 -- 172.21.15.7:0/2857266115 learned_addr learned my addr 172.21.15.7:0/2857266115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.494+0000 7feea1400640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feea4155380 0x7feea41451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.494+0000 7feea1e00640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee880039b0 0x7feea4159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.494+0000 7feea2800640 1 -- 172.21.15.7:0/2857266115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee880039b0 msgr2=0x7feea4159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.494+0000 7feea2800640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee880039b0 0x7feea4159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.494+0000 7feea2800640 1 -- 172.21.15.7:0/2857266115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feea4155380 msgr2=0x7feea41451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.494+0000 7feea2800640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feea4155380 0x7feea41451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.494+0000 7feea2800640 1 -- 172.21.15.7:0/2857266115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea4145720 con 0x7feea40cf2d0 2024-04-25T11:15:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.494+0000 7feea1e00640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee880039b0 0x7feea4159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.495+0000 7feea2800640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feea40cf2d0 0x7feea40cf6e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7feea40b4d30 tx=0x7fee9c00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.495+0000 7fee9a000640 1 -- 172.21.15.7:0/2857266115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c018070 con 0x7feea40cf2d0 2024-04-25T11:15:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.495+0000 7fee9a000640 1 -- 172.21.15.7:0/2857266115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9c0035f0 con 0x7feea40cf2d0 2024-04-25T11:15:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.495+0000 7fee9a000640 1 -- 172.21.15.7:0/2857266115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0039e0 con 0x7feea40cf2d0 2024-04-25T11:15:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.495+0000 7feea9000640 1 -- 172.21.15.7:0/2857266115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea4155790 con 0x7feea40cf2d0 2024-04-25T11:15:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.495+0000 7feea9000640 1 -- 172.21.15.7:0/2857266115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea41545c0 con 0x7feea40cf2d0 2024-04-25T11:15:55.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.496+0000 7feea9000640 1 -- 172.21.15.7:0/2857266115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea406a100 con 0x7feea40cf2d0 2024-04-25T11:15:55.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.498+0000 7fee9a000640 1 -- 172.21.15.7:0/2857266115 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fee9c01b070 con 0x7feea40cf2d0 2024-04-25T11:15:55.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.499+0000 7fee9a000640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee840744b0 0x7fee840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:55.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.499+0000 7fee9a000640 1 -- 172.21.15.7:0/2857266115 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5335+0+0 (secure 0 0 0) 0x7fee9c094600 con 0x7feea40cf2d0 2024-04-25T11:15:55.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.499+0000 7feea1e00640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee840744b0 0x7fee840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:55.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.500+0000 7feea1e00640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee840744b0 0x7fee840769c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fee94033000 tx=0x7fee9402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:55.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.503+0000 7fee9a000640 1 -- 172.21.15.7:0/2857266115 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feea4145720 con 0x7feea40cf2d0 2024-04-25T11:15:55.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.748+0000 7feea9000640 1 -- 172.21.15.7:0/2857266115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feea40b22e0 con 0x7feea40cf2d0 2024-04-25T11:15:55.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.749+0000 7fee9a000640 1 -- 172.21.15.7:0/2857266115 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+3851 (secure 0 0 0) 0x7fee9c05ef90 con 0x7feea40cf2d0 2024-04-25T11:15:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:55.751 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/779859707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":779859707},{"type":"v1","addr":"172.21.15.87:6834","nonce":779859707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5839,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/454388957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":454388957},{"type":"v1","addr":"172.21.15.87:6835","nonce":454388957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6525,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/2318271315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2318271315},{"type":"v1","addr":"172.21.15.7:6836","nonce":2318271315}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6861,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/25101514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":25101514},{"type":"v1","addr":"172.21.15.7:6837","nonce":25101514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[]} 2024-04-25T11:15:55.751 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 45 2024-04-25T11:15:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.752+0000 7feea9000640 1 -- 172.21.15.7:0/2857266115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee840744b0 msgr2=0x7fee840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.752+0000 7feea9000640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee840744b0 0x7fee840769c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fee94033000 tx=0x7fee9402e000 comp rx=0 tx=0).stop 2024-04-25T11:15:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.752+0000 7feea9000640 1 -- 172.21.15.7:0/2857266115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feea40cf2d0 msgr2=0x7feea40cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.752+0000 7feea9000640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feea40cf2d0 0x7feea40cf6e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7feea40b4d30 tx=0x7fee9c00d890 comp rx=0 tx=0).stop 2024-04-25T11:15:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.752+0000 7feea9000640 1 -- 172.21.15.7:0/2857266115 shutdown_connections 2024-04-25T11:15:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.752+0000 7feea9000640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fee840744b0 0x7fee840769c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.752+0000 7feea9000640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feea40cf2d0 0x7feea40cf6e0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.752+0000 7feea9000640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feea4155380 0x7feea41451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.752+0000 7feea9000640 1 --2- 172.21.15.7:0/2857266115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fee880039b0 0x7feea4159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.752+0000 7feea9000640 1 -- 172.21.15.7:0/2857266115 >> 172.21.15.7:0/2857266115 conn(0x7feea4074200 msgr2=0x7feea40b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.753+0000 7feea9000640 1 -- 172.21.15.7:0/2857266115 shutdown_connections 2024-04-25T11:15:55.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:55.753+0000 7feea9000640 1 -- 172.21.15.7:0/2857266115 wait complete. 2024-04-25T11:15:55.779 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:15:55.779 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:15:55.780 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:15:55.780 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:15:55.780 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:15:55.780 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:15:55.780 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:15:55.780 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:15:55.780 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:15:55.780 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:15:55.781 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:15:55.781 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:15:55.781 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:15:55.781 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:15:55.781 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:15:55.781 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:15:55.820 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:15:55.820 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:15:55.824 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:15:55.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:15:55.826 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:15:55.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:15:55.827 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:15:55.827 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:15:55.873 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:15:55.874 DEBUG:teuthology.parallel:result is None 2024-04-25T11:15:55.874 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:15:55.875 DEBUG:teuthology.parallel:result is None 2024-04-25T11:15:55.880 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:15:55.880 DEBUG:teuthology.parallel:result is None 2024-04-25T11:15:55.881 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:15:55.881 DEBUG:teuthology.parallel:result is None 2024-04-25T11:15:55.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:15:56.101 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:15:56.104 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:15:56.104 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:15:56.105 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:15:56.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.138+0000 7f27afe00640 1 Processor -- start 2024-04-25T11:15:56.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.138+0000 7f27afe00640 1 -- start start 2024-04-25T11:15:56.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.140+0000 7f27afe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f27b0070520 0x7f27b00b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:56.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.140+0000 7f27afe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27b00b0e40 con 0x7f27b0156d90 2024-04-25T11:15:56.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.140+0000 7f27afe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27b006f760 con 0x7f27b0164160 2024-04-25T11:15:56.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.140+0000 7f27afe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f27b006fa40 con 0x7f27b0070520 2024-04-25T11:15:56.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.140+0000 7f27aea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f27b0070520 0x7f27b00b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:56.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.140+0000 7f27aea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f27b0070520 0x7f27b00b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41666/0 (socket says 172.21.15.7:41666) 2024-04-25T11:15:56.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.140+0000 7f27aea00640 1 -- 172.21.15.7:0/4074689167 learned_addr learned my addr 172.21.15.7:0/4074689167 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:56.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.141+0000 7f27aea00640 1 -- 172.21.15.7:0/4074689167 >> v1:172.21.15.87:6789/0 conn(0x7f27b0164160 legacy=0x7f27b006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:56.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.141+0000 7f27aea00640 1 -- 172.21.15.7:0/4074689167 >> v1:172.21.15.7:6789/0 conn(0x7f27b0156d90 legacy=0x7f27b0163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:56.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.141+0000 7f27aea00640 1 -- 172.21.15.7:0/4074689167 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b006fd20 con 0x7f27b0070520 2024-04-25T11:15:56.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.142+0000 7f27aea00640 1 --2- 172.21.15.7:0/4074689167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27b0070520 0x7f27b00b0900 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009870 tx=0x7f27a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cb425f4fbb79e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:56.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.142+0000 7f27ad600640 1 -- 172.21.15.7:0/4074689167 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0037070 con 0x7f27b0070520 2024-04-25T11:15:56.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.142+0000 7f27ad600640 1 -- 172.21.15.7:0/4074689167 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a0032440 con 0x7f27b0070520 2024-04-25T11:15:56.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.142+0000 7f27ad600640 1 -- 172.21.15.7:0/4074689167 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0032810 con 0x7f27b0070520 2024-04-25T11:15:56.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.143+0000 7f27afe00640 1 -- 172.21.15.7:0/4074689167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27b0070520 msgr2=0x7f27b00b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:56.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.143+0000 7f27afe00640 1 --2- 172.21.15.7:0/4074689167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27b0070520 0x7f27b00b0900 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009870 tx=0x7f27a002a270 comp rx=0 tx=0).stop 2024-04-25T11:15:56.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.143+0000 7f27afe00640 1 -- 172.21.15.7:0/4074689167 shutdown_connections 2024-04-25T11:15:56.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.143+0000 7f27afe00640 1 --2- 172.21.15.7:0/4074689167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27b0070520 0x7f27b00b0900 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:56.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.143+0000 7f27afe00640 1 -- 172.21.15.7:0/4074689167 >> 172.21.15.7:0/4074689167 conn(0x7f27b00d1540 msgr2=0x7f27b00d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:56.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.143+0000 7f27afe00640 1 -- 172.21.15.7:0/4074689167 shutdown_connections 2024-04-25T11:15:56.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.143+0000 7f27afe00640 1 -- 172.21.15.7:0/4074689167 wait complete. 2024-04-25T11:15:56.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.145+0000 7f27afe00640 1 Processor -- start 2024-04-25T11:15:56.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.146+0000 7f27afe00640 1 -- start start 2024-04-25T11:15:56.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.147+0000 7f27afe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27b0070520 0x7f27b01f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:56.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.147+0000 7f27afe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27b0156d90 0x7f27b0157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:56.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.147+0000 7f27afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27b0164160 0x7f27b0157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:56.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.147+0000 7f27afe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0070ab0 con 0x7f27b0164160 2024-04-25T11:15:56.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.147+0000 7f27afe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0070930 con 0x7f27b0156d90 2024-04-25T11:15:56.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.147+0000 7f27afe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f27b01571a0 con 0x7f27b0070520 2024-04-25T11:15:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.148+0000 7f27af400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27b0164160 0x7f27b0157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.148+0000 7f27af400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27b0164160 0x7f27b0157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41682/0 (socket says 172.21.15.7:41682) 2024-04-25T11:15:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.148+0000 7f27af400640 1 -- 172.21.15.7:0/3618110713 learned_addr learned my addr 172.21.15.7:0/3618110713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.148+0000 7f27ae000640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27b0156d90 0x7f27b0157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.148+0000 7f27aea00640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27b0070520 0x7f27b01f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.148+0000 7f27af400640 1 -- 172.21.15.7:0/3618110713 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27b0070520 msgr2=0x7f27b01f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.148+0000 7f27af400640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27b0070520 0x7f27b01f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.148+0000 7f27af400640 1 -- 172.21.15.7:0/3618110713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27b0156d90 msgr2=0x7f27b0157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.148+0000 7f27af400640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27b0156d90 0x7f27b0157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.148+0000 7f27af400640 1 -- 172.21.15.7:0/3618110713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b01fab70 con 0x7f27b0164160 2024-04-25T11:15:56.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.148+0000 7f27af400640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27b0164160 0x7f27b0157b90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f27a800c760 tx=0x7f27a800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.149+0000 7f27a6a00640 1 -- 172.21.15.7:0/3618110713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8016020 con 0x7f27b0164160 2024-04-25T11:15:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.149+0000 7f27a6a00640 1 -- 172.21.15.7:0/3618110713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a8006d50 con 0x7f27b0164160 2024-04-25T11:15:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.149+0000 7f27a6a00640 1 -- 172.21.15.7:0/3618110713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8003bb0 con 0x7f27b0164160 2024-04-25T11:15:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.149+0000 7f27afe00640 1 -- 172.21.15.7:0/3618110713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b01f9db0 con 0x7f27b0164160 2024-04-25T11:15:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.149+0000 7f27afe00640 1 -- 172.21.15.7:0/3618110713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b01fa370 con 0x7f27b0164160 2024-04-25T11:15:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.150+0000 7f27afe00640 1 -- 172.21.15.7:0/3618110713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b00d1e40 con 0x7f27b0164160 2024-04-25T11:15:56.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.151+0000 7f27a6a00640 1 -- 172.21.15.7:0/3618110713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f27a80050d0 con 0x7f27b0164160 2024-04-25T11:15:56.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.152+0000 7f27a6a00640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2790074500 0x7f2790076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:56.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.152+0000 7f27a6a00640 1 -- 172.21.15.7:0/3618110713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5335+0+0 (secure 0 0 0) 0x7f27a8094aa0 con 0x7f27b0164160 2024-04-25T11:15:56.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.153+0000 7f27aea00640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2790074500 0x7f2790076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:56.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.153+0000 7f27aea00640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2790074500 0x7f2790076a10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f27a0034000 tx=0x7f27a002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:56.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.157+0000 7f27a6a00640 1 -- 172.21.15.7:0/3618110713 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27b00d1e40 con 0x7f27b0164160 2024-04-25T11:15:56.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:56.377+0000 7f27afe00640 1 -- 172.21.15.7:0/3618110713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f27b0158180 con 0x7f27b0164160 2024-04-25T11:15:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.194+0000 7f27a6a00640 1 -- 172.21.15.7:0/3618110713 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) ==== 97+0+0 (secure 0 0 0) 0x7f27a8020020 con 0x7f27b0164160 2024-04-25T11:15:57.196 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-04-25T11:15:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.198+0000 7f27afe00640 1 -- 172.21.15.7:0/3618110713 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2790074500 msgr2=0x7f2790076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.198+0000 7f27afe00640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2790074500 0x7f2790076a10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f27a0034000 tx=0x7f27a002e000 comp rx=0 tx=0).stop 2024-04-25T11:15:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.198+0000 7f27afe00640 1 -- 172.21.15.7:0/3618110713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27b0164160 msgr2=0x7f27b0157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.198+0000 7f27afe00640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27b0164160 0x7f27b0157b90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f27a800c760 tx=0x7f27a800cb60 comp rx=0 tx=0).stop 2024-04-25T11:15:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.199+0000 7f27afe00640 1 -- 172.21.15.7:0/3618110713 shutdown_connections 2024-04-25T11:15:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.199+0000 7f27afe00640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2790074500 0x7f2790076a10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.199+0000 7f27afe00640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27b0164160 0x7f27b0157b90 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.199+0000 7f27afe00640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27b0156d90 0x7f27b0157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.199+0000 7f27afe00640 1 --2- 172.21.15.7:0/3618110713 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27b0070520 0x7f27b01f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.199+0000 7f27afe00640 1 -- 172.21.15.7:0/3618110713 >> 172.21.15.7:0/3618110713 conn(0x7f27b00d1540 msgr2=0x7f27b00b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.200+0000 7f27afe00640 1 -- 172.21.15.7:0/3618110713 shutdown_connections 2024-04-25T11:15:57.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.200+0000 7f27afe00640 1 -- 172.21.15.7:0/3618110713 wait complete. 2024-04-25T11:15:57.228 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:15:57.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.507+0000 7f8983200640 1 Processor -- start 2024-04-25T11:15:57.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.507+0000 7f8983200640 1 -- start start 2024-04-25T11:15:57.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.509+0000 7f8983200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f898406dc90 0x7f89840d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:57.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.510+0000 7f8983200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89840cffa0 con 0x7f8984070810 2024-04-25T11:15:57.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.510+0000 7f8983200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8984160d90 con 0x7f8984070c20 2024-04-25T11:15:57.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.510+0000 7f8983200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8984161070 con 0x7f898406dc90 2024-04-25T11:15:57.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.510+0000 7f8981e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8984070810 0x7f898406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53652/0 (socket says 172.21.15.7:53652) 2024-04-25T11:15:57.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.510+0000 7f8981e00640 1 -- 172.21.15.7:0/2962434283 learned_addr learned my addr 172.21.15.7:0/2962434283 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:57.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.510+0000 7f8981400640 1 --2- 172.21.15.7:0/2962434283 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f898406dc90 0x7f89840d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:57.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.511+0000 7f8980a00640 1 -- 172.21.15.7:0/2962434283 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1365073675 0 0) 0x7f8974001110 con 0x7f8984070810 2024-04-25T11:15:57.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.511+0000 7f8980a00640 1 -- 172.21.15.7:0/2962434283 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8968003520 con 0x7f8984070810 2024-04-25T11:15:57.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.511+0000 7f8980a00640 1 -- 172.21.15.7:0/2962434283 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172222587 0 0) 0x7f8984160d90 con 0x7f8984070c20 2024-04-25T11:15:57.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.511+0000 7f8980a00640 1 -- 172.21.15.7:0/2962434283 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8974001110 con 0x7f8984070c20 2024-04-25T11:15:57.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.511+0000 7f8980a00640 1 -- 172.21.15.7:0/2962434283 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3323757569 0 0) 0x7f8968003520 con 0x7f8984070810 2024-04-25T11:15:57.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.511+0000 7f8981400640 1 -- 172.21.15.7:0/2962434283 >> v1:172.21.15.87:6789/0 conn(0x7f8984070c20 legacy=0x7f89840d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:57.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.511+0000 7f8981400640 1 -- 172.21.15.7:0/2962434283 >> v1:172.21.15.7:6789/0 conn(0x7f8984070810 legacy=0x7f898406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:57.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.511+0000 7f8981400640 1 -- 172.21.15.7:0/2962434283 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8984161350 con 0x7f898406dc90 2024-04-25T11:15:57.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.512+0000 7f8981400640 1 --2- 172.21.15.7:0/2962434283 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f898406dc90 0x7f89840d0460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89700097d0 tx=0x7f897002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa2e5c596aab3a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:57.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.513+0000 7f8980a00640 1 -- 172.21.15.7:0/2962434283 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970037070 con 0x7f898406dc90 2024-04-25T11:15:57.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.513+0000 7f8980a00640 1 -- 172.21.15.7:0/2962434283 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897002ac30 con 0x7f898406dc90 2024-04-25T11:15:57.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.513+0000 7f8980a00640 1 -- 172.21.15.7:0/2962434283 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974001840 con 0x7f898406dc90 2024-04-25T11:15:57.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.514+0000 7f8983200640 1 -- 172.21.15.7:0/2962434283 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f898406dc90 msgr2=0x7f89840d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:57.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.514+0000 7f8983200640 1 --2- 172.21.15.7:0/2962434283 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f898406dc90 0x7f89840d0460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f89700097d0 tx=0x7f897002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.514+0000 7f8983200640 1 -- 172.21.15.7:0/2962434283 shutdown_connections 2024-04-25T11:15:57.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.514+0000 7f8983200640 1 --2- 172.21.15.7:0/2962434283 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f898406dc90 0x7f89840d0460 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.514+0000 7f8983200640 1 -- 172.21.15.7:0/2962434283 >> 172.21.15.7:0/2962434283 conn(0x7f8984073090 msgr2=0x7f89840734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:57.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.514+0000 7f8983200640 1 -- 172.21.15.7:0/2962434283 shutdown_connections 2024-04-25T11:15:57.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.514+0000 7f8983200640 1 -- 172.21.15.7:0/2962434283 wait complete. 2024-04-25T11:15:57.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.516+0000 7f8983200640 1 Processor -- start 2024-04-25T11:15:57.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.516+0000 7f8983200640 1 -- start start 2024-04-25T11:15:57.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.518+0000 7f8983200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8984070810 0x7f8984159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:57.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.518+0000 7f8983200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8984070c20 0x7f898413eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:57.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.518+0000 7f8983200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8984157210 0x7f898415fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:57.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.518+0000 7f8983200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8984143530 con 0x7f8984070810 2024-04-25T11:15:57.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.518+0000 7f8983200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89841433b0 con 0x7f8984070c20 2024-04-25T11:15:57.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.518+0000 7f8983200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f898415e4d0 con 0x7f8984157210 2024-04-25T11:15:57.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.518+0000 7f8981e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8984070810 0x7f8984159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:57.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.518+0000 7f8981e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8984070810 0x7f8984159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41700/0 (socket says 172.21.15.7:41700) 2024-04-25T11:15:57.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.518+0000 7f8981e00640 1 -- 172.21.15.7:0/1658731141 learned_addr learned my addr 172.21.15.7:0/1658731141 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:57.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.519+0000 7f8982800640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8984157210 0x7f898415fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:57.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.519+0000 7f8981400640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8984070c20 0x7f898413eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:57.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.519+0000 7f8981e00640 1 -- 172.21.15.7:0/1658731141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8984157210 msgr2=0x7f898415fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:57.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.519+0000 7f8981e00640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8984157210 0x7f898415fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.519+0000 7f8981e00640 1 -- 172.21.15.7:0/1658731141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8984070c20 msgr2=0x7f898413eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:57.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.519+0000 7f8981e00640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8984070c20 0x7f898413eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.519+0000 7f8981e00640 1 -- 172.21.15.7:0/1658731141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898415f3d0 con 0x7f8984070810 2024-04-25T11:15:57.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.519+0000 7f8982800640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8984157210 0x7f898415fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:15:57.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.519+0000 7f8981e00640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8984070810 0x7f8984159bb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f89740013b0 tx=0x7f897400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:57.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.520+0000 7f897a000640 1 -- 172.21.15.7:0/1658731141 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974017070 con 0x7f8984070810 2024-04-25T11:15:57.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.520+0000 7f897a000640 1 -- 172.21.15.7:0/1658731141 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8974005af0 con 0x7f8984070810 2024-04-25T11:15:57.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.520+0000 7f897a000640 1 -- 172.21.15.7:0/1658731141 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8974007070 con 0x7f8984070810 2024-04-25T11:15:57.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.520+0000 7f8983200640 1 -- 172.21.15.7:0/1658731141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898415b210 con 0x7f8984070810 2024-04-25T11:15:57.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.520+0000 7f8983200640 1 -- 172.21.15.7:0/1658731141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898415f950 con 0x7f8984070810 2024-04-25T11:15:57.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.521+0000 7f8983200640 1 -- 172.21.15.7:0/1658731141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8984050740 con 0x7f8984070810 2024-04-25T11:15:57.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.522+0000 7f897a000640 1 -- 172.21.15.7:0/1658731141 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f897401a070 con 0x7f8984070810 2024-04-25T11:15:57.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.523+0000 7f897a000640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89640744b0 0x7f89640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:57.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.524+0000 7f897a000640 1 -- 172.21.15.7:0/1658731141 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5249+0+0 (secure 0 0 0) 0x7f89740949f0 con 0x7f8984070810 2024-04-25T11:15:57.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.524+0000 7f8981400640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89640744b0 0x7f89640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:57.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.525+0000 7f8981400640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89640744b0 0x7f89640769c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8970034000 tx=0x7f897002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.529+0000 7f897a000640 1 -- 172.21.15.7:0/1658731141 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89840cffa0 con 0x7f8984070810 2024-04-25T11:15:57.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.807+0000 7f8983200640 1 -- 172.21.15.7:0/1658731141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f89840b3100 con 0x7f8984070810 2024-04-25T11:15:57.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.808+0000 7f897a000640 1 -- 172.21.15.7:0/1658731141 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f897405f2a0 con 0x7f8984070810 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w==", 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw==", 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw==", 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:57.811 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A==", 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA==", 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-04-25T11:15:57.813 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg==", 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw==", 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.815 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-04-25T11:15:57.815 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ==", 2024-04-25T11:15:57.815 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.815 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:57.815 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:57.815 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.815 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw==", 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng==", 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:57.816 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ==", 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A==", 2024-04-25T11:15:57.817 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg==", 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA==", 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg==", 2024-04-25T11:15:57.819 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.820 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:57.820 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:15:57.820 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:15:57.820 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:15:57.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.820 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-04-25T11:15:57.820 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ==", 2024-04-25T11:15:57.821 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.821 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:15:57.821 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:15:57.821 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:15:57.821 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:15:57.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.822 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-04-25T11:15:57.822 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDZOSpmS8ObHxAAkXXXg1OI1YFC2/OZER3woA==", 2024-04-25T11:15:57.822 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.822 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:15:57.822 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:15:57.823 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:15:57.823 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.823 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.823 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-04-25T11:15:57.824 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw==", 2024-04-25T11:15:57.824 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.824 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:15:57.824 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:15:57.824 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:15:57.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:57.825 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-04-25T11:15:57.825 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w==", 2024-04-25T11:15:57.825 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:15:57.825 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:15:57.825 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:15:57.826 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:15:57.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:57.826 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-25T11:15:57.826 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:15:57.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.811+0000 7f8983200640 1 -- 172.21.15.7:0/1658731141 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89640744b0 msgr2=0x7f89640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:57.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.811+0000 7f8983200640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89640744b0 0x7f89640769c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8970034000 tx=0x7f897002e000 comp rx=0 tx=0).stop 2024-04-25T11:15:57.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.811+0000 7f8983200640 1 -- 172.21.15.7:0/1658731141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8984070810 msgr2=0x7f8984159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.811+0000 7f8983200640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8984070810 0x7f8984159bb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f89740013b0 tx=0x7f897400cb90 comp rx=0 tx=0).stop 2024-04-25T11:15:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.811+0000 7f8983200640 1 -- 172.21.15.7:0/1658731141 shutdown_connections 2024-04-25T11:15:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.811+0000 7f8983200640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89640744b0 0x7f89640769c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.811+0000 7f8983200640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8984157210 0x7f898415fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.811+0000 7f8983200640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8984070c20 0x7f898413eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.811+0000 7f8983200640 1 --2- 172.21.15.7:0/1658731141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8984070810 0x7f8984159bb0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.811+0000 7f8983200640 1 -- 172.21.15.7:0/1658731141 >> 172.21.15.7:0/1658731141 conn(0x7f8984073090 msgr2=0x7f89840d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:57.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.812+0000 7f8983200640 1 -- 172.21.15.7:0/1658731141 shutdown_connections 2024-04-25T11:15:57.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:57.812+0000 7f8983200640 1 -- 172.21.15.7:0/1658731141 wait complete. 2024-04-25T11:15:57.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:15:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.117+0000 7f5c80e00640 1 Processor -- start 2024-04-25T11:15:58.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.117+0000 7f5c80e00640 1 -- start start 2024-04-25T11:15:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.120+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c0b6790 0x7f5c7c164640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.120+0000 7f5c80e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c7c0aea00 con 0x7f5c7c15de70 2024-04-25T11:15:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.120+0000 7f5c80e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c7c0b13e0 con 0x7f5c7c0b0c20 2024-04-25T11:15:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.120+0000 7f5c80e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7c0cdb70 con 0x7f5c7c0b6790 2024-04-25T11:15:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.120+0000 7f5c79e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5c7c15de70 0x7f5c7c0b0510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53668/0 (socket says 172.21.15.7:53668) 2024-04-25T11:15:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.120+0000 7f5c79e00640 1 -- 172.21.15.7:0/4081138919 learned_addr learned my addr 172.21.15.7:0/4081138919 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.121+0000 7f5c71400640 1 --2- 172.21.15.7:0/4081138919 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c0b6790 0x7f5c7c164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.121+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3578587809 0 0) 0x7f5c7c0aea00 con 0x7f5c7c15de70 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.121+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c60003520 con 0x7f5c7c15de70 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 427794507 0 0) 0x7f5c60003520 con 0x7f5c7c15de70 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c0b6790 msgr2=0x7f5c7c164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 --2- 172.21.15.7:0/4081138919 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c0b6790 0x7f5c7c164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 >> v1:172.21.15.87:6789/0 conn(0x7f5c7c0b0c20 legacy=0x7f5c7c1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c0cddc0 con 0x7f5c7c15de70 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f5c6c001670 con 0x7f5c7c15de70 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 >> v1:172.21.15.7:6789/0 conn(0x7f5c7c15de70 legacy=0x7f5c7c0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 --2- 172.21.15.7:0/4081138919 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c0b6790 0x7f5c600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 --2- 172.21.15.7:0/4081138919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c60003f40 0x7f5c60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 --2- 172.21.15.7:0/4081138919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 0x7f5c60005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c0cdb70 con 0x7f5c60004d40 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6000bb80 con 0x7f5c60003f40 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6000bd80 con 0x7f5c7c0b6790 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c7a800640 1 --2- 172.21.15.7:0/4081138919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 0x7f5c60005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.122+0000 7f5c71400640 1 --2- 172.21.15.7:0/4081138919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c60003f40 0x7f5c60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.123+0000 7f5c7a800640 1 -- 172.21.15.7:0/4081138919 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c0b6790 msgr2=0x7f5c600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.123+0000 7f5c7a800640 1 --2- 172.21.15.7:0/4081138919 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c0b6790 0x7f5c600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.123+0000 7f5c7a800640 1 -- 172.21.15.7:0/4081138919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c60003f40 msgr2=0x7f5c60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.123+0000 7f5c7a800640 1 --2- 172.21.15.7:0/4081138919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c60003f40 0x7f5c60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.123+0000 7f5c7a800640 1 -- 172.21.15.7:0/4081138919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c60003520 con 0x7f5c60004d40 2024-04-25T11:15:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.124+0000 7f5c7a800640 1 --2- 172.21.15.7:0/4081138919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 0x7f5c60005150 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0b4740 tx=0x7f5c60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d23e4872a9048 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.124+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74037070 con 0x7f5c60004d40 2024-04-25T11:15:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.124+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c74032440 con 0x7f5c60004d40 2024-04-25T11:15:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.124+0000 7f5c79400640 1 -- 172.21.15.7:0/4081138919 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74001b30 con 0x7f5c60004d40 2024-04-25T11:15:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.125+0000 7f5c80e00640 1 -- 172.21.15.7:0/4081138919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 msgr2=0x7f5c60005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.125+0000 7f5c80e00640 1 --2- 172.21.15.7:0/4081138919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 0x7f5c60005150 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0b4740 tx=0x7f5c60005830 comp rx=0 tx=0).stop 2024-04-25T11:15:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.125+0000 7f5c7a800640 1 -- 172.21.15.7:0/4081138919 reap_dead start 2024-04-25T11:15:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.125+0000 7f5c80e00640 1 -- 172.21.15.7:0/4081138919 shutdown_connections 2024-04-25T11:15:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.125+0000 7f5c80e00640 1 -- 172.21.15.7:0/4081138919 >> 172.21.15.7:0/4081138919 conn(0x7f5c7c074200 msgr2=0x7f5c7c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.125+0000 7f5c80e00640 1 -- 172.21.15.7:0/4081138919 shutdown_connections 2024-04-25T11:15:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.125+0000 7f5c80e00640 1 -- 172.21.15.7:0/4081138919 wait complete. 2024-04-25T11:15:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.127+0000 7f5c80e00640 1 Processor -- start 2024-04-25T11:15:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.128+0000 7f5c80e00640 1 -- start start 2024-04-25T11:15:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.129+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 0x7f5c7c15c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.129+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c15aab0 0x7f5c7c142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.129+0000 7f5c80e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c7c1598b0 0x7f5c7c159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.129+0000 7f5c80e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c06f300 con 0x7f5c60004d40 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.129+0000 7f5c80e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c0aed30 con 0x7f5c7c1598b0 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.129+0000 7f5c80e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7c06f480 con 0x7f5c7c15aab0 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.130+0000 7f5c79e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 0x7f5c7c15c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.130+0000 7f5c79e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 0x7f5c7c15c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41726/0 (socket says 172.21.15.7:41726) 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.130+0000 7f5c79e00640 1 -- 172.21.15.7:0/3177427078 learned_addr learned my addr 172.21.15.7:0/3177427078 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.130+0000 7f5c71400640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c15aab0 0x7f5c7c142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.130+0000 7f5c79e00640 1 -- 172.21.15.7:0/3177427078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c15aab0 msgr2=0x7f5c7c142220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.130+0000 7f5c7a800640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c7c1598b0 0x7f5c7c159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.130+0000 7f5c79e00640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c15aab0 0x7f5c7c142220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.130+0000 7f5c79e00640 1 -- 172.21.15.7:0/3177427078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c7c1598b0 msgr2=0x7f5c7c159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.130+0000 7f5c79e00640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c7c1598b0 0x7f5c7c159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.130+0000 7f5c79e00640 1 -- 172.21.15.7:0/3177427078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c15a200 con 0x7f5c60004d40 2024-04-25T11:15:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.130+0000 7f5c71400640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c15aab0 0x7f5c7c142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.131+0000 7f5c79e00640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 0x7f5c7c15c2d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c760 tx=0x7f5c6c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.131+0000 7f5c72a00640 1 -- 172.21.15.7:0/3177427078 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c018070 con 0x7f5c60004d40 2024-04-25T11:15:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.131+0000 7f5c72a00640 1 -- 172.21.15.7:0/3177427078 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c007500 con 0x7f5c60004d40 2024-04-25T11:15:58.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.131+0000 7f5c72a00640 1 -- 172.21.15.7:0/3177427078 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c004a20 con 0x7f5c60004d40 2024-04-25T11:15:58.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.131+0000 7f5c80e00640 1 -- 172.21.15.7:0/3177427078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c15cbe0 con 0x7f5c60004d40 2024-04-25T11:15:58.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.131+0000 7f5c80e00640 1 -- 172.21.15.7:0/3177427078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c15d1a0 con 0x7f5c60004d40 2024-04-25T11:15:58.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.132+0000 7f5c80e00640 1 -- 172.21.15.7:0/3177427078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c7c06a100 con 0x7f5c60004d40 2024-04-25T11:15:58.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.134+0000 7f5c72a00640 1 -- 172.21.15.7:0/3177427078 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5c6c01b070 con 0x7f5c60004d40 2024-04-25T11:15:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.135+0000 7f5c72a00640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c580744b0 0x7f5c580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.135+0000 7f5c72a00640 1 -- 172.21.15.7:0/3177427078 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5249+0+0 (secure 0 0 0) 0x7f5c6c0945a0 con 0x7f5c60004d40 2024-04-25T11:15:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.135+0000 7f5c71400640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c580744b0 0x7f5c580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.136+0000 7f5c71400640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c580744b0 0x7f5c580769c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c164690 tx=0x7f5c64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:58.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.140+0000 7f5c72a00640 1 -- 172.21.15.7:0/3177427078 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c7c15a200 con 0x7f5c60004d40 2024-04-25T11:15:58.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.344+0000 7f5c80e00640 1 -- 172.21.15.7:0/3177427078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c7c070310 con 0x7f5c60004d40 2024-04-25T11:15:58.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.346+0000 7f5c72a00640 1 -- 172.21.15.7:0/3177427078 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+13069 (secure 0 0 0) 0x7f5c6c08be20 con 0x7f5c60004d40 2024-04-25T11:15:58.347 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:58.348 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:15:57.190183+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:15:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 -- 172.21.15.7:0/3177427078 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c580744b0 msgr2=0x7f5c580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c580744b0 0x7f5c580769c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c164690 tx=0x7f5c64008040 comp rx=0 tx=0).stop 2024-04-25T11:15:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 -- 172.21.15.7:0/3177427078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 msgr2=0x7f5c7c15c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 0x7f5c7c15c2d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c760 tx=0x7f5c6c00cc20 comp rx=0 tx=0).stop 2024-04-25T11:15:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 -- 172.21.15.7:0/3177427078 shutdown_connections 2024-04-25T11:15:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5c580744b0 0x7f5c580769c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c7c1598b0 0x7f5c7c159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7c15aab0 0x7f5c7c142220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 --2- 172.21.15.7:0/3177427078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c60004d40 0x7f5c7c15c2d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 -- 172.21.15.7:0/3177427078 >> 172.21.15.7:0/3177427078 conn(0x7f5c7c074200 msgr2=0x7f5c7c0b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 -- 172.21.15.7:0/3177427078 shutdown_connections 2024-04-25T11:15:58.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.349+0000 7f5c80e00640 1 -- 172.21.15.7:0/3177427078 wait complete. 2024-04-25T11:15:58.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:15:58.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:15:58.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.665+0000 7ff92b200640 1 Processor -- start 2024-04-25T11:15:58.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.665+0000 7ff92b200640 1 -- start start 2024-04-25T11:15:58.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.668+0000 7ff92b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c0cece0 0x7ff92c06f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.668+0000 7ff92b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff92c06fc70 0x7ff92c0b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.668+0000 7ff92b200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff92c070080 con 0x7ff92c0ce8d0 2024-04-25T11:15:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.668+0000 7ff92b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff92c0b4830 con 0x7ff92c06fc70 2024-04-25T11:15:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.668+0000 7ff92b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff92c0b4b10 con 0x7ff92c0cece0 2024-04-25T11:15:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.668+0000 7ff929e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff92c0ce8d0 0x7ff92c15c2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53678/0 (socket says 172.21.15.7:53678) 2024-04-25T11:15:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.668+0000 7ff929e00640 1 -- 172.21.15.7:0/4271865237 learned_addr learned my addr 172.21.15.7:0/4271865237 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff921400640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c0cece0 0x7ff92c06f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff92a800640 1 --2- 172.21.15.7:0/4271865237 >> v2:172.21.15.87:3300/0 conn(0x7ff92c06fc70 0x7ff92c0b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 256355340 0 0) 0x7ff91c001490 con 0x7ff92c0ce8d0 2024-04-25T11:15:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff910003520 con 0x7ff92c0ce8d0 2024-04-25T11:15:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1613329325 0 0) 0x7ff910003520 con 0x7ff92c0ce8d0 2024-04-25T11:15:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c0cece0 msgr2=0x7ff92c06f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff929400640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c0cece0 0x7ff92c06f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 >> v2:172.21.15.87:3300/0 conn(0x7ff92c06fc70 msgr2=0x7ff92c0b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff929400640 1 --2- 172.21.15.7:0/4271865237 >> v2:172.21.15.87:3300/0 conn(0x7ff92c06fc70 0x7ff92c0b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92c0b4df0 con 0x7ff92c0ce8d0 2024-04-25T11:15:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7ff91c001bc0 con 0x7ff92c0ce8d0 2024-04-25T11:15:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 >> v1:172.21.15.7:6789/0 conn(0x7ff92c0ce8d0 legacy=0x7ff92c15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.669+0000 7ff929400640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c0cece0 0x7ff9100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff929400640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff910003f30 0x7ff910004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff929400640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910004d40 0x7ff910005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c0b4830 con 0x7ff910004d40 2024-04-25T11:15:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c0b4b10 con 0x7ff910003f30 2024-04-25T11:15:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff91000dc20 con 0x7ff92c0cece0 2024-04-25T11:15:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff92a800640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910004d40 0x7ff910005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff929e00640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c0cece0 0x7ff9100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff921400640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff910003f30 0x7ff910004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff92a800640 1 -- 172.21.15.7:0/4271865237 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c0cece0 msgr2=0x7ff9100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff92a800640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c0cece0 0x7ff9100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff92a800640 1 -- 172.21.15.7:0/4271865237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff910003f30 msgr2=0x7ff910004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff92a800640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff910003f30 0x7ff910004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.670+0000 7ff92a800640 1 -- 172.21.15.7:0/4271865237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff910003520 con 0x7ff910004d40 2024-04-25T11:15:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.671+0000 7ff92a800640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910004d40 0x7ff910005150 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0cde50 tx=0x7ff910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0bc3927e7be7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.672+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924036070 con 0x7ff910004d40 2024-04-25T11:15:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.672+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff924031400 con 0x7ff910004d40 2024-04-25T11:15:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.672+0000 7ff929400640 1 -- 172.21.15.7:0/4271865237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924009690 con 0x7ff910004d40 2024-04-25T11:15:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.672+0000 7ff92b200640 1 -- 172.21.15.7:0/4271865237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910004d40 msgr2=0x7ff910005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.672+0000 7ff92b200640 1 --2- 172.21.15.7:0/4271865237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff910004d40 0x7ff910005150 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0cde50 tx=0x7ff910005830 comp rx=0 tx=0).stop 2024-04-25T11:15:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.672+0000 7ff92a800640 1 -- 172.21.15.7:0/4271865237 reap_dead start 2024-04-25T11:15:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.673+0000 7ff92b200640 1 -- 172.21.15.7:0/4271865237 shutdown_connections 2024-04-25T11:15:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.673+0000 7ff92b200640 1 -- 172.21.15.7:0/4271865237 >> 172.21.15.7:0/4271865237 conn(0x7ff92c0740f0 msgr2=0x7ff92c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:58.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.673+0000 7ff92b200640 1 -- 172.21.15.7:0/4271865237 shutdown_connections 2024-04-25T11:15:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.673+0000 7ff92b200640 1 -- 172.21.15.7:0/4271865237 wait complete. 2024-04-25T11:15:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.675+0000 7ff92b200640 1 Processor -- start 2024-04-25T11:15:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.675+0000 7ff92b200640 1 -- start start 2024-04-25T11:15:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.677+0000 7ff92b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff910004d40 0x7ff92c145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.677+0000 7ff92b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c1605b0 0x7ff92c157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.677+0000 7ff92b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c153cf0 0x7ff92c157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.677+0000 7ff92b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c070e70 con 0x7ff92c153cf0 2024-04-25T11:15:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.677+0000 7ff92b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c15c9f0 con 0x7ff910004d40 2024-04-25T11:15:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.677+0000 7ff92b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff92c070ff0 con 0x7ff92c1605b0 2024-04-25T11:15:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.677+0000 7ff92a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c153cf0 0x7ff92c157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.677+0000 7ff92a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c153cf0 0x7ff92c157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41744/0 (socket says 172.21.15.7:41744) 2024-04-25T11:15:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.677+0000 7ff92a800640 1 -- 172.21.15.7:0/1524462374 learned_addr learned my addr 172.21.15.7:0/1524462374 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.678+0000 7ff929e00640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff910004d40 0x7ff92c145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.678+0000 7ff921400640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c1605b0 0x7ff92c157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.678+0000 7ff92a800640 1 -- 172.21.15.7:0/1524462374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c1605b0 msgr2=0x7ff92c157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.678+0000 7ff92a800640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c1605b0 0x7ff92c157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.678+0000 7ff92a800640 1 -- 172.21.15.7:0/1524462374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff910004d40 msgr2=0x7ff92c145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.678+0000 7ff92a800640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff910004d40 0x7ff92c145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.678+0000 7ff92a800640 1 -- 172.21.15.7:0/1524462374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92c155060 con 0x7ff92c153cf0 2024-04-25T11:15:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.678+0000 7ff92a800640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c153cf0 0x7ff92c157aa0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff924034040 tx=0x7ff924004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.679+0000 7ff922a00640 1 -- 172.21.15.7:0/1524462374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924036040 con 0x7ff92c153cf0 2024-04-25T11:15:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.679+0000 7ff922a00640 1 -- 172.21.15.7:0/1524462374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff924041070 con 0x7ff92c153cf0 2024-04-25T11:15:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.679+0000 7ff922a00640 1 -- 172.21.15.7:0/1524462374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924004520 con 0x7ff92c153cf0 2024-04-25T11:15:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.679+0000 7ff92b200640 1 -- 172.21.15.7:0/1524462374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92c146330 con 0x7ff92c153cf0 2024-04-25T11:15:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.679+0000 7ff92b200640 1 -- 172.21.15.7:0/1524462374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92c1555b0 con 0x7ff92c153cf0 2024-04-25T11:15:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.680+0000 7ff92b200640 1 -- 172.21.15.7:0/1524462374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff92c050740 con 0x7ff92c153cf0 2024-04-25T11:15:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.681+0000 7ff922a00640 1 -- 172.21.15.7:0/1524462374 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff924039070 con 0x7ff92c153cf0 2024-04-25T11:15:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.682+0000 7ff922a00640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9080744b0 0x7ff9080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.682+0000 7ff922a00640 1 -- 172.21.15.7:0/1524462374 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5249+0+0 (secure 0 0 0) 0x7ff9240b4850 con 0x7ff92c153cf0 2024-04-25T11:15:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.683+0000 7ff929e00640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9080744b0 0x7ff9080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:58.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.683+0000 7ff929e00640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9080744b0 0x7ff9080769c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff91c001630 tx=0x7ff91c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.687+0000 7ff922a00640 1 -- 172.21.15.7:0/1524462374 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff92404a020 con 0x7ff92c153cf0 2024-04-25T11:15:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.890+0000 7ff92b200640 1 -- 172.21.15.7:0/1524462374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff92c15e520 con 0x7ff92c153cf0 2024-04-25T11:15:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.891+0000 7ff922a00640 1 -- 172.21.15.7:0/1524462374 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7ff92407f220 con 0x7ff92c153cf0 2024-04-25T11:15:58.892 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:15:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 -- 172.21.15.7:0/1524462374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9080744b0 msgr2=0x7ff9080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9080744b0 0x7ff9080769c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff91c001630 tx=0x7ff91c009040 comp rx=0 tx=0).stop 2024-04-25T11:15:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 -- 172.21.15.7:0/1524462374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c153cf0 msgr2=0x7ff92c157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c153cf0 0x7ff92c157aa0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff924034040 tx=0x7ff924004730 comp rx=0 tx=0).stop 2024-04-25T11:15:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 -- 172.21.15.7:0/1524462374 shutdown_connections 2024-04-25T11:15:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9080744b0 0x7ff9080769c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff92c153cf0 0x7ff92c157aa0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff92c1605b0 0x7ff92c157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 --2- 172.21.15.7:0/1524462374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff910004d40 0x7ff92c145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 -- 172.21.15.7:0/1524462374 >> 172.21.15.7:0/1524462374 conn(0x7ff92c0740f0 msgr2=0x7ff92c14cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 -- 172.21.15.7:0/1524462374 shutdown_connections 2024-04-25T11:15:58.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:58.894+0000 7ff92b200640 1 -- 172.21.15.7:0/1524462374 wait complete. 2024-04-25T11:15:58.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:15:59.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.204+0000 7fab7a200640 1 Processor -- start 2024-04-25T11:15:59.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.204+0000 7fab7a200640 1 -- start start 2024-04-25T11:15:59.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.206+0000 7fab7a200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab7415baa0 0x7fab7415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.207+0000 7fab7a200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab7415e980 con 0x7fab74070640 2024-04-25T11:15:59.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.207+0000 7fab7a200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab7415ec60 con 0x7fab7406d580 2024-04-25T11:15:59.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.207+0000 7fab7a200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fab7415ef40 con 0x7fab7415baa0 2024-04-25T11:15:59.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.207+0000 7fab73e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fab74070640 0x7fab74161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53694/0 (socket says 172.21.15.7:53694) 2024-04-25T11:15:59.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.207+0000 7fab73e00640 1 -- 172.21.15.7:0/1471317735 learned_addr learned my addr 172.21.15.7:0/1471317735 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:59.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.207+0000 7fab78e00640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab7415baa0 0x7fab7415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.207+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4178317510 0 0) 0x7fab7415e980 con 0x7fab74070640 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.207+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab5c003520 con 0x7fab74070640 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2383591000 0 0) 0x7fab5c003520 con 0x7fab74070640 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab7415baa0 msgr2=0x7fab7415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab7415baa0 0x7fab7415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 >> v1:172.21.15.87:6789/0 conn(0x7fab7406d580 legacy=0x7fab7406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab741619c0 con 0x7fab74070640 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fab60001670 con 0x7fab74070640 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 >> v1:172.21.15.7:6789/0 conn(0x7fab74070640 legacy=0x7fab74161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5c0039e0 0x7fab5c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab5c004800 0x7fab5c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5c005150 0x7fab5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab7415ef40 con 0x7fab5c005150 2024-04-25T11:15:59.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c00bf00 con 0x7fab5c0039e0 2024-04-25T11:15:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c00c100 con 0x7fab5c004800 2024-04-25T11:15:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.208+0000 7fab79800640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5c005150 0x7fab5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.209+0000 7fab73e00640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab5c004800 0x7fab5c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.209+0000 7fab78e00640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5c0039e0 0x7fab5c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.209+0000 7fab79800640 1 -- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab5c004800 msgr2=0x7fab5c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.209+0000 7fab79800640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab5c004800 0x7fab5c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.209+0000 7fab79800640 1 -- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5c0039e0 msgr2=0x7fab5c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.209+0000 7fab79800640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5c0039e0 0x7fab5c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.209+0000 7fab79800640 1 -- 172.21.15.7:0/1471317735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c003520 con 0x7fab5c005150 2024-04-25T11:15:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.209+0000 7fab73e00640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab5c004800 0x7fab5c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:59.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.209+0000 7fab78e00640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5c0039e0 0x7fab5c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:15:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.210+0000 7fab79800640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5c005150 0x7fab5c00b9c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fab74147bb0 tx=0x7fab5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ea5a345fe7ae5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.210+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c001a70 con 0x7fab5c005150 2024-04-25T11:15:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.210+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c02ae10 con 0x7fab5c005150 2024-04-25T11:15:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.210+0000 7fab73400640 1 -- 172.21.15.7:0/1471317735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c003bb0 con 0x7fab5c005150 2024-04-25T11:15:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.211+0000 7fab7a200640 1 -- 172.21.15.7:0/1471317735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5c005150 msgr2=0x7fab5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.211+0000 7fab7a200640 1 --2- 172.21.15.7:0/1471317735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5c005150 0x7fab5c00b9c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fab74147bb0 tx=0x7fab5c005830 comp rx=0 tx=0).stop 2024-04-25T11:15:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.211+0000 7fab79800640 1 -- 172.21.15.7:0/1471317735 reap_dead start 2024-04-25T11:15:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.211+0000 7fab7a200640 1 -- 172.21.15.7:0/1471317735 shutdown_connections 2024-04-25T11:15:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.211+0000 7fab7a200640 1 -- 172.21.15.7:0/1471317735 >> 172.21.15.7:0/1471317735 conn(0x7fab74073090 msgr2=0x7fab740734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:59.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.211+0000 7fab7a200640 1 -- 172.21.15.7:0/1471317735 shutdown_connections 2024-04-25T11:15:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.211+0000 7fab7a200640 1 -- 172.21.15.7:0/1471317735 wait complete. 2024-04-25T11:15:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.214+0000 7fab7a200640 1 Processor -- start 2024-04-25T11:15:59.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.214+0000 7fab7a200640 1 -- start start 2024-04-25T11:15:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.215+0000 7fab7a200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5c005150 0x7fab74141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.215+0000 7fab7a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab740b4480 0x7fab74145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.215+0000 7fab7a200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab740b5820 0x7fab74146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.215+0000 7fab7a200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab74162770 con 0x7fab740b4480 2024-04-25T11:15:59.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.215+0000 7fab7a200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fab741625f0 con 0x7fab5c005150 2024-04-25T11:15:59.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.215+0000 7fab7a200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fab7415baa0 con 0x7fab740b5820 2024-04-25T11:15:59.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.216+0000 7fab73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab740b4480 0x7fab74145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.216+0000 7fab73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab740b4480 0x7fab74145e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41766/0 (socket says 172.21.15.7:41766) 2024-04-25T11:15:59.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.216+0000 7fab73e00640 1 -- 172.21.15.7:0/4019792361 learned_addr learned my addr 172.21.15.7:0/4019792361 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:59.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.216+0000 7fab79800640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab740b5820 0x7fab74146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.216+0000 7fab73e00640 1 -- 172.21.15.7:0/4019792361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab740b5820 msgr2=0x7fab74146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.216+0000 7fab73e00640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab740b5820 0x7fab74146390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.216+0000 7fab73e00640 1 -- 172.21.15.7:0/4019792361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5c005150 msgr2=0x7fab74141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.216+0000 7fab73e00640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5c005150 0x7fab74141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.216+0000 7fab73e00640 1 -- 172.21.15.7:0/4019792361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab74142f30 con 0x7fab740b4480 2024-04-25T11:15:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.216+0000 7fab73e00640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab740b4480 0x7fab74145e50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fab6000d800 tx=0x7fab6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.217+0000 7fab70c00640 1 -- 172.21.15.7:0/4019792361 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60019070 con 0x7fab740b4480 2024-04-25T11:15:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.217+0000 7fab70c00640 1 -- 172.21.15.7:0/4019792361 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab600074e0 con 0x7fab740b4480 2024-04-25T11:15:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.217+0000 7fab7a200640 1 -- 172.21.15.7:0/4019792361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab74143210 con 0x7fab740b4480 2024-04-25T11:15:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.217+0000 7fab70c00640 1 -- 172.21.15.7:0/4019792361 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab60004a00 con 0x7fab740b4480 2024-04-25T11:15:59.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.217+0000 7fab7a200640 1 -- 172.21.15.7:0/4019792361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7413f7f0 con 0x7fab740b4480 2024-04-25T11:15:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.218+0000 7fab7a200640 1 -- 172.21.15.7:0/4019792361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab740b4b80 con 0x7fab740b4480 2024-04-25T11:15:59.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.219+0000 7fab70c00640 1 -- 172.21.15.7:0/4019792361 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fab6001c070 con 0x7fab740b4480 2024-04-25T11:15:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.220+0000 7fab70c00640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab540744b0 0x7fab540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.220+0000 7fab70c00640 1 -- 172.21.15.7:0/4019792361 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5249+0+0 (secure 0 0 0) 0x7fab60094b90 con 0x7fab740b4480 2024-04-25T11:15:59.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.220+0000 7fab78e00640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab540744b0 0x7fab540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.221+0000 7fab78e00640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab540744b0 0x7fab540769c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fab680099e0 tx=0x7fab68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:59.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.225+0000 7fab70c00640 1 -- 172.21.15.7:0/4019792361 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab6005f4d0 con 0x7fab740b4480 2024-04-25T11:15:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.428+0000 7fab7a200640 1 -- 172.21.15.7:0/4019792361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fab74050740 con 0x7fab740b4480 2024-04-25T11:15:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.429+0000 7fab70c00640 1 -- 172.21.15.7:0/4019792361 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) ==== 84+0+65 (secure 0 0 0) 0x7fab6005edd0 con 0x7fab740b4480 2024-04-25T11:15:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:15:59.431 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-25T11:15:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:15:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-25T11:15:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-25T11:15:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:15:59.431 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-25T11:15:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 -- 172.21.15.7:0/4019792361 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab540744b0 msgr2=0x7fab540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab540744b0 0x7fab540769c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fab680099e0 tx=0x7fab68008040 comp rx=0 tx=0).stop 2024-04-25T11:15:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 -- 172.21.15.7:0/4019792361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab740b4480 msgr2=0x7fab74145e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab740b4480 0x7fab74145e50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fab6000d800 tx=0x7fab6000dcc0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 -- 172.21.15.7:0/4019792361 shutdown_connections 2024-04-25T11:15:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab540744b0 0x7fab540769c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab740b5820 0x7fab74146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab740b4480 0x7fab74145e50 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 --2- 172.21.15.7:0/4019792361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab5c005150 0x7fab74141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 -- 172.21.15.7:0/4019792361 >> 172.21.15.7:0/4019792361 conn(0x7fab74073090 msgr2=0x7fab74143a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 -- 172.21.15.7:0/4019792361 shutdown_connections 2024-04-25T11:15:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.432+0000 7fab7a200640 1 -- 172.21.15.7:0/4019792361 wait complete. 2024-04-25T11:15:59.462 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:15:59.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:15:59.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.747+0000 7f3deb200640 1 Processor -- start 2024-04-25T11:15:59.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.747+0000 7f3deb200640 1 -- start start 2024-04-25T11:15:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.749+0000 7f3deb200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3dec1440a0 0x7f3dec15f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.749+0000 7f3deb200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dec15e620 con 0x7f3dec0cf370 2024-04-25T11:15:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.749+0000 7f3deb200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dec15e900 con 0x7f3dec06dc90 2024-04-25T11:15:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.749+0000 7f3deb200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3dec15ebe0 con 0x7f3dec1440a0 2024-04-25T11:15:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.749+0000 7f3de9400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3dec0cf370 0x7f3dec06d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53700/0 (socket says 172.21.15.7:53700) 2024-04-25T11:15:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.749+0000 7f3de9400640 1 -- 172.21.15.7:0/1047852097 learned_addr learned my addr 172.21.15.7:0/1047852097 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:15:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.750+0000 7f3de9e00640 1 --2- 172.21.15.7:0/1047852097 >> v2:172.21.15.87:3300/0 conn(0x7f3dec1440a0 0x7f3dec15f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.750+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2949217305 0 0) 0x7f3dec15e620 con 0x7f3dec0cf370 2024-04-25T11:15:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.750+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dd0003520 con 0x7f3dec0cf370 2024-04-25T11:15:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.750+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3131483268 0 0) 0x7f3dd0003520 con 0x7f3dec0cf370 2024-04-25T11:15:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.750+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 >> v2:172.21.15.87:3300/0 conn(0x7f3dec1440a0 msgr2=0x7f3dec15f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.750+0000 7f3de8a00640 1 --2- 172.21.15.7:0/1047852097 >> v2:172.21.15.87:3300/0 conn(0x7f3dec1440a0 0x7f3dec15f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.750+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 >> v1:172.21.15.87:6789/0 conn(0x7f3dec06dc90 legacy=0x7f3dec06f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec15eec0 con 0x7f3dec0cf370 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f3dd8001670 con 0x7f3dec0cf370 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 >> v1:172.21.15.7:6789/0 conn(0x7f3dec0cf370 legacy=0x7f3dec06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3de8a00640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd0003a00 0x7f3dd0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3de8a00640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dd0004800 0x7f3dd0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3de8a00640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dd00052a0 0x7f3dd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec15ebe0 con 0x7f3dd00052a0 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd000c230 con 0x7f3dd0004800 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd000c430 con 0x7f3dd0003a00 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3dea800640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dd00052a0 0x7f3dd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3de9e00640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd0003a00 0x7f3dd0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.751+0000 7f3de9400640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dd0004800 0x7f3dd0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.752+0000 7f3dea800640 1 -- 172.21.15.7:0/1047852097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd0003a00 msgr2=0x7f3dd0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.752+0000 7f3dea800640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd0003a00 0x7f3dd0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.752+0000 7f3dea800640 1 -- 172.21.15.7:0/1047852097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dd0004800 msgr2=0x7f3dd0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:15:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.752+0000 7f3dea800640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dd0004800 0x7f3dd0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.752+0000 7f3dea800640 1 -- 172.21.15.7:0/1047852097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd0003520 con 0x7f3dd00052a0 2024-04-25T11:15:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.752+0000 7f3de9400640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dd0004800 0x7f3dd0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:15:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.753+0000 7f3dea800640 1 -- 172.21.15.7:0/1047852097 reap_dead start 2024-04-25T11:15:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.753+0000 7f3dea800640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dd00052a0 0x7f3dd000b9c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3de4001e50 tx=0x7f3dd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f86373c4b125c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.753+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4001a70 con 0x7f3dd00052a0 2024-04-25T11:15:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.753+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3de4003970 con 0x7f3dd00052a0 2024-04-25T11:15:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.753+0000 7f3de8a00640 1 -- 172.21.15.7:0/1047852097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4003d60 con 0x7f3dd00052a0 2024-04-25T11:15:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.754+0000 7f3deb200640 1 -- 172.21.15.7:0/1047852097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dd00052a0 msgr2=0x7f3dd000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.754+0000 7f3deb200640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dd00052a0 0x7f3dd000b9c0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f3de4001e50 tx=0x7f3dd0005830 comp rx=0 tx=0).stop 2024-04-25T11:15:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.754+0000 7f3deb200640 1 -- 172.21.15.7:0/1047852097 shutdown_connections 2024-04-25T11:15:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.754+0000 7f3deb200640 1 --2- 172.21.15.7:0/1047852097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dd00052a0 0x7f3dd000b9c0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.754+0000 7f3deb200640 1 -- 172.21.15.7:0/1047852097 >> 172.21.15.7:0/1047852097 conn(0x7f3dec073160 msgr2=0x7f3dec073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:15:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.754+0000 7f3deb200640 1 -- 172.21.15.7:0/1047852097 shutdown_connections 2024-04-25T11:15:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.754+0000 7f3deb200640 1 -- 172.21.15.7:0/1047852097 wait complete. 2024-04-25T11:15:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.756+0000 7f3deb200640 1 Processor -- start 2024-04-25T11:15:59.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.757+0000 7f3deb200640 1 -- start start 2024-04-25T11:15:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.758+0000 7f3deb200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd00052a0 0x7f3dec15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.758+0000 7f3deb200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dec1f5220 0x7f3dec1f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.758+0000 7f3deb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec0b52c0 0x7f3dec0b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.758+0000 7f3deb200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec15e380 con 0x7f3dec0b52c0 2024-04-25T11:15:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.758+0000 7f3deb200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec15e200 con 0x7f3dec1f5220 2024-04-25T11:15:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.758+0000 7f3deb200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3dec0cf780 con 0x7f3dd00052a0 2024-04-25T11:15:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.758+0000 7f3dea800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec0b52c0 0x7f3dec0b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.758+0000 7f3dea800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec0b52c0 0x7f3dec0b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41792/0 (socket says 172.21.15.7:41792) 2024-04-25T11:15:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.758+0000 7f3dea800640 1 -- 172.21.15.7:0/3839597047 learned_addr learned my addr 172.21.15.7:0/3839597047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:15:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.759+0000 7f3de9400640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dec1f5220 0x7f3dec1f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.759+0000 7f3de9e00640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd00052a0 0x7f3dec15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.759+0000 7f3dea800640 1 -- 172.21.15.7:0/3839597047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd00052a0 msgr2=0x7f3dec15b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.759+0000 7f3dea800640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd00052a0 0x7f3dec15b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.759+0000 7f3dea800640 1 -- 172.21.15.7:0/3839597047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dec1f5220 msgr2=0x7f3dec1f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:15:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.759+0000 7f3dea800640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dec1f5220 0x7f3dec1f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:15:59.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.759+0000 7f3dea800640 1 -- 172.21.15.7:0/3839597047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec0b5c10 con 0x7f3dec0b52c0 2024-04-25T11:15:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.759+0000 7f3dea800640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec0b52c0 0x7f3dec0b56d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3de402a920 tx=0x7f3de40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.760+0000 7f3de2000640 1 -- 172.21.15.7:0/3839597047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4001a70 con 0x7f3dec0b52c0 2024-04-25T11:15:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.760+0000 7f3de2000640 1 -- 172.21.15.7:0/3839597047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3de402ac70 con 0x7f3dec0b52c0 2024-04-25T11:15:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.760+0000 7f3de2000640 1 -- 172.21.15.7:0/3839597047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4006d70 con 0x7f3dec0b52c0 2024-04-25T11:15:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.760+0000 7f3deb200640 1 -- 172.21.15.7:0/3839597047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dec15a150 con 0x7f3dec0b52c0 2024-04-25T11:15:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.760+0000 7f3deb200640 1 -- 172.21.15.7:0/3839597047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dec15a710 con 0x7f3dec0b52c0 2024-04-25T11:15:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.761+0000 7f3deb200640 1 -- 172.21.15.7:0/3839597047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dec050740 con 0x7f3dec0b52c0 2024-04-25T11:15:59.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.762+0000 7f3de2000640 1 -- 172.21.15.7:0/3839597047 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3de4007030 con 0x7f3dec0b52c0 2024-04-25T11:15:59.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.764+0000 7f3de2000640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dcc0744e0 0x7f3dcc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:15:59.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.764+0000 7f3de2000640 1 -- 172.21.15.7:0/3839597047 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5249+0+0 (secure 0 0 0) 0x7f3de40b4330 con 0x7f3dec0b52c0 2024-04-25T11:15:59.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.764+0000 7f3de9e00640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dcc0744e0 0x7f3dcc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:15:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.765+0000 7f3de9e00640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dcc0744e0 0x7f3dcc0769f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0711f0 tx=0x7f3ddc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:15:59.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.769+0000 7f3de2000640 1 -- 172.21.15.7:0/3839597047 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3dec0b5c10 con 0x7f3dec0b52c0 2024-04-25T11:15:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:15:59.977+0000 7f3deb200640 1 -- 172.21.15.7:0/3839597047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3dec159ee0 con 0x7f3dec0b52c0 2024-04-25T11:16:00.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.216+0000 7f3de2000640 1 -- 172.21.15.7:0/3839597047 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) ==== 138+0+0 (secure 0 0 0) 0x7f3de407ebe0 con 0x7f3dec0b52c0 2024-04-25T11:16:00.217 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-25T11:16:00.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.220+0000 7f3deb200640 1 -- 172.21.15.7:0/3839597047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dcc0744e0 msgr2=0x7f3dcc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:00.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.220+0000 7f3deb200640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dcc0744e0 0x7f3dcc0769f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0711f0 tx=0x7f3ddc008040 comp rx=0 tx=0).stop 2024-04-25T11:16:00.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.220+0000 7f3deb200640 1 -- 172.21.15.7:0/3839597047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec0b52c0 msgr2=0x7f3dec0b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:00.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.220+0000 7f3deb200640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec0b52c0 0x7f3dec0b56d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f3de402a920 tx=0x7f3de40014a0 comp rx=0 tx=0).stop 2024-04-25T11:16:00.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.220+0000 7f3deb200640 1 -- 172.21.15.7:0/3839597047 shutdown_connections 2024-04-25T11:16:00.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.220+0000 7f3deb200640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3dcc0744e0 0x7f3dcc0769f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:00.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.220+0000 7f3deb200640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dec0b52c0 0x7f3dec0b56d0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:00.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.220+0000 7f3deb200640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3dec1f5220 0x7f3dec1f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:00.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.220+0000 7f3deb200640 1 --2- 172.21.15.7:0/3839597047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3dd00052a0 0x7f3dec15b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:00.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.220+0000 7f3deb200640 1 -- 172.21.15.7:0/3839597047 >> 172.21.15.7:0/3839597047 conn(0x7f3dec073160 msgr2=0x7f3dec143620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:00.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.221+0000 7f3deb200640 1 -- 172.21.15.7:0/3839597047 shutdown_connections 2024-04-25T11:16:00.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.221+0000 7f3deb200640 1 -- 172.21.15.7:0/3839597047 wait complete. 2024-04-25T11:16:00.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:16:00.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.544+0000 7fedbe000640 1 Processor -- start 2024-04-25T11:16:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.544+0000 7fedbe000640 1 -- start start 2024-04-25T11:16:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.547+0000 7fedbe000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fedb8070d60 0x7fedb815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.547+0000 7fedbe000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedb806f470 con 0x7fedb806dc90 2024-04-25T11:16:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.547+0000 7fedbe000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedb806f750 con 0x7fedb809f790 2024-04-25T11:16:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.547+0000 7fedbe000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fedb815e2a0 con 0x7fedb8070d60 2024-04-25T11:16:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.547+0000 7fedaea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fedb806dc90 0x7fedb80ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59480/0 (socket says 172.21.15.7:59480) 2024-04-25T11:16:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.547+0000 7fedaea00640 1 -- 172.21.15.7:0/2346025154 learned_addr learned my addr 172.21.15.7:0/2346025154 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.548+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472180980 0 0) 0x7fedb806f470 con 0x7fedb806dc90 2024-04-25T11:16:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.548+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed9c003520 con 0x7fedb806dc90 2024-04-25T11:16:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.548+0000 7fedb7e00640 1 --2- 172.21.15.7:0/2346025154 >> v2:172.21.15.87:3300/0 conn(0x7fedb8070d60 0x7fedb815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.548+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 195090423 0 0) 0x7fed9c003520 con 0x7fedb806dc90 2024-04-25T11:16:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.548+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 >> v2:172.21.15.87:3300/0 conn(0x7fedb8070d60 msgr2=0x7fedb815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.548+0000 7fedb6a00640 1 --2- 172.21.15.7:0/2346025154 >> v2:172.21.15.87:3300/0 conn(0x7fedb8070d60 0x7fedb815dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.548+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 >> v1:172.21.15.87:6789/0 conn(0x7fedb809f790 legacy=0x7fedb806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.548+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb806fa30 con 0x7fedb806dc90 2024-04-25T11:16:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.548+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7feda0001670 con 0x7fedb806dc90 2024-04-25T11:16:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.549+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 >> v1:172.21.15.7:6789/0 conn(0x7fedb806dc90 legacy=0x7fedb80ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.549+0000 7fedb6a00640 1 --2- 172.21.15.7:0/2346025154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed9c0039b0 0x7fed9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.549+0000 7fedb6a00640 1 --2- 172.21.15.7:0/2346025154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c004800 0x7fed9c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.549+0000 7fedb6a00640 1 --2- 172.21.15.7:0/2346025154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed9c005350 0x7fed9c00ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.549+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c00c2b0 con 0x7fed9c004800 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.549+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c00c460 con 0x7fed9c005350 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.549+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fed9c00c660 con 0x7fed9c0039b0 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.549+0000 7fedb7e00640 1 --2- 172.21.15.7:0/2346025154 >> v2:172.21.15.87:3300/0 conn(0x7fedb8070d60 0x7fedb815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.549+0000 7fedaea00640 1 --2- 172.21.15.7:0/2346025154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c004800 0x7fed9c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.549+0000 7fedb7400640 1 --2- 172.21.15.7:0/2346025154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed9c0039b0 0x7fed9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.549+0000 7fedb7e00640 1 --2- 172.21.15.7:0/2346025154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed9c005350 0x7fed9c00ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.550+0000 7fedaea00640 1 -- 172.21.15.7:0/2346025154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed9c0039b0 msgr2=0x7fed9c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.550+0000 7fedaea00640 1 --2- 172.21.15.7:0/2346025154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed9c0039b0 0x7fed9c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.550+0000 7fedaea00640 1 -- 172.21.15.7:0/2346025154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed9c005350 msgr2=0x7fed9c00ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.550+0000 7fedaea00640 1 --2- 172.21.15.7:0/2346025154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed9c005350 0x7fed9c00ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.550+0000 7fedaea00640 1 -- 172.21.15.7:0/2346025154 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c003520 con 0x7fed9c004800 2024-04-25T11:16:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.550+0000 7fedb7e00640 1 -- 172.21.15.7:0/2346025154 reap_dead start 2024-04-25T11:16:00.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.551+0000 7fedaea00640 1 --2- 172.21.15.7:0/2346025154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c004800 0x7fed9c004c10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7feda000c240 tx=0x7fed9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f39329b4b810be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:00.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.551+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0007cf0 con 0x7fed9c004800 2024-04-25T11:16:00.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.551+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda00058d0 con 0x7fed9c004800 2024-04-25T11:16:00.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.551+0000 7fedb6a00640 1 -- 172.21.15.7:0/2346025154 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0005d20 con 0x7fed9c004800 2024-04-25T11:16:00.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.552+0000 7fedbe000640 1 -- 172.21.15.7:0/2346025154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c004800 msgr2=0x7fed9c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:00.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.552+0000 7fedbe000640 1 --2- 172.21.15.7:0/2346025154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c004800 0x7fed9c004c10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7feda000c240 tx=0x7fed9c005830 comp rx=0 tx=0).stop 2024-04-25T11:16:00.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.552+0000 7fedbe000640 1 -- 172.21.15.7:0/2346025154 shutdown_connections 2024-04-25T11:16:00.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.552+0000 7fedbe000640 1 --2- 172.21.15.7:0/2346025154 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed9c004800 0x7fed9c004c10 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:00.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.552+0000 7fedbe000640 1 -- 172.21.15.7:0/2346025154 >> 172.21.15.7:0/2346025154 conn(0x7fedb80d1d40 msgr2=0x7fedb8073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:00.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.552+0000 7fedbe000640 1 -- 172.21.15.7:0/2346025154 shutdown_connections 2024-04-25T11:16:00.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.552+0000 7fedbe000640 1 -- 172.21.15.7:0/2346025154 wait complete. 2024-04-25T11:16:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.555+0000 7fedbe000640 1 Processor -- start 2024-04-25T11:16:00.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.555+0000 7fedbe000640 1 -- start start 2024-04-25T11:16:00.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.556+0000 7fedbe000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed9c004800 0x7fedb8160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.556+0000 7fedbe000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb809ec40 0x7fedb809c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.556+0000 7fedbe000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb809ce80 0x7fedb809d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.556+0000 7fedbe000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fedb806f220 con 0x7fedb809ce80 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.556+0000 7fedbe000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fedb8074550 con 0x7fedb809ec40 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.556+0000 7fedbe000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fedb8070080 con 0x7fed9c004800 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.557+0000 7fedb7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb809ce80 0x7fedb809d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.557+0000 7fedb7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb809ce80 0x7fedb809d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60598/0 (socket says 172.21.15.7:60598) 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.557+0000 7fedb7e00640 1 -- 172.21.15.7:0/2311949091 learned_addr learned my addr 172.21.15.7:0/2311949091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.557+0000 7fedaea00640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb809ec40 0x7fedb809c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.557+0000 7fedb7400640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed9c004800 0x7fedb8160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.557+0000 7fedb7e00640 1 -- 172.21.15.7:0/2311949091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed9c004800 msgr2=0x7fedb8160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.557+0000 7fedb7e00640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed9c004800 0x7fedb8160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.557+0000 7fedb7e00640 1 -- 172.21.15.7:0/2311949091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb809ec40 msgr2=0x7fedb809c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.557+0000 7fedb7e00640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb809ec40 0x7fedb809c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:00.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.557+0000 7fedb7e00640 1 -- 172.21.15.7:0/2311949091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb809b6e0 con 0x7fedb809ce80 2024-04-25T11:16:00.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.557+0000 7fedb7400640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed9c004800 0x7fedb8160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:00.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.558+0000 7fedb7e00640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb809ce80 0x7fedb809d290 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fedb00070b0 tx=0x7fedb000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:00.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.558+0000 7fedafe00640 1 -- 172.21.15.7:0/2311949091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0003da0 con 0x7fedb809ce80 2024-04-25T11:16:00.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.558+0000 7fedafe00640 1 -- 172.21.15.7:0/2311949091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb000ce90 con 0x7fedb809ce80 2024-04-25T11:16:00.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.558+0000 7fedafe00640 1 -- 172.21.15.7:0/2311949091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0004270 con 0x7fedb809ce80 2024-04-25T11:16:00.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.558+0000 7fedbe000640 1 -- 172.21.15.7:0/2311949091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb809b9c0 con 0x7fedb809ce80 2024-04-25T11:16:00.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.558+0000 7fedbe000640 1 -- 172.21.15.7:0/2311949091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb815e900 con 0x7fedb809ce80 2024-04-25T11:16:00.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.560+0000 7fedbe000640 1 -- 172.21.15.7:0/2311949091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb8050740 con 0x7fedb809ce80 2024-04-25T11:16:00.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.561+0000 7fedafe00640 1 -- 172.21.15.7:0/2311949091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fedb0005d10 con 0x7fedb809ce80 2024-04-25T11:16:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.562+0000 7fedafe00640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed940744b0 0x7fed940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:00.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.562+0000 7fedafe00640 1 -- 172.21.15.7:0/2311949091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5619+0+0 (secure 0 0 0) 0x7fedb0095d20 con 0x7fedb809ce80 2024-04-25T11:16:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.563+0000 7fedb7400640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed940744b0 0x7fed940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:00.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.564+0000 7fedb7400640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed940744b0 0x7fed940769c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fedb806aef0 tx=0x7feda8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:00.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.567+0000 7fedafe00640 1 -- 172.21.15.7:0/2311949091 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedb0020070 con 0x7fedb809ce80 2024-04-25T11:16:00.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:00.772+0000 7fedbe000640 1 -- 172.21.15.7:0/2311949091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fedb8073c20 con 0x7fedb809ce80 2024-04-25T11:16:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.237+0000 7fedafe00640 1 -- 172.21.15.7:0/2311949091 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) ==== 170+0+0 (secure 0 0 0) 0x7fedb0060670 con 0x7fedb809ce80 2024-04-25T11:16:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-25T11:16:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.243+0000 7fedbe000640 1 -- 172.21.15.7:0/2311949091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed940744b0 msgr2=0x7fed940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.243+0000 7fedbe000640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed940744b0 0x7fed940769c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fedb806aef0 tx=0x7feda8009040 comp rx=0 tx=0).stop 2024-04-25T11:16:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.243+0000 7fedbe000640 1 -- 172.21.15.7:0/2311949091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb809ce80 msgr2=0x7fedb809d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.244+0000 7fedbe000640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb809ce80 0x7fedb809d290 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fedb00070b0 tx=0x7fedb000c7b0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.246+0000 7fedbe000640 1 -- 172.21.15.7:0/2311949091 shutdown_connections 2024-04-25T11:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.246+0000 7fedbe000640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed940744b0 0x7fed940769c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.246+0000 7fedbe000640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb809ce80 0x7fedb809d290 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.246+0000 7fedbe000640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb809ec40 0x7fedb809c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.246+0000 7fedbe000640 1 --2- 172.21.15.7:0/2311949091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed9c004800 0x7fedb8160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.246+0000 7fedbe000640 1 -- 172.21.15.7:0/2311949091 >> 172.21.15.7:0/2311949091 conn(0x7fedb80d1d40 msgr2=0x7fedb80d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.250+0000 7fedbe000640 1 -- 172.21.15.7:0/2311949091 shutdown_connections 2024-04-25T11:16:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.250+0000 7fedbe000640 1 -- 172.21.15.7:0/2311949091 wait complete. 2024-04-25T11:16:01.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:16:01.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.549+0000 7efdee000640 1 Processor -- start 2024-04-25T11:16:01.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.549+0000 7efdee000640 1 -- start start 2024-04-25T11:16:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.552+0000 7efdee000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efde815dfb0 0x7efde80b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.552+0000 7efdee000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d0b40 0x7efde806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.552+0000 7efdee000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efde80d0f50 0x7efde806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.552+0000 7efdee000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efde80b09b0 con 0x7efde815dfb0 2024-04-25T11:16:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.552+0000 7efdee000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efde815e3c0 con 0x7efde80d0f50 2024-04-25T11:16:01.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.552+0000 7efdee000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efde80cf800 con 0x7efde80d0b40 2024-04-25T11:16:01.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.552+0000 7efde7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efde815dfb0 0x7efde80b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:01.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.552+0000 7efde7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efde815dfb0 0x7efde80b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60612/0 (socket says 172.21.15.7:60612) 2024-04-25T11:16:01.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.552+0000 7efde7400640 1 -- 172.21.15.7:0/3202464086 learned_addr learned my addr 172.21.15.7:0/3202464086 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.552+0000 7efddea00640 1 --2- 172.21.15.7:0/3202464086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d0b40 0x7efde806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.553+0000 7efde7e00640 1 --2- 172.21.15.7:0/3202464086 >> v2:172.21.15.87:3300/0 conn(0x7efde80d0f50 0x7efde806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.553+0000 7efde7400640 1 -- 172.21.15.7:0/3202464086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d0b40 msgr2=0x7efde806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.553+0000 7efde7400640 1 --2- 172.21.15.7:0/3202464086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d0b40 0x7efde806f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.553+0000 7efde7400640 1 -- 172.21.15.7:0/3202464086 >> v2:172.21.15.87:3300/0 conn(0x7efde80d0f50 msgr2=0x7efde806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.553+0000 7efde7400640 1 --2- 172.21.15.7:0/3202464086 >> v2:172.21.15.87:3300/0 conn(0x7efde80d0f50 0x7efde806f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.553+0000 7efde7400640 1 -- 172.21.15.7:0/3202464086 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde80cfaa0 con 0x7efde815dfb0 2024-04-25T11:16:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.554+0000 7efde7400640 1 --2- 172.21.15.7:0/3202464086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde815dfb0 0x7efde80b0470 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7efde8071810 tx=0x7efdd802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355ce99478633c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.554+0000 7efde6a00640 1 -- 172.21.15.7:0/3202464086 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8037070 con 0x7efde815dfb0 2024-04-25T11:16:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.554+0000 7efde6a00640 1 -- 172.21.15.7:0/3202464086 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd8032440 con 0x7efde815dfb0 2024-04-25T11:16:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.554+0000 7efde6a00640 1 -- 172.21.15.7:0/3202464086 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8032810 con 0x7efde815dfb0 2024-04-25T11:16:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.555+0000 7efdee000640 1 -- 172.21.15.7:0/3202464086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde815dfb0 msgr2=0x7efde80b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.555+0000 7efdee000640 1 --2- 172.21.15.7:0/3202464086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde815dfb0 0x7efde80b0470 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7efde8071810 tx=0x7efdd802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.555+0000 7efdee000640 1 -- 172.21.15.7:0/3202464086 shutdown_connections 2024-04-25T11:16:01.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.555+0000 7efdee000640 1 --2- 172.21.15.7:0/3202464086 >> v2:172.21.15.87:3300/0 conn(0x7efde80d0f50 0x7efde806f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.555+0000 7efdee000640 1 --2- 172.21.15.7:0/3202464086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d0b40 0x7efde806f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.555+0000 7efdee000640 1 --2- 172.21.15.7:0/3202464086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde815dfb0 0x7efde80b0470 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.555+0000 7efdee000640 1 -- 172.21.15.7:0/3202464086 >> 172.21.15.7:0/3202464086 conn(0x7efde80742c0 msgr2=0x7efde806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.556+0000 7efdee000640 1 -- 172.21.15.7:0/3202464086 shutdown_connections 2024-04-25T11:16:01.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.556+0000 7efdee000640 1 -- 172.21.15.7:0/3202464086 wait complete. 2024-04-25T11:16:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.559+0000 7efdee000640 1 Processor -- start 2024-04-25T11:16:01.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.559+0000 7efdee000640 1 -- start start 2024-04-25T11:16:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.561+0000 7efdee000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde80d0b40 0x7efde81fe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.561+0000 7efdee000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d0f50 0x7efde81fe920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.561+0000 7efdee000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efde815dfb0 0x7efde81530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.561+0000 7efdee000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efde80d0340 con 0x7efde80d0b40 2024-04-25T11:16:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.561+0000 7efdee000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efde80d0060 con 0x7efde815dfb0 2024-04-25T11:16:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.561+0000 7efdee000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efde80d0620 con 0x7efde80d0f50 2024-04-25T11:16:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.562+0000 7efde7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde80d0b40 0x7efde81fe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.562+0000 7efde7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde80d0b40 0x7efde81fe3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60626/0 (socket says 172.21.15.7:60626) 2024-04-25T11:16:01.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.562+0000 7efde7400640 1 -- 172.21.15.7:0/2492961910 learned_addr learned my addr 172.21.15.7:0/2492961910 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:01.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.562+0000 7efde7400640 1 -- 172.21.15.7:0/2492961910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d0f50 msgr2=0x7efde81fe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:01.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.562+0000 7efddea00640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d0f50 0x7efde81fe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.562+0000 7efde7400640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d0f50 0x7efde81fe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.562+0000 7efde7400640 1 -- 172.21.15.7:0/2492961910 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efde815dfb0 msgr2=0x7efde81530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.562+0000 7efde7400640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efde815dfb0 0x7efde81530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.562+0000 7efde7400640 1 -- 172.21.15.7:0/2492961910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde81e9410 con 0x7efde80d0b40 2024-04-25T11:16:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.562+0000 7efddea00640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d0f50 0x7efde81fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.563+0000 7efde7400640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde80d0b40 0x7efde81fe3e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7efdd802a220 tx=0x7efdd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.563+0000 7efddfe00640 1 -- 172.21.15.7:0/2492961910 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8041020 con 0x7efde80d0b40 2024-04-25T11:16:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.563+0000 7efddfe00640 1 -- 172.21.15.7:0/2492961910 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd80306c0 con 0x7efde80d0b40 2024-04-25T11:16:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.563+0000 7efddfe00640 1 -- 172.21.15.7:0/2492961910 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8037040 con 0x7efde80d0b40 2024-04-25T11:16:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.563+0000 7efdee000640 1 -- 172.21.15.7:0/2492961910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde81e96f0 con 0x7efde80d0b40 2024-04-25T11:16:01.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.563+0000 7efdee000640 1 -- 172.21.15.7:0/2492961910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde81e9cb0 con 0x7efde80d0b40 2024-04-25T11:16:01.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.564+0000 7efdee000640 1 -- 172.21.15.7:0/2492961910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde8050740 con 0x7efde80d0b40 2024-04-25T11:16:01.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.566+0000 7efddfe00640 1 -- 172.21.15.7:0/2492961910 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efdd8044020 con 0x7efde80d0b40 2024-04-25T11:16:01.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.567+0000 7efddfe00640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 0x7efdc40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:01.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.567+0000 7efddfe00640 1 -- 172.21.15.7:0/2492961910 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6001+0+0 (secure 0 0 0) 0x7efdd80b5c10 con 0x7efde80d0b40 2024-04-25T11:16:01.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.567+0000 7efddea00640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 0x7efdc40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:01.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.568+0000 7efddea00640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 0x7efdc40769c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efdd0006fd0 tx=0x7efdd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:01.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.573+0000 7efddfe00640 1 -- 172.21.15.7:0/2492961910 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efdd804b030 con 0x7efde80d0b40 2024-04-25T11:16:01.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:01.817+0000 7efdee000640 1 -- 172.21.15.7:0/2492961910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efde8068cf0 con 0x7efde80d0b40 2024-04-25T11:16:02.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.241+0000 7efddfe00640 1 -- 172.21.15.7:0/2492961910 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v48) ==== 174+0+0 (secure 0 0 0) 0x7efdd8080260 con 0x7efde80d0b40 2024-04-25T11:16:02.243 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T11:16:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.246+0000 7efdee000640 1 -- 172.21.15.7:0/2492961910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 msgr2=0x7efdc40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.246+0000 7efdee000640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 0x7efdc40769c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efdd0006fd0 tx=0x7efdd0008040 comp rx=0 tx=0).stop 2024-04-25T11:16:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.246+0000 7efdee000640 1 -- 172.21.15.7:0/2492961910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde80d0b40 msgr2=0x7efde81fe3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:02.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.246+0000 7efdee000640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde80d0b40 0x7efde81fe3e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7efdd802a220 tx=0x7efdd80046c0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.247+0000 7efdee000640 1 -- 172.21.15.7:0/2492961910 shutdown_connections 2024-04-25T11:16:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.247+0000 7efdee000640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdc40744b0 0x7efdc40769c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.247+0000 7efdee000640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efde815dfb0 0x7efde81530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.247+0000 7efdee000640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efde80d0f50 0x7efde81fe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.247+0000 7efdee000640 1 --2- 172.21.15.7:0/2492961910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efde80d0b40 0x7efde81fe3e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.247+0000 7efdee000640 1 -- 172.21.15.7:0/2492961910 >> 172.21.15.7:0/2492961910 conn(0x7efde80742c0 msgr2=0x7efde806fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:02.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.248+0000 7efdee000640 1 -- 172.21.15.7:0/2492961910 shutdown_connections 2024-04-25T11:16:02.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.249+0000 7efdee000640 1 -- 172.21.15.7:0/2492961910 wait complete. 2024-04-25T11:16:02.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:16:02.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.541+0000 7ff90ae00640 1 Processor -- start 2024-04-25T11:16:02.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.541+0000 7ff90ae00640 1 -- start start 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.543+0000 7ff90ae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9040aed40 0x7ff90415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.543+0000 7ff90ae00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9040b5220 0x7ff9040ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.544+0000 7ff90ae00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9040b64b0 con 0x7ff9040af420 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.544+0000 7ff90ae00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff90415e1f0 con 0x7ff9040b5220 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.544+0000 7ff90ae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff90406c610 con 0x7ff9040aed40 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.544+0000 7ff909000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9040b5220 0x7ff9040ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.544+0000 7ff909000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9040b5220 0x7ff9040ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60628/0 (socket says 172.21.15.7:60628) 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.544+0000 7ff909000640 1 -- 172.21.15.7:0/3186666750 learned_addr learned my addr 172.21.15.7:0/3186666750 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.544+0000 7ff909000640 1 -- 172.21.15.7:0/3186666750 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9040aed40 msgr2=0x7ff90415dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.544+0000 7ff909a00640 1 --2- 172.21.15.7:0/3186666750 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9040aed40 0x7ff90415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.544+0000 7ff909000640 1 --2- 172.21.15.7:0/3186666750 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9040aed40 0x7ff90415dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.544+0000 7ff909000640 1 -- 172.21.15.7:0/3186666750 >> v1:172.21.15.87:6789/0 conn(0x7ff9040af420 legacy=0x7ff9040af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.544+0000 7ff909000640 1 -- 172.21.15.7:0/3186666750 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90406c8f0 con 0x7ff9040b5220 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.545+0000 7ff909000640 1 --2- 172.21.15.7:0/3186666750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9040b5220 0x7ff9040ce4a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009870 tx=0x7ff8f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc2a069465aca6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.546+0000 7ff903e00640 1 -- 172.21.15.7:0/3186666750 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0037070 con 0x7ff9040b5220 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.546+0000 7ff903e00640 1 -- 172.21.15.7:0/3186666750 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f0032440 con 0x7ff9040b5220 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.546+0000 7ff903e00640 1 -- 172.21.15.7:0/3186666750 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0032810 con 0x7ff9040b5220 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.546+0000 7ff90ae00640 1 -- 172.21.15.7:0/3186666750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9040b5220 msgr2=0x7ff9040ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.546+0000 7ff90ae00640 1 --2- 172.21.15.7:0/3186666750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9040b5220 0x7ff9040ce4a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009870 tx=0x7ff8f002a210 comp rx=0 tx=0).stop 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.547+0000 7ff90ae00640 1 -- 172.21.15.7:0/3186666750 shutdown_connections 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.547+0000 7ff90ae00640 1 --2- 172.21.15.7:0/3186666750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9040b5220 0x7ff9040ce4a0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.547+0000 7ff90ae00640 1 --2- 172.21.15.7:0/3186666750 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9040aed40 0x7ff90415dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.547+0000 7ff90ae00640 1 -- 172.21.15.7:0/3186666750 >> 172.21.15.7:0/3186666750 conn(0x7ff9040b3820 msgr2=0x7ff9040b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.547+0000 7ff90ae00640 1 -- 172.21.15.7:0/3186666750 shutdown_connections 2024-04-25T11:16:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.547+0000 7ff90ae00640 1 -- 172.21.15.7:0/3186666750 wait complete. 2024-04-25T11:16:02.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.549+0000 7ff90ae00640 1 Processor -- start 2024-04-25T11:16:02.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.549+0000 7ff90ae00640 1 -- start start 2024-04-25T11:16:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.551+0000 7ff90ae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9040aed40 0x7ff904142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.551+0000 7ff90ae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9040af420 0x7ff9040d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.551+0000 7ff90ae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9040b5220 0x7ff90415bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.551+0000 7ff90ae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff9040b6110 con 0x7ff9040af420 2024-04-25T11:16:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.551+0000 7ff90ae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff90406ceb0 con 0x7ff9040aed40 2024-04-25T11:16:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.551+0000 7ff90ae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff9040b6290 con 0x7ff9040b5220 2024-04-25T11:16:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.551+0000 7ff909000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9040af420 0x7ff9040d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.551+0000 7ff909000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9040af420 0x7ff9040d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60630/0 (socket says 172.21.15.7:60630) 2024-04-25T11:16:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.551+0000 7ff909000640 1 -- 172.21.15.7:0/204297495 learned_addr learned my addr 172.21.15.7:0/204297495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff90a400640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9040b5220 0x7ff90415bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:02.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff909000640 1 -- 172.21.15.7:0/204297495 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9040b5220 msgr2=0x7ff90415bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff909000640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9040b5220 0x7ff90415bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff909000640 1 -- 172.21.15.7:0/204297495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9040aed40 msgr2=0x7ff904142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff909000640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9040aed40 0x7ff904142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff909000640 1 -- 172.21.15.7:0/204297495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90415fff0 con 0x7ff9040af420 2024-04-25T11:16:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff90a400640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9040b5220 0x7ff90415bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff909000640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9040af420 0x7ff9040d04d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00090b0 tx=0x7ff8f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff901600640 1 -- 172.21.15.7:0/204297495 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0037040 con 0x7ff9040af420 2024-04-25T11:16:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff901600640 1 -- 172.21.15.7:0/204297495 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f0042070 con 0x7ff9040af420 2024-04-25T11:16:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff901600640 1 -- 172.21.15.7:0/204297495 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0030bd0 con 0x7ff9040af420 2024-04-25T11:16:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.552+0000 7ff90ae00640 1 -- 172.21.15.7:0/204297495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9041602d0 con 0x7ff9040af420 2024-04-25T11:16:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.553+0000 7ff90ae00640 1 -- 172.21.15.7:0/204297495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff904160890 con 0x7ff9040af420 2024-04-25T11:16:02.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.554+0000 7ff90ae00640 1 -- 172.21.15.7:0/204297495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff904070f00 con 0x7ff9040af420 2024-04-25T11:16:02.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.555+0000 7ff901600640 1 -- 172.21.15.7:0/204297495 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff8f0047460 con 0x7ff9040af420 2024-04-25T11:16:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.556+0000 7ff901600640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8e40744e0 0x7ff8e40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.556+0000 7ff901600640 1 -- 172.21.15.7:0/204297495 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7ff8f00b60f0 con 0x7ff9040af420 2024-04-25T11:16:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.556+0000 7ff909a00640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8e40744e0 0x7ff8e40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:02.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.557+0000 7ff909a00640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8e40744e0 0x7ff8e40769f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff904071200 tx=0x7ff8f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:02.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.560+0000 7ff901600640 1 -- 172.21.15.7:0/204297495 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8f0080650 con 0x7ff9040af420 2024-04-25T11:16:02.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.761+0000 7ff90ae00640 1 -- 172.21.15.7:0/204297495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9040b3e80 con 0x7ff9040af420 2024-04-25T11:16:02.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.765+0000 7ff901600640 1 -- 172.21.15.7:0/204297495 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16802 (secure 0 0 0) 0x7ff8f007ff50 con 0x7ff9040af420 2024-04-25T11:16:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:02.767 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:16:01.820446+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:59.999922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T11:16:00.796013+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:16:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 -- 172.21.15.7:0/204297495 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8e40744e0 msgr2=0x7ff8e40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8e40744e0 0x7ff8e40769f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff904071200 tx=0x7ff8f8008040 comp rx=0 tx=0).stop 2024-04-25T11:16:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 -- 172.21.15.7:0/204297495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9040af420 msgr2=0x7ff9040d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9040af420 0x7ff9040d04d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00090b0 tx=0x7ff8f00046c0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 -- 172.21.15.7:0/204297495 shutdown_connections 2024-04-25T11:16:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8e40744e0 0x7ff8e40769f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9040b5220 0x7ff90415bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9040af420 0x7ff9040d04d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 --2- 172.21.15.7:0/204297495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9040aed40 0x7ff904142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 -- 172.21.15.7:0/204297495 >> 172.21.15.7:0/204297495 conn(0x7ff9040b3820 msgr2=0x7ff90406fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 -- 172.21.15.7:0/204297495 shutdown_connections 2024-04-25T11:16:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:02.768+0000 7ff90ae00640 1 -- 172.21.15.7:0/204297495 wait complete. 2024-04-25T11:16:02.796 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:16:03.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.071+0000 7f9331600640 1 Processor -- start 2024-04-25T11:16:03.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.071+0000 7f9331600640 1 -- start start 2024-04-25T11:16:03.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.073+0000 7f9331600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f932c0b0290 0x7f932c0b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.073+0000 7f9331600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f932c06f300 0x7f932c0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.073+0000 7f9331600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c0b4980 con 0x7f932c06dc90 2024-04-25T11:16:03.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.074+0000 7f9331600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f932c0b4c60 con 0x7f932c06f300 2024-04-25T11:16:03.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.074+0000 7f9331600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f932c0b4f40 con 0x7f932c0b0290 2024-04-25T11:16:03.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.074+0000 7f932b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f932c06f300 0x7f932c0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:03.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.074+0000 7f932aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f932c06dc90 0x7f932c0afb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59486/0 (socket says 172.21.15.7:59486) 2024-04-25T11:16:03.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.074+0000 7f932aa00640 1 -- 172.21.15.7:0/3238312012 learned_addr learned my addr 172.21.15.7:0/3238312012 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:03.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.074+0000 7f932a000640 1 --2- 172.21.15.7:0/3238312012 >> v2:172.21.15.87:3300/0 conn(0x7f932c0b0290 0x7f932c0b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:03.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.074+0000 7f932b400640 1 -- 172.21.15.7:0/3238312012 >> v2:172.21.15.87:3300/0 conn(0x7f932c0b0290 msgr2=0x7f932c0b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:03.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.074+0000 7f932b400640 1 --2- 172.21.15.7:0/3238312012 >> v2:172.21.15.87:3300/0 conn(0x7f932c0b0290 0x7f932c0b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.074+0000 7f932b400640 1 -- 172.21.15.7:0/3238312012 >> v1:172.21.15.7:6789/0 conn(0x7f932c06dc90 legacy=0x7f932c0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:03.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.075+0000 7f932b400640 1 -- 172.21.15.7:0/3238312012 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c0b5220 con 0x7f932c06f300 2024-04-25T11:16:03.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.074+0000 7f9329600640 1 -- 172.21.15.7:0/3238312012 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2643943476 0 0) 0x7f932c0b4980 con 0x7f932c06dc90 2024-04-25T11:16:03.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.076+0000 7f932b400640 1 --2- 172.21.15.7:0/3238312012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f932c06f300 0x7f932c0b5f60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f932c071e80 tx=0x7f932402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6346c111e7437ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:03.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.076+0000 7f9329600640 1 -- 172.21.15.7:0/3238312012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324007740 con 0x7f932c06f300 2024-04-25T11:16:03.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.076+0000 7f9329600640 1 -- 172.21.15.7:0/3238312012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93240078e0 con 0x7f932c06f300 2024-04-25T11:16:03.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.076+0000 7f9329600640 1 -- 172.21.15.7:0/3238312012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324007cb0 con 0x7f932c06f300 2024-04-25T11:16:03.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.077+0000 7f9331600640 1 -- 172.21.15.7:0/3238312012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f932c06f300 msgr2=0x7f932c0b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:03.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.077+0000 7f9331600640 1 --2- 172.21.15.7:0/3238312012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f932c06f300 0x7f932c0b5f60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f932c071e80 tx=0x7f932402a680 comp rx=0 tx=0).stop 2024-04-25T11:16:03.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.077+0000 7f9331600640 1 -- 172.21.15.7:0/3238312012 shutdown_connections 2024-04-25T11:16:03.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.077+0000 7f9331600640 1 --2- 172.21.15.7:0/3238312012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f932c06f300 0x7f932c0b5f60 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.077+0000 7f9331600640 1 --2- 172.21.15.7:0/3238312012 >> v2:172.21.15.87:3300/0 conn(0x7f932c0b0290 0x7f932c0b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.077+0000 7f9331600640 1 -- 172.21.15.7:0/3238312012 >> 172.21.15.7:0/3238312012 conn(0x7f932c074200 msgr2=0x7f932c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:03.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.077+0000 7f9331600640 1 -- 172.21.15.7:0/3238312012 shutdown_connections 2024-04-25T11:16:03.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.077+0000 7f9331600640 1 -- 172.21.15.7:0/3238312012 wait complete. 2024-04-25T11:16:03.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.079+0000 7f9331600640 1 Processor -- start 2024-04-25T11:16:03.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.079+0000 7f9331600640 1 -- start start 2024-04-25T11:16:03.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.081+0000 7f9331600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f932c06f300 0x7f932c15b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.081+0000 7f9331600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f932c1f4fd0 0x7f932c1f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.081+0000 7f9331600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f932c1f5920 0x7f932c1f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.081+0000 7f9331600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0ce920 con 0x7f932c1f4fd0 2024-04-25T11:16:03.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.081+0000 7f9331600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f932c15d220 con 0x7f932c1f5920 2024-04-25T11:16:03.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.081+0000 7f9331600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f932c0ceaa0 con 0x7f932c06f300 2024-04-25T11:16:03.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.081+0000 7f932a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f932c1f4fd0 0x7f932c1f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:03.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.081+0000 7f932a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f932c1f4fd0 0x7f932c1f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60650/0 (socket says 172.21.15.7:60650) 2024-04-25T11:16:03.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.081+0000 7f932a000640 1 -- 172.21.15.7:0/1945768473 learned_addr learned my addr 172.21.15.7:0/1945768473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f932aa00640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f932c06f300 0x7f932c15b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f932a000640 1 -- 172.21.15.7:0/1945768473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f932c06f300 msgr2=0x7f932c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f932a000640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f932c06f300 0x7f932c15b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f932a000640 1 -- 172.21.15.7:0/1945768473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f932c1f5920 msgr2=0x7f932c1f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f932a000640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f932c1f5920 0x7f932c1f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f932a000640 1 -- 172.21.15.7:0/1945768473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c0d0a50 con 0x7f932c1f4fd0 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f932aa00640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f932c06f300 0x7f932c15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f932a000640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f932c1f4fd0 0x7f932c1f53e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f931800d620 tx=0x7f931800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f9322a00640 1 -- 172.21.15.7:0/1945768473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931800b620 con 0x7f932c1f4fd0 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f9322a00640 1 -- 172.21.15.7:0/1945768473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9318019070 con 0x7f932c1f4fd0 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f9331600640 1 -- 172.21.15.7:0/1945768473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c0cfed0 con 0x7f932c1f4fd0 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.082+0000 7f9322a00640 1 -- 172.21.15.7:0/1945768473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931800ba00 con 0x7f932c1f4fd0 2024-04-25T11:16:03.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.083+0000 7f9331600640 1 -- 172.21.15.7:0/1945768473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c0d0490 con 0x7f932c1f4fd0 2024-04-25T11:16:03.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.084+0000 7f9331600640 1 -- 172.21.15.7:0/1945768473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c06a100 con 0x7f932c1f4fd0 2024-04-25T11:16:03.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.085+0000 7f9322a00640 1 -- 172.21.15.7:0/1945768473 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f931800bba0 con 0x7f932c1f4fd0 2024-04-25T11:16:03.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.086+0000 7f9322a00640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f930c0744b0 0x7f930c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.086+0000 7f9322a00640 1 -- 172.21.15.7:0/1945768473 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f9318095810 con 0x7f932c1f4fd0 2024-04-25T11:16:03.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.086+0000 7f932aa00640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f930c0744b0 0x7f930c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:03.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.087+0000 7f932aa00640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f930c0744b0 0x7f930c0769c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f931c000f60 tx=0x7f931c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:03.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.090+0000 7f9322a00640 1 -- 172.21.15.7:0/1945768473 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f931805fe00 con 0x7f932c1f4fd0 2024-04-25T11:16:03.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.290+0000 7f9331600640 1 -- 172.21.15.7:0/1945768473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f932c0b03f0 con 0x7f932c1f4fd0 2024-04-25T11:16:03.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.293+0000 7f9322a00640 1 -- 172.21.15.7:0/1945768473 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16789 (secure 0 0 0) 0x7f931805f700 con 0x7f932c1f4fd0 2024-04-25T11:16:03.295 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:03.295 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":92,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:16:03.266074+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:59.999922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T11:16:00.796013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:16:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.295+0000 7f9331600640 1 -- 172.21.15.7:0/1945768473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f930c0744b0 msgr2=0x7f930c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.295+0000 7f9331600640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f930c0744b0 0x7f930c0769c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f931c000f60 tx=0x7f931c009040 comp rx=0 tx=0).stop 2024-04-25T11:16:03.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.296+0000 7f9331600640 1 -- 172.21.15.7:0/1945768473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f932c1f4fd0 msgr2=0x7f932c1f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.296+0000 7f9331600640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f932c1f4fd0 0x7f932c1f53e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f931800d620 tx=0x7f931800dae0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.296+0000 7f9331600640 1 -- 172.21.15.7:0/1945768473 shutdown_connections 2024-04-25T11:16:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.296+0000 7f9331600640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f930c0744b0 0x7f930c0769c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.296+0000 7f9331600640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f932c1f5920 0x7f932c1f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.296+0000 7f9331600640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f932c1f4fd0 0x7f932c1f53e0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.296+0000 7f9331600640 1 --2- 172.21.15.7:0/1945768473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f932c06f300 0x7f932c15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.296+0000 7f9331600640 1 -- 172.21.15.7:0/1945768473 >> 172.21.15.7:0/1945768473 conn(0x7f932c074200 msgr2=0x7f932c154f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.296+0000 7f9331600640 1 -- 172.21.15.7:0/1945768473 shutdown_connections 2024-04-25T11:16:03.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.296+0000 7f9331600640 1 -- 172.21.15.7:0/1945768473 wait complete. 2024-04-25T11:16:03.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:16:03.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.608+0000 7f90c2200640 1 Processor -- start 2024-04-25T11:16:03.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.609+0000 7f90c2200640 1 -- start start 2024-04-25T11:16:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.611+0000 7f90c2200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90bc0b6720 0x7f90bc164710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.611+0000 7f90c2200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f90bc157320 0x7f90bc160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.611+0000 7f90c2200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90bc06f3a0 0x7f90bc06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.611+0000 7f90c2200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f90bc160a60 con 0x7f90bc0b6720 2024-04-25T11:16:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.611+0000 7f90c2200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f90bc0b1390 con 0x7f90bc157320 2024-04-25T11:16:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.611+0000 7f90c2200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f90bc0ae880 con 0x7f90bc06f3a0 2024-04-25T11:16:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.612+0000 7f90bb400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90bc0b6720 0x7f90bc164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.612+0000 7f90bb400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90bc0b6720 0x7f90bc164710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60652/0 (socket says 172.21.15.7:60652) 2024-04-25T11:16:03.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.612+0000 7f90bb400640 1 -- 172.21.15.7:0/942597170 learned_addr learned my addr 172.21.15.7:0/942597170 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.612+0000 7f90bbe00640 1 --2- 172.21.15.7:0/942597170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90bc06f3a0 0x7f90bc06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.612+0000 7f90b3e00640 1 --2- 172.21.15.7:0/942597170 >> v2:172.21.15.87:3300/0 conn(0x7f90bc157320 0x7f90bc160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.612+0000 7f90bb400640 1 -- 172.21.15.7:0/942597170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90bc06f3a0 msgr2=0x7f90bc06f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.612+0000 7f90bb400640 1 --2- 172.21.15.7:0/942597170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90bc06f3a0 0x7f90bc06f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.612+0000 7f90bb400640 1 -- 172.21.15.7:0/942597170 >> v2:172.21.15.87:3300/0 conn(0x7f90bc157320 msgr2=0x7f90bc160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:03.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.612+0000 7f90bb400640 1 --2- 172.21.15.7:0/942597170 >> v2:172.21.15.87:3300/0 conn(0x7f90bc157320 0x7f90bc160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.612+0000 7f90bb400640 1 -- 172.21.15.7:0/942597170 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc160710 con 0x7f90bc0b6720 2024-04-25T11:16:03.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.613+0000 7f90b3e00640 1 --2- 172.21.15.7:0/942597170 >> v2:172.21.15.87:3300/0 conn(0x7f90bc157320 0x7f90bc160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:03.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.614+0000 7f90bb400640 1 --2- 172.21.15.7:0/942597170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90bc0b6720 0x7f90bc164710 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f90bc072550 tx=0x7f90ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eda40e3fa4dcbfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:03.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.614+0000 7f90baa00640 1 -- 172.21.15.7:0/942597170 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac037070 con 0x7f90bc0b6720 2024-04-25T11:16:03.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.615+0000 7f90baa00640 1 -- 172.21.15.7:0/942597170 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ac032440 con 0x7f90bc0b6720 2024-04-25T11:16:03.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.615+0000 7f90baa00640 1 -- 172.21.15.7:0/942597170 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac032810 con 0x7f90bc0b6720 2024-04-25T11:16:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.615+0000 7f90c2200640 1 -- 172.21.15.7:0/942597170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90bc0b6720 msgr2=0x7f90bc164710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.615+0000 7f90c2200640 1 --2- 172.21.15.7:0/942597170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90bc0b6720 0x7f90bc164710 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f90bc072550 tx=0x7f90ac02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.615+0000 7f90c2200640 1 -- 172.21.15.7:0/942597170 shutdown_connections 2024-04-25T11:16:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.615+0000 7f90c2200640 1 --2- 172.21.15.7:0/942597170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90bc06f3a0 0x7f90bc06f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.615+0000 7f90c2200640 1 --2- 172.21.15.7:0/942597170 >> v2:172.21.15.87:3300/0 conn(0x7f90bc157320 0x7f90bc160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.615+0000 7f90c2200640 1 --2- 172.21.15.7:0/942597170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90bc0b6720 0x7f90bc164710 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.615+0000 7f90c2200640 1 -- 172.21.15.7:0/942597170 >> 172.21.15.7:0/942597170 conn(0x7f90bc0742c0 msgr2=0x7f90bc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.615+0000 7f90c2200640 1 -- 172.21.15.7:0/942597170 shutdown_connections 2024-04-25T11:16:03.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.615+0000 7f90c2200640 1 -- 172.21.15.7:0/942597170 wait complete. 2024-04-25T11:16:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.618+0000 7f90c2200640 1 Processor -- start 2024-04-25T11:16:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.618+0000 7f90c2200640 1 -- start start 2024-04-25T11:16:03.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.619+0000 7f90c2200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90bc06f3a0 0x7f90bc14a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.619+0000 7f90c2200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90bc0b6720 0x7f90bc15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.619+0000 7f90c2200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90bc157320 0x7f90bc15c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.619+0000 7f90c2200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc06fcf0 con 0x7f90bc0b6720 2024-04-25T11:16:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.619+0000 7f90c2200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc0aee00 con 0x7f90bc157320 2024-04-25T11:16:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.619+0000 7f90c2200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f90bc161430 con 0x7f90bc06f3a0 2024-04-25T11:16:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.620+0000 7f90b3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90bc0b6720 0x7f90bc15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.620+0000 7f90b3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90bc0b6720 0x7f90bc15baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60666/0 (socket says 172.21.15.7:60666) 2024-04-25T11:16:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.620+0000 7f90b3e00640 1 -- 172.21.15.7:0/2252150402 learned_addr learned my addr 172.21.15.7:0/2252150402 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:03.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.620+0000 7f90bbe00640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90bc157320 0x7f90bc15c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.620+0000 7f90b3e00640 1 -- 172.21.15.7:0/2252150402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90bc06f3a0 msgr2=0x7f90bc14a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.620+0000 7f90b3e00640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90bc06f3a0 0x7f90bc14a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.620+0000 7f90b3e00640 1 -- 172.21.15.7:0/2252150402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90bc157320 msgr2=0x7f90bc15c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.620+0000 7f90b3e00640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90bc157320 0x7f90bc15c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.620+0000 7f90b3e00640 1 -- 172.21.15.7:0/2252150402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc1547c0 con 0x7f90bc0b6720 2024-04-25T11:16:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.620+0000 7f90bbe00640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90bc157320 0x7f90bc15c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.621+0000 7f90b3e00640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90bc0b6720 0x7f90bc15baf0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c7f0 tx=0x7f90a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.621+0000 7f90b3400640 1 -- 172.21.15.7:0/2252150402 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4015070 con 0x7f90bc0b6720 2024-04-25T11:16:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.621+0000 7f90b3400640 1 -- 172.21.15.7:0/2252150402 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a4007820 con 0x7f90bc0b6720 2024-04-25T11:16:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.621+0000 7f90b3400640 1 -- 172.21.15.7:0/2252150402 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4007bd0 con 0x7f90bc0b6720 2024-04-25T11:16:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.621+0000 7f90c2200640 1 -- 172.21.15.7:0/2252150402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc158f30 con 0x7f90bc0b6720 2024-04-25T11:16:03.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.621+0000 7f90c2200640 1 -- 172.21.15.7:0/2252150402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc1594f0 con 0x7f90bc0b6720 2024-04-25T11:16:03.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.622+0000 7f90c2200640 1 -- 172.21.15.7:0/2252150402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90bc050740 con 0x7f90bc0b6720 2024-04-25T11:16:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.623+0000 7f90b3400640 1 -- 172.21.15.7:0/2252150402 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f90a4018070 con 0x7f90bc0b6720 2024-04-25T11:16:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.624+0000 7f90b3400640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90980744b0 0x7f90980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.624+0000 7f90b3400640 1 -- 172.21.15.7:0/2252150402 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f90a4095580 con 0x7f90bc0b6720 2024-04-25T11:16:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.625+0000 7f90bb400640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90980744b0 0x7f90980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.625+0000 7f90bb400640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90980744b0 0x7f90980769c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f90ac034000 tx=0x7f90ac02c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:03.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.629+0000 7f90b3400640 1 -- 172.21.15.7:0/2252150402 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90a405fae0 con 0x7f90bc0b6720 2024-04-25T11:16:03.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:03.875+0000 7f90c2200640 1 -- 172.21.15.7:0/2252150402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f90bc15de90 con 0x7f90bc0b6720 2024-04-25T11:16:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.290+0000 7f90b3400640 1 -- 172.21.15.7:0/2252150402 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) ==== 118+0+0 (secure 0 0 0) 0x7f90a405f3e0 con 0x7f90bc0b6720 2024-04-25T11:16:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.293+0000 7f90c2200640 1 -- 172.21.15.7:0/2252150402 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90980744b0 msgr2=0x7f90980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.293+0000 7f90c2200640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90980744b0 0x7f90980769c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f90ac034000 tx=0x7f90ac02c000 comp rx=0 tx=0).stop 2024-04-25T11:16:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.293+0000 7f90c2200640 1 -- 172.21.15.7:0/2252150402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90bc0b6720 msgr2=0x7f90bc15baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.293+0000 7f90c2200640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90bc0b6720 0x7f90bc15baf0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c7f0 tx=0x7f90a400ccb0 comp rx=0 tx=0).stop 2024-04-25T11:16:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.294+0000 7f90c2200640 1 -- 172.21.15.7:0/2252150402 shutdown_connections 2024-04-25T11:16:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.294+0000 7f90c2200640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90980744b0 0x7f90980769c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.294+0000 7f90c2200640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90bc157320 0x7f90bc15c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.294+0000 7f90c2200640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90bc0b6720 0x7f90bc15baf0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.294+0000 7f90c2200640 1 --2- 172.21.15.7:0/2252150402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90bc06f3a0 0x7f90bc14a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.294+0000 7f90c2200640 1 -- 172.21.15.7:0/2252150402 >> 172.21.15.7:0/2252150402 conn(0x7f90bc0742c0 msgr2=0x7f90bc0b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.294+0000 7f90c2200640 1 -- 172.21.15.7:0/2252150402 shutdown_connections 2024-04-25T11:16:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.294+0000 7f90c2200640 1 -- 172.21.15.7:0/2252150402 wait complete. 2024-04-25T11:16:04.323 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:16:04.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:16:04.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.611+0000 7f8a11200640 1 Processor -- start 2024-04-25T11:16:04.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.611+0000 7f8a11200640 1 -- start start 2024-04-25T11:16:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.614+0000 7f8a11200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a0c0b0470 0x7f8a0c06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.614+0000 7f8a11200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0c15e3c0 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.614+0000 7f8a11200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0c0cf800 con 0x7f8a0c15dfb0 2024-04-25T11:16:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.614+0000 7f8a11200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0c0d1250 con 0x7f8a0c0b0470 2024-04-25T11:16:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.614+0000 7f8a01400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a0c0b0470 0x7f8a0c06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.615+0000 7f8a01400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a0c0b0470 0x7f8a0c06f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60674/0 (socket says 172.21.15.7:60674) 2024-04-25T11:16:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.615+0000 7f8a01400640 1 -- 172.21.15.7:0/1613498245 learned_addr learned my addr 172.21.15.7:0/1613498245 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.615+0000 7f8a01400640 1 -- 172.21.15.7:0/1613498245 >> v1:172.21.15.87:6789/0 conn(0x7f8a0c15dfb0 legacy=0x7f8a0c0d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.615+0000 7f8a01400640 1 -- 172.21.15.7:0/1613498245 >> v1:172.21.15.7:6789/0 conn(0x7f8a0c06f8e0 legacy=0x7f8a0c0aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.615+0000 7f8a01400640 1 -- 172.21.15.7:0/1613498245 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c0cfaa0 con 0x7f8a0c0b0470 2024-04-25T11:16:04.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.616+0000 7f8a01400640 1 --2- 172.21.15.7:0/1613498245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a0c0b0470 0x7f8a0c06f3a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009870 tx=0x7f89f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b094698cfd38d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:04.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.617+0000 7f8a09400640 1 -- 172.21.15.7:0/1613498245 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4037070 con 0x7f8a0c0b0470 2024-04-25T11:16:04.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.617+0000 7f8a09400640 1 -- 172.21.15.7:0/1613498245 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f4032440 con 0x7f8a0c0b0470 2024-04-25T11:16:04.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.617+0000 7f8a09400640 1 -- 172.21.15.7:0/1613498245 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4032810 con 0x7f8a0c0b0470 2024-04-25T11:16:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.618+0000 7f8a11200640 1 -- 172.21.15.7:0/1613498245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a0c0b0470 msgr2=0x7f8a0c06f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.618+0000 7f8a11200640 1 --2- 172.21.15.7:0/1613498245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a0c0b0470 0x7f8a0c06f3a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f89f4009870 tx=0x7f89f402a270 comp rx=0 tx=0).stop 2024-04-25T11:16:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.618+0000 7f8a11200640 1 -- 172.21.15.7:0/1613498245 shutdown_connections 2024-04-25T11:16:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.618+0000 7f8a11200640 1 --2- 172.21.15.7:0/1613498245 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a0c0b0470 0x7f8a0c06f3a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.618+0000 7f8a11200640 1 -- 172.21.15.7:0/1613498245 >> 172.21.15.7:0/1613498245 conn(0x7f8a0c0742c0 msgr2=0x7f8a0c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.618+0000 7f8a11200640 1 -- 172.21.15.7:0/1613498245 shutdown_connections 2024-04-25T11:16:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.618+0000 7f8a11200640 1 -- 172.21.15.7:0/1613498245 wait complete. 2024-04-25T11:16:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.620+0000 7f8a11200640 1 Processor -- start 2024-04-25T11:16:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.620+0000 7f8a11200640 1 -- start start 2024-04-25T11:16:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.622+0000 7f8a11200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a0c06f8e0 0x7f8a0c0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.622+0000 7f8a11200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a0c0b0470 0x7f8a0c158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:04.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.622+0000 7f8a11200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a0c15dfb0 0x7f8a0c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:04.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.622+0000 7f8a11200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0d0340 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.622+0000 7f8a11200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c0cfe40 con 0x7f8a0c15dfb0 2024-04-25T11:16:04.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.622+0000 7f8a11200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c0d0620 con 0x7f8a0c0b0470 2024-04-25T11:16:04.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.622+0000 7f8a09e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a0c06f8e0 0x7f8a0c0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:04.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.623+0000 7f8a09e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a0c06f8e0 0x7f8a0c0cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60678/0 (socket says 172.21.15.7:60678) 2024-04-25T11:16:04.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.623+0000 7f8a09e00640 1 -- 172.21.15.7:0/3738154270 learned_addr learned my addr 172.21.15.7:0/3738154270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.623+0000 7f8a01400640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a0c0b0470 0x7f8a0c158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.623+0000 7f8a0a800640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a0c15dfb0 0x7f8a0c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.623+0000 7f8a09e00640 1 -- 172.21.15.7:0/3738154270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a0c0b0470 msgr2=0x7f8a0c158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.623+0000 7f8a09e00640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a0c0b0470 0x7f8a0c158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.623+0000 7f8a09e00640 1 -- 172.21.15.7:0/3738154270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a0c15dfb0 msgr2=0x7f8a0c0cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.623+0000 7f8a09e00640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a0c15dfb0 0x7f8a0c0cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.623+0000 7f8a09e00640 1 -- 172.21.15.7:0/3738154270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c159c80 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.623+0000 7f8a01400640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a0c0b0470 0x7f8a0c158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.623+0000 7f8a09e00640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a0c06f8e0 0x7f8a0c0cac90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c760 tx=0x7f89fc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.624+0000 7f8a02a00640 1 -- 172.21.15.7:0/3738154270 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc003b90 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.624+0000 7f8a02a00640 1 -- 172.21.15.7:0/3738154270 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89fc003d30 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.624+0000 7f8a02a00640 1 -- 172.21.15.7:0/3738154270 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00aa90 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.624+0000 7f8a11200640 1 -- 172.21.15.7:0/3738154270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c159f60 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.624+0000 7f8a11200640 1 -- 172.21.15.7:0/3738154270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c0cb220 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.625+0000 7f8a11200640 1 -- 172.21.15.7:0/3738154270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c0cb480 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.626+0000 7f8a02a00640 1 -- 172.21.15.7:0/3738154270 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f89fc00ac30 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.627+0000 7f8a02a00640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89e80744b0 0x7f89e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:04.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.627+0000 7f8a02a00640 1 -- 172.21.15.7:0/3738154270 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f89fc095280 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.628+0000 7f8a01400640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89e80744b0 0x7f89e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:04.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.628+0000 7f8a01400640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89e80744b0 0x7f89e80769c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f89f4034000 tx=0x7f89f402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:04.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.632+0000 7f8a02a00640 1 -- 172.21.15.7:0/3738154270 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89fc05f870 con 0x7f8a0c06f8e0 2024-04-25T11:16:04.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:04.867+0000 7f8a11200640 1 -- 172.21.15.7:0/3738154270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8a0c050740 con 0x7f8a0c06f8e0 2024-04-25T11:16:05.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.307+0000 7f8a02a00640 1 -- 172.21.15.7:0/3738154270 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7f89fc05f170 con 0x7f8a0c06f8e0 2024-04-25T11:16:05.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.311+0000 7f8a11200640 1 -- 172.21.15.7:0/3738154270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89e80744b0 msgr2=0x7f89e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:05.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.311+0000 7f8a11200640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89e80744b0 0x7f89e80769c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f89f4034000 tx=0x7f89f402e000 comp rx=0 tx=0).stop 2024-04-25T11:16:05.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.311+0000 7f8a11200640 1 -- 172.21.15.7:0/3738154270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a0c06f8e0 msgr2=0x7f8a0c0cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:05.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.311+0000 7f8a11200640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a0c06f8e0 0x7f8a0c0cac90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00c760 tx=0x7f89fc00cc20 comp rx=0 tx=0).stop 2024-04-25T11:16:05.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.312+0000 7f8a11200640 1 -- 172.21.15.7:0/3738154270 shutdown_connections 2024-04-25T11:16:05.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.312+0000 7f8a11200640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89e80744b0 0x7f89e80769c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.312+0000 7f8a11200640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a0c15dfb0 0x7f8a0c0cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.312+0000 7f8a11200640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a0c0b0470 0x7f8a0c158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.312+0000 7f8a11200640 1 --2- 172.21.15.7:0/3738154270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a0c06f8e0 0x7f8a0c0cac90 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.312+0000 7f8a11200640 1 -- 172.21.15.7:0/3738154270 >> 172.21.15.7:0/3738154270 conn(0x7f8a0c0742c0 msgr2=0x7f8a0c0684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.312+0000 7f8a11200640 1 -- 172.21.15.7:0/3738154270 shutdown_connections 2024-04-25T11:16:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.312+0000 7f8a11200640 1 -- 172.21.15.7:0/3738154270 wait complete. 2024-04-25T11:16:05.346 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:16:05.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:16:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.631+0000 7fcd2cc00640 1 Processor -- start 2024-04-25T11:16:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.631+0000 7fcd2cc00640 1 -- start start 2024-04-25T11:16:05.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.633+0000 7fcd2cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd2809c110 0x7fcd280cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:05.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.633+0000 7fcd2cc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd280d1cf0 con 0x7fcd280d31a0 2024-04-25T11:16:05.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.633+0000 7fcd2cc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd280d1fd0 con 0x7fcd280d2d90 2024-04-25T11:16:05.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.633+0000 7fcd2cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcd280d22b0 con 0x7fcd2809c110 2024-04-25T11:16:05.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.634+0000 7fcd27e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcd280d31a0 0x7fcd2806dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59502/0 (socket says 172.21.15.7:59502) 2024-04-25T11:16:05.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.634+0000 7fcd27e00640 1 -- 172.21.15.7:0/432155704 learned_addr learned my addr 172.21.15.7:0/432155704 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.634+0000 7fcd26a00640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd2809c110 0x7fcd280cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.634+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 455828343 0 0) 0x7fcd280d1cf0 con 0x7fcd280d31a0 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd0c003520 con 0x7fcd280d31a0 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1909969375 0 0) 0x7fcd0c003520 con 0x7fcd280d31a0 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd2809c110 msgr2=0x7fcd280cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd2809c110 0x7fcd280cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 >> v1:172.21.15.87:6789/0 conn(0x7fcd280d2d90 legacy=0x7fcd2809ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd280d2590 con 0x7fcd280d31a0 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fcd200019f0 con 0x7fcd280d31a0 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 >> v1:172.21.15.7:6789/0 conn(0x7fcd280d31a0 legacy=0x7fcd2806dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd0c0039e0 0x7fcd0c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 0x7fcd0c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd0c0051a0 0x7fcd0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcd280d22b0 con 0x7fcd0c004800 2024-04-25T11:16:05.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c00c110 con 0x7fcd0c0051a0 2024-04-25T11:16:05.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0c00c310 con 0x7fcd0c0039e0 2024-04-25T11:16:05.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.635+0000 7fcd26a00640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 0x7fcd0c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:05.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.636+0000 7fcd27400640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd0c0039e0 0x7fcd0c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:05.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.636+0000 7fcd27e00640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd0c0051a0 0x7fcd0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:05.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.636+0000 7fcd26a00640 1 -- 172.21.15.7:0/432155704 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd0c0039e0 msgr2=0x7fcd0c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:05.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.636+0000 7fcd26a00640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd0c0039e0 0x7fcd0c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.636+0000 7fcd26a00640 1 -- 172.21.15.7:0/432155704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd0c0051a0 msgr2=0x7fcd0c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:05.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.636+0000 7fcd26a00640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd0c0051a0 0x7fcd0c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.636+0000 7fcd26a00640 1 -- 172.21.15.7:0/432155704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c003520 con 0x7fcd0c004800 2024-04-25T11:16:05.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.637+0000 7fcd26a00640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 0x7fcd0c004c10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fcd28074290 tx=0x7fcd0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d83bc7a118a6d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:05.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.638+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14036070 con 0x7fcd0c004800 2024-04-25T11:16:05.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.638+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd14031400 con 0x7fcd0c004800 2024-04-25T11:16:05.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.638+0000 7fcd26000640 1 -- 172.21.15.7:0/432155704 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14009650 con 0x7fcd0c004800 2024-04-25T11:16:05.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.638+0000 7fcd2cc00640 1 -- 172.21.15.7:0/432155704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 msgr2=0x7fcd0c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:05.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.638+0000 7fcd2cc00640 1 --2- 172.21.15.7:0/432155704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 0x7fcd0c004c10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fcd28074290 tx=0x7fcd0c005830 comp rx=0 tx=0).stop 2024-04-25T11:16:05.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.638+0000 7fcd27e00640 1 -- 172.21.15.7:0/432155704 reap_dead start 2024-04-25T11:16:05.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.638+0000 7fcd2cc00640 1 -- 172.21.15.7:0/432155704 shutdown_connections 2024-04-25T11:16:05.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.638+0000 7fcd2cc00640 1 -- 172.21.15.7:0/432155704 >> 172.21.15.7:0/432155704 conn(0x7fcd280b5250 msgr2=0x7fcd280b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:05.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.639+0000 7fcd2cc00640 1 -- 172.21.15.7:0/432155704 shutdown_connections 2024-04-25T11:16:05.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.639+0000 7fcd2cc00640 1 -- 172.21.15.7:0/432155704 wait complete. 2024-04-25T11:16:05.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.641+0000 7fcd2cc00640 1 Processor -- start 2024-04-25T11:16:05.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.641+0000 7fcd2cc00640 1 -- start start 2024-04-25T11:16:05.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.643+0000 7fcd2cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 0x7fcd281631a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:05.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.643+0000 7fcd2cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd28151be0 0x7fcd280b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:05.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.643+0000 7fcd2cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd280b1b10 0x7fcd281f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:05.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.643+0000 7fcd2cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcd280b91e0 con 0x7fcd0c004800 2024-04-25T11:16:05.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.643+0000 7fcd2cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcd280d19a0 con 0x7fcd280b1b10 2024-04-25T11:16:05.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.643+0000 7fcd2cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcd2809c110 con 0x7fcd28151be0 2024-04-25T11:16:05.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.643+0000 7fcd27400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 0x7fcd281631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:05.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.643+0000 7fcd27400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 0x7fcd281631a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60696/0 (socket says 172.21.15.7:60696) 2024-04-25T11:16:05.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.643+0000 7fcd27400640 1 -- 172.21.15.7:0/389370578 learned_addr learned my addr 172.21.15.7:0/389370578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.643+0000 7fcd26a00640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd28151be0 0x7fcd280b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.644+0000 7fcd27400640 1 -- 172.21.15.7:0/389370578 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd28151be0 msgr2=0x7fcd280b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.644+0000 7fcd27400640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd28151be0 0x7fcd280b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.644+0000 7fcd27400640 1 -- 172.21.15.7:0/389370578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd280b1b10 msgr2=0x7fcd281f96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.644+0000 7fcd27e00640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd280b1b10 0x7fcd281f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.644+0000 7fcd27400640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd280b1b10 0x7fcd281f96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.644+0000 7fcd27400640 1 -- 172.21.15.7:0/389370578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd280b1f20 con 0x7fcd0c004800 2024-04-25T11:16:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.644+0000 7fcd27400640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 0x7fcd281631a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800e4e0 tx=0x7fcd1800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.644+0000 7fcd1f400640 1 -- 172.21.15.7:0/389370578 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd180052e0 con 0x7fcd0c004800 2024-04-25T11:16:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.644+0000 7fcd1f400640 1 -- 172.21.15.7:0/389370578 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd18018070 con 0x7fcd0c004800 2024-04-25T11:16:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.644+0000 7fcd1f400640 1 -- 172.21.15.7:0/389370578 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18003640 con 0x7fcd0c004800 2024-04-25T11:16:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.644+0000 7fcd2cc00640 1 -- 172.21.15.7:0/389370578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd28161c00 con 0x7fcd0c004800 2024-04-25T11:16:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.645+0000 7fcd2cc00640 1 -- 172.21.15.7:0/389370578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd281621c0 con 0x7fcd0c004800 2024-04-25T11:16:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.646+0000 7fcd2cc00640 1 -- 172.21.15.7:0/389370578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd28072510 con 0x7fcd0c004800 2024-04-25T11:16:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.647+0000 7fcd1f400640 1 -- 172.21.15.7:0/389370578 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcd180037e0 con 0x7fcd0c004800 2024-04-25T11:16:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.649+0000 7fcd1f400640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd080744b0 0x7fcd080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.649+0000 7fcd1f400640 1 -- 172.21.15.7:0/389370578 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7fcd180950a0 con 0x7fcd0c004800 2024-04-25T11:16:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.649+0000 7fcd26a00640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd080744b0 0x7fcd080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.650+0000 7fcd26a00640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd080744b0 0x7fcd080769c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcd14033000 tx=0x7fcd14008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.654+0000 7fcd1f400640 1 -- 172.21.15.7:0/389370578 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd280b1f20 con 0x7fcd0c004800 2024-04-25T11:16:05.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.897+0000 7fcd2cc00640 1 -- 172.21.15.7:0/389370578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd2809c6e0 con 0x7fcd0c004800 2024-04-25T11:16:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.898+0000 7fcd1f400640 1 -- 172.21.15.7:0/389370578 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5168 (secure 0 0 0) 0x7fcd1805f640 con 0x7fcd0c004800 2024-04-25T11:16:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:05.900 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/684015857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":684015857},{"type":"v1","addr":"172.21.15.7:6836","nonce":684015857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/493684223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":493684223},{"type":"v1","addr":"172.21.15.87:6835","nonce":493684223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6109,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/1959070928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1959070928},{"type":"v1","addr":"172.21.15.87:6833","nonce":1959070928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:16:01.820362+0000","modified":"2024-04-25T11:16:04.870476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/336889324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":336889324},{"type":"v1","addr":"172.21.15.7:6837","nonce":336889324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6960,"qdb_cluster":[6960]},"id":6}]} 2024-04-25T11:16:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.901+0000 7fcd2cc00640 1 -- 172.21.15.7:0/389370578 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd080744b0 msgr2=0x7fcd080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.901+0000 7fcd2cc00640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd080744b0 0x7fcd080769c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcd14033000 tx=0x7fcd14008030 comp rx=0 tx=0).stop 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.901+0000 7fcd2cc00640 1 -- 172.21.15.7:0/389370578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 msgr2=0x7fcd281631a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.901+0000 7fcd2cc00640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 0x7fcd281631a0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800e4e0 tx=0x7fcd1800e9a0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.901+0000 7fcd2cc00640 1 -- 172.21.15.7:0/389370578 shutdown_connections 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.901+0000 7fcd2cc00640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd080744b0 0x7fcd080769c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.902+0000 7fcd2cc00640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd280b1b10 0x7fcd281f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.902+0000 7fcd2cc00640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcd28151be0 0x7fcd280b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.902+0000 7fcd2cc00640 1 --2- 172.21.15.7:0/389370578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd0c004800 0x7fcd281631a0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.902+0000 7fcd2cc00640 1 -- 172.21.15.7:0/389370578 >> 172.21.15.7:0/389370578 conn(0x7fcd280b5250 msgr2=0x7fcd28072940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.902+0000 7fcd2cc00640 1 -- 172.21.15.7:0/389370578 shutdown_connections 2024-04-25T11:16:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:05.902+0000 7fcd2cc00640 1 -- 172.21.15.7:0/389370578 wait complete. 2024-04-25T11:16:05.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:16:06.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.216+0000 7f8567000640 1 Processor -- start 2024-04-25T11:16:06.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.217+0000 7f8567000640 1 -- start start 2024-04-25T11:16:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.219+0000 7f8567000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8560156cd0 0x7f8560163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.219+0000 7f8567000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85600b5080 con 0x7f85601570e0 2024-04-25T11:16:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.220+0000 7f8567000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85600b5360 con 0x7f8560164690 2024-04-25T11:16:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.220+0000 7f8567000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f85600b5640 con 0x7f8560156cd0 2024-04-25T11:16:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.220+0000 7f8566600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f85601570e0 0x7f85600b0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59512/0 (socket says 172.21.15.7:59512) 2024-04-25T11:16:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.220+0000 7f8566600640 1 -- 172.21.15.7:0/1814897120 learned_addr learned my addr 172.21.15.7:0/1814897120 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.220+0000 7f8565200640 1 --2- 172.21.15.7:0/1814897120 >> v2:172.21.15.87:3300/0 conn(0x7f8560156cd0 0x7f8560163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.220+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4252263427 0 0) 0x7f85600b5080 con 0x7f85601570e0 2024-04-25T11:16:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8548003520 con 0x7f85601570e0 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 966455520 0 0) 0x7f8548003520 con 0x7f85601570e0 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 >> v2:172.21.15.87:3300/0 conn(0x7f8560156cd0 msgr2=0x7f8560163ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 --2- 172.21.15.7:0/1814897120 >> v2:172.21.15.87:3300/0 conn(0x7f8560156cd0 0x7f8560163ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 >> v1:172.21.15.87:6789/0 conn(0x7f8560164690 legacy=0x7f8560070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85600b5920 con 0x7f85601570e0 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f85580019f0 con 0x7f85601570e0 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 >> v1:172.21.15.7:6789/0 conn(0x7f85601570e0 legacy=0x7f85600b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 --2- 172.21.15.7:0/1814897120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8548003a00 0x7f8548003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 --2- 172.21.15.7:0/1814897120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8548004800 0x7f8548004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 --2- 172.21.15.7:0/1814897120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85480052a0 0x7f854800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85600b5640 con 0x7f8548003a00 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f854800c230 con 0x7f85480052a0 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.221+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f854800c430 con 0x7f8548004800 2024-04-25T11:16:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.222+0000 7f8565c00640 1 --2- 172.21.15.7:0/1814897120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8548003a00 0x7f8548003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.222+0000 7f8565200640 1 --2- 172.21.15.7:0/1814897120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8548004800 0x7f8548004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.222+0000 7f8565c00640 1 -- 172.21.15.7:0/1814897120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8548004800 msgr2=0x7f8548004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.222+0000 7f8565c00640 1 --2- 172.21.15.7:0/1814897120 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8548004800 0x7f8548004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.222+0000 7f8565c00640 1 -- 172.21.15.7:0/1814897120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85480052a0 msgr2=0x7f854800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.222+0000 7f8565c00640 1 --2- 172.21.15.7:0/1814897120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85480052a0 0x7f854800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.222+0000 7f8565c00640 1 -- 172.21.15.7:0/1814897120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8548003520 con 0x7f8548003a00 2024-04-25T11:16:06.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.222+0000 7f8566600640 1 -- 172.21.15.7:0/1814897120 reap_dead start 2024-04-25T11:16:06.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.223+0000 7f8565c00640 1 --2- 172.21.15.7:0/1814897120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8548003a00 0x7f8548003e70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8554001750 tx=0x7f8548005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9757f87c4397f046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:06.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.223+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554001370 con 0x7f8548003a00 2024-04-25T11:16:06.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.223+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85540052d0 con 0x7f8548003a00 2024-04-25T11:16:06.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.223+0000 7f855fe00640 1 -- 172.21.15.7:0/1814897120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855402abe0 con 0x7f8548003a00 2024-04-25T11:16:06.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.224+0000 7f8567000640 1 -- 172.21.15.7:0/1814897120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8548003a00 msgr2=0x7f8548003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.224+0000 7f8567000640 1 --2- 172.21.15.7:0/1814897120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8548003a00 0x7f8548003e70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8554001750 tx=0x7f8548005830 comp rx=0 tx=0).stop 2024-04-25T11:16:06.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.224+0000 7f8567000640 1 -- 172.21.15.7:0/1814897120 shutdown_connections 2024-04-25T11:16:06.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.224+0000 7f8567000640 1 --2- 172.21.15.7:0/1814897120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8548003a00 0x7f8548003e70 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.224+0000 7f8567000640 1 -- 172.21.15.7:0/1814897120 >> 172.21.15.7:0/1814897120 conn(0x7f85600b3820 msgr2=0x7f85600b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:06.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.225+0000 7f8567000640 1 -- 172.21.15.7:0/1814897120 shutdown_connections 2024-04-25T11:16:06.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.225+0000 7f8567000640 1 -- 172.21.15.7:0/1814897120 wait complete. 2024-04-25T11:16:06.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.228+0000 7f8567000640 1 Processor -- start 2024-04-25T11:16:06.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.228+0000 7f8567000640 1 -- start start 2024-04-25T11:16:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.230+0000 7f8567000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8548003a00 0x7f85601f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.230+0000 7f8567000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f856014f600 0x7f85600b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.230+0000 7f8567000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85601f7580 0x7f85601f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.230+0000 7f8567000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8560070ba0 con 0x7f85601f7580 2024-04-25T11:16:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.230+0000 7f8567000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8560070a20 con 0x7f856014f600 2024-04-25T11:16:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.230+0000 7f8567000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8560070d20 con 0x7f8548003a00 2024-04-25T11:16:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.230+0000 7f8566600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85601f7580 0x7f85601f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.230+0000 7f8566600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85601f7580 0x7f85601f7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60714/0 (socket says 172.21.15.7:60714) 2024-04-25T11:16:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.230+0000 7f8566600640 1 -- 172.21.15.7:0/3381870038 learned_addr learned my addr 172.21.15.7:0/3381870038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.230+0000 7f8565c00640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8548003a00 0x7f85601f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.231+0000 7f8565200640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f856014f600 0x7f85600b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.231+0000 7f8566600640 1 -- 172.21.15.7:0/3381870038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8548003a00 msgr2=0x7f85601f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.231+0000 7f8566600640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8548003a00 0x7f85601f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.231+0000 7f8566600640 1 -- 172.21.15.7:0/3381870038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f856014f600 msgr2=0x7f85600b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.231+0000 7f8566600640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f856014f600 0x7f85600b2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.231+0000 7f8566600640 1 -- 172.21.15.7:0/3381870038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85600b2cb0 con 0x7f85601f7580 2024-04-25T11:16:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.231+0000 7f8566600640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85601f7580 0x7f85601f7bb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8560071510 tx=0x7f855800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.231+0000 7f855d600640 1 -- 172.21.15.7:0/3381870038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855800a610 con 0x7f85601f7580 2024-04-25T11:16:06.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.231+0000 7f855d600640 1 -- 172.21.15.7:0/3381870038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855800a7d0 con 0x7f85601f7580 2024-04-25T11:16:06.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.231+0000 7f855d600640 1 -- 172.21.15.7:0/3381870038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855800abc0 con 0x7f85601f7580 2024-04-25T11:16:06.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.231+0000 7f8567000640 1 -- 172.21.15.7:0/3381870038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85600b16d0 con 0x7f85601f7580 2024-04-25T11:16:06.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.232+0000 7f8567000640 1 -- 172.21.15.7:0/3381870038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85600b1c90 con 0x7f85601f7580 2024-04-25T11:16:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.233+0000 7f8567000640 1 -- 172.21.15.7:0/3381870038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8560050740 con 0x7f85601f7580 2024-04-25T11:16:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.235+0000 7f855d600640 1 -- 172.21.15.7:0/3381870038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8558007070 con 0x7f85601f7580 2024-04-25T11:16:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.236+0000 7f855d600640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8540074530 0x7f8540076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.236+0000 7f855d600640 1 -- 172.21.15.7:0/3381870038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f8558094c30 con 0x7f85601f7580 2024-04-25T11:16:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.236+0000 7f8565c00640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8540074530 0x7f8540076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.237+0000 7f8565c00640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8540074530 0x7f8540076a40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8554032000 tx=0x7f855402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.241+0000 7f855d600640 1 -- 172.21.15.7:0/3381870038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f855809b080 con 0x7f85601f7580 2024-04-25T11:16:06.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.451+0000 7f8567000640 1 -- 172.21.15.7:0/3381870038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85600b1420 con 0x7f85601f7580 2024-04-25T11:16:06.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.455+0000 7f855d600640 1 -- 172.21.15.7:0/3381870038 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16789 (secure 0 0 0) 0x7f855805f290 con 0x7f85601f7580 2024-04-25T11:16:06.457 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:06.457 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":92,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:16:03.266074+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:59.999922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T11:16:00.796013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:16:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.457+0000 7f8567000640 1 -- 172.21.15.7:0/3381870038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8540074530 msgr2=0x7f8540076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.458+0000 7f8567000640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8540074530 0x7f8540076a40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8554032000 tx=0x7f855402e000 comp rx=0 tx=0).stop 2024-04-25T11:16:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.458+0000 7f8567000640 1 -- 172.21.15.7:0/3381870038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85601f7580 msgr2=0x7f85601f7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.458+0000 7f8567000640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85601f7580 0x7f85601f7bb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8560071510 tx=0x7f855800ccd0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.458+0000 7f8567000640 1 -- 172.21.15.7:0/3381870038 shutdown_connections 2024-04-25T11:16:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.458+0000 7f8567000640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8540074530 0x7f8540076a40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.458+0000 7f8567000640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85601f7580 0x7f85601f7bb0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.458+0000 7f8567000640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f856014f600 0x7f85600b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.458+0000 7f8567000640 1 --2- 172.21.15.7:0/3381870038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8548003a00 0x7f85601f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.458+0000 7f8567000640 1 -- 172.21.15.7:0/3381870038 >> 172.21.15.7:0/3381870038 conn(0x7f85600b3820 msgr2=0x7f85600af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.458+0000 7f8567000640 1 -- 172.21.15.7:0/3381870038 shutdown_connections 2024-04-25T11:16:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.458+0000 7f8567000640 1 -- 172.21.15.7:0/3381870038 wait complete. 2024-04-25T11:16:06.489 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:16:06.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:16:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.759+0000 7f3d7ee00640 1 Processor -- start 2024-04-25T11:16:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.759+0000 7f3d7ee00640 1 -- start start 2024-04-25T11:16:06.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.760+0000 7f3d7ee00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780aed40 0x7f3d7815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.760+0000 7f3d7ee00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d780cebb0 con 0x7f3d780af420 2024-04-25T11:16:06.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.760+0000 7f3d7ee00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d780b64b0 con 0x7f3d780b5220 2024-04-25T11:16:06.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.760+0000 7f3d7ee00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3d7815e1f0 con 0x7f3d780aed40 2024-04-25T11:16:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.760+0000 7f3d7e400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3d780af420 0x7f3d780af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59524/0 (socket says 172.21.15.7:59524) 2024-04-25T11:16:06.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.760+0000 7f3d7e400640 1 -- 172.21.15.7:0/3800774552 learned_addr learned my addr 172.21.15.7:0/3800774552 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d7da00640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780aed40 0x7f3d7815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818858060 0 0) 0x7f3d70001490 con 0x7f3d780af420 2024-04-25T11:16:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d60003520 con 0x7f3d780af420 2024-04-25T11:16:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 36807833 0 0) 0x7f3d60003520 con 0x7f3d780af420 2024-04-25T11:16:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780aed40 msgr2=0x7f3d7815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780aed40 0x7f3d7815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 >> v1:172.21.15.87:6789/0 conn(0x7f3d780b5220 legacy=0x7f3d780ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7806c610 con 0x7f3d780af420 2024-04-25T11:16:06.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f3d70001bc0 con 0x7f3d780af420 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 >> v1:172.21.15.7:6789/0 conn(0x7f3d780af420 legacy=0x7f3d780af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d600039e0 0x7f3d60003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780aed40 0x7f3d60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d60004d40 0x7f3d60005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7815e1f0 con 0x7f3d60004d40 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6000bbf0 con 0x7f3d600039e0 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.761+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6000bdf0 con 0x7f3d780aed40 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.762+0000 7f3d7e400640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d60004d40 0x7f3d60005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.762+0000 7f3d7d000640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780aed40 0x7f3d60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.762+0000 7f3d7da00640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d600039e0 0x7f3d60003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.762+0000 7f3d7e400640 1 -- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780aed40 msgr2=0x7f3d60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.762+0000 7f3d7e400640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780aed40 0x7f3d60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.762+0000 7f3d7e400640 1 -- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d600039e0 msgr2=0x7f3d60003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:06.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.762+0000 7f3d7e400640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d600039e0 0x7f3d60003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.762+0000 7f3d7e400640 1 -- 172.21.15.7:0/3800774552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d60003520 con 0x7f3d60004d40 2024-04-25T11:16:06.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.762+0000 7f3d7d000640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780aed40 0x7f3d60004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:06.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.763+0000 7f3d7e400640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d60004d40 0x7f3d60005210 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000aa30 tx=0x7f3d60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b2cfdc08116444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.763+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000a650 con 0x7f3d60004d40 2024-04-25T11:16:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.763+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7002f070 con 0x7f3d60004d40 2024-04-25T11:16:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.763+0000 7f3d77e00640 1 -- 172.21.15.7:0/3800774552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70034470 con 0x7f3d60004d40 2024-04-25T11:16:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.764+0000 7f3d7ee00640 1 -- 172.21.15.7:0/3800774552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d60004d40 msgr2=0x7f3d60005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.764+0000 7f3d7ee00640 1 --2- 172.21.15.7:0/3800774552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d60004d40 0x7f3d60005210 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000aa30 tx=0x7f3d60005830 comp rx=0 tx=0).stop 2024-04-25T11:16:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.764+0000 7f3d7e400640 1 -- 172.21.15.7:0/3800774552 reap_dead start 2024-04-25T11:16:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.764+0000 7f3d7ee00640 1 -- 172.21.15.7:0/3800774552 shutdown_connections 2024-04-25T11:16:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.764+0000 7f3d7ee00640 1 -- 172.21.15.7:0/3800774552 >> 172.21.15.7:0/3800774552 conn(0x7f3d780b3820 msgr2=0x7f3d780b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.764+0000 7f3d7ee00640 1 -- 172.21.15.7:0/3800774552 shutdown_connections 2024-04-25T11:16:06.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.764+0000 7f3d7ee00640 1 -- 172.21.15.7:0/3800774552 wait complete. 2024-04-25T11:16:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.766+0000 7f3d7ee00640 1 Processor -- start 2024-04-25T11:16:06.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.767+0000 7f3d7ee00640 1 -- start start 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.768+0000 7f3d7ee00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d60004d40 0x7f3d7809ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.768+0000 7f3d7ee00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d7815b7c0 0x7f3d7815bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.768+0000 7f3d7ee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d78160570 0x7f3d78160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.768+0000 7f3d7ee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7806ceb0 con 0x7f3d78160570 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.768+0000 7f3d7ee00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7806cbd0 con 0x7f3d60004d40 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.768+0000 7f3d7ee00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3d780b6110 con 0x7f3d7815b7c0 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.768+0000 7f3d7e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d78160570 0x7f3d78160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.768+0000 7f3d7e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d78160570 0x7f3d78160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60730/0 (socket says 172.21.15.7:60730) 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.768+0000 7f3d7e400640 1 -- 172.21.15.7:0/1789663931 learned_addr learned my addr 172.21.15.7:0/1789663931 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.769+0000 7f3d7da00640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d60004d40 0x7f3d7809ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.769+0000 7f3d7d000640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d7815b7c0 0x7f3d7815bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.769+0000 7f3d7e400640 1 -- 172.21.15.7:0/1789663931 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d7815b7c0 msgr2=0x7f3d7815bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.769+0000 7f3d7e400640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d7815b7c0 0x7f3d7815bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.769+0000 7f3d7e400640 1 -- 172.21.15.7:0/1789663931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d60004d40 msgr2=0x7f3d7809ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.769+0000 7f3d7e400640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d60004d40 0x7f3d7809ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.769+0000 7f3d7e400640 1 -- 172.21.15.7:0/1789663931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d78160ec0 con 0x7f3d78160570 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.769+0000 7f3d7e400640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d78160570 0x7f3d78160980 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000a0b0 tx=0x7f3d7002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.769+0000 7f3d75600640 1 -- 172.21.15.7:0/1789663931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000a650 con 0x7f3d78160570 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.770+0000 7f3d75600640 1 -- 172.21.15.7:0/1789663931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d70004d60 con 0x7f3d78160570 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.770+0000 7f3d7ee00640 1 -- 172.21.15.7:0/1789663931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7815c530 con 0x7f3d78160570 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.770+0000 7f3d75600640 1 -- 172.21.15.7:0/1789663931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d700346b0 con 0x7f3d78160570 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.770+0000 7f3d7ee00640 1 -- 172.21.15.7:0/1789663931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7809ba70 con 0x7f3d78160570 2024-04-25T11:16:06.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.771+0000 7f3d7ee00640 1 -- 172.21.15.7:0/1789663931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d7815c2a0 con 0x7f3d78160570 2024-04-25T11:16:06.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.772+0000 7f3d75600640 1 -- 172.21.15.7:0/1789663931 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3d70034850 con 0x7f3d78160570 2024-04-25T11:16:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.774+0000 7f3d75600640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d580744b0 0x7f3d580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.774+0000 7f3d75600640 1 -- 172.21.15.7:0/1789663931 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f3d700b56f0 con 0x7f3d78160570 2024-04-25T11:16:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.774+0000 7f3d7da00640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d580744b0 0x7f3d580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.775+0000 7f3d7da00640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d580744b0 0x7f3d580769c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006fd0 tx=0x7f3d6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.779+0000 7f3d75600640 1 -- 172.21.15.7:0/1789663931 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d78160ec0 con 0x7f3d78160570 2024-04-25T11:16:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.977+0000 7f3d7ee00640 1 -- 172.21.15.7:0/1789663931 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d78050740 con 0x7f3d580744b0 2024-04-25T11:16:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:06.996+0000 7f3d75600640 1 -- 172.21.15.7:0/1789663931 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252062 (secure 0 0 0) 0x7f3d78050740 con 0x7f3d580744b0 2024-04-25T11:16:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:07.003 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.003+0000 7f3d7ee00640 1 -- 172.21.15.7:0/1789663931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d580744b0 msgr2=0x7f3d580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.003+0000 7f3d7ee00640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d580744b0 0x7f3d580769c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006fd0 tx=0x7f3d6c008040 comp rx=0 tx=0).stop 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.003+0000 7f3d7ee00640 1 -- 172.21.15.7:0/1789663931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d78160570 msgr2=0x7f3d78160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.003+0000 7f3d7ee00640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d78160570 0x7f3d78160980 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000a0b0 tx=0x7f3d7002d070 comp rx=0 tx=0).stop 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.003+0000 7f3d7ee00640 1 -- 172.21.15.7:0/1789663931 shutdown_connections 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.003+0000 7f3d7ee00640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d580744b0 0x7f3d580769c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.003+0000 7f3d7ee00640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d78160570 0x7f3d78160980 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.003+0000 7f3d7ee00640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d7815b7c0 0x7f3d7815bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.003+0000 7f3d7ee00640 1 --2- 172.21.15.7:0/1789663931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d60004d40 0x7f3d7809ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.003+0000 7f3d7ee00640 1 -- 172.21.15.7:0/1789663931 >> 172.21.15.7:0/1789663931 conn(0x7f3d780b3820 msgr2=0x7f3d780b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.003+0000 7f3d7ee00640 1 -- 172.21.15.7:0/1789663931 shutdown_connections 2024-04-25T11:16:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.004+0000 7f3d7ee00640 1 -- 172.21.15.7:0/1789663931 wait complete. 2024-04-25T11:16:07.032 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-04-25T11:16:05.913020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259452,"kb_used_data":9992,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715268,"statfs":{"total":773094113280,"available":772828434432,"internally_reserved":0,"allocated":10231808,"data_stored":5288340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.693892"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286941+0000","last_change":"2024-04-25T11:16:02.288357+0000","last_active":"2024-04-25T11:16:03.286941+0000","last_peered":"2024-04-25T11:16:03.286941+0000","last_clean":"2024-04-25T11:16:03.286941+0000","last_became_active":"2024-04-25T11:16:02.288032+0000","last_became_peered":"2024-04-25T11:16:02.288032+0000","last_unstale":"2024-04-25T11:16:03.286941+0000","last_undegraded":"2024-04-25T11:16:03.286941+0000","last_fullsized":"2024-04-25T11:16:03.286941+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:27.026977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260677+0000","last_change":"2024-04-25T11:16:02.295973+0000","last_active":"2024-04-25T11:16:04.260677+0000","last_peered":"2024-04-25T11:16:04.260677+0000","last_clean":"2024-04-25T11:16:04.260677+0000","last_became_active":"2024-04-25T11:16:02.294026+0000","last_became_peered":"2024-04-25T11:16:02.294026+0000","last_unstale":"2024-04-25T11:16:04.260677+0000","last_undegraded":"2024-04-25T11:16:04.260677+0000","last_fullsized":"2024-04-25T11:16:04.260677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:56.524518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286436+0000","last_change":"2024-04-25T11:16:02.289595+0000","last_active":"2024-04-25T11:16:03.286436+0000","last_peered":"2024-04-25T11:16:03.286436+0000","last_clean":"2024-04-25T11:16:03.286436+0000","last_became_active":"2024-04-25T11:16:02.289273+0000","last_became_peered":"2024-04-25T11:16:02.289273+0000","last_unstale":"2024-04-25T11:16:03.286436+0000","last_undegraded":"2024-04-25T11:16:03.286436+0000","last_fullsized":"2024-04-25T11:16:03.286436+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:44.561500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.167152+0000","last_change":"2024-04-25T11:16:02.281914+0000","last_active":"2024-04-25T11:16:04.167152+0000","last_peered":"2024-04-25T11:16:04.167152+0000","last_clean":"2024-04-25T11:16:04.167152+0000","last_became_active":"2024-04-25T11:16:02.281583+0000","last_became_peered":"2024-04-25T11:16:02.281583+0000","last_unstale":"2024-04-25T11:16:04.167152+0000","last_undegraded":"2024-04-25T11:16:04.167152+0000","last_fullsized":"2024-04-25T11:16:04.167152+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.373033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865007+0000","last_change":"2024-04-25T11:16:02.285417+0000","last_active":"2024-04-25T11:16:03.865007+0000","last_peered":"2024-04-25T11:16:03.865007+0000","last_clean":"2024-04-25T11:16:03.865007+0000","last_became_active":"2024-04-25T11:16:02.284961+0000","last_became_peered":"2024-04-25T11:16:02.284961+0000","last_unstale":"2024-04-25T11:16:03.865007+0000","last_undegraded":"2024-04-25T11:16:03.865007+0000","last_fullsized":"2024-04-25T11:16:03.865007+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:16.629378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735642+0000","last_change":"2024-04-25T11:16:02.283167+0000","last_active":"2024-04-25T11:16:03.735642+0000","last_peered":"2024-04-25T11:16:03.735642+0000","last_clean":"2024-04-25T11:16:03.735642+0000","last_became_active":"2024-04-25T11:16:02.282827+0000","last_became_peered":"2024-04-25T11:16:02.282827+0000","last_unstale":"2024-04-25T11:16:03.735642+0000","last_undegraded":"2024-04-25T11:16:03.735642+0000","last_fullsized":"2024-04-25T11:16:03.735642+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:01.680419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.287146+0000","last_change":"2024-04-25T11:16:02.292576+0000","last_active":"2024-04-25T11:16:03.287146+0000","last_peered":"2024-04-25T11:16:03.287146+0000","last_clean":"2024-04-25T11:16:03.287146+0000","last_became_active":"2024-04-25T11:16:02.292021+0000","last_became_peered":"2024-04-25T11:16:02.292021+0000","last_unstale":"2024-04-25T11:16:03.287146+0000","last_undegraded":"2024-04-25T11:16:03.287146+0000","last_fullsized":"2024-04-25T11:16:03.287146+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:44.942770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166343+0000","last_change":"2024-04-25T11:16:02.288441+0000","last_active":"2024-04-25T11:16:04.166343+0000","last_peered":"2024-04-25T11:16:04.166343+0000","last_clean":"2024-04-25T11:16:04.166343+0000","last_became_active":"2024-04-25T11:16:02.287436+0000","last_became_peered":"2024-04-25T11:16:02.287436+0000","last_unstale":"2024-04-25T11:16:04.166343+0000","last_undegraded":"2024-04-25T11:16:04.166343+0000","last_fullsized":"2024-04-25T11:16:04.166343+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:24.601300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865640+0000","last_change":"2024-04-25T11:16:02.292575+0000","last_active":"2024-04-25T11:16:03.865640+0000","last_peered":"2024-04-25T11:16:03.865640+0000","last_clean":"2024-04-25T11:16:03.865640+0000","last_became_active":"2024-04-25T11:16:02.291980+0000","last_became_peered":"2024-04-25T11:16:02.291980+0000","last_unstale":"2024-04-25T11:16:03.865640+0000","last_undegraded":"2024-04-25T11:16:03.865640+0000","last_fullsized":"2024-04-25T11:16:03.865640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:57.943148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282323+0000","last_change":"2024-04-25T11:16:02.285510+0000","last_active":"2024-04-25T11:16:03.282323+0000","last_peered":"2024-04-25T11:16:03.282323+0000","last_clean":"2024-04-25T11:16:03.282323+0000","last_became_active":"2024-04-25T11:16:02.285124+0000","last_became_peered":"2024-04-25T11:16:02.285124+0000","last_unstale":"2024-04-25T11:16:03.282323+0000","last_undegraded":"2024-04-25T11:16:03.282323+0000","last_fullsized":"2024-04-25T11:16:03.282323+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:45.538413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865752+0000","last_change":"2024-04-25T11:16:02.290101+0000","last_active":"2024-04-25T11:16:03.865752+0000","last_peered":"2024-04-25T11:16:03.865752+0000","last_clean":"2024-04-25T11:16:03.865752+0000","last_became_active":"2024-04-25T11:16:02.289751+0000","last_became_peered":"2024-04-25T11:16:02.289751+0000","last_unstale":"2024-04-25T11:16:03.865752+0000","last_undegraded":"2024-04-25T11:16:03.865752+0000","last_fullsized":"2024-04-25T11:16:03.865752+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:58.330695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.281927+0000","last_change":"2024-04-25T11:16:02.297591+0000","last_active":"2024-04-25T11:16:03.281927+0000","last_peered":"2024-04-25T11:16:03.281927+0000","last_clean":"2024-04-25T11:16:03.281927+0000","last_became_active":"2024-04-25T11:16:02.296575+0000","last_became_peered":"2024-04-25T11:16:02.296575+0000","last_unstale":"2024-04-25T11:16:03.281927+0000","last_undegraded":"2024-04-25T11:16:03.281927+0000","last_fullsized":"2024-04-25T11:16:03.281927+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.503780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166713+0000","last_change":"2024-04-25T11:16:02.296709+0000","last_active":"2024-04-25T11:16:04.166713+0000","last_peered":"2024-04-25T11:16:04.166713+0000","last_clean":"2024-04-25T11:16:04.166713+0000","last_became_active":"2024-04-25T11:16:02.286486+0000","last_became_peered":"2024-04-25T11:16:02.286486+0000","last_unstale":"2024-04-25T11:16:04.166713+0000","last_undegraded":"2024-04-25T11:16:04.166713+0000","last_fullsized":"2024-04-25T11:16:04.166713+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:16.629378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.864629+0000","last_change":"2024-04-25T11:16:02.287493+0000","last_active":"2024-04-25T11:16:03.864629+0000","last_peered":"2024-04-25T11:16:03.864629+0000","last_clean":"2024-04-25T11:16:03.864629+0000","last_became_active":"2024-04-25T11:16:02.287164+0000","last_became_peered":"2024-04-25T11:16:02.287164+0000","last_unstale":"2024-04-25T11:16:03.864629+0000","last_undegraded":"2024-04-25T11:16:03.864629+0000","last_fullsized":"2024-04-25T11:16:03.864629+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:56.007748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166637+0000","last_change":"2024-04-25T11:16:02.301429+0000","last_active":"2024-04-25T11:16:04.166637+0000","last_peered":"2024-04-25T11:16:04.166637+0000","last_clean":"2024-04-25T11:16:04.166637+0000","last_became_active":"2024-04-25T11:16:02.298578+0000","last_became_peered":"2024-04-25T11:16:02.298578+0000","last_unstale":"2024-04-25T11:16:04.166637+0000","last_undegraded":"2024-04-25T11:16:04.166637+0000","last_fullsized":"2024-04-25T11:16:04.166637+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:43.279367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735260+0000","last_change":"2024-04-25T11:16:02.292961+0000","last_active":"2024-04-25T11:16:03.735260+0000","last_peered":"2024-04-25T11:16:03.735260+0000","last_clean":"2024-04-25T11:16:03.735260+0000","last_became_active":"2024-04-25T11:16:02.292600+0000","last_became_peered":"2024-04-25T11:16:02.292600+0000","last_unstale":"2024-04-25T11:16:03.735260+0000","last_undegraded":"2024-04-25T11:16:03.735260+0000","last_fullsized":"2024-04-25T11:16:03.735260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:30.745430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736630+0000","last_change":"2024-04-25T11:16:02.292733+0000","last_active":"2024-04-25T11:16:03.736630+0000","last_peered":"2024-04-25T11:16:03.736630+0000","last_clean":"2024-04-25T11:16:03.736630+0000","last_became_active":"2024-04-25T11:16:02.292355+0000","last_became_peered":"2024-04-25T11:16:02.292355+0000","last_unstale":"2024-04-25T11:16:03.736630+0000","last_undegraded":"2024-04-25T11:16:03.736630+0000","last_fullsized":"2024-04-25T11:16:03.736630+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:21.565918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286973+0000","last_change":"2024-04-25T11:16:02.278720+0000","last_active":"2024-04-25T11:16:03.286973+0000","last_peered":"2024-04-25T11:16:03.286973+0000","last_clean":"2024-04-25T11:16:03.286973+0000","last_became_active":"2024-04-25T11:16:02.278374+0000","last_became_peered":"2024-04-25T11:16:02.278374+0000","last_unstale":"2024-04-25T11:16:03.286973+0000","last_undegraded":"2024-04-25T11:16:03.286973+0000","last_fullsized":"2024-04-25T11:16:03.286973+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:28.752688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865647+0000","last_change":"2024-04-25T11:16:02.287670+0000","last_active":"2024-04-25T11:16:03.865647+0000","last_peered":"2024-04-25T11:16:03.865647+0000","last_clean":"2024-04-25T11:16:03.865647+0000","last_became_active":"2024-04-25T11:16:02.284697+0000","last_became_peered":"2024-04-25T11:16:02.284697+0000","last_unstale":"2024-04-25T11:16:03.865647+0000","last_undegraded":"2024-04-25T11:16:03.865647+0000","last_fullsized":"2024-04-25T11:16:03.865647+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:37.507516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.167651+0000","last_change":"2024-04-25T11:16:02.298769+0000","last_active":"2024-04-25T11:16:04.167651+0000","last_peered":"2024-04-25T11:16:04.167651+0000","last_clean":"2024-04-25T11:16:04.167651+0000","last_became_active":"2024-04-25T11:16:02.298389+0000","last_became_peered":"2024-04-25T11:16:02.298389+0000","last_unstale":"2024-04-25T11:16:04.167651+0000","last_undegraded":"2024-04-25T11:16:04.167651+0000","last_fullsized":"2024-04-25T11:16:04.167651+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:37.507516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.864771+0000","last_change":"2024-04-25T11:16:02.290304+0000","last_active":"2024-04-25T11:16:03.864771+0000","last_peered":"2024-04-25T11:16:03.864771+0000","last_clean":"2024-04-25T11:16:03.864771+0000","last_became_active":"2024-04-25T11:16:02.289027+0000","last_became_peered":"2024-04-25T11:16:02.289027+0000","last_unstale":"2024-04-25T11:16:03.864771+0000","last_undegraded":"2024-04-25T11:16:03.864771+0000","last_fullsized":"2024-04-25T11:16:03.864771+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:43.279367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.864607+0000","last_change":"2024-04-25T11:16:02.282389+0000","last_active":"2024-04-25T11:16:03.864607+0000","last_peered":"2024-04-25T11:16:03.864607+0000","last_clean":"2024-04-25T11:16:03.864607+0000","last_became_active":"2024-04-25T11:16:02.281784+0000","last_became_peered":"2024-04-25T11:16:02.281784+0000","last_unstale":"2024-04-25T11:16:03.864607+0000","last_undegraded":"2024-04-25T11:16:03.864607+0000","last_fullsized":"2024-04-25T11:16:03.864607+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:24.601300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.354719+0000","last_change":"2024-04-25T11:16:02.284202+0000","last_active":"2024-04-25T11:16:04.354719+0000","last_peered":"2024-04-25T11:16:04.354719+0000","last_clean":"2024-04-25T11:16:04.354719+0000","last_became_active":"2024-04-25T11:16:02.281661+0000","last_became_peered":"2024-04-25T11:16:02.281661+0000","last_unstale":"2024-04-25T11:16:04.354719+0000","last_undegraded":"2024-04-25T11:16:04.354719+0000","last_fullsized":"2024-04-25T11:16:04.354719+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.373033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286442+0000","last_change":"2024-04-25T11:16:02.283002+0000","last_active":"2024-04-25T11:16:03.286442+0000","last_peered":"2024-04-25T11:16:03.286442+0000","last_clean":"2024-04-25T11:16:03.286442+0000","last_became_active":"2024-04-25T11:16:02.282691+0000","last_became_peered":"2024-04-25T11:16:02.282691+0000","last_unstale":"2024-04-25T11:16:03.286442+0000","last_undegraded":"2024-04-25T11:16:03.286442+0000","last_fullsized":"2024-04-25T11:16:03.286442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:32.202992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260740+0000","last_change":"2024-04-25T11:16:02.282642+0000","last_active":"2024-04-25T11:16:04.260740+0000","last_peered":"2024-04-25T11:16:04.260740+0000","last_clean":"2024-04-25T11:16:04.260740+0000","last_became_active":"2024-04-25T11:16:02.278247+0000","last_became_peered":"2024-04-25T11:16:02.278247+0000","last_unstale":"2024-04-25T11:16:04.260740+0000","last_undegraded":"2024-04-25T11:16:04.260740+0000","last_fullsized":"2024-04-25T11:16:04.260740+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:21.104807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865677+0000","last_change":"2024-04-25T11:16:02.282592+0000","last_active":"2024-04-25T11:16:03.865677+0000","last_peered":"2024-04-25T11:16:03.865677+0000","last_clean":"2024-04-25T11:16:03.865677+0000","last_became_active":"2024-04-25T11:16:02.282242+0000","last_became_peered":"2024-04-25T11:16:02.282242+0000","last_unstale":"2024-04-25T11:16:03.865677+0000","last_undegraded":"2024-04-25T11:16:03.865677+0000","last_fullsized":"2024-04-25T11:16:03.865677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:23.157139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282474+0000","last_change":"2024-04-25T11:16:02.286210+0000","last_active":"2024-04-25T11:16:03.282474+0000","last_peered":"2024-04-25T11:16:03.282474+0000","last_clean":"2024-04-25T11:16:03.282474+0000","last_became_active":"2024-04-25T11:16:02.285549+0000","last_became_peered":"2024-04-25T11:16:02.285549+0000","last_unstale":"2024-04-25T11:16:03.282474+0000","last_undegraded":"2024-04-25T11:16:03.282474+0000","last_fullsized":"2024-04-25T11:16:03.282474+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:32.773109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.168164+0000","last_change":"2024-04-25T11:16:02.298006+0000","last_active":"2024-04-25T11:16:04.168164+0000","last_peered":"2024-04-25T11:16:04.168164+0000","last_clean":"2024-04-25T11:16:04.168164+0000","last_became_active":"2024-04-25T11:16:02.297149+0000","last_became_peered":"2024-04-25T11:16:02.297149+0000","last_unstale":"2024-04-25T11:16:04.168164+0000","last_undegraded":"2024-04-25T11:16:04.168164+0000","last_fullsized":"2024-04-25T11:16:04.168164+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:58.330695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286748+0000","last_change":"2024-04-25T11:16:02.273364+0000","last_active":"2024-04-25T11:16:03.286748+0000","last_peered":"2024-04-25T11:16:03.286748+0000","last_clean":"2024-04-25T11:16:03.286748+0000","last_became_active":"2024-04-25T11:16:02.272983+0000","last_became_peered":"2024-04-25T11:16:02.272983+0000","last_unstale":"2024-04-25T11:16:03.286748+0000","last_undegraded":"2024-04-25T11:16:03.286748+0000","last_fullsized":"2024-04-25T11:16:03.286748+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:27.026977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865261+0000","last_change":"2024-04-25T11:16:02.282123+0000","last_active":"2024-04-25T11:16:03.865261+0000","last_peered":"2024-04-25T11:16:03.865261+0000","last_clean":"2024-04-25T11:16:03.865261+0000","last_became_active":"2024-04-25T11:16:02.281395+0000","last_became_peered":"2024-04-25T11:16:02.281395+0000","last_unstale":"2024-04-25T11:16:03.865261+0000","last_undegraded":"2024-04-25T11:16:03.865261+0000","last_fullsized":"2024-04-25T11:16:03.865261+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:20.163095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260500+0000","last_change":"2024-04-25T11:16:02.294104+0000","last_active":"2024-04-25T11:16:04.260500+0000","last_peered":"2024-04-25T11:16:04.260500+0000","last_clean":"2024-04-25T11:16:04.260500+0000","last_became_active":"2024-04-25T11:16:02.293729+0000","last_became_peered":"2024-04-25T11:16:02.293729+0000","last_unstale":"2024-04-25T11:16:04.260500+0000","last_undegraded":"2024-04-25T11:16:04.260500+0000","last_fullsized":"2024-04-25T11:16:04.260500+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:02.468542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735746+0000","last_change":"2024-04-25T11:16:02.292985+0000","last_active":"2024-04-25T11:16:03.735746+0000","last_peered":"2024-04-25T11:16:03.735746+0000","last_clean":"2024-04-25T11:16:03.735746+0000","last_became_active":"2024-04-25T11:16:02.292677+0000","last_became_peered":"2024-04-25T11:16:02.292677+0000","last_unstale":"2024-04-25T11:16:03.735746+0000","last_undegraded":"2024-04-25T11:16:03.735746+0000","last_fullsized":"2024-04-25T11:16:03.735746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:47.889192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736174+0000","last_change":"2024-04-25T11:16:02.288846+0000","last_active":"2024-04-25T11:16:03.736174+0000","last_peered":"2024-04-25T11:16:03.736174+0000","last_clean":"2024-04-25T11:16:03.736174+0000","last_became_active":"2024-04-25T11:16:02.288522+0000","last_became_peered":"2024-04-25T11:16:02.288522+0000","last_unstale":"2024-04-25T11:16:03.736174+0000","last_undegraded":"2024-04-25T11:16:03.736174+0000","last_fullsized":"2024-04-25T11:16:03.736174+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:56.985747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.354766+0000","last_change":"2024-04-25T11:16:01.279001+0000","last_active":"2024-04-25T11:16:04.354766+0000","last_peered":"2024-04-25T11:16:04.354766+0000","last_clean":"2024-04-25T11:16:04.354766+0000","last_became_active":"2024-04-25T11:16:01.278677+0000","last_became_peered":"2024-04-25T11:16:01.278677+0000","last_unstale":"2024-04-25T11:16:04.354766+0000","last_undegraded":"2024-04-25T11:16:04.354766+0000","last_fullsized":"2024-04-25T11:16:04.354766+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:50.006353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.737002+0000","last_change":"2024-04-25T11:16:02.279010+0000","last_active":"2024-04-25T11:16:03.737002+0000","last_peered":"2024-04-25T11:16:03.737002+0000","last_clean":"2024-04-25T11:16:03.737002+0000","last_became_active":"2024-04-25T11:16:02.278707+0000","last_became_peered":"2024-04-25T11:16:02.278707+0000","last_unstale":"2024-04-25T11:16:03.737002+0000","last_undegraded":"2024-04-25T11:16:03.737002+0000","last_fullsized":"2024-04-25T11:16:03.737002+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:29.517820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865768+0000","last_change":"2024-04-25T11:16:01.272632+0000","last_active":"2024-04-25T11:16:03.865768+0000","last_peered":"2024-04-25T11:16:03.865768+0000","last_clean":"2024-04-25T11:16:03.865768+0000","last_became_active":"2024-04-25T11:16:01.272274+0000","last_became_peered":"2024-04-25T11:16:01.272274+0000","last_unstale":"2024-04-25T11:16:03.865768+0000","last_undegraded":"2024-04-25T11:16:03.865768+0000","last_fullsized":"2024-04-25T11:16:03.865768+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:30.088920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286946+0000","last_change":"2024-04-25T11:16:02.280647+0000","last_active":"2024-04-25T11:16:03.286946+0000","last_peered":"2024-04-25T11:16:03.286946+0000","last_clean":"2024-04-25T11:16:03.286946+0000","last_became_active":"2024-04-25T11:16:02.280287+0000","last_became_peered":"2024-04-25T11:16:02.280287+0000","last_unstale":"2024-04-25T11:16:03.286946+0000","last_undegraded":"2024-04-25T11:16:03.286946+0000","last_fullsized":"2024-04-25T11:16:03.286946+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:57.183354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286325+0000","last_change":"2024-04-25T11:16:01.271087+0000","last_active":"2024-04-25T11:16:03.286325+0000","last_peered":"2024-04-25T11:16:03.286325+0000","last_clean":"2024-04-25T11:16:03.286325+0000","last_became_active":"2024-04-25T11:16:01.269870+0000","last_became_peered":"2024-04-25T11:16:01.269870+0000","last_unstale":"2024-04-25T11:16:03.286325+0000","last_undegraded":"2024-04-25T11:16:03.286325+0000","last_fullsized":"2024-04-25T11:16:03.286325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:42.902663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736987+0000","last_change":"2024-04-25T11:16:02.288313+0000","last_active":"2024-04-25T11:16:03.736987+0000","last_peered":"2024-04-25T11:16:03.736987+0000","last_clean":"2024-04-25T11:16:03.736987+0000","last_became_active":"2024-04-25T11:16:02.287985+0000","last_became_peered":"2024-04-25T11:16:02.287985+0000","last_unstale":"2024-04-25T11:16:03.736987+0000","last_undegraded":"2024-04-25T11:16:03.736987+0000","last_fullsized":"2024-04-25T11:16:03.736987+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:39.405723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.355259+0000","last_change":"2024-04-25T11:16:01.280222+0000","last_active":"2024-04-25T11:16:04.355259+0000","last_peered":"2024-04-25T11:16:04.355259+0000","last_clean":"2024-04-25T11:16:04.355259+0000","last_became_active":"2024-04-25T11:16:01.279542+0000","last_became_peered":"2024-04-25T11:16:01.279542+0000","last_unstale":"2024-04-25T11:16:04.355259+0000","last_undegraded":"2024-04-25T11:16:04.355259+0000","last_fullsized":"2024-04-25T11:16:04.355259+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:55.026567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166449+0000","last_change":"2024-04-25T11:16:02.287801+0000","last_active":"2024-04-25T11:16:04.166449+0000","last_peered":"2024-04-25T11:16:04.166449+0000","last_clean":"2024-04-25T11:16:04.166449+0000","last_became_active":"2024-04-25T11:16:02.286111+0000","last_became_peered":"2024-04-25T11:16:02.286111+0000","last_unstale":"2024-04-25T11:16:04.166449+0000","last_undegraded":"2024-04-25T11:16:04.166449+0000","last_fullsized":"2024-04-25T11:16:04.166449+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:56.007748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865212+0000","last_change":"2024-04-25T11:16:01.273862+0000","last_active":"2024-04-25T11:16:03.865212+0000","last_peered":"2024-04-25T11:16:03.865212+0000","last_clean":"2024-04-25T11:16:03.865212+0000","last_became_active":"2024-04-25T11:16:01.272213+0000","last_became_peered":"2024-04-25T11:16:01.272213+0000","last_unstale":"2024-04-25T11:16:03.865212+0000","last_undegraded":"2024-04-25T11:16:03.865212+0000","last_fullsized":"2024-04-25T11:16:03.865212+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:09.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.354829+0000","last_change":"2024-04-25T11:16:02.291447+0000","last_active":"2024-04-25T11:16:04.354829+0000","last_peered":"2024-04-25T11:16:04.354829+0000","last_clean":"2024-04-25T11:16:04.354829+0000","last_became_active":"2024-04-25T11:16:02.290787+0000","last_became_peered":"2024-04-25T11:16:02.290787+0000","last_unstale":"2024-04-25T11:16:04.354829+0000","last_undegraded":"2024-04-25T11:16:04.354829+0000","last_fullsized":"2024-04-25T11:16:04.354829+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:57.943148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.167938+0000","last_change":"2024-04-25T11:16:01.262486+0000","last_active":"2024-04-25T11:16:04.167938+0000","last_peered":"2024-04-25T11:16:04.167938+0000","last_clean":"2024-04-25T11:16:04.167938+0000","last_became_active":"2024-04-25T11:16:01.262118+0000","last_became_peered":"2024-04-25T11:16:01.262118+0000","last_unstale":"2024-04-25T11:16:04.167938+0000","last_undegraded":"2024-04-25T11:16:04.167938+0000","last_fullsized":"2024-04-25T11:16:04.167938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:02.944477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736156+0000","last_change":"2024-04-25T11:16:02.281014+0000","last_active":"2024-04-25T11:16:03.736156+0000","last_peered":"2024-04-25T11:16:03.736156+0000","last_clean":"2024-04-25T11:16:03.736156+0000","last_became_active":"2024-04-25T11:16:02.280592+0000","last_became_peered":"2024-04-25T11:16:02.280592+0000","last_unstale":"2024-04-25T11:16:03.736156+0000","last_undegraded":"2024-04-25T11:16:03.736156+0000","last_fullsized":"2024-04-25T11:16:03.736156+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:07.774808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286752+0000","last_change":"2024-04-25T11:16:01.262719+0000","last_active":"2024-04-25T11:16:03.286752+0000","last_peered":"2024-04-25T11:16:03.286752+0000","last_clean":"2024-04-25T11:16:03.286752+0000","last_became_active":"2024-04-25T11:16:01.262353+0000","last_became_peered":"2024-04-25T11:16:01.262353+0000","last_unstale":"2024-04-25T11:16:03.286752+0000","last_undegraded":"2024-04-25T11:16:03.286752+0000","last_fullsized":"2024-04-25T11:16:03.286752+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:55.507929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.167219+0000","last_change":"2024-04-25T11:16:02.275001+0000","last_active":"2024-04-25T11:16:04.167219+0000","last_peered":"2024-04-25T11:16:04.167219+0000","last_clean":"2024-04-25T11:16:04.167219+0000","last_became_active":"2024-04-25T11:16:02.274651+0000","last_became_peered":"2024-04-25T11:16:02.274651+0000","last_unstale":"2024-04-25T11:16:04.167219+0000","last_undegraded":"2024-04-25T11:16:04.167219+0000","last_fullsized":"2024-04-25T11:16:04.167219+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:57.943148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.285815+0000","last_change":"2024-04-25T11:16:01.275205+0000","last_active":"2024-04-25T11:16:03.285815+0000","last_peered":"2024-04-25T11:16:03.285815+0000","last_clean":"2024-04-25T11:16:03.285815+0000","last_became_active":"2024-04-25T11:16:01.274817+0000","last_became_peered":"2024-04-25T11:16:01.274817+0000","last_unstale":"2024-04-25T11:16:03.285815+0000","last_undegraded":"2024-04-25T11:16:03.285815+0000","last_fullsized":"2024-04-25T11:16:03.285815+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:55.262046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865309+0000","last_change":"2024-04-25T11:16:02.296794+0000","last_active":"2024-04-25T11:16:03.865309+0000","last_peered":"2024-04-25T11:16:03.865309+0000","last_clean":"2024-04-25T11:16:03.865309+0000","last_became_active":"2024-04-25T11:16:02.296448+0000","last_became_peered":"2024-04-25T11:16:02.296448+0000","last_unstale":"2024-04-25T11:16:03.865309+0000","last_undegraded":"2024-04-25T11:16:03.865309+0000","last_fullsized":"2024-04-25T11:16:03.865309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.373033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260826+0000","last_change":"2024-04-25T11:16:01.274570+0000","last_active":"2024-04-25T11:16:04.260826+0000","last_peered":"2024-04-25T11:16:04.260826+0000","last_clean":"2024-04-25T11:16:04.260826+0000","last_became_active":"2024-04-25T11:16:01.271312+0000","last_became_peered":"2024-04-25T11:16:01.271312+0000","last_unstale":"2024-04-25T11:16:04.260826+0000","last_undegraded":"2024-04-25T11:16:04.260826+0000","last_fullsized":"2024-04-25T11:16:04.260826+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:19.146506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286144+0000","last_change":"2024-04-25T11:16:02.288766+0000","last_active":"2024-04-25T11:16:03.286144+0000","last_peered":"2024-04-25T11:16:03.286144+0000","last_clean":"2024-04-25T11:16:03.286144+0000","last_became_active":"2024-04-25T11:16:02.288159+0000","last_became_peered":"2024-04-25T11:16:02.288159+0000","last_unstale":"2024-04-25T11:16:03.286144+0000","last_undegraded":"2024-04-25T11:16:03.286144+0000","last_fullsized":"2024-04-25T11:16:03.286144+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:47.098885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286876+0000","last_change":"2024-04-25T11:16:01.274826+0000","last_active":"2024-04-25T11:16:03.286876+0000","last_peered":"2024-04-25T11:16:03.286876+0000","last_clean":"2024-04-25T11:16:03.286876+0000","last_became_active":"2024-04-25T11:16:01.274420+0000","last_became_peered":"2024-04-25T11:16:01.274420+0000","last_unstale":"2024-04-25T11:16:03.286876+0000","last_undegraded":"2024-04-25T11:16:03.286876+0000","last_fullsized":"2024-04-25T11:16:03.286876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:20.088218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286759+0000","last_change":"2024-04-25T11:16:02.289178+0000","last_active":"2024-04-25T11:16:03.286759+0000","last_peered":"2024-04-25T11:16:03.286759+0000","last_clean":"2024-04-25T11:16:03.286759+0000","last_became_active":"2024-04-25T11:16:02.288800+0000","last_became_peered":"2024-04-25T11:16:02.288800+0000","last_unstale":"2024-04-25T11:16:03.286759+0000","last_undegraded":"2024-04-25T11:16:03.286759+0000","last_fullsized":"2024-04-25T11:16:03.286759+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:38.948917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"91'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865872+0000","last_change":"2024-04-25T11:16:01.283227+0000","last_active":"2024-04-25T11:16:03.865872+0000","last_peered":"2024-04-25T11:16:03.865872+0000","last_clean":"2024-04-25T11:16:03.865872+0000","last_became_active":"2024-04-25T11:16:01.282668+0000","last_became_peered":"2024-04-25T11:16:01.282668+0000","last_unstale":"2024-04-25T11:16:03.865872+0000","last_undegraded":"2024-04-25T11:16:03.865872+0000","last_fullsized":"2024-04-25T11:16:03.865872+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:03.390971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.167736+0000","last_change":"2024-04-25T11:16:02.286305+0000","last_active":"2024-04-25T11:16:04.167736+0000","last_peered":"2024-04-25T11:16:04.167736+0000","last_clean":"2024-04-25T11:16:04.167736+0000","last_became_active":"2024-04-25T11:16:02.285888+0000","last_became_peered":"2024-04-25T11:16:02.285888+0000","last_unstale":"2024-04-25T11:16:04.167736+0000","last_undegraded":"2024-04-25T11:16:04.167736+0000","last_fullsized":"2024-04-25T11:16:04.167736+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:23.157139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"91'6","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736483+0000","last_change":"2024-04-25T11:16:01.278949+0000","last_active":"2024-04-25T11:16:03.736483+0000","last_peered":"2024-04-25T11:16:03.736483+0000","last_clean":"2024-04-25T11:16:03.736483+0000","last_became_active":"2024-04-25T11:16:01.278583+0000","last_became_peered":"2024-04-25T11:16:01.278583+0000","last_unstale":"2024-04-25T11:16:03.736483+0000","last_undegraded":"2024-04-25T11:16:03.736483+0000","last_fullsized":"2024-04-25T11:16:03.736483+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:46.082296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166909+0000","last_change":"2024-04-25T11:16:02.288151+0000","last_active":"2024-04-25T11:16:04.166909+0000","last_peered":"2024-04-25T11:16:04.166909+0000","last_clean":"2024-04-25T11:16:04.166909+0000","last_became_active":"2024-04-25T11:16:02.286563+0000","last_became_peered":"2024-04-25T11:16:02.286563+0000","last_unstale":"2024-04-25T11:16:04.166909+0000","last_undegraded":"2024-04-25T11:16:04.166909+0000","last_fullsized":"2024-04-25T11:16:04.166909+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:20.163095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.287007+0000","last_change":"2024-04-25T11:16:01.275564+0000","last_active":"2024-04-25T11:16:03.287007+0000","last_peered":"2024-04-25T11:16:03.287007+0000","last_clean":"2024-04-25T11:16:03.287007+0000","last_became_active":"2024-04-25T11:16:01.275224+0000","last_became_peered":"2024-04-25T11:16:01.275224+0000","last_unstale":"2024-04-25T11:16:03.287007+0000","last_undegraded":"2024-04-25T11:16:03.287007+0000","last_fullsized":"2024-04-25T11:16:03.287007+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:53.999519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.864527+0000","last_change":"2024-04-25T11:16:02.283621+0000","last_active":"2024-04-25T11:16:03.864527+0000","last_peered":"2024-04-25T11:16:03.864527+0000","last_clean":"2024-04-25T11:16:03.864527+0000","last_became_active":"2024-04-25T11:16:02.283277+0000","last_became_peered":"2024-04-25T11:16:02.283277+0000","last_unstale":"2024-04-25T11:16:03.864527+0000","last_undegraded":"2024-04-25T11:16:03.864527+0000","last_fullsized":"2024-04-25T11:16:03.864527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:19.134963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735992+0000","last_change":"2024-04-25T11:16:01.280826+0000","last_active":"2024-04-25T11:16:03.735992+0000","last_peered":"2024-04-25T11:16:03.735992+0000","last_clean":"2024-04-25T11:16:03.735992+0000","last_became_active":"2024-04-25T11:16:01.280479+0000","last_became_peered":"2024-04-25T11:16:01.280479+0000","last_unstale":"2024-04-25T11:16:03.735992+0000","last_undegraded":"2024-04-25T11:16:03.735992+0000","last_fullsized":"2024-04-25T11:16:03.735992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:37.932328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.283210+0000","last_change":"2024-04-25T11:16:02.285587+0000","last_active":"2024-04-25T11:16:03.283210+0000","last_peered":"2024-04-25T11:16:03.283210+0000","last_clean":"2024-04-25T11:16:03.283210+0000","last_became_active":"2024-04-25T11:16:02.285034+0000","last_became_peered":"2024-04-25T11:16:02.285034+0000","last_unstale":"2024-04-25T11:16:03.283210+0000","last_undegraded":"2024-04-25T11:16:03.283210+0000","last_fullsized":"2024-04-25T11:16:03.283210+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:23.121600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166762+0000","last_change":"2024-04-25T11:16:01.274918+0000","last_active":"2024-04-25T11:16:04.166762+0000","last_peered":"2024-04-25T11:16:04.166762+0000","last_clean":"2024-04-25T11:16:04.166762+0000","last_became_active":"2024-04-25T11:16:01.274571+0000","last_became_peered":"2024-04-25T11:16:01.274571+0000","last_unstale":"2024-04-25T11:16:04.166762+0000","last_undegraded":"2024-04-25T11:16:04.166762+0000","last_fullsized":"2024-04-25T11:16:04.166762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:59.956699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282226+0000","last_change":"2024-04-25T11:16:02.295492+0000","last_active":"2024-04-25T11:16:03.282226+0000","last_peered":"2024-04-25T11:16:03.282226+0000","last_clean":"2024-04-25T11:16:03.282226+0000","last_became_active":"2024-04-25T11:16:02.294312+0000","last_became_peered":"2024-04-25T11:16:02.294312+0000","last_unstale":"2024-04-25T11:16:03.282226+0000","last_undegraded":"2024-04-25T11:16:03.282226+0000","last_fullsized":"2024-04-25T11:16:03.282226+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:11.916916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166910+0000","last_change":"2024-04-25T11:16:01.270880+0000","last_active":"2024-04-25T11:16:04.166910+0000","last_peered":"2024-04-25T11:16:04.166910+0000","last_clean":"2024-04-25T11:16:04.166910+0000","last_became_active":"2024-04-25T11:16:01.270491+0000","last_became_peered":"2024-04-25T11:16:01.270491+0000","last_unstale":"2024-04-25T11:16:04.166910+0000","last_undegraded":"2024-04-25T11:16:04.166910+0000","last_fullsized":"2024-04-25T11:16:04.166910+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:52.750148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736602+0000","last_change":"2024-04-25T11:16:02.290143+0000","last_active":"2024-04-25T11:16:03.736602+0000","last_peered":"2024-04-25T11:16:03.736602+0000","last_clean":"2024-04-25T11:16:03.736602+0000","last_became_active":"2024-04-25T11:16:02.289804+0000","last_became_peered":"2024-04-25T11:16:02.289804+0000","last_unstale":"2024-04-25T11:16:03.736602+0000","last_undegraded":"2024-04-25T11:16:03.736602+0000","last_fullsized":"2024-04-25T11:16:03.736602+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:50.474889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.354927+0000","last_change":"2024-04-25T11:16:01.276949+0000","last_active":"2024-04-25T11:16:04.354927+0000","last_peered":"2024-04-25T11:16:04.354927+0000","last_clean":"2024-04-25T11:16:04.354927+0000","last_became_active":"2024-04-25T11:16:01.274385+0000","last_became_peered":"2024-04-25T11:16:01.274385+0000","last_unstale":"2024-04-25T11:16:04.354927+0000","last_undegraded":"2024-04-25T11:16:04.354927+0000","last_fullsized":"2024-04-25T11:16:04.354927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:10.287071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.355038+0000","last_change":"2024-04-25T11:16:02.291608+0000","last_active":"2024-04-25T11:16:04.355038+0000","last_peered":"2024-04-25T11:16:04.355038+0000","last_clean":"2024-04-25T11:16:04.355038+0000","last_became_active":"2024-04-25T11:16:02.290471+0000","last_became_peered":"2024-04-25T11:16:02.290471+0000","last_unstale":"2024-04-25T11:16:04.355038+0000","last_undegraded":"2024-04-25T11:16:04.355038+0000","last_fullsized":"2024-04-25T11:16:04.355038+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:37.507516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.287056+0000","last_change":"2024-04-25T11:16:01.279096+0000","last_active":"2024-04-25T11:16:03.287056+0000","last_peered":"2024-04-25T11:16:03.287056+0000","last_clean":"2024-04-25T11:16:03.287056+0000","last_became_active":"2024-04-25T11:16:01.278704+0000","last_became_peered":"2024-04-25T11:16:01.278704+0000","last_unstale":"2024-04-25T11:16:03.287056+0000","last_undegraded":"2024-04-25T11:16:03.287056+0000","last_fullsized":"2024-04-25T11:16:03.287056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:20.088218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865834+0000","last_change":"2024-04-25T11:16:02.297639+0000","last_active":"2024-04-25T11:16:03.865834+0000","last_peered":"2024-04-25T11:16:03.865834+0000","last_clean":"2024-04-25T11:16:03.865834+0000","last_became_active":"2024-04-25T11:16:02.297300+0000","last_became_peered":"2024-04-25T11:16:02.297300+0000","last_unstale":"2024-04-25T11:16:03.865834+0000","last_undegraded":"2024-04-25T11:16:03.865834+0000","last_fullsized":"2024-04-25T11:16:03.865834+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:44.760419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865959+0000","last_change":"2024-04-25T11:16:01.279391+0000","last_active":"2024-04-25T11:16:03.865959+0000","last_peered":"2024-04-25T11:16:03.865959+0000","last_clean":"2024-04-25T11:16:03.865959+0000","last_became_active":"2024-04-25T11:16:01.278789+0000","last_became_peered":"2024-04-25T11:16:01.278789+0000","last_unstale":"2024-04-25T11:16:03.865959+0000","last_undegraded":"2024-04-25T11:16:03.865959+0000","last_fullsized":"2024-04-25T11:16:03.865959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:33.805095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735458+0000","last_change":"2024-04-25T11:16:02.282903+0000","last_active":"2024-04-25T11:16:03.735458+0000","last_peered":"2024-04-25T11:16:03.735458+0000","last_clean":"2024-04-25T11:16:03.735458+0000","last_became_active":"2024-04-25T11:16:02.282559+0000","last_became_peered":"2024-04-25T11:16:02.282559+0000","last_unstale":"2024-04-25T11:16:03.735458+0000","last_undegraded":"2024-04-25T11:16:03.735458+0000","last_fullsized":"2024-04-25T11:16:03.735458+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:20.457810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.259725+0000","last_change":"2024-04-25T11:16:01.274062+0000","last_active":"2024-04-25T11:16:04.259725+0000","last_peered":"2024-04-25T11:16:04.259725+0000","last_clean":"2024-04-25T11:16:04.259725+0000","last_became_active":"2024-04-25T11:16:01.273734+0000","last_became_peered":"2024-04-25T11:16:01.273734+0000","last_unstale":"2024-04-25T11:16:04.259725+0000","last_undegraded":"2024-04-25T11:16:04.259725+0000","last_fullsized":"2024-04-25T11:16:04.259725+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:15.612789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.259935+0000","last_change":"2024-04-25T11:16:02.292488+0000","last_active":"2024-04-25T11:16:04.259935+0000","last_peered":"2024-04-25T11:16:04.259935+0000","last_clean":"2024-04-25T11:16:04.259935+0000","last_became_active":"2024-04-25T11:16:02.292115+0000","last_became_peered":"2024-04-25T11:16:02.292115+0000","last_unstale":"2024-04-25T11:16:04.259935+0000","last_undegraded":"2024-04-25T11:16:04.259935+0000","last_fullsized":"2024-04-25T11:16:04.259935+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:01.519084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282981+0000","last_change":"2024-04-25T11:16:01.265639+0000","last_active":"2024-04-25T11:16:03.282981+0000","last_peered":"2024-04-25T11:16:03.282981+0000","last_clean":"2024-04-25T11:16:03.282981+0000","last_became_active":"2024-04-25T11:16:01.265140+0000","last_became_peered":"2024-04-25T11:16:01.265140+0000","last_unstale":"2024-04-25T11:16:03.282981+0000","last_undegraded":"2024-04-25T11:16:03.282981+0000","last_fullsized":"2024-04-25T11:16:03.282981+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:34.092415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260100+0000","last_change":"2024-04-25T11:16:02.294790+0000","last_active":"2024-04-25T11:16:04.260100+0000","last_peered":"2024-04-25T11:16:04.260100+0000","last_clean":"2024-04-25T11:16:04.260100+0000","last_became_active":"2024-04-25T11:16:02.294373+0000","last_became_peered":"2024-04-25T11:16:02.294373+0000","last_unstale":"2024-04-25T11:16:04.260100+0000","last_undegraded":"2024-04-25T11:16:04.260100+0000","last_fullsized":"2024-04-25T11:16:04.260100+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:18.362912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260145+0000","last_change":"2024-04-25T11:16:01.275344+0000","last_active":"2024-04-25T11:16:04.260145+0000","last_peered":"2024-04-25T11:16:04.260145+0000","last_clean":"2024-04-25T11:16:04.260145+0000","last_became_active":"2024-04-25T11:16:01.274976+0000","last_became_peered":"2024-04-25T11:16:01.274976+0000","last_unstale":"2024-04-25T11:16:04.260145+0000","last_undegraded":"2024-04-25T11:16:04.260145+0000","last_fullsized":"2024-04-25T11:16:04.260145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:42.262778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736309+0000","last_change":"2024-04-25T11:16:02.278172+0000","last_active":"2024-04-25T11:16:03.736309+0000","last_peered":"2024-04-25T11:16:03.736309+0000","last_clean":"2024-04-25T11:16:03.736309+0000","last_became_active":"2024-04-25T11:16:02.277866+0000","last_became_peered":"2024-04-25T11:16:02.277866+0000","last_unstale":"2024-04-25T11:16:03.736309+0000","last_undegraded":"2024-04-25T11:16:03.736309+0000","last_fullsized":"2024-04-25T11:16:03.736309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:52.045231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282405+0000","last_change":"2024-04-25T11:16:01.268470+0000","last_active":"2024-04-25T11:16:03.282405+0000","last_peered":"2024-04-25T11:16:03.282405+0000","last_clean":"2024-04-25T11:16:03.282405+0000","last_became_active":"2024-04-25T11:16:01.268123+0000","last_became_peered":"2024-04-25T11:16:01.268123+0000","last_unstale":"2024-04-25T11:16:03.282405+0000","last_undegraded":"2024-04-25T11:16:03.282405+0000","last_fullsized":"2024-04-25T11:16:03.282405+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.656308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.355093+0000","last_change":"2024-04-25T11:16:02.290944+0000","last_active":"2024-04-25T11:16:04.355093+0000","last_peered":"2024-04-25T11:16:04.355093+0000","last_clean":"2024-04-25T11:16:04.355093+0000","last_became_active":"2024-04-25T11:16:02.287882+0000","last_became_peered":"2024-04-25T11:16:02.287882+0000","last_unstale":"2024-04-25T11:16:04.355093+0000","last_undegraded":"2024-04-25T11:16:04.355093+0000","last_fullsized":"2024-04-25T11:16:04.355093+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:23.157139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286683+0000","last_change":"2024-04-25T11:16:01.276047+0000","last_active":"2024-04-25T11:16:03.286683+0000","last_peered":"2024-04-25T11:16:03.286683+0000","last_clean":"2024-04-25T11:16:03.286683+0000","last_became_active":"2024-04-25T11:16:01.275671+0000","last_became_peered":"2024-04-25T11:16:01.275671+0000","last_unstale":"2024-04-25T11:16:03.286683+0000","last_undegraded":"2024-04-25T11:16:03.286683+0000","last_fullsized":"2024-04-25T11:16:03.286683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:55.507929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.737238+0000","last_change":"2024-04-25T11:16:02.289254+0000","last_active":"2024-04-25T11:16:03.737238+0000","last_peered":"2024-04-25T11:16:03.737238+0000","last_clean":"2024-04-25T11:16:03.737238+0000","last_became_active":"2024-04-25T11:16:02.288919+0000","last_became_peered":"2024-04-25T11:16:02.288919+0000","last_unstale":"2024-04-25T11:16:03.737238+0000","last_undegraded":"2024-04-25T11:16:03.737238+0000","last_fullsized":"2024-04-25T11:16:03.737238+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:04.908247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.866003+0000","last_change":"2024-04-25T11:16:01.283124+0000","last_active":"2024-04-25T11:16:03.866003+0000","last_peered":"2024-04-25T11:16:03.866003+0000","last_clean":"2024-04-25T11:16:03.866003+0000","last_became_active":"2024-04-25T11:16:01.282805+0000","last_became_peered":"2024-04-25T11:16:01.282805+0000","last_unstale":"2024-04-25T11:16:03.866003+0000","last_undegraded":"2024-04-25T11:16:03.866003+0000","last_fullsized":"2024-04-25T11:16:03.866003+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:55.208037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735937+0000","last_change":"2024-04-25T11:16:02.297866+0000","last_active":"2024-04-25T11:16:03.735937+0000","last_peered":"2024-04-25T11:16:03.735937+0000","last_clean":"2024-04-25T11:16:03.735937+0000","last_became_active":"2024-04-25T11:16:02.297511+0000","last_became_peered":"2024-04-25T11:16:02.297511+0000","last_unstale":"2024-04-25T11:16:03.735937+0000","last_undegraded":"2024-04-25T11:16:03.735937+0000","last_fullsized":"2024-04-25T11:16:03.735937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:21.707181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.866043+0000","last_change":"2024-04-25T11:16:01.281747+0000","last_active":"2024-04-25T11:16:03.866043+0000","last_peered":"2024-04-25T11:16:03.866043+0000","last_clean":"2024-04-25T11:16:03.866043+0000","last_became_active":"2024-04-25T11:16:01.281422+0000","last_became_peered":"2024-04-25T11:16:01.281422+0000","last_unstale":"2024-04-25T11:16:03.866043+0000","last_undegraded":"2024-04-25T11:16:03.866043+0000","last_fullsized":"2024-04-25T11:16:03.866043+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:02.944477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260910+0000","last_change":"2024-04-25T11:16:02.289682+0000","last_active":"2024-04-25T11:16:04.260910+0000","last_peered":"2024-04-25T11:16:04.260910+0000","last_clean":"2024-04-25T11:16:04.260910+0000","last_became_active":"2024-04-25T11:16:02.289224+0000","last_became_peered":"2024-04-25T11:16:02.289224+0000","last_unstale":"2024-04-25T11:16:04.260910+0000","last_undegraded":"2024-04-25T11:16:04.260910+0000","last_fullsized":"2024-04-25T11:16:04.260910+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:55.016108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.355309+0000","last_change":"2024-04-25T11:16:01.277618+0000","last_active":"2024-04-25T11:16:04.355309+0000","last_peered":"2024-04-25T11:16:04.355309+0000","last_clean":"2024-04-25T11:16:04.355309+0000","last_became_active":"2024-04-25T11:16:01.277214+0000","last_became_peered":"2024-04-25T11:16:01.277214+0000","last_unstale":"2024-04-25T11:16:04.355309+0000","last_undegraded":"2024-04-25T11:16:04.355309+0000","last_fullsized":"2024-04-25T11:16:04.355309+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:18.118375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282081+0000","last_change":"2024-04-25T11:16:02.276760+0000","last_active":"2024-04-25T11:16:03.282081+0000","last_peered":"2024-04-25T11:16:03.282081+0000","last_clean":"2024-04-25T11:16:03.282081+0000","last_became_active":"2024-04-25T11:16:02.276428+0000","last_became_peered":"2024-04-25T11:16:02.276428+0000","last_unstale":"2024-04-25T11:16:03.282081+0000","last_undegraded":"2024-04-25T11:16:03.282081+0000","last_fullsized":"2024-04-25T11:16:03.282081+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:23.674905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736562+0000","last_change":"2024-04-25T11:16:01.274998+0000","last_active":"2024-04-25T11:16:03.736562+0000","last_peered":"2024-04-25T11:16:03.736562+0000","last_clean":"2024-04-25T11:16:03.736562+0000","last_became_active":"2024-04-25T11:16:01.274605+0000","last_became_peered":"2024-04-25T11:16:01.274605+0000","last_unstale":"2024-04-25T11:16:03.736562+0000","last_undegraded":"2024-04-25T11:16:03.736562+0000","last_fullsized":"2024-04-25T11:16:03.736562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:31.186403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286893+0000","last_change":"2024-04-25T11:16:02.274185+0000","last_active":"2024-04-25T11:16:03.286893+0000","last_peered":"2024-04-25T11:16:03.286893+0000","last_clean":"2024-04-25T11:16:03.286893+0000","last_became_active":"2024-04-25T11:16:02.273867+0000","last_became_peered":"2024-04-25T11:16:02.273867+0000","last_unstale":"2024-04-25T11:16:03.286893+0000","last_undegraded":"2024-04-25T11:16:03.286893+0000","last_fullsized":"2024-04-25T11:16:03.286893+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:47.098885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.259948+0000","last_change":"2024-04-25T11:16:01.269570+0000","last_active":"2024-04-25T11:16:04.259948+0000","last_peered":"2024-04-25T11:16:04.259948+0000","last_clean":"2024-04-25T11:16:04.259948+0000","last_became_active":"2024-04-25T11:16:01.268117+0000","last_became_peered":"2024-04-25T11:16:01.268117+0000","last_unstale":"2024-04-25T11:16:04.259948+0000","last_undegraded":"2024-04-25T11:16:04.259948+0000","last_fullsized":"2024-04-25T11:16:04.259948+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:54.991159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.355148+0000","last_change":"2024-04-25T11:16:02.285776+0000","last_active":"2024-04-25T11:16:04.355148+0000","last_peered":"2024-04-25T11:16:04.355148+0000","last_clean":"2024-04-25T11:16:04.355148+0000","last_became_active":"2024-04-25T11:16:02.285389+0000","last_became_peered":"2024-04-25T11:16:02.285389+0000","last_unstale":"2024-04-25T11:16:04.355148+0000","last_undegraded":"2024-04-25T11:16:04.355148+0000","last_fullsized":"2024-04-25T11:16:04.355148+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:58.330695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.354891+0000","last_change":"2024-04-25T11:16:01.270160+0000","last_active":"2024-04-25T11:16:04.354891+0000","last_peered":"2024-04-25T11:16:04.354891+0000","last_clean":"2024-04-25T11:16:04.354891+0000","last_became_active":"2024-04-25T11:16:01.269804+0000","last_became_peered":"2024-04-25T11:16:01.269804+0000","last_unstale":"2024-04-25T11:16:04.354891+0000","last_undegraded":"2024-04-25T11:16:04.354891+0000","last_fullsized":"2024-04-25T11:16:04.354891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:02.739967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736267+0000","last_change":"2024-04-25T11:16:02.293262+0000","last_active":"2024-04-25T11:16:03.736267+0000","last_peered":"2024-04-25T11:16:03.736267+0000","last_clean":"2024-04-25T11:16:03.736267+0000","last_became_active":"2024-04-25T11:16:02.293005+0000","last_became_peered":"2024-04-25T11:16:02.293005+0000","last_unstale":"2024-04-25T11:16:03.736267+0000","last_undegraded":"2024-04-25T11:16:03.736267+0000","last_fullsized":"2024-04-25T11:16:03.736267+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:15.504099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.285817+0000","last_change":"2024-04-25T11:16:01.277681+0000","last_active":"2024-04-25T11:16:03.285817+0000","last_peered":"2024-04-25T11:16:03.285817+0000","last_clean":"2024-04-25T11:16:03.285817+0000","last_became_active":"2024-04-25T11:16:01.277162+0000","last_became_peered":"2024-04-25T11:16:01.277162+0000","last_unstale":"2024-04-25T11:16:03.285817+0000","last_undegraded":"2024-04-25T11:16:03.285817+0000","last_fullsized":"2024-04-25T11:16:03.285817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:01.451953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286851+0000","last_change":"2024-04-25T11:16:02.279810+0000","last_active":"2024-04-25T11:16:03.286851+0000","last_peered":"2024-04-25T11:16:03.286851+0000","last_clean":"2024-04-25T11:16:03.286851+0000","last_became_active":"2024-04-25T11:16:02.279402+0000","last_became_peered":"2024-04-25T11:16:02.279402+0000","last_unstale":"2024-04-25T11:16:03.286851+0000","last_undegraded":"2024-04-25T11:16:03.286851+0000","last_fullsized":"2024-04-25T11:16:03.286851+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:28.752688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.168133+0000","last_change":"2024-04-25T11:16:01.274468+0000","last_active":"2024-04-25T11:16:04.168133+0000","last_peered":"2024-04-25T11:16:04.168133+0000","last_clean":"2024-04-25T11:16:04.168133+0000","last_became_active":"2024-04-25T11:16:01.274143+0000","last_became_peered":"2024-04-25T11:16:01.274143+0000","last_unstale":"2024-04-25T11:16:04.168133+0000","last_undegraded":"2024-04-25T11:16:04.168133+0000","last_fullsized":"2024-04-25T11:16:04.168133+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:23.017026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":251,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286293+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:16:03.286293+0000","last_peered":"2024-04-25T11:16:03.286293+0000","last_clean":"2024-04-25T11:16:03.286293+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:16:03.286293+0000","last_undegraded":"2024-04-25T11:16:03.286293+0000","last_fullsized":"2024-04-25T11:16:03.286293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476809,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1724416,"data_stored":1103272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476810,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":515064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476809,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":512848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476810,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":513008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509513,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":512848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509514,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":512918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1740800,"data_stored":1105534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542218,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":512848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:16:07.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:16:07.202 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-04-25T11:16:07.201+0000 7f8e47000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:16:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.355+0000 7fbdd2600640 1 Processor -- start 2024-04-25T11:16:07.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.355+0000 7fbdd2600640 1 -- start start 2024-04-25T11:16:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdd2600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbdcc0b2390 0x7fbdcc15c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdd2600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdcc160090 0x7fbdcc1604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdd2600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdcc1609e0 con 0x7fbdcc06cd70 2024-04-25T11:16:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdd2600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbdcc0b2900 con 0x7fbdcc160090 2024-04-25T11:16:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdd2600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbdcc0b2be0 con 0x7fbdcc0b2390 2024-04-25T11:16:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdc3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdcc160090 0x7fbdcc1604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdc3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdcc160090 0x7fbdcc1604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60740/0 (socket says 172.21.15.7:60740) 2024-04-25T11:16:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdc3e00640 1 -- 172.21.15.7:0/3232930504 learned_addr learned my addr 172.21.15.7:0/3232930504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdc3e00640 1 -- 172.21.15.7:0/3232930504 >> v2:172.21.15.87:3300/0 conn(0x7fbdcc0b2390 msgr2=0x7fbdcc15c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdd1200640 1 --2- 172.21.15.7:0/3232930504 >> v2:172.21.15.87:3300/0 conn(0x7fbdcc0b2390 0x7fbdcc15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdc3e00640 1 --2- 172.21.15.7:0/3232930504 >> v2:172.21.15.87:3300/0 conn(0x7fbdcc0b2390 0x7fbdcc15c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdc3e00640 1 -- 172.21.15.7:0/3232930504 >> v1:172.21.15.87:6789/0 conn(0x7fbdcc06cd70 legacy=0x7fbdcc06d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.358+0000 7fbdc3e00640 1 -- 172.21.15.7:0/3232930504 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc0b2ec0 con 0x7fbdcc160090 2024-04-25T11:16:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.359+0000 7fbdc3e00640 1 --2- 172.21.15.7:0/3232930504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdcc160090 0x7fbdcc1604a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc073f90 tx=0x7fbdb402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d550d6c787499777 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.359+0000 7fbdcbe00640 1 -- 172.21.15.7:0/3232930504 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4037070 con 0x7fbdcc160090 2024-04-25T11:16:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.360+0000 7fbdcbe00640 1 -- 172.21.15.7:0/3232930504 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb4032440 con 0x7fbdcc160090 2024-04-25T11:16:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.360+0000 7fbdcbe00640 1 -- 172.21.15.7:0/3232930504 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4032810 con 0x7fbdcc160090 2024-04-25T11:16:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.360+0000 7fbdd2600640 1 -- 172.21.15.7:0/3232930504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdcc160090 msgr2=0x7fbdcc1604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.360+0000 7fbdd2600640 1 --2- 172.21.15.7:0/3232930504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdcc160090 0x7fbdcc1604a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc073f90 tx=0x7fbdb402a230 comp rx=0 tx=0).stop 2024-04-25T11:16:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.361+0000 7fbdd2600640 1 -- 172.21.15.7:0/3232930504 shutdown_connections 2024-04-25T11:16:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.361+0000 7fbdd2600640 1 --2- 172.21.15.7:0/3232930504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdcc160090 0x7fbdcc1604a0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.361+0000 7fbdd2600640 1 --2- 172.21.15.7:0/3232930504 >> v2:172.21.15.87:3300/0 conn(0x7fbdcc0b2390 0x7fbdcc15c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.361+0000 7fbdd2600640 1 -- 172.21.15.7:0/3232930504 >> 172.21.15.7:0/3232930504 conn(0x7fbdcc073b60 msgr2=0x7fbdcc0b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.361+0000 7fbdd2600640 1 -- 172.21.15.7:0/3232930504 shutdown_connections 2024-04-25T11:16:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.361+0000 7fbdd2600640 1 -- 172.21.15.7:0/3232930504 wait complete. 2024-04-25T11:16:07.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.364+0000 7fbdd2600640 1 Processor -- start 2024-04-25T11:16:07.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.364+0000 7fbdd2600640 1 -- start start 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdd2600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdcc06cd70 0x7fbdcc145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdd2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdcc0b2390 0x7fbdcc147c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdd2600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdcc160090 0x7fbdcc1481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdd2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc0b0810 con 0x7fbdcc0b2390 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdd2600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc06c850 con 0x7fbdcc06cd70 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdd2600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbdcc0b0990 con 0x7fbdcc160090 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdcc0b2390 0x7fbdcc147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdcc0b2390 0x7fbdcc147c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60744/0 (socket says 172.21.15.7:60744) 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc3e00640 1 -- 172.21.15.7:0/1760579901 learned_addr learned my addr 172.21.15.7:0/1760579901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdd1200640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdcc06cd70 0x7fbdcc145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdd1c00640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdcc160090 0x7fbdcc1481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc3e00640 1 -- 172.21.15.7:0/1760579901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdcc160090 msgr2=0x7fbdcc1481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc3e00640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdcc160090 0x7fbdcc1481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc3e00640 1 -- 172.21.15.7:0/1760579901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdcc06cd70 msgr2=0x7fbdcc145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc3e00640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdcc06cd70 0x7fbdcc145540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc3e00640 1 -- 172.21.15.7:0/1760579901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc165800 con 0x7fbdcc0b2390 2024-04-25T11:16:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc3e00640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdcc0b2390 0x7fbdcc147c60 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4000c00 tx=0x7fbdb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc9600640 1 -- 172.21.15.7:0/1760579901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4037040 con 0x7fbdcc0b2390 2024-04-25T11:16:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc9600640 1 -- 172.21.15.7:0/1760579901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb4042070 con 0x7fbdcc0b2390 2024-04-25T11:16:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdd2600640 1 -- 172.21.15.7:0/1760579901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdcc165ae0 con 0x7fbdcc0b2390 2024-04-25T11:16:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdc9600640 1 -- 172.21.15.7:0/1760579901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4030bb0 con 0x7fbdcc0b2390 2024-04-25T11:16:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.366+0000 7fbdd2600640 1 -- 172.21.15.7:0/1760579901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdcc1660a0 con 0x7fbdcc0b2390 2024-04-25T11:16:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.368+0000 7fbdd2600640 1 -- 172.21.15.7:0/1760579901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdcc050740 con 0x7fbdcc0b2390 2024-04-25T11:16:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.369+0000 7fbdc9600640 1 -- 172.21.15.7:0/1760579901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbdb4047460 con 0x7fbdcc0b2390 2024-04-25T11:16:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.370+0000 7fbdc9600640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbda80744b0 0x7fbda80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.370+0000 7fbdc9600640 1 -- 172.21.15.7:0/1760579901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7fbdb40b61a0 con 0x7fbdcc0b2390 2024-04-25T11:16:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.371+0000 7fbdd1200640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbda80744b0 0x7fbda80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.371+0000 7fbdd1200640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbda80744b0 0x7fbda80769c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0070b0 tx=0x7fbdbc007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.375+0000 7fbdc9600640 1 -- 172.21.15.7:0/1760579901 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbdcc1609e0 con 0x7fbdcc0b2390 2024-04-25T11:16:07.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.574+0000 7fbdd2600640 1 -- 172.21.15.7:0/1760579901 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbdcc1655b0 con 0x7fbda80744b0 2024-04-25T11:16:07.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.582+0000 7fbdc9600640 1 -- 172.21.15.7:0/1760579901 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252062 (secure 0 0 0) 0x7fbdcc1655b0 con 0x7fbda80744b0 2024-04-25T11:16:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:07.588 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:16:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.587+0000 7fbdd2600640 1 -- 172.21.15.7:0/1760579901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbda80744b0 msgr2=0x7fbda80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.587+0000 7fbdd2600640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbda80744b0 0x7fbda80769c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc0070b0 tx=0x7fbdbc007040 comp rx=0 tx=0).stop 2024-04-25T11:16:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.588+0000 7fbdd2600640 1 -- 172.21.15.7:0/1760579901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdcc0b2390 msgr2=0x7fbdcc147c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.588+0000 7fbdd2600640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdcc0b2390 0x7fbdcc147c60 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4000c00 tx=0x7fbdb40046c0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.588+0000 7fbdd2600640 1 -- 172.21.15.7:0/1760579901 shutdown_connections 2024-04-25T11:16:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.588+0000 7fbdd2600640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbda80744b0 0x7fbda80769c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.588+0000 7fbdd2600640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdcc160090 0x7fbdcc1481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.588+0000 7fbdd2600640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdcc0b2390 0x7fbdcc147c60 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.588+0000 7fbdd2600640 1 --2- 172.21.15.7:0/1760579901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdcc06cd70 0x7fbdcc145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.588+0000 7fbdd2600640 1 -- 172.21.15.7:0/1760579901 >> 172.21.15.7:0/1760579901 conn(0x7fbdcc073b60 msgr2=0x7fbdcc06d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:07.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.588+0000 7fbdd2600640 1 -- 172.21.15.7:0/1760579901 shutdown_connections 2024-04-25T11:16:07.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.588+0000 7fbdd2600640 1 -- 172.21.15.7:0/1760579901 wait complete. 2024-04-25T11:16:07.615 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-04-25T11:16:05.913020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259452,"kb_used_data":9992,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715268,"statfs":{"total":773094113280,"available":772828434432,"internally_reserved":0,"allocated":10231808,"data_stored":5288340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.693892"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286941+0000","last_change":"2024-04-25T11:16:02.288357+0000","last_active":"2024-04-25T11:16:03.286941+0000","last_peered":"2024-04-25T11:16:03.286941+0000","last_clean":"2024-04-25T11:16:03.286941+0000","last_became_active":"2024-04-25T11:16:02.288032+0000","last_became_peered":"2024-04-25T11:16:02.288032+0000","last_unstale":"2024-04-25T11:16:03.286941+0000","last_undegraded":"2024-04-25T11:16:03.286941+0000","last_fullsized":"2024-04-25T11:16:03.286941+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:27.026977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260677+0000","last_change":"2024-04-25T11:16:02.295973+0000","last_active":"2024-04-25T11:16:04.260677+0000","last_peered":"2024-04-25T11:16:04.260677+0000","last_clean":"2024-04-25T11:16:04.260677+0000","last_became_active":"2024-04-25T11:16:02.294026+0000","last_became_peered":"2024-04-25T11:16:02.294026+0000","last_unstale":"2024-04-25T11:16:04.260677+0000","last_undegraded":"2024-04-25T11:16:04.260677+0000","last_fullsized":"2024-04-25T11:16:04.260677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:56.524518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286436+0000","last_change":"2024-04-25T11:16:02.289595+0000","last_active":"2024-04-25T11:16:03.286436+0000","last_peered":"2024-04-25T11:16:03.286436+0000","last_clean":"2024-04-25T11:16:03.286436+0000","last_became_active":"2024-04-25T11:16:02.289273+0000","last_became_peered":"2024-04-25T11:16:02.289273+0000","last_unstale":"2024-04-25T11:16:03.286436+0000","last_undegraded":"2024-04-25T11:16:03.286436+0000","last_fullsized":"2024-04-25T11:16:03.286436+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:44.561500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.167152+0000","last_change":"2024-04-25T11:16:02.281914+0000","last_active":"2024-04-25T11:16:04.167152+0000","last_peered":"2024-04-25T11:16:04.167152+0000","last_clean":"2024-04-25T11:16:04.167152+0000","last_became_active":"2024-04-25T11:16:02.281583+0000","last_became_peered":"2024-04-25T11:16:02.281583+0000","last_unstale":"2024-04-25T11:16:04.167152+0000","last_undegraded":"2024-04-25T11:16:04.167152+0000","last_fullsized":"2024-04-25T11:16:04.167152+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.373033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865007+0000","last_change":"2024-04-25T11:16:02.285417+0000","last_active":"2024-04-25T11:16:03.865007+0000","last_peered":"2024-04-25T11:16:03.865007+0000","last_clean":"2024-04-25T11:16:03.865007+0000","last_became_active":"2024-04-25T11:16:02.284961+0000","last_became_peered":"2024-04-25T11:16:02.284961+0000","last_unstale":"2024-04-25T11:16:03.865007+0000","last_undegraded":"2024-04-25T11:16:03.865007+0000","last_fullsized":"2024-04-25T11:16:03.865007+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:16.629378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735642+0000","last_change":"2024-04-25T11:16:02.283167+0000","last_active":"2024-04-25T11:16:03.735642+0000","last_peered":"2024-04-25T11:16:03.735642+0000","last_clean":"2024-04-25T11:16:03.735642+0000","last_became_active":"2024-04-25T11:16:02.282827+0000","last_became_peered":"2024-04-25T11:16:02.282827+0000","last_unstale":"2024-04-25T11:16:03.735642+0000","last_undegraded":"2024-04-25T11:16:03.735642+0000","last_fullsized":"2024-04-25T11:16:03.735642+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:01.680419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.287146+0000","last_change":"2024-04-25T11:16:02.292576+0000","last_active":"2024-04-25T11:16:03.287146+0000","last_peered":"2024-04-25T11:16:03.287146+0000","last_clean":"2024-04-25T11:16:03.287146+0000","last_became_active":"2024-04-25T11:16:02.292021+0000","last_became_peered":"2024-04-25T11:16:02.292021+0000","last_unstale":"2024-04-25T11:16:03.287146+0000","last_undegraded":"2024-04-25T11:16:03.287146+0000","last_fullsized":"2024-04-25T11:16:03.287146+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:44.942770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166343+0000","last_change":"2024-04-25T11:16:02.288441+0000","last_active":"2024-04-25T11:16:04.166343+0000","last_peered":"2024-04-25T11:16:04.166343+0000","last_clean":"2024-04-25T11:16:04.166343+0000","last_became_active":"2024-04-25T11:16:02.287436+0000","last_became_peered":"2024-04-25T11:16:02.287436+0000","last_unstale":"2024-04-25T11:16:04.166343+0000","last_undegraded":"2024-04-25T11:16:04.166343+0000","last_fullsized":"2024-04-25T11:16:04.166343+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:24.601300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865640+0000","last_change":"2024-04-25T11:16:02.292575+0000","last_active":"2024-04-25T11:16:03.865640+0000","last_peered":"2024-04-25T11:16:03.865640+0000","last_clean":"2024-04-25T11:16:03.865640+0000","last_became_active":"2024-04-25T11:16:02.291980+0000","last_became_peered":"2024-04-25T11:16:02.291980+0000","last_unstale":"2024-04-25T11:16:03.865640+0000","last_undegraded":"2024-04-25T11:16:03.865640+0000","last_fullsized":"2024-04-25T11:16:03.865640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:57.943148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282323+0000","last_change":"2024-04-25T11:16:02.285510+0000","last_active":"2024-04-25T11:16:03.282323+0000","last_peered":"2024-04-25T11:16:03.282323+0000","last_clean":"2024-04-25T11:16:03.282323+0000","last_became_active":"2024-04-25T11:16:02.285124+0000","last_became_peered":"2024-04-25T11:16:02.285124+0000","last_unstale":"2024-04-25T11:16:03.282323+0000","last_undegraded":"2024-04-25T11:16:03.282323+0000","last_fullsized":"2024-04-25T11:16:03.282323+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:45.538413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865752+0000","last_change":"2024-04-25T11:16:02.290101+0000","last_active":"2024-04-25T11:16:03.865752+0000","last_peered":"2024-04-25T11:16:03.865752+0000","last_clean":"2024-04-25T11:16:03.865752+0000","last_became_active":"2024-04-25T11:16:02.289751+0000","last_became_peered":"2024-04-25T11:16:02.289751+0000","last_unstale":"2024-04-25T11:16:03.865752+0000","last_undegraded":"2024-04-25T11:16:03.865752+0000","last_fullsized":"2024-04-25T11:16:03.865752+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:58.330695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.281927+0000","last_change":"2024-04-25T11:16:02.297591+0000","last_active":"2024-04-25T11:16:03.281927+0000","last_peered":"2024-04-25T11:16:03.281927+0000","last_clean":"2024-04-25T11:16:03.281927+0000","last_became_active":"2024-04-25T11:16:02.296575+0000","last_became_peered":"2024-04-25T11:16:02.296575+0000","last_unstale":"2024-04-25T11:16:03.281927+0000","last_undegraded":"2024-04-25T11:16:03.281927+0000","last_fullsized":"2024-04-25T11:16:03.281927+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.503780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166713+0000","last_change":"2024-04-25T11:16:02.296709+0000","last_active":"2024-04-25T11:16:04.166713+0000","last_peered":"2024-04-25T11:16:04.166713+0000","last_clean":"2024-04-25T11:16:04.166713+0000","last_became_active":"2024-04-25T11:16:02.286486+0000","last_became_peered":"2024-04-25T11:16:02.286486+0000","last_unstale":"2024-04-25T11:16:04.166713+0000","last_undegraded":"2024-04-25T11:16:04.166713+0000","last_fullsized":"2024-04-25T11:16:04.166713+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:16.629378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.864629+0000","last_change":"2024-04-25T11:16:02.287493+0000","last_active":"2024-04-25T11:16:03.864629+0000","last_peered":"2024-04-25T11:16:03.864629+0000","last_clean":"2024-04-25T11:16:03.864629+0000","last_became_active":"2024-04-25T11:16:02.287164+0000","last_became_peered":"2024-04-25T11:16:02.287164+0000","last_unstale":"2024-04-25T11:16:03.864629+0000","last_undegraded":"2024-04-25T11:16:03.864629+0000","last_fullsized":"2024-04-25T11:16:03.864629+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:56.007748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166637+0000","last_change":"2024-04-25T11:16:02.301429+0000","last_active":"2024-04-25T11:16:04.166637+0000","last_peered":"2024-04-25T11:16:04.166637+0000","last_clean":"2024-04-25T11:16:04.166637+0000","last_became_active":"2024-04-25T11:16:02.298578+0000","last_became_peered":"2024-04-25T11:16:02.298578+0000","last_unstale":"2024-04-25T11:16:04.166637+0000","last_undegraded":"2024-04-25T11:16:04.166637+0000","last_fullsized":"2024-04-25T11:16:04.166637+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:43.279367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735260+0000","last_change":"2024-04-25T11:16:02.292961+0000","last_active":"2024-04-25T11:16:03.735260+0000","last_peered":"2024-04-25T11:16:03.735260+0000","last_clean":"2024-04-25T11:16:03.735260+0000","last_became_active":"2024-04-25T11:16:02.292600+0000","last_became_peered":"2024-04-25T11:16:02.292600+0000","last_unstale":"2024-04-25T11:16:03.735260+0000","last_undegraded":"2024-04-25T11:16:03.735260+0000","last_fullsized":"2024-04-25T11:16:03.735260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:30.745430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736630+0000","last_change":"2024-04-25T11:16:02.292733+0000","last_active":"2024-04-25T11:16:03.736630+0000","last_peered":"2024-04-25T11:16:03.736630+0000","last_clean":"2024-04-25T11:16:03.736630+0000","last_became_active":"2024-04-25T11:16:02.292355+0000","last_became_peered":"2024-04-25T11:16:02.292355+0000","last_unstale":"2024-04-25T11:16:03.736630+0000","last_undegraded":"2024-04-25T11:16:03.736630+0000","last_fullsized":"2024-04-25T11:16:03.736630+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:21.565918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286973+0000","last_change":"2024-04-25T11:16:02.278720+0000","last_active":"2024-04-25T11:16:03.286973+0000","last_peered":"2024-04-25T11:16:03.286973+0000","last_clean":"2024-04-25T11:16:03.286973+0000","last_became_active":"2024-04-25T11:16:02.278374+0000","last_became_peered":"2024-04-25T11:16:02.278374+0000","last_unstale":"2024-04-25T11:16:03.286973+0000","last_undegraded":"2024-04-25T11:16:03.286973+0000","last_fullsized":"2024-04-25T11:16:03.286973+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:28.752688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865647+0000","last_change":"2024-04-25T11:16:02.287670+0000","last_active":"2024-04-25T11:16:03.865647+0000","last_peered":"2024-04-25T11:16:03.865647+0000","last_clean":"2024-04-25T11:16:03.865647+0000","last_became_active":"2024-04-25T11:16:02.284697+0000","last_became_peered":"2024-04-25T11:16:02.284697+0000","last_unstale":"2024-04-25T11:16:03.865647+0000","last_undegraded":"2024-04-25T11:16:03.865647+0000","last_fullsized":"2024-04-25T11:16:03.865647+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:37.507516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.167651+0000","last_change":"2024-04-25T11:16:02.298769+0000","last_active":"2024-04-25T11:16:04.167651+0000","last_peered":"2024-04-25T11:16:04.167651+0000","last_clean":"2024-04-25T11:16:04.167651+0000","last_became_active":"2024-04-25T11:16:02.298389+0000","last_became_peered":"2024-04-25T11:16:02.298389+0000","last_unstale":"2024-04-25T11:16:04.167651+0000","last_undegraded":"2024-04-25T11:16:04.167651+0000","last_fullsized":"2024-04-25T11:16:04.167651+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:37.507516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.864771+0000","last_change":"2024-04-25T11:16:02.290304+0000","last_active":"2024-04-25T11:16:03.864771+0000","last_peered":"2024-04-25T11:16:03.864771+0000","last_clean":"2024-04-25T11:16:03.864771+0000","last_became_active":"2024-04-25T11:16:02.289027+0000","last_became_peered":"2024-04-25T11:16:02.289027+0000","last_unstale":"2024-04-25T11:16:03.864771+0000","last_undegraded":"2024-04-25T11:16:03.864771+0000","last_fullsized":"2024-04-25T11:16:03.864771+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:43.279367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.864607+0000","last_change":"2024-04-25T11:16:02.282389+0000","last_active":"2024-04-25T11:16:03.864607+0000","last_peered":"2024-04-25T11:16:03.864607+0000","last_clean":"2024-04-25T11:16:03.864607+0000","last_became_active":"2024-04-25T11:16:02.281784+0000","last_became_peered":"2024-04-25T11:16:02.281784+0000","last_unstale":"2024-04-25T11:16:03.864607+0000","last_undegraded":"2024-04-25T11:16:03.864607+0000","last_fullsized":"2024-04-25T11:16:03.864607+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:24.601300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.354719+0000","last_change":"2024-04-25T11:16:02.284202+0000","last_active":"2024-04-25T11:16:04.354719+0000","last_peered":"2024-04-25T11:16:04.354719+0000","last_clean":"2024-04-25T11:16:04.354719+0000","last_became_active":"2024-04-25T11:16:02.281661+0000","last_became_peered":"2024-04-25T11:16:02.281661+0000","last_unstale":"2024-04-25T11:16:04.354719+0000","last_undegraded":"2024-04-25T11:16:04.354719+0000","last_fullsized":"2024-04-25T11:16:04.354719+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.373033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286442+0000","last_change":"2024-04-25T11:16:02.283002+0000","last_active":"2024-04-25T11:16:03.286442+0000","last_peered":"2024-04-25T11:16:03.286442+0000","last_clean":"2024-04-25T11:16:03.286442+0000","last_became_active":"2024-04-25T11:16:02.282691+0000","last_became_peered":"2024-04-25T11:16:02.282691+0000","last_unstale":"2024-04-25T11:16:03.286442+0000","last_undegraded":"2024-04-25T11:16:03.286442+0000","last_fullsized":"2024-04-25T11:16:03.286442+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:32.202992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260740+0000","last_change":"2024-04-25T11:16:02.282642+0000","last_active":"2024-04-25T11:16:04.260740+0000","last_peered":"2024-04-25T11:16:04.260740+0000","last_clean":"2024-04-25T11:16:04.260740+0000","last_became_active":"2024-04-25T11:16:02.278247+0000","last_became_peered":"2024-04-25T11:16:02.278247+0000","last_unstale":"2024-04-25T11:16:04.260740+0000","last_undegraded":"2024-04-25T11:16:04.260740+0000","last_fullsized":"2024-04-25T11:16:04.260740+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:21.104807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865677+0000","last_change":"2024-04-25T11:16:02.282592+0000","last_active":"2024-04-25T11:16:03.865677+0000","last_peered":"2024-04-25T11:16:03.865677+0000","last_clean":"2024-04-25T11:16:03.865677+0000","last_became_active":"2024-04-25T11:16:02.282242+0000","last_became_peered":"2024-04-25T11:16:02.282242+0000","last_unstale":"2024-04-25T11:16:03.865677+0000","last_undegraded":"2024-04-25T11:16:03.865677+0000","last_fullsized":"2024-04-25T11:16:03.865677+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:23.157139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282474+0000","last_change":"2024-04-25T11:16:02.286210+0000","last_active":"2024-04-25T11:16:03.282474+0000","last_peered":"2024-04-25T11:16:03.282474+0000","last_clean":"2024-04-25T11:16:03.282474+0000","last_became_active":"2024-04-25T11:16:02.285549+0000","last_became_peered":"2024-04-25T11:16:02.285549+0000","last_unstale":"2024-04-25T11:16:03.282474+0000","last_undegraded":"2024-04-25T11:16:03.282474+0000","last_fullsized":"2024-04-25T11:16:03.282474+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:32.773109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.168164+0000","last_change":"2024-04-25T11:16:02.298006+0000","last_active":"2024-04-25T11:16:04.168164+0000","last_peered":"2024-04-25T11:16:04.168164+0000","last_clean":"2024-04-25T11:16:04.168164+0000","last_became_active":"2024-04-25T11:16:02.297149+0000","last_became_peered":"2024-04-25T11:16:02.297149+0000","last_unstale":"2024-04-25T11:16:04.168164+0000","last_undegraded":"2024-04-25T11:16:04.168164+0000","last_fullsized":"2024-04-25T11:16:04.168164+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:58.330695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286748+0000","last_change":"2024-04-25T11:16:02.273364+0000","last_active":"2024-04-25T11:16:03.286748+0000","last_peered":"2024-04-25T11:16:03.286748+0000","last_clean":"2024-04-25T11:16:03.286748+0000","last_became_active":"2024-04-25T11:16:02.272983+0000","last_became_peered":"2024-04-25T11:16:02.272983+0000","last_unstale":"2024-04-25T11:16:03.286748+0000","last_undegraded":"2024-04-25T11:16:03.286748+0000","last_fullsized":"2024-04-25T11:16:03.286748+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:27.026977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865261+0000","last_change":"2024-04-25T11:16:02.282123+0000","last_active":"2024-04-25T11:16:03.865261+0000","last_peered":"2024-04-25T11:16:03.865261+0000","last_clean":"2024-04-25T11:16:03.865261+0000","last_became_active":"2024-04-25T11:16:02.281395+0000","last_became_peered":"2024-04-25T11:16:02.281395+0000","last_unstale":"2024-04-25T11:16:03.865261+0000","last_undegraded":"2024-04-25T11:16:03.865261+0000","last_fullsized":"2024-04-25T11:16:03.865261+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:20.163095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260500+0000","last_change":"2024-04-25T11:16:02.294104+0000","last_active":"2024-04-25T11:16:04.260500+0000","last_peered":"2024-04-25T11:16:04.260500+0000","last_clean":"2024-04-25T11:16:04.260500+0000","last_became_active":"2024-04-25T11:16:02.293729+0000","last_became_peered":"2024-04-25T11:16:02.293729+0000","last_unstale":"2024-04-25T11:16:04.260500+0000","last_undegraded":"2024-04-25T11:16:04.260500+0000","last_fullsized":"2024-04-25T11:16:04.260500+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:02.468542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735746+0000","last_change":"2024-04-25T11:16:02.292985+0000","last_active":"2024-04-25T11:16:03.735746+0000","last_peered":"2024-04-25T11:16:03.735746+0000","last_clean":"2024-04-25T11:16:03.735746+0000","last_became_active":"2024-04-25T11:16:02.292677+0000","last_became_peered":"2024-04-25T11:16:02.292677+0000","last_unstale":"2024-04-25T11:16:03.735746+0000","last_undegraded":"2024-04-25T11:16:03.735746+0000","last_fullsized":"2024-04-25T11:16:03.735746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:47.889192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736174+0000","last_change":"2024-04-25T11:16:02.288846+0000","last_active":"2024-04-25T11:16:03.736174+0000","last_peered":"2024-04-25T11:16:03.736174+0000","last_clean":"2024-04-25T11:16:03.736174+0000","last_became_active":"2024-04-25T11:16:02.288522+0000","last_became_peered":"2024-04-25T11:16:02.288522+0000","last_unstale":"2024-04-25T11:16:03.736174+0000","last_undegraded":"2024-04-25T11:16:03.736174+0000","last_fullsized":"2024-04-25T11:16:03.736174+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:56.985747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.354766+0000","last_change":"2024-04-25T11:16:01.279001+0000","last_active":"2024-04-25T11:16:04.354766+0000","last_peered":"2024-04-25T11:16:04.354766+0000","last_clean":"2024-04-25T11:16:04.354766+0000","last_became_active":"2024-04-25T11:16:01.278677+0000","last_became_peered":"2024-04-25T11:16:01.278677+0000","last_unstale":"2024-04-25T11:16:04.354766+0000","last_undegraded":"2024-04-25T11:16:04.354766+0000","last_fullsized":"2024-04-25T11:16:04.354766+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:50.006353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.737002+0000","last_change":"2024-04-25T11:16:02.279010+0000","last_active":"2024-04-25T11:16:03.737002+0000","last_peered":"2024-04-25T11:16:03.737002+0000","last_clean":"2024-04-25T11:16:03.737002+0000","last_became_active":"2024-04-25T11:16:02.278707+0000","last_became_peered":"2024-04-25T11:16:02.278707+0000","last_unstale":"2024-04-25T11:16:03.737002+0000","last_undegraded":"2024-04-25T11:16:03.737002+0000","last_fullsized":"2024-04-25T11:16:03.737002+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:29.517820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865768+0000","last_change":"2024-04-25T11:16:01.272632+0000","last_active":"2024-04-25T11:16:03.865768+0000","last_peered":"2024-04-25T11:16:03.865768+0000","last_clean":"2024-04-25T11:16:03.865768+0000","last_became_active":"2024-04-25T11:16:01.272274+0000","last_became_peered":"2024-04-25T11:16:01.272274+0000","last_unstale":"2024-04-25T11:16:03.865768+0000","last_undegraded":"2024-04-25T11:16:03.865768+0000","last_fullsized":"2024-04-25T11:16:03.865768+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:30.088920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286946+0000","last_change":"2024-04-25T11:16:02.280647+0000","last_active":"2024-04-25T11:16:03.286946+0000","last_peered":"2024-04-25T11:16:03.286946+0000","last_clean":"2024-04-25T11:16:03.286946+0000","last_became_active":"2024-04-25T11:16:02.280287+0000","last_became_peered":"2024-04-25T11:16:02.280287+0000","last_unstale":"2024-04-25T11:16:03.286946+0000","last_undegraded":"2024-04-25T11:16:03.286946+0000","last_fullsized":"2024-04-25T11:16:03.286946+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:57.183354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286325+0000","last_change":"2024-04-25T11:16:01.271087+0000","last_active":"2024-04-25T11:16:03.286325+0000","last_peered":"2024-04-25T11:16:03.286325+0000","last_clean":"2024-04-25T11:16:03.286325+0000","last_became_active":"2024-04-25T11:16:01.269870+0000","last_became_peered":"2024-04-25T11:16:01.269870+0000","last_unstale":"2024-04-25T11:16:03.286325+0000","last_undegraded":"2024-04-25T11:16:03.286325+0000","last_fullsized":"2024-04-25T11:16:03.286325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:42.902663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736987+0000","last_change":"2024-04-25T11:16:02.288313+0000","last_active":"2024-04-25T11:16:03.736987+0000","last_peered":"2024-04-25T11:16:03.736987+0000","last_clean":"2024-04-25T11:16:03.736987+0000","last_became_active":"2024-04-25T11:16:02.287985+0000","last_became_peered":"2024-04-25T11:16:02.287985+0000","last_unstale":"2024-04-25T11:16:03.736987+0000","last_undegraded":"2024-04-25T11:16:03.736987+0000","last_fullsized":"2024-04-25T11:16:03.736987+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:39.405723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.355259+0000","last_change":"2024-04-25T11:16:01.280222+0000","last_active":"2024-04-25T11:16:04.355259+0000","last_peered":"2024-04-25T11:16:04.355259+0000","last_clean":"2024-04-25T11:16:04.355259+0000","last_became_active":"2024-04-25T11:16:01.279542+0000","last_became_peered":"2024-04-25T11:16:01.279542+0000","last_unstale":"2024-04-25T11:16:04.355259+0000","last_undegraded":"2024-04-25T11:16:04.355259+0000","last_fullsized":"2024-04-25T11:16:04.355259+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:55.026567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166449+0000","last_change":"2024-04-25T11:16:02.287801+0000","last_active":"2024-04-25T11:16:04.166449+0000","last_peered":"2024-04-25T11:16:04.166449+0000","last_clean":"2024-04-25T11:16:04.166449+0000","last_became_active":"2024-04-25T11:16:02.286111+0000","last_became_peered":"2024-04-25T11:16:02.286111+0000","last_unstale":"2024-04-25T11:16:04.166449+0000","last_undegraded":"2024-04-25T11:16:04.166449+0000","last_fullsized":"2024-04-25T11:16:04.166449+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:56.007748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865212+0000","last_change":"2024-04-25T11:16:01.273862+0000","last_active":"2024-04-25T11:16:03.865212+0000","last_peered":"2024-04-25T11:16:03.865212+0000","last_clean":"2024-04-25T11:16:03.865212+0000","last_became_active":"2024-04-25T11:16:01.272213+0000","last_became_peered":"2024-04-25T11:16:01.272213+0000","last_unstale":"2024-04-25T11:16:03.865212+0000","last_undegraded":"2024-04-25T11:16:03.865212+0000","last_fullsized":"2024-04-25T11:16:03.865212+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:09.627667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.354829+0000","last_change":"2024-04-25T11:16:02.291447+0000","last_active":"2024-04-25T11:16:04.354829+0000","last_peered":"2024-04-25T11:16:04.354829+0000","last_clean":"2024-04-25T11:16:04.354829+0000","last_became_active":"2024-04-25T11:16:02.290787+0000","last_became_peered":"2024-04-25T11:16:02.290787+0000","last_unstale":"2024-04-25T11:16:04.354829+0000","last_undegraded":"2024-04-25T11:16:04.354829+0000","last_fullsized":"2024-04-25T11:16:04.354829+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:57.943148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.167938+0000","last_change":"2024-04-25T11:16:01.262486+0000","last_active":"2024-04-25T11:16:04.167938+0000","last_peered":"2024-04-25T11:16:04.167938+0000","last_clean":"2024-04-25T11:16:04.167938+0000","last_became_active":"2024-04-25T11:16:01.262118+0000","last_became_peered":"2024-04-25T11:16:01.262118+0000","last_unstale":"2024-04-25T11:16:04.167938+0000","last_undegraded":"2024-04-25T11:16:04.167938+0000","last_fullsized":"2024-04-25T11:16:04.167938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:02.944477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736156+0000","last_change":"2024-04-25T11:16:02.281014+0000","last_active":"2024-04-25T11:16:03.736156+0000","last_peered":"2024-04-25T11:16:03.736156+0000","last_clean":"2024-04-25T11:16:03.736156+0000","last_became_active":"2024-04-25T11:16:02.280592+0000","last_became_peered":"2024-04-25T11:16:02.280592+0000","last_unstale":"2024-04-25T11:16:03.736156+0000","last_undegraded":"2024-04-25T11:16:03.736156+0000","last_fullsized":"2024-04-25T11:16:03.736156+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:07.774808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286752+0000","last_change":"2024-04-25T11:16:01.262719+0000","last_active":"2024-04-25T11:16:03.286752+0000","last_peered":"2024-04-25T11:16:03.286752+0000","last_clean":"2024-04-25T11:16:03.286752+0000","last_became_active":"2024-04-25T11:16:01.262353+0000","last_became_peered":"2024-04-25T11:16:01.262353+0000","last_unstale":"2024-04-25T11:16:03.286752+0000","last_undegraded":"2024-04-25T11:16:03.286752+0000","last_fullsized":"2024-04-25T11:16:03.286752+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:55.507929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.167219+0000","last_change":"2024-04-25T11:16:02.275001+0000","last_active":"2024-04-25T11:16:04.167219+0000","last_peered":"2024-04-25T11:16:04.167219+0000","last_clean":"2024-04-25T11:16:04.167219+0000","last_became_active":"2024-04-25T11:16:02.274651+0000","last_became_peered":"2024-04-25T11:16:02.274651+0000","last_unstale":"2024-04-25T11:16:04.167219+0000","last_undegraded":"2024-04-25T11:16:04.167219+0000","last_fullsized":"2024-04-25T11:16:04.167219+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:57.943148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.285815+0000","last_change":"2024-04-25T11:16:01.275205+0000","last_active":"2024-04-25T11:16:03.285815+0000","last_peered":"2024-04-25T11:16:03.285815+0000","last_clean":"2024-04-25T11:16:03.285815+0000","last_became_active":"2024-04-25T11:16:01.274817+0000","last_became_peered":"2024-04-25T11:16:01.274817+0000","last_unstale":"2024-04-25T11:16:03.285815+0000","last_undegraded":"2024-04-25T11:16:03.285815+0000","last_fullsized":"2024-04-25T11:16:03.285815+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:55.262046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865309+0000","last_change":"2024-04-25T11:16:02.296794+0000","last_active":"2024-04-25T11:16:03.865309+0000","last_peered":"2024-04-25T11:16:03.865309+0000","last_clean":"2024-04-25T11:16:03.865309+0000","last_became_active":"2024-04-25T11:16:02.296448+0000","last_became_peered":"2024-04-25T11:16:02.296448+0000","last_unstale":"2024-04-25T11:16:03.865309+0000","last_undegraded":"2024-04-25T11:16:03.865309+0000","last_fullsized":"2024-04-25T11:16:03.865309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.373033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260826+0000","last_change":"2024-04-25T11:16:01.274570+0000","last_active":"2024-04-25T11:16:04.260826+0000","last_peered":"2024-04-25T11:16:04.260826+0000","last_clean":"2024-04-25T11:16:04.260826+0000","last_became_active":"2024-04-25T11:16:01.271312+0000","last_became_peered":"2024-04-25T11:16:01.271312+0000","last_unstale":"2024-04-25T11:16:04.260826+0000","last_undegraded":"2024-04-25T11:16:04.260826+0000","last_fullsized":"2024-04-25T11:16:04.260826+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:19.146506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286144+0000","last_change":"2024-04-25T11:16:02.288766+0000","last_active":"2024-04-25T11:16:03.286144+0000","last_peered":"2024-04-25T11:16:03.286144+0000","last_clean":"2024-04-25T11:16:03.286144+0000","last_became_active":"2024-04-25T11:16:02.288159+0000","last_became_peered":"2024-04-25T11:16:02.288159+0000","last_unstale":"2024-04-25T11:16:03.286144+0000","last_undegraded":"2024-04-25T11:16:03.286144+0000","last_fullsized":"2024-04-25T11:16:03.286144+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:47.098885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286876+0000","last_change":"2024-04-25T11:16:01.274826+0000","last_active":"2024-04-25T11:16:03.286876+0000","last_peered":"2024-04-25T11:16:03.286876+0000","last_clean":"2024-04-25T11:16:03.286876+0000","last_became_active":"2024-04-25T11:16:01.274420+0000","last_became_peered":"2024-04-25T11:16:01.274420+0000","last_unstale":"2024-04-25T11:16:03.286876+0000","last_undegraded":"2024-04-25T11:16:03.286876+0000","last_fullsized":"2024-04-25T11:16:03.286876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:20.088218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286759+0000","last_change":"2024-04-25T11:16:02.289178+0000","last_active":"2024-04-25T11:16:03.286759+0000","last_peered":"2024-04-25T11:16:03.286759+0000","last_clean":"2024-04-25T11:16:03.286759+0000","last_became_active":"2024-04-25T11:16:02.288800+0000","last_became_peered":"2024-04-25T11:16:02.288800+0000","last_unstale":"2024-04-25T11:16:03.286759+0000","last_undegraded":"2024-04-25T11:16:03.286759+0000","last_fullsized":"2024-04-25T11:16:03.286759+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:38.948917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"91'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865872+0000","last_change":"2024-04-25T11:16:01.283227+0000","last_active":"2024-04-25T11:16:03.865872+0000","last_peered":"2024-04-25T11:16:03.865872+0000","last_clean":"2024-04-25T11:16:03.865872+0000","last_became_active":"2024-04-25T11:16:01.282668+0000","last_became_peered":"2024-04-25T11:16:01.282668+0000","last_unstale":"2024-04-25T11:16:03.865872+0000","last_undegraded":"2024-04-25T11:16:03.865872+0000","last_fullsized":"2024-04-25T11:16:03.865872+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:03.390971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.167736+0000","last_change":"2024-04-25T11:16:02.286305+0000","last_active":"2024-04-25T11:16:04.167736+0000","last_peered":"2024-04-25T11:16:04.167736+0000","last_clean":"2024-04-25T11:16:04.167736+0000","last_became_active":"2024-04-25T11:16:02.285888+0000","last_became_peered":"2024-04-25T11:16:02.285888+0000","last_unstale":"2024-04-25T11:16:04.167736+0000","last_undegraded":"2024-04-25T11:16:04.167736+0000","last_fullsized":"2024-04-25T11:16:04.167736+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:23.157139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"91'6","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736483+0000","last_change":"2024-04-25T11:16:01.278949+0000","last_active":"2024-04-25T11:16:03.736483+0000","last_peered":"2024-04-25T11:16:03.736483+0000","last_clean":"2024-04-25T11:16:03.736483+0000","last_became_active":"2024-04-25T11:16:01.278583+0000","last_became_peered":"2024-04-25T11:16:01.278583+0000","last_unstale":"2024-04-25T11:16:03.736483+0000","last_undegraded":"2024-04-25T11:16:03.736483+0000","last_fullsized":"2024-04-25T11:16:03.736483+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:46.082296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166909+0000","last_change":"2024-04-25T11:16:02.288151+0000","last_active":"2024-04-25T11:16:04.166909+0000","last_peered":"2024-04-25T11:16:04.166909+0000","last_clean":"2024-04-25T11:16:04.166909+0000","last_became_active":"2024-04-25T11:16:02.286563+0000","last_became_peered":"2024-04-25T11:16:02.286563+0000","last_unstale":"2024-04-25T11:16:04.166909+0000","last_undegraded":"2024-04-25T11:16:04.166909+0000","last_fullsized":"2024-04-25T11:16:04.166909+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:20.163095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.287007+0000","last_change":"2024-04-25T11:16:01.275564+0000","last_active":"2024-04-25T11:16:03.287007+0000","last_peered":"2024-04-25T11:16:03.287007+0000","last_clean":"2024-04-25T11:16:03.287007+0000","last_became_active":"2024-04-25T11:16:01.275224+0000","last_became_peered":"2024-04-25T11:16:01.275224+0000","last_unstale":"2024-04-25T11:16:03.287007+0000","last_undegraded":"2024-04-25T11:16:03.287007+0000","last_fullsized":"2024-04-25T11:16:03.287007+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:53.999519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.864527+0000","last_change":"2024-04-25T11:16:02.283621+0000","last_active":"2024-04-25T11:16:03.864527+0000","last_peered":"2024-04-25T11:16:03.864527+0000","last_clean":"2024-04-25T11:16:03.864527+0000","last_became_active":"2024-04-25T11:16:02.283277+0000","last_became_peered":"2024-04-25T11:16:02.283277+0000","last_unstale":"2024-04-25T11:16:03.864527+0000","last_undegraded":"2024-04-25T11:16:03.864527+0000","last_fullsized":"2024-04-25T11:16:03.864527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:19.134963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735992+0000","last_change":"2024-04-25T11:16:01.280826+0000","last_active":"2024-04-25T11:16:03.735992+0000","last_peered":"2024-04-25T11:16:03.735992+0000","last_clean":"2024-04-25T11:16:03.735992+0000","last_became_active":"2024-04-25T11:16:01.280479+0000","last_became_peered":"2024-04-25T11:16:01.280479+0000","last_unstale":"2024-04-25T11:16:03.735992+0000","last_undegraded":"2024-04-25T11:16:03.735992+0000","last_fullsized":"2024-04-25T11:16:03.735992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:37.932328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.283210+0000","last_change":"2024-04-25T11:16:02.285587+0000","last_active":"2024-04-25T11:16:03.283210+0000","last_peered":"2024-04-25T11:16:03.283210+0000","last_clean":"2024-04-25T11:16:03.283210+0000","last_became_active":"2024-04-25T11:16:02.285034+0000","last_became_peered":"2024-04-25T11:16:02.285034+0000","last_unstale":"2024-04-25T11:16:03.283210+0000","last_undegraded":"2024-04-25T11:16:03.283210+0000","last_fullsized":"2024-04-25T11:16:03.283210+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:23.121600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166762+0000","last_change":"2024-04-25T11:16:01.274918+0000","last_active":"2024-04-25T11:16:04.166762+0000","last_peered":"2024-04-25T11:16:04.166762+0000","last_clean":"2024-04-25T11:16:04.166762+0000","last_became_active":"2024-04-25T11:16:01.274571+0000","last_became_peered":"2024-04-25T11:16:01.274571+0000","last_unstale":"2024-04-25T11:16:04.166762+0000","last_undegraded":"2024-04-25T11:16:04.166762+0000","last_fullsized":"2024-04-25T11:16:04.166762+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:59.956699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282226+0000","last_change":"2024-04-25T11:16:02.295492+0000","last_active":"2024-04-25T11:16:03.282226+0000","last_peered":"2024-04-25T11:16:03.282226+0000","last_clean":"2024-04-25T11:16:03.282226+0000","last_became_active":"2024-04-25T11:16:02.294312+0000","last_became_peered":"2024-04-25T11:16:02.294312+0000","last_unstale":"2024-04-25T11:16:03.282226+0000","last_undegraded":"2024-04-25T11:16:03.282226+0000","last_fullsized":"2024-04-25T11:16:03.282226+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:11.916916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.166910+0000","last_change":"2024-04-25T11:16:01.270880+0000","last_active":"2024-04-25T11:16:04.166910+0000","last_peered":"2024-04-25T11:16:04.166910+0000","last_clean":"2024-04-25T11:16:04.166910+0000","last_became_active":"2024-04-25T11:16:01.270491+0000","last_became_peered":"2024-04-25T11:16:01.270491+0000","last_unstale":"2024-04-25T11:16:04.166910+0000","last_undegraded":"2024-04-25T11:16:04.166910+0000","last_fullsized":"2024-04-25T11:16:04.166910+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:52.750148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736602+0000","last_change":"2024-04-25T11:16:02.290143+0000","last_active":"2024-04-25T11:16:03.736602+0000","last_peered":"2024-04-25T11:16:03.736602+0000","last_clean":"2024-04-25T11:16:03.736602+0000","last_became_active":"2024-04-25T11:16:02.289804+0000","last_became_peered":"2024-04-25T11:16:02.289804+0000","last_unstale":"2024-04-25T11:16:03.736602+0000","last_undegraded":"2024-04-25T11:16:03.736602+0000","last_fullsized":"2024-04-25T11:16:03.736602+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:50.474889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.354927+0000","last_change":"2024-04-25T11:16:01.276949+0000","last_active":"2024-04-25T11:16:04.354927+0000","last_peered":"2024-04-25T11:16:04.354927+0000","last_clean":"2024-04-25T11:16:04.354927+0000","last_became_active":"2024-04-25T11:16:01.274385+0000","last_became_peered":"2024-04-25T11:16:01.274385+0000","last_unstale":"2024-04-25T11:16:04.354927+0000","last_undegraded":"2024-04-25T11:16:04.354927+0000","last_fullsized":"2024-04-25T11:16:04.354927+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:10.287071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.355038+0000","last_change":"2024-04-25T11:16:02.291608+0000","last_active":"2024-04-25T11:16:04.355038+0000","last_peered":"2024-04-25T11:16:04.355038+0000","last_clean":"2024-04-25T11:16:04.355038+0000","last_became_active":"2024-04-25T11:16:02.290471+0000","last_became_peered":"2024-04-25T11:16:02.290471+0000","last_unstale":"2024-04-25T11:16:04.355038+0000","last_undegraded":"2024-04-25T11:16:04.355038+0000","last_fullsized":"2024-04-25T11:16:04.355038+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:37.507516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.287056+0000","last_change":"2024-04-25T11:16:01.279096+0000","last_active":"2024-04-25T11:16:03.287056+0000","last_peered":"2024-04-25T11:16:03.287056+0000","last_clean":"2024-04-25T11:16:03.287056+0000","last_became_active":"2024-04-25T11:16:01.278704+0000","last_became_peered":"2024-04-25T11:16:01.278704+0000","last_unstale":"2024-04-25T11:16:03.287056+0000","last_undegraded":"2024-04-25T11:16:03.287056+0000","last_fullsized":"2024-04-25T11:16:03.287056+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:20.088218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865834+0000","last_change":"2024-04-25T11:16:02.297639+0000","last_active":"2024-04-25T11:16:03.865834+0000","last_peered":"2024-04-25T11:16:03.865834+0000","last_clean":"2024-04-25T11:16:03.865834+0000","last_became_active":"2024-04-25T11:16:02.297300+0000","last_became_peered":"2024-04-25T11:16:02.297300+0000","last_unstale":"2024-04-25T11:16:03.865834+0000","last_undegraded":"2024-04-25T11:16:03.865834+0000","last_fullsized":"2024-04-25T11:16:03.865834+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:44.760419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.865959+0000","last_change":"2024-04-25T11:16:01.279391+0000","last_active":"2024-04-25T11:16:03.865959+0000","last_peered":"2024-04-25T11:16:03.865959+0000","last_clean":"2024-04-25T11:16:03.865959+0000","last_became_active":"2024-04-25T11:16:01.278789+0000","last_became_peered":"2024-04-25T11:16:01.278789+0000","last_unstale":"2024-04-25T11:16:03.865959+0000","last_undegraded":"2024-04-25T11:16:03.865959+0000","last_fullsized":"2024-04-25T11:16:03.865959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:33.805095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735458+0000","last_change":"2024-04-25T11:16:02.282903+0000","last_active":"2024-04-25T11:16:03.735458+0000","last_peered":"2024-04-25T11:16:03.735458+0000","last_clean":"2024-04-25T11:16:03.735458+0000","last_became_active":"2024-04-25T11:16:02.282559+0000","last_became_peered":"2024-04-25T11:16:02.282559+0000","last_unstale":"2024-04-25T11:16:03.735458+0000","last_undegraded":"2024-04-25T11:16:03.735458+0000","last_fullsized":"2024-04-25T11:16:03.735458+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:20.457810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.259725+0000","last_change":"2024-04-25T11:16:01.274062+0000","last_active":"2024-04-25T11:16:04.259725+0000","last_peered":"2024-04-25T11:16:04.259725+0000","last_clean":"2024-04-25T11:16:04.259725+0000","last_became_active":"2024-04-25T11:16:01.273734+0000","last_became_peered":"2024-04-25T11:16:01.273734+0000","last_unstale":"2024-04-25T11:16:04.259725+0000","last_undegraded":"2024-04-25T11:16:04.259725+0000","last_fullsized":"2024-04-25T11:16:04.259725+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:15.612789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.259935+0000","last_change":"2024-04-25T11:16:02.292488+0000","last_active":"2024-04-25T11:16:04.259935+0000","last_peered":"2024-04-25T11:16:04.259935+0000","last_clean":"2024-04-25T11:16:04.259935+0000","last_became_active":"2024-04-25T11:16:02.292115+0000","last_became_peered":"2024-04-25T11:16:02.292115+0000","last_unstale":"2024-04-25T11:16:04.259935+0000","last_undegraded":"2024-04-25T11:16:04.259935+0000","last_fullsized":"2024-04-25T11:16:04.259935+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:01.519084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282981+0000","last_change":"2024-04-25T11:16:01.265639+0000","last_active":"2024-04-25T11:16:03.282981+0000","last_peered":"2024-04-25T11:16:03.282981+0000","last_clean":"2024-04-25T11:16:03.282981+0000","last_became_active":"2024-04-25T11:16:01.265140+0000","last_became_peered":"2024-04-25T11:16:01.265140+0000","last_unstale":"2024-04-25T11:16:03.282981+0000","last_undegraded":"2024-04-25T11:16:03.282981+0000","last_fullsized":"2024-04-25T11:16:03.282981+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:34.092415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260100+0000","last_change":"2024-04-25T11:16:02.294790+0000","last_active":"2024-04-25T11:16:04.260100+0000","last_peered":"2024-04-25T11:16:04.260100+0000","last_clean":"2024-04-25T11:16:04.260100+0000","last_became_active":"2024-04-25T11:16:02.294373+0000","last_became_peered":"2024-04-25T11:16:02.294373+0000","last_unstale":"2024-04-25T11:16:04.260100+0000","last_undegraded":"2024-04-25T11:16:04.260100+0000","last_fullsized":"2024-04-25T11:16:04.260100+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:18.362912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260145+0000","last_change":"2024-04-25T11:16:01.275344+0000","last_active":"2024-04-25T11:16:04.260145+0000","last_peered":"2024-04-25T11:16:04.260145+0000","last_clean":"2024-04-25T11:16:04.260145+0000","last_became_active":"2024-04-25T11:16:01.274976+0000","last_became_peered":"2024-04-25T11:16:01.274976+0000","last_unstale":"2024-04-25T11:16:04.260145+0000","last_undegraded":"2024-04-25T11:16:04.260145+0000","last_fullsized":"2024-04-25T11:16:04.260145+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:42.262778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736309+0000","last_change":"2024-04-25T11:16:02.278172+0000","last_active":"2024-04-25T11:16:03.736309+0000","last_peered":"2024-04-25T11:16:03.736309+0000","last_clean":"2024-04-25T11:16:03.736309+0000","last_became_active":"2024-04-25T11:16:02.277866+0000","last_became_peered":"2024-04-25T11:16:02.277866+0000","last_unstale":"2024-04-25T11:16:03.736309+0000","last_undegraded":"2024-04-25T11:16:03.736309+0000","last_fullsized":"2024-04-25T11:16:03.736309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:52.045231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282405+0000","last_change":"2024-04-25T11:16:01.268470+0000","last_active":"2024-04-25T11:16:03.282405+0000","last_peered":"2024-04-25T11:16:03.282405+0000","last_clean":"2024-04-25T11:16:03.282405+0000","last_became_active":"2024-04-25T11:16:01.268123+0000","last_became_peered":"2024-04-25T11:16:01.268123+0000","last_unstale":"2024-04-25T11:16:03.282405+0000","last_undegraded":"2024-04-25T11:16:03.282405+0000","last_fullsized":"2024-04-25T11:16:03.282405+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:50.656308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.355093+0000","last_change":"2024-04-25T11:16:02.290944+0000","last_active":"2024-04-25T11:16:04.355093+0000","last_peered":"2024-04-25T11:16:04.355093+0000","last_clean":"2024-04-25T11:16:04.355093+0000","last_became_active":"2024-04-25T11:16:02.287882+0000","last_became_peered":"2024-04-25T11:16:02.287882+0000","last_unstale":"2024-04-25T11:16:04.355093+0000","last_undegraded":"2024-04-25T11:16:04.355093+0000","last_fullsized":"2024-04-25T11:16:04.355093+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:23.157139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286683+0000","last_change":"2024-04-25T11:16:01.276047+0000","last_active":"2024-04-25T11:16:03.286683+0000","last_peered":"2024-04-25T11:16:03.286683+0000","last_clean":"2024-04-25T11:16:03.286683+0000","last_became_active":"2024-04-25T11:16:01.275671+0000","last_became_peered":"2024-04-25T11:16:01.275671+0000","last_unstale":"2024-04-25T11:16:03.286683+0000","last_undegraded":"2024-04-25T11:16:03.286683+0000","last_fullsized":"2024-04-25T11:16:03.286683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:55.507929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.737238+0000","last_change":"2024-04-25T11:16:02.289254+0000","last_active":"2024-04-25T11:16:03.737238+0000","last_peered":"2024-04-25T11:16:03.737238+0000","last_clean":"2024-04-25T11:16:03.737238+0000","last_became_active":"2024-04-25T11:16:02.288919+0000","last_became_peered":"2024-04-25T11:16:02.288919+0000","last_unstale":"2024-04-25T11:16:03.737238+0000","last_undegraded":"2024-04-25T11:16:03.737238+0000","last_fullsized":"2024-04-25T11:16:03.737238+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:04.908247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.866003+0000","last_change":"2024-04-25T11:16:01.283124+0000","last_active":"2024-04-25T11:16:03.866003+0000","last_peered":"2024-04-25T11:16:03.866003+0000","last_clean":"2024-04-25T11:16:03.866003+0000","last_became_active":"2024-04-25T11:16:01.282805+0000","last_became_peered":"2024-04-25T11:16:01.282805+0000","last_unstale":"2024-04-25T11:16:03.866003+0000","last_undegraded":"2024-04-25T11:16:03.866003+0000","last_fullsized":"2024-04-25T11:16:03.866003+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:55.208037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.735937+0000","last_change":"2024-04-25T11:16:02.297866+0000","last_active":"2024-04-25T11:16:03.735937+0000","last_peered":"2024-04-25T11:16:03.735937+0000","last_clean":"2024-04-25T11:16:03.735937+0000","last_became_active":"2024-04-25T11:16:02.297511+0000","last_became_peered":"2024-04-25T11:16:02.297511+0000","last_unstale":"2024-04-25T11:16:03.735937+0000","last_undegraded":"2024-04-25T11:16:03.735937+0000","last_fullsized":"2024-04-25T11:16:03.735937+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:21.707181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.866043+0000","last_change":"2024-04-25T11:16:01.281747+0000","last_active":"2024-04-25T11:16:03.866043+0000","last_peered":"2024-04-25T11:16:03.866043+0000","last_clean":"2024-04-25T11:16:03.866043+0000","last_became_active":"2024-04-25T11:16:01.281422+0000","last_became_peered":"2024-04-25T11:16:01.281422+0000","last_unstale":"2024-04-25T11:16:03.866043+0000","last_undegraded":"2024-04-25T11:16:03.866043+0000","last_fullsized":"2024-04-25T11:16:03.866043+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:02.944477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.260910+0000","last_change":"2024-04-25T11:16:02.289682+0000","last_active":"2024-04-25T11:16:04.260910+0000","last_peered":"2024-04-25T11:16:04.260910+0000","last_clean":"2024-04-25T11:16:04.260910+0000","last_became_active":"2024-04-25T11:16:02.289224+0000","last_became_peered":"2024-04-25T11:16:02.289224+0000","last_unstale":"2024-04-25T11:16:04.260910+0000","last_undegraded":"2024-04-25T11:16:04.260910+0000","last_fullsized":"2024-04-25T11:16:04.260910+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:55.016108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.355309+0000","last_change":"2024-04-25T11:16:01.277618+0000","last_active":"2024-04-25T11:16:04.355309+0000","last_peered":"2024-04-25T11:16:04.355309+0000","last_clean":"2024-04-25T11:16:04.355309+0000","last_became_active":"2024-04-25T11:16:01.277214+0000","last_became_peered":"2024-04-25T11:16:01.277214+0000","last_unstale":"2024-04-25T11:16:04.355309+0000","last_undegraded":"2024-04-25T11:16:04.355309+0000","last_fullsized":"2024-04-25T11:16:04.355309+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:18.118375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.282081+0000","last_change":"2024-04-25T11:16:02.276760+0000","last_active":"2024-04-25T11:16:03.282081+0000","last_peered":"2024-04-25T11:16:03.282081+0000","last_clean":"2024-04-25T11:16:03.282081+0000","last_became_active":"2024-04-25T11:16:02.276428+0000","last_became_peered":"2024-04-25T11:16:02.276428+0000","last_unstale":"2024-04-25T11:16:03.282081+0000","last_undegraded":"2024-04-25T11:16:03.282081+0000","last_fullsized":"2024-04-25T11:16:03.282081+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:23.674905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736562+0000","last_change":"2024-04-25T11:16:01.274998+0000","last_active":"2024-04-25T11:16:03.736562+0000","last_peered":"2024-04-25T11:16:03.736562+0000","last_clean":"2024-04-25T11:16:03.736562+0000","last_became_active":"2024-04-25T11:16:01.274605+0000","last_became_peered":"2024-04-25T11:16:01.274605+0000","last_unstale":"2024-04-25T11:16:03.736562+0000","last_undegraded":"2024-04-25T11:16:03.736562+0000","last_fullsized":"2024-04-25T11:16:03.736562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:31.186403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286893+0000","last_change":"2024-04-25T11:16:02.274185+0000","last_active":"2024-04-25T11:16:03.286893+0000","last_peered":"2024-04-25T11:16:03.286893+0000","last_clean":"2024-04-25T11:16:03.286893+0000","last_became_active":"2024-04-25T11:16:02.273867+0000","last_became_peered":"2024-04-25T11:16:02.273867+0000","last_unstale":"2024-04-25T11:16:03.286893+0000","last_undegraded":"2024-04-25T11:16:03.286893+0000","last_fullsized":"2024-04-25T11:16:03.286893+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:47.098885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.259948+0000","last_change":"2024-04-25T11:16:01.269570+0000","last_active":"2024-04-25T11:16:04.259948+0000","last_peered":"2024-04-25T11:16:04.259948+0000","last_clean":"2024-04-25T11:16:04.259948+0000","last_became_active":"2024-04-25T11:16:01.268117+0000","last_became_peered":"2024-04-25T11:16:01.268117+0000","last_unstale":"2024-04-25T11:16:04.259948+0000","last_undegraded":"2024-04-25T11:16:04.259948+0000","last_fullsized":"2024-04-25T11:16:04.259948+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:54.991159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.355148+0000","last_change":"2024-04-25T11:16:02.285776+0000","last_active":"2024-04-25T11:16:04.355148+0000","last_peered":"2024-04-25T11:16:04.355148+0000","last_clean":"2024-04-25T11:16:04.355148+0000","last_became_active":"2024-04-25T11:16:02.285389+0000","last_became_peered":"2024-04-25T11:16:02.285389+0000","last_unstale":"2024-04-25T11:16:04.355148+0000","last_undegraded":"2024-04-25T11:16:04.355148+0000","last_fullsized":"2024-04-25T11:16:04.355148+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:58.330695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.354891+0000","last_change":"2024-04-25T11:16:01.270160+0000","last_active":"2024-04-25T11:16:04.354891+0000","last_peered":"2024-04-25T11:16:04.354891+0000","last_clean":"2024-04-25T11:16:04.354891+0000","last_became_active":"2024-04-25T11:16:01.269804+0000","last_became_peered":"2024-04-25T11:16:01.269804+0000","last_unstale":"2024-04-25T11:16:04.354891+0000","last_undegraded":"2024-04-25T11:16:04.354891+0000","last_fullsized":"2024-04-25T11:16:04.354891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:02.739967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.736267+0000","last_change":"2024-04-25T11:16:02.293262+0000","last_active":"2024-04-25T11:16:03.736267+0000","last_peered":"2024-04-25T11:16:03.736267+0000","last_clean":"2024-04-25T11:16:03.736267+0000","last_became_active":"2024-04-25T11:16:02.293005+0000","last_became_peered":"2024-04-25T11:16:02.293005+0000","last_unstale":"2024-04-25T11:16:03.736267+0000","last_undegraded":"2024-04-25T11:16:03.736267+0000","last_fullsized":"2024-04-25T11:16:03.736267+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:15.504099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.285817+0000","last_change":"2024-04-25T11:16:01.277681+0000","last_active":"2024-04-25T11:16:03.285817+0000","last_peered":"2024-04-25T11:16:03.285817+0000","last_clean":"2024-04-25T11:16:03.285817+0000","last_became_active":"2024-04-25T11:16:01.277162+0000","last_became_peered":"2024-04-25T11:16:01.277162+0000","last_unstale":"2024-04-25T11:16:03.285817+0000","last_undegraded":"2024-04-25T11:16:03.285817+0000","last_fullsized":"2024-04-25T11:16:03.285817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:01.451953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286851+0000","last_change":"2024-04-25T11:16:02.279810+0000","last_active":"2024-04-25T11:16:03.286851+0000","last_peered":"2024-04-25T11:16:03.286851+0000","last_clean":"2024-04-25T11:16:03.286851+0000","last_became_active":"2024-04-25T11:16:02.279402+0000","last_became_peered":"2024-04-25T11:16:02.279402+0000","last_unstale":"2024-04-25T11:16:03.286851+0000","last_undegraded":"2024-04-25T11:16:03.286851+0000","last_fullsized":"2024-04-25T11:16:03.286851+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:01.219171+0000","last_clean_scrub_stamp":"2024-04-25T11:16:01.219171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:28.752688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:04.168133+0000","last_change":"2024-04-25T11:16:01.274468+0000","last_active":"2024-04-25T11:16:04.168133+0000","last_peered":"2024-04-25T11:16:04.168133+0000","last_clean":"2024-04-25T11:16:04.168133+0000","last_became_active":"2024-04-25T11:16:01.274143+0000","last_became_peered":"2024-04-25T11:16:01.274143+0000","last_unstale":"2024-04-25T11:16:04.168133+0000","last_undegraded":"2024-04-25T11:16:04.168133+0000","last_fullsized":"2024-04-25T11:16:04.168133+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:16:00.202582+0000","last_clean_scrub_stamp":"2024-04-25T11:16:00.202582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:23.017026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":251,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T11:16:03.286293+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:16:03.286293+0000","last_peered":"2024-04-25T11:16:03.286293+0000","last_clean":"2024-04-25T11:16:03.286293+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:16:03.286293+0000","last_undegraded":"2024-04-25T11:16:03.286293+0000","last_fullsized":"2024-04-25T11:16:03.286293+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476809,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1724416,"data_stored":1103272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476810,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":515064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476809,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":512848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476810,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1130496,"data_stored":513008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509513,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":512848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509514,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1126400,"data_stored":512918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509513,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1740800,"data_stored":1105534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542218,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":512848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:16:07.619 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:16:07.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:16:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.956+0000 7f984f000640 1 Processor -- start 2024-04-25T11:16:07.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.956+0000 7f984f000640 1 -- start start 2024-04-25T11:16:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.958+0000 7f984f000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9848153e90 0x7f9848070a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.958+0000 7f984f000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f984806dc90 0x7f98480b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.958+0000 7f984f000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98480b0630 con 0x7f98480b6370 2024-04-25T11:16:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.958+0000 7f984f000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f98480705d0 con 0x7f9848153e90 2024-04-25T11:16:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.958+0000 7f984f000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f98480b08b0 con 0x7f984806dc90 2024-04-25T11:16:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.958+0000 7f984d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f984806dc90 0x7f98480b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.958+0000 7f984d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f984806dc90 0x7f98480b6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:36946/0 (socket says 172.21.15.7:36946) 2024-04-25T11:16:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.958+0000 7f984d000640 1 -- 172.21.15.7:0/719261433 learned_addr learned my addr 172.21.15.7:0/719261433 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.959+0000 7f9847e00640 1 --2- 172.21.15.7:0/719261433 >> v2:172.21.15.87:3300/0 conn(0x7f9848153e90 0x7f9848070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.959+0000 7f984d000640 1 -- 172.21.15.7:0/719261433 >> v2:172.21.15.87:3300/0 conn(0x7f9848153e90 msgr2=0x7f9848070a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.959+0000 7f984d000640 1 --2- 172.21.15.7:0/719261433 >> v2:172.21.15.87:3300/0 conn(0x7f9848153e90 0x7f9848070a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.959+0000 7f984d000640 1 -- 172.21.15.7:0/719261433 >> v1:172.21.15.87:6789/0 conn(0x7f98480b6370 legacy=0x7f984806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.959+0000 7f984d000640 1 -- 172.21.15.7:0/719261433 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98481542a0 con 0x7f984806dc90 2024-04-25T11:16:07.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.961+0000 7f984d000640 1 --2- 172.21.15.7:0/719261433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f984806dc90 0x7f98480b6780 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9840009e50 tx=0x7f984002c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=686e3edd9a0b37e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:07.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.961+0000 7f9846a00640 1 -- 172.21.15.7:0/719261433 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840032610 con 0x7f984806dc90 2024-04-25T11:16:07.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.961+0000 7f9846a00640 1 -- 172.21.15.7:0/719261433 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9840037070 con 0x7f984806dc90 2024-04-25T11:16:07.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.961+0000 7f9846a00640 1 -- 172.21.15.7:0/719261433 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98400329d0 con 0x7f984806dc90 2024-04-25T11:16:07.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.962+0000 7f984f000640 1 -- 172.21.15.7:0/719261433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f984806dc90 msgr2=0x7f98480b6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.962+0000 7f984f000640 1 --2- 172.21.15.7:0/719261433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f984806dc90 0x7f98480b6780 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9840009e50 tx=0x7f984002c920 comp rx=0 tx=0).stop 2024-04-25T11:16:07.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.962+0000 7f984f000640 1 -- 172.21.15.7:0/719261433 shutdown_connections 2024-04-25T11:16:07.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.962+0000 7f984f000640 1 --2- 172.21.15.7:0/719261433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f984806dc90 0x7f98480b6780 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.962+0000 7f984f000640 1 --2- 172.21.15.7:0/719261433 >> v2:172.21.15.87:3300/0 conn(0x7f9848153e90 0x7f9848070a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.962+0000 7f984f000640 1 -- 172.21.15.7:0/719261433 >> 172.21.15.7:0/719261433 conn(0x7f98480d1d40 msgr2=0x7f9848073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:07.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.962+0000 7f984f000640 1 -- 172.21.15.7:0/719261433 shutdown_connections 2024-04-25T11:16:07.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.962+0000 7f984f000640 1 -- 172.21.15.7:0/719261433 wait complete. 2024-04-25T11:16:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.966+0000 7f984f000640 1 Processor -- start 2024-04-25T11:16:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.966+0000 7f984f000640 1 -- start start 2024-04-25T11:16:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f984f000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98480b6370 0x7f9848156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f984f000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9848153e90 0x7f9848142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f984f000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984814c1e0 0x7f984806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f984f000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9848070fa0 con 0x7f98480b6370 2024-04-25T11:16:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f984f000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f98480b0e50 con 0x7f984814c1e0 2024-04-25T11:16:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f984f000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9848071120 con 0x7f9848153e90 2024-04-25T11:16:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f9847e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98480b6370 0x7f9848156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f9847e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98480b6370 0x7f9848156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60758/0 (socket says 172.21.15.7:60758) 2024-04-25T11:16:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f9847e00640 1 -- 172.21.15.7:0/2295275587 learned_addr learned my addr 172.21.15.7:0/2295275587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f9847400640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9848153e90 0x7f9848142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f984d000640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984814c1e0 0x7f984806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f9847e00640 1 -- 172.21.15.7:0/2295275587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9848153e90 msgr2=0x7f9848142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f9847e00640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9848153e90 0x7f9848142030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f9847e00640 1 -- 172.21.15.7:0/2295275587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984814c1e0 msgr2=0x7f984806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f9847e00640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984814c1e0 0x7f984806d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f9847e00640 1 -- 172.21.15.7:0/2295275587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9848159cb0 con 0x7f98480b6370 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.967+0000 7f9847400640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9848153e90 0x7f9848142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.968+0000 7f9847e00640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98480b6370 0x7f9848156cb0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f983800e870 tx=0x7f983800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.968+0000 7f983fe00640 1 -- 172.21.15.7:0/2295275587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838017070 con 0x7f98480b6370 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.968+0000 7f983fe00640 1 -- 172.21.15.7:0/2295275587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983800c940 con 0x7f98480b6370 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.968+0000 7f984f000640 1 -- 172.21.15.7:0/2295275587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9848159f90 con 0x7f98480b6370 2024-04-25T11:16:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.968+0000 7f983fe00640 1 -- 172.21.15.7:0/2295275587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983800cd30 con 0x7f98480b6370 2024-04-25T11:16:07.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.968+0000 7f984f000640 1 -- 172.21.15.7:0/2295275587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984815a550 con 0x7f98480b6370 2024-04-25T11:16:07.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.969+0000 7f984f000640 1 -- 172.21.15.7:0/2295275587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9848050740 con 0x7f98480b6370 2024-04-25T11:16:07.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.972+0000 7f983fe00640 1 -- 172.21.15.7:0/2295275587 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f983801a070 con 0x7f98480b6370 2024-04-25T11:16:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.973+0000 7f983fe00640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f98280744b0 0x7f98280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.973+0000 7f983fe00640 1 -- 172.21.15.7:0/2295275587 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f9838095680 con 0x7f98480b6370 2024-04-25T11:16:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.973+0000 7f9847400640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f98280744b0 0x7f98280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:07.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.973+0000 7f9847400640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f98280744b0 0x7f98280769c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f984814ff20 tx=0x7f9834009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:07.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:07.977+0000 7f983fe00640 1 -- 172.21.15.7:0/2295275587 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f983805fc70 con 0x7f98480b6370 2024-04-25T11:16:08.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.255+0000 7f984f000640 1 -- 172.21.15.7:0/2295275587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f984815b320 con 0x7f98480b6370 2024-04-25T11:16:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.262+0000 7f983fe00640 1 -- 172.21.15.7:0/2295275587 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f983805f570 con 0x7f98480b6370 2024-04-25T11:16:08.262 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-25T11:16:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:16:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-04-25T11:16:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-04-25T11:16:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:16:08.263 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-04-25T11:16:08.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 -- 172.21.15.7:0/2295275587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f98280744b0 msgr2=0x7f98280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:08.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f98280744b0 0x7f98280769c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f984814ff20 tx=0x7f9834009040 comp rx=0 tx=0).stop 2024-04-25T11:16:08.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 -- 172.21.15.7:0/2295275587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98480b6370 msgr2=0x7f9848156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:08.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98480b6370 0x7f9848156cb0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f983800e870 tx=0x7f983800ed30 comp rx=0 tx=0).stop 2024-04-25T11:16:08.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 -- 172.21.15.7:0/2295275587 shutdown_connections 2024-04-25T11:16:08.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f98280744b0 0x7f98280769c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984814c1e0 0x7f984806d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9848153e90 0x7f9848142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 --2- 172.21.15.7:0/2295275587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98480b6370 0x7f9848156cb0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 -- 172.21.15.7:0/2295275587 >> 172.21.15.7:0/2295275587 conn(0x7f98480d1d40 msgr2=0x7f984815b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:08.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 -- 172.21.15.7:0/2295275587 shutdown_connections 2024-04-25T11:16:08.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.265+0000 7f984f000640 1 -- 172.21.15.7:0/2295275587 wait complete. 2024-04-25T11:16:08.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:16:08.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.578+0000 7efe89200640 1 Processor -- start 2024-04-25T11:16:08.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.578+0000 7efe89200640 1 -- start start 2024-04-25T11:16:08.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.581+0000 7efe89200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe84070a70 0x7efe840ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.581+0000 7efe89200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efe840b0be0 0x7efe840b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.581+0000 7efe89200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe8415e190 con 0x7efe840aedc0 2024-04-25T11:16:08.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.581+0000 7efe89200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efe8415d610 con 0x7efe840b0be0 2024-04-25T11:16:08.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.581+0000 7efe89200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe8415d8f0 con 0x7efe84070a70 2024-04-25T11:16:08.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.581+0000 7efe7aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efe840aedc0 0x7efe840b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59526/0 (socket says 172.21.15.7:59526) 2024-04-25T11:16:08.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.581+0000 7efe7aa00640 1 -- 172.21.15.7:0/2497025094 learned_addr learned my addr 172.21.15.7:0/2497025094 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe83e00640 1 --2- 172.21.15.7:0/2497025094 >> v2:172.21.15.87:3300/0 conn(0x7efe840b0be0 0x7efe840b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe83400640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe84070a70 0x7efe840ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 876615483 0 0) 0x7efe8415e190 con 0x7efe840aedc0 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe68005730 con 0x7efe840aedc0 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3089371426 0 0) 0x7efe68005730 con 0x7efe840aedc0 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe84070a70 msgr2=0x7efe840ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe84070a70 0x7efe840ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 >> v2:172.21.15.87:3300/0 conn(0x7efe840b0be0 msgr2=0x7efe840b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 --2- 172.21.15.7:0/2497025094 >> v2:172.21.15.87:3300/0 conn(0x7efe840b0be0 0x7efe840b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8415dbd0 con 0x7efe840aedc0 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7efe6c001670 con 0x7efe840aedc0 2024-04-25T11:16:08.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe83e00640 1 --2- 172.21.15.7:0/2497025094 >> v2:172.21.15.87:3300/0 conn(0x7efe840b0be0 0x7efe840b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:16:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 >> v1:172.21.15.7:6789/0 conn(0x7efe840aedc0 legacy=0x7efe840b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe68005bc0 0x7efe68006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe84070a70 0x7efe68006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.582+0000 7efe82a00640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe68006f50 0x7efe68007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.583+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe8415d610 con 0x7efe68006f50 2024-04-25T11:16:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.583+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efe8415d8f0 con 0x7efe68005bc0 2024-04-25T11:16:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.583+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe6800dc00 con 0x7efe84070a70 2024-04-25T11:16:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.583+0000 7efe83e00640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe68006f50 0x7efe68007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.583+0000 7efe83400640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe68005bc0 0x7efe68006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.583+0000 7efe7aa00640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe84070a70 0x7efe68006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.583+0000 7efe83e00640 1 -- 172.21.15.7:0/2497025094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe84070a70 msgr2=0x7efe68006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.583+0000 7efe83e00640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe84070a70 0x7efe68006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.583+0000 7efe83e00640 1 -- 172.21.15.7:0/2497025094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe68005bc0 msgr2=0x7efe68006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.583+0000 7efe83e00640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe68005bc0 0x7efe68006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.583+0000 7efe83e00640 1 -- 172.21.15.7:0/2497025094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe68005730 con 0x7efe68006f50 2024-04-25T11:16:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.585+0000 7efe83e00640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe68006f50 0x7efe68007360 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009c90 tx=0x7efe68007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d86a4bf0f76934 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.585+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c036070 con 0x7efe68006f50 2024-04-25T11:16:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.585+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c031400 con 0x7efe68006f50 2024-04-25T11:16:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.585+0000 7efe82a00640 1 -- 172.21.15.7:0/2497025094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c009860 con 0x7efe68006f50 2024-04-25T11:16:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.586+0000 7efe89200640 1 -- 172.21.15.7:0/2497025094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe68006f50 msgr2=0x7efe68007360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.586+0000 7efe89200640 1 --2- 172.21.15.7:0/2497025094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe68006f50 0x7efe68007360 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009c90 tx=0x7efe68007a40 comp rx=0 tx=0).stop 2024-04-25T11:16:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.586+0000 7efe83e00640 1 -- 172.21.15.7:0/2497025094 reap_dead start 2024-04-25T11:16:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.586+0000 7efe89200640 1 -- 172.21.15.7:0/2497025094 shutdown_connections 2024-04-25T11:16:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.586+0000 7efe89200640 1 -- 172.21.15.7:0/2497025094 >> 172.21.15.7:0/2497025094 conn(0x7efe840b3820 msgr2=0x7efe840b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.586+0000 7efe89200640 1 -- 172.21.15.7:0/2497025094 shutdown_connections 2024-04-25T11:16:08.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.586+0000 7efe89200640 1 -- 172.21.15.7:0/2497025094 wait complete. 2024-04-25T11:16:08.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.588+0000 7efe89200640 1 Processor -- start 2024-04-25T11:16:08.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.588+0000 7efe89200640 1 -- start start 2024-04-25T11:16:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.590+0000 7efe89200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe68006f50 0x7efe840ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.590+0000 7efe89200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe840d0590 0x7efe8415f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.590+0000 7efe89200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe8415f850 0x7efe8415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.590+0000 7efe89200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe8415d2f0 con 0x7efe840d0590 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.590+0000 7efe89200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efe8415d170 con 0x7efe8415f850 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.590+0000 7efe89200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe840ae880 con 0x7efe68006f50 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.590+0000 7efe7aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe840d0590 0x7efe8415f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.590+0000 7efe7aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe840d0590 0x7efe8415f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60768/0 (socket says 172.21.15.7:60768) 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.590+0000 7efe7aa00640 1 -- 172.21.15.7:0/2892388395 learned_addr learned my addr 172.21.15.7:0/2892388395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe83e00640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe8415f850 0x7efe8415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe83400640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe68006f50 0x7efe840ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe7aa00640 1 -- 172.21.15.7:0/2892388395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe68006f50 msgr2=0x7efe840ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe7aa00640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe68006f50 0x7efe840ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe7aa00640 1 -- 172.21.15.7:0/2892388395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe8415f850 msgr2=0x7efe8415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe7aa00640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe8415f850 0x7efe8415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe7aa00640 1 -- 172.21.15.7:0/2892388395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe840cea10 con 0x7efe840d0590 2024-04-25T11:16:08.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe83400640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe68006f50 0x7efe840ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:08.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe7aa00640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe840d0590 0x7efe8415f310 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00e730 tx=0x7efe6c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:08.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe7be00640 1 -- 172.21.15.7:0/2892388395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c00c6c0 con 0x7efe840d0590 2024-04-25T11:16:08.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe7be00640 1 -- 172.21.15.7:0/2892388395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c018070 con 0x7efe840d0590 2024-04-25T11:16:08.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.591+0000 7efe89200640 1 -- 172.21.15.7:0/2892388395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe840cecf0 con 0x7efe840d0590 2024-04-25T11:16:08.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.592+0000 7efe7be00640 1 -- 172.21.15.7:0/2892388395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c00caa0 con 0x7efe840d0590 2024-04-25T11:16:08.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.592+0000 7efe89200640 1 -- 172.21.15.7:0/2892388395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe840cf2b0 con 0x7efe840d0590 2024-04-25T11:16:08.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.593+0000 7efe89200640 1 -- 172.21.15.7:0/2892388395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe84050740 con 0x7efe840d0590 2024-04-25T11:16:08.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.594+0000 7efe7be00640 1 -- 172.21.15.7:0/2892388395 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efe6c00cc40 con 0x7efe840d0590 2024-04-25T11:16:08.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.595+0000 7efe7be00640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe600744e0 0x7efe600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:08.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.595+0000 7efe7be00640 1 -- 172.21.15.7:0/2892388395 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7efe6c0951f0 con 0x7efe840d0590 2024-04-25T11:16:08.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.595+0000 7efe83400640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe600744e0 0x7efe600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:08.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.596+0000 7efe83400640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe600744e0 0x7efe600769f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efe74007440 tx=0x7efe7400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:08.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.600+0000 7efe7be00640 1 -- 172.21.15.7:0/2892388395 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe6c05f7e0 con 0x7efe840d0590 2024-04-25T11:16:08.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.841+0000 7efe89200640 1 -- 172.21.15.7:0/2892388395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe840ce7a0 con 0x7efe840d0590 2024-04-25T11:16:08.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.842+0000 7efe7be00640 1 -- 172.21.15.7:0/2892388395 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5168 (secure 0 0 0) 0x7efe6c05f0e0 con 0x7efe840d0590 2024-04-25T11:16:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:08.843 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/684015857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":684015857},{"type":"v1","addr":"172.21.15.7:6836","nonce":684015857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/493684223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":493684223},{"type":"v1","addr":"172.21.15.87:6835","nonce":493684223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6109,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/1959070928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1959070928},{"type":"v1","addr":"172.21.15.87:6833","nonce":1959070928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:16:01.820362+0000","modified":"2024-04-25T11:16:04.870476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/336889324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":336889324},{"type":"v1","addr":"172.21.15.7:6837","nonce":336889324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6960,"qdb_cluster":[6960]},"id":6}]} 2024-04-25T11:16:08.843 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2024-04-25T11:16:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 -- 172.21.15.7:0/2892388395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe600744e0 msgr2=0x7efe600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe600744e0 0x7efe600769f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efe74007440 tx=0x7efe7400b040 comp rx=0 tx=0).stop 2024-04-25T11:16:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 -- 172.21.15.7:0/2892388395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe840d0590 msgr2=0x7efe8415f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe840d0590 0x7efe8415f310 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00e730 tx=0x7efe6c00ebf0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 -- 172.21.15.7:0/2892388395 shutdown_connections 2024-04-25T11:16:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe600744e0 0x7efe600769f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe8415f850 0x7efe8415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe840d0590 0x7efe8415f310 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 --2- 172.21.15.7:0/2892388395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe68006f50 0x7efe840ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 -- 172.21.15.7:0/2892388395 >> 172.21.15.7:0/2892388395 conn(0x7efe840b3820 msgr2=0x7efe840b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 -- 172.21.15.7:0/2892388395 shutdown_connections 2024-04-25T11:16:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:08.845+0000 7efe89200640 1 -- 172.21.15.7:0/2892388395 wait complete. 2024-04-25T11:16:09.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:16:01.820362+0000', 'modified': '2024-04-25T11:16:04.870476+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6960}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6960': {'gid': 6960, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6837/336889324', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6835', 'nonce': 336889324}, {'type': 'v1', 'addr': '172.21.15.7:6837', 'nonce': 336889324}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6960, 'qdb_cluster': [6960]} 2024-04-25T11:16:09.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:16:09.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:16:09.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.572+0000 7f65e5e00640 1 Processor -- start 2024-04-25T11:16:09.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.572+0000 7f65e5e00640 1 -- start start 2024-04-25T11:16:09.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.574+0000 7f65e5e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0156d90 0x7f65e0160460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.575+0000 7f65e5e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e006f760 con 0x7f65e01609a0 2024-04-25T11:16:09.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.575+0000 7f65e5e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65e006fa40 con 0x7f65e0070520 2024-04-25T11:16:09.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.575+0000 7f65e5e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65e006fd20 con 0x7f65e0156d90 2024-04-25T11:16:09.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.575+0000 7f65e5400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f65e01609a0 0x7f65e0160db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59534/0 (socket says 172.21.15.7:59534) 2024-04-25T11:16:09.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.575+0000 7f65e5400640 1 -- 172.21.15.7:0/3462650554 learned_addr learned my addr 172.21.15.7:0/3462650554 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:09.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.575+0000 7f65dfe00640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0156d90 0x7f65e0160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.575+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1840206334 0 0) 0x7f65e006f760 con 0x7f65e01609a0 2024-04-25T11:16:09.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.575+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65c8003520 con 0x7f65e01609a0 2024-04-25T11:16:09.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 272183118 0 0) 0x7f65c8003520 con 0x7f65e01609a0 2024-04-25T11:16:09.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0156d90 msgr2=0x7f65e0160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0156d90 0x7f65e0160460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 >> v1:172.21.15.87:6789/0 conn(0x7f65e0070520 legacy=0x7f65e00b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e0070000 con 0x7f65e01609a0 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f65d80019f0 con 0x7f65e01609a0 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 >> v1:172.21.15.7:6789/0 conn(0x7f65e01609a0 legacy=0x7f65e0160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0156d90 0x7f65c8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c8003f80 0x7f65c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c8004d40 0x7f65c8005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65e006fd20 con 0x7f65c8003f80 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65c800bc00 con 0x7f65c8004d40 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65c800be00 con 0x7f65e0156d90 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.576+0000 7f65dfe00640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c8003f80 0x7f65c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.577+0000 7f65e5400640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c8004d40 0x7f65c8005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.577+0000 7f65e4a00640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0156d90 0x7f65c8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.577+0000 7f65dfe00640 1 -- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0156d90 msgr2=0x7f65c8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.577+0000 7f65dfe00640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0156d90 0x7f65c8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.577+0000 7f65dfe00640 1 -- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c8004d40 msgr2=0x7f65c8005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.577+0000 7f65dfe00640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65c8004d40 0x7f65c8005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.577+0000 7f65dfe00640 1 -- 172.21.15.7:0/3462650554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c8003520 con 0x7f65c8003f80 2024-04-25T11:16:09.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.577+0000 7f65e4a00640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0156d90 0x7f65c8003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:09.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.578+0000 7f65dfe00640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c8003f80 0x7f65c8004800 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009ac0 tx=0x7f65c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2803ce9283cf5a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:09.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.578+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc036070 con 0x7f65c8003f80 2024-04-25T11:16:09.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.578+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65cc031400 con 0x7f65c8003f80 2024-04-25T11:16:09.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.578+0000 7f65df400640 1 -- 172.21.15.7:0/3462650554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc009690 con 0x7f65c8003f80 2024-04-25T11:16:09.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.579+0000 7f65e5e00640 1 -- 172.21.15.7:0/3462650554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c8003f80 msgr2=0x7f65c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.579+0000 7f65e5e00640 1 --2- 172.21.15.7:0/3462650554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65c8003f80 0x7f65c8004800 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009ac0 tx=0x7f65c8005830 comp rx=0 tx=0).stop 2024-04-25T11:16:09.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.579+0000 7f65e5400640 1 -- 172.21.15.7:0/3462650554 reap_dead start 2024-04-25T11:16:09.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.579+0000 7f65e5e00640 1 -- 172.21.15.7:0/3462650554 shutdown_connections 2024-04-25T11:16:09.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.579+0000 7f65e5e00640 1 -- 172.21.15.7:0/3462650554 >> 172.21.15.7:0/3462650554 conn(0x7f65e00d1540 msgr2=0x7f65e00d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:09.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.579+0000 7f65e5e00640 1 -- 172.21.15.7:0/3462650554 shutdown_connections 2024-04-25T11:16:09.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.579+0000 7f65e5e00640 1 -- 172.21.15.7:0/3462650554 wait complete. 2024-04-25T11:16:09.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.582+0000 7f65e5e00640 1 Processor -- start 2024-04-25T11:16:09.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.582+0000 7f65e5e00640 1 -- start start 2024-04-25T11:16:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.584+0000 7f65e5e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65c8003f80 0x7f65e015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.584+0000 7f65e5e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00b6280 0x7f65e0155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.584+0000 7f65e5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0155b90 0x7f65e01614e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.584+0000 7f65e5e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65e0157320 con 0x7f65e0155b90 2024-04-25T11:16:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.584+0000 7f65e5e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65e01571a0 con 0x7f65e00b6280 2024-04-25T11:16:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.584+0000 7f65e5e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65e0160460 con 0x7f65c8003f80 2024-04-25T11:16:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.584+0000 7f65e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0155b90 0x7f65e01614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.584+0000 7f65e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0155b90 0x7f65e01614e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60790/0 (socket says 172.21.15.7:60790) 2024-04-25T11:16:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.584+0000 7f65e5400640 1 -- 172.21.15.7:0/2608640834 learned_addr learned my addr 172.21.15.7:0/2608640834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.584+0000 7f65e4a00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65c8003f80 0x7f65e015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65dfe00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00b6280 0x7f65e0155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65e5400640 1 -- 172.21.15.7:0/2608640834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65c8003f80 msgr2=0x7f65e015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65e5400640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65c8003f80 0x7f65e015a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65e5400640 1 -- 172.21.15.7:0/2608640834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00b6280 msgr2=0x7f65e0155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65e5400640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00b6280 0x7f65e0155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65e5400640 1 -- 172.21.15.7:0/2608640834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e0155fa0 con 0x7f65e0155b90 2024-04-25T11:16:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65e4a00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65c8003f80 0x7f65e015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65e5400640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0155b90 0x7f65e01614e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f65e0074c70 tx=0x7f65d800e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65dcc00640 1 -- 172.21.15.7:0/2608640834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d800eb70 con 0x7f65e0155b90 2024-04-25T11:16:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65dcc00640 1 -- 172.21.15.7:0/2608640834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d800ed30 con 0x7f65e0155b90 2024-04-25T11:16:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65e5e00640 1 -- 172.21.15.7:0/2608640834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e00b4750 con 0x7f65e0155b90 2024-04-25T11:16:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65dcc00640 1 -- 172.21.15.7:0/2608640834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d80146c0 con 0x7f65e0155b90 2024-04-25T11:16:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.585+0000 7f65e5e00640 1 -- 172.21.15.7:0/2608640834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e00b4d10 con 0x7f65e0155b90 2024-04-25T11:16:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.589+0000 7f65dcc00640 1 -- 172.21.15.7:0/2608640834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f65d8014860 con 0x7f65e0155b90 2024-04-25T11:16:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.590+0000 7f65dcc00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65c00744b0 0x7f65c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.590+0000 7f65dcc00640 1 -- 172.21.15.7:0/2608640834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f65d8095410 con 0x7f65e0155b90 2024-04-25T11:16:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.590+0000 7f65ee37b740 1 Processor -- start 2024-04-25T11:16:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.590+0000 7f65ee37b740 1 -- start start 2024-04-25T11:16:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.590+0000 7f65e4a00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65c00744b0 0x7f65c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.591+0000 7f65e4a00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65c00744b0 0x7f65c00769c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f65d40016f0 tx=0x7f65d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0db43b0 0x5578c11c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c11c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65ee37b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5578c10e3600 con 0x5578c0ddd270 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65ee37b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5578c1080100 con 0x5578c0dbc590 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65ee37b740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5578c10df5c0 con 0x5578c0db43b0 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65e4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65e4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11aa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60802/0 (socket says 172.21.15.7:60802) 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65e4a00640 1 -- 172.21.15.7:0/2188290006 learned_addr learned my addr 172.21.15.7:0/2188290006 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65e4a00640 1 -- 172.21.15.7:0/2188290006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0db43b0 msgr2=0x5578c11c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65e4a00640 1 --2- 172.21.15.7:0/2188290006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0db43b0 0x5578c11c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65e4a00640 1 -- 172.21.15.7:0/2188290006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 msgr2=0x5578c11c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65e4a00640 1 --2- 172.21.15.7:0/2188290006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c11c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.592+0000 7f65e4a00640 1 -- 172.21.15.7:0/2188290006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578c0df52d0 con 0x5578c0ddd270 2024-04-25T11:16:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.593+0000 7f65e4a00640 1 --2- 172.21.15.7:0/2188290006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11aa210 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f65d400ebc0 tx=0x7f65d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3544925afcb7993e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:09.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.593+0000 7f65d1200640 1 -- 172.21.15.7:0/2188290006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d401a050 con 0x5578c0ddd270 2024-04-25T11:16:09.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.593+0000 7f65d1200640 1 -- 172.21.15.7:0/2188290006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d4024070 con 0x5578c0ddd270 2024-04-25T11:16:09.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.593+0000 7f65d1200640 1 -- 172.21.15.7:0/2188290006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4013dc0 con 0x5578c0ddd270 2024-04-25T11:16:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.594+0000 7f65ee37b740 1 -- 172.21.15.7:0/2188290006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 msgr2=0x5578c11aa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.594+0000 7f65ee37b740 1 --2- 172.21.15.7:0/2188290006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11aa210 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f65d400ebc0 tx=0x7f65d4011000 comp rx=0 tx=0).stop 2024-04-25T11:16:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.594+0000 7f65ee37b740 1 -- 172.21.15.7:0/2188290006 shutdown_connections 2024-04-25T11:16:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.594+0000 7f65ee37b740 1 --2- 172.21.15.7:0/2188290006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c11c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.594+0000 7f65ee37b740 1 --2- 172.21.15.7:0/2188290006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0db43b0 0x5578c11c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.594+0000 7f65ee37b740 1 --2- 172.21.15.7:0/2188290006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11aa210 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.594+0000 7f65ee37b740 1 -- 172.21.15.7:0/2188290006 >> 172.21.15.7:0/2188290006 conn(0x5578c0dd13c0 msgr2=0x5578c117f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.594+0000 7f65ee37b740 1 -- 172.21.15.7:0/2188290006 shutdown_connections 2024-04-25T11:16:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.594+0000 7f65ee37b740 1 -- 172.21.15.7:0/2188290006 wait complete. 2024-04-25T11:16:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.595+0000 7f65ee37b740 1 -- start start 2024-04-25T11:16:09.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.595+0000 7f65ee37b740 1 Processor -- start 2024-04-25T11:16:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0db43b0 0x5578c11bfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c11ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11be650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65ee37b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5578c106af70 con 0x5578c0ddd270 2024-04-25T11:16:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65ee37b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5578c11abc60 con 0x5578c0dbc590 2024-04-25T11:16:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65ee37b740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5578c11bd360 con 0x5578c0db43b0 2024-04-25T11:16:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65ee37b740 10 client.0 _ll_register_callbacks cb 0x5578c10d9480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:16:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11be650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11be650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60806/0 (socket says 172.21.15.7:60806) 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65e5400640 1 -- 172.21.15.7:0/990691515 learned_addr learned my addr 172.21.15.7:0/990691515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65dfe00640 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c11ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.597+0000 7f65e4a00640 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0db43b0 0x5578c11bfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65e5400640 1 -- 172.21.15.7:0/990691515 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0db43b0 msgr2=0x5578c11bfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65e5400640 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0db43b0 0x5578c11bfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65e5400640 1 -- 172.21.15.7:0/990691515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 msgr2=0x5578c11ab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65e5400640 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c11ab3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65e5400640 1 -- 172.21.15.7:0/990691515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578c0ebf6b0 con 0x5578c0ddd270 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65dfe00640 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c11ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65e5400640 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11be650 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f65d805f200 tx=0x7f65d805b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5327aab7560a340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65c5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65c5600640 1 -- 172.21.15.7:0/990691515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8063380 con 0x5578c0ddd270 2024-04-25T11:16:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65c5600640 1 -- 172.21.15.7:0/990691515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f65d8093b70 con 0x5578c0ddd270 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65ee37b740 1 -- 172.21.15.7:0/990691515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65d4024070 con 0x5578c0ddd270 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65c5600640 1 -- 172.21.15.7:0/990691515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f65d8091050 con 0x5578c0ddd270 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65ee37b740 10 client.7089 fetch_fsmap learned FSMap version 52 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.598+0000 7f65ee37b740 1 -- 172.21.15.7:0/990691515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5578c11beb90 con 0x5578c0ddd270 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.599+0000 7f65c5600640 1 -- 172.21.15.7:0/990691515 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d4024070 con 0x5578c0ddd270 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.599+0000 7f65c5600640 1 -- 172.21.15.7:0/990691515 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f65d80932f0 con 0x5578c0ddd270 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.599+0000 7f65ee37b740 10 client.7089 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.599+0000 7f65ee37b740 10 client.7089 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.599+0000 7f65ee37b740 20 client.7089 populate_metadata read hostname 'smithi007' 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.599+0000 7f65ee37b740 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.7:6835/336889324,v1:172.21.15.7:6837/336889324] conn(0x5578c0d9be40 0x5578c11bc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.599+0000 7f65ee37b740 4 client.7089 mds_command: new command op to 6960 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.599+0000 7f65ee37b740 1 -- 172.21.15.7:0/990691515 --> [v2:172.21.15.7:6835/336889324,v1:172.21.15.7:6837/336889324] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578c10f1210 con 0x5578c0d9be40 2024-04-25T11:16:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.599+0000 7f65e4a00640 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.7:6835/336889324,v1:172.21.15.7:6837/336889324] conn(0x5578c0d9be40 0x5578c11bc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.600+0000 7f65e4a00640 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.7:6835/336889324,v1:172.21.15.7:6837/336889324] conn(0x5578c0d9be40 0x5578c11bc920 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4179bdb6b9468403 server_cookie=13170edad144b18a in_seq=0 out_seq=0 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.600+0000 7f65c5600640 10 client.7089 ms_handle_connect on v2:172.21.15.7:6835/336889324 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65c5600640 1 -- 172.21.15.7:0/990691515 <== mds.0 v2:172.21.15.7:6835/336889324 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578c10f1210 con 0x5578c0d9be40 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65c5600640 10 client.7089 handle_command_reply: tid=0 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65ee37b740 1 client.7089 shutdown 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65ee37b740 1 -- 172.21.15.7:0/990691515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 msgr2=0x5578c11be650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65ee37b740 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11be650 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f65d805f200 tx=0x7f65d805b7b0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65ee37b740 1 -- 172.21.15.7:0/990691515 shutdown_connections 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65ee37b740 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11be650 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65ee37b740 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c11ab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65ee37b740 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.7:6835/336889324,v1:172.21.15.7:6837/336889324] conn(0x5578c0d9be40 0x5578c11bc920 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65ee37b740 1 --2- 172.21.15.7:0/990691515 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0db43b0 0x5578c11bfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65ee37b740 1 -- 172.21.15.7:0/990691515 >> 172.21.15.7:0/990691515 conn(0x5578c0dd13c0 msgr2=0x5578c117f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65c5600640 0 client.7089 ms_handle_reset on v2:172.21.15.7:6835/336889324 2024-04-25T11:16:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65ee37b740 1 -- 172.21.15.7:0/990691515 shutdown_connections 2024-04-25T11:16:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.601+0000 7f65ee37b740 1 -- 172.21.15.7:0/990691515 wait complete. 2024-04-25T11:16:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.602+0000 7f65ee37b740 20 client.7089 trim_cache size 0 max 16384 2024-04-25T11:16:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.622+0000 7f65ee37b740 1 Processor -- start 2024-04-25T11:16:09.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.622+0000 7f65ee37b740 1 -- start start 2024-04-25T11:16:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.623+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0db43b0 0x5578c11c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0dbc590 0x5578c11e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11e8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65ee37b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65e02024f0 con 0x5578c0ddd270 2024-04-25T11:16:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65ee37b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65e01590a0 con 0x5578c0db43b0 2024-04-25T11:16:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65ee37b740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65e0201fd0 con 0x5578c0dbc590 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11e8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65e5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11e8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60808/0 (socket says 172.21.15.7:60808) 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65e5400640 1 -- 172.21.15.7:0/2358416662 learned_addr learned my addr 172.21.15.7:0/2358416662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65e4a00640 1 --2- 172.21.15.7:0/2358416662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0db43b0 0x5578c11c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65dfe00640 1 --2- 172.21.15.7:0/2358416662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0dbc590 0x5578c11e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65e5400640 1 -- 172.21.15.7:0/2358416662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0dbc590 msgr2=0x5578c11e7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65e5400640 1 --2- 172.21.15.7:0/2358416662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0dbc590 0x5578c11e7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65e5400640 1 -- 172.21.15.7:0/2358416662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0db43b0 msgr2=0x5578c11c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65e5400640 1 --2- 172.21.15.7:0/2358416662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0db43b0 0x5578c11c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65e5400640 1 -- 172.21.15.7:0/2358416662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578c11c1420 con 0x5578c0ddd270 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.624+0000 7f65dfe00640 1 --2- 172.21.15.7:0/2358416662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0dbc590 0x5578c11e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.625+0000 7f65e5400640 1 --2- 172.21.15.7:0/2358416662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11e8180 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f65d805ef70 tx=0x7f65d8062000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bb9c78c750a531 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.625+0000 7f65c7e00640 1 -- 172.21.15.7:0/2358416662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d805ed80 con 0x5578c0ddd270 2024-04-25T11:16:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.625+0000 7f65c7e00640 1 -- 172.21.15.7:0/2358416662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d8063620 con 0x5578c0ddd270 2024-04-25T11:16:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.625+0000 7f65c7e00640 1 -- 172.21.15.7:0/2358416662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8063380 con 0x5578c0ddd270 2024-04-25T11:16:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.626+0000 7f65ee37b740 1 -- 172.21.15.7:0/2358416662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 msgr2=0x5578c11e8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.626+0000 7f65ee37b740 1 --2- 172.21.15.7:0/2358416662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11e8180 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f65d805ef70 tx=0x7f65d8062000 comp rx=0 tx=0).stop 2024-04-25T11:16:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.626+0000 7f65ee37b740 1 -- 172.21.15.7:0/2358416662 shutdown_connections 2024-04-25T11:16:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.626+0000 7f65ee37b740 1 --2- 172.21.15.7:0/2358416662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0ddd270 0x5578c11e8180 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.626+0000 7f65ee37b740 1 --2- 172.21.15.7:0/2358416662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0dbc590 0x5578c11e7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.626+0000 7f65ee37b740 1 --2- 172.21.15.7:0/2358416662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0db43b0 0x5578c11c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.626+0000 7f65ee37b740 1 -- 172.21.15.7:0/2358416662 >> 172.21.15.7:0/2358416662 conn(0x5578c0dd13c0 msgr2=0x5578c11aa210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.626+0000 7f65ee37b740 1 -- 172.21.15.7:0/2358416662 shutdown_connections 2024-04-25T11:16:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.626+0000 7f65ee37b740 1 -- 172.21.15.7:0/2358416662 wait complete. 2024-04-25T11:16:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.627+0000 7f65ee37b740 1 -- start start 2024-04-25T11:16:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.627+0000 7f65ee37b740 1 Processor -- start 2024-04-25T11:16:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0db43b0 0x5578c1179c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c1241980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65ee37b740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0ddd270 0x5578c11c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65ee37b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5578c11c4080 con 0x5578c0db43b0 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65ee37b740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5578c11bee80 con 0x5578c0dbc590 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65ee37b740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5578c11c4640 con 0x5578c0ddd270 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65ee37b740 10 client.0 _ll_register_callbacks cb 0x7f65d800eee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65e4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0db43b0 0x5578c1179c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65e4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0db43b0 0x5578c1179c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60818/0 (socket says 172.21.15.7:60818) 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65e4a00640 1 -- 172.21.15.7:0/1060489307 learned_addr learned my addr 172.21.15.7:0/1060489307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65e4a00640 1 -- 172.21.15.7:0/1060489307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0ddd270 msgr2=0x5578c11c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65e4a00640 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0ddd270 0x5578c11c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65e4a00640 1 -- 172.21.15.7:0/1060489307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 msgr2=0x5578c1241980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65dfe00640 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c1241980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65e4a00640 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c1241980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.629+0000 7f65e4a00640 1 -- 172.21.15.7:0/1060489307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578c1174fb0 con 0x5578c0db43b0 2024-04-25T11:16:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.630+0000 7f65e4a00640 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0db43b0 0x5578c1179c10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009010 tx=0x7f65d4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf32dee84cdb75b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.630+0000 7f65c5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.630+0000 7f65c5600640 1 -- 172.21.15.7:0/1060489307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d401a040 con 0x5578c0db43b0 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.630+0000 7f65c5600640 1 -- 172.21.15.7:0/1060489307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f65d4013dd0 con 0x5578c0db43b0 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.630+0000 7f65ee37b740 1 -- 172.21.15.7:0/1060489307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65d8063620 con 0x5578c0db43b0 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.630+0000 7f65c5600640 1 -- 172.21.15.7:0/1060489307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f65d4044050 con 0x5578c0db43b0 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.630+0000 7f65ee37b740 10 client.7095 fetch_fsmap learned FSMap version 52 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.630+0000 7f65ee37b740 1 -- 172.21.15.7:0/1060489307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5578c11bb8e0 con 0x5578c0db43b0 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.631+0000 7f65c5600640 1 -- 172.21.15.7:0/1060489307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d8063620 con 0x5578c0db43b0 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.631+0000 7f65c5600640 1 -- 172.21.15.7:0/1060489307 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f65d4047020 con 0x5578c0db43b0 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.631+0000 7f65ee37b740 10 client.7095 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.631+0000 7f65ee37b740 10 client.7095 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.631+0000 7f65ee37b740 20 client.7095 populate_metadata read hostname 'smithi007' 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.631+0000 7f65ee37b740 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.7:6835/336889324,v1:172.21.15.7:6837/336889324] conn(0x5578c11d4430 0x5578c11a5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.631+0000 7f65ee37b740 4 client.7095 mds_command: new command op to 6960 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:16:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.631+0000 7f65ee37b740 1 -- 172.21.15.7:0/1060489307 --> [v2:172.21.15.7:6835/336889324,v1:172.21.15.7:6837/336889324] -- command(tid 0: {"prefix": "status"}) -- 0x5578c10eb080 con 0x5578c11d4430 2024-04-25T11:16:09.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.631+0000 7f65dfe00640 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.7:6835/336889324,v1:172.21.15.7:6837/336889324] conn(0x5578c11d4430 0x5578c11a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65dfe00640 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.7:6835/336889324,v1:172.21.15.7:6837/336889324] conn(0x5578c11d4430 0x5578c11a5390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=182767f7a598501e server_cookie=69dc41efcf6a528f in_seq=0 out_seq=0 2024-04-25T11:16:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65c5600640 10 client.7095 ms_handle_connect on v2:172.21.15.7:6835/336889324 2024-04-25T11:16:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65c5600640 1 -- 172.21.15.7:0/1060489307 <== mds.0 v2:172.21.15.7:6835/336889324 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5578c10eb080 con 0x5578c11d4430 2024-04-25T11:16:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65c5600640 10 client.7095 handle_command_reply: tid=0 2024-04-25T11:16:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65ee37b740 1 client.7095 shutdown 2024-04-25T11:16:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65ee37b740 1 -- 172.21.15.7:0/1060489307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0db43b0 msgr2=0x5578c1179c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65ee37b740 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0db43b0 0x5578c1179c10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009010 tx=0x7f65d4015000 comp rx=0 tx=0).stop 2024-04-25T11:16:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65ee37b740 1 -- 172.21.15.7:0/1060489307 shutdown_connections 2024-04-25T11:16:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65ee37b740 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.7:6835/336889324,v1:172.21.15.7:6837/336889324] conn(0x5578c11d4430 0x5578c11a5390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65ee37b740 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5578c0ddd270 0x5578c11c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65ee37b740 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5578c0dbc590 0x5578c1241980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65c5600640 0 client.7095 ms_handle_reset on v2:172.21.15.7:6835/336889324 2024-04-25T11:16:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65ee37b740 1 --2- 172.21.15.7:0/1060489307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578c0db43b0 0x5578c1179c10 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65ee37b740 1 -- 172.21.15.7:0/1060489307 >> 172.21.15.7:0/1060489307 conn(0x5578c0dd13c0 msgr2=0x5578c11a66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.632+0000 7f65ee37b740 1 -- 172.21.15.7:0/1060489307 shutdown_connections 2024-04-25T11:16:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65ee37b740 1 -- 172.21.15.7:0/1060489307 wait complete. 2024-04-25T11:16:09.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65ee37b740 20 client.7095 trim_cache size 0 max 16384 2024-04-25T11:16:09.636 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:16:09.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:16:09.637 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:16:09.637 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 6960, 2024-04-25T11:16:09.637 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:16:09.637 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:16:09.637 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:16:09.637 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.3876886759999998, 2024-04-25T11:16:09.637 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 52, 2024-04-25T11:16:09.637 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 92, 2024-04-25T11:16:09.637 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T11:16:09.637 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.522838672000001 2024-04-25T11:16:09.637 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:16:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65e5e00640 1 -- 172.21.15.7:0/2608640834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65c00744b0 msgr2=0x7f65c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65e5e00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65c00744b0 0x7f65c00769c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f65d40016f0 tx=0x7f65d4009040 comp rx=0 tx=0).stop 2024-04-25T11:16:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65e5e00640 1 -- 172.21.15.7:0/2608640834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0155b90 msgr2=0x7f65e01614e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65e5e00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0155b90 0x7f65e01614e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f65e0074c70 tx=0x7f65d800e160 comp rx=0 tx=0).stop 2024-04-25T11:16:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65e5e00640 1 -- 172.21.15.7:0/2608640834 shutdown_connections 2024-04-25T11:16:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65e5e00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65c00744b0 0x7f65c00769c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65e5e00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0155b90 0x7f65e01614e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65e5e00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e00b6280 0x7f65e0155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65e5e00640 1 --2- 172.21.15.7:0/2608640834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65c8003f80 0x7f65e015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.633+0000 7f65e5e00640 1 -- 172.21.15.7:0/2608640834 >> 172.21.15.7:0/2608640834 conn(0x7f65e00d1540 msgr2=0x7f65e01e9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.634+0000 7f65e5e00640 1 -- 172.21.15.7:0/2608640834 shutdown_connections 2024-04-25T11:16:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.634+0000 7f65e5e00640 1 -- 172.21.15.7:0/2608640834 wait complete. 2024-04-25T11:16:09.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:16:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.923+0000 7fed55800640 1 Processor -- start 2024-04-25T11:16:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.923+0000 7fed55800640 1 -- start start 2024-04-25T11:16:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.925+0000 7fed55800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed5006dc90 0x7fed500d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.925+0000 7fed55800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed50070c20 0x7fed500d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.925+0000 7fed55800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed500cffa0 con 0x7fed50070810 2024-04-25T11:16:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.925+0000 7fed55800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fed50160d90 con 0x7fed5006dc90 2024-04-25T11:16:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.925+0000 7fed55800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fed50161070 con 0x7fed50070c20 2024-04-25T11:16:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.926+0000 7fed47400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed5006dc90 0x7fed500d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.926+0000 7fed4fe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fed50070810 0x7fed5006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:60814/0 (socket says 172.21.15.7:60814) 2024-04-25T11:16:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.926+0000 7fed4fe00640 1 -- 172.21.15.7:0/3010193034 learned_addr learned my addr 172.21.15.7:0/3010193034 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.926+0000 7fed54e00640 1 --2- 172.21.15.7:0/3010193034 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed50070c20 0x7fed500d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.926+0000 7fed47400640 1 -- 172.21.15.7:0/3010193034 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed50070c20 msgr2=0x7fed500d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.926+0000 7fed47400640 1 --2- 172.21.15.7:0/3010193034 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed50070c20 0x7fed500d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.926+0000 7fed47400640 1 -- 172.21.15.7:0/3010193034 >> v1:172.21.15.87:6789/0 conn(0x7fed50070810 legacy=0x7fed5006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.926+0000 7fed47400640 1 -- 172.21.15.7:0/3010193034 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed50161350 con 0x7fed5006dc90 2024-04-25T11:16:09.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.928+0000 7fed47400640 1 --2- 172.21.15.7:0/3010193034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5006dc90 0x7fed500d0460 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fed380097a0 tx=0x7fed3802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dbeb47aef89769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:09.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.928+0000 7fed4f400640 1 -- 172.21.15.7:0/3010193034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38037070 con 0x7fed5006dc90 2024-04-25T11:16:09.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.928+0000 7fed4f400640 1 -- 172.21.15.7:0/3010193034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed38032440 con 0x7fed5006dc90 2024-04-25T11:16:09.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.928+0000 7fed4f400640 1 -- 172.21.15.7:0/3010193034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38032810 con 0x7fed5006dc90 2024-04-25T11:16:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.929+0000 7fed55800640 1 -- 172.21.15.7:0/3010193034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5006dc90 msgr2=0x7fed500d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.929+0000 7fed55800640 1 --2- 172.21.15.7:0/3010193034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5006dc90 0x7fed500d0460 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fed380097a0 tx=0x7fed3802a180 comp rx=0 tx=0).stop 2024-04-25T11:16:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.929+0000 7fed55800640 1 -- 172.21.15.7:0/3010193034 shutdown_connections 2024-04-25T11:16:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.929+0000 7fed55800640 1 --2- 172.21.15.7:0/3010193034 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed50070c20 0x7fed500d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.929+0000 7fed55800640 1 --2- 172.21.15.7:0/3010193034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5006dc90 0x7fed500d0460 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.929+0000 7fed55800640 1 -- 172.21.15.7:0/3010193034 >> 172.21.15.7:0/3010193034 conn(0x7fed50073090 msgr2=0x7fed500734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.929+0000 7fed55800640 1 -- 172.21.15.7:0/3010193034 shutdown_connections 2024-04-25T11:16:09.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.929+0000 7fed55800640 1 -- 172.21.15.7:0/3010193034 wait complete. 2024-04-25T11:16:09.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.931+0000 7fed55800640 1 Processor -- start 2024-04-25T11:16:09.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.932+0000 7fed55800640 1 -- start start 2024-04-25T11:16:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.933+0000 7fed55800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed50070810 0x7fed5013d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.933+0000 7fed55800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed50070c20 0x7fed5015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.933+0000 7fed55800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed5015af70 0x7fed5015f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.933+0000 7fed55800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed50143530 con 0x7fed50070810 2024-04-25T11:16:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.933+0000 7fed55800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fed501433b0 con 0x7fed5015af70 2024-04-25T11:16:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.933+0000 7fed55800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fed5015e4d0 con 0x7fed50070c20 2024-04-25T11:16:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.933+0000 7fed4fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed50070810 0x7fed5013d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.934+0000 7fed4fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed50070810 0x7fed5013d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48724/0 (socket says 172.21.15.7:48724) 2024-04-25T11:16:09.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.934+0000 7fed4fe00640 1 -- 172.21.15.7:0/2223885437 learned_addr learned my addr 172.21.15.7:0/2223885437 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.934+0000 7fed47400640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed50070c20 0x7fed5015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.934+0000 7fed4fe00640 1 -- 172.21.15.7:0/2223885437 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed50070c20 msgr2=0x7fed5015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.934+0000 7fed4fe00640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed50070c20 0x7fed5015a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.934+0000 7fed4fe00640 1 -- 172.21.15.7:0/2223885437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed5015af70 msgr2=0x7fed5015f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.934+0000 7fed4fe00640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed5015af70 0x7fed5015f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.934+0000 7fed4fe00640 1 -- 172.21.15.7:0/2223885437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5015bf80 con 0x7fed50070810 2024-04-25T11:16:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.934+0000 7fed4fe00640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed50070810 0x7fed5013d820 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fed4000c480 tx=0x7fed4000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.934+0000 7fed4cc00640 1 -- 172.21.15.7:0/2223885437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40017070 con 0x7fed50070810 2024-04-25T11:16:09.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.935+0000 7fed4cc00640 1 -- 172.21.15.7:0/2223885437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed40005ce0 con 0x7fed50070810 2024-04-25T11:16:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.935+0000 7fed55800640 1 -- 172.21.15.7:0/2223885437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5015b6e0 con 0x7fed50070810 2024-04-25T11:16:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.935+0000 7fed4cc00640 1 -- 172.21.15.7:0/2223885437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed400072d0 con 0x7fed50070810 2024-04-25T11:16:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.935+0000 7fed55800640 1 -- 172.21.15.7:0/2223885437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5015bca0 con 0x7fed50070810 2024-04-25T11:16:09.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.936+0000 7fed55800640 1 -- 172.21.15.7:0/2223885437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed50050740 con 0x7fed50070810 2024-04-25T11:16:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.937+0000 7fed4cc00640 1 -- 172.21.15.7:0/2223885437 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed4001a070 con 0x7fed50070810 2024-04-25T11:16:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.938+0000 7fed4cc00640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed2c0744b0 0x7fed2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.938+0000 7fed4cc00640 1 -- 172.21.15.7:0/2223885437 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7fed40094600 con 0x7fed50070810 2024-04-25T11:16:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.938+0000 7fed47400640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed2c0744b0 0x7fed2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:09.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.939+0000 7fed47400640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed2c0744b0 0x7fed2c0769c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fed38034000 tx=0x7fed3802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:09.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:09.942+0000 7fed4cc00640 1 -- 172.21.15.7:0/2223885437 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed500cffa0 con 0x7fed50070810 2024-04-25T11:16:10.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.191+0000 7fed55800640 1 -- 172.21.15.7:0/2223885437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fed500d0890 con 0x7fed50070810 2024-04-25T11:16:10.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.192+0000 7fed4cc00640 1 -- 172.21.15.7:0/2223885437 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7fed4005e5a0 con 0x7fed50070810 2024-04-25T11:16:10.192 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:16:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 -- 172.21.15.7:0/2223885437 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed2c0744b0 msgr2=0x7fed2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed2c0744b0 0x7fed2c0769c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fed38034000 tx=0x7fed3802e000 comp rx=0 tx=0).stop 2024-04-25T11:16:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 -- 172.21.15.7:0/2223885437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed50070810 msgr2=0x7fed5013d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed50070810 0x7fed5013d820 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fed4000c480 tx=0x7fed4000c940 comp rx=0 tx=0).stop 2024-04-25T11:16:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 -- 172.21.15.7:0/2223885437 shutdown_connections 2024-04-25T11:16:10.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed2c0744b0 0x7fed2c0769c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed5015af70 0x7fed5015f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed50070c20 0x7fed5015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 --2- 172.21.15.7:0/2223885437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed50070810 0x7fed5013d820 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 -- 172.21.15.7:0/2223885437 >> 172.21.15.7:0/2223885437 conn(0x7fed50073090 msgr2=0x7fed500b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 -- 172.21.15.7:0/2223885437 shutdown_connections 2024-04-25T11:16:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10.195+0000 7fed55800640 1 -- 172.21.15.7:0/2223885437 wait complete. 2024-04-25T11:16:10.224 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:16:10.225 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:16:10.225 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:16:10.225 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:16:10.225 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:16:10.225 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:16:10.225 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:16:10.225 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:16:10.225 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:16:10.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:10.225 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:16:10.277 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:16:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:16:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:16:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:16:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:10.277 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:16:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:16:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:16:10.277 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:16:10.277 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42205sec preferred_lft 42205sec 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:16:10.278 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6098:c7ff:fe8a:4a30/64 scope link 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::646a:35ff:fe7b:ad0/64 scope link 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4465:c4ff:fe62:f75f/64 scope link 2024-04-25T11:16:10.279 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:10.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:10.281 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:16:10.360 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:16:10.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:10.364 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:16:10.416 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:16:10.416 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:16:10.416 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:16:10.416 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:16:10.416 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:16:10.417 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:16:10.417 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:16:10.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:16:10.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:16:10.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:16:10.418 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:16:10.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:16:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:16:10.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:10.639 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:16:10.685 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:16:10.685 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:16:10.686 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:16:10.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:10.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:16:10.821 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:16:10.821 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:16:10.876 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:16:10.876 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:16:10.877 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:16:10.931 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:16:10.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:10.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:16:11.146 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:16:11.147 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:16:11.190 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:16:11.190 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:16:11.190 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:16:11.190 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:16:11.190 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:16:11.192 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:16:11.192 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:16:11.192 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:16:11.192 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:16:11.192 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:16:11.197 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:16:11.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:11.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:16:11.276 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:16:11.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:16:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:16:11.426 INFO:teuthology.orchestra.run.smithi007.stdout:client6205 2024-04-25T11:16:11.427 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:16:11.427 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:16:11.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:16:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.765+0000 7f1945e00640 1 Processor -- start 2024-04-25T11:16:11.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.765+0000 7f1945e00640 1 -- start start 2024-04-25T11:16:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.767+0000 7f1945e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f194015e2f0 0x7f1940161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.767+0000 7f1945e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940162420 0x7f194006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.767+0000 7f1945e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f194015e700 con 0x7f194015baa0 2024-04-25T11:16:11.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.767+0000 7f1945e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1940162830 con 0x7f194015e2f0 2024-04-25T11:16:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.767+0000 7f1945e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f194015e980 con 0x7f1940162420 2024-04-25T11:16:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.768+0000 7f193fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f194015e2f0 0x7f1940161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.768+0000 7f193fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f194015e2f0 0x7f1940161ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48738/0 (socket says 172.21.15.7:48738) 2024-04-25T11:16:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.768+0000 7f193fe00640 1 -- 172.21.15.7:0/1584138297 learned_addr learned my addr 172.21.15.7:0/1584138297 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.768+0000 7f1945400640 1 --2- 172.21.15.7:0/1584138297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940162420 0x7f194006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.768+0000 7f193fe00640 1 -- 172.21.15.7:0/1584138297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940162420 msgr2=0x7f194006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.768+0000 7f193fe00640 1 --2- 172.21.15.7:0/1584138297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940162420 0x7f194006d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.768+0000 7f193fe00640 1 -- 172.21.15.7:0/1584138297 >> v1:172.21.15.87:6789/0 conn(0x7f194015baa0 legacy=0x7f1940070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.768+0000 7f193fe00640 1 -- 172.21.15.7:0/1584138297 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194015ec60 con 0x7f194015e2f0 2024-04-25T11:16:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.768+0000 7f1945400640 1 --2- 172.21.15.7:0/1584138297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940162420 0x7f194006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:11.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.770+0000 7f193fe00640 1 --2- 172.21.15.7:0/1584138297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f194015e2f0 0x7f1940161ee0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1940074530 tx=0x7f192c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6863312af1d049 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.770+0000 7f193f400640 1 -- 172.21.15.7:0/1584138297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c037070 con 0x7f194015e2f0 2024-04-25T11:16:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.770+0000 7f193f400640 1 -- 172.21.15.7:0/1584138297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f192c032440 con 0x7f194015e2f0 2024-04-25T11:16:11.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.770+0000 7f193f400640 1 -- 172.21.15.7:0/1584138297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c032810 con 0x7f194015e2f0 2024-04-25T11:16:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.771+0000 7f1945e00640 1 -- 172.21.15.7:0/1584138297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f194015e2f0 msgr2=0x7f1940161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.771+0000 7f1945e00640 1 --2- 172.21.15.7:0/1584138297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f194015e2f0 0x7f1940161ee0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1940074530 tx=0x7f192c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:16:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.771+0000 7f1945e00640 1 -- 172.21.15.7:0/1584138297 shutdown_connections 2024-04-25T11:16:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.771+0000 7f1945e00640 1 --2- 172.21.15.7:0/1584138297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940162420 0x7f194006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.771+0000 7f1945e00640 1 --2- 172.21.15.7:0/1584138297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f194015e2f0 0x7f1940161ee0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.771+0000 7f1945e00640 1 -- 172.21.15.7:0/1584138297 >> 172.21.15.7:0/1584138297 conn(0x7f1940073090 msgr2=0x7f19400734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.771+0000 7f1945e00640 1 -- 172.21.15.7:0/1584138297 shutdown_connections 2024-04-25T11:16:11.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.771+0000 7f1945e00640 1 -- 172.21.15.7:0/1584138297 wait complete. 2024-04-25T11:16:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.773+0000 7f1945e00640 1 Processor -- start 2024-04-25T11:16:11.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.774+0000 7f1945e00640 1 -- start start 2024-04-25T11:16:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.775+0000 7f1945e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f194015e2f0 0x7f194013f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.775+0000 7f1945e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940162420 0x7f19401e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:11.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.775+0000 7f1945e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19401e80f0 0x7f19401442d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.775+0000 7f1945e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f194006dac0 con 0x7f19401e80f0 2024-04-25T11:16:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.775+0000 7f1945e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f19401619c0 con 0x7f194015e2f0 2024-04-25T11:16:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.775+0000 7f1945e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f194006dc40 con 0x7f1940162420 2024-04-25T11:16:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.775+0000 7f1945400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19401e80f0 0x7f19401442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.775+0000 7f1945400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19401e80f0 0x7f19401442d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48742/0 (socket says 172.21.15.7:48742) 2024-04-25T11:16:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.775+0000 7f1945400640 1 -- 172.21.15.7:0/3806509911 learned_addr learned my addr 172.21.15.7:0/3806509911 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.776+0000 7f1944a00640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f194015e2f0 0x7f194013f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.776+0000 7f193fe00640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940162420 0x7f19401e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.776+0000 7f1945400640 1 -- 172.21.15.7:0/3806509911 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940162420 msgr2=0x7f19401e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.776+0000 7f1945400640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940162420 0x7f19401e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.776+0000 7f1945400640 1 -- 172.21.15.7:0/3806509911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f194015e2f0 msgr2=0x7f194013f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.776+0000 7f1945400640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f194015e2f0 0x7f194013f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.776+0000 7f1945400640 1 -- 172.21.15.7:0/3806509911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19401f6560 con 0x7f19401e80f0 2024-04-25T11:16:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.776+0000 7f1945400640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19401e80f0 0x7f19401442d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f193800ccf0 tx=0x7f193800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.776+0000 7f193cc00640 1 -- 172.21.15.7:0/3806509911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193800f680 con 0x7f19401e80f0 2024-04-25T11:16:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.777+0000 7f193cc00640 1 -- 172.21.15.7:0/3806509911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193800f840 con 0x7f19401e80f0 2024-04-25T11:16:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.777+0000 7f193cc00640 1 -- 172.21.15.7:0/3806509911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193800fc30 con 0x7f19401e80f0 2024-04-25T11:16:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.777+0000 7f1945e00640 1 -- 172.21.15.7:0/3806509911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19401e8870 con 0x7f19401e80f0 2024-04-25T11:16:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.777+0000 7f1945e00640 1 -- 172.21.15.7:0/3806509911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19401e8e30 con 0x7f19401e80f0 2024-04-25T11:16:11.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.778+0000 7f1945e00640 1 -- 172.21.15.7:0/3806509911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19401e85c0 con 0x7f19401e80f0 2024-04-25T11:16:11.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.779+0000 7f193cc00640 1 -- 172.21.15.7:0/3806509911 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f19380213f0 con 0x7f19401e80f0 2024-04-25T11:16:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.781+0000 7f193cc00640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19200744e0 0x7f19200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:11.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.782+0000 7f193cc00640 1 -- 172.21.15.7:0/3806509911 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f1938096150 con 0x7f19401e80f0 2024-04-25T11:16:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.782+0000 7f1944a00640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19200744e0 0x7f19200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.783+0000 7f1944a00640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19200744e0 0x7f19200769f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1934006fd0 tx=0x7f1934008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:11.787+0000 7f193cc00640 1 -- 172.21.15.7:0/3806509911 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19401f6560 con 0x7f19401e80f0 2024-04-25T11:16:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.035+0000 7f1945e00640 1 -- 172.21.15.7:0/3806509911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f19400cfe60 con 0x7f19401e80f0 2024-04-25T11:16:12.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.035+0000 7f193cc00640 1 -- 172.21.15.7:0/3806509911 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7f1938014030 con 0x7f19401e80f0 2024-04-25T11:16:12.036 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:16:12.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.038+0000 7f1945e00640 1 -- 172.21.15.7:0/3806509911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19200744e0 msgr2=0x7f19200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:12.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.038+0000 7f1945e00640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19200744e0 0x7f19200769f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1934006fd0 tx=0x7f1934008040 comp rx=0 tx=0).stop 2024-04-25T11:16:12.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.039+0000 7f1945e00640 1 -- 172.21.15.7:0/3806509911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19401e80f0 msgr2=0x7f19401442d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:12.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.039+0000 7f1945e00640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19401e80f0 0x7f19401442d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f193800ccf0 tx=0x7f193800a650 comp rx=0 tx=0).stop 2024-04-25T11:16:12.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.039+0000 7f1945e00640 1 -- 172.21.15.7:0/3806509911 shutdown_connections 2024-04-25T11:16:12.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.039+0000 7f1945e00640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19200744e0 0x7f19200769f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:12.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.039+0000 7f1945e00640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f19401e80f0 0x7f19401442d0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:12.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.039+0000 7f1945e00640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940162420 0x7f19401e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:12.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.039+0000 7f1945e00640 1 --2- 172.21.15.7:0/3806509911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f194015e2f0 0x7f194013f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:12.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.039+0000 7f1945e00640 1 -- 172.21.15.7:0/3806509911 >> 172.21.15.7:0/3806509911 conn(0x7f1940073090 msgr2=0x7f194015bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:12.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.039+0000 7f1945e00640 1 -- 172.21.15.7:0/3806509911 shutdown_connections 2024-04-25T11:16:12.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:12.039+0000 7f1945e00640 1 -- 172.21.15.7:0/3806509911 wait complete. 2024-04-25T11:16:12.065 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:16:12.065 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:16:12.065 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:16:12.065 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:16:12.065 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:16:12.065 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:16:12.065 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:16:12.065 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:16:12.065 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:16:12.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:12.066 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:16:12.126 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:16:12.126 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:16:12.126 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:16:12.126 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:12.126 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:16:12.126 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:12.126 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42204sec preferred_lft 42204sec 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:16:12.127 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::58f2:6bff:fe8b:8b4c/64 scope link 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::98fe:1bff:fe9f:d71c/64 scope link 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::3c38:a8ff:fec7:ea00/64 scope link 2024-04-25T11:16:12.128 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:16:12.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:12.129 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:16:12.210 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:16:12.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:12.214 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:16:12.263 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:16:12.263 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:16:12.263 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:16:12.263 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:16:12.263 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:16:12.264 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:16:12.264 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:16:12.265 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:16:12.265 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:16:12.265 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:16:12.265 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:16:12.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:16:12.273 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:16:12.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:16:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:16:12.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:16:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:16:12.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:12.460 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:16:12.554 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:16:12.554 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:16:12.555 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:16:12.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:12.556 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T11:16:12.639 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:16:12.639 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:16:12.694 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:16:12.694 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:16:12.694 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:16:12.743 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:16:12.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:12.743 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:16:12.939 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:16:12.940 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:16:12.984 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:16:12.984 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:16:12.984 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:16:12.984 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 1@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:16:12.984 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:16:12.990 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:16:12.990 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:16:12.991 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:16:12.991 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:16:12.991 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:16:12.992 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:16:12.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:16:12.992 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:16:13.068 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:16:13.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:16:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:16:13.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:16:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:16:13.186 INFO:teuthology.orchestra.run.smithi087.stdout:client7110 2024-04-25T11:16:13.187 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:16:13.187 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:16:13.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-04-25T11:16:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.535+0000 7f57e7e00640 1 Processor -- start 2024-04-25T11:16:13.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.535+0000 7f57e7e00640 1 -- start start 2024-04-25T11:16:13.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.537+0000 7f57e7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57e015dfb0 0x7f57e00b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.537+0000 7f57e7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57e006fab0 0x7f57e00aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.537+0000 7f57e7e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57e015e3c0 con 0x7f57e00d0b40 2024-04-25T11:16:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.537+0000 7f57e7e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f57e00b09b0 con 0x7f57e015dfb0 2024-04-25T11:16:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.537+0000 7f57e7e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f57e00cf800 con 0x7f57e006fab0 2024-04-25T11:16:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.538+0000 7f57e5400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57e015dfb0 0x7f57e00b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.538+0000 7f57e4a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f57e00d0b40 0x7f57e006f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37518/0 (socket says 172.21.15.7:37518) 2024-04-25T11:16:13.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.538+0000 7f57e4a00640 1 -- 172.21.15.7:0/3195745152 learned_addr learned my addr 172.21.15.7:0/3195745152 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.538+0000 7f57e5e00640 1 --2- 172.21.15.7:0/3195745152 >> v2:172.21.15.87:3300/0 conn(0x7f57e006fab0 0x7f57e00aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.538+0000 7f57e5400640 1 -- 172.21.15.7:0/3195745152 >> v2:172.21.15.87:3300/0 conn(0x7f57e006fab0 msgr2=0x7f57e00aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.538+0000 7f57e5400640 1 --2- 172.21.15.7:0/3195745152 >> v2:172.21.15.87:3300/0 conn(0x7f57e006fab0 0x7f57e00aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.538+0000 7f57e5400640 1 -- 172.21.15.7:0/3195745152 >> v1:172.21.15.7:6789/0 conn(0x7f57e00d0b40 legacy=0x7f57e006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.538+0000 7f57e5400640 1 -- 172.21.15.7:0/3195745152 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e00cfaa0 con 0x7f57e015dfb0 2024-04-25T11:16:13.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.538+0000 7f57dfe00640 1 -- 172.21.15.7:0/3195745152 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 132773586 0 0) 0x7f57e015e3c0 con 0x7f57e00d0b40 2024-04-25T11:16:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.539+0000 7f57e5400640 1 --2- 172.21.15.7:0/3195745152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e015dfb0 0x7f57e00b0470 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f57e009fb50 tx=0x7f57d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a459f7b40bc56bce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.540+0000 7f57dfe00640 1 -- 172.21.15.7:0/3195745152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0037070 con 0x7f57e015dfb0 2024-04-25T11:16:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.540+0000 7f57dfe00640 1 -- 172.21.15.7:0/3195745152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d0032440 con 0x7f57e015dfb0 2024-04-25T11:16:13.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.540+0000 7f57dfe00640 1 -- 172.21.15.7:0/3195745152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0032810 con 0x7f57e015dfb0 2024-04-25T11:16:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.541+0000 7f57e7e00640 1 -- 172.21.15.7:0/3195745152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e015dfb0 msgr2=0x7f57e00b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.541+0000 7f57e7e00640 1 --2- 172.21.15.7:0/3195745152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e015dfb0 0x7f57e00b0470 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f57e009fb50 tx=0x7f57d002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:16:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.541+0000 7f57e7e00640 1 -- 172.21.15.7:0/3195745152 shutdown_connections 2024-04-25T11:16:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.541+0000 7f57e7e00640 1 --2- 172.21.15.7:0/3195745152 >> v2:172.21.15.87:3300/0 conn(0x7f57e006fab0 0x7f57e00aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.541+0000 7f57e7e00640 1 --2- 172.21.15.7:0/3195745152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e015dfb0 0x7f57e00b0470 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.541+0000 7f57e7e00640 1 -- 172.21.15.7:0/3195745152 >> 172.21.15.7:0/3195745152 conn(0x7f57e00742c0 msgr2=0x7f57e006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.541+0000 7f57e7e00640 1 -- 172.21.15.7:0/3195745152 shutdown_connections 2024-04-25T11:16:13.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.541+0000 7f57e7e00640 1 -- 172.21.15.7:0/3195745152 wait complete. 2024-04-25T11:16:13.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.543+0000 7f57e7e00640 1 Processor -- start 2024-04-25T11:16:13.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.543+0000 7f57e7e00640 1 -- start start 2024-04-25T11:16:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.545+0000 7f57e7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57e006fab0 0x7f57e014d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:13.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.545+0000 7f57e7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57e00d0b40 0x7f57e0149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.545+0000 7f57e7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e015dfb0 0x7f57e015c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.545+0000 7f57e7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57e00d0340 con 0x7f57e015dfb0 2024-04-25T11:16:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.545+0000 7f57e7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57e00d0060 con 0x7f57e006fab0 2024-04-25T11:16:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.545+0000 7f57e7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57e00d0620 con 0x7f57e00d0b40 2024-04-25T11:16:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.545+0000 7f57e5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e015dfb0 0x7f57e015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.545+0000 7f57e5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e015dfb0 0x7f57e015c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48766/0 (socket says 172.21.15.7:48766) 2024-04-25T11:16:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.545+0000 7f57e5e00640 1 -- 172.21.15.7:0/2952794641 learned_addr learned my addr 172.21.15.7:0/2952794641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:13.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.546+0000 7f57e4a00640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57e00d0b40 0x7f57e0149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:13.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.546+0000 7f57e5e00640 1 -- 172.21.15.7:0/2952794641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57e00d0b40 msgr2=0x7f57e0149ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:13.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.546+0000 7f57e5e00640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57e00d0b40 0x7f57e0149ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:13.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.546+0000 7f57e5400640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57e006fab0 0x7f57e014d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.546+0000 7f57e5e00640 1 -- 172.21.15.7:0/2952794641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57e006fab0 msgr2=0x7f57e014d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.546+0000 7f57e5e00640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57e006fab0 0x7f57e014d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.546+0000 7f57e5e00640 1 -- 172.21.15.7:0/2952794641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e014d810 con 0x7f57e015dfb0 2024-04-25T11:16:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.546+0000 7f57e5400640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57e006fab0 0x7f57e014d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.546+0000 7f57e5e00640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e015dfb0 0x7f57e015c680 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f57d8007e60 tx=0x7f57d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.547+0000 7f57dd600640 1 -- 172.21.15.7:0/2952794641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8014420 con 0x7f57e015dfb0 2024-04-25T11:16:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.547+0000 7f57dd600640 1 -- 172.21.15.7:0/2952794641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d8019070 con 0x7f57e015dfb0 2024-04-25T11:16:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.547+0000 7f57dd600640 1 -- 172.21.15.7:0/2952794641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8014820 con 0x7f57e015dfb0 2024-04-25T11:16:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.547+0000 7f57e7e00640 1 -- 172.21.15.7:0/2952794641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e014a820 con 0x7f57e015dfb0 2024-04-25T11:16:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.547+0000 7f57e7e00640 1 -- 172.21.15.7:0/2952794641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e01f9e70 con 0x7f57e015dfb0 2024-04-25T11:16:13.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.548+0000 7f57e7e00640 1 -- 172.21.15.7:0/2952794641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e014a590 con 0x7f57e015dfb0 2024-04-25T11:16:13.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.549+0000 7f57dd600640 1 -- 172.21.15.7:0/2952794641 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f57d80149c0 con 0x7f57e015dfb0 2024-04-25T11:16:13.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.550+0000 7f57dd600640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57c00744b0 0x7f57c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.550+0000 7f57dd600640 1 -- 172.21.15.7:0/2952794641 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f57d8095cd0 con 0x7f57e015dfb0 2024-04-25T11:16:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.551+0000 7f57e5400640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57c00744b0 0x7f57c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:13.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.552+0000 7f57e5400640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57c00744b0 0x7f57c00769c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f57d0034000 tx=0x7f57d002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.555+0000 7f57dd600640 1 -- 172.21.15.7:0/2952794641 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57d8060230 con 0x7f57e015dfb0 2024-04-25T11:16:13.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.759+0000 7f57e7e00640 1 -- 172.21.15.7:0/2952794641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f57e0050740 con 0x7f57e015dfb0 2024-04-25T11:16:13.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.766+0000 7f57dd600640 1 -- 172.21.15.7:0/2952794641 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v25)=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f57d805fb30 con 0x7f57e015dfb0 2024-04-25T11:16:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.769+0000 7f57e7e00640 1 -- 172.21.15.7:0/2952794641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57c00744b0 msgr2=0x7f57c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.769+0000 7f57e7e00640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57c00744b0 0x7f57c00769c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f57d0034000 tx=0x7f57d002e000 comp rx=0 tx=0).stop 2024-04-25T11:16:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.770+0000 7f57e7e00640 1 -- 172.21.15.7:0/2952794641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e015dfb0 msgr2=0x7f57e015c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.770+0000 7f57e7e00640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e015dfb0 0x7f57e015c680 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f57d8007e60 tx=0x7f57d800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:16:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.770+0000 7f57e7e00640 1 -- 172.21.15.7:0/2952794641 shutdown_connections 2024-04-25T11:16:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.770+0000 7f57e7e00640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57c00744b0 0x7f57c00769c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.770+0000 7f57e7e00640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57e015dfb0 0x7f57e015c680 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.770+0000 7f57e7e00640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57e00d0b40 0x7f57e0149ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.770+0000 7f57e7e00640 1 --2- 172.21.15.7:0/2952794641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57e006fab0 0x7f57e014d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.770+0000 7f57e7e00640 1 -- 172.21.15.7:0/2952794641 >> 172.21.15.7:0/2952794641 conn(0x7f57e00742c0 msgr2=0x7f57e006f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.770+0000 7f57e7e00640 1 -- 172.21.15.7:0/2952794641 shutdown_connections 2024-04-25T11:16:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:13.770+0000 7f57e7e00640 1 -- 172.21.15.7:0/2952794641 wait complete. 2024-04-25T11:16:13.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2024-04-25T11:16:14.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.083+0000 7fcf43e00640 1 Processor -- start 2024-04-25T11:16:14.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.083+0000 7fcf43e00640 1 -- start start 2024-04-25T11:16:14.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.085+0000 7fcf43e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcf440d4d80 0x7fcf440d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.085+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf440d5190 0x7fcf440d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.085+0000 7fcf43e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf44161030 con 0x7fcf441616e0 2024-04-25T11:16:14.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.085+0000 7fcf43e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcf44161310 con 0x7fcf440d4d80 2024-04-25T11:16:14.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.085+0000 7fcf43e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf440d4580 con 0x7fcf440d5190 2024-04-25T11:16:14.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.086+0000 7fcf43400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcf441616e0 0x7fcf44161af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37522/0 (socket says 172.21.15.7:37522) 2024-04-25T11:16:14.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.086+0000 7fcf43400640 1 -- 172.21.15.7:0/3153326972 learned_addr learned my addr 172.21.15.7:0/3153326972 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.086+0000 7fcf3be00640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf440d5190 0x7fcf440d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.086+0000 7fcf42a00640 1 --2- 172.21.15.7:0/3153326972 >> v2:172.21.15.87:3300/0 conn(0x7fcf440d4d80 0x7fcf440d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.086+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3780604344 0 0) 0x7fcf44161030 con 0x7fcf441616e0 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.086+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf28003520 con 0x7fcf441616e0 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1415992291 0 0) 0x7fcf28003520 con 0x7fcf441616e0 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf440d5190 msgr2=0x7fcf440d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf440d5190 0x7fcf440d2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 >> v2:172.21.15.87:3300/0 conn(0x7fcf440d4d80 msgr2=0x7fcf440d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 --2- 172.21.15.7:0/3153326972 >> v2:172.21.15.87:3300/0 conn(0x7fcf440d4d80 0x7fcf440d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf440d4860 con 0x7fcf441616e0 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fcf3c0019f0 con 0x7fcf441616e0 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 >> v1:172.21.15.7:6789/0 conn(0x7fcf441616e0 legacy=0x7fcf44161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf280039b0 0x7fcf28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf440d5190 0x7fcf28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf28004d40 0x7fcf28005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44161310 con 0x7fcf280039b0 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf440d4580 con 0x7fcf28004d40 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2800b9c0 con 0x7fcf440d5190 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf42a00640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf280039b0 0x7fcf28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.087+0000 7fcf3be00640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf440d5190 0x7fcf28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.088+0000 7fcf42a00640 1 -- 172.21.15.7:0/3153326972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf440d5190 msgr2=0x7fcf28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.088+0000 7fcf42a00640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf440d5190 0x7fcf28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.088+0000 7fcf42a00640 1 -- 172.21.15.7:0/3153326972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf28004d40 msgr2=0x7fcf28005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.088+0000 7fcf43400640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf28004d40 0x7fcf28005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.088+0000 7fcf42a00640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf28004d40 0x7fcf28005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.088+0000 7fcf42a00640 1 -- 172.21.15.7:0/3153326972 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf28003520 con 0x7fcf280039b0 2024-04-25T11:16:14.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.089+0000 7fcf42a00640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf280039b0 0x7fcf28003e20 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009ac0 tx=0x7fcf28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fa4d20edd5efa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:14.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.089+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34036070 con 0x7fcf280039b0 2024-04-25T11:16:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.089+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf34031400 con 0x7fcf280039b0 2024-04-25T11:16:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.089+0000 7fcf42000640 1 -- 172.21.15.7:0/3153326972 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34009690 con 0x7fcf280039b0 2024-04-25T11:16:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.090+0000 7fcf43e00640 1 -- 172.21.15.7:0/3153326972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf280039b0 msgr2=0x7fcf28003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.090+0000 7fcf43e00640 1 --2- 172.21.15.7:0/3153326972 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf280039b0 0x7fcf28003e20 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009ac0 tx=0x7fcf28005830 comp rx=0 tx=0).stop 2024-04-25T11:16:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.090+0000 7fcf43400640 1 -- 172.21.15.7:0/3153326972 reap_dead start 2024-04-25T11:16:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.090+0000 7fcf43e00640 1 -- 172.21.15.7:0/3153326972 shutdown_connections 2024-04-25T11:16:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.090+0000 7fcf43e00640 1 -- 172.21.15.7:0/3153326972 >> 172.21.15.7:0/3153326972 conn(0x7fcf4406c750 msgr2=0x7fcf4406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.090+0000 7fcf43e00640 1 -- 172.21.15.7:0/3153326972 shutdown_connections 2024-04-25T11:16:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.090+0000 7fcf43e00640 1 -- 172.21.15.7:0/3153326972 wait complete. 2024-04-25T11:16:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.093+0000 7fcf43e00640 1 Processor -- start 2024-04-25T11:16:14.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.093+0000 7fcf43e00640 1 -- start start 2024-04-25T11:16:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.095+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf280039b0 0x7fcf440c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.095+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf441562a0 0x7fcf441566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.095+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf441f9750 0x7fcf441f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.095+0000 7fcf43e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcf440d2e40 con 0x7fcf441562a0 2024-04-25T11:16:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.095+0000 7fcf43e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf440d2cc0 con 0x7fcf441f9750 2024-04-25T11:16:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.095+0000 7fcf43e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf440d2fc0 con 0x7fcf280039b0 2024-04-25T11:16:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.095+0000 7fcf3be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf441562a0 0x7fcf441566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.095+0000 7fcf3be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf441562a0 0x7fcf441566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48790/0 (socket says 172.21.15.7:48790) 2024-04-25T11:16:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.095+0000 7fcf3be00640 1 -- 172.21.15.7:0/3381563832 learned_addr learned my addr 172.21.15.7:0/3381563832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.096+0000 7fcf42a00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf280039b0 0x7fcf440c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.096+0000 7fcf43400640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf441f9750 0x7fcf441f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.096+0000 7fcf3be00640 1 -- 172.21.15.7:0/3381563832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf280039b0 msgr2=0x7fcf440c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.096+0000 7fcf3be00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf280039b0 0x7fcf440c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.096+0000 7fcf3be00640 1 -- 172.21.15.7:0/3381563832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf441f9750 msgr2=0x7fcf441f9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.096+0000 7fcf3be00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf441f9750 0x7fcf441f9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.096+0000 7fcf3be00640 1 -- 172.21.15.7:0/3381563832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf441fa0a0 con 0x7fcf441562a0 2024-04-25T11:16:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.096+0000 7fcf3be00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf441562a0 0x7fcf441566b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c011630 tx=0x7fcf2c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.096+0000 7fcf3aa00640 1 -- 172.21.15.7:0/3381563832 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00f920 con 0x7fcf441562a0 2024-04-25T11:16:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.097+0000 7fcf3aa00640 1 -- 172.21.15.7:0/3381563832 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2c00fac0 con 0x7fcf441562a0 2024-04-25T11:16:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.097+0000 7fcf3aa00640 1 -- 172.21.15.7:0/3381563832 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00fe90 con 0x7fcf441562a0 2024-04-25T11:16:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.097+0000 7fcf43e00640 1 -- 172.21.15.7:0/3381563832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf44156fd0 con 0x7fcf441562a0 2024-04-25T11:16:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.097+0000 7fcf43e00640 1 -- 172.21.15.7:0/3381563832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf44157590 con 0x7fcf441562a0 2024-04-25T11:16:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.098+0000 7fcf43e00640 1 -- 172.21.15.7:0/3381563832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf44050740 con 0x7fcf441562a0 2024-04-25T11:16:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.099+0000 7fcf3aa00640 1 -- 172.21.15.7:0/3381563832 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf2c015460 con 0x7fcf441562a0 2024-04-25T11:16:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.101+0000 7fcf3aa00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf20074680 0x7fcf20076b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.101+0000 7fcf3aa00640 1 -- 172.21.15.7:0/3381563832 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7fcf2c095df0 con 0x7fcf441562a0 2024-04-25T11:16:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.101+0000 7fcf42a00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf20074680 0x7fcf20076b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.102+0000 7fcf42a00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf20074680 0x7fcf20076b90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcf34033000 tx=0x7fcf34008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.106+0000 7fcf3aa00640 1 -- 172.21.15.7:0/3381563832 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf441fa0a0 con 0x7fcf441562a0 2024-04-25T11:16:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.312+0000 7fcf43e00640 1 -- 172.21.15.7:0/3381563832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7fcf4415e060 con 0x7fcf441562a0 2024-04-25T11:16:14.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.318+0000 7fcf3aa00640 1 -- 172.21.15.7:0/3381563832 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v26)=0 v26) ==== 119+0+0 (secure 0 0 0) 0x7fcf2c019030 con 0x7fcf441562a0 2024-04-25T11:16:14.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 -- 172.21.15.7:0/3381563832 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf20074680 msgr2=0x7fcf20076b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf20074680 0x7fcf20076b90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcf34033000 tx=0x7fcf34008030 comp rx=0 tx=0).stop 2024-04-25T11:16:14.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 -- 172.21.15.7:0/3381563832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf441562a0 msgr2=0x7fcf441566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf441562a0 0x7fcf441566b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c011630 tx=0x7fcf2c011af0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 -- 172.21.15.7:0/3381563832 shutdown_connections 2024-04-25T11:16:14.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf20074680 0x7fcf20076b90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf441f9750 0x7fcf441f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf441562a0 0x7fcf441566b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 --2- 172.21.15.7:0/3381563832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf280039b0 0x7fcf440c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 -- 172.21.15.7:0/3381563832 >> 172.21.15.7:0/3381563832 conn(0x7fcf4406c750 msgr2=0x7fcf440c9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:14.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 -- 172.21.15.7:0/3381563832 shutdown_connections 2024-04-25T11:16:14.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.321+0000 7fcf43e00640 1 -- 172.21.15.7:0/3381563832 wait complete. 2024-04-25T11:16:14.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2024-04-25T11:16:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.637+0000 7f54af600640 1 Processor -- start 2024-04-25T11:16:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.638+0000 7f54af600640 1 -- start start 2024-04-25T11:16:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.640+0000 7f54af600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54a815dfb0 0x7f54a80b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.640+0000 7f54af600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f54a80d0b40 0x7f54a806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.640+0000 7f54af600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a815e3c0 con 0x7f54a80d0f50 2024-04-25T11:16:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.640+0000 7f54af600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f54a80b09b0 con 0x7f54a80d0b40 2024-04-25T11:16:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.640+0000 7f54af600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f54a80cf800 con 0x7f54a815dfb0 2024-04-25T11:16:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.641+0000 7f54ad600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f54a80d0f50 0x7f54a8161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37526/0 (socket says 172.21.15.7:37526) 2024-04-25T11:16:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.641+0000 7f54ad600640 1 -- 172.21.15.7:0/575914816 learned_addr learned my addr 172.21.15.7:0/575914816 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.641+0000 7f54a7e00640 1 --2- 172.21.15.7:0/575914816 >> v2:172.21.15.7:3300/0 conn(0x7f54a80d0b40 0x7f54a806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.641+0000 7f54acc00640 1 --2- 172.21.15.7:0/575914816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54a815dfb0 0x7f54a80b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.641+0000 7f54a7400640 1 -- 172.21.15.7:0/575914816 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147524110 0 0) 0x7f54a815e3c0 con 0x7f54a80d0f50 2024-04-25T11:16:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.641+0000 7f54a7400640 1 -- 172.21.15.7:0/575914816 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5490003520 con 0x7f54a80d0f50 2024-04-25T11:16:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.641+0000 7f54a7e00640 1 -- 172.21.15.7:0/575914816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54a815dfb0 msgr2=0x7f54a80b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.641+0000 7f54a7e00640 1 --2- 172.21.15.7:0/575914816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54a815dfb0 0x7f54a80b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.641+0000 7f54a7e00640 1 -- 172.21.15.7:0/575914816 >> v1:172.21.15.7:6789/0 conn(0x7f54a80d0f50 legacy=0x7f54a8161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.641+0000 7f54a7e00640 1 -- 172.21.15.7:0/575914816 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a80cfaa0 con 0x7f54a80d0b40 2024-04-25T11:16:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.643+0000 7f54a7e00640 1 --2- 172.21.15.7:0/575914816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a80d0b40 0x7f54a806f3a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f54940097a0 tx=0x7f549402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac01c0349ddf2a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.643+0000 7f54a7400640 1 -- 172.21.15.7:0/575914816 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494037070 con 0x7f54a80d0b40 2024-04-25T11:16:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.643+0000 7f54a7400640 1 -- 172.21.15.7:0/575914816 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5494032440 con 0x7f54a80d0b40 2024-04-25T11:16:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.643+0000 7f54a7400640 1 -- 172.21.15.7:0/575914816 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494032810 con 0x7f54a80d0b40 2024-04-25T11:16:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.644+0000 7f54af600640 1 -- 172.21.15.7:0/575914816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a80d0b40 msgr2=0x7f54a806f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.644+0000 7f54af600640 1 --2- 172.21.15.7:0/575914816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a80d0b40 0x7f54a806f3a0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f54940097a0 tx=0x7f549402a180 comp rx=0 tx=0).stop 2024-04-25T11:16:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.644+0000 7f54af600640 1 -- 172.21.15.7:0/575914816 shutdown_connections 2024-04-25T11:16:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.644+0000 7f54af600640 1 --2- 172.21.15.7:0/575914816 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a80d0b40 0x7f54a806f3a0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.644+0000 7f54af600640 1 --2- 172.21.15.7:0/575914816 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54a815dfb0 0x7f54a80b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.644+0000 7f54af600640 1 -- 172.21.15.7:0/575914816 >> 172.21.15.7:0/575914816 conn(0x7f54a80742c0 msgr2=0x7f54a806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.644+0000 7f54af600640 1 -- 172.21.15.7:0/575914816 shutdown_connections 2024-04-25T11:16:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.644+0000 7f54af600640 1 -- 172.21.15.7:0/575914816 wait complete. 2024-04-25T11:16:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.647+0000 7f54af600640 1 Processor -- start 2024-04-25T11:16:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.647+0000 7f54af600640 1 -- start start 2024-04-25T11:16:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.648+0000 7f54af600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a80d0b40 0x7f54a8151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.648+0000 7f54af600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a80d0f50 0x7f54a80b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.648+0000 7f54af600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54a815dfb0 0x7f54a80aef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.648+0000 7f54af600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54a80d0340 con 0x7f54a80d0f50 2024-04-25T11:16:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.648+0000 7f54af600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54a80d0060 con 0x7f54a80d0b40 2024-04-25T11:16:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.648+0000 7f54af600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f54a80d0620 con 0x7f54a815dfb0 2024-04-25T11:16:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.649+0000 7f54a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a80d0f50 0x7f54a80b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.649+0000 7f54a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a80d0f50 0x7f54a80b5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48816/0 (socket says 172.21.15.7:48816) 2024-04-25T11:16:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.649+0000 7f54a7e00640 1 -- 172.21.15.7:0/1169347917 learned_addr learned my addr 172.21.15.7:0/1169347917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.649+0000 7f54a7e00640 1 -- 172.21.15.7:0/1169347917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54a815dfb0 msgr2=0x7f54a80aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.649+0000 7f54a7e00640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54a815dfb0 0x7f54a80aef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.649+0000 7f54a7e00640 1 -- 172.21.15.7:0/1169347917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a80d0b40 msgr2=0x7f54a8151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.649+0000 7f54a7e00640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a80d0b40 0x7f54a8151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.649+0000 7f54a7e00640 1 -- 172.21.15.7:0/1169347917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a8159be0 con 0x7f54a80d0f50 2024-04-25T11:16:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.649+0000 7f54a7e00640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a80d0f50 0x7f54a80b5800 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f54940098c0 tx=0x7f54940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.650+0000 7f54a4c00640 1 -- 172.21.15.7:0/1169347917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494041030 con 0x7f54a80d0f50 2024-04-25T11:16:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.650+0000 7f54a4c00640 1 -- 172.21.15.7:0/1169347917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549403d4e0 con 0x7f54a80d0f50 2024-04-25T11:16:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.650+0000 7f54af600640 1 -- 172.21.15.7:0/1169347917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a8159ec0 con 0x7f54a80d0f50 2024-04-25T11:16:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.650+0000 7f54a4c00640 1 -- 172.21.15.7:0/1169347917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494037040 con 0x7f54a80d0f50 2024-04-25T11:16:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.650+0000 7f54af600640 1 -- 172.21.15.7:0/1169347917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a815a480 con 0x7f54a80d0f50 2024-04-25T11:16:14.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.651+0000 7f54af600640 1 -- 172.21.15.7:0/1169347917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a80b62a0 con 0x7f54a80d0f50 2024-04-25T11:16:14.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.653+0000 7f54a4c00640 1 -- 172.21.15.7:0/1169347917 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5494044070 con 0x7f54a80d0f50 2024-04-25T11:16:14.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.654+0000 7f54a4c00640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54880744b0 0x7f54880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:14.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.654+0000 7f54a4c00640 1 -- 172.21.15.7:0/1169347917 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f54940b5a00 con 0x7f54a80d0f50 2024-04-25T11:16:14.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.654+0000 7f54acc00640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54880744b0 0x7f54880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:14.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.655+0000 7f54acc00640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54880744b0 0x7f54880769c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f54a8072df0 tx=0x7f549c009500 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:14.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.659+0000 7f54a4c00640 1 -- 172.21.15.7:0/1169347917 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5494080000 con 0x7f54a80d0f50 2024-04-25T11:16:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.865+0000 7f54af600640 1 -- 172.21.15.7:0/1169347917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7f54a8050740 con 0x7f54a80d0f50 2024-04-25T11:16:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.874+0000 7f54a4c00640 1 -- 172.21.15.7:0/1169347917 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v27)=0 v27) ==== 128+0+0 (secure 0 0 0) 0x7f549407f900 con 0x7f54a80d0f50 2024-04-25T11:16:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.876+0000 7f54af600640 1 -- 172.21.15.7:0/1169347917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54880744b0 msgr2=0x7f54880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.876+0000 7f54af600640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54880744b0 0x7f54880769c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f54a8072df0 tx=0x7f549c009500 comp rx=0 tx=0).stop 2024-04-25T11:16:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.876+0000 7f54af600640 1 -- 172.21.15.7:0/1169347917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a80d0f50 msgr2=0x7f54a80b5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.876+0000 7f54af600640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a80d0f50 0x7f54a80b5800 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f54940098c0 tx=0x7f54940046c0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.877+0000 7f54af600640 1 -- 172.21.15.7:0/1169347917 shutdown_connections 2024-04-25T11:16:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.877+0000 7f54af600640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54880744b0 0x7f54880769c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.877+0000 7f54af600640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54a815dfb0 0x7f54a80aef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.877+0000 7f54af600640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54a80d0f50 0x7f54a80b5800 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.877+0000 7f54af600640 1 --2- 172.21.15.7:0/1169347917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54a80d0b40 0x7f54a8151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.877+0000 7f54af600640 1 -- 172.21.15.7:0/1169347917 >> 172.21.15.7:0/1169347917 conn(0x7f54a80742c0 msgr2=0x7f54a8070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.877+0000 7f54af600640 1 -- 172.21.15.7:0/1169347917 shutdown_connections 2024-04-25T11:16:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:14.877+0000 7f54af600640 1 -- 172.21.15.7:0/1169347917 wait complete. 2024-04-25T11:16:14.908 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 800 2024-04-25T11:16:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.200+0000 7f9d74c00640 1 Processor -- start 2024-04-25T11:16:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.200+0000 7f9d74c00640 1 -- start start 2024-04-25T11:16:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.202+0000 7f9d74c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9d7015dfb0 0x7f9d700b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.202+0000 7f9d74c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700d0b40 0x7f9d7006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.202+0000 7f9d74c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d7015e3c0 con 0x7f9d700d0f50 2024-04-25T11:16:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.202+0000 7f9d74c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9d700b09b0 con 0x7f9d7015dfb0 2024-04-25T11:16:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.202+0000 7f9d74c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9d700cf800 con 0x7f9d700d0b40 2024-04-25T11:16:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.203+0000 7f9d6de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9d7015dfb0 0x7f9d700b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.203+0000 7f9d6e800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9d700d0f50 0x7f9d70161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37528/0 (socket says 172.21.15.7:37528) 2024-04-25T11:16:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.203+0000 7f9d6e800640 1 -- 172.21.15.7:0/251907732 learned_addr learned my addr 172.21.15.7:0/251907732 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.203+0000 7f9d6d400640 1 --2- 172.21.15.7:0/251907732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700d0b40 0x7f9d7006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.203+0000 7f9d6de00640 1 -- 172.21.15.7:0/251907732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700d0b40 msgr2=0x7f9d7006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.203+0000 7f9d6de00640 1 --2- 172.21.15.7:0/251907732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700d0b40 0x7f9d7006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.203+0000 7f9d6de00640 1 -- 172.21.15.7:0/251907732 >> v1:172.21.15.7:6789/0 conn(0x7f9d700d0f50 legacy=0x7f9d70161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.203+0000 7f9d6de00640 1 -- 172.21.15.7:0/251907732 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d700cfaa0 con 0x7f9d7015dfb0 2024-04-25T11:16:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.203+0000 7f9d6d400640 1 --2- 172.21.15.7:0/251907732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700d0b40 0x7f9d7006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.204+0000 7f9d6de00640 1 --2- 172.21.15.7:0/251907732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d7015dfb0 0x7f9d700b0470 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9d600097a0 tx=0x7f9d6002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6656b7b9f551313 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.205+0000 7f9d6ca00640 1 -- 172.21.15.7:0/251907732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60037070 con 0x7f9d7015dfb0 2024-04-25T11:16:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.205+0000 7f9d6ca00640 1 -- 172.21.15.7:0/251907732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d60032440 con 0x7f9d7015dfb0 2024-04-25T11:16:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.205+0000 7f9d6ca00640 1 -- 172.21.15.7:0/251907732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60032810 con 0x7f9d7015dfb0 2024-04-25T11:16:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.205+0000 7f9d74c00640 1 -- 172.21.15.7:0/251907732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d7015dfb0 msgr2=0x7f9d700b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.205+0000 7f9d74c00640 1 --2- 172.21.15.7:0/251907732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d7015dfb0 0x7f9d700b0470 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9d600097a0 tx=0x7f9d6002a180 comp rx=0 tx=0).stop 2024-04-25T11:16:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.206+0000 7f9d74c00640 1 -- 172.21.15.7:0/251907732 shutdown_connections 2024-04-25T11:16:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.206+0000 7f9d74c00640 1 --2- 172.21.15.7:0/251907732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700d0b40 0x7f9d7006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.206+0000 7f9d74c00640 1 --2- 172.21.15.7:0/251907732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d7015dfb0 0x7f9d700b0470 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.206+0000 7f9d74c00640 1 -- 172.21.15.7:0/251907732 >> 172.21.15.7:0/251907732 conn(0x7f9d700742c0 msgr2=0x7f9d7006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.206+0000 7f9d74c00640 1 -- 172.21.15.7:0/251907732 shutdown_connections 2024-04-25T11:16:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.206+0000 7f9d74c00640 1 -- 172.21.15.7:0/251907732 wait complete. 2024-04-25T11:16:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.208+0000 7f9d74c00640 1 Processor -- start 2024-04-25T11:16:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.208+0000 7f9d74c00640 1 -- start start 2024-04-25T11:16:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d74c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700d0b40 0x7f9d700b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d74c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700d0f50 0x7f9d700cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d74c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d7015dfb0 0x7f9d70148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d74c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9d700d0340 con 0x7f9d700d0b40 2024-04-25T11:16:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d74c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9d700d0060 con 0x7f9d7015dfb0 2024-04-25T11:16:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d74c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9d700d0620 con 0x7f9d700d0f50 2024-04-25T11:16:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d6de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700d0b40 0x7f9d700b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d6de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700d0b40 0x7f9d700b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48828/0 (socket says 172.21.15.7:48828) 2024-04-25T11:16:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d6de00640 1 -- 172.21.15.7:0/3055451699 learned_addr learned my addr 172.21.15.7:0/3055451699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d6de00640 1 -- 172.21.15.7:0/3055451699 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700d0f50 msgr2=0x7f9d700cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d6de00640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700d0f50 0x7f9d700cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d6de00640 1 -- 172.21.15.7:0/3055451699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d7015dfb0 msgr2=0x7f9d70148180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d6de00640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d7015dfb0 0x7f9d70148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.210+0000 7f9d6de00640 1 -- 172.21.15.7:0/3055451699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d700b4df0 con 0x7f9d700d0b40 2024-04-25T11:16:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.211+0000 7f9d6de00640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700d0b40 0x7f9d700b5e90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9d600098c0 tx=0x7f9d600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.211+0000 7f9d66000640 1 -- 172.21.15.7:0/3055451699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60037040 con 0x7f9d700d0b40 2024-04-25T11:16:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.211+0000 7f9d66000640 1 -- 172.21.15.7:0/3055451699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d60030770 con 0x7f9d700d0b40 2024-04-25T11:16:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.211+0000 7f9d66000640 1 -- 172.21.15.7:0/3055451699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60030b40 con 0x7f9d700d0b40 2024-04-25T11:16:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.211+0000 7f9d74c00640 1 -- 172.21.15.7:0/3055451699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d700b50d0 con 0x7f9d700d0b40 2024-04-25T11:16:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.211+0000 7f9d74c00640 1 -- 172.21.15.7:0/3055451699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d700b5690 con 0x7f9d700d0b40 2024-04-25T11:16:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.213+0000 7f9d74c00640 1 -- 172.21.15.7:0/3055451699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d70050740 con 0x7f9d700d0b40 2024-04-25T11:16:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.214+0000 7f9d66000640 1 -- 172.21.15.7:0/3055451699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9d60044020 con 0x7f9d700d0b40 2024-04-25T11:16:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.215+0000 7f9d66000640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d4c0744b0 0x7f9d4c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.215+0000 7f9d66000640 1 -- 172.21.15.7:0/3055451699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f9d600b5810 con 0x7f9d700d0b40 2024-04-25T11:16:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.215+0000 7f9d6d400640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d4c0744b0 0x7f9d4c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.216+0000 7f9d6d400640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d4c0744b0 0x7f9d4c0769c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9d701f95a0 tx=0x7f9d58008330 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.220+0000 7f9d66000640 1 -- 172.21.15.7:0/3055451699 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9d60032e20 con 0x7f9d700d0b40 2024-04-25T11:16:15.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.424+0000 7f9d74c00640 1 -- 172.21.15.7:0/3055451699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f9d70070fb0 con 0x7f9d700d0b40 2024-04-25T11:16:15.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.433+0000 7f9d66000640 1 -- 172.21.15.7:0/3055451699 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v28)=0 v28) ==== 124+0+0 (secure 0 0 0) 0x7f9d60086e20 con 0x7f9d700d0b40 2024-04-25T11:16:15.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 -- 172.21.15.7:0/3055451699 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d4c0744b0 msgr2=0x7f9d4c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d4c0744b0 0x7f9d4c0769c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9d701f95a0 tx=0x7f9d58008330 comp rx=0 tx=0).stop 2024-04-25T11:16:15.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 -- 172.21.15.7:0/3055451699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700d0b40 msgr2=0x7f9d700b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700d0b40 0x7f9d700b5e90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9d600098c0 tx=0x7f9d600046c0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 -- 172.21.15.7:0/3055451699 shutdown_connections 2024-04-25T11:16:15.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9d4c0744b0 0x7f9d4c0769c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d7015dfb0 0x7f9d70148180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d700d0f50 0x7f9d700cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 --2- 172.21.15.7:0/3055451699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d700d0b40 0x7f9d700b5e90 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 -- 172.21.15.7:0/3055451699 >> 172.21.15.7:0/3055451699 conn(0x7f9d700742c0 msgr2=0x7f9d7006cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:15.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 -- 172.21.15.7:0/3055451699 shutdown_connections 2024-04-25T11:16:15.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.436+0000 7f9d74c00640 1 -- 172.21.15.7:0/3055451699 wait complete. 2024-04-25T11:16:15.466 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-04-25T11:16:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.752+0000 7f169b400640 1 Processor -- start 2024-04-25T11:16:15.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.753+0000 7f169b400640 1 -- start start 2024-04-25T11:16:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.755+0000 7f169b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16940affc0 0x7f1694167d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.755+0000 7f169b400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16940b5730 con 0x7f169415aaa0 2024-04-25T11:16:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.755+0000 7f169b400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f169406f5b0 con 0x7f16940d0ee0 2024-04-25T11:16:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.755+0000 7f169b400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f16940af290 con 0x7f16940affc0 2024-04-25T11:16:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.755+0000 7f1698a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f169415aaa0 0x7f16940b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37540/0 (socket says 172.21.15.7:37540) 2024-04-25T11:16:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.755+0000 7f1698a00640 1 -- 172.21.15.7:0/1297848360 learned_addr learned my addr 172.21.15.7:0/1297848360 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:15.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.755+0000 7f1699400640 1 --2- 172.21.15.7:0/1297848360 >> v2:172.21.15.7:3300/0 conn(0x7f16940affc0 0x7f1694167d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.756+0000 7f1693400640 1 -- 172.21.15.7:0/1297848360 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3503565556 0 0) 0x7f1688001110 con 0x7f169415aaa0 2024-04-25T11:16:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.756+0000 7f1693400640 1 -- 172.21.15.7:0/1297848360 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f167c005730 con 0x7f169415aaa0 2024-04-25T11:16:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.756+0000 7f1699400640 1 -- 172.21.15.7:0/1297848360 >> v1:172.21.15.87:6789/0 conn(0x7f16940d0ee0 legacy=0x7f16940b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.756+0000 7f1699400640 1 -- 172.21.15.7:0/1297848360 >> v1:172.21.15.7:6789/0 conn(0x7f169415aaa0 legacy=0x7f16940b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.756+0000 7f1699400640 1 -- 172.21.15.7:0/1297848360 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16940d14a0 con 0x7f16940affc0 2024-04-25T11:16:15.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.757+0000 7f1699400640 1 --2- 172.21.15.7:0/1297848360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16940affc0 0x7f1694167d20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f168c009d90 tx=0x7f168c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42ed84f8e3f0216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:15.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.758+0000 7f1693400640 1 -- 172.21.15.7:0/1297848360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c007740 con 0x7f16940affc0 2024-04-25T11:16:15.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.758+0000 7f1693400640 1 -- 172.21.15.7:0/1297848360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f168c0078e0 con 0x7f16940affc0 2024-04-25T11:16:15.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.758+0000 7f1693400640 1 -- 172.21.15.7:0/1297848360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c007c90 con 0x7f16940affc0 2024-04-25T11:16:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.759+0000 7f169b400640 1 -- 172.21.15.7:0/1297848360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16940affc0 msgr2=0x7f1694167d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.759+0000 7f169b400640 1 --2- 172.21.15.7:0/1297848360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16940affc0 0x7f1694167d20 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f168c009d90 tx=0x7f168c02a660 comp rx=0 tx=0).stop 2024-04-25T11:16:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.759+0000 7f169b400640 1 -- 172.21.15.7:0/1297848360 shutdown_connections 2024-04-25T11:16:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.759+0000 7f169b400640 1 --2- 172.21.15.7:0/1297848360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16940affc0 0x7f1694167d20 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.759+0000 7f169b400640 1 -- 172.21.15.7:0/1297848360 >> 172.21.15.7:0/1297848360 conn(0x7f16940742c0 msgr2=0x7f169406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.759+0000 7f169b400640 1 -- 172.21.15.7:0/1297848360 shutdown_connections 2024-04-25T11:16:15.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.759+0000 7f169b400640 1 -- 172.21.15.7:0/1297848360 wait complete. 2024-04-25T11:16:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.761+0000 7f169b400640 1 Processor -- start 2024-04-25T11:16:15.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.761+0000 7f169b400640 1 -- start start 2024-04-25T11:16:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.763+0000 7f169b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16940affc0 0x7f1694158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.763+0000 7f169b400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16940d0ee0 0x7f1694150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.763+0000 7f169b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f169415aaa0 0x7f169415af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.763+0000 7f169b400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16940afaa0 con 0x7f169415aaa0 2024-04-25T11:16:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.763+0000 7f169b400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f16940af7c0 con 0x7f16940affc0 2024-04-25T11:16:15.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.763+0000 7f169b400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f169406fc30 con 0x7f16940d0ee0 2024-04-25T11:16:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.763+0000 7f1699400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f169415aaa0 0x7f169415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.763+0000 7f1699400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f169415aaa0 0x7f169415af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48846/0 (socket says 172.21.15.7:48846) 2024-04-25T11:16:15.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.763+0000 7f1699400640 1 -- 172.21.15.7:0/3617357923 learned_addr learned my addr 172.21.15.7:0/3617357923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1698a00640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16940affc0 0x7f1694158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1693e00640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16940d0ee0 0x7f1694150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1699400640 1 -- 172.21.15.7:0/3617357923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16940d0ee0 msgr2=0x7f1694150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1699400640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16940d0ee0 0x7f1694150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1699400640 1 -- 172.21.15.7:0/3617357923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16940affc0 msgr2=0x7f1694158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1699400640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16940affc0 0x7f1694158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1699400640 1 -- 172.21.15.7:0/3617357923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1694158d90 con 0x7f169415aaa0 2024-04-25T11:16:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1698a00640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16940affc0 0x7f1694158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1699400640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f169415aaa0 0x7f169415af10 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f168c009d90 tx=0x7f168c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1690c00640 1 -- 172.21.15.7:0/3617357923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c03f070 con 0x7f169415aaa0 2024-04-25T11:16:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1690c00640 1 -- 172.21.15.7:0/3617357923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f168c031c40 con 0x7f169415aaa0 2024-04-25T11:16:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.764+0000 7f1690c00640 1 -- 172.21.15.7:0/3617357923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c009450 con 0x7f169415aaa0 2024-04-25T11:16:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.765+0000 7f169b400640 1 -- 172.21.15.7:0/3617357923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1694159070 con 0x7f169415aaa0 2024-04-25T11:16:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.765+0000 7f169b400640 1 -- 172.21.15.7:0/3617357923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1694159630 con 0x7f169415aaa0 2024-04-25T11:16:15.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.766+0000 7f169b400640 1 -- 172.21.15.7:0/3617357923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1694050740 con 0x7f169415aaa0 2024-04-25T11:16:15.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.767+0000 7f1690c00640 1 -- 172.21.15.7:0/3617357923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f168c003990 con 0x7f169415aaa0 2024-04-25T11:16:15.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.768+0000 7f1690c00640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16740744b0 0x7f16740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:15.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.768+0000 7f1690c00640 1 -- 172.21.15.7:0/3617357923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f168c0b64f0 con 0x7f169415aaa0 2024-04-25T11:16:15.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.769+0000 7f1698a00640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16740744b0 0x7f16740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:15.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.769+0000 7f1698a00640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16740744b0 0x7f16740769c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f16880013b0 tx=0x7f1688009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:15.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.773+0000 7f1690c00640 1 -- 172.21.15.7:0/3617357923 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f168c080ae0 con 0x7f169415aaa0 2024-04-25T11:16:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.982+0000 7f169b400640 1 -- 172.21.15.7:0/3617357923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f169415cd50 con 0x7f169415aaa0 2024-04-25T11:16:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.982+0000 7f1690c00640 1 -- 172.21.15.7:0/3617357923 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v28) ==== 106+0+4 (secure 0 0 0) 0x7f168c0803e0 con 0x7f169415aaa0 2024-04-25T11:16:15.983 INFO:teuthology.orchestra.run.smithi007.stdout:100 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.985+0000 7f169b400640 1 -- 172.21.15.7:0/3617357923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16740744b0 msgr2=0x7f16740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.985+0000 7f169b400640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16740744b0 0x7f16740769c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f16880013b0 tx=0x7f1688009040 comp rx=0 tx=0).stop 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.985+0000 7f169b400640 1 -- 172.21.15.7:0/3617357923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f169415aaa0 msgr2=0x7f169415af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.985+0000 7f169b400640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f169415aaa0 0x7f169415af10 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f168c009d90 tx=0x7f168c004a40 comp rx=0 tx=0).stop 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.985+0000 7f169b400640 1 -- 172.21.15.7:0/3617357923 shutdown_connections 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.985+0000 7f169b400640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f16740744b0 0x7f16740769c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.985+0000 7f169b400640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f169415aaa0 0x7f169415af10 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.985+0000 7f169b400640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16940d0ee0 0x7f1694150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.985+0000 7f169b400640 1 --2- 172.21.15.7:0/3617357923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16940affc0 0x7f1694158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.985+0000 7f169b400640 1 -- 172.21.15.7:0/3617357923 >> 172.21.15.7:0/3617357923 conn(0x7f16940742c0 msgr2=0x7f16940cdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.986+0000 7f169b400640 1 -- 172.21.15.7:0/3617357923 shutdown_connections 2024-04-25T11:16:15.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:15.986+0000 7f169b400640 1 -- 172.21.15.7:0/3617357923 wait complete. 2024-04-25T11:16:16.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-04-25T11:16:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.305+0000 7f9061a00640 1 Processor -- start 2024-04-25T11:16:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.305+0000 7f9061a00640 1 -- start start 2024-04-25T11:16:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.307+0000 7f9061a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f905c145e40 0x7f905c146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.307+0000 7f9061a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f905c0b4870 0x7f905c0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.307+0000 7f9061a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f905c0b51c0 0x7f905c143a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.307+0000 7f9061a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f905c145b70 con 0x7f905c0b4870 2024-04-25T11:16:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.307+0000 7f9061a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f905c0b66b0 con 0x7f905c145e40 2024-04-25T11:16:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.307+0000 7f9061a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f905c0b5b30 con 0x7f905c0b51c0 2024-04-25T11:16:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.307+0000 7f905b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f905c0b4870 0x7f905c0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.308+0000 7f905b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f905c0b4870 0x7f905c0b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48850/0 (socket says 172.21.15.7:48850) 2024-04-25T11:16:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.308+0000 7f905b400640 1 -- 172.21.15.7:0/745360976 learned_addr learned my addr 172.21.15.7:0/745360976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.308+0000 7f9061000640 1 --2- 172.21.15.7:0/745360976 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f905c0b51c0 0x7f905c143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.308+0000 7f905be00640 1 --2- 172.21.15.7:0/745360976 >> v2:172.21.15.87:3300/0 conn(0x7f905c145e40 0x7f905c146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.308+0000 7f905b400640 1 -- 172.21.15.7:0/745360976 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f905c0b51c0 msgr2=0x7f905c143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.308+0000 7f905b400640 1 --2- 172.21.15.7:0/745360976 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f905c0b51c0 0x7f905c143a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.308+0000 7f905b400640 1 -- 172.21.15.7:0/745360976 >> v2:172.21.15.87:3300/0 conn(0x7f905c145e40 msgr2=0x7f905c146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.308+0000 7f905b400640 1 --2- 172.21.15.7:0/745360976 >> v2:172.21.15.87:3300/0 conn(0x7f905c145e40 0x7f905c146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.308+0000 7f905b400640 1 -- 172.21.15.7:0/745360976 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c0b5e10 con 0x7f905c0b4870 2024-04-25T11:16:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.309+0000 7f905b400640 1 --2- 172.21.15.7:0/745360976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f905c0b4870 0x7f905c0b4c80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f905c06ded0 tx=0x7f904802c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf981660d56d8078 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.310+0000 7f905aa00640 1 -- 172.21.15.7:0/745360976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048039070 con 0x7f905c0b4870 2024-04-25T11:16:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.310+0000 7f905aa00640 1 -- 172.21.15.7:0/745360976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9048034640 con 0x7f905c0b4870 2024-04-25T11:16:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.310+0000 7f905aa00640 1 -- 172.21.15.7:0/745360976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048034a10 con 0x7f905c0b4870 2024-04-25T11:16:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.310+0000 7f9061a00640 1 -- 172.21.15.7:0/745360976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f905c0b4870 msgr2=0x7f905c0b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.310+0000 7f9061a00640 1 --2- 172.21.15.7:0/745360976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f905c0b4870 0x7f905c0b4c80 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f905c06ded0 tx=0x7f904802c350 comp rx=0 tx=0).stop 2024-04-25T11:16:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.310+0000 7f9061a00640 1 -- 172.21.15.7:0/745360976 shutdown_connections 2024-04-25T11:16:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.310+0000 7f9061a00640 1 --2- 172.21.15.7:0/745360976 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f905c0b51c0 0x7f905c143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.310+0000 7f9061a00640 1 --2- 172.21.15.7:0/745360976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f905c0b4870 0x7f905c0b4c80 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.310+0000 7f9061a00640 1 --2- 172.21.15.7:0/745360976 >> v2:172.21.15.87:3300/0 conn(0x7f905c145e40 0x7f905c146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.310+0000 7f9061a00640 1 -- 172.21.15.7:0/745360976 >> 172.21.15.7:0/745360976 conn(0x7f905c0740f0 msgr2=0x7f905c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.311+0000 7f9061a00640 1 -- 172.21.15.7:0/745360976 shutdown_connections 2024-04-25T11:16:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.311+0000 7f9061a00640 1 -- 172.21.15.7:0/745360976 wait complete. 2024-04-25T11:16:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.313+0000 7f9061a00640 1 Processor -- start 2024-04-25T11:16:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.313+0000 7f9061a00640 1 -- start start 2024-04-25T11:16:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.315+0000 7f9061a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f905c0b4870 0x7f905c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.315+0000 7f9061a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f905c0b51c0 0x7f905c1d6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.315+0000 7f9061a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f905c145e40 0x7f905c1d7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.315+0000 7f9061a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f905c0b55d0 con 0x7f905c0b4870 2024-04-25T11:16:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.315+0000 7f9061a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f905c0b63d0 con 0x7f905c0b51c0 2024-04-25T11:16:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.315+0000 7f9061a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f905c0b5750 con 0x7f905c145e40 2024-04-25T11:16:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.315+0000 7f905be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f905c0b4870 0x7f905c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.315+0000 7f905be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f905c0b4870 0x7f905c0b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48862/0 (socket says 172.21.15.7:48862) 2024-04-25T11:16:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.315+0000 7f905be00640 1 -- 172.21.15.7:0/1956086562 learned_addr learned my addr 172.21.15.7:0/1956086562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.316+0000 7f9061000640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f905c145e40 0x7f905c1d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.316+0000 7f905b400640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f905c0b51c0 0x7f905c1d6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.316+0000 7f905be00640 1 -- 172.21.15.7:0/1956086562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f905c145e40 msgr2=0x7f905c1d7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.316+0000 7f905be00640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f905c145e40 0x7f905c1d7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.316+0000 7f905be00640 1 -- 172.21.15.7:0/1956086562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f905c0b51c0 msgr2=0x7f905c1d6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.316+0000 7f905be00640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f905c0b51c0 0x7f905c1d6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.316+0000 7f905be00640 1 -- 172.21.15.7:0/1956086562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c0b30d0 con 0x7f905c0b4870 2024-04-25T11:16:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.316+0000 7f905be00640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f905c0b4870 0x7f905c0b2b90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f904c00d5f0 tx=0x7f904c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.317+0000 7f9053e00640 1 -- 172.21.15.7:0/1956086562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c0125f0 con 0x7f905c0b4870 2024-04-25T11:16:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.317+0000 7f9053e00640 1 -- 172.21.15.7:0/1956086562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904c019070 con 0x7f905c0b4870 2024-04-25T11:16:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.317+0000 7f9053e00640 1 -- 172.21.15.7:0/1956086562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c0129d0 con 0x7f905c0b4870 2024-04-25T11:16:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.317+0000 7f9061a00640 1 -- 172.21.15.7:0/1956086562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905c1d7ae0 con 0x7f905c0b4870 2024-04-25T11:16:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.317+0000 7f9061a00640 1 -- 172.21.15.7:0/1956086562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905c1d51a0 con 0x7f905c0b4870 2024-04-25T11:16:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.318+0000 7f9061a00640 1 -- 172.21.15.7:0/1956086562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f905c050740 con 0x7f905c0b4870 2024-04-25T11:16:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.319+0000 7f9053e00640 1 -- 172.21.15.7:0/1956086562 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f904c012b70 con 0x7f905c0b4870 2024-04-25T11:16:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.320+0000 7f9053e00640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f903c0744e0 0x7f903c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.320+0000 7f9053e00640 1 -- 172.21.15.7:0/1956086562 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f904c095850 con 0x7f905c0b4870 2024-04-25T11:16:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.321+0000 7f905b400640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f903c0744e0 0x7f903c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.321+0000 7f905b400640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f903c0744e0 0x7f903c0769f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9048036000 tx=0x7f9048030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.325+0000 7f9053e00640 1 -- 172.21.15.7:0/1956086562 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f904c05fe50 con 0x7f905c0b4870 2024-04-25T11:16:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.528+0000 7f9061a00640 1 -- 172.21.15.7:0/1956086562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f905c145110 con 0x7f905c0b4870 2024-04-25T11:16:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.529+0000 7f9053e00640 1 -- 172.21.15.7:0/1956086562 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v28) ==== 112+0+10 (secure 0 0 0) 0x7f904c05f750 con 0x7f905c0b4870 2024-04-25T11:16:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:60.000000 2024-04-25T11:16:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 -- 172.21.15.7:0/1956086562 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f903c0744e0 msgr2=0x7f903c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f903c0744e0 0x7f903c0769f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9048036000 tx=0x7f9048030000 comp rx=0 tx=0).stop 2024-04-25T11:16:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 -- 172.21.15.7:0/1956086562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f905c0b4870 msgr2=0x7f905c0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f905c0b4870 0x7f905c0b2b90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f904c00d5f0 tx=0x7f904c00dab0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 -- 172.21.15.7:0/1956086562 shutdown_connections 2024-04-25T11:16:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f903c0744e0 0x7f903c0769f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f905c145e40 0x7f905c1d7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f905c0b51c0 0x7f905c1d6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 --2- 172.21.15.7:0/1956086562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f905c0b4870 0x7f905c0b2b90 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 -- 172.21.15.7:0/1956086562 >> 172.21.15.7:0/1956086562 conn(0x7f905c0740f0 msgr2=0x7f905c143970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 -- 172.21.15.7:0/1956086562 shutdown_connections 2024-04-25T11:16:16.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16.532+0000 7f9061a00640 1 -- 172.21.15.7:0/1956086562 wait complete. 2024-04-25T11:16:16.560 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:16:16.560 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:16:16.613 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:16:16.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:16:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:16:16.734 INFO:teuthology.orchestra.run.smithi007.stdout:client6205 2024-04-25T11:16:16.734 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:16:16.735 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> import fcntl 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> import sys 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> n = 400 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> if not os.path.exists(abs_path): 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(abs_path) 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> handles = [] 2024-04-25T11:16:16.789 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(0, n): 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> fname = "file_"+str(i) 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> path = os.path.join(abs_path, fname) 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> handles.append(open(path, '"'"'w'"'"')) 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> print("waiting with handles open", file=sys.stderr) 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> print("open_n_background: keeping files open", file=sys.stderr) 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> if os.read(0, 4096) == b"": 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> except BlockingIOError: 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> pass 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(2) 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> print("stdin closed, goodbye!", file=sys.stderr) 2024-04-25T11:16:16.790 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-04-25T11:16:16.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:16:17.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.058+0000 7f6f3ea00640 1 Processor -- start 2024-04-25T11:16:17.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.058+0000 7f6f3ea00640 1 -- start start 2024-04-25T11:16:17.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.060+0000 7f6f3ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f380cf720 0x7f6f3806e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.060+0000 7f6f3ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6f3815d5b0 0x7f6f3815d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.060+0000 7f6f3ea00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f3806e9d0 con 0x7f6f38160850 2024-04-25T11:16:17.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.060+0000 7f6f3ea00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6f3806efb0 con 0x7f6f380cf720 2024-04-25T11:16:17.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.060+0000 7f6f3ea00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6f380ceeb0 con 0x7f6f3815d5b0 2024-04-25T11:16:17.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.061+0000 7f6f3cc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6f38160850 0x7f6f380b75b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37546/0 (socket says 172.21.15.7:37546) 2024-04-25T11:16:17.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.061+0000 7f6f3cc00640 1 -- 172.21.15.7:0/2705347442 learned_addr learned my addr 172.21.15.7:0/2705347442 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:17.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.061+0000 7f6f3d600640 1 --2- 172.21.15.7:0/2705347442 >> v2:172.21.15.7:3300/0 conn(0x7f6f380cf720 0x7f6f3806e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.061+0000 7f6f37e00640 1 -- 172.21.15.7:0/2705347442 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3386441776 0 0) 0x7f6f3806e9d0 con 0x7f6f38160850 2024-04-25T11:16:17.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.061+0000 7f6f37e00640 1 -- 172.21.15.7:0/2705347442 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f20003520 con 0x7f6f38160850 2024-04-25T11:16:17.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.061+0000 7f6f3e000640 1 --2- 172.21.15.7:0/2705347442 >> v2:172.21.15.87:3300/0 conn(0x7f6f3815d5b0 0x7f6f3815d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.062+0000 7f6f3d600640 1 -- 172.21.15.7:0/2705347442 >> v2:172.21.15.87:3300/0 conn(0x7f6f3815d5b0 msgr2=0x7f6f3815d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:17.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.062+0000 7f6f3d600640 1 --2- 172.21.15.7:0/2705347442 >> v2:172.21.15.87:3300/0 conn(0x7f6f3815d5b0 0x7f6f3815d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.062+0000 7f6f3d600640 1 -- 172.21.15.7:0/2705347442 >> v1:172.21.15.7:6789/0 conn(0x7f6f38160850 legacy=0x7f6f380b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.062+0000 7f6f3d600640 1 -- 172.21.15.7:0/2705347442 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f380cf100 con 0x7f6f380cf720 2024-04-25T11:16:17.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.062+0000 7f6f3e000640 1 --2- 172.21.15.7:0/2705347442 >> v2:172.21.15.87:3300/0 conn(0x7f6f3815d5b0 0x7f6f3815d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.063+0000 7f6f3d600640 1 --2- 172.21.15.7:0/2705347442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380cf720 0x7f6f3806e090 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f6f3806fe10 tx=0x7f6f2c02c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bbd2a1d914d75c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.063+0000 7f6f37e00640 1 -- 172.21.15.7:0/2705347442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c039070 con 0x7f6f380cf720 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.063+0000 7f6f37e00640 1 -- 172.21.15.7:0/2705347442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c034640 con 0x7f6f380cf720 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.063+0000 7f6f37e00640 1 -- 172.21.15.7:0/2705347442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0349f0 con 0x7f6f380cf720 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.064+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2705347442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380cf720 msgr2=0x7f6f3806e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.064+0000 7f6f3ea00640 1 --2- 172.21.15.7:0/2705347442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380cf720 0x7f6f3806e090 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f6f3806fe10 tx=0x7f6f2c02c360 comp rx=0 tx=0).stop 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.064+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2705347442 shutdown_connections 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.064+0000 7f6f3ea00640 1 --2- 172.21.15.7:0/2705347442 >> v2:172.21.15.87:3300/0 conn(0x7f6f3815d5b0 0x7f6f3815d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.064+0000 7f6f3ea00640 1 --2- 172.21.15.7:0/2705347442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f380cf720 0x7f6f3806e090 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.064+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2705347442 >> 172.21.15.7:0/2705347442 conn(0x7f6f3809b540 msgr2=0x7f6f3809b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.064+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2705347442 shutdown_connections 2024-04-25T11:16:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.065+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2705347442 wait complete. 2024-04-25T11:16:17.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.067+0000 7f6f3ea00640 1 Processor -- start 2024-04-25T11:16:17.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.067+0000 7f6f3ea00640 1 -- start start 2024-04-25T11:16:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.068+0000 7f6f3ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f380cf720 0x7f6f3815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.068+0000 7f6f3ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f3815d5b0 0x7f6f38149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.068+0000 7f6f3ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f38160850 0x7f6f3815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.068+0000 7f6f3ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6f380cf3e0 con 0x7f6f38160850 2024-04-25T11:16:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3ea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6f380b7cc0 con 0x7f6f380cf720 2024-04-25T11:16:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3ea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6f38160c60 con 0x7f6f3815d5b0 2024-04-25T11:16:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f38160850 0x7f6f3815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f38160850 0x7f6f3815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48868/0 (socket says 172.21.15.7:48868) 2024-04-25T11:16:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3e000640 1 -- 172.21.15.7:0/2256378941 learned_addr learned my addr 172.21.15.7:0/2256378941 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3d600640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f380cf720 0x7f6f3815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3e000640 1 -- 172.21.15.7:0/2256378941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f3815d5b0 msgr2=0x7f6f38149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3cc00640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f3815d5b0 0x7f6f38149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3e000640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f3815d5b0 0x7f6f38149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3e000640 1 -- 172.21.15.7:0/2256378941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f380cf720 msgr2=0x7f6f3815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3e000640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f380cf720 0x7f6f3815c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3e000640 1 -- 172.21.15.7:0/2256378941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3814a510 con 0x7f6f38160850 2024-04-25T11:16:17.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3cc00640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f3815d5b0 0x7f6f38149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:17.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.069+0000 7f6f3d600640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f380cf720 0x7f6f3815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:17.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.070+0000 7f6f3e000640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f38160850 0x7f6f3815ac70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000cdb0 tx=0x7f6f3000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:17.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.070+0000 7f6f35600640 1 -- 172.21.15.7:0/2256378941 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30003b90 con 0x7f6f38160850 2024-04-25T11:16:17.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.070+0000 7f6f35600640 1 -- 172.21.15.7:0/2256378941 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f30003d50 con 0x7f6f38160850 2024-04-25T11:16:17.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.070+0000 7f6f35600640 1 -- 172.21.15.7:0/2256378941 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30004260 con 0x7f6f38160850 2024-04-25T11:16:17.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.070+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2256378941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f380c6490 con 0x7f6f38160850 2024-04-25T11:16:17.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.070+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2256378941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f380c8130 con 0x7f6f38160850 2024-04-25T11:16:17.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.072+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2256378941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f380c61f0 con 0x7f6f38160850 2024-04-25T11:16:17.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.074+0000 7f6f35600640 1 -- 172.21.15.7:0/2256378941 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6f30004400 con 0x7f6f38160850 2024-04-25T11:16:17.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.075+0000 7f6f35600640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f180744b0 0x7f6f180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.075+0000 7f6f35600640 1 -- 172.21.15.7:0/2256378941 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f6f30095d40 con 0x7f6f38160850 2024-04-25T11:16:17.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.075+0000 7f6f3d600640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f180744b0 0x7f6f180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.076+0000 7f6f3d600640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f180744b0 0x7f6f180769c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c036000 tx=0x7f6f2c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:17.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.080+0000 7f6f35600640 1 -- 172.21.15.7:0/2256378941 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6f380c61f0 con 0x7f6f38160850 2024-04-25T11:16:17.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.375+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2256378941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f38050740 con 0x7f6f38160850 2024-04-25T11:16:17.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.376+0000 7f6f35600640 1 -- 172.21.15.7:0/2256378941 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5168 (secure 0 0 0) 0x7f6f30014030 con 0x7f6f38160850 2024-04-25T11:16:17.376 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:17.377 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/684015857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":684015857},{"type":"v1","addr":"172.21.15.7:6836","nonce":684015857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/493684223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":493684223},{"type":"v1","addr":"172.21.15.87:6835","nonce":493684223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6109,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/1959070928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1959070928},{"type":"v1","addr":"172.21.15.87:6833","nonce":1959070928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:16:01.820362+0000","modified":"2024-04-25T11:16:04.870476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/336889324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":336889324},{"type":"v1","addr":"172.21.15.7:6837","nonce":336889324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6960,"qdb_cluster":[6960]},"id":6}]} 2024-04-25T11:16:17.377 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2024-04-25T11:16:17.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.379+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2256378941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f180744b0 msgr2=0x7f6f180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.379+0000 7f6f3ea00640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f180744b0 0x7f6f180769c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c036000 tx=0x7f6f2c030000 comp rx=0 tx=0).stop 2024-04-25T11:16:17.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.380+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2256378941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f38160850 msgr2=0x7f6f3815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.380+0000 7f6f3ea00640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f38160850 0x7f6f3815ac70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000cdb0 tx=0x7f6f3000a650 comp rx=0 tx=0).stop 2024-04-25T11:16:17.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.380+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2256378941 shutdown_connections 2024-04-25T11:16:17.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.380+0000 7f6f3ea00640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6f180744b0 0x7f6f180769c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.380+0000 7f6f3ea00640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f38160850 0x7f6f3815ac70 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.380+0000 7f6f3ea00640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6f3815d5b0 0x7f6f38149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.380+0000 7f6f3ea00640 1 --2- 172.21.15.7:0/2256378941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f380cf720 0x7f6f3815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.380+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2256378941 >> 172.21.15.7:0/2256378941 conn(0x7f6f3809b540 msgr2=0x7f6f380d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:17.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.380+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2256378941 shutdown_connections 2024-04-25T11:16:17.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.380+0000 7f6f3ea00640 1 -- 172.21.15.7:0/2256378941 wait complete. 2024-04-25T11:16:17.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T11:16:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:waiting with handles open 2024-04-25T11:16:17.504 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:17.697 INFO:teuthology.orchestra.run.smithi007.stdout:[{"id":7110,"entity":{"name":{"type":"client","num":7110},"addr":{"type":"any","addr":"172.21.15.87:0","nonce":365669307}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":4.7039249190000003,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9857184629780731,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.7110 172.21.15.87:0/365669307","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi087","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":6205,"entity":{"name":{"type":"client","num":6205},"addr":{"type":"any","addr":"172.21.15.7:0","nonce":2025869673}},"state":"open","num_leases":1,"num_caps":402,"request_load_avg":424,"uptime":6.4988962690000003,"requests_in_flight":400,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":120.43937732608636,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":100.48717084108519,"halflife":1.5},"recall_caps_throttle2o":{"value":69.304449251005849,"halflife":0.5},"session_cache_liveness":{"value":804.16726636190174,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6205 172.21.15.7:0/2025869673","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi007","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T11:16:17.698 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi087", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.87:0", "nonce": 365669307, "type": "any" }, "name": { "num": 7110, "type": "client" } }, "id": 7110, "inst": "client.7110 172.21.15.87:0/365669307", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.985718462978073 }, "state": "open", "uptime": 4.703924919 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi007", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.7:0", "nonce": 2025869673, "type": "any" }, "name": { "num": 6205, "type": "client" } }, "id": 6205, "inst": "client.6205 172.21.15.7:0/2025869673", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 402, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 120.43937732608636 }, "recall_caps_throttle": { "halflife": 1.5, "value": 100.48717084108519 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 69.30444925100585 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 424, "requests_in_flight": 400, "session_cache_liveness": { "halflife": 300, "value": 804.1672663619017 }, "state": "open", "uptime": 6.498896269 } ] 2024-04-25T11:16:17.698 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-04-25T11:16:17.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:16:17.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.969+0000 7f8106400640 1 Processor -- start 2024-04-25T11:16:17.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.969+0000 7f8106400640 1 -- start start 2024-04-25T11:16:17.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.971+0000 7f8106400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81000b2a90 0x7f810006d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.971+0000 7f8106400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f810006dda0 0x7f8100164ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.971+0000 7f8106400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81000d0f50 con 0x7f81000b9050 2024-04-25T11:16:17.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.971+0000 7f8106400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f81000d0d00 con 0x7f810006dda0 2024-04-25T11:16:17.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.971+0000 7f8106400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f810009c970 con 0x7f81000b2a90 2024-04-25T11:16:17.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.972+0000 7f80ffe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f81000b9050 0x7f8100164490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37550/0 (socket says 172.21.15.7:37550) 2024-04-25T11:16:17.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.972+0000 7f8105a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f810006dda0 0x7f8100164ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.972+0000 7f80ffe00640 1 -- 172.21.15.7:0/1826360329 learned_addr learned my addr 172.21.15.7:0/1826360329 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:17.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.972+0000 7f8105000640 1 --2- 172.21.15.7:0/1826360329 >> v2:172.21.15.87:3300/0 conn(0x7f81000b2a90 0x7f810006d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.972+0000 7f8105a00640 1 -- 172.21.15.7:0/1826360329 >> v2:172.21.15.87:3300/0 conn(0x7f81000b2a90 msgr2=0x7f810006d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:17.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.972+0000 7f8105a00640 1 --2- 172.21.15.7:0/1826360329 >> v2:172.21.15.87:3300/0 conn(0x7f81000b2a90 0x7f810006d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.972+0000 7f8105a00640 1 -- 172.21.15.7:0/1826360329 >> v1:172.21.15.7:6789/0 conn(0x7f81000b9050 legacy=0x7f8100164490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.972+0000 7f8105a00640 1 -- 172.21.15.7:0/1826360329 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810009cbc0 con 0x7f810006dda0 2024-04-25T11:16:17.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.972+0000 7f80ff400640 1 -- 172.21.15.7:0/1826360329 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 828299454 0 0) 0x7f81000d0f50 con 0x7f81000b9050 2024-04-25T11:16:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.973+0000 7f8105a00640 1 --2- 172.21.15.7:0/1826360329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f810006dda0 0x7f8100164ba0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8100073d60 tx=0x7f80f802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc17e343b8369dd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.974+0000 7f80ff400640 1 -- 172.21.15.7:0/1826360329 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8007740 con 0x7f810006dda0 2024-04-25T11:16:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.974+0000 7f80ff400640 1 -- 172.21.15.7:0/1826360329 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f80078e0 con 0x7f810006dda0 2024-04-25T11:16:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.974+0000 7f80ff400640 1 -- 172.21.15.7:0/1826360329 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8007cb0 con 0x7f810006dda0 2024-04-25T11:16:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.974+0000 7f8106400640 1 -- 172.21.15.7:0/1826360329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f810006dda0 msgr2=0x7f8100164ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.974+0000 7f8106400640 1 --2- 172.21.15.7:0/1826360329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f810006dda0 0x7f8100164ba0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8100073d60 tx=0x7f80f802a5f0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.974+0000 7f8106400640 1 -- 172.21.15.7:0/1826360329 shutdown_connections 2024-04-25T11:16:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.974+0000 7f8106400640 1 --2- 172.21.15.7:0/1826360329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f810006dda0 0x7f8100164ba0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.974+0000 7f8106400640 1 --2- 172.21.15.7:0/1826360329 >> v2:172.21.15.87:3300/0 conn(0x7f81000b2a90 0x7f810006d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.974+0000 7f8106400640 1 -- 172.21.15.7:0/1826360329 >> 172.21.15.7:0/1826360329 conn(0x7f81000d01e0 msgr2=0x7f81000d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.975+0000 7f8106400640 1 -- 172.21.15.7:0/1826360329 shutdown_connections 2024-04-25T11:16:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.975+0000 7f8106400640 1 -- 172.21.15.7:0/1826360329 wait complete. 2024-04-25T11:16:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.977+0000 7f8106400640 1 Processor -- start 2024-04-25T11:16:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.977+0000 7f8106400640 1 -- start start 2024-04-25T11:16:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.978+0000 7f8106400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f810006dda0 0x7f8100163d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.978+0000 7f8106400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81000b2a90 0x7f81001f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.978+0000 7f8106400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81000b9050 0x7f81000ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.978+0000 7f8106400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81000b4290 con 0x7f81000b9050 2024-04-25T11:16:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.978+0000 7f8106400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81000b4110 con 0x7f810006dda0 2024-04-25T11:16:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.978+0000 7f8106400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f81000b4410 con 0x7f81000b2a90 2024-04-25T11:16:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.979+0000 7f8105a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81000b9050 0x7f81000ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.979+0000 7f8105a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81000b9050 0x7f81000ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48880/0 (socket says 172.21.15.7:48880) 2024-04-25T11:16:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.979+0000 7f8105a00640 1 -- 172.21.15.7:0/1227192254 learned_addr learned my addr 172.21.15.7:0/1227192254 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.979+0000 7f80ffe00640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81000b2a90 0x7f81001f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.979+0000 7f8105000640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f810006dda0 0x7f8100163d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.979+0000 7f8105a00640 1 -- 172.21.15.7:0/1227192254 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81000b2a90 msgr2=0x7f81001f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.979+0000 7f8105a00640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81000b2a90 0x7f81001f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.979+0000 7f8105a00640 1 -- 172.21.15.7:0/1227192254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f810006dda0 msgr2=0x7f8100163d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.979+0000 7f8105a00640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f810006dda0 0x7f8100163d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.979+0000 7f8105a00640 1 -- 172.21.15.7:0/1227192254 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81000cbf40 con 0x7f81000b9050 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.979+0000 7f80ffe00640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81000b2a90 0x7f81001f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.980+0000 7f8105a00640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81000b9050 0x7f81000ccfe0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f80f802abe0 tx=0x7f80f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.980+0000 7f80fcc00640 1 -- 172.21.15.7:0/1227192254 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f803f070 con 0x7f81000b9050 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.980+0000 7f80fcc00640 1 -- 172.21.15.7:0/1227192254 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f8031c40 con 0x7f81000b9050 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.980+0000 7f80fcc00640 1 -- 172.21.15.7:0/1227192254 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8009450 con 0x7f81000b9050 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.980+0000 7f8106400640 1 -- 172.21.15.7:0/1227192254 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81000cc220 con 0x7f81000b9050 2024-04-25T11:16:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.980+0000 7f8106400640 1 -- 172.21.15.7:0/1227192254 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81000cc7e0 con 0x7f81000b9050 2024-04-25T11:16:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.982+0000 7f8106400640 1 -- 172.21.15.7:0/1227192254 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8100050740 con 0x7f81000b9050 2024-04-25T11:16:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.983+0000 7f80fcc00640 1 -- 172.21.15.7:0/1227192254 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f80f8003990 con 0x7f81000b9050 2024-04-25T11:16:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.984+0000 7f80fcc00640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80e00744e0 0x7f80e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.984+0000 7f80fcc00640 1 -- 172.21.15.7:0/1227192254 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f80f80b6560 con 0x7f81000b9050 2024-04-25T11:16:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.984+0000 7f8105000640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80e00744e0 0x7f80e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.985+0000 7f8105000640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80e00744e0 0x7f80e00769f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f80f4006fd0 tx=0x7f80f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:17.989+0000 7f80fcc00640 1 -- 172.21.15.7:0/1227192254 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f80f8080b60 con 0x7f81000b9050 2024-04-25T11:16:18.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.225+0000 7f8106400640 1 -- 172.21.15.7:0/1227192254 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f81000b8c00 con 0x7f81000b9050 2024-04-25T11:16:18.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.226+0000 7f80fcc00640 1 -- 172.21.15.7:0/1227192254 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f80f8080460 con 0x7f81000b9050 2024-04-25T11:16:18.226 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:18.226 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.228+0000 7f8106400640 1 -- 172.21.15.7:0/1227192254 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80e00744e0 msgr2=0x7f80e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.228+0000 7f8106400640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80e00744e0 0x7f80e00769f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f80f4006fd0 tx=0x7f80f4008040 comp rx=0 tx=0).stop 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.228+0000 7f8106400640 1 -- 172.21.15.7:0/1227192254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81000b9050 msgr2=0x7f81000ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.228+0000 7f8106400640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81000b9050 0x7f81000ccfe0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f80f802abe0 tx=0x7f80f8004a40 comp rx=0 tx=0).stop 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.228+0000 7f8106400640 1 -- 172.21.15.7:0/1227192254 shutdown_connections 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.228+0000 7f8106400640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80e00744e0 0x7f80e00769f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.228+0000 7f8106400640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81000b9050 0x7f81000ccfe0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.228+0000 7f8106400640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f81000b2a90 0x7f81001f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.228+0000 7f8106400640 1 --2- 172.21.15.7:0/1227192254 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f810006dda0 0x7f8100163d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.229+0000 7f8106400640 1 -- 172.21.15.7:0/1227192254 >> 172.21.15.7:0/1227192254 conn(0x7f81000d01e0 msgr2=0x7f8100071390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.229+0000 7f8106400640 1 -- 172.21.15.7:0/1227192254 shutdown_connections 2024-04-25T11:16:18.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:18.229+0000 7f8106400640 1 -- 172.21.15.7:0/1227192254 wait complete. 2024-04-25T11:16:18.254 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-04-25T11:16:18.254 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:16:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:21.508 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:23.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:16:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.548+0000 7fc45ca00640 1 Processor -- start 2024-04-25T11:16:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.548+0000 7fc45ca00640 1 -- start start 2024-04-25T11:16:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.550+0000 7fc45ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc458070810 0x7fc45806d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.550+0000 7fc45ca00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4580cffa0 con 0x7fc458070c20 2024-04-25T11:16:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.550+0000 7fc45ca00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc45806ded0 con 0x7fc45806dac0 2024-04-25T11:16:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.550+0000 7fc45ca00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc458160d90 con 0x7fc458070810 2024-04-25T11:16:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.550+0000 7fc457400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc458070810 0x7fc45806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.550+0000 7fc456a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc458070c20 0x7fc45815d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40712/0 (socket says 172.21.15.7:40712) 2024-04-25T11:16:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.550+0000 7fc456a00640 1 -- 172.21.15.7:0/1945570219 learned_addr learned my addr 172.21.15.7:0/1945570219 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.551+0000 7fc457400640 1 -- 172.21.15.7:0/1945570219 >> v1:172.21.15.87:6789/0 conn(0x7fc45806dac0 legacy=0x7fc45815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.551+0000 7fc457400640 1 -- 172.21.15.7:0/1945570219 >> v1:172.21.15.7:6789/0 conn(0x7fc458070c20 legacy=0x7fc45815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.551+0000 7fc457400640 1 -- 172.21.15.7:0/1945570219 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc458161070 con 0x7fc458070810 2024-04-25T11:16:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.552+0000 7fc457400640 1 --2- 172.21.15.7:0/1945570219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc458070810 0x7fc45806d580 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc4400097a0 tx=0x7fc44002c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f3817253a45e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.552+0000 7fc456000640 1 -- 172.21.15.7:0/1945570219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440039070 con 0x7fc458070810 2024-04-25T11:16:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.552+0000 7fc456000640 1 -- 172.21.15.7:0/1945570219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc440034640 con 0x7fc458070810 2024-04-25T11:16:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.552+0000 7fc456000640 1 -- 172.21.15.7:0/1945570219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440034a10 con 0x7fc458070810 2024-04-25T11:16:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.553+0000 7fc45ca00640 1 -- 172.21.15.7:0/1945570219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc458070810 msgr2=0x7fc45806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.553+0000 7fc45ca00640 1 --2- 172.21.15.7:0/1945570219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc458070810 0x7fc45806d580 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc4400097a0 tx=0x7fc44002c3f0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.553+0000 7fc45ca00640 1 -- 172.21.15.7:0/1945570219 shutdown_connections 2024-04-25T11:16:23.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.553+0000 7fc45ca00640 1 --2- 172.21.15.7:0/1945570219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc458070810 0x7fc45806d580 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.553+0000 7fc45ca00640 1 -- 172.21.15.7:0/1945570219 >> 172.21.15.7:0/1945570219 conn(0x7fc458073090 msgr2=0x7fc4580734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:23.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.553+0000 7fc45ca00640 1 -- 172.21.15.7:0/1945570219 shutdown_connections 2024-04-25T11:16:23.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.553+0000 7fc45ca00640 1 -- 172.21.15.7:0/1945570219 wait complete. 2024-04-25T11:16:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.556+0000 7fc45ca00640 1 Processor -- start 2024-04-25T11:16:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.556+0000 7fc45ca00640 1 -- start start 2024-04-25T11:16:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.557+0000 7fc45ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45806dac0 0x7fc4581f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.557+0000 7fc45ca00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc458070810 0x7fc4581f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.557+0000 7fc45ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc458070c20 0x7fc458145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.557+0000 7fc45ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc45815e0f0 con 0x7fc458070c20 2024-04-25T11:16:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.557+0000 7fc45ca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc458161630 con 0x7fc45806dac0 2024-04-25T11:16:23.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.557+0000 7fc45ca00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc4581433b0 con 0x7fc458070810 2024-04-25T11:16:23.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.558+0000 7fc457e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc458070c20 0x7fc458145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:23.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.558+0000 7fc457e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc458070c20 0x7fc458145080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39514/0 (socket says 172.21.15.7:39514) 2024-04-25T11:16:23.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.558+0000 7fc457e00640 1 -- 172.21.15.7:0/3440444573 learned_addr learned my addr 172.21.15.7:0/3440444573 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:23.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.558+0000 7fc457400640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45806dac0 0x7fc4581f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:23.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.558+0000 7fc456a00640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc458070810 0x7fc4581f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.558+0000 7fc457e00640 1 -- 172.21.15.7:0/3440444573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc458070810 msgr2=0x7fc4581f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.558+0000 7fc457e00640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc458070810 0x7fc4581f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.558+0000 7fc457e00640 1 -- 172.21.15.7:0/3440444573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45806dac0 msgr2=0x7fc4581f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.558+0000 7fc457e00640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45806dac0 0x7fc4581f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.558+0000 7fc457e00640 1 -- 172.21.15.7:0/3440444573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4581f1da0 con 0x7fc458070c20 2024-04-25T11:16:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.558+0000 7fc457400640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45806dac0 0x7fc4581f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.559+0000 7fc457e00640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc458070c20 0x7fc458145080 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fc45000b410 tx=0x7fc45000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.559+0000 7fc44f400640 1 -- 172.21.15.7:0/3440444573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450015030 con 0x7fc458070c20 2024-04-25T11:16:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.559+0000 7fc44f400640 1 -- 172.21.15.7:0/3440444573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc450004020 con 0x7fc458070c20 2024-04-25T11:16:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.559+0000 7fc44f400640 1 -- 172.21.15.7:0/3440444573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450004410 con 0x7fc458070c20 2024-04-25T11:16:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.559+0000 7fc45ca00640 1 -- 172.21.15.7:0/3440444573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4580cce10 con 0x7fc458070c20 2024-04-25T11:16:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.559+0000 7fc45ca00640 1 -- 172.21.15.7:0/3440444573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4580cd3d0 con 0x7fc458070c20 2024-04-25T11:16:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.560+0000 7fc45ca00640 1 -- 172.21.15.7:0/3440444573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc458050740 con 0x7fc458070c20 2024-04-25T11:16:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.561+0000 7fc44f400640 1 -- 172.21.15.7:0/3440444573 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc450018070 con 0x7fc458070c20 2024-04-25T11:16:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.562+0000 7fc44f400640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4340744b0 0x7fc4340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.563+0000 7fc44f400640 1 -- 172.21.15.7:0/3440444573 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7fc450094e10 con 0x7fc458070c20 2024-04-25T11:16:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.563+0000 7fc457400640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4340744b0 0x7fc4340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:23.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.564+0000 7fc457400640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4340744b0 0x7fc4340769c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc440036000 tx=0x7fc440030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:23.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.568+0000 7fc44f400640 1 -- 172.21.15.7:0/3440444573 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc450098030 con 0x7fc458070c20 2024-04-25T11:16:23.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.817+0000 7fc45ca00640 1 -- 172.21.15.7:0/3440444573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc4581f2d70 con 0x7fc458070c20 2024-04-25T11:16:23.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.818+0000 7fc44f400640 1 -- 172.21.15.7:0/3440444573 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fc45005f490 con 0x7fc458070c20 2024-04-25T11:16:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:23.818 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:16:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 -- 172.21.15.7:0/3440444573 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4340744b0 msgr2=0x7fc4340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4340744b0 0x7fc4340769c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc440036000 tx=0x7fc440030000 comp rx=0 tx=0).stop 2024-04-25T11:16:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 -- 172.21.15.7:0/3440444573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc458070c20 msgr2=0x7fc458145080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc458070c20 0x7fc458145080 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fc45000b410 tx=0x7fc45000b8d0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 -- 172.21.15.7:0/3440444573 shutdown_connections 2024-04-25T11:16:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4340744b0 0x7fc4340769c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc458070c20 0x7fc458145080 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc458070810 0x7fc4581f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 --2- 172.21.15.7:0/3440444573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc45806dac0 0x7fc4581f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 -- 172.21.15.7:0/3440444573 >> 172.21.15.7:0/3440444573 conn(0x7fc458073090 msgr2=0x7fc4580b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 -- 172.21.15.7:0/3440444573 shutdown_connections 2024-04-25T11:16:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:23.821+0000 7fc45ca00640 1 -- 172.21.15.7:0/3440444573 wait complete. 2024-04-25T11:16:23.849 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:16:23.849 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-04-25T11:16:23.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:16:24.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.135+0000 7fd96bc00640 1 Processor -- start 2024-04-25T11:16:24.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.135+0000 7fd96bc00640 1 -- start start 2024-04-25T11:16:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.137+0000 7fd96bc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd964070ca0 0x7fd9640b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.138+0000 7fd96bc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd964160320 con 0x7fd964146310 2024-04-25T11:16:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.138+0000 7fd96bc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd964160600 con 0x7fd9640b65e0 2024-04-25T11:16:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.138+0000 7fd96bc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd9641608e0 con 0x7fd964070ca0 2024-04-25T11:16:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.138+0000 7fd969200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd964146310 0x7fd9640d0d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40726/0 (socket says 172.21.15.7:40726) 2024-04-25T11:16:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.138+0000 7fd969200640 1 -- 172.21.15.7:0/4102006985 learned_addr learned my addr 172.21.15.7:0/4102006985 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:24.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.138+0000 7fd963e00640 1 --2- 172.21.15.7:0/4102006985 >> v2:172.21.15.7:3300/0 conn(0x7fd964070ca0 0x7fd9640b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.138+0000 7fd963400640 1 -- 172.21.15.7:0/4102006985 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3292190664 0 0) 0x7fd964160320 con 0x7fd964146310 2024-04-25T11:16:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.138+0000 7fd963400640 1 -- 172.21.15.7:0/4102006985 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd94c003520 con 0x7fd964146310 2024-04-25T11:16:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.139+0000 7fd963e00640 1 -- 172.21.15.7:0/4102006985 >> v1:172.21.15.87:6789/0 conn(0x7fd9640b65e0 legacy=0x7fd96406f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.139+0000 7fd963e00640 1 -- 172.21.15.7:0/4102006985 >> v1:172.21.15.7:6789/0 conn(0x7fd964146310 legacy=0x7fd9640d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:24.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.139+0000 7fd963e00640 1 -- 172.21.15.7:0/4102006985 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd964160bc0 con 0x7fd964070ca0 2024-04-25T11:16:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.140+0000 7fd963e00640 1 --2- 172.21.15.7:0/4102006985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964070ca0 0x7fd9640b60a0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fd95400a940 tx=0x7fd95402b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18be29a47670169 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.140+0000 7fd963400640 1 -- 172.21.15.7:0/4102006985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95400b040 con 0x7fd964070ca0 2024-04-25T11:16:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.140+0000 7fd963400640 1 -- 172.21.15.7:0/4102006985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd954030710 con 0x7fd964070ca0 2024-04-25T11:16:24.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.140+0000 7fd963400640 1 -- 172.21.15.7:0/4102006985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954030ac0 con 0x7fd964070ca0 2024-04-25T11:16:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.141+0000 7fd96bc00640 1 -- 172.21.15.7:0/4102006985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964070ca0 msgr2=0x7fd9640b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.141+0000 7fd96bc00640 1 --2- 172.21.15.7:0/4102006985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964070ca0 0x7fd9640b60a0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fd95400a940 tx=0x7fd95402b630 comp rx=0 tx=0).stop 2024-04-25T11:16:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.141+0000 7fd96bc00640 1 -- 172.21.15.7:0/4102006985 shutdown_connections 2024-04-25T11:16:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.141+0000 7fd96bc00640 1 --2- 172.21.15.7:0/4102006985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964070ca0 0x7fd9640b60a0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.141+0000 7fd96bc00640 1 -- 172.21.15.7:0/4102006985 >> 172.21.15.7:0/4102006985 conn(0x7fd9640732b0 msgr2=0x7fd9640736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.141+0000 7fd96bc00640 1 -- 172.21.15.7:0/4102006985 shutdown_connections 2024-04-25T11:16:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.141+0000 7fd96bc00640 1 -- 172.21.15.7:0/4102006985 wait complete. 2024-04-25T11:16:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.143+0000 7fd96bc00640 1 Processor -- start 2024-04-25T11:16:24.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.143+0000 7fd96bc00640 1 -- start start 2024-04-25T11:16:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.145+0000 7fd96bc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd964070ca0 0x7fd96414af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.145+0000 7fd96bc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9640b65e0 0x7fd9641406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.145+0000 7fd96bc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964146310 0x7fd9640afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.145+0000 7fd96bc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd964073f50 con 0x7fd964146310 2024-04-25T11:16:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.145+0000 7fd96bc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd964073dd0 con 0x7fd964070ca0 2024-04-25T11:16:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.145+0000 7fd96bc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9640740d0 con 0x7fd9640b65e0 2024-04-25T11:16:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.145+0000 7fd969c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964146310 0x7fd9640afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:24.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.145+0000 7fd969c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964146310 0x7fd9640afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39538/0 (socket says 172.21.15.7:39538) 2024-04-25T11:16:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.145+0000 7fd969c00640 1 -- 172.21.15.7:0/1078808290 learned_addr learned my addr 172.21.15.7:0/1078808290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.146+0000 7fd963e00640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9640b65e0 0x7fd9641406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.146+0000 7fd969200640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd964070ca0 0x7fd96414af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.146+0000 7fd969c00640 1 -- 172.21.15.7:0/1078808290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9640b65e0 msgr2=0x7fd9641406e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.146+0000 7fd969c00640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9640b65e0 0x7fd9641406e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.146+0000 7fd969c00640 1 -- 172.21.15.7:0/1078808290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd964070ca0 msgr2=0x7fd96414af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.146+0000 7fd969c00640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd964070ca0 0x7fd96414af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:24.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.146+0000 7fd969c00640 1 -- 172.21.15.7:0/1078808290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9641411a0 con 0x7fd964146310 2024-04-25T11:16:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.146+0000 7fd969c00640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964146310 0x7fd9640afe70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00be60 tx=0x7fd95c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.146+0000 7fd960c00640 1 -- 172.21.15.7:0/1078808290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c019070 con 0x7fd964146310 2024-04-25T11:16:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.146+0000 7fd960c00640 1 -- 172.21.15.7:0/1078808290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd95c014440 con 0x7fd964146310 2024-04-25T11:16:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.146+0000 7fd960c00640 1 -- 172.21.15.7:0/1078808290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c014870 con 0x7fd964146310 2024-04-25T11:16:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.147+0000 7fd96bc00640 1 -- 172.21.15.7:0/1078808290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd964141480 con 0x7fd964146310 2024-04-25T11:16:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.147+0000 7fd96bc00640 1 -- 172.21.15.7:0/1078808290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd964141a40 con 0x7fd964146310 2024-04-25T11:16:24.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.148+0000 7fd96bc00640 1 -- 172.21.15.7:0/1078808290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd964140ef0 con 0x7fd964146310 2024-04-25T11:16:24.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.149+0000 7fd960c00640 1 -- 172.21.15.7:0/1078808290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd95c01c070 con 0x7fd964146310 2024-04-25T11:16:24.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.150+0000 7fd960c00640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744e0 0x7fd9480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:24.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.150+0000 7fd960c00640 1 -- 172.21.15.7:0/1078808290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7fd95c0953a0 con 0x7fd964146310 2024-04-25T11:16:24.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.150+0000 7fd969200640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744e0 0x7fd9480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:24.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.151+0000 7fd969200640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744e0 0x7fd9480769f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd958005880 tx=0x7fd958009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.155+0000 7fd960c00640 1 -- 172.21.15.7:0/1078808290 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd95c05f900 con 0x7fd964146310 2024-04-25T11:16:24.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.401+0000 7fd96bc00640 1 -- 172.21.15.7:0/1078808290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd964050740 con 0x7fd964146310 2024-04-25T11:16:24.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.402+0000 7fd960c00640 1 -- 172.21.15.7:0/1078808290 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fd95c05f200 con 0x7fd964146310 2024-04-25T11:16:24.402 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:24.402 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:16:24.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.404+0000 7fd96bc00640 1 -- 172.21.15.7:0/1078808290 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744e0 msgr2=0x7fd9480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:24.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.404+0000 7fd96bc00640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744e0 0x7fd9480769f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd958005880 tx=0x7fd958009040 comp rx=0 tx=0).stop 2024-04-25T11:16:24.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.404+0000 7fd96bc00640 1 -- 172.21.15.7:0/1078808290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964146310 msgr2=0x7fd9640afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:24.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.404+0000 7fd96bc00640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964146310 0x7fd9640afe70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00be60 tx=0x7fd95c00bf50 comp rx=0 tx=0).stop 2024-04-25T11:16:24.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.405+0000 7fd96bc00640 1 -- 172.21.15.7:0/1078808290 shutdown_connections 2024-04-25T11:16:24.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.405+0000 7fd96bc00640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744e0 0x7fd9480769f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:24.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.405+0000 7fd96bc00640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd964146310 0x7fd9640afe70 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:24.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.405+0000 7fd96bc00640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9640b65e0 0x7fd9641406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:24.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.405+0000 7fd96bc00640 1 --2- 172.21.15.7:0/1078808290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd964070ca0 0x7fd96414af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:24.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.405+0000 7fd96bc00640 1 -- 172.21.15.7:0/1078808290 >> 172.21.15.7:0/1078808290 conn(0x7fd9640732b0 msgr2=0x7fd9640b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:24.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.405+0000 7fd96bc00640 1 -- 172.21.15.7:0/1078808290 shutdown_connections 2024-04-25T11:16:24.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:24.405+0000 7fd96bc00640 1 -- 172.21.15.7:0/1078808290 wait complete. 2024-04-25T11:16:24.435 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:16:24.436 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:16:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:27.514 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:29.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:16:29.516 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.729+0000 7f5b2e600640 1 Processor -- start 2024-04-25T11:16:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.729+0000 7f5b2e600640 1 -- start start 2024-04-25T11:16:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.731+0000 7f5b2e600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b280b76d0 0x7f5b280b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.731+0000 7f5b2e600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b2806c570 0x7f5b28161160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.731+0000 7f5b2e600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b281616a0 con 0x7f5b280ce950 2024-04-25T11:16:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.731+0000 7f5b2e600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5b280c89a0 con 0x7f5b2806c570 2024-04-25T11:16:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.731+0000 7f5b2e600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5b280c8c40 con 0x7f5b280b76d0 2024-04-25T11:16:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.731+0000 7f5b1fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b2806c570 0x7f5b28161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.731+0000 7f5b1fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b2806c570 0x7f5b28161160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39540/0 (socket says 172.21.15.7:39540) 2024-04-25T11:16:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.731+0000 7f5b1fe00640 1 -- 172.21.15.7:0/1451072686 learned_addr learned my addr 172.21.15.7:0/1451072686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.732+0000 7f5b27400640 1 --2- 172.21.15.7:0/1451072686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b280b76d0 0x7f5b280b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.732+0000 7f5b1fe00640 1 -- 172.21.15.7:0/1451072686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b280b76d0 msgr2=0x7f5b280b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.732+0000 7f5b1fe00640 1 --2- 172.21.15.7:0/1451072686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b280b76d0 0x7f5b280b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.732+0000 7f5b1fe00640 1 -- 172.21.15.7:0/1451072686 >> v1:172.21.15.87:6789/0 conn(0x7f5b280ce950 legacy=0x7f5b280ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.732+0000 7f5b1fe00640 1 -- 172.21.15.7:0/1451072686 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b280c6fb0 con 0x7f5b2806c570 2024-04-25T11:16:29.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.733+0000 7f5b1fe00640 1 --2- 172.21.15.7:0/1451072686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806c570 0x7f5b28161160 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009870 tx=0x7f5b1002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20239c07832394ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.733+0000 7f5b26a00640 1 -- 172.21.15.7:0/1451072686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10037070 con 0x7f5b2806c570 2024-04-25T11:16:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.733+0000 7f5b26a00640 1 -- 172.21.15.7:0/1451072686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b10032440 con 0x7f5b2806c570 2024-04-25T11:16:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.733+0000 7f5b26a00640 1 -- 172.21.15.7:0/1451072686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10032810 con 0x7f5b2806c570 2024-04-25T11:16:29.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.734+0000 7f5b2e600640 1 -- 172.21.15.7:0/1451072686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806c570 msgr2=0x7f5b28161160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.734+0000 7f5b2e600640 1 --2- 172.21.15.7:0/1451072686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806c570 0x7f5b28161160 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009870 tx=0x7f5b1002a210 comp rx=0 tx=0).stop 2024-04-25T11:16:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.734+0000 7f5b2e600640 1 -- 172.21.15.7:0/1451072686 shutdown_connections 2024-04-25T11:16:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.734+0000 7f5b2e600640 1 --2- 172.21.15.7:0/1451072686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b2806c570 0x7f5b28161160 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.734+0000 7f5b2e600640 1 --2- 172.21.15.7:0/1451072686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b280b76d0 0x7f5b280b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.734+0000 7f5b2e600640 1 -- 172.21.15.7:0/1451072686 >> 172.21.15.7:0/1451072686 conn(0x7f5b2809b540 msgr2=0x7f5b280b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.734+0000 7f5b2e600640 1 -- 172.21.15.7:0/1451072686 shutdown_connections 2024-04-25T11:16:29.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.734+0000 7f5b2e600640 1 -- 172.21.15.7:0/1451072686 wait complete. 2024-04-25T11:16:29.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.737+0000 7f5b2e600640 1 Processor -- start 2024-04-25T11:16:29.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.737+0000 7f5b2e600640 1 -- start start 2024-04-25T11:16:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.738+0000 7f5b2e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2806c570 0x7f5b28151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.738+0000 7f5b2e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b280b76d0 0x7f5b28161880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.738+0000 7f5b2e600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b280ce950 0x7f5b28161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.738+0000 7f5b2e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b280c7930 con 0x7f5b280b76d0 2024-04-25T11:16:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.738+0000 7f5b2e600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5b280c77b0 con 0x7f5b2806c570 2024-04-25T11:16:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.738+0000 7f5b2e600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5b280c7ab0 con 0x7f5b280ce950 2024-04-25T11:16:29.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.739+0000 7f5b1fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b280b76d0 0x7f5b28161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.739+0000 7f5b1fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b280b76d0 0x7f5b28161880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39550/0 (socket says 172.21.15.7:39550) 2024-04-25T11:16:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.739+0000 7f5b1fe00640 1 -- 172.21.15.7:0/3983512964 learned_addr learned my addr 172.21.15.7:0/3983512964 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.739+0000 7f5b1fe00640 1 -- 172.21.15.7:0/3983512964 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b280ce950 msgr2=0x7f5b28161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.739+0000 7f5b1fe00640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b280ce950 0x7f5b28161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.739+0000 7f5b1fe00640 1 -- 172.21.15.7:0/3983512964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2806c570 msgr2=0x7f5b28151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.739+0000 7f5b27400640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2806c570 0x7f5b28151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.739+0000 7f5b1fe00640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2806c570 0x7f5b28151c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:29.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.739+0000 7f5b1fe00640 1 -- 172.21.15.7:0/3983512964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b280d0510 con 0x7f5b280b76d0 2024-04-25T11:16:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.739+0000 7f5b1fe00640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b280b76d0 0x7f5b28161880 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009870 tx=0x7f5b100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.740+0000 7f5b1f400640 1 -- 172.21.15.7:0/3983512964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10041020 con 0x7f5b280b76d0 2024-04-25T11:16:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.740+0000 7f5b1f400640 1 -- 172.21.15.7:0/3983512964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b100306c0 con 0x7f5b280b76d0 2024-04-25T11:16:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.740+0000 7f5b1f400640 1 -- 172.21.15.7:0/3983512964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b10037040 con 0x7f5b280b76d0 2024-04-25T11:16:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.740+0000 7f5b2e600640 1 -- 172.21.15.7:0/3983512964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b280d07d0 con 0x7f5b280b76d0 2024-04-25T11:16:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.740+0000 7f5b2e600640 1 -- 172.21.15.7:0/3983512964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b280d0d90 con 0x7f5b280b76d0 2024-04-25T11:16:29.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.741+0000 7f5b2e600640 1 -- 172.21.15.7:0/3983512964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b28050740 con 0x7f5b280b76d0 2024-04-25T11:16:29.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.743+0000 7f5b1f400640 1 -- 172.21.15.7:0/3983512964 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5b10044020 con 0x7f5b280b76d0 2024-04-25T11:16:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.744+0000 7f5b1f400640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b040744b0 0x7f5b040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.744+0000 7f5b1f400640 1 -- 172.21.15.7:0/3983512964 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f5b100b5c00 con 0x7f5b280b76d0 2024-04-25T11:16:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.744+0000 7f5b27400640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b040744b0 0x7f5b040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:29.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.745+0000 7f5b27400640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b040744b0 0x7f5b040769c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5b18006fd0 tx=0x7f5b18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:29.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.749+0000 7f5b1f400640 1 -- 172.21.15.7:0/3983512964 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5b1004b030 con 0x7f5b280b76d0 2024-04-25T11:16:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.997+0000 7f5b2e600640 1 -- 172.21.15.7:0/3983512964 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5b280746d0 con 0x7f5b280b76d0 2024-04-25T11:16:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:29.998+0000 7f5b1f400640 1 -- 172.21.15.7:0/3983512964 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f5b281616a0 con 0x7f5b280b76d0 2024-04-25T11:16:29.998 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:29.998 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:16:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 -- 172.21.15.7:0/3983512964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b040744b0 msgr2=0x7f5b040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b040744b0 0x7f5b040769c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5b18006fd0 tx=0x7f5b18008040 comp rx=0 tx=0).stop 2024-04-25T11:16:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 -- 172.21.15.7:0/3983512964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b280b76d0 msgr2=0x7f5b28161880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b280b76d0 0x7f5b28161880 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009870 tx=0x7f5b100046c0 comp rx=0 tx=0).stop 2024-04-25T11:16:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 -- 172.21.15.7:0/3983512964 shutdown_connections 2024-04-25T11:16:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5b040744b0 0x7f5b040769c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b280ce950 0x7f5b28161dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b280b76d0 0x7f5b28161880 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 --2- 172.21.15.7:0/3983512964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b2806c570 0x7f5b28151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 -- 172.21.15.7:0/3983512964 >> 172.21.15.7:0/3983512964 conn(0x7f5b2809b540 msgr2=0x7f5b2815f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 -- 172.21.15.7:0/3983512964 shutdown_connections 2024-04-25T11:16:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:30.001+0000 7f5b2e600640 1 -- 172.21.15.7:0/3983512964 wait complete. 2024-04-25T11:16:30.032 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:16:30.032 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:16:31.518 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:33.520 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:35.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:16:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.325+0000 7f6ae2e00640 1 Processor -- start 2024-04-25T11:16:35.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.325+0000 7f6ae2e00640 1 -- start start 2024-04-25T11:16:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.328+0000 7f6ae2e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6adc0b4a10 0x7f6adc0d53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.328+0000 7f6ae2e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6adc0b7b50 0x7f6adc0b9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.328+0000 7f6ae2e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6adc1604a0 con 0x7f6adc160090 2024-04-25T11:16:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.328+0000 7f6ae2e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6adc0d1490 con 0x7f6adc0b7b50 2024-04-25T11:16:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.328+0000 7f6ae2e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6adc0b9550 con 0x7f6adc0b4a10 2024-04-25T11:16:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.328+0000 7f6ae2400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6adc0b7b50 0x7f6adc0b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:35.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.328+0000 7f6ae2400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6adc0b7b50 0x7f6adc0b9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41674/0 (socket says 172.21.15.7:41674) 2024-04-25T11:16:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.328+0000 7f6ae2400640 1 -- 172.21.15.7:0/3640943008 learned_addr learned my addr 172.21.15.7:0/3640943008 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.328+0000 7f6ae1a00640 1 --2- 172.21.15.7:0/3640943008 >> v2:172.21.15.87:3300/0 conn(0x7f6adc0b4a10 0x7f6adc0d53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.329+0000 7f6ae2400640 1 -- 172.21.15.7:0/3640943008 >> v2:172.21.15.87:3300/0 conn(0x7f6adc0b4a10 msgr2=0x7f6adc0d53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.329+0000 7f6ae2400640 1 --2- 172.21.15.7:0/3640943008 >> v2:172.21.15.87:3300/0 conn(0x7f6adc0b4a10 0x7f6adc0d53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.329+0000 7f6ae2400640 1 -- 172.21.15.7:0/3640943008 >> v1:172.21.15.87:6789/0 conn(0x7f6adc160090 legacy=0x7f6adc0b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.329+0000 7f6ae2400640 1 -- 172.21.15.7:0/3640943008 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc0d0880 con 0x7f6adc0b7b50 2024-04-25T11:16:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.330+0000 7f6ae2400640 1 --2- 172.21.15.7:0/3640943008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6adc0b7b50 0x7f6adc0b9010 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009e50 tx=0x7f6ad402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6506741d81d8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.330+0000 7f6adbe00640 1 -- 172.21.15.7:0/3640943008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4036070 con 0x7f6adc0b7b50 2024-04-25T11:16:35.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.330+0000 7f6adbe00640 1 -- 172.21.15.7:0/3640943008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ad4031600 con 0x7f6adc0b7b50 2024-04-25T11:16:35.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.330+0000 7f6adbe00640 1 -- 172.21.15.7:0/3640943008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad402ae60 con 0x7f6adc0b7b50 2024-04-25T11:16:35.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.331+0000 7f6ae2e00640 1 -- 172.21.15.7:0/3640943008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6adc0b7b50 msgr2=0x7f6adc0b9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.331+0000 7f6ae2e00640 1 --2- 172.21.15.7:0/3640943008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6adc0b7b50 0x7f6adc0b9010 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009e50 tx=0x7f6ad402a680 comp rx=0 tx=0).stop 2024-04-25T11:16:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.331+0000 7f6ae2e00640 1 -- 172.21.15.7:0/3640943008 shutdown_connections 2024-04-25T11:16:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.331+0000 7f6ae2e00640 1 --2- 172.21.15.7:0/3640943008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6adc0b7b50 0x7f6adc0b9010 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.331+0000 7f6ae2e00640 1 --2- 172.21.15.7:0/3640943008 >> v2:172.21.15.87:3300/0 conn(0x7f6adc0b4a10 0x7f6adc0d53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.331+0000 7f6ae2e00640 1 -- 172.21.15.7:0/3640943008 >> 172.21.15.7:0/3640943008 conn(0x7f6adc0d5de0 msgr2=0x7f6adc0d61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.331+0000 7f6ae2e00640 1 -- 172.21.15.7:0/3640943008 shutdown_connections 2024-04-25T11:16:35.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.332+0000 7f6ae2e00640 1 -- 172.21.15.7:0/3640943008 wait complete. 2024-04-25T11:16:35.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.335+0000 7f6ae2e00640 1 Processor -- start 2024-04-25T11:16:35.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.335+0000 7f6ae2e00640 1 -- start start 2024-04-25T11:16:35.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.337+0000 7f6ae2e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6adc0b4a10 0x7f6adc14ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:35.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.337+0000 7f6ae2e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6adc160090 0x7f6adc0cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:35.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.337+0000 7f6ae2e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6adc0cb720 0x7f6adc1616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:35.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.337+0000 7f6ae2e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc0d1120 con 0x7f6adc160090 2024-04-25T11:16:35.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.337+0000 7f6ae2e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc0d0e40 con 0x7f6adc0b4a10 2024-04-25T11:16:35.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.337+0000 7f6ae2e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6adc163d40 con 0x7f6adc0cb720 2024-04-25T11:16:35.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.337+0000 7f6ae1000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6adc160090 0x7f6adc0cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:35.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.337+0000 7f6ae1000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6adc160090 0x7f6adc0cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41684/0 (socket says 172.21.15.7:41684) 2024-04-25T11:16:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.337+0000 7f6ae1000640 1 -- 172.21.15.7:0/305764730 learned_addr learned my addr 172.21.15.7:0/305764730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.337+0000 7f6ae1a00640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6adc0b4a10 0x7f6adc14ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.337+0000 7f6ae2400640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6adc0cb720 0x7f6adc1616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.338+0000 7f6ae1000640 1 -- 172.21.15.7:0/305764730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6adc0cb720 msgr2=0x7f6adc1616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.338+0000 7f6ae1000640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6adc0cb720 0x7f6adc1616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.338+0000 7f6ae1000640 1 -- 172.21.15.7:0/305764730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6adc0b4a10 msgr2=0x7f6adc14ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.338+0000 7f6ae1000640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6adc0b4a10 0x7f6adc14ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.338+0000 7f6ae1000640 1 -- 172.21.15.7:0/305764730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc0cbb30 con 0x7f6adc160090 2024-04-25T11:16:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.338+0000 7f6ae1000640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6adc160090 0x7f6adc0cb1e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800b3f0 tx=0x7f6ac800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:35.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.338+0000 7f6ad9600640 1 -- 172.21.15.7:0/305764730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8015020 con 0x7f6adc160090 2024-04-25T11:16:35.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.338+0000 7f6ad9600640 1 -- 172.21.15.7:0/305764730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ac8011830 con 0x7f6adc160090 2024-04-25T11:16:35.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.338+0000 7f6ad9600640 1 -- 172.21.15.7:0/305764730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8011c20 con 0x7f6adc160090 2024-04-25T11:16:35.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.339+0000 7f6ae2e00640 1 -- 172.21.15.7:0/305764730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6adc161210 con 0x7f6adc160090 2024-04-25T11:16:35.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.339+0000 7f6ae2e00640 1 -- 172.21.15.7:0/305764730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6adc0cdc60 con 0x7f6adc160090 2024-04-25T11:16:35.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.340+0000 7f6ae2e00640 1 -- 172.21.15.7:0/305764730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6adc0cd6d0 con 0x7f6adc160090 2024-04-25T11:16:35.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.341+0000 7f6ad9600640 1 -- 172.21.15.7:0/305764730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6ac8018020 con 0x7f6adc160090 2024-04-25T11:16:35.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.342+0000 7f6ad9600640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6abc0744e0 0x7f6abc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:35.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.342+0000 7f6ad9600640 1 -- 172.21.15.7:0/305764730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f6ac80954d0 con 0x7f6adc160090 2024-04-25T11:16:35.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.343+0000 7f6ae1a00640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6abc0744e0 0x7f6abc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:35.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.343+0000 7f6ae1a00640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6abc0744e0 0x7f6abc0769f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0006fd0 tx=0x7f6ad0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:35.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.346+0000 7f6ad9600640 1 -- 172.21.15.7:0/305764730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6ac801f030 con 0x7f6adc160090 2024-04-25T11:16:35.523 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.595+0000 7f6ae2e00640 1 -- 172.21.15.7:0/305764730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6adc050740 con 0x7f6adc160090 2024-04-25T11:16:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.596+0000 7f6ad9600640 1 -- 172.21.15.7:0/305764730 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f6adc0cbb30 con 0x7f6adc160090 2024-04-25T11:16:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:35.597 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:16:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.599+0000 7f6ae2e00640 1 -- 172.21.15.7:0/305764730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6abc0744e0 msgr2=0x7f6abc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.599+0000 7f6ae2e00640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6abc0744e0 0x7f6abc0769f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0006fd0 tx=0x7f6ad0008040 comp rx=0 tx=0).stop 2024-04-25T11:16:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.599+0000 7f6ae2e00640 1 -- 172.21.15.7:0/305764730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6adc160090 msgr2=0x7f6adc0cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.599+0000 7f6ae2e00640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6adc160090 0x7f6adc0cb1e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800b3f0 tx=0x7f6ac800b8b0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.599+0000 7f6ae2e00640 1 -- 172.21.15.7:0/305764730 shutdown_connections 2024-04-25T11:16:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.599+0000 7f6ae2e00640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6abc0744e0 0x7f6abc0769f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.599+0000 7f6ae2e00640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6adc0cb720 0x7f6adc1616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.599+0000 7f6ae2e00640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6adc160090 0x7f6adc0cb1e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.599+0000 7f6ae2e00640 1 --2- 172.21.15.7:0/305764730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6adc0b4a10 0x7f6adc14ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.599+0000 7f6ae2e00640 1 -- 172.21.15.7:0/305764730 >> 172.21.15.7:0/305764730 conn(0x7f6adc0d5de0 msgr2=0x7f6adc15bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.600+0000 7f6ae2e00640 1 -- 172.21.15.7:0/305764730 shutdown_connections 2024-04-25T11:16:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:35.600+0000 7f6ae2e00640 1 -- 172.21.15.7:0/305764730 wait complete. 2024-04-25T11:16:35.628 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:16:35.628 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:16:37.525 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:40.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:16:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.928+0000 7f7643c00640 1 Processor -- start 2024-04-25T11:16:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.928+0000 7f7643c00640 1 -- start start 2024-04-25T11:16:40.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.930+0000 7f7643c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c1604b0 0x7f763c0af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.930+0000 7f7643c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f763c0d2f20 con 0x7f763c06ebd0 2024-04-25T11:16:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.930+0000 7f7643c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f763c0d3200 con 0x7f763c09bc50 2024-04-25T11:16:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.930+0000 7f7643c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f763c0b00b0 con 0x7f763c1604b0 2024-04-25T11:16:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.931+0000 7f7641200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f763c06ebd0 0x7f763c09b540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46292/0 (socket says 172.21.15.7:46292) 2024-04-25T11:16:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.931+0000 7f7641200640 1 -- 172.21.15.7:0/3278741083 learned_addr learned my addr 172.21.15.7:0/3278741083 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.931+0000 7f7641c00640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c1604b0 0x7f763c0af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.931+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1191804501 0 0) 0x7f763c0d2f20 con 0x7f763c06ebd0 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.931+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7624003520 con 0x7f763c06ebd0 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 477624891 0 0) 0x7f7624003520 con 0x7f763c06ebd0 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c1604b0 msgr2=0x7f763c0af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c1604b0 0x7f763c0af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 >> v1:172.21.15.87:6789/0 conn(0x7f763c09bc50 legacy=0x7f763c0b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763c0d34e0 con 0x7f763c06ebd0 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f7630001670 con 0x7f763c06ebd0 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 >> v1:172.21.15.7:6789/0 conn(0x7f763c06ebd0 legacy=0x7f763c09b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c1604b0 0x7f76240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 0x7f7624004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7624004d40 0x7f7624005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f763c0b00b0 con 0x7f7624003f60 2024-04-25T11:16:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f762400bb50 con 0x7f7624004d40 2024-04-25T11:16:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f762400bd50 con 0x7f763c1604b0 2024-04-25T11:16:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.932+0000 7f763be00640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 0x7f7624004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.933+0000 7f7641200640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c1604b0 0x7f76240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.933+0000 7f7641c00640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7624004d40 0x7f7624005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.933+0000 7f763be00640 1 -- 172.21.15.7:0/3278741083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c1604b0 msgr2=0x7f76240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.933+0000 7f763be00640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c1604b0 0x7f76240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.933+0000 7f763be00640 1 -- 172.21.15.7:0/3278741083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7624004d40 msgr2=0x7f7624005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.933+0000 7f763be00640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7624004d40 0x7f7624005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.933+0000 7f763be00640 1 -- 172.21.15.7:0/3278741083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7624003520 con 0x7f7624003f60 2024-04-25T11:16:40.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.934+0000 7f763be00640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 0x7f7624004800 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f763c09c4e0 tx=0x7f7624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f99b799cb64492d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:40.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.934+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628001370 con 0x7f7624003f60 2024-04-25T11:16:40.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.934+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76280052d0 con 0x7f7624003f60 2024-04-25T11:16:40.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.934+0000 7f763b400640 1 -- 172.21.15.7:0/3278741083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762802ab50 con 0x7f7624003f60 2024-04-25T11:16:40.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.935+0000 7f7643c00640 1 -- 172.21.15.7:0/3278741083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 msgr2=0x7f7624004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:40.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.935+0000 7f7643c00640 1 --2- 172.21.15.7:0/3278741083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 0x7f7624004800 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f763c09c4e0 tx=0x7f7624005830 comp rx=0 tx=0).stop 2024-04-25T11:16:40.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.935+0000 7f7641c00640 1 -- 172.21.15.7:0/3278741083 reap_dead start 2024-04-25T11:16:40.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.935+0000 7f7643c00640 1 -- 172.21.15.7:0/3278741083 shutdown_connections 2024-04-25T11:16:40.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.935+0000 7f7643c00640 1 -- 172.21.15.7:0/3278741083 >> 172.21.15.7:0/3278741083 conn(0x7f763c0b7560 msgr2=0x7f763c0b7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:40.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.935+0000 7f7643c00640 1 -- 172.21.15.7:0/3278741083 shutdown_connections 2024-04-25T11:16:40.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.935+0000 7f7643c00640 1 -- 172.21.15.7:0/3278741083 wait complete. 2024-04-25T11:16:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.938+0000 7f7643c00640 1 Processor -- start 2024-04-25T11:16:40.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.938+0000 7f7643c00640 1 -- start start 2024-04-25T11:16:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.939+0000 7f7643c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 0x7f763c0a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.939+0000 7f7643c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f763c09d960 0x7f763c1442f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.939+0000 7f7643c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c153080 0x7f763c153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.939+0000 7f7643c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f763c0af6a0 con 0x7f7624003f60 2024-04-25T11:16:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.939+0000 7f7643c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f763c0b0670 con 0x7f763c09d960 2024-04-25T11:16:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.939+0000 7f7643c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f763c0af820 con 0x7f763c153080 2024-04-25T11:16:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.940+0000 7f7641200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 0x7f763c0a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.940+0000 7f7641200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 0x7f763c0a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48116/0 (socket says 172.21.15.7:48116) 2024-04-25T11:16:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.940+0000 7f7641200640 1 -- 172.21.15.7:0/2989052206 learned_addr learned my addr 172.21.15.7:0/2989052206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.940+0000 7f763be00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f763c09d960 0x7f763c1442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.940+0000 7f7641c00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c153080 0x7f763c153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.940+0000 7f7641200640 1 -- 172.21.15.7:0/2989052206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c153080 msgr2=0x7f763c153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.940+0000 7f7641200640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c153080 0x7f763c153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.940+0000 7f7641200640 1 -- 172.21.15.7:0/2989052206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f763c09d960 msgr2=0x7f763c1442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.940+0000 7f7641200640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f763c09d960 0x7f763c1442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.940+0000 7f7641200640 1 -- 172.21.15.7:0/2989052206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763c09e1e0 con 0x7f7624003f60 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.940+0000 7f763be00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f763c09d960 0x7f763c1442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.941+0000 7f7641200640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 0x7f763c0a0470 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f763000e7b0 tx=0x7f763000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.941+0000 7f7638c00640 1 -- 172.21.15.7:0/2989052206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763000ee60 con 0x7f7624003f60 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.941+0000 7f7638c00640 1 -- 172.21.15.7:0/2989052206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7630018070 con 0x7f7624003f60 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.941+0000 7f7643c00640 1 -- 172.21.15.7:0/2989052206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763c1539d0 con 0x7f7624003f60 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.941+0000 7f7638c00640 1 -- 172.21.15.7:0/2989052206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763000cad0 con 0x7f7624003f60 2024-04-25T11:16:40.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.941+0000 7f7643c00640 1 -- 172.21.15.7:0/2989052206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763c154dd0 con 0x7f7624003f60 2024-04-25T11:16:40.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.942+0000 7f7643c00640 1 -- 172.21.15.7:0/2989052206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f763c050740 con 0x7f7624003f60 2024-04-25T11:16:40.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.943+0000 7f7638c00640 1 -- 172.21.15.7:0/2989052206 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f763000cc70 con 0x7f7624003f60 2024-04-25T11:16:40.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.944+0000 7f7638c00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f761c0744e0 0x7f761c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:40.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.944+0000 7f7638c00640 1 -- 172.21.15.7:0/2989052206 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f7630095210 con 0x7f7624003f60 2024-04-25T11:16:40.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.945+0000 7f763be00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f761c0744e0 0x7f761c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:40.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.945+0000 7f763be00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f761c0744e0 0x7f761c0769f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f763c09c4e0 tx=0x7f762802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:40.949+0000 7f7638c00640 1 -- 172.21.15.7:0/2989052206 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f763c1539d0 con 0x7f7624003f60 2024-04-25T11:16:41.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.197+0000 7f7643c00640 1 -- 172.21.15.7:0/2989052206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f763c0b2550 con 0x7f7624003f60 2024-04-25T11:16:41.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.198+0000 7f7638c00640 1 -- 172.21.15.7:0/2989052206 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f763c09e1e0 con 0x7f7624003f60 2024-04-25T11:16:41.198 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:41.199 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:16:41.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 -- 172.21.15.7:0/2989052206 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f761c0744e0 msgr2=0x7f761c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:41.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f761c0744e0 0x7f761c0769f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f763c09c4e0 tx=0x7f762802e000 comp rx=0 tx=0).stop 2024-04-25T11:16:41.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 -- 172.21.15.7:0/2989052206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 msgr2=0x7f763c0a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:41.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 0x7f763c0a0470 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f763000e7b0 tx=0x7f763000ec70 comp rx=0 tx=0).stop 2024-04-25T11:16:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 -- 172.21.15.7:0/2989052206 shutdown_connections 2024-04-25T11:16:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f761c0744e0 0x7f761c0769f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f763c153080 0x7f763c153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f763c09d960 0x7f763c1442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 --2- 172.21.15.7:0/2989052206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7624003f60 0x7f763c0a0470 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 -- 172.21.15.7:0/2989052206 >> 172.21.15.7:0/2989052206 conn(0x7f763c0b7560 msgr2=0x7f763c06e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 -- 172.21.15.7:0/2989052206 shutdown_connections 2024-04-25T11:16:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:41.201+0000 7f7643c00640 1 -- 172.21.15.7:0/2989052206 wait complete. 2024-04-25T11:16:41.230 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:16:41.230 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:16:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:42.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:16:42.425 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:16:43.531 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:45.534 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:46.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:16:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.522+0000 7ff1e6000640 1 Processor -- start 2024-04-25T11:16:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.522+0000 7ff1e6000640 1 -- start start 2024-04-25T11:16:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.525+0000 7ff1e6000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1e015e2f0 0x7ff1e0161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.525+0000 7ff1e6000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1e015e700 con 0x7ff1e015baa0 2024-04-25T11:16:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.525+0000 7ff1e6000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1e015e980 con 0x7ff1e0162420 2024-04-25T11:16:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.525+0000 7ff1e6000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff1e0162830 con 0x7ff1e015e2f0 2024-04-25T11:16:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.525+0000 7ff1d7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1e015e2f0 0x7ff1e0161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.525+0000 7ff1e4c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff1e015baa0 0x7ff1e0070640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46308/0 (socket says 172.21.15.7:46308) 2024-04-25T11:16:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.525+0000 7ff1e4c00640 1 -- 172.21.15.7:0/983300666 learned_addr learned my addr 172.21.15.7:0/983300666 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.526+0000 7ff1d7e00640 1 -- 172.21.15.7:0/983300666 >> v1:172.21.15.87:6789/0 conn(0x7ff1e0162420 legacy=0x7ff1e006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.526+0000 7ff1d7e00640 1 -- 172.21.15.7:0/983300666 >> v1:172.21.15.7:6789/0 conn(0x7ff1e015baa0 legacy=0x7ff1e0070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.526+0000 7ff1d7e00640 1 -- 172.21.15.7:0/983300666 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e015ec60 con 0x7ff1e015e2f0 2024-04-25T11:16:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.526+0000 7ff1dfe00640 1 -- 172.21.15.7:0/983300666 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3164251884 0 0) 0x7ff1d0001110 con 0x7ff1e015baa0 2024-04-25T11:16:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.527+0000 7ff1d7e00640 1 --2- 172.21.15.7:0/983300666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e015e2f0 0x7ff1e0161ee0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80097a0 tx=0x7ff1c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15245fe6b997451d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.527+0000 7ff1dfe00640 1 -- 172.21.15.7:0/983300666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8037070 con 0x7ff1e015e2f0 2024-04-25T11:16:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.527+0000 7ff1dfe00640 1 -- 172.21.15.7:0/983300666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c8032440 con 0x7ff1e015e2f0 2024-04-25T11:16:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.527+0000 7ff1dfe00640 1 -- 172.21.15.7:0/983300666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8032810 con 0x7ff1e015e2f0 2024-04-25T11:16:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.528+0000 7ff1e6000640 1 -- 172.21.15.7:0/983300666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e015e2f0 msgr2=0x7ff1e0161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.528+0000 7ff1e6000640 1 --2- 172.21.15.7:0/983300666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e015e2f0 0x7ff1e0161ee0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80097a0 tx=0x7ff1c802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:16:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.528+0000 7ff1e6000640 1 -- 172.21.15.7:0/983300666 shutdown_connections 2024-04-25T11:16:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.528+0000 7ff1e6000640 1 --2- 172.21.15.7:0/983300666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e015e2f0 0x7ff1e0161ee0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.528+0000 7ff1e6000640 1 -- 172.21.15.7:0/983300666 >> 172.21.15.7:0/983300666 conn(0x7ff1e0073090 msgr2=0x7ff1e00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.528+0000 7ff1e6000640 1 -- 172.21.15.7:0/983300666 shutdown_connections 2024-04-25T11:16:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.528+0000 7ff1e6000640 1 -- 172.21.15.7:0/983300666 wait complete. 2024-04-25T11:16:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.530+0000 7ff1e6000640 1 Processor -- start 2024-04-25T11:16:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.531+0000 7ff1e6000640 1 -- start start 2024-04-25T11:16:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.532+0000 7ff1e6000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1e015e2f0 0x7ff1e01f9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.532+0000 7ff1e6000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1e0162420 0x7ff1e01f9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.532+0000 7ff1e6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e015c920 0x7ff1e015cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.532+0000 7ff1e6000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e006dc90 con 0x7ff1e015c920 2024-04-25T11:16:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.532+0000 7ff1e6000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e01619c0 con 0x7ff1e015e2f0 2024-04-25T11:16:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.532+0000 7ff1e6000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e006de10 con 0x7ff1e0162420 2024-04-25T11:16:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.533+0000 7ff1e5600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e015c920 0x7ff1e015cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.533+0000 7ff1e5600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e015c920 0x7ff1e015cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48122/0 (socket says 172.21.15.7:48122) 2024-04-25T11:16:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.533+0000 7ff1e5600640 1 -- 172.21.15.7:0/1798729890 learned_addr learned my addr 172.21.15.7:0/1798729890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.533+0000 7ff1e4c00640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1e015e2f0 0x7ff1e01f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.533+0000 7ff1d7e00640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1e0162420 0x7ff1e01f9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.533+0000 7ff1e5600640 1 -- 172.21.15.7:0/1798729890 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1e0162420 msgr2=0x7ff1e01f9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.533+0000 7ff1e5600640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1e0162420 0x7ff1e01f9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.533+0000 7ff1e5600640 1 -- 172.21.15.7:0/1798729890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1e015e2f0 msgr2=0x7ff1e01f9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.533+0000 7ff1e5600640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1e015e2f0 0x7ff1e01f9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.533+0000 7ff1e5600640 1 -- 172.21.15.7:0/1798729890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e015d980 con 0x7ff1e015c920 2024-04-25T11:16:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.533+0000 7ff1e5600640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e015c920 0x7ff1e015cd30 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800b9e0 tx=0x7ff1d800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.534+0000 7ff1dd600640 1 -- 172.21.15.7:0/1798729890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8003b70 con 0x7ff1e015c920 2024-04-25T11:16:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.534+0000 7ff1dd600640 1 -- 172.21.15.7:0/1798729890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d8003d10 con 0x7ff1e015c920 2024-04-25T11:16:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.534+0000 7ff1dd600640 1 -- 172.21.15.7:0/1798729890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d800cb60 con 0x7ff1e015c920 2024-04-25T11:16:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.534+0000 7ff1e6000640 1 -- 172.21.15.7:0/1798729890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e0134640 con 0x7ff1e015c920 2024-04-25T11:16:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.534+0000 7ff1e6000640 1 -- 172.21.15.7:0/1798729890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e01361a0 con 0x7ff1e015c920 2024-04-25T11:16:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.535+0000 7ff1e6000640 1 -- 172.21.15.7:0/1798729890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e0074020 con 0x7ff1e015c920 2024-04-25T11:16:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.537+0000 7ff1dd600640 1 -- 172.21.15.7:0/1798729890 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff1d800cd00 con 0x7ff1e015c920 2024-04-25T11:16:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.538+0000 7ff1dd600640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1bc0744e0 0x7ff1bc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.538+0000 7ff1dd600640 1 -- 172.21.15.7:0/1798729890 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7ff1d8095320 con 0x7ff1e015c920 2024-04-25T11:16:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.538+0000 7ff1e4c00640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1bc0744e0 0x7ff1bc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.539+0000 7ff1e4c00640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1bc0744e0 0x7ff1bc0769f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0000f60 tx=0x7ff1d0007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.543+0000 7ff1dd600640 1 -- 172.21.15.7:0/1798729890 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff1d805f880 con 0x7ff1e015c920 2024-04-25T11:16:46.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.791+0000 7ff1e6000640 1 -- 172.21.15.7:0/1798729890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff1e00688f0 con 0x7ff1e015c920 2024-04-25T11:16:46.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.792+0000 7ff1dd600640 1 -- 172.21.15.7:0/1798729890 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7ff1d805f180 con 0x7ff1e015c920 2024-04-25T11:16:46.792 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:46.792 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:16:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.794+0000 7ff1e6000640 1 -- 172.21.15.7:0/1798729890 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1bc0744e0 msgr2=0x7ff1bc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.794+0000 7ff1e6000640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1bc0744e0 0x7ff1bc0769f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0000f60 tx=0x7ff1d0007c00 comp rx=0 tx=0).stop 2024-04-25T11:16:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.794+0000 7ff1e6000640 1 -- 172.21.15.7:0/1798729890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e015c920 msgr2=0x7ff1e015cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.794+0000 7ff1e6000640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e015c920 0x7ff1e015cd30 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800b9e0 tx=0x7ff1d800bea0 comp rx=0 tx=0).stop 2024-04-25T11:16:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.795+0000 7ff1e6000640 1 -- 172.21.15.7:0/1798729890 shutdown_connections 2024-04-25T11:16:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.795+0000 7ff1e6000640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff1bc0744e0 0x7ff1bc0769f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.795+0000 7ff1e6000640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1e015c920 0x7ff1e015cd30 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.795+0000 7ff1e6000640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1e0162420 0x7ff1e01f9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.795+0000 7ff1e6000640 1 --2- 172.21.15.7:0/1798729890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1e015e2f0 0x7ff1e01f9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:46.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.795+0000 7ff1e6000640 1 -- 172.21.15.7:0/1798729890 >> 172.21.15.7:0/1798729890 conn(0x7ff1e0073090 msgr2=0x7ff1e006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:46.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.795+0000 7ff1e6000640 1 -- 172.21.15.7:0/1798729890 shutdown_connections 2024-04-25T11:16:46.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:46.795+0000 7ff1e6000640 1 -- 172.21.15.7:0/1798729890 wait complete. 2024-04-25T11:16:46.823 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:16:46.823 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:16:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:49.538 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:51.540 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:51.823 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:16:52.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.115+0000 7f6c10c00640 1 Processor -- start 2024-04-25T11:16:52.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.115+0000 7f6c10c00640 1 -- start start 2024-04-25T11:16:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.118+0000 7f6c10c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c06d580 0x7f6c0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.118+0000 7f6c10c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c0c0b5660 con 0x7f6c0c0b6960 2024-04-25T11:16:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.118+0000 7f6c10c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c0c06ded0 con 0x7f6c0c0d1350 2024-04-25T11:16:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.118+0000 7f6c10c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0c0b64d0 con 0x7f6c0c06d580 2024-04-25T11:16:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.118+0000 7f6c09e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6c0c0b6960 0x7f6c0c160220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52110/0 (socket says 172.21.15.7:52110) 2024-04-25T11:16:52.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.118+0000 7f6c09e00640 1 -- 172.21.15.7:0/2108336261 learned_addr learned my addr 172.21.15.7:0/2108336261 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:52.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.118+0000 7f6c0a800640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c06d580 0x7f6c0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.118+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2371592023 0 0) 0x7f6bfc001110 con 0x7f6c0c0b6960 2024-04-25T11:16:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bf0003520 con 0x7f6c0c0b6960 2024-04-25T11:16:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3076263055 0 0) 0x7f6bf0003520 con 0x7f6c0c0b6960 2024-04-25T11:16:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c06d580 msgr2=0x7f6c0c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c06d580 0x7f6c0c06d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 >> v1:172.21.15.87:6789/0 conn(0x7f6c0c0d1350 legacy=0x7f6c0c1648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0c0b58e0 con 0x7f6c0c0b6960 2024-04-25T11:16:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f6bfc001840 con 0x7f6c0c0b6960 2024-04-25T11:16:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 >> v1:172.21.15.7:6789/0 conn(0x7f6c0c0b6960 legacy=0x7f6c0c160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:52.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c06d580 0x7f6bf0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:52.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf0003fd0 0x7f6bf0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:52.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 0x7f6bf00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:52.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c0b64d0 con 0x7f6bf0004d40 2024-04-25T11:16:52.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf000bbd0 con 0x7f6bf0003fd0 2024-04-25T11:16:52.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.119+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6bf000bdd0 con 0x7f6c0c06d580 2024-04-25T11:16:52.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.120+0000 7f6c0a800640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 0x7f6bf00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:52.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.120+0000 7f6c03e00640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf0003fd0 0x7f6bf0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:52.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.120+0000 7f6c09e00640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c06d580 0x7f6bf0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:52.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.120+0000 7f6c0a800640 1 -- 172.21.15.7:0/2108336261 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c06d580 msgr2=0x7f6bf0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:52.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.120+0000 7f6c0a800640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c06d580 0x7f6bf0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:52.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.120+0000 7f6c0a800640 1 -- 172.21.15.7:0/2108336261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf0003fd0 msgr2=0x7f6bf0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:52.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.120+0000 7f6c0a800640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf0003fd0 0x7f6bf0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:52.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.120+0000 7f6c0a800640 1 -- 172.21.15.7:0/2108336261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf0003520 con 0x7f6bf0004d40 2024-04-25T11:16:52.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.121+0000 7f6c0a800640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 0x7f6bf00051d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c073260 tx=0x7f6bf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3be694fc50abb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.122+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04009c10 con 0x7f6bf0004d40 2024-04-25T11:16:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.122+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c04036070 con 0x7f6bf0004d40 2024-04-25T11:16:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.122+0000 7f6c09400640 1 -- 172.21.15.7:0/2108336261 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040317f0 con 0x7f6bf0004d40 2024-04-25T11:16:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.123+0000 7f6c10c00640 1 -- 172.21.15.7:0/2108336261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 msgr2=0x7f6bf00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.123+0000 7f6c10c00640 1 --2- 172.21.15.7:0/2108336261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 0x7f6bf00051d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c073260 tx=0x7f6bf0005830 comp rx=0 tx=0).stop 2024-04-25T11:16:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.123+0000 7f6c0a800640 1 -- 172.21.15.7:0/2108336261 reap_dead start 2024-04-25T11:16:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.123+0000 7f6c10c00640 1 -- 172.21.15.7:0/2108336261 shutdown_connections 2024-04-25T11:16:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.123+0000 7f6c10c00640 1 -- 172.21.15.7:0/2108336261 >> 172.21.15.7:0/2108336261 conn(0x7f6c0c0732b0 msgr2=0x7f6c0c0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.123+0000 7f6c10c00640 1 -- 172.21.15.7:0/2108336261 shutdown_connections 2024-04-25T11:16:52.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.123+0000 7f6c10c00640 1 -- 172.21.15.7:0/2108336261 wait complete. 2024-04-25T11:16:52.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.126+0000 7f6c10c00640 1 Processor -- start 2024-04-25T11:16:52.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.126+0000 7f6c10c00640 1 -- start start 2024-04-25T11:16:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.127+0000 7f6c10c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 0x7f6c0c14a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.127+0000 7f6c10c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c1581e0 0x7f6c0c1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.127+0000 7f6c10c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0c0afb30 0x7f6c0c0aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.127+0000 7f6c10c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c0b6160 con 0x7f6bf0004d40 2024-04-25T11:16:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.127+0000 7f6c10c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0c0b5e80 con 0x7f6c0c0afb30 2024-04-25T11:16:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.127+0000 7f6c10c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6c0c06d990 con 0x7f6c0c1581e0 2024-04-25T11:16:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.128+0000 7f6c09e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 0x7f6c0c14a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.128+0000 7f6c09e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 0x7f6c0c14a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48106/0 (socket says 172.21.15.7:48106) 2024-04-25T11:16:52.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.128+0000 7f6c09e00640 1 -- 172.21.15.7:0/547060155 learned_addr learned my addr 172.21.15.7:0/547060155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.128+0000 7f6c03e00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c1581e0 0x7f6c0c1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.128+0000 7f6c0a800640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0c0afb30 0x7f6c0c0aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.128+0000 7f6c09e00640 1 -- 172.21.15.7:0/547060155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c1581e0 msgr2=0x7f6c0c1585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.128+0000 7f6c09e00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c1581e0 0x7f6c0c1585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.128+0000 7f6c09e00640 1 -- 172.21.15.7:0/547060155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0c0afb30 msgr2=0x7f6c0c0aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.128+0000 7f6c09e00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0c0afb30 0x7f6c0c0aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.128+0000 7f6c09e00640 1 -- 172.21.15.7:0/547060155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c0c0b0480 con 0x7f6bf0004d40 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.128+0000 7f6c0a800640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0c0afb30 0x7f6c0c0aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.129+0000 7f6c09e00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 0x7f6c0c14a250 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00e7e0 tx=0x7f6bfc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.129+0000 7f6c02000640 1 -- 172.21.15.7:0/547060155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00c8d0 con 0x7f6bf0004d40 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.129+0000 7f6c02000640 1 -- 172.21.15.7:0/547060155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bfc018070 con 0x7f6bf0004d40 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.129+0000 7f6c02000640 1 -- 172.21.15.7:0/547060155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00ccd0 con 0x7f6bf0004d40 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.129+0000 7f6c10c00640 1 -- 172.21.15.7:0/547060155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c0c162280 con 0x7f6bf0004d40 2024-04-25T11:16:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.129+0000 7f6c10c00640 1 -- 172.21.15.7:0/547060155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c0c162520 con 0x7f6bf0004d40 2024-04-25T11:16:52.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.130+0000 7f6c10c00640 1 -- 172.21.15.7:0/547060155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c0c050740 con 0x7f6bf0004d40 2024-04-25T11:16:52.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.132+0000 7f6c02000640 1 -- 172.21.15.7:0/547060155 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6bfc01d460 con 0x7f6bf0004d40 2024-04-25T11:16:52.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.133+0000 7f6c02000640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6be80744b0 0x7f6be80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:52.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.133+0000 7f6c02000640 1 -- 172.21.15.7:0/547060155 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f6bfc0953d0 con 0x7f6bf0004d40 2024-04-25T11:16:52.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.133+0000 7f6c03e00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6be80744b0 0x7f6be80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:52.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.134+0000 7f6c03e00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6be80744b0 0x7f6be80769c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c15b4e0 tx=0x7f6bf4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:52.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.138+0000 7f6c02000640 1 -- 172.21.15.7:0/547060155 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6c0c0b0480 con 0x7f6bf0004d40 2024-04-25T11:16:52.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.382+0000 7f6c10c00640 1 -- 172.21.15.7:0/547060155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6c0c074130 con 0x7f6bf0004d40 2024-04-25T11:16:52.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.383+0000 7f6c02000640 1 -- 172.21.15.7:0/547060155 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f6c0c0b5660 con 0x7f6bf0004d40 2024-04-25T11:16:52.384 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:52.384 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:16:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.385+0000 7f6c10c00640 1 -- 172.21.15.7:0/547060155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6be80744b0 msgr2=0x7f6be80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.385+0000 7f6c10c00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6be80744b0 0x7f6be80769c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c15b4e0 tx=0x7f6bf4009040 comp rx=0 tx=0).stop 2024-04-25T11:16:52.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.385+0000 7f6c10c00640 1 -- 172.21.15.7:0/547060155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 msgr2=0x7f6c0c14a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:52.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.385+0000 7f6c10c00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 0x7f6c0c14a250 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00e7e0 tx=0x7f6bfc00eca0 comp rx=0 tx=0).stop 2024-04-25T11:16:52.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.385+0000 7f6c10c00640 1 -- 172.21.15.7:0/547060155 shutdown_connections 2024-04-25T11:16:52.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.385+0000 7f6c10c00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6be80744b0 0x7f6be80769c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:52.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.385+0000 7f6c10c00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c0c0afb30 0x7f6c0c0aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:52.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.385+0000 7f6c10c00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6c0c1581e0 0x7f6c0c1585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:52.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.386+0000 7f6c10c00640 1 --2- 172.21.15.7:0/547060155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6bf0004d40 0x7f6c0c14a250 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:52.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.386+0000 7f6c10c00640 1 -- 172.21.15.7:0/547060155 >> 172.21.15.7:0/547060155 conn(0x7f6c0c0732b0 msgr2=0x7f6c0c0b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:52.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.386+0000 7f6c10c00640 1 -- 172.21.15.7:0/547060155 shutdown_connections 2024-04-25T11:16:52.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:52.386+0000 7f6c10c00640 1 -- 172.21.15.7:0/547060155 wait complete. 2024-04-25T11:16:52.415 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:16:52.415 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:16:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:57.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:16:57.546 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:16:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.708+0000 7f3b52a00640 1 Processor -- start 2024-04-25T11:16:57.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.708+0000 7f3b52a00640 1 -- start start 2024-04-25T11:16:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.710+0000 7f3b52a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c070810 0x7f3b4c06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.711+0000 7f3b52a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4c0cffa0 con 0x7f3b4c070c20 2024-04-25T11:16:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.711+0000 7f3b52a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4c06ded0 con 0x7f3b4c06dac0 2024-04-25T11:16:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.711+0000 7f3b52a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c160d90 con 0x7f3b4c070810 2024-04-25T11:16:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.711+0000 7f3b50c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3b4c070c20 0x7f3b4c15d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52122/0 (socket says 172.21.15.7:52122) 2024-04-25T11:16:57.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.711+0000 7f3b50c00640 1 -- 172.21.15.7:0/3320546419 learned_addr learned my addr 172.21.15.7:0/3320546419 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:16:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.711+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585237115 0 0) 0x7f3b38001110 con 0x7f3b4c070c20 2024-04-25T11:16:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.711+0000 7f3b51600640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c070810 0x7f3b4c06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b34005730 con 0x7f3b4c070c20 2024-04-25T11:16:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1074560337 0 0) 0x7f3b34005730 con 0x7f3b4c070c20 2024-04-25T11:16:57.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c070810 msgr2=0x7f3b4c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c070810 0x7f3b4c06d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 >> v1:172.21.15.87:6789/0 conn(0x7f3b4c06dac0 legacy=0x7f3b4c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c161070 con 0x7f3b4c070c20 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f3b38001840 con 0x7f3b4c070c20 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 >> v1:172.21.15.7:6789/0 conn(0x7f3b4c070c20 legacy=0x7f3b4c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c070810 0x7f3b34005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b340061c0 0x7f3b34006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b34006f50 0x7f3b340073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c160d90 con 0x7f3b340061c0 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3400ddc0 con 0x7f3b34006f50 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.712+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3400dfc0 con 0x7f3b4c070810 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.713+0000 7f3b50c00640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b340061c0 0x7f3b34006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.713+0000 7f3b52000640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b34006f50 0x7f3b340073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:57.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.713+0000 7f3b51600640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c070810 0x7f3b34005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.713+0000 7f3b50c00640 1 -- 172.21.15.7:0/3320546419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c070810 msgr2=0x7f3b34005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.713+0000 7f3b50c00640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c070810 0x7f3b34005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.713+0000 7f3b50c00640 1 -- 172.21.15.7:0/3320546419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b34006f50 msgr2=0x7f3b340073c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:16:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.713+0000 7f3b50c00640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b34006f50 0x7f3b340073c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.713+0000 7f3b50c00640 1 -- 172.21.15.7:0/3320546419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b34005730 con 0x7f3b340061c0 2024-04-25T11:16:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.714+0000 7f3b50c00640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b340061c0 0x7f3b34006a10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0cdfa0 tx=0x7f3b34007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6d426c54936bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.714+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38036070 con 0x7f3b340061c0 2024-04-25T11:16:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.714+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b38005ad0 con 0x7f3b340061c0 2024-04-25T11:16:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.714+0000 7f3b4be00640 1 -- 172.21.15.7:0/3320546419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38007e10 con 0x7f3b340061c0 2024-04-25T11:16:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.715+0000 7f3b52a00640 1 -- 172.21.15.7:0/3320546419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b340061c0 msgr2=0x7f3b34006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.715+0000 7f3b52a00640 1 --2- 172.21.15.7:0/3320546419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b340061c0 0x7f3b34006a10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0cdfa0 tx=0x7f3b34007a40 comp rx=0 tx=0).stop 2024-04-25T11:16:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.715+0000 7f3b52000640 1 -- 172.21.15.7:0/3320546419 reap_dead start 2024-04-25T11:16:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.715+0000 7f3b52a00640 1 -- 172.21.15.7:0/3320546419 shutdown_connections 2024-04-25T11:16:57.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.715+0000 7f3b52a00640 1 -- 172.21.15.7:0/3320546419 >> 172.21.15.7:0/3320546419 conn(0x7f3b4c073090 msgr2=0x7f3b4c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.715+0000 7f3b52a00640 1 -- 172.21.15.7:0/3320546419 shutdown_connections 2024-04-25T11:16:57.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.715+0000 7f3b52a00640 1 -- 172.21.15.7:0/3320546419 wait complete. 2024-04-25T11:16:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.718+0000 7f3b52a00640 1 Processor -- start 2024-04-25T11:16:57.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.718+0000 7f3b52a00640 1 -- start start 2024-04-25T11:16:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.719+0000 7f3b52a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b340061c0 0x7f3b4c1f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.720+0000 7f3b52a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c13c1f0 0x7f3b4c13c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.720+0000 7f3b52a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c13e4b0 0x7f3b4c13e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.720+0000 7f3b52a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c15e0f0 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.720+0000 7f3b52a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c161630 con 0x7f3b340061c0 2024-04-25T11:16:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.720+0000 7f3b52a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c1433b0 con 0x7f3b4c13e4b0 2024-04-25T11:16:57.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.720+0000 7f3b50c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c13c1f0 0x7f3b4c13c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.720+0000 7f3b50c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c13c1f0 0x7f3b4c13c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48124/0 (socket says 172.21.15.7:48124) 2024-04-25T11:16:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.720+0000 7f3b50c00640 1 -- 172.21.15.7:0/3752118286 learned_addr learned my addr 172.21.15.7:0/3752118286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:16:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.720+0000 7f3b51600640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b340061c0 0x7f3b4c1f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:57.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.720+0000 7f3b52000640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c13e4b0 0x7f3b4c13e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.720+0000 7f3b50c00640 1 -- 172.21.15.7:0/3752118286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c13e4b0 msgr2=0x7f3b4c13e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.721+0000 7f3b50c00640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c13e4b0 0x7f3b4c13e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.721+0000 7f3b50c00640 1 -- 172.21.15.7:0/3752118286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b340061c0 msgr2=0x7f3b4c1f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.721+0000 7f3b50c00640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b340061c0 0x7f3b4c1f27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.721+0000 7f3b50c00640 1 -- 172.21.15.7:0/3752118286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c13ee00 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.721+0000 7f3b50c00640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c13c1f0 0x7f3b4c13c600 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800c3b0 tx=0x7f3b4c161240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.721+0000 7f3b49600640 1 -- 172.21.15.7:0/3752118286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38036040 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.721+0000 7f3b49600640 1 -- 172.21.15.7:0/3752118286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b38007720 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.721+0000 7f3b49600640 1 -- 172.21.15.7:0/3752118286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380015a0 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.721+0000 7f3b52a00640 1 -- 172.21.15.7:0/3752118286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c13ce70 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.722+0000 7f3b52a00640 1 -- 172.21.15.7:0/3752118286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c13d430 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.723+0000 7f3b52a00640 1 -- 172.21.15.7:0/3752118286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c13cbc0 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.724+0000 7f3b49600640 1 -- 172.21.15.7:0/3752118286 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3b38031400 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.725+0000 7f3b49600640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b2c0744b0 0x7f3b2c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:16:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.725+0000 7f3b49600640 1 -- 172.21.15.7:0/3752118286 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f3b380b5b40 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.726+0000 7f3b51600640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b2c0744b0 0x7f3b2c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:16:57.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.726+0000 7f3b51600640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b2c0744b0 0x7f3b2c0769c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009a10 tx=0x7f3b40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:16:57.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.731+0000 7f3b49600640 1 -- 172.21.15.7:0/3752118286 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b3804a030 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.982+0000 7f3b52a00640 1 -- 172.21.15.7:0/3752118286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3b4c050740 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.983+0000 7f3b49600640 1 -- 172.21.15.7:0/3752118286 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f3b4c13ee00 con 0x7f3b4c13c1f0 2024-04-25T11:16:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:16:57.984 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:16:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 -- 172.21.15.7:0/3752118286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b2c0744b0 msgr2=0x7f3b2c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b2c0744b0 0x7f3b2c0769c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009a10 tx=0x7f3b40008040 comp rx=0 tx=0).stop 2024-04-25T11:16:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 -- 172.21.15.7:0/3752118286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c13c1f0 msgr2=0x7f3b4c13c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:16:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c13c1f0 0x7f3b4c13c600 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800c3b0 tx=0x7f3b4c161240 comp rx=0 tx=0).stop 2024-04-25T11:16:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 -- 172.21.15.7:0/3752118286 shutdown_connections 2024-04-25T11:16:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b2c0744b0 0x7f3b2c0769c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4c13e4b0 0x7f3b4c13e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b4c13c1f0 0x7f3b4c13c600 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 --2- 172.21.15.7:0/3752118286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b340061c0 0x7f3b4c1f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:16:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 -- 172.21.15.7:0/3752118286 >> 172.21.15.7:0/3752118286 conn(0x7f3b4c073090 msgr2=0x7f3b4c0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:16:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 -- 172.21.15.7:0/3752118286 shutdown_connections 2024-04-25T11:16:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:16:57.986+0000 7f3b52a00640 1 -- 172.21.15.7:0/3752118286 wait complete. 2024-04-25T11:16:58.015 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:16:58.015 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:16:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:01.549 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:03.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:17:03.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.313+0000 7f71c7200640 1 Processor -- start 2024-04-25T11:17:03.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.314+0000 7f71c7200640 1 -- start start 2024-04-25T11:17:03.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.316+0000 7f71c7200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71c015baa0 0x7f71c015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:03.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.316+0000 7f71c7200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c015e980 con 0x7f71c0070640 2024-04-25T11:17:03.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.316+0000 7f71c7200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71c015ec60 con 0x7f71c006d580 2024-04-25T11:17:03.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.316+0000 7f71c7200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f71c015ef40 con 0x7f71c015baa0 2024-04-25T11:17:03.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.316+0000 7f71c5e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71c015baa0 0x7f71c015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:03.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.316+0000 7f71c5400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f71c0070640 0x7f71c0161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39164/0 (socket says 172.21.15.7:39164) 2024-04-25T11:17:03.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.316+0000 7f71c5400640 1 -- 172.21.15.7:0/2335143093 learned_addr learned my addr 172.21.15.7:0/2335143093 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:17:03.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.317+0000 7f71c5e00640 1 -- 172.21.15.7:0/2335143093 >> v1:172.21.15.87:6789/0 conn(0x7f71c006d580 legacy=0x7f71c006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:03.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.317+0000 7f71c5e00640 1 -- 172.21.15.7:0/2335143093 >> v1:172.21.15.7:6789/0 conn(0x7f71c0070640 legacy=0x7f71c0161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:03.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.317+0000 7f71c5e00640 1 -- 172.21.15.7:0/2335143093 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c01619c0 con 0x7f71c015baa0 2024-04-25T11:17:03.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.318+0000 7f71c5e00640 1 --2- 172.21.15.7:0/2335143093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c015baa0 0x7f71c015e2f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f71b40097a0 tx=0x7f71b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6dbe224007be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:03.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.318+0000 7f71c4a00640 1 -- 172.21.15.7:0/2335143093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4037070 con 0x7f71c015baa0 2024-04-25T11:17:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.318+0000 7f71c4a00640 1 -- 172.21.15.7:0/2335143093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b4032440 con 0x7f71c015baa0 2024-04-25T11:17:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.318+0000 7f71c4a00640 1 -- 172.21.15.7:0/2335143093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4032810 con 0x7f71c015baa0 2024-04-25T11:17:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.319+0000 7f71c7200640 1 -- 172.21.15.7:0/2335143093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c015baa0 msgr2=0x7f71c015e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:03.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.319+0000 7f71c7200640 1 --2- 172.21.15.7:0/2335143093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c015baa0 0x7f71c015e2f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f71b40097a0 tx=0x7f71b402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:17:03.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.319+0000 7f71c7200640 1 -- 172.21.15.7:0/2335143093 shutdown_connections 2024-04-25T11:17:03.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.319+0000 7f71c7200640 1 --2- 172.21.15.7:0/2335143093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c015baa0 0x7f71c015e2f0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:03.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.319+0000 7f71c7200640 1 -- 172.21.15.7:0/2335143093 >> 172.21.15.7:0/2335143093 conn(0x7f71c0073090 msgr2=0x7f71c00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:03.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.319+0000 7f71c7200640 1 -- 172.21.15.7:0/2335143093 shutdown_connections 2024-04-25T11:17:03.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.319+0000 7f71c7200640 1 -- 172.21.15.7:0/2335143093 wait complete. 2024-04-25T11:17:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.322+0000 7f71c7200640 1 Processor -- start 2024-04-25T11:17:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.322+0000 7f71c7200640 1 -- start start 2024-04-25T11:17:03.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.323+0000 7f71c7200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71c006d580 0x7f71c01f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:03.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.323+0000 7f71c7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0070640 0x7f71c00aa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:03.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.323+0000 7f71c7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71c00aa550 0x7f71c0145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.323+0000 7f71c7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0162770 con 0x7f71c0070640 2024-04-25T11:17:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.323+0000 7f71c7200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01625f0 con 0x7f71c006d580 2024-04-25T11:17:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.323+0000 7f71c7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f71c015f220 con 0x7f71c00aa550 2024-04-25T11:17:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.324+0000 7f71c5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0070640 0x7f71c00aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.324+0000 7f71c5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0070640 0x7f71c00aa010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55538/0 (socket says 172.21.15.7:55538) 2024-04-25T11:17:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.324+0000 7f71c5400640 1 -- 172.21.15.7:0/3535962389 learned_addr learned my addr 172.21.15.7:0/3535962389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.324+0000 7f71c5e00640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71c006d580 0x7f71c01f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.324+0000 7f71c6800640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71c00aa550 0x7f71c0145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.324+0000 7f71c5400640 1 -- 172.21.15.7:0/3535962389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71c00aa550 msgr2=0x7f71c0145d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.324+0000 7f71c5400640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71c00aa550 0x7f71c0145d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.324+0000 7f71c5400640 1 -- 172.21.15.7:0/3535962389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71c006d580 msgr2=0x7f71c01f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.324+0000 7f71c5400640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71c006d580 0x7f71c01f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.324+0000 7f71c5400640 1 -- 172.21.15.7:0/3535962389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c00aad80 con 0x7f71c0070640 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.325+0000 7f71c5400640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0070640 0x7f71c00aa010 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f71b000c3c0 tx=0x7f71b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.325+0000 7f71ba000640 1 -- 172.21.15.7:0/3535962389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0017070 con 0x7f71c0070640 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.325+0000 7f71c7200640 1 -- 172.21.15.7:0/3535962389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c00ab060 con 0x7f71c0070640 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.325+0000 7f71ba000640 1 -- 172.21.15.7:0/3535962389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b0005ce0 con 0x7f71c0070640 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.325+0000 7f71ba000640 1 -- 172.21.15.7:0/3535962389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00072f0 con 0x7f71c0070640 2024-04-25T11:17:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.325+0000 7f71c7200640 1 -- 172.21.15.7:0/3535962389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c00ab620 con 0x7f71c0070640 2024-04-25T11:17:03.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.327+0000 7f71c7200640 1 -- 172.21.15.7:0/3535962389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c00aaaf0 con 0x7f71c0070640 2024-04-25T11:17:03.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.328+0000 7f71ba000640 1 -- 172.21.15.7:0/3535962389 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f71b001a070 con 0x7f71c0070640 2024-04-25T11:17:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.329+0000 7f71ba000640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71a40744b0 0x7f71a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:03.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.329+0000 7f71ba000640 1 -- 172.21.15.7:0/3535962389 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f71b0094810 con 0x7f71c0070640 2024-04-25T11:17:03.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.330+0000 7f71c5e00640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71a40744b0 0x7f71a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:03.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.331+0000 7f71c5e00640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71a40744b0 0x7f71a40769c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f71b4034000 tx=0x7f71b402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:03.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.334+0000 7f71ba000640 1 -- 172.21.15.7:0/3535962389 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f71b005ed90 con 0x7f71c0070640 2024-04-25T11:17:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:03.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.582+0000 7f71c7200640 1 -- 172.21.15.7:0/3535962389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f71c0050740 con 0x7f71c0070640 2024-04-25T11:17:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.583+0000 7f71ba000640 1 -- 172.21.15.7:0/3535962389 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f71b005e690 con 0x7f71c0070640 2024-04-25T11:17:03.585 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:17:03.585 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:17:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.586+0000 7f71c7200640 1 -- 172.21.15.7:0/3535962389 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71a40744b0 msgr2=0x7f71a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.586+0000 7f71c7200640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71a40744b0 0x7f71a40769c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f71b4034000 tx=0x7f71b402e000 comp rx=0 tx=0).stop 2024-04-25T11:17:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.586+0000 7f71c7200640 1 -- 172.21.15.7:0/3535962389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0070640 msgr2=0x7f71c00aa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.586+0000 7f71c7200640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0070640 0x7f71c00aa010 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f71b000c3c0 tx=0x7f71b000c880 comp rx=0 tx=0).stop 2024-04-25T11:17:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.586+0000 7f71c7200640 1 -- 172.21.15.7:0/3535962389 shutdown_connections 2024-04-25T11:17:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.587+0000 7f71c7200640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f71a40744b0 0x7f71a40769c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.587+0000 7f71c7200640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71c00aa550 0x7f71c0145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.587+0000 7f71c7200640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71c0070640 0x7f71c00aa010 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.587+0000 7f71c7200640 1 --2- 172.21.15.7:0/3535962389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71c006d580 0x7f71c01f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.587+0000 7f71c7200640 1 -- 172.21.15.7:0/3535962389 >> 172.21.15.7:0/3535962389 conn(0x7f71c0073090 msgr2=0x7f71c015f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.587+0000 7f71c7200640 1 -- 172.21.15.7:0/3535962389 shutdown_connections 2024-04-25T11:17:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:03.587+0000 7f71c7200640 1 -- 172.21.15.7:0/3535962389 wait complete. 2024-04-25T11:17:03.618 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:17:03.618 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:17:05.553 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:07.555 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:08.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:17:08.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.908+0000 7fe16b200640 1 Processor -- start 2024-04-25T11:17:08.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.908+0000 7fe16b200640 1 -- start start 2024-04-25T11:17:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.911+0000 7fe16b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe16c15baa0 0x7fe16c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.911+0000 7fe16b200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe16c15e980 con 0x7fe16c070640 2024-04-25T11:17:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.911+0000 7fe16b200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe16c15ec60 con 0x7fe16c06d580 2024-04-25T11:17:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.911+0000 7fe16b200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe16c15ef40 con 0x7fe16c15baa0 2024-04-25T11:17:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.911+0000 7fe163e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe16c070640 0x7fe16c161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39178/0 (socket says 172.21.15.7:39178) 2024-04-25T11:17:08.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.911+0000 7fe169e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe16c15baa0 0x7fe16c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.911+0000 7fe163e00640 1 -- 172.21.15.7:0/183060975 learned_addr learned my addr 172.21.15.7:0/183060975 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:17:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.912+0000 7fe169e00640 1 -- 172.21.15.7:0/183060975 >> v1:172.21.15.87:6789/0 conn(0x7fe16c06d580 legacy=0x7fe16c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.912+0000 7fe169e00640 1 -- 172.21.15.7:0/183060975 >> v1:172.21.15.7:6789/0 conn(0x7fe16c070640 legacy=0x7fe16c161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:08.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.912+0000 7fe169e00640 1 -- 172.21.15.7:0/183060975 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16c1619c0 con 0x7fe16c15baa0 2024-04-25T11:17:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.913+0000 7fe169e00640 1 --2- 172.21.15.7:0/183060975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c15baa0 0x7fe16c15e2f0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0097a0 tx=0x7fe15c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4f9277e4f47284 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.913+0000 7fe169400640 1 -- 172.21.15.7:0/183060975 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c037070 con 0x7fe16c15baa0 2024-04-25T11:17:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.913+0000 7fe169400640 1 -- 172.21.15.7:0/183060975 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe15c032440 con 0x7fe16c15baa0 2024-04-25T11:17:08.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.913+0000 7fe169400640 1 -- 172.21.15.7:0/183060975 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c032810 con 0x7fe16c15baa0 2024-04-25T11:17:08.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.914+0000 7fe16b200640 1 -- 172.21.15.7:0/183060975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c15baa0 msgr2=0x7fe16c15e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:08.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.914+0000 7fe16b200640 1 --2- 172.21.15.7:0/183060975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c15baa0 0x7fe16c15e2f0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0097a0 tx=0x7fe15c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:17:08.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.914+0000 7fe16b200640 1 -- 172.21.15.7:0/183060975 shutdown_connections 2024-04-25T11:17:08.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.914+0000 7fe16b200640 1 --2- 172.21.15.7:0/183060975 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c15baa0 0x7fe16c15e2f0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:08.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.914+0000 7fe16b200640 1 -- 172.21.15.7:0/183060975 >> 172.21.15.7:0/183060975 conn(0x7fe16c073090 msgr2=0x7fe16c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:08.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.914+0000 7fe16b200640 1 -- 172.21.15.7:0/183060975 shutdown_connections 2024-04-25T11:17:08.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.914+0000 7fe16b200640 1 -- 172.21.15.7:0/183060975 wait complete. 2024-04-25T11:17:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.916+0000 7fe16b200640 1 Processor -- start 2024-04-25T11:17:08.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.917+0000 7fe16b200640 1 -- start start 2024-04-25T11:17:08.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.918+0000 7fe16b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c06d580 0x7fe16c1e8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:08.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.918+0000 7fe16b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe16c070640 0x7fe16c1eaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:08.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.918+0000 7fe16b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c1eaf90 0x7fe16c15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:08.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.918+0000 7fe16b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c162770 con 0x7fe16c06d580 2024-04-25T11:17:08.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.918+0000 7fe16b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1625f0 con 0x7fe16c1eaf90 2024-04-25T11:17:08.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.918+0000 7fe16b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe16c15f220 con 0x7fe16c070640 2024-04-25T11:17:08.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.918+0000 7fe169e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c06d580 0x7fe16c1e8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:08.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.918+0000 7fe169e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c06d580 0x7fe16c1e8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55562/0 (socket says 172.21.15.7:55562) 2024-04-25T11:17:08.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.918+0000 7fe169e00640 1 -- 172.21.15.7:0/3092717275 learned_addr learned my addr 172.21.15.7:0/3092717275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:08.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.919+0000 7fe16a800640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c1eaf90 0x7fe16c15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:08.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.919+0000 7fe169e00640 1 -- 172.21.15.7:0/3092717275 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe16c070640 msgr2=0x7fe16c1eaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:08.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.919+0000 7fe169e00640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe16c070640 0x7fe16c1eaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:08.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.919+0000 7fe169e00640 1 -- 172.21.15.7:0/3092717275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c1eaf90 msgr2=0x7fe16c15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:08.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.919+0000 7fe169e00640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c1eaf90 0x7fe16c15caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:08.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.919+0000 7fe169e00640 1 -- 172.21.15.7:0/3092717275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16c1eb3a0 con 0x7fe16c06d580 2024-04-25T11:17:08.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.919+0000 7fe169e00640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c06d580 0x7fe16c1e8dc0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe15c000c00 tx=0x7fe15c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.920+0000 7fe162000640 1 -- 172.21.15.7:0/3092717275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c041070 con 0x7fe16c06d580 2024-04-25T11:17:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.920+0000 7fe162000640 1 -- 172.21.15.7:0/3092717275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe15c03c890 con 0x7fe16c06d580 2024-04-25T11:17:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.920+0000 7fe162000640 1 -- 172.21.15.7:0/3092717275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c037040 con 0x7fe16c06d580 2024-04-25T11:17:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.920+0000 7fe16b200640 1 -- 172.21.15.7:0/3092717275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16c1e99d0 con 0x7fe16c06d580 2024-04-25T11:17:08.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.920+0000 7fe16b200640 1 -- 172.21.15.7:0/3092717275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16c1e9f90 con 0x7fe16c06d580 2024-04-25T11:17:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.921+0000 7fe16b200640 1 -- 172.21.15.7:0/3092717275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe16c050740 con 0x7fe16c06d580 2024-04-25T11:17:08.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.922+0000 7fe162000640 1 -- 172.21.15.7:0/3092717275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe15c044070 con 0x7fe16c06d580 2024-04-25T11:17:08.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.923+0000 7fe162000640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 0x7fe1480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:08.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.924+0000 7fe162000640 1 -- 172.21.15.7:0/3092717275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7fe15c0b6140 con 0x7fe16c06d580 2024-04-25T11:17:08.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.924+0000 7fe163e00640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 0x7fe1480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:08.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.925+0000 7fe163e00640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 0x7fe1480769c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe154000f60 tx=0x7fe154007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:08.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:08.929+0000 7fe162000640 1 -- 172.21.15.7:0/3092717275 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe16c1eb3a0 con 0x7fe16c06d580 2024-04-25T11:17:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.174+0000 7fe16b200640 1 -- 172.21.15.7:0/3092717275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe16c074020 con 0x7fe16c06d580 2024-04-25T11:17:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.175+0000 7fe162000640 1 -- 172.21.15.7:0/3092717275 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+308 (secure 0 0 0) 0x7fe15c080630 con 0x7fe16c06d580 2024-04-25T11:17:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:17:09.176 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 2 pgs peering","count":2},"muted":false}},"mutes":[]} 2024-04-25T11:17:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.177+0000 7fe16b200640 1 -- 172.21.15.7:0/3092717275 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 msgr2=0x7fe1480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.177+0000 7fe16b200640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 0x7fe1480769c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe154000f60 tx=0x7fe154007a80 comp rx=0 tx=0).stop 2024-04-25T11:17:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.177+0000 7fe16b200640 1 -- 172.21.15.7:0/3092717275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c06d580 msgr2=0x7fe16c1e8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.177+0000 7fe16b200640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c06d580 0x7fe16c1e8dc0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe15c000c00 tx=0x7fe15c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:17:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.178+0000 7fe16b200640 1 -- 172.21.15.7:0/3092717275 shutdown_connections 2024-04-25T11:17:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.178+0000 7fe16b200640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1480744b0 0x7fe1480769c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.178+0000 7fe16b200640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c1eaf90 0x7fe16c15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.178+0000 7fe16b200640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe16c070640 0x7fe16c1eaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:09.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.178+0000 7fe16b200640 1 --2- 172.21.15.7:0/3092717275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c06d580 0x7fe16c1e8dc0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.178+0000 7fe16b200640 1 -- 172.21.15.7:0/3092717275 >> 172.21.15.7:0/3092717275 conn(0x7fe16c073090 msgr2=0x7fe16c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.178+0000 7fe16b200640 1 -- 172.21.15.7:0/3092717275 shutdown_connections 2024-04-25T11:17:09.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:09.178+0000 7fe16b200640 1 -- 172.21.15.7:0/3092717275 wait complete. 2024-04-25T11:17:09.208 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache', 'Reduced data availability: 2 pgs peering']) 2024-04-25T11:17:09.208 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:17:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:11.559 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:12.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:17:12.563 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:17:13.561 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:14.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:17:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.505+0000 7f13b9c00640 1 Processor -- start 2024-04-25T11:17:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.505+0000 7f13b9c00640 1 -- start start 2024-04-25T11:17:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.508+0000 7f13b9c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13b415e2f0 0x7f13b4161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.508+0000 7f13b9c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13b4162420 0x7f13b406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.508+0000 7f13b9c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13b415e700 con 0x7f13b415baa0 2024-04-25T11:17:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.508+0000 7f13b9c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f13b4162830 con 0x7f13b4162420 2024-04-25T11:17:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.508+0000 7f13b9c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f13b415e980 con 0x7f13b415e2f0 2024-04-25T11:17:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.508+0000 7f13b3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f13b415baa0 0x7f13b4070640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55876/0 (socket says 172.21.15.7:55876) 2024-04-25T11:17:14.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.508+0000 7f13b3e00640 1 -- 172.21.15.7:0/604230280 learned_addr learned my addr 172.21.15.7:0/604230280 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:17:14.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.508+0000 7f13b9200640 1 --2- 172.21.15.7:0/604230280 >> v2:172.21.15.7:3300/0 conn(0x7f13b4162420 0x7f13b406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:14.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.508+0000 7f13ab400640 1 --2- 172.21.15.7:0/604230280 >> v2:172.21.15.87:3300/0 conn(0x7f13b415e2f0 0x7f13b4161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:14.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.509+0000 7f13b3400640 1 -- 172.21.15.7:0/604230280 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122682545 0 0) 0x7f13a4001490 con 0x7f13b415baa0 2024-04-25T11:17:14.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.509+0000 7f13b3400640 1 -- 172.21.15.7:0/604230280 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1398003520 con 0x7f13b415baa0 2024-04-25T11:17:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.509+0000 7f13b9200640 1 -- 172.21.15.7:0/604230280 >> v2:172.21.15.87:3300/0 conn(0x7f13b415e2f0 msgr2=0x7f13b4161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.509+0000 7f13b9200640 1 --2- 172.21.15.7:0/604230280 >> v2:172.21.15.87:3300/0 conn(0x7f13b415e2f0 0x7f13b4161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.509+0000 7f13b9200640 1 -- 172.21.15.7:0/604230280 >> v1:172.21.15.7:6789/0 conn(0x7f13b415baa0 legacy=0x7f13b4070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.509+0000 7f13b9200640 1 -- 172.21.15.7:0/604230280 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b415ec60 con 0x7f13b4162420 2024-04-25T11:17:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.510+0000 7f13b9200640 1 --2- 172.21.15.7:0/604230280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4162420 0x7f13b406d580 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f13b4074530 tx=0x7f13ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39c15d2ca4c0d6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:14.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.510+0000 7f13b3400640 1 -- 172.21.15.7:0/604230280 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac037070 con 0x7f13b4162420 2024-04-25T11:17:14.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.510+0000 7f13b3400640 1 -- 172.21.15.7:0/604230280 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13ac032440 con 0x7f13b4162420 2024-04-25T11:17:14.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.510+0000 7f13b3400640 1 -- 172.21.15.7:0/604230280 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac032810 con 0x7f13b4162420 2024-04-25T11:17:14.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.511+0000 7f13b9c00640 1 -- 172.21.15.7:0/604230280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4162420 msgr2=0x7f13b406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.511+0000 7f13b9c00640 1 --2- 172.21.15.7:0/604230280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4162420 0x7f13b406d580 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f13b4074530 tx=0x7f13ac02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:17:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.511+0000 7f13b9c00640 1 -- 172.21.15.7:0/604230280 shutdown_connections 2024-04-25T11:17:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.511+0000 7f13b9c00640 1 --2- 172.21.15.7:0/604230280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4162420 0x7f13b406d580 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:14.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.511+0000 7f13b9c00640 1 --2- 172.21.15.7:0/604230280 >> v2:172.21.15.87:3300/0 conn(0x7f13b415e2f0 0x7f13b4161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:14.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.511+0000 7f13b9c00640 1 -- 172.21.15.7:0/604230280 >> 172.21.15.7:0/604230280 conn(0x7f13b4073090 msgr2=0x7f13b40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:14.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.511+0000 7f13b9c00640 1 -- 172.21.15.7:0/604230280 shutdown_connections 2024-04-25T11:17:14.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.511+0000 7f13b9c00640 1 -- 172.21.15.7:0/604230280 wait complete. 2024-04-25T11:17:14.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.514+0000 7f13b9c00640 1 Processor -- start 2024-04-25T11:17:14.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.514+0000 7f13b9c00640 1 -- start start 2024-04-25T11:17:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.516+0000 7f13b9c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13b415e2f0 0x7f13b414d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.516+0000 7f13b9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4162420 0x7f13b41ea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:14.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.516+0000 7f13b9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13b4160610 0x7f13b4160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.516+0000 7f13b9c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f13b406dac0 con 0x7f13b4162420 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.516+0000 7f13b9c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41619c0 con 0x7f13b4160610 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.516+0000 7f13b9c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13b406dc40 con 0x7f13b415e2f0 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.516+0000 7f13ab400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4162420 0x7f13b41ea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.516+0000 7f13ab400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4162420 0x7f13b41ea2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56724/0 (socket says 172.21.15.7:56724) 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.516+0000 7f13ab400640 1 -- 172.21.15.7:0/2335896557 learned_addr learned my addr 172.21.15.7:0/2335896557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.516+0000 7f13b3e00640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13b415e2f0 0x7f13b414d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.516+0000 7f13b9200640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13b4160610 0x7f13b4160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13ab400640 1 -- 172.21.15.7:0/2335896557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13b415e2f0 msgr2=0x7f13b414d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13ab400640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13b415e2f0 0x7f13b414d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13ab400640 1 -- 172.21.15.7:0/2335896557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13b4160610 msgr2=0x7f13b4160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13ab400640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13b4160610 0x7f13b4160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13ab400640 1 -- 172.21.15.7:0/2335896557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b4160f60 con 0x7f13b4162420 2024-04-25T11:17:14.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13b9200640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13b4160610 0x7f13b4160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:17:14.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13ab400640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4162420 0x7f13b41ea2a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f139c00d620 tx=0x7f139c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:14.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13b0c00640 1 -- 172.21.15.7:0/2335896557 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c00b620 con 0x7f13b4162420 2024-04-25T11:17:14.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13b0c00640 1 -- 172.21.15.7:0/2335896557 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c019070 con 0x7f13b4162420 2024-04-25T11:17:14.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13b0c00640 1 -- 172.21.15.7:0/2335896557 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c00ba00 con 0x7f13b4162420 2024-04-25T11:17:14.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13b9c00640 1 -- 172.21.15.7:0/2335896557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b41ea7e0 con 0x7f13b4162420 2024-04-25T11:17:14.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.517+0000 7f13b9c00640 1 -- 172.21.15.7:0/2335896557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b41e7d50 con 0x7f13b4162420 2024-04-25T11:17:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.519+0000 7f13b9c00640 1 -- 172.21.15.7:0/2335896557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b4071260 con 0x7f13b4162420 2024-04-25T11:17:14.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.520+0000 7f13b0c00640 1 -- 172.21.15.7:0/2335896557 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f139c00bba0 con 0x7f13b4162420 2024-04-25T11:17:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.521+0000 7f13b0c00640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1390074530 0x7f1390076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.521+0000 7f13b0c00640 1 -- 172.21.15.7:0/2335896557 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f139c095880 con 0x7f13b4162420 2024-04-25T11:17:14.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.521+0000 7f13b3e00640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1390074530 0x7f1390076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:14.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.522+0000 7f13b3e00640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1390074530 0x7f1390076a40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f13a4005c00 tx=0x7f13a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:14.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.526+0000 7f13b0c00640 1 -- 172.21.15.7:0/2335896557 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f13b4160f60 con 0x7f13b4162420 2024-04-25T11:17:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.772+0000 7f13b9c00640 1 -- 172.21.15.7:0/2335896557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f13b4074020 con 0x7f13b4162420 2024-04-25T11:17:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.773+0000 7f13b0c00640 1 -- 172.21.15.7:0/2335896557 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f139c066e20 con 0x7f13b4162420 2024-04-25T11:17:14.773 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:17:14.773 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.775+0000 7f13b9c00640 1 -- 172.21.15.7:0/2335896557 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1390074530 msgr2=0x7f1390076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.775+0000 7f13b9c00640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1390074530 0x7f1390076a40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f13a4005c00 tx=0x7f13a4009040 comp rx=0 tx=0).stop 2024-04-25T11:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.775+0000 7f13b9c00640 1 -- 172.21.15.7:0/2335896557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4162420 msgr2=0x7f13b41ea2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.775+0000 7f13b9c00640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4162420 0x7f13b41ea2a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f139c00d620 tx=0x7f139c00dae0 comp rx=0 tx=0).stop 2024-04-25T11:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.776+0000 7f13b9c00640 1 -- 172.21.15.7:0/2335896557 shutdown_connections 2024-04-25T11:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.776+0000 7f13b9c00640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1390074530 0x7f1390076a40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.776+0000 7f13b9c00640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13b4160610 0x7f13b4160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.776+0000 7f13b9c00640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4162420 0x7f13b41ea2a0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.776+0000 7f13b9c00640 1 --2- 172.21.15.7:0/2335896557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13b415e2f0 0x7f13b414d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.776+0000 7f13b9c00640 1 -- 172.21.15.7:0/2335896557 >> 172.21.15.7:0/2335896557 conn(0x7f13b4073090 msgr2=0x7f13b4068780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:14.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.776+0000 7f13b9c00640 1 -- 172.21.15.7:0/2335896557 shutdown_connections 2024-04-25T11:17:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:14.776+0000 7f13b9c00640 1 -- 172.21.15.7:0/2335896557 wait complete. 2024-04-25T11:17:14.804 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:17:14.804 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:17:15.563 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:19.567 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:19.806 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:17:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.101+0000 7f4d30600640 1 Processor -- start 2024-04-25T11:17:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.101+0000 7f4d30600640 1 -- start start 2024-04-25T11:17:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.104+0000 7f4d30600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d2806dc90 0x7f4d28070390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.104+0000 7f4d30600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d28165b20 con 0x7f4d28162330 2024-04-25T11:17:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.104+0000 7f4d30600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d2806f2f0 con 0x7f4d280708d0 2024-04-25T11:17:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.104+0000 7f4d30600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2806f5d0 con 0x7f4d2806dc90 2024-04-25T11:17:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.104+0000 7f4d2dc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4d28162330 0x7f4d2806d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54128/0 (socket says 172.21.15.7:54128) 2024-04-25T11:17:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.104+0000 7f4d2dc00640 1 -- 172.21.15.7:0/2041955766 learned_addr learned my addr 172.21.15.7:0/2041955766 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:17:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2481740035 0 0) 0x7f4d28165b20 con 0x7f4d28162330 2024-04-25T11:17:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d10003520 con 0x7f4d28162330 2024-04-25T11:17:20.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d2d200640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d2806dc90 0x7f4d28070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1971722270 0 0) 0x7f4d10003520 con 0x7f4d28162330 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d2806dc90 msgr2=0x7f4d28070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d2806dc90 0x7f4d28070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 >> v1:172.21.15.87:6789/0 conn(0x7f4d280708d0 legacy=0x7f4d280b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2806f8b0 con 0x7f4d28162330 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f4d1c001670 con 0x7f4d28162330 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 >> v1:172.21.15.7:6789/0 conn(0x7f4d28162330 legacy=0x7f4d2806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d100039e0 0x7f4d10003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d10004800 0x7f4d10004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d10005270 0x7f4d1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2806f5d0 con 0x7f4d10004800 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1000bf50 con 0x7f4d100039e0 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.105+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1000c150 con 0x7f4d10005270 2024-04-25T11:17:20.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.106+0000 7f4d2d200640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d10004800 0x7f4d10004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.106+0000 7f4d2dc00640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d100039e0 0x7f4d10003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.106+0000 7f4d2d200640 1 -- 172.21.15.7:0/2041955766 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d10005270 msgr2=0x7f4d1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.106+0000 7f4d2d200640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d10005270 0x7f4d1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.106+0000 7f4d2d200640 1 -- 172.21.15.7:0/2041955766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d100039e0 msgr2=0x7f4d10003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.106+0000 7f4d2d200640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d100039e0 0x7f4d10003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.106+0000 7f4d2d200640 1 -- 172.21.15.7:0/2041955766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d10003520 con 0x7f4d10004800 2024-04-25T11:17:20.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.106+0000 7f4d2dc00640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d100039e0 0x7f4d10003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:17:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.107+0000 7f4d2d200640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d10004800 0x7f4d10004c10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009ac0 tx=0x7f4d10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7135c1b13e9f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:20.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.108+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14036070 con 0x7f4d10004800 2024-04-25T11:17:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.108+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d14031400 con 0x7f4d10004800 2024-04-25T11:17:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.108+0000 7f4d27e00640 1 -- 172.21.15.7:0/2041955766 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14009690 con 0x7f4d10004800 2024-04-25T11:17:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.108+0000 7f4d30600640 1 -- 172.21.15.7:0/2041955766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d10004800 msgr2=0x7f4d10004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.108+0000 7f4d30600640 1 --2- 172.21.15.7:0/2041955766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d10004800 0x7f4d10004c10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009ac0 tx=0x7f4d10005830 comp rx=0 tx=0).stop 2024-04-25T11:17:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.108+0000 7f4d2e600640 1 -- 172.21.15.7:0/2041955766 reap_dead start 2024-04-25T11:17:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.108+0000 7f4d30600640 1 -- 172.21.15.7:0/2041955766 shutdown_connections 2024-04-25T11:17:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.108+0000 7f4d30600640 1 -- 172.21.15.7:0/2041955766 >> 172.21.15.7:0/2041955766 conn(0x7f4d280732b0 msgr2=0x7f4d280736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.108+0000 7f4d30600640 1 -- 172.21.15.7:0/2041955766 shutdown_connections 2024-04-25T11:17:20.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.108+0000 7f4d30600640 1 -- 172.21.15.7:0/2041955766 wait complete. 2024-04-25T11:17:20.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.111+0000 7f4d30600640 1 Processor -- start 2024-04-25T11:17:20.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.111+0000 7f4d30600640 1 -- start start 2024-04-25T11:17:20.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.112+0000 7f4d30600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d10004800 0x7f4d2809b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:20.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.112+0000 7f4d30600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d2809dae0 0x7f4d28160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:20.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.113+0000 7f4d30600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d2806c570 0x7f4d2806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:20.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.113+0000 7f4d30600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28070e60 con 0x7f4d2806c570 2024-04-25T11:17:20.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.113+0000 7f4d30600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28070ce0 con 0x7f4d2809dae0 2024-04-25T11:17:20.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.113+0000 7f4d30600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2806dc90 con 0x7f4d10004800 2024-04-25T11:17:20.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.113+0000 7f4d2e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d2806c570 0x7f4d2806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:20.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.113+0000 7f4d2e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d2806c570 0x7f4d2806c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38380/0 (socket says 172.21.15.7:38380) 2024-04-25T11:17:20.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.113+0000 7f4d2e600640 1 -- 172.21.15.7:0/2024656049 learned_addr learned my addr 172.21.15.7:0/2024656049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.113+0000 7f4d2d200640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d2809dae0 0x7f4d28160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.113+0000 7f4d2dc00640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d10004800 0x7f4d2809b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d2e600640 1 -- 172.21.15.7:0/2024656049 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d10004800 msgr2=0x7f4d2809b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d2e600640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d10004800 0x7f4d2809b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d2e600640 1 -- 172.21.15.7:0/2024656049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d2809dae0 msgr2=0x7f4d28160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d2e600640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d2809dae0 0x7f4d28160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d2e600640 1 -- 172.21.15.7:0/2024656049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2806cec0 con 0x7f4d2806c570 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d2d200640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d2809dae0 0x7f4d28160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d2e600640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d2806c570 0x7f4d2806c980 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4d2809fbb0 tx=0x7f4d2000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d25600640 1 -- 172.21.15.7:0/2024656049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20017070 con 0x7f4d2806c570 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d25600640 1 -- 172.21.15.7:0/2024656049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d20007070 con 0x7f4d2806c570 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d25600640 1 -- 172.21.15.7:0/2024656049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20007460 con 0x7f4d2806c570 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d30600640 1 -- 172.21.15.7:0/2024656049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2809def0 con 0x7f4d2806c570 2024-04-25T11:17:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.114+0000 7f4d30600640 1 -- 172.21.15.7:0/2024656049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2809e6f0 con 0x7f4d2806c570 2024-04-25T11:17:20.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.116+0000 7f4d30600640 1 -- 172.21.15.7:0/2024656049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d2809e180 con 0x7f4d2806c570 2024-04-25T11:17:20.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.117+0000 7f4d25600640 1 -- 172.21.15.7:0/2024656049 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4d20007600 con 0x7f4d2806c570 2024-04-25T11:17:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.118+0000 7f4d25600640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4d080744e0 0x7f4d080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.118+0000 7f4d25600640 1 -- 172.21.15.7:0/2024656049 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f4d20094520 con 0x7f4d2806c570 2024-04-25T11:17:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.118+0000 7f4d2dc00640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4d080744e0 0x7f4d080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:20.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.119+0000 7f4d2dc00640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4d080744e0 0x7f4d080769f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0017e0 tx=0x7f4d1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:20.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.124+0000 7f4d25600640 1 -- 172.21.15.7:0/2024656049 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4d2806cec0 con 0x7f4d2806c570 2024-04-25T11:17:20.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.369+0000 7f4d30600640 1 -- 172.21.15.7:0/2024656049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4d281fa560 con 0x7f4d2806c570 2024-04-25T11:17:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.370+0000 7f4d25600640 1 -- 172.21.15.7:0/2024656049 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f4d2005eaa0 con 0x7f4d2806c570 2024-04-25T11:17:20.370 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:17:20.371 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:17:20.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 -- 172.21.15.7:0/2024656049 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4d080744e0 msgr2=0x7f4d080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:20.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4d080744e0 0x7f4d080769f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0017e0 tx=0x7f4d1c009040 comp rx=0 tx=0).stop 2024-04-25T11:17:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 -- 172.21.15.7:0/2024656049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d2806c570 msgr2=0x7f4d2806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d2806c570 0x7f4d2806c980 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4d2809fbb0 tx=0x7f4d2000c960 comp rx=0 tx=0).stop 2024-04-25T11:17:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 -- 172.21.15.7:0/2024656049 shutdown_connections 2024-04-25T11:17:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4d080744e0 0x7f4d080769f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d2806c570 0x7f4d2806c980 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d2809dae0 0x7f4d28160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 --2- 172.21.15.7:0/2024656049 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d10004800 0x7f4d2809b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:20.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 -- 172.21.15.7:0/2024656049 >> 172.21.15.7:0/2024656049 conn(0x7f4d280732b0 msgr2=0x7f4d280b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:20.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 -- 172.21.15.7:0/2024656049 shutdown_connections 2024-04-25T11:17:20.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:20.373+0000 7f4d30600640 1 -- 172.21.15.7:0/2024656049 wait complete. 2024-04-25T11:17:20.401 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:17:20.401 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:17:21.569 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:25.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:17:25.573 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:25.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.694+0000 7f73ba000640 1 Processor -- start 2024-04-25T11:17:25.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.694+0000 7f73ba000640 1 -- start start 2024-04-25T11:17:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.697+0000 7f73ba000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f73b4160060 0x7f73b40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.697+0000 7f73ba000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f73b4074060 0x7f73b4074470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.697+0000 7f73ba000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b40af550 con 0x7f73b406d580 2024-04-25T11:17:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.697+0000 7f73ba000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f73b40b0b30 con 0x7f73b4074060 2024-04-25T11:17:25.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.697+0000 7f73ba000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f73b40af830 con 0x7f73b4160060 2024-04-25T11:17:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.697+0000 7f73b3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f73b4074060 0x7f73b4074470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.697+0000 7f73b9600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f73b406d580 0x7f73b406d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54136/0 (socket says 172.21.15.7:54136) 2024-04-25T11:17:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.697+0000 7f73b9600640 1 -- 172.21.15.7:0/3026291020 learned_addr learned my addr 172.21.15.7:0/3026291020 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:17:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.697+0000 7f73b8c00640 1 --2- 172.21.15.7:0/3026291020 >> v2:172.21.15.87:3300/0 conn(0x7f73b4160060 0x7f73b40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.698+0000 7f73b3e00640 1 -- 172.21.15.7:0/3026291020 >> v2:172.21.15.87:3300/0 conn(0x7f73b4160060 msgr2=0x7f73b40b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.698+0000 7f73b3e00640 1 --2- 172.21.15.7:0/3026291020 >> v2:172.21.15.87:3300/0 conn(0x7f73b4160060 0x7f73b40b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.698+0000 7f73b3e00640 1 -- 172.21.15.7:0/3026291020 >> v1:172.21.15.7:6789/0 conn(0x7f73b406d580 legacy=0x7f73b406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:25.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.698+0000 7f73b3e00640 1 -- 172.21.15.7:0/3026291020 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b40afb10 con 0x7f73b4074060 2024-04-25T11:17:25.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.699+0000 7f73b3e00640 1 --2- 172.21.15.7:0/3026291020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b4074060 0x7f73b4074470 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f73b4072a30 tx=0x7f739c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f62911702f9194 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:25.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.699+0000 7f73b3400640 1 -- 172.21.15.7:0/3026291020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c007740 con 0x7f73b4074060 2024-04-25T11:17:25.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.699+0000 7f73b3400640 1 -- 172.21.15.7:0/3026291020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f739c0078e0 con 0x7f73b4074060 2024-04-25T11:17:25.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.699+0000 7f73b3400640 1 -- 172.21.15.7:0/3026291020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c007cb0 con 0x7f73b4074060 2024-04-25T11:17:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.700+0000 7f73ba000640 1 -- 172.21.15.7:0/3026291020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b4074060 msgr2=0x7f73b4074470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.700+0000 7f73ba000640 1 --2- 172.21.15.7:0/3026291020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b4074060 0x7f73b4074470 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f73b4072a30 tx=0x7f739c02a680 comp rx=0 tx=0).stop 2024-04-25T11:17:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.700+0000 7f73ba000640 1 -- 172.21.15.7:0/3026291020 shutdown_connections 2024-04-25T11:17:25.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.700+0000 7f73ba000640 1 --2- 172.21.15.7:0/3026291020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b4074060 0x7f73b4074470 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:25.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.700+0000 7f73ba000640 1 --2- 172.21.15.7:0/3026291020 >> v2:172.21.15.87:3300/0 conn(0x7f73b4160060 0x7f73b40b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:25.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.700+0000 7f73ba000640 1 -- 172.21.15.7:0/3026291020 >> 172.21.15.7:0/3026291020 conn(0x7f73b4073540 msgr2=0x7f73b4073950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:25.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.700+0000 7f73ba000640 1 -- 172.21.15.7:0/3026291020 shutdown_connections 2024-04-25T11:17:25.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.700+0000 7f73ba000640 1 -- 172.21.15.7:0/3026291020 wait complete. 2024-04-25T11:17:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.703+0000 7f73ba000640 1 Processor -- start 2024-04-25T11:17:25.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.703+0000 7f73ba000640 1 -- start start 2024-04-25T11:17:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.704+0000 7f73ba000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73b406d580 0x7f73b41f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.704+0000 7f73ba000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73b4074060 0x7f73b41553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.704+0000 7f73ba000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b4160060 0x7f73b4157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.704+0000 7f73ba000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f73b40af100 con 0x7f73b4160060 2024-04-25T11:17:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.704+0000 7f73ba000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f73b40b00d0 con 0x7f73b406d580 2024-04-25T11:17:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.704+0000 7f73ba000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f73b40af280 con 0x7f73b4074060 2024-04-25T11:17:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.704+0000 7f73b9600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b4160060 0x7f73b4157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.705+0000 7f73b9600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b4160060 0x7f73b4157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38396/0 (socket says 172.21.15.7:38396) 2024-04-25T11:17:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.705+0000 7f73b9600640 1 -- 172.21.15.7:0/4056501940 learned_addr learned my addr 172.21.15.7:0/4056501940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.705+0000 7f73b8c00640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73b406d580 0x7f73b41f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:25.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.705+0000 7f73b3e00640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73b4074060 0x7f73b41553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:25.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.705+0000 7f73b9600640 1 -- 172.21.15.7:0/4056501940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73b4074060 msgr2=0x7f73b41553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.705+0000 7f73b9600640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73b4074060 0x7f73b41553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.705+0000 7f73b9600640 1 -- 172.21.15.7:0/4056501940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73b406d580 msgr2=0x7f73b41f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.705+0000 7f73b9600640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73b406d580 0x7f73b41f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.705+0000 7f73b9600640 1 -- 172.21.15.7:0/4056501940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b415c6b0 con 0x7f73b4160060 2024-04-25T11:17:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.706+0000 7f73b9600640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b4160060 0x7f73b4157710 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00c3a0 tx=0x7f73ac00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.706+0000 7f73b0c00640 1 -- 172.21.15.7:0/4056501940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac017070 con 0x7f73b4160060 2024-04-25T11:17:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.706+0000 7f73b0c00640 1 -- 172.21.15.7:0/4056501940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ac005ce0 con 0x7f73b4160060 2024-04-25T11:17:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.706+0000 7f73b0c00640 1 -- 172.21.15.7:0/4056501940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0072d0 con 0x7f73b4160060 2024-04-25T11:17:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.706+0000 7f73ba000640 1 -- 172.21.15.7:0/4056501940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b415c990 con 0x7f73b4160060 2024-04-25T11:17:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.706+0000 7f73ba000640 1 -- 172.21.15.7:0/4056501940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b415cf50 con 0x7f73b4160060 2024-04-25T11:17:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.707+0000 7f73ba000640 1 -- 172.21.15.7:0/4056501940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b4050740 con 0x7f73b4160060 2024-04-25T11:17:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.709+0000 7f73b0c00640 1 -- 172.21.15.7:0/4056501940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f73ac01a070 con 0x7f73b4160060 2024-04-25T11:17:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.710+0000 7f73b0c00640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73900744b0 0x7f73900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.710+0000 7f73b0c00640 1 -- 172.21.15.7:0/4056501940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f73ac094730 con 0x7f73b4160060 2024-04-25T11:17:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.710+0000 7f73b8c00640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73900744b0 0x7f73900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.711+0000 7f73b8c00640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73900744b0 0x7f73900769c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f73a4006fd0 tx=0x7f73a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:25.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.715+0000 7f73b0c00640 1 -- 172.21.15.7:0/4056501940 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f73ac05e5d0 con 0x7f73b4160060 2024-04-25T11:17:25.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.963+0000 7f73ba000640 1 -- 172.21.15.7:0/4056501940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f73b41630b0 con 0x7f73b4160060 2024-04-25T11:17:25.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.964+0000 7f73b0c00640 1 -- 172.21.15.7:0/4056501940 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f73ac0628d0 con 0x7f73b4160060 2024-04-25T11:17:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:17:25.982 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:17:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.967+0000 7f73ba000640 1 -- 172.21.15.7:0/4056501940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73900744b0 msgr2=0x7f73900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.967+0000 7f73ba000640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73900744b0 0x7f73900769c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f73a4006fd0 tx=0x7f73a4008040 comp rx=0 tx=0).stop 2024-04-25T11:17:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.967+0000 7f73ba000640 1 -- 172.21.15.7:0/4056501940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b4160060 msgr2=0x7f73b4157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.967+0000 7f73ba000640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b4160060 0x7f73b4157710 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00c3a0 tx=0x7f73ac00c860 comp rx=0 tx=0).stop 2024-04-25T11:17:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.967+0000 7f73ba000640 1 -- 172.21.15.7:0/4056501940 shutdown_connections 2024-04-25T11:17:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.967+0000 7f73ba000640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f73900744b0 0x7f73900769c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.967+0000 7f73ba000640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73b4160060 0x7f73b4157710 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.967+0000 7f73ba000640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73b4074060 0x7f73b41553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.967+0000 7f73ba000640 1 --2- 172.21.15.7:0/4056501940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73b406d580 0x7f73b41f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.967+0000 7f73ba000640 1 -- 172.21.15.7:0/4056501940 >> 172.21.15.7:0/4056501940 conn(0x7f73b4073540 msgr2=0x7f73b415b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.967+0000 7f73ba000640 1 -- 172.21.15.7:0/4056501940 shutdown_connections 2024-04-25T11:17:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:25.968+0000 7f73ba000640 1 -- 172.21.15.7:0/4056501940 wait complete. 2024-04-25T11:17:25.996 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:17:25.996 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:17:27.575 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:30.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:17:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.291+0000 7f83c1000640 1 Processor -- start 2024-04-25T11:17:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.291+0000 7f83c1000640 1 -- start start 2024-04-25T11:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.294+0000 7f83c1000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc0b5190 0x7f83bc0d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.294+0000 7f83c1000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f83bc06f390 0x7f83bc06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.294+0000 7f83c1000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83bc0b55a0 con 0x7f83bc06d580 2024-04-25T11:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.294+0000 7f83c1000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f83bc15d560 con 0x7f83bc06f390 2024-04-25T11:17:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.294+0000 7f83c1000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f83bc15d820 con 0x7f83bc0b5190 2024-04-25T11:17:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.295+0000 7f83ba800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f83bc06f390 0x7f83bc06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.295+0000 7f83ba800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f83bc06f390 0x7f83bc06f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:33820/0 (socket says 172.21.15.7:33820) 2024-04-25T11:17:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.295+0000 7f83ba800640 1 -- 172.21.15.7:0/1852058821 learned_addr learned my addr 172.21.15.7:0/1852058821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.295+0000 7f83b9e00640 1 --2- 172.21.15.7:0/1852058821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc0b5190 0x7f83bc0d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.296+0000 7f83b9e00640 1 -- 172.21.15.7:0/1852058821 >> v2:172.21.15.87:3300/0 conn(0x7f83bc06f390 msgr2=0x7f83bc06f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.296+0000 7f83b9e00640 1 --2- 172.21.15.7:0/1852058821 >> v2:172.21.15.87:3300/0 conn(0x7f83bc06f390 0x7f83bc06f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.296+0000 7f83b9e00640 1 -- 172.21.15.7:0/1852058821 >> v1:172.21.15.87:6789/0 conn(0x7f83bc06d580 legacy=0x7f83bc06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.296+0000 7f83b9e00640 1 -- 172.21.15.7:0/1852058821 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc15db00 con 0x7f83bc0b5190 2024-04-25T11:17:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.296+0000 7f83ba800640 1 --2- 172.21.15.7:0/1852058821 >> v2:172.21.15.87:3300/0 conn(0x7f83bc06f390 0x7f83bc06f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:17:31.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.297+0000 7f83b9e00640 1 --2- 172.21.15.7:0/1852058821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc0b5190 0x7f83bc0d0ff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0097a0 tx=0x7f83ac02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff2a185dcca6fd0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:31.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.298+0000 7f83b9400640 1 -- 172.21.15.7:0/1852058821 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac037070 con 0x7f83bc0b5190 2024-04-25T11:17:31.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.298+0000 7f83b9400640 1 -- 172.21.15.7:0/1852058821 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ac032440 con 0x7f83bc0b5190 2024-04-25T11:17:31.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.298+0000 7f83b9400640 1 -- 172.21.15.7:0/1852058821 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac032810 con 0x7f83bc0b5190 2024-04-25T11:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.299+0000 7f83c1000640 1 -- 172.21.15.7:0/1852058821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc0b5190 msgr2=0x7f83bc0d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.299+0000 7f83c1000640 1 --2- 172.21.15.7:0/1852058821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc0b5190 0x7f83bc0d0ff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f83ac0097a0 tx=0x7f83ac02a180 comp rx=0 tx=0).stop 2024-04-25T11:17:31.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.299+0000 7f83c1000640 1 -- 172.21.15.7:0/1852058821 shutdown_connections 2024-04-25T11:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.299+0000 7f83c1000640 1 --2- 172.21.15.7:0/1852058821 >> v2:172.21.15.87:3300/0 conn(0x7f83bc06f390 0x7f83bc06f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.299+0000 7f83c1000640 1 --2- 172.21.15.7:0/1852058821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc0b5190 0x7f83bc0d0ff0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.299+0000 7f83c1000640 1 -- 172.21.15.7:0/1852058821 >> 172.21.15.7:0/1852058821 conn(0x7f83bc0732b0 msgr2=0x7f83bc0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.299+0000 7f83c1000640 1 -- 172.21.15.7:0/1852058821 shutdown_connections 2024-04-25T11:17:31.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.299+0000 7f83c1000640 1 -- 172.21.15.7:0/1852058821 wait complete. 2024-04-25T11:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.302+0000 7f83c1000640 1 Processor -- start 2024-04-25T11:17:31.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.302+0000 7f83c1000640 1 -- start start 2024-04-25T11:17:31.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.303+0000 7f83c1000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc06d580 0x7f83bc14e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:31.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.303+0000 7f83c1000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83bc06f390 0x7f83bc1f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:31.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.303+0000 7f83c1000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83bc0b5190 0x7f83bc14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:31.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.303+0000 7f83c1000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc0d1530 con 0x7f83bc06f390 2024-04-25T11:17:31.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.304+0000 7f83c1000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc15e0c0 con 0x7f83bc0b5190 2024-04-25T11:17:31.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.304+0000 7f83c1000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f83bc09f8e0 con 0x7f83bc06d580 2024-04-25T11:17:31.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.304+0000 7f83b3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83bc06f390 0x7f83bc1f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:31.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.304+0000 7f83b3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83bc06f390 0x7f83bc1f5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50032/0 (socket says 172.21.15.7:50032) 2024-04-25T11:17:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.304+0000 7f83b3e00640 1 -- 172.21.15.7:0/3933714459 learned_addr learned my addr 172.21.15.7:0/3933714459 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.304+0000 7f83b9e00640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc06d580 0x7f83bc14e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.304+0000 7f83ba800640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83bc0b5190 0x7f83bc14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.304+0000 7f83b3e00640 1 -- 172.21.15.7:0/3933714459 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc06d580 msgr2=0x7f83bc14e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.304+0000 7f83b3e00640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc06d580 0x7f83bc14e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.304+0000 7f83b3e00640 1 -- 172.21.15.7:0/3933714459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83bc0b5190 msgr2=0x7f83bc14fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.304+0000 7f83b3e00640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83bc0b5190 0x7f83bc14fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.305+0000 7f83b3e00640 1 -- 172.21.15.7:0/3933714459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc1f6500 con 0x7f83bc06f390 2024-04-25T11:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.305+0000 7f83b3e00640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83bc06f390 0x7f83bc1f5fc0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f83a400c3f0 tx=0x7f83a400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.305+0000 7f83b2000640 1 -- 172.21.15.7:0/3933714459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a40052d0 con 0x7f83bc06f390 2024-04-25T11:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.305+0000 7f83b2000640 1 -- 172.21.15.7:0/3933714459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a4005490 con 0x7f83bc06f390 2024-04-25T11:17:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.305+0000 7f83c1000640 1 -- 172.21.15.7:0/3933714459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc14ee30 con 0x7f83bc06f390 2024-04-25T11:17:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.305+0000 7f83b2000640 1 -- 172.21.15.7:0/3933714459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4007260 con 0x7f83bc06f390 2024-04-25T11:17:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.305+0000 7f83c1000640 1 -- 172.21.15.7:0/3933714459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc14f3f0 con 0x7f83bc06f390 2024-04-25T11:17:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.307+0000 7f83c1000640 1 -- 172.21.15.7:0/3933714459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83bc050740 con 0x7f83bc06f390 2024-04-25T11:17:31.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.308+0000 7f83b2000640 1 -- 172.21.15.7:0/3933714459 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f83a4007400 con 0x7f83bc06f390 2024-04-25T11:17:31.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.309+0000 7f83b2000640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f83980744e0 0x7f83980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:31.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.309+0000 7f83b2000640 1 -- 172.21.15.7:0/3933714459 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f83a4094870 con 0x7f83bc06f390 2024-04-25T11:17:31.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.309+0000 7f83b9e00640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f83980744e0 0x7f83980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:31.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.310+0000 7f83b9e00640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f83980744e0 0x7f83980769f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f83ac034000 tx=0x7f83ac02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.314+0000 7f83b2000640 1 -- 172.21.15.7:0/3933714459 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f83a405ed60 con 0x7f83bc06f390 2024-04-25T11:17:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.565+0000 7f83c1000640 1 -- 172.21.15.7:0/3933714459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f83bc074430 con 0x7f83bc06f390 2024-04-25T11:17:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.566+0000 7f83b2000640 1 -- 172.21.15.7:0/3933714459 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f83a405e660 con 0x7f83bc06f390 2024-04-25T11:17:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:17:31.566 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:17:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 -- 172.21.15.7:0/3933714459 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f83980744e0 msgr2=0x7f83980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f83980744e0 0x7f83980769f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f83ac034000 tx=0x7f83ac02e000 comp rx=0 tx=0).stop 2024-04-25T11:17:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 -- 172.21.15.7:0/3933714459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83bc06f390 msgr2=0x7f83bc1f5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83bc06f390 0x7f83bc1f5fc0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f83a400c3f0 tx=0x7f83a400c8b0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 -- 172.21.15.7:0/3933714459 shutdown_connections 2024-04-25T11:17:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f83980744e0 0x7f83980769f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83bc0b5190 0x7f83bc14fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83bc06f390 0x7f83bc1f5fc0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 --2- 172.21.15.7:0/3933714459 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc06d580 0x7f83bc14e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 -- 172.21.15.7:0/3933714459 >> 172.21.15.7:0/3933714459 conn(0x7f83bc0732b0 msgr2=0x7f83bc06f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 -- 172.21.15.7:0/3933714459 shutdown_connections 2024-04-25T11:17:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:31.569+0000 7f83c1000640 1 -- 172.21.15.7:0/3933714459 wait complete. 2024-04-25T11:17:31.578 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:31.598 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:17:31.598 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:17:33.581 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:35.583 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:36.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:17:36.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.899+0000 7f4605c00640 1 Processor -- start 2024-04-25T11:17:36.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.899+0000 7f4605c00640 1 -- start start 2024-04-25T11:17:36.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.902+0000 7f4605c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f460015f180 0x7f4600161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:36.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.902+0000 7f4605c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46001621a0 0x7f460006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:36.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.902+0000 7f4605c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460006ded0 con 0x7f460006dac0 2024-04-25T11:17:36.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.902+0000 7f4605c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f460015f590 con 0x7f460015f180 2024-04-25T11:17:36.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.902+0000 7f4605c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4600158ae0 con 0x7f46001621a0 2024-04-25T11:17:36.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.902+0000 7f45fea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f460015f180 0x7f4600161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:36.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.902+0000 7f45fea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f460015f180 0x7f4600161c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50036/0 (socket says 172.21.15.7:50036) 2024-04-25T11:17:36.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.902+0000 7f45fea00640 1 -- 172.21.15.7:0/2945957083 learned_addr learned my addr 172.21.15.7:0/2945957083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:36.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.902+0000 7f45fe000640 1 --2- 172.21.15.7:0/2945957083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46001621a0 0x7f460006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:36.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.903+0000 7f45fea00640 1 -- 172.21.15.7:0/2945957083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46001621a0 msgr2=0x7f460006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:36.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.903+0000 7f45fea00640 1 --2- 172.21.15.7:0/2945957083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46001621a0 0x7f460006d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:36.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.903+0000 7f45fea00640 1 -- 172.21.15.7:0/2945957083 >> v1:172.21.15.87:6789/0 conn(0x7f460006dac0 legacy=0x7f460006f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:36.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.903+0000 7f45fea00640 1 -- 172.21.15.7:0/2945957083 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4600162700 con 0x7f460015f180 2024-04-25T11:17:36.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.904+0000 7f45fea00640 1 --2- 172.21.15.7:0/2945957083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f460015f180 0x7f4600161c60 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f45f00098a0 tx=0x7f45f002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23e2033f073011e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:36.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.904+0000 7f45fd600640 1 -- 172.21.15.7:0/2945957083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0037070 con 0x7f460015f180 2024-04-25T11:17:36.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.905+0000 7f45fd600640 1 -- 172.21.15.7:0/2945957083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f0032440 con 0x7f460015f180 2024-04-25T11:17:36.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.905+0000 7f45fd600640 1 -- 172.21.15.7:0/2945957083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0032810 con 0x7f460015f180 2024-04-25T11:17:36.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.905+0000 7f4605c00640 1 -- 172.21.15.7:0/2945957083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f460015f180 msgr2=0x7f4600161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.905+0000 7f4605c00640 1 --2- 172.21.15.7:0/2945957083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f460015f180 0x7f4600161c60 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f45f00098a0 tx=0x7f45f002a290 comp rx=0 tx=0).stop 2024-04-25T11:17:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.905+0000 7f4605c00640 1 -- 172.21.15.7:0/2945957083 shutdown_connections 2024-04-25T11:17:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.905+0000 7f4605c00640 1 --2- 172.21.15.7:0/2945957083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46001621a0 0x7f460006d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.905+0000 7f4605c00640 1 --2- 172.21.15.7:0/2945957083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f460015f180 0x7f4600161c60 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.905+0000 7f4605c00640 1 -- 172.21.15.7:0/2945957083 >> 172.21.15.7:0/2945957083 conn(0x7f46000732b0 msgr2=0x7f46000736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.905+0000 7f4605c00640 1 -- 172.21.15.7:0/2945957083 shutdown_connections 2024-04-25T11:17:36.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.905+0000 7f4605c00640 1 -- 172.21.15.7:0/2945957083 wait complete. 2024-04-25T11:17:36.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.908+0000 7f4605c00640 1 Processor -- start 2024-04-25T11:17:36.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.908+0000 7f4605c00640 1 -- start start 2024-04-25T11:17:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.909+0000 7f4605c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f460006dac0 0x7f46001f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.909+0000 7f4605c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f460015f180 0x7f46001f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f4605c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46001621a0 0x7f460015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f4605c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f460015ec60 con 0x7f460006dac0 2024-04-25T11:17:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f4605c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f460015e980 con 0x7f460015f180 2024-04-25T11:17:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f4605c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f460015e670 con 0x7f46001621a0 2024-04-25T11:17:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f45fea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f460006dac0 0x7f46001f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f45fea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f460006dac0 0x7f46001f7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50042/0 (socket says 172.21.15.7:50042) 2024-04-25T11:17:36.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f45fea00640 1 -- 172.21.15.7:0/3109812058 learned_addr learned my addr 172.21.15.7:0/3109812058 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:36.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f45fe000640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f460015f180 0x7f46001f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f45fea00640 1 -- 172.21.15.7:0/3109812058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46001621a0 msgr2=0x7f460015d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f45fea00640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46001621a0 0x7f460015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f45fea00640 1 -- 172.21.15.7:0/3109812058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f460015f180 msgr2=0x7f46001f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f45fea00640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f460015f180 0x7f46001f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.910+0000 7f45fea00640 1 -- 172.21.15.7:0/3109812058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460014e020 con 0x7f460006dac0 2024-04-25T11:17:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.911+0000 7f45fea00640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f460006dac0 0x7f46001f7670 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f45f00098a0 tx=0x7f45f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.911+0000 7f45f6a00640 1 -- 172.21.15.7:0/3109812058 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0041020 con 0x7f460006dac0 2024-04-25T11:17:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.911+0000 7f45f6a00640 1 -- 172.21.15.7:0/3109812058 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f00306c0 con 0x7f460006dac0 2024-04-25T11:17:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.911+0000 7f4605c00640 1 -- 172.21.15.7:0/3109812058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460014e300 con 0x7f460006dac0 2024-04-25T11:17:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.911+0000 7f45f6a00640 1 -- 172.21.15.7:0/3109812058 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0037040 con 0x7f460006dac0 2024-04-25T11:17:36.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.911+0000 7f4605c00640 1 -- 172.21.15.7:0/3109812058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f460014e8c0 con 0x7f460006dac0 2024-04-25T11:17:36.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.913+0000 7f4605c00640 1 -- 172.21.15.7:0/3109812058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4600050740 con 0x7f460006dac0 2024-04-25T11:17:36.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.914+0000 7f45f6a00640 1 -- 172.21.15.7:0/3109812058 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f45f0044020 con 0x7f460006dac0 2024-04-25T11:17:36.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.915+0000 7f45f6a00640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45e00744b0 0x7f45e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:36.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.915+0000 7f45f6a00640 1 -- 172.21.15.7:0/3109812058 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f45f00b5cf0 con 0x7f460006dac0 2024-04-25T11:17:36.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.915+0000 7f45fe000640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45e00744b0 0x7f45e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:36.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.916+0000 7f45fe000640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45e00744b0 0x7f45e00769c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f45ec006fd0 tx=0x7f45ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:36.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:36.920+0000 7f45f6a00640 1 -- 172.21.15.7:0/3109812058 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f45f004b030 con 0x7f460006dac0 2024-04-25T11:17:37.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.167+0000 7f4605c00640 1 -- 172.21.15.7:0/3109812058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4600074250 con 0x7f460006dac0 2024-04-25T11:17:37.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.168+0000 7f45f6a00640 1 -- 172.21.15.7:0/3109812058 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f45f00801e0 con 0x7f460006dac0 2024-04-25T11:17:37.168 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:17:37.168 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:17:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 -- 172.21.15.7:0/3109812058 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45e00744b0 msgr2=0x7f45e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45e00744b0 0x7f45e00769c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f45ec006fd0 tx=0x7f45ec008040 comp rx=0 tx=0).stop 2024-04-25T11:17:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 -- 172.21.15.7:0/3109812058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f460006dac0 msgr2=0x7f46001f7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f460006dac0 0x7f46001f7670 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f45f00098a0 tx=0x7f45f00046c0 comp rx=0 tx=0).stop 2024-04-25T11:17:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 -- 172.21.15.7:0/3109812058 shutdown_connections 2024-04-25T11:17:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45e00744b0 0x7f45e00769c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46001621a0 0x7f460015d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f460015f180 0x7f46001f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 --2- 172.21.15.7:0/3109812058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f460006dac0 0x7f46001f7670 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 -- 172.21.15.7:0/3109812058 >> 172.21.15.7:0/3109812058 conn(0x7f46000732b0 msgr2=0x7f460006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:37.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 -- 172.21.15.7:0/3109812058 shutdown_connections 2024-04-25T11:17:37.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:37.171+0000 7f4605c00640 1 -- 172.21.15.7:0/3109812058 wait complete. 2024-04-25T11:17:37.198 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:17:37.199 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:17:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:39.587 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:41.589 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:42.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:17:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.498+0000 7f9ef9600640 1 Processor -- start 2024-04-25T11:17:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.498+0000 7f9ef9600640 1 -- start start 2024-04-25T11:17:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.501+0000 7f9ef9600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9ef406d580 0x7f9ef406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.501+0000 7f9ef9600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406f1e0 0x7f9ef406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.501+0000 7f9ef9600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ef406ded0 con 0x7f9ef415f180 2024-04-25T11:17:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.501+0000 7f9ef9600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9ef415f590 con 0x7f9ef406d580 2024-04-25T11:17:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.501+0000 7f9ef9600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef4158ae0 con 0x7f9ef406f1e0 2024-04-25T11:17:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.502+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406f1e0 0x7f9ef406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.501+0000 7f9ef2000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9ef406d580 0x7f9ef406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.502+0000 7f9ef2a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9ef415f180 0x7f9ef4161c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:40374/0 (socket says 172.21.15.7:40374) 2024-04-25T11:17:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.502+0000 7f9ef2a00640 1 -- 172.21.15.7:0/3076721536 learned_addr learned my addr 172.21.15.7:0/3076721536 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:17:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.503+0000 7f9ef2000640 1 -- 172.21.15.7:0/3076721536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406f1e0 msgr2=0x7f9ef406f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.503+0000 7f9ef2000640 1 --2- 172.21.15.7:0/3076721536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406f1e0 0x7f9ef406f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.503+0000 7f9ef2000640 1 -- 172.21.15.7:0/3076721536 >> v1:172.21.15.87:6789/0 conn(0x7f9ef415f180 legacy=0x7f9ef4161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.503+0000 7f9ef2000640 1 -- 172.21.15.7:0/3076721536 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef4162700 con 0x7f9ef406d580 2024-04-25T11:17:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.503+0000 7f9ef3400640 1 --2- 172.21.15.7:0/3076721536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406f1e0 0x7f9ef406f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:17:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.504+0000 7f9ef2000640 1 --2- 172.21.15.7:0/3076721536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ef406d580 0x7f9ef406d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00097d0 tx=0x7f9ee002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab86debf98fb1efd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.504+0000 7f9ef1600640 1 -- 172.21.15.7:0/3076721536 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0037070 con 0x7f9ef406d580 2024-04-25T11:17:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.504+0000 7f9ef1600640 1 -- 172.21.15.7:0/3076721536 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee0032440 con 0x7f9ef406d580 2024-04-25T11:17:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.504+0000 7f9ef1600640 1 -- 172.21.15.7:0/3076721536 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0032810 con 0x7f9ef406d580 2024-04-25T11:17:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.505+0000 7f9ef9600640 1 -- 172.21.15.7:0/3076721536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ef406d580 msgr2=0x7f9ef406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.505+0000 7f9ef9600640 1 --2- 172.21.15.7:0/3076721536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ef406d580 0x7f9ef406d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00097d0 tx=0x7f9ee002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:17:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.505+0000 7f9ef9600640 1 -- 172.21.15.7:0/3076721536 shutdown_connections 2024-04-25T11:17:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.505+0000 7f9ef9600640 1 --2- 172.21.15.7:0/3076721536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406f1e0 0x7f9ef406f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.505+0000 7f9ef9600640 1 --2- 172.21.15.7:0/3076721536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ef406d580 0x7f9ef406d990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.505+0000 7f9ef9600640 1 -- 172.21.15.7:0/3076721536 >> 172.21.15.7:0/3076721536 conn(0x7f9ef40732b0 msgr2=0x7f9ef40736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.505+0000 7f9ef9600640 1 -- 172.21.15.7:0/3076721536 shutdown_connections 2024-04-25T11:17:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.505+0000 7f9ef9600640 1 -- 172.21.15.7:0/3076721536 wait complete. 2024-04-25T11:17:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.508+0000 7f9ef9600640 1 Processor -- start 2024-04-25T11:17:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.508+0000 7f9ef9600640 1 -- start start 2024-04-25T11:17:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.509+0000 7f9ef9600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406d580 0x7f9ef414cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.509+0000 7f9ef9600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ef406f1e0 0x7f9ef41e8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.509+0000 7f9ef9600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ef415f180 0x7f9ef4155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.509+0000 7f9ef9600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef415ec60 con 0x7f9ef415f180 2024-04-25T11:17:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.509+0000 7f9ef9600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef415e980 con 0x7f9ef406f1e0 2024-04-25T11:17:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.509+0000 7f9ef9600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef415e670 con 0x7f9ef406d580 2024-04-25T11:17:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ef415f180 0x7f9ef4155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ef415f180 0x7f9ef4155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33084/0 (socket says 172.21.15.7:33084) 2024-04-25T11:17:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef3400640 1 -- 172.21.15.7:0/385683103 learned_addr learned my addr 172.21.15.7:0/385683103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef2a00640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406d580 0x7f9ef414cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef2000640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ef406f1e0 0x7f9ef41e8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef3400640 1 -- 172.21.15.7:0/385683103 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406d580 msgr2=0x7f9ef414cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef3400640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406d580 0x7f9ef414cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef3400640 1 -- 172.21.15.7:0/385683103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ef406f1e0 msgr2=0x7f9ef41e8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef3400640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ef406f1e0 0x7f9ef41e8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef3400640 1 -- 172.21.15.7:0/385683103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef41e9c50 con 0x7f9ef415f180 2024-04-25T11:17:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef2a00640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406d580 0x7f9ef414cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:17:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.510+0000 7f9ef2000640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ef406f1e0 0x7f9ef41e8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:17:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.511+0000 7f9ef3400640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ef415f180 0x7f9ef4155490 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00ef90 tx=0x7f9eec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.511+0000 7f9eeaa00640 1 -- 172.21.15.7:0/385683103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec018020 con 0x7f9ef415f180 2024-04-25T11:17:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.511+0000 7f9eeaa00640 1 -- 172.21.15.7:0/385683103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eec014480 con 0x7f9ef415f180 2024-04-25T11:17:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.511+0000 7f9eeaa00640 1 -- 172.21.15.7:0/385683103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec014850 con 0x7f9ef415f180 2024-04-25T11:17:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.511+0000 7f9ef9600640 1 -- 172.21.15.7:0/385683103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef40b4bf0 con 0x7f9ef415f180 2024-04-25T11:17:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.511+0000 7f9ef9600640 1 -- 172.21.15.7:0/385683103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef40b51b0 con 0x7f9ef415f180 2024-04-25T11:17:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.512+0000 7f9ef9600640 1 -- 172.21.15.7:0/385683103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef4050740 con 0x7f9ef415f180 2024-04-25T11:17:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.514+0000 7f9eeaa00640 1 -- 172.21.15.7:0/385683103 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9eec005d10 con 0x7f9ef415f180 2024-04-25T11:17:42.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.516+0000 7f9eeaa00640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ed40744b0 0x7f9ed40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:42.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.516+0000 7f9eeaa00640 1 -- 172.21.15.7:0/385683103 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f9eec096180 con 0x7f9ef415f180 2024-04-25T11:17:42.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.516+0000 7f9ef2a00640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ed40744b0 0x7f9ed40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:42.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.517+0000 7f9ef2a00640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ed40744b0 0x7f9ed40769c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005880 tx=0x7f9ee4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:42.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.521+0000 7f9eeaa00640 1 -- 172.21.15.7:0/385683103 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9eec022030 con 0x7f9ef415f180 2024-04-25T11:17:42.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:17:42.693 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:17:42.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.770+0000 7f9ef9600640 1 -- 172.21.15.7:0/385683103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9ef4074250 con 0x7f9ef415f180 2024-04-25T11:17:42.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.770+0000 7f9eeaa00640 1 -- 172.21.15.7:0/385683103 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f9eec010040 con 0x7f9ef415f180 2024-04-25T11:17:42.771 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:17:42.771 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 -- 172.21.15.7:0/385683103 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ed40744b0 msgr2=0x7f9ed40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ed40744b0 0x7f9ed40769c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005880 tx=0x7f9ee4009040 comp rx=0 tx=0).stop 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 -- 172.21.15.7:0/385683103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ef415f180 msgr2=0x7f9ef4155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ef415f180 0x7f9ef4155490 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00ef90 tx=0x7f9eec00c490 comp rx=0 tx=0).stop 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 -- 172.21.15.7:0/385683103 shutdown_connections 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ed40744b0 0x7f9ed40769c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ef415f180 0x7f9ef4155490 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ef406f1e0 0x7f9ef41e8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 --2- 172.21.15.7:0/385683103 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ef406d580 0x7f9ef414cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 -- 172.21.15.7:0/385683103 >> 172.21.15.7:0/385683103 conn(0x7f9ef40732b0 msgr2=0x7f9ef40b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 -- 172.21.15.7:0/385683103 shutdown_connections 2024-04-25T11:17:42.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:42.773+0000 7f9ef9600640 1 -- 172.21.15.7:0/385683103 wait complete. 2024-04-25T11:17:42.800 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:17:42.800 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:17:43.592 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:47.596 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:47.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:17:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.090+0000 7f1b6fe00640 1 Processor -- start 2024-04-25T11:17:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.091+0000 7f1b6fe00640 1 -- start start 2024-04-25T11:17:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.093+0000 7f1b6fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b7015e2f0 0x7f1b70161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.093+0000 7f1b6fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b70162420 0x7f1b7006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.093+0000 7f1b6fe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7015e700 con 0x7f1b7015baa0 2024-04-25T11:17:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.093+0000 7f1b6fe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1b70162830 con 0x7f1b70162420 2024-04-25T11:17:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.093+0000 7f1b6fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7015e980 con 0x7f1b7015e2f0 2024-04-25T11:17:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.093+0000 7f1b6ea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1b7015baa0 0x7f1b70070640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52356/0 (socket says 172.21.15.7:52356) 2024-04-25T11:17:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.093+0000 7f1b6ea00640 1 -- 172.21.15.7:0/1568969498 learned_addr learned my addr 172.21.15.7:0/1568969498 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:17:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.093+0000 7f1b6f400640 1 --2- 172.21.15.7:0/1568969498 >> v2:172.21.15.7:3300/0 conn(0x7f1b70162420 0x7f1b7006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.094+0000 7f1b67e00640 1 --2- 172.21.15.7:0/1568969498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b7015e2f0 0x7f1b70161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.094+0000 7f1b6e000640 1 -- 172.21.15.7:0/1568969498 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1152755280 0 0) 0x7f1b60001110 con 0x7f1b7015baa0 2024-04-25T11:17:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.094+0000 7f1b6e000640 1 -- 172.21.15.7:0/1568969498 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b54003520 con 0x7f1b7015baa0 2024-04-25T11:17:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.094+0000 7f1b6f400640 1 -- 172.21.15.7:0/1568969498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b7015e2f0 msgr2=0x7f1b70161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.094+0000 7f1b6f400640 1 --2- 172.21.15.7:0/1568969498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b7015e2f0 0x7f1b70161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.094+0000 7f1b6f400640 1 -- 172.21.15.7:0/1568969498 >> v1:172.21.15.7:6789/0 conn(0x7f1b7015baa0 legacy=0x7f1b70070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:48.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.094+0000 7f1b6f400640 1 -- 172.21.15.7:0/1568969498 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7015ec60 con 0x7f1b70162420 2024-04-25T11:17:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.095+0000 7f1b6f400640 1 --2- 172.21.15.7:0/1568969498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70162420 0x7f1b7006d580 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009d90 tx=0x7f1b6802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229a16e4da423257 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.095+0000 7f1b6e000640 1 -- 172.21.15.7:0/1568969498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68007740 con 0x7f1b70162420 2024-04-25T11:17:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.095+0000 7f1b6e000640 1 -- 172.21.15.7:0/1568969498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b680078e0 con 0x7f1b70162420 2024-04-25T11:17:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.095+0000 7f1b6e000640 1 -- 172.21.15.7:0/1568969498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68007cb0 con 0x7f1b70162420 2024-04-25T11:17:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.096+0000 7f1b6fe00640 1 -- 172.21.15.7:0/1568969498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70162420 msgr2=0x7f1b7006d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.096+0000 7f1b6fe00640 1 --2- 172.21.15.7:0/1568969498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70162420 0x7f1b7006d580 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009d90 tx=0x7f1b6802a660 comp rx=0 tx=0).stop 2024-04-25T11:17:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.096+0000 7f1b6fe00640 1 -- 172.21.15.7:0/1568969498 shutdown_connections 2024-04-25T11:17:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.096+0000 7f1b6fe00640 1 --2- 172.21.15.7:0/1568969498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b70162420 0x7f1b7006d580 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.096+0000 7f1b6fe00640 1 --2- 172.21.15.7:0/1568969498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b7015e2f0 0x7f1b70161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.096+0000 7f1b6fe00640 1 -- 172.21.15.7:0/1568969498 >> 172.21.15.7:0/1568969498 conn(0x7f1b70073090 msgr2=0x7f1b700734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.096+0000 7f1b6fe00640 1 -- 172.21.15.7:0/1568969498 shutdown_connections 2024-04-25T11:17:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.096+0000 7f1b6fe00640 1 -- 172.21.15.7:0/1568969498 wait complete. 2024-04-25T11:17:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.099+0000 7f1b6fe00640 1 Processor -- start 2024-04-25T11:17:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.099+0000 7f1b6fe00640 1 -- start start 2024-04-25T11:17:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.100+0000 7f1b6fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b7015e2f0 0x7f1b701e9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.100+0000 7f1b6fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b70162420 0x7f1b701eb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.100+0000 7f1b6fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b701ebe10 0x7f1b7015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.100+0000 7f1b6fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7006dac0 con 0x7f1b701ebe10 2024-04-25T11:17:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.100+0000 7f1b6fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701619c0 con 0x7f1b7015e2f0 2024-04-25T11:17:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.100+0000 7f1b6fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7006dc40 con 0x7f1b70162420 2024-04-25T11:17:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b6f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b701ebe10 0x7f1b7015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b6f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b701ebe10 0x7f1b7015d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33102/0 (socket says 172.21.15.7:33102) 2024-04-25T11:17:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b6f400640 1 -- 172.21.15.7:0/3257383971 learned_addr learned my addr 172.21.15.7:0/3257383971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b67e00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b70162420 0x7f1b701eb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b6ea00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b7015e2f0 0x7f1b701e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b6f400640 1 -- 172.21.15.7:0/3257383971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b70162420 msgr2=0x7f1b701eb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b6f400640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b70162420 0x7f1b701eb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b6f400640 1 -- 172.21.15.7:0/3257383971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b7015e2f0 msgr2=0x7f1b701e9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b6f400640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b7015e2f0 0x7f1b701e9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b6f400640 1 -- 172.21.15.7:0/3257383971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b701e9550 con 0x7f1b701ebe10 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b6ea00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b7015e2f0 0x7f1b701e9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.101+0000 7f1b67e00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b70162420 0x7f1b701eb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.102+0000 7f1b6f400640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b701ebe10 0x7f1b7015d370 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1b6802ac50 tx=0x7f1b68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.102+0000 7f1b66a00640 1 -- 172.21.15.7:0/3257383971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6803f070 con 0x7f1b701ebe10 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.102+0000 7f1b66a00640 1 -- 172.21.15.7:0/3257383971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b68031bc0 con 0x7f1b701ebe10 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.102+0000 7f1b66a00640 1 -- 172.21.15.7:0/3257383971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68009450 con 0x7f1b701ebe10 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.102+0000 7f1b6fe00640 1 -- 172.21.15.7:0/3257383971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b701ec590 con 0x7f1b701ebe10 2024-04-25T11:17:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.102+0000 7f1b6fe00640 1 -- 172.21.15.7:0/3257383971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b701fa230 con 0x7f1b701ebe10 2024-04-25T11:17:48.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.103+0000 7f1b6fe00640 1 -- 172.21.15.7:0/3257383971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b70071010 con 0x7f1b701ebe10 2024-04-25T11:17:48.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.105+0000 7f1b66a00640 1 -- 172.21.15.7:0/3257383971 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1b6803b410 con 0x7f1b701ebe10 2024-04-25T11:17:48.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.106+0000 7f1b66a00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b4c0744e0 0x7f1b4c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:48.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.106+0000 7f1b66a00640 1 -- 172.21.15.7:0/3257383971 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f1b680b6c30 con 0x7f1b701ebe10 2024-04-25T11:17:48.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.106+0000 7f1b6ea00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b4c0744e0 0x7f1b4c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:48.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.107+0000 7f1b6ea00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b4c0744e0 0x7f1b4c0769f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1b600012b0 tx=0x7f1b60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.111+0000 7f1b66a00640 1 -- 172.21.15.7:0/3257383971 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b68080bf0 con 0x7f1b701ebe10 2024-04-25T11:17:48.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.354+0000 7f1b6fe00640 1 -- 172.21.15.7:0/3257383971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1b70074020 con 0x7f1b701ebe10 2024-04-25T11:17:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.355+0000 7f1b66a00640 1 -- 172.21.15.7:0/3257383971 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f1b68084ef0 con 0x7f1b701ebe10 2024-04-25T11:17:48.355 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:17:48.355 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:17:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.357+0000 7f1b6fe00640 1 -- 172.21.15.7:0/3257383971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b4c0744e0 msgr2=0x7f1b4c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.357+0000 7f1b6fe00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b4c0744e0 0x7f1b4c0769f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1b600012b0 tx=0x7f1b60009040 comp rx=0 tx=0).stop 2024-04-25T11:17:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.357+0000 7f1b6fe00640 1 -- 172.21.15.7:0/3257383971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b701ebe10 msgr2=0x7f1b7015d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.357+0000 7f1b6fe00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b701ebe10 0x7f1b7015d370 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1b6802ac50 tx=0x7f1b68004a40 comp rx=0 tx=0).stop 2024-04-25T11:17:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.357+0000 7f1b6fe00640 1 -- 172.21.15.7:0/3257383971 shutdown_connections 2024-04-25T11:17:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.357+0000 7f1b6fe00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b4c0744e0 0x7f1b4c0769f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.357+0000 7f1b6fe00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b701ebe10 0x7f1b7015d370 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.357+0000 7f1b6fe00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b70162420 0x7f1b701eb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.357+0000 7f1b6fe00640 1 --2- 172.21.15.7:0/3257383971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b7015e2f0 0x7f1b701e9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.357+0000 7f1b6fe00640 1 -- 172.21.15.7:0/3257383971 >> 172.21.15.7:0/3257383971 conn(0x7f1b70073090 msgr2=0x7f1b7006cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.358+0000 7f1b6fe00640 1 -- 172.21.15.7:0/3257383971 shutdown_connections 2024-04-25T11:17:48.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:48.358+0000 7f1b6fe00640 1 -- 172.21.15.7:0/3257383971 wait complete. 2024-04-25T11:17:48.384 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:17:48.384 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:17:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:51.600 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:53.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:17:53.602 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:53.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.683+0000 7f75f0a00640 1 Processor -- start 2024-04-25T11:17:53.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.683+0000 7f75f0a00640 1 -- start start 2024-04-25T11:17:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.685+0000 7f75f0a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f75ec08cad0 0x7f75ec09cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.685+0000 7f75f0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75ec0b9040 0x7f75ec06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.685+0000 7f75f0a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f75ec06e8d0 0x7f75ec0b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.685+0000 7f75f0a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f75ec06db50 con 0x7f75ec08cad0 2024-04-25T11:17:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.685+0000 7f75f0a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f75ec0d0da0 con 0x7f75ec06e8d0 2024-04-25T11:17:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.685+0000 7f75f0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f75ec0d1080 con 0x7f75ec0b9040 2024-04-25T11:17:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.686+0000 7f75eb400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f75ec08cad0 0x7f75ec09cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.686+0000 7f75eb400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f75ec08cad0 0x7f75ec09cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47586/0 (socket says 172.21.15.7:47586) 2024-04-25T11:17:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.686+0000 7f75eb400640 1 -- 172.21.15.7:0/4049366533 learned_addr learned my addr 172.21.15.7:0/4049366533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.686+0000 7f75eaa00640 1 --2- 172.21.15.7:0/4049366533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75ec0b9040 0x7f75ec06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.686+0000 7f75ebe00640 1 --2- 172.21.15.7:0/4049366533 >> v2:172.21.15.87:3300/0 conn(0x7f75ec06e8d0 0x7f75ec0b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.686+0000 7f75eb400640 1 -- 172.21.15.7:0/4049366533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75ec0b9040 msgr2=0x7f75ec06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.686+0000 7f75eb400640 1 --2- 172.21.15.7:0/4049366533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75ec0b9040 0x7f75ec06e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.686+0000 7f75eb400640 1 -- 172.21.15.7:0/4049366533 >> v2:172.21.15.87:3300/0 conn(0x7f75ec06e8d0 msgr2=0x7f75ec0b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.686+0000 7f75eb400640 1 --2- 172.21.15.7:0/4049366533 >> v2:172.21.15.87:3300/0 conn(0x7f75ec06e8d0 0x7f75ec0b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.686+0000 7f75eb400640 1 -- 172.21.15.7:0/4049366533 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec06dd90 con 0x7f75ec08cad0 2024-04-25T11:17:53.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.686+0000 7f75ebe00640 1 --2- 172.21.15.7:0/4049366533 >> v2:172.21.15.87:3300/0 conn(0x7f75ec06e8d0 0x7f75ec0b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:17:53.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.688+0000 7f75eb400640 1 --2- 172.21.15.7:0/4049366533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec08cad0 0x7f75ec09cbc0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f75ec074120 tx=0x7f75dc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2379636761d283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:53.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.688+0000 7f75ea000640 1 -- 172.21.15.7:0/4049366533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc004030 con 0x7f75ec08cad0 2024-04-25T11:17:53.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.688+0000 7f75ea000640 1 -- 172.21.15.7:0/4049366533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75dc0041d0 con 0x7f75ec08cad0 2024-04-25T11:17:53.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.688+0000 7f75ea000640 1 -- 172.21.15.7:0/4049366533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc003610 con 0x7f75ec08cad0 2024-04-25T11:17:53.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.689+0000 7f75f0a00640 1 -- 172.21.15.7:0/4049366533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec08cad0 msgr2=0x7f75ec09cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:53.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.689+0000 7f75f0a00640 1 --2- 172.21.15.7:0/4049366533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec08cad0 0x7f75ec09cbc0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f75ec074120 tx=0x7f75dc02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.689+0000 7f75f0a00640 1 -- 172.21.15.7:0/4049366533 shutdown_connections 2024-04-25T11:17:53.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.689+0000 7f75f0a00640 1 --2- 172.21.15.7:0/4049366533 >> v2:172.21.15.87:3300/0 conn(0x7f75ec06e8d0 0x7f75ec0b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.689+0000 7f75f0a00640 1 --2- 172.21.15.7:0/4049366533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75ec0b9040 0x7f75ec06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.689+0000 7f75f0a00640 1 --2- 172.21.15.7:0/4049366533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec08cad0 0x7f75ec09cbc0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.689+0000 7f75f0a00640 1 -- 172.21.15.7:0/4049366533 >> 172.21.15.7:0/4049366533 conn(0x7f75ec0d0280 msgr2=0x7f75ec0d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.689+0000 7f75f0a00640 1 -- 172.21.15.7:0/4049366533 shutdown_connections 2024-04-25T11:17:53.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.689+0000 7f75f0a00640 1 -- 172.21.15.7:0/4049366533 wait complete. 2024-04-25T11:17:53.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.691+0000 7f75f0a00640 1 Processor -- start 2024-04-25T11:17:53.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.691+0000 7f75f0a00640 1 -- start start 2024-04-25T11:17:53.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.693+0000 7f75f0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75ec06e8d0 0x7f75ec1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:53.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.693+0000 7f75f0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75ec08cad0 0x7f75ec155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:53.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.693+0000 7f75f0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec0b9040 0x7f75ec1564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:53.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.693+0000 7f75f0a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec0d2ed0 con 0x7f75ec0b9040 2024-04-25T11:17:53.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.693+0000 7f75f0a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec09d100 con 0x7f75ec06e8d0 2024-04-25T11:17:53.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.693+0000 7f75f0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f75ec0d3050 con 0x7f75ec08cad0 2024-04-25T11:17:53.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.693+0000 7f75ebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec0b9040 0x7f75ec1564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:53.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.693+0000 7f75ebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec0b9040 0x7f75ec1564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47596/0 (socket says 172.21.15.7:47596) 2024-04-25T11:17:53.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.693+0000 7f75ebe00640 1 -- 172.21.15.7:0/700794591 learned_addr learned my addr 172.21.15.7:0/700794591 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.694+0000 7f75eb400640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75ec06e8d0 0x7f75ec1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.694+0000 7f75eaa00640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75ec08cad0 0x7f75ec155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.694+0000 7f75ebe00640 1 -- 172.21.15.7:0/700794591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75ec08cad0 msgr2=0x7f75ec155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.694+0000 7f75ebe00640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75ec08cad0 0x7f75ec155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.694+0000 7f75ebe00640 1 -- 172.21.15.7:0/700794591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75ec06e8d0 msgr2=0x7f75ec1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.694+0000 7f75ebe00640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75ec06e8d0 0x7f75ec1537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.694+0000 7f75ebe00640 1 -- 172.21.15.7:0/700794591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec0b3ff0 con 0x7f75ec0b9040 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.694+0000 7f75ebe00640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec0b9040 0x7f75ec1564d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f75e400c590 tx=0x7f75e400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.694+0000 7f75e3400640 1 -- 172.21.15.7:0/700794591 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4015070 con 0x7f75ec0b9040 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.695+0000 7f75e3400640 1 -- 172.21.15.7:0/700794591 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e4011800 con 0x7f75ec0b9040 2024-04-25T11:17:53.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.695+0000 7f75e3400640 1 -- 172.21.15.7:0/700794591 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4011bf0 con 0x7f75ec0b9040 2024-04-25T11:17:53.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.695+0000 7f75f0a00640 1 -- 172.21.15.7:0/700794591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ec154150 con 0x7f75ec0b9040 2024-04-25T11:17:53.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.695+0000 7f75f0a00640 1 -- 172.21.15.7:0/700794591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ec154710 con 0x7f75ec0b9040 2024-04-25T11:17:53.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.696+0000 7f75f0a00640 1 -- 172.21.15.7:0/700794591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ec0d1850 con 0x7f75ec0b9040 2024-04-25T11:17:53.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.697+0000 7f75e3400640 1 -- 172.21.15.7:0/700794591 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f75e4018070 con 0x7f75ec0b9040 2024-04-25T11:17:53.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.698+0000 7f75e3400640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75c80744b0 0x7f75c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:53.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.698+0000 7f75e3400640 1 -- 172.21.15.7:0/700794591 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f75e40952e0 con 0x7f75ec0b9040 2024-04-25T11:17:53.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.699+0000 7f75eb400640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75c80744b0 0x7f75c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:53.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.699+0000 7f75eb400640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75c80744b0 0x7f75c80769c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f75dc033000 tx=0x7f75dc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.704+0000 7f75e3400640 1 -- 172.21.15.7:0/700794591 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f75ec0d1850 con 0x7f75ec0b9040 2024-04-25T11:17:53.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.944+0000 7f75f0a00640 1 -- 172.21.15.7:0/700794591 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f75ec0c7330 con 0x7f75ec0b9040 2024-04-25T11:17:53.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.944+0000 7f75e3400640 1 -- 172.21.15.7:0/700794591 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f75ec0c7330 con 0x7f75ec0b9040 2024-04-25T11:17:53.945 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:17:53.946 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:17:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 -- 172.21.15.7:0/700794591 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75c80744b0 msgr2=0x7f75c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75c80744b0 0x7f75c80769c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f75dc033000 tx=0x7f75dc02e000 comp rx=0 tx=0).stop 2024-04-25T11:17:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 -- 172.21.15.7:0/700794591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec0b9040 msgr2=0x7f75ec1564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:53.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec0b9040 0x7f75ec1564d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f75e400c590 tx=0x7f75e400ca50 comp rx=0 tx=0).stop 2024-04-25T11:17:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 -- 172.21.15.7:0/700794591 shutdown_connections 2024-04-25T11:17:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75c80744b0 0x7f75c80769c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75ec0b9040 0x7f75ec1564d0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75ec08cad0 0x7f75ec155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 --2- 172.21.15.7:0/700794591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75ec06e8d0 0x7f75ec1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 -- 172.21.15.7:0/700794591 >> 172.21.15.7:0/700794591 conn(0x7f75ec0d0280 msgr2=0x7f75ec163de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 -- 172.21.15.7:0/700794591 shutdown_connections 2024-04-25T11:17:53.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:53.947+0000 7f75f0a00640 1 -- 172.21.15.7:0/700794591 wait complete. 2024-04-25T11:17:53.975 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:17:53.975 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:17:55.604 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:17:58.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:17:59.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.269+0000 7ff5bd400640 1 Processor -- start 2024-04-25T11:17:59.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.269+0000 7ff5bd400640 1 -- start start 2024-04-25T11:17:59.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.271+0000 7ff5bd400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5b814ab20 0x7ff5b806f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:59.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.271+0000 7ff5bd400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5b806dc90 0x7ff5b80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:59.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.271+0000 7ff5bd400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5b806f920 con 0x7ff5b8168ac0 2024-04-25T11:17:59.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.271+0000 7ff5bd400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff5b806fc00 con 0x7ff5b806dc90 2024-04-25T11:17:59.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.271+0000 7ff5bd400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b806fee0 con 0x7ff5b814ab20 2024-04-25T11:17:59.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.272+0000 7ff5b7400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff5b8168ac0 0x7ff5b806d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38498/0 (socket says 172.21.15.7:38498) 2024-04-25T11:17:59.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.272+0000 7ff5b7400640 1 -- 172.21.15.7:0/1726632611 learned_addr learned my addr 172.21.15.7:0/1726632611 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:17:59.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.272+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3496889115 0 0) 0x7ff5b806f920 con 0x7ff5b8168ac0 2024-04-25T11:17:59.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.272+0000 7ff5bca00640 1 --2- 172.21.15.7:0/1726632611 >> v2:172.21.15.87:3300/0 conn(0x7ff5b806dc90 0x7ff5b80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:59.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.272+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff59c003520 con 0x7ff5b8168ac0 2024-04-25T11:17:59.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.272+0000 7ff5b7e00640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5b814ab20 0x7ff5b806f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:59.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1777144528 0 0) 0x7ff59c003520 con 0x7ff5b8168ac0 2024-04-25T11:17:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5b814ab20 msgr2=0x7ff5b806f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5b814ab20 0x7ff5b806f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 >> v2:172.21.15.87:3300/0 conn(0x7ff5b806dc90 msgr2=0x7ff5b80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 --2- 172.21.15.7:0/1726632611 >> v2:172.21.15.87:3300/0 conn(0x7ff5b806dc90 0x7ff5b80b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b80b4690 con 0x7ff5b8168ac0 2024-04-25T11:17:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7ff5a4001690 con 0x7ff5b8168ac0 2024-04-25T11:17:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 >> v1:172.21.15.7:6789/0 conn(0x7ff5b8168ac0 legacy=0x7ff5b806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff59c0039b0 0x7ff59c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff59c004800 0x7ff59c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c005150 0x7ff59c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:59.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b806fc00 con 0x7ff59c005150 2024-04-25T11:17:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b806fee0 con 0x7ff59c0039b0 2024-04-25T11:17:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff59c00bf00 con 0x7ff59c004800 2024-04-25T11:17:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.273+0000 7ff5bca00640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c005150 0x7ff59c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.274+0000 7ff5b7400640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff59c004800 0x7ff59c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.274+0000 7ff5b7e00640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff59c0039b0 0x7ff59c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.274+0000 7ff5bca00640 1 -- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff59c004800 msgr2=0x7ff59c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.274+0000 7ff5bca00640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff59c004800 0x7ff59c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.274+0000 7ff5bca00640 1 -- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff59c0039b0 msgr2=0x7ff59c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:17:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.274+0000 7ff5bca00640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff59c0039b0 0x7ff59c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.274+0000 7ff5bca00640 1 -- 172.21.15.7:0/1726632611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59c003520 con 0x7ff59c005150 2024-04-25T11:17:59.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.274+0000 7ff5b7e00640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff59c0039b0 0x7ff59c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:17:59.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.275+0000 7ff5bca00640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c005150 0x7ff59c00b9c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009fc0 tx=0x7ff59c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed53e3028773d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:59.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.275+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0009c10 con 0x7ff59c005150 2024-04-25T11:17:59.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.275+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b0036070 con 0x7ff59c005150 2024-04-25T11:17:59.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.275+0000 7ff5b6a00640 1 -- 172.21.15.7:0/1726632611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b002d070 con 0x7ff59c005150 2024-04-25T11:17:59.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.276+0000 7ff5bd400640 1 -- 172.21.15.7:0/1726632611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c005150 msgr2=0x7ff59c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:59.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.276+0000 7ff5bd400640 1 --2- 172.21.15.7:0/1726632611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff59c005150 0x7ff59c00b9c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009fc0 tx=0x7ff59c005830 comp rx=0 tx=0).stop 2024-04-25T11:17:59.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.276+0000 7ff5bca00640 1 -- 172.21.15.7:0/1726632611 reap_dead start 2024-04-25T11:17:59.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.276+0000 7ff5bd400640 1 -- 172.21.15.7:0/1726632611 shutdown_connections 2024-04-25T11:17:59.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.276+0000 7ff5bd400640 1 -- 172.21.15.7:0/1726632611 >> 172.21.15.7:0/1726632611 conn(0x7ff5b8073090 msgr2=0x7ff5b80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:59.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.276+0000 7ff5bd400640 1 -- 172.21.15.7:0/1726632611 shutdown_connections 2024-04-25T11:17:59.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.276+0000 7ff5bd400640 1 -- 172.21.15.7:0/1726632611 wait complete. 2024-04-25T11:17:59.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.279+0000 7ff5bd400640 1 Processor -- start 2024-04-25T11:17:59.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.279+0000 7ff5bd400640 1 -- start start 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.280+0000 7ff5bd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff59c005150 0x7ff5b81467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.280+0000 7ff5bd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b8149d90 0x7ff5b8161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.280+0000 7ff5bd400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5b81fa2a0 0x7ff5b8162df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.280+0000 7ff5bd400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b814aca0 con 0x7ff5b8149d90 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.280+0000 7ff5bd400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b814ab20 con 0x7ff59c005150 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.280+0000 7ff5bd400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b806f0f0 con 0x7ff5b81fa2a0 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5b7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b8149d90 0x7ff5b8161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5b7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b8149d90 0x7ff5b8161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47622/0 (socket says 172.21.15.7:47622) 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5b7400640 1 -- 172.21.15.7:0/2851409489 learned_addr learned my addr 172.21.15.7:0/2851409489 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5b7e00640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff59c005150 0x7ff5b81467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5bca00640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5b81fa2a0 0x7ff5b8162df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5b7400640 1 -- 172.21.15.7:0/2851409489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5b81fa2a0 msgr2=0x7ff5b8162df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:59.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5b7400640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5b81fa2a0 0x7ff5b8162df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:59.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5b7400640 1 -- 172.21.15.7:0/2851409489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff59c005150 msgr2=0x7ff5b81467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:59.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5b7400640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff59c005150 0x7ff5b81467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:59.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5b7400640 1 -- 172.21.15.7:0/2851409489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b81603e0 con 0x7ff5b8149d90 2024-04-25T11:17:59.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5b7e00640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff59c005150 0x7ff5b81467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:17:59.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.281+0000 7ff5b7400640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b8149d90 0x7ff5b8161480 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400e780 tx=0x7ff5a400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:59.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.282+0000 7ff5afe00640 1 -- 172.21.15.7:0/2851409489 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400ee90 con 0x7ff5b8149d90 2024-04-25T11:17:59.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.282+0000 7ff5afe00640 1 -- 172.21.15.7:0/2851409489 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a4018070 con 0x7ff5b8149d90 2024-04-25T11:17:59.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.282+0000 7ff5afe00640 1 -- 172.21.15.7:0/2851409489 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400cb10 con 0x7ff5b8149d90 2024-04-25T11:17:59.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.282+0000 7ff5bd400640 1 -- 172.21.15.7:0/2851409489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b81606c0 con 0x7ff5b8149d90 2024-04-25T11:17:59.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.282+0000 7ff5bd400640 1 -- 172.21.15.7:0/2851409489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b8160c80 con 0x7ff5b8149d90 2024-04-25T11:17:59.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.283+0000 7ff5bd400640 1 -- 172.21.15.7:0/2851409489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b8050740 con 0x7ff5b8149d90 2024-04-25T11:17:59.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.284+0000 7ff5afe00640 1 -- 172.21.15.7:0/2851409489 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff5a400ccb0 con 0x7ff5b8149d90 2024-04-25T11:17:59.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.285+0000 7ff5afe00640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff598074650 0x7ff598076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:17:59.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.285+0000 7ff5afe00640 1 -- 172.21.15.7:0/2851409489 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7ff5a40953f0 con 0x7ff5b8149d90 2024-04-25T11:17:59.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.286+0000 7ff5b7e00640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff598074650 0x7ff598076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:17:59.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.286+0000 7ff5b7e00640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff598074650 0x7ff598076b60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009aa0 tx=0x7ff5a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:17:59.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.291+0000 7ff5afe00640 1 -- 172.21.15.7:0/2851409489 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5a405f8e0 con 0x7ff5b8149d90 2024-04-25T11:17:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.540+0000 7ff5bd400640 1 -- 172.21.15.7:0/2851409489 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff5b8165ec0 con 0x7ff5b8149d90 2024-04-25T11:17:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.541+0000 7ff5afe00640 1 -- 172.21.15.7:0/2851409489 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7ff5a405f1e0 con 0x7ff5b8149d90 2024-04-25T11:17:59.542 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:17:59.543 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:17:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.544+0000 7ff5bd400640 1 -- 172.21.15.7:0/2851409489 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff598074650 msgr2=0x7ff598076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.544+0000 7ff5bd400640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff598074650 0x7ff598076b60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009aa0 tx=0x7ff5a8008040 comp rx=0 tx=0).stop 2024-04-25T11:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.544+0000 7ff5bd400640 1 -- 172.21.15.7:0/2851409489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b8149d90 msgr2=0x7ff5b8161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.544+0000 7ff5bd400640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b8149d90 0x7ff5b8161480 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400e780 tx=0x7ff5a400ec40 comp rx=0 tx=0).stop 2024-04-25T11:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.545+0000 7ff5bd400640 1 -- 172.21.15.7:0/2851409489 shutdown_connections 2024-04-25T11:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.545+0000 7ff5bd400640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff598074650 0x7ff598076b60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.545+0000 7ff5bd400640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5b81fa2a0 0x7ff5b8162df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.545+0000 7ff5bd400640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5b8149d90 0x7ff5b8161480 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.545+0000 7ff5bd400640 1 --2- 172.21.15.7:0/2851409489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff59c005150 0x7ff5b81467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.545+0000 7ff5bd400640 1 -- 172.21.15.7:0/2851409489 >> 172.21.15.7:0/2851409489 conn(0x7ff5b8073090 msgr2=0x7ff5b80cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:17:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.545+0000 7ff5bd400640 1 -- 172.21.15.7:0/2851409489 shutdown_connections 2024-04-25T11:17:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:17:59.545+0000 7ff5bd400640 1 -- 172.21.15.7:0/2851409489 wait complete. 2024-04-25T11:17:59.576 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:17:59.576 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:17:59.607 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:01.610 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:03.612 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:04.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:18:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.862+0000 7f7c32800640 1 Processor -- start 2024-04-25T11:18:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.863+0000 7f7c32800640 1 -- start start 2024-04-25T11:18:04.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.865+0000 7f7c32800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c2c0703a0 0x7f7c2c167ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:04.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.865+0000 7f7c32800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c2c0b55a0 con 0x7f7c2c06d580 2024-04-25T11:18:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.865+0000 7f7c32800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c2c15d560 con 0x7f7c2c0b5190 2024-04-25T11:18:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.865+0000 7f7c32800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7c2c15d820 con 0x7f7c2c0703a0 2024-04-25T11:18:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.865+0000 7f7c23e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7c2c06d580 0x7f7c2c06d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56104/0 (socket says 172.21.15.7:56104) 2024-04-25T11:18:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.866+0000 7f7c23e00640 1 -- 172.21.15.7:0/2215647936 learned_addr learned my addr 172.21.15.7:0/2215647936 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.866+0000 7f7c2be00640 1 --2- 172.21.15.7:0/2215647936 >> v2:172.21.15.87:3300/0 conn(0x7f7c2c0703a0 0x7f7c2c167ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:04.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.866+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189003121 0 0) 0x7f7c2c0b55a0 con 0x7f7c2c06d580 2024-04-25T11:18:04.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.866+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c10003520 con 0x7f7c2c06d580 2024-04-25T11:18:04.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 554988962 0 0) 0x7f7c10003520 con 0x7f7c2c06d580 2024-04-25T11:18:04.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 >> v2:172.21.15.87:3300/0 conn(0x7f7c2c0703a0 msgr2=0x7f7c2c167ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:04.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 --2- 172.21.15.7:0/2215647936 >> v2:172.21.15.87:3300/0 conn(0x7f7c2c0703a0 0x7f7c2c167ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:04.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 >> v1:172.21.15.87:6789/0 conn(0x7f7c2c0b5190 legacy=0x7f7c2c0d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:04.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c15db00 con 0x7f7c2c06d580 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f7c14001670 con 0x7f7c2c06d580 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 >> v1:172.21.15.7:6789/0 conn(0x7f7c2c06d580 legacy=0x7f7c2c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 --2- 172.21.15.7:0/2215647936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 0x7f7c10003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 --2- 172.21.15.7:0/2215647936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c10004800 0x7f7c10004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 --2- 172.21.15.7:0/2215647936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c100052a0 0x7f7c1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c15d820 con 0x7f7c10003a00 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1000c230 con 0x7f7c10004800 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1000c430 con 0x7f7c100052a0 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2b400640 1 --2- 172.21.15.7:0/2215647936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 0x7f7c10003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c23e00640 1 --2- 172.21.15.7:0/2215647936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c10004800 0x7f7c10004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.867+0000 7f7c2be00640 1 --2- 172.21.15.7:0/2215647936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c100052a0 0x7f7c1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.868+0000 7f7c2b400640 1 -- 172.21.15.7:0/2215647936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c100052a0 msgr2=0x7f7c1000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.868+0000 7f7c2b400640 1 --2- 172.21.15.7:0/2215647936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c100052a0 0x7f7c1000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:04.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.868+0000 7f7c2b400640 1 -- 172.21.15.7:0/2215647936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c10004800 msgr2=0x7f7c10004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:04.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.868+0000 7f7c2b400640 1 --2- 172.21.15.7:0/2215647936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c10004800 0x7f7c10004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:04.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.868+0000 7f7c2b400640 1 -- 172.21.15.7:0/2215647936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c10003520 con 0x7f7c10003a00 2024-04-25T11:18:04.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.868+0000 7f7c2be00640 1 -- 172.21.15.7:0/2215647936 reap_dead start 2024-04-25T11:18:04.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.869+0000 7f7c2b400640 1 --2- 172.21.15.7:0/2215647936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 0x7f7c10003e70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0727a0 tx=0x7f7c10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527ece3bbc31ac90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:04.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.869+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c001370 con 0x7f7c10003a00 2024-04-25T11:18:04.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.869+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1c0052d0 con 0x7f7c10003a00 2024-04-25T11:18:04.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.869+0000 7f7c2aa00640 1 -- 172.21.15.7:0/2215647936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c02ab90 con 0x7f7c10003a00 2024-04-25T11:18:04.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.870+0000 7f7c32800640 1 -- 172.21.15.7:0/2215647936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 msgr2=0x7f7c10003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:04.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.870+0000 7f7c32800640 1 --2- 172.21.15.7:0/2215647936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 0x7f7c10003e70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0727a0 tx=0x7f7c10005830 comp rx=0 tx=0).stop 2024-04-25T11:18:04.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.870+0000 7f7c32800640 1 -- 172.21.15.7:0/2215647936 shutdown_connections 2024-04-25T11:18:04.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.870+0000 7f7c32800640 1 --2- 172.21.15.7:0/2215647936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 0x7f7c10003e70 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:04.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.870+0000 7f7c32800640 1 -- 172.21.15.7:0/2215647936 >> 172.21.15.7:0/2215647936 conn(0x7f7c2c0732b0 msgr2=0x7f7c2c0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:04.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.870+0000 7f7c32800640 1 -- 172.21.15.7:0/2215647936 shutdown_connections 2024-04-25T11:18:04.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.870+0000 7f7c32800640 1 -- 172.21.15.7:0/2215647936 wait complete. 2024-04-25T11:18:04.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.873+0000 7f7c32800640 1 Processor -- start 2024-04-25T11:18:04.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.873+0000 7f7c32800640 1 -- start start 2024-04-25T11:18:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.874+0000 7f7c32800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 0x7f7c2c14bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.874+0000 7f7c32800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c2c14fe10 0x7f7c2c150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.874+0000 7f7c32800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c2c0d0740 0x7f7c2c0d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.874+0000 7f7c32800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c09f8e0 con 0x7f7c10003a00 2024-04-25T11:18:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.874+0000 7f7c32800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c15e0c0 con 0x7f7c2c0d0740 2024-04-25T11:18:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.874+0000 7f7c32800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2c09fa60 con 0x7f7c2c14fe10 2024-04-25T11:18:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.875+0000 7f7c2b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 0x7f7c2c14bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.875+0000 7f7c2b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 0x7f7c2c14bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60408/0 (socket says 172.21.15.7:60408) 2024-04-25T11:18:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.875+0000 7f7c2b400640 1 -- 172.21.15.7:0/516216022 learned_addr learned my addr 172.21.15.7:0/516216022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.875+0000 7f7c23e00640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c2c14fe10 0x7f7c2c150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.875+0000 7f7c2be00640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c2c0d0740 0x7f7c2c0d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.875+0000 7f7c2b400640 1 -- 172.21.15.7:0/516216022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c2c14fe10 msgr2=0x7f7c2c150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.875+0000 7f7c2b400640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c2c14fe10 0x7f7c2c150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.875+0000 7f7c2b400640 1 -- 172.21.15.7:0/516216022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c2c0d0740 msgr2=0x7f7c2c0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.875+0000 7f7c2b400640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c2c0d0740 0x7f7c2c0d0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.875+0000 7f7c2b400640 1 -- 172.21.15.7:0/516216022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c0d1090 con 0x7f7c10003a00 2024-04-25T11:18:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.876+0000 7f7c2b400640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 0x7f7c2c14bfb0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0017e0 tx=0x7f7c1c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.876+0000 7f7c23400640 1 -- 172.21.15.7:0/516216022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c03f020 con 0x7f7c10003a00 2024-04-25T11:18:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.876+0000 7f7c23400640 1 -- 172.21.15.7:0/516216022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1c005480 con 0x7f7c10003a00 2024-04-25T11:18:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.876+0000 7f7c23400640 1 -- 172.21.15.7:0/516216022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c037040 con 0x7f7c10003a00 2024-04-25T11:18:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.876+0000 7f7c32800640 1 -- 172.21.15.7:0/516216022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c2c150d40 con 0x7f7c10003a00 2024-04-25T11:18:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.876+0000 7f7c32800640 1 -- 172.21.15.7:0/516216022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c2c151090 con 0x7f7c10003a00 2024-04-25T11:18:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.877+0000 7f7c32800640 1 -- 172.21.15.7:0/516216022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c2c050740 con 0x7f7c10003a00 2024-04-25T11:18:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.879+0000 7f7c23400640 1 -- 172.21.15.7:0/516216022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7c1c042020 con 0x7f7c10003a00 2024-04-25T11:18:04.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.880+0000 7f7c23400640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7c080744b0 0x7f7c080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:04.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.880+0000 7f7c23400640 1 -- 172.21.15.7:0/516216022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f7c1c0b51f0 con 0x7f7c10003a00 2024-04-25T11:18:04.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.880+0000 7f7c23e00640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7c080744b0 0x7f7c080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:04.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.881+0000 7f7c23e00640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7c080744b0 0x7f7c080769c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7c14001050 tx=0x7f7c14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:04.885+0000 7f7c23400640 1 -- 172.21.15.7:0/516216022 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7c1c049030 con 0x7f7c10003a00 2024-04-25T11:18:05.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.131+0000 7f7c32800640 1 -- 172.21.15.7:0/516216022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7c2c0b1240 con 0x7f7c10003a00 2024-04-25T11:18:05.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.132+0000 7f7c23400640 1 -- 172.21.15.7:0/516216022 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f7c2c0d1090 con 0x7f7c10003a00 2024-04-25T11:18:05.133 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:05.133 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:18:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 -- 172.21.15.7:0/516216022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7c080744b0 msgr2=0x7f7c080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7c080744b0 0x7f7c080769c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7c14001050 tx=0x7f7c14009040 comp rx=0 tx=0).stop 2024-04-25T11:18:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 -- 172.21.15.7:0/516216022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 msgr2=0x7f7c2c14bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 0x7f7c2c14bfb0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0017e0 tx=0x7f7c1c001120 comp rx=0 tx=0).stop 2024-04-25T11:18:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 -- 172.21.15.7:0/516216022 shutdown_connections 2024-04-25T11:18:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7c080744b0 0x7f7c080769c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c2c0d0740 0x7f7c2c0d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c2c14fe10 0x7f7c2c150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:05.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 --2- 172.21.15.7:0/516216022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c10003a00 0x7f7c2c14bfb0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:05.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 -- 172.21.15.7:0/516216022 >> 172.21.15.7:0/516216022 conn(0x7f7c2c0732b0 msgr2=0x7f7c2c0b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:05.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 -- 172.21.15.7:0/516216022 shutdown_connections 2024-04-25T11:18:05.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:05.135+0000 7f7c32800640 1 -- 172.21.15.7:0/516216022 wait complete. 2024-04-25T11:18:05.165 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:18:05.165 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:18:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:07.615 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:10.166 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:18:10.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.464+0000 7f959fe00640 1 Processor -- start 2024-04-25T11:18:10.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.464+0000 7f959fe00640 1 -- start start 2024-04-25T11:18:10.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.467+0000 7f959fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f95a014ab20 0x7f95a006f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:10.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.467+0000 7f959fe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a006f920 con 0x7f95a006dc90 2024-04-25T11:18:10.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.467+0000 7f959fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a006fc00 con 0x7f95a0168ac0 2024-04-25T11:18:10.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.467+0000 7f959fe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f95a006fee0 con 0x7f95a014ab20 2024-04-25T11:18:10.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.467+0000 7f959f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f95a006dc90 0x7f95a00b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35470/0 (socket says 172.21.15.7:35470) 2024-04-25T11:18:10.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.467+0000 7f959f400640 1 -- 172.21.15.7:0/1476323307 learned_addr learned my addr 172.21.15.7:0/1476323307 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:10.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959ea00640 1 --2- 172.21.15.7:0/1476323307 >> v2:172.21.15.87:3300/0 conn(0x7f95a014ab20 0x7f95a006f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:10.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 272581331 0 0) 0x7f95a006f920 con 0x7f95a006dc90 2024-04-25T11:18:10.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9584003520 con 0x7f95a006dc90 2024-04-25T11:18:10.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2654024284 0 0) 0x7f9584003520 con 0x7f95a006dc90 2024-04-25T11:18:10.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 >> v2:172.21.15.87:3300/0 conn(0x7f95a014ab20 msgr2=0x7f95a006f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:10.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 --2- 172.21.15.7:0/1476323307 >> v2:172.21.15.87:3300/0 conn(0x7f95a014ab20 0x7f95a006f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:10.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 >> v1:172.21.15.87:6789/0 conn(0x7f95a0168ac0 legacy=0x7f95a006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:10.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a00b4690 con 0x7f95a006dc90 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f9598001670 con 0x7f95a006dc90 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 >> v1:172.21.15.7:6789/0 conn(0x7f95a006dc90 legacy=0x7f95a00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 --2- 172.21.15.7:0/1476323307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95840039b0 0x7f9584003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 --2- 172.21.15.7:0/1476323307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9584004800 0x7f9584004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 --2- 172.21.15.7:0/1476323307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9584005250 0x7f958400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95a006fee0 con 0x7f9584005250 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f958400c1e0 con 0x7f95840039b0 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.468+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f958400c3e0 con 0x7f9584004800 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.469+0000 7f959f400640 1 --2- 172.21.15.7:0/1476323307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9584005250 0x7f958400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.469+0000 7f959ea00640 1 --2- 172.21.15.7:0/1476323307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95840039b0 0x7f9584003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.469+0000 7f959f400640 1 -- 172.21.15.7:0/1476323307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9584004800 msgr2=0x7f9584004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.469+0000 7f959f400640 1 --2- 172.21.15.7:0/1476323307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9584004800 0x7f9584004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.469+0000 7f959f400640 1 -- 172.21.15.7:0/1476323307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95840039b0 msgr2=0x7f9584003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.469+0000 7f959f400640 1 --2- 172.21.15.7:0/1476323307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95840039b0 0x7f9584003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:10.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.469+0000 7f959f400640 1 -- 172.21.15.7:0/1476323307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9584003520 con 0x7f9584005250 2024-04-25T11:18:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.470+0000 7f959f400640 1 -- 172.21.15.7:0/1476323307 reap_dead start 2024-04-25T11:18:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.470+0000 7f959f400640 1 --2- 172.21.15.7:0/1476323307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9584005250 0x7f958400b9c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f959800c210 tx=0x7f9584005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbec5e6dc6a96a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.471+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598007d10 con 0x7f9584005250 2024-04-25T11:18:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.471+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95980058d0 con 0x7f9584005250 2024-04-25T11:18:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.471+0000 7f959d600640 1 -- 172.21.15.7:0/1476323307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598005cc0 con 0x7f9584005250 2024-04-25T11:18:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.471+0000 7f959fe00640 1 -- 172.21.15.7:0/1476323307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9584005250 msgr2=0x7f958400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.471+0000 7f959fe00640 1 --2- 172.21.15.7:0/1476323307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9584005250 0x7f958400b9c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f959800c210 tx=0x7f9584005830 comp rx=0 tx=0).stop 2024-04-25T11:18:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.472+0000 7f959fe00640 1 -- 172.21.15.7:0/1476323307 shutdown_connections 2024-04-25T11:18:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.472+0000 7f959fe00640 1 --2- 172.21.15.7:0/1476323307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9584005250 0x7f958400b9c0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.472+0000 7f959fe00640 1 -- 172.21.15.7:0/1476323307 >> 172.21.15.7:0/1476323307 conn(0x7f95a0073090 msgr2=0x7f95a00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:10.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.472+0000 7f959fe00640 1 -- 172.21.15.7:0/1476323307 shutdown_connections 2024-04-25T11:18:10.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.472+0000 7f959fe00640 1 -- 172.21.15.7:0/1476323307 wait complete. 2024-04-25T11:18:10.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.474+0000 7f959fe00640 1 Processor -- start 2024-04-25T11:18:10.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.474+0000 7f959fe00640 1 -- start start 2024-04-25T11:18:10.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.476+0000 7f959fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9584005250 0x7f95a015b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:10.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.476+0000 7f959fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95a015d8e0 0x7f95a0149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:10.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.476+0000 7f959fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95a00cdc30 0x7f95a015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:10.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.476+0000 7f959fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95a014aca0 con 0x7f95a00cdc30 2024-04-25T11:18:10.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.476+0000 7f959fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f95a014ab20 con 0x7f9584005250 2024-04-25T11:18:10.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.476+0000 7f959fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f95a006f0f0 con 0x7f95a015d8e0 2024-04-25T11:18:10.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.476+0000 7f959f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95a00cdc30 0x7f95a015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:10.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.476+0000 7f959f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95a00cdc30 0x7f95a015aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40226/0 (socket says 172.21.15.7:40226) 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.476+0000 7f959f400640 1 -- 172.21.15.7:0/1633235833 learned_addr learned my addr 172.21.15.7:0/1633235833 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.477+0000 7f959ea00640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9584005250 0x7f95a015b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.477+0000 7f959e000640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95a015d8e0 0x7f95a0149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.477+0000 7f959f400640 1 -- 172.21.15.7:0/1633235833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95a015d8e0 msgr2=0x7f95a0149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.477+0000 7f959f400640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95a015d8e0 0x7f95a0149810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.477+0000 7f959f400640 1 -- 172.21.15.7:0/1633235833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9584005250 msgr2=0x7f95a015b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.477+0000 7f959f400640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9584005250 0x7f95a015b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.477+0000 7f959f400640 1 -- 172.21.15.7:0/1633235833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a015e4a0 con 0x7f95a00cdc30 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.477+0000 7f959f400640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95a00cdc30 0x7f95a015aa20 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f959802ad70 tx=0x7f95a006faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.477+0000 7f9596a00640 1 -- 172.21.15.7:0/1633235833 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598041070 con 0x7f95a00cdc30 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.478+0000 7f9596a00640 1 -- 172.21.15.7:0/1633235833 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9598000ed0 con 0x7f95a00cdc30 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.478+0000 7f959fe00640 1 -- 172.21.15.7:0/1633235833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a015e780 con 0x7f95a00cdc30 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.478+0000 7f9596a00640 1 -- 172.21.15.7:0/1633235833 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598007d10 con 0x7f95a00cdc30 2024-04-25T11:18:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.478+0000 7f959fe00640 1 -- 172.21.15.7:0/1633235833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a015bd30 con 0x7f95a00cdc30 2024-04-25T11:18:10.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.479+0000 7f959fe00640 1 -- 172.21.15.7:0/1633235833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a00ce4c0 con 0x7f95a00cdc30 2024-04-25T11:18:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.480+0000 7f9596a00640 1 -- 172.21.15.7:0/1633235833 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9598044070 con 0x7f95a00cdc30 2024-04-25T11:18:10.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.482+0000 7f9596a00640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f957c0744b0 0x7f957c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:10.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.482+0000 7f9596a00640 1 -- 172.21.15.7:0/1633235833 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f95980b49d0 con 0x7f95a00cdc30 2024-04-25T11:18:10.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.482+0000 7f959ea00640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f957c0744b0 0x7f957c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:10.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.483+0000 7f959ea00640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f957c0744b0 0x7f957c0769c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9588009a10 tx=0x7f9588008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:10.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.486+0000 7f9596a00640 1 -- 172.21.15.7:0/1633235833 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f95a00ce4c0 con 0x7f95a00cdc30 2024-04-25T11:18:10.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.734+0000 7f959fe00640 1 -- 172.21.15.7:0/1633235833 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f95a0050740 con 0x7f95a00cdc30 2024-04-25T11:18:10.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.735+0000 7f9596a00640 1 -- 172.21.15.7:0/1633235833 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f959807ef50 con 0x7f95a00cdc30 2024-04-25T11:18:10.736 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:10.736 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.737+0000 7f959fe00640 1 -- 172.21.15.7:0/1633235833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f957c0744b0 msgr2=0x7f957c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.737+0000 7f959fe00640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f957c0744b0 0x7f957c0769c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9588009a10 tx=0x7f9588008040 comp rx=0 tx=0).stop 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.737+0000 7f959fe00640 1 -- 172.21.15.7:0/1633235833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95a00cdc30 msgr2=0x7f95a015aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.737+0000 7f959fe00640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95a00cdc30 0x7f95a015aa20 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f959802ad70 tx=0x7f95a006faf0 comp rx=0 tx=0).stop 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.738+0000 7f959fe00640 1 -- 172.21.15.7:0/1633235833 shutdown_connections 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.738+0000 7f959fe00640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f957c0744b0 0x7f957c0769c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.738+0000 7f959fe00640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95a00cdc30 0x7f95a015aa20 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.738+0000 7f959fe00640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95a015d8e0 0x7f95a0149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.738+0000 7f959fe00640 1 --2- 172.21.15.7:0/1633235833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9584005250 0x7f95a015b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.738+0000 7f959fe00640 1 -- 172.21.15.7:0/1633235833 >> 172.21.15.7:0/1633235833 conn(0x7f95a0073090 msgr2=0x7f95a01670d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.738+0000 7f959fe00640 1 -- 172.21.15.7:0/1633235833 shutdown_connections 2024-04-25T11:18:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:10.738+0000 7f959fe00640 1 -- 172.21.15.7:0/1633235833 wait complete. 2024-04-25T11:18:10.765 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:18:10.765 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:18:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:12.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:18:12.832 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:18:13.622 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:15.624 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:15.766 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:18:16.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.062+0000 7f3e1ba00640 1 Processor -- start 2024-04-25T11:18:16.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.062+0000 7f3e1ba00640 1 -- start start 2024-04-25T11:18:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.065+0000 7f3e1ba00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e1415f180 0x7f3e14161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.065+0000 7f3e1ba00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3e1406dc90 0x7f3e1406f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.065+0000 7f3e1ba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e14162700 con 0x7f3e141621a0 2024-04-25T11:18:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.065+0000 7f3e1ba00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3e1415f590 con 0x7f3e1415f180 2024-04-25T11:18:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.065+0000 7f3e1ba00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3e14158ae0 con 0x7f3e1406dc90 2024-04-25T11:18:16.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.066+0000 7f3e19000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e1415f180 0x7f3e14161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.066+0000 7f3e19000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e1415f180 0x7f3e14161c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40240/0 (socket says 172.21.15.7:40240) 2024-04-25T11:18:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.066+0000 7f3e19000640 1 -- 172.21.15.7:0/1227512104 learned_addr learned my addr 172.21.15.7:0/1227512104 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.066+0000 7f3e19a00640 1 --2- 172.21.15.7:0/1227512104 >> v2:172.21.15.87:3300/0 conn(0x7f3e1406dc90 0x7f3e1406f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.066+0000 7f3e19000640 1 -- 172.21.15.7:0/1227512104 >> v2:172.21.15.87:3300/0 conn(0x7f3e1406dc90 msgr2=0x7f3e1406f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.066+0000 7f3e19000640 1 --2- 172.21.15.7:0/1227512104 >> v2:172.21.15.87:3300/0 conn(0x7f3e1406dc90 0x7f3e1406f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.066+0000 7f3e19000640 1 -- 172.21.15.7:0/1227512104 >> v1:172.21.15.87:6789/0 conn(0x7f3e141621a0 legacy=0x7f3e1406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:16.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.066+0000 7f3e19000640 1 -- 172.21.15.7:0/1227512104 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e141629e0 con 0x7f3e1415f180 2024-04-25T11:18:16.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.068+0000 7f3e19000640 1 --2- 172.21.15.7:0/1227512104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e1415f180 0x7f3e14161c60 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f3e1415f750 tx=0x7f3e0402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a0301d6eef6506 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:16.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.068+0000 7f3e13400640 1 -- 172.21.15.7:0/1227512104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04036070 con 0x7f3e1415f180 2024-04-25T11:18:16.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.068+0000 7f3e13400640 1 -- 172.21.15.7:0/1227512104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e04031600 con 0x7f3e1415f180 2024-04-25T11:18:16.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.068+0000 7f3e13400640 1 -- 172.21.15.7:0/1227512104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0402ae60 con 0x7f3e1415f180 2024-04-25T11:18:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.069+0000 7f3e1ba00640 1 -- 172.21.15.7:0/1227512104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e1415f180 msgr2=0x7f3e14161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.069+0000 7f3e1ba00640 1 --2- 172.21.15.7:0/1227512104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e1415f180 0x7f3e14161c60 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f3e1415f750 tx=0x7f3e0402a680 comp rx=0 tx=0).stop 2024-04-25T11:18:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.069+0000 7f3e1ba00640 1 -- 172.21.15.7:0/1227512104 shutdown_connections 2024-04-25T11:18:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.069+0000 7f3e1ba00640 1 --2- 172.21.15.7:0/1227512104 >> v2:172.21.15.87:3300/0 conn(0x7f3e1406dc90 0x7f3e1406f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.069+0000 7f3e1ba00640 1 --2- 172.21.15.7:0/1227512104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e1415f180 0x7f3e14161c60 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.069+0000 7f3e1ba00640 1 -- 172.21.15.7:0/1227512104 >> 172.21.15.7:0/1227512104 conn(0x7f3e140732b0 msgr2=0x7f3e140736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.070+0000 7f3e1ba00640 1 -- 172.21.15.7:0/1227512104 shutdown_connections 2024-04-25T11:18:16.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.070+0000 7f3e1ba00640 1 -- 172.21.15.7:0/1227512104 wait complete. 2024-04-25T11:18:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.072+0000 7f3e1ba00640 1 Processor -- start 2024-04-25T11:18:16.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.072+0000 7f3e1ba00640 1 -- start start 2024-04-25T11:18:16.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.074+0000 7f3e1ba00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e1415f180 0x7f3e141503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:16.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.074+0000 7f3e1ba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e141621a0 0x7f3e14155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:16.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.074+0000 7f3e1ba00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e14152460 0x7f3e140afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:16.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.074+0000 7f3e1ba00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1415e670 con 0x7f3e1415f180 2024-04-25T11:18:16.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.074+0000 7f3e1ba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1415ec60 con 0x7f3e141621a0 2024-04-25T11:18:16.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.074+0000 7f3e1ba00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3e140d1310 con 0x7f3e14152460 2024-04-25T11:18:16.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.074+0000 7f3e19000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e1415f180 0x7f3e141503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:16.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.074+0000 7f3e19000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e1415f180 0x7f3e141503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40248/0 (socket says 172.21.15.7:40248) 2024-04-25T11:18:16.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.074+0000 7f3e19000640 1 -- 172.21.15.7:0/2186898462 learned_addr learned my addr 172.21.15.7:0/2186898462 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:16.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.075+0000 7f3e19a00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e14152460 0x7f3e140afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.075+0000 7f3e13e00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e141621a0 0x7f3e14155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.075+0000 7f3e19000640 1 -- 172.21.15.7:0/2186898462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e14152460 msgr2=0x7f3e140afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.075+0000 7f3e19000640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e14152460 0x7f3e140afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.075+0000 7f3e19000640 1 -- 172.21.15.7:0/2186898462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e141621a0 msgr2=0x7f3e14155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.075+0000 7f3e19000640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e141621a0 0x7f3e14155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.075+0000 7f3e19000640 1 -- 172.21.15.7:0/2186898462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e14155570 con 0x7f3e1415f180 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.075+0000 7f3e13e00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e141621a0 0x7f3e14155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.075+0000 7f3e19000640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e1415f180 0x7f3e141503a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009f70 tx=0x7f3e04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.076+0000 7f3e10c00640 1 -- 172.21.15.7:0/2186898462 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04041020 con 0x7f3e1415f180 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.076+0000 7f3e10c00640 1 -- 172.21.15.7:0/2186898462 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e04008790 con 0x7f3e1415f180 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.076+0000 7f3e1ba00640 1 -- 172.21.15.7:0/2186898462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e14155a80 con 0x7f3e1415f180 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.076+0000 7f3e10c00640 1 -- 172.21.15.7:0/2186898462 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04036040 con 0x7f3e1415f180 2024-04-25T11:18:16.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.076+0000 7f3e1ba00640 1 -- 172.21.15.7:0/2186898462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e14156040 con 0x7f3e1415f180 2024-04-25T11:18:16.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.077+0000 7f3e1ba00640 1 -- 172.21.15.7:0/2186898462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e14050740 con 0x7f3e1415f180 2024-04-25T11:18:16.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.078+0000 7f3e10c00640 1 -- 172.21.15.7:0/2186898462 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3e04044020 con 0x7f3e1415f180 2024-04-25T11:18:16.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.079+0000 7f3e10c00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3df40744b0 0x7f3df40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:16.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.080+0000 7f3e10c00640 1 -- 172.21.15.7:0/2186898462 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f3e040b6360 con 0x7f3e1415f180 2024-04-25T11:18:16.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.080+0000 7f3e13e00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3df40744b0 0x7f3df40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:16.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.081+0000 7f3e13e00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3df40744b0 0x7f3df40769c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3e140d0fd0 tx=0x7f3e00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:16.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.085+0000 7f3e10c00640 1 -- 172.21.15.7:0/2186898462 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e0404b020 con 0x7f3e1415f180 2024-04-25T11:18:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.330+0000 7f3e1ba00640 1 -- 172.21.15.7:0/2186898462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3e14074250 con 0x7f3e1415f180 2024-04-25T11:18:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.332+0000 7f3e10c00640 1 -- 172.21.15.7:0/2186898462 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f3e04080850 con 0x7f3e1415f180 2024-04-25T11:18:16.331 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:16.331 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:18:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.334+0000 7f3e1ba00640 1 -- 172.21.15.7:0/2186898462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3df40744b0 msgr2=0x7f3df40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.334+0000 7f3e1ba00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3df40744b0 0x7f3df40769c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3e140d0fd0 tx=0x7f3e00008040 comp rx=0 tx=0).stop 2024-04-25T11:18:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.334+0000 7f3e1ba00640 1 -- 172.21.15.7:0/2186898462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e1415f180 msgr2=0x7f3e141503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.334+0000 7f3e1ba00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e1415f180 0x7f3e141503a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009f70 tx=0x7f3e04004a40 comp rx=0 tx=0).stop 2024-04-25T11:18:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.335+0000 7f3e1ba00640 1 -- 172.21.15.7:0/2186898462 shutdown_connections 2024-04-25T11:18:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.335+0000 7f3e1ba00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3df40744b0 0x7f3df40769c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.335+0000 7f3e1ba00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e14152460 0x7f3e140afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.335+0000 7f3e1ba00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e141621a0 0x7f3e14155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.335+0000 7f3e1ba00640 1 --2- 172.21.15.7:0/2186898462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e1415f180 0x7f3e141503a0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.335+0000 7f3e1ba00640 1 -- 172.21.15.7:0/2186898462 >> 172.21.15.7:0/2186898462 conn(0x7f3e140732b0 msgr2=0x7f3e141580b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.335+0000 7f3e1ba00640 1 -- 172.21.15.7:0/2186898462 shutdown_connections 2024-04-25T11:18:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:16.335+0000 7f3e1ba00640 1 -- 172.21.15.7:0/2186898462 wait complete. 2024-04-25T11:18:16.365 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:18:16.365 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:18:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:21.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:18:21.630 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:21.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.663+0000 7fd401c00640 1 Processor -- start 2024-04-25T11:18:21.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.663+0000 7fd401c00640 1 -- start start 2024-04-25T11:18:21.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.666+0000 7fd401c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc160180 0x7fd3fc163c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.666+0000 7fd401c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3fc164bb0 con 0x7fd3fc0d06c0 2024-04-25T11:18:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.666+0000 7fd401c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3fc164310 con 0x7fd3fc0d2550 2024-04-25T11:18:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.666+0000 7fd401c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd3fc0d4510 con 0x7fd3fc160180 2024-04-25T11:18:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.666+0000 7fd3fa000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd3fc0d06c0 0x7fd3fc0b9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33818/0 (socket says 172.21.15.7:33818) 2024-04-25T11:18:21.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.666+0000 7fd3fa000640 1 -- 172.21.15.7:0/2571375217 learned_addr learned my addr 172.21.15.7:0/2571375217 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.667+0000 7fd3faa00640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc160180 0x7fd3fc163c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.667+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237380627 0 0) 0x7fd3fc164bb0 con 0x7fd3fc0d06c0 2024-04-25T11:18:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.667+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3e0003520 con 0x7fd3fc0d06c0 2024-04-25T11:18:21.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.667+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4173019500 0 0) 0x7fd3e0003520 con 0x7fd3fc0d06c0 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.667+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc160180 msgr2=0x7fd3fc163c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.667+0000 7fd3f9600640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc160180 0x7fd3fc163c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.667+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 >> v1:172.21.15.87:6789/0 conn(0x7fd3fc0d2550 legacy=0x7fd3fc0d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc1645f0 con 0x7fd3fc0d06c0 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fd3e8001690 con 0x7fd3fc0d06c0 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 >> v1:172.21.15.7:6789/0 conn(0x7fd3fc0d06c0 legacy=0x7fd3fc0b9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3f9600640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3e00039e0 0x7fd3e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3f9600640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc160180 0x7fd3e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3f9600640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3e0004d40 0x7fd3e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc0d4510 con 0x7fd3e00039e0 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e000bb80 con 0x7fd3e0004d40 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd3e000bd80 con 0x7fd3fc160180 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3faa00640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3e00039e0 0x7fd3e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:21.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3fa000640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc160180 0x7fd3e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3fb400640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3e0004d40 0x7fd3e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.668+0000 7fd3faa00640 1 -- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc160180 msgr2=0x7fd3e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.669+0000 7fd3faa00640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc160180 0x7fd3e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.669+0000 7fd3faa00640 1 -- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3e0004d40 msgr2=0x7fd3e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.669+0000 7fd3faa00640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3e0004d40 0x7fd3e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.669+0000 7fd3faa00640 1 -- 172.21.15.7:0/2571375217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e0003520 con 0x7fd3e00039e0 2024-04-25T11:18:21.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.669+0000 7fd3fb400640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3e0004d40 0x7fd3e00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.670+0000 7fd3faa00640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3e00039e0 0x7fd3e0003e70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009ac0 tx=0x7fd3e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad3dbe9eac815e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.670+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec036070 con 0x7fd3e00039e0 2024-04-25T11:18:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.670+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ec031400 con 0x7fd3e00039e0 2024-04-25T11:18:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.670+0000 7fd3f9600640 1 -- 172.21.15.7:0/2571375217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec009690 con 0x7fd3e00039e0 2024-04-25T11:18:21.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.671+0000 7fd401c00640 1 -- 172.21.15.7:0/2571375217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3e00039e0 msgr2=0x7fd3e0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.671+0000 7fd401c00640 1 --2- 172.21.15.7:0/2571375217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3e00039e0 0x7fd3e0003e70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009ac0 tx=0x7fd3e0005830 comp rx=0 tx=0).stop 2024-04-25T11:18:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.671+0000 7fd3fb400640 1 -- 172.21.15.7:0/2571375217 reap_dead start 2024-04-25T11:18:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.671+0000 7fd401c00640 1 -- 172.21.15.7:0/2571375217 shutdown_connections 2024-04-25T11:18:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.671+0000 7fd401c00640 1 -- 172.21.15.7:0/2571375217 >> 172.21.15.7:0/2571375217 conn(0x7fd3fc06c750 msgr2=0x7fd3fc06cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.671+0000 7fd401c00640 1 -- 172.21.15.7:0/2571375217 shutdown_connections 2024-04-25T11:18:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.671+0000 7fd401c00640 1 -- 172.21.15.7:0/2571375217 wait complete. 2024-04-25T11:18:21.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.673+0000 7fd401c00640 1 Processor -- start 2024-04-25T11:18:21.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.673+0000 7fd401c00640 1 -- start start 2024-04-25T11:18:21.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.675+0000 7fd401c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3e00039e0 0x7fd3fc0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.675+0000 7fd401c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc1f7730 0x7fd3fc1f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.675+0000 7fd401c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3fc15ad70 0x7fd3fc15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.675+0000 7fd401c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc163c80 con 0x7fd3fc1f7730 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.675+0000 7fd401c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc0d0ad0 con 0x7fd3fc15ad70 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.675+0000 7fd401c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd3fc163e00 con 0x7fd3e00039e0 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.675+0000 7fd3fa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc1f7730 0x7fd3fc1f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.675+0000 7fd3fa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc1f7730 0x7fd3fc1f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54986/0 (socket says 172.21.15.7:54986) 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.675+0000 7fd3fa000640 1 -- 172.21.15.7:0/3751941632 learned_addr learned my addr 172.21.15.7:0/3751941632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.676+0000 7fd3faa00640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3e00039e0 0x7fd3fc0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.676+0000 7fd3fb400640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3fc15ad70 0x7fd3fc15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.676+0000 7fd3fa000640 1 -- 172.21.15.7:0/3751941632 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3e00039e0 msgr2=0x7fd3fc0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:21.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.676+0000 7fd3fa000640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3e00039e0 0x7fd3fc0cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.676+0000 7fd3fa000640 1 -- 172.21.15.7:0/3751941632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3fc15ad70 msgr2=0x7fd3fc15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.676+0000 7fd3fa000640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3fc15ad70 0x7fd3fc15b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.676+0000 7fd3fa000640 1 -- 172.21.15.7:0/3751941632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc1f7530 con 0x7fd3fc1f7730 2024-04-25T11:18:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.676+0000 7fd3fa000640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc1f7730 0x7fd3fc1f6ff0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e780 tx=0x7fd3e800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.677+0000 7fd3f2a00640 1 -- 172.21.15.7:0/3751941632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800c8d0 con 0x7fd3fc1f7730 2024-04-25T11:18:21.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.677+0000 7fd3f2a00640 1 -- 172.21.15.7:0/3751941632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e8018070 con 0x7fd3fc1f7730 2024-04-25T11:18:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.677+0000 7fd3f2a00640 1 -- 172.21.15.7:0/3751941632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800ccd0 con 0x7fd3fc1f7730 2024-04-25T11:18:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.677+0000 7fd401c00640 1 -- 172.21.15.7:0/3751941632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3fc15b6c0 con 0x7fd3fc1f7730 2024-04-25T11:18:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.677+0000 7fd401c00640 1 -- 172.21.15.7:0/3751941632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3fc161a60 con 0x7fd3fc1f7730 2024-04-25T11:18:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.678+0000 7fd401c00640 1 -- 172.21.15.7:0/3751941632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3fc050740 con 0x7fd3fc1f7730 2024-04-25T11:18:21.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.679+0000 7fd3f2a00640 1 -- 172.21.15.7:0/3751941632 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd3e801d460 con 0x7fd3fc1f7730 2024-04-25T11:18:21.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.680+0000 7fd3f2a00640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3dc0744e0 0x7fd3dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:21.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.680+0000 7fd3f2a00640 1 -- 172.21.15.7:0/3751941632 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7fd3e8095270 con 0x7fd3fc1f7730 2024-04-25T11:18:21.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.681+0000 7fd3faa00640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3dc0744e0 0x7fd3dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:21.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.681+0000 7fd3faa00640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3dc0744e0 0x7fd3dc0769f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec033000 tx=0x7fd3ec008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:21.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.685+0000 7fd3f2a00640 1 -- 172.21.15.7:0/3751941632 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3fc15b6c0 con 0x7fd3fc1f7730 2024-04-25T11:18:21.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.935+0000 7fd401c00640 1 -- 172.21.15.7:0/3751941632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd3fc158f30 con 0x7fd3fc1f7730 2024-04-25T11:18:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.936+0000 7fd3f2a00640 1 -- 172.21.15.7:0/3751941632 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fd3fc1f7530 con 0x7fd3fc1f7730 2024-04-25T11:18:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:21.936 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:18:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.939+0000 7fd401c00640 1 -- 172.21.15.7:0/3751941632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3dc0744e0 msgr2=0x7fd3dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.939+0000 7fd401c00640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3dc0744e0 0x7fd3dc0769f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec033000 tx=0x7fd3ec008030 comp rx=0 tx=0).stop 2024-04-25T11:18:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.939+0000 7fd401c00640 1 -- 172.21.15.7:0/3751941632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc1f7730 msgr2=0x7fd3fc1f6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.939+0000 7fd401c00640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc1f7730 0x7fd3fc1f6ff0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e780 tx=0x7fd3e800ec40 comp rx=0 tx=0).stop 2024-04-25T11:18:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.940+0000 7fd401c00640 1 -- 172.21.15.7:0/3751941632 shutdown_connections 2024-04-25T11:18:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.940+0000 7fd401c00640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3dc0744e0 0x7fd3dc0769f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.940+0000 7fd401c00640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3fc15ad70 0x7fd3fc15b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.940+0000 7fd401c00640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc1f7730 0x7fd3fc1f6ff0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.940+0000 7fd401c00640 1 --2- 172.21.15.7:0/3751941632 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3e00039e0 0x7fd3fc0cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.940+0000 7fd401c00640 1 -- 172.21.15.7:0/3751941632 >> 172.21.15.7:0/3751941632 conn(0x7fd3fc06c750 msgr2=0x7fd3fc0b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.940+0000 7fd401c00640 1 -- 172.21.15.7:0/3751941632 shutdown_connections 2024-04-25T11:18:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:21.940+0000 7fd401c00640 1 -- 172.21.15.7:0/3751941632 wait complete. 2024-04-25T11:18:21.969 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:18:21.969 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:18:23.632 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:25.634 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:26.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:18:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.264+0000 7f3d77e00640 1 Processor -- start 2024-04-25T11:18:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.264+0000 7f3d77e00640 1 -- start start 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.266+0000 7f3d77e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d7808cad0 0x7f3d7809cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.266+0000 7f3d77e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d780b9040 0x7f3d7806e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.267+0000 7f3d77e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d7806dd90 con 0x7f3d7806e8d0 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.267+0000 7f3d77e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3d7806db50 con 0x7f3d780b9040 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.267+0000 7f3d77e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3d780d0da0 con 0x7f3d7808cad0 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.267+0000 7f3d6fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d780b9040 0x7f3d7806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.267+0000 7f3d6fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d780b9040 0x7f3d7806e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54990/0 (socket says 172.21.15.7:54990) 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.267+0000 7f3d6fe00640 1 -- 172.21.15.7:0/422744102 learned_addr learned my addr 172.21.15.7:0/422744102 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.267+0000 7f3d76a00640 1 --2- 172.21.15.7:0/422744102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d7808cad0 0x7f3d7809cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.267+0000 7f3d6fe00640 1 -- 172.21.15.7:0/422744102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d7808cad0 msgr2=0x7f3d7809cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.267+0000 7f3d6fe00640 1 --2- 172.21.15.7:0/422744102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d7808cad0 0x7f3d7809cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.267+0000 7f3d6fe00640 1 -- 172.21.15.7:0/422744102 >> v1:172.21.15.87:6789/0 conn(0x7f3d7806e8d0 legacy=0x7f3d780b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.267+0000 7f3d6fe00640 1 -- 172.21.15.7:0/422744102 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7806d870 con 0x7f3d780b9040 2024-04-25T11:18:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.269+0000 7f3d6fe00640 1 --2- 172.21.15.7:0/422744102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d780b9040 0x7f3d7806e390 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009870 tx=0x7f3d6002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248d9c4707e82a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.269+0000 7f3d76000640 1 -- 172.21.15.7:0/422744102 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60037070 con 0x7f3d780b9040 2024-04-25T11:18:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.269+0000 7f3d76000640 1 -- 172.21.15.7:0/422744102 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d60032440 con 0x7f3d780b9040 2024-04-25T11:18:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.269+0000 7f3d76000640 1 -- 172.21.15.7:0/422744102 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60032810 con 0x7f3d780b9040 2024-04-25T11:18:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.270+0000 7f3d77e00640 1 -- 172.21.15.7:0/422744102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d780b9040 msgr2=0x7f3d7806e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.270+0000 7f3d77e00640 1 --2- 172.21.15.7:0/422744102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d780b9040 0x7f3d7806e390 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009870 tx=0x7f3d6002a210 comp rx=0 tx=0).stop 2024-04-25T11:18:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.270+0000 7f3d77e00640 1 -- 172.21.15.7:0/422744102 shutdown_connections 2024-04-25T11:18:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.270+0000 7f3d77e00640 1 --2- 172.21.15.7:0/422744102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d780b9040 0x7f3d7806e390 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.270+0000 7f3d77e00640 1 --2- 172.21.15.7:0/422744102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d7808cad0 0x7f3d7809cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.270+0000 7f3d77e00640 1 -- 172.21.15.7:0/422744102 >> 172.21.15.7:0/422744102 conn(0x7f3d780d0280 msgr2=0x7f3d780d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.270+0000 7f3d77e00640 1 -- 172.21.15.7:0/422744102 shutdown_connections 2024-04-25T11:18:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.270+0000 7f3d77e00640 1 -- 172.21.15.7:0/422744102 wait complete. 2024-04-25T11:18:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.273+0000 7f3d77e00640 1 Processor -- start 2024-04-25T11:18:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.273+0000 7f3d77e00640 1 -- start start 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.275+0000 7f3d77e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7806e8d0 0x7f3d7815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.275+0000 7f3d77e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d7808cad0 0x7f3d7815ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.275+0000 7f3d77e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780b9040 0x7f3d780b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.275+0000 7f3d77e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d780d2ed0 con 0x7f3d7808cad0 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.275+0000 7f3d77e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3d780d1080 con 0x7f3d7806e8d0 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.275+0000 7f3d77e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3d780d3050 con 0x7f3d780b9040 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.275+0000 7f3d6fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d7808cad0 0x7f3d7815ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.275+0000 7f3d6fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d7808cad0 0x7f3d7815ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54996/0 (socket says 172.21.15.7:54996) 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.275+0000 7f3d6fe00640 1 -- 172.21.15.7:0/1903850225 learned_addr learned my addr 172.21.15.7:0/1903850225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.275+0000 7f3d76a00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7806e8d0 0x7f3d7815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.276+0000 7f3d77400640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780b9040 0x7f3d780b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.276+0000 7f3d6fe00640 1 -- 172.21.15.7:0/1903850225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780b9040 msgr2=0x7f3d780b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.276+0000 7f3d6fe00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780b9040 0x7f3d780b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.276+0000 7f3d6fe00640 1 -- 172.21.15.7:0/1903850225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7806e8d0 msgr2=0x7f3d7815ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.276+0000 7f3d6fe00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7806e8d0 0x7f3d7815ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.276+0000 7f3d6fe00640 1 -- 172.21.15.7:0/1903850225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7815f470 con 0x7f3d7808cad0 2024-04-25T11:18:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.276+0000 7f3d6fe00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d7808cad0 0x7f3d7815ef30 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f3d60000c00 tx=0x7f3d600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.276+0000 7f3d6ea00640 1 -- 172.21.15.7:0/1903850225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60041020 con 0x7f3d7808cad0 2024-04-25T11:18:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.276+0000 7f3d6ea00640 1 -- 172.21.15.7:0/1903850225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d600306e0 con 0x7f3d7808cad0 2024-04-25T11:18:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.276+0000 7f3d6ea00640 1 -- 172.21.15.7:0/1903850225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60037040 con 0x7f3d7808cad0 2024-04-25T11:18:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.277+0000 7f3d77e00640 1 -- 172.21.15.7:0/1903850225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d780b32a0 con 0x7f3d7808cad0 2024-04-25T11:18:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.277+0000 7f3d77e00640 1 -- 172.21.15.7:0/1903850225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d780b3860 con 0x7f3d7808cad0 2024-04-25T11:18:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.278+0000 7f3d77e00640 1 -- 172.21.15.7:0/1903850225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d780b1930 con 0x7f3d7808cad0 2024-04-25T11:18:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.279+0000 7f3d6ea00640 1 -- 172.21.15.7:0/1903850225 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3d60044020 con 0x7f3d7808cad0 2024-04-25T11:18:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.280+0000 7f3d6ea00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d540744b0 0x7f3d540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.280+0000 7f3d6ea00640 1 -- 172.21.15.7:0/1903850225 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f3d600b5ca0 con 0x7f3d7808cad0 2024-04-25T11:18:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.281+0000 7f3d76a00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d540744b0 0x7f3d540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.281+0000 7f3d76a00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d540744b0 0x7f3d540769c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3d780517c0 tx=0x7f3d68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.286+0000 7f3d6ea00640 1 -- 172.21.15.7:0/1903850225 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3d6004b030 con 0x7f3d7808cad0 2024-04-25T11:18:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.528+0000 7f3d77e00640 1 -- 172.21.15.7:0/1903850225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3d78050740 con 0x7f3d7808cad0 2024-04-25T11:18:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.529+0000 7f3d6ea00640 1 -- 172.21.15.7:0/1903850225 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f3d600802c0 con 0x7f3d7808cad0 2024-04-25T11:18:27.529 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:27.529 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:18:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.531+0000 7f3d77e00640 1 -- 172.21.15.7:0/1903850225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d540744b0 msgr2=0x7f3d540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.531+0000 7f3d77e00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d540744b0 0x7f3d540769c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3d780517c0 tx=0x7f3d68008040 comp rx=0 tx=0).stop 2024-04-25T11:18:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.531+0000 7f3d77e00640 1 -- 172.21.15.7:0/1903850225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d7808cad0 msgr2=0x7f3d7815ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.531+0000 7f3d77e00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d7808cad0 0x7f3d7815ef30 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f3d60000c00 tx=0x7f3d600046c0 comp rx=0 tx=0).stop 2024-04-25T11:18:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.532+0000 7f3d77e00640 1 -- 172.21.15.7:0/1903850225 shutdown_connections 2024-04-25T11:18:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.532+0000 7f3d77e00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3d540744b0 0x7f3d540769c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.532+0000 7f3d77e00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d780b9040 0x7f3d780b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.532+0000 7f3d77e00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d7808cad0 0x7f3d7815ef30 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.532+0000 7f3d77e00640 1 --2- 172.21.15.7:0/1903850225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d7806e8d0 0x7f3d7815ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.532+0000 7f3d77e00640 1 -- 172.21.15.7:0/1903850225 >> 172.21.15.7:0/1903850225 conn(0x7f3d780d0280 msgr2=0x7f3d78163040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.532+0000 7f3d77e00640 1 -- 172.21.15.7:0/1903850225 shutdown_connections 2024-04-25T11:18:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:27.532+0000 7f3d77e00640 1 -- 172.21.15.7:0/1903850225 wait complete. 2024-04-25T11:18:27.559 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:18:27.560 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:18:27.636 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:29.639 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:31.641 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:32.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:18:32.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.859+0000 7f3173200640 1 Processor -- start 2024-04-25T11:18:32.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.859+0000 7f3173200640 1 -- start start 2024-04-25T11:18:32.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.862+0000 7f3173200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f317406fb00 0x7f31740b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:32.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.862+0000 7f3173200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31740af2b0 con 0x7f317406d580 2024-04-25T11:18:32.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.862+0000 7f3173200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31740af550 con 0x7f317415ff10 2024-04-25T11:18:32.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.862+0000 7f3173200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f31740af830 con 0x7f317406fb00 2024-04-25T11:18:32.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.862+0000 7f3172800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f317406d580 0x7f317406d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39638/0 (socket says 172.21.15.7:39638) 2024-04-25T11:18:32.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.862+0000 7f3172800640 1 -- 172.21.15.7:0/2447679342 learned_addr learned my addr 172.21.15.7:0/2447679342 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:32.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.862+0000 7f3171e00640 1 --2- 172.21.15.7:0/2447679342 >> v2:172.21.15.7:3300/0 conn(0x7f317406fb00 0x7f31740b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:32.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.863+0000 7f3171e00640 1 -- 172.21.15.7:0/2447679342 >> v1:172.21.15.87:6789/0 conn(0x7f317415ff10 legacy=0x7f31740b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:32.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.863+0000 7f3171e00640 1 -- 172.21.15.7:0/2447679342 >> v1:172.21.15.7:6789/0 conn(0x7f317406d580 legacy=0x7f317406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:32.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.863+0000 7f3171e00640 1 -- 172.21.15.7:0/2447679342 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31740afb10 con 0x7f317406fb00 2024-04-25T11:18:32.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.863+0000 7f3170a00640 1 -- 172.21.15.7:0/2447679342 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3878443503 0 0) 0x7f31740af2b0 con 0x7f317406d580 2024-04-25T11:18:32.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.865+0000 7f3171e00640 1 --2- 172.21.15.7:0/2447679342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f317406fb00 0x7f31740b0030 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f31640097a0 tx=0x7f316402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a50f7624a343b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:32.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.865+0000 7f3170a00640 1 -- 172.21.15.7:0/2447679342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164037070 con 0x7f317406fb00 2024-04-25T11:18:32.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.865+0000 7f3170a00640 1 -- 172.21.15.7:0/2447679342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3164032440 con 0x7f317406fb00 2024-04-25T11:18:32.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.865+0000 7f3170a00640 1 -- 172.21.15.7:0/2447679342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164032810 con 0x7f317406fb00 2024-04-25T11:18:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.866+0000 7f3173200640 1 -- 172.21.15.7:0/2447679342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f317406fb00 msgr2=0x7f31740b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.866+0000 7f3173200640 1 --2- 172.21.15.7:0/2447679342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f317406fb00 0x7f31740b0030 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f31640097a0 tx=0x7f316402a230 comp rx=0 tx=0).stop 2024-04-25T11:18:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.866+0000 7f3173200640 1 -- 172.21.15.7:0/2447679342 shutdown_connections 2024-04-25T11:18:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.866+0000 7f3173200640 1 --2- 172.21.15.7:0/2447679342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f317406fb00 0x7f31740b0030 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:32.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.866+0000 7f3173200640 1 -- 172.21.15.7:0/2447679342 >> 172.21.15.7:0/2447679342 conn(0x7f3174073090 msgr2=0x7f31740734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:32.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.866+0000 7f3173200640 1 -- 172.21.15.7:0/2447679342 shutdown_connections 2024-04-25T11:18:32.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.866+0000 7f3173200640 1 -- 172.21.15.7:0/2447679342 wait complete. 2024-04-25T11:18:32.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.869+0000 7f3173200640 1 Processor -- start 2024-04-25T11:18:32.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.869+0000 7f3173200640 1 -- start start 2024-04-25T11:18:32.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.870+0000 7f3173200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f317406d580 0x7f317415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.870+0000 7f3173200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f317406fb00 0x7f31741470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.870+0000 7f3173200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317415ff10 0x7f31740b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.870+0000 7f3173200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31740b4690 con 0x7f317406d580 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.870+0000 7f3173200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3174160320 con 0x7f317406fb00 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.870+0000 7f3173200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f31740b4810 con 0x7f317415ff10 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.871+0000 7f3171e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f317406d580 0x7f317415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.871+0000 7f3171e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f317406d580 0x7f317415ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45434/0 (socket says 172.21.15.7:45434) 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.871+0000 7f3171e00640 1 -- 172.21.15.7:0/662294683 learned_addr learned my addr 172.21.15.7:0/662294683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.871+0000 7f3171400640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f317406fb00 0x7f31741470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.871+0000 7f3172800640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317415ff10 0x7f31740b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.871+0000 7f3171e00640 1 -- 172.21.15.7:0/662294683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317415ff10 msgr2=0x7f31740b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.871+0000 7f3171e00640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317415ff10 0x7f31740b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:32.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.871+0000 7f3171e00640 1 -- 172.21.15.7:0/662294683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f317406fb00 msgr2=0x7f31741470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:32.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.871+0000 7f3171e00640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f317406fb00 0x7f31741470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:32.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.871+0000 7f3171e00640 1 -- 172.21.15.7:0/662294683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31740b2980 con 0x7f317406d580 2024-04-25T11:18:32.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.872+0000 7f3171e00640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f317406d580 0x7f317415ca70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f31640098c0 tx=0x7f31640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:32.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.872+0000 7f316a000640 1 -- 172.21.15.7:0/662294683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164037040 con 0x7f317406d580 2024-04-25T11:18:32.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.872+0000 7f316a000640 1 -- 172.21.15.7:0/662294683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3164042070 con 0x7f317406d580 2024-04-25T11:18:32.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.872+0000 7f316a000640 1 -- 172.21.15.7:0/662294683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164030a70 con 0x7f317406d580 2024-04-25T11:18:32.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.872+0000 7f3173200640 1 -- 172.21.15.7:0/662294683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31740b2c60 con 0x7f317406d580 2024-04-25T11:18:32.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.872+0000 7f3173200640 1 -- 172.21.15.7:0/662294683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317415a0c0 con 0x7f317406d580 2024-04-25T11:18:32.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.873+0000 7f3173200640 1 -- 172.21.15.7:0/662294683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3174050740 con 0x7f317406d580 2024-04-25T11:18:32.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.875+0000 7f316a000640 1 -- 172.21.15.7:0/662294683 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3164030c10 con 0x7f317406d580 2024-04-25T11:18:32.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.876+0000 7f316a000640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31540744e0 0x7f31540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:32.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.876+0000 7f316a000640 1 -- 172.21.15.7:0/662294683 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f31640b5fe0 con 0x7f317406d580 2024-04-25T11:18:32.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.876+0000 7f3171400640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31540744e0 0x7f31540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:32.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.877+0000 7f3171400640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31540744e0 0x7f31540769f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f31741f9630 tx=0x7f3160009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:32.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:32.881+0000 7f316a000640 1 -- 172.21.15.7:0/662294683 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3164080510 con 0x7f317406d580 2024-04-25T11:18:33.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.128+0000 7f3173200640 1 -- 172.21.15.7:0/662294683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3174157800 con 0x7f317406d580 2024-04-25T11:18:33.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.128+0000 7f316a000640 1 -- 172.21.15.7:0/662294683 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7f316407fe10 con 0x7f317406d580 2024-04-25T11:18:33.128 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:33.128 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:18:33.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.131+0000 7f3173200640 1 -- 172.21.15.7:0/662294683 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31540744e0 msgr2=0x7f31540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:33.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.131+0000 7f3173200640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31540744e0 0x7f31540769f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f31741f9630 tx=0x7f3160009040 comp rx=0 tx=0).stop 2024-04-25T11:18:33.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.131+0000 7f3173200640 1 -- 172.21.15.7:0/662294683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f317406d580 msgr2=0x7f317415ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:33.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.131+0000 7f3173200640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f317406d580 0x7f317415ca70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f31640098c0 tx=0x7f31640046c0 comp rx=0 tx=0).stop 2024-04-25T11:18:33.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.131+0000 7f3173200640 1 -- 172.21.15.7:0/662294683 shutdown_connections 2024-04-25T11:18:33.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.131+0000 7f3173200640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f31540744e0 0x7f31540769f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:33.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.131+0000 7f3173200640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317415ff10 0x7f31740b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:33.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.131+0000 7f3173200640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f317406fb00 0x7f31741470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:33.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.131+0000 7f3173200640 1 --2- 172.21.15.7:0/662294683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f317406d580 0x7f317415ca70 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:33.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.131+0000 7f3173200640 1 -- 172.21.15.7:0/662294683 >> 172.21.15.7:0/662294683 conn(0x7f3174073090 msgr2=0x7f3174158220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:33.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.132+0000 7f3173200640 1 -- 172.21.15.7:0/662294683 shutdown_connections 2024-04-25T11:18:33.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:33.132+0000 7f3173200640 1 -- 172.21.15.7:0/662294683 wait complete. 2024-04-25T11:18:33.161 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:18:33.161 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:18:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:35.645 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:37.647 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:38.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:18:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.448+0000 7fa5af200640 1 Processor -- start 2024-04-25T11:18:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.448+0000 7fa5af200640 1 -- start start 2024-04-25T11:18:38.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.450+0000 7fa5af200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa5b00b9040 0x7fa5b006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:38.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.450+0000 7fa5af200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5b006e8d0 0x7fa5b00b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.450+0000 7fa5af200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5b006dd90 con 0x7fa5b008cad0 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.450+0000 7fa5af200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa5b006db50 con 0x7fa5b006e8d0 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.450+0000 7fa5af200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa5b00d0da0 con 0x7fa5b00b9040 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.450+0000 7fa5ade00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa5b008cad0 0x7fa5b009cbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39652/0 (socket says 172.21.15.7:39652) 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.450+0000 7fa5ade00640 1 -- 172.21.15.7:0/4167831166 learned_addr learned my addr 172.21.15.7:0/4167831166 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.450+0000 7fa5ae800640 1 --2- 172.21.15.7:0/4167831166 >> v2:172.21.15.7:3300/0 conn(0x7fa5b006e8d0 0x7fa5b00b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.450+0000 7fa5a5400640 1 --2- 172.21.15.7:0/4167831166 >> v2:172.21.15.87:3300/0 conn(0x7fa5b00b9040 0x7fa5b006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.451+0000 7fa5ad400640 1 -- 172.21.15.7:0/4167831166 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1412391566 0 0) 0x7fa5b006dd90 con 0x7fa5b008cad0 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.451+0000 7fa5ad400640 1 -- 172.21.15.7:0/4167831166 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa594003520 con 0x7fa5b008cad0 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.451+0000 7fa5ae800640 1 -- 172.21.15.7:0/4167831166 >> v2:172.21.15.87:3300/0 conn(0x7fa5b00b9040 msgr2=0x7fa5b006e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.451+0000 7fa5ae800640 1 --2- 172.21.15.7:0/4167831166 >> v2:172.21.15.87:3300/0 conn(0x7fa5b00b9040 0x7fa5b006e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:38.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.451+0000 7fa5ae800640 1 -- 172.21.15.7:0/4167831166 >> v1:172.21.15.7:6789/0 conn(0x7fa5b008cad0 legacy=0x7fa5b009cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.451+0000 7fa5ae800640 1 -- 172.21.15.7:0/4167831166 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b006d870 con 0x7fa5b006e8d0 2024-04-25T11:18:38.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.452+0000 7fa5ae800640 1 --2- 172.21.15.7:0/4167831166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b006e8d0 0x7fa5b00b7060 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00690a0 tx=0x7fa5a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51931eee5d5c56b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.452+0000 7fa5ad400640 1 -- 172.21.15.7:0/4167831166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8037070 con 0x7fa5b006e8d0 2024-04-25T11:18:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.452+0000 7fa5ad400640 1 -- 172.21.15.7:0/4167831166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a8032440 con 0x7fa5b006e8d0 2024-04-25T11:18:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.452+0000 7fa5ad400640 1 -- 172.21.15.7:0/4167831166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80327f0 con 0x7fa5b006e8d0 2024-04-25T11:18:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.453+0000 7fa5af200640 1 -- 172.21.15.7:0/4167831166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b006e8d0 msgr2=0x7fa5b00b7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:38.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.453+0000 7fa5af200640 1 --2- 172.21.15.7:0/4167831166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b006e8d0 0x7fa5b00b7060 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00690a0 tx=0x7fa5a802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:18:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.453+0000 7fa5af200640 1 -- 172.21.15.7:0/4167831166 shutdown_connections 2024-04-25T11:18:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.453+0000 7fa5af200640 1 --2- 172.21.15.7:0/4167831166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b006e8d0 0x7fa5b00b7060 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.453+0000 7fa5af200640 1 --2- 172.21.15.7:0/4167831166 >> v2:172.21.15.87:3300/0 conn(0x7fa5b00b9040 0x7fa5b006e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.453+0000 7fa5af200640 1 -- 172.21.15.7:0/4167831166 >> 172.21.15.7:0/4167831166 conn(0x7fa5b00d0280 msgr2=0x7fa5b00d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.453+0000 7fa5af200640 1 -- 172.21.15.7:0/4167831166 shutdown_connections 2024-04-25T11:18:38.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.453+0000 7fa5af200640 1 -- 172.21.15.7:0/4167831166 wait complete. 2024-04-25T11:18:38.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.456+0000 7fa5af200640 1 Processor -- start 2024-04-25T11:18:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.456+0000 7fa5af200640 1 -- start start 2024-04-25T11:18:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.457+0000 7fa5af200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b006e8d0 0x7fa5b015aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.457+0000 7fa5af200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b008cad0 0x7fa5b015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.457+0000 7fa5af200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5b00b9040 0x7fa5b015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.457+0000 7fa5af200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b00b75a0 con 0x7fa5b008cad0 2024-04-25T11:18:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.457+0000 7fa5af200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b00d1080 con 0x7fa5b006e8d0 2024-04-25T11:18:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.457+0000 7fa5af200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b00b7720 con 0x7fa5b00b9040 2024-04-25T11:18:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.457+0000 7fa5a5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b008cad0 0x7fa5b015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.458+0000 7fa5a5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b008cad0 0x7fa5b015cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45446/0 (socket says 172.21.15.7:45446) 2024-04-25T11:18:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.458+0000 7fa5a5400640 1 -- 172.21.15.7:0/3711436872 learned_addr learned my addr 172.21.15.7:0/3711436872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.458+0000 7fa5ade00640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b006e8d0 0x7fa5b015aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.458+0000 7fa5ae800640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5b00b9040 0x7fa5b015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.458+0000 7fa5a5400640 1 -- 172.21.15.7:0/3711436872 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5b00b9040 msgr2=0x7fa5b015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.458+0000 7fa5a5400640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5b00b9040 0x7fa5b015b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.458+0000 7fa5a5400640 1 -- 172.21.15.7:0/3711436872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b006e8d0 msgr2=0x7fa5b015aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.458+0000 7fa5a5400640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b006e8d0 0x7fa5b015aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.458+0000 7fa5a5400640 1 -- 172.21.15.7:0/3711436872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b015cff0 con 0x7fa5b008cad0 2024-04-25T11:18:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.458+0000 7fa5ade00640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b006e8d0 0x7fa5b015aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.458+0000 7fa5a5400640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b008cad0 0x7fa5b015cab0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa59800d5f0 tx=0x7fa59800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.459+0000 7fa5a6a00640 1 -- 172.21.15.7:0/3711436872 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5980125f0 con 0x7fa5b008cad0 2024-04-25T11:18:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.459+0000 7fa5a6a00640 1 -- 172.21.15.7:0/3711436872 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa598019070 con 0x7fa5b008cad0 2024-04-25T11:18:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.459+0000 7fa5a6a00640 1 -- 172.21.15.7:0/3711436872 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5980129d0 con 0x7fa5b008cad0 2024-04-25T11:18:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.459+0000 7fa5af200640 1 -- 172.21.15.7:0/3711436872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b015d500 con 0x7fa5b008cad0 2024-04-25T11:18:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.459+0000 7fa5af200640 1 -- 172.21.15.7:0/3711436872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b015dac0 con 0x7fa5b008cad0 2024-04-25T11:18:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.460+0000 7fa5af200640 1 -- 172.21.15.7:0/3711436872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b015d270 con 0x7fa5b008cad0 2024-04-25T11:18:38.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.461+0000 7fa5a6a00640 1 -- 172.21.15.7:0/3711436872 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa598012b70 con 0x7fa5b008cad0 2024-04-25T11:18:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.462+0000 7fa5a6a00640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa58c0744b0 0x7fa58c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.463+0000 7fa5a6a00640 1 -- 172.21.15.7:0/3711436872 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7fa598095880 con 0x7fa5b008cad0 2024-04-25T11:18:38.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.463+0000 7fa5ade00640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa58c0744b0 0x7fa58c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:38.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.464+0000 7fa5ade00640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa58c0744b0 0x7fa58c0769c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00016e0 tx=0x7fa5a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.467+0000 7fa5a6a00640 1 -- 172.21.15.7:0/3711436872 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa59805fe00 con 0x7fa5b008cad0 2024-04-25T11:18:38.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.711+0000 7fa5af200640 1 -- 172.21.15.7:0/3711436872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa5b0050740 con 0x7fa5b008cad0 2024-04-25T11:18:38.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.713+0000 7fa5a6a00640 1 -- 172.21.15.7:0/3711436872 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+174 (secure 0 0 0) 0x7fa59805f700 con 0x7fa5b008cad0 2024-04-25T11:18:38.713 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:38.713 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:18:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.715+0000 7fa5af200640 1 -- 172.21.15.7:0/3711436872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa58c0744b0 msgr2=0x7fa58c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.715+0000 7fa5af200640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa58c0744b0 0x7fa58c0769c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00016e0 tx=0x7fa5a0009040 comp rx=0 tx=0).stop 2024-04-25T11:18:38.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.715+0000 7fa5af200640 1 -- 172.21.15.7:0/3711436872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b008cad0 msgr2=0x7fa5b015cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.715+0000 7fa5af200640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b008cad0 0x7fa5b015cab0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa59800d5f0 tx=0x7fa59800dab0 comp rx=0 tx=0).stop 2024-04-25T11:18:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.716+0000 7fa5af200640 1 -- 172.21.15.7:0/3711436872 shutdown_connections 2024-04-25T11:18:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.716+0000 7fa5af200640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa58c0744b0 0x7fa58c0769c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.716+0000 7fa5af200640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5b00b9040 0x7fa5b015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.716+0000 7fa5af200640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b008cad0 0x7fa5b015cab0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.716+0000 7fa5af200640 1 --2- 172.21.15.7:0/3711436872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b006e8d0 0x7fa5b015aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.716+0000 7fa5af200640 1 -- 172.21.15.7:0/3711436872 >> 172.21.15.7:0/3711436872 conn(0x7fa5b00d0280 msgr2=0x7fa5b00686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.716+0000 7fa5af200640 1 -- 172.21.15.7:0/3711436872 shutdown_connections 2024-04-25T11:18:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38.716+0000 7fa5af200640 1 -- 172.21.15.7:0/3711436872 wait complete. 2024-04-25T11:18:38.744 DEBUG:tasks.ceph_test_case:Not found expected summary strings yet (['1 MDSs report oversized cache']) 2024-04-25T11:18:38.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:38 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-25T11:18:39.649 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:18:39.650 INFO:teuthology.orchestra.run.smithi007.stderr:stdin closed, goodbye! 2024-04-25T11:18:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:18:39.702 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:18:39.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:18:39.858 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:18:39.969 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:18:39.969 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:18:39.969 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 400\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi007') 2024-04-25T11:18:39.970 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:18:39.970 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:18:40.032 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:18:40.032 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:18:40.086 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:18:40.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.372+0000 7fa95d400640 1 Processor -- start 2024-04-25T11:18:40.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.372+0000 7fa95d400640 1 -- start start 2024-04-25T11:18:40.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.374+0000 7fa95d400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa958160f80 0x7fa95809b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.374+0000 7fa95d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa95809ba80 0x7fa958169fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.374+0000 7fa95d400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95809be90 con 0x7fa9580b1bf0 2024-04-25T11:18:40.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.374+0000 7fa95d400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa9580b07e0 con 0x7fa95809ba80 2024-04-25T11:18:40.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.374+0000 7fa95d400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa958163af0 con 0x7fa958160f80 2024-04-25T11:18:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.375+0000 7fa95ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa95809ba80 0x7fa958169fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.375+0000 7fa957e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa9580b1bf0 0x7fa9580b26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47770/0 (socket says 172.21.15.7:47770) 2024-04-25T11:18:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.375+0000 7fa957e00640 1 -- 172.21.15.7:0/1762270352 learned_addr learned my addr 172.21.15.7:0/1762270352 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.375+0000 7fa957400640 1 --2- 172.21.15.7:0/1762270352 >> v2:172.21.15.87:3300/0 conn(0x7fa958160f80 0x7fa95809b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.375+0000 7fa95ca00640 1 -- 172.21.15.7:0/1762270352 >> v2:172.21.15.87:3300/0 conn(0x7fa958160f80 msgr2=0x7fa95809b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.375+0000 7fa95ca00640 1 --2- 172.21.15.7:0/1762270352 >> v2:172.21.15.87:3300/0 conn(0x7fa958160f80 0x7fa95809b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.375+0000 7fa95ca00640 1 -- 172.21.15.7:0/1762270352 >> v1:172.21.15.7:6789/0 conn(0x7fa9580b1bf0 legacy=0x7fa9580b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.375+0000 7fa95ca00640 1 -- 172.21.15.7:0/1762270352 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9580b2db0 con 0x7fa95809ba80 2024-04-25T11:18:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.375+0000 7fa956a00640 1 -- 172.21.15.7:0/1762270352 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1971682846 0 0) 0x7fa948001490 con 0x7fa9580b1bf0 2024-04-25T11:18:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.376+0000 7fa95ca00640 1 --2- 172.21.15.7:0/1762270352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95809ba80 0x7fa958169fb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa95809c400 tx=0x7fa95002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06fa3eba129a6be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.377+0000 7fa956a00640 1 -- 172.21.15.7:0/1762270352 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950037070 con 0x7fa95809ba80 2024-04-25T11:18:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.377+0000 7fa956a00640 1 -- 172.21.15.7:0/1762270352 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa950032440 con 0x7fa95809ba80 2024-04-25T11:18:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.377+0000 7fa956a00640 1 -- 172.21.15.7:0/1762270352 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95002ae60 con 0x7fa95809ba80 2024-04-25T11:18:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.377+0000 7fa95d400640 1 -- 172.21.15.7:0/1762270352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95809ba80 msgr2=0x7fa958169fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.377+0000 7fa95d400640 1 --2- 172.21.15.7:0/1762270352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95809ba80 0x7fa958169fb0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa95809c400 tx=0x7fa95002a110 comp rx=0 tx=0).stop 2024-04-25T11:18:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.377+0000 7fa95d400640 1 -- 172.21.15.7:0/1762270352 shutdown_connections 2024-04-25T11:18:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.377+0000 7fa95d400640 1 --2- 172.21.15.7:0/1762270352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa95809ba80 0x7fa958169fb0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.377+0000 7fa95d400640 1 --2- 172.21.15.7:0/1762270352 >> v2:172.21.15.87:3300/0 conn(0x7fa958160f80 0x7fa95809b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.377+0000 7fa95d400640 1 -- 172.21.15.7:0/1762270352 >> 172.21.15.7:0/1762270352 conn(0x7fa958074460 msgr2=0x7fa9580b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.378+0000 7fa95d400640 1 -- 172.21.15.7:0/1762270352 shutdown_connections 2024-04-25T11:18:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.378+0000 7fa95d400640 1 -- 172.21.15.7:0/1762270352 wait complete. 2024-04-25T11:18:40.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.380+0000 7fa95d400640 1 Processor -- start 2024-04-25T11:18:40.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.380+0000 7fa95d400640 1 -- start start 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.381+0000 7fa95d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa95809ba80 0x7fa95814e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa95d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9580b1bf0 0x7fa95806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa95d400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa958160f80 0x7fa95815ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa95d400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9580b0220 con 0x7fa9580b1bf0 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa95d400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa9580b21f0 con 0x7fa95809ba80 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa95d400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa9580b0500 con 0x7fa958160f80 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa957400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9580b1bf0 0x7fa95806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa957400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9580b1bf0 0x7fa95806e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50822/0 (socket says 172.21.15.7:50822) 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa957400640 1 -- 172.21.15.7:0/1779141603 learned_addr learned my addr 172.21.15.7:0/1779141603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa957e00640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa95809ba80 0x7fa95814e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa95ca00640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa958160f80 0x7fa95815ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa957400640 1 -- 172.21.15.7:0/1779141603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa958160f80 msgr2=0x7fa95815ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa957400640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa958160f80 0x7fa95815ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa957400640 1 -- 172.21.15.7:0/1779141603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa95809ba80 msgr2=0x7fa95814e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa957400640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa95809ba80 0x7fa95814e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa957400640 1 -- 172.21.15.7:0/1779141603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa958163540 con 0x7fa9580b1bf0 2024-04-25T11:18:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.382+0000 7fa957e00640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa95809ba80 0x7fa95814e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.383+0000 7fa957400640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9580b1bf0 0x7fa95806e5e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa94000d5f0 tx=0x7fa94000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.383+0000 7fa94fe00640 1 -- 172.21.15.7:0/1779141603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9400125f0 con 0x7fa9580b1bf0 2024-04-25T11:18:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.383+0000 7fa94fe00640 1 -- 172.21.15.7:0/1779141603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa940019070 con 0x7fa9580b1bf0 2024-04-25T11:18:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.383+0000 7fa95d400640 1 -- 172.21.15.7:0/1779141603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95806cf80 con 0x7fa9580b1bf0 2024-04-25T11:18:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.383+0000 7fa94fe00640 1 -- 172.21.15.7:0/1779141603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9400129d0 con 0x7fa9580b1bf0 2024-04-25T11:18:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.383+0000 7fa95d400640 1 -- 172.21.15.7:0/1779141603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95806d540 con 0x7fa9580b1bf0 2024-04-25T11:18:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.385+0000 7fa95d400640 1 -- 172.21.15.7:0/1779141603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa958050740 con 0x7fa9580b1bf0 2024-04-25T11:18:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.386+0000 7fa94fe00640 1 -- 172.21.15.7:0/1779141603 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa940012b70 con 0x7fa9580b1bf0 2024-04-25T11:18:40.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.387+0000 7fa94fe00640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9340744b0 0x7fa9340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.388+0000 7fa94fe00640 1 -- 172.21.15.7:0/1779141603 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7fa940095880 con 0x7fa9580b1bf0 2024-04-25T11:18:40.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.388+0000 7fa957e00640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9340744b0 0x7fa9340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.388+0000 7fa957e00640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9340744b0 0x7fa9340769c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa948007fd0 tx=0x7fa948009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:40.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.392+0000 7fa94fe00640 1 -- 172.21.15.7:0/1779141603 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa94005fe00 con 0x7fa9580b1bf0 2024-04-25T11:18:40.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.600+0000 7fa95d400640 1 -- 172.21.15.7:0/1779141603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9580b2840 con 0x7fa9580b1bf0 2024-04-25T11:18:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.602+0000 7fa94fe00640 1 -- 172.21.15.7:0/1779141603 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16983 (secure 0 0 0) 0x7fa94005f700 con 0x7fa9580b1bf0 2024-04-25T11:18:40.602 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:40.602 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:17:04.183733+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:59.999922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T11:16:00.796013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":5}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:18:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.605+0000 7fa95d400640 1 -- 172.21.15.7:0/1779141603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9340744b0 msgr2=0x7fa9340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.605+0000 7fa95d400640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9340744b0 0x7fa9340769c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa948007fd0 tx=0x7fa948009040 comp rx=0 tx=0).stop 2024-04-25T11:18:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.605+0000 7fa95d400640 1 -- 172.21.15.7:0/1779141603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9580b1bf0 msgr2=0x7fa95806e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.605+0000 7fa95d400640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9580b1bf0 0x7fa95806e5e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa94000d5f0 tx=0x7fa94000dab0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.605+0000 7fa95d400640 1 -- 172.21.15.7:0/1779141603 shutdown_connections 2024-04-25T11:18:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.605+0000 7fa95d400640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9340744b0 0x7fa9340769c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.605+0000 7fa95d400640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa958160f80 0x7fa95815ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.605+0000 7fa95d400640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9580b1bf0 0x7fa95806e5e0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.605+0000 7fa95d400640 1 --2- 172.21.15.7:0/1779141603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa95809ba80 0x7fa95814e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.606+0000 7fa95d400640 1 -- 172.21.15.7:0/1779141603 >> 172.21.15.7:0/1779141603 conn(0x7fa958074460 msgr2=0x7fa9580b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:40.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.606+0000 7fa95d400640 1 -- 172.21.15.7:0/1779141603 shutdown_connections 2024-04-25T11:18:40.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.606+0000 7fa95d400640 1 -- 172.21.15.7:0/1779141603 wait complete. 2024-04-25T11:18:40.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:15:59.999922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:16:00.796013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:18:40.635 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:18:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.924+0000 7f3c2da00640 1 Processor -- start 2024-04-25T11:18:40.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.924+0000 7f3c2da00640 1 -- start start 2024-04-25T11:18:40.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.927+0000 7f3c2da00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c2815aaa0 0x7f3c280affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.927+0000 7f3c2da00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c280d0ee0 0x7f3c280b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.927+0000 7f3c2da00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3c280b4e50 0x7f3c280b5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.927+0000 7f3c2da00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3c280af290 con 0x7f3c2815aaa0 2024-04-25T11:18:40.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.927+0000 7f3c2da00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3c2806f5b0 con 0x7f3c280b4e50 2024-04-25T11:18:40.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.927+0000 7f3c2da00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3c280d14a0 con 0x7f3c280d0ee0 2024-04-25T11:18:40.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.928+0000 7f3c26a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c2815aaa0 0x7f3c280affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.928+0000 7f3c26a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c2815aaa0 0x7f3c280affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50826/0 (socket says 172.21.15.7:50826) 2024-04-25T11:18:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.928+0000 7f3c26a00640 1 -- 172.21.15.7:0/2273594819 learned_addr learned my addr 172.21.15.7:0/2273594819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.928+0000 7f3c1fe00640 1 --2- 172.21.15.7:0/2273594819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c280d0ee0 0x7f3c280b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.928+0000 7f3c27400640 1 --2- 172.21.15.7:0/2273594819 >> v2:172.21.15.87:3300/0 conn(0x7f3c280b4e50 0x7f3c280b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.928+0000 7f3c26a00640 1 -- 172.21.15.7:0/2273594819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c280d0ee0 msgr2=0x7f3c280b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.928+0000 7f3c26a00640 1 --2- 172.21.15.7:0/2273594819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c280d0ee0 0x7f3c280b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.928+0000 7f3c26a00640 1 -- 172.21.15.7:0/2273594819 >> v2:172.21.15.87:3300/0 conn(0x7f3c280b4e50 msgr2=0x7f3c280b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.928+0000 7f3c26a00640 1 --2- 172.21.15.7:0/2273594819 >> v2:172.21.15.87:3300/0 conn(0x7f3c280b4e50 0x7f3c280b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.928+0000 7f3c26a00640 1 -- 172.21.15.7:0/2273594819 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c280af4e0 con 0x7f3c2815aaa0 2024-04-25T11:18:40.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.930+0000 7f3c26a00640 1 --2- 172.21.15.7:0/2273594819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2815aaa0 0x7f3c280affc0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3c280b4670 tx=0x7f3c1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2ab56a7f2f556c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.930+0000 7f3c26000640 1 -- 172.21.15.7:0/2273594819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18037070 con 0x7f3c2815aaa0 2024-04-25T11:18:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.930+0000 7f3c26000640 1 -- 172.21.15.7:0/2273594819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c18032440 con 0x7f3c2815aaa0 2024-04-25T11:18:40.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.930+0000 7f3c26000640 1 -- 172.21.15.7:0/2273594819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18032810 con 0x7f3c2815aaa0 2024-04-25T11:18:40.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.931+0000 7f3c2da00640 1 -- 172.21.15.7:0/2273594819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2815aaa0 msgr2=0x7f3c280affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.931+0000 7f3c2da00640 1 --2- 172.21.15.7:0/2273594819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2815aaa0 0x7f3c280affc0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f3c280b4670 tx=0x7f3c1802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.932+0000 7f3c2da00640 1 -- 172.21.15.7:0/2273594819 shutdown_connections 2024-04-25T11:18:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.932+0000 7f3c2da00640 1 --2- 172.21.15.7:0/2273594819 >> v2:172.21.15.87:3300/0 conn(0x7f3c280b4e50 0x7f3c280b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.932+0000 7f3c2da00640 1 --2- 172.21.15.7:0/2273594819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c280d0ee0 0x7f3c280b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.932+0000 7f3c2da00640 1 --2- 172.21.15.7:0/2273594819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c2815aaa0 0x7f3c280affc0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.932+0000 7f3c2da00640 1 -- 172.21.15.7:0/2273594819 >> 172.21.15.7:0/2273594819 conn(0x7f3c280742c0 msgr2=0x7f3c2806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:40.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.932+0000 7f3c2da00640 1 -- 172.21.15.7:0/2273594819 shutdown_connections 2024-04-25T11:18:40.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.932+0000 7f3c2da00640 1 -- 172.21.15.7:0/2273594819 wait complete. 2024-04-25T11:18:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.934+0000 7f3c2da00640 1 Processor -- start 2024-04-25T11:18:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.934+0000 7f3c2da00640 1 -- start start 2024-04-25T11:18:40.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.936+0000 7f3c2da00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c280b4e50 0x7f3c28167f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.936+0000 7f3c2da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c280d0ee0 0x7f3c28158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.936+0000 7f3c2da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2815aaa0 0x7f3c281685b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.936+0000 7f3c2da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2815d3a0 con 0x7f3c280d0ee0 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.936+0000 7f3c2da00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3c280afaa0 con 0x7f3c2815aaa0 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.936+0000 7f3c2da00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2815d520 con 0x7f3c280b4e50 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.936+0000 7f3c1fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c280d0ee0 0x7f3c28158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.936+0000 7f3c1fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c280d0ee0 0x7f3c28158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50828/0 (socket says 172.21.15.7:50828) 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.936+0000 7f3c1fe00640 1 -- 172.21.15.7:0/1040750661 learned_addr learned my addr 172.21.15.7:0/1040750661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.937+0000 7f3c26a00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c280b4e50 0x7f3c28167f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.937+0000 7f3c27400640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2815aaa0 0x7f3c281685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.937+0000 7f3c1fe00640 1 -- 172.21.15.7:0/1040750661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c280b4e50 msgr2=0x7f3c28167f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.937+0000 7f3c1fe00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c280b4e50 0x7f3c28167f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.937+0000 7f3c1fe00640 1 -- 172.21.15.7:0/1040750661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2815aaa0 msgr2=0x7f3c281685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:40.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.937+0000 7f3c1fe00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2815aaa0 0x7f3c281685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:40.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.937+0000 7f3c1fe00640 1 -- 172.21.15.7:0/1040750661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2814e0e0 con 0x7f3c280d0ee0 2024-04-25T11:18:40.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.937+0000 7f3c27400640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2815aaa0 0x7f3c281685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:40.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.937+0000 7f3c1fe00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c280d0ee0 0x7f3c28158a60 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000d620 tx=0x7f3c1000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:40.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.937+0000 7f3c1ea00640 1 -- 172.21.15.7:0/1040750661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000b620 con 0x7f3c280d0ee0 2024-04-25T11:18:40.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.938+0000 7f3c1ea00640 1 -- 172.21.15.7:0/1040750661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c10019070 con 0x7f3c280d0ee0 2024-04-25T11:18:40.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.938+0000 7f3c1ea00640 1 -- 172.21.15.7:0/1040750661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000ba00 con 0x7f3c280d0ee0 2024-04-25T11:18:40.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.938+0000 7f3c2da00640 1 -- 172.21.15.7:0/1040750661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2814e3c0 con 0x7f3c280d0ee0 2024-04-25T11:18:40.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.938+0000 7f3c2da00640 1 -- 172.21.15.7:0/1040750661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2814e980 con 0x7f3c280d0ee0 2024-04-25T11:18:40.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.939+0000 7f3c2da00640 1 -- 172.21.15.7:0/1040750661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c28050740 con 0x7f3c280d0ee0 2024-04-25T11:18:40.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.940+0000 7f3c1ea00640 1 -- 172.21.15.7:0/1040750661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3c1000bba0 con 0x7f3c280d0ee0 2024-04-25T11:18:40.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.941+0000 7f3c1ea00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c040744e0 0x7f3c040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:40.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.941+0000 7f3c1ea00640 1 -- 172.21.15.7:0/1040750661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f3c100958f0 con 0x7f3c280d0ee0 2024-04-25T11:18:40.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.942+0000 7f3c26a00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c040744e0 0x7f3c040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:40.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.942+0000 7f3c26a00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c040744e0 0x7f3c040769f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3c18034000 tx=0x7f3c1802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:40.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:40.946+0000 7f3c1ea00640 1 -- 172.21.15.7:0/1040750661 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3c1005fde0 con 0x7f3c280d0ee0 2024-04-25T11:18:41.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.152+0000 7f3c2da00640 1 -- 172.21.15.7:0/1040750661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c280b1580 con 0x7f3c280d0ee0 2024-04-25T11:18:41.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.153+0000 7f3c1ea00640 1 -- 172.21.15.7:0/1040750661 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f3c1005f6e0 con 0x7f3c280d0ee0 2024-04-25T11:18:41.152 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:18:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 -- 172.21.15.7:0/1040750661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c040744e0 msgr2=0x7f3c040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c040744e0 0x7f3c040769f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3c18034000 tx=0x7f3c1802e000 comp rx=0 tx=0).stop 2024-04-25T11:18:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 -- 172.21.15.7:0/1040750661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c280d0ee0 msgr2=0x7f3c28158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c280d0ee0 0x7f3c28158a60 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000d620 tx=0x7f3c1000dae0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 -- 172.21.15.7:0/1040750661 shutdown_connections 2024-04-25T11:18:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3c040744e0 0x7f3c040769f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c2815aaa0 0x7f3c281685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c280d0ee0 0x7f3c28158a60 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 --2- 172.21.15.7:0/1040750661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c280b4e50 0x7f3c28167f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 -- 172.21.15.7:0/1040750661 >> 172.21.15.7:0/1040750661 conn(0x7f3c280742c0 msgr2=0x7f3c2814f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 -- 172.21.15.7:0/1040750661 shutdown_connections 2024-04-25T11:18:41.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.156+0000 7f3c2da00640 1 -- 172.21.15.7:0/1040750661 wait complete. 2024-04-25T11:18:41.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:18:41.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.465+0000 7f4570e00640 1 Processor -- start 2024-04-25T11:18:41.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.465+0000 7f4570e00640 1 -- start start 2024-04-25T11:18:41.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.468+0000 7f4570e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f456c144a80 0x7f456c146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:41.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.468+0000 7f4570e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f456c0b66b0 con 0x7f456c0b4f80 2024-04-25T11:18:41.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.468+0000 7f4570e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f456c0b5b30 con 0x7f456c145e40 2024-04-25T11:18:41.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.468+0000 7f4570e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f456c145b70 con 0x7f456c144a80 2024-04-25T11:18:41.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.468+0000 7f456be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f456c0b4f80 0x7f456c1606f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47778/0 (socket says 172.21.15.7:47778) 2024-04-25T11:18:41.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.468+0000 7f456be00640 1 -- 172.21.15.7:0/3830905773 learned_addr learned my addr 172.21.15.7:0/3830905773 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:41.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.468+0000 7f456aa00640 1 --2- 172.21.15.7:0/3830905773 >> v2:172.21.15.7:3300/0 conn(0x7f456c144a80 0x7f456c146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:41.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.469+0000 7f456a000640 1 -- 172.21.15.7:0/3830905773 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1471527142 0 0) 0x7f456c0b66b0 con 0x7f456c0b4f80 2024-04-25T11:18:41.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.469+0000 7f456a000640 1 -- 172.21.15.7:0/3830905773 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4550003520 con 0x7f456c0b4f80 2024-04-25T11:18:41.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.469+0000 7f456aa00640 1 -- 172.21.15.7:0/3830905773 >> v1:172.21.15.87:6789/0 conn(0x7f456c145e40 legacy=0x7f456c0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:41.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.469+0000 7f456aa00640 1 -- 172.21.15.7:0/3830905773 >> v1:172.21.15.7:6789/0 conn(0x7f456c0b4f80 legacy=0x7f456c1606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:41.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.469+0000 7f456aa00640 1 -- 172.21.15.7:0/3830905773 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c0b5e10 con 0x7f456c144a80 2024-04-25T11:18:41.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.470+0000 7f456aa00640 1 --2- 172.21.15.7:0/3830905773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f456c144a80 0x7f456c146250 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f45540097a0 tx=0x7f455402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc794926ab813adb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:41.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.470+0000 7f456a000640 1 -- 172.21.15.7:0/3830905773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554037070 con 0x7f456c144a80 2024-04-25T11:18:41.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.470+0000 7f456a000640 1 -- 172.21.15.7:0/3830905773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4554032440 con 0x7f456c144a80 2024-04-25T11:18:41.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.470+0000 7f456a000640 1 -- 172.21.15.7:0/3830905773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45540327f0 con 0x7f456c144a80 2024-04-25T11:18:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.471+0000 7f4570e00640 1 -- 172.21.15.7:0/3830905773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f456c144a80 msgr2=0x7f456c146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.471+0000 7f4570e00640 1 --2- 172.21.15.7:0/3830905773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f456c144a80 0x7f456c146250 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f45540097a0 tx=0x7f455402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.471+0000 7f4570e00640 1 -- 172.21.15.7:0/3830905773 shutdown_connections 2024-04-25T11:18:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.471+0000 7f4570e00640 1 --2- 172.21.15.7:0/3830905773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f456c144a80 0x7f456c146250 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.471+0000 7f4570e00640 1 -- 172.21.15.7:0/3830905773 >> 172.21.15.7:0/3830905773 conn(0x7f456c0740f0 msgr2=0x7f456c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.472+0000 7f4570e00640 1 -- 172.21.15.7:0/3830905773 shutdown_connections 2024-04-25T11:18:41.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.472+0000 7f4570e00640 1 -- 172.21.15.7:0/3830905773 wait complete. 2024-04-25T11:18:41.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.474+0000 7f4570e00640 1 Processor -- start 2024-04-25T11:18:41.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.474+0000 7f4570e00640 1 -- start start 2024-04-25T11:18:41.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f4570e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f456c0b4f80 0x7f456c0aac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:41.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f4570e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f456c144a80 0x7f456c0ab170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f4570e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f456c145e40 0x7f456c0cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f4570e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f456c161e10 con 0x7f456c145e40 2024-04-25T11:18:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f4570e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f456c0b63d0 con 0x7f456c144a80 2024-04-25T11:18:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f4570e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f456c161f90 con 0x7f456c0b4f80 2024-04-25T11:18:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f456be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f456c145e40 0x7f456c0cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f456be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f456c145e40 0x7f456c0cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50844/0 (socket says 172.21.15.7:50844) 2024-04-25T11:18:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f456be00640 1 -- 172.21.15.7:0/3451223201 learned_addr learned my addr 172.21.15.7:0/3451223201 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:41.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f456b400640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f456c0b4f80 0x7f456c0aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:41.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f456be00640 1 -- 172.21.15.7:0/3451223201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f456c0b4f80 msgr2=0x7f456c0aac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:41.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f456be00640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f456c0b4f80 0x7f456c0aac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f456aa00640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f456c144a80 0x7f456c0ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:41.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f456be00640 1 -- 172.21.15.7:0/3451223201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f456c144a80 msgr2=0x7f456c0ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:41.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.476+0000 7f456be00640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f456c144a80 0x7f456c0ab170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.477+0000 7f456be00640 1 -- 172.21.15.7:0/3451223201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c0ac250 con 0x7f456c145e40 2024-04-25T11:18:41.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.477+0000 7f456b400640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f456c0b4f80 0x7f456c0aac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.477+0000 7f456be00640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f456c145e40 0x7f456c0cd1e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f456400c390 tx=0x7f456400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.477+0000 7f4563400640 1 -- 172.21.15.7:0/3451223201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564017070 con 0x7f456c145e40 2024-04-25T11:18:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.477+0000 7f4563400640 1 -- 172.21.15.7:0/3451223201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4564005ce0 con 0x7f456c145e40 2024-04-25T11:18:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.477+0000 7f4570e00640 1 -- 172.21.15.7:0/3451223201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c0ab9b0 con 0x7f456c145e40 2024-04-25T11:18:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.477+0000 7f4563400640 1 -- 172.21.15.7:0/3451223201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45640072d0 con 0x7f456c145e40 2024-04-25T11:18:41.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.477+0000 7f4570e00640 1 -- 172.21.15.7:0/3451223201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c0abf70 con 0x7f456c145e40 2024-04-25T11:18:41.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.478+0000 7f4570e00640 1 -- 172.21.15.7:0/3451223201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456c06eef0 con 0x7f456c145e40 2024-04-25T11:18:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.480+0000 7f4563400640 1 -- 172.21.15.7:0/3451223201 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f456401a070 con 0x7f456c145e40 2024-04-25T11:18:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.481+0000 7f4563400640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45480744b0 0x7f45480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.481+0000 7f4563400640 1 -- 172.21.15.7:0/3451223201 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f4564094720 con 0x7f456c145e40 2024-04-25T11:18:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.481+0000 7f456b400640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45480744b0 0x7f45480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:41.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.482+0000 7f456b400640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45480744b0 0x7f45480769c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f455c0012e0 tx=0x7f455c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:41.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.486+0000 7f4563400640 1 -- 172.21.15.7:0/3451223201 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f456c06eef0 con 0x7f456c145e40 2024-04-25T11:18:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.693+0000 7f4570e00640 1 -- 172.21.15.7:0/3451223201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f456c0ab740 con 0x7f456c145e40 2024-04-25T11:18:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.694+0000 7f4563400640 1 -- 172.21.15.7:0/3451223201 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f456405e5c0 con 0x7f456c145e40 2024-04-25T11:18:41.693 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:18:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 -- 172.21.15.7:0/3451223201 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45480744b0 msgr2=0x7f45480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45480744b0 0x7f45480769c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f455c0012e0 tx=0x7f455c009040 comp rx=0 tx=0).stop 2024-04-25T11:18:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 -- 172.21.15.7:0/3451223201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f456c145e40 msgr2=0x7f456c0cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f456c145e40 0x7f456c0cd1e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f456400c390 tx=0x7f456400c850 comp rx=0 tx=0).stop 2024-04-25T11:18:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 -- 172.21.15.7:0/3451223201 shutdown_connections 2024-04-25T11:18:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45480744b0 0x7f45480769c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f456c145e40 0x7f456c0cd1e0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f456c144a80 0x7f456c0ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 --2- 172.21.15.7:0/3451223201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f456c0b4f80 0x7f456c0aac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 -- 172.21.15.7:0/3451223201 >> 172.21.15.7:0/3451223201 conn(0x7f456c0740f0 msgr2=0x7f456c142f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 -- 172.21.15.7:0/3451223201 shutdown_connections 2024-04-25T11:18:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:41.697+0000 7f4570e00640 1 -- 172.21.15.7:0/3451223201 wait complete. 2024-04-25T11:18:41.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:18:42.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.014+0000 7fbcc6400640 1 Processor -- start 2024-04-25T11:18:42.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.014+0000 7fbcc6400640 1 -- start start 2024-04-25T11:18:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.016+0000 7fbcc6400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbcc00b6240 0x7fbcc015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.016+0000 7fbcc6400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcc0164e00 con 0x7fbcc00aef90 2024-04-25T11:18:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.016+0000 7fbcc6400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcc01650e0 con 0x7fbcc00b6920 2024-04-25T11:18:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.016+0000 7fbcc6400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc01653c0 con 0x7fbcc00b6240 2024-04-25T11:18:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.017+0000 7fbcbf400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbcc00b6240 0x7fbcc015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.017+0000 7fbcbfe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbcc00aef90 0x7fbcc00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47794/0 (socket says 172.21.15.7:47794) 2024-04-25T11:18:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.017+0000 7fbcbfe00640 1 -- 172.21.15.7:0/2759616469 learned_addr learned my addr 172.21.15.7:0/2759616469 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.017+0000 7fbcbf400640 1 -- 172.21.15.7:0/2759616469 >> v1:172.21.15.87:6789/0 conn(0x7fbcc00b6920 legacy=0x7fbcc00ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.017+0000 7fbcbf400640 1 -- 172.21.15.7:0/2759616469 >> v1:172.21.15.7:6789/0 conn(0x7fbcc00aef90 legacy=0x7fbcc00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.017+0000 7fbcbe000640 1 -- 172.21.15.7:0/2759616469 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917410539 0 0) 0x7fbcc0164e00 con 0x7fbcc00aef90 2024-04-25T11:18:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.017+0000 7fbcbf400640 1 -- 172.21.15.7:0/2759616469 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc01656a0 con 0x7fbcc00b6240 2024-04-25T11:18:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.018+0000 7fbcbf400640 1 --2- 172.21.15.7:0/2759616469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcc00b6240 0x7fbcc015eb00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00097a0 tx=0x7fbcb002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346b02cee95f75e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.019+0000 7fbcbe000640 1 -- 172.21.15.7:0/2759616469 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0037070 con 0x7fbcc00b6240 2024-04-25T11:18:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.019+0000 7fbcbe000640 1 -- 172.21.15.7:0/2759616469 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb002ae50 con 0x7fbcc00b6240 2024-04-25T11:18:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.019+0000 7fbcbe000640 1 -- 172.21.15.7:0/2759616469 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0032660 con 0x7fbcc00b6240 2024-04-25T11:18:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.019+0000 7fbcc6400640 1 -- 172.21.15.7:0/2759616469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcc00b6240 msgr2=0x7fbcc015eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.019+0000 7fbcc6400640 1 --2- 172.21.15.7:0/2759616469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcc00b6240 0x7fbcc015eb00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00097a0 tx=0x7fbcb002a140 comp rx=0 tx=0).stop 2024-04-25T11:18:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.020+0000 7fbcc6400640 1 -- 172.21.15.7:0/2759616469 shutdown_connections 2024-04-25T11:18:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.020+0000 7fbcc6400640 1 --2- 172.21.15.7:0/2759616469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcc00b6240 0x7fbcc015eb00 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.020+0000 7fbcc6400640 1 -- 172.21.15.7:0/2759616469 >> 172.21.15.7:0/2759616469 conn(0x7fbcc00742c0 msgr2=0x7fbcc006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.020+0000 7fbcc6400640 1 -- 172.21.15.7:0/2759616469 shutdown_connections 2024-04-25T11:18:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.020+0000 7fbcc6400640 1 -- 172.21.15.7:0/2759616469 wait complete. 2024-04-25T11:18:42.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.022+0000 7fbcc6400640 1 Processor -- start 2024-04-25T11:18:42.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.022+0000 7fbcc6400640 1 -- start start 2024-04-25T11:18:42.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.024+0000 7fbcc6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcc00aef90 0x7fbcc014f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.024+0000 7fbcc6400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc00b6240 0x7fbcc0156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.024+0000 7fbcc6400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00b6920 0x7fbcc0156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.024+0000 7fbcc6400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0164ba0 con 0x7fbcc00aef90 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.024+0000 7fbcc6400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc0164a20 con 0x7fbcc00b6920 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.024+0000 7fbcc6400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc00b5300 con 0x7fbcc00b6240 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.024+0000 7fbcbf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcc00aef90 0x7fbcc014f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.024+0000 7fbcbf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcc00aef90 0x7fbcc014f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50858/0 (socket says 172.21.15.7:50858) 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.024+0000 7fbcbf400640 1 -- 172.21.15.7:0/3564464215 learned_addr learned my addr 172.21.15.7:0/3564464215 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.024+0000 7fbcbea00640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc00b6240 0x7fbcc0156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.025+0000 7fbcbfe00640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00b6920 0x7fbcc0156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.025+0000 7fbcbf400640 1 -- 172.21.15.7:0/3564464215 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc00b6240 msgr2=0x7fbcc0156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.025+0000 7fbcbf400640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc00b6240 0x7fbcc0156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.025+0000 7fbcbf400640 1 -- 172.21.15.7:0/3564464215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00b6920 msgr2=0x7fbcc0156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.025+0000 7fbcbf400640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00b6920 0x7fbcc0156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.025+0000 7fbcbf400640 1 -- 172.21.15.7:0/3564464215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc015a800 con 0x7fbcc00aef90 2024-04-25T11:18:42.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.025+0000 7fbcbfe00640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00b6920 0x7fbcc0156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:42.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.025+0000 7fbcbf400640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcc00aef90 0x7fbcc014f980 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0035040 tx=0x7fbcb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:42.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.025+0000 7fbcb7400640 1 -- 172.21.15.7:0/3564464215 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0037040 con 0x7fbcc00aef90 2024-04-25T11:18:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.025+0000 7fbcb7400640 1 -- 172.21.15.7:0/3564464215 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb0004030 con 0x7fbcc00aef90 2024-04-25T11:18:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.026+0000 7fbcb7400640 1 -- 172.21.15.7:0/3564464215 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0006e10 con 0x7fbcc00aef90 2024-04-25T11:18:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.026+0000 7fbcc6400640 1 -- 172.21.15.7:0/3564464215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc015a520 con 0x7fbcc00aef90 2024-04-25T11:18:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.026+0000 7fbcc6400640 1 -- 172.21.15.7:0/3564464215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc0153a50 con 0x7fbcc00aef90 2024-04-25T11:18:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.027+0000 7fbcc6400640 1 -- 172.21.15.7:0/3564464215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc0050740 con 0x7fbcc00aef90 2024-04-25T11:18:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.028+0000 7fbcb7400640 1 -- 172.21.15.7:0/3564464215 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbcb003d650 con 0x7fbcc00aef90 2024-04-25T11:18:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.029+0000 7fbcb7400640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbca00744e0 0x7fbca00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.029+0000 7fbcb7400640 1 -- 172.21.15.7:0/3564464215 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7fbcb00b58a0 con 0x7fbcc00aef90 2024-04-25T11:18:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.030+0000 7fbcbea00640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbca00744e0 0x7fbca00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.031+0000 7fbcbea00640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbca00744e0 0x7fbca00769f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbcac006fd0 tx=0x7fbcac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.035+0000 7fbcb7400640 1 -- 172.21.15.7:0/3564464215 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbcb0042020 con 0x7fbcc00aef90 2024-04-25T11:18:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.237+0000 7fbcc6400640 1 -- 172.21.15.7:0/3564464215 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbcc00710c0 con 0x7fbcc00aef90 2024-04-25T11:18:42.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.238+0000 7fbcb7400640 1 -- 172.21.15.7:0/3564464215 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fbcb007fd90 con 0x7fbcc00aef90 2024-04-25T11:18:42.237 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:18:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.240+0000 7fbcc6400640 1 -- 172.21.15.7:0/3564464215 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbca00744e0 msgr2=0x7fbca00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.241+0000 7fbcc6400640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbca00744e0 0x7fbca00769f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbcac006fd0 tx=0x7fbcac008040 comp rx=0 tx=0).stop 2024-04-25T11:18:42.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.241+0000 7fbcc6400640 1 -- 172.21.15.7:0/3564464215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcc00aef90 msgr2=0x7fbcc014f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.241+0000 7fbcc6400640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcc00aef90 0x7fbcc014f980 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0035040 tx=0x7fbcb00046c0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.241+0000 7fbcc6400640 1 -- 172.21.15.7:0/3564464215 shutdown_connections 2024-04-25T11:18:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.241+0000 7fbcc6400640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbca00744e0 0x7fbca00769f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.241+0000 7fbcc6400640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00b6920 0x7fbcc0156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.241+0000 7fbcc6400640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc00b6240 0x7fbcc0156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.241+0000 7fbcc6400640 1 --2- 172.21.15.7:0/3564464215 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcc00aef90 0x7fbcc014f980 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.241+0000 7fbcc6400640 1 -- 172.21.15.7:0/3564464215 >> 172.21.15.7:0/3564464215 conn(0x7fbcc00742c0 msgr2=0x7fbcc00b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.241+0000 7fbcc6400640 1 -- 172.21.15.7:0/3564464215 shutdown_connections 2024-04-25T11:18:42.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.241+0000 7fbcc6400640 1 -- 172.21.15.7:0/3564464215 wait complete. 2024-04-25T11:18:42.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:18:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.542+0000 7f544d600640 1 Processor -- start 2024-04-25T11:18:42.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.542+0000 7f544d600640 1 -- start start 2024-04-25T11:18:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.544+0000 7f544d600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54480b2ab0 0x7f54480cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.544+0000 7f544d600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54480b3a90 con 0x7f54480cf3d0 2024-04-25T11:18:42.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.544+0000 7f544d600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f544809cf90 con 0x7f54480b4d00 2024-04-25T11:18:42.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.544+0000 7f544d600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f54480b3d30 con 0x7f54480b2ab0 2024-04-25T11:18:42.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.545+0000 7f544cc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f54480cf3d0 0x7f54480d1dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47806/0 (socket says 172.21.15.7:47806) 2024-04-25T11:18:42.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.545+0000 7f544cc00640 1 -- 172.21.15.7:0/2085399156 learned_addr learned my addr 172.21.15.7:0/2085399156 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.545+0000 7f543fe00640 1 --2- 172.21.15.7:0/2085399156 >> v2:172.21.15.87:3300/0 conn(0x7f54480b2ab0 0x7f54480cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.545+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3403895357 0 0) 0x7f54480b3a90 con 0x7f54480cf3d0 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.545+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f542c003520 con 0x7f54480cf3d0 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.545+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4073943066 0 0) 0x7f542c003520 con 0x7f54480cf3d0 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 >> v2:172.21.15.87:3300/0 conn(0x7f54480b2ab0 msgr2=0x7f54480cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 --2- 172.21.15.7:0/2085399156 >> v2:172.21.15.87:3300/0 conn(0x7f54480b2ab0 0x7f54480cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 >> v1:172.21.15.87:6789/0 conn(0x7f54480b4d00 legacy=0x7f544809bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54480b4010 con 0x7f54480cf3d0 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f5440001a10 con 0x7f54480cf3d0 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 >> v1:172.21.15.7:6789/0 conn(0x7f54480cf3d0 legacy=0x7f54480d1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 --2- 172.21.15.7:0/2085399156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f542c0039b0 0x7f542c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 --2- 172.21.15.7:0/2085399156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c004800 0x7f542c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 --2- 172.21.15.7:0/2085399156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c005270 0x7f542c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54480b3d30 con 0x7f542c004800 2024-04-25T11:18:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f542c00c200 con 0x7f542c0039b0 2024-04-25T11:18:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f542c00c400 con 0x7f542c005270 2024-04-25T11:18:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f543fe00640 1 --2- 172.21.15.7:0/2085399156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c004800 0x7f542c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f543fe00640 1 -- 172.21.15.7:0/2085399156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c005270 msgr2=0x7f542c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f543fe00640 1 --2- 172.21.15.7:0/2085399156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c005270 0x7f542c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f543fe00640 1 -- 172.21.15.7:0/2085399156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f542c0039b0 msgr2=0x7f542c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f543fe00640 1 --2- 172.21.15.7:0/2085399156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f542c0039b0 0x7f542c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f543fe00640 1 -- 172.21.15.7:0/2085399156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542c003520 con 0x7f542c004800 2024-04-25T11:18:42.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.546+0000 7f544cc00640 1 -- 172.21.15.7:0/2085399156 reap_dead start 2024-04-25T11:18:42.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.547+0000 7f543fe00640 1 --2- 172.21.15.7:0/2085399156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c004800 0x7f542c004c10 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f543000ba30 tx=0x7f542c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47681fc2ec94353 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.548+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430030070 con 0x7f542c004800 2024-04-25T11:18:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.548+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543002ade0 con 0x7f542c004800 2024-04-25T11:18:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.548+0000 7f5447400640 1 -- 172.21.15.7:0/2085399156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543000b600 con 0x7f542c004800 2024-04-25T11:18:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.548+0000 7f544d600640 1 -- 172.21.15.7:0/2085399156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c004800 msgr2=0x7f542c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.548+0000 7f544d600640 1 --2- 172.21.15.7:0/2085399156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c004800 0x7f542c004c10 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f543000ba30 tx=0x7f542c005830 comp rx=0 tx=0).stop 2024-04-25T11:18:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.549+0000 7f544d600640 1 -- 172.21.15.7:0/2085399156 shutdown_connections 2024-04-25T11:18:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.549+0000 7f544d600640 1 --2- 172.21.15.7:0/2085399156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c004800 0x7f542c004c10 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.549+0000 7f544d600640 1 -- 172.21.15.7:0/2085399156 >> 172.21.15.7:0/2085399156 conn(0x7f54480b5240 msgr2=0x7f54480b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.549+0000 7f544d600640 1 -- 172.21.15.7:0/2085399156 shutdown_connections 2024-04-25T11:18:42.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.549+0000 7f544d600640 1 -- 172.21.15.7:0/2085399156 wait complete. 2024-04-25T11:18:42.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.551+0000 7f544d600640 1 Processor -- start 2024-04-25T11:18:42.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.551+0000 7f544d600640 1 -- start start 2024-04-25T11:18:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.553+0000 7f544d600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c004800 0x7f54481dbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.553+0000 7f544d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54480c7680 0x7f54480c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.553+0000 7f544d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54480c9b20 0x7f54480c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.553+0000 7f544d600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f544809cc80 con 0x7f54480c7680 2024-04-25T11:18:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.553+0000 7f544d600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f544809cb00 con 0x7f54480c9b20 2024-04-25T11:18:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.553+0000 7f544d600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f54480cee90 con 0x7f542c004800 2024-04-25T11:18:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.553+0000 7f543fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54480c7680 0x7f54480c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.553+0000 7f543fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54480c7680 0x7f54480c7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50880/0 (socket says 172.21.15.7:50880) 2024-04-25T11:18:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.553+0000 7f543fe00640 1 -- 172.21.15.7:0/3533657736 learned_addr learned my addr 172.21.15.7:0/3533657736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.554+0000 7f5447e00640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c004800 0x7f54481dbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.554+0000 7f544cc00640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54480c9b20 0x7f54480c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.554+0000 7f543fe00640 1 -- 172.21.15.7:0/3533657736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c004800 msgr2=0x7f54481dbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.554+0000 7f543fe00640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c004800 0x7f54481dbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.554+0000 7f543fe00640 1 -- 172.21.15.7:0/3533657736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54480c9b20 msgr2=0x7f54480c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.554+0000 7f543fe00640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54480c9b20 0x7f54480c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.554+0000 7f543fe00640 1 -- 172.21.15.7:0/3533657736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54480ca470 con 0x7f54480c7680 2024-04-25T11:18:42.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.554+0000 7f543fe00640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54480c7680 0x7f54480c7a90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f543002a8d0 tx=0x7f54300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.555+0000 7f5444c00640 1 -- 172.21.15.7:0/3533657736 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430043030 con 0x7f54480c7680 2024-04-25T11:18:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.555+0000 7f5444c00640 1 -- 172.21.15.7:0/3533657736 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543002abe0 con 0x7f54480c7680 2024-04-25T11:18:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.555+0000 7f5444c00640 1 -- 172.21.15.7:0/3533657736 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543000b600 con 0x7f54480c7680 2024-04-25T11:18:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.555+0000 7f544d600640 1 -- 172.21.15.7:0/3533657736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54480c83d0 con 0x7f54480c7680 2024-04-25T11:18:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.555+0000 7f544d600640 1 -- 172.21.15.7:0/3533657736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54480c8990 con 0x7f54480c7680 2024-04-25T11:18:42.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.556+0000 7f544d600640 1 -- 172.21.15.7:0/3533657736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5448050740 con 0x7f54480c7680 2024-04-25T11:18:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.557+0000 7f5444c00640 1 -- 172.21.15.7:0/3533657736 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5430046070 con 0x7f54480c7680 2024-04-25T11:18:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.558+0000 7f5444c00640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54240744b0 0x7f54240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.558+0000 7f5444c00640 1 -- 172.21.15.7:0/3533657736 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f54300b5fe0 con 0x7f54480c7680 2024-04-25T11:18:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.558+0000 7f5447e00640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54240744b0 0x7f54240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:42.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.559+0000 7f5447e00640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54240744b0 0x7f54240769c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5438007fd0 tx=0x7f5438009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:42.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.563+0000 7f5444c00640 1 -- 172.21.15.7:0/3533657736 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f54480ca470 con 0x7f54480c7680 2024-04-25T11:18:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.778+0000 7f544d600640 1 -- 172.21.15.7:0/3533657736 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f54480b9090 con 0x7f54480c7680 2024-04-25T11:18:42.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.779+0000 7f5444c00640 1 -- 172.21.15.7:0/3533657736 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v94) ==== 80+0+0 (secure 0 0 0) 0x7f54300804f0 con 0x7f54480c7680 2024-04-25T11:18:42.778 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:18:42.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.781+0000 7f544d600640 1 -- 172.21.15.7:0/3533657736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54240744b0 msgr2=0x7f54240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.781+0000 7f544d600640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54240744b0 0x7f54240769c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5438007fd0 tx=0x7f5438009040 comp rx=0 tx=0).stop 2024-04-25T11:18:42.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.781+0000 7f544d600640 1 -- 172.21.15.7:0/3533657736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54480c7680 msgr2=0x7f54480c7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:42.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.781+0000 7f544d600640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54480c7680 0x7f54480c7a90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f543002a8d0 tx=0x7f54300046c0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.782+0000 7f544d600640 1 -- 172.21.15.7:0/3533657736 shutdown_connections 2024-04-25T11:18:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.782+0000 7f544d600640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54240744b0 0x7f54240769c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.782+0000 7f544d600640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54480c9b20 0x7f54480c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.782+0000 7f544d600640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54480c7680 0x7f54480c7a90 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.782+0000 7f544d600640 1 --2- 172.21.15.7:0/3533657736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c004800 0x7f54481dbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.782+0000 7f544d600640 1 -- 172.21.15.7:0/3533657736 >> 172.21.15.7:0/3533657736 conn(0x7f54480b5240 msgr2=0x7f54480724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.782+0000 7f544d600640 1 -- 172.21.15.7:0/3533657736 shutdown_connections 2024-04-25T11:18:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:42.782+0000 7f544d600640 1 -- 172.21.15.7:0/3533657736 wait complete. 2024-04-25T11:18:42.806 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:18:42.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:18:42.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:18:42.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:18:42.959 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:18:42.962 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:18:42.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:18:42.962 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:18:43.041 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:18:43.041 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:18:43.041 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:18:43.081 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:18:43.081 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:18:43.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:18:43.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.454+0000 7f634d000640 1 Processor -- start 2024-04-25T11:18:43.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.454+0000 7f634d000640 1 -- start start 2024-04-25T11:18:43.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.456+0000 7f634d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63480b0290 0x7f63480b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.456+0000 7f634d000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63480b4980 con 0x7f634806f300 2024-04-25T11:18:43.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.456+0000 7f634d000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63480b4c60 con 0x7f634806dc90 2024-04-25T11:18:43.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.456+0000 7f634d000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63480b4f40 con 0x7f63480b0290 2024-04-25T11:18:43.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.457+0000 7f6346800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f634806f300 0x7f63480b5f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47808/0 (socket says 172.21.15.7:47808) 2024-04-25T11:18:43.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.457+0000 7f6346800640 1 -- 172.21.15.7:0/305449552 learned_addr learned my addr 172.21.15.7:0/305449552 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:43.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.457+0000 7f633d400640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63480b0290 0x7f63480b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.457+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193168229 0 0) 0x7f63480b4980 con 0x7f634806f300 2024-04-25T11:18:43.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.457+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f632c003520 con 0x7f634806f300 2024-04-25T11:18:43.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.457+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 685975859 0 0) 0x7f632c003520 con 0x7f634806f300 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63480b0290 msgr2=0x7f63480b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63480b0290 0x7f63480b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 >> v1:172.21.15.87:6789/0 conn(0x7f634806dc90 legacy=0x7f63480afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63480b5220 con 0x7f634806f300 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f63400019f0 con 0x7f634806f300 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 >> v1:172.21.15.7:6789/0 conn(0x7f634806f300 legacy=0x7f63480b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f632c0039e0 0x7f632c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c004370 0x7f632c0048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c0050c0 0x7f632c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63480b4f40 con 0x7f632c0050c0 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f632c00bf60 con 0x7f632c004370 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f632c00c160 con 0x7f632c0039e0 2024-04-25T11:18:43.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6346800640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c0050c0 0x7f632c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.458+0000 7f6345e00640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f632c0039e0 0x7f632c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.459+0000 7f633d400640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c004370 0x7f632c0048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.459+0000 7f6346800640 1 -- 172.21.15.7:0/305449552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f632c0039e0 msgr2=0x7f632c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:43.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.459+0000 7f6346800640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f632c0039e0 0x7f632c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.459+0000 7f6346800640 1 -- 172.21.15.7:0/305449552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c004370 msgr2=0x7f632c0048c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:43.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.459+0000 7f6346800640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c004370 0x7f632c0048c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.459+0000 7f6346800640 1 -- 172.21.15.7:0/305449552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c003520 con 0x7f632c0050c0 2024-04-25T11:18:43.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.460+0000 7f6346800640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c0050c0 0x7f632c00b9c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6348073280 tx=0x7f632c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b52fba1bcfcb03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:43.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.460+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634000a460 con 0x7f632c0050c0 2024-04-25T11:18:43.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.460+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f634002f070 con 0x7f632c0050c0 2024-04-25T11:18:43.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.460+0000 7f6345400640 1 -- 172.21.15.7:0/305449552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634002bc60 con 0x7f632c0050c0 2024-04-25T11:18:43.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.461+0000 7f634d000640 1 -- 172.21.15.7:0/305449552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c0050c0 msgr2=0x7f632c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:43.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.461+0000 7f634d000640 1 --2- 172.21.15.7:0/305449552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f632c0050c0 0x7f632c00b9c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6348073280 tx=0x7f632c005830 comp rx=0 tx=0).stop 2024-04-25T11:18:43.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.461+0000 7f6346800640 1 -- 172.21.15.7:0/305449552 reap_dead start 2024-04-25T11:18:43.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.461+0000 7f634d000640 1 -- 172.21.15.7:0/305449552 shutdown_connections 2024-04-25T11:18:43.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.461+0000 7f634d000640 1 -- 172.21.15.7:0/305449552 >> 172.21.15.7:0/305449552 conn(0x7f6348074200 msgr2=0x7f634806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:43.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.461+0000 7f634d000640 1 -- 172.21.15.7:0/305449552 shutdown_connections 2024-04-25T11:18:43.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.461+0000 7f634d000640 1 -- 172.21.15.7:0/305449552 wait complete. 2024-04-25T11:18:43.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.463+0000 7f634d000640 1 Processor -- start 2024-04-25T11:18:43.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.464+0000 7f634d000640 1 -- start start 2024-04-25T11:18:43.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.465+0000 7f634d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f632c0050c0 0x7f63480b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.465+0000 7f634d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634814fd70 0x7f634814f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.465+0000 7f634d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f634806c570 0x7f634806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.465+0000 7f634d000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63480ce920 con 0x7f634814fd70 2024-04-25T11:18:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.465+0000 7f634d000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f634815d220 con 0x7f634806c570 2024-04-25T11:18:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.465+0000 7f634d000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63480ceaa0 con 0x7f632c0050c0 2024-04-25T11:18:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.466+0000 7f633d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634814fd70 0x7f634814f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.466+0000 7f633d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634814fd70 0x7f634814f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50888/0 (socket says 172.21.15.7:50888) 2024-04-25T11:18:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.466+0000 7f633d400640 1 -- 172.21.15.7:0/3954870349 learned_addr learned my addr 172.21.15.7:0/3954870349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.466+0000 7f6345e00640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f632c0050c0 0x7f63480b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.466+0000 7f633d400640 1 -- 172.21.15.7:0/3954870349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f632c0050c0 msgr2=0x7f63480b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.466+0000 7f633d400640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f632c0050c0 0x7f63480b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.466+0000 7f633d400640 1 -- 172.21.15.7:0/3954870349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f634806c570 msgr2=0x7f634806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.466+0000 7f633d400640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f634806c570 0x7f634806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.466+0000 7f633d400640 1 -- 172.21.15.7:0/3954870349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63481f5d10 con 0x7f634814fd70 2024-04-25T11:18:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.467+0000 7f633d400640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634814fd70 0x7f634814f480 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6330011630 tx=0x7f6330011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.467+0000 7f633ea00640 1 -- 172.21.15.7:0/3954870349 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330015870 con 0x7f634814fd70 2024-04-25T11:18:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.467+0000 7f633ea00640 1 -- 172.21.15.7:0/3954870349 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6330015a10 con 0x7f634814fd70 2024-04-25T11:18:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.467+0000 7f633ea00640 1 -- 172.21.15.7:0/3954870349 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330015dc0 con 0x7f634814fd70 2024-04-25T11:18:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.467+0000 7f634d000640 1 -- 172.21.15.7:0/3954870349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63480b3030 con 0x7f634814fd70 2024-04-25T11:18:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.467+0000 7f634d000640 1 -- 172.21.15.7:0/3954870349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63481f6d00 con 0x7f634814fd70 2024-04-25T11:18:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.468+0000 7f634d000640 1 -- 172.21.15.7:0/3954870349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f634806cec0 con 0x7f634814fd70 2024-04-25T11:18:43.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.469+0000 7f633ea00640 1 -- 172.21.15.7:0/3954870349 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f633000f490 con 0x7f634814fd70 2024-04-25T11:18:43.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.470+0000 7f633ea00640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63240744e0 0x7f63240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.471+0000 7f633ea00640 1 -- 172.21.15.7:0/3954870349 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f6330095c40 con 0x7f634814fd70 2024-04-25T11:18:43.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.471+0000 7f6345e00640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63240744e0 0x7f63240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.472+0000 7f6345e00640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63240744e0 0x7f63240769f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6338007fd0 tx=0x7f6338009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.475+0000 7f633ea00640 1 -- 172.21.15.7:0/3954870349 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f634806cec0 con 0x7f634814fd70 2024-04-25T11:18:43.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.682+0000 7f634d000640 1 -- 172.21.15.7:0/3954870349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6348070fb0 con 0x7f634814fd70 2024-04-25T11:18:43.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.684+0000 7f633ea00640 1 -- 172.21.15.7:0/3954870349 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16983 (secure 0 0 0) 0x7f6330019030 con 0x7f634814fd70 2024-04-25T11:18:43.684 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:43.684 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:17:04.183733+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:59.999922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T11:16:00.796013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":5}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 -- 172.21.15.7:0/3954870349 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63240744e0 msgr2=0x7f63240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63240744e0 0x7f63240769f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6338007fd0 tx=0x7f6338009040 comp rx=0 tx=0).stop 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 -- 172.21.15.7:0/3954870349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634814fd70 msgr2=0x7f634814f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634814fd70 0x7f634814f480 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6330011630 tx=0x7f6330011af0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 -- 172.21.15.7:0/3954870349 shutdown_connections 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63240744e0 0x7f63240769f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f634806c570 0x7f634806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f634814fd70 0x7f634814f480 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 --2- 172.21.15.7:0/3954870349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f632c0050c0 0x7f63480b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 -- 172.21.15.7:0/3954870349 >> 172.21.15.7:0/3954870349 conn(0x7f6348074200 msgr2=0x7f6348156050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 -- 172.21.15.7:0/3954870349 shutdown_connections 2024-04-25T11:18:43.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.687+0000 7f634d000640 1 -- 172.21.15.7:0/3954870349 wait complete. 2024-04-25T11:18:43.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:15:59.999922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:16:00.796013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:18:43.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:18:43.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.984+0000 7f8836e00640 1 Processor -- start 2024-04-25T11:18:43.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.984+0000 7f8836e00640 1 -- start start 2024-04-25T11:18:43.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.986+0000 7f8836e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88300d0480 0x7f88300d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.986+0000 7f8836e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f883006eea0 0x7f883006f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.986+0000 7f8836e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88300ce570 con 0x7f88301583c0 2024-04-25T11:18:43.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.986+0000 7f8836e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f883006f7f0 con 0x7f88300d0480 2024-04-25T11:18:43.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.986+0000 7f8836e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f883015f190 con 0x7f883006eea0 2024-04-25T11:18:43.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.986+0000 7f8835000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f88300d0480 0x7f88300d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.986+0000 7f8835a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f88301583c0 0x7f88300cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47824/0 (socket says 172.21.15.7:47824) 2024-04-25T11:18:43.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.986+0000 7f8835a00640 1 -- 172.21.15.7:0/1676408425 learned_addr learned my addr 172.21.15.7:0/1676408425 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:43.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.987+0000 7f8836400640 1 --2- 172.21.15.7:0/1676408425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f883006eea0 0x7f883006f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.987+0000 7f8835000640 1 -- 172.21.15.7:0/1676408425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f883006eea0 msgr2=0x7f883006f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:43.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.987+0000 7f8835000640 1 --2- 172.21.15.7:0/1676408425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f883006eea0 0x7f883006f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.987+0000 7f8835000640 1 -- 172.21.15.7:0/1676408425 >> v1:172.21.15.7:6789/0 conn(0x7f88301583c0 legacy=0x7f88300cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:43.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.987+0000 7f882fe00640 1 -- 172.21.15.7:0/1676408425 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2733804942 0 0) 0x7f88300ce570 con 0x7f88301583c0 2024-04-25T11:18:43.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.987+0000 7f8835000640 1 -- 172.21.15.7:0/1676408425 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88300ce850 con 0x7f88300d0480 2024-04-25T11:18:43.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.987+0000 7f8836400640 1 --2- 172.21.15.7:0/1676408425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f883006eea0 0x7f883006f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.988+0000 7f8835000640 1 --2- 172.21.15.7:0/1676408425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88300d0480 0x7f88300d0890 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f881c0097a0 tx=0x7f881c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a504a541176ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.988+0000 7f882fe00640 1 -- 172.21.15.7:0/1676408425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c037070 con 0x7f88300d0480 2024-04-25T11:18:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.988+0000 7f882fe00640 1 -- 172.21.15.7:0/1676408425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f881c032440 con 0x7f88300d0480 2024-04-25T11:18:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.988+0000 7f882fe00640 1 -- 172.21.15.7:0/1676408425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c032810 con 0x7f88300d0480 2024-04-25T11:18:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.989+0000 7f8836e00640 1 -- 172.21.15.7:0/1676408425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88300d0480 msgr2=0x7f88300d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.989+0000 7f8836e00640 1 --2- 172.21.15.7:0/1676408425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88300d0480 0x7f88300d0890 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f881c0097a0 tx=0x7f881c02a180 comp rx=0 tx=0).stop 2024-04-25T11:18:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.989+0000 7f8836e00640 1 -- 172.21.15.7:0/1676408425 shutdown_connections 2024-04-25T11:18:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.989+0000 7f8836e00640 1 --2- 172.21.15.7:0/1676408425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f883006eea0 0x7f883006f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.989+0000 7f8836e00640 1 --2- 172.21.15.7:0/1676408425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88300d0480 0x7f88300d0890 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.989+0000 7f8836e00640 1 -- 172.21.15.7:0/1676408425 >> 172.21.15.7:0/1676408425 conn(0x7f88300740f0 msgr2=0x7f883006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.989+0000 7f8836e00640 1 -- 172.21.15.7:0/1676408425 shutdown_connections 2024-04-25T11:18:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.989+0000 7f8836e00640 1 -- 172.21.15.7:0/1676408425 wait complete. 2024-04-25T11:18:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.992+0000 7f8836e00640 1 Processor -- start 2024-04-25T11:18:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.992+0000 7f8836e00640 1 -- start start 2024-04-25T11:18:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.993+0000 7f8836e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883006eea0 0x7f88300c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.993+0000 7f8836e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88300d0480 0x7f8830147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.993+0000 7f8836e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88301583c0 0x7f883015cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.993+0000 7f8836e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88300cf0f0 con 0x7f883006eea0 2024-04-25T11:18:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.993+0000 7f8836e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f88300cee10 con 0x7f88301583c0 2024-04-25T11:18:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.993+0000 7f8836e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f88300cfd20 con 0x7f88300d0480 2024-04-25T11:18:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8835a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883006eea0 0x7f88300c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8835a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883006eea0 0x7f88300c7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50906/0 (socket says 172.21.15.7:50906) 2024-04-25T11:18:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8835a00640 1 -- 172.21.15.7:0/1173685661 learned_addr learned my addr 172.21.15.7:0/1173685661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8835000640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88300d0480 0x7f8830147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8836400640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88301583c0 0x7f883015cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8835a00640 1 -- 172.21.15.7:0/1173685661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88300d0480 msgr2=0x7f8830147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:43.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8835a00640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88300d0480 0x7f8830147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8835a00640 1 -- 172.21.15.7:0/1173685661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88301583c0 msgr2=0x7f883015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8835a00640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88301583c0 0x7f883015cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8835a00640 1 -- 172.21.15.7:0/1173685661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88300c7550 con 0x7f883006eea0 2024-04-25T11:18:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8836400640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88301583c0 0x7f883015cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.994+0000 7f8835000640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88300d0480 0x7f8830147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.995+0000 7f8835a00640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883006eea0 0x7f88300c7010 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f882400c780 tx=0x7f882400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.995+0000 7f882d600640 1 -- 172.21.15.7:0/1173685661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824018070 con 0x7f883006eea0 2024-04-25T11:18:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.995+0000 7f882d600640 1 -- 172.21.15.7:0/1173685661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f882400a640 con 0x7f883006eea0 2024-04-25T11:18:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.995+0000 7f8836e00640 1 -- 172.21.15.7:0/1173685661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88300c8300 con 0x7f883006eea0 2024-04-25T11:18:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.995+0000 7f882d600640 1 -- 172.21.15.7:0/1173685661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882400aa30 con 0x7f883006eea0 2024-04-25T11:18:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.995+0000 7f8836e00640 1 -- 172.21.15.7:0/1173685661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88300c7a60 con 0x7f883006eea0 2024-04-25T11:18:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.996+0000 7f8836e00640 1 -- 172.21.15.7:0/1173685661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8830050740 con 0x7f883006eea0 2024-04-25T11:18:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.998+0000 7f882d600640 1 -- 172.21.15.7:0/1173685661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f882401b070 con 0x7f883006eea0 2024-04-25T11:18:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.999+0000 7f882d600640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88100744b0 0x7f88100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.999+0000 7f882d600640 1 -- 172.21.15.7:0/1173685661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f8824094a90 con 0x7f883006eea0 2024-04-25T11:18:44.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:43.999+0000 7f8835000640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88100744b0 0x7f88100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:44.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.000+0000 7f8835000640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88100744b0 0x7f88100769c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f881c034000 tx=0x7f881c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:44.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.004+0000 7f882d600640 1 -- 172.21.15.7:0/1173685661 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f882405ef80 con 0x7f883006eea0 2024-04-25T11:18:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.204+0000 7f8836e00640 1 -- 172.21.15.7:0/1173685661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8830070dd0 con 0x7f883006eea0 2024-04-25T11:18:44.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.205+0000 7f882d600640 1 -- 172.21.15.7:0/1173685661 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f882405e880 con 0x7f883006eea0 2024-04-25T11:18:44.204 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:18:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.207+0000 7f8836e00640 1 -- 172.21.15.7:0/1173685661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88100744b0 msgr2=0x7f88100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.207+0000 7f8836e00640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88100744b0 0x7f88100769c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f881c034000 tx=0x7f881c02e000 comp rx=0 tx=0).stop 2024-04-25T11:18:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.207+0000 7f8836e00640 1 -- 172.21.15.7:0/1173685661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883006eea0 msgr2=0x7f88300c7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.207+0000 7f8836e00640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883006eea0 0x7f88300c7010 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f882400c780 tx=0x7f882400cc40 comp rx=0 tx=0).stop 2024-04-25T11:18:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.208+0000 7f8836e00640 1 -- 172.21.15.7:0/1173685661 shutdown_connections 2024-04-25T11:18:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.208+0000 7f8836e00640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f88100744b0 0x7f88100769c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.208+0000 7f8836e00640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88301583c0 0x7f883015cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.208+0000 7f8836e00640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88300d0480 0x7f8830147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.208+0000 7f8836e00640 1 --2- 172.21.15.7:0/1173685661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883006eea0 0x7f88300c7010 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.208+0000 7f8836e00640 1 -- 172.21.15.7:0/1173685661 >> 172.21.15.7:0/1173685661 conn(0x7f88300740f0 msgr2=0x7f88300cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.208+0000 7f8836e00640 1 -- 172.21.15.7:0/1173685661 shutdown_connections 2024-04-25T11:18:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.208+0000 7f8836e00640 1 -- 172.21.15.7:0/1173685661 wait complete. 2024-04-25T11:18:44.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:18:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.510+0000 7efc7ea00640 1 Processor -- start 2024-04-25T11:18:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.511+0000 7efc7ea00640 1 -- start start 2024-04-25T11:18:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.513+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc780b0470 0x7efc7806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.513+0000 7efc7ea00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc7815e3c0 con 0x7efc7806f8e0 2024-04-25T11:18:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.513+0000 7efc7ea00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc780cf800 con 0x7efc7815dfb0 2024-04-25T11:18:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.513+0000 7efc7ea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efc780d1250 con 0x7efc780b0470 2024-04-25T11:18:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.513+0000 7efc7ca00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efc7806f8e0 0x7efc780aea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47826/0 (socket says 172.21.15.7:47826) 2024-04-25T11:18:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.513+0000 7efc7ca00640 1 -- 172.21.15.7:0/2137289021 learned_addr learned my addr 172.21.15.7:0/2137289021 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.514+0000 7efc6fe00640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc780b0470 0x7efc7806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.514+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2593121544 0 0) 0x7efc7815e3c0 con 0x7efc7806f8e0 2024-04-25T11:18:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.514+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc5c003520 con 0x7efc7806f8e0 2024-04-25T11:18:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.514+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 128970926 0 0) 0x7efc5c003520 con 0x7efc7806f8e0 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.514+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc780b0470 msgr2=0x7efc7806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.514+0000 7efc77400640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc780b0470 0x7efc7806f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.514+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 >> v1:172.21.15.87:6789/0 conn(0x7efc7815dfb0 legacy=0x7efc780d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.514+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc780cfaa0 con 0x7efc7806f8e0 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.514+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7efc700019f0 con 0x7efc7806f8e0 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.514+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 >> v1:172.21.15.7:6789/0 conn(0x7efc7806f8e0 legacy=0x7efc780aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc77400640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc780b0470 0x7efc5c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc77400640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc5c003fb0 0x7efc5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc77400640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c004d40 0x7efc5c0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc780d1250 con 0x7efc5c004d40 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efc5c00bbb0 con 0x7efc5c003fb0 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efc5c00bdb0 con 0x7efc780b0470 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc7ca00640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c004d40 0x7efc5c0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc6fe00640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc5c003fb0 0x7efc5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc77e00640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc780b0470 0x7efc5c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc7ca00640 1 -- 172.21.15.7:0/2137289021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc780b0470 msgr2=0x7efc5c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc7ca00640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc780b0470 0x7efc5c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc7ca00640 1 -- 172.21.15.7:0/2137289021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc5c003fb0 msgr2=0x7efc5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc7ca00640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc5c003fb0 0x7efc5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.515+0000 7efc7ca00640 1 -- 172.21.15.7:0/2137289021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5c003520 con 0x7efc5c004d40 2024-04-25T11:18:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.516+0000 7efc7ca00640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c004d40 0x7efc5c0051b0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7efc780b4690 tx=0x7efc5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9c0abe1e9e0959 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.517+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7000a3b0 con 0x7efc5c004d40 2024-04-25T11:18:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.517+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7002f070 con 0x7efc5c004d40 2024-04-25T11:18:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.517+0000 7efc77400640 1 -- 172.21.15.7:0/2137289021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7002bbe0 con 0x7efc5c004d40 2024-04-25T11:18:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.517+0000 7efc7ea00640 1 -- 172.21.15.7:0/2137289021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c004d40 msgr2=0x7efc5c0051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.517+0000 7efc7ea00640 1 --2- 172.21.15.7:0/2137289021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc5c004d40 0x7efc5c0051b0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7efc780b4690 tx=0x7efc5c005830 comp rx=0 tx=0).stop 2024-04-25T11:18:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.517+0000 7efc7ca00640 1 -- 172.21.15.7:0/2137289021 reap_dead start 2024-04-25T11:18:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.518+0000 7efc7ea00640 1 -- 172.21.15.7:0/2137289021 shutdown_connections 2024-04-25T11:18:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.518+0000 7efc7ea00640 1 -- 172.21.15.7:0/2137289021 >> 172.21.15.7:0/2137289021 conn(0x7efc780742c0 msgr2=0x7efc7806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.518+0000 7efc7ea00640 1 -- 172.21.15.7:0/2137289021 shutdown_connections 2024-04-25T11:18:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.518+0000 7efc7ea00640 1 -- 172.21.15.7:0/2137289021 wait complete. 2024-04-25T11:18:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.520+0000 7efc7ea00640 1 Processor -- start 2024-04-25T11:18:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.520+0000 7efc7ea00640 1 -- start start 2024-04-25T11:18:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc78159ad0 0x7efc78156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc7815a930 0x7efc7814fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc7ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc78152b90 0x7efc78152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc7ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc780d0340 con 0x7efc7815a930 2024-04-25T11:18:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc7ea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efc780d0060 con 0x7efc78152b90 2024-04-25T11:18:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc7ea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efc780d0620 con 0x7efc78159ad0 2024-04-25T11:18:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc6fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc7815a930 0x7efc7814fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc6fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc7815a930 0x7efc7814fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50922/0 (socket says 172.21.15.7:50922) 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc6fe00640 1 -- 172.21.15.7:0/3871870508 learned_addr learned my addr 172.21.15.7:0/3871870508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc77e00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc78159ad0 0x7efc78156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc7ca00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc78152b90 0x7efc78152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc6fe00640 1 -- 172.21.15.7:0/3871870508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc78159ad0 msgr2=0x7efc78156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc6fe00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc78159ad0 0x7efc78156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.522+0000 7efc6fe00640 1 -- 172.21.15.7:0/3871870508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc78152b90 msgr2=0x7efc78152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.523+0000 7efc6fe00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc78152b90 0x7efc78152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.523+0000 7efc6fe00640 1 -- 172.21.15.7:0/3871870508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc781534e0 con 0x7efc7815a930 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.523+0000 7efc7ca00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc78152b90 0x7efc78152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.523+0000 7efc6fe00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc7815a930 0x7efc7814fd10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7efc60011600 tx=0x7efc60011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.523+0000 7efc74c00640 1 -- 172.21.15.7:0/3871870508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6000f920 con 0x7efc7815a930 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.523+0000 7efc74c00640 1 -- 172.21.15.7:0/3871870508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6000fac0 con 0x7efc7815a930 2024-04-25T11:18:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.523+0000 7efc74c00640 1 -- 172.21.15.7:0/3871870508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6000fe70 con 0x7efc7815a930 2024-04-25T11:18:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.523+0000 7efc7ea00640 1 -- 172.21.15.7:0/3871870508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7815a0b0 con 0x7efc7815a930 2024-04-25T11:18:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.523+0000 7efc7ea00640 1 -- 172.21.15.7:0/3871870508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7814f510 con 0x7efc7815a930 2024-04-25T11:18:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.525+0000 7efc7ea00640 1 -- 172.21.15.7:0/3871870508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc78050740 con 0x7efc7815a930 2024-04-25T11:18:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.526+0000 7efc74c00640 1 -- 172.21.15.7:0/3871870508 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efc60015460 con 0x7efc7815a930 2024-04-25T11:18:44.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.527+0000 7efc74c00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc540744b0 0x7efc540769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:44.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.527+0000 7efc74c00640 1 -- 172.21.15.7:0/3871870508 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7efc600959e0 con 0x7efc7815a930 2024-04-25T11:18:44.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.528+0000 7efc77e00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc540744b0 0x7efc540769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:44.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.528+0000 7efc77e00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc540744b0 0x7efc540769c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efc7806f890 tx=0x7efc68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:44.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.532+0000 7efc74c00640 1 -- 172.21.15.7:0/3871870508 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc781534e0 con 0x7efc7815a930 2024-04-25T11:18:44.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.735+0000 7efc7ea00640 1 -- 172.21.15.7:0/3871870508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efc78070fb0 con 0x7efc7815a930 2024-04-25T11:18:44.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.736+0000 7efc74c00640 1 -- 172.21.15.7:0/3871870508 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7efc60019030 con 0x7efc7815a930 2024-04-25T11:18:44.735 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:18:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 -- 172.21.15.7:0/3871870508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc540744b0 msgr2=0x7efc540769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc540744b0 0x7efc540769c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efc7806f890 tx=0x7efc68009040 comp rx=0 tx=0).stop 2024-04-25T11:18:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 -- 172.21.15.7:0/3871870508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc7815a930 msgr2=0x7efc7814fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc7815a930 0x7efc7814fd10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7efc60011600 tx=0x7efc60011ac0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 -- 172.21.15.7:0/3871870508 shutdown_connections 2024-04-25T11:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc540744b0 0x7efc540769c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc78152b90 0x7efc78152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc7815a930 0x7efc7814fd10 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 --2- 172.21.15.7:0/3871870508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc78159ad0 0x7efc78156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 -- 172.21.15.7:0/3871870508 >> 172.21.15.7:0/3871870508 conn(0x7efc780742c0 msgr2=0x7efc7815b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 -- 172.21.15.7:0/3871870508 shutdown_connections 2024-04-25T11:18:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:44.739+0000 7efc7ea00640 1 -- 172.21.15.7:0/3871870508 wait complete. 2024-04-25T11:18:44.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:18:45.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.049+0000 7f5795a00640 1 Processor -- start 2024-04-25T11:18:45.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.050+0000 7f5795a00640 1 -- start start 2024-04-25T11:18:45.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.051+0000 7f5795a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57900b4690 0x7f57900b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.051+0000 7f5795a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5790143b10 0x7f5790143f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.051+0000 7f5795a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57900b5c40 con 0x7f5790144460 2024-04-25T11:18:45.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.051+0000 7f5795a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f57900b5f20 con 0x7f5790143b10 2024-04-25T11:18:45.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.051+0000 7f5795a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57900b6200 con 0x7f57900b4690 2024-04-25T11:18:45.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.052+0000 7f5795000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5790144460 0x7f5790144870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47836/0 (socket says 172.21.15.7:47836) 2024-04-25T11:18:45.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.052+0000 7f5795000640 1 -- 172.21.15.7:0/2538520330 learned_addr learned my addr 172.21.15.7:0/2538520330 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:45.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.052+0000 7f5787e00640 1 --2- 172.21.15.7:0/2538520330 >> v2:172.21.15.87:3300/0 conn(0x7f5790143b10 0x7f5790143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.052+0000 7f578fe00640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57900b4690 0x7f57900b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.052+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2111101698 0 0) 0x7f57900b5c40 con 0x7f5790144460 2024-04-25T11:18:45.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.052+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5774003520 con 0x7f5790144460 2024-04-25T11:18:45.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2129299598 0 0) 0x7f5774003520 con 0x7f5790144460 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57900b4690 msgr2=0x7f57900b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57900b4690 0x7f57900b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 >> v2:172.21.15.87:3300/0 conn(0x7f5790143b10 msgr2=0x7f5790143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 --2- 172.21.15.7:0/2538520330 >> v2:172.21.15.87:3300/0 conn(0x7f5790143b10 0x7f5790143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57900b64e0 con 0x7f5790144460 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f57880019f0 con 0x7f5790144460 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 >> v1:172.21.15.7:6789/0 conn(0x7f5790144460 legacy=0x7f5790144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57740039b0 0x7f5774003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57900b4690 0x7f5774004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5774004d40 0x7f5774005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57900b5f20 con 0x7f5774004d40 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57900b6200 con 0x7f57740039b0 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f577400ba80 con 0x7f57900b4690 2024-04-25T11:18:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.053+0000 7f5795000640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5774004d40 0x7f5774005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.054+0000 7f578fe00640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57740039b0 0x7f5774003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.054+0000 7f5787e00640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57900b4690 0x7f5774004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.054+0000 7f5795000640 1 -- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57900b4690 msgr2=0x7f5774004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.054+0000 7f5795000640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57900b4690 0x7f5774004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.054+0000 7f5795000640 1 -- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57740039b0 msgr2=0x7f5774003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.054+0000 7f5795000640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57740039b0 0x7f5774003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.054+0000 7f5795000640 1 -- 172.21.15.7:0/2538520330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5774003520 con 0x7f5774004d40 2024-04-25T11:18:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.054+0000 7f578fe00640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57740039b0 0x7f5774003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:18:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.055+0000 7f5795000640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5774004d40 0x7f5774005150 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f578800a780 tx=0x7f5774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acae062d7779cff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.055+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578800a3d0 con 0x7f5774004d40 2024-04-25T11:18:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.055+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f578802f070 con 0x7f5774004d40 2024-04-25T11:18:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.055+0000 7f578f400640 1 -- 172.21.15.7:0/2538520330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578802bc00 con 0x7f5774004d40 2024-04-25T11:18:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.056+0000 7f5795a00640 1 -- 172.21.15.7:0/2538520330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5774004d40 msgr2=0x7f5774005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.056+0000 7f5795a00640 1 --2- 172.21.15.7:0/2538520330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5774004d40 0x7f5774005150 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f578800a780 tx=0x7f5774005830 comp rx=0 tx=0).stop 2024-04-25T11:18:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.056+0000 7f5795000640 1 -- 172.21.15.7:0/2538520330 reap_dead start 2024-04-25T11:18:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.056+0000 7f5795a00640 1 -- 172.21.15.7:0/2538520330 shutdown_connections 2024-04-25T11:18:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.056+0000 7f5795a00640 1 -- 172.21.15.7:0/2538520330 >> 172.21.15.7:0/2538520330 conn(0x7f5790074140 msgr2=0x7f579006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.056+0000 7f5795a00640 1 -- 172.21.15.7:0/2538520330 shutdown_connections 2024-04-25T11:18:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.056+0000 7f5795a00640 1 -- 172.21.15.7:0/2538520330 wait complete. 2024-04-25T11:18:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.058+0000 7f5795a00640 1 Processor -- start 2024-04-25T11:18:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.059+0000 7f5795a00640 1 -- start start 2024-04-25T11:18:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.060+0000 7f5795a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5774004d40 0x7f57900ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.060+0000 7f5795a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5790156ec0 0x7f579015e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.060+0000 7f5795a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f579015edd0 0x7f57901f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.060+0000 7f5795a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5790143200 con 0x7f5790156ec0 2024-04-25T11:18:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.060+0000 7f5795a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5790070fa0 con 0x7f579015edd0 2024-04-25T11:18:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.060+0000 7f5795a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5790143380 con 0x7f5774004d40 2024-04-25T11:18:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.060+0000 7f5787e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5790156ec0 0x7f579015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.060+0000 7f5787e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5790156ec0 0x7f579015e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50950/0 (socket says 172.21.15.7:50950) 2024-04-25T11:18:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.060+0000 7f5787e00640 1 -- 172.21.15.7:0/723876868 learned_addr learned my addr 172.21.15.7:0/723876868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:45.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.061+0000 7f5795000640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f579015edd0 0x7f57901f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.061+0000 7f578fe00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5774004d40 0x7f57900ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.061+0000 7f5787e00640 1 -- 172.21.15.7:0/723876868 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5774004d40 msgr2=0x7f57900ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:45.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.061+0000 7f5787e00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5774004d40 0x7f57900ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.061+0000 7f5787e00640 1 -- 172.21.15.7:0/723876868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f579015edd0 msgr2=0x7f57901f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:45.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.061+0000 7f5787e00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f579015edd0 0x7f57901f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.061+0000 7f5787e00640 1 -- 172.21.15.7:0/723876868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579015f1e0 con 0x7f5790156ec0 2024-04-25T11:18:45.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.061+0000 7f5787e00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5790156ec0 0x7f579015e890 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5778011700 tx=0x7f5778011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.062+0000 7f578cc00640 1 -- 172.21.15.7:0/723876868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577800f9c0 con 0x7f5790156ec0 2024-04-25T11:18:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.062+0000 7f578cc00640 1 -- 172.21.15.7:0/723876868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577800fb60 con 0x7f5790156ec0 2024-04-25T11:18:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.062+0000 7f578cc00640 1 -- 172.21.15.7:0/723876868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778015460 con 0x7f5790156ec0 2024-04-25T11:18:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.062+0000 7f5795a00640 1 -- 172.21.15.7:0/723876868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57900ac060 con 0x7f5790156ec0 2024-04-25T11:18:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.062+0000 7f5795a00640 1 -- 172.21.15.7:0/723876868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57900ac600 con 0x7f5790156ec0 2024-04-25T11:18:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.063+0000 7f5795a00640 1 -- 172.21.15.7:0/723876868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5790050740 con 0x7f5790156ec0 2024-04-25T11:18:45.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.064+0000 7f578cc00640 1 -- 172.21.15.7:0/723876868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5778015600 con 0x7f5790156ec0 2024-04-25T11:18:45.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.066+0000 7f578cc00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f576c0744b0 0x7f576c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.066+0000 7f578cc00640 1 -- 172.21.15.7:0/723876868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f5778095e80 con 0x7f5790156ec0 2024-04-25T11:18:45.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.066+0000 7f578fe00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f576c0744b0 0x7f576c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.067+0000 7f578fe00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f576c0744b0 0x7f576c0769c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f57800098b0 tx=0x7f5780008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:45.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.071+0000 7f578cc00640 1 -- 172.21.15.7:0/723876868 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f579015f1e0 con 0x7f5790156ec0 2024-04-25T11:18:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.275+0000 7f5795a00640 1 -- 172.21.15.7:0/723876868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f57900d1320 con 0x7f5790156ec0 2024-04-25T11:18:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.276+0000 7f578cc00640 1 -- 172.21.15.7:0/723876868 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f5778019030 con 0x7f5790156ec0 2024-04-25T11:18:45.275 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:18:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 -- 172.21.15.7:0/723876868 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f576c0744b0 msgr2=0x7f576c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f576c0744b0 0x7f576c0769c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f57800098b0 tx=0x7f5780008040 comp rx=0 tx=0).stop 2024-04-25T11:18:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 -- 172.21.15.7:0/723876868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5790156ec0 msgr2=0x7f579015e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5790156ec0 0x7f579015e890 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5778011700 tx=0x7f5778011bc0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 -- 172.21.15.7:0/723876868 shutdown_connections 2024-04-25T11:18:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f576c0744b0 0x7f576c0769c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f579015edd0 0x7f57901f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5790156ec0 0x7f579015e890 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 --2- 172.21.15.7:0/723876868 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5774004d40 0x7f57900ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 -- 172.21.15.7:0/723876868 >> 172.21.15.7:0/723876868 conn(0x7f5790074140 msgr2=0x7f5790159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 -- 172.21.15.7:0/723876868 shutdown_connections 2024-04-25T11:18:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.279+0000 7f5795a00640 1 -- 172.21.15.7:0/723876868 wait complete. 2024-04-25T11:18:45.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:18:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.594+0000 7f7133000640 1 Processor -- start 2024-04-25T11:18:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.594+0000 7f7133000640 1 -- start start 2024-04-25T11:18:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.596+0000 7f7133000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f712c06ed90 0x7f712c1600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.596+0000 7f7133000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f712c0b3960 0x7f712c0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.596+0000 7f7133000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f712c0b7960 0x7f712c0d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.596+0000 7f7133000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f712c160630 con 0x7f712c0b7960 2024-04-25T11:18:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.596+0000 7f7133000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f712c0af730 con 0x7f712c0b3960 2024-04-25T11:18:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.596+0000 7f7133000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f712c0b7d70 con 0x7f712c06ed90 2024-04-25T11:18:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.597+0000 7f7132600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f712c0b7960 0x7f712c0d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.597+0000 7f7132600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f712c0b7960 0x7f712c0d17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50964/0 (socket says 172.21.15.7:50964) 2024-04-25T11:18:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.597+0000 7f7132600640 1 -- 172.21.15.7:0/517251792 learned_addr learned my addr 172.21.15.7:0/517251792 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.597+0000 7f7131c00640 1 --2- 172.21.15.7:0/517251792 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f712c06ed90 0x7f712c1600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.597+0000 7f7131200640 1 --2- 172.21.15.7:0/517251792 >> v2:172.21.15.87:3300/0 conn(0x7f712c0b3960 0x7f712c0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.597+0000 7f7132600640 1 -- 172.21.15.7:0/517251792 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f712c06ed90 msgr2=0x7f712c1600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.597+0000 7f7132600640 1 --2- 172.21.15.7:0/517251792 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f712c06ed90 0x7f712c1600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.597+0000 7f7132600640 1 -- 172.21.15.7:0/517251792 >> v2:172.21.15.87:3300/0 conn(0x7f712c0b3960 msgr2=0x7f712c0b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.597+0000 7f7132600640 1 --2- 172.21.15.7:0/517251792 >> v2:172.21.15.87:3300/0 conn(0x7f712c0b3960 0x7f712c0b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.597+0000 7f7132600640 1 -- 172.21.15.7:0/517251792 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c0cf850 con 0x7f712c0b7960 2024-04-25T11:18:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.599+0000 7f7132600640 1 --2- 172.21.15.7:0/517251792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f712c0b7960 0x7f712c0d17b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f712c0d3720 tx=0x7f712402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2766d9190511cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.599+0000 7f712be00640 1 -- 172.21.15.7:0/517251792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124036070 con 0x7f712c0b7960 2024-04-25T11:18:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.599+0000 7f712be00640 1 -- 172.21.15.7:0/517251792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124031600 con 0x7f712c0b7960 2024-04-25T11:18:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.599+0000 7f712be00640 1 -- 172.21.15.7:0/517251792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71240319d0 con 0x7f712c0b7960 2024-04-25T11:18:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.600+0000 7f7133000640 1 -- 172.21.15.7:0/517251792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f712c0b7960 msgr2=0x7f712c0d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.600+0000 7f7133000640 1 --2- 172.21.15.7:0/517251792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f712c0b7960 0x7f712c0d17b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f712c0d3720 tx=0x7f712402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.600+0000 7f7133000640 1 -- 172.21.15.7:0/517251792 shutdown_connections 2024-04-25T11:18:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.600+0000 7f7133000640 1 --2- 172.21.15.7:0/517251792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f712c0b7960 0x7f712c0d17b0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.600+0000 7f7133000640 1 --2- 172.21.15.7:0/517251792 >> v2:172.21.15.87:3300/0 conn(0x7f712c0b3960 0x7f712c0b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.600+0000 7f7133000640 1 --2- 172.21.15.7:0/517251792 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f712c06ed90 0x7f712c1600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.600+0000 7f7133000640 1 -- 172.21.15.7:0/517251792 >> 172.21.15.7:0/517251792 conn(0x7f712c09b540 msgr2=0x7f712c09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.600+0000 7f7133000640 1 -- 172.21.15.7:0/517251792 shutdown_connections 2024-04-25T11:18:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.600+0000 7f7133000640 1 -- 172.21.15.7:0/517251792 wait complete. 2024-04-25T11:18:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.602+0000 7f7133000640 1 Processor -- start 2024-04-25T11:18:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.603+0000 7f7133000640 1 -- start start 2024-04-25T11:18:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.604+0000 7f7133000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f712c06ed90 0x7f712c14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.604+0000 7f7133000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f712c0b3960 0x7f712c150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.604+0000 7f7133000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f712c0b7960 0x7f712c0cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.604+0000 7f7133000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f712c0b3d70 con 0x7f712c0b7960 2024-04-25T11:18:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.604+0000 7f7133000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f712c0cfe10 con 0x7f712c06ed90 2024-04-25T11:18:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.604+0000 7f7133000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f712c0b3ef0 con 0x7f712c0b3960 2024-04-25T11:18:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.604+0000 7f7132600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f712c0b7960 0x7f712c0cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.605+0000 7f7132600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f712c0b7960 0x7f712c0cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50966/0 (socket says 172.21.15.7:50966) 2024-04-25T11:18:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.605+0000 7f7132600640 1 -- 172.21.15.7:0/942547050 learned_addr learned my addr 172.21.15.7:0/942547050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.605+0000 7f7131200640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f712c0b3960 0x7f712c150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.605+0000 7f7132600640 1 -- 172.21.15.7:0/942547050 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f712c0b3960 msgr2=0x7f712c150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.605+0000 7f7131c00640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f712c06ed90 0x7f712c14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.605+0000 7f7132600640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f712c0b3960 0x7f712c150ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.605+0000 7f7132600640 1 -- 172.21.15.7:0/942547050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f712c06ed90 msgr2=0x7f712c14d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.605+0000 7f7132600640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f712c06ed90 0x7f712c14d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.605+0000 7f7132600640 1 -- 172.21.15.7:0/942547050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712c151020 con 0x7f712c0b7960 2024-04-25T11:18:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.605+0000 7f7132600640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f712c0b7960 0x7f712c0cde00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f712402ac90 tx=0x7f7124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.606+0000 7f7129600640 1 -- 172.21.15.7:0/942547050 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712403f020 con 0x7f712c0b7960 2024-04-25T11:18:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.606+0000 7f7129600640 1 -- 172.21.15.7:0/942547050 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124037040 con 0x7f712c0b7960 2024-04-25T11:18:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.606+0000 7f7129600640 1 -- 172.21.15.7:0/942547050 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124036070 con 0x7f712c0b7960 2024-04-25T11:18:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.606+0000 7f7133000640 1 -- 172.21.15.7:0/942547050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712c1f6410 con 0x7f712c0b7960 2024-04-25T11:18:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.606+0000 7f7133000640 1 -- 172.21.15.7:0/942547050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f712c1f69d0 con 0x7f712c0b7960 2024-04-25T11:18:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.607+0000 7f7133000640 1 -- 172.21.15.7:0/942547050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712c050740 con 0x7f712c0b7960 2024-04-25T11:18:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.608+0000 7f7129600640 1 -- 172.21.15.7:0/942547050 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7124042020 con 0x7f712c0b7960 2024-04-25T11:18:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.609+0000 7f7129600640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f710c0744e0 0x7f710c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.609+0000 7f7129600640 1 -- 172.21.15.7:0/942547050 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f71240b5d00 con 0x7f712c0b7960 2024-04-25T11:18:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.610+0000 7f7131c00640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f710c0744e0 0x7f710c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.610+0000 7f7131c00640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f710c0744e0 0x7f710c0769f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f711c006fd0 tx=0x7f711c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.614+0000 7f7129600640 1 -- 172.21.15.7:0/942547050 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7124049020 con 0x7f712c0b7960 2024-04-25T11:18:45.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.830+0000 7f7133000640 1 -- 172.21.15.7:0/942547050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f712c1f61a0 con 0x7f712c0b7960 2024-04-25T11:18:45.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.831+0000 7f7129600640 1 -- 172.21.15.7:0/942547050 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v94) ==== 80+0+0 (secure 0 0 0) 0x7f71240801f0 con 0x7f712c0b7960 2024-04-25T11:18:45.830 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:18:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.835+0000 7f7133000640 1 -- 172.21.15.7:0/942547050 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f710c0744e0 msgr2=0x7f710c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.835+0000 7f7133000640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f710c0744e0 0x7f710c0769f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f711c006fd0 tx=0x7f711c008040 comp rx=0 tx=0).stop 2024-04-25T11:18:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.835+0000 7f7133000640 1 -- 172.21.15.7:0/942547050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f712c0b7960 msgr2=0x7f712c0cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.835+0000 7f7133000640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f712c0b7960 0x7f712c0cde00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f712402ac90 tx=0x7f7124004a40 comp rx=0 tx=0).stop 2024-04-25T11:18:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.835+0000 7f7133000640 1 -- 172.21.15.7:0/942547050 shutdown_connections 2024-04-25T11:18:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.836+0000 7f7133000640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f710c0744e0 0x7f710c0769f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.836+0000 7f7133000640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f712c0b7960 0x7f712c0cde00 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.836+0000 7f7133000640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f712c0b3960 0x7f712c150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.836+0000 7f7133000640 1 --2- 172.21.15.7:0/942547050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f712c06ed90 0x7f712c14d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.836+0000 7f7133000640 1 -- 172.21.15.7:0/942547050 >> 172.21.15.7:0/942547050 conn(0x7f712c09b540 msgr2=0x7f712c070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.836+0000 7f7133000640 1 -- 172.21.15.7:0/942547050 shutdown_connections 2024-04-25T11:18:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:45.836+0000 7f7133000640 1 -- 172.21.15.7:0/942547050 wait complete. 2024-04-25T11:18:45.860 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:18:45.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:18:45.861 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T11:18:45.961 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:18:45.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:18:45.961 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:18:46.014 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:18:46.014 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:18:46.014 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:18:46.073 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:18:46.073 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:18:46.073 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:18:46.073 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:18:46.130 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:18:46.130 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:18:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.423+0000 7f3552800640 1 Processor -- start 2024-04-25T11:18:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.423+0000 7f3552800640 1 -- start start 2024-04-25T11:18:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.426+0000 7f3552800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f354c0b4aa0 0x7f354c0ae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.426+0000 7f3552800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f354c164710 0x7f354c0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.426+0000 7f3552800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f354c164c80 con 0x7f354c0b0da0 2024-04-25T11:18:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.426+0000 7f3552800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f354c1655b0 con 0x7f354c0b4aa0 2024-04-25T11:18:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.426+0000 7f3552800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f354c165a40 con 0x7f354c164710 2024-04-25T11:18:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.426+0000 7f354be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f354c0b0da0 0x7f354c0b4f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47850/0 (socket says 172.21.15.7:47850) 2024-04-25T11:18:46.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.426+0000 7f354be00640 1 -- 172.21.15.7:0/4073783637 learned_addr learned my addr 172.21.15.7:0/4073783637 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354b400640 1 --2- 172.21.15.7:0/4073783637 >> v2:172.21.15.87:3300/0 conn(0x7f354c0b4aa0 0x7f354c0ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354aa00640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f354c164710 0x7f354c0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4093900359 0 0) 0x7f354c164c80 con 0x7f354c0b0da0 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3530003520 con 0x7f354c0b0da0 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4144495438 0 0) 0x7f3530003520 con 0x7f354c0b0da0 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f354c164710 msgr2=0x7f354c0b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354a000640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f354c164710 0x7f354c0b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 >> v2:172.21.15.87:3300/0 conn(0x7f354c0b4aa0 msgr2=0x7f354c0ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354a000640 1 --2- 172.21.15.7:0/4073783637 >> v2:172.21.15.87:3300/0 conn(0x7f354c0b4aa0 0x7f354c0ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c164f60 con 0x7f354c0b0da0 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f35440019f0 con 0x7f354c0b0da0 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 >> v1:172.21.15.7:6789/0 conn(0x7f354c0b0da0 legacy=0x7f354c0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.427+0000 7f354a000640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35300039b0 0x7f3530003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354a000640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3530004800 0x7f3530004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354a000640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f354c164710 0x7f3530005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:46.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f354c1655b0 con 0x7f3530004800 2024-04-25T11:18:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f354c165a40 con 0x7f35300039b0 2024-04-25T11:18:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f353000b9f0 con 0x7f354c164710 2024-04-25T11:18:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354aa00640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3530004800 0x7f3530004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354b400640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35300039b0 0x7f3530003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354be00640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f354c164710 0x7f3530005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354aa00640 1 -- 172.21.15.7:0/4073783637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f354c164710 msgr2=0x7f3530005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354aa00640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f354c164710 0x7f3530005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354aa00640 1 -- 172.21.15.7:0/4073783637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35300039b0 msgr2=0x7f3530003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354aa00640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35300039b0 0x7f3530003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:46.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.428+0000 7f354aa00640 1 -- 172.21.15.7:0/4073783637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3530003520 con 0x7f3530004800 2024-04-25T11:18:46.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.430+0000 7f354aa00640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3530004800 0x7f3530004c10 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3534009ac0 tx=0x7f3530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e06a15cb2a4e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:46.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.430+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534036070 con 0x7f3530004800 2024-04-25T11:18:46.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.430+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3534031400 con 0x7f3530004800 2024-04-25T11:18:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.430+0000 7f354a000640 1 -- 172.21.15.7:0/4073783637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534009690 con 0x7f3530004800 2024-04-25T11:18:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.431+0000 7f3552800640 1 -- 172.21.15.7:0/4073783637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3530004800 msgr2=0x7f3530004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.431+0000 7f3552800640 1 --2- 172.21.15.7:0/4073783637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3530004800 0x7f3530004c10 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3534009ac0 tx=0x7f3530005830 comp rx=0 tx=0).stop 2024-04-25T11:18:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.431+0000 7f354be00640 1 -- 172.21.15.7:0/4073783637 reap_dead start 2024-04-25T11:18:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.431+0000 7f3552800640 1 -- 172.21.15.7:0/4073783637 shutdown_connections 2024-04-25T11:18:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.431+0000 7f3552800640 1 -- 172.21.15.7:0/4073783637 >> 172.21.15.7:0/4073783637 conn(0x7f354c074200 msgr2=0x7f354c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.431+0000 7f3552800640 1 -- 172.21.15.7:0/4073783637 shutdown_connections 2024-04-25T11:18:46.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.431+0000 7f3552800640 1 -- 172.21.15.7:0/4073783637 wait complete. 2024-04-25T11:18:46.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.433+0000 7f3552800640 1 Processor -- start 2024-04-25T11:18:46.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.434+0000 7f3552800640 1 -- start start 2024-04-25T11:18:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.435+0000 7f3552800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3530004800 0x7f354c158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:46.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.435+0000 7f3552800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c09f140 0x7f354c15df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:46.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.435+0000 7f3552800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f354c15e480 0x7f354c15e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:46.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.435+0000 7f3552800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f354c06f300 con 0x7f354c09f140 2024-04-25T11:18:46.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.435+0000 7f3552800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f354c0af340 con 0x7f354c15e480 2024-04-25T11:18:46.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.435+0000 7f3552800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f354c0b11b0 con 0x7f3530004800 2024-04-25T11:18:46.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.436+0000 7f354aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c09f140 0x7f354c15df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:46.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.436+0000 7f354aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c09f140 0x7f354c15df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50986/0 (socket says 172.21.15.7:50986) 2024-04-25T11:18:46.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.436+0000 7f354aa00640 1 -- 172.21.15.7:0/2741436268 learned_addr learned my addr 172.21.15.7:0/2741436268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.436+0000 7f354b400640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3530004800 0x7f354c158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.436+0000 7f354be00640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f354c15e480 0x7f354c15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.436+0000 7f354aa00640 1 -- 172.21.15.7:0/2741436268 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3530004800 msgr2=0x7f354c158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.436+0000 7f354aa00640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3530004800 0x7f354c158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.436+0000 7f354aa00640 1 -- 172.21.15.7:0/2741436268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f354c15e480 msgr2=0x7f354c15e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.436+0000 7f354aa00640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f354c15e480 0x7f354c15e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.436+0000 7f354aa00640 1 -- 172.21.15.7:0/2741436268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c1fab10 con 0x7f354c09f140 2024-04-25T11:18:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.436+0000 7f354be00640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f354c15e480 0x7f354c15e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.437+0000 7f354aa00640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c09f140 0x7f354c15df40 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f3534009ac0 tx=0x7f3534004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.437+0000 7f3543400640 1 -- 172.21.15.7:0/2741436268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534040020 con 0x7f354c09f140 2024-04-25T11:18:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.437+0000 7f3543400640 1 -- 172.21.15.7:0/2741436268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3534003940 con 0x7f354c09f140 2024-04-25T11:18:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.437+0000 7f3543400640 1 -- 172.21.15.7:0/2741436268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534036040 con 0x7f354c09f140 2024-04-25T11:18:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.437+0000 7f3552800640 1 -- 172.21.15.7:0/2741436268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c15cea0 con 0x7f354c09f140 2024-04-25T11:18:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.437+0000 7f3552800640 1 -- 172.21.15.7:0/2741436268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c15d460 con 0x7f354c09f140 2024-04-25T11:18:46.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.438+0000 7f3552800640 1 -- 172.21.15.7:0/2741436268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354c06a100 con 0x7f354c09f140 2024-04-25T11:18:46.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.439+0000 7f3543400640 1 -- 172.21.15.7:0/2741436268 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3534043020 con 0x7f354c09f140 2024-04-25T11:18:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.440+0000 7f3543400640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35280744b0 0x7f35280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.441+0000 7f3543400640 1 -- 172.21.15.7:0/2741436268 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f35340b5680 con 0x7f354c09f140 2024-04-25T11:18:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.441+0000 7f354b400640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35280744b0 0x7f35280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:46.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.442+0000 7f354b400640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35280744b0 0x7f35280769c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f354c072200 tx=0x7f353c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.446+0000 7f3543400640 1 -- 172.21.15.7:0/2741436268 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f353404a070 con 0x7f354c09f140 2024-04-25T11:18:46.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.691+0000 7f3552800640 1 -- 172.21.15.7:0/2741436268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f354c165d00 con 0x7f354c09f140 2024-04-25T11:18:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.692+0000 7f3543400640 1 -- 172.21.15.7:0/2741436268 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5169 (secure 0 0 0) 0x7f353407fc90 con 0x7f354c09f140 2024-04-25T11:18:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:46.692 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/684015857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":684015857},{"type":"v1","addr":"172.21.15.7:6836","nonce":684015857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/493684223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":493684223},{"type":"v1","addr":"172.21.15.87:6835","nonce":493684223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6109,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/1959070928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1959070928},{"type":"v1","addr":"172.21.15.87:6833","nonce":1959070928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:16:01.820362+0000","modified":"2024-04-25T11:18:46.323260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":44,"addr":"172.21.15.7:6837/336889324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":336889324},{"type":"v1","addr":"172.21.15.7:6837","nonce":336889324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6960,"qdb_cluster":[6960]},"id":6}]} 2024-04-25T11:18:46.692 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-04-25T11:18:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 -- 172.21.15.7:0/2741436268 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35280744b0 msgr2=0x7f35280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35280744b0 0x7f35280769c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f354c072200 tx=0x7f353c008040 comp rx=0 tx=0).stop 2024-04-25T11:18:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 -- 172.21.15.7:0/2741436268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c09f140 msgr2=0x7f354c15df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c09f140 0x7f354c15df40 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f3534009ac0 tx=0x7f3534004730 comp rx=0 tx=0).stop 2024-04-25T11:18:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 -- 172.21.15.7:0/2741436268 shutdown_connections 2024-04-25T11:18:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35280744b0 0x7f35280769c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f354c15e480 0x7f354c15e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f354c09f140 0x7f354c15df40 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 --2- 172.21.15.7:0/2741436268 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3530004800 0x7f354c158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 -- 172.21.15.7:0/2741436268 >> 172.21.15.7:0/2741436268 conn(0x7f354c074200 msgr2=0x7f354c0b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 -- 172.21.15.7:0/2741436268 shutdown_connections 2024-04-25T11:18:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:46.695+0000 7f3552800640 1 -- 172.21.15.7:0/2741436268 wait complete. 2024-04-25T11:18:46.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:18:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.010+0000 7f848ac00640 1 Processor -- start 2024-04-25T11:18:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.010+0000 7f848ac00640 1 -- start start 2024-04-25T11:18:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.013+0000 7f848ac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84840b5220 0x7f848415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.013+0000 7f848ac00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84840af420 0x7f84840af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.013+0000 7f848ac00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84840cebb0 con 0x7f84840aed40 2024-04-25T11:18:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.013+0000 7f848ac00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f848415e1f0 con 0x7f84840af420 2024-04-25T11:18:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.013+0000 7f848ac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f84840afd70 con 0x7f84840b5220 2024-04-25T11:18:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.013+0000 7f848a200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84840af420 0x7f84840af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.013+0000 7f848a200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84840af420 0x7f84840af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50992/0 (socket says 172.21.15.7:50992) 2024-04-25T11:18:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.013+0000 7f848a200640 1 -- 172.21.15.7:0/1593676076 learned_addr learned my addr 172.21.15.7:0/1593676076 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:47.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.014+0000 7f8488e00640 1 --2- 172.21.15.7:0/1593676076 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84840b5220 0x7f848415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.014+0000 7f848a200640 1 -- 172.21.15.7:0/1593676076 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84840b5220 msgr2=0x7f848415dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:47.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.014+0000 7f848a200640 1 --2- 172.21.15.7:0/1593676076 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84840b5220 0x7f848415dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.014+0000 7f848a200640 1 -- 172.21.15.7:0/1593676076 >> v1:172.21.15.87:6789/0 conn(0x7f84840aed40 legacy=0x7f84840ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:47.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.014+0000 7f848a200640 1 -- 172.21.15.7:0/1593676076 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84840b64b0 con 0x7f84840af420 2024-04-25T11:18:47.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.014+0000 7f8488e00640 1 --2- 172.21.15.7:0/1593676076 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84840b5220 0x7f848415dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:47.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.015+0000 7f848a200640 1 --2- 172.21.15.7:0/1593676076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84840af420 0x7f84840af830 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f847c009870 tx=0x7f847c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1513f15d4d1836 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:47.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.015+0000 7f8483e00640 1 -- 172.21.15.7:0/1593676076 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c037070 con 0x7f84840af420 2024-04-25T11:18:47.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.015+0000 7f8483e00640 1 -- 172.21.15.7:0/1593676076 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847c032440 con 0x7f84840af420 2024-04-25T11:18:47.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.015+0000 7f8483e00640 1 -- 172.21.15.7:0/1593676076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c032810 con 0x7f84840af420 2024-04-25T11:18:47.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.016+0000 7f848ac00640 1 -- 172.21.15.7:0/1593676076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84840af420 msgr2=0x7f84840af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:47.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.016+0000 7f848ac00640 1 --2- 172.21.15.7:0/1593676076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84840af420 0x7f84840af830 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f847c009870 tx=0x7f847c02a210 comp rx=0 tx=0).stop 2024-04-25T11:18:47.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.016+0000 7f848ac00640 1 -- 172.21.15.7:0/1593676076 shutdown_connections 2024-04-25T11:18:47.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.016+0000 7f848ac00640 1 --2- 172.21.15.7:0/1593676076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84840af420 0x7f84840af830 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.016+0000 7f848ac00640 1 --2- 172.21.15.7:0/1593676076 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84840b5220 0x7f848415dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.016+0000 7f848ac00640 1 -- 172.21.15.7:0/1593676076 >> 172.21.15.7:0/1593676076 conn(0x7f84840b3820 msgr2=0x7f84840b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:47.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.016+0000 7f848ac00640 1 -- 172.21.15.7:0/1593676076 shutdown_connections 2024-04-25T11:18:47.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.016+0000 7f848ac00640 1 -- 172.21.15.7:0/1593676076 wait complete. 2024-04-25T11:18:47.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.019+0000 7f848ac00640 1 Processor -- start 2024-04-25T11:18:47.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.019+0000 7f848ac00640 1 -- start start 2024-04-25T11:18:47.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.020+0000 7f848ac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84840aed40 0x7f848415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.020+0000 7f848ac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84840af420 0x7f848414ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f848ac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84840b5220 0x7f8484160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f848ac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f848406cbd0 con 0x7f84840af420 2024-04-25T11:18:47.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f848ac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f848406c8f0 con 0x7f84840aed40 2024-04-25T11:18:47.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f848ac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f848406ceb0 con 0x7f84840b5220 2024-04-25T11:18:47.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f8488e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84840af420 0x7f848414ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f8488e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84840af420 0x7f848414ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51004/0 (socket says 172.21.15.7:51004) 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f8488e00640 1 -- 172.21.15.7:0/2796070423 learned_addr learned my addr 172.21.15.7:0/2796070423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f848a200640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84840b5220 0x7f8484160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f8489800640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84840aed40 0x7f848415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f8488e00640 1 -- 172.21.15.7:0/2796070423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84840b5220 msgr2=0x7f8484160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f8488e00640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84840b5220 0x7f8484160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.021+0000 7f8488e00640 1 -- 172.21.15.7:0/2796070423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84840aed40 msgr2=0x7f848415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.022+0000 7f8488e00640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84840aed40 0x7f848415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.022+0000 7f8488e00640 1 -- 172.21.15.7:0/2796070423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848415f2c0 con 0x7f84840af420 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.022+0000 7f8488e00640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84840af420 0x7f848414ccc0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f847000c820 tx=0x7f847000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.022+0000 7f8481600640 1 -- 172.21.15.7:0/2796070423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470015070 con 0x7f84840af420 2024-04-25T11:18:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.022+0000 7f8481600640 1 -- 172.21.15.7:0/2796070423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8470007a30 con 0x7f84840af420 2024-04-25T11:18:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.022+0000 7f8481600640 1 -- 172.21.15.7:0/2796070423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470007de0 con 0x7f84840af420 2024-04-25T11:18:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.022+0000 7f848ac00640 1 -- 172.21.15.7:0/2796070423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84841e7280 con 0x7f84840af420 2024-04-25T11:18:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.022+0000 7f848ac00640 1 -- 172.21.15.7:0/2796070423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84841e7520 con 0x7f84840af420 2024-04-25T11:18:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.024+0000 7f848ac00640 1 -- 172.21.15.7:0/2796070423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f848415e570 con 0x7f84840af420 2024-04-25T11:18:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.025+0000 7f8481600640 1 -- 172.21.15.7:0/2796070423 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8470018070 con 0x7f84840af420 2024-04-25T11:18:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.026+0000 7f8481600640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84640744b0 0x7f84640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.026+0000 7f8481600640 1 -- 172.21.15.7:0/2796070423 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f84700953f0 con 0x7f84840af420 2024-04-25T11:18:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.026+0000 7f8489800640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84640744b0 0x7f84640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.027+0000 7f8489800640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84640744b0 0x7f84640769c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8484071200 tx=0x7f84780094d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.031+0000 7f8481600640 1 -- 172.21.15.7:0/2796070423 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f847005f8e0 con 0x7f84840af420 2024-04-25T11:18:47.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.242+0000 7f848ac00640 1 -- 172.21.15.7:0/2796070423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8484050740 con 0x7f84840af420 2024-04-25T11:18:47.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.245+0000 7f8481600640 1 -- 172.21.15.7:0/2796070423 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16983 (secure 0 0 0) 0x7f847005f1e0 con 0x7f84840af420 2024-04-25T11:18:47.244 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:47.244 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:17:04.183733+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:59.999922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T11:16:00.796013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":5}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:18:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 -- 172.21.15.7:0/2796070423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84640744b0 msgr2=0x7f84640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84640744b0 0x7f84640769c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8484071200 tx=0x7f84780094d0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 -- 172.21.15.7:0/2796070423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84840af420 msgr2=0x7f848414ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:47.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84840af420 0x7f848414ccc0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f847000c820 tx=0x7f847000cce0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 -- 172.21.15.7:0/2796070423 shutdown_connections 2024-04-25T11:18:47.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f84640744b0 0x7f84640769c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84840b5220 0x7f8484160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84840af420 0x7f848414ccc0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 --2- 172.21.15.7:0/2796070423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84840aed40 0x7f848415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 -- 172.21.15.7:0/2796070423 >> 172.21.15.7:0/2796070423 conn(0x7f84840b3820 msgr2=0x7f84840b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:47.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 -- 172.21.15.7:0/2796070423 shutdown_connections 2024-04-25T11:18:47.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.247+0000 7f848ac00640 1 -- 172.21.15.7:0/2796070423 wait complete. 2024-04-25T11:18:47.272 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:15:59.999922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:16:00.796013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:18:47.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:18:47.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.561+0000 7fc49c200640 1 Processor -- start 2024-04-25T11:18:47.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.561+0000 7fc49c200640 1 -- start start 2024-04-25T11:18:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.564+0000 7fc49c200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc49415dfb0 0x7fc4940b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.564+0000 7fc49c200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc49406fab0 0x7fc4940aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.564+0000 7fc49c200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc49415e3c0 con 0x7fc4940d0b40 2024-04-25T11:18:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.564+0000 7fc49c200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc4940b09b0 con 0x7fc49406fab0 2024-04-25T11:18:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.564+0000 7fc49c200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc4940cf800 con 0x7fc49415dfb0 2024-04-25T11:18:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.564+0000 7fc49a200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc49406fab0 0x7fc4940aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.564+0000 7fc49a200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc49406fab0 0x7fc4940aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51016/0 (socket says 172.21.15.7:51016) 2024-04-25T11:18:47.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.564+0000 7fc49a200640 1 -- 172.21.15.7:0/2400677378 learned_addr learned my addr 172.21.15.7:0/2400677378 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:47.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.564+0000 7fc499800640 1 --2- 172.21.15.7:0/2400677378 >> v2:172.21.15.87:3300/0 conn(0x7fc49415dfb0 0x7fc4940b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.565+0000 7fc49a200640 1 -- 172.21.15.7:0/2400677378 >> v2:172.21.15.87:3300/0 conn(0x7fc49415dfb0 msgr2=0x7fc4940b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:47.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.565+0000 7fc49a200640 1 --2- 172.21.15.7:0/2400677378 >> v2:172.21.15.87:3300/0 conn(0x7fc49415dfb0 0x7fc4940b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.565+0000 7fc49a200640 1 -- 172.21.15.7:0/2400677378 >> v1:172.21.15.87:6789/0 conn(0x7fc4940d0b40 legacy=0x7fc49406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:47.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.565+0000 7fc49a200640 1 -- 172.21.15.7:0/2400677378 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4940cfaa0 con 0x7fc49406fab0 2024-04-25T11:18:47.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.566+0000 7fc49a200640 1 --2- 172.21.15.7:0/2400677378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc49406fab0 0x7fc4940aea90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fc494072b10 tx=0x7fc48c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d7b8e951fec17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:47.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.566+0000 7fc493e00640 1 -- 172.21.15.7:0/2400677378 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c036070 con 0x7fc49406fab0 2024-04-25T11:18:47.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.566+0000 7fc493e00640 1 -- 172.21.15.7:0/2400677378 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48c031600 con 0x7fc49406fab0 2024-04-25T11:18:47.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.566+0000 7fc493e00640 1 -- 172.21.15.7:0/2400677378 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0319d0 con 0x7fc49406fab0 2024-04-25T11:18:47.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.567+0000 7fc49c200640 1 -- 172.21.15.7:0/2400677378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc49406fab0 msgr2=0x7fc4940aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:47.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.567+0000 7fc49c200640 1 --2- 172.21.15.7:0/2400677378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc49406fab0 0x7fc4940aea90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fc494072b10 tx=0x7fc48c02a6f0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.567+0000 7fc49c200640 1 -- 172.21.15.7:0/2400677378 shutdown_connections 2024-04-25T11:18:47.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.567+0000 7fc49c200640 1 --2- 172.21.15.7:0/2400677378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc49406fab0 0x7fc4940aea90 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.567+0000 7fc49c200640 1 --2- 172.21.15.7:0/2400677378 >> v2:172.21.15.87:3300/0 conn(0x7fc49415dfb0 0x7fc4940b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.567+0000 7fc49c200640 1 -- 172.21.15.7:0/2400677378 >> 172.21.15.7:0/2400677378 conn(0x7fc4940742c0 msgr2=0x7fc49406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:47.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.567+0000 7fc49c200640 1 -- 172.21.15.7:0/2400677378 shutdown_connections 2024-04-25T11:18:47.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.567+0000 7fc49c200640 1 -- 172.21.15.7:0/2400677378 wait complete. 2024-04-25T11:18:47.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.570+0000 7fc49c200640 1 Processor -- start 2024-04-25T11:18:47.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.570+0000 7fc49c200640 1 -- start start 2024-04-25T11:18:47.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.571+0000 7fc49c200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc49406fab0 0x7fc49415d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.571+0000 7fc49c200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4940d0b40 0x7fc494152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.571+0000 7fc49c200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc49415dfb0 0x7fc4940b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.571+0000 7fc49c200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc4940d0340 con 0x7fc49415dfb0 2024-04-25T11:18:47.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.571+0000 7fc49c200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc4940d0060 con 0x7fc49406fab0 2024-04-25T11:18:47.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.571+0000 7fc49c200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc4940d0620 con 0x7fc4940d0b40 2024-04-25T11:18:47.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.572+0000 7fc49a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc49415dfb0 0x7fc4940b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.572+0000 7fc49a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc49415dfb0 0x7fc4940b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51032/0 (socket says 172.21.15.7:51032) 2024-04-25T11:18:47.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.572+0000 7fc49a200640 1 -- 172.21.15.7:0/812519836 learned_addr learned my addr 172.21.15.7:0/812519836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:47.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.572+0000 7fc498e00640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4940d0b40 0x7fc494152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.572+0000 7fc499800640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc49406fab0 0x7fc49415d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.572+0000 7fc49a200640 1 -- 172.21.15.7:0/812519836 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4940d0b40 msgr2=0x7fc494152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.572+0000 7fc49a200640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4940d0b40 0x7fc494152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.572+0000 7fc49a200640 1 -- 172.21.15.7:0/812519836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc49406fab0 msgr2=0x7fc49415d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.572+0000 7fc49a200640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc49406fab0 0x7fc49415d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.572+0000 7fc49a200640 1 -- 172.21.15.7:0/812519836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49415b7d0 con 0x7fc49415dfb0 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.572+0000 7fc499800640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc49406fab0 0x7fc49415d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.573+0000 7fc49a200640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc49415dfb0 0x7fc4940b53f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009f40 tx=0x7fc48c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.573+0000 7fc491600640 1 -- 172.21.15.7:0/812519836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c036050 con 0x7fc49415dfb0 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.573+0000 7fc491600640 1 -- 172.21.15.7:0/812519836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48c042070 con 0x7fc49415dfb0 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.573+0000 7fc49c200640 1 -- 172.21.15.7:0/812519836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49415bab0 con 0x7fc49415dfb0 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.573+0000 7fc491600640 1 -- 172.21.15.7:0/812519836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c009510 con 0x7fc49415dfb0 2024-04-25T11:18:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.573+0000 7fc49c200640 1 -- 172.21.15.7:0/812519836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49415c070 con 0x7fc49415dfb0 2024-04-25T11:18:47.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.575+0000 7fc49c200640 1 -- 172.21.15.7:0/812519836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc494050740 con 0x7fc49415dfb0 2024-04-25T11:18:47.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.576+0000 7fc491600640 1 -- 172.21.15.7:0/812519836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc48c007bd0 con 0x7fc49415dfb0 2024-04-25T11:18:47.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.577+0000 7fc491600640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4740744b0 0x7fc4740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:47.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.577+0000 7fc491600640 1 -- 172.21.15.7:0/812519836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7fc48c0b63f0 con 0x7fc49415dfb0 2024-04-25T11:18:47.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.577+0000 7fc499800640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4740744b0 0x7fc4740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:47.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.578+0000 7fc499800640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4740744b0 0x7fc4740769c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc488006fd0 tx=0x7fc488008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:47.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.582+0000 7fc491600640 1 -- 172.21.15.7:0/812519836 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc48c0808e0 con 0x7fc49415dfb0 2024-04-25T11:18:47.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.791+0000 7fc49c200640 1 -- 172.21.15.7:0/812519836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc494070fb0 con 0x7fc49415dfb0 2024-04-25T11:18:47.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.792+0000 7fc491600640 1 -- 172.21.15.7:0/812519836 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fc48c0801e0 con 0x7fc49415dfb0 2024-04-25T11:18:47.791 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:18:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 -- 172.21.15.7:0/812519836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4740744b0 msgr2=0x7fc4740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4740744b0 0x7fc4740769c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc488006fd0 tx=0x7fc488008040 comp rx=0 tx=0).stop 2024-04-25T11:18:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 -- 172.21.15.7:0/812519836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc49415dfb0 msgr2=0x7fc4940b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc49415dfb0 0x7fc4940b53f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009f40 tx=0x7fc48c004a40 comp rx=0 tx=0).stop 2024-04-25T11:18:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 -- 172.21.15.7:0/812519836 shutdown_connections 2024-04-25T11:18:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc4740744b0 0x7fc4740769c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc49415dfb0 0x7fc4940b53f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4940d0b40 0x7fc494152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 --2- 172.21.15.7:0/812519836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc49406fab0 0x7fc49415d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 -- 172.21.15.7:0/812519836 >> 172.21.15.7:0/812519836 conn(0x7fc4940742c0 msgr2=0x7fc49406cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 -- 172.21.15.7:0/812519836 shutdown_connections 2024-04-25T11:18:47.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:47.796+0000 7fc49c200640 1 -- 172.21.15.7:0/812519836 wait complete. 2024-04-25T11:18:47.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:18:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.115+0000 7fb7e9c00640 1 Processor -- start 2024-04-25T11:18:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.116+0000 7fb7e9c00640 1 -- start start 2024-04-25T11:18:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.118+0000 7fb7e9c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7e40b0470 0x7fb7e406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.118+0000 7fb7e9c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e406f8e0 0x7fb7e40aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.118+0000 7fb7e9c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7e415e3c0 con 0x7fb7e415dfb0 2024-04-25T11:18:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.118+0000 7fb7e9c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb7e40cf800 con 0x7fb7e40b0470 2024-04-25T11:18:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.118+0000 7fb7e9c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e40d1250 con 0x7fb7e406f8e0 2024-04-25T11:18:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.119+0000 7fb7dbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7e40b0470 0x7fb7e406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.119+0000 7fb7dbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7e40b0470 0x7fb7e406f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51034/0 (socket says 172.21.15.7:51034) 2024-04-25T11:18:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.119+0000 7fb7dbe00640 1 -- 172.21.15.7:0/3236762817 learned_addr learned my addr 172.21.15.7:0/3236762817 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.119+0000 7fb7e3400640 1 --2- 172.21.15.7:0/3236762817 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e406f8e0 0x7fb7e40aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.119+0000 7fb7dbe00640 1 -- 172.21.15.7:0/3236762817 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e406f8e0 msgr2=0x7fb7e40aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.119+0000 7fb7dbe00640 1 --2- 172.21.15.7:0/3236762817 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e406f8e0 0x7fb7e40aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.119+0000 7fb7dbe00640 1 -- 172.21.15.7:0/3236762817 >> v1:172.21.15.87:6789/0 conn(0x7fb7e415dfb0 legacy=0x7fb7e40d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.119+0000 7fb7dbe00640 1 -- 172.21.15.7:0/3236762817 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e40cfaa0 con 0x7fb7e40b0470 2024-04-25T11:18:48.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.121+0000 7fb7dbe00640 1 --2- 172.21.15.7:0/3236762817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7e40b0470 0x7fb7e406f3a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009870 tx=0x7fb7cc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294ffbba5eefe116 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:48.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.121+0000 7fb7e2000640 1 -- 172.21.15.7:0/3236762817 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc037070 con 0x7fb7e40b0470 2024-04-25T11:18:48.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.121+0000 7fb7e2000640 1 -- 172.21.15.7:0/3236762817 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7cc032440 con 0x7fb7e40b0470 2024-04-25T11:18:48.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.121+0000 7fb7e2000640 1 -- 172.21.15.7:0/3236762817 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc032810 con 0x7fb7e40b0470 2024-04-25T11:18:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.122+0000 7fb7e9c00640 1 -- 172.21.15.7:0/3236762817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7e40b0470 msgr2=0x7fb7e406f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.122+0000 7fb7e9c00640 1 --2- 172.21.15.7:0/3236762817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7e40b0470 0x7fb7e406f3a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc009870 tx=0x7fb7cc02a250 comp rx=0 tx=0).stop 2024-04-25T11:18:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.122+0000 7fb7e9c00640 1 -- 172.21.15.7:0/3236762817 shutdown_connections 2024-04-25T11:18:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.122+0000 7fb7e9c00640 1 --2- 172.21.15.7:0/3236762817 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e406f8e0 0x7fb7e40aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.122+0000 7fb7e9c00640 1 --2- 172.21.15.7:0/3236762817 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7e40b0470 0x7fb7e406f3a0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.122+0000 7fb7e9c00640 1 -- 172.21.15.7:0/3236762817 >> 172.21.15.7:0/3236762817 conn(0x7fb7e40742c0 msgr2=0x7fb7e406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.122+0000 7fb7e9c00640 1 -- 172.21.15.7:0/3236762817 shutdown_connections 2024-04-25T11:18:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.122+0000 7fb7e9c00640 1 -- 172.21.15.7:0/3236762817 wait complete. 2024-04-25T11:18:48.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.124+0000 7fb7e9c00640 1 Processor -- start 2024-04-25T11:18:48.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.125+0000 7fb7e9c00640 1 -- start start 2024-04-25T11:18:48.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.126+0000 7fb7e9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7e406f8e0 0x7fb7e4144f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.126+0000 7fb7e9c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e40b0470 0x7fb7e4154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.126+0000 7fb7e9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7e415dfb0 0x7fb7e4154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.126+0000 7fb7e9c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e40d0340 con 0x7fb7e415dfb0 2024-04-25T11:18:48.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.126+0000 7fb7e9c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e40d0060 con 0x7fb7e406f8e0 2024-04-25T11:18:48.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.126+0000 7fb7e9c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e40d0620 con 0x7fb7e40b0470 2024-04-25T11:18:48.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.127+0000 7fb7e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7e415dfb0 0x7fb7e4154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.127+0000 7fb7e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7e415dfb0 0x7fb7e4154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51040/0 (socket says 172.21.15.7:51040) 2024-04-25T11:18:48.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.127+0000 7fb7e3400640 1 -- 172.21.15.7:0/398931779 learned_addr learned my addr 172.21.15.7:0/398931779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:48.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.127+0000 7fb7dbe00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e40b0470 0x7fb7e4154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.127+0000 7fb7e2a00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7e406f8e0 0x7fb7e4144f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.127+0000 7fb7e3400640 1 -- 172.21.15.7:0/398931779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e40b0470 msgr2=0x7fb7e4154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:48.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.127+0000 7fb7e3400640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e40b0470 0x7fb7e4154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.127+0000 7fb7e3400640 1 -- 172.21.15.7:0/398931779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7e406f8e0 msgr2=0x7fb7e4144f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.127+0000 7fb7e3400640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7e406f8e0 0x7fb7e4144f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.127+0000 7fb7e3400640 1 -- 172.21.15.7:0/398931779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e4153360 con 0x7fb7e415dfb0 2024-04-25T11:18:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.127+0000 7fb7e2a00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7e406f8e0 0x7fb7e4144f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.128+0000 7fb7e3400640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7e415dfb0 0x7fb7e4154940 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc007e60 tx=0x7fb7dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.128+0000 7fb7daa00640 1 -- 172.21.15.7:0/398931779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc014420 con 0x7fb7e415dfb0 2024-04-25T11:18:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.128+0000 7fb7daa00640 1 -- 172.21.15.7:0/398931779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc019070 con 0x7fb7e415dfb0 2024-04-25T11:18:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.128+0000 7fb7daa00640 1 -- 172.21.15.7:0/398931779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc014820 con 0x7fb7e415dfb0 2024-04-25T11:18:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.128+0000 7fb7e9c00640 1 -- 172.21.15.7:0/398931779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e4153640 con 0x7fb7e415dfb0 2024-04-25T11:18:48.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.128+0000 7fb7e9c00640 1 -- 172.21.15.7:0/398931779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e4153c00 con 0x7fb7e415dfb0 2024-04-25T11:18:48.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.129+0000 7fb7e9c00640 1 -- 172.21.15.7:0/398931779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e41530d0 con 0x7fb7e415dfb0 2024-04-25T11:18:48.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.131+0000 7fb7daa00640 1 -- 172.21.15.7:0/398931779 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb7dc0149c0 con 0x7fb7e415dfb0 2024-04-25T11:18:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.132+0000 7fb7daa00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7c00744b0 0x7fb7c00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.132+0000 7fb7daa00640 1 -- 172.21.15.7:0/398931779 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7fb7dc095cd0 con 0x7fb7e415dfb0 2024-04-25T11:18:48.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.132+0000 7fb7e2a00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7c00744b0 0x7fb7c00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.133+0000 7fb7e2a00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7c00744b0 0x7fb7c00769c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4000f60 tx=0x7fb7d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:48.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.137+0000 7fb7daa00640 1 -- 172.21.15.7:0/398931779 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7dc05fc00 con 0x7fb7e415dfb0 2024-04-25T11:18:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.340+0000 7fb7e9c00640 1 -- 172.21.15.7:0/398931779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb7e4050740 con 0x7fb7e415dfb0 2024-04-25T11:18:48.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.341+0000 7fb7daa00640 1 -- 172.21.15.7:0/398931779 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fb7dc063f00 con 0x7fb7e415dfb0 2024-04-25T11:18:48.340 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:18:48.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 -- 172.21.15.7:0/398931779 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7c00744b0 msgr2=0x7fb7c00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7c00744b0 0x7fb7c00769c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb7d4000f60 tx=0x7fb7d4009040 comp rx=0 tx=0).stop 2024-04-25T11:18:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 -- 172.21.15.7:0/398931779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7e415dfb0 msgr2=0x7fb7e4154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7e415dfb0 0x7fb7e4154940 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc007e60 tx=0x7fb7dc00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 -- 172.21.15.7:0/398931779 shutdown_connections 2024-04-25T11:18:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb7c00744b0 0x7fb7c00769c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7e415dfb0 0x7fb7e4154940 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e40b0470 0x7fb7e4154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 --2- 172.21.15.7:0/398931779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7e406f8e0 0x7fb7e4144f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 -- 172.21.15.7:0/398931779 >> 172.21.15.7:0/398931779 conn(0x7fb7e40742c0 msgr2=0x7fb7e406cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 -- 172.21.15.7:0/398931779 shutdown_connections 2024-04-25T11:18:48.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.344+0000 7fb7e9c00640 1 -- 172.21.15.7:0/398931779 wait complete. 2024-04-25T11:18:48.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:18:48.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.665+0000 7f0f7d000640 1 Processor -- start 2024-04-25T11:18:48.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.666+0000 7f0f7d000640 1 -- start start 2024-04-25T11:18:48.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.668+0000 7f0f7d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f7815dfb0 0x7f0f780b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.668+0000 7f0f7d000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f780d0b40 0x7f0f7806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.668+0000 7f0f7d000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0f780d0f50 0x7f0f7806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.668+0000 7f0f7d000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0f780b09b0 con 0x7f0f780d0b40 2024-04-25T11:18:48.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.668+0000 7f0f7d000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0f7815e3c0 con 0x7f0f780d0f50 2024-04-25T11:18:48.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.668+0000 7f0f7d000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0f780cf800 con 0x7f0f7815dfb0 2024-04-25T11:18:48.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.669+0000 7f0f75400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f780d0b40 0x7f0f7806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.669+0000 7f0f75400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f780d0b40 0x7f0f7806f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51044/0 (socket says 172.21.15.7:51044) 2024-04-25T11:18:48.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.669+0000 7f0f75400640 1 -- 172.21.15.7:0/2508495627 learned_addr learned my addr 172.21.15.7:0/2508495627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:48.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.669+0000 7f0f75e00640 1 --2- 172.21.15.7:0/2508495627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f7815dfb0 0x7f0f780b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.669+0000 7f0f76800640 1 --2- 172.21.15.7:0/2508495627 >> v2:172.21.15.87:3300/0 conn(0x7f0f780d0f50 0x7f0f7806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.669+0000 7f0f75400640 1 -- 172.21.15.7:0/2508495627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f7815dfb0 msgr2=0x7f0f780b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:48.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.669+0000 7f0f75400640 1 --2- 172.21.15.7:0/2508495627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f7815dfb0 0x7f0f780b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.669+0000 7f0f75400640 1 -- 172.21.15.7:0/2508495627 >> v2:172.21.15.87:3300/0 conn(0x7f0f780d0f50 msgr2=0x7f0f7806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:48.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.669+0000 7f0f75400640 1 --2- 172.21.15.7:0/2508495627 >> v2:172.21.15.87:3300/0 conn(0x7f0f780d0f50 0x7f0f7806f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.669+0000 7f0f75400640 1 -- 172.21.15.7:0/2508495627 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f780cfaa0 con 0x7f0f780d0b40 2024-04-25T11:18:48.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.670+0000 7f0f75400640 1 --2- 172.21.15.7:0/2508495627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f780d0b40 0x7f0f7806f3a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f0f78161a10 tx=0x7f0f6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f802db15270c46d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:48.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.671+0000 7f0f74a00640 1 -- 172.21.15.7:0/2508495627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60037070 con 0x7f0f780d0b40 2024-04-25T11:18:48.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.671+0000 7f0f74a00640 1 -- 172.21.15.7:0/2508495627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f60032440 con 0x7f0f780d0b40 2024-04-25T11:18:48.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.671+0000 7f0f74a00640 1 -- 172.21.15.7:0/2508495627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60032810 con 0x7f0f780d0b40 2024-04-25T11:18:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.671+0000 7f0f7d000640 1 -- 172.21.15.7:0/2508495627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f780d0b40 msgr2=0x7f0f7806f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.671+0000 7f0f7d000640 1 --2- 172.21.15.7:0/2508495627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f780d0b40 0x7f0f7806f3a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f0f78161a10 tx=0x7f0f6002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.672+0000 7f0f7d000640 1 -- 172.21.15.7:0/2508495627 shutdown_connections 2024-04-25T11:18:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.672+0000 7f0f7d000640 1 --2- 172.21.15.7:0/2508495627 >> v2:172.21.15.87:3300/0 conn(0x7f0f780d0f50 0x7f0f7806f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.672+0000 7f0f7d000640 1 --2- 172.21.15.7:0/2508495627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f780d0b40 0x7f0f7806f3a0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.672+0000 7f0f7d000640 1 --2- 172.21.15.7:0/2508495627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f7815dfb0 0x7f0f780b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.672+0000 7f0f7d000640 1 -- 172.21.15.7:0/2508495627 >> 172.21.15.7:0/2508495627 conn(0x7f0f780742c0 msgr2=0x7f0f7806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.672+0000 7f0f7d000640 1 -- 172.21.15.7:0/2508495627 shutdown_connections 2024-04-25T11:18:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.672+0000 7f0f7d000640 1 -- 172.21.15.7:0/2508495627 wait complete. 2024-04-25T11:18:48.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.674+0000 7f0f7d000640 1 Processor -- start 2024-04-25T11:18:48.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.674+0000 7f0f7d000640 1 -- start start 2024-04-25T11:18:48.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.676+0000 7f0f7d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f780d0b40 0x7f0f7814a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.676+0000 7f0f7d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f780d0f50 0x7f0f7814d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.676+0000 7f0f7d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7815dfb0 0x7f0f7806c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.676+0000 7f0f7d000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f780d0340 con 0x7f0f7815dfb0 2024-04-25T11:18:48.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.676+0000 7f0f7d000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0f780d0060 con 0x7f0f780d0b40 2024-04-25T11:18:48.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.676+0000 7f0f7d000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0f780d0620 con 0x7f0f780d0f50 2024-04-25T11:18:48.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.677+0000 7f0f76800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7815dfb0 0x7f0f7806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.677+0000 7f0f76800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7815dfb0 0x7f0f7806c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51058/0 (socket says 172.21.15.7:51058) 2024-04-25T11:18:48.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.677+0000 7f0f76800640 1 -- 172.21.15.7:0/2426843493 learned_addr learned my addr 172.21.15.7:0/2426843493 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:48.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.677+0000 7f0f75e00640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f780d0b40 0x7f0f7814a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.677+0000 7f0f75400640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f780d0f50 0x7f0f7814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.677+0000 7f0f76800640 1 -- 172.21.15.7:0/2426843493 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f780d0f50 msgr2=0x7f0f7814d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:48.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.677+0000 7f0f76800640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f780d0f50 0x7f0f7814d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.677+0000 7f0f76800640 1 -- 172.21.15.7:0/2426843493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f780d0b40 msgr2=0x7f0f7814a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:48.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.677+0000 7f0f76800640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f780d0b40 0x7f0f7814a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.677+0000 7f0f76800640 1 -- 172.21.15.7:0/2426843493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f78158c30 con 0x7f0f7815dfb0 2024-04-25T11:18:48.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.678+0000 7f0f76800640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7815dfb0 0x7f0f7806c570 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000cbb0 tx=0x7f0f7000a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:48.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.678+0000 7f0f6e000640 1 -- 172.21.15.7:0/2426843493 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f70015070 con 0x7f0f7815dfb0 2024-04-25T11:18:48.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.678+0000 7f0f6e000640 1 -- 172.21.15.7:0/2426843493 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f7000abd0 con 0x7f0f7815dfb0 2024-04-25T11:18:48.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.678+0000 7f0f7d000640 1 -- 172.21.15.7:0/2426843493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7814d850 con 0x7f0f7815dfb0 2024-04-25T11:18:48.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.678+0000 7f0f6e000640 1 -- 172.21.15.7:0/2426843493 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7000d040 con 0x7f0f7815dfb0 2024-04-25T11:18:48.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.678+0000 7f0f7d000640 1 -- 172.21.15.7:0/2426843493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f780cd060 con 0x7f0f7815dfb0 2024-04-25T11:18:48.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.679+0000 7f0f7d000640 1 -- 172.21.15.7:0/2426843493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f78050740 con 0x7f0f7815dfb0 2024-04-25T11:18:48.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.681+0000 7f0f6e000640 1 -- 172.21.15.7:0/2426843493 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0f70018070 con 0x7f0f7815dfb0 2024-04-25T11:18:48.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.682+0000 7f0f6e000640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f540744e0 0x7f0f540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:48.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.682+0000 7f0f6e000640 1 -- 172.21.15.7:0/2426843493 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f0f70094fd0 con 0x7f0f7815dfb0 2024-04-25T11:18:48.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.682+0000 7f0f75e00640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f540744e0 0x7f0f540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:48.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.683+0000 7f0f75e00640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f540744e0 0x7f0f540769f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006fd0 tx=0x7f0f68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:48.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.686+0000 7f0f6e000640 1 -- 172.21.15.7:0/2426843493 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f7005f5e0 con 0x7f0f7815dfb0 2024-04-25T11:18:48.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.886+0000 7f0f7d000640 1 -- 172.21.15.7:0/2426843493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0f780ccb10 con 0x7f0f7815dfb0 2024-04-25T11:18:48.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.887+0000 7f0f6e000640 1 -- 172.21.15.7:0/2426843493 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f0f7005eee0 con 0x7f0f7815dfb0 2024-04-25T11:18:48.886 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:18:48.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.890+0000 7f0f7d000640 1 -- 172.21.15.7:0/2426843493 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f540744e0 msgr2=0x7f0f540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:48.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.890+0000 7f0f7d000640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f540744e0 0x7f0f540769f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006fd0 tx=0x7f0f68008040 comp rx=0 tx=0).stop 2024-04-25T11:18:48.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.890+0000 7f0f7d000640 1 -- 172.21.15.7:0/2426843493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7815dfb0 msgr2=0x7f0f7806c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:48.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.890+0000 7f0f7d000640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7815dfb0 0x7f0f7806c570 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0f7000cbb0 tx=0x7f0f7000a4c0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.890+0000 7f0f7d000640 1 -- 172.21.15.7:0/2426843493 shutdown_connections 2024-04-25T11:18:48.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.890+0000 7f0f7d000640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f540744e0 0x7f0f540769f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.890+0000 7f0f7d000640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f7815dfb0 0x7f0f7806c570 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.890+0000 7f0f7d000640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f780d0f50 0x7f0f7814d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.890+0000 7f0f7d000640 1 --2- 172.21.15.7:0/2426843493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f780d0b40 0x7f0f7814a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:48.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.890+0000 7f0f7d000640 1 -- 172.21.15.7:0/2426843493 >> 172.21.15.7:0/2426843493 conn(0x7f0f780742c0 msgr2=0x7f0f7806d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:48.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.891+0000 7f0f7d000640 1 -- 172.21.15.7:0/2426843493 shutdown_connections 2024-04-25T11:18:48.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:48.891+0000 7f0f7d000640 1 -- 172.21.15.7:0/2426843493 wait complete. 2024-04-25T11:18:48.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:18:49.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.208+0000 7f1cdda00640 1 Processor -- start 2024-04-25T11:18:49.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.208+0000 7f1cdda00640 1 -- start start 2024-04-25T11:18:49.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.210+0000 7f1cdda00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1cd8070a70 0x7f1cd80ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.210+0000 7f1cdda00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd80aedc0 0x7f1cd80b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.210+0000 7f1cdda00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cd80b0a10 0x7f1cd81607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.210+0000 7f1cdda00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd815e190 con 0x7f1cd80b0a10 2024-04-25T11:18:49.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.210+0000 7f1cdda00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd815d610 con 0x7f1cd8070a70 2024-04-25T11:18:49.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.210+0000 7f1cdda00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd815d8f0 con 0x7f1cd80aedc0 2024-04-25T11:18:49.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.210+0000 7f1cdd000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cd80b0a10 0x7f1cd81607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.210+0000 7f1cdd000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cd80b0a10 0x7f1cd81607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51060/0 (socket says 172.21.15.7:51060) 2024-04-25T11:18:49.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.210+0000 7f1cdd000640 1 -- 172.21.15.7:0/3558092169 learned_addr learned my addr 172.21.15.7:0/3558092169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:49.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.211+0000 7f1ccf400640 1 --2- 172.21.15.7:0/3558092169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd80aedc0 0x7f1cd80b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.211+0000 7f1cd7e00640 1 --2- 172.21.15.7:0/3558092169 >> v2:172.21.15.87:3300/0 conn(0x7f1cd8070a70 0x7f1cd80ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.211+0000 7f1cdd000640 1 -- 172.21.15.7:0/3558092169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd80aedc0 msgr2=0x7f1cd80b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:49.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.211+0000 7f1cdd000640 1 --2- 172.21.15.7:0/3558092169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd80aedc0 0x7f1cd80b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.211+0000 7f1cdd000640 1 -- 172.21.15.7:0/3558092169 >> v2:172.21.15.87:3300/0 conn(0x7f1cd8070a70 msgr2=0x7f1cd80ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:49.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.211+0000 7f1cdd000640 1 --2- 172.21.15.7:0/3558092169 >> v2:172.21.15.87:3300/0 conn(0x7f1cd8070a70 0x7f1cd80ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.211+0000 7f1cdd000640 1 -- 172.21.15.7:0/3558092169 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd815dbd0 con 0x7f1cd80b0a10 2024-04-25T11:18:49.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.212+0000 7f1cdd000640 1 --2- 172.21.15.7:0/3558092169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd80b0a10 0x7f1cd81607b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80725e0 tx=0x7f1cd002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb57ae50a5c17aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:49.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.213+0000 7f1cd7400640 1 -- 172.21.15.7:0/3558092169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0036070 con 0x7f1cd80b0a10 2024-04-25T11:18:49.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.213+0000 7f1cd7400640 1 -- 172.21.15.7:0/3558092169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd0031600 con 0x7f1cd80b0a10 2024-04-25T11:18:49.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.213+0000 7f1cd7400640 1 -- 172.21.15.7:0/3558092169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00319d0 con 0x7f1cd80b0a10 2024-04-25T11:18:49.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.213+0000 7f1cdda00640 1 -- 172.21.15.7:0/3558092169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd80b0a10 msgr2=0x7f1cd81607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:49.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.213+0000 7f1cdda00640 1 --2- 172.21.15.7:0/3558092169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd80b0a10 0x7f1cd81607b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80725e0 tx=0x7f1cd002a6a0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.214+0000 7f1cdda00640 1 -- 172.21.15.7:0/3558092169 shutdown_connections 2024-04-25T11:18:49.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.214+0000 7f1cdda00640 1 --2- 172.21.15.7:0/3558092169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd80b0a10 0x7f1cd81607b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.214+0000 7f1cdda00640 1 --2- 172.21.15.7:0/3558092169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd80aedc0 0x7f1cd80b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.214+0000 7f1cdda00640 1 --2- 172.21.15.7:0/3558092169 >> v2:172.21.15.87:3300/0 conn(0x7f1cd8070a70 0x7f1cd80ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.214+0000 7f1cdda00640 1 -- 172.21.15.7:0/3558092169 >> 172.21.15.7:0/3558092169 conn(0x7f1cd80b3820 msgr2=0x7f1cd80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:49.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.214+0000 7f1cdda00640 1 -- 172.21.15.7:0/3558092169 shutdown_connections 2024-04-25T11:18:49.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.214+0000 7f1cdda00640 1 -- 172.21.15.7:0/3558092169 wait complete. 2024-04-25T11:18:49.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.216+0000 7f1cdda00640 1 Processor -- start 2024-04-25T11:18:49.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.216+0000 7f1cdda00640 1 -- start start 2024-04-25T11:18:49.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.218+0000 7f1cdda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cd8070a70 0x7f1cd8147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.218+0000 7f1cdda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd80aedc0 0x7f1cd815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.218+0000 7f1cdda00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd80b0a10 0x7f1cd80cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.218+0000 7f1cdda00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd815d170 con 0x7f1cd80aedc0 2024-04-25T11:18:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.218+0000 7f1cdda00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd80b0e20 con 0x7f1cd8070a70 2024-04-25T11:18:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.218+0000 7f1cdda00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd815d2f0 con 0x7f1cd80b0a10 2024-04-25T11:18:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.218+0000 7f1ccf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd80aedc0 0x7f1cd815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.218+0000 7f1ccf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd80aedc0 0x7f1cd815ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51076/0 (socket says 172.21.15.7:51076) 2024-04-25T11:18:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.218+0000 7f1ccf400640 1 -- 172.21.15.7:0/2675032286 learned_addr learned my addr 172.21.15.7:0/2675032286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.218+0000 7f1cdd000640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd80b0a10 0x7f1cd80cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.218+0000 7f1cd7e00640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cd8070a70 0x7f1cd8147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.219+0000 7f1ccf400640 1 -- 172.21.15.7:0/2675032286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd80b0a10 msgr2=0x7f1cd80cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.219+0000 7f1ccf400640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd80b0a10 0x7f1cd80cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.219+0000 7f1ccf400640 1 -- 172.21.15.7:0/2675032286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cd8070a70 msgr2=0x7f1cd8147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.219+0000 7f1ccf400640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cd8070a70 0x7f1cd8147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.219+0000 7f1ccf400640 1 -- 172.21.15.7:0/2675032286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd81f72e0 con 0x7f1cd80aedc0 2024-04-25T11:18:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.219+0000 7f1cd7e00640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cd8070a70 0x7f1cd8147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.219+0000 7f1ccf400640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd80aedc0 0x7f1cd815ff90 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000e8a0 tx=0x7f1cc000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.219+0000 7f1cd4c00640 1 -- 172.21.15.7:0/2675032286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0017070 con 0x7f1cd80aedc0 2024-04-25T11:18:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.219+0000 7f1cd4c00640 1 -- 172.21.15.7:0/2675032286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc000c8e0 con 0x7f1cd80aedc0 2024-04-25T11:18:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.219+0000 7f1cd4c00640 1 -- 172.21.15.7:0/2675032286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000ccd0 con 0x7f1cd80aedc0 2024-04-25T11:18:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.219+0000 7f1cdda00640 1 -- 172.21.15.7:0/2675032286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd81f77e0 con 0x7f1cd80aedc0 2024-04-25T11:18:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.220+0000 7f1cdda00640 1 -- 172.21.15.7:0/2675032286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd81f7da0 con 0x7f1cd80aedc0 2024-04-25T11:18:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.221+0000 7f1cdda00640 1 -- 172.21.15.7:0/2675032286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd8050740 con 0x7f1cd80aedc0 2024-04-25T11:18:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.222+0000 7f1cd4c00640 1 -- 172.21.15.7:0/2675032286 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1cc001a070 con 0x7f1cd80aedc0 2024-04-25T11:18:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.223+0000 7f1cd4c00640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb4074530 0x7f1cb4076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.223+0000 7f1cd4c00640 1 -- 172.21.15.7:0/2675032286 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f1cc0095640 con 0x7f1cd80aedc0 2024-04-25T11:18:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.223+0000 7f1cd7e00640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb4074530 0x7f1cb4076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.224+0000 7f1cd7e00640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb4074530 0x7f1cb4076a40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8006fd0 tx=0x7f1cc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:49.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.228+0000 7f1cd4c00640 1 -- 172.21.15.7:0/2675032286 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1cc005fbb0 con 0x7f1cd80aedc0 2024-04-25T11:18:49.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.478+0000 7f1cdda00640 1 -- 172.21.15.7:0/2675032286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cd8151260 con 0x7f1cd80aedc0 2024-04-25T11:18:49.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.484+0000 7f1cd4c00640 1 -- 172.21.15.7:0/2675032286 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5169 (secure 0 0 0) 0x7f1cc005f4b0 con 0x7f1cd80aedc0 2024-04-25T11:18:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:49.483 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/684015857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":684015857},{"type":"v1","addr":"172.21.15.7:6836","nonce":684015857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/493684223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":493684223},{"type":"v1","addr":"172.21.15.87:6835","nonce":493684223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6109,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/1959070928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1959070928},{"type":"v1","addr":"172.21.15.87:6833","nonce":1959070928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:16:01.820362+0000","modified":"2024-04-25T11:18:46.323260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":44,"addr":"172.21.15.7:6837/336889324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":336889324},{"type":"v1","addr":"172.21.15.7:6837","nonce":336889324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6960,"qdb_cluster":[6960]},"id":6}]} 2024-04-25T11:18:49.484 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-04-25T11:18:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.486+0000 7f1cdda00640 1 -- 172.21.15.7:0/2675032286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb4074530 msgr2=0x7f1cb4076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.486+0000 7f1cdda00640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb4074530 0x7f1cb4076a40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8006fd0 tx=0x7f1cc8008040 comp rx=0 tx=0).stop 2024-04-25T11:18:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.486+0000 7f1cdda00640 1 -- 172.21.15.7:0/2675032286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd80aedc0 msgr2=0x7f1cd815ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.486+0000 7f1cdda00640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd80aedc0 0x7f1cd815ff90 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000e8a0 tx=0x7f1cc000ed60 comp rx=0 tx=0).stop 2024-04-25T11:18:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.487+0000 7f1cdda00640 1 -- 172.21.15.7:0/2675032286 shutdown_connections 2024-04-25T11:18:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.487+0000 7f1cdda00640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb4074530 0x7f1cb4076a40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.487+0000 7f1cdda00640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd80b0a10 0x7f1cd80cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.487+0000 7f1cdda00640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd80aedc0 0x7f1cd815ff90 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.487+0000 7f1cdda00640 1 --2- 172.21.15.7:0/2675032286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cd8070a70 0x7f1cd8147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.487+0000 7f1cdda00640 1 -- 172.21.15.7:0/2675032286 >> 172.21.15.7:0/2675032286 conn(0x7f1cd80b3820 msgr2=0x7f1cd806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.487+0000 7f1cdda00640 1 -- 172.21.15.7:0/2675032286 shutdown_connections 2024-04-25T11:18:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.487+0000 7f1cdda00640 1 -- 172.21.15.7:0/2675032286 wait complete. 2024-04-25T11:18:49.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:18:49.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.793+0000 7fce44400640 1 Processor -- start 2024-04-25T11:18:49.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.793+0000 7fce44400640 1 -- start start 2024-04-25T11:18:49.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.796+0000 7fce44400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fce3c160800 0x7fce3c156610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.796+0000 7fce44400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce3c15fc40 0x7fce3c160050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.796+0000 7fce44400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce3c160590 con 0x7fce3c0713c0 2024-04-25T11:18:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.796+0000 7fce44400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fce3c0b11d0 con 0x7fce3c15fc40 2024-04-25T11:18:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.796+0000 7fce44400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fce3c0b0c00 con 0x7fce3c160800 2024-04-25T11:18:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.796+0000 7fce42400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce3c15fc40 0x7fce3c160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.797+0000 7fce42400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce3c15fc40 0x7fce3c160050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51078/0 (socket says 172.21.15.7:51078) 2024-04-25T11:18:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.797+0000 7fce42400640 1 -- 172.21.15.7:0/3429329774 learned_addr learned my addr 172.21.15.7:0/3429329774 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.797+0000 7fce41000640 1 --2- 172.21.15.7:0/3429329774 >> v2:172.21.15.87:3300/0 conn(0x7fce3c160800 0x7fce3c156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.797+0000 7fce42400640 1 -- 172.21.15.7:0/3429329774 >> v2:172.21.15.87:3300/0 conn(0x7fce3c160800 msgr2=0x7fce3c156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.797+0000 7fce42400640 1 --2- 172.21.15.7:0/3429329774 >> v2:172.21.15.87:3300/0 conn(0x7fce3c160800 0x7fce3c156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.797+0000 7fce42400640 1 -- 172.21.15.7:0/3429329774 >> v1:172.21.15.87:6789/0 conn(0x7fce3c0713c0 legacy=0x7fce3c0aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.797+0000 7fce42400640 1 -- 172.21.15.7:0/3429329774 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c0b6760 con 0x7fce3c15fc40 2024-04-25T11:18:49.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.799+0000 7fce42400640 1 --2- 172.21.15.7:0/3429329774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce3c15fc40 0x7fce3c160050 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0747e0 tx=0x7fce3402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657f2ea4eba39b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.799+0000 7fce3be00640 1 -- 172.21.15.7:0/3429329774 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34036070 con 0x7fce3c15fc40 2024-04-25T11:18:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.799+0000 7fce3be00640 1 -- 172.21.15.7:0/3429329774 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce34031600 con 0x7fce3c15fc40 2024-04-25T11:18:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.799+0000 7fce3be00640 1 -- 172.21.15.7:0/3429329774 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce340319d0 con 0x7fce3c15fc40 2024-04-25T11:18:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.800+0000 7fce44400640 1 -- 172.21.15.7:0/3429329774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce3c15fc40 msgr2=0x7fce3c160050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.800+0000 7fce44400640 1 --2- 172.21.15.7:0/3429329774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce3c15fc40 0x7fce3c160050 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0747e0 tx=0x7fce3402a660 comp rx=0 tx=0).stop 2024-04-25T11:18:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.800+0000 7fce44400640 1 -- 172.21.15.7:0/3429329774 shutdown_connections 2024-04-25T11:18:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.800+0000 7fce44400640 1 --2- 172.21.15.7:0/3429329774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce3c15fc40 0x7fce3c160050 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.800+0000 7fce44400640 1 --2- 172.21.15.7:0/3429329774 >> v2:172.21.15.87:3300/0 conn(0x7fce3c160800 0x7fce3c156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.800+0000 7fce44400640 1 -- 172.21.15.7:0/3429329774 >> 172.21.15.7:0/3429329774 conn(0x7fce3c074200 msgr2=0x7fce3c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.800+0000 7fce44400640 1 -- 172.21.15.7:0/3429329774 shutdown_connections 2024-04-25T11:18:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.800+0000 7fce44400640 1 -- 172.21.15.7:0/3429329774 wait complete. 2024-04-25T11:18:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.802+0000 7fce44400640 1 Processor -- start 2024-04-25T11:18:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.802+0000 7fce44400640 1 -- start start 2024-04-25T11:18:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce44400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce3c0713c0 0x7fce3c1f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce44400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce3c15fc40 0x7fce3c14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce44400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce3c160800 0x7fce3c14f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce44400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c0b5b00 con 0x7fce3c15fc40 2024-04-25T11:18:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce44400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce3c0b0e60 con 0x7fce3c0713c0 2024-04-25T11:18:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce44400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce3c0b5c80 con 0x7fce3c160800 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce41000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce3c15fc40 0x7fce3c14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce41000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce3c15fc40 0x7fce3c14ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51080/0 (socket says 172.21.15.7:51080) 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce41000640 1 -- 172.21.15.7:0/3236372173 learned_addr learned my addr 172.21.15.7:0/3236372173 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce41a00640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce3c0713c0 0x7fce3c1f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce42400640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce3c160800 0x7fce3c14f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.804+0000 7fce41000640 1 -- 172.21.15.7:0/3236372173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce3c160800 msgr2=0x7fce3c14f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.805+0000 7fce41000640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce3c160800 0x7fce3c14f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.805+0000 7fce41000640 1 -- 172.21.15.7:0/3236372173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce3c0713c0 msgr2=0x7fce3c1f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.805+0000 7fce41000640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce3c0713c0 0x7fce3c1f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.805+0000 7fce41000640 1 -- 172.21.15.7:0/3236372173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3c0d01e0 con 0x7fce3c15fc40 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.805+0000 7fce41a00640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce3c0713c0 0x7fce3c1f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.805+0000 7fce41000640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce3c15fc40 0x7fce3c14ee00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fce2800d620 tx=0x7fce2800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.805+0000 7fce39600640 1 -- 172.21.15.7:0/3236372173 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2800b620 con 0x7fce3c15fc40 2024-04-25T11:18:49.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.805+0000 7fce39600640 1 -- 172.21.15.7:0/3236372173 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce28019070 con 0x7fce3c15fc40 2024-04-25T11:18:49.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.805+0000 7fce39600640 1 -- 172.21.15.7:0/3236372173 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2800ba00 con 0x7fce3c15fc40 2024-04-25T11:18:49.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.805+0000 7fce44400640 1 -- 172.21.15.7:0/3236372173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3c0d04c0 con 0x7fce3c15fc40 2024-04-25T11:18:49.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.805+0000 7fce44400640 1 -- 172.21.15.7:0/3236372173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3c0d0a80 con 0x7fce3c15fc40 2024-04-25T11:18:49.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.806+0000 7fce44400640 1 -- 172.21.15.7:0/3236372173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3c06a100 con 0x7fce3c15fc40 2024-04-25T11:18:49.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.808+0000 7fce39600640 1 -- 172.21.15.7:0/3236372173 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce2800bba0 con 0x7fce3c15fc40 2024-04-25T11:18:49.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.809+0000 7fce39600640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce1c0744b0 0x7fce1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.809+0000 7fce39600640 1 -- 172.21.15.7:0/3236372173 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7fce28095870 con 0x7fce3c15fc40 2024-04-25T11:18:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.809+0000 7fce41a00640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce1c0744b0 0x7fce1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:49.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.810+0000 7fce41a00640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce1c0744b0 0x7fce1c0769c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fce3000a830 tx=0x7fce30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:49.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:49.816+0000 7fce39600640 1 -- 172.21.15.7:0/3236372173 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce2805fdf0 con 0x7fce3c15fc40 2024-04-25T11:18:50.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.024+0000 7fce44400640 1 -- 172.21.15.7:0/3236372173 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce3c0b4c30 con 0x7fce3c15fc40 2024-04-25T11:18:50.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.028+0000 7fce39600640 1 -- 172.21.15.7:0/3236372173 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16983 (secure 0 0 0) 0x7fce2805f6f0 con 0x7fce3c15fc40 2024-04-25T11:18:50.028 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:50.028 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:17:04.183733+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:59.999922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T11:16:00.796013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":5}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:18:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.031+0000 7fce44400640 1 -- 172.21.15.7:0/3236372173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce1c0744b0 msgr2=0x7fce1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.031+0000 7fce44400640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce1c0744b0 0x7fce1c0769c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fce3000a830 tx=0x7fce30008040 comp rx=0 tx=0).stop 2024-04-25T11:18:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.031+0000 7fce44400640 1 -- 172.21.15.7:0/3236372173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce3c15fc40 msgr2=0x7fce3c14ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.031+0000 7fce44400640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce3c15fc40 0x7fce3c14ee00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fce2800d620 tx=0x7fce2800dae0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.031+0000 7fce44400640 1 -- 172.21.15.7:0/3236372173 shutdown_connections 2024-04-25T11:18:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.031+0000 7fce44400640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce1c0744b0 0x7fce1c0769c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.031+0000 7fce44400640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce3c160800 0x7fce3c14f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.031+0000 7fce44400640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce3c15fc40 0x7fce3c14ee00 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.031+0000 7fce44400640 1 --2- 172.21.15.7:0/3236372173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce3c0713c0 0x7fce3c1f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.031+0000 7fce44400640 1 -- 172.21.15.7:0/3236372173 >> 172.21.15.7:0/3236372173 conn(0x7fce3c074200 msgr2=0x7fce3c15e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.032+0000 7fce44400640 1 -- 172.21.15.7:0/3236372173 shutdown_connections 2024-04-25T11:18:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.032+0000 7fce44400640 1 -- 172.21.15.7:0/3236372173 wait complete. 2024-04-25T11:18:50.058 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:18:50.058 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:18:50.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.350+0000 7f27c1e00640 1 Processor -- start 2024-04-25T11:18:50.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.350+0000 7f27c1e00640 1 -- start start 2024-04-25T11:18:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.353+0000 7f27c1e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f27bc0b4aa0 0x7f27bc0ae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.353+0000 7f27c1e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27bc164c80 con 0x7f27bc164710 2024-04-25T11:18:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.353+0000 7f27c1e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27bc164f60 con 0x7f27bc0b0f70 2024-04-25T11:18:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.353+0000 7f27c1e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f27bc1655b0 con 0x7f27bc0b4aa0 2024-04-25T11:18:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.353+0000 7f27baa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f27bc0b4aa0 0x7f27bc0ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.353+0000 7f27b3e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f27bc164710 0x7f27bc0b0860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45258/0 (socket says 172.21.15.7:45258) 2024-04-25T11:18:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.353+0000 7f27baa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f27bc0b4aa0 0x7f27bc0ae960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46122/0 (socket says 172.21.15.7:46122) 2024-04-25T11:18:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.353+0000 7f27b3e00640 1 -- 172.21.15.7:0/558313394 learned_addr learned my addr 172.21.15.7:0/558313394 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:50.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.354+0000 7f27baa00640 1 -- 172.21.15.7:0/558313394 >> v1:172.21.15.87:6789/0 conn(0x7f27bc0b0f70 legacy=0x7f27bc1680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:50.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.354+0000 7f27baa00640 1 -- 172.21.15.7:0/558313394 >> v1:172.21.15.7:6789/0 conn(0x7f27bc164710 legacy=0x7f27bc0b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.354+0000 7f27baa00640 1 -- 172.21.15.7:0/558313394 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc165240 con 0x7f27bc0b4aa0 2024-04-25T11:18:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.355+0000 7f27baa00640 1 --2- 172.21.15.7:0/558313394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc0b4aa0 0x7f27bc0ae960 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0097a0 tx=0x7f27ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445e9ebb2e94dfc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.355+0000 7f27ba000640 1 -- 172.21.15.7:0/558313394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac037070 con 0x7f27bc0b4aa0 2024-04-25T11:18:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.355+0000 7f27ba000640 1 -- 172.21.15.7:0/558313394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27ac032440 con 0x7f27bc0b4aa0 2024-04-25T11:18:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.356+0000 7f27ba000640 1 -- 172.21.15.7:0/558313394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac032810 con 0x7f27bc0b4aa0 2024-04-25T11:18:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.356+0000 7f27c1e00640 1 -- 172.21.15.7:0/558313394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc0b4aa0 msgr2=0x7f27bc0ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.356+0000 7f27c1e00640 1 --2- 172.21.15.7:0/558313394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc0b4aa0 0x7f27bc0ae960 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0097a0 tx=0x7f27ac02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.357+0000 7f27c1e00640 1 -- 172.21.15.7:0/558313394 shutdown_connections 2024-04-25T11:18:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.357+0000 7f27c1e00640 1 --2- 172.21.15.7:0/558313394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc0b4aa0 0x7f27bc0ae960 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.357+0000 7f27c1e00640 1 -- 172.21.15.7:0/558313394 >> 172.21.15.7:0/558313394 conn(0x7f27bc074200 msgr2=0x7f27bc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.357+0000 7f27c1e00640 1 -- 172.21.15.7:0/558313394 shutdown_connections 2024-04-25T11:18:50.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.357+0000 7f27c1e00640 1 -- 172.21.15.7:0/558313394 wait complete. 2024-04-25T11:18:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.359+0000 7f27c1e00640 1 Processor -- start 2024-04-25T11:18:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.359+0000 7f27c1e00640 1 -- start start 2024-04-25T11:18:50.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.361+0000 7f27c1e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27bc0b0f70 0x7f27bc159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:50.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.361+0000 7f27c1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27bc0b4aa0 0x7f27bc0b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:50.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.361+0000 7f27c1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc164710 0x7f27bc1495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:50.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.361+0000 7f27c1e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc06f300 con 0x7f27bc164710 2024-04-25T11:18:50.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.361+0000 7f27c1e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc165a40 con 0x7f27bc0b4aa0 2024-04-25T11:18:50.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.361+0000 7f27c1e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f27bc0aeea0 con 0x7f27bc0b0f70 2024-04-25T11:18:50.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.361+0000 7f27bb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc164710 0x7f27bc1495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:50.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.361+0000 7f27bb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc164710 0x7f27bc1495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46132/0 (socket says 172.21.15.7:46132) 2024-04-25T11:18:50.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.361+0000 7f27bb400640 1 -- 172.21.15.7:0/340858555 learned_addr learned my addr 172.21.15.7:0/340858555 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:50.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.362+0000 7f27b3e00640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27bc0b4aa0 0x7f27bc0b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.362+0000 7f27baa00640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27bc0b0f70 0x7f27bc159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.362+0000 7f27bb400640 1 -- 172.21.15.7:0/340858555 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27bc0b0f70 msgr2=0x7f27bc159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.362+0000 7f27bb400640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27bc0b0f70 0x7f27bc159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.362+0000 7f27bb400640 1 -- 172.21.15.7:0/340858555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27bc0b4aa0 msgr2=0x7f27bc0b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.362+0000 7f27bb400640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27bc0b4aa0 0x7f27bc0b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.362+0000 7f27bb400640 1 -- 172.21.15.7:0/340858555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc15cbe0 con 0x7f27bc164710 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.362+0000 7f27bb400640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc164710 0x7f27bc1495a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f27b400ba10 tx=0x7f27b400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.362+0000 7f27b3400640 1 -- 172.21.15.7:0/340858555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4019070 con 0x7f27bc164710 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.363+0000 7f27b3400640 1 -- 172.21.15.7:0/340858555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b400cb40 con 0x7f27bc164710 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.363+0000 7f27c1e00640 1 -- 172.21.15.7:0/340858555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc14a270 con 0x7f27bc164710 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.363+0000 7f27b3400640 1 -- 172.21.15.7:0/340858555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4014440 con 0x7f27bc164710 2024-04-25T11:18:50.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.363+0000 7f27c1e00640 1 -- 172.21.15.7:0/340858555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc14a830 con 0x7f27bc164710 2024-04-25T11:18:50.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.364+0000 7f27c1e00640 1 -- 172.21.15.7:0/340858555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc06a100 con 0x7f27bc164710 2024-04-25T11:18:50.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.365+0000 7f27b3400640 1 -- 172.21.15.7:0/340858555 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f27b401c070 con 0x7f27bc164710 2024-04-25T11:18:50.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.366+0000 7f27b3400640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27980744e0 0x7f27980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:50.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.366+0000 7f27b3400640 1 -- 172.21.15.7:0/340858555 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f27b4095060 con 0x7f27bc164710 2024-04-25T11:18:50.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.366+0000 7f27baa00640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27980744e0 0x7f27980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:50.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.367+0000 7f27baa00640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27980744e0 0x7f27980769f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f27ac034000 tx=0x7f27ac02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:50.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.371+0000 7f27b3400640 1 -- 172.21.15.7:0/340858555 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27b405f550 con 0x7f27bc164710 2024-04-25T11:18:50.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.616+0000 7f27c1e00640 1 -- 172.21.15.7:0/340858555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27bc15c970 con 0x7f27bc164710 2024-04-25T11:18:50.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.617+0000 7f27b3400640 1 -- 172.21.15.7:0/340858555 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5169 (secure 0 0 0) 0x7f27b405ee50 con 0x7f27bc164710 2024-04-25T11:18:50.616 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:50.617 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/684015857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":684015857},{"type":"v1","addr":"172.21.15.7:6836","nonce":684015857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/493684223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":493684223},{"type":"v1","addr":"172.21.15.87:6835","nonce":493684223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6109,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/1959070928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1959070928},{"type":"v1","addr":"172.21.15.87:6833","nonce":1959070928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:16:01.820362+0000","modified":"2024-04-25T11:18:46.323260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":44,"addr":"172.21.15.7:6837/336889324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":336889324},{"type":"v1","addr":"172.21.15.7:6837","nonce":336889324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6960,"qdb_cluster":[6960]},"id":6}]} 2024-04-25T11:18:50.617 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-04-25T11:18:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.620+0000 7f27c1e00640 1 -- 172.21.15.7:0/340858555 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27980744e0 msgr2=0x7f27980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.620+0000 7f27c1e00640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27980744e0 0x7f27980769f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f27ac034000 tx=0x7f27ac02e000 comp rx=0 tx=0).stop 2024-04-25T11:18:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.620+0000 7f27c1e00640 1 -- 172.21.15.7:0/340858555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc164710 msgr2=0x7f27bc1495a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.620+0000 7f27c1e00640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc164710 0x7f27bc1495a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f27b400ba10 tx=0x7f27b400bed0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.620+0000 7f27c1e00640 1 -- 172.21.15.7:0/340858555 shutdown_connections 2024-04-25T11:18:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.620+0000 7f27c1e00640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27980744e0 0x7f27980769f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.620+0000 7f27c1e00640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27bc164710 0x7f27bc1495a0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.620+0000 7f27c1e00640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27bc0b4aa0 0x7f27bc0b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.621+0000 7f27c1e00640 1 --2- 172.21.15.7:0/340858555 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27bc0b0f70 0x7f27bc159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.621+0000 7f27c1e00640 1 -- 172.21.15.7:0/340858555 >> 172.21.15.7:0/340858555 conn(0x7f27bc074200 msgr2=0x7f27bc0b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.621+0000 7f27c1e00640 1 -- 172.21.15.7:0/340858555 shutdown_connections 2024-04-25T11:18:50.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.621+0000 7f27c1e00640 1 -- 172.21.15.7:0/340858555 wait complete. 2024-04-25T11:18:50.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:18:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.933+0000 7f0db7200640 1 Processor -- start 2024-04-25T11:18:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.933+0000 7f0db7200640 1 -- start start 2024-04-25T11:18:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.935+0000 7f0db7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db00b04d0 0x7f0db00b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.935+0000 7f0db7200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0db00b4a80 0x7f0db00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.935+0000 7f0db7200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db015e190 con 0x7f0db0070a70 2024-04-25T11:18:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.935+0000 7f0db7200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0db015d610 con 0x7f0db00b4a80 2024-04-25T11:18:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.935+0000 7f0db7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0db015d8f0 con 0x7f0db00b04d0 2024-04-25T11:18:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.936+0000 7f0db6800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0db00b4a80 0x7f0db00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.936+0000 7f0db6800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0db00b4a80 0x7f0db00b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46144/0 (socket says 172.21.15.7:46144) 2024-04-25T11:18:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.936+0000 7f0db6800640 1 -- 172.21.15.7:0/2963685843 learned_addr learned my addr 172.21.15.7:0/2963685843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:50.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.936+0000 7f0db5400640 1 --2- 172.21.15.7:0/2963685843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db00b04d0 0x7f0db00b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:50.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.936+0000 7f0db6800640 1 -- 172.21.15.7:0/2963685843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db00b04d0 msgr2=0x7f0db00b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:50.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.936+0000 7f0db6800640 1 --2- 172.21.15.7:0/2963685843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db00b04d0 0x7f0db00b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.937+0000 7f0db6800640 1 -- 172.21.15.7:0/2963685843 >> v1:172.21.15.87:6789/0 conn(0x7f0db0070a70 legacy=0x7f0db00ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.937+0000 7f0db6800640 1 -- 172.21.15.7:0/2963685843 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db015dbd0 con 0x7f0db00b4a80 2024-04-25T11:18:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.938+0000 7f0db6800640 1 --2- 172.21.15.7:0/2963685843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db00b4a80 0x7f0db00b4e90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00d150 tx=0x7f0dac02c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43753ef07f4e9d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.938+0000 7f0db4a00640 1 -- 172.21.15.7:0/2963685843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac009600 con 0x7f0db00b4a80 2024-04-25T11:18:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.938+0000 7f0db4a00640 1 -- 172.21.15.7:0/2963685843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dac037070 con 0x7f0db00b4a80 2024-04-25T11:18:50.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.938+0000 7f0db4a00640 1 -- 172.21.15.7:0/2963685843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dac0099c0 con 0x7f0db00b4a80 2024-04-25T11:18:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.939+0000 7f0db7200640 1 -- 172.21.15.7:0/2963685843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db00b4a80 msgr2=0x7f0db00b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.939+0000 7f0db7200640 1 --2- 172.21.15.7:0/2963685843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db00b4a80 0x7f0db00b4e90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00d150 tx=0x7f0dac02c930 comp rx=0 tx=0).stop 2024-04-25T11:18:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.939+0000 7f0db7200640 1 -- 172.21.15.7:0/2963685843 shutdown_connections 2024-04-25T11:18:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.939+0000 7f0db7200640 1 --2- 172.21.15.7:0/2963685843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db00b4a80 0x7f0db00b4e90 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.939+0000 7f0db7200640 1 --2- 172.21.15.7:0/2963685843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db00b04d0 0x7f0db00b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.939+0000 7f0db7200640 1 -- 172.21.15.7:0/2963685843 >> 172.21.15.7:0/2963685843 conn(0x7f0db00b3820 msgr2=0x7f0db00b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.939+0000 7f0db7200640 1 -- 172.21.15.7:0/2963685843 shutdown_connections 2024-04-25T11:18:50.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.939+0000 7f0db7200640 1 -- 172.21.15.7:0/2963685843 wait complete. 2024-04-25T11:18:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.942+0000 7f0db7200640 1 Processor -- start 2024-04-25T11:18:50.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.942+0000 7f0db7200640 1 -- start start 2024-04-25T11:18:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.944+0000 7f0db7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db0070a70 0x7f0db009b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.944+0000 7f0db7200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0db00b04d0 0x7f0db01f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.944+0000 7f0db7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db00b4a80 0x7f0db01f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.944+0000 7f0db7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0db00b0e20 con 0x7f0db0070a70 2024-04-25T11:18:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.944+0000 7f0db7200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0db00aef90 con 0x7f0db00b04d0 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.944+0000 7f0db7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0db015d170 con 0x7f0db00b4a80 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.944+0000 7f0db5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db0070a70 0x7f0db009b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.944+0000 7f0db5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db0070a70 0x7f0db009b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46160/0 (socket says 172.21.15.7:46160) 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.944+0000 7f0db5e00640 1 -- 172.21.15.7:0/467313483 learned_addr learned my addr 172.21.15.7:0/467313483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.944+0000 7f0db6800640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db00b4a80 0x7f0db01f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.944+0000 7f0db5e00640 1 -- 172.21.15.7:0/467313483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db00b4a80 msgr2=0x7f0db01f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.945+0000 7f0db5e00640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db00b4a80 0x7f0db01f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.945+0000 7f0db5e00640 1 -- 172.21.15.7:0/467313483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0db00b04d0 msgr2=0x7f0db01f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.945+0000 7f0db5e00640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0db00b04d0 0x7f0db01f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.945+0000 7f0db5e00640 1 -- 172.21.15.7:0/467313483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db01fa8f0 con 0x7f0db0070a70 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.945+0000 7f0db5e00640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db0070a70 0x7f0db009b540 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0da400c3c0 tx=0x7f0da400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.945+0000 7f0daa000640 1 -- 172.21.15.7:0/467313483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4017070 con 0x7f0db0070a70 2024-04-25T11:18:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.945+0000 7f0daa000640 1 -- 172.21.15.7:0/467313483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da40052d0 con 0x7f0db0070a70 2024-04-25T11:18:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.945+0000 7f0daa000640 1 -- 172.21.15.7:0/467313483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4003830 con 0x7f0db0070a70 2024-04-25T11:18:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.945+0000 7f0db7200640 1 -- 172.21.15.7:0/467313483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db015b540 con 0x7f0db0070a70 2024-04-25T11:18:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.945+0000 7f0db7200640 1 -- 172.21.15.7:0/467313483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db015bb00 con 0x7f0db0070a70 2024-04-25T11:18:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.946+0000 7f0db7200640 1 -- 172.21.15.7:0/467313483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db0050740 con 0x7f0db0070a70 2024-04-25T11:18:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.948+0000 7f0daa000640 1 -- 172.21.15.7:0/467313483 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0da401a070 con 0x7f0db0070a70 2024-04-25T11:18:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.949+0000 7f0daa000640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744b0 0x7f0d940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.949+0000 7f0daa000640 1 -- 172.21.15.7:0/467313483 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f0da4094810 con 0x7f0db0070a70 2024-04-25T11:18:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.949+0000 7f0db5400640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744b0 0x7f0d940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.950+0000 7f0db5400640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744b0 0x7f0d940769c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006fd0 tx=0x7f0da0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:50.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:50.954+0000 7f0daa000640 1 -- 172.21.15.7:0/467313483 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0da405ed00 con 0x7f0db0070a70 2024-04-25T11:18:51.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.200+0000 7f0db7200640 1 -- 172.21.15.7:0/467313483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0db009f960 con 0x7f0db0070a70 2024-04-25T11:18:51.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.202+0000 7f0daa000640 1 -- 172.21.15.7:0/467313483 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5169 (secure 0 0 0) 0x7f0da405e600 con 0x7f0db0070a70 2024-04-25T11:18:51.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:51.201 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/684015857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":684015857},{"type":"v1","addr":"172.21.15.7:6836","nonce":684015857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/493684223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":493684223},{"type":"v1","addr":"172.21.15.87:6835","nonce":493684223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6109,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/1959070928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1959070928},{"type":"v1","addr":"172.21.15.87:6833","nonce":1959070928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:16:01.820362+0000","modified":"2024-04-25T11:18:46.323260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6960},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6960":{"gid":6960,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":44,"addr":"172.21.15.7:6837/336889324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":336889324},{"type":"v1","addr":"172.21.15.7:6837","nonce":336889324}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6960,"qdb_cluster":[6960]},"id":6}]} 2024-04-25T11:18:51.201 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2024-04-25T11:18:51.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.204+0000 7f0db7200640 1 -- 172.21.15.7:0/467313483 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744b0 msgr2=0x7f0d940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:51.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.204+0000 7f0db7200640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744b0 0x7f0d940769c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0da0006fd0 tx=0x7f0da0008040 comp rx=0 tx=0).stop 2024-04-25T11:18:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.204+0000 7f0db7200640 1 -- 172.21.15.7:0/467313483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db0070a70 msgr2=0x7f0db009b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.204+0000 7f0db7200640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db0070a70 0x7f0db009b540 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0da400c3c0 tx=0x7f0da400c880 comp rx=0 tx=0).stop 2024-04-25T11:18:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.204+0000 7f0db7200640 1 -- 172.21.15.7:0/467313483 shutdown_connections 2024-04-25T11:18:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.204+0000 7f0db7200640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744b0 0x7f0d940769c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.204+0000 7f0db7200640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db00b4a80 0x7f0db01f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.204+0000 7f0db7200640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0db00b04d0 0x7f0db01f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.204+0000 7f0db7200640 1 --2- 172.21.15.7:0/467313483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db0070a70 0x7f0db009b540 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.204+0000 7f0db7200640 1 -- 172.21.15.7:0/467313483 >> 172.21.15.7:0/467313483 conn(0x7f0db00b3820 msgr2=0x7f0db00ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.205+0000 7f0db7200640 1 -- 172.21.15.7:0/467313483 shutdown_connections 2024-04-25T11:18:51.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.205+0000 7f0db7200640 1 -- 172.21.15.7:0/467313483 wait complete. 2024-04-25T11:18:51.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:18:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.520+0000 7f59f3800640 1 Processor -- start 2024-04-25T11:18:51.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.520+0000 7f59f3800640 1 -- start start 2024-04-25T11:18:51.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.523+0000 7f59f3800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec0b0f70 0x7f59ec16f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:51.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.523+0000 7f59f3800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ec0af070 con 0x7f59ec0b4aa0 2024-04-25T11:18:51.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.523+0000 7f59f3800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ec164c80 con 0x7f59ec164710 2024-04-25T11:18:51.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.523+0000 7f59f3800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f59ec1655b0 con 0x7f59ec0b0f70 2024-04-25T11:18:51.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.523+0000 7f59f0e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f59ec0b4aa0 0x7f59ec0ae960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45268/0 (socket says 172.21.15.7:45268) 2024-04-25T11:18:51.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.524+0000 7f59f0e00640 1 -- 172.21.15.7:0/874847849 learned_addr learned my addr 172.21.15.7:0/874847849 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.524+0000 7f59f1800640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec0b0f70 0x7f59ec16f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:51.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.524+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2219612815 0 0) 0x7f59ec0af070 con 0x7f59ec0b4aa0 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.524+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59d0003520 con 0x7f59ec0b4aa0 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1185219189 0 0) 0x7f59d0003520 con 0x7f59ec0b4aa0 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec0b0f70 msgr2=0x7f59ec16f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec0b0f70 0x7f59ec16f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 >> v1:172.21.15.87:6789/0 conn(0x7f59ec164710 legacy=0x7f59ec0b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ec164f60 con 0x7f59ec0b4aa0 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f59dc0019f0 con 0x7f59ec0b4aa0 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 >> v1:172.21.15.7:6789/0 conn(0x7f59ec0b4aa0 legacy=0x7f59ec0ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec0b0f70 0x7f59d00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59d0003f60 0x7f59d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59d0004d40 0x7f59d0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec1655b0 con 0x7f59d0004d40 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f59d000bb50 con 0x7f59d0003f60 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f59d000bd50 con 0x7f59ec0b0f70 2024-04-25T11:18:51.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59f1800640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59d0004d40 0x7f59d0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59f0e00640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec0b0f70 0x7f59d00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59ebe00640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59d0003f60 0x7f59d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59f1800640 1 -- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec0b0f70 msgr2=0x7f59d00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59f1800640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec0b0f70 0x7f59d00039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59f1800640 1 -- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59d0003f60 msgr2=0x7f59d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59f1800640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59d0003f60 0x7f59d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59f1800640 1 -- 172.21.15.7:0/874847849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d0003520 con 0x7f59d0004d40 2024-04-25T11:18:51.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.525+0000 7f59ebe00640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59d0003f60 0x7f59d0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:51.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.526+0000 7f59f1800640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59d0004d40 0x7f59d0005150 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0b15a0 tx=0x7f59d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c03fdd7244906f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:51.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.527+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4036070 con 0x7f59d0004d40 2024-04-25T11:18:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.527+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e4031400 con 0x7f59d0004d40 2024-04-25T11:18:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.527+0000 7f59eb400640 1 -- 172.21.15.7:0/874847849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4009690 con 0x7f59d0004d40 2024-04-25T11:18:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.527+0000 7f59f3800640 1 -- 172.21.15.7:0/874847849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59d0004d40 msgr2=0x7f59d0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.527+0000 7f59f3800640 1 --2- 172.21.15.7:0/874847849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59d0004d40 0x7f59d0005150 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0b15a0 tx=0x7f59d0005830 comp rx=0 tx=0).stop 2024-04-25T11:18:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.527+0000 7f59f1800640 1 -- 172.21.15.7:0/874847849 reap_dead start 2024-04-25T11:18:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.528+0000 7f59f3800640 1 -- 172.21.15.7:0/874847849 shutdown_connections 2024-04-25T11:18:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.528+0000 7f59f3800640 1 -- 172.21.15.7:0/874847849 >> 172.21.15.7:0/874847849 conn(0x7f59ec074200 msgr2=0x7f59ec06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.528+0000 7f59f3800640 1 -- 172.21.15.7:0/874847849 shutdown_connections 2024-04-25T11:18:51.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.528+0000 7f59f3800640 1 -- 172.21.15.7:0/874847849 wait complete. 2024-04-25T11:18:51.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.530+0000 7f59f3800640 1 Processor -- start 2024-04-25T11:18:51.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.530+0000 7f59f3800640 1 -- start start 2024-04-25T11:18:51.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.531+0000 7f59f3800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ec156b30 0x7f59ec1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:51.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f3800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec152330 0x7f59ec1fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:51.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f3800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59ec1facf0 0x7f59ec0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:51.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f3800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec0af340 con 0x7f59ec1facf0 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f3800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f59ec165a40 con 0x7f59ec156b30 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f3800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f59ec06f300 con 0x7f59ec152330 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59ec1facf0 0x7f59ec0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59ec1facf0 0x7f59ec0b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46174/0 (socket says 172.21.15.7:46174) 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f1800640 1 -- 172.21.15.7:0/2275627128 learned_addr learned my addr 172.21.15.7:0/2275627128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f0e00640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ec156b30 0x7f59ec1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f1800640 1 -- 172.21.15.7:0/2275627128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec152330 msgr2=0x7f59ec1fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f1800640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec152330 0x7f59ec1fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f1800640 1 -- 172.21.15.7:0/2275627128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ec156b30 msgr2=0x7f59ec1588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f1800640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ec156b30 0x7f59ec1588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f1800640 1 -- 172.21.15.7:0/2275627128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ec157a40 con 0x7f59ec1facf0 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.532+0000 7f59f0e00640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ec156b30 0x7f59ec1588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:51.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.533+0000 7f59f1800640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59ec1facf0 0x7f59ec0b2620 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009b80 tx=0x7f59e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.533+0000 7f59e8c00640 1 -- 172.21.15.7:0/2275627128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4036040 con 0x7f59ec1facf0 2024-04-25T11:18:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.533+0000 7f59e8c00640 1 -- 172.21.15.7:0/2275627128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e4031ca0 con 0x7f59ec1facf0 2024-04-25T11:18:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.533+0000 7f59e8c00640 1 -- 172.21.15.7:0/2275627128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4009690 con 0x7f59ec1facf0 2024-04-25T11:18:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.533+0000 7f59f3800640 1 -- 172.21.15.7:0/2275627128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ec1571a0 con 0x7f59ec1facf0 2024-04-25T11:18:51.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.533+0000 7f59f3800640 1 -- 172.21.15.7:0/2275627128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ec157760 con 0x7f59ec1facf0 2024-04-25T11:18:51.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.534+0000 7f59f3800640 1 -- 172.21.15.7:0/2275627128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ec06a100 con 0x7f59ec1facf0 2024-04-25T11:18:51.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.536+0000 7f59e8c00640 1 -- 172.21.15.7:0/2275627128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f59e4045070 con 0x7f59ec1facf0 2024-04-25T11:18:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.537+0000 7f59e8c00640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f59c80744b0 0x7f59c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.537+0000 7f59e8c00640 1 -- 172.21.15.7:0/2275627128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f59e40b4d60 con 0x7f59ec1facf0 2024-04-25T11:18:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.537+0000 7f59f0e00640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f59c80744b0 0x7f59c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:51.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.538+0000 7f59f0e00640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f59c80744b0 0x7f59c80769c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f59dc002230 tx=0x7f59dc0017f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:51.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.542+0000 7f59e8c00640 1 -- 172.21.15.7:0/2275627128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f59e407f370 con 0x7f59ec1facf0 2024-04-25T11:18:51.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.747+0000 7f59f3800640 1 -- 172.21.15.7:0/2275627128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59ec167b40 con 0x7f59ec1facf0 2024-04-25T11:18:51.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.749+0000 7f59e8c00640 1 -- 172.21.15.7:0/2275627128 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16983 (secure 0 0 0) 0x7f59e4085070 con 0x7f59ec1facf0 2024-04-25T11:18:51.748 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:18:51.748 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:17:04.183733+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:15:59.999922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T11:16:00.796013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.12","mappings":[{"from":1,"to":5}]},{"pgid":"14.24","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:18:51.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 -- 172.21.15.7:0/2275627128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f59c80744b0 msgr2=0x7f59c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:51.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f59c80744b0 0x7f59c80769c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f59dc002230 tx=0x7f59dc0017f0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 -- 172.21.15.7:0/2275627128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59ec1facf0 msgr2=0x7f59ec0b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:51.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59ec1facf0 0x7f59ec0b2620 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009b80 tx=0x7f59e4004730 comp rx=0 tx=0).stop 2024-04-25T11:18:51.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 -- 172.21.15.7:0/2275627128 shutdown_connections 2024-04-25T11:18:51.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f59c80744b0 0x7f59c80769c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59ec1facf0 0x7f59ec0b2620 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59ec152330 0x7f59ec1fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 --2- 172.21.15.7:0/2275627128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59ec156b30 0x7f59ec1588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:51.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 -- 172.21.15.7:0/2275627128 >> 172.21.15.7:0/2275627128 conn(0x7f59ec074200 msgr2=0x7f59ec06d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:51.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 -- 172.21.15.7:0/2275627128 shutdown_connections 2024-04-25T11:18:51.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:51.752+0000 7f59f3800640 1 -- 172.21.15.7:0/2275627128 wait complete. 2024-04-25T11:18:51.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:18:52.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.061+0000 7f9229c00640 1 Processor -- start 2024-04-25T11:18:52.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.061+0000 7f9229c00640 1 -- start start 2024-04-25T11:18:52.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.063+0000 7f9229c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f922415eed0 0x7f92240cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.063+0000 7f9229c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f92241416a0 0x7f92240b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.063+0000 7f9229c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f922406c570 con 0x7f9224141290 2024-04-25T11:18:52.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.063+0000 7f9229c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f922406c850 con 0x7f92241416a0 2024-04-25T11:18:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.063+0000 7f9229c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f922406cb30 con 0x7f922415eed0 2024-04-25T11:18:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.064+0000 7f921be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9224141290 0x7f922406d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45270/0 (socket says 172.21.15.7:45270) 2024-04-25T11:18:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.064+0000 7f921be00640 1 -- 172.21.15.7:0/3669994188 learned_addr learned my addr 172.21.15.7:0/3669994188 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.064+0000 7f9223e00640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f922415eed0 0x7f92240cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.064+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3470366069 0 0) 0x7f922406c570 con 0x7f9224141290 2024-04-25T11:18:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.064+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9208003520 con 0x7f9224141290 2024-04-25T11:18:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.064+0000 7f9229200640 1 --2- 172.21.15.7:0/3669994188 >> v2:172.21.15.87:3300/0 conn(0x7f92241416a0 0x7f92240b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 701822408 0 0) 0x7f9208003520 con 0x7f9224141290 2024-04-25T11:18:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f922415eed0 msgr2=0x7f92240cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f922415eed0 0x7f92240cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 >> v2:172.21.15.87:3300/0 conn(0x7f92241416a0 msgr2=0x7f92240b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 --2- 172.21.15.7:0/3669994188 >> v2:172.21.15.87:3300/0 conn(0x7f92241416a0 0x7f92240b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922406ce10 con 0x7f9224141290 2024-04-25T11:18:52.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f920c001670 con 0x7f9224141290 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9229200640 1 --2- 172.21.15.7:0/3669994188 >> v2:172.21.15.87:3300/0 conn(0x7f92241416a0 0x7f92240b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 >> v1:172.21.15.7:6789/0 conn(0x7f9224141290 legacy=0x7f922406d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 0x7f9208003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f922415eed0 0x7f9208004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9208004d40 0x7f9208005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f922406c850 con 0x7f92080039b0 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f922406cb30 con 0x7f9208004d40 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f920800bb20 con 0x7f922415eed0 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.065+0000 7f9223e00640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 0x7f9208003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.066+0000 7f921be00640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f922415eed0 0x7f9208004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.066+0000 7f9229200640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9208004d40 0x7f9208005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.066+0000 7f9223e00640 1 -- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f922415eed0 msgr2=0x7f9208004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.066+0000 7f9223e00640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f922415eed0 0x7f9208004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.066+0000 7f9223e00640 1 -- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9208004d40 msgr2=0x7f9208005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.066+0000 7f9223e00640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9208004d40 0x7f9208005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.066+0000 7f9223e00640 1 -- 172.21.15.7:0/3669994188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9208003520 con 0x7f92080039b0 2024-04-25T11:18:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.066+0000 7f921be00640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f922415eed0 0x7f9208004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.066+0000 7f9229200640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9208004d40 0x7f9208005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.067+0000 7f9223e00640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 0x7f9208003e20 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9224074c70 tx=0x7f9208005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766b0a6481443043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.067+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214030070 con 0x7f92080039b0 2024-04-25T11:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.067+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921402ada0 con 0x7f92080039b0 2024-04-25T11:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.067+0000 7f9223400640 1 -- 172.21.15.7:0/3669994188 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921400b610 con 0x7f92080039b0 2024-04-25T11:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.068+0000 7f9229c00640 1 -- 172.21.15.7:0/3669994188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 msgr2=0x7f9208003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.068+0000 7f9229c00640 1 --2- 172.21.15.7:0/3669994188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 0x7f9208003e20 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9224074c70 tx=0x7f9208005830 comp rx=0 tx=0).stop 2024-04-25T11:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.068+0000 7f9229200640 1 -- 172.21.15.7:0/3669994188 reap_dead start 2024-04-25T11:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.068+0000 7f9229c00640 1 -- 172.21.15.7:0/3669994188 shutdown_connections 2024-04-25T11:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.068+0000 7f9229c00640 1 -- 172.21.15.7:0/3669994188 >> 172.21.15.7:0/3669994188 conn(0x7f92240d1540 msgr2=0x7f92240d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.068+0000 7f9229c00640 1 -- 172.21.15.7:0/3669994188 shutdown_connections 2024-04-25T11:18:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.068+0000 7f9229c00640 1 -- 172.21.15.7:0/3669994188 wait complete. 2024-04-25T11:18:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.070+0000 7f9229c00640 1 Processor -- start 2024-04-25T11:18:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.071+0000 7f9229c00640 1 -- start start 2024-04-25T11:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.072+0000 7f9229c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 0x7f9224153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.072+0000 7f9229c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92240ccc00 0x7f92240c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.072+0000 7f9229c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92240cd010 0x7f92240c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.072+0000 7f9229c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f922406dd20 con 0x7f92080039b0 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.072+0000 7f9229c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9224141ab0 con 0x7f92240cd010 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.072+0000 7f9229c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f922406dea0 con 0x7f92240ccc00 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.072+0000 7f9223e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 0x7f9224153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.072+0000 7f9223e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 0x7f9224153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46180/0 (socket says 172.21.15.7:46180) 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.072+0000 7f9223e00640 1 -- 172.21.15.7:0/1311733234 learned_addr learned my addr 172.21.15.7:0/1311733234 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.073+0000 7f921be00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92240ccc00 0x7f92240c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.073+0000 7f9229200640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92240cd010 0x7f92240c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.073+0000 7f9223e00640 1 -- 172.21.15.7:0/1311733234 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92240ccc00 msgr2=0x7f92240c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.073+0000 7f9223e00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92240ccc00 0x7f92240c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.073+0000 7f9223e00640 1 -- 172.21.15.7:0/1311733234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92240cd010 msgr2=0x7f92240c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.073+0000 7f9223e00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92240cd010 0x7f92240c6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.073+0000 7f9223e00640 1 -- 172.21.15.7:0/1311733234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9224154610 con 0x7f92080039b0 2024-04-25T11:18:52.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.073+0000 7f921be00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92240ccc00 0x7f92240c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:52.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.073+0000 7f9223e00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 0x7f9224153ce0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f921400bb00 tx=0x7f9214004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:52.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.073+0000 7f9220c00640 1 -- 172.21.15.7:0/1311733234 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9214043020 con 0x7f92080039b0 2024-04-25T11:18:52.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.074+0000 7f9220c00640 1 -- 172.21.15.7:0/1311733234 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9214035eb0 con 0x7f92080039b0 2024-04-25T11:18:52.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.074+0000 7f9220c00640 1 -- 172.21.15.7:0/1311733234 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921403f8a0 con 0x7f92080039b0 2024-04-25T11:18:52.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.074+0000 7f9229c00640 1 -- 172.21.15.7:0/1311733234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92241548f0 con 0x7f92080039b0 2024-04-25T11:18:52.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.074+0000 7f9229c00640 1 -- 172.21.15.7:0/1311733234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9224154eb0 con 0x7f92080039b0 2024-04-25T11:18:52.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.075+0000 7f9229c00640 1 -- 172.21.15.7:0/1311733234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9224050740 con 0x7f92080039b0 2024-04-25T11:18:52.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.076+0000 7f9220c00640 1 -- 172.21.15.7:0/1311733234 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9214046020 con 0x7f92080039b0 2024-04-25T11:18:52.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.078+0000 7f9220c00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92000744e0 0x7f92000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.078+0000 7f9220c00640 1 -- 172.21.15.7:0/1311733234 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6217+0+0 (secure 0 0 0) 0x7f92140b6060 con 0x7f92080039b0 2024-04-25T11:18:52.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.078+0000 7f921be00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92000744e0 0x7f92000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.079+0000 7f921be00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92000744e0 0x7f92000769f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f920c0017e0 tx=0x7f920c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:52.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.083+0000 7f9220c00640 1 -- 172.21.15.7:0/1311733234 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f921404d030 con 0x7f92080039b0 2024-04-25T11:18:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.329+0000 7f9229c00640 1 -- 172.21.15.7:0/1311733234 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f92240687a0 con 0x7f92080039b0 2024-04-25T11:18:52.329 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-04-25T11:18:52.330+0000 7fac34e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:18:52.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.333+0000 7f9220c00640 1 -- 172.21.15.7:0/1311733234 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) ==== 161+0+0 (secure 0 0 0) 0x7f9214080550 con 0x7f92080039b0 2024-04-25T11:18:52.332 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:18:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.337+0000 7f9229c00640 1 -- 172.21.15.7:0/1311733234 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92000744e0 msgr2=0x7f92000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.337+0000 7f9229c00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92000744e0 0x7f92000769f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f920c0017e0 tx=0x7f920c009040 comp rx=0 tx=0).stop 2024-04-25T11:18:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.337+0000 7f9229c00640 1 -- 172.21.15.7:0/1311733234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 msgr2=0x7f9224153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.337+0000 7f9229c00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 0x7f9224153ce0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f921400bb00 tx=0x7f9214004730 comp rx=0 tx=0).stop 2024-04-25T11:18:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.337+0000 7f9229c00640 1 -- 172.21.15.7:0/1311733234 shutdown_connections 2024-04-25T11:18:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.337+0000 7f9229c00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92000744e0 0x7f92000769f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.337+0000 7f9229c00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92240cd010 0x7f92240c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.337+0000 7f9229c00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92240ccc00 0x7f92240c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.337+0000 7f9229c00640 1 --2- 172.21.15.7:0/1311733234 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92080039b0 0x7f9224153ce0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.337+0000 7f9229c00640 1 -- 172.21.15.7:0/1311733234 >> 172.21.15.7:0/1311733234 conn(0x7f92240d1540 msgr2=0x7f922406fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.338+0000 7f9229c00640 1 -- 172.21.15.7:0/1311733234 shutdown_connections 2024-04-25T11:18:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.338+0000 7f9229c00640 1 -- 172.21.15.7:0/1311733234 wait complete. 2024-04-25T11:18:52.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:18:52.508 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:18:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.660+0000 7f15fce00640 1 Processor -- start 2024-04-25T11:18:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.660+0000 7f15fce00640 1 -- start start 2024-04-25T11:18:52.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.663+0000 7f15fce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f15f815dfb0 0x7f15f80b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.663+0000 7f15fce00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15f80d0b40 0x7f15f806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.663+0000 7f15fce00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15f815e3c0 con 0x7f15f80d0f50 2024-04-25T11:18:52.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.663+0000 7f15fce00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f15f80b09b0 con 0x7f15f80d0b40 2024-04-25T11:18:52.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.663+0000 7f15fce00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f15f80cf800 con 0x7f15f815dfb0 2024-04-25T11:18:52.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.663+0000 7f15ed400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15f80d0b40 0x7f15f806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.663+0000 7f15ed400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15f80d0b40 0x7f15f806f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46210/0 (socket says 172.21.15.7:46210) 2024-04-25T11:18:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.663+0000 7f15ed400640 1 -- 172.21.15.7:0/721938218 learned_addr learned my addr 172.21.15.7:0/721938218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.664+0000 7f15ed400640 1 -- 172.21.15.7:0/721938218 >> v2:172.21.15.87:3300/0 conn(0x7f15f815dfb0 msgr2=0x7f15f80b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.664+0000 7f15ed400640 1 --2- 172.21.15.7:0/721938218 >> v2:172.21.15.87:3300/0 conn(0x7f15f815dfb0 0x7f15f80b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.664+0000 7f15ed400640 1 -- 172.21.15.7:0/721938218 >> v1:172.21.15.87:6789/0 conn(0x7f15f80d0f50 legacy=0x7f15f8161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.664+0000 7f15ed400640 1 -- 172.21.15.7:0/721938218 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f80cfaa0 con 0x7f15f80d0b40 2024-04-25T11:18:52.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.665+0000 7f15ed400640 1 --2- 172.21.15.7:0/721938218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f80d0b40 0x7f15f806f3a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f15f8072b10 tx=0x7f15e002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dddc79e1f433633 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:52.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.666+0000 7f15f5400640 1 -- 172.21.15.7:0/721938218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0037070 con 0x7f15f80d0b40 2024-04-25T11:18:52.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.666+0000 7f15f5400640 1 -- 172.21.15.7:0/721938218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e0032440 con 0x7f15f80d0b40 2024-04-25T11:18:52.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.666+0000 7f15f5400640 1 -- 172.21.15.7:0/721938218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0032810 con 0x7f15f80d0b40 2024-04-25T11:18:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.667+0000 7f15fce00640 1 -- 172.21.15.7:0/721938218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f80d0b40 msgr2=0x7f15f806f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.667+0000 7f15fce00640 1 --2- 172.21.15.7:0/721938218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f80d0b40 0x7f15f806f3a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f15f8072b10 tx=0x7f15e002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.667+0000 7f15fce00640 1 -- 172.21.15.7:0/721938218 shutdown_connections 2024-04-25T11:18:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.667+0000 7f15fce00640 1 --2- 172.21.15.7:0/721938218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f80d0b40 0x7f15f806f3a0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.667+0000 7f15fce00640 1 --2- 172.21.15.7:0/721938218 >> v2:172.21.15.87:3300/0 conn(0x7f15f815dfb0 0x7f15f80b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.667+0000 7f15fce00640 1 -- 172.21.15.7:0/721938218 >> 172.21.15.7:0/721938218 conn(0x7f15f80742c0 msgr2=0x7f15f806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:52.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.667+0000 7f15fce00640 1 -- 172.21.15.7:0/721938218 shutdown_connections 2024-04-25T11:18:52.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.667+0000 7f15fce00640 1 -- 172.21.15.7:0/721938218 wait complete. 2024-04-25T11:18:52.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.669+0000 7f15fce00640 1 Processor -- start 2024-04-25T11:18:52.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.669+0000 7f15fce00640 1 -- start start 2024-04-25T11:18:52.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.671+0000 7f15fce00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15f80d0b40 0x7f15f80b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.671+0000 7f15fce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15f80d0f50 0x7f15f81f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.671+0000 7f15fce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f815dfb0 0x7f15f81f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.671+0000 7f15fce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15f80d0340 con 0x7f15f815dfb0 2024-04-25T11:18:52.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.671+0000 7f15fce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f15f80d0060 con 0x7f15f80d0f50 2024-04-25T11:18:52.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.671+0000 7f15fce00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f15f80d0620 con 0x7f15f80d0b40 2024-04-25T11:18:52.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.671+0000 7f15f6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f815dfb0 0x7f15f81f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.671+0000 7f15f6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f815dfb0 0x7f15f81f7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46220/0 (socket says 172.21.15.7:46220) 2024-04-25T11:18:52.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.671+0000 7f15f6800640 1 -- 172.21.15.7:0/1181077730 learned_addr learned my addr 172.21.15.7:0/1181077730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:52.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.672+0000 7f15f5e00640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15f80d0b40 0x7f15f80b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.672+0000 7f15ed400640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15f80d0f50 0x7f15f81f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.672+0000 7f15f6800640 1 -- 172.21.15.7:0/1181077730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15f80d0b40 msgr2=0x7f15f80b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:52.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.672+0000 7f15f6800640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15f80d0b40 0x7f15f80b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.672+0000 7f15f6800640 1 -- 172.21.15.7:0/1181077730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15f80d0f50 msgr2=0x7f15f81f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:52.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.672+0000 7f15f6800640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15f80d0f50 0x7f15f81f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.672+0000 7f15f6800640 1 -- 172.21.15.7:0/1181077730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f80b5f80 con 0x7f15f815dfb0 2024-04-25T11:18:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.672+0000 7f15f6800640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f815dfb0 0x7f15f81f7040 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f15f000b410 tx=0x7f15f000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.673+0000 7f15eea00640 1 -- 172.21.15.7:0/1181077730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0015030 con 0x7f15f815dfb0 2024-04-25T11:18:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.673+0000 7f15eea00640 1 -- 172.21.15.7:0/1181077730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f0004020 con 0x7f15f815dfb0 2024-04-25T11:18:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.673+0000 7f15eea00640 1 -- 172.21.15.7:0/1181077730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0004410 con 0x7f15f815dfb0 2024-04-25T11:18:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.673+0000 7f15fce00640 1 -- 172.21.15.7:0/1181077730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f8159280 con 0x7f15f815dfb0 2024-04-25T11:18:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.673+0000 7f15fce00640 1 -- 172.21.15.7:0/1181077730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f8159840 con 0x7f15f815dfb0 2024-04-25T11:18:52.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.674+0000 7f15fce00640 1 -- 172.21.15.7:0/1181077730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f80684e0 con 0x7f15f815dfb0 2024-04-25T11:18:52.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.675+0000 7f15eea00640 1 -- 172.21.15.7:0/1181077730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f15f0018070 con 0x7f15f815dfb0 2024-04-25T11:18:52.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.676+0000 7f15eea00640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15d40744b0 0x7f15d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:52.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.676+0000 7f15eea00640 1 -- 172.21.15.7:0/1181077730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6303+0+0 (secure 0 0 0) 0x7f15f0094fa0 con 0x7f15f815dfb0 2024-04-25T11:18:52.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.677+0000 7f15f5e00640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15d40744b0 0x7f15d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:52.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.677+0000 7f15f5e00640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15d40744b0 0x7f15d40769c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f15f8072df0 tx=0x7f15e80094f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:52.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.681+0000 7f15eea00640 1 -- 172.21.15.7:0/1181077730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15f80684e0 con 0x7f15f815dfb0 2024-04-25T11:18:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:52.932+0000 7f15fce00640 1 -- 172.21.15.7:0/1181077730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f15f8050740 con 0x7f15f815dfb0 2024-04-25T11:18:53.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.521+0000 7f15eea00640 1 -- 172.21.15.7:0/1181077730 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7f15f005f4d0 con 0x7f15f815dfb0 2024-04-25T11:18:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.525+0000 7f15fce00640 1 -- 172.21.15.7:0/1181077730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15d40744b0 msgr2=0x7f15d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.525+0000 7f15fce00640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15d40744b0 0x7f15d40769c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f15f8072df0 tx=0x7f15e80094f0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.525+0000 7f15fce00640 1 -- 172.21.15.7:0/1181077730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f815dfb0 msgr2=0x7f15f81f7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.525+0000 7f15fce00640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f815dfb0 0x7f15f81f7040 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f15f000b410 tx=0x7f15f000b8d0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.525+0000 7f15fce00640 1 -- 172.21.15.7:0/1181077730 shutdown_connections 2024-04-25T11:18:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.525+0000 7f15fce00640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15d40744b0 0x7f15d40769c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.525+0000 7f15fce00640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f815dfb0 0x7f15f81f7040 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.525+0000 7f15fce00640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15f80d0f50 0x7f15f81f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.525+0000 7f15fce00640 1 --2- 172.21.15.7:0/1181077730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15f80d0b40 0x7f15f80b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.525+0000 7f15fce00640 1 -- 172.21.15.7:0/1181077730 >> 172.21.15.7:0/1181077730 conn(0x7f15f80742c0 msgr2=0x7f15f806fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:53.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.525+0000 7f15fce00640 1 -- 172.21.15.7:0/1181077730 shutdown_connections 2024-04-25T11:18:53.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.526+0000 7f15fce00640 1 -- 172.21.15.7:0/1181077730 wait complete. 2024-04-25T11:18:53.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:18:53.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.853+0000 7fb349a00640 1 Processor -- start 2024-04-25T11:18:53.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.853+0000 7fb349a00640 1 -- start start 2024-04-25T11:18:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.855+0000 7fb349a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb344075bf0 0x7fb3440b1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.855+0000 7fb349a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb34406d730 0x7fb3441607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.856+0000 7fb349a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3440b1ec0 con 0x7fb3440b03f0 2024-04-25T11:18:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.856+0000 7fb349a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb3440b2d80 con 0x7fb34406d730 2024-04-25T11:18:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.856+0000 7fb349a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb3440b2170 con 0x7fb344075bf0 2024-04-25T11:18:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.856+0000 7fb349000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb34406d730 0x7fb3441607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.856+0000 7fb349000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb34406d730 0x7fb3441607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46228/0 (socket says 172.21.15.7:46228) 2024-04-25T11:18:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.856+0000 7fb349000640 1 -- 172.21.15.7:0/320647290 learned_addr learned my addr 172.21.15.7:0/320647290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.856+0000 7fb343e00640 1 --2- 172.21.15.7:0/320647290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb344075bf0 0x7fb3440b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.857+0000 7fb349000640 1 -- 172.21.15.7:0/320647290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb344075bf0 msgr2=0x7fb3440b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.857+0000 7fb349000640 1 --2- 172.21.15.7:0/320647290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb344075bf0 0x7fb3440b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.857+0000 7fb349000640 1 -- 172.21.15.7:0/320647290 >> v1:172.21.15.87:6789/0 conn(0x7fb3440b03f0 legacy=0x7fb3440b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.857+0000 7fb349000640 1 -- 172.21.15.7:0/320647290 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3440b2450 con 0x7fb34406d730 2024-04-25T11:18:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.858+0000 7fb349000640 1 --2- 172.21.15.7:0/320647290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34406d730 0x7fb3441607c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009e50 tx=0x7fb33c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a67ef1cd18e8e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.858+0000 7fb343400640 1 -- 172.21.15.7:0/320647290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c036070 con 0x7fb34406d730 2024-04-25T11:18:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.858+0000 7fb343400640 1 -- 172.21.15.7:0/320647290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33c031600 con 0x7fb34406d730 2024-04-25T11:18:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.858+0000 7fb343400640 1 -- 172.21.15.7:0/320647290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0319d0 con 0x7fb34406d730 2024-04-25T11:18:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.859+0000 7fb349a00640 1 -- 172.21.15.7:0/320647290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34406d730 msgr2=0x7fb3441607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.859+0000 7fb349a00640 1 --2- 172.21.15.7:0/320647290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34406d730 0x7fb3441607c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009e50 tx=0x7fb33c02a710 comp rx=0 tx=0).stop 2024-04-25T11:18:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.859+0000 7fb349a00640 1 -- 172.21.15.7:0/320647290 shutdown_connections 2024-04-25T11:18:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.859+0000 7fb349a00640 1 --2- 172.21.15.7:0/320647290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34406d730 0x7fb3441607c0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.859+0000 7fb349a00640 1 --2- 172.21.15.7:0/320647290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb344075bf0 0x7fb3440b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.859+0000 7fb349a00640 1 -- 172.21.15.7:0/320647290 >> 172.21.15.7:0/320647290 conn(0x7fb344073bf0 msgr2=0x7fb34409b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:53.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.859+0000 7fb349a00640 1 -- 172.21.15.7:0/320647290 shutdown_connections 2024-04-25T11:18:53.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.859+0000 7fb349a00640 1 -- 172.21.15.7:0/320647290 wait complete. 2024-04-25T11:18:53.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.862+0000 7fb349a00640 1 Processor -- start 2024-04-25T11:18:53.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.862+0000 7fb349a00640 1 -- start start 2024-04-25T11:18:53.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.864+0000 7fb349a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb34406d730 0x7fb344157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:53.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.864+0000 7fb349a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb344075bf0 0x7fb34409f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:53.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.864+0000 7fb349a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440b03f0 0x7fb34409c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:53.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.864+0000 7fb349a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3440b0f10 con 0x7fb3440b03f0 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.864+0000 7fb349a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb3440b2a10 con 0x7fb344075bf0 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.864+0000 7fb349a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb34409eb90 con 0x7fb34406d730 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.864+0000 7fb349000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440b03f0 0x7fb34409c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.864+0000 7fb349000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440b03f0 0x7fb34409c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46240/0 (socket says 172.21.15.7:46240) 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.864+0000 7fb349000640 1 -- 172.21.15.7:0/200548454 learned_addr learned my addr 172.21.15.7:0/200548454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.864+0000 7fb343e00640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb34406d730 0x7fb344157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.864+0000 7fb33be00640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb344075bf0 0x7fb34409f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.865+0000 7fb349000640 1 -- 172.21.15.7:0/200548454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb34406d730 msgr2=0x7fb344157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.865+0000 7fb349000640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb34406d730 0x7fb344157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.865+0000 7fb349000640 1 -- 172.21.15.7:0/200548454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb344075bf0 msgr2=0x7fb34409f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.865+0000 7fb349000640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb344075bf0 0x7fb34409f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:53.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.865+0000 7fb349000640 1 -- 172.21.15.7:0/200548454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34409f820 con 0x7fb3440b03f0 2024-04-25T11:18:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.865+0000 7fb349000640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440b03f0 0x7fb34409c650 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb33c034040 tx=0x7fb33c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.865+0000 7fb340c00640 1 -- 172.21.15.7:0/200548454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c036050 con 0x7fb3440b03f0 2024-04-25T11:18:53.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.865+0000 7fb340c00640 1 -- 172.21.15.7:0/200548454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33c042070 con 0x7fb3440b03f0 2024-04-25T11:18:53.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.865+0000 7fb340c00640 1 -- 172.21.15.7:0/200548454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c009510 con 0x7fb3440b03f0 2024-04-25T11:18:53.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.865+0000 7fb349a00640 1 -- 172.21.15.7:0/200548454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb344156eb0 con 0x7fb3440b03f0 2024-04-25T11:18:53.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.866+0000 7fb349a00640 1 -- 172.21.15.7:0/200548454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb344157470 con 0x7fb3440b03f0 2024-04-25T11:18:53.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.867+0000 7fb349a00640 1 -- 172.21.15.7:0/200548454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb344050740 con 0x7fb3440b03f0 2024-04-25T11:18:53.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.868+0000 7fb340c00640 1 -- 172.21.15.7:0/200548454 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb33c007bd0 con 0x7fb3440b03f0 2024-04-25T11:18:53.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.869+0000 7fb340c00640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3200744b0 0x7fb3200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:53.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.869+0000 7fb340c00640 1 -- 172.21.15.7:0/200548454 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6303+0+0 (secure 0 0 0) 0x7fb33c0b63d0 con 0x7fb3440b03f0 2024-04-25T11:18:53.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.869+0000 7fb343e00640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3200744b0 0x7fb3200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.870+0000 7fb343e00640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3200744b0 0x7fb3200769c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb334006fd0 tx=0x7fb334008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:53.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:53.874+0000 7fb340c00640 1 -- 172.21.15.7:0/200548454 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb33c080870 con 0x7fb3440b03f0 2024-04-25T11:18:54.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.080+0000 7fb349a00640 1 -- 172.21.15.7:0/200548454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb344160180 con 0x7fb3440b03f0 2024-04-25T11:18:54.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.526+0000 7fb340c00640 1 -- 172.21.15.7:0/200548454 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7fb33c080170 con 0x7fb3440b03f0 2024-04-25T11:18:54.525 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:18:54.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.530+0000 7fb349a00640 1 -- 172.21.15.7:0/200548454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3200744b0 msgr2=0x7fb3200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:54.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.530+0000 7fb349a00640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3200744b0 0x7fb3200769c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb334006fd0 tx=0x7fb334008040 comp rx=0 tx=0).stop 2024-04-25T11:18:54.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.530+0000 7fb349a00640 1 -- 172.21.15.7:0/200548454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440b03f0 msgr2=0x7fb34409c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:54.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.530+0000 7fb349a00640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440b03f0 0x7fb34409c650 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb33c034040 tx=0x7fb33c004a40 comp rx=0 tx=0).stop 2024-04-25T11:18:54.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.530+0000 7fb349a00640 1 -- 172.21.15.7:0/200548454 shutdown_connections 2024-04-25T11:18:54.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.530+0000 7fb349a00640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3200744b0 0x7fb3200769c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:54.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.530+0000 7fb349a00640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3440b03f0 0x7fb34409c650 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:54.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.530+0000 7fb349a00640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb344075bf0 0x7fb34409f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:54.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.530+0000 7fb349a00640 1 --2- 172.21.15.7:0/200548454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb34406d730 0x7fb344157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:54.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.530+0000 7fb349a00640 1 -- 172.21.15.7:0/200548454 >> 172.21.15.7:0/200548454 conn(0x7fb344073bf0 msgr2=0x7fb34415b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:54.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.532+0000 7fb349a00640 1 -- 172.21.15.7:0/200548454 shutdown_connections 2024-04-25T11:18:54.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.534+0000 7fb349a00640 1 -- 172.21.15.7:0/200548454 wait complete. 2024-04-25T11:18:54.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:18:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.842+0000 7f2d79800640 1 Processor -- start 2024-04-25T11:18:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.842+0000 7f2d79800640 1 -- start start 2024-04-25T11:18:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.845+0000 7f2d79800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d7406d580 0x7f2d7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.845+0000 7f2d79800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d7406ded0 con 0x7f2d740b4c60 2024-04-25T11:18:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.845+0000 7f2d79800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d74070e40 con 0x7f2d74143220 2024-04-25T11:18:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.845+0000 7f2d79800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2d740d02f0 con 0x7f2d7406d580 2024-04-25T11:18:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.845+0000 7f2d6be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d7406d580 0x7f2d7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.845+0000 7f2d6be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2d7406d580 0x7f2d7406d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46246/0 (socket says 172.21.15.7:46246) 2024-04-25T11:18:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.845+0000 7f2d78e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2d740b4c60 0x7f2d740b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45286/0 (socket says 172.21.15.7:45286) 2024-04-25T11:18:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.845+0000 7f2d6be00640 1 -- 172.21.15.7:0/1435844521 learned_addr learned my addr 172.21.15.7:0/1435844521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.846+0000 7f2d6be00640 1 -- 172.21.15.7:0/1435844521 >> v1:172.21.15.87:6789/0 conn(0x7f2d74143220 legacy=0x7f2d740d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.846+0000 7f2d6be00640 1 -- 172.21.15.7:0/1435844521 >> v1:172.21.15.7:6789/0 conn(0x7f2d740b4c60 legacy=0x7f2d740b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.846+0000 7f2d73400640 1 -- 172.21.15.7:0/1435844521 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 106210205 0 0) 0x7f2d6c001490 con 0x7f2d740b4c60 2024-04-25T11:18:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.846+0000 7f2d6be00640 1 -- 172.21.15.7:0/1435844521 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d740d05d0 con 0x7f2d7406d580 2024-04-25T11:18:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.847+0000 7f2d6be00640 1 --2- 172.21.15.7:0/1435844521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7406d580 0x7f2d7406d990 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009870 tx=0x7f2d5c02c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29d796fb6ad3560 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.848+0000 7f2d73400640 1 -- 172.21.15.7:0/1435844521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c038070 con 0x7f2d7406d580 2024-04-25T11:18:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.848+0000 7f2d73400640 1 -- 172.21.15.7:0/1435844521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d5c031710 con 0x7f2d7406d580 2024-04-25T11:18:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.848+0000 7f2d73400640 1 -- 172.21.15.7:0/1435844521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c031ae0 con 0x7f2d7406d580 2024-04-25T11:18:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.848+0000 7f2d79800640 1 -- 172.21.15.7:0/1435844521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7406d580 msgr2=0x7f2d7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.848+0000 7f2d79800640 1 --2- 172.21.15.7:0/1435844521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7406d580 0x7f2d7406d990 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009870 tx=0x7f2d5c02c480 comp rx=0 tx=0).stop 2024-04-25T11:18:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.848+0000 7f2d79800640 1 -- 172.21.15.7:0/1435844521 shutdown_connections 2024-04-25T11:18:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.848+0000 7f2d79800640 1 --2- 172.21.15.7:0/1435844521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7406d580 0x7f2d7406d990 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.848+0000 7f2d79800640 1 -- 172.21.15.7:0/1435844521 >> 172.21.15.7:0/1435844521 conn(0x7f2d74073160 msgr2=0x7f2d74073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.849+0000 7f2d79800640 1 -- 172.21.15.7:0/1435844521 shutdown_connections 2024-04-25T11:18:54.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.849+0000 7f2d79800640 1 -- 172.21.15.7:0/1435844521 wait complete. 2024-04-25T11:18:54.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.851+0000 7f2d79800640 1 Processor -- start 2024-04-25T11:18:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.851+0000 7f2d79800640 1 -- start start 2024-04-25T11:18:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d79800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7406d580 0x7f2d7413b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d79800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d740b4c60 0x7f2d74151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d79800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d74143220 0x7f2d7413c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d79800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d740d00b0 con 0x7f2d7406d580 2024-04-25T11:18:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d79800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2d740cff30 con 0x7f2d740b4c60 2024-04-25T11:18:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d79800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2d740700d0 con 0x7f2d74143220 2024-04-25T11:18:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7406d580 0x7f2d7413b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7406d580 0x7f2d7413b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46262/0 (socket says 172.21.15.7:46262) 2024-04-25T11:18:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d73e00640 1 -- 172.21.15.7:0/2912256114 learned_addr learned my addr 172.21.15.7:0/2912256114 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d6be00640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d740b4c60 0x7f2d74151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d78e00640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d74143220 0x7f2d7413c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.853+0000 7f2d73e00640 1 -- 172.21.15.7:0/2912256114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d74143220 msgr2=0x7f2d7413c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.854+0000 7f2d73e00640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d74143220 0x7f2d7413c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.854+0000 7f2d73e00640 1 -- 172.21.15.7:0/2912256114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d740b4c60 msgr2=0x7f2d74151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.854+0000 7f2d73e00640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d740b4c60 0x7f2d74151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.854+0000 7f2d73e00640 1 -- 172.21.15.7:0/2912256114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7414fc20 con 0x7f2d7406d580 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.854+0000 7f2d73e00640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7406d580 0x7f2d7413b7a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c430 tx=0x7f2d6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.854+0000 7f2d70c00640 1 -- 172.21.15.7:0/2912256114 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64017070 con 0x7f2d7406d580 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.854+0000 7f2d70c00640 1 -- 172.21.15.7:0/2912256114 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d640052d0 con 0x7f2d7406d580 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.854+0000 7f2d79800640 1 -- 172.21.15.7:0/2912256114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d74150120 con 0x7f2d7406d580 2024-04-25T11:18:54.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.854+0000 7f2d70c00640 1 -- 172.21.15.7:0/2912256114 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64003830 con 0x7f2d7406d580 2024-04-25T11:18:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.854+0000 7f2d79800640 1 -- 172.21.15.7:0/2912256114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d74141ea0 con 0x7f2d7406d580 2024-04-25T11:18:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.856+0000 7f2d79800640 1 -- 172.21.15.7:0/2912256114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d74050740 con 0x7f2d7406d580 2024-04-25T11:18:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.857+0000 7f2d70c00640 1 -- 172.21.15.7:0/2912256114 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d6401a070 con 0x7f2d7406d580 2024-04-25T11:18:54.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.858+0000 7f2d70c00640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d500744b0 0x7f2d500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:54.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.859+0000 7f2d70c00640 1 -- 172.21.15.7:0/2912256114 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5865+0+0 (secure 0 0 0) 0x7f2d64094700 con 0x7f2d7406d580 2024-04-25T11:18:54.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.859+0000 7f2d6be00640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d500744b0 0x7f2d500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:54.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.860+0000 7f2d6be00640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d500744b0 0x7f2d500769c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c035000 tx=0x7f2d5c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:54.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:54.864+0000 7f2d70c00640 1 -- 172.21.15.7:0/2912256114 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d6405ed50 con 0x7f2d7406d580 2024-04-25T11:18:55.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.069+0000 7f2d79800640 1 -- 172.21.15.7:0/2912256114 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2d74074320 con 0x7f2d7406d580 2024-04-25T11:18:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.544+0000 7f2d70c00640 1 -- 172.21.15.7:0/2912256114 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7f2d6405e650 con 0x7f2d7406d580 2024-04-25T11:18:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-25T11:18:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.547+0000 7f2d79800640 1 -- 172.21.15.7:0/2912256114 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d500744b0 msgr2=0x7f2d500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.547+0000 7f2d79800640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d500744b0 0x7f2d500769c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c035000 tx=0x7f2d5c008030 comp rx=0 tx=0).stop 2024-04-25T11:18:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.547+0000 7f2d79800640 1 -- 172.21.15.7:0/2912256114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7406d580 msgr2=0x7f2d7413b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.547+0000 7f2d79800640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7406d580 0x7f2d7413b7a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c430 tx=0x7f2d6400c8f0 comp rx=0 tx=0).stop 2024-04-25T11:18:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.548+0000 7f2d79800640 1 -- 172.21.15.7:0/2912256114 shutdown_connections 2024-04-25T11:18:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.548+0000 7f2d79800640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d500744b0 0x7f2d500769c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.548+0000 7f2d79800640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d74143220 0x7f2d7413c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.548+0000 7f2d79800640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d740b4c60 0x7f2d74151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.548+0000 7f2d79800640 1 --2- 172.21.15.7:0/2912256114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d7406d580 0x7f2d7413b7a0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.548+0000 7f2d79800640 1 -- 172.21.15.7:0/2912256114 >> 172.21.15.7:0/2912256114 conn(0x7f2d74073160 msgr2=0x7f2d740b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.548+0000 7f2d79800640 1 -- 172.21.15.7:0/2912256114 shutdown_connections 2024-04-25T11:18:55.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.548+0000 7f2d79800640 1 -- 172.21.15.7:0/2912256114 wait complete. 2024-04-25T11:18:55.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-04-25T11:18:55.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.869+0000 7fda83800640 1 Processor -- start 2024-04-25T11:18:55.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.869+0000 7fda83800640 1 -- start start 2024-04-25T11:18:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.871+0000 7fda83800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda7c0b21c0 0x7fda7c15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.871+0000 7fda83800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda7c161230 0x7fda7c06ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.871+0000 7fda83800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda7c0b29a0 con 0x7fda7c0d1f80 2024-04-25T11:18:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.871+0000 7fda83800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fda7c06c570 con 0x7fda7c161230 2024-04-25T11:18:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.872+0000 7fda83800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c0d2540 con 0x7fda7c0b21c0 2024-04-25T11:18:55.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.872+0000 7fda81800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda7c161230 0x7fda7c06ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:55.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.872+0000 7fda80e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fda7c0d1f80 0x7fda7c0b1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45294/0 (socket says 172.21.15.7:45294) 2024-04-25T11:18:55.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.872+0000 7fda80e00640 1 -- 172.21.15.7:0/4018238001 learned_addr learned my addr 172.21.15.7:0/4018238001 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:55.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.872+0000 7fda7be00640 1 --2- 172.21.15.7:0/4018238001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda7c0b21c0 0x7fda7c15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:55.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.872+0000 7fda7b400640 1 -- 172.21.15.7:0/4018238001 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3263174907 0 0) 0x7fda7c0b29a0 con 0x7fda7c0d1f80 2024-04-25T11:18:55.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.872+0000 7fda7b400640 1 -- 172.21.15.7:0/4018238001 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda64003520 con 0x7fda7c0d1f80 2024-04-25T11:18:55.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.873+0000 7fda81800640 1 -- 172.21.15.7:0/4018238001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda7c0b21c0 msgr2=0x7fda7c15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:55.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.873+0000 7fda81800640 1 --2- 172.21.15.7:0/4018238001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda7c0b21c0 0x7fda7c15dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:55.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.873+0000 7fda81800640 1 -- 172.21.15.7:0/4018238001 >> v1:172.21.15.7:6789/0 conn(0x7fda7c0d1f80 legacy=0x7fda7c0b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:55.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.873+0000 7fda81800640 1 -- 172.21.15.7:0/4018238001 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c0d2820 con 0x7fda7c161230 2024-04-25T11:18:55.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.874+0000 7fda81800640 1 --2- 172.21.15.7:0/4018238001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c161230 0x7fda7c06ca00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fda74009d90 tx=0x7fda7402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd25ab573394c120 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:55.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.874+0000 7fda7b400640 1 -- 172.21.15.7:0/4018238001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74031600 con 0x7fda7c161230 2024-04-25T11:18:55.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.874+0000 7fda7b400640 1 -- 172.21.15.7:0/4018238001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda740317a0 con 0x7fda7c161230 2024-04-25T11:18:55.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.874+0000 7fda7b400640 1 -- 172.21.15.7:0/4018238001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74031b70 con 0x7fda7c161230 2024-04-25T11:18:55.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.875+0000 7fda83800640 1 -- 172.21.15.7:0/4018238001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c161230 msgr2=0x7fda7c06ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:55.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.875+0000 7fda83800640 1 --2- 172.21.15.7:0/4018238001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c161230 0x7fda7c06ca00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fda74009d90 tx=0x7fda7402a5d0 comp rx=0 tx=0).stop 2024-04-25T11:18:55.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.875+0000 7fda83800640 1 -- 172.21.15.7:0/4018238001 shutdown_connections 2024-04-25T11:18:55.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.875+0000 7fda83800640 1 --2- 172.21.15.7:0/4018238001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c161230 0x7fda7c06ca00 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:55.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.875+0000 7fda83800640 1 --2- 172.21.15.7:0/4018238001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda7c0b21c0 0x7fda7c15dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:55.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.875+0000 7fda83800640 1 -- 172.21.15.7:0/4018238001 >> 172.21.15.7:0/4018238001 conn(0x7fda7c073d30 msgr2=0x7fda7c09b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:55.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.875+0000 7fda83800640 1 -- 172.21.15.7:0/4018238001 shutdown_connections 2024-04-25T11:18:55.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.875+0000 7fda83800640 1 -- 172.21.15.7:0/4018238001 wait complete. 2024-04-25T11:18:55.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.878+0000 7fda83800640 1 Processor -- start 2024-04-25T11:18:55.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.878+0000 7fda83800640 1 -- start start 2024-04-25T11:18:55.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.879+0000 7fda83800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda7c0d1f80 0x7fda7c0d1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:55.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.879+0000 7fda83800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda7c161230 0x7fda7c1483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:55.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.879+0000 7fda83800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c0cf3a0 0x7fda7c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:55.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.879+0000 7fda83800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c161640 con 0x7fda7c0cf3a0 2024-04-25T11:18:55.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.879+0000 7fda83800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c0d2de0 con 0x7fda7c161230 2024-04-25T11:18:55.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.879+0000 7fda83800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c06cf40 con 0x7fda7c0d1f80 2024-04-25T11:18:55.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.880+0000 7fda81800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c0cf3a0 0x7fda7c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:55.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.880+0000 7fda81800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c0cf3a0 0x7fda7c0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46270/0 (socket says 172.21.15.7:46270) 2024-04-25T11:18:55.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.880+0000 7fda81800640 1 -- 172.21.15.7:0/2389492153 learned_addr learned my addr 172.21.15.7:0/2389492153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:55.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.880+0000 7fda80e00640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda7c0d1f80 0x7fda7c0d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:55.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.880+0000 7fda7be00640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda7c161230 0x7fda7c1483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.880+0000 7fda81800640 1 -- 172.21.15.7:0/2389492153 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda7c0d1f80 msgr2=0x7fda7c0d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.880+0000 7fda81800640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda7c0d1f80 0x7fda7c0d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.880+0000 7fda81800640 1 -- 172.21.15.7:0/2389492153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda7c161230 msgr2=0x7fda7c1483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.880+0000 7fda81800640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda7c161230 0x7fda7c1483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.880+0000 7fda81800640 1 -- 172.21.15.7:0/2389492153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c0d04f0 con 0x7fda7c0cf3a0 2024-04-25T11:18:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.880+0000 7fda80e00640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda7c0d1f80 0x7fda7c0d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:55.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.881+0000 7fda81800640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c0cf3a0 0x7fda7c0cf7b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fda74009eb0 tx=0x7fda74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:55.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.881+0000 7fda78c00640 1 -- 172.21.15.7:0/2389492153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7403f070 con 0x7fda7c0cf3a0 2024-04-25T11:18:55.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.881+0000 7fda78c00640 1 -- 172.21.15.7:0/2389492153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda74030b60 con 0x7fda7c0cf3a0 2024-04-25T11:18:55.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.881+0000 7fda78c00640 1 -- 172.21.15.7:0/2389492153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74009450 con 0x7fda7c0cf3a0 2024-04-25T11:18:55.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.881+0000 7fda83800640 1 -- 172.21.15.7:0/2389492153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7c0d07d0 con 0x7fda7c0cf3a0 2024-04-25T11:18:55.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.881+0000 7fda83800640 1 -- 172.21.15.7:0/2389492153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7c0d0d90 con 0x7fda7c0cf3a0 2024-04-25T11:18:55.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.882+0000 7fda83800640 1 -- 172.21.15.7:0/2389492153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7c050740 con 0x7fda7c0cf3a0 2024-04-25T11:18:55.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.884+0000 7fda78c00640 1 -- 172.21.15.7:0/2389492153 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fda7403b410 con 0x7fda7c0cf3a0 2024-04-25T11:18:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.885+0000 7fda78c00640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fda5c0744b0 0x7fda5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.885+0000 7fda78c00640 1 -- 172.21.15.7:0/2389492153 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7fda740b67b0 con 0x7fda7c0cf3a0 2024-04-25T11:18:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.885+0000 7fda80e00640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fda5c0744b0 0x7fda5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.886+0000 7fda80e00640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fda5c0744b0 0x7fda5c0769c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fda70000f60 tx=0x7fda70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:55.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:55.890+0000 7fda78c00640 1 -- 172.21.15.7:0/2389492153 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fda740809c0 con 0x7fda7c0cf3a0 2024-04-25T11:18:56.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.099+0000 7fda83800640 1 -- 172.21.15.7:0/2389492153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7fda7c06cd00 con 0x7fda7c0cf3a0 2024-04-25T11:18:56.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.105+0000 7fda78c00640 1 -- 172.21.15.7:0/2389492153 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v29) ==== 102+0+0 (secure 0 0 0) 0x7fda74084cc0 con 0x7fda7c0cf3a0 2024-04-25T11:18:56.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.108+0000 7fda83800640 1 -- 172.21.15.7:0/2389492153 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fda5c0744b0 msgr2=0x7fda5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:56.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.108+0000 7fda83800640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fda5c0744b0 0x7fda5c0769c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fda70000f60 tx=0x7fda70009040 comp rx=0 tx=0).stop 2024-04-25T11:18:56.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.108+0000 7fda83800640 1 -- 172.21.15.7:0/2389492153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c0cf3a0 msgr2=0x7fda7c0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:56.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.108+0000 7fda83800640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c0cf3a0 0x7fda7c0cf7b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fda74009eb0 tx=0x7fda74004a40 comp rx=0 tx=0).stop 2024-04-25T11:18:56.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.108+0000 7fda83800640 1 -- 172.21.15.7:0/2389492153 shutdown_connections 2024-04-25T11:18:56.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.108+0000 7fda83800640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fda5c0744b0 0x7fda5c0769c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.108+0000 7fda83800640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c0cf3a0 0x7fda7c0cf7b0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.108+0000 7fda83800640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda7c161230 0x7fda7c1483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.108+0000 7fda83800640 1 --2- 172.21.15.7:0/2389492153 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fda7c0d1f80 0x7fda7c0d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.108+0000 7fda83800640 1 -- 172.21.15.7:0/2389492153 >> 172.21.15.7:0/2389492153 conn(0x7fda7c073d30 msgr2=0x7fda7c15abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:56.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.109+0000 7fda83800640 1 -- 172.21.15.7:0/2389492153 shutdown_connections 2024-04-25T11:18:56.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.109+0000 7fda83800640 1 -- 172.21.15.7:0/2389492153 wait complete. 2024-04-25T11:18:56.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-04-25T11:18:56.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.443+0000 7f537a400640 1 Processor -- start 2024-04-25T11:18:56.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.443+0000 7f537a400640 1 -- start start 2024-04-25T11:18:56.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.445+0000 7f537a400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f537415aaa0 0x7f53740affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:56.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.445+0000 7f537a400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f53740d0ee0 0x7f53740b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:56.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.445+0000 7f537a400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f537406f5b0 con 0x7f53740b4e50 2024-04-25T11:18:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.445+0000 7f537a400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f53740af290 con 0x7f537415aaa0 2024-04-25T11:18:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.445+0000 7f537a400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f53740d14a0 con 0x7f53740d0ee0 2024-04-25T11:18:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.445+0000 7f5373400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f537415aaa0 0x7f53740affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.445+0000 7f5373e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f53740b4e50 0x7f537415d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45304/0 (socket says 172.21.15.7:45304) 2024-04-25T11:18:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.445+0000 7f5373e00640 1 -- 172.21.15.7:0/145933020 learned_addr learned my addr 172.21.15.7:0/145933020 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.446+0000 7f5372a00640 1 --2- 172.21.15.7:0/145933020 >> v2:172.21.15.87:3300/0 conn(0x7f53740d0ee0 0x7f53740b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.446+0000 7f5373400640 1 -- 172.21.15.7:0/145933020 >> v2:172.21.15.87:3300/0 conn(0x7f53740d0ee0 msgr2=0x7f53740b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.446+0000 7f5373400640 1 --2- 172.21.15.7:0/145933020 >> v2:172.21.15.87:3300/0 conn(0x7f53740d0ee0 0x7f53740b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.446+0000 7f5373400640 1 -- 172.21.15.7:0/145933020 >> v1:172.21.15.7:6789/0 conn(0x7f53740b4e50 legacy=0x7f537415d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.446+0000 7f5373400640 1 -- 172.21.15.7:0/145933020 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53740af4e0 con 0x7f537415aaa0 2024-04-25T11:18:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.446+0000 7f5372000640 1 -- 172.21.15.7:0/145933020 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3513469990 0 0) 0x7f537406f5b0 con 0x7f53740b4e50 2024-04-25T11:18:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.446+0000 7f5372a00640 1 --2- 172.21.15.7:0/145933020 >> v2:172.21.15.87:3300/0 conn(0x7f53740d0ee0 0x7f53740b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.447+0000 7f5373400640 1 --2- 172.21.15.7:0/145933020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f537415aaa0 0x7f53740affc0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f53740af410 tx=0x7f536402c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57197dc638bc4ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.447+0000 7f5372000640 1 -- 172.21.15.7:0/145933020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364031710 con 0x7f537415aaa0 2024-04-25T11:18:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.447+0000 7f5372000640 1 -- 172.21.15.7:0/145933020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53640318b0 con 0x7f537415aaa0 2024-04-25T11:18:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.447+0000 7f5372000640 1 -- 172.21.15.7:0/145933020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364031ca0 con 0x7f537415aaa0 2024-04-25T11:18:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.448+0000 7f537a400640 1 -- 172.21.15.7:0/145933020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f537415aaa0 msgr2=0x7f53740affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.448+0000 7f537a400640 1 --2- 172.21.15.7:0/145933020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f537415aaa0 0x7f53740affc0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f53740af410 tx=0x7f536402c7c0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.448+0000 7f537a400640 1 -- 172.21.15.7:0/145933020 shutdown_connections 2024-04-25T11:18:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.448+0000 7f537a400640 1 --2- 172.21.15.7:0/145933020 >> v2:172.21.15.87:3300/0 conn(0x7f53740d0ee0 0x7f53740b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.448+0000 7f537a400640 1 --2- 172.21.15.7:0/145933020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f537415aaa0 0x7f53740affc0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.448+0000 7f537a400640 1 -- 172.21.15.7:0/145933020 >> 172.21.15.7:0/145933020 conn(0x7f53740742c0 msgr2=0x7f537406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.448+0000 7f537a400640 1 -- 172.21.15.7:0/145933020 shutdown_connections 2024-04-25T11:18:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.448+0000 7f537a400640 1 -- 172.21.15.7:0/145933020 wait complete. 2024-04-25T11:18:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.451+0000 7f537a400640 1 Processor -- start 2024-04-25T11:18:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.451+0000 7f537a400640 1 -- start start 2024-04-25T11:18:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.452+0000 7f537a400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53740b4e50 0x7f537415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.452+0000 7f537a400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53740d0ee0 0x7f5374159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.452+0000 7f537a400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f537415aaa0 0x7f5374148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:56.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.453+0000 7f537a400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f53740b5260 con 0x7f53740d0ee0 2024-04-25T11:18:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.453+0000 7f537a400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f53740afaa0 con 0x7f53740b4e50 2024-04-25T11:18:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.453+0000 7f537a400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f53740b53e0 con 0x7f537415aaa0 2024-04-25T11:18:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.453+0000 7f5372a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53740d0ee0 0x7f5374159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.453+0000 7f5372a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53740d0ee0 0x7f5374159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46280/0 (socket says 172.21.15.7:46280) 2024-04-25T11:18:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.453+0000 7f5372a00640 1 -- 172.21.15.7:0/85772671 learned_addr learned my addr 172.21.15.7:0/85772671 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.453+0000 7f5372a00640 1 -- 172.21.15.7:0/85772671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f537415aaa0 msgr2=0x7f5374148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.453+0000 7f5372a00640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f537415aaa0 0x7f5374148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.453+0000 7f5372a00640 1 -- 172.21.15.7:0/85772671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53740b4e50 msgr2=0x7f537415e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:56.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.453+0000 7f5372a00640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53740b4e50 0x7f537415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.453+0000 7f5372a00640 1 -- 172.21.15.7:0/85772671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5374149b50 con 0x7f53740d0ee0 2024-04-25T11:18:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.454+0000 7f5372a00640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53740d0ee0 0x7f5374159170 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ce00 tx=0x7f535c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.454+0000 7f536b400640 1 -- 172.21.15.7:0/85772671 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c00f890 con 0x7f53740d0ee0 2024-04-25T11:18:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.454+0000 7f536b400640 1 -- 172.21.15.7:0/85772671 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535c00fa30 con 0x7f53740d0ee0 2024-04-25T11:18:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.454+0000 7f536b400640 1 -- 172.21.15.7:0/85772671 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c00fe40 con 0x7f53740d0ee0 2024-04-25T11:18:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.454+0000 7f537a400640 1 -- 172.21.15.7:0/85772671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5374148d90 con 0x7f53740d0ee0 2024-04-25T11:18:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.454+0000 7f537a400640 1 -- 172.21.15.7:0/85772671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5374149350 con 0x7f53740d0ee0 2024-04-25T11:18:56.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.456+0000 7f537a400640 1 -- 172.21.15.7:0/85772671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5374068cd0 con 0x7f53740d0ee0 2024-04-25T11:18:56.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.457+0000 7f536b400640 1 -- 172.21.15.7:0/85772671 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f535c018400 con 0x7f53740d0ee0 2024-04-25T11:18:56.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.458+0000 7f536b400640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53500744b0 0x7f53500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:56.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.458+0000 7f536b400640 1 -- 172.21.15.7:0/85772671 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7f535c095e20 con 0x7f53740d0ee0 2024-04-25T11:18:56.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.458+0000 7f5373400640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53500744b0 0x7f53500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:56.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.459+0000 7f5373400640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53500744b0 0x7f53500769c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5364036000 tx=0x7f5364030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.463+0000 7f536b400640 1 -- 172.21.15.7:0/85772671 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5374068cd0 con 0x7f53740d0ee0 2024-04-25T11:18:56.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.670+0000 7f537a400640 1 -- 172.21.15.7:0/85772671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7f5374050740 con 0x7f53740d0ee0 2024-04-25T11:18:56.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.681+0000 7f536b400640 1 -- 172.21.15.7:0/85772671 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v30) ==== 107+0+0 (secure 0 0 0) 0x7f535c014030 con 0x7f53740d0ee0 2024-04-25T11:18:56.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.683+0000 7f537a400640 1 -- 172.21.15.7:0/85772671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53500744b0 msgr2=0x7f53500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:56.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.683+0000 7f537a400640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53500744b0 0x7f53500769c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5364036000 tx=0x7f5364030000 comp rx=0 tx=0).stop 2024-04-25T11:18:56.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.683+0000 7f537a400640 1 -- 172.21.15.7:0/85772671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53740d0ee0 msgr2=0x7f5374159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:56.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.683+0000 7f537a400640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53740d0ee0 0x7f5374159170 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ce00 tx=0x7f535c00a650 comp rx=0 tx=0).stop 2024-04-25T11:18:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.684+0000 7f537a400640 1 -- 172.21.15.7:0/85772671 shutdown_connections 2024-04-25T11:18:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.684+0000 7f537a400640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53500744b0 0x7f53500769c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.684+0000 7f537a400640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f537415aaa0 0x7f5374148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.684+0000 7f537a400640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53740d0ee0 0x7f5374159170 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.684+0000 7f537a400640 1 --2- 172.21.15.7:0/85772671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53740b4e50 0x7f537415e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:56.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.684+0000 7f537a400640 1 -- 172.21.15.7:0/85772671 >> 172.21.15.7:0/85772671 conn(0x7f53740742c0 msgr2=0x7f53740b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:56.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.684+0000 7f537a400640 1 -- 172.21.15.7:0/85772671 shutdown_connections 2024-04-25T11:18:56.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.684+0000 7f537a400640 1 -- 172.21.15.7:0/85772671 wait complete. 2024-04-25T11:18:56.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:18:56.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.997+0000 7f67af200640 1 Processor -- start 2024-04-25T11:18:56.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.997+0000 7f67af200640 1 -- start start 2024-04-25T11:18:56.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.999+0000 7f67af200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67b00b2400 0x7f67b00b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:56.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:56.999+0000 7f67af200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67b00b2950 0x7f67b00b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67af200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f67b009b8f0 0x7f67b009bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67af200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f67b00d2240 con 0x7f67b00b2950 2024-04-25T11:18:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67af200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f67b009c240 con 0x7f67b009b8f0 2024-04-25T11:18:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67af200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f67b00d1c30 con 0x7f67b00b2400 2024-04-25T11:18:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67ad400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67b00b2950 0x7f67b00b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67ad400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67b00b2950 0x7f67b00b8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46292/0 (socket says 172.21.15.7:46292) 2024-04-25T11:18:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67ad400640 1 -- 172.21.15.7:0/4165821386 learned_addr learned my addr 172.21.15.7:0/4165821386 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:57.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67ade00640 1 --2- 172.21.15.7:0/4165821386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67b00b2400 0x7f67b00b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67ae800640 1 --2- 172.21.15.7:0/4165821386 >> v2:172.21.15.87:3300/0 conn(0x7f67b009b8f0 0x7f67b009bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67ad400640 1 -- 172.21.15.7:0/4165821386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67b00b2400 msgr2=0x7f67b00b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:57.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67ad400640 1 --2- 172.21.15.7:0/4165821386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67b00b2400 0x7f67b00b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67ad400640 1 -- 172.21.15.7:0/4165821386 >> v2:172.21.15.87:3300/0 conn(0x7f67b009b8f0 msgr2=0x7f67b009bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:57.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67ad400640 1 --2- 172.21.15.7:0/4165821386 >> v2:172.21.15.87:3300/0 conn(0x7f67b009b8f0 0x7f67b009bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.000+0000 7f67ad400640 1 -- 172.21.15.7:0/4165821386 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b006e780 con 0x7f67b00b2950 2024-04-25T11:18:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.002+0000 7f67ad400640 1 --2- 172.21.15.7:0/4165821386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b00b2950 0x7f67b00b8c70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f67b0074190 tx=0x7f679802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ae63f977945925 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.002+0000 7f67aca00640 1 -- 172.21.15.7:0/4165821386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798037070 con 0x7f67b00b2950 2024-04-25T11:18:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.002+0000 7f67aca00640 1 -- 172.21.15.7:0/4165821386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6798032440 con 0x7f67b00b2950 2024-04-25T11:18:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.002+0000 7f67aca00640 1 -- 172.21.15.7:0/4165821386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798032810 con 0x7f67b00b2950 2024-04-25T11:18:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.003+0000 7f67af200640 1 -- 172.21.15.7:0/4165821386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b00b2950 msgr2=0x7f67b00b8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.003+0000 7f67af200640 1 --2- 172.21.15.7:0/4165821386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b00b2950 0x7f67b00b8c70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f67b0074190 tx=0x7f679802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.003+0000 7f67af200640 1 -- 172.21.15.7:0/4165821386 shutdown_connections 2024-04-25T11:18:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.003+0000 7f67af200640 1 --2- 172.21.15.7:0/4165821386 >> v2:172.21.15.87:3300/0 conn(0x7f67b009b8f0 0x7f67b009bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.003+0000 7f67af200640 1 --2- 172.21.15.7:0/4165821386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b00b2950 0x7f67b00b8c70 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.003+0000 7f67af200640 1 --2- 172.21.15.7:0/4165821386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67b00b2400 0x7f67b00b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.003+0000 7f67af200640 1 -- 172.21.15.7:0/4165821386 >> 172.21.15.7:0/4165821386 conn(0x7f67b00b5210 msgr2=0x7f67b00b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.003+0000 7f67af200640 1 -- 172.21.15.7:0/4165821386 shutdown_connections 2024-04-25T11:18:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.003+0000 7f67af200640 1 -- 172.21.15.7:0/4165821386 wait complete. 2024-04-25T11:18:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.005+0000 7f67af200640 1 Processor -- start 2024-04-25T11:18:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.006+0000 7f67af200640 1 -- start start 2024-04-25T11:18:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.007+0000 7f67af200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67b009b8f0 0x7f67b01fa470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.007+0000 7f67af200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b00b2400 0x7f67b015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.007+0000 7f67af200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67b00b2950 0x7f67b00723e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.007+0000 7f67af200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67b006e430 con 0x7f67b00b2400 2024-04-25T11:18:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.007+0000 7f67af200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f67b00b91b0 con 0x7f67b009b8f0 2024-04-25T11:18:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.007+0000 7f67af200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f67b006ea60 con 0x7f67b00b2950 2024-04-25T11:18:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.008+0000 7f67ad400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b00b2400 0x7f67b015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.008+0000 7f67ad400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b00b2400 0x7f67b015cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46300/0 (socket says 172.21.15.7:46300) 2024-04-25T11:18:57.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.008+0000 7f67ad400640 1 -- 172.21.15.7:0/662872193 learned_addr learned my addr 172.21.15.7:0/662872193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:57.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.008+0000 7f67ade00640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67b009b8f0 0x7f67b01fa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.008+0000 7f67ae800640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67b00b2950 0x7f67b00723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.008+0000 7f67ad400640 1 -- 172.21.15.7:0/662872193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67b00b2950 msgr2=0x7f67b00723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.008+0000 7f67ad400640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67b00b2950 0x7f67b00723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.008+0000 7f67ad400640 1 -- 172.21.15.7:0/662872193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67b009b8f0 msgr2=0x7f67b01fa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.008+0000 7f67ad400640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67b009b8f0 0x7f67b01fa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.008+0000 7f67ad400640 1 -- 172.21.15.7:0/662872193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b01fa9b0 con 0x7f67b00b2400 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.009+0000 7f67ad400640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b00b2400 0x7f67b015cff0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6798000c00 tx=0x7f67980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.009+0000 7f67a6000640 1 -- 172.21.15.7:0/662872193 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798037040 con 0x7f67b00b2400 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.009+0000 7f67a6000640 1 -- 172.21.15.7:0/662872193 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6798042070 con 0x7f67b00b2400 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.009+0000 7f67a6000640 1 -- 172.21.15.7:0/662872193 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798030bb0 con 0x7f67b00b2400 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.009+0000 7f67af200640 1 -- 172.21.15.7:0/662872193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b00c9750 con 0x7f67b00b2400 2024-04-25T11:18:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.009+0000 7f67af200640 1 -- 172.21.15.7:0/662872193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b00c9d10 con 0x7f67b00b2400 2024-04-25T11:18:57.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.010+0000 7f67af200640 1 -- 172.21.15.7:0/662872193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b0050740 con 0x7f67b00b2400 2024-04-25T11:18:57.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.012+0000 7f67a6000640 1 -- 172.21.15.7:0/662872193 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6798047460 con 0x7f67b00b2400 2024-04-25T11:18:57.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.013+0000 7f67a6000640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f678c0744e0 0x7f678c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.013+0000 7f67a6000640 1 -- 172.21.15.7:0/662872193 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7f67980b5dc0 con 0x7f67b00b2400 2024-04-25T11:18:57.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.013+0000 7f67ade00640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f678c0744e0 0x7f678c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.014+0000 7f67ade00640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f678c0744e0 0x7f678c0769f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f67a0006fd0 tx=0x7f67a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:57.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.018+0000 7f67a6000640 1 -- 172.21.15.7:0/662872193 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f67b01fa9b0 con 0x7f67b00b2400 2024-04-25T11:18:57.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.225+0000 7f67af200640 1 -- 172.21.15.7:0/662872193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f67b00c94b0 con 0x7f67b00b2400 2024-04-25T11:18:57.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.232+0000 7f67a6000640 1 -- 172.21.15.7:0/662872193 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f6798080620 con 0x7f67b00b2400 2024-04-25T11:18:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 -- 172.21.15.7:0/662872193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f678c0744e0 msgr2=0x7f678c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f678c0744e0 0x7f678c0769f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f67a0006fd0 tx=0x7f67a0008040 comp rx=0 tx=0).stop 2024-04-25T11:18:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 -- 172.21.15.7:0/662872193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b00b2400 msgr2=0x7f67b015cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b00b2400 0x7f67b015cff0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f6798000c00 tx=0x7f67980046c0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 -- 172.21.15.7:0/662872193 shutdown_connections 2024-04-25T11:18:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f678c0744e0 0x7f678c0769f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67b00b2950 0x7f67b00723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67b00b2400 0x7f67b015cff0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 --2- 172.21.15.7:0/662872193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67b009b8f0 0x7f67b01fa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 -- 172.21.15.7:0/662872193 >> 172.21.15.7:0/662872193 conn(0x7f67b00b5210 msgr2=0x7f67b00b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:57.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 -- 172.21.15.7:0/662872193 shutdown_connections 2024-04-25T11:18:57.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.235+0000 7f67af200640 1 -- 172.21.15.7:0/662872193 wait complete. 2024-04-25T11:18:57.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-04-25T11:18:57.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.550+0000 7fb7bda00640 1 Processor -- start 2024-04-25T11:18:57.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.550+0000 7fb7bda00640 1 -- start start 2024-04-25T11:18:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.552+0000 7fb7bda00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7b81583c0 0x7fb7b80cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.552+0000 7fb7bda00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7b80cfb50 0x7fb7b80d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.552+0000 7fb7bda00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b80ce570 con 0x7fb7b80d09c0 2024-04-25T11:18:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.552+0000 7fb7bda00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b815f190 con 0x7fb7b80cfb50 2024-04-25T11:18:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.552+0000 7fb7bda00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b80ce850 con 0x7fb7b81583c0 2024-04-25T11:18:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.552+0000 7fb7b7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7b80cfb50 0x7fb7b80d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.552+0000 7fb7b7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7b80cfb50 0x7fb7b80d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46312/0 (socket says 172.21.15.7:46312) 2024-04-25T11:18:57.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.552+0000 7fb7b7400640 1 -- 172.21.15.7:0/2689831477 learned_addr learned my addr 172.21.15.7:0/2689831477 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.553+0000 7fb7b7e00640 1 --2- 172.21.15.7:0/2689831477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7b81583c0 0x7fb7b80cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.553+0000 7fb7b7400640 1 -- 172.21.15.7:0/2689831477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7b81583c0 msgr2=0x7fb7b80cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.553+0000 7fb7b7400640 1 --2- 172.21.15.7:0/2689831477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7b81583c0 0x7fb7b80cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.553+0000 7fb7b7400640 1 -- 172.21.15.7:0/2689831477 >> v1:172.21.15.87:6789/0 conn(0x7fb7b80d09c0 legacy=0x7fb7b806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.553+0000 7fb7b7400640 1 -- 172.21.15.7:0/2689831477 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b80ceb30 con 0x7fb7b80cfb50 2024-04-25T11:18:57.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.553+0000 7fb7b7e00640 1 --2- 172.21.15.7:0/2689831477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7b81583c0 0x7fb7b80cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.554+0000 7fb7b7400640 1 --2- 172.21.15.7:0/2689831477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b80cfb50 0x7fb7b80d0480 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009870 tx=0x7fb7a402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0eb6b9adc3c84a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.554+0000 7fb7b6a00640 1 -- 172.21.15.7:0/2689831477 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4037070 con 0x7fb7b80cfb50 2024-04-25T11:18:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.554+0000 7fb7b6a00640 1 -- 172.21.15.7:0/2689831477 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a4032440 con 0x7fb7b80cfb50 2024-04-25T11:18:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.554+0000 7fb7b6a00640 1 -- 172.21.15.7:0/2689831477 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4032810 con 0x7fb7b80cfb50 2024-04-25T11:18:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.555+0000 7fb7bda00640 1 -- 172.21.15.7:0/2689831477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b80cfb50 msgr2=0x7fb7b80d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.555+0000 7fb7bda00640 1 --2- 172.21.15.7:0/2689831477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b80cfb50 0x7fb7b80d0480 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009870 tx=0x7fb7a402a200 comp rx=0 tx=0).stop 2024-04-25T11:18:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.555+0000 7fb7bda00640 1 -- 172.21.15.7:0/2689831477 shutdown_connections 2024-04-25T11:18:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.555+0000 7fb7bda00640 1 --2- 172.21.15.7:0/2689831477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b80cfb50 0x7fb7b80d0480 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.555+0000 7fb7bda00640 1 --2- 172.21.15.7:0/2689831477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7b81583c0 0x7fb7b80cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.555+0000 7fb7bda00640 1 -- 172.21.15.7:0/2689831477 >> 172.21.15.7:0/2689831477 conn(0x7fb7b80740f0 msgr2=0x7fb7b806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.555+0000 7fb7bda00640 1 -- 172.21.15.7:0/2689831477 shutdown_connections 2024-04-25T11:18:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.555+0000 7fb7bda00640 1 -- 172.21.15.7:0/2689831477 wait complete. 2024-04-25T11:18:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.558+0000 7fb7bda00640 1 Processor -- start 2024-04-25T11:18:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.558+0000 7fb7bda00640 1 -- start start 2024-04-25T11:18:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.560+0000 7fb7bda00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7b80cfb50 0x7fb7b80b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.560+0000 7fb7bda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7b80d09c0 0x7fb7b8144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.560+0000 7fb7bda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b81583c0 0x7fb7b81426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.560+0000 7fb7bda00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b806f5b0 con 0x7fb7b81583c0 2024-04-25T11:18:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.560+0000 7fb7bda00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b80cf0f0 con 0x7fb7b80d09c0 2024-04-25T11:18:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.560+0000 7fb7bda00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b806f730 con 0x7fb7b80cfb50 2024-04-25T11:18:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.560+0000 7fb7bd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b81583c0 0x7fb7b81426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.560+0000 7fb7bd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b81583c0 0x7fb7b81426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46324/0 (socket says 172.21.15.7:46324) 2024-04-25T11:18:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.560+0000 7fb7bd000640 1 -- 172.21.15.7:0/3831292357 learned_addr learned my addr 172.21.15.7:0/3831292357 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.560+0000 7fb7b7e00640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7b80cfb50 0x7fb7b80b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.560+0000 7fb7b7400640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7b80d09c0 0x7fb7b8144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.561+0000 7fb7bd000640 1 -- 172.21.15.7:0/3831292357 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7b80cfb50 msgr2=0x7fb7b80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.561+0000 7fb7bd000640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7b80cfb50 0x7fb7b80b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.561+0000 7fb7bd000640 1 -- 172.21.15.7:0/3831292357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7b80d09c0 msgr2=0x7fb7b8144a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.561+0000 7fb7bd000640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7b80d09c0 0x7fb7b8144a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.561+0000 7fb7bd000640 1 -- 172.21.15.7:0/3831292357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b80b4d70 con 0x7fb7b81583c0 2024-04-25T11:18:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.561+0000 7fb7bd000640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b81583c0 0x7fb7b81426f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000b970 tx=0x7fb7b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.561+0000 7fb7afe00640 1 -- 172.21.15.7:0/3831292357 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0019070 con 0x7fb7b81583c0 2024-04-25T11:18:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.561+0000 7fb7afe00640 1 -- 172.21.15.7:0/3831292357 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b000cb40 con 0x7fb7b81583c0 2024-04-25T11:18:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.561+0000 7fb7bda00640 1 -- 172.21.15.7:0/3831292357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b80b5050 con 0x7fb7b81583c0 2024-04-25T11:18:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.561+0000 7fb7afe00640 1 -- 172.21.15.7:0/3831292357 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0014440 con 0x7fb7b81583c0 2024-04-25T11:18:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.562+0000 7fb7bda00640 1 -- 172.21.15.7:0/3831292357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b80b5610 con 0x7fb7b81583c0 2024-04-25T11:18:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.564+0000 7fb7bda00640 1 -- 172.21.15.7:0/3831292357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b8050740 con 0x7fb7b81583c0 2024-04-25T11:18:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.564+0000 7fb7afe00640 1 -- 172.21.15.7:0/3831292357 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb7b001c070 con 0x7fb7b81583c0 2024-04-25T11:18:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.565+0000 7fb7afe00640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb798074500 0x7fb798076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.565+0000 7fb7afe00640 1 -- 172.21.15.7:0/3831292357 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7fb7b0094c70 con 0x7fb7b81583c0 2024-04-25T11:18:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.565+0000 7fb7b7e00640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb798074500 0x7fb798076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.566+0000 7fb7b7e00640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb798074500 0x7fb798076a10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80aa990 tx=0x7fb7a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:57.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.579+0000 7fb7afe00640 1 -- 172.21.15.7:0/3831292357 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb7b005ef90 con 0x7fb7b81583c0 2024-04-25T11:18:57.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.785+0000 7fb7bda00640 1 -- 172.21.15.7:0/3831292357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7fb7b8070dd0 con 0x7fb7b81583c0 2024-04-25T11:18:57.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.791+0000 7fb7afe00640 1 -- 172.21.15.7:0/3831292357 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v32) ==== 111+0+0 (secure 0 0 0) 0x7fb7b0063290 con 0x7fb7b81583c0 2024-04-25T11:18:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 -- 172.21.15.7:0/3831292357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb798074500 msgr2=0x7fb798076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb798074500 0x7fb798076a10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80aa990 tx=0x7fb7a8008040 comp rx=0 tx=0).stop 2024-04-25T11:18:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 -- 172.21.15.7:0/3831292357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b81583c0 msgr2=0x7fb7b81426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b81583c0 0x7fb7b81426f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000b970 tx=0x7fb7b000be30 comp rx=0 tx=0).stop 2024-04-25T11:18:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 -- 172.21.15.7:0/3831292357 shutdown_connections 2024-04-25T11:18:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb798074500 0x7fb798076a10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b81583c0 0x7fb7b81426f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7b80d09c0 0x7fb7b8144a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 --2- 172.21.15.7:0/3831292357 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7b80cfb50 0x7fb7b80b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 -- 172.21.15.7:0/3831292357 >> 172.21.15.7:0/3831292357 conn(0x7fb7b80740f0 msgr2=0x7fb7b806cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 -- 172.21.15.7:0/3831292357 shutdown_connections 2024-04-25T11:18:57.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:57.794+0000 7fb7bda00640 1 -- 172.21.15.7:0/3831292357 wait complete. 2024-04-25T11:18:57.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-04-25T11:18:58.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.115+0000 7f5dada00640 1 Processor -- start 2024-04-25T11:18:58.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.115+0000 7f5dada00640 1 -- start start 2024-04-25T11:18:58.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.118+0000 7f5dada00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5da815dfb0 0x7f5da80b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.118+0000 7f5dada00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5da806fab0 0x7f5da80aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.118+0000 7f5dada00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da815e3c0 con 0x7f5da80d0b40 2024-04-25T11:18:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.118+0000 7f5dada00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5da80b09b0 con 0x7f5da815dfb0 2024-04-25T11:18:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.118+0000 7f5dada00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5da80cf800 con 0x7f5da806fab0 2024-04-25T11:18:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.118+0000 7f5da6a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5da815dfb0 0x7f5da80b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.118+0000 7f5da6000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5da80d0b40 0x7f5da806f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45318/0 (socket says 172.21.15.7:45318) 2024-04-25T11:18:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.118+0000 7f5da6a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5da815dfb0 0x7f5da80b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46334/0 (socket says 172.21.15.7:46334) 2024-04-25T11:18:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.118+0000 7f5da6000640 1 -- 172.21.15.7:0/86661808 learned_addr learned my addr 172.21.15.7:0/86661808 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:58.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.119+0000 7f5da7400640 1 --2- 172.21.15.7:0/86661808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5da806fab0 0x7f5da80aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:58.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.119+0000 7f5da6a00640 1 -- 172.21.15.7:0/86661808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5da806fab0 msgr2=0x7f5da80aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:58.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.119+0000 7f5da6a00640 1 --2- 172.21.15.7:0/86661808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5da806fab0 0x7f5da80aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.119+0000 7f5da6a00640 1 -- 172.21.15.7:0/86661808 >> v1:172.21.15.7:6789/0 conn(0x7f5da80d0b40 legacy=0x7f5da806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:58.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.119+0000 7f5da6a00640 1 -- 172.21.15.7:0/86661808 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da80cfaa0 con 0x7f5da815dfb0 2024-04-25T11:18:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.120+0000 7f5da6a00640 1 --2- 172.21.15.7:0/86661808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5da815dfb0 0x7f5da80b0470 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f5d980097a0 tx=0x7f5d9802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65b73d382b77de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.120+0000 7f5da5600640 1 -- 172.21.15.7:0/86661808 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98037070 con 0x7f5da815dfb0 2024-04-25T11:18:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.120+0000 7f5da5600640 1 -- 172.21.15.7:0/86661808 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d98032440 con 0x7f5da815dfb0 2024-04-25T11:18:58.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.120+0000 7f5da5600640 1 -- 172.21.15.7:0/86661808 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98032810 con 0x7f5da815dfb0 2024-04-25T11:18:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.121+0000 7f5dada00640 1 -- 172.21.15.7:0/86661808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5da815dfb0 msgr2=0x7f5da80b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.121+0000 7f5dada00640 1 --2- 172.21.15.7:0/86661808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5da815dfb0 0x7f5da80b0470 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f5d980097a0 tx=0x7f5d9802a180 comp rx=0 tx=0).stop 2024-04-25T11:18:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.121+0000 7f5dada00640 1 -- 172.21.15.7:0/86661808 shutdown_connections 2024-04-25T11:18:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.121+0000 7f5dada00640 1 --2- 172.21.15.7:0/86661808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5da806fab0 0x7f5da80aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.121+0000 7f5dada00640 1 --2- 172.21.15.7:0/86661808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5da815dfb0 0x7f5da80b0470 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.121+0000 7f5dada00640 1 -- 172.21.15.7:0/86661808 >> 172.21.15.7:0/86661808 conn(0x7f5da80742c0 msgr2=0x7f5da806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.121+0000 7f5dada00640 1 -- 172.21.15.7:0/86661808 shutdown_connections 2024-04-25T11:18:58.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.121+0000 7f5dada00640 1 -- 172.21.15.7:0/86661808 wait complete. 2024-04-25T11:18:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.124+0000 7f5dada00640 1 Processor -- start 2024-04-25T11:18:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.124+0000 7f5dada00640 1 -- start start 2024-04-25T11:18:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.125+0000 7f5dada00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5da806fab0 0x7f5da815e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.125+0000 7f5dada00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5da80d0b40 0x7f5da8142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.125+0000 7f5dada00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5da815dfb0 0x7f5da8155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.125+0000 7f5dada00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5da80d0340 con 0x7f5da80d0b40 2024-04-25T11:18:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.125+0000 7f5dada00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5da80d0060 con 0x7f5da806fab0 2024-04-25T11:18:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.125+0000 7f5dada00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5da80d0620 con 0x7f5da815dfb0 2024-04-25T11:18:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.125+0000 7f5da6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5da80d0b40 0x7f5da8142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.126+0000 7f5da6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5da80d0b40 0x7f5da8142120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46340/0 (socket says 172.21.15.7:46340) 2024-04-25T11:18:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.126+0000 7f5da6000640 1 -- 172.21.15.7:0/3816872152 learned_addr learned my addr 172.21.15.7:0/3816872152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.126+0000 7f5da6a00640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5da806fab0 0x7f5da815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.126+0000 7f5da7400640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5da815dfb0 0x7f5da8155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.126+0000 7f5da6000640 1 -- 172.21.15.7:0/3816872152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5da815dfb0 msgr2=0x7f5da8155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.126+0000 7f5da6000640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5da815dfb0 0x7f5da8155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.126+0000 7f5da6000640 1 -- 172.21.15.7:0/3816872152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5da806fab0 msgr2=0x7f5da815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.126+0000 7f5da6000640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5da806fab0 0x7f5da815e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.126+0000 7f5da6000640 1 -- 172.21.15.7:0/3816872152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da8142660 con 0x7f5da80d0b40 2024-04-25T11:18:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.126+0000 7f5da6000640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5da80d0b40 0x7f5da8142120 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c3c0 tx=0x7f5d9000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.127+0000 7f5d9ea00640 1 -- 172.21.15.7:0/3816872152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90017070 con 0x7f5da80d0b40 2024-04-25T11:18:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.127+0000 7f5d9ea00640 1 -- 172.21.15.7:0/3816872152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d90005ce0 con 0x7f5da80d0b40 2024-04-25T11:18:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.127+0000 7f5d9ea00640 1 -- 172.21.15.7:0/3816872152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d900072f0 con 0x7f5da80d0b40 2024-04-25T11:18:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.127+0000 7f5dada00640 1 -- 172.21.15.7:0/3816872152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da81542b0 con 0x7f5da80d0b40 2024-04-25T11:18:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.127+0000 7f5dada00640 1 -- 172.21.15.7:0/3816872152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da8154870 con 0x7f5da80d0b40 2024-04-25T11:18:58.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.128+0000 7f5dada00640 1 -- 172.21.15.7:0/3816872152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da8050740 con 0x7f5da80d0b40 2024-04-25T11:18:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.130+0000 7f5d9ea00640 1 -- 172.21.15.7:0/3816872152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5d9001a070 con 0x7f5da80d0b40 2024-04-25T11:18:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.131+0000 7f5d9ea00640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d840744e0 0x7f5d840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.132+0000 7f5d9ea00640 1 -- 172.21.15.7:0/3816872152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7f5d90094290 con 0x7f5da80d0b40 2024-04-25T11:18:58.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.132+0000 7f5da6a00640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d840744e0 0x7f5d840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:58.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.133+0000 7f5da6a00640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d840744e0 0x7f5d840769f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5d98034000 tx=0x7f5d9802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:58.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.136+0000 7f5d9ea00640 1 -- 172.21.15.7:0/3816872152 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5d9005ec20 con 0x7f5da80d0b40 2024-04-25T11:18:58.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.344+0000 7f5dada00640 1 -- 172.21.15.7:0/3816872152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7f5da8070fb0 con 0x7f5da80d0b40 2024-04-25T11:18:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.353+0000 7f5d9ea00640 1 -- 172.21.15.7:0/3816872152 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v33) ==== 105+0+0 (secure 0 0 0) 0x7f5d9005e520 con 0x7f5da80d0b40 2024-04-25T11:18:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 -- 172.21.15.7:0/3816872152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d840744e0 msgr2=0x7f5d840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d840744e0 0x7f5d840769f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5d98034000 tx=0x7f5d9802e000 comp rx=0 tx=0).stop 2024-04-25T11:18:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 -- 172.21.15.7:0/3816872152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5da80d0b40 msgr2=0x7f5da8142120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5da80d0b40 0x7f5da8142120 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c3c0 tx=0x7f5d9000c7c0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 -- 172.21.15.7:0/3816872152 shutdown_connections 2024-04-25T11:18:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5d840744e0 0x7f5d840769f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5da815dfb0 0x7f5da8155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5da80d0b40 0x7f5da8142120 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 --2- 172.21.15.7:0/3816872152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5da806fab0 0x7f5da815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 -- 172.21.15.7:0/3816872152 >> 172.21.15.7:0/3816872152 conn(0x7f5da80742c0 msgr2=0x7f5da806c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:58.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 -- 172.21.15.7:0/3816872152 shutdown_connections 2024-04-25T11:18:58.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.356+0000 7f5dada00640 1 -- 172.21.15.7:0/3816872152 wait complete. 2024-04-25T11:18:58.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set' 2024-04-25T11:18:58.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.666+0000 7f5eafe00640 1 Processor -- start 2024-04-25T11:18:58.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.666+0000 7f5eafe00640 1 -- start start 2024-04-25T11:18:58.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.669+0000 7f5eafe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5eb006fb00 0x7f5eb00b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.669+0000 7f5eafe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5eb006d580 0x7f5eb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.669+0000 7f5eafe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb006ded0 con 0x7f5eb015ff10 2024-04-25T11:18:58.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.669+0000 7f5eafe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb00af2b0 con 0x7f5eb006fb00 2024-04-25T11:18:58.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.669+0000 7f5eafe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb00af550 con 0x7f5eb006d580 2024-04-25T11:18:58.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.669+0000 7f5eaea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5eb006fb00 0x7f5eb00b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:58.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.669+0000 7f5eaea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5eb006fb00 0x7f5eb00b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46354/0 (socket says 172.21.15.7:46354) 2024-04-25T11:18:58.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.669+0000 7f5eaea00640 1 -- 172.21.15.7:0/3960733512 learned_addr learned my addr 172.21.15.7:0/3960733512 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:58.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.670+0000 7f5eaea00640 1 -- 172.21.15.7:0/3960733512 >> v2:172.21.15.87:3300/0 conn(0x7f5eb006d580 msgr2=0x7f5eb006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:58.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.670+0000 7f5eaf400640 1 --2- 172.21.15.7:0/3960733512 >> v2:172.21.15.87:3300/0 conn(0x7f5eb006d580 0x7f5eb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:58.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.670+0000 7f5eaea00640 1 --2- 172.21.15.7:0/3960733512 >> v2:172.21.15.87:3300/0 conn(0x7f5eb006d580 0x7f5eb006d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.670+0000 7f5eaea00640 1 -- 172.21.15.7:0/3960733512 >> v1:172.21.15.87:6789/0 conn(0x7f5eb015ff10 legacy=0x7f5eb00b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:58.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.670+0000 7f5eaea00640 1 -- 172.21.15.7:0/3960733512 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb00af830 con 0x7f5eb006fb00 2024-04-25T11:18:58.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.670+0000 7f5eaf400640 1 --2- 172.21.15.7:0/3960733512 >> v2:172.21.15.87:3300/0 conn(0x7f5eb006d580 0x7f5eb006d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:58.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.671+0000 7f5eaea00640 1 --2- 172.21.15.7:0/3960733512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb006fb00 0x7f5eb00b0030 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0074920 tx=0x7f5ea002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b749469b7e1115 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:58.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.671+0000 7f5eae000640 1 -- 172.21.15.7:0/3960733512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0037070 con 0x7f5eb006fb00 2024-04-25T11:18:58.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.671+0000 7f5eae000640 1 -- 172.21.15.7:0/3960733512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea0032440 con 0x7f5eb006fb00 2024-04-25T11:18:58.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.671+0000 7f5eae000640 1 -- 172.21.15.7:0/3960733512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0032810 con 0x7f5eb006fb00 2024-04-25T11:18:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.672+0000 7f5eafe00640 1 -- 172.21.15.7:0/3960733512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb006fb00 msgr2=0x7f5eb00b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.672+0000 7f5eafe00640 1 --2- 172.21.15.7:0/3960733512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb006fb00 0x7f5eb00b0030 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0074920 tx=0x7f5ea002a230 comp rx=0 tx=0).stop 2024-04-25T11:18:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.672+0000 7f5eafe00640 1 -- 172.21.15.7:0/3960733512 shutdown_connections 2024-04-25T11:18:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.672+0000 7f5eafe00640 1 --2- 172.21.15.7:0/3960733512 >> v2:172.21.15.87:3300/0 conn(0x7f5eb006d580 0x7f5eb006d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.672+0000 7f5eafe00640 1 --2- 172.21.15.7:0/3960733512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb006fb00 0x7f5eb00b0030 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.672+0000 7f5eafe00640 1 -- 172.21.15.7:0/3960733512 >> 172.21.15.7:0/3960733512 conn(0x7f5eb0073090 msgr2=0x7f5eb00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.673+0000 7f5eafe00640 1 -- 172.21.15.7:0/3960733512 shutdown_connections 2024-04-25T11:18:58.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.673+0000 7f5eafe00640 1 -- 172.21.15.7:0/3960733512 wait complete. 2024-04-25T11:18:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.676+0000 7f5eafe00640 1 Processor -- start 2024-04-25T11:18:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.676+0000 7f5eafe00640 1 -- start start 2024-04-25T11:18:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eafe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb006d580 0x7f5eb01fa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eafe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eb015ff10 0x7f5eb01f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eafe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eb01e99c0 0x7f5eb015b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eafe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb0160320 con 0x7f5eb006d580 2024-04-25T11:18:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eafe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb0070070 con 0x7f5eb015ff10 2024-04-25T11:18:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eafe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb00b4690 con 0x7f5eb01e99c0 2024-04-25T11:18:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eaea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb006d580 0x7f5eb01fa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eaea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb006d580 0x7f5eb01fa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46370/0 (socket says 172.21.15.7:46370) 2024-04-25T11:18:58.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eaea00640 1 -- 172.21.15.7:0/374066423 learned_addr learned my addr 172.21.15.7:0/374066423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5ea7e00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eb015ff10 0x7f5eb01f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eaea00640 1 -- 172.21.15.7:0/374066423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eb01e99c0 msgr2=0x7f5eb015b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eaea00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eb01e99c0 0x7f5eb015b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eaea00640 1 -- 172.21.15.7:0/374066423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eb015ff10 msgr2=0x7f5eb01f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.678+0000 7f5eaea00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eb015ff10 0x7f5eb01f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:58.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.679+0000 7f5eaea00640 1 -- 172.21.15.7:0/374066423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb01eaa40 con 0x7f5eb006d580 2024-04-25T11:18:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.679+0000 7f5ea7e00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eb015ff10 0x7f5eb01f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:18:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.679+0000 7f5eaea00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb006d580 0x7f5eb01fa650 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5ea002a7a0 tx=0x7f5ea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.679+0000 7f5ea6a00640 1 -- 172.21.15.7:0/374066423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0037040 con 0x7f5eb006d580 2024-04-25T11:18:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.679+0000 7f5ea6a00640 1 -- 172.21.15.7:0/374066423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea0042070 con 0x7f5eb006d580 2024-04-25T11:18:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.679+0000 7f5ea6a00640 1 -- 172.21.15.7:0/374066423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0030b90 con 0x7f5eb006d580 2024-04-25T11:18:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.679+0000 7f5eafe00640 1 -- 172.21.15.7:0/374066423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb01ea1a0 con 0x7f5eb006d580 2024-04-25T11:18:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.680+0000 7f5eafe00640 1 -- 172.21.15.7:0/374066423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb01ea760 con 0x7f5eb006d580 2024-04-25T11:18:58.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.681+0000 7f5eafe00640 1 -- 172.21.15.7:0/374066423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb0050740 con 0x7f5eb006d580 2024-04-25T11:18:58.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.682+0000 7f5ea6a00640 1 -- 172.21.15.7:0/374066423 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5ea0047460 con 0x7f5eb006d580 2024-04-25T11:18:58.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.683+0000 7f5ea6a00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e8c0744e0 0x7f5e8c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:58.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.683+0000 7f5ea6a00640 1 -- 172.21.15.7:0/374066423 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7f5ea00b5c40 con 0x7f5eb006d580 2024-04-25T11:18:58.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.684+0000 7f5ea7e00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e8c0744e0 0x7f5e8c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:58.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.684+0000 7f5ea7e00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e8c0744e0 0x7f5e8c0769f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5eb01593f0 tx=0x7f5e98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.688+0000 7f5ea6a00640 1 -- 172.21.15.7:0/374066423 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5eb006ded0 con 0x7f5eb006d580 2024-04-25T11:18:58.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:58.966+0000 7f5eafe00640 1 -- 172.21.15.7:0/374066423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]} v 0) -- 0x7f5eb015c200 con 0x7f5eb006d580 2024-04-25T11:18:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.563+0000 7f5ea6a00640 1 -- 172.21.15.7:0/374066423 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_min_caps_working_set"]}]=0 v433) ==== 160+0+0 (secure 0 0 0) 0x7f5ea0080530 con 0x7f5eb006d580 2024-04-25T11:18:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.566+0000 7f5eafe00640 1 -- 172.21.15.7:0/374066423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e8c0744e0 msgr2=0x7f5e8c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.566+0000 7f5eafe00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e8c0744e0 0x7f5e8c0769f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5eb01593f0 tx=0x7f5e98008040 comp rx=0 tx=0).stop 2024-04-25T11:18:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.566+0000 7f5eafe00640 1 -- 172.21.15.7:0/374066423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb006d580 msgr2=0x7f5eb01fa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.566+0000 7f5eafe00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb006d580 0x7f5eb01fa650 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5ea002a7a0 tx=0x7f5ea00046c0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.566+0000 7f5eafe00640 1 -- 172.21.15.7:0/374066423 shutdown_connections 2024-04-25T11:18:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.566+0000 7f5eafe00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e8c0744e0 0x7f5e8c0769f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.566+0000 7f5eafe00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5eb01e99c0 0x7f5eb015b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.566+0000 7f5eafe00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5eb015ff10 0x7f5eb01f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.566+0000 7f5eafe00640 1 --2- 172.21.15.7:0/374066423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5eb006d580 0x7f5eb01fa650 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.566+0000 7f5eafe00640 1 -- 172.21.15.7:0/374066423 >> 172.21.15.7:0/374066423 conn(0x7f5eb0073090 msgr2=0x7f5eb01583b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:59.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.567+0000 7f5eafe00640 1 -- 172.21.15.7:0/374066423 shutdown_connections 2024-04-25T11:18:59.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.567+0000 7f5eafe00640 1 -- 172.21.15.7:0/374066423 wait complete. 2024-04-25T11:18:59.600 INFO:tasks.cephfs_test_runner:When a client has inodes pinned in its cache (open files), that the MDS ... ok 2024-04-25T11:18:59.600 INFO:tasks.cephfs_test_runner:Starting test: test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) When a client does not advance its oldest tid, the MDS should notice that 2024-04-25T11:18:59.600 INFO:tasks.cephfs_test_runner:test_client_oldest_tid (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-25T11:18:59.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:18:59.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.894+0000 7f50d9000640 1 Processor -- start 2024-04-25T11:18:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.894+0000 7f50d9000640 1 -- start start 2024-04-25T11:18:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.896+0000 7f50d9000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f50d4164690 0x7f50d4070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.897+0000 7f50d9000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d41641e0 0x7f50d40b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.897+0000 7f50d9000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d40b5080 con 0x7f50d4156cd0 2024-04-25T11:18:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.897+0000 7f50d9000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f50d40b5360 con 0x7f50d4164690 2024-04-25T11:18:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.897+0000 7f50d9000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f50d40b5640 con 0x7f50d41641e0 2024-04-25T11:18:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.897+0000 7f50d3400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f50d4164690 0x7f50d4070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.897+0000 7f50d2a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f50d4156cd0 0x7f50d4163ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35830/0 (socket says 172.21.15.7:35830) 2024-04-25T11:18:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.897+0000 7f50d2a00640 1 -- 172.21.15.7:0/1755332264 learned_addr learned my addr 172.21.15.7:0/1755332264 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:18:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.897+0000 7f50d3e00640 1 --2- 172.21.15.7:0/1755332264 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d41641e0 0x7f50d40b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.898+0000 7f50d3400640 1 -- 172.21.15.7:0/1755332264 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d41641e0 msgr2=0x7f50d40b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.898+0000 7f50d3400640 1 --2- 172.21.15.7:0/1755332264 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d41641e0 0x7f50d40b0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.898+0000 7f50d3400640 1 -- 172.21.15.7:0/1755332264 >> v1:172.21.15.7:6789/0 conn(0x7f50d4156cd0 legacy=0x7f50d4163ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.898+0000 7f50d3400640 1 -- 172.21.15.7:0/1755332264 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d40b5920 con 0x7f50d4164690 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.899+0000 7f50d3400640 1 --2- 172.21.15.7:0/1755332264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d4164690 0x7f50d4070a20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f50c40097a0 tx=0x7f50c402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e74698e3fc10ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.899+0000 7f50d2000640 1 -- 172.21.15.7:0/1755332264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4037070 con 0x7f50d4164690 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.899+0000 7f50d2000640 1 -- 172.21.15.7:0/1755332264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c402ae40 con 0x7f50d4164690 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.899+0000 7f50d2000640 1 -- 172.21.15.7:0/1755332264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4032660 con 0x7f50d4164690 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.900+0000 7f50d9000640 1 -- 172.21.15.7:0/1755332264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d4164690 msgr2=0x7f50d4070a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.900+0000 7f50d9000640 1 --2- 172.21.15.7:0/1755332264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d4164690 0x7f50d4070a20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f50c40097a0 tx=0x7f50c402a0f0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.900+0000 7f50d9000640 1 -- 172.21.15.7:0/1755332264 shutdown_connections 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.900+0000 7f50d9000640 1 --2- 172.21.15.7:0/1755332264 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d41641e0 0x7f50d40b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.900+0000 7f50d9000640 1 --2- 172.21.15.7:0/1755332264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d4164690 0x7f50d4070a20 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.900+0000 7f50d9000640 1 -- 172.21.15.7:0/1755332264 >> 172.21.15.7:0/1755332264 conn(0x7f50d40b3820 msgr2=0x7f50d40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.900+0000 7f50d9000640 1 -- 172.21.15.7:0/1755332264 shutdown_connections 2024-04-25T11:18:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.900+0000 7f50d9000640 1 -- 172.21.15.7:0/1755332264 wait complete. 2024-04-25T11:18:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.902+0000 7f50d9000640 1 Processor -- start 2024-04-25T11:18:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.903+0000 7f50d9000640 1 -- start start 2024-04-25T11:18:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.904+0000 7f50d9000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50d4156cd0 0x7f50d415e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.904+0000 7f50d9000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d4164690 0x7f50d41480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.904+0000 7f50d9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d4159fd0 0x7f50d4157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.904+0000 7f50d9000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50d41570e0 con 0x7f50d4159fd0 2024-04-25T11:18:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.904+0000 7f50d9000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4070f60 con 0x7f50d4156cd0 2024-04-25T11:18:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.904+0000 7f50d9000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f50d4157260 con 0x7f50d4164690 2024-04-25T11:18:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.905+0000 7f50d3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d4159fd0 0x7f50d4157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.905+0000 7f50d3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d4159fd0 0x7f50d4157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55452/0 (socket says 172.21.15.7:55452) 2024-04-25T11:18:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.905+0000 7f50d3e00640 1 -- 172.21.15.7:0/1213279496 learned_addr learned my addr 172.21.15.7:0/1213279496 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:18:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.905+0000 7f50d2a00640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d4164690 0x7f50d41480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.905+0000 7f50d3400640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50d4156cd0 0x7f50d415e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.905+0000 7f50d3e00640 1 -- 172.21.15.7:0/1213279496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d4164690 msgr2=0x7f50d41480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.905+0000 7f50d3e00640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d4164690 0x7f50d41480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.905+0000 7f50d3e00640 1 -- 172.21.15.7:0/1213279496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50d4156cd0 msgr2=0x7f50d415e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:18:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.905+0000 7f50d3e00640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50d4156cd0 0x7f50d415e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:18:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.905+0000 7f50d3e00640 1 -- 172.21.15.7:0/1213279496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d41485e0 con 0x7f50d4159fd0 2024-04-25T11:18:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.906+0000 7f50d3e00640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d4159fd0 0x7f50d4157ef0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f50cc007e90 tx=0x7f50cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.906+0000 7f50cb400640 1 -- 172.21.15.7:0/1213279496 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc014420 con 0x7f50d4159fd0 2024-04-25T11:18:59.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.906+0000 7f50cb400640 1 -- 172.21.15.7:0/1213279496 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50cc019070 con 0x7f50d4159fd0 2024-04-25T11:18:59.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.906+0000 7f50d9000640 1 -- 172.21.15.7:0/1213279496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d415ed90 con 0x7f50d4159fd0 2024-04-25T11:18:59.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.906+0000 7f50cb400640 1 -- 172.21.15.7:0/1213279496 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc014800 con 0x7f50d4159fd0 2024-04-25T11:18:59.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.906+0000 7f50d9000640 1 -- 172.21.15.7:0/1213279496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d415f350 con 0x7f50d4159fd0 2024-04-25T11:18:59.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.908+0000 7f50d9000640 1 -- 172.21.15.7:0/1213279496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d4050740 con 0x7f50d4159fd0 2024-04-25T11:18:59.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.909+0000 7f50cb400640 1 -- 172.21.15.7:0/1213279496 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f50cc0149a0 con 0x7f50d4159fd0 2024-04-25T11:18:59.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.910+0000 7f50cb400640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50b40744b0 0x7f50b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:18:59.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.910+0000 7f50cb400640 1 -- 172.21.15.7:0/1213279496 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7f50cc095a30 con 0x7f50d4159fd0 2024-04-25T11:18:59.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.910+0000 7f50d3400640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50b40744b0 0x7f50b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:18:59.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.911+0000 7f50d3400640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50b40744b0 0x7f50b40769c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f50c4034000 tx=0x7f50c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:18:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:18:59.915+0000 7f50cb400640 1 -- 172.21.15.7:0/1213279496 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50cc05fc40 con 0x7f50d4159fd0 2024-04-25T11:19:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.128+0000 7f50d9000640 1 -- 172.21.15.7:0/1213279496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50d406f930 con 0x7f50d4159fd0 2024-04-25T11:19:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.130+0000 7f50cb400640 1 -- 172.21.15.7:0/1213279496 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+13194 (secure 0 0 0) 0x7f50cc063f40 con 0x7f50d4159fd0 2024-04-25T11:19:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:00.130 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":97,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:18:55.541885+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6837/336889324":"2024-04-26T11:18:52.330861+0000","172.21.15.7:6835/336889324":"2024-04-26T11:18:52.330861+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 -- 172.21.15.7:0/1213279496 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50b40744b0 msgr2=0x7f50b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50b40744b0 0x7f50b40769c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f50c4034000 tx=0x7f50c402e000 comp rx=0 tx=0).stop 2024-04-25T11:19:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 -- 172.21.15.7:0/1213279496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d4159fd0 msgr2=0x7f50d4157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d4159fd0 0x7f50d4157ef0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f50cc007e90 tx=0x7f50cc00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 -- 172.21.15.7:0/1213279496 shutdown_connections 2024-04-25T11:19:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50b40744b0 0x7f50b40769c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50d4159fd0 0x7f50d4157ef0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d4164690 0x7f50d41480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 --2- 172.21.15.7:0/1213279496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50d4156cd0 0x7f50d415e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 -- 172.21.15.7:0/1213279496 >> 172.21.15.7:0/1213279496 conn(0x7f50d40b3820 msgr2=0x7f50d40b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 -- 172.21.15.7:0/1213279496 shutdown_connections 2024-04-25T11:19:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.133+0000 7f50d9000640 1 -- 172.21.15.7:0/1213279496 wait complete. 2024-04-25T11:19:00.160 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:00.160 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.444+0000 7f8253e00640 1 Processor -- start 2024-04-25T11:19:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.444+0000 7f8253e00640 1 -- start start 2024-04-25T11:19:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.447+0000 7f8253e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8254144a80 0x7f8254146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.447+0000 7f8253e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82540b4f80 0x7f82540b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.447+0000 7f8253e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82540b66b0 con 0x7f8254145e40 2024-04-25T11:19:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.447+0000 7f8253e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8254145b70 con 0x7f8254144a80 2024-04-25T11:19:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.447+0000 7f8253e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82540b58d0 con 0x7f82540b4f80 2024-04-25T11:19:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.448+0000 7f8252a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8254145e40 0x7f82540b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:38848/0 (socket says 172.21.15.7:38848) 2024-04-25T11:19:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.448+0000 7f8252a00640 1 -- 172.21.15.7:0/1422777108 learned_addr learned my addr 172.21.15.7:0/1422777108 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.448+0000 7f8253400640 1 --2- 172.21.15.7:0/1422777108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82540b4f80 0x7f82540b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.448+0000 7f824be00640 1 --2- 172.21.15.7:0/1422777108 >> v2:172.21.15.87:3300/0 conn(0x7f8254144a80 0x7f8254146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.449+0000 7f8253400640 1 -- 172.21.15.7:0/1422777108 >> v2:172.21.15.87:3300/0 conn(0x7f8254144a80 msgr2=0x7f8254146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.449+0000 7f8253400640 1 --2- 172.21.15.7:0/1422777108 >> v2:172.21.15.87:3300/0 conn(0x7f8254144a80 0x7f8254146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.449+0000 7f8253400640 1 -- 172.21.15.7:0/1422777108 >> v1:172.21.15.87:6789/0 conn(0x7f8254145e40 legacy=0x7f82540b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.449+0000 7f8253400640 1 -- 172.21.15.7:0/1422777108 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82540b5b30 con 0x7f82540b4f80 2024-04-25T11:19:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.450+0000 7f8253400640 1 --2- 172.21.15.7:0/1422777108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82540b4f80 0x7f82540b5390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f824c009d90 tx=0x7f824c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52939314e2ed9e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:00.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.451+0000 7f8252000640 1 -- 172.21.15.7:0/1422777108 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c031600 con 0x7f82540b4f80 2024-04-25T11:19:00.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.451+0000 7f8252000640 1 -- 172.21.15.7:0/1422777108 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824c0317a0 con 0x7f82540b4f80 2024-04-25T11:19:00.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.451+0000 7f8252000640 1 -- 172.21.15.7:0/1422777108 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c031b70 con 0x7f82540b4f80 2024-04-25T11:19:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.451+0000 7f8253e00640 1 -- 172.21.15.7:0/1422777108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82540b4f80 msgr2=0x7f82540b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.451+0000 7f8253e00640 1 --2- 172.21.15.7:0/1422777108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82540b4f80 0x7f82540b5390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f824c009d90 tx=0x7f824c02a5d0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.451+0000 7f8253e00640 1 -- 172.21.15.7:0/1422777108 shutdown_connections 2024-04-25T11:19:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.451+0000 7f8253e00640 1 --2- 172.21.15.7:0/1422777108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82540b4f80 0x7f82540b5390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.451+0000 7f8253e00640 1 --2- 172.21.15.7:0/1422777108 >> v2:172.21.15.87:3300/0 conn(0x7f8254144a80 0x7f8254146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.452+0000 7f8253e00640 1 -- 172.21.15.7:0/1422777108 >> 172.21.15.7:0/1422777108 conn(0x7f82540740f0 msgr2=0x7f825406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.452+0000 7f8253e00640 1 -- 172.21.15.7:0/1422777108 shutdown_connections 2024-04-25T11:19:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.452+0000 7f8253e00640 1 -- 172.21.15.7:0/1422777108 wait complete. 2024-04-25T11:19:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.454+0000 7f8253e00640 1 Processor -- start 2024-04-25T11:19:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.454+0000 7f8253e00640 1 -- start start 2024-04-25T11:19:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.456+0000 7f8253e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82540b4f80 0x7f82541ea3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.456+0000 7f8253e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8254144a80 0x7f825413df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.456+0000 7f8253e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254145e40 0x7f82541e8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.456+0000 7f8253e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82540b63d0 con 0x7f8254145e40 2024-04-25T11:19:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.456+0000 7f8253e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82540b60f0 con 0x7f8254144a80 2024-04-25T11:19:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.456+0000 7f8253e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82541606f0 con 0x7f82540b4f80 2024-04-25T11:19:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.456+0000 7f8253400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254145e40 0x7f82541e8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.456+0000 7f8253400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254145e40 0x7f82541e8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55464/0 (socket says 172.21.15.7:55464) 2024-04-25T11:19:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.456+0000 7f8253400640 1 -- 172.21.15.7:0/3893002029 learned_addr learned my addr 172.21.15.7:0/3893002029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.456+0000 7f8252a00640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82540b4f80 0x7f82541ea3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.457+0000 7f824be00640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8254144a80 0x7f825413df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.457+0000 7f8253400640 1 -- 172.21.15.7:0/3893002029 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82540b4f80 msgr2=0x7f82541ea3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.457+0000 7f8253400640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82540b4f80 0x7f82541ea3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.457+0000 7f8253400640 1 -- 172.21.15.7:0/3893002029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8254144a80 msgr2=0x7f825413df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.457+0000 7f8253400640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8254144a80 0x7f825413df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.457+0000 7f8253400640 1 -- 172.21.15.7:0/3893002029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82541ea930 con 0x7f8254145e40 2024-04-25T11:19:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.457+0000 7f8253400640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254145e40 0x7f82541e8480 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f824c02abc0 tx=0x7f824c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.458+0000 7f824aa00640 1 -- 172.21.15.7:0/3893002029 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c03f070 con 0x7f8254145e40 2024-04-25T11:19:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.458+0000 7f824aa00640 1 -- 172.21.15.7:0/3893002029 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824c030c00 con 0x7f8254145e40 2024-04-25T11:19:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.458+0000 7f8253e00640 1 -- 172.21.15.7:0/3893002029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825413ee30 con 0x7f8254145e40 2024-04-25T11:19:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.458+0000 7f824aa00640 1 -- 172.21.15.7:0/3893002029 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c009450 con 0x7f8254145e40 2024-04-25T11:19:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.458+0000 7f8253e00640 1 -- 172.21.15.7:0/3893002029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825413f3f0 con 0x7f8254145e40 2024-04-25T11:19:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.459+0000 7f8253e00640 1 -- 172.21.15.7:0/3893002029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825413eba0 con 0x7f8254145e40 2024-04-25T11:19:00.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.461+0000 7f824aa00640 1 -- 172.21.15.7:0/3893002029 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f824c03b410 con 0x7f8254145e40 2024-04-25T11:19:00.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.461+0000 7f824aa00640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82300744b0 0x7f82300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.462+0000 7f824aa00640 1 -- 172.21.15.7:0/3893002029 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7f824c0b6340 con 0x7f8254145e40 2024-04-25T11:19:00.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.462+0000 7f8252a00640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82300744b0 0x7f82300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.463+0000 7f8252a00640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82300744b0 0x7f82300769c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8244000f60 tx=0x7f8244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.467+0000 7f824aa00640 1 -- 172.21.15.7:0/3893002029 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f824c080ba0 con 0x7f8254145e40 2024-04-25T11:19:00.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.673+0000 7f8253e00640 1 -- 172.21.15.7:0/3893002029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8254050740 con 0x7f8254145e40 2024-04-25T11:19:00.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.673+0000 7f824aa00640 1 -- 172.21.15.7:0/3893002029 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f824c0804a0 con 0x7f8254145e40 2024-04-25T11:19:00.673 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:19:00.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 -- 172.21.15.7:0/3893002029 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82300744b0 msgr2=0x7f82300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82300744b0 0x7f82300769c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8244000f60 tx=0x7f8244009040 comp rx=0 tx=0).stop 2024-04-25T11:19:00.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 -- 172.21.15.7:0/3893002029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254145e40 msgr2=0x7f82541e8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254145e40 0x7f82541e8480 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f824c02abc0 tx=0x7f824c004a40 comp rx=0 tx=0).stop 2024-04-25T11:19:00.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 -- 172.21.15.7:0/3893002029 shutdown_connections 2024-04-25T11:19:00.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82300744b0 0x7f82300769c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8254145e40 0x7f82541e8480 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8254144a80 0x7f825413df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 --2- 172.21.15.7:0/3893002029 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82540b4f80 0x7f82541ea3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 -- 172.21.15.7:0/3893002029 >> 172.21.15.7:0/3893002029 conn(0x7f82540740f0 msgr2=0x7f82540ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:00.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 -- 172.21.15.7:0/3893002029 shutdown_connections 2024-04-25T11:19:00.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.676+0000 7f8253e00640 1 -- 172.21.15.7:0/3893002029 wait complete. 2024-04-25T11:19:00.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2024-04-25T11:19:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.981+0000 7ff31be00640 1 Processor -- start 2024-04-25T11:19:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.981+0000 7ff31be00640 1 -- start start 2024-04-25T11:19:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.984+0000 7ff31be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff31c0b0030 0x7ff31c15ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.984+0000 7ff31be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff31c06d580 0x7ff31c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.984+0000 7ff31be00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff31c06ded0 con 0x7ff31c06fb00 2024-04-25T11:19:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.984+0000 7ff31be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff31c160450 con 0x7ff31c06d580 2024-04-25T11:19:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.984+0000 7ff31be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff31c0af2b0 con 0x7ff31c0b0030 2024-04-25T11:19:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.984+0000 7ff31aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff31c06fb00 0x7ff31c0b0ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35840/0 (socket says 172.21.15.7:35840) 2024-04-25T11:19:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.984+0000 7ff31aa00640 1 -- 172.21.15.7:0/2091206869 learned_addr learned my addr 172.21.15.7:0/2091206869 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.985+0000 7ff31a000640 1 --2- 172.21.15.7:0/2091206869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff31c0b0030 0x7ff31c15ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.985+0000 7ff31b400640 1 --2- 172.21.15.7:0/2091206869 >> v2:172.21.15.87:3300/0 conn(0x7ff31c06d580 0x7ff31c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.985+0000 7ff319600640 1 -- 172.21.15.7:0/2091206869 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2958458631 0 0) 0x7ff30c001110 con 0x7ff31c06fb00 2024-04-25T11:19:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.985+0000 7ff319600640 1 -- 172.21.15.7:0/2091206869 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff300003520 con 0x7ff31c06fb00 2024-04-25T11:19:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.985+0000 7ff31a000640 1 -- 172.21.15.7:0/2091206869 >> v2:172.21.15.87:3300/0 conn(0x7ff31c06d580 msgr2=0x7ff31c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.985+0000 7ff31a000640 1 --2- 172.21.15.7:0/2091206869 >> v2:172.21.15.87:3300/0 conn(0x7ff31c06d580 0x7ff31c06d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.985+0000 7ff31a000640 1 -- 172.21.15.7:0/2091206869 >> v1:172.21.15.7:6789/0 conn(0x7ff31c06fb00 legacy=0x7ff31c0b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.985+0000 7ff319600640 1 -- 172.21.15.7:0/2091206869 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1859986786 0 0) 0x7ff300003520 con 0x7ff31c06fb00 2024-04-25T11:19:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.985+0000 7ff31a000640 1 -- 172.21.15.7:0/2091206869 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c0b0440 con 0x7ff31c0b0030 2024-04-25T11:19:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.987+0000 7ff31a000640 1 --2- 172.21.15.7:0/2091206869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff31c0b0030 0x7ff31c15ff10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3080097a0 tx=0x7ff30802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=692eaac748e2e6a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.987+0000 7ff319600640 1 -- 172.21.15.7:0/2091206869 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308037070 con 0x7ff31c0b0030 2024-04-25T11:19:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.987+0000 7ff319600640 1 -- 172.21.15.7:0/2091206869 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30802aba0 con 0x7ff31c0b0030 2024-04-25T11:19:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.987+0000 7ff319600640 1 -- 172.21.15.7:0/2091206869 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c001840 con 0x7ff31c0b0030 2024-04-25T11:19:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.988+0000 7ff31be00640 1 -- 172.21.15.7:0/2091206869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff31c0b0030 msgr2=0x7ff31c15ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.988+0000 7ff31be00640 1 --2- 172.21.15.7:0/2091206869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff31c0b0030 0x7ff31c15ff10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3080097a0 tx=0x7ff30802a110 comp rx=0 tx=0).stop 2024-04-25T11:19:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.988+0000 7ff31be00640 1 -- 172.21.15.7:0/2091206869 shutdown_connections 2024-04-25T11:19:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.988+0000 7ff31be00640 1 --2- 172.21.15.7:0/2091206869 >> v2:172.21.15.87:3300/0 conn(0x7ff31c06d580 0x7ff31c06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.988+0000 7ff31be00640 1 --2- 172.21.15.7:0/2091206869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff31c0b0030 0x7ff31c15ff10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.988+0000 7ff31be00640 1 -- 172.21.15.7:0/2091206869 >> 172.21.15.7:0/2091206869 conn(0x7ff31c073090 msgr2=0x7ff31c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.988+0000 7ff31be00640 1 -- 172.21.15.7:0/2091206869 shutdown_connections 2024-04-25T11:19:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.988+0000 7ff31be00640 1 -- 172.21.15.7:0/2091206869 wait complete. 2024-04-25T11:19:00.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.991+0000 7ff31be00640 1 Processor -- start 2024-04-25T11:19:00.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.991+0000 7ff31be00640 1 -- start start 2024-04-25T11:19:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.992+0000 7ff31be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff31c06d580 0x7ff31c14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.992+0000 7ff31be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff31c06fb00 0x7ff31c162e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.992+0000 7ff31be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff31c0b0030 0x7ff31c150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.992+0000 7ff31be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c0afb10 con 0x7ff31c06d580 2024-04-25T11:19:00.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.992+0000 7ff31be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c0af830 con 0x7ff31c06fb00 2024-04-25T11:19:00.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.992+0000 7ff31be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff31c070070 con 0x7ff31c0b0030 2024-04-25T11:19:00.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.993+0000 7ff31aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff31c06d580 0x7ff31c14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.993+0000 7ff31aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff31c06d580 0x7ff31c14f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55466/0 (socket says 172.21.15.7:55466) 2024-04-25T11:19:00.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.993+0000 7ff31aa00640 1 -- 172.21.15.7:0/3255587359 learned_addr learned my addr 172.21.15.7:0/3255587359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.993+0000 7ff31b400640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff31c0b0030 0x7ff31c150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.993+0000 7ff31a000640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff31c06fb00 0x7ff31c162e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.993+0000 7ff31aa00640 1 -- 172.21.15.7:0/3255587359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff31c0b0030 msgr2=0x7ff31c150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.993+0000 7ff31aa00640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff31c0b0030 0x7ff31c150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.993+0000 7ff31aa00640 1 -- 172.21.15.7:0/3255587359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff31c06fb00 msgr2=0x7ff31c162e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.993+0000 7ff31aa00640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff31c06fb00 0x7ff31c162e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:00.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.993+0000 7ff31aa00640 1 -- 172.21.15.7:0/3255587359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c14ff20 con 0x7ff31c06d580 2024-04-25T11:19:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.993+0000 7ff31a000640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff31c06fb00 0x7ff31c162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.994+0000 7ff31aa00640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff31c06d580 0x7ff31c14f9e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c4d0 tx=0x7ff30c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.994+0000 7ff313400640 1 -- 172.21.15.7:0/3255587359 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c017070 con 0x7ff31c06d580 2024-04-25T11:19:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.994+0000 7ff313400640 1 -- 172.21.15.7:0/3255587359 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30c005ce0 con 0x7ff31c06d580 2024-04-25T11:19:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.994+0000 7ff313400640 1 -- 172.21.15.7:0/3255587359 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c007310 con 0x7ff31c06d580 2024-04-25T11:19:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.994+0000 7ff31be00640 1 -- 172.21.15.7:0/3255587359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31c1ea990 con 0x7ff31c06d580 2024-04-25T11:19:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.994+0000 7ff31be00640 1 -- 172.21.15.7:0/3255587359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31c1eaf50 con 0x7ff31c06d580 2024-04-25T11:19:00.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.995+0000 7ff31be00640 1 -- 172.21.15.7:0/3255587359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31c15ff10 con 0x7ff31c06d580 2024-04-25T11:19:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.997+0000 7ff313400640 1 -- 172.21.15.7:0/3255587359 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff30c01a070 con 0x7ff31c06d580 2024-04-25T11:19:00.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.998+0000 7ff313400640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff2fc0744e0 0x7ff2fc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:00.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.998+0000 7ff313400640 1 -- 172.21.15.7:0/3255587359 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7ff30c094610 con 0x7ff31c06d580 2024-04-25T11:19:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.998+0000 7ff31a000640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff2fc0744e0 0x7ff2fc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:00.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:00.999+0000 7ff31a000640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff2fc0744e0 0x7ff2fc0769f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff308034000 tx=0x7ff30802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.003+0000 7ff313400640 1 -- 172.21.15.7:0/3255587359 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff31c15ff10 con 0x7ff31c06d580 2024-04-25T11:19:01.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.273+0000 7ff31be00640 1 -- 172.21.15.7:0/3255587359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) -- 0x7ff31c050740 con 0x7ff31c06d580 2024-04-25T11:19:01.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.577+0000 7ff313400640 1 -- 172.21.15.7:0/3255587359 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v435) ==== 153+0+0 (secure 0 0 0) 0x7ff31c06ded0 con 0x7ff31c06d580 2024-04-25T11:19:01.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.581+0000 7ff31be00640 1 -- 172.21.15.7:0/3255587359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff2fc0744e0 msgr2=0x7ff2fc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:01.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.581+0000 7ff31be00640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff2fc0744e0 0x7ff2fc0769f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff308034000 tx=0x7ff30802e000 comp rx=0 tx=0).stop 2024-04-25T11:19:01.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.581+0000 7ff31be00640 1 -- 172.21.15.7:0/3255587359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff31c06d580 msgr2=0x7ff31c14f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:01.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.581+0000 7ff31be00640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff31c06d580 0x7ff31c14f9e0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c4d0 tx=0x7ff30c00c990 comp rx=0 tx=0).stop 2024-04-25T11:19:01.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.581+0000 7ff31be00640 1 -- 172.21.15.7:0/3255587359 shutdown_connections 2024-04-25T11:19:01.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.581+0000 7ff31be00640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff2fc0744e0 0x7ff2fc0769f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.581+0000 7ff31be00640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff31c0b0030 0x7ff31c150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.581+0000 7ff31be00640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff31c06fb00 0x7ff31c162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.581+0000 7ff31be00640 1 --2- 172.21.15.7:0/3255587359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff31c06d580 0x7ff31c14f9e0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.581+0000 7ff31be00640 1 -- 172.21.15.7:0/3255587359 >> 172.21.15.7:0/3255587359 conn(0x7ff31c073090 msgr2=0x7ff31c163b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:01.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.582+0000 7ff31be00640 1 -- 172.21.15.7:0/3255587359 shutdown_connections 2024-04-25T11:19:01.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.582+0000 7ff31be00640 1 -- 172.21.15.7:0/3255587359 wait complete. 2024-04-25T11:19:01.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:19:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.905+0000 7fe70be00640 1 Processor -- start 2024-04-25T11:19:01.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.905+0000 7fe70be00640 1 -- start start 2024-04-25T11:19:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.908+0000 7fe70be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe70c0b7ac0 0x7fe70c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.908+0000 7fe70be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe70c0b29e0 0x7fe70c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.908+0000 7fe70be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe70c0d1740 con 0x7fe70c1604c0 2024-04-25T11:19:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.908+0000 7fe70be00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe70c0cffe0 con 0x7fe70c0b7ac0 2024-04-25T11:19:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.908+0000 7fe70be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe70c0d2310 con 0x7fe70c0b29e0 2024-04-25T11:19:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.908+0000 7fe70a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe70c0b7ac0 0x7fe70c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.908+0000 7fe70a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe70c0b7ac0 0x7fe70c0af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55472/0 (socket says 172.21.15.7:55472) 2024-04-25T11:19:01.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.908+0000 7fe70a000640 1 -- 172.21.15.7:0/1964749909 learned_addr learned my addr 172.21.15.7:0/1964749909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:01.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.909+0000 7fe70b400640 1 --2- 172.21.15.7:0/1964749909 >> v2:172.21.15.87:3300/0 conn(0x7fe70c0b29e0 0x7fe70c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.909+0000 7fe70a000640 1 -- 172.21.15.7:0/1964749909 >> v2:172.21.15.87:3300/0 conn(0x7fe70c0b29e0 msgr2=0x7fe70c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:01.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.909+0000 7fe70a000640 1 --2- 172.21.15.7:0/1964749909 >> v2:172.21.15.87:3300/0 conn(0x7fe70c0b29e0 0x7fe70c0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.909+0000 7fe70a000640 1 -- 172.21.15.7:0/1964749909 >> v1:172.21.15.87:6789/0 conn(0x7fe70c1604c0 legacy=0x7fe70c0b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:01.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.909+0000 7fe70a000640 1 -- 172.21.15.7:0/1964749909 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c0d19e0 con 0x7fe70c0b7ac0 2024-04-25T11:19:01.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.910+0000 7fe70a000640 1 --2- 172.21.15.7:0/1964749909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c0b7ac0 0x7fe70c0af3e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe70c073c00 tx=0x7fe6f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca04bf0353211f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:01.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.910+0000 7fe709600640 1 -- 172.21.15.7:0/1964749909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8037070 con 0x7fe70c0b7ac0 2024-04-25T11:19:01.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.910+0000 7fe709600640 1 -- 172.21.15.7:0/1964749909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f8032440 con 0x7fe70c0b7ac0 2024-04-25T11:19:01.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.910+0000 7fe709600640 1 -- 172.21.15.7:0/1964749909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8032810 con 0x7fe70c0b7ac0 2024-04-25T11:19:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.911+0000 7fe70be00640 1 -- 172.21.15.7:0/1964749909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c0b7ac0 msgr2=0x7fe70c0af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.911+0000 7fe70be00640 1 --2- 172.21.15.7:0/1964749909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c0b7ac0 0x7fe70c0af3e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe70c073c00 tx=0x7fe6f802a230 comp rx=0 tx=0).stop 2024-04-25T11:19:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.911+0000 7fe70be00640 1 -- 172.21.15.7:0/1964749909 shutdown_connections 2024-04-25T11:19:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.911+0000 7fe70be00640 1 --2- 172.21.15.7:0/1964749909 >> v2:172.21.15.87:3300/0 conn(0x7fe70c0b29e0 0x7fe70c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.911+0000 7fe70be00640 1 --2- 172.21.15.7:0/1964749909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c0b7ac0 0x7fe70c0af3e0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.911+0000 7fe70be00640 1 -- 172.21.15.7:0/1964749909 >> 172.21.15.7:0/1964749909 conn(0x7fe70c09b540 msgr2=0x7fe70c09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.912+0000 7fe70be00640 1 -- 172.21.15.7:0/1964749909 shutdown_connections 2024-04-25T11:19:01.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.912+0000 7fe70be00640 1 -- 172.21.15.7:0/1964749909 wait complete. 2024-04-25T11:19:01.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.915+0000 7fe70be00640 1 Processor -- start 2024-04-25T11:19:01.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.915+0000 7fe70be00640 1 -- start start 2024-04-25T11:19:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.917+0000 7fe70be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe70c0b29e0 0x7fe70c1f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.917+0000 7fe70be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c1604c0 0x7fe70c15d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.917+0000 7fe70be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe70c09fd30 0x7fe70c161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.917+0000 7fe70be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c0d1fa0 con 0x7fe70c1604c0 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.917+0000 7fe70be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c0cfd80 con 0x7fe70c09fd30 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.917+0000 7fe70be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe70c0d27a0 con 0x7fe70c0b29e0 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.917+0000 7fe70a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c1604c0 0x7fe70c15d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.917+0000 7fe70a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c1604c0 0x7fe70c15d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55484/0 (socket says 172.21.15.7:55484) 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.917+0000 7fe70a000640 1 -- 172.21.15.7:0/1623049354 learned_addr learned my addr 172.21.15.7:0/1623049354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.918+0000 7fe70aa00640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe70c0b29e0 0x7fe70c1f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.918+0000 7fe70b400640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe70c09fd30 0x7fe70c161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.918+0000 7fe70a000640 1 -- 172.21.15.7:0/1623049354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe70c0b29e0 msgr2=0x7fe70c1f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.918+0000 7fe70a000640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe70c0b29e0 0x7fe70c1f6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.918+0000 7fe70a000640 1 -- 172.21.15.7:0/1623049354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe70c09fd30 msgr2=0x7fe70c161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.918+0000 7fe70a000640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe70c09fd30 0x7fe70c161e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:01.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.918+0000 7fe70a000640 1 -- 172.21.15.7:0/1623049354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c1608d0 con 0x7fe70c1604c0 2024-04-25T11:19:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.918+0000 7fe70b400640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe70c09fd30 0x7fe70c161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.918+0000 7fe70a000640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c1604c0 0x7fe70c15d850 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8035040 tx=0x7fe6f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.918+0000 7fe702a00640 1 -- 172.21.15.7:0/1623049354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8037040 con 0x7fe70c1604c0 2024-04-25T11:19:01.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.919+0000 7fe702a00640 1 -- 172.21.15.7:0/1623049354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f8004230 con 0x7fe70c1604c0 2024-04-25T11:19:01.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.919+0000 7fe702a00640 1 -- 172.21.15.7:0/1623049354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8003610 con 0x7fe70c1604c0 2024-04-25T11:19:01.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.919+0000 7fe70be00640 1 -- 172.21.15.7:0/1623049354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c1f6aa0 con 0x7fe70c1604c0 2024-04-25T11:19:01.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.919+0000 7fe70be00640 1 -- 172.21.15.7:0/1623049354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c1610b0 con 0x7fe70c1604c0 2024-04-25T11:19:01.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.920+0000 7fe70be00640 1 -- 172.21.15.7:0/1623049354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70c050740 con 0x7fe70c1604c0 2024-04-25T11:19:01.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.921+0000 7fe702a00640 1 -- 172.21.15.7:0/1623049354 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe6f80037b0 con 0x7fe70c1604c0 2024-04-25T11:19:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.922+0000 7fe702a00640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6ec0744b0 0x7fe6ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.922+0000 7fe702a00640 1 -- 172.21.15.7:0/1623049354 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7fe6f80b5db0 con 0x7fe70c1604c0 2024-04-25T11:19:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.923+0000 7fe70aa00640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6ec0744b0 0x7fe6ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:01.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.923+0000 7fe70aa00640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6ec0744b0 0x7fe6ec0769c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc006fd0 tx=0x7fe6fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:01.930+0000 7fe702a00640 1 -- 172.21.15.7:0/1623049354 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe6f8080740 con 0x7fe70c1604c0 2024-04-25T11:19:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.129+0000 7fe70be00640 1 -- 172.21.15.7:0/1623049354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe70c160b60 con 0x7fe70c1604c0 2024-04-25T11:19:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.135+0000 7fe702a00640 1 -- 172.21.15.7:0/1623049354 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fe6f8080040 con 0x7fe70c1604c0 2024-04-25T11:19:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.138+0000 7fe70be00640 1 -- 172.21.15.7:0/1623049354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6ec0744b0 msgr2=0x7fe6ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.138+0000 7fe70be00640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6ec0744b0 0x7fe6ec0769c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc006fd0 tx=0x7fe6fc008040 comp rx=0 tx=0).stop 2024-04-25T11:19:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.138+0000 7fe70be00640 1 -- 172.21.15.7:0/1623049354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c1604c0 msgr2=0x7fe70c15d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.138+0000 7fe70be00640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c1604c0 0x7fe70c15d850 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8035040 tx=0x7fe6f80046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.138+0000 7fe70be00640 1 -- 172.21.15.7:0/1623049354 shutdown_connections 2024-04-25T11:19:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.138+0000 7fe70be00640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6ec0744b0 0x7fe6ec0769c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.138+0000 7fe70be00640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe70c09fd30 0x7fe70c161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.138+0000 7fe70be00640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe70c1604c0 0x7fe70c15d850 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.138+0000 7fe70be00640 1 --2- 172.21.15.7:0/1623049354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe70c0b29e0 0x7fe70c1f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.138+0000 7fe70be00640 1 -- 172.21.15.7:0/1623049354 >> 172.21.15.7:0/1623049354 conn(0x7fe70c09b540 msgr2=0x7fe70c0aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.139+0000 7fe70be00640 1 -- 172.21.15.7:0/1623049354 shutdown_connections 2024-04-25T11:19:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.139+0000 7fe70be00640 1 -- 172.21.15.7:0/1623049354 wait complete. 2024-04-25T11:19:02.163 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:19:02.163 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:19:02.262 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:19:02.376 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:19:02.376 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:19:02.376 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:02.435 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:19:02.436 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:19:02.436 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:02.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:19:02.493 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:19:02.493 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:02.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:19:02.553 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:19:02.554 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:02.605 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:19:02.605 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:19:02.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:02.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.890+0000 7fb721000640 1 Processor -- start 2024-04-25T11:19:02.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.890+0000 7fb721000640 1 -- start start 2024-04-25T11:19:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.892+0000 7fb721000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb71c06d930 0x7fb71c1619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.892+0000 7fb721000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71c075b10 con 0x7fb71c162330 2024-04-25T11:19:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.892+0000 7fb721000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71c0b3480 con 0x7fb71c161f20 2024-04-25T11:19:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.892+0000 7fb721000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb71c15f4a0 con 0x7fb71c06d930 2024-04-25T11:19:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.892+0000 7fb71be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb71c162330 0x7fb71c15ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35846/0 (socket says 172.21.15.7:35846) 2024-04-25T11:19:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.892+0000 7fb71be00640 1 -- 172.21.15.7:0/1464789181 learned_addr learned my addr 172.21.15.7:0/1464789181 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71b400640 1 --2- 172.21.15.7:0/1464789181 >> v2:172.21.15.87:3300/0 conn(0x7fb71c06d930 0x7fb71c1619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4138891128 0 0) 0x7fb71c075b10 con 0x7fb71c162330 2024-04-25T11:19:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb700003520 con 0x7fb71c162330 2024-04-25T11:19:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3251749967 0 0) 0x7fb700003520 con 0x7fb71c162330 2024-04-25T11:19:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 >> v2:172.21.15.87:3300/0 conn(0x7fb71c06d930 msgr2=0x7fb71c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 --2- 172.21.15.7:0/1464789181 >> v2:172.21.15.87:3300/0 conn(0x7fb71c06d930 0x7fb71c1619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 >> v1:172.21.15.87:6789/0 conn(0x7fb71c161f20 legacy=0x7fb71c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c0b3760 con 0x7fb71c162330 2024-04-25T11:19:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fb7140019f0 con 0x7fb71c162330 2024-04-25T11:19:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 >> v1:172.21.15.7:6789/0 conn(0x7fb71c162330 legacy=0x7fb71c15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7000039b0 0x7fb700003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 0x7fb700004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb700005250 0x7fb70000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c15f4a0 con 0x7fb700004800 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb70000c1e0 con 0x7fb700005250 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.893+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb70000c3e0 con 0x7fb7000039b0 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.894+0000 7fb71aa00640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 0x7fb700004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.894+0000 7fb71be00640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb700005250 0x7fb70000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.894+0000 7fb71b400640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7000039b0 0x7fb700003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.894+0000 7fb71aa00640 1 -- 172.21.15.7:0/1464789181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7000039b0 msgr2=0x7fb700003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.894+0000 7fb71aa00640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7000039b0 0x7fb700003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.894+0000 7fb71aa00640 1 -- 172.21.15.7:0/1464789181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb700005250 msgr2=0x7fb70000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.894+0000 7fb71aa00640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb700005250 0x7fb70000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.894+0000 7fb71aa00640 1 -- 172.21.15.7:0/1464789181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb700003520 con 0x7fb700004800 2024-04-25T11:19:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.894+0000 7fb71be00640 1 -- 172.21.15.7:0/1464789181 reap_dead start 2024-04-25T11:19:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.894+0000 7fb71b400640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7000039b0 0x7fb700003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.895+0000 7fb71aa00640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 0x7fb700004c10 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0740b0 tx=0x7fb700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e46a42fa671cfa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.896+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704001370 con 0x7fb700004800 2024-04-25T11:19:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.896+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7040052d0 con 0x7fb700004800 2024-04-25T11:19:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.896+0000 7fb71a000640 1 -- 172.21.15.7:0/1464789181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70402ab70 con 0x7fb700004800 2024-04-25T11:19:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.896+0000 7fb721000640 1 -- 172.21.15.7:0/1464789181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 msgr2=0x7fb700004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.896+0000 7fb721000640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 0x7fb700004c10 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0740b0 tx=0x7fb700005830 comp rx=0 tx=0).stop 2024-04-25T11:19:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.897+0000 7fb721000640 1 -- 172.21.15.7:0/1464789181 shutdown_connections 2024-04-25T11:19:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.897+0000 7fb721000640 1 --2- 172.21.15.7:0/1464789181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 0x7fb700004c10 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.897+0000 7fb721000640 1 -- 172.21.15.7:0/1464789181 >> 172.21.15.7:0/1464789181 conn(0x7fb71c073b60 msgr2=0x7fb71c0b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.897+0000 7fb721000640 1 -- 172.21.15.7:0/1464789181 shutdown_connections 2024-04-25T11:19:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.897+0000 7fb721000640 1 -- 172.21.15.7:0/1464789181 wait complete. 2024-04-25T11:19:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.899+0000 7fb721000640 1 Processor -- start 2024-04-25T11:19:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.899+0000 7fb721000640 1 -- start start 2024-04-25T11:19:02.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.901+0000 7fb721000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 0x7fb71c14b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.901+0000 7fb721000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb71c0b00e0 0x7fb71c070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.901+0000 7fb721000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb71c0b04f0 0x7fb71c0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.901+0000 7fb721000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c075db0 con 0x7fb700004800 2024-04-25T11:19:02.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.901+0000 7fb721000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c162740 con 0x7fb71c0b04f0 2024-04-25T11:19:02.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.901+0000 7fb721000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c06d5b0 con 0x7fb71c0b00e0 2024-04-25T11:19:02.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.901+0000 7fb71b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 0x7fb71c14b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.901+0000 7fb71b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 0x7fb71c14b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55506/0 (socket says 172.21.15.7:55506) 2024-04-25T11:19:02.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.901+0000 7fb71b400640 1 -- 172.21.15.7:0/503444601 learned_addr learned my addr 172.21.15.7:0/503444601 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:02.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.901+0000 7fb71be00640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb71c0b04f0 0x7fb71c0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.901+0000 7fb71aa00640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb71c0b00e0 0x7fb71c070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb71b400640 1 -- 172.21.15.7:0/503444601 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb71c0b00e0 msgr2=0x7fb71c070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb71b400640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb71c0b00e0 0x7fb71c070090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb71b400640 1 -- 172.21.15.7:0/503444601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb71c0b04f0 msgr2=0x7fb71c0705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb71b400640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb71c0b04f0 0x7fb71c0705d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb71b400640 1 -- 172.21.15.7:0/503444601 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c1539b0 con 0x7fb700004800 2024-04-25T11:19:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb71be00640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb71c0b04f0 0x7fb71c0705d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb71b400640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 0x7fb71c14b730 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0107c0 tx=0x7fb70c010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb713400640 1 -- 172.21.15.7:0/503444601 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00f040 con 0x7fb700004800 2024-04-25T11:19:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb713400640 1 -- 172.21.15.7:0/503444601 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70c00eb60 con 0x7fb700004800 2024-04-25T11:19:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb721000640 1 -- 172.21.15.7:0/503444601 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c0b0cc0 con 0x7fb700004800 2024-04-25T11:19:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb713400640 1 -- 172.21.15.7:0/503444601 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c010e70 con 0x7fb700004800 2024-04-25T11:19:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.902+0000 7fb721000640 1 -- 172.21.15.7:0/503444601 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c0af0a0 con 0x7fb700004800 2024-04-25T11:19:02.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.904+0000 7fb721000640 1 -- 172.21.15.7:0/503444601 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c050740 con 0x7fb700004800 2024-04-25T11:19:02.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.905+0000 7fb713400640 1 -- 172.21.15.7:0/503444601 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb70c01c400 con 0x7fb700004800 2024-04-25T11:19:02.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.906+0000 7fb713400640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb6f80744b0 0x7fb6f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.906+0000 7fb713400640 1 -- 172.21.15.7:0/503444601 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7fb70c0955e0 con 0x7fb700004800 2024-04-25T11:19:02.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.907+0000 7fb71aa00640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb6f80744b0 0x7fb6f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:02.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.908+0000 7fb71aa00640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb6f80744b0 0x7fb6f80769c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb704032000 tx=0x7fb70402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:02.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:02.911+0000 7fb713400640 1 -- 172.21.15.7:0/503444601 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb70c018030 con 0x7fb700004800 2024-04-25T11:19:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.153+0000 7fb721000640 1 -- 172.21.15.7:0/503444601 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb71c0d2d50 con 0x7fb700004800 2024-04-25T11:19:03.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.155+0000 7fb713400640 1 -- 172.21.15.7:0/503444601 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+3857 (secure 0 0 0) 0x7fb70c05fed0 con 0x7fb700004800 2024-04-25T11:19:03.154 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:03.155 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5555,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/684015857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":684015857},{"type":"v1","addr":"172.21.15.7:6836","nonce":684015857}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":5561,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/493684223","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":493684223},{"type":"v1","addr":"172.21.15.87:6835","nonce":493684223}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47},{"gid":6109,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/1959070928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1959070928},{"type":"v1","addr":"172.21.15.87:6833","nonce":1959070928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":46},{"gid":7503,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2016945879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2016945879},{"type":"v1","addr":"172.21.15.7:6837","nonce":2016945879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[]} 2024-04-25T11:19:03.155 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 60 2024-04-25T11:19:03.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.157+0000 7fb721000640 1 -- 172.21.15.7:0/503444601 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb6f80744b0 msgr2=0x7fb6f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.157+0000 7fb721000640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb6f80744b0 0x7fb6f80769c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb704032000 tx=0x7fb70402e000 comp rx=0 tx=0).stop 2024-04-25T11:19:03.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.157+0000 7fb721000640 1 -- 172.21.15.7:0/503444601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 msgr2=0x7fb71c14b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.157+0000 7fb721000640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 0x7fb71c14b730 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0107c0 tx=0x7fb70c010c80 comp rx=0 tx=0).stop 2024-04-25T11:19:03.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.157+0000 7fb721000640 1 -- 172.21.15.7:0/503444601 shutdown_connections 2024-04-25T11:19:03.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.158+0000 7fb721000640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb6f80744b0 0x7fb6f80769c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.158+0000 7fb721000640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb71c0b04f0 0x7fb71c0705d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.158+0000 7fb721000640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb71c0b00e0 0x7fb71c070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.158+0000 7fb721000640 1 --2- 172.21.15.7:0/503444601 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb700004800 0x7fb71c14b730 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.158+0000 7fb721000640 1 -- 172.21.15.7:0/503444601 >> 172.21.15.7:0/503444601 conn(0x7fb71c073b60 msgr2=0x7fb71c0b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.158+0000 7fb721000640 1 -- 172.21.15.7:0/503444601 shutdown_connections 2024-04-25T11:19:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.158+0000 7fb721000640 1 -- 172.21.15.7:0/503444601 wait complete. 2024-04-25T11:19:03.186 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:19:03.186 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:19:03.187 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:19:03.187 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:19:03.187 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:19:03.187 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:19:03.187 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:19:03.187 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:19:03.187 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:19:03.188 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:19:03.188 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:19:03.188 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:19:03.188 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:19:03.188 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:19:03.188 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:19:03.188 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:19:03.218 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:19:03.218 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:19:03.222 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:19:03.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:19:03.224 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:19:03.224 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:19:03.231 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:19:03.231 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:19:03.273 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:19:03.273 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:03.277 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:19:03.277 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:03.279 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:19:03.280 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:03.284 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:19:03.284 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:03.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:19:03.503 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:19:03.503 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:19:03.504 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:19:03.504 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:19:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.546+0000 7f6ccf600640 1 Processor -- start 2024-04-25T11:19:03.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.546+0000 7f6ccf600640 1 -- start start 2024-04-25T11:19:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.548+0000 7f6ccf600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6cc815f180 0x7f6cc8161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.548+0000 7f6ccf600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6cc81621a0 0x7f6cc806d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.548+0000 7f6ccf600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6cc806dac0 0x7f6cc806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.548+0000 7f6ccf600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6cc815f590 con 0x7f6cc81621a0 2024-04-25T11:19:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.548+0000 7f6ccf600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6cc8158ae0 con 0x7f6cc815f180 2024-04-25T11:19:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.548+0000 7f6ccf600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc806ded0 con 0x7f6cc806dac0 2024-04-25T11:19:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.548+0000 7f6cc7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6cc81621a0 0x7f6cc806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.548+0000 7f6cc7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6cc81621a0 0x7f6cc806d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55540/0 (socket says 172.21.15.7:55540) 2024-04-25T11:19:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.548+0000 7f6cc7e00640 1 -- 172.21.15.7:0/1965962856 learned_addr learned my addr 172.21.15.7:0/1965962856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.548+0000 7f6cccc00640 1 --2- 172.21.15.7:0/1965962856 >> v2:172.21.15.87:3300/0 conn(0x7f6cc815f180 0x7f6cc8161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.549+0000 7f6cc7e00640 1 -- 172.21.15.7:0/1965962856 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6cc806dac0 msgr2=0x7f6cc806f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:03.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.549+0000 7f6cc7e00640 1 --2- 172.21.15.7:0/1965962856 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6cc806dac0 0x7f6cc806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.549+0000 7f6cc7e00640 1 -- 172.21.15.7:0/1965962856 >> v2:172.21.15.87:3300/0 conn(0x7f6cc815f180 msgr2=0x7f6cc8161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:03.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.549+0000 7f6cc7e00640 1 --2- 172.21.15.7:0/1965962856 >> v2:172.21.15.87:3300/0 conn(0x7f6cc815f180 0x7f6cc8161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.549+0000 7f6cc7e00640 1 -- 172.21.15.7:0/1965962856 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc8162700 con 0x7f6cc81621a0 2024-04-25T11:19:03.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.550+0000 7f6cc7e00640 1 --2- 172.21.15.7:0/1965962856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc81621a0 0x7f6cc806d580 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8070340 tx=0x7f6cb402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b64f47a745ce3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.550+0000 7f6cc7400640 1 -- 172.21.15.7:0/1965962856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4037070 con 0x7f6cc81621a0 2024-04-25T11:19:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.550+0000 7f6cc7400640 1 -- 172.21.15.7:0/1965962856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4032440 con 0x7f6cc81621a0 2024-04-25T11:19:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.550+0000 7f6cc7400640 1 -- 172.21.15.7:0/1965962856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4032810 con 0x7f6cc81621a0 2024-04-25T11:19:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.551+0000 7f6ccf600640 1 -- 172.21.15.7:0/1965962856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc81621a0 msgr2=0x7f6cc806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.551+0000 7f6ccf600640 1 --2- 172.21.15.7:0/1965962856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc81621a0 0x7f6cc806d580 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8070340 tx=0x7f6cb402a210 comp rx=0 tx=0).stop 2024-04-25T11:19:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.551+0000 7f6ccf600640 1 -- 172.21.15.7:0/1965962856 shutdown_connections 2024-04-25T11:19:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.551+0000 7f6ccf600640 1 --2- 172.21.15.7:0/1965962856 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6cc806dac0 0x7f6cc806f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.551+0000 7f6ccf600640 1 --2- 172.21.15.7:0/1965962856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc81621a0 0x7f6cc806d580 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.551+0000 7f6ccf600640 1 --2- 172.21.15.7:0/1965962856 >> v2:172.21.15.87:3300/0 conn(0x7f6cc815f180 0x7f6cc8161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.551+0000 7f6ccf600640 1 -- 172.21.15.7:0/1965962856 >> 172.21.15.7:0/1965962856 conn(0x7f6cc80732b0 msgr2=0x7f6cc80736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:03.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.551+0000 7f6ccf600640 1 -- 172.21.15.7:0/1965962856 shutdown_connections 2024-04-25T11:19:03.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.551+0000 7f6ccf600640 1 -- 172.21.15.7:0/1965962856 wait complete. 2024-04-25T11:19:03.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.554+0000 7f6ccf600640 1 Processor -- start 2024-04-25T11:19:03.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.554+0000 7f6ccf600640 1 -- start start 2024-04-25T11:19:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.555+0000 7f6ccf600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc806dac0 0x7f6cc81464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.555+0000 7f6ccf600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc815f180 0x7f6cc8157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6ccf600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6cc81621a0 0x7f6cc81fa040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6ccf600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc815ec60 con 0x7f6cc815f180 2024-04-25T11:19:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6ccf600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc815e980 con 0x7f6cc806dac0 2024-04-25T11:19:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6ccf600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc815e670 con 0x7f6cc81621a0 2024-04-25T11:19:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6cc7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc815f180 0x7f6cc8157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6cc7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc815f180 0x7f6cc8157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55548/0 (socket says 172.21.15.7:55548) 2024-04-25T11:19:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6cc7e00640 1 -- 172.21.15.7:0/1546482587 learned_addr learned my addr 172.21.15.7:0/1546482587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6cccc00640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc806dac0 0x7f6cc81464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6ccd600640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6cc81621a0 0x7f6cc81fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6cc7e00640 1 -- 172.21.15.7:0/1546482587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6cc81621a0 msgr2=0x7f6cc81fa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6cc7e00640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6cc81621a0 0x7f6cc81fa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6cc7e00640 1 -- 172.21.15.7:0/1546482587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc806dac0 msgr2=0x7f6cc81464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6cc7e00640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc806dac0 0x7f6cc81464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:03.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.556+0000 7f6cc7e00640 1 -- 172.21.15.7:0/1546482587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc8157a60 con 0x7f6cc815f180 2024-04-25T11:19:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.557+0000 7f6cc7e00640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc815f180 0x7f6cc8157080 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009870 tx=0x7f6cb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.557+0000 7f6cc4c00640 1 -- 172.21.15.7:0/1546482587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4041020 con 0x7f6cc815f180 2024-04-25T11:19:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.557+0000 7f6cc4c00640 1 -- 172.21.15.7:0/1546482587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4030640 con 0x7f6cc815f180 2024-04-25T11:19:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.557+0000 7f6cc4c00640 1 -- 172.21.15.7:0/1546482587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4037040 con 0x7f6cc815f180 2024-04-25T11:19:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.557+0000 7f6ccf600640 1 -- 172.21.15.7:0/1546482587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc8157d40 con 0x7f6cc815f180 2024-04-25T11:19:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.558+0000 7f6ccf600640 1 -- 172.21.15.7:0/1546482587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc8158300 con 0x7f6cc815f180 2024-04-25T11:19:03.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.559+0000 7f6ccf600640 1 -- 172.21.15.7:0/1546482587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc81577b0 con 0x7f6cc815f180 2024-04-25T11:19:03.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.560+0000 7f6cc4c00640 1 -- 172.21.15.7:0/1546482587 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6cb4044020 con 0x7f6cc815f180 2024-04-25T11:19:03.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.561+0000 7f6cc4c00640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ca8074500 0x7f6ca8076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:03.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.561+0000 7f6cc4c00640 1 -- 172.21.15.7:0/1546482587 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5335+0+0 (secure 0 0 0) 0x7f6cb40b57d0 con 0x7f6cc815f180 2024-04-25T11:19:03.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.561+0000 7f6cccc00640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ca8074500 0x7f6ca8076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:03.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.562+0000 7f6cccc00640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ca8074500 0x7f6ca8076a10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8070310 tx=0x7f6cbc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.566+0000 7f6cc4c00640 1 -- 172.21.15.7:0/1546482587 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cb404b030 con 0x7f6cc815f180 2024-04-25T11:19:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:03.789+0000 7f6ccf600640 1 -- 172.21.15.7:0/1546482587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6cc8050740 con 0x7f6cc815f180 2024-04-25T11:19:04.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.640+0000 7f6cc4c00640 1 -- 172.21.15.7:0/1546482587 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v102) ==== 97+0+0 (secure 0 0 0) 0x7f6cb407fa60 con 0x7f6cc815f180 2024-04-25T11:19:04.640 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 -- 172.21.15.7:0/1546482587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ca8074500 msgr2=0x7f6ca8076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ca8074500 0x7f6ca8076a10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8070310 tx=0x7f6cbc009190 comp rx=0 tx=0).stop 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 -- 172.21.15.7:0/1546482587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc815f180 msgr2=0x7f6cc8157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc815f180 0x7f6cc8157080 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009870 tx=0x7f6cb40046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 -- 172.21.15.7:0/1546482587 shutdown_connections 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6ca8074500 0x7f6ca8076a10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6cc81621a0 0x7f6cc81fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6cc815f180 0x7f6cc8157080 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 --2- 172.21.15.7:0/1546482587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc806dac0 0x7f6cc81464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 -- 172.21.15.7:0/1546482587 >> 172.21.15.7:0/1546482587 conn(0x7f6cc80732b0 msgr2=0x7f6cc815dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 -- 172.21.15.7:0/1546482587 shutdown_connections 2024-04-25T11:19:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.644+0000 7f6ccf600640 1 -- 172.21.15.7:0/1546482587 wait complete. 2024-04-25T11:19:04.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:19:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.955+0000 7fd1e2a00640 1 Processor -- start 2024-04-25T11:19:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.956+0000 7fd1e2a00640 1 -- start start 2024-04-25T11:19:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.958+0000 7fd1e2a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1dc15e2f0 0x7fd1dc161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.958+0000 7fd1e2a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1dc162420 0x7fd1dc06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.958+0000 7fd1e2a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1dc15e700 con 0x7fd1dc15baa0 2024-04-25T11:19:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.958+0000 7fd1e2a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd1dc162830 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.958+0000 7fd1e2a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd1dc15e980 con 0x7fd1dc162420 2024-04-25T11:19:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.958+0000 7fd1e0c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1dc15e2f0 0x7fd1dc161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.958+0000 7fd1e0c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1dc15e2f0 0x7fd1dc161ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55556/0 (socket says 172.21.15.7:55556) 2024-04-25T11:19:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.958+0000 7fd1e0c00640 1 -- 172.21.15.7:0/3820552627 learned_addr learned my addr 172.21.15.7:0/3820552627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.959+0000 7fd1e0c00640 1 -- 172.21.15.7:0/3820552627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1dc162420 msgr2=0x7fd1dc06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.959+0000 7fd1e2000640 1 --2- 172.21.15.7:0/3820552627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1dc162420 0x7fd1dc06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.959+0000 7fd1e0c00640 1 --2- 172.21.15.7:0/3820552627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1dc162420 0x7fd1dc06d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.959+0000 7fd1e0c00640 1 -- 172.21.15.7:0/3820552627 >> v1:172.21.15.87:6789/0 conn(0x7fd1dc15baa0 legacy=0x7fd1dc070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.959+0000 7fd1e0c00640 1 -- 172.21.15.7:0/3820552627 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1dc15ec60 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.960+0000 7fd1e0c00640 1 --2- 172.21.15.7:0/3820552627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc15e2f0 0x7fd1dc161ee0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc074530 tx=0x7fd1cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ea20041a1213eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.961+0000 7fd1dbe00640 1 -- 172.21.15.7:0/3820552627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc036070 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.961+0000 7fd1dbe00640 1 -- 172.21.15.7:0/3820552627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1cc031600 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.961+0000 7fd1dbe00640 1 -- 172.21.15.7:0/3820552627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc0319d0 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.961+0000 7fd1e2a00640 1 -- 172.21.15.7:0/3820552627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc15e2f0 msgr2=0x7fd1dc161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.961+0000 7fd1e2a00640 1 --2- 172.21.15.7:0/3820552627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc15e2f0 0x7fd1dc161ee0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc074530 tx=0x7fd1cc02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.962+0000 7fd1e2a00640 1 -- 172.21.15.7:0/3820552627 shutdown_connections 2024-04-25T11:19:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.962+0000 7fd1e2a00640 1 --2- 172.21.15.7:0/3820552627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1dc162420 0x7fd1dc06d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.962+0000 7fd1e2a00640 1 --2- 172.21.15.7:0/3820552627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc15e2f0 0x7fd1dc161ee0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.962+0000 7fd1e2a00640 1 -- 172.21.15.7:0/3820552627 >> 172.21.15.7:0/3820552627 conn(0x7fd1dc073090 msgr2=0x7fd1dc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:04.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.962+0000 7fd1e2a00640 1 -- 172.21.15.7:0/3820552627 shutdown_connections 2024-04-25T11:19:04.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.962+0000 7fd1e2a00640 1 -- 172.21.15.7:0/3820552627 wait complete. 2024-04-25T11:19:04.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.964+0000 7fd1e2a00640 1 Processor -- start 2024-04-25T11:19:04.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.964+0000 7fd1e2a00640 1 -- start start 2024-04-25T11:19:04.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.966+0000 7fd1e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc15e2f0 0x7fd1dc141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.966+0000 7fd1e2a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1dc162420 0x7fd1dc138880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.966+0000 7fd1e2a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1dc135560 0x7fd1dc134070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.966+0000 7fd1e2a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc06dac0 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.966+0000 7fd1e2a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1619c0 con 0x7fd1dc135560 2024-04-25T11:19:04.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.966+0000 7fd1e2a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd1dc06dc40 con 0x7fd1dc162420 2024-04-25T11:19:04.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.966+0000 7fd1e1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc15e2f0 0x7fd1dc141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.966+0000 7fd1e1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc15e2f0 0x7fd1dc141bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55572/0 (socket says 172.21.15.7:55572) 2024-04-25T11:19:04.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.966+0000 7fd1e1600640 1 -- 172.21.15.7:0/2954415789 learned_addr learned my addr 172.21.15.7:0/2954415789 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:04.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1e2000640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1dc135560 0x7fd1dc134070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1e1600640 1 -- 172.21.15.7:0/2954415789 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1dc162420 msgr2=0x7fd1dc138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1e1600640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1dc162420 0x7fd1dc138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1e1600640 1 -- 172.21.15.7:0/2954415789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1dc135560 msgr2=0x7fd1dc134070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1e1600640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1dc135560 0x7fd1dc134070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1e1600640 1 -- 172.21.15.7:0/2954415789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1dc132fd0 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1e1600640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc15e2f0 0x7fd1dc141bb0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000b420 tx=0x7fd1d000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1d9600640 1 -- 172.21.15.7:0/2954415789 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0011830 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1d9600640 1 -- 172.21.15.7:0/2954415789 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d0016070 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1e2a00640 1 -- 172.21.15.7:0/2954415789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1dc1332b0 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1d9600640 1 -- 172.21.15.7:0/2954415789 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0011c10 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.967+0000 7fd1e2a00640 1 -- 172.21.15.7:0/2954415789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1dc133870 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.968+0000 7fd1e2a00640 1 -- 172.21.15.7:0/2954415789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1dc0712f0 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.970+0000 7fd1d9600640 1 -- 172.21.15.7:0/2954415789 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd1d001b460 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.972+0000 7fd1d9600640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1c00744e0 0x7fd1c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:04.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.972+0000 7fd1d9600640 1 -- 172.21.15.7:0/2954415789 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7fd1d0094bc0 con 0x7fd1dc15e2f0 2024-04-25T11:19:04.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.972+0000 7fd1e0c00640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1c00744e0 0x7fd1c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:04.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.973+0000 7fd1e0c00640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1c00744e0 0x7fd1c00769f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc033000 tx=0x7fd1cc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:04.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:04.977+0000 7fd1d9600640 1 -- 172.21.15.7:0/2954415789 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1dc0712f0 con 0x7fd1dc15e2f0 2024-04-25T11:19:05.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.245+0000 7fd1e2a00640 1 -- 172.21.15.7:0/2954415789 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd1dc074020 con 0x7fd1dc15e2f0 2024-04-25T11:19:05.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.246+0000 7fd1d9600640 1 -- 172.21.15.7:0/2954415789 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7fd1d0020020 con 0x7fd1dc15e2f0 2024-04-25T11:19:05.246 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:05.246 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:19:05.246 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-04-25T11:19:05.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.246 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-04-25T11:19:05.246 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w==", 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw==", 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:19:05.247 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw==", 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A==", 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA==", 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:05.249 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg==", 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw==", 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ==", 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:05.251 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw==", 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.253 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-04-25T11:19:05.253 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng==", 2024-04-25T11:19:05.253 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.253 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:05.253 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:05.253 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.253 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ==", 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A==", 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:05.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg==", 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA==", 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg==", 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:05.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ==", 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDZOSpmS8ObHxAAkXXXg1OI1YFC2/OZER3woA==", 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw==", 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.258 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w==", 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-25T11:19:05.259 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:19:05.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.249+0000 7fd1e2a00640 1 -- 172.21.15.7:0/2954415789 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1c00744e0 msgr2=0x7fd1c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.249+0000 7fd1e2a00640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1c00744e0 0x7fd1c00769f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc033000 tx=0x7fd1cc02e000 comp rx=0 tx=0).stop 2024-04-25T11:19:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.249+0000 7fd1e2a00640 1 -- 172.21.15.7:0/2954415789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc15e2f0 msgr2=0x7fd1dc141bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.249+0000 7fd1e2a00640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc15e2f0 0x7fd1dc141bb0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000b420 tx=0x7fd1d000b8e0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.249+0000 7fd1e2a00640 1 -- 172.21.15.7:0/2954415789 shutdown_connections 2024-04-25T11:19:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.249+0000 7fd1e2a00640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1c00744e0 0x7fd1c00769f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.249+0000 7fd1e2a00640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1dc135560 0x7fd1dc134070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.249+0000 7fd1e2a00640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1dc162420 0x7fd1dc138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.249+0000 7fd1e2a00640 1 --2- 172.21.15.7:0/2954415789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1dc15e2f0 0x7fd1dc141bb0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.249+0000 7fd1e2a00640 1 -- 172.21.15.7:0/2954415789 >> 172.21.15.7:0/2954415789 conn(0x7fd1dc073090 msgr2=0x7fd1dc0cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.250+0000 7fd1e2a00640 1 -- 172.21.15.7:0/2954415789 shutdown_connections 2024-04-25T11:19:05.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.250+0000 7fd1e2a00640 1 -- 172.21.15.7:0/2954415789 wait complete. 2024-04-25T11:19:05.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:05.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.557+0000 7f82e3e00640 1 Processor -- start 2024-04-25T11:19:05.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.557+0000 7f82e3e00640 1 -- start start 2024-04-25T11:19:05.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.559+0000 7f82e3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f82e40b9010 0x7f82e4161340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.560+0000 7f82e3e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e4160740 con 0x7f82e40b2ae0 2024-04-25T11:19:05.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.560+0000 7f82e3e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e406dc20 con 0x7f82e409c000 2024-04-25T11:19:05.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.560+0000 7f82e3e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f82e4161880 con 0x7f82e40b9010 2024-04-25T11:19:05.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.560+0000 7f82e2000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f82e40b2ae0 0x7f82e409b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35874/0 (socket says 172.21.15.7:35874) 2024-04-25T11:19:05.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.560+0000 7f82e2000640 1 -- 172.21.15.7:0/3297717650 learned_addr learned my addr 172.21.15.7:0/3297717650 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:05.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.560+0000 7f82e2a00640 1 --2- 172.21.15.7:0/3297717650 >> v2:172.21.15.87:3300/0 conn(0x7f82e40b9010 0x7f82e4161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.560+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546006909 0 0) 0x7f82e4160740 con 0x7f82e40b2ae0 2024-04-25T11:19:05.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.560+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82c4003520 con 0x7f82e40b2ae0 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3250767546 0 0) 0x7f82c4003520 con 0x7f82e40b2ae0 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 >> v2:172.21.15.87:3300/0 conn(0x7f82e40b9010 msgr2=0x7f82e4161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 --2- 172.21.15.7:0/3297717650 >> v2:172.21.15.87:3300/0 conn(0x7f82e40b9010 0x7f82e4161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 >> v1:172.21.15.87:6789/0 conn(0x7f82e409c000 legacy=0x7f82e40d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e406df00 con 0x7f82e40b2ae0 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f82cc0019f0 con 0x7f82e40b2ae0 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 >> v1:172.21.15.7:6789/0 conn(0x7f82e40b2ae0 legacy=0x7f82e409b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 --2- 172.21.15.7:0/3297717650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c4003a00 0x7f82c4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 --2- 172.21.15.7:0/3297717650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4004800 0x7f82c4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 --2- 172.21.15.7:0/3297717650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82c40052a0 0x7f82c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82e4161880 con 0x7f82c4004800 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82c400c230 con 0x7f82c4003a00 2024-04-25T11:19:05.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82c400c430 con 0x7f82c40052a0 2024-04-25T11:19:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.561+0000 7f82e2000640 1 --2- 172.21.15.7:0/3297717650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4004800 0x7f82c4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.562+0000 7f82e2a00640 1 --2- 172.21.15.7:0/3297717650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c4003a00 0x7f82c4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.562+0000 7f82e3400640 1 --2- 172.21.15.7:0/3297717650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82c40052a0 0x7f82c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.562+0000 7f82e2000640 1 -- 172.21.15.7:0/3297717650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82c40052a0 msgr2=0x7f82c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.562+0000 7f82e2000640 1 --2- 172.21.15.7:0/3297717650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82c40052a0 0x7f82c400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.562+0000 7f82e2000640 1 -- 172.21.15.7:0/3297717650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c4003a00 msgr2=0x7f82c4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.562+0000 7f82e2000640 1 --2- 172.21.15.7:0/3297717650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c4003a00 0x7f82c4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.562+0000 7f82e2000640 1 -- 172.21.15.7:0/3297717650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c4003520 con 0x7f82c4004800 2024-04-25T11:19:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.562+0000 7f82e3400640 1 -- 172.21.15.7:0/3297717650 reap_dead start 2024-04-25T11:19:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.563+0000 7f82e2000640 1 --2- 172.21.15.7:0/3297717650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4004800 0x7f82c4004c10 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00a780 tx=0x7f82c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc94c3b2c3284b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.563+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc00a3d0 con 0x7f82c4004800 2024-04-25T11:19:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.563+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82cc02f070 con 0x7f82c4004800 2024-04-25T11:19:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.563+0000 7f82e1600640 1 -- 172.21.15.7:0/3297717650 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc02bc60 con 0x7f82c4004800 2024-04-25T11:19:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.564+0000 7f82e3e00640 1 -- 172.21.15.7:0/3297717650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4004800 msgr2=0x7f82c4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.564+0000 7f82e3e00640 1 --2- 172.21.15.7:0/3297717650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4004800 0x7f82c4004c10 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00a780 tx=0x7f82c4005830 comp rx=0 tx=0).stop 2024-04-25T11:19:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.564+0000 7f82e3e00640 1 -- 172.21.15.7:0/3297717650 shutdown_connections 2024-04-25T11:19:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.564+0000 7f82e3e00640 1 --2- 172.21.15.7:0/3297717650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82c4004800 0x7f82c4004c10 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.564+0000 7f82e3e00640 1 -- 172.21.15.7:0/3297717650 >> 172.21.15.7:0/3297717650 conn(0x7f82e40b5210 msgr2=0x7f82e40b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.564+0000 7f82e3e00640 1 -- 172.21.15.7:0/3297717650 shutdown_connections 2024-04-25T11:19:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.564+0000 7f82e3e00640 1 -- 172.21.15.7:0/3297717650 wait complete. 2024-04-25T11:19:05.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.567+0000 7f82e3e00640 1 Processor -- start 2024-04-25T11:19:05.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.567+0000 7f82e3e00640 1 -- start start 2024-04-25T11:19:05.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.568+0000 7f82e3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c4004800 0x7f82e4154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.568+0000 7f82e3e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82e4154bd0 0x7f82e40b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.568+0000 7f82e3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e40b7c90 0x7f82e40b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.569+0000 7f82e3e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82e406e7a0 con 0x7f82e40b7c90 2024-04-25T11:19:05.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.569+0000 7f82e3e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82e406e4c0 con 0x7f82c4004800 2024-04-25T11:19:05.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.569+0000 7f82e3e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82e406ea80 con 0x7f82e4154bd0 2024-04-25T11:19:05.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.569+0000 7f82e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e40b7c90 0x7f82e40b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.569+0000 7f82e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e40b7c90 0x7f82e40b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55584/0 (socket says 172.21.15.7:55584) 2024-04-25T11:19:05.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.569+0000 7f82e3400640 1 -- 172.21.15.7:0/268090295 learned_addr learned my addr 172.21.15.7:0/268090295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:05.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.569+0000 7f82e2a00640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c4004800 0x7f82e4154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.569+0000 7f82e2000640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82e4154bd0 0x7f82e40b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82e3400640 1 -- 172.21.15.7:0/268090295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82e4154bd0 msgr2=0x7f82e40b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82e3400640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82e4154bd0 0x7f82e40b7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82e3400640 1 -- 172.21.15.7:0/268090295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c4004800 msgr2=0x7f82e4154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82e3400640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c4004800 0x7f82e4154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82e3400640 1 -- 172.21.15.7:0/268090295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e40b2590 con 0x7f82e40b7c90 2024-04-25T11:19:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82e2a00640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c4004800 0x7f82e4154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82e3400640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e40b7c90 0x7f82e40b0250 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f82e40d13d0 tx=0x7f82dc00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82daa00640 1 -- 172.21.15.7:0/268090295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc018070 con 0x7f82e40b7c90 2024-04-25T11:19:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82daa00640 1 -- 172.21.15.7:0/268090295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82dc0035f0 con 0x7f82e40b7c90 2024-04-25T11:19:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82e3e00640 1 -- 172.21.15.7:0/268090295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e40b80a0 con 0x7f82e40b7c90 2024-04-25T11:19:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82daa00640 1 -- 172.21.15.7:0/268090295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0039e0 con 0x7f82e40b7c90 2024-04-25T11:19:05.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.570+0000 7f82e3e00640 1 -- 172.21.15.7:0/268090295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e4157b50 con 0x7f82e40b7c90 2024-04-25T11:19:05.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.571+0000 7f82e3e00640 1 -- 172.21.15.7:0/268090295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e41575e0 con 0x7f82e40b7c90 2024-04-25T11:19:05.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.573+0000 7f82daa00640 1 -- 172.21.15.7:0/268090295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f82dc01b070 con 0x7f82e40b7c90 2024-04-25T11:19:05.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.574+0000 7f82daa00640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82c00744e0 0x7f82c00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:05.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.574+0000 7f82daa00640 1 -- 172.21.15.7:0/268090295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7f82dc094710 con 0x7f82e40b7c90 2024-04-25T11:19:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.574+0000 7f82e2a00640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82c00744e0 0x7f82c00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:05.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.575+0000 7f82e2a00640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82c00744e0 0x7f82c00769f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f82e406c310 tx=0x7f82d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:05.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.578+0000 7f82daa00640 1 -- 172.21.15.7:0/268090295 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82dc05efc0 con 0x7f82e40b7c90 2024-04-25T11:19:05.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.783+0000 7f82e3e00640 1 -- 172.21.15.7:0/268090295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82e4050740 con 0x7f82e40b7c90 2024-04-25T11:19:05.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.786+0000 7f82daa00640 1 -- 172.21.15.7:0/268090295 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+13070 (secure 0 0 0) 0x7f82dc05e8c0 con 0x7f82e40b7c90 2024-04-25T11:19:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:05.786 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":102,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:04.628691+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 -- 172.21.15.7:0/268090295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82c00744e0 msgr2=0x7f82c00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82c00744e0 0x7f82c00769f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f82e406c310 tx=0x7f82d4008040 comp rx=0 tx=0).stop 2024-04-25T11:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 -- 172.21.15.7:0/268090295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e40b7c90 msgr2=0x7f82e40b0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e40b7c90 0x7f82e40b0250 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f82e40d13d0 tx=0x7f82dc00d890 comp rx=0 tx=0).stop 2024-04-25T11:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 -- 172.21.15.7:0/268090295 shutdown_connections 2024-04-25T11:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82c00744e0 0x7f82c00769f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82e40b7c90 0x7f82e40b0250 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82e4154bd0 0x7f82e40b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 --2- 172.21.15.7:0/268090295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82c4004800 0x7f82e4154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 -- 172.21.15.7:0/268090295 >> 172.21.15.7:0/268090295 conn(0x7f82e40b5210 msgr2=0x7f82e40b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:05.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 -- 172.21.15.7:0/268090295 shutdown_connections 2024-04-25T11:19:05.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:05.789+0000 7f82e3e00640 1 -- 172.21.15.7:0/268090295 wait complete. 2024-04-25T11:19:05.819 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:05.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:06.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.106+0000 7f7a17800640 1 Processor -- start 2024-04-25T11:19:06.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.106+0000 7f7a17800640 1 -- start start 2024-04-25T11:19:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.108+0000 7f7a17800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7a100aef90 0x7f7a1015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.108+0000 7f7a17800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100b6920 0x7f7a100b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.108+0000 7f7a17800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a10164e00 con 0x7f7a100b6240 2024-04-25T11:19:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.108+0000 7f7a17800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7a101650e0 con 0x7f7a100aef90 2024-04-25T11:19:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.108+0000 7f7a17800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7a101653c0 con 0x7f7a100b6920 2024-04-25T11:19:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.109+0000 7f7a0fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7a100aef90 0x7f7a1015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.109+0000 7f7a0fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7a100aef90 0x7f7a1015eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55588/0 (socket says 172.21.15.7:55588) 2024-04-25T11:19:06.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.109+0000 7f7a0fe00640 1 -- 172.21.15.7:0/374519246 learned_addr learned my addr 172.21.15.7:0/374519246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.109+0000 7f7a15800640 1 --2- 172.21.15.7:0/374519246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100b6920 0x7f7a100b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.109+0000 7f7a0fe00640 1 -- 172.21.15.7:0/374519246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100b6920 msgr2=0x7f7a100b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.109+0000 7f7a0fe00640 1 --2- 172.21.15.7:0/374519246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100b6920 0x7f7a100b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.109+0000 7f7a0fe00640 1 -- 172.21.15.7:0/374519246 >> v1:172.21.15.87:6789/0 conn(0x7f7a100b6240 legacy=0x7f7a100ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.109+0000 7f7a0fe00640 1 -- 172.21.15.7:0/374519246 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a101656a0 con 0x7f7a100aef90 2024-04-25T11:19:06.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.109+0000 7f7a15800640 1 --2- 172.21.15.7:0/374519246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100b6920 0x7f7a100b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:06.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.111+0000 7f7a0fe00640 1 --2- 172.21.15.7:0/374519246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a100aef90 0x7f7a1015eb00 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009870 tx=0x7f79fc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72871e753fa6ac30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:06.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.111+0000 7f7a0f400640 1 -- 172.21.15.7:0/374519246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc037070 con 0x7f7a100aef90 2024-04-25T11:19:06.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.111+0000 7f7a0f400640 1 -- 172.21.15.7:0/374519246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79fc032440 con 0x7f7a100aef90 2024-04-25T11:19:06.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.111+0000 7f7a0f400640 1 -- 172.21.15.7:0/374519246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79fc032810 con 0x7f7a100aef90 2024-04-25T11:19:06.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.112+0000 7f7a17800640 1 -- 172.21.15.7:0/374519246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a100aef90 msgr2=0x7f7a1015eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.112+0000 7f7a17800640 1 --2- 172.21.15.7:0/374519246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a100aef90 0x7f7a1015eb00 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009870 tx=0x7f79fc02a210 comp rx=0 tx=0).stop 2024-04-25T11:19:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.112+0000 7f7a17800640 1 -- 172.21.15.7:0/374519246 shutdown_connections 2024-04-25T11:19:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.112+0000 7f7a17800640 1 --2- 172.21.15.7:0/374519246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100b6920 0x7f7a100b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.112+0000 7f7a17800640 1 --2- 172.21.15.7:0/374519246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a100aef90 0x7f7a1015eb00 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.112+0000 7f7a17800640 1 -- 172.21.15.7:0/374519246 >> 172.21.15.7:0/374519246 conn(0x7f7a100742c0 msgr2=0x7f7a1006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.112+0000 7f7a17800640 1 -- 172.21.15.7:0/374519246 shutdown_connections 2024-04-25T11:19:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.112+0000 7f7a17800640 1 -- 172.21.15.7:0/374519246 wait complete. 2024-04-25T11:19:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.115+0000 7f7a17800640 1 Processor -- start 2024-04-25T11:19:06.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.115+0000 7f7a17800640 1 -- start start 2024-04-25T11:19:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.116+0000 7f7a17800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100aef90 0x7f7a101fe1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.116+0000 7f7a17800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a100b6240 0x7f7a10159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.116+0000 7f7a17800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a100b6920 0x7f7a1015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.116+0000 7f7a17800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10164ba0 con 0x7f7a100b6920 2024-04-25T11:19:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.116+0000 7f7a17800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10164a20 con 0x7f7a100b6240 2024-04-25T11:19:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.116+0000 7f7a17800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7a100b5130 con 0x7f7a100aef90 2024-04-25T11:19:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.117+0000 7f7a15800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a100b6920 0x7f7a1015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.117+0000 7f7a15800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a100b6920 0x7f7a1015a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55594/0 (socket says 172.21.15.7:55594) 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.117+0000 7f7a15800640 1 -- 172.21.15.7:0/2690439791 learned_addr learned my addr 172.21.15.7:0/2690439791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.117+0000 7f7a0fe00640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a100b6240 0x7f7a10159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.117+0000 7f7a14e00640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100aef90 0x7f7a101fe1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.117+0000 7f7a15800640 1 -- 172.21.15.7:0/2690439791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100aef90 msgr2=0x7f7a101fe1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.117+0000 7f7a15800640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100aef90 0x7f7a101fe1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.117+0000 7f7a15800640 1 -- 172.21.15.7:0/2690439791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a100b6240 msgr2=0x7f7a10159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.117+0000 7f7a15800640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a100b6240 0x7f7a10159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.117+0000 7f7a15800640 1 -- 172.21.15.7:0/2690439791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a10147390 con 0x7f7a100b6920 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.117+0000 7f7a14e00640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100aef90 0x7f7a101fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.118+0000 7f7a15800640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a100b6920 0x7f7a1015a1b0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800c5f0 tx=0x7f7a0800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.118+0000 7f7a0cc00640 1 -- 172.21.15.7:0/2690439791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08015070 con 0x7f7a100b6920 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.118+0000 7f7a0cc00640 1 -- 172.21.15.7:0/2690439791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a08011820 con 0x7f7a100b6920 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.118+0000 7f7a0cc00640 1 -- 172.21.15.7:0/2690439791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a08011c10 con 0x7f7a100b6920 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.118+0000 7f7a17800640 1 -- 172.21.15.7:0/2690439791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a10147630 con 0x7f7a100b6920 2024-04-25T11:19:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.118+0000 7f7a17800640 1 -- 172.21.15.7:0/2690439791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a10147bf0 con 0x7f7a100b6920 2024-04-25T11:19:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.119+0000 7f7a17800640 1 -- 172.21.15.7:0/2690439791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a10050740 con 0x7f7a100b6920 2024-04-25T11:19:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.121+0000 7f7a0cc00640 1 -- 172.21.15.7:0/2690439791 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7a08018070 con 0x7f7a100b6920 2024-04-25T11:19:06.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.122+0000 7f7a0cc00640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f79f00744b0 0x7f79f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.122+0000 7f7a0cc00640 1 -- 172.21.15.7:0/2690439791 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7f7a08094fa0 con 0x7f7a100b6920 2024-04-25T11:19:06.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.122+0000 7f7a14e00640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f79f00744b0 0x7f79f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.123+0000 7f7a14e00640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f79f00744b0 0x7f79f00769c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7a10071850 tx=0x7f7a000073a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.127+0000 7f7a0cc00640 1 -- 172.21.15.7:0/2690439791 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7a0805f860 con 0x7f7a100b6920 2024-04-25T11:19:06.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.335+0000 7f7a17800640 1 -- 172.21.15.7:0/2690439791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a10164360 con 0x7f7a100b6920 2024-04-25T11:19:06.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.336+0000 7f7a0cc00640 1 -- 172.21.15.7:0/2690439791 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7f7a0805f160 con 0x7f7a100b6920 2024-04-25T11:19:06.335 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:19:06.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 -- 172.21.15.7:0/2690439791 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f79f00744b0 msgr2=0x7f79f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f79f00744b0 0x7f79f00769c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7a10071850 tx=0x7f7a000073a0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 -- 172.21.15.7:0/2690439791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a100b6920 msgr2=0x7f7a1015a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a100b6920 0x7f7a1015a1b0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800c5f0 tx=0x7f7a0800cab0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 -- 172.21.15.7:0/2690439791 shutdown_connections 2024-04-25T11:19:06.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f79f00744b0 0x7f79f00769c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a100b6920 0x7f7a1015a1b0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a100b6240 0x7f7a10159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 --2- 172.21.15.7:0/2690439791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a100aef90 0x7f7a101fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 -- 172.21.15.7:0/2690439791 >> 172.21.15.7:0/2690439791 conn(0x7f7a100742c0 msgr2=0x7f7a100b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:06.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 -- 172.21.15.7:0/2690439791 shutdown_connections 2024-04-25T11:19:06.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.339+0000 7f7a17800640 1 -- 172.21.15.7:0/2690439791 wait complete. 2024-04-25T11:19:06.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:19:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.643+0000 7f1bd5000640 1 Processor -- start 2024-04-25T11:19:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.643+0000 7f1bd5000640 1 -- start start 2024-04-25T11:19:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.646+0000 7f1bd5000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bd006dc90 0x7f1bd006f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.646+0000 7f1bd5000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd00af550 con 0x7f1bd00b05f0 2024-04-25T11:19:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.646+0000 7f1bd5000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bd00af830 con 0x7f1bd0160060 2024-04-25T11:19:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.646+0000 7f1bd5000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd00afb10 con 0x7f1bd006dc90 2024-04-25T11:19:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.646+0000 7f1bcea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1bd00b05f0 0x7f1bd006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35880/0 (socket says 172.21.15.7:35880) 2024-04-25T11:19:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.646+0000 7f1bcea00640 1 -- 172.21.15.7:0/2279349874 learned_addr learned my addr 172.21.15.7:0/2279349874 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.646+0000 7f1bcfe00640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bd006dc90 0x7f1bd006f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.646+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1272404848 0 0) 0x7f1bd00af550 con 0x7f1bd00b05f0 2024-04-25T11:19:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.646+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bb4003520 con 0x7f1bd00b05f0 2024-04-25T11:19:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 360365675 0 0) 0x7f1bb4003520 con 0x7f1bd00b05f0 2024-04-25T11:19:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bd006dc90 msgr2=0x7f1bd006f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bd006dc90 0x7f1bd006f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 >> v1:172.21.15.87:6789/0 conn(0x7f1bd0160060 legacy=0x7f1bd0074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd00afdf0 con 0x7f1bd00b05f0 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1bb8001670 con 0x7f1bd00b05f0 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bcfe00640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bd006dc90 0x7f1bd006f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 >> v1:172.21.15.7:6789/0 conn(0x7f1bd00b05f0 legacy=0x7f1bd006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb40039e0 0x7f1bb4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bb4004800 0x7f1bb4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb40051a0 0x7f1bb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd00afb10 con 0x7f1bb40039e0 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb400bfd0 con 0x7f1bb40051a0 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1bb400c1a0 con 0x7f1bb4004800 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.647+0000 7f1bcf400640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb40039e0 0x7f1bb4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.648+0000 7f1bcea00640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bb4004800 0x7f1bb4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.648+0000 7f1bcf400640 1 -- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bb4004800 msgr2=0x7f1bb4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.648+0000 7f1bcfe00640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb40051a0 0x7f1bb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.648+0000 7f1bcf400640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bb4004800 0x7f1bb4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.648+0000 7f1bcf400640 1 -- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb40051a0 msgr2=0x7f1bb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.648+0000 7f1bcf400640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bb40051a0 0x7f1bb400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.648+0000 7f1bcf400640 1 -- 172.21.15.7:0/2279349874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb4003520 con 0x7f1bb40039e0 2024-04-25T11:19:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.648+0000 7f1bcea00640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bb4004800 0x7f1bb4004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.649+0000 7f1bcf400640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb40039e0 0x7f1bb4003e50 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0001750 tx=0x7f1bb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e24d07eed81ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.649+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0001370 con 0x7f1bb40039e0 2024-04-25T11:19:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.649+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc00052d0 con 0x7f1bb40039e0 2024-04-25T11:19:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.649+0000 7f1bce000640 1 -- 172.21.15.7:0/2279349874 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc002abe0 con 0x7f1bb40039e0 2024-04-25T11:19:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.650+0000 7f1bd5000640 1 -- 172.21.15.7:0/2279349874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb40039e0 msgr2=0x7f1bb4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.650+0000 7f1bd5000640 1 --2- 172.21.15.7:0/2279349874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bb40039e0 0x7f1bb4003e50 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0001750 tx=0x7f1bb4005830 comp rx=0 tx=0).stop 2024-04-25T11:19:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.650+0000 7f1bcfe00640 1 -- 172.21.15.7:0/2279349874 reap_dead start 2024-04-25T11:19:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.650+0000 7f1bd5000640 1 -- 172.21.15.7:0/2279349874 shutdown_connections 2024-04-25T11:19:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.650+0000 7f1bd5000640 1 -- 172.21.15.7:0/2279349874 >> 172.21.15.7:0/2279349874 conn(0x7f1bd0073540 msgr2=0x7f1bd0073950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.650+0000 7f1bd5000640 1 -- 172.21.15.7:0/2279349874 shutdown_connections 2024-04-25T11:19:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.650+0000 7f1bd5000640 1 -- 172.21.15.7:0/2279349874 wait complete. 2024-04-25T11:19:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.652+0000 7f1bd5000640 1 Processor -- start 2024-04-25T11:19:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.653+0000 7f1bd5000640 1 -- start start 2024-04-25T11:19:06.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.654+0000 7f1bd5000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bb40039e0 0x7f1bd0155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.654+0000 7f1bd5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd015e180 0x7f1bd015e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.654+0000 7f1bd5000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bd01f5e00 0x7f1bd01f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.654+0000 7f1bd5000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd00b0a00 con 0x7f1bd015e180 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.654+0000 7f1bd5000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd0074770 con 0x7f1bd01f5e00 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.654+0000 7f1bd5000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd00af100 con 0x7f1bb40039e0 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.654+0000 7f1bcea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd015e180 0x7f1bd015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.654+0000 7f1bcea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd015e180 0x7f1bd015e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55612/0 (socket says 172.21.15.7:55612) 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.654+0000 7f1bcea00640 1 -- 172.21.15.7:0/3446493418 learned_addr learned my addr 172.21.15.7:0/3446493418 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.655+0000 7f1bcf400640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bb40039e0 0x7f1bd0155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.655+0000 7f1bcfe00640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bd01f5e00 0x7f1bd01f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.655+0000 7f1bcea00640 1 -- 172.21.15.7:0/3446493418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bb40039e0 msgr2=0x7f1bd0155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.655+0000 7f1bcea00640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bb40039e0 0x7f1bd0155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.655+0000 7f1bcea00640 1 -- 172.21.15.7:0/3446493418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bd01f5e00 msgr2=0x7f1bd01f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.655+0000 7f1bcea00640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bd01f5e00 0x7f1bd01f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.655+0000 7f1bcea00640 1 -- 172.21.15.7:0/3446493418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd015ef20 con 0x7f1bd015e180 2024-04-25T11:19:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.655+0000 7f1bcea00640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd015e180 0x7f1bd015e590 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800d770 tx=0x7f1bb800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.655+0000 7f1bc7400640 1 -- 172.21.15.7:0/3446493418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8019070 con 0x7f1bd015e180 2024-04-25T11:19:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.656+0000 7f1bc7400640 1 -- 172.21.15.7:0/3446493418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb80074e0 con 0x7f1bd015e180 2024-04-25T11:19:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.656+0000 7f1bc7400640 1 -- 172.21.15.7:0/3446493418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb80049e0 con 0x7f1bd015e180 2024-04-25T11:19:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.656+0000 7f1bd5000640 1 -- 172.21.15.7:0/3446493418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd01ea6b0 con 0x7f1bd015e180 2024-04-25T11:19:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.656+0000 7f1bd5000640 1 -- 172.21.15.7:0/3446493418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd01eac70 con 0x7f1bd015e180 2024-04-25T11:19:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.657+0000 7f1bd5000640 1 -- 172.21.15.7:0/3446493418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bd015ec90 con 0x7f1bd015e180 2024-04-25T11:19:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.658+0000 7f1bc7400640 1 -- 172.21.15.7:0/3446493418 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1bb8004b80 con 0x7f1bd015e180 2024-04-25T11:19:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.659+0000 7f1bc7400640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bac0744b0 0x7f1bac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.659+0000 7f1bc7400640 1 -- 172.21.15.7:0/3446493418 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7f1bb8094a80 con 0x7f1bd015e180 2024-04-25T11:19:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.659+0000 7f1bcf400640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bac0744b0 0x7f1bac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.660+0000 7f1bcf400640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bac0744b0 0x7f1bac0769c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0032000 tx=0x7f1bc002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:06.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.664+0000 7f1bc7400640 1 -- 172.21.15.7:0/3446493418 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1bb805f3d0 con 0x7f1bd015e180 2024-04-25T11:19:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.872+0000 7f1bd5000640 1 -- 172.21.15.7:0/3446493418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1bd0050740 con 0x7f1bd015e180 2024-04-25T11:19:06.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.873+0000 7f1bc7400640 1 -- 172.21.15.7:0/3446493418 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v102) ==== 84+0+65 (secure 0 0 0) 0x7f1bb805ecd0 con 0x7f1bd015e180 2024-04-25T11:19:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:06.873 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-25T11:19:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-25T11:19:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-25T11:19:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:06.873 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-25T11:19:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 -- 172.21.15.7:0/3446493418 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bac0744b0 msgr2=0x7f1bac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bac0744b0 0x7f1bac0769c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0032000 tx=0x7f1bc002e000 comp rx=0 tx=0).stop 2024-04-25T11:19:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 -- 172.21.15.7:0/3446493418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd015e180 msgr2=0x7f1bd015e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd015e180 0x7f1bd015e590 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f1bb800d770 tx=0x7f1bb800dc30 comp rx=0 tx=0).stop 2024-04-25T11:19:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 -- 172.21.15.7:0/3446493418 shutdown_connections 2024-04-25T11:19:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bac0744b0 0x7f1bac0769c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bd01f5e00 0x7f1bd01f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bd015e180 0x7f1bd015e590 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 --2- 172.21.15.7:0/3446493418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1bb40039e0 0x7f1bd0155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 -- 172.21.15.7:0/3446493418 >> 172.21.15.7:0/3446493418 conn(0x7f1bd0073540 msgr2=0x7f1bd006f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 -- 172.21.15.7:0/3446493418 shutdown_connections 2024-04-25T11:19:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:06.876+0000 7f1bd5000640 1 -- 172.21.15.7:0/3446493418 wait complete. 2024-04-25T11:19:06.902 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:19:06.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:19:07.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.192+0000 7f9c8f200640 1 Processor -- start 2024-04-25T11:19:07.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.193+0000 7f9c8f200640 1 -- start start 2024-04-25T11:19:07.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.195+0000 7f9c8f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c900b52f0 0x7f9c90160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.195+0000 7f9c8f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9c9006d580 0x7f9c9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.195+0000 7f9c8f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c9006ded0 con 0x7f9c900b5a40 2024-04-25T11:19:07.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.195+0000 7f9c8f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9c9006eee0 con 0x7f9c900b52f0 2024-04-25T11:19:07.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.195+0000 7f9c8f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9c90070320 con 0x7f9c9006d580 2024-04-25T11:19:07.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.196+0000 7f9c8de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c900b52f0 0x7f9c90160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:07.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.196+0000 7f9c87e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9c900b5a40 0x7f9c9006fc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35894/0 (socket says 172.21.15.7:35894) 2024-04-25T11:19:07.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.196+0000 7f9c87e00640 1 -- 172.21.15.7:0/4233615558 learned_addr learned my addr 172.21.15.7:0/4233615558 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:07.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.196+0000 7f9c8de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c900b52f0 0x7f9c90160030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55622/0 (socket says 172.21.15.7:55622) 2024-04-25T11:19:07.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.196+0000 7f9c8de00640 1 -- 172.21.15.7:0/4233615558 >> v2:172.21.15.87:3300/0 conn(0x7f9c9006d580 msgr2=0x7f9c9006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:07.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.196+0000 7f9c8de00640 1 --2- 172.21.15.7:0/4233615558 >> v2:172.21.15.87:3300/0 conn(0x7f9c9006d580 0x7f9c9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.196+0000 7f9c8de00640 1 -- 172.21.15.7:0/4233615558 >> v1:172.21.15.7:6789/0 conn(0x7f9c900b5a40 legacy=0x7f9c9006fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:07.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.196+0000 7f9c8de00640 1 -- 172.21.15.7:0/4233615558 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c900b6270 con 0x7f9c900b52f0 2024-04-25T11:19:07.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.197+0000 7f9c8de00640 1 --2- 172.21.15.7:0/4233615558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c900b52f0 0x7f9c90160030 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f9c90075b90 tx=0x7f9c8002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c97bd0be9b17e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.197+0000 7f9c8d400640 1 -- 172.21.15.7:0/4233615558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80037070 con 0x7f9c900b52f0 2024-04-25T11:19:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.198+0000 7f9c8d400640 1 -- 172.21.15.7:0/4233615558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8002ad30 con 0x7f9c900b52f0 2024-04-25T11:19:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.198+0000 7f9c8d400640 1 -- 172.21.15.7:0/4233615558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80032660 con 0x7f9c900b52f0 2024-04-25T11:19:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.198+0000 7f9c8f200640 1 -- 172.21.15.7:0/4233615558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c900b52f0 msgr2=0x7f9c90160030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.198+0000 7f9c8f200640 1 --2- 172.21.15.7:0/4233615558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c900b52f0 0x7f9c90160030 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f9c90075b90 tx=0x7f9c8002a0a0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.198+0000 7f9c8f200640 1 -- 172.21.15.7:0/4233615558 shutdown_connections 2024-04-25T11:19:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.198+0000 7f9c8f200640 1 --2- 172.21.15.7:0/4233615558 >> v2:172.21.15.87:3300/0 conn(0x7f9c9006d580 0x7f9c9006d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.198+0000 7f9c8f200640 1 --2- 172.21.15.7:0/4233615558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c900b52f0 0x7f9c90160030 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.198+0000 7f9c8f200640 1 -- 172.21.15.7:0/4233615558 >> 172.21.15.7:0/4233615558 conn(0x7f9c90073160 msgr2=0x7f9c90073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.198+0000 7f9c8f200640 1 -- 172.21.15.7:0/4233615558 shutdown_connections 2024-04-25T11:19:07.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.198+0000 7f9c8f200640 1 -- 172.21.15.7:0/4233615558 wait complete. 2024-04-25T11:19:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.201+0000 7f9c8f200640 1 Processor -- start 2024-04-25T11:19:07.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.201+0000 7f9c8f200640 1 -- start start 2024-04-25T11:19:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.202+0000 7f9c8f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c9006d580 0x7f9c900cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.202+0000 7f9c8f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c900b52f0 0x7f9c900d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.202+0000 7f9c8f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c900b5a40 0x7f9c900c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.202+0000 7f9c8f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9006f6f0 con 0x7f9c9006d580 2024-04-25T11:19:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.202+0000 7f9c8f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9006f410 con 0x7f9c900b52f0 2024-04-25T11:19:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.202+0000 7f9c8f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9c900b5e50 con 0x7f9c900b5a40 2024-04-25T11:19:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.203+0000 7f9c8de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c9006d580 0x7f9c900cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.203+0000 7f9c8de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c9006d580 0x7f9c900cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55634/0 (socket says 172.21.15.7:55634) 2024-04-25T11:19:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.203+0000 7f9c8de00640 1 -- 172.21.15.7:0/1567822176 learned_addr learned my addr 172.21.15.7:0/1567822176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.203+0000 7f9c8de00640 1 -- 172.21.15.7:0/1567822176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c900b5a40 msgr2=0x7f9c900c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.203+0000 7f9c8de00640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c900b5a40 0x7f9c900c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.203+0000 7f9c8de00640 1 -- 172.21.15.7:0/1567822176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c900b52f0 msgr2=0x7f9c900d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.203+0000 7f9c8de00640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c900b52f0 0x7f9c900d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.203+0000 7f9c8de00640 1 -- 172.21.15.7:0/1567822176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9015e2e0 con 0x7f9c9006d580 2024-04-25T11:19:07.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.203+0000 7f9c8de00640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c9006d580 0x7f9c900cc3f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f9c8002a0e0 tx=0x7f9c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.204+0000 7f9c86000640 1 -- 172.21.15.7:0/1567822176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80037040 con 0x7f9c9006d580 2024-04-25T11:19:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.204+0000 7f9c86000640 1 -- 172.21.15.7:0/1567822176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c80032e10 con 0x7f9c9006d580 2024-04-25T11:19:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.204+0000 7f9c86000640 1 -- 172.21.15.7:0/1567822176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80004250 con 0x7f9c9006d580 2024-04-25T11:19:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.204+0000 7f9c8f200640 1 -- 172.21.15.7:0/1567822176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c900cb630 con 0x7f9c9006d580 2024-04-25T11:19:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.204+0000 7f9c8f200640 1 -- 172.21.15.7:0/1567822176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c900cbbf0 con 0x7f9c9006d580 2024-04-25T11:19:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.205+0000 7f9c8f200640 1 -- 172.21.15.7:0/1567822176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c90050740 con 0x7f9c9006d580 2024-04-25T11:19:07.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.206+0000 7f9c86000640 1 -- 172.21.15.7:0/1567822176 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9c8003a070 con 0x7f9c9006d580 2024-04-25T11:19:07.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.208+0000 7f9c86000640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c6c0744e0 0x7f9c6c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.208+0000 7f9c86000640 1 -- 172.21.15.7:0/1567822176 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5249+0+0 (secure 0 0 0) 0x7f9c800b5350 con 0x7f9c9006d580 2024-04-25T11:19:07.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.208+0000 7f9c87e00640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c6c0744e0 0x7f9c6c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:07.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.209+0000 7f9c87e00640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c6c0744e0 0x7f9c6c0769f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9c78007fd0 tx=0x7f9c780079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.213+0000 7f9c86000640 1 -- 172.21.15.7:0/1567822176 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9c9015e2e0 con 0x7f9c9006d580 2024-04-25T11:19:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.412+0000 7f9c8f200640 1 -- 172.21.15.7:0/1567822176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9c90163190 con 0x7f9c9006d580 2024-04-25T11:19:07.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.648+0000 7f9c86000640 1 -- 172.21.15.7:0/1567822176 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v103) ==== 138+0+0 (secure 0 0 0) 0x7f9c80041020 con 0x7f9c9006d580 2024-04-25T11:19:07.648 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-25T11:19:07.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.652+0000 7f9c8f200640 1 -- 172.21.15.7:0/1567822176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c6c0744e0 msgr2=0x7f9c6c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:07.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.652+0000 7f9c8f200640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c6c0744e0 0x7f9c6c0769f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9c78007fd0 tx=0x7f9c780079a0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.652+0000 7f9c8f200640 1 -- 172.21.15.7:0/1567822176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c9006d580 msgr2=0x7f9c900cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:07.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.652+0000 7f9c8f200640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c9006d580 0x7f9c900cc3f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f9c8002a0e0 tx=0x7f9c800046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.653+0000 7f9c8f200640 1 -- 172.21.15.7:0/1567822176 shutdown_connections 2024-04-25T11:19:07.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.653+0000 7f9c8f200640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c6c0744e0 0x7f9c6c0769f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.653+0000 7f9c8f200640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9c900b5a40 0x7f9c900c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.653+0000 7f9c8f200640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9c900b52f0 0x7f9c900d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.653+0000 7f9c8f200640 1 --2- 172.21.15.7:0/1567822176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c9006d580 0x7f9c900cc3f0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.653+0000 7f9c8f200640 1 -- 172.21.15.7:0/1567822176 >> 172.21.15.7:0/1567822176 conn(0x7f9c90073160 msgr2=0x7f9c900b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:07.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.653+0000 7f9c8f200640 1 -- 172.21.15.7:0/1567822176 shutdown_connections 2024-04-25T11:19:07.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.653+0000 7f9c8f200640 1 -- 172.21.15.7:0/1567822176 wait complete. 2024-04-25T11:19:07.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:19:07.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.960+0000 7f7e61000640 1 Processor -- start 2024-04-25T11:19:07.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.960+0000 7f7e61000640 1 -- start start 2024-04-25T11:19:07.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.963+0000 7f7e61000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c0b6370 0x7f7e5c0b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.963+0000 7f7e61000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5c071170 con 0x7f7e5c06d580 2024-04-25T11:19:07.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.963+0000 7f7e61000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5c0b0630 con 0x7f7e5c153e90 2024-04-25T11:19:07.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.963+0000 7f7e61000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c0705d0 con 0x7f7e5c0b6370 2024-04-25T11:19:07.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.963+0000 7f7e5a800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7e5c06d580 0x7f7e5c06d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35904/0 (socket says 172.21.15.7:35904) 2024-04-25T11:19:07.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.963+0000 7f7e5a800640 1 -- 172.21.15.7:0/176617299 learned_addr learned my addr 172.21.15.7:0/176617299 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e59400640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c0b6370 0x7f7e5c0b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336349480 0 0) 0x7f7e5c071170 con 0x7f7e5c06d580 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e40005730 con 0x7f7e5c06d580 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2188656429 0 0) 0x7f7e40005730 con 0x7f7e5c06d580 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c0b6370 msgr2=0x7f7e5c0b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c0b6370 0x7f7e5c0b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 >> v1:172.21.15.87:6789/0 conn(0x7f7e5c153e90 legacy=0x7f7e5c070a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c0b08b0 con 0x7f7e5c06d580 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f7e540019f0 con 0x7f7e5c06d580 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 >> v1:172.21.15.7:6789/0 conn(0x7f7e5c06d580 legacy=0x7f7e5c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e40005bf0 0x7f7e40006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 0x7f7e40006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c0b6370 0x7f7e400074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c0705d0 con 0x7f7e40006a10 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4000dde0 con 0x7f7e40005bf0 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.964+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7e4000dfe0 con 0x7f7e5c0b6370 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.965+0000 7f7e59400640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 0x7f7e40006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.965+0000 7f7e59e00640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e40005bf0 0x7f7e40006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.965+0000 7f7e5a800640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c0b6370 0x7f7e400074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.965+0000 7f7e59400640 1 -- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c0b6370 msgr2=0x7f7e400074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.965+0000 7f7e59400640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c0b6370 0x7f7e400074e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.965+0000 7f7e59400640 1 -- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e40005bf0 msgr2=0x7f7e40006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.965+0000 7f7e59400640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e40005bf0 0x7f7e40006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.965+0000 7f7e59400640 1 -- 172.21.15.7:0/176617299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e40005730 con 0x7f7e40006a10 2024-04-25T11:19:07.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.965+0000 7f7e5a800640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c0b6370 0x7f7e400074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:07.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.966+0000 7f7e59400640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 0x7f7e40006e20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800ba30 tx=0x7f7e40007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93a59d1a6d7dc04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.967+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48030070 con 0x7f7e40006a10 2024-04-25T11:19:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.967+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4802ae00 con 0x7f7e40006a10 2024-04-25T11:19:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.967+0000 7f7e58a00640 1 -- 172.21.15.7:0/176617299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4800b600 con 0x7f7e40006a10 2024-04-25T11:19:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.968+0000 7f7e61000640 1 -- 172.21.15.7:0/176617299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 msgr2=0x7f7e40006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.968+0000 7f7e61000640 1 --2- 172.21.15.7:0/176617299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 0x7f7e40006e20 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800ba30 tx=0x7f7e40007a40 comp rx=0 tx=0).stop 2024-04-25T11:19:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.968+0000 7f7e5a800640 1 -- 172.21.15.7:0/176617299 reap_dead start 2024-04-25T11:19:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.968+0000 7f7e61000640 1 -- 172.21.15.7:0/176617299 shutdown_connections 2024-04-25T11:19:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.968+0000 7f7e61000640 1 -- 172.21.15.7:0/176617299 >> 172.21.15.7:0/176617299 conn(0x7f7e5c0d1d40 msgr2=0x7f7e5c073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.968+0000 7f7e61000640 1 -- 172.21.15.7:0/176617299 shutdown_connections 2024-04-25T11:19:07.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.968+0000 7f7e61000640 1 -- 172.21.15.7:0/176617299 wait complete. 2024-04-25T11:19:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.970+0000 7f7e61000640 1 Processor -- start 2024-04-25T11:19:07.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.970+0000 7f7e61000640 1 -- start start 2024-04-25T11:19:07.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.972+0000 7f7e61000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 0x7f7e5c146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.972+0000 7f7e61000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e5c1544c0 0x7f7e5c156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.972+0000 7f7e61000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c147680 0x7f7e5c156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.972+0000 7f7e61000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c0b0e50 con 0x7f7e40006a10 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.972+0000 7f7e61000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c0b0b70 con 0x7f7e5c1544c0 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.972+0000 7f7e61000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c0b6780 con 0x7f7e5c147680 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.972+0000 7f7e59e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 0x7f7e5c146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.972+0000 7f7e59e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 0x7f7e5c146fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55652/0 (socket says 172.21.15.7:55652) 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.972+0000 7f7e59e00640 1 -- 172.21.15.7:0/3350199691 learned_addr learned my addr 172.21.15.7:0/3350199691 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.973+0000 7f7e5a800640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c147680 0x7f7e5c156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.973+0000 7f7e59400640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e5c1544c0 0x7f7e5c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.973+0000 7f7e59e00640 1 -- 172.21.15.7:0/3350199691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c147680 msgr2=0x7f7e5c156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.973+0000 7f7e59e00640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c147680 0x7f7e5c156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.973+0000 7f7e59e00640 1 -- 172.21.15.7:0/3350199691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e5c1544c0 msgr2=0x7f7e5c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.973+0000 7f7e59e00640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e5c1544c0 0x7f7e5c156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.973+0000 7f7e59e00640 1 -- 172.21.15.7:0/3350199691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c0d1010 con 0x7f7e40006a10 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.973+0000 7f7e59400640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e5c1544c0 0x7f7e5c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:07.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.973+0000 7f7e59e00640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 0x7f7e5c146fc0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00e4c0 tx=0x7f7e4c00e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:07.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.974+0000 7f7e52000640 1 -- 172.21.15.7:0/3350199691 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0052e0 con 0x7f7e40006a10 2024-04-25T11:19:07.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.974+0000 7f7e52000640 1 -- 172.21.15.7:0/3350199691 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c018070 con 0x7f7e40006a10 2024-04-25T11:19:07.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.974+0000 7f7e52000640 1 -- 172.21.15.7:0/3350199691 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c003830 con 0x7f7e40006a10 2024-04-25T11:19:07.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.974+0000 7f7e61000640 1 -- 172.21.15.7:0/3350199691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5c154a60 con 0x7f7e40006a10 2024-04-25T11:19:07.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.974+0000 7f7e61000640 1 -- 172.21.15.7:0/3350199691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5c0d0810 con 0x7f7e40006a10 2024-04-25T11:19:07.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.975+0000 7f7e61000640 1 -- 172.21.15.7:0/3350199691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5c050740 con 0x7f7e40006a10 2024-04-25T11:19:07.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.977+0000 7f7e52000640 1 -- 172.21.15.7:0/3350199691 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e4c0039d0 con 0x7f7e40006a10 2024-04-25T11:19:07.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.978+0000 7f7e52000640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e3c0744e0 0x7f7e3c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:07.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.978+0000 7f7e52000640 1 -- 172.21.15.7:0/3350199691 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5619+0+0 (secure 0 0 0) 0x7f7e4c094ec0 con 0x7f7e40006a10 2024-04-25T11:19:07.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.979+0000 7f7e59400640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e3c0744e0 0x7f7e3c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:07.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.979+0000 7f7e59400640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e3c0744e0 0x7f7e3c0769f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800ba00 tx=0x7f7e48030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:07.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:07.984+0000 7f7e52000640 1 -- 172.21.15.7:0/3350199691 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e5c0b0630 con 0x7f7e40006a10 2024-04-25T11:19:08.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.188+0000 7f7e61000640 1 -- 172.21.15.7:0/3350199691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7e5c15b240 con 0x7f7e40006a10 2024-04-25T11:19:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.669+0000 7f7e52000640 1 -- 172.21.15.7:0/3350199691 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v104) ==== 170+0+0 (secure 0 0 0) 0x7f7e4c05f750 con 0x7f7e40006a10 2024-04-25T11:19:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-25T11:19:08.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.673+0000 7f7e61000640 1 -- 172.21.15.7:0/3350199691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e3c0744e0 msgr2=0x7f7e3c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:08.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.673+0000 7f7e61000640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e3c0744e0 0x7f7e3c0769f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800ba00 tx=0x7f7e48030000 comp rx=0 tx=0).stop 2024-04-25T11:19:08.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.673+0000 7f7e61000640 1 -- 172.21.15.7:0/3350199691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 msgr2=0x7f7e5c146fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:08.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.673+0000 7f7e61000640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 0x7f7e5c146fc0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00e4c0 tx=0x7f7e4c00e980 comp rx=0 tx=0).stop 2024-04-25T11:19:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.674+0000 7f7e61000640 1 -- 172.21.15.7:0/3350199691 shutdown_connections 2024-04-25T11:19:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.674+0000 7f7e61000640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e3c0744e0 0x7f7e3c0769f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.674+0000 7f7e61000640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e5c147680 0x7f7e5c156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.674+0000 7f7e61000640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e5c1544c0 0x7f7e5c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.674+0000 7f7e61000640 1 --2- 172.21.15.7:0/3350199691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e40006a10 0x7f7e5c146fc0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.674+0000 7f7e61000640 1 -- 172.21.15.7:0/3350199691 >> 172.21.15.7:0/3350199691 conn(0x7f7e5c0d1d40 msgr2=0x7f7e5c073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.674+0000 7f7e61000640 1 -- 172.21.15.7:0/3350199691 shutdown_connections 2024-04-25T11:19:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.674+0000 7f7e61000640 1 -- 172.21.15.7:0/3350199691 wait complete. 2024-04-25T11:19:08.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:19:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.975+0000 7f4cf7e00640 1 Processor -- start 2024-04-25T11:19:08.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.976+0000 7f4cf7e00640 1 -- start start 2024-04-25T11:19:08.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.978+0000 7f4cf7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cf81583c0 0x7f4cf80cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.978+0000 7f4cf7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4cf80cfb50 0x7f4cf80d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.978+0000 7f4cf7e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf80ce570 con 0x7f4cf80d09c0 2024-04-25T11:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.978+0000 7f4cf7e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf815f190 con 0x7f4cf81583c0 2024-04-25T11:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.978+0000 7f4cf7e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf80ce850 con 0x7f4cf80cfb50 2024-04-25T11:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.978+0000 7f4cf6a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cf81583c0 0x7f4cf80cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.978+0000 7f4cf6a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cf81583c0 0x7f4cf80cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55662/0 (socket says 172.21.15.7:55662) 2024-04-25T11:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.978+0000 7f4cf6a00640 1 -- 172.21.15.7:0/1452339896 learned_addr learned my addr 172.21.15.7:0/1452339896 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:08.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.978+0000 7f4cf7400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4cf80d09c0 0x7f4cf806eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35912/0 (socket says 172.21.15.7:35912) 2024-04-25T11:19:08.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.979+0000 7f4cee000640 1 --2- 172.21.15.7:0/1452339896 >> v2:172.21.15.87:3300/0 conn(0x7f4cf80cfb50 0x7f4cf80d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:08.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.979+0000 7f4cf6a00640 1 -- 172.21.15.7:0/1452339896 >> v2:172.21.15.87:3300/0 conn(0x7f4cf80cfb50 msgr2=0x7f4cf80d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:08.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.979+0000 7f4cf6a00640 1 --2- 172.21.15.7:0/1452339896 >> v2:172.21.15.87:3300/0 conn(0x7f4cf80cfb50 0x7f4cf80d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:08.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.979+0000 7f4cf6a00640 1 -- 172.21.15.7:0/1452339896 >> v1:172.21.15.7:6789/0 conn(0x7f4cf80d09c0 legacy=0x7f4cf806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:08.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.979+0000 7f4cf6a00640 1 -- 172.21.15.7:0/1452339896 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf80ceb30 con 0x7f4cf81583c0 2024-04-25T11:19:08.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.979+0000 7f4cee000640 1 --2- 172.21.15.7:0/1452339896 >> v2:172.21.15.87:3300/0 conn(0x7f4cf80cfb50 0x7f4cf80d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.980+0000 7f4cf6a00640 1 --2- 172.21.15.7:0/1452339896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf81583c0 0x7f4cf80cf610 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8070ad0 tx=0x7f4ce802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc46c69124476f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.980+0000 7f4cf6000640 1 -- 172.21.15.7:0/1452339896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8037070 con 0x7f4cf81583c0 2024-04-25T11:19:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.980+0000 7f4cf6000640 1 -- 172.21.15.7:0/1452339896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce8032440 con 0x7f4cf81583c0 2024-04-25T11:19:08.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.980+0000 7f4cf6000640 1 -- 172.21.15.7:0/1452339896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8032810 con 0x7f4cf81583c0 2024-04-25T11:19:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.981+0000 7f4cf7e00640 1 -- 172.21.15.7:0/1452339896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf81583c0 msgr2=0x7f4cf80cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.981+0000 7f4cf7e00640 1 --2- 172.21.15.7:0/1452339896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf81583c0 0x7f4cf80cf610 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8070ad0 tx=0x7f4ce802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:19:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.981+0000 7f4cf7e00640 1 -- 172.21.15.7:0/1452339896 shutdown_connections 2024-04-25T11:19:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.981+0000 7f4cf7e00640 1 --2- 172.21.15.7:0/1452339896 >> v2:172.21.15.87:3300/0 conn(0x7f4cf80cfb50 0x7f4cf80d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.981+0000 7f4cf7e00640 1 --2- 172.21.15.7:0/1452339896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf81583c0 0x7f4cf80cf610 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.981+0000 7f4cf7e00640 1 -- 172.21.15.7:0/1452339896 >> 172.21.15.7:0/1452339896 conn(0x7f4cf80740f0 msgr2=0x7f4cf806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.981+0000 7f4cf7e00640 1 -- 172.21.15.7:0/1452339896 shutdown_connections 2024-04-25T11:19:08.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.981+0000 7f4cf7e00640 1 -- 172.21.15.7:0/1452339896 wait complete. 2024-04-25T11:19:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.984+0000 7f4cf7e00640 1 Processor -- start 2024-04-25T11:19:08.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.984+0000 7f4cf7e00640 1 -- start start 2024-04-25T11:19:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.985+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cf80cfb50 0x7f4cf814e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.985+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf80d09c0 0x7f4cf815cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.985+0000 7f4cf7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf81583c0 0x7f4cf815d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.985+0000 7f4cf7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf806f5b0 con 0x7f4cf80d09c0 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.985+0000 7f4cf7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf80cf0f0 con 0x7f4cf81583c0 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.985+0000 7f4cf7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf806f730 con 0x7f4cf80cfb50 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.986+0000 7f4cee000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf80d09c0 0x7f4cf815cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.986+0000 7f4cee000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf80d09c0 0x7f4cf815cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55664/0 (socket says 172.21.15.7:55664) 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.986+0000 7f4cee000640 1 -- 172.21.15.7:0/640327506 learned_addr learned my addr 172.21.15.7:0/640327506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.986+0000 7f4cf7400640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf81583c0 0x7f4cf815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.986+0000 7f4cf6a00640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cf80cfb50 0x7f4cf814e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.986+0000 7f4cee000640 1 -- 172.21.15.7:0/640327506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cf80cfb50 msgr2=0x7f4cf814e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.986+0000 7f4cee000640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cf80cfb50 0x7f4cf814e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.986+0000 7f4cee000640 1 -- 172.21.15.7:0/640327506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf81583c0 msgr2=0x7f4cf815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:08.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.986+0000 7f4cee000640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf81583c0 0x7f4cf815d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:08.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.986+0000 7f4cee000640 1 -- 172.21.15.7:0/640327506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf81504e0 con 0x7f4cf80d09c0 2024-04-25T11:19:08.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.987+0000 7f4cee000640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf80d09c0 0x7f4cf815cce0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c7f0 tx=0x7f4ce000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.987+0000 7f4cef400640 1 -- 172.21.15.7:0/640327506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0015070 con 0x7f4cf80d09c0 2024-04-25T11:19:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.987+0000 7f4cef400640 1 -- 172.21.15.7:0/640327506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce0007820 con 0x7f4cf80d09c0 2024-04-25T11:19:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.987+0000 7f4cef400640 1 -- 172.21.15.7:0/640327506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0007bf0 con 0x7f4cf80d09c0 2024-04-25T11:19:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.987+0000 7f4cf7e00640 1 -- 172.21.15.7:0/640327506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf81507c0 con 0x7f4cf80d09c0 2024-04-25T11:19:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.987+0000 7f4cf7e00640 1 -- 172.21.15.7:0/640327506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf81f6530 con 0x7f4cf80d09c0 2024-04-25T11:19:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.988+0000 7f4cf7e00640 1 -- 172.21.15.7:0/640327506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf81e8e20 con 0x7f4cf80d09c0 2024-04-25T11:19:08.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.990+0000 7f4cef400640 1 -- 172.21.15.7:0/640327506 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4ce0018070 con 0x7f4cf80d09c0 2024-04-25T11:19:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.991+0000 7f4cef400640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cd40744b0 0x7f4cd40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.991+0000 7f4cef400640 1 -- 172.21.15.7:0/640327506 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6001+0+0 (secure 0 0 0) 0x7f4ce0095260 con 0x7f4cf80d09c0 2024-04-25T11:19:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.991+0000 7f4cf6a00640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cd40744b0 0x7f4cd40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:08.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.992+0000 7f4cf6a00640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cd40744b0 0x7f4cd40769c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8034000 tx=0x7f4ce802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:08.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:08.997+0000 7f4cef400640 1 -- 172.21.15.7:0/640327506 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4ce005f8b0 con 0x7f4cf80d09c0 2024-04-25T11:19:09.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.242+0000 7f4cf7e00640 1 -- 172.21.15.7:0/640327506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4cf8050740 con 0x7f4cf80d09c0 2024-04-25T11:19:09.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.673+0000 7f4cef400640 1 -- 172.21.15.7:0/640327506 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v63) ==== 174+0+0 (secure 0 0 0) 0x7f4ce005f1b0 con 0x7f4cf80d09c0 2024-04-25T11:19:09.673 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T11:19:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.678+0000 7f4cf7e00640 1 -- 172.21.15.7:0/640327506 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cd40744b0 msgr2=0x7f4cd40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.678+0000 7f4cf7e00640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cd40744b0 0x7f4cd40769c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8034000 tx=0x7f4ce802e000 comp rx=0 tx=0).stop 2024-04-25T11:19:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.678+0000 7f4cf7e00640 1 -- 172.21.15.7:0/640327506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf80d09c0 msgr2=0x7f4cf815cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.678+0000 7f4cf7e00640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf80d09c0 0x7f4cf815cce0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c7f0 tx=0x7f4ce000ccb0 comp rx=0 tx=0).stop 2024-04-25T11:19:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.678+0000 7f4cf7e00640 1 -- 172.21.15.7:0/640327506 shutdown_connections 2024-04-25T11:19:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.678+0000 7f4cf7e00640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cd40744b0 0x7f4cd40769c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.678+0000 7f4cf7e00640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cf81583c0 0x7f4cf815d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.678+0000 7f4cf7e00640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf80d09c0 0x7f4cf815cce0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.678+0000 7f4cf7e00640 1 --2- 172.21.15.7:0/640327506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cf80cfb50 0x7f4cf814e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.679+0000 7f4cf7e00640 1 -- 172.21.15.7:0/640327506 >> 172.21.15.7:0/640327506 conn(0x7f4cf80740f0 msgr2=0x7f4cf815f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.679+0000 7f4cf7e00640 1 -- 172.21.15.7:0/640327506 shutdown_connections 2024-04-25T11:19:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.680+0000 7f4cf7e00640 1 -- 172.21.15.7:0/640327506 wait complete. 2024-04-25T11:19:09.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:09.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.982+0000 7f1f31000640 1 Processor -- start 2024-04-25T11:19:09.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.982+0000 7f1f31000640 1 -- start start 2024-04-25T11:19:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.985+0000 7f1f31000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1f2c0b5220 0x7f1f2c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.985+0000 7f1f31000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f2c0af420 0x7f1f2c0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.985+0000 7f1f31000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f2c0cebb0 con 0x7f1f2c0aed40 2024-04-25T11:19:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.985+0000 7f1f31000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1f2c15e1f0 con 0x7f1f2c0af420 2024-04-25T11:19:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.985+0000 7f1f31000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1f2c0afd70 con 0x7f1f2c0b5220 2024-04-25T11:19:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.985+0000 7f1f2be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f2c0af420 0x7f1f2c0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.985+0000 7f1f2be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f2c0af420 0x7f1f2c0af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51626/0 (socket says 172.21.15.7:51626) 2024-04-25T11:19:09.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.985+0000 7f1f2be00640 1 -- 172.21.15.7:0/1890574448 learned_addr learned my addr 172.21.15.7:0/1890574448 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:09.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.986+0000 7f1f23e00640 1 --2- 172.21.15.7:0/1890574448 >> v2:172.21.15.87:3300/0 conn(0x7f1f2c0b5220 0x7f1f2c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:09.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.986+0000 7f1f2be00640 1 -- 172.21.15.7:0/1890574448 >> v2:172.21.15.87:3300/0 conn(0x7f1f2c0b5220 msgr2=0x7f1f2c15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:09.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.986+0000 7f1f2be00640 1 --2- 172.21.15.7:0/1890574448 >> v2:172.21.15.87:3300/0 conn(0x7f1f2c0b5220 0x7f1f2c15dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:09.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.986+0000 7f1f2be00640 1 -- 172.21.15.7:0/1890574448 >> v1:172.21.15.87:6789/0 conn(0x7f1f2c0aed40 legacy=0x7f1f2c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:09.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.986+0000 7f1f2be00640 1 -- 172.21.15.7:0/1890574448 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2c0b64b0 con 0x7f1f2c0af420 2024-04-25T11:19:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.987+0000 7f1f2be00640 1 --2- 172.21.15.7:0/1890574448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f2c0af420 0x7f1f2c0af830 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0b0a90 tx=0x7f1f2402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfa9784dbfbaa7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.987+0000 7f1f2aa00640 1 -- 172.21.15.7:0/1890574448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24036070 con 0x7f1f2c0af420 2024-04-25T11:19:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.987+0000 7f1f2aa00640 1 -- 172.21.15.7:0/1890574448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f24031600 con 0x7f1f2c0af420 2024-04-25T11:19:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.987+0000 7f1f2aa00640 1 -- 172.21.15.7:0/1890574448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f240319d0 con 0x7f1f2c0af420 2024-04-25T11:19:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.988+0000 7f1f31000640 1 -- 172.21.15.7:0/1890574448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f2c0af420 msgr2=0x7f1f2c0af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.988+0000 7f1f31000640 1 --2- 172.21.15.7:0/1890574448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f2c0af420 0x7f1f2c0af830 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0b0a90 tx=0x7f1f2402a6f0 comp rx=0 tx=0).stop 2024-04-25T11:19:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.988+0000 7f1f31000640 1 -- 172.21.15.7:0/1890574448 shutdown_connections 2024-04-25T11:19:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.988+0000 7f1f31000640 1 --2- 172.21.15.7:0/1890574448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f2c0af420 0x7f1f2c0af830 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.988+0000 7f1f31000640 1 --2- 172.21.15.7:0/1890574448 >> v2:172.21.15.87:3300/0 conn(0x7f1f2c0b5220 0x7f1f2c15dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.988+0000 7f1f31000640 1 -- 172.21.15.7:0/1890574448 >> 172.21.15.7:0/1890574448 conn(0x7f1f2c0b3820 msgr2=0x7f1f2c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.988+0000 7f1f31000640 1 -- 172.21.15.7:0/1890574448 shutdown_connections 2024-04-25T11:19:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.988+0000 7f1f31000640 1 -- 172.21.15.7:0/1890574448 wait complete. 2024-04-25T11:19:09.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.991+0000 7f1f31000640 1 Processor -- start 2024-04-25T11:19:09.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.991+0000 7f1f31000640 1 -- start start 2024-04-25T11:19:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.992+0000 7f1f31000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c0aed40 0x7f1f2c15edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.992+0000 7f1f31000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f2c0af420 0x7f1f2c145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.992+0000 7f1f31000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f2c0b5220 0x7f1f2c1559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.992+0000 7f1f31000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c06cbd0 con 0x7f1f2c0b5220 2024-04-25T11:19:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.992+0000 7f1f31000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c06c8f0 con 0x7f1f2c0aed40 2024-04-25T11:19:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.992+0000 7f1f31000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2c06ceb0 con 0x7f1f2c0af420 2024-04-25T11:19:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.992+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f2c0b5220 0x7f1f2c1559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.993+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f2c0b5220 0x7f1f2c1559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51634/0 (socket says 172.21.15.7:51634) 2024-04-25T11:19:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.993+0000 7f1f2be00640 1 -- 172.21.15.7:0/722118401 learned_addr learned my addr 172.21.15.7:0/722118401 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.993+0000 7f1f23e00640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f2c0af420 0x7f1f2c145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.993+0000 7f1f2b400640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c0aed40 0x7f1f2c15edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.993+0000 7f1f2be00640 1 -- 172.21.15.7:0/722118401 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f2c0af420 msgr2=0x7f1f2c145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.993+0000 7f1f2be00640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f2c0af420 0x7f1f2c145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.993+0000 7f1f2be00640 1 -- 172.21.15.7:0/722118401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c0aed40 msgr2=0x7f1f2c15edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.993+0000 7f1f2be00640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c0aed40 0x7f1f2c15edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.993+0000 7f1f2be00640 1 -- 172.21.15.7:0/722118401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2c0cf400 con 0x7f1f2c0b5220 2024-04-25T11:19:09.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.993+0000 7f1f2be00640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f2c0b5220 0x7f1f2c1559a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009f40 tx=0x7f1f24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:09.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.994+0000 7f1f23400640 1 -- 172.21.15.7:0/722118401 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24036050 con 0x7f1f2c0b5220 2024-04-25T11:19:09.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.994+0000 7f1f23400640 1 -- 172.21.15.7:0/722118401 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f24042070 con 0x7f1f2c0b5220 2024-04-25T11:19:09.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.994+0000 7f1f23400640 1 -- 172.21.15.7:0/722118401 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24009510 con 0x7f1f2c0b5220 2024-04-25T11:19:09.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.994+0000 7f1f31000640 1 -- 172.21.15.7:0/722118401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2c0cf6e0 con 0x7f1f2c0b5220 2024-04-25T11:19:09.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.994+0000 7f1f31000640 1 -- 172.21.15.7:0/722118401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f2c0cfca0 con 0x7f1f2c0b5220 2024-04-25T11:19:09.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.995+0000 7f1f31000640 1 -- 172.21.15.7:0/722118401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f2c0b1970 con 0x7f1f2c0b5220 2024-04-25T11:19:09.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.996+0000 7f1f23400640 1 -- 172.21.15.7:0/722118401 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1f24007bd0 con 0x7f1f2c0b5220 2024-04-25T11:19:09.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.997+0000 7f1f23400640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f080744b0 0x7f1f080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:09.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.998+0000 7f1f23400640 1 -- 172.21.15.7:0/722118401 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f1f240b6390 con 0x7f1f2c0b5220 2024-04-25T11:19:09.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.998+0000 7f1f2b400640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f080744b0 0x7f1f080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:09.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:09.998+0000 7f1f2b400640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f080744b0 0x7f1f080769c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c006fd0 tx=0x7f1f1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.002+0000 7f1f23400640 1 -- 172.21.15.7:0/722118401 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f2c0cebb0 con 0x7f1f2c0b5220 2024-04-25T11:19:10.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.197+0000 7f1f31000640 1 -- 172.21.15.7:0/722118401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f2c06e0b0 con 0x7f1f2c0b5220 2024-04-25T11:19:10.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.200+0000 7f1f23400640 1 -- 172.21.15.7:0/722118401 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16801 (secure 0 0 0) 0x7f1f240808f0 con 0x7f1f2c0b5220 2024-04-25T11:19:10.200 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:10.200 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":105,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:09.244769+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:07.435764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:08.209216+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 -- 172.21.15.7:0/722118401 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f080744b0 msgr2=0x7f1f080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f080744b0 0x7f1f080769c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c006fd0 tx=0x7f1f1c008040 comp rx=0 tx=0).stop 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 -- 172.21.15.7:0/722118401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f2c0b5220 msgr2=0x7f1f2c1559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f2c0b5220 0x7f1f2c1559a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009f40 tx=0x7f1f24004a40 comp rx=0 tx=0).stop 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 -- 172.21.15.7:0/722118401 shutdown_connections 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1f080744b0 0x7f1f080769c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f2c0b5220 0x7f1f2c1559a0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f2c0af420 0x7f1f2c145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 --2- 172.21.15.7:0/722118401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c0aed40 0x7f1f2c15edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 -- 172.21.15.7:0/722118401 >> 172.21.15.7:0/722118401 conn(0x7f1f2c0b3820 msgr2=0x7f1f2c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 -- 172.21.15.7:0/722118401 shutdown_connections 2024-04-25T11:19:10.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.203+0000 7f1f31000640 1 -- 172.21.15.7:0/722118401 wait complete. 2024-04-25T11:19:10.227 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.510+0000 7f4999c00640 1 Processor -- start 2024-04-25T11:19:10.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.510+0000 7f4999c00640 1 -- start start 2024-04-25T11:19:10.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.512+0000 7f4999c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4994070a70 0x7f49940ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.512+0000 7f4999c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f49940aedc0 0x7f49940b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.512+0000 7f4999c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49940b0a10 0x7f49941607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.512+0000 7f4999c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f499415e190 con 0x7f4994070a70 2024-04-25T11:19:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.512+0000 7f4999c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f499415d610 con 0x7f49940aedc0 2024-04-25T11:19:10.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.512+0000 7f4999c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f499415d8f0 con 0x7f49940b0a10 2024-04-25T11:19:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.513+0000 7f4993e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4994070a70 0x7f49940ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.513+0000 7f4993e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4994070a70 0x7f49940ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51644/0 (socket says 172.21.15.7:51644) 2024-04-25T11:19:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.513+0000 7f4993e00640 1 -- 172.21.15.7:0/3062333639 learned_addr learned my addr 172.21.15.7:0/3062333639 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.513+0000 7f498be00640 1 --2- 172.21.15.7:0/3062333639 >> v2:172.21.15.87:3300/0 conn(0x7f49940aedc0 0x7f49940b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.513+0000 7f4999200640 1 --2- 172.21.15.7:0/3062333639 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49940b0a10 0x7f49941607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.513+0000 7f4993e00640 1 -- 172.21.15.7:0/3062333639 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49940b0a10 msgr2=0x7f49941607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.513+0000 7f4993e00640 1 --2- 172.21.15.7:0/3062333639 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49940b0a10 0x7f49941607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.513+0000 7f4993e00640 1 -- 172.21.15.7:0/3062333639 >> v2:172.21.15.87:3300/0 conn(0x7f49940aedc0 msgr2=0x7f49940b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.513+0000 7f4993e00640 1 --2- 172.21.15.7:0/3062333639 >> v2:172.21.15.87:3300/0 conn(0x7f49940aedc0 0x7f49940b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.513+0000 7f4993e00640 1 -- 172.21.15.7:0/3062333639 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499415dbd0 con 0x7f4994070a70 2024-04-25T11:19:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.514+0000 7f4993e00640 1 --2- 172.21.15.7:0/3062333639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4994070a70 0x7f49940ae880 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4994071180 tx=0x7f498402c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75686fc66199773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.515+0000 7f4993400640 1 -- 172.21.15.7:0/3062333639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984038070 con 0x7f4994070a70 2024-04-25T11:19:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.515+0000 7f4993400640 1 -- 172.21.15.7:0/3062333639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4984031710 con 0x7f4994070a70 2024-04-25T11:19:10.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.515+0000 7f4993400640 1 -- 172.21.15.7:0/3062333639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984031ae0 con 0x7f4994070a70 2024-04-25T11:19:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.515+0000 7f4999c00640 1 -- 172.21.15.7:0/3062333639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4994070a70 msgr2=0x7f49940ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.515+0000 7f4999c00640 1 --2- 172.21.15.7:0/3062333639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4994070a70 0x7f49940ae880 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4994071180 tx=0x7f498402c3f0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.516+0000 7f4999c00640 1 -- 172.21.15.7:0/3062333639 shutdown_connections 2024-04-25T11:19:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.516+0000 7f4999c00640 1 --2- 172.21.15.7:0/3062333639 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49940b0a10 0x7f49941607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.516+0000 7f4999c00640 1 --2- 172.21.15.7:0/3062333639 >> v2:172.21.15.87:3300/0 conn(0x7f49940aedc0 0x7f49940b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.516+0000 7f4999c00640 1 --2- 172.21.15.7:0/3062333639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4994070a70 0x7f49940ae880 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.516+0000 7f4999c00640 1 -- 172.21.15.7:0/3062333639 >> 172.21.15.7:0/3062333639 conn(0x7f49940b3820 msgr2=0x7f49940b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.516+0000 7f4999c00640 1 -- 172.21.15.7:0/3062333639 shutdown_connections 2024-04-25T11:19:10.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.516+0000 7f4999c00640 1 -- 172.21.15.7:0/3062333639 wait complete. 2024-04-25T11:19:10.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.518+0000 7f4999c00640 1 Processor -- start 2024-04-25T11:19:10.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.518+0000 7f4999c00640 1 -- start start 2024-04-25T11:19:10.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.520+0000 7f4999c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4994070a70 0x7f49940b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:10.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.520+0000 7f4999c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49940aedc0 0x7f49941605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:10.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.520+0000 7f4999c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49940b0a10 0x7f4994160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:10.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.520+0000 7f4999c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f499415d170 con 0x7f49940aedc0 2024-04-25T11:19:10.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.520+0000 7f4999c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49940b0e20 con 0x7f49940b0a10 2024-04-25T11:19:10.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.520+0000 7f4999c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f499415d2f0 con 0x7f4994070a70 2024-04-25T11:19:10.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.520+0000 7f498be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49940aedc0 0x7f49941605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:10.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.520+0000 7f498be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49940aedc0 0x7f49941605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51648/0 (socket says 172.21.15.7:51648) 2024-04-25T11:19:10.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.520+0000 7f498be00640 1 -- 172.21.15.7:0/1355499714 learned_addr learned my addr 172.21.15.7:0/1355499714 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:10.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f4999200640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49940b0a10 0x7f4994160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f4993e00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4994070a70 0x7f49940b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f498be00640 1 -- 172.21.15.7:0/1355499714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4994070a70 msgr2=0x7f49940b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f498be00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4994070a70 0x7f49940b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f498be00640 1 -- 172.21.15.7:0/1355499714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49940b0a10 msgr2=0x7f4994160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f498be00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49940b0a10 0x7f4994160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f498be00640 1 -- 172.21.15.7:0/1355499714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49940b1630 con 0x7f49940aedc0 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f4999200640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49940b0a10 0x7f4994160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f498be00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49940aedc0 0x7f49941605d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f497c00d830 tx=0x7f497c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f4990c00640 1 -- 172.21.15.7:0/1355499714 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c018030 con 0x7f49940aedc0 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f4990c00640 1 -- 172.21.15.7:0/1355499714 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497c00be10 con 0x7f49940aedc0 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f4990c00640 1 -- 172.21.15.7:0/1355499714 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c0098b0 con 0x7f49940aedc0 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.521+0000 7f4999c00640 1 -- 172.21.15.7:0/1355499714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49940b22c0 con 0x7f49940aedc0 2024-04-25T11:19:10.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.522+0000 7f4999c00640 1 -- 172.21.15.7:0/1355499714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4994155aa0 con 0x7f49940aedc0 2024-04-25T11:19:10.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.523+0000 7f4999c00640 1 -- 172.21.15.7:0/1355499714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4994050740 con 0x7f49940aedc0 2024-04-25T11:19:10.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.524+0000 7f4990c00640 1 -- 172.21.15.7:0/1355499714 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f497c01b070 con 0x7f49940aedc0 2024-04-25T11:19:10.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.525+0000 7f4990c00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49700744e0 0x7f49700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:10.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.525+0000 7f4990c00640 1 -- 172.21.15.7:0/1355499714 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6101+0+0 (secure 0 0 0) 0x7f497c0955d0 con 0x7f49940aedc0 2024-04-25T11:19:10.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.525+0000 7f4993e00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49700744e0 0x7f49700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:10.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.526+0000 7f4993e00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49700744e0 0x7f49700769f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4984035000 tx=0x7f4984008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:10.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.530+0000 7f4990c00640 1 -- 172.21.15.7:0/1355499714 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f497c05fbc0 con 0x7f49940aedc0 2024-04-25T11:19:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.741+0000 7f4999c00640 1 -- 172.21.15.7:0/1355499714 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49940b0490 con 0x7f49940aedc0 2024-04-25T11:19:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.743+0000 7f4990c00640 1 -- 172.21.15.7:0/1355499714 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16788 (secure 0 0 0) 0x7f497c05f4c0 con 0x7f49940aedc0 2024-04-25T11:19:10.742 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:10.743 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:10.691402+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:07.435764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:08.209216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.745+0000 7f4999c00640 1 -- 172.21.15.7:0/1355499714 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49700744e0 msgr2=0x7f49700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.745+0000 7f4999c00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49700744e0 0x7f49700769f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4984035000 tx=0x7f4984008030 comp rx=0 tx=0).stop 2024-04-25T11:19:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.745+0000 7f4999c00640 1 -- 172.21.15.7:0/1355499714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49940aedc0 msgr2=0x7f49941605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.745+0000 7f4999c00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49940aedc0 0x7f49941605d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f497c00d830 tx=0x7f497c00dcf0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.745+0000 7f4999c00640 1 -- 172.21.15.7:0/1355499714 shutdown_connections 2024-04-25T11:19:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.745+0000 7f4999c00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49700744e0 0x7f49700769f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.745+0000 7f4999c00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49940b0a10 0x7f4994160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.745+0000 7f4999c00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49940aedc0 0x7f49941605d0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.745+0000 7f4999c00640 1 --2- 172.21.15.7:0/1355499714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4994070a70 0x7f49940b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.745+0000 7f4999c00640 1 -- 172.21.15.7:0/1355499714 >> 172.21.15.7:0/1355499714 conn(0x7f49940b3820 msgr2=0x7f499406d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.746+0000 7f4999c00640 1 -- 172.21.15.7:0/1355499714 shutdown_connections 2024-04-25T11:19:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:10.746+0000 7f4999c00640 1 -- 172.21.15.7:0/1355499714 wait complete. 2024-04-25T11:19:10.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:19:11.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.049+0000 7f422de00640 1 Processor -- start 2024-04-25T11:19:11.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.049+0000 7f422de00640 1 -- start start 2024-04-25T11:19:11.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.051+0000 7f422de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42281583c0 0x7f42280cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:11.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.051+0000 7f422de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f42280cfb50 0x7f42280d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:11.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.051+0000 7f422de00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42280ce570 con 0x7f42280d09c0 2024-04-25T11:19:11.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.051+0000 7f422de00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f422815f190 con 0x7f42281583c0 2024-04-25T11:19:11.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.051+0000 7f422de00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f42280ce850 con 0x7f42280cfb50 2024-04-25T11:19:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.052+0000 7f422ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42281583c0 0x7f42280cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.052+0000 7f422ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42281583c0 0x7f42280cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51658/0 (socket says 172.21.15.7:51658) 2024-04-25T11:19:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.052+0000 7f422ca00640 1 -- 172.21.15.7:0/3176994060 learned_addr learned my addr 172.21.15.7:0/3176994060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.052+0000 7f422ca00640 1 -- 172.21.15.7:0/3176994060 >> v2:172.21.15.87:3300/0 conn(0x7f42280cfb50 msgr2=0x7f42280d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.052+0000 7f4227e00640 1 --2- 172.21.15.7:0/3176994060 >> v2:172.21.15.87:3300/0 conn(0x7f42280cfb50 0x7f42280d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.052+0000 7f4227400640 1 -- 172.21.15.7:0/3176994060 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 849568326 0 0) 0x7f42280ce570 con 0x7f42280d09c0 2024-04-25T11:19:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.052+0000 7f422ca00640 1 --2- 172.21.15.7:0/3176994060 >> v2:172.21.15.87:3300/0 conn(0x7f42280cfb50 0x7f42280d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.052+0000 7f422ca00640 1 -- 172.21.15.7:0/3176994060 >> v1:172.21.15.7:6789/0 conn(0x7f42280d09c0 legacy=0x7f422806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.052+0000 7f422ca00640 1 -- 172.21.15.7:0/3176994060 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42280ceb30 con 0x7f42281583c0 2024-04-25T11:19:11.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.052+0000 7f4227e00640 1 --2- 172.21.15.7:0/3176994060 >> v2:172.21.15.87:3300/0 conn(0x7f42280cfb50 0x7f42280d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.053+0000 7f422ca00640 1 --2- 172.21.15.7:0/3176994060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42281583c0 0x7f42280cf610 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4228070ad0 tx=0x7f421c02a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8956c6679a9ea8d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.054+0000 7f4227400640 1 -- 172.21.15.7:0/3176994060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c037070 con 0x7f42281583c0 2024-04-25T11:19:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.054+0000 7f4227400640 1 -- 172.21.15.7:0/3176994060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f421c032440 con 0x7f42281583c0 2024-04-25T11:19:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.054+0000 7f4227400640 1 -- 172.21.15.7:0/3176994060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c032810 con 0x7f42281583c0 2024-04-25T11:19:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.054+0000 7f422de00640 1 -- 172.21.15.7:0/3176994060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42281583c0 msgr2=0x7f42280cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.054+0000 7f422de00640 1 --2- 172.21.15.7:0/3176994060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42281583c0 0x7f42280cf610 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4228070ad0 tx=0x7f421c02a1f0 comp rx=0 tx=0).stop 2024-04-25T11:19:11.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.055+0000 7f422de00640 1 -- 172.21.15.7:0/3176994060 shutdown_connections 2024-04-25T11:19:11.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.055+0000 7f422de00640 1 --2- 172.21.15.7:0/3176994060 >> v2:172.21.15.87:3300/0 conn(0x7f42280cfb50 0x7f42280d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:11.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.055+0000 7f422de00640 1 --2- 172.21.15.7:0/3176994060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42281583c0 0x7f42280cf610 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:11.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.055+0000 7f422de00640 1 -- 172.21.15.7:0/3176994060 >> 172.21.15.7:0/3176994060 conn(0x7f42280740f0 msgr2=0x7f422806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:11.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.055+0000 7f422de00640 1 -- 172.21.15.7:0/3176994060 shutdown_connections 2024-04-25T11:19:11.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.055+0000 7f422de00640 1 -- 172.21.15.7:0/3176994060 wait complete. 2024-04-25T11:19:11.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.057+0000 7f422de00640 1 Processor -- start 2024-04-25T11:19:11.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.057+0000 7f422de00640 1 -- start start 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.059+0000 7f422de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42280cfb50 0x7f4228146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.059+0000 7f422de00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42280d09c0 0x7f42281e8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.059+0000 7f422de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42281583c0 0x7f42281e88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.059+0000 7f422de00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f422806f5b0 con 0x7f42281583c0 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.059+0000 7f422de00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f42280cf0f0 con 0x7f42280cfb50 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.059+0000 7f422de00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f422806f730 con 0x7f42280d09c0 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.059+0000 7f422d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42281583c0 0x7f42281e88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.059+0000 7f422d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42281583c0 0x7f42281e88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51662/0 (socket says 172.21.15.7:51662) 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.059+0000 7f422d400640 1 -- 172.21.15.7:0/4260450560 learned_addr learned my addr 172.21.15.7:0/4260450560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.059+0000 7f422ca00640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42280cfb50 0x7f4228146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.059+0000 7f4227e00640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42280d09c0 0x7f42281e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:11.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.060+0000 7f422d400640 1 -- 172.21.15.7:0/4260450560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42280d09c0 msgr2=0x7f42281e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:11.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.060+0000 7f422d400640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42280d09c0 0x7f42281e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:11.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.060+0000 7f422d400640 1 -- 172.21.15.7:0/4260450560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42280cfb50 msgr2=0x7f4228146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:11.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.060+0000 7f422d400640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42280cfb50 0x7f4228146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:11.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.060+0000 7f422d400640 1 -- 172.21.15.7:0/4260450560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42281472d0 con 0x7f42281583c0 2024-04-25T11:19:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.060+0000 7f422d400640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42281583c0 0x7f42281e88d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f422000c7b0 tx=0x7f422000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.060+0000 7f4224c00640 1 -- 172.21.15.7:0/4260450560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422000ce60 con 0x7f42281583c0 2024-04-25T11:19:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.060+0000 7f4224c00640 1 -- 172.21.15.7:0/4260450560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4220003da0 con 0x7f42281583c0 2024-04-25T11:19:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.060+0000 7f4224c00640 1 -- 172.21.15.7:0/4260450560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42200052b0 con 0x7f42281583c0 2024-04-25T11:19:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.060+0000 7f422de00640 1 -- 172.21.15.7:0/4260450560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42281eb5c0 con 0x7f42281583c0 2024-04-25T11:19:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.061+0000 7f422de00640 1 -- 172.21.15.7:0/4260450560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42281e9c20 con 0x7f42281583c0 2024-04-25T11:19:11.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.062+0000 7f422de00640 1 -- 172.21.15.7:0/4260450560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42280c6950 con 0x7f42281583c0 2024-04-25T11:19:11.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.063+0000 7f4224c00640 1 -- 172.21.15.7:0/4260450560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4220005450 con 0x7f42281583c0 2024-04-25T11:19:11.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.065+0000 7f4224c00640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42080744b0 0x7f42080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:11.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.065+0000 7f4224c00640 1 -- 172.21.15.7:0/4260450560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f4220095210 con 0x7f42281583c0 2024-04-25T11:19:11.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.065+0000 7f422ca00640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42080744b0 0x7f42080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:11.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.066+0000 7f422ca00640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42080744b0 0x7f42080769c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f421c034000 tx=0x7f421c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:11.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.070+0000 7f4224c00640 1 -- 172.21.15.7:0/4260450560 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f422005f800 con 0x7f42281583c0 2024-04-25T11:19:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.318+0000 7f422de00640 1 -- 172.21.15.7:0/4260450560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4228050740 con 0x7f42281583c0 2024-04-25T11:19:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.716+0000 7f4224c00640 1 -- 172.21.15.7:0/4260450560 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) ==== 118+0+0 (secure 0 0 0) 0x7f422005f100 con 0x7f42281583c0 2024-04-25T11:19:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.720+0000 7f422de00640 1 -- 172.21.15.7:0/4260450560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42080744b0 msgr2=0x7f42080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:11.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.720+0000 7f422de00640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42080744b0 0x7f42080769c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f421c034000 tx=0x7f421c02e000 comp rx=0 tx=0).stop 2024-04-25T11:19:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.720+0000 7f422de00640 1 -- 172.21.15.7:0/4260450560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42281583c0 msgr2=0x7f42281e88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.720+0000 7f422de00640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42281583c0 0x7f42281e88d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f422000c7b0 tx=0x7f422000cc70 comp rx=0 tx=0).stop 2024-04-25T11:19:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.720+0000 7f422de00640 1 -- 172.21.15.7:0/4260450560 shutdown_connections 2024-04-25T11:19:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.720+0000 7f422de00640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42080744b0 0x7f42080769c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.720+0000 7f422de00640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42281583c0 0x7f42281e88d0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.720+0000 7f422de00640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42280d09c0 0x7f42281e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.720+0000 7f422de00640 1 --2- 172.21.15.7:0/4260450560 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42280cfb50 0x7f4228146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.720+0000 7f422de00640 1 -- 172.21.15.7:0/4260450560 >> 172.21.15.7:0/4260450560 conn(0x7f42280740f0 msgr2=0x7f422815f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.721+0000 7f422de00640 1 -- 172.21.15.7:0/4260450560 shutdown_connections 2024-04-25T11:19:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:11.721+0000 7f422de00640 1 -- 172.21.15.7:0/4260450560 wait complete. 2024-04-25T11:19:11.756 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:19:11.756 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:19:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.048+0000 7f761b000640 1 Processor -- start 2024-04-25T11:19:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.048+0000 7f761b000640 1 -- start start 2024-04-25T11:19:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.050+0000 7f761b000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76140d4d80 0x7f76140d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.050+0000 7f761b000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7614161030 con 0x7f76140d5190 2024-04-25T11:19:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.050+0000 7f761b000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7614161310 con 0x7f7614161df0 2024-04-25T11:19:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.050+0000 7f761b000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f76140d4580 con 0x7f76140d4d80 2024-04-25T11:19:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.051+0000 7f7619c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76140d4d80 0x7f76140d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.051+0000 7f7619c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76140d4d80 0x7f76140d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51676/0 (socket says 172.21.15.7:51676) 2024-04-25T11:19:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.051+0000 7f7619c00640 1 -- 172.21.15.7:0/2765491605 learned_addr learned my addr 172.21.15.7:0/2765491605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.051+0000 7f7619200640 1 --1- 172.21.15.7:0/2765491605 >> v1:172.21.15.7:6789/0 conn(0x7f76140d5190 0x7f76141616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37942/0 (socket says 172.21.15.7:37942) 2024-04-25T11:19:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.051+0000 7f7619c00640 1 -- 172.21.15.7:0/2765491605 >> v1:172.21.15.87:6789/0 conn(0x7f7614161df0 legacy=0x7f76141601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.051+0000 7f7619c00640 1 -- 172.21.15.7:0/2765491605 >> v1:172.21.15.7:6789/0 conn(0x7f76140d5190 legacy=0x7f76141616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.051+0000 7f7619c00640 1 -- 172.21.15.7:0/2765491605 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76140d4860 con 0x7f76140d4d80 2024-04-25T11:19:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.053+0000 7f7619c00640 1 --2- 172.21.15.7:0/2765491605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76140d4d80 0x7f76140d2780 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f7604009870 tx=0x7f760402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e253906baf127bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.053+0000 7f7613e00640 1 -- 172.21.15.7:0/2765491605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604037070 con 0x7f76140d4d80 2024-04-25T11:19:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.053+0000 7f7613e00640 1 -- 172.21.15.7:0/2765491605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7604032440 con 0x7f76140d4d80 2024-04-25T11:19:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.053+0000 7f7613e00640 1 -- 172.21.15.7:0/2765491605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604032810 con 0x7f76140d4d80 2024-04-25T11:19:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.054+0000 7f761b000640 1 -- 172.21.15.7:0/2765491605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76140d4d80 msgr2=0x7f76140d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.054+0000 7f761b000640 1 --2- 172.21.15.7:0/2765491605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76140d4d80 0x7f76140d2780 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f7604009870 tx=0x7f760402a2a0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.054+0000 7f761b000640 1 -- 172.21.15.7:0/2765491605 shutdown_connections 2024-04-25T11:19:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.054+0000 7f761b000640 1 --2- 172.21.15.7:0/2765491605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76140d4d80 0x7f76140d2780 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.054+0000 7f761b000640 1 -- 172.21.15.7:0/2765491605 >> 172.21.15.7:0/2765491605 conn(0x7f761406c750 msgr2=0x7f761406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.054+0000 7f761b000640 1 -- 172.21.15.7:0/2765491605 shutdown_connections 2024-04-25T11:19:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.054+0000 7f761b000640 1 -- 172.21.15.7:0/2765491605 wait complete. 2024-04-25T11:19:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.057+0000 7f761b000640 1 Processor -- start 2024-04-25T11:19:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.057+0000 7f761b000640 1 -- start start 2024-04-25T11:19:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.058+0000 7f761b000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76140d4d80 0x7f76140b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.058+0000 7f761b000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76140d5190 0x7f761415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.058+0000 7f761b000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7614161df0 0x7f76140ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.058+0000 7f761b000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76140d2e40 con 0x7f76140d4d80 2024-04-25T11:19:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.058+0000 7f761b000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f76140d2cc0 con 0x7f76140d5190 2024-04-25T11:19:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.058+0000 7f761b000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f76140d2fc0 con 0x7f7614161df0 2024-04-25T11:19:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f7619c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76140d4d80 0x7f76140b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f7619c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76140d4d80 0x7f76140b76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51692/0 (socket says 172.21.15.7:51692) 2024-04-25T11:19:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f7619c00640 1 -- 172.21.15.7:0/4202343317 learned_addr learned my addr 172.21.15.7:0/4202343317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f7619200640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76140d5190 0x7f761415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f761a600640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7614161df0 0x7f76140ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f7619c00640 1 -- 172.21.15.7:0/4202343317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7614161df0 msgr2=0x7f76140ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f7619c00640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7614161df0 0x7f76140ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f7619c00640 1 -- 172.21.15.7:0/4202343317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76140d5190 msgr2=0x7f761415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f7619c00640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76140d5190 0x7f761415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f7619c00640 1 -- 172.21.15.7:0/4202343317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76140b7c30 con 0x7f76140d4d80 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f7619200640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76140d5190 0x7f761415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.059+0000 7f761a600640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7614161df0 0x7f76140ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.060+0000 7f7619c00640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76140d4d80 0x7f76140b76f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7604009870 tx=0x7f76040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.060+0000 7f7611600640 1 -- 172.21.15.7:0/4202343317 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604037040 con 0x7f76140d4d80 2024-04-25T11:19:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.060+0000 7f7611600640 1 -- 172.21.15.7:0/4202343317 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7604042070 con 0x7f76140d4d80 2024-04-25T11:19:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.060+0000 7f7611600640 1 -- 172.21.15.7:0/4202343317 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604030bb0 con 0x7f76140d4d80 2024-04-25T11:19:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.060+0000 7f761b000640 1 -- 172.21.15.7:0/4202343317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76140cdd50 con 0x7f76140d4d80 2024-04-25T11:19:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.060+0000 7f761b000640 1 -- 172.21.15.7:0/4202343317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76140ce310 con 0x7f76140d4d80 2024-04-25T11:19:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.061+0000 7f761b000640 1 -- 172.21.15.7:0/4202343317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7614050740 con 0x7f76140d4d80 2024-04-25T11:19:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.063+0000 7f7611600640 1 -- 172.21.15.7:0/4202343317 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7604047460 con 0x7f76140d4d80 2024-04-25T11:19:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.064+0000 7f7611600640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f40744e0 0x7f75f40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.064+0000 7f7611600640 1 -- 172.21.15.7:0/4202343317 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f76040b5f90 con 0x7f76140d4d80 2024-04-25T11:19:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.064+0000 7f7619200640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f40744e0 0x7f75f40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.065+0000 7f7619200640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f40744e0 0x7f75f40769f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f76140d0140 tx=0x7f7600009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:12.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.069+0000 7f7611600640 1 -- 172.21.15.7:0/4202343317 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7604080620 con 0x7f76140d4d80 2024-04-25T11:19:12.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.316+0000 7f761b000640 1 -- 172.21.15.7:0/4202343317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f76141ebe00 con 0x7f76140d4d80 2024-04-25T11:19:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.727+0000 7f7611600640 1 -- 172.21.15.7:0/4202343317 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) ==== 122+0+0 (secure 0 0 0) 0x7f760407ff20 con 0x7f76140d4d80 2024-04-25T11:19:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.732+0000 7f761b000640 1 -- 172.21.15.7:0/4202343317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f40744e0 msgr2=0x7f75f40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.732+0000 7f761b000640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f40744e0 0x7f75f40769f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f76140d0140 tx=0x7f7600009040 comp rx=0 tx=0).stop 2024-04-25T11:19:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.732+0000 7f761b000640 1 -- 172.21.15.7:0/4202343317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76140d4d80 msgr2=0x7f76140b76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.732+0000 7f761b000640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76140d4d80 0x7f76140b76f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f7604009870 tx=0x7f76040046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.732+0000 7f761b000640 1 -- 172.21.15.7:0/4202343317 shutdown_connections 2024-04-25T11:19:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.732+0000 7f761b000640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75f40744e0 0x7f75f40769f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.732+0000 7f761b000640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7614161df0 0x7f76140ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.732+0000 7f761b000640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76140d5190 0x7f761415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.732+0000 7f761b000640 1 --2- 172.21.15.7:0/4202343317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76140d4d80 0x7f76140b76f0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.732+0000 7f761b000640 1 -- 172.21.15.7:0/4202343317 >> 172.21.15.7:0/4202343317 conn(0x7f761406c750 msgr2=0x7f761415ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.733+0000 7f761b000640 1 -- 172.21.15.7:0/4202343317 shutdown_connections 2024-04-25T11:19:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:12.733+0000 7f761b000640 1 -- 172.21.15.7:0/4202343317 wait complete. 2024-04-25T11:19:12.764 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:19:12.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.055+0000 7f10f0a00640 1 Processor -- start 2024-04-25T11:19:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.055+0000 7f10f0a00640 1 -- start start 2024-04-25T11:19:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.057+0000 7f10f0a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f10ec06ffd0 0x7f10ec164380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.058+0000 7f10f0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10ec0b5f10 0x7f10ec0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.058+0000 7f10f0a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10ec0b47d0 con 0x7f10ec0af390 2024-04-25T11:19:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.058+0000 7f10f0a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f10ec0b6320 con 0x7f10ec06ffd0 2024-04-25T11:19:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.058+0000 7f10f0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f10ec0b4a40 con 0x7f10ec0b5f10 2024-04-25T11:19:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.058+0000 7f10eb400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f10ec06ffd0 0x7f10ec164380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.058+0000 7f10eaa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f10ec0af390 0x7f10ec0b5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37944/0 (socket says 172.21.15.7:37944) 2024-04-25T11:19:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.058+0000 7f10eaa00640 1 -- 172.21.15.7:0/804386441 learned_addr learned my addr 172.21.15.7:0/804386441 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:13.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.058+0000 7f10ea000640 1 -- 172.21.15.7:0/804386441 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2772209413 0 0) 0x7f10ec0b47d0 con 0x7f10ec0af390 2024-04-25T11:19:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.058+0000 7f10ea000640 1 -- 172.21.15.7:0/804386441 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10d0003520 con 0x7f10ec0af390 2024-04-25T11:19:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.058+0000 7f10ebe00640 1 --2- 172.21.15.7:0/804386441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10ec0b5f10 0x7f10ec0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.058+0000 7f10eb400640 1 -- 172.21.15.7:0/804386441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10ec0b5f10 msgr2=0x7f10ec0af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.059+0000 7f10eb400640 1 --2- 172.21.15.7:0/804386441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10ec0b5f10 0x7f10ec0af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.059+0000 7f10eb400640 1 -- 172.21.15.7:0/804386441 >> v1:172.21.15.7:6789/0 conn(0x7f10ec0af390 legacy=0x7f10ec0b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.059+0000 7f10eb400640 1 -- 172.21.15.7:0/804386441 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec0b4d20 con 0x7f10ec06ffd0 2024-04-25T11:19:13.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.059+0000 7f10ebe00640 1 --2- 172.21.15.7:0/804386441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10ec0b5f10 0x7f10ec0af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.060+0000 7f10eb400640 1 --2- 172.21.15.7:0/804386441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10ec06ffd0 0x7f10ec164380 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0097a0 tx=0x7f10dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8236e589bca4f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.060+0000 7f10ea000640 1 -- 172.21.15.7:0/804386441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc037070 con 0x7f10ec06ffd0 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.060+0000 7f10ea000640 1 -- 172.21.15.7:0/804386441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc032440 con 0x7f10ec06ffd0 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.060+0000 7f10ea000640 1 -- 172.21.15.7:0/804386441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0327f0 con 0x7f10ec06ffd0 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.061+0000 7f10f0a00640 1 -- 172.21.15.7:0/804386441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10ec06ffd0 msgr2=0x7f10ec164380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.061+0000 7f10f0a00640 1 --2- 172.21.15.7:0/804386441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10ec06ffd0 0x7f10ec164380 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0097a0 tx=0x7f10dc02a180 comp rx=0 tx=0).stop 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.061+0000 7f10f0a00640 1 -- 172.21.15.7:0/804386441 shutdown_connections 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.061+0000 7f10f0a00640 1 --2- 172.21.15.7:0/804386441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10ec0b5f10 0x7f10ec0af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.061+0000 7f10f0a00640 1 --2- 172.21.15.7:0/804386441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10ec06ffd0 0x7f10ec164380 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.061+0000 7f10f0a00640 1 -- 172.21.15.7:0/804386441 >> 172.21.15.7:0/804386441 conn(0x7f10ec0b3820 msgr2=0x7f10ec0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.061+0000 7f10f0a00640 1 -- 172.21.15.7:0/804386441 shutdown_connections 2024-04-25T11:19:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.061+0000 7f10f0a00640 1 -- 172.21.15.7:0/804386441 wait complete. 2024-04-25T11:19:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.063+0000 7f10f0a00640 1 Processor -- start 2024-04-25T11:19:13.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.064+0000 7f10f0a00640 1 -- start start 2024-04-25T11:19:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.065+0000 7f10f0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10ec06ffd0 0x7f10ec14e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.065+0000 7f10f0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10ec0af390 0x7f10ec0b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.065+0000 7f10f0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10ec0b5f10 0x7f10ec0b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.065+0000 7f10f0a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec06fa70 con 0x7f10ec06ffd0 2024-04-25T11:19:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.065+0000 7f10f0a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec0b52e0 con 0x7f10ec0b5f10 2024-04-25T11:19:13.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.065+0000 7f10f0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f10ec06fbf0 con 0x7f10ec0af390 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.065+0000 7f10eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10ec06ffd0 0x7f10ec14e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.065+0000 7f10eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10ec06ffd0 0x7f10ec14e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51696/0 (socket says 172.21.15.7:51696) 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.066+0000 7f10eb400640 1 -- 172.21.15.7:0/1126646226 learned_addr learned my addr 172.21.15.7:0/1126646226 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.066+0000 7f10ebe00640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10ec0b5f10 0x7f10ec0b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.066+0000 7f10eaa00640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10ec0af390 0x7f10ec0b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.066+0000 7f10eb400640 1 -- 172.21.15.7:0/1126646226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10ec0af390 msgr2=0x7f10ec0b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.066+0000 7f10eb400640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10ec0af390 0x7f10ec0b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.066+0000 7f10eb400640 1 -- 172.21.15.7:0/1126646226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10ec0b5f10 msgr2=0x7f10ec0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.066+0000 7f10eb400640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10ec0b5f10 0x7f10ec0b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.066+0000 7f10eb400640 1 -- 172.21.15.7:0/1126646226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec1614a0 con 0x7f10ec06ffd0 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.066+0000 7f10eb400640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10ec06ffd0 0x7f10ec14e6f0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f10dc035040 tx=0x7f10dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.067+0000 7f10e3400640 1 -- 172.21.15.7:0/1126646226 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc041020 con 0x7f10ec06ffd0 2024-04-25T11:19:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.067+0000 7f10e3400640 1 -- 172.21.15.7:0/1126646226 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc030700 con 0x7f10ec06ffd0 2024-04-25T11:19:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.067+0000 7f10e3400640 1 -- 172.21.15.7:0/1126646226 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc037040 con 0x7f10ec06ffd0 2024-04-25T11:19:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.067+0000 7f10f0a00640 1 -- 172.21.15.7:0/1126646226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ec160c00 con 0x7f10ec06ffd0 2024-04-25T11:19:13.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.067+0000 7f10f0a00640 1 -- 172.21.15.7:0/1126646226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ec1611c0 con 0x7f10ec06ffd0 2024-04-25T11:19:13.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:19:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.068+0000 7f10f0a00640 1 -- 172.21.15.7:0/1126646226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ec050740 con 0x7f10ec06ffd0 2024-04-25T11:19:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.069+0000 7f10e3400640 1 -- 172.21.15.7:0/1126646226 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f10dc044020 con 0x7f10ec06ffd0 2024-04-25T11:19:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.070+0000 7f10e3400640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10c80744b0 0x7f10c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.070+0000 7f10e3400640 1 -- 172.21.15.7:0/1126646226 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f10dc0b56b0 con 0x7f10ec06ffd0 2024-04-25T11:19:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.071+0000 7f10eaa00640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10c80744b0 0x7f10c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.073 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:19:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.072+0000 7f10eaa00640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10c80744b0 0x7f10c80769c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f10ec167a50 tx=0x7f10d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.075+0000 7f10e3400640 1 -- 172.21.15.7:0/1126646226 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10dc04b020 con 0x7f10ec06ffd0 2024-04-25T11:19:13.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.321+0000 7f10f0a00640 1 -- 172.21.15.7:0/1126646226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10ec0af8d0 con 0x7f10ec06ffd0 2024-04-25T11:19:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.322+0000 7f10e3400640 1 -- 172.21.15.7:0/1126646226 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5171 (secure 0 0 0) 0x7f10dc07fca0 con 0x7f10ec06ffd0 2024-04-25T11:19:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:13.322 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5846,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/4132210096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":4132210096},{"type":"v1","addr":"172.21.15.87:6834","nonce":4132210096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2657338269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2657338269},{"type":"v1","addr":"172.21.15.87:6835","nonce":2657338269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6457,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/138020048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":138020048},{"type":"v1","addr":"172.21.15.7:6836","nonce":138020048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:09.244634+0000","modified":"2024-04-25T11:19:12.318984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/990441706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":990441706},{"type":"v1","addr":"172.21.15.7:6837","nonce":990441706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7623,"qdb_cluster":[7623]},"id":7}]} 2024-04-25T11:19:13.322 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 67 2024-04-25T11:19:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 -- 172.21.15.7:0/1126646226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10c80744b0 msgr2=0x7f10c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10c80744b0 0x7f10c80769c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f10ec167a50 tx=0x7f10d4009040 comp rx=0 tx=0).stop 2024-04-25T11:19:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 -- 172.21.15.7:0/1126646226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10ec06ffd0 msgr2=0x7f10ec14e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10ec06ffd0 0x7f10ec14e6f0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f10dc035040 tx=0x7f10dc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 -- 172.21.15.7:0/1126646226 shutdown_connections 2024-04-25T11:19:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10c80744b0 0x7f10c80769c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10ec0b5f10 0x7f10ec0b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10ec0af390 0x7f10ec0b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 --2- 172.21.15.7:0/1126646226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10ec06ffd0 0x7f10ec14e6f0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 -- 172.21.15.7:0/1126646226 >> 172.21.15.7:0/1126646226 conn(0x7f10ec0b3820 msgr2=0x7f10ec167fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 -- 172.21.15.7:0/1126646226 shutdown_connections 2024-04-25T11:19:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.326+0000 7f10f0a00640 1 -- 172.21.15.7:0/1126646226 wait complete. 2024-04-25T11:19:13.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:13.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.640+0000 7f1b2b400640 1 Processor -- start 2024-04-25T11:19:13.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.641+0000 7f1b2b400640 1 -- start start 2024-04-25T11:19:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.643+0000 7f1b2b400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2406f300 0x7f1b240b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.643+0000 7f1b2b400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b240b4980 con 0x7f1b240b0290 2024-04-25T11:19:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.643+0000 7f1b2b400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b240b4c60 con 0x7f1b2406dc90 2024-04-25T11:19:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.643+0000 7f1b2b400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1b240b4f40 con 0x7f1b2406f300 2024-04-25T11:19:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.643+0000 7f1b23e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1b240b0290 0x7f1b240b5a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37958/0 (socket says 172.21.15.7:37958) 2024-04-25T11:19:13.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.644+0000 7f1b23e00640 1 -- 172.21.15.7:0/1782771072 learned_addr learned my addr 172.21.15.7:0/1782771072 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:13.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.644+0000 7f1b29400640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2406f300 0x7f1b240b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.644+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2103090847 0 0) 0x7f1b240b4980 con 0x7f1b240b0290 2024-04-25T11:19:13.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.644+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b0c003520 con 0x7f1b240b0290 2024-04-25T11:19:13.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2468249255 0 0) 0x7f1b0c003520 con 0x7f1b240b0290 2024-04-25T11:19:13.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2406f300 msgr2=0x7f1b240b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:13.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2406f300 0x7f1b240b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 >> v1:172.21.15.87:6789/0 conn(0x7f1b2406dc90 legacy=0x7f1b240afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b240b5220 con 0x7f1b240b0290 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1b10001670 con 0x7f1b240b0290 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 >> v1:172.21.15.7:6789/0 conn(0x7f1b240b0290 legacy=0x7f1b240b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b0c0039e0 0x7f1b0c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 0x7f1b0c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2406f300 0x7f1b0c0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b240b4f40 con 0x7f1b0c004800 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1b0c00bb50 con 0x7f1b0c0039e0 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1b0c00bd50 con 0x7f1b2406f300 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b23e00640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 0x7f1b0c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b28a00640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b0c0039e0 0x7f1b0c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.645+0000 7f1b29400640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2406f300 0x7f1b0c0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.646+0000 7f1b23e00640 1 -- 172.21.15.7:0/1782771072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2406f300 msgr2=0x7f1b0c0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:13.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.646+0000 7f1b23e00640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2406f300 0x7f1b0c0052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.646+0000 7f1b23e00640 1 -- 172.21.15.7:0/1782771072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b0c0039e0 msgr2=0x7f1b0c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.646+0000 7f1b23e00640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b0c0039e0 0x7f1b0c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.646+0000 7f1b23e00640 1 -- 172.21.15.7:0/1782771072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b0c003520 con 0x7f1b0c004800 2024-04-25T11:19:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.647+0000 7f1b23e00640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 0x7f1b0c004c10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c110 tx=0x7f1b0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad7bebda8fac74c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.647+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10007c40 con 0x7f1b0c004800 2024-04-25T11:19:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.647+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b10005af0 con 0x7f1b0c004800 2024-04-25T11:19:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.647+0000 7f1b23400640 1 -- 172.21.15.7:0/1782771072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10007070 con 0x7f1b0c004800 2024-04-25T11:19:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.648+0000 7f1b2b400640 1 -- 172.21.15.7:0/1782771072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 msgr2=0x7f1b0c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.648+0000 7f1b2b400640 1 --2- 172.21.15.7:0/1782771072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 0x7f1b0c004c10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000c110 tx=0x7f1b0c005830 comp rx=0 tx=0).stop 2024-04-25T11:19:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.648+0000 7f1b29400640 1 -- 172.21.15.7:0/1782771072 reap_dead start 2024-04-25T11:19:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.648+0000 7f1b2b400640 1 -- 172.21.15.7:0/1782771072 shutdown_connections 2024-04-25T11:19:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.648+0000 7f1b2b400640 1 -- 172.21.15.7:0/1782771072 >> 172.21.15.7:0/1782771072 conn(0x7f1b24074200 msgr2=0x7f1b2406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.648+0000 7f1b2b400640 1 -- 172.21.15.7:0/1782771072 shutdown_connections 2024-04-25T11:19:13.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.648+0000 7f1b2b400640 1 -- 172.21.15.7:0/1782771072 wait complete. 2024-04-25T11:19:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.650+0000 7f1b2b400640 1 Processor -- start 2024-04-25T11:19:13.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.651+0000 7f1b2b400640 1 -- start start 2024-04-25T11:19:13.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.652+0000 7f1b2b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 0x7f1b2415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.652+0000 7f1b2b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b24148df0 0x7f1b24149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.652+0000 7f1b2b400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b24153b80 0x7f1b24153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.652+0000 7f1b2b400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b240ce920 con 0x7f1b0c004800 2024-04-25T11:19:13.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.652+0000 7f1b2b400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2415d220 con 0x7f1b24148df0 2024-04-25T11:19:13.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.652+0000 7f1b2b400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1b240ceaa0 con 0x7f1b24153b80 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b28a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 0x7f1b2415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b28a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 0x7f1b2415ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51718/0 (socket says 172.21.15.7:51718) 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b28a00640 1 -- 172.21.15.7:0/4229055720 learned_addr learned my addr 172.21.15.7:0/4229055720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b23e00640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b24148df0 0x7f1b24149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b29400640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b24153b80 0x7f1b24153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b28a00640 1 -- 172.21.15.7:0/4229055720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b24153b80 msgr2=0x7f1b24153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b28a00640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b24153b80 0x7f1b24153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b28a00640 1 -- 172.21.15.7:0/4229055720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b24148df0 msgr2=0x7f1b24149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b28a00640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b24148df0 0x7f1b24149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b28a00640 1 -- 172.21.15.7:0/4229055720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b24149740 con 0x7f1b0c004800 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b29400640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b24153b80 0x7f1b24153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b23e00640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b24148df0 0x7f1b24149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.653+0000 7f1b28a00640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 0x7f1b2415ba50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800e470 tx=0x7f1b1800e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.654+0000 7f1b20c00640 1 -- 172.21.15.7:0/4229055720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b180053a0 con 0x7f1b0c004800 2024-04-25T11:19:13.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.654+0000 7f1b20c00640 1 -- 172.21.15.7:0/4229055720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b18018070 con 0x7f1b0c004800 2024-04-25T11:19:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.654+0000 7f1b2b400640 1 -- 172.21.15.7:0/4229055720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b24154ab0 con 0x7f1b0c004800 2024-04-25T11:19:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.654+0000 7f1b20c00640 1 -- 172.21.15.7:0/4229055720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18003830 con 0x7f1b0c004800 2024-04-25T11:19:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.654+0000 7f1b2b400640 1 -- 172.21.15.7:0/4229055720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b241573c0 con 0x7f1b0c004800 2024-04-25T11:19:13.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.655+0000 7f1b2b400640 1 -- 172.21.15.7:0/4229055720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2406a100 con 0x7f1b0c004800 2024-04-25T11:19:13.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.657+0000 7f1b20c00640 1 -- 172.21.15.7:0/4229055720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1b180039d0 con 0x7f1b0c004800 2024-04-25T11:19:13.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.658+0000 7f1b20c00640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b040744b0 0x7f1b040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:13.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.658+0000 7f1b20c00640 1 -- 172.21.15.7:0/4229055720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f1b18095070 con 0x7f1b0c004800 2024-04-25T11:19:13.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.659+0000 7f1b23e00640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b040744b0 0x7f1b040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:13.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.660+0000 7f1b23e00640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b040744b0 0x7f1b040769c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1b10032000 tx=0x7f1b10009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:13.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.664+0000 7f1b20c00640 1 -- 172.21.15.7:0/4229055720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1b1805f5d0 con 0x7f1b0c004800 2024-04-25T11:19:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.871+0000 7f1b2b400640 1 -- 172.21.15.7:0/4229055720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b24070fb0 con 0x7f1b0c004800 2024-04-25T11:19:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.873+0000 7f1b20c00640 1 -- 172.21.15.7:0/4229055720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16788 (secure 0 0 0) 0x7f1b1805eed0 con 0x7f1b0c004800 2024-04-25T11:19:13.872 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:10.691402+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:07.435764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:08.209216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.875+0000 7f1b2b400640 1 -- 172.21.15.7:0/4229055720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b040744b0 msgr2=0x7f1b040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.875+0000 7f1b2b400640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b040744b0 0x7f1b040769c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1b10032000 tx=0x7f1b10009030 comp rx=0 tx=0).stop 2024-04-25T11:19:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.875+0000 7f1b2b400640 1 -- 172.21.15.7:0/4229055720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 msgr2=0x7f1b2415ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.875+0000 7f1b2b400640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 0x7f1b2415ba50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800e470 tx=0x7f1b1800e930 comp rx=0 tx=0).stop 2024-04-25T11:19:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.876+0000 7f1b2b400640 1 -- 172.21.15.7:0/4229055720 shutdown_connections 2024-04-25T11:19:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.876+0000 7f1b2b400640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1b040744b0 0x7f1b040769c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.876+0000 7f1b2b400640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b24153b80 0x7f1b24153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.876+0000 7f1b2b400640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b24148df0 0x7f1b24149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.876+0000 7f1b2b400640 1 --2- 172.21.15.7:0/4229055720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b0c004800 0x7f1b2415ba50 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.876+0000 7f1b2b400640 1 -- 172.21.15.7:0/4229055720 >> 172.21.15.7:0/4229055720 conn(0x7f1b24074200 msgr2=0x7f1b240d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.876+0000 7f1b2b400640 1 -- 172.21.15.7:0/4229055720 shutdown_connections 2024-04-25T11:19:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:13.876+0000 7f1b2b400640 1 -- 172.21.15.7:0/4229055720 wait complete. 2024-04-25T11:19:13.902 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:19:13.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:19:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.181+0000 7fc31fc00640 1 Processor -- start 2024-04-25T11:19:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.181+0000 7fc31fc00640 1 -- start start 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.183+0000 7fc31fc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc31815de70 0x7fc318164640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.184+0000 7fc31fc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3180b6790 0x7fc3180b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.184+0000 7fc31fc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3180b0a50 0x7fc3180b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.184+0000 7fc31fc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc3180cdb70 con 0x7fc3180b6790 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.184+0000 7fc31fc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc3180aed30 con 0x7fc31815de70 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.184+0000 7fc31fc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc3180aea00 con 0x7fc3180b0a50 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.184+0000 7fc317e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3180b6790 0x7fc3180b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.184+0000 7fc317e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3180b6790 0x7fc3180b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51726/0 (socket says 172.21.15.7:51726) 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.184+0000 7fc317e00640 1 -- 172.21.15.7:0/581936925 learned_addr learned my addr 172.21.15.7:0/581936925 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.184+0000 7fc31d200640 1 --2- 172.21.15.7:0/581936925 >> v2:172.21.15.87:3300/0 conn(0x7fc31815de70 0x7fc318164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.184+0000 7fc31dc00640 1 --2- 172.21.15.7:0/581936925 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3180b0a50 0x7fc3180b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.185+0000 7fc317e00640 1 -- 172.21.15.7:0/581936925 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3180b0a50 msgr2=0x7fc3180b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.185+0000 7fc317e00640 1 --2- 172.21.15.7:0/581936925 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3180b0a50 0x7fc3180b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.185+0000 7fc317e00640 1 -- 172.21.15.7:0/581936925 >> v2:172.21.15.87:3300/0 conn(0x7fc31815de70 msgr2=0x7fc318164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.185+0000 7fc317e00640 1 --2- 172.21.15.7:0/581936925 >> v2:172.21.15.87:3300/0 conn(0x7fc31815de70 0x7fc318164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.185+0000 7fc317e00640 1 -- 172.21.15.7:0/581936925 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3180b13e0 con 0x7fc3180b6790 2024-04-25T11:19:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.186+0000 7fc317e00640 1 --2- 172.21.15.7:0/581936925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3180b6790 0x7fc3180b0510 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc3180727c0 tx=0x7fc30002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27299f539ecec4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.186+0000 7fc317400640 1 -- 172.21.15.7:0/581936925 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300037070 con 0x7fc3180b6790 2024-04-25T11:19:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.186+0000 7fc317400640 1 -- 172.21.15.7:0/581936925 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc300032440 con 0x7fc3180b6790 2024-04-25T11:19:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.186+0000 7fc317400640 1 -- 172.21.15.7:0/581936925 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300032810 con 0x7fc3180b6790 2024-04-25T11:19:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.187+0000 7fc31fc00640 1 -- 172.21.15.7:0/581936925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3180b6790 msgr2=0x7fc3180b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.187+0000 7fc31fc00640 1 --2- 172.21.15.7:0/581936925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3180b6790 0x7fc3180b0510 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fc3180727c0 tx=0x7fc30002a140 comp rx=0 tx=0).stop 2024-04-25T11:19:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.187+0000 7fc31fc00640 1 -- 172.21.15.7:0/581936925 shutdown_connections 2024-04-25T11:19:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.187+0000 7fc31fc00640 1 --2- 172.21.15.7:0/581936925 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3180b0a50 0x7fc3180b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.187+0000 7fc31fc00640 1 --2- 172.21.15.7:0/581936925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3180b6790 0x7fc3180b0510 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.187+0000 7fc31fc00640 1 --2- 172.21.15.7:0/581936925 >> v2:172.21.15.87:3300/0 conn(0x7fc31815de70 0x7fc318164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.187+0000 7fc31fc00640 1 -- 172.21.15.7:0/581936925 >> 172.21.15.7:0/581936925 conn(0x7fc318074200 msgr2=0x7fc31806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.188+0000 7fc31fc00640 1 -- 172.21.15.7:0/581936925 shutdown_connections 2024-04-25T11:19:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.188+0000 7fc31fc00640 1 -- 172.21.15.7:0/581936925 wait complete. 2024-04-25T11:19:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.190+0000 7fc31fc00640 1 Processor -- start 2024-04-25T11:19:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.190+0000 7fc31fc00640 1 -- start start 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.192+0000 7fc31fc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3180b0a50 0x7fc31815d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.192+0000 7fc31fc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3180b6790 0x7fc318155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.192+0000 7fc31fc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc31815de70 0x7fc318155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.192+0000 7fc31fc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3180b0e60 con 0x7fc3180b0a50 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.192+0000 7fc31fc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc3180ce0a0 con 0x7fc31815de70 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.192+0000 7fc31fc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc31806f300 con 0x7fc3180b6790 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.192+0000 7fc31d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3180b0a50 0x7fc31815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.192+0000 7fc31d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3180b0a50 0x7fc31815d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51734/0 (socket says 172.21.15.7:51734) 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.192+0000 7fc31d200640 1 -- 172.21.15.7:0/4197060583 learned_addr learned my addr 172.21.15.7:0/4197060583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.192+0000 7fc317e00640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3180b6790 0x7fc318155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.193+0000 7fc31dc00640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc31815de70 0x7fc318155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.193+0000 7fc31d200640 1 -- 172.21.15.7:0/4197060583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3180b6790 msgr2=0x7fc318155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.193+0000 7fc31d200640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3180b6790 0x7fc318155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.193+0000 7fc31d200640 1 -- 172.21.15.7:0/4197060583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc31815de70 msgr2=0x7fc318155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.193+0000 7fc31d200640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc31815de70 0x7fc318155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.193+0000 7fc31d200640 1 -- 172.21.15.7:0/4197060583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc318152810 con 0x7fc3180b0a50 2024-04-25T11:19:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.193+0000 7fc31d200640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3180b0a50 0x7fc31815d280 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc30800d5d0 tx=0x7fc30800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.193+0000 7fc314c00640 1 -- 172.21.15.7:0/4197060583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308004240 con 0x7fc3180b0a50 2024-04-25T11:19:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.193+0000 7fc314c00640 1 -- 172.21.15.7:0/4197060583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3080043e0 con 0x7fc3180b0a50 2024-04-25T11:19:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.193+0000 7fc314c00640 1 -- 172.21.15.7:0/4197060583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308003830 con 0x7fc3180b0a50 2024-04-25T11:19:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.193+0000 7fc31fc00640 1 -- 172.21.15.7:0/4197060583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc318153760 con 0x7fc3180b0a50 2024-04-25T11:19:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.194+0000 7fc31fc00640 1 -- 172.21.15.7:0/4197060583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3180cf2f0 con 0x7fc3180b0a50 2024-04-25T11:19:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.195+0000 7fc31fc00640 1 -- 172.21.15.7:0/4197060583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31806a100 con 0x7fc3180b0a50 2024-04-25T11:19:14.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.196+0000 7fc314c00640 1 -- 172.21.15.7:0/4197060583 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc3080039d0 con 0x7fc3180b0a50 2024-04-25T11:19:14.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.197+0000 7fc314c00640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc2f40744b0 0x7fc2f40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.197+0000 7fc314c00640 1 -- 172.21.15.7:0/4197060583 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fc308095540 con 0x7fc3180b0a50 2024-04-25T11:19:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.197+0000 7fc317e00640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc2f40744b0 0x7fc2f40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.198+0000 7fc317e00640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc2f40744b0 0x7fc2f40769c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc300034000 tx=0x7fc30002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.202+0000 7fc314c00640 1 -- 172.21.15.7:0/4197060583 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc30805fbc0 con 0x7fc3180b0a50 2024-04-25T11:19:14.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.405+0000 7fc31fc00640 1 -- 172.21.15.7:0/4197060583 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3180701e0 con 0x7fc2f40744b0 2024-04-25T11:19:14.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.427+0000 7fc314c00640 1 -- 172.21.15.7:0/4197060583 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252434 (secure 0 0 0) 0x7fc3180701e0 con 0x7fc2f40744b0 2024-04-25T11:19:14.430 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:14.431 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:19:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 -- 172.21.15.7:0/4197060583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc2f40744b0 msgr2=0x7fc2f40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc2f40744b0 0x7fc2f40769c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc300034000 tx=0x7fc30002e000 comp rx=0 tx=0).stop 2024-04-25T11:19:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 -- 172.21.15.7:0/4197060583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3180b0a50 msgr2=0x7fc31815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3180b0a50 0x7fc31815d280 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fc30800d5d0 tx=0x7fc30800da90 comp rx=0 tx=0).stop 2024-04-25T11:19:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 -- 172.21.15.7:0/4197060583 shutdown_connections 2024-04-25T11:19:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc2f40744b0 0x7fc2f40769c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc31815de70 0x7fc318155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3180b6790 0x7fc318155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 --2- 172.21.15.7:0/4197060583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3180b0a50 0x7fc31815d280 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 -- 172.21.15.7:0/4197060583 >> 172.21.15.7:0/4197060583 conn(0x7fc318074200 msgr2=0x7fc31815e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:14.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 -- 172.21.15.7:0/4197060583 shutdown_connections 2024-04-25T11:19:14.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.433+0000 7fc31fc00640 1 -- 172.21.15.7:0/4197060583 wait complete. 2024-04-25T11:19:14.463 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-04-25T11:19:13.996567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260764,"kb_used_data":11272,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713956,"statfs":{"total":773094113280,"available":772827090944,"internally_reserved":0,"allocated":11542528,"data_stored":5917370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001930"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.723919+0000","last_change":"2024-04-25T11:19:09.713598+0000","last_active":"2024-04-25T11:19:09.723919+0000","last_peered":"2024-04-25T11:19:09.723919+0000","last_clean":"2024-04-25T11:19:09.723919+0000","last_became_active":"2024-04-25T11:19:09.713098+0000","last_became_peered":"2024-04-25T11:19:09.713098+0000","last_unstale":"2024-04-25T11:19:09.723919+0000","last_undegraded":"2024-04-25T11:19:09.723919+0000","last_fullsized":"2024-04-25T11:19:09.723919+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:26.755621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.870024+0000","last_change":"2024-04-25T11:19:09.702747+0000","last_active":"2024-04-25T11:19:10.870024+0000","last_peered":"2024-04-25T11:19:10.870024+0000","last_clean":"2024-04-25T11:19:10.870024+0000","last_became_active":"2024-04-25T11:19:09.702454+0000","last_became_peered":"2024-04-25T11:19:09.702454+0000","last_unstale":"2024-04-25T11:19:10.870024+0000","last_undegraded":"2024-04-25T11:19:10.870024+0000","last_fullsized":"2024-04-25T11:19:10.870024+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:36.384892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.730240+0000","last_change":"2024-04-25T11:19:09.726556+0000","last_active":"2024-04-25T11:19:09.730240+0000","last_peered":"2024-04-25T11:19:09.730240+0000","last_clean":"2024-04-25T11:19:09.730240+0000","last_became_active":"2024-04-25T11:19:09.726247+0000","last_became_peered":"2024-04-25T11:19:09.726247+0000","last_unstale":"2024-04-25T11:19:09.730240+0000","last_undegraded":"2024-04-25T11:19:09.730240+0000","last_fullsized":"2024-04-25T11:19:09.730240+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:48.612468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.719957+0000","last_change":"2024-04-25T11:19:09.712961+0000","last_active":"2024-04-25T11:19:09.719957+0000","last_peered":"2024-04-25T11:19:09.719957+0000","last_clean":"2024-04-25T11:19:09.719957+0000","last_became_active":"2024-04-25T11:19:09.712468+0000","last_became_peered":"2024-04-25T11:19:09.712468+0000","last_unstale":"2024-04-25T11:19:09.719957+0000","last_undegraded":"2024-04-25T11:19:09.719957+0000","last_fullsized":"2024-04-25T11:19:09.719957+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.992694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724695+0000","last_change":"2024-04-25T11:19:09.707134+0000","last_active":"2024-04-25T11:19:09.724695+0000","last_peered":"2024-04-25T11:19:09.724695+0000","last_clean":"2024-04-25T11:19:09.724695+0000","last_became_active":"2024-04-25T11:19:09.706819+0000","last_became_peered":"2024-04-25T11:19:09.706819+0000","last_unstale":"2024-04-25T11:19:09.724695+0000","last_undegraded":"2024-04-25T11:19:09.724695+0000","last_fullsized":"2024-04-25T11:19:09.724695+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:03.370541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.868824+0000","last_change":"2024-04-25T11:19:09.707059+0000","last_active":"2024-04-25T11:19:10.868824+0000","last_peered":"2024-04-25T11:19:10.868824+0000","last_clean":"2024-04-25T11:19:10.868824+0000","last_became_active":"2024-04-25T11:19:09.705959+0000","last_became_peered":"2024-04-25T11:19:09.705959+0000","last_unstale":"2024-04-25T11:19:10.868824+0000","last_undegraded":"2024-04-25T11:19:10.868824+0000","last_fullsized":"2024-04-25T11:19:10.868824+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:00.515175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.868753+0000","last_change":"2024-04-25T11:19:09.706669+0000","last_active":"2024-04-25T11:19:10.868753+0000","last_peered":"2024-04-25T11:19:10.868753+0000","last_clean":"2024-04-25T11:19:10.868753+0000","last_became_active":"2024-04-25T11:19:09.706243+0000","last_became_peered":"2024-04-25T11:19:09.706243+0000","last_unstale":"2024-04-25T11:19:10.868753+0000","last_undegraded":"2024-04-25T11:19:10.868753+0000","last_fullsized":"2024-04-25T11:19:10.868753+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:52.054644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016509+0000","last_change":"2024-04-25T11:19:09.711279+0000","last_active":"2024-04-25T11:19:11.016509+0000","last_peered":"2024-04-25T11:19:11.016509+0000","last_clean":"2024-04-25T11:19:11.016509+0000","last_became_active":"2024-04-25T11:19:09.710912+0000","last_became_peered":"2024-04-25T11:19:09.710912+0000","last_unstale":"2024-04-25T11:19:11.016509+0000","last_undegraded":"2024-04-25T11:19:11.016509+0000","last_fullsized":"2024-04-25T11:19:11.016509+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:27.095256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.870140+0000","last_change":"2024-04-25T11:19:09.710225+0000","last_active":"2024-04-25T11:19:10.870140+0000","last_peered":"2024-04-25T11:19:10.870140+0000","last_clean":"2024-04-25T11:19:10.870140+0000","last_became_active":"2024-04-25T11:19:09.709881+0000","last_became_peered":"2024-04-25T11:19:09.709881+0000","last_unstale":"2024-04-25T11:19:10.870140+0000","last_undegraded":"2024-04-25T11:19:10.870140+0000","last_fullsized":"2024-04-25T11:19:10.870140+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:55.241977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.719748+0000","last_change":"2024-04-25T11:19:09.712962+0000","last_active":"2024-04-25T11:19:09.719748+0000","last_peered":"2024-04-25T11:19:09.719748+0000","last_clean":"2024-04-25T11:19:09.719748+0000","last_became_active":"2024-04-25T11:19:09.712638+0000","last_became_peered":"2024-04-25T11:19:09.712638+0000","last_unstale":"2024-04-25T11:19:09.719748+0000","last_undegraded":"2024-04-25T11:19:09.719748+0000","last_fullsized":"2024-04-25T11:19:09.719748+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:29.719599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.726057+0000","last_change":"2024-04-25T11:19:09.722604+0000","last_active":"2024-04-25T11:19:09.726057+0000","last_peered":"2024-04-25T11:19:09.726057+0000","last_clean":"2024-04-25T11:19:09.726057+0000","last_became_active":"2024-04-25T11:19:09.721158+0000","last_became_peered":"2024-04-25T11:19:09.721158+0000","last_unstale":"2024-04-25T11:19:09.726057+0000","last_undegraded":"2024-04-25T11:19:09.726057+0000","last_fullsized":"2024-04-25T11:19:09.726057+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:24.997412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.722053+0000","last_change":"2024-04-25T11:19:09.712564+0000","last_active":"2024-04-25T11:19:10.722053+0000","last_peered":"2024-04-25T11:19:10.722053+0000","last_clean":"2024-04-25T11:19:10.722053+0000","last_became_active":"2024-04-25T11:19:09.711650+0000","last_became_peered":"2024-04-25T11:19:09.711650+0000","last_unstale":"2024-04-25T11:19:10.722053+0000","last_undegraded":"2024-04-25T11:19:10.722053+0000","last_fullsized":"2024-04-25T11:19:10.722053+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:10.756423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.728273+0000","last_change":"2024-04-25T11:19:09.726015+0000","last_active":"2024-04-25T11:19:09.728273+0000","last_peered":"2024-04-25T11:19:09.728273+0000","last_clean":"2024-04-25T11:19:09.728273+0000","last_became_active":"2024-04-25T11:19:09.724929+0000","last_became_peered":"2024-04-25T11:19:09.724929+0000","last_unstale":"2024-04-25T11:19:09.728273+0000","last_undegraded":"2024-04-25T11:19:09.728273+0000","last_fullsized":"2024-04-25T11:19:09.728273+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:05.665209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.716723+0000","last_change":"2024-04-25T11:19:09.712734+0000","last_active":"2024-04-25T11:19:09.716723+0000","last_peered":"2024-04-25T11:19:09.716723+0000","last_clean":"2024-04-25T11:19:09.716723+0000","last_became_active":"2024-04-25T11:19:09.712376+0000","last_became_peered":"2024-04-25T11:19:09.712376+0000","last_unstale":"2024-04-25T11:19:09.716723+0000","last_undegraded":"2024-04-25T11:19:09.716723+0000","last_fullsized":"2024-04-25T11:19:09.716723+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:54.334703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.722010+0000","last_change":"2024-04-25T11:19:09.718928+0000","last_active":"2024-04-25T11:19:09.722010+0000","last_peered":"2024-04-25T11:19:09.722010+0000","last_clean":"2024-04-25T11:19:09.722010+0000","last_became_active":"2024-04-25T11:19:09.718615+0000","last_became_peered":"2024-04-25T11:19:09.718615+0000","last_unstale":"2024-04-25T11:19:09.722010+0000","last_undegraded":"2024-04-25T11:19:09.722010+0000","last_fullsized":"2024-04-25T11:19:09.722010+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:03.964007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.705551+0000","last_change":"2024-04-25T11:19:09.700355+0000","last_active":"2024-04-25T11:19:09.705551+0000","last_peered":"2024-04-25T11:19:09.705551+0000","last_clean":"2024-04-25T11:19:09.705551+0000","last_became_active":"2024-04-25T11:19:09.699968+0000","last_became_peered":"2024-04-25T11:19:09.699968+0000","last_unstale":"2024-04-25T11:19:09.705551+0000","last_undegraded":"2024-04-25T11:19:09.705551+0000","last_fullsized":"2024-04-25T11:19:09.705551+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:29.649252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.730044+0000","last_change":"2024-04-25T11:19:09.727011+0000","last_active":"2024-04-25T11:19:09.730044+0000","last_peered":"2024-04-25T11:19:09.730044+0000","last_clean":"2024-04-25T11:19:09.730044+0000","last_became_active":"2024-04-25T11:19:09.726648+0000","last_became_peered":"2024-04-25T11:19:09.726648+0000","last_unstale":"2024-04-25T11:19:09.730044+0000","last_undegraded":"2024-04-25T11:19:09.730044+0000","last_fullsized":"2024-04-25T11:19:09.730044+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:59.653285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.870301+0000","last_change":"2024-04-25T11:19:09.703842+0000","last_active":"2024-04-25T11:19:10.870301+0000","last_peered":"2024-04-25T11:19:10.870301+0000","last_clean":"2024-04-25T11:19:10.870301+0000","last_became_active":"2024-04-25T11:19:09.703499+0000","last_became_peered":"2024-04-25T11:19:09.703499+0000","last_unstale":"2024-04-25T11:19:10.870301+0000","last_undegraded":"2024-04-25T11:19:10.870301+0000","last_fullsized":"2024-04-25T11:19:10.870301+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:55.796067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727603+0000","last_change":"2024-04-25T11:19:09.721422+0000","last_active":"2024-04-25T11:19:09.727603+0000","last_peered":"2024-04-25T11:19:09.727603+0000","last_clean":"2024-04-25T11:19:09.727603+0000","last_became_active":"2024-04-25T11:19:09.721035+0000","last_became_peered":"2024-04-25T11:19:09.721035+0000","last_unstale":"2024-04-25T11:19:09.727603+0000","last_undegraded":"2024-04-25T11:19:09.727603+0000","last_fullsized":"2024-04-25T11:19:09.727603+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:01.467633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.722153+0000","last_change":"2024-04-25T11:19:09.717947+0000","last_active":"2024-04-25T11:19:10.722153+0000","last_peered":"2024-04-25T11:19:10.722153+0000","last_clean":"2024-04-25T11:19:10.722153+0000","last_became_active":"2024-04-25T11:19:09.716428+0000","last_became_peered":"2024-04-25T11:19:09.716428+0000","last_unstale":"2024-04-25T11:19:10.722153+0000","last_undegraded":"2024-04-25T11:19:10.722153+0000","last_fullsized":"2024-04-25T11:19:10.722153+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.992694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017828+0000","last_change":"2024-04-25T11:19:09.718569+0000","last_active":"2024-04-25T11:19:11.017828+0000","last_peered":"2024-04-25T11:19:11.017828+0000","last_clean":"2024-04-25T11:19:11.017828+0000","last_became_active":"2024-04-25T11:19:09.718087+0000","last_became_peered":"2024-04-25T11:19:09.718087+0000","last_unstale":"2024-04-25T11:19:11.017828+0000","last_undegraded":"2024-04-25T11:19:11.017828+0000","last_fullsized":"2024-04-25T11:19:11.017828+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:08.743237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.730115+0000","last_change":"2024-04-25T11:19:09.724581+0000","last_active":"2024-04-25T11:19:09.730115+0000","last_peered":"2024-04-25T11:19:09.730115+0000","last_clean":"2024-04-25T11:19:09.730115+0000","last_became_active":"2024-04-25T11:19:09.720794+0000","last_became_peered":"2024-04-25T11:19:09.720794+0000","last_unstale":"2024-04-25T11:19:09.730115+0000","last_undegraded":"2024-04-25T11:19:09.730115+0000","last_fullsized":"2024-04-25T11:19:09.730115+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:52.626259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.731035+0000","last_change":"2024-04-25T11:19:09.728777+0000","last_active":"2024-04-25T11:19:09.731035+0000","last_peered":"2024-04-25T11:19:09.731035+0000","last_clean":"2024-04-25T11:19:09.731035+0000","last_became_active":"2024-04-25T11:19:09.728206+0000","last_became_peered":"2024-04-25T11:19:09.728206+0000","last_unstale":"2024-04-25T11:19:09.731035+0000","last_undegraded":"2024-04-25T11:19:09.731035+0000","last_fullsized":"2024-04-25T11:19:09.731035+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:06.243770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724468+0000","last_change":"2024-04-25T11:19:09.710323+0000","last_active":"2024-04-25T11:19:09.724468+0000","last_peered":"2024-04-25T11:19:09.724468+0000","last_clean":"2024-04-25T11:19:09.724468+0000","last_became_active":"2024-04-25T11:19:09.709992+0000","last_became_peered":"2024-04-25T11:19:09.709992+0000","last_unstale":"2024-04-25T11:19:09.724468+0000","last_undegraded":"2024-04-25T11:19:09.724468+0000","last_fullsized":"2024-04-25T11:19:09.724468+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:00.961608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.721947+0000","last_change":"2024-04-25T11:19:09.727646+0000","last_active":"2024-04-25T11:19:10.721947+0000","last_peered":"2024-04-25T11:19:10.721947+0000","last_clean":"2024-04-25T11:19:10.721947+0000","last_became_active":"2024-04-25T11:19:09.727136+0000","last_became_peered":"2024-04-25T11:19:09.727136+0000","last_unstale":"2024-04-25T11:19:10.721947+0000","last_undegraded":"2024-04-25T11:19:10.721947+0000","last_fullsized":"2024-04-25T11:19:10.721947+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:00.099157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.721638+0000","last_change":"2024-04-25T11:19:09.723571+0000","last_active":"2024-04-25T11:19:10.721638+0000","last_peered":"2024-04-25T11:19:10.721638+0000","last_clean":"2024-04-25T11:19:10.721638+0000","last_became_active":"2024-04-25T11:19:09.721892+0000","last_became_peered":"2024-04-25T11:19:09.721892+0000","last_unstale":"2024-04-25T11:19:10.721638+0000","last_undegraded":"2024-04-25T11:19:10.721638+0000","last_fullsized":"2024-04-25T11:19:10.721638+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:00.515175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.723926+0000","last_change":"2024-04-25T11:19:09.716265+0000","last_active":"2024-04-25T11:19:09.723926+0000","last_peered":"2024-04-25T11:19:09.723926+0000","last_clean":"2024-04-25T11:19:09.723926+0000","last_became_active":"2024-04-25T11:19:09.715853+0000","last_became_peered":"2024-04-25T11:19:09.715853+0000","last_unstale":"2024-04-25T11:19:09.723926+0000","last_undegraded":"2024-04-25T11:19:09.723926+0000","last_fullsized":"2024-04-25T11:19:09.723926+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:29.719599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.869032+0000","last_change":"2024-04-25T11:19:09.722428+0000","last_active":"2024-04-25T11:19:10.869032+0000","last_peered":"2024-04-25T11:19:10.869032+0000","last_clean":"2024-04-25T11:19:10.869032+0000","last_became_active":"2024-04-25T11:19:09.722102+0000","last_became_peered":"2024-04-25T11:19:09.722102+0000","last_unstale":"2024-04-25T11:19:10.869032+0000","last_undegraded":"2024-04-25T11:19:10.869032+0000","last_fullsized":"2024-04-25T11:19:10.869032+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:10.756423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.018003+0000","last_change":"2024-04-25T11:19:09.704082+0000","last_active":"2024-04-25T11:19:11.018003+0000","last_peered":"2024-04-25T11:19:11.018003+0000","last_clean":"2024-04-25T11:19:11.018003+0000","last_became_active":"2024-04-25T11:19:09.703648+0000","last_became_peered":"2024-04-25T11:19:09.703648+0000","last_unstale":"2024-04-25T11:19:11.018003+0000","last_undegraded":"2024-04-25T11:19:11.018003+0000","last_fullsized":"2024-04-25T11:19:11.018003+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:17.941867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.721262+0000","last_change":"2024-04-25T11:19:09.727980+0000","last_active":"2024-04-25T11:19:10.721262+0000","last_peered":"2024-04-25T11:19:10.721262+0000","last_clean":"2024-04-25T11:19:10.721262+0000","last_became_active":"2024-04-25T11:19:09.727594+0000","last_became_peered":"2024-04-25T11:19:09.727594+0000","last_unstale":"2024-04-25T11:19:10.721262+0000","last_undegraded":"2024-04-25T11:19:10.721262+0000","last_fullsized":"2024-04-25T11:19:10.721262+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:14.566202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.868605+0000","last_change":"2024-04-25T11:19:09.705190+0000","last_active":"2024-04-25T11:19:10.868605+0000","last_peered":"2024-04-25T11:19:10.868605+0000","last_clean":"2024-04-25T11:19:10.868605+0000","last_became_active":"2024-04-25T11:19:09.704854+0000","last_became_peered":"2024-04-25T11:19:09.704854+0000","last_unstale":"2024-04-25T11:19:10.868605+0000","last_undegraded":"2024-04-25T11:19:10.868605+0000","last_fullsized":"2024-04-25T11:19:10.868605+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:14.566202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016479+0000","last_change":"2024-04-25T11:19:09.717396+0000","last_active":"2024-04-25T11:19:11.016479+0000","last_peered":"2024-04-25T11:19:11.016479+0000","last_clean":"2024-04-25T11:19:11.016479+0000","last_became_active":"2024-04-25T11:19:09.717097+0000","last_became_peered":"2024-04-25T11:19:09.717097+0000","last_unstale":"2024-04-25T11:19:11.016479+0000","last_undegraded":"2024-04-25T11:19:11.016479+0000","last_fullsized":"2024-04-25T11:19:11.016479+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:49.344998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727414+0000","last_change":"2024-04-25T11:19:09.720502+0000","last_active":"2024-04-25T11:19:09.727414+0000","last_peered":"2024-04-25T11:19:09.727414+0000","last_clean":"2024-04-25T11:19:09.727414+0000","last_became_active":"2024-04-25T11:19:09.720161+0000","last_became_peered":"2024-04-25T11:19:09.720161+0000","last_unstale":"2024-04-25T11:19:09.727414+0000","last_undegraded":"2024-04-25T11:19:09.727414+0000","last_fullsized":"2024-04-25T11:19:09.727414+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:18.203565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.722799+0000","last_change":"2024-04-25T11:19:08.712020+0000","last_active":"2024-04-25T11:19:09.722799+0000","last_peered":"2024-04-25T11:19:09.722799+0000","last_clean":"2024-04-25T11:19:09.722799+0000","last_became_active":"2024-04-25T11:19:08.711698+0000","last_became_peered":"2024-04-25T11:19:08.711698+0000","last_unstale":"2024-04-25T11:19:09.722799+0000","last_undegraded":"2024-04-25T11:19:09.722799+0000","last_fullsized":"2024-04-25T11:19:09.722799+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:09.736832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016792+0000","last_change":"2024-04-25T11:19:09.716136+0000","last_active":"2024-04-25T11:19:11.016792+0000","last_peered":"2024-04-25T11:19:11.016792+0000","last_clean":"2024-04-25T11:19:11.016792+0000","last_became_active":"2024-04-25T11:19:09.715126+0000","last_became_peered":"2024-04-25T11:19:09.715126+0000","last_unstale":"2024-04-25T11:19:11.016792+0000","last_undegraded":"2024-04-25T11:19:11.016792+0000","last_fullsized":"2024-04-25T11:19:11.016792+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:03.060032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017545+0000","last_change":"2024-04-25T11:19:08.710525+0000","last_active":"2024-04-25T11:19:11.017545+0000","last_peered":"2024-04-25T11:19:11.017545+0000","last_clean":"2024-04-25T11:19:11.017545+0000","last_became_active":"2024-04-25T11:19:08.710180+0000","last_became_peered":"2024-04-25T11:19:08.710180+0000","last_unstale":"2024-04-25T11:19:11.017545+0000","last_undegraded":"2024-04-25T11:19:11.017545+0000","last_fullsized":"2024-04-25T11:19:11.017545+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:04.053134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.721603+0000","last_change":"2024-04-25T11:19:09.730212+0000","last_active":"2024-04-25T11:19:10.721603+0000","last_peered":"2024-04-25T11:19:10.721603+0000","last_clean":"2024-04-25T11:19:10.721603+0000","last_became_active":"2024-04-25T11:19:09.729741+0000","last_became_peered":"2024-04-25T11:19:09.729741+0000","last_unstale":"2024-04-25T11:19:10.721603+0000","last_undegraded":"2024-04-25T11:19:10.721603+0000","last_fullsized":"2024-04-25T11:19:10.721603+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:52.054644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.720824+0000","last_change":"2024-04-25T11:19:08.714470+0000","last_active":"2024-04-25T11:19:10.720824+0000","last_peered":"2024-04-25T11:19:10.720824+0000","last_clean":"2024-04-25T11:19:10.720824+0000","last_became_active":"2024-04-25T11:19:08.713751+0000","last_became_peered":"2024-04-25T11:19:08.713751+0000","last_unstale":"2024-04-25T11:19:10.720824+0000","last_undegraded":"2024-04-25T11:19:10.720824+0000","last_fullsized":"2024-04-25T11:19:10.720824+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:42.107371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.726242+0000","last_change":"2024-04-25T11:19:09.722954+0000","last_active":"2024-04-25T11:19:09.726242+0000","last_peered":"2024-04-25T11:19:09.726242+0000","last_clean":"2024-04-25T11:19:09.726242+0000","last_became_active":"2024-04-25T11:19:09.721378+0000","last_became_peered":"2024-04-25T11:19:09.721378+0000","last_unstale":"2024-04-25T11:19:09.726242+0000","last_undegraded":"2024-04-25T11:19:09.726242+0000","last_fullsized":"2024-04-25T11:19:09.726242+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:17.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.728762+0000","last_change":"2024-04-25T11:19:08.700455+0000","last_active":"2024-04-25T11:19:09.728762+0000","last_peered":"2024-04-25T11:19:09.728762+0000","last_clean":"2024-04-25T11:19:09.728762+0000","last_became_active":"2024-04-25T11:19:08.700082+0000","last_became_peered":"2024-04-25T11:19:08.700082+0000","last_unstale":"2024-04-25T11:19:09.728762+0000","last_undegraded":"2024-04-25T11:19:09.728762+0000","last_fullsized":"2024-04-25T11:19:09.728762+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:25.169079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727999+0000","last_change":"2024-04-25T11:19:09.725542+0000","last_active":"2024-04-25T11:19:09.727999+0000","last_peered":"2024-04-25T11:19:09.727999+0000","last_clean":"2024-04-25T11:19:09.727999+0000","last_became_active":"2024-04-25T11:19:09.725230+0000","last_became_peered":"2024-04-25T11:19:09.725230+0000","last_unstale":"2024-04-25T11:19:09.727999+0000","last_undegraded":"2024-04-25T11:19:09.727999+0000","last_fullsized":"2024-04-25T11:19:09.727999+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:22.367292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.720357+0000","last_change":"2024-04-25T11:19:08.711289+0000","last_active":"2024-04-25T11:19:09.720357+0000","last_peered":"2024-04-25T11:19:09.720357+0000","last_clean":"2024-04-25T11:19:09.720357+0000","last_became_active":"2024-04-25T11:19:08.710956+0000","last_became_peered":"2024-04-25T11:19:08.710956+0000","last_unstale":"2024-04-25T11:19:09.720357+0000","last_undegraded":"2024-04-25T11:19:09.720357+0000","last_fullsized":"2024-04-25T11:19:09.720357+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:21.347701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727631+0000","last_change":"2024-04-25T11:19:09.721805+0000","last_active":"2024-04-25T11:19:09.727631+0000","last_peered":"2024-04-25T11:19:09.727631+0000","last_clean":"2024-04-25T11:19:09.727631+0000","last_became_active":"2024-04-25T11:19:09.721040+0000","last_became_peered":"2024-04-25T11:19:09.721040+0000","last_unstale":"2024-04-25T11:19:09.727631+0000","last_undegraded":"2024-04-25T11:19:09.727631+0000","last_fullsized":"2024-04-25T11:19:09.727631+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:45.937225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724004+0000","last_change":"2024-04-25T11:19:08.706544+0000","last_active":"2024-04-25T11:19:09.724004+0000","last_peered":"2024-04-25T11:19:09.724004+0000","last_clean":"2024-04-25T11:19:09.724004+0000","last_became_active":"2024-04-25T11:19:08.706160+0000","last_became_peered":"2024-04-25T11:19:08.706160+0000","last_unstale":"2024-04-25T11:19:09.724004+0000","last_undegraded":"2024-04-25T11:19:09.724004+0000","last_fullsized":"2024-04-25T11:19:09.724004+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:02.350950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.870325+0000","last_change":"2024-04-25T11:19:09.724104+0000","last_active":"2024-04-25T11:19:10.870325+0000","last_peered":"2024-04-25T11:19:10.870325+0000","last_clean":"2024-04-25T11:19:10.870325+0000","last_became_active":"2024-04-25T11:19:09.723101+0000","last_became_peered":"2024-04-25T11:19:09.723101+0000","last_unstale":"2024-04-25T11:19:10.870325+0000","last_undegraded":"2024-04-25T11:19:10.870325+0000","last_fullsized":"2024-04-25T11:19:10.870325+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:22.367292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.720880+0000","last_change":"2024-04-25T11:19:08.708941+0000","last_active":"2024-04-25T11:19:10.720880+0000","last_peered":"2024-04-25T11:19:10.720880+0000","last_clean":"2024-04-25T11:19:10.720880+0000","last_became_active":"2024-04-25T11:19:08.706905+0000","last_became_peered":"2024-04-25T11:19:08.706905+0000","last_unstale":"2024-04-25T11:19:10.720880+0000","last_undegraded":"2024-04-25T11:19:10.720880+0000","last_fullsized":"2024-04-25T11:19:10.720880+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:02.350950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.725199+0000","last_change":"2024-04-25T11:19:09.723854+0000","last_active":"2024-04-25T11:19:09.725199+0000","last_peered":"2024-04-25T11:19:09.725199+0000","last_clean":"2024-04-25T11:19:09.725199+0000","last_became_active":"2024-04-25T11:19:09.723505+0000","last_became_peered":"2024-04-25T11:19:09.723505+0000","last_unstale":"2024-04-25T11:19:09.725199+0000","last_undegraded":"2024-04-25T11:19:09.725199+0000","last_fullsized":"2024-04-25T11:19:09.725199+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:55.796067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.704222+0000","last_change":"2024-04-25T11:19:08.698071+0000","last_active":"2024-04-25T11:19:09.704222+0000","last_peered":"2024-04-25T11:19:09.704222+0000","last_clean":"2024-04-25T11:19:09.704222+0000","last_became_active":"2024-04-25T11:19:08.697726+0000","last_became_peered":"2024-04-25T11:19:08.697726+0000","last_unstale":"2024-04-25T11:19:09.704222+0000","last_undegraded":"2024-04-25T11:19:09.704222+0000","last_fullsized":"2024-04-25T11:19:09.704222+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:59.495584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727442+0000","last_change":"2024-04-25T11:19:09.720710+0000","last_active":"2024-04-25T11:19:09.727442+0000","last_peered":"2024-04-25T11:19:09.727442+0000","last_clean":"2024-04-25T11:19:09.727442+0000","last_became_active":"2024-04-25T11:19:09.720352+0000","last_became_peered":"2024-04-25T11:19:09.720352+0000","last_unstale":"2024-04-25T11:19:09.727442+0000","last_undegraded":"2024-04-25T11:19:09.727442+0000","last_fullsized":"2024-04-25T11:19:09.727442+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:22.639028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.722029+0000","last_change":"2024-04-25T11:19:08.708614+0000","last_active":"2024-04-25T11:19:10.722029+0000","last_peered":"2024-04-25T11:19:10.722029+0000","last_clean":"2024-04-25T11:19:10.722029+0000","last_became_active":"2024-04-25T11:19:08.707687+0000","last_became_peered":"2024-04-25T11:19:08.707687+0000","last_unstale":"2024-04-25T11:19:10.722029+0000","last_undegraded":"2024-04-25T11:19:10.722029+0000","last_fullsized":"2024-04-25T11:19:10.722029+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:37.873967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.715984+0000","last_change":"2024-04-25T11:19:09.703672+0000","last_active":"2024-04-25T11:19:09.715984+0000","last_peered":"2024-04-25T11:19:09.715984+0000","last_clean":"2024-04-25T11:19:09.715984+0000","last_became_active":"2024-04-25T11:19:09.703220+0000","last_became_peered":"2024-04-25T11:19:09.703220+0000","last_unstale":"2024-04-25T11:19:09.715984+0000","last_undegraded":"2024-04-25T11:19:09.715984+0000","last_fullsized":"2024-04-25T11:19:09.715984+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:46.676680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.707897+0000","last_change":"2024-04-25T11:19:08.708515+0000","last_active":"2024-04-25T11:19:09.707897+0000","last_peered":"2024-04-25T11:19:09.707897+0000","last_clean":"2024-04-25T11:19:09.707897+0000","last_became_active":"2024-04-25T11:19:08.708238+0000","last_became_peered":"2024-04-25T11:19:08.708238+0000","last_unstale":"2024-04-25T11:19:09.707897+0000","last_undegraded":"2024-04-25T11:19:09.707897+0000","last_fullsized":"2024-04-25T11:19:09.707897+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:19.821060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017177+0000","last_change":"2024-04-25T11:19:09.715409+0000","last_active":"2024-04-25T11:19:11.017177+0000","last_peered":"2024-04-25T11:19:11.017177+0000","last_clean":"2024-04-25T11:19:11.017177+0000","last_became_active":"2024-04-25T11:19:09.715104+0000","last_became_peered":"2024-04-25T11:19:09.715104+0000","last_unstale":"2024-04-25T11:19:11.017177+0000","last_undegraded":"2024-04-25T11:19:11.017177+0000","last_fullsized":"2024-04-25T11:19:11.017177+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:22.848256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.729545+0000","last_change":"2024-04-25T11:19:08.715017+0000","last_active":"2024-04-25T11:19:09.729545+0000","last_peered":"2024-04-25T11:19:09.729545+0000","last_clean":"2024-04-25T11:19:09.729545+0000","last_became_active":"2024-04-25T11:19:08.714696+0000","last_became_peered":"2024-04-25T11:19:08.714696+0000","last_unstale":"2024-04-25T11:19:09.729545+0000","last_undegraded":"2024-04-25T11:19:09.729545+0000","last_fullsized":"2024-04-25T11:19:09.729545+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:54.776476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.716583+0000","last_change":"2024-04-25T11:19:09.703394+0000","last_active":"2024-04-25T11:19:09.716583+0000","last_peered":"2024-04-25T11:19:09.716583+0000","last_clean":"2024-04-25T11:19:09.716583+0000","last_became_active":"2024-04-25T11:19:09.702476+0000","last_became_peered":"2024-04-25T11:19:09.702476+0000","last_unstale":"2024-04-25T11:19:09.716583+0000","last_undegraded":"2024-04-25T11:19:09.716583+0000","last_fullsized":"2024-04-25T11:19:09.716583+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:56.814924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.869387+0000","last_change":"2024-04-25T11:19:08.714074+0000","last_active":"2024-04-25T11:19:10.869387+0000","last_peered":"2024-04-25T11:19:10.869387+0000","last_clean":"2024-04-25T11:19:10.869387+0000","last_became_active":"2024-04-25T11:19:08.713485+0000","last_became_peered":"2024-04-25T11:19:08.713485+0000","last_unstale":"2024-04-25T11:19:10.869387+0000","last_undegraded":"2024-04-25T11:19:10.869387+0000","last_fullsized":"2024-04-25T11:19:10.869387+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:25.169079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.726002+0000","last_change":"2024-04-25T11:19:09.720431+0000","last_active":"2024-04-25T11:19:09.726002+0000","last_peered":"2024-04-25T11:19:09.726002+0000","last_clean":"2024-04-25T11:19:09.726002+0000","last_became_active":"2024-04-25T11:19:09.720097+0000","last_became_peered":"2024-04-25T11:19:09.720097+0000","last_unstale":"2024-04-25T11:19:09.726002+0000","last_undegraded":"2024-04-25T11:19:09.726002+0000","last_fullsized":"2024-04-25T11:19:09.726002+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:20.924888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.729392+0000","last_change":"2024-04-25T11:19:08.707366+0000","last_active":"2024-04-25T11:19:09.729392+0000","last_peered":"2024-04-25T11:19:09.729392+0000","last_clean":"2024-04-25T11:19:09.729392+0000","last_became_active":"2024-04-25T11:19:08.707047+0000","last_became_peered":"2024-04-25T11:19:08.707047+0000","last_unstale":"2024-04-25T11:19:09.729392+0000","last_undegraded":"2024-04-25T11:19:09.729392+0000","last_fullsized":"2024-04-25T11:19:09.729392+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:09.125332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.869851+0000","last_change":"2024-04-25T11:19:09.714775+0000","last_active":"2024-04-25T11:19:10.869851+0000","last_peered":"2024-04-25T11:19:10.869851+0000","last_clean":"2024-04-25T11:19:10.869851+0000","last_became_active":"2024-04-25T11:19:09.711665+0000","last_became_peered":"2024-04-25T11:19:09.711665+0000","last_unstale":"2024-04-25T11:19:10.869851+0000","last_undegraded":"2024-04-25T11:19:10.869851+0000","last_fullsized":"2024-04-25T11:19:10.869851+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.992694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.706985+0000","last_change":"2024-04-25T11:19:08.699583+0000","last_active":"2024-04-25T11:19:09.706985+0000","last_peered":"2024-04-25T11:19:09.706985+0000","last_clean":"2024-04-25T11:19:09.706985+0000","last_became_active":"2024-04-25T11:19:08.697764+0000","last_became_peered":"2024-04-25T11:19:08.697764+0000","last_unstale":"2024-04-25T11:19:09.706985+0000","last_undegraded":"2024-04-25T11:19:09.706985+0000","last_fullsized":"2024-04-25T11:19:09.706985+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:59.079566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016924+0000","last_change":"2024-04-25T11:19:09.713065+0000","last_active":"2024-04-25T11:19:11.016924+0000","last_peered":"2024-04-25T11:19:11.016924+0000","last_clean":"2024-04-25T11:19:11.016924+0000","last_became_active":"2024-04-25T11:19:09.712730+0000","last_became_peered":"2024-04-25T11:19:09.712730+0000","last_unstale":"2024-04-25T11:19:11.016924+0000","last_undegraded":"2024-04-25T11:19:11.016924+0000","last_fullsized":"2024-04-25T11:19:11.016924+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:11.612910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017278+0000","last_change":"2024-04-25T11:19:08.704569+0000","last_active":"2024-04-25T11:19:11.017278+0000","last_peered":"2024-04-25T11:19:11.017278+0000","last_clean":"2024-04-25T11:19:11.017278+0000","last_became_active":"2024-04-25T11:19:08.704204+0000","last_became_peered":"2024-04-25T11:19:08.704204+0000","last_unstale":"2024-04-25T11:19:11.017278+0000","last_undegraded":"2024-04-25T11:19:11.017278+0000","last_fullsized":"2024-04-25T11:19:11.017278+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:12.947432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.730018+0000","last_change":"2024-04-25T11:19:09.723317+0000","last_active":"2024-04-25T11:19:09.730018+0000","last_peered":"2024-04-25T11:19:09.730018+0000","last_clean":"2024-04-25T11:19:09.730018+0000","last_became_active":"2024-04-25T11:19:09.723015+0000","last_became_peered":"2024-04-25T11:19:09.723015+0000","last_unstale":"2024-04-25T11:19:09.730018+0000","last_undegraded":"2024-04-25T11:19:09.730018+0000","last_fullsized":"2024-04-25T11:19:09.730018+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:09.229793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.720269+0000","last_change":"2024-04-25T11:19:08.712591+0000","last_active":"2024-04-25T11:19:10.720269+0000","last_peered":"2024-04-25T11:19:10.720269+0000","last_clean":"2024-04-25T11:19:10.720269+0000","last_became_active":"2024-04-25T11:19:08.712114+0000","last_became_peered":"2024-04-25T11:19:08.712114+0000","last_unstale":"2024-04-25T11:19:10.720269+0000","last_undegraded":"2024-04-25T11:19:10.720269+0000","last_fullsized":"2024-04-25T11:19:10.720269+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:45.657089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017224+0000","last_change":"2024-04-25T11:19:09.705557+0000","last_active":"2024-04-25T11:19:11.017224+0000","last_peered":"2024-04-25T11:19:11.017224+0000","last_clean":"2024-04-25T11:19:11.017224+0000","last_became_active":"2024-04-25T11:19:09.705200+0000","last_became_peered":"2024-04-25T11:19:09.705200+0000","last_unstale":"2024-04-25T11:19:11.017224+0000","last_undegraded":"2024-04-25T11:19:11.017224+0000","last_fullsized":"2024-04-25T11:19:11.017224+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:06.816077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.870047+0000","last_change":"2024-04-25T11:19:08.715055+0000","last_active":"2024-04-25T11:19:10.870047+0000","last_peered":"2024-04-25T11:19:10.870047+0000","last_clean":"2024-04-25T11:19:10.870047+0000","last_became_active":"2024-04-25T11:19:08.714750+0000","last_became_peered":"2024-04-25T11:19:08.714750+0000","last_unstale":"2024-04-25T11:19:10.870047+0000","last_undegraded":"2024-04-25T11:19:10.870047+0000","last_fullsized":"2024-04-25T11:19:10.870047+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:45.657089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.868958+0000","last_change":"2024-04-25T11:19:09.716609+0000","last_active":"2024-04-25T11:19:10.868958+0000","last_peered":"2024-04-25T11:19:10.868958+0000","last_clean":"2024-04-25T11:19:10.868958+0000","last_became_active":"2024-04-25T11:19:09.716261+0000","last_became_peered":"2024-04-25T11:19:09.716261+0000","last_unstale":"2024-04-25T11:19:10.868958+0000","last_undegraded":"2024-04-25T11:19:10.868958+0000","last_fullsized":"2024-04-25T11:19:10.868958+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:00.099157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.725730+0000","last_change":"2024-04-25T11:19:08.708933+0000","last_active":"2024-04-25T11:19:09.725730+0000","last_peered":"2024-04-25T11:19:09.725730+0000","last_clean":"2024-04-25T11:19:09.725730+0000","last_became_active":"2024-04-25T11:19:08.708577+0000","last_became_peered":"2024-04-25T11:19:08.708577+0000","last_unstale":"2024-04-25T11:19:09.725730+0000","last_undegraded":"2024-04-25T11:19:09.725730+0000","last_fullsized":"2024-04-25T11:19:09.725730+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:04.645618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724642+0000","last_change":"2024-04-25T11:19:09.703849+0000","last_active":"2024-04-25T11:19:09.724642+0000","last_peered":"2024-04-25T11:19:09.724642+0000","last_clean":"2024-04-25T11:19:09.724642+0000","last_became_active":"2024-04-25T11:19:09.702990+0000","last_became_peered":"2024-04-25T11:19:09.702990+0000","last_unstale":"2024-04-25T11:19:09.724642+0000","last_undegraded":"2024-04-25T11:19:09.724642+0000","last_fullsized":"2024-04-25T11:19:09.724642+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:38.893558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.709178+0000","last_change":"2024-04-25T11:19:08.713021+0000","last_active":"2024-04-25T11:19:09.709178+0000","last_peered":"2024-04-25T11:19:09.709178+0000","last_clean":"2024-04-25T11:19:09.709178+0000","last_became_active":"2024-04-25T11:19:08.712663+0000","last_became_peered":"2024-04-25T11:19:08.712663+0000","last_unstale":"2024-04-25T11:19:09.709178+0000","last_undegraded":"2024-04-25T11:19:09.709178+0000","last_fullsized":"2024-04-25T11:19:09.709178+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:59.942017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727516+0000","last_change":"2024-04-25T11:19:09.720420+0000","last_active":"2024-04-25T11:19:09.727516+0000","last_peered":"2024-04-25T11:19:09.727516+0000","last_clean":"2024-04-25T11:19:09.727516+0000","last_became_active":"2024-04-25T11:19:09.720099+0000","last_became_peered":"2024-04-25T11:19:09.720099+0000","last_unstale":"2024-04-25T11:19:09.727516+0000","last_undegraded":"2024-04-25T11:19:09.727516+0000","last_fullsized":"2024-04-25T11:19:09.727516+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:45.126171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.018330+0000","last_change":"2024-04-25T11:19:08.707288+0000","last_active":"2024-04-25T11:19:11.018330+0000","last_peered":"2024-04-25T11:19:11.018330+0000","last_clean":"2024-04-25T11:19:11.018330+0000","last_became_active":"2024-04-25T11:19:08.706963+0000","last_became_peered":"2024-04-25T11:19:08.706963+0000","last_unstale":"2024-04-25T11:19:11.018330+0000","last_undegraded":"2024-04-25T11:19:11.018330+0000","last_fullsized":"2024-04-25T11:19:11.018330+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:57.697723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.721603+0000","last_change":"2024-04-25T11:19:09.708506+0000","last_active":"2024-04-25T11:19:09.721603+0000","last_peered":"2024-04-25T11:19:09.721603+0000","last_clean":"2024-04-25T11:19:09.721603+0000","last_became_active":"2024-04-25T11:19:09.708162+0000","last_became_peered":"2024-04-25T11:19:09.708162+0000","last_unstale":"2024-04-25T11:19:09.721603+0000","last_undegraded":"2024-04-25T11:19:09.721603+0000","last_fullsized":"2024-04-25T11:19:09.721603+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:53.262229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016553+0000","last_change":"2024-04-25T11:19:08.698313+0000","last_active":"2024-04-25T11:19:11.016553+0000","last_peered":"2024-04-25T11:19:11.016553+0000","last_clean":"2024-04-25T11:19:11.016553+0000","last_became_active":"2024-04-25T11:19:08.697950+0000","last_became_peered":"2024-04-25T11:19:08.697950+0000","last_unstale":"2024-04-25T11:19:11.016553+0000","last_undegraded":"2024-04-25T11:19:11.016553+0000","last_fullsized":"2024-04-25T11:19:11.016553+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:58.014541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724409+0000","last_change":"2024-04-25T11:19:09.720265+0000","last_active":"2024-04-25T11:19:09.724409+0000","last_peered":"2024-04-25T11:19:09.724409+0000","last_clean":"2024-04-25T11:19:09.724409+0000","last_became_active":"2024-04-25T11:19:09.719926+0000","last_became_peered":"2024-04-25T11:19:09.719926+0000","last_unstale":"2024-04-25T11:19:09.724409+0000","last_undegraded":"2024-04-25T11:19:09.724409+0000","last_fullsized":"2024-04-25T11:19:09.724409+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:36.384892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.868824+0000","last_change":"2024-04-25T11:19:08.715722+0000","last_active":"2024-04-25T11:19:10.868824+0000","last_peered":"2024-04-25T11:19:10.868824+0000","last_clean":"2024-04-25T11:19:10.868824+0000","last_became_active":"2024-04-25T11:19:08.714740+0000","last_became_peered":"2024-04-25T11:19:08.714740+0000","last_unstale":"2024-04-25T11:19:10.868824+0000","last_undegraded":"2024-04-25T11:19:10.868824+0000","last_fullsized":"2024-04-25T11:19:10.868824+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:42.107371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017859+0000","last_change":"2024-04-25T11:19:09.716107+0000","last_active":"2024-04-25T11:19:11.017859+0000","last_peered":"2024-04-25T11:19:11.017859+0000","last_clean":"2024-04-25T11:19:11.017859+0000","last_became_active":"2024-04-25T11:19:09.715223+0000","last_became_peered":"2024-04-25T11:19:09.715223+0000","last_unstale":"2024-04-25T11:19:11.017859+0000","last_undegraded":"2024-04-25T11:19:11.017859+0000","last_fullsized":"2024-04-25T11:19:11.017859+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:51.484231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016882+0000","last_change":"2024-04-25T11:19:08.698434+0000","last_active":"2024-04-25T11:19:11.016882+0000","last_peered":"2024-04-25T11:19:11.016882+0000","last_clean":"2024-04-25T11:19:11.016882+0000","last_became_active":"2024-04-25T11:19:08.698069+0000","last_became_peered":"2024-04-25T11:19:08.698069+0000","last_unstale":"2024-04-25T11:19:11.016882+0000","last_undegraded":"2024-04-25T11:19:11.016882+0000","last_fullsized":"2024-04-25T11:19:11.016882+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:02.881374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.723957+0000","last_change":"2024-04-25T11:19:09.714095+0000","last_active":"2024-04-25T11:19:09.723957+0000","last_peered":"2024-04-25T11:19:09.723957+0000","last_clean":"2024-04-25T11:19:09.723957+0000","last_became_active":"2024-04-25T11:19:09.713552+0000","last_became_peered":"2024-04-25T11:19:09.713552+0000","last_unstale":"2024-04-25T11:19:09.723957+0000","last_undegraded":"2024-04-25T11:19:09.723957+0000","last_fullsized":"2024-04-25T11:19:09.723957+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:20.533817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.729454+0000","last_change":"2024-04-25T11:19:08.705001+0000","last_active":"2024-04-25T11:19:09.729454+0000","last_peered":"2024-04-25T11:19:09.729454+0000","last_clean":"2024-04-25T11:19:09.729454+0000","last_became_active":"2024-04-25T11:19:08.703645+0000","last_became_peered":"2024-04-25T11:19:08.703645+0000","last_unstale":"2024-04-25T11:19:09.729454+0000","last_undegraded":"2024-04-25T11:19:09.729454+0000","last_fullsized":"2024-04-25T11:19:09.729454+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:42.107371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.720489+0000","last_change":"2024-04-25T11:19:09.720750+0000","last_active":"2024-04-25T11:19:10.720489+0000","last_peered":"2024-04-25T11:19:10.720489+0000","last_clean":"2024-04-25T11:19:10.720489+0000","last_became_active":"2024-04-25T11:19:09.720422+0000","last_became_peered":"2024-04-25T11:19:09.720422+0000","last_unstale":"2024-04-25T11:19:10.720489+0000","last_undegraded":"2024-04-25T11:19:10.720489+0000","last_fullsized":"2024-04-25T11:19:10.720489+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:26.953283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.708073+0000","last_change":"2024-04-25T11:19:08.715297+0000","last_active":"2024-04-25T11:19:09.708073+0000","last_peered":"2024-04-25T11:19:09.708073+0000","last_clean":"2024-04-25T11:19:09.708073+0000","last_became_active":"2024-04-25T11:19:08.714853+0000","last_became_peered":"2024-04-25T11:19:08.714853+0000","last_unstale":"2024-04-25T11:19:09.708073+0000","last_undegraded":"2024-04-25T11:19:09.708073+0000","last_fullsized":"2024-04-25T11:19:09.708073+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:36.110794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724937+0000","last_change":"2024-04-25T11:19:09.721488+0000","last_active":"2024-04-25T11:19:09.724937+0000","last_peered":"2024-04-25T11:19:09.724937+0000","last_clean":"2024-04-25T11:19:09.724937+0000","last_became_active":"2024-04-25T11:19:09.720577+0000","last_became_peered":"2024-04-25T11:19:09.720577+0000","last_unstale":"2024-04-25T11:19:09.724937+0000","last_undegraded":"2024-04-25T11:19:09.724937+0000","last_fullsized":"2024-04-25T11:19:09.724937+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:55.241977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.869778+0000","last_change":"2024-04-25T11:19:08.711679+0000","last_active":"2024-04-25T11:19:10.869778+0000","last_peered":"2024-04-25T11:19:10.869778+0000","last_clean":"2024-04-25T11:19:10.869778+0000","last_became_active":"2024-04-25T11:19:08.710796+0000","last_became_peered":"2024-04-25T11:19:08.710796+0000","last_unstale":"2024-04-25T11:19:10.869778+0000","last_undegraded":"2024-04-25T11:19:10.869778+0000","last_fullsized":"2024-04-25T11:19:10.869778+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:09.125332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.723131+0000","last_change":"2024-04-25T11:19:09.716679+0000","last_active":"2024-04-25T11:19:09.723131+0000","last_peered":"2024-04-25T11:19:09.723131+0000","last_clean":"2024-04-25T11:19:09.723131+0000","last_became_active":"2024-04-25T11:19:09.714281+0000","last_became_peered":"2024-04-25T11:19:09.714281+0000","last_unstale":"2024-04-25T11:19:09.723131+0000","last_undegraded":"2024-04-25T11:19:09.723131+0000","last_fullsized":"2024-04-25T11:19:09.723131+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:52.626259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.690181+0000","last_change":"2024-04-25T11:19:08.712088+0000","last_active":"2024-04-25T11:19:09.690181+0000","last_peered":"2024-04-25T11:19:09.690181+0000","last_clean":"2024-04-25T11:19:09.690181+0000","last_became_active":"2024-04-25T11:19:08.711696+0000","last_became_peered":"2024-04-25T11:19:08.711696+0000","last_unstale":"2024-04-25T11:19:09.690181+0000","last_undegraded":"2024-04-25T11:19:09.690181+0000","last_fullsized":"2024-04-25T11:19:09.690181+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:41.646122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017056+0000","last_change":"2024-04-25T11:19:09.717055+0000","last_active":"2024-04-25T11:19:11.017056+0000","last_peered":"2024-04-25T11:19:11.017056+0000","last_clean":"2024-04-25T11:19:11.017056+0000","last_became_active":"2024-04-25T11:19:09.716567+0000","last_became_peered":"2024-04-25T11:19:09.716567+0000","last_unstale":"2024-04-25T11:19:11.017056+0000","last_undegraded":"2024-04-25T11:19:11.017056+0000","last_fullsized":"2024-04-25T11:19:11.017056+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:34.145608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.720822+0000","last_change":"2024-04-25T11:19:08.710120+0000","last_active":"2024-04-25T11:19:10.720822+0000","last_peered":"2024-04-25T11:19:10.720822+0000","last_clean":"2024-04-25T11:19:10.720822+0000","last_became_active":"2024-04-25T11:19:08.709776+0000","last_became_peered":"2024-04-25T11:19:08.709776+0000","last_unstale":"2024-04-25T11:19:10.720822+0000","last_undegraded":"2024-04-25T11:19:10.720822+0000","last_fullsized":"2024-04-25T11:19:10.720822+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:55.795333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017825+0000","last_change":"2024-04-25T11:19:09.711296+0000","last_active":"2024-04-25T11:19:11.017825+0000","last_peered":"2024-04-25T11:19:11.017825+0000","last_clean":"2024-04-25T11:19:11.017825+0000","last_became_active":"2024-04-25T11:19:09.710981+0000","last_became_peered":"2024-04-25T11:19:09.710981+0000","last_unstale":"2024-04-25T11:19:11.017825+0000","last_undegraded":"2024-04-25T11:19:11.017825+0000","last_fullsized":"2024-04-25T11:19:11.017825+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:39.203607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017054+0000","last_change":"2024-04-25T11:19:08.704110+0000","last_active":"2024-04-25T11:19:11.017054+0000","last_peered":"2024-04-25T11:19:11.017054+0000","last_clean":"2024-04-25T11:19:11.017054+0000","last_became_active":"2024-04-25T11:19:08.703615+0000","last_became_peered":"2024-04-25T11:19:08.703615+0000","last_unstale":"2024-04-25T11:19:11.017054+0000","last_undegraded":"2024-04-25T11:19:11.017054+0000","last_fullsized":"2024-04-25T11:19:11.017054+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:01.092687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.725339+0000","last_change":"2024-04-25T11:19:09.720844+0000","last_active":"2024-04-25T11:19:09.725339+0000","last_peered":"2024-04-25T11:19:09.725339+0000","last_clean":"2024-04-25T11:19:09.725339+0000","last_became_active":"2024-04-25T11:19:09.720532+0000","last_became_peered":"2024-04-25T11:19:09.720532+0000","last_unstale":"2024-04-25T11:19:09.725339+0000","last_undegraded":"2024-04-25T11:19:09.725339+0000","last_fullsized":"2024-04-25T11:19:09.725339+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:03.663753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.869023+0000","last_change":"2024-04-25T11:19:08.710917+0000","last_active":"2024-04-25T11:19:10.869023+0000","last_peered":"2024-04-25T11:19:10.869023+0000","last_clean":"2024-04-25T11:19:10.869023+0000","last_became_active":"2024-04-25T11:19:08.710541+0000","last_became_peered":"2024-04-25T11:19:08.710541+0000","last_unstale":"2024-04-25T11:19:10.869023+0000","last_undegraded":"2024-04-25T11:19:10.869023+0000","last_fullsized":"2024-04-25T11:19:10.869023+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:55.795333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.722208+0000","last_change":"2024-04-25T11:19:09.718243+0000","last_active":"2024-04-25T11:19:09.722208+0000","last_peered":"2024-04-25T11:19:09.722208+0000","last_clean":"2024-04-25T11:19:09.722208+0000","last_became_active":"2024-04-25T11:19:09.717755+0000","last_became_peered":"2024-04-25T11:19:09.717755+0000","last_unstale":"2024-04-25T11:19:09.722208+0000","last_undegraded":"2024-04-25T11:19:09.722208+0000","last_fullsized":"2024-04-25T11:19:09.722208+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:21.308335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017210+0000","last_change":"2024-04-25T11:19:08.705550+0000","last_active":"2024-04-25T11:19:11.017210+0000","last_peered":"2024-04-25T11:19:11.017210+0000","last_clean":"2024-04-25T11:19:11.017210+0000","last_became_active":"2024-04-25T11:19:08.705237+0000","last_became_peered":"2024-04-25T11:19:08.705237+0000","last_unstale":"2024-04-25T11:19:11.017210+0000","last_undegraded":"2024-04-25T11:19:11.017210+0000","last_fullsized":"2024-04-25T11:19:11.017210+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:41.620818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724047+0000","last_change":"2024-04-25T11:19:09.718648+0000","last_active":"2024-04-25T11:19:09.724047+0000","last_peered":"2024-04-25T11:19:09.724047+0000","last_clean":"2024-04-25T11:19:09.724047+0000","last_became_active":"2024-04-25T11:19:09.715913+0000","last_became_peered":"2024-04-25T11:19:09.715913+0000","last_unstale":"2024-04-25T11:19:09.724047+0000","last_undegraded":"2024-04-25T11:19:09.724047+0000","last_fullsized":"2024-04-25T11:19:09.724047+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:07.906131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017052+0000","last_change":"2024-04-25T11:19:08.709112+0000","last_active":"2024-04-25T11:19:11.017052+0000","last_peered":"2024-04-25T11:19:11.017052+0000","last_clean":"2024-04-25T11:19:11.017052+0000","last_became_active":"2024-04-25T11:19:08.708338+0000","last_became_peered":"2024-04-25T11:19:08.708338+0000","last_unstale":"2024-04-25T11:19:11.017052+0000","last_undegraded":"2024-04-25T11:19:11.017052+0000","last_fullsized":"2024-04-25T11:19:11.017052+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:24.203477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":278,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.687528+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:19:09.687528+0000","last_peered":"2024-04-25T11:19:09.687528+0000","last_clean":"2024-04-25T11:19:09.687528+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:19:09.687528+0000","last_undegraded":"2024-04-25T11:19:09.687528+0000","last_fullsized":"2024-04-25T11:19:09.687528+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476846,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1892352,"data_stored":1183177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476847,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1310720,"data_stored":594865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476846,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":592649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476847,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1298432,"data_stored":592741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509550,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":593231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509551,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":585482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509550,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1871872,"data_stored":1175850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542256,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1306624,"data_stored":599375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:19:14.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:19:14.632 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-04-25T11:19:14.633+0000 7fc13ec00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:19:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.783+0000 7f1e09a00640 1 Processor -- start 2024-04-25T11:19:14.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.783+0000 7f1e09a00640 1 -- start start 2024-04-25T11:19:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.785+0000 7f1e09a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1e040b5220 0x7f1e0415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.785+0000 7f1e09a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e040af420 0x7f1e040af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.785+0000 7f1e09a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e040cebb0 con 0x7f1e040aed40 2024-04-25T11:19:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.785+0000 7f1e09a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1e0415e1f0 con 0x7f1e040b5220 2024-04-25T11:19:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.785+0000 7f1e09a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1e040afd70 con 0x7f1e040af420 2024-04-25T11:19:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.785+0000 7f1e03400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1e040b5220 0x7f1e0415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.785+0000 7f1e03400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1e040b5220 0x7f1e0415dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51736/0 (socket says 172.21.15.7:51736) 2024-04-25T11:19:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.785+0000 7f1e03400640 1 -- 172.21.15.7:0/4028999076 learned_addr learned my addr 172.21.15.7:0/4028999076 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.786+0000 7f1e09000640 1 --2- 172.21.15.7:0/4028999076 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e040af420 0x7f1e040af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.786+0000 7f1e03400640 1 -- 172.21.15.7:0/4028999076 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e040af420 msgr2=0x7f1e040af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.786+0000 7f1e03400640 1 --2- 172.21.15.7:0/4028999076 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e040af420 0x7f1e040af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.786+0000 7f1e03400640 1 -- 172.21.15.7:0/4028999076 >> v1:172.21.15.7:6789/0 conn(0x7f1e040aed40 legacy=0x7f1e040ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.786+0000 7f1e02a00640 1 -- 172.21.15.7:0/4028999076 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613750854 0 0) 0x7f1df4001110 con 0x7f1e040aed40 2024-04-25T11:19:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.786+0000 7f1e03400640 1 -- 172.21.15.7:0/4028999076 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e040b64b0 con 0x7f1e040b5220 2024-04-25T11:19:14.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.787+0000 7f1e03400640 1 --2- 172.21.15.7:0/4028999076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e040b5220 0x7f1e0415dcb0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009870 tx=0x7f1dec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a971eca6471d925 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:14.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.787+0000 7f1e02a00640 1 -- 172.21.15.7:0/4028999076 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec037070 con 0x7f1e040b5220 2024-04-25T11:19:14.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.787+0000 7f1e02a00640 1 -- 172.21.15.7:0/4028999076 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dec032440 con 0x7f1e040b5220 2024-04-25T11:19:14.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.787+0000 7f1e02a00640 1 -- 172.21.15.7:0/4028999076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec032810 con 0x7f1e040b5220 2024-04-25T11:19:14.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.788+0000 7f1e09a00640 1 -- 172.21.15.7:0/4028999076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e040b5220 msgr2=0x7f1e0415dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:14.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.788+0000 7f1e09a00640 1 --2- 172.21.15.7:0/4028999076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e040b5220 0x7f1e0415dcb0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009870 tx=0x7f1dec02a270 comp rx=0 tx=0).stop 2024-04-25T11:19:14.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.788+0000 7f1e09a00640 1 -- 172.21.15.7:0/4028999076 shutdown_connections 2024-04-25T11:19:14.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.788+0000 7f1e09a00640 1 --2- 172.21.15.7:0/4028999076 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e040af420 0x7f1e040af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.788+0000 7f1e09a00640 1 --2- 172.21.15.7:0/4028999076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e040b5220 0x7f1e0415dcb0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.788+0000 7f1e09a00640 1 -- 172.21.15.7:0/4028999076 >> 172.21.15.7:0/4028999076 conn(0x7f1e040b3820 msgr2=0x7f1e040b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:14.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.788+0000 7f1e09a00640 1 -- 172.21.15.7:0/4028999076 shutdown_connections 2024-04-25T11:19:14.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.788+0000 7f1e09a00640 1 -- 172.21.15.7:0/4028999076 wait complete. 2024-04-25T11:19:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.791+0000 7f1e09a00640 1 Processor -- start 2024-04-25T11:19:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.791+0000 7f1e09a00640 1 -- start start 2024-04-25T11:19:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.792+0000 7f1e09a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e040aed40 0x7f1e040d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.792+0000 7f1e09a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e040af420 0x7f1e04149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.792+0000 7f1e09a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e040b5220 0x7f1e040ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.792+0000 7f1e09a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0406cbd0 con 0x7f1e040b5220 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.792+0000 7f1e09a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0406c8f0 con 0x7f1e040aed40 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.792+0000 7f1e09a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0406ceb0 con 0x7f1e040af420 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.793+0000 7f1e09000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e040b5220 0x7f1e040ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.793+0000 7f1e09000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e040b5220 0x7f1e040ced90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51750/0 (socket says 172.21.15.7:51750) 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.793+0000 7f1e09000640 1 -- 172.21.15.7:0/3561388413 learned_addr learned my addr 172.21.15.7:0/3561388413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.793+0000 7f1e03400640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e040af420 0x7f1e04149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.793+0000 7f1e03e00640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e040aed40 0x7f1e040d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.793+0000 7f1e09000640 1 -- 172.21.15.7:0/3561388413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e040af420 msgr2=0x7f1e04149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.793+0000 7f1e09000640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e040af420 0x7f1e04149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.793+0000 7f1e09000640 1 -- 172.21.15.7:0/3561388413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e040aed40 msgr2=0x7f1e040d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.793+0000 7f1e09000640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e040aed40 0x7f1e040d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.793+0000 7f1e09000640 1 -- 172.21.15.7:0/3561388413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e040cd180 con 0x7f1e040b5220 2024-04-25T11:19:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.794+0000 7f1e09000640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e040b5220 0x7f1e040ced90 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc007e60 tx=0x7f1dfc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.794+0000 7f1dfbe00640 1 -- 172.21.15.7:0/3561388413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc014420 con 0x7f1e040b5220 2024-04-25T11:19:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.794+0000 7f1dfbe00640 1 -- 172.21.15.7:0/3561388413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dfc019070 con 0x7f1e040b5220 2024-04-25T11:19:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.794+0000 7f1dfbe00640 1 -- 172.21.15.7:0/3561388413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc014820 con 0x7f1e040b5220 2024-04-25T11:19:14.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.794+0000 7f1e09a00640 1 -- 172.21.15.7:0/3561388413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e040cd460 con 0x7f1e040b5220 2024-04-25T11:19:14.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.794+0000 7f1e09a00640 1 -- 172.21.15.7:0/3561388413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e040cf980 con 0x7f1e040b5220 2024-04-25T11:19:14.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.796+0000 7f1e09a00640 1 -- 172.21.15.7:0/3561388413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e040cfbe0 con 0x7f1e040b5220 2024-04-25T11:19:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.797+0000 7f1dfbe00640 1 -- 172.21.15.7:0/3561388413 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1dfc0149c0 con 0x7f1e040b5220 2024-04-25T11:19:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.798+0000 7f1dfbe00640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1de00744b0 0x7f1de00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.798+0000 7f1dfbe00640 1 -- 172.21.15.7:0/3561388413 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f1dfc095c60 con 0x7f1e040b5220 2024-04-25T11:19:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.798+0000 7f1e03e00640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1de00744b0 0x7f1de00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.799+0000 7f1e03e00640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1de00744b0 0x7f1de00769c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1df4007fd0 tx=0x7f1df4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:14.802+0000 7f1dfbe00640 1 -- 172.21.15.7:0/3561388413 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1dfc05fc00 con 0x7f1e040b5220 2024-04-25T11:19:15.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.011+0000 7f1e09a00640 1 -- 172.21.15.7:0/3561388413 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e04050740 con 0x7f1de00744b0 2024-04-25T11:19:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.031+0000 7f1dfbe00640 1 -- 172.21.15.7:0/3561388413 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252434 (secure 0 0 0) 0x7f1e04050740 con 0x7f1de00744b0 2024-04-25T11:19:15.034 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:19:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.037+0000 7f1e09a00640 1 -- 172.21.15.7:0/3561388413 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1de00744b0 msgr2=0x7f1de00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.037+0000 7f1e09a00640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1de00744b0 0x7f1de00769c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1df4007fd0 tx=0x7f1df4009040 comp rx=0 tx=0).stop 2024-04-25T11:19:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.037+0000 7f1e09a00640 1 -- 172.21.15.7:0/3561388413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e040b5220 msgr2=0x7f1e040ced90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.037+0000 7f1e09a00640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e040b5220 0x7f1e040ced90 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc007e60 tx=0x7f1dfc00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.038+0000 7f1e09a00640 1 -- 172.21.15.7:0/3561388413 shutdown_connections 2024-04-25T11:19:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.038+0000 7f1e09a00640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1de00744b0 0x7f1de00769c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.038+0000 7f1e09a00640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e040b5220 0x7f1e040ced90 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.038+0000 7f1e09a00640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e040af420 0x7f1e04149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.038+0000 7f1e09a00640 1 --2- 172.21.15.7:0/3561388413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e040aed40 0x7f1e040d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.038+0000 7f1e09a00640 1 -- 172.21.15.7:0/3561388413 >> 172.21.15.7:0/3561388413 conn(0x7f1e040b3820 msgr2=0x7f1e040ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.038+0000 7f1e09a00640 1 -- 172.21.15.7:0/3561388413 shutdown_connections 2024-04-25T11:19:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.038+0000 7f1e09a00640 1 -- 172.21.15.7:0/3561388413 wait complete. 2024-04-25T11:19:15.066 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-04-25T11:19:13.996567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260764,"kb_used_data":11272,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713956,"statfs":{"total":773094113280,"available":772827090944,"internally_reserved":0,"allocated":11542528,"data_stored":5917370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001930"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.723919+0000","last_change":"2024-04-25T11:19:09.713598+0000","last_active":"2024-04-25T11:19:09.723919+0000","last_peered":"2024-04-25T11:19:09.723919+0000","last_clean":"2024-04-25T11:19:09.723919+0000","last_became_active":"2024-04-25T11:19:09.713098+0000","last_became_peered":"2024-04-25T11:19:09.713098+0000","last_unstale":"2024-04-25T11:19:09.723919+0000","last_undegraded":"2024-04-25T11:19:09.723919+0000","last_fullsized":"2024-04-25T11:19:09.723919+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:26.755621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.870024+0000","last_change":"2024-04-25T11:19:09.702747+0000","last_active":"2024-04-25T11:19:10.870024+0000","last_peered":"2024-04-25T11:19:10.870024+0000","last_clean":"2024-04-25T11:19:10.870024+0000","last_became_active":"2024-04-25T11:19:09.702454+0000","last_became_peered":"2024-04-25T11:19:09.702454+0000","last_unstale":"2024-04-25T11:19:10.870024+0000","last_undegraded":"2024-04-25T11:19:10.870024+0000","last_fullsized":"2024-04-25T11:19:10.870024+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:36.384892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.730240+0000","last_change":"2024-04-25T11:19:09.726556+0000","last_active":"2024-04-25T11:19:09.730240+0000","last_peered":"2024-04-25T11:19:09.730240+0000","last_clean":"2024-04-25T11:19:09.730240+0000","last_became_active":"2024-04-25T11:19:09.726247+0000","last_became_peered":"2024-04-25T11:19:09.726247+0000","last_unstale":"2024-04-25T11:19:09.730240+0000","last_undegraded":"2024-04-25T11:19:09.730240+0000","last_fullsized":"2024-04-25T11:19:09.730240+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:48.612468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.719957+0000","last_change":"2024-04-25T11:19:09.712961+0000","last_active":"2024-04-25T11:19:09.719957+0000","last_peered":"2024-04-25T11:19:09.719957+0000","last_clean":"2024-04-25T11:19:09.719957+0000","last_became_active":"2024-04-25T11:19:09.712468+0000","last_became_peered":"2024-04-25T11:19:09.712468+0000","last_unstale":"2024-04-25T11:19:09.719957+0000","last_undegraded":"2024-04-25T11:19:09.719957+0000","last_fullsized":"2024-04-25T11:19:09.719957+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.992694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724695+0000","last_change":"2024-04-25T11:19:09.707134+0000","last_active":"2024-04-25T11:19:09.724695+0000","last_peered":"2024-04-25T11:19:09.724695+0000","last_clean":"2024-04-25T11:19:09.724695+0000","last_became_active":"2024-04-25T11:19:09.706819+0000","last_became_peered":"2024-04-25T11:19:09.706819+0000","last_unstale":"2024-04-25T11:19:09.724695+0000","last_undegraded":"2024-04-25T11:19:09.724695+0000","last_fullsized":"2024-04-25T11:19:09.724695+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:03.370541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.868824+0000","last_change":"2024-04-25T11:19:09.707059+0000","last_active":"2024-04-25T11:19:10.868824+0000","last_peered":"2024-04-25T11:19:10.868824+0000","last_clean":"2024-04-25T11:19:10.868824+0000","last_became_active":"2024-04-25T11:19:09.705959+0000","last_became_peered":"2024-04-25T11:19:09.705959+0000","last_unstale":"2024-04-25T11:19:10.868824+0000","last_undegraded":"2024-04-25T11:19:10.868824+0000","last_fullsized":"2024-04-25T11:19:10.868824+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:00.515175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.868753+0000","last_change":"2024-04-25T11:19:09.706669+0000","last_active":"2024-04-25T11:19:10.868753+0000","last_peered":"2024-04-25T11:19:10.868753+0000","last_clean":"2024-04-25T11:19:10.868753+0000","last_became_active":"2024-04-25T11:19:09.706243+0000","last_became_peered":"2024-04-25T11:19:09.706243+0000","last_unstale":"2024-04-25T11:19:10.868753+0000","last_undegraded":"2024-04-25T11:19:10.868753+0000","last_fullsized":"2024-04-25T11:19:10.868753+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:52.054644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016509+0000","last_change":"2024-04-25T11:19:09.711279+0000","last_active":"2024-04-25T11:19:11.016509+0000","last_peered":"2024-04-25T11:19:11.016509+0000","last_clean":"2024-04-25T11:19:11.016509+0000","last_became_active":"2024-04-25T11:19:09.710912+0000","last_became_peered":"2024-04-25T11:19:09.710912+0000","last_unstale":"2024-04-25T11:19:11.016509+0000","last_undegraded":"2024-04-25T11:19:11.016509+0000","last_fullsized":"2024-04-25T11:19:11.016509+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:27.095256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.870140+0000","last_change":"2024-04-25T11:19:09.710225+0000","last_active":"2024-04-25T11:19:10.870140+0000","last_peered":"2024-04-25T11:19:10.870140+0000","last_clean":"2024-04-25T11:19:10.870140+0000","last_became_active":"2024-04-25T11:19:09.709881+0000","last_became_peered":"2024-04-25T11:19:09.709881+0000","last_unstale":"2024-04-25T11:19:10.870140+0000","last_undegraded":"2024-04-25T11:19:10.870140+0000","last_fullsized":"2024-04-25T11:19:10.870140+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:55.241977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.719748+0000","last_change":"2024-04-25T11:19:09.712962+0000","last_active":"2024-04-25T11:19:09.719748+0000","last_peered":"2024-04-25T11:19:09.719748+0000","last_clean":"2024-04-25T11:19:09.719748+0000","last_became_active":"2024-04-25T11:19:09.712638+0000","last_became_peered":"2024-04-25T11:19:09.712638+0000","last_unstale":"2024-04-25T11:19:09.719748+0000","last_undegraded":"2024-04-25T11:19:09.719748+0000","last_fullsized":"2024-04-25T11:19:09.719748+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:29.719599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.726057+0000","last_change":"2024-04-25T11:19:09.722604+0000","last_active":"2024-04-25T11:19:09.726057+0000","last_peered":"2024-04-25T11:19:09.726057+0000","last_clean":"2024-04-25T11:19:09.726057+0000","last_became_active":"2024-04-25T11:19:09.721158+0000","last_became_peered":"2024-04-25T11:19:09.721158+0000","last_unstale":"2024-04-25T11:19:09.726057+0000","last_undegraded":"2024-04-25T11:19:09.726057+0000","last_fullsized":"2024-04-25T11:19:09.726057+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:24.997412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.722053+0000","last_change":"2024-04-25T11:19:09.712564+0000","last_active":"2024-04-25T11:19:10.722053+0000","last_peered":"2024-04-25T11:19:10.722053+0000","last_clean":"2024-04-25T11:19:10.722053+0000","last_became_active":"2024-04-25T11:19:09.711650+0000","last_became_peered":"2024-04-25T11:19:09.711650+0000","last_unstale":"2024-04-25T11:19:10.722053+0000","last_undegraded":"2024-04-25T11:19:10.722053+0000","last_fullsized":"2024-04-25T11:19:10.722053+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:10.756423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.728273+0000","last_change":"2024-04-25T11:19:09.726015+0000","last_active":"2024-04-25T11:19:09.728273+0000","last_peered":"2024-04-25T11:19:09.728273+0000","last_clean":"2024-04-25T11:19:09.728273+0000","last_became_active":"2024-04-25T11:19:09.724929+0000","last_became_peered":"2024-04-25T11:19:09.724929+0000","last_unstale":"2024-04-25T11:19:09.728273+0000","last_undegraded":"2024-04-25T11:19:09.728273+0000","last_fullsized":"2024-04-25T11:19:09.728273+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:05.665209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.716723+0000","last_change":"2024-04-25T11:19:09.712734+0000","last_active":"2024-04-25T11:19:09.716723+0000","last_peered":"2024-04-25T11:19:09.716723+0000","last_clean":"2024-04-25T11:19:09.716723+0000","last_became_active":"2024-04-25T11:19:09.712376+0000","last_became_peered":"2024-04-25T11:19:09.712376+0000","last_unstale":"2024-04-25T11:19:09.716723+0000","last_undegraded":"2024-04-25T11:19:09.716723+0000","last_fullsized":"2024-04-25T11:19:09.716723+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:54.334703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.722010+0000","last_change":"2024-04-25T11:19:09.718928+0000","last_active":"2024-04-25T11:19:09.722010+0000","last_peered":"2024-04-25T11:19:09.722010+0000","last_clean":"2024-04-25T11:19:09.722010+0000","last_became_active":"2024-04-25T11:19:09.718615+0000","last_became_peered":"2024-04-25T11:19:09.718615+0000","last_unstale":"2024-04-25T11:19:09.722010+0000","last_undegraded":"2024-04-25T11:19:09.722010+0000","last_fullsized":"2024-04-25T11:19:09.722010+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:03.964007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.705551+0000","last_change":"2024-04-25T11:19:09.700355+0000","last_active":"2024-04-25T11:19:09.705551+0000","last_peered":"2024-04-25T11:19:09.705551+0000","last_clean":"2024-04-25T11:19:09.705551+0000","last_became_active":"2024-04-25T11:19:09.699968+0000","last_became_peered":"2024-04-25T11:19:09.699968+0000","last_unstale":"2024-04-25T11:19:09.705551+0000","last_undegraded":"2024-04-25T11:19:09.705551+0000","last_fullsized":"2024-04-25T11:19:09.705551+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:29.649252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.730044+0000","last_change":"2024-04-25T11:19:09.727011+0000","last_active":"2024-04-25T11:19:09.730044+0000","last_peered":"2024-04-25T11:19:09.730044+0000","last_clean":"2024-04-25T11:19:09.730044+0000","last_became_active":"2024-04-25T11:19:09.726648+0000","last_became_peered":"2024-04-25T11:19:09.726648+0000","last_unstale":"2024-04-25T11:19:09.730044+0000","last_undegraded":"2024-04-25T11:19:09.730044+0000","last_fullsized":"2024-04-25T11:19:09.730044+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:59.653285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.870301+0000","last_change":"2024-04-25T11:19:09.703842+0000","last_active":"2024-04-25T11:19:10.870301+0000","last_peered":"2024-04-25T11:19:10.870301+0000","last_clean":"2024-04-25T11:19:10.870301+0000","last_became_active":"2024-04-25T11:19:09.703499+0000","last_became_peered":"2024-04-25T11:19:09.703499+0000","last_unstale":"2024-04-25T11:19:10.870301+0000","last_undegraded":"2024-04-25T11:19:10.870301+0000","last_fullsized":"2024-04-25T11:19:10.870301+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:55.796067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727603+0000","last_change":"2024-04-25T11:19:09.721422+0000","last_active":"2024-04-25T11:19:09.727603+0000","last_peered":"2024-04-25T11:19:09.727603+0000","last_clean":"2024-04-25T11:19:09.727603+0000","last_became_active":"2024-04-25T11:19:09.721035+0000","last_became_peered":"2024-04-25T11:19:09.721035+0000","last_unstale":"2024-04-25T11:19:09.727603+0000","last_undegraded":"2024-04-25T11:19:09.727603+0000","last_fullsized":"2024-04-25T11:19:09.727603+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:01.467633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.722153+0000","last_change":"2024-04-25T11:19:09.717947+0000","last_active":"2024-04-25T11:19:10.722153+0000","last_peered":"2024-04-25T11:19:10.722153+0000","last_clean":"2024-04-25T11:19:10.722153+0000","last_became_active":"2024-04-25T11:19:09.716428+0000","last_became_peered":"2024-04-25T11:19:09.716428+0000","last_unstale":"2024-04-25T11:19:10.722153+0000","last_undegraded":"2024-04-25T11:19:10.722153+0000","last_fullsized":"2024-04-25T11:19:10.722153+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.992694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017828+0000","last_change":"2024-04-25T11:19:09.718569+0000","last_active":"2024-04-25T11:19:11.017828+0000","last_peered":"2024-04-25T11:19:11.017828+0000","last_clean":"2024-04-25T11:19:11.017828+0000","last_became_active":"2024-04-25T11:19:09.718087+0000","last_became_peered":"2024-04-25T11:19:09.718087+0000","last_unstale":"2024-04-25T11:19:11.017828+0000","last_undegraded":"2024-04-25T11:19:11.017828+0000","last_fullsized":"2024-04-25T11:19:11.017828+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:08.743237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.730115+0000","last_change":"2024-04-25T11:19:09.724581+0000","last_active":"2024-04-25T11:19:09.730115+0000","last_peered":"2024-04-25T11:19:09.730115+0000","last_clean":"2024-04-25T11:19:09.730115+0000","last_became_active":"2024-04-25T11:19:09.720794+0000","last_became_peered":"2024-04-25T11:19:09.720794+0000","last_unstale":"2024-04-25T11:19:09.730115+0000","last_undegraded":"2024-04-25T11:19:09.730115+0000","last_fullsized":"2024-04-25T11:19:09.730115+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:52.626259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.731035+0000","last_change":"2024-04-25T11:19:09.728777+0000","last_active":"2024-04-25T11:19:09.731035+0000","last_peered":"2024-04-25T11:19:09.731035+0000","last_clean":"2024-04-25T11:19:09.731035+0000","last_became_active":"2024-04-25T11:19:09.728206+0000","last_became_peered":"2024-04-25T11:19:09.728206+0000","last_unstale":"2024-04-25T11:19:09.731035+0000","last_undegraded":"2024-04-25T11:19:09.731035+0000","last_fullsized":"2024-04-25T11:19:09.731035+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:06.243770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724468+0000","last_change":"2024-04-25T11:19:09.710323+0000","last_active":"2024-04-25T11:19:09.724468+0000","last_peered":"2024-04-25T11:19:09.724468+0000","last_clean":"2024-04-25T11:19:09.724468+0000","last_became_active":"2024-04-25T11:19:09.709992+0000","last_became_peered":"2024-04-25T11:19:09.709992+0000","last_unstale":"2024-04-25T11:19:09.724468+0000","last_undegraded":"2024-04-25T11:19:09.724468+0000","last_fullsized":"2024-04-25T11:19:09.724468+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:00.961608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.721947+0000","last_change":"2024-04-25T11:19:09.727646+0000","last_active":"2024-04-25T11:19:10.721947+0000","last_peered":"2024-04-25T11:19:10.721947+0000","last_clean":"2024-04-25T11:19:10.721947+0000","last_became_active":"2024-04-25T11:19:09.727136+0000","last_became_peered":"2024-04-25T11:19:09.727136+0000","last_unstale":"2024-04-25T11:19:10.721947+0000","last_undegraded":"2024-04-25T11:19:10.721947+0000","last_fullsized":"2024-04-25T11:19:10.721947+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:00.099157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.721638+0000","last_change":"2024-04-25T11:19:09.723571+0000","last_active":"2024-04-25T11:19:10.721638+0000","last_peered":"2024-04-25T11:19:10.721638+0000","last_clean":"2024-04-25T11:19:10.721638+0000","last_became_active":"2024-04-25T11:19:09.721892+0000","last_became_peered":"2024-04-25T11:19:09.721892+0000","last_unstale":"2024-04-25T11:19:10.721638+0000","last_undegraded":"2024-04-25T11:19:10.721638+0000","last_fullsized":"2024-04-25T11:19:10.721638+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:00.515175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.723926+0000","last_change":"2024-04-25T11:19:09.716265+0000","last_active":"2024-04-25T11:19:09.723926+0000","last_peered":"2024-04-25T11:19:09.723926+0000","last_clean":"2024-04-25T11:19:09.723926+0000","last_became_active":"2024-04-25T11:19:09.715853+0000","last_became_peered":"2024-04-25T11:19:09.715853+0000","last_unstale":"2024-04-25T11:19:09.723926+0000","last_undegraded":"2024-04-25T11:19:09.723926+0000","last_fullsized":"2024-04-25T11:19:09.723926+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:29.719599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.869032+0000","last_change":"2024-04-25T11:19:09.722428+0000","last_active":"2024-04-25T11:19:10.869032+0000","last_peered":"2024-04-25T11:19:10.869032+0000","last_clean":"2024-04-25T11:19:10.869032+0000","last_became_active":"2024-04-25T11:19:09.722102+0000","last_became_peered":"2024-04-25T11:19:09.722102+0000","last_unstale":"2024-04-25T11:19:10.869032+0000","last_undegraded":"2024-04-25T11:19:10.869032+0000","last_fullsized":"2024-04-25T11:19:10.869032+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:10.756423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.018003+0000","last_change":"2024-04-25T11:19:09.704082+0000","last_active":"2024-04-25T11:19:11.018003+0000","last_peered":"2024-04-25T11:19:11.018003+0000","last_clean":"2024-04-25T11:19:11.018003+0000","last_became_active":"2024-04-25T11:19:09.703648+0000","last_became_peered":"2024-04-25T11:19:09.703648+0000","last_unstale":"2024-04-25T11:19:11.018003+0000","last_undegraded":"2024-04-25T11:19:11.018003+0000","last_fullsized":"2024-04-25T11:19:11.018003+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:17.941867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.721262+0000","last_change":"2024-04-25T11:19:09.727980+0000","last_active":"2024-04-25T11:19:10.721262+0000","last_peered":"2024-04-25T11:19:10.721262+0000","last_clean":"2024-04-25T11:19:10.721262+0000","last_became_active":"2024-04-25T11:19:09.727594+0000","last_became_peered":"2024-04-25T11:19:09.727594+0000","last_unstale":"2024-04-25T11:19:10.721262+0000","last_undegraded":"2024-04-25T11:19:10.721262+0000","last_fullsized":"2024-04-25T11:19:10.721262+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:14.566202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.868605+0000","last_change":"2024-04-25T11:19:09.705190+0000","last_active":"2024-04-25T11:19:10.868605+0000","last_peered":"2024-04-25T11:19:10.868605+0000","last_clean":"2024-04-25T11:19:10.868605+0000","last_became_active":"2024-04-25T11:19:09.704854+0000","last_became_peered":"2024-04-25T11:19:09.704854+0000","last_unstale":"2024-04-25T11:19:10.868605+0000","last_undegraded":"2024-04-25T11:19:10.868605+0000","last_fullsized":"2024-04-25T11:19:10.868605+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:14.566202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016479+0000","last_change":"2024-04-25T11:19:09.717396+0000","last_active":"2024-04-25T11:19:11.016479+0000","last_peered":"2024-04-25T11:19:11.016479+0000","last_clean":"2024-04-25T11:19:11.016479+0000","last_became_active":"2024-04-25T11:19:09.717097+0000","last_became_peered":"2024-04-25T11:19:09.717097+0000","last_unstale":"2024-04-25T11:19:11.016479+0000","last_undegraded":"2024-04-25T11:19:11.016479+0000","last_fullsized":"2024-04-25T11:19:11.016479+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:49.344998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727414+0000","last_change":"2024-04-25T11:19:09.720502+0000","last_active":"2024-04-25T11:19:09.727414+0000","last_peered":"2024-04-25T11:19:09.727414+0000","last_clean":"2024-04-25T11:19:09.727414+0000","last_became_active":"2024-04-25T11:19:09.720161+0000","last_became_peered":"2024-04-25T11:19:09.720161+0000","last_unstale":"2024-04-25T11:19:09.727414+0000","last_undegraded":"2024-04-25T11:19:09.727414+0000","last_fullsized":"2024-04-25T11:19:09.727414+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:18.203565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.722799+0000","last_change":"2024-04-25T11:19:08.712020+0000","last_active":"2024-04-25T11:19:09.722799+0000","last_peered":"2024-04-25T11:19:09.722799+0000","last_clean":"2024-04-25T11:19:09.722799+0000","last_became_active":"2024-04-25T11:19:08.711698+0000","last_became_peered":"2024-04-25T11:19:08.711698+0000","last_unstale":"2024-04-25T11:19:09.722799+0000","last_undegraded":"2024-04-25T11:19:09.722799+0000","last_fullsized":"2024-04-25T11:19:09.722799+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:09.736832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016792+0000","last_change":"2024-04-25T11:19:09.716136+0000","last_active":"2024-04-25T11:19:11.016792+0000","last_peered":"2024-04-25T11:19:11.016792+0000","last_clean":"2024-04-25T11:19:11.016792+0000","last_became_active":"2024-04-25T11:19:09.715126+0000","last_became_peered":"2024-04-25T11:19:09.715126+0000","last_unstale":"2024-04-25T11:19:11.016792+0000","last_undegraded":"2024-04-25T11:19:11.016792+0000","last_fullsized":"2024-04-25T11:19:11.016792+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:03.060032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017545+0000","last_change":"2024-04-25T11:19:08.710525+0000","last_active":"2024-04-25T11:19:11.017545+0000","last_peered":"2024-04-25T11:19:11.017545+0000","last_clean":"2024-04-25T11:19:11.017545+0000","last_became_active":"2024-04-25T11:19:08.710180+0000","last_became_peered":"2024-04-25T11:19:08.710180+0000","last_unstale":"2024-04-25T11:19:11.017545+0000","last_undegraded":"2024-04-25T11:19:11.017545+0000","last_fullsized":"2024-04-25T11:19:11.017545+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:04.053134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.721603+0000","last_change":"2024-04-25T11:19:09.730212+0000","last_active":"2024-04-25T11:19:10.721603+0000","last_peered":"2024-04-25T11:19:10.721603+0000","last_clean":"2024-04-25T11:19:10.721603+0000","last_became_active":"2024-04-25T11:19:09.729741+0000","last_became_peered":"2024-04-25T11:19:09.729741+0000","last_unstale":"2024-04-25T11:19:10.721603+0000","last_undegraded":"2024-04-25T11:19:10.721603+0000","last_fullsized":"2024-04-25T11:19:10.721603+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:52.054644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.720824+0000","last_change":"2024-04-25T11:19:08.714470+0000","last_active":"2024-04-25T11:19:10.720824+0000","last_peered":"2024-04-25T11:19:10.720824+0000","last_clean":"2024-04-25T11:19:10.720824+0000","last_became_active":"2024-04-25T11:19:08.713751+0000","last_became_peered":"2024-04-25T11:19:08.713751+0000","last_unstale":"2024-04-25T11:19:10.720824+0000","last_undegraded":"2024-04-25T11:19:10.720824+0000","last_fullsized":"2024-04-25T11:19:10.720824+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:42.107371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.726242+0000","last_change":"2024-04-25T11:19:09.722954+0000","last_active":"2024-04-25T11:19:09.726242+0000","last_peered":"2024-04-25T11:19:09.726242+0000","last_clean":"2024-04-25T11:19:09.726242+0000","last_became_active":"2024-04-25T11:19:09.721378+0000","last_became_peered":"2024-04-25T11:19:09.721378+0000","last_unstale":"2024-04-25T11:19:09.726242+0000","last_undegraded":"2024-04-25T11:19:09.726242+0000","last_fullsized":"2024-04-25T11:19:09.726242+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:17.076484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.728762+0000","last_change":"2024-04-25T11:19:08.700455+0000","last_active":"2024-04-25T11:19:09.728762+0000","last_peered":"2024-04-25T11:19:09.728762+0000","last_clean":"2024-04-25T11:19:09.728762+0000","last_became_active":"2024-04-25T11:19:08.700082+0000","last_became_peered":"2024-04-25T11:19:08.700082+0000","last_unstale":"2024-04-25T11:19:09.728762+0000","last_undegraded":"2024-04-25T11:19:09.728762+0000","last_fullsized":"2024-04-25T11:19:09.728762+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:25.169079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727999+0000","last_change":"2024-04-25T11:19:09.725542+0000","last_active":"2024-04-25T11:19:09.727999+0000","last_peered":"2024-04-25T11:19:09.727999+0000","last_clean":"2024-04-25T11:19:09.727999+0000","last_became_active":"2024-04-25T11:19:09.725230+0000","last_became_peered":"2024-04-25T11:19:09.725230+0000","last_unstale":"2024-04-25T11:19:09.727999+0000","last_undegraded":"2024-04-25T11:19:09.727999+0000","last_fullsized":"2024-04-25T11:19:09.727999+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:22.367292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.720357+0000","last_change":"2024-04-25T11:19:08.711289+0000","last_active":"2024-04-25T11:19:09.720357+0000","last_peered":"2024-04-25T11:19:09.720357+0000","last_clean":"2024-04-25T11:19:09.720357+0000","last_became_active":"2024-04-25T11:19:08.710956+0000","last_became_peered":"2024-04-25T11:19:08.710956+0000","last_unstale":"2024-04-25T11:19:09.720357+0000","last_undegraded":"2024-04-25T11:19:09.720357+0000","last_fullsized":"2024-04-25T11:19:09.720357+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:21.347701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727631+0000","last_change":"2024-04-25T11:19:09.721805+0000","last_active":"2024-04-25T11:19:09.727631+0000","last_peered":"2024-04-25T11:19:09.727631+0000","last_clean":"2024-04-25T11:19:09.727631+0000","last_became_active":"2024-04-25T11:19:09.721040+0000","last_became_peered":"2024-04-25T11:19:09.721040+0000","last_unstale":"2024-04-25T11:19:09.727631+0000","last_undegraded":"2024-04-25T11:19:09.727631+0000","last_fullsized":"2024-04-25T11:19:09.727631+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:45.937225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724004+0000","last_change":"2024-04-25T11:19:08.706544+0000","last_active":"2024-04-25T11:19:09.724004+0000","last_peered":"2024-04-25T11:19:09.724004+0000","last_clean":"2024-04-25T11:19:09.724004+0000","last_became_active":"2024-04-25T11:19:08.706160+0000","last_became_peered":"2024-04-25T11:19:08.706160+0000","last_unstale":"2024-04-25T11:19:09.724004+0000","last_undegraded":"2024-04-25T11:19:09.724004+0000","last_fullsized":"2024-04-25T11:19:09.724004+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:02.350950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.870325+0000","last_change":"2024-04-25T11:19:09.724104+0000","last_active":"2024-04-25T11:19:10.870325+0000","last_peered":"2024-04-25T11:19:10.870325+0000","last_clean":"2024-04-25T11:19:10.870325+0000","last_became_active":"2024-04-25T11:19:09.723101+0000","last_became_peered":"2024-04-25T11:19:09.723101+0000","last_unstale":"2024-04-25T11:19:10.870325+0000","last_undegraded":"2024-04-25T11:19:10.870325+0000","last_fullsized":"2024-04-25T11:19:10.870325+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:22.367292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.720880+0000","last_change":"2024-04-25T11:19:08.708941+0000","last_active":"2024-04-25T11:19:10.720880+0000","last_peered":"2024-04-25T11:19:10.720880+0000","last_clean":"2024-04-25T11:19:10.720880+0000","last_became_active":"2024-04-25T11:19:08.706905+0000","last_became_peered":"2024-04-25T11:19:08.706905+0000","last_unstale":"2024-04-25T11:19:10.720880+0000","last_undegraded":"2024-04-25T11:19:10.720880+0000","last_fullsized":"2024-04-25T11:19:10.720880+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:02.350950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.725199+0000","last_change":"2024-04-25T11:19:09.723854+0000","last_active":"2024-04-25T11:19:09.725199+0000","last_peered":"2024-04-25T11:19:09.725199+0000","last_clean":"2024-04-25T11:19:09.725199+0000","last_became_active":"2024-04-25T11:19:09.723505+0000","last_became_peered":"2024-04-25T11:19:09.723505+0000","last_unstale":"2024-04-25T11:19:09.725199+0000","last_undegraded":"2024-04-25T11:19:09.725199+0000","last_fullsized":"2024-04-25T11:19:09.725199+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:55.796067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.704222+0000","last_change":"2024-04-25T11:19:08.698071+0000","last_active":"2024-04-25T11:19:09.704222+0000","last_peered":"2024-04-25T11:19:09.704222+0000","last_clean":"2024-04-25T11:19:09.704222+0000","last_became_active":"2024-04-25T11:19:08.697726+0000","last_became_peered":"2024-04-25T11:19:08.697726+0000","last_unstale":"2024-04-25T11:19:09.704222+0000","last_undegraded":"2024-04-25T11:19:09.704222+0000","last_fullsized":"2024-04-25T11:19:09.704222+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:59.495584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727442+0000","last_change":"2024-04-25T11:19:09.720710+0000","last_active":"2024-04-25T11:19:09.727442+0000","last_peered":"2024-04-25T11:19:09.727442+0000","last_clean":"2024-04-25T11:19:09.727442+0000","last_became_active":"2024-04-25T11:19:09.720352+0000","last_became_peered":"2024-04-25T11:19:09.720352+0000","last_unstale":"2024-04-25T11:19:09.727442+0000","last_undegraded":"2024-04-25T11:19:09.727442+0000","last_fullsized":"2024-04-25T11:19:09.727442+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:22.639028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.722029+0000","last_change":"2024-04-25T11:19:08.708614+0000","last_active":"2024-04-25T11:19:10.722029+0000","last_peered":"2024-04-25T11:19:10.722029+0000","last_clean":"2024-04-25T11:19:10.722029+0000","last_became_active":"2024-04-25T11:19:08.707687+0000","last_became_peered":"2024-04-25T11:19:08.707687+0000","last_unstale":"2024-04-25T11:19:10.722029+0000","last_undegraded":"2024-04-25T11:19:10.722029+0000","last_fullsized":"2024-04-25T11:19:10.722029+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:37.873967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.715984+0000","last_change":"2024-04-25T11:19:09.703672+0000","last_active":"2024-04-25T11:19:09.715984+0000","last_peered":"2024-04-25T11:19:09.715984+0000","last_clean":"2024-04-25T11:19:09.715984+0000","last_became_active":"2024-04-25T11:19:09.703220+0000","last_became_peered":"2024-04-25T11:19:09.703220+0000","last_unstale":"2024-04-25T11:19:09.715984+0000","last_undegraded":"2024-04-25T11:19:09.715984+0000","last_fullsized":"2024-04-25T11:19:09.715984+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:46.676680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.707897+0000","last_change":"2024-04-25T11:19:08.708515+0000","last_active":"2024-04-25T11:19:09.707897+0000","last_peered":"2024-04-25T11:19:09.707897+0000","last_clean":"2024-04-25T11:19:09.707897+0000","last_became_active":"2024-04-25T11:19:08.708238+0000","last_became_peered":"2024-04-25T11:19:08.708238+0000","last_unstale":"2024-04-25T11:19:09.707897+0000","last_undegraded":"2024-04-25T11:19:09.707897+0000","last_fullsized":"2024-04-25T11:19:09.707897+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:19.821060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017177+0000","last_change":"2024-04-25T11:19:09.715409+0000","last_active":"2024-04-25T11:19:11.017177+0000","last_peered":"2024-04-25T11:19:11.017177+0000","last_clean":"2024-04-25T11:19:11.017177+0000","last_became_active":"2024-04-25T11:19:09.715104+0000","last_became_peered":"2024-04-25T11:19:09.715104+0000","last_unstale":"2024-04-25T11:19:11.017177+0000","last_undegraded":"2024-04-25T11:19:11.017177+0000","last_fullsized":"2024-04-25T11:19:11.017177+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:22.848256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.729545+0000","last_change":"2024-04-25T11:19:08.715017+0000","last_active":"2024-04-25T11:19:09.729545+0000","last_peered":"2024-04-25T11:19:09.729545+0000","last_clean":"2024-04-25T11:19:09.729545+0000","last_became_active":"2024-04-25T11:19:08.714696+0000","last_became_peered":"2024-04-25T11:19:08.714696+0000","last_unstale":"2024-04-25T11:19:09.729545+0000","last_undegraded":"2024-04-25T11:19:09.729545+0000","last_fullsized":"2024-04-25T11:19:09.729545+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:54.776476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.716583+0000","last_change":"2024-04-25T11:19:09.703394+0000","last_active":"2024-04-25T11:19:09.716583+0000","last_peered":"2024-04-25T11:19:09.716583+0000","last_clean":"2024-04-25T11:19:09.716583+0000","last_became_active":"2024-04-25T11:19:09.702476+0000","last_became_peered":"2024-04-25T11:19:09.702476+0000","last_unstale":"2024-04-25T11:19:09.716583+0000","last_undegraded":"2024-04-25T11:19:09.716583+0000","last_fullsized":"2024-04-25T11:19:09.716583+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:56.814924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.869387+0000","last_change":"2024-04-25T11:19:08.714074+0000","last_active":"2024-04-25T11:19:10.869387+0000","last_peered":"2024-04-25T11:19:10.869387+0000","last_clean":"2024-04-25T11:19:10.869387+0000","last_became_active":"2024-04-25T11:19:08.713485+0000","last_became_peered":"2024-04-25T11:19:08.713485+0000","last_unstale":"2024-04-25T11:19:10.869387+0000","last_undegraded":"2024-04-25T11:19:10.869387+0000","last_fullsized":"2024-04-25T11:19:10.869387+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:25.169079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.726002+0000","last_change":"2024-04-25T11:19:09.720431+0000","last_active":"2024-04-25T11:19:09.726002+0000","last_peered":"2024-04-25T11:19:09.726002+0000","last_clean":"2024-04-25T11:19:09.726002+0000","last_became_active":"2024-04-25T11:19:09.720097+0000","last_became_peered":"2024-04-25T11:19:09.720097+0000","last_unstale":"2024-04-25T11:19:09.726002+0000","last_undegraded":"2024-04-25T11:19:09.726002+0000","last_fullsized":"2024-04-25T11:19:09.726002+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:20.924888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.729392+0000","last_change":"2024-04-25T11:19:08.707366+0000","last_active":"2024-04-25T11:19:09.729392+0000","last_peered":"2024-04-25T11:19:09.729392+0000","last_clean":"2024-04-25T11:19:09.729392+0000","last_became_active":"2024-04-25T11:19:08.707047+0000","last_became_peered":"2024-04-25T11:19:08.707047+0000","last_unstale":"2024-04-25T11:19:09.729392+0000","last_undegraded":"2024-04-25T11:19:09.729392+0000","last_fullsized":"2024-04-25T11:19:09.729392+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:09.125332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.869851+0000","last_change":"2024-04-25T11:19:09.714775+0000","last_active":"2024-04-25T11:19:10.869851+0000","last_peered":"2024-04-25T11:19:10.869851+0000","last_clean":"2024-04-25T11:19:10.869851+0000","last_became_active":"2024-04-25T11:19:09.711665+0000","last_became_peered":"2024-04-25T11:19:09.711665+0000","last_unstale":"2024-04-25T11:19:10.869851+0000","last_undegraded":"2024-04-25T11:19:10.869851+0000","last_fullsized":"2024-04-25T11:19:10.869851+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:24.992694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.706985+0000","last_change":"2024-04-25T11:19:08.699583+0000","last_active":"2024-04-25T11:19:09.706985+0000","last_peered":"2024-04-25T11:19:09.706985+0000","last_clean":"2024-04-25T11:19:09.706985+0000","last_became_active":"2024-04-25T11:19:08.697764+0000","last_became_peered":"2024-04-25T11:19:08.697764+0000","last_unstale":"2024-04-25T11:19:09.706985+0000","last_undegraded":"2024-04-25T11:19:09.706985+0000","last_fullsized":"2024-04-25T11:19:09.706985+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:59.079566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016924+0000","last_change":"2024-04-25T11:19:09.713065+0000","last_active":"2024-04-25T11:19:11.016924+0000","last_peered":"2024-04-25T11:19:11.016924+0000","last_clean":"2024-04-25T11:19:11.016924+0000","last_became_active":"2024-04-25T11:19:09.712730+0000","last_became_peered":"2024-04-25T11:19:09.712730+0000","last_unstale":"2024-04-25T11:19:11.016924+0000","last_undegraded":"2024-04-25T11:19:11.016924+0000","last_fullsized":"2024-04-25T11:19:11.016924+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:11.612910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017278+0000","last_change":"2024-04-25T11:19:08.704569+0000","last_active":"2024-04-25T11:19:11.017278+0000","last_peered":"2024-04-25T11:19:11.017278+0000","last_clean":"2024-04-25T11:19:11.017278+0000","last_became_active":"2024-04-25T11:19:08.704204+0000","last_became_peered":"2024-04-25T11:19:08.704204+0000","last_unstale":"2024-04-25T11:19:11.017278+0000","last_undegraded":"2024-04-25T11:19:11.017278+0000","last_fullsized":"2024-04-25T11:19:11.017278+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:12.947432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.730018+0000","last_change":"2024-04-25T11:19:09.723317+0000","last_active":"2024-04-25T11:19:09.730018+0000","last_peered":"2024-04-25T11:19:09.730018+0000","last_clean":"2024-04-25T11:19:09.730018+0000","last_became_active":"2024-04-25T11:19:09.723015+0000","last_became_peered":"2024-04-25T11:19:09.723015+0000","last_unstale":"2024-04-25T11:19:09.730018+0000","last_undegraded":"2024-04-25T11:19:09.730018+0000","last_fullsized":"2024-04-25T11:19:09.730018+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:09.229793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.720269+0000","last_change":"2024-04-25T11:19:08.712591+0000","last_active":"2024-04-25T11:19:10.720269+0000","last_peered":"2024-04-25T11:19:10.720269+0000","last_clean":"2024-04-25T11:19:10.720269+0000","last_became_active":"2024-04-25T11:19:08.712114+0000","last_became_peered":"2024-04-25T11:19:08.712114+0000","last_unstale":"2024-04-25T11:19:10.720269+0000","last_undegraded":"2024-04-25T11:19:10.720269+0000","last_fullsized":"2024-04-25T11:19:10.720269+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:45.657089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017224+0000","last_change":"2024-04-25T11:19:09.705557+0000","last_active":"2024-04-25T11:19:11.017224+0000","last_peered":"2024-04-25T11:19:11.017224+0000","last_clean":"2024-04-25T11:19:11.017224+0000","last_became_active":"2024-04-25T11:19:09.705200+0000","last_became_peered":"2024-04-25T11:19:09.705200+0000","last_unstale":"2024-04-25T11:19:11.017224+0000","last_undegraded":"2024-04-25T11:19:11.017224+0000","last_fullsized":"2024-04-25T11:19:11.017224+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:06.816077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.870047+0000","last_change":"2024-04-25T11:19:08.715055+0000","last_active":"2024-04-25T11:19:10.870047+0000","last_peered":"2024-04-25T11:19:10.870047+0000","last_clean":"2024-04-25T11:19:10.870047+0000","last_became_active":"2024-04-25T11:19:08.714750+0000","last_became_peered":"2024-04-25T11:19:08.714750+0000","last_unstale":"2024-04-25T11:19:10.870047+0000","last_undegraded":"2024-04-25T11:19:10.870047+0000","last_fullsized":"2024-04-25T11:19:10.870047+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:45.657089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.868958+0000","last_change":"2024-04-25T11:19:09.716609+0000","last_active":"2024-04-25T11:19:10.868958+0000","last_peered":"2024-04-25T11:19:10.868958+0000","last_clean":"2024-04-25T11:19:10.868958+0000","last_became_active":"2024-04-25T11:19:09.716261+0000","last_became_peered":"2024-04-25T11:19:09.716261+0000","last_unstale":"2024-04-25T11:19:10.868958+0000","last_undegraded":"2024-04-25T11:19:10.868958+0000","last_fullsized":"2024-04-25T11:19:10.868958+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:00.099157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.725730+0000","last_change":"2024-04-25T11:19:08.708933+0000","last_active":"2024-04-25T11:19:09.725730+0000","last_peered":"2024-04-25T11:19:09.725730+0000","last_clean":"2024-04-25T11:19:09.725730+0000","last_became_active":"2024-04-25T11:19:08.708577+0000","last_became_peered":"2024-04-25T11:19:08.708577+0000","last_unstale":"2024-04-25T11:19:09.725730+0000","last_undegraded":"2024-04-25T11:19:09.725730+0000","last_fullsized":"2024-04-25T11:19:09.725730+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:04.645618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724642+0000","last_change":"2024-04-25T11:19:09.703849+0000","last_active":"2024-04-25T11:19:09.724642+0000","last_peered":"2024-04-25T11:19:09.724642+0000","last_clean":"2024-04-25T11:19:09.724642+0000","last_became_active":"2024-04-25T11:19:09.702990+0000","last_became_peered":"2024-04-25T11:19:09.702990+0000","last_unstale":"2024-04-25T11:19:09.724642+0000","last_undegraded":"2024-04-25T11:19:09.724642+0000","last_fullsized":"2024-04-25T11:19:09.724642+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:38.893558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.709178+0000","last_change":"2024-04-25T11:19:08.713021+0000","last_active":"2024-04-25T11:19:09.709178+0000","last_peered":"2024-04-25T11:19:09.709178+0000","last_clean":"2024-04-25T11:19:09.709178+0000","last_became_active":"2024-04-25T11:19:08.712663+0000","last_became_peered":"2024-04-25T11:19:08.712663+0000","last_unstale":"2024-04-25T11:19:09.709178+0000","last_undegraded":"2024-04-25T11:19:09.709178+0000","last_fullsized":"2024-04-25T11:19:09.709178+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:59.942017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.727516+0000","last_change":"2024-04-25T11:19:09.720420+0000","last_active":"2024-04-25T11:19:09.727516+0000","last_peered":"2024-04-25T11:19:09.727516+0000","last_clean":"2024-04-25T11:19:09.727516+0000","last_became_active":"2024-04-25T11:19:09.720099+0000","last_became_peered":"2024-04-25T11:19:09.720099+0000","last_unstale":"2024-04-25T11:19:09.727516+0000","last_undegraded":"2024-04-25T11:19:09.727516+0000","last_fullsized":"2024-04-25T11:19:09.727516+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:45.126171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.018330+0000","last_change":"2024-04-25T11:19:08.707288+0000","last_active":"2024-04-25T11:19:11.018330+0000","last_peered":"2024-04-25T11:19:11.018330+0000","last_clean":"2024-04-25T11:19:11.018330+0000","last_became_active":"2024-04-25T11:19:08.706963+0000","last_became_peered":"2024-04-25T11:19:08.706963+0000","last_unstale":"2024-04-25T11:19:11.018330+0000","last_undegraded":"2024-04-25T11:19:11.018330+0000","last_fullsized":"2024-04-25T11:19:11.018330+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:57.697723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.721603+0000","last_change":"2024-04-25T11:19:09.708506+0000","last_active":"2024-04-25T11:19:09.721603+0000","last_peered":"2024-04-25T11:19:09.721603+0000","last_clean":"2024-04-25T11:19:09.721603+0000","last_became_active":"2024-04-25T11:19:09.708162+0000","last_became_peered":"2024-04-25T11:19:09.708162+0000","last_unstale":"2024-04-25T11:19:09.721603+0000","last_undegraded":"2024-04-25T11:19:09.721603+0000","last_fullsized":"2024-04-25T11:19:09.721603+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:53.262229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016553+0000","last_change":"2024-04-25T11:19:08.698313+0000","last_active":"2024-04-25T11:19:11.016553+0000","last_peered":"2024-04-25T11:19:11.016553+0000","last_clean":"2024-04-25T11:19:11.016553+0000","last_became_active":"2024-04-25T11:19:08.697950+0000","last_became_peered":"2024-04-25T11:19:08.697950+0000","last_unstale":"2024-04-25T11:19:11.016553+0000","last_undegraded":"2024-04-25T11:19:11.016553+0000","last_fullsized":"2024-04-25T11:19:11.016553+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:58.014541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724409+0000","last_change":"2024-04-25T11:19:09.720265+0000","last_active":"2024-04-25T11:19:09.724409+0000","last_peered":"2024-04-25T11:19:09.724409+0000","last_clean":"2024-04-25T11:19:09.724409+0000","last_became_active":"2024-04-25T11:19:09.719926+0000","last_became_peered":"2024-04-25T11:19:09.719926+0000","last_unstale":"2024-04-25T11:19:09.724409+0000","last_undegraded":"2024-04-25T11:19:09.724409+0000","last_fullsized":"2024-04-25T11:19:09.724409+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:36.384892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.868824+0000","last_change":"2024-04-25T11:19:08.715722+0000","last_active":"2024-04-25T11:19:10.868824+0000","last_peered":"2024-04-25T11:19:10.868824+0000","last_clean":"2024-04-25T11:19:10.868824+0000","last_became_active":"2024-04-25T11:19:08.714740+0000","last_became_peered":"2024-04-25T11:19:08.714740+0000","last_unstale":"2024-04-25T11:19:10.868824+0000","last_undegraded":"2024-04-25T11:19:10.868824+0000","last_fullsized":"2024-04-25T11:19:10.868824+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:42.107371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017859+0000","last_change":"2024-04-25T11:19:09.716107+0000","last_active":"2024-04-25T11:19:11.017859+0000","last_peered":"2024-04-25T11:19:11.017859+0000","last_clean":"2024-04-25T11:19:11.017859+0000","last_became_active":"2024-04-25T11:19:09.715223+0000","last_became_peered":"2024-04-25T11:19:09.715223+0000","last_unstale":"2024-04-25T11:19:11.017859+0000","last_undegraded":"2024-04-25T11:19:11.017859+0000","last_fullsized":"2024-04-25T11:19:11.017859+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:51.484231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.016882+0000","last_change":"2024-04-25T11:19:08.698434+0000","last_active":"2024-04-25T11:19:11.016882+0000","last_peered":"2024-04-25T11:19:11.016882+0000","last_clean":"2024-04-25T11:19:11.016882+0000","last_became_active":"2024-04-25T11:19:08.698069+0000","last_became_peered":"2024-04-25T11:19:08.698069+0000","last_unstale":"2024-04-25T11:19:11.016882+0000","last_undegraded":"2024-04-25T11:19:11.016882+0000","last_fullsized":"2024-04-25T11:19:11.016882+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:02.881374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.723957+0000","last_change":"2024-04-25T11:19:09.714095+0000","last_active":"2024-04-25T11:19:09.723957+0000","last_peered":"2024-04-25T11:19:09.723957+0000","last_clean":"2024-04-25T11:19:09.723957+0000","last_became_active":"2024-04-25T11:19:09.713552+0000","last_became_peered":"2024-04-25T11:19:09.713552+0000","last_unstale":"2024-04-25T11:19:09.723957+0000","last_undegraded":"2024-04-25T11:19:09.723957+0000","last_fullsized":"2024-04-25T11:19:09.723957+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:20.533817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.729454+0000","last_change":"2024-04-25T11:19:08.705001+0000","last_active":"2024-04-25T11:19:09.729454+0000","last_peered":"2024-04-25T11:19:09.729454+0000","last_clean":"2024-04-25T11:19:09.729454+0000","last_became_active":"2024-04-25T11:19:08.703645+0000","last_became_peered":"2024-04-25T11:19:08.703645+0000","last_unstale":"2024-04-25T11:19:09.729454+0000","last_undegraded":"2024-04-25T11:19:09.729454+0000","last_fullsized":"2024-04-25T11:19:09.729454+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:42.107371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.720489+0000","last_change":"2024-04-25T11:19:09.720750+0000","last_active":"2024-04-25T11:19:10.720489+0000","last_peered":"2024-04-25T11:19:10.720489+0000","last_clean":"2024-04-25T11:19:10.720489+0000","last_became_active":"2024-04-25T11:19:09.720422+0000","last_became_peered":"2024-04-25T11:19:09.720422+0000","last_unstale":"2024-04-25T11:19:10.720489+0000","last_undegraded":"2024-04-25T11:19:10.720489+0000","last_fullsized":"2024-04-25T11:19:10.720489+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:26.953283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.708073+0000","last_change":"2024-04-25T11:19:08.715297+0000","last_active":"2024-04-25T11:19:09.708073+0000","last_peered":"2024-04-25T11:19:09.708073+0000","last_clean":"2024-04-25T11:19:09.708073+0000","last_became_active":"2024-04-25T11:19:08.714853+0000","last_became_peered":"2024-04-25T11:19:08.714853+0000","last_unstale":"2024-04-25T11:19:09.708073+0000","last_undegraded":"2024-04-25T11:19:09.708073+0000","last_fullsized":"2024-04-25T11:19:09.708073+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:36.110794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724937+0000","last_change":"2024-04-25T11:19:09.721488+0000","last_active":"2024-04-25T11:19:09.724937+0000","last_peered":"2024-04-25T11:19:09.724937+0000","last_clean":"2024-04-25T11:19:09.724937+0000","last_became_active":"2024-04-25T11:19:09.720577+0000","last_became_peered":"2024-04-25T11:19:09.720577+0000","last_unstale":"2024-04-25T11:19:09.724937+0000","last_undegraded":"2024-04-25T11:19:09.724937+0000","last_fullsized":"2024-04-25T11:19:09.724937+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:55.241977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.869778+0000","last_change":"2024-04-25T11:19:08.711679+0000","last_active":"2024-04-25T11:19:10.869778+0000","last_peered":"2024-04-25T11:19:10.869778+0000","last_clean":"2024-04-25T11:19:10.869778+0000","last_became_active":"2024-04-25T11:19:08.710796+0000","last_became_peered":"2024-04-25T11:19:08.710796+0000","last_unstale":"2024-04-25T11:19:10.869778+0000","last_undegraded":"2024-04-25T11:19:10.869778+0000","last_fullsized":"2024-04-25T11:19:10.869778+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:09.125332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.723131+0000","last_change":"2024-04-25T11:19:09.716679+0000","last_active":"2024-04-25T11:19:09.723131+0000","last_peered":"2024-04-25T11:19:09.723131+0000","last_clean":"2024-04-25T11:19:09.723131+0000","last_became_active":"2024-04-25T11:19:09.714281+0000","last_became_peered":"2024-04-25T11:19:09.714281+0000","last_unstale":"2024-04-25T11:19:09.723131+0000","last_undegraded":"2024-04-25T11:19:09.723131+0000","last_fullsized":"2024-04-25T11:19:09.723131+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:52.626259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.690181+0000","last_change":"2024-04-25T11:19:08.712088+0000","last_active":"2024-04-25T11:19:09.690181+0000","last_peered":"2024-04-25T11:19:09.690181+0000","last_clean":"2024-04-25T11:19:09.690181+0000","last_became_active":"2024-04-25T11:19:08.711696+0000","last_became_peered":"2024-04-25T11:19:08.711696+0000","last_unstale":"2024-04-25T11:19:09.690181+0000","last_undegraded":"2024-04-25T11:19:09.690181+0000","last_fullsized":"2024-04-25T11:19:09.690181+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:41.646122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017056+0000","last_change":"2024-04-25T11:19:09.717055+0000","last_active":"2024-04-25T11:19:11.017056+0000","last_peered":"2024-04-25T11:19:11.017056+0000","last_clean":"2024-04-25T11:19:11.017056+0000","last_became_active":"2024-04-25T11:19:09.716567+0000","last_became_peered":"2024-04-25T11:19:09.716567+0000","last_unstale":"2024-04-25T11:19:11.017056+0000","last_undegraded":"2024-04-25T11:19:11.017056+0000","last_fullsized":"2024-04-25T11:19:11.017056+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:34.145608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.720822+0000","last_change":"2024-04-25T11:19:08.710120+0000","last_active":"2024-04-25T11:19:10.720822+0000","last_peered":"2024-04-25T11:19:10.720822+0000","last_clean":"2024-04-25T11:19:10.720822+0000","last_became_active":"2024-04-25T11:19:08.709776+0000","last_became_peered":"2024-04-25T11:19:08.709776+0000","last_unstale":"2024-04-25T11:19:10.720822+0000","last_undegraded":"2024-04-25T11:19:10.720822+0000","last_fullsized":"2024-04-25T11:19:10.720822+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:55.795333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017825+0000","last_change":"2024-04-25T11:19:09.711296+0000","last_active":"2024-04-25T11:19:11.017825+0000","last_peered":"2024-04-25T11:19:11.017825+0000","last_clean":"2024-04-25T11:19:11.017825+0000","last_became_active":"2024-04-25T11:19:09.710981+0000","last_became_peered":"2024-04-25T11:19:09.710981+0000","last_unstale":"2024-04-25T11:19:11.017825+0000","last_undegraded":"2024-04-25T11:19:11.017825+0000","last_fullsized":"2024-04-25T11:19:11.017825+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:39.203607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017054+0000","last_change":"2024-04-25T11:19:08.704110+0000","last_active":"2024-04-25T11:19:11.017054+0000","last_peered":"2024-04-25T11:19:11.017054+0000","last_clean":"2024-04-25T11:19:11.017054+0000","last_became_active":"2024-04-25T11:19:08.703615+0000","last_became_peered":"2024-04-25T11:19:08.703615+0000","last_unstale":"2024-04-25T11:19:11.017054+0000","last_undegraded":"2024-04-25T11:19:11.017054+0000","last_fullsized":"2024-04-25T11:19:11.017054+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:01.092687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.725339+0000","last_change":"2024-04-25T11:19:09.720844+0000","last_active":"2024-04-25T11:19:09.725339+0000","last_peered":"2024-04-25T11:19:09.725339+0000","last_clean":"2024-04-25T11:19:09.725339+0000","last_became_active":"2024-04-25T11:19:09.720532+0000","last_became_peered":"2024-04-25T11:19:09.720532+0000","last_unstale":"2024-04-25T11:19:09.725339+0000","last_undegraded":"2024-04-25T11:19:09.725339+0000","last_fullsized":"2024-04-25T11:19:09.725339+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:22:03.663753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:10.869023+0000","last_change":"2024-04-25T11:19:08.710917+0000","last_active":"2024-04-25T11:19:10.869023+0000","last_peered":"2024-04-25T11:19:10.869023+0000","last_clean":"2024-04-25T11:19:10.869023+0000","last_became_active":"2024-04-25T11:19:08.710541+0000","last_became_peered":"2024-04-25T11:19:08.710541+0000","last_unstale":"2024-04-25T11:19:10.869023+0000","last_undegraded":"2024-04-25T11:19:10.869023+0000","last_fullsized":"2024-04-25T11:19:10.869023+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:55.795333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.722208+0000","last_change":"2024-04-25T11:19:09.718243+0000","last_active":"2024-04-25T11:19:09.722208+0000","last_peered":"2024-04-25T11:19:09.722208+0000","last_clean":"2024-04-25T11:19:09.722208+0000","last_became_active":"2024-04-25T11:19:09.717755+0000","last_became_peered":"2024-04-25T11:19:09.717755+0000","last_unstale":"2024-04-25T11:19:09.722208+0000","last_undegraded":"2024-04-25T11:19:09.722208+0000","last_fullsized":"2024-04-25T11:19:09.722208+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:21.308335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017210+0000","last_change":"2024-04-25T11:19:08.705550+0000","last_active":"2024-04-25T11:19:11.017210+0000","last_peered":"2024-04-25T11:19:11.017210+0000","last_clean":"2024-04-25T11:19:11.017210+0000","last_became_active":"2024-04-25T11:19:08.705237+0000","last_became_peered":"2024-04-25T11:19:08.705237+0000","last_unstale":"2024-04-25T11:19:11.017210+0000","last_undegraded":"2024-04-25T11:19:11.017210+0000","last_fullsized":"2024-04-25T11:19:11.017210+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:41.620818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.724047+0000","last_change":"2024-04-25T11:19:09.718648+0000","last_active":"2024-04-25T11:19:09.724047+0000","last_peered":"2024-04-25T11:19:09.724047+0000","last_clean":"2024-04-25T11:19:09.724047+0000","last_became_active":"2024-04-25T11:19:09.715913+0000","last_became_peered":"2024-04-25T11:19:09.715913+0000","last_unstale":"2024-04-25T11:19:09.724047+0000","last_undegraded":"2024-04-25T11:19:09.724047+0000","last_fullsized":"2024-04-25T11:19:09.724047+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:08.651212+0000","last_clean_scrub_stamp":"2024-04-25T11:19:08.651212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:07.906131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T11:19:11.017052+0000","last_change":"2024-04-25T11:19:08.709112+0000","last_active":"2024-04-25T11:19:11.017052+0000","last_peered":"2024-04-25T11:19:11.017052+0000","last_clean":"2024-04-25T11:19:11.017052+0000","last_became_active":"2024-04-25T11:19:08.708338+0000","last_became_peered":"2024-04-25T11:19:08.708338+0000","last_unstale":"2024-04-25T11:19:11.017052+0000","last_undegraded":"2024-04-25T11:19:11.017052+0000","last_fullsized":"2024-04-25T11:19:11.017052+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:07.631621+0000","last_clean_scrub_stamp":"2024-04-25T11:19:07.631621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:24.203477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":278,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-25T11:19:09.687528+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:19:09.687528+0000","last_peered":"2024-04-25T11:19:09.687528+0000","last_clean":"2024-04-25T11:19:09.687528+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:19:09.687528+0000","last_undegraded":"2024-04-25T11:19:09.687528+0000","last_fullsized":"2024-04-25T11:19:09.687528+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476846,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1892352,"data_stored":1183177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476847,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1310720,"data_stored":594865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476846,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":592649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476847,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1298432,"data_stored":592741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509550,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":593231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509551,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":585482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509550,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1871872,"data_stored":1175850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542256,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1306624,"data_stored":599375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:19:15.069 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:19:15.070 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:19:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.403+0000 7fd404400640 1 Processor -- start 2024-04-25T11:19:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.403+0000 7fd404400640 1 -- start start 2024-04-25T11:19:15.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.406+0000 7fd404400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3fc09f790 0x7fd3fc15dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:15.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.406+0000 7fd404400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd3fc070d60 0x7fd3fc06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:15.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.406+0000 7fd404400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc06dac0 0x7fd3fc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.406+0000 7fd404400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd3fc15e2a0 con 0x7fd3fc09f790 2024-04-25T11:19:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.406+0000 7fd404400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd3fc074550 con 0x7fd3fc070d60 2024-04-25T11:19:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.406+0000 7fd404400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd3fc06f220 con 0x7fd3fc06dac0 2024-04-25T11:19:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.406+0000 7fd401a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3fc09f790 0x7fd3fc15dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.406+0000 7fd401a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3fc09f790 0x7fd3fc15dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51766/0 (socket says 172.21.15.7:51766) 2024-04-25T11:19:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.406+0000 7fd401a00640 1 -- 172.21.15.7:0/144091810 learned_addr learned my addr 172.21.15.7:0/144091810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.407+0000 7fd402400640 1 --2- 172.21.15.7:0/144091810 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc06dac0 0x7fd3fc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.407+0000 7fd401000640 1 --2- 172.21.15.7:0/144091810 >> v2:172.21.15.87:3300/0 conn(0x7fd3fc070d60 0x7fd3fc06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.407+0000 7fd401a00640 1 -- 172.21.15.7:0/144091810 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc06dac0 msgr2=0x7fd3fc0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.407+0000 7fd401a00640 1 --2- 172.21.15.7:0/144091810 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc06dac0 0x7fd3fc0ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.407+0000 7fd401a00640 1 -- 172.21.15.7:0/144091810 >> v2:172.21.15.87:3300/0 conn(0x7fd3fc070d60 msgr2=0x7fd3fc06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.407+0000 7fd401a00640 1 --2- 172.21.15.7:0/144091810 >> v2:172.21.15.87:3300/0 conn(0x7fd3fc070d60 0x7fd3fc06d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.407+0000 7fd401a00640 1 -- 172.21.15.7:0/144091810 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc06ded0 con 0x7fd3fc09f790 2024-04-25T11:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.407+0000 7fd402400640 1 --2- 172.21.15.7:0/144091810 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc06dac0 0x7fd3fc0ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.409+0000 7fd401a00640 1 --2- 172.21.15.7:0/144091810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc09f790 0x7fd3fc15dd60 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0731d0 tx=0x7fd3f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980d1087a33ec6d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.409+0000 7fd3fbe00640 1 -- 172.21.15.7:0/144091810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0037070 con 0x7fd3fc09f790 2024-04-25T11:19:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.409+0000 7fd3fbe00640 1 -- 172.21.15.7:0/144091810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f0032440 con 0x7fd3fc09f790 2024-04-25T11:19:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.409+0000 7fd3fbe00640 1 -- 172.21.15.7:0/144091810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0032810 con 0x7fd3fc09f790 2024-04-25T11:19:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.410+0000 7fd404400640 1 -- 172.21.15.7:0/144091810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc09f790 msgr2=0x7fd3fc15dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.410+0000 7fd404400640 1 --2- 172.21.15.7:0/144091810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc09f790 0x7fd3fc15dd60 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc0731d0 tx=0x7fd3f002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.410+0000 7fd404400640 1 -- 172.21.15.7:0/144091810 shutdown_connections 2024-04-25T11:19:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.410+0000 7fd404400640 1 --2- 172.21.15.7:0/144091810 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc06dac0 0x7fd3fc0ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.410+0000 7fd404400640 1 --2- 172.21.15.7:0/144091810 >> v2:172.21.15.87:3300/0 conn(0x7fd3fc070d60 0x7fd3fc06d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.410+0000 7fd404400640 1 --2- 172.21.15.7:0/144091810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc09f790 0x7fd3fc15dd60 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.410+0000 7fd404400640 1 -- 172.21.15.7:0/144091810 >> 172.21.15.7:0/144091810 conn(0x7fd3fc0d1d40 msgr2=0x7fd3fc073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.410+0000 7fd404400640 1 -- 172.21.15.7:0/144091810 shutdown_connections 2024-04-25T11:19:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.410+0000 7fd404400640 1 -- 172.21.15.7:0/144091810 wait complete. 2024-04-25T11:19:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.413+0000 7fd404400640 1 Processor -- start 2024-04-25T11:19:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.413+0000 7fd404400640 1 -- start start 2024-04-25T11:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.414+0000 7fd404400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc06dac0 0x7fd3fc0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd404400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc070d60 0x7fd3fc15ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd404400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3fc09f790 0x7fd3fc14bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd404400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc06fa30 con 0x7fd3fc06dac0 2024-04-25T11:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd404400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc06f750 con 0x7fd3fc09f790 2024-04-25T11:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd404400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd3fc06fd10 con 0x7fd3fc070d60 2024-04-25T11:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd401a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc06dac0 0x7fd3fc0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd401a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc06dac0 0x7fd3fc0d00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51776/0 (socket says 172.21.15.7:51776) 2024-04-25T11:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd401a00640 1 -- 172.21.15.7:0/4267229491 learned_addr learned my addr 172.21.15.7:0/4267229491 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd401a00640 1 -- 172.21.15.7:0/4267229491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc070d60 msgr2=0x7fd3fc15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd401a00640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc070d60 0x7fd3fc15ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd401a00640 1 -- 172.21.15.7:0/4267229491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3fc09f790 msgr2=0x7fd3fc14bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd401a00640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3fc09f790 0x7fd3fc14bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.415+0000 7fd401a00640 1 -- 172.21.15.7:0/4267229491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc15e780 con 0x7fd3fc06dac0 2024-04-25T11:19:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.416+0000 7fd401a00640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc06dac0 0x7fd3fc0d00d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.416+0000 7fd3f9600640 1 -- 172.21.15.7:0/4267229491 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0041020 con 0x7fd3fc06dac0 2024-04-25T11:19:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.416+0000 7fd3f9600640 1 -- 172.21.15.7:0/4267229491 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f00306c0 con 0x7fd3fc06dac0 2024-04-25T11:19:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.416+0000 7fd404400640 1 -- 172.21.15.7:0/4267229491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3fc15ea60 con 0x7fd3fc06dac0 2024-04-25T11:19:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.416+0000 7fd3f9600640 1 -- 172.21.15.7:0/4267229491 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0037040 con 0x7fd3fc06dac0 2024-04-25T11:19:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.416+0000 7fd404400640 1 -- 172.21.15.7:0/4267229491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3fc15f020 con 0x7fd3fc06dac0 2024-04-25T11:19:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.417+0000 7fd404400640 1 -- 172.21.15.7:0/4267229491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3fc050740 con 0x7fd3fc06dac0 2024-04-25T11:19:15.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.419+0000 7fd3f9600640 1 -- 172.21.15.7:0/4267229491 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd3f0044020 con 0x7fd3fc06dac0 2024-04-25T11:19:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.420+0000 7fd3f9600640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3e00744b0 0x7fd3e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:15.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.420+0000 7fd3f9600640 1 -- 172.21.15.7:0/4267229491 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fd3f00b5c60 con 0x7fd3fc06dac0 2024-04-25T11:19:15.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.421+0000 7fd401000640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3e00744b0 0x7fd3e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:15.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.421+0000 7fd401000640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3e00744b0 0x7fd3e00769c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec006fd0 tx=0x7fd3ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:15.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.425+0000 7fd3f9600640 1 -- 172.21.15.7:0/4267229491 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd3f004b030 con 0x7fd3fc06dac0 2024-04-25T11:19:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.710+0000 7fd404400640 1 -- 172.21.15.7:0/4267229491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd3fc158320 con 0x7fd3fc06dac0 2024-04-25T11:19:15.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.717+0000 7fd3f9600640 1 -- 172.21.15.7:0/4267229491 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7fd3fc06ded0 con 0x7fd3fc06dac0 2024-04-25T11:19:15.716 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-25T11:19:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:19:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-04-25T11:19:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-04-25T11:19:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:19:15.717 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 -- 172.21.15.7:0/4267229491 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3e00744b0 msgr2=0x7fd3e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3e00744b0 0x7fd3e00769c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec006fd0 tx=0x7fd3ec008040 comp rx=0 tx=0).stop 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 -- 172.21.15.7:0/4267229491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc06dac0 msgr2=0x7fd3fc0d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc06dac0 0x7fd3fc0d00d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f00046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 -- 172.21.15.7:0/4267229491 shutdown_connections 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd3e00744b0 0x7fd3e00769c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3fc09f790 0x7fd3fc14bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd3fc070d60 0x7fd3fc15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 --2- 172.21.15.7:0/4267229491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3fc06dac0 0x7fd3fc0d00d0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 -- 172.21.15.7:0/4267229491 >> 172.21.15.7:0/4267229491 conn(0x7fd3fc0d1d40 msgr2=0x7fd3fc0b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 -- 172.21.15.7:0/4267229491 shutdown_connections 2024-04-25T11:19:15.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:15.720+0000 7fd404400640 1 -- 172.21.15.7:0/4267229491 wait complete. 2024-04-25T11:19:15.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.030+0000 7fe96fe00640 1 Processor -- start 2024-04-25T11:19:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.030+0000 7fe96fe00640 1 -- start start 2024-04-25T11:19:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.032+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe970164690 0x7fe970070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.032+0000 7fe96fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe970156cd0 0x7fe970163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.032+0000 7fe96fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9700b5080 con 0x7fe9701570e0 2024-04-25T11:19:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.032+0000 7fe96fe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe9700b5360 con 0x7fe970156cd0 2024-04-25T11:19:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.032+0000 7fe96fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe9700b5640 con 0x7fe970164690 2024-04-25T11:19:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.033+0000 7fe96e000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe970156cd0 0x7fe970163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.033+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe970164690 0x7fe970070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.033+0000 7fe96e000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe970156cd0 0x7fe970163ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:59242/0 (socket says 172.21.15.7:59242) 2024-04-25T11:19:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.033+0000 7fe96e000640 1 -- 172.21.15.7:0/2825821562 learned_addr learned my addr 172.21.15.7:0/2825821562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.033+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe970164690 0x7fe970070a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:32790/0 (socket says 172.21.15.7:32790) 2024-04-25T11:19:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.033+0000 7fe96f400640 1 --1- 172.21.15.7:0/2825821562 >> v1:172.21.15.87:6789/0 conn(0x7fe9701570e0 0x7fe9700b0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:55056/0 (socket says 172.21.15.7:55056) 2024-04-25T11:19:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.034+0000 7fe96ea00640 1 -- 172.21.15.7:0/2825821562 >> v2:172.21.15.87:3300/0 conn(0x7fe970156cd0 msgr2=0x7fe970163ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.034+0000 7fe96ea00640 1 --2- 172.21.15.7:0/2825821562 >> v2:172.21.15.87:3300/0 conn(0x7fe970156cd0 0x7fe970163ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.034+0000 7fe96ea00640 1 -- 172.21.15.7:0/2825821562 >> v1:172.21.15.87:6789/0 conn(0x7fe9701570e0 legacy=0x7fe9700b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.034+0000 7fe96ea00640 1 -- 172.21.15.7:0/2825821562 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9700b5920 con 0x7fe970164690 2024-04-25T11:19:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.034+0000 7fe96e000640 1 --2- 172.21.15.7:0/2825821562 >> v2:172.21.15.87:3300/0 conn(0x7fe970156cd0 0x7fe970163ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.036+0000 7fe96ea00640 1 --2- 172.21.15.7:0/2825821562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe970164690 0x7fe970070a20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097a0 tx=0x7fe96002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68eac39b75f6d150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.036+0000 7fe96d600640 1 -- 172.21.15.7:0/2825821562 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960037070 con 0x7fe970164690 2024-04-25T11:19:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.036+0000 7fe96d600640 1 -- 172.21.15.7:0/2825821562 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96002ae50 con 0x7fe970164690 2024-04-25T11:19:16.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.036+0000 7fe96d600640 1 -- 172.21.15.7:0/2825821562 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960032660 con 0x7fe970164690 2024-04-25T11:19:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.036+0000 7fe96fe00640 1 -- 172.21.15.7:0/2825821562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe970164690 msgr2=0x7fe970070a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.036+0000 7fe96fe00640 1 --2- 172.21.15.7:0/2825821562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe970164690 0x7fe970070a20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe9600097a0 tx=0x7fe96002a100 comp rx=0 tx=0).stop 2024-04-25T11:19:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.036+0000 7fe96fe00640 1 -- 172.21.15.7:0/2825821562 shutdown_connections 2024-04-25T11:19:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.036+0000 7fe96fe00640 1 --2- 172.21.15.7:0/2825821562 >> v2:172.21.15.87:3300/0 conn(0x7fe970156cd0 0x7fe970163ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.036+0000 7fe96fe00640 1 --2- 172.21.15.7:0/2825821562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe970164690 0x7fe970070a20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.036+0000 7fe96fe00640 1 -- 172.21.15.7:0/2825821562 >> 172.21.15.7:0/2825821562 conn(0x7fe9700b3820 msgr2=0x7fe9700b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.038+0000 7fe96fe00640 1 -- 172.21.15.7:0/2825821562 shutdown_connections 2024-04-25T11:19:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.038+0000 7fe96fe00640 1 -- 172.21.15.7:0/2825821562 wait complete. 2024-04-25T11:19:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.040+0000 7fe96fe00640 1 Processor -- start 2024-04-25T11:19:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.040+0000 7fe96fe00640 1 -- start start 2024-04-25T11:19:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe970156cd0 0x7fe9701f9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9701570e0 0x7fe9701453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe970164690 0x7fe97006cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe970164010 con 0x7fe970156cd0 2024-04-25T11:19:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe970070f60 con 0x7fe970164690 2024-04-25T11:19:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe970164190 con 0x7fe9701570e0 2024-04-25T11:19:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe970156cd0 0x7fe9701f9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe970156cd0 0x7fe9701f9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51790/0 (socket says 172.21.15.7:51790) 2024-04-25T11:19:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96ea00640 1 -- 172.21.15.7:0/664584820 learned_addr learned my addr 172.21.15.7:0/664584820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96f400640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe970164690 0x7fe97006cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96ea00640 1 -- 172.21.15.7:0/664584820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9701570e0 msgr2=0x7fe9701453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.042+0000 7fe96e000640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9701570e0 0x7fe9701453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.043+0000 7fe96ea00640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9701570e0 0x7fe9701453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.043+0000 7fe96ea00640 1 -- 172.21.15.7:0/664584820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe970164690 msgr2=0x7fe97006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.043+0000 7fe96ea00640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe970164690 0x7fe97006cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.043+0000 7fe96ea00640 1 -- 172.21.15.7:0/664584820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9701fa110 con 0x7fe970156cd0 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.043+0000 7fe96e000640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9701570e0 0x7fe9701453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.043+0000 7fe96ea00640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe970156cd0 0x7fe9701f9bd0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe9600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.043+0000 7fe966a00640 1 -- 172.21.15.7:0/664584820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960041070 con 0x7fe970156cd0 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.043+0000 7fe966a00640 1 -- 172.21.15.7:0/664584820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96003c460 con 0x7fe970156cd0 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.043+0000 7fe966a00640 1 -- 172.21.15.7:0/664584820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960037040 con 0x7fe970156cd0 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.043+0000 7fe96fe00640 1 -- 172.21.15.7:0/664584820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9701fa590 con 0x7fe970156cd0 2024-04-25T11:19:16.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.043+0000 7fe96fe00640 1 -- 172.21.15.7:0/664584820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe970153a10 con 0x7fe970156cd0 2024-04-25T11:19:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.045+0000 7fe96fe00640 1 -- 172.21.15.7:0/664584820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe970050740 con 0x7fe970156cd0 2024-04-25T11:19:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.046+0000 7fe966a00640 1 -- 172.21.15.7:0/664584820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe960044070 con 0x7fe970156cd0 2024-04-25T11:19:16.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.047+0000 7fe966a00640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe9500744b0 0x7fe9500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:16.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.047+0000 7fe966a00640 1 -- 172.21.15.7:0/664584820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fe9600b5a10 con 0x7fe970156cd0 2024-04-25T11:19:16.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.047+0000 7fe96e000640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe9500744b0 0x7fe9500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:16.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.048+0000 7fe96e000640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe9500744b0 0x7fe9500769c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe9701610f0 tx=0x7fe95c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:16.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.053+0000 7fe966a00640 1 -- 172.21.15.7:0/664584820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe960080000 con 0x7fe970156cd0 2024-04-25T11:19:16.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.296+0000 7fe96fe00640 1 -- 172.21.15.7:0/664584820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9700b1140 con 0x7fe970156cd0 2024-04-25T11:19:16.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.298+0000 7fe966a00640 1 -- 172.21.15.7:0/664584820 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5171 (secure 0 0 0) 0x7fe96007f900 con 0x7fe970156cd0 2024-04-25T11:19:16.297 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:16.297 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5846,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/4132210096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":4132210096},{"type":"v1","addr":"172.21.15.87:6834","nonce":4132210096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2657338269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2657338269},{"type":"v1","addr":"172.21.15.87:6835","nonce":2657338269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6457,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/138020048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":138020048},{"type":"v1","addr":"172.21.15.7:6836","nonce":138020048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:09.244634+0000","modified":"2024-04-25T11:19:12.318984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/990441706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":990441706},{"type":"v1","addr":"172.21.15.7:6837","nonce":990441706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7623,"qdb_cluster":[7623]},"id":7}]} 2024-04-25T11:19:16.297 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 67 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 -- 172.21.15.7:0/664584820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe9500744b0 msgr2=0x7fe9500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe9500744b0 0x7fe9500769c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fe9701610f0 tx=0x7fe95c008040 comp rx=0 tx=0).stop 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 -- 172.21.15.7:0/664584820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe970156cd0 msgr2=0x7fe9701f9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe970156cd0 0x7fe9701f9bd0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fe960000c00 tx=0x7fe9600046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 -- 172.21.15.7:0/664584820 shutdown_connections 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe9500744b0 0x7fe9500769c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe970164690 0x7fe97006cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9701570e0 0x7fe9701453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 --2- 172.21.15.7:0/664584820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe970156cd0 0x7fe9701f9bd0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 -- 172.21.15.7:0/664584820 >> 172.21.15.7:0/664584820 conn(0x7fe9700b3820 msgr2=0x7fe970162ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 -- 172.21.15.7:0/664584820 shutdown_connections 2024-04-25T11:19:16.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:16.301+0000 7fe96fe00640 1 -- 172.21.15.7:0/664584820 wait complete. 2024-04-25T11:19:16.727 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:19:09.244634+0000', 'modified': '2024-04-25T11:19:12.318984+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7623}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7623': {'gid': 7623, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6837/990441706', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 990441706}, {'type': 'v1', 'addr': '172.21.15.7:6837', 'nonce': 990441706}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7623, 'qdb_cluster': [7623]} 2024-04-25T11:19:16.728 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:19:16.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:19:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.026+0000 7f0784a00640 1 Processor -- start 2024-04-25T11:19:17.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.027+0000 7f0784a00640 1 -- start start 2024-04-25T11:19:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.029+0000 7f0784a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07800d0330 0x7f07800d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.029+0000 7f0784a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f078006f3f0 0x7f078006f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.029+0000 7f0784a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07800d0c80 con 0x7f078006c570 2024-04-25T11:19:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.029+0000 7f0784a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f07800b5c20 con 0x7f078006f3f0 2024-04-25T11:19:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.029+0000 7f0784a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f07800b5f00 con 0x7f07800d0330 2024-04-25T11:19:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.030+0000 7f077f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f078006c570 0x7f07800b53b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37986/0 (socket says 172.21.15.7:37986) 2024-04-25T11:19:17.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.030+0000 7f077f400640 1 -- 172.21.15.7:0/748720686 learned_addr learned my addr 172.21.15.7:0/748720686 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.030+0000 7f077fe00640 1 --2- 172.21.15.7:0/748720686 >> v2:172.21.15.87:3300/0 conn(0x7f078006f3f0 0x7f078006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.030+0000 7f077ea00640 1 --2- 172.21.15.7:0/748720686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07800d0330 0x7f07800d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.030+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982322260 0 0) 0x7f0770001490 con 0x7f078006c570 2024-04-25T11:19:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0764003520 con 0x7f078006c570 2024-04-25T11:19:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 567913606 0 0) 0x7f0764003520 con 0x7f078006c570 2024-04-25T11:19:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07800d0330 msgr2=0x7f07800d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 --2- 172.21.15.7:0/748720686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07800d0330 0x7f07800d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 >> v2:172.21.15.87:3300/0 conn(0x7f078006f3f0 msgr2=0x7f078006f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 --2- 172.21.15.7:0/748720686 >> v2:172.21.15.87:3300/0 conn(0x7f078006f3f0 0x7f078006f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07800b61e0 con 0x7f078006c570 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0770001bc0 con 0x7f078006c570 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 >> v1:172.21.15.7:6789/0 conn(0x7f078006c570 legacy=0x7f07800b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 --2- 172.21.15.7:0/748720686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07640039b0 0x7f0764003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 --2- 172.21.15.7:0/748720686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07800d0330 0x7f0764004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 --2- 172.21.15.7:0/748720686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 0x7f0764005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07800b5c20 con 0x7f0764004d40 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07800b5f00 con 0x7f07640039b0 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f076400b9f0 con 0x7f07800d0330 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.031+0000 7f077fe00640 1 --2- 172.21.15.7:0/748720686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 0x7f0764005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.032+0000 7f077f400640 1 --2- 172.21.15.7:0/748720686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07640039b0 0x7f0764003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.032+0000 7f077fe00640 1 -- 172.21.15.7:0/748720686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07800d0330 msgr2=0x7f0764004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.032+0000 7f077fe00640 1 --2- 172.21.15.7:0/748720686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07800d0330 0x7f0764004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.032+0000 7f077fe00640 1 -- 172.21.15.7:0/748720686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07640039b0 msgr2=0x7f0764003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.032+0000 7f077fe00640 1 --2- 172.21.15.7:0/748720686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07640039b0 0x7f0764003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.032+0000 7f077fe00640 1 -- 172.21.15.7:0/748720686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0764003520 con 0x7f0764004d40 2024-04-25T11:19:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.033+0000 7f077fe00640 1 --2- 172.21.15.7:0/748720686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 0x7f0764005150 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f0780074220 tx=0x7f0764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7dca2806cf27a9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.033+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778036070 con 0x7f0764004d40 2024-04-25T11:19:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.033+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0778031400 con 0x7f0764004d40 2024-04-25T11:19:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.033+0000 7f077e000640 1 -- 172.21.15.7:0/748720686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778009690 con 0x7f0764004d40 2024-04-25T11:19:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.034+0000 7f0784a00640 1 -- 172.21.15.7:0/748720686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 msgr2=0x7f0764005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.034+0000 7f0784a00640 1 --2- 172.21.15.7:0/748720686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 0x7f0764005150 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f0780074220 tx=0x7f0764005830 comp rx=0 tx=0).stop 2024-04-25T11:19:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.034+0000 7f077fe00640 1 -- 172.21.15.7:0/748720686 reap_dead start 2024-04-25T11:19:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.034+0000 7f0784a00640 1 -- 172.21.15.7:0/748720686 shutdown_connections 2024-04-25T11:19:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.034+0000 7f0784a00640 1 -- 172.21.15.7:0/748720686 >> 172.21.15.7:0/748720686 conn(0x7f07800d1540 msgr2=0x7f07800d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.034+0000 7f0784a00640 1 -- 172.21.15.7:0/748720686 shutdown_connections 2024-04-25T11:19:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.034+0000 7f0784a00640 1 -- 172.21.15.7:0/748720686 wait complete. 2024-04-25T11:19:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.037+0000 7f0784a00640 1 Processor -- start 2024-04-25T11:19:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.037+0000 7f0784a00640 1 -- start start 2024-04-25T11:19:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.038+0000 7f0784a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 0x7f07801f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.038+0000 7f0784a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07801f62c0 0x7f07800c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.038+0000 7f0784a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07800c7450 0x7f07800af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.038+0000 7f0784a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07801432a0 con 0x7f0764004d40 2024-04-25T11:19:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.038+0000 7f0784a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07800b67a0 con 0x7f07800c7450 2024-04-25T11:19:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f0784a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0780143420 con 0x7f07801f62c0 2024-04-25T11:19:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f077f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 0x7f07801f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f077f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 0x7f07801f5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51816/0 (socket says 172.21.15.7:51816) 2024-04-25T11:19:17.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f077f400640 1 -- 172.21.15.7:0/1386167132 learned_addr learned my addr 172.21.15.7:0/1386167132 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f077fe00640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07800c7450 0x7f07800af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f077ea00640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07801f62c0 0x7f07800c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f077f400640 1 -- 172.21.15.7:0/1386167132 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07801f62c0 msgr2=0x7f07800c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f077f400640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07801f62c0 0x7f07800c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f077f400640 1 -- 172.21.15.7:0/1386167132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07800c7450 msgr2=0x7f07800af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f077f400640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07800c7450 0x7f07800af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f077f400640 1 -- 172.21.15.7:0/1386167132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07800c7c00 con 0x7f0764004d40 2024-04-25T11:19:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.039+0000 7f077fe00640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07800c7450 0x7f07800af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.040+0000 7f077f400640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 0x7f07801f5d80 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f0770005fd0 tx=0x7f077000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:17.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.040+0000 7f0777400640 1 -- 172.21.15.7:0/1386167132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770017020 con 0x7f0764004d40 2024-04-25T11:19:17.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.040+0000 7f0777400640 1 -- 172.21.15.7:0/1386167132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077000ee50 con 0x7f0764004d40 2024-04-25T11:19:17.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.040+0000 7f0777400640 1 -- 172.21.15.7:0/1386167132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770013640 con 0x7f0764004d40 2024-04-25T11:19:17.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.040+0000 7f0784a00640 1 -- 172.21.15.7:0/1386167132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07800c7ee0 con 0x7f0764004d40 2024-04-25T11:19:17.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.040+0000 7f0784a00640 1 -- 172.21.15.7:0/1386167132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07801f6a60 con 0x7f0764004d40 2024-04-25T11:19:17.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.043+0000 7f0777400640 1 -- 172.21.15.7:0/1386167132 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f077001a020 con 0x7f0764004d40 2024-04-25T11:19:17.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.044+0000 7f0777400640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07600744b0 0x7f07600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.044+0000 7f0777400640 1 -- 172.21.15.7:0/1386167132 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f0770095d30 con 0x7f0764004d40 2024-04-25T11:19:17.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.044+0000 7f077ea00640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07600744b0 0x7f07600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.045+0000 7f078cfab740 1 Processor -- start 2024-04-25T11:19:17.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.045+0000 7f078cfab740 1 -- start start 2024-04-25T11:19:17.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.045+0000 7f077ea00640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07600744b0 0x7f07600769c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f076c004770 tx=0x7f076c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:17.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.046+0000 7f078cfab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589390a8270 0x558939475210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.046+0000 7f078cfab740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55893907f3b0 0x55893948ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.046+0000 7f078cfab740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558939087590 0x558939490480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.046+0000 7f078cfab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5589393ae600 con 0x5589390a8270 2024-04-25T11:19:17.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.046+0000 7f078cfab740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55893934b100 con 0x558939087590 2024-04-25T11:19:17.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.046+0000 7f078cfab740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5589393aa5c0 con 0x55893907f3b0 2024-04-25T11:19:17.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.046+0000 7f077f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589390a8270 0x558939475210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.046+0000 7f077f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589390a8270 0x558939475210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51832/0 (socket says 172.21.15.7:51832) 2024-04-25T11:19:17.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.047+0000 7f077f400640 1 -- 172.21.15.7:0/1854730903 learned_addr learned my addr 172.21.15.7:0/1854730903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.047+0000 7f077ea00640 1 --2- 172.21.15.7:0/1854730903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55893907f3b0 0x55893948ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.047+0000 7f077f400640 1 -- 172.21.15.7:0/1854730903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55893907f3b0 msgr2=0x55893948ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.047+0000 7f077f400640 1 --2- 172.21.15.7:0/1854730903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55893907f3b0 0x55893948ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.047+0000 7f077f400640 1 -- 172.21.15.7:0/1854730903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558939087590 msgr2=0x558939490480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.047+0000 7f077f400640 1 --2- 172.21.15.7:0/1854730903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558939087590 0x558939490480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.047+0000 7f077f400640 1 -- 172.21.15.7:0/1854730903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589390c02d0 con 0x5589390a8270 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.047+0000 7f077f400640 1 --2- 172.21.15.7:0/1854730903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589390a8270 0x558939475210 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f077005bd30 tx=0x7f07700683c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0be3ea0f145691 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.047+0000 7f076b400640 1 -- 172.21.15.7:0/1854730903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077005fcb0 con 0x5589390a8270 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.048+0000 7f076b400640 1 -- 172.21.15.7:0/1854730903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0770064060 con 0x5589390a8270 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.048+0000 7f076b400640 1 -- 172.21.15.7:0/1854730903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770068a80 con 0x5589390a8270 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.048+0000 7f078cfab740 1 -- 172.21.15.7:0/1854730903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589390a8270 msgr2=0x558939475210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.048+0000 7f078cfab740 1 --2- 172.21.15.7:0/1854730903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589390a8270 0x558939475210 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f077005bd30 tx=0x7f07700683c0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.048+0000 7f078cfab740 1 -- 172.21.15.7:0/1854730903 shutdown_connections 2024-04-25T11:19:17.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.048+0000 7f078cfab740 1 --2- 172.21.15.7:0/1854730903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558939087590 0x558939490480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.048+0000 7f078cfab740 1 --2- 172.21.15.7:0/1854730903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55893907f3b0 0x55893948ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.048+0000 7f078cfab740 1 --2- 172.21.15.7:0/1854730903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5589390a8270 0x558939475210 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.048+0000 7f078cfab740 1 -- 172.21.15.7:0/1854730903 >> 172.21.15.7:0/1854730903 conn(0x55893909c3c0 msgr2=0x55893944a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.049+0000 7f078cfab740 1 -- 172.21.15.7:0/1854730903 shutdown_connections 2024-04-25T11:19:17.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.049+0000 7f078cfab740 1 -- 172.21.15.7:0/1854730903 wait complete. 2024-04-25T11:19:17.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.050+0000 7f078cfab740 1 -- start start 2024-04-25T11:19:17.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.050+0000 7f078cfab740 1 Processor -- start 2024-04-25T11:19:17.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.052+0000 7f078cfab740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55893907f3b0 0x55893948bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.052+0000 7f078cfab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558939087590 0x558939467930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.052+0000 7f078cfab740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5589390a8270 0x55893948e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.052+0000 7f078cfab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558939335f70 con 0x558939087590 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.052+0000 7f078cfab740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558939467e70 con 0x55893907f3b0 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.052+0000 7f078cfab740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55893948ca60 con 0x5589390a8270 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.052+0000 7f078cfab740 10 client.0 _ll_register_callbacks cb 0x5589393a4480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.052+0000 7f077ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558939087590 0x558939467930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.052+0000 7f077ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558939087590 0x558939467930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51840/0 (socket says 172.21.15.7:51840) 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.052+0000 7f077ea00640 1 -- 172.21.15.7:0/455953916 learned_addr learned my addr 172.21.15.7:0/455953916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.052+0000 7f077f400640 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55893907f3b0 0x55893948bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f077ea00640 1 -- 172.21.15.7:0/455953916 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5589390a8270 msgr2=0x55893948e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f077fe00640 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5589390a8270 0x55893948e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f077ea00640 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5589390a8270 0x55893948e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f077ea00640 1 -- 172.21.15.7:0/455953916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55893907f3b0 msgr2=0x55893948bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f077ea00640 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55893907f3b0 0x55893948bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f077ea00640 1 -- 172.21.15.7:0/455953916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55893918a6b0 con 0x558939087590 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f077f400640 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55893907f3b0 0x55893948bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f077fe00640 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5589390a8270 0x55893948e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f077ea00640 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558939087590 0x558939467930 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f076c015720 tx=0x7f076c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e035890d349efce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f075fe00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f075fe00640 1 -- 172.21.15.7:0/455953916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c00f040 con 0x558939087590 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f078cfab740 1 -- 172.21.15.7:0/455953916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0770064060 con 0x558939087590 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.053+0000 7f075fe00640 1 -- 172.21.15.7:0/455953916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f076c016830 con 0x558939087590 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.054+0000 7f075fe00640 1 -- 172.21.15.7:0/455953916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f076c02f050 con 0x558939087590 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.054+0000 7f078cfab740 10 client.7758 fetch_fsmap learned FSMap version 67 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.054+0000 7f078cfab740 1 -- 172.21.15.7:0/455953916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x558939468290 con 0x558939087590 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.054+0000 7f075fe00640 1 -- 172.21.15.7:0/455953916 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0770064060 con 0x558939087590 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.054+0000 7f075fe00640 1 -- 172.21.15.7:0/455953916 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f076c032020 con 0x558939087590 2024-04-25T11:19:17.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.054+0000 7f078cfab740 10 client.7758 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T11:19:17.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.054+0000 7f078cfab740 10 client.7758 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:19:17.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.054+0000 7f078cfab740 20 client.7758 populate_metadata read hostname 'smithi007' 2024-04-25T11:19:17.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.054+0000 7f078cfab740 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.7:6834/990441706,v1:172.21.15.7:6837/990441706] conn(0x558939066e40 0x55893948da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.054+0000 7f078cfab740 4 client.7758 mds_command: new command op to 7623 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:19:17.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.054+0000 7f078cfab740 1 -- 172.21.15.7:0/455953916 --> [v2:172.21.15.7:6834/990441706,v1:172.21.15.7:6837/990441706] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589393bc210 con 0x558939066e40 2024-04-25T11:19:17.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.055+0000 7f077f400640 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.7:6834/990441706,v1:172.21.15.7:6837/990441706] conn(0x558939066e40 0x55893948da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.055+0000 7f077f400640 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.7:6834/990441706,v1:172.21.15.7:6837/990441706] conn(0x558939066e40 0x55893948da40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f69b2914af9087 server_cookie=3a66f7b612867bd0 in_seq=0 out_seq=0 2024-04-25T11:19:17.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.055+0000 7f075fe00640 10 client.7758 ms_handle_connect on v2:172.21.15.7:6834/990441706 2024-04-25T11:19:17.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.056+0000 7f075fe00640 1 -- 172.21.15.7:0/455953916 <== mds.0 v2:172.21.15.7:6834/990441706 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5589393bc210 con 0x558939066e40 2024-04-25T11:19:17.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.056+0000 7f075fe00640 10 client.7758 handle_command_reply: tid=0 2024-04-25T11:19:17.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.056+0000 7f078cfab740 1 client.7758 shutdown 2024-04-25T11:19:17.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.056+0000 7f078cfab740 1 -- 172.21.15.7:0/455953916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558939087590 msgr2=0x558939467930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.056+0000 7f078cfab740 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558939087590 0x558939467930 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f076c015720 tx=0x7f076c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.057+0000 7f078cfab740 1 -- 172.21.15.7:0/455953916 shutdown_connections 2024-04-25T11:19:17.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.057+0000 7f078cfab740 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.7:6834/990441706,v1:172.21.15.7:6837/990441706] conn(0x558939066e40 0x55893948da40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.057+0000 7f078cfab740 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5589390a8270 0x55893948e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.057+0000 7f078cfab740 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558939087590 0x558939467930 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.057+0000 7f078cfab740 1 --2- 172.21.15.7:0/455953916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55893907f3b0 0x55893948bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.057+0000 7f075fe00640 0 client.7758 ms_handle_reset on v2:172.21.15.7:6834/990441706 2024-04-25T11:19:17.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.057+0000 7f078cfab740 1 -- 172.21.15.7:0/455953916 >> 172.21.15.7:0/455953916 conn(0x55893909c3c0 msgr2=0x55893944a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:17.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.057+0000 7f078cfab740 1 -- 172.21.15.7:0/455953916 shutdown_connections 2024-04-25T11:19:17.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.057+0000 7f078cfab740 1 -- 172.21.15.7:0/455953916 wait complete. 2024-04-25T11:19:17.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.057+0000 7f078cfab740 20 client.7758 trim_cache size 0 max 16384 2024-04-25T11:19:17.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.078+0000 7f078cfab740 1 Processor -- start 2024-04-25T11:19:17.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.078+0000 7f078cfab740 1 -- start start 2024-04-25T11:19:17.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.079+0000 7f078cfab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.079+0000 7f078cfab740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558939087590 0x5589394918d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.079+0000 7f078cfab740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5589390a8270 0x558939491e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.079+0000 7f078cfab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07802028e0 con 0x55893907f3b0 2024-04-25T11:19:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.079+0000 7f078cfab740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07801f8270 con 0x558939087590 2024-04-25T11:19:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.079+0000 7f078cfab740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0780202290 con 0x5589390a8270 2024-04-25T11:19:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.079+0000 7f077f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.080+0000 7f077f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51842/0 (socket says 172.21.15.7:51842) 2024-04-25T11:19:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.080+0000 7f077f400640 1 -- 172.21.15.7:0/2349468387 learned_addr learned my addr 172.21.15.7:0/2349468387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.080+0000 7f077f400640 1 -- 172.21.15.7:0/2349468387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5589390a8270 msgr2=0x558939491e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.080+0000 7f077f400640 1 --2- 172.21.15.7:0/2349468387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5589390a8270 0x558939491e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.080+0000 7f077ea00640 1 --2- 172.21.15.7:0/2349468387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558939087590 0x5589394918d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.080+0000 7f077f400640 1 -- 172.21.15.7:0/2349468387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558939087590 msgr2=0x5589394918d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.080+0000 7f077f400640 1 --2- 172.21.15.7:0/2349468387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558939087590 0x5589394918d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.080+0000 7f077f400640 1 -- 172.21.15.7:0/2349468387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893948ba20 con 0x55893907f3b0 2024-04-25T11:19:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.080+0000 7f077ea00640 1 --2- 172.21.15.7:0/2349468387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558939087590 0x5589394918d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.080+0000 7f077f400640 1 --2- 172.21.15.7:0/2349468387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f077005c2a0 tx=0x7f0770095000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20477625e64ffd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.081+0000 7f076aa00640 1 -- 172.21.15.7:0/2349468387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077005fcb0 con 0x55893907f3b0 2024-04-25T11:19:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.081+0000 7f076aa00640 1 -- 172.21.15.7:0/2349468387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077005bf40 con 0x55893907f3b0 2024-04-25T11:19:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.081+0000 7f076aa00640 1 -- 172.21.15.7:0/2349468387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07700813b0 con 0x55893907f3b0 2024-04-25T11:19:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.081+0000 7f078cfab740 1 -- 172.21.15.7:0/2349468387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 msgr2=0x558939477920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.081+0000 7f078cfab740 1 --2- 172.21.15.7:0/2349468387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f077005c2a0 tx=0x7f0770095000 comp rx=0 tx=0).stop 2024-04-25T11:19:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.082+0000 7f078cfab740 1 -- 172.21.15.7:0/2349468387 shutdown_connections 2024-04-25T11:19:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.082+0000 7f078cfab740 1 --2- 172.21.15.7:0/2349468387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5589390a8270 0x558939491e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.082+0000 7f078cfab740 1 --2- 172.21.15.7:0/2349468387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558939087590 0x5589394918d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.082+0000 7f078cfab740 1 --2- 172.21.15.7:0/2349468387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.082+0000 7f078cfab740 1 -- 172.21.15.7:0/2349468387 >> 172.21.15.7:0/2349468387 conn(0x55893909c3c0 msgr2=0x558939474910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.082+0000 7f078cfab740 1 -- 172.21.15.7:0/2349468387 shutdown_connections 2024-04-25T11:19:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.082+0000 7f078cfab740 1 -- 172.21.15.7:0/2349468387 wait complete. 2024-04-25T11:19:17.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.083+0000 7f078cfab740 1 -- start start 2024-04-25T11:19:17.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.083+0000 7f078cfab740 1 Processor -- start 2024-04-25T11:19:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.084+0000 7f078cfab740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.084+0000 7f078cfab740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x558939087590 0x5589394b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.084+0000 7f078cfab740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5589390a8270 0x55893948d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.084+0000 7f078cfab740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55893948a640 con 0x55893907f3b0 2024-04-25T11:19:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.084+0000 7f078cfab740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55893948c650 con 0x5589390a8270 2024-04-25T11:19:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.084+0000 7f078cfab740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55893948ac90 con 0x558939087590 2024-04-25T11:19:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.084+0000 7f078cfab740 10 client.0 _ll_register_callbacks cb 0x7f076c006dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:19:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.085+0000 7f077f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.085+0000 7f077f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51856/0 (socket says 172.21.15.7:51856) 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.085+0000 7f077f400640 1 -- 172.21.15.7:0/2174430456 learned_addr learned my addr 172.21.15.7:0/2174430456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.085+0000 7f077fe00640 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5589390a8270 0x55893948d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.085+0000 7f077ea00640 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x558939087590 0x5589394b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.085+0000 7f077f400640 1 -- 172.21.15.7:0/2174430456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x558939087590 msgr2=0x5589394b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.085+0000 7f077f400640 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x558939087590 0x5589394b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.085+0000 7f077f400640 1 -- 172.21.15.7:0/2174430456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5589390a8270 msgr2=0x55893948d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.085+0000 7f077f400640 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5589390a8270 0x55893948d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.085+0000 7f077f400640 1 -- 172.21.15.7:0/2174430456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589394fed50 con 0x55893907f3b0 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.085+0000 7f077fe00640 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5589390a8270 0x55893948d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.086+0000 7f077f400640 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f077005cf20 tx=0x7f0770062040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c105c687c0594bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.086+0000 7f075fe00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.086+0000 7f075fe00640 1 -- 172.21.15.7:0/2174430456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07700813b0 con 0x55893907f3b0 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.086+0000 7f075fe00640 1 -- 172.21.15.7:0/2174430456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f077005ef10 con 0x55893907f3b0 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.086+0000 7f078cfab740 1 -- 172.21.15.7:0/2174430456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f077005bf40 con 0x55893907f3b0 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.086+0000 7f075fe00640 1 -- 172.21.15.7:0/2174430456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f077005fcb0 con 0x55893907f3b0 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.086+0000 7f078cfab740 10 client.7764 fetch_fsmap learned FSMap version 67 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.086+0000 7f078cfab740 1 -- 172.21.15.7:0/2174430456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5589394fed50 con 0x55893907f3b0 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.087+0000 7f075fe00640 1 -- 172.21.15.7:0/2174430456 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077005bf40 con 0x55893907f3b0 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.087+0000 7f075fe00640 1 -- 172.21.15.7:0/2174430456 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f0770066270 con 0x55893907f3b0 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.087+0000 7f078cfab740 10 client.7764 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.087+0000 7f078cfab740 10 client.7764 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.087+0000 7f078cfab740 20 client.7764 populate_metadata read hostname 'smithi007' 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.087+0000 7f078cfab740 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.7:6834/990441706,v1:172.21.15.7:6837/990441706] conn(0x5589394757d0 0x55893948dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.087+0000 7f078cfab740 4 client.7764 mds_command: new command op to 7623 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.087+0000 7f078cfab740 1 -- 172.21.15.7:0/2174430456 --> [v2:172.21.15.7:6834/990441706,v1:172.21.15.7:6837/990441706] -- command(tid 0: {"prefix": "status"}) -- 0x5589393b6080 con 0x5589394757d0 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.087+0000 7f077ea00640 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.7:6834/990441706,v1:172.21.15.7:6837/990441706] conn(0x5589394757d0 0x55893948dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f077ea00640 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.7:6834/990441706,v1:172.21.15.7:6837/990441706] conn(0x5589394757d0 0x55893948dee0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b1d00c8adca7929 server_cookie=8fa8f5f77cb71ef2 in_seq=0 out_seq=0 2024-04-25T11:19:17.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f075fe00640 10 client.7764 ms_handle_connect on v2:172.21.15.7:6834/990441706 2024-04-25T11:19:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f075fe00640 1 -- 172.21.15.7:0/2174430456 <== mds.0 v2:172.21.15.7:6834/990441706 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5589393b6080 con 0x5589394757d0 2024-04-25T11:19:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f075fe00640 10 client.7764 handle_command_reply: tid=0 2024-04-25T11:19:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f078cfab740 1 client.7764 shutdown 2024-04-25T11:19:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f078cfab740 1 -- 172.21.15.7:0/2174430456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 msgr2=0x558939477920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f078cfab740 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f077005cf20 tx=0x7f0770062040 comp rx=0 tx=0).stop 2024-04-25T11:19:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f078cfab740 1 -- 172.21.15.7:0/2174430456 shutdown_connections 2024-04-25T11:19:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f078cfab740 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.7:6834/990441706,v1:172.21.15.7:6837/990441706] conn(0x5589394757d0 0x55893948dee0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f078cfab740 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5589390a8270 0x55893948d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f078cfab740 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x558939087590 0x5589394b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f078cfab740 1 --2- 172.21.15.7:0/2174430456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55893907f3b0 0x558939477920 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f075fe00640 0 client.7764 ms_handle_reset on v2:172.21.15.7:6834/990441706 2024-04-25T11:19:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.088+0000 7f078cfab740 1 -- 172.21.15.7:0/2174430456 >> 172.21.15.7:0/2174430456 conn(0x55893909c3c0 msgr2=0x558939474910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f078cfab740 1 -- 172.21.15.7:0/2174430456 shutdown_connections 2024-04-25T11:19:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f078cfab740 1 -- 172.21.15.7:0/2174430456 wait complete. 2024-04-25T11:19:17.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f078cfab740 20 client.7764 trim_cache size 0 max 16384 2024-04-25T11:19:17.090 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:19:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:19:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:19:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 7623, 2024-04-25T11:19:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:19:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:19:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:19:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.4111285130000004, 2024-04-25T11:19:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 67, 2024-04-25T11:19:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 106, 2024-04-25T11:19:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 106, 2024-04-25T11:19:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.576831311999999 2024-04-25T11:19:17.091 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:19:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f0784a00640 1 -- 172.21.15.7:0/1386167132 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07600744b0 msgr2=0x7f07600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f0784a00640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07600744b0 0x7f07600769c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f076c004770 tx=0x7f076c008040 comp rx=0 tx=0).stop 2024-04-25T11:19:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f0784a00640 1 -- 172.21.15.7:0/1386167132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 msgr2=0x7f07801f5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f0784a00640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 0x7f07801f5d80 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f0770005fd0 tx=0x7f077000e530 comp rx=0 tx=0).stop 2024-04-25T11:19:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f0784a00640 1 -- 172.21.15.7:0/1386167132 shutdown_connections 2024-04-25T11:19:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f0784a00640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07600744b0 0x7f07600769c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f0784a00640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07800c7450 0x7f07800af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f0784a00640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07801f62c0 0x7f07800c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f0784a00640 1 --2- 172.21.15.7:0/1386167132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0764004d40 0x7f07801f5d80 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.089+0000 7f0784a00640 1 -- 172.21.15.7:0/1386167132 >> 172.21.15.7:0/1386167132 conn(0x7f07800d1540 msgr2=0x7f0780070490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.090+0000 7f0784a00640 1 -- 172.21.15.7:0/1386167132 shutdown_connections 2024-04-25T11:19:17.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.090+0000 7f0784a00640 1 -- 172.21.15.7:0/1386167132 wait complete. 2024-04-25T11:19:17.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:19:17.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.390+0000 7f5e8fe00640 1 Processor -- start 2024-04-25T11:19:17.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.390+0000 7f5e8fe00640 1 -- start start 2024-04-25T11:19:17.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.392+0000 7f5e8fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e90070810 0x7f5e9006d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.392+0000 7f5e8fe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e900cffa0 con 0x7f5e90070c20 2024-04-25T11:19:17.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.392+0000 7f5e8fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e9006ded0 con 0x7f5e9006dac0 2024-04-25T11:19:17.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.392+0000 7f5e8fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5e90160d90 con 0x7f5e90070810 2024-04-25T11:19:17.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.392+0000 7f5e8e000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5e90070c20 0x7f5e9015d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37994/0 (socket says 172.21.15.7:37994) 2024-04-25T11:19:17.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.392+0000 7f5e8e000640 1 -- 172.21.15.7:0/4058868494 learned_addr learned my addr 172.21.15.7:0/4058868494 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:17.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.392+0000 7f5e8ea00640 1 --2- 172.21.15.7:0/4058868494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e90070810 0x7f5e9006d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 994734532 0 0) 0x7f5e80001490 con 0x7f5e90070c20 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e74005730 con 0x7f5e90070c20 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4212529130 0 0) 0x7f5e74005730 con 0x7f5e90070c20 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e90070810 msgr2=0x7f5e9006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 --2- 172.21.15.7:0/4058868494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e90070810 0x7f5e9006d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 >> v1:172.21.15.87:6789/0 conn(0x7f5e9006dac0 legacy=0x7f5e9015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e90161070 con 0x7f5e90070c20 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f5e80001bc0 con 0x7f5e90070c20 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 >> v1:172.21.15.7:6789/0 conn(0x7f5e90070c20 legacy=0x7f5e9015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 --2- 172.21.15.7:0/4058868494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e74005bf0 0x7f5e74006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 --2- 172.21.15.7:0/4058868494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e74006a10 0x7f5e74006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 --2- 172.21.15.7:0/4058868494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e90070810 0x7f5e740074e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90160d90 con 0x7f5e74005bf0 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7400dd60 con 0x7f5e74006a10 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.393+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7400df60 con 0x7f5e90070810 2024-04-25T11:19:17.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.394+0000 7f5e8ea00640 1 --2- 172.21.15.7:0/4058868494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e74005bf0 0x7f5e74006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.394+0000 7f5e8e000640 1 --2- 172.21.15.7:0/4058868494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e74006a10 0x7f5e74006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.394+0000 7f5e8ea00640 1 -- 172.21.15.7:0/4058868494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e90070810 msgr2=0x7f5e740074e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.394+0000 7f5e8ea00640 1 --2- 172.21.15.7:0/4058868494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e90070810 0x7f5e740074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.394+0000 7f5e8ea00640 1 -- 172.21.15.7:0/4058868494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e74006a10 msgr2=0x7f5e74006e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.394+0000 7f5e8ea00640 1 --2- 172.21.15.7:0/4058868494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e74006a10 0x7f5e74006e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.394+0000 7f5e8ea00640 1 -- 172.21.15.7:0/4058868494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e74005730 con 0x7f5e74005bf0 2024-04-25T11:19:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.395+0000 7f5e8ea00640 1 --2- 172.21.15.7:0/4058868494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e74005bf0 0x7f5e74006060 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009ac0 tx=0x7f5e74007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6d2e8d0c625c03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.395+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78036070 con 0x7f5e74005bf0 2024-04-25T11:19:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.395+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e78031400 con 0x7f5e74005bf0 2024-04-25T11:19:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.395+0000 7f5e8d600640 1 -- 172.21.15.7:0/4058868494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78009690 con 0x7f5e74005bf0 2024-04-25T11:19:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.396+0000 7f5e8fe00640 1 -- 172.21.15.7:0/4058868494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e74005bf0 msgr2=0x7f5e74006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.396+0000 7f5e8fe00640 1 --2- 172.21.15.7:0/4058868494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e74005bf0 0x7f5e74006060 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009ac0 tx=0x7f5e74007a40 comp rx=0 tx=0).stop 2024-04-25T11:19:17.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.396+0000 7f5e8f400640 1 -- 172.21.15.7:0/4058868494 reap_dead start 2024-04-25T11:19:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.396+0000 7f5e8fe00640 1 -- 172.21.15.7:0/4058868494 shutdown_connections 2024-04-25T11:19:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.396+0000 7f5e8fe00640 1 -- 172.21.15.7:0/4058868494 >> 172.21.15.7:0/4058868494 conn(0x7f5e90073090 msgr2=0x7f5e900734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.396+0000 7f5e8fe00640 1 -- 172.21.15.7:0/4058868494 shutdown_connections 2024-04-25T11:19:17.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.396+0000 7f5e8fe00640 1 -- 172.21.15.7:0/4058868494 wait complete. 2024-04-25T11:19:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.399+0000 7f5e8fe00640 1 Processor -- start 2024-04-25T11:19:17.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.399+0000 7f5e8fe00640 1 -- start start 2024-04-25T11:19:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.400+0000 7f5e8fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e9015e4d0 0x7f5e9015d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.400+0000 7f5e8fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e90160410 0x7f5e90143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.400+0000 7f5e8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e90144310 0x7f5e901f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.400+0000 7f5e8fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9015e0f0 con 0x7f5e90144310 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.400+0000 7f5e8fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90161630 con 0x7f5e90160410 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.400+0000 7f5e8fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5e901433b0 con 0x7f5e9015e4d0 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.401+0000 7f5e8f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e90144310 0x7f5e901f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.401+0000 7f5e8f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e90144310 0x7f5e901f58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51870/0 (socket says 172.21.15.7:51870) 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.401+0000 7f5e8f400640 1 -- 172.21.15.7:0/2434640869 learned_addr learned my addr 172.21.15.7:0/2434640869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.401+0000 7f5e8ea00640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e9015e4d0 0x7f5e9015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.401+0000 7f5e8e000640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e90160410 0x7f5e90143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.401+0000 7f5e8f400640 1 -- 172.21.15.7:0/2434640869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e9015e4d0 msgr2=0x7f5e9015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.401+0000 7f5e8f400640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e9015e4d0 0x7f5e9015d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.401+0000 7f5e8f400640 1 -- 172.21.15.7:0/2434640869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e90160410 msgr2=0x7f5e90143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.401+0000 7f5e8f400640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e90160410 0x7f5e90143dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.401+0000 7f5e8f400640 1 -- 172.21.15.7:0/2434640869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e90144720 con 0x7f5e90144310 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.402+0000 7f5e8f400640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e90144310 0x7f5e901f58f0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f5e90073040 tx=0x7f5e8800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.402+0000 7f5e86a00640 1 -- 172.21.15.7:0/2434640869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880072e0 con 0x7f5e90144310 2024-04-25T11:19:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.402+0000 7f5e86a00640 1 -- 172.21.15.7:0/2434640869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e88007480 con 0x7f5e90144310 2024-04-25T11:19:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.402+0000 7f5e86a00640 1 -- 172.21.15.7:0/2434640869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88004260 con 0x7f5e90144310 2024-04-25T11:19:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.402+0000 7f5e8fe00640 1 -- 172.21.15.7:0/2434640869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9015ea40 con 0x7f5e90144310 2024-04-25T11:19:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.402+0000 7f5e8fe00640 1 -- 172.21.15.7:0/2434640869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e9015f000 con 0x7f5e90144310 2024-04-25T11:19:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.403+0000 7f5e8fe00640 1 -- 172.21.15.7:0/2434640869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e900cdb30 con 0x7f5e90144310 2024-04-25T11:19:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.405+0000 7f5e86a00640 1 -- 172.21.15.7:0/2434640869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5e880057c0 con 0x7f5e90144310 2024-04-25T11:19:17.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.406+0000 7f5e86a00640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e6c0744b0 0x7f5e6c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:17.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.406+0000 7f5e86a00640 1 -- 172.21.15.7:0/2434640869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f5e88094560 con 0x7f5e90144310 2024-04-25T11:19:17.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.406+0000 7f5e8ea00640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e6c0744b0 0x7f5e6c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.407+0000 7f5e8ea00640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e6c0744b0 0x7f5e6c0769c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5e78033000 tx=0x7f5e78008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.412+0000 7f5e86a00640 1 -- 172.21.15.7:0/2434640869 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e900cdb30 con 0x7f5e90144310 2024-04-25T11:19:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.657+0000 7f5e8fe00640 1 -- 172.21.15.7:0/2434640869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5e90050740 con 0x7f5e90144310 2024-04-25T11:19:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.658+0000 7f5e86a00640 1 -- 172.21.15.7:0/2434640869 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f5e90144720 con 0x7f5e90144310 2024-04-25T11:19:17.656 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:19:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 -- 172.21.15.7:0/2434640869 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e6c0744b0 msgr2=0x7f5e6c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e6c0744b0 0x7f5e6c0769c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5e78033000 tx=0x7f5e78008030 comp rx=0 tx=0).stop 2024-04-25T11:19:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 -- 172.21.15.7:0/2434640869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e90144310 msgr2=0x7f5e901f58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e90144310 0x7f5e901f58f0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f5e90073040 tx=0x7f5e8800c820 comp rx=0 tx=0).stop 2024-04-25T11:19:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 -- 172.21.15.7:0/2434640869 shutdown_connections 2024-04-25T11:19:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e6c0744b0 0x7f5e6c0769c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e90144310 0x7f5e901f58f0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e90160410 0x7f5e90143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 --2- 172.21.15.7:0/2434640869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e9015e4d0 0x7f5e9015d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 -- 172.21.15.7:0/2434640869 >> 172.21.15.7:0/2434640869 conn(0x7f5e90073090 msgr2=0x7f5e9006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 -- 172.21.15.7:0/2434640869 shutdown_connections 2024-04-25T11:19:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17.661+0000 7f5e8fe00640 1 -- 172.21.15.7:0/2434640869 wait complete. 2024-04-25T11:19:17.683 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:19:17.683 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:19:17.683 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:19:17.683 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:19:17.683 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:17.683 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:19:17.683 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:19:17.683 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:19:17.683 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:19:17.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:17.684 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:19:17.737 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:19:17.738 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:19:17.739 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:17.739 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:19:17.739 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:19:17.739 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42017sec preferred_lft 42017sec 2024-04-25T11:19:17.739 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:19:17.739 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:17.739 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:17.739 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:17.739 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:19:17.739 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6098:c7ff:fe8a:4a30/64 scope link 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::646a:35ff:fe7b:ad0/64 scope link 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4465:c4ff:fe62:f75f/64 scope link 2024-04-25T11:19:17.740 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:17.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:17.741 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:19:17.818 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:19:17.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:17.823 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:19:17.874 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:19:17.874 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:19:17.874 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:19:17.874 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:19:17.875 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:19:17.875 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:17.875 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:19:17.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:19:17.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:19:17.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:19:17.875 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:19:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:18.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:18.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:18.098 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:19:18.150 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:19:18.151 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:19:18.151 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:19:18.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:18.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:19:18.232 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:18.232 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:18.288 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:19:18.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:18.288 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:18.342 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:19:18.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:18.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:18.553 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:18.553 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:19:18.600 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:18.600 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:19:18.600 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:19:18.600 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:19:18.600 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:19:18.601 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:19:18.602 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:19:18.602 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:19:18.602 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:19:18.602 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:19:18.607 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:19:18.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:18.607 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:18.688 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:19:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:19:18.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:18.822 INFO:teuthology.orchestra.run.smithi007.stdout:client5918 2024-04-25T11:19:18.822 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:19:18.822 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:18.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:19:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.159+0000 7fd4f3200640 1 Processor -- start 2024-04-25T11:19:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.160+0000 7fd4f3200640 1 -- start start 2024-04-25T11:19:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.162+0000 7fd4f3200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4f415baa0 0x7fd4f415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.162+0000 7fd4f3200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f4070640 0x7fd4f4161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.162+0000 7fd4f3200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4f415e980 con 0x7fd4f4162420 2024-04-25T11:19:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.162+0000 7fd4f3200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd4f4162830 con 0x7fd4f415baa0 2024-04-25T11:19:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.162+0000 7fd4f3200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f415ec60 con 0x7fd4f4070640 2024-04-25T11:19:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.163+0000 7fd4f2800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd4f4162420 0x7fd4f406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38008/0 (socket says 172.21.15.7:38008) 2024-04-25T11:19:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.163+0000 7fd4f2800640 1 -- 172.21.15.7:0/3108832413 learned_addr learned my addr 172.21.15.7:0/3108832413 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.163+0000 7fd4f1e00640 1 --2- 172.21.15.7:0/3108832413 >> v2:172.21.15.7:3300/0 conn(0x7fd4f415baa0 0x7fd4f415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.163+0000 7fd4e9400640 1 --2- 172.21.15.7:0/3108832413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f4070640 0x7fd4f4161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.164+0000 7fd4f1e00640 1 -- 172.21.15.7:0/3108832413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f4070640 msgr2=0x7fd4f4161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.164+0000 7fd4f1e00640 1 --2- 172.21.15.7:0/3108832413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f4070640 0x7fd4f4161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.164+0000 7fd4f1e00640 1 -- 172.21.15.7:0/3108832413 >> v1:172.21.15.7:6789/0 conn(0x7fd4f4162420 legacy=0x7fd4f406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.164+0000 7fd4f1e00640 1 -- 172.21.15.7:0/3108832413 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f415ef40 con 0x7fd4f415baa0 2024-04-25T11:19:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.164+0000 7fd4f1400640 1 -- 172.21.15.7:0/3108832413 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574732670 0 0) 0x7fd4f415e980 con 0x7fd4f4162420 2024-04-25T11:19:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.165+0000 7fd4f1e00640 1 --2- 172.21.15.7:0/3108832413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f415baa0 0x7fd4f415e2f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40097a0 tx=0x7fd4e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a793cbcf9206d190 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.166+0000 7fd4f1400640 1 -- 172.21.15.7:0/3108832413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4037070 con 0x7fd4f415baa0 2024-04-25T11:19:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.166+0000 7fd4f1400640 1 -- 172.21.15.7:0/3108832413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e4032440 con 0x7fd4f415baa0 2024-04-25T11:19:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.166+0000 7fd4f1400640 1 -- 172.21.15.7:0/3108832413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4032810 con 0x7fd4f415baa0 2024-04-25T11:19:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.166+0000 7fd4f3200640 1 -- 172.21.15.7:0/3108832413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f415baa0 msgr2=0x7fd4f415e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.166+0000 7fd4f3200640 1 --2- 172.21.15.7:0/3108832413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f415baa0 0x7fd4f415e2f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40097a0 tx=0x7fd4e402a180 comp rx=0 tx=0).stop 2024-04-25T11:19:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.166+0000 7fd4f3200640 1 -- 172.21.15.7:0/3108832413 shutdown_connections 2024-04-25T11:19:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.166+0000 7fd4f3200640 1 --2- 172.21.15.7:0/3108832413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f4070640 0x7fd4f4161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.166+0000 7fd4f3200640 1 --2- 172.21.15.7:0/3108832413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f415baa0 0x7fd4f415e2f0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.166+0000 7fd4f3200640 1 -- 172.21.15.7:0/3108832413 >> 172.21.15.7:0/3108832413 conn(0x7fd4f4073090 msgr2=0x7fd4f40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.167+0000 7fd4f3200640 1 -- 172.21.15.7:0/3108832413 shutdown_connections 2024-04-25T11:19:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.167+0000 7fd4f3200640 1 -- 172.21.15.7:0/3108832413 wait complete. 2024-04-25T11:19:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.169+0000 7fd4f3200640 1 Processor -- start 2024-04-25T11:19:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.169+0000 7fd4f3200640 1 -- start start 2024-04-25T11:19:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.171+0000 7fd4f3200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f4070640 0x7fd4f40b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.171+0000 7fd4f3200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f4162420 0x7fd4f413fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.171+0000 7fd4f3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f415cdf0 0x7fd4f415d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.171+0000 7fd4f3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4070bd0 con 0x7fd4f415cdf0 2024-04-25T11:19:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.171+0000 7fd4f3200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4070a50 con 0x7fd4f4070640 2024-04-25T11:19:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.171+0000 7fd4f3200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f406dc90 con 0x7fd4f4162420 2024-04-25T11:19:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.171+0000 7fd4f2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f415cdf0 0x7fd4f415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.171+0000 7fd4f2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f415cdf0 0x7fd4f415d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51890/0 (socket says 172.21.15.7:51890) 2024-04-25T11:19:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.171+0000 7fd4f2800640 1 -- 172.21.15.7:0/3214252803 learned_addr learned my addr 172.21.15.7:0/3214252803 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.171+0000 7fd4f1e00640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f4070640 0x7fd4f40b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.171+0000 7fd4e9400640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f4162420 0x7fd4f413fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4f2800640 1 -- 172.21.15.7:0/3214252803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f4162420 msgr2=0x7fd4f413fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4f2800640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f4162420 0x7fd4f413fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4f2800640 1 -- 172.21.15.7:0/3214252803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f4070640 msgr2=0x7fd4f40b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4f2800640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f4070640 0x7fd4f40b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4f2800640 1 -- 172.21.15.7:0/3214252803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f415d740 con 0x7fd4f415cdf0 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4e9400640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f4162420 0x7fd4f413fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4f2800640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f415cdf0 0x7fd4f415d200 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c6d0 tx=0x7fd4ec00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4eaa00640 1 -- 172.21.15.7:0/3214252803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec018070 con 0x7fd4f415cdf0 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4eaa00640 1 -- 172.21.15.7:0/3214252803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ec004d60 con 0x7fd4f415cdf0 2024-04-25T11:19:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4eaa00640 1 -- 172.21.15.7:0/3214252803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec00a880 con 0x7fd4f415cdf0 2024-04-25T11:19:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4f3200640 1 -- 172.21.15.7:0/3214252803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f40afc00 con 0x7fd4f415cdf0 2024-04-25T11:19:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.172+0000 7fd4f3200640 1 -- 172.21.15.7:0/3214252803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f40b01c0 con 0x7fd4f415cdf0 2024-04-25T11:19:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.173+0000 7fd4f3200640 1 -- 172.21.15.7:0/3214252803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f4074020 con 0x7fd4f415cdf0 2024-04-25T11:19:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.175+0000 7fd4eaa00640 1 -- 172.21.15.7:0/3214252803 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd4ec00ac30 con 0x7fd4f415cdf0 2024-04-25T11:19:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.176+0000 7fd4eaa00640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4d00744b0 0x7fd4d00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.176+0000 7fd4eaa00640 1 -- 172.21.15.7:0/3214252803 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fd4ec094bc0 con 0x7fd4f415cdf0 2024-04-25T11:19:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.176+0000 7fd4f1e00640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4d00744b0 0x7fd4d00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.177+0000 7fd4f1e00640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4d00744b0 0x7fd4d00769c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4034000 tx=0x7fd4e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.181+0000 7fd4eaa00640 1 -- 172.21.15.7:0/3214252803 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4f415d740 con 0x7fd4f415cdf0 2024-04-25T11:19:19.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.427+0000 7fd4f3200640 1 -- 172.21.15.7:0/3214252803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd4f4068780 con 0x7fd4f415cdf0 2024-04-25T11:19:19.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.428+0000 7fd4eaa00640 1 -- 172.21.15.7:0/3214252803 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7fd4ec05f1b0 con 0x7fd4f415cdf0 2024-04-25T11:19:19.427 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:19:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 -- 172.21.15.7:0/3214252803 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4d00744b0 msgr2=0x7fd4d00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4d00744b0 0x7fd4d00769c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4034000 tx=0x7fd4e402e000 comp rx=0 tx=0).stop 2024-04-25T11:19:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 -- 172.21.15.7:0/3214252803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f415cdf0 msgr2=0x7fd4f415d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f415cdf0 0x7fd4f415d200 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c6d0 tx=0x7fd4ec00cb90 comp rx=0 tx=0).stop 2024-04-25T11:19:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 -- 172.21.15.7:0/3214252803 shutdown_connections 2024-04-25T11:19:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4d00744b0 0x7fd4d00769c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4f415cdf0 0x7fd4f415d200 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f4162420 0x7fd4f413fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 --2- 172.21.15.7:0/3214252803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f4070640 0x7fd4f40b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 -- 172.21.15.7:0/3214252803 >> 172.21.15.7:0/3214252803 conn(0x7fd4f4073090 msgr2=0x7fd4f406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 -- 172.21.15.7:0/3214252803 shutdown_connections 2024-04-25T11:19:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:19.431+0000 7fd4f3200640 1 -- 172.21.15.7:0/3214252803 wait complete. 2024-04-25T11:19:19.459 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:19:19.459 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:19:19.459 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:19:19.459 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:19:19.460 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:19.460 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:19:19.460 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:19:19.460 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:19:19.460 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:19:19.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:19.460 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:19:19.519 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:19.520 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:19.520 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:19.520 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:19.520 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:19.520 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:19.520 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:19.520 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:19.520 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:19:19.520 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:19.520 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:19.520 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42016sec preferred_lft 42016sec 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:19.521 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::58f2:6bff:fe8b:8b4c/64 scope link 2024-04-25T11:19:19.522 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:19.522 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:19:19.522 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:19.522 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::98fe:1bff:fe9f:d71c/64 scope link 2024-04-25T11:19:19.522 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:19.522 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:19:19.522 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:19:19.522 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::3c38:a8ff:fec7:ea00/64 scope link 2024-04-25T11:19:19.522 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:19.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:19.523 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:19:19.610 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:19:19.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:19.615 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:19:19.662 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:19:19.662 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:19:19.662 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:19:19.662 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:19:19.663 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:19:19.663 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:19.663 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:19:19.663 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:19:19.663 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:19:19.663 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:19:19.663 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:19:19.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:19:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:19.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:19:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:19.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:19.865 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:19:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:19:19.915 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:19:19.916 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:19:19.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:19.916 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T11:19:20.046 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:20.046 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:20.101 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:19:20.101 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:20.101 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:20.151 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:19:20.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:20.151 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:20.354 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:20.354 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:19:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:20.399 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:19:20.399 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:19:20.399 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 1@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:19:20.399 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:19:20.401 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:19:20.401 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:19:20.401 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:19:20.401 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:19:20.401 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:19:20.406 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:19:20.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:20.406 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:20.480 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:19:20.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:19:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:19:20.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:20.599 INFO:teuthology.orchestra.run.smithi087.stdout:client5924 2024-04-25T11:19:20.599 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:19:20.599 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:20.652 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:19:20.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:19:20.770 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:19:20.872 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:19:20.872 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:19:20.872 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:19:20.872 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:20.935 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:19:20.935 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:20.991 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:21.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.277+0000 7f22ebe00640 1 Processor -- start 2024-04-25T11:19:21.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.277+0000 7f22ebe00640 1 -- start start 2024-04-25T11:19:21.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.279+0000 7f22ebe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec163ac0 0x7f22ec0b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.279+0000 7f22ebe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22ec15f580 0x7f22ec15f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.280+0000 7f22ebe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22ec0d35a0 con 0x7f22ec06dcd0 2024-04-25T11:19:21.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.280+0000 7f22ebe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f22ec164310 con 0x7f22ec15f580 2024-04-25T11:19:21.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.280+0000 7f22ebe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec1645f0 con 0x7f22ec163ac0 2024-04-25T11:19:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.280+0000 7f22eb400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22ec15f580 0x7f22ec15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.280+0000 7f22eb400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22ec15f580 0x7f22ec15f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43260/0 (socket says 172.21.15.7:43260) 2024-04-25T11:19:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.280+0000 7f22eb400640 1 -- 172.21.15.7:0/1929991847 learned_addr learned my addr 172.21.15.7:0/1929991847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.280+0000 7f22eaa00640 1 --2- 172.21.15.7:0/1929991847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec163ac0 0x7f22ec0b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.280+0000 7f22eb400640 1 -- 172.21.15.7:0/1929991847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec163ac0 msgr2=0x7f22ec0b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.280+0000 7f22eb400640 1 --2- 172.21.15.7:0/1929991847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec163ac0 0x7f22ec0b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.280+0000 7f22eb400640 1 -- 172.21.15.7:0/1929991847 >> v1:172.21.15.87:6789/0 conn(0x7f22ec06dcd0 legacy=0x7f22ec06e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.281+0000 7f22eb400640 1 -- 172.21.15.7:0/1929991847 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec1648d0 con 0x7f22ec15f580 2024-04-25T11:19:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.281+0000 7f22eaa00640 1 --2- 172.21.15.7:0/1929991847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec163ac0 0x7f22ec0b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.282+0000 7f22eb400640 1 --2- 172.21.15.7:0/1929991847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15f580 0x7f22ec15f990 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009e50 tx=0x7f22e402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baeaa24318de3eef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.282+0000 7f22ea000640 1 -- 172.21.15.7:0/1929991847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4036070 con 0x7f22ec15f580 2024-04-25T11:19:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.282+0000 7f22ea000640 1 -- 172.21.15.7:0/1929991847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e4031600 con 0x7f22ec15f580 2024-04-25T11:19:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.282+0000 7f22ea000640 1 -- 172.21.15.7:0/1929991847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e40319d0 con 0x7f22ec15f580 2024-04-25T11:19:21.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.283+0000 7f22ebe00640 1 -- 172.21.15.7:0/1929991847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15f580 msgr2=0x7f22ec15f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:21.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.283+0000 7f22ebe00640 1 --2- 172.21.15.7:0/1929991847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15f580 0x7f22ec15f990 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009e50 tx=0x7f22e402a640 comp rx=0 tx=0).stop 2024-04-25T11:19:21.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.283+0000 7f22ebe00640 1 -- 172.21.15.7:0/1929991847 shutdown_connections 2024-04-25T11:19:21.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.283+0000 7f22ebe00640 1 --2- 172.21.15.7:0/1929991847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15f580 0x7f22ec15f990 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.283+0000 7f22ebe00640 1 --2- 172.21.15.7:0/1929991847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec163ac0 0x7f22ec0b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.283+0000 7f22ebe00640 1 -- 172.21.15.7:0/1929991847 >> 172.21.15.7:0/1929991847 conn(0x7f22ec0d29e0 msgr2=0x7f22ec0d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.283+0000 7f22ebe00640 1 -- 172.21.15.7:0/1929991847 shutdown_connections 2024-04-25T11:19:21.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.283+0000 7f22ebe00640 1 -- 172.21.15.7:0/1929991847 wait complete. 2024-04-25T11:19:21.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.285+0000 7f22ebe00640 1 Processor -- start 2024-04-25T11:19:21.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.286+0000 7f22ebe00640 1 -- start start 2024-04-25T11:19:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.287+0000 7f22ebe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec06dcd0 0x7f22ec0d0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.287+0000 7f22ebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15f580 0x7f22ec0ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.287+0000 7f22ebe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22ec163ac0 0x7f22ec0ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.287+0000 7f22ebe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec164050 con 0x7f22ec15f580 2024-04-25T11:19:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.287+0000 7f22ebe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec163ed0 con 0x7f22ec163ac0 2024-04-25T11:19:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.287+0000 7f22ebe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f22ec15fed0 con 0x7f22ec06dcd0 2024-04-25T11:19:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22e3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15f580 0x7f22ec0ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22e3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15f580 0x7f22ec0ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43266/0 (socket says 172.21.15.7:43266) 2024-04-25T11:19:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22e3e00640 1 -- 172.21.15.7:0/2307608044 learned_addr learned my addr 172.21.15.7:0/2307608044 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22eaa00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec06dcd0 0x7f22ec0d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22eb400640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22ec163ac0 0x7f22ec0ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22e3e00640 1 -- 172.21.15.7:0/2307608044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec06dcd0 msgr2=0x7f22ec0d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22e3e00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec06dcd0 0x7f22ec0d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22e3e00640 1 -- 172.21.15.7:0/2307608044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22ec163ac0 msgr2=0x7f22ec0ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22e3e00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22ec163ac0 0x7f22ec0ac8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22e3e00640 1 -- 172.21.15.7:0/2307608044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec0d0ab0 con 0x7f22ec15f580 2024-04-25T11:19:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22eaa00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec06dcd0 0x7f22ec0d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.288+0000 7f22e3e00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15f580 0x7f22ec0ac3b0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f22d400b3c0 tx=0x7f22d400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.289+0000 7f22e3400640 1 -- 172.21.15.7:0/2307608044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4011830 con 0x7f22ec15f580 2024-04-25T11:19:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.289+0000 7f22e3400640 1 -- 172.21.15.7:0/2307608044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d4016070 con 0x7f22ec15f580 2024-04-25T11:19:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.289+0000 7f22e3400640 1 -- 172.21.15.7:0/2307608044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4011c10 con 0x7f22ec15f580 2024-04-25T11:19:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.289+0000 7f22ebe00640 1 -- 172.21.15.7:0/2307608044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec0b3650 con 0x7f22ec15f580 2024-04-25T11:19:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.289+0000 7f22ebe00640 1 -- 172.21.15.7:0/2307608044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec0b3c10 con 0x7f22ec15f580 2024-04-25T11:19:21.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.290+0000 7f22ebe00640 1 -- 172.21.15.7:0/2307608044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ec0aefc0 con 0x7f22ec15f580 2024-04-25T11:19:21.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.291+0000 7f22e3400640 1 -- 172.21.15.7:0/2307608044 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f22d401b460 con 0x7f22ec15f580 2024-04-25T11:19:21.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.292+0000 7f22e3400640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22c80744b0 0x7f22c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.293+0000 7f22e3400640 1 -- 172.21.15.7:0/2307608044 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f22d4094e90 con 0x7f22ec15f580 2024-04-25T11:19:21.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.293+0000 7f22eaa00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22c80744b0 0x7f22c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.294+0000 7f22eaa00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22c80744b0 0x7f22c80769c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f22dc006fd0 tx=0x7f22dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.297+0000 7f22e3400640 1 -- 172.21.15.7:0/2307608044 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f22d4020020 con 0x7f22ec15f580 2024-04-25T11:19:21.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.505+0000 7f22ebe00640 1 -- 172.21.15.7:0/2307608044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22ec050740 con 0x7f22ec15f580 2024-04-25T11:19:21.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.507+0000 7f22e3400640 1 -- 172.21.15.7:0/2307608044 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16788 (secure 0 0 0) 0x7f22ec0d0ab0 con 0x7f22ec15f580 2024-04-25T11:19:21.506 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:21.506 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:10.691402+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:07.435764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:08.209216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:21.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 -- 172.21.15.7:0/2307608044 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22c80744b0 msgr2=0x7f22c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:21.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22c80744b0 0x7f22c80769c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f22dc006fd0 tx=0x7f22dc008040 comp rx=0 tx=0).stop 2024-04-25T11:19:21.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 -- 172.21.15.7:0/2307608044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15f580 msgr2=0x7f22ec0ac3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:21.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15f580 0x7f22ec0ac3b0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f22d400b3c0 tx=0x7f22d400b880 comp rx=0 tx=0).stop 2024-04-25T11:19:21.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 -- 172.21.15.7:0/2307608044 shutdown_connections 2024-04-25T11:19:21.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f22c80744b0 0x7f22c80769c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22ec163ac0 0x7f22ec0ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15f580 0x7f22ec0ac3b0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 --2- 172.21.15.7:0/2307608044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22ec06dcd0 0x7f22ec0d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 -- 172.21.15.7:0/2307608044 >> 172.21.15.7:0/2307608044 conn(0x7f22ec0d29e0 msgr2=0x7f22ec161ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:21.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 -- 172.21.15.7:0/2307608044 shutdown_connections 2024-04-25T11:19:21.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.510+0000 7f22ebe00640 1 -- 172.21.15.7:0/2307608044 wait complete. 2024-04-25T11:19:21.535 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:07.435764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:08.209216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:21.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:21.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.829+0000 7f2fcaa00640 1 Processor -- start 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.830+0000 7f2fcaa00640 1 -- start start 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.832+0000 7f2fcaa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc415aaa0 0x7f2fc40affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.832+0000 7f2fcaa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2fc40d0ee0 0x7f2fc40b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.832+0000 7f2fcaa00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc406f5b0 con 0x7f2fc40b4e50 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.832+0000 7f2fcaa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc40af290 con 0x7f2fc40d0ee0 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.832+0000 7f2fcaa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc40d14a0 con 0x7f2fc415aaa0 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.833+0000 7f2fc8a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2fc40b4e50 0x7f2fc415d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37736/0 (socket says 172.21.15.7:37736) 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.833+0000 7f2fc8a00640 1 -- 172.21.15.7:0/445591505 learned_addr learned my addr 172.21.15.7:0/445591505 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.833+0000 7f2fc3400640 1 --2- 172.21.15.7:0/445591505 >> v2:172.21.15.87:3300/0 conn(0x7f2fc40d0ee0 0x7f2fc40b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.833+0000 7f2fc3e00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc415aaa0 0x7f2fc40affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.833+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1065803375 0 0) 0x7f2fc406f5b0 con 0x7f2fc40b4e50 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.833+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fa8005730 con 0x7f2fc40b4e50 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1093970886 0 0) 0x7f2fa8005730 con 0x7f2fc40b4e50 2024-04-25T11:19:21.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc415aaa0 msgr2=0x7f2fc40affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc415aaa0 0x7f2fc40affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 >> v2:172.21.15.87:3300/0 conn(0x7f2fc40d0ee0 msgr2=0x7f2fc40b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 --2- 172.21.15.7:0/445591505 >> v2:172.21.15.87:3300/0 conn(0x7f2fc40d0ee0 0x7f2fc40b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc40af4e0 con 0x7f2fc40b4e50 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2fbc0019f0 con 0x7f2fc40b4e50 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 >> v1:172.21.15.7:6789/0 conn(0x7f2fc40b4e50 legacy=0x7f2fc415d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa8005bc0 0x7f2fa8006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc415aaa0 0x7f2fa8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8006f50 0x7f2fa8007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc40af290 con 0x7f2fa8005bc0 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc40d14a0 con 0x7f2fa8006f50 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa800dcb0 con 0x7f2fc415aaa0 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.834+0000 7f2fc3e00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa8005bc0 0x7f2fa8006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.835+0000 7f2fc8a00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8006f50 0x7f2fa8007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.835+0000 7f2fc3e00640 1 -- 172.21.15.7:0/445591505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc415aaa0 msgr2=0x7f2fa8006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.835+0000 7f2fc3e00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc415aaa0 0x7f2fa8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.835+0000 7f2fc3e00640 1 -- 172.21.15.7:0/445591505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8006f50 msgr2=0x7f2fa8007360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.835+0000 7f2fc3e00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8006f50 0x7f2fa8007360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.835+0000 7f2fc3e00640 1 -- 172.21.15.7:0/445591505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa8005730 con 0x7f2fa8005bc0 2024-04-25T11:19:21.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.835+0000 7f2fc8a00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8006f50 0x7f2fa8007360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.836+0000 7f2fc3e00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa8005bc0 0x7f2fa8006030 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009ac0 tx=0x7f2fa8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674a0db8c06f1306 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.836+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4036070 con 0x7f2fa8005bc0 2024-04-25T11:19:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.836+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb4031400 con 0x7f2fa8005bc0 2024-04-25T11:19:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.836+0000 7f2fc2a00640 1 -- 172.21.15.7:0/445591505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4009690 con 0x7f2fa8005bc0 2024-04-25T11:19:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.837+0000 7f2fcaa00640 1 -- 172.21.15.7:0/445591505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa8005bc0 msgr2=0x7f2fa8006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.837+0000 7f2fcaa00640 1 --2- 172.21.15.7:0/445591505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa8005bc0 0x7f2fa8006030 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009ac0 tx=0x7f2fa8007a40 comp rx=0 tx=0).stop 2024-04-25T11:19:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.837+0000 7f2fc8a00640 1 -- 172.21.15.7:0/445591505 reap_dead start 2024-04-25T11:19:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.837+0000 7f2fcaa00640 1 -- 172.21.15.7:0/445591505 shutdown_connections 2024-04-25T11:19:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.837+0000 7f2fcaa00640 1 -- 172.21.15.7:0/445591505 >> 172.21.15.7:0/445591505 conn(0x7f2fc40742c0 msgr2=0x7f2fc406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:21.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.837+0000 7f2fcaa00640 1 -- 172.21.15.7:0/445591505 shutdown_connections 2024-04-25T11:19:21.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.837+0000 7f2fcaa00640 1 -- 172.21.15.7:0/445591505 wait complete. 2024-04-25T11:19:21.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.840+0000 7f2fcaa00640 1 Processor -- start 2024-04-25T11:19:21.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.840+0000 7f2fcaa00640 1 -- start start 2024-04-25T11:19:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.841+0000 7f2fcaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8005bc0 0x7f2fc4159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.841+0000 7f2fcaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc414cc90 0x7f2fc4160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.841+0000 7f2fcaa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160dc0 0x7f2fc414a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.841+0000 7f2fcaa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc40affc0 con 0x7f2fc414cc90 2024-04-25T11:19:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.841+0000 7f2fcaa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc40afaa0 con 0x7f2fa8005bc0 2024-04-25T11:19:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.841+0000 7f2fcaa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc40b0140 con 0x7f2fc4160dc0 2024-04-25T11:19:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.842+0000 7f2fc3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc414cc90 0x7f2fc4160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.842+0000 7f2fc3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc414cc90 0x7f2fc4160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43292/0 (socket says 172.21.15.7:43292) 2024-04-25T11:19:21.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.842+0000 7f2fc3400640 1 -- 172.21.15.7:0/315339805 learned_addr learned my addr 172.21.15.7:0/315339805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.842+0000 7f2fc3e00640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8005bc0 0x7f2fc4159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.842+0000 7f2fc8a00640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160dc0 0x7f2fc414a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.842+0000 7f2fc3400640 1 -- 172.21.15.7:0/315339805 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160dc0 msgr2=0x7f2fc414a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.842+0000 7f2fc3400640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160dc0 0x7f2fc414a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.842+0000 7f2fc3400640 1 -- 172.21.15.7:0/315339805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8005bc0 msgr2=0x7f2fc4159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.842+0000 7f2fc3400640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8005bc0 0x7f2fc4159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.842+0000 7f2fc3400640 1 -- 172.21.15.7:0/315339805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc415a370 con 0x7f2fc414cc90 2024-04-25T11:19:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.843+0000 7f2fc3400640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc414cc90 0x7f2fc4160880 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000e7f0 tx=0x7f2fb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.843+0000 7f2fbbe00640 1 -- 172.21.15.7:0/315339805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0017070 con 0x7f2fc414cc90 2024-04-25T11:19:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.843+0000 7f2fbbe00640 1 -- 172.21.15.7:0/315339805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb000c940 con 0x7f2fc414cc90 2024-04-25T11:19:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.843+0000 7f2fbbe00640 1 -- 172.21.15.7:0/315339805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb000cd30 con 0x7f2fc414cc90 2024-04-25T11:19:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.843+0000 7f2fcaa00640 1 -- 172.21.15.7:0/315339805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc41f99a0 con 0x7f2fc414cc90 2024-04-25T11:19:21.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.843+0000 7f2fcaa00640 1 -- 172.21.15.7:0/315339805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc41f9f60 con 0x7f2fc414cc90 2024-04-25T11:19:21.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.844+0000 7f2fcaa00640 1 -- 172.21.15.7:0/315339805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc4050740 con 0x7f2fc414cc90 2024-04-25T11:19:21.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.846+0000 7f2fbbe00640 1 -- 172.21.15.7:0/315339805 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2fb001a070 con 0x7f2fc414cc90 2024-04-25T11:19:21.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.847+0000 7f2fbbe00640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:21.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.847+0000 7f2fbbe00640 1 -- 172.21.15.7:0/315339805 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f2fb0095330 con 0x7f2fc414cc90 2024-04-25T11:19:21.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.847+0000 7f2fc3e00640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:21.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.848+0000 7f2fc3e00640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4033000 tx=0x7f2fb4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:21.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:21.852+0000 7f2fbbe00640 1 -- 172.21.15.7:0/315339805 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fb005f890 con 0x7f2fc414cc90 2024-04-25T11:19:22.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.058+0000 7f2fcaa00640 1 -- 172.21.15.7:0/315339805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2fc4070f20 con 0x7f2fc414cc90 2024-04-25T11:19:22.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.059+0000 7f2fbbe00640 1 -- 172.21.15.7:0/315339805 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7f2fb005f190 con 0x7f2fc414cc90 2024-04-25T11:19:22.057 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.061+0000 7f2fcaa00640 1 -- 172.21.15.7:0/315339805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 msgr2=0x7f2fa40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.061+0000 7f2fcaa00640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4033000 tx=0x7f2fb4008030 comp rx=0 tx=0).stop 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.061+0000 7f2fcaa00640 1 -- 172.21.15.7:0/315339805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc414cc90 msgr2=0x7f2fc4160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.061+0000 7f2fcaa00640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc414cc90 0x7f2fc4160880 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000e7f0 tx=0x7f2fb000ecb0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.061+0000 7f2fcaa00640 1 -- 172.21.15.7:0/315339805 shutdown_connections 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.061+0000 7f2fcaa00640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.061+0000 7f2fcaa00640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4160dc0 0x7f2fc414a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.061+0000 7f2fcaa00640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc414cc90 0x7f2fc4160880 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.061+0000 7f2fcaa00640 1 --2- 172.21.15.7:0/315339805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8005bc0 0x7f2fc4159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.061+0000 7f2fcaa00640 1 -- 172.21.15.7:0/315339805 >> 172.21.15.7:0/315339805 conn(0x7f2fc40742c0 msgr2=0x7f2fc40d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.062+0000 7f2fcaa00640 1 -- 172.21.15.7:0/315339805 shutdown_connections 2024-04-25T11:19:22.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.062+0000 7f2fcaa00640 1 -- 172.21.15.7:0/315339805 wait complete. 2024-04-25T11:19:22.087 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:19:22.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.378+0000 7f504ea00640 1 Processor -- start 2024-04-25T11:19:22.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.379+0000 7f504ea00640 1 -- start start 2024-04-25T11:19:22.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.381+0000 7f504ea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f50480cff00 0x7f50480d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.381+0000 7f504ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5048167de0 0x7f50481681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.381+0000 7f504ea00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5048168730 con 0x7f50481498d0 2024-04-25T11:19:22.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.381+0000 7f504ea00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f504806eea0 con 0x7f50480cff00 2024-04-25T11:19:22.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.381+0000 7f504ea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f50480d09e0 con 0x7f5048167de0 2024-04-25T11:19:22.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.382+0000 7f504d600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f50480cff00 0x7f50480d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.382+0000 7f504cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5048167de0 0x7f50481681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.382+0000 7f504d600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f50480cff00 0x7f50480d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:55608/0 (socket says 172.21.15.7:55608) 2024-04-25T11:19:22.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.382+0000 7f504d600640 1 -- 172.21.15.7:0/2057652762 learned_addr learned my addr 172.21.15.7:0/2057652762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.383+0000 7f504d600640 1 -- 172.21.15.7:0/2057652762 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5048167de0 msgr2=0x7f50481681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.383+0000 7f504d600640 1 --2- 172.21.15.7:0/2057652762 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5048167de0 0x7f50481681f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.383+0000 7f504d600640 1 -- 172.21.15.7:0/2057652762 >> v1:172.21.15.87:6789/0 conn(0x7f50481498d0 legacy=0x7f5048149ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.383+0000 7f504d600640 1 -- 172.21.15.7:0/2057652762 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50480d0cc0 con 0x7f50480cff00 2024-04-25T11:19:22.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.383+0000 7f5047e00640 1 -- 172.21.15.7:0/2057652762 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 377717621 0 0) 0x7f5040001110 con 0x7f50481498d0 2024-04-25T11:19:22.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.384+0000 7f504d600640 1 --2- 172.21.15.7:0/2057652762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50480cff00 0x7f50480d0310 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f504806f020 tx=0x7f503802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=800bf2b020a685c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:22.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.385+0000 7f5047e00640 1 -- 172.21.15.7:0/2057652762 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038036070 con 0x7f50480cff00 2024-04-25T11:19:22.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.385+0000 7f5047e00640 1 -- 172.21.15.7:0/2057652762 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5038031600 con 0x7f50480cff00 2024-04-25T11:19:22.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.385+0000 7f5047e00640 1 -- 172.21.15.7:0/2057652762 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50380319d0 con 0x7f50480cff00 2024-04-25T11:19:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.385+0000 7f504ea00640 1 -- 172.21.15.7:0/2057652762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50480cff00 msgr2=0x7f50480d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.385+0000 7f504ea00640 1 --2- 172.21.15.7:0/2057652762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50480cff00 0x7f50480d0310 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f504806f020 tx=0x7f503802a710 comp rx=0 tx=0).stop 2024-04-25T11:19:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.386+0000 7f504ea00640 1 -- 172.21.15.7:0/2057652762 shutdown_connections 2024-04-25T11:19:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.386+0000 7f504ea00640 1 --2- 172.21.15.7:0/2057652762 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5048167de0 0x7f50481681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.386+0000 7f504ea00640 1 --2- 172.21.15.7:0/2057652762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50480cff00 0x7f50480d0310 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.386+0000 7f504ea00640 1 -- 172.21.15.7:0/2057652762 >> 172.21.15.7:0/2057652762 conn(0x7f50480740f0 msgr2=0x7f504806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.386+0000 7f504ea00640 1 -- 172.21.15.7:0/2057652762 shutdown_connections 2024-04-25T11:19:22.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.386+0000 7f504ea00640 1 -- 172.21.15.7:0/2057652762 wait complete. 2024-04-25T11:19:22.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.388+0000 7f504ea00640 1 Processor -- start 2024-04-25T11:19:22.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.388+0000 7f504ea00640 1 -- start start 2024-04-25T11:19:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.390+0000 7f504ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50480cff00 0x7f5048134680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.390+0000 7f504ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50481498d0 0x7f50481465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.390+0000 7f504ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5048167de0 0x7f50481425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.390+0000 7f504ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5048168a30 con 0x7f50481498d0 2024-04-25T11:19:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.390+0000 7f504ea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f50480d1280 con 0x7f5048167de0 2024-04-25T11:19:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.390+0000 7f504ea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5048168bb0 con 0x7f50480cff00 2024-04-25T11:19:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.390+0000 7f504cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50481498d0 0x7f50481465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.390+0000 7f504cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50481498d0 0x7f50481465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43300/0 (socket says 172.21.15.7:43300) 2024-04-25T11:19:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.390+0000 7f504cc00640 1 -- 172.21.15.7:0/1471669219 learned_addr learned my addr 172.21.15.7:0/1471669219 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:22.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.391+0000 7f504d600640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50480cff00 0x7f5048134680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.391+0000 7f504e000640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5048167de0 0x7f50481425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.391+0000 7f504cc00640 1 -- 172.21.15.7:0/1471669219 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50480cff00 msgr2=0x7f5048134680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.391+0000 7f504cc00640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50480cff00 0x7f5048134680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.391+0000 7f504cc00640 1 -- 172.21.15.7:0/1471669219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5048167de0 msgr2=0x7f50481425c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.391+0000 7f504cc00640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5048167de0 0x7f50481425c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.391+0000 7f504cc00640 1 -- 172.21.15.7:0/1471669219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5048149080 con 0x7f50481498d0 2024-04-25T11:19:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.391+0000 7f504cc00640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50481498d0 0x7f50481465e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f503400e8a0 tx=0x7f503400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.392+0000 7f5045600640 1 -- 172.21.15.7:0/1471669219 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034017070 con 0x7f50481498d0 2024-04-25T11:19:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.392+0000 7f5045600640 1 -- 172.21.15.7:0/1471669219 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f503400c940 con 0x7f50481498d0 2024-04-25T11:19:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.392+0000 7f504ea00640 1 -- 172.21.15.7:0/1471669219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5048149360 con 0x7f50481498d0 2024-04-25T11:19:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.392+0000 7f5045600640 1 -- 172.21.15.7:0/1471669219 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503400cd30 con 0x7f50481498d0 2024-04-25T11:19:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.392+0000 7f504ea00640 1 -- 172.21.15.7:0/1471669219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5048144a60 con 0x7f50481498d0 2024-04-25T11:19:22.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.393+0000 7f504ea00640 1 -- 172.21.15.7:0/1471669219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5048148df0 con 0x7f50481498d0 2024-04-25T11:19:22.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.394+0000 7f5045600640 1 -- 172.21.15.7:0/1471669219 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f503401a070 con 0x7f50481498d0 2024-04-25T11:19:22.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.395+0000 7f5045600640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50280744b0 0x7f50280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.396+0000 7f5045600640 1 -- 172.21.15.7:0/1471669219 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f5034095770 con 0x7f50481498d0 2024-04-25T11:19:22.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.396+0000 7f504d600640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50280744b0 0x7f50280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.397+0000 7f504d600640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50280744b0 0x7f50280769c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5038033000 tx=0x7f5038009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:22.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.401+0000 7f5045600640 1 -- 172.21.15.7:0/1471669219 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f503405fc50 con 0x7f50481498d0 2024-04-25T11:19:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.606+0000 7f504ea00640 1 -- 172.21.15.7:0/1471669219 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5048050740 con 0x7f50481498d0 2024-04-25T11:19:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.607+0000 7f5045600640 1 -- 172.21.15.7:0/1471669219 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7f503405f550 con 0x7f50481498d0 2024-04-25T11:19:22.606 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:19:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.610+0000 7f504ea00640 1 -- 172.21.15.7:0/1471669219 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50280744b0 msgr2=0x7f50280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.610+0000 7f504ea00640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50280744b0 0x7f50280769c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5038033000 tx=0x7f5038009000 comp rx=0 tx=0).stop 2024-04-25T11:19:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.610+0000 7f504ea00640 1 -- 172.21.15.7:0/1471669219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50481498d0 msgr2=0x7f50481465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.610+0000 7f504ea00640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50481498d0 0x7f50481465e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f503400e8a0 tx=0x7f503400ed60 comp rx=0 tx=0).stop 2024-04-25T11:19:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.610+0000 7f504ea00640 1 -- 172.21.15.7:0/1471669219 shutdown_connections 2024-04-25T11:19:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.610+0000 7f504ea00640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50280744b0 0x7f50280769c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.610+0000 7f504ea00640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5048167de0 0x7f50481425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.610+0000 7f504ea00640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50481498d0 0x7f50481465e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.610+0000 7f504ea00640 1 --2- 172.21.15.7:0/1471669219 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50480cff00 0x7f5048134680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.610+0000 7f504ea00640 1 -- 172.21.15.7:0/1471669219 >> 172.21.15.7:0/1471669219 conn(0x7f50480740f0 msgr2=0x7f5048164830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.610+0000 7f504ea00640 1 -- 172.21.15.7:0/1471669219 shutdown_connections 2024-04-25T11:19:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.611+0000 7f504ea00640 1 -- 172.21.15.7:0/1471669219 wait complete. 2024-04-25T11:19:22.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:19:22.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.929+0000 7f5a5ae00640 1 Processor -- start 2024-04-25T11:19:22.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.929+0000 7f5a5ae00640 1 -- start start 2024-04-25T11:19:22.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.932+0000 7f5a5ae00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a540b4690 0x7f5a540b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.932+0000 7f5a5ae00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5a54144630 0x7f5a54144a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.932+0000 7f5a5ae00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a540b5c40 con 0x7f5a54143b10 2024-04-25T11:19:22.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.932+0000 7f5a5ae00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5a540b5f20 con 0x7f5a540b4690 2024-04-25T11:19:22.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.932+0000 7f5a5ae00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5a540b6200 con 0x7f5a54144630 2024-04-25T11:19:22.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.932+0000 7f5a59000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5a54143b10 0x7f5a54143f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37738/0 (socket says 172.21.15.7:37738) 2024-04-25T11:19:22.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.932+0000 7f5a59000640 1 -- 172.21.15.7:0/1799296511 learned_addr learned my addr 172.21.15.7:0/1799296511 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:22.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.932+0000 7f5a59a00640 1 --2- 172.21.15.7:0/1799296511 >> v2:172.21.15.7:3300/0 conn(0x7f5a540b4690 0x7f5a540b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.933+0000 7f5a5a400640 1 --2- 172.21.15.7:0/1799296511 >> v2:172.21.15.87:3300/0 conn(0x7f5a54144630 0x7f5a54144a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.933+0000 7f5a53e00640 1 -- 172.21.15.7:0/1799296511 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1798906264 0 0) 0x7f5a540b5c40 con 0x7f5a54143b10 2024-04-25T11:19:22.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.933+0000 7f5a53e00640 1 -- 172.21.15.7:0/1799296511 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a3c005730 con 0x7f5a54143b10 2024-04-25T11:19:22.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.933+0000 7f5a59a00640 1 -- 172.21.15.7:0/1799296511 >> v2:172.21.15.87:3300/0 conn(0x7f5a54144630 msgr2=0x7f5a54144a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:22.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.933+0000 7f5a59a00640 1 --2- 172.21.15.7:0/1799296511 >> v2:172.21.15.87:3300/0 conn(0x7f5a54144630 0x7f5a54144a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.933+0000 7f5a59a00640 1 -- 172.21.15.7:0/1799296511 >> v1:172.21.15.7:6789/0 conn(0x7f5a54143b10 legacy=0x7f5a54143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.933+0000 7f5a59a00640 1 -- 172.21.15.7:0/1799296511 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a540b64e0 con 0x7f5a540b4690 2024-04-25T11:19:22.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.934+0000 7f5a59a00640 1 --2- 172.21.15.7:0/1799296511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540b4690 0x7f5a540b5580 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f5a540d0e80 tx=0x7f5a4802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208c05868f4e1110 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:22.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.934+0000 7f5a53e00640 1 -- 172.21.15.7:0/1799296511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48037070 con 0x7f5a540b4690 2024-04-25T11:19:22.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.935+0000 7f5a53e00640 1 -- 172.21.15.7:0/1799296511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a48032440 con 0x7f5a540b4690 2024-04-25T11:19:22.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.935+0000 7f5a53e00640 1 -- 172.21.15.7:0/1799296511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48032810 con 0x7f5a540b4690 2024-04-25T11:19:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.935+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1799296511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540b4690 msgr2=0x7f5a540b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.935+0000 7f5a5ae00640 1 --2- 172.21.15.7:0/1799296511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540b4690 0x7f5a540b5580 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f5a540d0e80 tx=0x7f5a4802a230 comp rx=0 tx=0).stop 2024-04-25T11:19:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.935+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1799296511 shutdown_connections 2024-04-25T11:19:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.935+0000 7f5a5ae00640 1 --2- 172.21.15.7:0/1799296511 >> v2:172.21.15.87:3300/0 conn(0x7f5a54144630 0x7f5a54144a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.935+0000 7f5a5ae00640 1 --2- 172.21.15.7:0/1799296511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540b4690 0x7f5a540b5580 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.935+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1799296511 >> 172.21.15.7:0/1799296511 conn(0x7f5a54074140 msgr2=0x7f5a5406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.936+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1799296511 shutdown_connections 2024-04-25T11:19:22.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.936+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1799296511 wait complete. 2024-04-25T11:19:22.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.938+0000 7f5a5ae00640 1 Processor -- start 2024-04-25T11:19:22.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.938+0000 7f5a5ae00640 1 -- start start 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.940+0000 7f5a5ae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540b4690 0x7f5a540ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.940+0000 7f5a5ae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a54143b10 0x7f5a541e9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.940+0000 7f5a5ae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a54144630 0x7f5a541e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.940+0000 7f5a5ae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54143200 con 0x7f5a540b4690 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.940+0000 7f5a5ae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5a54070fa0 con 0x7f5a54144630 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.940+0000 7f5a5ae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5a54143380 con 0x7f5a54143b10 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.940+0000 7f5a59a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540b4690 0x7f5a540ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.940+0000 7f5a59a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540b4690 0x7f5a540ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43308/0 (socket says 172.21.15.7:43308) 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.940+0000 7f5a59a00640 1 -- 172.21.15.7:0/1351447739 learned_addr learned my addr 172.21.15.7:0/1351447739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.941+0000 7f5a5a400640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a54144630 0x7f5a541e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.941+0000 7f5a59000640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a54143b10 0x7f5a541e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.941+0000 7f5a59a00640 1 -- 172.21.15.7:0/1351447739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a54143b10 msgr2=0x7f5a541e9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.941+0000 7f5a59a00640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a54143b10 0x7f5a541e9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.941+0000 7f5a59a00640 1 -- 172.21.15.7:0/1351447739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a54144630 msgr2=0x7f5a541e51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:22.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.941+0000 7f5a59a00640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a54144630 0x7f5a541e51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:22.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.941+0000 7f5a59a00640 1 -- 172.21.15.7:0/1351447739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a541e7fc0 con 0x7f5a540b4690 2024-04-25T11:19:22.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.941+0000 7f5a59000640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a54143b10 0x7f5a541e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:22.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.941+0000 7f5a5a400640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a54144630 0x7f5a541e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:22.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.941+0000 7f5a59a00640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540b4690 0x7f5a540ae3b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009890 tx=0x7f5a480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:22.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.942+0000 7f5a51600640 1 -- 172.21.15.7:0/1351447739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48041020 con 0x7f5a540b4690 2024-04-25T11:19:22.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.942+0000 7f5a51600640 1 -- 172.21.15.7:0/1351447739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a4803d700 con 0x7f5a540b4690 2024-04-25T11:19:22.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.942+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1351447739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a541e8260 con 0x7f5a540b4690 2024-04-25T11:19:22.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.942+0000 7f5a51600640 1 -- 172.21.15.7:0/1351447739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48037040 con 0x7f5a540b4690 2024-04-25T11:19:22.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.942+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1351447739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a541e8820 con 0x7f5a540b4690 2024-04-25T11:19:22.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.943+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1351447739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a54144ad0 con 0x7f5a540b4690 2024-04-25T11:19:22.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.944+0000 7f5a51600640 1 -- 172.21.15.7:0/1351447739 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5a48044020 con 0x7f5a540b4690 2024-04-25T11:19:22.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.945+0000 7f5a51600640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a340744e0 0x7f5a340769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:22.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.946+0000 7f5a51600640 1 -- 172.21.15.7:0/1351447739 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f5a480b5920 con 0x7f5a540b4690 2024-04-25T11:19:22.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.946+0000 7f5a59000640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a340744e0 0x7f5a340769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:22.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.947+0000 7f5a59000640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a340744e0 0x7f5a340769f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5a5406db20 tx=0x7f5a40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:22.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:22.951+0000 7f5a51600640 1 -- 172.21.15.7:0/1351447739 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a54144ad0 con 0x7f5a540b4690 2024-04-25T11:19:23.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.153+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1351447739 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5a54050740 con 0x7f5a540b4690 2024-04-25T11:19:23.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.154+0000 7f5a51600640 1 -- 172.21.15.7:0/1351447739 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7f5a4804b070 con 0x7f5a540b4690 2024-04-25T11:19:23.152 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1351447739 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a340744e0 msgr2=0x7f5a340769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a340744e0 0x7f5a340769f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5a5406db20 tx=0x7f5a40009040 comp rx=0 tx=0).stop 2024-04-25T11:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1351447739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540b4690 msgr2=0x7f5a540ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540b4690 0x7f5a540ae3b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009890 tx=0x7f5a480046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1351447739 shutdown_connections 2024-04-25T11:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a340744e0 0x7f5a340769f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a54144630 0x7f5a541e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a54143b10 0x7f5a541e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 --2- 172.21.15.7:0/1351447739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a540b4690 0x7f5a540ae3b0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1351447739 >> 172.21.15.7:0/1351447739 conn(0x7f5a54074140 msgr2=0x7f5a54144f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1351447739 shutdown_connections 2024-04-25T11:19:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.157+0000 7f5a5ae00640 1 -- 172.21.15.7:0/1351447739 wait complete. 2024-04-25T11:19:23.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:19:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.464+0000 7f5e55c00640 1 Processor -- start 2024-04-25T11:19:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.464+0000 7f5e55c00640 1 -- start start 2024-04-25T11:19:23.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.466+0000 7f5e55c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5e5006d580 0x7f5e5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.466+0000 7f5e55c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e5006ded0 con 0x7f5e5006f1e0 2024-04-25T11:19:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.466+0000 7f5e55c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e50162700 con 0x7f5e5015f180 2024-04-25T11:19:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.466+0000 7f5e55c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5e5015f590 con 0x7f5e5006d580 2024-04-25T11:19:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.467+0000 7f5e4e000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5e5006d580 0x7f5e5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.467+0000 7f5e4e000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5e5006d580 0x7f5e5006d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43318/0 (socket says 172.21.15.7:43318) 2024-04-25T11:19:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.467+0000 7f5e4e000640 1 -- 172.21.15.7:0/1704989501 learned_addr learned my addr 172.21.15.7:0/1704989501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.467+0000 7f5e4e000640 1 -- 172.21.15.7:0/1704989501 >> v1:172.21.15.87:6789/0 conn(0x7f5e5015f180 legacy=0x7f5e50161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.467+0000 7f5e4e000640 1 -- 172.21.15.7:0/1704989501 >> v1:172.21.15.7:6789/0 conn(0x7f5e5006f1e0 legacy=0x7f5e5006f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.467+0000 7f5e4e000640 1 -- 172.21.15.7:0/1704989501 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e501629e0 con 0x7f5e5006d580 2024-04-25T11:19:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.468+0000 7f5e4e000640 1 --2- 172.21.15.7:0/1704989501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5006d580 0x7f5e5006d990 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009870 tx=0x7f5e3c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216f76b3b3f6cffb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.469+0000 7f5e4d600640 1 -- 172.21.15.7:0/1704989501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c037070 con 0x7f5e5006d580 2024-04-25T11:19:23.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.469+0000 7f5e4d600640 1 -- 172.21.15.7:0/1704989501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c032440 con 0x7f5e5006d580 2024-04-25T11:19:23.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.469+0000 7f5e4d600640 1 -- 172.21.15.7:0/1704989501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c032810 con 0x7f5e5006d580 2024-04-25T11:19:23.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.469+0000 7f5e55c00640 1 -- 172.21.15.7:0/1704989501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5006d580 msgr2=0x7f5e5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.469+0000 7f5e55c00640 1 --2- 172.21.15.7:0/1704989501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5006d580 0x7f5e5006d990 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c009870 tx=0x7f5e3c02a210 comp rx=0 tx=0).stop 2024-04-25T11:19:23.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.470+0000 7f5e55c00640 1 -- 172.21.15.7:0/1704989501 shutdown_connections 2024-04-25T11:19:23.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.470+0000 7f5e55c00640 1 --2- 172.21.15.7:0/1704989501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5006d580 0x7f5e5006d990 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.470+0000 7f5e55c00640 1 -- 172.21.15.7:0/1704989501 >> 172.21.15.7:0/1704989501 conn(0x7f5e500732b0 msgr2=0x7f5e500736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:23.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.470+0000 7f5e55c00640 1 -- 172.21.15.7:0/1704989501 shutdown_connections 2024-04-25T11:19:23.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.470+0000 7f5e55c00640 1 -- 172.21.15.7:0/1704989501 wait complete. 2024-04-25T11:19:23.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.473+0000 7f5e55c00640 1 Processor -- start 2024-04-25T11:19:23.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.473+0000 7f5e55c00640 1 -- start start 2024-04-25T11:19:23.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.475+0000 7f5e55c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5006d580 0x7f5e5009bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.475+0000 7f5e55c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5006f1e0 0x7f5e5009ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.475+0000 7f5e55c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e5015f180 0x7f5e501607d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.475+0000 7f5e55c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5015ec60 con 0x7f5e5006d580 2024-04-25T11:19:23.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.475+0000 7f5e55c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5e50158ae0 con 0x7f5e5006f1e0 2024-04-25T11:19:23.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.475+0000 7f5e55c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5e5015e670 con 0x7f5e5015f180 2024-04-25T11:19:23.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.475+0000 7f5e4ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5006d580 0x7f5e5009bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.476+0000 7f5e4ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5006d580 0x7f5e5009bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43320/0 (socket says 172.21.15.7:43320) 2024-04-25T11:19:23.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.476+0000 7f5e4ea00640 1 -- 172.21.15.7:0/1425455455 learned_addr learned my addr 172.21.15.7:0/1425455455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:23.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.476+0000 7f5e4e000640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5006f1e0 0x7f5e5009ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.476+0000 7f5e4f400640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e5015f180 0x7f5e501607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.476+0000 7f5e4ea00640 1 -- 172.21.15.7:0/1425455455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e5015f180 msgr2=0x7f5e501607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.476+0000 7f5e4ea00640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e5015f180 0x7f5e501607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.476+0000 7f5e4ea00640 1 -- 172.21.15.7:0/1425455455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5006f1e0 msgr2=0x7f5e5009ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.476+0000 7f5e4ea00640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5006f1e0 0x7f5e5009ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.476+0000 7f5e4ea00640 1 -- 172.21.15.7:0/1425455455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e50143d60 con 0x7f5e5006d580 2024-04-25T11:19:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.476+0000 7f5e4e000640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5006f1e0 0x7f5e5009ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.477+0000 7f5e4ea00640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5006d580 0x7f5e5009bf00 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000b9f0 tx=0x7f5e4000beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.477+0000 7f5e46a00640 1 -- 172.21.15.7:0/1425455455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40003b70 con 0x7f5e5006d580 2024-04-25T11:19:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.477+0000 7f5e46a00640 1 -- 172.21.15.7:0/1425455455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e40003d10 con 0x7f5e5006d580 2024-04-25T11:19:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.477+0000 7f5e46a00640 1 -- 172.21.15.7:0/1425455455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000cbb0 con 0x7f5e5006d580 2024-04-25T11:19:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.477+0000 7f5e55c00640 1 -- 172.21.15.7:0/1425455455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e50159370 con 0x7f5e5006d580 2024-04-25T11:19:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.478+0000 7f5e55c00640 1 -- 172.21.15.7:0/1425455455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e50144000 con 0x7f5e5006d580 2024-04-25T11:19:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.479+0000 7f5e55c00640 1 -- 172.21.15.7:0/1425455455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e50050740 con 0x7f5e5006d580 2024-04-25T11:19:23.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.480+0000 7f5e46a00640 1 -- 172.21.15.7:0/1425455455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5e40003ec0 con 0x7f5e5006d580 2024-04-25T11:19:23.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.481+0000 7f5e46a00640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e300744e0 0x7f5e300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:23.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.481+0000 7f5e46a00640 1 -- 172.21.15.7:0/1425455455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f5e40095270 con 0x7f5e5006d580 2024-04-25T11:19:23.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.482+0000 7f5e4e000640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e300744e0 0x7f5e300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:23.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.483+0000 7f5e4e000640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e300744e0 0x7f5e300769f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c034000 tx=0x7f5e3c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:23.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.486+0000 7f5e46a00640 1 -- 172.21.15.7:0/1425455455 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5e4005f180 con 0x7f5e5006d580 2024-04-25T11:19:23.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.706+0000 7f5e55c00640 1 -- 172.21.15.7:0/1425455455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5e500685e0 con 0x7f5e5006d580 2024-04-25T11:19:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.707+0000 7f5e46a00640 1 -- 172.21.15.7:0/1425455455 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f5e40063480 con 0x7f5e5006d580 2024-04-25T11:19:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:19:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.709+0000 7f5e55c00640 1 -- 172.21.15.7:0/1425455455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e300744e0 msgr2=0x7f5e300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.709+0000 7f5e55c00640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e300744e0 0x7f5e300769f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c034000 tx=0x7f5e3c02e000 comp rx=0 tx=0).stop 2024-04-25T11:19:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.709+0000 7f5e55c00640 1 -- 172.21.15.7:0/1425455455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5006d580 msgr2=0x7f5e5009bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.709+0000 7f5e55c00640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5006d580 0x7f5e5009bf00 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000b9f0 tx=0x7f5e4000beb0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.710+0000 7f5e55c00640 1 -- 172.21.15.7:0/1425455455 shutdown_connections 2024-04-25T11:19:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.710+0000 7f5e55c00640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5e300744e0 0x7f5e300769f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.710+0000 7f5e55c00640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5e5015f180 0x7f5e501607d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.710+0000 7f5e55c00640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5006f1e0 0x7f5e5009ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.710+0000 7f5e55c00640 1 --2- 172.21.15.7:0/1425455455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e5006d580 0x7f5e5009bf00 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:23.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.710+0000 7f5e55c00640 1 -- 172.21.15.7:0/1425455455 >> 172.21.15.7:0/1425455455 conn(0x7f5e500732b0 msgr2=0x7f5e50154310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:23.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.710+0000 7f5e55c00640 1 -- 172.21.15.7:0/1425455455 shutdown_connections 2024-04-25T11:19:23.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:23.710+0000 7f5e55c00640 1 -- 172.21.15.7:0/1425455455 wait complete. 2024-04-25T11:19:23.734 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:19:23.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:23.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:23.835 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:19:23.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:23.836 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:19:23.887 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:19:23.887 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:19:23.887 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:23.946 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:19:23.946 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:23.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:24.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.292+0000 7fe7f5600640 1 Processor -- start 2024-04-25T11:19:24.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.292+0000 7fe7f5600640 1 -- start start 2024-04-25T11:19:24.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.294+0000 7fe7f5600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe7f0164710 0x7fe7f00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.294+0000 7fe7f5600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7f00b0da0 0x7fe7f01680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.294+0000 7fe7f5600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f00af070 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.294+0000 7fe7f5600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f01655b0 con 0x7fe7f00b0da0 2024-04-25T11:19:24.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.294+0000 7fe7f5600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f0165a40 con 0x7fe7f0164710 2024-04-25T11:19:24.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.295+0000 7fe7eea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe7f00b4aa0 0x7fe7f00ae960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37746/0 (socket says 172.21.15.7:37746) 2024-04-25T11:19:24.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.295+0000 7fe7eea00640 1 -- 172.21.15.7:0/1510883543 learned_addr learned my addr 172.21.15.7:0/1510883543 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:24.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.295+0000 7fe7ef400640 1 --2- 172.21.15.7:0/1510883543 >> v2:172.21.15.7:3300/0 conn(0x7fe7f00b0da0 0x7fe7f01680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.295+0000 7fe7ed600640 1 -- 172.21.15.7:0/1510883543 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3699664198 0 0) 0x7fe7f00af070 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.295+0000 7fe7ee000640 1 --2- 172.21.15.7:0/1510883543 >> v2:172.21.15.87:3300/0 conn(0x7fe7f0164710 0x7fe7f00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.295+0000 7fe7ed600640 1 -- 172.21.15.7:0/1510883543 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7d4003520 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.295+0000 7fe7ef400640 1 -- 172.21.15.7:0/1510883543 >> v2:172.21.15.87:3300/0 conn(0x7fe7f0164710 msgr2=0x7fe7f00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:24.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.295+0000 7fe7ef400640 1 --2- 172.21.15.7:0/1510883543 >> v2:172.21.15.87:3300/0 conn(0x7fe7f0164710 0x7fe7f00b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.295+0000 7fe7ef400640 1 -- 172.21.15.7:0/1510883543 >> v1:172.21.15.7:6789/0 conn(0x7fe7f00b4aa0 legacy=0x7fe7f00ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.295+0000 7fe7ef400640 1 -- 172.21.15.7:0/1510883543 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f0164c80 con 0x7fe7f00b0da0 2024-04-25T11:19:24.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.297+0000 7fe7ef400640 1 --2- 172.21.15.7:0/1510883543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7f00b0da0 0x7fe7f01680b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00d12a0 tx=0x7fe7e802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4197decf2ceb9eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.297+0000 7fe7ed600640 1 -- 172.21.15.7:0/1510883543 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8007740 con 0x7fe7f00b0da0 2024-04-25T11:19:24.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.297+0000 7fe7ed600640 1 -- 172.21.15.7:0/1510883543 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e80078e0 con 0x7fe7f00b0da0 2024-04-25T11:19:24.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.297+0000 7fe7ed600640 1 -- 172.21.15.7:0/1510883543 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8007cb0 con 0x7fe7f00b0da0 2024-04-25T11:19:24.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.298+0000 7fe7f5600640 1 -- 172.21.15.7:0/1510883543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7f00b0da0 msgr2=0x7fe7f01680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.298+0000 7fe7f5600640 1 --2- 172.21.15.7:0/1510883543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7f00b0da0 0x7fe7f01680b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00d12a0 tx=0x7fe7e802a680 comp rx=0 tx=0).stop 2024-04-25T11:19:24.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.298+0000 7fe7f5600640 1 -- 172.21.15.7:0/1510883543 shutdown_connections 2024-04-25T11:19:24.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.298+0000 7fe7f5600640 1 --2- 172.21.15.7:0/1510883543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7f00b0da0 0x7fe7f01680b0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.298+0000 7fe7f5600640 1 --2- 172.21.15.7:0/1510883543 >> v2:172.21.15.87:3300/0 conn(0x7fe7f0164710 0x7fe7f00b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.298+0000 7fe7f5600640 1 -- 172.21.15.7:0/1510883543 >> 172.21.15.7:0/1510883543 conn(0x7fe7f0074200 msgr2=0x7fe7f006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:24.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.298+0000 7fe7f5600640 1 -- 172.21.15.7:0/1510883543 shutdown_connections 2024-04-25T11:19:24.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.298+0000 7fe7f5600640 1 -- 172.21.15.7:0/1510883543 wait complete. 2024-04-25T11:19:24.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.300+0000 7fe7f5600640 1 Processor -- start 2024-04-25T11:19:24.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.300+0000 7fe7f5600640 1 -- start start 2024-04-25T11:19:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.302+0000 7fe7f5600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7f00b0da0 0x7fe7f009ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.302+0000 7fe7f5600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7f00b4aa0 0x7fe7f00b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.302+0000 7fe7f5600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7f0164710 0x7fe7f00b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.302+0000 7fe7f5600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f00af340 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.302+0000 7fe7f5600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f0165240 con 0x7fe7f00b0da0 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.302+0000 7fe7f5600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f006f300 con 0x7fe7f0164710 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.302+0000 7fe7ee000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7f00b4aa0 0x7fe7f00b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.302+0000 7fe7ee000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7f00b4aa0 0x7fe7f00b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43350/0 (socket says 172.21.15.7:43350) 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.302+0000 7fe7ee000640 1 -- 172.21.15.7:0/4286441356 learned_addr learned my addr 172.21.15.7:0/4286441356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.303+0000 7fe7eea00640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7f00b0da0 0x7fe7f009ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.303+0000 7fe7ee000640 1 -- 172.21.15.7:0/4286441356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7f0164710 msgr2=0x7fe7f00b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.303+0000 7fe7ee000640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7f0164710 0x7fe7f00b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.303+0000 7fe7ee000640 1 -- 172.21.15.7:0/4286441356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7f00b0da0 msgr2=0x7fe7f009ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.303+0000 7fe7ee000640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7f00b0da0 0x7fe7f009ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.303+0000 7fe7ee000640 1 -- 172.21.15.7:0/4286441356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f01fac70 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.303+0000 7fe7ee000640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7f00b4aa0 0x7fe7f00b28a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c7f0 tx=0x7fe7dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.303+0000 7fe7e6a00640 1 -- 172.21.15.7:0/4286441356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc015070 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.303+0000 7fe7e6a00640 1 -- 172.21.15.7:0/4286441356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7dc004220 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.303+0000 7fe7f5600640 1 -- 172.21.15.7:0/4286441356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f01faf50 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.303+0000 7fe7e6a00640 1 -- 172.21.15.7:0/4286441356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc007620 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.304+0000 7fe7f5600640 1 -- 172.21.15.7:0/4286441356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f01fb510 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.305+0000 7fe7f5600640 1 -- 172.21.15.7:0/4286441356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f01678b0 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.306+0000 7fe7e6a00640 1 -- 172.21.15.7:0/4286441356 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe7dc0079d0 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.307+0000 7fe7e6a00640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7d00744b0 0x7fe7d00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.307+0000 7fe7e6a00640 1 -- 172.21.15.7:0/4286441356 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fe7dc095480 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.307+0000 7fe7eea00640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7d00744b0 0x7fe7d00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.308+0000 7fe7eea00640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7d00744b0 0x7fe7d00769c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0000f60 tx=0x7fe7e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.312+0000 7fe7e6a00640 1 -- 172.21.15.7:0/4286441356 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe7f01678b0 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.516+0000 7fe7f5600640 1 -- 172.21.15.7:0/4286441356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7f009f390 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.519+0000 7fe7e6a00640 1 -- 172.21.15.7:0/4286441356 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16788 (secure 0 0 0) 0x7fe7dc05f9e0 con 0x7fe7f00b4aa0 2024-04-25T11:19:24.518 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:24.518 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:10.691402+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:07.435764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:08.209216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 -- 172.21.15.7:0/4286441356 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7d00744b0 msgr2=0x7fe7d00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7d00744b0 0x7fe7d00769c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0000f60 tx=0x7fe7e0009040 comp rx=0 tx=0).stop 2024-04-25T11:19:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 -- 172.21.15.7:0/4286441356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7f00b4aa0 msgr2=0x7fe7f00b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7f00b4aa0 0x7fe7f00b28a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00c7f0 tx=0x7fe7dc00ccb0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 -- 172.21.15.7:0/4286441356 shutdown_connections 2024-04-25T11:19:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe7d00744b0 0x7fe7d00769c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7f0164710 0x7fe7f00b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7f00b4aa0 0x7fe7f00b28a0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 --2- 172.21.15.7:0/4286441356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7f00b0da0 0x7fe7f009ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 -- 172.21.15.7:0/4286441356 >> 172.21.15.7:0/4286441356 conn(0x7fe7f0074200 msgr2=0x7fe7f01623c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 -- 172.21.15.7:0/4286441356 shutdown_connections 2024-04-25T11:19:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.522+0000 7fe7f5600640 1 -- 172.21.15.7:0/4286441356 wait complete. 2024-04-25T11:19:24.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:07.435764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:08.209216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:24.551 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:24.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.838+0000 7f10d2200640 1 Processor -- start 2024-04-25T11:19:24.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.838+0000 7f10d2200640 1 -- start start 2024-04-25T11:19:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.841+0000 7f10d2200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f10cc157320 0x7f10cc164710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.841+0000 7f10d2200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10cc06f3a0 0x7f10cc06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.841+0000 7f10d2200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc0b1390 con 0x7f10cc0b6720 2024-04-25T11:19:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.841+0000 7f10d2200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f10cc160a60 con 0x7f10cc157320 2024-04-25T11:19:24.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.841+0000 7f10d2200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc0ae880 con 0x7f10cc06f3a0 2024-04-25T11:19:24.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.842+0000 7f10c2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f10cc157320 0x7f10cc164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.842+0000 7f10c2a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f10cc157320 0x7f10cc164710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:55666/0 (socket says 172.21.15.7:55666) 2024-04-25T11:19:24.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.842+0000 7f10c2a00640 1 -- 172.21.15.7:0/672951286 learned_addr learned my addr 172.21.15.7:0/672951286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:24.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.842+0000 7f10cbe00640 1 --2- 172.21.15.7:0/672951286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10cc06f3a0 0x7f10cc06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.843+0000 7f10c2a00640 1 -- 172.21.15.7:0/672951286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10cc06f3a0 msgr2=0x7f10cc06f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.843+0000 7f10c2a00640 1 --2- 172.21.15.7:0/672951286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10cc06f3a0 0x7f10cc06f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.843+0000 7f10c2a00640 1 -- 172.21.15.7:0/672951286 >> v1:172.21.15.87:6789/0 conn(0x7f10cc0b6720 legacy=0x7f10cc160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.843+0000 7f10c2a00640 1 -- 172.21.15.7:0/672951286 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc160710 con 0x7f10cc157320 2024-04-25T11:19:24.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.844+0000 7f10c2a00640 1 --2- 172.21.15.7:0/672951286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10cc157320 0x7f10cc164710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009870 tx=0x7f10b402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0e45e6dde580660 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.845+0000 7f10caa00640 1 -- 172.21.15.7:0/672951286 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4037070 con 0x7f10cc157320 2024-04-25T11:19:24.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.845+0000 7f10caa00640 1 -- 172.21.15.7:0/672951286 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b4032440 con 0x7f10cc157320 2024-04-25T11:19:24.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.845+0000 7f10caa00640 1 -- 172.21.15.7:0/672951286 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4032810 con 0x7f10cc157320 2024-04-25T11:19:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.846+0000 7f10d2200640 1 -- 172.21.15.7:0/672951286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10cc157320 msgr2=0x7f10cc164710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.846+0000 7f10d2200640 1 --2- 172.21.15.7:0/672951286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10cc157320 0x7f10cc164710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009870 tx=0x7f10b402a270 comp rx=0 tx=0).stop 2024-04-25T11:19:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.846+0000 7f10d2200640 1 -- 172.21.15.7:0/672951286 shutdown_connections 2024-04-25T11:19:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.846+0000 7f10d2200640 1 --2- 172.21.15.7:0/672951286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10cc06f3a0 0x7f10cc06f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.846+0000 7f10d2200640 1 --2- 172.21.15.7:0/672951286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10cc157320 0x7f10cc164710 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.846+0000 7f10d2200640 1 -- 172.21.15.7:0/672951286 >> 172.21.15.7:0/672951286 conn(0x7f10cc0742c0 msgr2=0x7f10cc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.847+0000 7f10d2200640 1 -- 172.21.15.7:0/672951286 shutdown_connections 2024-04-25T11:19:24.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.847+0000 7f10d2200640 1 -- 172.21.15.7:0/672951286 wait complete. 2024-04-25T11:19:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.849+0000 7f10d2200640 1 Processor -- start 2024-04-25T11:19:24.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.849+0000 7f10d2200640 1 -- start start 2024-04-25T11:19:24.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10d2200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10cc06f3a0 0x7f10cc142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10d2200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10cc0b6720 0x7f10cc143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10d2200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10cc157320 0x7f10cc0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10d2200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0aee00 con 0x7f10cc06f3a0 2024-04-25T11:19:24.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10d2200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0aeb20 con 0x7f10cc0b6720 2024-04-25T11:19:24.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10d2200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc06fcf0 con 0x7f10cc157320 2024-04-25T11:19:24.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10cb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10cc06f3a0 0x7f10cc142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10cb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10cc06f3a0 0x7f10cc142d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43354/0 (socket says 172.21.15.7:43354) 2024-04-25T11:19:24.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10cb400640 1 -- 172.21.15.7:0/4177955470 learned_addr learned my addr 172.21.15.7:0/4177955470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10c2a00640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10cc0b6720 0x7f10cc143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10cb400640 1 -- 172.21.15.7:0/4177955470 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10cc157320 msgr2=0x7f10cc0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.851+0000 7f10cb400640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10cc157320 0x7f10cc0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.852+0000 7f10cb400640 1 -- 172.21.15.7:0/4177955470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10cc0b6720 msgr2=0x7f10cc143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.852+0000 7f10cb400640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10cc0b6720 0x7f10cc143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.852+0000 7f10cb400640 1 -- 172.21.15.7:0/4177955470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc149900 con 0x7f10cc06f3a0 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.852+0000 7f10cb400640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10cc06f3a0 0x7f10cc142d90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00c6b0 tx=0x7f10bc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.852+0000 7f10c3e00640 1 -- 172.21.15.7:0/4177955470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc016020 con 0x7f10cc06f3a0 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.852+0000 7f10c3e00640 1 -- 172.21.15.7:0/4177955470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10bc006d50 con 0x7f10cc06f3a0 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.852+0000 7f10d2200640 1 -- 172.21.15.7:0/4177955470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc143a60 con 0x7f10cc06f3a0 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.852+0000 7f10c3e00640 1 -- 172.21.15.7:0/4177955470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc003bb0 con 0x7f10cc06f3a0 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.852+0000 7f10d2200640 1 -- 172.21.15.7:0/4177955470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc157b90 con 0x7f10cc06f3a0 2024-04-25T11:19:24.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.853+0000 7f10d2200640 1 -- 172.21.15.7:0/4177955470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc050740 con 0x7f10cc06f3a0 2024-04-25T11:19:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.855+0000 7f10c3e00640 1 -- 172.21.15.7:0/4177955470 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f10bc0050d0 con 0x7f10cc06f3a0 2024-04-25T11:19:24.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.856+0000 7f10c3e00640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10a80744b0 0x7f10a80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:24.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.856+0000 7f10c3e00640 1 -- 172.21.15.7:0/4177955470 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f10bc094ff0 con 0x7f10cc06f3a0 2024-04-25T11:19:24.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.856+0000 7f10c2a00640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10a80744b0 0x7f10a80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:24.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.857+0000 7f10c2a00640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10a80744b0 0x7f10a80769c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f10b4034000 tx=0x7f10b402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:24.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:24.861+0000 7f10c3e00640 1 -- 172.21.15.7:0/4177955470 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f10bc020020 con 0x7f10cc06f3a0 2024-04-25T11:19:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.071+0000 7f10d2200640 1 -- 172.21.15.7:0/4177955470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10cc157e60 con 0x7f10cc06f3a0 2024-04-25T11:19:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.072+0000 7f10c3e00640 1 -- 172.21.15.7:0/4177955470 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7f10bc05f690 con 0x7f10cc06f3a0 2024-04-25T11:19:25.071 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:19:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.075+0000 7f10d2200640 1 -- 172.21.15.7:0/4177955470 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10a80744b0 msgr2=0x7f10a80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.075+0000 7f10d2200640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10a80744b0 0x7f10a80769c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f10b4034000 tx=0x7f10b402e000 comp rx=0 tx=0).stop 2024-04-25T11:19:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.075+0000 7f10d2200640 1 -- 172.21.15.7:0/4177955470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10cc06f3a0 msgr2=0x7f10cc142d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.075+0000 7f10d2200640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10cc06f3a0 0x7f10cc142d90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00c6b0 tx=0x7f10bc00cb70 comp rx=0 tx=0).stop 2024-04-25T11:19:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.075+0000 7f10d2200640 1 -- 172.21.15.7:0/4177955470 shutdown_connections 2024-04-25T11:19:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.075+0000 7f10d2200640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f10a80744b0 0x7f10a80769c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.075+0000 7f10d2200640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f10cc157320 0x7f10cc0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.075+0000 7f10d2200640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f10cc0b6720 0x7f10cc143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.075+0000 7f10d2200640 1 --2- 172.21.15.7:0/4177955470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10cc06f3a0 0x7f10cc142d90 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.075+0000 7f10d2200640 1 -- 172.21.15.7:0/4177955470 >> 172.21.15.7:0/4177955470 conn(0x7f10cc0742c0 msgr2=0x7f10cc0d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.076+0000 7f10d2200640 1 -- 172.21.15.7:0/4177955470 shutdown_connections 2024-04-25T11:19:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.076+0000 7f10d2200640 1 -- 172.21.15.7:0/4177955470 wait complete. 2024-04-25T11:19:25.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:19:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.397+0000 7f90da400640 1 Processor -- start 2024-04-25T11:19:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.397+0000 7f90da400640 1 -- start start 2024-04-25T11:19:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.399+0000 7f90da400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f90d4145e40 0x7f90d4146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.400+0000 7f90da400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90d4144a80 0x7f90d40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.400+0000 7f90da400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90d40b66b0 con 0x7f90d40b4870 2024-04-25T11:19:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.400+0000 7f90da400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f90d4145b70 con 0x7f90d4145e40 2024-04-25T11:19:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.400+0000 7f90da400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f90d40b58d0 con 0x7f90d4144a80 2024-04-25T11:19:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.400+0000 7f90cbe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f90d40b4870 0x7f90d40b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37762/0 (socket says 172.21.15.7:37762) 2024-04-25T11:19:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.400+0000 7f90cbe00640 1 -- 172.21.15.7:0/3984064689 learned_addr learned my addr 172.21.15.7:0/3984064689 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.400+0000 7f90d9000640 1 --2- 172.21.15.7:0/3984064689 >> v2:172.21.15.87:3300/0 conn(0x7f90d4145e40 0x7f90d4146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.400+0000 7f90d9a00640 1 --2- 172.21.15.7:0/3984064689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90d4144a80 0x7f90d40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.401+0000 7f90d3e00640 1 -- 172.21.15.7:0/3984064689 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3036176436 0 0) 0x7f90d40b66b0 con 0x7f90d40b4870 2024-04-25T11:19:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.401+0000 7f90d3e00640 1 -- 172.21.15.7:0/3984064689 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90b4003520 con 0x7f90d40b4870 2024-04-25T11:19:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.401+0000 7f90d9000640 1 -- 172.21.15.7:0/3984064689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90d4144a80 msgr2=0x7f90d40b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.401+0000 7f90d9000640 1 --2- 172.21.15.7:0/3984064689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90d4144a80 0x7f90d40b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.401+0000 7f90d9000640 1 -- 172.21.15.7:0/3984064689 >> v1:172.21.15.7:6789/0 conn(0x7f90d40b4870 legacy=0x7f90d40b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.401+0000 7f90d3e00640 1 -- 172.21.15.7:0/3984064689 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2055216886 0 0) 0x7f90b4003520 con 0x7f90d40b4870 2024-04-25T11:19:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.401+0000 7f90d9000640 1 -- 172.21.15.7:0/3984064689 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d40b5b30 con 0x7f90d4145e40 2024-04-25T11:19:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.403+0000 7f90d9000640 1 --2- 172.21.15.7:0/3984064689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90d4145e40 0x7f90d4146250 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f90c40097a0 tx=0x7f90c402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6581002a5a613a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:25.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.403+0000 7f90d3e00640 1 -- 172.21.15.7:0/3984064689 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4037070 con 0x7f90d4145e40 2024-04-25T11:19:25.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.403+0000 7f90d3e00640 1 -- 172.21.15.7:0/3984064689 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c402ab10 con 0x7f90d4145e40 2024-04-25T11:19:25.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.403+0000 7f90d3e00640 1 -- 172.21.15.7:0/3984064689 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc001690 con 0x7f90d4145e40 2024-04-25T11:19:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.404+0000 7f90da400640 1 -- 172.21.15.7:0/3984064689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90d4145e40 msgr2=0x7f90d4146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.404+0000 7f90da400640 1 --2- 172.21.15.7:0/3984064689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90d4145e40 0x7f90d4146250 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f90c40097a0 tx=0x7f90c402a080 comp rx=0 tx=0).stop 2024-04-25T11:19:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.404+0000 7f90da400640 1 -- 172.21.15.7:0/3984064689 shutdown_connections 2024-04-25T11:19:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.404+0000 7f90da400640 1 --2- 172.21.15.7:0/3984064689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90d4144a80 0x7f90d40b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.404+0000 7f90da400640 1 --2- 172.21.15.7:0/3984064689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90d4145e40 0x7f90d4146250 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.404+0000 7f90da400640 1 -- 172.21.15.7:0/3984064689 >> 172.21.15.7:0/3984064689 conn(0x7f90d40740f0 msgr2=0x7f90d406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.404+0000 7f90da400640 1 -- 172.21.15.7:0/3984064689 shutdown_connections 2024-04-25T11:19:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.404+0000 7f90da400640 1 -- 172.21.15.7:0/3984064689 wait complete. 2024-04-25T11:19:25.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.407+0000 7f90da400640 1 Processor -- start 2024-04-25T11:19:25.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.407+0000 7f90da400640 1 -- start start 2024-04-25T11:19:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.408+0000 7f90da400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90d40b4870 0x7f90d41530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.408+0000 7f90da400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90d4144a80 0x7f90d4142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.409+0000 7f90da400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90d4145e40 0x7f90d4142a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.409+0000 7f90da400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90d40b63d0 con 0x7f90d4144a80 2024-04-25T11:19:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.409+0000 7f90da400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f90d40b60f0 con 0x7f90d4145e40 2024-04-25T11:19:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.409+0000 7f90da400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f90d41606f0 con 0x7f90d40b4870 2024-04-25T11:19:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.409+0000 7f90cbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90d4144a80 0x7f90d4142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.409+0000 7f90cbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90d4144a80 0x7f90d4142550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43360/0 (socket says 172.21.15.7:43360) 2024-04-25T11:19:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.409+0000 7f90cbe00640 1 -- 172.21.15.7:0/3577796479 learned_addr learned my addr 172.21.15.7:0/3577796479 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.409+0000 7f90d9a00640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90d4145e40 0x7f90d4142a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.409+0000 7f90d9000640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90d40b4870 0x7f90d41530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.410+0000 7f90cbe00640 1 -- 172.21.15.7:0/3577796479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90d40b4870 msgr2=0x7f90d41530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.410+0000 7f90cbe00640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90d40b4870 0x7f90d41530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.410+0000 7f90cbe00640 1 -- 172.21.15.7:0/3577796479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90d4145e40 msgr2=0x7f90d4142a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.410+0000 7f90cbe00640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90d4145e40 0x7f90d4142a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.410+0000 7f90cbe00640 1 -- 172.21.15.7:0/3577796479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d41f7a80 con 0x7f90d4144a80 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.410+0000 7f90cbe00640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90d4144a80 0x7f90d4142550 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c4a0 tx=0x7f90bc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.410+0000 7f90d1600640 1 -- 172.21.15.7:0/3577796479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc018070 con 0x7f90d4144a80 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.410+0000 7f90d1600640 1 -- 172.21.15.7:0/3577796479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90bc007500 con 0x7f90d4144a80 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.410+0000 7f90d1600640 1 -- 172.21.15.7:0/3577796479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc004a20 con 0x7f90d4144a80 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.410+0000 7f90da400640 1 -- 172.21.15.7:0/3577796479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d41f7d60 con 0x7f90d4144a80 2024-04-25T11:19:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.410+0000 7f90da400640 1 -- 172.21.15.7:0/3577796479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d41f8320 con 0x7f90d4144a80 2024-04-25T11:19:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.412+0000 7f90da400640 1 -- 172.21.15.7:0/3577796479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d41f77f0 con 0x7f90d4144a80 2024-04-25T11:19:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.413+0000 7f90d1600640 1 -- 172.21.15.7:0/3577796479 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f90bc004bc0 con 0x7f90d4144a80 2024-04-25T11:19:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.414+0000 7f90d1600640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90b00744b0 0x7f90b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.414+0000 7f90d1600640 1 -- 172.21.15.7:0/3577796479 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f90bc0946f0 con 0x7f90d4144a80 2024-04-25T11:19:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.414+0000 7f90d9000640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90b00744b0 0x7f90b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.415+0000 7f90d9000640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90b00744b0 0x7f90b00769c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f90c4034000 tx=0x7f90c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.419+0000 7f90d1600640 1 -- 172.21.15.7:0/3577796479 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f90bc05ece0 con 0x7f90d4144a80 2024-04-25T11:19:25.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.626+0000 7f90da400640 1 -- 172.21.15.7:0/3577796479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f90d4050740 con 0x7f90d4144a80 2024-04-25T11:19:25.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.627+0000 7f90d1600640 1 -- 172.21.15.7:0/3577796479 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7f90bc05e5e0 con 0x7f90d4144a80 2024-04-25T11:19:25.625 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:19:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 -- 172.21.15.7:0/3577796479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90b00744b0 msgr2=0x7f90b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90b00744b0 0x7f90b00769c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f90c4034000 tx=0x7f90c402e000 comp rx=0 tx=0).stop 2024-04-25T11:19:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 -- 172.21.15.7:0/3577796479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90d4144a80 msgr2=0x7f90d4142550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90d4144a80 0x7f90d4142550 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00c4a0 tx=0x7f90bc00c960 comp rx=0 tx=0).stop 2024-04-25T11:19:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 -- 172.21.15.7:0/3577796479 shutdown_connections 2024-04-25T11:19:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90b00744b0 0x7f90b00769c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90d4145e40 0x7f90d4142a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90d4144a80 0x7f90d4142550 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 --2- 172.21.15.7:0/3577796479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90d40b4870 0x7f90d41530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 -- 172.21.15.7:0/3577796479 >> 172.21.15.7:0/3577796479 conn(0x7f90d40740f0 msgr2=0x7f90d40c9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 -- 172.21.15.7:0/3577796479 shutdown_connections 2024-04-25T11:19:25.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.630+0000 7f90da400640 1 -- 172.21.15.7:0/3577796479 wait complete. 2024-04-25T11:19:25.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:19:25.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.944+0000 7f5dcee00640 1 Processor -- start 2024-04-25T11:19:25.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.945+0000 7f5dcee00640 1 -- start start 2024-04-25T11:19:25.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.947+0000 7f5dcee00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5dc8160470 0x7f5dc806eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.947+0000 7f5dcee00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc8160b70 con 0x7f5dc809b540 2024-04-25T11:19:25.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.947+0000 7f5dcee00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc8160e50 con 0x7f5dc815db10 2024-04-25T11:19:25.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.947+0000 7f5dcee00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc80b2da0 con 0x7f5dc8160470 2024-04-25T11:19:25.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.947+0000 7f5dce400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5dc809b540 0x7f5dc809b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37774/0 (socket says 172.21.15.7:37774) 2024-04-25T11:19:25.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.947+0000 7f5dce400640 1 -- 172.21.15.7:0/4194046302 learned_addr learned my addr 172.21.15.7:0/4194046302 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:25.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.947+0000 7f5dcd000640 1 --2- 172.21.15.7:0/4194046302 >> v2:172.21.15.87:3300/0 conn(0x7f5dc8160470 0x7f5dc806eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.948+0000 7f5dc7e00640 1 -- 172.21.15.7:0/4194046302 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2593673678 0 0) 0x7f5dc8160b70 con 0x7f5dc809b540 2024-04-25T11:19:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.948+0000 7f5dc7e00640 1 -- 172.21.15.7:0/4194046302 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5db0003520 con 0x7f5dc809b540 2024-04-25T11:19:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.948+0000 7f5dcd000640 1 -- 172.21.15.7:0/4194046302 >> v1:172.21.15.87:6789/0 conn(0x7f5dc815db10 legacy=0x7f5dc80b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.948+0000 7f5dcd000640 1 -- 172.21.15.7:0/4194046302 >> v1:172.21.15.7:6789/0 conn(0x7f5dc809b540 legacy=0x7f5dc809b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.948+0000 7f5dcd000640 1 -- 172.21.15.7:0/4194046302 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc8161130 con 0x7f5dc8160470 2024-04-25T11:19:25.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.950+0000 7f5dcd000640 1 --2- 172.21.15.7:0/4194046302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dc8160470 0x7f5dc806eb50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5db40097d0 tx=0x7f5db402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a6ffeddf3e275b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.951+0000 7f5dc7e00640 1 -- 172.21.15.7:0/4194046302 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4037070 con 0x7f5dc8160470 2024-04-25T11:19:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.951+0000 7f5dc7e00640 1 -- 172.21.15.7:0/4194046302 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db4032440 con 0x7f5dc8160470 2024-04-25T11:19:25.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.951+0000 7f5dc7e00640 1 -- 172.21.15.7:0/4194046302 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4032810 con 0x7f5dc8160470 2024-04-25T11:19:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.952+0000 7f5dcee00640 1 -- 172.21.15.7:0/4194046302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dc8160470 msgr2=0x7f5dc806eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.952+0000 7f5dcee00640 1 --2- 172.21.15.7:0/4194046302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dc8160470 0x7f5dc806eb50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5db40097d0 tx=0x7f5db402a250 comp rx=0 tx=0).stop 2024-04-25T11:19:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.952+0000 7f5dcee00640 1 -- 172.21.15.7:0/4194046302 shutdown_connections 2024-04-25T11:19:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.952+0000 7f5dcee00640 1 --2- 172.21.15.7:0/4194046302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dc8160470 0x7f5dc806eb50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.952+0000 7f5dcee00640 1 -- 172.21.15.7:0/4194046302 >> 172.21.15.7:0/4194046302 conn(0x7f5dc8074460 msgr2=0x7f5dc80b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.952+0000 7f5dcee00640 1 -- 172.21.15.7:0/4194046302 shutdown_connections 2024-04-25T11:19:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.952+0000 7f5dcee00640 1 -- 172.21.15.7:0/4194046302 wait complete. 2024-04-25T11:19:25.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.954+0000 7f5dcee00640 1 Processor -- start 2024-04-25T11:19:25.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.955+0000 7f5dcee00640 1 -- start start 2024-04-25T11:19:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.956+0000 7f5dcee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc809b540 0x7f5dc81f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.956+0000 7f5dcee00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5dc815db10 0x7f5dc815b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.956+0000 7f5dcee00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dc8160470 0x7f5dc81f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.956+0000 7f5dcee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc8160880 con 0x7f5dc809b540 2024-04-25T11:19:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.956+0000 7f5dcee00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc806f090 con 0x7f5dc8160470 2024-04-25T11:19:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.956+0000 7f5dcee00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc80d29f0 con 0x7f5dc815db10 2024-04-25T11:19:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.957+0000 7f5dcda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc809b540 0x7f5dc81f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.957+0000 7f5dcda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc809b540 0x7f5dc81f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43366/0 (socket says 172.21.15.7:43366) 2024-04-25T11:19:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.957+0000 7f5dcda00640 1 -- 172.21.15.7:0/121937697 learned_addr learned my addr 172.21.15.7:0/121937697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:25.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.957+0000 7f5dce400640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dc8160470 0x7f5dc81f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.957+0000 7f5dcda00640 1 -- 172.21.15.7:0/121937697 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5dc815db10 msgr2=0x7f5dc815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.957+0000 7f5dcda00640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5dc815db10 0x7f5dc815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.957+0000 7f5dcda00640 1 -- 172.21.15.7:0/121937697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dc8160470 msgr2=0x7f5dc81f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.957+0000 7f5dcda00640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dc8160470 0x7f5dc81f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.957+0000 7f5dcda00640 1 -- 172.21.15.7:0/121937697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc815bd70 con 0x7f5dc809b540 2024-04-25T11:19:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.958+0000 7f5dcda00640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc809b540 0x7f5dc81f5f20 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c3f0 tx=0x7f5dbc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.958+0000 7f5dc5600640 1 -- 172.21.15.7:0/121937697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc0052d0 con 0x7f5dc809b540 2024-04-25T11:19:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.958+0000 7f5dc5600640 1 -- 172.21.15.7:0/121937697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dbc005490 con 0x7f5dc809b540 2024-04-25T11:19:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.958+0000 7f5dc5600640 1 -- 172.21.15.7:0/121937697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc007260 con 0x7f5dc809b540 2024-04-25T11:19:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.958+0000 7f5dcee00640 1 -- 172.21.15.7:0/121937697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc815c050 con 0x7f5dc809b540 2024-04-25T11:19:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.958+0000 7f5dcee00640 1 -- 172.21.15.7:0/121937697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc815c610 con 0x7f5dc809b540 2024-04-25T11:19:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.959+0000 7f5dcee00640 1 -- 172.21.15.7:0/121937697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc8050740 con 0x7f5dc809b540 2024-04-25T11:19:25.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.961+0000 7f5dc5600640 1 -- 172.21.15.7:0/121937697 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5dbc007400 con 0x7f5dc809b540 2024-04-25T11:19:25.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.962+0000 7f5dc5600640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5da80744e0 0x7f5da80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:25.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.962+0000 7f5dc5600640 1 -- 172.21.15.7:0/121937697 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f5dbc094910 con 0x7f5dc809b540 2024-04-25T11:19:25.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.962+0000 7f5dcd000640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5da80744e0 0x7f5da80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:25.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.963+0000 7f5dcd000640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5da80744e0 0x7f5da80769f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5db4034000 tx=0x7f5db402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:25.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:25.967+0000 7f5dc5600640 1 -- 172.21.15.7:0/121937697 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5dbc05ee70 con 0x7f5dc809b540 2024-04-25T11:19:26.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.174+0000 7f5dcee00640 1 -- 172.21.15.7:0/121937697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5dc80b2470 con 0x7f5dc809b540 2024-04-25T11:19:26.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.175+0000 7f5dc5600640 1 -- 172.21.15.7:0/121937697 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7f5dbc05e770 con 0x7f5dc809b540 2024-04-25T11:19:26.173 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:19:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.177+0000 7f5dcee00640 1 -- 172.21.15.7:0/121937697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5da80744e0 msgr2=0x7f5da80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.177+0000 7f5dcee00640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5da80744e0 0x7f5da80769f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5db4034000 tx=0x7f5db402c000 comp rx=0 tx=0).stop 2024-04-25T11:19:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.177+0000 7f5dcee00640 1 -- 172.21.15.7:0/121937697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc809b540 msgr2=0x7f5dc81f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.177+0000 7f5dcee00640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc809b540 0x7f5dc81f5f20 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00c3f0 tx=0x7f5dbc00c8b0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.178+0000 7f5dcee00640 1 -- 172.21.15.7:0/121937697 shutdown_connections 2024-04-25T11:19:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.178+0000 7f5dcee00640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5da80744e0 0x7f5da80769f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.178+0000 7f5dcee00640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5dc8160470 0x7f5dc81f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.178+0000 7f5dcee00640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5dc815db10 0x7f5dc815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.178+0000 7f5dcee00640 1 --2- 172.21.15.7:0/121937697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dc809b540 0x7f5dc81f5f20 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.178+0000 7f5dcee00640 1 -- 172.21.15.7:0/121937697 >> 172.21.15.7:0/121937697 conn(0x7f5dc8074460 msgr2=0x7f5dc80b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:26.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.178+0000 7f5dcee00640 1 -- 172.21.15.7:0/121937697 shutdown_connections 2024-04-25T11:19:26.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.178+0000 7f5dcee00640 1 -- 172.21.15.7:0/121937697 wait complete. 2024-04-25T11:19:26.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:19:26.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.481+0000 7f57bda00640 1 Processor -- start 2024-04-25T11:19:26.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.481+0000 7f57bda00640 1 -- start start 2024-04-25T11:19:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.483+0000 7f57bda00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57b8141290 0x7f57b81416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.483+0000 7f57bda00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57b806d610 0x7f57b806da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.483+0000 7f57bda00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57b806c570 con 0x7f57b815eed0 2024-04-25T11:19:26.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.483+0000 7f57bda00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f57b806c850 con 0x7f57b8141290 2024-04-25T11:19:26.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.483+0000 7f57bda00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57b806cb30 con 0x7f57b806d610 2024-04-25T11:19:26.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.483+0000 7f57b7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57b8141290 0x7f57b81416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.483+0000 7f57b7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f57b8141290 0x7f57b81416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43378/0 (socket says 172.21.15.7:43378) 2024-04-25T11:19:26.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.483+0000 7f57b7400640 1 -- 172.21.15.7:0/3220808372 learned_addr learned my addr 172.21.15.7:0/3220808372 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:26.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.484+0000 7f57bd000640 1 --2- 172.21.15.7:0/3220808372 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57b806d610 0x7f57b806da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.484+0000 7f57b7400640 1 -- 172.21.15.7:0/3220808372 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57b806d610 msgr2=0x7f57b806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:26.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.484+0000 7f57b7400640 1 --2- 172.21.15.7:0/3220808372 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57b806d610 0x7f57b806da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.484+0000 7f57b7400640 1 -- 172.21.15.7:0/3220808372 >> v1:172.21.15.87:6789/0 conn(0x7f57b815eed0 legacy=0x7f57b80cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:26.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.484+0000 7f57b7400640 1 -- 172.21.15.7:0/3220808372 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b806ce10 con 0x7f57b8141290 2024-04-25T11:19:26.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.485+0000 7f57b7400640 1 --2- 172.21.15.7:0/3220808372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b8141290 0x7f57b81416a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f57b806eed0 tx=0x7f57a002a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f34694d771afb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:26.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.485+0000 7f57b6a00640 1 -- 172.21.15.7:0/3220808372 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a0036070 con 0x7f57b8141290 2024-04-25T11:19:26.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.485+0000 7f57b6a00640 1 -- 172.21.15.7:0/3220808372 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a0031600 con 0x7f57b8141290 2024-04-25T11:19:26.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.485+0000 7f57b6a00640 1 -- 172.21.15.7:0/3220808372 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a00319d0 con 0x7f57b8141290 2024-04-25T11:19:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.486+0000 7f57bda00640 1 -- 172.21.15.7:0/3220808372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b8141290 msgr2=0x7f57b81416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.486+0000 7f57bda00640 1 --2- 172.21.15.7:0/3220808372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b8141290 0x7f57b81416a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f57b806eed0 tx=0x7f57a002a6b0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.486+0000 7f57bda00640 1 -- 172.21.15.7:0/3220808372 shutdown_connections 2024-04-25T11:19:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.486+0000 7f57bda00640 1 --2- 172.21.15.7:0/3220808372 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57b806d610 0x7f57b806da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.486+0000 7f57bda00640 1 --2- 172.21.15.7:0/3220808372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b8141290 0x7f57b81416a0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.486+0000 7f57bda00640 1 -- 172.21.15.7:0/3220808372 >> 172.21.15.7:0/3220808372 conn(0x7f57b80d1540 msgr2=0x7f57b80d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.486+0000 7f57bda00640 1 -- 172.21.15.7:0/3220808372 shutdown_connections 2024-04-25T11:19:26.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.486+0000 7f57bda00640 1 -- 172.21.15.7:0/3220808372 wait complete. 2024-04-25T11:19:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.489+0000 7f57bda00640 1 Processor -- start 2024-04-25T11:19:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.489+0000 7f57bda00640 1 -- start start 2024-04-25T11:19:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.490+0000 7f57bda00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57b806d610 0x7f57b81e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.490+0000 7f57bda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b8141290 0x7f57b81e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.490+0000 7f57bda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b815eed0 0x7f57b8133b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.490+0000 7f57bda00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8140f20 con 0x7f57b815eed0 2024-04-25T11:19:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.490+0000 7f57bda00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57b8140da0 con 0x7f57b8141290 2024-04-25T11:19:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.490+0000 7f57bda00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57b80b4590 con 0x7f57b806d610 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.491+0000 7f57bd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b815eed0 0x7f57b8133b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.491+0000 7f57bd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b815eed0 0x7f57b8133b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43392/0 (socket says 172.21.15.7:43392) 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.491+0000 7f57bd000640 1 -- 172.21.15.7:0/2833477131 learned_addr learned my addr 172.21.15.7:0/2833477131 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.491+0000 7f57b7400640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b8141290 0x7f57b81e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.491+0000 7f57b7e00640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57b806d610 0x7f57b81e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.491+0000 7f57bd000640 1 -- 172.21.15.7:0/2833477131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57b806d610 msgr2=0x7f57b81e64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.492+0000 7f57bd000640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57b806d610 0x7f57b81e64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.492+0000 7f57bd000640 1 -- 172.21.15.7:0/2833477131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b8141290 msgr2=0x7f57b81e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.492+0000 7f57bd000640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b8141290 0x7f57b81e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.492+0000 7f57bd000640 1 -- 172.21.15.7:0/2833477131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b81e82e0 con 0x7f57b815eed0 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.492+0000 7f57b7400640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b8141290 0x7f57b81e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.492+0000 7f57bd000640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b815eed0 0x7f57b8133b70 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f57b000c5f0 tx=0x7f57b000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:26.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.492+0000 7f57afe00640 1 -- 172.21.15.7:0/2833477131 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0015070 con 0x7f57b815eed0 2024-04-25T11:19:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.492+0000 7f57afe00640 1 -- 172.21.15.7:0/2833477131 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b0004220 con 0x7f57b815eed0 2024-04-25T11:19:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.492+0000 7f57afe00640 1 -- 172.21.15.7:0/2833477131 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0003610 con 0x7f57b815eed0 2024-04-25T11:19:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.492+0000 7f57bda00640 1 -- 172.21.15.7:0/2833477131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b81340b0 con 0x7f57b815eed0 2024-04-25T11:19:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.492+0000 7f57bda00640 1 -- 172.21.15.7:0/2833477131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b81e6fe0 con 0x7f57b815eed0 2024-04-25T11:19:26.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.494+0000 7f57bda00640 1 -- 172.21.15.7:0/2833477131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b81e79e0 con 0x7f57b815eed0 2024-04-25T11:19:26.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.495+0000 7f57afe00640 1 -- 172.21.15.7:0/2833477131 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f57b00037b0 con 0x7f57b815eed0 2024-04-25T11:19:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.496+0000 7f57afe00640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57940744b0 0x7f57940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.497+0000 7f57afe00640 1 -- 172.21.15.7:0/2833477131 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f57b00951d0 con 0x7f57b815eed0 2024-04-25T11:19:26.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.497+0000 7f57b7e00640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57940744b0 0x7f57940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:26.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.498+0000 7f57b7e00640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57940744b0 0x7f57940769c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f57a8006fd0 tx=0x7f57a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:26.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.502+0000 7f57afe00640 1 -- 172.21.15.7:0/2833477131 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f57b005f800 con 0x7f57b815eed0 2024-04-25T11:19:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.718+0000 7f57bda00640 1 -- 172.21.15.7:0/2833477131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f57b81e6a90 con 0x7f57b815eed0 2024-04-25T11:19:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.719+0000 7f57afe00640 1 -- 172.21.15.7:0/2833477131 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f57b005f100 con 0x7f57b815eed0 2024-04-25T11:19:26.717 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:19:26.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.722+0000 7f57bda00640 1 -- 172.21.15.7:0/2833477131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57940744b0 msgr2=0x7f57940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.722+0000 7f57bda00640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57940744b0 0x7f57940769c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f57a8006fd0 tx=0x7f57a8008040 comp rx=0 tx=0).stop 2024-04-25T11:19:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.722+0000 7f57bda00640 1 -- 172.21.15.7:0/2833477131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b815eed0 msgr2=0x7f57b8133b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.722+0000 7f57bda00640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b815eed0 0x7f57b8133b70 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f57b000c5f0 tx=0x7f57b000cab0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.722+0000 7f57bda00640 1 -- 172.21.15.7:0/2833477131 shutdown_connections 2024-04-25T11:19:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.722+0000 7f57bda00640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f57940744b0 0x7f57940769c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.722+0000 7f57bda00640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57b815eed0 0x7f57b8133b70 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.722+0000 7f57bda00640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57b8141290 0x7f57b81e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.722+0000 7f57bda00640 1 --2- 172.21.15.7:0/2833477131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57b806d610 0x7f57b81e64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.722+0000 7f57bda00640 1 -- 172.21.15.7:0/2833477131 >> 172.21.15.7:0/2833477131 conn(0x7f57b80d1540 msgr2=0x7f57b80cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.722+0000 7f57bda00640 1 -- 172.21.15.7:0/2833477131 shutdown_connections 2024-04-25T11:19:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:26.723+0000 7f57bda00640 1 -- 172.21.15.7:0/2833477131 wait complete. 2024-04-25T11:19:26.745 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:19:26.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:26.746 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:26.869 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:19:26.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:26.869 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:19:26.924 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:19:26.924 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:19:26.924 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:26.984 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:19:26.984 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:19:26.984 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:19:26.985 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:27.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:19:27.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:27.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.333+0000 7f7daaa00640 1 Processor -- start 2024-04-25T11:19:27.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.333+0000 7f7daaa00640 1 -- start start 2024-04-25T11:19:27.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.336+0000 7f7daaa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7da415de70 0x7f7da4164640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.336+0000 7f7daaa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7da40b0c20 0x7f7da41606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.336+0000 7f7daaa00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7da40aea00 con 0x7f7da40b6790 2024-04-25T11:19:27.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.336+0000 7f7daaa00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7da40cdb70 con 0x7f7da40b0c20 2024-04-25T11:19:27.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.336+0000 7f7daaa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7da40aed30 con 0x7f7da415de70 2024-04-25T11:19:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.336+0000 7f7da8a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7da40b0c20 0x7f7da41606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.336+0000 7f7da8a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7da40b0c20 0x7f7da41606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43400/0 (socket says 172.21.15.7:43400) 2024-04-25T11:19:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.336+0000 7f7da8a00640 1 -- 172.21.15.7:0/2737306018 learned_addr learned my addr 172.21.15.7:0/2737306018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.337+0000 7f7da3e00640 1 --2- 172.21.15.7:0/2737306018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7da415de70 0x7f7da4164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.337+0000 7f7da8a00640 1 -- 172.21.15.7:0/2737306018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7da415de70 msgr2=0x7f7da4164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.337+0000 7f7da8a00640 1 --2- 172.21.15.7:0/2737306018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7da415de70 0x7f7da4164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.337+0000 7f7da8a00640 1 -- 172.21.15.7:0/2737306018 >> v1:172.21.15.7:6789/0 conn(0x7f7da40b6790 legacy=0x7f7da40b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.337+0000 7f7da2a00640 1 -- 172.21.15.7:0/2737306018 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3628565726 0 0) 0x7f7da40aea00 con 0x7f7da40b6790 2024-04-25T11:19:27.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.337+0000 7f7da8a00640 1 -- 172.21.15.7:0/2737306018 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da40b13e0 con 0x7f7da40b0c20 2024-04-25T11:19:27.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.338+0000 7f7da8a00640 1 --2- 172.21.15.7:0/2737306018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7da40b0c20 0x7f7da41606a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f7da40cdcf0 tx=0x7f7d9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f285e01f9c66b282 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.338+0000 7f7da2a00640 1 -- 172.21.15.7:0/2737306018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c037070 con 0x7f7da40b0c20 2024-04-25T11:19:27.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.338+0000 7f7da2a00640 1 -- 172.21.15.7:0/2737306018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c032440 con 0x7f7da40b0c20 2024-04-25T11:19:27.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.338+0000 7f7da2a00640 1 -- 172.21.15.7:0/2737306018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c032810 con 0x7f7da40b0c20 2024-04-25T11:19:27.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.339+0000 7f7daaa00640 1 -- 172.21.15.7:0/2737306018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7da40b0c20 msgr2=0x7f7da41606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.339+0000 7f7daaa00640 1 --2- 172.21.15.7:0/2737306018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7da40b0c20 0x7f7da41606a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f7da40cdcf0 tx=0x7f7d9c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.339+0000 7f7daaa00640 1 -- 172.21.15.7:0/2737306018 shutdown_connections 2024-04-25T11:19:27.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.339+0000 7f7daaa00640 1 --2- 172.21.15.7:0/2737306018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7da40b0c20 0x7f7da41606a0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.339+0000 7f7daaa00640 1 --2- 172.21.15.7:0/2737306018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7da415de70 0x7f7da4164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.339+0000 7f7daaa00640 1 -- 172.21.15.7:0/2737306018 >> 172.21.15.7:0/2737306018 conn(0x7f7da4074200 msgr2=0x7f7da406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:27.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.339+0000 7f7daaa00640 1 -- 172.21.15.7:0/2737306018 shutdown_connections 2024-04-25T11:19:27.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.339+0000 7f7daaa00640 1 -- 172.21.15.7:0/2737306018 wait complete. 2024-04-25T11:19:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.342+0000 7f7daaa00640 1 Processor -- start 2024-04-25T11:19:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.342+0000 7f7daaa00640 1 -- start start 2024-04-25T11:19:27.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.343+0000 7f7daaa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7da40b0c20 0x7f7da414e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.343+0000 7f7daaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7da40b6790 0x7f7da4154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.343+0000 7f7daaa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7da415de70 0x7f7da4154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.343+0000 7f7daaa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7da406f300 con 0x7f7da40b6790 2024-04-25T11:19:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.343+0000 7f7daaa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7da40ce0a0 con 0x7f7da415de70 2024-04-25T11:19:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.343+0000 7f7daaa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7da406f480 con 0x7f7da40b0c20 2024-04-25T11:19:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.344+0000 7f7da3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7da40b6790 0x7f7da4154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.344+0000 7f7da3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7da40b6790 0x7f7da4154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43402/0 (socket says 172.21.15.7:43402) 2024-04-25T11:19:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.344+0000 7f7da3400640 1 -- 172.21.15.7:0/785194927 learned_addr learned my addr 172.21.15.7:0/785194927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.344+0000 7f7da3e00640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7da40b0c20 0x7f7da414e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.344+0000 7f7da3400640 1 -- 172.21.15.7:0/785194927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7da40b0c20 msgr2=0x7f7da414e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.344+0000 7f7da3400640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7da40b0c20 0x7f7da414e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.344+0000 7f7da8a00640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7da415de70 0x7f7da4154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.344+0000 7f7da3400640 1 -- 172.21.15.7:0/785194927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7da415de70 msgr2=0x7f7da4154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.344+0000 7f7da3400640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7da415de70 0x7f7da4154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.344+0000 7f7da3400640 1 -- 172.21.15.7:0/785194927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da41553d0 con 0x7f7da40b6790 2024-04-25T11:19:27.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.345+0000 7f7da3400640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7da40b6790 0x7f7da4154770 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c440 tx=0x7f7d9000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.345+0000 7f7d9be00640 1 -- 172.21.15.7:0/785194927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d90017070 con 0x7f7da40b6790 2024-04-25T11:19:27.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.345+0000 7f7d9be00640 1 -- 172.21.15.7:0/785194927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d900035f0 con 0x7f7da40b6790 2024-04-25T11:19:27.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.345+0000 7f7d9be00640 1 -- 172.21.15.7:0/785194927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d900039e0 con 0x7f7da40b6790 2024-04-25T11:19:27.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.345+0000 7f7daaa00640 1 -- 172.21.15.7:0/785194927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da41556b0 con 0x7f7da40b6790 2024-04-25T11:19:27.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.345+0000 7f7daaa00640 1 -- 172.21.15.7:0/785194927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da4153190 con 0x7f7da40b6790 2024-04-25T11:19:27.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.346+0000 7f7daaa00640 1 -- 172.21.15.7:0/785194927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da406a100 con 0x7f7da40b6790 2024-04-25T11:19:27.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.348+0000 7f7d9be00640 1 -- 172.21.15.7:0/785194927 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7d9001a070 con 0x7f7da40b6790 2024-04-25T11:19:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.349+0000 7f7d9be00640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d840744b0 0x7f7d840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.349+0000 7f7d9be00640 1 -- 172.21.15.7:0/785194927 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f7d900945e0 con 0x7f7da40b6790 2024-04-25T11:19:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.349+0000 7f7da3e00640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d840744b0 0x7f7d840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.350+0000 7f7da3e00640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d840744b0 0x7f7d840769c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7da406edc0 tx=0x7f7d9400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.353+0000 7f7d9be00640 1 -- 172.21.15.7:0/785194927 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7d9005e5b0 con 0x7f7da40b6790 2024-04-25T11:19:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.602+0000 7f7daaa00640 1 -- 172.21.15.7:0/785194927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7da4152f20 con 0x7f7da40b6790 2024-04-25T11:19:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.603+0000 7f7d9be00640 1 -- 172.21.15.7:0/785194927 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5171 (secure 0 0 0) 0x7f7d900628b0 con 0x7f7da40b6790 2024-04-25T11:19:27.601 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:27.602 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5846,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/4132210096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":4132210096},{"type":"v1","addr":"172.21.15.87:6834","nonce":4132210096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2657338269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2657338269},{"type":"v1","addr":"172.21.15.87:6835","nonce":2657338269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6457,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/138020048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":138020048},{"type":"v1","addr":"172.21.15.7:6836","nonce":138020048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:09.244634+0000","modified":"2024-04-25T11:19:12.318984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/990441706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":990441706},{"type":"v1","addr":"172.21.15.7:6837","nonce":990441706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7623,"qdb_cluster":[7623]},"id":7}]} 2024-04-25T11:19:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 67 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.606+0000 7f7daaa00640 1 -- 172.21.15.7:0/785194927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d840744b0 msgr2=0x7f7d840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.606+0000 7f7daaa00640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d840744b0 0x7f7d840769c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7da406edc0 tx=0x7f7d9400b040 comp rx=0 tx=0).stop 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.606+0000 7f7daaa00640 1 -- 172.21.15.7:0/785194927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7da40b6790 msgr2=0x7f7da4154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.606+0000 7f7daaa00640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7da40b6790 0x7f7da4154770 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c440 tx=0x7f7d9000c900 comp rx=0 tx=0).stop 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.606+0000 7f7daaa00640 1 -- 172.21.15.7:0/785194927 shutdown_connections 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.606+0000 7f7daaa00640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7d840744b0 0x7f7d840769c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.606+0000 7f7daaa00640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7da415de70 0x7f7da4154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.606+0000 7f7daaa00640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7da40b6790 0x7f7da4154770 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.606+0000 7f7daaa00640 1 --2- 172.21.15.7:0/785194927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7da40b0c20 0x7f7da414e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.606+0000 7f7daaa00640 1 -- 172.21.15.7:0/785194927 >> 172.21.15.7:0/785194927 conn(0x7f7da4074200 msgr2=0x7f7da406f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.606+0000 7f7daaa00640 1 -- 172.21.15.7:0/785194927 shutdown_connections 2024-04-25T11:19:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.607+0000 7f7daaa00640 1 -- 172.21.15.7:0/785194927 wait complete. 2024-04-25T11:19:27.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.910+0000 7f76e1e00640 1 Processor -- start 2024-04-25T11:19:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.910+0000 7f76e1e00640 1 -- start start 2024-04-25T11:19:27.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.912+0000 7f76e1e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76dc0aed40 0x7f76dc15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.912+0000 7f76e1e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76dc0b5220 0x7f76dc0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.912+0000 7f76e1e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76dc0af420 0x7f76dc0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.912+0000 7f76e1e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f76dc15e1f0 con 0x7f76dc0aed40 2024-04-25T11:19:27.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.912+0000 7f76e1e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f76dc0afd70 con 0x7f76dc0af420 2024-04-25T11:19:27.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.912+0000 7f76e1e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f76dc0b64b0 con 0x7f76dc0b5220 2024-04-25T11:19:27.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.912+0000 7f76e0a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76dc0aed40 0x7f76dc15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.912+0000 7f76e0a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f76dc0aed40 0x7f76dc15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43418/0 (socket says 172.21.15.7:43418) 2024-04-25T11:19:27.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.912+0000 7f76e0a00640 1 -- 172.21.15.7:0/447335018 learned_addr learned my addr 172.21.15.7:0/447335018 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:27.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.913+0000 7f76dbe00640 1 --2- 172.21.15.7:0/447335018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76dc0b5220 0x7f76dc0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.913+0000 7f76e0a00640 1 -- 172.21.15.7:0/447335018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76dc0b5220 msgr2=0x7f76dc0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:27.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.913+0000 7f76e0a00640 1 --2- 172.21.15.7:0/447335018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76dc0b5220 0x7f76dc0ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.913+0000 7f76e0a00640 1 -- 172.21.15.7:0/447335018 >> v2:172.21.15.87:3300/0 conn(0x7f76dc0af420 msgr2=0x7f76dc0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:27.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.913+0000 7f76e0a00640 1 --2- 172.21.15.7:0/447335018 >> v2:172.21.15.87:3300/0 conn(0x7f76dc0af420 0x7f76dc0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.913+0000 7f76e0a00640 1 -- 172.21.15.7:0/447335018 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc06c610 con 0x7f76dc0aed40 2024-04-25T11:19:27.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.914+0000 7f76e0a00640 1 --2- 172.21.15.7:0/447335018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc0aed40 0x7f76dc15dcb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f76dc073080 tx=0x7f76cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3102e43aa935e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.914+0000 7f76db400640 1 -- 172.21.15.7:0/447335018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc037070 con 0x7f76dc0aed40 2024-04-25T11:19:27.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.914+0000 7f76db400640 1 -- 172.21.15.7:0/447335018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc032440 con 0x7f76dc0aed40 2024-04-25T11:19:27.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.914+0000 7f76db400640 1 -- 172.21.15.7:0/447335018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc032810 con 0x7f76dc0aed40 2024-04-25T11:19:27.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.915+0000 7f76e1e00640 1 -- 172.21.15.7:0/447335018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc0aed40 msgr2=0x7f76dc15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.915+0000 7f76e1e00640 1 --2- 172.21.15.7:0/447335018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc0aed40 0x7f76dc15dcb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f76dc073080 tx=0x7f76cc02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.915+0000 7f76e1e00640 1 -- 172.21.15.7:0/447335018 shutdown_connections 2024-04-25T11:19:27.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.915+0000 7f76e1e00640 1 --2- 172.21.15.7:0/447335018 >> v2:172.21.15.87:3300/0 conn(0x7f76dc0af420 0x7f76dc0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.915+0000 7f76e1e00640 1 --2- 172.21.15.7:0/447335018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76dc0b5220 0x7f76dc0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.915+0000 7f76e1e00640 1 --2- 172.21.15.7:0/447335018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc0aed40 0x7f76dc15dcb0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.915+0000 7f76e1e00640 1 -- 172.21.15.7:0/447335018 >> 172.21.15.7:0/447335018 conn(0x7f76dc0b3820 msgr2=0x7f76dc0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:27.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.916+0000 7f76e1e00640 1 -- 172.21.15.7:0/447335018 shutdown_connections 2024-04-25T11:19:27.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.916+0000 7f76e1e00640 1 -- 172.21.15.7:0/447335018 wait complete. 2024-04-25T11:19:27.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.918+0000 7f76e1e00640 1 Processor -- start 2024-04-25T11:19:27.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.918+0000 7f76e1e00640 1 -- start start 2024-04-25T11:19:27.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.919+0000 7f76e1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc0aed40 0x7f76dc160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.919+0000 7f76e1e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76dc0af420 0x7f76dc0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76dc0b5220 0x7f76dc0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e1e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc06ceb0 con 0x7f76dc0aed40 2024-04-25T11:19:27.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e1e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f76dc06cbd0 con 0x7f76dc0b5220 2024-04-25T11:19:27.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e1e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f76dc0b6110 con 0x7f76dc0af420 2024-04-25T11:19:27.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc0aed40 0x7f76dc160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc0aed40 0x7f76dc160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43432/0 (socket says 172.21.15.7:43432) 2024-04-25T11:19:27.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e0a00640 1 -- 172.21.15.7:0/3970606570 learned_addr learned my addr 172.21.15.7:0/3970606570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:27.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e0a00640 1 -- 172.21.15.7:0/3970606570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76dc0af420 msgr2=0x7f76dc0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e0a00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76dc0af420 0x7f76dc0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e0a00640 1 -- 172.21.15.7:0/3970606570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76dc0b5220 msgr2=0x7f76dc0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:27.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e0a00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76dc0b5220 0x7f76dc0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:27.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e0a00640 1 -- 172.21.15.7:0/3970606570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76dc0cbe60 con 0x7f76dc0aed40 2024-04-25T11:19:27.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.920+0000 7f76e0a00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc0aed40 0x7f76dc160bf0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f76cc000c00 tx=0x7f76cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.921+0000 7f76d8c00640 1 -- 172.21.15.7:0/3970606570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc041020 con 0x7f76dc0aed40 2024-04-25T11:19:27.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.921+0000 7f76d8c00640 1 -- 172.21.15.7:0/3970606570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76cc030970 con 0x7f76dc0aed40 2024-04-25T11:19:27.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.921+0000 7f76e1e00640 1 -- 172.21.15.7:0/3970606570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76dc06f7f0 con 0x7f76dc0aed40 2024-04-25T11:19:27.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.921+0000 7f76d8c00640 1 -- 172.21.15.7:0/3970606570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc037040 con 0x7f76dc0aed40 2024-04-25T11:19:27.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.921+0000 7f76e1e00640 1 -- 172.21.15.7:0/3970606570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76dc1f73c0 con 0x7f76dc0aed40 2024-04-25T11:19:27.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.922+0000 7f76e1e00640 1 -- 172.21.15.7:0/3970606570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76dc050740 con 0x7f76dc0aed40 2024-04-25T11:19:27.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.923+0000 7f76d8c00640 1 -- 172.21.15.7:0/3970606570 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f76cc044020 con 0x7f76dc0aed40 2024-04-25T11:19:27.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.924+0000 7f76d8c00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76b80744e0 0x7f76b80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:27.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.924+0000 7f76d8c00640 1 -- 172.21.15.7:0/3970606570 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f76cc0b5ea0 con 0x7f76dc0aed40 2024-04-25T11:19:27.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.925+0000 7f76dbe00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76b80744e0 0x7f76b80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:27.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.925+0000 7f76dbe00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76b80744e0 0x7f76b80769f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f76dc156150 tx=0x7f76c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:27.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:27.929+0000 7f76d8c00640 1 -- 172.21.15.7:0/3970606570 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76cc04b030 con 0x7f76dc0aed40 2024-04-25T11:19:28.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.126+0000 7f76e1e00640 1 -- 172.21.15.7:0/3970606570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76dc15ac70 con 0x7f76dc0aed40 2024-04-25T11:19:28.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.129+0000 7f76d8c00640 1 -- 172.21.15.7:0/3970606570 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16788 (secure 0 0 0) 0x7f76cc080400 con 0x7f76dc0aed40 2024-04-25T11:19:28.127 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:28.128 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:10.691402+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:07.435764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:08.209216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:28.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.131+0000 7f76e1e00640 1 -- 172.21.15.7:0/3970606570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76b80744e0 msgr2=0x7f76b80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.131+0000 7f76e1e00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76b80744e0 0x7f76b80769f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f76dc156150 tx=0x7f76c4008040 comp rx=0 tx=0).stop 2024-04-25T11:19:28.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.131+0000 7f76e1e00640 1 -- 172.21.15.7:0/3970606570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc0aed40 msgr2=0x7f76dc160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.131+0000 7f76e1e00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc0aed40 0x7f76dc160bf0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f76cc000c00 tx=0x7f76cc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.132+0000 7f76e1e00640 1 -- 172.21.15.7:0/3970606570 shutdown_connections 2024-04-25T11:19:28.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.132+0000 7f76e1e00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76b80744e0 0x7f76b80769f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.132+0000 7f76e1e00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76dc0b5220 0x7f76dc0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.132+0000 7f76e1e00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76dc0af420 0x7f76dc0cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.132+0000 7f76e1e00640 1 --2- 172.21.15.7:0/3970606570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76dc0aed40 0x7f76dc160bf0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.132+0000 7f76e1e00640 1 -- 172.21.15.7:0/3970606570 >> 172.21.15.7:0/3970606570 conn(0x7f76dc0b3820 msgr2=0x7f76dc15d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:28.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.132+0000 7f76e1e00640 1 -- 172.21.15.7:0/3970606570 shutdown_connections 2024-04-25T11:19:28.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.132+0000 7f76e1e00640 1 -- 172.21.15.7:0/3970606570 wait complete. 2024-04-25T11:19:28.155 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:07.435764+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:08.209216+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:28.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.429+0000 7f4f71000640 1 Processor -- start 2024-04-25T11:19:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.429+0000 7f4f71000640 1 -- start start 2024-04-25T11:19:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.432+0000 7f4f71000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f6c1490f0 0x7f4f6c149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.432+0000 7f4f71000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4f6c0b01d0 0x7f4f6c0b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.432+0000 7f4f71000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f6c06f280 con 0x7f4f6c146180 2024-04-25T11:19:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.432+0000 7f4f71000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4f6c146590 con 0x7f4f6c1490f0 2024-04-25T11:19:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.432+0000 7f4f71000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6c06f560 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.432+0000 7f4f63e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f6c1490f0 0x7f4f6c149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.432+0000 7f4f63e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f6c1490f0 0x7f4f6c149500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43434/0 (socket says 172.21.15.7:43434) 2024-04-25T11:19:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.432+0000 7f4f63e00640 1 -- 172.21.15.7:0/2161951342 learned_addr learned my addr 172.21.15.7:0/2161951342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.433+0000 7f4f6be00640 1 --2- 172.21.15.7:0/2161951342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4f6c0b01d0 0x7f4f6c0b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.433+0000 7f4f63e00640 1 -- 172.21.15.7:0/2161951342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4f6c0b01d0 msgr2=0x7f4f6c0b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.433+0000 7f4f63e00640 1 --2- 172.21.15.7:0/2161951342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4f6c0b01d0 0x7f4f6c0b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.433+0000 7f4f63e00640 1 -- 172.21.15.7:0/2161951342 >> v1:172.21.15.87:6789/0 conn(0x7f4f6c146180 legacy=0x7f4f6c070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.433+0000 7f4f63e00640 1 -- 172.21.15.7:0/2161951342 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6c06f840 con 0x7f4f6c1490f0 2024-04-25T11:19:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.434+0000 7f4f63e00640 1 --2- 172.21.15.7:0/2161951342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f6c1490f0 0x7f4f6c149500 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009870 tx=0x7f4f5402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef7cc17e1150e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.434+0000 7f4f6aa00640 1 -- 172.21.15.7:0/2161951342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54037070 con 0x7f4f6c1490f0 2024-04-25T11:19:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.434+0000 7f4f6aa00640 1 -- 172.21.15.7:0/2161951342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f54032440 con 0x7f4f6c1490f0 2024-04-25T11:19:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.434+0000 7f4f6aa00640 1 -- 172.21.15.7:0/2161951342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54032810 con 0x7f4f6c1490f0 2024-04-25T11:19:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.435+0000 7f4f71000640 1 -- 172.21.15.7:0/2161951342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f6c1490f0 msgr2=0x7f4f6c149500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.435+0000 7f4f71000640 1 --2- 172.21.15.7:0/2161951342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f6c1490f0 0x7f4f6c149500 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009870 tx=0x7f4f5402a210 comp rx=0 tx=0).stop 2024-04-25T11:19:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.435+0000 7f4f71000640 1 -- 172.21.15.7:0/2161951342 shutdown_connections 2024-04-25T11:19:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.435+0000 7f4f71000640 1 --2- 172.21.15.7:0/2161951342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4f6c0b01d0 0x7f4f6c0b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.435+0000 7f4f71000640 1 --2- 172.21.15.7:0/2161951342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f6c1490f0 0x7f4f6c149500 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.435+0000 7f4f71000640 1 -- 172.21.15.7:0/2161951342 >> 172.21.15.7:0/2161951342 conn(0x7f4f6c0740f0 msgr2=0x7f4f6c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.435+0000 7f4f71000640 1 -- 172.21.15.7:0/2161951342 shutdown_connections 2024-04-25T11:19:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.435+0000 7f4f71000640 1 -- 172.21.15.7:0/2161951342 wait complete. 2024-04-25T11:19:28.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.438+0000 7f4f71000640 1 Processor -- start 2024-04-25T11:19:28.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.438+0000 7f4f71000640 1 -- start start 2024-04-25T11:19:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f71000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f6c0b01d0 0x7f4f6c0d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f71000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4f6c146180 0x7f4f6c15f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f71000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f6c1490f0 0x7f4f6c0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f71000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6c0b0b20 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f71000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6c06fe00 con 0x7f4f6c1490f0 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f71000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6c06eea0 con 0x7f4f6c146180 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f6b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f6c0b01d0 0x7f4f6c0d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f6b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f6c0b01d0 0x7f4f6c0d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43450/0 (socket says 172.21.15.7:43450) 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f6b400640 1 -- 172.21.15.7:0/2164265469 learned_addr learned my addr 172.21.15.7:0/2164265469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f63e00640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4f6c146180 0x7f4f6c15f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f6be00640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f6c1490f0 0x7f4f6c0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f6b400640 1 -- 172.21.15.7:0/2164265469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4f6c146180 msgr2=0x7f4f6c15f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f6b400640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4f6c146180 0x7f4f6c15f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f6b400640 1 -- 172.21.15.7:0/2164265469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f6c1490f0 msgr2=0x7f4f6c0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f6b400640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f6c1490f0 0x7f4f6c0cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.441+0000 7f4f6b400640 1 -- 172.21.15.7:0/2164265469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6c1417a0 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.442+0000 7f4f6b400640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f6c0b01d0 0x7f4f6c0d0a30 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00b3f0 tx=0x7f4f5c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:28.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.442+0000 7f4f63400640 1 -- 172.21.15.7:0/2164265469 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c015020 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.442+0000 7f4f63400640 1 -- 172.21.15.7:0/2164265469 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5c011830 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.442+0000 7f4f63400640 1 -- 172.21.15.7:0/2164265469 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c011c20 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.442+0000 7f4f71000640 1 -- 172.21.15.7:0/2164265469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f6c141a80 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.442+0000 7f4f71000640 1 -- 172.21.15.7:0/2164265469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f6c142040 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.444+0000 7f4f71000640 1 -- 172.21.15.7:0/2164265469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f6c050740 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.445+0000 7f4f63400640 1 -- 172.21.15.7:0/2164265469 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4f5c018020 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.446+0000 7f4f63400640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4f480744e0 0x7f4f480769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.446+0000 7f4f63400640 1 -- 172.21.15.7:0/2164265469 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f4f5c094ff0 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.447+0000 7f4f63e00640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4f480744e0 0x7f4f480769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.447+0000 7f4f63e00640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4f480744e0 0x7f4f480769f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4f54034000 tx=0x7f4f5402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.451+0000 7f4f63400640 1 -- 172.21.15.7:0/2164265469 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4f5c01f030 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.645+0000 7f4f71000640 1 -- 172.21.15.7:0/2164265469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4f6c06d380 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.646+0000 7f4f63400640 1 -- 172.21.15.7:0/2164265469 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) ==== 93+0+10 (secure 0 0 0) 0x7f4f5c05f640 con 0x7f4f6c0b01d0 2024-04-25T11:19:28.645 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:19:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.648+0000 7f4f71000640 1 -- 172.21.15.7:0/2164265469 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4f480744e0 msgr2=0x7f4f480769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.648+0000 7f4f71000640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4f480744e0 0x7f4f480769f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4f54034000 tx=0x7f4f5402e000 comp rx=0 tx=0).stop 2024-04-25T11:19:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.649+0000 7f4f71000640 1 -- 172.21.15.7:0/2164265469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f6c0b01d0 msgr2=0x7f4f6c0d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.649+0000 7f4f71000640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f6c0b01d0 0x7f4f6c0d0a30 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00b3f0 tx=0x7f4f5c00b8b0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.649+0000 7f4f71000640 1 -- 172.21.15.7:0/2164265469 shutdown_connections 2024-04-25T11:19:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.649+0000 7f4f71000640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4f480744e0 0x7f4f480769f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.649+0000 7f4f71000640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f6c1490f0 0x7f4f6c0cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.649+0000 7f4f71000640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4f6c146180 0x7f4f6c15f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.649+0000 7f4f71000640 1 --2- 172.21.15.7:0/2164265469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f6c0b01d0 0x7f4f6c0d0a30 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.649+0000 7f4f71000640 1 -- 172.21.15.7:0/2164265469 >> 172.21.15.7:0/2164265469 conn(0x7f4f6c0740f0 msgr2=0x7f4f6c0b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.649+0000 7f4f71000640 1 -- 172.21.15.7:0/2164265469 shutdown_connections 2024-04-25T11:19:28.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.649+0000 7f4f71000640 1 -- 172.21.15.7:0/2164265469 wait complete. 2024-04-25T11:19:28.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:19:28.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.946+0000 7fddca400640 1 Processor -- start 2024-04-25T11:19:28.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.947+0000 7fddca400640 1 -- start start 2024-04-25T11:19:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.949+0000 7fddca400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fddc406de10 0x7fddc40b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.949+0000 7fddca400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fddc40b7a10 0x7fddc40d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.949+0000 7fddca400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddc40b7e20 con 0x7fddc40d1cd0 2024-04-25T11:19:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.949+0000 7fddca400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fddc41603d0 con 0x7fddc40b7a10 2024-04-25T11:19:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.949+0000 7fddca400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fddc406d090 con 0x7fddc406de10 2024-04-25T11:19:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.949+0000 7fddc3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fddc40b7a10 0x7fddc40d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.949+0000 7fddc3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fddc40b7a10 0x7fddc40d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43456/0 (socket says 172.21.15.7:43456) 2024-04-25T11:19:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.949+0000 7fddc3e00640 1 -- 172.21.15.7:0/994656621 learned_addr learned my addr 172.21.15.7:0/994656621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.950+0000 7fddc9000640 1 --2- 172.21.15.7:0/994656621 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fddc406de10 0x7fddc40b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.950+0000 7fddc3e00640 1 -- 172.21.15.7:0/994656621 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fddc406de10 msgr2=0x7fddc40b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:28.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.950+0000 7fddc3e00640 1 --2- 172.21.15.7:0/994656621 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fddc406de10 0x7fddc40b74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.950+0000 7fddc3e00640 1 -- 172.21.15.7:0/994656621 >> v1:172.21.15.87:6789/0 conn(0x7fddc40d1cd0 legacy=0x7fddc40b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:28.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.950+0000 7fddc3e00640 1 -- 172.21.15.7:0/994656621 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc40d3490 con 0x7fddc40b7a10 2024-04-25T11:19:28.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.951+0000 7fddc3e00640 1 --2- 172.21.15.7:0/994656621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc40b7a10 0x7fddc40d1790 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009870 tx=0x7fddb002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac27a09a0aa6fef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:28.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.951+0000 7fddc3400640 1 -- 172.21.15.7:0/994656621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0037070 con 0x7fddc40b7a10 2024-04-25T11:19:28.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.951+0000 7fddc3400640 1 -- 172.21.15.7:0/994656621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb0032440 con 0x7fddc40b7a10 2024-04-25T11:19:28.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.951+0000 7fddc3400640 1 -- 172.21.15.7:0/994656621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0032810 con 0x7fddc40b7a10 2024-04-25T11:19:28.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.952+0000 7fddca400640 1 -- 172.21.15.7:0/994656621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc40b7a10 msgr2=0x7fddc40d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.952+0000 7fddca400640 1 --2- 172.21.15.7:0/994656621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc40b7a10 0x7fddc40d1790 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009870 tx=0x7fddb002a210 comp rx=0 tx=0).stop 2024-04-25T11:19:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.952+0000 7fddca400640 1 -- 172.21.15.7:0/994656621 shutdown_connections 2024-04-25T11:19:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.952+0000 7fddca400640 1 --2- 172.21.15.7:0/994656621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc40b7a10 0x7fddc40d1790 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.952+0000 7fddca400640 1 --2- 172.21.15.7:0/994656621 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fddc406de10 0x7fddc40b74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.952+0000 7fddca400640 1 -- 172.21.15.7:0/994656621 >> 172.21.15.7:0/994656621 conn(0x7fddc409b540 msgr2=0x7fddc409b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.952+0000 7fddca400640 1 -- 172.21.15.7:0/994656621 shutdown_connections 2024-04-25T11:19:28.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.952+0000 7fddca400640 1 -- 172.21.15.7:0/994656621 wait complete. 2024-04-25T11:19:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.955+0000 7fddca400640 1 Processor -- start 2024-04-25T11:19:28.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.955+0000 7fddca400640 1 -- start start 2024-04-25T11:19:28.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.956+0000 7fddca400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fddc406de10 0x7fddc41556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.956+0000 7fddca400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddc40b7a10 0x7fddc40b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.956+0000 7fddca400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc40d1cd0 0x7fddc40b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.956+0000 7fddca400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fddc406d8f0 con 0x7fddc40d1cd0 2024-04-25T11:19:28.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.956+0000 7fddca400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fddc406d610 con 0x7fddc40b7a10 2024-04-25T11:19:28.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.956+0000 7fddca400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fddc40d20e0 con 0x7fddc406de10 2024-04-25T11:19:28.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.957+0000 7fddc9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc40d1cd0 0x7fddc40b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.957+0000 7fddc9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc40d1cd0 0x7fddc40b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43472/0 (socket says 172.21.15.7:43472) 2024-04-25T11:19:28.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.957+0000 7fddc9a00640 1 -- 172.21.15.7:0/156588947 learned_addr learned my addr 172.21.15.7:0/156588947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:28.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.957+0000 7fddc3e00640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddc40b7a10 0x7fddc40b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.957+0000 7fddc9000640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fddc406de10 0x7fddc41556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.957+0000 7fddc9a00640 1 -- 172.21.15.7:0/156588947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fddc406de10 msgr2=0x7fddc41556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.957+0000 7fddc9a00640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fddc406de10 0x7fddc41556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.957+0000 7fddc9a00640 1 -- 172.21.15.7:0/156588947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddc40b7a10 msgr2=0x7fddc40b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:28.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.957+0000 7fddc9a00640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddc40b7a10 0x7fddc40b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:28.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.957+0000 7fddc9a00640 1 -- 172.21.15.7:0/156588947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc4161610 con 0x7fddc40d1cd0 2024-04-25T11:19:28.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.958+0000 7fddc9a00640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc40d1cd0 0x7fddc40b4930 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00b9e0 tx=0x7fddbc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:28.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.958+0000 7fddc0c00640 1 -- 172.21.15.7:0/156588947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc003b70 con 0x7fddc40d1cd0 2024-04-25T11:19:28.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.958+0000 7fddc0c00640 1 -- 172.21.15.7:0/156588947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddbc003d10 con 0x7fddc40d1cd0 2024-04-25T11:19:28.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.958+0000 7fddc0c00640 1 -- 172.21.15.7:0/156588947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00caa0 con 0x7fddc40d1cd0 2024-04-25T11:19:28.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.958+0000 7fddca400640 1 -- 172.21.15.7:0/156588947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc41618f0 con 0x7fddc40d1cd0 2024-04-25T11:19:28.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.958+0000 7fddca400640 1 -- 172.21.15.7:0/156588947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc4161eb0 con 0x7fddc40d1cd0 2024-04-25T11:19:28.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.959+0000 7fddca400640 1 -- 172.21.15.7:0/156588947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc4050740 con 0x7fddc40d1cd0 2024-04-25T11:19:28.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.960+0000 7fddc0c00640 1 -- 172.21.15.7:0/156588947 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fddbc00cc40 con 0x7fddc40d1cd0 2024-04-25T11:19:28.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.961+0000 7fddc0c00640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdda40744e0 0x7fdda40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:28.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.961+0000 7fddc0c00640 1 -- 172.21.15.7:0/156588947 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fddbc0951a0 con 0x7fddc40d1cd0 2024-04-25T11:19:28.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.962+0000 7fddc9000640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdda40744e0 0x7fdda40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:28.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.963+0000 7fddc9000640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdda40744e0 0x7fdda40769f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fddb8006fd0 tx=0x7fddb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:28.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:28.966+0000 7fddc0c00640 1 -- 172.21.15.7:0/156588947 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fddbc05f700 con 0x7fddc40d1cd0 2024-04-25T11:19:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.171+0000 7fddca400640 1 -- 172.21.15.7:0/156588947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fddc406e4c0 con 0x7fddc40d1cd0 2024-04-25T11:19:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.172+0000 7fddc0c00640 1 -- 172.21.15.7:0/156588947 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) ==== 104+0+11 (secure 0 0 0) 0x7fddbc05f000 con 0x7fddc40d1cd0 2024-04-25T11:19:29.170 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:19:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 -- 172.21.15.7:0/156588947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdda40744e0 msgr2=0x7fdda40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdda40744e0 0x7fdda40769f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fddb8006fd0 tx=0x7fddb8008040 comp rx=0 tx=0).stop 2024-04-25T11:19:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 -- 172.21.15.7:0/156588947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc40d1cd0 msgr2=0x7fddc40b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc40d1cd0 0x7fddc40b4930 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00b9e0 tx=0x7fddbc00bea0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 -- 172.21.15.7:0/156588947 shutdown_connections 2024-04-25T11:19:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdda40744e0 0x7fdda40769f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc40d1cd0 0x7fddc40b4930 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fddc40b7a10 0x7fddc40b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 --2- 172.21.15.7:0/156588947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fddc406de10 0x7fddc41556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 -- 172.21.15.7:0/156588947 >> 172.21.15.7:0/156588947 conn(0x7fddc409b540 msgr2=0x7fddc40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 -- 172.21.15.7:0/156588947 shutdown_connections 2024-04-25T11:19:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.175+0000 7fddca400640 1 -- 172.21.15.7:0/156588947 wait complete. 2024-04-25T11:19:29.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:19:29.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.480+0000 7f5c10600640 1 Processor -- start 2024-04-25T11:19:29.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.480+0000 7f5c10600640 1 -- start start 2024-04-25T11:19:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.482+0000 7f5c10600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5c0815dfb0 0x7f5c080b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.482+0000 7f5c10600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5c080d0b40 0x7f5c0806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.483+0000 7f5c10600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c0815e3c0 con 0x7f5c080d0f50 2024-04-25T11:19:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.483+0000 7f5c10600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5c080b09b0 con 0x7f5c0815dfb0 2024-04-25T11:19:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.483+0000 7f5c10600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5c080cf800 con 0x7f5c080d0b40 2024-04-25T11:19:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.483+0000 7f5c0dc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5c0815dfb0 0x7f5c080b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.483+0000 7f5c0dc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5c0815dfb0 0x7f5c080b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43478/0 (socket says 172.21.15.7:43478) 2024-04-25T11:19:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.483+0000 7f5c0dc00640 1 -- 172.21.15.7:0/3650806523 learned_addr learned my addr 172.21.15.7:0/3650806523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:29.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.483+0000 7f5c0d200640 1 --2- 172.21.15.7:0/3650806523 >> v2:172.21.15.87:3300/0 conn(0x7f5c080d0b40 0x7f5c0806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.484+0000 7f5c0dc00640 1 -- 172.21.15.7:0/3650806523 >> v2:172.21.15.87:3300/0 conn(0x7f5c080d0b40 msgr2=0x7f5c0806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:29.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.484+0000 7f5c0dc00640 1 --2- 172.21.15.7:0/3650806523 >> v2:172.21.15.87:3300/0 conn(0x7f5c080d0b40 0x7f5c0806f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.484+0000 7f5c0dc00640 1 -- 172.21.15.7:0/3650806523 >> v1:172.21.15.87:6789/0 conn(0x7f5c080d0f50 legacy=0x7f5c08161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.484+0000 7f5c0dc00640 1 -- 172.21.15.7:0/3650806523 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c080cfaa0 con 0x7f5c0815dfb0 2024-04-25T11:19:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.484+0000 7f5c0d200640 1 --2- 172.21.15.7:0/3650806523 >> v2:172.21.15.87:3300/0 conn(0x7f5c080d0b40 0x7f5c0806f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.485+0000 7f5c0dc00640 1 --2- 172.21.15.7:0/3650806523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0815dfb0 0x7f5c080b0470 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f5c080b4670 tx=0x7f5bfc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254d16813ad041b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.485+0000 7f5c07e00640 1 -- 172.21.15.7:0/3650806523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc036070 con 0x7f5c0815dfb0 2024-04-25T11:19:29.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.485+0000 7f5c07e00640 1 -- 172.21.15.7:0/3650806523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bfc031600 con 0x7f5c0815dfb0 2024-04-25T11:19:29.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.485+0000 7f5c07e00640 1 -- 172.21.15.7:0/3650806523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0319d0 con 0x7f5c0815dfb0 2024-04-25T11:19:29.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.486+0000 7f5c10600640 1 -- 172.21.15.7:0/3650806523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0815dfb0 msgr2=0x7f5c080b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.486+0000 7f5c10600640 1 --2- 172.21.15.7:0/3650806523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0815dfb0 0x7f5c080b0470 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f5c080b4670 tx=0x7f5bfc02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.486+0000 7f5c10600640 1 -- 172.21.15.7:0/3650806523 shutdown_connections 2024-04-25T11:19:29.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.486+0000 7f5c10600640 1 --2- 172.21.15.7:0/3650806523 >> v2:172.21.15.87:3300/0 conn(0x7f5c080d0b40 0x7f5c0806f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.486+0000 7f5c10600640 1 --2- 172.21.15.7:0/3650806523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0815dfb0 0x7f5c080b0470 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.486+0000 7f5c10600640 1 -- 172.21.15.7:0/3650806523 >> 172.21.15.7:0/3650806523 conn(0x7f5c080742c0 msgr2=0x7f5c0806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:29.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.486+0000 7f5c10600640 1 -- 172.21.15.7:0/3650806523 shutdown_connections 2024-04-25T11:19:29.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.487+0000 7f5c10600640 1 -- 172.21.15.7:0/3650806523 wait complete. 2024-04-25T11:19:29.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.489+0000 7f5c10600640 1 Processor -- start 2024-04-25T11:19:29.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.489+0000 7f5c10600640 1 -- start start 2024-04-25T11:19:29.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c10600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c080d0b40 0x7f5c081593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:29.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c10600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c080d0f50 0x7f5c081531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:29.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c10600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c0815dfb0 0x7f5c0815eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:29.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c10600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c080d0340 con 0x7f5c080d0f50 2024-04-25T11:19:29.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c10600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5c080d0060 con 0x7f5c0815dfb0 2024-04-25T11:19:29.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c10600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c080d0620 con 0x7f5c080d0b40 2024-04-25T11:19:29.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c0d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c080d0f50 0x7f5c081531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:29.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c0d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c080d0f50 0x7f5c081531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43486/0 (socket says 172.21.15.7:43486) 2024-04-25T11:19:29.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c0d200640 1 -- 172.21.15.7:0/1608603022 learned_addr learned my addr 172.21.15.7:0/1608603022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:29.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c0d200640 1 -- 172.21.15.7:0/1608603022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c080d0b40 msgr2=0x7f5c081593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c0d200640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c080d0b40 0x7f5c081593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c0d200640 1 -- 172.21.15.7:0/1608603022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c0815dfb0 msgr2=0x7f5c0815eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.491+0000 7f5c0e600640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c0815dfb0 0x7f5c0815eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:29.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.492+0000 7f5c0d200640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c0815dfb0 0x7f5c0815eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.492+0000 7f5c0d200640 1 -- 172.21.15.7:0/1608603022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0815bcc0 con 0x7f5c080d0f50 2024-04-25T11:19:29.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.492+0000 7f5c0e600640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c0815dfb0 0x7f5c0815eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:29.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.492+0000 7f5c0d200640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c080d0f50 0x7f5c081531a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400d7d0 tx=0x7f5bf400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:29.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.492+0000 7f5c05600640 1 -- 172.21.15.7:0/1608603022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4004030 con 0x7f5c080d0f50 2024-04-25T11:19:29.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.492+0000 7f5c05600640 1 -- 172.21.15.7:0/1608603022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf40041d0 con 0x7f5c080d0f50 2024-04-25T11:19:29.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.492+0000 7f5c10600640 1 -- 172.21.15.7:0/1608603022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0815bfa0 con 0x7f5c080d0f50 2024-04-25T11:19:29.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.492+0000 7f5c05600640 1 -- 172.21.15.7:0/1608603022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf40096a0 con 0x7f5c080d0f50 2024-04-25T11:19:29.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.492+0000 7f5c10600640 1 -- 172.21.15.7:0/1608603022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0815c560 con 0x7f5c080d0f50 2024-04-25T11:19:29.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.494+0000 7f5c10600640 1 -- 172.21.15.7:0/1608603022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c0815ba30 con 0x7f5c080d0f50 2024-04-25T11:19:29.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.495+0000 7f5c05600640 1 -- 172.21.15.7:0/1608603022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5bf4009840 con 0x7f5c080d0f50 2024-04-25T11:19:29.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.496+0000 7f5c05600640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be80744b0 0x7f5be80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:29.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.496+0000 7f5c05600640 1 -- 172.21.15.7:0/1608603022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f5bf4095690 con 0x7f5c080d0f50 2024-04-25T11:19:29.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.496+0000 7f5c0dc00640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be80744b0 0x7f5be80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:29.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.497+0000 7f5c0dc00640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be80744b0 0x7f5be80769c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc033000 tx=0x7f5bfc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:29.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.501+0000 7f5c05600640 1 -- 172.21.15.7:0/1608603022 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5bf405fbf0 con 0x7f5c080d0f50 2024-04-25T11:19:29.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.704+0000 7f5c10600640 1 -- 172.21.15.7:0/1608603022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5c08050740 con 0x7f5c080d0f50 2024-04-25T11:19:29.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.705+0000 7f5c05600640 1 -- 172.21.15.7:0/1608603022 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) ==== 100+0+11 (secure 0 0 0) 0x7f5bf405f4f0 con 0x7f5c080d0f50 2024-04-25T11:19:29.703 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:19:29.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 -- 172.21.15.7:0/1608603022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be80744b0 msgr2=0x7f5be80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be80744b0 0x7f5be80769c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc033000 tx=0x7f5bfc02e000 comp rx=0 tx=0).stop 2024-04-25T11:19:29.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 -- 172.21.15.7:0/1608603022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c080d0f50 msgr2=0x7f5c081531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:29.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c080d0f50 0x7f5c081531a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400d7d0 tx=0x7f5bf400dc90 comp rx=0 tx=0).stop 2024-04-25T11:19:29.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 -- 172.21.15.7:0/1608603022 shutdown_connections 2024-04-25T11:19:29.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be80744b0 0x7f5be80769c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c0815dfb0 0x7f5c0815eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c080d0f50 0x7f5c081531a0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 --2- 172.21.15.7:0/1608603022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c080d0b40 0x7f5c081593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:29.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 -- 172.21.15.7:0/1608603022 >> 172.21.15.7:0/1608603022 conn(0x7f5c080742c0 msgr2=0x7f5c08070080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:29.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 -- 172.21.15.7:0/1608603022 shutdown_connections 2024-04-25T11:19:29.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:29.708+0000 7f5c10600640 1 -- 172.21.15.7:0/1608603022 wait complete. 2024-04-25T11:19:29.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:30.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.014+0000 7fb3be600640 1 Processor -- start 2024-04-25T11:19:30.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.015+0000 7fb3be600640 1 -- start start 2024-04-25T11:19:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.017+0000 7fb3be600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb3b80afb30 0x7fb3b815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.017+0000 7fb3be600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b80cebb0 con 0x7fb3b80aed40 2024-04-25T11:19:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.017+0000 7fb3be600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b80b64b0 con 0x7fb3b80b5220 2024-04-25T11:19:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.017+0000 7fb3be600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b815e1f0 con 0x7fb3b80afb30 2024-04-25T11:19:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.017+0000 7fb3bdc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb3b80afb30 0x7fb3b815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.018+0000 7fb3bd200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb3b80aed40 0x7fb3b80ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50530/0 (socket says 172.21.15.7:50530) 2024-04-25T11:19:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.018+0000 7fb3bd200640 1 -- 172.21.15.7:0/275381561 learned_addr learned my addr 172.21.15.7:0/275381561 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.018+0000 7fb3bdc00640 1 -- 172.21.15.7:0/275381561 >> v1:172.21.15.87:6789/0 conn(0x7fb3b80b5220 legacy=0x7fb3b80af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.018+0000 7fb3bdc00640 1 -- 172.21.15.7:0/275381561 >> v1:172.21.15.7:6789/0 conn(0x7fb3b80aed40 legacy=0x7fb3b80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.018+0000 7fb3bdc00640 1 -- 172.21.15.7:0/275381561 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b806c610 con 0x7fb3b80afb30 2024-04-25T11:19:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.018+0000 7fb3b7400640 1 -- 172.21.15.7:0/275381561 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3458078222 0 0) 0x7fb3ac001110 con 0x7fb3b80aed40 2024-04-25T11:19:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.019+0000 7fb3bdc00640 1 --2- 172.21.15.7:0/275381561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b80afb30 0x7fb3b815dcb0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009d90 tx=0x7fb3b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3da37e087ed4fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.019+0000 7fb3b7400640 1 -- 172.21.15.7:0/275381561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0007740 con 0x7fb3b80afb30 2024-04-25T11:19:30.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.019+0000 7fb3b7400640 1 -- 172.21.15.7:0/275381561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b00078e0 con 0x7fb3b80afb30 2024-04-25T11:19:30.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.019+0000 7fb3b7400640 1 -- 172.21.15.7:0/275381561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0007cb0 con 0x7fb3b80afb30 2024-04-25T11:19:30.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.020+0000 7fb3be600640 1 -- 172.21.15.7:0/275381561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b80afb30 msgr2=0x7fb3b815dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.020+0000 7fb3be600640 1 --2- 172.21.15.7:0/275381561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b80afb30 0x7fb3b815dcb0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009d90 tx=0x7fb3b002a6a0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.020+0000 7fb3be600640 1 -- 172.21.15.7:0/275381561 shutdown_connections 2024-04-25T11:19:30.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.020+0000 7fb3be600640 1 --2- 172.21.15.7:0/275381561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b80afb30 0x7fb3b815dcb0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.020+0000 7fb3be600640 1 -- 172.21.15.7:0/275381561 >> 172.21.15.7:0/275381561 conn(0x7fb3b80b3820 msgr2=0x7fb3b80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:30.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.020+0000 7fb3be600640 1 -- 172.21.15.7:0/275381561 shutdown_connections 2024-04-25T11:19:30.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.020+0000 7fb3be600640 1 -- 172.21.15.7:0/275381561 wait complete. 2024-04-25T11:19:30.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.023+0000 7fb3be600640 1 Processor -- start 2024-04-25T11:19:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.023+0000 7fb3be600640 1 -- start start 2024-04-25T11:19:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.024+0000 7fb3be600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b80aed40 0x7fb3b8159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.024+0000 7fb3be600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3b80b5220 0x7fb3b81501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.024+0000 7fb3be600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b80b1380 0x7fb3b80b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.024+0000 7fb3be600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b806ceb0 con 0x7fb3b80b1380 2024-04-25T11:19:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.024+0000 7fb3be600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b806cbd0 con 0x7fb3b80aed40 2024-04-25T11:19:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.024+0000 7fb3be600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b80b6110 con 0x7fb3b80b5220 2024-04-25T11:19:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3bdc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b80b1380 0x7fb3b80b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3bdc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b80b1380 0x7fb3b80b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47088/0 (socket says 172.21.15.7:47088) 2024-04-25T11:19:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3bdc00640 1 -- 172.21.15.7:0/1845032551 learned_addr learned my addr 172.21.15.7:0/1845032551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3bd200640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b80aed40 0x7fb3b8159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3b7e00640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3b80b5220 0x7fb3b81501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3bdc00640 1 -- 172.21.15.7:0/1845032551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3b80b5220 msgr2=0x7fb3b81501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3bdc00640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3b80b5220 0x7fb3b81501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3bdc00640 1 -- 172.21.15.7:0/1845032551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b80aed40 msgr2=0x7fb3b8159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3bdc00640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b80aed40 0x7fb3b8159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3bdc00640 1 -- 172.21.15.7:0/1845032551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b80b1cd0 con 0x7fb3b80b1380 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3b7e00640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3b80b5220 0x7fb3b81501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.025+0000 7fb3bdc00640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b80b1380 0x7fb3b80b1790 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009eb0 tx=0x7fb3b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.026+0000 7fb3b4c00640 1 -- 172.21.15.7:0/1845032551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0040070 con 0x7fb3b80b1380 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.026+0000 7fb3b4c00640 1 -- 172.21.15.7:0/1845032551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3b0031c90 con 0x7fb3b80b1380 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.026+0000 7fb3b4c00640 1 -- 172.21.15.7:0/1845032551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0045680 con 0x7fb3b80b1380 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.026+0000 7fb3be600640 1 -- 172.21.15.7:0/1845032551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b81f91c0 con 0x7fb3b80b1380 2024-04-25T11:19:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.026+0000 7fb3be600640 1 -- 172.21.15.7:0/1845032551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b81f9740 con 0x7fb3b80b1380 2024-04-25T11:19:30.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.027+0000 7fb3be600640 1 -- 172.21.15.7:0/1845032551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b815a3c0 con 0x7fb3b80b1380 2024-04-25T11:19:30.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.028+0000 7fb3b4c00640 1 -- 172.21.15.7:0/1845032551 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb3b0039040 con 0x7fb3b80b1380 2024-04-25T11:19:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.029+0000 7fb3b4c00640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3980744b0 0x7fb3980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.029+0000 7fb3b4c00640 1 -- 172.21.15.7:0/1845032551 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fb3b00b6e30 con 0x7fb3b80b1380 2024-04-25T11:19:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.029+0000 7fb3bd200640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3980744b0 0x7fb3980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.030+0000 7fb3bd200640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3980744b0 0x7fb3980769c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac000f60 tx=0x7fb3ac007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.034+0000 7fb3b4c00640 1 -- 172.21.15.7:0/1845032551 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb3b80b1cd0 con 0x7fb3b80b1380 2024-04-25T11:19:30.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.274+0000 7fb3be600640 1 -- 172.21.15.7:0/1845032551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3b81578e0 con 0x7fb3b80b1380 2024-04-25T11:19:30.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.276+0000 7fb3b4c00640 1 -- 172.21.15.7:0/1845032551 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5171 (secure 0 0 0) 0x7fb3b0081450 con 0x7fb3b80b1380 2024-04-25T11:19:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:30.274 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5846,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/4132210096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":4132210096},{"type":"v1","addr":"172.21.15.87:6834","nonce":4132210096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2657338269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2657338269},{"type":"v1","addr":"172.21.15.87:6835","nonce":2657338269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6457,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/138020048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":138020048},{"type":"v1","addr":"172.21.15.7:6836","nonce":138020048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:09.244634+0000","modified":"2024-04-25T11:19:12.318984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/990441706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":990441706},{"type":"v1","addr":"172.21.15.7:6837","nonce":990441706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7623,"qdb_cluster":[7623]},"id":7}]} 2024-04-25T11:19:30.274 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 67 2024-04-25T11:19:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.278+0000 7fb3be600640 1 -- 172.21.15.7:0/1845032551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3980744b0 msgr2=0x7fb3980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.278+0000 7fb3be600640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3980744b0 0x7fb3980769c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac000f60 tx=0x7fb3ac007c00 comp rx=0 tx=0).stop 2024-04-25T11:19:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.278+0000 7fb3be600640 1 -- 172.21.15.7:0/1845032551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b80b1380 msgr2=0x7fb3b80b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.278+0000 7fb3be600640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b80b1380 0x7fb3b80b1790 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009eb0 tx=0x7fb3b0004a40 comp rx=0 tx=0).stop 2024-04-25T11:19:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.279+0000 7fb3be600640 1 -- 172.21.15.7:0/1845032551 shutdown_connections 2024-04-25T11:19:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.279+0000 7fb3be600640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb3980744b0 0x7fb3980769c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.279+0000 7fb3be600640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b80b1380 0x7fb3b80b1790 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.279+0000 7fb3be600640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3b80b5220 0x7fb3b81501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.279+0000 7fb3be600640 1 --2- 172.21.15.7:0/1845032551 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3b80aed40 0x7fb3b8159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.279+0000 7fb3be600640 1 -- 172.21.15.7:0/1845032551 >> 172.21.15.7:0/1845032551 conn(0x7fb3b80b3820 msgr2=0x7fb3b80af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:30.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.279+0000 7fb3be600640 1 -- 172.21.15.7:0/1845032551 shutdown_connections 2024-04-25T11:19:30.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.279+0000 7fb3be600640 1 -- 172.21.15.7:0/1845032551 wait complete. 2024-04-25T11:19:30.307 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.592+0000 7f2aff800640 1 Processor -- start 2024-04-25T11:19:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.592+0000 7f2aff800640 1 -- start start 2024-04-25T11:19:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.594+0000 7f2aff800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2af806f300 0x7f2af80b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.594+0000 7f2aff800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af80b4980 con 0x7f2af80b0290 2024-04-25T11:19:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.594+0000 7f2aff800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2af80b4c60 con 0x7f2af806dc90 2024-04-25T11:19:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.594+0000 7f2aff800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2af80b4f40 con 0x7f2af806f300 2024-04-25T11:19:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.595+0000 7f2af7e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2af80b0290 0x7f2af80b5a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50532/0 (socket says 172.21.15.7:50532) 2024-04-25T11:19:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.595+0000 7f2af7e00640 1 -- 172.21.15.7:0/2567966616 learned_addr learned my addr 172.21.15.7:0/2567966616 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.595+0000 7f2afd800640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2af806f300 0x7f2af80b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2892770207 0 0) 0x7f2af80b4980 con 0x7f2af80b0290 2024-04-25T11:19:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2adc003520 con 0x7f2af80b0290 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2362680225 0 0) 0x7f2af80b4c60 con 0x7f2af806dc90 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2af80b4980 con 0x7f2af806dc90 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2892177847 0 0) 0x7f2adc003520 con 0x7f2af80b0290 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2af806f300 msgr2=0x7f2af80b6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2af806f300 0x7f2af80b6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 >> v1:172.21.15.87:6789/0 conn(0x7f2af806dc90 legacy=0x7f2af80afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af80b5220 con 0x7f2af80b0290 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2ae0001670 con 0x7f2af80b0290 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 >> v1:172.21.15.7:6789/0 conn(0x7f2af80b0290 legacy=0x7f2af80b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2af806f300 0x7f2adc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2adc004120 0x7f2adc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2adc004d40 0x7f2adc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2af80b4f40 con 0x7f2adc004d40 2024-04-25T11:19:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc00bb80 con 0x7f2adc004120 2024-04-25T11:19:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.596+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2adc00bd80 con 0x7f2af806f300 2024-04-25T11:19:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.597+0000 7f2afd800640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2adc004d40 0x7f2adc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.597+0000 7f2af7e00640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2adc004120 0x7f2adc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.597+0000 7f2afce00640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2af806f300 0x7f2adc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.597+0000 7f2afd800640 1 -- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2af806f300 msgr2=0x7f2adc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.597+0000 7f2afd800640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2af806f300 0x7f2adc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.597+0000 7f2afd800640 1 -- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2adc004120 msgr2=0x7f2adc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.597+0000 7f2afd800640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2adc004120 0x7f2adc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.597+0000 7f2afd800640 1 -- 172.21.15.7:0/2567966616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc003520 con 0x7f2adc004d40 2024-04-25T11:19:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.597+0000 7f2afce00640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2af806f300 0x7f2adc003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.597+0000 7f2af7e00640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2adc004120 0x7f2adc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:30.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.598+0000 7f2afd800640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2adc004d40 0x7f2adc005190 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009a90 tx=0x7f2adc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8876ef80526061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.599+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0036070 con 0x7f2adc004d40 2024-04-25T11:19:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.599+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af0031400 con 0x7f2adc004d40 2024-04-25T11:19:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.599+0000 7f2af7400640 1 -- 172.21.15.7:0/2567966616 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0009690 con 0x7f2adc004d40 2024-04-25T11:19:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.599+0000 7f2aff800640 1 -- 172.21.15.7:0/2567966616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2adc004d40 msgr2=0x7f2adc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.599+0000 7f2aff800640 1 --2- 172.21.15.7:0/2567966616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2adc004d40 0x7f2adc005190 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009a90 tx=0x7f2adc005830 comp rx=0 tx=0).stop 2024-04-25T11:19:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.599+0000 7f2afd800640 1 -- 172.21.15.7:0/2567966616 reap_dead start 2024-04-25T11:19:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.599+0000 7f2aff800640 1 -- 172.21.15.7:0/2567966616 shutdown_connections 2024-04-25T11:19:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.599+0000 7f2aff800640 1 -- 172.21.15.7:0/2567966616 >> 172.21.15.7:0/2567966616 conn(0x7f2af8074200 msgr2=0x7f2af806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.600+0000 7f2aff800640 1 -- 172.21.15.7:0/2567966616 shutdown_connections 2024-04-25T11:19:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.600+0000 7f2aff800640 1 -- 172.21.15.7:0/2567966616 wait complete. 2024-04-25T11:19:30.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.602+0000 7f2aff800640 1 Processor -- start 2024-04-25T11:19:30.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.602+0000 7f2aff800640 1 -- start start 2024-04-25T11:19:30.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2aff800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2adc004d40 0x7f2af8145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2aff800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af80b2690 0x7f2af80b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2aff800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2af814c100 0x7f2af814c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2aff800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2af80ce920 con 0x7f2af80b2690 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2aff800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2af815d220 con 0x7f2af814c100 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2aff800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2af80ceaa0 con 0x7f2adc004d40 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2af7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af80b2690 0x7f2af80b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2af7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af80b2690 0x7f2af80b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47100/0 (socket says 172.21.15.7:47100) 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2af7e00640 1 -- 172.21.15.7:0/4202863072 learned_addr learned my addr 172.21.15.7:0/4202863072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2afce00640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2adc004d40 0x7f2af8145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2af7e00640 1 -- 172.21.15.7:0/4202863072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2adc004d40 msgr2=0x7f2af8145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2af7e00640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2adc004d40 0x7f2af8145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2af7e00640 1 -- 172.21.15.7:0/4202863072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2af814c100 msgr2=0x7f2af814c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2af7e00640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2af814c100 0x7f2af814c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2af7e00640 1 -- 172.21.15.7:0/4202863072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af814ca50 con 0x7f2af80b2690 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.604+0000 7f2afce00640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2adc004d40 0x7f2af8145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.605+0000 7f2af7e00640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af80b2690 0x7f2af80b2aa0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000d840 tx=0x7f2ae000dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.605+0000 7f2af4c00640 1 -- 172.21.15.7:0/4202863072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0019070 con 0x7f2af80b2690 2024-04-25T11:19:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.605+0000 7f2af4c00640 1 -- 172.21.15.7:0/4202863072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae00074e0 con 0x7f2af80b2690 2024-04-25T11:19:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.605+0000 7f2af4c00640 1 -- 172.21.15.7:0/4202863072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae00049e0 con 0x7f2af80b2690 2024-04-25T11:19:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.605+0000 7f2aff800640 1 -- 172.21.15.7:0/4202863072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af814aaf0 con 0x7f2af80b2690 2024-04-25T11:19:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.605+0000 7f2aff800640 1 -- 172.21.15.7:0/4202863072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af8149d30 con 0x7f2af80b2690 2024-04-25T11:19:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.606+0000 7f2aff800640 1 -- 172.21.15.7:0/4202863072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af80b3170 con 0x7f2af80b2690 2024-04-25T11:19:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.607+0000 7f2af4c00640 1 -- 172.21.15.7:0/4202863072 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2ae0004b80 con 0x7f2af80b2690 2024-04-25T11:19:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.609+0000 7f2af4c00640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2ad40744b0 0x7f2ad40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.609+0000 7f2af4c00640 1 -- 172.21.15.7:0/4202863072 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f2ae0094ea0 con 0x7f2af80b2690 2024-04-25T11:19:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.609+0000 7f2afce00640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2ad40744b0 0x7f2ad40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.610+0000 7f2afce00640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2ad40744b0 0x7f2ad40769c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8007fd0 tx=0x7f2ae8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.614+0000 7f2af4c00640 1 -- 172.21.15.7:0/4202863072 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2af814ca50 con 0x7f2af80b2690 2024-04-25T11:19:30.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.810+0000 7f2aff800640 1 -- 172.21.15.7:0/4202863072 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2af806a100 con 0x7f2af80b2690 2024-04-25T11:19:30.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.812+0000 7f2af4c00640 1 -- 172.21.15.7:0/4202863072 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16788 (secure 0 0 0) 0x7f2ae005f400 con 0x7f2af80b2690 2024-04-25T11:19:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:30.811 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:10.691402+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:07.435764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:08.209216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 -- 172.21.15.7:0/4202863072 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2ad40744b0 msgr2=0x7f2ad40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2ad40744b0 0x7f2ad40769c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8007fd0 tx=0x7f2ae8009040 comp rx=0 tx=0).stop 2024-04-25T11:19:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 -- 172.21.15.7:0/4202863072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af80b2690 msgr2=0x7f2af80b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af80b2690 0x7f2af80b2aa0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000d840 tx=0x7f2ae000dd00 comp rx=0 tx=0).stop 2024-04-25T11:19:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 -- 172.21.15.7:0/4202863072 shutdown_connections 2024-04-25T11:19:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2ad40744b0 0x7f2ad40769c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2af814c100 0x7f2af814c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2af80b2690 0x7f2af80b2aa0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 --2- 172.21.15.7:0/4202863072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2adc004d40 0x7f2af8145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:30.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 -- 172.21.15.7:0/4202863072 >> 172.21.15.7:0/4202863072 conn(0x7f2af8074200 msgr2=0x7f2af80af000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:30.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 -- 172.21.15.7:0/4202863072 shutdown_connections 2024-04-25T11:19:30.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:30.816+0000 7f2aff800640 1 -- 172.21.15.7:0/4202863072 wait complete. 2024-04-25T11:19:30.842 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:19:30.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.124+0000 7fbf84400640 1 Processor -- start 2024-04-25T11:19:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.124+0000 7fbf84400640 1 -- start start 2024-04-25T11:19:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.126+0000 7fbf84400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf7c0af540 0x7fbf7c09f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.126+0000 7fbf84400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf7c15bc40 0x7fbf7c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.126+0000 7fbf84400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf7c15f4e0 con 0x7fbf7c15ef90 2024-04-25T11:19:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.126+0000 7fbf84400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbf7c071190 con 0x7fbf7c15bc40 2024-04-25T11:19:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.126+0000 7fbf84400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7c0b0560 con 0x7fbf7c0af540 2024-04-25T11:19:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.127+0000 7fbf82400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf7c15bc40 0x7fbf7c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.127+0000 7fbf82400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf7c15bc40 0x7fbf7c0cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47102/0 (socket says 172.21.15.7:47102) 2024-04-25T11:19:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.127+0000 7fbf82400640 1 -- 172.21.15.7:0/4049081078 learned_addr learned my addr 172.21.15.7:0/4049081078 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.127+0000 7fbf81000640 1 --2- 172.21.15.7:0/4049081078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf7c0af540 0x7fbf7c09f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.127+0000 7fbf82400640 1 -- 172.21.15.7:0/4049081078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf7c0af540 msgr2=0x7fbf7c09f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.127+0000 7fbf82400640 1 --2- 172.21.15.7:0/4049081078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf7c0af540 0x7fbf7c09f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.127+0000 7fbf82400640 1 -- 172.21.15.7:0/4049081078 >> v1:172.21.15.87:6789/0 conn(0x7fbf7c15ef90 legacy=0x7fbf7c0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.127+0000 7fbf82400640 1 -- 172.21.15.7:0/4049081078 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c0b0840 con 0x7fbf7c15bc40 2024-04-25T11:19:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.128+0000 7fbf82400640 1 --2- 172.21.15.7:0/4049081078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7c15bc40 0x7fbf7c0cf980 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009870 tx=0x7fbf7402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d53847cd71e171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.129+0000 7fbf7be00640 1 -- 172.21.15.7:0/4049081078 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74037070 con 0x7fbf7c15bc40 2024-04-25T11:19:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.129+0000 7fbf7be00640 1 -- 172.21.15.7:0/4049081078 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf74032440 con 0x7fbf7c15bc40 2024-04-25T11:19:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.129+0000 7fbf7be00640 1 -- 172.21.15.7:0/4049081078 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74032810 con 0x7fbf7c15bc40 2024-04-25T11:19:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.130+0000 7fbf84400640 1 -- 172.21.15.7:0/4049081078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7c15bc40 msgr2=0x7fbf7c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.130+0000 7fbf84400640 1 --2- 172.21.15.7:0/4049081078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7c15bc40 0x7fbf7c0cf980 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009870 tx=0x7fbf7402a180 comp rx=0 tx=0).stop 2024-04-25T11:19:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.130+0000 7fbf84400640 1 -- 172.21.15.7:0/4049081078 shutdown_connections 2024-04-25T11:19:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.130+0000 7fbf84400640 1 --2- 172.21.15.7:0/4049081078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7c15bc40 0x7fbf7c0cf980 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.130+0000 7fbf84400640 1 --2- 172.21.15.7:0/4049081078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf7c0af540 0x7fbf7c09f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.130+0000 7fbf84400640 1 -- 172.21.15.7:0/4049081078 >> 172.21.15.7:0/4049081078 conn(0x7fbf7c074200 msgr2=0x7fbf7c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.130+0000 7fbf84400640 1 -- 172.21.15.7:0/4049081078 shutdown_connections 2024-04-25T11:19:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.130+0000 7fbf84400640 1 -- 172.21.15.7:0/4049081078 wait complete. 2024-04-25T11:19:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.132+0000 7fbf84400640 1 Processor -- start 2024-04-25T11:19:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.132+0000 7fbf84400640 1 -- start start 2024-04-25T11:19:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.134+0000 7fbf84400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7c15bc40 0x7fbf7c0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.134+0000 7fbf84400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf7c15ef90 0x7fbf7c0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.134+0000 7fbf84400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c0cb390 0x7fbf7c14ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.134+0000 7fbf84400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c0b1320 con 0x7fbf7c15bc40 2024-04-25T11:19:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.134+0000 7fbf84400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7c0b0e00 con 0x7fbf7c0cb390 2024-04-25T11:19:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.134+0000 7fbf84400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7c0afb80 con 0x7fbf7c15ef90 2024-04-25T11:19:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.134+0000 7fbf81a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7c15bc40 0x7fbf7c0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.134+0000 7fbf81a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7c15bc40 0x7fbf7c0b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47104/0 (socket says 172.21.15.7:47104) 2024-04-25T11:19:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.134+0000 7fbf81a00640 1 -- 172.21.15.7:0/578073618 learned_addr learned my addr 172.21.15.7:0/578073618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.135+0000 7fbf81000640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf7c15ef90 0x7fbf7c0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.135+0000 7fbf82400640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c0cb390 0x7fbf7c14ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.135+0000 7fbf81a00640 1 -- 172.21.15.7:0/578073618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf7c15ef90 msgr2=0x7fbf7c0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.135+0000 7fbf81a00640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf7c15ef90 0x7fbf7c0cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.135+0000 7fbf81a00640 1 -- 172.21.15.7:0/578073618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c0cb390 msgr2=0x7fbf7c14ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.135+0000 7fbf81a00640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c0cb390 0x7fbf7c14ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.135+0000 7fbf81a00640 1 -- 172.21.15.7:0/578073618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7c0b6750 con 0x7fbf7c15bc40 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.135+0000 7fbf82400640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c0cb390 0x7fbf7c14ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.135+0000 7fbf81a00640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7c15bc40 0x7fbf7c0b6210 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00ba80 tx=0x7fbf6c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.136+0000 7fbf79600640 1 -- 172.21.15.7:0/578073618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c00cc00 con 0x7fbf7c15bc40 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.136+0000 7fbf79600640 1 -- 172.21.15.7:0/578073618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6c00cda0 con 0x7fbf7c15bc40 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.136+0000 7fbf84400640 1 -- 172.21.15.7:0/578073618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7c0cbbc0 con 0x7fbf7c15bc40 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.136+0000 7fbf79600640 1 -- 172.21.15.7:0/578073618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c003bb0 con 0x7fbf7c15bc40 2024-04-25T11:19:31.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.136+0000 7fbf84400640 1 -- 172.21.15.7:0/578073618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7c0cc180 con 0x7fbf7c15bc40 2024-04-25T11:19:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.137+0000 7fbf84400640 1 -- 172.21.15.7:0/578073618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7c06a100 con 0x7fbf7c15bc40 2024-04-25T11:19:31.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.139+0000 7fbf79600640 1 -- 172.21.15.7:0/578073618 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbf6c014440 con 0x7fbf7c15bc40 2024-04-25T11:19:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.140+0000 7fbf79600640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf5c0744b0 0x7fbf5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.140+0000 7fbf79600640 1 -- 172.21.15.7:0/578073618 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fbf6c0952e0 con 0x7fbf7c15bc40 2024-04-25T11:19:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.141+0000 7fbf81000640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf5c0744b0 0x7fbf5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.142+0000 7fbf81000640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf5c0744b0 0x7fbf5c0769c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbf68006fd0 tx=0x7fbf68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.145+0000 7fbf79600640 1 -- 172.21.15.7:0/578073618 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf6c05f8d0 con 0x7fbf7c15bc40 2024-04-25T11:19:31.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.392+0000 7fbf84400640 1 -- 172.21.15.7:0/578073618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf7c0cfa10 con 0x7fbf7c15bc40 2024-04-25T11:19:31.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.393+0000 7fbf79600640 1 -- 172.21.15.7:0/578073618 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5171 (secure 0 0 0) 0x7fbf6c05f1d0 con 0x7fbf7c15bc40 2024-04-25T11:19:31.392 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:31.392 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5846,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/4132210096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":4132210096},{"type":"v1","addr":"172.21.15.87:6834","nonce":4132210096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2657338269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2657338269},{"type":"v1","addr":"172.21.15.87:6835","nonce":2657338269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6457,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/138020048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":138020048},{"type":"v1","addr":"172.21.15.7:6836","nonce":138020048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:09.244634+0000","modified":"2024-04-25T11:19:12.318984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/990441706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":990441706},{"type":"v1","addr":"172.21.15.7:6837","nonce":990441706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7623,"qdb_cluster":[7623]},"id":7}]} 2024-04-25T11:19:31.392 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 67 2024-04-25T11:19:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 -- 172.21.15.7:0/578073618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf5c0744b0 msgr2=0x7fbf5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf5c0744b0 0x7fbf5c0769c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbf68006fd0 tx=0x7fbf68008040 comp rx=0 tx=0).stop 2024-04-25T11:19:31.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 -- 172.21.15.7:0/578073618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7c15bc40 msgr2=0x7fbf7c0b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7c15bc40 0x7fbf7c0b6210 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00ba80 tx=0x7fbf6c00bf40 comp rx=0 tx=0).stop 2024-04-25T11:19:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 -- 172.21.15.7:0/578073618 shutdown_connections 2024-04-25T11:19:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf5c0744b0 0x7fbf5c0769c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf7c0cb390 0x7fbf7c14ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf7c15ef90 0x7fbf7c0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 --2- 172.21.15.7:0/578073618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf7c15bc40 0x7fbf7c0b6210 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 -- 172.21.15.7:0/578073618 >> 172.21.15.7:0/578073618 conn(0x7fbf7c074200 msgr2=0x7fbf7c161060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 -- 172.21.15.7:0/578073618 shutdown_connections 2024-04-25T11:19:31.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.396+0000 7fbf84400640 1 -- 172.21.15.7:0/578073618 wait complete. 2024-04-25T11:19:31.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:31.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.704+0000 7f0264a00640 1 Processor -- start 2024-04-25T11:19:31.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.704+0000 7f0264a00640 1 -- start start 2024-04-25T11:19:31.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.706+0000 7f0264a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f02600b04d0 0x7f02600b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.706+0000 7f0264a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02600b4a80 0x7f02600b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.706+0000 7f0264a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f026015e190 con 0x7f0260070a70 2024-04-25T11:19:31.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.706+0000 7f0264a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f026015d610 con 0x7f02600b04d0 2024-04-25T11:19:31.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.706+0000 7f0264a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f026015d8f0 con 0x7f02600b4a80 2024-04-25T11:19:31.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.707+0000 7f025e800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0260070a70 0x7f02600ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50540/0 (socket says 172.21.15.7:50540) 2024-04-25T11:19:31.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.707+0000 7f025e800640 1 -- 172.21.15.7:0/3039407619 learned_addr learned my addr 172.21.15.7:0/3039407619 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.707+0000 7f025de00640 1 --2- 172.21.15.7:0/3039407619 >> v2:172.21.15.87:3300/0 conn(0x7f02600b04d0 0x7f02600b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.707+0000 7f025f200640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02600b4a80 0x7f02600b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.707+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2094434370 0 0) 0x7f026015e190 con 0x7f0260070a70 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.707+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0244003520 con 0x7f0260070a70 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4161747994 0 0) 0x7f0244003520 con 0x7f0260070a70 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02600b4a80 msgr2=0x7f02600b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02600b4a80 0x7f02600b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 >> v2:172.21.15.87:3300/0 conn(0x7f02600b04d0 msgr2=0x7f02600b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 --2- 172.21.15.7:0/3039407619 >> v2:172.21.15.87:3300/0 conn(0x7f02600b04d0 0x7f02600b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026015dbd0 con 0x7f0260070a70 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0250001a10 con 0x7f0260070a70 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 >> v1:172.21.15.7:6789/0 conn(0x7f0260070a70 legacy=0x7f02600ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02600b4a80 0x7f02440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244003f80 0x7f0244004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0244004d40 0x7f0244005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f026015d610 con 0x7f0244004d40 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f026015d8f0 con 0x7f0244003f80 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f024400ba70 con 0x7f02600b4a80 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.708+0000 7f025f200640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0244004d40 0x7f0244005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.709+0000 7f025de00640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244003f80 0x7f0244004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.709+0000 7f025e800640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02600b4a80 0x7f02440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.709+0000 7f025f200640 1 -- 172.21.15.7:0/3039407619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02600b4a80 msgr2=0x7f02440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.709+0000 7f025f200640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02600b4a80 0x7f02440039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.709+0000 7f025f200640 1 -- 172.21.15.7:0/3039407619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244003f80 msgr2=0x7f0244004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.709+0000 7f025f200640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0244003f80 0x7f0244004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.709+0000 7f025f200640 1 -- 172.21.15.7:0/3039407619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0244003520 con 0x7f0244004d40 2024-04-25T11:19:31.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.710+0000 7f025f200640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0244004d40 0x7f0244005150 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f0258009650 tx=0x7f0244005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89050509ad09a956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.710+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258038070 con 0x7f0244004d40 2024-04-25T11:19:31.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.710+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0258033400 con 0x7f0244004d40 2024-04-25T11:19:31.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.710+0000 7f025d400640 1 -- 172.21.15.7:0/3039407619 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258009690 con 0x7f0244004d40 2024-04-25T11:19:31.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.711+0000 7f0264a00640 1 -- 172.21.15.7:0/3039407619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0244004d40 msgr2=0x7f0244005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.711+0000 7f0264a00640 1 --2- 172.21.15.7:0/3039407619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0244004d40 0x7f0244005150 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f0258009650 tx=0x7f0244005830 comp rx=0 tx=0).stop 2024-04-25T11:19:31.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.711+0000 7f025f200640 1 -- 172.21.15.7:0/3039407619 reap_dead start 2024-04-25T11:19:31.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.711+0000 7f0264a00640 1 -- 172.21.15.7:0/3039407619 shutdown_connections 2024-04-25T11:19:31.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.711+0000 7f0264a00640 1 -- 172.21.15.7:0/3039407619 >> 172.21.15.7:0/3039407619 conn(0x7f02600b3820 msgr2=0x7f02600b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:31.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.711+0000 7f0264a00640 1 -- 172.21.15.7:0/3039407619 shutdown_connections 2024-04-25T11:19:31.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.711+0000 7f0264a00640 1 -- 172.21.15.7:0/3039407619 wait complete. 2024-04-25T11:19:31.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.714+0000 7f0264a00640 1 Processor -- start 2024-04-25T11:19:31.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.714+0000 7f0264a00640 1 -- start start 2024-04-25T11:19:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.715+0000 7f0264a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0260153c50 0x7f0260142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.715+0000 7f0264a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0260151dc0 0x7f02601521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.715+0000 7f0264a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f026013f750 0x7f026013fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.715+0000 7f0264a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02600b0e20 con 0x7f0260151dc0 2024-04-25T11:19:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.715+0000 7f0264a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f02600aef90 con 0x7f026013f750 2024-04-25T11:19:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.715+0000 7f0264a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f026015d170 con 0x7f0260153c50 2024-04-25T11:19:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.716+0000 7f025de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0260151dc0 0x7f02601521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.716+0000 7f025de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0260151dc0 0x7f02601521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47126/0 (socket says 172.21.15.7:47126) 2024-04-25T11:19:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.716+0000 7f025de00640 1 -- 172.21.15.7:0/315355057 learned_addr learned my addr 172.21.15.7:0/315355057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:31.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.716+0000 7f025e800640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0260153c50 0x7f0260142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.716+0000 7f025de00640 1 -- 172.21.15.7:0/315355057 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0260153c50 msgr2=0x7f0260142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.716+0000 7f025f200640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f026013f750 0x7f026013fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.716+0000 7f025de00640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0260153c50 0x7f0260142e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.716+0000 7f025de00640 1 -- 172.21.15.7:0/315355057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f026013f750 msgr2=0x7f026013fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.716+0000 7f025de00640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f026013f750 0x7f026013fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.716+0000 7f025de00640 1 -- 172.21.15.7:0/315355057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0260143380 con 0x7f0260151dc0 2024-04-25T11:19:31.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.717+0000 7f025de00640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0260151dc0 0x7f02601521d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f024c011600 tx=0x7f024c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.717+0000 7f0256a00640 1 -- 172.21.15.7:0/315355057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c00f920 con 0x7f0260151dc0 2024-04-25T11:19:31.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.717+0000 7f0256a00640 1 -- 172.21.15.7:0/315355057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c00fac0 con 0x7f0260151dc0 2024-04-25T11:19:31.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.717+0000 7f0264a00640 1 -- 172.21.15.7:0/315355057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0260152710 con 0x7f0260151dc0 2024-04-25T11:19:31.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.717+0000 7f0256a00640 1 -- 172.21.15.7:0/315355057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c00fe70 con 0x7f0260151dc0 2024-04-25T11:19:31.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.717+0000 7f0264a00640 1 -- 172.21.15.7:0/315355057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02601542c0 con 0x7f0260151dc0 2024-04-25T11:19:31.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.718+0000 7f0264a00640 1 -- 172.21.15.7:0/315355057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0260050740 con 0x7f0260151dc0 2024-04-25T11:19:31.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.720+0000 7f0256a00640 1 -- 172.21.15.7:0/315355057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f024c015460 con 0x7f0260151dc0 2024-04-25T11:19:31.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.721+0000 7f0256a00640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02400744b0 0x7f02400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:31.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.721+0000 7f0256a00640 1 -- 172.21.15.7:0/315355057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f024c095d30 con 0x7f0260151dc0 2024-04-25T11:19:31.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.722+0000 7f025e800640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02400744b0 0x7f02400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:31.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.722+0000 7f025e800640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02400744b0 0x7f02400769c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f026015fab0 tx=0x7f02500012d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:31.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.727+0000 7f0256a00640 1 -- 172.21.15.7:0/315355057 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0260152710 con 0x7f0260151dc0 2024-04-25T11:19:31.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.976+0000 7f0264a00640 1 -- 172.21.15.7:0/315355057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0260159250 con 0x7f0260151dc0 2024-04-25T11:19:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.978+0000 7f0256a00640 1 -- 172.21.15.7:0/315355057 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5171 (secure 0 0 0) 0x7f0260143380 con 0x7f0260151dc0 2024-04-25T11:19:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:31.977 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5846,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/4132210096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":4132210096},{"type":"v1","addr":"172.21.15.87:6834","nonce":4132210096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2657338269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2657338269},{"type":"v1","addr":"172.21.15.87:6835","nonce":2657338269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6457,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/138020048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":138020048},{"type":"v1","addr":"172.21.15.7:6836","nonce":138020048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:09.244634+0000","modified":"2024-04-25T11:19:12.318984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7623},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7623":{"gid":7623,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/990441706","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":990441706},{"type":"v1","addr":"172.21.15.7:6837","nonce":990441706}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7623,"qdb_cluster":[7623]},"id":7}]} 2024-04-25T11:19:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 67 2024-04-25T11:19:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 -- 172.21.15.7:0/315355057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02400744b0 msgr2=0x7f02400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02400744b0 0x7f02400769c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f026015fab0 tx=0x7f02500012d0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 -- 172.21.15.7:0/315355057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0260151dc0 msgr2=0x7f02601521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0260151dc0 0x7f02601521d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f024c011600 tx=0x7f024c011ac0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 -- 172.21.15.7:0/315355057 shutdown_connections 2024-04-25T11:19:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02400744b0 0x7f02400769c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f026013f750 0x7f026013fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0260151dc0 0x7f02601521d0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 --2- 172.21.15.7:0/315355057 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0260153c50 0x7f0260142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 -- 172.21.15.7:0/315355057 >> 172.21.15.7:0/315355057 conn(0x7f02600b3820 msgr2=0x7f02600d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 -- 172.21.15.7:0/315355057 shutdown_connections 2024-04-25T11:19:31.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:31.981+0000 7f0264a00640 1 -- 172.21.15.7:0/315355057 wait complete. 2024-04-25T11:19:32.006 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:32.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.296+0000 7f15d6200640 1 Processor -- start 2024-04-25T11:19:32.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.296+0000 7f15d6200640 1 -- start start 2024-04-25T11:19:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.298+0000 7f15d6200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15d00b1610 0x7f15d00cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.298+0000 7f15d6200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f15d00b1d50 0x7f15d00af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.299+0000 7f15d6200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15d00b24d0 con 0x7f15d00b73d0 2024-04-25T11:19:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.299+0000 7f15d6200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f15d00b2790 con 0x7f15d00b1610 2024-04-25T11:19:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.299+0000 7f15d6200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f15d00b2a70 con 0x7f15d00b1d50 2024-04-25T11:19:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.299+0000 7f15d4e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15d00b1610 0x7f15d00cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.299+0000 7f15d4e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15d00b1610 0x7f15d00cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47136/0 (socket says 172.21.15.7:47136) 2024-04-25T11:19:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.299+0000 7f15d4e00640 1 -- 172.21.15.7:0/1704335362 learned_addr learned my addr 172.21.15.7:0/1704335362 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.299+0000 7f15d5800640 1 --1- 172.21.15.7:0/1704335362 >> v1:172.21.15.7:6789/0 conn(0x7f15d00b73d0 0x7f15d00b77e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50544/0 (socket says 172.21.15.7:50544) 2024-04-25T11:19:32.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.299+0000 7f15cfe00640 1 --2- 172.21.15.7:0/1704335362 >> v2:172.21.15.87:3300/0 conn(0x7f15d00b1d50 0x7f15d00af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.300+0000 7f15d4e00640 1 -- 172.21.15.7:0/1704335362 >> v2:172.21.15.87:3300/0 conn(0x7f15d00b1d50 msgr2=0x7f15d00af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.300+0000 7f15d4e00640 1 --2- 172.21.15.7:0/1704335362 >> v2:172.21.15.87:3300/0 conn(0x7f15d00b1d50 0x7f15d00af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.300+0000 7f15d4e00640 1 -- 172.21.15.7:0/1704335362 >> v1:172.21.15.7:6789/0 conn(0x7f15d00b73d0 legacy=0x7f15d00b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.300+0000 7f15d4e00640 1 -- 172.21.15.7:0/1704335362 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d00b2d50 con 0x7f15d00b1610 2024-04-25T11:19:32.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.301+0000 7f15d4e00640 1 --2- 172.21.15.7:0/1704335362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00b1610 0x7f15d00cecf0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f15d00733d0 tx=0x7f15c402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab911a55586e1a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.301+0000 7f15cf400640 1 -- 172.21.15.7:0/1704335362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4037070 con 0x7f15d00b1610 2024-04-25T11:19:32.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.301+0000 7f15cf400640 1 -- 172.21.15.7:0/1704335362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c4032440 con 0x7f15d00b1610 2024-04-25T11:19:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.301+0000 7f15cf400640 1 -- 172.21.15.7:0/1704335362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4032810 con 0x7f15d00b1610 2024-04-25T11:19:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.302+0000 7f15d6200640 1 -- 172.21.15.7:0/1704335362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00b1610 msgr2=0x7f15d00cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.302+0000 7f15d6200640 1 --2- 172.21.15.7:0/1704335362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00b1610 0x7f15d00cecf0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f15d00733d0 tx=0x7f15c402a200 comp rx=0 tx=0).stop 2024-04-25T11:19:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.302+0000 7f15d6200640 1 -- 172.21.15.7:0/1704335362 shutdown_connections 2024-04-25T11:19:32.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.302+0000 7f15d6200640 1 --2- 172.21.15.7:0/1704335362 >> v2:172.21.15.87:3300/0 conn(0x7f15d00b1d50 0x7f15d00af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.302+0000 7f15d6200640 1 --2- 172.21.15.7:0/1704335362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00b1610 0x7f15d00cecf0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.302+0000 7f15d6200640 1 -- 172.21.15.7:0/1704335362 >> 172.21.15.7:0/1704335362 conn(0x7f15d009b540 msgr2=0x7f15d009b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.302+0000 7f15d6200640 1 -- 172.21.15.7:0/1704335362 shutdown_connections 2024-04-25T11:19:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.302+0000 7f15d6200640 1 -- 172.21.15.7:0/1704335362 wait complete. 2024-04-25T11:19:32.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.304+0000 7f15d6200640 1 Processor -- start 2024-04-25T11:19:32.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.305+0000 7f15d6200640 1 -- start start 2024-04-25T11:19:32.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.306+0000 7f15d6200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d00b1610 0x7f15d014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.306+0000 7f15d6200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d00b1d50 0x7f15d01f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.306+0000 7f15d6200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00b73d0 0x7f15d0160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.306+0000 7f15d6200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15d00afa00 con 0x7f15d00b73d0 2024-04-25T11:19:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.306+0000 7f15d6200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f15d00b2160 con 0x7f15d00b1610 2024-04-25T11:19:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.306+0000 7f15d6200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f15d00afb80 con 0x7f15d00b1d50 2024-04-25T11:19:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.306+0000 7f15d5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00b73d0 0x7f15d0160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.307+0000 7f15d5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00b73d0 0x7f15d0160a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47148/0 (socket says 172.21.15.7:47148) 2024-04-25T11:19:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.307+0000 7f15d5800640 1 -- 172.21.15.7:0/2381424070 learned_addr learned my addr 172.21.15.7:0/2381424070 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.307+0000 7f15d4e00640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d00b1610 0x7f15d014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.307+0000 7f15cfe00640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d00b1d50 0x7f15d01f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.307+0000 7f15d5800640 1 -- 172.21.15.7:0/2381424070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d00b1d50 msgr2=0x7f15d01f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.307+0000 7f15d5800640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d00b1d50 0x7f15d01f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.307+0000 7f15d5800640 1 -- 172.21.15.7:0/2381424070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d00b1610 msgr2=0x7f15d014c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.307+0000 7f15d5800640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d00b1610 0x7f15d014c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.307+0000 7f15d5800640 1 -- 172.21.15.7:0/2381424070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d014c630 con 0x7f15d00b73d0 2024-04-25T11:19:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.307+0000 7f15cfe00640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d00b1d50 0x7f15d01f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.307+0000 7f15d5800640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00b73d0 0x7f15d0160a40 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c730 tx=0x7f15c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.308+0000 7f15ccc00640 1 -- 172.21.15.7:0/2381424070 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8017070 con 0x7f15d00b73d0 2024-04-25T11:19:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.308+0000 7f15ccc00640 1 -- 172.21.15.7:0/2381424070 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c800ce40 con 0x7f15d00b73d0 2024-04-25T11:19:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.308+0000 7f15ccc00640 1 -- 172.21.15.7:0/2381424070 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c800a840 con 0x7f15d00b73d0 2024-04-25T11:19:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.308+0000 7f15d6200640 1 -- 172.21.15.7:0/2381424070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d00c7ec0 con 0x7f15d00b73d0 2024-04-25T11:19:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.308+0000 7f15d6200640 1 -- 172.21.15.7:0/2381424070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d00c8480 con 0x7f15d00b73d0 2024-04-25T11:19:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.309+0000 7f15d6200640 1 -- 172.21.15.7:0/2381424070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d0050740 con 0x7f15d00b73d0 2024-04-25T11:19:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.310+0000 7f15ccc00640 1 -- 172.21.15.7:0/2381424070 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f15c801a070 con 0x7f15d00b73d0 2024-04-25T11:19:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.311+0000 7f15ccc00640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b00744b0 0x7f15b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.311+0000 7f15ccc00640 1 -- 172.21.15.7:0/2381424070 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f15c8094930 con 0x7f15d00b73d0 2024-04-25T11:19:32.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.312+0000 7f15d4e00640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b00744b0 0x7f15b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.312+0000 7f15d4e00640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b00744b0 0x7f15b00769c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f15c4034000 tx=0x7f15c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.316+0000 7f15ccc00640 1 -- 172.21.15.7:0/2381424070 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f15d014c630 con 0x7f15d00b73d0 2024-04-25T11:19:32.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.522+0000 7f15d6200640 1 -- 172.21.15.7:0/2381424070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15d00c94e0 con 0x7f15d00b73d0 2024-04-25T11:19:32.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.526+0000 7f15ccc00640 1 -- 172.21.15.7:0/2381424070 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16788 (secure 0 0 0) 0x7f15c805e840 con 0x7f15d00b73d0 2024-04-25T11:19:32.524 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:32.524 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:10.691402+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:07.435764+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:08.209216+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.528+0000 7f15d6200640 1 -- 172.21.15.7:0/2381424070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b00744b0 msgr2=0x7f15b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.528+0000 7f15d6200640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b00744b0 0x7f15b00769c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f15c4034000 tx=0x7f15c402e000 comp rx=0 tx=0).stop 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.528+0000 7f15d6200640 1 -- 172.21.15.7:0/2381424070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00b73d0 msgr2=0x7f15d0160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.528+0000 7f15d6200640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00b73d0 0x7f15d0160a40 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c730 tx=0x7f15c800cbf0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.529+0000 7f15d6200640 1 -- 172.21.15.7:0/2381424070 shutdown_connections 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.529+0000 7f15d6200640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15b00744b0 0x7f15b00769c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.529+0000 7f15d6200640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00b73d0 0x7f15d0160a40 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.529+0000 7f15d6200640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15d00b1d50 0x7f15d01f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.529+0000 7f15d6200640 1 --2- 172.21.15.7:0/2381424070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d00b1610 0x7f15d014c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.529+0000 7f15d6200640 1 -- 172.21.15.7:0/2381424070 >> 172.21.15.7:0/2381424070 conn(0x7f15d009b540 msgr2=0x7f15d0068860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.529+0000 7f15d6200640 1 -- 172.21.15.7:0/2381424070 shutdown_connections 2024-04-25T11:19:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.529+0000 7f15d6200640 1 -- 172.21.15.7:0/2381424070 wait complete. 2024-04-25T11:19:32.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:19:32.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.833+0000 7fbf7f200640 1 Processor -- start 2024-04-25T11:19:32.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.834+0000 7fbf7f200640 1 -- start start 2024-04-25T11:19:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.835+0000 7fbf7f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf8015eed0 0x7fbf800cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.836+0000 7fbf7f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf801416a0 0x7fbf800b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.836+0000 7fbf7f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf8006c570 con 0x7fbf80141290 2024-04-25T11:19:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.836+0000 7fbf7f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbf8006c850 con 0x7fbf801416a0 2024-04-25T11:19:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.836+0000 7fbf7f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8006cb30 con 0x7fbf8015eed0 2024-04-25T11:19:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.836+0000 7fbf7e800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf801416a0 0x7fbf800b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.836+0000 7fbf7e800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf801416a0 0x7fbf800b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47152/0 (socket says 172.21.15.7:47152) 2024-04-25T11:19:32.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.836+0000 7fbf7e800640 1 -- 172.21.15.7:0/3559312412 learned_addr learned my addr 172.21.15.7:0/3559312412 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:32.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.836+0000 7fbf7de00640 1 --2- 172.21.15.7:0/3559312412 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf8015eed0 0x7fbf800cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.836+0000 7fbf7e800640 1 -- 172.21.15.7:0/3559312412 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf8015eed0 msgr2=0x7fbf800cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.836+0000 7fbf7e800640 1 --2- 172.21.15.7:0/3559312412 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf8015eed0 0x7fbf800cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.836+0000 7fbf7e800640 1 -- 172.21.15.7:0/3559312412 >> v1:172.21.15.87:6789/0 conn(0x7fbf80141290 legacy=0x7fbf8006d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:32.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.837+0000 7fbf7e800640 1 -- 172.21.15.7:0/3559312412 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8006ce10 con 0x7fbf801416a0 2024-04-25T11:19:32.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.838+0000 7fbf7e800640 1 --2- 172.21.15.7:0/3559312412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf801416a0 0x7fbf800b4590 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009e50 tx=0x7fbf7802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eeb0826751dd90f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.838+0000 7fbf7ca00640 1 -- 172.21.15.7:0/3559312412 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78036070 con 0x7fbf801416a0 2024-04-25T11:19:32.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.838+0000 7fbf7ca00640 1 -- 172.21.15.7:0/3559312412 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf78031600 con 0x7fbf801416a0 2024-04-25T11:19:32.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.838+0000 7fbf7ca00640 1 -- 172.21.15.7:0/3559312412 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf780319d0 con 0x7fbf801416a0 2024-04-25T11:19:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.839+0000 7fbf7f200640 1 -- 172.21.15.7:0/3559312412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf801416a0 msgr2=0x7fbf800b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.839+0000 7fbf7f200640 1 --2- 172.21.15.7:0/3559312412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf801416a0 0x7fbf800b4590 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009e50 tx=0x7fbf7802a6c0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.839+0000 7fbf7f200640 1 -- 172.21.15.7:0/3559312412 shutdown_connections 2024-04-25T11:19:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.839+0000 7fbf7f200640 1 --2- 172.21.15.7:0/3559312412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf801416a0 0x7fbf800b4590 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.839+0000 7fbf7f200640 1 --2- 172.21.15.7:0/3559312412 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf8015eed0 0x7fbf800cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.839+0000 7fbf7f200640 1 -- 172.21.15.7:0/3559312412 >> 172.21.15.7:0/3559312412 conn(0x7fbf800d1540 msgr2=0x7fbf800d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.839+0000 7fbf7f200640 1 -- 172.21.15.7:0/3559312412 shutdown_connections 2024-04-25T11:19:32.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.839+0000 7fbf7f200640 1 -- 172.21.15.7:0/3559312412 wait complete. 2024-04-25T11:19:32.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.841+0000 7fbf7f200640 1 Processor -- start 2024-04-25T11:19:32.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.841+0000 7fbf7f200640 1 -- start start 2024-04-25T11:19:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.843+0000 7fbf7f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf80141290 0x7fbf801f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.843+0000 7fbf7f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf801416a0 0x7fbf801f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.843+0000 7fbf7f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf8015eed0 0x7fbf8015a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.843+0000 7fbf7f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80141ab0 con 0x7fbf8015eed0 2024-04-25T11:19:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.843+0000 7fbf7f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbf800cdcc0 con 0x7fbf80141290 2024-04-25T11:19:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.843+0000 7fbf7f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8006dd20 con 0x7fbf801416a0 2024-04-25T11:19:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.843+0000 7fbf7e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf8015eed0 0x7fbf8015a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.843+0000 7fbf7e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf8015eed0 0x7fbf8015a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47164/0 (socket says 172.21.15.7:47164) 2024-04-25T11:19:32.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.843+0000 7fbf7e800640 1 -- 172.21.15.7:0/1133787665 learned_addr learned my addr 172.21.15.7:0/1133787665 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.844+0000 7fbf7de00640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf80141290 0x7fbf801f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.844+0000 7fbf7d400640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf801416a0 0x7fbf801f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.844+0000 7fbf7e800640 1 -- 172.21.15.7:0/1133787665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf801416a0 msgr2=0x7fbf801f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.844+0000 7fbf7e800640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf801416a0 0x7fbf801f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.844+0000 7fbf7e800640 1 -- 172.21.15.7:0/1133787665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf80141290 msgr2=0x7fbf801f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.844+0000 7fbf7e800640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf80141290 0x7fbf801f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.844+0000 7fbf7e800640 1 -- 172.21.15.7:0/1133787665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf801f42d0 con 0x7fbf8015eed0 2024-04-25T11:19:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.844+0000 7fbf7e800640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf8015eed0 0x7fbf8015a3b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009f70 tx=0x7fbf78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.844+0000 7fbf76000640 1 -- 172.21.15.7:0/1133787665 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78036050 con 0x7fbf8015eed0 2024-04-25T11:19:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.845+0000 7fbf76000640 1 -- 172.21.15.7:0/1133787665 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf78042070 con 0x7fbf8015eed0 2024-04-25T11:19:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.845+0000 7fbf76000640 1 -- 172.21.15.7:0/1133787665 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78009510 con 0x7fbf8015eed0 2024-04-25T11:19:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.845+0000 7fbf7f200640 1 -- 172.21.15.7:0/1133787665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf801f45b0 con 0x7fbf8015eed0 2024-04-25T11:19:32.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.845+0000 7fbf7f200640 1 -- 172.21.15.7:0/1133787665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8015c140 con 0x7fbf8015eed0 2024-04-25T11:19:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.846+0000 7fbf7f200640 1 -- 172.21.15.7:0/1133787665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf80050740 con 0x7fbf8015eed0 2024-04-25T11:19:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.847+0000 7fbf76000640 1 -- 172.21.15.7:0/1133787665 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbf78007bd0 con 0x7fbf8015eed0 2024-04-25T11:19:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.849+0000 7fbf76000640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf600744b0 0x7fbf600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.849+0000 7fbf76000640 1 -- 172.21.15.7:0/1133787665 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fbf780b6340 con 0x7fbf8015eed0 2024-04-25T11:19:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.849+0000 7fbf7de00640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf600744b0 0x7fbf600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:32.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.850+0000 7fbf7de00640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf600744b0 0x7fbf600769c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbf70006fd0 tx=0x7fbf70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:32.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:32.854+0000 7fbf76000640 1 -- 172.21.15.7:0/1133787665 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbf780808a0 con 0x7fbf8015eed0 2024-04-25T11:19:33.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.096+0000 7fbf7f200640 1 -- 172.21.15.7:0/1133787665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbf801588a0 con 0x7fbf8015eed0 2024-04-25T11:19:33.095 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-04-25T11:19:33.097+0000 7fac34e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:19:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.101+0000 7fbf76000640 1 -- 172.21.15.7:0/1133787665 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7fbf780801a0 con 0x7fbf8015eed0 2024-04-25T11:19:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:19:33.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.103+0000 7fbf7f200640 1 -- 172.21.15.7:0/1133787665 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf600744b0 msgr2=0x7fbf600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.103+0000 7fbf7f200640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf600744b0 0x7fbf600769c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbf70006fd0 tx=0x7fbf70008040 comp rx=0 tx=0).stop 2024-04-25T11:19:33.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.103+0000 7fbf7f200640 1 -- 172.21.15.7:0/1133787665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf8015eed0 msgr2=0x7fbf8015a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.103+0000 7fbf7f200640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf8015eed0 0x7fbf8015a3b0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009f70 tx=0x7fbf78004a40 comp rx=0 tx=0).stop 2024-04-25T11:19:33.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.104+0000 7fbf7f200640 1 -- 172.21.15.7:0/1133787665 shutdown_connections 2024-04-25T11:19:33.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.104+0000 7fbf7f200640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbf600744b0 0x7fbf600769c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.104+0000 7fbf7f200640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf8015eed0 0x7fbf8015a3b0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.104+0000 7fbf7f200640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf801416a0 0x7fbf801f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.104+0000 7fbf7f200640 1 --2- 172.21.15.7:0/1133787665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf80141290 0x7fbf801f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.104+0000 7fbf7f200640 1 -- 172.21.15.7:0/1133787665 >> 172.21.15.7:0/1133787665 conn(0x7fbf800d1540 msgr2=0x7fbf8006d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.105+0000 7fbf7f200640 1 -- 172.21.15.7:0/1133787665 shutdown_connections 2024-04-25T11:19:33.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.105+0000 7fbf7f200640 1 -- 172.21.15.7:0/1133787665 wait complete. 2024-04-25T11:19:33.113 INFO:tasks.ceph.mds.c.smithi007.stderr: -2403> 2024-04-25T11:19:14.633+0000 7fc13ec00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:19:33.132 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:19:33.244 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:19:33.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.391+0000 7f72a0c00640 1 Processor -- start 2024-04-25T11:19:33.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.391+0000 7f72a0c00640 1 -- start start 2024-04-25T11:19:33.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.393+0000 7f72a0c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f729c0d0480 0x7f729c0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.394+0000 7f72a0c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f729c06eea0 0x7f729c06f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.394+0000 7f72a0c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f729c0ce570 con 0x7f729c1583c0 2024-04-25T11:19:33.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.394+0000 7f72a0c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f729c06f7f0 con 0x7f729c06eea0 2024-04-25T11:19:33.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.394+0000 7f72a0c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f729c15f190 con 0x7f729c0d0480 2024-04-25T11:19:33.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.394+0000 7f729be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f729c06eea0 0x7f729c06f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.394+0000 7f729aa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f729c0d0480 0x7f729c0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.394+0000 7f729be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f729c06eea0 0x7f729c06f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:44120/0 (socket says 172.21.15.7:44120) 2024-04-25T11:19:33.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.394+0000 7f729be00640 1 -- 172.21.15.7:0/1224430151 learned_addr learned my addr 172.21.15.7:0/1224430151 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.395+0000 7f729be00640 1 -- 172.21.15.7:0/1224430151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f729c0d0480 msgr2=0x7f729c0d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.395+0000 7f729be00640 1 --2- 172.21.15.7:0/1224430151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f729c0d0480 0x7f729c0d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.395+0000 7f729be00640 1 -- 172.21.15.7:0/1224430151 >> v1:172.21.15.87:6789/0 conn(0x7f729c1583c0 legacy=0x7f729c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.395+0000 7f729be00640 1 -- 172.21.15.7:0/1224430151 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c0ce850 con 0x7f729c06eea0 2024-04-25T11:19:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.395+0000 7f729aa00640 1 --2- 172.21.15.7:0/1224430151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f729c0d0480 0x7f729c0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.397+0000 7f729be00640 1 --2- 172.21.15.7:0/1224430151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f729c06eea0 0x7f729c06f2b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7294009e50 tx=0x7f729402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e54fa43e0a2dd13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.397+0000 7f729a000640 1 -- 172.21.15.7:0/1224430151 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294036070 con 0x7f729c06eea0 2024-04-25T11:19:33.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.397+0000 7f729a000640 1 -- 172.21.15.7:0/1224430151 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7294031600 con 0x7f729c06eea0 2024-04-25T11:19:33.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.397+0000 7f729a000640 1 -- 172.21.15.7:0/1224430151 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940319d0 con 0x7f729c06eea0 2024-04-25T11:19:33.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.398+0000 7f72a0c00640 1 -- 172.21.15.7:0/1224430151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f729c06eea0 msgr2=0x7f729c06f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.398+0000 7f72a0c00640 1 --2- 172.21.15.7:0/1224430151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f729c06eea0 0x7f729c06f2b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7294009e50 tx=0x7f729402a710 comp rx=0 tx=0).stop 2024-04-25T11:19:33.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.398+0000 7f72a0c00640 1 -- 172.21.15.7:0/1224430151 shutdown_connections 2024-04-25T11:19:33.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.398+0000 7f72a0c00640 1 --2- 172.21.15.7:0/1224430151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f729c06eea0 0x7f729c06f2b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.398+0000 7f72a0c00640 1 --2- 172.21.15.7:0/1224430151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f729c0d0480 0x7f729c0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.398+0000 7f72a0c00640 1 -- 172.21.15.7:0/1224430151 >> 172.21.15.7:0/1224430151 conn(0x7f729c0740f0 msgr2=0x7f729c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:33.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.398+0000 7f72a0c00640 1 -- 172.21.15.7:0/1224430151 shutdown_connections 2024-04-25T11:19:33.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.398+0000 7f72a0c00640 1 -- 172.21.15.7:0/1224430151 wait complete. 2024-04-25T11:19:33.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.400+0000 7f72a0c00640 1 Processor -- start 2024-04-25T11:19:33.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.400+0000 7f72a0c00640 1 -- start start 2024-04-25T11:19:33.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.402+0000 7f72a0c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f729c06eea0 0x7f729c141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.402+0000 7f72a0c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f729c0d0480 0x7f729c1e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.402+0000 7f72a0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f729c1583c0 0x7f729c1e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.402+0000 7f72a0c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f729c0cf0f0 con 0x7f729c1583c0 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.402+0000 7f72a0c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f729c0cee10 con 0x7f729c0d0480 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.402+0000 7f72a0c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f729c0cfd20 con 0x7f729c06eea0 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.402+0000 7f729be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f729c1583c0 0x7f729c1e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.402+0000 7f729be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f729c1583c0 0x7f729c1e7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47190/0 (socket says 172.21.15.7:47190) 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.402+0000 7f729be00640 1 -- 172.21.15.7:0/3048713763 learned_addr learned my addr 172.21.15.7:0/3048713763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.402+0000 7f729b400640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f729c06eea0 0x7f729c141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.403+0000 7f729aa00640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f729c0d0480 0x7f729c1e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.403+0000 7f729be00640 1 -- 172.21.15.7:0/3048713763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f729c06eea0 msgr2=0x7f729c141050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.403+0000 7f729be00640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f729c06eea0 0x7f729c141050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.403+0000 7f729be00640 1 -- 172.21.15.7:0/3048713763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f729c0d0480 msgr2=0x7f729c1e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.403+0000 7f729be00640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f729c0d0480 0x7f729c1e77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.403+0000 7f729be00640 1 -- 172.21.15.7:0/3048713763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c1e85b0 con 0x7f729c1583c0 2024-04-25T11:19:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.403+0000 7f729be00640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f729c1583c0 0x7f729c1e7d20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7294009f70 tx=0x7f7294004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.403+0000 7f7293400640 1 -- 172.21.15.7:0/3048713763 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294036050 con 0x7f729c1583c0 2024-04-25T11:19:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.403+0000 7f7293400640 1 -- 172.21.15.7:0/3048713763 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7294042070 con 0x7f729c1583c0 2024-04-25T11:19:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.403+0000 7f7293400640 1 -- 172.21.15.7:0/3048713763 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294009510 con 0x7f729c1583c0 2024-04-25T11:19:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.404+0000 7f72a0c00640 1 -- 172.21.15.7:0/3048713763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729c158900 con 0x7f729c1583c0 2024-04-25T11:19:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.404+0000 7f72a0c00640 1 -- 172.21.15.7:0/3048713763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f729c158ec0 con 0x7f729c1583c0 2024-04-25T11:19:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.405+0000 7f72a0c00640 1 -- 172.21.15.7:0/3048713763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f729c1e8320 con 0x7f729c1583c0 2024-04-25T11:19:33.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.406+0000 7f7293400640 1 -- 172.21.15.7:0/3048713763 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7294007c50 con 0x7f729c1583c0 2024-04-25T11:19:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.407+0000 7f7293400640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f727c0744e0 0x7f727c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.407+0000 7f7293400640 1 -- 172.21.15.7:0/3048713763 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6187+0+0 (secure 0 0 0) 0x7f72940b6280 con 0x7f729c1583c0 2024-04-25T11:19:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.407+0000 7f729b400640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f727c0744e0 0x7f727c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:33.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.408+0000 7f729b400640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f727c0744e0 0x7f727c0769f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f728c007fd0 tx=0x7f728c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:33.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.412+0000 7f7293400640 1 -- 172.21.15.7:0/3048713763 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7294080820 con 0x7f729c1583c0 2024-04-25T11:19:33.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:33.646+0000 7f72a0c00640 1 -- 172.21.15.7:0/3048713763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f729c050740 con 0x7f729c1583c0 2024-04-25T11:19:34.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.105+0000 7f7293400640 1 -- 172.21.15.7:0/3048713763 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7f7294080120 con 0x7f729c1583c0 2024-04-25T11:19:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.108+0000 7f72a0c00640 1 -- 172.21.15.7:0/3048713763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f727c0744e0 msgr2=0x7f727c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.108+0000 7f72a0c00640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f727c0744e0 0x7f727c0769f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f728c007fd0 tx=0x7f728c009040 comp rx=0 tx=0).stop 2024-04-25T11:19:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.108+0000 7f72a0c00640 1 -- 172.21.15.7:0/3048713763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f729c1583c0 msgr2=0x7f729c1e7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.108+0000 7f72a0c00640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f729c1583c0 0x7f729c1e7d20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f7294009f70 tx=0x7f7294004a40 comp rx=0 tx=0).stop 2024-04-25T11:19:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.108+0000 7f72a0c00640 1 -- 172.21.15.7:0/3048713763 shutdown_connections 2024-04-25T11:19:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.108+0000 7f72a0c00640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f727c0744e0 0x7f727c0769f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.108+0000 7f72a0c00640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f729c1583c0 0x7f729c1e7d20 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.108+0000 7f72a0c00640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f729c0d0480 0x7f729c1e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.108+0000 7f72a0c00640 1 --2- 172.21.15.7:0/3048713763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f729c06eea0 0x7f729c141050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.108+0000 7f72a0c00640 1 -- 172.21.15.7:0/3048713763 >> 172.21.15.7:0/3048713763 conn(0x7f729c0740f0 msgr2=0x7f729c152f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.109+0000 7f72a0c00640 1 -- 172.21.15.7:0/3048713763 shutdown_connections 2024-04-25T11:19:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.109+0000 7f72a0c00640 1 -- 172.21.15.7:0/3048713763 wait complete. 2024-04-25T11:19:34.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:19:34.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.429+0000 7fa823200640 1 Processor -- start 2024-04-25T11:19:34.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.430+0000 7fa823200640 1 -- start start 2024-04-25T11:19:34.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.432+0000 7fa823200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa8241603e0 0x7fa824168980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:34.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.432+0000 7fa823200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8240b4630 con 0x7fa8241610d0 2024-04-25T11:19:34.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.432+0000 7fa823200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8240b48d0 con 0x7fa82406d580 2024-04-25T11:19:34.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.432+0000 7fa823200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa8240b4bb0 con 0x7fa8241603e0 2024-04-25T11:19:34.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.432+0000 7fa821e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa8241610d0 0x7fa8240b5b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50558/0 (socket says 172.21.15.7:50558) 2024-04-25T11:19:34.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.432+0000 7fa821e00640 1 -- 172.21.15.7:0/2690489676 learned_addr learned my addr 172.21.15.7:0/2690489676 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:34.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.433+0000 7fa822800640 1 --2- 172.21.15.7:0/2690489676 >> v2:172.21.15.87:3300/0 conn(0x7fa8241603e0 0x7fa824168980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:34.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.433+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3239147876 0 0) 0x7fa8240b4630 con 0x7fa8241610d0 2024-04-25T11:19:34.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.433+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa808003520 con 0x7fa8241610d0 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.433+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3809586779 0 0) 0x7fa808003520 con 0x7fa8241610d0 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.433+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 >> v2:172.21.15.87:3300/0 conn(0x7fa8241603e0 msgr2=0x7fa824168980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.433+0000 7fa821400640 1 --2- 172.21.15.7:0/2690489676 >> v2:172.21.15.87:3300/0 conn(0x7fa8241603e0 0x7fa824168980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.433+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 >> v1:172.21.15.87:6789/0 conn(0x7fa82406d580 legacy=0x7fa82406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.433+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8240b4e90 con 0x7fa8241610d0 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.433+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fa814001670 con 0x7fa8241610d0 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.433+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 >> v1:172.21.15.7:6789/0 conn(0x7fa8241610d0 legacy=0x7fa8240b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa821400640 1 --2- 172.21.15.7:0/2690489676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa808003a00 0x7fa808003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa821400640 1 --2- 172.21.15.7:0/2690489676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa808004800 0x7fa808004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa821400640 1 --2- 172.21.15.7:0/2690489676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa808005270 0x7fa80800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8240b4bb0 con 0x7fa808005270 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa80800c190 con 0x7fa808003a00 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa80800c390 con 0x7fa808004800 2024-04-25T11:19:34.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa822800640 1 --2- 172.21.15.7:0/2690489676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa808005270 0x7fa80800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:34.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa821e00640 1 --2- 172.21.15.7:0/2690489676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa808003a00 0x7fa808003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:34.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa81be00640 1 --2- 172.21.15.7:0/2690489676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa808004800 0x7fa808004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:34.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa822800640 1 -- 172.21.15.7:0/2690489676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa808004800 msgr2=0x7fa808004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:34.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa822800640 1 --2- 172.21.15.7:0/2690489676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa808004800 0x7fa808004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:34.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa822800640 1 -- 172.21.15.7:0/2690489676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa808003a00 msgr2=0x7fa808003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:34.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa822800640 1 --2- 172.21.15.7:0/2690489676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa808003a00 0x7fa808003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:34.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.434+0000 7fa822800640 1 -- 172.21.15.7:0/2690489676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa808003520 con 0x7fa808005270 2024-04-25T11:19:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.435+0000 7fa822800640 1 -- 172.21.15.7:0/2690489676 reap_dead start 2024-04-25T11:19:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.436+0000 7fa822800640 1 --2- 172.21.15.7:0/2690489676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa808005270 0x7fa80800b9c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009f90 tx=0x7fa808005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9c68a63ac3d094 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.436+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c035020 con 0x7fa808005270 2024-04-25T11:19:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.436+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa81c003b90 con 0x7fa808005270 2024-04-25T11:19:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.436+0000 7fa821400640 1 -- 172.21.15.7:0/2690489676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c009c10 con 0x7fa808005270 2024-04-25T11:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.436+0000 7fa823200640 1 -- 172.21.15.7:0/2690489676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa808005270 msgr2=0x7fa80800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.436+0000 7fa823200640 1 --2- 172.21.15.7:0/2690489676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa808005270 0x7fa80800b9c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009f90 tx=0x7fa808005830 comp rx=0 tx=0).stop 2024-04-25T11:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.437+0000 7fa823200640 1 -- 172.21.15.7:0/2690489676 shutdown_connections 2024-04-25T11:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.437+0000 7fa823200640 1 --2- 172.21.15.7:0/2690489676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa808005270 0x7fa80800b9c0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.437+0000 7fa823200640 1 -- 172.21.15.7:0/2690489676 >> 172.21.15.7:0/2690489676 conn(0x7fa824073160 msgr2=0x7fa824073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.437+0000 7fa823200640 1 -- 172.21.15.7:0/2690489676 shutdown_connections 2024-04-25T11:19:34.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.437+0000 7fa823200640 1 -- 172.21.15.7:0/2690489676 wait complete. 2024-04-25T11:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.439+0000 7fa823200640 1 Processor -- start 2024-04-25T11:19:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.439+0000 7fa823200640 1 -- start start 2024-04-25T11:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa823200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa808005270 0x7fa82413cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa823200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8241440a0 0x7fa824159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:34.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa823200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8240ae3b0 0x7fa8240ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa823200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa82415f0d0 con 0x7fa8241440a0 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa823200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241614e0 con 0x7fa8240ae3b0 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa823200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa82406d990 con 0x7fa808005270 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa81be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8241440a0 0x7fa824159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa81be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8241440a0 0x7fa824159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47204/0 (socket says 172.21.15.7:47204) 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa81be00640 1 -- 172.21.15.7:0/2115657761 learned_addr learned my addr 172.21.15.7:0/2115657761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa821e00640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa808005270 0x7fa82413cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa822800640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8240ae3b0 0x7fa8240ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa81be00640 1 -- 172.21.15.7:0/2115657761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa808005270 msgr2=0x7fa82413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa81be00640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa808005270 0x7fa82413cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa81be00640 1 -- 172.21.15.7:0/2115657761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8240ae3b0 msgr2=0x7fa8240ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.441+0000 7fa81be00640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8240ae3b0 0x7fa8240ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.442+0000 7fa81be00640 1 -- 172.21.15.7:0/2115657761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8240aed00 con 0x7fa8241440a0 2024-04-25T11:19:34.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.442+0000 7fa81be00640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8241440a0 0x7fa824159ea0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00e410 tx=0x7fa80c00e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.442+0000 7fa81a000640 1 -- 172.21.15.7:0/2115657761 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c019070 con 0x7fa8241440a0 2024-04-25T11:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.442+0000 7fa81a000640 1 -- 172.21.15.7:0/2115657761 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80c0052e0 con 0x7fa8241440a0 2024-04-25T11:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.442+0000 7fa81a000640 1 -- 172.21.15.7:0/2115657761 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0037f0 con 0x7fa8241440a0 2024-04-25T11:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.442+0000 7fa823200640 1 -- 172.21.15.7:0/2115657761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82415a7f0 con 0x7fa8241440a0 2024-04-25T11:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.442+0000 7fa823200640 1 -- 172.21.15.7:0/2115657761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82415adb0 con 0x7fa8241440a0 2024-04-25T11:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.444+0000 7fa823200640 1 -- 172.21.15.7:0/2115657761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa824050740 con 0x7fa8241440a0 2024-04-25T11:19:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.445+0000 7fa81a000640 1 -- 172.21.15.7:0/2115657761 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa80c01c070 con 0x7fa8241440a0 2024-04-25T11:19:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.446+0000 7fa81a000640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8000744b0 0x7fa8000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.446+0000 7fa81a000640 1 -- 172.21.15.7:0/2115657761 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6187+0+0 (secure 0 0 0) 0x7fa80c0948b0 con 0x7fa8241440a0 2024-04-25T11:19:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.446+0000 7fa821e00640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8000744b0 0x7fa8000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.447+0000 7fa821e00640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8000744b0 0x7fa8000769c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa814000c00 tx=0x7fa814009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.451+0000 7fa81a000640 1 -- 172.21.15.7:0/2115657761 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa8240aed00 con 0x7fa8241440a0 2024-04-25T11:19:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:34.657+0000 7fa823200640 1 -- 172.21.15.7:0/2115657761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa82415b470 con 0x7fa8241440a0 2024-04-25T11:19:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.108+0000 7fa81a000640 1 -- 172.21.15.7:0/2115657761 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7fa80c05e890 con 0x7fa8241440a0 2024-04-25T11:19:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:19:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.112+0000 7fa823200640 1 -- 172.21.15.7:0/2115657761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8000744b0 msgr2=0x7fa8000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.112+0000 7fa823200640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8000744b0 0x7fa8000769c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa814000c00 tx=0x7fa814009040 comp rx=0 tx=0).stop 2024-04-25T11:19:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.112+0000 7fa823200640 1 -- 172.21.15.7:0/2115657761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8241440a0 msgr2=0x7fa824159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.112+0000 7fa823200640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8241440a0 0x7fa824159ea0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00e410 tx=0x7fa80c00e8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.112+0000 7fa823200640 1 -- 172.21.15.7:0/2115657761 shutdown_connections 2024-04-25T11:19:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.112+0000 7fa823200640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8000744b0 0x7fa8000769c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.112+0000 7fa823200640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa8240ae3b0 0x7fa8240ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.112+0000 7fa823200640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8241440a0 0x7fa824159ea0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.112+0000 7fa823200640 1 --2- 172.21.15.7:0/2115657761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa808005270 0x7fa82413cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.112+0000 7fa823200640 1 -- 172.21.15.7:0/2115657761 >> 172.21.15.7:0/2115657761 conn(0x7fa824073160 msgr2=0x7fa824070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.113+0000 7fa823200640 1 -- 172.21.15.7:0/2115657761 shutdown_connections 2024-04-25T11:19:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.113+0000 7fa823200640 1 -- 172.21.15.7:0/2115657761 wait complete. 2024-04-25T11:19:35.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:19:35.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.425+0000 7f9de9c00640 1 Processor -- start 2024-04-25T11:19:35.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.425+0000 7f9de9c00640 1 -- start start 2024-04-25T11:19:35.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.427+0000 7f9de9c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9de40b48a0 0x7f9de415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.427+0000 7f9de9c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9de40aec60 0x7f9de40b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.427+0000 7f9de9c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de40b4da0 con 0x7f9de40b0940 2024-04-25T11:19:35.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.427+0000 7f9de9c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9de40b5080 con 0x7f9de40aec60 2024-04-25T11:19:35.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.427+0000 7f9de9c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9de40b5360 con 0x7f9de40b48a0 2024-04-25T11:19:35.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.428+0000 7f9ddbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9de40aec60 0x7f9de40b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.428+0000 7f9ddbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9de40aec60 0x7f9de40b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47206/0 (socket says 172.21.15.7:47206) 2024-04-25T11:19:35.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.428+0000 7f9ddbe00640 1 -- 172.21.15.7:0/1462874326 learned_addr learned my addr 172.21.15.7:0/1462874326 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:35.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.428+0000 7f9de2a00640 1 --2- 172.21.15.7:0/1462874326 >> v2:172.21.15.87:3300/0 conn(0x7f9de40b48a0 0x7f9de415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.428+0000 7f9ddbe00640 1 -- 172.21.15.7:0/1462874326 >> v2:172.21.15.87:3300/0 conn(0x7f9de40b48a0 msgr2=0x7f9de415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:35.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.428+0000 7f9ddbe00640 1 --2- 172.21.15.7:0/1462874326 >> v2:172.21.15.87:3300/0 conn(0x7f9de40b48a0 0x7f9de415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.428+0000 7f9ddbe00640 1 -- 172.21.15.7:0/1462874326 >> v1:172.21.15.7:6789/0 conn(0x7f9de40b0940 legacy=0x7f9de40b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.428+0000 7f9ddbe00640 1 -- 172.21.15.7:0/1462874326 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de40b5640 con 0x7f9de40aec60 2024-04-25T11:19:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.429+0000 7f9ddbe00640 1 --2- 172.21.15.7:0/1462874326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de40aec60 0x7f9de40b5e40 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9de4071390 tx=0x7f9dcc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e812d878bd00c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.430+0000 7f9de2000640 1 -- 172.21.15.7:0/1462874326 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc037070 con 0x7f9de40aec60 2024-04-25T11:19:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.430+0000 7f9de2000640 1 -- 172.21.15.7:0/1462874326 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc032440 con 0x7f9de40aec60 2024-04-25T11:19:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.430+0000 7f9de2000640 1 -- 172.21.15.7:0/1462874326 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc032810 con 0x7f9de40aec60 2024-04-25T11:19:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.431+0000 7f9de9c00640 1 -- 172.21.15.7:0/1462874326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de40aec60 msgr2=0x7f9de40b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.431+0000 7f9de9c00640 1 --2- 172.21.15.7:0/1462874326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de40aec60 0x7f9de40b5e40 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9de4071390 tx=0x7f9dcc02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.431+0000 7f9de9c00640 1 -- 172.21.15.7:0/1462874326 shutdown_connections 2024-04-25T11:19:35.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.431+0000 7f9de9c00640 1 --2- 172.21.15.7:0/1462874326 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de40aec60 0x7f9de40b5e40 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.431+0000 7f9de9c00640 1 --2- 172.21.15.7:0/1462874326 >> v2:172.21.15.87:3300/0 conn(0x7f9de40b48a0 0x7f9de415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.431+0000 7f9de9c00640 1 -- 172.21.15.7:0/1462874326 >> 172.21.15.7:0/1462874326 conn(0x7f9de4073920 msgr2=0x7f9de4073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:35.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.431+0000 7f9de9c00640 1 -- 172.21.15.7:0/1462874326 shutdown_connections 2024-04-25T11:19:35.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.431+0000 7f9de9c00640 1 -- 172.21.15.7:0/1462874326 wait complete. 2024-04-25T11:19:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.433+0000 7f9de9c00640 1 Processor -- start 2024-04-25T11:19:35.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.433+0000 7f9de9c00640 1 -- start start 2024-04-25T11:19:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.435+0000 7f9de9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de40aec60 0x7f9de4160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.435+0000 7f9de9c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9de40b0940 0x7f9de40cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.435+0000 7f9de9c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9de40b48a0 0x7f9de41458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.435+0000 7f9de9c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9de40b6380 con 0x7f9de40aec60 2024-04-25T11:19:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.435+0000 7f9de9c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9de415e1d0 con 0x7f9de40b48a0 2024-04-25T11:19:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.435+0000 7f9de9c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9de40b03b0 con 0x7f9de40b0940 2024-04-25T11:19:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.435+0000 7f9de2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de40aec60 0x7f9de4160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.435+0000 7f9de2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de40aec60 0x7f9de4160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47216/0 (socket says 172.21.15.7:47216) 2024-04-25T11:19:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.435+0000 7f9de2a00640 1 -- 172.21.15.7:0/4116428581 learned_addr learned my addr 172.21.15.7:0/4116428581 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9ddbe00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9de40b0940 0x7f9de40cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9de3400640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9de40b48a0 0x7f9de41458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9de2a00640 1 -- 172.21.15.7:0/4116428581 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9de40b0940 msgr2=0x7f9de40cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9de2a00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9de40b0940 0x7f9de40cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9de2a00640 1 -- 172.21.15.7:0/4116428581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9de40b48a0 msgr2=0x7f9de41458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9de2a00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9de40b48a0 0x7f9de41458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9de2a00640 1 -- 172.21.15.7:0/4116428581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de415fe80 con 0x7f9de40aec60 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9ddbe00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9de40b0940 0x7f9de40cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9de3400640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9de40b48a0 0x7f9de41458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9de2a00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de40aec60 0x7f9de4160f20 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400d600 tx=0x7f9dd400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9ddaa00640 1 -- 172.21.15.7:0/4116428581 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4004240 con 0x7f9de40aec60 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9ddaa00640 1 -- 172.21.15.7:0/4116428581 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd40043e0 con 0x7f9de40aec60 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.436+0000 7f9ddaa00640 1 -- 172.21.15.7:0/4116428581 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4003830 con 0x7f9de40aec60 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.437+0000 7f9de9c00640 1 -- 172.21.15.7:0/4116428581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de4160160 con 0x7f9de40aec60 2024-04-25T11:19:35.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.437+0000 7f9de9c00640 1 -- 172.21.15.7:0/4116428581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de4160720 con 0x7f9de40aec60 2024-04-25T11:19:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.438+0000 7f9de9c00640 1 -- 172.21.15.7:0/4116428581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de406a100 con 0x7f9de40aec60 2024-04-25T11:19:35.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.439+0000 7f9ddaa00640 1 -- 172.21.15.7:0/4116428581 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9dd40039d0 con 0x7f9de40aec60 2024-04-25T11:19:35.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.440+0000 7f9ddaa00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9dc00744b0 0x7f9dc00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:35.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.440+0000 7f9ddaa00640 1 -- 172.21.15.7:0/4116428581 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5749+0+0 (secure 0 0 0) 0x7f9dd40955e0 con 0x7f9de40aec60 2024-04-25T11:19:35.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.440+0000 7f9ddbe00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9dc00744b0 0x7f9dc00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:35.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.441+0000 7f9ddbe00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9dc00744b0 0x7f9dc00769c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc034000 tx=0x7f9dcc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:35.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.445+0000 7f9ddaa00640 1 -- 172.21.15.7:0/4116428581 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9dd405fca0 con 0x7f9de40aec60 2024-04-25T11:19:35.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:35.650+0000 7f9de9c00640 1 -- 172.21.15.7:0/4116428581 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9de40b68d0 con 0x7f9de40aec60 2024-04-25T11:19:36.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.111+0000 7f9ddaa00640 1 -- 172.21.15.7:0/4116428581 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7f9dd405f5a0 con 0x7f9de40aec60 2024-04-25T11:19:36.110 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-25T11:19:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.115+0000 7f9de9c00640 1 -- 172.21.15.7:0/4116428581 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9dc00744b0 msgr2=0x7f9dc00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.115+0000 7f9de9c00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9dc00744b0 0x7f9dc00769c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc034000 tx=0x7f9dcc02e000 comp rx=0 tx=0).stop 2024-04-25T11:19:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.115+0000 7f9de9c00640 1 -- 172.21.15.7:0/4116428581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de40aec60 msgr2=0x7f9de4160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.115+0000 7f9de9c00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de40aec60 0x7f9de4160f20 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400d600 tx=0x7f9dd400dac0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.116+0000 7f9de9c00640 1 -- 172.21.15.7:0/4116428581 shutdown_connections 2024-04-25T11:19:36.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.116+0000 7f9de9c00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9dc00744b0 0x7f9dc00769c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.116+0000 7f9de9c00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9de40b48a0 0x7f9de41458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.116+0000 7f9de9c00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9de40b0940 0x7f9de40cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.116+0000 7f9de9c00640 1 --2- 172.21.15.7:0/4116428581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9de40aec60 0x7f9de4160f20 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.116+0000 7f9de9c00640 1 -- 172.21.15.7:0/4116428581 >> 172.21.15.7:0/4116428581 conn(0x7f9de4073920 msgr2=0x7f9de40d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:36.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.116+0000 7f9de9c00640 1 -- 172.21.15.7:0/4116428581 shutdown_connections 2024-04-25T11:19:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.116+0000 7f9de9c00640 1 -- 172.21.15.7:0/4116428581 wait complete. 2024-04-25T11:19:36.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:19:36.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.424+0000 7f4a13200640 1 Processor -- start 2024-04-25T11:19:36.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.424+0000 7f4a13200640 1 -- start start 2024-04-25T11:19:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.426+0000 7f4a13200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a0c0d1150 0x7f4a0c0bb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.427+0000 7f4a13200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a0c06e710 0x7f4a0c15f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.427+0000 7f4a13200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0c0b8f30 con 0x7f4a0c0d4400 2024-04-25T11:19:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.427+0000 7f4a13200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4a0c0b9260 con 0x7f4a0c0d1150 2024-04-25T11:19:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.427+0000 7f4a13200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0c0d1560 con 0x7f4a0c06e710 2024-04-25T11:19:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.427+0000 7f4a11e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a0c0d1150 0x7f4a0c0bb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.427+0000 7f4a11e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a0c0d1150 0x7f4a0c0bb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47228/0 (socket says 172.21.15.7:47228) 2024-04-25T11:19:36.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.427+0000 7f4a11e00640 1 -- 172.21.15.7:0/49585554 learned_addr learned my addr 172.21.15.7:0/49585554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.428+0000 7f4a12800640 1 --2- 172.21.15.7:0/49585554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a0c06e710 0x7f4a0c15f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.428+0000 7f4a11e00640 1 -- 172.21.15.7:0/49585554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a0c06e710 msgr2=0x7f4a0c15f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.428+0000 7f4a11e00640 1 --2- 172.21.15.7:0/49585554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a0c06e710 0x7f4a0c15f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.428+0000 7f4a11e00640 1 -- 172.21.15.7:0/49585554 >> v1:172.21.15.87:6789/0 conn(0x7f4a0c0d4400 legacy=0x7f4a0c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:36.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.428+0000 7f4a11e00640 1 -- 172.21.15.7:0/49585554 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c0d4be0 con 0x7f4a0c0d1150 2024-04-25T11:19:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.429+0000 7f4a11e00640 1 --2- 172.21.15.7:0/49585554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c0d1150 0x7f4a0c0bb2f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0d31f0 tx=0x7f4a0002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573f88e5bb6ca494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.430+0000 7f4a10a00640 1 -- 172.21.15.7:0/49585554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00037070 con 0x7f4a0c0d1150 2024-04-25T11:19:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.430+0000 7f4a10a00640 1 -- 172.21.15.7:0/49585554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a0002ae50 con 0x7f4a0c0d1150 2024-04-25T11:19:36.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.430+0000 7f4a10a00640 1 -- 172.21.15.7:0/49585554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00032660 con 0x7f4a0c0d1150 2024-04-25T11:19:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.431+0000 7f4a13200640 1 -- 172.21.15.7:0/49585554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c0d1150 msgr2=0x7f4a0c0bb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.431+0000 7f4a13200640 1 --2- 172.21.15.7:0/49585554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c0d1150 0x7f4a0c0bb2f0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0d31f0 tx=0x7f4a0002a140 comp rx=0 tx=0).stop 2024-04-25T11:19:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.431+0000 7f4a13200640 1 -- 172.21.15.7:0/49585554 shutdown_connections 2024-04-25T11:19:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.431+0000 7f4a13200640 1 --2- 172.21.15.7:0/49585554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a0c06e710 0x7f4a0c15f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.431+0000 7f4a13200640 1 --2- 172.21.15.7:0/49585554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c0d1150 0x7f4a0c0bb2f0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.431+0000 7f4a13200640 1 -- 172.21.15.7:0/49585554 >> 172.21.15.7:0/49585554 conn(0x7f4a0c06d170 msgr2=0x7f4a0c0d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.431+0000 7f4a13200640 1 -- 172.21.15.7:0/49585554 shutdown_connections 2024-04-25T11:19:36.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.431+0000 7f4a13200640 1 -- 172.21.15.7:0/49585554 wait complete. 2024-04-25T11:19:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.433+0000 7f4a13200640 1 Processor -- start 2024-04-25T11:19:36.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.433+0000 7f4a13200640 1 -- start start 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.435+0000 7f4a13200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a0c0d1150 0x7f4a0c14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.435+0000 7f4a13200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a0c0d4400 0x7f4a0c0ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.435+0000 7f4a13200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c15db10 0x7f4a0c1f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.435+0000 7f4a13200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c0d4810 con 0x7f4a0c15db10 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.435+0000 7f4a13200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c0d51a0 con 0x7f4a0c0d4400 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.435+0000 7f4a13200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0c0d4990 con 0x7f4a0c0d1150 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.435+0000 7f4a12800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c15db10 0x7f4a0c1f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.435+0000 7f4a12800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c15db10 0x7f4a0c1f9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47232/0 (socket says 172.21.15.7:47232) 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.435+0000 7f4a12800640 1 -- 172.21.15.7:0/1467848097 learned_addr learned my addr 172.21.15.7:0/1467848097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.436+0000 7f4a11e00640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a0c0d1150 0x7f4a0c14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.436+0000 7f4a11400640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a0c0d4400 0x7f4a0c0ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.436+0000 7f4a12800640 1 -- 172.21.15.7:0/1467848097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a0c0d1150 msgr2=0x7f4a0c14dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.436+0000 7f4a12800640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a0c0d1150 0x7f4a0c14dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.436+0000 7f4a12800640 1 -- 172.21.15.7:0/1467848097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a0c0d4400 msgr2=0x7f4a0c0ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.436+0000 7f4a12800640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a0c0d4400 0x7f4a0c0ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.436+0000 7f4a12800640 1 -- 172.21.15.7:0/1467848097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c14e4c0 con 0x7f4a0c15db10 2024-04-25T11:19:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.436+0000 7f4a11400640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a0c0d4400 0x7f4a0c0ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:36.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.436+0000 7f4a12800640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c15db10 0x7f4a0c1f9790 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f4a08007e90 tx=0x7f4a0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.436+0000 7f4a06000640 1 -- 172.21.15.7:0/1467848097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08014420 con 0x7f4a0c15db10 2024-04-25T11:19:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.436+0000 7f4a06000640 1 -- 172.21.15.7:0/1467848097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a08019070 con 0x7f4a0c15db10 2024-04-25T11:19:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.437+0000 7f4a06000640 1 -- 172.21.15.7:0/1467848097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08014800 con 0x7f4a0c15db10 2024-04-25T11:19:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.437+0000 7f4a13200640 1 -- 172.21.15.7:0/1467848097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c14e760 con 0x7f4a0c15db10 2024-04-25T11:19:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.437+0000 7f4a13200640 1 -- 172.21.15.7:0/1467848097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c14ed20 con 0x7f4a0c15db10 2024-04-25T11:19:36.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.438+0000 7f4a13200640 1 -- 172.21.15.7:0/1467848097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c0cb310 con 0x7f4a0c15db10 2024-04-25T11:19:36.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.439+0000 7f4a06000640 1 -- 172.21.15.7:0/1467848097 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4a080149a0 con 0x7f4a0c15db10 2024-04-25T11:19:36.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.441+0000 7f4a06000640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49f00744b0 0x7f49f00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.441+0000 7f4a06000640 1 -- 172.21.15.7:0/1467848097 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7f4a08095a30 con 0x7f4a0c15db10 2024-04-25T11:19:36.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.441+0000 7f4a11e00640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49f00744b0 0x7f49f00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.442+0000 7f4a11e00640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49f00744b0 0x7f49f00769c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4a00034000 tx=0x7f4a0002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.446+0000 7f4a06000640 1 -- 172.21.15.7:0/1467848097 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4a0805fc40 con 0x7f4a0c15db10 2024-04-25T11:19:36.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.651+0000 7f4a13200640 1 -- 172.21.15.7:0/1467848097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f4a0c050740 con 0x7f4a0c15db10 2024-04-25T11:19:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.657+0000 7f4a06000640 1 -- 172.21.15.7:0/1467848097 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7f4a08063f40 con 0x7f4a0c15db10 2024-04-25T11:19:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.660+0000 7f4a13200640 1 -- 172.21.15.7:0/1467848097 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49f00744b0 msgr2=0x7f49f00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.660+0000 7f4a13200640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49f00744b0 0x7f49f00769c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4a00034000 tx=0x7f4a0002e000 comp rx=0 tx=0).stop 2024-04-25T11:19:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.660+0000 7f4a13200640 1 -- 172.21.15.7:0/1467848097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c15db10 msgr2=0x7f4a0c1f9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.660+0000 7f4a13200640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c15db10 0x7f4a0c1f9790 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f4a08007e90 tx=0x7f4a0800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.660+0000 7f4a13200640 1 -- 172.21.15.7:0/1467848097 shutdown_connections 2024-04-25T11:19:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.660+0000 7f4a13200640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49f00744b0 0x7f49f00769c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.660+0000 7f4a13200640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a0c15db10 0x7f4a0c1f9790 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.660+0000 7f4a13200640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a0c0d4400 0x7f4a0c0ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.660+0000 7f4a13200640 1 --2- 172.21.15.7:0/1467848097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a0c0d1150 0x7f4a0c14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.660+0000 7f4a13200640 1 -- 172.21.15.7:0/1467848097 >> 172.21.15.7:0/1467848097 conn(0x7f4a0c06d170 msgr2=0x7f4a0c0b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.661+0000 7f4a13200640 1 -- 172.21.15.7:0/1467848097 shutdown_connections 2024-04-25T11:19:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.661+0000 7f4a13200640 1 -- 172.21.15.7:0/1467848097 wait complete. 2024-04-25T11:19:36.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid' 2024-04-25T11:19:36.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.974+0000 7f5105400640 1 Processor -- start 2024-04-25T11:19:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.975+0000 7f5105400640 1 -- start start 2024-04-25T11:19:36.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.977+0000 7f5105400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f510006dc90 0x7f5100070390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.977+0000 7f5105400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51000708d0 0x7f51000b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.977+0000 7f5105400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5100165b20 con 0x7f5100162330 2024-04-25T11:19:36.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.977+0000 7f5105400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f510006f2f0 con 0x7f510006dc90 2024-04-25T11:19:36.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.977+0000 7f5105400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f510006f5d0 con 0x7f51000708d0 2024-04-25T11:19:36.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.978+0000 7f50ff200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51000708d0 0x7f51000b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.978+0000 7f50f5e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f510006dc90 0x7f5100070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.978+0000 7f50ff200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51000708d0 0x7f51000b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:52726/0 (socket says 172.21.15.7:52726) 2024-04-25T11:19:36.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.978+0000 7f50fe800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f5100162330 0x7f510006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:32774/0 (socket says 172.21.15.7:32774) 2024-04-25T11:19:36.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.978+0000 7f50ff200640 1 -- 172.21.15.7:0/1475028571 learned_addr learned my addr 172.21.15.7:0/1475028571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:36.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.979+0000 7f50ff200640 1 -- 172.21.15.7:0/1475028571 >> v2:172.21.15.87:3300/0 conn(0x7f510006dc90 msgr2=0x7f5100070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:36.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.979+0000 7f50ff200640 1 --2- 172.21.15.7:0/1475028571 >> v2:172.21.15.87:3300/0 conn(0x7f510006dc90 0x7f5100070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.979+0000 7f50ff200640 1 -- 172.21.15.7:0/1475028571 >> v1:172.21.15.87:6789/0 conn(0x7f5100162330 legacy=0x7f510006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.979+0000 7f50ff200640 1 -- 172.21.15.7:0/1475028571 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510006f8b0 con 0x7f51000708d0 2024-04-25T11:19:36.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.981+0000 7f50ff200640 1 --2- 172.21.15.7:0/1475028571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51000708d0 0x7f51000b5940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009e50 tx=0x7f50f802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0f1c0717b8bec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.981+0000 7f50fde00640 1 -- 172.21.15.7:0/1475028571 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8036070 con 0x7f51000708d0 2024-04-25T11:19:36.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.981+0000 7f50fde00640 1 -- 172.21.15.7:0/1475028571 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f8031600 con 0x7f51000708d0 2024-04-25T11:19:36.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.981+0000 7f50fde00640 1 -- 172.21.15.7:0/1475028571 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f80319d0 con 0x7f51000708d0 2024-04-25T11:19:36.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.982+0000 7f5105400640 1 -- 172.21.15.7:0/1475028571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51000708d0 msgr2=0x7f51000b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.982+0000 7f5105400640 1 --2- 172.21.15.7:0/1475028571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51000708d0 0x7f51000b5940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009e50 tx=0x7f50f802a710 comp rx=0 tx=0).stop 2024-04-25T11:19:36.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.982+0000 7f5105400640 1 -- 172.21.15.7:0/1475028571 shutdown_connections 2024-04-25T11:19:36.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.982+0000 7f5105400640 1 --2- 172.21.15.7:0/1475028571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51000708d0 0x7f51000b5940 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.982+0000 7f5105400640 1 --2- 172.21.15.7:0/1475028571 >> v2:172.21.15.87:3300/0 conn(0x7f510006dc90 0x7f5100070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.982+0000 7f5105400640 1 -- 172.21.15.7:0/1475028571 >> 172.21.15.7:0/1475028571 conn(0x7f51000732b0 msgr2=0x7f51000736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:36.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.982+0000 7f5105400640 1 -- 172.21.15.7:0/1475028571 shutdown_connections 2024-04-25T11:19:36.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.982+0000 7f5105400640 1 -- 172.21.15.7:0/1475028571 wait complete. 2024-04-25T11:19:36.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.984+0000 7f5105400640 1 Processor -- start 2024-04-25T11:19:36.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.985+0000 7f5105400640 1 -- start start 2024-04-25T11:19:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.986+0000 7f5105400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51000708d0 0x7f51000d0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.986+0000 7f5105400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5100162330 0x7f510014a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.986+0000 7f5105400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510009b540 0x7f510009d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.986+0000 7f5105400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5100070e60 con 0x7f510009b540 2024-04-25T11:19:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.986+0000 7f5105400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5100070ce0 con 0x7f51000708d0 2024-04-25T11:19:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.986+0000 7f5105400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f51000b5e80 con 0x7f5100162330 2024-04-25T11:19:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.987+0000 7f50ff200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510009b540 0x7f510009d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.987+0000 7f50ff200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510009b540 0x7f510009d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47234/0 (socket says 172.21.15.7:47234) 2024-04-25T11:19:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.987+0000 7f50ff200640 1 -- 172.21.15.7:0/595686189 learned_addr learned my addr 172.21.15.7:0/595686189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.987+0000 7f50f5e00640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5100162330 0x7f510014a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.987+0000 7f50fe800640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51000708d0 0x7f51000d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.987+0000 7f50ff200640 1 -- 172.21.15.7:0/595686189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5100162330 msgr2=0x7f510014a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.987+0000 7f50ff200640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5100162330 0x7f510014a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.987+0000 7f50ff200640 1 -- 172.21.15.7:0/595686189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51000708d0 msgr2=0x7f51000d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:36.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.987+0000 7f50ff200640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51000708d0 0x7f51000d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:36.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.987+0000 7f50ff200640 1 -- 172.21.15.7:0/595686189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51000cf8c0 con 0x7f510009b540 2024-04-25T11:19:36.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.988+0000 7f50ff200640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510009b540 0x7f510009d570 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009f70 tx=0x7f50f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.988+0000 7f50f7400640 1 -- 172.21.15.7:0/595686189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8041020 con 0x7f510009b540 2024-04-25T11:19:36.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.988+0000 7f50f7400640 1 -- 172.21.15.7:0/595686189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f8008730 con 0x7f510009b540 2024-04-25T11:19:36.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.988+0000 7f5105400640 1 -- 172.21.15.7:0/595686189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51000cfba0 con 0x7f510009b540 2024-04-25T11:19:36.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.988+0000 7f50f7400640 1 -- 172.21.15.7:0/595686189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8036040 con 0x7f510009b540 2024-04-25T11:19:36.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.988+0000 7f5105400640 1 -- 172.21.15.7:0/595686189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51000d0160 con 0x7f510009b540 2024-04-25T11:19:36.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.989+0000 7f5105400640 1 -- 172.21.15.7:0/595686189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510009bd30 con 0x7f510009b540 2024-04-25T11:19:36.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.991+0000 7f50f7400640 1 -- 172.21.15.7:0/595686189 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f50f8044020 con 0x7f510009b540 2024-04-25T11:19:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.992+0000 7f50f7400640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50dc0744b0 0x7f50dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.992+0000 7f50f7400640 1 -- 172.21.15.7:0/595686189 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7f50f80b5ff0 con 0x7f510009b540 2024-04-25T11:19:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.992+0000 7f50fe800640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50dc0744b0 0x7f50dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:36.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.993+0000 7f50fe800640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50dc0744b0 0x7f50dc0769c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f50f0000f60 tx=0x7f50f0007c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:36.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:36.997+0000 7f50f7400640 1 -- 172.21.15.7:0/595686189 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50f804b020 con 0x7f510009b540 2024-04-25T11:19:37.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:37.274+0000 7f5105400640 1 -- 172.21.15.7:0/595686189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]} v 0) -- 0x7f5100050740 con 0x7f510009b540 2024-04-25T11:19:38.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.143+0000 7f50f7400640 1 -- 172.21.15.7:0/595686189 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_oldest_tid"]}]=0 v471) ==== 150+0+0 (secure 0 0 0) 0x7f50f80808e0 con 0x7f510009b540 2024-04-25T11:19:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.147+0000 7f5105400640 1 -- 172.21.15.7:0/595686189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50dc0744b0 msgr2=0x7f50dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.147+0000 7f5105400640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50dc0744b0 0x7f50dc0769c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f50f0000f60 tx=0x7f50f0007c80 comp rx=0 tx=0).stop 2024-04-25T11:19:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.147+0000 7f5105400640 1 -- 172.21.15.7:0/595686189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510009b540 msgr2=0x7f510009d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.147+0000 7f5105400640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510009b540 0x7f510009d570 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009f70 tx=0x7f50f8004a40 comp rx=0 tx=0).stop 2024-04-25T11:19:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.147+0000 7f5105400640 1 -- 172.21.15.7:0/595686189 shutdown_connections 2024-04-25T11:19:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.147+0000 7f5105400640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50dc0744b0 0x7f50dc0769c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.147+0000 7f5105400640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f510009b540 0x7f510009d570 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.147+0000 7f5105400640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5100162330 0x7f510014a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.147+0000 7f5105400640 1 --2- 172.21.15.7:0/595686189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51000708d0 0x7f51000d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.147+0000 7f5105400640 1 -- 172.21.15.7:0/595686189 >> 172.21.15.7:0/595686189 conn(0x7f51000732b0 msgr2=0x7f5100166c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.148+0000 7f5105400640 1 -- 172.21.15.7:0/595686189 shutdown_connections 2024-04-25T11:19:38.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.148+0000 7f5105400640 1 -- 172.21.15.7:0/595686189 wait complete. 2024-04-25T11:19:38.173 INFO:tasks.cephfs_test_runner:When a client does not advance its oldest tid, the MDS should notice that ... skipped 'Require FUSE client to inject client release failure' 2024-04-25T11:19:38.174 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-25T11:19:38.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.457+0000 7fa966a00640 1 Processor -- start 2024-04-25T11:19:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.457+0000 7fa966a00640 1 -- start start 2024-04-25T11:19:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.459+0000 7fa966a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9600b04d0 0x7fa9600b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.459+0000 7fa966a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9600b4a80 0x7fa9600b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.459+0000 7fa966a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96015e190 con 0x7fa960070a70 2024-04-25T11:19:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.459+0000 7fa966a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa96015d610 con 0x7fa9600b04d0 2024-04-25T11:19:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.459+0000 7fa966a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa96015d8f0 con 0x7fa9600b4a80 2024-04-25T11:19:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.459+0000 7fa964c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9600b04d0 0x7fa9600b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.459+0000 7fa965600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa960070a70 0x7fa9600ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50572/0 (socket says 172.21.15.7:50572) 2024-04-25T11:19:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.459+0000 7fa965600640 1 -- 172.21.15.7:0/1328825032 learned_addr learned my addr 172.21.15.7:0/1328825032 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.460+0000 7fa966000640 1 --2- 172.21.15.7:0/1328825032 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9600b4a80 0x7fa9600b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.460+0000 7fa964c00640 1 -- 172.21.15.7:0/1328825032 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9600b4a80 msgr2=0x7fa9600b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.460+0000 7fa964c00640 1 --2- 172.21.15.7:0/1328825032 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9600b4a80 0x7fa9600b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.460+0000 7fa964c00640 1 -- 172.21.15.7:0/1328825032 >> v1:172.21.15.7:6789/0 conn(0x7fa960070a70 legacy=0x7fa9600ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.460+0000 7fa964c00640 1 -- 172.21.15.7:0/1328825032 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96015dbd0 con 0x7fa9600b04d0 2024-04-25T11:19:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.460+0000 7fa95fe00640 1 -- 172.21.15.7:0/1328825032 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2188323167 0 0) 0x7fa96015e190 con 0x7fa960070a70 2024-04-25T11:19:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.461+0000 7fa964c00640 1 --2- 172.21.15.7:0/1328825032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9600b04d0 0x7fa9600b08e0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0097a0 tx=0x7fa94c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127e157f5d72c9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.462+0000 7fa95fe00640 1 -- 172.21.15.7:0/1328825032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c037070 con 0x7fa9600b04d0 2024-04-25T11:19:38.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.462+0000 7fa95fe00640 1 -- 172.21.15.7:0/1328825032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c032440 con 0x7fa9600b04d0 2024-04-25T11:19:38.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.462+0000 7fa95fe00640 1 -- 172.21.15.7:0/1328825032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c032810 con 0x7fa9600b04d0 2024-04-25T11:19:38.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.462+0000 7fa966a00640 1 -- 172.21.15.7:0/1328825032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9600b04d0 msgr2=0x7fa9600b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.462+0000 7fa966a00640 1 --2- 172.21.15.7:0/1328825032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9600b04d0 0x7fa9600b08e0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0097a0 tx=0x7fa94c02a180 comp rx=0 tx=0).stop 2024-04-25T11:19:38.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.463+0000 7fa966a00640 1 -- 172.21.15.7:0/1328825032 shutdown_connections 2024-04-25T11:19:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.463+0000 7fa966a00640 1 --2- 172.21.15.7:0/1328825032 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9600b4a80 0x7fa9600b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.463+0000 7fa966a00640 1 --2- 172.21.15.7:0/1328825032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9600b04d0 0x7fa9600b08e0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.463+0000 7fa966a00640 1 -- 172.21.15.7:0/1328825032 >> 172.21.15.7:0/1328825032 conn(0x7fa9600b3820 msgr2=0x7fa9600b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.463+0000 7fa966a00640 1 -- 172.21.15.7:0/1328825032 shutdown_connections 2024-04-25T11:19:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.463+0000 7fa966a00640 1 -- 172.21.15.7:0/1328825032 wait complete. 2024-04-25T11:19:38.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.465+0000 7fa966a00640 1 Processor -- start 2024-04-25T11:19:38.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.465+0000 7fa966a00640 1 -- start start 2024-04-25T11:19:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.467+0000 7fa966a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa960070a70 0x7fa96009e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.467+0000 7fa966a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9600b04d0 0x7fa96009bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.467+0000 7fa966a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9600b4a80 0x7fa96014b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.467+0000 7fa966a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600b0e20 con 0x7fa960070a70 2024-04-25T11:19:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.467+0000 7fa966a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa9600aef90 con 0x7fa9600b4a80 2024-04-25T11:19:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.467+0000 7fa966a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa96015d170 con 0x7fa9600b04d0 2024-04-25T11:19:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.467+0000 7fa965600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa960070a70 0x7fa96009e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.467+0000 7fa965600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa960070a70 0x7fa96009e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47244/0 (socket says 172.21.15.7:47244) 2024-04-25T11:19:38.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.467+0000 7fa965600640 1 -- 172.21.15.7:0/2530528958 learned_addr learned my addr 172.21.15.7:0/2530528958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa966000640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9600b4a80 0x7fa96014b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa964c00640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9600b04d0 0x7fa96009bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa965600640 1 -- 172.21.15.7:0/2530528958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9600b04d0 msgr2=0x7fa96009bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa965600640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9600b04d0 0x7fa96009bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa965600640 1 -- 172.21.15.7:0/2530528958 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9600b4a80 msgr2=0x7fa96014b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa965600640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9600b4a80 0x7fa96014b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa965600640 1 -- 172.21.15.7:0/2530528958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96014a380 con 0x7fa960070a70 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa965600640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa960070a70 0x7fa96009e7a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa95000cb10 tx=0x7fa95000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa95d600640 1 -- 172.21.15.7:0/2530528958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950017070 con 0x7fa960070a70 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa95d600640 1 -- 172.21.15.7:0/2530528958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa95000ac70 con 0x7fa960070a70 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa966a00640 1 -- 172.21.15.7:0/2530528958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96014a660 con 0x7fa960070a70 2024-04-25T11:19:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa95d600640 1 -- 172.21.15.7:0/2530528958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950012620 con 0x7fa960070a70 2024-04-25T11:19:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.468+0000 7fa966a00640 1 -- 172.21.15.7:0/2530528958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96014ac20 con 0x7fa960070a70 2024-04-25T11:19:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.470+0000 7fa966a00640 1 -- 172.21.15.7:0/2530528958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa960050740 con 0x7fa960070a70 2024-04-25T11:19:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.471+0000 7fa95d600640 1 -- 172.21.15.7:0/2530528958 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa95001a070 con 0x7fa960070a70 2024-04-25T11:19:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.472+0000 7fa95d600640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9400744b0 0x7fa9400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.472+0000 7fa95d600640 1 -- 172.21.15.7:0/2530528958 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7fa950094ec0 con 0x7fa960070a70 2024-04-25T11:19:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.473+0000 7fa964c00640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9400744b0 0x7fa9400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.473+0000 7fa964c00640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9400744b0 0x7fa9400769c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa94c034000 tx=0x7fa94c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:38.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.477+0000 7fa95d600640 1 -- 172.21.15.7:0/2530528958 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa95000ae20 con 0x7fa960070a70 2024-04-25T11:19:38.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.683+0000 7fa966a00640 1 -- 172.21.15.7:0/2530528958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa96015a350 con 0x7fa960070a70 2024-04-25T11:19:38.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.685+0000 7fa95d600640 1 -- 172.21.15.7:0/2530528958 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+13203 (secure 0 0 0) 0x7fa95005f7b0 con 0x7fa960070a70 2024-04-25T11:19:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:38.683 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":109,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:36.108364+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6837/990441706":"2024-04-26T11:19:33.098043+0000","172.21.15.7:6834/990441706":"2024-04-26T11:19:33.098043+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 -- 172.21.15.7:0/2530528958 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9400744b0 msgr2=0x7fa9400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9400744b0 0x7fa9400769c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa94c034000 tx=0x7fa94c02e000 comp rx=0 tx=0).stop 2024-04-25T11:19:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 -- 172.21.15.7:0/2530528958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa960070a70 msgr2=0x7fa96009e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa960070a70 0x7fa96009e7a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa95000cb10 tx=0x7fa95000a060 comp rx=0 tx=0).stop 2024-04-25T11:19:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 -- 172.21.15.7:0/2530528958 shutdown_connections 2024-04-25T11:19:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9400744b0 0x7fa9400769c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9600b4a80 0x7fa96014b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9600b04d0 0x7fa96009bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 --2- 172.21.15.7:0/2530528958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa960070a70 0x7fa96009e7a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 -- 172.21.15.7:0/2530528958 >> 172.21.15.7:0/2530528958 conn(0x7fa9600b3820 msgr2=0x7fa9600684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 -- 172.21.15.7:0/2530528958 shutdown_connections 2024-04-25T11:19:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:38.688+0000 7fa966a00640 1 -- 172.21.15.7:0/2530528958 wait complete. 2024-04-25T11:19:38.716 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:38.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.010+0000 7fb59f200640 1 Processor -- start 2024-04-25T11:19:39.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.010+0000 7fb59f200640 1 -- start start 2024-04-25T11:19:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.012+0000 7fb59f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb5981583c0 0x7fb5980cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.012+0000 7fb59f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb59806eea0 0x7fb59806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.012+0000 7fb59f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5980ce570 con 0x7fb5980cfb50 2024-04-25T11:19:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.012+0000 7fb59f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb59806f7f0 con 0x7fb5981583c0 2024-04-25T11:19:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.012+0000 7fb59f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb59815f190 con 0x7fb59806eea0 2024-04-25T11:19:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.013+0000 7fb59de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb5981583c0 0x7fb5980cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.013+0000 7fb59de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb5981583c0 0x7fb5980cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47248/0 (socket says 172.21.15.7:47248) 2024-04-25T11:19:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.013+0000 7fb59de00640 1 -- 172.21.15.7:0/2220521578 learned_addr learned my addr 172.21.15.7:0/2220521578 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:39.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.013+0000 7fb59e800640 1 --2- 172.21.15.7:0/2220521578 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb59806eea0 0x7fb59806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.013+0000 7fb59de00640 1 -- 172.21.15.7:0/2220521578 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb59806eea0 msgr2=0x7fb59806f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.013+0000 7fb59de00640 1 --2- 172.21.15.7:0/2220521578 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb59806eea0 0x7fb59806f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.013+0000 7fb59de00640 1 -- 172.21.15.7:0/2220521578 >> v1:172.21.15.87:6789/0 conn(0x7fb5980cfb50 legacy=0x7fb5980d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.014+0000 7fb59de00640 1 -- 172.21.15.7:0/2220521578 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5980ce850 con 0x7fb5981583c0 2024-04-25T11:19:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.015+0000 7fb59de00640 1 --2- 172.21.15.7:0/2220521578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5981583c0 0x7fb5980cf610 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb58c009870 tx=0x7fb58c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f281a2a8efb94abd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.015+0000 7fb59ca00640 1 -- 172.21.15.7:0/2220521578 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c037070 con 0x7fb5981583c0 2024-04-25T11:19:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.015+0000 7fb59ca00640 1 -- 172.21.15.7:0/2220521578 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58c032440 con 0x7fb5981583c0 2024-04-25T11:19:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.015+0000 7fb59ca00640 1 -- 172.21.15.7:0/2220521578 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c032810 con 0x7fb5981583c0 2024-04-25T11:19:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.016+0000 7fb59f200640 1 -- 172.21.15.7:0/2220521578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5981583c0 msgr2=0x7fb5980cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.016+0000 7fb59f200640 1 --2- 172.21.15.7:0/2220521578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5981583c0 0x7fb5980cf610 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb58c009870 tx=0x7fb58c02a270 comp rx=0 tx=0).stop 2024-04-25T11:19:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.016+0000 7fb59f200640 1 -- 172.21.15.7:0/2220521578 shutdown_connections 2024-04-25T11:19:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.016+0000 7fb59f200640 1 --2- 172.21.15.7:0/2220521578 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb59806eea0 0x7fb59806f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.016+0000 7fb59f200640 1 --2- 172.21.15.7:0/2220521578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5981583c0 0x7fb5980cf610 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.016+0000 7fb59f200640 1 -- 172.21.15.7:0/2220521578 >> 172.21.15.7:0/2220521578 conn(0x7fb5980740f0 msgr2=0x7fb59806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.016+0000 7fb59f200640 1 -- 172.21.15.7:0/2220521578 shutdown_connections 2024-04-25T11:19:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.016+0000 7fb59f200640 1 -- 172.21.15.7:0/2220521578 wait complete. 2024-04-25T11:19:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.019+0000 7fb59f200640 1 Processor -- start 2024-04-25T11:19:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.019+0000 7fb59f200640 1 -- start start 2024-04-25T11:19:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.021+0000 7fb59f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb59806eea0 0x7fb5981f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.021+0000 7fb59f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5980cfb50 0x7fb5981514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.021+0000 7fb59f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5981583c0 0x7fb59806cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.021+0000 7fb59f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb5980cf0f0 con 0x7fb5981583c0 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.021+0000 7fb59f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb5980cee10 con 0x7fb59806eea0 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.021+0000 7fb59f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb5980d0b90 con 0x7fb5980cfb50 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.021+0000 7fb59e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5981583c0 0x7fb59806cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.021+0000 7fb59e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5981583c0 0x7fb59806cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47260/0 (socket says 172.21.15.7:47260) 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.021+0000 7fb59e800640 1 -- 172.21.15.7:0/3977315115 learned_addr learned my addr 172.21.15.7:0/3977315115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.021+0000 7fb59de00640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb59806eea0 0x7fb5981f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.022+0000 7fb59d400640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5980cfb50 0x7fb5981514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.022+0000 7fb59e800640 1 -- 172.21.15.7:0/3977315115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5980cfb50 msgr2=0x7fb5981514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.022+0000 7fb59e800640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5980cfb50 0x7fb5981514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.022+0000 7fb59e800640 1 -- 172.21.15.7:0/3977315115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb59806eea0 msgr2=0x7fb5981f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.022+0000 7fb59e800640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb59806eea0 0x7fb5981f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.022+0000 7fb59e800640 1 -- 172.21.15.7:0/3977315115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb598151a30 con 0x7fb5981583c0 2024-04-25T11:19:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.022+0000 7fb59de00640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb59806eea0 0x7fb5981f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.022+0000 7fb59e800640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5981583c0 0x7fb59806cee0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb594007e90 tx=0x7fb59400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.022+0000 7fb592000640 1 -- 172.21.15.7:0/3977315115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594014420 con 0x7fb5981583c0 2024-04-25T11:19:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.022+0000 7fb592000640 1 -- 172.21.15.7:0/3977315115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb594019070 con 0x7fb5981583c0 2024-04-25T11:19:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.023+0000 7fb59f200640 1 -- 172.21.15.7:0/3977315115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb598159bb0 con 0x7fb5981583c0 2024-04-25T11:19:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.023+0000 7fb592000640 1 -- 172.21.15.7:0/3977315115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594014800 con 0x7fb5981583c0 2024-04-25T11:19:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.023+0000 7fb59f200640 1 -- 172.21.15.7:0/3977315115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59815a170 con 0x7fb5981583c0 2024-04-25T11:19:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.024+0000 7fb59f200640 1 -- 172.21.15.7:0/3977315115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb598159920 con 0x7fb5981583c0 2024-04-25T11:19:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.025+0000 7fb592000640 1 -- 172.21.15.7:0/3977315115 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb5940149a0 con 0x7fb5981583c0 2024-04-25T11:19:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.026+0000 7fb592000640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb57c0744b0 0x7fb57c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.026+0000 7fb592000640 1 -- 172.21.15.7:0/3977315115 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7fb594095860 con 0x7fb5981583c0 2024-04-25T11:19:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.026+0000 7fb59de00640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb57c0744b0 0x7fb57c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.027+0000 7fb59de00640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb57c0744b0 0x7fb57c0769c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb58c034000 tx=0x7fb58c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.031+0000 7fb592000640 1 -- 172.21.15.7:0/3977315115 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb59405fb00 con 0x7fb5981583c0 2024-04-25T11:19:39.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.239+0000 7fb59f200640 1 -- 172.21.15.7:0/3977315115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb598050740 con 0x7fb5981583c0 2024-04-25T11:19:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.240+0000 7fb592000640 1 -- 172.21.15.7:0/3977315115 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7fb594063e00 con 0x7fb5981583c0 2024-04-25T11:19:39.238 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:19:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 -- 172.21.15.7:0/3977315115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb57c0744b0 msgr2=0x7fb57c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb57c0744b0 0x7fb57c0769c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb58c034000 tx=0x7fb58c02e000 comp rx=0 tx=0).stop 2024-04-25T11:19:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 -- 172.21.15.7:0/3977315115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5981583c0 msgr2=0x7fb59806cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5981583c0 0x7fb59806cee0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb594007e90 tx=0x7fb59400e4b0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 -- 172.21.15.7:0/3977315115 shutdown_connections 2024-04-25T11:19:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb57c0744b0 0x7fb57c0769c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5981583c0 0x7fb59806cee0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5980cfb50 0x7fb5981514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 --2- 172.21.15.7:0/3977315115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb59806eea0 0x7fb5981f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 -- 172.21.15.7:0/3977315115 >> 172.21.15.7:0/3977315115 conn(0x7fb5980740f0 msgr2=0x7fb598161150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 -- 172.21.15.7:0/3977315115 shutdown_connections 2024-04-25T11:19:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.243+0000 7fb59f200640 1 -- 172.21.15.7:0/3977315115 wait complete. 2024-04-25T11:19:39.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2024-04-25T11:19:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.551+0000 7f6980600640 1 Processor -- start 2024-04-25T11:19:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.551+0000 7f6980600640 1 -- start start 2024-04-25T11:19:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.553+0000 7f6980600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f697806d580 0x7f697806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.553+0000 7f6980600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f697806ded0 con 0x7f697806f1e0 2024-04-25T11:19:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.553+0000 7f6980600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6978162700 con 0x7f697815f180 2024-04-25T11:19:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.553+0000 7f6980600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f697815f590 con 0x7f697806d580 2024-04-25T11:19:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.554+0000 7f697d200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f697806d580 0x7f697806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.554+0000 7f697d200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f697806d580 0x7f697806d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47266/0 (socket says 172.21.15.7:47266) 2024-04-25T11:19:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.554+0000 7f697d200640 1 -- 172.21.15.7:0/685755944 learned_addr learned my addr 172.21.15.7:0/685755944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.554+0000 7f697d200640 1 -- 172.21.15.7:0/685755944 >> v1:172.21.15.87:6789/0 conn(0x7f697815f180 legacy=0x7f6978161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.554+0000 7f697d200640 1 -- 172.21.15.7:0/685755944 >> v1:172.21.15.7:6789/0 conn(0x7f697806f1e0 legacy=0x7f697806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.554+0000 7f697d200640 1 -- 172.21.15.7:0/685755944 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69781629e0 con 0x7f697806d580 2024-04-25T11:19:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.556+0000 7f697d200640 1 --2- 172.21.15.7:0/685755944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697806d580 0x7f697806d990 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6964009870 tx=0x7f696402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0af271233a34a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.556+0000 7f6977e00640 1 -- 172.21.15.7:0/685755944 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964037070 con 0x7f697806d580 2024-04-25T11:19:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.556+0000 7f6977e00640 1 -- 172.21.15.7:0/685755944 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6964032440 con 0x7f697806d580 2024-04-25T11:19:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.556+0000 7f6977e00640 1 -- 172.21.15.7:0/685755944 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964032810 con 0x7f697806d580 2024-04-25T11:19:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.557+0000 7f6980600640 1 -- 172.21.15.7:0/685755944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697806d580 msgr2=0x7f697806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.557+0000 7f6980600640 1 --2- 172.21.15.7:0/685755944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697806d580 0x7f697806d990 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6964009870 tx=0x7f696402a210 comp rx=0 tx=0).stop 2024-04-25T11:19:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.557+0000 7f6980600640 1 -- 172.21.15.7:0/685755944 shutdown_connections 2024-04-25T11:19:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.557+0000 7f6980600640 1 --2- 172.21.15.7:0/685755944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697806d580 0x7f697806d990 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.557+0000 7f6980600640 1 -- 172.21.15.7:0/685755944 >> 172.21.15.7:0/685755944 conn(0x7f69780732b0 msgr2=0x7f69780736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.557+0000 7f6980600640 1 -- 172.21.15.7:0/685755944 shutdown_connections 2024-04-25T11:19:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.557+0000 7f6980600640 1 -- 172.21.15.7:0/685755944 wait complete. 2024-04-25T11:19:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.560+0000 7f6980600640 1 Processor -- start 2024-04-25T11:19:39.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.561+0000 7f6980600640 1 -- start start 2024-04-25T11:19:39.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.563+0000 7f6980600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f697806d580 0x7f69781fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.563+0000 7f6980600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f697806f1e0 0x7f69780b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.563+0000 7f6980600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697815f180 0x7f697815d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.563+0000 7f6980600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f697815ec60 con 0x7f697815f180 2024-04-25T11:19:39.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.563+0000 7f6980600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6978158ae0 con 0x7f697806f1e0 2024-04-25T11:19:39.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.563+0000 7f6980600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f697815e670 con 0x7f697806d580 2024-04-25T11:19:39.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.563+0000 7f697e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697815f180 0x7f697815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.563+0000 7f697e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697815f180 0x7f697815d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47274/0 (socket says 172.21.15.7:47274) 2024-04-25T11:19:39.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.563+0000 7f697e600640 1 -- 172.21.15.7:0/408902320 learned_addr learned my addr 172.21.15.7:0/408902320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.563+0000 7f697d200640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f697806f1e0 0x7f69780b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.564+0000 7f697dc00640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f697806d580 0x7f69781fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.564+0000 7f697e600640 1 -- 172.21.15.7:0/408902320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f697806d580 msgr2=0x7f69781fa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.564+0000 7f697e600640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f697806d580 0x7f69781fa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.564+0000 7f697e600640 1 -- 172.21.15.7:0/408902320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f697806f1e0 msgr2=0x7f69780b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.564+0000 7f697e600640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f697806f1e0 0x7f69780b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.564+0000 7f697e600640 1 -- 172.21.15.7:0/408902320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69780b5460 con 0x7f697815f180 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.564+0000 7f697dc00640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f697806d580 0x7f69781fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.564+0000 7f697e600640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697815f180 0x7f697815d250 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f697000c6b0 tx=0x7f697000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.564+0000 7f6975600640 1 -- 172.21.15.7:0/408902320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970003b70 con 0x7f697815f180 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.565+0000 7f6975600640 1 -- 172.21.15.7:0/408902320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6970003d10 con 0x7f697815f180 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.565+0000 7f6980600640 1 -- 172.21.15.7:0/408902320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69780b5740 con 0x7f697815f180 2024-04-25T11:19:39.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.565+0000 7f6975600640 1 -- 172.21.15.7:0/408902320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970005310 con 0x7f697815f180 2024-04-25T11:19:39.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.565+0000 7f6980600640 1 -- 172.21.15.7:0/408902320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697809b7d0 con 0x7f697815f180 2024-04-25T11:19:39.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.566+0000 7f6980600640 1 -- 172.21.15.7:0/408902320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69780b51b0 con 0x7f697815f180 2024-04-25T11:19:39.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.567+0000 7f6975600640 1 -- 172.21.15.7:0/408902320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f69700054b0 con 0x7f697815f180 2024-04-25T11:19:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.568+0000 7f6975600640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69580744b0 0x7f69580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.568+0000 7f6975600640 1 -- 172.21.15.7:0/408902320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7f6970094f80 con 0x7f697815f180 2024-04-25T11:19:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.569+0000 7f697dc00640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69580744b0 0x7f69580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.569+0000 7f697dc00640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69580744b0 0x7f69580769c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6978070310 tx=0x7f696c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.573+0000 7f6975600640 1 -- 172.21.15.7:0/408902320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f697806ded0 con 0x7f697815f180 2024-04-25T11:19:39.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:39.855+0000 7f6980600640 1 -- 172.21.15.7:0/408902320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) -- 0x7f6978050740 con 0x7f697815f180 2024-04-25T11:19:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.150+0000 7f6975600640 1 -- 172.21.15.7:0/408902320 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v473) ==== 146+0+0 (secure 0 0 0) 0x7f697005f870 con 0x7f697815f180 2024-04-25T11:19:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 -- 172.21.15.7:0/408902320 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69580744b0 msgr2=0x7f69580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69580744b0 0x7f69580769c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6978070310 tx=0x7f696c008040 comp rx=0 tx=0).stop 2024-04-25T11:19:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 -- 172.21.15.7:0/408902320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697815f180 msgr2=0x7f697815d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697815f180 0x7f697815d250 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f697000c6b0 tx=0x7f697000cb70 comp rx=0 tx=0).stop 2024-04-25T11:19:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 -- 172.21.15.7:0/408902320 shutdown_connections 2024-04-25T11:19:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f69580744b0 0x7f69580769c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f697815f180 0x7f697815d250 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f697806f1e0 0x7f69780b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 --2- 172.21.15.7:0/408902320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f697806d580 0x7f69781fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 -- 172.21.15.7:0/408902320 >> 172.21.15.7:0/408902320 conn(0x7f69780732b0 msgr2=0x7f6978068780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 -- 172.21.15.7:0/408902320 shutdown_connections 2024-04-25T11:19:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.154+0000 7f6980600640 1 -- 172.21.15.7:0/408902320 wait complete. 2024-04-25T11:19:40.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:19:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.484+0000 7f11cf200640 1 Processor -- start 2024-04-25T11:19:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.485+0000 7f11cf200640 1 -- start start 2024-04-25T11:19:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.487+0000 7f11cf200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d0146250 0x7f11d0161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.487+0000 7f11cf200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11d00b66b0 con 0x7f11d0145e40 2024-04-25T11:19:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.487+0000 7f11cf200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11d00b5b30 con 0x7f11d0144a80 2024-04-25T11:19:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.487+0000 7f11cf200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f11d0145b70 con 0x7f11d0146250 2024-04-25T11:19:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.488+0000 7f11cde00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f11d0145e40 0x7f11d00b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55214/0 (socket says 172.21.15.7:55214) 2024-04-25T11:19:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.488+0000 7f11cde00640 1 -- 172.21.15.7:0/1462004674 learned_addr learned my addr 172.21.15.7:0/1462004674 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.488+0000 7f11ce800640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d0146250 0x7f11d0161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.488+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2312631045 0 0) 0x7f11d00b66b0 con 0x7f11d0145e40 2024-04-25T11:19:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.488+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11b4003520 con 0x7f11d0145e40 2024-04-25T11:19:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3535182051 0 0) 0x7f11b4003520 con 0x7f11d0145e40 2024-04-25T11:19:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d0146250 msgr2=0x7f11d0161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d0146250 0x7f11d0161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 >> v1:172.21.15.87:6789/0 conn(0x7f11d0144a80 legacy=0x7f11d00b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d00b5e10 con 0x7f11d0145e40 2024-04-25T11:19:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f11c0001690 con 0x7f11d0145e40 2024-04-25T11:19:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 >> v1:172.21.15.7:6789/0 conn(0x7f11d0145e40 legacy=0x7f11d00b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b40039e0 0x7f11b4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d0146250 0x7f11b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b4004d40 0x7f11b40051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f11b400bbc0 con 0x7f11b4004d40 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f11b400bd70 con 0x7f11b40039e0 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f11b400bf70 con 0x7f11d0146250 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.489+0000 7f11ce800640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b4004d40 0x7f11b40051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.490+0000 7f11cde00640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b40039e0 0x7f11b4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.490+0000 7f11c7e00640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d0146250 0x7f11b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.490+0000 7f11ce800640 1 -- 172.21.15.7:0/1462004674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d0146250 msgr2=0x7f11b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.490+0000 7f11ce800640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d0146250 0x7f11b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.490+0000 7f11ce800640 1 -- 172.21.15.7:0/1462004674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b40039e0 msgr2=0x7f11b4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.490+0000 7f11ce800640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b40039e0 0x7f11b4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.490+0000 7f11ce800640 1 -- 172.21.15.7:0/1462004674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b4003520 con 0x7f11b4004d40 2024-04-25T11:19:40.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.491+0000 7f11ce800640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b4004d40 0x7f11b40051b0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009fc0 tx=0x7f11b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50fbfb745abccfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.491+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8034020 con 0x7f11b4004d40 2024-04-25T11:19:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.491+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c80305f0 con 0x7f11b4004d40 2024-04-25T11:19:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.491+0000 7f11cd400640 1 -- 172.21.15.7:0/1462004674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8009c10 con 0x7f11b4004d40 2024-04-25T11:19:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.492+0000 7f11cf200640 1 -- 172.21.15.7:0/1462004674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b4004d40 msgr2=0x7f11b40051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.492+0000 7f11cf200640 1 --2- 172.21.15.7:0/1462004674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b4004d40 0x7f11b40051b0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009fc0 tx=0x7f11b4005830 comp rx=0 tx=0).stop 2024-04-25T11:19:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.492+0000 7f11ce800640 1 -- 172.21.15.7:0/1462004674 reap_dead start 2024-04-25T11:19:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.492+0000 7f11cf200640 1 -- 172.21.15.7:0/1462004674 shutdown_connections 2024-04-25T11:19:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.492+0000 7f11cf200640 1 -- 172.21.15.7:0/1462004674 >> 172.21.15.7:0/1462004674 conn(0x7f11d00740f0 msgr2=0x7f11d006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.492+0000 7f11cf200640 1 -- 172.21.15.7:0/1462004674 shutdown_connections 2024-04-25T11:19:40.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.492+0000 7f11cf200640 1 -- 172.21.15.7:0/1462004674 wait complete. 2024-04-25T11:19:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.495+0000 7f11cf200640 1 Processor -- start 2024-04-25T11:19:40.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.495+0000 7f11cf200640 1 -- start start 2024-04-25T11:19:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.496+0000 7f11cf200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4004d40 0x7f11d015c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.496+0000 7f11cf200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11d012f3d0 0x7f11d012d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.496+0000 7f11cf200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d012db40 0x7f11d012fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.496+0000 7f11cf200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f11d00b5690 con 0x7f11d012f3d0 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.496+0000 7f11cf200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f11d00b63d0 con 0x7f11b4004d40 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.496+0000 7f11cf200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f11d00b5810 con 0x7f11d012db40 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.497+0000 7f11c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11d012f3d0 0x7f11d012d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.497+0000 7f11c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11d012f3d0 0x7f11d012d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33684/0 (socket says 172.21.15.7:33684) 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.497+0000 7f11c7e00640 1 -- 172.21.15.7:0/3848114057 learned_addr learned my addr 172.21.15.7:0/3848114057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.497+0000 7f11cde00640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4004d40 0x7f11d015c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.497+0000 7f11ce800640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d012db40 0x7f11d012fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.497+0000 7f11c7e00640 1 -- 172.21.15.7:0/3848114057 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d012db40 msgr2=0x7f11d012fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.497+0000 7f11c7e00640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d012db40 0x7f11d012fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.497+0000 7f11c7e00640 1 -- 172.21.15.7:0/3848114057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4004d40 msgr2=0x7f11d015c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.497+0000 7f11c7e00640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4004d40 0x7f11d015c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.497+0000 7f11c7e00640 1 -- 172.21.15.7:0/3848114057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d012e330 con 0x7f11d012f3d0 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.497+0000 7f11cde00640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4004d40 0x7f11d015c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:40.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.498+0000 7f11c7e00640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11d012f3d0 0x7f11d012d600 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f11b800e550 tx=0x7f11b800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.498+0000 7f11c6000640 1 -- 172.21.15.7:0/3848114057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b80052e0 con 0x7f11d012f3d0 2024-04-25T11:19:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.498+0000 7f11c6000640 1 -- 172.21.15.7:0/3848114057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b8018070 con 0x7f11d012f3d0 2024-04-25T11:19:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.498+0000 7f11cf200640 1 -- 172.21.15.7:0/3848114057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d012e610 con 0x7f11d012f3d0 2024-04-25T11:19:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.498+0000 7f11c6000640 1 -- 172.21.15.7:0/3848114057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8003810 con 0x7f11d012f3d0 2024-04-25T11:19:40.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.498+0000 7f11cf200640 1 -- 172.21.15.7:0/3848114057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d012ebd0 con 0x7f11d012f3d0 2024-04-25T11:19:40.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.499+0000 7f11cf200640 1 -- 172.21.15.7:0/3848114057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d012e0a0 con 0x7f11d012f3d0 2024-04-25T11:19:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.501+0000 7f11c6000640 1 -- 172.21.15.7:0/3848114057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f11b80039b0 con 0x7f11d012f3d0 2024-04-25T11:19:40.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.502+0000 7f11c6000640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11ac0744b0 0x7f11ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:40.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.503+0000 7f11c6000640 1 -- 172.21.15.7:0/3848114057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7f11b8094e40 con 0x7f11d012f3d0 2024-04-25T11:19:40.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.503+0000 7f11cde00640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11ac0744b0 0x7f11ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:40.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.504+0000 7f11cde00640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11ac0744b0 0x7f11ac0769c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f11c0001050 tx=0x7f11c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.508+0000 7f11c6000640 1 -- 172.21.15.7:0/3848114057 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f11b805f6a0 con 0x7f11d012f3d0 2024-04-25T11:19:40.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.713+0000 7f11cf200640 1 -- 172.21.15.7:0/3848114057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f11d0050740 con 0x7f11d012f3d0 2024-04-25T11:19:40.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.720+0000 7f11c6000640 1 -- 172.21.15.7:0/3848114057 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f11b805efa0 con 0x7f11d012f3d0 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.722+0000 7f11cf200640 1 -- 172.21.15.7:0/3848114057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11ac0744b0 msgr2=0x7f11ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.722+0000 7f11cf200640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11ac0744b0 0x7f11ac0769c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f11c0001050 tx=0x7f11c0009040 comp rx=0 tx=0).stop 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.722+0000 7f11cf200640 1 -- 172.21.15.7:0/3848114057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11d012f3d0 msgr2=0x7f11d012d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.722+0000 7f11cf200640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11d012f3d0 0x7f11d012d600 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f11b800e550 tx=0x7f11b800ea10 comp rx=0 tx=0).stop 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.723+0000 7f11cf200640 1 -- 172.21.15.7:0/3848114057 shutdown_connections 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.723+0000 7f11cf200640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11ac0744b0 0x7f11ac0769c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.723+0000 7f11cf200640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11d012db40 0x7f11d012fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.723+0000 7f11cf200640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11d012f3d0 0x7f11d012d600 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.723+0000 7f11cf200640 1 --2- 172.21.15.7:0/3848114057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4004d40 0x7f11d015c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.723+0000 7f11cf200640 1 -- 172.21.15.7:0/3848114057 >> 172.21.15.7:0/3848114057 conn(0x7f11d00740f0 msgr2=0x7f11d006d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.723+0000 7f11cf200640 1 -- 172.21.15.7:0/3848114057 shutdown_connections 2024-04-25T11:19:40.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:40.723+0000 7f11cf200640 1 -- 172.21.15.7:0/3848114057 wait complete. 2024-04-25T11:19:40.748 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:19:40.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:19:40.867 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:19:40.987 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:19:40.987 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:19:40.987 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:41.043 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:19:41.043 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:19:41.044 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:41.099 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:19:41.099 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:19:41.100 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:41.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:19:41.160 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:19:41.160 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:41.216 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:19:41.217 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:19:41.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:41.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.509+0000 7fc93be00640 1 Processor -- start 2024-04-25T11:19:41.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.509+0000 7fc93be00640 1 -- start start 2024-04-25T11:19:41.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.511+0000 7fc93be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc93c160590 0x7fc93c0d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.511+0000 7fc93be00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc93c0d35a0 con 0x7fc93c163ac0 2024-04-25T11:19:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.511+0000 7fc93be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc93c164310 con 0x7fc93c06dcd0 2024-04-25T11:19:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.511+0000 7fc93be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc93c1645f0 con 0x7fc93c160590 2024-04-25T11:19:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.512+0000 7fc93aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc93c163ac0 0x7fc93c0b4b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55226/0 (socket says 172.21.15.7:55226) 2024-04-25T11:19:41.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.512+0000 7fc93aa00640 1 -- 172.21.15.7:0/4287212824 learned_addr learned my addr 172.21.15.7:0/4287212824 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.512+0000 7fc93b400640 1 --2- 172.21.15.7:0/4287212824 >> v2:172.21.15.87:3300/0 conn(0x7fc93c160590 0x7fc93c0d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.512+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1201294940 0 0) 0x7fc93c0d35a0 con 0x7fc93c163ac0 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc91c003520 con 0x7fc93c163ac0 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1087458953 0 0) 0x7fc91c003520 con 0x7fc93c163ac0 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 >> v2:172.21.15.87:3300/0 conn(0x7fc93c160590 msgr2=0x7fc93c0d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 --2- 172.21.15.7:0/4287212824 >> v2:172.21.15.87:3300/0 conn(0x7fc93c160590 0x7fc93c0d3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 >> v1:172.21.15.87:6789/0 conn(0x7fc93c06dcd0 legacy=0x7fc93c06e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c1648d0 con 0x7fc93c163ac0 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fc92c001670 con 0x7fc93c163ac0 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 >> v1:172.21.15.7:6789/0 conn(0x7fc93c163ac0 legacy=0x7fc93c0b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 --2- 172.21.15.7:0/4287212824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 0x7fc91c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 --2- 172.21.15.7:0/4287212824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc91c004800 0x7fc91c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 --2- 172.21.15.7:0/4287212824 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc91c0052a0 0x7fc91c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c1645f0 con 0x7fc91c003a00 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc91c00c230 con 0x7fc91c004800 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc91c00c430 con 0x7fc91c0052a0 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.513+0000 7fc93aa00640 1 --2- 172.21.15.7:0/4287212824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 0x7fc91c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.514+0000 7fc93a000640 1 --2- 172.21.15.7:0/4287212824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc91c004800 0x7fc91c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.514+0000 7fc93aa00640 1 -- 172.21.15.7:0/4287212824 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc91c0052a0 msgr2=0x7fc91c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.514+0000 7fc93aa00640 1 --2- 172.21.15.7:0/4287212824 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc91c0052a0 0x7fc91c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.514+0000 7fc93aa00640 1 -- 172.21.15.7:0/4287212824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc91c004800 msgr2=0x7fc91c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.514+0000 7fc93aa00640 1 --2- 172.21.15.7:0/4287212824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc91c004800 0x7fc91c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.514+0000 7fc93aa00640 1 -- 172.21.15.7:0/4287212824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91c003520 con 0x7fc91c003a00 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.514+0000 7fc93b400640 1 -- 172.21.15.7:0/4287212824 reap_dead start 2024-04-25T11:19:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.514+0000 7fc93a000640 1 --2- 172.21.15.7:0/4287212824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc91c004800 0x7fc91c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:41.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.515+0000 7fc93aa00640 1 --2- 172.21.15.7:0/4287212824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 0x7fc91c003e70 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00c110 tx=0x7fc91c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bead5b51d3da2c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.515+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c007c40 con 0x7fc91c003a00 2024-04-25T11:19:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.515+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92c005af0 con 0x7fc91c003a00 2024-04-25T11:19:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.515+0000 7fc939600640 1 -- 172.21.15.7:0/4287212824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c007070 con 0x7fc91c003a00 2024-04-25T11:19:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.516+0000 7fc93be00640 1 -- 172.21.15.7:0/4287212824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 msgr2=0x7fc91c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.516+0000 7fc93be00640 1 --2- 172.21.15.7:0/4287212824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 0x7fc91c003e70 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00c110 tx=0x7fc91c005830 comp rx=0 tx=0).stop 2024-04-25T11:19:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.516+0000 7fc93be00640 1 -- 172.21.15.7:0/4287212824 shutdown_connections 2024-04-25T11:19:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.516+0000 7fc93be00640 1 --2- 172.21.15.7:0/4287212824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 0x7fc91c003e70 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.516+0000 7fc93be00640 1 -- 172.21.15.7:0/4287212824 >> 172.21.15.7:0/4287212824 conn(0x7fc93c0d29e0 msgr2=0x7fc93c0d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.516+0000 7fc93be00640 1 -- 172.21.15.7:0/4287212824 shutdown_connections 2024-04-25T11:19:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.516+0000 7fc93be00640 1 -- 172.21.15.7:0/4287212824 wait complete. 2024-04-25T11:19:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.519+0000 7fc93be00640 1 Processor -- start 2024-04-25T11:19:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.519+0000 7fc93be00640 1 -- start start 2024-04-25T11:19:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.520+0000 7fc93be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 0x7fc93c1548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.520+0000 7fc93be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc93c158eb0 0x7fc93c1592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc93c157c60 0x7fc93c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c06e0e0 con 0x7fc91c003a00 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc93c06e260 con 0x7fc93c158eb0 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc93c06e3e0 con 0x7fc93c157c60 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 0x7fc93c1548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 0x7fc93c1548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33696/0 (socket says 172.21.15.7:33696) 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93aa00640 1 -- 172.21.15.7:0/230882689 learned_addr learned my addr 172.21.15.7:0/230882689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93a000640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc93c158eb0 0x7fc93c1592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93b400640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc93c157c60 0x7fc93c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93aa00640 1 -- 172.21.15.7:0/230882689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc93c157c60 msgr2=0x7fc93c158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93aa00640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc93c157c60 0x7fc93c158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93aa00640 1 -- 172.21.15.7:0/230882689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc93c158eb0 msgr2=0x7fc93c1592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93aa00640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc93c158eb0 0x7fc93c1592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93aa00640 1 -- 172.21.15.7:0/230882689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c1585b0 con 0x7fc91c003a00 2024-04-25T11:19:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.521+0000 7fc93a000640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc93c158eb0 0x7fc93c1592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.522+0000 7fc93aa00640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 0x7fc93c1548e0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00c230 tx=0x7fc93c1644e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.522+0000 7fc932a00640 1 -- 172.21.15.7:0/230882689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c037040 con 0x7fc91c003a00 2024-04-25T11:19:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.522+0000 7fc932a00640 1 -- 172.21.15.7:0/230882689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92c041070 con 0x7fc91c003a00 2024-04-25T11:19:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.522+0000 7fc93be00640 1 -- 172.21.15.7:0/230882689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93c159be0 con 0x7fc91c003a00 2024-04-25T11:19:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.522+0000 7fc932a00640 1 -- 172.21.15.7:0/230882689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c001670 con 0x7fc91c003a00 2024-04-25T11:19:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.522+0000 7fc93be00640 1 -- 172.21.15.7:0/230882689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc93c155cb0 con 0x7fc91c003a00 2024-04-25T11:19:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.523+0000 7fc93be00640 1 -- 172.21.15.7:0/230882689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc93c050740 con 0x7fc91c003a00 2024-04-25T11:19:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.525+0000 7fc932a00640 1 -- 172.21.15.7:0/230882689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc92c004a20 con 0x7fc91c003a00 2024-04-25T11:19:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.526+0000 7fc932a00640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9180744b0 0x7fc9180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.526+0000 7fc932a00640 1 -- 172.21.15.7:0/230882689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5335+0+0 (secure 0 0 0) 0x7fc92c0b50c0 con 0x7fc91c003a00 2024-04-25T11:19:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.526+0000 7fc93a000640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9180744b0 0x7fc9180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.527+0000 7fc93a000640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9180744b0 0x7fc9180769c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc93c15f560 tx=0x7fc924009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.531+0000 7fc932a00640 1 -- 172.21.15.7:0/230882689 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc93c1585b0 con 0x7fc91c003a00 2024-04-25T11:19:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.773+0000 7fc93be00640 1 -- 172.21.15.7:0/230882689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc93c15f590 con 0x7fc91c003a00 2024-04-25T11:19:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.775+0000 7fc932a00640 1 -- 172.21.15.7:0/230882689 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+3860 (secure 0 0 0) 0x7fc92c07f8a0 con 0x7fc91c003a00 2024-04-25T11:19:41.773 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:41.773 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":69,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5846,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/4132210096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":4132210096},{"type":"v1","addr":"172.21.15.87:6834","nonce":4132210096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2657338269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2657338269},{"type":"v1","addr":"172.21.15.87:6835","nonce":2657338269}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6457,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/138020048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":138020048},{"type":"v1","addr":"172.21.15.7:6836","nonce":138020048}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":7926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1416028387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1416028387},{"type":"v1","addr":"172.21.15.7:6837","nonce":1416028387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-04-25T11:19:41.773 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 69 2024-04-25T11:19:41.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.777+0000 7fc93be00640 1 -- 172.21.15.7:0/230882689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9180744b0 msgr2=0x7fc9180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.777+0000 7fc93be00640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9180744b0 0x7fc9180769c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc93c15f560 tx=0x7fc924009040 comp rx=0 tx=0).stop 2024-04-25T11:19:41.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.777+0000 7fc93be00640 1 -- 172.21.15.7:0/230882689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 msgr2=0x7fc93c1548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:41.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.777+0000 7fc93be00640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 0x7fc93c1548e0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00c230 tx=0x7fc93c1644e0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.777+0000 7fc93be00640 1 -- 172.21.15.7:0/230882689 shutdown_connections 2024-04-25T11:19:41.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.777+0000 7fc93be00640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9180744b0 0x7fc9180769c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.777+0000 7fc93be00640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc93c157c60 0x7fc93c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.777+0000 7fc93be00640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc93c158eb0 0x7fc93c1592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.777+0000 7fc93be00640 1 --2- 172.21.15.7:0/230882689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc91c003a00 0x7fc93c1548e0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:41.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.777+0000 7fc93be00640 1 -- 172.21.15.7:0/230882689 >> 172.21.15.7:0/230882689 conn(0x7fc93c0d29e0 msgr2=0x7fc93c0b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:41.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.777+0000 7fc93be00640 1 -- 172.21.15.7:0/230882689 shutdown_connections 2024-04-25T11:19:41.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:41.778+0000 7fc93be00640 1 -- 172.21.15.7:0/230882689 wait complete. 2024-04-25T11:19:41.801 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:19:41.801 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:19:41.801 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:19:41.801 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:19:41.802 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:19:41.802 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:19:41.802 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:19:41.802 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:19:41.802 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:19:41.802 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:19:41.803 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:19:41.803 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:19:41.803 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:19:41.803 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:19:41.803 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:19:41.803 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:19:41.831 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:19:41.831 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:19:41.843 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:19:41.843 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:19:41.844 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:19:41.844 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:19:41.847 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:19:41.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:19:41.890 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:19:41.891 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:41.894 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:19:41.895 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:41.897 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:19:41.897 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:41.901 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:19:41.901 DEBUG:teuthology.parallel:result is None 2024-04-25T11:19:41.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:19:42.110 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:19:42.111 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:19:42.122 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:19:42.122 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:19:42.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.154+0000 7f2b51600640 1 Processor -- start 2024-04-25T11:19:42.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.154+0000 7f2b51600640 1 -- start start 2024-04-25T11:19:42.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.157+0000 7f2b51600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b4c06d580 0x7f2b4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:42.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.157+0000 7f2b51600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2b4c06f1e0 0x7f2b4c06f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:42.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.157+0000 7f2b51600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b4c06ded0 con 0x7f2b4c15f180 2024-04-25T11:19:42.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.157+0000 7f2b51600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2b4c15f590 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.157+0000 7f2b51600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c158ae0 con 0x7f2b4c06d580 2024-04-25T11:19:42.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.157+0000 7f2b4aa00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f2b4c15f180 0x7f2b4c161c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:45034/0 (socket says 172.21.15.7:45034) 2024-04-25T11:19:42.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.157+0000 7f2b4aa00640 1 -- 172.21.15.7:0/3466123722 learned_addr learned my addr 172.21.15.7:0/3466123722 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:42.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.157+0000 7f2b4b400640 1 --2- 172.21.15.7:0/3466123722 >> v2:172.21.15.87:3300/0 conn(0x7f2b4c06f1e0 0x7f2b4c06f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:42.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.157+0000 7f2b4a000640 1 --2- 172.21.15.7:0/3466123722 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b4c06d580 0x7f2b4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:42.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.158+0000 7f2b4b400640 1 -- 172.21.15.7:0/3466123722 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b4c06d580 msgr2=0x7f2b4c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:42.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.158+0000 7f2b4b400640 1 --2- 172.21.15.7:0/3466123722 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b4c06d580 0x7f2b4c06d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:42.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.158+0000 7f2b4b400640 1 -- 172.21.15.7:0/3466123722 >> v1:172.21.15.87:6789/0 conn(0x7f2b4c15f180 legacy=0x7f2b4c161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:42.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.158+0000 7f2b4b400640 1 -- 172.21.15.7:0/3466123722 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c162700 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.159+0000 7f2b4b400640 1 --2- 172.21.15.7:0/3466123722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4c06f1e0 0x7f2b4c06f5f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009dc0 tx=0x7f2b4402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d3144660664ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:42.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.159+0000 7f2b49600640 1 -- 172.21.15.7:0/3466123722 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4402ae60 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.159+0000 7f2b49600640 1 -- 172.21.15.7:0/3466123722 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b44007740 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.159+0000 7f2b49600640 1 -- 172.21.15.7:0/3466123722 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44007b10 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.160+0000 7f2b51600640 1 -- 172.21.15.7:0/3466123722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4c06f1e0 msgr2=0x7f2b4c06f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:42.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.160+0000 7f2b51600640 1 --2- 172.21.15.7:0/3466123722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4c06f1e0 0x7f2b4c06f5f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009dc0 tx=0x7f2b4402a680 comp rx=0 tx=0).stop 2024-04-25T11:19:42.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.160+0000 7f2b51600640 1 -- 172.21.15.7:0/3466123722 shutdown_connections 2024-04-25T11:19:42.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.160+0000 7f2b51600640 1 --2- 172.21.15.7:0/3466123722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4c06f1e0 0x7f2b4c06f5f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:42.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.160+0000 7f2b51600640 1 --2- 172.21.15.7:0/3466123722 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b4c06d580 0x7f2b4c06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:42.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.160+0000 7f2b51600640 1 -- 172.21.15.7:0/3466123722 >> 172.21.15.7:0/3466123722 conn(0x7f2b4c0732b0 msgr2=0x7f2b4c0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:42.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.160+0000 7f2b51600640 1 -- 172.21.15.7:0/3466123722 shutdown_connections 2024-04-25T11:19:42.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.160+0000 7f2b51600640 1 -- 172.21.15.7:0/3466123722 wait complete. 2024-04-25T11:19:42.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.163+0000 7f2b51600640 1 Processor -- start 2024-04-25T11:19:42.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.163+0000 7f2b51600640 1 -- start start 2024-04-25T11:19:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.165+0000 7f2b51600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4c06d580 0x7f2b4c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.165+0000 7f2b51600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b4c06f1e0 0x7f2b4c1f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.165+0000 7f2b51600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b4c15f180 0x7f2b4c14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.165+0000 7f2b51600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c15ec60 con 0x7f2b4c15f180 2024-04-25T11:19:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.165+0000 7f2b51600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c15e980 con 0x7f2b4c06d580 2024-04-25T11:19:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.165+0000 7f2b51600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4c15e670 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.165+0000 7f2b4b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b4c15f180 0x7f2b4c14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.165+0000 7f2b4b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b4c15f180 0x7f2b4c14cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33744/0 (socket says 172.21.15.7:33744) 2024-04-25T11:19:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.165+0000 7f2b4b400640 1 -- 172.21.15.7:0/897646532 learned_addr learned my addr 172.21.15.7:0/897646532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:42.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.165+0000 7f2b4aa00640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4c06d580 0x7f2b4c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:42.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.165+0000 7f2b4a000640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b4c06f1e0 0x7f2b4c1f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:42.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.166+0000 7f2b4a000640 1 -- 172.21.15.7:0/897646532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4c06d580 msgr2=0x7f2b4c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:42.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.166+0000 7f2b4a000640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4c06d580 0x7f2b4c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:42.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.166+0000 7f2b4a000640 1 -- 172.21.15.7:0/897646532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b4c15f180 msgr2=0x7f2b4c14cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:42.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.166+0000 7f2b4a000640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b4c15f180 0x7f2b4c14cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:42.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.166+0000 7f2b4a000640 1 -- 172.21.15.7:0/897646532 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c1f5c50 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.166+0000 7f2b4aa00640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4c06d580 0x7f2b4c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:42.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.166+0000 7f2b4a000640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b4c06f1e0 0x7f2b4c1f5710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800e8a0 tx=0x7f2b3800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:42.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.168+0000 7f2b42a00640 1 -- 172.21.15.7:0/897646532 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38017070 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.168+0000 7f2b42a00640 1 -- 172.21.15.7:0/897646532 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3800c940 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.168+0000 7f2b42a00640 1 -- 172.21.15.7:0/897646532 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38009e60 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.168+0000 7f2b51600640 1 -- 172.21.15.7:0/897646532 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4c1f45e0 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.169+0000 7f2b51600640 1 -- 172.21.15.7:0/897646532 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4c151e90 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.170+0000 7f2b51600640 1 -- 172.21.15.7:0/897646532 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4c050740 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.171+0000 7f2b42a00640 1 -- 172.21.15.7:0/897646532 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2b3801a070 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.173+0000 7f2b42a00640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b2c074530 0x7f2b2c076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:42.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.173+0000 7f2b42a00640 1 -- 172.21.15.7:0/897646532 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(111..111 src has 1..111) ==== 5335+0+0 (secure 0 0 0) 0x7f2b38094e40 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.173+0000 7f2b4aa00640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b2c074530 0x7f2b2c076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:42.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.174+0000 7f2b4aa00640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b2c074530 0x7f2b2c076a40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005880 tx=0x7f2b3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:42.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.179+0000 7f2b42a00640 1 -- 172.21.15.7:0/897646532 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b3805f0d0 con 0x7f2b4c06f1e0 2024-04-25T11:19:42.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:42.400+0000 7f2b51600640 1 -- 172.21.15.7:0/897646532 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2b4c074250 con 0x7f2b4c06f1e0 2024-04-25T11:19:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.197+0000 7f2b42a00640 1 -- 172.21.15.7:0/897646532 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v114) ==== 97+0+0 (secure 0 0 0) 0x7f2b380633d0 con 0x7f2b4c06f1e0 2024-04-25T11:19:43.195 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-04-25T11:19:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.200+0000 7f2b51600640 1 -- 172.21.15.7:0/897646532 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b2c074530 msgr2=0x7f2b2c076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.200+0000 7f2b51600640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b2c074530 0x7f2b2c076a40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c005880 tx=0x7f2b3c009040 comp rx=0 tx=0).stop 2024-04-25T11:19:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.200+0000 7f2b51600640 1 -- 172.21.15.7:0/897646532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b4c06f1e0 msgr2=0x7f2b4c1f5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.200+0000 7f2b51600640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b4c06f1e0 0x7f2b4c1f5710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800e8a0 tx=0x7f2b3800ed60 comp rx=0 tx=0).stop 2024-04-25T11:19:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.201+0000 7f2b51600640 1 -- 172.21.15.7:0/897646532 shutdown_connections 2024-04-25T11:19:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.201+0000 7f2b51600640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b2c074530 0x7f2b2c076a40 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.201+0000 7f2b51600640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b4c15f180 0x7f2b4c14cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.201+0000 7f2b51600640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b4c06f1e0 0x7f2b4c1f5710 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.201+0000 7f2b51600640 1 --2- 172.21.15.7:0/897646532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4c06d580 0x7f2b4c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.201+0000 7f2b51600640 1 -- 172.21.15.7:0/897646532 >> 172.21.15.7:0/897646532 conn(0x7f2b4c0732b0 msgr2=0x7f2b4c161d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.201+0000 7f2b51600640 1 -- 172.21.15.7:0/897646532 shutdown_connections 2024-04-25T11:19:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.201+0000 7f2b51600640 1 -- 172.21.15.7:0/897646532 wait complete. 2024-04-25T11:19:43.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:19:43.205 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:19:43.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:19:43.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.543+0000 7f42aba00640 1 Processor -- start 2024-04-25T11:19:43.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.543+0000 7f42aba00640 1 -- start start 2024-04-25T11:19:43.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.545+0000 7f42aba00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42a40d1350 0x7f42a41648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:43.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.546+0000 7f42aba00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42a40b5660 con 0x7f42a40b6960 2024-04-25T11:19:43.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.546+0000 7f42aba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42a40b58e0 con 0x7f42a406d580 2024-04-25T11:19:43.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.546+0000 7f42aba00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f42a40b64d0 con 0x7f42a40d1350 2024-04-25T11:19:43.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.546+0000 7f42a9000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f42a40b6960 0x7f42a4160220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55266/0 (socket says 172.21.15.7:55266) 2024-04-25T11:19:43.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.546+0000 7f42a9000640 1 -- 172.21.15.7:0/2008622061 learned_addr learned my addr 172.21.15.7:0/2008622061 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:43.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.546+0000 7f42a3e00640 1 --2- 172.21.15.7:0/2008622061 >> v2:172.21.15.7:3300/0 conn(0x7f42a40d1350 0x7f42a41648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:43.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.546+0000 7f42a3400640 1 -- 172.21.15.7:0/2008622061 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 80328015 0 0) 0x7f4298001110 con 0x7f42a40b6960 2024-04-25T11:19:43.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.546+0000 7f42a3400640 1 -- 172.21.15.7:0/2008622061 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f428c003520 con 0x7f42a40b6960 2024-04-25T11:19:43.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.547+0000 7f42a3e00640 1 -- 172.21.15.7:0/2008622061 >> v1:172.21.15.87:6789/0 conn(0x7f42a406d580 legacy=0x7f42a406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:43.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.547+0000 7f42a3e00640 1 -- 172.21.15.7:0/2008622061 >> v1:172.21.15.7:6789/0 conn(0x7f42a40b6960 legacy=0x7f42a4160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:43.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.547+0000 7f42a3400640 1 -- 172.21.15.7:0/2008622061 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1975367597 0 0) 0x7f428c003520 con 0x7f42a40b6960 2024-04-25T11:19:43.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.547+0000 7f42a3e00640 1 -- 172.21.15.7:0/2008622061 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a4164df0 con 0x7f42a40d1350 2024-04-25T11:19:43.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.548+0000 7f42a3e00640 1 --2- 172.21.15.7:0/2008622061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a40d1350 0x7f42a41648b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f42900097a0 tx=0x7f429002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f118a6414f0f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:43.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.548+0000 7f42a3400640 1 -- 172.21.15.7:0/2008622061 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290037070 con 0x7f42a40d1350 2024-04-25T11:19:43.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.548+0000 7f42a3400640 1 -- 172.21.15.7:0/2008622061 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f429002ac50 con 0x7f42a40d1350 2024-04-25T11:19:43.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.548+0000 7f42a3400640 1 -- 172.21.15.7:0/2008622061 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298001840 con 0x7f42a40d1350 2024-04-25T11:19:43.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.549+0000 7f42aba00640 1 -- 172.21.15.7:0/2008622061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a40d1350 msgr2=0x7f42a41648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:43.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.549+0000 7f42aba00640 1 --2- 172.21.15.7:0/2008622061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a40d1350 0x7f42a41648b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f42900097a0 tx=0x7f429002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.549+0000 7f42aba00640 1 -- 172.21.15.7:0/2008622061 shutdown_connections 2024-04-25T11:19:43.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.549+0000 7f42aba00640 1 --2- 172.21.15.7:0/2008622061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a40d1350 0x7f42a41648b0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.549+0000 7f42aba00640 1 -- 172.21.15.7:0/2008622061 >> 172.21.15.7:0/2008622061 conn(0x7f42a40732b0 msgr2=0x7f42a40736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:43.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.549+0000 7f42aba00640 1 -- 172.21.15.7:0/2008622061 shutdown_connections 2024-04-25T11:19:43.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.549+0000 7f42aba00640 1 -- 172.21.15.7:0/2008622061 wait complete. 2024-04-25T11:19:43.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.551+0000 7f42aba00640 1 Processor -- start 2024-04-25T11:19:43.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.552+0000 7f42aba00640 1 -- start start 2024-04-25T11:19:43.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.553+0000 7f42aba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a406d580 0x7f42a4143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:43.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.553+0000 7f42aba00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42a40b6960 0x7f42a40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:43.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.553+0000 7f42aba00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a40d1350 0x7f42a40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:43.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.553+0000 7f42aba00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42a40b6160 con 0x7f42a40d1350 2024-04-25T11:19:43.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.553+0000 7f42aba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f42a40b5e80 con 0x7f42a406d580 2024-04-25T11:19:43.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.553+0000 7f42aba00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f42a416b260 con 0x7f42a40b6960 2024-04-25T11:19:43.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.553+0000 7f42a9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a40d1350 0x7f42a40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:43.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.554+0000 7f42a9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a40d1350 0x7f42a40b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33764/0 (socket says 172.21.15.7:33764) 2024-04-25T11:19:43.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.554+0000 7f42a9a00640 1 -- 172.21.15.7:0/2225490463 learned_addr learned my addr 172.21.15.7:0/2225490463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:43.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.554+0000 7f42a9000640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a406d580 0x7f42a4143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:43.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.554+0000 7f42a3e00640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42a40b6960 0x7f42a40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:43.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.554+0000 7f42a9a00640 1 -- 172.21.15.7:0/2225490463 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42a40b6960 msgr2=0x7f42a40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:43.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.554+0000 7f42a9a00640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42a40b6960 0x7f42a40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.554+0000 7f42a9a00640 1 -- 172.21.15.7:0/2225490463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a406d580 msgr2=0x7f42a4143f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:43.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.554+0000 7f42a9a00640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a406d580 0x7f42a4143f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.554+0000 7f42a9a00640 1 -- 172.21.15.7:0/2225490463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a41603c0 con 0x7f42a40d1350 2024-04-25T11:19:43.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.554+0000 7f42a9000640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a406d580 0x7f42a4143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:43.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.554+0000 7f42a9a00640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a40d1350 0x7f42a40b04b0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c6b0 tx=0x7f429c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:43.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.555+0000 7f42a0c00640 1 -- 172.21.15.7:0/2225490463 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c003b70 con 0x7f42a40d1350 2024-04-25T11:19:43.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.555+0000 7f42a0c00640 1 -- 172.21.15.7:0/2225490463 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f429c003d10 con 0x7f42a40d1350 2024-04-25T11:19:43.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.555+0000 7f42a0c00640 1 -- 172.21.15.7:0/2225490463 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429c005310 con 0x7f42a40d1350 2024-04-25T11:19:43.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.555+0000 7f42aba00640 1 -- 172.21.15.7:0/2225490463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a41606a0 con 0x7f42a40d1350 2024-04-25T11:19:43.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.555+0000 7f42aba00640 1 -- 172.21.15.7:0/2225490463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a4160c60 con 0x7f42a40d1350 2024-04-25T11:19:43.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.556+0000 7f42aba00640 1 -- 172.21.15.7:0/2225490463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a4050740 con 0x7f42a40d1350 2024-04-25T11:19:43.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.557+0000 7f42a0c00640 1 -- 172.21.15.7:0/2225490463 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f429c0054b0 con 0x7f42a40d1350 2024-04-25T11:19:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.559+0000 7f42a0c00640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42840744b0 0x7f42840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.559+0000 7f42a0c00640 1 -- 172.21.15.7:0/2225490463 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5249+0+0 (secure 0 0 0) 0x7f429c094f30 con 0x7f42a40d1350 2024-04-25T11:19:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.559+0000 7f42a9000640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42840744b0 0x7f42840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:43.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.560+0000 7f42a9000640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42840744b0 0x7f42840769c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4298001eb0 tx=0x7f4298009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.564+0000 7f42a0c00640 1 -- 172.21.15.7:0/2225490463 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f429c05f7e0 con 0x7f42a40d1350 2024-04-25T11:19:43.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.852+0000 7f42aba00640 1 -- 172.21.15.7:0/2225490463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f42a4074130 con 0x7f42a40d1350 2024-04-25T11:19:43.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.853+0000 7f42a0c00640 1 -- 172.21.15.7:0/2225490463 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f429c05f0e0 con 0x7f42a40d1350 2024-04-25T11:19:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:43.851 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:19:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-04-25T11:19:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-04-25T11:19:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w==", 2024-04-25T11:19:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:43.851 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw==", 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-04-25T11:19:43.852 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw==", 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A==", 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA==", 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-04-25T11:19:43.854 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg==", 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw==", 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ==", 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw==", 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.856 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng==", 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-04-25T11:19:43.857 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ==", 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A==", 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg==", 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA==", 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:43.859 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg==", 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.860 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ==", 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDZOSpmS8ObHxAAkXXXg1OI1YFC2/OZER3woA==", 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:19:43.861 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw==", 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-04-25T11:19:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w==", 2024-04-25T11:19:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:19:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:19:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:19:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:19:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-25T11:19:43.863 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 -- 172.21.15.7:0/2225490463 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42840744b0 msgr2=0x7f42840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42840744b0 0x7f42840769c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4298001eb0 tx=0x7f4298009040 comp rx=0 tx=0).stop 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 -- 172.21.15.7:0/2225490463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a40d1350 msgr2=0x7f42a40b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a40d1350 0x7f42a40b04b0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c6b0 tx=0x7f429c00cb70 comp rx=0 tx=0).stop 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 -- 172.21.15.7:0/2225490463 shutdown_connections 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42840744b0 0x7f42840769c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42a40d1350 0x7f42a40b04b0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42a40b6960 0x7f42a40aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 --2- 172.21.15.7:0/2225490463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42a406d580 0x7f42a4143f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 -- 172.21.15.7:0/2225490463 >> 172.21.15.7:0/2225490463 conn(0x7f42a40732b0 msgr2=0x7f42a40b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 -- 172.21.15.7:0/2225490463 shutdown_connections 2024-04-25T11:19:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:43.856+0000 7f42aba00640 1 -- 172.21.15.7:0/2225490463 wait complete. 2024-04-25T11:19:43.880 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:44.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.162+0000 7f8064a00640 1 Processor -- start 2024-04-25T11:19:44.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.162+0000 7f8064a00640 1 -- start start 2024-04-25T11:19:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.164+0000 7f8064a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80600b03b0 0x7f80600b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.164+0000 7f8064a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f806006ef10 con 0x7f8060070890 2024-04-25T11:19:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.164+0000 7f8064a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80600b6390 con 0x7f806015c390 2024-04-25T11:19:44.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.164+0000 7f8064a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f80600b07c0 con 0x7f80600b03b0 2024-04-25T11:19:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.165+0000 7f805f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80600b03b0 0x7f80600b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.165+0000 7f805e800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8060070890 0x7f806006f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55278/0 (socket says 172.21.15.7:55278) 2024-04-25T11:19:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.165+0000 7f805e800640 1 -- 172.21.15.7:0/1062767956 learned_addr learned my addr 172.21.15.7:0/1062767956 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.165+0000 7f805f200640 1 -- 172.21.15.7:0/1062767956 >> v1:172.21.15.87:6789/0 conn(0x7f806015c390 legacy=0x7f80600afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.165+0000 7f805f200640 1 -- 172.21.15.7:0/1062767956 >> v1:172.21.15.7:6789/0 conn(0x7f8060070890 legacy=0x7f806006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.165+0000 7f805de00640 1 -- 172.21.15.7:0/1062767956 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3098501731 0 0) 0x7f806006ef10 con 0x7f8060070890 2024-04-25T11:19:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.165+0000 7f805f200640 1 -- 172.21.15.7:0/1062767956 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80600b6670 con 0x7f80600b03b0 2024-04-25T11:19:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.166+0000 7f805f200640 1 --2- 172.21.15.7:0/1062767956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80600b03b0 0x7f80600b5140 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8058009d90 tx=0x7f805802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b642bc78d8d481 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.167+0000 7f805de00640 1 -- 172.21.15.7:0/1062767956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058031600 con 0x7f80600b03b0 2024-04-25T11:19:44.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.167+0000 7f805de00640 1 -- 172.21.15.7:0/1062767956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80580317a0 con 0x7f80600b03b0 2024-04-25T11:19:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.167+0000 7f805de00640 1 -- 172.21.15.7:0/1062767956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058031b70 con 0x7f80600b03b0 2024-04-25T11:19:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.167+0000 7f8064a00640 1 -- 172.21.15.7:0/1062767956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80600b03b0 msgr2=0x7f80600b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.167+0000 7f8064a00640 1 --2- 172.21.15.7:0/1062767956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80600b03b0 0x7f80600b5140 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8058009d90 tx=0x7f805802a610 comp rx=0 tx=0).stop 2024-04-25T11:19:44.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.168+0000 7f8064a00640 1 -- 172.21.15.7:0/1062767956 shutdown_connections 2024-04-25T11:19:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.168+0000 7f8064a00640 1 --2- 172.21.15.7:0/1062767956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80600b03b0 0x7f80600b5140 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.168+0000 7f8064a00640 1 -- 172.21.15.7:0/1062767956 >> 172.21.15.7:0/1062767956 conn(0x7f80600b3820 msgr2=0x7f80600b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.168+0000 7f8064a00640 1 -- 172.21.15.7:0/1062767956 shutdown_connections 2024-04-25T11:19:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.168+0000 7f8064a00640 1 -- 172.21.15.7:0/1062767956 wait complete. 2024-04-25T11:19:44.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.170+0000 7f8064a00640 1 Processor -- start 2024-04-25T11:19:44.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.170+0000 7f8064a00640 1 -- start start 2024-04-25T11:19:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f8064a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060070890 0x7f806015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f8064a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80600b03b0 0x7f8060141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f8064a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f806015c390 0x7f80600d0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f8064a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f806015c7a0 con 0x7f8060070890 2024-04-25T11:19:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f8064a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f806006f210 con 0x7f806015c390 2024-04-25T11:19:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f8064a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f80600b6950 con 0x7f80600b03b0 2024-04-25T11:19:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f805e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060070890 0x7f806015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f805e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060070890 0x7f806015c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33784/0 (socket says 172.21.15.7:33784) 2024-04-25T11:19:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f805e800640 1 -- 172.21.15.7:0/1026190239 learned_addr learned my addr 172.21.15.7:0/1026190239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f8055e00640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80600b03b0 0x7f8060141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f805e800640 1 -- 172.21.15.7:0/1026190239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80600b03b0 msgr2=0x7f8060141b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f805f200640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f806015c390 0x7f80600d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.172+0000 7f805e800640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80600b03b0 0x7f8060141b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.173+0000 7f805e800640 1 -- 172.21.15.7:0/1026190239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f806015c390 msgr2=0x7f80600d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.173+0000 7f805e800640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f806015c390 0x7f80600d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.173+0000 7f805e800640 1 -- 172.21.15.7:0/1026190239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806015ef00 con 0x7f8060070890 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.173+0000 7f805f200640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f806015c390 0x7f80600d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.173+0000 7f805e800640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060070890 0x7f806015c9b0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f805000c410 tx=0x7f805000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.173+0000 7f8057400640 1 -- 172.21.15.7:0/1026190239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80500074e0 con 0x7f8060070890 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.173+0000 7f8057400640 1 -- 172.21.15.7:0/1026190239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80500047e0 con 0x7f8060070890 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.173+0000 7f8064a00640 1 -- 172.21.15.7:0/1026190239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80600cf770 con 0x7f8060070890 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.173+0000 7f8057400640 1 -- 172.21.15.7:0/1026190239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8050004bd0 con 0x7f8060070890 2024-04-25T11:19:44.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.173+0000 7f8064a00640 1 -- 172.21.15.7:0/1026190239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80600cfd30 con 0x7f8060070890 2024-04-25T11:19:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.175+0000 7f8064a00640 1 -- 172.21.15.7:0/1026190239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8060050740 con 0x7f8060070890 2024-04-25T11:19:44.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.176+0000 7f8057400640 1 -- 172.21.15.7:0/1026190239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8050011070 con 0x7f8060070890 2024-04-25T11:19:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.178+0000 7f8057400640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 0x7f803c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.178+0000 7f8057400640 1 -- 172.21.15.7:0/1026190239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5249+0+0 (secure 0 0 0) 0x7f8050094cc0 con 0x7f8060070890 2024-04-25T11:19:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.178+0000 7f8055e00640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 0x7f803c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.179+0000 7f8055e00640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 0x7f803c0769f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8060068b10 tx=0x7f8048008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:44.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.185+0000 7f8057400640 1 -- 172.21.15.7:0/1026190239 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f805009b080 con 0x7f8060070890 2024-04-25T11:19:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.395+0000 7f8064a00640 1 -- 172.21.15.7:0/1026190239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8060070440 con 0x7f8060070890 2024-04-25T11:19:44.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.398+0000 7f8057400640 1 -- 172.21.15.7:0/1026190239 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+13078 (secure 0 0 0) 0x7f805005f570 con 0x7f8060070890 2024-04-25T11:19:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:44.396 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":114,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:43.183943+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 -- 172.21.15.7:0/1026190239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 msgr2=0x7f803c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 0x7f803c0769f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8060068b10 tx=0x7f8048008040 comp rx=0 tx=0).stop 2024-04-25T11:19:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 -- 172.21.15.7:0/1026190239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060070890 msgr2=0x7f806015c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060070890 0x7f806015c9b0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f805000c410 tx=0x7f805000c8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 -- 172.21.15.7:0/1026190239 shutdown_connections 2024-04-25T11:19:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f803c0744e0 0x7f803c0769f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f806015c390 0x7f80600d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80600b03b0 0x7f8060141b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 --2- 172.21.15.7:0/1026190239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060070890 0x7f806015c9b0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 -- 172.21.15.7:0/1026190239 >> 172.21.15.7:0/1026190239 conn(0x7f80600b3820 msgr2=0x7f8060152e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 -- 172.21.15.7:0/1026190239 shutdown_connections 2024-04-25T11:19:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.401+0000 7f8064a00640 1 -- 172.21.15.7:0/1026190239 wait complete. 2024-04-25T11:19:44.423 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:19:44.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:19:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.702+0000 7f1221200640 1 Processor -- start 2024-04-25T11:19:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.702+0000 7f1221200640 1 -- start start 2024-04-25T11:19:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f1221200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f121c0cff00 0x7f121c0d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f1221200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f121c06f330 0x7f121c06f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f1221200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f121c0d09e0 con 0x7f121c167de0 2024-04-25T11:19:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f1221200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f121c06eea0 con 0x7f121c0cff00 2024-04-25T11:19:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f1221200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f121c0d0cc0 con 0x7f121c06f330 2024-04-25T11:19:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f121b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f121c0cff00 0x7f121c0d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f121b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f121c0cff00 0x7f121c0d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33796/0 (socket says 172.21.15.7:33796) 2024-04-25T11:19:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f121b400640 1 -- 172.21.15.7:0/1265239301 learned_addr learned my addr 172.21.15.7:0/1265239301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f121b400640 1 -- 172.21.15.7:0/1265239301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f121c06f330 msgr2=0x7f121c06f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f121b400640 1 --2- 172.21.15.7:0/1265239301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f121c06f330 0x7f121c06f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f121b400640 1 -- 172.21.15.7:0/1265239301 >> v1:172.21.15.87:6789/0 conn(0x7f121c167de0 legacy=0x7f121c1681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.705+0000 7f121b400640 1 -- 172.21.15.7:0/1265239301 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c0d0fa0 con 0x7f121c0cff00 2024-04-25T11:19:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.706+0000 7f121b400640 1 --2- 172.21.15.7:0/1265239301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f121c0cff00 0x7f121c0d0310 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f121c072330 tx=0x7f120c02a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5f0062f0aa7d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.707+0000 7f121aa00640 1 -- 172.21.15.7:0/1265239301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c037070 con 0x7f121c0cff00 2024-04-25T11:19:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.707+0000 7f121aa00640 1 -- 172.21.15.7:0/1265239301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f120c032440 con 0x7f121c0cff00 2024-04-25T11:19:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.707+0000 7f121aa00640 1 -- 172.21.15.7:0/1265239301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c032810 con 0x7f121c0cff00 2024-04-25T11:19:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.708+0000 7f1221200640 1 -- 172.21.15.7:0/1265239301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f121c0cff00 msgr2=0x7f121c0d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.708+0000 7f1221200640 1 --2- 172.21.15.7:0/1265239301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f121c0cff00 0x7f121c0d0310 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f121c072330 tx=0x7f120c02a1f0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.708+0000 7f1221200640 1 -- 172.21.15.7:0/1265239301 shutdown_connections 2024-04-25T11:19:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.708+0000 7f1221200640 1 --2- 172.21.15.7:0/1265239301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f121c06f330 0x7f121c06f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.708+0000 7f1221200640 1 --2- 172.21.15.7:0/1265239301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f121c0cff00 0x7f121c0d0310 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.708+0000 7f1221200640 1 -- 172.21.15.7:0/1265239301 >> 172.21.15.7:0/1265239301 conn(0x7f121c0740f0 msgr2=0x7f121c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.708+0000 7f1221200640 1 -- 172.21.15.7:0/1265239301 shutdown_connections 2024-04-25T11:19:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.708+0000 7f1221200640 1 -- 172.21.15.7:0/1265239301 wait complete. 2024-04-25T11:19:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.710+0000 7f1221200640 1 Processor -- start 2024-04-25T11:19:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.710+0000 7f1221200640 1 -- start start 2024-04-25T11:19:44.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f1221200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f121c06f330 0x7f121c1fac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f1221200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f121c0cff00 0x7f121c140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f1221200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c167de0 0x7f121c147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f1221200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f121c168bb0 con 0x7f121c0cff00 2024-04-25T11:19:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f1221200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f121c168a30 con 0x7f121c167de0 2024-04-25T11:19:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f1221200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f121c147340 con 0x7f121c06f330 2024-04-25T11:19:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f1212a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f121c0cff00 0x7f121c140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f1212a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f121c0cff00 0x7f121c140590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33804/0 (socket says 172.21.15.7:33804) 2024-04-25T11:19:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f1212a00640 1 -- 172.21.15.7:0/1852141539 learned_addr learned my addr 172.21.15.7:0/1852141539 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:44.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f1212a00640 1 -- 172.21.15.7:0/1852141539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f121c06f330 msgr2=0x7f121c1fac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f121b400640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f121c06f330 0x7f121c1fac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f121be00640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c167de0 0x7f121c147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.712+0000 7f1212a00640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f121c06f330 0x7f121c1fac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.713+0000 7f1212a00640 1 -- 172.21.15.7:0/1852141539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c167de0 msgr2=0x7f121c147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.713+0000 7f1212a00640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c167de0 0x7f121c147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.713+0000 7f1212a00640 1 -- 172.21.15.7:0/1852141539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c15a6b0 con 0x7f121c0cff00 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.713+0000 7f121be00640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c167de0 0x7f121c147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.713+0000 7f1212a00640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f121c0cff00 0x7f121c140590 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f120400b410 tx=0x7f120400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.713+0000 7f1213e00640 1 -- 172.21.15.7:0/1852141539 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204015020 con 0x7f121c0cff00 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.713+0000 7f1213e00640 1 -- 172.21.15.7:0/1852141539 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1204011830 con 0x7f121c0cff00 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.713+0000 7f1213e00640 1 -- 172.21.15.7:0/1852141539 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204011c20 con 0x7f121c0cff00 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.713+0000 7f1221200640 1 -- 172.21.15.7:0/1852141539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121c148250 con 0x7f121c0cff00 2024-04-25T11:19:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.714+0000 7f1221200640 1 -- 172.21.15.7:0/1852141539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121c148810 con 0x7f121c0cff00 2024-04-25T11:19:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.715+0000 7f1221200640 1 -- 172.21.15.7:0/1852141539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f121c147fc0 con 0x7f121c0cff00 2024-04-25T11:19:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.716+0000 7f1213e00640 1 -- 172.21.15.7:0/1852141539 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1204018020 con 0x7f121c0cff00 2024-04-25T11:19:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.717+0000 7f1213e00640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11f80744b0 0x7f11f80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.717+0000 7f1213e00640 1 -- 172.21.15.7:0/1852141539 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5249+0+0 (secure 0 0 0) 0x7f12040950a0 con 0x7f121c0cff00 2024-04-25T11:19:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.717+0000 7f121b400640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11f80744b0 0x7f11f80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:44.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.718+0000 7f121b400640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11f80744b0 0x7f11f80769c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f120c034000 tx=0x7f120c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:44.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.722+0000 7f1213e00640 1 -- 172.21.15.7:0/1852141539 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f120401f020 con 0x7f121c0cff00 2024-04-25T11:19:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.926+0000 7f1221200640 1 -- 172.21.15.7:0/1852141539 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f121c050740 con 0x7f121c0cff00 2024-04-25T11:19:44.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.927+0000 7f1213e00640 1 -- 172.21.15.7:0/1852141539 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7f121c15a6b0 con 0x7f121c0cff00 2024-04-25T11:19:44.925 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:19:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.930+0000 7f1221200640 1 -- 172.21.15.7:0/1852141539 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11f80744b0 msgr2=0x7f11f80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.930+0000 7f1221200640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11f80744b0 0x7f11f80769c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f120c034000 tx=0x7f120c02e000 comp rx=0 tx=0).stop 2024-04-25T11:19:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.930+0000 7f1221200640 1 -- 172.21.15.7:0/1852141539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f121c0cff00 msgr2=0x7f121c140590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.930+0000 7f1221200640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f121c0cff00 0x7f121c140590 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f120400b410 tx=0x7f120400b8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.930+0000 7f1221200640 1 -- 172.21.15.7:0/1852141539 shutdown_connections 2024-04-25T11:19:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.930+0000 7f1221200640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f11f80744b0 0x7f11f80769c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.930+0000 7f1221200640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f121c167de0 0x7f121c147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.930+0000 7f1221200640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f121c0cff00 0x7f121c140590 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.930+0000 7f1221200640 1 --2- 172.21.15.7:0/1852141539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f121c06f330 0x7f121c1fac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:44.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.930+0000 7f1221200640 1 -- 172.21.15.7:0/1852141539 >> 172.21.15.7:0/1852141539 conn(0x7f121c0740f0 msgr2=0x7f121c0b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.931+0000 7f1221200640 1 -- 172.21.15.7:0/1852141539 shutdown_connections 2024-04-25T11:19:44.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:44.931+0000 7f1221200640 1 -- 172.21.15.7:0/1852141539 wait complete. 2024-04-25T11:19:44.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:19:45.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.230+0000 7f2fd3e00640 1 Processor -- start 2024-04-25T11:19:45.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.230+0000 7f2fd3e00640 1 -- start start 2024-04-25T11:19:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.232+0000 7f2fd3e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fd415baa0 0x7f2fd415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.232+0000 7f2fd3e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd415e980 con 0x7f2fd4070640 2024-04-25T11:19:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.232+0000 7f2fd3e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd415ec60 con 0x7f2fd406d580 2024-04-25T11:19:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.232+0000 7f2fd3e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd415ef40 con 0x7f2fd415baa0 2024-04-25T11:19:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.233+0000 7f2fd2000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2fd4070640 0x7f2fd4161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55292/0 (socket says 172.21.15.7:55292) 2024-04-25T11:19:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.233+0000 7f2fd2000640 1 -- 172.21.15.7:0/1392357932 learned_addr learned my addr 172.21.15.7:0/1392357932 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.233+0000 7f2fd2a00640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fd415baa0 0x7f2fd415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.233+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2408025939 0 0) 0x7f2fd415e980 con 0x7f2fd4070640 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.233+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fb8003520 con 0x7f2fd4070640 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4283268018 0 0) 0x7f2fb8003520 con 0x7f2fd4070640 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fd415baa0 msgr2=0x7f2fd415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fd415baa0 0x7f2fd415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 >> v1:172.21.15.87:6789/0 conn(0x7f2fd406d580 legacy=0x7f2fd406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd41619c0 con 0x7f2fd4070640 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2fc0001670 con 0x7f2fd4070640 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 >> v1:172.21.15.7:6789/0 conn(0x7f2fd4070640 legacy=0x7f2fd4161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fb80039e0 0x7f2fb8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fb8004370 0x7f2fb80048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb80050c0 0x7f2fb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd415ef40 con 0x7f2fb80050c0 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb800bf60 con 0x7f2fb8004370 2024-04-25T11:19:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb800c160 con 0x7f2fb80039e0 2024-04-25T11:19:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.234+0000 7f2fd3400640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb80050c0 0x7f2fb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.235+0000 7f2fd2000640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fb8004370 0x7f2fb80048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.235+0000 7f2fd2a00640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fb80039e0 0x7f2fb8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.235+0000 7f2fd3400640 1 -- 172.21.15.7:0/1392357932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fb80039e0 msgr2=0x7f2fb8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.235+0000 7f2fd3400640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fb80039e0 0x7f2fb8003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.235+0000 7f2fd3400640 1 -- 172.21.15.7:0/1392357932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fb8004370 msgr2=0x7f2fb80048c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.235+0000 7f2fd3400640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fb8004370 0x7f2fb80048c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.235+0000 7f2fd3400640 1 -- 172.21.15.7:0/1392357932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb8003520 con 0x7f2fb80050c0 2024-04-25T11:19:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.236+0000 7f2fd3400640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb80050c0 0x7f2fb800b9c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4073040 tx=0x7f2fb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c825d05b34abdb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.236+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc001a70 con 0x7f2fb80050c0 2024-04-25T11:19:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.236+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fcc02add0 con 0x7f2fb80050c0 2024-04-25T11:19:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.236+0000 7f2fd1600640 1 -- 172.21.15.7:0/1392357932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc003bb0 con 0x7f2fb80050c0 2024-04-25T11:19:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.237+0000 7f2fd3e00640 1 -- 172.21.15.7:0/1392357932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb80050c0 msgr2=0x7f2fb800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.237+0000 7f2fd3e00640 1 --2- 172.21.15.7:0/1392357932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fb80050c0 0x7f2fb800b9c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4073040 tx=0x7f2fb8005830 comp rx=0 tx=0).stop 2024-04-25T11:19:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.237+0000 7f2fd3400640 1 -- 172.21.15.7:0/1392357932 reap_dead start 2024-04-25T11:19:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.237+0000 7f2fd3e00640 1 -- 172.21.15.7:0/1392357932 shutdown_connections 2024-04-25T11:19:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.237+0000 7f2fd3e00640 1 -- 172.21.15.7:0/1392357932 >> 172.21.15.7:0/1392357932 conn(0x7f2fd4073090 msgr2=0x7f2fd40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.237+0000 7f2fd3e00640 1 -- 172.21.15.7:0/1392357932 shutdown_connections 2024-04-25T11:19:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.237+0000 7f2fd3e00640 1 -- 172.21.15.7:0/1392357932 wait complete. 2024-04-25T11:19:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.240+0000 7f2fd3e00640 1 Processor -- start 2024-04-25T11:19:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.240+0000 7f2fd3e00640 1 -- start start 2024-04-25T11:19:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.241+0000 7f2fd3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fb80050c0 0x7f2fd41f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.241+0000 7f2fd3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd41421b0 0x7f2fd41425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd3e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fd4145b70 0x7f2fd4145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd3e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd4162770 con 0x7f2fd41421b0 2024-04-25T11:19:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd3e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd41625f0 con 0x7f2fb80050c0 2024-04-25T11:19:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd3e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd415baa0 con 0x7f2fd4145b70 2024-04-25T11:19:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd41421b0 0x7f2fd41425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd41421b0 0x7f2fd41425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33826/0 (socket says 172.21.15.7:33826) 2024-04-25T11:19:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd2000640 1 -- 172.21.15.7:0/662411778 learned_addr learned my addr 172.21.15.7:0/662411778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd2a00640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fb80050c0 0x7f2fd41f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd3400640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fd4145b70 0x7f2fd4145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd2000640 1 -- 172.21.15.7:0/662411778 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fd4145b70 msgr2=0x7f2fd4145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd2000640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fd4145b70 0x7f2fd4145f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd2000640 1 -- 172.21.15.7:0/662411778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fb80050c0 msgr2=0x7f2fd41f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd2000640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fb80050c0 0x7f2fd41f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.242+0000 7f2fd2000640 1 -- 172.21.15.7:0/662411778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd41464c0 con 0x7f2fd41421b0 2024-04-25T11:19:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.243+0000 7f2fd2000640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd41421b0 0x7f2fd41425c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000e800 tx=0x7f2fc000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.243+0000 7f2fcaa00640 1 -- 172.21.15.7:0/662411778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000c870 con 0x7f2fd41421b0 2024-04-25T11:19:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.243+0000 7f2fcaa00640 1 -- 172.21.15.7:0/662411778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc0018070 con 0x7f2fd41421b0 2024-04-25T11:19:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.243+0000 7f2fcaa00640 1 -- 172.21.15.7:0/662411778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000cc70 con 0x7f2fd41421b0 2024-04-25T11:19:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.243+0000 7f2fd3e00640 1 -- 172.21.15.7:0/662411778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd41f6e60 con 0x7f2fd41421b0 2024-04-25T11:19:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.243+0000 7f2fd3e00640 1 -- 172.21.15.7:0/662411778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd41f7420 con 0x7f2fd41421b0 2024-04-25T11:19:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.244+0000 7f2fd3e00640 1 -- 172.21.15.7:0/662411778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd4050740 con 0x7f2fd41421b0 2024-04-25T11:19:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.245+0000 7f2fcaa00640 1 -- 172.21.15.7:0/662411778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2fc001d460 con 0x7f2fd41421b0 2024-04-25T11:19:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.246+0000 7f2fcaa00640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fb40744b0 0x7f2fb40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.246+0000 7f2fcaa00640 1 -- 172.21.15.7:0/662411778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5249+0+0 (secure 0 0 0) 0x7f2fc0094e30 con 0x7f2fd41421b0 2024-04-25T11:19:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.247+0000 7f2fd2a00640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fb40744b0 0x7f2fb40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.247+0000 7f2fd2a00640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fb40744b0 0x7f2fb40769c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009a10 tx=0x7f2fc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:45.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.251+0000 7f2fcaa00640 1 -- 172.21.15.7:0/662411778 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fd41464c0 con 0x7f2fd41421b0 2024-04-25T11:19:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.457+0000 7f2fd3e00640 1 -- 172.21.15.7:0/662411778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2fd4074020 con 0x7f2fd41421b0 2024-04-25T11:19:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.458+0000 7f2fcaa00640 1 -- 172.21.15.7:0/662411778 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v114) ==== 84+0+65 (secure 0 0 0) 0x7f2fc005f800 con 0x7f2fd41421b0 2024-04-25T11:19:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:45.455 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-25T11:19:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:19:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-25T11:19:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-25T11:19:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:19:45.456 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.460+0000 7f2fd3e00640 1 -- 172.21.15.7:0/662411778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fb40744b0 msgr2=0x7f2fb40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.460+0000 7f2fd3e00640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fb40744b0 0x7f2fb40769c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009a10 tx=0x7f2fc4008040 comp rx=0 tx=0).stop 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.460+0000 7f2fd3e00640 1 -- 172.21.15.7:0/662411778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd41421b0 msgr2=0x7f2fd41425c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.460+0000 7f2fd3e00640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd41421b0 0x7f2fd41425c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000e800 tx=0x7f2fc000ecc0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.460+0000 7f2fd3e00640 1 -- 172.21.15.7:0/662411778 shutdown_connections 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.460+0000 7f2fd3e00640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fb40744b0 0x7f2fb40769c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.460+0000 7f2fd3e00640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fd4145b70 0x7f2fd4145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.460+0000 7f2fd3e00640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fd41421b0 0x7f2fd41425c0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.460+0000 7f2fd3e00640 1 --2- 172.21.15.7:0/662411778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fb80050c0 0x7f2fd41f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.460+0000 7f2fd3e00640 1 -- 172.21.15.7:0/662411778 >> 172.21.15.7:0/662411778 conn(0x7f2fd4073090 msgr2=0x7f2fd40cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.461+0000 7f2fd3e00640 1 -- 172.21.15.7:0/662411778 shutdown_connections 2024-04-25T11:19:45.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.461+0000 7f2fd3e00640 1 -- 172.21.15.7:0/662411778 wait complete. 2024-04-25T11:19:45.483 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:19:45.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:19:45.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.769+0000 7f7253200640 1 Processor -- start 2024-04-25T11:19:45.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.770+0000 7f7253200640 1 -- start start 2024-04-25T11:19:45.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.772+0000 7f7253200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f725414d090 0x7f725406eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.772+0000 7f7253200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f725406d580 0x7f725406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.772+0000 7f7253200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f725406ded0 con 0x7f72540cee40 2024-04-25T11:19:45.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.772+0000 7f7253200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f725406f4c0 con 0x7f725406d580 2024-04-25T11:19:45.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.772+0000 7f7253200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f725406f7a0 con 0x7f725414d090 2024-04-25T11:19:45.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.772+0000 7f724be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f72540cee40 0x7f7254070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55294/0 (socket says 172.21.15.7:55294) 2024-04-25T11:19:45.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.772+0000 7f724be00640 1 -- 172.21.15.7:0/3095017184 learned_addr learned my addr 172.21.15.7:0/3095017184 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:45.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.773+0000 7f7252800640 1 --2- 172.21.15.7:0/3095017184 >> v2:172.21.15.7:3300/0 conn(0x7f725406d580 0x7f725406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.773+0000 7f7251e00640 1 --2- 172.21.15.7:0/3095017184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f725414d090 0x7f725406eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.773+0000 7f7251400640 1 -- 172.21.15.7:0/3095017184 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1368360370 0 0) 0x7f723c001110 con 0x7f72540cee40 2024-04-25T11:19:45.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.773+0000 7f7251400640 1 -- 172.21.15.7:0/3095017184 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7238003520 con 0x7f72540cee40 2024-04-25T11:19:45.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.773+0000 7f7252800640 1 -- 172.21.15.7:0/3095017184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f725414d090 msgr2=0x7f725406eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:45.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.773+0000 7f7252800640 1 --2- 172.21.15.7:0/3095017184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f725414d090 0x7f725406eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.773+0000 7f7252800640 1 -- 172.21.15.7:0/3095017184 >> v1:172.21.15.7:6789/0 conn(0x7f72540cee40 legacy=0x7f7254070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.773+0000 7f7252800640 1 -- 172.21.15.7:0/3095017184 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725406fa80 con 0x7f725406d580 2024-04-25T11:19:45.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.774+0000 7f7252800640 1 --2- 172.21.15.7:0/3095017184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f725406d580 0x7f725406d990 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f724c009d90 tx=0x7f724c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2360effad4d3357f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:45.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.774+0000 7f7251400640 1 -- 172.21.15.7:0/3095017184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c031600 con 0x7f725406d580 2024-04-25T11:19:45.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.775+0000 7f7251400640 1 -- 172.21.15.7:0/3095017184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f724c0317a0 con 0x7f725406d580 2024-04-25T11:19:45.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.775+0000 7f7251400640 1 -- 172.21.15.7:0/3095017184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c031b70 con 0x7f725406d580 2024-04-25T11:19:45.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.775+0000 7f7253200640 1 -- 172.21.15.7:0/3095017184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f725406d580 msgr2=0x7f725406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.775+0000 7f7253200640 1 --2- 172.21.15.7:0/3095017184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f725406d580 0x7f725406d990 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f724c009d90 tx=0x7f724c02a5d0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.775+0000 7f7253200640 1 -- 172.21.15.7:0/3095017184 shutdown_connections 2024-04-25T11:19:45.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.775+0000 7f7253200640 1 --2- 172.21.15.7:0/3095017184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f725406d580 0x7f725406d990 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.775+0000 7f7253200640 1 --2- 172.21.15.7:0/3095017184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f725414d090 0x7f725406eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.775+0000 7f7253200640 1 -- 172.21.15.7:0/3095017184 >> 172.21.15.7:0/3095017184 conn(0x7f7254073160 msgr2=0x7f7254073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.776+0000 7f7253200640 1 -- 172.21.15.7:0/3095017184 shutdown_connections 2024-04-25T11:19:45.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.776+0000 7f7253200640 1 -- 172.21.15.7:0/3095017184 wait complete. 2024-04-25T11:19:45.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.778+0000 7f7253200640 1 Processor -- start 2024-04-25T11:19:45.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.778+0000 7f7253200640 1 -- start start 2024-04-25T11:19:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f7253200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f725406d580 0x7f7254166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f7253200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72540cee40 0x7f7254164930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f7253200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725414d090 0x7f7254146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f7253200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72540cf250 con 0x7f72540cee40 2024-04-25T11:19:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f7253200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f72540b4690 con 0x7f725414d090 2024-04-25T11:19:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f7253200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f72540b4bb0 con 0x7f725406d580 2024-04-25T11:19:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f724be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72540cee40 0x7f7254164930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f724be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72540cee40 0x7f7254164930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33844/0 (socket says 172.21.15.7:33844) 2024-04-25T11:19:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f724be00640 1 -- 172.21.15.7:0/3015346441 learned_addr learned my addr 172.21.15.7:0/3015346441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:45.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f7251e00640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f725406d580 0x7f7254166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f7252800640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725414d090 0x7f7254146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f724be00640 1 -- 172.21.15.7:0/3015346441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f725406d580 msgr2=0x7f7254166ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f724be00640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f725406d580 0x7f7254166ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f724be00640 1 -- 172.21.15.7:0/3015346441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725414d090 msgr2=0x7f7254146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.780+0000 7f724be00640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725414d090 0x7f7254146830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.781+0000 7f724be00640 1 -- 172.21.15.7:0/3015346441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7254165e40 con 0x7f72540cee40 2024-04-25T11:19:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.781+0000 7f724be00640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72540cee40 0x7f7254164930 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f723c00c340 tx=0x7f723c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.781+0000 7f724a000640 1 -- 172.21.15.7:0/3015346441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c017070 con 0x7f72540cee40 2024-04-25T11:19:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.781+0000 7f724a000640 1 -- 172.21.15.7:0/3015346441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f723c005ce0 con 0x7f72540cee40 2024-04-25T11:19:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.781+0000 7f724a000640 1 -- 172.21.15.7:0/3015346441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c0072f0 con 0x7f72540cee40 2024-04-25T11:19:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.781+0000 7f7253200640 1 -- 172.21.15.7:0/3015346441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7254166120 con 0x7f72540cee40 2024-04-25T11:19:45.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.781+0000 7f7253200640 1 -- 172.21.15.7:0/3015346441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72541666e0 con 0x7f72540cee40 2024-04-25T11:19:45.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.783+0000 7f7253200640 1 -- 172.21.15.7:0/3015346441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7254050740 con 0x7f72540cee40 2024-04-25T11:19:45.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.784+0000 7f724a000640 1 -- 172.21.15.7:0/3015346441 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f723c01a070 con 0x7f72540cee40 2024-04-25T11:19:45.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.785+0000 7f724a000640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72300744b0 0x7f72300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:45.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.785+0000 7f724a000640 1 -- 172.21.15.7:0/3015346441 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5249+0+0 (secure 0 0 0) 0x7f723c094510 con 0x7f72540cee40 2024-04-25T11:19:45.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.785+0000 7f7251e00640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72300744b0 0x7f72300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:45.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.786+0000 7f7251e00640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72300744b0 0x7f72300769c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7244009a30 tx=0x7f7244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:45.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.790+0000 7f724a000640 1 -- 172.21.15.7:0/3015346441 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f725406ded0 con 0x7f72540cee40 2024-04-25T11:19:45.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:45.995+0000 7f7253200640 1 -- 172.21.15.7:0/3015346441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f72540cfec0 con 0x7f72540cee40 2024-04-25T11:19:46.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.213+0000 7f724a000640 1 -- 172.21.15.7:0/3015346441 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v115) ==== 138+0+0 (secure 0 0 0) 0x7f723c05edc0 con 0x7f72540cee40 2024-04-25T11:19:46.212 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-25T11:19:46.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.218+0000 7f7253200640 1 -- 172.21.15.7:0/3015346441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72300744b0 msgr2=0x7f72300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:46.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.218+0000 7f7253200640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72300744b0 0x7f72300769c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7244009a30 tx=0x7f7244008040 comp rx=0 tx=0).stop 2024-04-25T11:19:46.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.218+0000 7f7253200640 1 -- 172.21.15.7:0/3015346441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72540cee40 msgr2=0x7f7254164930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:46.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.218+0000 7f7253200640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72540cee40 0x7f7254164930 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f723c00c340 tx=0x7f723c00c800 comp rx=0 tx=0).stop 2024-04-25T11:19:46.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.218+0000 7f7253200640 1 -- 172.21.15.7:0/3015346441 shutdown_connections 2024-04-25T11:19:46.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.218+0000 7f7253200640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f72300744b0 0x7f72300769c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:46.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.218+0000 7f7253200640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f725414d090 0x7f7254146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:46.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.218+0000 7f7253200640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72540cee40 0x7f7254164930 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:46.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.218+0000 7f7253200640 1 --2- 172.21.15.7:0/3015346441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f725406d580 0x7f7254166ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:46.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.218+0000 7f7253200640 1 -- 172.21.15.7:0/3015346441 >> 172.21.15.7:0/3015346441 conn(0x7f7254073160 msgr2=0x7f7254073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:46.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.219+0000 7f7253200640 1 -- 172.21.15.7:0/3015346441 shutdown_connections 2024-04-25T11:19:46.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.219+0000 7f7253200640 1 -- 172.21.15.7:0/3015346441 wait complete. 2024-04-25T11:19:46.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:19:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.541+0000 7f1728600640 1 Processor -- start 2024-04-25T11:19:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.541+0000 7f1728600640 1 -- start start 2024-04-25T11:19:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.544+0000 7f1728600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17200b9190 0x7f17200d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.544+0000 7f1728600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17200d2b90 0x7f17200d4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.544+0000 7f1728600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1720163e20 con 0x7f1720160410 2024-04-25T11:19:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.544+0000 7f1728600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1720164100 con 0x7f17200b9190 2024-04-25T11:19:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.544+0000 7f1728600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f17201643e0 con 0x7f17200d2b90 2024-04-25T11:19:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.544+0000 7f1725200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17200b9190 0x7f17200d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.544+0000 7f1725200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17200b9190 0x7f17200d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33856/0 (socket says 172.21.15.7:33856) 2024-04-25T11:19:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.544+0000 7f1725200640 1 -- 172.21.15.7:0/405649191 learned_addr learned my addr 172.21.15.7:0/405649191 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.544+0000 7f1725c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1720160410 0x7f17200d2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55308/0 (socket says 172.21.15.7:55308) 2024-04-25T11:19:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.545+0000 7f1726600640 1 --2- 172.21.15.7:0/405649191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17200d2b90 0x7f17200d4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.545+0000 7f1725200640 1 -- 172.21.15.7:0/405649191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17200d2b90 msgr2=0x7f17200d4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.545+0000 7f1725200640 1 --2- 172.21.15.7:0/405649191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17200d2b90 0x7f17200d4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.545+0000 7f1725200640 1 -- 172.21.15.7:0/405649191 >> v1:172.21.15.7:6789/0 conn(0x7f1720160410 legacy=0x7f17200d2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.545+0000 7f1725200640 1 -- 172.21.15.7:0/405649191 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17201646c0 con 0x7f17200b9190 2024-04-25T11:19:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.545+0000 7f171fe00640 1 -- 172.21.15.7:0/405649191 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2880125317 0 0) 0x7f1720163e20 con 0x7f1720160410 2024-04-25T11:19:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.546+0000 7f1725200640 1 --2- 172.21.15.7:0/405649191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17200b9190 0x7f17200d4660 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f170c009870 tx=0x7f170c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6402fbf67008a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.547+0000 7f171fe00640 1 -- 172.21.15.7:0/405649191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c037070 con 0x7f17200b9190 2024-04-25T11:19:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.547+0000 7f171fe00640 1 -- 172.21.15.7:0/405649191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170c032440 con 0x7f17200b9190 2024-04-25T11:19:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.547+0000 7f171fe00640 1 -- 172.21.15.7:0/405649191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c032810 con 0x7f17200b9190 2024-04-25T11:19:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.547+0000 7f1728600640 1 -- 172.21.15.7:0/405649191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17200b9190 msgr2=0x7f17200d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.547+0000 7f1728600640 1 --2- 172.21.15.7:0/405649191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17200b9190 0x7f17200d4660 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f170c009870 tx=0x7f170c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:19:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.548+0000 7f1728600640 1 -- 172.21.15.7:0/405649191 shutdown_connections 2024-04-25T11:19:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.548+0000 7f1728600640 1 --2- 172.21.15.7:0/405649191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17200d2b90 0x7f17200d4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.548+0000 7f1728600640 1 --2- 172.21.15.7:0/405649191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17200b9190 0x7f17200d4660 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.548+0000 7f1728600640 1 -- 172.21.15.7:0/405649191 >> 172.21.15.7:0/405649191 conn(0x7f172006c750 msgr2=0x7f172006cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.548+0000 7f1728600640 1 -- 172.21.15.7:0/405649191 shutdown_connections 2024-04-25T11:19:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.548+0000 7f1728600640 1 -- 172.21.15.7:0/405649191 wait complete. 2024-04-25T11:19:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.551+0000 7f1728600640 1 Processor -- start 2024-04-25T11:19:46.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.552+0000 7f1728600640 1 -- start start 2024-04-25T11:19:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.554+0000 7f1728600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17200b9190 0x7f17201541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.554+0000 7f1728600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17200d2b90 0x7f17201eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.554+0000 7f1728600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1720160410 0x7f17200b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.554+0000 7f1728600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17200d17d0 con 0x7f17200d2b90 2024-04-25T11:19:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.554+0000 7f1728600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f17200d1650 con 0x7f17200b9190 2024-04-25T11:19:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.554+0000 7f1728600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f17200b4860 con 0x7f1720160410 2024-04-25T11:19:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.554+0000 7f1725200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17200d2b90 0x7f17201eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.554+0000 7f1725200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17200d2b90 0x7f17201eb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33870/0 (socket says 172.21.15.7:33870) 2024-04-25T11:19:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.554+0000 7f1725200640 1 -- 172.21.15.7:0/2982446894 learned_addr learned my addr 172.21.15.7:0/2982446894 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.555+0000 7f1725200640 1 -- 172.21.15.7:0/2982446894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1720160410 msgr2=0x7f17200b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.555+0000 7f1725200640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1720160410 0x7f17200b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.555+0000 7f1725200640 1 -- 172.21.15.7:0/2982446894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17200b9190 msgr2=0x7f17201541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.555+0000 7f1725c00640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17200b9190 0x7f17201541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.555+0000 7f1725200640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17200b9190 0x7f17201541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.555+0000 7f1725200640 1 -- 172.21.15.7:0/2982446894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17201ea010 con 0x7f17200d2b90 2024-04-25T11:19:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.555+0000 7f1725c00640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17200b9190 0x7f17201541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.555+0000 7f1725200640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17200d2b90 0x7f17201eb0b0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f170c02a200 tx=0x7f170c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:46.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.555+0000 7f171d600640 1 -- 172.21.15.7:0/2982446894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c041020 con 0x7f17200d2b90 2024-04-25T11:19:46.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.555+0000 7f171d600640 1 -- 172.21.15.7:0/2982446894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f170c0306c0 con 0x7f17200d2b90 2024-04-25T11:19:46.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.556+0000 7f1728600640 1 -- 172.21.15.7:0/2982446894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17201ea2f0 con 0x7f17200d2b90 2024-04-25T11:19:46.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.556+0000 7f171d600640 1 -- 172.21.15.7:0/2982446894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170c037040 con 0x7f17200d2b90 2024-04-25T11:19:46.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.556+0000 7f1728600640 1 -- 172.21.15.7:0/2982446894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17201ea8b0 con 0x7f17200d2b90 2024-04-25T11:19:46.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.557+0000 7f1728600640 1 -- 172.21.15.7:0/2982446894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17201e9d80 con 0x7f17200d2b90 2024-04-25T11:19:46.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.558+0000 7f171d600640 1 -- 172.21.15.7:0/2982446894 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f170c044020 con 0x7f17200d2b90 2024-04-25T11:19:46.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.559+0000 7f171d600640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17000744b0 0x7f17000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:46.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.559+0000 7f171d600640 1 -- 172.21.15.7:0/2982446894 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5619+0+0 (secure 0 0 0) 0x7f170c0b5b10 con 0x7f17200d2b90 2024-04-25T11:19:46.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.560+0000 7f1725c00640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17000744b0 0x7f17000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:46.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.561+0000 7f1725c00640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17000744b0 0x7f17000769c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1714000f60 tx=0x7f1714009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:46.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.565+0000 7f171d600640 1 -- 172.21.15.7:0/2982446894 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f170c04b030 con 0x7f17200d2b90 2024-04-25T11:19:46.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:46.772+0000 7f1728600640 1 -- 172.21.15.7:0/2982446894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1720050740 con 0x7f17200d2b90 2024-04-25T11:19:47.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.221+0000 7f171d600640 1 -- 172.21.15.7:0/2982446894 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v116) ==== 170+0+0 (secure 0 0 0) 0x7f170c080250 con 0x7f17200d2b90 2024-04-25T11:19:47.219 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-25T11:19:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.225+0000 7f1728600640 1 -- 172.21.15.7:0/2982446894 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17000744b0 msgr2=0x7f17000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.225+0000 7f1728600640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17000744b0 0x7f17000769c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1714000f60 tx=0x7f1714009040 comp rx=0 tx=0).stop 2024-04-25T11:19:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.225+0000 7f1728600640 1 -- 172.21.15.7:0/2982446894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17200d2b90 msgr2=0x7f17201eb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.225+0000 7f1728600640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17200d2b90 0x7f17201eb0b0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f170c02a200 tx=0x7f170c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.226+0000 7f1728600640 1 -- 172.21.15.7:0/2982446894 shutdown_connections 2024-04-25T11:19:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.226+0000 7f1728600640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17000744b0 0x7f17000769c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.226+0000 7f1728600640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1720160410 0x7f17200b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.226+0000 7f1728600640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17200d2b90 0x7f17201eb0b0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.226+0000 7f1728600640 1 --2- 172.21.15.7:0/2982446894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17200b9190 0x7f17201541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.226+0000 7f1728600640 1 -- 172.21.15.7:0/2982446894 >> 172.21.15.7:0/2982446894 conn(0x7f172006c750 msgr2=0x7f172015c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:47.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.226+0000 7f1728600640 1 -- 172.21.15.7:0/2982446894 shutdown_connections 2024-04-25T11:19:47.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.227+0000 7f1728600640 1 -- 172.21.15.7:0/2982446894 wait complete. 2024-04-25T11:19:47.278 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:19:47.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.536+0000 7fa2d7600640 1 Processor -- start 2024-04-25T11:19:47.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.536+0000 7fa2d7600640 1 -- start start 2024-04-25T11:19:47.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.538+0000 7fa2d7600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2d015aaa0 0x7fa2d00affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:47.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.538+0000 7fa2d7600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2d006f5b0 con 0x7fa2d00d0ee0 2024-04-25T11:19:47.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.538+0000 7fa2d7600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2d00d14a0 con 0x7fa2d00b5020 2024-04-25T11:19:47.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.538+0000 7fa2d7600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d00af290 con 0x7fa2d015aaa0 2024-04-25T11:19:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.539+0000 7fa2cfe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa2d00d0ee0 0x7fa2d00b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55324/0 (socket says 172.21.15.7:55324) 2024-04-25T11:19:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.539+0000 7fa2cfe00640 1 -- 172.21.15.7:0/2208055285 learned_addr learned my addr 172.21.15.7:0/2208055285 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.539+0000 7fa2d4c00640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2d015aaa0 0x7fa2d00affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.539+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569833688 0 0) 0x7fa2d006f5b0 con 0x7fa2d00d0ee0 2024-04-25T11:19:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.539+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2b4005730 con 0x7fa2d00d0ee0 2024-04-25T11:19:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4111546469 0 0) 0x7fa2b4005730 con 0x7fa2d00d0ee0 2024-04-25T11:19:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2d015aaa0 msgr2=0x7fa2d00affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2d015aaa0 0x7fa2d00affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 >> v1:172.21.15.87:6789/0 conn(0x7fa2d00b5020 legacy=0x7fa2d0160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d00af4e0 con 0x7fa2d00d0ee0 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fa2bc0019f0 con 0x7fa2d00d0ee0 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 >> v1:172.21.15.7:6789/0 conn(0x7fa2d00d0ee0 legacy=0x7fa2d00b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2d015aaa0 0x7fa2b4005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b4006170 0x7fa2b4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b4006f50 0x7fa2b4007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d00af290 con 0x7fa2b4006170 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b400dd60 con 0x7fa2b4006f50 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b400df60 con 0x7fa2d015aaa0 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2cfe00640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b4006170 0x7fa2b4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.540+0000 7fa2d4c00640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2d015aaa0 0x7fa2b4005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.541+0000 7fa2cfe00640 1 -- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2d015aaa0 msgr2=0x7fa2b4005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.541+0000 7fa2cfe00640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2d015aaa0 0x7fa2b4005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.541+0000 7fa2d5600640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b4006f50 0x7fa2b4007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.541+0000 7fa2cfe00640 1 -- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b4006f50 msgr2=0x7fa2b4007360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:47.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.541+0000 7fa2cfe00640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b4006f50 0x7fa2b4007360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:47.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.541+0000 7fa2cfe00640 1 -- 172.21.15.7:0/2208055285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b4005730 con 0x7fa2b4006170 2024-04-25T11:19:47.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.541+0000 7fa2d5600640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b4006f50 0x7fa2b4007360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:47.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.542+0000 7fa2cfe00640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b4006170 0x7fa2b4006a10 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00a830 tx=0x7fa2b4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a6ba9513af99b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:47.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.542+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00a480 con 0x7fa2b4006170 2024-04-25T11:19:47.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.542+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2bc02f070 con 0x7fa2b4006170 2024-04-25T11:19:47.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.542+0000 7fa2cf400640 1 -- 172.21.15.7:0/2208055285 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc02bcb0 con 0x7fa2b4006170 2024-04-25T11:19:47.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.543+0000 7fa2d7600640 1 -- 172.21.15.7:0/2208055285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b4006170 msgr2=0x7fa2b4006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:47.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.543+0000 7fa2d7600640 1 --2- 172.21.15.7:0/2208055285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2b4006170 0x7fa2b4006a10 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00a830 tx=0x7fa2b4007a40 comp rx=0 tx=0).stop 2024-04-25T11:19:47.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.543+0000 7fa2d5600640 1 -- 172.21.15.7:0/2208055285 reap_dead start 2024-04-25T11:19:47.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.543+0000 7fa2d7600640 1 -- 172.21.15.7:0/2208055285 shutdown_connections 2024-04-25T11:19:47.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.543+0000 7fa2d7600640 1 -- 172.21.15.7:0/2208055285 >> 172.21.15.7:0/2208055285 conn(0x7fa2d00742c0 msgr2=0x7fa2d006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:47.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.543+0000 7fa2d7600640 1 -- 172.21.15.7:0/2208055285 shutdown_connections 2024-04-25T11:19:47.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.543+0000 7fa2d7600640 1 -- 172.21.15.7:0/2208055285 wait complete. 2024-04-25T11:19:47.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.546+0000 7fa2d7600640 1 Processor -- start 2024-04-25T11:19:47.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.546+0000 7fa2d7600640 1 -- start start 2024-04-25T11:19:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.547+0000 7fa2d7600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2b4006170 0x7fa2d0154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.547+0000 7fa2d7600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2d0159a40 0x7fa2d01e7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.547+0000 7fa2d7600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01fa4b0 0x7fa2d01e7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.547+0000 7fa2d7600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d00b5430 con 0x7fa2d01fa4b0 2024-04-25T11:19:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.547+0000 7fa2d7600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d00afaa0 con 0x7fa2d0159a40 2024-04-25T11:19:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.547+0000 7fa2d7600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d00b55b0 con 0x7fa2b4006170 2024-04-25T11:19:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.548+0000 7fa2d5600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01fa4b0 0x7fa2d01e7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.548+0000 7fa2d5600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01fa4b0 0x7fa2d01e7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33902/0 (socket says 172.21.15.7:33902) 2024-04-25T11:19:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.548+0000 7fa2d5600640 1 -- 172.21.15.7:0/3153357288 learned_addr learned my addr 172.21.15.7:0/3153357288 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.548+0000 7fa2cfe00640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2d0159a40 0x7fa2d01e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.548+0000 7fa2d4c00640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2b4006170 0x7fa2d0154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:47.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.548+0000 7fa2d5600640 1 -- 172.21.15.7:0/3153357288 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2b4006170 msgr2=0x7fa2d0154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:47.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.548+0000 7fa2d5600640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2b4006170 0x7fa2d0154980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:47.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.548+0000 7fa2d5600640 1 -- 172.21.15.7:0/3153357288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2d0159a40 msgr2=0x7fa2d01e7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:47.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.548+0000 7fa2d5600640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2d0159a40 0x7fa2d01e7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:47.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.548+0000 7fa2d5600640 1 -- 172.21.15.7:0/3153357288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d015a1b0 con 0x7fa2d01fa4b0 2024-04-25T11:19:47.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.548+0000 7fa2cfe00640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2d0159a40 0x7fa2d01e7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.549+0000 7fa2d5600640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01fa4b0 0x7fa2d01e7a80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0072dd0 tx=0x7fa2c800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.549+0000 7fa2ccc00640 1 -- 172.21.15.7:0/3153357288 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8005480 con 0x7fa2d01fa4b0 2024-04-25T11:19:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.549+0000 7fa2ccc00640 1 -- 172.21.15.7:0/3153357288 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2c8007030 con 0x7fa2d01fa4b0 2024-04-25T11:19:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.549+0000 7fa2d7600640 1 -- 172.21.15.7:0/3153357288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d01f99d0 con 0x7fa2d01fa4b0 2024-04-25T11:19:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.549+0000 7fa2ccc00640 1 -- 172.21.15.7:0/3153357288 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8007440 con 0x7fa2d01fa4b0 2024-04-25T11:19:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.549+0000 7fa2d7600640 1 -- 172.21.15.7:0/3153357288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d01f9f90 con 0x7fa2d01fa4b0 2024-04-25T11:19:47.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.550+0000 7fa2d7600640 1 -- 172.21.15.7:0/3153357288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d0159f20 con 0x7fa2d01fa4b0 2024-04-25T11:19:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.552+0000 7fa2ccc00640 1 -- 172.21.15.7:0/3153357288 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa2c801b020 con 0x7fa2d01fa4b0 2024-04-25T11:19:47.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.553+0000 7fa2ccc00640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2b00744b0 0x7fa2b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:47.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.553+0000 7fa2ccc00640 1 -- 172.21.15.7:0/3153357288 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 6001+0+0 (secure 0 0 0) 0x7fa2c8094c90 con 0x7fa2d01fa4b0 2024-04-25T11:19:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.553+0000 7fa2d4c00640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2b00744b0 0x7fa2b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.554+0000 7fa2d4c00640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2b00744b0 0x7fa2b00769c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009950 tx=0x7fa2c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:47.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.558+0000 7fa2ccc00640 1 -- 172.21.15.7:0/3153357288 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa2c805f380 con 0x7fa2d01fa4b0 2024-04-25T11:19:47.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:47.806+0000 7fa2d7600640 1 -- 172.21.15.7:0/3153357288 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa2d0050740 con 0x7fa2d01fa4b0 2024-04-25T11:19:48.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.281+0000 7fa2ccc00640 1 -- 172.21.15.7:0/3153357288 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v72) ==== 174+0+0 (secure 0 0 0) 0x7fa2c805ec80 con 0x7fa2d01fa4b0 2024-04-25T11:19:48.283 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T11:19:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.290+0000 7fa2d7600640 1 -- 172.21.15.7:0/3153357288 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2b00744b0 msgr2=0x7fa2b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.290+0000 7fa2d7600640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2b00744b0 0x7fa2b00769c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009950 tx=0x7fa2c0008040 comp rx=0 tx=0).stop 2024-04-25T11:19:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.290+0000 7fa2d7600640 1 -- 172.21.15.7:0/3153357288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01fa4b0 msgr2=0x7fa2d01e7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.290+0000 7fa2d7600640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01fa4b0 0x7fa2d01e7a80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0072dd0 tx=0x7fa2c800d890 comp rx=0 tx=0).stop 2024-04-25T11:19:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.291+0000 7fa2d7600640 1 -- 172.21.15.7:0/3153357288 shutdown_connections 2024-04-25T11:19:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.291+0000 7fa2d7600640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2b00744b0 0x7fa2b00769c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.291+0000 7fa2d7600640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01fa4b0 0x7fa2d01e7a80 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.291+0000 7fa2d7600640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2d0159a40 0x7fa2d01e7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.291+0000 7fa2d7600640 1 --2- 172.21.15.7:0/3153357288 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2b4006170 0x7fa2d0154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.291+0000 7fa2d7600640 1 -- 172.21.15.7:0/3153357288 >> 172.21.15.7:0/3153357288 conn(0x7fa2d00742c0 msgr2=0x7fa2d015b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.292+0000 7fa2d7600640 1 -- 172.21.15.7:0/3153357288 shutdown_connections 2024-04-25T11:19:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.293+0000 7fa2d7600640 1 -- 172.21.15.7:0/3153357288 wait complete. 2024-04-25T11:19:48.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:48.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.589+0000 7fedb1600640 1 Processor -- start 2024-04-25T11:19:48.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.589+0000 7fedb1600640 1 -- start start 2024-04-25T11:19:48.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.591+0000 7fedb1600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fedac0b1610 0x7fedac0cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.592+0000 7fedb1600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedac0b1d50 0x7fedac0af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.592+0000 7fedb1600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedac0b24d0 con 0x7fedac0b73d0 2024-04-25T11:19:48.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.592+0000 7fedb1600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fedac0b2790 con 0x7fedac0b1610 2024-04-25T11:19:48.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.592+0000 7fedb1600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fedac0b2a70 con 0x7fedac0b1d50 2024-04-25T11:19:48.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.592+0000 7fedb0c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fedac0b73d0 0x7fedac0b77e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55332/0 (socket says 172.21.15.7:55332) 2024-04-25T11:19:48.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.592+0000 7fedb0c00640 1 -- 172.21.15.7:0/1004961230 learned_addr learned my addr 172.21.15.7:0/1004961230 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.592+0000 7fedabe00640 1 --2- 172.21.15.7:0/1004961230 >> v2:172.21.15.87:3300/0 conn(0x7fedac0b1610 0x7fedac0cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.592+0000 7fedab400640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedac0b1d50 0x7fedac0af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.592+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3666994856 0 0) 0x7fedac0b24d0 con 0x7fedac0b73d0 2024-04-25T11:19:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed8c003520 con 0x7fedac0b73d0 2024-04-25T11:19:48.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3024400656 0 0) 0x7fed8c003520 con 0x7fedac0b73d0 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedac0b1d50 msgr2=0x7fedac0af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedac0b1d50 0x7fedac0af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 >> v2:172.21.15.87:3300/0 conn(0x7fedac0b1610 msgr2=0x7fedac0cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 --2- 172.21.15.7:0/1004961230 >> v2:172.21.15.87:3300/0 conn(0x7fedac0b1610 0x7fedac0cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedac0b2d50 con 0x7fedac0b73d0 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7feda4001670 con 0x7fedac0b73d0 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 >> v1:172.21.15.7:6789/0 conn(0x7fedac0b73d0 legacy=0x7fedac0b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed8c0039b0 0x7fed8c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedac0b1d50 0x7fed8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c004d40 0x7fed8c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fedac0b2790 con 0x7fed8c004d40 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fedac0b2a70 con 0x7fed8c0039b0 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.593+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fed8c00ba10 con 0x7fedac0b1d50 2024-04-25T11:19:48.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.594+0000 7fedb0c00640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c004d40 0x7fed8c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.594+0000 7fedab400640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedac0b1d50 0x7fed8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.594+0000 7fedabe00640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed8c0039b0 0x7fed8c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.594+0000 7fedb0c00640 1 -- 172.21.15.7:0/1004961230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedac0b1d50 msgr2=0x7fed8c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:48.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.594+0000 7fedb0c00640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedac0b1d50 0x7fed8c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.594+0000 7fedb0c00640 1 -- 172.21.15.7:0/1004961230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed8c0039b0 msgr2=0x7fed8c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:48.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.594+0000 7fedb0c00640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed8c0039b0 0x7fed8c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.594+0000 7fedb0c00640 1 -- 172.21.15.7:0/1004961230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c003520 con 0x7fed8c004d40 2024-04-25T11:19:48.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.595+0000 7fedb0c00640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c004d40 0x7fed8c005150 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fedac074190 tx=0x7fed8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b77933540f70e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:48.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.595+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4007d50 con 0x7fed8c004d40 2024-04-25T11:19:48.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.596+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda40058d0 con 0x7fed8c004d40 2024-04-25T11:19:48.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.596+0000 7fedaaa00640 1 -- 172.21.15.7:0/1004961230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4005ce0 con 0x7fed8c004d40 2024-04-25T11:19:48.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.596+0000 7fedb1600640 1 -- 172.21.15.7:0/1004961230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c004d40 msgr2=0x7fed8c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.596+0000 7fedb1600640 1 --2- 172.21.15.7:0/1004961230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed8c004d40 0x7fed8c005150 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fedac074190 tx=0x7fed8c005830 comp rx=0 tx=0).stop 2024-04-25T11:19:48.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.596+0000 7fedb0c00640 1 -- 172.21.15.7:0/1004961230 reap_dead start 2024-04-25T11:19:48.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.596+0000 7fedb1600640 1 -- 172.21.15.7:0/1004961230 shutdown_connections 2024-04-25T11:19:48.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.596+0000 7fedb1600640 1 -- 172.21.15.7:0/1004961230 >> 172.21.15.7:0/1004961230 conn(0x7fedac09b540 msgr2=0x7fedac09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:48.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.596+0000 7fedb1600640 1 -- 172.21.15.7:0/1004961230 shutdown_connections 2024-04-25T11:19:48.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.597+0000 7fedb1600640 1 -- 172.21.15.7:0/1004961230 wait complete. 2024-04-25T11:19:48.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.599+0000 7fedb1600640 1 Processor -- start 2024-04-25T11:19:48.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.599+0000 7fedb1600640 1 -- start start 2024-04-25T11:19:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb1600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed8c004d40 0x7fedac1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb1600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedac0c9960 0x7fedac155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedac155aa0 0x7fedac1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb1600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fedac0af4c0 con 0x7fedac155aa0 2024-04-25T11:19:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb1600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fedac0b2160 con 0x7fedac0c9960 2024-04-25T11:19:48.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb1600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fedac0af640 con 0x7fed8c004d40 2024-04-25T11:19:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedac155aa0 0x7fedac1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedac155aa0 0x7fedac1567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33918/0 (socket says 172.21.15.7:33918) 2024-04-25T11:19:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb0c00640 1 -- 172.21.15.7:0/2045234176 learned_addr learned my addr 172.21.15.7:0/2045234176 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedabe00640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed8c004d40 0x7fedac1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedab400640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedac0c9960 0x7fedac155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb0c00640 1 -- 172.21.15.7:0/2045234176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed8c004d40 msgr2=0x7fedac1597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb0c00640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed8c004d40 0x7fedac1597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb0c00640 1 -- 172.21.15.7:0/2045234176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedac0c9960 msgr2=0x7fedac155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb0c00640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedac0c9960 0x7fedac155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.601+0000 7fedb0c00640 1 -- 172.21.15.7:0/2045234176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedac1f4430 con 0x7fedac155aa0 2024-04-25T11:19:48.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.602+0000 7fedb0c00640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedac155aa0 0x7fedac1567a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7feda402adf0 tx=0x7fedac0b2f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.602+0000 7feda3e00640 1 -- 172.21.15.7:0/2045234176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4041070 con 0x7fedac155aa0 2024-04-25T11:19:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.602+0000 7feda3e00640 1 -- 172.21.15.7:0/2045234176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda4007070 con 0x7fedac155aa0 2024-04-25T11:19:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.602+0000 7feda3e00640 1 -- 172.21.15.7:0/2045234176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4038040 con 0x7fedac155aa0 2024-04-25T11:19:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.602+0000 7fedb1600640 1 -- 172.21.15.7:0/2045234176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedac156580 con 0x7fedac155aa0 2024-04-25T11:19:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.602+0000 7fedb1600640 1 -- 172.21.15.7:0/2045234176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedac154a80 con 0x7fedac155aa0 2024-04-25T11:19:48.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.604+0000 7fedb1600640 1 -- 172.21.15.7:0/2045234176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedac0cf7e0 con 0x7fedac155aa0 2024-04-25T11:19:48.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.605+0000 7feda3e00640 1 -- 172.21.15.7:0/2045234176 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feda4044070 con 0x7fedac155aa0 2024-04-25T11:19:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.606+0000 7feda3e00640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed880744b0 0x7fed880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:48.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.606+0000 7feda3e00640 1 -- 172.21.15.7:0/2045234176 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6101+0+0 (secure 0 0 0) 0x7feda40b4620 con 0x7fedac155aa0 2024-04-25T11:19:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.606+0000 7fedabe00640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed880744b0 0x7fed880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:48.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.607+0000 7fedabe00640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed880744b0 0x7fed880769c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0099d0 tx=0x7fed9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:48.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.611+0000 7feda3e00640 1 -- 172.21.15.7:0/2045234176 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedac156580 con 0x7fedac155aa0 2024-04-25T11:19:48.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.817+0000 7fedb1600640 1 -- 172.21.15.7:0/2045234176 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedac0c94e0 con 0x7fedac155aa0 2024-04-25T11:19:48.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.819+0000 7feda3e00640 1 -- 172.21.15.7:0/2045234176 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16869 (secure 0 0 0) 0x7feda40058a0 con 0x7fedac155aa0 2024-04-25T11:19:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:48.818 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":117,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:47.807601+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:46.014943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:46.794047+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:48.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 -- 172.21.15.7:0/2045234176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed880744b0 msgr2=0x7fed880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed880744b0 0x7fed880769c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0099d0 tx=0x7fed9c008040 comp rx=0 tx=0).stop 2024-04-25T11:19:48.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 -- 172.21.15.7:0/2045234176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedac155aa0 msgr2=0x7fedac1567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:48.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedac155aa0 0x7fedac1567a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7feda402adf0 tx=0x7fedac0b2f20 comp rx=0 tx=0).stop 2024-04-25T11:19:48.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 -- 172.21.15.7:0/2045234176 shutdown_connections 2024-04-25T11:19:48.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed880744b0 0x7fed880769c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedac155aa0 0x7fedac1567a0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedac0c9960 0x7fedac155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 --2- 172.21.15.7:0/2045234176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed8c004d40 0x7fedac1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 -- 172.21.15.7:0/2045234176 >> 172.21.15.7:0/2045234176 conn(0x7fedac09b540 msgr2=0x7fedac0b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 -- 172.21.15.7:0/2045234176 shutdown_connections 2024-04-25T11:19:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:48.822+0000 7fedb1600640 1 -- 172.21.15.7:0/2045234176 wait complete. 2024-04-25T11:19:48.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.126+0000 7efed4a00640 1 Processor -- start 2024-04-25T11:19:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.126+0000 7efed4a00640 1 -- start start 2024-04-25T11:19:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.129+0000 7efed4a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efed0070a70 0x7efed00ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.129+0000 7efed4a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efed00aedc0 0x7efed00b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.129+0000 7efed4a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efed00b0a10 0x7efed01607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.129+0000 7efed4a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efed015e190 con 0x7efed0070a70 2024-04-25T11:19:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.129+0000 7efed4a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efed015d610 con 0x7efed00b0a10 2024-04-25T11:19:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.129+0000 7efed4a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efed015d8f0 con 0x7efed00aedc0 2024-04-25T11:19:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.129+0000 7efece800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efed0070a70 0x7efed00ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.129+0000 7efece800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efed0070a70 0x7efed00ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33930/0 (socket says 172.21.15.7:33930) 2024-04-25T11:19:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.129+0000 7efece800640 1 -- 172.21.15.7:0/1145410053 learned_addr learned my addr 172.21.15.7:0/1145410053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.130+0000 7efecf200640 1 --2- 172.21.15.7:0/1145410053 >> v2:172.21.15.87:3300/0 conn(0x7efed00b0a10 0x7efed01607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.130+0000 7efec7e00640 1 --2- 172.21.15.7:0/1145410053 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efed00aedc0 0x7efed00b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.130+0000 7efece800640 1 -- 172.21.15.7:0/1145410053 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efed00aedc0 msgr2=0x7efed00b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.130+0000 7efece800640 1 --2- 172.21.15.7:0/1145410053 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efed00aedc0 0x7efed00b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.130+0000 7efece800640 1 -- 172.21.15.7:0/1145410053 >> v2:172.21.15.87:3300/0 conn(0x7efed00b0a10 msgr2=0x7efed01607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.130+0000 7efece800640 1 --2- 172.21.15.7:0/1145410053 >> v2:172.21.15.87:3300/0 conn(0x7efed00b0a10 0x7efed01607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.130+0000 7efece800640 1 -- 172.21.15.7:0/1145410053 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed015dbd0 con 0x7efed0070a70 2024-04-25T11:19:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.131+0000 7efece800640 1 --2- 172.21.15.7:0/1145410053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed0070a70 0x7efed00ae880 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7efed0161850 tx=0x7efec002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fb1d1f867f48d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.131+0000 7efecde00640 1 -- 172.21.15.7:0/1145410053 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0037070 con 0x7efed0070a70 2024-04-25T11:19:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.131+0000 7efecde00640 1 -- 172.21.15.7:0/1145410053 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec0032440 con 0x7efed0070a70 2024-04-25T11:19:49.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.131+0000 7efecde00640 1 -- 172.21.15.7:0/1145410053 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0032810 con 0x7efed0070a70 2024-04-25T11:19:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.132+0000 7efed4a00640 1 -- 172.21.15.7:0/1145410053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed0070a70 msgr2=0x7efed00ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.132+0000 7efed4a00640 1 --2- 172.21.15.7:0/1145410053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed0070a70 0x7efed00ae880 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7efed0161850 tx=0x7efec002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.132+0000 7efed4a00640 1 -- 172.21.15.7:0/1145410053 shutdown_connections 2024-04-25T11:19:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.132+0000 7efed4a00640 1 --2- 172.21.15.7:0/1145410053 >> v2:172.21.15.87:3300/0 conn(0x7efed00b0a10 0x7efed01607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.132+0000 7efed4a00640 1 --2- 172.21.15.7:0/1145410053 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efed00aedc0 0x7efed00b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.132+0000 7efed4a00640 1 --2- 172.21.15.7:0/1145410053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed0070a70 0x7efed00ae880 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.132+0000 7efed4a00640 1 -- 172.21.15.7:0/1145410053 >> 172.21.15.7:0/1145410053 conn(0x7efed00b3820 msgr2=0x7efed00b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.132+0000 7efed4a00640 1 -- 172.21.15.7:0/1145410053 shutdown_connections 2024-04-25T11:19:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.132+0000 7efed4a00640 1 -- 172.21.15.7:0/1145410053 wait complete. 2024-04-25T11:19:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.134+0000 7efed4a00640 1 Processor -- start 2024-04-25T11:19:49.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.135+0000 7efed4a00640 1 -- start start 2024-04-25T11:19:49.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.136+0000 7efed4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed0070a70 0x7efed01610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.136+0000 7efed4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efed00aedc0 0x7efed00ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.136+0000 7efed4a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efed00b0a10 0x7efed00cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.136+0000 7efed4a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efed015d170 con 0x7efed0070a70 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.136+0000 7efed4a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efed00b0e20 con 0x7efed00aedc0 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.136+0000 7efed4a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efed015d2f0 con 0x7efed00b0a10 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.136+0000 7efece800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed0070a70 0x7efed01610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.136+0000 7efece800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed0070a70 0x7efed01610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33938/0 (socket says 172.21.15.7:33938) 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.136+0000 7efece800640 1 -- 172.21.15.7:0/2978323100 learned_addr learned my addr 172.21.15.7:0/2978323100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.137+0000 7efec7e00640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efed00aedc0 0x7efed00ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.137+0000 7efecf200640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efed00b0a10 0x7efed00cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.137+0000 7efece800640 1 -- 172.21.15.7:0/2978323100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efed00b0a10 msgr2=0x7efed00cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.137+0000 7efece800640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efed00b0a10 0x7efed00cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.137+0000 7efece800640 1 -- 172.21.15.7:0/2978323100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efed00aedc0 msgr2=0x7efed00ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.137+0000 7efece800640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efed00aedc0 0x7efed00ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.137+0000 7efece800640 1 -- 172.21.15.7:0/2978323100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed00cc560 con 0x7efed0070a70 2024-04-25T11:19:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.137+0000 7efece800640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed0070a70 0x7efed01610c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7efec0009960 tx=0x7efec00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.138+0000 7efec6a00640 1 -- 172.21.15.7:0/2978323100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0041020 con 0x7efed0070a70 2024-04-25T11:19:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.138+0000 7efec6a00640 1 -- 172.21.15.7:0/2978323100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec0030800 con 0x7efed0070a70 2024-04-25T11:19:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.138+0000 7efec6a00640 1 -- 172.21.15.7:0/2978323100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0037040 con 0x7efed0070a70 2024-04-25T11:19:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.138+0000 7efed4a00640 1 -- 172.21.15.7:0/2978323100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed015cd10 con 0x7efed0070a70 2024-04-25T11:19:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.138+0000 7efed4a00640 1 -- 172.21.15.7:0/2978323100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed01605e0 con 0x7efed0070a70 2024-04-25T11:19:49.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.139+0000 7efed4a00640 1 -- 172.21.15.7:0/2978323100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed0050740 con 0x7efed0070a70 2024-04-25T11:19:49.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.140+0000 7efec6a00640 1 -- 172.21.15.7:0/2978323100 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efec0044020 con 0x7efed0070a70 2024-04-25T11:19:49.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.141+0000 7efec6a00640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efeac0744e0 0x7efeac0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.142+0000 7efec6a00640 1 -- 172.21.15.7:0/2978323100 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6101+0+0 (secure 0 0 0) 0x7efec00b59f0 con 0x7efed0070a70 2024-04-25T11:19:49.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.142+0000 7efec7e00640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efeac0744e0 0x7efeac0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.142+0000 7efec7e00640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efeac0744e0 0x7efeac0769f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efeb8006fd0 tx=0x7efeb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:49.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.146+0000 7efec6a00640 1 -- 172.21.15.7:0/2978323100 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efec004b070 con 0x7efed0070a70 2024-04-25T11:19:49.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.353+0000 7efed4a00640 1 -- 172.21.15.7:0/2978323100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efed00b1420 con 0x7efed0070a70 2024-04-25T11:19:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.357+0000 7efec6a00640 1 -- 172.21.15.7:0/2978323100 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16856 (secure 0 0 0) 0x7efec007ff50 con 0x7efed0070a70 2024-04-25T11:19:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:49.355 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:49.277316+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:46.014943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:46.794047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:49.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 -- 172.21.15.7:0/2978323100 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efeac0744e0 msgr2=0x7efeac0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:49.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efeac0744e0 0x7efeac0769f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efeb8006fd0 tx=0x7efeb8008040 comp rx=0 tx=0).stop 2024-04-25T11:19:49.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 -- 172.21.15.7:0/2978323100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed0070a70 msgr2=0x7efed01610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:49.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed0070a70 0x7efed01610c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7efec0009960 tx=0x7efec00046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 -- 172.21.15.7:0/2978323100 shutdown_connections 2024-04-25T11:19:49.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efeac0744e0 0x7efeac0769f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efed00b0a10 0x7efed00cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efed00aedc0 0x7efed00ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 --2- 172.21.15.7:0/2978323100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efed0070a70 0x7efed01610c0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 -- 172.21.15.7:0/2978323100 >> 172.21.15.7:0/2978323100 conn(0x7efed00b3820 msgr2=0x7efed00b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:49.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 -- 172.21.15.7:0/2978323100 shutdown_connections 2024-04-25T11:19:49.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.360+0000 7efed4a00640 1 -- 172.21.15.7:0/2978323100 wait complete. 2024-04-25T11:19:49.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:19:49.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.677+0000 7f7fe8a00640 1 Processor -- start 2024-04-25T11:19:49.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.678+0000 7f7fe8a00640 1 -- start start 2024-04-25T11:19:49.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.680+0000 7f7fe8a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fe415aaa0 0x7f7fe40affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.680+0000 7f7fe8a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fe40d0ee0 0x7f7fe40b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.680+0000 7f7fe8a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe406f5b0 con 0x7f7fe40b4e50 2024-04-25T11:19:49.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.680+0000 7f7fe8a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7fe40af290 con 0x7f7fe415aaa0 2024-04-25T11:19:49.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.680+0000 7f7fe8a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe40d14a0 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.680+0000 7f7fe1e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fe415aaa0 0x7f7fe40affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.681+0000 7f7fe1e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fe415aaa0 0x7f7fe40affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33946/0 (socket says 172.21.15.7:33946) 2024-04-25T11:19:49.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.681+0000 7f7fe1e00640 1 -- 172.21.15.7:0/2495639286 learned_addr learned my addr 172.21.15.7:0/2495639286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:49.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.681+0000 7f7fe1e00640 1 -- 172.21.15.7:0/2495639286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fe40d0ee0 msgr2=0x7f7fe40b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:49.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.681+0000 7f7fe1e00640 1 --2- 172.21.15.7:0/2495639286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fe40d0ee0 0x7f7fe40b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.681+0000 7f7fe1e00640 1 -- 172.21.15.7:0/2495639286 >> v1:172.21.15.87:6789/0 conn(0x7f7fe40b4e50 legacy=0x7f7fe415d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:49.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.681+0000 7f7fe1e00640 1 -- 172.21.15.7:0/2495639286 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe40af4e0 con 0x7f7fe415aaa0 2024-04-25T11:19:49.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.682+0000 7f7fe1e00640 1 --2- 172.21.15.7:0/2495639286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe415aaa0 0x7f7fe40affc0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009870 tx=0x7f7fd402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a5344ee188a463 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:49.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.682+0000 7f7fe0a00640 1 -- 172.21.15.7:0/2495639286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4037070 con 0x7f7fe415aaa0 2024-04-25T11:19:49.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.682+0000 7f7fe0a00640 1 -- 172.21.15.7:0/2495639286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd4032440 con 0x7f7fe415aaa0 2024-04-25T11:19:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.682+0000 7f7fe0a00640 1 -- 172.21.15.7:0/2495639286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4032810 con 0x7f7fe415aaa0 2024-04-25T11:19:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.683+0000 7f7fe8a00640 1 -- 172.21.15.7:0/2495639286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe415aaa0 msgr2=0x7f7fe40affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.683+0000 7f7fe8a00640 1 --2- 172.21.15.7:0/2495639286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe415aaa0 0x7f7fe40affc0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009870 tx=0x7f7fd402a180 comp rx=0 tx=0).stop 2024-04-25T11:19:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.683+0000 7f7fe8a00640 1 -- 172.21.15.7:0/2495639286 shutdown_connections 2024-04-25T11:19:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.683+0000 7f7fe8a00640 1 --2- 172.21.15.7:0/2495639286 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fe40d0ee0 0x7f7fe40b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.683+0000 7f7fe8a00640 1 --2- 172.21.15.7:0/2495639286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe415aaa0 0x7f7fe40affc0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.683+0000 7f7fe8a00640 1 -- 172.21.15.7:0/2495639286 >> 172.21.15.7:0/2495639286 conn(0x7f7fe40742c0 msgr2=0x7f7fe406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.683+0000 7f7fe8a00640 1 -- 172.21.15.7:0/2495639286 shutdown_connections 2024-04-25T11:19:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.684+0000 7f7fe8a00640 1 -- 172.21.15.7:0/2495639286 wait complete. 2024-04-25T11:19:49.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.687+0000 7f7fe8a00640 1 Processor -- start 2024-04-25T11:19:49.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.687+0000 7f7fe8a00640 1 -- start start 2024-04-25T11:19:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.689+0000 7f7fe8a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fe40b4e50 0x7f7fe41463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.689+0000 7f7fe8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe40d0ee0 0x7f7fe415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.689+0000 7f7fe8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe415aaa0 0x7f7fe4158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.689+0000 7f7fe8a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe40b5260 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.689+0000 7f7fe8a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe40afaa0 con 0x7f7fe415aaa0 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.689+0000 7f7fe8a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe40b53e0 con 0x7f7fe40b4e50 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.689+0000 7f7fe1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe40d0ee0 0x7f7fe415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.689+0000 7f7fe1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe40d0ee0 0x7f7fe415ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33960/0 (socket says 172.21.15.7:33960) 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.689+0000 7f7fe1400640 1 -- 172.21.15.7:0/3250853686 learned_addr learned my addr 172.21.15.7:0/3250853686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.690+0000 7f7fe1e00640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fe40b4e50 0x7f7fe41463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.690+0000 7f7fe1400640 1 -- 172.21.15.7:0/3250853686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fe40b4e50 msgr2=0x7f7fe41463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.690+0000 7f7fe1400640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fe40b4e50 0x7f7fe41463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.690+0000 7f7fe2800640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe415aaa0 0x7f7fe4158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.690+0000 7f7fe1400640 1 -- 172.21.15.7:0/3250853686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe415aaa0 msgr2=0x7f7fe4158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.690+0000 7f7fe1400640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe415aaa0 0x7f7fe4158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.690+0000 7f7fe1400640 1 -- 172.21.15.7:0/3250853686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe41f73c0 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.690+0000 7f7fe1400640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe40d0ee0 0x7f7fe415ef20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000b410 tx=0x7f7fd000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.690+0000 7f7fda000640 1 -- 172.21.15.7:0/3250853686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0015020 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.691+0000 7f7fda000640 1 -- 172.21.15.7:0/3250853686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fd0011850 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.691+0000 7f7fda000640 1 -- 172.21.15.7:0/3250853686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0011c40 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.691+0000 7f7fe8a00640 1 -- 172.21.15.7:0/3250853686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe41f76a0 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.691+0000 7f7fe8a00640 1 -- 172.21.15.7:0/3250853686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe415ea00 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.692+0000 7f7fe8a00640 1 -- 172.21.15.7:0/3250853686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe41f7870 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.694+0000 7f7fda000640 1 -- 172.21.15.7:0/3250853686 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7fd0018020 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.695+0000 7f7fda000640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7fc40744b0 0x7f7fc40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:49.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.696+0000 7f7fda000640 1 -- 172.21.15.7:0/3250853686 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f7fd00959a0 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.696+0000 7f7fe1e00640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7fc40744b0 0x7f7fc40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:49.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.696+0000 7f7fe1e00640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7fc40744b0 0x7f7fc40769c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4034000 tx=0x7f7fd402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.699+0000 7f7fda000640 1 -- 172.21.15.7:0/3250853686 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7fd001f020 con 0x7f7fe40d0ee0 2024-04-25T11:19:49.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:49.943+0000 7f7fe8a00640 1 -- 172.21.15.7:0/3250853686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7fe4050740 con 0x7f7fe40d0ee0 2024-04-25T11:19:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.304+0000 7f7fda000640 1 -- 172.21.15.7:0/3250853686 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) ==== 118+0+0 (secure 0 0 0) 0x7f7fd005ff80 con 0x7f7fe40d0ee0 2024-04-25T11:19:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 -- 172.21.15.7:0/3250853686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7fc40744b0 msgr2=0x7f7fc40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7fc40744b0 0x7f7fc40769c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4034000 tx=0x7f7fd402e000 comp rx=0 tx=0).stop 2024-04-25T11:19:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 -- 172.21.15.7:0/3250853686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe40d0ee0 msgr2=0x7f7fe415ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe40d0ee0 0x7f7fe415ef20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000b410 tx=0x7f7fd000b8d0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 -- 172.21.15.7:0/3250853686 shutdown_connections 2024-04-25T11:19:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7fc40744b0 0x7f7fc40769c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fe415aaa0 0x7f7fe4158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe40d0ee0 0x7f7fe415ef20 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 --2- 172.21.15.7:0/3250853686 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7fe40b4e50 0x7f7fe41463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 -- 172.21.15.7:0/3250853686 >> 172.21.15.7:0/3250853686 conn(0x7f7fe40742c0 msgr2=0x7f7fe406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 -- 172.21.15.7:0/3250853686 shutdown_connections 2024-04-25T11:19:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.308+0000 7f7fe8a00640 1 -- 172.21.15.7:0/3250853686 wait complete. 2024-04-25T11:19:50.343 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:19:50.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:19:50.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.635+0000 7fec08a00640 1 Processor -- start 2024-04-25T11:19:50.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.635+0000 7fec08a00640 1 -- start start 2024-04-25T11:19:50.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.638+0000 7fec08a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fec040d0480 0x7fec040d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:50.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.638+0000 7fec08a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec040ce570 con 0x7fec041583c0 2024-04-25T11:19:50.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.638+0000 7fec08a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec040ce850 con 0x7fec0406eea0 2024-04-25T11:19:50.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.638+0000 7fec08a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fec0415f190 con 0x7fec040d0480 2024-04-25T11:19:50.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.638+0000 7fec03400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fec041583c0 0x7fec040cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58764/0 (socket says 172.21.15.7:58764) 2024-04-25T11:19:50.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.638+0000 7fec03400640 1 -- 172.21.15.7:0/1472616766 learned_addr learned my addr 172.21.15.7:0/1472616766 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:50.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.638+0000 7fec02a00640 1 --2- 172.21.15.7:0/1472616766 >> v2:172.21.15.7:3300/0 conn(0x7fec040d0480 0x7fec040d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:50.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.639+0000 7fec02000640 1 -- 172.21.15.7:0/1472616766 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2900590916 0 0) 0x7fec040ce570 con 0x7fec041583c0 2024-04-25T11:19:50.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.639+0000 7fec02000640 1 -- 172.21.15.7:0/1472616766 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febe8003520 con 0x7fec041583c0 2024-04-25T11:19:50.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.639+0000 7fec02a00640 1 -- 172.21.15.7:0/1472616766 >> v1:172.21.15.87:6789/0 conn(0x7fec0406eea0 legacy=0x7fec040b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:50.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.639+0000 7fec02a00640 1 -- 172.21.15.7:0/1472616766 >> v1:172.21.15.7:6789/0 conn(0x7fec041583c0 legacy=0x7fec040cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:50.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.639+0000 7fec02a00640 1 -- 172.21.15.7:0/1472616766 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec040ceb30 con 0x7fec040d0480 2024-04-25T11:19:50.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.640+0000 7fec02a00640 1 --2- 172.21.15.7:0/1472616766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040d0480 0x7fec040d0890 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7febf00097a0 tx=0x7febf002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6c2f35f371a9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:50.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.640+0000 7fec02000640 1 -- 172.21.15.7:0/1472616766 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0037070 con 0x7fec040d0480 2024-04-25T11:19:50.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.640+0000 7fec02000640 1 -- 172.21.15.7:0/1472616766 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf0032440 con 0x7fec040d0480 2024-04-25T11:19:50.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.640+0000 7fec02000640 1 -- 172.21.15.7:0/1472616766 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0032810 con 0x7fec040d0480 2024-04-25T11:19:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.641+0000 7fec08a00640 1 -- 172.21.15.7:0/1472616766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040d0480 msgr2=0x7fec040d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.641+0000 7fec08a00640 1 --2- 172.21.15.7:0/1472616766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040d0480 0x7fec040d0890 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7febf00097a0 tx=0x7febf002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.641+0000 7fec08a00640 1 -- 172.21.15.7:0/1472616766 shutdown_connections 2024-04-25T11:19:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.641+0000 7fec08a00640 1 --2- 172.21.15.7:0/1472616766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec040d0480 0x7fec040d0890 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.641+0000 7fec08a00640 1 -- 172.21.15.7:0/1472616766 >> 172.21.15.7:0/1472616766 conn(0x7fec040740f0 msgr2=0x7fec0406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:50.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.641+0000 7fec08a00640 1 -- 172.21.15.7:0/1472616766 shutdown_connections 2024-04-25T11:19:50.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.641+0000 7fec08a00640 1 -- 172.21.15.7:0/1472616766 wait complete. 2024-04-25T11:19:50.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.644+0000 7fec08a00640 1 Processor -- start 2024-04-25T11:19:50.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.644+0000 7fec08a00640 1 -- start start 2024-04-25T11:19:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.645+0000 7fec08a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec0406eea0 0x7fec041e7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.645+0000 7fec08a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec040d0480 0x7fec041f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.645+0000 7fec08a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec041583c0 0x7fec041f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec08a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fec040cfd20 con 0x7fec041583c0 2024-04-25T11:19:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec08a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fec040cf0f0 con 0x7fec040d0480 2024-04-25T11:19:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec08a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fec040b0e70 con 0x7fec0406eea0 2024-04-25T11:19:50.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec03e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec041583c0 0x7fec041f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec03e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec041583c0 0x7fec041f9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42420/0 (socket says 172.21.15.7:42420) 2024-04-25T11:19:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec03e00640 1 -- 172.21.15.7:0/2978956316 learned_addr learned my addr 172.21.15.7:0/2978956316 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec02a00640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec040d0480 0x7fec041f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec03400640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec0406eea0 0x7fec041e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec03e00640 1 -- 172.21.15.7:0/2978956316 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec0406eea0 msgr2=0x7fec041e7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec03e00640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec0406eea0 0x7fec041e7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec03e00640 1 -- 172.21.15.7:0/2978956316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec040d0480 msgr2=0x7fec041f9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec03e00640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec040d0480 0x7fec041f9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec03e00640 1 -- 172.21.15.7:0/2978956316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec041ea080 con 0x7fec041583c0 2024-04-25T11:19:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.646+0000 7fec02a00640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec040d0480 0x7fec041f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:50.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.647+0000 7fec03e00640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec041583c0 0x7fec041f9e40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7febfc00c6b0 tx=0x7febfc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.647+0000 7febfb400640 1 -- 172.21.15.7:0/2978956316 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc003b70 con 0x7fec041583c0 2024-04-25T11:19:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.647+0000 7febfb400640 1 -- 172.21.15.7:0/2978956316 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febfc003d10 con 0x7fec041583c0 2024-04-25T11:19:50.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.647+0000 7fec08a00640 1 -- 172.21.15.7:0/2978956316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec041ea360 con 0x7fec041583c0 2024-04-25T11:19:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.647+0000 7febfb400640 1 -- 172.21.15.7:0/2978956316 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc005310 con 0x7fec041583c0 2024-04-25T11:19:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.647+0000 7fec08a00640 1 -- 172.21.15.7:0/2978956316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec04152130 con 0x7fec041583c0 2024-04-25T11:19:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.648+0000 7fec08a00640 1 -- 172.21.15.7:0/2978956316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec041e7bd0 con 0x7fec041583c0 2024-04-25T11:19:50.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.650+0000 7febfb400640 1 -- 172.21.15.7:0/2978956316 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7febfc0054b0 con 0x7fec041583c0 2024-04-25T11:19:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.651+0000 7febfb400640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febe40744b0 0x7febe40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.651+0000 7febfb400640 1 -- 172.21.15.7:0/2978956316 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7febfc095270 con 0x7fec041583c0 2024-04-25T11:19:50.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.651+0000 7fec03400640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febe40744b0 0x7febe40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:50.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.652+0000 7fec03400640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febe40744b0 0x7febe40769c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7febf4005880 tx=0x7febf4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:50.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.656+0000 7febfb400640 1 -- 172.21.15.7:0/2978956316 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febfc05f870 con 0x7fec041583c0 2024-04-25T11:19:50.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:50.898+0000 7fec08a00640 1 -- 172.21.15.7:0/2978956316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fec04152390 con 0x7fec041583c0 2024-04-25T11:19:51.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.308+0000 7febfb400640 1 -- 172.21.15.7:0/2978956316 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) ==== 122+0+0 (secure 0 0 0) 0x7febfc05f170 con 0x7fec041583c0 2024-04-25T11:19:51.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.311+0000 7fec08a00640 1 -- 172.21.15.7:0/2978956316 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febe40744b0 msgr2=0x7febe40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.311+0000 7fec08a00640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febe40744b0 0x7febe40769c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7febf4005880 tx=0x7febf4009040 comp rx=0 tx=0).stop 2024-04-25T11:19:51.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.311+0000 7fec08a00640 1 -- 172.21.15.7:0/2978956316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec041583c0 msgr2=0x7fec041f9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.311+0000 7fec08a00640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec041583c0 0x7fec041f9e40 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7febfc00c6b0 tx=0x7febfc00cb70 comp rx=0 tx=0).stop 2024-04-25T11:19:51.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.312+0000 7fec08a00640 1 -- 172.21.15.7:0/2978956316 shutdown_connections 2024-04-25T11:19:51.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.312+0000 7fec08a00640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febe40744b0 0x7febe40769c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.312+0000 7fec08a00640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec041583c0 0x7fec041f9e40 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.312+0000 7fec08a00640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec040d0480 0x7fec041f9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.312+0000 7fec08a00640 1 --2- 172.21.15.7:0/2978956316 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec0406eea0 0x7fec041e7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.312+0000 7fec08a00640 1 -- 172.21.15.7:0/2978956316 >> 172.21.15.7:0/2978956316 conn(0x7fec040740f0 msgr2=0x7fec040b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:51.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.312+0000 7fec08a00640 1 -- 172.21.15.7:0/2978956316 shutdown_connections 2024-04-25T11:19:51.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.312+0000 7fec08a00640 1 -- 172.21.15.7:0/2978956316 wait complete. 2024-04-25T11:19:51.344 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:19:51.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.636+0000 7f4b98a00640 1 Processor -- start 2024-04-25T11:19:51.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.636+0000 7f4b98a00640 1 -- start start 2024-04-25T11:19:51.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.639+0000 7f4b98a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4b940aed40 0x7f4b9415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.639+0000 7f4b98a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b940cebb0 con 0x7f4b940b5220 2024-04-25T11:19:51.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.639+0000 7f4b98a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b940b64b0 con 0x7f4b940af420 2024-04-25T11:19:51.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.639+0000 7f4b98a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4b9415e1f0 con 0x7f4b940aed40 2024-04-25T11:19:51.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.639+0000 7f4b91e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4b940b5220 0x7f4b940ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58770/0 (socket says 172.21.15.7:58770) 2024-04-25T11:19:51.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.639+0000 7f4b91e00640 1 -- 172.21.15.7:0/2533906780 learned_addr learned my addr 172.21.15.7:0/2533906780 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.639+0000 7f4b92800640 1 --2- 172.21.15.7:0/2533906780 >> v2:172.21.15.87:3300/0 conn(0x7f4b940aed40 0x7f4b9415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.640+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1877770081 0 0) 0x7f4b80001110 con 0x7f4b940b5220 2024-04-25T11:19:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.640+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b78003520 con 0x7f4b940b5220 2024-04-25T11:19:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.640+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 221011041 0 0) 0x7f4b78003520 con 0x7f4b940b5220 2024-04-25T11:19:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.640+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 >> v2:172.21.15.87:3300/0 conn(0x7f4b940aed40 msgr2=0x7f4b9415dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.640+0000 7f4b91400640 1 --2- 172.21.15.7:0/2533906780 >> v2:172.21.15.87:3300/0 conn(0x7f4b940aed40 0x7f4b9415dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.640+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 >> v1:172.21.15.87:6789/0 conn(0x7f4b940af420 legacy=0x7f4b940af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.640+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9406c610 con 0x7f4b940b5220 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.640+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f4b80001840 con 0x7f4b940b5220 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.640+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 >> v1:172.21.15.7:6789/0 conn(0x7f4b940b5220 legacy=0x7f4b940ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.640+0000 7f4b91400640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b780039b0 0x7f4b78003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b91400640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 0x7f4b78004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b91400640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b780051c0 0x7f4b7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9415e1f0 con 0x7f4b78004800 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7800c160 con 0x7f4b780039b0 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7800c360 con 0x7f4b780051c0 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b91e00640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 0x7f4b78004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b92800640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b780039b0 0x7f4b78003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b91e00640 1 -- 172.21.15.7:0/2533906780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b780051c0 msgr2=0x7f4b7800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b93200640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b780051c0 0x7f4b7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b91e00640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b780051c0 0x7f4b7800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b91e00640 1 -- 172.21.15.7:0/2533906780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b780039b0 msgr2=0x7f4b78003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b91e00640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b780039b0 0x7f4b78003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b91e00640 1 -- 172.21.15.7:0/2533906780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b78003520 con 0x7f4b78004800 2024-04-25T11:19:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b93200640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b780051c0 0x7f4b7800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:51.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.641+0000 7f4b93200640 1 -- 172.21.15.7:0/2533906780 reap_dead start 2024-04-25T11:19:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.643+0000 7f4b91e00640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 0x7f4b78004c10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000c4c0 tx=0x7f4b78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f3b18024499951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.643+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000c0e0 con 0x7f4b78004800 2024-04-25T11:19:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.643+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b80034070 con 0x7f4b78004800 2024-04-25T11:19:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.643+0000 7f4b91400640 1 -- 172.21.15.7:0/2533906780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80005d70 con 0x7f4b78004800 2024-04-25T11:19:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.644+0000 7f4b98a00640 1 -- 172.21.15.7:0/2533906780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 msgr2=0x7f4b78004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.644+0000 7f4b98a00640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 0x7f4b78004c10 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000c4c0 tx=0x7f4b78005830 comp rx=0 tx=0).stop 2024-04-25T11:19:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.644+0000 7f4b98a00640 1 -- 172.21.15.7:0/2533906780 shutdown_connections 2024-04-25T11:19:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.644+0000 7f4b98a00640 1 --2- 172.21.15.7:0/2533906780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 0x7f4b78004c10 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.644+0000 7f4b98a00640 1 -- 172.21.15.7:0/2533906780 >> 172.21.15.7:0/2533906780 conn(0x7f4b940b3820 msgr2=0x7f4b940b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.644+0000 7f4b98a00640 1 -- 172.21.15.7:0/2533906780 shutdown_connections 2024-04-25T11:19:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.644+0000 7f4b98a00640 1 -- 172.21.15.7:0/2533906780 wait complete. 2024-04-25T11:19:51.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.646+0000 7f4b98a00640 1 Processor -- start 2024-04-25T11:19:51.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.647+0000 7f4b98a00640 1 -- start start 2024-04-25T11:19:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.648+0000 7f4b98a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 0x7f4b9409bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.648+0000 7f4b98a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b9414d110 0x7f4b9415bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.648+0000 7f4b98a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b9414d520 0x7f4b9415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.648+0000 7f4b98a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9406ceb0 con 0x7f4b78004800 2024-04-25T11:19:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.648+0000 7f4b98a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9406cbd0 con 0x7f4b9414d110 2024-04-25T11:19:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.648+0000 7f4b98a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4b940b6110 con 0x7f4b9414d520 2024-04-25T11:19:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.648+0000 7f4b92800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 0x7f4b9409bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.649+0000 7f4b92800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 0x7f4b9409bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42434/0 (socket says 172.21.15.7:42434) 2024-04-25T11:19:51.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.649+0000 7f4b92800640 1 -- 172.21.15.7:0/1770372691 learned_addr learned my addr 172.21.15.7:0/1770372691 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.649+0000 7f4b91e00640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b9414d110 0x7f4b9415bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.649+0000 7f4b93200640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b9414d520 0x7f4b9415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.649+0000 7f4b92800640 1 -- 172.21.15.7:0/1770372691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b9414d520 msgr2=0x7f4b9415c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.649+0000 7f4b92800640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b9414d520 0x7f4b9415c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.649+0000 7f4b92800640 1 -- 172.21.15.7:0/1770372691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b9414d110 msgr2=0x7f4b9415bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.649+0000 7f4b92800640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b9414d110 0x7f4b9415bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.649+0000 7f4b92800640 1 -- 172.21.15.7:0/1770372691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9409e850 con 0x7f4b78004800 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.649+0000 7f4b91e00640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b9414d110 0x7f4b9415bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.650+0000 7f4b92800640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 0x7f4b9409bf00 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f4b84011620 tx=0x7f4b84011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.650+0000 7f4b8aa00640 1 -- 172.21.15.7:0/1770372691 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8400f920 con 0x7f4b78004800 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.650+0000 7f4b8aa00640 1 -- 172.21.15.7:0/1770372691 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8400fac0 con 0x7f4b78004800 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.650+0000 7f4b8aa00640 1 -- 172.21.15.7:0/1770372691 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8400fe90 con 0x7f4b78004800 2024-04-25T11:19:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.650+0000 7f4b98a00640 1 -- 172.21.15.7:0/1770372691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9409eb30 con 0x7f4b78004800 2024-04-25T11:19:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.650+0000 7f4b98a00640 1 -- 172.21.15.7:0/1770372691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9409f0f0 con 0x7f4b78004800 2024-04-25T11:19:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.651+0000 7f4b98a00640 1 -- 172.21.15.7:0/1770372691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b94050740 con 0x7f4b78004800 2024-04-25T11:19:51.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.653+0000 7f4b8aa00640 1 -- 172.21.15.7:0/1770372691 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4b84015460 con 0x7f4b78004800 2024-04-25T11:19:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.654+0000 7f4b8aa00640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b74074650 0x7f4b74076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.654+0000 7f4b8aa00640 1 -- 172.21.15.7:0/1770372691 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f4b84095e20 con 0x7f4b78004800 2024-04-25T11:19:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.654+0000 7f4b91e00640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b74074650 0x7f4b74076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:51.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.655+0000 7f4b91e00640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b74074650 0x7f4b74076b60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4b80033000 tx=0x7f4b8002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:51.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.659+0000 7f4b8aa00640 1 -- 172.21.15.7:0/1770372691 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4b84019030 con 0x7f4b78004800 2024-04-25T11:19:51.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.907+0000 7f4b98a00640 1 -- 172.21.15.7:0/1770372691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b9415d3f0 con 0x7f4b78004800 2024-04-25T11:19:51.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.908+0000 7f4b8aa00640 1 -- 172.21.15.7:0/1770372691 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5174 (secure 0 0 0) 0x7f4b84060410 con 0x7f4b78004800 2024-04-25T11:19:51.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:51.907 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/326382713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":326382713},{"type":"v1","addr":"172.21.15.7:6837","nonce":326382713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2007810270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2007810270},{"type":"v1","addr":"172.21.15.87:6835","nonce":2007810270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1753656512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1753656512},{"type":"v1","addr":"172.21.15.7:6836","nonce":1753656512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:47.807534+0000","modified":"2024-04-25T11:19:50.899375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1924056839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1924056839},{"type":"v1","addr":"172.21.15.87:6834","nonce":1924056839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8007,"qdb_cluster":[8007]},"id":8}]} 2024-04-25T11:19:51.907 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2024-04-25T11:19:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.911+0000 7f4b98a00640 1 -- 172.21.15.7:0/1770372691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b74074650 msgr2=0x7f4b74076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.911+0000 7f4b98a00640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b74074650 0x7f4b74076b60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4b80033000 tx=0x7f4b8002e000 comp rx=0 tx=0).stop 2024-04-25T11:19:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.911+0000 7f4b98a00640 1 -- 172.21.15.7:0/1770372691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 msgr2=0x7f4b9409bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.911+0000 7f4b98a00640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 0x7f4b9409bf00 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f4b84011620 tx=0x7f4b84011ae0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.912+0000 7f4b98a00640 1 -- 172.21.15.7:0/1770372691 shutdown_connections 2024-04-25T11:19:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.912+0000 7f4b98a00640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4b74074650 0x7f4b74076b60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.912+0000 7f4b98a00640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b9414d520 0x7f4b9415c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.912+0000 7f4b98a00640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b9414d110 0x7f4b9415bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.912+0000 7f4b98a00640 1 --2- 172.21.15.7:0/1770372691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b78004800 0x7f4b9409bf00 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.912+0000 7f4b98a00640 1 -- 172.21.15.7:0/1770372691 >> 172.21.15.7:0/1770372691 conn(0x7f4b940b3820 msgr2=0x7f4b940d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.912+0000 7f4b98a00640 1 -- 172.21.15.7:0/1770372691 shutdown_connections 2024-04-25T11:19:51.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:51.912+0000 7f4b98a00640 1 -- 172.21.15.7:0/1770372691 wait complete. 2024-04-25T11:19:51.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:19:52.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.226+0000 7f0757a00640 1 Processor -- start 2024-04-25T11:19:52.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.226+0000 7f0757a00640 1 -- start start 2024-04-25T11:19:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.228+0000 7f0757a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f075015bc40 0x7f075009f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.228+0000 7f0757a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f075015f4e0 con 0x7f075015ef90 2024-04-25T11:19:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.228+0000 7f0757a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07500b0560 con 0x7f07500af540 2024-04-25T11:19:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.228+0000 7f0757a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0750071190 con 0x7f075015bc40 2024-04-25T11:19:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.228+0000 7f0755000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f075015ef90 0x7f07500aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58780/0 (socket says 172.21.15.7:58780) 2024-04-25T11:19:52.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.228+0000 7f0755000640 1 -- 172.21.15.7:0/1412884951 learned_addr learned my addr 172.21.15.7:0/1412884951 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:52.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.229+0000 7f0755a00640 1 --2- 172.21.15.7:0/1412884951 >> v2:172.21.15.87:3300/0 conn(0x7f075015bc40 0x7f075009f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.229+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 360762683 0 0) 0x7f075015f4e0 con 0x7f075015ef90 2024-04-25T11:19:52.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.229+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0734003520 con 0x7f075015ef90 2024-04-25T11:19:52.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.229+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2161470619 0 0) 0x7f0734003520 con 0x7f075015ef90 2024-04-25T11:19:52.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.229+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 >> v2:172.21.15.87:3300/0 conn(0x7f075015bc40 msgr2=0x7f075009f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.229+0000 7f074f400640 1 --2- 172.21.15.7:0/1412884951 >> v2:172.21.15.87:3300/0 conn(0x7f075015bc40 0x7f075009f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.229+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 >> v1:172.21.15.87:6789/0 conn(0x7f07500af540 legacy=0x7f07500cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.229+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07500b0840 con 0x7f075015ef90 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.229+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0740001670 con 0x7f075015ef90 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 >> v1:172.21.15.7:6789/0 conn(0x7f075015ef90 legacy=0x7f07500aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f074f400640 1 --2- 172.21.15.7:0/1412884951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07340039b0 0x7f0734003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f074f400640 1 --2- 172.21.15.7:0/1412884951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0734004800 0x7f0734004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f074f400640 1 --2- 172.21.15.7:0/1412884951 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0734005250 0x7f073400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0750071190 con 0x7f07340039b0 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f073400c1e0 con 0x7f0734004800 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f073400c3e0 con 0x7f0734005250 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f0755000640 1 --2- 172.21.15.7:0/1412884951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07340039b0 0x7f0734003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f0755a00640 1 --2- 172.21.15.7:0/1412884951 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0734005250 0x7f073400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f074fe00640 1 --2- 172.21.15.7:0/1412884951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0734004800 0x7f0734004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f0755000640 1 -- 172.21.15.7:0/1412884951 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0734005250 msgr2=0x7f073400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f0755000640 1 --2- 172.21.15.7:0/1412884951 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0734005250 0x7f073400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f0755000640 1 -- 172.21.15.7:0/1412884951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0734004800 msgr2=0x7f0734004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f0755000640 1 --2- 172.21.15.7:0/1412884951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0734004800 0x7f0734004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.230+0000 7f0755000640 1 -- 172.21.15.7:0/1412884951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0734003520 con 0x7f07340039b0 2024-04-25T11:19:52.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.231+0000 7f0755a00640 1 -- 172.21.15.7:0/1412884951 reap_dead start 2024-04-25T11:19:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.231+0000 7f0755000640 1 --2- 172.21.15.7:0/1412884951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07340039b0 0x7f0734003e20 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f074000c240 tx=0x7f0734005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64661e3046f8f91a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.232+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740007d10 con 0x7f07340039b0 2024-04-25T11:19:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.232+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0740005ad0 con 0x7f07340039b0 2024-04-25T11:19:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.232+0000 7f074f400640 1 -- 172.21.15.7:0/1412884951 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740007070 con 0x7f07340039b0 2024-04-25T11:19:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.232+0000 7f0757a00640 1 -- 172.21.15.7:0/1412884951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07340039b0 msgr2=0x7f0734003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.232+0000 7f0757a00640 1 --2- 172.21.15.7:0/1412884951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07340039b0 0x7f0734003e20 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f074000c240 tx=0x7f0734005830 comp rx=0 tx=0).stop 2024-04-25T11:19:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.233+0000 7f0757a00640 1 -- 172.21.15.7:0/1412884951 shutdown_connections 2024-04-25T11:19:52.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.233+0000 7f0757a00640 1 --2- 172.21.15.7:0/1412884951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07340039b0 0x7f0734003e20 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.233+0000 7f0757a00640 1 -- 172.21.15.7:0/1412884951 >> 172.21.15.7:0/1412884951 conn(0x7f0750074200 msgr2=0x7f075006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:52.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.233+0000 7f0757a00640 1 -- 172.21.15.7:0/1412884951 shutdown_connections 2024-04-25T11:19:52.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.233+0000 7f0757a00640 1 -- 172.21.15.7:0/1412884951 wait complete. 2024-04-25T11:19:52.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.235+0000 7f0757a00640 1 Processor -- start 2024-04-25T11:19:52.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.235+0000 7f0757a00640 1 -- start start 2024-04-25T11:19:52.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.237+0000 7f0757a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07340039b0 0x7f0750154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.237+0000 7f0757a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0750153130 0x7f0750149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.237+0000 7f0757a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07501562d0 0x7f07501566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.237+0000 7f0757a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07500b1320 con 0x7f07501562d0 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.237+0000 7f0757a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07500b0e00 con 0x7f0750153130 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.237+0000 7f0757a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f07500afb80 con 0x7f07340039b0 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.237+0000 7f0755a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07501562d0 0x7f07501566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.237+0000 7f0755a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07501562d0 0x7f07501566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42452/0 (socket says 172.21.15.7:42452) 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.237+0000 7f0755a00640 1 -- 172.21.15.7:0/195929547 learned_addr learned my addr 172.21.15.7:0/195929547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f074fe00640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0750153130 0x7f0750149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f0755000640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07340039b0 0x7f0750154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f0755a00640 1 -- 172.21.15.7:0/195929547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07340039b0 msgr2=0x7f0750154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f0755a00640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07340039b0 0x7f0750154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f0755a00640 1 -- 172.21.15.7:0/195929547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0750153130 msgr2=0x7f0750149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f0755a00640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0750153130 0x7f0750149060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f0755a00640 1 -- 172.21.15.7:0/195929547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0750156c20 con 0x7f07501562d0 2024-04-25T11:19:52.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f0755000640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07340039b0 0x7f0750154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:52.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f0755a00640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07501562d0 0x7f07501566e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f07500ce140 tx=0x7f074800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:52.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f074cc00640 1 -- 172.21.15.7:0/195929547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748016620 con 0x7f07501562d0 2024-04-25T11:19:52.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f074cc00640 1 -- 172.21.15.7:0/195929547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f074801b070 con 0x7f07501562d0 2024-04-25T11:19:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f074cc00640 1 -- 172.21.15.7:0/195929547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748016a20 con 0x7f07501562d0 2024-04-25T11:19:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.238+0000 7f0757a00640 1 -- 172.21.15.7:0/195929547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0750153ea0 con 0x7f07501562d0 2024-04-25T11:19:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.239+0000 7f0757a00640 1 -- 172.21.15.7:0/195929547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075015c460 con 0x7f07501562d0 2024-04-25T11:19:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.240+0000 7f0757a00640 1 -- 172.21.15.7:0/195929547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075006a100 con 0x7f07501562d0 2024-04-25T11:19:52.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.241+0000 7f074cc00640 1 -- 172.21.15.7:0/195929547 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0748005d10 con 0x7f07501562d0 2024-04-25T11:19:52.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.242+0000 7f074cc00640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f072c0744e0 0x7f072c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.242+0000 7f074cc00640 1 -- 172.21.15.7:0/195929547 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f0748095da0 con 0x7f07501562d0 2024-04-25T11:19:52.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.242+0000 7f0755000640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f072c0744e0 0x7f072c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.243+0000 7f0755000640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f072c0744e0 0x7f072c0769f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0740032000 tx=0x7f0740034040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:52.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.247+0000 7f074cc00640 1 -- 172.21.15.7:0/195929547 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0750156c20 con 0x7f07501562d0 2024-04-25T11:19:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.458+0000 7f0757a00640 1 -- 172.21.15.7:0/195929547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07500b3230 con 0x7f07501562d0 2024-04-25T11:19:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.461+0000 7f074cc00640 1 -- 172.21.15.7:0/195929547 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16856 (secure 0 0 0) 0x7f0748060300 con 0x7f07501562d0 2024-04-25T11:19:52.459 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:52.459 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:49.277316+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:46.014943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:46.794047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:19:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.463+0000 7f0757a00640 1 -- 172.21.15.7:0/195929547 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f072c0744e0 msgr2=0x7f072c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.463+0000 7f0757a00640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f072c0744e0 0x7f072c0769f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0740032000 tx=0x7f0740034040 comp rx=0 tx=0).stop 2024-04-25T11:19:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.464+0000 7f0757a00640 1 -- 172.21.15.7:0/195929547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07501562d0 msgr2=0x7f07501566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.464+0000 7f0757a00640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07501562d0 0x7f07501566e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f07500ce140 tx=0x7f074800f520 comp rx=0 tx=0).stop 2024-04-25T11:19:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.464+0000 7f0757a00640 1 -- 172.21.15.7:0/195929547 shutdown_connections 2024-04-25T11:19:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.464+0000 7f0757a00640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f072c0744e0 0x7f072c0769f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.464+0000 7f0757a00640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07501562d0 0x7f07501566e0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.464+0000 7f0757a00640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0750153130 0x7f0750149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.464+0000 7f0757a00640 1 --2- 172.21.15.7:0/195929547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07340039b0 0x7f0750154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.464+0000 7f0757a00640 1 -- 172.21.15.7:0/195929547 >> 172.21.15.7:0/195929547 conn(0x7f0750074200 msgr2=0x7f075006fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.464+0000 7f0757a00640 1 -- 172.21.15.7:0/195929547 shutdown_connections 2024-04-25T11:19:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.464+0000 7f0757a00640 1 -- 172.21.15.7:0/195929547 wait complete. 2024-04-25T11:19:52.489 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:19:52.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:19:52.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.775+0000 7f2fe6e00640 1 Processor -- start 2024-04-25T11:19:52.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.775+0000 7f2fe6e00640 1 -- start start 2024-04-25T11:19:52.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.777+0000 7f2fe6e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2fe0163bb0 0x7f2fe00d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.778+0000 7f2fe6e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fe006c570 0x7f2fe006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.778+0000 7f2fe6e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe006cec0 con 0x7f2fe00bae90 2024-04-25T11:19:52.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.778+0000 7f2fe6e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe00d08e0 con 0x7f2fe0163bb0 2024-04-25T11:19:52.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.778+0000 7f2fe6e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe00b51c0 con 0x7f2fe006c570 2024-04-25T11:19:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.779+0000 7f2fe5a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2fe0163bb0 0x7f2fe00d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.779+0000 7f2fe6400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fe006c570 0x7f2fe006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.779+0000 7f2fe5a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2fe0163bb0 0x7f2fe00d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:36998/0 (socket says 172.21.15.7:36998) 2024-04-25T11:19:52.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.779+0000 7f2fe5a00640 1 -- 172.21.15.7:0/3698263700 learned_addr learned my addr 172.21.15.7:0/3698263700 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.779+0000 7f2fe6400640 1 -- 172.21.15.7:0/3698263700 >> v2:172.21.15.87:3300/0 conn(0x7f2fe0163bb0 msgr2=0x7f2fe00d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.779+0000 7f2fe6400640 1 --2- 172.21.15.7:0/3698263700 >> v2:172.21.15.87:3300/0 conn(0x7f2fe0163bb0 0x7f2fe00d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.780+0000 7f2fe6400640 1 -- 172.21.15.7:0/3698263700 >> v1:172.21.15.87:6789/0 conn(0x7f2fe00bae90 legacy=0x7f2fe00b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.780+0000 7f2fe6400640 1 -- 172.21.15.7:0/3698263700 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe0164120 con 0x7f2fe006c570 2024-04-25T11:19:52.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.780+0000 7f2fe5a00640 1 --2- 172.21.15.7:0/3698263700 >> v2:172.21.15.87:3300/0 conn(0x7f2fe0163bb0 0x7f2fe00d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:19:52.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.780+0000 7f2fdfe00640 1 -- 172.21.15.7:0/3698263700 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2256707029 0 0) 0x7f2fcc001110 con 0x7f2fe00bae90 2024-04-25T11:19:52.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.781+0000 7f2fe6400640 1 --2- 172.21.15.7:0/3698263700 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fe006c570 0x7f2fe006c980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009d90 tx=0x7f2fd802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c53405aca393cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.782+0000 7f2fdfe00640 1 -- 172.21.15.7:0/3698263700 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8007740 con 0x7f2fe006c570 2024-04-25T11:19:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.782+0000 7f2fdfe00640 1 -- 172.21.15.7:0/3698263700 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd80078e0 con 0x7f2fe006c570 2024-04-25T11:19:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.782+0000 7f2fdfe00640 1 -- 172.21.15.7:0/3698263700 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8007cb0 con 0x7f2fe006c570 2024-04-25T11:19:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.782+0000 7f2fe6e00640 1 -- 172.21.15.7:0/3698263700 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fe006c570 msgr2=0x7f2fe006c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.782+0000 7f2fe6e00640 1 --2- 172.21.15.7:0/3698263700 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fe006c570 0x7f2fe006c980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009d90 tx=0x7f2fd802a660 comp rx=0 tx=0).stop 2024-04-25T11:19:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.783+0000 7f2fe6e00640 1 -- 172.21.15.7:0/3698263700 shutdown_connections 2024-04-25T11:19:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.783+0000 7f2fe6e00640 1 --2- 172.21.15.7:0/3698263700 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fe006c570 0x7f2fe006c980 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.783+0000 7f2fe6e00640 1 --2- 172.21.15.7:0/3698263700 >> v2:172.21.15.87:3300/0 conn(0x7f2fe0163bb0 0x7f2fe00d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.783+0000 7f2fe6e00640 1 -- 172.21.15.7:0/3698263700 >> 172.21.15.7:0/3698263700 conn(0x7f2fe006eb00 msgr2=0x7f2fe00d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.783+0000 7f2fe6e00640 1 -- 172.21.15.7:0/3698263700 shutdown_connections 2024-04-25T11:19:52.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.783+0000 7f2fe6e00640 1 -- 172.21.15.7:0/3698263700 wait complete. 2024-04-25T11:19:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.786+0000 7f2fe6e00640 1 Processor -- start 2024-04-25T11:19:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.786+0000 7f2fe6e00640 1 -- start start 2024-04-25T11:19:52.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.788+0000 7f2fe6e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fe006c570 0x7f2fe00cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.788+0000 7f2fe6e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe00bae90 0x7f2fe014bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.789+0000 7f2fe6e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fe0163bb0 0x7f2fe014c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.789+0000 7f2fe6e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe01649c0 con 0x7f2fe00bae90 2024-04-25T11:19:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.789+0000 7f2fe6e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe01646e0 con 0x7f2fe0163bb0 2024-04-25T11:19:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.789+0000 7f2fe6e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe00d4170 con 0x7f2fe006c570 2024-04-25T11:19:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.789+0000 7f2fe5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe00bae90 0x7f2fe014bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.789+0000 7f2fe5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe00bae90 0x7f2fe014bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42454/0 (socket says 172.21.15.7:42454) 2024-04-25T11:19:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.789+0000 7f2fe5000640 1 -- 172.21.15.7:0/1740036761 learned_addr learned my addr 172.21.15.7:0/1740036761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.789+0000 7f2fe6400640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fe0163bb0 0x7f2fe014c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.789+0000 7f2fe5a00640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fe006c570 0x7f2fe00cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.789+0000 7f2fe5000640 1 -- 172.21.15.7:0/1740036761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fe006c570 msgr2=0x7f2fe00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.790+0000 7f2fe5000640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fe006c570 0x7f2fe00cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.790+0000 7f2fe5000640 1 -- 172.21.15.7:0/1740036761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fe0163bb0 msgr2=0x7f2fe014c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.790+0000 7f2fe5000640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fe0163bb0 0x7f2fe014c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.790+0000 7f2fe5000640 1 -- 172.21.15.7:0/1740036761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe00c9f00 con 0x7f2fe00bae90 2024-04-25T11:19:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.790+0000 7f2fe6400640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fe0163bb0 0x7f2fe014c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.790+0000 7f2fe5000640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe00bae90 0x7f2fe014bb20 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00c3e0 tx=0x7f2fcc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.790+0000 7f2fdd600640 1 -- 172.21.15.7:0/1740036761 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc017070 con 0x7f2fe00bae90 2024-04-25T11:19:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.790+0000 7f2fdd600640 1 -- 172.21.15.7:0/1740036761 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fcc0052d0 con 0x7f2fe00bae90 2024-04-25T11:19:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.790+0000 7f2fdd600640 1 -- 172.21.15.7:0/1740036761 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc003830 con 0x7f2fe00bae90 2024-04-25T11:19:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.790+0000 7f2fe6e00640 1 -- 172.21.15.7:0/1740036761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe00ca1e0 con 0x7f2fe00bae90 2024-04-25T11:19:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.790+0000 7f2fe6e00640 1 -- 172.21.15.7:0/1740036761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe00cbd40 con 0x7f2fe00bae90 2024-04-25T11:19:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.792+0000 7f2fe6e00640 1 -- 172.21.15.7:0/1740036761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe0050740 con 0x7f2fe00bae90 2024-04-25T11:19:52.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.793+0000 7f2fdd600640 1 -- 172.21.15.7:0/1740036761 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2fcc01a070 con 0x7f2fe00bae90 2024-04-25T11:19:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.794+0000 7f2fdd600640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fc00744b0 0x7f2fc00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.794+0000 7f2fdd600640 1 -- 172.21.15.7:0/1740036761 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f2fcc0947d0 con 0x7f2fe00bae90 2024-04-25T11:19:52.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.794+0000 7f2fe5a00640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fc00744b0 0x7f2fc00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.795+0000 7f2fe5a00640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fc00744b0 0x7f2fc00769c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40099e0 tx=0x7f2fd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.799+0000 7f2fdd600640 1 -- 172.21.15.7:0/1740036761 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fe006cec0 con 0x7f2fe00bae90 2024-04-25T11:19:52.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:52.995+0000 7f2fe6e00640 1 -- 172.21.15.7:0/1740036761 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2fe00c9cb0 con 0x7f2fc00744b0 2024-04-25T11:19:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.014+0000 7f2fdd600640 1 -- 172.21.15.7:0/1740036761 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252451 (secure 0 0 0) 0x7f2fe00c9cb0 con 0x7f2fc00744b0 2024-04-25T11:19:53.015 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:53.017 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:19:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.020+0000 7f2fe6e00640 1 -- 172.21.15.7:0/1740036761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fc00744b0 msgr2=0x7f2fc00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.020+0000 7f2fe6e00640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fc00744b0 0x7f2fc00769c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2fd40099e0 tx=0x7f2fd4008040 comp rx=0 tx=0).stop 2024-04-25T11:19:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.020+0000 7f2fe6e00640 1 -- 172.21.15.7:0/1740036761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe00bae90 msgr2=0x7f2fe014bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:53.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.020+0000 7f2fe6e00640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe00bae90 0x7f2fe014bb20 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00c3e0 tx=0x7f2fcc00c8a0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.020+0000 7f2fe6e00640 1 -- 172.21.15.7:0/1740036761 shutdown_connections 2024-04-25T11:19:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.020+0000 7f2fe6e00640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fc00744b0 0x7f2fc00769c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.020+0000 7f2fe6e00640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fe0163bb0 0x7f2fe014c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.020+0000 7f2fe6e00640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fe00bae90 0x7f2fe014bb20 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.020+0000 7f2fe6e00640 1 --2- 172.21.15.7:0/1740036761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fe006c570 0x7f2fe00cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.020+0000 7f2fe6e00640 1 -- 172.21.15.7:0/1740036761 >> 172.21.15.7:0/1740036761 conn(0x7f2fe006eb00 msgr2=0x7f2fe00b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.020+0000 7f2fe6e00640 1 -- 172.21.15.7:0/1740036761 shutdown_connections 2024-04-25T11:19:53.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.021+0000 7f2fe6e00640 1 -- 172.21.15.7:0/1740036761 wait complete. 2024-04-25T11:19:53.045 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-04-25T11:19:52.011489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270376,"kb_used_data":12580,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754704344,"statfs":{"total":773094113280,"available":772817248256,"internally_reserved":0,"allocated":12881920,"data_stored":6545960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.795679"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731540+0000","last_change":"2024-04-25T11:19:48.291100+0000","last_active":"2024-04-25T11:19:49.731540+0000","last_peered":"2024-04-25T11:19:49.731540+0000","last_clean":"2024-04-25T11:19:49.731540+0000","last_became_active":"2024-04-25T11:19:48.290780+0000","last_became_peered":"2024-04-25T11:19:48.290780+0000","last_unstale":"2024-04-25T11:19:49.731540+0000","last_undegraded":"2024-04-25T11:19:49.731540+0000","last_fullsized":"2024-04-25T11:19:49.731540+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:01.201766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.304485+0000","last_change":"2024-04-25T11:19:48.296231+0000","last_active":"2024-04-25T11:19:48.304485+0000","last_peered":"2024-04-25T11:19:48.304485+0000","last_clean":"2024-04-25T11:19:48.304485+0000","last_became_active":"2024-04-25T11:19:48.295829+0000","last_became_peered":"2024-04-25T11:19:48.295829+0000","last_unstale":"2024-04-25T11:19:48.304485+0000","last_undegraded":"2024-04-25T11:19:48.304485+0000","last_fullsized":"2024-04-25T11:19:48.304485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:12.674057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.306342+0000","last_change":"2024-04-25T11:19:48.294215+0000","last_active":"2024-04-25T11:19:48.306342+0000","last_peered":"2024-04-25T11:19:48.306342+0000","last_clean":"2024-04-25T11:19:48.306342+0000","last_became_active":"2024-04-25T11:19:48.293889+0000","last_became_peered":"2024-04-25T11:19:48.293889+0000","last_unstale":"2024-04-25T11:19:48.306342+0000","last_undegraded":"2024-04-25T11:19:48.306342+0000","last_fullsized":"2024-04-25T11:19:48.306342+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:43.185766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298809+0000","last_change":"2024-04-25T11:19:48.289536+0000","last_active":"2024-04-25T11:19:49.298809+0000","last_peered":"2024-04-25T11:19:49.298809+0000","last_clean":"2024-04-25T11:19:49.298809+0000","last_became_active":"2024-04-25T11:19:48.288968+0000","last_became_peered":"2024-04-25T11:19:48.288968+0000","last_unstale":"2024-04-25T11:19:49.298809+0000","last_undegraded":"2024-04-25T11:19:49.298809+0000","last_fullsized":"2024-04-25T11:19:49.298809+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:43.845296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.730732+0000","last_change":"2024-04-25T11:19:48.292007+0000","last_active":"2024-04-25T11:19:49.730732+0000","last_peered":"2024-04-25T11:19:49.730732+0000","last_clean":"2024-04-25T11:19:49.730732+0000","last_became_active":"2024-04-25T11:19:48.291700+0000","last_became_peered":"2024-04-25T11:19:48.291700+0000","last_unstale":"2024-04-25T11:19:49.730732+0000","last_undegraded":"2024-04-25T11:19:49.730732+0000","last_fullsized":"2024-04-25T11:19:49.730732+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:05.318358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303024+0000","last_change":"2024-04-25T11:19:48.309091+0000","last_active":"2024-04-25T11:19:49.303024+0000","last_peered":"2024-04-25T11:19:49.303024+0000","last_clean":"2024-04-25T11:19:49.303024+0000","last_became_active":"2024-04-25T11:19:48.308619+0000","last_became_peered":"2024-04-25T11:19:48.308619+0000","last_unstale":"2024-04-25T11:19:49.303024+0000","last_undegraded":"2024-04-25T11:19:49.303024+0000","last_fullsized":"2024-04-25T11:19:49.303024+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:52.931994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.302862+0000","last_change":"2024-04-25T11:19:48.312988+0000","last_active":"2024-04-25T11:19:49.302862+0000","last_peered":"2024-04-25T11:19:49.302862+0000","last_clean":"2024-04-25T11:19:49.302862+0000","last_became_active":"2024-04-25T11:19:48.309097+0000","last_became_peered":"2024-04-25T11:19:48.309097+0000","last_unstale":"2024-04-25T11:19:49.302862+0000","last_undegraded":"2024-04-25T11:19:49.302862+0000","last_fullsized":"2024-04-25T11:19:49.302862+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:30.300749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.300155+0000","last_change":"2024-04-25T11:19:48.291497+0000","last_active":"2024-04-25T11:19:48.300155+0000","last_peered":"2024-04-25T11:19:48.300155+0000","last_clean":"2024-04-25T11:19:48.300155+0000","last_became_active":"2024-04-25T11:19:48.291160+0000","last_became_peered":"2024-04-25T11:19:48.291160+0000","last_unstale":"2024-04-25T11:19:48.300155+0000","last_undegraded":"2024-04-25T11:19:48.300155+0000","last_fullsized":"2024-04-25T11:19:48.300155+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:23.899534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299608+0000","last_change":"2024-04-25T11:19:48.301014+0000","last_active":"2024-04-25T11:19:49.299608+0000","last_peered":"2024-04-25T11:19:49.299608+0000","last_clean":"2024-04-25T11:19:49.299608+0000","last_became_active":"2024-04-25T11:19:48.300711+0000","last_became_peered":"2024-04-25T11:19:48.300711+0000","last_unstale":"2024-04-25T11:19:49.299608+0000","last_undegraded":"2024-04-25T11:19:49.299608+0000","last_fullsized":"2024-04-25T11:19:49.299608+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:30.513081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.312862+0000","last_change":"2024-04-25T11:19:48.305992+0000","last_active":"2024-04-25T11:19:48.312862+0000","last_peered":"2024-04-25T11:19:48.312862+0000","last_clean":"2024-04-25T11:19:48.312862+0000","last_became_active":"2024-04-25T11:19:48.305637+0000","last_became_peered":"2024-04-25T11:19:48.305637+0000","last_unstale":"2024-04-25T11:19:48.312862+0000","last_undegraded":"2024-04-25T11:19:48.312862+0000","last_fullsized":"2024-04-25T11:19:48.312862+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:31.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731632+0000","last_change":"2024-04-25T11:19:48.298644+0000","last_active":"2024-04-25T11:19:49.731632+0000","last_peered":"2024-04-25T11:19:49.731632+0000","last_clean":"2024-04-25T11:19:49.731632+0000","last_became_active":"2024-04-25T11:19:48.295822+0000","last_became_peered":"2024-04-25T11:19:48.295822+0000","last_unstale":"2024-04-25T11:19:49.731632+0000","last_undegraded":"2024-04-25T11:19:49.731632+0000","last_fullsized":"2024-04-25T11:19:49.731632+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:24.499963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303965+0000","last_change":"2024-04-25T11:19:48.309922+0000","last_active":"2024-04-25T11:19:49.303965+0000","last_peered":"2024-04-25T11:19:49.303965+0000","last_clean":"2024-04-25T11:19:49.303965+0000","last_became_active":"2024-04-25T11:19:48.309086+0000","last_became_peered":"2024-04-25T11:19:48.309086+0000","last_unstale":"2024-04-25T11:19:49.303965+0000","last_undegraded":"2024-04-25T11:19:49.303965+0000","last_fullsized":"2024-04-25T11:19:49.303965+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:04.823507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.293356+0000","last_change":"2024-04-25T11:19:48.290035+0000","last_active":"2024-04-25T11:19:48.293356+0000","last_peered":"2024-04-25T11:19:48.293356+0000","last_clean":"2024-04-25T11:19:48.293356+0000","last_became_active":"2024-04-25T11:19:48.289648+0000","last_became_peered":"2024-04-25T11:19:48.289648+0000","last_unstale":"2024-04-25T11:19:48.293356+0000","last_undegraded":"2024-04-25T11:19:48.293356+0000","last_fullsized":"2024-04-25T11:19:48.293356+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:30.400046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.298223+0000","last_change":"2024-04-25T11:19:48.293151+0000","last_active":"2024-04-25T11:19:48.298223+0000","last_peered":"2024-04-25T11:19:48.298223+0000","last_clean":"2024-04-25T11:19:48.298223+0000","last_became_active":"2024-04-25T11:19:48.292149+0000","last_became_peered":"2024-04-25T11:19:48.292149+0000","last_unstale":"2024-04-25T11:19:48.298223+0000","last_undegraded":"2024-04-25T11:19:48.298223+0000","last_fullsized":"2024-04-25T11:19:48.298223+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:40.119303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731003+0000","last_change":"2024-04-25T11:19:48.301827+0000","last_active":"2024-04-25T11:19:49.731003+0000","last_peered":"2024-04-25T11:19:49.731003+0000","last_clean":"2024-04-25T11:19:49.731003+0000","last_became_active":"2024-04-25T11:19:48.301517+0000","last_became_peered":"2024-04-25T11:19:48.301517+0000","last_unstale":"2024-04-25T11:19:49.731003+0000","last_undegraded":"2024-04-25T11:19:49.731003+0000","last_fullsized":"2024-04-25T11:19:49.731003+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:23.688909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.302725+0000","last_change":"2024-04-25T11:19:48.304232+0000","last_active":"2024-04-25T11:19:49.302725+0000","last_peered":"2024-04-25T11:19:49.302725+0000","last_clean":"2024-04-25T11:19:49.302725+0000","last_became_active":"2024-04-25T11:19:48.303358+0000","last_became_peered":"2024-04-25T11:19:48.303358+0000","last_unstale":"2024-04-25T11:19:49.302725+0000","last_undegraded":"2024-04-25T11:19:49.302725+0000","last_fullsized":"2024-04-25T11:19:49.302725+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:56.548714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.310358+0000","last_change":"2024-04-25T11:19:48.298929+0000","last_active":"2024-04-25T11:19:48.310358+0000","last_peered":"2024-04-25T11:19:48.310358+0000","last_clean":"2024-04-25T11:19:48.310358+0000","last_became_active":"2024-04-25T11:19:48.298575+0000","last_became_peered":"2024-04-25T11:19:48.298575+0000","last_unstale":"2024-04-25T11:19:48.310358+0000","last_undegraded":"2024-04-25T11:19:48.310358+0000","last_fullsized":"2024-04-25T11:19:48.310358+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:59.403388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.293439+0000","last_change":"2024-04-25T11:19:48.280394+0000","last_active":"2024-04-25T11:19:48.293439+0000","last_peered":"2024-04-25T11:19:48.293439+0000","last_clean":"2024-04-25T11:19:48.293439+0000","last_became_active":"2024-04-25T11:19:48.279885+0000","last_became_peered":"2024-04-25T11:19:48.279885+0000","last_unstale":"2024-04-25T11:19:48.293439+0000","last_undegraded":"2024-04-25T11:19:48.293439+0000","last_fullsized":"2024-04-25T11:19:48.293439+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:27.175206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296317+0000","last_change":"2024-04-25T11:19:48.314502+0000","last_active":"2024-04-25T11:19:49.296317+0000","last_peered":"2024-04-25T11:19:49.296317+0000","last_clean":"2024-04-25T11:19:49.296317+0000","last_became_active":"2024-04-25T11:19:48.311402+0000","last_became_peered":"2024-04-25T11:19:48.311402+0000","last_unstale":"2024-04-25T11:19:49.296317+0000","last_undegraded":"2024-04-25T11:19:49.296317+0000","last_fullsized":"2024-04-25T11:19:49.296317+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:56.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.317069+0000","last_change":"2024-04-25T11:19:48.311358+0000","last_active":"2024-04-25T11:19:48.317069+0000","last_peered":"2024-04-25T11:19:48.317069+0000","last_clean":"2024-04-25T11:19:48.317069+0000","last_became_active":"2024-04-25T11:19:48.311003+0000","last_became_peered":"2024-04-25T11:19:48.311003+0000","last_unstale":"2024-04-25T11:19:48.317069+0000","last_undegraded":"2024-04-25T11:19:48.317069+0000","last_fullsized":"2024-04-25T11:19:48.317069+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:19.376102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299210+0000","last_change":"2024-04-25T11:19:48.297975+0000","last_active":"2024-04-25T11:19:49.299210+0000","last_peered":"2024-04-25T11:19:49.299210+0000","last_clean":"2024-04-25T11:19:49.299210+0000","last_became_active":"2024-04-25T11:19:48.297339+0000","last_became_peered":"2024-04-25T11:19:48.297339+0000","last_unstale":"2024-04-25T11:19:49.299210+0000","last_undegraded":"2024-04-25T11:19:49.299210+0000","last_fullsized":"2024-04-25T11:19:49.299210+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:16.477832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.730941+0000","last_change":"2024-04-25T11:19:48.299350+0000","last_active":"2024-04-25T11:19:49.730941+0000","last_peered":"2024-04-25T11:19:49.730941+0000","last_clean":"2024-04-25T11:19:49.730941+0000","last_became_active":"2024-04-25T11:19:48.299004+0000","last_became_peered":"2024-04-25T11:19:48.299004+0000","last_unstale":"2024-04-25T11:19:49.730941+0000","last_undegraded":"2024-04-25T11:19:49.730941+0000","last_fullsized":"2024-04-25T11:19:49.730941+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:59.096555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299044+0000","last_change":"2024-04-25T11:19:48.304575+0000","last_active":"2024-04-25T11:19:49.299044+0000","last_peered":"2024-04-25T11:19:49.299044+0000","last_clean":"2024-04-25T11:19:49.299044+0000","last_became_active":"2024-04-25T11:19:48.302053+0000","last_became_peered":"2024-04-25T11:19:48.302053+0000","last_unstale":"2024-04-25T11:19:49.299044+0000","last_undegraded":"2024-04-25T11:19:49.299044+0000","last_fullsized":"2024-04-25T11:19:49.299044+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:12.404652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.294837+0000","last_change":"2024-04-25T11:19:48.296299+0000","last_active":"2024-04-25T11:19:49.294837+0000","last_peered":"2024-04-25T11:19:49.294837+0000","last_clean":"2024-04-25T11:19:49.294837+0000","last_became_active":"2024-04-25T11:19:48.295681+0000","last_became_peered":"2024-04-25T11:19:48.295681+0000","last_unstale":"2024-04-25T11:19:49.294837+0000","last_undegraded":"2024-04-25T11:19:49.294837+0000","last_fullsized":"2024-04-25T11:19:49.294837+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:56.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298061+0000","last_change":"2024-04-25T11:19:48.297924+0000","last_active":"2024-04-25T11:19:49.298061+0000","last_peered":"2024-04-25T11:19:49.298061+0000","last_clean":"2024-04-25T11:19:49.298061+0000","last_became_active":"2024-04-25T11:19:48.293163+0000","last_became_peered":"2024-04-25T11:19:48.293163+0000","last_unstale":"2024-04-25T11:19:49.298061+0000","last_undegraded":"2024-04-25T11:19:49.298061+0000","last_fullsized":"2024-04-25T11:19:49.298061+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:44.806508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.298886+0000","last_change":"2024-04-25T11:19:48.294110+0000","last_active":"2024-04-25T11:19:48.298886+0000","last_peered":"2024-04-25T11:19:48.298886+0000","last_clean":"2024-04-25T11:19:48.298886+0000","last_became_active":"2024-04-25T11:19:48.292284+0000","last_became_peered":"2024-04-25T11:19:48.292284+0000","last_unstale":"2024-04-25T11:19:48.298886+0000","last_undegraded":"2024-04-25T11:19:48.298886+0000","last_fullsized":"2024-04-25T11:19:48.298886+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:00.207963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296771+0000","last_change":"2024-04-25T11:19:48.314086+0000","last_active":"2024-04-25T11:19:49.296771+0000","last_peered":"2024-04-25T11:19:49.296771+0000","last_clean":"2024-04-25T11:19:49.296771+0000","last_became_active":"2024-04-25T11:19:48.310126+0000","last_became_peered":"2024-04-25T11:19:48.310126+0000","last_unstale":"2024-04-25T11:19:49.296771+0000","last_undegraded":"2024-04-25T11:19:49.296771+0000","last_fullsized":"2024-04-25T11:19:49.296771+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:01.201766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304086+0000","last_change":"2024-04-25T11:19:48.302539+0000","last_active":"2024-04-25T11:19:49.304086+0000","last_peered":"2024-04-25T11:19:49.304086+0000","last_clean":"2024-04-25T11:19:49.304086+0000","last_became_active":"2024-04-25T11:19:48.302117+0000","last_became_peered":"2024-04-25T11:19:48.302117+0000","last_unstale":"2024-04-25T11:19:49.304086+0000","last_undegraded":"2024-04-25T11:19:49.304086+0000","last_fullsized":"2024-04-25T11:19:49.304086+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:47.151066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.301059+0000","last_change":"2024-04-25T11:19:48.299544+0000","last_active":"2024-04-25T11:19:48.301059+0000","last_peered":"2024-04-25T11:19:48.301059+0000","last_clean":"2024-04-25T11:19:48.301059+0000","last_became_active":"2024-04-25T11:19:48.299180+0000","last_became_peered":"2024-04-25T11:19:48.299180+0000","last_unstale":"2024-04-25T11:19:48.301059+0000","last_undegraded":"2024-04-25T11:19:48.301059+0000","last_fullsized":"2024-04-25T11:19:48.301059+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:36.484233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.298176+0000","last_change":"2024-04-25T11:19:48.289715+0000","last_active":"2024-04-25T11:19:48.298176+0000","last_peered":"2024-04-25T11:19:48.298176+0000","last_clean":"2024-04-25T11:19:48.298176+0000","last_became_active":"2024-04-25T11:19:48.289301+0000","last_became_peered":"2024-04-25T11:19:48.289301+0000","last_unstale":"2024-04-25T11:19:48.298176+0000","last_undegraded":"2024-04-25T11:19:48.298176+0000","last_fullsized":"2024-04-25T11:19:48.298176+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:44.806508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.295371+0000","last_change":"2024-04-25T11:19:48.290338+0000","last_active":"2024-04-25T11:19:49.295371+0000","last_peered":"2024-04-25T11:19:49.295371+0000","last_clean":"2024-04-25T11:19:49.295371+0000","last_became_active":"2024-04-25T11:19:48.289989+0000","last_became_peered":"2024-04-25T11:19:48.289989+0000","last_unstale":"2024-04-25T11:19:49.295371+0000","last_undegraded":"2024-04-25T11:19:49.295371+0000","last_fullsized":"2024-04-25T11:19:49.295371+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:40.030371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.295050+0000","last_change":"2024-04-25T11:19:48.295982+0000","last_active":"2024-04-25T11:19:49.295050+0000","last_peered":"2024-04-25T11:19:49.295050+0000","last_clean":"2024-04-25T11:19:49.295050+0000","last_became_active":"2024-04-25T11:19:48.295276+0000","last_became_peered":"2024-04-25T11:19:48.295276+0000","last_unstale":"2024-04-25T11:19:49.295050+0000","last_undegraded":"2024-04-25T11:19:49.295050+0000","last_fullsized":"2024-04-25T11:19:49.295050+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:01.201766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.313445+0000","last_change":"2024-04-25T11:19:48.305570+0000","last_active":"2024-04-25T11:19:48.313445+0000","last_peered":"2024-04-25T11:19:48.313445+0000","last_clean":"2024-04-25T11:19:48.313445+0000","last_became_active":"2024-04-25T11:19:48.305175+0000","last_became_peered":"2024-04-25T11:19:48.305175+0000","last_unstale":"2024-04-25T11:19:48.313445+0000","last_undegraded":"2024-04-25T11:19:48.313445+0000","last_fullsized":"2024-04-25T11:19:48.313445+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:15.693123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.295966+0000","last_change":"2024-04-25T11:19:47.261599+0000","last_active":"2024-04-25T11:19:49.295966+0000","last_peered":"2024-04-25T11:19:49.295966+0000","last_clean":"2024-04-25T11:19:49.295966+0000","last_became_active":"2024-04-25T11:19:47.261269+0000","last_became_peered":"2024-04-25T11:19:47.261269+0000","last_unstale":"2024-04-25T11:19:49.295966+0000","last_undegraded":"2024-04-25T11:19:49.295966+0000","last_fullsized":"2024-04-25T11:19:49.295966+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:42.042741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304131+0000","last_change":"2024-04-25T11:19:48.304592+0000","last_active":"2024-04-25T11:19:49.304131+0000","last_peered":"2024-04-25T11:19:49.304131+0000","last_clean":"2024-04-25T11:19:49.304131+0000","last_became_active":"2024-04-25T11:19:48.304241+0000","last_became_peered":"2024-04-25T11:19:48.304241+0000","last_unstale":"2024-04-25T11:19:49.304131+0000","last_undegraded":"2024-04-25T11:19:49.304131+0000","last_fullsized":"2024-04-25T11:19:49.304131+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:50.353341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.316570+0000","last_change":"2024-04-25T11:19:47.259114+0000","last_active":"2024-04-25T11:19:48.316570+0000","last_peered":"2024-04-25T11:19:48.316570+0000","last_clean":"2024-04-25T11:19:48.316570+0000","last_became_active":"2024-04-25T11:19:47.258626+0000","last_became_peered":"2024-04-25T11:19:47.258626+0000","last_unstale":"2024-04-25T11:19:48.316570+0000","last_undegraded":"2024-04-25T11:19:48.316570+0000","last_fullsized":"2024-04-25T11:19:48.316570+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:45.930118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297566+0000","last_change":"2024-04-25T11:19:48.307393+0000","last_active":"2024-04-25T11:19:49.297566+0000","last_peered":"2024-04-25T11:19:49.297566+0000","last_clean":"2024-04-25T11:19:49.297566+0000","last_became_active":"2024-04-25T11:19:48.307010+0000","last_became_peered":"2024-04-25T11:19:48.307010+0000","last_unstale":"2024-04-25T11:19:49.297566+0000","last_undegraded":"2024-04-25T11:19:49.297566+0000","last_fullsized":"2024-04-25T11:19:49.297566+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:47.792531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299674+0000","last_change":"2024-04-25T11:19:47.255903+0000","last_active":"2024-04-25T11:19:49.299674+0000","last_peered":"2024-04-25T11:19:49.299674+0000","last_clean":"2024-04-25T11:19:49.299674+0000","last_became_active":"2024-04-25T11:19:47.255258+0000","last_became_peered":"2024-04-25T11:19:47.255258+0000","last_unstale":"2024-04-25T11:19:49.299674+0000","last_undegraded":"2024-04-25T11:19:49.299674+0000","last_fullsized":"2024-04-25T11:19:49.299674+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:46.778083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299752+0000","last_change":"2024-04-25T11:19:48.301009+0000","last_active":"2024-04-25T11:19:49.299752+0000","last_peered":"2024-04-25T11:19:49.299752+0000","last_clean":"2024-04-25T11:19:49.299752+0000","last_became_active":"2024-04-25T11:19:48.300259+0000","last_became_peered":"2024-04-25T11:19:48.300259+0000","last_unstale":"2024-04-25T11:19:49.299752+0000","last_undegraded":"2024-04-25T11:19:49.299752+0000","last_fullsized":"2024-04-25T11:19:49.299752+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:22.874651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.321625+0000","last_change":"2024-04-25T11:19:47.248687+0000","last_active":"2024-04-25T11:19:48.321625+0000","last_peered":"2024-04-25T11:19:48.321625+0000","last_clean":"2024-04-25T11:19:48.321625+0000","last_became_active":"2024-04-25T11:19:47.248303+0000","last_became_peered":"2024-04-25T11:19:47.248303+0000","last_unstale":"2024-04-25T11:19:48.321625+0000","last_undegraded":"2024-04-25T11:19:48.321625+0000","last_fullsized":"2024-04-25T11:19:48.321625+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:37.075222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.298746+0000","last_change":"2024-04-25T11:19:48.292006+0000","last_active":"2024-04-25T11:19:48.298746+0000","last_peered":"2024-04-25T11:19:48.298746+0000","last_clean":"2024-04-25T11:19:48.298746+0000","last_became_active":"2024-04-25T11:19:48.291652+0000","last_became_peered":"2024-04-25T11:19:48.291652+0000","last_unstale":"2024-04-25T11:19:48.298746+0000","last_undegraded":"2024-04-25T11:19:48.298746+0000","last_fullsized":"2024-04-25T11:19:48.298746+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:01.249219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731159+0000","last_change":"2024-04-25T11:19:47.264898+0000","last_active":"2024-04-25T11:19:49.731159+0000","last_peered":"2024-04-25T11:19:49.731159+0000","last_clean":"2024-04-25T11:19:49.731159+0000","last_became_active":"2024-04-25T11:19:47.264573+0000","last_became_peered":"2024-04-25T11:19:47.264573+0000","last_unstale":"2024-04-25T11:19:49.731159+0000","last_undegraded":"2024-04-25T11:19:49.731159+0000","last_fullsized":"2024-04-25T11:19:49.731159+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:45.930118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731053+0000","last_change":"2024-04-25T11:19:48.296475+0000","last_active":"2024-04-25T11:19:49.731053+0000","last_peered":"2024-04-25T11:19:49.731053+0000","last_clean":"2024-04-25T11:19:49.731053+0000","last_became_active":"2024-04-25T11:19:48.296158+0000","last_became_peered":"2024-04-25T11:19:48.296158+0000","last_unstale":"2024-04-25T11:19:49.731053+0000","last_undegraded":"2024-04-25T11:19:49.731053+0000","last_fullsized":"2024-04-25T11:19:49.731053+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:56.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303271+0000","last_change":"2024-04-25T11:19:47.271179+0000","last_active":"2024-04-25T11:19:49.303271+0000","last_peered":"2024-04-25T11:19:49.303271+0000","last_clean":"2024-04-25T11:19:49.303271+0000","last_became_active":"2024-04-25T11:19:47.270853+0000","last_became_peered":"2024-04-25T11:19:47.270853+0000","last_unstale":"2024-04-25T11:19:49.303271+0000","last_undegraded":"2024-04-25T11:19:49.303271+0000","last_fullsized":"2024-04-25T11:19:49.303271+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:50.630486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.293711+0000","last_change":"2024-04-25T11:19:48.281107+0000","last_active":"2024-04-25T11:19:48.293711+0000","last_peered":"2024-04-25T11:19:48.293711+0000","last_clean":"2024-04-25T11:19:48.293711+0000","last_became_active":"2024-04-25T11:19:48.280714+0000","last_became_peered":"2024-04-25T11:19:48.280714+0000","last_unstale":"2024-04-25T11:19:48.293711+0000","last_undegraded":"2024-04-25T11:19:48.293711+0000","last_fullsized":"2024-04-25T11:19:48.293711+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:38.216023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296040+0000","last_change":"2024-04-25T11:19:47.250262+0000","last_active":"2024-04-25T11:19:49.296040+0000","last_peered":"2024-04-25T11:19:49.296040+0000","last_clean":"2024-04-25T11:19:49.296040+0000","last_became_active":"2024-04-25T11:19:47.249571+0000","last_became_peered":"2024-04-25T11:19:47.249571+0000","last_unstale":"2024-04-25T11:19:49.296040+0000","last_undegraded":"2024-04-25T11:19:49.296040+0000","last_fullsized":"2024-04-25T11:19:49.296040+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:45.930118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.298696+0000","last_change":"2024-04-25T11:19:48.291799+0000","last_active":"2024-04-25T11:19:48.298696+0000","last_peered":"2024-04-25T11:19:48.298696+0000","last_clean":"2024-04-25T11:19:48.298696+0000","last_became_active":"2024-04-25T11:19:48.291449+0000","last_became_peered":"2024-04-25T11:19:48.291449+0000","last_unstale":"2024-04-25T11:19:48.298696+0000","last_undegraded":"2024-04-25T11:19:48.298696+0000","last_fullsized":"2024-04-25T11:19:48.298696+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:08.012932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303156+0000","last_change":"2024-04-25T11:19:47.263182+0000","last_active":"2024-04-25T11:19:49.303156+0000","last_peered":"2024-04-25T11:19:49.303156+0000","last_clean":"2024-04-25T11:19:49.303156+0000","last_became_active":"2024-04-25T11:19:47.262824+0000","last_became_peered":"2024-04-25T11:19:47.262824+0000","last_unstale":"2024-04-25T11:19:49.303156+0000","last_undegraded":"2024-04-25T11:19:49.303156+0000","last_fullsized":"2024-04-25T11:19:49.303156+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.671190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.295663+0000","last_change":"2024-04-25T11:19:48.306114+0000","last_active":"2024-04-25T11:19:49.295663+0000","last_peered":"2024-04-25T11:19:49.295663+0000","last_clean":"2024-04-25T11:19:49.295663+0000","last_became_active":"2024-04-25T11:19:48.305426+0000","last_became_peered":"2024-04-25T11:19:48.305426+0000","last_unstale":"2024-04-25T11:19:49.295663+0000","last_undegraded":"2024-04-25T11:19:49.295663+0000","last_fullsized":"2024-04-25T11:19:49.295663+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:23.688909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296358+0000","last_change":"2024-04-25T11:19:47.265821+0000","last_active":"2024-04-25T11:19:49.296358+0000","last_peered":"2024-04-25T11:19:49.296358+0000","last_clean":"2024-04-25T11:19:49.296358+0000","last_became_active":"2024-04-25T11:19:47.265396+0000","last_became_peered":"2024-04-25T11:19:47.265396+0000","last_unstale":"2024-04-25T11:19:49.296358+0000","last_undegraded":"2024-04-25T11:19:49.296358+0000","last_fullsized":"2024-04-25T11:19:49.296358+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:12.508447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297929+0000","last_change":"2024-04-25T11:19:48.281087+0000","last_active":"2024-04-25T11:19:49.297929+0000","last_peered":"2024-04-25T11:19:49.297929+0000","last_clean":"2024-04-25T11:19:49.297929+0000","last_became_active":"2024-04-25T11:19:48.280578+0000","last_became_peered":"2024-04-25T11:19:48.280578+0000","last_unstale":"2024-04-25T11:19:49.297929+0000","last_undegraded":"2024-04-25T11:19:49.297929+0000","last_fullsized":"2024-04-25T11:19:49.297929+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:38.216023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297685+0000","last_change":"2024-04-25T11:19:47.258465+0000","last_active":"2024-04-25T11:19:49.297685+0000","last_peered":"2024-04-25T11:19:49.297685+0000","last_clean":"2024-04-25T11:19:49.297685+0000","last_became_active":"2024-04-25T11:19:47.258061+0000","last_became_peered":"2024-04-25T11:19:47.258061+0000","last_unstale":"2024-04-25T11:19:49.297685+0000","last_undegraded":"2024-04-25T11:19:49.297685+0000","last_fullsized":"2024-04-25T11:19:49.297685+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:29.385598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.306593+0000","last_change":"2024-04-25T11:19:48.301628+0000","last_active":"2024-04-25T11:19:48.306593+0000","last_peered":"2024-04-25T11:19:48.306593+0000","last_clean":"2024-04-25T11:19:48.306593+0000","last_became_active":"2024-04-25T11:19:48.301296+0000","last_became_peered":"2024-04-25T11:19:48.301296+0000","last_unstale":"2024-04-25T11:19:48.306593+0000","last_undegraded":"2024-04-25T11:19:48.306593+0000","last_fullsized":"2024-04-25T11:19:48.306593+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:46.417609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298410+0000","last_change":"2024-04-25T11:19:47.263397+0000","last_active":"2024-04-25T11:19:49.298410+0000","last_peered":"2024-04-25T11:19:49.298410+0000","last_clean":"2024-04-25T11:19:49.298410+0000","last_became_active":"2024-04-25T11:19:47.262983+0000","last_became_peered":"2024-04-25T11:19:47.262983+0000","last_unstale":"2024-04-25T11:19:49.298410+0000","last_undegraded":"2024-04-25T11:19:49.298410+0000","last_fullsized":"2024-04-25T11:19:49.298410+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:16.259744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298271+0000","last_change":"2024-04-25T11:19:48.294009+0000","last_active":"2024-04-25T11:19:49.298271+0000","last_peered":"2024-04-25T11:19:49.298271+0000","last_clean":"2024-04-25T11:19:49.298271+0000","last_became_active":"2024-04-25T11:19:48.291682+0000","last_became_peered":"2024-04-25T11:19:48.291682+0000","last_unstale":"2024-04-25T11:19:49.298271+0000","last_undegraded":"2024-04-25T11:19:49.298271+0000","last_fullsized":"2024-04-25T11:19:49.298271+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:48.173226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"117'6","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.323294+0000","last_change":"2024-04-25T11:19:47.263550+0000","last_active":"2024-04-25T11:19:48.323294+0000","last_peered":"2024-04-25T11:19:48.323294+0000","last_clean":"2024-04-25T11:19:48.323294+0000","last_became_active":"2024-04-25T11:19:47.263170+0000","last_became_peered":"2024-04-25T11:19:47.263170+0000","last_unstale":"2024-04-25T11:19:48.323294+0000","last_undegraded":"2024-04-25T11:19:48.323294+0000","last_fullsized":"2024-04-25T11:19:48.323294+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:00.187318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.295363+0000","last_change":"2024-04-25T11:19:48.303426+0000","last_active":"2024-04-25T11:19:49.295363+0000","last_peered":"2024-04-25T11:19:49.295363+0000","last_clean":"2024-04-25T11:19:49.295363+0000","last_became_active":"2024-04-25T11:19:48.302608+0000","last_became_peered":"2024-04-25T11:19:48.302608+0000","last_unstale":"2024-04-25T11:19:49.295363+0000","last_undegraded":"2024-04-25T11:19:49.295363+0000","last_fullsized":"2024-04-25T11:19:49.295363+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:59.096555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.321759+0000","last_change":"2024-04-25T11:19:47.260450+0000","last_active":"2024-04-25T11:19:48.321759+0000","last_peered":"2024-04-25T11:19:48.321759+0000","last_clean":"2024-04-25T11:19:48.321759+0000","last_became_active":"2024-04-25T11:19:47.260145+0000","last_became_peered":"2024-04-25T11:19:47.260145+0000","last_unstale":"2024-04-25T11:19:48.321759+0000","last_undegraded":"2024-04-25T11:19:48.321759+0000","last_fullsized":"2024-04-25T11:19:48.321759+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:55.751855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.309790+0000","last_change":"2024-04-25T11:19:48.295943+0000","last_active":"2024-04-25T11:19:48.309790+0000","last_peered":"2024-04-25T11:19:48.309790+0000","last_clean":"2024-04-25T11:19:48.309790+0000","last_became_active":"2024-04-25T11:19:48.295568+0000","last_became_peered":"2024-04-25T11:19:48.295568+0000","last_unstale":"2024-04-25T11:19:48.309790+0000","last_undegraded":"2024-04-25T11:19:48.309790+0000","last_fullsized":"2024-04-25T11:19:48.309790+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:07.267851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.276715+0000","last_change":"2024-04-25T11:19:47.239820+0000","last_active":"2024-04-25T11:19:48.276715+0000","last_peered":"2024-04-25T11:19:48.276715+0000","last_clean":"2024-04-25T11:19:48.276715+0000","last_became_active":"2024-04-25T11:19:47.239358+0000","last_became_peered":"2024-04-25T11:19:47.239358+0000","last_unstale":"2024-04-25T11:19:48.276715+0000","last_undegraded":"2024-04-25T11:19:48.276715+0000","last_fullsized":"2024-04-25T11:19:48.276715+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:46.778083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.297035+0000","last_change":"2024-04-25T11:19:48.290975+0000","last_active":"2024-04-25T11:19:48.297035+0000","last_peered":"2024-04-25T11:19:48.297035+0000","last_clean":"2024-04-25T11:19:48.297035+0000","last_became_active":"2024-04-25T11:19:48.290375+0000","last_became_peered":"2024-04-25T11:19:48.290375+0000","last_unstale":"2024-04-25T11:19:48.297035+0000","last_undegraded":"2024-04-25T11:19:48.297035+0000","last_fullsized":"2024-04-25T11:19:48.297035+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:50.496441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.272153+0000","last_change":"2024-04-25T11:19:47.259544+0000","last_active":"2024-04-25T11:19:48.272153+0000","last_peered":"2024-04-25T11:19:48.272153+0000","last_clean":"2024-04-25T11:19:48.272153+0000","last_became_active":"2024-04-25T11:19:47.256173+0000","last_became_peered":"2024-04-25T11:19:47.256173+0000","last_unstale":"2024-04-25T11:19:48.272153+0000","last_undegraded":"2024-04-25T11:19:48.272153+0000","last_fullsized":"2024-04-25T11:19:48.272153+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:39.015923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297588+0000","last_change":"2024-04-25T11:19:48.307395+0000","last_active":"2024-04-25T11:19:49.297588+0000","last_peered":"2024-04-25T11:19:49.297588+0000","last_clean":"2024-04-25T11:19:49.297588+0000","last_became_active":"2024-04-25T11:19:48.306776+0000","last_became_peered":"2024-04-25T11:19:48.306776+0000","last_unstale":"2024-04-25T11:19:49.297588+0000","last_undegraded":"2024-04-25T11:19:49.297588+0000","last_fullsized":"2024-04-25T11:19:49.297588+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:27.175206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.319677+0000","last_change":"2024-04-25T11:19:47.267880+0000","last_active":"2024-04-25T11:19:48.319677+0000","last_peered":"2024-04-25T11:19:48.319677+0000","last_clean":"2024-04-25T11:19:48.319677+0000","last_became_active":"2024-04-25T11:19:47.266288+0000","last_became_peered":"2024-04-25T11:19:47.266288+0000","last_unstale":"2024-04-25T11:19:48.319677+0000","last_undegraded":"2024-04-25T11:19:48.319677+0000","last_fullsized":"2024-04-25T11:19:48.319677+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:16.259744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731630+0000","last_change":"2024-04-25T11:19:48.300670+0000","last_active":"2024-04-25T11:19:49.731630+0000","last_peered":"2024-04-25T11:19:49.731630+0000","last_clean":"2024-04-25T11:19:49.731630+0000","last_became_active":"2024-04-25T11:19:48.300304+0000","last_became_peered":"2024-04-25T11:19:48.300304+0000","last_unstale":"2024-04-25T11:19:49.731630+0000","last_undegraded":"2024-04-25T11:19:49.731630+0000","last_fullsized":"2024-04-25T11:19:49.731630+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:40.030371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.319503+0000","last_change":"2024-04-25T11:19:47.266268+0000","last_active":"2024-04-25T11:19:48.319503+0000","last_peered":"2024-04-25T11:19:48.319503+0000","last_clean":"2024-04-25T11:19:48.319503+0000","last_became_active":"2024-04-25T11:19:47.265332+0000","last_became_peered":"2024-04-25T11:19:47.265332+0000","last_unstale":"2024-04-25T11:19:48.319503+0000","last_undegraded":"2024-04-25T11:19:48.319503+0000","last_fullsized":"2024-04-25T11:19:48.319503+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:34.128714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304644+0000","last_change":"2024-04-25T11:19:48.312255+0000","last_active":"2024-04-25T11:19:49.304644+0000","last_peered":"2024-04-25T11:19:49.304644+0000","last_clean":"2024-04-25T11:19:49.304644+0000","last_became_active":"2024-04-25T11:19:48.311090+0000","last_became_peered":"2024-04-25T11:19:48.311090+0000","last_unstale":"2024-04-25T11:19:49.304644+0000","last_undegraded":"2024-04-25T11:19:49.304644+0000","last_fullsized":"2024-04-25T11:19:49.304644+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:17.845098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299573+0000","last_change":"2024-04-25T11:19:47.256564+0000","last_active":"2024-04-25T11:19:49.299573+0000","last_peered":"2024-04-25T11:19:49.299573+0000","last_clean":"2024-04-25T11:19:49.299573+0000","last_became_active":"2024-04-25T11:19:47.255603+0000","last_became_peered":"2024-04-25T11:19:47.255603+0000","last_unstale":"2024-04-25T11:19:49.299573+0000","last_undegraded":"2024-04-25T11:19:49.299573+0000","last_fullsized":"2024-04-25T11:19:49.299573+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:26.160757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298312+0000","last_change":"2024-04-25T11:19:48.276843+0000","last_active":"2024-04-25T11:19:49.298312+0000","last_peered":"2024-04-25T11:19:49.298312+0000","last_clean":"2024-04-25T11:19:49.298312+0000","last_became_active":"2024-04-25T11:19:48.276450+0000","last_became_peered":"2024-04-25T11:19:48.276450+0000","last_unstale":"2024-04-25T11:19:49.298312+0000","last_undegraded":"2024-04-25T11:19:49.298312+0000","last_fullsized":"2024-04-25T11:19:49.298312+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:23.899534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299242+0000","last_change":"2024-04-25T11:19:47.269430+0000","last_active":"2024-04-25T11:19:49.299242+0000","last_peered":"2024-04-25T11:19:49.299242+0000","last_clean":"2024-04-25T11:19:49.299242+0000","last_became_active":"2024-04-25T11:19:47.269090+0000","last_became_peered":"2024-04-25T11:19:47.269090+0000","last_unstale":"2024-04-25T11:19:49.299242+0000","last_undegraded":"2024-04-25T11:19:49.299242+0000","last_fullsized":"2024-04-25T11:19:49.299242+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:39.015923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.302626+0000","last_change":"2024-04-25T11:19:48.303957+0000","last_active":"2024-04-25T11:19:49.302626+0000","last_peered":"2024-04-25T11:19:49.302626+0000","last_clean":"2024-04-25T11:19:49.302626+0000","last_became_active":"2024-04-25T11:19:48.302860+0000","last_became_peered":"2024-04-25T11:19:48.302860+0000","last_unstale":"2024-04-25T11:19:49.302626+0000","last_undegraded":"2024-04-25T11:19:49.302626+0000","last_fullsized":"2024-04-25T11:19:49.302626+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:43.323898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303434+0000","last_change":"2024-04-25T11:19:47.267653+0000","last_active":"2024-04-25T11:19:49.303434+0000","last_peered":"2024-04-25T11:19:49.303434+0000","last_clean":"2024-04-25T11:19:49.303434+0000","last_became_active":"2024-04-25T11:19:47.267317+0000","last_became_peered":"2024-04-25T11:19:47.267317+0000","last_unstale":"2024-04-25T11:19:49.303434+0000","last_undegraded":"2024-04-25T11:19:49.303434+0000","last_fullsized":"2024-04-25T11:19:49.303434+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:47.171241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.299201+0000","last_change":"2024-04-25T11:19:48.290999+0000","last_active":"2024-04-25T11:19:48.299201+0000","last_peered":"2024-04-25T11:19:48.299201+0000","last_clean":"2024-04-25T11:19:48.299201+0000","last_became_active":"2024-04-25T11:19:48.289317+0000","last_became_peered":"2024-04-25T11:19:48.289317+0000","last_unstale":"2024-04-25T11:19:48.299201+0000","last_undegraded":"2024-04-25T11:19:48.299201+0000","last_fullsized":"2024-04-25T11:19:48.299201+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:48.173226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298693+0000","last_change":"2024-04-25T11:19:47.257548+0000","last_active":"2024-04-25T11:19:49.298693+0000","last_peered":"2024-04-25T11:19:49.298693+0000","last_clean":"2024-04-25T11:19:49.298693+0000","last_became_active":"2024-04-25T11:19:47.257245+0000","last_became_peered":"2024-04-25T11:19:47.257245+0000","last_unstale":"2024-04-25T11:19:49.298693+0000","last_undegraded":"2024-04-25T11:19:49.298693+0000","last_fullsized":"2024-04-25T11:19:49.298693+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:00.187318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.306196+0000","last_change":"2024-04-25T11:19:48.294491+0000","last_active":"2024-04-25T11:19:48.306196+0000","last_peered":"2024-04-25T11:19:48.306196+0000","last_clean":"2024-04-25T11:19:48.306196+0000","last_became_active":"2024-04-25T11:19:48.292416+0000","last_became_peered":"2024-04-25T11:19:48.292416+0000","last_unstale":"2024-04-25T11:19:48.306196+0000","last_undegraded":"2024-04-25T11:19:48.306196+0000","last_fullsized":"2024-04-25T11:19:48.306196+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.076716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.272465+0000","last_change":"2024-04-25T11:19:47.263206+0000","last_active":"2024-04-25T11:19:48.272465+0000","last_peered":"2024-04-25T11:19:48.272465+0000","last_clean":"2024-04-25T11:19:48.272465+0000","last_became_active":"2024-04-25T11:19:47.262829+0000","last_became_peered":"2024-04-25T11:19:47.262829+0000","last_unstale":"2024-04-25T11:19:48.272465+0000","last_undegraded":"2024-04-25T11:19:48.272465+0000","last_fullsized":"2024-04-25T11:19:48.272465+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:23.485514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296792+0000","last_change":"2024-04-25T11:19:48.291641+0000","last_active":"2024-04-25T11:19:49.296792+0000","last_peered":"2024-04-25T11:19:49.296792+0000","last_clean":"2024-04-25T11:19:49.296792+0000","last_became_active":"2024-04-25T11:19:48.288628+0000","last_became_peered":"2024-04-25T11:19:48.288628+0000","last_unstale":"2024-04-25T11:19:49.296792+0000","last_undegraded":"2024-04-25T11:19:49.296792+0000","last_fullsized":"2024-04-25T11:19:49.296792+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:24.499963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296722+0000","last_change":"2024-04-25T11:19:47.254409+0000","last_active":"2024-04-25T11:19:49.296722+0000","last_peered":"2024-04-25T11:19:49.296722+0000","last_clean":"2024-04-25T11:19:49.296722+0000","last_became_active":"2024-04-25T11:19:47.254033+0000","last_became_peered":"2024-04-25T11:19:47.254033+0000","last_unstale":"2024-04-25T11:19:49.296722+0000","last_undegraded":"2024-04-25T11:19:49.296722+0000","last_fullsized":"2024-04-25T11:19:49.296722+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:23.485514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304467+0000","last_change":"2024-04-25T11:19:48.302987+0000","last_active":"2024-04-25T11:19:49.304467+0000","last_peered":"2024-04-25T11:19:49.304467+0000","last_clean":"2024-04-25T11:19:49.304467+0000","last_became_active":"2024-04-25T11:19:48.302547+0000","last_became_peered":"2024-04-25T11:19:48.302547+0000","last_unstale":"2024-04-25T11:19:49.304467+0000","last_undegraded":"2024-04-25T11:19:49.304467+0000","last_fullsized":"2024-04-25T11:19:49.304467+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:51.228850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304723+0000","last_change":"2024-04-25T11:19:47.268843+0000","last_active":"2024-04-25T11:19:49.304723+0000","last_peered":"2024-04-25T11:19:49.304723+0000","last_clean":"2024-04-25T11:19:49.304723+0000","last_became_active":"2024-04-25T11:19:47.267167+0000","last_became_peered":"2024-04-25T11:19:47.267167+0000","last_unstale":"2024-04-25T11:19:49.304723+0000","last_undegraded":"2024-04-25T11:19:49.304723+0000","last_fullsized":"2024-04-25T11:19:49.304723+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:29.994468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.310214+0000","last_change":"2024-04-25T11:19:48.295452+0000","last_active":"2024-04-25T11:19:48.310214+0000","last_peered":"2024-04-25T11:19:48.310214+0000","last_clean":"2024-04-25T11:19:48.310214+0000","last_became_active":"2024-04-25T11:19:48.295030+0000","last_became_peered":"2024-04-25T11:19:48.295030+0000","last_unstale":"2024-04-25T11:19:48.310214+0000","last_undegraded":"2024-04-25T11:19:48.310214+0000","last_fullsized":"2024-04-25T11:19:48.310214+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.057189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"118'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.301137+0000","last_change":"2024-04-25T11:19:47.267333+0000","last_active":"2024-04-25T11:19:49.301137+0000","last_peered":"2024-04-25T11:19:49.301137+0000","last_clean":"2024-04-25T11:19:49.301137+0000","last_became_active":"2024-04-25T11:19:47.267013+0000","last_became_peered":"2024-04-25T11:19:47.267013+0000","last_unstale":"2024-04-25T11:19:49.301137+0000","last_undegraded":"2024-04-25T11:19:49.301137+0000","last_fullsized":"2024-04-25T11:19:49.301137+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:00.234771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299365+0000","last_change":"2024-04-25T11:19:48.290489+0000","last_active":"2024-04-25T11:19:49.299365+0000","last_peered":"2024-04-25T11:19:49.299365+0000","last_clean":"2024-04-25T11:19:49.299365+0000","last_became_active":"2024-04-25T11:19:48.290168+0000","last_became_peered":"2024-04-25T11:19:48.290168+0000","last_unstale":"2024-04-25T11:19:49.299365+0000","last_undegraded":"2024-04-25T11:19:49.299365+0000","last_fullsized":"2024-04-25T11:19:49.299365+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:27.031412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296631+0000","last_change":"2024-04-25T11:19:47.265807+0000","last_active":"2024-04-25T11:19:49.296631+0000","last_peered":"2024-04-25T11:19:49.296631+0000","last_clean":"2024-04-25T11:19:49.296631+0000","last_became_active":"2024-04-25T11:19:47.265396+0000","last_became_peered":"2024-04-25T11:19:47.265396+0000","last_unstale":"2024-04-25T11:19:49.296631+0000","last_undegraded":"2024-04-25T11:19:49.296631+0000","last_fullsized":"2024-04-25T11:19:49.296631+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:06.253402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.303131+0000","last_change":"2024-04-25T11:19:48.301298+0000","last_active":"2024-04-25T11:19:48.303131+0000","last_peered":"2024-04-25T11:19:48.303131+0000","last_clean":"2024-04-25T11:19:48.303131+0000","last_became_active":"2024-04-25T11:19:48.300888+0000","last_became_peered":"2024-04-25T11:19:48.300888+0000","last_unstale":"2024-04-25T11:19:48.303131+0000","last_undegraded":"2024-04-25T11:19:48.303131+0000","last_fullsized":"2024-04-25T11:19:48.303131+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:34.374026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304593+0000","last_change":"2024-04-25T11:19:47.265317+0000","last_active":"2024-04-25T11:19:49.304593+0000","last_peered":"2024-04-25T11:19:49.304593+0000","last_clean":"2024-04-25T11:19:49.304593+0000","last_became_active":"2024-04-25T11:19:47.264636+0000","last_became_peered":"2024-04-25T11:19:47.264636+0000","last_unstale":"2024-04-25T11:19:49.304593+0000","last_undegraded":"2024-04-25T11:19:49.304593+0000","last_fullsized":"2024-04-25T11:19:49.304593+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:06.680032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303238+0000","last_change":"2024-04-25T11:19:48.303689+0000","last_active":"2024-04-25T11:19:49.303238+0000","last_peered":"2024-04-25T11:19:49.303238+0000","last_clean":"2024-04-25T11:19:49.303238+0000","last_became_active":"2024-04-25T11:19:48.302135+0000","last_became_peered":"2024-04-25T11:19:48.302135+0000","last_unstale":"2024-04-25T11:19:49.303238+0000","last_undegraded":"2024-04-25T11:19:49.303238+0000","last_fullsized":"2024-04-25T11:19:49.303238+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:44.398813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731182+0000","last_change":"2024-04-25T11:19:47.259012+0000","last_active":"2024-04-25T11:19:49.731182+0000","last_peered":"2024-04-25T11:19:49.731182+0000","last_clean":"2024-04-25T11:19:49.731182+0000","last_became_active":"2024-04-25T11:19:47.257947+0000","last_became_peered":"2024-04-25T11:19:47.257947+0000","last_unstale":"2024-04-25T11:19:49.731182+0000","last_undegraded":"2024-04-25T11:19:49.731182+0000","last_fullsized":"2024-04-25T11:19:49.731182+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:16.259744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.304877+0000","last_change":"2024-04-25T11:19:48.295181+0000","last_active":"2024-04-25T11:19:48.304877+0000","last_peered":"2024-04-25T11:19:48.304877+0000","last_clean":"2024-04-25T11:19:48.304877+0000","last_became_active":"2024-04-25T11:19:48.294861+0000","last_became_peered":"2024-04-25T11:19:48.294861+0000","last_unstale":"2024-04-25T11:19:48.304877+0000","last_undegraded":"2024-04-25T11:19:48.304877+0000","last_fullsized":"2024-04-25T11:19:48.304877+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:52.969575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731025+0000","last_change":"2024-04-25T11:19:47.258297+0000","last_active":"2024-04-25T11:19:49.731025+0000","last_peered":"2024-04-25T11:19:49.731025+0000","last_clean":"2024-04-25T11:19:49.731025+0000","last_became_active":"2024-04-25T11:19:47.257964+0000","last_became_peered":"2024-04-25T11:19:47.257964+0000","last_unstale":"2024-04-25T11:19:49.731025+0000","last_undegraded":"2024-04-25T11:19:49.731025+0000","last_fullsized":"2024-04-25T11:19:49.731025+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:29.385598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298570+0000","last_change":"2024-04-25T11:19:48.309559+0000","last_active":"2024-04-25T11:19:49.298570+0000","last_peered":"2024-04-25T11:19:49.298570+0000","last_clean":"2024-04-25T11:19:49.298570+0000","last_became_active":"2024-04-25T11:19:48.309212+0000","last_became_peered":"2024-04-25T11:19:48.309212+0000","last_unstale":"2024-04-25T11:19:49.298570+0000","last_undegraded":"2024-04-25T11:19:49.298570+0000","last_fullsized":"2024-04-25T11:19:49.298570+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:46.417609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.302923+0000","last_change":"2024-04-25T11:19:47.267575+0000","last_active":"2024-04-25T11:19:49.302923+0000","last_peered":"2024-04-25T11:19:49.302923+0000","last_clean":"2024-04-25T11:19:49.302923+0000","last_became_active":"2024-04-25T11:19:47.267161+0000","last_became_peered":"2024-04-25T11:19:47.267161+0000","last_unstale":"2024-04-25T11:19:49.302923+0000","last_undegraded":"2024-04-25T11:19:49.302923+0000","last_fullsized":"2024-04-25T11:19:49.302923+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:44.763800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.310275+0000","last_change":"2024-04-25T11:19:48.295474+0000","last_active":"2024-04-25T11:19:48.310275+0000","last_peered":"2024-04-25T11:19:48.310275+0000","last_clean":"2024-04-25T11:19:48.310275+0000","last_became_active":"2024-04-25T11:19:48.295107+0000","last_became_peered":"2024-04-25T11:19:48.295107+0000","last_unstale":"2024-04-25T11:19:48.310275+0000","last_undegraded":"2024-04-25T11:19:48.310275+0000","last_fullsized":"2024-04-25T11:19:48.310275+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:13.522895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297011+0000","last_change":"2024-04-25T11:19:47.260306+0000","last_active":"2024-04-25T11:19:49.297011+0000","last_peered":"2024-04-25T11:19:49.297011+0000","last_clean":"2024-04-25T11:19:49.297011+0000","last_became_active":"2024-04-25T11:19:47.259366+0000","last_became_peered":"2024-04-25T11:19:47.259366+0000","last_unstale":"2024-04-25T11:19:49.297011+0000","last_undegraded":"2024-04-25T11:19:49.297011+0000","last_fullsized":"2024-04-25T11:19:49.297011+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:55.751855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299431+0000","last_change":"2024-04-25T11:19:48.297647+0000","last_active":"2024-04-25T11:19:49.299431+0000","last_peered":"2024-04-25T11:19:49.299431+0000","last_clean":"2024-04-25T11:19:49.299431+0000","last_became_active":"2024-04-25T11:19:48.296834+0000","last_became_peered":"2024-04-25T11:19:48.296834+0000","last_unstale":"2024-04-25T11:19:49.299431+0000","last_undegraded":"2024-04-25T11:19:49.299431+0000","last_fullsized":"2024-04-25T11:19:49.299431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.203635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"117'8","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.322354+0000","last_change":"2024-04-25T11:19:47.262351+0000","last_active":"2024-04-25T11:19:48.322354+0000","last_peered":"2024-04-25T11:19:48.322354+0000","last_clean":"2024-04-25T11:19:48.322354+0000","last_became_active":"2024-04-25T11:19:47.261686+0000","last_became_peered":"2024-04-25T11:19:47.261686+0000","last_unstale":"2024-04-25T11:19:48.322354+0000","last_undegraded":"2024-04-25T11:19:48.322354+0000","last_fullsized":"2024-04-25T11:19:48.322354+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:22.674461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":304,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297696+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:19:49.297696+0000","last_peered":"2024-04-25T11:19:49.297696+0000","last_clean":"2024-04-25T11:19:49.297696+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:19:49.297696+0000","last_undegraded":"2024-04-25T11:19:49.297696+0000","last_fullsized":"2024-04-25T11:19:49.297696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2068480,"data_stored":1263704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":1462272,"data_stored":673142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1466368,"data_stored":672720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":1470464,"data_stored":674172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509558,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":667514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509559,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":665890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509558,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2056192,"data_stored":1262950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542263,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1445888,"data_stored":665868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:19:53.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:19:53.194 INFO:tasks.ceph.mds.d.smithi087.stderr:2024-04-25T11:19:53.196+0000 7f2712c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:19:53.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.373+0000 7f543fe00640 1 Processor -- start 2024-04-25T11:19:53.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.373+0000 7f543fe00640 1 -- start start 2024-04-25T11:19:53.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.375+0000 7f543fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54400b1610 0x7f54400cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.375+0000 7f543fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f54400b1d50 0x7f54400af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.375+0000 7f543fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54400b24d0 con 0x7f54400b73d0 2024-04-25T11:19:53.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.375+0000 7f543fe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f54400b2790 con 0x7f54400b1d50 2024-04-25T11:19:53.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.375+0000 7f543fe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f54400b2a70 con 0x7f54400b1610 2024-04-25T11:19:53.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.375+0000 7f543e000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f54400b1d50 0x7f54400af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.375+0000 7f543e000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f54400b1d50 0x7f54400af4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42464/0 (socket says 172.21.15.7:42464) 2024-04-25T11:19:53.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.375+0000 7f543e000640 1 -- 172.21.15.7:0/2897462827 learned_addr learned my addr 172.21.15.7:0/2897462827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.376+0000 7f543ea00640 1 --2- 172.21.15.7:0/2897462827 >> v2:172.21.15.87:3300/0 conn(0x7f54400b1610 0x7f54400cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.376+0000 7f543e000640 1 -- 172.21.15.7:0/2897462827 >> v2:172.21.15.87:3300/0 conn(0x7f54400b1610 msgr2=0x7f54400cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.376+0000 7f543e000640 1 --2- 172.21.15.7:0/2897462827 >> v2:172.21.15.87:3300/0 conn(0x7f54400b1610 0x7f54400cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.376+0000 7f543e000640 1 -- 172.21.15.7:0/2897462827 >> v1:172.21.15.87:6789/0 conn(0x7f54400b73d0 legacy=0x7f54400b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.376+0000 7f543e000640 1 -- 172.21.15.7:0/2897462827 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54400b2d50 con 0x7f54400b1d50 2024-04-25T11:19:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.376+0000 7f543ea00640 1 --2- 172.21.15.7:0/2897462827 >> v2:172.21.15.87:3300/0 conn(0x7f54400b1610 0x7f54400cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:53.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.377+0000 7f543e000640 1 --2- 172.21.15.7:0/2897462827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54400b1d50 0x7f54400af4c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f544006fe70 tx=0x7f542802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc5f1edba078ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:53.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.377+0000 7f543d600640 1 -- 172.21.15.7:0/2897462827 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428037070 con 0x7f54400b1d50 2024-04-25T11:19:53.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.377+0000 7f543d600640 1 -- 172.21.15.7:0/2897462827 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5428032440 con 0x7f54400b1d50 2024-04-25T11:19:53.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.377+0000 7f543d600640 1 -- 172.21.15.7:0/2897462827 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428032810 con 0x7f54400b1d50 2024-04-25T11:19:53.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.378+0000 7f543fe00640 1 -- 172.21.15.7:0/2897462827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54400b1d50 msgr2=0x7f54400af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:53.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.378+0000 7f543fe00640 1 --2- 172.21.15.7:0/2897462827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54400b1d50 0x7f54400af4c0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f544006fe70 tx=0x7f542802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.378+0000 7f543fe00640 1 -- 172.21.15.7:0/2897462827 shutdown_connections 2024-04-25T11:19:53.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.378+0000 7f543fe00640 1 --2- 172.21.15.7:0/2897462827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54400b1d50 0x7f54400af4c0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.378+0000 7f543fe00640 1 --2- 172.21.15.7:0/2897462827 >> v2:172.21.15.87:3300/0 conn(0x7f54400b1610 0x7f54400cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.378+0000 7f543fe00640 1 -- 172.21.15.7:0/2897462827 >> 172.21.15.7:0/2897462827 conn(0x7f544009b540 msgr2=0x7f544009b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:53.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.378+0000 7f543fe00640 1 -- 172.21.15.7:0/2897462827 shutdown_connections 2024-04-25T11:19:53.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.378+0000 7f543fe00640 1 -- 172.21.15.7:0/2897462827 wait complete. 2024-04-25T11:19:53.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.380+0000 7f543fe00640 1 Processor -- start 2024-04-25T11:19:53.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.381+0000 7f543fe00640 1 -- start start 2024-04-25T11:19:53.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.382+0000 7f543fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54400b1610 0x7f54400c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.382+0000 7f543fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54400b1d50 0x7f5440070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.382+0000 7f543fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54400b73d0 0x7f5440071450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.382+0000 7f543fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54400afa00 con 0x7f54400b1d50 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.382+0000 7f543fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54400b2160 con 0x7f54400b73d0 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.382+0000 7f543fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f54400afb80 con 0x7f54400b1610 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.383+0000 7f543e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54400b1d50 0x7f5440070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.383+0000 7f543e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54400b1d50 0x7f5440070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42466/0 (socket says 172.21.15.7:42466) 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.383+0000 7f543e000640 1 -- 172.21.15.7:0/2979448020 learned_addr learned my addr 172.21.15.7:0/2979448020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.383+0000 7f543e000640 1 -- 172.21.15.7:0/2979448020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54400b1610 msgr2=0x7f54400c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.383+0000 7f543ea00640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54400b1610 0x7f54400c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.383+0000 7f543f400640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54400b73d0 0x7f5440071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.383+0000 7f543e000640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54400b1610 0x7f54400c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.383+0000 7f543e000640 1 -- 172.21.15.7:0/2979448020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54400b73d0 msgr2=0x7f5440071450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.383+0000 7f543e000640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54400b73d0 0x7f5440071450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.383+0000 7f543e000640 1 -- 172.21.15.7:0/2979448020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54401f4f80 con 0x7f54400b1d50 2024-04-25T11:19:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.383+0000 7f543e000640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54400b1d50 0x7f5440070f10 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5428000c00 tx=0x7f54280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.384+0000 7f5436a00640 1 -- 172.21.15.7:0/2979448020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428041020 con 0x7f54400b1d50 2024-04-25T11:19:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.384+0000 7f5436a00640 1 -- 172.21.15.7:0/2979448020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5428030970 con 0x7f54400b1d50 2024-04-25T11:19:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.384+0000 7f5436a00640 1 -- 172.21.15.7:0/2979448020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428037040 con 0x7f54400b1d50 2024-04-25T11:19:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.384+0000 7f543fe00640 1 -- 172.21.15.7:0/2979448020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54400c6630 con 0x7f54400b1d50 2024-04-25T11:19:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.384+0000 7f543fe00640 1 -- 172.21.15.7:0/2979448020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54401f41c0 con 0x7f54400b1d50 2024-04-25T11:19:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.385+0000 7f543fe00640 1 -- 172.21.15.7:0/2979448020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5440050740 con 0x7f54400b1d50 2024-04-25T11:19:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.386+0000 7f5436a00640 1 -- 172.21.15.7:0/2979448020 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5428044020 con 0x7f54400b1d50 2024-04-25T11:19:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.387+0000 7f5436a00640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f541c074500 0x7f541c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.388+0000 7f5436a00640 1 -- 172.21.15.7:0/2979448020 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f54280b5d90 con 0x7f54400b1d50 2024-04-25T11:19:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.388+0000 7f543ea00640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f541c074500 0x7f541c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.388+0000 7f543ea00640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f541c074500 0x7f541c076a10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f543000b840 tx=0x7f5430008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:53.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.392+0000 7f5436a00640 1 -- 172.21.15.7:0/2979448020 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f542804b030 con 0x7f54400b1d50 2024-04-25T11:19:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.585+0000 7f543fe00640 1 -- 172.21.15.7:0/2979448020 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54400c94e0 con 0x7f541c074500 2024-04-25T11:19:53.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.602+0000 7f5436a00640 1 -- 172.21.15.7:0/2979448020 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252451 (secure 0 0 0) 0x7f54400c94e0 con 0x7f541c074500 2024-04-25T11:19:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:53.605 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:19:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.608+0000 7f543fe00640 1 -- 172.21.15.7:0/2979448020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f541c074500 msgr2=0x7f541c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.608+0000 7f543fe00640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f541c074500 0x7f541c076a10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f543000b840 tx=0x7f5430008040 comp rx=0 tx=0).stop 2024-04-25T11:19:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.608+0000 7f543fe00640 1 -- 172.21.15.7:0/2979448020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54400b1d50 msgr2=0x7f5440070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.608+0000 7f543fe00640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54400b1d50 0x7f5440070f10 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f5428000c00 tx=0x7f54280046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.609+0000 7f543fe00640 1 -- 172.21.15.7:0/2979448020 shutdown_connections 2024-04-25T11:19:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.609+0000 7f543fe00640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f541c074500 0x7f541c076a10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.609+0000 7f543fe00640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54400b73d0 0x7f5440071450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.609+0000 7f543fe00640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54400b1d50 0x7f5440070f10 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.609+0000 7f543fe00640 1 --2- 172.21.15.7:0/2979448020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54400b1610 0x7f54400c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.609+0000 7f543fe00640 1 -- 172.21.15.7:0/2979448020 >> 172.21.15.7:0/2979448020 conn(0x7f544009b540 msgr2=0x7f54400707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.609+0000 7f543fe00640 1 -- 172.21.15.7:0/2979448020 shutdown_connections 2024-04-25T11:19:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.609+0000 7f543fe00640 1 -- 172.21.15.7:0/2979448020 wait complete. 2024-04-25T11:19:53.634 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-04-25T11:19:52.011489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":270376,"kb_used_data":12580,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754704344,"statfs":{"total":773094113280,"available":772817248256,"internally_reserved":0,"allocated":12881920,"data_stored":6545960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.795679"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731540+0000","last_change":"2024-04-25T11:19:48.291100+0000","last_active":"2024-04-25T11:19:49.731540+0000","last_peered":"2024-04-25T11:19:49.731540+0000","last_clean":"2024-04-25T11:19:49.731540+0000","last_became_active":"2024-04-25T11:19:48.290780+0000","last_became_peered":"2024-04-25T11:19:48.290780+0000","last_unstale":"2024-04-25T11:19:49.731540+0000","last_undegraded":"2024-04-25T11:19:49.731540+0000","last_fullsized":"2024-04-25T11:19:49.731540+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:01.201766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.304485+0000","last_change":"2024-04-25T11:19:48.296231+0000","last_active":"2024-04-25T11:19:48.304485+0000","last_peered":"2024-04-25T11:19:48.304485+0000","last_clean":"2024-04-25T11:19:48.304485+0000","last_became_active":"2024-04-25T11:19:48.295829+0000","last_became_peered":"2024-04-25T11:19:48.295829+0000","last_unstale":"2024-04-25T11:19:48.304485+0000","last_undegraded":"2024-04-25T11:19:48.304485+0000","last_fullsized":"2024-04-25T11:19:48.304485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:12.674057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.306342+0000","last_change":"2024-04-25T11:19:48.294215+0000","last_active":"2024-04-25T11:19:48.306342+0000","last_peered":"2024-04-25T11:19:48.306342+0000","last_clean":"2024-04-25T11:19:48.306342+0000","last_became_active":"2024-04-25T11:19:48.293889+0000","last_became_peered":"2024-04-25T11:19:48.293889+0000","last_unstale":"2024-04-25T11:19:48.306342+0000","last_undegraded":"2024-04-25T11:19:48.306342+0000","last_fullsized":"2024-04-25T11:19:48.306342+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:43.185766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298809+0000","last_change":"2024-04-25T11:19:48.289536+0000","last_active":"2024-04-25T11:19:49.298809+0000","last_peered":"2024-04-25T11:19:49.298809+0000","last_clean":"2024-04-25T11:19:49.298809+0000","last_became_active":"2024-04-25T11:19:48.288968+0000","last_became_peered":"2024-04-25T11:19:48.288968+0000","last_unstale":"2024-04-25T11:19:49.298809+0000","last_undegraded":"2024-04-25T11:19:49.298809+0000","last_fullsized":"2024-04-25T11:19:49.298809+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:43.845296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.730732+0000","last_change":"2024-04-25T11:19:48.292007+0000","last_active":"2024-04-25T11:19:49.730732+0000","last_peered":"2024-04-25T11:19:49.730732+0000","last_clean":"2024-04-25T11:19:49.730732+0000","last_became_active":"2024-04-25T11:19:48.291700+0000","last_became_peered":"2024-04-25T11:19:48.291700+0000","last_unstale":"2024-04-25T11:19:49.730732+0000","last_undegraded":"2024-04-25T11:19:49.730732+0000","last_fullsized":"2024-04-25T11:19:49.730732+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:05.318358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303024+0000","last_change":"2024-04-25T11:19:48.309091+0000","last_active":"2024-04-25T11:19:49.303024+0000","last_peered":"2024-04-25T11:19:49.303024+0000","last_clean":"2024-04-25T11:19:49.303024+0000","last_became_active":"2024-04-25T11:19:48.308619+0000","last_became_peered":"2024-04-25T11:19:48.308619+0000","last_unstale":"2024-04-25T11:19:49.303024+0000","last_undegraded":"2024-04-25T11:19:49.303024+0000","last_fullsized":"2024-04-25T11:19:49.303024+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:52.931994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.302862+0000","last_change":"2024-04-25T11:19:48.312988+0000","last_active":"2024-04-25T11:19:49.302862+0000","last_peered":"2024-04-25T11:19:49.302862+0000","last_clean":"2024-04-25T11:19:49.302862+0000","last_became_active":"2024-04-25T11:19:48.309097+0000","last_became_peered":"2024-04-25T11:19:48.309097+0000","last_unstale":"2024-04-25T11:19:49.302862+0000","last_undegraded":"2024-04-25T11:19:49.302862+0000","last_fullsized":"2024-04-25T11:19:49.302862+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:30.300749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.300155+0000","last_change":"2024-04-25T11:19:48.291497+0000","last_active":"2024-04-25T11:19:48.300155+0000","last_peered":"2024-04-25T11:19:48.300155+0000","last_clean":"2024-04-25T11:19:48.300155+0000","last_became_active":"2024-04-25T11:19:48.291160+0000","last_became_peered":"2024-04-25T11:19:48.291160+0000","last_unstale":"2024-04-25T11:19:48.300155+0000","last_undegraded":"2024-04-25T11:19:48.300155+0000","last_fullsized":"2024-04-25T11:19:48.300155+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:23.899534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299608+0000","last_change":"2024-04-25T11:19:48.301014+0000","last_active":"2024-04-25T11:19:49.299608+0000","last_peered":"2024-04-25T11:19:49.299608+0000","last_clean":"2024-04-25T11:19:49.299608+0000","last_became_active":"2024-04-25T11:19:48.300711+0000","last_became_peered":"2024-04-25T11:19:48.300711+0000","last_unstale":"2024-04-25T11:19:49.299608+0000","last_undegraded":"2024-04-25T11:19:49.299608+0000","last_fullsized":"2024-04-25T11:19:49.299608+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:30.513081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.312862+0000","last_change":"2024-04-25T11:19:48.305992+0000","last_active":"2024-04-25T11:19:48.312862+0000","last_peered":"2024-04-25T11:19:48.312862+0000","last_clean":"2024-04-25T11:19:48.312862+0000","last_became_active":"2024-04-25T11:19:48.305637+0000","last_became_peered":"2024-04-25T11:19:48.305637+0000","last_unstale":"2024-04-25T11:19:48.312862+0000","last_undegraded":"2024-04-25T11:19:48.312862+0000","last_fullsized":"2024-04-25T11:19:48.312862+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:31.824966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731632+0000","last_change":"2024-04-25T11:19:48.298644+0000","last_active":"2024-04-25T11:19:49.731632+0000","last_peered":"2024-04-25T11:19:49.731632+0000","last_clean":"2024-04-25T11:19:49.731632+0000","last_became_active":"2024-04-25T11:19:48.295822+0000","last_became_peered":"2024-04-25T11:19:48.295822+0000","last_unstale":"2024-04-25T11:19:49.731632+0000","last_undegraded":"2024-04-25T11:19:49.731632+0000","last_fullsized":"2024-04-25T11:19:49.731632+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:24.499963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303965+0000","last_change":"2024-04-25T11:19:48.309922+0000","last_active":"2024-04-25T11:19:49.303965+0000","last_peered":"2024-04-25T11:19:49.303965+0000","last_clean":"2024-04-25T11:19:49.303965+0000","last_became_active":"2024-04-25T11:19:48.309086+0000","last_became_peered":"2024-04-25T11:19:48.309086+0000","last_unstale":"2024-04-25T11:19:49.303965+0000","last_undegraded":"2024-04-25T11:19:49.303965+0000","last_fullsized":"2024-04-25T11:19:49.303965+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:04.823507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.293356+0000","last_change":"2024-04-25T11:19:48.290035+0000","last_active":"2024-04-25T11:19:48.293356+0000","last_peered":"2024-04-25T11:19:48.293356+0000","last_clean":"2024-04-25T11:19:48.293356+0000","last_became_active":"2024-04-25T11:19:48.289648+0000","last_became_peered":"2024-04-25T11:19:48.289648+0000","last_unstale":"2024-04-25T11:19:48.293356+0000","last_undegraded":"2024-04-25T11:19:48.293356+0000","last_fullsized":"2024-04-25T11:19:48.293356+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:30.400046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.298223+0000","last_change":"2024-04-25T11:19:48.293151+0000","last_active":"2024-04-25T11:19:48.298223+0000","last_peered":"2024-04-25T11:19:48.298223+0000","last_clean":"2024-04-25T11:19:48.298223+0000","last_became_active":"2024-04-25T11:19:48.292149+0000","last_became_peered":"2024-04-25T11:19:48.292149+0000","last_unstale":"2024-04-25T11:19:48.298223+0000","last_undegraded":"2024-04-25T11:19:48.298223+0000","last_fullsized":"2024-04-25T11:19:48.298223+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:40.119303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731003+0000","last_change":"2024-04-25T11:19:48.301827+0000","last_active":"2024-04-25T11:19:49.731003+0000","last_peered":"2024-04-25T11:19:49.731003+0000","last_clean":"2024-04-25T11:19:49.731003+0000","last_became_active":"2024-04-25T11:19:48.301517+0000","last_became_peered":"2024-04-25T11:19:48.301517+0000","last_unstale":"2024-04-25T11:19:49.731003+0000","last_undegraded":"2024-04-25T11:19:49.731003+0000","last_fullsized":"2024-04-25T11:19:49.731003+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:23.688909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.302725+0000","last_change":"2024-04-25T11:19:48.304232+0000","last_active":"2024-04-25T11:19:49.302725+0000","last_peered":"2024-04-25T11:19:49.302725+0000","last_clean":"2024-04-25T11:19:49.302725+0000","last_became_active":"2024-04-25T11:19:48.303358+0000","last_became_peered":"2024-04-25T11:19:48.303358+0000","last_unstale":"2024-04-25T11:19:49.302725+0000","last_undegraded":"2024-04-25T11:19:49.302725+0000","last_fullsized":"2024-04-25T11:19:49.302725+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:56.548714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.310358+0000","last_change":"2024-04-25T11:19:48.298929+0000","last_active":"2024-04-25T11:19:48.310358+0000","last_peered":"2024-04-25T11:19:48.310358+0000","last_clean":"2024-04-25T11:19:48.310358+0000","last_became_active":"2024-04-25T11:19:48.298575+0000","last_became_peered":"2024-04-25T11:19:48.298575+0000","last_unstale":"2024-04-25T11:19:48.310358+0000","last_undegraded":"2024-04-25T11:19:48.310358+0000","last_fullsized":"2024-04-25T11:19:48.310358+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:59.403388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.293439+0000","last_change":"2024-04-25T11:19:48.280394+0000","last_active":"2024-04-25T11:19:48.293439+0000","last_peered":"2024-04-25T11:19:48.293439+0000","last_clean":"2024-04-25T11:19:48.293439+0000","last_became_active":"2024-04-25T11:19:48.279885+0000","last_became_peered":"2024-04-25T11:19:48.279885+0000","last_unstale":"2024-04-25T11:19:48.293439+0000","last_undegraded":"2024-04-25T11:19:48.293439+0000","last_fullsized":"2024-04-25T11:19:48.293439+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:27.175206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296317+0000","last_change":"2024-04-25T11:19:48.314502+0000","last_active":"2024-04-25T11:19:49.296317+0000","last_peered":"2024-04-25T11:19:49.296317+0000","last_clean":"2024-04-25T11:19:49.296317+0000","last_became_active":"2024-04-25T11:19:48.311402+0000","last_became_peered":"2024-04-25T11:19:48.311402+0000","last_unstale":"2024-04-25T11:19:49.296317+0000","last_undegraded":"2024-04-25T11:19:49.296317+0000","last_fullsized":"2024-04-25T11:19:49.296317+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:56.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.317069+0000","last_change":"2024-04-25T11:19:48.311358+0000","last_active":"2024-04-25T11:19:48.317069+0000","last_peered":"2024-04-25T11:19:48.317069+0000","last_clean":"2024-04-25T11:19:48.317069+0000","last_became_active":"2024-04-25T11:19:48.311003+0000","last_became_peered":"2024-04-25T11:19:48.311003+0000","last_unstale":"2024-04-25T11:19:48.317069+0000","last_undegraded":"2024-04-25T11:19:48.317069+0000","last_fullsized":"2024-04-25T11:19:48.317069+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:19.376102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299210+0000","last_change":"2024-04-25T11:19:48.297975+0000","last_active":"2024-04-25T11:19:49.299210+0000","last_peered":"2024-04-25T11:19:49.299210+0000","last_clean":"2024-04-25T11:19:49.299210+0000","last_became_active":"2024-04-25T11:19:48.297339+0000","last_became_peered":"2024-04-25T11:19:48.297339+0000","last_unstale":"2024-04-25T11:19:49.299210+0000","last_undegraded":"2024-04-25T11:19:49.299210+0000","last_fullsized":"2024-04-25T11:19:49.299210+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:16.477832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.730941+0000","last_change":"2024-04-25T11:19:48.299350+0000","last_active":"2024-04-25T11:19:49.730941+0000","last_peered":"2024-04-25T11:19:49.730941+0000","last_clean":"2024-04-25T11:19:49.730941+0000","last_became_active":"2024-04-25T11:19:48.299004+0000","last_became_peered":"2024-04-25T11:19:48.299004+0000","last_unstale":"2024-04-25T11:19:49.730941+0000","last_undegraded":"2024-04-25T11:19:49.730941+0000","last_fullsized":"2024-04-25T11:19:49.730941+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:59.096555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299044+0000","last_change":"2024-04-25T11:19:48.304575+0000","last_active":"2024-04-25T11:19:49.299044+0000","last_peered":"2024-04-25T11:19:49.299044+0000","last_clean":"2024-04-25T11:19:49.299044+0000","last_became_active":"2024-04-25T11:19:48.302053+0000","last_became_peered":"2024-04-25T11:19:48.302053+0000","last_unstale":"2024-04-25T11:19:49.299044+0000","last_undegraded":"2024-04-25T11:19:49.299044+0000","last_fullsized":"2024-04-25T11:19:49.299044+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:12.404652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.294837+0000","last_change":"2024-04-25T11:19:48.296299+0000","last_active":"2024-04-25T11:19:49.294837+0000","last_peered":"2024-04-25T11:19:49.294837+0000","last_clean":"2024-04-25T11:19:49.294837+0000","last_became_active":"2024-04-25T11:19:48.295681+0000","last_became_peered":"2024-04-25T11:19:48.295681+0000","last_unstale":"2024-04-25T11:19:49.294837+0000","last_undegraded":"2024-04-25T11:19:49.294837+0000","last_fullsized":"2024-04-25T11:19:49.294837+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:56.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298061+0000","last_change":"2024-04-25T11:19:48.297924+0000","last_active":"2024-04-25T11:19:49.298061+0000","last_peered":"2024-04-25T11:19:49.298061+0000","last_clean":"2024-04-25T11:19:49.298061+0000","last_became_active":"2024-04-25T11:19:48.293163+0000","last_became_peered":"2024-04-25T11:19:48.293163+0000","last_unstale":"2024-04-25T11:19:49.298061+0000","last_undegraded":"2024-04-25T11:19:49.298061+0000","last_fullsized":"2024-04-25T11:19:49.298061+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:44.806508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.298886+0000","last_change":"2024-04-25T11:19:48.294110+0000","last_active":"2024-04-25T11:19:48.298886+0000","last_peered":"2024-04-25T11:19:48.298886+0000","last_clean":"2024-04-25T11:19:48.298886+0000","last_became_active":"2024-04-25T11:19:48.292284+0000","last_became_peered":"2024-04-25T11:19:48.292284+0000","last_unstale":"2024-04-25T11:19:48.298886+0000","last_undegraded":"2024-04-25T11:19:48.298886+0000","last_fullsized":"2024-04-25T11:19:48.298886+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:00.207963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296771+0000","last_change":"2024-04-25T11:19:48.314086+0000","last_active":"2024-04-25T11:19:49.296771+0000","last_peered":"2024-04-25T11:19:49.296771+0000","last_clean":"2024-04-25T11:19:49.296771+0000","last_became_active":"2024-04-25T11:19:48.310126+0000","last_became_peered":"2024-04-25T11:19:48.310126+0000","last_unstale":"2024-04-25T11:19:49.296771+0000","last_undegraded":"2024-04-25T11:19:49.296771+0000","last_fullsized":"2024-04-25T11:19:49.296771+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:01.201766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304086+0000","last_change":"2024-04-25T11:19:48.302539+0000","last_active":"2024-04-25T11:19:49.304086+0000","last_peered":"2024-04-25T11:19:49.304086+0000","last_clean":"2024-04-25T11:19:49.304086+0000","last_became_active":"2024-04-25T11:19:48.302117+0000","last_became_peered":"2024-04-25T11:19:48.302117+0000","last_unstale":"2024-04-25T11:19:49.304086+0000","last_undegraded":"2024-04-25T11:19:49.304086+0000","last_fullsized":"2024-04-25T11:19:49.304086+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:47.151066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.301059+0000","last_change":"2024-04-25T11:19:48.299544+0000","last_active":"2024-04-25T11:19:48.301059+0000","last_peered":"2024-04-25T11:19:48.301059+0000","last_clean":"2024-04-25T11:19:48.301059+0000","last_became_active":"2024-04-25T11:19:48.299180+0000","last_became_peered":"2024-04-25T11:19:48.299180+0000","last_unstale":"2024-04-25T11:19:48.301059+0000","last_undegraded":"2024-04-25T11:19:48.301059+0000","last_fullsized":"2024-04-25T11:19:48.301059+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:36.484233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.298176+0000","last_change":"2024-04-25T11:19:48.289715+0000","last_active":"2024-04-25T11:19:48.298176+0000","last_peered":"2024-04-25T11:19:48.298176+0000","last_clean":"2024-04-25T11:19:48.298176+0000","last_became_active":"2024-04-25T11:19:48.289301+0000","last_became_peered":"2024-04-25T11:19:48.289301+0000","last_unstale":"2024-04-25T11:19:48.298176+0000","last_undegraded":"2024-04-25T11:19:48.298176+0000","last_fullsized":"2024-04-25T11:19:48.298176+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:44.806508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.295371+0000","last_change":"2024-04-25T11:19:48.290338+0000","last_active":"2024-04-25T11:19:49.295371+0000","last_peered":"2024-04-25T11:19:49.295371+0000","last_clean":"2024-04-25T11:19:49.295371+0000","last_became_active":"2024-04-25T11:19:48.289989+0000","last_became_peered":"2024-04-25T11:19:48.289989+0000","last_unstale":"2024-04-25T11:19:49.295371+0000","last_undegraded":"2024-04-25T11:19:49.295371+0000","last_fullsized":"2024-04-25T11:19:49.295371+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:40.030371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.295050+0000","last_change":"2024-04-25T11:19:48.295982+0000","last_active":"2024-04-25T11:19:49.295050+0000","last_peered":"2024-04-25T11:19:49.295050+0000","last_clean":"2024-04-25T11:19:49.295050+0000","last_became_active":"2024-04-25T11:19:48.295276+0000","last_became_peered":"2024-04-25T11:19:48.295276+0000","last_unstale":"2024-04-25T11:19:49.295050+0000","last_undegraded":"2024-04-25T11:19:49.295050+0000","last_fullsized":"2024-04-25T11:19:49.295050+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:01.201766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.313445+0000","last_change":"2024-04-25T11:19:48.305570+0000","last_active":"2024-04-25T11:19:48.313445+0000","last_peered":"2024-04-25T11:19:48.313445+0000","last_clean":"2024-04-25T11:19:48.313445+0000","last_became_active":"2024-04-25T11:19:48.305175+0000","last_became_peered":"2024-04-25T11:19:48.305175+0000","last_unstale":"2024-04-25T11:19:48.313445+0000","last_undegraded":"2024-04-25T11:19:48.313445+0000","last_fullsized":"2024-04-25T11:19:48.313445+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:15.693123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.295966+0000","last_change":"2024-04-25T11:19:47.261599+0000","last_active":"2024-04-25T11:19:49.295966+0000","last_peered":"2024-04-25T11:19:49.295966+0000","last_clean":"2024-04-25T11:19:49.295966+0000","last_became_active":"2024-04-25T11:19:47.261269+0000","last_became_peered":"2024-04-25T11:19:47.261269+0000","last_unstale":"2024-04-25T11:19:49.295966+0000","last_undegraded":"2024-04-25T11:19:49.295966+0000","last_fullsized":"2024-04-25T11:19:49.295966+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:42.042741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304131+0000","last_change":"2024-04-25T11:19:48.304592+0000","last_active":"2024-04-25T11:19:49.304131+0000","last_peered":"2024-04-25T11:19:49.304131+0000","last_clean":"2024-04-25T11:19:49.304131+0000","last_became_active":"2024-04-25T11:19:48.304241+0000","last_became_peered":"2024-04-25T11:19:48.304241+0000","last_unstale":"2024-04-25T11:19:49.304131+0000","last_undegraded":"2024-04-25T11:19:49.304131+0000","last_fullsized":"2024-04-25T11:19:49.304131+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:50.353341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.316570+0000","last_change":"2024-04-25T11:19:47.259114+0000","last_active":"2024-04-25T11:19:48.316570+0000","last_peered":"2024-04-25T11:19:48.316570+0000","last_clean":"2024-04-25T11:19:48.316570+0000","last_became_active":"2024-04-25T11:19:47.258626+0000","last_became_peered":"2024-04-25T11:19:47.258626+0000","last_unstale":"2024-04-25T11:19:48.316570+0000","last_undegraded":"2024-04-25T11:19:48.316570+0000","last_fullsized":"2024-04-25T11:19:48.316570+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:45.930118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297566+0000","last_change":"2024-04-25T11:19:48.307393+0000","last_active":"2024-04-25T11:19:49.297566+0000","last_peered":"2024-04-25T11:19:49.297566+0000","last_clean":"2024-04-25T11:19:49.297566+0000","last_became_active":"2024-04-25T11:19:48.307010+0000","last_became_peered":"2024-04-25T11:19:48.307010+0000","last_unstale":"2024-04-25T11:19:49.297566+0000","last_undegraded":"2024-04-25T11:19:49.297566+0000","last_fullsized":"2024-04-25T11:19:49.297566+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:47.792531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299674+0000","last_change":"2024-04-25T11:19:47.255903+0000","last_active":"2024-04-25T11:19:49.299674+0000","last_peered":"2024-04-25T11:19:49.299674+0000","last_clean":"2024-04-25T11:19:49.299674+0000","last_became_active":"2024-04-25T11:19:47.255258+0000","last_became_peered":"2024-04-25T11:19:47.255258+0000","last_unstale":"2024-04-25T11:19:49.299674+0000","last_undegraded":"2024-04-25T11:19:49.299674+0000","last_fullsized":"2024-04-25T11:19:49.299674+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:46.778083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299752+0000","last_change":"2024-04-25T11:19:48.301009+0000","last_active":"2024-04-25T11:19:49.299752+0000","last_peered":"2024-04-25T11:19:49.299752+0000","last_clean":"2024-04-25T11:19:49.299752+0000","last_became_active":"2024-04-25T11:19:48.300259+0000","last_became_peered":"2024-04-25T11:19:48.300259+0000","last_unstale":"2024-04-25T11:19:49.299752+0000","last_undegraded":"2024-04-25T11:19:49.299752+0000","last_fullsized":"2024-04-25T11:19:49.299752+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:22.874651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.321625+0000","last_change":"2024-04-25T11:19:47.248687+0000","last_active":"2024-04-25T11:19:48.321625+0000","last_peered":"2024-04-25T11:19:48.321625+0000","last_clean":"2024-04-25T11:19:48.321625+0000","last_became_active":"2024-04-25T11:19:47.248303+0000","last_became_peered":"2024-04-25T11:19:47.248303+0000","last_unstale":"2024-04-25T11:19:48.321625+0000","last_undegraded":"2024-04-25T11:19:48.321625+0000","last_fullsized":"2024-04-25T11:19:48.321625+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:37.075222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.298746+0000","last_change":"2024-04-25T11:19:48.292006+0000","last_active":"2024-04-25T11:19:48.298746+0000","last_peered":"2024-04-25T11:19:48.298746+0000","last_clean":"2024-04-25T11:19:48.298746+0000","last_became_active":"2024-04-25T11:19:48.291652+0000","last_became_peered":"2024-04-25T11:19:48.291652+0000","last_unstale":"2024-04-25T11:19:48.298746+0000","last_undegraded":"2024-04-25T11:19:48.298746+0000","last_fullsized":"2024-04-25T11:19:48.298746+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:01.249219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731159+0000","last_change":"2024-04-25T11:19:47.264898+0000","last_active":"2024-04-25T11:19:49.731159+0000","last_peered":"2024-04-25T11:19:49.731159+0000","last_clean":"2024-04-25T11:19:49.731159+0000","last_became_active":"2024-04-25T11:19:47.264573+0000","last_became_peered":"2024-04-25T11:19:47.264573+0000","last_unstale":"2024-04-25T11:19:49.731159+0000","last_undegraded":"2024-04-25T11:19:49.731159+0000","last_fullsized":"2024-04-25T11:19:49.731159+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:45.930118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731053+0000","last_change":"2024-04-25T11:19:48.296475+0000","last_active":"2024-04-25T11:19:49.731053+0000","last_peered":"2024-04-25T11:19:49.731053+0000","last_clean":"2024-04-25T11:19:49.731053+0000","last_became_active":"2024-04-25T11:19:48.296158+0000","last_became_peered":"2024-04-25T11:19:48.296158+0000","last_unstale":"2024-04-25T11:19:49.731053+0000","last_undegraded":"2024-04-25T11:19:49.731053+0000","last_fullsized":"2024-04-25T11:19:49.731053+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:56.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303271+0000","last_change":"2024-04-25T11:19:47.271179+0000","last_active":"2024-04-25T11:19:49.303271+0000","last_peered":"2024-04-25T11:19:49.303271+0000","last_clean":"2024-04-25T11:19:49.303271+0000","last_became_active":"2024-04-25T11:19:47.270853+0000","last_became_peered":"2024-04-25T11:19:47.270853+0000","last_unstale":"2024-04-25T11:19:49.303271+0000","last_undegraded":"2024-04-25T11:19:49.303271+0000","last_fullsized":"2024-04-25T11:19:49.303271+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:50.630486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.293711+0000","last_change":"2024-04-25T11:19:48.281107+0000","last_active":"2024-04-25T11:19:48.293711+0000","last_peered":"2024-04-25T11:19:48.293711+0000","last_clean":"2024-04-25T11:19:48.293711+0000","last_became_active":"2024-04-25T11:19:48.280714+0000","last_became_peered":"2024-04-25T11:19:48.280714+0000","last_unstale":"2024-04-25T11:19:48.293711+0000","last_undegraded":"2024-04-25T11:19:48.293711+0000","last_fullsized":"2024-04-25T11:19:48.293711+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:38.216023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296040+0000","last_change":"2024-04-25T11:19:47.250262+0000","last_active":"2024-04-25T11:19:49.296040+0000","last_peered":"2024-04-25T11:19:49.296040+0000","last_clean":"2024-04-25T11:19:49.296040+0000","last_became_active":"2024-04-25T11:19:47.249571+0000","last_became_peered":"2024-04-25T11:19:47.249571+0000","last_unstale":"2024-04-25T11:19:49.296040+0000","last_undegraded":"2024-04-25T11:19:49.296040+0000","last_fullsized":"2024-04-25T11:19:49.296040+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:45.930118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.298696+0000","last_change":"2024-04-25T11:19:48.291799+0000","last_active":"2024-04-25T11:19:48.298696+0000","last_peered":"2024-04-25T11:19:48.298696+0000","last_clean":"2024-04-25T11:19:48.298696+0000","last_became_active":"2024-04-25T11:19:48.291449+0000","last_became_peered":"2024-04-25T11:19:48.291449+0000","last_unstale":"2024-04-25T11:19:48.298696+0000","last_undegraded":"2024-04-25T11:19:48.298696+0000","last_fullsized":"2024-04-25T11:19:48.298696+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:08.012932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303156+0000","last_change":"2024-04-25T11:19:47.263182+0000","last_active":"2024-04-25T11:19:49.303156+0000","last_peered":"2024-04-25T11:19:49.303156+0000","last_clean":"2024-04-25T11:19:49.303156+0000","last_became_active":"2024-04-25T11:19:47.262824+0000","last_became_peered":"2024-04-25T11:19:47.262824+0000","last_unstale":"2024-04-25T11:19:49.303156+0000","last_undegraded":"2024-04-25T11:19:49.303156+0000","last_fullsized":"2024-04-25T11:19:49.303156+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:49.671190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.295663+0000","last_change":"2024-04-25T11:19:48.306114+0000","last_active":"2024-04-25T11:19:49.295663+0000","last_peered":"2024-04-25T11:19:49.295663+0000","last_clean":"2024-04-25T11:19:49.295663+0000","last_became_active":"2024-04-25T11:19:48.305426+0000","last_became_peered":"2024-04-25T11:19:48.305426+0000","last_unstale":"2024-04-25T11:19:49.295663+0000","last_undegraded":"2024-04-25T11:19:49.295663+0000","last_fullsized":"2024-04-25T11:19:49.295663+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:23.688909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296358+0000","last_change":"2024-04-25T11:19:47.265821+0000","last_active":"2024-04-25T11:19:49.296358+0000","last_peered":"2024-04-25T11:19:49.296358+0000","last_clean":"2024-04-25T11:19:49.296358+0000","last_became_active":"2024-04-25T11:19:47.265396+0000","last_became_peered":"2024-04-25T11:19:47.265396+0000","last_unstale":"2024-04-25T11:19:49.296358+0000","last_undegraded":"2024-04-25T11:19:49.296358+0000","last_fullsized":"2024-04-25T11:19:49.296358+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:12.508447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297929+0000","last_change":"2024-04-25T11:19:48.281087+0000","last_active":"2024-04-25T11:19:49.297929+0000","last_peered":"2024-04-25T11:19:49.297929+0000","last_clean":"2024-04-25T11:19:49.297929+0000","last_became_active":"2024-04-25T11:19:48.280578+0000","last_became_peered":"2024-04-25T11:19:48.280578+0000","last_unstale":"2024-04-25T11:19:49.297929+0000","last_undegraded":"2024-04-25T11:19:49.297929+0000","last_fullsized":"2024-04-25T11:19:49.297929+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:38.216023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297685+0000","last_change":"2024-04-25T11:19:47.258465+0000","last_active":"2024-04-25T11:19:49.297685+0000","last_peered":"2024-04-25T11:19:49.297685+0000","last_clean":"2024-04-25T11:19:49.297685+0000","last_became_active":"2024-04-25T11:19:47.258061+0000","last_became_peered":"2024-04-25T11:19:47.258061+0000","last_unstale":"2024-04-25T11:19:49.297685+0000","last_undegraded":"2024-04-25T11:19:49.297685+0000","last_fullsized":"2024-04-25T11:19:49.297685+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:29.385598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.306593+0000","last_change":"2024-04-25T11:19:48.301628+0000","last_active":"2024-04-25T11:19:48.306593+0000","last_peered":"2024-04-25T11:19:48.306593+0000","last_clean":"2024-04-25T11:19:48.306593+0000","last_became_active":"2024-04-25T11:19:48.301296+0000","last_became_peered":"2024-04-25T11:19:48.301296+0000","last_unstale":"2024-04-25T11:19:48.306593+0000","last_undegraded":"2024-04-25T11:19:48.306593+0000","last_fullsized":"2024-04-25T11:19:48.306593+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:46.417609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298410+0000","last_change":"2024-04-25T11:19:47.263397+0000","last_active":"2024-04-25T11:19:49.298410+0000","last_peered":"2024-04-25T11:19:49.298410+0000","last_clean":"2024-04-25T11:19:49.298410+0000","last_became_active":"2024-04-25T11:19:47.262983+0000","last_became_peered":"2024-04-25T11:19:47.262983+0000","last_unstale":"2024-04-25T11:19:49.298410+0000","last_undegraded":"2024-04-25T11:19:49.298410+0000","last_fullsized":"2024-04-25T11:19:49.298410+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:16.259744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298271+0000","last_change":"2024-04-25T11:19:48.294009+0000","last_active":"2024-04-25T11:19:49.298271+0000","last_peered":"2024-04-25T11:19:49.298271+0000","last_clean":"2024-04-25T11:19:49.298271+0000","last_became_active":"2024-04-25T11:19:48.291682+0000","last_became_peered":"2024-04-25T11:19:48.291682+0000","last_unstale":"2024-04-25T11:19:49.298271+0000","last_undegraded":"2024-04-25T11:19:49.298271+0000","last_fullsized":"2024-04-25T11:19:49.298271+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:48.173226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"117'6","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.323294+0000","last_change":"2024-04-25T11:19:47.263550+0000","last_active":"2024-04-25T11:19:48.323294+0000","last_peered":"2024-04-25T11:19:48.323294+0000","last_clean":"2024-04-25T11:19:48.323294+0000","last_became_active":"2024-04-25T11:19:47.263170+0000","last_became_peered":"2024-04-25T11:19:47.263170+0000","last_unstale":"2024-04-25T11:19:48.323294+0000","last_undegraded":"2024-04-25T11:19:48.323294+0000","last_fullsized":"2024-04-25T11:19:48.323294+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:00.187318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.295363+0000","last_change":"2024-04-25T11:19:48.303426+0000","last_active":"2024-04-25T11:19:49.295363+0000","last_peered":"2024-04-25T11:19:49.295363+0000","last_clean":"2024-04-25T11:19:49.295363+0000","last_became_active":"2024-04-25T11:19:48.302608+0000","last_became_peered":"2024-04-25T11:19:48.302608+0000","last_unstale":"2024-04-25T11:19:49.295363+0000","last_undegraded":"2024-04-25T11:19:49.295363+0000","last_fullsized":"2024-04-25T11:19:49.295363+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:59.096555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.321759+0000","last_change":"2024-04-25T11:19:47.260450+0000","last_active":"2024-04-25T11:19:48.321759+0000","last_peered":"2024-04-25T11:19:48.321759+0000","last_clean":"2024-04-25T11:19:48.321759+0000","last_became_active":"2024-04-25T11:19:47.260145+0000","last_became_peered":"2024-04-25T11:19:47.260145+0000","last_unstale":"2024-04-25T11:19:48.321759+0000","last_undegraded":"2024-04-25T11:19:48.321759+0000","last_fullsized":"2024-04-25T11:19:48.321759+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:55.751855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.309790+0000","last_change":"2024-04-25T11:19:48.295943+0000","last_active":"2024-04-25T11:19:48.309790+0000","last_peered":"2024-04-25T11:19:48.309790+0000","last_clean":"2024-04-25T11:19:48.309790+0000","last_became_active":"2024-04-25T11:19:48.295568+0000","last_became_peered":"2024-04-25T11:19:48.295568+0000","last_unstale":"2024-04-25T11:19:48.309790+0000","last_undegraded":"2024-04-25T11:19:48.309790+0000","last_fullsized":"2024-04-25T11:19:48.309790+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:07.267851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.276715+0000","last_change":"2024-04-25T11:19:47.239820+0000","last_active":"2024-04-25T11:19:48.276715+0000","last_peered":"2024-04-25T11:19:48.276715+0000","last_clean":"2024-04-25T11:19:48.276715+0000","last_became_active":"2024-04-25T11:19:47.239358+0000","last_became_peered":"2024-04-25T11:19:47.239358+0000","last_unstale":"2024-04-25T11:19:48.276715+0000","last_undegraded":"2024-04-25T11:19:48.276715+0000","last_fullsized":"2024-04-25T11:19:48.276715+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:46.778083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.297035+0000","last_change":"2024-04-25T11:19:48.290975+0000","last_active":"2024-04-25T11:19:48.297035+0000","last_peered":"2024-04-25T11:19:48.297035+0000","last_clean":"2024-04-25T11:19:48.297035+0000","last_became_active":"2024-04-25T11:19:48.290375+0000","last_became_peered":"2024-04-25T11:19:48.290375+0000","last_unstale":"2024-04-25T11:19:48.297035+0000","last_undegraded":"2024-04-25T11:19:48.297035+0000","last_fullsized":"2024-04-25T11:19:48.297035+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:50.496441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.272153+0000","last_change":"2024-04-25T11:19:47.259544+0000","last_active":"2024-04-25T11:19:48.272153+0000","last_peered":"2024-04-25T11:19:48.272153+0000","last_clean":"2024-04-25T11:19:48.272153+0000","last_became_active":"2024-04-25T11:19:47.256173+0000","last_became_peered":"2024-04-25T11:19:47.256173+0000","last_unstale":"2024-04-25T11:19:48.272153+0000","last_undegraded":"2024-04-25T11:19:48.272153+0000","last_fullsized":"2024-04-25T11:19:48.272153+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:39.015923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297588+0000","last_change":"2024-04-25T11:19:48.307395+0000","last_active":"2024-04-25T11:19:49.297588+0000","last_peered":"2024-04-25T11:19:49.297588+0000","last_clean":"2024-04-25T11:19:49.297588+0000","last_became_active":"2024-04-25T11:19:48.306776+0000","last_became_peered":"2024-04-25T11:19:48.306776+0000","last_unstale":"2024-04-25T11:19:49.297588+0000","last_undegraded":"2024-04-25T11:19:49.297588+0000","last_fullsized":"2024-04-25T11:19:49.297588+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:27.175206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"117'1","reported_seq":13,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.319677+0000","last_change":"2024-04-25T11:19:47.267880+0000","last_active":"2024-04-25T11:19:48.319677+0000","last_peered":"2024-04-25T11:19:48.319677+0000","last_clean":"2024-04-25T11:19:48.319677+0000","last_became_active":"2024-04-25T11:19:47.266288+0000","last_became_peered":"2024-04-25T11:19:47.266288+0000","last_unstale":"2024-04-25T11:19:48.319677+0000","last_undegraded":"2024-04-25T11:19:48.319677+0000","last_fullsized":"2024-04-25T11:19:48.319677+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:16.259744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731630+0000","last_change":"2024-04-25T11:19:48.300670+0000","last_active":"2024-04-25T11:19:49.731630+0000","last_peered":"2024-04-25T11:19:49.731630+0000","last_clean":"2024-04-25T11:19:49.731630+0000","last_became_active":"2024-04-25T11:19:48.300304+0000","last_became_peered":"2024-04-25T11:19:48.300304+0000","last_unstale":"2024-04-25T11:19:49.731630+0000","last_undegraded":"2024-04-25T11:19:49.731630+0000","last_fullsized":"2024-04-25T11:19:49.731630+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:40.030371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"117'2","reported_seq":14,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.319503+0000","last_change":"2024-04-25T11:19:47.266268+0000","last_active":"2024-04-25T11:19:48.319503+0000","last_peered":"2024-04-25T11:19:48.319503+0000","last_clean":"2024-04-25T11:19:48.319503+0000","last_became_active":"2024-04-25T11:19:47.265332+0000","last_became_peered":"2024-04-25T11:19:47.265332+0000","last_unstale":"2024-04-25T11:19:48.319503+0000","last_undegraded":"2024-04-25T11:19:48.319503+0000","last_fullsized":"2024-04-25T11:19:48.319503+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:34.128714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304644+0000","last_change":"2024-04-25T11:19:48.312255+0000","last_active":"2024-04-25T11:19:49.304644+0000","last_peered":"2024-04-25T11:19:49.304644+0000","last_clean":"2024-04-25T11:19:49.304644+0000","last_became_active":"2024-04-25T11:19:48.311090+0000","last_became_peered":"2024-04-25T11:19:48.311090+0000","last_unstale":"2024-04-25T11:19:49.304644+0000","last_undegraded":"2024-04-25T11:19:49.304644+0000","last_fullsized":"2024-04-25T11:19:49.304644+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:17.845098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299573+0000","last_change":"2024-04-25T11:19:47.256564+0000","last_active":"2024-04-25T11:19:49.299573+0000","last_peered":"2024-04-25T11:19:49.299573+0000","last_clean":"2024-04-25T11:19:49.299573+0000","last_became_active":"2024-04-25T11:19:47.255603+0000","last_became_peered":"2024-04-25T11:19:47.255603+0000","last_unstale":"2024-04-25T11:19:49.299573+0000","last_undegraded":"2024-04-25T11:19:49.299573+0000","last_fullsized":"2024-04-25T11:19:49.299573+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:26.160757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298312+0000","last_change":"2024-04-25T11:19:48.276843+0000","last_active":"2024-04-25T11:19:49.298312+0000","last_peered":"2024-04-25T11:19:49.298312+0000","last_clean":"2024-04-25T11:19:49.298312+0000","last_became_active":"2024-04-25T11:19:48.276450+0000","last_became_peered":"2024-04-25T11:19:48.276450+0000","last_unstale":"2024-04-25T11:19:49.298312+0000","last_undegraded":"2024-04-25T11:19:49.298312+0000","last_fullsized":"2024-04-25T11:19:49.298312+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:23.899534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299242+0000","last_change":"2024-04-25T11:19:47.269430+0000","last_active":"2024-04-25T11:19:49.299242+0000","last_peered":"2024-04-25T11:19:49.299242+0000","last_clean":"2024-04-25T11:19:49.299242+0000","last_became_active":"2024-04-25T11:19:47.269090+0000","last_became_peered":"2024-04-25T11:19:47.269090+0000","last_unstale":"2024-04-25T11:19:49.299242+0000","last_undegraded":"2024-04-25T11:19:49.299242+0000","last_fullsized":"2024-04-25T11:19:49.299242+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:39.015923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.302626+0000","last_change":"2024-04-25T11:19:48.303957+0000","last_active":"2024-04-25T11:19:49.302626+0000","last_peered":"2024-04-25T11:19:49.302626+0000","last_clean":"2024-04-25T11:19:49.302626+0000","last_became_active":"2024-04-25T11:19:48.302860+0000","last_became_peered":"2024-04-25T11:19:48.302860+0000","last_unstale":"2024-04-25T11:19:49.302626+0000","last_undegraded":"2024-04-25T11:19:49.302626+0000","last_fullsized":"2024-04-25T11:19:49.302626+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:43.323898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303434+0000","last_change":"2024-04-25T11:19:47.267653+0000","last_active":"2024-04-25T11:19:49.303434+0000","last_peered":"2024-04-25T11:19:49.303434+0000","last_clean":"2024-04-25T11:19:49.303434+0000","last_became_active":"2024-04-25T11:19:47.267317+0000","last_became_peered":"2024-04-25T11:19:47.267317+0000","last_unstale":"2024-04-25T11:19:49.303434+0000","last_undegraded":"2024-04-25T11:19:49.303434+0000","last_fullsized":"2024-04-25T11:19:49.303434+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:47.171241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.299201+0000","last_change":"2024-04-25T11:19:48.290999+0000","last_active":"2024-04-25T11:19:48.299201+0000","last_peered":"2024-04-25T11:19:48.299201+0000","last_clean":"2024-04-25T11:19:48.299201+0000","last_became_active":"2024-04-25T11:19:48.289317+0000","last_became_peered":"2024-04-25T11:19:48.289317+0000","last_unstale":"2024-04-25T11:19:48.299201+0000","last_undegraded":"2024-04-25T11:19:48.299201+0000","last_fullsized":"2024-04-25T11:19:48.299201+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:48.173226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298693+0000","last_change":"2024-04-25T11:19:47.257548+0000","last_active":"2024-04-25T11:19:49.298693+0000","last_peered":"2024-04-25T11:19:49.298693+0000","last_clean":"2024-04-25T11:19:49.298693+0000","last_became_active":"2024-04-25T11:19:47.257245+0000","last_became_peered":"2024-04-25T11:19:47.257245+0000","last_unstale":"2024-04-25T11:19:49.298693+0000","last_undegraded":"2024-04-25T11:19:49.298693+0000","last_fullsized":"2024-04-25T11:19:49.298693+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:00.187318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.306196+0000","last_change":"2024-04-25T11:19:48.294491+0000","last_active":"2024-04-25T11:19:48.306196+0000","last_peered":"2024-04-25T11:19:48.306196+0000","last_clean":"2024-04-25T11:19:48.306196+0000","last_became_active":"2024-04-25T11:19:48.292416+0000","last_became_peered":"2024-04-25T11:19:48.292416+0000","last_unstale":"2024-04-25T11:19:48.306196+0000","last_undegraded":"2024-04-25T11:19:48.306196+0000","last_fullsized":"2024-04-25T11:19:48.306196+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:11.076716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.272465+0000","last_change":"2024-04-25T11:19:47.263206+0000","last_active":"2024-04-25T11:19:48.272465+0000","last_peered":"2024-04-25T11:19:48.272465+0000","last_clean":"2024-04-25T11:19:48.272465+0000","last_became_active":"2024-04-25T11:19:47.262829+0000","last_became_peered":"2024-04-25T11:19:47.262829+0000","last_unstale":"2024-04-25T11:19:48.272465+0000","last_undegraded":"2024-04-25T11:19:48.272465+0000","last_fullsized":"2024-04-25T11:19:48.272465+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:23.485514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296792+0000","last_change":"2024-04-25T11:19:48.291641+0000","last_active":"2024-04-25T11:19:49.296792+0000","last_peered":"2024-04-25T11:19:49.296792+0000","last_clean":"2024-04-25T11:19:49.296792+0000","last_became_active":"2024-04-25T11:19:48.288628+0000","last_became_peered":"2024-04-25T11:19:48.288628+0000","last_unstale":"2024-04-25T11:19:49.296792+0000","last_undegraded":"2024-04-25T11:19:49.296792+0000","last_fullsized":"2024-04-25T11:19:49.296792+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:24.499963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296722+0000","last_change":"2024-04-25T11:19:47.254409+0000","last_active":"2024-04-25T11:19:49.296722+0000","last_peered":"2024-04-25T11:19:49.296722+0000","last_clean":"2024-04-25T11:19:49.296722+0000","last_became_active":"2024-04-25T11:19:47.254033+0000","last_became_peered":"2024-04-25T11:19:47.254033+0000","last_unstale":"2024-04-25T11:19:49.296722+0000","last_undegraded":"2024-04-25T11:19:49.296722+0000","last_fullsized":"2024-04-25T11:19:49.296722+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:23.485514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304467+0000","last_change":"2024-04-25T11:19:48.302987+0000","last_active":"2024-04-25T11:19:49.304467+0000","last_peered":"2024-04-25T11:19:49.304467+0000","last_clean":"2024-04-25T11:19:49.304467+0000","last_became_active":"2024-04-25T11:19:48.302547+0000","last_became_peered":"2024-04-25T11:19:48.302547+0000","last_unstale":"2024-04-25T11:19:49.304467+0000","last_undegraded":"2024-04-25T11:19:49.304467+0000","last_fullsized":"2024-04-25T11:19:49.304467+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:51.228850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304723+0000","last_change":"2024-04-25T11:19:47.268843+0000","last_active":"2024-04-25T11:19:49.304723+0000","last_peered":"2024-04-25T11:19:49.304723+0000","last_clean":"2024-04-25T11:19:49.304723+0000","last_became_active":"2024-04-25T11:19:47.267167+0000","last_became_peered":"2024-04-25T11:19:47.267167+0000","last_unstale":"2024-04-25T11:19:49.304723+0000","last_undegraded":"2024-04-25T11:19:49.304723+0000","last_fullsized":"2024-04-25T11:19:49.304723+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:29.994468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.310214+0000","last_change":"2024-04-25T11:19:48.295452+0000","last_active":"2024-04-25T11:19:48.310214+0000","last_peered":"2024-04-25T11:19:48.310214+0000","last_clean":"2024-04-25T11:19:48.310214+0000","last_became_active":"2024-04-25T11:19:48.295030+0000","last_became_peered":"2024-04-25T11:19:48.295030+0000","last_unstale":"2024-04-25T11:19:48.310214+0000","last_undegraded":"2024-04-25T11:19:48.310214+0000","last_fullsized":"2024-04-25T11:19:48.310214+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.057189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"118'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.301137+0000","last_change":"2024-04-25T11:19:47.267333+0000","last_active":"2024-04-25T11:19:49.301137+0000","last_peered":"2024-04-25T11:19:49.301137+0000","last_clean":"2024-04-25T11:19:49.301137+0000","last_became_active":"2024-04-25T11:19:47.267013+0000","last_became_peered":"2024-04-25T11:19:47.267013+0000","last_unstale":"2024-04-25T11:19:49.301137+0000","last_undegraded":"2024-04-25T11:19:49.301137+0000","last_fullsized":"2024-04-25T11:19:49.301137+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:00.234771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299365+0000","last_change":"2024-04-25T11:19:48.290489+0000","last_active":"2024-04-25T11:19:49.299365+0000","last_peered":"2024-04-25T11:19:49.299365+0000","last_clean":"2024-04-25T11:19:49.299365+0000","last_became_active":"2024-04-25T11:19:48.290168+0000","last_became_peered":"2024-04-25T11:19:48.290168+0000","last_unstale":"2024-04-25T11:19:49.299365+0000","last_undegraded":"2024-04-25T11:19:49.299365+0000","last_fullsized":"2024-04-25T11:19:49.299365+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:27.031412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.296631+0000","last_change":"2024-04-25T11:19:47.265807+0000","last_active":"2024-04-25T11:19:49.296631+0000","last_peered":"2024-04-25T11:19:49.296631+0000","last_clean":"2024-04-25T11:19:49.296631+0000","last_became_active":"2024-04-25T11:19:47.265396+0000","last_became_peered":"2024-04-25T11:19:47.265396+0000","last_unstale":"2024-04-25T11:19:49.296631+0000","last_undegraded":"2024-04-25T11:19:49.296631+0000","last_fullsized":"2024-04-25T11:19:49.296631+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:06.253402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.303131+0000","last_change":"2024-04-25T11:19:48.301298+0000","last_active":"2024-04-25T11:19:48.303131+0000","last_peered":"2024-04-25T11:19:48.303131+0000","last_clean":"2024-04-25T11:19:48.303131+0000","last_became_active":"2024-04-25T11:19:48.300888+0000","last_became_peered":"2024-04-25T11:19:48.300888+0000","last_unstale":"2024-04-25T11:19:48.303131+0000","last_undegraded":"2024-04-25T11:19:48.303131+0000","last_fullsized":"2024-04-25T11:19:48.303131+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:34.374026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.304593+0000","last_change":"2024-04-25T11:19:47.265317+0000","last_active":"2024-04-25T11:19:49.304593+0000","last_peered":"2024-04-25T11:19:49.304593+0000","last_clean":"2024-04-25T11:19:49.304593+0000","last_became_active":"2024-04-25T11:19:47.264636+0000","last_became_peered":"2024-04-25T11:19:47.264636+0000","last_unstale":"2024-04-25T11:19:49.304593+0000","last_undegraded":"2024-04-25T11:19:49.304593+0000","last_fullsized":"2024-04-25T11:19:49.304593+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:06.680032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.303238+0000","last_change":"2024-04-25T11:19:48.303689+0000","last_active":"2024-04-25T11:19:49.303238+0000","last_peered":"2024-04-25T11:19:49.303238+0000","last_clean":"2024-04-25T11:19:49.303238+0000","last_became_active":"2024-04-25T11:19:48.302135+0000","last_became_peered":"2024-04-25T11:19:48.302135+0000","last_unstale":"2024-04-25T11:19:49.303238+0000","last_undegraded":"2024-04-25T11:19:49.303238+0000","last_fullsized":"2024-04-25T11:19:49.303238+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:44.398813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731182+0000","last_change":"2024-04-25T11:19:47.259012+0000","last_active":"2024-04-25T11:19:49.731182+0000","last_peered":"2024-04-25T11:19:49.731182+0000","last_clean":"2024-04-25T11:19:49.731182+0000","last_became_active":"2024-04-25T11:19:47.257947+0000","last_became_peered":"2024-04-25T11:19:47.257947+0000","last_unstale":"2024-04-25T11:19:49.731182+0000","last_undegraded":"2024-04-25T11:19:49.731182+0000","last_fullsized":"2024-04-25T11:19:49.731182+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:16.259744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.304877+0000","last_change":"2024-04-25T11:19:48.295181+0000","last_active":"2024-04-25T11:19:48.304877+0000","last_peered":"2024-04-25T11:19:48.304877+0000","last_clean":"2024-04-25T11:19:48.304877+0000","last_became_active":"2024-04-25T11:19:48.294861+0000","last_became_peered":"2024-04-25T11:19:48.294861+0000","last_unstale":"2024-04-25T11:19:48.304877+0000","last_undegraded":"2024-04-25T11:19:48.304877+0000","last_fullsized":"2024-04-25T11:19:48.304877+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:52.969575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.731025+0000","last_change":"2024-04-25T11:19:47.258297+0000","last_active":"2024-04-25T11:19:49.731025+0000","last_peered":"2024-04-25T11:19:49.731025+0000","last_clean":"2024-04-25T11:19:49.731025+0000","last_became_active":"2024-04-25T11:19:47.257964+0000","last_became_peered":"2024-04-25T11:19:47.257964+0000","last_unstale":"2024-04-25T11:19:49.731025+0000","last_undegraded":"2024-04-25T11:19:49.731025+0000","last_fullsized":"2024-04-25T11:19:49.731025+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:29.385598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.298570+0000","last_change":"2024-04-25T11:19:48.309559+0000","last_active":"2024-04-25T11:19:49.298570+0000","last_peered":"2024-04-25T11:19:49.298570+0000","last_clean":"2024-04-25T11:19:49.298570+0000","last_became_active":"2024-04-25T11:19:48.309212+0000","last_became_peered":"2024-04-25T11:19:48.309212+0000","last_unstale":"2024-04-25T11:19:49.298570+0000","last_undegraded":"2024-04-25T11:19:49.298570+0000","last_fullsized":"2024-04-25T11:19:49.298570+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:46.417609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.302923+0000","last_change":"2024-04-25T11:19:47.267575+0000","last_active":"2024-04-25T11:19:49.302923+0000","last_peered":"2024-04-25T11:19:49.302923+0000","last_clean":"2024-04-25T11:19:49.302923+0000","last_became_active":"2024-04-25T11:19:47.267161+0000","last_became_peered":"2024-04-25T11:19:47.267161+0000","last_unstale":"2024-04-25T11:19:49.302923+0000","last_undegraded":"2024-04-25T11:19:49.302923+0000","last_fullsized":"2024-04-25T11:19:49.302923+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:44.763800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.310275+0000","last_change":"2024-04-25T11:19:48.295474+0000","last_active":"2024-04-25T11:19:48.310275+0000","last_peered":"2024-04-25T11:19:48.310275+0000","last_clean":"2024-04-25T11:19:48.310275+0000","last_became_active":"2024-04-25T11:19:48.295107+0000","last_became_peered":"2024-04-25T11:19:48.295107+0000","last_unstale":"2024-04-25T11:19:48.310275+0000","last_undegraded":"2024-04-25T11:19:48.310275+0000","last_fullsized":"2024-04-25T11:19:48.310275+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:13.522895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297011+0000","last_change":"2024-04-25T11:19:47.260306+0000","last_active":"2024-04-25T11:19:49.297011+0000","last_peered":"2024-04-25T11:19:49.297011+0000","last_clean":"2024-04-25T11:19:49.297011+0000","last_became_active":"2024-04-25T11:19:47.259366+0000","last_became_peered":"2024-04-25T11:19:47.259366+0000","last_unstale":"2024-04-25T11:19:49.297011+0000","last_undegraded":"2024-04-25T11:19:49.297011+0000","last_fullsized":"2024-04-25T11:19:49.297011+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:55.751855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.299431+0000","last_change":"2024-04-25T11:19:48.297647+0000","last_active":"2024-04-25T11:19:49.299431+0000","last_peered":"2024-04-25T11:19:49.299431+0000","last_clean":"2024-04-25T11:19:49.299431+0000","last_became_active":"2024-04-25T11:19:48.296834+0000","last_became_peered":"2024-04-25T11:19:48.296834+0000","last_unstale":"2024-04-25T11:19:49.299431+0000","last_undegraded":"2024-04-25T11:19:49.299431+0000","last_fullsized":"2024-04-25T11:19:49.299431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:47.213950+0000","last_clean_scrub_stamp":"2024-04-25T11:19:47.213950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:33.203635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"117'8","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-04-25T11:19:48.322354+0000","last_change":"2024-04-25T11:19:47.262351+0000","last_active":"2024-04-25T11:19:48.322354+0000","last_peered":"2024-04-25T11:19:48.322354+0000","last_clean":"2024-04-25T11:19:48.322354+0000","last_became_active":"2024-04-25T11:19:47.261686+0000","last_became_peered":"2024-04-25T11:19:47.261686+0000","last_unstale":"2024-04-25T11:19:48.322354+0000","last_undegraded":"2024-04-25T11:19:48.322354+0000","last_fullsized":"2024-04-25T11:19:48.322354+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:19:46.199502+0000","last_clean_scrub_stamp":"2024-04-25T11:19:46.199502+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:22.674461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":304,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-04-25T11:19:49.297696+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:19:49.297696+0000","last_peered":"2024-04-25T11:19:49.297696+0000","last_clean":"2024-04-25T11:19:49.297696+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:19:49.297696+0000","last_undegraded":"2024-04-25T11:19:49.297696+0000","last_fullsized":"2024-04-25T11:19:49.297696+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2068480,"data_stored":1263704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":1462272,"data_stored":673142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1466368,"data_stored":672720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":1470464,"data_stored":674172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509558,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":667514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509559,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":665890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509558,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2056192,"data_stored":1262950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542263,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1445888,"data_stored":665868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:19:53.638 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:19:53.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:19:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.974+0000 7fa16e600640 1 Processor -- start 2024-04-25T11:19:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.974+0000 7fa16e600640 1 -- start start 2024-04-25T11:19:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.977+0000 7fa16e600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa1680ccd20 0x7fa1680cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.977+0000 7fa16e600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1680cf260 0x7fa1680afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.977+0000 7fa16e600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1680b1410 con 0x7fa1680cd670 2024-04-25T11:19:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.977+0000 7fa16e600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa1680d2590 con 0x7fa1680cf260 2024-04-25T11:19:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.977+0000 7fa16e600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa168160580 con 0x7fa1680ccd20 2024-04-25T11:19:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.978+0000 7fa16dc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1680cf260 0x7fa1680afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.978+0000 7fa16dc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1680cf260 0x7fa1680afbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42480/0 (socket says 172.21.15.7:42480) 2024-04-25T11:19:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.978+0000 7fa16dc00640 1 -- 172.21.15.7:0/4007280606 learned_addr learned my addr 172.21.15.7:0/4007280606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.978+0000 7fa16d200640 1 --2- 172.21.15.7:0/4007280606 >> v2:172.21.15.87:3300/0 conn(0x7fa1680ccd20 0x7fa1680cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.978+0000 7fa16dc00640 1 -- 172.21.15.7:0/4007280606 >> v2:172.21.15.87:3300/0 conn(0x7fa1680ccd20 msgr2=0x7fa1680cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.978+0000 7fa16dc00640 1 --2- 172.21.15.7:0/4007280606 >> v2:172.21.15.87:3300/0 conn(0x7fa1680ccd20 0x7fa1680cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.978+0000 7fa16dc00640 1 -- 172.21.15.7:0/4007280606 >> v1:172.21.15.87:6789/0 conn(0x7fa1680cd670 legacy=0x7fa1680af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.978+0000 7fa16dc00640 1 -- 172.21.15.7:0/4007280606 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1680d2830 con 0x7fa1680cf260 2024-04-25T11:19:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.979+0000 7fa16dc00640 1 --2- 172.21.15.7:0/4007280606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1680cf260 0x7fa1680afbd0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fa1680d37a0 tx=0x7fa16002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a458c04c6c112fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.980+0000 7fa167400640 1 -- 172.21.15.7:0/4007280606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa160036070 con 0x7fa1680cf260 2024-04-25T11:19:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.980+0000 7fa167400640 1 -- 172.21.15.7:0/4007280606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa160031600 con 0x7fa1680cf260 2024-04-25T11:19:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.980+0000 7fa167400640 1 -- 172.21.15.7:0/4007280606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1600319d0 con 0x7fa1680cf260 2024-04-25T11:19:53.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.980+0000 7fa16e600640 1 -- 172.21.15.7:0/4007280606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1680cf260 msgr2=0x7fa1680afbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.980+0000 7fa16e600640 1 --2- 172.21.15.7:0/4007280606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1680cf260 0x7fa1680afbd0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fa1680d37a0 tx=0x7fa16002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.981+0000 7fa16e600640 1 -- 172.21.15.7:0/4007280606 shutdown_connections 2024-04-25T11:19:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.981+0000 7fa16e600640 1 --2- 172.21.15.7:0/4007280606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1680cf260 0x7fa1680afbd0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.981+0000 7fa16e600640 1 --2- 172.21.15.7:0/4007280606 >> v2:172.21.15.87:3300/0 conn(0x7fa1680ccd20 0x7fa1680cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.981+0000 7fa16e600640 1 -- 172.21.15.7:0/4007280606 >> 172.21.15.7:0/4007280606 conn(0x7fa16809b540 msgr2=0x7fa16809b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.981+0000 7fa16e600640 1 -- 172.21.15.7:0/4007280606 shutdown_connections 2024-04-25T11:19:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.981+0000 7fa16e600640 1 -- 172.21.15.7:0/4007280606 wait complete. 2024-04-25T11:19:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.983+0000 7fa16e600640 1 Processor -- start 2024-04-25T11:19:53.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.983+0000 7fa16e600640 1 -- start start 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.985+0000 7fa16e600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1680ccd20 0x7fa1681569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.985+0000 7fa16e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1680cd670 0x7fa168070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.985+0000 7fa16e600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1680cf260 0x7fa1680707c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.985+0000 7fa16e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa1680d30d0 con 0x7fa1680cd670 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.985+0000 7fa16e600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa1680d2df0 con 0x7fa1680cf260 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.985+0000 7fa16e600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa1680cda80 con 0x7fa1680ccd20 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.985+0000 7fa167e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1680cd670 0x7fa168070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.985+0000 7fa167e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1680cd670 0x7fa168070280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42494/0 (socket says 172.21.15.7:42494) 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.985+0000 7fa167e00640 1 -- 172.21.15.7:0/3006727753 learned_addr learned my addr 172.21.15.7:0/3006727753 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.985+0000 7fa16d200640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1680ccd20 0x7fa1681569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.986+0000 7fa16dc00640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1680cf260 0x7fa1680707c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.986+0000 7fa167e00640 1 -- 172.21.15.7:0/3006727753 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1680ccd20 msgr2=0x7fa1681569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.986+0000 7fa167e00640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1680ccd20 0x7fa1681569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.986+0000 7fa167e00640 1 -- 172.21.15.7:0/3006727753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1680cf260 msgr2=0x7fa1680707c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.986+0000 7fa167e00640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1680cf260 0x7fa1680707c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.986+0000 7fa167e00640 1 -- 172.21.15.7:0/3006727753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1681573a0 con 0x7fa1680cd670 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.986+0000 7fa167e00640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1680cd670 0x7fa168070280 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fa15000b420 tx=0x7fa15000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:53.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.986+0000 7fa164c00640 1 -- 172.21.15.7:0/3006727753 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150011830 con 0x7fa1680cd670 2024-04-25T11:19:53.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.987+0000 7fa164c00640 1 -- 172.21.15.7:0/3006727753 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa150016070 con 0x7fa1680cd670 2024-04-25T11:19:53.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.987+0000 7fa164c00640 1 -- 172.21.15.7:0/3006727753 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150011c10 con 0x7fa1680cd670 2024-04-25T11:19:53.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.987+0000 7fa16e600640 1 -- 172.21.15.7:0/3006727753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa168157680 con 0x7fa1680cd670 2024-04-25T11:19:53.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.987+0000 7fa16e600640 1 -- 172.21.15.7:0/3006727753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1680b24b0 con 0x7fa1680cd670 2024-04-25T11:19:53.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.988+0000 7fa16e600640 1 -- 172.21.15.7:0/3006727753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa168157ba0 con 0x7fa1680cd670 2024-04-25T11:19:53.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.990+0000 7fa164c00640 1 -- 172.21.15.7:0/3006727753 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa150004020 con 0x7fa1680cd670 2024-04-25T11:19:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.991+0000 7fa164c00640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1440744b0 0x7fa1440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.991+0000 7fa164c00640 1 -- 172.21.15.7:0/3006727753 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fa150095840 con 0x7fa1680cd670 2024-04-25T11:19:53.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.991+0000 7fa16d200640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1440744b0 0x7fa1440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:53.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.992+0000 7fa16d200640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1440744b0 0x7fa1440769c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa158006fd0 tx=0x7fa158008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:53.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:53.995+0000 7fa164c00640 1 -- 172.21.15.7:0/3006727753 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa168157ba0 con 0x7fa1680cd670 2024-04-25T11:19:54.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.292+0000 7fa16e600640 1 -- 172.21.15.7:0/3006727753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa168050740 con 0x7fa1680cd670 2024-04-25T11:19:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.298+0000 7fa164c00640 1 -- 172.21.15.7:0/3006727753 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7fa150020020 con 0x7fa1680cd670 2024-04-25T11:19:54.296 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-25T11:19:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:19:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-04-25T11:19:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-04-25T11:19:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:19:54.297 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-04-25T11:19:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.302+0000 7fa16e600640 1 -- 172.21.15.7:0/3006727753 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1440744b0 msgr2=0x7fa1440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.302+0000 7fa16e600640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1440744b0 0x7fa1440769c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa158006fd0 tx=0x7fa158008040 comp rx=0 tx=0).stop 2024-04-25T11:19:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.302+0000 7fa16e600640 1 -- 172.21.15.7:0/3006727753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1680cd670 msgr2=0x7fa168070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.302+0000 7fa16e600640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1680cd670 0x7fa168070280 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fa15000b420 tx=0x7fa15000b8e0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.302+0000 7fa16e600640 1 -- 172.21.15.7:0/3006727753 shutdown_connections 2024-04-25T11:19:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.302+0000 7fa16e600640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa1440744b0 0x7fa1440769c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.302+0000 7fa16e600640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1680cf260 0x7fa1680707c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.302+0000 7fa16e600640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1680cd670 0x7fa168070280 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.302+0000 7fa16e600640 1 --2- 172.21.15.7:0/3006727753 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1680ccd20 0x7fa1681569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.302+0000 7fa16e600640 1 -- 172.21.15.7:0/3006727753 >> 172.21.15.7:0/3006727753 conn(0x7fa16809b540 msgr2=0x7fa16806e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.303+0000 7fa16e600640 1 -- 172.21.15.7:0/3006727753 shutdown_connections 2024-04-25T11:19:54.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.303+0000 7fa16e600640 1 -- 172.21.15.7:0/3006727753 wait complete. 2024-04-25T11:19:54.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:19:54.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.615+0000 7fa547600640 1 Processor -- start 2024-04-25T11:19:54.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.615+0000 7fa547600640 1 -- start start 2024-04-25T11:19:54.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.617+0000 7fa547600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa54015ef90 0x7fa54009f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.617+0000 7fa547600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5400af540 0x7fa5400cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.618+0000 7fa547600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa54015f4e0 con 0x7fa54015bc40 2024-04-25T11:19:54.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.618+0000 7fa547600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa540071190 con 0x7fa5400af540 2024-04-25T11:19:54.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.618+0000 7fa547600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa5400b0560 con 0x7fa54015ef90 2024-04-25T11:19:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.618+0000 7fa545600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5400af540 0x7fa5400cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.618+0000 7fa53fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa54015bc40 0x7fa5400aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58782/0 (socket says 172.21.15.7:58782) 2024-04-25T11:19:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.618+0000 7fa53fe00640 1 -- 172.21.15.7:0/282353921 learned_addr learned my addr 172.21.15.7:0/282353921 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.618+0000 7fa544c00640 1 --2- 172.21.15.7:0/282353921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa54015ef90 0x7fa54009f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.618+0000 7fa545600640 1 -- 172.21.15.7:0/282353921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa54015ef90 msgr2=0x7fa54009f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.618+0000 7fa545600640 1 --2- 172.21.15.7:0/282353921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa54015ef90 0x7fa54009f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.618+0000 7fa545600640 1 -- 172.21.15.7:0/282353921 >> v1:172.21.15.7:6789/0 conn(0x7fa54015bc40 legacy=0x7fa5400aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.619+0000 7fa545600640 1 -- 172.21.15.7:0/282353921 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5400b0840 con 0x7fa5400af540 2024-04-25T11:19:54.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.620+0000 7fa545600640 1 --2- 172.21.15.7:0/282353921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5400af540 0x7fa5400cf980 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fa538009d90 tx=0x7fa53802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb8b3464722ad08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:54.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.620+0000 7fa53f400640 1 -- 172.21.15.7:0/282353921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538031600 con 0x7fa5400af540 2024-04-25T11:19:54.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.620+0000 7fa53f400640 1 -- 172.21.15.7:0/282353921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5380317a0 con 0x7fa5400af540 2024-04-25T11:19:54.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.620+0000 7fa53f400640 1 -- 172.21.15.7:0/282353921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa538031b70 con 0x7fa5400af540 2024-04-25T11:19:54.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.621+0000 7fa547600640 1 -- 172.21.15.7:0/282353921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5400af540 msgr2=0x7fa5400cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.621+0000 7fa547600640 1 --2- 172.21.15.7:0/282353921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5400af540 0x7fa5400cf980 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fa538009d90 tx=0x7fa53802a5d0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.621+0000 7fa547600640 1 -- 172.21.15.7:0/282353921 shutdown_connections 2024-04-25T11:19:54.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.621+0000 7fa547600640 1 --2- 172.21.15.7:0/282353921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5400af540 0x7fa5400cf980 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.621+0000 7fa547600640 1 --2- 172.21.15.7:0/282353921 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa54015ef90 0x7fa54009f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.621+0000 7fa547600640 1 -- 172.21.15.7:0/282353921 >> 172.21.15.7:0/282353921 conn(0x7fa540074200 msgr2=0x7fa54006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:54.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.621+0000 7fa547600640 1 -- 172.21.15.7:0/282353921 shutdown_connections 2024-04-25T11:19:54.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.621+0000 7fa547600640 1 -- 172.21.15.7:0/282353921 wait complete. 2024-04-25T11:19:54.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.623+0000 7fa547600640 1 Processor -- start 2024-04-25T11:19:54.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.623+0000 7fa547600640 1 -- start start 2024-04-25T11:19:54.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.625+0000 7fa547600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54015bc40 0x7fa540148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.625+0000 7fa547600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54015ef90 0x7fa5400cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.625+0000 7fa547600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5400cd510 0x7fa5401594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.625+0000 7fa547600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa5400b1320 con 0x7fa54015bc40 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.625+0000 7fa547600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa5400b0e00 con 0x7fa54015ef90 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.625+0000 7fa547600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa5400afb80 con 0x7fa5400cd510 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.625+0000 7fa544c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54015bc40 0x7fa540148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.625+0000 7fa544c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54015bc40 0x7fa540148fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42506/0 (socket says 172.21.15.7:42506) 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.625+0000 7fa544c00640 1 -- 172.21.15.7:0/1347259884 learned_addr learned my addr 172.21.15.7:0/1347259884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.626+0000 7fa53fe00640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54015ef90 0x7fa5400cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.626+0000 7fa545600640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5400cd510 0x7fa5401594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.626+0000 7fa544c00640 1 -- 172.21.15.7:0/1347259884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5400cd510 msgr2=0x7fa5401594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.626+0000 7fa544c00640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5400cd510 0x7fa5401594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.626+0000 7fa544c00640 1 -- 172.21.15.7:0/1347259884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54015ef90 msgr2=0x7fa5400cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.626+0000 7fa544c00640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54015ef90 0x7fa5400cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.626+0000 7fa544c00640 1 -- 172.21.15.7:0/1347259884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5401498b0 con 0x7fa54015bc40 2024-04-25T11:19:54.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.626+0000 7fa545600640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5400cd510 0x7fa5401594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:54.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.626+0000 7fa53fe00640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54015ef90 0x7fa5400cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:54.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.626+0000 7fa544c00640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54015bc40 0x7fa540148fb0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa53400d620 tx=0x7fa53400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:54.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.626+0000 7fa53cc00640 1 -- 172.21.15.7:0/1347259884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53400b620 con 0x7fa54015bc40 2024-04-25T11:19:54.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.627+0000 7fa53cc00640 1 -- 172.21.15.7:0/1347259884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa534019070 con 0x7fa54015bc40 2024-04-25T11:19:54.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.627+0000 7fa53cc00640 1 -- 172.21.15.7:0/1347259884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53400ba00 con 0x7fa54015bc40 2024-04-25T11:19:54.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.627+0000 7fa547600640 1 -- 172.21.15.7:0/1347259884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5400cc750 con 0x7fa54015bc40 2024-04-25T11:19:54.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.627+0000 7fa547600640 1 -- 172.21.15.7:0/1347259884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5400ccd10 con 0x7fa54015bc40 2024-04-25T11:19:54.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.628+0000 7fa547600640 1 -- 172.21.15.7:0/1347259884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa54006a100 con 0x7fa54015bc40 2024-04-25T11:19:54.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.629+0000 7fa53cc00640 1 -- 172.21.15.7:0/1347259884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa53400bba0 con 0x7fa54015bc40 2024-04-25T11:19:54.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.630+0000 7fa53cc00640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5200744b0 0x7fa5200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:54.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.630+0000 7fa53cc00640 1 -- 172.21.15.7:0/1347259884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fa5340958a0 con 0x7fa54015bc40 2024-04-25T11:19:54.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.631+0000 7fa53fe00640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5200744b0 0x7fa5200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:54.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.631+0000 7fa53fe00640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5200744b0 0x7fa5200769c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa54015a200 tx=0x7fa52c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:54.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.638+0000 7fa53cc00640 1 -- 172.21.15.7:0/1347259884 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa53405fe00 con 0x7fa54015bc40 2024-04-25T11:19:54.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.885+0000 7fa547600640 1 -- 172.21.15.7:0/1347259884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5400cdc90 con 0x7fa54015bc40 2024-04-25T11:19:54.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.886+0000 7fa53cc00640 1 -- 172.21.15.7:0/1347259884 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5174 (secure 0 0 0) 0x7fa53405f700 con 0x7fa54015bc40 2024-04-25T11:19:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:19:54.884 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/326382713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":326382713},{"type":"v1","addr":"172.21.15.7:6837","nonce":326382713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2007810270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2007810270},{"type":"v1","addr":"172.21.15.87:6835","nonce":2007810270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1753656512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1753656512},{"type":"v1","addr":"172.21.15.7:6836","nonce":1753656512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:47.807534+0000","modified":"2024-04-25T11:19:50.899375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1924056839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1924056839},{"type":"v1","addr":"172.21.15.87:6834","nonce":1924056839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8007,"qdb_cluster":[8007]},"id":8}]} 2024-04-25T11:19:54.884 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2024-04-25T11:19:54.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.889+0000 7fa547600640 1 -- 172.21.15.7:0/1347259884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5200744b0 msgr2=0x7fa5200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.889+0000 7fa547600640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5200744b0 0x7fa5200769c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa54015a200 tx=0x7fa52c009040 comp rx=0 tx=0).stop 2024-04-25T11:19:54.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.889+0000 7fa547600640 1 -- 172.21.15.7:0/1347259884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54015bc40 msgr2=0x7fa540148fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:54.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.889+0000 7fa547600640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54015bc40 0x7fa540148fb0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa53400d620 tx=0x7fa53400dae0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.889+0000 7fa547600640 1 -- 172.21.15.7:0/1347259884 shutdown_connections 2024-04-25T11:19:54.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.889+0000 7fa547600640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa5200744b0 0x7fa5200769c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.889+0000 7fa547600640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5400cd510 0x7fa5401594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.889+0000 7fa547600640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa54015ef90 0x7fa5400cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.889+0000 7fa547600640 1 --2- 172.21.15.7:0/1347259884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa54015bc40 0x7fa540148fb0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:54.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.889+0000 7fa547600640 1 -- 172.21.15.7:0/1347259884 >> 172.21.15.7:0/1347259884 conn(0x7fa540074200 msgr2=0x7fa5400d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:54.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.890+0000 7fa547600640 1 -- 172.21.15.7:0/1347259884 shutdown_connections 2024-04-25T11:19:54.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:54.890+0000 7fa547600640 1 -- 172.21.15.7:0/1347259884 wait complete. 2024-04-25T11:19:55.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:19:47.807534+0000', 'modified': '2024-04-25T11:19:50.899375+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8007}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8007': {'gid': 8007, 'name': 'd', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6834/1924056839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6832', 'nonce': 1924056839}, {'type': 'v1', 'addr': '172.21.15.87:6834', 'nonce': 1924056839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8007, 'qdb_cluster': [8007]} 2024-04-25T11:19:55.319 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:19:55.319 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T11:19:55.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.609+0000 7fce6aa00640 1 Processor -- start 2024-04-25T11:19:55.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.609+0000 7fce6aa00640 1 -- start start 2024-04-25T11:19:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.611+0000 7fce6aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fce640d3260 0x7fce6406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.611+0000 7fce6aa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6406e790 0x7fce640b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.611+0000 7fce6aa00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce640b8e10 con 0x7fce640b4070 2024-04-25T11:19:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.612+0000 7fce6aa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fce6409cc20 con 0x7fce640d3260 2024-04-25T11:19:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.612+0000 7fce6aa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce6409cf00 con 0x7fce6406e790 2024-04-25T11:19:55.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.612+0000 7fce6a000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fce640b4070 0x7fce640d1d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58788/0 (socket says 172.21.15.7:58788) 2024-04-25T11:19:55.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.612+0000 7fce6a000640 1 -- 172.21.15.7:0/79347777 learned_addr learned my addr 172.21.15.7:0/79347777 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.612+0000 7fce69600640 1 --2- 172.21.15.7:0/79347777 >> v2:172.21.15.87:3300/0 conn(0x7fce640d3260 0x7fce6406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.612+0000 7fce68c00640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6406e790 0x7fce640b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.612+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3270477236 0 0) 0x7fce640b8e10 con 0x7fce640b4070 2024-04-25T11:19:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce4c003520 con 0x7fce640b4070 2024-04-25T11:19:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2551283720 0 0) 0x7fce4c003520 con 0x7fce640b4070 2024-04-25T11:19:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6406e790 msgr2=0x7fce640b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6406e790 0x7fce640b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 >> v2:172.21.15.87:3300/0 conn(0x7fce640d3260 msgr2=0x7fce6406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 --2- 172.21.15.7:0/79347777 >> v2:172.21.15.87:3300/0 conn(0x7fce640d3260 0x7fce6406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce64074950 con 0x7fce640b4070 2024-04-25T11:19:55.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fce5c001670 con 0x7fce640b4070 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 >> v1:172.21.15.7:6789/0 conn(0x7fce640b4070 legacy=0x7fce640d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce4c0039b0 0x7fce4c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6406e790 0x7fce4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce4c004d40 0x7fce4c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce6409cc20 con 0x7fce4c004d40 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce6409cf00 con 0x7fce4c0039b0 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce4c00ba10 con 0x7fce6406e790 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.613+0000 7fce6a000640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce4c004d40 0x7fce4c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.614+0000 7fce69600640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce4c0039b0 0x7fce4c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.614+0000 7fce68c00640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6406e790 0x7fce4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.614+0000 7fce6a000640 1 -- 172.21.15.7:0/79347777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6406e790 msgr2=0x7fce4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.614+0000 7fce6a000640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6406e790 0x7fce4c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.614+0000 7fce6a000640 1 -- 172.21.15.7:0/79347777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce4c0039b0 msgr2=0x7fce4c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.614+0000 7fce6a000640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce4c0039b0 0x7fce4c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.614+0000 7fce6a000640 1 -- 172.21.15.7:0/79347777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4c003520 con 0x7fce4c004d40 2024-04-25T11:19:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.615+0000 7fce6a000640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce4c004d40 0x7fce4c005150 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fce640763f0 tx=0x7fce4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9398fe2e744da7e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.615+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c007d50 con 0x7fce4c004d40 2024-04-25T11:19:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.615+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce5c0058d0 con 0x7fce4c004d40 2024-04-25T11:19:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.615+0000 7fce63e00640 1 -- 172.21.15.7:0/79347777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c005d60 con 0x7fce4c004d40 2024-04-25T11:19:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.616+0000 7fce6aa00640 1 -- 172.21.15.7:0/79347777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce4c004d40 msgr2=0x7fce4c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.616+0000 7fce6aa00640 1 --2- 172.21.15.7:0/79347777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce4c004d40 0x7fce4c005150 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fce640763f0 tx=0x7fce4c005830 comp rx=0 tx=0).stop 2024-04-25T11:19:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.616+0000 7fce6a000640 1 -- 172.21.15.7:0/79347777 reap_dead start 2024-04-25T11:19:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.616+0000 7fce6aa00640 1 -- 172.21.15.7:0/79347777 shutdown_connections 2024-04-25T11:19:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.616+0000 7fce6aa00640 1 -- 172.21.15.7:0/79347777 >> 172.21.15.7:0/79347777 conn(0x7fce640d0240 msgr2=0x7fce640d0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.616+0000 7fce6aa00640 1 -- 172.21.15.7:0/79347777 shutdown_connections 2024-04-25T11:19:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.616+0000 7fce6aa00640 1 -- 172.21.15.7:0/79347777 wait complete. 2024-04-25T11:19:55.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.619+0000 7fce6aa00640 1 Processor -- start 2024-04-25T11:19:55.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.619+0000 7fce6aa00640 1 -- start start 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce6aa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce641dafd0 0x7fce641d8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce6aa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce641d9570 0x7fce640c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce6aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce641d9980 0x7fce640c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce6aa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce640d1090 con 0x7fce641d9980 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce6aa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce640b9080 con 0x7fce641d9570 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce6aa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce6406da60 con 0x7fce641dafd0 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce6a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce641d9980 0x7fce640c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce6a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce641d9980 0x7fce640c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42530/0 (socket says 172.21.15.7:42530) 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce6a000640 1 -- 172.21.15.7:0/1728932102 learned_addr learned my addr 172.21.15.7:0/1728932102 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce68c00640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce641d9570 0x7fce640c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce69600640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce641dafd0 0x7fce641d8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.621+0000 7fce6a000640 1 -- 172.21.15.7:0/1728932102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce641dafd0 msgr2=0x7fce641d8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce6a000640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce641dafd0 0x7fce641d8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce6a000640 1 -- 172.21.15.7:0/1728932102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce641d9570 msgr2=0x7fce640c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce6a000640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce641d9570 0x7fce640c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce6a000640 1 -- 172.21.15.7:0/1728932102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce641d9240 con 0x7fce641d9980 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce68c00640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce641d9570 0x7fce640c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce69600640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce641dafd0 0x7fce641d8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce6a000640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce641d9980 0x7fce640c92c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c2d0 tx=0x7fce5c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce61600640 1 -- 172.21.15.7:0/1728932102 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c038040 con 0x7fce641d9980 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce61600640 1 -- 172.21.15.7:0/1728932102 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce5c041070 con 0x7fce641d9980 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce6aa00640 1 -- 172.21.15.7:0/1728932102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce641d9f30 con 0x7fce641d9980 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce61600640 1 -- 172.21.15.7:0/1728932102 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c007d50 con 0x7fce641d9980 2024-04-25T11:19:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.622+0000 7fce6aa00640 1 -- 172.21.15.7:0/1728932102 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce641da4f0 con 0x7fce641d9980 2024-04-25T11:19:55.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.625+0000 7fce61600640 1 -- 172.21.15.7:0/1728932102 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce5c007270 con 0x7fce641d9980 2024-04-25T11:19:55.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.626+0000 7fce61600640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744b0 0x7fce440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.626+0000 7fce61600640 1 -- 172.21.15.7:0/1728932102 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fce5c0b4fa0 con 0x7fce641d9980 2024-04-25T11:19:55.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.626+0000 7fce69600640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744b0 0x7fce440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.627+0000 7fce73035740 1 Processor -- start 2024-04-25T11:19:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.627+0000 7fce73035740 1 -- start start 2024-04-25T11:19:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.627+0000 7fce69600640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744b0 0x7fce440769c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fce6406a050 tx=0x7fce5800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.629+0000 7fce73035740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca62a09270 0x55ca62dd6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.629+0000 7fce73035740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62df0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.629+0000 7fce73035740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca629e8590 0x55ca62df1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.629+0000 7fce73035740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ca62d0f600 con 0x55ca629e03b0 2024-04-25T11:19:55.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.629+0000 7fce73035740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ca62cac100 con 0x55ca629e8590 2024-04-25T11:19:55.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.629+0000 7fce73035740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55ca62d0b5c0 con 0x55ca62a09270 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.629+0000 7fce68c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62df0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.629+0000 7fce68c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62df0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42532/0 (socket says 172.21.15.7:42532) 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.629+0000 7fce68c00640 1 -- 172.21.15.7:0/4098173045 learned_addr learned my addr 172.21.15.7:0/4098173045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.629+0000 7fce6a000640 1 --2- 172.21.15.7:0/4098173045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca629e8590 0x55ca62df1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.629+0000 7fce69600640 1 --2- 172.21.15.7:0/4098173045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca62a09270 0x55ca62dd6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.630+0000 7fce68c00640 1 -- 172.21.15.7:0/4098173045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca62a09270 msgr2=0x55ca62dd6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.630+0000 7fce68c00640 1 --2- 172.21.15.7:0/4098173045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca62a09270 0x55ca62dd6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.630+0000 7fce68c00640 1 -- 172.21.15.7:0/4098173045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca629e8590 msgr2=0x55ca62df1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.630+0000 7fce68c00640 1 --2- 172.21.15.7:0/4098173045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca629e8590 0x55ca62df1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.630+0000 7fce68c00640 1 -- 172.21.15.7:0/4098173045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca62a212d0 con 0x55ca629e03b0 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.630+0000 7fce69600640 1 --2- 172.21.15.7:0/4098173045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca62a09270 0x55ca62dd6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.630+0000 7fce68c00640 1 --2- 172.21.15.7:0/4098173045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62df0f40 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fce50009b00 tx=0x7fce50009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91a483fb49511af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.630+0000 7fce55c00640 1 -- 172.21.15.7:0/4098173045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50019070 con 0x55ca629e03b0 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.630+0000 7fce55c00640 1 -- 172.21.15.7:0/4098173045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce50011c20 con 0x55ca629e03b0 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.630+0000 7fce55c00640 1 -- 172.21.15.7:0/4098173045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50015460 con 0x55ca629e03b0 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.631+0000 7fce73035740 1 -- 172.21.15.7:0/4098173045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 msgr2=0x55ca62df0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.631+0000 7fce73035740 1 --2- 172.21.15.7:0/4098173045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62df0f40 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fce50009b00 tx=0x7fce50009190 comp rx=0 tx=0).stop 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.631+0000 7fce73035740 1 -- 172.21.15.7:0/4098173045 shutdown_connections 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.631+0000 7fce73035740 1 --2- 172.21.15.7:0/4098173045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca629e8590 0x55ca62df1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.631+0000 7fce73035740 1 --2- 172.21.15.7:0/4098173045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62df0f40 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.631+0000 7fce73035740 1 --2- 172.21.15.7:0/4098173045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca62a09270 0x55ca62dd6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.631+0000 7fce73035740 1 -- 172.21.15.7:0/4098173045 >> 172.21.15.7:0/4098173045 conn(0x55ca629fd3c0 msgr2=0x55ca62dab870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.631+0000 7fce73035740 1 -- 172.21.15.7:0/4098173045 shutdown_connections 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.631+0000 7fce73035740 1 -- 172.21.15.7:0/4098173045 wait complete. 2024-04-25T11:19:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.632+0000 7fce73035740 1 -- start start 2024-04-25T11:19:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.633+0000 7fce73035740 1 Processor -- start 2024-04-25T11:19:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.634+0000 7fce73035740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca629e03b0 0x55ca62dd0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.634+0000 7fce73035740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e8590 0x55ca62deb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.634+0000 7fce73035740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62a09270 0x55ca62df0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.634+0000 7fce73035740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ca62c96f70 con 0x55ca629e8590 2024-04-25T11:19:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.634+0000 7fce73035740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ca62debe40 con 0x55ca62a09270 2024-04-25T11:19:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.634+0000 7fce73035740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55ca62df03b0 con 0x55ca629e03b0 2024-04-25T11:19:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.634+0000 7fce73035740 10 client.0 _ll_register_callbacks cb 0x55ca62d05480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:19:55.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.635+0000 7fce68c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e8590 0x55ca62deb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.635+0000 7fce68c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e8590 0x55ca62deb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42540/0 (socket says 172.21.15.7:42540) 2024-04-25T11:19:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.635+0000 7fce68c00640 1 -- 172.21.15.7:0/119838432 learned_addr learned my addr 172.21.15.7:0/119838432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:55.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.635+0000 7fce69600640 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca629e03b0 0x55ca62dd0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.635+0000 7fce68c00640 1 -- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca629e03b0 msgr2=0x55ca62dd0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.635+0000 7fce6a000640 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62a09270 0x55ca62df0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.635+0000 7fce68c00640 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca629e03b0 0x55ca62dd0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.635+0000 7fce68c00640 1 -- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62a09270 msgr2=0x55ca62df0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.635+0000 7fce68c00640 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62a09270 0x55ca62df0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.635+0000 7fce68c00640 1 -- 172.21.15.7:0/119838432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca62aeb6b0 con 0x55ca629e8590 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.636+0000 7fce68c00640 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e8590 0x55ca62deb4d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fce50018000 tx=0x7fce500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3a98017abb71c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.636+0000 7fce4a000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.636+0000 7fce4a000640 1 -- 172.21.15.7:0/119838432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5002c060 con 0x55ca629e8590 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.636+0000 7fce73035740 1 -- 172.21.15.7:0/119838432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce50011c20 con 0x55ca629e8590 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.636+0000 7fce4a000640 1 -- 172.21.15.7:0/119838432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fce50011a20 con 0x55ca629e8590 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.636+0000 7fce4a000640 1 -- 172.21.15.7:0/119838432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fce50032050 con 0x55ca629e8590 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.636+0000 7fce73035740 10 client.8142 fetch_fsmap learned FSMap version 76 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.636+0000 7fce73035740 1 -- 172.21.15.7:0/119838432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ca62df0d80 con 0x55ca629e8590 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.636+0000 7fce4a000640 1 -- 172.21.15.7:0/119838432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce50011c20 con 0x55ca629e8590 2024-04-25T11:19:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.637+0000 7fce4a000640 1 -- 172.21.15.7:0/119838432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7fce50035020 con 0x55ca629e8590 2024-04-25T11:19:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.637+0000 7fce73035740 10 client.8142 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T11:19:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.637+0000 7fce73035740 10 client.8142 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:19:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.637+0000 7fce73035740 20 client.8142 populate_metadata read hostname 'smithi007' 2024-04-25T11:19:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.637+0000 7fce73035740 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:6832/1924056839,v1:172.21.15.87:6834/1924056839] conn(0x55ca629c7e40 0x55ca62deea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.637+0000 7fce73035740 4 client.8142 mds_command: new command op to 8007 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:19:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.637+0000 7fce73035740 1 -- 172.21.15.7:0/119838432 --> [v2:172.21.15.87:6832/1924056839,v1:172.21.15.87:6834/1924056839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca62d1d210 con 0x55ca629c7e40 2024-04-25T11:19:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.638+0000 7fce69600640 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:6832/1924056839,v1:172.21.15.87:6834/1924056839] conn(0x55ca629c7e40 0x55ca62deea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.639+0000 7fce69600640 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:6832/1924056839,v1:172.21.15.87:6834/1924056839] conn(0x55ca629c7e40 0x55ca62deea20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6274f4e25080b8 server_cookie=d302d398711b9c84 in_seq=0 out_seq=0 2024-04-25T11:19:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.639+0000 7fce4a000640 10 client.8142 ms_handle_connect on v2:172.21.15.87:6832/1924056839 2024-04-25T11:19:55.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce4a000640 1 -- 172.21.15.7:0/119838432 <== mds.0 v2:172.21.15.87:6832/1924056839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ca62d1d210 con 0x55ca629c7e40 2024-04-25T11:19:55.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce4a000640 10 client.8142 handle_command_reply: tid=0 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 1 client.8142 shutdown 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 1 -- 172.21.15.7:0/119838432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e8590 msgr2=0x55ca62deb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e8590 0x55ca62deb4d0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fce50018000 tx=0x7fce500046c0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 1 -- 172.21.15.7:0/119838432 shutdown_connections 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:6832/1924056839,v1:172.21.15.87:6834/1924056839] conn(0x55ca629c7e40 0x55ca62deea20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62a09270 0x55ca62df0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e8590 0x55ca62deb4d0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 1 --2- 172.21.15.7:0/119838432 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca629e03b0 0x55ca62dd0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce4a000640 0 client.8142 ms_handle_reset on v2:172.21.15.87:6832/1924056839 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 1 -- 172.21.15.7:0/119838432 >> 172.21.15.7:0/119838432 conn(0x55ca629fd3c0 msgr2=0x55ca62dd2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 1 -- 172.21.15.7:0/119838432 shutdown_connections 2024-04-25T11:19:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 1 -- 172.21.15.7:0/119838432 wait complete. 2024-04-25T11:19:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.641+0000 7fce73035740 20 client.8142 trim_cache size 0 max 16384 2024-04-25T11:19:55.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.662+0000 7fce73035740 1 Processor -- start 2024-04-25T11:19:55.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.663+0000 7fce73035740 1 -- start start 2024-04-25T11:19:55.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.664+0000 7fce73035740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62deec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.664+0000 7fce73035740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca629e8590 0x55ca62df11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.664+0000 7fce73035740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62a09270 0x55ca62df1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.664+0000 7fce73035740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce64202a40 con 0x55ca629e03b0 2024-04-25T11:19:55.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.664+0000 7fce73035740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce64201da0 con 0x55ca62a09270 2024-04-25T11:19:55.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.664+0000 7fce73035740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce642023f0 con 0x55ca629e8590 2024-04-25T11:19:55.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.664+0000 7fce69600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62deec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.665+0000 7fce69600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62deec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42546/0 (socket says 172.21.15.7:42546) 2024-04-25T11:19:55.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.665+0000 7fce69600640 1 -- 172.21.15.7:0/3197765617 learned_addr learned my addr 172.21.15.7:0/3197765617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:55.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.665+0000 7fce6a000640 1 --2- 172.21.15.7:0/3197765617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62a09270 0x55ca62df1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.665+0000 7fce68c00640 1 --2- 172.21.15.7:0/3197765617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca629e8590 0x55ca62df11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.665+0000 7fce69600640 1 -- 172.21.15.7:0/3197765617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca629e8590 msgr2=0x55ca62df11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.665+0000 7fce69600640 1 --2- 172.21.15.7:0/3197765617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca629e8590 0x55ca62df11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.665+0000 7fce69600640 1 -- 172.21.15.7:0/3197765617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62a09270 msgr2=0x55ca62df1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.665+0000 7fce69600640 1 --2- 172.21.15.7:0/3197765617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62a09270 0x55ca62df1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.665+0000 7fce69600640 1 -- 172.21.15.7:0/3197765617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca62da20a0 con 0x55ca629e03b0 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.665+0000 7fce69600640 1 --2- 172.21.15.7:0/3197765617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62deec10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fce58018990 tx=0x7fce58015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46993c4d52282445 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.666+0000 7fce55200640 1 -- 172.21.15.7:0/3197765617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58017b30 con 0x55ca629e03b0 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.666+0000 7fce55200640 1 -- 172.21.15.7:0/3197765617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce5802d070 con 0x55ca629e03b0 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.666+0000 7fce55200640 1 -- 172.21.15.7:0/3197765617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce580270d0 con 0x55ca629e03b0 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.667+0000 7fce73035740 1 -- 172.21.15.7:0/3197765617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 msgr2=0x55ca62deec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.667+0000 7fce73035740 1 --2- 172.21.15.7:0/3197765617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62deec10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fce58018990 tx=0x7fce58015000 comp rx=0 tx=0).stop 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.667+0000 7fce73035740 1 -- 172.21.15.7:0/3197765617 shutdown_connections 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.667+0000 7fce73035740 1 --2- 172.21.15.7:0/3197765617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca62a09270 0x55ca62df1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.667+0000 7fce73035740 1 --2- 172.21.15.7:0/3197765617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca629e8590 0x55ca62df11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.667+0000 7fce73035740 1 --2- 172.21.15.7:0/3197765617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62deec10 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.667+0000 7fce73035740 1 -- 172.21.15.7:0/3197765617 >> 172.21.15.7:0/3197765617 conn(0x55ca629fd3c0 msgr2=0x55ca62da1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.667+0000 7fce73035740 1 -- 172.21.15.7:0/3197765617 shutdown_connections 2024-04-25T11:19:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.667+0000 7fce73035740 1 -- 172.21.15.7:0/3197765617 wait complete. 2024-04-25T11:19:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.668+0000 7fce73035740 1 -- start start 2024-04-25T11:19:55.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.668+0000 7fce73035740 1 Processor -- start 2024-04-25T11:19:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce73035740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62e70c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce73035740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca629e8590 0x55ca62e71150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce73035740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca62a09270 0x55ca62de8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce73035740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ca62dd9fc0 con 0x55ca629e03b0 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce73035740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ca62df1070 con 0x55ca629e8590 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce73035740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55ca62de99a0 con 0x55ca62a09270 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce73035740 10 client.0 _ll_register_callbacks cb 0x7fce50009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce69600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62e70c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce69600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62e70c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42550/0 (socket says 172.21.15.7:42550) 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce69600640 1 -- 172.21.15.7:0/532144445 learned_addr learned my addr 172.21.15.7:0/532144445 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce69600640 1 -- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca62a09270 msgr2=0x55ca62de8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce6a000640 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca62a09270 0x55ca62de8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce68c00640 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca629e8590 0x55ca62e71150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce69600640 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca62a09270 0x55ca62de8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce69600640 1 -- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca629e8590 msgr2=0x55ca62e71150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce69600640 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca629e8590 0x55ca62e71150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.670+0000 7fce69600640 1 -- 172.21.15.7:0/532144445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca62dd8220 con 0x55ca629e03b0 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.671+0000 7fce69600640 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62e70c10 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fce58018750 tx=0x7fce58004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4261b0c12db1257a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.671+0000 7fce4a000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:19:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.671+0000 7fce4a000640 1 -- 172.21.15.7:0/532144445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58038020 con 0x55ca629e03b0 2024-04-25T11:19:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.671+0000 7fce4a000640 1 -- 172.21.15.7:0/532144445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fce580274e0 con 0x55ca629e03b0 2024-04-25T11:19:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.671+0000 7fce73035740 1 -- 172.21.15.7:0/532144445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5802d070 con 0x55ca629e03b0 2024-04-25T11:19:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.671+0000 7fce4a000640 1 -- 172.21.15.7:0/532144445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fce580276e0 con 0x55ca629e03b0 2024-04-25T11:19:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.671+0000 7fce73035740 10 client.8148 fetch_fsmap learned FSMap version 76 2024-04-25T11:19:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.671+0000 7fce73035740 1 -- 172.21.15.7:0/532144445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55ca62e71750 con 0x55ca629e03b0 2024-04-25T11:19:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.672+0000 7fce4a000640 1 -- 172.21.15.7:0/532144445 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce5802d070 con 0x55ca629e03b0 2024-04-25T11:19:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.672+0000 7fce4a000640 1 -- 172.21.15.7:0/532144445 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3494+0+0 (secure 0 0 0) 0x7fce58027880 con 0x55ca629e03b0 2024-04-25T11:19:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.672+0000 7fce73035740 10 client.8148 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T11:19:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.672+0000 7fce73035740 10 client.8148 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T11:19:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.672+0000 7fce73035740 20 client.8148 populate_metadata read hostname 'smithi007' 2024-04-25T11:19:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.672+0000 7fce73035740 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:6832/1924056839,v1:172.21.15.87:6834/1924056839] conn(0x55ca62dd24d0 0x55ca62dd28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.672+0000 7fce73035740 4 client.8148 mds_command: new command op to 8007 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:19:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.672+0000 7fce73035740 1 -- 172.21.15.7:0/532144445 --> [v2:172.21.15.87:6832/1924056839,v1:172.21.15.87:6834/1924056839] -- command(tid 0: {"prefix": "status"}) -- 0x55ca62d17080 con 0x55ca62dd24d0 2024-04-25T11:19:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.672+0000 7fce68c00640 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:6832/1924056839,v1:172.21.15.87:6834/1924056839] conn(0x55ca62dd24d0 0x55ca62dd28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.673+0000 7fce68c00640 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:6832/1924056839,v1:172.21.15.87:6834/1924056839] conn(0x55ca62dd24d0 0x55ca62dd28e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b443f472e3d017 server_cookie=b89a9c2f7aa03ec7 in_seq=0 out_seq=0 2024-04-25T11:19:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.673+0000 7fce4a000640 10 client.8148 ms_handle_connect on v2:172.21.15.87:6832/1924056839 2024-04-25T11:19:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce4a000640 1 -- 172.21.15.7:0/532144445 <== mds.0 v2:172.21.15.87:6832/1924056839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55ca62d17080 con 0x55ca62dd24d0 2024-04-25T11:19:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce4a000640 10 client.8148 handle_command_reply: tid=0 2024-04-25T11:19:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce73035740 1 client.8148 shutdown 2024-04-25T11:19:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce73035740 1 -- 172.21.15.7:0/532144445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 msgr2=0x55ca62e70c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce73035740 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62e70c10 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fce58018750 tx=0x7fce58004ab0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce73035740 1 -- 172.21.15.7:0/532144445 shutdown_connections 2024-04-25T11:19:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce73035740 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:6832/1924056839,v1:172.21.15.87:6834/1924056839] conn(0x55ca62dd24d0 0x55ca62dd28e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce73035740 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ca62a09270 0x55ca62de8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce73035740 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ca629e8590 0x55ca62e71150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce73035740 1 --2- 172.21.15.7:0/532144445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ca629e03b0 0x55ca62e70c10 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce73035740 1 -- 172.21.15.7:0/532144445 >> 172.21.15.7:0/532144445 conn(0x55ca629fd3c0 msgr2=0x55ca62e02c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.674+0000 7fce4a000640 0 client.8148 ms_handle_reset on v2:172.21.15.87:6832/1924056839 2024-04-25T11:19:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.675+0000 7fce73035740 1 -- 172.21.15.7:0/532144445 shutdown_connections 2024-04-25T11:19:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.675+0000 7fce73035740 1 -- 172.21.15.7:0/532144445 wait complete. 2024-04-25T11:19:55.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.675+0000 7fce73035740 20 client.8148 trim_cache size 0 max 16384 2024-04-25T11:19:55.674 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:19:55.674 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:19:55.674 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:19:55.674 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 8007, 2024-04-25T11:19:55.674 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:19:55.675 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:19:55.675 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:19:55.675 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.3951402450000003, 2024-04-25T11:19:55.675 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 76, 2024-04-25T11:19:55.675 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 118, 2024-04-25T11:19:55.675 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 118, 2024-04-25T11:19:55.675 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.545576329999999 2024-04-25T11:19:55.675 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:19:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 -- 172.21.15.7:0/1728932102 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744b0 msgr2=0x7fce440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744b0 0x7fce440769c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fce6406a050 tx=0x7fce5800b040 comp rx=0 tx=0).stop 2024-04-25T11:19:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 -- 172.21.15.7:0/1728932102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce641d9980 msgr2=0x7fce640c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce641d9980 0x7fce640c92c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00c2d0 tx=0x7fce5c001a40 comp rx=0 tx=0).stop 2024-04-25T11:19:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 -- 172.21.15.7:0/1728932102 shutdown_connections 2024-04-25T11:19:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744b0 0x7fce440769c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce641d9980 0x7fce640c92c0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce641d9570 0x7fce640c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 --2- 172.21.15.7:0/1728932102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce641dafd0 0x7fce641d8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 -- 172.21.15.7:0/1728932102 >> 172.21.15.7:0/1728932102 conn(0x7fce640d0240 msgr2=0x7fce640b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 -- 172.21.15.7:0/1728932102 shutdown_connections 2024-04-25T11:19:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.676+0000 7fce6aa00640 1 -- 172.21.15.7:0/1728932102 wait complete. 2024-04-25T11:19:55.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:19:55.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.984+0000 7f409d600640 1 Processor -- start 2024-04-25T11:19:55.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.984+0000 7f409d600640 1 -- start start 2024-04-25T11:19:55.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.986+0000 7f409d600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f409815baa0 0x7f409815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.986+0000 7f409d600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f409815e980 con 0x7f409806d580 2024-04-25T11:19:55.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.986+0000 7f409d600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f409815ec60 con 0x7f4098070640 2024-04-25T11:19:55.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.986+0000 7f409d600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f409815ef40 con 0x7f409815baa0 2024-04-25T11:19:55.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.986+0000 7f409cc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f409806d580 0x7f409806d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58800/0 (socket says 172.21.15.7:58800) 2024-04-25T11:19:55.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.987+0000 7f409cc00640 1 -- 172.21.15.7:0/4236294897 learned_addr learned my addr 172.21.15.7:0/4236294897 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:19:55.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.987+0000 7f4097e00640 1 --2- 172.21.15.7:0/4236294897 >> v2:172.21.15.87:3300/0 conn(0x7f409815baa0 0x7f409815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.987+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4289862816 0 0) 0x7f409815e980 con 0x7f409806d580 2024-04-25T11:19:55.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.987+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f407c003520 con 0x7f409806d580 2024-04-25T11:19:55.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.987+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2778171125 0 0) 0x7f407c003520 con 0x7f409806d580 2024-04-25T11:19:55.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.987+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 >> v2:172.21.15.87:3300/0 conn(0x7f409815baa0 msgr2=0x7f409815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4096a00640 1 --2- 172.21.15.7:0/4236294897 >> v2:172.21.15.87:3300/0 conn(0x7f409815baa0 0x7f409815e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 >> v1:172.21.15.87:6789/0 conn(0x7f4098070640 legacy=0x7f4098161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40981619c0 con 0x7f409806d580 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f4090001670 con 0x7f409806d580 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 >> v1:172.21.15.7:6789/0 conn(0x7f409806d580 legacy=0x7f409806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4096a00640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f407c0039b0 0x7f407c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4096a00640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f407c004800 0x7f407c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4096a00640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f407c0051c0 0x7f407c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f409815ef40 con 0x7f407c0051c0 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f407c00c160 con 0x7f407c0039b0 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f407c00c360 con 0x7f407c004800 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f409cc00640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f407c0051c0 0x7f407c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4097e00640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f407c0039b0 0x7f407c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.989+0000 7f409cc00640 1 -- 172.21.15.7:0/4236294897 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f407c004800 msgr2=0x7f407c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.988+0000 7f4097400640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f407c004800 0x7f407c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.989+0000 7f409cc00640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f407c004800 0x7f407c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.989+0000 7f409cc00640 1 -- 172.21.15.7:0/4236294897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f407c0039b0 msgr2=0x7f407c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:55.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.989+0000 7f409cc00640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f407c0039b0 0x7f407c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.989+0000 7f409cc00640 1 -- 172.21.15.7:0/4236294897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c003520 con 0x7f407c0051c0 2024-04-25T11:19:55.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.989+0000 7f4097400640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f407c004800 0x7f407c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:19:55.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.989+0000 7f409cc00640 1 -- 172.21.15.7:0/4236294897 reap_dead start 2024-04-25T11:19:55.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.990+0000 7f409cc00640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f407c0051c0 0x7f407c00b9c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f409000c210 tx=0x7f407c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67ff949bd92bc60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.990+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090007d10 con 0x7f407c0051c0 2024-04-25T11:19:55.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.990+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4090005ad0 con 0x7f407c0051c0 2024-04-25T11:19:55.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.990+0000 7f4096a00640 1 -- 172.21.15.7:0/4236294897 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090007070 con 0x7f407c0051c0 2024-04-25T11:19:55.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.991+0000 7f409d600640 1 -- 172.21.15.7:0/4236294897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f407c0051c0 msgr2=0x7f407c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.991+0000 7f409d600640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f407c0051c0 0x7f407c00b9c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f409000c210 tx=0x7f407c005830 comp rx=0 tx=0).stop 2024-04-25T11:19:55.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.991+0000 7f409d600640 1 -- 172.21.15.7:0/4236294897 shutdown_connections 2024-04-25T11:19:55.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.991+0000 7f409d600640 1 --2- 172.21.15.7:0/4236294897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f407c0051c0 0x7f407c00b9c0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.991+0000 7f409d600640 1 -- 172.21.15.7:0/4236294897 >> 172.21.15.7:0/4236294897 conn(0x7f4098073090 msgr2=0x7f40980734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:55.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.991+0000 7f409d600640 1 -- 172.21.15.7:0/4236294897 shutdown_connections 2024-04-25T11:19:55.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.991+0000 7f409d600640 1 -- 172.21.15.7:0/4236294897 wait complete. 2024-04-25T11:19:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.993+0000 7f409d600640 1 Processor -- start 2024-04-25T11:19:55.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.994+0000 7f409d600640 1 -- start start 2024-04-25T11:19:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.995+0000 7f409d600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f407c0051c0 0x7f40980aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.995+0000 7f409d600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f409815a640 0x7f40980c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f409815aa50 0x7f40980c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409d600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f409815bc20 con 0x7f409815aa50 2024-04-25T11:19:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409d600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f409815baa0 con 0x7f407c0051c0 2024-04-25T11:19:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409d600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f409815e2f0 con 0x7f409815a640 2024-04-25T11:19:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f409815aa50 0x7f40980c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f409815aa50 0x7f40980c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42562/0 (socket says 172.21.15.7:42562) 2024-04-25T11:19:55.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409cc00640 1 -- 172.21.15.7:0/2682639918 learned_addr learned my addr 172.21.15.7:0/2682639918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f4097e00640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f407c0051c0 0x7f40980aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f4097400640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f409815a640 0x7f40980c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409cc00640 1 -- 172.21.15.7:0/2682639918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f409815a640 msgr2=0x7f40980c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409cc00640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f409815a640 0x7f40980c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409cc00640 1 -- 172.21.15.7:0/2682639918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f407c0051c0 msgr2=0x7f40980aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409cc00640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f407c0051c0 0x7f40980aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.996+0000 7f409cc00640 1 -- 172.21.15.7:0/2682639918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40980af860 con 0x7f409815aa50 2024-04-25T11:19:55.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.997+0000 7f409cc00640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f409815aa50 0x7f40980c74e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f409000c210 tx=0x7f409815eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.997+0000 7f408fe00640 1 -- 172.21.15.7:0/2682639918 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090042070 con 0x7f409815aa50 2024-04-25T11:19:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.997+0000 7f408fe00640 1 -- 172.21.15.7:0/2682639918 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4090001640 con 0x7f409815aa50 2024-04-25T11:19:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.997+0000 7f408fe00640 1 -- 172.21.15.7:0/2682639918 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40900049e0 con 0x7f409815aa50 2024-04-25T11:19:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.997+0000 7f409d600640 1 -- 172.21.15.7:0/2682639918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40980afb40 con 0x7f409815aa50 2024-04-25T11:19:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.997+0000 7f409d600640 1 -- 172.21.15.7:0/2682639918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409815fd60 con 0x7f409815aa50 2024-04-25T11:19:55.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:55.998+0000 7f409d600640 1 -- 172.21.15.7:0/2682639918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4098050740 con 0x7f409815aa50 2024-04-25T11:19:55.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.000+0000 7f408fe00640 1 -- 172.21.15.7:0/2682639918 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f409003a070 con 0x7f409815aa50 2024-04-25T11:19:55.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.001+0000 7f408fe00640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40740744b0 0x7f40740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:55.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.001+0000 7f408fe00640 1 -- 172.21.15.7:0/2682639918 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f40900b42a0 con 0x7f409815aa50 2024-04-25T11:19:55.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.001+0000 7f4097e00640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40740744b0 0x7f40740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:55.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.002+0000 7f4097e00640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40740744b0 0x7f40740769c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4088009fd0 tx=0x7f408800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:56.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.006+0000 7f408fe00640 1 -- 172.21.15.7:0/2682639918 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f409007e800 con 0x7f409815aa50 2024-04-25T11:19:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.251+0000 7f409d600640 1 -- 172.21.15.7:0/2682639918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f40980afd10 con 0x7f409815aa50 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.251+0000 7f408fe00640 1 -- 172.21.15.7:0/2682639918 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7f409007e100 con 0x7f409815aa50 2024-04-25T11:19:56.249 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 -- 172.21.15.7:0/2682639918 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40740744b0 msgr2=0x7f40740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40740744b0 0x7f40740769c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4088009fd0 tx=0x7f408800b040 comp rx=0 tx=0).stop 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 -- 172.21.15.7:0/2682639918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f409815aa50 msgr2=0x7f40980c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f409815aa50 0x7f40980c74e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f409000c210 tx=0x7f409815eb50 comp rx=0 tx=0).stop 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 -- 172.21.15.7:0/2682639918 shutdown_connections 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40740744b0 0x7f40740769c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f409815aa50 0x7f40980c74e0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f409815a640 0x7f40980c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 --2- 172.21.15.7:0/2682639918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f407c0051c0 0x7f40980aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:56.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 -- 172.21.15.7:0/2682639918 >> 172.21.15.7:0/2682639918 conn(0x7f4098073090 msgr2=0x7f4098146fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 -- 172.21.15.7:0/2682639918 shutdown_connections 2024-04-25T11:19:56.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56.254+0000 7f409d600640 1 -- 172.21.15.7:0/2682639918 wait complete. 2024-04-25T11:19:56.277 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:19:56.277 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:19:56.278 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:19:56.278 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:19:56.278 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:56.278 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:19:56.278 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:19:56.278 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:19:56.278 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:19:56.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:56.278 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:19:56.329 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41978sec preferred_lft 41978sec 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:19:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6098:c7ff:fe8a:4a30/64 scope link 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::646a:35ff:fe7b:ad0/64 scope link 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:19:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:19:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4465:c4ff:fe62:f75f/64 scope link 2024-04-25T11:19:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:56.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:56.333 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:19:56.416 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:19:56.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:56.421 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:19:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:19:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:19:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:19:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:19:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:19:56.475 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:56.475 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:19:56.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:19:56.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:19:56.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:19:56.475 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:19:56.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:56.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:56.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:56.691 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:19:56.742 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:19:56.743 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:19:56.744 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:19:56.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:56.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:19:56.831 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:56.832 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:56.888 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:19:56.888 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:56.888 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:56.936 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:19:56.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:56.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:57.193 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:57.193 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:19:57.238 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:57.239 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:19:57.239 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:19:57.239 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:19:57.239 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:19:57.240 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:19:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:19:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:19:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:19:57.241 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:19:57.246 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:19:57.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:57.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:19:57.323 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:19:57.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:19:57.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:57.463 INFO:teuthology.orchestra.run.smithi007.stdout:client8157 2024-04-25T11:19:57.463 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:19:57.464 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:57.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:19:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.848+0000 7fd0ca400640 1 Processor -- start 2024-04-25T11:19:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.848+0000 7fd0ca400640 1 -- start start 2024-04-25T11:19:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.850+0000 7fd0ca400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0c415baa0 0x7fd0c415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.850+0000 7fd0ca400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd0c4070640 0x7fd0c4161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.850+0000 7fd0ca400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0c415e980 con 0x7fd0c4162420 2024-04-25T11:19:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.850+0000 7fd0ca400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd0c4162830 con 0x7fd0c4070640 2024-04-25T11:19:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.850+0000 7fd0ca400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c415ec60 con 0x7fd0c415baa0 2024-04-25T11:19:57.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.851+0000 7fd0c9000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0c415baa0 0x7fd0c415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.851+0000 7fd0c3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd0c4070640 0x7fd0c4161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.851+0000 7fd0c9000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0c415baa0 0x7fd0c415e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:43720/0 (socket says 172.21.15.7:43720) 2024-04-25T11:19:57.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.851+0000 7fd0c9000640 1 -- 172.21.15.7:0/1764609084 learned_addr learned my addr 172.21.15.7:0/1764609084 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.851+0000 7fd0c9a00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd0c4162420 0x7fd0c406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:57950/0 (socket says 172.21.15.7:57950) 2024-04-25T11:19:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.852+0000 7fd0c3e00640 1 -- 172.21.15.7:0/1764609084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0c415baa0 msgr2=0x7fd0c415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.852+0000 7fd0c3e00640 1 --2- 172.21.15.7:0/1764609084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0c415baa0 0x7fd0c415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.852+0000 7fd0c3e00640 1 -- 172.21.15.7:0/1764609084 >> v1:172.21.15.87:6789/0 conn(0x7fd0c4162420 legacy=0x7fd0c406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.852+0000 7fd0c3e00640 1 -- 172.21.15.7:0/1764609084 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c415ef40 con 0x7fd0c4070640 2024-04-25T11:19:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.853+0000 7fd0c3e00640 1 --2- 172.21.15.7:0/1764609084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0c4070640 0x7fd0c4161ee0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00097a0 tx=0x7fd0b002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4df00852dba63789 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.854+0000 7fd0c3400640 1 -- 172.21.15.7:0/1764609084 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0037070 con 0x7fd0c4070640 2024-04-25T11:19:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.854+0000 7fd0c3400640 1 -- 172.21.15.7:0/1764609084 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b0032440 con 0x7fd0c4070640 2024-04-25T11:19:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.854+0000 7fd0c3400640 1 -- 172.21.15.7:0/1764609084 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0032810 con 0x7fd0c4070640 2024-04-25T11:19:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.854+0000 7fd0ca400640 1 -- 172.21.15.7:0/1764609084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0c4070640 msgr2=0x7fd0c4161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.854+0000 7fd0ca400640 1 --2- 172.21.15.7:0/1764609084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0c4070640 0x7fd0c4161ee0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00097a0 tx=0x7fd0b002a180 comp rx=0 tx=0).stop 2024-04-25T11:19:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.855+0000 7fd0ca400640 1 -- 172.21.15.7:0/1764609084 shutdown_connections 2024-04-25T11:19:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.855+0000 7fd0ca400640 1 --2- 172.21.15.7:0/1764609084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0c4070640 0x7fd0c4161ee0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.855+0000 7fd0ca400640 1 --2- 172.21.15.7:0/1764609084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0c415baa0 0x7fd0c415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.855+0000 7fd0ca400640 1 -- 172.21.15.7:0/1764609084 >> 172.21.15.7:0/1764609084 conn(0x7fd0c4073090 msgr2=0x7fd0c40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.855+0000 7fd0ca400640 1 -- 172.21.15.7:0/1764609084 shutdown_connections 2024-04-25T11:19:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.855+0000 7fd0ca400640 1 -- 172.21.15.7:0/1764609084 wait complete. 2024-04-25T11:19:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.857+0000 7fd0ca400640 1 Processor -- start 2024-04-25T11:19:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.857+0000 7fd0ca400640 1 -- start start 2024-04-25T11:19:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.859+0000 7fd0ca400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4070640 0x7fd0c415d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.859+0000 7fd0ca400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0c4162420 0x7fd0c4133de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.859+0000 7fd0ca400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0c4139930 0x7fd0c4134320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.859+0000 7fd0ca400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4070bd0 con 0x7fd0c4070640 2024-04-25T11:19:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.859+0000 7fd0ca400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4070a50 con 0x7fd0c4162420 2024-04-25T11:19:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.859+0000 7fd0ca400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c406dc90 con 0x7fd0c4139930 2024-04-25T11:19:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.859+0000 7fd0c9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4070640 0x7fd0c415d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.859+0000 7fd0c9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4070640 0x7fd0c415d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42568/0 (socket says 172.21.15.7:42568) 2024-04-25T11:19:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.859+0000 7fd0c9000640 1 -- 172.21.15.7:0/3873179819 learned_addr learned my addr 172.21.15.7:0/3873179819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.859+0000 7fd0c3e00640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0c4162420 0x7fd0c4133de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.860+0000 7fd0c9a00640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0c4139930 0x7fd0c4134320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.860+0000 7fd0c9000640 1 -- 172.21.15.7:0/3873179819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0c4139930 msgr2=0x7fd0c4134320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.860+0000 7fd0c9000640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0c4139930 0x7fd0c4134320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.860+0000 7fd0c9000640 1 -- 172.21.15.7:0/3873179819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0c4162420 msgr2=0x7fd0c4133de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.860+0000 7fd0c9000640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0c4162420 0x7fd0c4133de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.860+0000 7fd0c9000640 1 -- 172.21.15.7:0/3873179819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c415c280 con 0x7fd0c4070640 2024-04-25T11:19:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.860+0000 7fd0c9000640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4070640 0x7fd0c415d320 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800e870 tx=0x7fd0b800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.860+0000 7fd0c0c00640 1 -- 172.21.15.7:0/3873179819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8017070 con 0x7fd0c4070640 2024-04-25T11:19:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.861+0000 7fd0c0c00640 1 -- 172.21.15.7:0/3873179819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b800c8e0 con 0x7fd0c4070640 2024-04-25T11:19:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.861+0000 7fd0ca400640 1 -- 172.21.15.7:0/3873179819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c415c560 con 0x7fd0c4070640 2024-04-25T11:19:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.861+0000 7fd0c0c00640 1 -- 172.21.15.7:0/3873179819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800ccd0 con 0x7fd0c4070640 2024-04-25T11:19:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.861+0000 7fd0ca400640 1 -- 172.21.15.7:0/3873179819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c415cb20 con 0x7fd0c4070640 2024-04-25T11:19:57.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.862+0000 7fd0ca400640 1 -- 172.21.15.7:0/3873179819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c4074020 con 0x7fd0c4070640 2024-04-25T11:19:57.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.863+0000 7fd0c0c00640 1 -- 172.21.15.7:0/3873179819 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd0b801a070 con 0x7fd0c4070640 2024-04-25T11:19:57.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.864+0000 7fd0c0c00640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0a40744e0 0x7fd0a40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:57.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.865+0000 7fd0c0c00640 1 -- 172.21.15.7:0/3873179819 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fd0b8095620 con 0x7fd0c4070640 2024-04-25T11:19:57.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.865+0000 7fd0c3e00640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0a40744e0 0x7fd0a40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:57.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.866+0000 7fd0c3e00640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0a40744e0 0x7fd0a40769f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0034000 tx=0x7fd0b002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:57.870+0000 7fd0c0c00640 1 -- 172.21.15.7:0/3873179819 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0b805fc50 con 0x7fd0c4070640 2024-04-25T11:19:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.124+0000 7fd0ca400640 1 -- 172.21.15.7:0/3873179819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd0c4161190 con 0x7fd0c4070640 2024-04-25T11:19:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.125+0000 7fd0c0c00640 1 -- 172.21.15.7:0/3873179819 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7fd0b805f550 con 0x7fd0c4070640 2024-04-25T11:19:58.124 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:19:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.127+0000 7fd0ca400640 1 -- 172.21.15.7:0/3873179819 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0a40744e0 msgr2=0x7fd0a40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.127+0000 7fd0ca400640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0a40744e0 0x7fd0a40769f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0034000 tx=0x7fd0b002e000 comp rx=0 tx=0).stop 2024-04-25T11:19:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.127+0000 7fd0ca400640 1 -- 172.21.15.7:0/3873179819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4070640 msgr2=0x7fd0c415d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.127+0000 7fd0ca400640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4070640 0x7fd0c415d320 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800e870 tx=0x7fd0b800ed30 comp rx=0 tx=0).stop 2024-04-25T11:19:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.128+0000 7fd0ca400640 1 -- 172.21.15.7:0/3873179819 shutdown_connections 2024-04-25T11:19:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.128+0000 7fd0ca400640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0a40744e0 0x7fd0a40769f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.128+0000 7fd0ca400640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0c4139930 0x7fd0c4134320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.128+0000 7fd0ca400640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0c4162420 0x7fd0c4133de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.128+0000 7fd0ca400640 1 --2- 172.21.15.7:0/3873179819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c4070640 0x7fd0c415d320 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.128+0000 7fd0ca400640 1 -- 172.21.15.7:0/3873179819 >> 172.21.15.7:0/3873179819 conn(0x7fd0c4073090 msgr2=0x7fd0c40b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.128+0000 7fd0ca400640 1 -- 172.21.15.7:0/3873179819 shutdown_connections 2024-04-25T11:19:58.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:58.128+0000 7fd0ca400640 1 -- 172.21.15.7:0/3873179819 wait complete. 2024-04-25T11:19:58.155 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:19:58.155 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:19:58.155 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:19:58.155 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:19:58.155 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:58.155 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:19:58.155 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:19:58.155 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:19:58.156 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:19:58.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:58.156 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:19:58.215 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:19:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:58.216 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:19:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:19:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41977sec preferred_lft 41977sec 2024-04-25T11:19:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:19:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:58.216 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:19:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:19:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:19:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::58f2:6bff:fe8b:8b4c/64 scope link 2024-04-25T11:19:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:58.217 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:19:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:19:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::98fe:1bff:fe9f:d71c/64 scope link 2024-04-25T11:19:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:58.217 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:19:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:19:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::3c38:a8ff:fec7:ea00/64 scope link 2024-04-25T11:19:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:19:58.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:58.217 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:19:58.299 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:19:58.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:58.303 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:19:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:19:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:19:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:19:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:19:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:19:58.353 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:19:58.353 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:19:58.353 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:19:58.354 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:19:58.354 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:19:58.354 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:19:58.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:19:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:19:58.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:19:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:58.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:58.562 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:19:58.610 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:19:58.610 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:19:58.611 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:19:58.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:58.611 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T11:19:58.689 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:58.690 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:58.742 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:19:58.743 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:19:58.743 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:58.794 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:19:58.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:58.794 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:59.006 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:59.006 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:19:59.050 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:19:59.050 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:19:59.050 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:19:59.050 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 1@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:19:59.050 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:19:59.052 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:19:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:19:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:19:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:19:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:19:59.056 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:19:59.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:19:59.056 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:19:59.121 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:19:59.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:19:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:19:59.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:19:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:19:59.233 INFO:teuthology.orchestra.run.smithi087.stdout:client6098 2024-04-25T11:19:59.233 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:19:59.233 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:19:59.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-04-25T11:19:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.584+0000 7f035be00640 1 Processor -- start 2024-04-25T11:19:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.584+0000 7f035be00640 1 -- start start 2024-04-25T11:19:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.587+0000 7f035be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f035c0b1660 0x7f035c06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.587+0000 7f035be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f035c0d23a0 0x7f035c0c7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.587+0000 7f035be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f035c0c8700 con 0x7f035c0b7440 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.587+0000 7f035be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f035c0c6cc0 con 0x7f035c0b1660 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.587+0000 7f035be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f035c0c7280 con 0x7f035c0d23a0 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.588+0000 7f035b400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f035c0d23a0 0x7f035c0c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.588+0000 7f035aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f035c0b1660 0x7f035c06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.588+0000 7f035aa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f035c0b1660 0x7f035c06c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:37150/0 (socket says 172.21.15.7:37150) 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.588+0000 7f035aa00640 1 -- 172.21.15.7:0/3579864732 learned_addr learned my addr 172.21.15.7:0/3579864732 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.589+0000 7f035aa00640 1 -- 172.21.15.7:0/3579864732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f035c0d23a0 msgr2=0x7f035c0c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.589+0000 7f035aa00640 1 --2- 172.21.15.7:0/3579864732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f035c0d23a0 0x7f035c0c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.589+0000 7f035aa00640 1 -- 172.21.15.7:0/3579864732 >> v1:172.21.15.87:6789/0 conn(0x7f035c0b7440 legacy=0x7f035c0d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.589+0000 7f035aa00640 1 -- 172.21.15.7:0/3579864732 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c0c89a0 con 0x7f035c0b1660 2024-04-25T11:19:59.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.591+0000 7f035aa00640 1 --2- 172.21.15.7:0/3579864732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f035c0b1660 0x7f035c06c570 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f034c009e50 tx=0x7f034c02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=452edc6d7a590db7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.591+0000 7f0359600640 1 -- 172.21.15.7:0/3579864732 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c036070 con 0x7f035c0b1660 2024-04-25T11:19:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.591+0000 7f0359600640 1 -- 172.21.15.7:0/3579864732 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c031600 con 0x7f035c0b1660 2024-04-25T11:19:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.591+0000 7f0359600640 1 -- 172.21.15.7:0/3579864732 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c0319d0 con 0x7f035c0b1660 2024-04-25T11:19:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.592+0000 7f035be00640 1 -- 172.21.15.7:0/3579864732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f035c0b1660 msgr2=0x7f035c06c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.592+0000 7f035be00640 1 --2- 172.21.15.7:0/3579864732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f035c0b1660 0x7f035c06c570 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f034c009e50 tx=0x7f034c02a710 comp rx=0 tx=0).stop 2024-04-25T11:19:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.592+0000 7f035be00640 1 -- 172.21.15.7:0/3579864732 shutdown_connections 2024-04-25T11:19:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.592+0000 7f035be00640 1 --2- 172.21.15.7:0/3579864732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f035c0d23a0 0x7f035c0c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.592+0000 7f035be00640 1 --2- 172.21.15.7:0/3579864732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f035c0b1660 0x7f035c06c570 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.592+0000 7f035be00640 1 -- 172.21.15.7:0/3579864732 >> 172.21.15.7:0/3579864732 conn(0x7f035c09b540 msgr2=0x7f035c0b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.592+0000 7f035be00640 1 -- 172.21.15.7:0/3579864732 shutdown_connections 2024-04-25T11:19:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.592+0000 7f035be00640 1 -- 172.21.15.7:0/3579864732 wait complete. 2024-04-25T11:19:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.595+0000 7f035be00640 1 Processor -- start 2024-04-25T11:19:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.595+0000 7f035be00640 1 -- start start 2024-04-25T11:19:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.597+0000 7f035be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f035c0b1660 0x7f035c159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.597+0000 7f035be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f035c0b7440 0x7f035c1f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.597+0000 7f035be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f035c1f9250 0x7f035c161d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.597+0000 7f035be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f035c0b3680 con 0x7f035c0b1660 2024-04-25T11:19:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.597+0000 7f035be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f035c06cab0 con 0x7f035c0b7440 2024-04-25T11:19:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.597+0000 7f035be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f035c0b3800 con 0x7f035c1f9250 2024-04-25T11:19:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.597+0000 7f035aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f035c0b1660 0x7f035c159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.597+0000 7f035aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f035c0b1660 0x7f035c159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42574/0 (socket says 172.21.15.7:42574) 2024-04-25T11:19:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.597+0000 7f035aa00640 1 -- 172.21.15.7:0/3707170171 learned_addr learned my addr 172.21.15.7:0/3707170171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:19:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.597+0000 7f035b400640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f035c1f9250 0x7f035c161d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.597+0000 7f035a000640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f035c0b7440 0x7f035c1f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.598+0000 7f035aa00640 1 -- 172.21.15.7:0/3707170171 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f035c1f9250 msgr2=0x7f035c161d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.598+0000 7f035aa00640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f035c1f9250 0x7f035c161d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.598+0000 7f035aa00640 1 -- 172.21.15.7:0/3707170171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f035c0b7440 msgr2=0x7f035c1f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.598+0000 7f035aa00640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f035c0b7440 0x7f035c1f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.598+0000 7f035aa00640 1 -- 172.21.15.7:0/3707170171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c1f6590 con 0x7f035c0b1660 2024-04-25T11:19:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.598+0000 7f035b400640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f035c1f9250 0x7f035c161d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.598+0000 7f035a000640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f035c0b7440 0x7f035c1f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:19:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.598+0000 7f035aa00640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f035c0b1660 0x7f035c159aa0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f034c009f70 tx=0x7f034c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.598+0000 7f0353400640 1 -- 172.21.15.7:0/3707170171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c041020 con 0x7f035c0b1660 2024-04-25T11:19:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.599+0000 7f0353400640 1 -- 172.21.15.7:0/3707170171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034c0087b0 con 0x7f035c0b1660 2024-04-25T11:19:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.599+0000 7f0353400640 1 -- 172.21.15.7:0/3707170171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c036040 con 0x7f035c0b1660 2024-04-25T11:19:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.599+0000 7f035be00640 1 -- 172.21.15.7:0/3707170171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035c159fe0 con 0x7f035c0b1660 2024-04-25T11:19:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.599+0000 7f035be00640 1 -- 172.21.15.7:0/3707170171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035c161260 con 0x7f035c0b1660 2024-04-25T11:19:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.600+0000 7f035be00640 1 -- 172.21.15.7:0/3707170171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035c050740 con 0x7f035c0b1660 2024-04-25T11:19:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.601+0000 7f0353400640 1 -- 172.21.15.7:0/3707170171 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f034c044020 con 0x7f035c0b1660 2024-04-25T11:19:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.603+0000 7f0353400640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f033c0744e0 0x7f033c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:19:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.603+0000 7f0353400640 1 -- 172.21.15.7:0/3707170171 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f034c0b6300 con 0x7f035c0b1660 2024-04-25T11:19:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.603+0000 7f035a000640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f033c0744e0 0x7f033c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:19:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.604+0000 7f035a000640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f033c0744e0 0x7f033c0769f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f035c15fcf0 tx=0x7f0348009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:19:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.608+0000 7f0353400640 1 -- 172.21.15.7:0/3707170171 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f034c04b020 con 0x7f035c0b1660 2024-04-25T11:19:59.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.813+0000 7f035be00640 1 -- 172.21.15.7:0/3707170171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f035c0d0fb0 con 0x7f035c0b1660 2024-04-25T11:19:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.820+0000 7f0353400640 1 -- 172.21.15.7:0/3707170171 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v37)=0 v37) ==== 121+0+0 (secure 0 0 0) 0x7f034c0808f0 con 0x7f035c0b1660 2024-04-25T11:19:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 -- 172.21.15.7:0/3707170171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f033c0744e0 msgr2=0x7f033c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f033c0744e0 0x7f033c0769f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f035c15fcf0 tx=0x7f0348009040 comp rx=0 tx=0).stop 2024-04-25T11:19:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 -- 172.21.15.7:0/3707170171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f035c0b1660 msgr2=0x7f035c159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:19:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f035c0b1660 0x7f035c159aa0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f034c009f70 tx=0x7f034c004a40 comp rx=0 tx=0).stop 2024-04-25T11:19:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 -- 172.21.15.7:0/3707170171 shutdown_connections 2024-04-25T11:19:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f033c0744e0 0x7f033c0769f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f035c1f9250 0x7f035c161d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f035c0b7440 0x7f035c1f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 --2- 172.21.15.7:0/3707170171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f035c0b1660 0x7f035c159aa0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:19:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 -- 172.21.15.7:0/3707170171 >> 172.21.15.7:0/3707170171 conn(0x7f035c09b540 msgr2=0x7f035c15ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:19:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 -- 172.21.15.7:0/3707170171 shutdown_connections 2024-04-25T11:19:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:19:59.823+0000 7f035be00640 1 -- 172.21.15.7:0/3707170171 wait complete. 2024-04-25T11:19:59.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 400 2024-04-25T11:20:00.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.121+0000 7febf3e00640 1 Processor -- start 2024-04-25T11:20:00.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.122+0000 7febf3e00640 1 -- start start 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.124+0000 7febf3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7febf40ce8d0 0x7febf40cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.124+0000 7febf3e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7febf415c2e0 0x7febf415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.124+0000 7febf3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7febf406f730 0x7febf406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.124+0000 7febf3e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7febf4070080 con 0x7febf406f730 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.124+0000 7febf3e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7febf40b4830 con 0x7febf40ce8d0 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.124+0000 7febf3e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7febf40b4b10 con 0x7febf415c2e0 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.125+0000 7febf3400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7febf406f730 0x7febf406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.125+0000 7febf3400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7febf406f730 0x7febf406fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54340/0 (socket says 172.21.15.7:54340) 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.125+0000 7febf3400640 1 -- 172.21.15.7:0/568983330 learned_addr learned my addr 172.21.15.7:0/568983330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.125+0000 7febf2000640 1 --2- 172.21.15.7:0/568983330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7febf415c2e0 0x7febf415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.125+0000 7febf2a00640 1 --2- 172.21.15.7:0/568983330 >> v2:172.21.15.87:3300/0 conn(0x7febf40ce8d0 0x7febf40cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.125+0000 7febf3400640 1 -- 172.21.15.7:0/568983330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7febf415c2e0 msgr2=0x7febf415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.125+0000 7febf3400640 1 --2- 172.21.15.7:0/568983330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7febf415c2e0 0x7febf415c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.125+0000 7febf3400640 1 -- 172.21.15.7:0/568983330 >> v2:172.21.15.87:3300/0 conn(0x7febf40ce8d0 msgr2=0x7febf40cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:00.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.125+0000 7febf3400640 1 --2- 172.21.15.7:0/568983330 >> v2:172.21.15.87:3300/0 conn(0x7febf40ce8d0 0x7febf40cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.125+0000 7febf3400640 1 -- 172.21.15.7:0/568983330 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf40b4df0 con 0x7febf406f730 2024-04-25T11:20:00.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.125+0000 7febf2a00640 1 --2- 172.21.15.7:0/568983330 >> v2:172.21.15.87:3300/0 conn(0x7febf40ce8d0 0x7febf40cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:00.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.126+0000 7febf3400640 1 --2- 172.21.15.7:0/568983330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf406f730 0x7febf406fb40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7febf4072330 tx=0x7febec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9192cf0c349d041 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.127+0000 7febf1600640 1 -- 172.21.15.7:0/568983330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec036070 con 0x7febf406f730 2024-04-25T11:20:00.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.127+0000 7febf1600640 1 -- 172.21.15.7:0/568983330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febec031600 con 0x7febf406f730 2024-04-25T11:20:00.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.127+0000 7febf1600640 1 -- 172.21.15.7:0/568983330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febec0319d0 con 0x7febf406f730 2024-04-25T11:20:00.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.127+0000 7febf3e00640 1 -- 172.21.15.7:0/568983330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf406f730 msgr2=0x7febf406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.127+0000 7febf3e00640 1 --2- 172.21.15.7:0/568983330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf406f730 0x7febf406fb40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7febf4072330 tx=0x7febec02a6a0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.128+0000 7febf3e00640 1 -- 172.21.15.7:0/568983330 shutdown_connections 2024-04-25T11:20:00.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.128+0000 7febf3e00640 1 --2- 172.21.15.7:0/568983330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf406f730 0x7febf406fb40 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.128+0000 7febf3e00640 1 --2- 172.21.15.7:0/568983330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7febf415c2e0 0x7febf415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.128+0000 7febf3e00640 1 --2- 172.21.15.7:0/568983330 >> v2:172.21.15.87:3300/0 conn(0x7febf40ce8d0 0x7febf40cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.128+0000 7febf3e00640 1 -- 172.21.15.7:0/568983330 >> 172.21.15.7:0/568983330 conn(0x7febf40740f0 msgr2=0x7febf406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.128+0000 7febf3e00640 1 -- 172.21.15.7:0/568983330 shutdown_connections 2024-04-25T11:20:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.128+0000 7febf3e00640 1 -- 172.21.15.7:0/568983330 wait complete. 2024-04-25T11:20:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.130+0000 7febf3e00640 1 Processor -- start 2024-04-25T11:20:00.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.130+0000 7febf3e00640 1 -- start start 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.132+0000 7febf3e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7febf406f730 0x7febf414bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.132+0000 7febf3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf40ce8d0 0x7febf4154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.132+0000 7febf3e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febf415c2e0 0x7febf4155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.132+0000 7febf3e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7febf4070ff0 con 0x7febf40ce8d0 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.132+0000 7febf3e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7febf4070e70 con 0x7febf415c2e0 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.132+0000 7febf3e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7febf406eea0 con 0x7febf406f730 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.132+0000 7febf2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf40ce8d0 0x7febf4154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.132+0000 7febf2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf40ce8d0 0x7febf4154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54346/0 (socket says 172.21.15.7:54346) 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.132+0000 7febf2000640 1 -- 172.21.15.7:0/1787135238 learned_addr learned my addr 172.21.15.7:0/1787135238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.132+0000 7febf2a00640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7febf406f730 0x7febf414bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febf3400640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febf415c2e0 0x7febf4155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febf2000640 1 -- 172.21.15.7:0/1787135238 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7febf406f730 msgr2=0x7febf414bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febf2000640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7febf406f730 0x7febf414bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febf2000640 1 -- 172.21.15.7:0/1787135238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febf415c2e0 msgr2=0x7febf4155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febf2000640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febf415c2e0 0x7febf4155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febf2000640 1 -- 172.21.15.7:0/1787135238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf414ca50 con 0x7febf40ce8d0 2024-04-25T11:20:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febf2a00640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7febf406f730 0x7febf414bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febf3400640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febf415c2e0 0x7febf4155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febf2000640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf40ce8d0 0x7febf4154f20 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7febe000d620 tx=0x7febe000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febeaa00640 1 -- 172.21.15.7:0/1787135238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe0015070 con 0x7febf40ce8d0 2024-04-25T11:20:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febeaa00640 1 -- 172.21.15.7:0/1787135238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe0011870 con 0x7febf40ce8d0 2024-04-25T11:20:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febeaa00640 1 -- 172.21.15.7:0/1787135238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe000b040 con 0x7febf40ce8d0 2024-04-25T11:20:00.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.133+0000 7febf3e00640 1 -- 172.21.15.7:0/1787135238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf414cd30 con 0x7febf40ce8d0 2024-04-25T11:20:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.134+0000 7febf3e00640 1 -- 172.21.15.7:0/1787135238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf414d2f0 con 0x7febf40ce8d0 2024-04-25T11:20:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.135+0000 7febf3e00640 1 -- 172.21.15.7:0/1787135238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf4050740 con 0x7febf40ce8d0 2024-04-25T11:20:00.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.136+0000 7febeaa00640 1 -- 172.21.15.7:0/1787135238 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7febe0018070 con 0x7febf40ce8d0 2024-04-25T11:20:00.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.138+0000 7febeaa00640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febd40744b0 0x7febd40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.138+0000 7febeaa00640 1 -- 172.21.15.7:0/1787135238 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7febe0095430 con 0x7febf40ce8d0 2024-04-25T11:20:00.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.138+0000 7febf2a00640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febd40744b0 0x7febd40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.139+0000 7febf2a00640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febd40744b0 0x7febd40769c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7febe4006fd0 tx=0x7febe4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.142+0000 7febeaa00640 1 -- 172.21.15.7:0/1787135238 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7febe005f990 con 0x7febf40ce8d0 2024-04-25T11:20:00.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.346+0000 7febf3e00640 1 -- 172.21.15.7:0/1787135238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7febf40709b0 con 0x7febf40ce8d0 2024-04-25T11:20:00.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.352+0000 7febeaa00640 1 -- 172.21.15.7:0/1787135238 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v38)=0 v38) ==== 119+0+0 (secure 0 0 0) 0x7febe005f290 con 0x7febf40ce8d0 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 -- 172.21.15.7:0/1787135238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febd40744b0 msgr2=0x7febd40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febd40744b0 0x7febd40769c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7febe4006fd0 tx=0x7febe4008040 comp rx=0 tx=0).stop 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 -- 172.21.15.7:0/1787135238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf40ce8d0 msgr2=0x7febf4154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf40ce8d0 0x7febf4154f20 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7febe000d620 tx=0x7febe000dae0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 -- 172.21.15.7:0/1787135238 shutdown_connections 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7febd40744b0 0x7febd40769c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7febf415c2e0 0x7febf4155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febf40ce8d0 0x7febf4154f20 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 --2- 172.21.15.7:0/1787135238 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7febf406f730 0x7febf414bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 -- 172.21.15.7:0/1787135238 >> 172.21.15.7:0/1787135238 conn(0x7febf40740f0 msgr2=0x7febf406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 -- 172.21.15.7:0/1787135238 shutdown_connections 2024-04-25T11:20:00.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.355+0000 7febf3e00640 1 -- 172.21.15.7:0/1787135238 wait complete. 2024-04-25T11:20:00.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 800 2024-04-25T11:20:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.657+0000 7fd835000640 1 Processor -- start 2024-04-25T11:20:00.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.657+0000 7fd835000640 1 -- start start 2024-04-25T11:20:00.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.660+0000 7fd835000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd830146180 0x7fd830070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.660+0000 7fd835000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8300b08e0 0x7fd830149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.660+0000 7fd835000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd830070860 con 0x7fd8301490f0 2024-04-25T11:20:00.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.660+0000 7fd835000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd830146590 con 0x7fd8300b08e0 2024-04-25T11:20:00.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.660+0000 7fd835000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd83006f280 con 0x7fd830146180 2024-04-25T11:20:00.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.660+0000 7fd82fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8300b08e0 0x7fd830149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.660+0000 7fd82fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8300b08e0 0x7fd830149500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54352/0 (socket says 172.21.15.7:54352) 2024-04-25T11:20:00.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.660+0000 7fd82fe00640 1 -- 172.21.15.7:0/1439118425 learned_addr learned my addr 172.21.15.7:0/1439118425 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:00.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.660+0000 7fd82f400640 1 --2- 172.21.15.7:0/1439118425 >> v2:172.21.15.87:3300/0 conn(0x7fd830146180 0x7fd830070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.661+0000 7fd82fe00640 1 -- 172.21.15.7:0/1439118425 >> v2:172.21.15.87:3300/0 conn(0x7fd830146180 msgr2=0x7fd830070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:00.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.661+0000 7fd82fe00640 1 --2- 172.21.15.7:0/1439118425 >> v2:172.21.15.87:3300/0 conn(0x7fd830146180 0x7fd830070320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.661+0000 7fd82fe00640 1 -- 172.21.15.7:0/1439118425 >> v1:172.21.15.7:6789/0 conn(0x7fd8301490f0 legacy=0x7fd8300b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.661+0000 7fd82fe00640 1 -- 172.21.15.7:0/1439118425 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83006f560 con 0x7fd8300b08e0 2024-04-25T11:20:00.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.662+0000 7fd82fe00640 1 --2- 172.21.15.7:0/1439118425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8300b08e0 0x7fd830149500 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fd83006e0d0 tx=0x7fd82802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ecff093b0766b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.662+0000 7fd82e000640 1 -- 172.21.15.7:0/1439118425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828036070 con 0x7fd8300b08e0 2024-04-25T11:20:00.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.662+0000 7fd82e000640 1 -- 172.21.15.7:0/1439118425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd828031600 con 0x7fd8300b08e0 2024-04-25T11:20:00.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.662+0000 7fd82e000640 1 -- 172.21.15.7:0/1439118425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8280319d0 con 0x7fd8300b08e0 2024-04-25T11:20:00.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.663+0000 7fd835000640 1 -- 172.21.15.7:0/1439118425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8300b08e0 msgr2=0x7fd830149500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.663+0000 7fd835000640 1 --2- 172.21.15.7:0/1439118425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8300b08e0 0x7fd830149500 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fd83006e0d0 tx=0x7fd82802a730 comp rx=0 tx=0).stop 2024-04-25T11:20:00.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.663+0000 7fd835000640 1 -- 172.21.15.7:0/1439118425 shutdown_connections 2024-04-25T11:20:00.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.663+0000 7fd835000640 1 --2- 172.21.15.7:0/1439118425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8300b08e0 0x7fd830149500 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.663+0000 7fd835000640 1 --2- 172.21.15.7:0/1439118425 >> v2:172.21.15.87:3300/0 conn(0x7fd830146180 0x7fd830070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.663+0000 7fd835000640 1 -- 172.21.15.7:0/1439118425 >> 172.21.15.7:0/1439118425 conn(0x7fd8300740f0 msgr2=0x7fd83006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:00.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.663+0000 7fd835000640 1 -- 172.21.15.7:0/1439118425 shutdown_connections 2024-04-25T11:20:00.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.663+0000 7fd835000640 1 -- 172.21.15.7:0/1439118425 wait complete. 2024-04-25T11:20:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.666+0000 7fd835000640 1 Processor -- start 2024-04-25T11:20:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.666+0000 7fd835000640 1 -- start start 2024-04-25T11:20:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.667+0000 7fd835000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8300b08e0 0x7fd830160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.667+0000 7fd835000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830146180 0x7fd830135390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.667+0000 7fd835000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8301490f0 0x7fd8301f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.667+0000 7fd835000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd83006fe00 con 0x7fd830146180 2024-04-25T11:20:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.667+0000 7fd835000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd83006fb20 con 0x7fd8300b08e0 2024-04-25T11:20:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.667+0000 7fd835000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd83006eea0 con 0x7fd8301490f0 2024-04-25T11:20:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830146180 0x7fd830135390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830146180 0x7fd830135390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54360/0 (socket says 172.21.15.7:54360) 2024-04-25T11:20:00.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82ea00640 1 -- 172.21.15.7:0/2955076170 learned_addr learned my addr 172.21.15.7:0/2955076170 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82fe00640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8301490f0 0x7fd8301f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82f400640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8300b08e0 0x7fd830160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82ea00640 1 -- 172.21.15.7:0/2955076170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8301490f0 msgr2=0x7fd8301f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82ea00640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8301490f0 0x7fd8301f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82ea00640 1 -- 172.21.15.7:0/2955076170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8300b08e0 msgr2=0x7fd830160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82ea00640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8300b08e0 0x7fd830160450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82ea00640 1 -- 172.21.15.7:0/2955076170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd830160990 con 0x7fd830146180 2024-04-25T11:20:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82fe00640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8301490f0 0x7fd8301f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.668+0000 7fd82f400640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8300b08e0 0x7fd830160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:00.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.669+0000 7fd82ea00640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830146180 0x7fd830135390 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00c3f0 tx=0x7fd81c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.669+0000 7fd827400640 1 -- 172.21.15.7:0/2955076170 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0052d0 con 0x7fd830146180 2024-04-25T11:20:00.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.669+0000 7fd827400640 1 -- 172.21.15.7:0/2955076170 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd81c005490 con 0x7fd830146180 2024-04-25T11:20:00.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.669+0000 7fd827400640 1 -- 172.21.15.7:0/2955076170 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c007260 con 0x7fd830146180 2024-04-25T11:20:00.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.669+0000 7fd835000640 1 -- 172.21.15.7:0/2955076170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8301358d0 con 0x7fd830146180 2024-04-25T11:20:00.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.669+0000 7fd835000640 1 -- 172.21.15.7:0/2955076170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8301331e0 con 0x7fd830146180 2024-04-25T11:20:00.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.670+0000 7fd835000640 1 -- 172.21.15.7:0/2955076170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8300b0190 con 0x7fd830146180 2024-04-25T11:20:00.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.672+0000 7fd827400640 1 -- 172.21.15.7:0/2955076170 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd81c007400 con 0x7fd830146180 2024-04-25T11:20:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.673+0000 7fd827400640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8100744b0 0x7fd8100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.673+0000 7fd827400640 1 -- 172.21.15.7:0/2955076170 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fd81c094880 con 0x7fd830146180 2024-04-25T11:20:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.673+0000 7fd82f400640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8100744b0 0x7fd8100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:00.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.674+0000 7fd82f400640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8100744b0 0x7fd8100769c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd820006fd0 tx=0x7fd820008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:00.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.681+0000 7fd827400640 1 -- 172.21.15.7:0/2955076170 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8300b0190 con 0x7fd830146180 2024-04-25T11:20:00.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.890+0000 7fd835000640 1 -- 172.21.15.7:0/2955076170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7fd830050740 con 0x7fd830146180 2024-04-25T11:20:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.898+0000 7fd827400640 1 -- 172.21.15.7:0/2955076170 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v39)=0 v39) ==== 128+0+0 (secure 0 0 0) 0x7fd81c05ee70 con 0x7fd830146180 2024-04-25T11:20:00.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.901+0000 7fd835000640 1 -- 172.21.15.7:0/2955076170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8100744b0 msgr2=0x7fd8100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.901+0000 7fd835000640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8100744b0 0x7fd8100769c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd820006fd0 tx=0x7fd820008040 comp rx=0 tx=0).stop 2024-04-25T11:20:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.901+0000 7fd835000640 1 -- 172.21.15.7:0/2955076170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830146180 msgr2=0x7fd830135390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.901+0000 7fd835000640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830146180 0x7fd830135390 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00c3f0 tx=0x7fd81c00c8b0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.901+0000 7fd835000640 1 -- 172.21.15.7:0/2955076170 shutdown_connections 2024-04-25T11:20:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.901+0000 7fd835000640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8100744b0 0x7fd8100769c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.901+0000 7fd835000640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8301490f0 0x7fd8301f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.901+0000 7fd835000640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd830146180 0x7fd830135390 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.901+0000 7fd835000640 1 --2- 172.21.15.7:0/2955076170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8300b08e0 0x7fd830160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.901+0000 7fd835000640 1 -- 172.21.15.7:0/2955076170 >> 172.21.15.7:0/2955076170 conn(0x7fd8300740f0 msgr2=0x7fd8300d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.902+0000 7fd835000640 1 -- 172.21.15.7:0/2955076170 shutdown_connections 2024-04-25T11:20:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:00.902+0000 7fd835000640 1 -- 172.21.15.7:0/2955076170 wait complete. 2024-04-25T11:20:00.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-04-25T11:20:01.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.211+0000 7fd57f200640 1 Processor -- start 2024-04-25T11:20:01.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.211+0000 7fd57f200640 1 -- start start 2024-04-25T11:20:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.213+0000 7fd57f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd580145e40 0x7fd580146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.213+0000 7fd57f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580144a80 0x7fd5800b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.213+0000 7fd57f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5800b66b0 con 0x7fd5800b4870 2024-04-25T11:20:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.213+0000 7fd57f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd580145b70 con 0x7fd580145e40 2024-04-25T11:20:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.213+0000 7fd57f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd5800b58d0 con 0x7fd580144a80 2024-04-25T11:20:01.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.214+0000 7fd57d400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd5800b4870 0x7fd5800b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54774/0 (socket says 172.21.15.7:54774) 2024-04-25T11:20:01.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.214+0000 7fd57d400640 1 -- 172.21.15.7:0/1998178689 learned_addr learned my addr 172.21.15.7:0/1998178689 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.214+0000 7fd57de00640 1 --2- 172.21.15.7:0/1998178689 >> v2:172.21.15.87:3300/0 conn(0x7fd580145e40 0x7fd580146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.214+0000 7fd57e800640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580144a80 0x7fd5800b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.214+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2088911742 0 0) 0x7fd5800b66b0 con 0x7fd5800b4870 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd564003520 con 0x7fd5800b4870 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 917347991 0 0) 0x7fd564003520 con 0x7fd5800b4870 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580144a80 msgr2=0x7fd5800b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580144a80 0x7fd5800b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 >> v2:172.21.15.87:3300/0 conn(0x7fd580145e40 msgr2=0x7fd580146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 --2- 172.21.15.7:0/1998178689 >> v2:172.21.15.87:3300/0 conn(0x7fd580145e40 0x7fd580146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5800b5b30 con 0x7fd5800b4870 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fd570001690 con 0x7fd5800b4870 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 >> v1:172.21.15.7:6789/0 conn(0x7fd5800b4870 legacy=0x7fd5800b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5640039b0 0x7fd564003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580144a80 0x7fd564004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd564004d40 0x7fd564005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5800b58d0 con 0x7fd564004d40 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd56400ba70 con 0x7fd5640039b0 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd56400bc20 con 0x7fd580144a80 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.215+0000 7fd57e800640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd564004d40 0x7fd564005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.216+0000 7fd57de00640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5640039b0 0x7fd564003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.216+0000 7fd57d400640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580144a80 0x7fd564004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.216+0000 7fd57e800640 1 -- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580144a80 msgr2=0x7fd564004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.216+0000 7fd57e800640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580144a80 0x7fd564004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.216+0000 7fd57e800640 1 -- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5640039b0 msgr2=0x7fd564003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.216+0000 7fd57e800640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5640039b0 0x7fd564003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.216+0000 7fd57e800640 1 -- 172.21.15.7:0/1998178689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd564003520 con 0x7fd564004d40 2024-04-25T11:20:01.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.216+0000 7fd57d400640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580144a80 0x7fd564004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.217+0000 7fd57e800640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd564004d40 0x7fd564005150 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd568009ac0 tx=0x7fd564005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39554ae2b7fa8cba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.217+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568036070 con 0x7fd564004d40 2024-04-25T11:20:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.217+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd568031400 con 0x7fd564004d40 2024-04-25T11:20:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.218+0000 7fd57ca00640 1 -- 172.21.15.7:0/1998178689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568009690 con 0x7fd564004d40 2024-04-25T11:20:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.218+0000 7fd57f200640 1 -- 172.21.15.7:0/1998178689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd564004d40 msgr2=0x7fd564005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.218+0000 7fd57f200640 1 --2- 172.21.15.7:0/1998178689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd564004d40 0x7fd564005150 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd568009ac0 tx=0x7fd564005830 comp rx=0 tx=0).stop 2024-04-25T11:20:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.218+0000 7fd57e800640 1 -- 172.21.15.7:0/1998178689 reap_dead start 2024-04-25T11:20:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.218+0000 7fd57f200640 1 -- 172.21.15.7:0/1998178689 shutdown_connections 2024-04-25T11:20:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.218+0000 7fd57f200640 1 -- 172.21.15.7:0/1998178689 >> 172.21.15.7:0/1998178689 conn(0x7fd5800740f0 msgr2=0x7fd58006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.218+0000 7fd57f200640 1 -- 172.21.15.7:0/1998178689 shutdown_connections 2024-04-25T11:20:01.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.218+0000 7fd57f200640 1 -- 172.21.15.7:0/1998178689 wait complete. 2024-04-25T11:20:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.221+0000 7fd57f200640 1 Processor -- start 2024-04-25T11:20:01.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.221+0000 7fd57f200640 1 -- start start 2024-04-25T11:20:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.222+0000 7fd57f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564004d40 0x7fd58015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.222+0000 7fd57f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580141db0 0x7fd5801421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.222+0000 7fd57f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5801345a0 0x7fd5801349b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.222+0000 7fd57f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5800b63d0 con 0x7fd5801345a0 2024-04-25T11:20:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.222+0000 7fd57f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5800b60f0 con 0x7fd564004d40 2024-04-25T11:20:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.222+0000 7fd57f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd5800b5390 con 0x7fd580141db0 2024-04-25T11:20:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.223+0000 7fd57e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5801345a0 0x7fd5801349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.223+0000 7fd57e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5801345a0 0x7fd5801349b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54382/0 (socket says 172.21.15.7:54382) 2024-04-25T11:20:01.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.223+0000 7fd57e800640 1 -- 172.21.15.7:0/3899522882 learned_addr learned my addr 172.21.15.7:0/3899522882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.223+0000 7fd57d400640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580141db0 0x7fd5801421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.223+0000 7fd57de00640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564004d40 0x7fd58015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.223+0000 7fd57e800640 1 -- 172.21.15.7:0/3899522882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580141db0 msgr2=0x7fd5801421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.223+0000 7fd57e800640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580141db0 0x7fd5801421c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.223+0000 7fd57e800640 1 -- 172.21.15.7:0/3899522882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564004d40 msgr2=0x7fd58015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.223+0000 7fd57e800640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564004d40 0x7fd58015c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.223+0000 7fd57e800640 1 -- 172.21.15.7:0/3899522882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd580142700 con 0x7fd5801345a0 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.223+0000 7fd57e800640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5801345a0 0x7fd5801349b0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fd568009be0 tx=0x7fd568004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.224+0000 7fd576000640 1 -- 172.21.15.7:0/3899522882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568036040 con 0x7fd5801345a0 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.224+0000 7fd576000640 1 -- 172.21.15.7:0/3899522882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd568041070 con 0x7fd5801345a0 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.224+0000 7fd576000640 1 -- 172.21.15.7:0/3899522882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568009690 con 0x7fd5801345a0 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.224+0000 7fd57f200640 1 -- 172.21.15.7:0/3899522882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd580139670 con 0x7fd5801345a0 2024-04-25T11:20:01.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.224+0000 7fd57f200640 1 -- 172.21.15.7:0/3899522882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5801388b0 con 0x7fd5801345a0 2024-04-25T11:20:01.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.225+0000 7fd57f200640 1 -- 172.21.15.7:0/3899522882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd580050740 con 0x7fd5801345a0 2024-04-25T11:20:01.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.226+0000 7fd576000640 1 -- 172.21.15.7:0/3899522882 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd568003a50 con 0x7fd5801345a0 2024-04-25T11:20:01.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.228+0000 7fd576000640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd55c0744e0 0x7fd55c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.228+0000 7fd576000640 1 -- 172.21.15.7:0/3899522882 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fd5680b4cc0 con 0x7fd5801345a0 2024-04-25T11:20:01.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.228+0000 7fd57de00640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd55c0744e0 0x7fd55c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.229+0000 7fd57de00640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd55c0744e0 0x7fd55c0769f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd578007380 tx=0x7fd57800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:01.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.232+0000 7fd576000640 1 -- 172.21.15.7:0/3899522882 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd56804a020 con 0x7fd5801345a0 2024-04-25T11:20:01.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.436+0000 7fd57f200640 1 -- 172.21.15.7:0/3899522882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7fd5800703f0 con 0x7fd5801345a0 2024-04-25T11:20:01.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.437+0000 7fd576000640 1 -- 172.21.15.7:0/3899522882 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v39) ==== 106+0+4 (secure 0 0 0) 0x7fd580142700 con 0x7fd5801345a0 2024-04-25T11:20:01.436 INFO:teuthology.orchestra.run.smithi007.stdout:100 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 -- 172.21.15.7:0/3899522882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd55c0744e0 msgr2=0x7fd55c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd55c0744e0 0x7fd55c0769f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd578007380 tx=0x7fd57800b040 comp rx=0 tx=0).stop 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 -- 172.21.15.7:0/3899522882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5801345a0 msgr2=0x7fd5801349b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5801345a0 0x7fd5801349b0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fd568009be0 tx=0x7fd568004730 comp rx=0 tx=0).stop 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 -- 172.21.15.7:0/3899522882 shutdown_connections 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd55c0744e0 0x7fd55c0769f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5801345a0 0x7fd5801349b0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd580141db0 0x7fd5801421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 --2- 172.21.15.7:0/3899522882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd564004d40 0x7fd58015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 -- 172.21.15.7:0/3899522882 >> 172.21.15.7:0/3899522882 conn(0x7fd5800740f0 msgr2=0x7fd58006f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 -- 172.21.15.7:0/3899522882 shutdown_connections 2024-04-25T11:20:01.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.440+0000 7fd57f200640 1 -- 172.21.15.7:0/3899522882 wait complete. 2024-04-25T11:20:01.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-04-25T11:20:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.757+0000 7f3191400640 1 Processor -- start 2024-04-25T11:20:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.757+0000 7f3191400640 1 -- start start 2024-04-25T11:20:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.759+0000 7f3191400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f318c0b2550 0x7f318c0b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.759+0000 7f3191400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f318c0b2ff0 0x7f318c0b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.759+0000 7f3191400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f318c160770 con 0x7f318c0b2a80 2024-04-25T11:20:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.759+0000 7f3191400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f318c06e960 con 0x7f318c0b2550 2024-04-25T11:20:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.759+0000 7f3191400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f318c0d1dd0 con 0x7f318c0b2ff0 2024-04-25T11:20:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.759+0000 7f318be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f318c0b2550 0x7f318c0b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.759+0000 7f318b400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f318c0b2a80 0x7f318c0d28b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54778/0 (socket says 172.21.15.7:54778) 2024-04-25T11:20:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.759+0000 7f318be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f318c0b2550 0x7f318c0b8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54398/0 (socket says 172.21.15.7:54398) 2024-04-25T11:20:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.759+0000 7f318b400640 1 -- 172.21.15.7:0/2787051788 learned_addr learned my addr 172.21.15.7:0/2787051788 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.759+0000 7f3190a00640 1 --2- 172.21.15.7:0/2787051788 >> v2:172.21.15.87:3300/0 conn(0x7f318c0b2ff0 0x7f318c0b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.760+0000 7f318be00640 1 -- 172.21.15.7:0/2787051788 >> v2:172.21.15.87:3300/0 conn(0x7f318c0b2ff0 msgr2=0x7f318c0b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.760+0000 7f318be00640 1 --2- 172.21.15.7:0/2787051788 >> v2:172.21.15.87:3300/0 conn(0x7f318c0b2ff0 0x7f318c0b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.760+0000 7f318be00640 1 -- 172.21.15.7:0/2787051788 >> v1:172.21.15.7:6789/0 conn(0x7f318c0b2a80 legacy=0x7f318c0d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.760+0000 7f318be00640 1 -- 172.21.15.7:0/2787051788 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c0d20b0 con 0x7f318c0b2550 2024-04-25T11:20:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.761+0000 7f318be00640 1 --2- 172.21.15.7:0/2787051788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f318c0b2550 0x7f318c0b8eb0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f318c08cec0 tx=0x7f317c02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6754a6e0325a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.761+0000 7f318aa00640 1 -- 172.21.15.7:0/2787051788 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c037070 con 0x7f318c0b2550 2024-04-25T11:20:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.761+0000 7f318aa00640 1 -- 172.21.15.7:0/2787051788 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317c02add0 con 0x7f318c0b2550 2024-04-25T11:20:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.761+0000 7f318aa00640 1 -- 172.21.15.7:0/2787051788 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c032660 con 0x7f318c0b2550 2024-04-25T11:20:01.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.762+0000 7f3191400640 1 -- 172.21.15.7:0/2787051788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f318c0b2550 msgr2=0x7f318c0b8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.762+0000 7f3191400640 1 --2- 172.21.15.7:0/2787051788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f318c0b2550 0x7f318c0b8eb0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f318c08cec0 tx=0x7f317c02a080 comp rx=0 tx=0).stop 2024-04-25T11:20:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.762+0000 7f3191400640 1 -- 172.21.15.7:0/2787051788 shutdown_connections 2024-04-25T11:20:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.762+0000 7f3191400640 1 --2- 172.21.15.7:0/2787051788 >> v2:172.21.15.87:3300/0 conn(0x7f318c0b2ff0 0x7f318c0b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.762+0000 7f3191400640 1 --2- 172.21.15.7:0/2787051788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f318c0b2550 0x7f318c0b8eb0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.762+0000 7f3191400640 1 -- 172.21.15.7:0/2787051788 >> 172.21.15.7:0/2787051788 conn(0x7f318c0b5240 msgr2=0x7f318c0b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.762+0000 7f3191400640 1 -- 172.21.15.7:0/2787051788 shutdown_connections 2024-04-25T11:20:01.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.762+0000 7f3191400640 1 -- 172.21.15.7:0/2787051788 wait complete. 2024-04-25T11:20:01.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.765+0000 7f3191400640 1 Processor -- start 2024-04-25T11:20:01.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.765+0000 7f3191400640 1 -- start start 2024-04-25T11:20:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f3191400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f318c0b2550 0x7f318c0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f3191400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f318c0b2a80 0x7f318c150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f3191400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f318c0b2ff0 0x7f318c1eb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f3191400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f318c06e5f0 con 0x7f318c0b2a80 2024-04-25T11:20:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f3191400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f318c0d1ac0 con 0x7f318c0b2550 2024-04-25T11:20:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f3191400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f318c0d2fc0 con 0x7f318c0b2ff0 2024-04-25T11:20:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f318b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f318c0b2a80 0x7f318c150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f318b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f318c0b2a80 0x7f318c150db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54406/0 (socket says 172.21.15.7:54406) 2024-04-25T11:20:01.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f318b400640 1 -- 172.21.15.7:0/3773058135 learned_addr learned my addr 172.21.15.7:0/3773058135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:01.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f318be00640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f318c0b2550 0x7f318c0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f318b400640 1 -- 172.21.15.7:0/3773058135 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f318c0b2ff0 msgr2=0x7f318c1eb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f318b400640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f318c0b2ff0 0x7f318c1eb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f318b400640 1 -- 172.21.15.7:0/3773058135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f318c0b2550 msgr2=0x7f318c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:01.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.767+0000 7f318b400640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f318c0b2550 0x7f318c0b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:01.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.768+0000 7f318b400640 1 -- 172.21.15.7:0/3773058135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c1512f0 con 0x7f318c0b2a80 2024-04-25T11:20:01.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.768+0000 7f318be00640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f318c0b2550 0x7f318c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:01.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.768+0000 7f318b400640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f318c0b2a80 0x7f318c150db0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f317800c470 tx=0x7f317800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:01.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.768+0000 7f3183e00640 1 -- 172.21.15.7:0/3773058135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178017070 con 0x7f318c0b2a80 2024-04-25T11:20:01.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.768+0000 7f3183e00640 1 -- 172.21.15.7:0/3773058135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3178005ce0 con 0x7f318c0b2a80 2024-04-25T11:20:01.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.768+0000 7f3183e00640 1 -- 172.21.15.7:0/3773058135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31780072d0 con 0x7f318c0b2a80 2024-04-25T11:20:01.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.768+0000 7f3191400640 1 -- 172.21.15.7:0/3773058135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c0b6cb0 con 0x7f318c0b2a80 2024-04-25T11:20:01.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.768+0000 7f3191400640 1 -- 172.21.15.7:0/3773058135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c0b7270 con 0x7f318c0b2a80 2024-04-25T11:20:01.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.769+0000 7f3191400640 1 -- 172.21.15.7:0/3773058135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318c050740 con 0x7f318c0b2a80 2024-04-25T11:20:01.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.771+0000 7f3183e00640 1 -- 172.21.15.7:0/3773058135 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f317801a070 con 0x7f318c0b2a80 2024-04-25T11:20:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.772+0000 7f3183e00640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f316c0744e0 0x7f316c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:01.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.772+0000 7f3183e00640 1 -- 172.21.15.7:0/3773058135 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f31780945c0 con 0x7f318c0b2a80 2024-04-25T11:20:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.772+0000 7f318be00640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f316c0744e0 0x7f316c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:01.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.773+0000 7f318be00640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f316c0744e0 0x7f316c0769f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f317c034000 tx=0x7f317c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:01.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.777+0000 7f3183e00640 1 -- 172.21.15.7:0/3773058135 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f317805e560 con 0x7f318c0b2a80 2024-04-25T11:20:01.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:01.992+0000 7f3191400640 1 -- 172.21.15.7:0/3773058135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f318c0b59d0 con 0x7f318c0b2a80 2024-04-25T11:20:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.000+0000 7f3183e00640 1 -- 172.21.15.7:0/3773058135 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v40)=0 v40) ==== 124+0+0 (secure 0 0 0) 0x7f3178062860 con 0x7f318c0b2a80 2024-04-25T11:20:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.003+0000 7f3191400640 1 -- 172.21.15.7:0/3773058135 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f316c0744e0 msgr2=0x7f316c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.003+0000 7f3191400640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f316c0744e0 0x7f316c0769f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f317c034000 tx=0x7f317c02e000 comp rx=0 tx=0).stop 2024-04-25T11:20:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.003+0000 7f3191400640 1 -- 172.21.15.7:0/3773058135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f318c0b2a80 msgr2=0x7f318c150db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.003+0000 7f3191400640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f318c0b2a80 0x7f318c150db0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f317800c470 tx=0x7f317800c930 comp rx=0 tx=0).stop 2024-04-25T11:20:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.003+0000 7f3191400640 1 -- 172.21.15.7:0/3773058135 shutdown_connections 2024-04-25T11:20:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.003+0000 7f3191400640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f316c0744e0 0x7f316c0769f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.003+0000 7f3191400640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f318c0b2ff0 0x7f318c1eb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.003+0000 7f3191400640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f318c0b2a80 0x7f318c150db0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.003+0000 7f3191400640 1 --2- 172.21.15.7:0/3773058135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f318c0b2550 0x7f318c0b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.003+0000 7f3191400640 1 -- 172.21.15.7:0/3773058135 >> 172.21.15.7:0/3773058135 conn(0x7f318c0b5240 msgr2=0x7f318c06d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.004+0000 7f3191400640 1 -- 172.21.15.7:0/3773058135 shutdown_connections 2024-04-25T11:20:02.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.004+0000 7f3191400640 1 -- 172.21.15.7:0/3773058135 wait complete. 2024-04-25T11:20:02.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-04-25T11:20:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.316+0000 7f333a000640 1 Processor -- start 2024-04-25T11:20:02.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.316+0000 7f333a000640 1 -- start start 2024-04-25T11:20:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.318+0000 7f333a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f33341583c0 0x7f33340cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.318+0000 7f333a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f33340cfb50 0x7f33340d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.318+0000 7f333a000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33340ce570 con 0x7f33340d09c0 2024-04-25T11:20:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.318+0000 7f333a000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f333415f190 con 0x7f33341583c0 2024-04-25T11:20:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.318+0000 7f333a000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f33340ce850 con 0x7f33340cfb50 2024-04-25T11:20:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.319+0000 7f3338c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f33341583c0 0x7f33340cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.319+0000 7f3339600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f33340d09c0 0x7f333406eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54790/0 (socket says 172.21.15.7:54790) 2024-04-25T11:20:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.319+0000 7f3339600640 1 -- 172.21.15.7:0/3715313503 learned_addr learned my addr 172.21.15.7:0/3715313503 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:02.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.319+0000 7f3333e00640 1 --2- 172.21.15.7:0/3715313503 >> v2:172.21.15.87:3300/0 conn(0x7f33340cfb50 0x7f33340d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.319+0000 7f3338c00640 1 -- 172.21.15.7:0/3715313503 >> v2:172.21.15.87:3300/0 conn(0x7f33340cfb50 msgr2=0x7f33340d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:02.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.319+0000 7f3338c00640 1 --2- 172.21.15.7:0/3715313503 >> v2:172.21.15.87:3300/0 conn(0x7f33340cfb50 0x7f33340d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.319+0000 7f3338c00640 1 -- 172.21.15.7:0/3715313503 >> v1:172.21.15.7:6789/0 conn(0x7f33340d09c0 legacy=0x7f333406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.319+0000 7f3333400640 1 -- 172.21.15.7:0/3715313503 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 949626858 0 0) 0x7f33340ce570 con 0x7f33340d09c0 2024-04-25T11:20:02.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.319+0000 7f3338c00640 1 -- 172.21.15.7:0/3715313503 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33340ceb30 con 0x7f33341583c0 2024-04-25T11:20:02.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.319+0000 7f3333e00640 1 --2- 172.21.15.7:0/3715313503 >> v2:172.21.15.87:3300/0 conn(0x7f33340cfb50 0x7f33340d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:02.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.320+0000 7f3338c00640 1 --2- 172.21.15.7:0/3715313503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33341583c0 0x7f33340cf610 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3334070ad0 tx=0x7f332802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d4f18b332b5d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.321+0000 7f3333400640 1 -- 172.21.15.7:0/3715313503 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328037070 con 0x7f33341583c0 2024-04-25T11:20:02.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.321+0000 7f3333400640 1 -- 172.21.15.7:0/3715313503 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3328032440 con 0x7f33341583c0 2024-04-25T11:20:02.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.321+0000 7f3333400640 1 -- 172.21.15.7:0/3715313503 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332802ae60 con 0x7f33341583c0 2024-04-25T11:20:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.321+0000 7f333a000640 1 -- 172.21.15.7:0/3715313503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33341583c0 msgr2=0x7f33340cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.321+0000 7f333a000640 1 --2- 172.21.15.7:0/3715313503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33341583c0 0x7f33340cf610 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3334070ad0 tx=0x7f332802a110 comp rx=0 tx=0).stop 2024-04-25T11:20:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.322+0000 7f333a000640 1 -- 172.21.15.7:0/3715313503 shutdown_connections 2024-04-25T11:20:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.322+0000 7f333a000640 1 --2- 172.21.15.7:0/3715313503 >> v2:172.21.15.87:3300/0 conn(0x7f33340cfb50 0x7f33340d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.322+0000 7f333a000640 1 --2- 172.21.15.7:0/3715313503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33341583c0 0x7f33340cf610 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.322+0000 7f333a000640 1 -- 172.21.15.7:0/3715313503 >> 172.21.15.7:0/3715313503 conn(0x7f33340740f0 msgr2=0x7f333406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.322+0000 7f333a000640 1 -- 172.21.15.7:0/3715313503 shutdown_connections 2024-04-25T11:20:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.322+0000 7f333a000640 1 -- 172.21.15.7:0/3715313503 wait complete. 2024-04-25T11:20:02.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.324+0000 7f333a000640 1 Processor -- start 2024-04-25T11:20:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.324+0000 7f333a000640 1 -- start start 2024-04-25T11:20:02.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.326+0000 7f333a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33340cfb50 0x7f3334143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.326+0000 7f333a000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f33340d09c0 0x7f3334152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.326+0000 7f333a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33341583c0 0x7f33340b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.326+0000 7f333a000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f333406f5b0 con 0x7f33340cfb50 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.326+0000 7f333a000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f33340cf0f0 con 0x7f33341583c0 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.326+0000 7f333a000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f333406f730 con 0x7f33340d09c0 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.326+0000 7f3338c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33340cfb50 0x7f3334143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.326+0000 7f3338c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33340cfb50 0x7f3334143f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54418/0 (socket says 172.21.15.7:54418) 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.326+0000 7f3338c00640 1 -- 172.21.15.7:0/382711661 learned_addr learned my addr 172.21.15.7:0/382711661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.327+0000 7f3333e00640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f33340d09c0 0x7f3334152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.327+0000 7f3339600640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33341583c0 0x7f33340b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.327+0000 7f3338c00640 1 -- 172.21.15.7:0/382711661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f33340d09c0 msgr2=0x7f3334152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.327+0000 7f3338c00640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f33340d09c0 0x7f3334152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.327+0000 7f3338c00640 1 -- 172.21.15.7:0/382711661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33341583c0 msgr2=0x7f33340b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.327+0000 7f3338c00640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33341583c0 0x7f33340b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.327+0000 7f3338c00640 1 -- 172.21.15.7:0/382711661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33341e8020 con 0x7f33340cfb50 2024-04-25T11:20:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.327+0000 7f3333e00640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f33340d09c0 0x7f3334152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.327+0000 7f3339600640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33341583c0 0x7f33340b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.327+0000 7f3338c00640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33340cfb50 0x7f3334143f20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f3328009890 tx=0x7f33280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.328+0000 7f3330c00640 1 -- 172.21.15.7:0/382711661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328037050 con 0x7f33340cfb50 2024-04-25T11:20:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.328+0000 7f3330c00640 1 -- 172.21.15.7:0/382711661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3328038040 con 0x7f33340cfb50 2024-04-25T11:20:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.328+0000 7f3330c00640 1 -- 172.21.15.7:0/382711661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328004470 con 0x7f33340cfb50 2024-04-25T11:20:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.328+0000 7f333a000640 1 -- 172.21.15.7:0/382711661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33341e8300 con 0x7f33340cfb50 2024-04-25T11:20:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.328+0000 7f333a000640 1 -- 172.21.15.7:0/382711661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33341e88c0 con 0x7f33340cfb50 2024-04-25T11:20:02.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.329+0000 7f333a000640 1 -- 172.21.15.7:0/382711661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3334050740 con 0x7f33340cfb50 2024-04-25T11:20:02.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.331+0000 7f3330c00640 1 -- 172.21.15.7:0/382711661 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3328030980 con 0x7f33340cfb50 2024-04-25T11:20:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.332+0000 7f3330c00640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33140744e0 0x7f33140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.332+0000 7f3330c00640 1 -- 172.21.15.7:0/382711661 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f33280b5fe0 con 0x7f33340cfb50 2024-04-25T11:20:02.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.332+0000 7f3333e00640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33140744e0 0x7f33140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.333+0000 7f3333e00640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33140744e0 0x7f33140769f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3320006fd0 tx=0x7f3320008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.336+0000 7f3330c00640 1 -- 172.21.15.7:0/382711661 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3328080580 con 0x7f33340cfb50 2024-04-25T11:20:02.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.552+0000 7f333a000640 1 -- 172.21.15.7:0/382711661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7f3334070dd0 con 0x7f33340cfb50 2024-04-25T11:20:02.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.553+0000 7f3330c00640 1 -- 172.21.15.7:0/382711661 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v40) ==== 106+0+8 (secure 0 0 0) 0x7f332807fe80 con 0x7f33340cfb50 2024-04-25T11:20:02.551 INFO:teuthology.orchestra.run.smithi007.stdout:1048576 2024-04-25T11:20:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 -- 172.21.15.7:0/382711661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33140744e0 msgr2=0x7f33140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33140744e0 0x7f33140769f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3320006fd0 tx=0x7f3320008040 comp rx=0 tx=0).stop 2024-04-25T11:20:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 -- 172.21.15.7:0/382711661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33340cfb50 msgr2=0x7f3334143f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33340cfb50 0x7f3334143f20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f3328009890 tx=0x7f33280046c0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 -- 172.21.15.7:0/382711661 shutdown_connections 2024-04-25T11:20:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f33140744e0 0x7f33140769f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33341583c0 0x7f33340b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f33340d09c0 0x7f3334152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 --2- 172.21.15.7:0/382711661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33340cfb50 0x7f3334143f20 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 -- 172.21.15.7:0/382711661 >> 172.21.15.7:0/382711661 conn(0x7f33340740f0 msgr2=0x7f3334161320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 -- 172.21.15.7:0/382711661 shutdown_connections 2024-04-25T11:20:02.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.556+0000 7f333a000640 1 -- 172.21.15.7:0/382711661 wait complete. 2024-04-25T11:20:02.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-04-25T11:20:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.869+0000 7f5c11e00640 1 Processor -- start 2024-04-25T11:20:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.869+0000 7f5c11e00640 1 -- start start 2024-04-25T11:20:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.872+0000 7f5c11e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5c0c15dfb0 0x7f5c0c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.872+0000 7f5c11e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0c06fab0 0x7f5c0c0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.872+0000 7f5c11e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c0c15e3c0 con 0x7f5c0c0d0b40 2024-04-25T11:20:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.872+0000 7f5c11e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5c0c0b09b0 con 0x7f5c0c15dfb0 2024-04-25T11:20:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.872+0000 7f5c11e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0c0cf800 con 0x7f5c0c06fab0 2024-04-25T11:20:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.872+0000 7f5c0aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5c0c15dfb0 0x7f5c0c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.872+0000 7f5c0aa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5c0c15dfb0 0x7f5c0c0b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54428/0 (socket says 172.21.15.7:54428) 2024-04-25T11:20:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.872+0000 7f5c0aa00640 1 -- 172.21.15.7:0/1790132847 learned_addr learned my addr 172.21.15.7:0/1790132847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.872+0000 7f5c0b400640 1 --2- 172.21.15.7:0/1790132847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0c06fab0 0x7f5c0c0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.873+0000 7f5c0aa00640 1 -- 172.21.15.7:0/1790132847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0c06fab0 msgr2=0x7f5c0c0aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.873+0000 7f5c0aa00640 1 --2- 172.21.15.7:0/1790132847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0c06fab0 0x7f5c0c0aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.873+0000 7f5c0aa00640 1 -- 172.21.15.7:0/1790132847 >> v1:172.21.15.87:6789/0 conn(0x7f5c0c0d0b40 legacy=0x7f5c0c06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.873+0000 7f5c0aa00640 1 -- 172.21.15.7:0/1790132847 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0c0cfaa0 con 0x7f5c0c15dfb0 2024-04-25T11:20:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.874+0000 7f5c0aa00640 1 --2- 172.21.15.7:0/1790132847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c15dfb0 0x7f5c0c0b0470 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009870 tx=0x7f5bfc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafb1ff08df32099 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.874+0000 7f5c09600640 1 -- 172.21.15.7:0/1790132847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc037070 con 0x7f5c0c15dfb0 2024-04-25T11:20:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.874+0000 7f5c09600640 1 -- 172.21.15.7:0/1790132847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bfc032440 con 0x7f5c0c15dfb0 2024-04-25T11:20:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.874+0000 7f5c09600640 1 -- 172.21.15.7:0/1790132847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc032810 con 0x7f5c0c15dfb0 2024-04-25T11:20:02.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.875+0000 7f5c11e00640 1 -- 172.21.15.7:0/1790132847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c15dfb0 msgr2=0x7f5c0c0b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.875+0000 7f5c11e00640 1 --2- 172.21.15.7:0/1790132847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c15dfb0 0x7f5c0c0b0470 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009870 tx=0x7f5bfc02a270 comp rx=0 tx=0).stop 2024-04-25T11:20:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.875+0000 7f5c11e00640 1 -- 172.21.15.7:0/1790132847 shutdown_connections 2024-04-25T11:20:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.875+0000 7f5c11e00640 1 --2- 172.21.15.7:0/1790132847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0c06fab0 0x7f5c0c0aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.875+0000 7f5c11e00640 1 --2- 172.21.15.7:0/1790132847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c15dfb0 0x7f5c0c0b0470 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.875+0000 7f5c11e00640 1 -- 172.21.15.7:0/1790132847 >> 172.21.15.7:0/1790132847 conn(0x7f5c0c0742c0 msgr2=0x7f5c0c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.875+0000 7f5c11e00640 1 -- 172.21.15.7:0/1790132847 shutdown_connections 2024-04-25T11:20:02.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.875+0000 7f5c11e00640 1 -- 172.21.15.7:0/1790132847 wait complete. 2024-04-25T11:20:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.878+0000 7f5c11e00640 1 Processor -- start 2024-04-25T11:20:02.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.878+0000 7f5c11e00640 1 -- start start 2024-04-25T11:20:02.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.879+0000 7f5c11e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c0c06fab0 0x7f5c0c152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.879+0000 7f5c11e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0d0b40 0x7f5c0c1601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.879+0000 7f5c11e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0c15dfb0 0x7f5c0c15c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.879+0000 7f5c11e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c0d0340 con 0x7f5c0c0d0b40 2024-04-25T11:20:02.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.879+0000 7f5c11e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c0d0060 con 0x7f5c0c06fab0 2024-04-25T11:20:02.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.879+0000 7f5c11e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0c0d0620 con 0x7f5c0c15dfb0 2024-04-25T11:20:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.880+0000 7f5c0a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0d0b40 0x7f5c0c1601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.880+0000 7f5c0a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0d0b40 0x7f5c0c1601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54434/0 (socket says 172.21.15.7:54434) 2024-04-25T11:20:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.880+0000 7f5c0a000640 1 -- 172.21.15.7:0/2781565325 learned_addr learned my addr 172.21.15.7:0/2781565325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.880+0000 7f5c0aa00640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c0c06fab0 0x7f5c0c152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.880+0000 7f5c0b400640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0c15dfb0 0x7f5c0c15c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.880+0000 7f5c0a000640 1 -- 172.21.15.7:0/2781565325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0c15dfb0 msgr2=0x7f5c0c15c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.880+0000 7f5c0a000640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0c15dfb0 0x7f5c0c15c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.880+0000 7f5c0a000640 1 -- 172.21.15.7:0/2781565325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c0c06fab0 msgr2=0x7f5c0c152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.880+0000 7f5c0a000640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c0c06fab0 0x7f5c0c152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.880+0000 7f5c0a000640 1 -- 172.21.15.7:0/2781565325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0c158260 con 0x7f5c0c0d0b40 2024-04-25T11:20:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.881+0000 7f5c0a000640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0d0b40 0x7f5c0c1601a0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800c3f0 tx=0x7f5bf800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.881+0000 7f5c03400640 1 -- 172.21.15.7:0/2781565325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80052d0 con 0x7f5c0c0d0b40 2024-04-25T11:20:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.881+0000 7f5c03400640 1 -- 172.21.15.7:0/2781565325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf8005490 con 0x7f5c0c0d0b40 2024-04-25T11:20:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.881+0000 7f5c03400640 1 -- 172.21.15.7:0/2781565325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8007260 con 0x7f5c0c0d0b40 2024-04-25T11:20:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.881+0000 7f5c11e00640 1 -- 172.21.15.7:0/2781565325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0c0cac90 con 0x7f5c0c0d0b40 2024-04-25T11:20:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.881+0000 7f5c11e00640 1 -- 172.21.15.7:0/2781565325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0c0cb3e0 con 0x7f5c0c0d0b40 2024-04-25T11:20:02.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.882+0000 7f5c11e00640 1 -- 172.21.15.7:0/2781565325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c0c050740 con 0x7f5c0c0d0b40 2024-04-25T11:20:02.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.884+0000 7f5c03400640 1 -- 172.21.15.7:0/2781565325 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5bf8007400 con 0x7f5c0c0d0b40 2024-04-25T11:20:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.885+0000 7f5c03400640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5bec0744e0 0x7f5bec0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.885+0000 7f5c03400640 1 -- 172.21.15.7:0/2781565325 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f5bf80947f0 con 0x7f5c0c0d0b40 2024-04-25T11:20:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.885+0000 7f5c0aa00640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5bec0744e0 0x7f5bec0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.886+0000 7f5c0aa00640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5bec0744e0 0x7f5bec0769f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc034000 tx=0x7f5bfc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:02.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:02.891+0000 7f5c03400640 1 -- 172.21.15.7:0/2781565325 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5bf805ede0 con 0x7f5c0c0d0b40 2024-04-25T11:20:03.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.095+0000 7f5c11e00640 1 -- 172.21.15.7:0/2781565325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f5c0c0cae60 con 0x7f5c0c0d0b40 2024-04-25T11:20:03.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.096+0000 7f5c03400640 1 -- 172.21.15.7:0/2781565325 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v40) ==== 112+0+10 (secure 0 0 0) 0x7f5bf805e6e0 con 0x7f5c0c0d0b40 2024-04-25T11:20:03.095 INFO:teuthology.orchestra.run.smithi007.stdout:60.000000 2024-04-25T11:20:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.098+0000 7f5c11e00640 1 -- 172.21.15.7:0/2781565325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5bec0744e0 msgr2=0x7f5bec0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.098+0000 7f5c11e00640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5bec0744e0 0x7f5bec0769f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc034000 tx=0x7f5bfc02e000 comp rx=0 tx=0).stop 2024-04-25T11:20:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.098+0000 7f5c11e00640 1 -- 172.21.15.7:0/2781565325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0d0b40 msgr2=0x7f5c0c1601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.098+0000 7f5c11e00640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0d0b40 0x7f5c0c1601a0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800c3f0 tx=0x7f5bf800c7f0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.099+0000 7f5c11e00640 1 -- 172.21.15.7:0/2781565325 shutdown_connections 2024-04-25T11:20:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.099+0000 7f5c11e00640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5bec0744e0 0x7f5bec0769f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.099+0000 7f5c11e00640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0c15dfb0 0x7f5c0c15c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.099+0000 7f5c11e00640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0d0b40 0x7f5c0c1601a0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.099+0000 7f5c11e00640 1 --2- 172.21.15.7:0/2781565325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c0c06fab0 0x7f5c0c152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.099+0000 7f5c11e00640 1 -- 172.21.15.7:0/2781565325 >> 172.21.15.7:0/2781565325 conn(0x7f5c0c0742c0 msgr2=0x7f5c0c0ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:03.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.099+0000 7f5c11e00640 1 -- 172.21.15.7:0/2781565325 shutdown_connections 2024-04-25T11:20:03.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.099+0000 7f5c11e00640 1 -- 172.21.15.7:0/2781565325 wait complete. 2024-04-25T11:20:03.127 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:20:03.127 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:03.180 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:20:03.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:20:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:03.303 INFO:teuthology.orchestra.run.smithi007.stdout:client8157 2024-04-25T11:20:03.304 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:20:03.304 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:03.354 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-04-25T11:20:03.354 DEBUG:teuthology.orchestra.run.smithi007:> import fcntl 2024-04-25T11:20:03.354 DEBUG:teuthology.orchestra.run.smithi007:> import sys 2024-04-25T11:20:03.354 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-04-25T11:20:03.354 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-04-25T11:20:03.354 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:20:03.354 DEBUG:teuthology.orchestra.run.smithi007:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-25T11:20:03.354 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> n = 800 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> if not os.path.exists(abs_path): 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(abs_path) 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> handles = [] 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(0, n): 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> fname = "file_"+str(i) 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> path = os.path.join(abs_path, fname) 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> handles.append(open(path, '"'"'w'"'"')) 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> print("waiting with handles open", file=sys.stderr) 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> print("open_n_background: keeping files open", file=sys.stderr) 2024-04-25T11:20:03.355 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-04-25T11:20:03.356 DEBUG:teuthology.orchestra.run.smithi007:> if os.read(0, 4096) == b"": 2024-04-25T11:20:03.356 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-04-25T11:20:03.356 DEBUG:teuthology.orchestra.run.smithi007:> except BlockingIOError: 2024-04-25T11:20:03.356 DEBUG:teuthology.orchestra.run.smithi007:> pass 2024-04-25T11:20:03.356 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(2) 2024-04-25T11:20:03.356 DEBUG:teuthology.orchestra.run.smithi007:> print("stdin closed, goodbye!", file=sys.stderr) 2024-04-25T11:20:03.356 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-04-25T11:20:03.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.626+0000 7f07e2000640 1 Processor -- start 2024-04-25T11:20:03.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.626+0000 7f07e2000640 1 -- start start 2024-04-25T11:20:03.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.628+0000 7f07e2000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07dc0b3920 0x7f07dc0b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.628+0000 7f07e2000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f07dc0b3d30 0x7f07dc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.628+0000 7f07e2000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f07dc0d2520 0x7f07dc0d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.628+0000 7f07e2000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f07dc0cf290 con 0x7f07dc0b3d30 2024-04-25T11:20:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.628+0000 7f07e2000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f07dc0cf570 con 0x7f07dc0d2520 2024-04-25T11:20:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.628+0000 7f07e2000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc0cf850 con 0x7f07dc0b3920 2024-04-25T11:20:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.628+0000 7f07dbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f07dc0b3d30 0x7f07dc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.628+0000 7f07dbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f07dc0b3d30 0x7f07dc0b7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54440/0 (socket says 172.21.15.7:54440) 2024-04-25T11:20:03.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.628+0000 7f07dbe00640 1 -- 172.21.15.7:0/2574073681 learned_addr learned my addr 172.21.15.7:0/2574073681 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:03.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.628+0000 7f07e0c00640 1 --2- 172.21.15.7:0/2574073681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07dc0b3920 0x7f07dc0b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.629+0000 7f07dbe00640 1 -- 172.21.15.7:0/2574073681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07dc0b3920 msgr2=0x7f07dc0b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:03.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.629+0000 7f07e1600640 1 --2- 172.21.15.7:0/2574073681 >> v2:172.21.15.87:3300/0 conn(0x7f07dc0d2520 0x7f07dc0d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.629+0000 7f07dbe00640 1 --2- 172.21.15.7:0/2574073681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07dc0b3920 0x7f07dc0b73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.629+0000 7f07dbe00640 1 -- 172.21.15.7:0/2574073681 >> v2:172.21.15.87:3300/0 conn(0x7f07dc0d2520 msgr2=0x7f07dc0d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:03.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.629+0000 7f07dbe00640 1 --2- 172.21.15.7:0/2574073681 >> v2:172.21.15.87:3300/0 conn(0x7f07dc0d2520 0x7f07dc0d2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.629+0000 7f07dbe00640 1 -- 172.21.15.7:0/2574073681 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc0cfb30 con 0x7f07dc0b3d30 2024-04-25T11:20:03.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.629+0000 7f07e0c00640 1 --2- 172.21.15.7:0/2574073681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07dc0b3920 0x7f07dc0b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:03.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.630+0000 7f07dbe00640 1 --2- 172.21.15.7:0/2574073681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07dc0b3d30 0x7f07dc0b7910 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f07dc073990 tx=0x7f07c802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138fe36bc8f856f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.630+0000 7f07db400640 1 -- 172.21.15.7:0/2574073681 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8037070 con 0x7f07dc0b3d30 2024-04-25T11:20:03.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.630+0000 7f07db400640 1 -- 172.21.15.7:0/2574073681 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07c8032440 con 0x7f07dc0b3d30 2024-04-25T11:20:03.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.630+0000 7f07db400640 1 -- 172.21.15.7:0/2574073681 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8032810 con 0x7f07dc0b3d30 2024-04-25T11:20:03.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.631+0000 7f07e2000640 1 -- 172.21.15.7:0/2574073681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07dc0b3d30 msgr2=0x7f07dc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.631+0000 7f07e2000640 1 --2- 172.21.15.7:0/2574073681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07dc0b3d30 0x7f07dc0b7910 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f07dc073990 tx=0x7f07c802a140 comp rx=0 tx=0).stop 2024-04-25T11:20:03.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.631+0000 7f07e2000640 1 -- 172.21.15.7:0/2574073681 shutdown_connections 2024-04-25T11:20:03.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.631+0000 7f07e2000640 1 --2- 172.21.15.7:0/2574073681 >> v2:172.21.15.87:3300/0 conn(0x7f07dc0d2520 0x7f07dc0d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.631+0000 7f07e2000640 1 --2- 172.21.15.7:0/2574073681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07dc0b3d30 0x7f07dc0b7910 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.631+0000 7f07e2000640 1 --2- 172.21.15.7:0/2574073681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07dc0b3920 0x7f07dc0b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.631+0000 7f07e2000640 1 -- 172.21.15.7:0/2574073681 >> 172.21.15.7:0/2574073681 conn(0x7f07dc09b540 msgr2=0x7f07dc09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:03.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.631+0000 7f07e2000640 1 -- 172.21.15.7:0/2574073681 shutdown_connections 2024-04-25T11:20:03.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.631+0000 7f07e2000640 1 -- 172.21.15.7:0/2574073681 wait complete. 2024-04-25T11:20:03.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.633+0000 7f07e2000640 1 Processor -- start 2024-04-25T11:20:03.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.633+0000 7f07e2000640 1 -- start start 2024-04-25T11:20:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.635+0000 7f07e2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07dc0b3920 0x7f07dc156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.635+0000 7f07e2000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07dc0b3d30 0x7f07dc0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.635+0000 7f07e2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07dc0d2520 0x7f07dc0c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.635+0000 7f07e2000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc0d2e70 con 0x7f07dc0b3920 2024-04-25T11:20:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.635+0000 7f07e2000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc0cee60 con 0x7f07dc0d2520 2024-04-25T11:20:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.635+0000 7f07e2000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc0d2ff0 con 0x7f07dc0b3d30 2024-04-25T11:20:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.635+0000 7f07e0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07dc0b3920 0x7f07dc156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.635+0000 7f07e0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07dc0b3920 0x7f07dc156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54454/0 (socket says 172.21.15.7:54454) 2024-04-25T11:20:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.635+0000 7f07e0c00640 1 -- 172.21.15.7:0/3986178194 learned_addr learned my addr 172.21.15.7:0/3986178194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.636+0000 7f07dbe00640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07dc0b3d30 0x7f07dc0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.636+0000 7f07e1600640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07dc0d2520 0x7f07dc0c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.636+0000 7f07e0c00640 1 -- 172.21.15.7:0/3986178194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07dc0b3d30 msgr2=0x7f07dc0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.636+0000 7f07e0c00640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07dc0b3d30 0x7f07dc0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.636+0000 7f07e0c00640 1 -- 172.21.15.7:0/3986178194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07dc0d2520 msgr2=0x7f07dc0c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.636+0000 7f07e0c00640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07dc0d2520 0x7f07dc0c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.636+0000 7f07e0c00640 1 -- 172.21.15.7:0/3986178194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc0cc340 con 0x7f07dc0b3920 2024-04-25T11:20:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.636+0000 7f07e1600640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07dc0d2520 0x7f07dc0c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.636+0000 7f07e0c00640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07dc0b3920 0x7f07dc156e20 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f07d000c7f0 tx=0x7f07d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.637+0000 7f07d8c00640 1 -- 172.21.15.7:0/3986178194 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0015070 con 0x7f07dc0b3920 2024-04-25T11:20:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.637+0000 7f07d8c00640 1 -- 172.21.15.7:0/3986178194 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d0007820 con 0x7f07dc0b3920 2024-04-25T11:20:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.637+0000 7f07d8c00640 1 -- 172.21.15.7:0/3986178194 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0007c10 con 0x7f07dc0b3920 2024-04-25T11:20:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.637+0000 7f07e2000640 1 -- 172.21.15.7:0/3986178194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07dc144750 con 0x7f07dc0b3920 2024-04-25T11:20:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.637+0000 7f07e2000640 1 -- 172.21.15.7:0/3986178194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc1462b0 con 0x7f07dc0b3920 2024-04-25T11:20:03.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.638+0000 7f07e2000640 1 -- 172.21.15.7:0/3986178194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07dc144570 con 0x7f07dc0b3920 2024-04-25T11:20:03.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.639+0000 7f07d8c00640 1 -- 172.21.15.7:0/3986178194 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f07d0018070 con 0x7f07dc0b3920 2024-04-25T11:20:03.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.640+0000 7f07d8c00640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07bc0744b0 0x7f07bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:03.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.641+0000 7f07d8c00640 1 -- 172.21.15.7:0/3986178194 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f07d00955d0 con 0x7f07dc0b3920 2024-04-25T11:20:03.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.641+0000 7f07dbe00640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07bc0744b0 0x7f07bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.641+0000 7f07dbe00640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07bc0744b0 0x7f07bc0769c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f07c8034000 tx=0x7f07c802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.646+0000 7f07d8c00640 1 -- 172.21.15.7:0/3986178194 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07dc144570 con 0x7f07dc0b3920 2024-04-25T11:20:03.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.900+0000 7f07e2000640 1 -- 172.21.15.7:0/3986178194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07dc050740 con 0x7f07dc0b3920 2024-04-25T11:20:03.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.901+0000 7f07d8c00640 1 -- 172.21.15.7:0/3986178194 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5174 (secure 0 0 0) 0x7f07d005fab0 con 0x7f07dc0b3920 2024-04-25T11:20:03.900 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:03.900 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/326382713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":326382713},{"type":"v1","addr":"172.21.15.7:6837","nonce":326382713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2007810270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2007810270},{"type":"v1","addr":"172.21.15.87:6835","nonce":2007810270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1753656512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1753656512},{"type":"v1","addr":"172.21.15.7:6836","nonce":1753656512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:47.807534+0000","modified":"2024-04-25T11:19:50.899375+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1924056839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1924056839},{"type":"v1","addr":"172.21.15.87:6834","nonce":1924056839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8007,"qdb_cluster":[8007]},"id":8}]} 2024-04-25T11:20:03.900 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 -- 172.21.15.7:0/3986178194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07bc0744b0 msgr2=0x7f07bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07bc0744b0 0x7f07bc0769c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f07c8034000 tx=0x7f07c802e000 comp rx=0 tx=0).stop 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 -- 172.21.15.7:0/3986178194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07dc0b3920 msgr2=0x7f07dc156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07dc0b3920 0x7f07dc156e20 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f07d000c7f0 tx=0x7f07d000ccb0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 -- 172.21.15.7:0/3986178194 shutdown_connections 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07bc0744b0 0x7f07bc0769c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07dc0d2520 0x7f07dc0c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07dc0b3d30 0x7f07dc0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 --2- 172.21.15.7:0/3986178194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07dc0b3920 0x7f07dc156e20 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 -- 172.21.15.7:0/3986178194 >> 172.21.15.7:0/3986178194 conn(0x7f07dc09b540 msgr2=0x7f07dc0b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 -- 172.21.15.7:0/3986178194 shutdown_connections 2024-04-25T11:20:03.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:03.904+0000 7f07e2000640 1 -- 172.21.15.7:0/3986178194 wait complete. 2024-04-25T11:20:03.929 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-25T11:20:04.227 INFO:teuthology.orchestra.run.smithi087.stdout:[{"id":6098,"entity":{"name":{"type":"client","num":6098},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1133632146}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":5.1708852820000004,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9847786595908259,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6098 192.168.0.1:0/1133632146","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi087","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":8157,"entity":{"name":{"type":"client","num":8157},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":128224496}},"state":"open","num_leases":1,"num_caps":315,"request_load_avg":337,"uptime":6.9838450620000003,"requests_in_flight":313,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":630.44675433600605,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8157 192.168.0.1:0/128224496","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi007","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T11:20:04.228 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi087", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1133632146, "type": "any" }, "name": { "num": 6098, "type": "client" } }, "id": 6098, "inst": "client.6098 192.168.0.1:0/1133632146", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.984778659590826 }, "state": "open", "uptime": 5.170885282 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi007", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 128224496, "type": "any" }, "name": { "num": 8157, "type": "client" } }, "id": 8157, "inst": "client.8157 192.168.0.1:0/128224496", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 315, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 337, "requests_in_flight": 313, "session_cache_liveness": { "halflife": 300, "value": 630.446754336006 }, "state": "open", "uptime": 6.983845062 } ] 2024-04-25T11:20:04.228 DEBUG:tasks.ceph_test_case:wait_until_equal: 315 != 802, waiting (timeout=600)... 2024-04-25T11:20:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:waiting with handles open 2024-04-25T11:20:05.854 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:20:07.856 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:20:09.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:09.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.525+0000 7f768a200640 1 Processor -- start 2024-04-25T11:20:09.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.525+0000 7f768a200640 1 -- start start 2024-04-25T11:20:09.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.528+0000 7f768a200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f768409ba00 0x7f768409be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.528+0000 7f768a200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f768409c350 con 0x7f76840d1540 2024-04-25T11:20:09.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.528+0000 7f768a200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76840cea70 con 0x7f76840cfb10 2024-04-25T11:20:09.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.528+0000 7f768a200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f76840ced50 con 0x7f768409ba00 2024-04-25T11:20:09.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.528+0000 7f7688e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f76840d1540 0x7f76840ce270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54792/0 (socket says 172.21.15.7:54792) 2024-04-25T11:20:09.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.528+0000 7f7688e00640 1 -- 172.21.15.7:0/299422028 learned_addr learned my addr 172.21.15.7:0/299422028 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:09.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.529+0000 7f7689800640 1 --2- 172.21.15.7:0/299422028 >> v2:172.21.15.87:3300/0 conn(0x7f768409ba00 0x7f768409be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.529+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 402983139 0 0) 0x7f7674001290 con 0x7f76840d1540 2024-04-25T11:20:09.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.529+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7664003520 con 0x7f76840d1540 2024-04-25T11:20:09.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.529+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 674404906 0 0) 0x7f7664003520 con 0x7f76840d1540 2024-04-25T11:20:09.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.529+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 >> v2:172.21.15.87:3300/0 conn(0x7f768409ba00 msgr2=0x7f768409be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:09.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.529+0000 7f7683400640 1 --2- 172.21.15.7:0/299422028 >> v2:172.21.15.87:3300/0 conn(0x7f768409ba00 0x7f768409be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.529+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 >> v1:172.21.15.87:6789/0 conn(0x7f76840cfb10 legacy=0x7f76840b8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.529+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76840cf030 con 0x7f76840d1540 2024-04-25T11:20:09.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.529+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f76740019c0 con 0x7f76840d1540 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 >> v1:172.21.15.7:6789/0 conn(0x7f76840d1540 legacy=0x7f76840ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7683400640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7664003a00 0x7f7664003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7683400640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7664004800 0x7f7664004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7683400640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7664005270 0x7f766400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76840ced50 con 0x7f7664004800 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f766400c190 con 0x7f7664005270 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f766400c390 con 0x7f7664003a00 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7683e00640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7664004800 0x7f7664004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7689800640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7664005270 0x7f766400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7688e00640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7664003a00 0x7f7664003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7683e00640 1 -- 172.21.15.7:0/299422028 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7664003a00 msgr2=0x7f7664003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.530+0000 7f7683e00640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7664003a00 0x7f7664003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.531+0000 7f7683e00640 1 -- 172.21.15.7:0/299422028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7664005270 msgr2=0x7f766400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:09.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.531+0000 7f7683e00640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7664005270 0x7f766400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.531+0000 7f7683e00640 1 -- 172.21.15.7:0/299422028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7664003520 con 0x7f7664004800 2024-04-25T11:20:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.531+0000 7f7689800640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7664005270 0x7f766400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.531+0000 7f7689800640 1 -- 172.21.15.7:0/299422028 reap_dead start 2024-04-25T11:20:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.531+0000 7f7688e00640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7664003a00 0x7f7664003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.532+0000 7f7683e00640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7664004800 0x7f7664004c10 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f766c001e50 tx=0x7f7664005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f2110238af4270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.532+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c001a70 con 0x7f7664004800 2024-04-25T11:20:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.532+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766c02ae50 con 0x7f7664004800 2024-04-25T11:20:09.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.532+0000 7f7683400640 1 -- 172.21.15.7:0/299422028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c003bb0 con 0x7f7664004800 2024-04-25T11:20:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.533+0000 7f768a200640 1 -- 172.21.15.7:0/299422028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7664004800 msgr2=0x7f7664004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.533+0000 7f768a200640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7664004800 0x7f7664004c10 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f766c001e50 tx=0x7f7664005830 comp rx=0 tx=0).stop 2024-04-25T11:20:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.533+0000 7f768a200640 1 -- 172.21.15.7:0/299422028 shutdown_connections 2024-04-25T11:20:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.533+0000 7f768a200640 1 --2- 172.21.15.7:0/299422028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7664004800 0x7f7664004c10 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.533+0000 7f768a200640 1 -- 172.21.15.7:0/299422028 >> 172.21.15.7:0/299422028 conn(0x7f76840b5250 msgr2=0x7f76840b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.533+0000 7f768a200640 1 -- 172.21.15.7:0/299422028 shutdown_connections 2024-04-25T11:20:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.533+0000 7f768a200640 1 -- 172.21.15.7:0/299422028 wait complete. 2024-04-25T11:20:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.535+0000 7f768a200640 1 Processor -- start 2024-04-25T11:20:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.535+0000 7f768a200640 1 -- start start 2024-04-25T11:20:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.537+0000 7f768a200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7664004800 0x7f768414dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.537+0000 7f768a200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7684150ea0 0x7f76840ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.537+0000 7f768a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76840cd1b0 0x7f76841f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.537+0000 7f768a200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f768409be10 con 0x7f76840cd1b0 2024-04-25T11:20:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.537+0000 7f768a200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f76840d2680 con 0x7f7664004800 2024-04-25T11:20:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.537+0000 7f768a200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f768409bf90 con 0x7f7684150ea0 2024-04-25T11:20:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.537+0000 7f7689800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76840cd1b0 0x7f76841f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.537+0000 7f7689800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76840cd1b0 0x7f76841f9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54470/0 (socket says 172.21.15.7:54470) 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.537+0000 7f7689800640 1 -- 172.21.15.7:0/3181079901 learned_addr learned my addr 172.21.15.7:0/3181079901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.538+0000 7f7683e00640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7684150ea0 0x7f76840ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.538+0000 7f7688e00640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7664004800 0x7f768414dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.538+0000 7f7689800640 1 -- 172.21.15.7:0/3181079901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7684150ea0 msgr2=0x7f76840ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.538+0000 7f7689800640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7684150ea0 0x7f76840ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.538+0000 7f7689800640 1 -- 172.21.15.7:0/3181079901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7664004800 msgr2=0x7f768414dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.538+0000 7f7689800640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7664004800 0x7f768414dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.538+0000 7f7689800640 1 -- 172.21.15.7:0/3181079901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76840cd5c0 con 0x7f76840cd1b0 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.538+0000 7f7689800640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76840cd1b0 0x7f76841f9c20 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f76840b9460 tx=0x7f767c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.539+0000 7f7680c00640 1 -- 172.21.15.7:0/3181079901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c00f040 con 0x7f76840cd1b0 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.539+0000 7f7680c00640 1 -- 172.21.15.7:0/3181079901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767c01b070 con 0x7f76840cd1b0 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.539+0000 7f7680c00640 1 -- 172.21.15.7:0/3181079901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c004430 con 0x7f76840cd1b0 2024-04-25T11:20:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.539+0000 7f768a200640 1 -- 172.21.15.7:0/3181079901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76840ca300 con 0x7f76840cd1b0 2024-04-25T11:20:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.539+0000 7f768a200640 1 -- 172.21.15.7:0/3181079901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768415fd40 con 0x7f76840cd1b0 2024-04-25T11:20:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.540+0000 7f768a200640 1 -- 172.21.15.7:0/3181079901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768415f7d0 con 0x7f76840cd1b0 2024-04-25T11:20:09.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.542+0000 7f7680c00640 1 -- 172.21.15.7:0/3181079901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f767c00e6d0 con 0x7f76840cd1b0 2024-04-25T11:20:09.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.543+0000 7f7680c00640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76600744e0 0x7f76600769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:09.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.543+0000 7f7680c00640 1 -- 172.21.15.7:0/3181079901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f767c094d40 con 0x7f76840cd1b0 2024-04-25T11:20:09.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.543+0000 7f7688e00640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76600744e0 0x7f76600769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:09.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.544+0000 7f7688e00640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76600744e0 0x7f76600769f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7674001470 tx=0x7f7674009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.548+0000 7f7680c00640 1 -- 172.21.15.7:0/3181079901 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f767c024020 con 0x7f76840cd1b0 2024-04-25T11:20:09.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.794+0000 7f768a200640 1 -- 172.21.15.7:0/3181079901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f768409c6e0 con 0x7f76840cd1b0 2024-04-25T11:20:09.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.796+0000 7f7680c00640 1 -- 172.21.15.7:0/3181079901 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5174 (secure 0 0 0) 0x7f76840cd5c0 con 0x7f76840cd1b0 2024-04-25T11:20:09.794 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:09.795 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/326382713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":326382713},{"type":"v1","addr":"172.21.15.7:6837","nonce":326382713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2007810270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2007810270},{"type":"v1","addr":"172.21.15.87:6835","nonce":2007810270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1753656512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1753656512},{"type":"v1","addr":"172.21.15.7:6836","nonce":1753656512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:47.807534+0000","modified":"2024-04-25T11:20:08.325402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":8,"addr":"172.21.15.87:6834/1924056839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1924056839},{"type":"v1","addr":"172.21.15.87:6834","nonce":1924056839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8007,"qdb_cluster":[8007]},"id":8}]} 2024-04-25T11:20:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2024-04-25T11:20:09.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.798+0000 7f768a200640 1 -- 172.21.15.7:0/3181079901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76600744e0 msgr2=0x7f76600769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.798+0000 7f768a200640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76600744e0 0x7f76600769f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7674001470 tx=0x7f7674009040 comp rx=0 tx=0).stop 2024-04-25T11:20:09.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.798+0000 7f768a200640 1 -- 172.21.15.7:0/3181079901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76840cd1b0 msgr2=0x7f76841f9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:09.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.798+0000 7f768a200640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76840cd1b0 0x7f76841f9c20 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f76840b9460 tx=0x7f767c010c20 comp rx=0 tx=0).stop 2024-04-25T11:20:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.799+0000 7f768a200640 1 -- 172.21.15.7:0/3181079901 shutdown_connections 2024-04-25T11:20:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.799+0000 7f768a200640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76600744e0 0x7f76600769f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.799+0000 7f768a200640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76840cd1b0 0x7f76841f9c20 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.799+0000 7f768a200640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7684150ea0 0x7f76840ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.799+0000 7f768a200640 1 --2- 172.21.15.7:0/3181079901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7664004800 0x7f768414dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.799+0000 7f768a200640 1 -- 172.21.15.7:0/3181079901 >> 172.21.15.7:0/3181079901 conn(0x7f76840b5250 msgr2=0x7f76840b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.799+0000 7f768a200640 1 -- 172.21.15.7:0/3181079901 shutdown_connections 2024-04-25T11:20:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:09.799+0000 7f768a200640 1 -- 172.21.15.7:0/3181079901 wait complete. 2024-04-25T11:20:09.825 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-25T11:20:09.857 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:20:10.130 INFO:teuthology.orchestra.run.smithi087.stdout:[{"id":6098,"entity":{"name":{"type":"client","num":6098},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1133632146}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":11.0747543,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9443402237550038,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6098 192.168.0.1:0/1133632146","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi087","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":8157,"entity":{"name":{"type":"client","num":8157},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":128224496}},"state":"open","num_leases":1,"num_caps":802,"request_load_avg":774,"uptime":12.88771408,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2175.5023279114307,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":696.52521352792962,"halflife":1.5},"recall_caps_throttle2o":{"value":181.13855011330881,"halflife":0.5},"session_cache_liveness":{"value":1584.5961070553431,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":530,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8157 192.168.0.1:0/128224496","completed_requests":[{"tid":530,"created_ino":"0x100000003e9"},{"tid":531,"created_ino":"0x100000003ea"},{"tid":532,"created_ino":"0x100000003eb"},{"tid":533,"created_ino":"0x100000003ec"},{"tid":534,"created_ino":"0x100000003ed"},{"tid":535,"created_ino":"0x100000003ee"},{"tid":536,"created_ino":"0x100000003ef"},{"tid":537,"created_ino":"0x100000003f0"},{"tid":538,"created_ino":"0x100000003f1"},{"tid":539,"created_ino":"0x100000003f2"},{"tid":540,"created_ino":"0x100000003f3"},{"tid":541,"created_ino":"0x100000003f4"},{"tid":542,"created_ino":"0x100000003f5"},{"tid":543,"created_ino":"0x100000003f6"},{"tid":544,"created_ino":"0x100000003f7"},{"tid":545,"created_ino":"0x100000003f8"},{"tid":546,"created_ino":"0x100000003f9"},{"tid":547,"created_ino":"0x100000003fa"},{"tid":548,"created_ino":"0x100000003fb"},{"tid":549,"created_ino":"0x100000003fc"},{"tid":550,"created_ino":"0x100000003fd"},{"tid":551,"created_ino":"0x100000003fe"},{"tid":552,"created_ino":"0x100000003ff"},{"tid":553,"created_ino":"0x10000000400"},{"tid":554,"created_ino":"0x10000000401"},{"tid":555,"created_ino":"0x10000000402"},{"tid":556,"created_ino":"0x10000000403"},{"tid":557,"created_ino":"0x10000000404"},{"tid":558,"created_ino":"0x10000000405"},{"tid":559,"created_ino":"0x10000000406"},{"tid":560,"created_ino":"0x10000000407"},{"tid":561,"created_ino":"0x10000000408"},{"tid":562,"created_ino":"0x10000000409"},{"tid":563,"created_ino":"0x1000000040a"},{"tid":564,"created_ino":"0x1000000040b"},{"tid":565,"created_ino":"0x1000000040c"},{"tid":566,"created_ino":"0x1000000040d"},{"tid":567,"created_ino":"0x1000000040e"},{"tid":568,"created_ino":"0x1000000040f"},{"tid":569,"created_ino":"0x10000000410"},{"tid":570,"created_ino":"0x10000000411"},{"tid":571,"created_ino":"0x10000000412"},{"tid":572,"created_ino":"0x10000000413"},{"tid":573,"created_ino":"0x10000000414"},{"tid":574,"created_ino":"0x10000000415"},{"tid":575,"created_ino":"0x10000000416"},{"tid":576,"created_ino":"0x10000000417"},{"tid":577,"created_ino":"0x10000000418"},{"tid":578,"created_ino":"0x10000000419"},{"tid":579,"created_ino":"0x1000000041a"},{"tid":580,"created_ino":"0x1000000041b"},{"tid":581,"created_ino":"0x1000000041c"},{"tid":582,"created_ino":"0x1000000041d"},{"tid":583,"created_ino":"0x1000000041e"},{"tid":584,"created_ino":"0x1000000041f"},{"tid":585,"created_ino":"0x10000000420"},{"tid":586,"created_ino":"0x10000000421"},{"tid":587,"created_ino":"0x10000000422"},{"tid":588,"created_ino":"0x10000000423"},{"tid":589,"created_ino":"0x10000000424"},{"tid":590,"created_ino":"0x10000000425"},{"tid":591,"created_ino":"0x10000000426"},{"tid":592,"created_ino":"0x10000000427"},{"tid":593,"created_ino":"0x10000000428"},{"tid":594,"created_ino":"0x10000000429"},{"tid":595,"created_ino":"0x1000000042a"},{"tid":596,"created_ino":"0x1000000042b"},{"tid":597,"created_ino":"0x1000000042c"},{"tid":598,"created_ino":"0x1000000042d"},{"tid":599,"created_ino":"0x1000000042e"},{"tid":600,"created_ino":"0x1000000042f"},{"tid":601,"created_ino":"0x10000000430"},{"tid":602,"created_ino":"0x10000000431"},{"tid":603,"created_ino":"0x10000000432"},{"tid":604,"created_ino":"0x10000000433"},{"tid":605,"created_ino":"0x10000000434"},{"tid":606,"created_ino":"0x10000000435"},{"tid":607,"created_ino":"0x10000000436"},{"tid":608,"created_ino":"0x10000000437"},{"tid":609,"created_ino":"0x10000000438"},{"tid":610,"created_ino":"0x10000000439"},{"tid":611,"created_ino":"0x1000000043a"},{"tid":612,"created_ino":"0x1000000043b"},{"tid":613,"created_ino":"0x1000000043c"},{"tid":614,"created_ino":"0x1000000043d"},{"tid":615,"created_ino":"0x1000000043e"},{"tid":616,"created_ino":"0x1000000043f"},{"tid":617,"created_ino":"0x10000000440"},{"tid":618,"created_ino":"0x10000000441"},{"tid":619,"created_ino":"0x10000000442"},{"tid":620,"created_ino":"0x10000000443"},{"tid":621,"created_ino":"0x10000000444"},{"tid":622,"created_ino":"0x10000000445"},{"tid":623,"created_ino":"0x10000000446"},{"tid":624,"created_ino":"0x10000000447"},{"tid":625,"created_ino":"0x10000000448"},{"tid":626,"created_ino":"0x10000000449"},{"tid":627,"created_ino":"0x1000000044a"},{"tid":628,"created_ino":"0x1000000044b"},{"tid":629,"created_ino":"0x1000000044c"},{"tid":630,"created_ino":"0x1000000044d"},{"tid":631,"created_ino":"0x1000000044e"},{"tid":632,"created_ino":"0x1000000044f"},{"tid":633,"created_ino":"0x10000000450"},{"tid":634,"created_ino":"0x10000000451"},{"tid":635,"created_ino":"0x10000000452"},{"tid":636,"created_ino":"0x10000000453"},{"tid":637,"created_ino":"0x10000000454"},{"tid":638,"created_ino":"0x10000000455"},{"tid":639,"created_ino":"0x10000000456"},{"tid":640,"created_ino":"0x10000000457"},{"tid":641,"created_ino":"0x10000000458"},{"tid":642,"created_ino":"0x10000000459"},{"tid":643,"created_ino":"0x1000000045a"},{"tid":644,"created_ino":"0x1000000045b"},{"tid":645,"created_ino":"0x1000000045c"},{"tid":646,"created_ino":"0x1000000045d"},{"tid":647,"created_ino":"0x1000000045e"},{"tid":648,"created_ino":"0x1000000045f"},{"tid":649,"created_ino":"0x10000000460"},{"tid":650,"created_ino":"0x10000000461"},{"tid":651,"created_ino":"0x10000000462"},{"tid":652,"created_ino":"0x10000000463"},{"tid":653,"created_ino":"0x10000000464"},{"tid":654,"created_ino":"0x10000000465"},{"tid":655,"created_ino":"0x10000000466"},{"tid":656,"created_ino":"0x10000000467"},{"tid":657,"created_ino":"0x10000000468"},{"tid":658,"created_ino":"0x10000000469"},{"tid":659,"created_ino":"0x1000000046a"},{"tid":660,"created_ino":"0x1000000046b"},{"tid":661,"created_ino":"0x1000000046c"},{"tid":662,"created_ino":"0x1000000046d"},{"tid":663,"created_ino":"0x1000000046e"},{"tid":664,"created_ino":"0x1000000046f"},{"tid":665,"created_ino":"0x10000000470"},{"tid":666,"created_ino":"0x10000000471"},{"tid":667,"created_ino":"0x10000000472"},{"tid":668,"created_ino":"0x10000000473"},{"tid":669,"created_ino":"0x10000000474"},{"tid":670,"created_ino":"0x10000000475"},{"tid":671,"created_ino":"0x10000000476"},{"tid":672,"created_ino":"0x10000000477"},{"tid":673,"created_ino":"0x10000000478"},{"tid":674,"created_ino":"0x10000000479"},{"tid":675,"created_ino":"0x1000000047a"},{"tid":676,"created_ino":"0x1000000047b"},{"tid":677,"created_ino":"0x1000000047c"},{"tid":678,"created_ino":"0x1000000047d"},{"tid":679,"created_ino":"0x1000000047e"},{"tid":680,"created_ino":"0x1000000047f"},{"tid":681,"created_ino":"0x10000000480"},{"tid":682,"created_ino":"0x10000000481"},{"tid":683,"created_ino":"0x10000000482"},{"tid":684,"created_ino":"0x10000000483"},{"tid":685,"created_ino":"0x10000000484"},{"tid":686,"created_ino":"0x10000000485"},{"tid":687,"created_ino":"0x10000000486"},{"tid":688,"created_ino":"0x10000000487"},{"tid":689,"created_ino":"0x10000000488"},{"tid":690,"created_ino":"0x10000000489"},{"tid":691,"created_ino":"0x1000000048a"},{"tid":692,"created_ino":"0x1000000048b"},{"tid":693,"created_ino":"0x1000000048c"},{"tid":694,"created_ino":"0x1000000048d"},{"tid":695,"created_ino":"0x1000000048e"},{"tid":696,"created_ino":"0x1000000048f"},{"tid":697,"created_ino":"0x10000000490"},{"tid":698,"created_ino":"0x10000000491"},{"tid":699,"created_ino":"0x10000000492"},{"tid":700,"created_ino":"0x10000000493"},{"tid":701,"created_ino":"0x10000000494"},{"tid":702,"created_ino":"0x10000000495"},{"tid":703,"created_ino":"0x10000000496"},{"tid":704,"created_ino":"0x10000000497"},{"tid":705,"created_ino":"0x10000000498"},{"tid":706,"created_ino":"0x10000000499"},{"tid":707,"created_ino":"0x1000000049a"},{"tid":708,"created_ino":"0x1000000049b"},{"tid":709,"created_ino":"0x1000000049c"},{"tid":710,"created_ino":"0x1000000049d"},{"tid":711,"created_ino":"0x1000000049e"},{"tid":712,"created_ino":"0x1000000049f"},{"tid":713,"created_ino":"0x100000004a0"},{"tid":714,"created_ino":"0x100000004a1"},{"tid":715,"created_ino":"0x100000004a2"},{"tid":716,"created_ino":"0x100000004a3"},{"tid":717,"created_ino":"0x100000004a4"},{"tid":718,"created_ino":"0x100000004a5"},{"tid":719,"created_ino":"0x100000004a6"},{"tid":720,"created_ino":"0x100000004a7"},{"tid":721,"created_ino":"0x100000004a8"},{"tid":722,"created_ino":"0x100000004a9"},{"tid":723,"created_ino":"0x100000004aa"},{"tid":724,"created_ino":"0x100000004ab"},{"tid":725,"created_ino":"0x100000004ac"},{"tid":726,"created_ino":"0x100000004ad"},{"tid":727,"created_ino":"0x100000004ae"},{"tid":728,"created_ino":"0x100000004af"},{"tid":729,"created_ino":"0x100000004b0"},{"tid":730,"created_ino":"0x100000004b1"},{"tid":731,"created_ino":"0x100000004b2"},{"tid":732,"created_ino":"0x100000004b3"},{"tid":733,"created_ino":"0x100000004b4"},{"tid":734,"created_ino":"0x100000004b5"},{"tid":735,"created_ino":"0x100000004b6"},{"tid":736,"created_ino":"0x100000004b7"},{"tid":737,"created_ino":"0x100000004b8"},{"tid":738,"created_ino":"0x100000004b9"},{"tid":739,"created_ino":"0x100000004ba"},{"tid":740,"created_ino":"0x100000004bb"},{"tid":741,"created_ino":"0x100000004bc"},{"tid":742,"created_ino":"0x100000004bd"},{"tid":743,"created_ino":"0x100000004be"},{"tid":744,"created_ino":"0x100000004bf"},{"tid":745,"created_ino":"0x100000004c0"},{"tid":746,"created_ino":"0x100000004c1"},{"tid":747,"created_ino":"0x100000004c2"},{"tid":748,"created_ino":"0x100000004c3"},{"tid":749,"created_ino":"0x100000004c4"},{"tid":750,"created_ino":"0x100000004c5"},{"tid":751,"created_ino":"0x100000004c6"},{"tid":752,"created_ino":"0x100000004c7"},{"tid":753,"created_ino":"0x100000004c8"},{"tid":754,"created_ino":"0x100000004c9"},{"tid":755,"created_ino":"0x100000004ca"},{"tid":756,"created_ino":"0x100000004cb"},{"tid":757,"created_ino":"0x100000004cc"},{"tid":758,"created_ino":"0x100000004cd"},{"tid":759,"created_ino":"0x100000004ce"},{"tid":760,"created_ino":"0x100000004cf"},{"tid":761,"created_ino":"0x100000004d0"},{"tid":762,"created_ino":"0x100000004d1"},{"tid":763,"created_ino":"0x100000004d2"},{"tid":764,"created_ino":"0x100000004d3"},{"tid":765,"created_ino":"0x100000004d4"},{"tid":766,"created_ino":"0x100000004d5"},{"tid":767,"created_ino":"0x100000004d6"},{"tid":768,"created_ino":"0x100000004d7"},{"tid":769,"created_ino":"0x100000004d8"},{"tid":770,"created_ino":"0x100000004d9"},{"tid":771,"created_ino":"0x100000004da"},{"tid":772,"created_ino":"0x100000004db"},{"tid":773,"created_ino":"0x100000004dc"},{"tid":774,"created_ino":"0x100000004dd"},{"tid":775,"created_ino":"0x100000004de"},{"tid":776,"created_ino":"0x100000004df"},{"tid":777,"created_ino":"0x100000004e0"},{"tid":778,"created_ino":"0x100000004e1"},{"tid":779,"created_ino":"0x100000004e2"},{"tid":780,"created_ino":"0x100000004e3"},{"tid":781,"created_ino":"0x100000004e4"},{"tid":782,"created_ino":"0x100000004e5"},{"tid":783,"created_ino":"0x100000004e6"},{"tid":784,"created_ino":"0x100000004e7"},{"tid":785,"created_ino":"0x100000004e8"},{"tid":786,"created_ino":"0x100000004e9"},{"tid":787,"created_ino":"0x100000004ea"},{"tid":788,"created_ino":"0x100000004eb"},{"tid":789,"created_ino":"0x100000004ec"},{"tid":790,"created_ino":"0x100000004ed"},{"tid":791,"created_ino":"0x100000004ee"},{"tid":792,"created_ino":"0x100000004ef"},{"tid":793,"created_ino":"0x100000004f0"},{"tid":794,"created_ino":"0x100000004f1"},{"tid":795,"created_ino":"0x100000004f2"},{"tid":796,"created_ino":"0x100000004f3"},{"tid":797,"created_ino":"0x100000004f4"},{"tid":798,"created_ino":"0x100000004f5"},{"tid":799,"created_ino":"0x100000004f6"},{"tid":800,"created_ino":"0x100000004f7"},{"tid":801,"created_ino":"0x100000004f8"},{"tid":802,"created_ino":"0x100000004f9"},{"tid":803,"created_ino":"0x100000004fa"},{"tid":804,"created_ino":"0x100000004fb"},{"tid":805,"created_ino":"0x100000004fc"},{"tid":806,"created_ino":"0x100000004fd"},{"tid":807,"created_ino":"0x100000004fe"},{"tid":808,"created_ino":"0x100000004ff"},{"tid":809,"created_ino":"0x10000000500"},{"tid":810,"created_ino":"0x10000000501"},{"tid":811,"created_ino":"0x10000000502"},{"tid":812,"created_ino":"0x10000000503"},{"tid":813,"created_ino":"0x10000000504"},{"tid":814,"created_ino":"0x10000000505"},{"tid":815,"created_ino":"0x10000000506"},{"tid":816,"created_ino":"0x10000000507"},{"tid":817,"created_ino":"0x10000000508"},{"tid":818,"created_ino":"0x10000000509"},{"tid":819,"created_ino":"0x1000000050a"},{"tid":820,"created_ino":"0x1000000050b"},{"tid":821,"created_ino":"0x1000000050c"},{"tid":822,"created_ino":"0x1000000050d"},{"tid":823,"created_ino":"0x1000000050e"},{"tid":824,"created_ino":"0x1000000050f"},{"tid":825,"created_ino":"0x10000000510"},{"tid":826,"created_ino":"0x10000000511"},{"tid":827,"created_ino":"0x10000000512"},{"tid":828,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi007","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T11:20:10.132 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi087", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1133632146, "type": "any" }, "name": { "num": 6098, "type": "client" } }, "id": 6098, "inst": "client.6098 192.168.0.1:0/1133632146", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.944340223755004 }, "state": "open", "uptime": 11.0747543 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi007", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 530 }, { "created_ino": "0x100000003ea", "tid": 531 }, { "created_ino": "0x100000003eb", "tid": 532 }, { "created_ino": "0x100000003ec", "tid": 533 }, { "created_ino": "0x100000003ed", "tid": 534 }, { "created_ino": "0x100000003ee", "tid": 535 }, { "created_ino": "0x100000003ef", "tid": 536 }, { "created_ino": "0x100000003f0", "tid": 537 }, { "created_ino": "0x100000003f1", "tid": 538 }, { "created_ino": "0x100000003f2", "tid": 539 }, { "created_ino": "0x100000003f3", "tid": 540 }, { "created_ino": "0x100000003f4", "tid": 541 }, { "created_ino": "0x100000003f5", "tid": 542 }, { "created_ino": "0x100000003f6", "tid": 543 }, { "created_ino": "0x100000003f7", "tid": 544 }, { "created_ino": "0x100000003f8", "tid": 545 }, { "created_ino": "0x100000003f9", "tid": 546 }, { "created_ino": "0x100000003fa", "tid": 547 }, { "created_ino": "0x100000003fb", "tid": 548 }, { "created_ino": "0x100000003fc", "tid": 549 }, { "created_ino": "0x100000003fd", "tid": 550 }, { "created_ino": "0x100000003fe", "tid": 551 }, { "created_ino": "0x100000003ff", "tid": 552 }, { "created_ino": "0x10000000400", "tid": 553 }, { "created_ino": "0x10000000401", "tid": 554 }, { "created_ino": "0x10000000402", "tid": 555 }, { "created_ino": "0x10000000403", "tid": 556 }, { "created_ino": "0x10000000404", "tid": 557 }, { "created_ino": "0x10000000405", "tid": 558 }, { "created_ino": "0x10000000406", "tid": 559 }, { "created_ino": "0x10000000407", "tid": 560 }, { "created_ino": "0x10000000408", "tid": 561 }, { "created_ino": "0x10000000409", "tid": 562 }, { "created_ino": "0x1000000040a", "tid": 563 }, { "created_ino": "0x1000000040b", "tid": 564 }, { "created_ino": "0x1000000040c", "tid": 565 }, { "created_ino": "0x1000000040d", "tid": 566 }, { "created_ino": "0x1000000040e", "tid": 567 }, { "created_ino": "0x1000000040f", "tid": 568 }, { "created_ino": "0x10000000410", "tid": 569 }, { "created_ino": "0x10000000411", "tid": 570 }, { "created_ino": "0x10000000412", "tid": 571 }, { "created_ino": "0x10000000413", "tid": 572 }, { "created_ino": "0x10000000414", "tid": 573 }, { "created_ino": "0x10000000415", "tid": 574 }, { "created_ino": "0x10000000416", "tid": 575 }, { "created_ino": "0x10000000417", "tid": 576 }, { "created_ino": "0x10000000418", "tid": 577 }, { "created_ino": "0x10000000419", "tid": 578 }, { "created_ino": "0x1000000041a", "tid": 579 }, { "created_ino": "0x1000000041b", "tid": 580 }, { "created_ino": "0x1000000041c", "tid": 581 }, { "created_ino": "0x1000000041d", "tid": 582 }, { "created_ino": "0x1000000041e", "tid": 583 }, { "created_ino": "0x1000000041f", "tid": 584 }, { "created_ino": "0x10000000420", "tid": 585 }, { "created_ino": "0x10000000421", "tid": 586 }, { "created_ino": "0x10000000422", "tid": 587 }, { "created_ino": "0x10000000423", "tid": 588 }, { "created_ino": "0x10000000424", "tid": 589 }, { "created_ino": "0x10000000425", "tid": 590 }, { "created_ino": "0x10000000426", "tid": 591 }, { "created_ino": "0x10000000427", "tid": 592 }, { "created_ino": "0x10000000428", "tid": 593 }, { "created_ino": "0x10000000429", "tid": 594 }, { "created_ino": "0x1000000042a", "tid": 595 }, { "created_ino": "0x1000000042b", "tid": 596 }, { "created_ino": "0x1000000042c", "tid": 597 }, { "created_ino": "0x1000000042d", "tid": 598 }, { "created_ino": "0x1000000042e", "tid": 599 }, { "created_ino": "0x1000000042f", "tid": 600 }, { "created_ino": "0x10000000430", "tid": 601 }, { "created_ino": "0x10000000431", "tid": 602 }, { "created_ino": "0x10000000432", "tid": 603 }, { "created_ino": "0x10000000433", "tid": 604 }, { "created_ino": "0x10000000434", "tid": 605 }, { "created_ino": "0x10000000435", "tid": 606 }, { "created_ino": "0x10000000436", "tid": 607 }, { "created_ino": "0x10000000437", "tid": 608 }, { "created_ino": "0x10000000438", "tid": 609 }, { "created_ino": "0x10000000439", "tid": 610 }, { "created_ino": "0x1000000043a", "tid": 611 }, { "created_ino": "0x1000000043b", "tid": 612 }, { "created_ino": "0x1000000043c", "tid": 613 }, { "created_ino": "0x1000000043d", "tid": 614 }, { "created_ino": "0x1000000043e", "tid": 615 }, { "created_ino": "0x1000000043f", "tid": 616 }, { "created_ino": "0x10000000440", "tid": 617 }, { "created_ino": "0x10000000441", "tid": 618 }, { "created_ino": "0x10000000442", "tid": 619 }, { "created_ino": "0x10000000443", "tid": 620 }, { "created_ino": "0x10000000444", "tid": 621 }, { "created_ino": "0x10000000445", "tid": 622 }, { "created_ino": "0x10000000446", "tid": 623 }, { "created_ino": "0x10000000447", "tid": 624 }, { "created_ino": "0x10000000448", "tid": 625 }, { "created_ino": "0x10000000449", "tid": 626 }, { "created_ino": "0x1000000044a", "tid": 627 }, { "created_ino": "0x1000000044b", "tid": 628 }, { "created_ino": "0x1000000044c", "tid": 629 }, { "created_ino": "0x1000000044d", "tid": 630 }, { "created_ino": "0x1000000044e", "tid": 631 }, { "created_ino": "0x1000000044f", "tid": 632 }, { "created_ino": "0x10000000450", "tid": 633 }, { "created_ino": "0x10000000451", "tid": 634 }, { "created_ino": "0x10000000452", "tid": 635 }, { "created_ino": "0x10000000453", "tid": 636 }, { "created_ino": "0x10000000454", "tid": 637 }, { "created_ino": "0x10000000455", "tid": 638 }, { "created_ino": "0x10000000456", "tid": 639 }, { "created_ino": "0x10000000457", "tid": 640 }, { "created_ino": "0x10000000458", "tid": 641 }, { "created_ino": "0x10000000459", "tid": 642 }, { "created_ino": "0x1000000045a", "tid": 643 }, { "created_ino": "0x1000000045b", "tid": 644 }, { "created_ino": "0x1000000045c", "tid": 645 }, { "created_ino": "0x1000000045d", "tid": 646 }, { "created_ino": "0x1000000045e", "tid": 647 }, { "created_ino": "0x1000000045f", "tid": 648 }, { "created_ino": "0x10000000460", "tid": 649 }, { "created_ino": "0x10000000461", "tid": 650 }, { "created_ino": "0x10000000462", "tid": 651 }, { "created_ino": "0x10000000463", "tid": 652 }, { "created_ino": "0x10000000464", "tid": 653 }, { "created_ino": "0x10000000465", "tid": 654 }, { "created_ino": "0x10000000466", "tid": 655 }, { "created_ino": "0x10000000467", "tid": 656 }, { "created_ino": "0x10000000468", "tid": 657 }, { "created_ino": "0x10000000469", "tid": 658 }, { "created_ino": "0x1000000046a", "tid": 659 }, { "created_ino": "0x1000000046b", "tid": 660 }, { "created_ino": "0x1000000046c", "tid": 661 }, { "created_ino": "0x1000000046d", "tid": 662 }, { "created_ino": "0x1000000046e", "tid": 663 }, { "created_ino": "0x1000000046f", "tid": 664 }, { "created_ino": "0x10000000470", "tid": 665 }, { "created_ino": "0x10000000471", "tid": 666 }, { "created_ino": "0x10000000472", "tid": 667 }, { "created_ino": "0x10000000473", "tid": 668 }, { "created_ino": "0x10000000474", "tid": 669 }, { "created_ino": "0x10000000475", "tid": 670 }, { "created_ino": "0x10000000476", "tid": 671 }, { "created_ino": "0x10000000477", "tid": 672 }, { "created_ino": "0x10000000478", "tid": 673 }, { "created_ino": "0x10000000479", "tid": 674 }, { "created_ino": "0x1000000047a", "tid": 675 }, { "created_ino": "0x1000000047b", "tid": 676 }, { "created_ino": "0x1000000047c", "tid": 677 }, { "created_ino": "0x1000000047d", "tid": 678 }, { "created_ino": "0x1000000047e", "tid": 679 }, { "created_ino": "0x1000000047f", "tid": 680 }, { "created_ino": "0x10000000480", "tid": 681 }, { "created_ino": "0x10000000481", "tid": 682 }, { "created_ino": "0x10000000482", "tid": 683 }, { "created_ino": "0x10000000483", "tid": 684 }, { "created_ino": "0x10000000484", "tid": 685 }, { "created_ino": "0x10000000485", "tid": 686 }, { "created_ino": "0x10000000486", "tid": 687 }, { "created_ino": "0x10000000487", "tid": 688 }, { "created_ino": "0x10000000488", "tid": 689 }, { "created_ino": "0x10000000489", "tid": 690 }, { "created_ino": "0x1000000048a", "tid": 691 }, { "created_ino": "0x1000000048b", "tid": 692 }, { "created_ino": "0x1000000048c", "tid": 693 }, { "created_ino": "0x1000000048d", "tid": 694 }, { "created_ino": "0x1000000048e", "tid": 695 }, { "created_ino": "0x1000000048f", "tid": 696 }, { "created_ino": "0x10000000490", "tid": 697 }, { "created_ino": "0x10000000491", "tid": 698 }, { "created_ino": "0x10000000492", "tid": 699 }, { "created_ino": "0x10000000493", "tid": 700 }, { "created_ino": "0x10000000494", "tid": 701 }, { "created_ino": "0x10000000495", "tid": 702 }, { "created_ino": "0x10000000496", "tid": 703 }, { "created_ino": "0x10000000497", "tid": 704 }, { "created_ino": "0x10000000498", "tid": 705 }, { "created_ino": "0x10000000499", "tid": 706 }, { "created_ino": "0x1000000049a", "tid": 707 }, { "created_ino": "0x1000000049b", "tid": 708 }, { "created_ino": "0x1000000049c", "tid": 709 }, { "created_ino": "0x1000000049d", "tid": 710 }, { "created_ino": "0x1000000049e", "tid": 711 }, { "created_ino": "0x1000000049f", "tid": 712 }, { "created_ino": "0x100000004a0", "tid": 713 }, { "created_ino": "0x100000004a1", "tid": 714 }, { "created_ino": "0x100000004a2", "tid": 715 }, { "created_ino": "0x100000004a3", "tid": 716 }, { "created_ino": "0x100000004a4", "tid": 717 }, { "created_ino": "0x100000004a5", "tid": 718 }, { "created_ino": "0x100000004a6", "tid": 719 }, { "created_ino": "0x100000004a7", "tid": 720 }, { "created_ino": "0x100000004a8", "tid": 721 }, { "created_ino": "0x100000004a9", "tid": 722 }, { "created_ino": "0x100000004aa", "tid": 723 }, { "created_ino": "0x100000004ab", "tid": 724 }, { "created_ino": "0x100000004ac", "tid": 725 }, { "created_ino": "0x100000004ad", "tid": 726 }, { "created_ino": "0x100000004ae", "tid": 727 }, { "created_ino": "0x100000004af", "tid": 728 }, { "created_ino": "0x100000004b0", "tid": 729 }, { "created_ino": "0x100000004b1", "tid": 730 }, { "created_ino": "0x100000004b2", "tid": 731 }, { "created_ino": "0x100000004b3", "tid": 732 }, { "created_ino": "0x100000004b4", "tid": 733 }, { "created_ino": "0x100000004b5", "tid": 734 }, { "created_ino": "0x100000004b6", "tid": 735 }, { "created_ino": "0x100000004b7", "tid": 736 }, { "created_ino": "0x100000004b8", "tid": 737 }, { "created_ino": "0x100000004b9", "tid": 738 }, { "created_ino": "0x100000004ba", "tid": 739 }, { "created_ino": "0x100000004bb", "tid": 740 }, { "created_ino": "0x100000004bc", "tid": 741 }, { "created_ino": "0x100000004bd", "tid": 742 }, { "created_ino": "0x100000004be", "tid": 743 }, { "created_ino": "0x100000004bf", "tid": 744 }, { "created_ino": "0x100000004c0", "tid": 745 }, { "created_ino": "0x100000004c1", "tid": 746 }, { "created_ino": "0x100000004c2", "tid": 747 }, { "created_ino": "0x100000004c3", "tid": 748 }, { "created_ino": "0x100000004c4", "tid": 749 }, { "created_ino": "0x100000004c5", "tid": 750 }, { "created_ino": "0x100000004c6", "tid": 751 }, { "created_ino": "0x100000004c7", "tid": 752 }, { "created_ino": "0x100000004c8", "tid": 753 }, { "created_ino": "0x100000004c9", "tid": 754 }, { "created_ino": "0x100000004ca", "tid": 755 }, { "created_ino": "0x100000004cb", "tid": 756 }, { "created_ino": "0x100000004cc", "tid": 757 }, { "created_ino": "0x100000004cd", "tid": 758 }, { "created_ino": "0x100000004ce", "tid": 759 }, { "created_ino": "0x100000004cf", "tid": 760 }, { "created_ino": "0x100000004d0", "tid": 761 }, { "created_ino": "0x100000004d1", "tid": 762 }, { "created_ino": "0x100000004d2", "tid": 763 }, { "created_ino": "0x100000004d3", "tid": 764 }, { "created_ino": "0x100000004d4", "tid": 765 }, { "created_ino": "0x100000004d5", "tid": 766 }, { "created_ino": "0x100000004d6", "tid": 767 }, { "created_ino": "0x100000004d7", "tid": 768 }, { "created_ino": "0x100000004d8", "tid": 769 }, { "created_ino": "0x100000004d9", "tid": 770 }, { "created_ino": "0x100000004da", "tid": 771 }, { "created_ino": "0x100000004db", "tid": 772 }, { "created_ino": "0x100000004dc", "tid": 773 }, { "created_ino": "0x100000004dd", "tid": 774 }, { "created_ino": "0x100000004de", "tid": 775 }, { "created_ino": "0x100000004df", "tid": 776 }, { "created_ino": "0x100000004e0", "tid": 777 }, { "created_ino": "0x100000004e1", "tid": 778 }, { "created_ino": "0x100000004e2", "tid": 779 }, { "created_ino": "0x100000004e3", "tid": 780 }, { "created_ino": "0x100000004e4", "tid": 781 }, { "created_ino": "0x100000004e5", "tid": 782 }, { "created_ino": "0x100000004e6", "tid": 783 }, { "created_ino": "0x100000004e7", "tid": 784 }, { "created_ino": "0x100000004e8", "tid": 785 }, { "created_ino": "0x100000004e9", "tid": 786 }, { "created_ino": "0x100000004ea", "tid": 787 }, { "created_ino": "0x100000004eb", "tid": 788 }, { "created_ino": "0x100000004ec", "tid": 789 }, { "created_ino": "0x100000004ed", "tid": 790 }, { "created_ino": "0x100000004ee", "tid": 791 }, { "created_ino": "0x100000004ef", "tid": 792 }, { "created_ino": "0x100000004f0", "tid": 793 }, { "created_ino": "0x100000004f1", "tid": 794 }, { "created_ino": "0x100000004f2", "tid": 795 }, { "created_ino": "0x100000004f3", "tid": 796 }, { "created_ino": "0x100000004f4", "tid": 797 }, { "created_ino": "0x100000004f5", "tid": 798 }, { "created_ino": "0x100000004f6", "tid": 799 }, { "created_ino": "0x100000004f7", "tid": 800 }, { "created_ino": "0x100000004f8", "tid": 801 }, { "created_ino": "0x100000004f9", "tid": 802 }, { "created_ino": "0x100000004fa", "tid": 803 }, { "created_ino": "0x100000004fb", "tid": 804 }, { "created_ino": "0x100000004fc", "tid": 805 }, { "created_ino": "0x100000004fd", "tid": 806 }, { "created_ino": "0x100000004fe", "tid": 807 }, { "created_ino": "0x100000004ff", "tid": 808 }, { "created_ino": "0x10000000500", "tid": 809 }, { "created_ino": "0x10000000501", "tid": 810 }, { "created_ino": "0x10000000502", "tid": 811 }, { "created_ino": "0x10000000503", "tid": 812 }, { "created_ino": "0x10000000504", "tid": 813 }, { "created_ino": "0x10000000505", "tid": 814 }, { "created_ino": "0x10000000506", "tid": 815 }, { "created_ino": "0x10000000507", "tid": 816 }, { "created_ino": "0x10000000508", "tid": 817 }, { "created_ino": "0x10000000509", "tid": 818 }, { "created_ino": "0x1000000050a", "tid": 819 }, { "created_ino": "0x1000000050b", "tid": 820 }, { "created_ino": "0x1000000050c", "tid": 821 }, { "created_ino": "0x1000000050d", "tid": 822 }, { "created_ino": "0x1000000050e", "tid": 823 }, { "created_ino": "0x1000000050f", "tid": 824 }, { "created_ino": "0x10000000510", "tid": 825 }, { "created_ino": "0x10000000511", "tid": 826 }, { "created_ino": "0x10000000512", "tid": 827 }, { "created_ino": "0x10000000513", "tid": 828 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 128224496, "type": "any" }, "name": { "num": 8157, "type": "client" } }, "id": 8157, "inst": "client.8157 192.168.0.1:0/128224496", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 530, "num_caps": 802, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2175.5023279114307 }, "recall_caps_throttle": { "halflife": 1.5, "value": 696.5252135279296 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 181.1385501133088 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 774, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1584.596107055343 }, "state": "open", "uptime": 12.88771408 } ] 2024-04-25T11:20:10.132 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-04-25T11:20:10.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:20:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.428+0000 7f0dbda00640 1 Processor -- start 2024-04-25T11:20:10.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.428+0000 7f0dbda00640 1 -- start start 2024-04-25T11:20:10.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.430+0000 7f0dbda00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0db815baa0 0x7f0db815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.430+0000 7f0dbda00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0db806d580 0x7f0db806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.430+0000 7f0dbda00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0db806ded0 con 0x7f0db8070640 2024-04-25T11:20:10.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.430+0000 7f0dbda00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0db815e980 con 0x7f0db806d580 2024-04-25T11:20:10.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.430+0000 7f0dbda00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0db815ec60 con 0x7f0db815baa0 2024-04-25T11:20:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.430+0000 7f0dbd000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0db806d580 0x7f0db806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.430+0000 7f0dbd000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0db806d580 0x7f0db806d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42942/0 (socket says 172.21.15.7:42942) 2024-04-25T11:20:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.430+0000 7f0dbd000640 1 -- 172.21.15.7:0/3104728295 learned_addr learned my addr 172.21.15.7:0/3104728295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.430+0000 7f0db7e00640 1 --2- 172.21.15.7:0/3104728295 >> v2:172.21.15.87:3300/0 conn(0x7f0db815baa0 0x7f0db815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.431+0000 7f0dbd000640 1 -- 172.21.15.7:0/3104728295 >> v2:172.21.15.87:3300/0 conn(0x7f0db815baa0 msgr2=0x7f0db815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.431+0000 7f0dbd000640 1 --2- 172.21.15.7:0/3104728295 >> v2:172.21.15.87:3300/0 conn(0x7f0db815baa0 0x7f0db815e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.431+0000 7f0dbd000640 1 -- 172.21.15.7:0/3104728295 >> v1:172.21.15.87:6789/0 conn(0x7f0db8070640 legacy=0x7f0db8161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:10.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.431+0000 7f0dbd000640 1 -- 172.21.15.7:0/3104728295 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db815ef40 con 0x7f0db806d580 2024-04-25T11:20:10.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.432+0000 7f0dbd000640 1 --2- 172.21.15.7:0/3104728295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db806d580 0x7f0db806d990 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0db80b6d80 tx=0x7f0db002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea787a9b6dbb3d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.432+0000 7f0db7400640 1 -- 172.21.15.7:0/3104728295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0036070 con 0x7f0db806d580 2024-04-25T11:20:10.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.432+0000 7f0db7400640 1 -- 172.21.15.7:0/3104728295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db0031600 con 0x7f0db806d580 2024-04-25T11:20:10.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.432+0000 7f0db7400640 1 -- 172.21.15.7:0/3104728295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00319d0 con 0x7f0db806d580 2024-04-25T11:20:10.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.433+0000 7f0dbda00640 1 -- 172.21.15.7:0/3104728295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db806d580 msgr2=0x7f0db806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.433+0000 7f0dbda00640 1 --2- 172.21.15.7:0/3104728295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db806d580 0x7f0db806d990 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0db80b6d80 tx=0x7f0db002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.433+0000 7f0dbda00640 1 -- 172.21.15.7:0/3104728295 shutdown_connections 2024-04-25T11:20:10.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.433+0000 7f0dbda00640 1 --2- 172.21.15.7:0/3104728295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db806d580 0x7f0db806d990 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.433+0000 7f0dbda00640 1 --2- 172.21.15.7:0/3104728295 >> v2:172.21.15.87:3300/0 conn(0x7f0db815baa0 0x7f0db815e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.433+0000 7f0dbda00640 1 -- 172.21.15.7:0/3104728295 >> 172.21.15.7:0/3104728295 conn(0x7f0db8073090 msgr2=0x7f0db80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:10.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.433+0000 7f0dbda00640 1 -- 172.21.15.7:0/3104728295 shutdown_connections 2024-04-25T11:20:10.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.433+0000 7f0dbda00640 1 -- 172.21.15.7:0/3104728295 wait complete. 2024-04-25T11:20:10.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.435+0000 7f0dbda00640 1 Processor -- start 2024-04-25T11:20:10.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.436+0000 7f0dbda00640 1 -- start start 2024-04-25T11:20:10.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.437+0000 7f0dbda00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0db806d580 0x7f0db815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.437+0000 7f0dbda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db8070640 0x7f0db8145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.437+0000 7f0dbda00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db8159ae0 0x7f0db8147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.437+0000 7f0dbda00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8070bd0 con 0x7f0db8070640 2024-04-25T11:20:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.437+0000 7f0dbda00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0db8070a50 con 0x7f0db806d580 2024-04-25T11:20:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.437+0000 7f0dbda00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0db81625f0 con 0x7f0db8159ae0 2024-04-25T11:20:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.437+0000 7f0daf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db8070640 0x7f0db8145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.437+0000 7f0daf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db8070640 0x7f0db8145020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42946/0 (socket says 172.21.15.7:42946) 2024-04-25T11:20:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.437+0000 7f0daf400640 1 -- 172.21.15.7:0/1297611203 learned_addr learned my addr 172.21.15.7:0/1297611203 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.438+0000 7f0dbd000640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db8159ae0 0x7f0db8147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.438+0000 7f0db7e00640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0db806d580 0x7f0db815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.438+0000 7f0daf400640 1 -- 172.21.15.7:0/1297611203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db8159ae0 msgr2=0x7f0db8147910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.438+0000 7f0daf400640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db8159ae0 0x7f0db8147910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.438+0000 7f0daf400640 1 -- 172.21.15.7:0/1297611203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0db806d580 msgr2=0x7f0db815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.438+0000 7f0daf400640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0db806d580 0x7f0db815c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.438+0000 7f0daf400640 1 -- 172.21.15.7:0/1297611203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db815d090 con 0x7f0db8070640 2024-04-25T11:20:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.438+0000 7f0daf400640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db8070640 0x7f0db8145020 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0da000b3f0 tx=0x7f0da000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.438+0000 7f0db4c00640 1 -- 172.21.15.7:0/1297611203 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0015020 con 0x7f0db8070640 2024-04-25T11:20:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.439+0000 7f0db4c00640 1 -- 172.21.15.7:0/1297611203 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da0011830 con 0x7f0db8070640 2024-04-25T11:20:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.439+0000 7f0db4c00640 1 -- 172.21.15.7:0/1297611203 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da0011c20 con 0x7f0db8070640 2024-04-25T11:20:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.439+0000 7f0dbda00640 1 -- 172.21.15.7:0/1297611203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db815d370 con 0x7f0db8070640 2024-04-25T11:20:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.439+0000 7f0dbda00640 1 -- 172.21.15.7:0/1297611203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db815d930 con 0x7f0db8070640 2024-04-25T11:20:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.440+0000 7f0dbda00640 1 -- 172.21.15.7:0/1297611203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db815db00 con 0x7f0db8070640 2024-04-25T11:20:10.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.441+0000 7f0db4c00640 1 -- 172.21.15.7:0/1297611203 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0da0018020 con 0x7f0db8070640 2024-04-25T11:20:10.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.442+0000 7f0db4c00640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744e0 0x7f0d940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.442+0000 7f0db4c00640 1 -- 172.21.15.7:0/1297611203 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f0da00950e0 con 0x7f0db8070640 2024-04-25T11:20:10.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.442+0000 7f0db7e00640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744e0 0x7f0d940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:10.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.443+0000 7f0db7e00640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744e0 0x7f0d940769f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0da8006fd0 tx=0x7f0da8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.454+0000 7f0db4c00640 1 -- 172.21.15.7:0/1297611203 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0da001f030 con 0x7f0db8070640 2024-04-25T11:20:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.698+0000 7f0dbda00640 1 -- 172.21.15.7:0/1297611203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0db81458f0 con 0x7f0db8070640 2024-04-25T11:20:10.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.699+0000 7f0db4c00640 1 -- 172.21.15.7:0/1297611203 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f0da005f640 con 0x7f0db8070640 2024-04-25T11:20:10.697 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:10.697 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:20:10.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.701+0000 7f0dbda00640 1 -- 172.21.15.7:0/1297611203 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744e0 msgr2=0x7f0d940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.701+0000 7f0dbda00640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744e0 0x7f0d940769f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0da8006fd0 tx=0x7f0da8008040 comp rx=0 tx=0).stop 2024-04-25T11:20:10.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.701+0000 7f0dbda00640 1 -- 172.21.15.7:0/1297611203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db8070640 msgr2=0x7f0db8145020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:10.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.701+0000 7f0dbda00640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db8070640 0x7f0db8145020 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0da000b3f0 tx=0x7f0da000b8b0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.702+0000 7f0dbda00640 1 -- 172.21.15.7:0/1297611203 shutdown_connections 2024-04-25T11:20:10.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.702+0000 7f0dbda00640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0d940744e0 0x7f0d940769f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.702+0000 7f0dbda00640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0db8159ae0 0x7f0db8147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.702+0000 7f0dbda00640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db8070640 0x7f0db8145020 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.702+0000 7f0dbda00640 1 --2- 172.21.15.7:0/1297611203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0db806d580 0x7f0db815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.702+0000 7f0dbda00640 1 -- 172.21.15.7:0/1297611203 >> 172.21.15.7:0/1297611203 conn(0x7f0db8073090 msgr2=0x7f0db80b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.702+0000 7f0dbda00640 1 -- 172.21.15.7:0/1297611203 shutdown_connections 2024-04-25T11:20:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.702+0000 7f0dbda00640 1 -- 172.21.15.7:0/1297611203 wait complete. 2024-04-25T11:20:10.726 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:20:10.726 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-04-25T11:20:10.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:20:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.998+0000 7f1d56e00640 1 Processor -- start 2024-04-25T11:20:10.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:10.998+0000 7f1d56e00640 1 -- start start 2024-04-25T11:20:10.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.000+0000 7f1d56e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d5015baa0 0x7f1d5015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.001+0000 7f1d56e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1d5006d580 0x7f1d5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:10.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.001+0000 7f1d56e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d5006ded0 con 0x7f1d50070640 2024-04-25T11:20:10.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.001+0000 7f1d56e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1d5015e980 con 0x7f1d5006d580 2024-04-25T11:20:10.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.001+0000 7f1d56e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5015ec60 con 0x7f1d5015baa0 2024-04-25T11:20:10.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.001+0000 7f1d55000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1d50070640 0x7f1d50161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49944/0 (socket says 172.21.15.7:49944) 2024-04-25T11:20:10.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.001+0000 7f1d55000640 1 -- 172.21.15.7:0/347140349 learned_addr learned my addr 172.21.15.7:0/347140349 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.001+0000 7f1d55a00640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d5015baa0 0x7f1d5015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.001+0000 7f1d56400640 1 --2- 172.21.15.7:0/347140349 >> v2:172.21.15.87:3300/0 conn(0x7f1d5006d580 0x7f1d5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.001+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 687940252 0 0) 0x7f1d3c001110 con 0x7f1d50070640 2024-04-25T11:20:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d38003520 con 0x7f1d50070640 2024-04-25T11:20:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 315576467 0 0) 0x7f1d38003520 con 0x7f1d50070640 2024-04-25T11:20:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d5015baa0 msgr2=0x7f1d5015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d5015baa0 0x7f1d5015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 >> v2:172.21.15.87:3300/0 conn(0x7f1d5006d580 msgr2=0x7f1d5006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 --2- 172.21.15.7:0/347140349 >> v2:172.21.15.87:3300/0 conn(0x7f1d5006d580 0x7f1d5006d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5015ef40 con 0x7f1d50070640 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1d3c001840 con 0x7f1d50070640 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 >> v1:172.21.15.7:6789/0 conn(0x7f1d50070640 legacy=0x7f1d50161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d380039b0 0x7f1d38003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d38004340 0x7f1d38004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38004f80 0x7f1d3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5015e980 con 0x7f1d38004f80 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5015ec60 con 0x7f1d38004340 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3800bf00 con 0x7f1d380039b0 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.002+0000 7f1d56400640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38004f80 0x7f1d3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.003+0000 7f1d55000640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d38004340 0x7f1d38004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.003+0000 7f1d55a00640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d380039b0 0x7f1d38003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.003+0000 7f1d56400640 1 -- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d380039b0 msgr2=0x7f1d38003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.003+0000 7f1d56400640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d380039b0 0x7f1d38003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.003+0000 7f1d56400640 1 -- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d38004340 msgr2=0x7f1d38004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.003+0000 7f1d56400640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d38004340 0x7f1d38004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.003+0000 7f1d56400640 1 -- 172.21.15.7:0/347140349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d38003520 con 0x7f1d38004f80 2024-04-25T11:20:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.003+0000 7f1d55a00640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d380039b0 0x7f1d38003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.003+0000 7f1d55000640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d38004340 0x7f1d38004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.004+0000 7f1d56400640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38004f80 0x7f1d3800b9c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1d48007330 tx=0x7f1d38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db13f46109d5b83f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.004+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48009be0 con 0x7f1d38004f80 2024-04-25T11:20:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.004+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d48036600 con 0x7f1d38004f80 2024-04-25T11:20:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.004+0000 7f1d4fe00640 1 -- 172.21.15.7:0/347140349 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d480369f0 con 0x7f1d38004f80 2024-04-25T11:20:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.005+0000 7f1d56e00640 1 -- 172.21.15.7:0/347140349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38004f80 msgr2=0x7f1d3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.005+0000 7f1d56e00640 1 --2- 172.21.15.7:0/347140349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d38004f80 0x7f1d3800b9c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1d48007330 tx=0x7f1d38005830 comp rx=0 tx=0).stop 2024-04-25T11:20:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.005+0000 7f1d56400640 1 -- 172.21.15.7:0/347140349 reap_dead start 2024-04-25T11:20:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.005+0000 7f1d56e00640 1 -- 172.21.15.7:0/347140349 shutdown_connections 2024-04-25T11:20:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.005+0000 7f1d56e00640 1 -- 172.21.15.7:0/347140349 >> 172.21.15.7:0/347140349 conn(0x7f1d50073090 msgr2=0x7f1d500734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.005+0000 7f1d56e00640 1 -- 172.21.15.7:0/347140349 shutdown_connections 2024-04-25T11:20:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.005+0000 7f1d56e00640 1 -- 172.21.15.7:0/347140349 wait complete. 2024-04-25T11:20:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.007+0000 7f1d56e00640 1 Processor -- start 2024-04-25T11:20:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.007+0000 7f1d56e00640 1 -- start start 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.009+0000 7f1d56e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d38004f80 0x7f1d5015a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.009+0000 7f1d56e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d500afcd0 0x7f1d500b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.009+0000 7f1d56e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d500b00e0 0x7f1d500b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.009+0000 7f1d56e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50070bd0 con 0x7f1d500afcd0 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.009+0000 7f1d56e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50070a50 con 0x7f1d500b00e0 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.009+0000 7f1d56e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1d501625f0 con 0x7f1d38004f80 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.009+0000 7f1d55000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d500afcd0 0x7f1d500b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.009+0000 7f1d55000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d500afcd0 0x7f1d500b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42960/0 (socket says 172.21.15.7:42960) 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.009+0000 7f1d55000640 1 -- 172.21.15.7:0/3596104758 learned_addr learned my addr 172.21.15.7:0/3596104758 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.010+0000 7f1d56400640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d500b00e0 0x7f1d500b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.010+0000 7f1d55a00640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d38004f80 0x7f1d5015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.010+0000 7f1d55000640 1 -- 172.21.15.7:0/3596104758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d38004f80 msgr2=0x7f1d5015a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.010+0000 7f1d55000640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d38004f80 0x7f1d5015a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.010+0000 7f1d55000640 1 -- 172.21.15.7:0/3596104758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d500b00e0 msgr2=0x7f1d500b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.010+0000 7f1d55000640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d500b00e0 0x7f1d500b2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.010+0000 7f1d55000640 1 -- 172.21.15.7:0/3596104758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d500b5740 con 0x7f1d500afcd0 2024-04-25T11:20:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.010+0000 7f1d55000640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d500afcd0 0x7f1d500b25e0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00d8b0 tx=0x7f1d3c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.010+0000 7f1d4d600640 1 -- 172.21.15.7:0/3596104758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c019070 con 0x7f1d500afcd0 2024-04-25T11:20:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.011+0000 7f1d4d600640 1 -- 172.21.15.7:0/3596104758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3c0074e0 con 0x7f1d500afcd0 2024-04-25T11:20:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.011+0000 7f1d56e00640 1 -- 172.21.15.7:0/3596104758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d500b5a20 con 0x7f1d500afcd0 2024-04-25T11:20:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.011+0000 7f1d4d600640 1 -- 172.21.15.7:0/3596104758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0049e0 con 0x7f1d500afcd0 2024-04-25T11:20:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.011+0000 7f1d56e00640 1 -- 172.21.15.7:0/3596104758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d500b5fe0 con 0x7f1d500afcd0 2024-04-25T11:20:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.012+0000 7f1d56e00640 1 -- 172.21.15.7:0/3596104758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d500d04a0 con 0x7f1d500afcd0 2024-04-25T11:20:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.013+0000 7f1d4d600640 1 -- 172.21.15.7:0/3596104758 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1d3c01c070 con 0x7f1d500afcd0 2024-04-25T11:20:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.014+0000 7f1d4d600640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1d300744b0 0x7f1d300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.014+0000 7f1d4d600640 1 -- 172.21.15.7:0/3596104758 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f1d3c094f70 con 0x7f1d500afcd0 2024-04-25T11:20:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.014+0000 7f1d55a00640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1d300744b0 0x7f1d300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.015+0000 7f1d55a00640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1d300744b0 0x7f1d300769c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1d440099c0 tx=0x7f1d44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:11.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.019+0000 7f1d4d600640 1 -- 172.21.15.7:0/3596104758 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1d500d04a0 con 0x7f1d500afcd0 2024-04-25T11:20:11.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.254+0000 7f1d56e00640 1 -- 172.21.15.7:0/3596104758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1d50074020 con 0x7f1d500afcd0 2024-04-25T11:20:11.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.255+0000 7f1d4d600640 1 -- 172.21.15.7:0/3596104758 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f1d5006ded0 con 0x7f1d500afcd0 2024-04-25T11:20:11.253 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:11.253 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:20:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.257+0000 7f1d56e00640 1 -- 172.21.15.7:0/3596104758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1d300744b0 msgr2=0x7f1d300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.257+0000 7f1d56e00640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1d300744b0 0x7f1d300769c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1d440099c0 tx=0x7f1d44008040 comp rx=0 tx=0).stop 2024-04-25T11:20:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.257+0000 7f1d56e00640 1 -- 172.21.15.7:0/3596104758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d500afcd0 msgr2=0x7f1d500b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.258+0000 7f1d56e00640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d500afcd0 0x7f1d500b25e0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00d8b0 tx=0x7f1d3c00dd70 comp rx=0 tx=0).stop 2024-04-25T11:20:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.258+0000 7f1d56e00640 1 -- 172.21.15.7:0/3596104758 shutdown_connections 2024-04-25T11:20:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.258+0000 7f1d56e00640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1d300744b0 0x7f1d300769c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.258+0000 7f1d56e00640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d500b00e0 0x7f1d500b2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.258+0000 7f1d56e00640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d500afcd0 0x7f1d500b25e0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.258+0000 7f1d56e00640 1 --2- 172.21.15.7:0/3596104758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1d38004f80 0x7f1d5015a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.258+0000 7f1d56e00640 1 -- 172.21.15.7:0/3596104758 >> 172.21.15.7:0/3596104758 conn(0x7f1d50073090 msgr2=0x7f1d500684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.258+0000 7f1d56e00640 1 -- 172.21.15.7:0/3596104758 shutdown_connections 2024-04-25T11:20:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11.258+0000 7f1d56e00640 1 -- 172.21.15.7:0/3596104758 wait complete. 2024-04-25T11:20:11.282 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:20:11.282 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-04-25T11:20:11.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-25T11:20:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:20:11.861 INFO:teuthology.orchestra.run.smithi007.stderr:stdin closed, goodbye! 2024-04-25T11:20:11.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:11.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.262+0000 7f5c0a400640 1 Processor -- start 2024-04-25T11:20:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.262+0000 7f5c0a400640 1 -- start start 2024-04-25T11:20:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.264+0000 7f5c0a400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c040afb30 0x7f5c0415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.264+0000 7f5c0a400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c040cebb0 con 0x7f5c040b5220 2024-04-25T11:20:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.265+0000 7f5c0a400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c040b64b0 con 0x7f5c040aed40 2024-04-25T11:20:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.265+0000 7f5c0a400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0415e1f0 con 0x7f5c040afb30 2024-04-25T11:20:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.265+0000 7f5c03e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5c040b5220 0x7f5c040af420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49958/0 (socket says 172.21.15.7:49958) 2024-04-25T11:20:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.265+0000 7f5c03e00640 1 -- 172.21.15.7:0/3877343199 learned_addr learned my addr 172.21.15.7:0/3877343199 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:12.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.265+0000 7f5c09a00640 1 --2- 172.21.15.7:0/3877343199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c040afb30 0x7f5c0415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.265+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005235006 0 0) 0x7f5bf0001110 con 0x7f5c040b5220 2024-04-25T11:20:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.265+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5be8003520 con 0x7f5c040b5220 2024-04-25T11:20:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3926626329 0 0) 0x7f5be8003520 con 0x7f5c040b5220 2024-04-25T11:20:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c040afb30 msgr2=0x7f5c0415dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 --2- 172.21.15.7:0/3877343199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c040afb30 0x7f5c0415dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 >> v1:172.21.15.87:6789/0 conn(0x7f5c040aed40 legacy=0x7f5c040ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0406c610 con 0x7f5c040b5220 2024-04-25T11:20:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f5bf0001840 con 0x7f5c040b5220 2024-04-25T11:20:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 >> v1:172.21.15.7:6789/0 conn(0x7f5c040b5220 legacy=0x7f5c040af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 --2- 172.21.15.7:0/3877343199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5be80039e0 0x7f5be8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 --2- 172.21.15.7:0/3877343199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5be8004800 0x7f5be8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 --2- 172.21.15.7:0/3877343199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be8005150 0x7f5be800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0415e1f0 con 0x7f5be80039e0 2024-04-25T11:20:12.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5be800bf50 con 0x7f5be8005150 2024-04-25T11:20:12.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5be800c150 con 0x7f5be8004800 2024-04-25T11:20:12.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.266+0000 7f5c09000640 1 --2- 172.21.15.7:0/3877343199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5be80039e0 0x7f5be8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.267+0000 7f5c09000640 1 -- 172.21.15.7:0/3877343199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5be8004800 msgr2=0x7f5be8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.267+0000 7f5c09000640 1 --2- 172.21.15.7:0/3877343199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5be8004800 0x7f5be8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.267+0000 7f5c09000640 1 -- 172.21.15.7:0/3877343199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be8005150 msgr2=0x7f5be800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.267+0000 7f5c09000640 1 --2- 172.21.15.7:0/3877343199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be8005150 0x7f5be800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.267+0000 7f5c09000640 1 -- 172.21.15.7:0/3877343199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be8003520 con 0x7f5be80039e0 2024-04-25T11:20:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.268+0000 7f5c09000640 1 --2- 172.21.15.7:0/3877343199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5be80039e0 0x7f5be8003e50 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4001720 tx=0x7f5be8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ffa49e7b2366e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.268+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4001370 con 0x7f5be80039e0 2024-04-25T11:20:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.268+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf40052d0 con 0x7f5be80039e0 2024-04-25T11:20:12.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.268+0000 7f5c03400640 1 -- 172.21.15.7:0/3877343199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf402abc0 con 0x7f5be80039e0 2024-04-25T11:20:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.269+0000 7f5c0a400640 1 -- 172.21.15.7:0/3877343199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5be80039e0 msgr2=0x7f5be8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.269+0000 7f5c0a400640 1 --2- 172.21.15.7:0/3877343199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5be80039e0 0x7f5be8003e50 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4001720 tx=0x7f5be8005830 comp rx=0 tx=0).stop 2024-04-25T11:20:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.269+0000 7f5c09a00640 1 -- 172.21.15.7:0/3877343199 reap_dead start 2024-04-25T11:20:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.269+0000 7f5c0a400640 1 -- 172.21.15.7:0/3877343199 shutdown_connections 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.269+0000 7f5c0a400640 1 -- 172.21.15.7:0/3877343199 >> 172.21.15.7:0/3877343199 conn(0x7f5c040b3820 msgr2=0x7f5c040b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.269+0000 7f5c0a400640 1 -- 172.21.15.7:0/3877343199 shutdown_connections 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.269+0000 7f5c0a400640 1 -- 172.21.15.7:0/3877343199 wait complete. 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.271+0000 7f5c0a400640 1 Processor -- start 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.271+0000 7f5c0a400640 1 -- start start 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.273+0000 7f5c0a400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be80039e0 0x7f5c0414f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.273+0000 7f5c0a400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0406f7b0 0x7f5c040caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.273+0000 7f5c0a400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0406fbc0 0x7f5c040caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.273+0000 7f5c0a400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0406ceb0 con 0x7f5c0406f7b0 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.273+0000 7f5c0a400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0406cbd0 con 0x7f5be80039e0 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.273+0000 7f5c0a400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c040b6110 con 0x7f5c0406fbc0 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.273+0000 7f5c03e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0406f7b0 0x7f5c040caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.273+0000 7f5c03e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0406f7b0 0x7f5c040caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42976/0 (socket says 172.21.15.7:42976) 2024-04-25T11:20:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.273+0000 7f5c03e00640 1 -- 172.21.15.7:0/2374723687 learned_addr learned my addr 172.21.15.7:0/2374723687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.273+0000 7f5c09000640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be80039e0 0x7f5c0414f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c09a00640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0406fbc0 0x7f5c040caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c03e00640 1 -- 172.21.15.7:0/2374723687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0406fbc0 msgr2=0x7f5c040caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c03e00640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0406fbc0 0x7f5c040caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c03e00640 1 -- 172.21.15.7:0/2374723687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be80039e0 msgr2=0x7f5c0414f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c03e00640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be80039e0 0x7f5c0414f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c03e00640 1 -- 172.21.15.7:0/2374723687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c040cb530 con 0x7f5c0406f7b0 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c09a00640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0406fbc0 0x7f5c040caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c03e00640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0406f7b0 0x7f5c040caab0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000c6c0 tx=0x7f5bf000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c00c00640 1 -- 172.21.15.7:0/2374723687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0018070 con 0x7f5c0406f7b0 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c00c00640 1 -- 172.21.15.7:0/2374723687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf00072d0 con 0x7f5c0406f7b0 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c00c00640 1 -- 172.21.15.7:0/2374723687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf00047e0 con 0x7f5c0406f7b0 2024-04-25T11:20:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c0a400640 1 -- 172.21.15.7:0/2374723687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c040cb7f0 con 0x7f5c0406f7b0 2024-04-25T11:20:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.274+0000 7f5c0a400640 1 -- 172.21.15.7:0/2374723687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c040cbdb0 con 0x7f5c0406f7b0 2024-04-25T11:20:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.275+0000 7f5c0a400640 1 -- 172.21.15.7:0/2374723687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c040b45b0 con 0x7f5c0406f7b0 2024-04-25T11:20:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.277+0000 7f5c00c00640 1 -- 172.21.15.7:0/2374723687 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5bf0004980 con 0x7f5c0406f7b0 2024-04-25T11:20:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.278+0000 7f5c00c00640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be40744b0 0x7f5be40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:12.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.278+0000 7f5c00c00640 1 -- 172.21.15.7:0/2374723687 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f5bf0094b00 con 0x7f5c0406f7b0 2024-04-25T11:20:12.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.278+0000 7f5c09000640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be40744b0 0x7f5be40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:12.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.279+0000 7f5c09000640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be40744b0 0x7f5be40769c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4032000 tx=0x7f5bf402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.282+0000 7f5c00c00640 1 -- 172.21.15.7:0/2374723687 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5c040b45b0 con 0x7f5c0406f7b0 2024-04-25T11:20:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.519+0000 7f5c0a400640 1 -- 172.21.15.7:0/2374723687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c04050740 con 0x7f5c0406f7b0 2024-04-25T11:20:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.520+0000 7f5c00c00640 1 -- 172.21.15.7:0/2374723687 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+5174 (secure 0 0 0) 0x7f5c040cebb0 con 0x7f5c0406f7b0 2024-04-25T11:20:12.519 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:12.519 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/326382713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":326382713},{"type":"v1","addr":"172.21.15.7:6837","nonce":326382713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2007810270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2007810270},{"type":"v1","addr":"172.21.15.87:6835","nonce":2007810270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1753656512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1753656512},{"type":"v1","addr":"172.21.15.7:6836","nonce":1753656512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:47.807534+0000","modified":"2024-04-25T11:20:08.325402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":8,"addr":"172.21.15.87:6834/1924056839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1924056839},{"type":"v1","addr":"172.21.15.87:6834","nonce":1924056839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8007,"qdb_cluster":[8007]},"id":8}]} 2024-04-25T11:20:12.519 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2024-04-25T11:20:12.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.523+0000 7f5c0a400640 1 -- 172.21.15.7:0/2374723687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be40744b0 msgr2=0x7f5be40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.523+0000 7f5c0a400640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be40744b0 0x7f5be40769c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4032000 tx=0x7f5bf402e000 comp rx=0 tx=0).stop 2024-04-25T11:20:12.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.523+0000 7f5c0a400640 1 -- 172.21.15.7:0/2374723687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0406f7b0 msgr2=0x7f5c040caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:12.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.523+0000 7f5c0a400640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0406f7b0 0x7f5c040caab0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5bf000c6c0 tx=0x7f5bf000cb80 comp rx=0 tx=0).stop 2024-04-25T11:20:12.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.523+0000 7f5c0a400640 1 -- 172.21.15.7:0/2374723687 shutdown_connections 2024-04-25T11:20:12.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.523+0000 7f5c0a400640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5be40744b0 0x7f5be40769c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.523+0000 7f5c0a400640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c0406fbc0 0x7f5c040caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.523+0000 7f5c0a400640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0406f7b0 0x7f5c040caab0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.523+0000 7f5c0a400640 1 --2- 172.21.15.7:0/2374723687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be80039e0 0x7f5c0414f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:12.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.523+0000 7f5c0a400640 1 -- 172.21.15.7:0/2374723687 >> 172.21.15.7:0/2374723687 conn(0x7f5c040b3820 msgr2=0x7f5c040cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:12.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.524+0000 7f5c0a400640 1 -- 172.21.15.7:0/2374723687 shutdown_connections 2024-04-25T11:20:12.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:12.524+0000 7f5c0a400640 1 -- 172.21.15.7:0/2374723687 wait complete. 2024-04-25T11:20:12.550 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-04-25T11:20:12.850 INFO:teuthology.orchestra.run.smithi087.stdout:[{"id":6098,"entity":{"name":{"type":"client","num":6098},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":1133632146}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":13.794693956,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9258948464614818,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6098 192.168.0.1:0/1133632146","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi087","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":8157,"entity":{"name":{"type":"client","num":8157},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":128224496}},"state":"open","num_leases":1,"num_caps":402,"request_load_avg":750,"uptime":15.607653736,"requests_in_flight":0,"num_completed_requests":299,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":2889.3117087989899,"halflife":60},"release_caps":{"value":398.40083299350511,"halflife":60},"recall_caps_throttle":{"value":843.33602168075197,"halflife":1.5},"recall_caps_throttle2o":{"value":269.02943062274619,"halflife":0.5},"session_cache_liveness":{"value":1574.6691044567078,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":530,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.8157 192.168.0.1:0/128224496","completed_requests":[{"tid":530,"created_ino":"0x100000003e9"},{"tid":531,"created_ino":"0x100000003ea"},{"tid":532,"created_ino":"0x100000003eb"},{"tid":533,"created_ino":"0x100000003ec"},{"tid":534,"created_ino":"0x100000003ed"},{"tid":535,"created_ino":"0x100000003ee"},{"tid":536,"created_ino":"0x100000003ef"},{"tid":537,"created_ino":"0x100000003f0"},{"tid":538,"created_ino":"0x100000003f1"},{"tid":539,"created_ino":"0x100000003f2"},{"tid":540,"created_ino":"0x100000003f3"},{"tid":541,"created_ino":"0x100000003f4"},{"tid":542,"created_ino":"0x100000003f5"},{"tid":543,"created_ino":"0x100000003f6"},{"tid":544,"created_ino":"0x100000003f7"},{"tid":545,"created_ino":"0x100000003f8"},{"tid":546,"created_ino":"0x100000003f9"},{"tid":547,"created_ino":"0x100000003fa"},{"tid":548,"created_ino":"0x100000003fb"},{"tid":549,"created_ino":"0x100000003fc"},{"tid":550,"created_ino":"0x100000003fd"},{"tid":551,"created_ino":"0x100000003fe"},{"tid":552,"created_ino":"0x100000003ff"},{"tid":553,"created_ino":"0x10000000400"},{"tid":554,"created_ino":"0x10000000401"},{"tid":555,"created_ino":"0x10000000402"},{"tid":556,"created_ino":"0x10000000403"},{"tid":557,"created_ino":"0x10000000404"},{"tid":558,"created_ino":"0x10000000405"},{"tid":559,"created_ino":"0x10000000406"},{"tid":560,"created_ino":"0x10000000407"},{"tid":561,"created_ino":"0x10000000408"},{"tid":562,"created_ino":"0x10000000409"},{"tid":563,"created_ino":"0x1000000040a"},{"tid":564,"created_ino":"0x1000000040b"},{"tid":565,"created_ino":"0x1000000040c"},{"tid":566,"created_ino":"0x1000000040d"},{"tid":567,"created_ino":"0x1000000040e"},{"tid":568,"created_ino":"0x1000000040f"},{"tid":569,"created_ino":"0x10000000410"},{"tid":570,"created_ino":"0x10000000411"},{"tid":571,"created_ino":"0x10000000412"},{"tid":572,"created_ino":"0x10000000413"},{"tid":573,"created_ino":"0x10000000414"},{"tid":574,"created_ino":"0x10000000415"},{"tid":575,"created_ino":"0x10000000416"},{"tid":576,"created_ino":"0x10000000417"},{"tid":577,"created_ino":"0x10000000418"},{"tid":578,"created_ino":"0x10000000419"},{"tid":579,"created_ino":"0x1000000041a"},{"tid":580,"created_ino":"0x1000000041b"},{"tid":581,"created_ino":"0x1000000041c"},{"tid":582,"created_ino":"0x1000000041d"},{"tid":583,"created_ino":"0x1000000041e"},{"tid":584,"created_ino":"0x1000000041f"},{"tid":585,"created_ino":"0x10000000420"},{"tid":586,"created_ino":"0x10000000421"},{"tid":587,"created_ino":"0x10000000422"},{"tid":588,"created_ino":"0x10000000423"},{"tid":589,"created_ino":"0x10000000424"},{"tid":590,"created_ino":"0x10000000425"},{"tid":591,"created_ino":"0x10000000426"},{"tid":592,"created_ino":"0x10000000427"},{"tid":593,"created_ino":"0x10000000428"},{"tid":594,"created_ino":"0x10000000429"},{"tid":595,"created_ino":"0x1000000042a"},{"tid":596,"created_ino":"0x1000000042b"},{"tid":597,"created_ino":"0x1000000042c"},{"tid":598,"created_ino":"0x1000000042d"},{"tid":599,"created_ino":"0x1000000042e"},{"tid":600,"created_ino":"0x1000000042f"},{"tid":601,"created_ino":"0x10000000430"},{"tid":602,"created_ino":"0x10000000431"},{"tid":603,"created_ino":"0x10000000432"},{"tid":604,"created_ino":"0x10000000433"},{"tid":605,"created_ino":"0x10000000434"},{"tid":606,"created_ino":"0x10000000435"},{"tid":607,"created_ino":"0x10000000436"},{"tid":608,"created_ino":"0x10000000437"},{"tid":609,"created_ino":"0x10000000438"},{"tid":610,"created_ino":"0x10000000439"},{"tid":611,"created_ino":"0x1000000043a"},{"tid":612,"created_ino":"0x1000000043b"},{"tid":613,"created_ino":"0x1000000043c"},{"tid":614,"created_ino":"0x1000000043d"},{"tid":615,"created_ino":"0x1000000043e"},{"tid":616,"created_ino":"0x1000000043f"},{"tid":617,"created_ino":"0x10000000440"},{"tid":618,"created_ino":"0x10000000441"},{"tid":619,"created_ino":"0x10000000442"},{"tid":620,"created_ino":"0x10000000443"},{"tid":621,"created_ino":"0x10000000444"},{"tid":622,"created_ino":"0x10000000445"},{"tid":623,"created_ino":"0x10000000446"},{"tid":624,"created_ino":"0x10000000447"},{"tid":625,"created_ino":"0x10000000448"},{"tid":626,"created_ino":"0x10000000449"},{"tid":627,"created_ino":"0x1000000044a"},{"tid":628,"created_ino":"0x1000000044b"},{"tid":629,"created_ino":"0x1000000044c"},{"tid":630,"created_ino":"0x1000000044d"},{"tid":631,"created_ino":"0x1000000044e"},{"tid":632,"created_ino":"0x1000000044f"},{"tid":633,"created_ino":"0x10000000450"},{"tid":634,"created_ino":"0x10000000451"},{"tid":635,"created_ino":"0x10000000452"},{"tid":636,"created_ino":"0x10000000453"},{"tid":637,"created_ino":"0x10000000454"},{"tid":638,"created_ino":"0x10000000455"},{"tid":639,"created_ino":"0x10000000456"},{"tid":640,"created_ino":"0x10000000457"},{"tid":641,"created_ino":"0x10000000458"},{"tid":642,"created_ino":"0x10000000459"},{"tid":643,"created_ino":"0x1000000045a"},{"tid":644,"created_ino":"0x1000000045b"},{"tid":645,"created_ino":"0x1000000045c"},{"tid":646,"created_ino":"0x1000000045d"},{"tid":647,"created_ino":"0x1000000045e"},{"tid":648,"created_ino":"0x1000000045f"},{"tid":649,"created_ino":"0x10000000460"},{"tid":650,"created_ino":"0x10000000461"},{"tid":651,"created_ino":"0x10000000462"},{"tid":652,"created_ino":"0x10000000463"},{"tid":653,"created_ino":"0x10000000464"},{"tid":654,"created_ino":"0x10000000465"},{"tid":655,"created_ino":"0x10000000466"},{"tid":656,"created_ino":"0x10000000467"},{"tid":657,"created_ino":"0x10000000468"},{"tid":658,"created_ino":"0x10000000469"},{"tid":659,"created_ino":"0x1000000046a"},{"tid":660,"created_ino":"0x1000000046b"},{"tid":661,"created_ino":"0x1000000046c"},{"tid":662,"created_ino":"0x1000000046d"},{"tid":663,"created_ino":"0x1000000046e"},{"tid":664,"created_ino":"0x1000000046f"},{"tid":665,"created_ino":"0x10000000470"},{"tid":666,"created_ino":"0x10000000471"},{"tid":667,"created_ino":"0x10000000472"},{"tid":668,"created_ino":"0x10000000473"},{"tid":669,"created_ino":"0x10000000474"},{"tid":670,"created_ino":"0x10000000475"},{"tid":671,"created_ino":"0x10000000476"},{"tid":672,"created_ino":"0x10000000477"},{"tid":673,"created_ino":"0x10000000478"},{"tid":674,"created_ino":"0x10000000479"},{"tid":675,"created_ino":"0x1000000047a"},{"tid":676,"created_ino":"0x1000000047b"},{"tid":677,"created_ino":"0x1000000047c"},{"tid":678,"created_ino":"0x1000000047d"},{"tid":679,"created_ino":"0x1000000047e"},{"tid":680,"created_ino":"0x1000000047f"},{"tid":681,"created_ino":"0x10000000480"},{"tid":682,"created_ino":"0x10000000481"},{"tid":683,"created_ino":"0x10000000482"},{"tid":684,"created_ino":"0x10000000483"},{"tid":685,"created_ino":"0x10000000484"},{"tid":686,"created_ino":"0x10000000485"},{"tid":687,"created_ino":"0x10000000486"},{"tid":688,"created_ino":"0x10000000487"},{"tid":689,"created_ino":"0x10000000488"},{"tid":690,"created_ino":"0x10000000489"},{"tid":691,"created_ino":"0x1000000048a"},{"tid":692,"created_ino":"0x1000000048b"},{"tid":693,"created_ino":"0x1000000048c"},{"tid":694,"created_ino":"0x1000000048d"},{"tid":695,"created_ino":"0x1000000048e"},{"tid":696,"created_ino":"0x1000000048f"},{"tid":697,"created_ino":"0x10000000490"},{"tid":698,"created_ino":"0x10000000491"},{"tid":699,"created_ino":"0x10000000492"},{"tid":700,"created_ino":"0x10000000493"},{"tid":701,"created_ino":"0x10000000494"},{"tid":702,"created_ino":"0x10000000495"},{"tid":703,"created_ino":"0x10000000496"},{"tid":704,"created_ino":"0x10000000497"},{"tid":705,"created_ino":"0x10000000498"},{"tid":706,"created_ino":"0x10000000499"},{"tid":707,"created_ino":"0x1000000049a"},{"tid":708,"created_ino":"0x1000000049b"},{"tid":709,"created_ino":"0x1000000049c"},{"tid":710,"created_ino":"0x1000000049d"},{"tid":711,"created_ino":"0x1000000049e"},{"tid":712,"created_ino":"0x1000000049f"},{"tid":713,"created_ino":"0x100000004a0"},{"tid":714,"created_ino":"0x100000004a1"},{"tid":715,"created_ino":"0x100000004a2"},{"tid":716,"created_ino":"0x100000004a3"},{"tid":717,"created_ino":"0x100000004a4"},{"tid":718,"created_ino":"0x100000004a5"},{"tid":719,"created_ino":"0x100000004a6"},{"tid":720,"created_ino":"0x100000004a7"},{"tid":721,"created_ino":"0x100000004a8"},{"tid":722,"created_ino":"0x100000004a9"},{"tid":723,"created_ino":"0x100000004aa"},{"tid":724,"created_ino":"0x100000004ab"},{"tid":725,"created_ino":"0x100000004ac"},{"tid":726,"created_ino":"0x100000004ad"},{"tid":727,"created_ino":"0x100000004ae"},{"tid":728,"created_ino":"0x100000004af"},{"tid":729,"created_ino":"0x100000004b0"},{"tid":730,"created_ino":"0x100000004b1"},{"tid":731,"created_ino":"0x100000004b2"},{"tid":732,"created_ino":"0x100000004b3"},{"tid":733,"created_ino":"0x100000004b4"},{"tid":734,"created_ino":"0x100000004b5"},{"tid":735,"created_ino":"0x100000004b6"},{"tid":736,"created_ino":"0x100000004b7"},{"tid":737,"created_ino":"0x100000004b8"},{"tid":738,"created_ino":"0x100000004b9"},{"tid":739,"created_ino":"0x100000004ba"},{"tid":740,"created_ino":"0x100000004bb"},{"tid":741,"created_ino":"0x100000004bc"},{"tid":742,"created_ino":"0x100000004bd"},{"tid":743,"created_ino":"0x100000004be"},{"tid":744,"created_ino":"0x100000004bf"},{"tid":745,"created_ino":"0x100000004c0"},{"tid":746,"created_ino":"0x100000004c1"},{"tid":747,"created_ino":"0x100000004c2"},{"tid":748,"created_ino":"0x100000004c3"},{"tid":749,"created_ino":"0x100000004c4"},{"tid":750,"created_ino":"0x100000004c5"},{"tid":751,"created_ino":"0x100000004c6"},{"tid":752,"created_ino":"0x100000004c7"},{"tid":753,"created_ino":"0x100000004c8"},{"tid":754,"created_ino":"0x100000004c9"},{"tid":755,"created_ino":"0x100000004ca"},{"tid":756,"created_ino":"0x100000004cb"},{"tid":757,"created_ino":"0x100000004cc"},{"tid":758,"created_ino":"0x100000004cd"},{"tid":759,"created_ino":"0x100000004ce"},{"tid":760,"created_ino":"0x100000004cf"},{"tid":761,"created_ino":"0x100000004d0"},{"tid":762,"created_ino":"0x100000004d1"},{"tid":763,"created_ino":"0x100000004d2"},{"tid":764,"created_ino":"0x100000004d3"},{"tid":765,"created_ino":"0x100000004d4"},{"tid":766,"created_ino":"0x100000004d5"},{"tid":767,"created_ino":"0x100000004d6"},{"tid":768,"created_ino":"0x100000004d7"},{"tid":769,"created_ino":"0x100000004d8"},{"tid":770,"created_ino":"0x100000004d9"},{"tid":771,"created_ino":"0x100000004da"},{"tid":772,"created_ino":"0x100000004db"},{"tid":773,"created_ino":"0x100000004dc"},{"tid":774,"created_ino":"0x100000004dd"},{"tid":775,"created_ino":"0x100000004de"},{"tid":776,"created_ino":"0x100000004df"},{"tid":777,"created_ino":"0x100000004e0"},{"tid":778,"created_ino":"0x100000004e1"},{"tid":779,"created_ino":"0x100000004e2"},{"tid":780,"created_ino":"0x100000004e3"},{"tid":781,"created_ino":"0x100000004e4"},{"tid":782,"created_ino":"0x100000004e5"},{"tid":783,"created_ino":"0x100000004e6"},{"tid":784,"created_ino":"0x100000004e7"},{"tid":785,"created_ino":"0x100000004e8"},{"tid":786,"created_ino":"0x100000004e9"},{"tid":787,"created_ino":"0x100000004ea"},{"tid":788,"created_ino":"0x100000004eb"},{"tid":789,"created_ino":"0x100000004ec"},{"tid":790,"created_ino":"0x100000004ed"},{"tid":791,"created_ino":"0x100000004ee"},{"tid":792,"created_ino":"0x100000004ef"},{"tid":793,"created_ino":"0x100000004f0"},{"tid":794,"created_ino":"0x100000004f1"},{"tid":795,"created_ino":"0x100000004f2"},{"tid":796,"created_ino":"0x100000004f3"},{"tid":797,"created_ino":"0x100000004f4"},{"tid":798,"created_ino":"0x100000004f5"},{"tid":799,"created_ino":"0x100000004f6"},{"tid":800,"created_ino":"0x100000004f7"},{"tid":801,"created_ino":"0x100000004f8"},{"tid":802,"created_ino":"0x100000004f9"},{"tid":803,"created_ino":"0x100000004fa"},{"tid":804,"created_ino":"0x100000004fb"},{"tid":805,"created_ino":"0x100000004fc"},{"tid":806,"created_ino":"0x100000004fd"},{"tid":807,"created_ino":"0x100000004fe"},{"tid":808,"created_ino":"0x100000004ff"},{"tid":809,"created_ino":"0x10000000500"},{"tid":810,"created_ino":"0x10000000501"},{"tid":811,"created_ino":"0x10000000502"},{"tid":812,"created_ino":"0x10000000503"},{"tid":813,"created_ino":"0x10000000504"},{"tid":814,"created_ino":"0x10000000505"},{"tid":815,"created_ino":"0x10000000506"},{"tid":816,"created_ino":"0x10000000507"},{"tid":817,"created_ino":"0x10000000508"},{"tid":818,"created_ino":"0x10000000509"},{"tid":819,"created_ino":"0x1000000050a"},{"tid":820,"created_ino":"0x1000000050b"},{"tid":821,"created_ino":"0x1000000050c"},{"tid":822,"created_ino":"0x1000000050d"},{"tid":823,"created_ino":"0x1000000050e"},{"tid":824,"created_ino":"0x1000000050f"},{"tid":825,"created_ino":"0x10000000510"},{"tid":826,"created_ino":"0x10000000511"},{"tid":827,"created_ino":"0x10000000512"},{"tid":828,"created_ino":"0x10000000513"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000514","length":203},{"start":"0x100000005e0","length":501}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi007","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T11:20:12.853 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi087", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 1133632146, "type": "any" }, "name": { "num": 6098, "type": "client" } }, "id": 6098, "inst": "client.6098 192.168.0.1:0/1133632146", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.925894846461482 }, "state": "open", "uptime": 13.794693956 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi007", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x100000003e9", "tid": 530 }, { "created_ino": "0x100000003ea", "tid": 531 }, { "created_ino": "0x100000003eb", "tid": 532 }, { "created_ino": "0x100000003ec", "tid": 533 }, { "created_ino": "0x100000003ed", "tid": 534 }, { "created_ino": "0x100000003ee", "tid": 535 }, { "created_ino": "0x100000003ef", "tid": 536 }, { "created_ino": "0x100000003f0", "tid": 537 }, { "created_ino": "0x100000003f1", "tid": 538 }, { "created_ino": "0x100000003f2", "tid": 539 }, { "created_ino": "0x100000003f3", "tid": 540 }, { "created_ino": "0x100000003f4", "tid": 541 }, { "created_ino": "0x100000003f5", "tid": 542 }, { "created_ino": "0x100000003f6", "tid": 543 }, { "created_ino": "0x100000003f7", "tid": 544 }, { "created_ino": "0x100000003f8", "tid": 545 }, { "created_ino": "0x100000003f9", "tid": 546 }, { "created_ino": "0x100000003fa", "tid": 547 }, { "created_ino": "0x100000003fb", "tid": 548 }, { "created_ino": "0x100000003fc", "tid": 549 }, { "created_ino": "0x100000003fd", "tid": 550 }, { "created_ino": "0x100000003fe", "tid": 551 }, { "created_ino": "0x100000003ff", "tid": 552 }, { "created_ino": "0x10000000400", "tid": 553 }, { "created_ino": "0x10000000401", "tid": 554 }, { "created_ino": "0x10000000402", "tid": 555 }, { "created_ino": "0x10000000403", "tid": 556 }, { "created_ino": "0x10000000404", "tid": 557 }, { "created_ino": "0x10000000405", "tid": 558 }, { "created_ino": "0x10000000406", "tid": 559 }, { "created_ino": "0x10000000407", "tid": 560 }, { "created_ino": "0x10000000408", "tid": 561 }, { "created_ino": "0x10000000409", "tid": 562 }, { "created_ino": "0x1000000040a", "tid": 563 }, { "created_ino": "0x1000000040b", "tid": 564 }, { "created_ino": "0x1000000040c", "tid": 565 }, { "created_ino": "0x1000000040d", "tid": 566 }, { "created_ino": "0x1000000040e", "tid": 567 }, { "created_ino": "0x1000000040f", "tid": 568 }, { "created_ino": "0x10000000410", "tid": 569 }, { "created_ino": "0x10000000411", "tid": 570 }, { "created_ino": "0x10000000412", "tid": 571 }, { "created_ino": "0x10000000413", "tid": 572 }, { "created_ino": "0x10000000414", "tid": 573 }, { "created_ino": "0x10000000415", "tid": 574 }, { "created_ino": "0x10000000416", "tid": 575 }, { "created_ino": "0x10000000417", "tid": 576 }, { "created_ino": "0x10000000418", "tid": 577 }, { "created_ino": "0x10000000419", "tid": 578 }, { "created_ino": "0x1000000041a", "tid": 579 }, { "created_ino": "0x1000000041b", "tid": 580 }, { "created_ino": "0x1000000041c", "tid": 581 }, { "created_ino": "0x1000000041d", "tid": 582 }, { "created_ino": "0x1000000041e", "tid": 583 }, { "created_ino": "0x1000000041f", "tid": 584 }, { "created_ino": "0x10000000420", "tid": 585 }, { "created_ino": "0x10000000421", "tid": 586 }, { "created_ino": "0x10000000422", "tid": 587 }, { "created_ino": "0x10000000423", "tid": 588 }, { "created_ino": "0x10000000424", "tid": 589 }, { "created_ino": "0x10000000425", "tid": 590 }, { "created_ino": "0x10000000426", "tid": 591 }, { "created_ino": "0x10000000427", "tid": 592 }, { "created_ino": "0x10000000428", "tid": 593 }, { "created_ino": "0x10000000429", "tid": 594 }, { "created_ino": "0x1000000042a", "tid": 595 }, { "created_ino": "0x1000000042b", "tid": 596 }, { "created_ino": "0x1000000042c", "tid": 597 }, { "created_ino": "0x1000000042d", "tid": 598 }, { "created_ino": "0x1000000042e", "tid": 599 }, { "created_ino": "0x1000000042f", "tid": 600 }, { "created_ino": "0x10000000430", "tid": 601 }, { "created_ino": "0x10000000431", "tid": 602 }, { "created_ino": "0x10000000432", "tid": 603 }, { "created_ino": "0x10000000433", "tid": 604 }, { "created_ino": "0x10000000434", "tid": 605 }, { "created_ino": "0x10000000435", "tid": 606 }, { "created_ino": "0x10000000436", "tid": 607 }, { "created_ino": "0x10000000437", "tid": 608 }, { "created_ino": "0x10000000438", "tid": 609 }, { "created_ino": "0x10000000439", "tid": 610 }, { "created_ino": "0x1000000043a", "tid": 611 }, { "created_ino": "0x1000000043b", "tid": 612 }, { "created_ino": "0x1000000043c", "tid": 613 }, { "created_ino": "0x1000000043d", "tid": 614 }, { "created_ino": "0x1000000043e", "tid": 615 }, { "created_ino": "0x1000000043f", "tid": 616 }, { "created_ino": "0x10000000440", "tid": 617 }, { "created_ino": "0x10000000441", "tid": 618 }, { "created_ino": "0x10000000442", "tid": 619 }, { "created_ino": "0x10000000443", "tid": 620 }, { "created_ino": "0x10000000444", "tid": 621 }, { "created_ino": "0x10000000445", "tid": 622 }, { "created_ino": "0x10000000446", "tid": 623 }, { "created_ino": "0x10000000447", "tid": 624 }, { "created_ino": "0x10000000448", "tid": 625 }, { "created_ino": "0x10000000449", "tid": 626 }, { "created_ino": "0x1000000044a", "tid": 627 }, { "created_ino": "0x1000000044b", "tid": 628 }, { "created_ino": "0x1000000044c", "tid": 629 }, { "created_ino": "0x1000000044d", "tid": 630 }, { "created_ino": "0x1000000044e", "tid": 631 }, { "created_ino": "0x1000000044f", "tid": 632 }, { "created_ino": "0x10000000450", "tid": 633 }, { "created_ino": "0x10000000451", "tid": 634 }, { "created_ino": "0x10000000452", "tid": 635 }, { "created_ino": "0x10000000453", "tid": 636 }, { "created_ino": "0x10000000454", "tid": 637 }, { "created_ino": "0x10000000455", "tid": 638 }, { "created_ino": "0x10000000456", "tid": 639 }, { "created_ino": "0x10000000457", "tid": 640 }, { "created_ino": "0x10000000458", "tid": 641 }, { "created_ino": "0x10000000459", "tid": 642 }, { "created_ino": "0x1000000045a", "tid": 643 }, { "created_ino": "0x1000000045b", "tid": 644 }, { "created_ino": "0x1000000045c", "tid": 645 }, { "created_ino": "0x1000000045d", "tid": 646 }, { "created_ino": "0x1000000045e", "tid": 647 }, { "created_ino": "0x1000000045f", "tid": 648 }, { "created_ino": "0x10000000460", "tid": 649 }, { "created_ino": "0x10000000461", "tid": 650 }, { "created_ino": "0x10000000462", "tid": 651 }, { "created_ino": "0x10000000463", "tid": 652 }, { "created_ino": "0x10000000464", "tid": 653 }, { "created_ino": "0x10000000465", "tid": 654 }, { "created_ino": "0x10000000466", "tid": 655 }, { "created_ino": "0x10000000467", "tid": 656 }, { "created_ino": "0x10000000468", "tid": 657 }, { "created_ino": "0x10000000469", "tid": 658 }, { "created_ino": "0x1000000046a", "tid": 659 }, { "created_ino": "0x1000000046b", "tid": 660 }, { "created_ino": "0x1000000046c", "tid": 661 }, { "created_ino": "0x1000000046d", "tid": 662 }, { "created_ino": "0x1000000046e", "tid": 663 }, { "created_ino": "0x1000000046f", "tid": 664 }, { "created_ino": "0x10000000470", "tid": 665 }, { "created_ino": "0x10000000471", "tid": 666 }, { "created_ino": "0x10000000472", "tid": 667 }, { "created_ino": "0x10000000473", "tid": 668 }, { "created_ino": "0x10000000474", "tid": 669 }, { "created_ino": "0x10000000475", "tid": 670 }, { "created_ino": "0x10000000476", "tid": 671 }, { "created_ino": "0x10000000477", "tid": 672 }, { "created_ino": "0x10000000478", "tid": 673 }, { "created_ino": "0x10000000479", "tid": 674 }, { "created_ino": "0x1000000047a", "tid": 675 }, { "created_ino": "0x1000000047b", "tid": 676 }, { "created_ino": "0x1000000047c", "tid": 677 }, { "created_ino": "0x1000000047d", "tid": 678 }, { "created_ino": "0x1000000047e", "tid": 679 }, { "created_ino": "0x1000000047f", "tid": 680 }, { "created_ino": "0x10000000480", "tid": 681 }, { "created_ino": "0x10000000481", "tid": 682 }, { "created_ino": "0x10000000482", "tid": 683 }, { "created_ino": "0x10000000483", "tid": 684 }, { "created_ino": "0x10000000484", "tid": 685 }, { "created_ino": "0x10000000485", "tid": 686 }, { "created_ino": "0x10000000486", "tid": 687 }, { "created_ino": "0x10000000487", "tid": 688 }, { "created_ino": "0x10000000488", "tid": 689 }, { "created_ino": "0x10000000489", "tid": 690 }, { "created_ino": "0x1000000048a", "tid": 691 }, { "created_ino": "0x1000000048b", "tid": 692 }, { "created_ino": "0x1000000048c", "tid": 693 }, { "created_ino": "0x1000000048d", "tid": 694 }, { "created_ino": "0x1000000048e", "tid": 695 }, { "created_ino": "0x1000000048f", "tid": 696 }, { "created_ino": "0x10000000490", "tid": 697 }, { "created_ino": "0x10000000491", "tid": 698 }, { "created_ino": "0x10000000492", "tid": 699 }, { "created_ino": "0x10000000493", "tid": 700 }, { "created_ino": "0x10000000494", "tid": 701 }, { "created_ino": "0x10000000495", "tid": 702 }, { "created_ino": "0x10000000496", "tid": 703 }, { "created_ino": "0x10000000497", "tid": 704 }, { "created_ino": "0x10000000498", "tid": 705 }, { "created_ino": "0x10000000499", "tid": 706 }, { "created_ino": "0x1000000049a", "tid": 707 }, { "created_ino": "0x1000000049b", "tid": 708 }, { "created_ino": "0x1000000049c", "tid": 709 }, { "created_ino": "0x1000000049d", "tid": 710 }, { "created_ino": "0x1000000049e", "tid": 711 }, { "created_ino": "0x1000000049f", "tid": 712 }, { "created_ino": "0x100000004a0", "tid": 713 }, { "created_ino": "0x100000004a1", "tid": 714 }, { "created_ino": "0x100000004a2", "tid": 715 }, { "created_ino": "0x100000004a3", "tid": 716 }, { "created_ino": "0x100000004a4", "tid": 717 }, { "created_ino": "0x100000004a5", "tid": 718 }, { "created_ino": "0x100000004a6", "tid": 719 }, { "created_ino": "0x100000004a7", "tid": 720 }, { "created_ino": "0x100000004a8", "tid": 721 }, { "created_ino": "0x100000004a9", "tid": 722 }, { "created_ino": "0x100000004aa", "tid": 723 }, { "created_ino": "0x100000004ab", "tid": 724 }, { "created_ino": "0x100000004ac", "tid": 725 }, { "created_ino": "0x100000004ad", "tid": 726 }, { "created_ino": "0x100000004ae", "tid": 727 }, { "created_ino": "0x100000004af", "tid": 728 }, { "created_ino": "0x100000004b0", "tid": 729 }, { "created_ino": "0x100000004b1", "tid": 730 }, { "created_ino": "0x100000004b2", "tid": 731 }, { "created_ino": "0x100000004b3", "tid": 732 }, { "created_ino": "0x100000004b4", "tid": 733 }, { "created_ino": "0x100000004b5", "tid": 734 }, { "created_ino": "0x100000004b6", "tid": 735 }, { "created_ino": "0x100000004b7", "tid": 736 }, { "created_ino": "0x100000004b8", "tid": 737 }, { "created_ino": "0x100000004b9", "tid": 738 }, { "created_ino": "0x100000004ba", "tid": 739 }, { "created_ino": "0x100000004bb", "tid": 740 }, { "created_ino": "0x100000004bc", "tid": 741 }, { "created_ino": "0x100000004bd", "tid": 742 }, { "created_ino": "0x100000004be", "tid": 743 }, { "created_ino": "0x100000004bf", "tid": 744 }, { "created_ino": "0x100000004c0", "tid": 745 }, { "created_ino": "0x100000004c1", "tid": 746 }, { "created_ino": "0x100000004c2", "tid": 747 }, { "created_ino": "0x100000004c3", "tid": 748 }, { "created_ino": "0x100000004c4", "tid": 749 }, { "created_ino": "0x100000004c5", "tid": 750 }, { "created_ino": "0x100000004c6", "tid": 751 }, { "created_ino": "0x100000004c7", "tid": 752 }, { "created_ino": "0x100000004c8", "tid": 753 }, { "created_ino": "0x100000004c9", "tid": 754 }, { "created_ino": "0x100000004ca", "tid": 755 }, { "created_ino": "0x100000004cb", "tid": 756 }, { "created_ino": "0x100000004cc", "tid": 757 }, { "created_ino": "0x100000004cd", "tid": 758 }, { "created_ino": "0x100000004ce", "tid": 759 }, { "created_ino": "0x100000004cf", "tid": 760 }, { "created_ino": "0x100000004d0", "tid": 761 }, { "created_ino": "0x100000004d1", "tid": 762 }, { "created_ino": "0x100000004d2", "tid": 763 }, { "created_ino": "0x100000004d3", "tid": 764 }, { "created_ino": "0x100000004d4", "tid": 765 }, { "created_ino": "0x100000004d5", "tid": 766 }, { "created_ino": "0x100000004d6", "tid": 767 }, { "created_ino": "0x100000004d7", "tid": 768 }, { "created_ino": "0x100000004d8", "tid": 769 }, { "created_ino": "0x100000004d9", "tid": 770 }, { "created_ino": "0x100000004da", "tid": 771 }, { "created_ino": "0x100000004db", "tid": 772 }, { "created_ino": "0x100000004dc", "tid": 773 }, { "created_ino": "0x100000004dd", "tid": 774 }, { "created_ino": "0x100000004de", "tid": 775 }, { "created_ino": "0x100000004df", "tid": 776 }, { "created_ino": "0x100000004e0", "tid": 777 }, { "created_ino": "0x100000004e1", "tid": 778 }, { "created_ino": "0x100000004e2", "tid": 779 }, { "created_ino": "0x100000004e3", "tid": 780 }, { "created_ino": "0x100000004e4", "tid": 781 }, { "created_ino": "0x100000004e5", "tid": 782 }, { "created_ino": "0x100000004e6", "tid": 783 }, { "created_ino": "0x100000004e7", "tid": 784 }, { "created_ino": "0x100000004e8", "tid": 785 }, { "created_ino": "0x100000004e9", "tid": 786 }, { "created_ino": "0x100000004ea", "tid": 787 }, { "created_ino": "0x100000004eb", "tid": 788 }, { "created_ino": "0x100000004ec", "tid": 789 }, { "created_ino": "0x100000004ed", "tid": 790 }, { "created_ino": "0x100000004ee", "tid": 791 }, { "created_ino": "0x100000004ef", "tid": 792 }, { "created_ino": "0x100000004f0", "tid": 793 }, { "created_ino": "0x100000004f1", "tid": 794 }, { "created_ino": "0x100000004f2", "tid": 795 }, { "created_ino": "0x100000004f3", "tid": 796 }, { "created_ino": "0x100000004f4", "tid": 797 }, { "created_ino": "0x100000004f5", "tid": 798 }, { "created_ino": "0x100000004f6", "tid": 799 }, { "created_ino": "0x100000004f7", "tid": 800 }, { "created_ino": "0x100000004f8", "tid": 801 }, { "created_ino": "0x100000004f9", "tid": 802 }, { "created_ino": "0x100000004fa", "tid": 803 }, { "created_ino": "0x100000004fb", "tid": 804 }, { "created_ino": "0x100000004fc", "tid": 805 }, { "created_ino": "0x100000004fd", "tid": 806 }, { "created_ino": "0x100000004fe", "tid": 807 }, { "created_ino": "0x100000004ff", "tid": 808 }, { "created_ino": "0x10000000500", "tid": 809 }, { "created_ino": "0x10000000501", "tid": 810 }, { "created_ino": "0x10000000502", "tid": 811 }, { "created_ino": "0x10000000503", "tid": 812 }, { "created_ino": "0x10000000504", "tid": 813 }, { "created_ino": "0x10000000505", "tid": 814 }, { "created_ino": "0x10000000506", "tid": 815 }, { "created_ino": "0x10000000507", "tid": 816 }, { "created_ino": "0x10000000508", "tid": 817 }, { "created_ino": "0x10000000509", "tid": 818 }, { "created_ino": "0x1000000050a", "tid": 819 }, { "created_ino": "0x1000000050b", "tid": 820 }, { "created_ino": "0x1000000050c", "tid": 821 }, { "created_ino": "0x1000000050d", "tid": 822 }, { "created_ino": "0x1000000050e", "tid": 823 }, { "created_ino": "0x1000000050f", "tid": 824 }, { "created_ino": "0x10000000510", "tid": 825 }, { "created_ino": "0x10000000511", "tid": 826 }, { "created_ino": "0x10000000512", "tid": 827 }, { "created_ino": "0x10000000513", "tid": 828 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 128224496, "type": "any" }, "name": { "num": 8157, "type": "client" } }, "id": 8157, "inst": "client.8157 192.168.0.1:0/128224496", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 530, "num_caps": 402, "num_completed_flushes": 0, "num_completed_requests": 299, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 203, "start": "0x10000000514" }, { "length": 501, "start": "0x100000005e0" } ], "recall_caps": { "halflife": 60, "value": 2889.31170879899 }, "recall_caps_throttle": { "halflife": 1.5, "value": 843.336021680752 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 269.0294306227462 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 398.4008329935051 }, "request_load_avg": 750, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1574.6691044567078 }, "state": "open", "uptime": 15.607653736 } ] 2024-04-25T11:20:12.854 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:20:12.854 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:20:12.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:20:12.970 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:20:13.083 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:20:13.083 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:20:13.083 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 800\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi007') 2024-04-25T11:20:13.084 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:20:13.084 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:13.145 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:20:13.145 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:13.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:13.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:20:13.331 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:20:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.477+0000 7f514bc00640 1 Processor -- start 2024-04-25T11:20:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.477+0000 7f514bc00640 1 -- start start 2024-04-25T11:20:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.479+0000 7f514bc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51440b4aa0 0x7f51440ae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.479+0000 7f514bc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5144164c80 con 0x7f51440b0f70 2024-04-25T11:20:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.479+0000 7f514bc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5144164f60 con 0x7f5144164710 2024-04-25T11:20:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.479+0000 7f514bc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f51441655b0 con 0x7f51440b4aa0 2024-04-25T11:20:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.480+0000 7f5149c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f51440b0f70 0x7f51441680b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49960/0 (socket says 172.21.15.7:49960) 2024-04-25T11:20:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.480+0000 7f5149c00640 1 -- 172.21.15.7:0/3713232832 learned_addr learned my addr 172.21.15.7:0/3713232832 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.480+0000 7f5149200640 1 --2- 172.21.15.7:0/3713232832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51440b4aa0 0x7f51440ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.480+0000 7f5143400640 1 -- 172.21.15.7:0/3713232832 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1302048179 0 0) 0x7f5144164c80 con 0x7f51440b0f70 2024-04-25T11:20:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.480+0000 7f5143400640 1 -- 172.21.15.7:0/3713232832 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5128003520 con 0x7f51440b0f70 2024-04-25T11:20:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.481+0000 7f5149200640 1 -- 172.21.15.7:0/3713232832 >> v1:172.21.15.87:6789/0 conn(0x7f5144164710 legacy=0x7f51440b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.481+0000 7f5149200640 1 -- 172.21.15.7:0/3713232832 >> v1:172.21.15.7:6789/0 conn(0x7f51440b0f70 legacy=0x7f51441680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.481+0000 7f5149200640 1 -- 172.21.15.7:0/3713232832 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5144165240 con 0x7f51440b4aa0 2024-04-25T11:20:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.482+0000 7f5149200640 1 --2- 172.21.15.7:0/3713232832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51440b4aa0 0x7f51440ae960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f51340097d0 tx=0x7f513402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98c6399498c60aad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.482+0000 7f5143400640 1 -- 172.21.15.7:0/3713232832 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134037070 con 0x7f51440b4aa0 2024-04-25T11:20:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.482+0000 7f5143400640 1 -- 172.21.15.7:0/3713232832 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5134032440 con 0x7f51440b4aa0 2024-04-25T11:20:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.482+0000 7f5143400640 1 -- 172.21.15.7:0/3713232832 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51340327f0 con 0x7f51440b4aa0 2024-04-25T11:20:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.483+0000 7f514bc00640 1 -- 172.21.15.7:0/3713232832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51440b4aa0 msgr2=0x7f51440ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.483+0000 7f514bc00640 1 --2- 172.21.15.7:0/3713232832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51440b4aa0 0x7f51440ae960 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f51340097d0 tx=0x7f513402a210 comp rx=0 tx=0).stop 2024-04-25T11:20:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.483+0000 7f514bc00640 1 -- 172.21.15.7:0/3713232832 shutdown_connections 2024-04-25T11:20:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.483+0000 7f514bc00640 1 --2- 172.21.15.7:0/3713232832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51440b4aa0 0x7f51440ae960 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.483+0000 7f514bc00640 1 -- 172.21.15.7:0/3713232832 >> 172.21.15.7:0/3713232832 conn(0x7f5144074200 msgr2=0x7f514406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.483+0000 7f514bc00640 1 -- 172.21.15.7:0/3713232832 shutdown_connections 2024-04-25T11:20:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.483+0000 7f514bc00640 1 -- 172.21.15.7:0/3713232832 wait complete. 2024-04-25T11:20:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.486+0000 7f514bc00640 1 Processor -- start 2024-04-25T11:20:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.487+0000 7f514bc00640 1 -- start start 2024-04-25T11:20:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.489+0000 7f514bc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51440b0f70 0x7f5144152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.489+0000 7f514bc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51440b4aa0 0x7f51441611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.489+0000 7f514bc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5144164710 0x7f51441616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.489+0000 7f514bc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f514406f300 con 0x7f51440b4aa0 2024-04-25T11:20:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.489+0000 7f514bc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5144165a40 con 0x7f5144164710 2024-04-25T11:20:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.489+0000 7f514bc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f51440aeea0 con 0x7f51440b0f70 2024-04-25T11:20:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.489+0000 7f5143e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51440b4aa0 0x7f51441611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.489+0000 7f5143e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51440b4aa0 0x7f51441611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42990/0 (socket says 172.21.15.7:42990) 2024-04-25T11:20:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.489+0000 7f5143e00640 1 -- 172.21.15.7:0/684726312 learned_addr learned my addr 172.21.15.7:0/684726312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.490+0000 7f5149200640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51440b0f70 0x7f5144152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.490+0000 7f5149c00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5144164710 0x7f51441616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.490+0000 7f5143e00640 1 -- 172.21.15.7:0/684726312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51440b0f70 msgr2=0x7f5144152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.490+0000 7f5143e00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51440b0f70 0x7f5144152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.490+0000 7f5143e00640 1 -- 172.21.15.7:0/684726312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5144164710 msgr2=0x7f51441616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.490+0000 7f5143e00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5144164710 0x7f51441616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.490+0000 7f5143e00640 1 -- 172.21.15.7:0/684726312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5144156da0 con 0x7f51440b4aa0 2024-04-25T11:20:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.490+0000 7f5149c00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5144164710 0x7f51441616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.490+0000 7f5143e00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51440b4aa0 0x7f51441611a0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f512c00c3c0 tx=0x7f512c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.491+0000 7f5140c00640 1 -- 172.21.15.7:0/684726312 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c017070 con 0x7f51440b4aa0 2024-04-25T11:20:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.491+0000 7f5140c00640 1 -- 172.21.15.7:0/684726312 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f512c0052d0 con 0x7f51440b4aa0 2024-04-25T11:20:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.491+0000 7f5140c00640 1 -- 172.21.15.7:0/684726312 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c003830 con 0x7f51440b4aa0 2024-04-25T11:20:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.491+0000 7f514bc00640 1 -- 172.21.15.7:0/684726312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5144157080 con 0x7f51440b4aa0 2024-04-25T11:20:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.491+0000 7f514bc00640 1 -- 172.21.15.7:0/684726312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5144157640 con 0x7f51440b4aa0 2024-04-25T11:20:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.492+0000 7f514bc00640 1 -- 172.21.15.7:0/684726312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514406a100 con 0x7f51440b4aa0 2024-04-25T11:20:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.494+0000 7f5140c00640 1 -- 172.21.15.7:0/684726312 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f512c01a070 con 0x7f51440b4aa0 2024-04-25T11:20:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.495+0000 7f5140c00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51200744e0 0x7f51200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.495+0000 7f5140c00640 1 -- 172.21.15.7:0/684726312 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f512c094620 con 0x7f51440b4aa0 2024-04-25T11:20:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.495+0000 7f5149200640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51200744e0 0x7f51200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:13.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.496+0000 7f5149200640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51200744e0 0x7f51200769f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5134034000 tx=0x7f513402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:13.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.499+0000 7f5140c00640 1 -- 172.21.15.7:0/684726312 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f512c05e530 con 0x7f51440b4aa0 2024-04-25T11:20:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.708+0000 7f514bc00640 1 -- 172.21.15.7:0/684726312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51440d0bc0 con 0x7f51440b4aa0 2024-04-25T11:20:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.710+0000 7f5140c00640 1 -- 172.21.15.7:0/684726312 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16856 (secure 0 0 0) 0x7f512c062830 con 0x7f51440b4aa0 2024-04-25T11:20:13.708 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:13.709 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:49.277316+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:46.014943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:46.794047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 -- 172.21.15.7:0/684726312 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51200744e0 msgr2=0x7f51200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51200744e0 0x7f51200769f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5134034000 tx=0x7f513402e000 comp rx=0 tx=0).stop 2024-04-25T11:20:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 -- 172.21.15.7:0/684726312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51440b4aa0 msgr2=0x7f51441611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51440b4aa0 0x7f51441611a0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f512c00c3c0 tx=0x7f512c00c880 comp rx=0 tx=0).stop 2024-04-25T11:20:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 -- 172.21.15.7:0/684726312 shutdown_connections 2024-04-25T11:20:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f51200744e0 0x7f51200769f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5144164710 0x7f51441616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51440b4aa0 0x7f51441611a0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 --2- 172.21.15.7:0/684726312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51440b0f70 0x7f5144152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 -- 172.21.15.7:0/684726312 >> 172.21.15.7:0/684726312 conn(0x7f5144074200 msgr2=0x7f51441625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 -- 172.21.15.7:0/684726312 shutdown_connections 2024-04-25T11:20:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:13.713+0000 7f514bc00640 1 -- 172.21.15.7:0/684726312 wait complete. 2024-04-25T11:20:13.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:46.014943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:46.794047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:13.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.074+0000 7f364c400640 1 Processor -- start 2024-04-25T11:20:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.074+0000 7f364c400640 1 -- start start 2024-04-25T11:20:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.076+0000 7f364c400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f364415aaa0 0x7f36440affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.077+0000 7f364c400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36440b5020 0x7f3644160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.077+0000 7f364c400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f364406f5b0 con 0x7f36440d0ee0 2024-04-25T11:20:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.077+0000 7f364c400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f36440af290 con 0x7f364415aaa0 2024-04-25T11:20:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.077+0000 7f364c400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f36440d14a0 con 0x7f36440b5020 2024-04-25T11:20:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.077+0000 7f3649a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f364415aaa0 0x7f36440affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.077+0000 7f3649a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f364415aaa0 0x7f36440affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:32806/0 (socket says 172.21.15.7:32806) 2024-04-25T11:20:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.077+0000 7f3649a00640 1 -- 172.21.15.7:0/3418155699 learned_addr learned my addr 172.21.15.7:0/3418155699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.077+0000 7f364a400640 1 --2- 172.21.15.7:0/3418155699 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36440b5020 0x7f3644160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.078+0000 7f3649a00640 1 -- 172.21.15.7:0/3418155699 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36440b5020 msgr2=0x7f3644160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.078+0000 7f3649a00640 1 --2- 172.21.15.7:0/3418155699 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36440b5020 0x7f3644160990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.078+0000 7f3649a00640 1 -- 172.21.15.7:0/3418155699 >> v1:172.21.15.87:6789/0 conn(0x7f36440d0ee0 legacy=0x7f36440b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.078+0000 7f3649a00640 1 -- 172.21.15.7:0/3418155699 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36440af4e0 con 0x7f364415aaa0 2024-04-25T11:20:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.080+0000 7f3649a00640 1 --2- 172.21.15.7:0/3418155699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f364415aaa0 0x7f36440affc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3638009870 tx=0x7f363802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bc480893494f963 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.080+0000 7f3643e00640 1 -- 172.21.15.7:0/3418155699 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638037070 con 0x7f364415aaa0 2024-04-25T11:20:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.080+0000 7f3643e00640 1 -- 172.21.15.7:0/3418155699 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3638032440 con 0x7f364415aaa0 2024-04-25T11:20:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.080+0000 7f3643e00640 1 -- 172.21.15.7:0/3418155699 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638032810 con 0x7f364415aaa0 2024-04-25T11:20:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.081+0000 7f364c400640 1 -- 172.21.15.7:0/3418155699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f364415aaa0 msgr2=0x7f36440affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.081+0000 7f364c400640 1 --2- 172.21.15.7:0/3418155699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f364415aaa0 0x7f36440affc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3638009870 tx=0x7f363802a270 comp rx=0 tx=0).stop 2024-04-25T11:20:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.082+0000 7f364c400640 1 -- 172.21.15.7:0/3418155699 shutdown_connections 2024-04-25T11:20:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.082+0000 7f364c400640 1 --2- 172.21.15.7:0/3418155699 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36440b5020 0x7f3644160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.082+0000 7f364c400640 1 --2- 172.21.15.7:0/3418155699 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f364415aaa0 0x7f36440affc0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.082+0000 7f364c400640 1 -- 172.21.15.7:0/3418155699 >> 172.21.15.7:0/3418155699 conn(0x7f36440742c0 msgr2=0x7f364406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.082+0000 7f364c400640 1 -- 172.21.15.7:0/3418155699 shutdown_connections 2024-04-25T11:20:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.082+0000 7f364c400640 1 -- 172.21.15.7:0/3418155699 wait complete. 2024-04-25T11:20:14.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.085+0000 7f364c400640 1 Processor -- start 2024-04-25T11:20:14.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.085+0000 7f364c400640 1 -- start start 2024-04-25T11:20:14.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.088+0000 7f364c400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36440b5020 0x7f36440b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.088+0000 7f364c400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36440d0ee0 0x7f36441464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.088+0000 7f364c400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3644146a10 0x7f3644142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.088+0000 7f364c400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36440b5430 con 0x7f36440d0ee0 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.088+0000 7f364c400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f36440afaa0 con 0x7f36440b5020 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.088+0000 7f364c400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f36440b55b0 con 0x7f3644146a10 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.088+0000 7f3649000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36440d0ee0 0x7f36441464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.088+0000 7f3649000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36440d0ee0 0x7f36441464d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42994/0 (socket says 172.21.15.7:42994) 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.088+0000 7f3649000640 1 -- 172.21.15.7:0/4174762912 learned_addr learned my addr 172.21.15.7:0/4174762912 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.088+0000 7f3649a00640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36440b5020 0x7f36440b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.088+0000 7f364a400640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3644146a10 0x7f3644142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.089+0000 7f3649000640 1 -- 172.21.15.7:0/4174762912 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3644146a10 msgr2=0x7f3644142cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.089+0000 7f3649000640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3644146a10 0x7f3644142cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.089+0000 7f3649000640 1 -- 172.21.15.7:0/4174762912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36440b5020 msgr2=0x7f36440b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.089+0000 7f3649000640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36440b5020 0x7f36440b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.089+0000 7f3649000640 1 -- 172.21.15.7:0/4174762912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36440b30f0 con 0x7f36440d0ee0 2024-04-25T11:20:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.089+0000 7f3649000640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36440d0ee0 0x7f36441464d0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f363000c430 tx=0x7f363000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.089+0000 7f3641600640 1 -- 172.21.15.7:0/4174762912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630017070 con 0x7f36440d0ee0 2024-04-25T11:20:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.089+0000 7f3641600640 1 -- 172.21.15.7:0/4174762912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36300052d0 con 0x7f36440d0ee0 2024-04-25T11:20:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.089+0000 7f364c400640 1 -- 172.21.15.7:0/4174762912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3644158370 con 0x7f36440d0ee0 2024-04-25T11:20:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.089+0000 7f3641600640 1 -- 172.21.15.7:0/4174762912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3630003830 con 0x7f36440d0ee0 2024-04-25T11:20:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.089+0000 7f364c400640 1 -- 172.21.15.7:0/4174762912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3644158930 con 0x7f36440d0ee0 2024-04-25T11:20:14.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.091+0000 7f364c400640 1 -- 172.21.15.7:0/4174762912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3644050740 con 0x7f36440d0ee0 2024-04-25T11:20:14.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.092+0000 7f3641600640 1 -- 172.21.15.7:0/4174762912 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f363001a070 con 0x7f36440d0ee0 2024-04-25T11:20:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.093+0000 7f3641600640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36240744b0 0x7f36240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.093+0000 7f3641600640 1 -- 172.21.15.7:0/4174762912 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f36300946b0 con 0x7f36440d0ee0 2024-04-25T11:20:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.094+0000 7f3649a00640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36240744b0 0x7f36240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.094+0000 7f3649a00640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36240744b0 0x7f36240769c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3638034000 tx=0x7f363802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.100+0000 7f3641600640 1 -- 172.21.15.7:0/4174762912 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f363005ed40 con 0x7f36440d0ee0 2024-04-25T11:20:14.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.313+0000 7f364c400640 1 -- 172.21.15.7:0/4174762912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3644070f20 con 0x7f36440d0ee0 2024-04-25T11:20:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.314+0000 7f3641600640 1 -- 172.21.15.7:0/4174762912 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7f363005e640 con 0x7f36440d0ee0 2024-04-25T11:20:14.312 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:20:14.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 -- 172.21.15.7:0/4174762912 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36240744b0 msgr2=0x7f36240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36240744b0 0x7f36240769c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3638034000 tx=0x7f363802e000 comp rx=0 tx=0).stop 2024-04-25T11:20:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 -- 172.21.15.7:0/4174762912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36440d0ee0 msgr2=0x7f36441464d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36440d0ee0 0x7f36441464d0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f363000c430 tx=0x7f363000c8f0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 -- 172.21.15.7:0/4174762912 shutdown_connections 2024-04-25T11:20:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f36240744b0 0x7f36240769c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3644146a10 0x7f3644142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36440d0ee0 0x7f36441464d0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 --2- 172.21.15.7:0/4174762912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36440b5020 0x7f36440b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 -- 172.21.15.7:0/4174762912 >> 172.21.15.7:0/4174762912 conn(0x7f36440742c0 msgr2=0x7f36440684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 -- 172.21.15.7:0/4174762912 shutdown_connections 2024-04-25T11:20:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.317+0000 7f364c400640 1 -- 172.21.15.7:0/4174762912 wait complete. 2024-04-25T11:20:14.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:14.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.633+0000 7f68e3c00640 1 Processor -- start 2024-04-25T11:20:14.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.633+0000 7f68e3c00640 1 -- start start 2024-04-25T11:20:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.636+0000 7f68e3c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f68dc15dfb0 0x7f68dc0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.636+0000 7f68e3c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68dc0d0b40 0x7f68dc06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.636+0000 7f68e3c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68dc15e3c0 con 0x7f68dc0d0f50 2024-04-25T11:20:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.636+0000 7f68e3c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f68dc0b09b0 con 0x7f68dc15dfb0 2024-04-25T11:20:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.636+0000 7f68e3c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f68dc0cf800 con 0x7f68dc0d0b40 2024-04-25T11:20:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.636+0000 7f68e1200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f68dc15dfb0 0x7f68dc0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.637+0000 7f68e1200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f68dc15dfb0 0x7f68dc0b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:32818/0 (socket says 172.21.15.7:32818) 2024-04-25T11:20:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.637+0000 7f68e1c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f68dc0d0f50 0x7f68dc161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:55646/0 (socket says 172.21.15.7:55646) 2024-04-25T11:20:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.637+0000 7f68e1200640 1 -- 172.21.15.7:0/1022490893 learned_addr learned my addr 172.21.15.7:0/1022490893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.637+0000 7f68dbe00640 1 --2- 172.21.15.7:0/1022490893 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68dc0d0b40 0x7f68dc06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.638+0000 7f68e1200640 1 -- 172.21.15.7:0/1022490893 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68dc0d0b40 msgr2=0x7f68dc06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.638+0000 7f68e1200640 1 --2- 172.21.15.7:0/1022490893 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68dc0d0b40 0x7f68dc06f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.638+0000 7f68e1200640 1 -- 172.21.15.7:0/1022490893 >> v1:172.21.15.87:6789/0 conn(0x7f68dc0d0f50 legacy=0x7f68dc161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.638+0000 7f68e1200640 1 -- 172.21.15.7:0/1022490893 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc0cfaa0 con 0x7f68dc15dfb0 2024-04-25T11:20:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.638+0000 7f68db400640 1 -- 172.21.15.7:0/1022490893 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 368620524 0 0) 0x7f68dc15e3c0 con 0x7f68dc0d0f50 2024-04-25T11:20:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.639+0000 7f68e1200640 1 --2- 172.21.15.7:0/1022490893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68dc15dfb0 0x7f68dc0b0470 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009870 tx=0x7f68d002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c13e3776b2610276 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.640+0000 7f68db400640 1 -- 172.21.15.7:0/1022490893 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0037070 con 0x7f68dc15dfb0 2024-04-25T11:20:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.640+0000 7f68db400640 1 -- 172.21.15.7:0/1022490893 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d0032440 con 0x7f68dc15dfb0 2024-04-25T11:20:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.640+0000 7f68db400640 1 -- 172.21.15.7:0/1022490893 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0032810 con 0x7f68dc15dfb0 2024-04-25T11:20:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.641+0000 7f68e3c00640 1 -- 172.21.15.7:0/1022490893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68dc15dfb0 msgr2=0x7f68dc0b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.641+0000 7f68e3c00640 1 --2- 172.21.15.7:0/1022490893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68dc15dfb0 0x7f68dc0b0470 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009870 tx=0x7f68d002a270 comp rx=0 tx=0).stop 2024-04-25T11:20:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.641+0000 7f68e3c00640 1 -- 172.21.15.7:0/1022490893 shutdown_connections 2024-04-25T11:20:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.641+0000 7f68e3c00640 1 --2- 172.21.15.7:0/1022490893 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68dc0d0b40 0x7f68dc06f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.641+0000 7f68e3c00640 1 --2- 172.21.15.7:0/1022490893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68dc15dfb0 0x7f68dc0b0470 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.641+0000 7f68e3c00640 1 -- 172.21.15.7:0/1022490893 >> 172.21.15.7:0/1022490893 conn(0x7f68dc0742c0 msgr2=0x7f68dc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.641+0000 7f68e3c00640 1 -- 172.21.15.7:0/1022490893 shutdown_connections 2024-04-25T11:20:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.641+0000 7f68e3c00640 1 -- 172.21.15.7:0/1022490893 wait complete. 2024-04-25T11:20:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.643+0000 7f68e3c00640 1 Processor -- start 2024-04-25T11:20:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.644+0000 7f68e3c00640 1 -- start start 2024-04-25T11:20:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.645+0000 7f68e3c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc0d0b40 0x7f68dc15e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.645+0000 7f68e3c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68dc0d0f50 0x7f68dc15c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.645+0000 7f68e3c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68dc15dfb0 0x7f68dc15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.645+0000 7f68e3c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc0d0340 con 0x7f68dc0d0b40 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.645+0000 7f68e3c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc0d0060 con 0x7f68dc0d0f50 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.645+0000 7f68e3c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f68dc0d0620 con 0x7f68dc15dfb0 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.645+0000 7f68e1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc0d0b40 0x7f68dc15e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.646+0000 7f68e1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc0d0b40 0x7f68dc15e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42996/0 (socket says 172.21.15.7:42996) 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.646+0000 7f68e1200640 1 -- 172.21.15.7:0/194774674 learned_addr learned my addr 172.21.15.7:0/194774674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.646+0000 7f68e1c00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68dc15dfb0 0x7f68dc15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.646+0000 7f68dbe00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68dc0d0f50 0x7f68dc15c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.646+0000 7f68e1200640 1 -- 172.21.15.7:0/194774674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68dc15dfb0 msgr2=0x7f68dc15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.646+0000 7f68e1200640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68dc15dfb0 0x7f68dc15cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.646+0000 7f68e1200640 1 -- 172.21.15.7:0/194774674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68dc0d0f50 msgr2=0x7f68dc15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.646+0000 7f68e1200640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68dc0d0f50 0x7f68dc15c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.646+0000 7f68e1200640 1 -- 172.21.15.7:0/194774674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc15eb70 con 0x7f68dc0d0b40 2024-04-25T11:20:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.646+0000 7f68dbe00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68dc0d0f50 0x7f68dc15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.646+0000 7f68e1200640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc0d0b40 0x7f68dc15e630 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009870 tx=0x7f68d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.647+0000 7f68d8c00640 1 -- 172.21.15.7:0/194774674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0037040 con 0x7f68dc0d0b40 2024-04-25T11:20:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.647+0000 7f68d8c00640 1 -- 172.21.15.7:0/194774674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d0042070 con 0x7f68dc0d0b40 2024-04-25T11:20:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.647+0000 7f68d8c00640 1 -- 172.21.15.7:0/194774674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0030880 con 0x7f68dc0d0b40 2024-04-25T11:20:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.647+0000 7f68e3c00640 1 -- 172.21.15.7:0/194774674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68dc146010 con 0x7f68dc0d0b40 2024-04-25T11:20:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.647+0000 7f68e3c00640 1 -- 172.21.15.7:0/194774674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68dc159750 con 0x7f68dc0d0b40 2024-04-25T11:20:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.648+0000 7f68e3c00640 1 -- 172.21.15.7:0/194774674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68dc0690d0 con 0x7f68dc0d0b40 2024-04-25T11:20:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.649+0000 7f68d8c00640 1 -- 172.21.15.7:0/194774674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f68d0030a20 con 0x7f68dc0d0b40 2024-04-25T11:20:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.651+0000 7f68d8c00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68bc0744e0 0x7f68bc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.651+0000 7f68d8c00640 1 -- 172.21.15.7:0/194774674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f68d00b5ca0 con 0x7f68dc0d0b40 2024-04-25T11:20:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.651+0000 7f68dbe00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68bc0744e0 0x7f68bc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.652+0000 7f68dbe00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68bc0744e0 0x7f68bc0769f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f68dc1f78a0 tx=0x7f68c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:14.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.656+0000 7f68d8c00640 1 -- 172.21.15.7:0/194774674 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f68dc0690d0 con 0x7f68dc0d0b40 2024-04-25T11:20:14.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.860+0000 7f68e3c00640 1 -- 172.21.15.7:0/194774674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f68dc050740 con 0x7f68dc0d0b40 2024-04-25T11:20:14.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.861+0000 7f68d8c00640 1 -- 172.21.15.7:0/194774674 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7f68d0080290 con 0x7f68dc0d0b40 2024-04-25T11:20:14.860 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:20:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.864+0000 7f68e3c00640 1 -- 172.21.15.7:0/194774674 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68bc0744e0 msgr2=0x7f68bc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.864+0000 7f68e3c00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68bc0744e0 0x7f68bc0769f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f68dc1f78a0 tx=0x7f68c8008040 comp rx=0 tx=0).stop 2024-04-25T11:20:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.864+0000 7f68e3c00640 1 -- 172.21.15.7:0/194774674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc0d0b40 msgr2=0x7f68dc15e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.864+0000 7f68e3c00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc0d0b40 0x7f68dc15e630 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009870 tx=0x7f68d00046c0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.864+0000 7f68e3c00640 1 -- 172.21.15.7:0/194774674 shutdown_connections 2024-04-25T11:20:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.864+0000 7f68e3c00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f68bc0744e0 0x7f68bc0769f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.864+0000 7f68e3c00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f68dc15dfb0 0x7f68dc15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.864+0000 7f68e3c00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68dc0d0f50 0x7f68dc15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.864+0000 7f68e3c00640 1 --2- 172.21.15.7:0/194774674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68dc0d0b40 0x7f68dc15e630 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.865+0000 7f68e3c00640 1 -- 172.21.15.7:0/194774674 >> 172.21.15.7:0/194774674 conn(0x7f68dc0742c0 msgr2=0x7f68dc06c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.865+0000 7f68e3c00640 1 -- 172.21.15.7:0/194774674 shutdown_connections 2024-04-25T11:20:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:14.865+0000 7f68e3c00640 1 -- 172.21.15.7:0/194774674 wait complete. 2024-04-25T11:20:14.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.179+0000 7fbc23e00640 1 Processor -- start 2024-04-25T11:20:15.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.179+0000 7fbc23e00640 1 -- start start 2024-04-25T11:20:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.182+0000 7fbc23e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc1c15dfb0 0x7fbc1c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.182+0000 7fbc23e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc1c06fab0 0x7fbc1c0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.182+0000 7fbc23e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc1c15e3c0 con 0x7fbc1c0d0b40 2024-04-25T11:20:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.182+0000 7fbc23e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1c0b09b0 con 0x7fbc1c06fab0 2024-04-25T11:20:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.182+0000 7fbc23e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c0cf800 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.182+0000 7fbc20a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbc1c0d0b40 0x7fbc1c06f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49970/0 (socket says 172.21.15.7:49970) 2024-04-25T11:20:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.182+0000 7fbc20a00640 1 -- 172.21.15.7:0/1176420960 learned_addr learned my addr 172.21.15.7:0/1176420960 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:15.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.182+0000 7fbc21e00640 1 --2- 172.21.15.7:0/1176420960 >> v2:172.21.15.7:3300/0 conn(0x7fbc1c06fab0 0x7fbc1c0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.183+0000 7fbc21400640 1 --2- 172.21.15.7:0/1176420960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc1c15dfb0 0x7fbc1c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.183+0000 7fbc1be00640 1 -- 172.21.15.7:0/1176420960 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1944512171 0 0) 0x7fbc1c15e3c0 con 0x7fbc1c0d0b40 2024-04-25T11:20:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.183+0000 7fbc1be00640 1 -- 172.21.15.7:0/1176420960 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc04003520 con 0x7fbc1c0d0b40 2024-04-25T11:20:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.183+0000 7fbc21e00640 1 -- 172.21.15.7:0/1176420960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc1c15dfb0 msgr2=0x7fbc1c0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.183+0000 7fbc21e00640 1 --2- 172.21.15.7:0/1176420960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc1c15dfb0 0x7fbc1c0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.183+0000 7fbc21e00640 1 -- 172.21.15.7:0/1176420960 >> v1:172.21.15.7:6789/0 conn(0x7fbc1c0d0b40 legacy=0x7fbc1c06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.183+0000 7fbc21e00640 1 -- 172.21.15.7:0/1176420960 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c0cfaa0 con 0x7fbc1c06fab0 2024-04-25T11:20:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.184+0000 7fbc21e00640 1 --2- 172.21.15.7:0/1176420960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc1c06fab0 0x7fbc1c0aea90 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbc140097a0 tx=0x7fbc1402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5b1c9d6ffba250 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.184+0000 7fbc1be00640 1 -- 172.21.15.7:0/1176420960 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14037070 con 0x7fbc1c06fab0 2024-04-25T11:20:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.185+0000 7fbc1be00640 1 -- 172.21.15.7:0/1176420960 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc14032440 con 0x7fbc1c06fab0 2024-04-25T11:20:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.185+0000 7fbc1be00640 1 -- 172.21.15.7:0/1176420960 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14032810 con 0x7fbc1c06fab0 2024-04-25T11:20:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.185+0000 7fbc23e00640 1 -- 172.21.15.7:0/1176420960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc1c06fab0 msgr2=0x7fbc1c0aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.185+0000 7fbc23e00640 1 --2- 172.21.15.7:0/1176420960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc1c06fab0 0x7fbc1c0aea90 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbc140097a0 tx=0x7fbc1402a180 comp rx=0 tx=0).stop 2024-04-25T11:20:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.186+0000 7fbc23e00640 1 -- 172.21.15.7:0/1176420960 shutdown_connections 2024-04-25T11:20:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.186+0000 7fbc23e00640 1 --2- 172.21.15.7:0/1176420960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc1c06fab0 0x7fbc1c0aea90 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.186+0000 7fbc23e00640 1 --2- 172.21.15.7:0/1176420960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc1c15dfb0 0x7fbc1c0b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.186+0000 7fbc23e00640 1 -- 172.21.15.7:0/1176420960 >> 172.21.15.7:0/1176420960 conn(0x7fbc1c0742c0 msgr2=0x7fbc1c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.186+0000 7fbc23e00640 1 -- 172.21.15.7:0/1176420960 shutdown_connections 2024-04-25T11:20:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.186+0000 7fbc23e00640 1 -- 172.21.15.7:0/1176420960 wait complete. 2024-04-25T11:20:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.188+0000 7fbc23e00640 1 Processor -- start 2024-04-25T11:20:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.188+0000 7fbc23e00640 1 -- start start 2024-04-25T11:20:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.190+0000 7fbc23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc1c06fab0 0x7fbc1c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.190+0000 7fbc23e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc1c0d0b40 0x7fbc1c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.190+0000 7fbc23e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc1c15dfb0 0x7fbc1c0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.190+0000 7fbc23e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c0d0340 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.190+0000 7fbc23e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c0d0060 con 0x7fbc1c06fab0 2024-04-25T11:20:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.190+0000 7fbc23e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c0d0620 con 0x7fbc1c0d0b40 2024-04-25T11:20:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.190+0000 7fbc21e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc1c15dfb0 0x7fbc1c0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.190+0000 7fbc21e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc1c15dfb0 0x7fbc1c0cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43014/0 (socket says 172.21.15.7:43014) 2024-04-25T11:20:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.190+0000 7fbc21e00640 1 -- 172.21.15.7:0/2590148225 learned_addr learned my addr 172.21.15.7:0/2590148225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.191+0000 7fbc21400640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc1c06fab0 0x7fbc1c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.191+0000 7fbc20a00640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc1c0d0b40 0x7fbc1c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.191+0000 7fbc21e00640 1 -- 172.21.15.7:0/2590148225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc1c0d0b40 msgr2=0x7fbc1c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.191+0000 7fbc21e00640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc1c0d0b40 0x7fbc1c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.191+0000 7fbc21e00640 1 -- 172.21.15.7:0/2590148225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc1c06fab0 msgr2=0x7fbc1c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.191+0000 7fbc21e00640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc1c06fab0 0x7fbc1c0cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.191+0000 7fbc21e00640 1 -- 172.21.15.7:0/2590148225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c0b55f0 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.191+0000 7fbc20a00640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc1c0d0b40 0x7fbc1c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.191+0000 7fbc21e00640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc1c15dfb0 0x7fbc1c0cac90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fbc14035040 tx=0x7fbc140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.191+0000 7fbc19600640 1 -- 172.21.15.7:0/2590148225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14041020 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.192+0000 7fbc19600640 1 -- 172.21.15.7:0/2590148225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1403d640 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.192+0000 7fbc23e00640 1 -- 172.21.15.7:0/2590148225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c0b58d0 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.192+0000 7fbc19600640 1 -- 172.21.15.7:0/2590148225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14037040 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.192+0000 7fbc23e00640 1 -- 172.21.15.7:0/2590148225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c0b5e90 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.193+0000 7fbc23e00640 1 -- 172.21.15.7:0/2590148225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1c0b5360 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.195+0000 7fbc19600640 1 -- 172.21.15.7:0/2590148225 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbc14044020 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.196+0000 7fbc19600640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbbfc0744e0 0x7fbbfc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.196+0000 7fbc19600640 1 -- 172.21.15.7:0/2590148225 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fbc140b59b0 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.196+0000 7fbc21400640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbbfc0744e0 0x7fbbfc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.197+0000 7fbc21400640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbbfc0744e0 0x7fbbfc0769f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c072df0 tx=0x7fbc1000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.201+0000 7fbc19600640 1 -- 172.21.15.7:0/2590148225 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc1404b070 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.407+0000 7fbc23e00640 1 -- 172.21.15.7:0/2590148225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbc1c0af890 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.408+0000 7fbc19600640 1 -- 172.21.15.7:0/2590148225 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7fbc1c0af890 con 0x7fbc1c15dfb0 2024-04-25T11:20:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:20:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 -- 172.21.15.7:0/2590148225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbbfc0744e0 msgr2=0x7fbbfc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbbfc0744e0 0x7fbbfc0769f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c072df0 tx=0x7fbc1000b040 comp rx=0 tx=0).stop 2024-04-25T11:20:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 -- 172.21.15.7:0/2590148225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc1c15dfb0 msgr2=0x7fbc1c0cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc1c15dfb0 0x7fbc1c0cac90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fbc14035040 tx=0x7fbc140046c0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 -- 172.21.15.7:0/2590148225 shutdown_connections 2024-04-25T11:20:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbbfc0744e0 0x7fbbfc0769f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc1c15dfb0 0x7fbc1c0cac90 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc1c0d0b40 0x7fbc1c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 --2- 172.21.15.7:0/2590148225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc1c06fab0 0x7fbc1c0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 -- 172.21.15.7:0/2590148225 >> 172.21.15.7:0/2590148225 conn(0x7fbc1c0742c0 msgr2=0x7fbc1c06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 -- 172.21.15.7:0/2590148225 shutdown_connections 2024-04-25T11:20:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.411+0000 7fbc23e00640 1 -- 172.21.15.7:0/2590148225 wait complete. 2024-04-25T11:20:15.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:15.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.723+0000 7f2b84a00640 1 Processor -- start 2024-04-25T11:20:15.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.723+0000 7f2b84a00640 1 -- start start 2024-04-25T11:20:15.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.726+0000 7f2b84a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2b80141290 0x7f2b801416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.726+0000 7f2b84a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b8006c570 con 0x7f2b8015eed0 2024-04-25T11:20:15.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.726+0000 7f2b84a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b8006c850 con 0x7f2b8006d610 2024-04-25T11:20:15.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.726+0000 7f2b84a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2b8006cb30 con 0x7f2b80141290 2024-04-25T11:20:15.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.726+0000 7f2b7f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2b8015eed0 0x7f2b800cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49982/0 (socket says 172.21.15.7:49982) 2024-04-25T11:20:15.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.726+0000 7f2b7f400640 1 -- 172.21.15.7:0/266554617 learned_addr learned my addr 172.21.15.7:0/266554617 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:15.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b77e00640 1 --2- 172.21.15.7:0/266554617 >> v2:172.21.15.87:3300/0 conn(0x7f2b80141290 0x7f2b801416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 70028306 0 0) 0x7f2b8006c570 con 0x7f2b8015eed0 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b64003520 con 0x7f2b8015eed0 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2228089537 0 0) 0x7f2b64003520 con 0x7f2b8015eed0 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 >> v2:172.21.15.87:3300/0 conn(0x7f2b80141290 msgr2=0x7f2b801416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 --2- 172.21.15.7:0/266554617 >> v2:172.21.15.87:3300/0 conn(0x7f2b80141290 0x7f2b801416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 >> v1:172.21.15.87:6789/0 conn(0x7f2b8006d610 legacy=0x7f2b800b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8006ce10 con 0x7f2b8015eed0 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2b70001670 con 0x7f2b8015eed0 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 >> v1:172.21.15.7:6789/0 conn(0x7f2b8015eed0 legacy=0x7f2b800cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 --2- 172.21.15.7:0/266554617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b64003a00 0x7f2b64003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 --2- 172.21.15.7:0/266554617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b64004800 0x7f2b64004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.727+0000 7f2b7ea00640 1 --2- 172.21.15.7:0/266554617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b64005270 0x7f2b6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8006cb30 con 0x7f2b64003a00 2024-04-25T11:20:15.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6400c190 con 0x7f2b64005270 2024-04-25T11:20:15.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6400c390 con 0x7f2b64004800 2024-04-25T11:20:15.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b7f400640 1 --2- 172.21.15.7:0/266554617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b64003a00 0x7f2b64003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b77e00640 1 --2- 172.21.15.7:0/266554617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b64004800 0x7f2b64004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b7f400640 1 -- 172.21.15.7:0/266554617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b64004800 msgr2=0x7f2b64004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:15.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b7f400640 1 --2- 172.21.15.7:0/266554617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b64004800 0x7f2b64004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b7f400640 1 -- 172.21.15.7:0/266554617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b64005270 msgr2=0x7f2b6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:15.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b7f400640 1 --2- 172.21.15.7:0/266554617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b64005270 0x7f2b6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b7f400640 1 -- 172.21.15.7:0/266554617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b64003520 con 0x7f2b64003a00 2024-04-25T11:20:15.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b7fe00640 1 -- 172.21.15.7:0/266554617 reap_dead start 2024-04-25T11:20:15.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.728+0000 7f2b77e00640 1 --2- 172.21.15.7:0/266554617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b64004800 0x7f2b64004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:15.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.729+0000 7f2b7f400640 1 --2- 172.21.15.7:0/266554617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b64003a00 0x7f2b64003e70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2b80074c70 tx=0x7f2b64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6a90e574f5bdcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.730+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70007c40 con 0x7f2b64003a00 2024-04-25T11:20:15.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.730+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b70005af0 con 0x7f2b64003a00 2024-04-25T11:20:15.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.730+0000 7f2b7ea00640 1 -- 172.21.15.7:0/266554617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70007070 con 0x7f2b64003a00 2024-04-25T11:20:15.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.730+0000 7f2b84a00640 1 -- 172.21.15.7:0/266554617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b64003a00 msgr2=0x7f2b64003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.730+0000 7f2b84a00640 1 --2- 172.21.15.7:0/266554617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b64003a00 0x7f2b64003e70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2b80074c70 tx=0x7f2b64005830 comp rx=0 tx=0).stop 2024-04-25T11:20:15.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.730+0000 7f2b84a00640 1 -- 172.21.15.7:0/266554617 shutdown_connections 2024-04-25T11:20:15.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.730+0000 7f2b84a00640 1 --2- 172.21.15.7:0/266554617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b64003a00 0x7f2b64003e70 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.730+0000 7f2b84a00640 1 -- 172.21.15.7:0/266554617 >> 172.21.15.7:0/266554617 conn(0x7f2b800d1540 msgr2=0x7f2b800d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:15.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.731+0000 7f2b84a00640 1 -- 172.21.15.7:0/266554617 shutdown_connections 2024-04-25T11:20:15.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.731+0000 7f2b84a00640 1 -- 172.21.15.7:0/266554617 wait complete. 2024-04-25T11:20:15.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.733+0000 7f2b84a00640 1 Processor -- start 2024-04-25T11:20:15.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.733+0000 7f2b84a00640 1 -- start start 2024-04-25T11:20:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.735+0000 7f2b84a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b64003a00 0x7f2b8013b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.735+0000 7f2b84a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b80139a50 0x7f2b80139270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.735+0000 7f2b84a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8014c770 0x7f2b8014cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.735+0000 7f2b84a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80140f20 con 0x7f2b8014c770 2024-04-25T11:20:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.735+0000 7f2b84a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2b80140da0 con 0x7f2b64003a00 2024-04-25T11:20:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.735+0000 7f2b84a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b801416a0 con 0x7f2b80139a50 2024-04-25T11:20:15.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.735+0000 7f2b7fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8014c770 0x7f2b8014cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.735+0000 7f2b7fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8014c770 0x7f2b8014cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43042/0 (socket says 172.21.15.7:43042) 2024-04-25T11:20:15.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.735+0000 7f2b7fe00640 1 -- 172.21.15.7:0/1085023337 learned_addr learned my addr 172.21.15.7:0/1085023337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:15.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.735+0000 7f2b77e00640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b80139a50 0x7f2b80139270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.735+0000 7f2b7f400640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b64003a00 0x7f2b8013b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.736+0000 7f2b7fe00640 1 -- 172.21.15.7:0/1085023337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b80139a50 msgr2=0x7f2b80139270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.736+0000 7f2b7fe00640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b80139a50 0x7f2b80139270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.736+0000 7f2b7fe00640 1 -- 172.21.15.7:0/1085023337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b64003a00 msgr2=0x7f2b8013b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.736+0000 7f2b7fe00640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b64003a00 0x7f2b8013b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.736+0000 7f2b7fe00640 1 -- 172.21.15.7:0/1085023337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8013a4d0 con 0x7f2b8014c770 2024-04-25T11:20:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.736+0000 7f2b7fe00640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8014c770 0x7f2b8014cb80 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2b80072b20 tx=0x7f2b7800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.736+0000 7f2b77400640 1 -- 172.21.15.7:0/1085023337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78017070 con 0x7f2b8014c770 2024-04-25T11:20:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.736+0000 7f2b77400640 1 -- 172.21.15.7:0/1085023337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b78005d40 con 0x7f2b8014c770 2024-04-25T11:20:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.736+0000 7f2b77400640 1 -- 172.21.15.7:0/1085023337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7800a830 con 0x7f2b8014c770 2024-04-25T11:20:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.736+0000 7f2b84a00640 1 -- 172.21.15.7:0/1085023337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8013a7b0 con 0x7f2b8014c770 2024-04-25T11:20:15.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.736+0000 7f2b84a00640 1 -- 172.21.15.7:0/1085023337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8013ad70 con 0x7f2b8014c770 2024-04-25T11:20:15.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.738+0000 7f2b84a00640 1 -- 172.21.15.7:0/1085023337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b80050740 con 0x7f2b8014c770 2024-04-25T11:20:15.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.739+0000 7f2b77400640 1 -- 172.21.15.7:0/1085023337 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2b7801a070 con 0x7f2b8014c770 2024-04-25T11:20:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.741+0000 7f2b77400640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b5c0744b0 0x7f2b5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.741+0000 7f2b77400640 1 -- 172.21.15.7:0/1085023337 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f2b780945f0 con 0x7f2b8014c770 2024-04-25T11:20:15.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.741+0000 7f2b7f400640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b5c0744b0 0x7f2b5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:15.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.742+0000 7f2b7f400640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b5c0744b0 0x7f2b5c0769c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2b70032000 tx=0x7f2b7002d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:15.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.745+0000 7f2b77400640 1 -- 172.21.15.7:0/1085023337 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b7805ec80 con 0x7f2b8014c770 2024-04-25T11:20:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.968+0000 7f2b84a00640 1 -- 172.21.15.7:0/1085023337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2b8013a220 con 0x7f2b8014c770 2024-04-25T11:20:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.969+0000 7f2b77400640 1 -- 172.21.15.7:0/1085023337 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) ==== 80+0+0 (secure 0 0 0) 0x7f2b7805e580 con 0x7f2b8014c770 2024-04-25T11:20:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:20:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.971+0000 7f2b84a00640 1 -- 172.21.15.7:0/1085023337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b5c0744b0 msgr2=0x7f2b5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.971+0000 7f2b84a00640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b5c0744b0 0x7f2b5c0769c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2b70032000 tx=0x7f2b7002d000 comp rx=0 tx=0).stop 2024-04-25T11:20:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.971+0000 7f2b84a00640 1 -- 172.21.15.7:0/1085023337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8014c770 msgr2=0x7f2b8014cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.971+0000 7f2b84a00640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8014c770 0x7f2b8014cb80 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2b80072b20 tx=0x7f2b7800cb90 comp rx=0 tx=0).stop 2024-04-25T11:20:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.972+0000 7f2b84a00640 1 -- 172.21.15.7:0/1085023337 shutdown_connections 2024-04-25T11:20:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.972+0000 7f2b84a00640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b5c0744b0 0x7f2b5c0769c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.972+0000 7f2b84a00640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8014c770 0x7f2b8014cb80 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.972+0000 7f2b84a00640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b80139a50 0x7f2b80139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.972+0000 7f2b84a00640 1 --2- 172.21.15.7:0/1085023337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b64003a00 0x7f2b8013b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.972+0000 7f2b84a00640 1 -- 172.21.15.7:0/1085023337 >> 172.21.15.7:0/1085023337 conn(0x7f2b800d1540 msgr2=0x7f2b8015f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.972+0000 7f2b84a00640 1 -- 172.21.15.7:0/1085023337 shutdown_connections 2024-04-25T11:20:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:15.972+0000 7f2b84a00640 1 -- 172.21.15.7:0/1085023337 wait complete. 2024-04-25T11:20:15.996 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:20:15.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:15.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:16.330 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:20:16.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:16.330 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:20:16.371 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:20:16.371 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:20:16.371 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:16.410 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:20:16.410 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:16.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:16.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.787+0000 7fd9e9e00640 1 Processor -- start 2024-04-25T11:20:16.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.787+0000 7fd9e9e00640 1 -- start start 2024-04-25T11:20:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.790+0000 7fd9e9e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9e4164380 0x7fd9e40b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.790+0000 7fd9e9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9e40b5d40 0x7fd9e4167f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.790+0000 7fd9e9e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e40b47d0 con 0x7fd9e406ffd0 2024-04-25T11:20:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.790+0000 7fd9e9e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e40b4a40 con 0x7fd9e4164380 2024-04-25T11:20:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.790+0000 7fd9e9e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e40b4d20 con 0x7fd9e40b5d40 2024-04-25T11:20:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.790+0000 7fd9e3e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9e4164380 0x7fd9e40b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.790+0000 7fd9e8a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd9e406ffd0 0x7fd9e40af390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49994/0 (socket says 172.21.15.7:49994) 2024-04-25T11:20:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.790+0000 7fd9e8a00640 1 -- 172.21.15.7:0/3577495176 learned_addr learned my addr 172.21.15.7:0/3577495176 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.791+0000 7fd9e9400640 1 --2- 172.21.15.7:0/3577495176 >> v2:172.21.15.87:3300/0 conn(0x7fd9e40b5d40 0x7fd9e4167f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.791+0000 7fd9e3e00640 1 -- 172.21.15.7:0/3577495176 >> v2:172.21.15.87:3300/0 conn(0x7fd9e40b5d40 msgr2=0x7fd9e4167f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.791+0000 7fd9e3e00640 1 --2- 172.21.15.7:0/3577495176 >> v2:172.21.15.87:3300/0 conn(0x7fd9e40b5d40 0x7fd9e4167f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.791+0000 7fd9e3e00640 1 -- 172.21.15.7:0/3577495176 >> v1:172.21.15.7:6789/0 conn(0x7fd9e406ffd0 legacy=0x7fd9e40af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.791+0000 7fd9e3e00640 1 -- 172.21.15.7:0/3577495176 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e40b5000 con 0x7fd9e4164380 2024-04-25T11:20:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.792+0000 7fd9e3e00640 1 --2- 172.21.15.7:0/3577495176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9e4164380 0x7fd9e40b5800 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40712c0 tx=0x7fd9d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e838a6bd1db6a9f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.792+0000 7fd9e3400640 1 -- 172.21.15.7:0/3577495176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0037070 con 0x7fd9e4164380 2024-04-25T11:20:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.793+0000 7fd9e3400640 1 -- 172.21.15.7:0/3577495176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d0032440 con 0x7fd9e4164380 2024-04-25T11:20:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.793+0000 7fd9e3400640 1 -- 172.21.15.7:0/3577495176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0032810 con 0x7fd9e4164380 2024-04-25T11:20:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.793+0000 7fd9e9e00640 1 -- 172.21.15.7:0/3577495176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9e4164380 msgr2=0x7fd9e40b5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.793+0000 7fd9e9e00640 1 --2- 172.21.15.7:0/3577495176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9e4164380 0x7fd9e40b5800 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd9e40712c0 tx=0x7fd9d002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.793+0000 7fd9e9e00640 1 -- 172.21.15.7:0/3577495176 shutdown_connections 2024-04-25T11:20:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.793+0000 7fd9e9e00640 1 --2- 172.21.15.7:0/3577495176 >> v2:172.21.15.87:3300/0 conn(0x7fd9e40b5d40 0x7fd9e4167f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.793+0000 7fd9e9e00640 1 --2- 172.21.15.7:0/3577495176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9e4164380 0x7fd9e40b5800 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.793+0000 7fd9e9e00640 1 -- 172.21.15.7:0/3577495176 >> 172.21.15.7:0/3577495176 conn(0x7fd9e40b3820 msgr2=0x7fd9e40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.793+0000 7fd9e9e00640 1 -- 172.21.15.7:0/3577495176 shutdown_connections 2024-04-25T11:20:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.793+0000 7fd9e9e00640 1 -- 172.21.15.7:0/3577495176 wait complete. 2024-04-25T11:20:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.796+0000 7fd9e9e00640 1 Processor -- start 2024-04-25T11:20:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.796+0000 7fd9e9e00640 1 -- start start 2024-04-25T11:20:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.797+0000 7fd9e9e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9e406ffd0 0x7fd9e4153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.797+0000 7fd9e9e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9e40b5d40 0x7fd9e40b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.798+0000 7fd9e9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9e4164380 0x7fd9e4167050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.798+0000 7fd9e9e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e40afaa0 con 0x7fd9e4164380 2024-04-25T11:20:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.798+0000 7fd9e9e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4164790 con 0x7fd9e406ffd0 2024-04-25T11:20:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.798+0000 7fd9e9e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e406fa70 con 0x7fd9e40b5d40 2024-04-25T11:20:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.798+0000 7fd9e9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9e4164380 0x7fd9e4167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.798+0000 7fd9e9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9e4164380 0x7fd9e4167050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43052/0 (socket says 172.21.15.7:43052) 2024-04-25T11:20:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.798+0000 7fd9e9400640 1 -- 172.21.15.7:0/1671981955 learned_addr learned my addr 172.21.15.7:0/1671981955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.798+0000 7fd9e8a00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9e406ffd0 0x7fd9e4153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.798+0000 7fd9e3e00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9e40b5d40 0x7fd9e40b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.799+0000 7fd9e9400640 1 -- 172.21.15.7:0/1671981955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9e40b5d40 msgr2=0x7fd9e40b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.799+0000 7fd9e9400640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9e40b5d40 0x7fd9e40b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.799+0000 7fd9e9400640 1 -- 172.21.15.7:0/1671981955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9e406ffd0 msgr2=0x7fd9e4153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.799+0000 7fd9e9400640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9e406ffd0 0x7fd9e4153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.799+0000 7fd9e9400640 1 -- 172.21.15.7:0/1671981955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e41fa840 con 0x7fd9e4164380 2024-04-25T11:20:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.799+0000 7fd9e3e00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9e40b5d40 0x7fd9e40b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.799+0000 7fd9e9400640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9e4164380 0x7fd9e4167050 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc007e60 tx=0x7fd9dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.799+0000 7fd9e0c00640 1 -- 172.21.15.7:0/1671981955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc014420 con 0x7fd9e4164380 2024-04-25T11:20:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.799+0000 7fd9e0c00640 1 -- 172.21.15.7:0/1671981955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9dc019070 con 0x7fd9e4164380 2024-04-25T11:20:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.799+0000 7fd9e0c00640 1 -- 172.21.15.7:0/1671981955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc014820 con 0x7fd9e4164380 2024-04-25T11:20:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.799+0000 7fd9e9e00640 1 -- 172.21.15.7:0/1671981955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e40b11f0 con 0x7fd9e4164380 2024-04-25T11:20:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.800+0000 7fd9e9e00640 1 -- 172.21.15.7:0/1671981955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e4164c80 con 0x7fd9e4164380 2024-04-25T11:20:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.801+0000 7fd9e9e00640 1 -- 172.21.15.7:0/1671981955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e4050740 con 0x7fd9e4164380 2024-04-25T11:20:16.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.802+0000 7fd9e0c00640 1 -- 172.21.15.7:0/1671981955 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd9dc0149c0 con 0x7fd9e4164380 2024-04-25T11:20:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.804+0000 7fd9e0c00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9c40744b0 0x7fd9c40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.804+0000 7fd9e0c00640 1 -- 172.21.15.7:0/1671981955 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fd9dc095c40 con 0x7fd9e4164380 2024-04-25T11:20:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.804+0000 7fd9e8a00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9c40744b0 0x7fd9c40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.805+0000 7fd9e8a00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9c40744b0 0x7fd9c40769c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8007fd0 tx=0x7fd9d8007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:16.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:16.809+0000 7fd9e0c00640 1 -- 172.21.15.7:0/1671981955 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd9dc0601a0 con 0x7fd9e4164380 2024-04-25T11:20:17.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.013+0000 7fd9e9e00640 1 -- 172.21.15.7:0/1671981955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9e40739c0 con 0x7fd9e4164380 2024-04-25T11:20:17.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.016+0000 7fd9e0c00640 1 -- 172.21.15.7:0/1671981955 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16856 (secure 0 0 0) 0x7fd9dc05faa0 con 0x7fd9e4164380 2024-04-25T11:20:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:17.014 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:49.277316+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:46.014943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:46.794047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.018+0000 7fd9e9e00640 1 -- 172.21.15.7:0/1671981955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9c40744b0 msgr2=0x7fd9c40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.018+0000 7fd9e9e00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9c40744b0 0x7fd9c40769c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8007fd0 tx=0x7fd9d8007e80 comp rx=0 tx=0).stop 2024-04-25T11:20:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.018+0000 7fd9e9e00640 1 -- 172.21.15.7:0/1671981955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9e4164380 msgr2=0x7fd9e4167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.018+0000 7fd9e9e00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9e4164380 0x7fd9e4167050 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc007e60 tx=0x7fd9dc00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.019+0000 7fd9e9e00640 1 -- 172.21.15.7:0/1671981955 shutdown_connections 2024-04-25T11:20:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.019+0000 7fd9e9e00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9c40744b0 0x7fd9c40769c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.019+0000 7fd9e9e00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9e4164380 0x7fd9e4167050 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.019+0000 7fd9e9e00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9e40b5d40 0x7fd9e40b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.019+0000 7fd9e9e00640 1 --2- 172.21.15.7:0/1671981955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9e406ffd0 0x7fd9e4153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.019+0000 7fd9e9e00640 1 -- 172.21.15.7:0/1671981955 >> 172.21.15.7:0/1671981955 conn(0x7fd9e40b3820 msgr2=0x7fd9e406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.019+0000 7fd9e9e00640 1 -- 172.21.15.7:0/1671981955 shutdown_connections 2024-04-25T11:20:17.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.019+0000 7fd9e9e00640 1 -- 172.21.15.7:0/1671981955 wait complete. 2024-04-25T11:20:17.046 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:46.014943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:46.794047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:17.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:17.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.340+0000 7f4e2f200640 1 Processor -- start 2024-04-25T11:20:17.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.340+0000 7f4e2f200640 1 -- start start 2024-04-25T11:20:17.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.342+0000 7f4e2f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300ce8d0 0x7f4e300cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.342+0000 7f4e2f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4e3015c2e0 0x7f4e3015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.342+0000 7f4e2f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e300b4830 con 0x7f4e3015fef0 2024-04-25T11:20:17.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.342+0000 7f4e2f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4e300b4b10 con 0x7f4e3015c2e0 2024-04-25T11:20:17.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.342+0000 7f4e2f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e300b4df0 con 0x7f4e300ce8d0 2024-04-25T11:20:17.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.343+0000 7f4e2e800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4e3015fef0 0x7f4e30160300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50010/0 (socket says 172.21.15.7:50010) 2024-04-25T11:20:17.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.343+0000 7f4e2e800640 1 -- 172.21.15.7:0/4174859129 learned_addr learned my addr 172.21.15.7:0/4174859129 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:17.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.343+0000 7f4e2de00640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300ce8d0 0x7f4e300cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.343+0000 7f4e2d400640 1 --2- 172.21.15.7:0/4174859129 >> v2:172.21.15.87:3300/0 conn(0x7f4e3015c2e0 0x7f4e3015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.343+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3219008295 0 0) 0x7f4e300b4830 con 0x7f4e3015fef0 2024-04-25T11:20:17.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.343+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e14003520 con 0x7f4e3015fef0 2024-04-25T11:20:17.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1493462995 0 0) 0x7f4e14003520 con 0x7f4e3015fef0 2024-04-25T11:20:17.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300ce8d0 msgr2=0x7f4e300cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:17.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300ce8d0 0x7f4e300cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 >> v2:172.21.15.87:3300/0 conn(0x7f4e3015c2e0 msgr2=0x7f4e3015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 --2- 172.21.15.7:0/4174859129 >> v2:172.21.15.87:3300/0 conn(0x7f4e3015c2e0 0x7f4e3015c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e300b50d0 con 0x7f4e3015fef0 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f4e280019f0 con 0x7f4e3015fef0 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 >> v1:172.21.15.7:6789/0 conn(0x7f4e3015fef0 legacy=0x7f4e30160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e140039b0 0x7f4e14003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 0x7f4e14004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300ce8d0 0x7f4e14005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e300b4b10 con 0x7f4e14004800 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e300b4df0 con 0x7f4e140039b0 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1400b9f0 con 0x7f4e300ce8d0 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.344+0000 7f4e2d400640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 0x7f4e14004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.345+0000 7f4e2de00640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e140039b0 0x7f4e14003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.345+0000 7f4e2e800640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300ce8d0 0x7f4e14005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.345+0000 7f4e2d400640 1 -- 172.21.15.7:0/4174859129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300ce8d0 msgr2=0x7f4e14005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:17.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.345+0000 7f4e2d400640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e300ce8d0 0x7f4e14005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.345+0000 7f4e2d400640 1 -- 172.21.15.7:0/4174859129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e140039b0 msgr2=0x7f4e14003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:17.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.345+0000 7f4e2d400640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e140039b0 0x7f4e14003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.345+0000 7f4e2d400640 1 -- 172.21.15.7:0/4174859129 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e14003520 con 0x7f4e14004800 2024-04-25T11:20:17.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.346+0000 7f4e2d400640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 0x7f4e14004c10 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009ac0 tx=0x7f4e14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36ce973d8f540e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.346+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c036070 con 0x7f4e14004800 2024-04-25T11:20:17.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.346+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e1c031400 con 0x7f4e14004800 2024-04-25T11:20:17.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.346+0000 7f4e2ca00640 1 -- 172.21.15.7:0/4174859129 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c009690 con 0x7f4e14004800 2024-04-25T11:20:17.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.347+0000 7f4e2f200640 1 -- 172.21.15.7:0/4174859129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 msgr2=0x7f4e14004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.347+0000 7f4e2f200640 1 --2- 172.21.15.7:0/4174859129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 0x7f4e14004c10 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009ac0 tx=0x7f4e14005830 comp rx=0 tx=0).stop 2024-04-25T11:20:17.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.347+0000 7f4e2e800640 1 -- 172.21.15.7:0/4174859129 reap_dead start 2024-04-25T11:20:17.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.347+0000 7f4e2f200640 1 -- 172.21.15.7:0/4174859129 shutdown_connections 2024-04-25T11:20:17.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.347+0000 7f4e2f200640 1 -- 172.21.15.7:0/4174859129 >> 172.21.15.7:0/4174859129 conn(0x7f4e300740f0 msgr2=0x7f4e3006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:17.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.347+0000 7f4e2f200640 1 -- 172.21.15.7:0/4174859129 shutdown_connections 2024-04-25T11:20:17.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.347+0000 7f4e2f200640 1 -- 172.21.15.7:0/4174859129 wait complete. 2024-04-25T11:20:17.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.350+0000 7f4e2f200640 1 Processor -- start 2024-04-25T11:20:17.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.350+0000 7f4e2f200640 1 -- start start 2024-04-25T11:20:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 0x7f4e30142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e30151330 0x7f4e301e6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e3014b290 0x7f4e301e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30070030 con 0x7f4e14004800 2024-04-25T11:20:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3006feb0 con 0x7f4e3014b290 2024-04-25T11:20:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e30160a10 con 0x7f4e30151330 2024-04-25T11:20:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 0x7f4e30142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 0x7f4e30142e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43072/0 (socket says 172.21.15.7:43072) 2024-04-25T11:20:17.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2de00640 1 -- 172.21.15.7:0/2046210370 learned_addr learned my addr 172.21.15.7:0/2046210370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:17.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2de00640 1 -- 172.21.15.7:0/2046210370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e30151330 msgr2=0x7f4e301e6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2de00640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e30151330 0x7f4e301e6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2de00640 1 -- 172.21.15.7:0/2046210370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e3014b290 msgr2=0x7f4e301e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.353+0000 7f4e2e800640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e3014b290 0x7f4e301e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.354+0000 7f4e2de00640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e3014b290 0x7f4e301e72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.354+0000 7f4e2de00640 1 -- 172.21.15.7:0/2046210370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e301fa460 con 0x7f4e14004800 2024-04-25T11:20:17.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.354+0000 7f4e2de00640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 0x7f4e30142e50 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f4e200115d0 tx=0x7f4e20011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.354+0000 7f4e26000640 1 -- 172.21.15.7:0/2046210370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20015870 con 0x7f4e14004800 2024-04-25T11:20:17.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.354+0000 7f4e26000640 1 -- 172.21.15.7:0/2046210370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e20015a10 con 0x7f4e14004800 2024-04-25T11:20:17.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.354+0000 7f4e26000640 1 -- 172.21.15.7:0/2046210370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e20015e00 con 0x7f4e14004800 2024-04-25T11:20:17.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.354+0000 7f4e2f200640 1 -- 172.21.15.7:0/2046210370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3014c0c0 con 0x7f4e14004800 2024-04-25T11:20:17.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.355+0000 7f4e2f200640 1 -- 172.21.15.7:0/2046210370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e301f6250 con 0x7f4e14004800 2024-04-25T11:20:17.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.356+0000 7f4e2f200640 1 -- 172.21.15.7:0/2046210370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e300cf710 con 0x7f4e14004800 2024-04-25T11:20:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.357+0000 7f4e26000640 1 -- 172.21.15.7:0/2046210370 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4e2000f480 con 0x7f4e14004800 2024-04-25T11:20:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.358+0000 7f4e26000640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 0x7f4e100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.358+0000 7f4e26000640 1 -- 172.21.15.7:0/2046210370 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f4e200961f0 con 0x7f4e14004800 2024-04-25T11:20:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.358+0000 7f4e2d400640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 0x7f4e100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.359+0000 7f4e2d400640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 0x7f4e100769c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c033000 tx=0x7f4e1c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.365+0000 7f4e26000640 1 -- 172.21.15.7:0/2046210370 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e300cf710 con 0x7f4e14004800 2024-04-25T11:20:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.567+0000 7f4e2f200640 1 -- 172.21.15.7:0/2046210370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e30050740 con 0x7f4e14004800 2024-04-25T11:20:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.568+0000 7f4e26000640 1 -- 172.21.15.7:0/2046210370 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7f4e20019030 con 0x7f4e14004800 2024-04-25T11:20:17.567 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:20:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 -- 172.21.15.7:0/2046210370 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 msgr2=0x7f4e100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 0x7f4e100769c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c033000 tx=0x7f4e1c02e000 comp rx=0 tx=0).stop 2024-04-25T11:20:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 -- 172.21.15.7:0/2046210370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 msgr2=0x7f4e30142e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 0x7f4e30142e50 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f4e200115d0 tx=0x7f4e20011a90 comp rx=0 tx=0).stop 2024-04-25T11:20:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 -- 172.21.15.7:0/2046210370 shutdown_connections 2024-04-25T11:20:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e100744b0 0x7f4e100769c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e3014b290 0x7f4e301e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e30151330 0x7f4e301e6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 --2- 172.21.15.7:0/2046210370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e14004800 0x7f4e30142e50 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 -- 172.21.15.7:0/2046210370 >> 172.21.15.7:0/2046210370 conn(0x7f4e300740f0 msgr2=0x7f4e3006ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 -- 172.21.15.7:0/2046210370 shutdown_connections 2024-04-25T11:20:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.572+0000 7f4e2f200640 1 -- 172.21.15.7:0/2046210370 wait complete. 2024-04-25T11:20:17.596 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.880+0000 7f052f200640 1 Processor -- start 2024-04-25T11:20:17.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.880+0000 7f052f200640 1 -- start start 2024-04-25T11:20:17.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.881+0000 7f052f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05300d0890 0x7f05300b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.881+0000 7f052f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05300d0ca0 con 0x7f05300d0480 2024-04-25T11:20:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.881+0000 7f052f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05300ce570 con 0x7f05301583c0 2024-04-25T11:20:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.881+0000 7f052f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f053015f190 con 0x7f05300d0890 2024-04-25T11:20:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.882+0000 7f052e800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05300d0890 0x7f05300b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.882+0000 7f0527e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f05300d0480 0x7f053006eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50018/0 (socket says 172.21.15.7:50018) 2024-04-25T11:20:17.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.882+0000 7f0527e00640 1 -- 172.21.15.7:0/3980153953 learned_addr learned my addr 172.21.15.7:0/3980153953 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.882+0000 7f052d400640 1 -- 172.21.15.7:0/3980153953 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 663564891 0 0) 0x7f0518001110 con 0x7f05300d0480 2024-04-25T11:20:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.882+0000 7f052d400640 1 -- 172.21.15.7:0/3980153953 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0514003520 con 0x7f05300d0480 2024-04-25T11:20:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.882+0000 7f052e800640 1 -- 172.21.15.7:0/3980153953 >> v1:172.21.15.87:6789/0 conn(0x7f05301583c0 legacy=0x7f05300cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.882+0000 7f052e800640 1 -- 172.21.15.7:0/3980153953 >> v1:172.21.15.7:6789/0 conn(0x7f05300d0480 legacy=0x7f053006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.882+0000 7f052e800640 1 -- 172.21.15.7:0/3980153953 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05300ce850 con 0x7f05300d0890 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.883+0000 7f052e800640 1 --2- 172.21.15.7:0/3980153953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05300d0890 0x7f05300b0760 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0528009d90 tx=0x7f052802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf30c87d3ae9901 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.884+0000 7f052d400640 1 -- 172.21.15.7:0/3980153953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528007740 con 0x7f05300d0890 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.884+0000 7f052d400640 1 -- 172.21.15.7:0/3980153953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05280078e0 con 0x7f05300d0890 2024-04-25T11:20:17.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.884+0000 7f052d400640 1 -- 172.21.15.7:0/3980153953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528007cb0 con 0x7f05300d0890 2024-04-25T11:20:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.884+0000 7f052f200640 1 -- 172.21.15.7:0/3980153953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05300d0890 msgr2=0x7f05300b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.884+0000 7f052f200640 1 --2- 172.21.15.7:0/3980153953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05300d0890 0x7f05300b0760 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f0528009d90 tx=0x7f052802a660 comp rx=0 tx=0).stop 2024-04-25T11:20:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.884+0000 7f052f200640 1 -- 172.21.15.7:0/3980153953 shutdown_connections 2024-04-25T11:20:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.884+0000 7f052f200640 1 --2- 172.21.15.7:0/3980153953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05300d0890 0x7f05300b0760 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.884+0000 7f052f200640 1 -- 172.21.15.7:0/3980153953 >> 172.21.15.7:0/3980153953 conn(0x7f05300740f0 msgr2=0x7f053006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.885+0000 7f052f200640 1 -- 172.21.15.7:0/3980153953 shutdown_connections 2024-04-25T11:20:17.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.885+0000 7f052f200640 1 -- 172.21.15.7:0/3980153953 wait complete. 2024-04-25T11:20:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.888+0000 7f052f200640 1 Processor -- start 2024-04-25T11:20:17.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.888+0000 7f052f200640 1 -- start start 2024-04-25T11:20:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f052f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05300d0480 0x7f05301607a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f052f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05300d0890 0x7f0530156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f052f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05301583c0 0x7f05301566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f052f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05300cf0f0 con 0x7f05300d0890 2024-04-25T11:20:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f052f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f05300cee10 con 0x7f05300d0480 2024-04-25T11:20:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f052f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f05300cfd20 con 0x7f05301583c0 2024-04-25T11:20:17.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f0527e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05300d0890 0x7f0530156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f0527e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05300d0890 0x7f0530156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43098/0 (socket says 172.21.15.7:43098) 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f0527e00640 1 -- 172.21.15.7:0/1617404981 learned_addr learned my addr 172.21.15.7:0/1617404981 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f052de00640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05300d0480 0x7f05301607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f0527e00640 1 -- 172.21.15.7:0/1617404981 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05301583c0 msgr2=0x7f05301566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f0527e00640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05301583c0 0x7f05301566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f0527e00640 1 -- 172.21.15.7:0/1617404981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05300d0480 msgr2=0x7f05301607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f0527e00640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05300d0480 0x7f05301607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f0527e00640 1 -- 172.21.15.7:0/1617404981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0530161110 con 0x7f05300d0890 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.889+0000 7f052de00640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05300d0480 0x7f05301607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.890+0000 7f0527e00640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05300d0890 0x7f0530156170 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f051800c340 tx=0x7f051800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.890+0000 7f0526000640 1 -- 172.21.15.7:0/1617404981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0518017070 con 0x7f05300d0890 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.890+0000 7f0526000640 1 -- 172.21.15.7:0/1617404981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0518005ce0 con 0x7f05300d0890 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.890+0000 7f052f200640 1 -- 172.21.15.7:0/1617404981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05301613f0 con 0x7f05300d0890 2024-04-25T11:20:17.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.890+0000 7f0526000640 1 -- 172.21.15.7:0/1617404981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05180072f0 con 0x7f05300d0890 2024-04-25T11:20:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.890+0000 7f052f200640 1 -- 172.21.15.7:0/1617404981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05301619b0 con 0x7f05300d0890 2024-04-25T11:20:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.891+0000 7f052f200640 1 -- 172.21.15.7:0/1617404981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0530050740 con 0x7f05300d0890 2024-04-25T11:20:17.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.893+0000 7f0526000640 1 -- 172.21.15.7:0/1617404981 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f051801a070 con 0x7f05300d0890 2024-04-25T11:20:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.893+0000 7f0526000640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f050c0744e0 0x7f050c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.894+0000 7f0526000640 1 -- 172.21.15.7:0/1617404981 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f0518094720 con 0x7f05300d0890 2024-04-25T11:20:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.894+0000 7f052de00640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f050c0744e0 0x7f050c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:17.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.894+0000 7f052de00640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f050c0744e0 0x7f050c0769f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0520000f60 tx=0x7f0520009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:17.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:17.899+0000 7f0526000640 1 -- 172.21.15.7:0/1617404981 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f05300d0ca0 con 0x7f05300d0890 2024-04-25T11:20:18.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.101+0000 7f052f200640 1 -- 172.21.15.7:0/1617404981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0530070dd0 con 0x7f05300d0890 2024-04-25T11:20:18.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.102+0000 7f0526000640 1 -- 172.21.15.7:0/1617404981 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7f051805ec80 con 0x7f05300d0890 2024-04-25T11:20:18.101 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:20:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 -- 172.21.15.7:0/1617404981 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f050c0744e0 msgr2=0x7f050c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f050c0744e0 0x7f050c0769f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0520000f60 tx=0x7f0520009040 comp rx=0 tx=0).stop 2024-04-25T11:20:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 -- 172.21.15.7:0/1617404981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05300d0890 msgr2=0x7f0530156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05300d0890 0x7f0530156170 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f051800c340 tx=0x7f051800c800 comp rx=0 tx=0).stop 2024-04-25T11:20:18.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 -- 172.21.15.7:0/1617404981 shutdown_connections 2024-04-25T11:20:18.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f050c0744e0 0x7f050c0769f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05301583c0 0x7f05301566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05300d0890 0x7f0530156170 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 --2- 172.21.15.7:0/1617404981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05300d0480 0x7f05301607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 -- 172.21.15.7:0/1617404981 >> 172.21.15.7:0/1617404981 conn(0x7f05300740f0 msgr2=0x7f053015cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:18.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 -- 172.21.15.7:0/1617404981 shutdown_connections 2024-04-25T11:20:18.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.105+0000 7f052f200640 1 -- 172.21.15.7:0/1617404981 wait complete. 2024-04-25T11:20:18.132 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:18.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.413+0000 7fad71000640 1 Processor -- start 2024-04-25T11:20:18.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.413+0000 7fad71000640 1 -- start start 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.416+0000 7fad71000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fad6c06fc30 0x7fad6c070040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.416+0000 7fad71000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad6c070580 con 0x7fad6c070b60 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.416+0000 7fad71000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad6c15db10 con 0x7fad6c15e560 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.416+0000 7fad71000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fad6c15e0d0 con 0x7fad6c06fc30 2024-04-25T11:20:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.416+0000 7fad69e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fad6c070b60 0x7fad6c15d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50030/0 (socket says 172.21.15.7:50030) 2024-04-25T11:20:18.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.416+0000 7fad69e00640 1 -- 172.21.15.7:0/3708754548 learned_addr learned my addr 172.21.15.7:0/3708754548 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:18.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.417+0000 7fad63e00640 1 --2- 172.21.15.7:0/3708754548 >> v2:172.21.15.87:3300/0 conn(0x7fad6c06fc30 0x7fad6c070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.417+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3876541338 0 0) 0x7fad5c001110 con 0x7fad6c070b60 2024-04-25T11:20:18.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.417+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad50003520 con 0x7fad6c070b60 2024-04-25T11:20:18.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.417+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1617033089 0 0) 0x7fad50003520 con 0x7fad6c070b60 2024-04-25T11:20:18.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.417+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 >> v2:172.21.15.87:3300/0 conn(0x7fad6c06fc30 msgr2=0x7fad6c070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.417+0000 7fad69400640 1 --2- 172.21.15.7:0/3708754548 >> v2:172.21.15.87:3300/0 conn(0x7fad6c06fc30 0x7fad6c070040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.417+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 >> v1:172.21.15.87:6789/0 conn(0x7fad6c15e560 legacy=0x7fad6c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.417+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c0af3c0 con 0x7fad6c070b60 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.417+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fad5c001840 con 0x7fad6c070b60 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.417+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 >> v1:172.21.15.7:6789/0 conn(0x7fad6c070b60 legacy=0x7fad6c15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad69400640 1 --2- 172.21.15.7:0/3708754548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad500039b0 0x7fad50003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad69400640 1 --2- 172.21.15.7:0/3708754548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad50004800 0x7fad50004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad69400640 1 --2- 172.21.15.7:0/3708754548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50005270 0x7fad5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c15e0d0 con 0x7fad50005270 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fad5000c190 con 0x7fad50004800 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad5000c390 con 0x7fad500039b0 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad6a800640 1 --2- 172.21.15.7:0/3708754548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50005270 0x7fad5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad69e00640 1 --2- 172.21.15.7:0/3708754548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad500039b0 0x7fad50003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad6a800640 1 -- 172.21.15.7:0/3708754548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad500039b0 msgr2=0x7fad50003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad6a800640 1 --2- 172.21.15.7:0/3708754548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad500039b0 0x7fad50003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad6a800640 1 -- 172.21.15.7:0/3708754548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad50004800 msgr2=0x7fad50004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.418+0000 7fad6a800640 1 --2- 172.21.15.7:0/3708754548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad50004800 0x7fad50004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.420+0000 7fad6a800640 1 -- 172.21.15.7:0/3708754548 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c0af3c0 con 0x7fad50005270 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.420+0000 7fad69e00640 1 --2- 172.21.15.7:0/3708754548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad500039b0 0x7fad50003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.420+0000 7fad6a800640 1 -- 172.21.15.7:0/3708754548 reap_dead start 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.421+0000 7fad6a800640 1 --2- 172.21.15.7:0/3708754548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50005270 0x7fad5000b9c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fad64001e20 tx=0x7fad50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16482f596142e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.421+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64001a70 con 0x7fad50005270 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.421+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6402ae30 con 0x7fad50005270 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.421+0000 7fad69400640 1 -- 172.21.15.7:0/3708754548 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64003b70 con 0x7fad50005270 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.422+0000 7fad71000640 1 -- 172.21.15.7:0/3708754548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50005270 msgr2=0x7fad5000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.422+0000 7fad71000640 1 --2- 172.21.15.7:0/3708754548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50005270 0x7fad5000b9c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fad64001e20 tx=0x7fad50005830 comp rx=0 tx=0).stop 2024-04-25T11:20:18.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.422+0000 7fad71000640 1 -- 172.21.15.7:0/3708754548 shutdown_connections 2024-04-25T11:20:18.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.422+0000 7fad71000640 1 --2- 172.21.15.7:0/3708754548 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad50005270 0x7fad5000b9c0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.422+0000 7fad71000640 1 -- 172.21.15.7:0/3708754548 >> 172.21.15.7:0/3708754548 conn(0x7fad6c0742c0 msgr2=0x7fad6c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:18.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.422+0000 7fad71000640 1 -- 172.21.15.7:0/3708754548 shutdown_connections 2024-04-25T11:20:18.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.422+0000 7fad71000640 1 -- 172.21.15.7:0/3708754548 wait complete. 2024-04-25T11:20:18.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.423+0000 7fad71000640 1 Processor -- start 2024-04-25T11:20:18.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.423+0000 7fad71000640 1 -- start start 2024-04-25T11:20:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad71000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad50005270 0x7fad6c147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad71000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad6c1fa9a0 0x7fad6c147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad71000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad6c1f9e00 0x7fad6c1fa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad71000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c0af980 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad71000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c153b00 con 0x7fad6c1f9e00 2024-04-25T11:20:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad71000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad6c15dd60 con 0x7fad50005270 2024-04-25T11:20:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad63e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad6c1fa9a0 0x7fad6c147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad63e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad6c1fa9a0 0x7fad6c147520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43110/0 (socket says 172.21.15.7:43110) 2024-04-25T11:20:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad63e00640 1 -- 172.21.15.7:0/1824834955 learned_addr learned my addr 172.21.15.7:0/1824834955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad69e00640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad50005270 0x7fad6c147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad6a800640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad6c1f9e00 0x7fad6c1fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad63e00640 1 -- 172.21.15.7:0/1824834955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad50005270 msgr2=0x7fad6c147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad63e00640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad50005270 0x7fad6c147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad63e00640 1 -- 172.21.15.7:0/1824834955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad6c1f9e00 msgr2=0x7fad6c1fa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad63e00640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad6c1f9e00 0x7fad6c1fa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad63e00640 1 -- 172.21.15.7:0/1824834955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c14aa30 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad63e00640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad6c1fa9a0 0x7fad6c147520 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fad5400e860 tx=0x7fad54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad62000640 1 -- 172.21.15.7:0/1824834955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad54016070 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.426+0000 7fad62000640 1 -- 172.21.15.7:0/1824834955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad5400ca30 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.427+0000 7fad62000640 1 -- 172.21.15.7:0/1824834955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5400b040 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.427+0000 7fad71000640 1 -- 172.21.15.7:0/1824834955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c1fa750 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.427+0000 7fad71000640 1 -- 172.21.15.7:0/1824834955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c14af80 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.428+0000 7fad71000640 1 -- 172.21.15.7:0/1824834955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6c050740 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.431+0000 7fad62000640 1 -- 172.21.15.7:0/1824834955 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad54019070 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.431+0000 7fad62000640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad480743c0 0x7fad480768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.431+0000 7fad62000640 1 -- 172.21.15.7:0/1824834955 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fad54094f00 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.431+0000 7fad69e00640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad480743c0 0x7fad480768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.433+0000 7fad69e00640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad480743c0 0x7fad480768d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0012b0 tx=0x7fad5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.436+0000 7fad62000640 1 -- 172.21.15.7:0/1824834955 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad6c1fa750 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.638+0000 7fad71000640 1 -- 172.21.15.7:0/1824834955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad6c070940 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.640+0000 7fad62000640 1 -- 172.21.15.7:0/1824834955 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7fad5405f460 con 0x7fad6c1fa9a0 2024-04-25T11:20:18.637 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:20:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.643+0000 7fad71000640 1 -- 172.21.15.7:0/1824834955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad480743c0 msgr2=0x7fad480768d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.643+0000 7fad71000640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad480743c0 0x7fad480768d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0012b0 tx=0x7fad5c009040 comp rx=0 tx=0).stop 2024-04-25T11:20:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.643+0000 7fad71000640 1 -- 172.21.15.7:0/1824834955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad6c1fa9a0 msgr2=0x7fad6c147520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.643+0000 7fad71000640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad6c1fa9a0 0x7fad6c147520 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fad5400e860 tx=0x7fad54004730 comp rx=0 tx=0).stop 2024-04-25T11:20:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.643+0000 7fad71000640 1 -- 172.21.15.7:0/1824834955 shutdown_connections 2024-04-25T11:20:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.643+0000 7fad71000640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad480743c0 0x7fad480768d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.643+0000 7fad71000640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad6c1f9e00 0x7fad6c1fa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.643+0000 7fad71000640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad6c1fa9a0 0x7fad6c147520 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.643+0000 7fad71000640 1 --2- 172.21.15.7:0/1824834955 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad50005270 0x7fad6c147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.643+0000 7fad71000640 1 -- 172.21.15.7:0/1824834955 >> 172.21.15.7:0/1824834955 conn(0x7fad6c0742c0 msgr2=0x7fad6c152340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.643+0000 7fad71000640 1 -- 172.21.15.7:0/1824834955 shutdown_connections 2024-04-25T11:20:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.644+0000 7fad71000640 1 -- 172.21.15.7:0/1824834955 wait complete. 2024-04-25T11:20:18.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:20:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.958+0000 7fedab000640 1 Processor -- start 2024-04-25T11:20:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.958+0000 7fedab000640 1 -- start start 2024-04-25T11:20:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.960+0000 7fedab000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feda41602e0 0x7feda406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.960+0000 7fedab000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feda406e8b0 0x7feda40d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.960+0000 7fedab000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feda40d4ca0 0x7feda40b7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.960+0000 7fedab000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7feda406ecc0 con 0x7feda41602e0 2024-04-25T11:20:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.960+0000 7fedab000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7feda40b9060 con 0x7feda40d4ca0 2024-04-25T11:20:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.960+0000 7fedab000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feda40d5500 con 0x7feda406e8b0 2024-04-25T11:20:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.960+0000 7feda9c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feda41602e0 0x7feda406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.960+0000 7feda9c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feda41602e0 0x7feda406e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43114/0 (socket says 172.21.15.7:43114) 2024-04-25T11:20:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.960+0000 7feda9c00640 1 -- 172.21.15.7:0/3379052448 learned_addr learned my addr 172.21.15.7:0/3379052448 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.961+0000 7fedaa600640 1 --2- 172.21.15.7:0/3379052448 >> v2:172.21.15.87:3300/0 conn(0x7feda40d4ca0 0x7feda40b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.961+0000 7feda9200640 1 --2- 172.21.15.7:0/3379052448 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feda406e8b0 0x7feda40d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.961+0000 7feda9c00640 1 -- 172.21.15.7:0/3379052448 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feda406e8b0 msgr2=0x7feda40d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.961+0000 7feda9c00640 1 --2- 172.21.15.7:0/3379052448 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feda406e8b0 0x7feda40d4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.961+0000 7feda9c00640 1 -- 172.21.15.7:0/3379052448 >> v2:172.21.15.87:3300/0 conn(0x7feda40d4ca0 msgr2=0x7feda40b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.961+0000 7feda9c00640 1 --2- 172.21.15.7:0/3379052448 >> v2:172.21.15.87:3300/0 conn(0x7feda40d4ca0 0x7feda40b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.961+0000 7feda9c00640 1 -- 172.21.15.7:0/3379052448 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda40b9300 con 0x7feda41602e0 2024-04-25T11:20:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.962+0000 7feda9c00640 1 --2- 172.21.15.7:0/3379052448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda41602e0 0x7feda406e370 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7feda40d1fa0 tx=0x7fed9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429797715251e491 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.962+0000 7feda3e00640 1 -- 172.21.15.7:0/3379052448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98037070 con 0x7feda41602e0 2024-04-25T11:20:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.962+0000 7feda3e00640 1 -- 172.21.15.7:0/3379052448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed98032440 con 0x7feda41602e0 2024-04-25T11:20:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.962+0000 7feda3e00640 1 -- 172.21.15.7:0/3379052448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98032810 con 0x7feda41602e0 2024-04-25T11:20:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.963+0000 7fedab000640 1 -- 172.21.15.7:0/3379052448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda41602e0 msgr2=0x7feda406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.963+0000 7fedab000640 1 --2- 172.21.15.7:0/3379052448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda41602e0 0x7feda406e370 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7feda40d1fa0 tx=0x7fed9802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.963+0000 7fedab000640 1 -- 172.21.15.7:0/3379052448 shutdown_connections 2024-04-25T11:20:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.963+0000 7fedab000640 1 --2- 172.21.15.7:0/3379052448 >> v2:172.21.15.87:3300/0 conn(0x7feda40d4ca0 0x7feda40b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.963+0000 7fedab000640 1 --2- 172.21.15.7:0/3379052448 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feda406e8b0 0x7feda40d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.963+0000 7fedab000640 1 --2- 172.21.15.7:0/3379052448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda41602e0 0x7feda406e370 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.963+0000 7fedab000640 1 -- 172.21.15.7:0/3379052448 >> 172.21.15.7:0/3379052448 conn(0x7feda40d2670 msgr2=0x7feda40d2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.963+0000 7fedab000640 1 -- 172.21.15.7:0/3379052448 shutdown_connections 2024-04-25T11:20:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.963+0000 7fedab000640 1 -- 172.21.15.7:0/3379052448 wait complete. 2024-04-25T11:20:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.965+0000 7fedab000640 1 Processor -- start 2024-04-25T11:20:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.966+0000 7fedab000640 1 -- start start 2024-04-25T11:20:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.967+0000 7fedab000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda406e8b0 0x7feda4156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.967+0000 7fedab000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feda40d4ca0 0x7feda41475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.967+0000 7fedab000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda41602e0 0x7feda40cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.967+0000 7fedab000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feda40d50b0 con 0x7feda406e8b0 2024-04-25T11:20:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.967+0000 7fedab000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feda40b98c0 con 0x7feda41602e0 2024-04-25T11:20:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.967+0000 7fedab000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feda40d5230 con 0x7feda40d4ca0 2024-04-25T11:20:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.967+0000 7feda9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda406e8b0 0x7feda4156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.967+0000 7feda9c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda406e8b0 0x7feda4156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43118/0 (socket says 172.21.15.7:43118) 2024-04-25T11:20:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.967+0000 7feda9c00640 1 -- 172.21.15.7:0/1374632600 learned_addr learned my addr 172.21.15.7:0/1374632600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.968+0000 7feda9200640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feda40d4ca0 0x7feda41475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.968+0000 7feda9c00640 1 -- 172.21.15.7:0/1374632600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feda40d4ca0 msgr2=0x7feda41475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.968+0000 7feda9c00640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feda40d4ca0 0x7feda41475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.968+0000 7feda9c00640 1 -- 172.21.15.7:0/1374632600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda41602e0 msgr2=0x7feda40cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.968+0000 7feda9c00640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda41602e0 0x7feda40cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.968+0000 7feda9c00640 1 -- 172.21.15.7:0/1374632600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda4147ae0 con 0x7feda406e8b0 2024-04-25T11:20:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.968+0000 7feda9c00640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda406e8b0 0x7feda4156e60 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fed98000c00 tx=0x7fed980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.968+0000 7feda1600640 1 -- 172.21.15.7:0/1374632600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98041020 con 0x7feda406e8b0 2024-04-25T11:20:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.968+0000 7feda1600640 1 -- 172.21.15.7:0/1374632600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed980306c0 con 0x7feda406e8b0 2024-04-25T11:20:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.969+0000 7fedab000640 1 -- 172.21.15.7:0/1374632600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda4155dc0 con 0x7feda406e8b0 2024-04-25T11:20:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.969+0000 7feda1600640 1 -- 172.21.15.7:0/1374632600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98037040 con 0x7feda406e8b0 2024-04-25T11:20:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.969+0000 7fedab000640 1 -- 172.21.15.7:0/1374632600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda4156380 con 0x7feda406e8b0 2024-04-25T11:20:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.970+0000 7fedab000640 1 -- 172.21.15.7:0/1374632600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda4050740 con 0x7feda406e8b0 2024-04-25T11:20:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.971+0000 7feda1600640 1 -- 172.21.15.7:0/1374632600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed98044020 con 0x7feda406e8b0 2024-04-25T11:20:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.972+0000 7feda1600640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed840744e0 0x7fed840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.972+0000 7feda1600640 1 -- 172.21.15.7:0/1374632600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fed980b5ca0 con 0x7feda406e8b0 2024-04-25T11:20:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.973+0000 7feda9200640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed840744e0 0x7fed840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.973+0000 7feda9200640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed840744e0 0x7fed840769f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fed90006fd0 tx=0x7fed90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:18.977+0000 7feda1600640 1 -- 172.21.15.7:0/1374632600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed9804b030 con 0x7feda406e8b0 2024-04-25T11:20:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.197+0000 7fedab000640 1 -- 172.21.15.7:0/1374632600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feda4069c50 con 0x7feda406e8b0 2024-04-25T11:20:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.198+0000 7feda1600640 1 -- 172.21.15.7:0/1374632600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v118) ==== 80+0+0 (secure 0 0 0) 0x7feda4147ae0 con 0x7feda406e8b0 2024-04-25T11:20:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:20:19.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.201+0000 7fedab000640 1 -- 172.21.15.7:0/1374632600 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed840744e0 msgr2=0x7fed840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.201+0000 7fedab000640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed840744e0 0x7fed840769f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fed90006fd0 tx=0x7fed90008040 comp rx=0 tx=0).stop 2024-04-25T11:20:19.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.201+0000 7fedab000640 1 -- 172.21.15.7:0/1374632600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda406e8b0 msgr2=0x7feda4156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.201+0000 7fedab000640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda406e8b0 0x7feda4156e60 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fed98000c00 tx=0x7fed980046c0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.201+0000 7fedab000640 1 -- 172.21.15.7:0/1374632600 shutdown_connections 2024-04-25T11:20:19.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.201+0000 7fedab000640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed840744e0 0x7fed840769f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.202+0000 7fedab000640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda41602e0 0x7feda40cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.202+0000 7fedab000640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feda40d4ca0 0x7feda41475a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.202+0000 7fedab000640 1 --2- 172.21.15.7:0/1374632600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feda406e8b0 0x7feda4156e60 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.202+0000 7fedab000640 1 -- 172.21.15.7:0/1374632600 >> 172.21.15.7:0/1374632600 conn(0x7feda40d2670 msgr2=0x7feda40badb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:19.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.202+0000 7fedab000640 1 -- 172.21.15.7:0/1374632600 shutdown_connections 2024-04-25T11:20:19.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.202+0000 7fedab000640 1 -- 172.21.15.7:0/1374632600 wait complete. 2024-04-25T11:20:19.223 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:20:19.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:19.224 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T11:20:19.323 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:20:19.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:19.324 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:20:19.375 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:20:19.376 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:20:19.376 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:19.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:20:19.434 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:20:19.434 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:20:19.434 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:19.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:20:19.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.794+0000 7ffbeca00640 1 Processor -- start 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.794+0000 7ffbeca00640 1 -- start start 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.797+0000 7ffbeca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffbe8070890 0x7ffbe815c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.797+0000 7ffbeca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffbe806ce00 0x7ffbe806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.797+0000 7ffbeca00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbe806ef10 con 0x7ffbe80b5140 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.797+0000 7ffbeca00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ffbe806f210 con 0x7ffbe806ce00 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.797+0000 7ffbeca00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ffbe806d750 con 0x7ffbe8070890 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.797+0000 7ffbe7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffbe806ce00 0x7ffbe806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.797+0000 7ffbe7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffbe806ce00 0x7ffbe806d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43120/0 (socket says 172.21.15.7:43120) 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.797+0000 7ffbe7e00640 1 -- 172.21.15.7:0/1302537541 learned_addr learned my addr 172.21.15.7:0/1302537541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.798+0000 7ffbe7400640 1 --2- 172.21.15.7:0/1302537541 >> v2:172.21.15.87:3300/0 conn(0x7ffbe8070890 0x7ffbe815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.798+0000 7ffbe7e00640 1 -- 172.21.15.7:0/1302537541 >> v2:172.21.15.87:3300/0 conn(0x7ffbe8070890 msgr2=0x7ffbe815c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:19.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.798+0000 7ffbe7e00640 1 --2- 172.21.15.7:0/1302537541 >> v2:172.21.15.87:3300/0 conn(0x7ffbe8070890 0x7ffbe815c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.798+0000 7ffbe7e00640 1 -- 172.21.15.7:0/1302537541 >> v1:172.21.15.87:6789/0 conn(0x7ffbe80b5140 legacy=0x7ffbe806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:19.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.798+0000 7ffbe7e00640 1 -- 172.21.15.7:0/1302537541 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe80b6390 con 0x7ffbe806ce00 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.799+0000 7ffbe7e00640 1 --2- 172.21.15.7:0/1302537541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe806ce00 0x7ffbe806d210 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80744f0 tx=0x7ffbe002b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1b5444c8077071 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.799+0000 7ffbe6a00640 1 -- 172.21.15.7:0/1302537541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0037070 con 0x7ffbe806ce00 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.799+0000 7ffbe6a00640 1 -- 172.21.15.7:0/1302537541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbe0032600 con 0x7ffbe806ce00 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.799+0000 7ffbe6a00640 1 -- 172.21.15.7:0/1302537541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00329d0 con 0x7ffbe806ce00 2024-04-25T11:20:19.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.800+0000 7ffbeca00640 1 -- 172.21.15.7:0/1302537541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe806ce00 msgr2=0x7ffbe806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.800+0000 7ffbeca00640 1 --2- 172.21.15.7:0/1302537541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe806ce00 0x7ffbe806d210 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80744f0 tx=0x7ffbe002b770 comp rx=0 tx=0).stop 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.800+0000 7ffbeca00640 1 -- 172.21.15.7:0/1302537541 shutdown_connections 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.800+0000 7ffbeca00640 1 --2- 172.21.15.7:0/1302537541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe806ce00 0x7ffbe806d210 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.800+0000 7ffbeca00640 1 --2- 172.21.15.7:0/1302537541 >> v2:172.21.15.87:3300/0 conn(0x7ffbe8070890 0x7ffbe815c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.800+0000 7ffbeca00640 1 -- 172.21.15.7:0/1302537541 >> 172.21.15.7:0/1302537541 conn(0x7ffbe80b3820 msgr2=0x7ffbe80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.800+0000 7ffbeca00640 1 -- 172.21.15.7:0/1302537541 shutdown_connections 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.800+0000 7ffbeca00640 1 -- 172.21.15.7:0/1302537541 wait complete. 2024-04-25T11:20:19.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.802+0000 7ffbeca00640 1 Processor -- start 2024-04-25T11:20:19.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.803+0000 7ffbeca00640 1 -- start start 2024-04-25T11:20:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.804+0000 7ffbeca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe806ce00 0x7ffbe80c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.804+0000 7ffbeca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8070890 0x7ffbe809f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.804+0000 7ffbeca00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffbe80b5140 0x7ffbe81500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.804+0000 7ffbeca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe80b5550 con 0x7ffbe8070890 2024-04-25T11:20:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.804+0000 7ffbeca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe806fdb0 con 0x7ffbe806ce00 2024-04-25T11:20:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.804+0000 7ffbeca00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe80b6950 con 0x7ffbe80b5140 2024-04-25T11:20:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.805+0000 7ffbdea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8070890 0x7ffbe809f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.805+0000 7ffbdea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8070890 0x7ffbe809f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43128/0 (socket says 172.21.15.7:43128) 2024-04-25T11:20:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.805+0000 7ffbdea00640 1 -- 172.21.15.7:0/1251609272 learned_addr learned my addr 172.21.15.7:0/1251609272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.805+0000 7ffbe7400640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe806ce00 0x7ffbe80c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.805+0000 7ffbe7e00640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffbe80b5140 0x7ffbe81500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.805+0000 7ffbdea00640 1 -- 172.21.15.7:0/1251609272 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffbe80b5140 msgr2=0x7ffbe81500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.805+0000 7ffbdea00640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffbe80b5140 0x7ffbe81500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.805+0000 7ffbdea00640 1 -- 172.21.15.7:0/1251609272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe806ce00 msgr2=0x7ffbe80c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.805+0000 7ffbdea00640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe806ce00 0x7ffbe80c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.805+0000 7ffbdea00640 1 -- 172.21.15.7:0/1251609272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe809d520 con 0x7ffbe8070890 2024-04-25T11:20:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.805+0000 7ffbdea00640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8070890 0x7ffbe809f020 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000b420 tx=0x7ffbd000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.806+0000 7ffbdfe00640 1 -- 172.21.15.7:0/1251609272 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0011830 con 0x7ffbe8070890 2024-04-25T11:20:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.806+0000 7ffbdfe00640 1 -- 172.21.15.7:0/1251609272 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd0016070 con 0x7ffbe8070890 2024-04-25T11:20:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.806+0000 7ffbdfe00640 1 -- 172.21.15.7:0/1251609272 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0011c10 con 0x7ffbe8070890 2024-04-25T11:20:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.806+0000 7ffbeca00640 1 -- 172.21.15.7:0/1251609272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe809d800 con 0x7ffbe8070890 2024-04-25T11:20:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.806+0000 7ffbeca00640 1 -- 172.21.15.7:0/1251609272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe809ddc0 con 0x7ffbe8070890 2024-04-25T11:20:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.807+0000 7ffbeca00640 1 -- 172.21.15.7:0/1251609272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe8050740 con 0x7ffbe8070890 2024-04-25T11:20:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.808+0000 7ffbdfe00640 1 -- 172.21.15.7:0/1251609272 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ffbd001b460 con 0x7ffbe8070890 2024-04-25T11:20:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.809+0000 7ffbdfe00640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffbc40744e0 0x7ffbc40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.809+0000 7ffbdfe00640 1 -- 172.21.15.7:0/1251609272 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7ffbd0094ed0 con 0x7ffbe8070890 2024-04-25T11:20:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.810+0000 7ffbe7400640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffbc40744e0 0x7ffbc40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.810+0000 7ffbe7400640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffbc40744e0 0x7ffbc40769f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8006fd0 tx=0x7ffbd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:19.814+0000 7ffbdfe00640 1 -- 172.21.15.7:0/1251609272 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ffbd0020020 con 0x7ffbe8070890 2024-04-25T11:20:20.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.061+0000 7ffbeca00640 1 -- 172.21.15.7:0/1251609272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbe8070440 con 0x7ffbe8070890 2024-04-25T11:20:20.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.062+0000 7ffbdfe00640 1 -- 172.21.15.7:0/1251609272 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+5174 (secure 0 0 0) 0x7ffbd005f4d0 con 0x7ffbe8070890 2024-04-25T11:20:20.059 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:20.059 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":79,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/326382713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":326382713},{"type":"v1","addr":"172.21.15.7:6837","nonce":326382713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2007810270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2007810270},{"type":"v1","addr":"172.21.15.87:6835","nonce":2007810270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1753656512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1753656512},{"type":"v1","addr":"172.21.15.7:6836","nonce":1753656512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:47.807534+0000","modified":"2024-04-25T11:20:12.325419+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":9,"addr":"172.21.15.87:6834/1924056839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1924056839},{"type":"v1","addr":"172.21.15.87:6834","nonce":1924056839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8007,"qdb_cluster":[8007]},"id":8}]} 2024-04-25T11:20:20.060 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 79 2024-04-25T11:20:20.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 -- 172.21.15.7:0/1251609272 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffbc40744e0 msgr2=0x7ffbc40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffbc40744e0 0x7ffbc40769f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8006fd0 tx=0x7ffbd8008040 comp rx=0 tx=0).stop 2024-04-25T11:20:20.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 -- 172.21.15.7:0/1251609272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8070890 msgr2=0x7ffbe809f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8070890 0x7ffbe809f020 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000b420 tx=0x7ffbd000b8e0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 -- 172.21.15.7:0/1251609272 shutdown_connections 2024-04-25T11:20:20.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ffbc40744e0 0x7ffbc40769f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffbe80b5140 0x7ffbe81500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8070890 0x7ffbe809f020 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 --2- 172.21.15.7:0/1251609272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbe806ce00 0x7ffbe80c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 -- 172.21.15.7:0/1251609272 >> 172.21.15.7:0/1251609272 conn(0x7ffbe80b3820 msgr2=0x7ffbe80b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:20.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 -- 172.21.15.7:0/1251609272 shutdown_connections 2024-04-25T11:20:20.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.065+0000 7ffbeca00640 1 -- 172.21.15.7:0/1251609272 wait complete. 2024-04-25T11:20:20.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:20.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.374+0000 7f9693800640 1 Processor -- start 2024-04-25T11:20:20.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.374+0000 7f9693800640 1 -- start start 2024-04-25T11:20:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.376+0000 7f9693800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f968c15ef90 0x7f968c09f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.376+0000 7f9693800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f968c0af540 0x7f968c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.377+0000 7f9693800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968c15f4e0 con 0x7f968c15bc40 2024-04-25T11:20:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.377+0000 7f9693800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f968c071190 con 0x7f968c15ef90 2024-04-25T11:20:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.377+0000 7f9693800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f968c0b0560 con 0x7f968c0af540 2024-04-25T11:20:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.377+0000 7f968be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f968c15bc40 0x7f968c0aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46682/0 (socket says 172.21.15.7:46682) 2024-04-25T11:20:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.377+0000 7f968be00640 1 -- 172.21.15.7:0/3601825328 learned_addr learned my addr 172.21.15.7:0/3601825328 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.377+0000 7f9690e00640 1 --2- 172.21.15.7:0/3601825328 >> v2:172.21.15.87:3300/0 conn(0x7f968c15ef90 0x7f968c09f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.377+0000 7f9691800640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f968c0af540 0x7f968c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.377+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1125761495 0 0) 0x7f968c15f4e0 con 0x7f968c15bc40 2024-04-25T11:20:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.377+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9670003520 con 0x7f968c15bc40 2024-04-25T11:20:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 103139873 0 0) 0x7f9670003520 con 0x7f968c15bc40 2024-04-25T11:20:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f968c0af540 msgr2=0x7f968c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f968c0af540 0x7f968c0cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 >> v2:172.21.15.87:3300/0 conn(0x7f968c15ef90 msgr2=0x7f968c09f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 --2- 172.21.15.7:0/3601825328 >> v2:172.21.15.87:3300/0 conn(0x7f968c15ef90 0x7f968c09f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c0b0840 con 0x7f968c15bc40 2024-04-25T11:20:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f967c0019f0 con 0x7f968c15bc40 2024-04-25T11:20:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 >> v1:172.21.15.7:6789/0 conn(0x7f968c15bc40 legacy=0x7f968c0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96700039b0 0x7f9670003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9670004800 0x7f9670004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9670005150 0x7f967000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f968c071190 con 0x7f9670005150 2024-04-25T11:20:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f968c0b0560 con 0x7f96700039b0 2024-04-25T11:20:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f967000bf00 con 0x7f9670004800 2024-04-25T11:20:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.378+0000 7f9691800640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9670005150 0x7f967000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.379+0000 7f968be00640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9670004800 0x7f9670004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.379+0000 7f9690e00640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96700039b0 0x7f9670003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.379+0000 7f9691800640 1 -- 172.21.15.7:0/3601825328 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9670004800 msgr2=0x7f9670004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.379+0000 7f9691800640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9670004800 0x7f9670004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.379+0000 7f9691800640 1 -- 172.21.15.7:0/3601825328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96700039b0 msgr2=0x7f9670003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.379+0000 7f9691800640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96700039b0 0x7f9670003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.379+0000 7f9691800640 1 -- 172.21.15.7:0/3601825328 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9670003520 con 0x7f9670005150 2024-04-25T11:20:20.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.380+0000 7f9691800640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9670005150 0x7f967000b9c0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f9684009ac0 tx=0x7f9670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd61078fa4ee5332 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.380+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684036070 con 0x7f9670005150 2024-04-25T11:20:20.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.380+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9684031400 con 0x7f9670005150 2024-04-25T11:20:20.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.380+0000 7f968b400640 1 -- 172.21.15.7:0/3601825328 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684009690 con 0x7f9670005150 2024-04-25T11:20:20.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.381+0000 7f9693800640 1 -- 172.21.15.7:0/3601825328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9670005150 msgr2=0x7f967000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.381+0000 7f9693800640 1 --2- 172.21.15.7:0/3601825328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9670005150 0x7f967000b9c0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f9684009ac0 tx=0x7f9670005830 comp rx=0 tx=0).stop 2024-04-25T11:20:20.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.381+0000 7f9691800640 1 -- 172.21.15.7:0/3601825328 reap_dead start 2024-04-25T11:20:20.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.381+0000 7f9693800640 1 -- 172.21.15.7:0/3601825328 shutdown_connections 2024-04-25T11:20:20.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.381+0000 7f9693800640 1 -- 172.21.15.7:0/3601825328 >> 172.21.15.7:0/3601825328 conn(0x7f968c074200 msgr2=0x7f968c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:20.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.381+0000 7f9693800640 1 -- 172.21.15.7:0/3601825328 shutdown_connections 2024-04-25T11:20:20.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.381+0000 7f9693800640 1 -- 172.21.15.7:0/3601825328 wait complete. 2024-04-25T11:20:20.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.384+0000 7f9693800640 1 Processor -- start 2024-04-25T11:20:20.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.384+0000 7f9693800640 1 -- start start 2024-04-25T11:20:20.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.385+0000 7f9693800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9670005150 0x7f968c0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.385+0000 7f9693800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c0b5710 0x7f968c1f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.385+0000 7f9693800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f968c0b27b0 0x7f968c0b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.385+0000 7f9693800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f968c0b1320 con 0x7f968c0b5710 2024-04-25T11:20:20.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.385+0000 7f9693800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f968c0b0e00 con 0x7f9670005150 2024-04-25T11:20:20.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.385+0000 7f9693800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f968c0afb80 con 0x7f968c0b27b0 2024-04-25T11:20:20.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f968be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c0b5710 0x7f968c1f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f968be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c0b5710 0x7f968c1f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41384/0 (socket says 172.21.15.7:41384) 2024-04-25T11:20:20.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f968be00640 1 -- 172.21.15.7:0/3073585866 learned_addr learned my addr 172.21.15.7:0/3073585866 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:20.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f9691800640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f968c0b27b0 0x7f968c0b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f9690e00640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9670005150 0x7f968c0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f968be00640 1 -- 172.21.15.7:0/3073585866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f968c0b27b0 msgr2=0x7f968c0b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f968be00640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f968c0b27b0 0x7f968c0b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f968be00640 1 -- 172.21.15.7:0/3073585866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9670005150 msgr2=0x7f968c0cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f968be00640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9670005150 0x7f968c0cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f968be00640 1 -- 172.21.15.7:0/3073585866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c0b3100 con 0x7f968c0b5710 2024-04-25T11:20:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f9690e00640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9670005150 0x7f968c0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.386+0000 7f968be00640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c0b5710 0x7f968c1f5300 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f967c007b00 tx=0x7f967c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.387+0000 7f9688c00640 1 -- 172.21.15.7:0/3073585866 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c00ee30 con 0x7f968c0b5710 2024-04-25T11:20:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.387+0000 7f9688c00640 1 -- 172.21.15.7:0/3073585866 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f967c018070 con 0x7f968c0b5710 2024-04-25T11:20:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.387+0000 7f9688c00640 1 -- 172.21.15.7:0/3073585866 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c013680 con 0x7f968c0b5710 2024-04-25T11:20:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.387+0000 7f9693800640 1 -- 172.21.15.7:0/3073585866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968c0cbf60 con 0x7f968c0b5710 2024-04-25T11:20:20.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.387+0000 7f9693800640 1 -- 172.21.15.7:0/3073585866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968c0cc520 con 0x7f968c0b5710 2024-04-25T11:20:20.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.388+0000 7f9693800640 1 -- 172.21.15.7:0/3073585866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f968c06a100 con 0x7f968c0b5710 2024-04-25T11:20:20.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.389+0000 7f9688c00640 1 -- 172.21.15.7:0/3073585866 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f967c013820 con 0x7f968c0b5710 2024-04-25T11:20:20.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.390+0000 7f9688c00640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96680744e0 0x7f96680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.391+0000 7f9688c00640 1 -- 172.21.15.7:0/3073585866 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f967c095b80 con 0x7f968c0b5710 2024-04-25T11:20:20.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.391+0000 7f9690e00640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96680744e0 0x7f96680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.392+0000 7f9690e00640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96680744e0 0x7f96680769f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9674009a60 tx=0x7f9674008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.395+0000 7f9688c00640 1 -- 172.21.15.7:0/3073585866 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f968c0b3100 con 0x7f968c0b5710 2024-04-25T11:20:20.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.602+0000 7f9693800640 1 -- 172.21.15.7:0/3073585866 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f968c06d950 con 0x7f968c0b5710 2024-04-25T11:20:20.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.605+0000 7f9688c00640 1 -- 172.21.15.7:0/3073585866 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16856 (secure 0 0 0) 0x7f967c0600e0 con 0x7f968c0b5710 2024-04-25T11:20:20.603 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:20.603 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:49.277316+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:46.014943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:46.794047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 -- 172.21.15.7:0/3073585866 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96680744e0 msgr2=0x7f96680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96680744e0 0x7f96680769f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9674009a60 tx=0x7f9674008040 comp rx=0 tx=0).stop 2024-04-25T11:20:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 -- 172.21.15.7:0/3073585866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c0b5710 msgr2=0x7f968c1f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c0b5710 0x7f968c1f5300 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f967c007b00 tx=0x7f967c00e530 comp rx=0 tx=0).stop 2024-04-25T11:20:20.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 -- 172.21.15.7:0/3073585866 shutdown_connections 2024-04-25T11:20:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f96680744e0 0x7f96680769f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f968c0b27b0 0x7f968c0b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f968c0b5710 0x7f968c1f5300 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 --2- 172.21.15.7:0/3073585866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9670005150 0x7f968c0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 -- 172.21.15.7:0/3073585866 >> 172.21.15.7:0/3073585866 conn(0x7f968c074200 msgr2=0x7f968c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 -- 172.21.15.7:0/3073585866 shutdown_connections 2024-04-25T11:20:20.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.608+0000 7f9693800640 1 -- 172.21.15.7:0/3073585866 wait complete. 2024-04-25T11:20:20.633 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:19:46.014943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:19:46.794047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:20.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:20.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.928+0000 7f0950a00640 1 Processor -- start 2024-04-25T11:20:20.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.928+0000 7f0950a00640 1 -- start start 2024-04-25T11:20:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.930+0000 7f0950a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f094c1583c0 0x7f094c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.930+0000 7f0950a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f094c0cfb50 0x7f094c0d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.930+0000 7f0950a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f094c0d09c0 0x7f094c06eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.930+0000 7f0950a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f094c15f190 con 0x7f094c0cfb50 2024-04-25T11:20:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.931+0000 7f0950a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f094c0ce570 con 0x7f094c1583c0 2024-04-25T11:20:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.931+0000 7f0950a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f094c0ce850 con 0x7f094c0d09c0 2024-04-25T11:20:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.931+0000 7f0949e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f094c0cfb50 0x7f094c0d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.931+0000 7f0949e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f094c0cfb50 0x7f094c0d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41388/0 (socket says 172.21.15.7:41388) 2024-04-25T11:20:20.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.931+0000 7f0949e00640 1 -- 172.21.15.7:0/359739607 learned_addr learned my addr 172.21.15.7:0/359739607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:20.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.931+0000 7f094a800640 1 --2- 172.21.15.7:0/359739607 >> v2:172.21.15.87:3300/0 conn(0x7f094c1583c0 0x7f094c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.931+0000 7f094b200640 1 --2- 172.21.15.7:0/359739607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f094c0d09c0 0x7f094c06eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.932+0000 7f0949e00640 1 -- 172.21.15.7:0/359739607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f094c0d09c0 msgr2=0x7f094c06eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:20.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.932+0000 7f0949e00640 1 --2- 172.21.15.7:0/359739607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f094c0d09c0 0x7f094c06eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.932+0000 7f0949e00640 1 -- 172.21.15.7:0/359739607 >> v2:172.21.15.87:3300/0 conn(0x7f094c1583c0 msgr2=0x7f094c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:20.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.932+0000 7f0949e00640 1 --2- 172.21.15.7:0/359739607 >> v2:172.21.15.87:3300/0 conn(0x7f094c1583c0 0x7f094c0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.932+0000 7f0949e00640 1 -- 172.21.15.7:0/359739607 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094c0ceb30 con 0x7f094c0cfb50 2024-04-25T11:20:20.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.932+0000 7f094a800640 1 --2- 172.21.15.7:0/359739607 >> v2:172.21.15.87:3300/0 conn(0x7f094c1583c0 0x7f094c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:20.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.932+0000 7f094b200640 1 --2- 172.21.15.7:0/359739607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f094c0d09c0 0x7f094c06eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:20.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.933+0000 7f0949e00640 1 --2- 172.21.15.7:0/359739607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f094c0cfb50 0x7f094c0d0480 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f094c072330 tx=0x7f093802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946d9122f8f340e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.934+0000 7f0949400640 1 -- 172.21.15.7:0/359739607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938037070 con 0x7f094c0cfb50 2024-04-25T11:20:20.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.934+0000 7f0949400640 1 -- 172.21.15.7:0/359739607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0938032440 con 0x7f094c0cfb50 2024-04-25T11:20:20.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.934+0000 7f0949400640 1 -- 172.21.15.7:0/359739607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938032810 con 0x7f094c0cfb50 2024-04-25T11:20:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.934+0000 7f0950a00640 1 -- 172.21.15.7:0/359739607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f094c0cfb50 msgr2=0x7f094c0d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.934+0000 7f0950a00640 1 --2- 172.21.15.7:0/359739607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f094c0cfb50 0x7f094c0d0480 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f094c072330 tx=0x7f093802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.935+0000 7f0950a00640 1 -- 172.21.15.7:0/359739607 shutdown_connections 2024-04-25T11:20:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.935+0000 7f0950a00640 1 --2- 172.21.15.7:0/359739607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f094c0d09c0 0x7f094c06eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.935+0000 7f0950a00640 1 --2- 172.21.15.7:0/359739607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f094c0cfb50 0x7f094c0d0480 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.935+0000 7f0950a00640 1 --2- 172.21.15.7:0/359739607 >> v2:172.21.15.87:3300/0 conn(0x7f094c1583c0 0x7f094c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.935+0000 7f0950a00640 1 -- 172.21.15.7:0/359739607 >> 172.21.15.7:0/359739607 conn(0x7f094c0740f0 msgr2=0x7f094c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.935+0000 7f0950a00640 1 -- 172.21.15.7:0/359739607 shutdown_connections 2024-04-25T11:20:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.935+0000 7f0950a00640 1 -- 172.21.15.7:0/359739607 wait complete. 2024-04-25T11:20:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.937+0000 7f0950a00640 1 Processor -- start 2024-04-25T11:20:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.937+0000 7f0950a00640 1 -- start start 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f0950a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f094c0cfb50 0x7f094c152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f0950a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f094c0d09c0 0x7f094c1f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f0950a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f094c1583c0 0x7f094c155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f0950a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f094c06f3e0 con 0x7f094c0cfb50 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f0950a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f094c0cf0f0 con 0x7f094c1583c0 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f0950a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f094c06f560 con 0x7f094c0d09c0 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f094a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f094c0cfb50 0x7f094c152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f094a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f094c0cfb50 0x7f094c152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41400/0 (socket says 172.21.15.7:41400) 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f094a800640 1 -- 172.21.15.7:0/2308706246 learned_addr learned my addr 172.21.15.7:0/2308706246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f0949e00640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f094c0d09c0 0x7f094c1f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f094a800640 1 -- 172.21.15.7:0/2308706246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f094c0d09c0 msgr2=0x7f094c1f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.939+0000 7f094a800640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f094c0d09c0 0x7f094c1f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.940+0000 7f094a800640 1 -- 172.21.15.7:0/2308706246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f094c1583c0 msgr2=0x7f094c155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.940+0000 7f094b200640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f094c1583c0 0x7f094c155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.940+0000 7f094a800640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f094c1583c0 0x7f094c155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.940+0000 7f094a800640 1 -- 172.21.15.7:0/2308706246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094c1f6270 con 0x7f094c0cfb50 2024-04-25T11:20:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.940+0000 7f094a800640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f094c0cfb50 0x7f094c152390 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f093c00d800 tx=0x7f093c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.940+0000 7f0942a00640 1 -- 172.21.15.7:0/2308706246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c0096a0 con 0x7f094c0cfb50 2024-04-25T11:20:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.940+0000 7f0942a00640 1 -- 172.21.15.7:0/2308706246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f093c019070 con 0x7f094c0cfb50 2024-04-25T11:20:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.940+0000 7f0942a00640 1 -- 172.21.15.7:0/2308706246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c009a80 con 0x7f094c0cfb50 2024-04-25T11:20:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.940+0000 7f0950a00640 1 -- 172.21.15.7:0/2308706246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094c13dfd0 con 0x7f094c0cfb50 2024-04-25T11:20:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.940+0000 7f0950a00640 1 -- 172.21.15.7:0/2308706246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094c13e590 con 0x7f094c0cfb50 2024-04-25T11:20:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.942+0000 7f0950a00640 1 -- 172.21.15.7:0/2308706246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f094c050740 con 0x7f094c0cfb50 2024-04-25T11:20:20.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.943+0000 7f0942a00640 1 -- 172.21.15.7:0/2308706246 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f093c009c20 con 0x7f094c0cfb50 2024-04-25T11:20:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.944+0000 7f0942a00640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f092c0744b0 0x7f092c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.944+0000 7f0942a00640 1 -- 172.21.15.7:0/2308706246 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f093c0957d0 con 0x7f094c0cfb50 2024-04-25T11:20:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.944+0000 7f0949e00640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f092c0744b0 0x7f092c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.945+0000 7f0949e00640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f092c0744b0 0x7f092c0769c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0938034000 tx=0x7f093802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:20.949+0000 7f0942a00640 1 -- 172.21.15.7:0/2308706246 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f093c05fe50 con 0x7f094c0cfb50 2024-04-25T11:20:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.154+0000 7f0950a00640 1 -- 172.21.15.7:0/2308706246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f094c070dd0 con 0x7f094c0cfb50 2024-04-25T11:20:21.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.155+0000 7f0942a00640 1 -- 172.21.15.7:0/2308706246 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7f093c05f750 con 0x7f094c0cfb50 2024-04-25T11:20:21.152 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.158+0000 7f0950a00640 1 -- 172.21.15.7:0/2308706246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f092c0744b0 msgr2=0x7f092c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.158+0000 7f0950a00640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f092c0744b0 0x7f092c0769c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0938034000 tx=0x7f093802e000 comp rx=0 tx=0).stop 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.158+0000 7f0950a00640 1 -- 172.21.15.7:0/2308706246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f094c0cfb50 msgr2=0x7f094c152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.158+0000 7f0950a00640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f094c0cfb50 0x7f094c152390 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f093c00d800 tx=0x7f093c00dcc0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.158+0000 7f0950a00640 1 -- 172.21.15.7:0/2308706246 shutdown_connections 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.158+0000 7f0950a00640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f092c0744b0 0x7f092c0769c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.158+0000 7f0950a00640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f094c1583c0 0x7f094c155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.158+0000 7f0950a00640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f094c0d09c0 0x7f094c1f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.158+0000 7f0950a00640 1 --2- 172.21.15.7:0/2308706246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f094c0cfb50 0x7f094c152390 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.159+0000 7f0950a00640 1 -- 172.21.15.7:0/2308706246 >> 172.21.15.7:0/2308706246 conn(0x7f094c0740f0 msgr2=0x7f094c070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.159+0000 7f0950a00640 1 -- 172.21.15.7:0/2308706246 shutdown_connections 2024-04-25T11:20:21.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.159+0000 7f0950a00640 1 -- 172.21.15.7:0/2308706246 wait complete. 2024-04-25T11:20:21.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:20:21.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.478+0000 7fbc99200640 1 Processor -- start 2024-04-25T11:20:21.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.478+0000 7fbc99200640 1 -- start start 2024-04-25T11:20:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.481+0000 7fbc99200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc94146180 0x7fbc94070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.481+0000 7fbc99200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc941490f0 0x7fbc94149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.481+0000 7fbc99200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc94070860 con 0x7fbc940b01d0 2024-04-25T11:20:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.481+0000 7fbc99200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbc94146590 con 0x7fbc94146180 2024-04-25T11:20:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.481+0000 7fbc99200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbc9406f280 con 0x7fbc941490f0 2024-04-25T11:20:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.481+0000 7fbc93400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc94146180 0x7fbc94070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.481+0000 7fbc93400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc94146180 0x7fbc94070320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41408/0 (socket says 172.21.15.7:41408) 2024-04-25T11:20:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.481+0000 7fbc93400640 1 -- 172.21.15.7:0/1755927924 learned_addr learned my addr 172.21.15.7:0/1755927924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.482+0000 7fbc92a00640 1 --2- 172.21.15.7:0/1755927924 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc941490f0 0x7fbc94149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.482+0000 7fbc93400640 1 -- 172.21.15.7:0/1755927924 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc941490f0 msgr2=0x7fbc94149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.482+0000 7fbc93400640 1 --2- 172.21.15.7:0/1755927924 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc941490f0 0x7fbc94149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.482+0000 7fbc93400640 1 -- 172.21.15.7:0/1755927924 >> v1:172.21.15.87:6789/0 conn(0x7fbc940b01d0 legacy=0x7fbc940b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:21.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.482+0000 7fbc93400640 1 -- 172.21.15.7:0/1755927924 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9406f560 con 0x7fbc94146180 2024-04-25T11:20:21.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.483+0000 7fbc93400640 1 --2- 172.21.15.7:0/1755927924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94146180 0x7fbc94070320 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009870 tx=0x7fbc8402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596e9c3573e9ea4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:21.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.484+0000 7fbc92000640 1 -- 172.21.15.7:0/1755927924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84037070 con 0x7fbc94146180 2024-04-25T11:20:21.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.484+0000 7fbc92000640 1 -- 172.21.15.7:0/1755927924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc84032440 con 0x7fbc94146180 2024-04-25T11:20:21.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.484+0000 7fbc92000640 1 -- 172.21.15.7:0/1755927924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84032810 con 0x7fbc94146180 2024-04-25T11:20:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.485+0000 7fbc99200640 1 -- 172.21.15.7:0/1755927924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94146180 msgr2=0x7fbc94070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.485+0000 7fbc99200640 1 --2- 172.21.15.7:0/1755927924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94146180 0x7fbc94070320 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009870 tx=0x7fbc8402a210 comp rx=0 tx=0).stop 2024-04-25T11:20:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.485+0000 7fbc99200640 1 -- 172.21.15.7:0/1755927924 shutdown_connections 2024-04-25T11:20:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.485+0000 7fbc99200640 1 --2- 172.21.15.7:0/1755927924 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc941490f0 0x7fbc94149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.485+0000 7fbc99200640 1 --2- 172.21.15.7:0/1755927924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94146180 0x7fbc94070320 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.485+0000 7fbc99200640 1 -- 172.21.15.7:0/1755927924 >> 172.21.15.7:0/1755927924 conn(0x7fbc940740f0 msgr2=0x7fbc9406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.485+0000 7fbc99200640 1 -- 172.21.15.7:0/1755927924 shutdown_connections 2024-04-25T11:20:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.485+0000 7fbc99200640 1 -- 172.21.15.7:0/1755927924 wait complete. 2024-04-25T11:20:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.488+0000 7fbc99200640 1 Processor -- start 2024-04-25T11:20:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.488+0000 7fbc99200640 1 -- start start 2024-04-25T11:20:21.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.490+0000 7fbc99200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc940b01d0 0x7fbc941fa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.490+0000 7fbc99200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc94146180 0x7fbc9415f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.490+0000 7fbc99200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc941490f0 0x7fbc94147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.490+0000 7fbc99200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9406fe00 con 0x7fbc941490f0 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.490+0000 7fbc99200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9406fb20 con 0x7fbc940b01d0 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.490+0000 7fbc99200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbc9406eea0 con 0x7fbc94146180 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.490+0000 7fbc93e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc941490f0 0x7fbc94147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.490+0000 7fbc93e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc941490f0 0x7fbc94147d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41422/0 (socket says 172.21.15.7:41422) 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.490+0000 7fbc93e00640 1 -- 172.21.15.7:0/497451456 learned_addr learned my addr 172.21.15.7:0/497451456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.490+0000 7fbc93400640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc940b01d0 0x7fbc941fa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.490+0000 7fbc92a00640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc94146180 0x7fbc9415f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.491+0000 7fbc93e00640 1 -- 172.21.15.7:0/497451456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc94146180 msgr2=0x7fbc9415f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.491+0000 7fbc93e00640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc94146180 0x7fbc9415f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.491+0000 7fbc93e00640 1 -- 172.21.15.7:0/497451456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc940b01d0 msgr2=0x7fbc941fa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.491+0000 7fbc93e00640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc940b01d0 0x7fbc941fa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.491+0000 7fbc93e00640 1 -- 172.21.15.7:0/497451456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc941fa940 con 0x7fbc941490f0 2024-04-25T11:20:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.491+0000 7fbc93e00640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc941490f0 0x7fbc94147d00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00be60 tx=0x7fbc8c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.491+0000 7fbc8b400640 1 -- 172.21.15.7:0/497451456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c019070 con 0x7fbc941490f0 2024-04-25T11:20:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.491+0000 7fbc8b400640 1 -- 172.21.15.7:0/497451456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc8c014440 con 0x7fbc941490f0 2024-04-25T11:20:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.491+0000 7fbc99200640 1 -- 172.21.15.7:0/497451456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9415fc90 con 0x7fbc941490f0 2024-04-25T11:20:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.491+0000 7fbc8b400640 1 -- 172.21.15.7:0/497451456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c014870 con 0x7fbc941490f0 2024-04-25T11:20:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.492+0000 7fbc99200640 1 -- 172.21.15.7:0/497451456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc941f6aa0 con 0x7fbc941490f0 2024-04-25T11:20:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.493+0000 7fbc99200640 1 -- 172.21.15.7:0/497451456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc9406db50 con 0x7fbc941490f0 2024-04-25T11:20:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.494+0000 7fbc8b400640 1 -- 172.21.15.7:0/497451456 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbc8c01c070 con 0x7fbc941490f0 2024-04-25T11:20:21.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.496+0000 7fbc8b400640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc740744b0 0x7fbc740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:21.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.496+0000 7fbc8b400640 1 -- 172.21.15.7:0/497451456 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fbc8c0953e0 con 0x7fbc941490f0 2024-04-25T11:20:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.496+0000 7fbc93400640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc740744b0 0x7fbc740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.497+0000 7fbc93400640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc740744b0 0x7fbc740769c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbc84034000 tx=0x7fbc8402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.501+0000 7fbc8b400640 1 -- 172.21.15.7:0/497451456 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc9415fc90 con 0x7fbc941490f0 2024-04-25T11:20:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.704+0000 7fbc99200640 1 -- 172.21.15.7:0/497451456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbc9406d870 con 0x7fbc941490f0 2024-04-25T11:20:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.706+0000 7fbc8b400640 1 -- 172.21.15.7:0/497451456 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7fbc941fa940 con 0x7fbc941490f0 2024-04-25T11:20:21.703 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 -- 172.21.15.7:0/497451456 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc740744b0 msgr2=0x7fbc740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc740744b0 0x7fbc740769c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fbc84034000 tx=0x7fbc8402e000 comp rx=0 tx=0).stop 2024-04-25T11:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 -- 172.21.15.7:0/497451456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc941490f0 msgr2=0x7fbc94147d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc941490f0 0x7fbc94147d00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00be60 tx=0x7fbc8c00bf50 comp rx=0 tx=0).stop 2024-04-25T11:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 -- 172.21.15.7:0/497451456 shutdown_connections 2024-04-25T11:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc740744b0 0x7fbc740769c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc941490f0 0x7fbc94147d00 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc94146180 0x7fbc9415f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 --2- 172.21.15.7:0/497451456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc940b01d0 0x7fbc941fa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 -- 172.21.15.7:0/497451456 >> 172.21.15.7:0/497451456 conn(0x7fbc940740f0 msgr2=0x7fbc940b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 -- 172.21.15.7:0/497451456 shutdown_connections 2024-04-25T11:20:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:21.709+0000 7fbc99200640 1 -- 172.21.15.7:0/497451456 wait complete. 2024-04-25T11:20:21.732 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:20:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.018+0000 7f7e77e00640 1 Processor -- start 2024-04-25T11:20:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.018+0000 7f7e77e00640 1 -- start start 2024-04-25T11:20:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.020+0000 7f7e77e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e780d1150 0x7f7e780bb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.021+0000 7f7e77e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e7806e710 0x7f7e7815f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.021+0000 7f7e77e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e780b8f30 con 0x7f7e780d4400 2024-04-25T11:20:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.021+0000 7f7e77e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7e780b9260 con 0x7f7e780d1150 2024-04-25T11:20:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.021+0000 7f7e77e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7e780d1560 con 0x7f7e7806e710 2024-04-25T11:20:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.021+0000 7f7e76000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7e780d4400 0x7f7e7806e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46692/0 (socket says 172.21.15.7:46692) 2024-04-25T11:20:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.021+0000 7f7e76000640 1 -- 172.21.15.7:0/119891216 learned_addr learned my addr 172.21.15.7:0/119891216 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.021+0000 7f7e76a00640 1 --2- 172.21.15.7:0/119891216 >> v2:172.21.15.7:3300/0 conn(0x7f7e780d1150 0x7f7e780bb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.021+0000 7f7e77400640 1 --2- 172.21.15.7:0/119891216 >> v2:172.21.15.87:3300/0 conn(0x7f7e7806e710 0x7f7e7815f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.022+0000 7f7e75600640 1 -- 172.21.15.7:0/119891216 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3774257958 0 0) 0x7f7e780b8f30 con 0x7f7e780d4400 2024-04-25T11:20:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.022+0000 7f7e75600640 1 -- 172.21.15.7:0/119891216 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e5c003520 con 0x7f7e780d4400 2024-04-25T11:20:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.022+0000 7f7e76a00640 1 -- 172.21.15.7:0/119891216 >> v2:172.21.15.87:3300/0 conn(0x7f7e7806e710 msgr2=0x7f7e7815f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.022+0000 7f7e76a00640 1 --2- 172.21.15.7:0/119891216 >> v2:172.21.15.87:3300/0 conn(0x7f7e7806e710 0x7f7e7815f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.022+0000 7f7e76a00640 1 -- 172.21.15.7:0/119891216 >> v1:172.21.15.7:6789/0 conn(0x7f7e780d4400 legacy=0x7f7e7806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.022+0000 7f7e76a00640 1 -- 172.21.15.7:0/119891216 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e780d4be0 con 0x7f7e780d1150 2024-04-25T11:20:22.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.023+0000 7f7e76a00640 1 --2- 172.21.15.7:0/119891216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e780d1150 0x7f7e780bb2f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7e780d32b0 tx=0x7f7e6802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba809ed7ba793eef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:22.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.023+0000 7f7e75600640 1 -- 172.21.15.7:0/119891216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e68037070 con 0x7f7e780d1150 2024-04-25T11:20:22.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.024+0000 7f7e75600640 1 -- 172.21.15.7:0/119891216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e68032440 con 0x7f7e780d1150 2024-04-25T11:20:22.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.024+0000 7f7e75600640 1 -- 172.21.15.7:0/119891216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6802ae60 con 0x7f7e780d1150 2024-04-25T11:20:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.024+0000 7f7e77e00640 1 -- 172.21.15.7:0/119891216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e780d1150 msgr2=0x7f7e780bb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.024+0000 7f7e77e00640 1 --2- 172.21.15.7:0/119891216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e780d1150 0x7f7e780bb2f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7e780d32b0 tx=0x7f7e6802a110 comp rx=0 tx=0).stop 2024-04-25T11:20:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.025+0000 7f7e77e00640 1 -- 172.21.15.7:0/119891216 shutdown_connections 2024-04-25T11:20:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.025+0000 7f7e77e00640 1 --2- 172.21.15.7:0/119891216 >> v2:172.21.15.87:3300/0 conn(0x7f7e7806e710 0x7f7e7815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.025+0000 7f7e77e00640 1 --2- 172.21.15.7:0/119891216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e780d1150 0x7f7e780bb2f0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.025+0000 7f7e77e00640 1 -- 172.21.15.7:0/119891216 >> 172.21.15.7:0/119891216 conn(0x7f7e7806d170 msgr2=0x7f7e780d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.025+0000 7f7e77e00640 1 -- 172.21.15.7:0/119891216 shutdown_connections 2024-04-25T11:20:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.025+0000 7f7e77e00640 1 -- 172.21.15.7:0/119891216 wait complete. 2024-04-25T11:20:22.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.027+0000 7f7e77e00640 1 Processor -- start 2024-04-25T11:20:22.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.027+0000 7f7e77e00640 1 -- start start 2024-04-25T11:20:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e77e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e780d1150 0x7f7e7815e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e77e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e780d4400 0x7f7e7815f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e77e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e780cf680 0x7f7e7815fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e77e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e780d4810 con 0x7f7e780cf680 2024-04-25T11:20:22.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e77e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7e780d51a0 con 0x7f7e780d4400 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e77e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7e780d4990 con 0x7f7e780d1150 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e77400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e780cf680 0x7f7e7815fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e77400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e780cf680 0x7f7e7815fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41438/0 (socket says 172.21.15.7:41438) 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e77400640 1 -- 172.21.15.7:0/822107603 learned_addr learned my addr 172.21.15.7:0/822107603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e76000640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e780d4400 0x7f7e7815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e77400640 1 -- 172.21.15.7:0/822107603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e780d1150 msgr2=0x7f7e7815e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.029+0000 7f7e76a00640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e780d1150 0x7f7e7815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.030+0000 7f7e77400640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e780d1150 0x7f7e7815e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.030+0000 7f7e77400640 1 -- 172.21.15.7:0/822107603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e780d4400 msgr2=0x7f7e7815f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.030+0000 7f7e77400640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e780d4400 0x7f7e7815f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.030+0000 7f7e77400640 1 -- 172.21.15.7:0/822107603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e780cca60 con 0x7f7e780cf680 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.030+0000 7f7e76000640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e780d4400 0x7f7e7815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.030+0000 7f7e77400640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e780cf680 0x7f7e7815fcd0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7e70007e60 tx=0x7f7e7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.030+0000 7f7e6ea00640 1 -- 172.21.15.7:0/822107603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70014420 con 0x7f7e780cf680 2024-04-25T11:20:22.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.030+0000 7f7e6ea00640 1 -- 172.21.15.7:0/822107603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e70019070 con 0x7f7e780cf680 2024-04-25T11:20:22.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.030+0000 7f7e77e00640 1 -- 172.21.15.7:0/822107603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e780ccd20 con 0x7f7e780cf680 2024-04-25T11:20:22.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.030+0000 7f7e6ea00640 1 -- 172.21.15.7:0/822107603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70014820 con 0x7f7e780cf680 2024-04-25T11:20:22.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.030+0000 7f7e77e00640 1 -- 172.21.15.7:0/822107603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e780cd2e0 con 0x7f7e780cf680 2024-04-25T11:20:22.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.031+0000 7f7e77e00640 1 -- 172.21.15.7:0/822107603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e7806ece0 con 0x7f7e780cf680 2024-04-25T11:20:22.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.033+0000 7f7e6ea00640 1 -- 172.21.15.7:0/822107603 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e700149c0 con 0x7f7e780cf680 2024-04-25T11:20:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.034+0000 7f7e6ea00640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e580744b0 0x7f7e580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.034+0000 7f7e6ea00640 1 -- 172.21.15.7:0/822107603 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f7e70096b70 con 0x7f7e780cf680 2024-04-25T11:20:22.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.034+0000 7f7e76a00640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e580744b0 0x7f7e580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.035+0000 7f7e76a00640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e580744b0 0x7f7e580769c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7e68034000 tx=0x7f7e6802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:22.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.038+0000 7f7e6ea00640 1 -- 172.21.15.7:0/822107603 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e7806ece0 con 0x7f7e780cf680 2024-04-25T11:20:22.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.247+0000 7f7e77e00640 1 -- 172.21.15.7:0/822107603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7e7815dc30 con 0x7f7e780cf680 2024-04-25T11:20:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.248+0000 7f7e6ea00640 1 -- 172.21.15.7:0/822107603 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7f7e70061150 con 0x7f7e780cf680 2024-04-25T11:20:22.245 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:20:22.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 -- 172.21.15.7:0/822107603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e580744b0 msgr2=0x7f7e580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e580744b0 0x7f7e580769c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7e68034000 tx=0x7f7e6802e000 comp rx=0 tx=0).stop 2024-04-25T11:20:22.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 -- 172.21.15.7:0/822107603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e780cf680 msgr2=0x7f7e7815fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e780cf680 0x7f7e7815fcd0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7e70007e60 tx=0x7f7e7000e4b0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 -- 172.21.15.7:0/822107603 shutdown_connections 2024-04-25T11:20:22.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e580744b0 0x7f7e580769c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e780cf680 0x7f7e7815fcd0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e780d4400 0x7f7e7815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 --2- 172.21.15.7:0/822107603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e780d1150 0x7f7e7815e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 -- 172.21.15.7:0/822107603 >> 172.21.15.7:0/822107603 conn(0x7f7e7806d170 msgr2=0x7f7e780b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 -- 172.21.15.7:0/822107603 shutdown_connections 2024-04-25T11:20:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.251+0000 7f7e77e00640 1 -- 172.21.15.7:0/822107603 wait complete. 2024-04-25T11:20:22.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:22.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.564+0000 7fd8a7200640 1 Processor -- start 2024-04-25T11:20:22.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.564+0000 7fd8a7200640 1 -- start start 2024-04-25T11:20:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.567+0000 7fd8a7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8a8164690 0x7fd8a8070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.567+0000 7fd8a7200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8a81641e0 0x7fd8a80b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.567+0000 7fd8a7200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8a80b5080 con 0x7fd8a8156cd0 2024-04-25T11:20:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.567+0000 7fd8a7200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd8a80b5360 con 0x7fd8a81641e0 2024-04-25T11:20:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.567+0000 7fd8a7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a80b5640 con 0x7fd8a8164690 2024-04-25T11:20:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.567+0000 7fd8a6800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8a81641e0 0x7fd8a80b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.567+0000 7fd8a6800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd8a81641e0 0x7fd8a80b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41448/0 (socket says 172.21.15.7:41448) 2024-04-25T11:20:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.567+0000 7fd8a6800640 1 -- 172.21.15.7:0/1208676584 learned_addr learned my addr 172.21.15.7:0/1208676584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:22.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.567+0000 7fd8a5e00640 1 --2- 172.21.15.7:0/1208676584 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8a8164690 0x7fd8a8070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.568+0000 7fd8a6800640 1 -- 172.21.15.7:0/1208676584 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8a8164690 msgr2=0x7fd8a8070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:22.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.568+0000 7fd8a6800640 1 --2- 172.21.15.7:0/1208676584 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8a8164690 0x7fd8a8070a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.568+0000 7fd8a6800640 1 -- 172.21.15.7:0/1208676584 >> v1:172.21.15.87:6789/0 conn(0x7fd8a8156cd0 legacy=0x7fd8a8163ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:22.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.568+0000 7fd8a6800640 1 -- 172.21.15.7:0/1208676584 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a80b5920 con 0x7fd8a81641e0 2024-04-25T11:20:22.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.569+0000 7fd8a6800640 1 --2- 172.21.15.7:0/1208676584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a81641e0 0x7fd8a80b0210 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009e50 tx=0x7fd8a002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d85760f9402ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:22.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.569+0000 7fd8a4a00640 1 -- 172.21.15.7:0/1208676584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0036070 con 0x7fd8a81641e0 2024-04-25T11:20:22.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.569+0000 7fd8a4a00640 1 -- 172.21.15.7:0/1208676584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a0031600 con 0x7fd8a81641e0 2024-04-25T11:20:22.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.569+0000 7fd8a4a00640 1 -- 172.21.15.7:0/1208676584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00319d0 con 0x7fd8a81641e0 2024-04-25T11:20:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.570+0000 7fd8a7200640 1 -- 172.21.15.7:0/1208676584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a81641e0 msgr2=0x7fd8a80b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.570+0000 7fd8a7200640 1 --2- 172.21.15.7:0/1208676584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a81641e0 0x7fd8a80b0210 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009e50 tx=0x7fd8a002a630 comp rx=0 tx=0).stop 2024-04-25T11:20:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.570+0000 7fd8a7200640 1 -- 172.21.15.7:0/1208676584 shutdown_connections 2024-04-25T11:20:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.570+0000 7fd8a7200640 1 --2- 172.21.15.7:0/1208676584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a81641e0 0x7fd8a80b0210 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.570+0000 7fd8a7200640 1 --2- 172.21.15.7:0/1208676584 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8a8164690 0x7fd8a8070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.570+0000 7fd8a7200640 1 -- 172.21.15.7:0/1208676584 >> 172.21.15.7:0/1208676584 conn(0x7fd8a80b3820 msgr2=0x7fd8a80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:22.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.570+0000 7fd8a7200640 1 -- 172.21.15.7:0/1208676584 shutdown_connections 2024-04-25T11:20:22.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.570+0000 7fd8a7200640 1 -- 172.21.15.7:0/1208676584 wait complete. 2024-04-25T11:20:22.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.572+0000 7fd8a7200640 1 Processor -- start 2024-04-25T11:20:22.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.573+0000 7fd8a7200640 1 -- start start 2024-04-25T11:20:22.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.574+0000 7fd8a7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8a8156cd0 0x7fd8a8146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.574+0000 7fd8a7200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8a8164690 0x7fd8a8159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.574+0000 7fd8a7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a8159aa0 0x7fd8a80b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.574+0000 7fd8a7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a81570e0 con 0x7fd8a8159aa0 2024-04-25T11:20:22.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.574+0000 7fd8a7200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a8070f60 con 0x7fd8a8164690 2024-04-25T11:20:22.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.574+0000 7fd8a7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a8157260 con 0x7fd8a8156cd0 2024-04-25T11:20:22.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.575+0000 7fd8a6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a8159aa0 0x7fd8a80b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.575+0000 7fd8a6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a8159aa0 0x7fd8a80b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41464/0 (socket says 172.21.15.7:41464) 2024-04-25T11:20:22.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.575+0000 7fd8a6800640 1 -- 172.21.15.7:0/368726169 learned_addr learned my addr 172.21.15.7:0/368726169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:22.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.575+0000 7fd8a5e00640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8a8156cd0 0x7fd8a8146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.575+0000 7fd8a5400640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8a8164690 0x7fd8a8159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.575+0000 7fd8a6800640 1 -- 172.21.15.7:0/368726169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8a8156cd0 msgr2=0x7fd8a8146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.575+0000 7fd8a6800640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8a8156cd0 0x7fd8a8146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.575+0000 7fd8a6800640 1 -- 172.21.15.7:0/368726169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8a8164690 msgr2=0x7fd8a8159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.575+0000 7fd8a6800640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8a8164690 0x7fd8a8159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.575+0000 7fd8a6800640 1 -- 172.21.15.7:0/368726169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a815a320 con 0x7fd8a8159aa0 2024-04-25T11:20:22.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.575+0000 7fd8a6800640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a8159aa0 0x7fd8a80b2610 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009f70 tx=0x7fd8a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:22.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.576+0000 7fd89e000640 1 -- 172.21.15.7:0/368726169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0036050 con 0x7fd8a8159aa0 2024-04-25T11:20:22.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.576+0000 7fd89e000640 1 -- 172.21.15.7:0/368726169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a0042070 con 0x7fd8a8159aa0 2024-04-25T11:20:22.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.576+0000 7fd89e000640 1 -- 172.21.15.7:0/368726169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0009510 con 0x7fd8a8159aa0 2024-04-25T11:20:22.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.576+0000 7fd8a7200640 1 -- 172.21.15.7:0/368726169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a81f9870 con 0x7fd8a8159aa0 2024-04-25T11:20:22.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.576+0000 7fd8a7200640 1 -- 172.21.15.7:0/368726169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a81f9e30 con 0x7fd8a8159aa0 2024-04-25T11:20:22.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.577+0000 7fd8a7200640 1 -- 172.21.15.7:0/368726169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a815a090 con 0x7fd8a8159aa0 2024-04-25T11:20:22.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.578+0000 7fd89e000640 1 -- 172.21.15.7:0/368726169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd8a0007bd0 con 0x7fd8a8159aa0 2024-04-25T11:20:22.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.579+0000 7fd89e000640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8880744b0 0x7fd8880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:22.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.579+0000 7fd89e000640 1 -- 172.21.15.7:0/368726169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fd8a00b6310 con 0x7fd8a8159aa0 2024-04-25T11:20:22.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.580+0000 7fd8a5e00640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8880744b0 0x7fd8880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:22.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.580+0000 7fd8a5e00640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8880744b0 0x7fd8880769c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd898006fd0 tx=0x7fd898008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:22.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.584+0000 7fd89e000640 1 -- 172.21.15.7:0/368726169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd8a0080870 con 0x7fd8a8159aa0 2024-04-25T11:20:22.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.832+0000 7fd8a7200640 1 -- 172.21.15.7:0/368726169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8a8050740 con 0x7fd8a8159aa0 2024-04-25T11:20:22.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.834+0000 7fd89e000640 1 -- 172.21.15.7:0/368726169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5175 (secure 0 0 0) 0x7fd8a0080170 con 0x7fd8a8159aa0 2024-04-25T11:20:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:22.831 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/326382713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":326382713},{"type":"v1","addr":"172.21.15.7:6837","nonce":326382713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2007810270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2007810270},{"type":"v1","addr":"172.21.15.87:6835","nonce":2007810270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1753656512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1753656512},{"type":"v1","addr":"172.21.15.7:6836","nonce":1753656512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:47.807534+0000","modified":"2024-04-25T11:20:20.325877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":11,"addr":"172.21.15.87:6834/1924056839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1924056839},{"type":"v1","addr":"172.21.15.87:6834","nonce":1924056839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8007,"qdb_cluster":[8007]},"id":8}]} 2024-04-25T11:20:22.831 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 80 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.837+0000 7fd8a7200640 1 -- 172.21.15.7:0/368726169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8880744b0 msgr2=0x7fd8880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.837+0000 7fd8a7200640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8880744b0 0x7fd8880769c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd898006fd0 tx=0x7fd898008040 comp rx=0 tx=0).stop 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.837+0000 7fd8a7200640 1 -- 172.21.15.7:0/368726169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a8159aa0 msgr2=0x7fd8a80b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.837+0000 7fd8a7200640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a8159aa0 0x7fd8a80b2610 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009f70 tx=0x7fd8a0004a40 comp rx=0 tx=0).stop 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.837+0000 7fd8a7200640 1 -- 172.21.15.7:0/368726169 shutdown_connections 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.837+0000 7fd8a7200640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd8880744b0 0x7fd8880769c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.837+0000 7fd8a7200640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a8159aa0 0x7fd8a80b2610 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.837+0000 7fd8a7200640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8a8164690 0x7fd8a8159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.837+0000 7fd8a7200640 1 --2- 172.21.15.7:0/368726169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8a8156cd0 0x7fd8a8146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.837+0000 7fd8a7200640 1 -- 172.21.15.7:0/368726169 >> 172.21.15.7:0/368726169 conn(0x7fd8a80b3820 msgr2=0x7fd8a80b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.838+0000 7fd8a7200640 1 -- 172.21.15.7:0/368726169 shutdown_connections 2024-04-25T11:20:22.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:22.838+0000 7fd8a7200640 1 -- 172.21.15.7:0/368726169 wait complete. 2024-04-25T11:20:22.863 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:23.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.150+0000 7ff592800640 1 Processor -- start 2024-04-25T11:20:23.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.150+0000 7ff592800640 1 -- start start 2024-04-25T11:20:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.152+0000 7ff592800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff58c0afb30 0x7ff58c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.153+0000 7ff592800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff58c0cebb0 con 0x7ff58c0b5220 2024-04-25T11:20:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.153+0000 7ff592800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff58c0b64b0 con 0x7ff58c0aed40 2024-04-25T11:20:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.153+0000 7ff592800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff58c15e1f0 con 0x7ff58c0afb30 2024-04-25T11:20:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.153+0000 7ff590a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff58c0b5220 0x7ff58c0af420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46696/0 (socket says 172.21.15.7:46696) 2024-04-25T11:20:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.153+0000 7ff590a00640 1 -- 172.21.15.7:0/3024908570 learned_addr learned my addr 172.21.15.7:0/3024908570 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.153+0000 7ff591e00640 1 --2- 172.21.15.7:0/3024908570 >> v2:172.21.15.87:3300/0 conn(0x7ff58c0afb30 0x7ff58c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.153+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1598247260 0 0) 0x7ff578001110 con 0x7ff58c0b5220 2024-04-25T11:20:23.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff570003520 con 0x7ff58c0b5220 2024-04-25T11:20:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2670001214 0 0) 0x7ff570003520 con 0x7ff58c0b5220 2024-04-25T11:20:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 >> v2:172.21.15.87:3300/0 conn(0x7ff58c0afb30 msgr2=0x7ff58c15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 --2- 172.21.15.7:0/3024908570 >> v2:172.21.15.87:3300/0 conn(0x7ff58c0afb30 0x7ff58c15dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 >> v1:172.21.15.87:6789/0 conn(0x7ff58c0aed40 legacy=0x7ff58c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c06c610 con 0x7ff58c0b5220 2024-04-25T11:20:23.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7ff578001840 con 0x7ff58c0b5220 2024-04-25T11:20:23.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 >> v1:172.21.15.7:6789/0 conn(0x7ff58c0b5220 legacy=0x7ff58c0af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 --2- 172.21.15.7:0/3024908570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff570003a00 0x7ff570003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 --2- 172.21.15.7:0/3024908570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff570004800 0x7ff570004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 --2- 172.21.15.7:0/3024908570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 0x7ff57000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c15e1f0 con 0x7ff570005270 2024-04-25T11:20:23.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff57000c190 con 0x7ff570003a00 2024-04-25T11:20:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff57000c390 con 0x7ff570004800 2024-04-25T11:20:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.154+0000 7ff591e00640 1 --2- 172.21.15.7:0/3024908570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 0x7ff57000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.155+0000 7ff591400640 1 --2- 172.21.15.7:0/3024908570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff570003a00 0x7ff570003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.155+0000 7ff591e00640 1 -- 172.21.15.7:0/3024908570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff570004800 msgr2=0x7ff570004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.155+0000 7ff591e00640 1 --2- 172.21.15.7:0/3024908570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff570004800 0x7ff570004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.155+0000 7ff591e00640 1 -- 172.21.15.7:0/3024908570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff570003a00 msgr2=0x7ff570003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.155+0000 7ff591e00640 1 --2- 172.21.15.7:0/3024908570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff570003a00 0x7ff570003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.155+0000 7ff591e00640 1 -- 172.21.15.7:0/3024908570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff570003520 con 0x7ff570005270 2024-04-25T11:20:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.156+0000 7ff591e00640 1 -- 172.21.15.7:0/3024908570 reap_dead start 2024-04-25T11:20:23.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.156+0000 7ff591e00640 1 --2- 172.21.15.7:0/3024908570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 0x7ff57000b9c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff584009ac0 tx=0x7ff570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b164480f97df8621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.156+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584009690 con 0x7ff570005270 2024-04-25T11:20:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.156+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff584004020 con 0x7ff570005270 2024-04-25T11:20:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.156+0000 7ff58be00640 1 -- 172.21.15.7:0/3024908570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff584004410 con 0x7ff570005270 2024-04-25T11:20:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.157+0000 7ff592800640 1 -- 172.21.15.7:0/3024908570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 msgr2=0x7ff57000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.157+0000 7ff592800640 1 --2- 172.21.15.7:0/3024908570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 0x7ff57000b9c0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ff584009ac0 tx=0x7ff570005830 comp rx=0 tx=0).stop 2024-04-25T11:20:23.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.157+0000 7ff592800640 1 -- 172.21.15.7:0/3024908570 shutdown_connections 2024-04-25T11:20:23.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.157+0000 7ff592800640 1 --2- 172.21.15.7:0/3024908570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 0x7ff57000b9c0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.157+0000 7ff592800640 1 -- 172.21.15.7:0/3024908570 >> 172.21.15.7:0/3024908570 conn(0x7ff58c0b3820 msgr2=0x7ff58c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:23.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.157+0000 7ff592800640 1 -- 172.21.15.7:0/3024908570 shutdown_connections 2024-04-25T11:20:23.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.157+0000 7ff592800640 1 -- 172.21.15.7:0/3024908570 wait complete. 2024-04-25T11:20:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.160+0000 7ff592800640 1 Processor -- start 2024-04-25T11:20:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.160+0000 7ff592800640 1 -- start start 2024-04-25T11:20:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.161+0000 7ff592800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 0x7ff58c14f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.161+0000 7ff592800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff58c0702c0 0x7ff58c15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.161+0000 7ff592800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff58c0706d0 0x7ff58c15c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.161+0000 7ff592800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c06ceb0 con 0x7ff570005270 2024-04-25T11:20:23.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.161+0000 7ff592800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c06cbd0 con 0x7ff58c0706d0 2024-04-25T11:20:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.161+0000 7ff592800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c0b6110 con 0x7ff58c0702c0 2024-04-25T11:20:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.162+0000 7ff591400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 0x7ff58c14f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.162+0000 7ff591400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 0x7ff58c14f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41484/0 (socket says 172.21.15.7:41484) 2024-04-25T11:20:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.162+0000 7ff591400640 1 -- 172.21.15.7:0/3575546513 learned_addr learned my addr 172.21.15.7:0/3575546513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.162+0000 7ff591400640 1 -- 172.21.15.7:0/3575546513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff58c0702c0 msgr2=0x7ff58c15c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.162+0000 7ff591400640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff58c0702c0 0x7ff58c15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.162+0000 7ff591400640 1 -- 172.21.15.7:0/3575546513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff58c0706d0 msgr2=0x7ff58c15c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.162+0000 7ff591400640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff58c0706d0 0x7ff58c15c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.162+0000 7ff591400640 1 -- 172.21.15.7:0/3575546513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c15e650 con 0x7ff570005270 2024-04-25T11:20:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.162+0000 7ff591400640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 0x7ff58c14f0a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00d860 tx=0x7ff57c00dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:23.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.163+0000 7ff589600640 1 -- 172.21.15.7:0/3575546513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c003da0 con 0x7ff570005270 2024-04-25T11:20:23.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.163+0000 7ff589600640 1 -- 172.21.15.7:0/3575546513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57c017070 con 0x7ff570005270 2024-04-25T11:20:23.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.163+0000 7ff589600640 1 -- 172.21.15.7:0/3575546513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00b8f0 con 0x7ff570005270 2024-04-25T11:20:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.163+0000 7ff592800640 1 -- 172.21.15.7:0/3575546513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58c15e930 con 0x7ff570005270 2024-04-25T11:20:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.163+0000 7ff592800640 1 -- 172.21.15.7:0/3575546513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff58c15eef0 con 0x7ff570005270 2024-04-25T11:20:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.164+0000 7ff592800640 1 -- 172.21.15.7:0/3575546513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58c050740 con 0x7ff570005270 2024-04-25T11:20:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.165+0000 7ff589600640 1 -- 172.21.15.7:0/3575546513 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff57c00ba90 con 0x7ff570005270 2024-04-25T11:20:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.166+0000 7ff589600640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff56c0744b0 0x7ff56c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.166+0000 7ff589600640 1 -- 172.21.15.7:0/3575546513 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7ff57c095560 con 0x7ff570005270 2024-04-25T11:20:23.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.167+0000 7ff590a00640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff56c0744b0 0x7ff56c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.167+0000 7ff590a00640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff56c0744b0 0x7ff56c0769c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff5780013b0 tx=0x7ff578001260 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:23.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.171+0000 7ff589600640 1 -- 172.21.15.7:0/3575546513 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff57c05fb60 con 0x7ff570005270 2024-04-25T11:20:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.380+0000 7ff592800640 1 -- 172.21.15.7:0/3575546513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff58c069960 con 0x7ff570005270 2024-04-25T11:20:23.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.381+0000 7ff589600640 1 -- 172.21.15.7:0/3575546513 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16856 (secure 0 0 0) 0x7ff57c05f460 con 0x7ff570005270 2024-04-25T11:20:23.379 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:23.380 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:49.277316+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:46.014943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:46.794047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:23.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.384+0000 7ff592800640 1 -- 172.21.15.7:0/3575546513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff56c0744b0 msgr2=0x7ff56c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.384+0000 7ff592800640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff56c0744b0 0x7ff56c0769c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff5780013b0 tx=0x7ff578001260 comp rx=0 tx=0).stop 2024-04-25T11:20:23.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.384+0000 7ff592800640 1 -- 172.21.15.7:0/3575546513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 msgr2=0x7ff58c14f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.384+0000 7ff592800640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 0x7ff58c14f0a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00d860 tx=0x7ff57c00dd20 comp rx=0 tx=0).stop 2024-04-25T11:20:23.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.384+0000 7ff592800640 1 -- 172.21.15.7:0/3575546513 shutdown_connections 2024-04-25T11:20:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.384+0000 7ff592800640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff56c0744b0 0x7ff56c0769c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.384+0000 7ff592800640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff58c0706d0 0x7ff58c15c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.384+0000 7ff592800640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff58c0702c0 0x7ff58c15c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.384+0000 7ff592800640 1 --2- 172.21.15.7:0/3575546513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff570005270 0x7ff58c14f0a0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.384+0000 7ff592800640 1 -- 172.21.15.7:0/3575546513 >> 172.21.15.7:0/3575546513 conn(0x7ff58c0b3820 msgr2=0x7ff58c0b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:23.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.384+0000 7ff592800640 1 -- 172.21.15.7:0/3575546513 shutdown_connections 2024-04-25T11:20:23.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.385+0000 7ff592800640 1 -- 172.21.15.7:0/3575546513 wait complete. 2024-04-25T11:20:23.407 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:20:23.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.693+0000 7f2fcd400640 1 Processor -- start 2024-04-25T11:20:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.693+0000 7f2fcd400640 1 -- start start 2024-04-25T11:20:23.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.695+0000 7f2fcd400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2fc815ef90 0x7f2fc809f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.695+0000 7f2fcd400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2fc815bc40 0x7f2fc80aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.695+0000 7f2fcd400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc815f4e0 con 0x7f2fc80af370 2024-04-25T11:20:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.695+0000 7f2fcd400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc8071190 con 0x7f2fc815ef90 2024-04-25T11:20:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.695+0000 7f2fcd400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc80af780 con 0x7f2fc815bc40 2024-04-25T11:20:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.695+0000 7f2fc7200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2fc80af370 0x7f2fc8161b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46710/0 (socket says 172.21.15.7:46710) 2024-04-25T11:20:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.695+0000 7f2fc7200640 1 -- 172.21.15.7:0/2894139105 learned_addr learned my addr 172.21.15.7:0/2894139105 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.695+0000 7f2fc6800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2fc815ef90 0x7f2fc809f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.696+0000 7f2fbfe00640 1 --2- 172.21.15.7:0/2894139105 >> v2:172.21.15.87:3300/0 conn(0x7f2fc815bc40 0x7f2fc80aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.696+0000 7f2fc6800640 1 -- 172.21.15.7:0/2894139105 >> v2:172.21.15.87:3300/0 conn(0x7f2fc815bc40 msgr2=0x7f2fc80aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:23.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.696+0000 7f2fc6800640 1 --2- 172.21.15.7:0/2894139105 >> v2:172.21.15.87:3300/0 conn(0x7f2fc815bc40 0x7f2fc80aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.696+0000 7f2fc6800640 1 -- 172.21.15.7:0/2894139105 >> v1:172.21.15.7:6789/0 conn(0x7f2fc80af370 legacy=0x7f2fc8161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.696+0000 7f2fc6800640 1 -- 172.21.15.7:0/2894139105 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc80b0560 con 0x7f2fc815ef90 2024-04-25T11:20:23.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.697+0000 7f2fc6800640 1 --2- 172.21.15.7:0/2894139105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc815ef90 0x7f2fc809f790 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f2fc806f2b0 tx=0x7f2fb802e520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571690e838ae3ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:23.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.698+0000 7f2fc5e00640 1 -- 172.21.15.7:0/2894139105 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb803a070 con 0x7f2fc815ef90 2024-04-25T11:20:23.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.698+0000 7f2fc5e00640 1 -- 172.21.15.7:0/2894139105 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb8033710 con 0x7f2fc815ef90 2024-04-25T11:20:23.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.698+0000 7f2fc5e00640 1 -- 172.21.15.7:0/2894139105 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8033ae0 con 0x7f2fc815ef90 2024-04-25T11:20:23.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.698+0000 7f2fcd400640 1 -- 172.21.15.7:0/2894139105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc815ef90 msgr2=0x7f2fc809f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.698+0000 7f2fcd400640 1 --2- 172.21.15.7:0/2894139105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc815ef90 0x7f2fc809f790 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f2fc806f2b0 tx=0x7f2fb802e520 comp rx=0 tx=0).stop 2024-04-25T11:20:23.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.698+0000 7f2fcd400640 1 -- 172.21.15.7:0/2894139105 shutdown_connections 2024-04-25T11:20:23.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.698+0000 7f2fcd400640 1 --2- 172.21.15.7:0/2894139105 >> v2:172.21.15.87:3300/0 conn(0x7f2fc815bc40 0x7f2fc80aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.698+0000 7f2fcd400640 1 --2- 172.21.15.7:0/2894139105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc815ef90 0x7f2fc809f790 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.698+0000 7f2fcd400640 1 -- 172.21.15.7:0/2894139105 >> 172.21.15.7:0/2894139105 conn(0x7f2fc8074200 msgr2=0x7f2fc806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:23.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.699+0000 7f2fcd400640 1 -- 172.21.15.7:0/2894139105 shutdown_connections 2024-04-25T11:20:23.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.699+0000 7f2fcd400640 1 -- 172.21.15.7:0/2894139105 wait complete. 2024-04-25T11:20:23.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.701+0000 7f2fcd400640 1 Processor -- start 2024-04-25T11:20:23.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.701+0000 7f2fcd400640 1 -- start start 2024-04-25T11:20:23.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.702+0000 7f2fcd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc80af370 0x7f2fc81462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.702+0000 7f2fcd400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc815bc40 0x7f2fc8156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fcd400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc815ef90 0x7f2fc81f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fcd400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc80b0e00 con 0x7f2fc80af370 2024-04-25T11:20:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fcd400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc80b0b20 con 0x7f2fc815ef90 2024-04-25T11:20:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fcd400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc80b1320 con 0x7f2fc815bc40 2024-04-25T11:20:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fc6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc80af370 0x7f2fc81462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fc6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc80af370 0x7f2fc81462f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41510/0 (socket says 172.21.15.7:41510) 2024-04-25T11:20:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fc6800640 1 -- 172.21.15.7:0/4007048844 learned_addr learned my addr 172.21.15.7:0/4007048844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fbfe00640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc815bc40 0x7f2fc8156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fc7200640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc815ef90 0x7f2fc81f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fc6800640 1 -- 172.21.15.7:0/4007048844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc815bc40 msgr2=0x7f2fc8156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fc6800640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc815bc40 0x7f2fc8156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fc6800640 1 -- 172.21.15.7:0/4007048844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc815ef90 msgr2=0x7f2fc81f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fc6800640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc815ef90 0x7f2fc81f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fc6800640 1 -- 172.21.15.7:0/4007048844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc8146830 con 0x7f2fc80af370 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fbfe00640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc815bc40 0x7f2fc8156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.703+0000 7f2fc7200640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc815ef90 0x7f2fc81f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.704+0000 7f2fc6800640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc80af370 0x7f2fc81462f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009890 tx=0x7f2fb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.704+0000 7f2fbea00640 1 -- 172.21.15.7:0/4007048844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8044020 con 0x7f2fc80af370 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.704+0000 7f2fbea00640 1 -- 172.21.15.7:0/4007048844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb8035a40 con 0x7f2fc80af370 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.704+0000 7f2fbea00640 1 -- 172.21.15.7:0/4007048844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb803a040 con 0x7f2fc80af370 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.704+0000 7f2fcd400640 1 -- 172.21.15.7:0/4007048844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc8156ef0 con 0x7f2fc80af370 2024-04-25T11:20:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.704+0000 7f2fcd400640 1 -- 172.21.15.7:0/4007048844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc81574b0 con 0x7f2fc80af370 2024-04-25T11:20:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.706+0000 7f2fcd400640 1 -- 172.21.15.7:0/4007048844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc806a100 con 0x7f2fc80af370 2024-04-25T11:20:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.707+0000 7f2fbea00640 1 -- 172.21.15.7:0/4007048844 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2fb8047020 con 0x7f2fc80af370 2024-04-25T11:20:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.708+0000 7f2fbea00640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.708+0000 7f2fbea00640 1 -- 172.21.15.7:0/4007048844 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f2fb80b9f20 con 0x7f2fc80af370 2024-04-25T11:20:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.708+0000 7f2fbfe00640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:23.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.709+0000 7f2fbfe00640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006fd0 tx=0x7f2fb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.713+0000 7f2fbea00640 1 -- 172.21.15.7:0/4007048844 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fb804e030 con 0x7f2fc80af370 2024-04-25T11:20:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.966+0000 7f2fcd400640 1 -- 172.21.15.7:0/4007048844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fc80cfe40 con 0x7f2fc80af370 2024-04-25T11:20:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.967+0000 7f2fbea00640 1 -- 172.21.15.7:0/4007048844 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5175 (secure 0 0 0) 0x7f2fb80845b0 con 0x7f2fc80af370 2024-04-25T11:20:23.965 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:23.965 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/326382713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":326382713},{"type":"v1","addr":"172.21.15.7:6837","nonce":326382713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2007810270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2007810270},{"type":"v1","addr":"172.21.15.87:6835","nonce":2007810270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1753656512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1753656512},{"type":"v1","addr":"172.21.15.7:6836","nonce":1753656512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:47.807534+0000","modified":"2024-04-25T11:20:20.325877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":11,"addr":"172.21.15.87:6834/1924056839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1924056839},{"type":"v1","addr":"172.21.15.87:6834","nonce":1924056839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8007,"qdb_cluster":[8007]},"id":8}]} 2024-04-25T11:20:23.965 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 80 2024-04-25T11:20:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 -- 172.21.15.7:0/4007048844 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 msgr2=0x7f2fa40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006fd0 tx=0x7f2fb0008040 comp rx=0 tx=0).stop 2024-04-25T11:20:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 -- 172.21.15.7:0/4007048844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc80af370 msgr2=0x7f2fc81462f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc80af370 0x7f2fc81462f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009890 tx=0x7f2fb80046c0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 -- 172.21.15.7:0/4007048844 shutdown_connections 2024-04-25T11:20:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa40744b0 0x7f2fa40769c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc815ef90 0x7f2fc81f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc815bc40 0x7f2fc8156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 --2- 172.21.15.7:0/4007048844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fc80af370 0x7f2fc81462f0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 -- 172.21.15.7:0/4007048844 >> 172.21.15.7:0/4007048844 conn(0x7f2fc8074200 msgr2=0x7f2fc80afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 -- 172.21.15.7:0/4007048844 shutdown_connections 2024-04-25T11:20:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:23.970+0000 7f2fcd400640 1 -- 172.21.15.7:0/4007048844 wait complete. 2024-04-25T11:20:23.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:24.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.266+0000 7fd971a00640 1 Processor -- start 2024-04-25T11:20:24.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.266+0000 7fd971a00640 1 -- start start 2024-04-25T11:20:24.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.268+0000 7fd971a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd96c06e950 0x7fd96c0d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.268+0000 7fd971a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd96c0d0810 0x7fd96c0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.268+0000 7fd971a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd96c06ed60 con 0x7fd96c0b7880 2024-04-25T11:20:24.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.268+0000 7fd971a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd96c0d15b0 con 0x7fd96c0d0810 2024-04-25T11:20:24.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.268+0000 7fd971a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd96c0d12d0 con 0x7fd96c06e950 2024-04-25T11:20:24.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.269+0000 7fd96b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd96c0d0810 0x7fd96c0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.269+0000 7fd96b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd96c0d0810 0x7fd96c0d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41526/0 (socket says 172.21.15.7:41526) 2024-04-25T11:20:24.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.269+0000 7fd96b400640 1 -- 172.21.15.7:0/3328666545 learned_addr learned my addr 172.21.15.7:0/3328666545 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:24.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.269+0000 7fd96be00640 1 --2- 172.21.15.7:0/3328666545 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd96c06e950 0x7fd96c0d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.269+0000 7fd96b400640 1 -- 172.21.15.7:0/3328666545 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd96c06e950 msgr2=0x7fd96c0d4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.269+0000 7fd96b400640 1 --2- 172.21.15.7:0/3328666545 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd96c06e950 0x7fd96c0d4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.269+0000 7fd96b400640 1 -- 172.21.15.7:0/3328666545 >> v1:172.21.15.87:6789/0 conn(0x7fd96c0b7880 legacy=0x7fd96c0b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.269+0000 7fd96b400640 1 -- 172.21.15.7:0/3328666545 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c06de70 con 0x7fd96c0d0810 2024-04-25T11:20:24.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.271+0000 7fd96b400640 1 --2- 172.21.15.7:0/3328666545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd96c0d0810 0x7fd96c0d0c20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd9540098a0 tx=0x7fd95402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d321d9a85a2d8d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:24.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.271+0000 7fd96aa00640 1 -- 172.21.15.7:0/3328666545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954037070 con 0x7fd96c0d0810 2024-04-25T11:20:24.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.271+0000 7fd96aa00640 1 -- 172.21.15.7:0/3328666545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd954032440 con 0x7fd96c0d0810 2024-04-25T11:20:24.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.271+0000 7fd96aa00640 1 -- 172.21.15.7:0/3328666545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd954032810 con 0x7fd96c0d0810 2024-04-25T11:20:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.272+0000 7fd971a00640 1 -- 172.21.15.7:0/3328666545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd96c0d0810 msgr2=0x7fd96c0d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.272+0000 7fd971a00640 1 --2- 172.21.15.7:0/3328666545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd96c0d0810 0x7fd96c0d0c20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd9540098a0 tx=0x7fd95402a290 comp rx=0 tx=0).stop 2024-04-25T11:20:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.272+0000 7fd971a00640 1 -- 172.21.15.7:0/3328666545 shutdown_connections 2024-04-25T11:20:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.272+0000 7fd971a00640 1 --2- 172.21.15.7:0/3328666545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd96c0d0810 0x7fd96c0d0c20 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.272+0000 7fd971a00640 1 --2- 172.21.15.7:0/3328666545 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd96c06e950 0x7fd96c0d4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.272+0000 7fd971a00640 1 -- 172.21.15.7:0/3328666545 >> 172.21.15.7:0/3328666545 conn(0x7fd96c0d2840 msgr2=0x7fd96c0d2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.272+0000 7fd971a00640 1 -- 172.21.15.7:0/3328666545 shutdown_connections 2024-04-25T11:20:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.272+0000 7fd971a00640 1 -- 172.21.15.7:0/3328666545 wait complete. 2024-04-25T11:20:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.274+0000 7fd971a00640 1 Processor -- start 2024-04-25T11:20:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.275+0000 7fd971a00640 1 -- start start 2024-04-25T11:20:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.276+0000 7fd971a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd96c06e950 0x7fd96c1fa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.276+0000 7fd971a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c0b7880 0x7fd96c15d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.276+0000 7fd971a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd96c0d0810 0x7fd96c163c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.276+0000 7fd971a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c0d5190 con 0x7fd96c06e950 2024-04-25T11:20:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.276+0000 7fd971a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c06e430 con 0x7fd96c0b7880 2024-04-25T11:20:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.276+0000 7fd971a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd96c0d5310 con 0x7fd96c0d0810 2024-04-25T11:20:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.276+0000 7fd96be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd96c06e950 0x7fd96c1fa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.276+0000 7fd96be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd96c06e950 0x7fd96c1fa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41532/0 (socket says 172.21.15.7:41532) 2024-04-25T11:20:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.276+0000 7fd96be00640 1 -- 172.21.15.7:0/975927170 learned_addr learned my addr 172.21.15.7:0/975927170 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:24.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.277+0000 7fd96b400640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c0b7880 0x7fd96c15d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.277+0000 7fd971000640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd96c0d0810 0x7fd96c163c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.277+0000 7fd96be00640 1 -- 172.21.15.7:0/975927170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd96c0d0810 msgr2=0x7fd96c163c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.277+0000 7fd96be00640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd96c0d0810 0x7fd96c163c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.277+0000 7fd96be00640 1 -- 172.21.15.7:0/975927170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c0b7880 msgr2=0x7fd96c15d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.277+0000 7fd96be00640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c0b7880 0x7fd96c15d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.277+0000 7fd96be00640 1 -- 172.21.15.7:0/975927170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c1fabf0 con 0x7fd96c06e950 2024-04-25T11:20:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.277+0000 7fd96b400640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c0b7880 0x7fd96c15d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.277+0000 7fd96be00640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd96c06e950 0x7fd96c1fa6b0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00d5d0 tx=0x7fd95c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.278+0000 7fd963e00640 1 -- 172.21.15.7:0/975927170 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c004240 con 0x7fd96c06e950 2024-04-25T11:20:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.278+0000 7fd963e00640 1 -- 172.21.15.7:0/975927170 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd95c0043e0 con 0x7fd96c06e950 2024-04-25T11:20:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.278+0000 7fd963e00640 1 -- 172.21.15.7:0/975927170 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c003830 con 0x7fd96c06e950 2024-04-25T11:20:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.278+0000 7fd971a00640 1 -- 172.21.15.7:0/975927170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c0ce8a0 con 0x7fd96c06e950 2024-04-25T11:20:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.278+0000 7fd971a00640 1 -- 172.21.15.7:0/975927170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c0cf0b0 con 0x7fd96c06e950 2024-04-25T11:20:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.279+0000 7fd971a00640 1 -- 172.21.15.7:0/975927170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd96c15e260 con 0x7fd96c06e950 2024-04-25T11:20:24.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.280+0000 7fd963e00640 1 -- 172.21.15.7:0/975927170 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd95c0039d0 con 0x7fd96c06e950 2024-04-25T11:20:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.282+0000 7fd963e00640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744b0 0x7fd9480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.282+0000 7fd963e00640 1 -- 172.21.15.7:0/975927170 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fd95c0956f0 con 0x7fd96c06e950 2024-04-25T11:20:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.282+0000 7fd96b400640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744b0 0x7fd9480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.283+0000 7fd96b400640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744b0 0x7fd9480769c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd954034000 tx=0x7fd95402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:24.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.287+0000 7fd963e00640 1 -- 172.21.15.7:0/975927170 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd96c1fabf0 con 0x7fd96c06e950 2024-04-25T11:20:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.535+0000 7fd971a00640 1 -- 172.21.15.7:0/975927170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd96c050740 con 0x7fd96c06e950 2024-04-25T11:20:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.536+0000 7fd963e00640 1 -- 172.21.15.7:0/975927170 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5175 (secure 0 0 0) 0x7fd95c05fce0 con 0x7fd96c06e950 2024-04-25T11:20:24.534 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:24.534 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/326382713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":326382713},{"type":"v1","addr":"172.21.15.7:6837","nonce":326382713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2007810270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2007810270},{"type":"v1","addr":"172.21.15.87:6835","nonce":2007810270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1753656512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1753656512},{"type":"v1","addr":"172.21.15.7:6836","nonce":1753656512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:19:47.807534+0000","modified":"2024-04-25T11:20:20.325877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8007},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8007":{"gid":8007,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":11,"addr":"172.21.15.87:6834/1924056839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1924056839},{"type":"v1","addr":"172.21.15.87:6834","nonce":1924056839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8007,"qdb_cluster":[8007]},"id":8}]} 2024-04-25T11:20:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 80 2024-04-25T11:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.538+0000 7fd971a00640 1 -- 172.21.15.7:0/975927170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744b0 msgr2=0x7fd9480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.538+0000 7fd971a00640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744b0 0x7fd9480769c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd954034000 tx=0x7fd95402e000 comp rx=0 tx=0).stop 2024-04-25T11:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.538+0000 7fd971a00640 1 -- 172.21.15.7:0/975927170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd96c06e950 msgr2=0x7fd96c1fa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.538+0000 7fd971a00640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd96c06e950 0x7fd96c1fa6b0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00d5d0 tx=0x7fd95c00da90 comp rx=0 tx=0).stop 2024-04-25T11:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.539+0000 7fd971a00640 1 -- 172.21.15.7:0/975927170 shutdown_connections 2024-04-25T11:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.539+0000 7fd971a00640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9480744b0 0x7fd9480769c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.539+0000 7fd971a00640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd96c0d0810 0x7fd96c163c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.539+0000 7fd971a00640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd96c0b7880 0x7fd96c15d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.539+0000 7fd971a00640 1 --2- 172.21.15.7:0/975927170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd96c06e950 0x7fd96c1fa6b0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.539+0000 7fd971a00640 1 -- 172.21.15.7:0/975927170 >> 172.21.15.7:0/975927170 conn(0x7fd96c0d2840 msgr2=0x7fd96c0b9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.539+0000 7fd971a00640 1 -- 172.21.15.7:0/975927170 shutdown_connections 2024-04-25T11:20:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.539+0000 7fd971a00640 1 -- 172.21.15.7:0/975927170 wait complete. 2024-04-25T11:20:24.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.851+0000 7f82fd800640 1 Processor -- start 2024-04-25T11:20:24.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.851+0000 7f82fd800640 1 -- start start 2024-04-25T11:20:24.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.853+0000 7f82fd800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f82f80afb30 0x7f82f815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.854+0000 7f82fd800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f80cebb0 con 0x7f82f80b5220 2024-04-25T11:20:24.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.854+0000 7f82fd800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82f80b64b0 con 0x7f82f80aed40 2024-04-25T11:20:24.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.854+0000 7f82fd800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f82f815e1f0 con 0x7f82f80afb30 2024-04-25T11:20:24.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.854+0000 7f82f7400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f82f80b5220 0x7f82f80af420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46722/0 (socket says 172.21.15.7:46722) 2024-04-25T11:20:24.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.854+0000 7f82f7400640 1 -- 172.21.15.7:0/3229021394 learned_addr learned my addr 172.21.15.7:0/3229021394 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.854+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2207857764 0 0) 0x7f82e0001110 con 0x7f82f80b5220 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.854+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82dc003520 con 0x7f82f80b5220 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.854+0000 7f82fce00640 1 --2- 172.21.15.7:0/3229021394 >> v2:172.21.15.87:3300/0 conn(0x7f82f80afb30 0x7f82f815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1974561873 0 0) 0x7f82dc003520 con 0x7f82f80b5220 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 >> v2:172.21.15.87:3300/0 conn(0x7f82f80afb30 msgr2=0x7f82f815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 --2- 172.21.15.7:0/3229021394 >> v2:172.21.15.87:3300/0 conn(0x7f82f80afb30 0x7f82f815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 >> v1:172.21.15.87:6789/0 conn(0x7f82f80aed40 legacy=0x7f82f80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f806c610 con 0x7f82f80b5220 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f82e0001840 con 0x7f82f80b5220 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 >> v1:172.21.15.7:6789/0 conn(0x7f82f80b5220 legacy=0x7f82f80af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 --2- 172.21.15.7:0/3229021394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82dc0039b0 0x7f82dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 --2- 172.21.15.7:0/3229021394 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82dc004800 0x7f82dc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 --2- 172.21.15.7:0/3229021394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 0x7f82dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82f815e1f0 con 0x7f82dc005270 2024-04-25T11:20:24.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc00c1e0 con 0x7f82dc0039b0 2024-04-25T11:20:24.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82dc00c3e0 con 0x7f82dc004800 2024-04-25T11:20:24.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82fce00640 1 --2- 172.21.15.7:0/3229021394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 0x7f82dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82f7e00640 1 --2- 172.21.15.7:0/3229021394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82dc0039b0 0x7f82dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82fce00640 1 -- 172.21.15.7:0/3229021394 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82dc004800 msgr2=0x7f82dc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:24.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.855+0000 7f82fce00640 1 --2- 172.21.15.7:0/3229021394 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82dc004800 0x7f82dc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.856+0000 7f82fce00640 1 -- 172.21.15.7:0/3229021394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82dc0039b0 msgr2=0x7f82dc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:24.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.856+0000 7f82fce00640 1 --2- 172.21.15.7:0/3229021394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82dc0039b0 0x7f82dc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.856+0000 7f82fce00640 1 -- 172.21.15.7:0/3229021394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc003520 con 0x7f82dc005270 2024-04-25T11:20:24.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.856+0000 7f82fce00640 1 -- 172.21.15.7:0/3229021394 reap_dead start 2024-04-25T11:20:24.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.857+0000 7f82fce00640 1 --2- 172.21.15.7:0/3229021394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 0x7f82dc00b9c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f82f000ba30 tx=0x7f82dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602b1070d5ddfce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:24.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.857+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f0030070 con 0x7f82dc005270 2024-04-25T11:20:24.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.857+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f002ae50 con 0x7f82dc005270 2024-04-25T11:20:24.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.857+0000 7f82f6a00640 1 -- 172.21.15.7:0/3229021394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f000b600 con 0x7f82dc005270 2024-04-25T11:20:24.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.857+0000 7f82fd800640 1 -- 172.21.15.7:0/3229021394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 msgr2=0x7f82dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.857+0000 7f82fd800640 1 --2- 172.21.15.7:0/3229021394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 0x7f82dc00b9c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f82f000ba30 tx=0x7f82dc005830 comp rx=0 tx=0).stop 2024-04-25T11:20:24.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.858+0000 7f82fd800640 1 -- 172.21.15.7:0/3229021394 shutdown_connections 2024-04-25T11:20:24.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.858+0000 7f82fd800640 1 --2- 172.21.15.7:0/3229021394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 0x7f82dc00b9c0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.858+0000 7f82fd800640 1 -- 172.21.15.7:0/3229021394 >> 172.21.15.7:0/3229021394 conn(0x7f82f80b3820 msgr2=0x7f82f80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:24.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.858+0000 7f82fd800640 1 -- 172.21.15.7:0/3229021394 shutdown_connections 2024-04-25T11:20:24.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.858+0000 7f82fd800640 1 -- 172.21.15.7:0/3229021394 wait complete. 2024-04-25T11:20:24.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.860+0000 7f82fd800640 1 Processor -- start 2024-04-25T11:20:24.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.860+0000 7f82fd800640 1 -- start start 2024-04-25T11:20:24.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82fd800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 0x7f82f814bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82fd800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82f80d0c60 0x7f82f8158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82fd800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82f80d1070 0x7f82f8159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82fd800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82f806ceb0 con 0x7f82dc005270 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82fd800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82f806cbd0 con 0x7f82f80d1070 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82fd800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82f80b6110 con 0x7f82f80d0c60 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82f7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 0x7f82f814bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82f7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 0x7f82f814bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41544/0 (socket says 172.21.15.7:41544) 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82f7e00640 1 -- 172.21.15.7:0/546305676 learned_addr learned my addr 172.21.15.7:0/546305676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82f7400640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82f80d0c60 0x7f82f8158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82f7e00640 1 -- 172.21.15.7:0/546305676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82f80d0c60 msgr2=0x7f82f8158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82f7e00640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82f80d0c60 0x7f82f8158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82f7e00640 1 -- 172.21.15.7:0/546305676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82f80d1070 msgr2=0x7f82f8159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82f7e00640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82f80d1070 0x7f82f8159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.862+0000 7f82f7e00640 1 -- 172.21.15.7:0/546305676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f815e6e0 con 0x7f82dc005270 2024-04-25T11:20:24.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.863+0000 7f82f7e00640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 0x7f82f814bf70 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f82e800e800 tx=0x7f82e800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:24.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.863+0000 7f82efe00640 1 -- 172.21.15.7:0/546305676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e8003df0 con 0x7f82dc005270 2024-04-25T11:20:24.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.863+0000 7f82efe00640 1 -- 172.21.15.7:0/546305676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e800c6c0 con 0x7f82dc005270 2024-04-25T11:20:24.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.863+0000 7f82fd800640 1 -- 172.21.15.7:0/546305676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f815e9c0 con 0x7f82dc005270 2024-04-25T11:20:24.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.863+0000 7f82efe00640 1 -- 172.21.15.7:0/546305676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e800cab0 con 0x7f82dc005270 2024-04-25T11:20:24.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.863+0000 7f82fd800640 1 -- 172.21.15.7:0/546305676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f815ef80 con 0x7f82dc005270 2024-04-25T11:20:24.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.864+0000 7f82fd800640 1 -- 172.21.15.7:0/546305676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f8050740 con 0x7f82dc005270 2024-04-25T11:20:24.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.866+0000 7f82efe00640 1 -- 172.21.15.7:0/546305676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f82e800cc50 con 0x7f82dc005270 2024-04-25T11:20:24.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.867+0000 7f82efe00640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82d40744b0 0x7f82d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:24.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.867+0000 7f82efe00640 1 -- 172.21.15.7:0/546305676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7f82e8095900 con 0x7f82dc005270 2024-04-25T11:20:24.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.867+0000 7f82f7400640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82d40744b0 0x7f82d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:24.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.868+0000 7f82f7400640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82d40744b0 0x7f82d40769c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f82f81614c0 tx=0x7f82e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:24.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:24.871+0000 7f82efe00640 1 -- 172.21.15.7:0/546305676 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f82e805fef0 con 0x7f82dc005270 2024-04-25T11:20:25.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.073+0000 7f82fd800640 1 -- 172.21.15.7:0/546305676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82f80008d0 con 0x7f82dc005270 2024-04-25T11:20:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.074+0000 7f82efe00640 1 -- 172.21.15.7:0/546305676 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16856 (secure 0 0 0) 0x7f82e80660c0 con 0x7f82dc005270 2024-04-25T11:20:25.072 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:25.072 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:19:49.277316+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:19:46.014943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T11:19:46.794047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.077+0000 7f82fd800640 1 -- 172.21.15.7:0/546305676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82d40744b0 msgr2=0x7f82d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.077+0000 7f82fd800640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82d40744b0 0x7f82d40769c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f82f81614c0 tx=0x7f82e0009040 comp rx=0 tx=0).stop 2024-04-25T11:20:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.077+0000 7f82fd800640 1 -- 172.21.15.7:0/546305676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 msgr2=0x7f82f814bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.077+0000 7f82fd800640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 0x7f82f814bf70 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f82e800e800 tx=0x7f82e800ecc0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.077+0000 7f82fd800640 1 -- 172.21.15.7:0/546305676 shutdown_connections 2024-04-25T11:20:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.078+0000 7f82fd800640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82d40744b0 0x7f82d40769c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.078+0000 7f82fd800640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82f80d1070 0x7f82f8159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.078+0000 7f82fd800640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82f80d0c60 0x7f82f8158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.078+0000 7f82fd800640 1 --2- 172.21.15.7:0/546305676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82dc005270 0x7f82f814bf70 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.078+0000 7f82fd800640 1 -- 172.21.15.7:0/546305676 >> 172.21.15.7:0/546305676 conn(0x7f82f80b3820 msgr2=0x7f82f80b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.078+0000 7f82fd800640 1 -- 172.21.15.7:0/546305676 shutdown_connections 2024-04-25T11:20:25.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.078+0000 7f82fd800640 1 -- 172.21.15.7:0/546305676 wait complete. 2024-04-25T11:20:25.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:20:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.392+0000 7fbb69600640 1 Processor -- start 2024-04-25T11:20:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.392+0000 7fbb69600640 1 -- start start 2024-04-25T11:20:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.395+0000 7fbb69600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb6415eed0 0x7fbb640cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.395+0000 7fbb69600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbb641416a0 0x7fbb640b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.395+0000 7fbb69600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb6406c570 con 0x7fbb64141290 2024-04-25T11:20:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.395+0000 7fbb69600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbb6406c850 con 0x7fbb641416a0 2024-04-25T11:20:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.395+0000 7fbb69600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6406cb30 con 0x7fbb6415eed0 2024-04-25T11:20:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.396+0000 7fbb63e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb6415eed0 0x7fbb640cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.396+0000 7fbb63400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fbb64141290 0x7fbb6406d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:52150/0 (socket says 172.21.15.7:52150) 2024-04-25T11:20:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.396+0000 7fbb63400640 1 -- 172.21.15.7:0/1845855681 learned_addr learned my addr 172.21.15.7:0/1845855681 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.396+0000 7fbb68c00640 1 --2- 172.21.15.7:0/1845855681 >> v2:172.21.15.87:3300/0 conn(0x7fbb641416a0 0x7fbb640b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.397+0000 7fbb62a00640 1 -- 172.21.15.7:0/1845855681 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1554171030 0 0) 0x7fbb6406c570 con 0x7fbb64141290 2024-04-25T11:20:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.397+0000 7fbb63e00640 1 -- 172.21.15.7:0/1845855681 >> v2:172.21.15.87:3300/0 conn(0x7fbb641416a0 msgr2=0x7fbb640b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.397+0000 7fbb63e00640 1 --2- 172.21.15.7:0/1845855681 >> v2:172.21.15.87:3300/0 conn(0x7fbb641416a0 0x7fbb640b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.397+0000 7fbb63e00640 1 -- 172.21.15.7:0/1845855681 >> v1:172.21.15.87:6789/0 conn(0x7fbb64141290 legacy=0x7fbb6406d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.397+0000 7fbb63e00640 1 -- 172.21.15.7:0/1845855681 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6406ce10 con 0x7fbb6415eed0 2024-04-25T11:20:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.398+0000 7fbb63e00640 1 --2- 172.21.15.7:0/1845855681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb6415eed0 0x7fbb640cd780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbb540097a0 tx=0x7fbb5402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eefba8c2dc1e58a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.399+0000 7fbb62a00640 1 -- 172.21.15.7:0/1845855681 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54037070 con 0x7fbb6415eed0 2024-04-25T11:20:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.399+0000 7fbb62a00640 1 -- 172.21.15.7:0/1845855681 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb54032440 con 0x7fbb6415eed0 2024-04-25T11:20:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.399+0000 7fbb62a00640 1 -- 172.21.15.7:0/1845855681 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54032810 con 0x7fbb6415eed0 2024-04-25T11:20:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.400+0000 7fbb69600640 1 -- 172.21.15.7:0/1845855681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb6415eed0 msgr2=0x7fbb640cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.400+0000 7fbb69600640 1 --2- 172.21.15.7:0/1845855681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb6415eed0 0x7fbb640cd780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbb540097a0 tx=0x7fbb5402a180 comp rx=0 tx=0).stop 2024-04-25T11:20:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.400+0000 7fbb69600640 1 -- 172.21.15.7:0/1845855681 shutdown_connections 2024-04-25T11:20:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.400+0000 7fbb69600640 1 --2- 172.21.15.7:0/1845855681 >> v2:172.21.15.87:3300/0 conn(0x7fbb641416a0 0x7fbb640b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.400+0000 7fbb69600640 1 --2- 172.21.15.7:0/1845855681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb6415eed0 0x7fbb640cd780 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.400+0000 7fbb69600640 1 -- 172.21.15.7:0/1845855681 >> 172.21.15.7:0/1845855681 conn(0x7fbb640d1540 msgr2=0x7fbb640d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.400+0000 7fbb69600640 1 -- 172.21.15.7:0/1845855681 shutdown_connections 2024-04-25T11:20:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.400+0000 7fbb69600640 1 -- 172.21.15.7:0/1845855681 wait complete. 2024-04-25T11:20:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.402+0000 7fbb69600640 1 Processor -- start 2024-04-25T11:20:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.403+0000 7fbb69600640 1 -- start start 2024-04-25T11:20:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.404+0000 7fbb69600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64141290 0x7fbb641537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.404+0000 7fbb69600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb641416a0 0x7fbb641f9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.404+0000 7fbb69600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb6415eed0 0x7fbb64070100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.404+0000 7fbb69600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbb64141ab0 con 0x7fbb6415eed0 2024-04-25T11:20:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.404+0000 7fbb69600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbb640cdcc0 con 0x7fbb64141290 2024-04-25T11:20:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.404+0000 7fbb69600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6406dd20 con 0x7fbb641416a0 2024-04-25T11:20:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.404+0000 7fbb68c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb6415eed0 0x7fbb64070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.405+0000 7fbb68c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb6415eed0 0x7fbb64070100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41556/0 (socket says 172.21.15.7:41556) 2024-04-25T11:20:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.405+0000 7fbb68c00640 1 -- 172.21.15.7:0/501346473 learned_addr learned my addr 172.21.15.7:0/501346473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.405+0000 7fbb63e00640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64141290 0x7fbb641537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.405+0000 7fbb68c00640 1 -- 172.21.15.7:0/501346473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb641416a0 msgr2=0x7fbb641f9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.405+0000 7fbb68c00640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb641416a0 0x7fbb641f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.405+0000 7fbb68c00640 1 -- 172.21.15.7:0/501346473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64141290 msgr2=0x7fbb641537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.405+0000 7fbb68c00640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64141290 0x7fbb641537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.405+0000 7fbb68c00640 1 -- 172.21.15.7:0/501346473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb641f9de0 con 0x7fbb6415eed0 2024-04-25T11:20:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.405+0000 7fbb68c00640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb6415eed0 0x7fbb64070100 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00efc0 tx=0x7fbb5c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.406+0000 7fbb5be00640 1 -- 172.21.15.7:0/501346473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c018020 con 0x7fbb6415eed0 2024-04-25T11:20:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.406+0000 7fbb5be00640 1 -- 172.21.15.7:0/501346473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb5c014480 con 0x7fbb6415eed0 2024-04-25T11:20:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.406+0000 7fbb69600640 1 -- 172.21.15.7:0/501346473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6413da90 con 0x7fbb6415eed0 2024-04-25T11:20:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.406+0000 7fbb5be00640 1 -- 172.21.15.7:0/501346473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c009e60 con 0x7fbb6415eed0 2024-04-25T11:20:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.406+0000 7fbb69600640 1 -- 172.21.15.7:0/501346473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb641549d0 con 0x7fbb6415eed0 2024-04-25T11:20:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.407+0000 7fbb69600640 1 -- 172.21.15.7:0/501346473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb64050740 con 0x7fbb6415eed0 2024-04-25T11:20:25.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.409+0000 7fbb5be00640 1 -- 172.21.15.7:0/501346473 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbb5c005d10 con 0x7fbb6415eed0 2024-04-25T11:20:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.410+0000 7fbb5be00640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb440744e0 0x7fbb440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.410+0000 7fbb5be00640 1 -- 172.21.15.7:0/501346473 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 6101+0+0 (secure 0 0 0) 0x7fbb5c0960d0 con 0x7fbb6415eed0 2024-04-25T11:20:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.410+0000 7fbb63e00640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb440744e0 0x7fbb440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.411+0000 7fbb63e00640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb440744e0 0x7fbb440769f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbb54034000 tx=0x7fbb5402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.415+0000 7fbb5be00640 1 -- 172.21.15.7:0/501346473 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbb5c022030 con 0x7fbb6415eed0 2024-04-25T11:20:25.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.660+0000 7fbb69600640 1 -- 172.21.15.7:0/501346473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbb641f9b30 con 0x7fbb6415eed0 2024-04-25T11:20:25.658 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-04-25T11:20:25.662+0000 7fac34e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:20:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.666+0000 7fbb5be00640 1 -- 172.21.15.7:0/501346473 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) ==== 161+0+0 (secure 0 0 0) 0x7fbb5c010040 con 0x7fbb6415eed0 2024-04-25T11:20:25.663 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:20:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.669+0000 7fbb69600640 1 -- 172.21.15.7:0/501346473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb440744e0 msgr2=0x7fbb440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.669+0000 7fbb69600640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb440744e0 0x7fbb440769f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbb54034000 tx=0x7fbb5402e000 comp rx=0 tx=0).stop 2024-04-25T11:20:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.669+0000 7fbb69600640 1 -- 172.21.15.7:0/501346473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb6415eed0 msgr2=0x7fbb64070100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.669+0000 7fbb69600640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb6415eed0 0x7fbb64070100 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00efc0 tx=0x7fbb5c00c490 comp rx=0 tx=0).stop 2024-04-25T11:20:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.670+0000 7fbb69600640 1 -- 172.21.15.7:0/501346473 shutdown_connections 2024-04-25T11:20:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.670+0000 7fbb69600640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbb440744e0 0x7fbb440769f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.670+0000 7fbb69600640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb6415eed0 0x7fbb64070100 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.670+0000 7fbb69600640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb641416a0 0x7fbb641f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.670+0000 7fbb69600640 1 --2- 172.21.15.7:0/501346473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb64141290 0x7fbb641537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.670+0000 7fbb69600640 1 -- 172.21.15.7:0/501346473 >> 172.21.15.7:0/501346473 conn(0x7fbb640d1540 msgr2=0x7fbb641403b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.670+0000 7fbb69600640 1 -- 172.21.15.7:0/501346473 shutdown_connections 2024-04-25T11:20:25.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.670+0000 7fbb69600640 1 -- 172.21.15.7:0/501346473 wait complete. 2024-04-25T11:20:25.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:20:25.872 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:20:25.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.982+0000 7f7728a00640 1 Processor -- start 2024-04-25T11:20:25.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.982+0000 7f7728a00640 1 -- start start 2024-04-25T11:20:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.985+0000 7f7728a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f772415aaa0 0x7f77240affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.985+0000 7f7728a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f77240d0ee0 0x7f77240b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.985+0000 7f7728a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f772406f5b0 con 0x7f77240b4e50 2024-04-25T11:20:25.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.985+0000 7f7728a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f77240af290 con 0x7f772415aaa0 2024-04-25T11:20:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.985+0000 7f7728a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f77240d14a0 con 0x7f77240d0ee0 2024-04-25T11:20:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.985+0000 7f7721e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f772415aaa0 0x7f77240affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.985+0000 7f7722800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f77240b4e50 0x7f772415d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46724/0 (socket says 172.21.15.7:46724) 2024-04-25T11:20:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.985+0000 7f7722800640 1 -- 172.21.15.7:0/2508826315 learned_addr learned my addr 172.21.15.7:0/2508826315 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.985+0000 7f7721e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f772415aaa0 0x7f77240affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41558/0 (socket says 172.21.15.7:41558) 2024-04-25T11:20:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.985+0000 7f771be00640 1 --2- 172.21.15.7:0/2508826315 >> v2:172.21.15.87:3300/0 conn(0x7f77240d0ee0 0x7f77240b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.986+0000 7f7721e00640 1 -- 172.21.15.7:0/2508826315 >> v2:172.21.15.87:3300/0 conn(0x7f77240d0ee0 msgr2=0x7f77240b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.986+0000 7f7721e00640 1 --2- 172.21.15.7:0/2508826315 >> v2:172.21.15.87:3300/0 conn(0x7f77240d0ee0 0x7f77240b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.986+0000 7f7721e00640 1 -- 172.21.15.7:0/2508826315 >> v1:172.21.15.7:6789/0 conn(0x7f77240b4e50 legacy=0x7f772415d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.986+0000 7f7721e00640 1 -- 172.21.15.7:0/2508826315 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77240af4e0 con 0x7f772415aaa0 2024-04-25T11:20:25.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.986+0000 7f7721400640 1 -- 172.21.15.7:0/2508826315 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277817382 0 0) 0x7f772406f5b0 con 0x7f77240b4e50 2024-04-25T11:20:25.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.987+0000 7f7721e00640 1 --2- 172.21.15.7:0/2508826315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f772415aaa0 0x7f77240affc0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f77240af410 tx=0x7f771402c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c0dcf01cd10f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.987+0000 7f7721400640 1 -- 172.21.15.7:0/2508826315 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771403a070 con 0x7f772415aaa0 2024-04-25T11:20:25.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.987+0000 7f7721400640 1 -- 172.21.15.7:0/2508826315 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7714031070 con 0x7f772415aaa0 2024-04-25T11:20:25.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.987+0000 7f7721400640 1 -- 172.21.15.7:0/2508826315 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77140328f0 con 0x7f772415aaa0 2024-04-25T11:20:25.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.988+0000 7f7728a00640 1 -- 172.21.15.7:0/2508826315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f772415aaa0 msgr2=0x7f77240affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.988+0000 7f7728a00640 1 --2- 172.21.15.7:0/2508826315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f772415aaa0 0x7f77240affc0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f77240af410 tx=0x7f771402c7d0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.988+0000 7f7728a00640 1 -- 172.21.15.7:0/2508826315 shutdown_connections 2024-04-25T11:20:25.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.988+0000 7f7728a00640 1 --2- 172.21.15.7:0/2508826315 >> v2:172.21.15.87:3300/0 conn(0x7f77240d0ee0 0x7f77240b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.988+0000 7f7728a00640 1 --2- 172.21.15.7:0/2508826315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f772415aaa0 0x7f77240affc0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.988+0000 7f7728a00640 1 -- 172.21.15.7:0/2508826315 >> 172.21.15.7:0/2508826315 conn(0x7f77240742c0 msgr2=0x7f772406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:25.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.988+0000 7f7728a00640 1 -- 172.21.15.7:0/2508826315 shutdown_connections 2024-04-25T11:20:25.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.988+0000 7f7728a00640 1 -- 172.21.15.7:0/2508826315 wait complete. 2024-04-25T11:20:25.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.990+0000 7f7728a00640 1 Processor -- start 2024-04-25T11:20:25.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.991+0000 7f7728a00640 1 -- start start 2024-04-25T11:20:25.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.992+0000 7f7728a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77240b4e50 0x7f7724159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.992+0000 7f7728a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77240d0ee0 0x7f77241e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.992+0000 7f7728a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f772415aaa0 0x7f77241e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.992+0000 7f7728a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77240b5260 con 0x7f772415aaa0 2024-04-25T11:20:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.992+0000 7f7728a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f77240afaa0 con 0x7f77240b4e50 2024-04-25T11:20:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.992+0000 7f7728a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f77240b53e0 con 0x7f77240d0ee0 2024-04-25T11:20:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f7722800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f772415aaa0 0x7f77241e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f7722800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f772415aaa0 0x7f77241e7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41570/0 (socket says 172.21.15.7:41570) 2024-04-25T11:20:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f7722800640 1 -- 172.21.15.7:0/3580998388 learned_addr learned my addr 172.21.15.7:0/3580998388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f7721e00640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77240b4e50 0x7f7724159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f771be00640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77240d0ee0 0x7f77241e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f7722800640 1 -- 172.21.15.7:0/3580998388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77240d0ee0 msgr2=0x7f77241e75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f7722800640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77240d0ee0 0x7f77241e75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f7722800640 1 -- 172.21.15.7:0/3580998388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77240b4e50 msgr2=0x7f7724159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f7722800640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77240b4e50 0x7f7724159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f7722800640 1 -- 172.21.15.7:0/3580998388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77241f6580 con 0x7f772415aaa0 2024-04-25T11:20:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f771be00640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77240d0ee0 0x7f77241e75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.993+0000 7f7722800640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f772415aaa0 0x7f77241e7b10 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f771c00c730 tx=0x7f771c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.994+0000 7f771a000640 1 -- 172.21.15.7:0/3580998388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c018070 con 0x7f772415aaa0 2024-04-25T11:20:25.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.994+0000 7f771a000640 1 -- 172.21.15.7:0/3580998388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f771c00ce40 con 0x7f772415aaa0 2024-04-25T11:20:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.994+0000 7f7728a00640 1 -- 172.21.15.7:0/3580998388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772415a060 con 0x7f772415aaa0 2024-04-25T11:20:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.994+0000 7f771a000640 1 -- 172.21.15.7:0/3580998388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c00a860 con 0x7f772415aaa0 2024-04-25T11:20:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.994+0000 7f7728a00640 1 -- 172.21.15.7:0/3580998388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77241e6dd0 con 0x7f772415aaa0 2024-04-25T11:20:25.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.995+0000 7f7728a00640 1 -- 172.21.15.7:0/3580998388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f772415a6e0 con 0x7f772415aaa0 2024-04-25T11:20:25.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.996+0000 7f771a000640 1 -- 172.21.15.7:0/3580998388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f771c01b070 con 0x7f772415aaa0 2024-04-25T11:20:25.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.997+0000 7f771a000640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77000744b0 0x7f77000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:25.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.998+0000 7f771a000640 1 -- 172.21.15.7:0/3580998388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f771c0949c0 con 0x7f772415aaa0 2024-04-25T11:20:25.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.998+0000 7f7721e00640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77000744b0 0x7f77000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:25.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:25.999+0000 7f7721e00640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77000744b0 0x7f77000769c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7714037000 tx=0x7f7714032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:25.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.003+0000 7f771a000640 1 -- 172.21.15.7:0/3580998388 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f772415a6e0 con 0x7f772415aaa0 2024-04-25T11:20:26.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.250+0000 7f7728a00640 1 -- 172.21.15.7:0/3580998388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7724159df0 con 0x7f772415aaa0 2024-04-25T11:20:26.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.678+0000 7f771a000640 1 -- 172.21.15.7:0/3580998388 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) ==== 104+0+0 (secure 0 0 0) 0x7f771c05ef60 con 0x7f772415aaa0 2024-04-25T11:20:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.681+0000 7f7728a00640 1 -- 172.21.15.7:0/3580998388 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77000744b0 msgr2=0x7f77000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.681+0000 7f7728a00640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77000744b0 0x7f77000769c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7714037000 tx=0x7f7714032000 comp rx=0 tx=0).stop 2024-04-25T11:20:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.682+0000 7f7728a00640 1 -- 172.21.15.7:0/3580998388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f772415aaa0 msgr2=0x7f77241e7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.682+0000 7f7728a00640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f772415aaa0 0x7f77241e7b10 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f771c00c730 tx=0x7f771c00cbf0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.682+0000 7f7728a00640 1 -- 172.21.15.7:0/3580998388 shutdown_connections 2024-04-25T11:20:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.682+0000 7f7728a00640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f77000744b0 0x7f77000769c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.682+0000 7f7728a00640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f772415aaa0 0x7f77241e7b10 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.682+0000 7f7728a00640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77240d0ee0 0x7f77241e75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.682+0000 7f7728a00640 1 --2- 172.21.15.7:0/3580998388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77240b4e50 0x7f7724159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:26.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.682+0000 7f7728a00640 1 -- 172.21.15.7:0/3580998388 >> 172.21.15.7:0/3580998388 conn(0x7f77240742c0 msgr2=0x7f772406c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.682+0000 7f7728a00640 1 -- 172.21.15.7:0/3580998388 shutdown_connections 2024-04-25T11:20:26.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.682+0000 7f7728a00640 1 -- 172.21.15.7:0/3580998388 wait complete. 2024-04-25T11:20:26.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:20:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.998+0000 7f3b6ce00640 1 Processor -- start 2024-04-25T11:20:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:26.999+0000 7f3b6ce00640 1 -- start start 2024-04-25T11:20:26.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.001+0000 7f3b6ce00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3b680b48a0 0x7f3b6815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.001+0000 7f3b6ce00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b680aec60 0x7f3b680b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.001+0000 7f3b6ce00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b680b0940 0x7f3b680b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.001+0000 7f3b6ce00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3b680b1290 con 0x7f3b680b0940 2024-04-25T11:20:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.001+0000 7f3b6ce00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3b680b4da0 con 0x7f3b680b48a0 2024-04-25T11:20:26.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.001+0000 7f3b6ce00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3b680b5080 con 0x7f3b680aec60 2024-04-25T11:20:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b66800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b680b0940 0x7f3b680b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b66800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b680b0940 0x7f3b680b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41584/0 (socket says 172.21.15.7:41584) 2024-04-25T11:20:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b66800640 1 -- 172.21.15.7:0/3030522243 learned_addr learned my addr 172.21.15.7:0/3030522243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b65e00640 1 --2- 172.21.15.7:0/3030522243 >> v2:172.21.15.87:3300/0 conn(0x7f3b680b48a0 0x7f3b6815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b65400640 1 --2- 172.21.15.7:0/3030522243 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b680aec60 0x7f3b680b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b66800640 1 -- 172.21.15.7:0/3030522243 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b680aec60 msgr2=0x7f3b680b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b66800640 1 --2- 172.21.15.7:0/3030522243 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b680aec60 0x7f3b680b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b66800640 1 -- 172.21.15.7:0/3030522243 >> v2:172.21.15.87:3300/0 conn(0x7f3b680b48a0 msgr2=0x7f3b6815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:27.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b66800640 1 --2- 172.21.15.7:0/3030522243 >> v2:172.21.15.87:3300/0 conn(0x7f3b680b48a0 0x7f3b6815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b66800640 1 -- 172.21.15.7:0/3030522243 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b680b5360 con 0x7f3b680b0940 2024-04-25T11:20:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b65400640 1 --2- 172.21.15.7:0/3030522243 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b680aec60 0x7f3b680b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:27.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.002+0000 7f3b65e00640 1 --2- 172.21.15.7:0/3030522243 >> v2:172.21.15.87:3300/0 conn(0x7f3b680b48a0 0x7f3b6815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.003+0000 7f3b66800640 1 --2- 172.21.15.7:0/3030522243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b680b0940 0x7f3b680b0d50 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3b68071390 tx=0x7f3b6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593df1357e577c1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.004+0000 7f3b64a00640 1 -- 172.21.15.7:0/3030522243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60037070 con 0x7f3b680b0940 2024-04-25T11:20:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.004+0000 7f3b64a00640 1 -- 172.21.15.7:0/3030522243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b60032440 con 0x7f3b680b0940 2024-04-25T11:20:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.004+0000 7f3b64a00640 1 -- 172.21.15.7:0/3030522243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60032810 con 0x7f3b680b0940 2024-04-25T11:20:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.005+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3030522243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b680b0940 msgr2=0x7f3b680b0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.005+0000 7f3b6ce00640 1 --2- 172.21.15.7:0/3030522243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b680b0940 0x7f3b680b0d50 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f3b68071390 tx=0x7f3b6002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.005+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3030522243 shutdown_connections 2024-04-25T11:20:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.005+0000 7f3b6ce00640 1 --2- 172.21.15.7:0/3030522243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b680b0940 0x7f3b680b0d50 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.005+0000 7f3b6ce00640 1 --2- 172.21.15.7:0/3030522243 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b680aec60 0x7f3b680b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.005+0000 7f3b6ce00640 1 --2- 172.21.15.7:0/3030522243 >> v2:172.21.15.87:3300/0 conn(0x7f3b680b48a0 0x7f3b6815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.005+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3030522243 >> 172.21.15.7:0/3030522243 conn(0x7f3b68073920 msgr2=0x7f3b68073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.005+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3030522243 shutdown_connections 2024-04-25T11:20:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.005+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3030522243 wait complete. 2024-04-25T11:20:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.007+0000 7f3b6ce00640 1 Processor -- start 2024-04-25T11:20:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.007+0000 7f3b6ce00640 1 -- start start 2024-04-25T11:20:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.009+0000 7f3b6ce00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b680aec60 0x7f3b681fa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.009+0000 7f3b6ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b680b0940 0x7f3b681f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.009+0000 7f3b6ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b680b48a0 0x7f3b680700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.009+0000 7f3b6ce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6815e1d0 con 0x7f3b680b48a0 2024-04-25T11:20:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.009+0000 7f3b6ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3b680b5920 con 0x7f3b680b0940 2024-04-25T11:20:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.009+0000 7f3b6ce00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3b680b6380 con 0x7f3b680aec60 2024-04-25T11:20:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.009+0000 7f3b66800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b680b48a0 0x7f3b680700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.009+0000 7f3b66800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b680b48a0 0x7f3b680700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41598/0 (socket says 172.21.15.7:41598) 2024-04-25T11:20:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.009+0000 7f3b66800640 1 -- 172.21.15.7:0/3284729230 learned_addr learned my addr 172.21.15.7:0/3284729230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.010+0000 7f3b65400640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b680b0940 0x7f3b681f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.010+0000 7f3b65e00640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b680aec60 0x7f3b681fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.010+0000 7f3b66800640 1 -- 172.21.15.7:0/3284729230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b680aec60 msgr2=0x7f3b681fa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.010+0000 7f3b66800640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b680aec60 0x7f3b681fa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.010+0000 7f3b66800640 1 -- 172.21.15.7:0/3284729230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b680b0940 msgr2=0x7f3b681f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.010+0000 7f3b66800640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b680b0940 0x7f3b681f6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.010+0000 7f3b66800640 1 -- 172.21.15.7:0/3284729230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6814c590 con 0x7f3b680b48a0 2024-04-25T11:20:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.010+0000 7f3b66800640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b680b48a0 0x7f3b680700c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f3b6002a220 tx=0x7f3b600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.010+0000 7f3b5e000640 1 -- 172.21.15.7:0/3284729230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60037040 con 0x7f3b680b48a0 2024-04-25T11:20:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.011+0000 7f3b5e000640 1 -- 172.21.15.7:0/3284729230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b60042070 con 0x7f3b680b48a0 2024-04-25T11:20:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.011+0000 7f3b5e000640 1 -- 172.21.15.7:0/3284729230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60030bd0 con 0x7f3b680b48a0 2024-04-25T11:20:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.011+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3284729230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6814c870 con 0x7f3b680b48a0 2024-04-25T11:20:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.011+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3284729230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6814ce30 con 0x7f3b680b48a0 2024-04-25T11:20:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.012+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3284729230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b6814c300 con 0x7f3b680b48a0 2024-04-25T11:20:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.013+0000 7f3b5e000640 1 -- 172.21.15.7:0/3284729230 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3b60047460 con 0x7f3b680b48a0 2024-04-25T11:20:27.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.014+0000 7f3b5e000640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b440744e0 0x7f3b440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:27.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.014+0000 7f3b5e000640 1 -- 172.21.15.7:0/3284729230 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6187+0+0 (secure 0 0 0) 0x7f3b600b6140 con 0x7f3b680b48a0 2024-04-25T11:20:27.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.015+0000 7f3b65e00640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b440744e0 0x7f3b440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:27.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.015+0000 7f3b65e00640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b440744e0 0x7f3b440769f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3b680cf3f0 tx=0x7f3b58007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:27.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.020+0000 7f3b5e000640 1 -- 172.21.15.7:0/3284729230 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3b60080000 con 0x7f3b680b48a0 2024-04-25T11:20:27.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:27.220+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3284729230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3b6806a100 con 0x7f3b680b48a0 2024-04-25T11:20:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.222+0000 7f3b5e000640 1 -- 172.21.15.7:0/3284729230 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) ==== 181+0+0 (secure 0 0 0) 0x7f3b60084300 con 0x7f3b680b48a0 2024-04-25T11:20:28.220 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:20:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.226+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3284729230 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b440744e0 msgr2=0x7f3b440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.226+0000 7f3b6ce00640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b440744e0 0x7f3b440769f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3b680cf3f0 tx=0x7f3b58007040 comp rx=0 tx=0).stop 2024-04-25T11:20:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.226+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3284729230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b680b48a0 msgr2=0x7f3b680700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.226+0000 7f3b6ce00640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b680b48a0 0x7f3b680700c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f3b6002a220 tx=0x7f3b600046c0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.227+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3284729230 shutdown_connections 2024-04-25T11:20:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.227+0000 7f3b6ce00640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3b440744e0 0x7f3b440769f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.227+0000 7f3b6ce00640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b680b48a0 0x7f3b680700c0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.227+0000 7f3b6ce00640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b680b0940 0x7f3b681f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.227+0000 7f3b6ce00640 1 --2- 172.21.15.7:0/3284729230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b680aec60 0x7f3b681fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.227+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3284729230 >> 172.21.15.7:0/3284729230 conn(0x7f3b68073920 msgr2=0x7f3b680cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:28.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.228+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3284729230 shutdown_connections 2024-04-25T11:20:28.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.228+0000 7f3b6ce00640 1 -- 172.21.15.7:0/3284729230 wait complete. 2024-04-25T11:20:28.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:20:28.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.556+0000 7f876ac00640 1 Processor -- start 2024-04-25T11:20:28.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.557+0000 7f876ac00640 1 -- start start 2024-04-25T11:20:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.558+0000 7f876ac00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8764163a30 0x7f87640b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.558+0000 7f876ac00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87640d0b50 0x7f87640d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.558+0000 7f876ac00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87640d14a0 con 0x7f876406dcb0 2024-04-25T11:20:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.558+0000 7f876ac00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f876406e3b0 con 0x7f87640d0b50 2024-04-25T11:20:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.558+0000 7f876ac00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f876406e690 con 0x7f8764163a30 2024-04-25T11:20:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.559+0000 7f876a200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87640d0b50 0x7f87640d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.559+0000 7f876a200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87640d0b50 0x7f87640d0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41606/0 (socket says 172.21.15.7:41606) 2024-04-25T11:20:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.559+0000 7f876a200640 1 -- 172.21.15.7:0/2616732636 learned_addr learned my addr 172.21.15.7:0/2616732636 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.559+0000 7f8769800640 1 --2- 172.21.15.7:0/2616732636 >> v2:172.21.15.87:3300/0 conn(0x7f8764163a30 0x7f87640b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.559+0000 7f876a200640 1 -- 172.21.15.7:0/2616732636 >> v2:172.21.15.87:3300/0 conn(0x7f8764163a30 msgr2=0x7f87640b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.559+0000 7f876a200640 1 --2- 172.21.15.7:0/2616732636 >> v2:172.21.15.87:3300/0 conn(0x7f8764163a30 0x7f87640b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.559+0000 7f876a200640 1 -- 172.21.15.7:0/2616732636 >> v1:172.21.15.87:6789/0 conn(0x7f876406dcb0 legacy=0x7f87640d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.559+0000 7f876a200640 1 -- 172.21.15.7:0/2616732636 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876406e970 con 0x7f87640d0b50 2024-04-25T11:20:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.561+0000 7f876a200640 1 --2- 172.21.15.7:0/2616732636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87640d0b50 0x7f87640d0f60 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f87640d3890 tx=0x7f875c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7f281f0c4cdb4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.561+0000 7f8763e00640 1 -- 172.21.15.7:0/2616732636 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c036070 con 0x7f87640d0b50 2024-04-25T11:20:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.561+0000 7f8763e00640 1 -- 172.21.15.7:0/2616732636 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f875c031600 con 0x7f87640d0b50 2024-04-25T11:20:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.561+0000 7f8763e00640 1 -- 172.21.15.7:0/2616732636 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c0319d0 con 0x7f87640d0b50 2024-04-25T11:20:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.562+0000 7f876ac00640 1 -- 172.21.15.7:0/2616732636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87640d0b50 msgr2=0x7f87640d0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.562+0000 7f876ac00640 1 --2- 172.21.15.7:0/2616732636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87640d0b50 0x7f87640d0f60 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f87640d3890 tx=0x7f875c02a660 comp rx=0 tx=0).stop 2024-04-25T11:20:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.562+0000 7f876ac00640 1 -- 172.21.15.7:0/2616732636 shutdown_connections 2024-04-25T11:20:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.562+0000 7f876ac00640 1 --2- 172.21.15.7:0/2616732636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87640d0b50 0x7f87640d0f60 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.562+0000 7f876ac00640 1 --2- 172.21.15.7:0/2616732636 >> v2:172.21.15.87:3300/0 conn(0x7f8764163a30 0x7f87640b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.562+0000 7f876ac00640 1 -- 172.21.15.7:0/2616732636 >> 172.21.15.7:0/2616732636 conn(0x7f87640d2590 msgr2=0x7f87640d29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.562+0000 7f876ac00640 1 -- 172.21.15.7:0/2616732636 shutdown_connections 2024-04-25T11:20:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.562+0000 7f876ac00640 1 -- 172.21.15.7:0/2616732636 wait complete. 2024-04-25T11:20:28.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.564+0000 7f876ac00640 1 Processor -- start 2024-04-25T11:20:28.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.564+0000 7f876ac00640 1 -- start start 2024-04-25T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.566+0000 7f876ac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f876406dcb0 0x7f87640cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.566+0000 7f876ac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87640d0b50 0x7f87640b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.566+0000 7f876ac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8764163a30 0x7f87641f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.566+0000 7f876ac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f87640d4680 con 0x7f87640d0b50 2024-04-25T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.566+0000 7f876ac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f876406e0c0 con 0x7f876406dcb0 2024-04-25T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.566+0000 7f876ac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87640d4800 con 0x7f8764163a30 2024-04-25T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.566+0000 7f8768e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87640d0b50 0x7f87640b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.566+0000 7f8768e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87640d0b50 0x7f87640b71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41622/0 (socket says 172.21.15.7:41622) 2024-04-25T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.566+0000 7f8768e00640 1 -- 172.21.15.7:0/550233931 learned_addr learned my addr 172.21.15.7:0/550233931 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.567+0000 7f8769800640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f876406dcb0 0x7f87640cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.567+0000 7f876a200640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8764163a30 0x7f87641f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.567+0000 7f8768e00640 1 -- 172.21.15.7:0/550233931 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8764163a30 msgr2=0x7f87641f9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.567+0000 7f8768e00640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8764163a30 0x7f87641f9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.567+0000 7f8768e00640 1 -- 172.21.15.7:0/550233931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f876406dcb0 msgr2=0x7f87640cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.567+0000 7f8768e00640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f876406dcb0 0x7f87640cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.567+0000 7f8768e00640 1 -- 172.21.15.7:0/550233931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87640cda10 con 0x7f87640d0b50 2024-04-25T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.567+0000 7f8768e00640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87640d0b50 0x7f87640b71c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f875400b3f0 tx=0x7f875400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:28.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.567+0000 7f8761600640 1 -- 172.21.15.7:0/550233931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754015020 con 0x7f87640d0b50 2024-04-25T11:20:28.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.568+0000 7f8761600640 1 -- 172.21.15.7:0/550233931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8754011830 con 0x7f87640d0b50 2024-04-25T11:20:28.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.568+0000 7f8761600640 1 -- 172.21.15.7:0/550233931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754011c20 con 0x7f87640d0b50 2024-04-25T11:20:28.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.568+0000 7f876ac00640 1 -- 172.21.15.7:0/550233931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87640c9500 con 0x7f87640d0b50 2024-04-25T11:20:28.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.568+0000 7f876ac00640 1 -- 172.21.15.7:0/550233931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87640cd450 con 0x7f87640d0b50 2024-04-25T11:20:28.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.569+0000 7f876ac00640 1 -- 172.21.15.7:0/550233931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87640d3d50 con 0x7f87640d0b50 2024-04-25T11:20:28.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.570+0000 7f8761600640 1 -- 172.21.15.7:0/550233931 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8754018020 con 0x7f87640d0b50 2024-04-25T11:20:28.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.572+0000 7f8761600640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87480744b0 0x7f87480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:28.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.572+0000 7f8761600640 1 -- 172.21.15.7:0/550233931 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5749+0+0 (secure 0 0 0) 0x7f8754094e50 con 0x7f87640d0b50 2024-04-25T11:20:28.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.572+0000 7f8769800640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87480744b0 0x7f87480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:28.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.573+0000 7f8769800640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87480744b0 0x7f87480769c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8758006fd0 tx=0x7f8758008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:28.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.577+0000 7f8761600640 1 -- 172.21.15.7:0/550233931 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f875401f030 con 0x7f87640d0b50 2024-04-25T11:20:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:28.785+0000 7f876ac00640 1 -- 172.21.15.7:0/550233931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f87640cc230 con 0x7f87640d0b50 2024-04-25T11:20:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.226+0000 7f8761600640 1 -- 172.21.15.7:0/550233931 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) ==== 169+0+0 (secure 0 0 0) 0x7f875405f630 con 0x7f87640d0b50 2024-04-25T11:20:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-25T11:20:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.231+0000 7f876ac00640 1 -- 172.21.15.7:0/550233931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87480744b0 msgr2=0x7f87480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.231+0000 7f876ac00640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87480744b0 0x7f87480769c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8758006fd0 tx=0x7f8758008040 comp rx=0 tx=0).stop 2024-04-25T11:20:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.231+0000 7f876ac00640 1 -- 172.21.15.7:0/550233931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87640d0b50 msgr2=0x7f87640b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.231+0000 7f876ac00640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87640d0b50 0x7f87640b71c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f875400b3f0 tx=0x7f875400b8b0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.232+0000 7f876ac00640 1 -- 172.21.15.7:0/550233931 shutdown_connections 2024-04-25T11:20:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.232+0000 7f876ac00640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87480744b0 0x7f87480769c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.232+0000 7f876ac00640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8764163a30 0x7f87641f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.232+0000 7f876ac00640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87640d0b50 0x7f87640b71c0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.232+0000 7f876ac00640 1 --2- 172.21.15.7:0/550233931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f876406dcb0 0x7f87640cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.232+0000 7f876ac00640 1 -- 172.21.15.7:0/550233931 >> 172.21.15.7:0/550233931 conn(0x7f87640d2590 msgr2=0x7f87640d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.232+0000 7f876ac00640 1 -- 172.21.15.7:0/550233931 shutdown_connections 2024-04-25T11:20:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.232+0000 7f876ac00640 1 -- 172.21.15.7:0/550233931 wait complete. 2024-04-25T11:20:29.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-04-25T11:20:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.552+0000 7faa0c800640 1 Processor -- start 2024-04-25T11:20:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.552+0000 7faa0c800640 1 -- start start 2024-04-25T11:20:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.555+0000 7faa0c800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faa0415dfb0 0x7faa040b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.555+0000 7faa0c800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa0415e3c0 con 0x7faa040d0b40 2024-04-25T11:20:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.555+0000 7faa0c800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa040cf800 con 0x7faa0406fab0 2024-04-25T11:20:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.555+0000 7faa0c800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7faa040b09b0 con 0x7faa0415dfb0 2024-04-25T11:20:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.555+0000 7faa09400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faa040d0b40 0x7faa0406f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46740/0 (socket says 172.21.15.7:46740) 2024-04-25T11:20:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.555+0000 7faa09400640 1 -- 172.21.15.7:0/740668044 learned_addr learned my addr 172.21.15.7:0/740668044 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.555+0000 7faa09e00640 1 --2- 172.21.15.7:0/740668044 >> v2:172.21.15.87:3300/0 conn(0x7faa0415dfb0 0x7faa040b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1314473129 0 0) 0x7faa0415e3c0 con 0x7faa040d0b40 2024-04-25T11:20:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9ec003520 con 0x7faa040d0b40 2024-04-25T11:20:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 16903028 0 0) 0x7fa9ec003520 con 0x7faa040d0b40 2024-04-25T11:20:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 >> v2:172.21.15.87:3300/0 conn(0x7faa0415dfb0 msgr2=0x7faa040b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 --2- 172.21.15.7:0/740668044 >> v2:172.21.15.87:3300/0 conn(0x7faa0415dfb0 0x7faa040b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 >> v1:172.21.15.87:6789/0 conn(0x7faa0406fab0 legacy=0x7faa040aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa040cfaa0 con 0x7faa040d0b40 2024-04-25T11:20:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fa9f4001670 con 0x7faa040d0b40 2024-04-25T11:20:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 >> v1:172.21.15.7:6789/0 conn(0x7faa040d0b40 legacy=0x7faa0406f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 --2- 172.21.15.7:0/740668044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9ec003a00 0x7fa9ec003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 --2- 172.21.15.7:0/740668044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9ec004800 0x7fa9ec004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 --2- 172.21.15.7:0/740668044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9ec005220 0x7fa9ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa040b09b0 con 0x7fa9ec003a00 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa9ec00c1b0 con 0x7fa9ec004800 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.556+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ec00c3b0 con 0x7fa9ec005220 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.557+0000 7faa09e00640 1 --2- 172.21.15.7:0/740668044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9ec003a00 0x7fa9ec003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.557+0000 7faa09400640 1 --2- 172.21.15.7:0/740668044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9ec004800 0x7fa9ec004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.557+0000 7faa09e00640 1 -- 172.21.15.7:0/740668044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9ec005220 msgr2=0x7fa9ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.557+0000 7faa09e00640 1 --2- 172.21.15.7:0/740668044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9ec005220 0x7fa9ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.557+0000 7faa09e00640 1 -- 172.21.15.7:0/740668044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9ec004800 msgr2=0x7fa9ec004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.557+0000 7faa09e00640 1 --2- 172.21.15.7:0/740668044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9ec004800 0x7fa9ec004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.557+0000 7faa09e00640 1 -- 172.21.15.7:0/740668044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9ec003520 con 0x7fa9ec003a00 2024-04-25T11:20:29.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.557+0000 7faa09400640 1 --2- 172.21.15.7:0/740668044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9ec004800 0x7fa9ec004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:29.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.557+0000 7faa0a800640 1 -- 172.21.15.7:0/740668044 reap_dead start 2024-04-25T11:20:29.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.558+0000 7faa09e00640 1 --2- 172.21.15.7:0/740668044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9ec003a00 0x7fa9ec003e90 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009a90 tx=0x7fa9ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d05b0f34d342d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.558+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8036070 con 0x7fa9ec003a00 2024-04-25T11:20:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.559+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f8031400 con 0x7fa9ec003a00 2024-04-25T11:20:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.559+0000 7faa08a00640 1 -- 172.21.15.7:0/740668044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8009690 con 0x7fa9ec003a00 2024-04-25T11:20:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.559+0000 7faa0c800640 1 -- 172.21.15.7:0/740668044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9ec003a00 msgr2=0x7fa9ec003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.559+0000 7faa0c800640 1 --2- 172.21.15.7:0/740668044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9ec003a00 0x7fa9ec003e90 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009a90 tx=0x7fa9ec005830 comp rx=0 tx=0).stop 2024-04-25T11:20:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.560+0000 7faa0c800640 1 -- 172.21.15.7:0/740668044 shutdown_connections 2024-04-25T11:20:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.560+0000 7faa0c800640 1 --2- 172.21.15.7:0/740668044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9ec003a00 0x7fa9ec003e90 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.560+0000 7faa0c800640 1 -- 172.21.15.7:0/740668044 >> 172.21.15.7:0/740668044 conn(0x7faa040742c0 msgr2=0x7faa0406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.560+0000 7faa0c800640 1 -- 172.21.15.7:0/740668044 shutdown_connections 2024-04-25T11:20:29.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.560+0000 7faa0c800640 1 -- 172.21.15.7:0/740668044 wait complete. 2024-04-25T11:20:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.563+0000 7faa0c800640 1 Processor -- start 2024-04-25T11:20:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.563+0000 7faa0c800640 1 -- start start 2024-04-25T11:20:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.565+0000 7faa0c800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9ec003a00 0x7faa041604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.565+0000 7faa0c800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa0414f6b0 0x7faa041476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.565+0000 7faa0c800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa04158f40 0x7faa04159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.565+0000 7faa0c800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa040d0340 con 0x7faa0414f6b0 2024-04-25T11:20:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.565+0000 7faa0c800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faa040d0060 con 0x7fa9ec003a00 2024-04-25T11:20:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.565+0000 7faa0c800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faa040d0620 con 0x7faa04158f40 2024-04-25T11:20:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.565+0000 7faa09400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa0414f6b0 0x7faa041476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.566+0000 7faa09400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa0414f6b0 0x7faa041476e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41640/0 (socket says 172.21.15.7:41640) 2024-04-25T11:20:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.566+0000 7faa09400640 1 -- 172.21.15.7:0/2728549184 learned_addr learned my addr 172.21.15.7:0/2728549184 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.566+0000 7faa09e00640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9ec003a00 0x7faa041604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.566+0000 7faa0a800640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa04158f40 0x7faa04159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.566+0000 7faa09400640 1 -- 172.21.15.7:0/2728549184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa04158f40 msgr2=0x7faa04159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.566+0000 7faa09400640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa04158f40 0x7faa04159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.566+0000 7faa09400640 1 -- 172.21.15.7:0/2728549184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9ec003a00 msgr2=0x7faa041604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.566+0000 7faa09400640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9ec003a00 0x7faa041604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.566+0000 7faa09400640 1 -- 172.21.15.7:0/2728549184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa04159890 con 0x7faa0414f6b0 2024-04-25T11:20:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.566+0000 7faa09400640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa0414f6b0 0x7faa041476e0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400d7a0 tx=0x7fa9f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.567+0000 7fa9fe000640 1 -- 172.21.15.7:0/2728549184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4019070 con 0x7faa0414f6b0 2024-04-25T11:20:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.567+0000 7fa9fe000640 1 -- 172.21.15.7:0/2728549184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9f400de50 con 0x7faa0414f6b0 2024-04-25T11:20:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.567+0000 7fa9fe000640 1 -- 172.21.15.7:0/2728549184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40047e0 con 0x7faa0414f6b0 2024-04-25T11:20:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.567+0000 7faa0c800640 1 -- 172.21.15.7:0/2728549184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa04160a10 con 0x7faa0414f6b0 2024-04-25T11:20:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.567+0000 7faa0c800640 1 -- 172.21.15.7:0/2728549184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa040b5110 con 0x7faa0414f6b0 2024-04-25T11:20:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.568+0000 7faa0c800640 1 -- 172.21.15.7:0/2728549184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa04050740 con 0x7faa0414f6b0 2024-04-25T11:20:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.570+0000 7fa9fe000640 1 -- 172.21.15.7:0/2728549184 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa9f401c070 con 0x7faa0414f6b0 2024-04-25T11:20:29.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.570+0000 7fa9fe000640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9e80744e0 0x7fa9e80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:29.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.571+0000 7fa9fe000640 1 -- 172.21.15.7:0/2728549184 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7fa9f40949d0 con 0x7faa0414f6b0 2024-04-25T11:20:29.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.571+0000 7faa09e00640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9e80744e0 0x7fa9e80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:29.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.572+0000 7faa09e00640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9e80744e0 0x7fa9e80769f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8033000 tx=0x7fa9f8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.576+0000 7fa9fe000640 1 -- 172.21.15.7:0/2728549184 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faa04159890 con 0x7faa0414f6b0 2024-04-25T11:20:29.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.786+0000 7faa0c800640 1 -- 172.21.15.7:0/2728549184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7faa04070fb0 con 0x7faa0414f6b0 2024-04-25T11:20:29.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.793+0000 7fa9fe000640 1 -- 172.21.15.7:0/2728549184 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v41) ==== 102+0+0 (secure 0 0 0) 0x7fa9f405f360 con 0x7faa0414f6b0 2024-04-25T11:20:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 -- 172.21.15.7:0/2728549184 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9e80744e0 msgr2=0x7fa9e80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9e80744e0 0x7fa9e80769f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8033000 tx=0x7fa9f8008030 comp rx=0 tx=0).stop 2024-04-25T11:20:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 -- 172.21.15.7:0/2728549184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa0414f6b0 msgr2=0x7faa041476e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa0414f6b0 0x7faa041476e0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400d7a0 tx=0x7fa9f400dc60 comp rx=0 tx=0).stop 2024-04-25T11:20:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 -- 172.21.15.7:0/2728549184 shutdown_connections 2024-04-25T11:20:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa9e80744e0 0x7fa9e80769f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa04158f40 0x7faa04159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa0414f6b0 0x7faa041476e0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 --2- 172.21.15.7:0/2728549184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9ec003a00 0x7faa041604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:29.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 -- 172.21.15.7:0/2728549184 >> 172.21.15.7:0/2728549184 conn(0x7faa040742c0 msgr2=0x7faa0406f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:29.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 -- 172.21.15.7:0/2728549184 shutdown_connections 2024-04-25T11:20:29.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:29.796+0000 7faa0c800640 1 -- 172.21.15.7:0/2728549184 wait complete. 2024-04-25T11:20:29.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-04-25T11:20:30.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.105+0000 7f487fe00640 1 Processor -- start 2024-04-25T11:20:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.106+0000 7f487fe00640 1 -- start start 2024-04-25T11:20:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.108+0000 7f487fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4880144a80 0x7f4880146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.108+0000 7f487fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48800b4f80 0x7f48800b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.108+0000 7f487fe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48800b66b0 con 0x7f4880145e40 2024-04-25T11:20:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.108+0000 7f487fe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4880145b70 con 0x7f4880144a80 2024-04-25T11:20:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.108+0000 7f487fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48800b58d0 con 0x7f48800b4f80 2024-04-25T11:20:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.108+0000 7f487ea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4880145e40 0x7f48800b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58310/0 (socket says 172.21.15.7:58310) 2024-04-25T11:20:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.108+0000 7f487ea00640 1 -- 172.21.15.7:0/1554199510 learned_addr learned my addr 172.21.15.7:0/1554199510 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:30.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.108+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2100035370 0 0) 0x7f48800b66b0 con 0x7f4880145e40 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4864003520 con 0x7f4880145e40 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f4877e00640 1 --2- 172.21.15.7:0/1554199510 >> v2:172.21.15.87:3300/0 conn(0x7f4880144a80 0x7f4880146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487f400640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48800b4f80 0x7f48800b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3233088963 0 0) 0x7f4864003520 con 0x7f4880145e40 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48800b4f80 msgr2=0x7f48800b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48800b4f80 0x7f48800b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 >> v2:172.21.15.87:3300/0 conn(0x7f4880144a80 msgr2=0x7f4880146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 --2- 172.21.15.7:0/1554199510 >> v2:172.21.15.87:3300/0 conn(0x7f4880144a80 0x7f4880146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48800b5b30 con 0x7f4880145e40 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f4870001690 con 0x7f4880145e40 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 >> v1:172.21.15.7:6789/0 conn(0x7f4880145e40 legacy=0x7f48800b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48640039b0 0x7f4864003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48800b4f80 0x7f4864004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4864004d40 0x7f4864005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48800b58d0 con 0x7f4864004d40 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f486400bb80 con 0x7f48640039b0 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f486400bd80 con 0x7f48800b4f80 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.109+0000 7f487f400640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4864004d40 0x7f4864005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.110+0000 7f487ea00640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48640039b0 0x7f4864003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.110+0000 7f4877e00640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48800b4f80 0x7f4864004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.110+0000 7f487f400640 1 -- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48800b4f80 msgr2=0x7f4864004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.110+0000 7f487f400640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48800b4f80 0x7f4864004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.110+0000 7f487f400640 1 -- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48640039b0 msgr2=0x7f4864003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.110+0000 7f487f400640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48640039b0 0x7f4864003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.110+0000 7f487f400640 1 -- 172.21.15.7:0/1554199510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4864003520 con 0x7f4864004d40 2024-04-25T11:20:30.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.110+0000 7f487ea00640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48640039b0 0x7f4864003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.111+0000 7f487f400640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4864004d40 0x7f4864005150 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f4878007360 tx=0x7f4864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66dfab5db15a134b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:30.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.111+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878009be0 con 0x7f4864004d40 2024-04-25T11:20:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.112+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4878036600 con 0x7f4864004d40 2024-04-25T11:20:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.112+0000 7f487e000640 1 -- 172.21.15.7:0/1554199510 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48780369d0 con 0x7f4864004d40 2024-04-25T11:20:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.112+0000 7f487fe00640 1 -- 172.21.15.7:0/1554199510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4864004d40 msgr2=0x7f4864005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.112+0000 7f487fe00640 1 --2- 172.21.15.7:0/1554199510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4864004d40 0x7f4864005150 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f4878007360 tx=0x7f4864005830 comp rx=0 tx=0).stop 2024-04-25T11:20:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.112+0000 7f487f400640 1 -- 172.21.15.7:0/1554199510 reap_dead start 2024-04-25T11:20:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.112+0000 7f487fe00640 1 -- 172.21.15.7:0/1554199510 shutdown_connections 2024-04-25T11:20:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.112+0000 7f487fe00640 1 -- 172.21.15.7:0/1554199510 >> 172.21.15.7:0/1554199510 conn(0x7f48800740f0 msgr2=0x7f488006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.113+0000 7f487fe00640 1 -- 172.21.15.7:0/1554199510 shutdown_connections 2024-04-25T11:20:30.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.113+0000 7f487fe00640 1 -- 172.21.15.7:0/1554199510 wait complete. 2024-04-25T11:20:30.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.115+0000 7f487fe00640 1 Processor -- start 2024-04-25T11:20:30.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.115+0000 7f487fe00640 1 -- start start 2024-04-25T11:20:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f487fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4864004d40 0x7f48800cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f487fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48800aac10 0x7f48801f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f487fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48801358c0 0x7f4880135cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f487fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48800b63d0 con 0x7f48800aac10 2024-04-25T11:20:30.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f487fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48800b60f0 con 0x7f4864004d40 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f487fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48800b5390 con 0x7f48801358c0 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f4877e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48800aac10 0x7f48801f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f4877e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48800aac10 0x7f48801f58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52474/0 (socket says 172.21.15.7:52474) 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f4877e00640 1 -- 172.21.15.7:0/2874489566 learned_addr learned my addr 172.21.15.7:0/2874489566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f487ea00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4864004d40 0x7f48800cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f4877e00640 1 -- 172.21.15.7:0/2874489566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48801358c0 msgr2=0x7f4880135cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f487f400640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48801358c0 0x7f4880135cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f4877e00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48801358c0 0x7f4880135cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f4877e00640 1 -- 172.21.15.7:0/2874489566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4864004d40 msgr2=0x7f48800cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f4877e00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4864004d40 0x7f48800cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.117+0000 7f4877e00640 1 -- 172.21.15.7:0/2874489566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48800cd860 con 0x7f48800aac10 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.118+0000 7f487ea00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4864004d40 0x7f48800cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.118+0000 7f4877e00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48800aac10 0x7f48801f58a0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f486800f630 tx=0x7f486800faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.118+0000 7f4876a00640 1 -- 172.21.15.7:0/2874489566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868019020 con 0x7f48800aac10 2024-04-25T11:20:30.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.118+0000 7f4876a00640 1 -- 172.21.15.7:0/2874489566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4868014870 con 0x7f48800aac10 2024-04-25T11:20:30.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.118+0000 7f4876a00640 1 -- 172.21.15.7:0/2874489566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868014c20 con 0x7f48800aac10 2024-04-25T11:20:30.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.118+0000 7f487fe00640 1 -- 172.21.15.7:0/2874489566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48800cdb40 con 0x7f48800aac10 2024-04-25T11:20:30.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.118+0000 7f487fe00640 1 -- 172.21.15.7:0/2874489566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48800ce100 con 0x7f48800aac10 2024-04-25T11:20:30.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.120+0000 7f487fe00640 1 -- 172.21.15.7:0/2874489566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48800ce540 con 0x7f48800aac10 2024-04-25T11:20:30.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.121+0000 7f4876a00640 1 -- 172.21.15.7:0/2874489566 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f486801c020 con 0x7f48800aac10 2024-04-25T11:20:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.122+0000 7f4876a00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f485c0744b0 0x7f485c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.122+0000 7f4876a00640 1 -- 172.21.15.7:0/2874489566 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7f4868095a30 con 0x7f48800aac10 2024-04-25T11:20:30.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.122+0000 7f487ea00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f485c0744b0 0x7f485c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.123+0000 7f487ea00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f485c0744b0 0x7f485c0769c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f488006e950 tx=0x7f4870000f70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:30.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.127+0000 7f4876a00640 1 -- 172.21.15.7:0/2874489566 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4868023020 con 0x7f48800aac10 2024-04-25T11:20:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.346+0000 7f487fe00640 1 -- 172.21.15.7:0/2874489566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7f48800cd5f0 con 0x7f48800aac10 2024-04-25T11:20:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.355+0000 7f4876a00640 1 -- 172.21.15.7:0/2874489566 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v42) ==== 107+0+0 (secure 0 0 0) 0x7f4868060290 con 0x7f48800aac10 2024-04-25T11:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 -- 172.21.15.7:0/2874489566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f485c0744b0 msgr2=0x7f485c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f485c0744b0 0x7f485c0769c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f488006e950 tx=0x7f4870000f70 comp rx=0 tx=0).stop 2024-04-25T11:20:30.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 -- 172.21.15.7:0/2874489566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48800aac10 msgr2=0x7f48801f58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48800aac10 0x7f48801f58a0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f486800f630 tx=0x7f486800faf0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 -- 172.21.15.7:0/2874489566 shutdown_connections 2024-04-25T11:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f485c0744b0 0x7f485c0769c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48801358c0 0x7f4880135cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48800aac10 0x7f48801f58a0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 --2- 172.21.15.7:0/2874489566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4864004d40 0x7f48800cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 -- 172.21.15.7:0/2874489566 >> 172.21.15.7:0/2874489566 conn(0x7f48800740f0 msgr2=0x7f48800cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:30.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 -- 172.21.15.7:0/2874489566 shutdown_connections 2024-04-25T11:20:30.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.358+0000 7f487fe00640 1 -- 172.21.15.7:0/2874489566 wait complete. 2024-04-25T11:20:30.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:20:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.650+0000 7fb30a600640 1 Processor -- start 2024-04-25T11:20:30.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.651+0000 7fb30a600640 1 -- start start 2024-04-25T11:20:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.653+0000 7fb30a600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb304149620 0x7fb3041461a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.653+0000 7fb30a600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb304149040 0x7fb3040703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.653+0000 7fb30a600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb3040b01d0 0x7fb3040b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.653+0000 7fb30a600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb30406f330 con 0x7fb304149620 2024-04-25T11:20:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.653+0000 7fb30a600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb30406f610 con 0x7fb3040b01d0 2024-04-25T11:20:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.653+0000 7fb30a600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb30406f8f0 con 0x7fb304149040 2024-04-25T11:20:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.654+0000 7fb309200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb304149620 0x7fb3041461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.654+0000 7fb309200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb304149620 0x7fb3041461a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52484/0 (socket says 172.21.15.7:52484) 2024-04-25T11:20:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.654+0000 7fb309200640 1 -- 172.21.15.7:0/4256183663 learned_addr learned my addr 172.21.15.7:0/4256183663 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.654+0000 7fb309c00640 1 --2- 172.21.15.7:0/4256183663 >> v2:172.21.15.87:3300/0 conn(0x7fb3040b01d0 0x7fb3040b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.654+0000 7fb303e00640 1 --2- 172.21.15.7:0/4256183663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb304149040 0x7fb3040703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.654+0000 7fb309200640 1 -- 172.21.15.7:0/4256183663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb304149040 msgr2=0x7fb3040703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.654+0000 7fb309200640 1 --2- 172.21.15.7:0/4256183663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb304149040 0x7fb3040703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.654+0000 7fb309200640 1 -- 172.21.15.7:0/4256183663 >> v2:172.21.15.87:3300/0 conn(0x7fb3040b01d0 msgr2=0x7fb3040b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.654+0000 7fb309200640 1 --2- 172.21.15.7:0/4256183663 >> v2:172.21.15.87:3300/0 conn(0x7fb3040b01d0 0x7fb3040b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.654+0000 7fb309200640 1 -- 172.21.15.7:0/4256183663 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30406fbd0 con 0x7fb304149620 2024-04-25T11:20:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.656+0000 7fb309200640 1 --2- 172.21.15.7:0/4256183663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb304149620 0x7fb3041461a0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb304072050 tx=0x7fb2f802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdc6262dc0e7222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.656+0000 7fb303400640 1 -- 172.21.15.7:0/4256183663 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8037070 con 0x7fb304149620 2024-04-25T11:20:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.656+0000 7fb303400640 1 -- 172.21.15.7:0/4256183663 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2f8032440 con 0x7fb304149620 2024-04-25T11:20:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.656+0000 7fb303400640 1 -- 172.21.15.7:0/4256183663 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8032810 con 0x7fb304149620 2024-04-25T11:20:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.657+0000 7fb30a600640 1 -- 172.21.15.7:0/4256183663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb304149620 msgr2=0x7fb3041461a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.657+0000 7fb30a600640 1 --2- 172.21.15.7:0/4256183663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb304149620 0x7fb3041461a0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb304072050 tx=0x7fb2f802a140 comp rx=0 tx=0).stop 2024-04-25T11:20:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.657+0000 7fb30a600640 1 -- 172.21.15.7:0/4256183663 shutdown_connections 2024-04-25T11:20:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.657+0000 7fb30a600640 1 --2- 172.21.15.7:0/4256183663 >> v2:172.21.15.87:3300/0 conn(0x7fb3040b01d0 0x7fb3040b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.657+0000 7fb30a600640 1 --2- 172.21.15.7:0/4256183663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb304149040 0x7fb3040703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.657+0000 7fb30a600640 1 --2- 172.21.15.7:0/4256183663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb304149620 0x7fb3041461a0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.657+0000 7fb30a600640 1 -- 172.21.15.7:0/4256183663 >> 172.21.15.7:0/4256183663 conn(0x7fb304074140 msgr2=0x7fb30406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.657+0000 7fb30a600640 1 -- 172.21.15.7:0/4256183663 shutdown_connections 2024-04-25T11:20:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.657+0000 7fb30a600640 1 -- 172.21.15.7:0/4256183663 wait complete. 2024-04-25T11:20:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.659+0000 7fb30a600640 1 Processor -- start 2024-04-25T11:20:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.659+0000 7fb30a600640 1 -- start start 2024-04-25T11:20:30.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.661+0000 7fb30a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3040b01d0 0x7fb30415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.661+0000 7fb30a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb304149040 0x7fb3041ea230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.661+0000 7fb30a600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb304162700 0x7fb304162b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.661+0000 7fb30a600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb30406eea0 con 0x7fb3040b01d0 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.661+0000 7fb30a600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb3040b0b20 con 0x7fb304149040 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.661+0000 7fb30a600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb30406f020 con 0x7fb304162700 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.661+0000 7fb309200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3040b01d0 0x7fb30415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.661+0000 7fb309200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3040b01d0 0x7fb30415f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52488/0 (socket says 172.21.15.7:52488) 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.661+0000 7fb309200640 1 -- 172.21.15.7:0/2056235239 learned_addr learned my addr 172.21.15.7:0/2056235239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.661+0000 7fb303e00640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb304149040 0x7fb3041ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.662+0000 7fb309200640 1 -- 172.21.15.7:0/2056235239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb304162700 msgr2=0x7fb304162b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.662+0000 7fb309200640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb304162700 0x7fb304162b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.662+0000 7fb309200640 1 -- 172.21.15.7:0/2056235239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb304149040 msgr2=0x7fb3041ea230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.662+0000 7fb309200640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb304149040 0x7fb3041ea230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.662+0000 7fb309200640 1 -- 172.21.15.7:0/2056235239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb304163050 con 0x7fb3040b01d0 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.662+0000 7fb309200640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3040b01d0 0x7fb30415f4a0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb2f802a6b0 tx=0x7fb2f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.662+0000 7fb300c00640 1 -- 172.21.15.7:0/2056235239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8041020 con 0x7fb3040b01d0 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.662+0000 7fb300c00640 1 -- 172.21.15.7:0/2056235239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2f8030760 con 0x7fb3040b01d0 2024-04-25T11:20:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.662+0000 7fb300c00640 1 -- 172.21.15.7:0/2056235239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8037040 con 0x7fb3040b01d0 2024-04-25T11:20:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.662+0000 7fb30a600640 1 -- 172.21.15.7:0/2056235239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb304135110 con 0x7fb3040b01d0 2024-04-25T11:20:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.662+0000 7fb30a600640 1 -- 172.21.15.7:0/2056235239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb304160500 con 0x7fb3040b01d0 2024-04-25T11:20:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.663+0000 7fb30a600640 1 -- 172.21.15.7:0/2056235239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb304050740 con 0x7fb3040b01d0 2024-04-25T11:20:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.665+0000 7fb300c00640 1 -- 172.21.15.7:0/2056235239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb2f8044020 con 0x7fb3040b01d0 2024-04-25T11:20:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.666+0000 7fb300c00640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2e80744b0 0x7fb2e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.666+0000 7fb300c00640 1 -- 172.21.15.7:0/2056235239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7fb2f80b58b0 con 0x7fb3040b01d0 2024-04-25T11:20:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.666+0000 7fb303e00640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2e80744b0 0x7fb2e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.667+0000 7fb303e00640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2e80744b0 0x7fb2e80769c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4006fd0 tx=0x7fb2f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:30.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.670+0000 7fb300c00640 1 -- 172.21.15.7:0/2056235239 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb2f804b030 con 0x7fb3040b01d0 2024-04-25T11:20:30.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.869+0000 7fb30a600640 1 -- 172.21.15.7:0/2056235239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb30406d790 con 0x7fb3040b01d0 2024-04-25T11:20:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.877+0000 7fb300c00640 1 -- 172.21.15.7:0/2056235239 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) ==== 104+0+0 (secure 0 0 0) 0x7fb304163050 con 0x7fb3040b01d0 2024-04-25T11:20:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 -- 172.21.15.7:0/2056235239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2e80744b0 msgr2=0x7fb2e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2e80744b0 0x7fb2e80769c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4006fd0 tx=0x7fb2f4008040 comp rx=0 tx=0).stop 2024-04-25T11:20:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 -- 172.21.15.7:0/2056235239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3040b01d0 msgr2=0x7fb30415f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3040b01d0 0x7fb30415f4a0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb2f802a6b0 tx=0x7fb2f80046c0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 -- 172.21.15.7:0/2056235239 shutdown_connections 2024-04-25T11:20:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2e80744b0 0x7fb2e80769c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb304162700 0x7fb304162b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb304149040 0x7fb3041ea230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 --2- 172.21.15.7:0/2056235239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3040b01d0 0x7fb30415f4a0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 -- 172.21.15.7:0/2056235239 >> 172.21.15.7:0/2056235239 conn(0x7fb304074140 msgr2=0x7fb30406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 -- 172.21.15.7:0/2056235239 shutdown_connections 2024-04-25T11:20:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:30.880+0000 7fb30a600640 1 -- 172.21.15.7:0/2056235239 wait complete. 2024-04-25T11:20:30.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-04-25T11:20:31.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.190+0000 7f1744a00640 1 Processor -- start 2024-04-25T11:20:31.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.190+0000 7f1744a00640 1 -- start start 2024-04-25T11:20:31.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.192+0000 7f1744a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17401490f0 0x7f1740149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.193+0000 7f1744a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f174006f280 con 0x7f1740146180 2024-04-25T11:20:31.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.193+0000 7f1744a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f174006f560 con 0x7f17400b01d0 2024-04-25T11:20:31.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.193+0000 7f1744a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1740146590 con 0x7f17401490f0 2024-04-25T11:20:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.193+0000 7f173ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17401490f0 0x7f1740149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.193+0000 7f173ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17401490f0 0x7f1740149500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52502/0 (socket says 172.21.15.7:52502) 2024-04-25T11:20:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.193+0000 7f173f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1740146180 0x7f1740070320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58322/0 (socket says 172.21.15.7:58322) 2024-04-25T11:20:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.193+0000 7f173ea00640 1 -- 172.21.15.7:0/2776122570 learned_addr learned my addr 172.21.15.7:0/2776122570 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.194+0000 7f173ea00640 1 -- 172.21.15.7:0/2776122570 >> v1:172.21.15.87:6789/0 conn(0x7f17400b01d0 legacy=0x7f17400b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:31.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.194+0000 7f173ea00640 1 -- 172.21.15.7:0/2776122570 >> v1:172.21.15.7:6789/0 conn(0x7f1740146180 legacy=0x7f1740070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.194+0000 7f173ea00640 1 -- 172.21.15.7:0/2776122570 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174006f840 con 0x7f17401490f0 2024-04-25T11:20:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.195+0000 7f173ea00640 1 --2- 172.21.15.7:0/2776122570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17401490f0 0x7f1740149500 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f172c009870 tx=0x7f172c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceed569fc3f20d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.195+0000 7f173e000640 1 -- 172.21.15.7:0/2776122570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c037070 con 0x7f17401490f0 2024-04-25T11:20:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.196+0000 7f173e000640 1 -- 172.21.15.7:0/2776122570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172c032440 con 0x7f17401490f0 2024-04-25T11:20:31.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.196+0000 7f173e000640 1 -- 172.21.15.7:0/2776122570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c032810 con 0x7f17401490f0 2024-04-25T11:20:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.196+0000 7f1744a00640 1 -- 172.21.15.7:0/2776122570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17401490f0 msgr2=0x7f1740149500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.196+0000 7f1744a00640 1 --2- 172.21.15.7:0/2776122570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17401490f0 0x7f1740149500 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f172c009870 tx=0x7f172c02a2a0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.197+0000 7f1744a00640 1 -- 172.21.15.7:0/2776122570 shutdown_connections 2024-04-25T11:20:31.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.197+0000 7f1744a00640 1 --2- 172.21.15.7:0/2776122570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17401490f0 0x7f1740149500 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.197+0000 7f1744a00640 1 -- 172.21.15.7:0/2776122570 >> 172.21.15.7:0/2776122570 conn(0x7f17400740f0 msgr2=0x7f174006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:31.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.197+0000 7f1744a00640 1 -- 172.21.15.7:0/2776122570 shutdown_connections 2024-04-25T11:20:31.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.197+0000 7f1744a00640 1 -- 172.21.15.7:0/2776122570 wait complete. 2024-04-25T11:20:31.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.200+0000 7f1744a00640 1 Processor -- start 2024-04-25T11:20:31.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.200+0000 7f1744a00640 1 -- start start 2024-04-25T11:20:31.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.202+0000 7f1744a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17400b01d0 0x7f17401e9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.202+0000 7f1744a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1740146180 0x7f174015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.202+0000 7f1744a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17401490f0 0x7f1740146cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.202+0000 7f1744a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f174006eea0 con 0x7f17401490f0 2024-04-25T11:20:31.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.202+0000 7f1744a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f174006fe00 con 0x7f17400b01d0 2024-04-25T11:20:31.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.202+0000 7f1744a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f174006f020 con 0x7f1740146180 2024-04-25T11:20:31.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.202+0000 7f173fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17401490f0 0x7f1740146cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.202+0000 7f173fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17401490f0 0x7f1740146cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52514/0 (socket says 172.21.15.7:52514) 2024-04-25T11:20:31.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.202+0000 7f173fe00640 1 -- 172.21.15.7:0/2961979476 learned_addr learned my addr 172.21.15.7:0/2961979476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:31.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.203+0000 7f173ea00640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1740146180 0x7f174015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.203+0000 7f173f400640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17400b01d0 0x7f17401e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.203+0000 7f173fe00640 1 -- 172.21.15.7:0/2961979476 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1740146180 msgr2=0x7f174015f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.203+0000 7f173fe00640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1740146180 0x7f174015f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.203+0000 7f173fe00640 1 -- 172.21.15.7:0/2961979476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17400b01d0 msgr2=0x7f17401e9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.203+0000 7f173fe00640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17400b01d0 0x7f17401e9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.203+0000 7f173fe00640 1 -- 172.21.15.7:0/2961979476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17400ce2d0 con 0x7f17401490f0 2024-04-25T11:20:31.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.203+0000 7f173fe00640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17401490f0 0x7f1740146cb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f173800be60 tx=0x7f173800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:31.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.204+0000 7f1737400640 1 -- 172.21.15.7:0/2961979476 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738019070 con 0x7f17401490f0 2024-04-25T11:20:31.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.204+0000 7f1737400640 1 -- 172.21.15.7:0/2961979476 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1738014440 con 0x7f17401490f0 2024-04-25T11:20:31.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.204+0000 7f1737400640 1 -- 172.21.15.7:0/2961979476 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738014830 con 0x7f17401490f0 2024-04-25T11:20:31.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.204+0000 7f1744a00640 1 -- 172.21.15.7:0/2961979476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174015fc20 con 0x7f17401490f0 2024-04-25T11:20:31.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.204+0000 7f1744a00640 1 -- 172.21.15.7:0/2961979476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17401ea4d0 con 0x7f17401490f0 2024-04-25T11:20:31.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.205+0000 7f1744a00640 1 -- 172.21.15.7:0/2961979476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17401e9f60 con 0x7f17401490f0 2024-04-25T11:20:31.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.206+0000 7f1737400640 1 -- 172.21.15.7:0/2961979476 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f173801c070 con 0x7f17401490f0 2024-04-25T11:20:31.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.208+0000 7f1737400640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17200744b0 0x7f17200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.208+0000 7f1737400640 1 -- 172.21.15.7:0/2961979476 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7f17380950e0 con 0x7f17401490f0 2024-04-25T11:20:31.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.208+0000 7f173f400640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17200744b0 0x7f17200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.209+0000 7f173f400640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17200744b0 0x7f17200769c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1730000f60 tx=0x7f1730009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:31.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.213+0000 7f1737400640 1 -- 172.21.15.7:0/2961979476 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f173805f9d0 con 0x7f17401490f0 2024-04-25T11:20:31.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.417+0000 7f1744a00640 1 -- 172.21.15.7:0/2961979476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7f17400ce4a0 con 0x7f17401490f0 2024-04-25T11:20:31.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.425+0000 7f1737400640 1 -- 172.21.15.7:0/2961979476 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v44) ==== 111+0+0 (secure 0 0 0) 0x7f173805f2d0 con 0x7f17401490f0 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.428+0000 7f1744a00640 1 -- 172.21.15.7:0/2961979476 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17200744b0 msgr2=0x7f17200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.428+0000 7f1744a00640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17200744b0 0x7f17200769c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1730000f60 tx=0x7f1730009040 comp rx=0 tx=0).stop 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.428+0000 7f1744a00640 1 -- 172.21.15.7:0/2961979476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17401490f0 msgr2=0x7f1740146cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.428+0000 7f1744a00640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17401490f0 0x7f1740146cb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f173800be60 tx=0x7f173800bf50 comp rx=0 tx=0).stop 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.429+0000 7f1744a00640 1 -- 172.21.15.7:0/2961979476 shutdown_connections 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.429+0000 7f1744a00640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17200744b0 0x7f17200769c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.429+0000 7f1744a00640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17401490f0 0x7f1740146cb0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.429+0000 7f1744a00640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1740146180 0x7f174015f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.429+0000 7f1744a00640 1 --2- 172.21.15.7:0/2961979476 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17400b01d0 0x7f17401e9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.429+0000 7f1744a00640 1 -- 172.21.15.7:0/2961979476 >> 172.21.15.7:0/2961979476 conn(0x7f17400740f0 msgr2=0x7f1740143e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.429+0000 7f1744a00640 1 -- 172.21.15.7:0/2961979476 shutdown_connections 2024-04-25T11:20:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.429+0000 7f1744a00640 1 -- 172.21.15.7:0/2961979476 wait complete. 2024-04-25T11:20:31.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-04-25T11:20:31.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.741+0000 7fd927e00640 1 Processor -- start 2024-04-25T11:20:31.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.741+0000 7fd927e00640 1 -- start start 2024-04-25T11:20:31.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.744+0000 7fd927e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd928145e40 0x7fd928146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.744+0000 7fd927e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9280b4870 0x7fd9280b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.744+0000 7fd927e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9280b66b0 con 0x7fd9280b51c0 2024-04-25T11:20:31.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.744+0000 7fd927e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd928145b70 con 0x7fd9280b4870 2024-04-25T11:20:31.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.744+0000 7fd927e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd9280b5b30 con 0x7fd928145e40 2024-04-25T11:20:31.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.744+0000 7fd926000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9280b4870 0x7fd9280b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.745+0000 7fd927400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd9280b51c0 0x7fd928143a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58338/0 (socket says 172.21.15.7:58338) 2024-04-25T11:20:31.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.745+0000 7fd927400640 1 -- 172.21.15.7:0/2723104190 learned_addr learned my addr 172.21.15.7:0/2723104190 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:31.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.745+0000 7fd926a00640 1 --2- 172.21.15.7:0/2723104190 >> v2:172.21.15.87:3300/0 conn(0x7fd928145e40 0x7fd928146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.745+0000 7fd926000640 1 -- 172.21.15.7:0/2723104190 >> v2:172.21.15.87:3300/0 conn(0x7fd928145e40 msgr2=0x7fd928146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:31.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.745+0000 7fd926000640 1 --2- 172.21.15.7:0/2723104190 >> v2:172.21.15.87:3300/0 conn(0x7fd928145e40 0x7fd928146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.745+0000 7fd926000640 1 -- 172.21.15.7:0/2723104190 >> v1:172.21.15.7:6789/0 conn(0x7fd9280b51c0 legacy=0x7fd928143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.745+0000 7fd926000640 1 -- 172.21.15.7:0/2723104190 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9280b5e10 con 0x7fd9280b4870 2024-04-25T11:20:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.746+0000 7fd926000640 1 --2- 172.21.15.7:0/2723104190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9280b4870 0x7fd9280b4c80 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd92806ded0 tx=0x7fd91402c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900280fde8bda2af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.747+0000 7fd925600640 1 -- 172.21.15.7:0/2723104190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914039070 con 0x7fd9280b4870 2024-04-25T11:20:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.747+0000 7fd925600640 1 -- 172.21.15.7:0/2723104190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd914034640 con 0x7fd9280b4870 2024-04-25T11:20:31.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.747+0000 7fd925600640 1 -- 172.21.15.7:0/2723104190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914034a10 con 0x7fd9280b4870 2024-04-25T11:20:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.747+0000 7fd927e00640 1 -- 172.21.15.7:0/2723104190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9280b4870 msgr2=0x7fd9280b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.747+0000 7fd927e00640 1 --2- 172.21.15.7:0/2723104190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9280b4870 0x7fd9280b4c80 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd92806ded0 tx=0x7fd91402c320 comp rx=0 tx=0).stop 2024-04-25T11:20:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.747+0000 7fd927e00640 1 -- 172.21.15.7:0/2723104190 shutdown_connections 2024-04-25T11:20:31.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.747+0000 7fd927e00640 1 --2- 172.21.15.7:0/2723104190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9280b4870 0x7fd9280b4c80 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.747+0000 7fd927e00640 1 --2- 172.21.15.7:0/2723104190 >> v2:172.21.15.87:3300/0 conn(0x7fd928145e40 0x7fd928146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.747+0000 7fd927e00640 1 -- 172.21.15.7:0/2723104190 >> 172.21.15.7:0/2723104190 conn(0x7fd9280740f0 msgr2=0x7fd92806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.748+0000 7fd927e00640 1 -- 172.21.15.7:0/2723104190 shutdown_connections 2024-04-25T11:20:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.748+0000 7fd927e00640 1 -- 172.21.15.7:0/2723104190 wait complete. 2024-04-25T11:20:31.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.750+0000 7fd927e00640 1 Processor -- start 2024-04-25T11:20:31.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.750+0000 7fd927e00640 1 -- start start 2024-04-25T11:20:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.751+0000 7fd927e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9280b4870 0x7fd9280c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.751+0000 7fd927e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9280b51c0 0x7fd92813eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.751+0000 7fd927e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd928145e40 0x7fd928141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.751+0000 7fd927e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9280b55d0 con 0x7fd9280b4870 2024-04-25T11:20:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.751+0000 7fd927e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd9280b63d0 con 0x7fd928145e40 2024-04-25T11:20:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.751+0000 7fd927e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9280b5750 con 0x7fd9280b51c0 2024-04-25T11:20:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.752+0000 7fd926a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9280b4870 0x7fd9280c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.752+0000 7fd926a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9280b4870 0x7fd9280c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52532/0 (socket says 172.21.15.7:52532) 2024-04-25T11:20:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.752+0000 7fd926a00640 1 -- 172.21.15.7:0/1216243954 learned_addr learned my addr 172.21.15.7:0/1216243954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.752+0000 7fd926a00640 1 -- 172.21.15.7:0/1216243954 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9280b51c0 msgr2=0x7fd92813eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.752+0000 7fd926a00640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9280b51c0 0x7fd92813eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.752+0000 7fd926a00640 1 -- 172.21.15.7:0/1216243954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd928145e40 msgr2=0x7fd928141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.752+0000 7fd926a00640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd928145e40 0x7fd928141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.752+0000 7fd926a00640 1 -- 172.21.15.7:0/1216243954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9280c9830 con 0x7fd9280b4870 2024-04-25T11:20:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.753+0000 7fd926a00640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9280b4870 0x7fd9280c8c50 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd91800d5f0 tx=0x7fd91800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.753+0000 7fd91ea00640 1 -- 172.21.15.7:0/1216243954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9180125f0 con 0x7fd9280b4870 2024-04-25T11:20:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.753+0000 7fd91ea00640 1 -- 172.21.15.7:0/1216243954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd918019070 con 0x7fd9280b4870 2024-04-25T11:20:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.753+0000 7fd927e00640 1 -- 172.21.15.7:0/1216243954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9280c9b10 con 0x7fd9280b4870 2024-04-25T11:20:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.753+0000 7fd91ea00640 1 -- 172.21.15.7:0/1216243954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9180129d0 con 0x7fd9280b4870 2024-04-25T11:20:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.753+0000 7fd927e00640 1 -- 172.21.15.7:0/1216243954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9280c9190 con 0x7fd9280b4870 2024-04-25T11:20:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.754+0000 7fd927e00640 1 -- 172.21.15.7:0/1216243954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd928050740 con 0x7fd9280b4870 2024-04-25T11:20:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.755+0000 7fd91ea00640 1 -- 172.21.15.7:0/1216243954 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd918012b70 con 0x7fd9280b4870 2024-04-25T11:20:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.757+0000 7fd91ea00640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9080744b0 0x7fd9080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.757+0000 7fd91ea00640 1 -- 172.21.15.7:0/1216243954 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7fd9180955f0 con 0x7fd9280b4870 2024-04-25T11:20:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.757+0000 7fd926000640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9080744b0 0x7fd9080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.758+0000 7fd926000640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9080744b0 0x7fd9080769c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd914036000 tx=0x7fd91402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.762+0000 7fd91ea00640 1 -- 172.21.15.7:0/1216243954 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd91805fe50 con 0x7fd9280b4870 2024-04-25T11:20:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.968+0000 7fd927e00640 1 -- 172.21.15.7:0/1216243954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7fd928158ea0 con 0x7fd9280b4870 2024-04-25T11:20:31.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.977+0000 7fd91ea00640 1 -- 172.21.15.7:0/1216243954 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v45) ==== 105+0+0 (secure 0 0 0) 0x7fd91805f750 con 0x7fd9280b4870 2024-04-25T11:20:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.979+0000 7fd927e00640 1 -- 172.21.15.7:0/1216243954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9080744b0 msgr2=0x7fd9080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.979+0000 7fd927e00640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9080744b0 0x7fd9080769c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd914036000 tx=0x7fd91402e000 comp rx=0 tx=0).stop 2024-04-25T11:20:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.979+0000 7fd927e00640 1 -- 172.21.15.7:0/1216243954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9280b4870 msgr2=0x7fd9280c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:31.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.979+0000 7fd927e00640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9280b4870 0x7fd9280c8c50 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd91800d5f0 tx=0x7fd91800dab0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.979+0000 7fd927e00640 1 -- 172.21.15.7:0/1216243954 shutdown_connections 2024-04-25T11:20:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.979+0000 7fd927e00640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9080744b0 0x7fd9080769c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.979+0000 7fd927e00640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd928145e40 0x7fd928141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.979+0000 7fd927e00640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9280b51c0 0x7fd92813eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.979+0000 7fd927e00640 1 --2- 172.21.15.7:0/1216243954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9280b4870 0x7fd9280c8c50 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.980+0000 7fd927e00640 1 -- 172.21.15.7:0/1216243954 >> 172.21.15.7:0/1216243954 conn(0x7fd9280740f0 msgr2=0x7fd9280c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.980+0000 7fd927e00640 1 -- 172.21.15.7:0/1216243954 shutdown_connections 2024-04-25T11:20:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:31.980+0000 7fd927e00640 1 -- 172.21.15.7:0/1216243954 wait complete. 2024-04-25T11:20:32.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin' 2024-04-25T11:20:32.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.277+0000 7ff5af000640 1 Processor -- start 2024-04-25T11:20:32.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.278+0000 7ff5af000640 1 -- start start 2024-04-25T11:20:32.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.280+0000 7ff5af000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5a815f180 0x7ff5a8161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.280+0000 7ff5af000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5a806dc90 0x7ff5a806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.280+0000 7ff5af000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a8162700 con 0x7ff5a81621a0 2024-04-25T11:20:32.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.280+0000 7ff5af000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a815f590 con 0x7ff5a815f180 2024-04-25T11:20:32.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.280+0000 7ff5af000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a8158ae0 con 0x7ff5a806dc90 2024-04-25T11:20:32.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.281+0000 7ff5ad000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5a806dc90 0x7ff5a806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.281+0000 7ff5a7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5a815f180 0x7ff5a8161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.281+0000 7ff5a7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5a815f180 0x7ff5a8161c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:34952/0 (socket says 172.21.15.7:34952) 2024-04-25T11:20:32.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.281+0000 7ff5a7e00640 1 -- 172.21.15.7:0/3887258098 learned_addr learned my addr 172.21.15.7:0/3887258098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:32.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.282+0000 7ff5a7400640 1 -- 172.21.15.7:0/3887258098 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1462705273 0 0) 0x7ff5a8162700 con 0x7ff5a81621a0 2024-04-25T11:20:32.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.282+0000 7ff5a7400640 1 -- 172.21.15.7:0/3887258098 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff58c003520 con 0x7ff5a81621a0 2024-04-25T11:20:32.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.282+0000 7ff5a7e00640 1 -- 172.21.15.7:0/3887258098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5a806dc90 msgr2=0x7ff5a806f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:32.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.282+0000 7ff5a7e00640 1 --2- 172.21.15.7:0/3887258098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5a806dc90 0x7ff5a806f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.282+0000 7ff5a7e00640 1 -- 172.21.15.7:0/3887258098 >> v1:172.21.15.87:6789/0 conn(0x7ff5a81621a0 legacy=0x7ff5a806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.282+0000 7ff5a7e00640 1 -- 172.21.15.7:0/3887258098 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a81629e0 con 0x7ff5a815f180 2024-04-25T11:20:32.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.283+0000 7ff5a7e00640 1 --2- 172.21.15.7:0/3887258098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5a815f180 0x7ff5a8161c60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff598009870 tx=0x7ff59802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14850af68311a574 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:32.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.284+0000 7ff5a7400640 1 -- 172.21.15.7:0/3887258098 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598037070 con 0x7ff5a815f180 2024-04-25T11:20:32.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.284+0000 7ff5a7400640 1 -- 172.21.15.7:0/3887258098 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff598032440 con 0x7ff5a815f180 2024-04-25T11:20:32.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.284+0000 7ff5a7400640 1 -- 172.21.15.7:0/3887258098 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598032810 con 0x7ff5a815f180 2024-04-25T11:20:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.285+0000 7ff5af000640 1 -- 172.21.15.7:0/3887258098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5a815f180 msgr2=0x7ff5a8161c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.285+0000 7ff5af000640 1 --2- 172.21.15.7:0/3887258098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5a815f180 0x7ff5a8161c60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff598009870 tx=0x7ff59802a270 comp rx=0 tx=0).stop 2024-04-25T11:20:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.285+0000 7ff5af000640 1 -- 172.21.15.7:0/3887258098 shutdown_connections 2024-04-25T11:20:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.285+0000 7ff5af000640 1 --2- 172.21.15.7:0/3887258098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5a806dc90 0x7ff5a806f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.285+0000 7ff5af000640 1 --2- 172.21.15.7:0/3887258098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5a815f180 0x7ff5a8161c60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.285+0000 7ff5af000640 1 -- 172.21.15.7:0/3887258098 >> 172.21.15.7:0/3887258098 conn(0x7ff5a80732b0 msgr2=0x7ff5a80736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.285+0000 7ff5af000640 1 -- 172.21.15.7:0/3887258098 shutdown_connections 2024-04-25T11:20:32.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.285+0000 7ff5af000640 1 -- 172.21.15.7:0/3887258098 wait complete. 2024-04-25T11:20:32.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.287+0000 7ff5af000640 1 Processor -- start 2024-04-25T11:20:32.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.288+0000 7ff5af000640 1 -- start start 2024-04-25T11:20:32.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.289+0000 7ff5af000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a815f180 0x7ff5a81511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.289+0000 7ff5af000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5a81621a0 0x7ff5a8151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.289+0000 7ff5af000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5a8152200 0x7ff5a815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.289+0000 7ff5af000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a815e670 con 0x7ff5a815f180 2024-04-25T11:20:32.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.289+0000 7ff5af000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a815ec60 con 0x7ff5a8152200 2024-04-25T11:20:32.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.289+0000 7ff5af000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a80d1310 con 0x7ff5a81621a0 2024-04-25T11:20:32.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.290+0000 7ff5a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a815f180 0x7ff5a81511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.290+0000 7ff5a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a815f180 0x7ff5a81511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52534/0 (socket says 172.21.15.7:52534) 2024-04-25T11:20:32.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.290+0000 7ff5a7e00640 1 -- 172.21.15.7:0/3376103624 learned_addr learned my addr 172.21.15.7:0/3376103624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:32.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.290+0000 7ff5ad000640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5a8152200 0x7ff5a815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.290+0000 7ff59f400640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5a81621a0 0x7ff5a8151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.290+0000 7ff5a7e00640 1 -- 172.21.15.7:0/3376103624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5a81621a0 msgr2=0x7ff5a8151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.290+0000 7ff5a7e00640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5a81621a0 0x7ff5a8151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.290+0000 7ff5a7e00640 1 -- 172.21.15.7:0/3376103624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5a8152200 msgr2=0x7ff5a815d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:32.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.290+0000 7ff5a7e00640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5a8152200 0x7ff5a815d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:32.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.290+0000 7ff5a7e00640 1 -- 172.21.15.7:0/3376103624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a8152940 con 0x7ff5a815f180 2024-04-25T11:20:32.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.290+0000 7ff59f400640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5a81621a0 0x7ff5a8151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:32.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.291+0000 7ff5a7e00640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a815f180 0x7ff5a81511a0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff598009990 tx=0x7ff5980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:32.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.291+0000 7ff5a4c00640 1 -- 172.21.15.7:0/3376103624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598041020 con 0x7ff5a815f180 2024-04-25T11:20:32.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.291+0000 7ff5a4c00640 1 -- 172.21.15.7:0/3376103624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59803d6e0 con 0x7ff5a815f180 2024-04-25T11:20:32.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.291+0000 7ff5a4c00640 1 -- 172.21.15.7:0/3376103624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598037040 con 0x7ff5a815f180 2024-04-25T11:20:32.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.291+0000 7ff5af000640 1 -- 172.21.15.7:0/3376103624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a8152c20 con 0x7ff5a815f180 2024-04-25T11:20:32.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.291+0000 7ff5af000640 1 -- 172.21.15.7:0/3376103624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a80b4b40 con 0x7ff5a815f180 2024-04-25T11:20:32.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.292+0000 7ff5af000640 1 -- 172.21.15.7:0/3376103624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a8153140 con 0x7ff5a815f180 2024-04-25T11:20:32.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.294+0000 7ff5a4c00640 1 -- 172.21.15.7:0/3376103624 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff598044020 con 0x7ff5a815f180 2024-04-25T11:20:32.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.295+0000 7ff5a4c00640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5840744e0 0x7ff5840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:32.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.295+0000 7ff5a4c00640 1 -- 172.21.15.7:0/3376103624 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7ff5980b5790 con 0x7ff5a815f180 2024-04-25T11:20:32.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.295+0000 7ff59f400640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5840744e0 0x7ff5840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:32.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.296+0000 7ff59f400640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5840744e0 0x7ff5840769f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80b1270 tx=0x7ff590009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:32.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.300+0000 7ff5a4c00640 1 -- 172.21.15.7:0/3376103624 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff5a8153140 con 0x7ff5a815f180 2024-04-25T11:20:32.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:32.582+0000 7ff5af000640 1 -- 172.21.15.7:0/3376103624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]} v 0) -- 0x7ff5a8050740 con 0x7ff5a815f180 2024-04-25T11:20:33.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.257+0000 7ff5a4c00640 1 -- 172.21.15.7:0/3376103624 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin"]}]=0 v524) ==== 143+0+0 (secure 0 0 0) 0x7ff59804b030 con 0x7ff5a815f180 2024-04-25T11:20:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.261+0000 7ff5af000640 1 -- 172.21.15.7:0/3376103624 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5840744e0 msgr2=0x7ff5840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.261+0000 7ff5af000640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5840744e0 0x7ff5840769f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80b1270 tx=0x7ff590009040 comp rx=0 tx=0).stop 2024-04-25T11:20:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.261+0000 7ff5af000640 1 -- 172.21.15.7:0/3376103624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a815f180 msgr2=0x7ff5a81511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.261+0000 7ff5af000640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a815f180 0x7ff5a81511a0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff598009990 tx=0x7ff5980046c0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.261+0000 7ff5af000640 1 -- 172.21.15.7:0/3376103624 shutdown_connections 2024-04-25T11:20:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.261+0000 7ff5af000640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff5840744e0 0x7ff5840769f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.261+0000 7ff5af000640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5a8152200 0x7ff5a815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.261+0000 7ff5af000640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5a81621a0 0x7ff5a8151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.261+0000 7ff5af000640 1 --2- 172.21.15.7:0/3376103624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5a815f180 0x7ff5a81511a0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.261+0000 7ff5af000640 1 -- 172.21.15.7:0/3376103624 >> 172.21.15.7:0/3376103624 conn(0x7ff5a80732b0 msgr2=0x7ff5a8157eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:33.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.262+0000 7ff5af000640 1 -- 172.21.15.7:0/3376103624 shutdown_connections 2024-04-25T11:20:33.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.262+0000 7ff5af000640 1 -- 172.21.15.7:0/3376103624 wait complete. 2024-04-25T11:20:33.294 INFO:tasks.cephfs_test_runner:test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-04-25T11:20:33.294 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-25T11:20:33.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.581+0000 7f6126400640 1 Processor -- start 2024-04-25T11:20:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.581+0000 7f6126400640 1 -- start start 2024-04-25T11:20:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.583+0000 7f6126400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f61200aed40 0x7f612015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.583+0000 7f6126400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61200b5220 0x7f61200ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.583+0000 7f6126400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f61200af420 0x7f61200af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.583+0000 7f6126400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f612015e1f0 con 0x7f61200aed40 2024-04-25T11:20:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.583+0000 7f6126400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f61200afd70 con 0x7f61200af420 2024-04-25T11:20:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.583+0000 7f6126400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f61200b64b0 con 0x7f61200b5220 2024-04-25T11:20:33.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.583+0000 7f6125000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f61200aed40 0x7f612015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.584+0000 7f6125000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f61200aed40 0x7f612015dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52542/0 (socket says 172.21.15.7:52542) 2024-04-25T11:20:33.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.584+0000 7f6125000640 1 -- 172.21.15.7:0/2756647590 learned_addr learned my addr 172.21.15.7:0/2756647590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:33.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.584+0000 7f6125000640 1 -- 172.21.15.7:0/2756647590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61200b5220 msgr2=0x7f61200ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.584+0000 7f6125000640 1 --2- 172.21.15.7:0/2756647590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61200b5220 0x7f61200ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.584+0000 7f6125000640 1 -- 172.21.15.7:0/2756647590 >> v2:172.21.15.87:3300/0 conn(0x7f61200af420 msgr2=0x7f61200af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.584+0000 7f6125a00640 1 --2- 172.21.15.7:0/2756647590 >> v2:172.21.15.87:3300/0 conn(0x7f61200af420 0x7f61200af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.584+0000 7f6125000640 1 --2- 172.21.15.7:0/2756647590 >> v2:172.21.15.87:3300/0 conn(0x7f61200af420 0x7f61200af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.584+0000 7f6125000640 1 -- 172.21.15.7:0/2756647590 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612006c610 con 0x7f61200aed40 2024-04-25T11:20:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.584+0000 7f6125a00640 1 --2- 172.21.15.7:0/2756647590 >> v2:172.21.15.87:3300/0 conn(0x7f61200af420 0x7f61200af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.585+0000 7f6125000640 1 --2- 172.21.15.7:0/2756647590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61200aed40 0x7f612015dcb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f6120073080 tx=0x7f610802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2999e362c5d7cff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.586+0000 7f611fe00640 1 -- 172.21.15.7:0/2756647590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108037070 con 0x7f61200aed40 2024-04-25T11:20:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.586+0000 7f611fe00640 1 -- 172.21.15.7:0/2756647590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6108032440 con 0x7f61200aed40 2024-04-25T11:20:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.586+0000 7f611fe00640 1 -- 172.21.15.7:0/2756647590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108032810 con 0x7f61200aed40 2024-04-25T11:20:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.586+0000 7f6126400640 1 -- 172.21.15.7:0/2756647590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61200aed40 msgr2=0x7f612015dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.586+0000 7f6126400640 1 --2- 172.21.15.7:0/2756647590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61200aed40 0x7f612015dcb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f6120073080 tx=0x7f610802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.587+0000 7f6126400640 1 -- 172.21.15.7:0/2756647590 shutdown_connections 2024-04-25T11:20:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.587+0000 7f6126400640 1 --2- 172.21.15.7:0/2756647590 >> v2:172.21.15.87:3300/0 conn(0x7f61200af420 0x7f61200af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.587+0000 7f6126400640 1 --2- 172.21.15.7:0/2756647590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61200b5220 0x7f61200ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.587+0000 7f6126400640 1 --2- 172.21.15.7:0/2756647590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61200aed40 0x7f612015dcb0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.587+0000 7f6126400640 1 -- 172.21.15.7:0/2756647590 >> 172.21.15.7:0/2756647590 conn(0x7f61200b3820 msgr2=0x7f61200b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.587+0000 7f6126400640 1 -- 172.21.15.7:0/2756647590 shutdown_connections 2024-04-25T11:20:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.587+0000 7f6126400640 1 -- 172.21.15.7:0/2756647590 wait complete. 2024-04-25T11:20:33.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.589+0000 7f6126400640 1 Processor -- start 2024-04-25T11:20:33.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.589+0000 7f6126400640 1 -- start start 2024-04-25T11:20:33.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.591+0000 7f6126400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61200aed40 0x7f612015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.591+0000 7f6126400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61200af420 0x7f6120158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.591+0000 7f6126400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61200b5220 0x7f6120159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.591+0000 7f6126400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f612006ceb0 con 0x7f61200b5220 2024-04-25T11:20:33.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.591+0000 7f6126400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f612006cbd0 con 0x7f61200af420 2024-04-25T11:20:33.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.591+0000 7f6126400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f61200b6110 con 0x7f61200aed40 2024-04-25T11:20:33.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.591+0000 7f6125a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61200b5220 0x7f6120159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.591+0000 7f6125a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61200b5220 0x7f6120159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52554/0 (socket says 172.21.15.7:52554) 2024-04-25T11:20:33.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.591+0000 7f6125a00640 1 -- 172.21.15.7:0/1903701143 learned_addr learned my addr 172.21.15.7:0/1903701143 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f6117e00640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61200af420 0x7f6120158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f6125000640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61200aed40 0x7f612015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f6125a00640 1 -- 172.21.15.7:0/1903701143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61200aed40 msgr2=0x7f612015e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f6125a00640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61200aed40 0x7f612015e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f6125a00640 1 -- 172.21.15.7:0/1903701143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61200af420 msgr2=0x7f6120158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f6125a00640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61200af420 0x7f6120158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f6125a00640 1 -- 172.21.15.7:0/1903701143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6120159880 con 0x7f61200b5220 2024-04-25T11:20:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f6117e00640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61200af420 0x7f6120158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f6125a00640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61200b5220 0x7f6120159340 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f611800cd80 tx=0x7f611800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f611d600640 1 -- 172.21.15.7:0/1903701143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611800f680 con 0x7f61200b5220 2024-04-25T11:20:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f611d600640 1 -- 172.21.15.7:0/1903701143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611800f840 con 0x7f61200b5220 2024-04-25T11:20:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f6126400640 1 -- 172.21.15.7:0/1903701143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612006f220 con 0x7f61200b5220 2024-04-25T11:20:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.592+0000 7f611d600640 1 -- 172.21.15.7:0/1903701143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611800fc30 con 0x7f61200b5220 2024-04-25T11:20:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.593+0000 7f6126400640 1 -- 172.21.15.7:0/1903701143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612006f7e0 con 0x7f61200b5220 2024-04-25T11:20:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.594+0000 7f6126400640 1 -- 172.21.15.7:0/1903701143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6120050740 con 0x7f61200b5220 2024-04-25T11:20:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.595+0000 7f611d600640 1 -- 172.21.15.7:0/1903701143 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f61180213f0 con 0x7f61200b5220 2024-04-25T11:20:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.596+0000 7f611d600640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f60fc0744b0 0x7f60fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.597+0000 7f611d600640 1 -- 172.21.15.7:0/1903701143 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7f6118095de0 con 0x7f61200b5220 2024-04-25T11:20:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.597+0000 7f6125000640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f60fc0744b0 0x7f60fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.598+0000 7f6125000640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f60fc0744b0 0x7f60fc0769c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6108034000 tx=0x7f610802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.601+0000 7f611d600640 1 -- 172.21.15.7:0/1903701143 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6118014030 con 0x7f61200b5220 2024-04-25T11:20:33.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.807+0000 7f6126400640 1 -- 172.21.15.7:0/1903701143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f612015ea00 con 0x7f61200b5220 2024-04-25T11:20:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.809+0000 7f611d600640 1 -- 172.21.15.7:0/1903701143 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+13207 (secure 0 0 0) 0x7f61180606d0 con 0x7f61200b5220 2024-04-25T11:20:33.806 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:33.807 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":121,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:20:29.222758+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6834/1924056839":"2024-04-26T11:20:25.662212+0000","172.21.15.87:6832/1924056839":"2024-04-26T11:20:25.662212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:33.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.812+0000 7f6126400640 1 -- 172.21.15.7:0/1903701143 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f60fc0744b0 msgr2=0x7f60fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.812+0000 7f6126400640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f60fc0744b0 0x7f60fc0769c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6108034000 tx=0x7f610802e000 comp rx=0 tx=0).stop 2024-04-25T11:20:33.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.812+0000 7f6126400640 1 -- 172.21.15.7:0/1903701143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61200b5220 msgr2=0x7f6120159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:33.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.812+0000 7f6126400640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61200b5220 0x7f6120159340 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f611800cd80 tx=0x7f611800a650 comp rx=0 tx=0).stop 2024-04-25T11:20:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.812+0000 7f6126400640 1 -- 172.21.15.7:0/1903701143 shutdown_connections 2024-04-25T11:20:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.812+0000 7f6126400640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f60fc0744b0 0x7f60fc0769c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.812+0000 7f6126400640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61200b5220 0x7f6120159340 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.812+0000 7f6126400640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61200af420 0x7f6120158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.812+0000 7f6126400640 1 --2- 172.21.15.7:0/1903701143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61200aed40 0x7f612015e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.812+0000 7f6126400640 1 -- 172.21.15.7:0/1903701143 >> 172.21.15.7:0/1903701143 conn(0x7f61200b3820 msgr2=0x7f61200b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.813+0000 7f6126400640 1 -- 172.21.15.7:0/1903701143 shutdown_connections 2024-04-25T11:20:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:33.813+0000 7f6126400640 1 -- 172.21.15.7:0/1903701143 wait complete. 2024-04-25T11:20:33.836 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:33.836 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:34.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.128+0000 7f97db200640 1 Processor -- start 2024-04-25T11:20:34.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.128+0000 7f97db200640 1 -- start start 2024-04-25T11:20:34.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.130+0000 7f97db200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41583c0 0x7f97d40cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.130+0000 7f97db200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f97d40cfb50 0x7f97d40d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.131+0000 7f97db200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97d40ce570 con 0x7f97d40d09c0 2024-04-25T11:20:34.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.131+0000 7f97db200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f97d415f190 con 0x7f97d40cfb50 2024-04-25T11:20:34.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.131+0000 7f97db200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f97d40ce850 con 0x7f97d41583c0 2024-04-25T11:20:34.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.131+0000 7f97da800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f97d40d09c0 0x7f97d406eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58340/0 (socket says 172.21.15.7:58340) 2024-04-25T11:20:34.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.131+0000 7f97da800640 1 -- 172.21.15.7:0/3514094390 learned_addr learned my addr 172.21.15.7:0/3514094390 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.131+0000 7f97d9e00640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41583c0 0x7f97d40cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.131+0000 7f97d9400640 1 --2- 172.21.15.7:0/3514094390 >> v2:172.21.15.87:3300/0 conn(0x7f97d40cfb50 0x7f97d40d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.131+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 749579159 0 0) 0x7f97d40ce570 con 0x7f97d40d09c0 2024-04-25T11:20:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.131+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97c0003520 con 0x7f97d40d09c0 2024-04-25T11:20:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 952604800 0 0) 0x7f97c0003520 con 0x7f97d40d09c0 2024-04-25T11:20:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41583c0 msgr2=0x7f97d40cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41583c0 0x7f97d40cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 >> v2:172.21.15.87:3300/0 conn(0x7f97d40cfb50 msgr2=0x7f97d40d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 --2- 172.21.15.7:0/3514094390 >> v2:172.21.15.87:3300/0 conn(0x7f97d40cfb50 0x7f97d40d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d40ceb30 con 0x7f97d40d09c0 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f97d0001670 con 0x7f97d40d09c0 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 >> v1:172.21.15.7:6789/0 conn(0x7f97d40d09c0 legacy=0x7f97d406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 0x7f97c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97c0004800 0x7f97c0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41583c0 0x7f97c0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f97d415f190 con 0x7f97c00039b0 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f97d40ce850 con 0x7f97c0004800 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f97c000b9c0 con 0x7f97d41583c0 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.132+0000 7f97d9e00640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 0x7f97c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.133+0000 7f97da800640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41583c0 0x7f97c0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.133+0000 7f97d9400640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97c0004800 0x7f97c0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.133+0000 7f97d9e00640 1 -- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41583c0 msgr2=0x7f97c0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.133+0000 7f97d9e00640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41583c0 0x7f97c0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.133+0000 7f97d9e00640 1 -- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97c0004800 msgr2=0x7f97c0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:34.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.133+0000 7f97d9e00640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97c0004800 0x7f97c0004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.133+0000 7f97d9e00640 1 -- 172.21.15.7:0/3514094390 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c0003520 con 0x7f97c00039b0 2024-04-25T11:20:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.133+0000 7f97da800640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41583c0 0x7f97c0005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.134+0000 7f97d9e00640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 0x7f97c0003e20 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f97d40b43c0 tx=0x7f97c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f584df17f69e095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.134+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc037070 con 0x7f97c00039b0 2024-04-25T11:20:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.134+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97cc032630 con 0x7f97c00039b0 2024-04-25T11:20:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.134+0000 7f97d8a00640 1 -- 172.21.15.7:0/3514094390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc02e050 con 0x7f97c00039b0 2024-04-25T11:20:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.135+0000 7f97db200640 1 -- 172.21.15.7:0/3514094390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 msgr2=0x7f97c0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.135+0000 7f97db200640 1 --2- 172.21.15.7:0/3514094390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 0x7f97c0003e20 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f97d40b43c0 tx=0x7f97c0005830 comp rx=0 tx=0).stop 2024-04-25T11:20:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.135+0000 7f97da800640 1 -- 172.21.15.7:0/3514094390 reap_dead start 2024-04-25T11:20:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.135+0000 7f97db200640 1 -- 172.21.15.7:0/3514094390 shutdown_connections 2024-04-25T11:20:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.135+0000 7f97db200640 1 -- 172.21.15.7:0/3514094390 >> 172.21.15.7:0/3514094390 conn(0x7f97d40740f0 msgr2=0x7f97d406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.135+0000 7f97db200640 1 -- 172.21.15.7:0/3514094390 shutdown_connections 2024-04-25T11:20:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.135+0000 7f97db200640 1 -- 172.21.15.7:0/3514094390 wait complete. 2024-04-25T11:20:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.138+0000 7f97db200640 1 Processor -- start 2024-04-25T11:20:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.138+0000 7f97db200640 1 -- start start 2024-04-25T11:20:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.139+0000 7f97db200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 0x7f97d41f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.139+0000 7f97db200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41e78e0 0x7f97d41e7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.139+0000 7f97db200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97d4161780 0x7f97d4161b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.140+0000 7f97db200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f97d406f5b0 con 0x7f97c00039b0 2024-04-25T11:20:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.140+0000 7f97db200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f97d40cf0f0 con 0x7f97d4161780 2024-04-25T11:20:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.140+0000 7f97db200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f97d406f730 con 0x7f97d41e78e0 2024-04-25T11:20:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.140+0000 7f97d9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 0x7f97d41f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.140+0000 7f97d9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 0x7f97d41f5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52562/0 (socket says 172.21.15.7:52562) 2024-04-25T11:20:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.140+0000 7f97d9e00640 1 -- 172.21.15.7:0/2724078944 learned_addr learned my addr 172.21.15.7:0/2724078944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.140+0000 7f97d9e00640 1 -- 172.21.15.7:0/2724078944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41e78e0 msgr2=0x7f97d41e7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.140+0000 7f97d9e00640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41e78e0 0x7f97d41e7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.140+0000 7f97d9e00640 1 -- 172.21.15.7:0/2724078944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97d4161780 msgr2=0x7f97d4161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.140+0000 7f97d9e00640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97d4161780 0x7f97d4161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.140+0000 7f97d9e00640 1 -- 172.21.15.7:0/2724078944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d41620d0 con 0x7f97c00039b0 2024-04-25T11:20:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.141+0000 7f97d9e00640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 0x7f97d41f5bb0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e230 tx=0x7f97cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.141+0000 7f97ca000640 1 -- 172.21.15.7:0/2724078944 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc042020 con 0x7f97c00039b0 2024-04-25T11:20:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.141+0000 7f97ca000640 1 -- 172.21.15.7:0/2724078944 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97cc031680 con 0x7f97c00039b0 2024-04-25T11:20:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.141+0000 7f97ca000640 1 -- 172.21.15.7:0/2724078944 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc037040 con 0x7f97c00039b0 2024-04-25T11:20:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.141+0000 7f97db200640 1 -- 172.21.15.7:0/2724078944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d41f6700 con 0x7f97c00039b0 2024-04-25T11:20:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.141+0000 7f97db200640 1 -- 172.21.15.7:0/2724078944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d41f6cc0 con 0x7f97c00039b0 2024-04-25T11:20:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.142+0000 7f97db200640 1 -- 172.21.15.7:0/2724078944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d4050740 con 0x7f97c00039b0 2024-04-25T11:20:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.144+0000 7f97ca000640 1 -- 172.21.15.7:0/2724078944 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f97cc045020 con 0x7f97c00039b0 2024-04-25T11:20:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.145+0000 7f97ca000640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f97b80744b0 0x7f97b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.145+0000 7f97ca000640 1 -- 172.21.15.7:0/2724078944 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7f97cc0b5fa0 con 0x7f97c00039b0 2024-04-25T11:20:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.145+0000 7f97d9400640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f97b80744b0 0x7f97b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.146+0000 7f97d9400640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f97b80744b0 0x7f97b80769c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f97c40090e0 tx=0x7f97c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.149+0000 7f97ca000640 1 -- 172.21.15.7:0/2724078944 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f97d41620d0 con 0x7f97c00039b0 2024-04-25T11:20:34.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.355+0000 7f97db200640 1 -- 172.21.15.7:0/2724078944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97d4070dd0 con 0x7f97c00039b0 2024-04-25T11:20:34.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.356+0000 7f97ca000640 1 -- 172.21.15.7:0/2724078944 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7f97cc080890 con 0x7f97c00039b0 2024-04-25T11:20:34.353 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:20:34.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 -- 172.21.15.7:0/2724078944 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f97b80744b0 msgr2=0x7f97b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f97b80744b0 0x7f97b80769c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f97c40090e0 tx=0x7f97c4008040 comp rx=0 tx=0).stop 2024-04-25T11:20:34.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 -- 172.21.15.7:0/2724078944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 msgr2=0x7f97d41f5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 0x7f97d41f5bb0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e230 tx=0x7f97cc009750 comp rx=0 tx=0).stop 2024-04-25T11:20:34.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 -- 172.21.15.7:0/2724078944 shutdown_connections 2024-04-25T11:20:34.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f97b80744b0 0x7f97b80769c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97d4161780 0x7f97d4161b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97d41e78e0 0x7f97d41e7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 --2- 172.21.15.7:0/2724078944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97c00039b0 0x7f97d41f5bb0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 -- 172.21.15.7:0/2724078944 >> 172.21.15.7:0/2724078944 conn(0x7f97d40740f0 msgr2=0x7f97d40b57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:34.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 -- 172.21.15.7:0/2724078944 shutdown_connections 2024-04-25T11:20:34.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.359+0000 7f97db200640 1 -- 172.21.15.7:0/2724078944 wait complete. 2024-04-25T11:20:34.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2024-04-25T11:20:34.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.667+0000 7ff9bae00640 1 Processor -- start 2024-04-25T11:20:34.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.667+0000 7ff9bae00640 1 -- start start 2024-04-25T11:20:34.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.670+0000 7ff9bae00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff9b415f180 0x7ff9b4161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.670+0000 7ff9bae00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9b41621a0 0x7ff9b406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.670+0000 7ff9bae00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b406ded0 con 0x7ff9b406dac0 2024-04-25T11:20:34.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.670+0000 7ff9bae00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b415f590 con 0x7ff9b41621a0 2024-04-25T11:20:34.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.670+0000 7ff9bae00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b4158ae0 con 0x7ff9b415f180 2024-04-25T11:20:34.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.670+0000 7ff9abe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9b41621a0 0x7ff9b406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.670+0000 7ff9abe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9b41621a0 0x7ff9b406d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52566/0 (socket says 172.21.15.7:52566) 2024-04-25T11:20:34.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.670+0000 7ff9abe00640 1 -- 172.21.15.7:0/365954337 learned_addr learned my addr 172.21.15.7:0/365954337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.671+0000 7ff9b3e00640 1 --2- 172.21.15.7:0/365954337 >> v2:172.21.15.87:3300/0 conn(0x7ff9b415f180 0x7ff9b4161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.671+0000 7ff9abe00640 1 -- 172.21.15.7:0/365954337 >> v2:172.21.15.87:3300/0 conn(0x7ff9b415f180 msgr2=0x7ff9b4161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.671+0000 7ff9abe00640 1 --2- 172.21.15.7:0/365954337 >> v2:172.21.15.87:3300/0 conn(0x7ff9b415f180 0x7ff9b4161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.671+0000 7ff9abe00640 1 -- 172.21.15.7:0/365954337 >> v1:172.21.15.87:6789/0 conn(0x7ff9b406dac0 legacy=0x7ff9b406f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.671+0000 7ff9abe00640 1 -- 172.21.15.7:0/365954337 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b4162700 con 0x7ff9b41621a0 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.671+0000 7ff9b3e00640 1 --2- 172.21.15.7:0/365954337 >> v2:172.21.15.87:3300/0 conn(0x7ff9b415f180 0x7ff9b4161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.672+0000 7ff9abe00640 1 --2- 172.21.15.7:0/365954337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b41621a0 0x7ff9b406d580 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4070340 tx=0x7ff99c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2426565a11483cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:34.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.672+0000 7ff9b3400640 1 -- 172.21.15.7:0/365954337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c037070 con 0x7ff9b41621a0 2024-04-25T11:20:34.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.673+0000 7ff9b3400640 1 -- 172.21.15.7:0/365954337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff99c032440 con 0x7ff9b41621a0 2024-04-25T11:20:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.673+0000 7ff9b3400640 1 -- 172.21.15.7:0/365954337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c032810 con 0x7ff9b41621a0 2024-04-25T11:20:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.673+0000 7ff9bae00640 1 -- 172.21.15.7:0/365954337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b41621a0 msgr2=0x7ff9b406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.673+0000 7ff9bae00640 1 --2- 172.21.15.7:0/365954337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b41621a0 0x7ff9b406d580 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4070340 tx=0x7ff99c02a1c0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.673+0000 7ff9bae00640 1 -- 172.21.15.7:0/365954337 shutdown_connections 2024-04-25T11:20:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.673+0000 7ff9bae00640 1 --2- 172.21.15.7:0/365954337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b41621a0 0x7ff9b406d580 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.673+0000 7ff9bae00640 1 --2- 172.21.15.7:0/365954337 >> v2:172.21.15.87:3300/0 conn(0x7ff9b415f180 0x7ff9b4161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.673+0000 7ff9bae00640 1 -- 172.21.15.7:0/365954337 >> 172.21.15.7:0/365954337 conn(0x7ff9b40732b0 msgr2=0x7ff9b40736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.673+0000 7ff9bae00640 1 -- 172.21.15.7:0/365954337 shutdown_connections 2024-04-25T11:20:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.674+0000 7ff9bae00640 1 -- 172.21.15.7:0/365954337 wait complete. 2024-04-25T11:20:34.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.676+0000 7ff9bae00640 1 Processor -- start 2024-04-25T11:20:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.676+0000 7ff9bae00640 1 -- start start 2024-04-25T11:20:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.677+0000 7ff9bae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b406dac0 0x7ff9b415e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.678+0000 7ff9bae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9b415f180 0x7ff9b409b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.678+0000 7ff9bae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9b41621a0 0x7ff9b4156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.678+0000 7ff9bae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b415ec60 con 0x7ff9b406dac0 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.678+0000 7ff9bae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b415e980 con 0x7ff9b415f180 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.678+0000 7ff9bae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b415e670 con 0x7ff9b41621a0 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.678+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b406dac0 0x7ff9b415e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.678+0000 7ff9b3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b406dac0 0x7ff9b415e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52574/0 (socket says 172.21.15.7:52574) 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.678+0000 7ff9b3e00640 1 -- 172.21.15.7:0/2494305582 learned_addr learned my addr 172.21.15.7:0/2494305582 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.678+0000 7ff9abe00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9b415f180 0x7ff9b409b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.678+0000 7ff9b8e00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9b41621a0 0x7ff9b4156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.679+0000 7ff9b3e00640 1 -- 172.21.15.7:0/2494305582 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9b41621a0 msgr2=0x7ff9b4156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.679+0000 7ff9b3e00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9b41621a0 0x7ff9b4156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.679+0000 7ff9b3e00640 1 -- 172.21.15.7:0/2494305582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9b415f180 msgr2=0x7ff9b409b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.679+0000 7ff9b3e00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9b415f180 0x7ff9b409b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.679+0000 7ff9b3e00640 1 -- 172.21.15.7:0/2494305582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b41568c0 con 0x7ff9b406dac0 2024-04-25T11:20:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.679+0000 7ff9b3e00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b406dac0 0x7ff9b415e070 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400c820 tx=0x7ff9a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.679+0000 7ff9b0c00640 1 -- 172.21.15.7:0/2494305582 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4015070 con 0x7ff9b406dac0 2024-04-25T11:20:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.679+0000 7ff9b0c00640 1 -- 172.21.15.7:0/2494305582 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a4007a30 con 0x7ff9b406dac0 2024-04-25T11:20:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.679+0000 7ff9b0c00640 1 -- 172.21.15.7:0/2494305582 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4007e20 con 0x7ff9b406dac0 2024-04-25T11:20:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.679+0000 7ff9bae00640 1 -- 172.21.15.7:0/2494305582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b409de80 con 0x7ff9b406dac0 2024-04-25T11:20:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.679+0000 7ff9bae00640 1 -- 172.21.15.7:0/2494305582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b409e440 con 0x7ff9b406dac0 2024-04-25T11:20:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.680+0000 7ff9bae00640 1 -- 172.21.15.7:0/2494305582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b4050740 con 0x7ff9b406dac0 2024-04-25T11:20:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.682+0000 7ff9b0c00640 1 -- 172.21.15.7:0/2494305582 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff9a4018070 con 0x7ff9b406dac0 2024-04-25T11:20:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.683+0000 7ff9b0c00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9900744e0 0x7ff9900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.683+0000 7ff9b0c00640 1 -- 172.21.15.7:0/2494305582 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7ff9a4094ce0 con 0x7ff9b406dac0 2024-04-25T11:20:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.683+0000 7ff9abe00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9900744e0 0x7ff9900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.684+0000 7ff9abe00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9900744e0 0x7ff9900769f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff99c034000 tx=0x7ff99c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.688+0000 7ff9b0c00640 1 -- 172.21.15.7:0/2494305582 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff9b41568c0 con 0x7ff9b406dac0 2024-04-25T11:20:34.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:34.965+0000 7ff9bae00640 1 -- 172.21.15.7:0/2494305582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) -- 0x7ff9b409f5f0 con 0x7ff9b406dac0 2024-04-25T11:20:35.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.272+0000 7ff9b0c00640 1 -- 172.21.15.7:0/2494305582 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v526) ==== 154+0+0 (secure 0 0 0) 0x7ff9a405f540 con 0x7ff9b406dac0 2024-04-25T11:20:35.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.276+0000 7ff9bae00640 1 -- 172.21.15.7:0/2494305582 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9900744e0 msgr2=0x7ff9900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.276+0000 7ff9bae00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9900744e0 0x7ff9900769f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff99c034000 tx=0x7ff99c02e000 comp rx=0 tx=0).stop 2024-04-25T11:20:35.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.276+0000 7ff9bae00640 1 -- 172.21.15.7:0/2494305582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b406dac0 msgr2=0x7ff9b415e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.276+0000 7ff9bae00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b406dac0 0x7ff9b415e070 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400c820 tx=0x7ff9a400cce0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.276+0000 7ff9bae00640 1 -- 172.21.15.7:0/2494305582 shutdown_connections 2024-04-25T11:20:35.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.276+0000 7ff9bae00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff9900744e0 0x7ff9900769f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.276+0000 7ff9bae00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9b41621a0 0x7ff9b4156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.276+0000 7ff9bae00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9b415f180 0x7ff9b409b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.276+0000 7ff9bae00640 1 --2- 172.21.15.7:0/2494305582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9b406dac0 0x7ff9b415e070 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.276+0000 7ff9bae00640 1 -- 172.21.15.7:0/2494305582 >> 172.21.15.7:0/2494305582 conn(0x7ff9b40732b0 msgr2=0x7ff9b406f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:35.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.276+0000 7ff9bae00640 1 -- 172.21.15.7:0/2494305582 shutdown_connections 2024-04-25T11:20:35.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.277+0000 7ff9bae00640 1 -- 172.21.15.7:0/2494305582 wait complete. 2024-04-25T11:20:35.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:20:35.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.583+0000 7efeebe00640 1 Processor -- start 2024-04-25T11:20:35.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.583+0000 7efeebe00640 1 -- start start 2024-04-25T11:20:35.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.586+0000 7efeebe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efeec06fe40 0x7efeec167280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.586+0000 7efeebe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeec0b4830 con 0x7efeec0cece0 2024-04-25T11:20:35.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.586+0000 7efeebe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeec0b4b10 con 0x7efeec0ce8d0 2024-04-25T11:20:35.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.586+0000 7efeebe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efeec0b6600 con 0x7efeec06fe40 2024-04-25T11:20:35.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.586+0000 7efeea000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efeec0cece0 0x7efeec06f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58352/0 (socket says 172.21.15.7:58352) 2024-04-25T11:20:35.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.586+0000 7efeea000640 1 -- 172.21.15.7:0/1255748870 learned_addr learned my addr 172.21.15.7:0/1255748870 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:35.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.586+0000 7efeeb400640 1 --2- 172.21.15.7:0/1255748870 >> v2:172.21.15.7:3300/0 conn(0x7efeec06fe40 0x7efeec167280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.587+0000 7efee9600640 1 -- 172.21.15.7:0/1255748870 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2757276783 0 0) 0x7efeec0b4830 con 0x7efeec0cece0 2024-04-25T11:20:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.587+0000 7efee9600640 1 -- 172.21.15.7:0/1255748870 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efed0003520 con 0x7efeec0cece0 2024-04-25T11:20:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.587+0000 7efeeb400640 1 -- 172.21.15.7:0/1255748870 >> v1:172.21.15.87:6789/0 conn(0x7efeec0ce8d0 legacy=0x7efeec15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.587+0000 7efeeb400640 1 -- 172.21.15.7:0/1255748870 >> v1:172.21.15.7:6789/0 conn(0x7efeec0cece0 legacy=0x7efeec06f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.587+0000 7efeeb400640 1 -- 172.21.15.7:0/1255748870 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec0b4df0 con 0x7efeec06fe40 2024-04-25T11:20:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.588+0000 7efeeb400640 1 --2- 172.21.15.7:0/1255748870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeec06fe40 0x7efeec167280 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7efeec0b6780 tx=0x7efee402b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902e9008012c19c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.588+0000 7efee9600640 1 -- 172.21.15.7:0/1255748870 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4037070 con 0x7efeec06fe40 2024-04-25T11:20:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.588+0000 7efee9600640 1 -- 172.21.15.7:0/1255748870 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee4008640 con 0x7efeec06fe40 2024-04-25T11:20:35.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.588+0000 7efee9600640 1 -- 172.21.15.7:0/1255748870 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4008a10 con 0x7efeec06fe40 2024-04-25T11:20:35.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.589+0000 7efeebe00640 1 -- 172.21.15.7:0/1255748870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeec06fe40 msgr2=0x7efeec167280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.589+0000 7efeebe00640 1 --2- 172.21.15.7:0/1255748870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeec06fe40 0x7efeec167280 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7efeec0b6780 tx=0x7efee402b760 comp rx=0 tx=0).stop 2024-04-25T11:20:35.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.589+0000 7efeebe00640 1 -- 172.21.15.7:0/1255748870 shutdown_connections 2024-04-25T11:20:35.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.589+0000 7efeebe00640 1 --2- 172.21.15.7:0/1255748870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeec06fe40 0x7efeec167280 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.589+0000 7efeebe00640 1 -- 172.21.15.7:0/1255748870 >> 172.21.15.7:0/1255748870 conn(0x7efeec0740f0 msgr2=0x7efeec06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:35.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.589+0000 7efeebe00640 1 -- 172.21.15.7:0/1255748870 shutdown_connections 2024-04-25T11:20:35.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.589+0000 7efeebe00640 1 -- 172.21.15.7:0/1255748870 wait complete. 2024-04-25T11:20:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.591+0000 7efeebe00640 1 Processor -- start 2024-04-25T11:20:35.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.592+0000 7efeebe00640 1 -- start start 2024-04-25T11:20:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.593+0000 7efeebe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeec0ce8d0 0x7efeec155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.593+0000 7efeebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeec0cece0 0x7efeec15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.593+0000 7efeebe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efeec14e480 0x7efeec1601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.593+0000 7efeebe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efeec070e70 con 0x7efeec0cece0 2024-04-25T11:20:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.593+0000 7efeebe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efeec15c9f0 con 0x7efeec0ce8d0 2024-04-25T11:20:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.593+0000 7efeebe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efeec070ff0 con 0x7efeec14e480 2024-04-25T11:20:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.594+0000 7efeea000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeec0cece0 0x7efeec15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.594+0000 7efeea000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeec0cece0 0x7efeec15fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52588/0 (socket says 172.21.15.7:52588) 2024-04-25T11:20:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.594+0000 7efeea000640 1 -- 172.21.15.7:0/1621460782 learned_addr learned my addr 172.21.15.7:0/1621460782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:35.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.594+0000 7efeeaa00640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeec0ce8d0 0x7efeec155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.594+0000 7efeeb400640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efeec14e480 0x7efeec1601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.594+0000 7efeea000640 1 -- 172.21.15.7:0/1621460782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efeec14e480 msgr2=0x7efeec1601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.594+0000 7efeea000640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efeec14e480 0x7efeec1601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.594+0000 7efeea000640 1 -- 172.21.15.7:0/1621460782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeec0ce8d0 msgr2=0x7efeec155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.594+0000 7efeea000640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeec0ce8d0 0x7efeec155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.594+0000 7efeea000640 1 -- 172.21.15.7:0/1621460782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec1f9e00 con 0x7efeec0cece0 2024-04-25T11:20:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.595+0000 7efeea000640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeec0cece0 0x7efeec15fc60 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7efed800c3f0 tx=0x7efed800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.595+0000 7efee2a00640 1 -- 172.21.15.7:0/1621460782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8017070 con 0x7efeec0cece0 2024-04-25T11:20:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.595+0000 7efee2a00640 1 -- 172.21.15.7:0/1621460782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed8005ce0 con 0x7efeec0cece0 2024-04-25T11:20:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.595+0000 7efee2a00640 1 -- 172.21.15.7:0/1621460782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8007310 con 0x7efeec0cece0 2024-04-25T11:20:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.595+0000 7efeebe00640 1 -- 172.21.15.7:0/1621460782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeec1fa0e0 con 0x7efeec0cece0 2024-04-25T11:20:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.595+0000 7efeebe00640 1 -- 172.21.15.7:0/1621460782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeec1fa6a0 con 0x7efeec0cece0 2024-04-25T11:20:35.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.596+0000 7efeebe00640 1 -- 172.21.15.7:0/1621460782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeec159520 con 0x7efeec0cece0 2024-04-25T11:20:35.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.598+0000 7efee2a00640 1 -- 172.21.15.7:0/1621460782 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efed801a070 con 0x7efeec0cece0 2024-04-25T11:20:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.599+0000 7efee2a00640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efecc0744e0 0x7efecc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:35.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.599+0000 7efee2a00640 1 -- 172.21.15.7:0/1621460782 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7efed80944e0 con 0x7efeec0cece0 2024-04-25T11:20:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.599+0000 7efeeaa00640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efecc0744e0 0x7efecc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.600+0000 7efeeaa00640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efecc0744e0 0x7efecc0769f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efedc000f60 tx=0x7efedc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.603+0000 7efee2a00640 1 -- 172.21.15.7:0/1621460782 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efeec159520 con 0x7efeec0cece0 2024-04-25T11:20:35.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.811+0000 7efeebe00640 1 -- 172.21.15.7:0/1621460782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efeec068f10 con 0x7efeec0cece0 2024-04-25T11:20:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.821+0000 7efee2a00640 1 -- 172.21.15.7:0/1621460782 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7efeec068f10 con 0x7efeec0cece0 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 -- 172.21.15.7:0/1621460782 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efecc0744e0 msgr2=0x7efecc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efecc0744e0 0x7efecc0769f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efedc000f60 tx=0x7efedc009040 comp rx=0 tx=0).stop 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 -- 172.21.15.7:0/1621460782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeec0cece0 msgr2=0x7efeec15fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeec0cece0 0x7efeec15fc60 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7efed800c3f0 tx=0x7efed800c8b0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 -- 172.21.15.7:0/1621460782 shutdown_connections 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efecc0744e0 0x7efecc0769f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efeec14e480 0x7efeec1601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeec0cece0 0x7efeec15fc60 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 --2- 172.21.15.7:0/1621460782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeec0ce8d0 0x7efeec155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 -- 172.21.15.7:0/1621460782 >> 172.21.15.7:0/1621460782 conn(0x7efeec0740f0 msgr2=0x7efeec06f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 -- 172.21.15.7:0/1621460782 shutdown_connections 2024-04-25T11:20:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:35.824+0000 7efeebe00640 1 -- 172.21.15.7:0/1621460782 wait complete. 2024-04-25T11:20:35.851 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:20:35.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:20:35.957 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:20:36.071 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:20:36.071 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:20:36.071 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:36.132 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:20:36.133 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:20:36.133 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:36.192 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:20:36.192 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:20:36.192 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:36.252 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:20:36.253 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:20:36.253 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:36.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:20:36.310 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:20:36.311 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.607+0000 7fe17cc00640 1 Processor -- start 2024-04-25T11:20:36.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.607+0000 7fe17cc00640 1 -- start start 2024-04-25T11:20:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.609+0000 7fe17cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe17815bc40 0x7fe17809f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.610+0000 7fe17cc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe17815f4e0 con 0x7fe17815ef90 2024-04-25T11:20:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.610+0000 7fe17cc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1780b0560 con 0x7fe1780af540 2024-04-25T11:20:36.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.610+0000 7fe17cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe178071190 con 0x7fe17815bc40 2024-04-25T11:20:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.610+0000 7fe175e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe17815ef90 0x7fe1780aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58356/0 (socket says 172.21.15.7:58356) 2024-04-25T11:20:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.610+0000 7fe175e00640 1 -- 172.21.15.7:0/3680610681 learned_addr learned my addr 172.21.15.7:0/3680610681 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:36.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.610+0000 7fe176800640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe17815bc40 0x7fe17809f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.611+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2233444460 0 0) 0x7fe17815f4e0 con 0x7fe17815ef90 2024-04-25T11:20:36.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.611+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe15c003520 con 0x7fe17815ef90 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.611+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2854151593 0 0) 0x7fe15c003520 con 0x7fe17815ef90 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.611+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe17815bc40 msgr2=0x7fe17809f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.611+0000 7fe174a00640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe17815bc40 0x7fe17809f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.611+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 >> v1:172.21.15.87:6789/0 conn(0x7fe1780af540 legacy=0x7fe1780cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.611+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1780b0840 con 0x7fe17815ef90 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.611+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fe168001670 con 0x7fe17815ef90 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.611+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 >> v1:172.21.15.7:6789/0 conn(0x7fe17815ef90 legacy=0x7fe1780aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.611+0000 7fe174a00640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe17815bc40 0x7fe15c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe174a00640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe15c003f40 0x7fe15c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe174a00640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 0x7fe15c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe178071190 con 0x7fe15c004d40 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c00bb80 con 0x7fe15c003f40 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe15c00bd80 con 0x7fe17815bc40 2024-04-25T11:20:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe176800640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 0x7fe15c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe175400640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe15c003f40 0x7fe15c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe175e00640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe17815bc40 0x7fe15c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe176800640 1 -- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe17815bc40 msgr2=0x7fe15c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe176800640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe17815bc40 0x7fe15c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe176800640 1 -- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe15c003f40 msgr2=0x7fe15c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe176800640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe15c003f40 0x7fe15c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe176800640 1 -- 172.21.15.7:0/3680610681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c003520 con 0x7fe15c004d40 2024-04-25T11:20:36.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.612+0000 7fe175400640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe15c003f40 0x7fe15c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.614+0000 7fe176800640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 0x7fe15c005150 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fe170009fc0 tx=0x7fe15c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a1ffc104b5124d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.614+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170034020 con 0x7fe15c004d40 2024-04-25T11:20:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.614+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1700305f0 con 0x7fe15c004d40 2024-04-25T11:20:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.614+0000 7fe174a00640 1 -- 172.21.15.7:0/3680610681 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170009c10 con 0x7fe15c004d40 2024-04-25T11:20:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.615+0000 7fe17cc00640 1 -- 172.21.15.7:0/3680610681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 msgr2=0x7fe15c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.615+0000 7fe17cc00640 1 --2- 172.21.15.7:0/3680610681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 0x7fe15c005150 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fe170009fc0 tx=0x7fe15c005830 comp rx=0 tx=0).stop 2024-04-25T11:20:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.615+0000 7fe176800640 1 -- 172.21.15.7:0/3680610681 reap_dead start 2024-04-25T11:20:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.615+0000 7fe17cc00640 1 -- 172.21.15.7:0/3680610681 shutdown_connections 2024-04-25T11:20:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.615+0000 7fe17cc00640 1 -- 172.21.15.7:0/3680610681 >> 172.21.15.7:0/3680610681 conn(0x7fe178074200 msgr2=0x7fe17806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.615+0000 7fe17cc00640 1 -- 172.21.15.7:0/3680610681 shutdown_connections 2024-04-25T11:20:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.615+0000 7fe17cc00640 1 -- 172.21.15.7:0/3680610681 wait complete. 2024-04-25T11:20:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.617+0000 7fe17cc00640 1 Processor -- start 2024-04-25T11:20:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.617+0000 7fe17cc00640 1 -- start start 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.619+0000 7fe17cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 0x7fe1781f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.619+0000 7fe17cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe17815e030 0x7fe178160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.619+0000 7fe17cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1781609d0 0x7fe1780b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.619+0000 7fe17cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1780b1320 con 0x7fe15c004d40 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.619+0000 7fe17cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe1780b0e00 con 0x7fe17815e030 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.619+0000 7fe17cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe1780afb80 con 0x7fe1781609d0 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.619+0000 7fe175e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 0x7fe1781f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.619+0000 7fe175e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 0x7fe1781f31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52608/0 (socket says 172.21.15.7:52608) 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.619+0000 7fe175e00640 1 -- 172.21.15.7:0/3058493017 learned_addr learned my addr 172.21.15.7:0/3058493017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.620+0000 7fe176800640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1781609d0 0x7fe1780b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.620+0000 7fe175e00640 1 -- 172.21.15.7:0/3058493017 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1781609d0 msgr2=0x7fe1780b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.620+0000 7fe175e00640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1781609d0 0x7fe1780b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.620+0000 7fe175400640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe17815e030 0x7fe178160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.620+0000 7fe175e00640 1 -- 172.21.15.7:0/3058493017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe17815e030 msgr2=0x7fe178160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.620+0000 7fe175e00640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe17815e030 0x7fe178160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.620+0000 7fe175e00640 1 -- 172.21.15.7:0/3058493017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe178160de0 con 0x7fe15c004d40 2024-04-25T11:20:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.620+0000 7fe175e00640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 0x7fe1781f31a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fe16800e7f0 tx=0x7fe16800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.620+0000 7fe16e000640 1 -- 172.21.15.7:0/3058493017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16800c8d0 con 0x7fe15c004d40 2024-04-25T11:20:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.620+0000 7fe16e000640 1 -- 172.21.15.7:0/3058493017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe168018070 con 0x7fe15c004d40 2024-04-25T11:20:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.621+0000 7fe17cc00640 1 -- 172.21.15.7:0/3058493017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1780b28b0 con 0x7fe15c004d40 2024-04-25T11:20:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.621+0000 7fe16e000640 1 -- 172.21.15.7:0/3058493017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16800ccd0 con 0x7fe15c004d40 2024-04-25T11:20:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.621+0000 7fe17cc00640 1 -- 172.21.15.7:0/3058493017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe178152b90 con 0x7fe15c004d40 2024-04-25T11:20:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.622+0000 7fe17cc00640 1 -- 172.21.15.7:0/3058493017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1781526b0 con 0x7fe15c004d40 2024-04-25T11:20:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.623+0000 7fe16e000640 1 -- 172.21.15.7:0/3058493017 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe16801d460 con 0x7fe15c004d40 2024-04-25T11:20:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.624+0000 7fe16e000640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1580744b0 0x7fe1580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.624+0000 7fe16e000640 1 -- 172.21.15.7:0/3058493017 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5335+0+0 (secure 0 0 0) 0x7fe168095040 con 0x7fe15c004d40 2024-04-25T11:20:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.624+0000 7fe175400640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1580744b0 0x7fe1580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.625+0000 7fe175400640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1580744b0 0x7fe1580769c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe164007fd0 tx=0x7fe164009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:36.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.630+0000 7fe16e000640 1 -- 172.21.15.7:0/3058493017 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe178160de0 con 0x7fe15c004d40 2024-04-25T11:20:36.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.878+0000 7fe17cc00640 1 -- 172.21.15.7:0/3058493017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe17806a100 con 0x7fe15c004d40 2024-04-25T11:20:36.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.879+0000 7fe16e000640 1 -- 172.21.15.7:0/3058493017 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+3860 (secure 0 0 0) 0x7fe16805f930 con 0x7fe15c004d40 2024-04-25T11:20:36.877 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:36.877 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":83,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6023,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/326382713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":326382713},{"type":"v1","addr":"172.21.15.7:6837","nonce":326382713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6194,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2446876478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2446876478},{"type":"v1","addr":"172.21.15.87:6834","nonce":2446876478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":6673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2007810270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":2007810270},{"type":"v1","addr":"172.21.15.87:6835","nonce":2007810270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7998,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1753656512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1753656512},{"type":"v1","addr":"172.21.15.7:6836","nonce":1753656512}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[]} 2024-04-25T11:20:36.877 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 83 2024-04-25T11:20:36.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 -- 172.21.15.7:0/3058493017 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1580744b0 msgr2=0x7fe1580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1580744b0 0x7fe1580769c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe164007fd0 tx=0x7fe164009040 comp rx=0 tx=0).stop 2024-04-25T11:20:36.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 -- 172.21.15.7:0/3058493017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 msgr2=0x7fe1781f31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:36.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 0x7fe1781f31a0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fe16800e7f0 tx=0x7fe16800ecb0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 -- 172.21.15.7:0/3058493017 shutdown_connections 2024-04-25T11:20:36.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1580744b0 0x7fe1580769c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1781609d0 0x7fe1780b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe17815e030 0x7fe178160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 --2- 172.21.15.7:0/3058493017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe15c004d40 0x7fe1781f31a0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:36.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 -- 172.21.15.7:0/3058493017 >> 172.21.15.7:0/3058493017 conn(0x7fe178074200 msgr2=0x7fe17815b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:36.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 -- 172.21.15.7:0/3058493017 shutdown_connections 2024-04-25T11:20:36.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:36.883+0000 7fe17cc00640 1 -- 172.21.15.7:0/3058493017 wait complete. 2024-04-25T11:20:36.908 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:20:36.908 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:20:36.908 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:20:36.908 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:36.908 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:20:36.908 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:20:36.909 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:20:36.909 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:36.909 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:20:36.909 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:20:36.909 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:20:36.909 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:36.910 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:20:36.910 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:20:36.910 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:20:36.910 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:20:36.938 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:20:36.938 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:20:36.943 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:20:36.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:20:36.954 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:20:36.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:20:36.958 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:20:36.958 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:20:37.001 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:20:37.001 DEBUG:teuthology.parallel:result is None 2024-04-25T11:20:37.005 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:20:37.006 DEBUG:teuthology.parallel:result is None 2024-04-25T11:20:37.007 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:20:37.007 DEBUG:teuthology.parallel:result is None 2024-04-25T11:20:37.012 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:20:37.012 DEBUG:teuthology.parallel:result is None 2024-04-25T11:20:37.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:20:37.231 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:20:37.232 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:20:37.235 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:20:37.247 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:20:37.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.271+0000 7f6d06e00640 1 Processor -- start 2024-04-25T11:20:37.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.271+0000 7f6d06e00640 1 -- start start 2024-04-25T11:20:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.273+0000 7f6d06e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d000d1350 0x7f6d001648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.273+0000 7f6d06e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d000b5660 con 0x7f6d0006d580 2024-04-25T11:20:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.273+0000 7f6d06e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d000b58e0 con 0x7f6d000b6960 2024-04-25T11:20:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.273+0000 7f6d06e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6d000b64d0 con 0x7f6d000d1350 2024-04-25T11:20:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.273+0000 7f6cffe00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f6d000b6960 0x7f6d00160220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:56154/0 (socket says 172.21.15.7:56154) 2024-04-25T11:20:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.273+0000 7f6cffe00640 1 -- 172.21.15.7:0/3174745787 learned_addr learned my addr 172.21.15.7:0/3174745787 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.273+0000 7f6d04e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6d0006d580 0x7f6d0006d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58374/0 (socket says 172.21.15.7:58374) 2024-04-25T11:20:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.273+0000 7f6cf7e00640 1 --2- 172.21.15.7:0/3174745787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d000d1350 0x7f6d001648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.274+0000 7f6cff400640 1 -- 172.21.15.7:0/3174745787 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1513729217 0 0) 0x7f6d000b58e0 con 0x7f6d000b6960 2024-04-25T11:20:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.274+0000 7f6cff400640 1 -- 172.21.15.7:0/3174745787 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ce4003520 con 0x7f6d000b6960 2024-04-25T11:20:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.274+0000 7f6cff400640 1 -- 172.21.15.7:0/3174745787 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3802124977 0 0) 0x7f6cf8001490 con 0x7f6d0006d580 2024-04-25T11:20:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.274+0000 7f6cf7e00640 1 -- 172.21.15.7:0/3174745787 >> v1:172.21.15.87:6789/0 conn(0x7f6d000b6960 legacy=0x7f6d00160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.274+0000 7f6cf7e00640 1 -- 172.21.15.7:0/3174745787 >> v1:172.21.15.7:6789/0 conn(0x7f6d0006d580 legacy=0x7f6d0006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.274+0000 7f6cf7e00640 1 -- 172.21.15.7:0/3174745787 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d00164df0 con 0x7f6d000d1350 2024-04-25T11:20:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.275+0000 7f6cf7e00640 1 --2- 172.21.15.7:0/3174745787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d000d1350 0x7f6d001648b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80097d0 tx=0x7f6ce802a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=801a222167070f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:37.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.275+0000 7f6cff400640 1 -- 172.21.15.7:0/3174745787 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8037070 con 0x7f6d000d1350 2024-04-25T11:20:37.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.275+0000 7f6cff400640 1 -- 172.21.15.7:0/3174745787 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce8032440 con 0x7f6d000d1350 2024-04-25T11:20:37.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.275+0000 7f6cff400640 1 -- 172.21.15.7:0/3174745787 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8032810 con 0x7f6d000d1350 2024-04-25T11:20:37.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.276+0000 7f6d06e00640 1 -- 172.21.15.7:0/3174745787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d000d1350 msgr2=0x7f6d001648b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.276+0000 7f6d06e00640 1 --2- 172.21.15.7:0/3174745787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d000d1350 0x7f6d001648b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80097d0 tx=0x7f6ce802a200 comp rx=0 tx=0).stop 2024-04-25T11:20:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.276+0000 7f6d06e00640 1 -- 172.21.15.7:0/3174745787 shutdown_connections 2024-04-25T11:20:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.276+0000 7f6d06e00640 1 --2- 172.21.15.7:0/3174745787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d000d1350 0x7f6d001648b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.276+0000 7f6d06e00640 1 -- 172.21.15.7:0/3174745787 >> 172.21.15.7:0/3174745787 conn(0x7f6d000732b0 msgr2=0x7f6d000736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.276+0000 7f6d06e00640 1 -- 172.21.15.7:0/3174745787 shutdown_connections 2024-04-25T11:20:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.277+0000 7f6d06e00640 1 -- 172.21.15.7:0/3174745787 wait complete. 2024-04-25T11:20:37.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.280+0000 7f6d06e00640 1 Processor -- start 2024-04-25T11:20:37.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.280+0000 7f6d06e00640 1 -- start start 2024-04-25T11:20:37.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.282+0000 7f6d06e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0006d580 0x7f6d0014bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.282+0000 7f6d06e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d000b6960 0x7f6d001fa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.282+0000 7f6d06e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d000d1350 0x7f6d00156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.282+0000 7f6d06e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d000b6160 con 0x7f6d0006d580 2024-04-25T11:20:37.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.282+0000 7f6d06e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d000b5e80 con 0x7f6d000b6960 2024-04-25T11:20:37.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.282+0000 7f6d06e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0016b260 con 0x7f6d000d1350 2024-04-25T11:20:37.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.283+0000 7f6cffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0006d580 0x7f6d0014bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.283+0000 7f6cffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0006d580 0x7f6d0014bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52658/0 (socket says 172.21.15.7:52658) 2024-04-25T11:20:37.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.283+0000 7f6cffe00640 1 -- 172.21.15.7:0/3639952197 learned_addr learned my addr 172.21.15.7:0/3639952197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:37.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.283+0000 7f6cf7e00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d000b6960 0x7f6d001fa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.283+0000 7f6d04e00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d000d1350 0x7f6d00156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.283+0000 7f6cffe00640 1 -- 172.21.15.7:0/3639952197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d000d1350 msgr2=0x7f6d00156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.283+0000 7f6cffe00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d000d1350 0x7f6d00156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.283+0000 7f6cffe00640 1 -- 172.21.15.7:0/3639952197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d000b6960 msgr2=0x7f6d001fa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:37.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.283+0000 7f6cffe00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d000b6960 0x7f6d001fa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:37.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.283+0000 7f6cffe00640 1 -- 172.21.15.7:0/3639952197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d000b0840 con 0x7f6d0006d580 2024-04-25T11:20:37.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.284+0000 7f6cffe00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0006d580 0x7f6d0014bc90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000c3c0 tx=0x7f6cf000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:37.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.286+0000 7f6cfcc00640 1 -- 172.21.15.7:0/3639952197 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0017070 con 0x7f6d0006d580 2024-04-25T11:20:37.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.286+0000 7f6cfcc00640 1 -- 172.21.15.7:0/3639952197 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cf0005ce0 con 0x7f6d0006d580 2024-04-25T11:20:37.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.286+0000 7f6d06e00640 1 -- 172.21.15.7:0/3639952197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d000aed10 con 0x7f6d0006d580 2024-04-25T11:20:37.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.286+0000 7f6cfcc00640 1 -- 172.21.15.7:0/3639952197 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf00072f0 con 0x7f6d0006d580 2024-04-25T11:20:37.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.286+0000 7f6d06e00640 1 -- 172.21.15.7:0/3639952197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d000af2d0 con 0x7f6d0006d580 2024-04-25T11:20:37.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.288+0000 7f6d06e00640 1 -- 172.21.15.7:0/3639952197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d00050740 con 0x7f6d0006d580 2024-04-25T11:20:37.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.289+0000 7f6cfcc00640 1 -- 172.21.15.7:0/3639952197 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6cf001a070 con 0x7f6d0006d580 2024-04-25T11:20:37.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.290+0000 7f6cfcc00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6cdc0744b0 0x7f6cdc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:37.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.290+0000 7f6cfcc00640 1 -- 172.21.15.7:0/3639952197 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5335+0+0 (secure 0 0 0) 0x7f6cf0094410 con 0x7f6d0006d580 2024-04-25T11:20:37.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.290+0000 7f6cf7e00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6cdc0744b0 0x7f6cdc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:37.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.291+0000 7f6cf7e00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6cdc0744b0 0x7f6cdc0769c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8034000 tx=0x7f6ce802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:37.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.295+0000 7f6cfcc00640 1 -- 172.21.15.7:0/3639952197 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6cf005e6b0 con 0x7f6d0006d580 2024-04-25T11:20:37.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:37.503+0000 7f6d06e00640 1 -- 172.21.15.7:0/3639952197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6d000cd880 con 0x7f6d0006d580 2024-04-25T11:20:38.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.307+0000 7f6cfcc00640 1 -- 172.21.15.7:0/3639952197 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v126) ==== 97+0+0 (secure 0 0 0) 0x7f6cf00629b0 con 0x7f6d0006d580 2024-04-25T11:20:38.304 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-04-25T11:20:38.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.312+0000 7f6d06e00640 1 -- 172.21.15.7:0/3639952197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6cdc0744b0 msgr2=0x7f6cdc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.312+0000 7f6d06e00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6cdc0744b0 0x7f6cdc0769c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8034000 tx=0x7f6ce802e000 comp rx=0 tx=0).stop 2024-04-25T11:20:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.312+0000 7f6d06e00640 1 -- 172.21.15.7:0/3639952197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0006d580 msgr2=0x7f6d0014bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.312+0000 7f6d06e00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0006d580 0x7f6d0014bc90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000c3c0 tx=0x7f6cf000c880 comp rx=0 tx=0).stop 2024-04-25T11:20:38.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.312+0000 7f6d06e00640 1 -- 172.21.15.7:0/3639952197 shutdown_connections 2024-04-25T11:20:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.312+0000 7f6d06e00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6cdc0744b0 0x7f6cdc0769c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.312+0000 7f6d06e00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d000d1350 0x7f6d00156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.312+0000 7f6d06e00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d000b6960 0x7f6d001fa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.312+0000 7f6d06e00640 1 --2- 172.21.15.7:0/3639952197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d0006d580 0x7f6d0014bc90 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.312+0000 7f6d06e00640 1 -- 172.21.15.7:0/3639952197 >> 172.21.15.7:0/3639952197 conn(0x7f6d000732b0 msgr2=0x7f6d00163b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.312+0000 7f6d06e00640 1 -- 172.21.15.7:0/3639952197 shutdown_connections 2024-04-25T11:20:38.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.313+0000 7f6d06e00640 1 -- 172.21.15.7:0/3639952197 wait complete. 2024-04-25T11:20:38.340 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:20:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.612+0000 7fce4da00640 1 Processor -- start 2024-04-25T11:20:38.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.612+0000 7fce4da00640 1 -- start start 2024-04-25T11:20:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.615+0000 7fce4da00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce48160a10 0x7fce480d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.615+0000 7fce4da00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce48160e20 con 0x7fce4806d580 2024-04-25T11:20:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.615+0000 7fce4da00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce480b4810 con 0x7fce480cf710 2024-04-25T11:20:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.615+0000 7fce4da00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fce480d12b0 con 0x7fce48160a10 2024-04-25T11:20:38.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.616+0000 7fce47e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce48160a10 0x7fce480d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.616+0000 7fce3fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fce4806d580 0x7fce4806d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58376/0 (socket says 172.21.15.7:58376) 2024-04-25T11:20:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.616+0000 7fce3fe00640 1 -- 172.21.15.7:0/3157890494 learned_addr learned my addr 172.21.15.7:0/3157890494 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.616+0000 7fce47e00640 1 -- 172.21.15.7:0/3157890494 >> v1:172.21.15.87:6789/0 conn(0x7fce480cf710 legacy=0x7fce480cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.616+0000 7fce47e00640 1 -- 172.21.15.7:0/3157890494 >> v1:172.21.15.7:6789/0 conn(0x7fce4806d580 legacy=0x7fce4806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.617+0000 7fce47e00640 1 -- 172.21.15.7:0/3157890494 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce480b4af0 con 0x7fce48160a10 2024-04-25T11:20:38.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.616+0000 7fce47400640 1 -- 172.21.15.7:0/3157890494 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2571583090 0 0) 0x7fce48160e20 con 0x7fce4806d580 2024-04-25T11:20:38.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.618+0000 7fce47e00640 1 --2- 172.21.15.7:0/3157890494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce48160a10 0x7fce480d0d70 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fce380097a0 tx=0x7fce3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2a44e8c37bda08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.618+0000 7fce47400640 1 -- 172.21.15.7:0/3157890494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38037070 con 0x7fce48160a10 2024-04-25T11:20:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.618+0000 7fce47400640 1 -- 172.21.15.7:0/3157890494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce38032440 con 0x7fce48160a10 2024-04-25T11:20:38.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.618+0000 7fce47400640 1 -- 172.21.15.7:0/3157890494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38032810 con 0x7fce48160a10 2024-04-25T11:20:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.619+0000 7fce4da00640 1 -- 172.21.15.7:0/3157890494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce48160a10 msgr2=0x7fce480d0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.619+0000 7fce4da00640 1 --2- 172.21.15.7:0/3157890494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce48160a10 0x7fce480d0d70 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fce380097a0 tx=0x7fce3802a230 comp rx=0 tx=0).stop 2024-04-25T11:20:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.619+0000 7fce4da00640 1 -- 172.21.15.7:0/3157890494 shutdown_connections 2024-04-25T11:20:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.619+0000 7fce4da00640 1 --2- 172.21.15.7:0/3157890494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce48160a10 0x7fce480d0d70 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.619+0000 7fce4da00640 1 -- 172.21.15.7:0/3157890494 >> 172.21.15.7:0/3157890494 conn(0x7fce48073090 msgr2=0x7fce480734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.619+0000 7fce4da00640 1 -- 172.21.15.7:0/3157890494 shutdown_connections 2024-04-25T11:20:38.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.619+0000 7fce4da00640 1 -- 172.21.15.7:0/3157890494 wait complete. 2024-04-25T11:20:38.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.623+0000 7fce4da00640 1 Processor -- start 2024-04-25T11:20:38.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.623+0000 7fce4da00640 1 -- start start 2024-04-25T11:20:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.625+0000 7fce4da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce4806d580 0x7fce481442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.625+0000 7fce4da00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480cf710 0x7fce481f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.625+0000 7fce4da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce48160a10 0x7fce48158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.625+0000 7fce4da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce4815c650 con 0x7fce48160a10 2024-04-25T11:20:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.625+0000 7fce4da00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce480b50b0 con 0x7fce4806d580 2024-04-25T11:20:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.625+0000 7fce4da00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce4815c7d0 con 0x7fce480cf710 2024-04-25T11:20:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.625+0000 7fce4d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce48160a10 0x7fce48158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.625+0000 7fce4d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce48160a10 0x7fce48158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52680/0 (socket says 172.21.15.7:52680) 2024-04-25T11:20:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.625+0000 7fce4d000640 1 -- 172.21.15.7:0/4050724079 learned_addr learned my addr 172.21.15.7:0/4050724079 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.626+0000 7fce3fe00640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480cf710 0x7fce481f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.626+0000 7fce47e00640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce4806d580 0x7fce481442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.626+0000 7fce4d000640 1 -- 172.21.15.7:0/4050724079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480cf710 msgr2=0x7fce481f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.626+0000 7fce4d000640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480cf710 0x7fce481f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.626+0000 7fce4d000640 1 -- 172.21.15.7:0/4050724079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce4806d580 msgr2=0x7fce481442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.626+0000 7fce4d000640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce4806d580 0x7fce481442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.626+0000 7fce4d000640 1 -- 172.21.15.7:0/4050724079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce481f5260 con 0x7fce48160a10 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.626+0000 7fce4d000640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce48160a10 0x7fce48158920 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c680 tx=0x7fce4000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.627+0000 7fce44c00640 1 -- 172.21.15.7:0/4050724079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40003b70 con 0x7fce48160a10 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.627+0000 7fce44c00640 1 -- 172.21.15.7:0/4050724079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce40003d10 con 0x7fce48160a10 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.627+0000 7fce44c00640 1 -- 172.21.15.7:0/4050724079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce40005310 con 0x7fce48160a10 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.627+0000 7fce4da00640 1 -- 172.21.15.7:0/4050724079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce481f7c50 con 0x7fce48160a10 2024-04-25T11:20:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.627+0000 7fce4da00640 1 -- 172.21.15.7:0/4050724079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce481496c0 con 0x7fce48160a10 2024-04-25T11:20:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.628+0000 7fce4da00640 1 -- 172.21.15.7:0/4050724079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce48149920 con 0x7fce48160a10 2024-04-25T11:20:38.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.629+0000 7fce44c00640 1 -- 172.21.15.7:0/4050724079 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce400054b0 con 0x7fce48160a10 2024-04-25T11:20:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.631+0000 7fce44c00640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce240744e0 0x7fce240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:38.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.631+0000 7fce44c00640 1 -- 172.21.15.7:0/4050724079 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5249+0+0 (secure 0 0 0) 0x7fce40094f40 con 0x7fce48160a10 2024-04-25T11:20:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.631+0000 7fce47e00640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce240744e0 0x7fce240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:38.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.632+0000 7fce47e00640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce240744e0 0x7fce240769f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fce38034000 tx=0x7fce3802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:38.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.635+0000 7fce44c00640 1 -- 172.21.15.7:0/4050724079 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce4005f7f0 con 0x7fce48160a10 2024-04-25T11:20:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.910+0000 7fce4da00640 1 -- 172.21.15.7:0/4050724079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fce48050740 con 0x7fce48160a10 2024-04-25T11:20:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.911+0000 7fce44c00640 1 -- 172.21.15.7:0/4050724079 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7fce4005f0f0 con 0x7fce48160a10 2024-04-25T11:20:38.907 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w==", 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.908 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-04-25T11:20:38.909 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw==", 2024-04-25T11:20:38.909 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.909 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:20:38.909 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:20:38.909 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:20:38.909 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.909 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-04-25T11:20:38.909 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw==", 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A==", 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:20:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA==", 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg==", 2024-04-25T11:20:38.911 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw==", 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.912 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ==", 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw==", 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:20:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-04-25T11:20:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng==", 2024-04-25T11:20:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:20:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-04-25T11:20:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ==", 2024-04-25T11:20:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:20:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-04-25T11:20:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A==", 2024-04-25T11:20:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg==", 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA==", 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg==", 2024-04-25T11:20:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ==", 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:20:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDZOSpmS8ObHxAAkXXXg1OI1YFC2/OZER3woA==", 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-04-25T11:20:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw==", 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w==", 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-25T11:20:38.922 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:20:38.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 -- 172.21.15.7:0/4050724079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce240744e0 msgr2=0x7fce240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce240744e0 0x7fce240769f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fce38034000 tx=0x7fce3802e000 comp rx=0 tx=0).stop 2024-04-25T11:20:38.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 -- 172.21.15.7:0/4050724079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce48160a10 msgr2=0x7fce48158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:38.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce48160a10 0x7fce48158920 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c680 tx=0x7fce4000cb40 comp rx=0 tx=0).stop 2024-04-25T11:20:38.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 -- 172.21.15.7:0/4050724079 shutdown_connections 2024-04-25T11:20:38.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce240744e0 0x7fce240769f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce48160a10 0x7fce48158920 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480cf710 0x7fce481f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 --2- 172.21.15.7:0/4050724079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce4806d580 0x7fce481442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:38.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 -- 172.21.15.7:0/4050724079 >> 172.21.15.7:0/4050724079 conn(0x7fce48073090 msgr2=0x7fce4815fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:38.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 -- 172.21.15.7:0/4050724079 shutdown_connections 2024-04-25T11:20:38.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:38.913+0000 7fce4da00640 1 -- 172.21.15.7:0/4050724079 wait complete. 2024-04-25T11:20:38.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.215+0000 7f8132200640 1 Processor -- start 2024-04-25T11:20:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.215+0000 7f8132200640 1 -- start start 2024-04-25T11:20:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.217+0000 7f8132200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f812c0d2d90 0x7f812c09ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.217+0000 7f8132200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c0d31a0 0x7f812c09bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.217+0000 7f8132200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f812c0d1cf0 con 0x7f812c0cddb0 2024-04-25T11:20:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.217+0000 7f8132200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f812c0d1fd0 con 0x7f812c0d2d90 2024-04-25T11:20:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.217+0000 7f8132200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f812c0d22b0 con 0x7f812c0d31a0 2024-04-25T11:20:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.218+0000 7f8131800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f812c0cddb0 0x7f812c0ce1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58380/0 (socket says 172.21.15.7:58380) 2024-04-25T11:20:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.218+0000 7f8131800640 1 -- 172.21.15.7:0/4078914829 learned_addr learned my addr 172.21.15.7:0/4078914829 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.218+0000 7f8130e00640 1 --2- 172.21.15.7:0/4078914829 >> v2:172.21.15.87:3300/0 conn(0x7f812c0d2d90 0x7f812c09ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.218+0000 7f812be00640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c0d31a0 0x7f812c09bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.218+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403917538 0 0) 0x7f812c0d1cf0 con 0x7f812c0cddb0 2024-04-25T11:20:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.218+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8114003520 con 0x7f812c0cddb0 2024-04-25T11:20:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2260059370 0 0) 0x7f8114003520 con 0x7f812c0cddb0 2024-04-25T11:20:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c0d31a0 msgr2=0x7f812c09bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c0d31a0 0x7f812c09bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 >> v2:172.21.15.87:3300/0 conn(0x7f812c0d2d90 msgr2=0x7f812c09ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 --2- 172.21.15.7:0/4078914829 >> v2:172.21.15.87:3300/0 conn(0x7f812c0d2d90 0x7f812c09ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812c0d2590 con 0x7f812c0cddb0 2024-04-25T11:20:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f81240019f0 con 0x7f812c0cddb0 2024-04-25T11:20:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 >> v1:172.21.15.7:6789/0 conn(0x7f812c0cddb0 legacy=0x7f812c0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c0d31a0 0x7f81140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8114003f80 0x7f8114004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8114004d40 0x7f8114005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f812c0d1fd0 con 0x7f8114004d40 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f812c0d22b0 con 0x7f8114003f80 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f811400ba70 con 0x7f812c0d31a0 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.219+0000 7f8131800640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8114004d40 0x7f8114005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.220+0000 7f8130e00640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c0d31a0 0x7f81140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.220+0000 7f812be00640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8114003f80 0x7f8114004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.220+0000 7f8131800640 1 -- 172.21.15.7:0/4078914829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c0d31a0 msgr2=0x7f81140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.220+0000 7f8131800640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c0d31a0 0x7f81140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.220+0000 7f8131800640 1 -- 172.21.15.7:0/4078914829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8114003f80 msgr2=0x7f8114004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.220+0000 7f8131800640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8114003f80 0x7f8114004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.220+0000 7f8131800640 1 -- 172.21.15.7:0/4078914829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8114003520 con 0x7f8114004d40 2024-04-25T11:20:39.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.221+0000 7f8131800640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8114004d40 0x7f8114005150 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f812c164d90 tx=0x7f8114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd42186ea79efe79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:39.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.221+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812400a3f0 con 0x7f8114004d40 2024-04-25T11:20:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.221+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f812402f070 con 0x7f8114004d40 2024-04-25T11:20:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.221+0000 7f812b400640 1 -- 172.21.15.7:0/4078914829 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812402bc00 con 0x7f8114004d40 2024-04-25T11:20:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.222+0000 7f8132200640 1 -- 172.21.15.7:0/4078914829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8114004d40 msgr2=0x7f8114005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.222+0000 7f8132200640 1 --2- 172.21.15.7:0/4078914829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8114004d40 0x7f8114005150 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f812c164d90 tx=0x7f8114005830 comp rx=0 tx=0).stop 2024-04-25T11:20:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.222+0000 7f8131800640 1 -- 172.21.15.7:0/4078914829 reap_dead start 2024-04-25T11:20:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.222+0000 7f8132200640 1 -- 172.21.15.7:0/4078914829 shutdown_connections 2024-04-25T11:20:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.222+0000 7f8132200640 1 -- 172.21.15.7:0/4078914829 >> 172.21.15.7:0/4078914829 conn(0x7f812c0b5250 msgr2=0x7f812c0b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.222+0000 7f8132200640 1 -- 172.21.15.7:0/4078914829 shutdown_connections 2024-04-25T11:20:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.222+0000 7f8132200640 1 -- 172.21.15.7:0/4078914829 wait complete. 2024-04-25T11:20:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.225+0000 7f8132200640 1 Processor -- start 2024-04-25T11:20:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.225+0000 7f8132200640 1 -- start start 2024-04-25T11:20:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.226+0000 7f8132200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f812c156c10 0x7f812c0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.226+0000 7f8132200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f812c0cb190 0x7f812c1611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.226+0000 7f8132200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c1616f0 0x7f812c162260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.226+0000 7f8132200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f812c0b91e0 con 0x7f812c0cb190 2024-04-25T11:20:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.226+0000 7f8132200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f812c0d19a0 con 0x7f812c156c10 2024-04-25T11:20:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.226+0000 7f8132200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f812c09bf40 con 0x7f812c1616f0 2024-04-25T11:20:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.227+0000 7f812be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f812c0cb190 0x7f812c1611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.227+0000 7f812be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f812c0cb190 0x7f812c1611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52698/0 (socket says 172.21.15.7:52698) 2024-04-25T11:20:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.227+0000 7f812be00640 1 -- 172.21.15.7:0/2756094703 learned_addr learned my addr 172.21.15.7:0/2756094703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.227+0000 7f8130e00640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f812c156c10 0x7f812c0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.227+0000 7f8131800640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c1616f0 0x7f812c162260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.227+0000 7f812be00640 1 -- 172.21.15.7:0/2756094703 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c1616f0 msgr2=0x7f812c162260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.227+0000 7f812be00640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c1616f0 0x7f812c162260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.227+0000 7f812be00640 1 -- 172.21.15.7:0/2756094703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f812c156c10 msgr2=0x7f812c0cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.227+0000 7f812be00640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f812c156c10 0x7f812c0cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.227+0000 7f812be00640 1 -- 172.21.15.7:0/2756094703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812c0c7290 con 0x7f812c0cb190 2024-04-25T11:20:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.228+0000 7f812be00640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f812c0cb190 0x7f812c1611b0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f8118011600 tx=0x7f8118011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.228+0000 7f8128c00640 1 -- 172.21.15.7:0/2756094703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811800f920 con 0x7f812c0cb190 2024-04-25T11:20:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.228+0000 7f8128c00640 1 -- 172.21.15.7:0/2756094703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f811800fac0 con 0x7f812c0cb190 2024-04-25T11:20:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.228+0000 7f8128c00640 1 -- 172.21.15.7:0/2756094703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811800fe70 con 0x7f812c0cb190 2024-04-25T11:20:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.228+0000 7f8132200640 1 -- 172.21.15.7:0/2756094703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812c0c7550 con 0x7f812c0cb190 2024-04-25T11:20:39.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.228+0000 7f8132200640 1 -- 172.21.15.7:0/2756094703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812c0c7b10 con 0x7f812c0cb190 2024-04-25T11:20:39.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.229+0000 7f8132200640 1 -- 172.21.15.7:0/2756094703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f812c09c6e0 con 0x7f812c0cb190 2024-04-25T11:20:39.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.231+0000 7f8128c00640 1 -- 172.21.15.7:0/2756094703 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8118015460 con 0x7f812c0cb190 2024-04-25T11:20:39.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.231+0000 7f8128c00640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f810c0744b0 0x7f810c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.232+0000 7f8128c00640 1 -- 172.21.15.7:0/2756094703 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5249+0+0 (secure 0 0 0) 0x7f8118095600 con 0x7f812c0cb190 2024-04-25T11:20:39.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.232+0000 7f8130e00640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f810c0744b0 0x7f810c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.232+0000 7f8130e00640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f810c0744b0 0x7f810c0769c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f812c0a1650 tx=0x7f8120008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:39.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.238+0000 7f8128c00640 1 -- 172.21.15.7:0/2756094703 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8118019030 con 0x7f812c0cb190 2024-04-25T11:20:39.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.444+0000 7f8132200640 1 -- 172.21.15.7:0/2756094703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f812c0b5940 con 0x7f812c0cb190 2024-04-25T11:20:39.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.446+0000 7f8128c00640 1 -- 172.21.15.7:0/2756094703 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+13078 (secure 0 0 0) 0x7f811805ffd0 con 0x7f812c0cb190 2024-04-25T11:20:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:39.443 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":126,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:20:38.303762+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:39.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.448+0000 7f8132200640 1 -- 172.21.15.7:0/2756094703 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f810c0744b0 msgr2=0x7f810c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.448+0000 7f8132200640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f810c0744b0 0x7f810c0769c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f812c0a1650 tx=0x7f8120008040 comp rx=0 tx=0).stop 2024-04-25T11:20:39.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.448+0000 7f8132200640 1 -- 172.21.15.7:0/2756094703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f812c0cb190 msgr2=0x7f812c1611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.448+0000 7f8132200640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f812c0cb190 0x7f812c1611b0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f8118011600 tx=0x7f8118011ac0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.448+0000 7f8132200640 1 -- 172.21.15.7:0/2756094703 shutdown_connections 2024-04-25T11:20:39.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.448+0000 7f8132200640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f810c0744b0 0x7f810c0769c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.448+0000 7f8132200640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f812c1616f0 0x7f812c162260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.448+0000 7f8132200640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f812c0cb190 0x7f812c1611b0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.448+0000 7f8132200640 1 --2- 172.21.15.7:0/2756094703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f812c156c10 0x7f812c0cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.449+0000 7f8132200640 1 -- 172.21.15.7:0/2756094703 >> 172.21.15.7:0/2756094703 conn(0x7f812c0b5250 msgr2=0x7f812c072820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:39.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.449+0000 7f8132200640 1 -- 172.21.15.7:0/2756094703 shutdown_connections 2024-04-25T11:20:39.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.449+0000 7f8132200640 1 -- 172.21.15.7:0/2756094703 wait complete. 2024-04-25T11:20:39.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:20:39.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:20:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.753+0000 7f0636200640 1 Processor -- start 2024-04-25T11:20:39.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.753+0000 7f0636200640 1 -- start start 2024-04-25T11:20:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.756+0000 7f0636200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0630146180 0x7f0630070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.756+0000 7f0636200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f06300b08e0 0x7f0630149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.756+0000 7f0636200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0630070860 con 0x7f06301490f0 2024-04-25T11:20:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.756+0000 7f0636200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0630146590 con 0x7f0630146180 2024-04-25T11:20:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.756+0000 7f0636200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f063006f280 con 0x7f06300b08e0 2024-04-25T11:20:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.756+0000 7f0634e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0630146180 0x7f0630070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.756+0000 7f062fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f06301490f0 0x7f06300b01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58384/0 (socket says 172.21.15.7:58384) 2024-04-25T11:20:39.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.756+0000 7f062fe00640 1 -- 172.21.15.7:0/576048964 learned_addr learned my addr 172.21.15.7:0/576048964 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.757+0000 7f0635800640 1 --2- 172.21.15.7:0/576048964 >> v2:172.21.15.87:3300/0 conn(0x7f06300b08e0 0x7f0630149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.757+0000 7f0634e00640 1 -- 172.21.15.7:0/576048964 >> v2:172.21.15.87:3300/0 conn(0x7f06300b08e0 msgr2=0x7f0630149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.757+0000 7f0634e00640 1 --2- 172.21.15.7:0/576048964 >> v2:172.21.15.87:3300/0 conn(0x7f06300b08e0 0x7f0630149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.757+0000 7f0634e00640 1 -- 172.21.15.7:0/576048964 >> v1:172.21.15.7:6789/0 conn(0x7f06301490f0 legacy=0x7f06300b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.757+0000 7f0634e00640 1 -- 172.21.15.7:0/576048964 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063006f560 con 0x7f0630146180 2024-04-25T11:20:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.757+0000 7f062f400640 1 -- 172.21.15.7:0/576048964 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285714502 0 0) 0x7f0630070860 con 0x7f06301490f0 2024-04-25T11:20:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.757+0000 7f0635800640 1 --2- 172.21.15.7:0/576048964 >> v2:172.21.15.87:3300/0 conn(0x7f06300b08e0 0x7f0630149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.758+0000 7f0634e00640 1 --2- 172.21.15.7:0/576048964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0630146180 0x7f0630070320 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f0630164a70 tx=0x7f062402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c418440799d6da4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.758+0000 7f062f400640 1 -- 172.21.15.7:0/576048964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624037070 con 0x7f0630146180 2024-04-25T11:20:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.758+0000 7f062f400640 1 -- 172.21.15.7:0/576048964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0624032440 con 0x7f0630146180 2024-04-25T11:20:39.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.758+0000 7f062f400640 1 -- 172.21.15.7:0/576048964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624032810 con 0x7f0630146180 2024-04-25T11:20:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.759+0000 7f0636200640 1 -- 172.21.15.7:0/576048964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0630146180 msgr2=0x7f0630070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.759+0000 7f0636200640 1 --2- 172.21.15.7:0/576048964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0630146180 0x7f0630070320 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f0630164a70 tx=0x7f062402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.759+0000 7f0636200640 1 -- 172.21.15.7:0/576048964 shutdown_connections 2024-04-25T11:20:39.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.759+0000 7f0636200640 1 --2- 172.21.15.7:0/576048964 >> v2:172.21.15.87:3300/0 conn(0x7f06300b08e0 0x7f0630149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.759+0000 7f0636200640 1 --2- 172.21.15.7:0/576048964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0630146180 0x7f0630070320 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.759+0000 7f0636200640 1 -- 172.21.15.7:0/576048964 >> 172.21.15.7:0/576048964 conn(0x7f06300740f0 msgr2=0x7f063006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:39.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.759+0000 7f0636200640 1 -- 172.21.15.7:0/576048964 shutdown_connections 2024-04-25T11:20:39.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.759+0000 7f0636200640 1 -- 172.21.15.7:0/576048964 wait complete. 2024-04-25T11:20:39.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.762+0000 7f0636200640 1 Processor -- start 2024-04-25T11:20:39.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.762+0000 7f0636200640 1 -- start start 2024-04-25T11:20:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.763+0000 7f0636200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06300b08e0 0x7f06301483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.763+0000 7f0636200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0630146180 0x7f06301410c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.763+0000 7f0636200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06301490f0 0x7f0630141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.763+0000 7f0636200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f063006fe00 con 0x7f0630146180 2024-04-25T11:20:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.763+0000 7f0636200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f063006fb20 con 0x7f06300b08e0 2024-04-25T11:20:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.763+0000 7f0636200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f063006eea0 con 0x7f06301490f0 2024-04-25T11:20:39.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.763+0000 7f062fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0630146180 0x7f06301410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.764+0000 7f062fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0630146180 0x7f06301410c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52704/0 (socket says 172.21.15.7:52704) 2024-04-25T11:20:39.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.764+0000 7f062fe00640 1 -- 172.21.15.7:0/521596285 learned_addr learned my addr 172.21.15.7:0/521596285 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.764+0000 7f062fe00640 1 -- 172.21.15.7:0/521596285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06301490f0 msgr2=0x7f0630141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.764+0000 7f062fe00640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06301490f0 0x7f0630141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.764+0000 7f062fe00640 1 -- 172.21.15.7:0/521596285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06300b08e0 msgr2=0x7f06301483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.764+0000 7f062fe00640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06300b08e0 0x7f06301483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.764+0000 7f062fe00640 1 -- 172.21.15.7:0/521596285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0630161a60 con 0x7f0630146180 2024-04-25T11:20:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.764+0000 7f062fe00640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0630146180 0x7f06301410c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f061c00c460 tx=0x7f061c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.764+0000 7f062cc00640 1 -- 172.21.15.7:0/521596285 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c018070 con 0x7f0630146180 2024-04-25T11:20:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.765+0000 7f062cc00640 1 -- 172.21.15.7:0/521596285 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c007500 con 0x7f0630146180 2024-04-25T11:20:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.765+0000 7f0636200640 1 -- 172.21.15.7:0/521596285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06301609c0 con 0x7f0630146180 2024-04-25T11:20:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.765+0000 7f062cc00640 1 -- 172.21.15.7:0/521596285 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c004a20 con 0x7f0630146180 2024-04-25T11:20:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.765+0000 7f0636200640 1 -- 172.21.15.7:0/521596285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0630160f80 con 0x7f0630146180 2024-04-25T11:20:39.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.766+0000 7f0636200640 1 -- 172.21.15.7:0/521596285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0630148990 con 0x7f0630146180 2024-04-25T11:20:39.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.767+0000 7f062cc00640 1 -- 172.21.15.7:0/521596285 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f061c01b070 con 0x7f0630146180 2024-04-25T11:20:39.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.768+0000 7f062cc00640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06100744e0 0x7f06100769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:39.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.768+0000 7f062cc00640 1 -- 172.21.15.7:0/521596285 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5249+0+0 (secure 0 0 0) 0x7f061c094320 con 0x7f0630146180 2024-04-25T11:20:39.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.768+0000 7f0634e00640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06100744e0 0x7f06100769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:39.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.769+0000 7f0634e00640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06100744e0 0x7f06100769f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0624034000 tx=0x7f062402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:39.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.773+0000 7f062cc00640 1 -- 172.21.15.7:0/521596285 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f061c05ec60 con 0x7f0630146180 2024-04-25T11:20:39.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.967+0000 7f0636200640 1 -- 172.21.15.7:0/521596285 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0630050740 con 0x7f0630146180 2024-04-25T11:20:39.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.968+0000 7f062cc00640 1 -- 172.21.15.7:0/521596285 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) ==== 93+0+10 (secure 0 0 0) 0x7f061c05e560 con 0x7f0630146180 2024-04-25T11:20:39.965 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:20:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 -- 172.21.15.7:0/521596285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06100744e0 msgr2=0x7f06100769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06100744e0 0x7f06100769f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0624034000 tx=0x7f062402c000 comp rx=0 tx=0).stop 2024-04-25T11:20:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 -- 172.21.15.7:0/521596285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0630146180 msgr2=0x7f06301410c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0630146180 0x7f06301410c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f061c00c460 tx=0x7f061c00c920 comp rx=0 tx=0).stop 2024-04-25T11:20:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 -- 172.21.15.7:0/521596285 shutdown_connections 2024-04-25T11:20:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06100744e0 0x7f06100769f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06301490f0 0x7f0630141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0630146180 0x7f06301410c0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 --2- 172.21.15.7:0/521596285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06300b08e0 0x7f06301483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:39.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 -- 172.21.15.7:0/521596285 >> 172.21.15.7:0/521596285 conn(0x7f06300740f0 msgr2=0x7f0630163960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:39.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 -- 172.21.15.7:0/521596285 shutdown_connections 2024-04-25T11:20:39.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:39.971+0000 7f0636200640 1 -- 172.21.15.7:0/521596285 wait complete. 2024-04-25T11:20:39.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:20:40.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.276+0000 7f5f67200640 1 Processor -- start 2024-04-25T11:20:40.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.276+0000 7f5f67200640 1 -- start start 2024-04-25T11:20:40.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.278+0000 7f5f67200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5f60160a10 0x7f5f600d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.278+0000 7f5f67200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f6006d580 0x7f5f6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.278+0000 7f5f67200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f6006ded0 con 0x7f5f6006f0f0 2024-04-25T11:20:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.279+0000 7f5f67200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5f600d12b0 con 0x7f5f60160a10 2024-04-25T11:20:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.279+0000 7f5f67200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5f60160e20 con 0x7f5f6006d580 2024-04-25T11:20:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.279+0000 7f5f66800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5f6006f0f0 0x7f5f6006f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48412/0 (socket says 172.21.15.7:48412) 2024-04-25T11:20:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.279+0000 7f5f66800640 1 -- 172.21.15.7:0/1284759748 learned_addr learned my addr 172.21.15.7:0/1284759748 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:40.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.279+0000 7f5f65e00640 1 --2- 172.21.15.7:0/1284759748 >> v2:172.21.15.87:3300/0 conn(0x7f5f60160a10 0x7f5f600d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.279+0000 7f5f65400640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f6006d580 0x7f5f6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.279+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1784346496 0 0) 0x7f5f5c001490 con 0x7f5f6006f0f0 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f48003520 con 0x7f5f6006f0f0 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3329479443 0 0) 0x7f5f48003520 con 0x7f5f6006f0f0 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f6006d580 msgr2=0x7f5f6006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f6006d580 0x7f5f6006d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 >> v2:172.21.15.87:3300/0 conn(0x7f5f60160a10 msgr2=0x7f5f600d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 --2- 172.21.15.7:0/1284759748 >> v2:172.21.15.87:3300/0 conn(0x7f5f60160a10 0x7f5f600d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f600d0720 con 0x7f5f6006f0f0 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f5f5c001bc0 con 0x7f5f6006f0f0 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 >> v1:172.21.15.7:6789/0 conn(0x7f5f6006f0f0 legacy=0x7f5f6006f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f480039b0 0x7f5f48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f48004800 0x7f5f48004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f6006d580 0x7f5f48005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5f60160e20 con 0x7f5f48004800 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4800dbd0 con 0x7f5f480039b0 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4800dd50 con 0x7f5f6006d580 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.280+0000 7f5f65400640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f48004800 0x7f5f48004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.281+0000 7f5f65e00640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f480039b0 0x7f5f48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.281+0000 7f5f66800640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f6006d580 0x7f5f48005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.281+0000 7f5f65400640 1 -- 172.21.15.7:0/1284759748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f6006d580 msgr2=0x7f5f48005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.281+0000 7f5f65400640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f6006d580 0x7f5f48005150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.281+0000 7f5f65400640 1 -- 172.21.15.7:0/1284759748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f480039b0 msgr2=0x7f5f48003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.281+0000 7f5f65400640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f480039b0 0x7f5f48003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.281+0000 7f5f65400640 1 -- 172.21.15.7:0/1284759748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f48003520 con 0x7f5f48004800 2024-04-25T11:20:40.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.282+0000 7f5f65400640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f48004800 0x7f5f48004c10 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009ac0 tx=0x7f5f48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74f9e94db2d315b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.282+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50036070 con 0x7f5f48004800 2024-04-25T11:20:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.282+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f50031400 con 0x7f5f48004800 2024-04-25T11:20:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.282+0000 7f5f64a00640 1 -- 172.21.15.7:0/1284759748 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50009690 con 0x7f5f48004800 2024-04-25T11:20:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.283+0000 7f5f67200640 1 -- 172.21.15.7:0/1284759748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f48004800 msgr2=0x7f5f48004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.283+0000 7f5f67200640 1 --2- 172.21.15.7:0/1284759748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f48004800 0x7f5f48004c10 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009ac0 tx=0x7f5f48005830 comp rx=0 tx=0).stop 2024-04-25T11:20:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.283+0000 7f5f66800640 1 -- 172.21.15.7:0/1284759748 reap_dead start 2024-04-25T11:20:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.283+0000 7f5f67200640 1 -- 172.21.15.7:0/1284759748 shutdown_connections 2024-04-25T11:20:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.283+0000 7f5f67200640 1 -- 172.21.15.7:0/1284759748 >> 172.21.15.7:0/1284759748 conn(0x7f5f60073090 msgr2=0x7f5f600734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.283+0000 7f5f67200640 1 -- 172.21.15.7:0/1284759748 shutdown_connections 2024-04-25T11:20:40.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.283+0000 7f5f67200640 1 -- 172.21.15.7:0/1284759748 wait complete. 2024-04-25T11:20:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.286+0000 7f5f67200640 1 Processor -- start 2024-04-25T11:20:40.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.286+0000 7f5f67200640 1 -- start start 2024-04-25T11:20:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.287+0000 7f5f67200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f48004800 0x7f5f60145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.287+0000 7f5f67200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f601483b0 0x7f5f601487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.287+0000 7f5f67200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f601f68c0 0x7f5f601f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.287+0000 7f5f67200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5f600b4dd0 con 0x7f5f601f68c0 2024-04-25T11:20:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.287+0000 7f5f67200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f600b4af0 con 0x7f5f601483b0 2024-04-25T11:20:40.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.287+0000 7f5f67200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5f600b50b0 con 0x7f5f48004800 2024-04-25T11:20:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.288+0000 7f5f66800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f601f68c0 0x7f5f601f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.288+0000 7f5f66800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f601f68c0 0x7f5f601f6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36894/0 (socket says 172.21.15.7:36894) 2024-04-25T11:20:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.288+0000 7f5f66800640 1 -- 172.21.15.7:0/3907430374 learned_addr learned my addr 172.21.15.7:0/3907430374 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.288+0000 7f5f66800640 1 -- 172.21.15.7:0/3907430374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f48004800 msgr2=0x7f5f60145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.288+0000 7f5f66800640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f48004800 0x7f5f60145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.288+0000 7f5f66800640 1 -- 172.21.15.7:0/3907430374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f601483b0 msgr2=0x7f5f601487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.288+0000 7f5f66800640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f601483b0 0x7f5f601487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.288+0000 7f5f66800640 1 -- 172.21.15.7:0/3907430374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f60149310 con 0x7f5f601f68c0 2024-04-25T11:20:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.288+0000 7f5f66800640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f601f68c0 0x7f5f601f6cd0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c000f80 tx=0x7f5f5c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.289+0000 7f5f5a000640 1 -- 172.21.15.7:0/3907430374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c00bb80 con 0x7f5f601f68c0 2024-04-25T11:20:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.289+0000 7f5f5a000640 1 -- 172.21.15.7:0/3907430374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f5c00bd40 con 0x7f5f601f68c0 2024-04-25T11:20:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.289+0000 7f5f5a000640 1 -- 172.21.15.7:0/3907430374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c0072b0 con 0x7f5f601f68c0 2024-04-25T11:20:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.289+0000 7f5f67200640 1 -- 172.21.15.7:0/3907430374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f60149030 con 0x7f5f601f68c0 2024-04-25T11:20:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.289+0000 7f5f67200640 1 -- 172.21.15.7:0/3907430374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6015ac60 con 0x7f5f601f68c0 2024-04-25T11:20:40.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.290+0000 7f5f67200640 1 -- 172.21.15.7:0/3907430374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f60148da0 con 0x7f5f601f68c0 2024-04-25T11:20:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.291+0000 7f5f5a000640 1 -- 172.21.15.7:0/3907430374 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f5c01b020 con 0x7f5f601f68c0 2024-04-25T11:20:40.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.292+0000 7f5f5a000640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f440744e0 0x7f5f440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.292+0000 7f5f5a000640 1 -- 172.21.15.7:0/3907430374 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5249+0+0 (secure 0 0 0) 0x7f5f5c0953a0 con 0x7f5f601f68c0 2024-04-25T11:20:40.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.293+0000 7f5f65e00640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f440744e0 0x7f5f440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.293+0000 7f5f65e00640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f440744e0 0x7f5f440769f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5f60073040 tx=0x7f5f54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.297+0000 7f5f5a000640 1 -- 172.21.15.7:0/3907430374 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f6006ded0 con 0x7f5f601f68c0 2024-04-25T11:20:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.506+0000 7f5f67200640 1 -- 172.21.15.7:0/3907430374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5f6015a790 con 0x7f5f601f68c0 2024-04-25T11:20:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.507+0000 7f5f5a000640 1 -- 172.21.15.7:0/3907430374 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v126) ==== 84+0+65 (secure 0 0 0) 0x7f5f5c05fc50 con 0x7f5f601f68c0 2024-04-25T11:20:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:40.504 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-25T11:20:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:20:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-25T11:20:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-25T11:20:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:20:40.505 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-25T11:20:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.510+0000 7f5f67200640 1 -- 172.21.15.7:0/3907430374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f440744e0 msgr2=0x7f5f440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.510+0000 7f5f67200640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f440744e0 0x7f5f440769f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5f60073040 tx=0x7f5f54008040 comp rx=0 tx=0).stop 2024-04-25T11:20:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.510+0000 7f5f67200640 1 -- 172.21.15.7:0/3907430374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f601f68c0 msgr2=0x7f5f601f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.510+0000 7f5f67200640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f601f68c0 0x7f5f601f6cd0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c000f80 tx=0x7f5f5c00b4a0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.510+0000 7f5f67200640 1 -- 172.21.15.7:0/3907430374 shutdown_connections 2024-04-25T11:20:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.510+0000 7f5f67200640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f440744e0 0x7f5f440769f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.510+0000 7f5f67200640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f601f68c0 0x7f5f601f6cd0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.510+0000 7f5f67200640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f601483b0 0x7f5f601487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.510+0000 7f5f67200640 1 --2- 172.21.15.7:0/3907430374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f48004800 0x7f5f60145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.510+0000 7f5f67200640 1 -- 172.21.15.7:0/3907430374 >> 172.21.15.7:0/3907430374 conn(0x7f5f60073090 msgr2=0x7f5f6006cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.510+0000 7f5f67200640 1 -- 172.21.15.7:0/3907430374 shutdown_connections 2024-04-25T11:20:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.511+0000 7f5f67200640 1 -- 172.21.15.7:0/3907430374 wait complete. 2024-04-25T11:20:40.534 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:20:40.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:20:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.813+0000 7f6a96000640 1 Processor -- start 2024-04-25T11:20:40.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.813+0000 7f6a96000640 1 -- start start 2024-04-25T11:20:40.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.815+0000 7f6a96000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a900b48a0 0x7f6a9015dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.815+0000 7f6a96000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6a900aec60 0x7f6a900b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.816+0000 7f6a96000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a900b4da0 con 0x7f6a900b0940 2024-04-25T11:20:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.816+0000 7f6a96000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6a900b5080 con 0x7f6a900b48a0 2024-04-25T11:20:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.816+0000 7f6a96000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6a900b5360 con 0x7f6a900aec60 2024-04-25T11:20:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.816+0000 7f6a8f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a900b48a0 0x7f6a9015dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.816+0000 7f6a8f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a900b48a0 0x7f6a9015dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36906/0 (socket says 172.21.15.7:36906) 2024-04-25T11:20:40.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.816+0000 7f6a8f400640 1 -- 172.21.15.7:0/3509957711 learned_addr learned my addr 172.21.15.7:0/3509957711 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.816+0000 7f6a8ea00640 1 --2- 172.21.15.7:0/3509957711 >> v2:172.21.15.87:3300/0 conn(0x7f6a900aec60 0x7f6a900b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.817+0000 7f6a8f400640 1 -- 172.21.15.7:0/3509957711 >> v2:172.21.15.87:3300/0 conn(0x7f6a900aec60 msgr2=0x7f6a900b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.817+0000 7f6a8f400640 1 --2- 172.21.15.7:0/3509957711 >> v2:172.21.15.87:3300/0 conn(0x7f6a900aec60 0x7f6a900b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.817+0000 7f6a8f400640 1 -- 172.21.15.7:0/3509957711 >> v1:172.21.15.87:6789/0 conn(0x7f6a900b0940 legacy=0x7f6a900b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.817+0000 7f6a8f400640 1 -- 172.21.15.7:0/3509957711 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a900b5640 con 0x7f6a900b48a0 2024-04-25T11:20:40.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.818+0000 7f6a8f400640 1 --2- 172.21.15.7:0/3509957711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a900b48a0 0x7f6a9015dc90 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f6a90074770 tx=0x7f6a8002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7123845fe103200c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:40.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.818+0000 7f6a8e000640 1 -- 172.21.15.7:0/3509957711 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80037070 con 0x7f6a900b48a0 2024-04-25T11:20:40.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.818+0000 7f6a8e000640 1 -- 172.21.15.7:0/3509957711 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a80032440 con 0x7f6a900b48a0 2024-04-25T11:20:40.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.818+0000 7f6a8e000640 1 -- 172.21.15.7:0/3509957711 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80032810 con 0x7f6a900b48a0 2024-04-25T11:20:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.819+0000 7f6a96000640 1 -- 172.21.15.7:0/3509957711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a900b48a0 msgr2=0x7f6a9015dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.819+0000 7f6a96000640 1 --2- 172.21.15.7:0/3509957711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a900b48a0 0x7f6a9015dc90 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f6a90074770 tx=0x7f6a8002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.819+0000 7f6a96000640 1 -- 172.21.15.7:0/3509957711 shutdown_connections 2024-04-25T11:20:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.819+0000 7f6a96000640 1 --2- 172.21.15.7:0/3509957711 >> v2:172.21.15.87:3300/0 conn(0x7f6a900aec60 0x7f6a900b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.819+0000 7f6a96000640 1 --2- 172.21.15.7:0/3509957711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a900b48a0 0x7f6a9015dc90 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.819+0000 7f6a96000640 1 -- 172.21.15.7:0/3509957711 >> 172.21.15.7:0/3509957711 conn(0x7f6a90073920 msgr2=0x7f6a90073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:40.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.819+0000 7f6a96000640 1 -- 172.21.15.7:0/3509957711 shutdown_connections 2024-04-25T11:20:40.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.819+0000 7f6a96000640 1 -- 172.21.15.7:0/3509957711 wait complete. 2024-04-25T11:20:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.822+0000 7f6a96000640 1 Processor -- start 2024-04-25T11:20:40.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.822+0000 7f6a96000640 1 -- start start 2024-04-25T11:20:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.823+0000 7f6a96000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900aec60 0x7f6a901554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.823+0000 7f6a96000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a900b0940 0x7f6a901607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.823+0000 7f6a96000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a900b48a0 0x7f6a90160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.823+0000 7f6a96000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a900b6380 con 0x7f6a900b0940 2024-04-25T11:20:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.823+0000 7f6a96000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9015e1d0 con 0x7f6a900aec60 2024-04-25T11:20:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.823+0000 7f6a96000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6a900b03b0 con 0x7f6a900b48a0 2024-04-25T11:20:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.824+0000 7f6a8ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a900b0940 0x7f6a901607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.824+0000 7f6a8ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a900b0940 0x7f6a901607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36908/0 (socket says 172.21.15.7:36908) 2024-04-25T11:20:40.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.824+0000 7f6a8ea00640 1 -- 172.21.15.7:0/142278842 learned_addr learned my addr 172.21.15.7:0/142278842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:40.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.824+0000 7f6a8f400640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900aec60 0x7f6a901554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.824+0000 7f6a8ea00640 1 -- 172.21.15.7:0/142278842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a900b48a0 msgr2=0x7f6a90160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:40.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.824+0000 7f6a8ea00640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a900b48a0 0x7f6a90160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.824+0000 7f6a8ea00640 1 -- 172.21.15.7:0/142278842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900aec60 msgr2=0x7f6a901554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:40.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.824+0000 7f6a8ea00640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900aec60 0x7f6a901554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:40.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.824+0000 7f6a8ea00640 1 -- 172.21.15.7:0/142278842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a90155d40 con 0x7f6a900b0940 2024-04-25T11:20:40.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.824+0000 7f6a8ea00640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a900b0940 0x7f6a901607b0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00d5f0 tx=0x7f6a7c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:40.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.825+0000 7f6a87400640 1 -- 172.21.15.7:0/142278842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0125f0 con 0x7f6a900b0940 2024-04-25T11:20:40.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.825+0000 7f6a87400640 1 -- 172.21.15.7:0/142278842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a7c019070 con 0x7f6a900b0940 2024-04-25T11:20:40.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.825+0000 7f6a87400640 1 -- 172.21.15.7:0/142278842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0129d0 con 0x7f6a900b0940 2024-04-25T11:20:40.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.825+0000 7f6a96000640 1 -- 172.21.15.7:0/142278842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9015e570 con 0x7f6a900b0940 2024-04-25T11:20:40.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.825+0000 7f6a96000640 1 -- 172.21.15.7:0/142278842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9015eb30 con 0x7f6a900b0940 2024-04-25T11:20:40.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.826+0000 7f6a96000640 1 -- 172.21.15.7:0/142278842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a90155ab0 con 0x7f6a900b0940 2024-04-25T11:20:40.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.827+0000 7f6a87400640 1 -- 172.21.15.7:0/142278842 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6a7c012b70 con 0x7f6a900b0940 2024-04-25T11:20:40.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.828+0000 7f6a87400640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a700744e0 0x7f6a700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:40.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.828+0000 7f6a87400640 1 -- 172.21.15.7:0/142278842 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5249+0+0 (secure 0 0 0) 0x7f6a7c0954f0 con 0x7f6a900b0940 2024-04-25T11:20:40.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.829+0000 7f6a8f400640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a700744e0 0x7f6a700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:40.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.829+0000 7f6a8f400640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a700744e0 0x7f6a700769f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6a80034000 tx=0x7f6a8002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:40.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:40.834+0000 7f6a87400640 1 -- 172.21.15.7:0/142278842 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6a7c05fda0 con 0x7f6a900b0940 2024-04-25T11:20:41.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.030+0000 7f6a96000640 1 -- 172.21.15.7:0/142278842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6a9006a100 con 0x7f6a900b0940 2024-04-25T11:20:41.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.328+0000 7f6a87400640 1 -- 172.21.15.7:0/142278842 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v127) ==== 138+0+0 (secure 0 0 0) 0x7f6a7c05f6a0 con 0x7f6a900b0940 2024-04-25T11:20:41.325 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-25T11:20:41.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 -- 172.21.15.7:0/142278842 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a700744e0 msgr2=0x7f6a700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a700744e0 0x7f6a700769f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6a80034000 tx=0x7f6a8002e000 comp rx=0 tx=0).stop 2024-04-25T11:20:41.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 -- 172.21.15.7:0/142278842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a900b0940 msgr2=0x7f6a901607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a900b0940 0x7f6a901607b0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00d5f0 tx=0x7f6a7c00dab0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 -- 172.21.15.7:0/142278842 shutdown_connections 2024-04-25T11:20:41.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6a700744e0 0x7f6a700769f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6a900b48a0 0x7f6a90160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a900b0940 0x7f6a901607b0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 --2- 172.21.15.7:0/142278842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a900aec60 0x7f6a901554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 -- 172.21.15.7:0/142278842 >> 172.21.15.7:0/142278842 conn(0x7f6a90073920 msgr2=0x7f6a90156c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:41.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 -- 172.21.15.7:0/142278842 shutdown_connections 2024-04-25T11:20:41.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.332+0000 7f6a96000640 1 -- 172.21.15.7:0/142278842 wait complete. 2024-04-25T11:20:41.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:20:41.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.653+0000 7fc143200640 1 Processor -- start 2024-04-25T11:20:41.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.654+0000 7fc143200640 1 -- start start 2024-04-25T11:20:41.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.656+0000 7fc143200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc144145e40 0x7fc144146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.656+0000 7fc143200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc1440b4870 0x7fc1440b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.656+0000 7fc143200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1440b66b0 con 0x7fc1440b51c0 2024-04-25T11:20:41.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.656+0000 7fc143200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc144145b70 con 0x7fc144145e40 2024-04-25T11:20:41.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.656+0000 7fc143200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc1440b5b30 con 0x7fc1440b4870 2024-04-25T11:20:41.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.656+0000 7fc141e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc144145e40 0x7fc144146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.656+0000 7fc141e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc144145e40 0x7fc144146250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36916/0 (socket says 172.21.15.7:36916) 2024-04-25T11:20:41.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.656+0000 7fc141e00640 1 -- 172.21.15.7:0/2916183180 learned_addr learned my addr 172.21.15.7:0/2916183180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:41.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.657+0000 7fc13be00640 1 --2- 172.21.15.7:0/2916183180 >> v2:172.21.15.87:3300/0 conn(0x7fc1440b4870 0x7fc1440b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.657+0000 7fc141e00640 1 -- 172.21.15.7:0/2916183180 >> v2:172.21.15.87:3300/0 conn(0x7fc1440b4870 msgr2=0x7fc1440b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:41.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.657+0000 7fc141e00640 1 --2- 172.21.15.7:0/2916183180 >> v2:172.21.15.87:3300/0 conn(0x7fc1440b4870 0x7fc1440b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.657+0000 7fc141e00640 1 -- 172.21.15.7:0/2916183180 >> v1:172.21.15.87:6789/0 conn(0x7fc1440b51c0 legacy=0x7fc144143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:41.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.657+0000 7fc141e00640 1 -- 172.21.15.7:0/2916183180 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1440b5e10 con 0x7fc144145e40 2024-04-25T11:20:41.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.657+0000 7fc13be00640 1 --2- 172.21.15.7:0/2916183180 >> v2:172.21.15.87:3300/0 conn(0x7fc1440b4870 0x7fc1440b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:41.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.658+0000 7fc141e00640 1 --2- 172.21.15.7:0/2916183180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144145e40 0x7fc144146250 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc14406ded0 tx=0x7fc13402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c0d2aedd8e38b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:41.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.658+0000 7fc141400640 1 -- 172.21.15.7:0/2916183180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134037070 con 0x7fc144145e40 2024-04-25T11:20:41.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.658+0000 7fc141400640 1 -- 172.21.15.7:0/2916183180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc134032440 con 0x7fc144145e40 2024-04-25T11:20:41.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.658+0000 7fc141400640 1 -- 172.21.15.7:0/2916183180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134032810 con 0x7fc144145e40 2024-04-25T11:20:41.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.659+0000 7fc143200640 1 -- 172.21.15.7:0/2916183180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144145e40 msgr2=0x7fc144146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.659+0000 7fc143200640 1 --2- 172.21.15.7:0/2916183180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144145e40 0x7fc144146250 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc14406ded0 tx=0x7fc13402a180 comp rx=0 tx=0).stop 2024-04-25T11:20:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.659+0000 7fc143200640 1 -- 172.21.15.7:0/2916183180 shutdown_connections 2024-04-25T11:20:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.659+0000 7fc143200640 1 --2- 172.21.15.7:0/2916183180 >> v2:172.21.15.87:3300/0 conn(0x7fc1440b4870 0x7fc1440b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.659+0000 7fc143200640 1 --2- 172.21.15.7:0/2916183180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc144145e40 0x7fc144146250 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.659+0000 7fc143200640 1 -- 172.21.15.7:0/2916183180 >> 172.21.15.7:0/2916183180 conn(0x7fc1440740f0 msgr2=0x7fc14406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.659+0000 7fc143200640 1 -- 172.21.15.7:0/2916183180 shutdown_connections 2024-04-25T11:20:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.659+0000 7fc143200640 1 -- 172.21.15.7:0/2916183180 wait complete. 2024-04-25T11:20:41.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.662+0000 7fc143200640 1 Processor -- start 2024-04-25T11:20:41.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.662+0000 7fc143200640 1 -- start start 2024-04-25T11:20:41.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.664+0000 7fc143200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1440b4870 0x7fc14413b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.664+0000 7fc143200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1440b51c0 0x7fc1441e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.664+0000 7fc143200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc144145e40 0x7fc14413ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.664+0000 7fc143200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc1440b55d0 con 0x7fc1440b4870 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.664+0000 7fc143200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1440b63d0 con 0x7fc1440b51c0 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.664+0000 7fc143200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc1440b5750 con 0x7fc144145e40 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.664+0000 7fc141e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1440b4870 0x7fc14413b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.664+0000 7fc141e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1440b4870 0x7fc14413b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36922/0 (socket says 172.21.15.7:36922) 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.664+0000 7fc141e00640 1 -- 172.21.15.7:0/1605423031 learned_addr learned my addr 172.21.15.7:0/1605423031 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.665+0000 7fc13be00640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1440b51c0 0x7fc1441e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.665+0000 7fc142800640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc144145e40 0x7fc14413ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.665+0000 7fc141e00640 1 -- 172.21.15.7:0/1605423031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc144145e40 msgr2=0x7fc14413ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.665+0000 7fc141e00640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc144145e40 0x7fc14413ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.665+0000 7fc141e00640 1 -- 172.21.15.7:0/1605423031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1440b51c0 msgr2=0x7fc1441e1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.665+0000 7fc141e00640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1440b51c0 0x7fc1441e1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.665+0000 7fc141e00640 1 -- 172.21.15.7:0/1605423031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14413bed0 con 0x7fc1440b4870 2024-04-25T11:20:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.665+0000 7fc141e00640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1440b4870 0x7fc14413b990 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc134000c00 tx=0x7fc1340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.665+0000 7fc13a000640 1 -- 172.21.15.7:0/1605423031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134037040 con 0x7fc1440b4870 2024-04-25T11:20:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.666+0000 7fc13a000640 1 -- 172.21.15.7:0/1605423031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc134042070 con 0x7fc1440b4870 2024-04-25T11:20:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.666+0000 7fc13a000640 1 -- 172.21.15.7:0/1605423031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134030bb0 con 0x7fc1440b4870 2024-04-25T11:20:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.666+0000 7fc143200640 1 -- 172.21.15.7:0/1605423031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14413dbe0 con 0x7fc1440b4870 2024-04-25T11:20:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.666+0000 7fc143200640 1 -- 172.21.15.7:0/1605423031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14413e1a0 con 0x7fc1440b4870 2024-04-25T11:20:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.667+0000 7fc143200640 1 -- 172.21.15.7:0/1605423031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc144050740 con 0x7fc1440b4870 2024-04-25T11:20:41.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.668+0000 7fc13a000640 1 -- 172.21.15.7:0/1605423031 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc134047460 con 0x7fc1440b4870 2024-04-25T11:20:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.669+0000 7fc13a000640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc1200744b0 0x7fc1200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.669+0000 7fc13a000640 1 -- 172.21.15.7:0/1605423031 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5619+0+0 (secure 0 0 0) 0x7fc1340b5e60 con 0x7fc1440b4870 2024-04-25T11:20:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.670+0000 7fc13be00640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc1200744b0 0x7fc1200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.670+0000 7fc13be00640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc1200744b0 0x7fc1200769c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc1441f9d50 tx=0x7fc12c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:41.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.675+0000 7fc13a000640 1 -- 172.21.15.7:0/1605423031 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc134080080 con 0x7fc1440b4870 2024-04-25T11:20:41.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:41.892+0000 7fc143200640 1 -- 172.21.15.7:0/1605423031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc144140640 con 0x7fc1440b4870 2024-04-25T11:20:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.335+0000 7fc13a000640 1 -- 172.21.15.7:0/1605423031 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v128) ==== 170+0+0 (secure 0 0 0) 0x7fc134084380 con 0x7fc1440b4870 2024-04-25T11:20:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-25T11:20:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.339+0000 7fc143200640 1 -- 172.21.15.7:0/1605423031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc1200744b0 msgr2=0x7fc1200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.339+0000 7fc143200640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc1200744b0 0x7fc1200769c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc1441f9d50 tx=0x7fc12c008040 comp rx=0 tx=0).stop 2024-04-25T11:20:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.339+0000 7fc143200640 1 -- 172.21.15.7:0/1605423031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1440b4870 msgr2=0x7fc14413b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.339+0000 7fc143200640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1440b4870 0x7fc14413b990 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc134000c00 tx=0x7fc1340046c0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.339+0000 7fc143200640 1 -- 172.21.15.7:0/1605423031 shutdown_connections 2024-04-25T11:20:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.339+0000 7fc143200640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc1200744b0 0x7fc1200769c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.339+0000 7fc143200640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc144145e40 0x7fc14413ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.339+0000 7fc143200640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1440b51c0 0x7fc1441e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.339+0000 7fc143200640 1 --2- 172.21.15.7:0/1605423031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1440b4870 0x7fc14413b990 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.340+0000 7fc143200640 1 -- 172.21.15.7:0/1605423031 >> 172.21.15.7:0/1605423031 conn(0x7fc1440740f0 msgr2=0x7fc14406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.342+0000 7fc143200640 1 -- 172.21.15.7:0/1605423031 shutdown_connections 2024-04-25T11:20:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.342+0000 7fc143200640 1 -- 172.21.15.7:0/1605423031 wait complete. 2024-04-25T11:20:42.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:20:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.638+0000 7fa24e400640 1 Processor -- start 2024-04-25T11:20:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.638+0000 7fa24e400640 1 -- start start 2024-04-25T11:20:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.641+0000 7fa24e400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2481583c0 0x7fa2480cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.641+0000 7fa24e400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa24806eea0 0x7fa24806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.641+0000 7fa24e400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2480ce570 con 0x7fa2480cfb50 2024-04-25T11:20:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.641+0000 7fa24e400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa24806f7f0 con 0x7fa24806eea0 2024-04-25T11:20:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.641+0000 7fa24e400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa24815f190 con 0x7fa2481583c0 2024-04-25T11:20:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.641+0000 7fa24da00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa24806eea0 0x7fa24806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.641+0000 7fa24da00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa24806eea0 0x7fa24806f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36926/0 (socket says 172.21.15.7:36926) 2024-04-25T11:20:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.641+0000 7fa24da00640 1 -- 172.21.15.7:0/2797389318 learned_addr learned my addr 172.21.15.7:0/2797389318 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.642+0000 7fa24d000640 1 --2- 172.21.15.7:0/2797389318 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2481583c0 0x7fa2480cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.642+0000 7fa24da00640 1 -- 172.21.15.7:0/2797389318 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2481583c0 msgr2=0x7fa2480cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.642+0000 7fa24da00640 1 --2- 172.21.15.7:0/2797389318 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2481583c0 0x7fa2480cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.642+0000 7fa24da00640 1 -- 172.21.15.7:0/2797389318 >> v1:172.21.15.87:6789/0 conn(0x7fa2480cfb50 legacy=0x7fa2480d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.642+0000 7fa24da00640 1 -- 172.21.15.7:0/2797389318 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2480ce850 con 0x7fa24806eea0 2024-04-25T11:20:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.642+0000 7fa24d000640 1 --2- 172.21.15.7:0/2797389318 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2481583c0 0x7fa2480cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.643+0000 7fa24da00640 1 --2- 172.21.15.7:0/2797389318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24806eea0 0x7fa24806f2b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa240009870 tx=0x7fa24002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67890db20fa1bde6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.643+0000 7fa247400640 1 -- 172.21.15.7:0/2797389318 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240037070 con 0x7fa24806eea0 2024-04-25T11:20:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.643+0000 7fa247400640 1 -- 172.21.15.7:0/2797389318 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa240032440 con 0x7fa24806eea0 2024-04-25T11:20:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.643+0000 7fa247400640 1 -- 172.21.15.7:0/2797389318 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240032810 con 0x7fa24806eea0 2024-04-25T11:20:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.644+0000 7fa24e400640 1 -- 172.21.15.7:0/2797389318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24806eea0 msgr2=0x7fa24806f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.644+0000 7fa24e400640 1 --2- 172.21.15.7:0/2797389318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24806eea0 0x7fa24806f2b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa240009870 tx=0x7fa24002a200 comp rx=0 tx=0).stop 2024-04-25T11:20:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.644+0000 7fa24e400640 1 -- 172.21.15.7:0/2797389318 shutdown_connections 2024-04-25T11:20:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.644+0000 7fa24e400640 1 --2- 172.21.15.7:0/2797389318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24806eea0 0x7fa24806f2b0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.644+0000 7fa24e400640 1 --2- 172.21.15.7:0/2797389318 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2481583c0 0x7fa2480cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.644+0000 7fa24e400640 1 -- 172.21.15.7:0/2797389318 >> 172.21.15.7:0/2797389318 conn(0x7fa2480740f0 msgr2=0x7fa24806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.644+0000 7fa24e400640 1 -- 172.21.15.7:0/2797389318 shutdown_connections 2024-04-25T11:20:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.644+0000 7fa24e400640 1 -- 172.21.15.7:0/2797389318 wait complete. 2024-04-25T11:20:42.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.647+0000 7fa24e400640 1 Processor -- start 2024-04-25T11:20:42.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.647+0000 7fa24e400640 1 -- start start 2024-04-25T11:20:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.648+0000 7fa24e400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa24806eea0 0x7fa248157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa24e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2480cfb50 0x7fa2481f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa24e400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2481583c0 0x7fa24815cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa24e400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa2480cf0f0 con 0x7fa2480cfb50 2024-04-25T11:20:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa24e400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa2480cee10 con 0x7fa2481583c0 2024-04-25T11:20:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa24e400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa2480d0b90 con 0x7fa24806eea0 2024-04-25T11:20:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa247e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2480cfb50 0x7fa2481f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa247e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2480cfb50 0x7fa2481f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36928/0 (socket says 172.21.15.7:36928) 2024-04-25T11:20:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa247e00640 1 -- 172.21.15.7:0/449228804 learned_addr learned my addr 172.21.15.7:0/449228804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa24d000640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa24806eea0 0x7fa248157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa24da00640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2481583c0 0x7fa24815cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa247e00640 1 -- 172.21.15.7:0/449228804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa24806eea0 msgr2=0x7fa248157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa247e00640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa24806eea0 0x7fa248157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa247e00640 1 -- 172.21.15.7:0/449228804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2481583c0 msgr2=0x7fa24815cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa247e00640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2481583c0 0x7fa24815cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.649+0000 7fa247e00640 1 -- 172.21.15.7:0/449228804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2480b67d0 con 0x7fa2480cfb50 2024-04-25T11:20:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.650+0000 7fa247e00640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2480cfb50 0x7fa2481f60e0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa23800ba80 tx=0x7fa23800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.650+0000 7fa244c00640 1 -- 172.21.15.7:0/449228804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23800cba0 con 0x7fa2480cfb50 2024-04-25T11:20:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.650+0000 7fa244c00640 1 -- 172.21.15.7:0/449228804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa23800cd40 con 0x7fa2480cfb50 2024-04-25T11:20:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.650+0000 7fa244c00640 1 -- 172.21.15.7:0/449228804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa238003bb0 con 0x7fa2480cfb50 2024-04-25T11:20:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.650+0000 7fa24e400640 1 -- 172.21.15.7:0/449228804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa248144bf0 con 0x7fa2480cfb50 2024-04-25T11:20:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.650+0000 7fa24e400640 1 -- 172.21.15.7:0/449228804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2481453d0 con 0x7fa2480cfb50 2024-04-25T11:20:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.651+0000 7fa24e400640 1 -- 172.21.15.7:0/449228804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa248144e60 con 0x7fa2480cfb50 2024-04-25T11:20:42.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.653+0000 7fa244c00640 1 -- 172.21.15.7:0/449228804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa238014440 con 0x7fa2480cfb50 2024-04-25T11:20:42.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.654+0000 7fa244c00640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2240744e0 0x7fa2240769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:42.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.654+0000 7fa244c00640 1 -- 172.21.15.7:0/449228804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 6001+0+0 (secure 0 0 0) 0x7fa238095250 con 0x7fa2480cfb50 2024-04-25T11:20:42.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.654+0000 7fa24d000640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2240744e0 0x7fa2240769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:42.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.655+0000 7fa24d000640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2240744e0 0x7fa2240769f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa2480aa990 tx=0x7fa230008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:42.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.660+0000 7fa244c00640 1 -- 172.21.15.7:0/449228804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa23805f8a0 con 0x7fa2480cfb50 2024-04-25T11:20:42.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:42.907+0000 7fa24e400640 1 -- 172.21.15.7:0/449228804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa2480aa1a0 con 0x7fa2480cfb50 2024-04-25T11:20:43.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.397+0000 7fa244c00640 1 -- 172.21.15.7:0/449228804 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v85) ==== 174+0+0 (secure 0 0 0) 0x7fa2480aa1a0 con 0x7fa2480cfb50 2024-04-25T11:20:43.395 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 19 and data pool 20 2024-04-25T11:20:43.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.406+0000 7fa24e400640 1 -- 172.21.15.7:0/449228804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2240744e0 msgr2=0x7fa2240769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.406+0000 7fa24e400640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2240744e0 0x7fa2240769f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa2480aa990 tx=0x7fa230008040 comp rx=0 tx=0).stop 2024-04-25T11:20:43.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.406+0000 7fa24e400640 1 -- 172.21.15.7:0/449228804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2480cfb50 msgr2=0x7fa2481f60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.406+0000 7fa24e400640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2480cfb50 0x7fa2481f60e0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa23800ba80 tx=0x7fa23800bf40 comp rx=0 tx=0).stop 2024-04-25T11:20:43.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.410+0000 7fa24e400640 1 -- 172.21.15.7:0/449228804 shutdown_connections 2024-04-25T11:20:43.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.410+0000 7fa24e400640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2240744e0 0x7fa2240769f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.410+0000 7fa24e400640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2481583c0 0x7fa24815cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.410+0000 7fa24e400640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2480cfb50 0x7fa2481f60e0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.410+0000 7fa24e400640 1 --2- 172.21.15.7:0/449228804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa24806eea0 0x7fa248157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.410+0000 7fa24e400640 1 -- 172.21.15.7:0/449228804 >> 172.21.15.7:0/449228804 conn(0x7fa2480740f0 msgr2=0x7fa2480cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:43.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.412+0000 7fa24e400640 1 -- 172.21.15.7:0/449228804 shutdown_connections 2024-04-25T11:20:43.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.413+0000 7fa24e400640 1 -- 172.21.15.7:0/449228804 wait complete. 2024-04-25T11:20:43.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:43.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:20:43.499 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:20:43.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.705+0000 7fa90d200640 1 Processor -- start 2024-04-25T11:20:43.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.705+0000 7fa90d200640 1 -- start start 2024-04-25T11:20:43.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.707+0000 7fa90d200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9080b04d0 0x7fa9080b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.707+0000 7fa90d200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa9080b4a80 0x7fa9080b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.707+0000 7fa90d200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa90815e190 con 0x7fa908070a70 2024-04-25T11:20:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.707+0000 7fa90d200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa90815d610 con 0x7fa9080b4a80 2024-04-25T11:20:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.707+0000 7fa90d200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa90815d8f0 con 0x7fa9080b04d0 2024-04-25T11:20:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.708+0000 7fa907e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa9080b4a80 0x7fa9080b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.708+0000 7fa907400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa908070a70 0x7fa9080ae880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:44454/0 (socket says 172.21.15.7:44454) 2024-04-25T11:20:43.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.708+0000 7fa907400640 1 -- 172.21.15.7:0/3046668969 learned_addr learned my addr 172.21.15.7:0/3046668969 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:43.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.708+0000 7fa906a00640 1 --2- 172.21.15.7:0/3046668969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9080b04d0 0x7fa9080b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.709+0000 7fa907e00640 1 -- 172.21.15.7:0/3046668969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9080b04d0 msgr2=0x7fa9080b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.709+0000 7fa907e00640 1 --2- 172.21.15.7:0/3046668969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9080b04d0 0x7fa9080b08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.709+0000 7fa907e00640 1 -- 172.21.15.7:0/3046668969 >> v1:172.21.15.87:6789/0 conn(0x7fa908070a70 legacy=0x7fa9080ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.709+0000 7fa907e00640 1 -- 172.21.15.7:0/3046668969 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90815dbd0 con 0x7fa9080b4a80 2024-04-25T11:20:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.709+0000 7fa906a00640 1 --2- 172.21.15.7:0/3046668969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9080b04d0 0x7fa9080b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:20:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.710+0000 7fa907e00640 1 --2- 172.21.15.7:0/3046668969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080b4a80 0x7fa9080b4e90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa900009d90 tx=0x7fa90002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b82385146760bf7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.711+0000 7fa906000640 1 -- 172.21.15.7:0/3046668969 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900007740 con 0x7fa9080b4a80 2024-04-25T11:20:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.711+0000 7fa906000640 1 -- 172.21.15.7:0/3046668969 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9000078e0 con 0x7fa9080b4a80 2024-04-25T11:20:43.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.711+0000 7fa906000640 1 -- 172.21.15.7:0/3046668969 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900007cb0 con 0x7fa9080b4a80 2024-04-25T11:20:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.712+0000 7fa90d200640 1 -- 172.21.15.7:0/3046668969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080b4a80 msgr2=0x7fa9080b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.712+0000 7fa90d200640 1 --2- 172.21.15.7:0/3046668969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080b4a80 0x7fa9080b4e90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa900009d90 tx=0x7fa90002a660 comp rx=0 tx=0).stop 2024-04-25T11:20:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.712+0000 7fa90d200640 1 -- 172.21.15.7:0/3046668969 shutdown_connections 2024-04-25T11:20:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.712+0000 7fa90d200640 1 --2- 172.21.15.7:0/3046668969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080b4a80 0x7fa9080b4e90 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.712+0000 7fa90d200640 1 --2- 172.21.15.7:0/3046668969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9080b04d0 0x7fa9080b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.712+0000 7fa90d200640 1 -- 172.21.15.7:0/3046668969 >> 172.21.15.7:0/3046668969 conn(0x7fa9080b3820 msgr2=0x7fa9080b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.712+0000 7fa90d200640 1 -- 172.21.15.7:0/3046668969 shutdown_connections 2024-04-25T11:20:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.713+0000 7fa90d200640 1 -- 172.21.15.7:0/3046668969 wait complete. 2024-04-25T11:20:43.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.715+0000 7fa90d200640 1 Processor -- start 2024-04-25T11:20:43.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.715+0000 7fa90d200640 1 -- start start 2024-04-25T11:20:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.717+0000 7fa90d200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa908070a70 0x7fa90815ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.717+0000 7fa90d200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080b04d0 0x7fa9081509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.717+0000 7fa90d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9080b4a80 0x7fa90809cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.717+0000 7fa90d200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9080b0e20 con 0x7fa9080b4a80 2024-04-25T11:20:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.717+0000 7fa90d200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa9080aef90 con 0x7fa9080b04d0 2024-04-25T11:20:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.717+0000 7fa90d200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa90815d170 con 0x7fa908070a70 2024-04-25T11:20:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.717+0000 7fa907e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9080b4a80 0x7fa90809cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.717+0000 7fa907e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9080b4a80 0x7fa90809cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36940/0 (socket says 172.21.15.7:36940) 2024-04-25T11:20:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.717+0000 7fa907e00640 1 -- 172.21.15.7:0/782500501 learned_addr learned my addr 172.21.15.7:0/782500501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.718+0000 7fa907400640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa908070a70 0x7fa90815ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.718+0000 7fa906a00640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080b04d0 0x7fa9081509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.718+0000 7fa907e00640 1 -- 172.21.15.7:0/782500501 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa908070a70 msgr2=0x7fa90815ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.718+0000 7fa907e00640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa908070a70 0x7fa90815ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.718+0000 7fa907e00640 1 -- 172.21.15.7:0/782500501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080b04d0 msgr2=0x7fa9081509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.718+0000 7fa907e00640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080b04d0 0x7fa9081509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.718+0000 7fa907e00640 1 -- 172.21.15.7:0/782500501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9081513e0 con 0x7fa9080b4a80 2024-04-25T11:20:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.718+0000 7fa907400640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa908070a70 0x7fa90815ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.718+0000 7fa907e00640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9080b4a80 0x7fa90809cdb0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa900009eb0 tx=0x7fa900004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.719+0000 7fa8ff400640 1 -- 172.21.15.7:0/782500501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90003f070 con 0x7fa9080b4a80 2024-04-25T11:20:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.719+0000 7fa8ff400640 1 -- 172.21.15.7:0/782500501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa900031b40 con 0x7fa9080b4a80 2024-04-25T11:20:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.719+0000 7fa8ff400640 1 -- 172.21.15.7:0/782500501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900044460 con 0x7fa9080b4a80 2024-04-25T11:20:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.719+0000 7fa90d200640 1 -- 172.21.15.7:0/782500501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9081516c0 con 0x7fa9080b4a80 2024-04-25T11:20:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.719+0000 7fa90d200640 1 -- 172.21.15.7:0/782500501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa90815e400 con 0x7fa9080b4a80 2024-04-25T11:20:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.720+0000 7fa90d200640 1 -- 172.21.15.7:0/782500501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa908050740 con 0x7fa9080b4a80 2024-04-25T11:20:43.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.722+0000 7fa8ff400640 1 -- 172.21.15.7:0/782500501 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa90003b410 con 0x7fa9080b4a80 2024-04-25T11:20:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.723+0000 7fa8ff400640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8e80744b0 0x7fa8e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.723+0000 7fa8ff400640 1 -- 172.21.15.7:0/782500501 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7fa9000b6b70 con 0x7fa9080b4a80 2024-04-25T11:20:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.723+0000 7fa907400640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8e80744b0 0x7fa8e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.724+0000 7fa907400640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8e80744b0 0x7fa8e80769c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8000f60 tx=0x7fa8f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:43.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.728+0000 7fa8ff400640 1 -- 172.21.15.7:0/782500501 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa900080b10 con 0x7fa9080b4a80 2024-04-25T11:20:43.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.934+0000 7fa90d200640 1 -- 172.21.15.7:0/782500501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa90806e0b0 con 0x7fa9080b4a80 2024-04-25T11:20:43.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.938+0000 7fa8ff400640 1 -- 172.21.15.7:0/782500501 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16817 (secure 0 0 0) 0x7fa900084e10 con 0x7fa9080b4a80 2024-04-25T11:20:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:43.935 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":129,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:20:42.908847+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:41.047420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:41.908907+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 -- 172.21.15.7:0/782500501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8e80744b0 msgr2=0x7fa8e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8e80744b0 0x7fa8e80769c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8000f60 tx=0x7fa8f8009040 comp rx=0 tx=0).stop 2024-04-25T11:20:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 -- 172.21.15.7:0/782500501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9080b4a80 msgr2=0x7fa90809cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:43.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9080b4a80 0x7fa90809cdb0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa900009eb0 tx=0x7fa900004a40 comp rx=0 tx=0).stop 2024-04-25T11:20:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 -- 172.21.15.7:0/782500501 shutdown_connections 2024-04-25T11:20:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa8e80744b0 0x7fa8e80769c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9080b4a80 0x7fa90809cdb0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9080b04d0 0x7fa9081509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 --2- 172.21.15.7:0/782500501 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa908070a70 0x7fa90815ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 -- 172.21.15.7:0/782500501 >> 172.21.15.7:0/782500501 conn(0x7fa9080b3820 msgr2=0x7fa90806fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 -- 172.21.15.7:0/782500501 shutdown_connections 2024-04-25T11:20:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:43.941+0000 7fa90d200640 1 -- 172.21.15.7:0/782500501 wait complete. 2024-04-25T11:20:43.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.246+0000 7f2689400640 1 Processor -- start 2024-04-25T11:20:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.246+0000 7f2689400640 1 -- start start 2024-04-25T11:20:44.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.248+0000 7f2689400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2684070a70 0x7f26840ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.248+0000 7f2689400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26840b0be0 0x7f26840b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.248+0000 7f2689400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f268415e190 con 0x7f26840aedc0 2024-04-25T11:20:44.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.248+0000 7f2689400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f268415d610 con 0x7f26840b0be0 2024-04-25T11:20:44.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.248+0000 7f2689400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f268415d8f0 con 0x7f2684070a70 2024-04-25T11:20:44.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.249+0000 7f2688a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26840b0be0 0x7f26840b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.249+0000 7f2688a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26840b0be0 0x7f26840b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36950/0 (socket says 172.21.15.7:36950) 2024-04-25T11:20:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.249+0000 7f2688a00640 1 -- 172.21.15.7:0/249760884 learned_addr learned my addr 172.21.15.7:0/249760884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.249+0000 7f2683e00640 1 --2- 172.21.15.7:0/249760884 >> v2:172.21.15.87:3300/0 conn(0x7f2684070a70 0x7f26840ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.249+0000 7f2688a00640 1 -- 172.21.15.7:0/249760884 >> v2:172.21.15.87:3300/0 conn(0x7f2684070a70 msgr2=0x7f26840ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.249+0000 7f2688a00640 1 --2- 172.21.15.7:0/249760884 >> v2:172.21.15.87:3300/0 conn(0x7f2684070a70 0x7f26840ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.249+0000 7f2688a00640 1 -- 172.21.15.7:0/249760884 >> v1:172.21.15.87:6789/0 conn(0x7f26840aedc0 legacy=0x7f26840b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.249+0000 7f2688a00640 1 -- 172.21.15.7:0/249760884 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268415dbd0 con 0x7f26840b0be0 2024-04-25T11:20:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.250+0000 7f2688a00640 1 --2- 172.21.15.7:0/249760884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26840b0be0 0x7f26840b4a80 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f26840725e0 tx=0x7f267c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfa0bcabe50ba7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.251+0000 7f2682a00640 1 -- 172.21.15.7:0/249760884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c036070 con 0x7f26840b0be0 2024-04-25T11:20:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.251+0000 7f2682a00640 1 -- 172.21.15.7:0/249760884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f267c031600 con 0x7f26840b0be0 2024-04-25T11:20:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.251+0000 7f2682a00640 1 -- 172.21.15.7:0/249760884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c0319d0 con 0x7f26840b0be0 2024-04-25T11:20:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.251+0000 7f2689400640 1 -- 172.21.15.7:0/249760884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26840b0be0 msgr2=0x7f26840b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.251+0000 7f2689400640 1 --2- 172.21.15.7:0/249760884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26840b0be0 0x7f26840b4a80 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f26840725e0 tx=0x7f267c02a6f0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.252+0000 7f2689400640 1 -- 172.21.15.7:0/249760884 shutdown_connections 2024-04-25T11:20:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.252+0000 7f2689400640 1 --2- 172.21.15.7:0/249760884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26840b0be0 0x7f26840b4a80 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.252+0000 7f2689400640 1 --2- 172.21.15.7:0/249760884 >> v2:172.21.15.87:3300/0 conn(0x7f2684070a70 0x7f26840ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.252+0000 7f2689400640 1 -- 172.21.15.7:0/249760884 >> 172.21.15.7:0/249760884 conn(0x7f26840b3820 msgr2=0x7f26840b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.252+0000 7f2689400640 1 -- 172.21.15.7:0/249760884 shutdown_connections 2024-04-25T11:20:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.252+0000 7f2689400640 1 -- 172.21.15.7:0/249760884 wait complete. 2024-04-25T11:20:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.255+0000 7f2689400640 1 Processor -- start 2024-04-25T11:20:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.255+0000 7f2689400640 1 -- start start 2024-04-25T11:20:44.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.257+0000 7f2689400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2684070a70 0x7f268415ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.257+0000 7f2689400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26840aedc0 0x7f26840cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.257+0000 7f2689400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26840b0be0 0x7f26840cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.257+0000 7f2689400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f268415d2f0 con 0x7f2684070a70 2024-04-25T11:20:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.257+0000 7f2689400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f268415d170 con 0x7f26840b0be0 2024-04-25T11:20:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.257+0000 7f2689400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f26840b4fc0 con 0x7f26840aedc0 2024-04-25T11:20:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.257+0000 7f2683e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2684070a70 0x7f268415ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.257+0000 7f2683e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2684070a70 0x7f268415ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36958/0 (socket says 172.21.15.7:36958) 2024-04-25T11:20:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.257+0000 7f2683e00640 1 -- 172.21.15.7:0/1488000848 learned_addr learned my addr 172.21.15.7:0/1488000848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.257+0000 7f2683400640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26840aedc0 0x7f26840cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f2688a00640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26840b0be0 0x7f26840cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f2683e00640 1 -- 172.21.15.7:0/1488000848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26840aedc0 msgr2=0x7f26840cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f2683e00640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26840aedc0 0x7f26840cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f2683e00640 1 -- 172.21.15.7:0/1488000848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26840b0be0 msgr2=0x7f26840cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f2683e00640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26840b0be0 0x7f26840cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f2683e00640 1 -- 172.21.15.7:0/1488000848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26840ccf00 con 0x7f2684070a70 2024-04-25T11:20:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f2688a00640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26840b0be0 0x7f26840cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f2683e00640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2684070a70 0x7f268415ff20 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f267400d620 tx=0x7f267400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f267be00640 1 -- 172.21.15.7:0/1488000848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267400b620 con 0x7f2684070a70 2024-04-25T11:20:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f267be00640 1 -- 172.21.15.7:0/1488000848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2674019070 con 0x7f2684070a70 2024-04-25T11:20:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f267be00640 1 -- 172.21.15.7:0/1488000848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267400ba00 con 0x7f2684070a70 2024-04-25T11:20:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.258+0000 7f2689400640 1 -- 172.21.15.7:0/1488000848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268415aa90 con 0x7f2684070a70 2024-04-25T11:20:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.259+0000 7f2689400640 1 -- 172.21.15.7:0/1488000848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268415b050 con 0x7f2684070a70 2024-04-25T11:20:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.260+0000 7f2689400640 1 -- 172.21.15.7:0/1488000848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2684050740 con 0x7f2684070a70 2024-04-25T11:20:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.261+0000 7f267be00640 1 -- 172.21.15.7:0/1488000848 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f267400bba0 con 0x7f2684070a70 2024-04-25T11:20:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.262+0000 7f267be00640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26640744b0 0x7f26640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.262+0000 7f267be00640 1 -- 172.21.15.7:0/1488000848 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 6101+0+0 (secure 0 0 0) 0x7f2674095770 con 0x7f2684070a70 2024-04-25T11:20:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.262+0000 7f2683400640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26640744b0 0x7f26640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.263+0000 7f2683400640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26640744b0 0x7f26640769c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2670006fd0 tx=0x7f2670008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.267+0000 7f267be00640 1 -- 172.21.15.7:0/1488000848 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f267405fe00 con 0x7f2684070a70 2024-04-25T11:20:44.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.475+0000 7f2689400640 1 -- 172.21.15.7:0/1488000848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26840b5380 con 0x7f2684070a70 2024-04-25T11:20:44.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.477+0000 7f267be00640 1 -- 172.21.15.7:0/1488000848 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16804 (secure 0 0 0) 0x7f267405f700 con 0x7f2684070a70 2024-04-25T11:20:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:44.475 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":130,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:20:44.396251+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:41.047420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:41.908907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.480+0000 7f2689400640 1 -- 172.21.15.7:0/1488000848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26640744b0 msgr2=0x7f26640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.480+0000 7f2689400640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26640744b0 0x7f26640769c0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2670006fd0 tx=0x7f2670008040 comp rx=0 tx=0).stop 2024-04-25T11:20:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.480+0000 7f2689400640 1 -- 172.21.15.7:0/1488000848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2684070a70 msgr2=0x7f268415ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.480+0000 7f2689400640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2684070a70 0x7f268415ff20 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f267400d620 tx=0x7f267400dae0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.480+0000 7f2689400640 1 -- 172.21.15.7:0/1488000848 shutdown_connections 2024-04-25T11:20:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.480+0000 7f2689400640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26640744b0 0x7f26640769c0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.480+0000 7f2689400640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26840b0be0 0x7f26840cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.480+0000 7f2689400640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26840aedc0 0x7f26840cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.480+0000 7f2689400640 1 --2- 172.21.15.7:0/1488000848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2684070a70 0x7f268415ff20 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.480+0000 7f2689400640 1 -- 172.21.15.7:0/1488000848 >> 172.21.15.7:0/1488000848 conn(0x7f26840b3820 msgr2=0x7f26840b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:44.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.480+0000 7f2689400640 1 -- 172.21.15.7:0/1488000848 shutdown_connections 2024-04-25T11:20:44.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.481+0000 7f2689400640 1 -- 172.21.15.7:0/1488000848 wait complete. 2024-04-25T11:20:44.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:20:44.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.793+0000 7fe314a00640 1 Processor -- start 2024-04-25T11:20:44.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.793+0000 7fe314a00640 1 -- start start 2024-04-25T11:20:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.795+0000 7fe314a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe310145e40 0x7fe310146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.795+0000 7fe314a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3100b4870 0x7fe3100b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.795+0000 7fe314a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe3100b51c0 0x7fe310143a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.795+0000 7fe314a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe3100b5b30 con 0x7fe3100b51c0 2024-04-25T11:20:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.795+0000 7fe314a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe3100b5e10 con 0x7fe3100b4870 2024-04-25T11:20:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.795+0000 7fe314a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe3100b60f0 con 0x7fe310145e40 2024-04-25T11:20:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.796+0000 7fe30fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe3100b51c0 0x7fe310143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.796+0000 7fe30fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe3100b51c0 0x7fe310143a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36970/0 (socket says 172.21.15.7:36970) 2024-04-25T11:20:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.796+0000 7fe30fe00640 1 -- 172.21.15.7:0/3504172656 learned_addr learned my addr 172.21.15.7:0/3504172656 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.796+0000 7fe30f400640 1 --2- 172.21.15.7:0/3504172656 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe310145e40 0x7fe310146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.796+0000 7fe307e00640 1 --2- 172.21.15.7:0/3504172656 >> v2:172.21.15.87:3300/0 conn(0x7fe3100b4870 0x7fe3100b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.796+0000 7fe30fe00640 1 -- 172.21.15.7:0/3504172656 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe310145e40 msgr2=0x7fe310146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:44.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.796+0000 7fe30fe00640 1 --2- 172.21.15.7:0/3504172656 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe310145e40 0x7fe310146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.796+0000 7fe30fe00640 1 -- 172.21.15.7:0/3504172656 >> v2:172.21.15.87:3300/0 conn(0x7fe3100b4870 msgr2=0x7fe3100b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:44.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.796+0000 7fe30fe00640 1 --2- 172.21.15.7:0/3504172656 >> v2:172.21.15.87:3300/0 conn(0x7fe3100b4870 0x7fe3100b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.796+0000 7fe30fe00640 1 -- 172.21.15.7:0/3504172656 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3100b63d0 con 0x7fe3100b51c0 2024-04-25T11:20:44.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.796+0000 7fe307e00640 1 --2- 172.21.15.7:0/3504172656 >> v2:172.21.15.87:3300/0 conn(0x7fe3100b4870 0x7fe3100b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:44.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.797+0000 7fe30fe00640 1 --2- 172.21.15.7:0/3504172656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3100b51c0 0x7fe310143a70 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe3100b4390 tx=0x7fe30802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af952a7015e08a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.798+0000 7fe30ea00640 1 -- 172.21.15.7:0/3504172656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308036070 con 0x7fe3100b51c0 2024-04-25T11:20:44.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.798+0000 7fe30ea00640 1 -- 172.21.15.7:0/3504172656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe308031600 con 0x7fe3100b51c0 2024-04-25T11:20:44.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.798+0000 7fe30ea00640 1 -- 172.21.15.7:0/3504172656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3080319d0 con 0x7fe3100b51c0 2024-04-25T11:20:44.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.798+0000 7fe314a00640 1 -- 172.21.15.7:0/3504172656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3100b51c0 msgr2=0x7fe310143a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.798+0000 7fe314a00640 1 --2- 172.21.15.7:0/3504172656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3100b51c0 0x7fe310143a70 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe3100b4390 tx=0x7fe30802a610 comp rx=0 tx=0).stop 2024-04-25T11:20:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.798+0000 7fe314a00640 1 -- 172.21.15.7:0/3504172656 shutdown_connections 2024-04-25T11:20:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.798+0000 7fe314a00640 1 --2- 172.21.15.7:0/3504172656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3100b51c0 0x7fe310143a70 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.798+0000 7fe314a00640 1 --2- 172.21.15.7:0/3504172656 >> v2:172.21.15.87:3300/0 conn(0x7fe3100b4870 0x7fe3100b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.798+0000 7fe314a00640 1 --2- 172.21.15.7:0/3504172656 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe310145e40 0x7fe310146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.798+0000 7fe314a00640 1 -- 172.21.15.7:0/3504172656 >> 172.21.15.7:0/3504172656 conn(0x7fe3100740f0 msgr2=0x7fe31006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.799+0000 7fe314a00640 1 -- 172.21.15.7:0/3504172656 shutdown_connections 2024-04-25T11:20:44.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.799+0000 7fe314a00640 1 -- 172.21.15.7:0/3504172656 wait complete. 2024-04-25T11:20:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.801+0000 7fe314a00640 1 Processor -- start 2024-04-25T11:20:44.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.801+0000 7fe314a00640 1 -- start start 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.802+0000 7fe314a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3100b4870 0x7fe3100b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.802+0000 7fe314a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3100b51c0 0x7fe3100caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.802+0000 7fe314a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe310145e40 0x7fe3101d3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.803+0000 7fe314a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe310143fb0 con 0x7fe3100b4870 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.803+0000 7fe314a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe3100b57e0 con 0x7fe3100b51c0 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.803+0000 7fe314a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe310144130 con 0x7fe310145e40 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.803+0000 7fe30f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3100b4870 0x7fe3100b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.803+0000 7fe30f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3100b4870 0x7fe3100b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36974/0 (socket says 172.21.15.7:36974) 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.803+0000 7fe30f400640 1 -- 172.21.15.7:0/3070526884 learned_addr learned my addr 172.21.15.7:0/3070526884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.803+0000 7fe30f400640 1 -- 172.21.15.7:0/3070526884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe310145e40 msgr2=0x7fe3101d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.803+0000 7fe30f400640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe310145e40 0x7fe3101d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.803+0000 7fe30f400640 1 -- 172.21.15.7:0/3070526884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3100b51c0 msgr2=0x7fe3100caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.803+0000 7fe30f400640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3100b51c0 0x7fe3100caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.803+0000 7fe30f400640 1 -- 172.21.15.7:0/3070526884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3100cb470 con 0x7fe3100b4870 2024-04-25T11:20:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.804+0000 7fe30f400640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3100b4870 0x7fe3100b2dd0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fe30000d5d0 tx=0x7fe30000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.804+0000 7fe307400640 1 -- 172.21.15.7:0/3070526884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300004240 con 0x7fe3100b4870 2024-04-25T11:20:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.804+0000 7fe307400640 1 -- 172.21.15.7:0/3070526884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3000043e0 con 0x7fe3100b4870 2024-04-25T11:20:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.804+0000 7fe307400640 1 -- 172.21.15.7:0/3070526884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe300003830 con 0x7fe3100b4870 2024-04-25T11:20:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.804+0000 7fe314a00640 1 -- 172.21.15.7:0/3070526884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3100c9e30 con 0x7fe3100b4870 2024-04-25T11:20:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.804+0000 7fe314a00640 1 -- 172.21.15.7:0/3070526884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3101d1a00 con 0x7fe3100b4870 2024-04-25T11:20:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.805+0000 7fe314a00640 1 -- 172.21.15.7:0/3070526884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe310050740 con 0x7fe3100b4870 2024-04-25T11:20:44.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.806+0000 7fe307400640 1 -- 172.21.15.7:0/3070526884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe3000039d0 con 0x7fe3100b4870 2024-04-25T11:20:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.807+0000 7fe307400640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2ec0744b0 0x7fe2ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.808+0000 7fe307400640 1 -- 172.21.15.7:0/3070526884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7fe3000956f0 con 0x7fe3100b4870 2024-04-25T11:20:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.808+0000 7fe307e00640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2ec0744b0 0x7fe2ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:44.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.808+0000 7fe307e00640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2ec0744b0 0x7fe2ec0769c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8006b10 tx=0x7fe2f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:44.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:44.812+0000 7fe307400640 1 -- 172.21.15.7:0/3070526884 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe30005fce0 con 0x7fe3100b4870 2024-04-25T11:20:45.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.052+0000 7fe314a00640 1 -- 172.21.15.7:0/3070526884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe3100cb6a0 con 0x7fe3100b4870 2024-04-25T11:20:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.411+0000 7fe307400640 1 -- 172.21.15.7:0/3070526884 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v88) ==== 118+0+0 (secure 0 0 0) 0x7fe30005f5e0 con 0x7fe3100b4870 2024-04-25T11:20:45.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.414+0000 7fe314a00640 1 -- 172.21.15.7:0/3070526884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2ec0744b0 msgr2=0x7fe2ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:45.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.414+0000 7fe314a00640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2ec0744b0 0x7fe2ec0769c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8006b10 tx=0x7fe2f8008040 comp rx=0 tx=0).stop 2024-04-25T11:20:45.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.414+0000 7fe314a00640 1 -- 172.21.15.7:0/3070526884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3100b4870 msgr2=0x7fe3100b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:45.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.414+0000 7fe314a00640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3100b4870 0x7fe3100b2dd0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fe30000d5d0 tx=0x7fe30000da90 comp rx=0 tx=0).stop 2024-04-25T11:20:45.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.415+0000 7fe314a00640 1 -- 172.21.15.7:0/3070526884 shutdown_connections 2024-04-25T11:20:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.415+0000 7fe314a00640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe2ec0744b0 0x7fe2ec0769c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.415+0000 7fe314a00640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe310145e40 0x7fe3101d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.415+0000 7fe314a00640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3100b51c0 0x7fe3100caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.415+0000 7fe314a00640 1 --2- 172.21.15.7:0/3070526884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3100b4870 0x7fe3100b2dd0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.415+0000 7fe314a00640 1 -- 172.21.15.7:0/3070526884 >> 172.21.15.7:0/3070526884 conn(0x7fe3100740f0 msgr2=0x7fe3101e34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.415+0000 7fe314a00640 1 -- 172.21.15.7:0/3070526884 shutdown_connections 2024-04-25T11:20:45.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.415+0000 7fe314a00640 1 -- 172.21.15.7:0/3070526884 wait complete. 2024-04-25T11:20:45.445 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:20:45.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:20:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.738+0000 7f1bed000640 1 Processor -- start 2024-04-25T11:20:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.739+0000 7f1bed000640 1 -- start start 2024-04-25T11:20:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.741+0000 7f1bed000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1be81684f0 0x7f1be80d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.741+0000 7f1bed000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be806f330 0x7f1be806f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.741+0000 7f1bed000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1be80d09e0 con 0x7f1be80cff00 2024-04-25T11:20:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.741+0000 7f1bed000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1be806eea0 con 0x7f1be81684f0 2024-04-25T11:20:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.741+0000 7f1bed000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1be80d0cc0 con 0x7f1be806f330 2024-04-25T11:20:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.742+0000 7f1be7400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1be80cff00 0x7f1be8167de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48422/0 (socket says 172.21.15.7:48422) 2024-04-25T11:20:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.742+0000 7f1be7400640 1 -- 172.21.15.7:0/3554832292 learned_addr learned my addr 172.21.15.7:0/3554832292 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.742+0000 7f1be7e00640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be806f330 0x7f1be806f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.742+0000 7f1be6a00640 1 --2- 172.21.15.7:0/3554832292 >> v2:172.21.15.87:3300/0 conn(0x7f1be81684f0 0x7f1be80d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.742+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2825718029 0 0) 0x7f1be80d09e0 con 0x7f1be80cff00 2024-04-25T11:20:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.742+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bcc003520 con 0x7f1be80cff00 2024-04-25T11:20:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1133922432 0 0) 0x7f1bcc003520 con 0x7f1be80cff00 2024-04-25T11:20:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be806f330 msgr2=0x7f1be806f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be806f330 0x7f1be806f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 >> v2:172.21.15.87:3300/0 conn(0x7f1be81684f0 msgr2=0x7f1be80d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 --2- 172.21.15.7:0/3554832292 >> v2:172.21.15.87:3300/0 conn(0x7f1be81684f0 0x7f1be80d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be80d0fa0 con 0x7f1be80cff00 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1bd8001690 con 0x7f1be80cff00 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 >> v1:172.21.15.7:6789/0 conn(0x7f1be80cff00 legacy=0x7f1be8167de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be806f330 0x7f1bcc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bcc003f30 0x7f1bcc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc004d40 0x7f1bcc005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1be806eea0 con 0x7f1bcc004d40 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1be80d0cc0 con 0x7f1bcc003f30 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1bcc00ba10 con 0x7f1be806f330 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.743+0000 7f1be7e00640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc004d40 0x7f1bcc005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.744+0000 7f1be7400640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be806f330 0x7f1bcc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.744+0000 7f1be6a00640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bcc003f30 0x7f1bcc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.744+0000 7f1be7e00640 1 -- 172.21.15.7:0/3554832292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be806f330 msgr2=0x7f1bcc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.744+0000 7f1be7e00640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be806f330 0x7f1bcc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:45.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.744+0000 7f1be7e00640 1 -- 172.21.15.7:0/3554832292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bcc003f30 msgr2=0x7f1bcc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:45.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.744+0000 7f1be7e00640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bcc003f30 0x7f1bcc004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:45.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.744+0000 7f1be7e00640 1 -- 172.21.15.7:0/3554832292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc003520 con 0x7f1bcc004d40 2024-04-25T11:20:45.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.745+0000 7f1be7e00640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc004d40 0x7f1bcc005150 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009f90 tx=0x7f1bcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6263f0379506daa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:45.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.745+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0009c10 con 0x7f1bcc004d40 2024-04-25T11:20:45.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.745+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1be0036070 con 0x7f1bcc004d40 2024-04-25T11:20:45.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.745+0000 7f1be6000640 1 -- 172.21.15.7:0/3554832292 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0031850 con 0x7f1bcc004d40 2024-04-25T11:20:45.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.746+0000 7f1bed000640 1 -- 172.21.15.7:0/3554832292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc004d40 msgr2=0x7f1bcc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:45.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.746+0000 7f1bed000640 1 --2- 172.21.15.7:0/3554832292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc004d40 0x7f1bcc005150 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009f90 tx=0x7f1bcc005830 comp rx=0 tx=0).stop 2024-04-25T11:20:45.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.746+0000 7f1be7e00640 1 -- 172.21.15.7:0/3554832292 reap_dead start 2024-04-25T11:20:45.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.746+0000 7f1bed000640 1 -- 172.21.15.7:0/3554832292 shutdown_connections 2024-04-25T11:20:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.746+0000 7f1bed000640 1 -- 172.21.15.7:0/3554832292 >> 172.21.15.7:0/3554832292 conn(0x7f1be80740f0 msgr2=0x7f1be806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.746+0000 7f1bed000640 1 -- 172.21.15.7:0/3554832292 shutdown_connections 2024-04-25T11:20:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.746+0000 7f1bed000640 1 -- 172.21.15.7:0/3554832292 wait complete. 2024-04-25T11:20:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.749+0000 7f1bed000640 1 Processor -- start 2024-04-25T11:20:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.749+0000 7f1bed000640 1 -- start start 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.750+0000 7f1bed000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bcc004d40 0x7f1be81fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.750+0000 7f1bed000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be815d3c0 0x7f1be815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.750+0000 7f1bed000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be815dd10 0x7f1be8145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.750+0000 7f1bed000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1be8168bb0 con 0x7f1be815dd10 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.750+0000 7f1bed000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1be8168a30 con 0x7f1bcc004d40 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.750+0000 7f1bed000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1be8147340 con 0x7f1be815d3c0 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.751+0000 7f1be7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be815dd10 0x7f1be8145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.751+0000 7f1be7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be815dd10 0x7f1be8145320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36998/0 (socket says 172.21.15.7:36998) 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.751+0000 7f1be7e00640 1 -- 172.21.15.7:0/598513517 learned_addr learned my addr 172.21.15.7:0/598513517 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.751+0000 7f1be7400640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bcc004d40 0x7f1be81fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.751+0000 7f1be6a00640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be815d3c0 0x7f1be815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.751+0000 7f1be7e00640 1 -- 172.21.15.7:0/598513517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be815d3c0 msgr2=0x7f1be815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:45.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.751+0000 7f1be7e00640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be815d3c0 0x7f1be815d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.751+0000 7f1be7e00640 1 -- 172.21.15.7:0/598513517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bcc004d40 msgr2=0x7f1be81fa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.751+0000 7f1be7e00640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bcc004d40 0x7f1be81fa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.751+0000 7f1be7e00640 1 -- 172.21.15.7:0/598513517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be8144b40 con 0x7f1be815dd10 2024-04-25T11:20:45.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.751+0000 7f1be6a00640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be815d3c0 0x7f1be815d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.752+0000 7f1be7e00640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be815dd10 0x7f1be8145320 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f1be0033040 tx=0x7f1be0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.752+0000 7f1bdf400640 1 -- 172.21.15.7:0/598513517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0009c10 con 0x7f1be815dd10 2024-04-25T11:20:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.752+0000 7f1bdf400640 1 -- 172.21.15.7:0/598513517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1be0003990 con 0x7f1be815dd10 2024-04-25T11:20:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.752+0000 7f1bdf400640 1 -- 172.21.15.7:0/598513517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be0003d80 con 0x7f1be815dd10 2024-04-25T11:20:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.752+0000 7f1bed000640 1 -- 172.21.15.7:0/598513517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be81fa630 con 0x7f1be815dd10 2024-04-25T11:20:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.752+0000 7f1bed000640 1 -- 172.21.15.7:0/598513517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be815c960 con 0x7f1be815dd10 2024-04-25T11:20:45.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.753+0000 7f1bed000640 1 -- 172.21.15.7:0/598513517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be815cb60 con 0x7f1be815dd10 2024-04-25T11:20:45.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.754+0000 7f1bdf400640 1 -- 172.21.15.7:0/598513517 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1be0005d10 con 0x7f1be815dd10 2024-04-25T11:20:45.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.755+0000 7f1bdf400640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bc80744b0 0x7f1bc80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:45.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.756+0000 7f1bdf400640 1 -- 172.21.15.7:0/598513517 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f1be00b6810 con 0x7f1be815dd10 2024-04-25T11:20:45.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.756+0000 7f1be7400640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bc80744b0 0x7f1bc80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:45.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.756+0000 7f1be7400640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bc80744b0 0x7f1bc80769c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1be8070db0 tx=0x7f1bd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:45.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:45.761+0000 7f1bdf400640 1 -- 172.21.15.7:0/598513517 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1be002e050 con 0x7f1be815dd10 2024-04-25T11:20:46.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.007+0000 7f1bed000640 1 -- 172.21.15.7:0/598513517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1be8050740 con 0x7f1be815dd10 2024-04-25T11:20:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.425+0000 7f1bdf400640 1 -- 172.21.15.7:0/598513517 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v89) ==== 122+0+0 (secure 0 0 0) 0x7f1be0080e10 con 0x7f1be815dd10 2024-04-25T11:20:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.429+0000 7f1bed000640 1 -- 172.21.15.7:0/598513517 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bc80744b0 msgr2=0x7f1bc80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.429+0000 7f1bed000640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bc80744b0 0x7f1bc80769c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1be8070db0 tx=0x7f1bd8009040 comp rx=0 tx=0).stop 2024-04-25T11:20:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.429+0000 7f1bed000640 1 -- 172.21.15.7:0/598513517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be815dd10 msgr2=0x7f1be8145320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:46.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.429+0000 7f1bed000640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be815dd10 0x7f1be8145320 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f1be0033040 tx=0x7f1be0009750 comp rx=0 tx=0).stop 2024-04-25T11:20:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.429+0000 7f1bed000640 1 -- 172.21.15.7:0/598513517 shutdown_connections 2024-04-25T11:20:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.429+0000 7f1bed000640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1bc80744b0 0x7f1bc80769c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.430+0000 7f1bed000640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1be815dd10 0x7f1be8145320 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.430+0000 7f1bed000640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1be815d3c0 0x7f1be815d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.430+0000 7f1bed000640 1 --2- 172.21.15.7:0/598513517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1bcc004d40 0x7f1be81fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.430+0000 7f1bed000640 1 -- 172.21.15.7:0/598513517 >> 172.21.15.7:0/598513517 conn(0x7f1be80740f0 msgr2=0x7f1be8166160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.430+0000 7f1bed000640 1 -- 172.21.15.7:0/598513517 shutdown_connections 2024-04-25T11:20:46.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.430+0000 7f1bed000640 1 -- 172.21.15.7:0/598513517 wait complete. 2024-04-25T11:20:46.459 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:20:46.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:46.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.744+0000 7f42dd600640 1 Processor -- start 2024-04-25T11:20:46.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.744+0000 7f42dd600640 1 -- start start 2024-04-25T11:20:46.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.747+0000 7f42dd600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42d80af540 0x7f42d809f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:46.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.747+0000 7f42dd600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d815f4e0 con 0x7f42d815bc40 2024-04-25T11:20:46.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.747+0000 7f42dd600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d80b0560 con 0x7f42d815ef90 2024-04-25T11:20:46.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.747+0000 7f42dd600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f42d8071190 con 0x7f42d80af540 2024-04-25T11:20:46.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.747+0000 7f42d7400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f42d815bc40 0x7f42d80cf980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48430/0 (socket says 172.21.15.7:48430) 2024-04-25T11:20:46.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.747+0000 7f42d7400640 1 -- 172.21.15.7:0/1048124581 learned_addr learned my addr 172.21.15.7:0/1048124581 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:46.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.747+0000 7f42d6000640 1 --2- 172.21.15.7:0/1048124581 >> v2:172.21.15.7:3300/0 conn(0x7f42d80af540 0x7f42d809f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:46.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.748+0000 7f42d5600640 1 -- 172.21.15.7:0/1048124581 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2498068580 0 0) 0x7f42d815f4e0 con 0x7f42d815bc40 2024-04-25T11:20:46.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.748+0000 7f42d5600640 1 -- 172.21.15.7:0/1048124581 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42bc003520 con 0x7f42d815bc40 2024-04-25T11:20:46.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.748+0000 7f42d6000640 1 -- 172.21.15.7:0/1048124581 >> v1:172.21.15.87:6789/0 conn(0x7f42d815ef90 legacy=0x7f42d80aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:46.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.748+0000 7f42d6000640 1 -- 172.21.15.7:0/1048124581 >> v1:172.21.15.7:6789/0 conn(0x7f42d815bc40 legacy=0x7f42d80cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:46.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.748+0000 7f42d6000640 1 -- 172.21.15.7:0/1048124581 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d80b0840 con 0x7f42d80af540 2024-04-25T11:20:46.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.749+0000 7f42d6000640 1 --2- 172.21.15.7:0/1048124581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80af540 0x7f42d809f790 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f42c40097a0 tx=0x7f42c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1726eacd9beec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:46.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.749+0000 7f42d5600640 1 -- 172.21.15.7:0/1048124581 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4037070 con 0x7f42d80af540 2024-04-25T11:20:46.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.749+0000 7f42d5600640 1 -- 172.21.15.7:0/1048124581 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c4032440 con 0x7f42d80af540 2024-04-25T11:20:46.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.749+0000 7f42d5600640 1 -- 172.21.15.7:0/1048124581 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4032810 con 0x7f42d80af540 2024-04-25T11:20:46.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.750+0000 7f42dd600640 1 -- 172.21.15.7:0/1048124581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80af540 msgr2=0x7f42d809f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:46.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.750+0000 7f42dd600640 1 --2- 172.21.15.7:0/1048124581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80af540 0x7f42d809f790 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f42c40097a0 tx=0x7f42c402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:20:46.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.750+0000 7f42dd600640 1 -- 172.21.15.7:0/1048124581 shutdown_connections 2024-04-25T11:20:46.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.750+0000 7f42dd600640 1 --2- 172.21.15.7:0/1048124581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80af540 0x7f42d809f790 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:46.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.750+0000 7f42dd600640 1 -- 172.21.15.7:0/1048124581 >> 172.21.15.7:0/1048124581 conn(0x7f42d8074200 msgr2=0x7f42d806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:46.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.750+0000 7f42dd600640 1 -- 172.21.15.7:0/1048124581 shutdown_connections 2024-04-25T11:20:46.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.750+0000 7f42dd600640 1 -- 172.21.15.7:0/1048124581 wait complete. 2024-04-25T11:20:46.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.752+0000 7f42dd600640 1 Processor -- start 2024-04-25T11:20:46.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.753+0000 7f42dd600640 1 -- start start 2024-04-25T11:20:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.754+0000 7f42dd600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d815bc40 0x7f42d81f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.754+0000 7f42dd600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42d815ef90 0x7f42d8153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.754+0000 7f42dd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80cf940 0x7f42d80cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.754+0000 7f42dd600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42d80b1320 con 0x7f42d80cf940 2024-04-25T11:20:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.754+0000 7f42dd600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f42d80b0e00 con 0x7f42d815bc40 2024-04-25T11:20:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.754+0000 7f42dd600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f42d80afb80 con 0x7f42d815ef90 2024-04-25T11:20:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.755+0000 7f42d7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80cf940 0x7f42d80cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.755+0000 7f42d7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80cf940 0x7f42d80cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37008/0 (socket says 172.21.15.7:37008) 2024-04-25T11:20:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.755+0000 7f42d7400640 1 -- 172.21.15.7:0/2514673249 learned_addr learned my addr 172.21.15.7:0/2514673249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:46.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.755+0000 7f42d6a00640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d815bc40 0x7f42d81f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.755+0000 7f42d6000640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42d815ef90 0x7f42d8153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.755+0000 7f42d7400640 1 -- 172.21.15.7:0/2514673249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42d815ef90 msgr2=0x7f42d8153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.755+0000 7f42d7400640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42d815ef90 0x7f42d8153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.755+0000 7f42d7400640 1 -- 172.21.15.7:0/2514673249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d815bc40 msgr2=0x7f42d81f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.755+0000 7f42d7400640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d815bc40 0x7f42d81f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.755+0000 7f42d7400640 1 -- 172.21.15.7:0/2514673249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d80d0290 con 0x7f42d80cf940 2024-04-25T11:20:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.755+0000 7f42d6a00640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d815bc40 0x7f42d81f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.756+0000 7f42d7400640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80cf940 0x7f42d80cfd50 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f42d000caa0 tx=0x7f42d000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:46.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.756+0000 7f42cea00640 1 -- 172.21.15.7:0/2514673249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0017070 con 0x7f42d80cf940 2024-04-25T11:20:46.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.756+0000 7f42cea00640 1 -- 172.21.15.7:0/2514673249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d000aba0 con 0x7f42d80cf940 2024-04-25T11:20:46.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.756+0000 7f42cea00640 1 -- 172.21.15.7:0/2514673249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0012400 con 0x7f42d80cf940 2024-04-25T11:20:46.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.756+0000 7f42dd600640 1 -- 172.21.15.7:0/2514673249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d8153850 con 0x7f42d80cf940 2024-04-25T11:20:46.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.756+0000 7f42dd600640 1 -- 172.21.15.7:0/2514673249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d81561c0 con 0x7f42d80cf940 2024-04-25T11:20:46.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.757+0000 7f42dd600640 1 -- 172.21.15.7:0/2514673249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d806a100 con 0x7f42d80cf940 2024-04-25T11:20:46.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.759+0000 7f42cea00640 1 -- 172.21.15.7:0/2514673249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f42d001a070 con 0x7f42d80cf940 2024-04-25T11:20:46.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.760+0000 7f42cea00640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42b80744e0 0x7f42b80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:46.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.760+0000 7f42cea00640 1 -- 172.21.15.7:0/2514673249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f42d0095170 con 0x7f42d80cf940 2024-04-25T11:20:46.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.760+0000 7f42d6a00640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42b80744e0 0x7f42b80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:46.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.761+0000 7f42d6a00640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42b80744e0 0x7f42b80769f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f42c8000f60 tx=0x7f42c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:46.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:46.765+0000 7f42cea00640 1 -- 172.21.15.7:0/2514673249 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f42d80d0290 con 0x7f42d80cf940 2024-04-25T11:20:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.010+0000 7f42dd600640 1 -- 172.21.15.7:0/2514673249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42d806c7c0 con 0x7f42d80cf940 2024-04-25T11:20:47.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.011+0000 7f42cea00640 1 -- 172.21.15.7:0/2514673249 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5174 (secure 0 0 0) 0x7f42d005f6d0 con 0x7f42d80cf940 2024-04-25T11:20:47.008 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:47.008 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2288158899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2288158899},{"type":"v1","addr":"172.21.15.7:6837","nonce":2288158899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3545919245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3545919245},{"type":"v1","addr":"172.21.15.87:6835","nonce":3545919245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8478,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/951407928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":951407928},{"type":"v1","addr":"172.21.15.7:6836","nonce":951407928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:42.908797+0000","modified":"2024-04-25T11:20:46.008582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6833/1243096872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1243096872},{"type":"v1","addr":"172.21.15.87:6833","nonce":1243096872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8484,"qdb_cluster":[8484]},"id":9}]} 2024-04-25T11:20:47.009 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 89 2024-04-25T11:20:47.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.014+0000 7f42dd600640 1 -- 172.21.15.7:0/2514673249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42b80744e0 msgr2=0x7f42b80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.014+0000 7f42dd600640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42b80744e0 0x7f42b80769f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f42c8000f60 tx=0x7f42c8009040 comp rx=0 tx=0).stop 2024-04-25T11:20:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.014+0000 7f42dd600640 1 -- 172.21.15.7:0/2514673249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80cf940 msgr2=0x7f42d80cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.014+0000 7f42dd600640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80cf940 0x7f42d80cfd50 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f42d000caa0 tx=0x7f42d000cf60 comp rx=0 tx=0).stop 2024-04-25T11:20:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.015+0000 7f42dd600640 1 -- 172.21.15.7:0/2514673249 shutdown_connections 2024-04-25T11:20:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.015+0000 7f42dd600640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f42b80744e0 0x7f42b80769f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.015+0000 7f42dd600640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42d80cf940 0x7f42d80cfd50 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.015+0000 7f42dd600640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42d815ef90 0x7f42d8153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.015+0000 7f42dd600640 1 --2- 172.21.15.7:0/2514673249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d815bc40 0x7f42d81f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.015+0000 7f42dd600640 1 -- 172.21.15.7:0/2514673249 >> 172.21.15.7:0/2514673249 conn(0x7f42d8074200 msgr2=0x7f42d8159770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:47.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.015+0000 7f42dd600640 1 -- 172.21.15.7:0/2514673249 shutdown_connections 2024-04-25T11:20:47.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.015+0000 7f42dd600640 1 -- 172.21.15.7:0/2514673249 wait complete. 2024-04-25T11:20:47.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:20:47.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.330+0000 7fb8d1e00640 1 Processor -- start 2024-04-25T11:20:47.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.330+0000 7fb8d1e00640 1 -- start start 2024-04-25T11:20:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.332+0000 7fb8d1e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8cc0af540 0x7fb8cc09f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.332+0000 7fb8d1e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8cc15f4e0 con 0x7fb8cc15ef90 2024-04-25T11:20:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.332+0000 7fb8d1e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8cc0b0560 con 0x7fb8cc15bc40 2024-04-25T11:20:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.332+0000 7fb8d1e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb8cc071190 con 0x7fb8cc0af540 2024-04-25T11:20:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.332+0000 7fb8caa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb8cc15ef90 0x7fb8cc0aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48434/0 (socket says 172.21.15.7:48434) 2024-04-25T11:20:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.332+0000 7fb8caa00640 1 -- 172.21.15.7:0/4189577139 learned_addr learned my addr 172.21.15.7:0/4189577139 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 951828604 0 0) 0x7fb8cc15f4e0 con 0x7fb8cc15ef90 2024-04-25T11:20:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8b0003520 con 0x7fb8cc15ef90 2024-04-25T11:20:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8ca000640 1 --2- 172.21.15.7:0/4189577139 >> v2:172.21.15.87:3300/0 conn(0x7fb8cc0af540 0x7fb8cc09f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2562454105 0 0) 0x7fb8b0003520 con 0x7fb8cc15ef90 2024-04-25T11:20:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 >> v2:172.21.15.87:3300/0 conn(0x7fb8cc0af540 msgr2=0x7fb8cc09f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 --2- 172.21.15.7:0/4189577139 >> v2:172.21.15.87:3300/0 conn(0x7fb8cc0af540 0x7fb8cc09f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 >> v1:172.21.15.87:6789/0 conn(0x7fb8cc15bc40 legacy=0x7fb8cc0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc0b0840 con 0x7fb8cc15ef90 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fb8bc001670 con 0x7fb8cc15ef90 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 >> v1:172.21.15.7:6789/0 conn(0x7fb8cc15ef90 legacy=0x7fb8cc0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b0003a00 0x7fb8b0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8b0004800 0x7fb8b0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8b0005330 0x7fb8b000b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc071190 con 0x7fb8b0003a00 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b000c2c0 con 0x7fb8b0004800 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b000c4c0 con 0x7fb8b0005330 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.333+0000 7fb8caa00640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b0003a00 0x7fb8b0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.334+0000 7fb8ca000640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8b0004800 0x7fb8b0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.334+0000 7fb8caa00640 1 -- 172.21.15.7:0/4189577139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8b0005330 msgr2=0x7fb8b000b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.334+0000 7fb8cb400640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8b0005330 0x7fb8b000b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.334+0000 7fb8caa00640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8b0005330 0x7fb8b000b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.334+0000 7fb8caa00640 1 -- 172.21.15.7:0/4189577139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8b0004800 msgr2=0x7fb8b0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.334+0000 7fb8caa00640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8b0004800 0x7fb8b0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.334+0000 7fb8caa00640 1 -- 172.21.15.7:0/4189577139 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b0003520 con 0x7fb8b0003a00 2024-04-25T11:20:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.334+0000 7fb8ca000640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8b0004800 0x7fb8b0004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.334+0000 7fb8cb400640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8b0005330 0x7fb8b000b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.334+0000 7fb8cb400640 1 -- 172.21.15.7:0/4189577139 reap_dead start 2024-04-25T11:20:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.335+0000 7fb8caa00640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b0003a00 0x7fb8b0003e50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00c190 tx=0x7fb8b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245dd89488f54107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.335+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc007c60 con 0x7fb8b0003a00 2024-04-25T11:20:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.335+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8bc005ad0 con 0x7fb8b0003a00 2024-04-25T11:20:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.335+0000 7fb8c9600640 1 -- 172.21.15.7:0/4189577139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc007070 con 0x7fb8b0003a00 2024-04-25T11:20:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.336+0000 7fb8d1e00640 1 -- 172.21.15.7:0/4189577139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b0003a00 msgr2=0x7fb8b0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.336+0000 7fb8d1e00640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b0003a00 0x7fb8b0003e50 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00c190 tx=0x7fb8b0005830 comp rx=0 tx=0).stop 2024-04-25T11:20:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.336+0000 7fb8d1e00640 1 -- 172.21.15.7:0/4189577139 shutdown_connections 2024-04-25T11:20:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.336+0000 7fb8d1e00640 1 --2- 172.21.15.7:0/4189577139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8b0003a00 0x7fb8b0003e50 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.336+0000 7fb8d1e00640 1 -- 172.21.15.7:0/4189577139 >> 172.21.15.7:0/4189577139 conn(0x7fb8cc074200 msgr2=0x7fb8cc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.336+0000 7fb8d1e00640 1 -- 172.21.15.7:0/4189577139 shutdown_connections 2024-04-25T11:20:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.336+0000 7fb8d1e00640 1 -- 172.21.15.7:0/4189577139 wait complete. 2024-04-25T11:20:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.339+0000 7fb8d1e00640 1 Processor -- start 2024-04-25T11:20:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.339+0000 7fb8d1e00640 1 -- start start 2024-04-25T11:20:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.340+0000 7fb8d1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8b0003a00 0x7fb8cc0b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.340+0000 7fb8d1e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8cc1505c0 0x7fb8cc0b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.340+0000 7fb8d1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1500b0 0x7fb8cc15c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.340+0000 7fb8d1e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b1320 con 0x7fb8cc1500b0 2024-04-25T11:20:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.340+0000 7fb8d1e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b0e00 con 0x7fb8b0003a00 2024-04-25T11:20:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.340+0000 7fb8d1e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8cc0afb80 con 0x7fb8cc1505c0 2024-04-25T11:20:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.341+0000 7fb8cb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1500b0 0x7fb8cc15c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.341+0000 7fb8cb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1500b0 0x7fb8cc15c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37022/0 (socket says 172.21.15.7:37022) 2024-04-25T11:20:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.341+0000 7fb8cb400640 1 -- 172.21.15.7:0/2446286108 learned_addr learned my addr 172.21.15.7:0/2446286108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.341+0000 7fb8caa00640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8b0003a00 0x7fb8cc0b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.341+0000 7fb8ca000640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8cc1505c0 0x7fb8cc0b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.341+0000 7fb8cb400640 1 -- 172.21.15.7:0/2446286108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8cc1505c0 msgr2=0x7fb8cc0b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.341+0000 7fb8cb400640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8cc1505c0 0x7fb8cc0b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.341+0000 7fb8cb400640 1 -- 172.21.15.7:0/2446286108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8b0003a00 msgr2=0x7fb8cc0b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.341+0000 7fb8cb400640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8b0003a00 0x7fb8cc0b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.341+0000 7fb8cb400640 1 -- 172.21.15.7:0/2446286108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc0b4e20 con 0x7fb8cc1500b0 2024-04-25T11:20:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.342+0000 7fb8cb400640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1500b0 0x7fb8cc15c0b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0cf1e0 tx=0x7fb8c400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.342+0000 7fb8c3400640 1 -- 172.21.15.7:0/2446286108 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4006b50 con 0x7fb8cc1500b0 2024-04-25T11:20:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.342+0000 7fb8c3400640 1 -- 172.21.15.7:0/2446286108 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c4006d10 con 0x7fb8cc1500b0 2024-04-25T11:20:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.342+0000 7fb8c3400640 1 -- 172.21.15.7:0/2446286108 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4012840 con 0x7fb8cc1500b0 2024-04-25T11:20:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.342+0000 7fb8d1e00640 1 -- 172.21.15.7:0/2446286108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc0b5100 con 0x7fb8cc1500b0 2024-04-25T11:20:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.342+0000 7fb8d1e00640 1 -- 172.21.15.7:0/2446286108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc0b56c0 con 0x7fb8cc1500b0 2024-04-25T11:20:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.344+0000 7fb8d1e00640 1 -- 172.21.15.7:0/2446286108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8cc0b4b70 con 0x7fb8cc1500b0 2024-04-25T11:20:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.345+0000 7fb8c3400640 1 -- 172.21.15.7:0/2446286108 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb8c401b020 con 0x7fb8cc1500b0 2024-04-25T11:20:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.346+0000 7fb8c3400640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8ac0744b0 0x7fb8ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.346+0000 7fb8c3400640 1 -- 172.21.15.7:0/2446286108 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7fb8c40949b0 con 0x7fb8cc1500b0 2024-04-25T11:20:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.346+0000 7fb8caa00640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8ac0744b0 0x7fb8ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.347+0000 7fb8caa00640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8ac0744b0 0x7fb8ac0769c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc032000 tx=0x7fb8bc009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:47.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.351+0000 7fb8c3400640 1 -- 172.21.15.7:0/2446286108 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb8c405f030 con 0x7fb8cc1500b0 2024-04-25T11:20:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.557+0000 7fb8d1e00640 1 -- 172.21.15.7:0/2446286108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8cc06a100 con 0x7fb8cc1500b0 2024-04-25T11:20:47.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.559+0000 7fb8c3400640 1 -- 172.21.15.7:0/2446286108 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16804 (secure 0 0 0) 0x7fb8c405e930 con 0x7fb8cc1500b0 2024-04-25T11:20:47.557 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:47.557 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":130,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:20:44.396251+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:41.047420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:41.908907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:20:47.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 -- 172.21.15.7:0/2446286108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8ac0744b0 msgr2=0x7fb8ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:47.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8ac0744b0 0x7fb8ac0769c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc032000 tx=0x7fb8bc009030 comp rx=0 tx=0).stop 2024-04-25T11:20:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 -- 172.21.15.7:0/2446286108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1500b0 msgr2=0x7fb8cc15c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1500b0 0x7fb8cc15c0b0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0cf1e0 tx=0x7fb8c400d9d0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 -- 172.21.15.7:0/2446286108 shutdown_connections 2024-04-25T11:20:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb8ac0744b0 0x7fb8ac0769c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8cc1500b0 0x7fb8cc15c0b0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8cc1505c0 0x7fb8cc0b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 --2- 172.21.15.7:0/2446286108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8b0003a00 0x7fb8cc0b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 -- 172.21.15.7:0/2446286108 >> 172.21.15.7:0/2446286108 conn(0x7fb8cc074200 msgr2=0x7fb8cc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 -- 172.21.15.7:0/2446286108 shutdown_connections 2024-04-25T11:20:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.562+0000 7fb8d1e00640 1 -- 172.21.15.7:0/2446286108 wait complete. 2024-04-25T11:20:47.584 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:20:47.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:20:47.840 INFO:tasks.ceph.mds.b.smithi087.stderr:2024-04-25T11:20:47.843+0000 7f0799e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:20:47.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.871+0000 7fa34b200640 1 Processor -- start 2024-04-25T11:20:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.871+0000 7fa34b200640 1 -- start start 2024-04-25T11:20:47.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.873+0000 7fa34b200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa34c070a70 0x7fa34c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.873+0000 7fa34b200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa34c0aedc0 0x7fa34c0b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.873+0000 7fa34b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa34c0b0a10 0x7fa34c1607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.873+0000 7fa34b200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa34c15e190 con 0x7fa34c0aedc0 2024-04-25T11:20:47.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.873+0000 7fa34b200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa34c15d610 con 0x7fa34c070a70 2024-04-25T11:20:47.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.873+0000 7fa34b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa34c15d8f0 con 0x7fa34c0b0a10 2024-04-25T11:20:47.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.874+0000 7fa343e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa34c0aedc0 0x7fa34c0b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.874+0000 7fa343e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa34c0aedc0 0x7fa34c0b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37034/0 (socket says 172.21.15.7:37034) 2024-04-25T11:20:47.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.874+0000 7fa343e00640 1 -- 172.21.15.7:0/1279981898 learned_addr learned my addr 172.21.15.7:0/1279981898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.874+0000 7fa349e00640 1 --2- 172.21.15.7:0/1279981898 >> v2:172.21.15.87:3300/0 conn(0x7fa34c070a70 0x7fa34c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.874+0000 7fa34a800640 1 --2- 172.21.15.7:0/1279981898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa34c0b0a10 0x7fa34c1607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.874+0000 7fa343e00640 1 -- 172.21.15.7:0/1279981898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa34c0b0a10 msgr2=0x7fa34c1607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.874+0000 7fa343e00640 1 --2- 172.21.15.7:0/1279981898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa34c0b0a10 0x7fa34c1607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.874+0000 7fa343e00640 1 -- 172.21.15.7:0/1279981898 >> v2:172.21.15.87:3300/0 conn(0x7fa34c070a70 msgr2=0x7fa34c0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.874+0000 7fa343e00640 1 --2- 172.21.15.7:0/1279981898 >> v2:172.21.15.87:3300/0 conn(0x7fa34c070a70 0x7fa34c0ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.874+0000 7fa343e00640 1 -- 172.21.15.7:0/1279981898 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c15dbd0 con 0x7fa34c0aedc0 2024-04-25T11:20:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.876+0000 7fa343e00640 1 --2- 172.21.15.7:0/1279981898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa34c0aedc0 0x7fa34c0b04d0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0725e0 tx=0x7fa33402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1812f606aab0bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.876+0000 7fa349400640 1 -- 172.21.15.7:0/1279981898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334037070 con 0x7fa34c0aedc0 2024-04-25T11:20:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.876+0000 7fa349400640 1 -- 172.21.15.7:0/1279981898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa334032440 con 0x7fa34c0aedc0 2024-04-25T11:20:47.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.876+0000 7fa349400640 1 -- 172.21.15.7:0/1279981898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334032810 con 0x7fa34c0aedc0 2024-04-25T11:20:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.877+0000 7fa34b200640 1 -- 172.21.15.7:0/1279981898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa34c0aedc0 msgr2=0x7fa34c0b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.877+0000 7fa34b200640 1 --2- 172.21.15.7:0/1279981898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa34c0aedc0 0x7fa34c0b04d0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0725e0 tx=0x7fa33402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.877+0000 7fa34b200640 1 -- 172.21.15.7:0/1279981898 shutdown_connections 2024-04-25T11:20:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.877+0000 7fa34b200640 1 --2- 172.21.15.7:0/1279981898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa34c0b0a10 0x7fa34c1607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.877+0000 7fa34b200640 1 --2- 172.21.15.7:0/1279981898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa34c0aedc0 0x7fa34c0b04d0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.877+0000 7fa34b200640 1 --2- 172.21.15.7:0/1279981898 >> v2:172.21.15.87:3300/0 conn(0x7fa34c070a70 0x7fa34c0ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.877+0000 7fa34b200640 1 -- 172.21.15.7:0/1279981898 >> 172.21.15.7:0/1279981898 conn(0x7fa34c0b3820 msgr2=0x7fa34c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:47.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.877+0000 7fa34b200640 1 -- 172.21.15.7:0/1279981898 shutdown_connections 2024-04-25T11:20:47.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.877+0000 7fa34b200640 1 -- 172.21.15.7:0/1279981898 wait complete. 2024-04-25T11:20:47.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.879+0000 7fa34b200640 1 Processor -- start 2024-04-25T11:20:47.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.879+0000 7fa34b200640 1 -- start start 2024-04-25T11:20:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.881+0000 7fa34b200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa34c070a70 0x7fa34c14bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.881+0000 7fa34b200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa34c0aedc0 0x7fa34c15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.881+0000 7fa34b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa34c0b0a10 0x7fa34c15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.881+0000 7fa34b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c15d170 con 0x7fa34c0b0a10 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.881+0000 7fa34b200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c0b0e20 con 0x7fa34c070a70 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.881+0000 7fa34b200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa34c15d2f0 con 0x7fa34c0aedc0 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.881+0000 7fa34a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa34c0b0a10 0x7fa34c15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa34a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa34c0b0a10 0x7fa34c15ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37036/0 (socket says 172.21.15.7:37036) 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa34a800640 1 -- 172.21.15.7:0/1426087085 learned_addr learned my addr 172.21.15.7:0/1426087085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa343e00640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa34c0aedc0 0x7fa34c15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa349e00640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa34c070a70 0x7fa34c14bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa34a800640 1 -- 172.21.15.7:0/1426087085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa34c0aedc0 msgr2=0x7fa34c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa34a800640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa34c0aedc0 0x7fa34c15c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa34a800640 1 -- 172.21.15.7:0/1426087085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa34c070a70 msgr2=0x7fa34c14bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa34a800640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa34c070a70 0x7fa34c14bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa34a800640 1 -- 172.21.15.7:0/1426087085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34c14cc70 con 0x7fa34c0b0a10 2024-04-25T11:20:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa34a800640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa34c0b0a10 0x7fa34c15ba00 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa344007d80 tx=0x7fa34400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa342000640 1 -- 172.21.15.7:0/1426087085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344014420 con 0x7fa34c0b0a10 2024-04-25T11:20:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.882+0000 7fa342000640 1 -- 172.21.15.7:0/1426087085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa344019070 con 0x7fa34c0b0a10 2024-04-25T11:20:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.883+0000 7fa34b200640 1 -- 172.21.15.7:0/1426087085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34c14e8e0 con 0x7fa34c0b0a10 2024-04-25T11:20:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.883+0000 7fa342000640 1 -- 172.21.15.7:0/1426087085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344014800 con 0x7fa34c0b0a10 2024-04-25T11:20:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.883+0000 7fa34b200640 1 -- 172.21.15.7:0/1426087085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34c14eea0 con 0x7fa34c0b0a10 2024-04-25T11:20:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.884+0000 7fa34b200640 1 -- 172.21.15.7:0/1426087085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa34c050740 con 0x7fa34c0b0a10 2024-04-25T11:20:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.885+0000 7fa342000640 1 -- 172.21.15.7:0/1426087085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa3440149a0 con 0x7fa34c0b0a10 2024-04-25T11:20:47.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.886+0000 7fa342000640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3280744b0 0x7fa3280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:47.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.887+0000 7fa342000640 1 -- 172.21.15.7:0/1426087085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7fa344095cd0 con 0x7fa34c0b0a10 2024-04-25T11:20:47.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.887+0000 7fa349e00640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3280744b0 0x7fa3280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:47.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.887+0000 7fa349e00640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3280744b0 0x7fa3280769c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa33c006fd0 tx=0x7fa33c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:47.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:47.891+0000 7fa342000640 1 -- 172.21.15.7:0/1426087085 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa3440602c0 con 0x7fa34c0b0a10 2024-04-25T11:20:48.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.083+0000 7fa34b200640 1 -- 172.21.15.7:0/1426087085 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa34c160de0 con 0x7fa3280744b0 2024-04-25T11:20:48.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.095+0000 7fa342000640 1 -- 172.21.15.7:0/1426087085 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252463 (secure 0 0 0) 0x7fa34c160de0 con 0x7fa3280744b0 2024-04-25T11:20:48.096 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.100+0000 7fa34b200640 1 -- 172.21.15.7:0/1426087085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3280744b0 msgr2=0x7fa3280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.100+0000 7fa34b200640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3280744b0 0x7fa3280769c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa33c006fd0 tx=0x7fa33c008040 comp rx=0 tx=0).stop 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.100+0000 7fa34b200640 1 -- 172.21.15.7:0/1426087085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa34c0b0a10 msgr2=0x7fa34c15ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.100+0000 7fa34b200640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa34c0b0a10 0x7fa34c15ba00 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa344007d80 tx=0x7fa34400e4b0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.101+0000 7fa34b200640 1 -- 172.21.15.7:0/1426087085 shutdown_connections 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.101+0000 7fa34b200640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa3280744b0 0x7fa3280769c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.101+0000 7fa34b200640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa34c0b0a10 0x7fa34c15ba00 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.101+0000 7fa34b200640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa34c0aedc0 0x7fa34c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.101+0000 7fa34b200640 1 --2- 172.21.15.7:0/1426087085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa34c070a70 0x7fa34c14bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.101+0000 7fa34b200640 1 -- 172.21.15.7:0/1426087085 >> 172.21.15.7:0/1426087085 conn(0x7fa34c0b3820 msgr2=0x7fa34c0684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.101+0000 7fa34b200640 1 -- 172.21.15.7:0/1426087085 shutdown_connections 2024-04-25T11:20:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.101+0000 7fa34b200640 1 -- 172.21.15.7:0/1426087085 wait complete. 2024-04-25T11:20:48.122 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-04-25T11:20:48.031637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279892,"kb_used_data":13796,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754694828,"statfs":{"total":773094113280,"available":772807503872,"internally_reserved":0,"allocated":14127104,"data_stored":7131648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001864"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.419364+0000","last_change":"2024-04-25T11:20:43.413723+0000","last_active":"2024-04-25T11:20:43.419364+0000","last_peered":"2024-04-25T11:20:43.419364+0000","last_clean":"2024-04-25T11:20:43.419364+0000","last_became_active":"2024-04-25T11:20:43.413305+0000","last_became_peered":"2024-04-25T11:20:43.413305+0000","last_unstale":"2024-04-25T11:20:43.419364+0000","last_undegraded":"2024-04-25T11:20:43.419364+0000","last_fullsized":"2024-04-25T11:20:43.419364+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:40.208838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419781+0000","last_change":"2024-04-25T11:20:43.412353+0000","last_active":"2024-04-25T11:20:44.419781+0000","last_peered":"2024-04-25T11:20:44.419781+0000","last_clean":"2024-04-25T11:20:44.419781+0000","last_became_active":"2024-04-25T11:20:43.411031+0000","last_became_peered":"2024-04-25T11:20:43.411031+0000","last_unstale":"2024-04-25T11:20:44.419781+0000","last_undegraded":"2024-04-25T11:20:44.419781+0000","last_fullsized":"2024-04-25T11:20:44.419781+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:25.874772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422560+0000","last_change":"2024-04-25T11:20:43.425339+0000","last_active":"2024-04-25T11:20:44.422560+0000","last_peered":"2024-04-25T11:20:44.422560+0000","last_clean":"2024-04-25T11:20:44.422560+0000","last_became_active":"2024-04-25T11:20:43.424773+0000","last_became_peered":"2024-04-25T11:20:43.424773+0000","last_unstale":"2024-04-25T11:20:44.422560+0000","last_undegraded":"2024-04-25T11:20:44.422560+0000","last_fullsized":"2024-04-25T11:20:44.422560+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:26.671112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419161+0000","last_change":"2024-04-25T11:20:43.409738+0000","last_active":"2024-04-25T11:20:44.419161+0000","last_peered":"2024-04-25T11:20:44.419161+0000","last_clean":"2024-04-25T11:20:44.419161+0000","last_became_active":"2024-04-25T11:20:43.409403+0000","last_became_peered":"2024-04-25T11:20:43.409403+0000","last_unstale":"2024-04-25T11:20:44.419161+0000","last_undegraded":"2024-04-25T11:20:44.419161+0000","last_fullsized":"2024-04-25T11:20:44.419161+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:26.671112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599032+0000","last_change":"2024-04-25T11:20:43.417699+0000","last_active":"2024-04-25T11:20:44.599032+0000","last_peered":"2024-04-25T11:20:44.599032+0000","last_clean":"2024-04-25T11:20:44.599032+0000","last_became_active":"2024-04-25T11:20:43.417342+0000","last_became_peered":"2024-04-25T11:20:43.417342+0000","last_unstale":"2024-04-25T11:20:44.599032+0000","last_undegraded":"2024-04-25T11:20:44.599032+0000","last_fullsized":"2024-04-25T11:20:44.599032+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:37.741543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598801+0000","last_change":"2024-04-25T11:20:43.417042+0000","last_active":"2024-04-25T11:20:44.598801+0000","last_peered":"2024-04-25T11:20:44.598801+0000","last_clean":"2024-04-25T11:20:44.598801+0000","last_became_active":"2024-04-25T11:20:43.416676+0000","last_became_peered":"2024-04-25T11:20:43.416676+0000","last_unstale":"2024-04-25T11:20:44.598801+0000","last_undegraded":"2024-04-25T11:20:44.598801+0000","last_fullsized":"2024-04-25T11:20:44.598801+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:33.381386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421685+0000","last_change":"2024-04-25T11:20:43.414802+0000","last_active":"2024-04-25T11:20:44.421685+0000","last_peered":"2024-04-25T11:20:44.421685+0000","last_clean":"2024-04-25T11:20:44.421685+0000","last_became_active":"2024-04-25T11:20:43.414476+0000","last_became_peered":"2024-04-25T11:20:43.414476+0000","last_unstale":"2024-04-25T11:20:44.421685+0000","last_undegraded":"2024-04-25T11:20:44.421685+0000","last_fullsized":"2024-04-25T11:20:44.421685+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:53.766981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597973+0000","last_change":"2024-04-25T11:20:43.426167+0000","last_active":"2024-04-25T11:20:44.597973+0000","last_peered":"2024-04-25T11:20:44.597973+0000","last_clean":"2024-04-25T11:20:44.597973+0000","last_became_active":"2024-04-25T11:20:43.425785+0000","last_became_peered":"2024-04-25T11:20:43.425785+0000","last_unstale":"2024-04-25T11:20:44.597973+0000","last_undegraded":"2024-04-25T11:20:44.597973+0000","last_fullsized":"2024-04-25T11:20:44.597973+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:45.799504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599700+0000","last_change":"2024-04-25T11:20:43.414574+0000","last_active":"2024-04-25T11:20:44.599700+0000","last_peered":"2024-04-25T11:20:44.599700+0000","last_clean":"2024-04-25T11:20:44.599700+0000","last_became_active":"2024-04-25T11:20:43.413194+0000","last_became_peered":"2024-04-25T11:20:43.413194+0000","last_unstale":"2024-04-25T11:20:44.599700+0000","last_undegraded":"2024-04-25T11:20:44.599700+0000","last_fullsized":"2024-04-25T11:20:44.599700+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:38.437764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564706+0000","last_change":"2024-04-25T11:20:43.414707+0000","last_active":"2024-04-25T11:20:44.564706+0000","last_peered":"2024-04-25T11:20:44.564706+0000","last_clean":"2024-04-25T11:20:44.564706+0000","last_became_active":"2024-04-25T11:20:43.414348+0000","last_became_peered":"2024-04-25T11:20:43.414348+0000","last_unstale":"2024-04-25T11:20:44.564706+0000","last_undegraded":"2024-04-25T11:20:44.564706+0000","last_fullsized":"2024-04-25T11:20:44.564706+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:47.853731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419405+0000","last_change":"2024-04-25T11:20:43.417885+0000","last_active":"2024-04-25T11:20:44.419405+0000","last_peered":"2024-04-25T11:20:44.419405+0000","last_clean":"2024-04-25T11:20:44.419405+0000","last_became_active":"2024-04-25T11:20:43.415456+0000","last_became_peered":"2024-04-25T11:20:43.415456+0000","last_unstale":"2024-04-25T11:20:44.419405+0000","last_undegraded":"2024-04-25T11:20:44.419405+0000","last_fullsized":"2024-04-25T11:20:44.419405+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:08.190683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.418596+0000","last_change":"2024-04-25T11:20:43.409715+0000","last_active":"2024-04-25T11:20:43.418596+0000","last_peered":"2024-04-25T11:20:43.418596+0000","last_clean":"2024-04-25T11:20:43.418596+0000","last_became_active":"2024-04-25T11:20:43.409259+0000","last_became_peered":"2024-04-25T11:20:43.409259+0000","last_unstale":"2024-04-25T11:20:43.418596+0000","last_undegraded":"2024-04-25T11:20:43.418596+0000","last_fullsized":"2024-04-25T11:20:43.418596+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:58.995466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599337+0000","last_change":"2024-04-25T11:20:43.427371+0000","last_active":"2024-04-25T11:20:44.599337+0000","last_peered":"2024-04-25T11:20:44.599337+0000","last_clean":"2024-04-25T11:20:44.599337+0000","last_became_active":"2024-04-25T11:20:43.426966+0000","last_became_peered":"2024-04-25T11:20:43.426966+0000","last_unstale":"2024-04-25T11:20:44.599337+0000","last_undegraded":"2024-04-25T11:20:44.599337+0000","last_fullsized":"2024-04-25T11:20:44.599337+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:16.001880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.422029+0000","last_change":"2024-04-25T11:20:43.416071+0000","last_active":"2024-04-25T11:20:43.422029+0000","last_peered":"2024-04-25T11:20:43.422029+0000","last_clean":"2024-04-25T11:20:43.422029+0000","last_became_active":"2024-04-25T11:20:43.415749+0000","last_became_peered":"2024-04-25T11:20:43.415749+0000","last_unstale":"2024-04-25T11:20:43.422029+0000","last_undegraded":"2024-04-25T11:20:43.422029+0000","last_fullsized":"2024-04-25T11:20:43.422029+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:15.942085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.422452+0000","last_change":"2024-04-25T11:20:43.417524+0000","last_active":"2024-04-25T11:20:43.422452+0000","last_peered":"2024-04-25T11:20:43.422452+0000","last_clean":"2024-04-25T11:20:43.422452+0000","last_became_active":"2024-04-25T11:20:43.417155+0000","last_became_peered":"2024-04-25T11:20:43.417155+0000","last_unstale":"2024-04-25T11:20:43.422452+0000","last_undegraded":"2024-04-25T11:20:43.422452+0000","last_fullsized":"2024-04-25T11:20:43.422452+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:09.432829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597628+0000","last_change":"2024-04-25T11:20:43.404825+0000","last_active":"2024-04-25T11:20:44.597628+0000","last_peered":"2024-04-25T11:20:44.597628+0000","last_clean":"2024-04-25T11:20:44.597628+0000","last_became_active":"2024-04-25T11:20:43.404315+0000","last_became_peered":"2024-04-25T11:20:43.404315+0000","last_unstale":"2024-04-25T11:20:44.597628+0000","last_undegraded":"2024-04-25T11:20:44.597628+0000","last_fullsized":"2024-04-25T11:20:44.597628+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:25.874772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.600186+0000","last_change":"2024-04-25T11:20:43.411053+0000","last_active":"2024-04-25T11:20:44.600186+0000","last_peered":"2024-04-25T11:20:44.600186+0000","last_clean":"2024-04-25T11:20:44.600186+0000","last_became_active":"2024-04-25T11:20:43.409332+0000","last_became_peered":"2024-04-25T11:20:43.409332+0000","last_unstale":"2024-04-25T11:20:44.600186+0000","last_undegraded":"2024-04-25T11:20:44.600186+0000","last_fullsized":"2024-04-25T11:20:44.600186+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:51.662580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.596706+0000","last_change":"2024-04-25T11:20:43.420507+0000","last_active":"2024-04-25T11:20:44.596706+0000","last_peered":"2024-04-25T11:20:44.596706+0000","last_clean":"2024-04-25T11:20:44.596706+0000","last_became_active":"2024-04-25T11:20:43.418330+0000","last_became_peered":"2024-04-25T11:20:43.418330+0000","last_unstale":"2024-04-25T11:20:44.596706+0000","last_undegraded":"2024-04-25T11:20:44.596706+0000","last_fullsized":"2024-04-25T11:20:44.596706+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:53.766981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423381+0000","last_change":"2024-04-25T11:20:43.415336+0000","last_active":"2024-04-25T11:20:44.423381+0000","last_peered":"2024-04-25T11:20:44.423381+0000","last_clean":"2024-04-25T11:20:44.423381+0000","last_became_active":"2024-04-25T11:20:43.412492+0000","last_became_peered":"2024-04-25T11:20:43.412492+0000","last_unstale":"2024-04-25T11:20:44.423381+0000","last_undegraded":"2024-04-25T11:20:44.423381+0000","last_fullsized":"2024-04-25T11:20:44.423381+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:46.271892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.596950+0000","last_change":"2024-04-25T11:20:43.400897+0000","last_active":"2024-04-25T11:20:44.596950+0000","last_peered":"2024-04-25T11:20:44.596950+0000","last_clean":"2024-04-25T11:20:44.596950+0000","last_became_active":"2024-04-25T11:20:43.397411+0000","last_became_peered":"2024-04-25T11:20:43.397411+0000","last_unstale":"2024-04-25T11:20:44.596950+0000","last_undegraded":"2024-04-25T11:20:44.596950+0000","last_fullsized":"2024-04-25T11:20:44.596950+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:39.941085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.418438+0000","last_change":"2024-04-25T11:20:43.408303+0000","last_active":"2024-04-25T11:20:44.418438+0000","last_peered":"2024-04-25T11:20:44.418438+0000","last_clean":"2024-04-25T11:20:44.418438+0000","last_became_active":"2024-04-25T11:20:43.407917+0000","last_became_peered":"2024-04-25T11:20:43.407917+0000","last_unstale":"2024-04-25T11:20:44.418438+0000","last_undegraded":"2024-04-25T11:20:44.418438+0000","last_fullsized":"2024-04-25T11:20:44.418438+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:02.808346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.414530+0000","last_change":"2024-04-25T11:20:43.408247+0000","last_active":"2024-04-25T11:20:43.414530+0000","last_peered":"2024-04-25T11:20:43.414530+0000","last_clean":"2024-04-25T11:20:43.414530+0000","last_became_active":"2024-04-25T11:20:43.407771+0000","last_became_peered":"2024-04-25T11:20:43.407771+0000","last_unstale":"2024-04-25T11:20:43.414530+0000","last_undegraded":"2024-04-25T11:20:43.414530+0000","last_fullsized":"2024-04-25T11:20:43.414530+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:05.393647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.418810+0000","last_change":"2024-04-25T11:20:43.397553+0000","last_active":"2024-04-25T11:20:44.418810+0000","last_peered":"2024-04-25T11:20:44.418810+0000","last_clean":"2024-04-25T11:20:44.418810+0000","last_became_active":"2024-04-25T11:20:43.396684+0000","last_became_peered":"2024-04-25T11:20:43.396684+0000","last_unstale":"2024-04-25T11:20:44.418810+0000","last_undegraded":"2024-04-25T11:20:44.418810+0000","last_fullsized":"2024-04-25T11:20:44.418810+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:42.730597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597917+0000","last_change":"2024-04-25T11:20:43.400746+0000","last_active":"2024-04-25T11:20:44.597917+0000","last_peered":"2024-04-25T11:20:44.597917+0000","last_clean":"2024-04-25T11:20:44.597917+0000","last_became_active":"2024-04-25T11:20:43.400339+0000","last_became_peered":"2024-04-25T11:20:43.400339+0000","last_unstale":"2024-04-25T11:20:44.597917+0000","last_undegraded":"2024-04-25T11:20:44.597917+0000","last_fullsized":"2024-04-25T11:20:44.597917+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:36.184438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422573+0000","last_change":"2024-04-25T11:20:43.429453+0000","last_active":"2024-04-25T11:20:44.422573+0000","last_peered":"2024-04-25T11:20:44.422573+0000","last_clean":"2024-04-25T11:20:44.422573+0000","last_became_active":"2024-04-25T11:20:43.428920+0000","last_became_peered":"2024-04-25T11:20:43.428920+0000","last_unstale":"2024-04-25T11:20:44.422573+0000","last_undegraded":"2024-04-25T11:20:44.422573+0000","last_fullsized":"2024-04-25T11:20:44.422573+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:08.190683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422818+0000","last_change":"2024-04-25T11:20:43.424634+0000","last_active":"2024-04-25T11:20:44.422818+0000","last_peered":"2024-04-25T11:20:44.422818+0000","last_clean":"2024-04-25T11:20:44.422818+0000","last_became_active":"2024-04-25T11:20:43.424279+0000","last_became_peered":"2024-04-25T11:20:43.424279+0000","last_unstale":"2024-04-25T11:20:44.422818+0000","last_undegraded":"2024-04-25T11:20:44.422818+0000","last_fullsized":"2024-04-25T11:20:44.422818+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:25.874772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423792+0000","last_change":"2024-04-25T11:20:43.408193+0000","last_active":"2024-04-25T11:20:44.423792+0000","last_peered":"2024-04-25T11:20:44.423792+0000","last_clean":"2024-04-25T11:20:44.423792+0000","last_became_active":"2024-04-25T11:20:43.407856+0000","last_became_peered":"2024-04-25T11:20:43.407856+0000","last_unstale":"2024-04-25T11:20:44.423792+0000","last_undegraded":"2024-04-25T11:20:44.423792+0000","last_fullsized":"2024-04-25T11:20:44.423792+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:44.774829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564433+0000","last_change":"2024-04-25T11:20:43.416430+0000","last_active":"2024-04-25T11:20:44.564433+0000","last_peered":"2024-04-25T11:20:44.564433+0000","last_clean":"2024-04-25T11:20:44.564433+0000","last_became_active":"2024-04-25T11:20:43.414706+0000","last_became_peered":"2024-04-25T11:20:43.414706+0000","last_unstale":"2024-04-25T11:20:44.564433+0000","last_undegraded":"2024-04-25T11:20:44.564433+0000","last_fullsized":"2024-04-25T11:20:44.564433+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.802813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423209+0000","last_change":"2024-04-25T11:20:43.426856+0000","last_active":"2024-04-25T11:20:44.423209+0000","last_peered":"2024-04-25T11:20:44.423209+0000","last_clean":"2024-04-25T11:20:44.423209+0000","last_became_active":"2024-04-25T11:20:43.426305+0000","last_became_peered":"2024-04-25T11:20:43.426305+0000","last_unstale":"2024-04-25T11:20:44.423209+0000","last_undegraded":"2024-04-25T11:20:44.423209+0000","last_fullsized":"2024-04-25T11:20:44.423209+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:36.184438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421910+0000","last_change":"2024-04-25T11:20:43.415498+0000","last_active":"2024-04-25T11:20:44.421910+0000","last_peered":"2024-04-25T11:20:44.421910+0000","last_clean":"2024-04-25T11:20:44.421910+0000","last_became_active":"2024-04-25T11:20:43.415186+0000","last_became_peered":"2024-04-25T11:20:43.415186+0000","last_unstale":"2024-04-25T11:20:44.421910+0000","last_undegraded":"2024-04-25T11:20:44.421910+0000","last_fullsized":"2024-04-25T11:20:44.421910+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:02.808346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598214+0000","last_change":"2024-04-25T11:20:43.426123+0000","last_active":"2024-04-25T11:20:44.598214+0000","last_peered":"2024-04-25T11:20:44.598214+0000","last_clean":"2024-04-25T11:20:44.598214+0000","last_became_active":"2024-04-25T11:20:43.425219+0000","last_became_peered":"2024-04-25T11:20:43.425219+0000","last_unstale":"2024-04-25T11:20:44.598214+0000","last_undegraded":"2024-04-25T11:20:44.598214+0000","last_fullsized":"2024-04-25T11:20:44.598214+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:40.892114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564024+0000","last_change":"2024-04-25T11:20:43.420104+0000","last_active":"2024-04-25T11:20:44.564024+0000","last_peered":"2024-04-25T11:20:44.564024+0000","last_clean":"2024-04-25T11:20:44.564024+0000","last_became_active":"2024-04-25T11:20:43.419535+0000","last_became_peered":"2024-04-25T11:20:43.419535+0000","last_unstale":"2024-04-25T11:20:44.564024+0000","last_undegraded":"2024-04-25T11:20:44.564024+0000","last_fullsized":"2024-04-25T11:20:44.564024+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:12.156522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597994+0000","last_change":"2024-04-25T11:20:43.419261+0000","last_active":"2024-04-25T11:20:44.597994+0000","last_peered":"2024-04-25T11:20:44.597994+0000","last_clean":"2024-04-25T11:20:44.597994+0000","last_became_active":"2024-04-25T11:20:43.417934+0000","last_became_peered":"2024-04-25T11:20:43.417934+0000","last_unstale":"2024-04-25T11:20:44.597994+0000","last_undegraded":"2024-04-25T11:20:44.597994+0000","last_fullsized":"2024-04-25T11:20:44.597994+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:26.122782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"129'4","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422680+0000","last_change":"2024-04-25T11:20:42.373454+0000","last_active":"2024-04-25T11:20:44.422680+0000","last_peered":"2024-04-25T11:20:44.422680+0000","last_clean":"2024-04-25T11:20:44.422680+0000","last_became_active":"2024-04-25T11:20:42.373041+0000","last_became_peered":"2024-04-25T11:20:42.373041+0000","last_unstale":"2024-04-25T11:20:44.422680+0000","last_undegraded":"2024-04-25T11:20:44.422680+0000","last_fullsized":"2024-04-25T11:20:44.422680+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:15.845113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.418816+0000","last_change":"2024-04-25T11:20:43.411005+0000","last_active":"2024-04-25T11:20:43.418816+0000","last_peered":"2024-04-25T11:20:43.418816+0000","last_clean":"2024-04-25T11:20:43.418816+0000","last_became_active":"2024-04-25T11:20:43.410555+0000","last_became_peered":"2024-04-25T11:20:43.410555+0000","last_unstale":"2024-04-25T11:20:43.418816+0000","last_undegraded":"2024-04-25T11:20:43.418816+0000","last_fullsized":"2024-04-25T11:20:43.418816+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:43.394881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597213+0000","last_change":"2024-04-25T11:20:42.368323+0000","last_active":"2024-04-25T11:20:44.597213+0000","last_peered":"2024-04-25T11:20:44.597213+0000","last_clean":"2024-04-25T11:20:44.597213+0000","last_became_active":"2024-04-25T11:20:42.367758+0000","last_became_peered":"2024-04-25T11:20:42.367758+0000","last_unstale":"2024-04-25T11:20:44.597213+0000","last_undegraded":"2024-04-25T11:20:44.597213+0000","last_fullsized":"2024-04-25T11:20:44.597213+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:47.040419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.418478+0000","last_change":"2024-04-25T11:20:43.412691+0000","last_active":"2024-04-25T11:20:44.418478+0000","last_peered":"2024-04-25T11:20:44.418478+0000","last_clean":"2024-04-25T11:20:44.418478+0000","last_became_active":"2024-04-25T11:20:43.406640+0000","last_became_peered":"2024-04-25T11:20:43.406640+0000","last_unstale":"2024-04-25T11:20:44.418478+0000","last_undegraded":"2024-04-25T11:20:44.418478+0000","last_fullsized":"2024-04-25T11:20:44.418478+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:33.797487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423545+0000","last_change":"2024-04-25T11:20:42.371896+0000","last_active":"2024-04-25T11:20:44.423545+0000","last_peered":"2024-04-25T11:20:44.423545+0000","last_clean":"2024-04-25T11:20:44.423545+0000","last_became_active":"2024-04-25T11:20:42.370292+0000","last_became_peered":"2024-04-25T11:20:42.370292+0000","last_unstale":"2024-04-25T11:20:44.423545+0000","last_undegraded":"2024-04-25T11:20:44.423545+0000","last_fullsized":"2024-04-25T11:20:44.423545+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:24.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.422558+0000","last_change":"2024-04-25T11:20:43.418620+0000","last_active":"2024-04-25T11:20:43.422558+0000","last_peered":"2024-04-25T11:20:43.422558+0000","last_clean":"2024-04-25T11:20:43.422558+0000","last_became_active":"2024-04-25T11:20:43.415444+0000","last_became_peered":"2024-04-25T11:20:43.415444+0000","last_unstale":"2024-04-25T11:20:43.422558+0000","last_undegraded":"2024-04-25T11:20:43.422558+0000","last_fullsized":"2024-04-25T11:20:43.422558+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:27.669039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423024+0000","last_change":"2024-04-25T11:20:42.376264+0000","last_active":"2024-04-25T11:20:44.423024+0000","last_peered":"2024-04-25T11:20:44.423024+0000","last_clean":"2024-04-25T11:20:44.423024+0000","last_became_active":"2024-04-25T11:20:42.375524+0000","last_became_peered":"2024-04-25T11:20:42.375524+0000","last_unstale":"2024-04-25T11:20:44.423024+0000","last_undegraded":"2024-04-25T11:20:44.423024+0000","last_fullsized":"2024-04-25T11:20:44.423024+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:37.432323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598839+0000","last_change":"2024-04-25T11:20:43.426118+0000","last_active":"2024-04-25T11:20:44.598839+0000","last_peered":"2024-04-25T11:20:44.598839+0000","last_clean":"2024-04-25T11:20:44.598839+0000","last_became_active":"2024-04-25T11:20:43.425388+0000","last_became_peered":"2024-04-25T11:20:43.425388+0000","last_unstale":"2024-04-25T11:20:44.598839+0000","last_undegraded":"2024-04-25T11:20:44.598839+0000","last_fullsized":"2024-04-25T11:20:44.598839+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:02.808346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598125+0000","last_change":"2024-04-25T11:20:42.377296+0000","last_active":"2024-04-25T11:20:44.598125+0000","last_peered":"2024-04-25T11:20:44.598125+0000","last_clean":"2024-04-25T11:20:44.598125+0000","last_became_active":"2024-04-25T11:20:42.376970+0000","last_became_peered":"2024-04-25T11:20:42.376970+0000","last_unstale":"2024-04-25T11:20:44.598125+0000","last_undegraded":"2024-04-25T11:20:44.598125+0000","last_fullsized":"2024-04-25T11:20:44.598125+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:35.541447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598855+0000","last_change":"2024-04-25T11:20:43.414083+0000","last_active":"2024-04-25T11:20:44.598855+0000","last_peered":"2024-04-25T11:20:44.598855+0000","last_clean":"2024-04-25T11:20:44.598855+0000","last_became_active":"2024-04-25T11:20:43.411532+0000","last_became_peered":"2024-04-25T11:20:43.411532+0000","last_unstale":"2024-04-25T11:20:44.598855+0000","last_undegraded":"2024-04-25T11:20:44.598855+0000","last_fullsized":"2024-04-25T11:20:44.598855+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:46.758800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.596451+0000","last_change":"2024-04-25T11:20:42.373623+0000","last_active":"2024-04-25T11:20:44.596451+0000","last_peered":"2024-04-25T11:20:44.596451+0000","last_clean":"2024-04-25T11:20:44.596451+0000","last_became_active":"2024-04-25T11:20:42.370550+0000","last_became_peered":"2024-04-25T11:20:42.370550+0000","last_unstale":"2024-04-25T11:20:44.596451+0000","last_undegraded":"2024-04-25T11:20:44.596451+0000","last_fullsized":"2024-04-25T11:20:44.596451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:24.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597044+0000","last_change":"2024-04-25T11:20:43.401381+0000","last_active":"2024-04-25T11:20:44.597044+0000","last_peered":"2024-04-25T11:20:44.597044+0000","last_clean":"2024-04-25T11:20:44.597044+0000","last_became_active":"2024-04-25T11:20:43.399646+0000","last_became_peered":"2024-04-25T11:20:43.399646+0000","last_unstale":"2024-04-25T11:20:44.597044+0000","last_undegraded":"2024-04-25T11:20:44.597044+0000","last_fullsized":"2024-04-25T11:20:44.597044+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:37.561947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599160+0000","last_change":"2024-04-25T11:20:42.369518+0000","last_active":"2024-04-25T11:20:44.599160+0000","last_peered":"2024-04-25T11:20:44.599160+0000","last_clean":"2024-04-25T11:20:44.599160+0000","last_became_active":"2024-04-25T11:20:42.369100+0000","last_became_peered":"2024-04-25T11:20:42.369100+0000","last_unstale":"2024-04-25T11:20:44.599160+0000","last_undegraded":"2024-04-25T11:20:44.599160+0000","last_fullsized":"2024-04-25T11:20:44.599160+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:34.115268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423469+0000","last_change":"2024-04-25T11:20:43.412931+0000","last_active":"2024-04-25T11:20:44.423469+0000","last_peered":"2024-04-25T11:20:44.423469+0000","last_clean":"2024-04-25T11:20:44.423469+0000","last_became_active":"2024-04-25T11:20:43.412578+0000","last_became_peered":"2024-04-25T11:20:43.412578+0000","last_unstale":"2024-04-25T11:20:44.423469+0000","last_undegraded":"2024-04-25T11:20:44.423469+0000","last_fullsized":"2024-04-25T11:20:44.423469+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:44.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.420432+0000","last_change":"2024-04-25T11:20:42.376354+0000","last_active":"2024-04-25T11:20:44.420432+0000","last_peered":"2024-04-25T11:20:44.420432+0000","last_clean":"2024-04-25T11:20:44.420432+0000","last_became_active":"2024-04-25T11:20:42.376012+0000","last_became_peered":"2024-04-25T11:20:42.376012+0000","last_unstale":"2024-04-25T11:20:44.420432+0000","last_undegraded":"2024-04-25T11:20:44.420432+0000","last_fullsized":"2024-04-25T11:20:44.420432+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:47.365940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.421975+0000","last_change":"2024-04-25T11:20:43.415496+0000","last_active":"2024-04-25T11:20:43.421975+0000","last_peered":"2024-04-25T11:20:43.421975+0000","last_clean":"2024-04-25T11:20:43.421975+0000","last_became_active":"2024-04-25T11:20:43.415147+0000","last_became_peered":"2024-04-25T11:20:43.415147+0000","last_unstale":"2024-04-25T11:20:43.421975+0000","last_undegraded":"2024-04-25T11:20:43.421975+0000","last_fullsized":"2024-04-25T11:20:43.421975+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:11.851404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419432+0000","last_change":"2024-04-25T11:20:42.374689+0000","last_active":"2024-04-25T11:20:44.419432+0000","last_peered":"2024-04-25T11:20:44.419432+0000","last_clean":"2024-04-25T11:20:44.419432+0000","last_became_active":"2024-04-25T11:20:42.374235+0000","last_became_peered":"2024-04-25T11:20:42.374235+0000","last_unstale":"2024-04-25T11:20:44.419432+0000","last_undegraded":"2024-04-25T11:20:44.419432+0000","last_fullsized":"2024-04-25T11:20:44.419432+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:27.727738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599390+0000","last_change":"2024-04-25T11:20:43.410603+0000","last_active":"2024-04-25T11:20:44.599390+0000","last_peered":"2024-04-25T11:20:44.599390+0000","last_clean":"2024-04-25T11:20:44.599390+0000","last_became_active":"2024-04-25T11:20:43.410266+0000","last_became_peered":"2024-04-25T11:20:43.410266+0000","last_unstale":"2024-04-25T11:20:44.599390+0000","last_undegraded":"2024-04-25T11:20:44.599390+0000","last_fullsized":"2024-04-25T11:20:44.599390+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:19.371050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.403440+0000","last_change":"2024-04-25T11:20:42.380279+0000","last_active":"2024-04-25T11:20:43.403440+0000","last_peered":"2024-04-25T11:20:43.403440+0000","last_clean":"2024-04-25T11:20:43.403440+0000","last_became_active":"2024-04-25T11:20:42.379390+0000","last_became_peered":"2024-04-25T11:20:42.379390+0000","last_unstale":"2024-04-25T11:20:43.403440+0000","last_undegraded":"2024-04-25T11:20:43.403440+0000","last_fullsized":"2024-04-25T11:20:43.403440+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:28.286300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422041+0000","last_change":"2024-04-25T11:20:43.424857+0000","last_active":"2024-04-25T11:20:44.422041+0000","last_peered":"2024-04-25T11:20:44.422041+0000","last_clean":"2024-04-25T11:20:44.422041+0000","last_became_active":"2024-04-25T11:20:43.422860+0000","last_became_peered":"2024-04-25T11:20:43.422860+0000","last_unstale":"2024-04-25T11:20:44.422041+0000","last_undegraded":"2024-04-25T11:20:44.422041+0000","last_fullsized":"2024-04-25T11:20:44.422041+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:33.797487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"129'3","reported_seq":17,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423722+0000","last_change":"2024-04-25T11:20:42.372299+0000","last_active":"2024-04-25T11:20:44.423722+0000","last_peered":"2024-04-25T11:20:44.423722+0000","last_clean":"2024-04-25T11:20:44.423722+0000","last_became_active":"2024-04-25T11:20:42.371945+0000","last_became_peered":"2024-04-25T11:20:42.371945+0000","last_unstale":"2024-04-25T11:20:44.423722+0000","last_undegraded":"2024-04-25T11:20:44.423722+0000","last_fullsized":"2024-04-25T11:20:44.423722+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:47.040419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422504+0000","last_change":"2024-04-25T11:20:43.423703+0000","last_active":"2024-04-25T11:20:44.422504+0000","last_peered":"2024-04-25T11:20:44.422504+0000","last_clean":"2024-04-25T11:20:44.422504+0000","last_became_active":"2024-04-25T11:20:43.423369+0000","last_became_peered":"2024-04-25T11:20:43.423369+0000","last_unstale":"2024-04-25T11:20:44.422504+0000","last_undegraded":"2024-04-25T11:20:44.422504+0000","last_fullsized":"2024-04-25T11:20:44.422504+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:42.730597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"129'6","reported_seq":20,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423159+0000","last_change":"2024-04-25T11:20:42.376532+0000","last_active":"2024-04-25T11:20:44.423159+0000","last_peered":"2024-04-25T11:20:44.423159+0000","last_clean":"2024-04-25T11:20:44.423159+0000","last_became_active":"2024-04-25T11:20:42.375922+0000","last_became_peered":"2024-04-25T11:20:42.375922+0000","last_unstale":"2024-04-25T11:20:44.423159+0000","last_undegraded":"2024-04-25T11:20:44.423159+0000","last_fullsized":"2024-04-25T11:20:44.423159+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:50.657139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564682+0000","last_change":"2024-04-25T11:20:43.411891+0000","last_active":"2024-04-25T11:20:44.564682+0000","last_peered":"2024-04-25T11:20:44.564682+0000","last_clean":"2024-04-25T11:20:44.564682+0000","last_became_active":"2024-04-25T11:20:43.411541+0000","last_became_peered":"2024-04-25T11:20:43.411541+0000","last_unstale":"2024-04-25T11:20:44.564682+0000","last_undegraded":"2024-04-25T11:20:44.564682+0000","last_fullsized":"2024-04-25T11:20:44.564682+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:05.281320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.431962+0000","last_change":"2024-04-25T11:20:42.381155+0000","last_active":"2024-04-25T11:20:43.431962+0000","last_peered":"2024-04-25T11:20:43.431962+0000","last_clean":"2024-04-25T11:20:43.431962+0000","last_became_active":"2024-04-25T11:20:42.380803+0000","last_became_peered":"2024-04-25T11:20:42.380803+0000","last_unstale":"2024-04-25T11:20:43.431962+0000","last_undegraded":"2024-04-25T11:20:43.431962+0000","last_fullsized":"2024-04-25T11:20:43.431962+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:17.228130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423624+0000","last_change":"2024-04-25T11:20:43.417277+0000","last_active":"2024-04-25T11:20:44.423624+0000","last_peered":"2024-04-25T11:20:44.423624+0000","last_clean":"2024-04-25T11:20:44.423624+0000","last_became_active":"2024-04-25T11:20:43.416914+0000","last_became_peered":"2024-04-25T11:20:43.416914+0000","last_unstale":"2024-04-25T11:20:44.423624+0000","last_undegraded":"2024-04-25T11:20:44.423624+0000","last_fullsized":"2024-04-25T11:20:44.423624+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:37.741543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.420668+0000","last_change":"2024-04-25T11:20:42.367031+0000","last_active":"2024-04-25T11:20:44.420668+0000","last_peered":"2024-04-25T11:20:44.420668+0000","last_clean":"2024-04-25T11:20:44.420668+0000","last_became_active":"2024-04-25T11:20:42.366576+0000","last_became_peered":"2024-04-25T11:20:42.366576+0000","last_unstale":"2024-04-25T11:20:44.420668+0000","last_undegraded":"2024-04-25T11:20:44.420668+0000","last_fullsized":"2024-04-25T11:20:44.420668+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:29.172494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597082+0000","last_change":"2024-04-25T11:20:43.405902+0000","last_active":"2024-04-25T11:20:44.597082+0000","last_peered":"2024-04-25T11:20:44.597082+0000","last_clean":"2024-04-25T11:20:44.597082+0000","last_became_active":"2024-04-25T11:20:43.405559+0000","last_became_peered":"2024-04-25T11:20:43.405559+0000","last_unstale":"2024-04-25T11:20:44.597082+0000","last_undegraded":"2024-04-25T11:20:44.597082+0000","last_fullsized":"2024-04-25T11:20:44.597082+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:46.271892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"129'8","reported_seq":22,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597869+0000","last_change":"2024-04-25T11:20:42.366255+0000","last_active":"2024-04-25T11:20:44.597869+0000","last_peered":"2024-04-25T11:20:44.597869+0000","last_clean":"2024-04-25T11:20:44.597869+0000","last_became_active":"2024-04-25T11:20:42.364737+0000","last_became_peered":"2024-04-25T11:20:42.364737+0000","last_unstale":"2024-04-25T11:20:44.597869+0000","last_undegraded":"2024-04-25T11:20:44.597869+0000","last_fullsized":"2024-04-25T11:20:44.597869+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:12.016407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.420321+0000","last_change":"2024-04-25T11:20:43.432463+0000","last_active":"2024-04-25T11:20:44.420321+0000","last_peered":"2024-04-25T11:20:44.420321+0000","last_clean":"2024-04-25T11:20:44.420321+0000","last_became_active":"2024-04-25T11:20:43.428513+0000","last_became_peered":"2024-04-25T11:20:43.428513+0000","last_unstale":"2024-04-25T11:20:44.420321+0000","last_undegraded":"2024-04-25T11:20:44.420321+0000","last_fullsized":"2024-04-25T11:20:44.420321+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:40.892114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419527+0000","last_change":"2024-04-25T11:20:42.377113+0000","last_active":"2024-04-25T11:20:44.419527+0000","last_peered":"2024-04-25T11:20:44.419527+0000","last_clean":"2024-04-25T11:20:44.419527+0000","last_became_active":"2024-04-25T11:20:42.376770+0000","last_became_peered":"2024-04-25T11:20:42.376770+0000","last_unstale":"2024-04-25T11:20:44.419527+0000","last_undegraded":"2024-04-25T11:20:44.419527+0000","last_fullsized":"2024-04-25T11:20:44.419527+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:56.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421942+0000","last_change":"2024-04-25T11:20:43.404474+0000","last_active":"2024-04-25T11:20:44.421942+0000","last_peered":"2024-04-25T11:20:44.421942+0000","last_clean":"2024-04-25T11:20:44.421942+0000","last_became_active":"2024-04-25T11:20:43.404094+0000","last_became_peered":"2024-04-25T11:20:43.404094+0000","last_unstale":"2024-04-25T11:20:44.421942+0000","last_undegraded":"2024-04-25T11:20:44.421942+0000","last_fullsized":"2024-04-25T11:20:44.421942+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:39.941085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422831+0000","last_change":"2024-04-25T11:20:42.384484+0000","last_active":"2024-04-25T11:20:44.422831+0000","last_peered":"2024-04-25T11:20:44.422831+0000","last_clean":"2024-04-25T11:20:44.422831+0000","last_became_active":"2024-04-25T11:20:42.383490+0000","last_became_peered":"2024-04-25T11:20:42.383490+0000","last_unstale":"2024-04-25T11:20:44.422831+0000","last_undegraded":"2024-04-25T11:20:44.422831+0000","last_fullsized":"2024-04-25T11:20:44.422831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:50.930451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422256+0000","last_change":"2024-04-25T11:20:43.412744+0000","last_active":"2024-04-25T11:20:44.422256+0000","last_peered":"2024-04-25T11:20:44.422256+0000","last_clean":"2024-04-25T11:20:44.422256+0000","last_became_active":"2024-04-25T11:20:43.408707+0000","last_became_peered":"2024-04-25T11:20:43.408707+0000","last_unstale":"2024-04-25T11:20:44.422256+0000","last_undegraded":"2024-04-25T11:20:44.422256+0000","last_fullsized":"2024-04-25T11:20:44.422256+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:37.561947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.600279+0000","last_change":"2024-04-25T11:20:42.371276+0000","last_active":"2024-04-25T11:20:44.600279+0000","last_peered":"2024-04-25T11:20:44.600279+0000","last_clean":"2024-04-25T11:20:44.600279+0000","last_became_active":"2024-04-25T11:20:42.370892+0000","last_became_peered":"2024-04-25T11:20:42.370892+0000","last_unstale":"2024-04-25T11:20:44.600279+0000","last_undegraded":"2024-04-25T11:20:44.600279+0000","last_fullsized":"2024-04-25T11:20:44.600279+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:42.294115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564241+0000","last_change":"2024-04-25T11:20:43.425979+0000","last_active":"2024-04-25T11:20:44.564241+0000","last_peered":"2024-04-25T11:20:44.564241+0000","last_clean":"2024-04-25T11:20:44.564241+0000","last_became_active":"2024-04-25T11:20:43.424946+0000","last_became_peered":"2024-04-25T11:20:43.424946+0000","last_unstale":"2024-04-25T11:20:44.564241+0000","last_undegraded":"2024-04-25T11:20:44.564241+0000","last_fullsized":"2024-04-25T11:20:44.564241+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:43.445693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597354+0000","last_change":"2024-04-25T11:20:42.372055+0000","last_active":"2024-04-25T11:20:44.597354+0000","last_peered":"2024-04-25T11:20:44.597354+0000","last_clean":"2024-04-25T11:20:44.597354+0000","last_became_active":"2024-04-25T11:20:42.371748+0000","last_became_peered":"2024-04-25T11:20:42.371748+0000","last_unstale":"2024-04-25T11:20:44.597354+0000","last_undegraded":"2024-04-25T11:20:44.597354+0000","last_fullsized":"2024-04-25T11:20:44.597354+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:38.507238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597217+0000","last_change":"2024-04-25T11:20:43.408905+0000","last_active":"2024-04-25T11:20:44.597217+0000","last_peered":"2024-04-25T11:20:44.597217+0000","last_clean":"2024-04-25T11:20:44.597217+0000","last_became_active":"2024-04-25T11:20:43.408505+0000","last_became_peered":"2024-04-25T11:20:43.408505+0000","last_unstale":"2024-04-25T11:20:44.597217+0000","last_undegraded":"2024-04-25T11:20:44.597217+0000","last_fullsized":"2024-04-25T11:20:44.597217+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:44.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598484+0000","last_change":"2024-04-25T11:20:42.379960+0000","last_active":"2024-04-25T11:20:44.598484+0000","last_peered":"2024-04-25T11:20:44.598484+0000","last_clean":"2024-04-25T11:20:44.598484+0000","last_became_active":"2024-04-25T11:20:42.379582+0000","last_became_peered":"2024-04-25T11:20:42.379582+0000","last_unstale":"2024-04-25T11:20:44.598484+0000","last_undegraded":"2024-04-25T11:20:44.598484+0000","last_fullsized":"2024-04-25T11:20:44.598484+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:47.276923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.428685+0000","last_change":"2024-04-25T11:20:43.426267+0000","last_active":"2024-04-25T11:20:43.428685+0000","last_peered":"2024-04-25T11:20:43.428685+0000","last_clean":"2024-04-25T11:20:43.428685+0000","last_became_active":"2024-04-25T11:20:43.425937+0000","last_became_peered":"2024-04-25T11:20:43.425937+0000","last_unstale":"2024-04-25T11:20:43.428685+0000","last_undegraded":"2024-04-25T11:20:43.428685+0000","last_fullsized":"2024-04-25T11:20:43.428685+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:13.983664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.401277+0000","last_change":"2024-04-25T11:20:42.382049+0000","last_active":"2024-04-25T11:20:43.401277+0000","last_peered":"2024-04-25T11:20:43.401277+0000","last_clean":"2024-04-25T11:20:43.401277+0000","last_became_active":"2024-04-25T11:20:42.381621+0000","last_became_peered":"2024-04-25T11:20:42.381621+0000","last_unstale":"2024-04-25T11:20:43.401277+0000","last_undegraded":"2024-04-25T11:20:43.401277+0000","last_fullsized":"2024-04-25T11:20:43.401277+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:39.016017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598363+0000","last_change":"2024-04-25T11:20:43.413150+0000","last_active":"2024-04-25T11:20:44.598363+0000","last_peered":"2024-04-25T11:20:44.598363+0000","last_clean":"2024-04-25T11:20:44.598363+0000","last_became_active":"2024-04-25T11:20:43.412595+0000","last_became_peered":"2024-04-25T11:20:43.412595+0000","last_unstale":"2024-04-25T11:20:44.598363+0000","last_undegraded":"2024-04-25T11:20:44.598363+0000","last_fullsized":"2024-04-25T11:20:44.598363+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:58.042477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421970+0000","last_change":"2024-04-25T11:20:42.379509+0000","last_active":"2024-04-25T11:20:44.421970+0000","last_peered":"2024-04-25T11:20:44.421970+0000","last_clean":"2024-04-25T11:20:44.421970+0000","last_became_active":"2024-04-25T11:20:42.378547+0000","last_became_peered":"2024-04-25T11:20:42.378547+0000","last_unstale":"2024-04-25T11:20:44.421970+0000","last_undegraded":"2024-04-25T11:20:44.421970+0000","last_fullsized":"2024-04-25T11:20:44.421970+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:14.996438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419269+0000","last_change":"2024-04-25T11:20:43.420252+0000","last_active":"2024-04-25T11:20:44.419269+0000","last_peered":"2024-04-25T11:20:44.419269+0000","last_clean":"2024-04-25T11:20:44.419269+0000","last_became_active":"2024-04-25T11:20:43.419911+0000","last_became_peered":"2024-04-25T11:20:43.419911+0000","last_unstale":"2024-04-25T11:20:44.419269+0000","last_undegraded":"2024-04-25T11:20:44.419269+0000","last_fullsized":"2024-04-25T11:20:44.419269+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:39.553093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598038+0000","last_change":"2024-04-25T11:20:42.360966+0000","last_active":"2024-04-25T11:20:44.598038+0000","last_peered":"2024-04-25T11:20:44.598038+0000","last_clean":"2024-04-25T11:20:44.598038+0000","last_became_active":"2024-04-25T11:20:42.360596+0000","last_became_peered":"2024-04-25T11:20:42.360596+0000","last_unstale":"2024-04-25T11:20:44.598038+0000","last_undegraded":"2024-04-25T11:20:44.598038+0000","last_fullsized":"2024-04-25T11:20:44.598038+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:56.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.596694+0000","last_change":"2024-04-25T11:20:43.401914+0000","last_active":"2024-04-25T11:20:44.596694+0000","last_peered":"2024-04-25T11:20:44.596694+0000","last_clean":"2024-04-25T11:20:44.596694+0000","last_became_active":"2024-04-25T11:20:43.398788+0000","last_became_peered":"2024-04-25T11:20:43.398788+0000","last_unstale":"2024-04-25T11:20:44.596694+0000","last_undegraded":"2024-04-25T11:20:44.596694+0000","last_fullsized":"2024-04-25T11:20:44.596694+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:47.707614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.436474+0000","last_change":"2024-04-25T11:20:42.377964+0000","last_active":"2024-04-25T11:20:43.436474+0000","last_peered":"2024-04-25T11:20:43.436474+0000","last_clean":"2024-04-25T11:20:43.436474+0000","last_became_active":"2024-04-25T11:20:42.377418+0000","last_became_peered":"2024-04-25T11:20:42.377418+0000","last_unstale":"2024-04-25T11:20:43.436474+0000","last_undegraded":"2024-04-25T11:20:43.436474+0000","last_fullsized":"2024-04-25T11:20:43.436474+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:25.827894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421052+0000","last_change":"2024-04-25T11:20:43.414949+0000","last_active":"2024-04-25T11:20:44.421052+0000","last_peered":"2024-04-25T11:20:44.421052+0000","last_clean":"2024-04-25T11:20:44.421052+0000","last_became_active":"2024-04-25T11:20:43.414616+0000","last_became_peered":"2024-04-25T11:20:43.414616+0000","last_unstale":"2024-04-25T11:20:44.421052+0000","last_undegraded":"2024-04-25T11:20:44.421052+0000","last_fullsized":"2024-04-25T11:20:44.421052+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:46.758800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598827+0000","last_change":"2024-04-25T11:20:42.370410+0000","last_active":"2024-04-25T11:20:44.598827+0000","last_peered":"2024-04-25T11:20:44.598827+0000","last_clean":"2024-04-25T11:20:44.598827+0000","last_became_active":"2024-04-25T11:20:42.370102+0000","last_became_peered":"2024-04-25T11:20:42.370102+0000","last_unstale":"2024-04-25T11:20:44.598827+0000","last_undegraded":"2024-04-25T11:20:44.598827+0000","last_fullsized":"2024-04-25T11:20:44.598827+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:12.016407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421732+0000","last_change":"2024-04-25T11:20:43.415984+0000","last_active":"2024-04-25T11:20:44.421732+0000","last_peered":"2024-04-25T11:20:44.421732+0000","last_clean":"2024-04-25T11:20:44.421732+0000","last_became_active":"2024-04-25T11:20:43.415640+0000","last_became_peered":"2024-04-25T11:20:43.415640+0000","last_unstale":"2024-04-25T11:20:44.421732+0000","last_undegraded":"2024-04-25T11:20:44.421732+0000","last_fullsized":"2024-04-25T11:20:44.421732+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:47.707614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598429+0000","last_change":"2024-04-25T11:20:42.383705+0000","last_active":"2024-04-25T11:20:44.598429+0000","last_peered":"2024-04-25T11:20:44.598429+0000","last_clean":"2024-04-25T11:20:44.598429+0000","last_became_active":"2024-04-25T11:20:42.383361+0000","last_became_peered":"2024-04-25T11:20:42.383361+0000","last_unstale":"2024-04-25T11:20:44.598429+0000","last_undegraded":"2024-04-25T11:20:44.598429+0000","last_fullsized":"2024-04-25T11:20:44.598429+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:27.727738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564311+0000","last_change":"2024-04-25T11:20:43.416088+0000","last_active":"2024-04-25T11:20:44.564311+0000","last_peered":"2024-04-25T11:20:44.564311+0000","last_clean":"2024-04-25T11:20:44.564311+0000","last_became_active":"2024-04-25T11:20:43.415757+0000","last_became_peered":"2024-04-25T11:20:43.415757+0000","last_unstale":"2024-04-25T11:20:44.564311+0000","last_undegraded":"2024-04-25T11:20:44.564311+0000","last_fullsized":"2024-04-25T11:20:44.564311+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:11.139740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599849+0000","last_change":"2024-04-25T11:20:42.371221+0000","last_active":"2024-04-25T11:20:44.599849+0000","last_peered":"2024-04-25T11:20:44.599849+0000","last_clean":"2024-04-25T11:20:44.599849+0000","last_became_active":"2024-04-25T11:20:42.370464+0000","last_became_peered":"2024-04-25T11:20:42.370464+0000","last_unstale":"2024-04-25T11:20:44.599849+0000","last_undegraded":"2024-04-25T11:20:44.599849+0000","last_fullsized":"2024-04-25T11:20:44.599849+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:41.715534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.600524+0000","last_change":"2024-04-25T11:20:43.410926+0000","last_active":"2024-04-25T11:20:44.600524+0000","last_peered":"2024-04-25T11:20:44.600524+0000","last_clean":"2024-04-25T11:20:44.600524+0000","last_became_active":"2024-04-25T11:20:43.410605+0000","last_became_peered":"2024-04-25T11:20:43.410605+0000","last_unstale":"2024-04-25T11:20:44.600524+0000","last_undegraded":"2024-04-25T11:20:44.600524+0000","last_fullsized":"2024-04-25T11:20:44.600524+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.045860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419155+0000","last_change":"2024-04-25T11:20:42.384326+0000","last_active":"2024-04-25T11:20:44.419155+0000","last_peered":"2024-04-25T11:20:44.419155+0000","last_clean":"2024-04-25T11:20:44.419155+0000","last_became_active":"2024-04-25T11:20:42.383518+0000","last_became_peered":"2024-04-25T11:20:42.383518+0000","last_unstale":"2024-04-25T11:20:44.419155+0000","last_undegraded":"2024-04-25T11:20:44.419155+0000","last_fullsized":"2024-04-25T11:20:44.419155+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:12.016407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.600248+0000","last_change":"2024-04-25T11:20:43.415403+0000","last_active":"2024-04-25T11:20:44.600248+0000","last_peered":"2024-04-25T11:20:44.600248+0000","last_clean":"2024-04-25T11:20:44.600248+0000","last_became_active":"2024-04-25T11:20:43.412411+0000","last_became_peered":"2024-04-25T11:20:43.412411+0000","last_unstale":"2024-04-25T11:20:44.600248+0000","last_undegraded":"2024-04-25T11:20:44.600248+0000","last_fullsized":"2024-04-25T11:20:44.600248+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:25.414615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422193+0000","last_change":"2024-04-25T11:20:42.371478+0000","last_active":"2024-04-25T11:20:44.422193+0000","last_peered":"2024-04-25T11:20:44.422193+0000","last_clean":"2024-04-25T11:20:44.422193+0000","last_became_active":"2024-04-25T11:20:42.371144+0000","last_became_peered":"2024-04-25T11:20:42.371144+0000","last_unstale":"2024-04-25T11:20:44.422193+0000","last_undegraded":"2024-04-25T11:20:44.422193+0000","last_fullsized":"2024-04-25T11:20:44.422193+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.365609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598879+0000","last_change":"2024-04-25T11:20:43.415030+0000","last_active":"2024-04-25T11:20:44.598879+0000","last_peered":"2024-04-25T11:20:44.598879+0000","last_clean":"2024-04-25T11:20:44.598879+0000","last_became_active":"2024-04-25T11:20:43.414697+0000","last_became_peered":"2024-04-25T11:20:43.414697+0000","last_unstale":"2024-04-25T11:20:44.598879+0000","last_undegraded":"2024-04-25T11:20:44.598879+0000","last_fullsized":"2024-04-25T11:20:44.598879+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:55.138633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599551+0000","last_change":"2024-04-25T11:20:42.359739+0000","last_active":"2024-04-25T11:20:44.599551+0000","last_peered":"2024-04-25T11:20:44.599551+0000","last_clean":"2024-04-25T11:20:44.599551+0000","last_became_active":"2024-04-25T11:20:42.357458+0000","last_became_peered":"2024-04-25T11:20:42.357458+0000","last_unstale":"2024-04-25T11:20:44.599551+0000","last_undegraded":"2024-04-25T11:20:44.599551+0000","last_fullsized":"2024-04-25T11:20:44.599551+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:04.832828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423469+0000","last_change":"2024-04-25T11:20:43.422095+0000","last_active":"2024-04-25T11:20:44.423469+0000","last_peered":"2024-04-25T11:20:44.423469+0000","last_clean":"2024-04-25T11:20:44.423469+0000","last_became_active":"2024-04-25T11:20:43.421758+0000","last_became_peered":"2024-04-25T11:20:43.421758+0000","last_unstale":"2024-04-25T11:20:44.423469+0000","last_undegraded":"2024-04-25T11:20:44.423469+0000","last_fullsized":"2024-04-25T11:20:44.423469+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:58.042477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.596853+0000","last_change":"2024-04-25T11:20:42.375043+0000","last_active":"2024-04-25T11:20:44.596853+0000","last_peered":"2024-04-25T11:20:44.596853+0000","last_clean":"2024-04-25T11:20:44.596853+0000","last_became_active":"2024-04-25T11:20:42.374678+0000","last_became_peered":"2024-04-25T11:20:42.374678+0000","last_unstale":"2024-04-25T11:20:44.596853+0000","last_undegraded":"2024-04-25T11:20:44.596853+0000","last_fullsized":"2024-04-25T11:20:44.596853+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:50.657139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422560+0000","last_change":"2024-04-25T11:20:43.429330+0000","last_active":"2024-04-25T11:20:44.422560+0000","last_peered":"2024-04-25T11:20:44.422560+0000","last_clean":"2024-04-25T11:20:44.422560+0000","last_became_active":"2024-04-25T11:20:43.426542+0000","last_became_peered":"2024-04-25T11:20:43.426542+0000","last_unstale":"2024-04-25T11:20:44.422560+0000","last_undegraded":"2024-04-25T11:20:44.422560+0000","last_fullsized":"2024-04-25T11:20:44.422560+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:39.553093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422031+0000","last_change":"2024-04-25T11:20:42.374129+0000","last_active":"2024-04-25T11:20:44.422031+0000","last_peered":"2024-04-25T11:20:44.422031+0000","last_clean":"2024-04-25T11:20:44.422031+0000","last_became_active":"2024-04-25T11:20:42.373753+0000","last_became_peered":"2024-04-25T11:20:42.373753+0000","last_unstale":"2024-04-25T11:20:44.422031+0000","last_undegraded":"2024-04-25T11:20:44.422031+0000","last_fullsized":"2024-04-25T11:20:44.422031+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:54.133192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":329,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421345+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:20:44.421345+0000","last_peered":"2024-04-25T11:20:44.421345+0000","last_clean":"2024-04-25T11:20:44.421345+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:20:44.421345+0000","last_undegraded":"2024-04-25T11:20:44.421345+0000","last_fullsized":"2024-04-25T11:20:44.421345+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33352,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338488,"statfs":{"total":96636764160,"available":96602611712,"internally_reserved":0,"allocated":2224128,"data_stored":1337841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36924,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334916,"statfs":{"total":96636764160,"available":96598953984,"internally_reserved":0,"allocated":1622016,"data_stored":746443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1617920,"data_stored":745793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476866,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36916,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334924,"statfs":{"total":96636764160,"available":96598962176,"internally_reserved":0,"allocated":1613824,"data_stored":745771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509569,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334904,"statfs":{"total":96636764160,"available":96598941696,"internally_reserved":0,"allocated":1630208,"data_stored":747453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509570,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":739149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509569,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37496,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334344,"statfs":{"total":96636764160,"available":96598368256,"internally_reserved":0,"allocated":2203648,"data_stored":1329503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542274,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1609728,"data_stored":739695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:20:48.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:20:48.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.454+0000 7f5f3e400640 1 Processor -- start 2024-04-25T11:20:48.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.455+0000 7f5f3e400640 1 -- start start 2024-04-25T11:20:48.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.457+0000 7f5f3e400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380d0de0 0x7f5f380b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:48.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.457+0000 7f5f3e400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5f380ce7e0 0x7f5f380cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:48.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.457+0000 7f5f3e400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f3809cc20 con 0x7f5f380d2a90 2024-04-25T11:20:48.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.457+0000 7f5f3e400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5f3806eba0 con 0x7f5f380ce7e0 2024-04-25T11:20:48.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.457+0000 7f5f3e400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3809cf00 con 0x7f5f380d0de0 2024-04-25T11:20:48.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.458+0000 7f5f3da00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5f380ce7e0 0x7f5f380cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:48.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.458+0000 7f5f3d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380d0de0 0x7f5f380b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:48.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.458+0000 7f5f3da00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5f380ce7e0 0x7f5f380cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:57088/0 (socket says 172.21.15.7:57088) 2024-04-25T11:20:48.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.458+0000 7f5f3da00640 1 -- 172.21.15.7:0/1066267717 learned_addr learned my addr 172.21.15.7:0/1066267717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:48.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.458+0000 7f5f3d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380d0de0 0x7f5f380b89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:34868/0 (socket says 172.21.15.7:34868) 2024-04-25T11:20:48.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.458+0000 7f5f3d000640 1 -- 172.21.15.7:0/1066267717 >> v2:172.21.15.87:3300/0 conn(0x7f5f380ce7e0 msgr2=0x7f5f380cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:48.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.458+0000 7f5f3d000640 1 --2- 172.21.15.7:0/1066267717 >> v2:172.21.15.87:3300/0 conn(0x7f5f380ce7e0 0x7f5f380cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.459+0000 7f5f3d000640 1 -- 172.21.15.7:0/1066267717 >> v1:172.21.15.87:6789/0 conn(0x7f5f380d2a90 legacy=0x7f5f380d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.459+0000 7f5f3d000640 1 -- 172.21.15.7:0/1066267717 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3806e830 con 0x7f5f380d0de0 2024-04-25T11:20:48.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.460+0000 7f5f3d000640 1 --2- 172.21.15.7:0/1066267717 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380d0de0 0x7f5f380b89c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0097a0 tx=0x7f5f2c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64cb62ed3692cb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:48.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.460+0000 7f5f37400640 1 -- 172.21.15.7:0/1066267717 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c037070 con 0x7f5f380d0de0 2024-04-25T11:20:48.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.460+0000 7f5f37400640 1 -- 172.21.15.7:0/1066267717 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2c032440 con 0x7f5f380d0de0 2024-04-25T11:20:48.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.460+0000 7f5f37400640 1 -- 172.21.15.7:0/1066267717 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c032810 con 0x7f5f380d0de0 2024-04-25T11:20:48.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.461+0000 7f5f3e400640 1 -- 172.21.15.7:0/1066267717 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380d0de0 msgr2=0x7f5f380b89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:48.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.461+0000 7f5f3e400640 1 --2- 172.21.15.7:0/1066267717 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380d0de0 0x7f5f380b89c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0097a0 tx=0x7f5f2c02a180 comp rx=0 tx=0).stop 2024-04-25T11:20:48.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.461+0000 7f5f3e400640 1 -- 172.21.15.7:0/1066267717 shutdown_connections 2024-04-25T11:20:48.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.461+0000 7f5f3e400640 1 --2- 172.21.15.7:0/1066267717 >> v2:172.21.15.87:3300/0 conn(0x7f5f380ce7e0 0x7f5f380cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.461+0000 7f5f3e400640 1 --2- 172.21.15.7:0/1066267717 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380d0de0 0x7f5f380b89c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.461+0000 7f5f3e400640 1 -- 172.21.15.7:0/1066267717 >> 172.21.15.7:0/1066267717 conn(0x7f5f380d0250 msgr2=0x7f5f380d0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:48.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.461+0000 7f5f3e400640 1 -- 172.21.15.7:0/1066267717 shutdown_connections 2024-04-25T11:20:48.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.461+0000 7f5f3e400640 1 -- 172.21.15.7:0/1066267717 wait complete. 2024-04-25T11:20:48.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.464+0000 7f5f3e400640 1 Processor -- start 2024-04-25T11:20:48.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.464+0000 7f5f3e400640 1 -- start start 2024-04-25T11:20:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.466+0000 7f5f3e400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380ce7e0 0x7f5f380cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.466+0000 7f5f3e400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f380d0de0 0x7f5f380cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.466+0000 7f5f3e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f380d2a90 0x7f5f381e1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.466+0000 7f5f3e400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5f380d3020 con 0x7f5f380d2a90 2024-04-25T11:20:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.466+0000 7f5f3e400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f380d2ea0 con 0x7f5f380d0de0 2024-04-25T11:20:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.466+0000 7f5f3e400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5f380b9150 con 0x7f5f380ce7e0 2024-04-25T11:20:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.466+0000 7f5f3da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f380d2a90 0x7f5f381e1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.466+0000 7f5f3da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f380d2a90 0x7f5f381e1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37046/0 (socket says 172.21.15.7:37046) 2024-04-25T11:20:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.466+0000 7f5f3da00640 1 -- 172.21.15.7:0/3154931238 learned_addr learned my addr 172.21.15.7:0/3154931238 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:48.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.466+0000 7f5f3d000640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380ce7e0 0x7f5f380cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:48.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.466+0000 7f5f37e00640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f380d0de0 0x7f5f380cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.467+0000 7f5f3da00640 1 -- 172.21.15.7:0/3154931238 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380ce7e0 msgr2=0x7f5f380cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.467+0000 7f5f3da00640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380ce7e0 0x7f5f380cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.467+0000 7f5f3da00640 1 -- 172.21.15.7:0/3154931238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f380d0de0 msgr2=0x7f5f380cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.467+0000 7f5f3da00640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f380d0de0 0x7f5f380cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.467+0000 7f5f3da00640 1 -- 172.21.15.7:0/3154931238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f380cc0d0 con 0x7f5f380d2a90 2024-04-25T11:20:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.467+0000 7f5f3da00640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f380d2a90 0x7f5f381e1f10 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000ec10 tx=0x7f5f3000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.468+0000 7f5f34c00640 1 -- 172.21.15.7:0/3154931238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3000cb20 con 0x7f5f380d2a90 2024-04-25T11:20:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.468+0000 7f5f34c00640 1 -- 172.21.15.7:0/3154931238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f3000ccc0 con 0x7f5f380d2a90 2024-04-25T11:20:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.468+0000 7f5f3e400640 1 -- 172.21.15.7:0/3154931238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f380cc3b0 con 0x7f5f380d2a90 2024-04-25T11:20:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.468+0000 7f5f34c00640 1 -- 172.21.15.7:0/3154931238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f30003bd0 con 0x7f5f380d2a90 2024-04-25T11:20:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.468+0000 7f5f3e400640 1 -- 172.21.15.7:0/3154931238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f380cd9b0 con 0x7f5f380d2a90 2024-04-25T11:20:48.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.469+0000 7f5f3e400640 1 -- 172.21.15.7:0/3154931238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f38050740 con 0x7f5f380d2a90 2024-04-25T11:20:48.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.470+0000 7f5f34c00640 1 -- 172.21.15.7:0/3154931238 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5f30012050 con 0x7f5f380d2a90 2024-04-25T11:20:48.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.471+0000 7f5f34c00640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f180744e0 0x7f5f180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:48.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.471+0000 7f5f34c00640 1 -- 172.21.15.7:0/3154931238 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f5f30095740 con 0x7f5f380d2a90 2024-04-25T11:20:48.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.472+0000 7f5f3d000640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f180744e0 0x7f5f180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:48.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.472+0000 7f5f3d000640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f180744e0 0x7f5f180769f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c034000 tx=0x7f5f2c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:48.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.476+0000 7f5f34c00640 1 -- 172.21.15.7:0/3154931238 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5f30010040 con 0x7f5f380d2a90 2024-04-25T11:20:48.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.674+0000 7f5f3e400640 1 -- 172.21.15.7:0/3154931238 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f380b7070 con 0x7f5f180744e0 2024-04-25T11:20:48.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.693+0000 7f5f34c00640 1 -- 172.21.15.7:0/3154931238 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252463 (secure 0 0 0) 0x7f5f380b7070 con 0x7f5f180744e0 2024-04-25T11:20:48.692 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:48.695 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:20:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.700+0000 7f5f3e400640 1 -- 172.21.15.7:0/3154931238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f180744e0 msgr2=0x7f5f180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.700+0000 7f5f3e400640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f180744e0 0x7f5f180769f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c034000 tx=0x7f5f2c02e000 comp rx=0 tx=0).stop 2024-04-25T11:20:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.700+0000 7f5f3e400640 1 -- 172.21.15.7:0/3154931238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f380d2a90 msgr2=0x7f5f381e1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.700+0000 7f5f3e400640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f380d2a90 0x7f5f381e1f10 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000ec10 tx=0x7f5f3000c490 comp rx=0 tx=0).stop 2024-04-25T11:20:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.700+0000 7f5f3e400640 1 -- 172.21.15.7:0/3154931238 shutdown_connections 2024-04-25T11:20:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.700+0000 7f5f3e400640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5f180744e0 0x7f5f180769f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.700+0000 7f5f3e400640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f380d2a90 0x7f5f381e1f10 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.700+0000 7f5f3e400640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f380d0de0 0x7f5f380cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.700+0000 7f5f3e400640 1 --2- 172.21.15.7:0/3154931238 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f380ce7e0 0x7f5f380cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.700+0000 7f5f3e400640 1 -- 172.21.15.7:0/3154931238 >> 172.21.15.7:0/3154931238 conn(0x7f5f380d0250 msgr2=0x7f5f380cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.701+0000 7f5f3e400640 1 -- 172.21.15.7:0/3154931238 shutdown_connections 2024-04-25T11:20:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:48.701+0000 7f5f3e400640 1 -- 172.21.15.7:0/3154931238 wait complete. 2024-04-25T11:20:48.726 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-04-25T11:20:48.031637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279892,"kb_used_data":13796,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754694828,"statfs":{"total":773094113280,"available":772807503872,"internally_reserved":0,"allocated":14127104,"data_stored":7131648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001864"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.419364+0000","last_change":"2024-04-25T11:20:43.413723+0000","last_active":"2024-04-25T11:20:43.419364+0000","last_peered":"2024-04-25T11:20:43.419364+0000","last_clean":"2024-04-25T11:20:43.419364+0000","last_became_active":"2024-04-25T11:20:43.413305+0000","last_became_peered":"2024-04-25T11:20:43.413305+0000","last_unstale":"2024-04-25T11:20:43.419364+0000","last_undegraded":"2024-04-25T11:20:43.419364+0000","last_fullsized":"2024-04-25T11:20:43.419364+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:40.208838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419781+0000","last_change":"2024-04-25T11:20:43.412353+0000","last_active":"2024-04-25T11:20:44.419781+0000","last_peered":"2024-04-25T11:20:44.419781+0000","last_clean":"2024-04-25T11:20:44.419781+0000","last_became_active":"2024-04-25T11:20:43.411031+0000","last_became_peered":"2024-04-25T11:20:43.411031+0000","last_unstale":"2024-04-25T11:20:44.419781+0000","last_undegraded":"2024-04-25T11:20:44.419781+0000","last_fullsized":"2024-04-25T11:20:44.419781+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:25.874772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422560+0000","last_change":"2024-04-25T11:20:43.425339+0000","last_active":"2024-04-25T11:20:44.422560+0000","last_peered":"2024-04-25T11:20:44.422560+0000","last_clean":"2024-04-25T11:20:44.422560+0000","last_became_active":"2024-04-25T11:20:43.424773+0000","last_became_peered":"2024-04-25T11:20:43.424773+0000","last_unstale":"2024-04-25T11:20:44.422560+0000","last_undegraded":"2024-04-25T11:20:44.422560+0000","last_fullsized":"2024-04-25T11:20:44.422560+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:26.671112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419161+0000","last_change":"2024-04-25T11:20:43.409738+0000","last_active":"2024-04-25T11:20:44.419161+0000","last_peered":"2024-04-25T11:20:44.419161+0000","last_clean":"2024-04-25T11:20:44.419161+0000","last_became_active":"2024-04-25T11:20:43.409403+0000","last_became_peered":"2024-04-25T11:20:43.409403+0000","last_unstale":"2024-04-25T11:20:44.419161+0000","last_undegraded":"2024-04-25T11:20:44.419161+0000","last_fullsized":"2024-04-25T11:20:44.419161+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:26.671112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599032+0000","last_change":"2024-04-25T11:20:43.417699+0000","last_active":"2024-04-25T11:20:44.599032+0000","last_peered":"2024-04-25T11:20:44.599032+0000","last_clean":"2024-04-25T11:20:44.599032+0000","last_became_active":"2024-04-25T11:20:43.417342+0000","last_became_peered":"2024-04-25T11:20:43.417342+0000","last_unstale":"2024-04-25T11:20:44.599032+0000","last_undegraded":"2024-04-25T11:20:44.599032+0000","last_fullsized":"2024-04-25T11:20:44.599032+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:37.741543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598801+0000","last_change":"2024-04-25T11:20:43.417042+0000","last_active":"2024-04-25T11:20:44.598801+0000","last_peered":"2024-04-25T11:20:44.598801+0000","last_clean":"2024-04-25T11:20:44.598801+0000","last_became_active":"2024-04-25T11:20:43.416676+0000","last_became_peered":"2024-04-25T11:20:43.416676+0000","last_unstale":"2024-04-25T11:20:44.598801+0000","last_undegraded":"2024-04-25T11:20:44.598801+0000","last_fullsized":"2024-04-25T11:20:44.598801+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:33.381386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421685+0000","last_change":"2024-04-25T11:20:43.414802+0000","last_active":"2024-04-25T11:20:44.421685+0000","last_peered":"2024-04-25T11:20:44.421685+0000","last_clean":"2024-04-25T11:20:44.421685+0000","last_became_active":"2024-04-25T11:20:43.414476+0000","last_became_peered":"2024-04-25T11:20:43.414476+0000","last_unstale":"2024-04-25T11:20:44.421685+0000","last_undegraded":"2024-04-25T11:20:44.421685+0000","last_fullsized":"2024-04-25T11:20:44.421685+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:53.766981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597973+0000","last_change":"2024-04-25T11:20:43.426167+0000","last_active":"2024-04-25T11:20:44.597973+0000","last_peered":"2024-04-25T11:20:44.597973+0000","last_clean":"2024-04-25T11:20:44.597973+0000","last_became_active":"2024-04-25T11:20:43.425785+0000","last_became_peered":"2024-04-25T11:20:43.425785+0000","last_unstale":"2024-04-25T11:20:44.597973+0000","last_undegraded":"2024-04-25T11:20:44.597973+0000","last_fullsized":"2024-04-25T11:20:44.597973+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:45.799504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599700+0000","last_change":"2024-04-25T11:20:43.414574+0000","last_active":"2024-04-25T11:20:44.599700+0000","last_peered":"2024-04-25T11:20:44.599700+0000","last_clean":"2024-04-25T11:20:44.599700+0000","last_became_active":"2024-04-25T11:20:43.413194+0000","last_became_peered":"2024-04-25T11:20:43.413194+0000","last_unstale":"2024-04-25T11:20:44.599700+0000","last_undegraded":"2024-04-25T11:20:44.599700+0000","last_fullsized":"2024-04-25T11:20:44.599700+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:38.437764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564706+0000","last_change":"2024-04-25T11:20:43.414707+0000","last_active":"2024-04-25T11:20:44.564706+0000","last_peered":"2024-04-25T11:20:44.564706+0000","last_clean":"2024-04-25T11:20:44.564706+0000","last_became_active":"2024-04-25T11:20:43.414348+0000","last_became_peered":"2024-04-25T11:20:43.414348+0000","last_unstale":"2024-04-25T11:20:44.564706+0000","last_undegraded":"2024-04-25T11:20:44.564706+0000","last_fullsized":"2024-04-25T11:20:44.564706+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:47.853731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419405+0000","last_change":"2024-04-25T11:20:43.417885+0000","last_active":"2024-04-25T11:20:44.419405+0000","last_peered":"2024-04-25T11:20:44.419405+0000","last_clean":"2024-04-25T11:20:44.419405+0000","last_became_active":"2024-04-25T11:20:43.415456+0000","last_became_peered":"2024-04-25T11:20:43.415456+0000","last_unstale":"2024-04-25T11:20:44.419405+0000","last_undegraded":"2024-04-25T11:20:44.419405+0000","last_fullsized":"2024-04-25T11:20:44.419405+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:08.190683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.418596+0000","last_change":"2024-04-25T11:20:43.409715+0000","last_active":"2024-04-25T11:20:43.418596+0000","last_peered":"2024-04-25T11:20:43.418596+0000","last_clean":"2024-04-25T11:20:43.418596+0000","last_became_active":"2024-04-25T11:20:43.409259+0000","last_became_peered":"2024-04-25T11:20:43.409259+0000","last_unstale":"2024-04-25T11:20:43.418596+0000","last_undegraded":"2024-04-25T11:20:43.418596+0000","last_fullsized":"2024-04-25T11:20:43.418596+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:58.995466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599337+0000","last_change":"2024-04-25T11:20:43.427371+0000","last_active":"2024-04-25T11:20:44.599337+0000","last_peered":"2024-04-25T11:20:44.599337+0000","last_clean":"2024-04-25T11:20:44.599337+0000","last_became_active":"2024-04-25T11:20:43.426966+0000","last_became_peered":"2024-04-25T11:20:43.426966+0000","last_unstale":"2024-04-25T11:20:44.599337+0000","last_undegraded":"2024-04-25T11:20:44.599337+0000","last_fullsized":"2024-04-25T11:20:44.599337+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:16.001880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.422029+0000","last_change":"2024-04-25T11:20:43.416071+0000","last_active":"2024-04-25T11:20:43.422029+0000","last_peered":"2024-04-25T11:20:43.422029+0000","last_clean":"2024-04-25T11:20:43.422029+0000","last_became_active":"2024-04-25T11:20:43.415749+0000","last_became_peered":"2024-04-25T11:20:43.415749+0000","last_unstale":"2024-04-25T11:20:43.422029+0000","last_undegraded":"2024-04-25T11:20:43.422029+0000","last_fullsized":"2024-04-25T11:20:43.422029+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:15.942085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.422452+0000","last_change":"2024-04-25T11:20:43.417524+0000","last_active":"2024-04-25T11:20:43.422452+0000","last_peered":"2024-04-25T11:20:43.422452+0000","last_clean":"2024-04-25T11:20:43.422452+0000","last_became_active":"2024-04-25T11:20:43.417155+0000","last_became_peered":"2024-04-25T11:20:43.417155+0000","last_unstale":"2024-04-25T11:20:43.422452+0000","last_undegraded":"2024-04-25T11:20:43.422452+0000","last_fullsized":"2024-04-25T11:20:43.422452+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:09.432829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597628+0000","last_change":"2024-04-25T11:20:43.404825+0000","last_active":"2024-04-25T11:20:44.597628+0000","last_peered":"2024-04-25T11:20:44.597628+0000","last_clean":"2024-04-25T11:20:44.597628+0000","last_became_active":"2024-04-25T11:20:43.404315+0000","last_became_peered":"2024-04-25T11:20:43.404315+0000","last_unstale":"2024-04-25T11:20:44.597628+0000","last_undegraded":"2024-04-25T11:20:44.597628+0000","last_fullsized":"2024-04-25T11:20:44.597628+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:25.874772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.600186+0000","last_change":"2024-04-25T11:20:43.411053+0000","last_active":"2024-04-25T11:20:44.600186+0000","last_peered":"2024-04-25T11:20:44.600186+0000","last_clean":"2024-04-25T11:20:44.600186+0000","last_became_active":"2024-04-25T11:20:43.409332+0000","last_became_peered":"2024-04-25T11:20:43.409332+0000","last_unstale":"2024-04-25T11:20:44.600186+0000","last_undegraded":"2024-04-25T11:20:44.600186+0000","last_fullsized":"2024-04-25T11:20:44.600186+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:51.662580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.596706+0000","last_change":"2024-04-25T11:20:43.420507+0000","last_active":"2024-04-25T11:20:44.596706+0000","last_peered":"2024-04-25T11:20:44.596706+0000","last_clean":"2024-04-25T11:20:44.596706+0000","last_became_active":"2024-04-25T11:20:43.418330+0000","last_became_peered":"2024-04-25T11:20:43.418330+0000","last_unstale":"2024-04-25T11:20:44.596706+0000","last_undegraded":"2024-04-25T11:20:44.596706+0000","last_fullsized":"2024-04-25T11:20:44.596706+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:53.766981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423381+0000","last_change":"2024-04-25T11:20:43.415336+0000","last_active":"2024-04-25T11:20:44.423381+0000","last_peered":"2024-04-25T11:20:44.423381+0000","last_clean":"2024-04-25T11:20:44.423381+0000","last_became_active":"2024-04-25T11:20:43.412492+0000","last_became_peered":"2024-04-25T11:20:43.412492+0000","last_unstale":"2024-04-25T11:20:44.423381+0000","last_undegraded":"2024-04-25T11:20:44.423381+0000","last_fullsized":"2024-04-25T11:20:44.423381+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:46.271892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.596950+0000","last_change":"2024-04-25T11:20:43.400897+0000","last_active":"2024-04-25T11:20:44.596950+0000","last_peered":"2024-04-25T11:20:44.596950+0000","last_clean":"2024-04-25T11:20:44.596950+0000","last_became_active":"2024-04-25T11:20:43.397411+0000","last_became_peered":"2024-04-25T11:20:43.397411+0000","last_unstale":"2024-04-25T11:20:44.596950+0000","last_undegraded":"2024-04-25T11:20:44.596950+0000","last_fullsized":"2024-04-25T11:20:44.596950+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:39.941085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.418438+0000","last_change":"2024-04-25T11:20:43.408303+0000","last_active":"2024-04-25T11:20:44.418438+0000","last_peered":"2024-04-25T11:20:44.418438+0000","last_clean":"2024-04-25T11:20:44.418438+0000","last_became_active":"2024-04-25T11:20:43.407917+0000","last_became_peered":"2024-04-25T11:20:43.407917+0000","last_unstale":"2024-04-25T11:20:44.418438+0000","last_undegraded":"2024-04-25T11:20:44.418438+0000","last_fullsized":"2024-04-25T11:20:44.418438+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:02.808346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.414530+0000","last_change":"2024-04-25T11:20:43.408247+0000","last_active":"2024-04-25T11:20:43.414530+0000","last_peered":"2024-04-25T11:20:43.414530+0000","last_clean":"2024-04-25T11:20:43.414530+0000","last_became_active":"2024-04-25T11:20:43.407771+0000","last_became_peered":"2024-04-25T11:20:43.407771+0000","last_unstale":"2024-04-25T11:20:43.414530+0000","last_undegraded":"2024-04-25T11:20:43.414530+0000","last_fullsized":"2024-04-25T11:20:43.414530+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:05.393647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.418810+0000","last_change":"2024-04-25T11:20:43.397553+0000","last_active":"2024-04-25T11:20:44.418810+0000","last_peered":"2024-04-25T11:20:44.418810+0000","last_clean":"2024-04-25T11:20:44.418810+0000","last_became_active":"2024-04-25T11:20:43.396684+0000","last_became_peered":"2024-04-25T11:20:43.396684+0000","last_unstale":"2024-04-25T11:20:44.418810+0000","last_undegraded":"2024-04-25T11:20:44.418810+0000","last_fullsized":"2024-04-25T11:20:44.418810+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:42.730597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597917+0000","last_change":"2024-04-25T11:20:43.400746+0000","last_active":"2024-04-25T11:20:44.597917+0000","last_peered":"2024-04-25T11:20:44.597917+0000","last_clean":"2024-04-25T11:20:44.597917+0000","last_became_active":"2024-04-25T11:20:43.400339+0000","last_became_peered":"2024-04-25T11:20:43.400339+0000","last_unstale":"2024-04-25T11:20:44.597917+0000","last_undegraded":"2024-04-25T11:20:44.597917+0000","last_fullsized":"2024-04-25T11:20:44.597917+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:36.184438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422573+0000","last_change":"2024-04-25T11:20:43.429453+0000","last_active":"2024-04-25T11:20:44.422573+0000","last_peered":"2024-04-25T11:20:44.422573+0000","last_clean":"2024-04-25T11:20:44.422573+0000","last_became_active":"2024-04-25T11:20:43.428920+0000","last_became_peered":"2024-04-25T11:20:43.428920+0000","last_unstale":"2024-04-25T11:20:44.422573+0000","last_undegraded":"2024-04-25T11:20:44.422573+0000","last_fullsized":"2024-04-25T11:20:44.422573+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:08.190683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422818+0000","last_change":"2024-04-25T11:20:43.424634+0000","last_active":"2024-04-25T11:20:44.422818+0000","last_peered":"2024-04-25T11:20:44.422818+0000","last_clean":"2024-04-25T11:20:44.422818+0000","last_became_active":"2024-04-25T11:20:43.424279+0000","last_became_peered":"2024-04-25T11:20:43.424279+0000","last_unstale":"2024-04-25T11:20:44.422818+0000","last_undegraded":"2024-04-25T11:20:44.422818+0000","last_fullsized":"2024-04-25T11:20:44.422818+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:25.874772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423792+0000","last_change":"2024-04-25T11:20:43.408193+0000","last_active":"2024-04-25T11:20:44.423792+0000","last_peered":"2024-04-25T11:20:44.423792+0000","last_clean":"2024-04-25T11:20:44.423792+0000","last_became_active":"2024-04-25T11:20:43.407856+0000","last_became_peered":"2024-04-25T11:20:43.407856+0000","last_unstale":"2024-04-25T11:20:44.423792+0000","last_undegraded":"2024-04-25T11:20:44.423792+0000","last_fullsized":"2024-04-25T11:20:44.423792+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:44.774829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564433+0000","last_change":"2024-04-25T11:20:43.416430+0000","last_active":"2024-04-25T11:20:44.564433+0000","last_peered":"2024-04-25T11:20:44.564433+0000","last_clean":"2024-04-25T11:20:44.564433+0000","last_became_active":"2024-04-25T11:20:43.414706+0000","last_became_peered":"2024-04-25T11:20:43.414706+0000","last_unstale":"2024-04-25T11:20:44.564433+0000","last_undegraded":"2024-04-25T11:20:44.564433+0000","last_fullsized":"2024-04-25T11:20:44.564433+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:30.802813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423209+0000","last_change":"2024-04-25T11:20:43.426856+0000","last_active":"2024-04-25T11:20:44.423209+0000","last_peered":"2024-04-25T11:20:44.423209+0000","last_clean":"2024-04-25T11:20:44.423209+0000","last_became_active":"2024-04-25T11:20:43.426305+0000","last_became_peered":"2024-04-25T11:20:43.426305+0000","last_unstale":"2024-04-25T11:20:44.423209+0000","last_undegraded":"2024-04-25T11:20:44.423209+0000","last_fullsized":"2024-04-25T11:20:44.423209+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:36.184438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421910+0000","last_change":"2024-04-25T11:20:43.415498+0000","last_active":"2024-04-25T11:20:44.421910+0000","last_peered":"2024-04-25T11:20:44.421910+0000","last_clean":"2024-04-25T11:20:44.421910+0000","last_became_active":"2024-04-25T11:20:43.415186+0000","last_became_peered":"2024-04-25T11:20:43.415186+0000","last_unstale":"2024-04-25T11:20:44.421910+0000","last_undegraded":"2024-04-25T11:20:44.421910+0000","last_fullsized":"2024-04-25T11:20:44.421910+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:02.808346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598214+0000","last_change":"2024-04-25T11:20:43.426123+0000","last_active":"2024-04-25T11:20:44.598214+0000","last_peered":"2024-04-25T11:20:44.598214+0000","last_clean":"2024-04-25T11:20:44.598214+0000","last_became_active":"2024-04-25T11:20:43.425219+0000","last_became_peered":"2024-04-25T11:20:43.425219+0000","last_unstale":"2024-04-25T11:20:44.598214+0000","last_undegraded":"2024-04-25T11:20:44.598214+0000","last_fullsized":"2024-04-25T11:20:44.598214+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:40.892114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564024+0000","last_change":"2024-04-25T11:20:43.420104+0000","last_active":"2024-04-25T11:20:44.564024+0000","last_peered":"2024-04-25T11:20:44.564024+0000","last_clean":"2024-04-25T11:20:44.564024+0000","last_became_active":"2024-04-25T11:20:43.419535+0000","last_became_peered":"2024-04-25T11:20:43.419535+0000","last_unstale":"2024-04-25T11:20:44.564024+0000","last_undegraded":"2024-04-25T11:20:44.564024+0000","last_fullsized":"2024-04-25T11:20:44.564024+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:12.156522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597994+0000","last_change":"2024-04-25T11:20:43.419261+0000","last_active":"2024-04-25T11:20:44.597994+0000","last_peered":"2024-04-25T11:20:44.597994+0000","last_clean":"2024-04-25T11:20:44.597994+0000","last_became_active":"2024-04-25T11:20:43.417934+0000","last_became_peered":"2024-04-25T11:20:43.417934+0000","last_unstale":"2024-04-25T11:20:44.597994+0000","last_undegraded":"2024-04-25T11:20:44.597994+0000","last_fullsized":"2024-04-25T11:20:44.597994+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:26.122782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"129'4","reported_seq":18,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422680+0000","last_change":"2024-04-25T11:20:42.373454+0000","last_active":"2024-04-25T11:20:44.422680+0000","last_peered":"2024-04-25T11:20:44.422680+0000","last_clean":"2024-04-25T11:20:44.422680+0000","last_became_active":"2024-04-25T11:20:42.373041+0000","last_became_peered":"2024-04-25T11:20:42.373041+0000","last_unstale":"2024-04-25T11:20:44.422680+0000","last_undegraded":"2024-04-25T11:20:44.422680+0000","last_fullsized":"2024-04-25T11:20:44.422680+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:15.845113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.418816+0000","last_change":"2024-04-25T11:20:43.411005+0000","last_active":"2024-04-25T11:20:43.418816+0000","last_peered":"2024-04-25T11:20:43.418816+0000","last_clean":"2024-04-25T11:20:43.418816+0000","last_became_active":"2024-04-25T11:20:43.410555+0000","last_became_peered":"2024-04-25T11:20:43.410555+0000","last_unstale":"2024-04-25T11:20:43.418816+0000","last_undegraded":"2024-04-25T11:20:43.418816+0000","last_fullsized":"2024-04-25T11:20:43.418816+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:43.394881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597213+0000","last_change":"2024-04-25T11:20:42.368323+0000","last_active":"2024-04-25T11:20:44.597213+0000","last_peered":"2024-04-25T11:20:44.597213+0000","last_clean":"2024-04-25T11:20:44.597213+0000","last_became_active":"2024-04-25T11:20:42.367758+0000","last_became_peered":"2024-04-25T11:20:42.367758+0000","last_unstale":"2024-04-25T11:20:44.597213+0000","last_undegraded":"2024-04-25T11:20:44.597213+0000","last_fullsized":"2024-04-25T11:20:44.597213+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:47.040419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.418478+0000","last_change":"2024-04-25T11:20:43.412691+0000","last_active":"2024-04-25T11:20:44.418478+0000","last_peered":"2024-04-25T11:20:44.418478+0000","last_clean":"2024-04-25T11:20:44.418478+0000","last_became_active":"2024-04-25T11:20:43.406640+0000","last_became_peered":"2024-04-25T11:20:43.406640+0000","last_unstale":"2024-04-25T11:20:44.418478+0000","last_undegraded":"2024-04-25T11:20:44.418478+0000","last_fullsized":"2024-04-25T11:20:44.418478+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:33.797487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423545+0000","last_change":"2024-04-25T11:20:42.371896+0000","last_active":"2024-04-25T11:20:44.423545+0000","last_peered":"2024-04-25T11:20:44.423545+0000","last_clean":"2024-04-25T11:20:44.423545+0000","last_became_active":"2024-04-25T11:20:42.370292+0000","last_became_peered":"2024-04-25T11:20:42.370292+0000","last_unstale":"2024-04-25T11:20:44.423545+0000","last_undegraded":"2024-04-25T11:20:44.423545+0000","last_fullsized":"2024-04-25T11:20:44.423545+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:24.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.422558+0000","last_change":"2024-04-25T11:20:43.418620+0000","last_active":"2024-04-25T11:20:43.422558+0000","last_peered":"2024-04-25T11:20:43.422558+0000","last_clean":"2024-04-25T11:20:43.422558+0000","last_became_active":"2024-04-25T11:20:43.415444+0000","last_became_peered":"2024-04-25T11:20:43.415444+0000","last_unstale":"2024-04-25T11:20:43.422558+0000","last_undegraded":"2024-04-25T11:20:43.422558+0000","last_fullsized":"2024-04-25T11:20:43.422558+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:27.669039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423024+0000","last_change":"2024-04-25T11:20:42.376264+0000","last_active":"2024-04-25T11:20:44.423024+0000","last_peered":"2024-04-25T11:20:44.423024+0000","last_clean":"2024-04-25T11:20:44.423024+0000","last_became_active":"2024-04-25T11:20:42.375524+0000","last_became_peered":"2024-04-25T11:20:42.375524+0000","last_unstale":"2024-04-25T11:20:44.423024+0000","last_undegraded":"2024-04-25T11:20:44.423024+0000","last_fullsized":"2024-04-25T11:20:44.423024+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:37.432323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598839+0000","last_change":"2024-04-25T11:20:43.426118+0000","last_active":"2024-04-25T11:20:44.598839+0000","last_peered":"2024-04-25T11:20:44.598839+0000","last_clean":"2024-04-25T11:20:44.598839+0000","last_became_active":"2024-04-25T11:20:43.425388+0000","last_became_peered":"2024-04-25T11:20:43.425388+0000","last_unstale":"2024-04-25T11:20:44.598839+0000","last_undegraded":"2024-04-25T11:20:44.598839+0000","last_fullsized":"2024-04-25T11:20:44.598839+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:02.808346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598125+0000","last_change":"2024-04-25T11:20:42.377296+0000","last_active":"2024-04-25T11:20:44.598125+0000","last_peered":"2024-04-25T11:20:44.598125+0000","last_clean":"2024-04-25T11:20:44.598125+0000","last_became_active":"2024-04-25T11:20:42.376970+0000","last_became_peered":"2024-04-25T11:20:42.376970+0000","last_unstale":"2024-04-25T11:20:44.598125+0000","last_undegraded":"2024-04-25T11:20:44.598125+0000","last_fullsized":"2024-04-25T11:20:44.598125+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:35.541447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598855+0000","last_change":"2024-04-25T11:20:43.414083+0000","last_active":"2024-04-25T11:20:44.598855+0000","last_peered":"2024-04-25T11:20:44.598855+0000","last_clean":"2024-04-25T11:20:44.598855+0000","last_became_active":"2024-04-25T11:20:43.411532+0000","last_became_peered":"2024-04-25T11:20:43.411532+0000","last_unstale":"2024-04-25T11:20:44.598855+0000","last_undegraded":"2024-04-25T11:20:44.598855+0000","last_fullsized":"2024-04-25T11:20:44.598855+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:46.758800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.596451+0000","last_change":"2024-04-25T11:20:42.373623+0000","last_active":"2024-04-25T11:20:44.596451+0000","last_peered":"2024-04-25T11:20:44.596451+0000","last_clean":"2024-04-25T11:20:44.596451+0000","last_became_active":"2024-04-25T11:20:42.370550+0000","last_became_peered":"2024-04-25T11:20:42.370550+0000","last_unstale":"2024-04-25T11:20:44.596451+0000","last_undegraded":"2024-04-25T11:20:44.596451+0000","last_fullsized":"2024-04-25T11:20:44.596451+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:24.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597044+0000","last_change":"2024-04-25T11:20:43.401381+0000","last_active":"2024-04-25T11:20:44.597044+0000","last_peered":"2024-04-25T11:20:44.597044+0000","last_clean":"2024-04-25T11:20:44.597044+0000","last_became_active":"2024-04-25T11:20:43.399646+0000","last_became_peered":"2024-04-25T11:20:43.399646+0000","last_unstale":"2024-04-25T11:20:44.597044+0000","last_undegraded":"2024-04-25T11:20:44.597044+0000","last_fullsized":"2024-04-25T11:20:44.597044+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:37.561947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599160+0000","last_change":"2024-04-25T11:20:42.369518+0000","last_active":"2024-04-25T11:20:44.599160+0000","last_peered":"2024-04-25T11:20:44.599160+0000","last_clean":"2024-04-25T11:20:44.599160+0000","last_became_active":"2024-04-25T11:20:42.369100+0000","last_became_peered":"2024-04-25T11:20:42.369100+0000","last_unstale":"2024-04-25T11:20:44.599160+0000","last_undegraded":"2024-04-25T11:20:44.599160+0000","last_fullsized":"2024-04-25T11:20:44.599160+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:34.115268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423469+0000","last_change":"2024-04-25T11:20:43.412931+0000","last_active":"2024-04-25T11:20:44.423469+0000","last_peered":"2024-04-25T11:20:44.423469+0000","last_clean":"2024-04-25T11:20:44.423469+0000","last_became_active":"2024-04-25T11:20:43.412578+0000","last_became_peered":"2024-04-25T11:20:43.412578+0000","last_unstale":"2024-04-25T11:20:44.423469+0000","last_undegraded":"2024-04-25T11:20:44.423469+0000","last_fullsized":"2024-04-25T11:20:44.423469+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:44.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.420432+0000","last_change":"2024-04-25T11:20:42.376354+0000","last_active":"2024-04-25T11:20:44.420432+0000","last_peered":"2024-04-25T11:20:44.420432+0000","last_clean":"2024-04-25T11:20:44.420432+0000","last_became_active":"2024-04-25T11:20:42.376012+0000","last_became_peered":"2024-04-25T11:20:42.376012+0000","last_unstale":"2024-04-25T11:20:44.420432+0000","last_undegraded":"2024-04-25T11:20:44.420432+0000","last_fullsized":"2024-04-25T11:20:44.420432+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:47.365940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.421975+0000","last_change":"2024-04-25T11:20:43.415496+0000","last_active":"2024-04-25T11:20:43.421975+0000","last_peered":"2024-04-25T11:20:43.421975+0000","last_clean":"2024-04-25T11:20:43.421975+0000","last_became_active":"2024-04-25T11:20:43.415147+0000","last_became_peered":"2024-04-25T11:20:43.415147+0000","last_unstale":"2024-04-25T11:20:43.421975+0000","last_undegraded":"2024-04-25T11:20:43.421975+0000","last_fullsized":"2024-04-25T11:20:43.421975+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:11.851404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419432+0000","last_change":"2024-04-25T11:20:42.374689+0000","last_active":"2024-04-25T11:20:44.419432+0000","last_peered":"2024-04-25T11:20:44.419432+0000","last_clean":"2024-04-25T11:20:44.419432+0000","last_became_active":"2024-04-25T11:20:42.374235+0000","last_became_peered":"2024-04-25T11:20:42.374235+0000","last_unstale":"2024-04-25T11:20:44.419432+0000","last_undegraded":"2024-04-25T11:20:44.419432+0000","last_fullsized":"2024-04-25T11:20:44.419432+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:27.727738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599390+0000","last_change":"2024-04-25T11:20:43.410603+0000","last_active":"2024-04-25T11:20:44.599390+0000","last_peered":"2024-04-25T11:20:44.599390+0000","last_clean":"2024-04-25T11:20:44.599390+0000","last_became_active":"2024-04-25T11:20:43.410266+0000","last_became_peered":"2024-04-25T11:20:43.410266+0000","last_unstale":"2024-04-25T11:20:44.599390+0000","last_undegraded":"2024-04-25T11:20:44.599390+0000","last_fullsized":"2024-04-25T11:20:44.599390+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:19.371050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.403440+0000","last_change":"2024-04-25T11:20:42.380279+0000","last_active":"2024-04-25T11:20:43.403440+0000","last_peered":"2024-04-25T11:20:43.403440+0000","last_clean":"2024-04-25T11:20:43.403440+0000","last_became_active":"2024-04-25T11:20:42.379390+0000","last_became_peered":"2024-04-25T11:20:42.379390+0000","last_unstale":"2024-04-25T11:20:43.403440+0000","last_undegraded":"2024-04-25T11:20:43.403440+0000","last_fullsized":"2024-04-25T11:20:43.403440+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:28.286300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422041+0000","last_change":"2024-04-25T11:20:43.424857+0000","last_active":"2024-04-25T11:20:44.422041+0000","last_peered":"2024-04-25T11:20:44.422041+0000","last_clean":"2024-04-25T11:20:44.422041+0000","last_became_active":"2024-04-25T11:20:43.422860+0000","last_became_peered":"2024-04-25T11:20:43.422860+0000","last_unstale":"2024-04-25T11:20:44.422041+0000","last_undegraded":"2024-04-25T11:20:44.422041+0000","last_fullsized":"2024-04-25T11:20:44.422041+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:33.797487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"129'3","reported_seq":17,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423722+0000","last_change":"2024-04-25T11:20:42.372299+0000","last_active":"2024-04-25T11:20:44.423722+0000","last_peered":"2024-04-25T11:20:44.423722+0000","last_clean":"2024-04-25T11:20:44.423722+0000","last_became_active":"2024-04-25T11:20:42.371945+0000","last_became_peered":"2024-04-25T11:20:42.371945+0000","last_unstale":"2024-04-25T11:20:44.423722+0000","last_undegraded":"2024-04-25T11:20:44.423722+0000","last_fullsized":"2024-04-25T11:20:44.423722+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:47.040419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422504+0000","last_change":"2024-04-25T11:20:43.423703+0000","last_active":"2024-04-25T11:20:44.422504+0000","last_peered":"2024-04-25T11:20:44.422504+0000","last_clean":"2024-04-25T11:20:44.422504+0000","last_became_active":"2024-04-25T11:20:43.423369+0000","last_became_peered":"2024-04-25T11:20:43.423369+0000","last_unstale":"2024-04-25T11:20:44.422504+0000","last_undegraded":"2024-04-25T11:20:44.422504+0000","last_fullsized":"2024-04-25T11:20:44.422504+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:42.730597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"129'6","reported_seq":20,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423159+0000","last_change":"2024-04-25T11:20:42.376532+0000","last_active":"2024-04-25T11:20:44.423159+0000","last_peered":"2024-04-25T11:20:44.423159+0000","last_clean":"2024-04-25T11:20:44.423159+0000","last_became_active":"2024-04-25T11:20:42.375922+0000","last_became_peered":"2024-04-25T11:20:42.375922+0000","last_unstale":"2024-04-25T11:20:44.423159+0000","last_undegraded":"2024-04-25T11:20:44.423159+0000","last_fullsized":"2024-04-25T11:20:44.423159+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:50.657139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564682+0000","last_change":"2024-04-25T11:20:43.411891+0000","last_active":"2024-04-25T11:20:44.564682+0000","last_peered":"2024-04-25T11:20:44.564682+0000","last_clean":"2024-04-25T11:20:44.564682+0000","last_became_active":"2024-04-25T11:20:43.411541+0000","last_became_peered":"2024-04-25T11:20:43.411541+0000","last_unstale":"2024-04-25T11:20:44.564682+0000","last_undegraded":"2024-04-25T11:20:44.564682+0000","last_fullsized":"2024-04-25T11:20:44.564682+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:05.281320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.431962+0000","last_change":"2024-04-25T11:20:42.381155+0000","last_active":"2024-04-25T11:20:43.431962+0000","last_peered":"2024-04-25T11:20:43.431962+0000","last_clean":"2024-04-25T11:20:43.431962+0000","last_became_active":"2024-04-25T11:20:42.380803+0000","last_became_peered":"2024-04-25T11:20:42.380803+0000","last_unstale":"2024-04-25T11:20:43.431962+0000","last_undegraded":"2024-04-25T11:20:43.431962+0000","last_fullsized":"2024-04-25T11:20:43.431962+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:17.228130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423624+0000","last_change":"2024-04-25T11:20:43.417277+0000","last_active":"2024-04-25T11:20:44.423624+0000","last_peered":"2024-04-25T11:20:44.423624+0000","last_clean":"2024-04-25T11:20:44.423624+0000","last_became_active":"2024-04-25T11:20:43.416914+0000","last_became_peered":"2024-04-25T11:20:43.416914+0000","last_unstale":"2024-04-25T11:20:44.423624+0000","last_undegraded":"2024-04-25T11:20:44.423624+0000","last_fullsized":"2024-04-25T11:20:44.423624+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:37.741543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.420668+0000","last_change":"2024-04-25T11:20:42.367031+0000","last_active":"2024-04-25T11:20:44.420668+0000","last_peered":"2024-04-25T11:20:44.420668+0000","last_clean":"2024-04-25T11:20:44.420668+0000","last_became_active":"2024-04-25T11:20:42.366576+0000","last_became_peered":"2024-04-25T11:20:42.366576+0000","last_unstale":"2024-04-25T11:20:44.420668+0000","last_undegraded":"2024-04-25T11:20:44.420668+0000","last_fullsized":"2024-04-25T11:20:44.420668+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:29.172494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597082+0000","last_change":"2024-04-25T11:20:43.405902+0000","last_active":"2024-04-25T11:20:44.597082+0000","last_peered":"2024-04-25T11:20:44.597082+0000","last_clean":"2024-04-25T11:20:44.597082+0000","last_became_active":"2024-04-25T11:20:43.405559+0000","last_became_peered":"2024-04-25T11:20:43.405559+0000","last_unstale":"2024-04-25T11:20:44.597082+0000","last_undegraded":"2024-04-25T11:20:44.597082+0000","last_fullsized":"2024-04-25T11:20:44.597082+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:46.271892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"129'8","reported_seq":22,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597869+0000","last_change":"2024-04-25T11:20:42.366255+0000","last_active":"2024-04-25T11:20:44.597869+0000","last_peered":"2024-04-25T11:20:44.597869+0000","last_clean":"2024-04-25T11:20:44.597869+0000","last_became_active":"2024-04-25T11:20:42.364737+0000","last_became_peered":"2024-04-25T11:20:42.364737+0000","last_unstale":"2024-04-25T11:20:44.597869+0000","last_undegraded":"2024-04-25T11:20:44.597869+0000","last_fullsized":"2024-04-25T11:20:44.597869+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:12.016407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.420321+0000","last_change":"2024-04-25T11:20:43.432463+0000","last_active":"2024-04-25T11:20:44.420321+0000","last_peered":"2024-04-25T11:20:44.420321+0000","last_clean":"2024-04-25T11:20:44.420321+0000","last_became_active":"2024-04-25T11:20:43.428513+0000","last_became_peered":"2024-04-25T11:20:43.428513+0000","last_unstale":"2024-04-25T11:20:44.420321+0000","last_undegraded":"2024-04-25T11:20:44.420321+0000","last_fullsized":"2024-04-25T11:20:44.420321+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:40.892114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419527+0000","last_change":"2024-04-25T11:20:42.377113+0000","last_active":"2024-04-25T11:20:44.419527+0000","last_peered":"2024-04-25T11:20:44.419527+0000","last_clean":"2024-04-25T11:20:44.419527+0000","last_became_active":"2024-04-25T11:20:42.376770+0000","last_became_peered":"2024-04-25T11:20:42.376770+0000","last_unstale":"2024-04-25T11:20:44.419527+0000","last_undegraded":"2024-04-25T11:20:44.419527+0000","last_fullsized":"2024-04-25T11:20:44.419527+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:56.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421942+0000","last_change":"2024-04-25T11:20:43.404474+0000","last_active":"2024-04-25T11:20:44.421942+0000","last_peered":"2024-04-25T11:20:44.421942+0000","last_clean":"2024-04-25T11:20:44.421942+0000","last_became_active":"2024-04-25T11:20:43.404094+0000","last_became_peered":"2024-04-25T11:20:43.404094+0000","last_unstale":"2024-04-25T11:20:44.421942+0000","last_undegraded":"2024-04-25T11:20:44.421942+0000","last_fullsized":"2024-04-25T11:20:44.421942+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:39.941085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422831+0000","last_change":"2024-04-25T11:20:42.384484+0000","last_active":"2024-04-25T11:20:44.422831+0000","last_peered":"2024-04-25T11:20:44.422831+0000","last_clean":"2024-04-25T11:20:44.422831+0000","last_became_active":"2024-04-25T11:20:42.383490+0000","last_became_peered":"2024-04-25T11:20:42.383490+0000","last_unstale":"2024-04-25T11:20:44.422831+0000","last_undegraded":"2024-04-25T11:20:44.422831+0000","last_fullsized":"2024-04-25T11:20:44.422831+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:50.930451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422256+0000","last_change":"2024-04-25T11:20:43.412744+0000","last_active":"2024-04-25T11:20:44.422256+0000","last_peered":"2024-04-25T11:20:44.422256+0000","last_clean":"2024-04-25T11:20:44.422256+0000","last_became_active":"2024-04-25T11:20:43.408707+0000","last_became_peered":"2024-04-25T11:20:43.408707+0000","last_unstale":"2024-04-25T11:20:44.422256+0000","last_undegraded":"2024-04-25T11:20:44.422256+0000","last_fullsized":"2024-04-25T11:20:44.422256+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:37.561947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.600279+0000","last_change":"2024-04-25T11:20:42.371276+0000","last_active":"2024-04-25T11:20:44.600279+0000","last_peered":"2024-04-25T11:20:44.600279+0000","last_clean":"2024-04-25T11:20:44.600279+0000","last_became_active":"2024-04-25T11:20:42.370892+0000","last_became_peered":"2024-04-25T11:20:42.370892+0000","last_unstale":"2024-04-25T11:20:44.600279+0000","last_undegraded":"2024-04-25T11:20:44.600279+0000","last_fullsized":"2024-04-25T11:20:44.600279+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:42.294115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564241+0000","last_change":"2024-04-25T11:20:43.425979+0000","last_active":"2024-04-25T11:20:44.564241+0000","last_peered":"2024-04-25T11:20:44.564241+0000","last_clean":"2024-04-25T11:20:44.564241+0000","last_became_active":"2024-04-25T11:20:43.424946+0000","last_became_peered":"2024-04-25T11:20:43.424946+0000","last_unstale":"2024-04-25T11:20:44.564241+0000","last_undegraded":"2024-04-25T11:20:44.564241+0000","last_fullsized":"2024-04-25T11:20:44.564241+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:43.445693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597354+0000","last_change":"2024-04-25T11:20:42.372055+0000","last_active":"2024-04-25T11:20:44.597354+0000","last_peered":"2024-04-25T11:20:44.597354+0000","last_clean":"2024-04-25T11:20:44.597354+0000","last_became_active":"2024-04-25T11:20:42.371748+0000","last_became_peered":"2024-04-25T11:20:42.371748+0000","last_unstale":"2024-04-25T11:20:44.597354+0000","last_undegraded":"2024-04-25T11:20:44.597354+0000","last_fullsized":"2024-04-25T11:20:44.597354+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:38.507238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.597217+0000","last_change":"2024-04-25T11:20:43.408905+0000","last_active":"2024-04-25T11:20:44.597217+0000","last_peered":"2024-04-25T11:20:44.597217+0000","last_clean":"2024-04-25T11:20:44.597217+0000","last_became_active":"2024-04-25T11:20:43.408505+0000","last_became_peered":"2024-04-25T11:20:43.408505+0000","last_unstale":"2024-04-25T11:20:44.597217+0000","last_undegraded":"2024-04-25T11:20:44.597217+0000","last_fullsized":"2024-04-25T11:20:44.597217+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:44.372049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"129'2","reported_seq":16,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598484+0000","last_change":"2024-04-25T11:20:42.379960+0000","last_active":"2024-04-25T11:20:44.598484+0000","last_peered":"2024-04-25T11:20:44.598484+0000","last_clean":"2024-04-25T11:20:44.598484+0000","last_became_active":"2024-04-25T11:20:42.379582+0000","last_became_peered":"2024-04-25T11:20:42.379582+0000","last_unstale":"2024-04-25T11:20:44.598484+0000","last_undegraded":"2024-04-25T11:20:44.598484+0000","last_fullsized":"2024-04-25T11:20:44.598484+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:47.276923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.428685+0000","last_change":"2024-04-25T11:20:43.426267+0000","last_active":"2024-04-25T11:20:43.428685+0000","last_peered":"2024-04-25T11:20:43.428685+0000","last_clean":"2024-04-25T11:20:43.428685+0000","last_became_active":"2024-04-25T11:20:43.425937+0000","last_became_peered":"2024-04-25T11:20:43.425937+0000","last_unstale":"2024-04-25T11:20:43.428685+0000","last_undegraded":"2024-04-25T11:20:43.428685+0000","last_fullsized":"2024-04-25T11:20:43.428685+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:13.983664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.401277+0000","last_change":"2024-04-25T11:20:42.382049+0000","last_active":"2024-04-25T11:20:43.401277+0000","last_peered":"2024-04-25T11:20:43.401277+0000","last_clean":"2024-04-25T11:20:43.401277+0000","last_became_active":"2024-04-25T11:20:42.381621+0000","last_became_peered":"2024-04-25T11:20:42.381621+0000","last_unstale":"2024-04-25T11:20:43.401277+0000","last_undegraded":"2024-04-25T11:20:43.401277+0000","last_fullsized":"2024-04-25T11:20:43.401277+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:39.016017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598363+0000","last_change":"2024-04-25T11:20:43.413150+0000","last_active":"2024-04-25T11:20:44.598363+0000","last_peered":"2024-04-25T11:20:44.598363+0000","last_clean":"2024-04-25T11:20:44.598363+0000","last_became_active":"2024-04-25T11:20:43.412595+0000","last_became_peered":"2024-04-25T11:20:43.412595+0000","last_unstale":"2024-04-25T11:20:44.598363+0000","last_undegraded":"2024-04-25T11:20:44.598363+0000","last_fullsized":"2024-04-25T11:20:44.598363+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:58.042477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421970+0000","last_change":"2024-04-25T11:20:42.379509+0000","last_active":"2024-04-25T11:20:44.421970+0000","last_peered":"2024-04-25T11:20:44.421970+0000","last_clean":"2024-04-25T11:20:44.421970+0000","last_became_active":"2024-04-25T11:20:42.378547+0000","last_became_peered":"2024-04-25T11:20:42.378547+0000","last_unstale":"2024-04-25T11:20:44.421970+0000","last_undegraded":"2024-04-25T11:20:44.421970+0000","last_fullsized":"2024-04-25T11:20:44.421970+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:14.996438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419269+0000","last_change":"2024-04-25T11:20:43.420252+0000","last_active":"2024-04-25T11:20:44.419269+0000","last_peered":"2024-04-25T11:20:44.419269+0000","last_clean":"2024-04-25T11:20:44.419269+0000","last_became_active":"2024-04-25T11:20:43.419911+0000","last_became_peered":"2024-04-25T11:20:43.419911+0000","last_unstale":"2024-04-25T11:20:44.419269+0000","last_undegraded":"2024-04-25T11:20:44.419269+0000","last_fullsized":"2024-04-25T11:20:44.419269+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:39.553093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598038+0000","last_change":"2024-04-25T11:20:42.360966+0000","last_active":"2024-04-25T11:20:44.598038+0000","last_peered":"2024-04-25T11:20:44.598038+0000","last_clean":"2024-04-25T11:20:44.598038+0000","last_became_active":"2024-04-25T11:20:42.360596+0000","last_became_peered":"2024-04-25T11:20:42.360596+0000","last_unstale":"2024-04-25T11:20:44.598038+0000","last_undegraded":"2024-04-25T11:20:44.598038+0000","last_fullsized":"2024-04-25T11:20:44.598038+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:56.829256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.596694+0000","last_change":"2024-04-25T11:20:43.401914+0000","last_active":"2024-04-25T11:20:44.596694+0000","last_peered":"2024-04-25T11:20:44.596694+0000","last_clean":"2024-04-25T11:20:44.596694+0000","last_became_active":"2024-04-25T11:20:43.398788+0000","last_became_peered":"2024-04-25T11:20:43.398788+0000","last_unstale":"2024-04-25T11:20:44.596694+0000","last_undegraded":"2024-04-25T11:20:44.596694+0000","last_fullsized":"2024-04-25T11:20:44.596694+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:47.707614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"129'2","reported_seq":14,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-04-25T11:20:43.436474+0000","last_change":"2024-04-25T11:20:42.377964+0000","last_active":"2024-04-25T11:20:43.436474+0000","last_peered":"2024-04-25T11:20:43.436474+0000","last_clean":"2024-04-25T11:20:43.436474+0000","last_became_active":"2024-04-25T11:20:42.377418+0000","last_became_peered":"2024-04-25T11:20:42.377418+0000","last_unstale":"2024-04-25T11:20:43.436474+0000","last_undegraded":"2024-04-25T11:20:43.436474+0000","last_fullsized":"2024-04-25T11:20:43.436474+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:25.827894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421052+0000","last_change":"2024-04-25T11:20:43.414949+0000","last_active":"2024-04-25T11:20:44.421052+0000","last_peered":"2024-04-25T11:20:44.421052+0000","last_clean":"2024-04-25T11:20:44.421052+0000","last_became_active":"2024-04-25T11:20:43.414616+0000","last_became_peered":"2024-04-25T11:20:43.414616+0000","last_unstale":"2024-04-25T11:20:44.421052+0000","last_undegraded":"2024-04-25T11:20:44.421052+0000","last_fullsized":"2024-04-25T11:20:44.421052+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:46.758800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"129'1","reported_seq":15,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598827+0000","last_change":"2024-04-25T11:20:42.370410+0000","last_active":"2024-04-25T11:20:44.598827+0000","last_peered":"2024-04-25T11:20:44.598827+0000","last_clean":"2024-04-25T11:20:44.598827+0000","last_became_active":"2024-04-25T11:20:42.370102+0000","last_became_peered":"2024-04-25T11:20:42.370102+0000","last_unstale":"2024-04-25T11:20:44.598827+0000","last_undegraded":"2024-04-25T11:20:44.598827+0000","last_fullsized":"2024-04-25T11:20:44.598827+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:12.016407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421732+0000","last_change":"2024-04-25T11:20:43.415984+0000","last_active":"2024-04-25T11:20:44.421732+0000","last_peered":"2024-04-25T11:20:44.421732+0000","last_clean":"2024-04-25T11:20:44.421732+0000","last_became_active":"2024-04-25T11:20:43.415640+0000","last_became_peered":"2024-04-25T11:20:43.415640+0000","last_unstale":"2024-04-25T11:20:44.421732+0000","last_undegraded":"2024-04-25T11:20:44.421732+0000","last_fullsized":"2024-04-25T11:20:44.421732+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:47.707614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598429+0000","last_change":"2024-04-25T11:20:42.383705+0000","last_active":"2024-04-25T11:20:44.598429+0000","last_peered":"2024-04-25T11:20:44.598429+0000","last_clean":"2024-04-25T11:20:44.598429+0000","last_became_active":"2024-04-25T11:20:42.383361+0000","last_became_peered":"2024-04-25T11:20:42.383361+0000","last_unstale":"2024-04-25T11:20:44.598429+0000","last_undegraded":"2024-04-25T11:20:44.598429+0000","last_fullsized":"2024-04-25T11:20:44.598429+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:27.727738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.564311+0000","last_change":"2024-04-25T11:20:43.416088+0000","last_active":"2024-04-25T11:20:44.564311+0000","last_peered":"2024-04-25T11:20:44.564311+0000","last_clean":"2024-04-25T11:20:44.564311+0000","last_became_active":"2024-04-25T11:20:43.415757+0000","last_became_peered":"2024-04-25T11:20:43.415757+0000","last_unstale":"2024-04-25T11:20:44.564311+0000","last_undegraded":"2024-04-25T11:20:44.564311+0000","last_fullsized":"2024-04-25T11:20:44.564311+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:11.139740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599849+0000","last_change":"2024-04-25T11:20:42.371221+0000","last_active":"2024-04-25T11:20:44.599849+0000","last_peered":"2024-04-25T11:20:44.599849+0000","last_clean":"2024-04-25T11:20:44.599849+0000","last_became_active":"2024-04-25T11:20:42.370464+0000","last_became_peered":"2024-04-25T11:20:42.370464+0000","last_unstale":"2024-04-25T11:20:44.599849+0000","last_undegraded":"2024-04-25T11:20:44.599849+0000","last_fullsized":"2024-04-25T11:20:44.599849+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:41.715534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.600524+0000","last_change":"2024-04-25T11:20:43.410926+0000","last_active":"2024-04-25T11:20:44.600524+0000","last_peered":"2024-04-25T11:20:44.600524+0000","last_clean":"2024-04-25T11:20:44.600524+0000","last_became_active":"2024-04-25T11:20:43.410605+0000","last_became_peered":"2024-04-25T11:20:43.410605+0000","last_unstale":"2024-04-25T11:20:44.600524+0000","last_undegraded":"2024-04-25T11:20:44.600524+0000","last_fullsized":"2024-04-25T11:20:44.600524+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:48.045860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.419155+0000","last_change":"2024-04-25T11:20:42.384326+0000","last_active":"2024-04-25T11:20:44.419155+0000","last_peered":"2024-04-25T11:20:44.419155+0000","last_clean":"2024-04-25T11:20:44.419155+0000","last_became_active":"2024-04-25T11:20:42.383518+0000","last_became_peered":"2024-04-25T11:20:42.383518+0000","last_unstale":"2024-04-25T11:20:44.419155+0000","last_undegraded":"2024-04-25T11:20:44.419155+0000","last_fullsized":"2024-04-25T11:20:44.419155+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:12.016407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.600248+0000","last_change":"2024-04-25T11:20:43.415403+0000","last_active":"2024-04-25T11:20:44.600248+0000","last_peered":"2024-04-25T11:20:44.600248+0000","last_clean":"2024-04-25T11:20:44.600248+0000","last_became_active":"2024-04-25T11:20:43.412411+0000","last_became_peered":"2024-04-25T11:20:43.412411+0000","last_unstale":"2024-04-25T11:20:44.600248+0000","last_undegraded":"2024-04-25T11:20:44.600248+0000","last_fullsized":"2024-04-25T11:20:44.600248+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:25.414615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422193+0000","last_change":"2024-04-25T11:20:42.371478+0000","last_active":"2024-04-25T11:20:44.422193+0000","last_peered":"2024-04-25T11:20:44.422193+0000","last_clean":"2024-04-25T11:20:44.422193+0000","last_became_active":"2024-04-25T11:20:42.371144+0000","last_became_peered":"2024-04-25T11:20:42.371144+0000","last_unstale":"2024-04-25T11:20:44.422193+0000","last_undegraded":"2024-04-25T11:20:44.422193+0000","last_fullsized":"2024-04-25T11:20:44.422193+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:18.365609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.598879+0000","last_change":"2024-04-25T11:20:43.415030+0000","last_active":"2024-04-25T11:20:44.598879+0000","last_peered":"2024-04-25T11:20:44.598879+0000","last_clean":"2024-04-25T11:20:44.598879+0000","last_became_active":"2024-04-25T11:20:43.414697+0000","last_became_peered":"2024-04-25T11:20:43.414697+0000","last_unstale":"2024-04-25T11:20:44.598879+0000","last_undegraded":"2024-04-25T11:20:44.598879+0000","last_fullsized":"2024-04-25T11:20:44.598879+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:55.138633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.599551+0000","last_change":"2024-04-25T11:20:42.359739+0000","last_active":"2024-04-25T11:20:44.599551+0000","last_peered":"2024-04-25T11:20:44.599551+0000","last_clean":"2024-04-25T11:20:44.599551+0000","last_became_active":"2024-04-25T11:20:42.357458+0000","last_became_peered":"2024-04-25T11:20:42.357458+0000","last_unstale":"2024-04-25T11:20:44.599551+0000","last_undegraded":"2024-04-25T11:20:44.599551+0000","last_fullsized":"2024-04-25T11:20:44.599551+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:04.832828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.423469+0000","last_change":"2024-04-25T11:20:43.422095+0000","last_active":"2024-04-25T11:20:44.423469+0000","last_peered":"2024-04-25T11:20:44.423469+0000","last_clean":"2024-04-25T11:20:44.423469+0000","last_became_active":"2024-04-25T11:20:43.421758+0000","last_became_peered":"2024-04-25T11:20:43.421758+0000","last_unstale":"2024-04-25T11:20:44.423469+0000","last_undegraded":"2024-04-25T11:20:44.423469+0000","last_fullsized":"2024-04-25T11:20:44.423469+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:58.042477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.596853+0000","last_change":"2024-04-25T11:20:42.375043+0000","last_active":"2024-04-25T11:20:44.596853+0000","last_peered":"2024-04-25T11:20:44.596853+0000","last_clean":"2024-04-25T11:20:44.596853+0000","last_became_active":"2024-04-25T11:20:42.374678+0000","last_became_peered":"2024-04-25T11:20:42.374678+0000","last_unstale":"2024-04-25T11:20:44.596853+0000","last_undegraded":"2024-04-25T11:20:44.596853+0000","last_fullsized":"2024-04-25T11:20:44.596853+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:50.657139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422560+0000","last_change":"2024-04-25T11:20:43.429330+0000","last_active":"2024-04-25T11:20:44.422560+0000","last_peered":"2024-04-25T11:20:44.422560+0000","last_clean":"2024-04-25T11:20:44.422560+0000","last_became_active":"2024-04-25T11:20:43.426542+0000","last_became_peered":"2024-04-25T11:20:43.426542+0000","last_unstale":"2024-04-25T11:20:44.422560+0000","last_undegraded":"2024-04-25T11:20:44.422560+0000","last_fullsized":"2024-04-25T11:20:44.422560+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":128,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:42.327816+0000","last_clean_scrub_stamp":"2024-04-25T11:20:42.327816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:39.553093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.422031+0000","last_change":"2024-04-25T11:20:42.374129+0000","last_active":"2024-04-25T11:20:44.422031+0000","last_peered":"2024-04-25T11:20:44.422031+0000","last_clean":"2024-04-25T11:20:44.422031+0000","last_became_active":"2024-04-25T11:20:42.373753+0000","last_became_peered":"2024-04-25T11:20:42.373753+0000","last_unstale":"2024-04-25T11:20:44.422031+0000","last_undegraded":"2024-04-25T11:20:44.422031+0000","last_fullsized":"2024-04-25T11:20:44.422031+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:20:41.322375+0000","last_clean_scrub_stamp":"2024-04-25T11:20:41.322375+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:54.133192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":329,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T11:20:44.421345+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:20:44.421345+0000","last_peered":"2024-04-25T11:20:44.421345+0000","last_clean":"2024-04-25T11:20:44.421345+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:20:44.421345+0000","last_undegraded":"2024-04-25T11:20:44.421345+0000","last_fullsized":"2024-04-25T11:20:44.421345+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33352,"kb_used_data":2172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338488,"statfs":{"total":96636764160,"available":96602611712,"internally_reserved":0,"allocated":2224128,"data_stored":1337841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36924,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334916,"statfs":{"total":96636764160,"available":96598953984,"internally_reserved":0,"allocated":1622016,"data_stored":746443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1617920,"data_stored":745793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476866,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36916,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334924,"statfs":{"total":96636764160,"available":96598962176,"internally_reserved":0,"allocated":1613824,"data_stored":745771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509569,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36936,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334904,"statfs":{"total":96636764160,"available":96598941696,"internally_reserved":0,"allocated":1630208,"data_stored":747453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509570,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":739149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509569,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37496,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334344,"statfs":{"total":96636764160,"available":96598368256,"internally_reserved":0,"allocated":2203648,"data_stored":1329503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542274,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1609728,"data_stored":739695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:20:48.729 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:20:48.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:20:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.072+0000 7f2b89000640 1 Processor -- start 2024-04-25T11:20:49.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.072+0000 7f2b89000640 1 -- start start 2024-04-25T11:20:49.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.074+0000 7f2b89000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b840ce8d0 0x7f2b840cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.074+0000 7f2b89000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b8415c2e0 0x7f2b8415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.074+0000 7f2b89000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b840b4830 con 0x7f2b8415fef0 2024-04-25T11:20:49.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.074+0000 7f2b89000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2b840b4b10 con 0x7f2b8415c2e0 2024-04-25T11:20:49.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.074+0000 7f2b89000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b840b4df0 con 0x7f2b840ce8d0 2024-04-25T11:20:49.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.075+0000 7f2b82a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b8415c2e0 0x7f2b8415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.075+0000 7f2b83e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2b8415fef0 0x7f2b84160300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48446/0 (socket says 172.21.15.7:48446) 2024-04-25T11:20:49.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.075+0000 7f2b83e00640 1 -- 172.21.15.7:0/1465659595 learned_addr learned my addr 172.21.15.7:0/1465659595 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:49.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.075+0000 7f2b83400640 1 --2- 172.21.15.7:0/1465659595 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b840ce8d0 0x7f2b840cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.075+0000 7f2b82a00640 1 -- 172.21.15.7:0/1465659595 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b840ce8d0 msgr2=0x7f2b840cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:49.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.075+0000 7f2b82a00640 1 --2- 172.21.15.7:0/1465659595 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b840ce8d0 0x7f2b840cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.075+0000 7f2b82a00640 1 -- 172.21.15.7:0/1465659595 >> v1:172.21.15.7:6789/0 conn(0x7f2b8415fef0 legacy=0x7f2b84160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:49.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.075+0000 7f2b82a00640 1 -- 172.21.15.7:0/1465659595 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b840b50d0 con 0x7f2b8415c2e0 2024-04-25T11:20:49.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.075+0000 7f2b82000640 1 -- 172.21.15.7:0/1465659595 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1503353476 0 0) 0x7f2b840b4830 con 0x7f2b8415fef0 2024-04-25T11:20:49.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.077+0000 7f2b82a00640 1 --2- 172.21.15.7:0/1465659595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8415c2e0 0x7f2b8415c6f0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f2b700097a0 tx=0x7f2b7002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6643c0d83b81346 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:49.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.077+0000 7f2b82000640 1 -- 172.21.15.7:0/1465659595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70037070 con 0x7f2b8415c2e0 2024-04-25T11:20:49.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.077+0000 7f2b82000640 1 -- 172.21.15.7:0/1465659595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b70032440 con 0x7f2b8415c2e0 2024-04-25T11:20:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.077+0000 7f2b82000640 1 -- 172.21.15.7:0/1465659595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70032810 con 0x7f2b8415c2e0 2024-04-25T11:20:49.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.078+0000 7f2b89000640 1 -- 172.21.15.7:0/1465659595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8415c2e0 msgr2=0x7f2b8415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.078+0000 7f2b89000640 1 --2- 172.21.15.7:0/1465659595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8415c2e0 0x7f2b8415c6f0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f2b700097a0 tx=0x7f2b7002a180 comp rx=0 tx=0).stop 2024-04-25T11:20:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.078+0000 7f2b89000640 1 -- 172.21.15.7:0/1465659595 shutdown_connections 2024-04-25T11:20:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.078+0000 7f2b89000640 1 --2- 172.21.15.7:0/1465659595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b8415c2e0 0x7f2b8415c6f0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.078+0000 7f2b89000640 1 --2- 172.21.15.7:0/1465659595 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b840ce8d0 0x7f2b840cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.078+0000 7f2b89000640 1 -- 172.21.15.7:0/1465659595 >> 172.21.15.7:0/1465659595 conn(0x7f2b840740f0 msgr2=0x7f2b8406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.078+0000 7f2b89000640 1 -- 172.21.15.7:0/1465659595 shutdown_connections 2024-04-25T11:20:49.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.078+0000 7f2b89000640 1 -- 172.21.15.7:0/1465659595 wait complete. 2024-04-25T11:20:49.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.080+0000 7f2b89000640 1 Processor -- start 2024-04-25T11:20:49.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.081+0000 7f2b89000640 1 -- start start 2024-04-25T11:20:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.082+0000 7f2b89000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b840ce8d0 0x7f2b84155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.082+0000 7f2b89000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b8415c2e0 0x7f2b8413f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.082+0000 7f2b89000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b8415fef0 0x7f2b8413cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.082+0000 7f2b89000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2b84070030 con 0x7f2b840ce8d0 2024-04-25T11:20:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.082+0000 7f2b89000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8406feb0 con 0x7f2b8415c2e0 2024-04-25T11:20:49.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.082+0000 7f2b89000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b84160a10 con 0x7f2b8415fef0 2024-04-25T11:20:49.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b83400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b840ce8d0 0x7f2b84155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b83400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b840ce8d0 0x7f2b84155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37054/0 (socket says 172.21.15.7:37054) 2024-04-25T11:20:49.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b83400640 1 -- 172.21.15.7:0/2897501589 learned_addr learned my addr 172.21.15.7:0/2897501589 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:49.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b82a00640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b8415c2e0 0x7f2b8413f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b83e00640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b8415fef0 0x7f2b8413cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b83400640 1 -- 172.21.15.7:0/2897501589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b8415fef0 msgr2=0x7f2b8413cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b83400640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b8415fef0 0x7f2b8413cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b83400640 1 -- 172.21.15.7:0/2897501589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b8415c2e0 msgr2=0x7f2b8413f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b83400640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b8415c2e0 0x7f2b8413f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b83400640 1 -- 172.21.15.7:0/2897501589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8413e040 con 0x7f2b840ce8d0 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b82a00640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b8415c2e0 0x7f2b8413f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.083+0000 7f2b83400640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b840ce8d0 0x7f2b84155dd0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400d600 tx=0x7f2b7400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.084+0000 7f2b7b400640 1 -- 172.21.15.7:0/2897501589 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74004240 con 0x7f2b840ce8d0 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.084+0000 7f2b7b400640 1 -- 172.21.15.7:0/2897501589 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b740043e0 con 0x7f2b840ce8d0 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.084+0000 7f2b7b400640 1 -- 172.21.15.7:0/2897501589 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74003830 con 0x7f2b840ce8d0 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.084+0000 7f2b89000640 1 -- 172.21.15.7:0/2897501589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8413e320 con 0x7f2b840ce8d0 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.084+0000 7f2b89000640 1 -- 172.21.15.7:0/2897501589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8413e8e0 con 0x7f2b840ce8d0 2024-04-25T11:20:49.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.085+0000 7f2b89000640 1 -- 172.21.15.7:0/2897501589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b84050740 con 0x7f2b840ce8d0 2024-04-25T11:20:49.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.086+0000 7f2b7b400640 1 -- 172.21.15.7:0/2897501589 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2b740039d0 con 0x7f2b840ce8d0 2024-04-25T11:20:49.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.087+0000 7f2b7b400640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b640744e0 0x7f2b640769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.088+0000 7f2b7b400640 1 -- 172.21.15.7:0/2897501589 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f2b74095640 con 0x7f2b840ce8d0 2024-04-25T11:20:49.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.088+0000 7f2b82a00640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b640744e0 0x7f2b640769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.089+0000 7f2b82a00640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b640744e0 0x7f2b640769f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2b70034000 tx=0x7f2b7002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:49.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.093+0000 7f2b7b400640 1 -- 172.21.15.7:0/2897501589 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2b7405fc30 con 0x7f2b840ce8d0 2024-04-25T11:20:49.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.371+0000 7f2b89000640 1 -- 172.21.15.7:0/2897501589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2b840709b0 con 0x7f2b840ce8d0 2024-04-25T11:20:49.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.376+0000 7f2b7b400640 1 -- 172.21.15.7:0/2897501589 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f2b7405f530 con 0x7f2b840ce8d0 2024-04-25T11:20:49.372 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-25T11:20:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:20:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-04-25T11:20:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-04-25T11:20:49.373 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:20:49.373 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-04-25T11:20:49.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.378+0000 7f2b89000640 1 -- 172.21.15.7:0/2897501589 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b640744e0 msgr2=0x7f2b640769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:49.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.378+0000 7f2b89000640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b640744e0 0x7f2b640769f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2b70034000 tx=0x7f2b7002e000 comp rx=0 tx=0).stop 2024-04-25T11:20:49.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.378+0000 7f2b89000640 1 -- 172.21.15.7:0/2897501589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b840ce8d0 msgr2=0x7f2b84155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.378+0000 7f2b89000640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b840ce8d0 0x7f2b84155dd0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400d600 tx=0x7f2b7400dac0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.379+0000 7f2b89000640 1 -- 172.21.15.7:0/2897501589 shutdown_connections 2024-04-25T11:20:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.379+0000 7f2b89000640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2b640744e0 0x7f2b640769f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.379+0000 7f2b89000640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b8415fef0 0x7f2b8413cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.379+0000 7f2b89000640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b8415c2e0 0x7f2b8413f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.379+0000 7f2b89000640 1 --2- 172.21.15.7:0/2897501589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b840ce8d0 0x7f2b84155dd0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.379+0000 7f2b89000640 1 -- 172.21.15.7:0/2897501589 >> 172.21.15.7:0/2897501589 conn(0x7f2b840740f0 msgr2=0x7f2b8406c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.379+0000 7f2b89000640 1 -- 172.21.15.7:0/2897501589 shutdown_connections 2024-04-25T11:20:49.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.379+0000 7f2b89000640 1 -- 172.21.15.7:0/2897501589 wait complete. 2024-04-25T11:20:49.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.686+0000 7fc0abe00640 1 Processor -- start 2024-04-25T11:20:49.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.686+0000 7fc0abe00640 1 -- start start 2024-04-25T11:20:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.688+0000 7fc0abe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc0ac0aed40 0x7fc0ac15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.688+0000 7fc0abe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0b5220 0x7fc0ac0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.688+0000 7fc0abe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0ac0af420 0x7fc0ac0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.688+0000 7fc0abe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ac15e1f0 con 0x7fc0ac0aed40 2024-04-25T11:20:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.688+0000 7fc0abe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ac0afd70 con 0x7fc0ac0af420 2024-04-25T11:20:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.688+0000 7fc0abe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ac0b64b0 con 0x7fc0ac0b5220 2024-04-25T11:20:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0aaa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc0ac0aed40 0x7fc0ac15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0aaa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc0ac0aed40 0x7fc0ac15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37062/0 (socket says 172.21.15.7:37062) 2024-04-25T11:20:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0aaa00640 1 -- 172.21.15.7:0/3979145984 learned_addr learned my addr 172.21.15.7:0/3979145984 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0aa000640 1 --2- 172.21.15.7:0/3979145984 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0b5220 0x7fc0ac0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0aaa00640 1 -- 172.21.15.7:0/3979145984 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0b5220 msgr2=0x7fc0ac0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0ab400640 1 --2- 172.21.15.7:0/3979145984 >> v2:172.21.15.87:3300/0 conn(0x7fc0ac0af420 0x7fc0ac0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0aaa00640 1 --2- 172.21.15.7:0/3979145984 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0b5220 0x7fc0ac0ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0aaa00640 1 -- 172.21.15.7:0/3979145984 >> v2:172.21.15.87:3300/0 conn(0x7fc0ac0af420 msgr2=0x7fc0ac0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0aaa00640 1 --2- 172.21.15.7:0/3979145984 >> v2:172.21.15.87:3300/0 conn(0x7fc0ac0af420 0x7fc0ac0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0aaa00640 1 -- 172.21.15.7:0/3979145984 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac06c610 con 0x7fc0ac0aed40 2024-04-25T11:20:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0aa000640 1 --2- 172.21.15.7:0/3979145984 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0b5220 0x7fc0ac0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.689+0000 7fc0ab400640 1 --2- 172.21.15.7:0/3979145984 >> v2:172.21.15.87:3300/0 conn(0x7fc0ac0af420 0x7fc0ac0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:49.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.690+0000 7fc0aaa00640 1 --2- 172.21.15.7:0/3979145984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0aed40 0x7fc0ac15dcb0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac073080 tx=0x7fc09c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c23c53fd072ec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.691+0000 7fc0a9600640 1 -- 172.21.15.7:0/3979145984 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c037070 con 0x7fc0ac0aed40 2024-04-25T11:20:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.691+0000 7fc0a9600640 1 -- 172.21.15.7:0/3979145984 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc09c032440 con 0x7fc0ac0aed40 2024-04-25T11:20:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.691+0000 7fc0a9600640 1 -- 172.21.15.7:0/3979145984 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c032810 con 0x7fc0ac0aed40 2024-04-25T11:20:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.692+0000 7fc0abe00640 1 -- 172.21.15.7:0/3979145984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0aed40 msgr2=0x7fc0ac15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.692+0000 7fc0abe00640 1 --2- 172.21.15.7:0/3979145984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0aed40 0x7fc0ac15dcb0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac073080 tx=0x7fc09c02a1d0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.692+0000 7fc0abe00640 1 -- 172.21.15.7:0/3979145984 shutdown_connections 2024-04-25T11:20:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.692+0000 7fc0abe00640 1 --2- 172.21.15.7:0/3979145984 >> v2:172.21.15.87:3300/0 conn(0x7fc0ac0af420 0x7fc0ac0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.692+0000 7fc0abe00640 1 --2- 172.21.15.7:0/3979145984 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0b5220 0x7fc0ac0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.692+0000 7fc0abe00640 1 --2- 172.21.15.7:0/3979145984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0aed40 0x7fc0ac15dcb0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.692+0000 7fc0abe00640 1 -- 172.21.15.7:0/3979145984 >> 172.21.15.7:0/3979145984 conn(0x7fc0ac0b3820 msgr2=0x7fc0ac0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.692+0000 7fc0abe00640 1 -- 172.21.15.7:0/3979145984 shutdown_connections 2024-04-25T11:20:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.692+0000 7fc0abe00640 1 -- 172.21.15.7:0/3979145984 wait complete. 2024-04-25T11:20:49.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.694+0000 7fc0abe00640 1 Processor -- start 2024-04-25T11:20:49.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.694+0000 7fc0abe00640 1 -- start start 2024-04-25T11:20:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.696+0000 7fc0abe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0aed40 0x7fc0ac156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.696+0000 7fc0abe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0ac0af420 0x7fc0ac15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.696+0000 7fc0abe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0b5220 0x7fc0ac15ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.696+0000 7fc0abe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac06ceb0 con 0x7fc0ac0b5220 2024-04-25T11:20:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.696+0000 7fc0abe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac06cbd0 con 0x7fc0ac0af420 2024-04-25T11:20:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.696+0000 7fc0abe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ac0b6110 con 0x7fc0ac0aed40 2024-04-25T11:20:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.696+0000 7fc0ab400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0b5220 0x7fc0ac15ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.696+0000 7fc0ab400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0b5220 0x7fc0ac15ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37068/0 (socket says 172.21.15.7:37068) 2024-04-25T11:20:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.696+0000 7fc0ab400640 1 -- 172.21.15.7:0/2173586908 learned_addr learned my addr 172.21.15.7:0/2173586908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:49.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.697+0000 7fc0aaa00640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0aed40 0x7fc0ac156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.697+0000 7fc0aa000640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0ac0af420 0x7fc0ac15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.697+0000 7fc0ab400640 1 -- 172.21.15.7:0/2173586908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0aed40 msgr2=0x7fc0ac156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:49.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.697+0000 7fc0ab400640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0aed40 0x7fc0ac156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.697+0000 7fc0ab400640 1 -- 172.21.15.7:0/2173586908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0ac0af420 msgr2=0x7fc0ac15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:49.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.697+0000 7fc0ab400640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0ac0af420 0x7fc0ac15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.697+0000 7fc0ab400640 1 -- 172.21.15.7:0/2173586908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac142c10 con 0x7fc0ac0b5220 2024-04-25T11:20:49.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.697+0000 7fc0aaa00640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0aed40 0x7fc0ac156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:49.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.697+0000 7fc0ab400640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0b5220 0x7fc0ac15ca40 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400cca0 tx=0x7fc0a400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:49.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.698+0000 7fc0a2a00640 1 -- 172.21.15.7:0/2173586908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a400f680 con 0x7fc0ac0b5220 2024-04-25T11:20:49.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.698+0000 7fc0a2a00640 1 -- 172.21.15.7:0/2173586908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a400f840 con 0x7fc0ac0b5220 2024-04-25T11:20:49.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.698+0000 7fc0a2a00640 1 -- 172.21.15.7:0/2173586908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a400fc30 con 0x7fc0ac0b5220 2024-04-25T11:20:49.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.698+0000 7fc0abe00640 1 -- 172.21.15.7:0/2173586908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac142ef0 con 0x7fc0ac0b5220 2024-04-25T11:20:49.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.698+0000 7fc0abe00640 1 -- 172.21.15.7:0/2173586908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac1434b0 con 0x7fc0ac0b5220 2024-04-25T11:20:49.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.699+0000 7fc0abe00640 1 -- 172.21.15.7:0/2173586908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ac142980 con 0x7fc0ac0b5220 2024-04-25T11:20:49.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.700+0000 7fc0a2a00640 1 -- 172.21.15.7:0/2173586908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc0a400e3e0 con 0x7fc0ac0b5220 2024-04-25T11:20:49.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.701+0000 7fc0a2a00640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc08c0744b0 0x7fc08c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:49.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.701+0000 7fc0a2a00640 1 -- 172.21.15.7:0/2173586908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7fc0a40960f0 con 0x7fc0ac0b5220 2024-04-25T11:20:49.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.702+0000 7fc0aaa00640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc08c0744b0 0x7fc08c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:49.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.702+0000 7fc0aaa00640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc08c0744b0 0x7fc08c0769c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc09c034000 tx=0x7fc09c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.706+0000 7fc0a2a00640 1 -- 172.21.15.7:0/2173586908 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0a4014030 con 0x7fc0ac0b5220 2024-04-25T11:20:49.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.948+0000 7fc0abe00640 1 -- 172.21.15.7:0/2173586908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0ac050740 con 0x7fc0ac0b5220 2024-04-25T11:20:49.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.949+0000 7fc0a2a00640 1 -- 172.21.15.7:0/2173586908 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5174 (secure 0 0 0) 0x7fc0a40606e0 con 0x7fc0ac0b5220 2024-04-25T11:20:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:49.946 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2288158899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2288158899},{"type":"v1","addr":"172.21.15.7:6837","nonce":2288158899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3545919245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3545919245},{"type":"v1","addr":"172.21.15.87:6835","nonce":3545919245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8478,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/951407928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":951407928},{"type":"v1","addr":"172.21.15.7:6836","nonce":951407928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:42.908797+0000","modified":"2024-04-25T11:20:46.008582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6833/1243096872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1243096872},{"type":"v1","addr":"172.21.15.87:6833","nonce":1243096872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8484,"qdb_cluster":[8484]},"id":9}]} 2024-04-25T11:20:49.946 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 89 2024-04-25T11:20:49.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 -- 172.21.15.7:0/2173586908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc08c0744b0 msgr2=0x7fc08c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:49.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc08c0744b0 0x7fc08c0769c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc09c034000 tx=0x7fc09c02e000 comp rx=0 tx=0).stop 2024-04-25T11:20:49.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 -- 172.21.15.7:0/2173586908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0b5220 msgr2=0x7fc0ac15ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:49.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0b5220 0x7fc0ac15ca40 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fc0a400cca0 tx=0x7fc0a400a650 comp rx=0 tx=0).stop 2024-04-25T11:20:49.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 -- 172.21.15.7:0/2173586908 shutdown_connections 2024-04-25T11:20:49.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc08c0744b0 0x7fc08c0769c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0b5220 0x7fc0ac15ca40 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0ac0af420 0x7fc0ac15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 --2- 172.21.15.7:0/2173586908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ac0aed40 0x7fc0ac156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:49.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 -- 172.21.15.7:0/2173586908 >> 172.21.15.7:0/2173586908 conn(0x7fc0ac0b3820 msgr2=0x7fc0ac0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:49.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 -- 172.21.15.7:0/2173586908 shutdown_connections 2024-04-25T11:20:49.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:49.952+0000 7fc0abe00640 1 -- 172.21.15.7:0/2173586908 wait complete. 2024-04-25T11:20:50.377 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:20:42.908797+0000', 'modified': '2024-04-25T11:20:46.008582+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8484}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8484': {'gid': 8484, 'name': 'b', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6833/1243096872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6832', 'nonce': 1243096872}, {'type': 'v1', 'addr': '172.21.15.87:6833', 'nonce': 1243096872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8484, 'qdb_cluster': [8484]} 2024-04-25T11:20:50.377 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:20:50.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T11:20:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.679+0000 7f8fa2200640 1 Processor -- start 2024-04-25T11:20:50.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.679+0000 7f8fa2200640 1 -- start start 2024-04-25T11:20:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.681+0000 7f8fa2200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8f9c0d04c0 0x7f8f9c163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.681+0000 7f8fa2200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f9c0b7430 0x7f8f9c0b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.681+0000 7f8fa2200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f9c0b7d80 con 0x7f8f9c0bae70 2024-04-25T11:20:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.681+0000 7f8fa2200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8f9c164980 con 0x7f8f9c0b7430 2024-04-25T11:20:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.681+0000 7f8fa2200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9c1640e0 con 0x7f8f9c0d04c0 2024-04-25T11:20:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.681+0000 7f8fa1800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f9c0b7430 0x7f8f9c0b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.682+0000 7f8fa1800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f9c0b7430 0x7f8f9c0b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56578/0 (socket says 172.21.15.7:56578) 2024-04-25T11:20:50.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.682+0000 7f8fa1800640 1 -- 172.21.15.7:0/430146663 learned_addr learned my addr 172.21.15.7:0/430146663 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.682+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/430146663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8f9c0d04c0 0x7f8f9c163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.682+0000 7f8fa1800640 1 -- 172.21.15.7:0/430146663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8f9c0d04c0 msgr2=0x7f8f9c163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.682+0000 7f8fa1800640 1 --2- 172.21.15.7:0/430146663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8f9c0d04c0 0x7f8f9c163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.682+0000 7f8fa1800640 1 -- 172.21.15.7:0/430146663 >> v1:172.21.15.87:6789/0 conn(0x7f8f9c0bae70 legacy=0x7f8f9c0b8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.682+0000 7f8fa1800640 1 -- 172.21.15.7:0/430146663 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c1643c0 con 0x7f8f9c0b7430 2024-04-25T11:20:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.683+0000 7f8fa1800640 1 --2- 172.21.15.7:0/430146663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0b7430 0x7f8f9c0b7840 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009e50 tx=0x7f8f9402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84248de4672f02fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.683+0000 7f8f9b400640 1 -- 172.21.15.7:0/430146663 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94036070 con 0x7f8f9c0b7430 2024-04-25T11:20:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.683+0000 7f8f9b400640 1 -- 172.21.15.7:0/430146663 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f94031600 con 0x7f8f9c0b7430 2024-04-25T11:20:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.684+0000 7f8f9b400640 1 -- 172.21.15.7:0/430146663 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f940319d0 con 0x7f8f9c0b7430 2024-04-25T11:20:50.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.684+0000 7f8fa2200640 1 -- 172.21.15.7:0/430146663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0b7430 msgr2=0x7f8f9c0b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:50.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.684+0000 7f8fa2200640 1 --2- 172.21.15.7:0/430146663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0b7430 0x7f8f9c0b7840 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009e50 tx=0x7f8f9402a630 comp rx=0 tx=0).stop 2024-04-25T11:20:50.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.684+0000 7f8fa2200640 1 -- 172.21.15.7:0/430146663 shutdown_connections 2024-04-25T11:20:50.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.684+0000 7f8fa2200640 1 --2- 172.21.15.7:0/430146663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0b7430 0x7f8f9c0b7840 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.684+0000 7f8fa2200640 1 --2- 172.21.15.7:0/430146663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8f9c0d04c0 0x7f8f9c163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.684+0000 7f8fa2200640 1 -- 172.21.15.7:0/430146663 >> 172.21.15.7:0/430146663 conn(0x7f8f9c0d2610 msgr2=0x7f8f9c0d2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:50.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.684+0000 7f8fa2200640 1 -- 172.21.15.7:0/430146663 shutdown_connections 2024-04-25T11:20:50.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.684+0000 7f8fa2200640 1 -- 172.21.15.7:0/430146663 wait complete. 2024-04-25T11:20:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.687+0000 7f8fa2200640 1 Processor -- start 2024-04-25T11:20:50.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.687+0000 7f8fa2200640 1 -- start start 2024-04-25T11:20:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.688+0000 7f8fa2200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0b7430 0x7f8f9c0ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.688+0000 7f8fa2200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f9c0bae70 0x7f8f9c1f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.688+0000 7f8fa2200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8f9c0d04c0 0x7f8f9c1f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.688+0000 7f8fa2200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c0b51c0 con 0x7f8f9c0b7430 2024-04-25T11:20:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.688+0000 7f8fa2200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c0d4210 con 0x7f8f9c0bae70 2024-04-25T11:20:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.688+0000 7f8fa2200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9c0bb280 con 0x7f8f9c0d04c0 2024-04-25T11:20:50.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8fa0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0b7430 0x7f8f9c0ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8fa0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0b7430 0x7f8f9c0ca400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56588/0 (socket says 172.21.15.7:56588) 2024-04-25T11:20:50.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8fa0e00640 1 -- 172.21.15.7:0/234519752 learned_addr learned my addr 172.21.15.7:0/234519752 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8fa1800640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8f9c0d04c0 0x7f8f9c1f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8f9be00640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f9c0bae70 0x7f8f9c1f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8fa0e00640 1 -- 172.21.15.7:0/234519752 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8f9c0d04c0 msgr2=0x7f8f9c1f9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8f9c0d04c0 0x7f8f9c1f9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8fa0e00640 1 -- 172.21.15.7:0/234519752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f9c0bae70 msgr2=0x7f8f9c1f9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f9c0bae70 0x7f8f9c1f9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8fa0e00640 1 -- 172.21.15.7:0/234519752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c0cad10 con 0x7f8f9c0b7430 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8f9be00640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f9c0bae70 0x7f8f9c1f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.689+0000 7f8fa1800640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8f9c0d04c0 0x7f8f9c1f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.690+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0b7430 0x7f8f9c0ca400 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00d5f0 tx=0x7f8f8c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.690+0000 7f8f98c00640 1 -- 172.21.15.7:0/234519752 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c0125f0 con 0x7f8f9c0b7430 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.690+0000 7f8f98c00640 1 -- 172.21.15.7:0/234519752 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c019070 con 0x7f8f9c0b7430 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.690+0000 7f8f98c00640 1 -- 172.21.15.7:0/234519752 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c0129d0 con 0x7f8f9c0b7430 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.690+0000 7f8fa2200640 1 -- 172.21.15.7:0/234519752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9c0caff0 con 0x7f8f9c0b7430 2024-04-25T11:20:50.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.690+0000 7f8fa2200640 1 -- 172.21.15.7:0/234519752 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9c1f7770 con 0x7f8f9c0b7430 2024-04-25T11:20:50.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.693+0000 7f8f98c00640 1 -- 172.21.15.7:0/234519752 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8f8c012b70 con 0x7f8f9c0b7430 2024-04-25T11:20:50.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.694+0000 7f8f98c00640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8f7c0744e0 0x7f8f7c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.694+0000 7f8f98c00640 1 -- 172.21.15.7:0/234519752 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f8f8c0957e0 con 0x7f8f9c0b7430 2024-04-25T11:20:50.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.694+0000 7f8f9be00640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8f7c0744e0 0x7f8f7c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.694+0000 7f8faa818740 1 Processor -- start 2024-04-25T11:20:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.695+0000 7f8faa818740 1 -- start start 2024-04-25T11:20:50.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.695+0000 7f8f9be00640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8f7c0744e0 0x7f8f7c0769f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0b83f0 tx=0x7f8f88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.696+0000 7f8faa818740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0664f270 0x55be06a1c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.696+0000 7f8faa818740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a36f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.696+0000 7f8faa818740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be0662e590 0x55be06a37480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.696+0000 7f8faa818740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55be06955600 con 0x55be066263b0 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.696+0000 7f8faa818740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55be068f2100 con 0x55be0664f270 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.696+0000 7f8faa818740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55be069515c0 con 0x55be0662e590 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.696+0000 7f8f9be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a36f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.696+0000 7f8f9be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a36f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56604/0 (socket says 172.21.15.7:56604) 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.696+0000 7f8f9be00640 1 -- 172.21.15.7:0/2682333706 learned_addr learned my addr 172.21.15.7:0/2682333706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.697+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/2682333706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0664f270 0x55be06a1c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.697+0000 7f8fa1800640 1 --2- 172.21.15.7:0/2682333706 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be0662e590 0x55be06a37480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.697+0000 7f8f9be00640 1 -- 172.21.15.7:0/2682333706 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be0662e590 msgr2=0x55be06a37480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.697+0000 7f8f9be00640 1 --2- 172.21.15.7:0/2682333706 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be0662e590 0x55be06a37480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.697+0000 7f8f9be00640 1 -- 172.21.15.7:0/2682333706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0664f270 msgr2=0x55be06a1c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:50.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.697+0000 7f8f9be00640 1 --2- 172.21.15.7:0/2682333706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0664f270 0x55be06a1c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.697+0000 7f8f9be00640 1 -- 172.21.15.7:0/2682333706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be066672d0 con 0x55be066263b0 2024-04-25T11:20:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.697+0000 7f8f9be00640 1 --2- 172.21.15.7:0/2682333706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a36f40 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800ca60 tx=0x7f8f8800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137a1e451ecbace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:50.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.697+0000 7f8f91600640 1 -- 172.21.15.7:0/2682333706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88013db0 con 0x55be066263b0 2024-04-25T11:20:50.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.698+0000 7f8f91600640 1 -- 172.21.15.7:0/2682333706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f88004020 con 0x55be066263b0 2024-04-25T11:20:50.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.698+0000 7f8f91600640 1 -- 172.21.15.7:0/2682333706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f880043f0 con 0x55be066263b0 2024-04-25T11:20:50.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.698+0000 7f8faa818740 1 -- 172.21.15.7:0/2682333706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 msgr2=0x55be06a36f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:50.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.698+0000 7f8faa818740 1 --2- 172.21.15.7:0/2682333706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a36f40 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800ca60 tx=0x7f8f8800f000 comp rx=0 tx=0).stop 2024-04-25T11:20:50.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.698+0000 7f8faa818740 1 -- 172.21.15.7:0/2682333706 shutdown_connections 2024-04-25T11:20:50.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.698+0000 7f8faa818740 1 --2- 172.21.15.7:0/2682333706 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be0662e590 0x55be06a37480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.698+0000 7f8faa818740 1 --2- 172.21.15.7:0/2682333706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a36f40 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.698+0000 7f8faa818740 1 --2- 172.21.15.7:0/2682333706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0664f270 0x55be06a1c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.698+0000 7f8faa818740 1 -- 172.21.15.7:0/2682333706 >> 172.21.15.7:0/2682333706 conn(0x55be066433c0 msgr2=0x55be069f1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:50.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.699+0000 7f8faa818740 1 -- 172.21.15.7:0/2682333706 shutdown_connections 2024-04-25T11:20:50.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.699+0000 7f8faa818740 1 -- 172.21.15.7:0/2682333706 wait complete. 2024-04-25T11:20:50.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.700+0000 7f8faa818740 1 -- start start 2024-04-25T11:20:50.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.700+0000 7f8faa818740 1 Processor -- start 2024-04-25T11:20:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.701+0000 7f8faa818740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be066263b0 0x55be06a45f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.701+0000 7f8faa818740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0662e590 0x55be06a1f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.701+0000 7f8faa818740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.701+0000 7f8faa818740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55be068dcf70 con 0x55be0664f270 2024-04-25T11:20:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.701+0000 7f8faa818740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55be06a464c0 con 0x55be0662e590 2024-04-25T11:20:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.701+0000 7f8faa818740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55be06a46920 con 0x55be066263b0 2024-04-25T11:20:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.702+0000 7f8faa818740 10 client.0 _ll_register_callbacks cb 0x55be0694b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:20:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.702+0000 7f8fa1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.702+0000 7f8fa1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56618/0 (socket says 172.21.15.7:56618) 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.702+0000 7f8fa1800640 1 -- 172.21.15.7:0/1289938266 learned_addr learned my addr 172.21.15.7:0/1289938266 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.702+0000 7f8fa1800640 1 -- 172.21.15.7:0/1289938266 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be066263b0 msgr2=0x55be06a45f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.702+0000 7f8fa1800640 1 --2- 172.21.15.7:0/1289938266 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be066263b0 0x55be06a45f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.702+0000 7f8fa1800640 1 -- 172.21.15.7:0/1289938266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0662e590 msgr2=0x55be06a1f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.702+0000 7f8fa1800640 1 --2- 172.21.15.7:0/1289938266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0662e590 0x55be06a1f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.702+0000 7f8fa1800640 1 -- 172.21.15.7:0/1289938266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be067316b0 con 0x55be0664f270 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.703+0000 7f8fa1800640 1 --2- 172.21.15.7:0/1289938266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1fea0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009f70 tx=0x7f8f94004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc212737d8de5e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.703+0000 7f8f86000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.703+0000 7f8f86000640 1 -- 172.21.15.7:0/1289938266 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94042070 con 0x55be0664f270 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.703+0000 7f8f86000640 1 -- 172.21.15.7:0/1289938266 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f8f94031bd0 con 0x55be0664f270 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.703+0000 7f8faa818740 1 -- 172.21.15.7:0/1289938266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f88004020 con 0x55be0664f270 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.703+0000 7f8f86000640 1 -- 172.21.15.7:0/1289938266 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f8f9404c050 con 0x55be0664f270 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.703+0000 7f8faa818740 10 client.8616 fetch_fsmap learned FSMap version 89 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.703+0000 7f8faa818740 1 -- 172.21.15.7:0/1289938266 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55be06a17f90 con 0x55be0664f270 2024-04-25T11:20:50.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.703+0000 7f8f86000640 1 -- 172.21.15.7:0/1289938266 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f88004020 con 0x55be0664f270 2024-04-25T11:20:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.704+0000 7f8f86000640 1 -- 172.21.15.7:0/1289938266 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f8f9404f020 con 0x55be0664f270 2024-04-25T11:20:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.704+0000 7f8faa818740 10 client.8616 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T11:20:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.704+0000 7f8faa818740 10 client.8616 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:20:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.704+0000 7f8faa818740 20 client.8616 populate_metadata read hostname 'smithi007' 2024-04-25T11:20:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.704+0000 7f8faa818740 1 --2- 172.21.15.7:0/1289938266 >> [v2:172.21.15.87:6832/1243096872,v1:172.21.15.87:6833/1243096872] conn(0x55be0660de40 0x55be06a49a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.704+0000 7f8faa818740 4 client.8616 mds_command: new command op to 8484 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:20:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.704+0000 7f8faa818740 1 -- 172.21.15.7:0/1289938266 --> [v2:172.21.15.87:6832/1243096872,v1:172.21.15.87:6833/1243096872] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be06963210 con 0x55be0660de40 2024-04-25T11:20:50.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.705+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/1289938266 >> [v2:172.21.15.87:6832/1243096872,v1:172.21.15.87:6833/1243096872] conn(0x55be0660de40 0x55be06a49a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.705+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/1289938266 >> [v2:172.21.15.87:6832/1243096872,v1:172.21.15.87:6833/1243096872] conn(0x55be0660de40 0x55be06a49a70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce0094f9586a206b server_cookie=f23f1dfbd29a6977 in_seq=0 out_seq=0 2024-04-25T11:20:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.705+0000 7f8f86000640 10 client.8616 ms_handle_connect on v2:172.21.15.87:6832/1243096872 2024-04-25T11:20:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.707+0000 7f8f86000640 1 -- 172.21.15.7:0/1289938266 <== mds.0 v2:172.21.15.87:6832/1243096872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55be06963210 con 0x55be0660de40 2024-04-25T11:20:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.707+0000 7f8f86000640 10 client.8616 handle_command_reply: tid=0 2024-04-25T11:20:50.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.707+0000 7f8faa818740 1 client.8616 shutdown 2024-04-25T11:20:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.707+0000 7f8faa818740 1 -- 172.21.15.7:0/1289938266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 msgr2=0x55be06a1fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.707+0000 7f8faa818740 1 --2- 172.21.15.7:0/1289938266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1fea0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009f70 tx=0x7f8f94004ab0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.708+0000 7f8faa818740 1 -- 172.21.15.7:0/1289938266 shutdown_connections 2024-04-25T11:20:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.708+0000 7f8faa818740 1 --2- 172.21.15.7:0/1289938266 >> [v2:172.21.15.87:6832/1243096872,v1:172.21.15.87:6833/1243096872] conn(0x55be0660de40 0x55be06a49a70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.708+0000 7f8faa818740 1 --2- 172.21.15.7:0/1289938266 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1fea0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.708+0000 7f8faa818740 1 --2- 172.21.15.7:0/1289938266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0662e590 0x55be06a1f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.708+0000 7f8faa818740 1 --2- 172.21.15.7:0/1289938266 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be066263b0 0x55be06a45f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.708+0000 7f8f86000640 0 client.8616 ms_handle_reset on v2:172.21.15.87:6832/1243096872 2024-04-25T11:20:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.708+0000 7f8faa818740 1 -- 172.21.15.7:0/1289938266 >> 172.21.15.7:0/1289938266 conn(0x55be066433c0 msgr2=0x55be06a1c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.708+0000 7f8faa818740 1 -- 172.21.15.7:0/1289938266 shutdown_connections 2024-04-25T11:20:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.708+0000 7f8faa818740 1 -- 172.21.15.7:0/1289938266 wait complete. 2024-04-25T11:20:50.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.708+0000 7f8faa818740 20 client.8616 trim_cache size 0 max 16384 2024-04-25T11:20:50.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.729+0000 7f8faa818740 1 Processor -- start 2024-04-25T11:20:50.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.729+0000 7f8faa818740 1 -- start start 2024-04-25T11:20:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.730+0000 7f8faa818740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a38ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.730+0000 7f8faa818740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be0662e590 0x55be069f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.730+0000 7f8faa818740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0664f270 0x55be069f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.730+0000 7f8faa818740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c202190 con 0x55be066263b0 2024-04-25T11:20:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.730+0000 7f8faa818740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1f8a00 con 0x55be0664f270 2024-04-25T11:20:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.730+0000 7f8faa818740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9c201ce0 con 0x55be0662e590 2024-04-25T11:20:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.730+0000 7f8fa0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a38ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.730+0000 7f8fa0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a38ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56628/0 (socket says 172.21.15.7:56628) 2024-04-25T11:20:50.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.730+0000 7f8fa0e00640 1 -- 172.21.15.7:0/2322270610 learned_addr learned my addr 172.21.15.7:0/2322270610 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.731+0000 7f8fa0e00640 1 -- 172.21.15.7:0/2322270610 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be0662e590 msgr2=0x55be069f15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.731+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/2322270610 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be0662e590 0x55be069f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.731+0000 7f8fa0e00640 1 -- 172.21.15.7:0/2322270610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0664f270 msgr2=0x55be069f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.731+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/2322270610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0664f270 0x55be069f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.731+0000 7f8fa0e00640 1 -- 172.21.15.7:0/2322270610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be069f3bf0 con 0x55be066263b0 2024-04-25T11:20:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.731+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/2322270610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a38ac0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c05b6c0 tx=0x7f8f8c05b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7067488a57fb9314 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.731+0000 7f8f90c00640 1 -- 172.21.15.7:0/2322270610 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c091070 con 0x55be066263b0 2024-04-25T11:20:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.731+0000 7f8f90c00640 1 -- 172.21.15.7:0/2322270610 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c0639f0 con 0x55be066263b0 2024-04-25T11:20:50.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.731+0000 7f8f90c00640 1 -- 172.21.15.7:0/2322270610 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c0684a0 con 0x55be066263b0 2024-04-25T11:20:50.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.732+0000 7f8faa818740 1 -- 172.21.15.7:0/2322270610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 msgr2=0x55be06a38ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:50.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.732+0000 7f8faa818740 1 --2- 172.21.15.7:0/2322270610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a38ac0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c05b6c0 tx=0x7f8f8c05b630 comp rx=0 tx=0).stop 2024-04-25T11:20:50.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.732+0000 7f8faa818740 1 -- 172.21.15.7:0/2322270610 shutdown_connections 2024-04-25T11:20:50.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.732+0000 7f8faa818740 1 --2- 172.21.15.7:0/2322270610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0664f270 0x55be069f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.732+0000 7f8faa818740 1 --2- 172.21.15.7:0/2322270610 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be0662e590 0x55be069f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.732+0000 7f8faa818740 1 --2- 172.21.15.7:0/2322270610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be066263b0 0x55be06a38ac0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.732+0000 7f8faa818740 1 -- 172.21.15.7:0/2322270610 >> 172.21.15.7:0/2322270610 conn(0x55be066433c0 msgr2=0x55be06a1c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:50.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.732+0000 7f8faa818740 1 -- 172.21.15.7:0/2322270610 shutdown_connections 2024-04-25T11:20:50.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.732+0000 7f8faa818740 1 -- 172.21.15.7:0/2322270610 wait complete. 2024-04-25T11:20:50.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.733+0000 7f8faa818740 1 -- start start 2024-04-25T11:20:50.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.734+0000 7f8faa818740 1 Processor -- start 2024-04-25T11:20:50.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.735+0000 7f8faa818740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0662e590 0x55be06a1cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.735+0000 7f8faa818740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.735+0000 7f8faa818740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be06a9e1a0 0x55be06a1d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.735+0000 7f8faa818740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55be06a32b80 con 0x55be0664f270 2024-04-25T11:20:50.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.735+0000 7f8faa818740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55be06a326e0 con 0x55be0662e590 2024-04-25T11:20:50.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.735+0000 7f8faa818740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55be06a3ad00 con 0x55be06a9e1a0 2024-04-25T11:20:50.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.735+0000 7f8faa818740 10 client.0 _ll_register_callbacks cb 0x7f8f940317e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:20:50.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.735+0000 7f8f9be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.735+0000 7f8f9be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56638/0 (socket says 172.21.15.7:56638) 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.735+0000 7f8f9be00640 1 -- 172.21.15.7:0/4159789930 learned_addr learned my addr 172.21.15.7:0/4159789930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.736+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0662e590 0x55be06a1cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.736+0000 7f8f9be00640 1 -- 172.21.15.7:0/4159789930 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be06a9e1a0 msgr2=0x55be06a1d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.736+0000 7f8f9be00640 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be06a9e1a0 0x55be06a1d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.736+0000 7f8f9be00640 1 -- 172.21.15.7:0/4159789930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0662e590 msgr2=0x55be06a1cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.736+0000 7f8f9be00640 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0662e590 0x55be06a1cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.736+0000 7f8f9be00640 1 -- 172.21.15.7:0/4159789930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be06a33e00 con 0x55be0664f270 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.736+0000 7f8f9be00640 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1d4a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8f8801e000 tx=0x7f8f8800a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf24b9b5abbd363 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.736+0000 7f8f86000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8f86000640 1 -- 172.21.15.7:0/4159789930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88004240 con 0x55be0664f270 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8faa818740 1 -- 172.21.15.7:0/4159789930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f8c0639f0 con 0x55be0664f270 2024-04-25T11:20:50.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8f86000640 1 -- 172.21.15.7:0/4159789930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f8f880043e0 con 0x55be0664f270 2024-04-25T11:20:50.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8f86000640 1 -- 172.21.15.7:0/4159789930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f8f88036050 con 0x55be0664f270 2024-04-25T11:20:50.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8faa818740 10 client.8622 fetch_fsmap learned FSMap version 89 2024-04-25T11:20:50.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8faa818740 1 -- 172.21.15.7:0/4159789930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55be06a16970 con 0x55be0664f270 2024-04-25T11:20:50.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8f86000640 1 -- 172.21.15.7:0/4159789930 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c0639f0 con 0x55be0664f270 2024-04-25T11:20:50.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8f86000640 1 -- 172.21.15.7:0/4159789930 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f8f88039020 con 0x55be0664f270 2024-04-25T11:20:50.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8faa818740 10 client.8622 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T11:20:50.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8faa818740 10 client.8622 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T11:20:50.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8faa818740 20 client.8622 populate_metadata read hostname 'smithi007' 2024-04-25T11:20:50.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8faa818740 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.87:6832/1243096872,v1:172.21.15.87:6833/1243096872] conn(0x55be06a361a0 0x55be06a365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:50.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8faa818740 4 client.8622 mds_command: new command op to 8484 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:20:50.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.737+0000 7f8faa818740 1 -- 172.21.15.7:0/4159789930 --> [v2:172.21.15.87:6832/1243096872,v1:172.21.15.87:6833/1243096872] -- command(tid 0: {"prefix": "status"}) -- 0x55be0695d080 con 0x55be06a361a0 2024-04-25T11:20:50.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.738+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.87:6832/1243096872,v1:172.21.15.87:6833/1243096872] conn(0x55be06a361a0 0x55be06a365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:50.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.739+0000 7f8fa0e00640 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.87:6832/1243096872,v1:172.21.15.87:6833/1243096872] conn(0x55be06a361a0 0x55be06a365b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a02d39ad744596 server_cookie=f9c6e51cfabfe3ba in_seq=0 out_seq=0 2024-04-25T11:20:50.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.739+0000 7f8f86000640 10 client.8622 ms_handle_connect on v2:172.21.15.87:6832/1243096872 2024-04-25T11:20:50.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8f86000640 1 -- 172.21.15.7:0/4159789930 <== mds.0 v2:172.21.15.87:6832/1243096872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55be0695d080 con 0x55be06a361a0 2024-04-25T11:20:50.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8f86000640 10 client.8622 handle_command_reply: tid=0 2024-04-25T11:20:50.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 1 client.8622 shutdown 2024-04-25T11:20:50.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 1 -- 172.21.15.7:0/4159789930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 msgr2=0x55be06a1d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:50.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1d4a0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f8f8801e000 tx=0x7f8f8800a680 comp rx=0 tx=0).stop 2024-04-25T11:20:50.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 1 -- 172.21.15.7:0/4159789930 shutdown_connections 2024-04-25T11:20:50.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.87:6832/1243096872,v1:172.21.15.87:6833/1243096872] conn(0x55be06a361a0 0x55be06a365b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55be06a9e1a0 0x55be06a1d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55be0664f270 0x55be06a1d4a0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 1 --2- 172.21.15.7:0/4159789930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be0662e590 0x55be06a1cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8f86000640 0 client.8622 ms_handle_reset on v2:172.21.15.87:6832/1243096872 2024-04-25T11:20:50.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 1 -- 172.21.15.7:0/4159789930 >> 172.21.15.7:0/4159789930 conn(0x55be066433c0 msgr2=0x55be06a32d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:50.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 1 -- 172.21.15.7:0/4159789930 shutdown_connections 2024-04-25T11:20:50.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 1 -- 172.21.15.7:0/4159789930 wait complete. 2024-04-25T11:20:50.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.740+0000 7f8faa818740 20 client.8622 trim_cache size 0 max 16384 2024-04-25T11:20:50.739 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:20:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:20:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:20:50.739 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 8484, 2024-04-25T11:20:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:20:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:20:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:20:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.3427072860000004, 2024-04-25T11:20:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 89, 2024-04-25T11:20:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 130, 2024-04-25T11:20:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 130, 2024-04-25T11:20:50.740 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.496893623 2024-04-25T11:20:50.740 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:20:50.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 -- 172.21.15.7:0/234519752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8f7c0744e0 msgr2=0x7f8f7c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:50.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8f7c0744e0 0x7f8f7c0769f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0b83f0 tx=0x7f8f88008040 comp rx=0 tx=0).stop 2024-04-25T11:20:50.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 -- 172.21.15.7:0/234519752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0b7430 msgr2=0x7f8f9c0ca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:50.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0b7430 0x7f8f9c0ca400 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00d5f0 tx=0x7f8f8c00dab0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 -- 172.21.15.7:0/234519752 shutdown_connections 2024-04-25T11:20:50.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8f7c0744e0 0x7f8f7c0769f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8f9c0d04c0 0x7f8f9c1f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8f9c0bae70 0x7f8f9c1f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 --2- 172.21.15.7:0/234519752 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0b7430 0x7f8f9c0ca400 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:50.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 -- 172.21.15.7:0/234519752 >> 172.21.15.7:0/234519752 conn(0x7f8f9c0d2610 msgr2=0x7f8f9c0d2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:50.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 -- 172.21.15.7:0/234519752 shutdown_connections 2024-04-25T11:20:50.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:50.741+0000 7f8fa2200640 1 -- 172.21.15.7:0/234519752 wait complete. 2024-04-25T11:20:50.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:20:51.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.041+0000 7f85de800640 1 Processor -- start 2024-04-25T11:20:51.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.041+0000 7f85de800640 1 -- start start 2024-04-25T11:20:51.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.043+0000 7f85de800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f85d809b540 0x7f85d8161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.043+0000 7f85de800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85d8162000 0x7f85d806c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.044+0000 7f85de800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d80b2360 con 0x7f85d80b1d70 2024-04-25T11:20:51.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.044+0000 7f85de800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f85d806e130 con 0x7f85d8162000 2024-04-25T11:20:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.044+0000 7f85de800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f85d80b2900 con 0x7f85d809b540 2024-04-25T11:20:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.044+0000 7f85dd400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f85d80b1d70 0x7f85d806da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56432/0 (socket says 172.21.15.7:56432) 2024-04-25T11:20:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.044+0000 7f85dd400640 1 -- 172.21.15.7:0/3340826523 learned_addr learned my addr 172.21.15.7:0/3340826523 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.044+0000 7f85dde00640 1 --2- 172.21.15.7:0/3340826523 >> v2:172.21.15.7:3300/0 conn(0x7f85d8162000 0x7f85d806c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.044+0000 7f85dca00640 1 --2- 172.21.15.7:0/3340826523 >> v2:172.21.15.87:3300/0 conn(0x7f85d809b540 0x7f85d8161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.044+0000 7f85d7e00640 1 -- 172.21.15.7:0/3340826523 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2734052060 0 0) 0x7f85d80b2360 con 0x7f85d80b1d70 2024-04-25T11:20:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.044+0000 7f85d7e00640 1 -- 172.21.15.7:0/3340826523 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85c0003520 con 0x7f85d80b1d70 2024-04-25T11:20:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.045+0000 7f85dde00640 1 -- 172.21.15.7:0/3340826523 >> v2:172.21.15.87:3300/0 conn(0x7f85d809b540 msgr2=0x7f85d8161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.045+0000 7f85dde00640 1 --2- 172.21.15.7:0/3340826523 >> v2:172.21.15.87:3300/0 conn(0x7f85d809b540 0x7f85d8161ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.045+0000 7f85dde00640 1 -- 172.21.15.7:0/3340826523 >> v1:172.21.15.7:6789/0 conn(0x7f85d80b1d70 legacy=0x7f85d806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:51.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.045+0000 7f85dde00640 1 -- 172.21.15.7:0/3340826523 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d80b2620 con 0x7f85d8162000 2024-04-25T11:20:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.046+0000 7f85dde00640 1 --2- 172.21.15.7:0/3340826523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8162000 0x7f85d806c750 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f85d80b4f00 tx=0x7f85d002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47909fa7b1ac7cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.046+0000 7f85d7e00640 1 -- 172.21.15.7:0/3340826523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0007740 con 0x7f85d8162000 2024-04-25T11:20:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.046+0000 7f85d7e00640 1 -- 172.21.15.7:0/3340826523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d00078e0 con 0x7f85d8162000 2024-04-25T11:20:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.046+0000 7f85d7e00640 1 -- 172.21.15.7:0/3340826523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0007c90 con 0x7f85d8162000 2024-04-25T11:20:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.047+0000 7f85de800640 1 -- 172.21.15.7:0/3340826523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8162000 msgr2=0x7f85d806c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.047+0000 7f85de800640 1 --2- 172.21.15.7:0/3340826523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8162000 0x7f85d806c750 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f85d80b4f00 tx=0x7f85d002a680 comp rx=0 tx=0).stop 2024-04-25T11:20:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.047+0000 7f85de800640 1 -- 172.21.15.7:0/3340826523 shutdown_connections 2024-04-25T11:20:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.047+0000 7f85de800640 1 --2- 172.21.15.7:0/3340826523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d8162000 0x7f85d806c750 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.047+0000 7f85de800640 1 --2- 172.21.15.7:0/3340826523 >> v2:172.21.15.87:3300/0 conn(0x7f85d809b540 0x7f85d8161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.047+0000 7f85de800640 1 -- 172.21.15.7:0/3340826523 >> 172.21.15.7:0/3340826523 conn(0x7f85d8073b60 msgr2=0x7f85d80b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.047+0000 7f85de800640 1 -- 172.21.15.7:0/3340826523 shutdown_connections 2024-04-25T11:20:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.047+0000 7f85de800640 1 -- 172.21.15.7:0/3340826523 wait complete. 2024-04-25T11:20:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.050+0000 7f85de800640 1 Processor -- start 2024-04-25T11:20:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.050+0000 7f85de800640 1 -- start start 2024-04-25T11:20:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.051+0000 7f85de800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d809b540 0x7f85d809fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.051+0000 7f85de800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85d80b1d70 0x7f85d814c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.051+0000 7f85de800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85d8162000 0x7f85d809df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.051+0000 7f85de800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8162410 con 0x7f85d809b540 2024-04-25T11:20:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.051+0000 7f85de800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8161510 con 0x7f85d8162000 2024-04-25T11:20:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.051+0000 7f85de800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f85d809b950 con 0x7f85d80b1d70 2024-04-25T11:20:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.052+0000 7f85dd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d809b540 0x7f85d809fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.052+0000 7f85dd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d809b540 0x7f85d809fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56662/0 (socket says 172.21.15.7:56662) 2024-04-25T11:20:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.052+0000 7f85dd400640 1 -- 172.21.15.7:0/2666882037 learned_addr learned my addr 172.21.15.7:0/2666882037 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.052+0000 7f85dd400640 1 -- 172.21.15.7:0/2666882037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85d80b1d70 msgr2=0x7f85d814c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.052+0000 7f85dd400640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85d80b1d70 0x7f85d814c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.052+0000 7f85dd400640 1 -- 172.21.15.7:0/2666882037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85d8162000 msgr2=0x7f85d809df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.052+0000 7f85dde00640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85d8162000 0x7f85d809df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.052+0000 7f85dd400640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85d8162000 0x7f85d809df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.052+0000 7f85dd400640 1 -- 172.21.15.7:0/2666882037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d814cd50 con 0x7f85d809b540 2024-04-25T11:20:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.052+0000 7f85dd400640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d809b540 0x7f85d809fb80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00c3c0 tx=0x7f85cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.053+0000 7f85d5600640 1 -- 172.21.15.7:0/2666882037 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc017070 con 0x7f85d809b540 2024-04-25T11:20:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.053+0000 7f85d5600640 1 -- 172.21.15.7:0/2666882037 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85cc005ce0 con 0x7f85d809b540 2024-04-25T11:20:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.053+0000 7f85d5600640 1 -- 172.21.15.7:0/2666882037 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0072f0 con 0x7f85d809b540 2024-04-25T11:20:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.053+0000 7f85de800640 1 -- 172.21.15.7:0/2666882037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d814c810 con 0x7f85d809b540 2024-04-25T11:20:51.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.053+0000 7f85de800640 1 -- 172.21.15.7:0/2666882037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d809edc0 con 0x7f85d809b540 2024-04-25T11:20:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.054+0000 7f85de800640 1 -- 172.21.15.7:0/2666882037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8050740 con 0x7f85d809b540 2024-04-25T11:20:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.056+0000 7f85d5600640 1 -- 172.21.15.7:0/2666882037 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f85cc01a070 con 0x7f85d809b540 2024-04-25T11:20:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.057+0000 7f85d5600640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85b80744b0 0x7f85b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.057+0000 7f85d5600640 1 -- 172.21.15.7:0/2666882037 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f85cc094790 con 0x7f85d809b540 2024-04-25T11:20:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.057+0000 7f85dca00640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85b80744b0 0x7f85b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.058+0000 7f85dca00640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85b80744b0 0x7f85b80769c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f85c4006a90 tx=0x7f85c4006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:51.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.062+0000 7f85d5600640 1 -- 172.21.15.7:0/2666882037 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f85cc05ed90 con 0x7f85d809b540 2024-04-25T11:20:51.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.311+0000 7f85de800640 1 -- 172.21.15.7:0/2666882037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f85d815d7a0 con 0x7f85d809b540 2024-04-25T11:20:51.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.312+0000 7f85d5600640 1 -- 172.21.15.7:0/2666882037 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) ==== 53+0+73 (secure 0 0 0) 0x7f85cc05e690 con 0x7f85d809b540 2024-04-25T11:20:51.309 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:20:51.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 -- 172.21.15.7:0/2666882037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85b80744b0 msgr2=0x7f85b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:51.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85b80744b0 0x7f85b80769c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f85c4006a90 tx=0x7f85c4006a20 comp rx=0 tx=0).stop 2024-04-25T11:20:51.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 -- 172.21.15.7:0/2666882037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d809b540 msgr2=0x7f85d809fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:51.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d809b540 0x7f85d809fb80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00c3c0 tx=0x7f85cc00c880 comp rx=0 tx=0).stop 2024-04-25T11:20:51.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 -- 172.21.15.7:0/2666882037 shutdown_connections 2024-04-25T11:20:51.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f85b80744b0 0x7f85b80769c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85d8162000 0x7f85d809df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85d80b1d70 0x7f85d814c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 --2- 172.21.15.7:0/2666882037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d809b540 0x7f85d809fb80 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:51.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 -- 172.21.15.7:0/2666882037 >> 172.21.15.7:0/2666882037 conn(0x7f85d8073b60 msgr2=0x7f85d80d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:51.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 -- 172.21.15.7:0/2666882037 shutdown_connections 2024-04-25T11:20:51.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51.315+0000 7f85de800640 1 -- 172.21.15.7:0/2666882037 wait complete. 2024-04-25T11:20:51.338 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:20:51.339 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:20:51.339 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:20:51.339 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:20:51.339 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:51.339 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:20:51.339 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:20:51.339 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:20:51.339 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:20:51.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.340 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:20:51.393 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:20:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:20:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:20:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:20:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.394 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:20:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:20:51.394 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:20:51.394 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41923sec preferred_lft 41923sec 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:20:51.395 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6098:c7ff:fe8a:4a30/64 scope link 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::646a:35ff:fe7b:ad0/64 scope link 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4465:c4ff:fe62:f75f/64 scope link 2024-04-25T11:20:51.396 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:51.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.397 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:20:51.481 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:20:51.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.486 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:20:51.540 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:20:51.540 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:20:51.540 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:20:51.540 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:20:51.540 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:20:51.541 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:20:51.541 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:20:51.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:20:51.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:20:51.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:20:51.542 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:20:51.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:20:51.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:51.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.766 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:20:51.816 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:20:51.817 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:20:51.818 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:20:51.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:51.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:20:51.903 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:20:51.903 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:51.956 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:20:51.956 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:20:51.956 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:52.005 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:20:52.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:52.005 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:20:52.215 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:20:52.216 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:20:52.261 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:20:52.261 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:20:52.262 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:20:52.262 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:20:52.262 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:20:52.264 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:20:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:20:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:20:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:20:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:20:52.269 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:20:52.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:52.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:20:52.351 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:20:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:20:52.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:52.498 INFO:teuthology.orchestra.run.smithi007.stdout:client6302 2024-04-25T11:20:52.499 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:20:52.499 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:52.553 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:20:52.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.844+0000 7f0001800640 1 Processor -- start 2024-04-25T11:20:52.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.844+0000 7f0001800640 1 -- start start 2024-04-25T11:20:52.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.846+0000 7f0001800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efffc070ca0 0x7efffc0b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:52.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.846+0000 7f0001800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efffc0b65e0 0x7efffc06f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:52.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.846+0000 7f0001800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efffc160320 con 0x7efffc146310 2024-04-25T11:20:52.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.846+0000 7f0001800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efffc160600 con 0x7efffc070ca0 2024-04-25T11:20:52.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.846+0000 7f0001800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efffc1608e0 con 0x7efffc0b65e0 2024-04-25T11:20:52.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.847+0000 7efffaa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efffc146310 0x7efffc0d0d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56444/0 (socket says 172.21.15.7:56444) 2024-04-25T11:20:52.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.847+0000 7efffaa00640 1 -- 172.21.15.7:0/650405161 learned_addr learned my addr 172.21.15.7:0/650405161 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:52.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.847+0000 7efffa000640 1 --2- 172.21.15.7:0/650405161 >> v2:172.21.15.7:3300/0 conn(0x7efffc070ca0 0x7efffc0b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.847+0000 7efffb400640 1 --2- 172.21.15.7:0/650405161 >> v2:172.21.15.87:3300/0 conn(0x7efffc0b65e0 0x7efffc06f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.847+0000 7efff9600640 1 -- 172.21.15.7:0/650405161 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2895162574 0 0) 0x7efffc160320 con 0x7efffc146310 2024-04-25T11:20:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.847+0000 7efff9600640 1 -- 172.21.15.7:0/650405161 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effe0003520 con 0x7efffc146310 2024-04-25T11:20:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.847+0000 7efffa000640 1 -- 172.21.15.7:0/650405161 >> v2:172.21.15.87:3300/0 conn(0x7efffc0b65e0 msgr2=0x7efffc06f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:52.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.847+0000 7efffa000640 1 --2- 172.21.15.7:0/650405161 >> v2:172.21.15.87:3300/0 conn(0x7efffc0b65e0 0x7efffc06f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:52.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.847+0000 7efffa000640 1 -- 172.21.15.7:0/650405161 >> v1:172.21.15.7:6789/0 conn(0x7efffc146310 legacy=0x7efffc0d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:52.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.847+0000 7efffa000640 1 -- 172.21.15.7:0/650405161 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc160bc0 con 0x7efffc070ca0 2024-04-25T11:20:52.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.848+0000 7efffa000640 1 --2- 172.21.15.7:0/650405161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc070ca0 0x7efffc0b60a0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7efffc074bf0 tx=0x7effe802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5e8e5b8a315d1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:52.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.849+0000 7efff9600640 1 -- 172.21.15.7:0/650405161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8037070 con 0x7efffc070ca0 2024-04-25T11:20:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.849+0000 7efff9600640 1 -- 172.21.15.7:0/650405161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe8032440 con 0x7efffc070ca0 2024-04-25T11:20:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.849+0000 7efff9600640 1 -- 172.21.15.7:0/650405161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe80327f0 con 0x7efffc070ca0 2024-04-25T11:20:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.849+0000 7f0001800640 1 -- 172.21.15.7:0/650405161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc070ca0 msgr2=0x7efffc0b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.849+0000 7f0001800640 1 --2- 172.21.15.7:0/650405161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc070ca0 0x7efffc0b60a0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7efffc074bf0 tx=0x7effe802a230 comp rx=0 tx=0).stop 2024-04-25T11:20:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.850+0000 7f0001800640 1 -- 172.21.15.7:0/650405161 shutdown_connections 2024-04-25T11:20:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.850+0000 7f0001800640 1 --2- 172.21.15.7:0/650405161 >> v2:172.21.15.87:3300/0 conn(0x7efffc0b65e0 0x7efffc06f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.850+0000 7f0001800640 1 --2- 172.21.15.7:0/650405161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc070ca0 0x7efffc0b60a0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.850+0000 7f0001800640 1 -- 172.21.15.7:0/650405161 >> 172.21.15.7:0/650405161 conn(0x7efffc0732b0 msgr2=0x7efffc0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.850+0000 7f0001800640 1 -- 172.21.15.7:0/650405161 shutdown_connections 2024-04-25T11:20:52.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.850+0000 7f0001800640 1 -- 172.21.15.7:0/650405161 wait complete. 2024-04-25T11:20:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.852+0000 7f0001800640 1 Processor -- start 2024-04-25T11:20:52.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.852+0000 7f0001800640 1 -- start start 2024-04-25T11:20:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7f0001800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efffc070ca0 0x7efffc156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7f0001800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc0b65e0 0x7efffc13dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7f0001800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efffc146310 0x7efffc13e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7f0001800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efffc073f50 con 0x7efffc0b65e0 2024-04-25T11:20:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7f0001800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efffc073dd0 con 0x7efffc070ca0 2024-04-25T11:20:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7f0001800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efffc0740d0 con 0x7efffc146310 2024-04-25T11:20:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7efffa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc0b65e0 0x7efffc13dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7efffa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc0b65e0 0x7efffc13dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56684/0 (socket says 172.21.15.7:56684) 2024-04-25T11:20:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7efffa000640 1 -- 172.21.15.7:0/1505569708 learned_addr learned my addr 172.21.15.7:0/1505569708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7efffaa00640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efffc070ca0 0x7efffc156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:52.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7efffa000640 1 -- 172.21.15.7:0/1505569708 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efffc146310 msgr2=0x7efffc13e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7efffa000640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efffc146310 0x7efffc13e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7efffa000640 1 -- 172.21.15.7:0/1505569708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efffc070ca0 msgr2=0x7efffc156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7efffa000640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efffc070ca0 0x7efffc156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:52.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.854+0000 7efffa000640 1 -- 172.21.15.7:0/1505569708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc157a70 con 0x7efffc0b65e0 2024-04-25T11:20:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.855+0000 7efffa000640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc0b65e0 0x7efffc13dcf0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7effe802a270 tx=0x7effe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.855+0000 7efff2a00640 1 -- 172.21.15.7:0/1505569708 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8037040 con 0x7efffc0b65e0 2024-04-25T11:20:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.855+0000 7efff2a00640 1 -- 172.21.15.7:0/1505569708 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe80304c0 con 0x7efffc0b65e0 2024-04-25T11:20:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.855+0000 7efff2a00640 1 -- 172.21.15.7:0/1505569708 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8030890 con 0x7efffc0b65e0 2024-04-25T11:20:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.855+0000 7f0001800640 1 -- 172.21.15.7:0/1505569708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc157d50 con 0x7efffc0b65e0 2024-04-25T11:20:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.855+0000 7f0001800640 1 -- 172.21.15.7:0/1505569708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc158310 con 0x7efffc0b65e0 2024-04-25T11:20:52.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.857+0000 7f0001800640 1 -- 172.21.15.7:0/1505569708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efffc158750 con 0x7efffc0b65e0 2024-04-25T11:20:52.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.858+0000 7efff2a00640 1 -- 172.21.15.7:0/1505569708 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7effe8044020 con 0x7efffc0b65e0 2024-04-25T11:20:52.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.859+0000 7efff2a00640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effdc0744b0 0x7effdc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:52.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.859+0000 7efff2a00640 1 -- 172.21.15.7:0/1505569708 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7effe80b55e0 con 0x7efffc0b65e0 2024-04-25T11:20:52.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.859+0000 7efffaa00640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effdc0744b0 0x7effdc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:52.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.860+0000 7efffaa00640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effdc0744b0 0x7effdc0769c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7effec000f60 tx=0x7effec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:52.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:52.864+0000 7efff2a00640 1 -- 172.21.15.7:0/1505569708 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effe807fbd0 con 0x7efffc0b65e0 2024-04-25T11:20:53.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.117+0000 7f0001800640 1 -- 172.21.15.7:0/1505569708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efffc050740 con 0x7efffc0b65e0 2024-04-25T11:20:53.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.117+0000 7efff2a00640 1 -- 172.21.15.7:0/1505569708 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) ==== 53+0+73 (secure 0 0 0) 0x7effe807f4d0 con 0x7efffc0b65e0 2024-04-25T11:20:53.114 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:20:53.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.120+0000 7f0001800640 1 -- 172.21.15.7:0/1505569708 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effdc0744b0 msgr2=0x7effdc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:53.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.120+0000 7f0001800640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effdc0744b0 0x7effdc0769c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7effec000f60 tx=0x7effec009040 comp rx=0 tx=0).stop 2024-04-25T11:20:53.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.120+0000 7f0001800640 1 -- 172.21.15.7:0/1505569708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc0b65e0 msgr2=0x7efffc13dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:53.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.120+0000 7f0001800640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc0b65e0 0x7efffc13dcf0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7effe802a270 tx=0x7effe80046c0 comp rx=0 tx=0).stop 2024-04-25T11:20:53.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.120+0000 7f0001800640 1 -- 172.21.15.7:0/1505569708 shutdown_connections 2024-04-25T11:20:53.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.120+0000 7f0001800640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effdc0744b0 0x7effdc0769c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.120+0000 7f0001800640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efffc146310 0x7efffc13e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.120+0000 7f0001800640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efffc0b65e0 0x7efffc13dcf0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.120+0000 7f0001800640 1 --2- 172.21.15.7:0/1505569708 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efffc070ca0 0x7efffc156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.120+0000 7f0001800640 1 -- 172.21.15.7:0/1505569708 >> 172.21.15.7:0/1505569708 conn(0x7efffc0732b0 msgr2=0x7efffc0d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.121+0000 7f0001800640 1 -- 172.21.15.7:0/1505569708 shutdown_connections 2024-04-25T11:20:53.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:53.121+0000 7f0001800640 1 -- 172.21.15.7:0/1505569708 wait complete. 2024-04-25T11:20:53.148 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:20:53.148 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:20:53.148 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:20:53.148 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:20:53.148 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:20:53.148 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:20:53.148 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:20:53.148 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:20:53.148 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:20:53.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:53.149 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:20:53.208 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:20:53.208 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:20:53.208 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:20:53.208 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:53.208 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:20:53.208 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41922sec preferred_lft 41922sec 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:20:53.209 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::58f2:6bff:fe8b:8b4c/64 scope link 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::98fe:1bff:fe9f:d71c/64 scope link 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::3c38:a8ff:fec7:ea00/64 scope link 2024-04-25T11:20:53.210 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:20:53.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:53.211 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:20:53.290 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:20:53.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:53.294 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:20:53.344 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:20:53.344 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:20:53.344 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:20:53.344 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:20:53.344 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:20:53.345 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:20:53.345 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:20:53.345 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:20:53.345 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:20:53.345 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:20:53.345 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:20:53.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:20:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:20:53.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:20:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:53.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:53.551 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:20:53.600 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:20:53.601 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:20:53.601 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:20:53.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:53.601 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T11:20:53.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:20:53.677 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:20:53.729 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:20:53.729 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:20:53.729 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:20:53.780 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:20:53.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:53.780 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:20:53.980 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:20:53.980 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:20:54.022 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:20:54.022 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:20:54.022 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:20:54.022 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 1@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:20:54.022 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:20:54.023 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:20:54.024 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:20:54.024 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:20:54.024 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:20:54.024 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:20:54.028 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:20:54.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:20:54.029 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:20:54.092 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:20:54.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:20:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:20:54.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:20:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:54.200 INFO:teuthology.orchestra.run.smithi087.stdout:client6305 2024-04-25T11:20:54.200 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:20:54.200 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:54.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-04-25T11:20:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.553+0000 7f8096800640 1 Processor -- start 2024-04-25T11:20:54.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.553+0000 7f8096800640 1 -- start start 2024-04-25T11:20:54.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.556+0000 7f8096800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80901583c0 0x7f80900cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.556+0000 7f8096800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f80900cfb50 0x7f80900d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.556+0000 7f8096800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80900ce570 con 0x7f80900d09c0 2024-04-25T11:20:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.556+0000 7f8096800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f809015f190 con 0x7f80900cfb50 2024-04-25T11:20:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.556+0000 7f8096800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f80900ce850 con 0x7f80901583c0 2024-04-25T11:20:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.557+0000 7f8094a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f80900cfb50 0x7f80900d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.557+0000 7f8095400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80901583c0 0x7f80900cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.557+0000 7f8094a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f80900cfb50 0x7f80900d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:37976/0 (socket says 172.21.15.7:37976) 2024-04-25T11:20:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.557+0000 7f8094a00640 1 -- 172.21.15.7:0/1159594622 learned_addr learned my addr 172.21.15.7:0/1159594622 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.558+0000 7f8094a00640 1 -- 172.21.15.7:0/1159594622 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80901583c0 msgr2=0x7f80900cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.558+0000 7f8094a00640 1 --2- 172.21.15.7:0/1159594622 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80901583c0 0x7f80900cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.558+0000 7f8094a00640 1 -- 172.21.15.7:0/1159594622 >> v1:172.21.15.87:6789/0 conn(0x7f80900d09c0 legacy=0x7f809006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.558+0000 7f8094a00640 1 -- 172.21.15.7:0/1159594622 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80900ceb30 con 0x7f80900cfb50 2024-04-25T11:20:54.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.559+0000 7f8094a00640 1 --2- 172.21.15.7:0/1159594622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80900cfb50 0x7f80900d0480 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f807c009870 tx=0x7f807c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5548a460531e4379 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:54.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.560+0000 7f808fe00640 1 -- 172.21.15.7:0/1159594622 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c037070 con 0x7f80900cfb50 2024-04-25T11:20:54.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.560+0000 7f808fe00640 1 -- 172.21.15.7:0/1159594622 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f807c032440 con 0x7f80900cfb50 2024-04-25T11:20:54.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.560+0000 7f808fe00640 1 -- 172.21.15.7:0/1159594622 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c032810 con 0x7f80900cfb50 2024-04-25T11:20:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.560+0000 7f8096800640 1 -- 172.21.15.7:0/1159594622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80900cfb50 msgr2=0x7f80900d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.560+0000 7f8096800640 1 --2- 172.21.15.7:0/1159594622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80900cfb50 0x7f80900d0480 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f807c009870 tx=0x7f807c02a270 comp rx=0 tx=0).stop 2024-04-25T11:20:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.561+0000 7f8096800640 1 -- 172.21.15.7:0/1159594622 shutdown_connections 2024-04-25T11:20:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.561+0000 7f8096800640 1 --2- 172.21.15.7:0/1159594622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80900cfb50 0x7f80900d0480 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.561+0000 7f8096800640 1 --2- 172.21.15.7:0/1159594622 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80901583c0 0x7f80900cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.561+0000 7f8096800640 1 -- 172.21.15.7:0/1159594622 >> 172.21.15.7:0/1159594622 conn(0x7f80900740f0 msgr2=0x7f809006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.561+0000 7f8096800640 1 -- 172.21.15.7:0/1159594622 shutdown_connections 2024-04-25T11:20:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.561+0000 7f8096800640 1 -- 172.21.15.7:0/1159594622 wait complete. 2024-04-25T11:20:54.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.563+0000 7f8096800640 1 Processor -- start 2024-04-25T11:20:54.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.564+0000 7f8096800640 1 -- start start 2024-04-25T11:20:54.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.565+0000 7f8096800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80900cfb50 0x7f80901502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.565+0000 7f8096800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80900d09c0 0x7f80901544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.565+0000 7f8096800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80901583c0 0x7f809015cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.565+0000 7f8096800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f809006f5b0 con 0x7f80901583c0 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.565+0000 7f8096800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f80900cf0f0 con 0x7f80900cfb50 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.565+0000 7f8096800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f809006f730 con 0x7f80900d09c0 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.566+0000 7f8095e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80901583c0 0x7f809015cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.566+0000 7f8095e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80901583c0 0x7f809015cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56696/0 (socket says 172.21.15.7:56696) 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.566+0000 7f8095e00640 1 -- 172.21.15.7:0/3517273137 learned_addr learned my addr 172.21.15.7:0/3517273137 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.566+0000 7f8095400640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80900cfb50 0x7f80901502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.566+0000 7f8094a00640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80900d09c0 0x7f80901544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.566+0000 7f8095e00640 1 -- 172.21.15.7:0/3517273137 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80900d09c0 msgr2=0x7f80901544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.566+0000 7f8095e00640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80900d09c0 0x7f80901544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.566+0000 7f8095e00640 1 -- 172.21.15.7:0/3517273137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80900cfb50 msgr2=0x7f80901502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.566+0000 7f8095e00640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80900cfb50 0x7f80901502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.566+0000 7f8095e00640 1 -- 172.21.15.7:0/3517273137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8090154e60 con 0x7f80901583c0 2024-04-25T11:20:54.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.567+0000 7f8095e00640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80901583c0 0x7f809015cc70 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f808800c6b0 tx=0x7f808800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:54.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.567+0000 7f808d600640 1 -- 172.21.15.7:0/3517273137 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088003b70 con 0x7f80901583c0 2024-04-25T11:20:54.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.567+0000 7f808d600640 1 -- 172.21.15.7:0/3517273137 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8088003d30 con 0x7f80901583c0 2024-04-25T11:20:54.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.567+0000 7f8096800640 1 -- 172.21.15.7:0/3517273137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8090155140 con 0x7f80901583c0 2024-04-25T11:20:54.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.567+0000 7f808d600640 1 -- 172.21.15.7:0/3517273137 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088005310 con 0x7f80901583c0 2024-04-25T11:20:54.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.567+0000 7f8096800640 1 -- 172.21.15.7:0/3517273137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8090155700 con 0x7f80901583c0 2024-04-25T11:20:54.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.569+0000 7f8096800640 1 -- 172.21.15.7:0/3517273137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8090154bb0 con 0x7f80901583c0 2024-04-25T11:20:54.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.570+0000 7f808d600640 1 -- 172.21.15.7:0/3517273137 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f80880054b0 con 0x7f80901583c0 2024-04-25T11:20:54.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.571+0000 7f808d600640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80700744b0 0x7f80700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:54.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.571+0000 7f808d600640 1 -- 172.21.15.7:0/3517273137 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f80880952b0 con 0x7f80901583c0 2024-04-25T11:20:54.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.571+0000 7f8095400640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80700744b0 0x7f80700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.572+0000 7f8095400640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80700744b0 0x7f80700769c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f80900aa990 tx=0x7f8084008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:54.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.576+0000 7f808d600640 1 -- 172.21.15.7:0/3517273137 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f808805f810 con 0x7f80901583c0 2024-04-25T11:20:54.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.781+0000 7f8096800640 1 -- 172.21.15.7:0/3517273137 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f8090050740 con 0x7f80901583c0 2024-04-25T11:20:54.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.788+0000 7f808d600640 1 -- 172.21.15.7:0/3517273137 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v47)=0 v47) ==== 121+0+0 (secure 0 0 0) 0x7f808805f110 con 0x7f80901583c0 2024-04-25T11:20:54.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 -- 172.21.15.7:0/3517273137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80700744b0 msgr2=0x7f80700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:54.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80700744b0 0x7f80700769c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f80900aa990 tx=0x7f8084008040 comp rx=0 tx=0).stop 2024-04-25T11:20:54.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 -- 172.21.15.7:0/3517273137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80901583c0 msgr2=0x7f809015cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:54.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80901583c0 0x7f809015cc70 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f808800c6b0 tx=0x7f808800cb70 comp rx=0 tx=0).stop 2024-04-25T11:20:54.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 -- 172.21.15.7:0/3517273137 shutdown_connections 2024-04-25T11:20:54.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f80700744b0 0x7f80700769c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80901583c0 0x7f809015cc70 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80900d09c0 0x7f80901544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 --2- 172.21.15.7:0/3517273137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80900cfb50 0x7f80901502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:54.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 -- 172.21.15.7:0/3517273137 >> 172.21.15.7:0/3517273137 conn(0x7f80900740f0 msgr2=0x7f809015fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:54.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 -- 172.21.15.7:0/3517273137 shutdown_connections 2024-04-25T11:20:54.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:54.791+0000 7f8096800640 1 -- 172.21.15.7:0/3517273137 wait complete. 2024-04-25T11:20:54.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 100 2024-04-25T11:20:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.108+0000 7ff468400640 1 Processor -- start 2024-04-25T11:20:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.108+0000 7ff468400640 1 -- start start 2024-04-25T11:20:55.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.110+0000 7ff468400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46006fab0 0x7ff460168e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.110+0000 7ff468400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff46015e3c0 con 0x7ff4600b0470 2024-04-25T11:20:55.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.110+0000 7ff468400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4600cf800 con 0x7ff46015dfb0 2024-04-25T11:20:55.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.110+0000 7ff468400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff4600d1250 con 0x7ff46006fab0 2024-04-25T11:20:55.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.111+0000 7ff465000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff4600b0470 0x7ff46006f3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56460/0 (socket says 172.21.15.7:56460) 2024-04-25T11:20:55.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.111+0000 7ff465000640 1 -- 172.21.15.7:0/3761674746 learned_addr learned my addr 172.21.15.7:0/3761674746 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:55.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.111+0000 7ff466400640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46006fab0 0x7ff460168e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.111+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3993236233 0 0) 0x7ff46015e3c0 con 0x7ff4600b0470 2024-04-25T11:20:55.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.111+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff448003520 con 0x7ff4600b0470 2024-04-25T11:20:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4110748159 0 0) 0x7ff448003520 con 0x7ff4600b0470 2024-04-25T11:20:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46006fab0 msgr2=0x7ff460168e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46006fab0 0x7ff460168e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 >> v1:172.21.15.87:6789/0 conn(0x7ff46015dfb0 legacy=0x7ff4600d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4600cfaa0 con 0x7ff4600b0470 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7ff450001670 con 0x7ff4600b0470 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 >> v1:172.21.15.7:6789/0 conn(0x7ff4600b0470 legacy=0x7ff46006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46006fab0 0x7ff4480039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff448003f40 0x7ff448004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 0x7ff448005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4600d1250 con 0x7ff448004d40 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff44800bad0 con 0x7ff448003f40 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff44800bcd0 con 0x7ff46006fab0 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.112+0000 7ff466400640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 0x7ff448005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.113+0000 7ff465000640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff448003f40 0x7ff448004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.113+0000 7ff465a00640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46006fab0 0x7ff4480039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.113+0000 7ff466400640 1 -- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46006fab0 msgr2=0x7ff4480039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.113+0000 7ff466400640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46006fab0 0x7ff4480039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.113+0000 7ff466400640 1 -- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff448003f40 msgr2=0x7ff448004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.113+0000 7ff466400640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff448003f40 0x7ff448004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.113+0000 7ff466400640 1 -- 172.21.15.7:0/3761674746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff448003520 con 0x7ff448004d40 2024-04-25T11:20:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.113+0000 7ff465000640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff448003f40 0x7ff448004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.113+0000 7ff465a00640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46006fab0 0x7ff4480039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:55.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.114+0000 7ff466400640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 0x7ff448005150 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7ff458009fc0 tx=0x7ff448005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84468afe5589d02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.114+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458034020 con 0x7ff448004d40 2024-04-25T11:20:55.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.114+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4580305f0 con 0x7ff448004d40 2024-04-25T11:20:55.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.114+0000 7ff45fe00640 1 -- 172.21.15.7:0/3761674746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458009c10 con 0x7ff448004d40 2024-04-25T11:20:55.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.115+0000 7ff468400640 1 -- 172.21.15.7:0/3761674746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 msgr2=0x7ff448005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.115+0000 7ff468400640 1 --2- 172.21.15.7:0/3761674746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 0x7ff448005150 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7ff458009fc0 tx=0x7ff448005830 comp rx=0 tx=0).stop 2024-04-25T11:20:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.115+0000 7ff466400640 1 -- 172.21.15.7:0/3761674746 reap_dead start 2024-04-25T11:20:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.115+0000 7ff468400640 1 -- 172.21.15.7:0/3761674746 shutdown_connections 2024-04-25T11:20:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.115+0000 7ff468400640 1 -- 172.21.15.7:0/3761674746 >> 172.21.15.7:0/3761674746 conn(0x7ff4600742c0 msgr2=0x7ff46006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.115+0000 7ff468400640 1 -- 172.21.15.7:0/3761674746 shutdown_connections 2024-04-25T11:20:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.116+0000 7ff468400640 1 -- 172.21.15.7:0/3761674746 wait complete. 2024-04-25T11:20:55.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.118+0000 7ff468400640 1 Processor -- start 2024-04-25T11:20:55.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.118+0000 7ff468400640 1 -- start start 2024-04-25T11:20:55.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff468400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 0x7ff4601e9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff468400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff460161140 0x7ff46014a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff468400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46015cc40 0x7ff46015d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff468400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4600d0340 con 0x7ff448004d40 2024-04-25T11:20:55.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff468400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff4600d0060 con 0x7ff460161140 2024-04-25T11:20:55.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff468400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff4600d0620 con 0x7ff46015cc40 2024-04-25T11:20:55.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff465a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 0x7ff4601e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff465a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 0x7ff4601e9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56726/0 (socket says 172.21.15.7:56726) 2024-04-25T11:20:55.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff465a00640 1 -- 172.21.15.7:0/4287624454 learned_addr learned my addr 172.21.15.7:0/4287624454 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:55.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff465a00640 1 -- 172.21.15.7:0/4287624454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46015cc40 msgr2=0x7ff46015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff465a00640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46015cc40 0x7ff46015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff465000640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff460161140 0x7ff46014a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff465a00640 1 -- 172.21.15.7:0/4287624454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff460161140 msgr2=0x7ff46014a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff465a00640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff460161140 0x7ff46014a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.120+0000 7ff465a00640 1 -- 172.21.15.7:0/4287624454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4601e9b50 con 0x7ff448004d40 2024-04-25T11:20:55.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.121+0000 7ff465a00640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 0x7ff4601e9400 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7ff45400d750 tx=0x7ff45400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.121+0000 7ff45d600640 1 -- 172.21.15.7:0/4287624454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45400de60 con 0x7ff448004d40 2024-04-25T11:20:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.121+0000 7ff45d600640 1 -- 172.21.15.7:0/4287624454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff454005280 con 0x7ff448004d40 2024-04-25T11:20:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.121+0000 7ff45d600640 1 -- 172.21.15.7:0/4287624454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454007030 con 0x7ff448004d40 2024-04-25T11:20:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.121+0000 7ff468400640 1 -- 172.21.15.7:0/4287624454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46015d590 con 0x7ff448004d40 2024-04-25T11:20:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.121+0000 7ff468400640 1 -- 172.21.15.7:0/4287624454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4601e7a40 con 0x7ff448004d40 2024-04-25T11:20:55.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.123+0000 7ff468400640 1 -- 172.21.15.7:0/4287624454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff460050740 con 0x7ff448004d40 2024-04-25T11:20:55.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.124+0000 7ff45d600640 1 -- 172.21.15.7:0/4287624454 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff45401a020 con 0x7ff448004d40 2024-04-25T11:20:55.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.126+0000 7ff45d600640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4440744b0 0x7ff4440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.126+0000 7ff45d600640 1 -- 172.21.15.7:0/4287624454 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7ff454094d80 con 0x7ff448004d40 2024-04-25T11:20:55.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.127+0000 7ff465000640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4440744b0 0x7ff4440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.127+0000 7ff465000640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4440744b0 0x7ff4440769c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff46015f3f0 tx=0x7ff450001470 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.131+0000 7ff45d600640 1 -- 172.21.15.7:0/4287624454 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff46015d590 con 0x7ff448004d40 2024-04-25T11:20:55.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.339+0000 7ff468400640 1 -- 172.21.15.7:0/4287624454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7ff4600686f0 con 0x7ff448004d40 2024-04-25T11:20:55.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.348+0000 7ff45d600640 1 -- 172.21.15.7:0/4287624454 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v48)=0 v48) ==== 119+0+0 (secure 0 0 0) 0x7ff45405f380 con 0x7ff448004d40 2024-04-25T11:20:55.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.351+0000 7ff468400640 1 -- 172.21.15.7:0/4287624454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4440744b0 msgr2=0x7ff4440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.351+0000 7ff468400640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4440744b0 0x7ff4440769c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff46015f3f0 tx=0x7ff450001470 comp rx=0 tx=0).stop 2024-04-25T11:20:55.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.351+0000 7ff468400640 1 -- 172.21.15.7:0/4287624454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 msgr2=0x7ff4601e9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.351+0000 7ff468400640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 0x7ff4601e9400 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7ff45400d750 tx=0x7ff45400dc10 comp rx=0 tx=0).stop 2024-04-25T11:20:55.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.351+0000 7ff468400640 1 -- 172.21.15.7:0/4287624454 shutdown_connections 2024-04-25T11:20:55.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.351+0000 7ff468400640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4440744b0 0x7ff4440769c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.351+0000 7ff468400640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46015cc40 0x7ff46015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.351+0000 7ff468400640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff460161140 0x7ff46014a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.351+0000 7ff468400640 1 --2- 172.21.15.7:0/4287624454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff448004d40 0x7ff4601e9400 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.351+0000 7ff468400640 1 -- 172.21.15.7:0/4287624454 >> 172.21.15.7:0/4287624454 conn(0x7ff4600742c0 msgr2=0x7ff4600b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:55.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.351+0000 7ff468400640 1 -- 172.21.15.7:0/4287624454 shutdown_connections 2024-04-25T11:20:55.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.352+0000 7ff468400640 1 -- 172.21.15.7:0/4287624454 wait complete. 2024-04-25T11:20:55.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 200 2024-04-25T11:20:55.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.667+0000 7fdbbbe00640 1 Processor -- start 2024-04-25T11:20:55.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.667+0000 7fdbbbe00640 1 -- start start 2024-04-25T11:20:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.669+0000 7fdbbbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdbbc145e40 0x7fdbbc146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.669+0000 7fdbbbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdbbc0b4870 0x7fdbbc0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.669+0000 7fdbbbe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbbc0b66b0 con 0x7fdbbc0b51c0 2024-04-25T11:20:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.669+0000 7fdbbbe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fdbbc145b70 con 0x7fdbbc145e40 2024-04-25T11:20:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.670+0000 7fdbbbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdbbc0b5b30 con 0x7fdbbc0b4870 2024-04-25T11:20:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.670+0000 7fdbbb400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdbbc0b51c0 0x7fdbbc143a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56474/0 (socket says 172.21.15.7:56474) 2024-04-25T11:20:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.670+0000 7fdbbb400640 1 -- 172.21.15.7:0/2927483070 learned_addr learned my addr 172.21.15.7:0/2927483070 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.670+0000 7fdbbaa00640 1 --2- 172.21.15.7:0/2927483070 >> v2:172.21.15.87:3300/0 conn(0x7fdbbc145e40 0x7fdbbc146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.670+0000 7fdbb3e00640 1 --2- 172.21.15.7:0/2927483070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdbbc0b4870 0x7fdbbc0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.670+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 272169348 0 0) 0x7fdbbc0b66b0 con 0x7fdbbc0b51c0 2024-04-25T11:20:55.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.670+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb9c003520 con 0x7fdbbc0b51c0 2024-04-25T11:20:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 504110636 0 0) 0x7fdb9c003520 con 0x7fdbbc0b51c0 2024-04-25T11:20:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdbbc0b4870 msgr2=0x7fdbbc0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 --2- 172.21.15.7:0/2927483070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdbbc0b4870 0x7fdbbc0b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 >> v2:172.21.15.87:3300/0 conn(0x7fdbbc145e40 msgr2=0x7fdbbc146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 --2- 172.21.15.7:0/2927483070 >> v2:172.21.15.87:3300/0 conn(0x7fdbbc145e40 0x7fdbbc146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc0b5e10 con 0x7fdbbc0b51c0 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fdbb40019f0 con 0x7fdbbc0b51c0 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 >> v1:172.21.15.7:6789/0 conn(0x7fdbbc0b51c0 legacy=0x7fdbbc143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 --2- 172.21.15.7:0/2927483070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 --2- 172.21.15.7:0/2927483070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 0x7fdb9c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 --2- 172.21.15.7:0/2927483070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdbbc0b4870 0x7fdb9c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc0b5b30 con 0x7fdb9c004800 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c00ba70 con 0x7fdb9c0039b0 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c00bc70 con 0x7fdbbc0b4870 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.671+0000 7fdbb3e00640 1 --2- 172.21.15.7:0/2927483070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 0x7fdb9c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.672+0000 7fdbbaa00640 1 --2- 172.21.15.7:0/2927483070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.672+0000 7fdbb3e00640 1 -- 172.21.15.7:0/2927483070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdbbc0b4870 msgr2=0x7fdb9c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.672+0000 7fdbb3e00640 1 --2- 172.21.15.7:0/2927483070 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdbbc0b4870 0x7fdb9c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.672+0000 7fdbb3e00640 1 -- 172.21.15.7:0/2927483070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb9c0039b0 msgr2=0x7fdb9c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.672+0000 7fdbb3e00640 1 --2- 172.21.15.7:0/2927483070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb9c0039b0 0x7fdb9c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.672+0000 7fdbb3e00640 1 -- 172.21.15.7:0/2927483070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c003520 con 0x7fdb9c004800 2024-04-25T11:20:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.673+0000 7fdbb3e00640 1 --2- 172.21.15.7:0/2927483070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 0x7fdb9c004c10 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009650 tx=0x7fdb9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f84f198c413efb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.673+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4038070 con 0x7fdb9c004800 2024-04-25T11:20:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.673+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba4033400 con 0x7fdb9c004800 2024-04-25T11:20:55.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.673+0000 7fdbba000640 1 -- 172.21.15.7:0/2927483070 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4009690 con 0x7fdb9c004800 2024-04-25T11:20:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.674+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2927483070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 msgr2=0x7fdb9c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.674+0000 7fdbbbe00640 1 --2- 172.21.15.7:0/2927483070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 0x7fdb9c004c10 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fdba4009650 tx=0x7fdb9c005830 comp rx=0 tx=0).stop 2024-04-25T11:20:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.674+0000 7fdbbb400640 1 -- 172.21.15.7:0/2927483070 reap_dead start 2024-04-25T11:20:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.674+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2927483070 shutdown_connections 2024-04-25T11:20:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.674+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2927483070 >> 172.21.15.7:0/2927483070 conn(0x7fdbbc0740f0 msgr2=0x7fdbbc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.674+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2927483070 shutdown_connections 2024-04-25T11:20:55.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.674+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2927483070 wait complete. 2024-04-25T11:20:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.677+0000 7fdbbbe00640 1 Processor -- start 2024-04-25T11:20:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.677+0000 7fdbbbe00640 1 -- start start 2024-04-25T11:20:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.678+0000 7fdbbbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 0x7fdbbc15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.678+0000 7fdbbbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbbc13de40 0x7fdbbc13a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.678+0000 7fdbbbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdbbc1f6050 0x7fdbbc1f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.678+0000 7fdbbbe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc0b55d0 con 0x7fdb9c004800 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.678+0000 7fdbbbe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc0b63d0 con 0x7fdbbc13de40 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.678+0000 7fdbbbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdbbc0b5750 con 0x7fdbbc1f6050 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.678+0000 7fdbbaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 0x7fdbbc15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.678+0000 7fdbbaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 0x7fdbbc15c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56748/0 (socket says 172.21.15.7:56748) 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.679+0000 7fdbbaa00640 1 -- 172.21.15.7:0/2230414647 learned_addr learned my addr 172.21.15.7:0/2230414647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.679+0000 7fdbb3e00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbbc13de40 0x7fdbbc13a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.679+0000 7fdbbaa00640 1 -- 172.21.15.7:0/2230414647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdbbc1f6050 msgr2=0x7fdbbc1f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.679+0000 7fdbbaa00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdbbc1f6050 0x7fdbbc1f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.679+0000 7fdbbaa00640 1 -- 172.21.15.7:0/2230414647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbbc13de40 msgr2=0x7fdbbc13a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.679+0000 7fdbbaa00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbbc13de40 0x7fdbbc13a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.679+0000 7fdbbaa00640 1 -- 172.21.15.7:0/2230414647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc137970 con 0x7fdb9c004800 2024-04-25T11:20:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.679+0000 7fdbbaa00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 0x7fdbbc15c4a0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0116d0 tx=0x7fdbac011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.680+0000 7fdbb2a00640 1 -- 172.21.15.7:0/2230414647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac00f9c0 con 0x7fdb9c004800 2024-04-25T11:20:55.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.680+0000 7fdbb2a00640 1 -- 172.21.15.7:0/2230414647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbac00fb60 con 0x7fdb9c004800 2024-04-25T11:20:55.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.680+0000 7fdbb2a00640 1 -- 172.21.15.7:0/2230414647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac015460 con 0x7fdb9c004800 2024-04-25T11:20:55.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.680+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2230414647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbbc13cda0 con 0x7fdb9c004800 2024-04-25T11:20:55.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.680+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2230414647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbbc13d360 con 0x7fdb9c004800 2024-04-25T11:20:55.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.681+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2230414647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbbc050740 con 0x7fdb9c004800 2024-04-25T11:20:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.683+0000 7fdbb2a00640 1 -- 172.21.15.7:0/2230414647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdbac015600 con 0x7fdb9c004800 2024-04-25T11:20:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.684+0000 7fdbb2a00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdb980744b0 0x7fdb980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.684+0000 7fdbb2a00640 1 -- 172.21.15.7:0/2230414647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7fdbac095a20 con 0x7fdb9c004800 2024-04-25T11:20:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.684+0000 7fdbb3e00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdb980744b0 0x7fdb980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.685+0000 7fdbb3e00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdb980744b0 0x7fdb980769c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdba4035000 tx=0x7fdba4010040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:55.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.689+0000 7fdbb2a00640 1 -- 172.21.15.7:0/2230414647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdbac019030 con 0x7fdb9c004800 2024-04-25T11:20:55.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.894+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2230414647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7fdbbc145c00 con 0x7fdb9c004800 2024-04-25T11:20:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.902+0000 7fdbb2a00640 1 -- 172.21.15.7:0/2230414647 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v49)=0 v49) ==== 128+0+0 (secure 0 0 0) 0x7fdbac060020 con 0x7fdb9c004800 2024-04-25T11:20:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.904+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2230414647 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdb980744b0 msgr2=0x7fdb980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.904+0000 7fdbbbe00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdb980744b0 0x7fdb980769c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdba4035000 tx=0x7fdba4010040 comp rx=0 tx=0).stop 2024-04-25T11:20:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.904+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2230414647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 msgr2=0x7fdbbc15c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.904+0000 7fdbbbe00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 0x7fdbbc15c4a0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0116d0 tx=0x7fdbac011b90 comp rx=0 tx=0).stop 2024-04-25T11:20:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.904+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2230414647 shutdown_connections 2024-04-25T11:20:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.904+0000 7fdbbbe00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdb980744b0 0x7fdb980769c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.904+0000 7fdbbbe00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdbbc1f6050 0x7fdbbc1f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.904+0000 7fdbbbe00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdbbc13de40 0x7fdbbc13a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.904+0000 7fdbbbe00640 1 --2- 172.21.15.7:0/2230414647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9c004800 0x7fdbbc15c4a0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.905+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2230414647 >> 172.21.15.7:0/2230414647 conn(0x7fdbbc0740f0 msgr2=0x7fdbbc0cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.905+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2230414647 shutdown_connections 2024-04-25T11:20:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:55.905+0000 7fdbbbe00640 1 -- 172.21.15.7:0/2230414647 wait complete. 2024-04-25T11:20:55.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-04-25T11:20:56.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.207+0000 7f9a4de00640 1 Processor -- start 2024-04-25T11:20:56.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.207+0000 7f9a4de00640 1 -- start start 2024-04-25T11:20:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.210+0000 7f9a4de00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a481583c0 0x7f9a480cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.210+0000 7f9a4de00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9a480cfb50 0x7f9a480d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.210+0000 7f9a4de00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a480d09c0 0x7f9a4806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.210+0000 7f9a4de00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9a4815f190 con 0x7f9a480d09c0 2024-04-25T11:20:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.210+0000 7f9a4de00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9a480ce570 con 0x7f9a480cfb50 2024-04-25T11:20:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.210+0000 7f9a4de00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9a480ce850 con 0x7f9a481583c0 2024-04-25T11:20:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.210+0000 7f9a4d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a480d09c0 0x7f9a4806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.210+0000 7f9a4d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a480d09c0 0x7f9a4806eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56764/0 (socket says 172.21.15.7:56764) 2024-04-25T11:20:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.210+0000 7f9a4d400640 1 -- 172.21.15.7:0/2918430097 learned_addr learned my addr 172.21.15.7:0/2918430097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:56.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.210+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/2918430097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a481583c0 0x7f9a480cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.211+0000 7f9a4d400640 1 -- 172.21.15.7:0/2918430097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a481583c0 msgr2=0x7f9a480cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.211+0000 7f9a4d400640 1 --2- 172.21.15.7:0/2918430097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a481583c0 0x7f9a480cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.211+0000 7f9a47e00640 1 --2- 172.21.15.7:0/2918430097 >> v2:172.21.15.87:3300/0 conn(0x7f9a480cfb50 0x7f9a480d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.211+0000 7f9a4d400640 1 -- 172.21.15.7:0/2918430097 >> v2:172.21.15.87:3300/0 conn(0x7f9a480cfb50 msgr2=0x7f9a480d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.211+0000 7f9a4d400640 1 --2- 172.21.15.7:0/2918430097 >> v2:172.21.15.87:3300/0 conn(0x7f9a480cfb50 0x7f9a480d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.211+0000 7f9a4d400640 1 -- 172.21.15.7:0/2918430097 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a480ceb30 con 0x7f9a480d09c0 2024-04-25T11:20:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.211+0000 7f9a47e00640 1 --2- 172.21.15.7:0/2918430097 >> v2:172.21.15.87:3300/0 conn(0x7f9a480cfb50 0x7f9a480d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:56.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.212+0000 7f9a4d400640 1 --2- 172.21.15.7:0/2918430097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a480d09c0 0x7f9a4806eea0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f9a4806e0f0 tx=0x7f9a4002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd72f23bd547343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.212+0000 7f9a47400640 1 -- 172.21.15.7:0/2918430097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40036070 con 0x7f9a480d09c0 2024-04-25T11:20:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.212+0000 7f9a47400640 1 -- 172.21.15.7:0/2918430097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a40031600 con 0x7f9a480d09c0 2024-04-25T11:20:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.212+0000 7f9a47400640 1 -- 172.21.15.7:0/2918430097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a400319d0 con 0x7f9a480d09c0 2024-04-25T11:20:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.213+0000 7f9a4de00640 1 -- 172.21.15.7:0/2918430097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a480d09c0 msgr2=0x7f9a4806eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.213+0000 7f9a4de00640 1 --2- 172.21.15.7:0/2918430097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a480d09c0 0x7f9a4806eea0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f9a4806e0f0 tx=0x7f9a4002a6a0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.213+0000 7f9a4de00640 1 -- 172.21.15.7:0/2918430097 shutdown_connections 2024-04-25T11:20:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.213+0000 7f9a4de00640 1 --2- 172.21.15.7:0/2918430097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a480d09c0 0x7f9a4806eea0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.213+0000 7f9a4de00640 1 --2- 172.21.15.7:0/2918430097 >> v2:172.21.15.87:3300/0 conn(0x7f9a480cfb50 0x7f9a480d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.213+0000 7f9a4de00640 1 --2- 172.21.15.7:0/2918430097 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a481583c0 0x7f9a480cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.213+0000 7f9a4de00640 1 -- 172.21.15.7:0/2918430097 >> 172.21.15.7:0/2918430097 conn(0x7f9a480740f0 msgr2=0x7f9a4806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:56.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.214+0000 7f9a4de00640 1 -- 172.21.15.7:0/2918430097 shutdown_connections 2024-04-25T11:20:56.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.214+0000 7f9a4de00640 1 -- 172.21.15.7:0/2918430097 wait complete. 2024-04-25T11:20:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.216+0000 7f9a4de00640 1 Processor -- start 2024-04-25T11:20:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.216+0000 7f9a4de00640 1 -- start start 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.218+0000 7f9a4de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a480cfb50 0x7f9a48141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.218+0000 7f9a4de00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a480d09c0 0x7f9a4813f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.218+0000 7f9a4de00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a481583c0 0x7f9a48140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.218+0000 7f9a4de00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4806f3e0 con 0x7f9a480cfb50 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.218+0000 7f9a4de00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9a480cf0f0 con 0x7f9a480d09c0 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.218+0000 7f9a4de00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4806f560 con 0x7f9a481583c0 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.218+0000 7f9a4ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a480cfb50 0x7f9a48141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.218+0000 7f9a4ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a480cfb50 0x7f9a48141ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56776/0 (socket says 172.21.15.7:56776) 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.218+0000 7f9a4ca00640 1 -- 172.21.15.7:0/1031975655 learned_addr learned my addr 172.21.15.7:0/1031975655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.218+0000 7f9a47e00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a480d09c0 0x7f9a4813f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.219+0000 7f9a4d400640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a481583c0 0x7f9a48140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.219+0000 7f9a4ca00640 1 -- 172.21.15.7:0/1031975655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a481583c0 msgr2=0x7f9a48140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.219+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a481583c0 0x7f9a48140280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.219+0000 7f9a4ca00640 1 -- 172.21.15.7:0/1031975655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a480d09c0 msgr2=0x7f9a4813f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.219+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a480d09c0 0x7f9a4813f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.219+0000 7f9a4ca00640 1 -- 172.21.15.7:0/1031975655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a48140b00 con 0x7f9a480cfb50 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.219+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a480cfb50 0x7f9a48141ba0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00d5d0 tx=0x7f9a3c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.219+0000 7f9a44c00640 1 -- 172.21.15.7:0/1031975655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c004240 con 0x7f9a480cfb50 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.219+0000 7f9a44c00640 1 -- 172.21.15.7:0/1031975655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a3c0043e0 con 0x7f9a480cfb50 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.219+0000 7f9a44c00640 1 -- 172.21.15.7:0/1031975655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c003830 con 0x7f9a480cfb50 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.219+0000 7f9a4de00640 1 -- 172.21.15.7:0/1031975655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a48140de0 con 0x7f9a480cfb50 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.220+0000 7f9a4de00640 1 -- 172.21.15.7:0/1031975655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a481413a0 con 0x7f9a480cfb50 2024-04-25T11:20:56.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.221+0000 7f9a4de00640 1 -- 172.21.15.7:0/1031975655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a48050740 con 0x7f9a480cfb50 2024-04-25T11:20:56.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.222+0000 7f9a44c00640 1 -- 172.21.15.7:0/1031975655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9a3c0039d0 con 0x7f9a480cfb50 2024-04-25T11:20:56.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.223+0000 7f9a44c00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 0x7f9a280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.223+0000 7f9a44c00640 1 -- 172.21.15.7:0/1031975655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f9a3c095770 con 0x7f9a480cfb50 2024-04-25T11:20:56.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.224+0000 7f9a47e00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 0x7f9a280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.224+0000 7f9a47e00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 0x7f9a280769c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9a48076a40 tx=0x7f9a34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.228+0000 7f9a44c00640 1 -- 172.21.15.7:0/1031975655 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a3c05fcd0 con 0x7f9a480cfb50 2024-04-25T11:20:56.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.431+0000 7f9a4de00640 1 -- 172.21.15.7:0/1031975655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f9a480cdd80 con 0x7f9a480cfb50 2024-04-25T11:20:56.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.432+0000 7f9a44c00640 1 -- 172.21.15.7:0/1031975655 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v49) ==== 106+0+4 (secure 0 0 0) 0x7f9a3c05f5d0 con 0x7f9a480cfb50 2024-04-25T11:20:56.429 INFO:teuthology.orchestra.run.smithi007.stdout:100 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 -- 172.21.15.7:0/1031975655 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 msgr2=0x7f9a280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 0x7f9a280769c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9a48076a40 tx=0x7f9a34008040 comp rx=0 tx=0).stop 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 -- 172.21.15.7:0/1031975655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a480cfb50 msgr2=0x7f9a48141ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a480cfb50 0x7f9a48141ba0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00d5d0 tx=0x7f9a3c00da90 comp rx=0 tx=0).stop 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 -- 172.21.15.7:0/1031975655 shutdown_connections 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 0x7f9a280769c0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a481583c0 0x7f9a48140280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a480d09c0 0x7f9a4813f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 --2- 172.21.15.7:0/1031975655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a480cfb50 0x7f9a48141ba0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 -- 172.21.15.7:0/1031975655 >> 172.21.15.7:0/1031975655 conn(0x7f9a480740f0 msgr2=0x7f9a481529a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 -- 172.21.15.7:0/1031975655 shutdown_connections 2024-04-25T11:20:56.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.435+0000 7f9a4de00640 1 -- 172.21.15.7:0/1031975655 wait complete. 2024-04-25T11:20:56.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-04-25T11:20:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.748+0000 7f6445600640 1 Processor -- start 2024-04-25T11:20:56.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.748+0000 7f6445600640 1 -- start start 2024-04-25T11:20:56.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.751+0000 7f6445600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f64400d0ee0 0x7f64400affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.751+0000 7f6445600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644006f5b0 con 0x7f644015aaa0 2024-04-25T11:20:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.751+0000 7f6445600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64400d14a0 con 0x7f64400b5020 2024-04-25T11:20:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.751+0000 7f6445600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f64400af290 con 0x7f64400d0ee0 2024-04-25T11:20:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.751+0000 7f643ea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f644015aaa0 0x7f64400b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56490/0 (socket says 172.21.15.7:56490) 2024-04-25T11:20:56.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.751+0000 7f643ea00640 1 -- 172.21.15.7:0/1291733989 learned_addr learned my addr 172.21.15.7:0/1291733989 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.752+0000 7f643e000640 1 --2- 172.21.15.7:0/1291733989 >> v2:172.21.15.87:3300/0 conn(0x7f64400d0ee0 0x7f64400affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.752+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 156705141 0 0) 0x7f644006f5b0 con 0x7f644015aaa0 2024-04-25T11:20:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.752+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6424003520 con 0x7f644015aaa0 2024-04-25T11:20:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.752+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4079160773 0 0) 0x7f6424003520 con 0x7f644015aaa0 2024-04-25T11:20:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.752+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 >> v2:172.21.15.87:3300/0 conn(0x7f64400d0ee0 msgr2=0x7f64400affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.752+0000 7f643d600640 1 --2- 172.21.15.7:0/1291733989 >> v2:172.21.15.87:3300/0 conn(0x7f64400d0ee0 0x7f64400affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.752+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 >> v1:172.21.15.87:6789/0 conn(0x7f64400b5020 legacy=0x7f6440160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.752+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64400af4e0 con 0x7f644015aaa0 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.752+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f6430001670 con 0x7f644015aaa0 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.752+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 >> v1:172.21.15.7:6789/0 conn(0x7f644015aaa0 legacy=0x7f64400b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.752+0000 7f643d600640 1 --2- 172.21.15.7:0/1291733989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424003a00 0x7f6424003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643d600640 1 --2- 172.21.15.7:0/1291733989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424004800 0x7f6424004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643d600640 1 --2- 172.21.15.7:0/1291733989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6424005220 0x7f642400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f64400af290 con 0x7f6424003a00 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f642400c1b0 con 0x7f6424004800 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f642400c3b0 con 0x7f6424005220 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643ea00640 1 --2- 172.21.15.7:0/1291733989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424003a00 0x7f6424003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643e000640 1 --2- 172.21.15.7:0/1291733989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424004800 0x7f6424004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643ea00640 1 -- 172.21.15.7:0/1291733989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6424005220 msgr2=0x7f642400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643ea00640 1 --2- 172.21.15.7:0/1291733989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6424005220 0x7f642400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643ea00640 1 -- 172.21.15.7:0/1291733989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424004800 msgr2=0x7f6424004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:56.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643ea00640 1 --2- 172.21.15.7:0/1291733989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424004800 0x7f6424004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643ea00640 1 -- 172.21.15.7:0/1291733989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6424003520 con 0x7f6424003a00 2024-04-25T11:20:56.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.753+0000 7f643f400640 1 -- 172.21.15.7:0/1291733989 reap_dead start 2024-04-25T11:20:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.755+0000 7f643ea00640 1 --2- 172.21.15.7:0/1291733989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424003a00 0x7f6424003e90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f643000c110 tx=0x7f6424005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfd646ae1a58c22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.755+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430007c40 con 0x7f6424003a00 2024-04-25T11:20:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.755+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6430005af0 con 0x7f6424003a00 2024-04-25T11:20:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.755+0000 7f643d600640 1 -- 172.21.15.7:0/1291733989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6430007070 con 0x7f6424003a00 2024-04-25T11:20:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.756+0000 7f6445600640 1 -- 172.21.15.7:0/1291733989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424003a00 msgr2=0x7f6424003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.756+0000 7f6445600640 1 --2- 172.21.15.7:0/1291733989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424003a00 0x7f6424003e90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f643000c110 tx=0x7f6424005830 comp rx=0 tx=0).stop 2024-04-25T11:20:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.756+0000 7f6445600640 1 -- 172.21.15.7:0/1291733989 shutdown_connections 2024-04-25T11:20:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.756+0000 7f6445600640 1 --2- 172.21.15.7:0/1291733989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6424003a00 0x7f6424003e90 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.756+0000 7f6445600640 1 -- 172.21.15.7:0/1291733989 >> 172.21.15.7:0/1291733989 conn(0x7f64400742c0 msgr2=0x7f644006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.756+0000 7f6445600640 1 -- 172.21.15.7:0/1291733989 shutdown_connections 2024-04-25T11:20:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.756+0000 7f6445600640 1 -- 172.21.15.7:0/1291733989 wait complete. 2024-04-25T11:20:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.758+0000 7f6445600640 1 Processor -- start 2024-04-25T11:20:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.758+0000 7f6445600640 1 -- start start 2024-04-25T11:20:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.760+0000 7f6445600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424003a00 0x7f64401f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.760+0000 7f6445600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6440149e00 0x7f644014fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.760+0000 7f6445600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64401501d0 0x7f64401f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.760+0000 7f6445600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f64400b5430 con 0x7f64401501d0 2024-04-25T11:20:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.760+0000 7f6445600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f64400afaa0 con 0x7f6424003a00 2024-04-25T11:20:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.760+0000 7f6445600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f64400b55b0 con 0x7f6440149e00 2024-04-25T11:20:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.760+0000 7f643f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64401501d0 0x7f64401f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.761+0000 7f643f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64401501d0 0x7f64401f77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56790/0 (socket says 172.21.15.7:56790) 2024-04-25T11:20:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.761+0000 7f643f400640 1 -- 172.21.15.7:0/994841162 learned_addr learned my addr 172.21.15.7:0/994841162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.761+0000 7f643ea00640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424003a00 0x7f64401f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.761+0000 7f643e000640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6440149e00 0x7f644014fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.761+0000 7f643f400640 1 -- 172.21.15.7:0/994841162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6440149e00 msgr2=0x7f644014fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.761+0000 7f643f400640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6440149e00 0x7f644014fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.761+0000 7f643f400640 1 -- 172.21.15.7:0/994841162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424003a00 msgr2=0x7f64401f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.761+0000 7f643f400640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424003a00 0x7f64401f3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.761+0000 7f643f400640 1 -- 172.21.15.7:0/994841162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64401505e0 con 0x7f64401501d0 2024-04-25T11:20:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.761+0000 7f643ea00640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424003a00 0x7f64401f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:20:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.761+0000 7f643f400640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64401501d0 0x7f64401f77f0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f644006ec40 tx=0x7f643800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.762+0000 7f6436a00640 1 -- 172.21.15.7:0/994841162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438017070 con 0x7f64401501d0 2024-04-25T11:20:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.762+0000 7f6436a00640 1 -- 172.21.15.7:0/994841162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6438005d60 con 0x7f64401501d0 2024-04-25T11:20:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.762+0000 7f6436a00640 1 -- 172.21.15.7:0/994841162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643800a850 con 0x7f64401501d0 2024-04-25T11:20:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.762+0000 7f6445600640 1 -- 172.21.15.7:0/994841162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644014b0f0 con 0x7f64401501d0 2024-04-25T11:20:56.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.762+0000 7f6445600640 1 -- 172.21.15.7:0/994841162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644014ab30 con 0x7f64401501d0 2024-04-25T11:20:56.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.763+0000 7f6445600640 1 -- 172.21.15.7:0/994841162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644014a5c0 con 0x7f64401501d0 2024-04-25T11:20:56.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.764+0000 7f6436a00640 1 -- 172.21.15.7:0/994841162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f643801a070 con 0x7f64401501d0 2024-04-25T11:20:56.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.765+0000 7f6436a00640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64200744e0 0x7f64200769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:56.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.765+0000 7f6436a00640 1 -- 172.21.15.7:0/994841162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f6438094520 con 0x7f64401501d0 2024-04-25T11:20:56.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.766+0000 7f643ea00640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64200744e0 0x7f64200769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:56.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.766+0000 7f643ea00640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64200744e0 0x7f64200769f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6430032000 tx=0x7f6430009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:56.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.771+0000 7f6436a00640 1 -- 172.21.15.7:0/994841162 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64401505e0 con 0x7f64401501d0 2024-04-25T11:20:56.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.978+0000 7f6445600640 1 -- 172.21.15.7:0/994841162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f6440050740 con 0x7f64401501d0 2024-04-25T11:20:56.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.987+0000 7f6436a00640 1 -- 172.21.15.7:0/994841162 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v50)=0 v50) ==== 124+0+0 (secure 0 0 0) 0x7f643805ea80 con 0x7f64401501d0 2024-04-25T11:20:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.990+0000 7f6445600640 1 -- 172.21.15.7:0/994841162 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64200744e0 msgr2=0x7f64200769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.990+0000 7f6445600640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64200744e0 0x7f64200769f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6430032000 tx=0x7f6430009030 comp rx=0 tx=0).stop 2024-04-25T11:20:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.990+0000 7f6445600640 1 -- 172.21.15.7:0/994841162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64401501d0 msgr2=0x7f64401f77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.990+0000 7f6445600640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64401501d0 0x7f64401f77f0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f644006ec40 tx=0x7f643800cb90 comp rx=0 tx=0).stop 2024-04-25T11:20:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.990+0000 7f6445600640 1 -- 172.21.15.7:0/994841162 shutdown_connections 2024-04-25T11:20:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.990+0000 7f6445600640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64200744e0 0x7f64200769f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.990+0000 7f6445600640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64401501d0 0x7f64401f77f0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.990+0000 7f6445600640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6440149e00 0x7f644014fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.990+0000 7f6445600640 1 --2- 172.21.15.7:0/994841162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6424003a00 0x7f64401f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.990+0000 7f6445600640 1 -- 172.21.15.7:0/994841162 >> 172.21.15.7:0/994841162 conn(0x7f64400742c0 msgr2=0x7f644015dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.991+0000 7f6445600640 1 -- 172.21.15.7:0/994841162 shutdown_connections 2024-04-25T11:20:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:56.991+0000 7f6445600640 1 -- 172.21.15.7:0/994841162 wait complete. 2024-04-25T11:20:57.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-04-25T11:20:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.301+0000 7f6798c00640 1 Processor -- start 2024-04-25T11:20:57.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.301+0000 7f6798c00640 1 -- start start 2024-04-25T11:20:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.304+0000 7f6798c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f679406fc30 0x7f6794070040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.304+0000 7f6798c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f679415e560 0x7f679415e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.304+0000 7f6798c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6794070580 con 0x7f6794070b60 2024-04-25T11:20:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.304+0000 7f6798c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f679415e0d0 con 0x7f679406fc30 2024-04-25T11:20:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.304+0000 7f6798c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6794153b00 con 0x7f679415e560 2024-04-25T11:20:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.304+0000 7f6791400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f679406fc30 0x7f6794070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.304+0000 7f6791400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f679406fc30 0x7f6794070040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:38084/0 (socket says 172.21.15.7:38084) 2024-04-25T11:20:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.304+0000 7f6791400640 1 -- 172.21.15.7:0/2045261301 learned_addr learned my addr 172.21.15.7:0/2045261301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.304+0000 7f6792800640 1 --2- 172.21.15.7:0/2045261301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f679415e560 0x7f679415e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.305+0000 7f6792800640 1 -- 172.21.15.7:0/2045261301 >> v2:172.21.15.87:3300/0 conn(0x7f679406fc30 msgr2=0x7f6794070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.305+0000 7f6792800640 1 --2- 172.21.15.7:0/2045261301 >> v2:172.21.15.87:3300/0 conn(0x7f679406fc30 0x7f6794070040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.305+0000 7f6792800640 1 -- 172.21.15.7:0/2045261301 >> v1:172.21.15.87:6789/0 conn(0x7f6794070b60 legacy=0x7f679415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.305+0000 7f6792800640 1 -- 172.21.15.7:0/2045261301 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679415db10 con 0x7f679415e560 2024-04-25T11:20:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.305+0000 7f6791400640 1 --2- 172.21.15.7:0/2045261301 >> v2:172.21.15.87:3300/0 conn(0x7f679406fc30 0x7f6794070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:20:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.307+0000 7f6792800640 1 --2- 172.21.15.7:0/2045261301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f679415e560 0x7f679415e970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f678c009d90 tx=0x7f678c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c157d7e3d6e12912 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.307+0000 7f6790a00640 1 -- 172.21.15.7:0/2045261301 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c031600 con 0x7f679415e560 2024-04-25T11:20:57.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.307+0000 7f6790a00640 1 -- 172.21.15.7:0/2045261301 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f678c0317a0 con 0x7f679415e560 2024-04-25T11:20:57.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.307+0000 7f6790a00640 1 -- 172.21.15.7:0/2045261301 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678c031b70 con 0x7f679415e560 2024-04-25T11:20:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.308+0000 7f6798c00640 1 -- 172.21.15.7:0/2045261301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f679415e560 msgr2=0x7f679415e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.308+0000 7f6798c00640 1 --2- 172.21.15.7:0/2045261301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f679415e560 0x7f679415e970 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f678c009d90 tx=0x7f678c02a5d0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.308+0000 7f6798c00640 1 -- 172.21.15.7:0/2045261301 shutdown_connections 2024-04-25T11:20:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.308+0000 7f6798c00640 1 --2- 172.21.15.7:0/2045261301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f679415e560 0x7f679415e970 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.308+0000 7f6798c00640 1 --2- 172.21.15.7:0/2045261301 >> v2:172.21.15.87:3300/0 conn(0x7f679406fc30 0x7f6794070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.308+0000 7f6798c00640 1 -- 172.21.15.7:0/2045261301 >> 172.21.15.7:0/2045261301 conn(0x7f67940742c0 msgr2=0x7f679406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.308+0000 7f6798c00640 1 -- 172.21.15.7:0/2045261301 shutdown_connections 2024-04-25T11:20:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.308+0000 7f6798c00640 1 -- 172.21.15.7:0/2045261301 wait complete. 2024-04-25T11:20:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.311+0000 7f6798c00640 1 Processor -- start 2024-04-25T11:20:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.311+0000 7f6798c00640 1 -- start start 2024-04-25T11:20:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.312+0000 7f6798c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f679406fc30 0x7f679413f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.312+0000 7f6798c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6794070b60 0x7f6794141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.312+0000 7f6798c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f679415e560 0x7f67941584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.312+0000 7f6798c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67940af980 con 0x7f6794070b60 2024-04-25T11:20:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.312+0000 7f6798c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f67940af6a0 con 0x7f679406fc30 2024-04-25T11:20:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.312+0000 7f6798c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f679415dd60 con 0x7f679415e560 2024-04-25T11:20:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.313+0000 7f6791400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6794070b60 0x7f6794141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.313+0000 7f6791400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6794070b60 0x7f6794141090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56800/0 (socket says 172.21.15.7:56800) 2024-04-25T11:20:57.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.313+0000 7f6791400640 1 -- 172.21.15.7:0/7471724 learned_addr learned my addr 172.21.15.7:0/7471724 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.313+0000 7f6791e00640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f679406fc30 0x7f679413f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.313+0000 7f6792800640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f679415e560 0x7f67941584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.313+0000 7f6791400640 1 -- 172.21.15.7:0/7471724 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f679415e560 msgr2=0x7f67941584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.313+0000 7f6791400640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f679415e560 0x7f67941584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.313+0000 7f6791400640 1 -- 172.21.15.7:0/7471724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f679406fc30 msgr2=0x7f679413f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.313+0000 7f6791400640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f679406fc30 0x7f679413f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.313+0000 7f6791400640 1 -- 172.21.15.7:0/7471724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67941415d0 con 0x7f6794070b60 2024-04-25T11:20:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.313+0000 7f6791e00640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f679406fc30 0x7f679413f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.314+0000 7f6791400640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6794070b60 0x7f6794141090 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f677c00e8e0 tx=0x7f677c00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.314+0000 7f678a000640 1 -- 172.21.15.7:0/7471724 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c00b040 con 0x7f6794070b60 2024-04-25T11:20:57.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.314+0000 7f678a000640 1 -- 172.21.15.7:0/7471724 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c00cd00 con 0x7f6794070b60 2024-04-25T11:20:57.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.314+0000 7f678a000640 1 -- 172.21.15.7:0/7471724 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c004230 con 0x7f6794070b60 2024-04-25T11:20:57.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.314+0000 7f6798c00640 1 -- 172.21.15.7:0/7471724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679413fff0 con 0x7f6794070b60 2024-04-25T11:20:57.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.314+0000 7f6798c00640 1 -- 172.21.15.7:0/7471724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67941405b0 con 0x7f6794070b60 2024-04-25T11:20:57.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.315+0000 7f6798c00640 1 -- 172.21.15.7:0/7471724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6794050740 con 0x7f6794070b60 2024-04-25T11:20:57.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.317+0000 7f678a000640 1 -- 172.21.15.7:0/7471724 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f677c003610 con 0x7f6794070b60 2024-04-25T11:20:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.318+0000 7f678a000640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744b0 0x7f67700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.318+0000 7f678a000640 1 -- 172.21.15.7:0/7471724 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f677c0950b0 con 0x7f6794070b60 2024-04-25T11:20:57.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.318+0000 7f6791e00640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744b0 0x7f67700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.319+0000 7f6791e00640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744b0 0x7f67700769c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6784000f60 tx=0x7f6784009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.322+0000 7f678a000640 1 -- 172.21.15.7:0/7471724 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f677c016020 con 0x7f6794070b60 2024-04-25T11:20:57.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.528+0000 7f6798c00640 1 -- 172.21.15.7:0/7471724 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7f6794070940 con 0x7f6794070b60 2024-04-25T11:20:57.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.529+0000 7f678a000640 1 -- 172.21.15.7:0/7471724 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v50) ==== 106+0+8 (secure 0 0 0) 0x7f677c05f740 con 0x7f6794070b60 2024-04-25T11:20:57.526 INFO:teuthology.orchestra.run.smithi007.stdout:1048576 2024-04-25T11:20:57.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 -- 172.21.15.7:0/7471724 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744b0 msgr2=0x7f67700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744b0 0x7f67700769c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6784000f60 tx=0x7f6784009040 comp rx=0 tx=0).stop 2024-04-25T11:20:57.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 -- 172.21.15.7:0/7471724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6794070b60 msgr2=0x7f6794141090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6794070b60 0x7f6794141090 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f677c00e8e0 tx=0x7f677c00eda0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 -- 172.21.15.7:0/7471724 shutdown_connections 2024-04-25T11:20:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f67700744b0 0x7f67700769c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f679415e560 0x7f67941584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6794070b60 0x7f6794141090 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 --2- 172.21.15.7:0/7471724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f679406fc30 0x7f679413f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 -- 172.21.15.7:0/7471724 >> 172.21.15.7:0/7471724 conn(0x7f67940742c0 msgr2=0x7f6794155ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 -- 172.21.15.7:0/7471724 shutdown_connections 2024-04-25T11:20:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.533+0000 7f6798c00640 1 -- 172.21.15.7:0/7471724 wait complete. 2024-04-25T11:20:57.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-04-25T11:20:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.850+0000 7f3224c00640 1 Processor -- start 2024-04-25T11:20:57.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.850+0000 7f3224c00640 1 -- start start 2024-04-25T11:20:57.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.853+0000 7f3224c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32200d2430 0x7f32200b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.853+0000 7f3224c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f32200b3080 0x7f32201620e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.853+0000 7f3224c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32200d2840 con 0x7f32200b8770 2024-04-25T11:20:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.853+0000 7f3224c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f32200ceff0 con 0x7f32200b3080 2024-04-25T11:20:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.853+0000 7f3224c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f32200b8b80 con 0x7f32200d2430 2024-04-25T11:20:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.853+0000 7f321fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f32200b3080 0x7f32201620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.853+0000 7f321f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f32200b8770 0x7f32200ce8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56496/0 (socket says 172.21.15.7:56496) 2024-04-25T11:20:57.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.853+0000 7f321f400640 1 -- 172.21.15.7:0/3013409712 learned_addr learned my addr 172.21.15.7:0/3013409712 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.854+0000 7f321ea00640 1 --2- 172.21.15.7:0/3013409712 >> v2:172.21.15.87:3300/0 conn(0x7f32200d2430 0x7f32200b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.854+0000 7f321fe00640 1 -- 172.21.15.7:0/3013409712 >> v2:172.21.15.87:3300/0 conn(0x7f32200d2430 msgr2=0x7f32200b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.854+0000 7f321fe00640 1 --2- 172.21.15.7:0/3013409712 >> v2:172.21.15.87:3300/0 conn(0x7f32200d2430 0x7f32200b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.854+0000 7f321fe00640 1 -- 172.21.15.7:0/3013409712 >> v1:172.21.15.7:6789/0 conn(0x7f32200b8770 legacy=0x7f32200ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.854+0000 7f321fe00640 1 -- 172.21.15.7:0/3013409712 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32200d0e20 con 0x7f32200b3080 2024-04-25T11:20:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.855+0000 7f321fe00640 1 --2- 172.21.15.7:0/3013409712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200b3080 0x7f32201620e0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f3220074040 tx=0x7f321802a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2f6689bd2cafd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.855+0000 7f321e000640 1 -- 172.21.15.7:0/3013409712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218007740 con 0x7f32200b3080 2024-04-25T11:20:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.855+0000 7f321e000640 1 -- 172.21.15.7:0/3013409712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32180078e0 con 0x7f32200b3080 2024-04-25T11:20:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.855+0000 7f321e000640 1 -- 172.21.15.7:0/3013409712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3218007cb0 con 0x7f32200b3080 2024-04-25T11:20:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.856+0000 7f3224c00640 1 -- 172.21.15.7:0/3013409712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200b3080 msgr2=0x7f32201620e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.856+0000 7f3224c00640 1 --2- 172.21.15.7:0/3013409712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200b3080 0x7f32201620e0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f3220074040 tx=0x7f321802a550 comp rx=0 tx=0).stop 2024-04-25T11:20:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.856+0000 7f3224c00640 1 -- 172.21.15.7:0/3013409712 shutdown_connections 2024-04-25T11:20:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.856+0000 7f3224c00640 1 --2- 172.21.15.7:0/3013409712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200b3080 0x7f32201620e0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.856+0000 7f3224c00640 1 --2- 172.21.15.7:0/3013409712 >> v2:172.21.15.87:3300/0 conn(0x7f32200d2430 0x7f32200b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.856+0000 7f3224c00640 1 -- 172.21.15.7:0/3013409712 >> 172.21.15.7:0/3013409712 conn(0x7f32200d01d0 msgr2=0x7f32200d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.856+0000 7f3224c00640 1 -- 172.21.15.7:0/3013409712 shutdown_connections 2024-04-25T11:20:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.856+0000 7f3224c00640 1 -- 172.21.15.7:0/3013409712 wait complete. 2024-04-25T11:20:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.858+0000 7f3224c00640 1 Processor -- start 2024-04-25T11:20:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.859+0000 7f3224c00640 1 -- start start 2024-04-25T11:20:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.860+0000 7f3224c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32200b3080 0x7f32201603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.860+0000 7f3224c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200b8770 0x7f32200b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.860+0000 7f3224c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32200d2430 0x7f3220158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.860+0000 7f3224c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f322008cad0 con 0x7f32200b8770 2024-04-25T11:20:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.860+0000 7f3224c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f322009cf30 con 0x7f32200d2430 2024-04-25T11:20:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.860+0000 7f3224c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f322008cc50 con 0x7f32200b3080 2024-04-25T11:20:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.860+0000 7f321ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200b8770 0x7f32200b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.860+0000 7f321ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200b8770 0x7f32200b1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56810/0 (socket says 172.21.15.7:56810) 2024-04-25T11:20:57.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.860+0000 7f321ea00640 1 -- 172.21.15.7:0/1097557414 learned_addr learned my addr 172.21.15.7:0/1097557414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.861+0000 7f321fe00640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32200d2430 0x7f3220158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.861+0000 7f321ea00640 1 -- 172.21.15.7:0/1097557414 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32200b3080 msgr2=0x7f32201603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.861+0000 7f321ea00640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32200b3080 0x7f32201603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.861+0000 7f321ea00640 1 -- 172.21.15.7:0/1097557414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32200d2430 msgr2=0x7f3220158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.861+0000 7f321ea00640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32200d2430 0x7f3220158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.861+0000 7f321ea00640 1 -- 172.21.15.7:0/1097557414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32200ca0b0 con 0x7f32200b8770 2024-04-25T11:20:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.861+0000 7f321ea00640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200b8770 0x7f32200b1570 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f320c00d5a0 tx=0x7f320c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.861+0000 7f3217400640 1 -- 172.21.15.7:0/1097557414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c019070 con 0x7f32200b8770 2024-04-25T11:20:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.862+0000 7f3217400640 1 -- 172.21.15.7:0/1097557414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320c004240 con 0x7f32200b8770 2024-04-25T11:20:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.862+0000 7f3217400640 1 -- 172.21.15.7:0/1097557414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c003610 con 0x7f32200b8770 2024-04-25T11:20:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.862+0000 7f3224c00640 1 -- 172.21.15.7:0/1097557414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32200cd660 con 0x7f32200b8770 2024-04-25T11:20:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.862+0000 7f3224c00640 1 -- 172.21.15.7:0/1097557414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32200c8830 con 0x7f32200b8770 2024-04-25T11:20:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.863+0000 7f3224c00640 1 -- 172.21.15.7:0/1097557414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32200c8340 con 0x7f32200b8770 2024-04-25T11:20:57.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.865+0000 7f3217400640 1 -- 172.21.15.7:0/1097557414 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f320c01c070 con 0x7f32200b8770 2024-04-25T11:20:57.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.866+0000 7f3217400640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744b0 0x7f32000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:57.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.866+0000 7f3217400640 1 -- 172.21.15.7:0/1097557414 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f320c094c10 con 0x7f32200b8770 2024-04-25T11:20:57.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.866+0000 7f321f400640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744b0 0x7f32000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:57.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.867+0000 7f321f400640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744b0 0x7f32000769c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3210000f60 tx=0x7f3210007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:57.870+0000 7f3217400640 1 -- 172.21.15.7:0/1097557414 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f320c05ebc0 con 0x7f32200b8770 2024-04-25T11:20:58.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.077+0000 7f3224c00640 1 -- 172.21.15.7:0/1097557414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7f32200cd3f0 con 0x7f32200b8770 2024-04-25T11:20:58.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.078+0000 7f3217400640 1 -- 172.21.15.7:0/1097557414 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v50) ==== 112+0+10 (secure 0 0 0) 0x7f320c062ec0 con 0x7f32200b8770 2024-04-25T11:20:58.075 INFO:teuthology.orchestra.run.smithi007.stdout:60.000000 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 -- 172.21.15.7:0/1097557414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744b0 msgr2=0x7f32000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744b0 0x7f32000769c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3210000f60 tx=0x7f3210007a80 comp rx=0 tx=0).stop 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 -- 172.21.15.7:0/1097557414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200b8770 msgr2=0x7f32200b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200b8770 0x7f32200b1570 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f320c00d5a0 tx=0x7f320c00da60 comp rx=0 tx=0).stop 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 -- 172.21.15.7:0/1097557414 shutdown_connections 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f32000744b0 0x7f32000769c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32200d2430 0x7f3220158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32200b8770 0x7f32200b1570 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 --2- 172.21.15.7:0/1097557414 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32200b3080 0x7f32201603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 -- 172.21.15.7:0/1097557414 >> 172.21.15.7:0/1097557414 conn(0x7f32200d01d0 msgr2=0x7f32200ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 -- 172.21.15.7:0/1097557414 shutdown_connections 2024-04-25T11:20:58.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.081+0000 7f3224c00640 1 -- 172.21.15.7:0/1097557414 wait complete. 2024-04-25T11:20:58.109 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:20:58.109 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:58.161 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:20:58.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:20:58.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:20:58.287 INFO:teuthology.orchestra.run.smithi007.stdout:client6302 2024-04-25T11:20:58.287 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:20:58.287 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:20:58.340 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-04-25T11:20:58.340 DEBUG:teuthology.orchestra.run.smithi007:> import fcntl 2024-04-25T11:20:58.340 DEBUG:teuthology.orchestra.run.smithi007:> import sys 2024-04-25T11:20:58.340 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-04-25T11:20:58.340 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-04-25T11:20:58.340 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:20:58.340 DEBUG:teuthology.orchestra.run.smithi007:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> n = 200 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> abs_path = "/home/ubuntu/cephtest/mnt.0/subdir" 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> if not os.path.exists(abs_path): 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(abs_path) 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> handles = [] 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(0, n): 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> fname = "file_"+str(i) 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> path = os.path.join(abs_path, fname) 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> handles.append(open(path, '"'"'w'"'"')) 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> print("waiting with handles open", file=sys.stderr) 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> print("open_n_background: keeping files open", file=sys.stderr) 2024-04-25T11:20:58.341 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-04-25T11:20:58.342 DEBUG:teuthology.orchestra.run.smithi007:> if os.read(0, 4096) == b"": 2024-04-25T11:20:58.342 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-04-25T11:20:58.342 DEBUG:teuthology.orchestra.run.smithi007:> except BlockingIOError: 2024-04-25T11:20:58.342 DEBUG:teuthology.orchestra.run.smithi007:> pass 2024-04-25T11:20:58.342 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(2) 2024-04-25T11:20:58.342 DEBUG:teuthology.orchestra.run.smithi007:> print("stdin closed, goodbye!", file=sys.stderr) 2024-04-25T11:20:58.342 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-04-25T11:20:58.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:20:58.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.620+0000 7fcd03c00640 1 Processor -- start 2024-04-25T11:20:58.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.620+0000 7fcd03c00640 1 -- start start 2024-04-25T11:20:58.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.623+0000 7fcd03c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fccfc0b0290 0x7fccfc0b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.623+0000 7fcd03c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fccfc06f300 0x7fccfc0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.623+0000 7fcd03c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccfc0b4980 con 0x7fccfc06dc90 2024-04-25T11:20:58.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.623+0000 7fcd03c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fccfc0b4c60 con 0x7fccfc0b0290 2024-04-25T11:20:58.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.623+0000 7fcd03c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fccfc0b4f40 con 0x7fccfc06f300 2024-04-25T11:20:58.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.623+0000 7fccfbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fccfc0b0290 0x7fccfc0b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.623+0000 7fccfbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fccfc0b0290 0x7fccfc0b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56816/0 (socket says 172.21.15.7:56816) 2024-04-25T11:20:58.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.623+0000 7fccfbe00640 1 -- 172.21.15.7:0/3070625450 learned_addr learned my addr 172.21.15.7:0/3070625450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:58.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.623+0000 7fcd01c00640 1 --2- 172.21.15.7:0/3070625450 >> v2:172.21.15.87:3300/0 conn(0x7fccfc06f300 0x7fccfc0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.624+0000 7fccfbe00640 1 -- 172.21.15.7:0/3070625450 >> v2:172.21.15.87:3300/0 conn(0x7fccfc06f300 msgr2=0x7fccfc0b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:58.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.624+0000 7fccfbe00640 1 --2- 172.21.15.7:0/3070625450 >> v2:172.21.15.87:3300/0 conn(0x7fccfc06f300 0x7fccfc0b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.624+0000 7fccfbe00640 1 -- 172.21.15.7:0/3070625450 >> v1:172.21.15.87:6789/0 conn(0x7fccfc06dc90 legacy=0x7fccfc0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:58.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.624+0000 7fccfbe00640 1 -- 172.21.15.7:0/3070625450 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc0b5220 con 0x7fccfc0b0290 2024-04-25T11:20:58.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.625+0000 7fccfbe00640 1 --2- 172.21.15.7:0/3070625450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc0b0290 0x7fccfc0b5a20 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fccfc071e80 tx=0x7fcce802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb149e6b814171a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.625+0000 7fccfb400640 1 -- 172.21.15.7:0/3070625450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8036070 con 0x7fccfc0b0290 2024-04-25T11:20:58.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.625+0000 7fccfb400640 1 -- 172.21.15.7:0/3070625450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce8031600 con 0x7fccfc0b0290 2024-04-25T11:20:58.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.625+0000 7fccfb400640 1 -- 172.21.15.7:0/3070625450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80319d0 con 0x7fccfc0b0290 2024-04-25T11:20:58.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.626+0000 7fcd03c00640 1 -- 172.21.15.7:0/3070625450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc0b0290 msgr2=0x7fccfc0b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.626+0000 7fcd03c00640 1 --2- 172.21.15.7:0/3070625450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc0b0290 0x7fccfc0b5a20 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fccfc071e80 tx=0x7fcce802a6d0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.626+0000 7fcd03c00640 1 -- 172.21.15.7:0/3070625450 shutdown_connections 2024-04-25T11:20:58.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.626+0000 7fcd03c00640 1 --2- 172.21.15.7:0/3070625450 >> v2:172.21.15.87:3300/0 conn(0x7fccfc06f300 0x7fccfc0b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.626+0000 7fcd03c00640 1 --2- 172.21.15.7:0/3070625450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc0b0290 0x7fccfc0b5a20 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.626+0000 7fcd03c00640 1 -- 172.21.15.7:0/3070625450 >> 172.21.15.7:0/3070625450 conn(0x7fccfc074200 msgr2=0x7fccfc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:58.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.626+0000 7fcd03c00640 1 -- 172.21.15.7:0/3070625450 shutdown_connections 2024-04-25T11:20:58.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.626+0000 7fcd03c00640 1 -- 172.21.15.7:0/3070625450 wait complete. 2024-04-25T11:20:58.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.629+0000 7fcd03c00640 1 Processor -- start 2024-04-25T11:20:58.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.629+0000 7fcd03c00640 1 -- start start 2024-04-25T11:20:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.630+0000 7fcd03c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccfc06f300 0x7fccfc0b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.630+0000 7fcd03c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fccfc1494b0 0x7fccfc157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.630+0000 7fcd03c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc1498c0 0x7fccfc157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.630+0000 7fcd03c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc0ce920 con 0x7fccfc1498c0 2024-04-25T11:20:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.630+0000 7fcd03c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc15d220 con 0x7fccfc06f300 2024-04-25T11:20:58.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.630+0000 7fcd03c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fccfc0ceaa0 con 0x7fccfc1494b0 2024-04-25T11:20:58.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.631+0000 7fcd01c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc1498c0 0x7fccfc157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.631+0000 7fcd01c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc1498c0 0x7fccfc157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56830/0 (socket says 172.21.15.7:56830) 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.631+0000 7fcd01c00640 1 -- 172.21.15.7:0/473955923 learned_addr learned my addr 172.21.15.7:0/473955923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.631+0000 7fcd01200640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccfc06f300 0x7fccfc0b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.631+0000 7fccfbe00640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fccfc1494b0 0x7fccfc157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.631+0000 7fcd01c00640 1 -- 172.21.15.7:0/473955923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fccfc1494b0 msgr2=0x7fccfc157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.631+0000 7fcd01c00640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fccfc1494b0 0x7fccfc157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.631+0000 7fcd01c00640 1 -- 172.21.15.7:0/473955923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccfc06f300 msgr2=0x7fccfc0b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.631+0000 7fcd01c00640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccfc06f300 0x7fccfc0b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.631+0000 7fcd01c00640 1 -- 172.21.15.7:0/473955923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc0b2f50 con 0x7fccfc1498c0 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.632+0000 7fcd01c00640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc1498c0 0x7fccfc157f90 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fccf400d4e0 tx=0x7fccf400d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.632+0000 7fccf8c00640 1 -- 172.21.15.7:0/473955923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf40125f0 con 0x7fccfc1498c0 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.632+0000 7fccf8c00640 1 -- 172.21.15.7:0/473955923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf4019070 con 0x7fccfc1498c0 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.632+0000 7fcd03c00640 1 -- 172.21.15.7:0/473955923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccfc1f6020 con 0x7fccfc1498c0 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.632+0000 7fccf8c00640 1 -- 172.21.15.7:0/473955923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf40129f0 con 0x7fccfc1498c0 2024-04-25T11:20:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.632+0000 7fcd03c00640 1 -- 172.21.15.7:0/473955923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccfc1f65e0 con 0x7fccfc1498c0 2024-04-25T11:20:58.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.634+0000 7fcd03c00640 1 -- 172.21.15.7:0/473955923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccfc15cae0 con 0x7fccfc1498c0 2024-04-25T11:20:58.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.635+0000 7fccf8c00640 1 -- 172.21.15.7:0/473955923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fccf4012b90 con 0x7fccfc1498c0 2024-04-25T11:20:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.636+0000 7fccf8c00640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fccdc0744b0 0x7fccdc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.636+0000 7fccf8c00640 1 -- 172.21.15.7:0/473955923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7fccf4095780 con 0x7fccfc1498c0 2024-04-25T11:20:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.636+0000 7fcd01200640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fccdc0744b0 0x7fccdc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.637+0000 7fcd01200640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fccdc0744b0 0x7fccdc0769c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fccec006fd0 tx=0x7fccec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:58.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.641+0000 7fccf8c00640 1 -- 172.21.15.7:0/473955923 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fccfc15cae0 con 0x7fccfc1498c0 2024-04-25T11:20:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.890+0000 7fcd03c00640 1 -- 172.21.15.7:0/473955923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccfc06a100 con 0x7fccfc1498c0 2024-04-25T11:20:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.891+0000 7fccf8c00640 1 -- 172.21.15.7:0/473955923 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5174 (secure 0 0 0) 0x7fccf405fd70 con 0x7fccfc1498c0 2024-04-25T11:20:58.887 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:58.888 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2288158899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2288158899},{"type":"v1","addr":"172.21.15.7:6837","nonce":2288158899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3545919245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3545919245},{"type":"v1","addr":"172.21.15.87:6835","nonce":3545919245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8478,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/951407928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":951407928},{"type":"v1","addr":"172.21.15.7:6836","nonce":951407928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:42.908797+0000","modified":"2024-04-25T11:20:46.008582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6833/1243096872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1243096872},{"type":"v1","addr":"172.21.15.87:6833","nonce":1243096872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8484,"qdb_cluster":[8484]},"id":9}]} 2024-04-25T11:20:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 89 2024-04-25T11:20:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 -- 172.21.15.7:0/473955923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fccdc0744b0 msgr2=0x7fccdc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fccdc0744b0 0x7fccdc0769c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fccec006fd0 tx=0x7fccec008040 comp rx=0 tx=0).stop 2024-04-25T11:20:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 -- 172.21.15.7:0/473955923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc1498c0 msgr2=0x7fccfc157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc1498c0 0x7fccfc157f90 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fccf400d4e0 tx=0x7fccf400d9a0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 -- 172.21.15.7:0/473955923 shutdown_connections 2024-04-25T11:20:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fccdc0744b0 0x7fccdc0769c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc1498c0 0x7fccfc157f90 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fccfc1494b0 0x7fccfc157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 --2- 172.21.15.7:0/473955923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fccfc06f300 0x7fccfc0b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 -- 172.21.15.7:0/473955923 >> 172.21.15.7:0/473955923 conn(0x7fccfc074200 msgr2=0x7fccfc0af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 -- 172.21.15.7:0/473955923 shutdown_connections 2024-04-25T11:20:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:58.894+0000 7fcd03c00640 1 -- 172.21.15.7:0/473955923 wait complete. 2024-04-25T11:20:58.920 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-04-25T11:20:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:waiting with handles open 2024-04-25T11:20:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:20:59.210 INFO:teuthology.orchestra.run.smithi087.stdout:[{"id":6305,"entity":{"name":{"type":"client","num":6305},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3077954217}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":5.1788851009999997,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9848088339417731,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6305 192.168.0.1:0/3077954217","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi087","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":6302,"entity":{"name":{"type":"client","num":6302},"addr":{"type":"any","addr":"172.21.15.7:0","nonce":3184181386}},"state":"open","num_leases":1,"num_caps":202,"request_load_avg":225,"uptime":6.9398460310000001,"requests_in_flight":200,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":404.57157168419587,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6302 172.21.15.7:0/3184181386","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi007","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T11:20:59.211 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi087", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3077954217, "type": "any" }, "name": { "num": 6305, "type": "client" } }, "id": 6305, "inst": "client.6305 192.168.0.1:0/3077954217", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.984808833941773 }, "state": "open", "uptime": 5.178885101 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi007", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.7:0", "nonce": 3184181386, "type": "any" }, "name": { "num": 6302, "type": "client" } }, "id": 6302, "inst": "client.6302 172.21.15.7:0/3184181386", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 202, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 225, "requests_in_flight": 200, "session_cache_liveness": { "halflife": 300, "value": 404.5715716841959 }, "state": "open", "uptime": 6.939846031 } ] 2024-04-25T11:20:59.211 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-04-25T11:20:59.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:20:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.513+0000 7f584be00640 1 Processor -- start 2024-04-25T11:20:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.514+0000 7f584be00640 1 -- start start 2024-04-25T11:20:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.516+0000 7f584be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f584c15e2f0 0x7f584c161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.516+0000 7f584be00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f584c15e700 con 0x7f584c15baa0 2024-04-25T11:20:59.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.516+0000 7f584be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f584c15e980 con 0x7f584c162420 2024-04-25T11:20:59.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.516+0000 7f584be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f584c162830 con 0x7f584c15e2f0 2024-04-25T11:20:59.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.516+0000 7f584aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f584c15baa0 0x7f584c070640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56510/0 (socket says 172.21.15.7:56510) 2024-04-25T11:20:59.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.516+0000 7f584aa00640 1 -- 172.21.15.7:0/3768213350 learned_addr learned my addr 172.21.15.7:0/3768213350 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.517+0000 7f5842000640 1 --2- 172.21.15.7:0/3768213350 >> v2:172.21.15.87:3300/0 conn(0x7f584c15e2f0 0x7f584c161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.517+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1067033481 0 0) 0x7f583c001110 con 0x7f584c15baa0 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.517+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5830003520 con 0x7f584c15baa0 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.517+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 471091504 0 0) 0x7f5830003520 con 0x7f584c15baa0 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.517+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 >> v2:172.21.15.87:3300/0 conn(0x7f584c15e2f0 msgr2=0x7f584c161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.517+0000 7f584a000640 1 --2- 172.21.15.7:0/3768213350 >> v2:172.21.15.87:3300/0 conn(0x7f584c15e2f0 0x7f584c161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.517+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 >> v1:172.21.15.87:6789/0 conn(0x7f584c162420 legacy=0x7f584c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.517+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c15ec60 con 0x7f584c15baa0 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.517+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f583c001840 con 0x7f584c15baa0 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 >> v1:172.21.15.7:6789/0 conn(0x7f584c15baa0 legacy=0x7f584c070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584a000640 1 --2- 172.21.15.7:0/3768213350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5830003a00 0x7f5830003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584a000640 1 --2- 172.21.15.7:0/3768213350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5830004800 0x7f5830004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584a000640 1 --2- 172.21.15.7:0/3768213350 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5830005270 0x7f583000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f584c162830 con 0x7f5830003a00 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f583000c190 con 0x7f5830004800 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f583000c390 con 0x7f5830005270 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584aa00640 1 --2- 172.21.15.7:0/3768213350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5830003a00 0x7f5830003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f5842000640 1 --2- 172.21.15.7:0/3768213350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5830004800 0x7f5830004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584aa00640 1 -- 172.21.15.7:0/3768213350 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5830005270 msgr2=0x7f583000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584aa00640 1 --2- 172.21.15.7:0/3768213350 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5830005270 0x7f583000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584aa00640 1 -- 172.21.15.7:0/3768213350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5830004800 msgr2=0x7f5830004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584aa00640 1 --2- 172.21.15.7:0/3768213350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5830004800 0x7f5830004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584aa00640 1 -- 172.21.15.7:0/3768213350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5830003520 con 0x7f5830003a00 2024-04-25T11:20:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.518+0000 7f584b400640 1 -- 172.21.15.7:0/3768213350 reap_dead start 2024-04-25T11:20:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.519+0000 7f584aa00640 1 --2- 172.21.15.7:0/3768213350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5830003a00 0x7f5830003e70 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f583c00c320 tx=0x7f5830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfa16c04ff0dc8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.520+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c007e10 con 0x7f5830003a00 2024-04-25T11:20:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.520+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f583c005ad0 con 0x7f5830003a00 2024-04-25T11:20:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.520+0000 7f584a000640 1 -- 172.21.15.7:0/3768213350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c007070 con 0x7f5830003a00 2024-04-25T11:20:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.520+0000 7f584be00640 1 -- 172.21.15.7:0/3768213350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5830003a00 msgr2=0x7f5830003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.520+0000 7f584be00640 1 --2- 172.21.15.7:0/3768213350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5830003a00 0x7f5830003e70 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f583c00c320 tx=0x7f5830005830 comp rx=0 tx=0).stop 2024-04-25T11:20:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.521+0000 7f584be00640 1 -- 172.21.15.7:0/3768213350 shutdown_connections 2024-04-25T11:20:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.521+0000 7f584be00640 1 --2- 172.21.15.7:0/3768213350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5830003a00 0x7f5830003e70 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.521+0000 7f584be00640 1 -- 172.21.15.7:0/3768213350 >> 172.21.15.7:0/3768213350 conn(0x7f584c073090 msgr2=0x7f584c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.521+0000 7f584be00640 1 -- 172.21.15.7:0/3768213350 shutdown_connections 2024-04-25T11:20:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.521+0000 7f584be00640 1 -- 172.21.15.7:0/3768213350 wait complete. 2024-04-25T11:20:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.523+0000 7f584be00640 1 Processor -- start 2024-04-25T11:20:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.523+0000 7f584be00640 1 -- start start 2024-04-25T11:20:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.525+0000 7f584be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5830003a00 0x7f584c15da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.525+0000 7f584be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f584c0b0130 0x7f584c1f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.525+0000 7f584be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f584c1e7740 0x7f584c1e7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.525+0000 7f584be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f584c15baa0 con 0x7f584c1e7740 2024-04-25T11:20:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.525+0000 7f584be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1619c0 con 0x7f584c0b0130 2024-04-25T11:20:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.525+0000 7f584be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f584c15bc20 con 0x7f5830003a00 2024-04-25T11:20:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.525+0000 7f584b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f584c1e7740 0x7f584c1e7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.525+0000 7f584b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f584c1e7740 0x7f584c1e7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56848/0 (socket says 172.21.15.7:56848) 2024-04-25T11:20:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.525+0000 7f584b400640 1 -- 172.21.15.7:0/2966819396 learned_addr learned my addr 172.21.15.7:0/2966819396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:20:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f584aa00640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5830003a00 0x7f584c15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f5842000640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f584c0b0130 0x7f584c1f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f584b400640 1 -- 172.21.15.7:0/2966819396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5830003a00 msgr2=0x7f584c15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f584b400640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5830003a00 0x7f584c15da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f584b400640 1 -- 172.21.15.7:0/2966819396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f584c0b0130 msgr2=0x7f584c1f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f584b400640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f584c0b0130 0x7f584c1f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f584b400640 1 -- 172.21.15.7:0/2966819396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c1e8090 con 0x7f584c1e7740 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f584aa00640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5830003a00 0x7f584c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f584b400640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f584c1e7740 0x7f584c1e7b50 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f584c071500 tx=0x7f584400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f5843400640 1 -- 172.21.15.7:0/2966819396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844017070 con 0x7f584c1e7740 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f5843400640 1 -- 172.21.15.7:0/2966819396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5844005d40 con 0x7f584c1e7740 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f584be00640 1 -- 172.21.15.7:0/2966819396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584c0aeb90 con 0x7f584c1e7740 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.526+0000 7f5843400640 1 -- 172.21.15.7:0/2966819396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584400a850 con 0x7f584c1e7740 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.527+0000 7f584be00640 1 -- 172.21.15.7:0/2966819396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584c0af090 con 0x7f584c1e7740 2024-04-25T11:20:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.528+0000 7f584be00640 1 -- 172.21.15.7:0/2966819396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f584c0aee00 con 0x7f584c1e7740 2024-04-25T11:20:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.529+0000 7f5843400640 1 -- 172.21.15.7:0/2966819396 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f584401a070 con 0x7f584c1e7740 2024-04-25T11:20:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.530+0000 7f5843400640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58280744b0 0x7f58280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:20:59.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.530+0000 7f5843400640 1 -- 172.21.15.7:0/2966819396 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f5844094610 con 0x7f584c1e7740 2024-04-25T11:20:59.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.531+0000 7f584aa00640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58280744b0 0x7f58280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:20:59.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.532+0000 7f584aa00640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58280744b0 0x7f58280769c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f583c033000 tx=0x7f583c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:20:59.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.535+0000 7f5843400640 1 -- 172.21.15.7:0/2966819396 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f584c1e8090 con 0x7f584c1e7740 2024-04-25T11:20:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.779+0000 7f584be00640 1 -- 172.21.15.7:0/2966819396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f584c0ae970 con 0x7f584c1e7740 2024-04-25T11:20:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.780+0000 7f5843400640 1 -- 172.21.15.7:0/2966819396 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f584405ec00 con 0x7f584c1e7740 2024-04-25T11:20:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:20:59.777 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:20:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 -- 172.21.15.7:0/2966819396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58280744b0 msgr2=0x7f58280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58280744b0 0x7f58280769c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f583c033000 tx=0x7f583c02e000 comp rx=0 tx=0).stop 2024-04-25T11:20:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 -- 172.21.15.7:0/2966819396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f584c1e7740 msgr2=0x7f584c1e7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:20:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f584c1e7740 0x7f584c1e7b50 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f584c071500 tx=0x7f584400cb90 comp rx=0 tx=0).stop 2024-04-25T11:20:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 -- 172.21.15.7:0/2966819396 shutdown_connections 2024-04-25T11:20:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f58280744b0 0x7f58280769c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f584c1e7740 0x7f584c1e7b50 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f584c0b0130 0x7f584c1f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 --2- 172.21.15.7:0/2966819396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5830003a00 0x7f584c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:20:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 -- 172.21.15.7:0/2966819396 >> 172.21.15.7:0/2966819396 conn(0x7f584c073090 msgr2=0x7f584c0705d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:20:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 -- 172.21.15.7:0/2966819396 shutdown_connections 2024-04-25T11:20:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:20:59.783+0000 7f584be00640 1 -- 172.21.15.7:0/2966819396 wait complete. 2024-04-25T11:20:59.806 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-04-25T11:20:59.807 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:21:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:21:02.988 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:21:04.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:21:04.990 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:21:05.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.097+0000 7fbdf0a00640 1 Processor -- start 2024-04-25T11:21:05.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.098+0000 7fbdf0a00640 1 -- start start 2024-04-25T11:21:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.100+0000 7fbdf0a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbdec15e2f0 0x7fbdec161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.100+0000 7fbdf0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec162420 0x7fbdec06d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.100+0000 7fbdf0a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdec15e700 con 0x7fbdec15baa0 2024-04-25T11:21:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.100+0000 7fbdf0a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbdec162830 con 0x7fbdec15e2f0 2024-04-25T11:21:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.100+0000 7fbdf0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbdec15e980 con 0x7fbdec162420 2024-04-25T11:21:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.100+0000 7fbdea800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbdec15baa0 0x7fbdec070640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47290/0 (socket says 172.21.15.7:47290) 2024-04-25T11:21:05.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.100+0000 7fbdea800640 1 -- 172.21.15.7:0/3789619347 learned_addr learned my addr 172.21.15.7:0/3789619347 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbde1e00640 1 --2- 172.21.15.7:0/3789619347 >> v2:172.21.15.87:3300/0 conn(0x7fbdec15e2f0 0x7fbdec161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbdeb200640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec162420 0x7fbdec06d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2056520398 0 0) 0x7fbddc001110 con 0x7fbdec15baa0 2024-04-25T11:21:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdd0003520 con 0x7fbdec15baa0 2024-04-25T11:21:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 312813705 0 0) 0x7fbdd0003520 con 0x7fbdec15baa0 2024-04-25T11:21:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec162420 msgr2=0x7fbdec06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbde9e00640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec162420 0x7fbdec06d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 >> v2:172.21.15.87:3300/0 conn(0x7fbdec15e2f0 msgr2=0x7fbdec161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbde9e00640 1 --2- 172.21.15.7:0/3789619347 >> v2:172.21.15.87:3300/0 conn(0x7fbdec15e2f0 0x7fbdec161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec15ec60 con 0x7fbdec15baa0 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fbddc001860 con 0x7fbdec15baa0 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.101+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 >> v1:172.21.15.7:6789/0 conn(0x7fbdec15baa0 legacy=0x7fbdec070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde9e00640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec162420 0x7fbdd00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde9e00640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 0x7fbdd0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde9e00640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd0004d40 0x7fbdd0005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec162830 con 0x7fbdd0003f80 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec15e980 con 0x7fbdd0004d40 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd000ba70 con 0x7fbdec162420 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde1e00640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 0x7fbdd0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbdeb200640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd0004d40 0x7fbdd0005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbdea800640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec162420 0x7fbdd00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde1e00640 1 -- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec162420 msgr2=0x7fbdd00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde1e00640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec162420 0x7fbdd00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde1e00640 1 -- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd0004d40 msgr2=0x7fbdd0005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde1e00640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdd0004d40 0x7fbdd0005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.102+0000 7fbde1e00640 1 -- 172.21.15.7:0/3789619347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd0003520 con 0x7fbdd0003f80 2024-04-25T11:21:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.103+0000 7fbdea800640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec162420 0x7fbdd00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.104+0000 7fbde1e00640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 0x7fbdd0004800 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009ac0 tx=0x7fbdd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c742d36acdf4eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.104+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4036070 con 0x7fbdd0003f80 2024-04-25T11:21:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.104+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd4031400 con 0x7fbdd0003f80 2024-04-25T11:21:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.104+0000 7fbde9e00640 1 -- 172.21.15.7:0/3789619347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4009690 con 0x7fbdd0003f80 2024-04-25T11:21:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.105+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3789619347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 msgr2=0x7fbdd0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.105+0000 7fbdf0a00640 1 --2- 172.21.15.7:0/3789619347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 0x7fbdd0004800 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009ac0 tx=0x7fbdd0005830 comp rx=0 tx=0).stop 2024-04-25T11:21:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.105+0000 7fbdeb200640 1 -- 172.21.15.7:0/3789619347 reap_dead start 2024-04-25T11:21:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.105+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3789619347 shutdown_connections 2024-04-25T11:21:05.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.105+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3789619347 >> 172.21.15.7:0/3789619347 conn(0x7fbdec073090 msgr2=0x7fbdec0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:05.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.105+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3789619347 shutdown_connections 2024-04-25T11:21:05.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.105+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3789619347 wait complete. 2024-04-25T11:21:05.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.107+0000 7fbdf0a00640 1 Processor -- start 2024-04-25T11:21:05.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.108+0000 7fbdf0a00640 1 -- start start 2024-04-25T11:21:05.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.109+0000 7fbdf0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 0x7fbdec15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.109+0000 7fbdf0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdec141500 0x7fbdec1e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.109+0000 7fbdf0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec1f4480 0x7fbdec1e8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.109+0000 7fbdf0a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec06d580 con 0x7fbdd0003f80 2024-04-25T11:21:05.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.109+0000 7fbdf0a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec1619c0 con 0x7fbdec141500 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.109+0000 7fbdf0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbdec06d700 con 0x7fbdec1f4480 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.109+0000 7fbdea800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 0x7fbdec15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.110+0000 7fbdea800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 0x7fbdec15a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57388/0 (socket says 172.21.15.7:57388) 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.110+0000 7fbdea800640 1 -- 172.21.15.7:0/3048676152 learned_addr learned my addr 172.21.15.7:0/3048676152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.110+0000 7fbde1e00640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdec141500 0x7fbdec1e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.110+0000 7fbdeb200640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec1f4480 0x7fbdec1e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.110+0000 7fbdea800640 1 -- 172.21.15.7:0/3048676152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec1f4480 msgr2=0x7fbdec1e8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.110+0000 7fbdea800640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec1f4480 0x7fbdec1e8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.110+0000 7fbdea800640 1 -- 172.21.15.7:0/3048676152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdec141500 msgr2=0x7fbdec1e7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.110+0000 7fbdea800640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdec141500 0x7fbdec1e7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.110+0000 7fbdea800640 1 -- 172.21.15.7:0/3048676152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec15ab10 con 0x7fbdd0003f80 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.110+0000 7fbde1e00640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdec141500 0x7fbdec1e7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.110+0000 7fbdeb200640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec1f4480 0x7fbdec1e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:05.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.111+0000 7fbdea800640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 0x7fbdec15a5d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00d8e0 tx=0x7fbddc00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.111+0000 7fbde3400640 1 -- 172.21.15.7:0/3048676152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc019070 con 0x7fbdd0003f80 2024-04-25T11:21:05.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.111+0000 7fbde3400640 1 -- 172.21.15.7:0/3048676152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbddc0074e0 con 0x7fbdd0003f80 2024-04-25T11:21:05.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.111+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3048676152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdec143a00 con 0x7fbdd0003f80 2024-04-25T11:21:05.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.111+0000 7fbde3400640 1 -- 172.21.15.7:0/3048676152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0049e0 con 0x7fbdd0003f80 2024-04-25T11:21:05.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.111+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3048676152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdec143fc0 con 0x7fbdd0003f80 2024-04-25T11:21:05.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.112+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3048676152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdec074020 con 0x7fbdd0003f80 2024-04-25T11:21:05.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.113+0000 7fbde3400640 1 -- 172.21.15.7:0/3048676152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbddc01c070 con 0x7fbdd0003f80 2024-04-25T11:21:05.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.114+0000 7fbde3400640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdc80744b0 0x7fbdc80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.115+0000 7fbde3400640 1 -- 172.21.15.7:0/3048676152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7fbddc094fe0 con 0x7fbdd0003f80 2024-04-25T11:21:05.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.115+0000 7fbde1e00640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdc80744b0 0x7fbdc80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.116+0000 7fbde1e00640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdc80744b0 0x7fbdc80769c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4033000 tx=0x7fbdd4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.120+0000 7fbde3400640 1 -- 172.21.15.7:0/3048676152 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbdec15e700 con 0x7fbdd0003f80 2024-04-25T11:21:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.362+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3048676152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbdec06e0b0 con 0x7fbdd0003f80 2024-04-25T11:21:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.364+0000 7fbde3400640 1 -- 172.21.15.7:0/3048676152 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7fbddc05f580 con 0x7fbdd0003f80 2024-04-25T11:21:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:05.361 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:21:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3048676152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdc80744b0 msgr2=0x7fbdc80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdc80744b0 0x7fbdc80769c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4033000 tx=0x7fbdd4008030 comp rx=0 tx=0).stop 2024-04-25T11:21:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3048676152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 msgr2=0x7fbdec15a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 0x7fbdec15a5d0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00d8e0 tx=0x7fbddc00dda0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3048676152 shutdown_connections 2024-04-25T11:21:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbdc80744b0 0x7fbdc80769c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbdec1f4480 0x7fbdec1e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbdec141500 0x7fbdec1e7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 --2- 172.21.15.7:0/3048676152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdd0003f80 0x7fbdec15a5d0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3048676152 >> 172.21.15.7:0/3048676152 conn(0x7fbdec073090 msgr2=0x7fbdec0b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3048676152 shutdown_connections 2024-04-25T11:21:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.366+0000 7fbdf0a00640 1 -- 172.21.15.7:0/3048676152 wait complete. 2024-04-25T11:21:05.391 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:21:05.391 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-04-25T11:21:05.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:21:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.667+0000 7f6602c00640 1 Processor -- start 2024-04-25T11:21:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.667+0000 7f6602c00640 1 -- start start 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.669+0000 7f6602c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65fc070ca0 0x7f65fc073dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.670+0000 7f6602c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65fc160320 con 0x7f65fc146310 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.670+0000 7f6602c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65fc160600 con 0x7f65fc0b60a0 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.670+0000 7f6602c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65fc1608e0 con 0x7f65fc070ca0 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.670+0000 7f65fbe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f65fc146310 0x7f65fc0d0d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47302/0 (socket says 172.21.15.7:47302) 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.670+0000 7f65fbe00640 1 -- 172.21.15.7:0/2791062948 learned_addr learned my addr 172.21.15.7:0/2791062948 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.670+0000 7f65fb400640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65fc070ca0 0x7f65fc073dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1708712597 0 0) 0x7f65fc160320 con 0x7f65fc146310 2024-04-25T11:21:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65e0005730 con 0x7f65fc146310 2024-04-25T11:21:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 503667865 0 0) 0x7f65e0005730 con 0x7f65fc146310 2024-04-25T11:21:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65fc070ca0 msgr2=0x7f65fc073dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65fc070ca0 0x7f65fc073dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 >> v1:172.21.15.87:6789/0 conn(0x7f65fc0b60a0 legacy=0x7f65fc0b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc160bc0 con 0x7f65fc146310 2024-04-25T11:21:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f65ec001670 con 0x7f65fc146310 2024-04-25T11:21:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 >> v1:172.21.15.7:6789/0 conn(0x7f65fc146310 legacy=0x7f65fc0d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65fc070ca0 0x7f65e0005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e0006150 0x7f65e0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0006f50 0x7f65e0007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc1608e0 con 0x7f65e0006f50 2024-04-25T11:21:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65e000dd30 con 0x7f65e0006150 2024-04-25T11:21:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.671+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65e000df30 con 0x7f65fc070ca0 2024-04-25T11:21:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.672+0000 7f6600c00640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0006f50 0x7f65e0007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.672+0000 7f65fb400640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e0006150 0x7f65e0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.672+0000 7f65fbe00640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65fc070ca0 0x7f65e0005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.672+0000 7f6600c00640 1 -- 172.21.15.7:0/2791062948 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65fc070ca0 msgr2=0x7f65e0005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.672+0000 7f6600c00640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65fc070ca0 0x7f65e0005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.672+0000 7f6600c00640 1 -- 172.21.15.7:0/2791062948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e0006150 msgr2=0x7f65e0006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.672+0000 7f6600c00640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65e0006150 0x7f65e0006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.672+0000 7f6600c00640 1 -- 172.21.15.7:0/2791062948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e0005730 con 0x7f65e0006f50 2024-04-25T11:21:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.674+0000 7f6600c00640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0006f50 0x7f65e0007360 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f65f4001f60 tx=0x7f65e0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d28353d00df00c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.674+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4001b30 con 0x7f65e0006f50 2024-04-25T11:21:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.674+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f4031620 con 0x7f65e0006f50 2024-04-25T11:21:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.674+0000 7f65faa00640 1 -- 172.21.15.7:0/2791062948 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4031a10 con 0x7f65e0006f50 2024-04-25T11:21:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.675+0000 7f6602c00640 1 -- 172.21.15.7:0/2791062948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0006f50 msgr2=0x7f65e0007360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.675+0000 7f6602c00640 1 --2- 172.21.15.7:0/2791062948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65e0006f50 0x7f65e0007360 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f65f4001f60 tx=0x7f65e0007a40 comp rx=0 tx=0).stop 2024-04-25T11:21:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.675+0000 7f6600c00640 1 -- 172.21.15.7:0/2791062948 reap_dead start 2024-04-25T11:21:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.675+0000 7f6602c00640 1 -- 172.21.15.7:0/2791062948 shutdown_connections 2024-04-25T11:21:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.675+0000 7f6602c00640 1 -- 172.21.15.7:0/2791062948 >> 172.21.15.7:0/2791062948 conn(0x7f65fc0732b0 msgr2=0x7f65fc0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.675+0000 7f6602c00640 1 -- 172.21.15.7:0/2791062948 shutdown_connections 2024-04-25T11:21:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.675+0000 7f6602c00640 1 -- 172.21.15.7:0/2791062948 wait complete. 2024-04-25T11:21:05.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.677+0000 7f6602c00640 1 Processor -- start 2024-04-25T11:21:05.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.678+0000 7f6602c00640 1 -- start start 2024-04-25T11:21:05.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.679+0000 7f6602c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0006f50 0x7f65fc15f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.679+0000 7f6602c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65fc13fd70 0x7f65fc15bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.679+0000 7f6602c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc140180 0x7f65fc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.679+0000 7f6602c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc0741a0 con 0x7f65fc140180 2024-04-25T11:21:05.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.679+0000 7f6602c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc074020 con 0x7f65fc13fd70 2024-04-25T11:21:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.679+0000 7f6602c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65fc074320 con 0x7f65e0006f50 2024-04-25T11:21:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.680+0000 7f6600c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc140180 0x7f65fc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.680+0000 7f6600c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc140180 0x7f65fc0ae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57412/0 (socket says 172.21.15.7:57412) 2024-04-25T11:21:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.680+0000 7f6600c00640 1 -- 172.21.15.7:0/3771151889 learned_addr learned my addr 172.21.15.7:0/3771151889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.680+0000 7f65fb400640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65fc13fd70 0x7f65fc15bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.680+0000 7f65fbe00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0006f50 0x7f65fc15f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.680+0000 7f6600c00640 1 -- 172.21.15.7:0/3771151889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0006f50 msgr2=0x7f65fc15f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.680+0000 7f6600c00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0006f50 0x7f65fc15f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.680+0000 7f6600c00640 1 -- 172.21.15.7:0/3771151889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65fc13fd70 msgr2=0x7f65fc15bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.680+0000 7f6600c00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65fc13fd70 0x7f65fc15bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.680+0000 7f6600c00640 1 -- 172.21.15.7:0/3771151889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc15e380 con 0x7f65fc140180 2024-04-25T11:21:05.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.681+0000 7f6600c00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc140180 0x7f65fc0ae880 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f65f402aa00 tx=0x7f65f40015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.681+0000 7f65f3e00640 1 -- 172.21.15.7:0/3771151889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4006d60 con 0x7f65fc140180 2024-04-25T11:21:05.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.681+0000 7f65f3e00640 1 -- 172.21.15.7:0/3771151889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f40317d0 con 0x7f65fc140180 2024-04-25T11:21:05.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.681+0000 7f65f3e00640 1 -- 172.21.15.7:0/3771151889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f4031ba0 con 0x7f65fc140180 2024-04-25T11:21:05.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.681+0000 7f6602c00640 1 -- 172.21.15.7:0/3771151889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65fc13efb0 con 0x7f65fc140180 2024-04-25T11:21:05.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.681+0000 7f6602c00640 1 -- 172.21.15.7:0/3771151889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65fc13f570 con 0x7f65fc140180 2024-04-25T11:21:05.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.682+0000 7f6602c00640 1 -- 172.21.15.7:0/3771151889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65fc15e0f0 con 0x7f65fc140180 2024-04-25T11:21:05.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.683+0000 7f65f3e00640 1 -- 172.21.15.7:0/3771151889 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f65f4044020 con 0x7f65fc140180 2024-04-25T11:21:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.684+0000 7f65f3e00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65dc0744b0 0x7f65dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.685+0000 7f65f3e00640 1 -- 172.21.15.7:0/3771151889 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f65f40b5b80 con 0x7f65fc140180 2024-04-25T11:21:05.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.685+0000 7f65fbe00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65dc0744b0 0x7f65dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.686+0000 7f65fbe00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65dc0744b0 0x7f65dc0769c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f65ec001050 tx=0x7f65ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.690+0000 7f65f3e00640 1 -- 172.21.15.7:0/3771151889 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f65fc15e380 con 0x7f65fc140180 2024-04-25T11:21:05.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.935+0000 7f6602c00640 1 -- 172.21.15.7:0/3771151889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f65fc0b4b60 con 0x7f65fc140180 2024-04-25T11:21:05.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.935+0000 7f65f3e00640 1 -- 172.21.15.7:0/3771151889 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f65f40800e0 con 0x7f65fc140180 2024-04-25T11:21:05.933 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:05.933 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:21:05.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.938+0000 7f6602c00640 1 -- 172.21.15.7:0/3771151889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65dc0744b0 msgr2=0x7f65dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.938+0000 7f6602c00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65dc0744b0 0x7f65dc0769c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f65ec001050 tx=0x7f65ec009040 comp rx=0 tx=0).stop 2024-04-25T11:21:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.938+0000 7f6602c00640 1 -- 172.21.15.7:0/3771151889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc140180 msgr2=0x7f65fc0ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.938+0000 7f6602c00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc140180 0x7f65fc0ae880 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f65f402aa00 tx=0x7f65f40015b0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.938+0000 7f6602c00640 1 -- 172.21.15.7:0/3771151889 shutdown_connections 2024-04-25T11:21:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.938+0000 7f6602c00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f65dc0744b0 0x7f65dc0769c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.938+0000 7f6602c00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc140180 0x7f65fc0ae880 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.938+0000 7f6602c00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65fc13fd70 0x7f65fc15bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.938+0000 7f6602c00640 1 --2- 172.21.15.7:0/3771151889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65e0006f50 0x7f65fc15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.938+0000 7f6602c00640 1 -- 172.21.15.7:0/3771151889 >> 172.21.15.7:0/3771151889 conn(0x7f65fc0732b0 msgr2=0x7f65fc0d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.938+0000 7f6602c00640 1 -- 172.21.15.7:0/3771151889 shutdown_connections 2024-04-25T11:21:05.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05.939+0000 7f6602c00640 1 -- 172.21.15.7:0/3771151889 wait complete. 2024-04-25T11:21:05.960 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:21:05.960 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-04-25T11:21:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:05 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-25T11:21:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:21:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:stdin closed, goodbye! 2024-04-25T11:21:07.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:21:07.029 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:21:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.366+0000 7f2493200640 1 Processor -- start 2024-04-25T11:21:07.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.366+0000 7f2493200640 1 -- start start 2024-04-25T11:21:07.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.368+0000 7f2493200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f248c0b5220 0x7f248c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.368+0000 7f2493200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f248c0cebb0 con 0x7f248c0aed40 2024-04-25T11:21:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.368+0000 7f2493200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f248c0b64b0 con 0x7f248c0af420 2024-04-25T11:21:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.368+0000 7f2493200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f248c15e1f0 con 0x7f248c0b5220 2024-04-25T11:21:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.369+0000 7f2491400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f248c0b5220 0x7f248c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.369+0000 7f2491e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f248c0aed40 0x7f248c0ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47308/0 (socket says 172.21.15.7:47308) 2024-04-25T11:21:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.369+0000 7f2491e00640 1 -- 172.21.15.7:0/3423995062 learned_addr learned my addr 172.21.15.7:0/3423995062 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.370+0000 7f2491400640 1 -- 172.21.15.7:0/3423995062 >> v1:172.21.15.87:6789/0 conn(0x7f248c0af420 legacy=0x7f248c0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.370+0000 7f2491400640 1 -- 172.21.15.7:0/3423995062 >> v1:172.21.15.7:6789/0 conn(0x7f248c0aed40 legacy=0x7f248c0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.370+0000 7f2491400640 1 -- 172.21.15.7:0/3423995062 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c06c610 con 0x7f248c0b5220 2024-04-25T11:21:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.371+0000 7f2491400640 1 --2- 172.21.15.7:0/3423995062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f248c0b5220 0x7f248c15dcb0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f247c0097a0 tx=0x7f247c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bce866138b1d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.371+0000 7f2490a00640 1 -- 172.21.15.7:0/3423995062 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c037070 con 0x7f248c0b5220 2024-04-25T11:21:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.371+0000 7f2490a00640 1 -- 172.21.15.7:0/3423995062 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f247c032440 con 0x7f248c0b5220 2024-04-25T11:21:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.371+0000 7f2490a00640 1 -- 172.21.15.7:0/3423995062 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c032810 con 0x7f248c0b5220 2024-04-25T11:21:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.372+0000 7f2493200640 1 -- 172.21.15.7:0/3423995062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f248c0b5220 msgr2=0x7f248c15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.372+0000 7f2493200640 1 --2- 172.21.15.7:0/3423995062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f248c0b5220 0x7f248c15dcb0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f247c0097a0 tx=0x7f247c02a230 comp rx=0 tx=0).stop 2024-04-25T11:21:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.372+0000 7f2493200640 1 -- 172.21.15.7:0/3423995062 shutdown_connections 2024-04-25T11:21:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.372+0000 7f2493200640 1 --2- 172.21.15.7:0/3423995062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f248c0b5220 0x7f248c15dcb0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.372+0000 7f2493200640 1 -- 172.21.15.7:0/3423995062 >> 172.21.15.7:0/3423995062 conn(0x7f248c0b3820 msgr2=0x7f248c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.372+0000 7f2493200640 1 -- 172.21.15.7:0/3423995062 shutdown_connections 2024-04-25T11:21:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.372+0000 7f2493200640 1 -- 172.21.15.7:0/3423995062 wait complete. 2024-04-25T11:21:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.374+0000 7f2493200640 1 Processor -- start 2024-04-25T11:21:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.375+0000 7f2493200640 1 -- start start 2024-04-25T11:21:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.376+0000 7f2493200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f248c0aed40 0x7f248c146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.376+0000 7f2493200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c0af420 0x7f248c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.376+0000 7f2493200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f248c0b5220 0x7f248c155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.376+0000 7f2493200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f248c06ceb0 con 0x7f248c0b5220 2024-04-25T11:21:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.376+0000 7f2493200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f248c06cbd0 con 0x7f248c0af420 2024-04-25T11:21:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.376+0000 7f2493200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f248c0b6110 con 0x7f248c0aed40 2024-04-25T11:21:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.376+0000 7f2492800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f248c0b5220 0x7f248c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.377+0000 7f2492800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f248c0b5220 0x7f248c155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57436/0 (socket says 172.21.15.7:57436) 2024-04-25T11:21:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.377+0000 7f2492800640 1 -- 172.21.15.7:0/941109511 learned_addr learned my addr 172.21.15.7:0/941109511 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.377+0000 7f2491400640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c0af420 0x7f248c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.377+0000 7f2491e00640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f248c0aed40 0x7f248c146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.377+0000 7f2492800640 1 -- 172.21.15.7:0/941109511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f248c0aed40 msgr2=0x7f248c146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.377+0000 7f2492800640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f248c0aed40 0x7f248c146f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.377+0000 7f2492800640 1 -- 172.21.15.7:0/941109511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c0af420 msgr2=0x7f248c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.377+0000 7f2492800640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c0af420 0x7f248c15f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.377+0000 7f2492800640 1 -- 172.21.15.7:0/941109511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c153140 con 0x7f248c0b5220 2024-04-25T11:21:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.377+0000 7f2491400640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c0af420 0x7f248c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:21:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.377+0000 7f2492800640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f248c0b5220 0x7f248c155830 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f248800c6b0 tx=0x7f248800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.378+0000 7f2486000640 1 -- 172.21.15.7:0/941109511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488003b70 con 0x7f248c0b5220 2024-04-25T11:21:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.378+0000 7f2486000640 1 -- 172.21.15.7:0/941109511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2488003d30 con 0x7f248c0b5220 2024-04-25T11:21:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.378+0000 7f2493200640 1 -- 172.21.15.7:0/941109511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c153700 con 0x7f248c0b5220 2024-04-25T11:21:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.378+0000 7f2486000640 1 -- 172.21.15.7:0/941109511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488005310 con 0x7f248c0b5220 2024-04-25T11:21:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.378+0000 7f2493200640 1 -- 172.21.15.7:0/941109511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c153420 con 0x7f248c0b5220 2024-04-25T11:21:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.379+0000 7f2493200640 1 -- 172.21.15.7:0/941109511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248c0cd940 con 0x7f248c0b5220 2024-04-25T11:21:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.381+0000 7f2486000640 1 -- 172.21.15.7:0/941109511 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f24880054b0 con 0x7f248c0b5220 2024-04-25T11:21:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.382+0000 7f2486000640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24700744b0 0x7f24700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.382+0000 7f2486000640 1 -- 172.21.15.7:0/941109511 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f2488095290 con 0x7f248c0b5220 2024-04-25T11:21:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.382+0000 7f2491e00640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24700744b0 0x7f24700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.383+0000 7f2491e00640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24700744b0 0x7f24700769c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2480005880 tx=0x7f2480009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.388+0000 7f2486000640 1 -- 172.21.15.7:0/941109511 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f248805f890 con 0x7f248c0b5220 2024-04-25T11:21:07.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.630+0000 7f2493200640 1 -- 172.21.15.7:0/941109511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f248c152eb0 con 0x7f248c0b5220 2024-04-25T11:21:07.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.632+0000 7f2486000640 1 -- 172.21.15.7:0/941109511 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5174 (secure 0 0 0) 0x7f248805f190 con 0x7f248c0b5220 2024-04-25T11:21:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:07.629 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2288158899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2288158899},{"type":"v1","addr":"172.21.15.7:6837","nonce":2288158899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3545919245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3545919245},{"type":"v1","addr":"172.21.15.87:6835","nonce":3545919245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8478,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/951407928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":951407928},{"type":"v1","addr":"172.21.15.7:6836","nonce":951407928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:42.908797+0000","modified":"2024-04-25T11:21:03.439614+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":8,"addr":"172.21.15.87:6833/1243096872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1243096872},{"type":"v1","addr":"172.21.15.87:6833","nonce":1243096872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8484,"qdb_cluster":[8484]},"id":9}]} 2024-04-25T11:21:07.629 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 91 2024-04-25T11:21:07.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.634+0000 7f2493200640 1 -- 172.21.15.7:0/941109511 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24700744b0 msgr2=0x7f24700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.634+0000 7f2493200640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24700744b0 0x7f24700769c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2480005880 tx=0x7f2480009040 comp rx=0 tx=0).stop 2024-04-25T11:21:07.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.634+0000 7f2493200640 1 -- 172.21.15.7:0/941109511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f248c0b5220 msgr2=0x7f248c155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:07.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.634+0000 7f2493200640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f248c0b5220 0x7f248c155830 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f248800c6b0 tx=0x7f248800cb70 comp rx=0 tx=0).stop 2024-04-25T11:21:07.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.635+0000 7f2493200640 1 -- 172.21.15.7:0/941109511 shutdown_connections 2024-04-25T11:21:07.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.635+0000 7f2493200640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24700744b0 0x7f24700769c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.635+0000 7f2493200640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f248c0b5220 0x7f248c155830 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.635+0000 7f2493200640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f248c0af420 0x7f248c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.635+0000 7f2493200640 1 --2- 172.21.15.7:0/941109511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f248c0aed40 0x7f248c146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.635+0000 7f2493200640 1 -- 172.21.15.7:0/941109511 >> 172.21.15.7:0/941109511 conn(0x7f248c0b3820 msgr2=0x7f248c0d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.635+0000 7f2493200640 1 -- 172.21.15.7:0/941109511 shutdown_connections 2024-04-25T11:21:07.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:07.635+0000 7f2493200640 1 -- 172.21.15.7:0/941109511 wait complete. 2024-04-25T11:21:07.656 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json session ls 2024-04-25T11:21:07.968 INFO:teuthology.orchestra.run.smithi087.stdout:[{"id":6305,"entity":{"name":{"type":"client","num":6305},"addr":{"type":"any","addr":"192.168.0.1:0","nonce":3077954217}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":13.937690775,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9250119448060588,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6305 192.168.0.1:0/3077954217","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi087","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":6302,"entity":{"name":{"type":"client","num":6302},"addr":{"type":"any","addr":"172.21.15.7:0","nonce":3184181386}},"state":"open","num_leases":1,"num_caps":102,"request_load_avg":203,"uptime":15.698651705,"requests_in_flight":0,"num_completed_requests":200,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":755.1757119234444,"halflife":60},"release_caps":{"value":99.512534068972599,"halflife":60},"recall_caps_throttle":{"value":210.52761875898668,"halflife":1.5},"recall_caps_throttle2o":{"value":66.261607320003094,"halflife":0.5},"session_cache_liveness":{"value":396.46648932704096,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.6302 172.21.15.7:0/3184181386","completed_requests":[{"tid":29,"created_ino":"0x10000000001"},{"tid":30,"created_ino":"0x100000001f6"},{"tid":31,"created_ino":"0x100000001f7"},{"tid":32,"created_ino":"0x100000001f8"},{"tid":33,"created_ino":"0x100000001f9"},{"tid":34,"created_ino":"0x100000001fa"},{"tid":35,"created_ino":"0x100000001fb"},{"tid":36,"created_ino":"0x100000001fc"},{"tid":37,"created_ino":"0x100000001fd"},{"tid":38,"created_ino":"0x100000001fe"},{"tid":39,"created_ino":"0x100000001ff"},{"tid":40,"created_ino":"0x10000000200"},{"tid":41,"created_ino":"0x10000000201"},{"tid":42,"created_ino":"0x10000000202"},{"tid":43,"created_ino":"0x10000000203"},{"tid":44,"created_ino":"0x10000000204"},{"tid":45,"created_ino":"0x10000000205"},{"tid":46,"created_ino":"0x10000000206"},{"tid":47,"created_ino":"0x10000000207"},{"tid":48,"created_ino":"0x10000000208"},{"tid":49,"created_ino":"0x10000000209"},{"tid":50,"created_ino":"0x1000000020a"},{"tid":51,"created_ino":"0x1000000020b"},{"tid":52,"created_ino":"0x1000000020c"},{"tid":53,"created_ino":"0x1000000020d"},{"tid":54,"created_ino":"0x1000000020e"},{"tid":55,"created_ino":"0x1000000020f"},{"tid":56,"created_ino":"0x10000000210"},{"tid":57,"created_ino":"0x10000000211"},{"tid":58,"created_ino":"0x10000000212"},{"tid":59,"created_ino":"0x10000000213"},{"tid":60,"created_ino":"0x10000000214"},{"tid":61,"created_ino":"0x10000000215"},{"tid":62,"created_ino":"0x10000000216"},{"tid":63,"created_ino":"0x10000000217"},{"tid":64,"created_ino":"0x10000000218"},{"tid":65,"created_ino":"0x10000000219"},{"tid":66,"created_ino":"0x1000000021a"},{"tid":67,"created_ino":"0x1000000021b"},{"tid":68,"created_ino":"0x1000000021c"},{"tid":69,"created_ino":"0x1000000021d"},{"tid":70,"created_ino":"0x1000000021e"},{"tid":71,"created_ino":"0x1000000021f"},{"tid":72,"created_ino":"0x10000000220"},{"tid":73,"created_ino":"0x10000000221"},{"tid":74,"created_ino":"0x10000000222"},{"tid":75,"created_ino":"0x10000000223"},{"tid":76,"created_ino":"0x10000000224"},{"tid":77,"created_ino":"0x10000000225"},{"tid":78,"created_ino":"0x10000000226"},{"tid":79,"created_ino":"0x10000000227"},{"tid":80,"created_ino":"0x10000000228"},{"tid":81,"created_ino":"0x10000000229"},{"tid":82,"created_ino":"0x1000000022a"},{"tid":83,"created_ino":"0x1000000022b"},{"tid":84,"created_ino":"0x1000000022c"},{"tid":85,"created_ino":"0x1000000022d"},{"tid":86,"created_ino":"0x1000000022e"},{"tid":87,"created_ino":"0x1000000022f"},{"tid":88,"created_ino":"0x10000000230"},{"tid":89,"created_ino":"0x10000000231"},{"tid":90,"created_ino":"0x10000000232"},{"tid":91,"created_ino":"0x10000000233"},{"tid":92,"created_ino":"0x10000000234"},{"tid":93,"created_ino":"0x10000000235"},{"tid":94,"created_ino":"0x10000000236"},{"tid":95,"created_ino":"0x10000000237"},{"tid":96,"created_ino":"0x10000000238"},{"tid":97,"created_ino":"0x10000000239"},{"tid":98,"created_ino":"0x1000000023a"},{"tid":99,"created_ino":"0x1000000023b"},{"tid":100,"created_ino":"0x1000000023c"},{"tid":101,"created_ino":"0x1000000023d"},{"tid":102,"created_ino":"0x1000000023e"},{"tid":103,"created_ino":"0x1000000023f"},{"tid":104,"created_ino":"0x10000000240"},{"tid":105,"created_ino":"0x10000000241"},{"tid":106,"created_ino":"0x10000000242"},{"tid":107,"created_ino":"0x10000000243"},{"tid":108,"created_ino":"0x10000000244"},{"tid":109,"created_ino":"0x10000000245"},{"tid":110,"created_ino":"0x10000000246"},{"tid":111,"created_ino":"0x10000000247"},{"tid":112,"created_ino":"0x10000000248"},{"tid":113,"created_ino":"0x10000000249"},{"tid":114,"created_ino":"0x1000000024a"},{"tid":115,"created_ino":"0x1000000024b"},{"tid":116,"created_ino":"0x1000000024c"},{"tid":117,"created_ino":"0x1000000024d"},{"tid":118,"created_ino":"0x1000000024e"},{"tid":119,"created_ino":"0x1000000024f"},{"tid":120,"created_ino":"0x10000000250"},{"tid":121,"created_ino":"0x10000000251"},{"tid":122,"created_ino":"0x10000000252"},{"tid":123,"created_ino":"0x10000000253"},{"tid":124,"created_ino":"0x10000000254"},{"tid":125,"created_ino":"0x10000000255"},{"tid":126,"created_ino":"0x10000000256"},{"tid":127,"created_ino":"0x10000000257"},{"tid":128,"created_ino":"0x10000000258"},{"tid":129,"created_ino":"0x10000000259"},{"tid":130,"created_ino":"0x1000000025a"},{"tid":131,"created_ino":"0x1000000025b"},{"tid":132,"created_ino":"0x1000000025c"},{"tid":133,"created_ino":"0x1000000025d"},{"tid":134,"created_ino":"0x1000000025e"},{"tid":135,"created_ino":"0x1000000025f"},{"tid":136,"created_ino":"0x10000000260"},{"tid":137,"created_ino":"0x10000000261"},{"tid":138,"created_ino":"0x10000000262"},{"tid":139,"created_ino":"0x10000000263"},{"tid":140,"created_ino":"0x10000000264"},{"tid":141,"created_ino":"0x10000000265"},{"tid":142,"created_ino":"0x10000000266"},{"tid":143,"created_ino":"0x10000000267"},{"tid":144,"created_ino":"0x10000000268"},{"tid":145,"created_ino":"0x10000000269"},{"tid":146,"created_ino":"0x1000000026a"},{"tid":147,"created_ino":"0x1000000026b"},{"tid":148,"created_ino":"0x1000000026c"},{"tid":149,"created_ino":"0x1000000026d"},{"tid":150,"created_ino":"0x1000000026e"},{"tid":151,"created_ino":"0x1000000026f"},{"tid":152,"created_ino":"0x10000000270"},{"tid":153,"created_ino":"0x10000000271"},{"tid":154,"created_ino":"0x10000000272"},{"tid":155,"created_ino":"0x10000000273"},{"tid":156,"created_ino":"0x10000000274"},{"tid":157,"created_ino":"0x10000000275"},{"tid":158,"created_ino":"0x10000000276"},{"tid":159,"created_ino":"0x10000000277"},{"tid":160,"created_ino":"0x10000000278"},{"tid":161,"created_ino":"0x10000000279"},{"tid":162,"created_ino":"0x1000000027a"},{"tid":163,"created_ino":"0x1000000027b"},{"tid":164,"created_ino":"0x1000000027c"},{"tid":165,"created_ino":"0x1000000027d"},{"tid":166,"created_ino":"0x1000000027e"},{"tid":167,"created_ino":"0x1000000027f"},{"tid":168,"created_ino":"0x10000000280"},{"tid":169,"created_ino":"0x10000000281"},{"tid":170,"created_ino":"0x10000000282"},{"tid":171,"created_ino":"0x10000000283"},{"tid":172,"created_ino":"0x10000000284"},{"tid":173,"created_ino":"0x10000000285"},{"tid":174,"created_ino":"0x10000000286"},{"tid":175,"created_ino":"0x10000000287"},{"tid":176,"created_ino":"0x10000000288"},{"tid":177,"created_ino":"0x10000000289"},{"tid":178,"created_ino":"0x1000000028a"},{"tid":179,"created_ino":"0x1000000028b"},{"tid":180,"created_ino":"0x1000000028c"},{"tid":181,"created_ino":"0x1000000028d"},{"tid":182,"created_ino":"0x1000000028e"},{"tid":183,"created_ino":"0x1000000028f"},{"tid":184,"created_ino":"0x10000000290"},{"tid":185,"created_ino":"0x10000000291"},{"tid":186,"created_ino":"0x10000000292"},{"tid":187,"created_ino":"0x10000000293"},{"tid":188,"created_ino":"0x10000000294"},{"tid":189,"created_ino":"0x10000000295"},{"tid":190,"created_ino":"0x10000000296"},{"tid":191,"created_ino":"0x10000000297"},{"tid":192,"created_ino":"0x10000000298"},{"tid":193,"created_ino":"0x10000000299"},{"tid":194,"created_ino":"0x1000000029a"},{"tid":195,"created_ino":"0x1000000029b"},{"tid":196,"created_ino":"0x1000000029c"},{"tid":197,"created_ino":"0x1000000029d"},{"tid":198,"created_ino":"0x1000000029e"},{"tid":199,"created_ino":"0x1000000029f"},{"tid":200,"created_ino":"0x100000002a0"},{"tid":201,"created_ino":"0x100000002a1"},{"tid":202,"created_ino":"0x100000002a2"},{"tid":203,"created_ino":"0x100000002a3"},{"tid":204,"created_ino":"0x100000002a4"},{"tid":205,"created_ino":"0x100000002a5"},{"tid":206,"created_ino":"0x100000002a6"},{"tid":207,"created_ino":"0x100000002a7"},{"tid":208,"created_ino":"0x100000002a8"},{"tid":209,"created_ino":"0x100000002a9"},{"tid":210,"created_ino":"0x100000002aa"},{"tid":211,"created_ino":"0x100000002ab"},{"tid":212,"created_ino":"0x100000002ac"},{"tid":213,"created_ino":"0x100000002ad"},{"tid":214,"created_ino":"0x100000002ae"},{"tid":215,"created_ino":"0x100000002af"},{"tid":216,"created_ino":"0x100000002b0"},{"tid":217,"created_ino":"0x100000002b1"},{"tid":218,"created_ino":"0x100000002b2"},{"tid":219,"created_ino":"0x100000002b3"},{"tid":220,"created_ino":"0x100000002b4"},{"tid":221,"created_ino":"0x100000002b5"},{"tid":222,"created_ino":"0x100000002b6"},{"tid":223,"created_ino":"0x100000002b7"},{"tid":224,"created_ino":"0x100000002b8"},{"tid":225,"created_ino":"0x100000002b9"},{"tid":226,"created_ino":"0x100000002ba"},{"tid":227,"created_ino":"0x100000002bb"},{"tid":228,"created_ino":"0x100000002bc"}],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x100000002bd","length":802}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi007","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T11:21:07.970 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi087", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "192.168.0.1:0", "nonce": 3077954217, "type": "any" }, "name": { "num": 6305, "type": "client" } }, "id": 6305, "inst": "client.6305 192.168.0.1:0/3077954217", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.9250119448060588 }, "state": "open", "uptime": 13.937690775 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi007", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [ { "created_ino": "0x10000000001", "tid": 29 }, { "created_ino": "0x100000001f6", "tid": 30 }, { "created_ino": "0x100000001f7", "tid": 31 }, { "created_ino": "0x100000001f8", "tid": 32 }, { "created_ino": "0x100000001f9", "tid": 33 }, { "created_ino": "0x100000001fa", "tid": 34 }, { "created_ino": "0x100000001fb", "tid": 35 }, { "created_ino": "0x100000001fc", "tid": 36 }, { "created_ino": "0x100000001fd", "tid": 37 }, { "created_ino": "0x100000001fe", "tid": 38 }, { "created_ino": "0x100000001ff", "tid": 39 }, { "created_ino": "0x10000000200", "tid": 40 }, { "created_ino": "0x10000000201", "tid": 41 }, { "created_ino": "0x10000000202", "tid": 42 }, { "created_ino": "0x10000000203", "tid": 43 }, { "created_ino": "0x10000000204", "tid": 44 }, { "created_ino": "0x10000000205", "tid": 45 }, { "created_ino": "0x10000000206", "tid": 46 }, { "created_ino": "0x10000000207", "tid": 47 }, { "created_ino": "0x10000000208", "tid": 48 }, { "created_ino": "0x10000000209", "tid": 49 }, { "created_ino": "0x1000000020a", "tid": 50 }, { "created_ino": "0x1000000020b", "tid": 51 }, { "created_ino": "0x1000000020c", "tid": 52 }, { "created_ino": "0x1000000020d", "tid": 53 }, { "created_ino": "0x1000000020e", "tid": 54 }, { "created_ino": "0x1000000020f", "tid": 55 }, { "created_ino": "0x10000000210", "tid": 56 }, { "created_ino": "0x10000000211", "tid": 57 }, { "created_ino": "0x10000000212", "tid": 58 }, { "created_ino": "0x10000000213", "tid": 59 }, { "created_ino": "0x10000000214", "tid": 60 }, { "created_ino": "0x10000000215", "tid": 61 }, { "created_ino": "0x10000000216", "tid": 62 }, { "created_ino": "0x10000000217", "tid": 63 }, { "created_ino": "0x10000000218", "tid": 64 }, { "created_ino": "0x10000000219", "tid": 65 }, { "created_ino": "0x1000000021a", "tid": 66 }, { "created_ino": "0x1000000021b", "tid": 67 }, { "created_ino": "0x1000000021c", "tid": 68 }, { "created_ino": "0x1000000021d", "tid": 69 }, { "created_ino": "0x1000000021e", "tid": 70 }, { "created_ino": "0x1000000021f", "tid": 71 }, { "created_ino": "0x10000000220", "tid": 72 }, { "created_ino": "0x10000000221", "tid": 73 }, { "created_ino": "0x10000000222", "tid": 74 }, { "created_ino": "0x10000000223", "tid": 75 }, { "created_ino": "0x10000000224", "tid": 76 }, { "created_ino": "0x10000000225", "tid": 77 }, { "created_ino": "0x10000000226", "tid": 78 }, { "created_ino": "0x10000000227", "tid": 79 }, { "created_ino": "0x10000000228", "tid": 80 }, { "created_ino": "0x10000000229", "tid": 81 }, { "created_ino": "0x1000000022a", "tid": 82 }, { "created_ino": "0x1000000022b", "tid": 83 }, { "created_ino": "0x1000000022c", "tid": 84 }, { "created_ino": "0x1000000022d", "tid": 85 }, { "created_ino": "0x1000000022e", "tid": 86 }, { "created_ino": "0x1000000022f", "tid": 87 }, { "created_ino": "0x10000000230", "tid": 88 }, { "created_ino": "0x10000000231", "tid": 89 }, { "created_ino": "0x10000000232", "tid": 90 }, { "created_ino": "0x10000000233", "tid": 91 }, { "created_ino": "0x10000000234", "tid": 92 }, { "created_ino": "0x10000000235", "tid": 93 }, { "created_ino": "0x10000000236", "tid": 94 }, { "created_ino": "0x10000000237", "tid": 95 }, { "created_ino": "0x10000000238", "tid": 96 }, { "created_ino": "0x10000000239", "tid": 97 }, { "created_ino": "0x1000000023a", "tid": 98 }, { "created_ino": "0x1000000023b", "tid": 99 }, { "created_ino": "0x1000000023c", "tid": 100 }, { "created_ino": "0x1000000023d", "tid": 101 }, { "created_ino": "0x1000000023e", "tid": 102 }, { "created_ino": "0x1000000023f", "tid": 103 }, { "created_ino": "0x10000000240", "tid": 104 }, { "created_ino": "0x10000000241", "tid": 105 }, { "created_ino": "0x10000000242", "tid": 106 }, { "created_ino": "0x10000000243", "tid": 107 }, { "created_ino": "0x10000000244", "tid": 108 }, { "created_ino": "0x10000000245", "tid": 109 }, { "created_ino": "0x10000000246", "tid": 110 }, { "created_ino": "0x10000000247", "tid": 111 }, { "created_ino": "0x10000000248", "tid": 112 }, { "created_ino": "0x10000000249", "tid": 113 }, { "created_ino": "0x1000000024a", "tid": 114 }, { "created_ino": "0x1000000024b", "tid": 115 }, { "created_ino": "0x1000000024c", "tid": 116 }, { "created_ino": "0x1000000024d", "tid": 117 }, { "created_ino": "0x1000000024e", "tid": 118 }, { "created_ino": "0x1000000024f", "tid": 119 }, { "created_ino": "0x10000000250", "tid": 120 }, { "created_ino": "0x10000000251", "tid": 121 }, { "created_ino": "0x10000000252", "tid": 122 }, { "created_ino": "0x10000000253", "tid": 123 }, { "created_ino": "0x10000000254", "tid": 124 }, { "created_ino": "0x10000000255", "tid": 125 }, { "created_ino": "0x10000000256", "tid": 126 }, { "created_ino": "0x10000000257", "tid": 127 }, { "created_ino": "0x10000000258", "tid": 128 }, { "created_ino": "0x10000000259", "tid": 129 }, { "created_ino": "0x1000000025a", "tid": 130 }, { "created_ino": "0x1000000025b", "tid": 131 }, { "created_ino": "0x1000000025c", "tid": 132 }, { "created_ino": "0x1000000025d", "tid": 133 }, { "created_ino": "0x1000000025e", "tid": 134 }, { "created_ino": "0x1000000025f", "tid": 135 }, { "created_ino": "0x10000000260", "tid": 136 }, { "created_ino": "0x10000000261", "tid": 137 }, { "created_ino": "0x10000000262", "tid": 138 }, { "created_ino": "0x10000000263", "tid": 139 }, { "created_ino": "0x10000000264", "tid": 140 }, { "created_ino": "0x10000000265", "tid": 141 }, { "created_ino": "0x10000000266", "tid": 142 }, { "created_ino": "0x10000000267", "tid": 143 }, { "created_ino": "0x10000000268", "tid": 144 }, { "created_ino": "0x10000000269", "tid": 145 }, { "created_ino": "0x1000000026a", "tid": 146 }, { "created_ino": "0x1000000026b", "tid": 147 }, { "created_ino": "0x1000000026c", "tid": 148 }, { "created_ino": "0x1000000026d", "tid": 149 }, { "created_ino": "0x1000000026e", "tid": 150 }, { "created_ino": "0x1000000026f", "tid": 151 }, { "created_ino": "0x10000000270", "tid": 152 }, { "created_ino": "0x10000000271", "tid": 153 }, { "created_ino": "0x10000000272", "tid": 154 }, { "created_ino": "0x10000000273", "tid": 155 }, { "created_ino": "0x10000000274", "tid": 156 }, { "created_ino": "0x10000000275", "tid": 157 }, { "created_ino": "0x10000000276", "tid": 158 }, { "created_ino": "0x10000000277", "tid": 159 }, { "created_ino": "0x10000000278", "tid": 160 }, { "created_ino": "0x10000000279", "tid": 161 }, { "created_ino": "0x1000000027a", "tid": 162 }, { "created_ino": "0x1000000027b", "tid": 163 }, { "created_ino": "0x1000000027c", "tid": 164 }, { "created_ino": "0x1000000027d", "tid": 165 }, { "created_ino": "0x1000000027e", "tid": 166 }, { "created_ino": "0x1000000027f", "tid": 167 }, { "created_ino": "0x10000000280", "tid": 168 }, { "created_ino": "0x10000000281", "tid": 169 }, { "created_ino": "0x10000000282", "tid": 170 }, { "created_ino": "0x10000000283", "tid": 171 }, { "created_ino": "0x10000000284", "tid": 172 }, { "created_ino": "0x10000000285", "tid": 173 }, { "created_ino": "0x10000000286", "tid": 174 }, { "created_ino": "0x10000000287", "tid": 175 }, { "created_ino": "0x10000000288", "tid": 176 }, { "created_ino": "0x10000000289", "tid": 177 }, { "created_ino": "0x1000000028a", "tid": 178 }, { "created_ino": "0x1000000028b", "tid": 179 }, { "created_ino": "0x1000000028c", "tid": 180 }, { "created_ino": "0x1000000028d", "tid": 181 }, { "created_ino": "0x1000000028e", "tid": 182 }, { "created_ino": "0x1000000028f", "tid": 183 }, { "created_ino": "0x10000000290", "tid": 184 }, { "created_ino": "0x10000000291", "tid": 185 }, { "created_ino": "0x10000000292", "tid": 186 }, { "created_ino": "0x10000000293", "tid": 187 }, { "created_ino": "0x10000000294", "tid": 188 }, { "created_ino": "0x10000000295", "tid": 189 }, { "created_ino": "0x10000000296", "tid": 190 }, { "created_ino": "0x10000000297", "tid": 191 }, { "created_ino": "0x10000000298", "tid": 192 }, { "created_ino": "0x10000000299", "tid": 193 }, { "created_ino": "0x1000000029a", "tid": 194 }, { "created_ino": "0x1000000029b", "tid": 195 }, { "created_ino": "0x1000000029c", "tid": 196 }, { "created_ino": "0x1000000029d", "tid": 197 }, { "created_ino": "0x1000000029e", "tid": 198 }, { "created_ino": "0x1000000029f", "tid": 199 }, { "created_ino": "0x100000002a0", "tid": 200 }, { "created_ino": "0x100000002a1", "tid": 201 }, { "created_ino": "0x100000002a2", "tid": 202 }, { "created_ino": "0x100000002a3", "tid": 203 }, { "created_ino": "0x100000002a4", "tid": 204 }, { "created_ino": "0x100000002a5", "tid": 205 }, { "created_ino": "0x100000002a6", "tid": 206 }, { "created_ino": "0x100000002a7", "tid": 207 }, { "created_ino": "0x100000002a8", "tid": 208 }, { "created_ino": "0x100000002a9", "tid": 209 }, { "created_ino": "0x100000002aa", "tid": 210 }, { "created_ino": "0x100000002ab", "tid": 211 }, { "created_ino": "0x100000002ac", "tid": 212 }, { "created_ino": "0x100000002ad", "tid": 213 }, { "created_ino": "0x100000002ae", "tid": 214 }, { "created_ino": "0x100000002af", "tid": 215 }, { "created_ino": "0x100000002b0", "tid": 216 }, { "created_ino": "0x100000002b1", "tid": 217 }, { "created_ino": "0x100000002b2", "tid": 218 }, { "created_ino": "0x100000002b3", "tid": 219 }, { "created_ino": "0x100000002b4", "tid": 220 }, { "created_ino": "0x100000002b5", "tid": 221 }, { "created_ino": "0x100000002b6", "tid": 222 }, { "created_ino": "0x100000002b7", "tid": 223 }, { "created_ino": "0x100000002b8", "tid": 224 }, { "created_ino": "0x100000002b9", "tid": 225 }, { "created_ino": "0x100000002ba", "tid": 226 }, { "created_ino": "0x100000002bb", "tid": 227 }, { "created_ino": "0x100000002bc", "tid": 228 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.7:0", "nonce": 3184181386, "type": "any" }, "name": { "num": 6302, "type": "client" } }, "id": 6302, "inst": "client.6302 172.21.15.7:0/3184181386", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 102, "num_completed_flushes": 0, "num_completed_requests": 200, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 802, "start": "0x100000002bd" } ], "recall_caps": { "halflife": 60, "value": 755.1757119234444 }, "recall_caps_throttle": { "halflife": 1.5, "value": 210.52761875898668 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 66.2616073200031 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 99.5125340689726 }, "request_load_avg": 203, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 396.46648932704096 }, "state": "open", "uptime": 15.698651705 } ] 2024-04-25T11:21:07.971 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:21:07.971 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:21:07.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:21:08.092 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:21:08.200 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:21:08.200 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:21:08.200 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 200\nabs_path = "/home/ubuntu/cephtest/mnt.0/subdir"\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi007') 2024-04-25T11:21:08.200 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:21:08.200 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:08.260 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:21:08.260 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:08.316 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:21:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.605+0000 7f2a1a000640 1 Processor -- start 2024-04-25T11:21:08.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.605+0000 7f2a1a000640 1 -- start start 2024-04-25T11:21:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.607+0000 7f2a1a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2a140aed40 0x7f2a1415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.607+0000 7f2a1a000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a140b5220 0x7f2a140ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.608+0000 7f2a1a000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a140b64b0 con 0x7f2a140af420 2024-04-25T11:21:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.608+0000 7f2a1a000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2a1415e1f0 con 0x7f2a140aed40 2024-04-25T11:21:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.608+0000 7f2a1a000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1406c610 con 0x7f2a140b5220 2024-04-25T11:21:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.608+0000 7f2a18c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2a140aed40 0x7f2a1415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.608+0000 7f2a18c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2a140aed40 0x7f2a1415dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57442/0 (socket says 172.21.15.7:57442) 2024-04-25T11:21:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.608+0000 7f2a18c00640 1 -- 172.21.15.7:0/1010512032 learned_addr learned my addr 172.21.15.7:0/1010512032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.608+0000 7f2a18c00640 1 -- 172.21.15.7:0/1010512032 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a140b5220 msgr2=0x7f2a140ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.608+0000 7f2a18c00640 1 --2- 172.21.15.7:0/1010512032 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a140b5220 0x7f2a140ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.608+0000 7f2a18c00640 1 -- 172.21.15.7:0/1010512032 >> v1:172.21.15.87:6789/0 conn(0x7f2a140af420 legacy=0x7f2a140af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.608+0000 7f2a18c00640 1 -- 172.21.15.7:0/1010512032 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1406c8f0 con 0x7f2a140aed40 2024-04-25T11:21:08.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.610+0000 7f2a18c00640 1 --2- 172.21.15.7:0/1010512032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a140aed40 0x7f2a1415dcb0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2a1415e370 tx=0x7f2a0802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5169995a1200bc1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.610+0000 7f2a13400640 1 -- 172.21.15.7:0/1010512032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08037070 con 0x7f2a140aed40 2024-04-25T11:21:08.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.610+0000 7f2a13400640 1 -- 172.21.15.7:0/1010512032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a08032440 con 0x7f2a140aed40 2024-04-25T11:21:08.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.610+0000 7f2a13400640 1 -- 172.21.15.7:0/1010512032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08032810 con 0x7f2a140aed40 2024-04-25T11:21:08.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.611+0000 7f2a1a000640 1 -- 172.21.15.7:0/1010512032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a140aed40 msgr2=0x7f2a1415dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.611+0000 7f2a1a000640 1 --2- 172.21.15.7:0/1010512032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a140aed40 0x7f2a1415dcb0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2a1415e370 tx=0x7f2a0802a200 comp rx=0 tx=0).stop 2024-04-25T11:21:08.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.611+0000 7f2a1a000640 1 -- 172.21.15.7:0/1010512032 shutdown_connections 2024-04-25T11:21:08.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.611+0000 7f2a1a000640 1 --2- 172.21.15.7:0/1010512032 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a140b5220 0x7f2a140ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.611+0000 7f2a1a000640 1 --2- 172.21.15.7:0/1010512032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a140aed40 0x7f2a1415dcb0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.611+0000 7f2a1a000640 1 -- 172.21.15.7:0/1010512032 >> 172.21.15.7:0/1010512032 conn(0x7f2a140b3820 msgr2=0x7f2a140b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:08.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.611+0000 7f2a1a000640 1 -- 172.21.15.7:0/1010512032 shutdown_connections 2024-04-25T11:21:08.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.611+0000 7f2a1a000640 1 -- 172.21.15.7:0/1010512032 wait complete. 2024-04-25T11:21:08.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.613+0000 7f2a1a000640 1 Processor -- start 2024-04-25T11:21:08.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.613+0000 7f2a1a000640 1 -- start start 2024-04-25T11:21:08.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.615+0000 7f2a1a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a140aed40 0x7f2a141566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.615+0000 7f2a1a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a140af420 0x7f2a14160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.615+0000 7f2a1a000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a140b5220 0x7f2a140d0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.615+0000 7f2a1a000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2a140b6110 con 0x7f2a140af420 2024-04-25T11:21:08.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.615+0000 7f2a1a000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1406ceb0 con 0x7f2a140aed40 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.615+0000 7f2a1a000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2a140b6290 con 0x7f2a140b5220 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.615+0000 7f2a13e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a140af420 0x7f2a14160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.615+0000 7f2a13e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a140af420 0x7f2a14160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57454/0 (socket says 172.21.15.7:57454) 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.615+0000 7f2a13e00640 1 -- 172.21.15.7:0/4172224047 learned_addr learned my addr 172.21.15.7:0/4172224047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.615+0000 7f2a18c00640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a140aed40 0x7f2a141566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.616+0000 7f2a19600640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a140b5220 0x7f2a140d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.616+0000 7f2a13e00640 1 -- 172.21.15.7:0/4172224047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a140b5220 msgr2=0x7f2a140d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.616+0000 7f2a13e00640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a140b5220 0x7f2a140d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.616+0000 7f2a13e00640 1 -- 172.21.15.7:0/4172224047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a140aed40 msgr2=0x7f2a141566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.616+0000 7f2a13e00640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a140aed40 0x7f2a141566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.616+0000 7f2a13e00640 1 -- 172.21.15.7:0/4172224047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a141f9b70 con 0x7f2a140af420 2024-04-25T11:21:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.616+0000 7f2a13e00640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a140af420 0x7f2a14160670 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b410 tx=0x7f2a0000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.617+0000 7f2a10c00640 1 -- 172.21.15.7:0/4172224047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00015020 con 0x7f2a140af420 2024-04-25T11:21:08.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.617+0000 7f2a10c00640 1 -- 172.21.15.7:0/4172224047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a00011850 con 0x7f2a140af420 2024-04-25T11:21:08.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.617+0000 7f2a10c00640 1 -- 172.21.15.7:0/4172224047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00011c40 con 0x7f2a140af420 2024-04-25T11:21:08.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.617+0000 7f2a1a000640 1 -- 172.21.15.7:0/4172224047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a141f9e50 con 0x7f2a140af420 2024-04-25T11:21:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.617+0000 7f2a1a000640 1 -- 172.21.15.7:0/4172224047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a141fa410 con 0x7f2a140af420 2024-04-25T11:21:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.618+0000 7f2a1a000640 1 -- 172.21.15.7:0/4172224047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a14050740 con 0x7f2a140af420 2024-04-25T11:21:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.619+0000 7f2a10c00640 1 -- 172.21.15.7:0/4172224047 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2a00018020 con 0x7f2a140af420 2024-04-25T11:21:08.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.620+0000 7f2a10c00640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29f40744e0 0x7f29f40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:08.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.620+0000 7f2a10c00640 1 -- 172.21.15.7:0/4172224047 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f2a00095360 con 0x7f2a140af420 2024-04-25T11:21:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.621+0000 7f2a18c00640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29f40744e0 0x7f29f40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.621+0000 7f2a18c00640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29f40744e0 0x7f29f40769f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2a08034000 tx=0x7f2a0802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.625+0000 7f2a10c00640 1 -- 172.21.15.7:0/4172224047 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2a0001f020 con 0x7f2a140af420 2024-04-25T11:21:08.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.832+0000 7f2a1a000640 1 -- 172.21.15.7:0/4172224047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a140b3fb0 con 0x7f2a140af420 2024-04-25T11:21:08.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.836+0000 7f2a10c00640 1 -- 172.21.15.7:0/4172224047 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16804 (secure 0 0 0) 0x7f2a0005f960 con 0x7f2a140af420 2024-04-25T11:21:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:08.833 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":130,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:20:44.396251+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:41.047420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:41.908907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:08.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 -- 172.21.15.7:0/4172224047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29f40744e0 msgr2=0x7f29f40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29f40744e0 0x7f29f40769f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f2a08034000 tx=0x7f2a0802e000 comp rx=0 tx=0).stop 2024-04-25T11:21:08.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 -- 172.21.15.7:0/4172224047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a140af420 msgr2=0x7f2a14160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:08.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a140af420 0x7f2a14160670 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b410 tx=0x7f2a0000b8d0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 -- 172.21.15.7:0/4172224047 shutdown_connections 2024-04-25T11:21:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f29f40744e0 0x7f29f40769f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2a140b5220 0x7f2a140d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2a140af420 0x7f2a14160670 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 --2- 172.21.15.7:0/4172224047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2a140aed40 0x7f2a141566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 -- 172.21.15.7:0/4172224047 >> 172.21.15.7:0/4172224047 conn(0x7f2a140b3820 msgr2=0x7f2a140b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 -- 172.21.15.7:0/4172224047 shutdown_connections 2024-04-25T11:21:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:08.839+0000 7f2a1a000640 1 -- 172.21.15.7:0/4172224047 wait complete. 2024-04-25T11:21:08.866 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:20:41.047420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:20:41.908907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:21:08.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:21:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.152+0000 7f161ce00640 1 Processor -- start 2024-04-25T11:21:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.152+0000 7f161ce00640 1 -- start start 2024-04-25T11:21:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.154+0000 7f161ce00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16181490f0 0x7f1618149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.154+0000 7f161ce00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f161806f280 con 0x7f16180b01d0 2024-04-25T11:21:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.154+0000 7f161ce00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f161806f560 con 0x7f1618146180 2024-04-25T11:21:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.154+0000 7f161ce00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1618146590 con 0x7f16181490f0 2024-04-25T11:21:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.155+0000 7f1617e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f16180b01d0 0x7f16180b05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47322/0 (socket says 172.21.15.7:47322) 2024-04-25T11:21:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.155+0000 7f1617e00640 1 -- 172.21.15.7:0/3916427367 learned_addr learned my addr 172.21.15.7:0/3916427367 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.155+0000 7f1616a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16181490f0 0x7f1618149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.156+0000 7f1616a00640 1 -- 172.21.15.7:0/3916427367 >> v1:172.21.15.87:6789/0 conn(0x7f1618146180 legacy=0x7f1618070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.156+0000 7f1616a00640 1 -- 172.21.15.7:0/3916427367 >> v1:172.21.15.7:6789/0 conn(0x7f16180b01d0 legacy=0x7f16180b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.156+0000 7f1616a00640 1 -- 172.21.15.7:0/3916427367 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161806f840 con 0x7f16181490f0 2024-04-25T11:21:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.156+0000 7f1616000640 1 -- 172.21.15.7:0/3916427367 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 890010270 0 0) 0x7f161806f280 con 0x7f16180b01d0 2024-04-25T11:21:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.157+0000 7f1616a00640 1 --2- 172.21.15.7:0/3916427367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16181490f0 0x7f1618149500 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f16040097a0 tx=0x7f160402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2198a57e52cff62e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.157+0000 7f1616000640 1 -- 172.21.15.7:0/3916427367 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604037070 con 0x7f16181490f0 2024-04-25T11:21:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.157+0000 7f1616000640 1 -- 172.21.15.7:0/3916427367 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1604032440 con 0x7f16181490f0 2024-04-25T11:21:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.157+0000 7f1616000640 1 -- 172.21.15.7:0/3916427367 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604032810 con 0x7f16181490f0 2024-04-25T11:21:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.158+0000 7f161ce00640 1 -- 172.21.15.7:0/3916427367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16181490f0 msgr2=0x7f1618149500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.158+0000 7f161ce00640 1 --2- 172.21.15.7:0/3916427367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16181490f0 0x7f1618149500 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f16040097a0 tx=0x7f160402a230 comp rx=0 tx=0).stop 2024-04-25T11:21:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.158+0000 7f161ce00640 1 -- 172.21.15.7:0/3916427367 shutdown_connections 2024-04-25T11:21:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.158+0000 7f161ce00640 1 --2- 172.21.15.7:0/3916427367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16181490f0 0x7f1618149500 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.158+0000 7f161ce00640 1 -- 172.21.15.7:0/3916427367 >> 172.21.15.7:0/3916427367 conn(0x7f16180740f0 msgr2=0x7f161806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.158+0000 7f161ce00640 1 -- 172.21.15.7:0/3916427367 shutdown_connections 2024-04-25T11:21:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.158+0000 7f161ce00640 1 -- 172.21.15.7:0/3916427367 wait complete. 2024-04-25T11:21:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.161+0000 7f161ce00640 1 Processor -- start 2024-04-25T11:21:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.161+0000 7f161ce00640 1 -- start start 2024-04-25T11:21:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.162+0000 7f161ce00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16180b01d0 0x7f1618160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.162+0000 7f161ce00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1618146180 0x7f16181ea220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.162+0000 7f161ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16181490f0 0x7f16180cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.162+0000 7f161ce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f161806eea0 con 0x7f16181490f0 2024-04-25T11:21:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.162+0000 7f161ce00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f161806fe00 con 0x7f1618146180 2024-04-25T11:21:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.162+0000 7f161ce00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f161806f020 con 0x7f16180b01d0 2024-04-25T11:21:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.163+0000 7f1617e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16181490f0 0x7f16180cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.163+0000 7f1617e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16181490f0 0x7f16180cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57462/0 (socket says 172.21.15.7:57462) 2024-04-25T11:21:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.163+0000 7f1617e00640 1 -- 172.21.15.7:0/1015887236 learned_addr learned my addr 172.21.15.7:0/1015887236 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.163+0000 7f1616a00640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1618146180 0x7f16181ea220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.163+0000 7f1617400640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16180b01d0 0x7f1618160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.163+0000 7f1617e00640 1 -- 172.21.15.7:0/1015887236 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16180b01d0 msgr2=0x7f1618160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.163+0000 7f1617e00640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16180b01d0 0x7f1618160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.163+0000 7f1617e00640 1 -- 172.21.15.7:0/1015887236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1618146180 msgr2=0x7f16181ea220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.163+0000 7f1617e00640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1618146180 0x7f16181ea220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.163+0000 7f1617e00640 1 -- 172.21.15.7:0/1015887236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16180cdbd0 con 0x7f16181490f0 2024-04-25T11:21:09.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.164+0000 7f1617e00640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16181490f0 0x7f16180cec70 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f161000c780 tx=0x7f161000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.164+0000 7f160f400640 1 -- 172.21.15.7:0/1015887236 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610018070 con 0x7f16181490f0 2024-04-25T11:21:09.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.164+0000 7f160f400640 1 -- 172.21.15.7:0/1015887236 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161000ce30 con 0x7f16181490f0 2024-04-25T11:21:09.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.164+0000 7f161ce00640 1 -- 172.21.15.7:0/1015887236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16180cdeb0 con 0x7f16181490f0 2024-04-25T11:21:09.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.164+0000 7f160f400640 1 -- 172.21.15.7:0/1015887236 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161000a840 con 0x7f16181490f0 2024-04-25T11:21:09.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.164+0000 7f161ce00640 1 -- 172.21.15.7:0/1015887236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16180ce470 con 0x7f16181490f0 2024-04-25T11:21:09.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.165+0000 7f161ce00640 1 -- 172.21.15.7:0/1015887236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16180cd930 con 0x7f16181490f0 2024-04-25T11:21:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.167+0000 7f160f400640 1 -- 172.21.15.7:0/1015887236 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f161001b070 con 0x7f16181490f0 2024-04-25T11:21:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.168+0000 7f160f400640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15f80744e0 0x7f15f80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.168+0000 7f160f400640 1 -- 172.21.15.7:0/1015887236 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f16100948f0 con 0x7f16181490f0 2024-04-25T11:21:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.168+0000 7f1617400640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15f80744e0 0x7f15f80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.169+0000 7f1617400640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15f80744e0 0x7f15f80769f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1608000f60 tx=0x7f1608009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.173+0000 7f160f400640 1 -- 172.21.15.7:0/1015887236 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16180cd930 con 0x7f16181490f0 2024-04-25T11:21:09.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.381+0000 7f161ce00640 1 -- 172.21.15.7:0/1015887236 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1618050740 con 0x7f16181490f0 2024-04-25T11:21:09.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.382+0000 7f160f400640 1 -- 172.21.15.7:0/1015887236 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) ==== 93+0+10 (secure 0 0 0) 0x7f161005eee0 con 0x7f16181490f0 2024-04-25T11:21:09.380 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:21:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.385+0000 7f161ce00640 1 -- 172.21.15.7:0/1015887236 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15f80744e0 msgr2=0x7f15f80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.385+0000 7f161ce00640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15f80744e0 0x7f15f80769f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1608000f60 tx=0x7f1608009040 comp rx=0 tx=0).stop 2024-04-25T11:21:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.385+0000 7f161ce00640 1 -- 172.21.15.7:0/1015887236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16181490f0 msgr2=0x7f16180cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.385+0000 7f161ce00640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16181490f0 0x7f16180cec70 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f161000c780 tx=0x7f161000cc40 comp rx=0 tx=0).stop 2024-04-25T11:21:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.385+0000 7f161ce00640 1 -- 172.21.15.7:0/1015887236 shutdown_connections 2024-04-25T11:21:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.385+0000 7f161ce00640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15f80744e0 0x7f15f80769f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.385+0000 7f161ce00640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16181490f0 0x7f16180cec70 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.385+0000 7f161ce00640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1618146180 0x7f16181ea220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.385+0000 7f161ce00640 1 --2- 172.21.15.7:0/1015887236 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16180b01d0 0x7f1618160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.385+0000 7f161ce00640 1 -- 172.21.15.7:0/1015887236 >> 172.21.15.7:0/1015887236 conn(0x7f16180740f0 msgr2=0x7f1618145400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.385+0000 7f161ce00640 1 -- 172.21.15.7:0/1015887236 shutdown_connections 2024-04-25T11:21:09.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.386+0000 7f161ce00640 1 -- 172.21.15.7:0/1015887236 wait complete. 2024-04-25T11:21:09.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:21:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.699+0000 7efeb5c00640 1 Processor -- start 2024-04-25T11:21:09.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.699+0000 7efeb5c00640 1 -- start start 2024-04-25T11:21:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.702+0000 7efeb5c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efeb0145e40 0x7efeb0146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.702+0000 7efeb5c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb00b66b0 con 0x7efeb0144a80 2024-04-25T11:21:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.702+0000 7efeb5c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb00b5b30 con 0x7efeb00b4870 2024-04-25T11:21:09.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.702+0000 7efeb5c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efeb0145b70 con 0x7efeb0145e40 2024-04-25T11:21:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.702+0000 7efeb5200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efeb0144a80 0x7efeb01606f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47336/0 (socket says 172.21.15.7:47336) 2024-04-25T11:21:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.702+0000 7efeafe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efeb0145e40 0x7efeb0146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.702+0000 7efeb5200640 1 -- 172.21.15.7:0/3003251181 learned_addr learned my addr 172.21.15.7:0/3003251181 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.703+0000 7efeaea00640 1 -- 172.21.15.7:0/3003251181 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1502993646 0 0) 0x7efeb00b66b0 con 0x7efeb0144a80 2024-04-25T11:21:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.703+0000 7efeaea00640 1 -- 172.21.15.7:0/3003251181 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe94005730 con 0x7efeb0144a80 2024-04-25T11:21:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.703+0000 7efeafe00640 1 -- 172.21.15.7:0/3003251181 >> v1:172.21.15.87:6789/0 conn(0x7efeb00b4870 legacy=0x7efeb00b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.703+0000 7efeafe00640 1 -- 172.21.15.7:0/3003251181 >> v1:172.21.15.7:6789/0 conn(0x7efeb0144a80 legacy=0x7efeb01606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.703+0000 7efeafe00640 1 -- 172.21.15.7:0/3003251181 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb00b5e10 con 0x7efeb0145e40 2024-04-25T11:21:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.704+0000 7efeafe00640 1 --2- 172.21.15.7:0/3003251181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0145e40 0x7efeb0146250 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7efea00097a0 tx=0x7efea002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c236ae5e1f0bd8e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.704+0000 7efeaea00640 1 -- 172.21.15.7:0/3003251181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0037070 con 0x7efeb0145e40 2024-04-25T11:21:09.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.704+0000 7efeaea00640 1 -- 172.21.15.7:0/3003251181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea0032440 con 0x7efeb0145e40 2024-04-25T11:21:09.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.705+0000 7efeaea00640 1 -- 172.21.15.7:0/3003251181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00327f0 con 0x7efeb0145e40 2024-04-25T11:21:09.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.705+0000 7efeb5c00640 1 -- 172.21.15.7:0/3003251181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0145e40 msgr2=0x7efeb0146250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.705+0000 7efeb5c00640 1 --2- 172.21.15.7:0/3003251181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0145e40 0x7efeb0146250 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7efea00097a0 tx=0x7efea002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.705+0000 7efeb5c00640 1 -- 172.21.15.7:0/3003251181 shutdown_connections 2024-04-25T11:21:09.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.705+0000 7efeb5c00640 1 --2- 172.21.15.7:0/3003251181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb0145e40 0x7efeb0146250 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.705+0000 7efeb5c00640 1 -- 172.21.15.7:0/3003251181 >> 172.21.15.7:0/3003251181 conn(0x7efeb00740f0 msgr2=0x7efeb006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.705+0000 7efeb5c00640 1 -- 172.21.15.7:0/3003251181 shutdown_connections 2024-04-25T11:21:09.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.705+0000 7efeb5c00640 1 -- 172.21.15.7:0/3003251181 wait complete. 2024-04-25T11:21:09.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.708+0000 7efeb5c00640 1 Processor -- start 2024-04-25T11:21:09.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.708+0000 7efeb5c00640 1 -- start start 2024-04-25T11:21:09.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeb5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb00b4870 0x7efeb015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeb5c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb0144a80 0x7efeb0136320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeb5c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efeb0145e40 0x7efeb00b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeb5c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efeb0161e10 con 0x7efeb00b4870 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeb5c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efeb00b63d0 con 0x7efeb0144a80 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeb5c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efeb0161f90 con 0x7efeb0145e40 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeafe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb00b4870 0x7efeb015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeafe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb00b4870 0x7efeb015a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57484/0 (socket says 172.21.15.7:57484) 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeafe00640 1 -- 172.21.15.7:0/4167107216 learned_addr learned my addr 172.21.15.7:0/4167107216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeafe00640 1 -- 172.21.15.7:0/4167107216 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efeb0145e40 msgr2=0x7efeb00b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeafe00640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efeb0145e40 0x7efeb00b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeafe00640 1 -- 172.21.15.7:0/4167107216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb0144a80 msgr2=0x7efeb0136320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeafe00640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb0144a80 0x7efeb0136320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.710+0000 7efeafe00640 1 -- 172.21.15.7:0/4167107216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb013bd00 con 0x7efeb00b4870 2024-04-25T11:21:09.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.711+0000 7efeafe00640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb00b4870 0x7efeb015a630 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7efea0035040 tx=0x7efea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.711+0000 7efea7e00640 1 -- 172.21.15.7:0/4167107216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0037050 con 0x7efeb00b4870 2024-04-25T11:21:09.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.711+0000 7efea7e00640 1 -- 172.21.15.7:0/4167107216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea0038040 con 0x7efeb00b4870 2024-04-25T11:21:09.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.711+0000 7efea7e00640 1 -- 172.21.15.7:0/4167107216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0004470 con 0x7efeb00b4870 2024-04-25T11:21:09.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.711+0000 7efeb5c00640 1 -- 172.21.15.7:0/4167107216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb013bfe0 con 0x7efeb00b4870 2024-04-25T11:21:09.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.711+0000 7efeb5c00640 1 -- 172.21.15.7:0/4167107216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb013c5a0 con 0x7efeb00b4870 2024-04-25T11:21:09.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.713+0000 7efeb5c00640 1 -- 172.21.15.7:0/4167107216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb0050740 con 0x7efeb00b4870 2024-04-25T11:21:09.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.714+0000 7efea7e00640 1 -- 172.21.15.7:0/4167107216 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efea003c6d0 con 0x7efeb00b4870 2024-04-25T11:21:09.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.715+0000 7efea7e00640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe900744e0 0x7efe900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:09.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.715+0000 7efea7e00640 1 -- 172.21.15.7:0/4167107216 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7efea00b5c70 con 0x7efeb00b4870 2024-04-25T11:21:09.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.715+0000 7efeaf400640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe900744e0 0x7efe900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:09.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.716+0000 7efeaf400640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe900744e0 0x7efe900769f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7efe9c000f60 tx=0x7efe9c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:09.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.720+0000 7efea7e00640 1 -- 172.21.15.7:0/4167107216 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efea0080260 con 0x7efeb00b4870 2024-04-25T11:21:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.926+0000 7efeb5c00640 1 -- 172.21.15.7:0/4167107216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efeb006fab0 con 0x7efeb00b4870 2024-04-25T11:21:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.928+0000 7efea7e00640 1 -- 172.21.15.7:0/4167107216 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) ==== 104+0+11 (secure 0 0 0) 0x7efea007fb60 con 0x7efeb00b4870 2024-04-25T11:21:09.925 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:21:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 -- 172.21.15.7:0/4167107216 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe900744e0 msgr2=0x7efe900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe900744e0 0x7efe900769f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7efe9c000f60 tx=0x7efe9c0079b0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 -- 172.21.15.7:0/4167107216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb00b4870 msgr2=0x7efeb015a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb00b4870 0x7efeb015a630 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7efea0035040 tx=0x7efea00046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 -- 172.21.15.7:0/4167107216 shutdown_connections 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efe900744e0 0x7efe900769f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efeb0145e40 0x7efeb00b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb0144a80 0x7efeb0136320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 --2- 172.21.15.7:0/4167107216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efeb00b4870 0x7efeb015a630 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 -- 172.21.15.7:0/4167107216 >> 172.21.15.7:0/4167107216 conn(0x7efeb00740f0 msgr2=0x7efeb015c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 -- 172.21.15.7:0/4167107216 shutdown_connections 2024-04-25T11:21:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:09.931+0000 7efeb5c00640 1 -- 172.21.15.7:0/4167107216 wait complete. 2024-04-25T11:21:09.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:21:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.232+0000 7fd453200640 1 Processor -- start 2024-04-25T11:21:10.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.232+0000 7fd453200640 1 -- start start 2024-04-25T11:21:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.234+0000 7fd453200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4540b8ff0 0x7fd4540d3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.234+0000 7fd453200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4540b4b80 0x7fd4541634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.234+0000 7fd453200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd454163b40 con 0x7fd45406c8a0 2024-04-25T11:21:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.234+0000 7fd453200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd4540d54e0 con 0x7fd4540b4b80 2024-04-25T11:21:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.234+0000 7fd453200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd454164750 con 0x7fd4540b8ff0 2024-04-25T11:21:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.235+0000 7fd451400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4540b4b80 0x7fd4541634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.235+0000 7fd451400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4540b4b80 0x7fd4541634f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37480/0 (socket says 172.21.15.7:37480) 2024-04-25T11:21:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.235+0000 7fd451400640 1 -- 172.21.15.7:0/2074864474 learned_addr learned my addr 172.21.15.7:0/2074864474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.235+0000 7fd451e00640 1 --2- 172.21.15.7:0/2074864474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4540b8ff0 0x7fd4540d3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.235+0000 7fd451400640 1 -- 172.21.15.7:0/2074864474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4540b8ff0 msgr2=0x7fd4540d3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.235+0000 7fd451400640 1 --2- 172.21.15.7:0/2074864474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4540b8ff0 0x7fd4540d3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.235+0000 7fd451400640 1 -- 172.21.15.7:0/2074864474 >> v1:172.21.15.7:6789/0 conn(0x7fd45406c8a0 legacy=0x7fd45406ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.235+0000 7fd451400640 1 -- 172.21.15.7:0/2074864474 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd454163e20 con 0x7fd4540b4b80 2024-04-25T11:21:10.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.235+0000 7fd450a00640 1 -- 172.21.15.7:0/2074864474 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3819303574 0 0) 0x7fd454163b40 con 0x7fd45406c8a0 2024-04-25T11:21:10.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.237+0000 7fd451400640 1 --2- 172.21.15.7:0/2074864474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4540b4b80 0x7fd4541634f0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd440009870 tx=0x7fd44002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0c3977fcf1b52b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.237+0000 7fd450a00640 1 -- 172.21.15.7:0/2074864474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440037070 con 0x7fd4540b4b80 2024-04-25T11:21:10.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.237+0000 7fd450a00640 1 -- 172.21.15.7:0/2074864474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd440032440 con 0x7fd4540b4b80 2024-04-25T11:21:10.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.237+0000 7fd450a00640 1 -- 172.21.15.7:0/2074864474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440032810 con 0x7fd4540b4b80 2024-04-25T11:21:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.238+0000 7fd453200640 1 -- 172.21.15.7:0/2074864474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4540b4b80 msgr2=0x7fd4541634f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.238+0000 7fd453200640 1 --2- 172.21.15.7:0/2074864474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4540b4b80 0x7fd4541634f0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd440009870 tx=0x7fd44002a270 comp rx=0 tx=0).stop 2024-04-25T11:21:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.238+0000 7fd453200640 1 -- 172.21.15.7:0/2074864474 shutdown_connections 2024-04-25T11:21:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.238+0000 7fd453200640 1 --2- 172.21.15.7:0/2074864474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4540b4b80 0x7fd4541634f0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.238+0000 7fd453200640 1 --2- 172.21.15.7:0/2074864474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4540b8ff0 0x7fd4540d3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.238+0000 7fd453200640 1 -- 172.21.15.7:0/2074864474 >> 172.21.15.7:0/2074864474 conn(0x7fd4540d5b80 msgr2=0x7fd4540d5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.238+0000 7fd453200640 1 -- 172.21.15.7:0/2074864474 shutdown_connections 2024-04-25T11:21:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.238+0000 7fd453200640 1 -- 172.21.15.7:0/2074864474 wait complete. 2024-04-25T11:21:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.241+0000 7fd453200640 1 Processor -- start 2024-04-25T11:21:10.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.241+0000 7fd453200640 1 -- start start 2024-04-25T11:21:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.242+0000 7fd453200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd45406c8a0 0x7fd45415bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.242+0000 7fd453200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4540b4b80 0x7fd4540c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.242+0000 7fd453200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4540b8ff0 0x7fd4540ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.242+0000 7fd453200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4540b4850 con 0x7fd4540b8ff0 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.242+0000 7fd453200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4541643e0 con 0x7fd4540b4b80 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.242+0000 7fd453200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4540b6b40 con 0x7fd45406c8a0 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd452800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4540b8ff0 0x7fd4540ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd452800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4540b8ff0 0x7fd4540ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37482/0 (socket says 172.21.15.7:37482) 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd452800640 1 -- 172.21.15.7:0/3885492923 learned_addr learned my addr 172.21.15.7:0/3885492923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd451e00640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd45406c8a0 0x7fd45415bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd451400640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4540b4b80 0x7fd4540c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd452800640 1 -- 172.21.15.7:0/3885492923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd45406c8a0 msgr2=0x7fd45415bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd452800640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd45406c8a0 0x7fd45415bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd452800640 1 -- 172.21.15.7:0/3885492923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4540b4b80 msgr2=0x7fd4540c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd452800640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4540b4b80 0x7fd4540c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd452800640 1 -- 172.21.15.7:0/3885492923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4540cc4b0 con 0x7fd4540b8ff0 2024-04-25T11:21:10.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd451400640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4540b4b80 0x7fd4540c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.243+0000 7fd452800640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4540b8ff0 0x7fd4540ce060 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00c780 tx=0x7fd44c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.244+0000 7fd44a000640 1 -- 172.21.15.7:0/3885492923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c017070 con 0x7fd4540b8ff0 2024-04-25T11:21:10.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.244+0000 7fd44a000640 1 -- 172.21.15.7:0/3885492923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd44c003da0 con 0x7fd4540b8ff0 2024-04-25T11:21:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.244+0000 7fd44a000640 1 -- 172.21.15.7:0/3885492923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd44c005290 con 0x7fd4540b8ff0 2024-04-25T11:21:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.244+0000 7fd453200640 1 -- 172.21.15.7:0/3885492923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4540c8eb0 con 0x7fd4540b8ff0 2024-04-25T11:21:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.244+0000 7fd453200640 1 -- 172.21.15.7:0/3885492923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4540c9470 con 0x7fd4540b8ff0 2024-04-25T11:21:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.245+0000 7fd453200640 1 -- 172.21.15.7:0/3885492923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4540cc220 con 0x7fd4540b8ff0 2024-04-25T11:21:10.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.246+0000 7fd44a000640 1 -- 172.21.15.7:0/3885492923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd44c01a070 con 0x7fd4540b8ff0 2024-04-25T11:21:10.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.247+0000 7fd44a000640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4340744b0 0x7fd4340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.247+0000 7fd44a000640 1 -- 172.21.15.7:0/3885492923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7fd44c094ac0 con 0x7fd4540b8ff0 2024-04-25T11:21:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.248+0000 7fd451e00640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4340744b0 0x7fd4340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.248+0000 7fd451e00640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4340744b0 0x7fd4340769c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd444006fd0 tx=0x7fd444008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.252+0000 7fd44a000640 1 -- 172.21.15.7:0/3885492923 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd44c05f020 con 0x7fd4540b8ff0 2024-04-25T11:21:10.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.457+0000 7fd453200640 1 -- 172.21.15.7:0/3885492923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd454050740 con 0x7fd4540b8ff0 2024-04-25T11:21:10.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.458+0000 7fd44a000640 1 -- 172.21.15.7:0/3885492923 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) ==== 100+0+11 (secure 0 0 0) 0x7fd44c0670a0 con 0x7fd4540b8ff0 2024-04-25T11:21:10.456 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:21:10.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.461+0000 7fd453200640 1 -- 172.21.15.7:0/3885492923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4340744b0 msgr2=0x7fd4340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.461+0000 7fd453200640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4340744b0 0x7fd4340769c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd444006fd0 tx=0x7fd444008040 comp rx=0 tx=0).stop 2024-04-25T11:21:10.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.461+0000 7fd453200640 1 -- 172.21.15.7:0/3885492923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4540b8ff0 msgr2=0x7fd4540ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.461+0000 7fd453200640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4540b8ff0 0x7fd4540ce060 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00c780 tx=0x7fd44c00cc40 comp rx=0 tx=0).stop 2024-04-25T11:21:10.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.461+0000 7fd453200640 1 -- 172.21.15.7:0/3885492923 shutdown_connections 2024-04-25T11:21:10.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.461+0000 7fd453200640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4340744b0 0x7fd4340769c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.461+0000 7fd453200640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4540b8ff0 0x7fd4540ce060 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.461+0000 7fd453200640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4540b4b80 0x7fd4540c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.461+0000 7fd453200640 1 --2- 172.21.15.7:0/3885492923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd45406c8a0 0x7fd45415bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.461+0000 7fd453200640 1 -- 172.21.15.7:0/3885492923 >> 172.21.15.7:0/3885492923 conn(0x7fd4540d5b80 msgr2=0x7fd45406dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:10.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.462+0000 7fd453200640 1 -- 172.21.15.7:0/3885492923 shutdown_connections 2024-04-25T11:21:10.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.462+0000 7fd453200640 1 -- 172.21.15.7:0/3885492923 wait complete. 2024-04-25T11:21:10.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:21:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.774+0000 7f6b6fe00640 1 Processor -- start 2024-04-25T11:21:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.774+0000 7f6b6fe00640 1 -- start start 2024-04-25T11:21:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.776+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b700b7cc0 0x7f6b700d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.777+0000 7f6b6fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6b700d32b0 0x7f6b700d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.777+0000 7f6b6fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b7015dea0 con 0x7f6b700d2ea0 2024-04-25T11:21:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.777+0000 7f6b6fe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6b700d2a10 con 0x7f6b700d32b0 2024-04-25T11:21:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.777+0000 7f6b6fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7015e180 con 0x7f6b700b7cc0 2024-04-25T11:21:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.777+0000 7f6b6f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6b700d32b0 0x7f6b700d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.777+0000 7f6b6e000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b700b7cc0 0x7f6b700d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.777+0000 7f6b6f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6b700d32b0 0x7f6b700d0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:36034/0 (socket says 172.21.15.7:36034) 2024-04-25T11:21:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.777+0000 7f6b6f400640 1 -- 172.21.15.7:0/1193708109 learned_addr learned my addr 172.21.15.7:0/1193708109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.778+0000 7f6b6f400640 1 -- 172.21.15.7:0/1193708109 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b700b7cc0 msgr2=0x7f6b700d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.778+0000 7f6b6f400640 1 --2- 172.21.15.7:0/1193708109 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b700b7cc0 0x7f6b700d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.778+0000 7f6b6f400640 1 -- 172.21.15.7:0/1193708109 >> v1:172.21.15.87:6789/0 conn(0x7f6b700d2ea0 legacy=0x7f6b700b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.778+0000 7f6b6f400640 1 -- 172.21.15.7:0/1193708109 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b700d26a0 con 0x7f6b700d32b0 2024-04-25T11:21:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.778+0000 7f6b6e000640 1 --2- 172.21.15.7:0/1193708109 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b700b7cc0 0x7f6b700d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:21:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.779+0000 7f6b6f400640 1 --2- 172.21.15.7:0/1193708109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b700d32b0 0x7f6b700d0900 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009e50 tx=0x7f6b6802c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=630d6560fcc19ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.780+0000 7f6b6d600640 1 -- 172.21.15.7:0/1193708109 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68032610 con 0x7f6b700d32b0 2024-04-25T11:21:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.780+0000 7f6b6d600640 1 -- 172.21.15.7:0/1193708109 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b68037070 con 0x7f6b700d32b0 2024-04-25T11:21:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.780+0000 7f6b6d600640 1 -- 172.21.15.7:0/1193708109 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b680329d0 con 0x7f6b700d32b0 2024-04-25T11:21:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.781+0000 7f6b6fe00640 1 -- 172.21.15.7:0/1193708109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b700d32b0 msgr2=0x7f6b700d0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.781+0000 7f6b6fe00640 1 --2- 172.21.15.7:0/1193708109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b700d32b0 0x7f6b700d0900 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009e50 tx=0x7f6b6802c920 comp rx=0 tx=0).stop 2024-04-25T11:21:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.781+0000 7f6b6fe00640 1 -- 172.21.15.7:0/1193708109 shutdown_connections 2024-04-25T11:21:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.781+0000 7f6b6fe00640 1 --2- 172.21.15.7:0/1193708109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b700d32b0 0x7f6b700d0900 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.781+0000 7f6b6fe00640 1 --2- 172.21.15.7:0/1193708109 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b700b7cc0 0x7f6b700d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.781+0000 7f6b6fe00640 1 -- 172.21.15.7:0/1193708109 >> 172.21.15.7:0/1193708109 conn(0x7f6b7009b540 msgr2=0x7f6b7009b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.781+0000 7f6b6fe00640 1 -- 172.21.15.7:0/1193708109 shutdown_connections 2024-04-25T11:21:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.781+0000 7f6b6fe00640 1 -- 172.21.15.7:0/1193708109 wait complete. 2024-04-25T11:21:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.784+0000 7f6b6fe00640 1 Processor -- start 2024-04-25T11:21:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.785+0000 7f6b6fe00640 1 -- start start 2024-04-25T11:21:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.787+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b700d2ea0 0x7f6b700d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.787+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b700d32b0 0x7f6b700af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.787+0000 7f6b6fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b700afdc0 0x7f6b7014c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.787+0000 7f6b6fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7015d0d0 con 0x7f6b700d2ea0 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.787+0000 7f6b6fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7015cf50 con 0x7f6b700afdc0 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.787+0000 7f6b6fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7015d250 con 0x7f6b700d32b0 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.787+0000 7f6b6ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b700d2ea0 0x7f6b700d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.787+0000 7f6b6ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b700d2ea0 0x7f6b700d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37498/0 (socket says 172.21.15.7:37498) 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.787+0000 7f6b6ea00640 1 -- 172.21.15.7:0/3236947083 learned_addr learned my addr 172.21.15.7:0/3236947083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.787+0000 7f6b6e000640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b700d32b0 0x7f6b700af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b6f400640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b700afdc0 0x7f6b7014c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b6ea00640 1 -- 172.21.15.7:0/3236947083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b700d32b0 msgr2=0x7f6b700af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b6ea00640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b700d32b0 0x7f6b700af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b6ea00640 1 -- 172.21.15.7:0/3236947083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b700afdc0 msgr2=0x7f6b7014c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b6ea00640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b700afdc0 0x7f6b7014c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b6ea00640 1 -- 172.21.15.7:0/3236947083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b700cb130 con 0x7f6b700d2ea0 2024-04-25T11:21:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b6ea00640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b700d2ea0 0x7f6b700d0b80 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000c3c0 tx=0x7f6b6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b66a00640 1 -- 172.21.15.7:0/3236947083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b600052d0 con 0x7f6b700d2ea0 2024-04-25T11:21:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b6fe00640 1 -- 172.21.15.7:0/3236947083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b700cb410 con 0x7f6b700d2ea0 2024-04-25T11:21:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b66a00640 1 -- 172.21.15.7:0/3236947083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b60005490 con 0x7f6b700d2ea0 2024-04-25T11:21:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b66a00640 1 -- 172.21.15.7:0/3236947083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60007260 con 0x7f6b700d2ea0 2024-04-25T11:21:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.788+0000 7f6b6fe00640 1 -- 172.21.15.7:0/3236947083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b700cb9d0 con 0x7f6b700d2ea0 2024-04-25T11:21:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.789+0000 7f6b6fe00640 1 -- 172.21.15.7:0/3236947083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b70050740 con 0x7f6b700d2ea0 2024-04-25T11:21:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.791+0000 7f6b66a00640 1 -- 172.21.15.7:0/3236947083 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6b60007400 con 0x7f6b700d2ea0 2024-04-25T11:21:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.792+0000 7f6b66a00640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b500744e0 0x7f6b500769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.792+0000 7f6b66a00640 1 -- 172.21.15.7:0/3236947083 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f6b600949f0 con 0x7f6b700d2ea0 2024-04-25T11:21:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.793+0000 7f6b6e000640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b500744e0 0x7f6b500769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:10.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.793+0000 7f6b6e000640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b500744e0 0x7f6b500769f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6b700cd5e0 tx=0x7f6b5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:10.797+0000 7f6b66a00640 1 -- 172.21.15.7:0/3236947083 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b6005eed0 con 0x7f6b700d2ea0 2024-04-25T11:21:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.006+0000 7f6b6fe00640 1 -- 172.21.15.7:0/3236947083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6b700cee90 con 0x7f6b700d2ea0 2024-04-25T11:21:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.007+0000 7f6b66a00640 1 -- 172.21.15.7:0/3236947083 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v130) ==== 80+0+0 (secure 0 0 0) 0x7f6b6005e7d0 con 0x7f6b700d2ea0 2024-04-25T11:21:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:21:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.009+0000 7f6b6fe00640 1 -- 172.21.15.7:0/3236947083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b500744e0 msgr2=0x7f6b500769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.010+0000 7f6b6fe00640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b500744e0 0x7f6b500769f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6b700cd5e0 tx=0x7f6b5c008040 comp rx=0 tx=0).stop 2024-04-25T11:21:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.010+0000 7f6b6fe00640 1 -- 172.21.15.7:0/3236947083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b700d2ea0 msgr2=0x7f6b700d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.010+0000 7f6b6fe00640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b700d2ea0 0x7f6b700d0b80 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000c3c0 tx=0x7f6b6000c880 comp rx=0 tx=0).stop 2024-04-25T11:21:11.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.010+0000 7f6b6fe00640 1 -- 172.21.15.7:0/3236947083 shutdown_connections 2024-04-25T11:21:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.010+0000 7f6b6fe00640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b500744e0 0x7f6b500769f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.010+0000 7f6b6fe00640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b700afdc0 0x7f6b7014c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.010+0000 7f6b6fe00640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b700d32b0 0x7f6b700af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.010+0000 7f6b6fe00640 1 --2- 172.21.15.7:0/3236947083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b700d2ea0 0x7f6b700d0b80 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.010+0000 7f6b6fe00640 1 -- 172.21.15.7:0/3236947083 >> 172.21.15.7:0/3236947083 conn(0x7f6b7009b540 msgr2=0x7f6b700b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.010+0000 7f6b6fe00640 1 -- 172.21.15.7:0/3236947083 shutdown_connections 2024-04-25T11:21:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.010+0000 7f6b6fe00640 1 -- 172.21.15.7:0/3236947083 wait complete. 2024-04-25T11:21:11.036 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:21:11.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:11.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:21:11.256 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:21:11.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:11.257 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:21:11.296 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:21:11.296 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:21:11.297 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:11.336 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:21:11.336 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:11.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:21:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.713+0000 7f262b600640 1 Processor -- start 2024-04-25T11:21:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.713+0000 7f262b600640 1 -- start start 2024-04-25T11:21:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.715+0000 7f262b600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f262406f300 0x7f26240b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.715+0000 7f262b600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26240b4980 con 0x7f26240b0290 2024-04-25T11:21:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.715+0000 7f262b600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26240b4c60 con 0x7f262406dc90 2024-04-25T11:21:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.715+0000 7f262b600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f26240b4f40 con 0x7f262406f300 2024-04-25T11:21:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.716+0000 7f2629600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f262406f300 0x7f26240b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.716+0000 7f2623e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f26240b0290 0x7f26240b5a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39910/0 (socket says 172.21.15.7:39910) 2024-04-25T11:21:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.716+0000 7f2623e00640 1 -- 172.21.15.7:0/1408254768 learned_addr learned my addr 172.21.15.7:0/1408254768 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.716+0000 7f2629600640 1 -- 172.21.15.7:0/1408254768 >> v1:172.21.15.87:6789/0 conn(0x7f262406dc90 legacy=0x7f26240afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.716+0000 7f2629600640 1 -- 172.21.15.7:0/1408254768 >> v1:172.21.15.7:6789/0 conn(0x7f26240b0290 legacy=0x7f26240b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.716+0000 7f2629600640 1 -- 172.21.15.7:0/1408254768 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26240b5220 con 0x7f262406f300 2024-04-25T11:21:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.717+0000 7f2629600640 1 --2- 172.21.15.7:0/1408254768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f262406f300 0x7f26240b6130 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f261c009d90 tx=0x7f261c02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3f502d4456f3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.718+0000 7f2623400640 1 -- 172.21.15.7:0/1408254768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c032770 con 0x7f262406f300 2024-04-25T11:21:11.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.718+0000 7f2623400640 1 -- 172.21.15.7:0/1408254768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f261c038070 con 0x7f262406f300 2024-04-25T11:21:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.718+0000 7f2623400640 1 -- 172.21.15.7:0/1408254768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c032b30 con 0x7f262406f300 2024-04-25T11:21:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.718+0000 7f262b600640 1 -- 172.21.15.7:0/1408254768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f262406f300 msgr2=0x7f26240b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.718+0000 7f262b600640 1 --2- 172.21.15.7:0/1408254768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f262406f300 0x7f26240b6130 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f261c009d90 tx=0x7f261c02c8b0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.718+0000 7f262b600640 1 -- 172.21.15.7:0/1408254768 shutdown_connections 2024-04-25T11:21:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.718+0000 7f262b600640 1 --2- 172.21.15.7:0/1408254768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f262406f300 0x7f26240b6130 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.718+0000 7f262b600640 1 -- 172.21.15.7:0/1408254768 >> 172.21.15.7:0/1408254768 conn(0x7f2624074200 msgr2=0x7f262406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.718+0000 7f262b600640 1 -- 172.21.15.7:0/1408254768 shutdown_connections 2024-04-25T11:21:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.719+0000 7f262b600640 1 -- 172.21.15.7:0/1408254768 wait complete. 2024-04-25T11:21:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.721+0000 7f262b600640 1 Processor -- start 2024-04-25T11:21:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.721+0000 7f262b600640 1 -- start start 2024-04-25T11:21:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.723+0000 7f262b600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262406f300 0x7f262415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.723+0000 7f262b600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f262414b540 0x7f2624158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.723+0000 7f262b600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f262414b950 0x7f26241588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.723+0000 7f262b600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26240ce920 con 0x7f262414b540 2024-04-25T11:21:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.723+0000 7f262b600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f262415d220 con 0x7f262406f300 2024-04-25T11:21:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.723+0000 7f262b600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f26240ceaa0 con 0x7f262414b950 2024-04-25T11:21:11.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.723+0000 7f2623e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f262414b540 0x7f2624158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.723+0000 7f2623e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f262414b540 0x7f2624158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37510/0 (socket says 172.21.15.7:37510) 2024-04-25T11:21:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.723+0000 7f2623e00640 1 -- 172.21.15.7:0/324868210 learned_addr learned my addr 172.21.15.7:0/324868210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2629600640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f262414b950 0x7f26241588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2628c00640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262406f300 0x7f262415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2623e00640 1 -- 172.21.15.7:0/324868210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f262414b950 msgr2=0x7f26241588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2623e00640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f262414b950 0x7f26241588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2623e00640 1 -- 172.21.15.7:0/324868210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262406f300 msgr2=0x7f262415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2623e00640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262406f300 0x7f262415ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2623e00640 1 -- 172.21.15.7:0/324868210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262415a940 con 0x7f262414b540 2024-04-25T11:21:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2628c00640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262406f300 0x7f262415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2623e00640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f262414b540 0x7f2624158370 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f260c00c3c0 tx=0x7f260c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2620c00640 1 -- 172.21.15.7:0/324868210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c017070 con 0x7f262414b540 2024-04-25T11:21:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2620c00640 1 -- 172.21.15.7:0/324868210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f260c005ce0 con 0x7f262414b540 2024-04-25T11:21:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f2620c00640 1 -- 172.21.15.7:0/324868210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c007310 con 0x7f262414b540 2024-04-25T11:21:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.724+0000 7f262b600640 1 -- 172.21.15.7:0/324868210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262415ac20 con 0x7f262414b540 2024-04-25T11:21:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.725+0000 7f262b600640 1 -- 172.21.15.7:0/324868210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262415b1e0 con 0x7f262414b540 2024-04-25T11:21:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.726+0000 7f262b600640 1 -- 172.21.15.7:0/324868210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262406a100 con 0x7f262414b540 2024-04-25T11:21:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.727+0000 7f2620c00640 1 -- 172.21.15.7:0/324868210 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f260c01a070 con 0x7f262414b540 2024-04-25T11:21:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.728+0000 7f2620c00640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26000744b0 0x7f26000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.728+0000 7f2620c00640 1 -- 172.21.15.7:0/324868210 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f260c094710 con 0x7f262414b540 2024-04-25T11:21:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.728+0000 7f2628c00640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26000744b0 0x7f26000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.729+0000 7f2628c00640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26000744b0 0x7f26000769c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f26240730d0 tx=0x7f2614008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:11.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.733+0000 7f2620c00640 1 -- 172.21.15.7:0/324868210 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f260c05ed10 con 0x7f262414b540 2024-04-25T11:21:11.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.939+0000 7f262b600640 1 -- 172.21.15.7:0/324868210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2624070fb0 con 0x7f262414b540 2024-04-25T11:21:11.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.942+0000 7f2620c00640 1 -- 172.21.15.7:0/324868210 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16804 (secure 0 0 0) 0x7f260c05e610 con 0x7f262414b540 2024-04-25T11:21:11.939 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:11.939 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":130,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:20:44.396251+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:41.047420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:41.908907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:11.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 -- 172.21.15.7:0/324868210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26000744b0 msgr2=0x7f26000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26000744b0 0x7f26000769c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f26240730d0 tx=0x7f2614008040 comp rx=0 tx=0).stop 2024-04-25T11:21:11.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 -- 172.21.15.7:0/324868210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f262414b540 msgr2=0x7f2624158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:11.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f262414b540 0x7f2624158370 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f260c00c3c0 tx=0x7f260c00c880 comp rx=0 tx=0).stop 2024-04-25T11:21:11.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 -- 172.21.15.7:0/324868210 shutdown_connections 2024-04-25T11:21:11.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f26000744b0 0x7f26000769c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f262414b950 0x7f26241588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f262414b540 0x7f2624158370 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 --2- 172.21.15.7:0/324868210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f262406f300 0x7f262415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:11.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 -- 172.21.15.7:0/324868210 >> 172.21.15.7:0/324868210 conn(0x7f2624074200 msgr2=0x7f26240b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:11.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 -- 172.21.15.7:0/324868210 shutdown_connections 2024-04-25T11:21:11.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:11.945+0000 7f262b600640 1 -- 172.21.15.7:0/324868210 wait complete. 2024-04-25T11:21:11.969 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:20:41.047420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:20:41.908907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:21:11.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:21:12.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.268+0000 7fa434800640 1 Processor -- start 2024-04-25T11:21:12.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.268+0000 7fa434800640 1 -- start start 2024-04-25T11:21:12.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.271+0000 7fa434800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa42c0b6720 0x7fa42c164710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.271+0000 7fa434800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa42c0b1390 con 0x7fa42c06f3a0 2024-04-25T11:21:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.271+0000 7fa434800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa42c0ae880 con 0x7fa42c157320 2024-04-25T11:21:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.271+0000 7fa434800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa42c160a60 con 0x7fa42c0b6720 2024-04-25T11:21:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.271+0000 7fa432800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa42c06f3a0 0x7fa42c06f7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39920/0 (socket says 172.21.15.7:39920) 2024-04-25T11:21:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.271+0000 7fa432800640 1 -- 172.21.15.7:0/7753771 learned_addr learned my addr 172.21.15.7:0/7753771 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa431e00640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa42c0b6720 0x7fa42c164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3878868458 0 0) 0x7fa42c0b1390 con 0x7fa42c06f3a0 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa414003520 con 0x7fa42c06f3a0 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3957426823 0 0) 0x7fa414003520 con 0x7fa42c06f3a0 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa42c0b6720 msgr2=0x7fa42c164710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa42c0b6720 0x7fa42c164710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 >> v1:172.21.15.87:6789/0 conn(0x7fa42c157320 legacy=0x7fa42c160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c160710 con 0x7fa42c06f3a0 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fa4280019f0 con 0x7fa42c06f3a0 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 >> v1:172.21.15.7:6789/0 conn(0x7fa42c06f3a0 legacy=0x7fa42c06f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140039e0 0x7fa414003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa42c0b6720 0x7fa414004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa414004d40 0x7fa414005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c160a60 con 0x7fa4140039e0 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa41400bb50 con 0x7fa414004d40 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.272+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa41400bd00 con 0x7fa42c0b6720 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.273+0000 7fa431e00640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140039e0 0x7fa414003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.273+0000 7fa431e00640 1 -- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa42c0b6720 msgr2=0x7fa414004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.273+0000 7fa431400640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa42c0b6720 0x7fa414004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.273+0000 7fa431e00640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa42c0b6720 0x7fa414004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.273+0000 7fa431e00640 1 -- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa414004d40 msgr2=0x7fa414005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.273+0000 7fa432800640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa414004d40 0x7fa414005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.273+0000 7fa431e00640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa414004d40 0x7fa414005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.273+0000 7fa431e00640 1 -- 172.21.15.7:0/7753771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa414003520 con 0x7fa4140039e0 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.273+0000 7fa432800640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa414004d40 0x7fa414005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:12.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.273+0000 7fa431400640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa42c0b6720 0x7fa414004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.275+0000 7fa431e00640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140039e0 0x7fa414003e70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa420009a90 tx=0x7fa414005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939edc4df6a9fb83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.275+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420036070 con 0x7fa4140039e0 2024-04-25T11:21:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.275+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa420031400 con 0x7fa4140039e0 2024-04-25T11:21:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.275+0000 7fa430a00640 1 -- 172.21.15.7:0/7753771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420009690 con 0x7fa4140039e0 2024-04-25T11:21:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.276+0000 7fa434800640 1 -- 172.21.15.7:0/7753771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140039e0 msgr2=0x7fa414003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.276+0000 7fa434800640 1 --2- 172.21.15.7:0/7753771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4140039e0 0x7fa414003e70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa420009a90 tx=0x7fa414005830 comp rx=0 tx=0).stop 2024-04-25T11:21:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.276+0000 7fa432800640 1 -- 172.21.15.7:0/7753771 reap_dead start 2024-04-25T11:21:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.276+0000 7fa434800640 1 -- 172.21.15.7:0/7753771 shutdown_connections 2024-04-25T11:21:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.276+0000 7fa434800640 1 -- 172.21.15.7:0/7753771 >> 172.21.15.7:0/7753771 conn(0x7fa42c0742c0 msgr2=0x7fa42c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.276+0000 7fa434800640 1 -- 172.21.15.7:0/7753771 shutdown_connections 2024-04-25T11:21:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.276+0000 7fa434800640 1 -- 172.21.15.7:0/7753771 wait complete. 2024-04-25T11:21:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.278+0000 7fa434800640 1 Processor -- start 2024-04-25T11:21:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.278+0000 7fa434800640 1 -- start start 2024-04-25T11:21:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.280+0000 7fa434800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4140039e0 0x7fa42c144710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.280+0000 7fa434800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa42c14f5c0 0x7fa42c153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.280+0000 7fa434800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa42c1536c0 0x7fa42c159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.280+0000 7fa434800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c0aee00 con 0x7fa42c14f5c0 2024-04-25T11:21:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.280+0000 7fa434800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa42c0aeb20 con 0x7fa42c1536c0 2024-04-25T11:21:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.280+0000 7fa434800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa42c164710 con 0x7fa4140039e0 2024-04-25T11:21:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.280+0000 7fa431400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa42c14f5c0 0x7fa42c153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.280+0000 7fa431400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa42c14f5c0 0x7fa42c153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37526/0 (socket says 172.21.15.7:37526) 2024-04-25T11:21:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.280+0000 7fa431400640 1 -- 172.21.15.7:0/1743353676 learned_addr learned my addr 172.21.15.7:0/1743353676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.280+0000 7fa431e00640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4140039e0 0x7fa42c144710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.281+0000 7fa432800640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa42c1536c0 0x7fa42c159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.281+0000 7fa431400640 1 -- 172.21.15.7:0/1743353676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4140039e0 msgr2=0x7fa42c144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.281+0000 7fa431400640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4140039e0 0x7fa42c144710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.281+0000 7fa431400640 1 -- 172.21.15.7:0/1743353676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa42c1536c0 msgr2=0x7fa42c159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.281+0000 7fa431400640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa42c1536c0 0x7fa42c159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.281+0000 7fa431400640 1 -- 172.21.15.7:0/1743353676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42c153ad0 con 0x7fa42c14f5c0 2024-04-25T11:21:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.281+0000 7fa431e00640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4140039e0 0x7fa42c144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:21:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.281+0000 7fa431400640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa42c14f5c0 0x7fa42c153180 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00d7a0 tx=0x7fa41c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.281+0000 7fa426000640 1 -- 172.21.15.7:0/1743353676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c005070 con 0x7fa42c14f5c0 2024-04-25T11:21:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.282+0000 7fa426000640 1 -- 172.21.15.7:0/1743353676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa41c005210 con 0x7fa42c14f5c0 2024-04-25T11:21:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.282+0000 7fa426000640 1 -- 172.21.15.7:0/1743353676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41c007030 con 0x7fa42c14f5c0 2024-04-25T11:21:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.282+0000 7fa434800640 1 -- 172.21.15.7:0/1743353676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42c141a60 con 0x7fa42c14f5c0 2024-04-25T11:21:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.282+0000 7fa434800640 1 -- 172.21.15.7:0/1743353676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42c150250 con 0x7fa42c14f5c0 2024-04-25T11:21:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.283+0000 7fa434800640 1 -- 172.21.15.7:0/1743353676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa42c050740 con 0x7fa42c14f5c0 2024-04-25T11:21:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.284+0000 7fa426000640 1 -- 172.21.15.7:0/1743353676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa41c01a020 con 0x7fa42c14f5c0 2024-04-25T11:21:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.285+0000 7fa426000640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4100744b0 0x7fa4100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.285+0000 7fa426000640 1 -- 172.21.15.7:0/1743353676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7fa41c094cf0 con 0x7fa42c14f5c0 2024-04-25T11:21:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.286+0000 7fa431e00640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4100744b0 0x7fa4100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.286+0000 7fa431e00640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4100744b0 0x7fa4100769c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa420033000 tx=0x7fa420008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:12.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.290+0000 7fa426000640 1 -- 172.21.15.7:0/1743353676 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa42c153ad0 con 0x7fa42c14f5c0 2024-04-25T11:21:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.493+0000 7fa434800640 1 -- 172.21.15.7:0/1743353676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa42c151e90 con 0x7fa42c14f5c0 2024-04-25T11:21:12.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.495+0000 7fa426000640 1 -- 172.21.15.7:0/1743353676 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) ==== 93+0+10 (secure 0 0 0) 0x7fa41c05f250 con 0x7fa42c14f5c0 2024-04-25T11:21:12.515 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:21:12.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.497+0000 7fa434800640 1 -- 172.21.15.7:0/1743353676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4100744b0 msgr2=0x7fa4100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.497+0000 7fa434800640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4100744b0 0x7fa4100769c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa420033000 tx=0x7fa420008030 comp rx=0 tx=0).stop 2024-04-25T11:21:12.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.497+0000 7fa434800640 1 -- 172.21.15.7:0/1743353676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa42c14f5c0 msgr2=0x7fa42c153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.497+0000 7fa434800640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa42c14f5c0 0x7fa42c153180 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00d7a0 tx=0x7fa41c00dc60 comp rx=0 tx=0).stop 2024-04-25T11:21:12.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.497+0000 7fa434800640 1 -- 172.21.15.7:0/1743353676 shutdown_connections 2024-04-25T11:21:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.497+0000 7fa434800640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4100744b0 0x7fa4100769c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.497+0000 7fa434800640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa42c1536c0 0x7fa42c159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.497+0000 7fa434800640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa42c14f5c0 0x7fa42c153180 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.497+0000 7fa434800640 1 --2- 172.21.15.7:0/1743353676 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4140039e0 0x7fa42c144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.497+0000 7fa434800640 1 -- 172.21.15.7:0/1743353676 >> 172.21.15.7:0/1743353676 conn(0x7fa42c0742c0 msgr2=0x7fa42c0b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.498+0000 7fa434800640 1 -- 172.21.15.7:0/1743353676 shutdown_connections 2024-04-25T11:21:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.498+0000 7fa434800640 1 -- 172.21.15.7:0/1743353676 wait complete. 2024-04-25T11:21:12.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:21:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.811+0000 7f75cb000640 1 Processor -- start 2024-04-25T11:21:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.812+0000 7f75cb000640 1 -- start start 2024-04-25T11:21:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.814+0000 7f75cb000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f75c40b6240 0x7f75c415eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.814+0000 7f75cb000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c4164e00 con 0x7f75c40aef90 2024-04-25T11:21:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.814+0000 7f75cb000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c41650e0 con 0x7f75c40b6920 2024-04-25T11:21:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.814+0000 7f75cb000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f75c41653c0 con 0x7f75c40b6240 2024-04-25T11:21:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.815+0000 7f75c9000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f75c40aef90 0x7f75c40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39930/0 (socket says 172.21.15.7:39930) 2024-04-25T11:21:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.815+0000 7f75c9000640 1 -- 172.21.15.7:0/2382356384 learned_addr learned my addr 172.21.15.7:0/2382356384 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.815+0000 7f75c3e00640 1 --2- 172.21.15.7:0/2382356384 >> v2:172.21.15.87:3300/0 conn(0x7f75c40b6240 0x7f75c415eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.815+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3824587452 0 0) 0x7f75c4164e00 con 0x7f75c40aef90 2024-04-25T11:21:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75a8003520 con 0x7f75c40aef90 2024-04-25T11:21:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3959549309 0 0) 0x7f75a8003520 con 0x7f75c40aef90 2024-04-25T11:21:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 >> v2:172.21.15.87:3300/0 conn(0x7f75c40b6240 msgr2=0x7f75c415eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 --2- 172.21.15.7:0/2382356384 >> v2:172.21.15.87:3300/0 conn(0x7f75c40b6240 0x7f75c415eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 >> v1:172.21.15.87:6789/0 conn(0x7f75c40b6920 legacy=0x7f75c40ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c41656a0 con 0x7f75c40aef90 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f75bc0019f0 con 0x7f75c40aef90 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 >> v1:172.21.15.7:6789/0 conn(0x7f75c40aef90 legacy=0x7f75c40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 0x7f75a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75a8004800 0x7f75a8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a8005250 0x7f75a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75c41653c0 con 0x7f75a80039b0 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f75a800c1e0 con 0x7f75a8005250 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f75a800c3e0 con 0x7f75a8004800 2024-04-25T11:21:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.816+0000 7f75c3e00640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 0x7f75a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.817+0000 7f75c9000640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a8005250 0x7f75a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.817+0000 7f75c3e00640 1 -- 172.21.15.7:0/2382356384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75a8004800 msgr2=0x7f75a8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.817+0000 7f75c3400640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75a8004800 0x7f75a8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.817+0000 7f75c3e00640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75a8004800 0x7f75a8004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.817+0000 7f75c3e00640 1 -- 172.21.15.7:0/2382356384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a8005250 msgr2=0x7f75a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.817+0000 7f75c3e00640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a8005250 0x7f75a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.817+0000 7f75c3e00640 1 -- 172.21.15.7:0/2382356384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a8003520 con 0x7f75a80039b0 2024-04-25T11:21:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.817+0000 7f75c9000640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75a8005250 0x7f75a800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.817+0000 7f75c9000640 1 -- 172.21.15.7:0/2382356384 reap_dead start 2024-04-25T11:21:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.818+0000 7f75c3e00640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 0x7f75a8003e20 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f75c40714e0 tx=0x7f75a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9d2fea28f5eee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.818+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4036070 con 0x7f75a80039b0 2024-04-25T11:21:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.818+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b4031400 con 0x7f75a80039b0 2024-04-25T11:21:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.818+0000 7f75c2a00640 1 -- 172.21.15.7:0/2382356384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4009690 con 0x7f75a80039b0 2024-04-25T11:21:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.819+0000 7f75cb000640 1 -- 172.21.15.7:0/2382356384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 msgr2=0x7f75a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.819+0000 7f75cb000640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 0x7f75a8003e20 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f75c40714e0 tx=0x7f75a8005830 comp rx=0 tx=0).stop 2024-04-25T11:21:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.819+0000 7f75cb000640 1 -- 172.21.15.7:0/2382356384 shutdown_connections 2024-04-25T11:21:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.819+0000 7f75cb000640 1 --2- 172.21.15.7:0/2382356384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 0x7f75a8003e20 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.819+0000 7f75cb000640 1 -- 172.21.15.7:0/2382356384 >> 172.21.15.7:0/2382356384 conn(0x7f75c40742c0 msgr2=0x7f75c406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.819+0000 7f75cb000640 1 -- 172.21.15.7:0/2382356384 shutdown_connections 2024-04-25T11:21:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.819+0000 7f75cb000640 1 -- 172.21.15.7:0/2382356384 wait complete. 2024-04-25T11:21:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.822+0000 7f75cb000640 1 Processor -- start 2024-04-25T11:21:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.822+0000 7f75cb000640 1 -- start start 2024-04-25T11:21:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.823+0000 7f75cb000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 0x7f75c41fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.823+0000 7f75cb000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75c415f360 0x7f75c415f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.823+0000 7f75cb000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75c41fa8a0 0x7f75c41facb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.823+0000 7f75cb000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75c4164ba0 con 0x7f75a80039b0 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.823+0000 7f75cb000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f75c4164a20 con 0x7f75c415f360 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.823+0000 7f75cb000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f75c415eb00 con 0x7f75c41fa8a0 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.824+0000 7f75c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 0x7f75c41fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.824+0000 7f75c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 0x7f75c41fd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37534/0 (socket says 172.21.15.7:37534) 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.824+0000 7f75c3e00640 1 -- 172.21.15.7:0/3349346666 learned_addr learned my addr 172.21.15.7:0/3349346666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.824+0000 7f75c3400640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75c415f360 0x7f75c415f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.824+0000 7f75c9000640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75c41fa8a0 0x7f75c41facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.824+0000 7f75c3e00640 1 -- 172.21.15.7:0/3349346666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75c41fa8a0 msgr2=0x7f75c41facb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.824+0000 7f75c3e00640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75c41fa8a0 0x7f75c41facb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.824+0000 7f75c3e00640 1 -- 172.21.15.7:0/3349346666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75c415f360 msgr2=0x7f75c415f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.824+0000 7f75c3e00640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75c415f360 0x7f75c415f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.824+0000 7f75c3e00640 1 -- 172.21.15.7:0/3349346666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c41624d0 con 0x7f75a80039b0 2024-04-25T11:21:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.824+0000 7f75c3400640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75c415f360 0x7f75c415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:12.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.825+0000 7f75c3e00640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 0x7f75c41fd810 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f75b4000c00 tx=0x7f75b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:12.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.825+0000 7f75bbe00640 1 -- 172.21.15.7:0/3349346666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4040030 con 0x7f75a80039b0 2024-04-25T11:21:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.825+0000 7f75bbe00640 1 -- 172.21.15.7:0/3349346666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b403b460 con 0x7f75a80039b0 2024-04-25T11:21:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.825+0000 7f75bbe00640 1 -- 172.21.15.7:0/3349346666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4036040 con 0x7f75a80039b0 2024-04-25T11:21:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.825+0000 7f75cb000640 1 -- 172.21.15.7:0/3349346666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c4162740 con 0x7f75a80039b0 2024-04-25T11:21:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.825+0000 7f75cb000640 1 -- 172.21.15.7:0/3349346666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c4162d00 con 0x7f75a80039b0 2024-04-25T11:21:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.826+0000 7f75cb000640 1 -- 172.21.15.7:0/3349346666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c4050740 con 0x7f75a80039b0 2024-04-25T11:21:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.828+0000 7f75bbe00640 1 -- 172.21.15.7:0/3349346666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f75b4043070 con 0x7f75a80039b0 2024-04-25T11:21:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.829+0000 7f75bbe00640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75a40744b0 0x7f75a40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.829+0000 7f75bbe00640 1 -- 172.21.15.7:0/3349346666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f75b40b5370 con 0x7f75a80039b0 2024-04-25T11:21:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.829+0000 7f75c3400640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75a40744b0 0x7f75a40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:12.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.830+0000 7f75c3400640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75a40744b0 0x7f75a40769c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f75b0007fd0 tx=0x7f75b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:12.834+0000 7f75bbe00640 1 -- 172.21.15.7:0/3349346666 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f75b407f8d0 con 0x7f75a80039b0 2024-04-25T11:21:13.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.038+0000 7f75cb000640 1 -- 172.21.15.7:0/3349346666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f75c40710c0 con 0x7f75a80039b0 2024-04-25T11:21:13.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.040+0000 7f75bbe00640 1 -- 172.21.15.7:0/3349346666 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) ==== 104+0+11 (secure 0 0 0) 0x7f75b407f1d0 con 0x7f75a80039b0 2024-04-25T11:21:13.037 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:21:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.042+0000 7f75cb000640 1 -- 172.21.15.7:0/3349346666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75a40744b0 msgr2=0x7f75a40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.042+0000 7f75cb000640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75a40744b0 0x7f75a40769c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f75b0007fd0 tx=0x7f75b0009040 comp rx=0 tx=0).stop 2024-04-25T11:21:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.042+0000 7f75cb000640 1 -- 172.21.15.7:0/3349346666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 msgr2=0x7f75c41fd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.042+0000 7f75cb000640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 0x7f75c41fd810 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f75b4000c00 tx=0x7f75b4004730 comp rx=0 tx=0).stop 2024-04-25T11:21:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.043+0000 7f75cb000640 1 -- 172.21.15.7:0/3349346666 shutdown_connections 2024-04-25T11:21:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.043+0000 7f75cb000640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f75a40744b0 0x7f75a40769c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.043+0000 7f75cb000640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f75c41fa8a0 0x7f75c41facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.043+0000 7f75cb000640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f75c415f360 0x7f75c415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.043+0000 7f75cb000640 1 --2- 172.21.15.7:0/3349346666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75a80039b0 0x7f75c41fd810 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.043+0000 7f75cb000640 1 -- 172.21.15.7:0/3349346666 >> 172.21.15.7:0/3349346666 conn(0x7f75c40742c0 msgr2=0x7f75c40ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.043+0000 7f75cb000640 1 -- 172.21.15.7:0/3349346666 shutdown_connections 2024-04-25T11:21:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.043+0000 7f75cb000640 1 -- 172.21.15.7:0/3349346666 wait complete. 2024-04-25T11:21:13.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:21:13.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.359+0000 7f1cafe00640 1 Processor -- start 2024-04-25T11:21:13.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.359+0000 7f1cafe00640 1 -- start start 2024-04-25T11:21:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.361+0000 7f1cafe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1cb0070630 0x7f1cb00cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.362+0000 7f1cafe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cb015ecc0 0x7f1cb00d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.362+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cb00d0800 0x7f1cb006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.362+0000 7f1cafe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1cb00d0c10 con 0x7f1cb015ecc0 2024-04-25T11:21:13.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.362+0000 7f1cafe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1cb00b0be0 con 0x7f1cb0070630 2024-04-25T11:21:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.362+0000 7f1cafe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb00ce9e0 con 0x7f1cb00d0800 2024-04-25T11:21:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.362+0000 7f1cae000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cb015ecc0 0x7f1cb00d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.362+0000 7f1cae000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cb015ecc0 0x7f1cb00d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37538/0 (socket says 172.21.15.7:37538) 2024-04-25T11:21:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.362+0000 7f1cae000640 1 -- 172.21.15.7:0/2110733345 learned_addr learned my addr 172.21.15.7:0/2110733345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.362+0000 7f1caf400640 1 --2- 172.21.15.7:0/2110733345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cb00d0800 0x7f1cb006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.362+0000 7f1caea00640 1 --2- 172.21.15.7:0/2110733345 >> v2:172.21.15.87:3300/0 conn(0x7f1cb0070630 0x7f1cb00cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.363+0000 7f1cae000640 1 -- 172.21.15.7:0/2110733345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cb00d0800 msgr2=0x7f1cb006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.363+0000 7f1cae000640 1 --2- 172.21.15.7:0/2110733345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cb00d0800 0x7f1cb006eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.363+0000 7f1cae000640 1 -- 172.21.15.7:0/2110733345 >> v2:172.21.15.87:3300/0 conn(0x7f1cb0070630 msgr2=0x7f1cb00cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.363+0000 7f1cae000640 1 --2- 172.21.15.7:0/2110733345 >> v2:172.21.15.87:3300/0 conn(0x7f1cb0070630 0x7f1cb00cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.363+0000 7f1cae000640 1 -- 172.21.15.7:0/2110733345 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb00cecc0 con 0x7f1cb015ecc0 2024-04-25T11:21:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.363+0000 7f1caf400640 1 --2- 172.21.15.7:0/2110733345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cb00d0800 0x7f1cb006eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:21:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.364+0000 7f1cae000640 1 --2- 172.21.15.7:0/2110733345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb015ecc0 0x7f1cb00d02c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f1cb006e0f0 tx=0x7f1c9802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc57f0757184f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:13.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.364+0000 7f1cad600640 1 -- 172.21.15.7:0/2110733345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98037070 con 0x7f1cb015ecc0 2024-04-25T11:21:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.365+0000 7f1cad600640 1 -- 172.21.15.7:0/2110733345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c98032440 con 0x7f1cb015ecc0 2024-04-25T11:21:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.365+0000 7f1cad600640 1 -- 172.21.15.7:0/2110733345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98032810 con 0x7f1cb015ecc0 2024-04-25T11:21:13.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.365+0000 7f1cafe00640 1 -- 172.21.15.7:0/2110733345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb015ecc0 msgr2=0x7f1cb00d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.365+0000 7f1cafe00640 1 --2- 172.21.15.7:0/2110733345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb015ecc0 0x7f1cb00d02c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f1cb006e0f0 tx=0x7f1c9802a1d0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.365+0000 7f1cafe00640 1 -- 172.21.15.7:0/2110733345 shutdown_connections 2024-04-25T11:21:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.365+0000 7f1cafe00640 1 --2- 172.21.15.7:0/2110733345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cb00d0800 0x7f1cb006eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.365+0000 7f1cafe00640 1 --2- 172.21.15.7:0/2110733345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb015ecc0 0x7f1cb00d02c0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.365+0000 7f1cafe00640 1 --2- 172.21.15.7:0/2110733345 >> v2:172.21.15.87:3300/0 conn(0x7f1cb0070630 0x7f1cb00cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.365+0000 7f1cafe00640 1 -- 172.21.15.7:0/2110733345 >> 172.21.15.7:0/2110733345 conn(0x7f1cb0074140 msgr2=0x7f1cb006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.366+0000 7f1cafe00640 1 -- 172.21.15.7:0/2110733345 shutdown_connections 2024-04-25T11:21:13.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.366+0000 7f1cafe00640 1 -- 172.21.15.7:0/2110733345 wait complete. 2024-04-25T11:21:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.368+0000 7f1cafe00640 1 Processor -- start 2024-04-25T11:21:13.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.368+0000 7f1cafe00640 1 -- start start 2024-04-25T11:21:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.370+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb0070630 0x7f1cb0152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.370+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cb00d0800 0x7f1cb00ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.370+0000 7f1cafe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cb015ecc0 0x7f1cb01603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.370+0000 7f1cafe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb00cf560 con 0x7f1cb0070630 2024-04-25T11:21:13.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.370+0000 7f1cafe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb00cf280 con 0x7f1cb00d0800 2024-04-25T11:21:13.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.370+0000 7f1cafe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb015f0d0 con 0x7f1cb015ecc0 2024-04-25T11:21:13.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.370+0000 7f1caea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb0070630 0x7f1cb0152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.370+0000 7f1caea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb0070630 0x7f1cb0152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37550/0 (socket says 172.21.15.7:37550) 2024-04-25T11:21:13.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.370+0000 7f1caea00640 1 -- 172.21.15.7:0/4138819404 learned_addr learned my addr 172.21.15.7:0/4138819404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.370+0000 7f1caf400640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cb015ecc0 0x7f1cb01603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.370+0000 7f1cae000640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cb00d0800 0x7f1cb00ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.371+0000 7f1caea00640 1 -- 172.21.15.7:0/4138819404 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cb015ecc0 msgr2=0x7f1cb01603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.371+0000 7f1caea00640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cb015ecc0 0x7f1cb01603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.371+0000 7f1caea00640 1 -- 172.21.15.7:0/4138819404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cb00d0800 msgr2=0x7f1cb00ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.371+0000 7f1caea00640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cb00d0800 0x7f1cb00ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.371+0000 7f1caea00640 1 -- 172.21.15.7:0/4138819404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb00ac6b0 con 0x7f1cb0070630 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.371+0000 7f1cae000640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cb00d0800 0x7f1cb00ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.371+0000 7f1caea00640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb0070630 0x7f1cb0152050 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000d800 tx=0x7f1ca000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.371+0000 7f1ca6a00640 1 -- 172.21.15.7:0/4138819404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0018030 con 0x7f1cb0070630 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.371+0000 7f1ca6a00640 1 -- 172.21.15.7:0/4138819404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ca00096a0 con 0x7f1cb0070630 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.371+0000 7f1ca6a00640 1 -- 172.21.15.7:0/4138819404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca0009870 con 0x7f1cb0070630 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.371+0000 7f1cafe00640 1 -- 172.21.15.7:0/4138819404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb0152590 con 0x7f1cb0070630 2024-04-25T11:21:13.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.372+0000 7f1cafe00640 1 -- 172.21.15.7:0/4138819404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb00acc50 con 0x7f1cb0070630 2024-04-25T11:21:13.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.373+0000 7f1cafe00640 1 -- 172.21.15.7:0/4138819404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb0050740 con 0x7f1cb0070630 2024-04-25T11:21:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.374+0000 7f1ca6a00640 1 -- 172.21.15.7:0/4138819404 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ca001b070 con 0x7f1cb0070630 2024-04-25T11:21:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.376+0000 7f1ca6a00640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c8c0744b0 0x7f1c8c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.376+0000 7f1ca6a00640 1 -- 172.21.15.7:0/4138819404 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f1ca0095670 con 0x7f1cb0070630 2024-04-25T11:21:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.376+0000 7f1cae000640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c8c0744b0 0x7f1c8c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.377+0000 7f1cae000640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c8c0744b0 0x7f1c8c0769c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1c98034000 tx=0x7f1c9802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.381+0000 7f1ca6a00640 1 -- 172.21.15.7:0/4138819404 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ca005fc70 con 0x7f1cb0070630 2024-04-25T11:21:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.584+0000 7f1cafe00640 1 -- 172.21.15.7:0/4138819404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1cb0070dd0 con 0x7f1cb0070630 2024-04-25T11:21:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.585+0000 7f1ca6a00640 1 -- 172.21.15.7:0/4138819404 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) ==== 100+0+11 (secure 0 0 0) 0x7f1ca005f570 con 0x7f1cb0070630 2024-04-25T11:21:13.582 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:21:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 -- 172.21.15.7:0/4138819404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c8c0744b0 msgr2=0x7f1c8c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c8c0744b0 0x7f1c8c0769c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1c98034000 tx=0x7f1c9802e000 comp rx=0 tx=0).stop 2024-04-25T11:21:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 -- 172.21.15.7:0/4138819404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb0070630 msgr2=0x7f1cb0152050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb0070630 0x7f1cb0152050 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f1ca000d800 tx=0x7f1ca000dcc0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 -- 172.21.15.7:0/4138819404 shutdown_connections 2024-04-25T11:21:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c8c0744b0 0x7f1c8c0769c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cb015ecc0 0x7f1cb01603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cb00d0800 0x7f1cb00ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 --2- 172.21.15.7:0/4138819404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb0070630 0x7f1cb0152050 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 -- 172.21.15.7:0/4138819404 >> 172.21.15.7:0/4138819404 conn(0x7f1cb0074140 msgr2=0x7f1cb00afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:13.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 -- 172.21.15.7:0/4138819404 shutdown_connections 2024-04-25T11:21:13.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.588+0000 7f1cafe00640 1 -- 172.21.15.7:0/4138819404 wait complete. 2024-04-25T11:21:13.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:21:13.599 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:21:13.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:21:13.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.933+0000 7f02dbe00640 1 Processor -- start 2024-04-25T11:21:13.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.933+0000 7f02dbe00640 1 -- start start 2024-04-25T11:21:13.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.936+0000 7f02dbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02d4162330 0x7f02d406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.936+0000 7f02dbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f02d406dac0 0x7f02d4070390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.936+0000 7f02dbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02d40708d0 0x7f02d41681e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.936+0000 7f02dbe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f02d406ded0 con 0x7f02d4162330 2024-04-25T11:21:13.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.936+0000 7f02dbe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f02d4165b20 con 0x7f02d406dac0 2024-04-25T11:21:13.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.936+0000 7f02dbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f02d406f2f0 con 0x7f02d40708d0 2024-04-25T11:21:13.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.936+0000 7f02d9400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02d4162330 0x7f02d406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.936+0000 7f02d9400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02d4162330 0x7f02d406d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37556/0 (socket says 172.21.15.7:37556) 2024-04-25T11:21:13.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.936+0000 7f02d9400640 1 -- 172.21.15.7:0/2013997626 learned_addr learned my addr 172.21.15.7:0/2013997626 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:13.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.937+0000 7f02d9e00640 1 --2- 172.21.15.7:0/2013997626 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02d40708d0 0x7f02d41681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.937+0000 7f02d8a00640 1 --2- 172.21.15.7:0/2013997626 >> v2:172.21.15.87:3300/0 conn(0x7f02d406dac0 0x7f02d4070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.937+0000 7f02d9400640 1 -- 172.21.15.7:0/2013997626 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02d40708d0 msgr2=0x7f02d41681e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:13.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.937+0000 7f02d9400640 1 --2- 172.21.15.7:0/2013997626 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02d40708d0 0x7f02d41681e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.937+0000 7f02d9400640 1 -- 172.21.15.7:0/2013997626 >> v2:172.21.15.87:3300/0 conn(0x7f02d406dac0 msgr2=0x7f02d4070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:13.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.937+0000 7f02d9400640 1 --2- 172.21.15.7:0/2013997626 >> v2:172.21.15.87:3300/0 conn(0x7f02d406dac0 0x7f02d4070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.937+0000 7f02d9400640 1 -- 172.21.15.7:0/2013997626 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d406f5d0 con 0x7f02d4162330 2024-04-25T11:21:13.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.937+0000 7f02d8a00640 1 --2- 172.21.15.7:0/2013997626 >> v2:172.21.15.87:3300/0 conn(0x7f02d406dac0 0x7f02d4070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:13.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.938+0000 7f02d9400640 1 --2- 172.21.15.7:0/2013997626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4162330 0x7f02d406d580 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f02d40b56b0 tx=0x7f02c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496031426f049696 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:13.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.939+0000 7f02d3e00640 1 -- 172.21.15.7:0/2013997626 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8037070 con 0x7f02d4162330 2024-04-25T11:21:13.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.939+0000 7f02d3e00640 1 -- 172.21.15.7:0/2013997626 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c8032440 con 0x7f02d4162330 2024-04-25T11:21:13.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.939+0000 7f02d3e00640 1 -- 172.21.15.7:0/2013997626 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8032810 con 0x7f02d4162330 2024-04-25T11:21:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.939+0000 7f02dbe00640 1 -- 172.21.15.7:0/2013997626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4162330 msgr2=0x7f02d406d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.939+0000 7f02dbe00640 1 --2- 172.21.15.7:0/2013997626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4162330 0x7f02d406d580 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f02d40b56b0 tx=0x7f02c802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.939+0000 7f02dbe00640 1 -- 172.21.15.7:0/2013997626 shutdown_connections 2024-04-25T11:21:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.939+0000 7f02dbe00640 1 --2- 172.21.15.7:0/2013997626 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02d40708d0 0x7f02d41681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.939+0000 7f02dbe00640 1 --2- 172.21.15.7:0/2013997626 >> v2:172.21.15.87:3300/0 conn(0x7f02d406dac0 0x7f02d4070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.940+0000 7f02dbe00640 1 --2- 172.21.15.7:0/2013997626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4162330 0x7f02d406d580 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.940+0000 7f02dbe00640 1 -- 172.21.15.7:0/2013997626 >> 172.21.15.7:0/2013997626 conn(0x7f02d40732b0 msgr2=0x7f02d40736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:13.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.940+0000 7f02dbe00640 1 -- 172.21.15.7:0/2013997626 shutdown_connections 2024-04-25T11:21:13.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.940+0000 7f02dbe00640 1 -- 172.21.15.7:0/2013997626 wait complete. 2024-04-25T11:21:13.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.942+0000 7f02dbe00640 1 Processor -- start 2024-04-25T11:21:13.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.942+0000 7f02dbe00640 1 -- start start 2024-04-25T11:21:13.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.944+0000 7f02dbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d406dac0 0x7f02d4160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.944+0000 7f02dbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02d40708d0 0x7f02d40cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.944+0000 7f02dbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4162330 0x7f02d414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.944+0000 7f02dbe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02d4070ce0 con 0x7f02d4162330 2024-04-25T11:21:13.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.944+0000 7f02dbe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f02d406fb90 con 0x7f02d406dac0 2024-04-25T11:21:13.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.944+0000 7f02dbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f02d4070e60 con 0x7f02d40708d0 2024-04-25T11:21:13.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.944+0000 7f02d9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4162330 0x7f02d414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.944+0000 7f02d9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4162330 0x7f02d414a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37562/0 (socket says 172.21.15.7:37562) 2024-04-25T11:21:13.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.944+0000 7f02d9e00640 1 -- 172.21.15.7:0/1202870226 learned_addr learned my addr 172.21.15.7:0/1202870226 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:13.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.945+0000 7f02d9400640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d406dac0 0x7f02d4160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.945+0000 7f02d8a00640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02d40708d0 0x7f02d40cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.945+0000 7f02d9e00640 1 -- 172.21.15.7:0/1202870226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02d40708d0 msgr2=0x7f02d40cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:13.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.945+0000 7f02d9e00640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02d40708d0 0x7f02d40cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.945+0000 7f02d9e00640 1 -- 172.21.15.7:0/1202870226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d406dac0 msgr2=0x7f02d4160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:13.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.945+0000 7f02d9e00640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d406dac0 0x7f02d4160ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:13.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.945+0000 7f02d9e00640 1 -- 172.21.15.7:0/1202870226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d40d06b0 con 0x7f02d4162330 2024-04-25T11:21:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.945+0000 7f02d9e00640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4162330 0x7f02d414a3d0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f02cc007d40 tx=0x7f02cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.946+0000 7f02d1600640 1 -- 172.21.15.7:0/1202870226 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc014420 con 0x7f02d4162330 2024-04-25T11:21:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.946+0000 7f02d1600640 1 -- 172.21.15.7:0/1202870226 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02cc019070 con 0x7f02d4162330 2024-04-25T11:21:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.946+0000 7f02d1600640 1 -- 172.21.15.7:0/1202870226 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc014820 con 0x7f02d4162330 2024-04-25T11:21:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.946+0000 7f02dbe00640 1 -- 172.21.15.7:0/1202870226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d40d0990 con 0x7f02d4162330 2024-04-25T11:21:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.946+0000 7f02dbe00640 1 -- 172.21.15.7:0/1202870226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d40d0f50 con 0x7f02d4162330 2024-04-25T11:21:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.947+0000 7f02dbe00640 1 -- 172.21.15.7:0/1202870226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d4050740 con 0x7f02d4162330 2024-04-25T11:21:13.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.948+0000 7f02d1600640 1 -- 172.21.15.7:0/1202870226 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f02cc0149c0 con 0x7f02d4162330 2024-04-25T11:21:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.950+0000 7f02d1600640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02b40744b0 0x7f02b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.950+0000 7f02d1600640 1 -- 172.21.15.7:0/1202870226 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f02cc095cd0 con 0x7f02d4162330 2024-04-25T11:21:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.950+0000 7f02d9400640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02b40744b0 0x7f02b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:13.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.951+0000 7f02d9400640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02b40744b0 0x7f02b40769c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f02c8034000 tx=0x7f02c802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:13.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:13.954+0000 7f02d1600640 1 -- 172.21.15.7:0/1202870226 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f02cc05fbe0 con 0x7f02d4162330 2024-04-25T11:21:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.171+0000 7f02dbe00640 1 -- 172.21.15.7:0/1202870226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f02d4074430 con 0x7f02d4162330 2024-04-25T11:21:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.172+0000 7f02d1600640 1 -- 172.21.15.7:0/1202870226 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v130) ==== 80+0+0 (secure 0 0 0) 0x7f02cc063ee0 con 0x7f02d4162330 2024-04-25T11:21:14.169 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.176+0000 7f02dbe00640 1 -- 172.21.15.7:0/1202870226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02b40744b0 msgr2=0x7f02b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.176+0000 7f02dbe00640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02b40744b0 0x7f02b40769c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f02c8034000 tx=0x7f02c802e000 comp rx=0 tx=0).stop 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.176+0000 7f02dbe00640 1 -- 172.21.15.7:0/1202870226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4162330 msgr2=0x7f02d414a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.176+0000 7f02dbe00640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4162330 0x7f02d414a3d0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f02cc007d40 tx=0x7f02cc00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.176+0000 7f02dbe00640 1 -- 172.21.15.7:0/1202870226 shutdown_connections 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.176+0000 7f02dbe00640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f02b40744b0 0x7f02b40769c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.176+0000 7f02dbe00640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02d4162330 0x7f02d414a3d0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.176+0000 7f02dbe00640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02d40708d0 0x7f02d40cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.176+0000 7f02dbe00640 1 --2- 172.21.15.7:0/1202870226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d406dac0 0x7f02d4160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.176+0000 7f02dbe00640 1 -- 172.21.15.7:0/1202870226 >> 172.21.15.7:0/1202870226 conn(0x7f02d40732b0 msgr2=0x7f02d406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.176+0000 7f02dbe00640 1 -- 172.21.15.7:0/1202870226 shutdown_connections 2024-04-25T11:21:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.177+0000 7f02dbe00640 1 -- 172.21.15.7:0/1202870226 wait complete. 2024-04-25T11:21:14.200 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:21:14.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:14.201 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T11:21:14.304 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:21:14.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:14.305 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:21:14.357 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:21:14.357 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:21:14.358 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:14.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:21:14.415 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:21:14.416 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:21:14.416 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:14.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:21:14.473 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:21:14.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.771+0000 7f1f1d000640 1 Processor -- start 2024-04-25T11:21:14.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.771+0000 7f1f1d000640 1 -- start start 2024-04-25T11:21:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.773+0000 7f1f1d000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1f1815ef90 0x7f1f1809f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.773+0000 7f1f1d000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f1815bc40 0x7f1f180aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.773+0000 7f1f1d000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f1815f4e0 con 0x7f1f180af370 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.773+0000 7f1f1d000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1f18071190 con 0x7f1f1815bc40 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.773+0000 7f1f1d000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1f180af780 con 0x7f1f1815ef90 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.774+0000 7f1f15400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f1815bc40 0x7f1f180aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.774+0000 7f1f15400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f1815bc40 0x7f1f180aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37572/0 (socket says 172.21.15.7:37572) 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.774+0000 7f1f15400640 1 -- 172.21.15.7:0/650671595 learned_addr learned my addr 172.21.15.7:0/650671595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.774+0000 7f1f15e00640 1 --2- 172.21.15.7:0/650671595 >> v2:172.21.15.87:3300/0 conn(0x7f1f1815ef90 0x7f1f1809f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.774+0000 7f1f15400640 1 -- 172.21.15.7:0/650671595 >> v2:172.21.15.87:3300/0 conn(0x7f1f1815ef90 msgr2=0x7f1f1809f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.774+0000 7f1f15400640 1 --2- 172.21.15.7:0/650671595 >> v2:172.21.15.87:3300/0 conn(0x7f1f1815ef90 0x7f1f1809f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.774+0000 7f1f15400640 1 -- 172.21.15.7:0/650671595 >> v1:172.21.15.87:6789/0 conn(0x7f1f180af370 legacy=0x7f1f18161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.774+0000 7f1f15400640 1 -- 172.21.15.7:0/650671595 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f180b0560 con 0x7f1f1815bc40 2024-04-25T11:21:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.776+0000 7f1f15400640 1 --2- 172.21.15.7:0/650671595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1815bc40 0x7f1f180aee30 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f1f180b4740 tx=0x7f1f0002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f81c6950459ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.776+0000 7f1f14a00640 1 -- 172.21.15.7:0/650671595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00037070 con 0x7f1f1815bc40 2024-04-25T11:21:14.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.776+0000 7f1f14a00640 1 -- 172.21.15.7:0/650671595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f00032440 con 0x7f1f1815bc40 2024-04-25T11:21:14.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.776+0000 7f1f14a00640 1 -- 172.21.15.7:0/650671595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00032810 con 0x7f1f1815bc40 2024-04-25T11:21:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.777+0000 7f1f1d000640 1 -- 172.21.15.7:0/650671595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1815bc40 msgr2=0x7f1f180aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.777+0000 7f1f1d000640 1 --2- 172.21.15.7:0/650671595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1815bc40 0x7f1f180aee30 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f1f180b4740 tx=0x7f1f0002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:21:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.777+0000 7f1f1d000640 1 -- 172.21.15.7:0/650671595 shutdown_connections 2024-04-25T11:21:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.777+0000 7f1f1d000640 1 --2- 172.21.15.7:0/650671595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f1815bc40 0x7f1f180aee30 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.777+0000 7f1f1d000640 1 --2- 172.21.15.7:0/650671595 >> v2:172.21.15.87:3300/0 conn(0x7f1f1815ef90 0x7f1f1809f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.777+0000 7f1f1d000640 1 -- 172.21.15.7:0/650671595 >> 172.21.15.7:0/650671595 conn(0x7f1f18074200 msgr2=0x7f1f1806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.777+0000 7f1f1d000640 1 -- 172.21.15.7:0/650671595 shutdown_connections 2024-04-25T11:21:14.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.777+0000 7f1f1d000640 1 -- 172.21.15.7:0/650671595 wait complete. 2024-04-25T11:21:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.780+0000 7f1f1d000640 1 Processor -- start 2024-04-25T11:21:14.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.780+0000 7f1f1d000640 1 -- start start 2024-04-25T11:21:14.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.782+0000 7f1f1d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f180af370 0x7f1f180b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:14.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.782+0000 7f1f1d000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f1815bc40 0x7f1f181f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:14.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.782+0000 7f1f1d000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f1815ef90 0x7f1f1814af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:14.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.782+0000 7f1f1d000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f180b0e00 con 0x7f1f180af370 2024-04-25T11:21:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.782+0000 7f1f1d000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1f180b0b20 con 0x7f1f1815ef90 2024-04-25T11:21:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.782+0000 7f1f1d000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1f180b1320 con 0x7f1f1815bc40 2024-04-25T11:21:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.782+0000 7f1f15e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f180af370 0x7f1f180b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.782+0000 7f1f15e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f180af370 0x7f1f180b62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37584/0 (socket says 172.21.15.7:37584) 2024-04-25T11:21:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.783+0000 7f1f15e00640 1 -- 172.21.15.7:0/3017439067 learned_addr learned my addr 172.21.15.7:0/3017439067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.783+0000 7f1f15400640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f1815bc40 0x7f1f181f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.783+0000 7f1f16800640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f1815ef90 0x7f1f1814af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.783+0000 7f1f15e00640 1 -- 172.21.15.7:0/3017439067 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f1815bc40 msgr2=0x7f1f181f98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.783+0000 7f1f15e00640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f1815bc40 0x7f1f181f98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.783+0000 7f1f15e00640 1 -- 172.21.15.7:0/3017439067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f1815ef90 msgr2=0x7f1f1814af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.783+0000 7f1f15e00640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f1815ef90 0x7f1f1814af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.783+0000 7f1f15e00640 1 -- 172.21.15.7:0/3017439067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1814b4b0 con 0x7f1f180af370 2024-04-25T11:21:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.783+0000 7f1f15e00640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f180af370 0x7f1f180b62c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f1f08007e40 tx=0x7f1f0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.784+0000 7f1f0e000640 1 -- 172.21.15.7:0/3017439067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08014620 con 0x7f1f180af370 2024-04-25T11:21:14.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.784+0000 7f1f0e000640 1 -- 172.21.15.7:0/3017439067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f08019070 con 0x7f1f180af370 2024-04-25T11:21:14.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.784+0000 7f1f0e000640 1 -- 172.21.15.7:0/3017439067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08014a00 con 0x7f1f180af370 2024-04-25T11:21:14.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.784+0000 7f1f1d000640 1 -- 172.21.15.7:0/3017439067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f18157080 con 0x7f1f180af370 2024-04-25T11:21:14.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.784+0000 7f1f1d000640 1 -- 172.21.15.7:0/3017439067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f181fa3d0 con 0x7f1f180af370 2024-04-25T11:21:14.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.785+0000 7f1f1d000640 1 -- 172.21.15.7:0/3017439067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1806a100 con 0x7f1f180af370 2024-04-25T11:21:14.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.786+0000 7f1f0e000640 1 -- 172.21.15.7:0/3017439067 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1f08014ba0 con 0x7f1f180af370 2024-04-25T11:21:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.787+0000 7f1f0e000640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ef40744b0 0x7f1ef40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.787+0000 7f1f0e000640 1 -- 172.21.15.7:0/3017439067 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f1f08095cc0 con 0x7f1f180af370 2024-04-25T11:21:14.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.788+0000 7f1f15400640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ef40744b0 0x7f1ef40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:14.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.789+0000 7f1f15400640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ef40744b0 0x7f1ef40769c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1f00034000 tx=0x7f1f0002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:14.792+0000 7f1f0e000640 1 -- 172.21.15.7:0/3017439067 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1f0809c080 con 0x7f1f180af370 2024-04-25T11:21:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.047+0000 7f1f1d000640 1 -- 172.21.15.7:0/3017439067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f1809f8c0 con 0x7f1f180af370 2024-04-25T11:21:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.049+0000 7f1f0e000640 1 -- 172.21.15.7:0/3017439067 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5174 (secure 0 0 0) 0x7f1f08060220 con 0x7f1f180af370 2024-04-25T11:21:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:15.047 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":92,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2288158899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2288158899},{"type":"v1","addr":"172.21.15.7:6837","nonce":2288158899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3545919245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3545919245},{"type":"v1","addr":"172.21.15.87:6835","nonce":3545919245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8478,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/951407928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":951407928},{"type":"v1","addr":"172.21.15.7:6836","nonce":951407928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:42.908797+0000","modified":"2024-04-25T11:21:07.439019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":9,"addr":"172.21.15.87:6833/1243096872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1243096872},{"type":"v1","addr":"172.21.15.87:6833","nonce":1243096872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8484,"qdb_cluster":[8484]},"id":9}]} 2024-04-25T11:21:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 92 2024-04-25T11:21:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.051+0000 7f1f1d000640 1 -- 172.21.15.7:0/3017439067 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ef40744b0 msgr2=0x7f1ef40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.051+0000 7f1f1d000640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ef40744b0 0x7f1ef40769c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1f00034000 tx=0x7f1f0002e000 comp rx=0 tx=0).stop 2024-04-25T11:21:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.051+0000 7f1f1d000640 1 -- 172.21.15.7:0/3017439067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f180af370 msgr2=0x7f1f180b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.051+0000 7f1f1d000640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f180af370 0x7f1f180b62c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f1f08007e40 tx=0x7f1f0800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.051+0000 7f1f1d000640 1 -- 172.21.15.7:0/3017439067 shutdown_connections 2024-04-25T11:21:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.051+0000 7f1f1d000640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ef40744b0 0x7f1ef40769c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.051+0000 7f1f1d000640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f1815ef90 0x7f1f1814af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.051+0000 7f1f1d000640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f1815bc40 0x7f1f181f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.052+0000 7f1f1d000640 1 --2- 172.21.15.7:0/3017439067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f180af370 0x7f1f180b62c0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.052+0000 7f1f1d000640 1 -- 172.21.15.7:0/3017439067 >> 172.21.15.7:0/3017439067 conn(0x7f1f18074200 msgr2=0x7f1f180684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.052+0000 7f1f1d000640 1 -- 172.21.15.7:0/3017439067 shutdown_connections 2024-04-25T11:21:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.052+0000 7f1f1d000640 1 -- 172.21.15.7:0/3017439067 wait complete. 2024-04-25T11:21:15.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:21:15.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.361+0000 7ff384e00640 1 Processor -- start 2024-04-25T11:21:15.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.361+0000 7ff384e00640 1 -- start start 2024-04-25T11:21:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.363+0000 7ff384e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff38006dc70 0x7ff3800d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.363+0000 7ff384e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 0x7ff3800af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.363+0000 7ff384e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3800b2820 con 0x7ff3800d24f0 2024-04-25T11:21:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.363+0000 7ff384e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff3800b2260 con 0x7ff38006dc70 2024-04-25T11:21:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.363+0000 7ff384e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff3800b2540 con 0x7ff3800d2900 2024-04-25T11:21:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.364+0000 7ff37f400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff38006dc70 0x7ff3800d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.364+0000 7ff37fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 0x7ff3800af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.364+0000 7ff37fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 0x7ff3800af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:36466/0 (socket says 172.21.15.7:36466) 2024-04-25T11:21:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.364+0000 7ff37fe00640 1 -- 172.21.15.7:0/101889037 learned_addr learned my addr 172.21.15.7:0/101889037 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.364+0000 7ff37f400640 1 --2- 172.21.15.7:0/101889037 >> v2:172.21.15.87:3300/0 conn(0x7ff38006dc70 0x7ff3800d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:36146/0 (socket says 172.21.15.7:36146) 2024-04-25T11:21:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.365+0000 7ff37fe00640 1 -- 172.21.15.7:0/101889037 >> v2:172.21.15.87:3300/0 conn(0x7ff38006dc70 msgr2=0x7ff3800d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.365+0000 7ff37fe00640 1 --2- 172.21.15.7:0/101889037 >> v2:172.21.15.87:3300/0 conn(0x7ff38006dc70 0x7ff3800d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.365+0000 7ff37fe00640 1 -- 172.21.15.7:0/101889037 >> v1:172.21.15.87:6789/0 conn(0x7ff3800d24f0 legacy=0x7ff3800aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.365+0000 7ff37fe00640 1 -- 172.21.15.7:0/101889037 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38006d120 con 0x7ff3800d2900 2024-04-25T11:21:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.366+0000 7ff37fe00640 1 --2- 172.21.15.7:0/101889037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 0x7ff3800af6b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff378009e50 tx=0x7ff37802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e62c5cc73c5857e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.366+0000 7ff37e000640 1 -- 172.21.15.7:0/101889037 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378036070 con 0x7ff3800d2900 2024-04-25T11:21:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.367+0000 7ff37e000640 1 -- 172.21.15.7:0/101889037 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff378031600 con 0x7ff3800d2900 2024-04-25T11:21:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.367+0000 7ff37e000640 1 -- 172.21.15.7:0/101889037 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3780319d0 con 0x7ff3800d2900 2024-04-25T11:21:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.367+0000 7ff384e00640 1 -- 172.21.15.7:0/101889037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 msgr2=0x7ff3800af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.367+0000 7ff384e00640 1 --2- 172.21.15.7:0/101889037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 0x7ff3800af6b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff378009e50 tx=0x7ff37802a710 comp rx=0 tx=0).stop 2024-04-25T11:21:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.367+0000 7ff384e00640 1 -- 172.21.15.7:0/101889037 shutdown_connections 2024-04-25T11:21:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.367+0000 7ff384e00640 1 --2- 172.21.15.7:0/101889037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 0x7ff3800af6b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.367+0000 7ff384e00640 1 --2- 172.21.15.7:0/101889037 >> v2:172.21.15.87:3300/0 conn(0x7ff38006dc70 0x7ff3800d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.367+0000 7ff384e00640 1 -- 172.21.15.7:0/101889037 >> 172.21.15.7:0/101889037 conn(0x7ff380073b60 msgr2=0x7ff3800b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.367+0000 7ff384e00640 1 -- 172.21.15.7:0/101889037 shutdown_connections 2024-04-25T11:21:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.367+0000 7ff384e00640 1 -- 172.21.15.7:0/101889037 wait complete. 2024-04-25T11:21:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.370+0000 7ff384e00640 1 Processor -- start 2024-04-25T11:21:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.370+0000 7ff384e00640 1 -- start start 2024-04-25T11:21:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.372+0000 7ff384e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff38006dc70 0x7ff380151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.372+0000 7ff384e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3800d24f0 0x7ff3801518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.372+0000 7ff384e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 0x7ff380070a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.372+0000 7ff384e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff3800b1ea0 con 0x7ff3800d24f0 2024-04-25T11:21:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.372+0000 7ff384e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff38009b990 con 0x7ff38006dc70 2024-04-25T11:21:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.372+0000 7ff384e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff38009b540 con 0x7ff3800d2900 2024-04-25T11:21:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.372+0000 7ff37ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3800d24f0 0x7ff3801518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.372+0000 7ff37ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3800d24f0 0x7ff3801518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37590/0 (socket says 172.21.15.7:37590) 2024-04-25T11:21:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.372+0000 7ff37ea00640 1 -- 172.21.15.7:0/2403712853 learned_addr learned my addr 172.21.15.7:0/2403712853 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.372+0000 7ff37f400640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff38006dc70 0x7ff380151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff37ea00640 1 -- 172.21.15.7:0/2403712853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 msgr2=0x7ff380070a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff37fe00640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 0x7ff380070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff37ea00640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 0x7ff380070a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff37ea00640 1 -- 172.21.15.7:0/2403712853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff38006dc70 msgr2=0x7ff380151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff37ea00640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff38006dc70 0x7ff380151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff37ea00640 1 -- 172.21.15.7:0/2403712853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff380156b40 con 0x7ff3800d24f0 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff37f400640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff38006dc70 0x7ff380151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff37ea00640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3800d24f0 0x7ff3801518d0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c340 tx=0x7ff36c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff377400640 1 -- 172.21.15.7:0/2403712853 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c017070 con 0x7ff3800d24f0 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff377400640 1 -- 172.21.15.7:0/2403712853 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36c0052d0 con 0x7ff3800d24f0 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff377400640 1 -- 172.21.15.7:0/2403712853 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c003850 con 0x7ff3800d24f0 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.373+0000 7ff384e00640 1 -- 172.21.15.7:0/2403712853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff380154c10 con 0x7ff3800d24f0 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.374+0000 7ff384e00640 1 -- 172.21.15.7:0/2403712853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff380154930 con 0x7ff3800d24f0 2024-04-25T11:21:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.375+0000 7ff384e00640 1 -- 172.21.15.7:0/2403712853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3801543c0 con 0x7ff3800d24f0 2024-04-25T11:21:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.376+0000 7ff377400640 1 -- 172.21.15.7:0/2403712853 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff36c01a070 con 0x7ff3800d24f0 2024-04-25T11:21:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.377+0000 7ff377400640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3600744b0 0x7ff3600769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.377+0000 7ff377400640 1 -- 172.21.15.7:0/2403712853 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7ff36c0946d0 con 0x7ff3800d24f0 2024-04-25T11:21:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.377+0000 7ff37f400640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3600744b0 0x7ff3600769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.378+0000 7ff37f400640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3600744b0 0x7ff3600769c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff370009a20 tx=0x7ff370008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.382+0000 7ff377400640 1 -- 172.21.15.7:0/2403712853 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff36c05ecc0 con 0x7ff3800d24f0 2024-04-25T11:21:15.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.593+0000 7ff384e00640 1 -- 172.21.15.7:0/2403712853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff380050740 con 0x7ff3800d24f0 2024-04-25T11:21:15.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.596+0000 7ff377400640 1 -- 172.21.15.7:0/2403712853 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16804 (secure 0 0 0) 0x7ff36c05e5c0 con 0x7ff3800d24f0 2024-04-25T11:21:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:15.595 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":130,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:20:44.396251+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:41.047420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:41.908907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:15.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.598+0000 7ff384e00640 1 -- 172.21.15.7:0/2403712853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3600744b0 msgr2=0x7ff3600769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.598+0000 7ff384e00640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3600744b0 0x7ff3600769c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff370009a20 tx=0x7ff370008040 comp rx=0 tx=0).stop 2024-04-25T11:21:15.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.598+0000 7ff384e00640 1 -- 172.21.15.7:0/2403712853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3800d24f0 msgr2=0x7ff3801518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.598+0000 7ff384e00640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3800d24f0 0x7ff3801518d0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c340 tx=0x7ff36c00c800 comp rx=0 tx=0).stop 2024-04-25T11:21:15.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.599+0000 7ff384e00640 1 -- 172.21.15.7:0/2403712853 shutdown_connections 2024-04-25T11:21:15.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.599+0000 7ff384e00640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3600744b0 0x7ff3600769c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.599+0000 7ff384e00640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3800d2900 0x7ff380070a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.599+0000 7ff384e00640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3800d24f0 0x7ff3801518d0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.599+0000 7ff384e00640 1 --2- 172.21.15.7:0/2403712853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff38006dc70 0x7ff380151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.599+0000 7ff384e00640 1 -- 172.21.15.7:0/2403712853 >> 172.21.15.7:0/2403712853 conn(0x7ff380073b60 msgr2=0x7ff3800b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:15.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.599+0000 7ff384e00640 1 -- 172.21.15.7:0/2403712853 shutdown_connections 2024-04-25T11:21:15.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.599+0000 7ff384e00640 1 -- 172.21.15.7:0/2403712853 wait complete. 2024-04-25T11:21:15.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:20:41.047420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:20:41.908907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '130', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:21:15.624 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:21:15.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.918+0000 7fd977600640 1 Processor -- start 2024-04-25T11:21:15.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.918+0000 7fd977600640 1 -- start start 2024-04-25T11:21:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.921+0000 7fd977600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd97015dfb0 0x7fd9700b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.921+0000 7fd977600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9700d0b40 0x7fd97006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.921+0000 7fd977600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd97015e3c0 con 0x7fd9700d0f50 2024-04-25T11:21:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.921+0000 7fd977600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd9700b09b0 con 0x7fd9700d0b40 2024-04-25T11:21:15.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.921+0000 7fd977600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd9700cf800 con 0x7fd97015dfb0 2024-04-25T11:21:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.921+0000 7fd975600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd9700d0f50 0x7fd970161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39944/0 (socket says 172.21.15.7:39944) 2024-04-25T11:21:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.921+0000 7fd975600640 1 -- 172.21.15.7:0/3680220524 learned_addr learned my addr 172.21.15.7:0/3680220524 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.921+0000 7fd96fe00640 1 --2- 172.21.15.7:0/3680220524 >> v2:172.21.15.7:3300/0 conn(0x7fd9700d0b40 0x7fd97006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.921+0000 7fd974c00640 1 --2- 172.21.15.7:0/3680220524 >> v2:172.21.15.87:3300/0 conn(0x7fd97015dfb0 0x7fd9700b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.922+0000 7fd96f400640 1 -- 172.21.15.7:0/3680220524 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737848601 0 0) 0x7fd97015e3c0 con 0x7fd9700d0f50 2024-04-25T11:21:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.922+0000 7fd96f400640 1 -- 172.21.15.7:0/3680220524 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd958003520 con 0x7fd9700d0f50 2024-04-25T11:21:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.922+0000 7fd96fe00640 1 -- 172.21.15.7:0/3680220524 >> v2:172.21.15.87:3300/0 conn(0x7fd97015dfb0 msgr2=0x7fd9700b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.922+0000 7fd96fe00640 1 --2- 172.21.15.7:0/3680220524 >> v2:172.21.15.87:3300/0 conn(0x7fd97015dfb0 0x7fd9700b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.922+0000 7fd96fe00640 1 -- 172.21.15.7:0/3680220524 >> v1:172.21.15.7:6789/0 conn(0x7fd9700d0f50 legacy=0x7fd970161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.922+0000 7fd96fe00640 1 -- 172.21.15.7:0/3680220524 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9700cfaa0 con 0x7fd9700d0b40 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.923+0000 7fd96fe00640 1 --2- 172.21.15.7:0/3680220524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9700d0b40 0x7fd97006f3a0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd970072b10 tx=0x7fd95c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f78eb2fd9698f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.924+0000 7fd96f400640 1 -- 172.21.15.7:0/3680220524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c037070 con 0x7fd9700d0b40 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.924+0000 7fd96f400640 1 -- 172.21.15.7:0/3680220524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd95c032440 con 0x7fd9700d0b40 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.924+0000 7fd96f400640 1 -- 172.21.15.7:0/3680220524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c02ae60 con 0x7fd9700d0b40 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.924+0000 7fd977600640 1 -- 172.21.15.7:0/3680220524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9700d0b40 msgr2=0x7fd97006f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.924+0000 7fd977600640 1 --2- 172.21.15.7:0/3680220524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9700d0b40 0x7fd97006f3a0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd970072b10 tx=0x7fd95c02a110 comp rx=0 tx=0).stop 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.925+0000 7fd977600640 1 -- 172.21.15.7:0/3680220524 shutdown_connections 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.925+0000 7fd977600640 1 --2- 172.21.15.7:0/3680220524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9700d0b40 0x7fd97006f3a0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.925+0000 7fd977600640 1 --2- 172.21.15.7:0/3680220524 >> v2:172.21.15.87:3300/0 conn(0x7fd97015dfb0 0x7fd9700b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.925+0000 7fd977600640 1 -- 172.21.15.7:0/3680220524 >> 172.21.15.7:0/3680220524 conn(0x7fd9700742c0 msgr2=0x7fd97006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.925+0000 7fd977600640 1 -- 172.21.15.7:0/3680220524 shutdown_connections 2024-04-25T11:21:15.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.925+0000 7fd977600640 1 -- 172.21.15.7:0/3680220524 wait complete. 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.927+0000 7fd977600640 1 Processor -- start 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.927+0000 7fd977600640 1 -- start start 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.929+0000 7fd977600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9700d0b40 0x7fd97015b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.929+0000 7fd977600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9700d0f50 0x7fd97014c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.929+0000 7fd977600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd97015dfb0 0x7fd9701582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.929+0000 7fd977600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd9700d0340 con 0x7fd9700d0f50 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.929+0000 7fd977600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd9700d0060 con 0x7fd9700d0b40 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.929+0000 7fd977600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9700d0620 con 0x7fd97015dfb0 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.929+0000 7fd96fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9700d0f50 0x7fd97014c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.929+0000 7fd96fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9700d0f50 0x7fd97014c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37606/0 (socket says 172.21.15.7:37606) 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.929+0000 7fd96fe00640 1 -- 172.21.15.7:0/2393950379 learned_addr learned my addr 172.21.15.7:0/2393950379 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.929+0000 7fd974c00640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9700d0b40 0x7fd97015b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.930+0000 7fd975600640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd97015dfb0 0x7fd9701582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.930+0000 7fd96fe00640 1 -- 172.21.15.7:0/2393950379 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd97015dfb0 msgr2=0x7fd9701582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.930+0000 7fd96fe00640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd97015dfb0 0x7fd9701582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.930+0000 7fd96fe00640 1 -- 172.21.15.7:0/2393950379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9700d0b40 msgr2=0x7fd97015b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.930+0000 7fd96fe00640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9700d0b40 0x7fd97015b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.930+0000 7fd96fe00640 1 -- 172.21.15.7:0/2393950379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97015bb70 con 0x7fd9700d0f50 2024-04-25T11:21:15.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.930+0000 7fd975600640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd97015dfb0 0x7fd9701582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.930+0000 7fd96fe00640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9700d0f50 0x7fd97014c0c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd95c02a150 tx=0x7fd95c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.931+0000 7fd96cc00640 1 -- 172.21.15.7:0/2393950379 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c037040 con 0x7fd9700d0f50 2024-04-25T11:21:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.931+0000 7fd96cc00640 1 -- 172.21.15.7:0/2393950379 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd95c042070 con 0x7fd9700d0f50 2024-04-25T11:21:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.931+0000 7fd96cc00640 1 -- 172.21.15.7:0/2393950379 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c032810 con 0x7fd9700d0f50 2024-04-25T11:21:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.931+0000 7fd977600640 1 -- 172.21.15.7:0/2393950379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97015c080 con 0x7fd9700d0f50 2024-04-25T11:21:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.931+0000 7fd977600640 1 -- 172.21.15.7:0/2393950379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97015c640 con 0x7fd9700d0f50 2024-04-25T11:21:15.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.932+0000 7fd977600640 1 -- 172.21.15.7:0/2393950379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd970050740 con 0x7fd9700d0f50 2024-04-25T11:21:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.933+0000 7fd96cc00640 1 -- 172.21.15.7:0/2393950379 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd95c0329b0 con 0x7fd9700d0f50 2024-04-25T11:21:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.935+0000 7fd96cc00640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9500744b0 0x7fd9500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.935+0000 7fd96cc00640 1 -- 172.21.15.7:0/2393950379 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7fd95c0b5d40 con 0x7fd9700d0f50 2024-04-25T11:21:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.935+0000 7fd974c00640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9500744b0 0x7fd9500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:15.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.936+0000 7fd974c00640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9500744b0 0x7fd9500769c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd970072df0 tx=0x7fd964008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:15.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:15.940+0000 7fd96cc00640 1 -- 172.21.15.7:0/2393950379 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd95c0802a0 con 0x7fd9700d0f50 2024-04-25T11:21:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.136+0000 7fd977600640 1 -- 172.21.15.7:0/2393950379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd970070fb0 con 0x7fd9700d0f50 2024-04-25T11:21:16.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.137+0000 7fd96cc00640 1 -- 172.21.15.7:0/2393950379 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) ==== 93+0+10 (secure 0 0 0) 0x7fd95c07fba0 con 0x7fd9700d0f50 2024-04-25T11:21:16.134 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:21:16.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.139+0000 7fd977600640 1 -- 172.21.15.7:0/2393950379 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9500744b0 msgr2=0x7fd9500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.139+0000 7fd977600640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9500744b0 0x7fd9500769c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd970072df0 tx=0x7fd964008040 comp rx=0 tx=0).stop 2024-04-25T11:21:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.139+0000 7fd977600640 1 -- 172.21.15.7:0/2393950379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9700d0f50 msgr2=0x7fd97014c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.139+0000 7fd977600640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9700d0f50 0x7fd97014c0c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd95c02a150 tx=0x7fd95c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.139+0000 7fd977600640 1 -- 172.21.15.7:0/2393950379 shutdown_connections 2024-04-25T11:21:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.139+0000 7fd977600640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd9500744b0 0x7fd9500769c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.139+0000 7fd977600640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd97015dfb0 0x7fd9701582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.139+0000 7fd977600640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9700d0f50 0x7fd97014c0c0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.139+0000 7fd977600640 1 --2- 172.21.15.7:0/2393950379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9700d0b40 0x7fd97015b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.139+0000 7fd977600640 1 -- 172.21.15.7:0/2393950379 >> 172.21.15.7:0/2393950379 conn(0x7fd9700742c0 msgr2=0x7fd9700ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.140+0000 7fd977600640 1 -- 172.21.15.7:0/2393950379 shutdown_connections 2024-04-25T11:21:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.140+0000 7fd977600640 1 -- 172.21.15.7:0/2393950379 wait complete. 2024-04-25T11:21:16.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:21:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.444+0000 7efe18c00640 1 Processor -- start 2024-04-25T11:21:16.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.445+0000 7efe18c00640 1 -- start start 2024-04-25T11:21:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.447+0000 7efe18c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe140d0ee0 0x7efe140affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.447+0000 7efe18c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efe140b5020 0x7efe14160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.448+0000 7efe18c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1406f5b0 con 0x7efe1415aaa0 2024-04-25T11:21:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.448+0000 7efe18c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efe140af290 con 0x7efe140b5020 2024-04-25T11:21:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.448+0000 7efe18c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe140d14a0 con 0x7efe140d0ee0 2024-04-25T11:21:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.448+0000 7efe11e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efe1415aaa0 0x7efe140b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39960/0 (socket says 172.21.15.7:39960) 2024-04-25T11:21:16.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.448+0000 7efe11e00640 1 -- 172.21.15.7:0/316697583 learned_addr learned my addr 172.21.15.7:0/316697583 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:16.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.448+0000 7efe11400640 1 --2- 172.21.15.7:0/316697583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe140d0ee0 0x7efe140affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.448+0000 7efe12800640 1 --2- 172.21.15.7:0/316697583 >> v2:172.21.15.87:3300/0 conn(0x7efe140b5020 0x7efe14160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.449+0000 7efe10a00640 1 -- 172.21.15.7:0/316697583 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2359328066 0 0) 0x7efe1406f5b0 con 0x7efe1415aaa0 2024-04-25T11:21:16.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.449+0000 7efe10a00640 1 -- 172.21.15.7:0/316697583 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdf8003520 con 0x7efe1415aaa0 2024-04-25T11:21:16.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.449+0000 7efe11400640 1 -- 172.21.15.7:0/316697583 >> v2:172.21.15.87:3300/0 conn(0x7efe140b5020 msgr2=0x7efe14160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:16.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.449+0000 7efe11400640 1 --2- 172.21.15.7:0/316697583 >> v2:172.21.15.87:3300/0 conn(0x7efe140b5020 0x7efe14160990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.449+0000 7efe11400640 1 -- 172.21.15.7:0/316697583 >> v1:172.21.15.7:6789/0 conn(0x7efe1415aaa0 legacy=0x7efe140b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:16.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.449+0000 7efe11400640 1 -- 172.21.15.7:0/316697583 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe140af4e0 con 0x7efe140d0ee0 2024-04-25T11:21:16.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.451+0000 7efe11400640 1 --2- 172.21.15.7:0/316697583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe140d0ee0 0x7efe140affc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0097a0 tx=0x7efdfc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d4ed473cf822bd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:16.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.452+0000 7efe10a00640 1 -- 172.21.15.7:0/316697583 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc037070 con 0x7efe140d0ee0 2024-04-25T11:21:16.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.452+0000 7efe10a00640 1 -- 172.21.15.7:0/316697583 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdfc032440 con 0x7efe140d0ee0 2024-04-25T11:21:16.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.452+0000 7efe10a00640 1 -- 172.21.15.7:0/316697583 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc032810 con 0x7efe140d0ee0 2024-04-25T11:21:16.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.453+0000 7efe18c00640 1 -- 172.21.15.7:0/316697583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe140d0ee0 msgr2=0x7efe140affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:16.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.453+0000 7efe18c00640 1 --2- 172.21.15.7:0/316697583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe140d0ee0 0x7efe140affc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0097a0 tx=0x7efdfc02a180 comp rx=0 tx=0).stop 2024-04-25T11:21:16.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.453+0000 7efe18c00640 1 -- 172.21.15.7:0/316697583 shutdown_connections 2024-04-25T11:21:16.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.453+0000 7efe18c00640 1 --2- 172.21.15.7:0/316697583 >> v2:172.21.15.87:3300/0 conn(0x7efe140b5020 0x7efe14160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.453+0000 7efe18c00640 1 --2- 172.21.15.7:0/316697583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe140d0ee0 0x7efe140affc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.453+0000 7efe18c00640 1 -- 172.21.15.7:0/316697583 >> 172.21.15.7:0/316697583 conn(0x7efe140742c0 msgr2=0x7efe1406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.453+0000 7efe18c00640 1 -- 172.21.15.7:0/316697583 shutdown_connections 2024-04-25T11:21:16.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.453+0000 7efe18c00640 1 -- 172.21.15.7:0/316697583 wait complete. 2024-04-25T11:21:16.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.456+0000 7efe18c00640 1 Processor -- start 2024-04-25T11:21:16.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.456+0000 7efe18c00640 1 -- start start 2024-04-25T11:21:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.457+0000 7efe18c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe140b5020 0x7efe14159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.457+0000 7efe18c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe140d0ee0 0x7efe14143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe18c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe1415aaa0 0x7efe14143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe18c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe140b5430 con 0x7efe140b5020 2024-04-25T11:21:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe18c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efe140afaa0 con 0x7efe1415aaa0 2024-04-25T11:21:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe18c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe140b55b0 con 0x7efe140d0ee0 2024-04-25T11:21:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe11e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe140b5020 0x7efe14159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe11e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe140b5020 0x7efe14159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37616/0 (socket says 172.21.15.7:37616) 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe11e00640 1 -- 172.21.15.7:0/1422984455 learned_addr learned my addr 172.21.15.7:0/1422984455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe11400640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe140d0ee0 0x7efe14143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe11e00640 1 -- 172.21.15.7:0/1422984455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe140d0ee0 msgr2=0x7efe14143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe11e00640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe140d0ee0 0x7efe14143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe11e00640 1 -- 172.21.15.7:0/1422984455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe1415aaa0 msgr2=0x7efe14143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe12800640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe1415aaa0 0x7efe14143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe11e00640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe1415aaa0 0x7efe14143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.458+0000 7efe11e00640 1 -- 172.21.15.7:0/1422984455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe14158bb0 con 0x7efe140b5020 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.459+0000 7efe11e00640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe140b5020 0x7efe14159c50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7efe0400caa0 tx=0x7efe0400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.459+0000 7efe0a000640 1 -- 172.21.15.7:0/1422984455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04017070 con 0x7efe140b5020 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.459+0000 7efe0a000640 1 -- 172.21.15.7:0/1422984455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0400ac00 con 0x7efe140b5020 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.459+0000 7efe0a000640 1 -- 172.21.15.7:0/1422984455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04012400 con 0x7efe140b5020 2024-04-25T11:21:16.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.459+0000 7efe18c00640 1 -- 172.21.15.7:0/1422984455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe14158e90 con 0x7efe140b5020 2024-04-25T11:21:16.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.459+0000 7efe18c00640 1 -- 172.21.15.7:0/1422984455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe14159450 con 0x7efe140b5020 2024-04-25T11:21:16.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.460+0000 7efe18c00640 1 -- 172.21.15.7:0/1422984455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe14050740 con 0x7efe140b5020 2024-04-25T11:21:16.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.462+0000 7efe0a000640 1 -- 172.21.15.7:0/1422984455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efe0401a070 con 0x7efe140b5020 2024-04-25T11:21:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.463+0000 7efe0a000640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdf00744b0 0x7efdf00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.463+0000 7efe0a000640 1 -- 172.21.15.7:0/1422984455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7efe040952c0 con 0x7efe140b5020 2024-04-25T11:21:16.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.463+0000 7efe11400640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdf00744b0 0x7efdf00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.464+0000 7efe11400640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdf00744b0 0x7efdf00769c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efdfc034000 tx=0x7efdfc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:16.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.468+0000 7efe0a000640 1 -- 172.21.15.7:0/1422984455 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efe0405f7a0 con 0x7efe140b5020 2024-04-25T11:21:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.670+0000 7efe18c00640 1 -- 172.21.15.7:0/1422984455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efe14070f20 con 0x7efe140b5020 2024-04-25T11:21:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.671+0000 7efe0a000640 1 -- 172.21.15.7:0/1422984455 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) ==== 104+0+11 (secure 0 0 0) 0x7efe0405f0a0 con 0x7efe140b5020 2024-04-25T11:21:16.668 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:21:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.673+0000 7efe18c00640 1 -- 172.21.15.7:0/1422984455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdf00744b0 msgr2=0x7efdf00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.673+0000 7efe18c00640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdf00744b0 0x7efdf00769c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efdfc034000 tx=0x7efdfc02e000 comp rx=0 tx=0).stop 2024-04-25T11:21:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.673+0000 7efe18c00640 1 -- 172.21.15.7:0/1422984455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe140b5020 msgr2=0x7efe14159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.673+0000 7efe18c00640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe140b5020 0x7efe14159c50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7efe0400caa0 tx=0x7efe0400cf60 comp rx=0 tx=0).stop 2024-04-25T11:21:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.674+0000 7efe18c00640 1 -- 172.21.15.7:0/1422984455 shutdown_connections 2024-04-25T11:21:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.674+0000 7efe18c00640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efdf00744b0 0x7efdf00769c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.674+0000 7efe18c00640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe1415aaa0 0x7efe14143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.674+0000 7efe18c00640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe140d0ee0 0x7efe14143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.674+0000 7efe18c00640 1 --2- 172.21.15.7:0/1422984455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe140b5020 0x7efe14159c50 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.674+0000 7efe18c00640 1 -- 172.21.15.7:0/1422984455 >> 172.21.15.7:0/1422984455 conn(0x7efe140742c0 msgr2=0x7efe1406cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.674+0000 7efe18c00640 1 -- 172.21.15.7:0/1422984455 shutdown_connections 2024-04-25T11:21:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.674+0000 7efe18c00640 1 -- 172.21.15.7:0/1422984455 wait complete. 2024-04-25T11:21:16.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:21:16.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.988+0000 7f1252a00640 1 Processor -- start 2024-04-25T11:21:16.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.988+0000 7f1252a00640 1 -- start start 2024-04-25T11:21:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.991+0000 7f1252a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15dfb0 0x7f124c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.991+0000 7f1252a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f124c0d0b40 0x7f124c06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.991+0000 7f1252a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f124c15e3c0 con 0x7f124c0d0f50 2024-04-25T11:21:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.991+0000 7f1252a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f124c0b09b0 con 0x7f124c0d0b40 2024-04-25T11:21:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.991+0000 7f1252a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f124c0cf800 con 0x7f124c15dfb0 2024-04-25T11:21:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.991+0000 7f1250a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f124c0d0f50 0x7f124c161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39976/0 (socket says 172.21.15.7:39976) 2024-04-25T11:21:16.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.991+0000 7f1250a00640 1 -- 172.21.15.7:0/2677777257 learned_addr learned my addr 172.21.15.7:0/2677777257 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124b400640 1 --2- 172.21.15.7:0/2677777257 >> v2:172.21.15.87:3300/0 conn(0x7f124c0d0b40 0x7f124c06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.991+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2663167151 0 0) 0x7f124c15e3c0 con 0x7f124c0d0f50 2024-04-25T11:21:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124be00640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15dfb0 0x7f124c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1230003520 con 0x7f124c0d0f50 2024-04-25T11:21:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3966514158 0 0) 0x7f1230003520 con 0x7f124c0d0f50 2024-04-25T11:21:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15dfb0 msgr2=0x7f124c0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:16.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15dfb0 0x7f124c0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 >> v2:172.21.15.87:3300/0 conn(0x7f124c0d0b40 msgr2=0x7f124c06f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 --2- 172.21.15.7:0/2677777257 >> v2:172.21.15.87:3300/0 conn(0x7f124c0d0b40 0x7f124c06f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c0cfaa0 con 0x7f124c0d0f50 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1244001670 con 0x7f124c0d0f50 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 >> v1:172.21.15.7:6789/0 conn(0x7f124c0d0f50 legacy=0x7f124c161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 0x7f1230003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1230004800 0x7f1230004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15dfb0 0x7f1230005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f124c0b09b0 con 0x7f12300039b0 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.992+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f124c0cf800 con 0x7f1230004800 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.993+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f123000b9c0 con 0x7f124c15dfb0 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.993+0000 7f124be00640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 0x7f1230003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.993+0000 7f124b400640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1230004800 0x7f1230004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.993+0000 7f1250a00640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15dfb0 0x7f1230005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.993+0000 7f124be00640 1 -- 172.21.15.7:0/2677777257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15dfb0 msgr2=0x7f1230005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:16.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.993+0000 7f124be00640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15dfb0 0x7f1230005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.993+0000 7f124be00640 1 -- 172.21.15.7:0/2677777257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1230004800 msgr2=0x7f1230004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.993+0000 7f124be00640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1230004800 0x7f1230004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.994+0000 7f124be00640 1 -- 172.21.15.7:0/2677777257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1230003520 con 0x7f12300039b0 2024-04-25T11:21:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.995+0000 7f124be00640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 0x7f1230003e20 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f123c009f90 tx=0x7f1230005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b3fc04e9fe23db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.995+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c009c10 con 0x7f12300039b0 2024-04-25T11:21:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.995+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f123c036070 con 0x7f12300039b0 2024-04-25T11:21:16.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.995+0000 7f124aa00640 1 -- 172.21.15.7:0/2677777257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c031810 con 0x7f12300039b0 2024-04-25T11:21:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.996+0000 7f1252a00640 1 -- 172.21.15.7:0/2677777257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 msgr2=0x7f1230003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.996+0000 7f1252a00640 1 --2- 172.21.15.7:0/2677777257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 0x7f1230003e20 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f123c009f90 tx=0x7f1230005830 comp rx=0 tx=0).stop 2024-04-25T11:21:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.996+0000 7f1250a00640 1 -- 172.21.15.7:0/2677777257 reap_dead start 2024-04-25T11:21:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.996+0000 7f1252a00640 1 -- 172.21.15.7:0/2677777257 shutdown_connections 2024-04-25T11:21:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.996+0000 7f1252a00640 1 -- 172.21.15.7:0/2677777257 >> 172.21.15.7:0/2677777257 conn(0x7f124c0742c0 msgr2=0x7f124c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:16.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.996+0000 7f1252a00640 1 -- 172.21.15.7:0/2677777257 shutdown_connections 2024-04-25T11:21:16.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.996+0000 7f1252a00640 1 -- 172.21.15.7:0/2677777257 wait complete. 2024-04-25T11:21:16.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.999+0000 7f1252a00640 1 Processor -- start 2024-04-25T11:21:16.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:16.999+0000 7f1252a00640 1 -- start start 2024-04-25T11:21:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.000+0000 7f1252a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 0x7f124c1f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.001+0000 7f1252a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15a390 0x7f124c158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.001+0000 7f1252a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f124c1500a0 0x7f124c159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.001+0000 7f1252a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f124c0d0340 con 0x7f12300039b0 2024-04-25T11:21:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.001+0000 7f1252a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f124c0d0060 con 0x7f124c1500a0 2024-04-25T11:21:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.001+0000 7f1252a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f124c0d0620 con 0x7f124c15a390 2024-04-25T11:21:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.001+0000 7f124be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 0x7f124c1f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.001+0000 7f124be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 0x7f124c1f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37630/0 (socket says 172.21.15.7:37630) 2024-04-25T11:21:16.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.001+0000 7f124be00640 1 -- 172.21.15.7:0/1520284124 learned_addr learned my addr 172.21.15.7:0/1520284124 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.001+0000 7f124b400640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15a390 0x7f124c158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.001+0000 7f1250a00640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f124c1500a0 0x7f124c159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.002+0000 7f124be00640 1 -- 172.21.15.7:0/1520284124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15a390 msgr2=0x7f124c158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.002+0000 7f124be00640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15a390 0x7f124c158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.002+0000 7f124be00640 1 -- 172.21.15.7:0/1520284124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f124c1500a0 msgr2=0x7f124c159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.002+0000 7f124be00640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f124c1500a0 0x7f124c159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.002+0000 7f124be00640 1 -- 172.21.15.7:0/1520284124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c15cc00 con 0x7f12300039b0 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.002+0000 7f124be00640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 0x7f124c1f6030 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f123c00e230 tx=0x7f123c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.002+0000 7f1243e00640 1 -- 172.21.15.7:0/1520284124 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c045430 con 0x7f12300039b0 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.002+0000 7f1243e00640 1 -- 172.21.15.7:0/1520284124 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f123c0455d0 con 0x7f12300039b0 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.002+0000 7f1243e00640 1 -- 172.21.15.7:0/1520284124 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c0459c0 con 0x7f12300039b0 2024-04-25T11:21:17.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.002+0000 7f1252a00640 1 -- 172.21.15.7:0/1520284124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c15cee0 con 0x7f12300039b0 2024-04-25T11:21:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.003+0000 7f1252a00640 1 -- 172.21.15.7:0/1520284124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c15d4a0 con 0x7f12300039b0 2024-04-25T11:21:17.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.004+0000 7f1252a00640 1 -- 172.21.15.7:0/1520284124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124c050740 con 0x7f12300039b0 2024-04-25T11:21:17.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.005+0000 7f1243e00640 1 -- 172.21.15.7:0/1520284124 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f123c005d10 con 0x7f12300039b0 2024-04-25T11:21:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.006+0000 7f1243e00640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f122c0744b0 0x7f122c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.006+0000 7f1243e00640 1 -- 172.21.15.7:0/1520284124 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f123c0b6ba0 con 0x7f12300039b0 2024-04-25T11:21:17.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.007+0000 7f124b400640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f122c0744b0 0x7f122c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:17.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.007+0000 7f124b400640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f122c0744b0 0x7f122c0769c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f124c1604b0 tx=0x7f1238008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:17.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.011+0000 7f1243e00640 1 -- 172.21.15.7:0/1520284124 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f123c02e050 con 0x7f12300039b0 2024-04-25T11:21:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.220+0000 7f1252a00640 1 -- 172.21.15.7:0/1520284124 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f124c070fb0 con 0x7f12300039b0 2024-04-25T11:21:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.221+0000 7f1243e00640 1 -- 172.21.15.7:0/1520284124 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) ==== 100+0+11 (secure 0 0 0) 0x7f123c043e40 con 0x7f12300039b0 2024-04-25T11:21:17.218 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:21:17.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 -- 172.21.15.7:0/1520284124 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f122c0744b0 msgr2=0x7f122c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f122c0744b0 0x7f122c0769c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f124c1604b0 tx=0x7f1238008040 comp rx=0 tx=0).stop 2024-04-25T11:21:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 -- 172.21.15.7:0/1520284124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 msgr2=0x7f124c1f6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 0x7f124c1f6030 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f123c00e230 tx=0x7f123c009750 comp rx=0 tx=0).stop 2024-04-25T11:21:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 -- 172.21.15.7:0/1520284124 shutdown_connections 2024-04-25T11:21:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f122c0744b0 0x7f122c0769c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f124c1500a0 0x7f124c159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f124c15a390 0x7f124c158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 --2- 172.21.15.7:0/1520284124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12300039b0 0x7f124c1f6030 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 -- 172.21.15.7:0/1520284124 >> 172.21.15.7:0/1520284124 conn(0x7f124c0742c0 msgr2=0x7f124c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 -- 172.21.15.7:0/1520284124 shutdown_connections 2024-04-25T11:21:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.224+0000 7f1252a00640 1 -- 172.21.15.7:0/1520284124 wait complete. 2024-04-25T11:21:17.249 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:21:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.535+0000 7f8a1ca00640 1 Processor -- start 2024-04-25T11:21:17.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.536+0000 7f8a1ca00640 1 -- start start 2024-04-25T11:21:17.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.538+0000 7f8a1ca00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a1806dc70 0x7f8a180d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.538+0000 7f8a1ca00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a180b2820 con 0x7f8a180d24f0 2024-04-25T11:21:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.538+0000 7f8a1ca00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a180b2260 con 0x7f8a180d2900 2024-04-25T11:21:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.538+0000 7f8a1ca00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f8a180b2540 con 0x7f8a1806dc70 2024-04-25T11:21:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.539+0000 7f8a17400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a1806dc70 0x7f8a180d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.539+0000 7f8a0ea00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8a180d24f0 0x7f8a180aefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39986/0 (socket says 172.21.15.7:39986) 2024-04-25T11:21:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.539+0000 7f8a0ea00640 1 -- 172.21.15.7:0/2936036914 learned_addr learned my addr 172.21.15.7:0/2936036914 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:17.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.539+0000 7f8a17400640 1 -- 172.21.15.7:0/2936036914 >> v1:172.21.15.87:6789/0 conn(0x7f8a180d2900 legacy=0x7f8a180af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.539+0000 7f8a17400640 1 -- 172.21.15.7:0/2936036914 >> v1:172.21.15.7:6789/0 conn(0x7f8a180d24f0 legacy=0x7f8a180aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.539+0000 7f8a17400640 1 -- 172.21.15.7:0/2936036914 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1806d120 con 0x7f8a1806dc70 2024-04-25T11:21:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.539+0000 7f8a16a00640 1 -- 172.21.15.7:0/2936036914 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1383151528 0 0) 0x7f8a180b2820 con 0x7f8a180d24f0 2024-04-25T11:21:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.540+0000 7f8a17400640 1 --2- 172.21.15.7:0/2936036914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1806dc70 0x7f8a180d1fb0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f8a080097a0 tx=0x7f8a0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ea05edc4b1f7fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.541+0000 7f8a16a00640 1 -- 172.21.15.7:0/2936036914 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08037070 con 0x7f8a1806dc70 2024-04-25T11:21:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.541+0000 7f8a16a00640 1 -- 172.21.15.7:0/2936036914 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a08032440 con 0x7f8a1806dc70 2024-04-25T11:21:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.541+0000 7f8a16a00640 1 -- 172.21.15.7:0/2936036914 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08032810 con 0x7f8a1806dc70 2024-04-25T11:21:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.541+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2936036914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1806dc70 msgr2=0x7f8a180d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.542+0000 7f8a1ca00640 1 --2- 172.21.15.7:0/2936036914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1806dc70 0x7f8a180d1fb0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f8a080097a0 tx=0x7f8a0802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.542+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2936036914 shutdown_connections 2024-04-25T11:21:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.542+0000 7f8a1ca00640 1 --2- 172.21.15.7:0/2936036914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1806dc70 0x7f8a180d1fb0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.542+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2936036914 >> 172.21.15.7:0/2936036914 conn(0x7f8a18073b60 msgr2=0x7f8a180b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.542+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2936036914 shutdown_connections 2024-04-25T11:21:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.542+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2936036914 wait complete. 2024-04-25T11:21:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.544+0000 7f8a1ca00640 1 Processor -- start 2024-04-25T11:21:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.544+0000 7f8a1ca00640 1 -- start start 2024-04-25T11:21:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.546+0000 7f8a1ca00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1806dc70 0x7f8a181f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.546+0000 7f8a1ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a180d24f0 0x7f8a18151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.546+0000 7f8a1ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a180d2900 0x7f8a18070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.546+0000 7f8a1ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a180b1ea0 con 0x7f8a180d2900 2024-04-25T11:21:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.546+0000 7f8a1ca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1809b990 con 0x7f8a180d24f0 2024-04-25T11:21:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.546+0000 7f8a1ca00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1809b540 con 0x7f8a1806dc70 2024-04-25T11:21:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.546+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a180d2900 0x7f8a18070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:17.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.546+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a180d2900 0x7f8a18070c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37654/0 (socket says 172.21.15.7:37654) 2024-04-25T11:21:17.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.546+0000 7f8a17e00640 1 -- 172.21.15.7:0/2293131403 learned_addr learned my addr 172.21.15.7:0/2293131403 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:17.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a0ea00640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a180d24f0 0x7f8a18151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:17.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a17400640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1806dc70 0x7f8a181f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:17.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a17e00640 1 -- 172.21.15.7:0/2293131403 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1806dc70 msgr2=0x7f8a181f6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:17.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a17e00640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1806dc70 0x7f8a181f6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a17e00640 1 -- 172.21.15.7:0/2293131403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a180d24f0 msgr2=0x7f8a18151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:17.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a17e00640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a180d24f0 0x7f8a18151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a17e00640 1 -- 172.21.15.7:0/2293131403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a18152320 con 0x7f8a180d2900 2024-04-25T11:21:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a17400640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1806dc70 0x7f8a181f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a17e00640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a180d2900 0x7f8a18070c90 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000ba10 tx=0x7f8a1000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a0fe00640 1 -- 172.21.15.7:0/2293131403 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10019070 con 0x7f8a180d2900 2024-04-25T11:21:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a0fe00640 1 -- 172.21.15.7:0/2293131403 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1000cb40 con 0x7f8a180d2900 2024-04-25T11:21:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a0fe00640 1 -- 172.21.15.7:0/2293131403 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10014440 con 0x7f8a180d2900 2024-04-25T11:21:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.547+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2293131403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a181f9760 con 0x7f8a180d2900 2024-04-25T11:21:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.548+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2293131403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a181f9c90 con 0x7f8a180d2900 2024-04-25T11:21:17.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.549+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2293131403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a181f9a00 con 0x7f8a180d2900 2024-04-25T11:21:17.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.550+0000 7f8a0fe00640 1 -- 172.21.15.7:0/2293131403 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8a1001c070 con 0x7f8a180d2900 2024-04-25T11:21:17.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.551+0000 7f8a0fe00640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f40744e0 0x7f89f40769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:17.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.551+0000 7f8a0fe00640 1 -- 172.21.15.7:0/2293131403 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f8a10094fa0 con 0x7f8a180d2900 2024-04-25T11:21:17.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.551+0000 7f8a17400640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f40744e0 0x7f89f40769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:17.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.552+0000 7f8a17400640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f40744e0 0x7f89f40769f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8a08034000 tx=0x7f8a0802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:17.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.556+0000 7f8a0fe00640 1 -- 172.21.15.7:0/2293131403 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8a1005f500 con 0x7f8a180d2900 2024-04-25T11:21:17.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.803+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2293131403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a181524f0 con 0x7f8a180d2900 2024-04-25T11:21:17.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.804+0000 7f8a0fe00640 1 -- 172.21.15.7:0/2293131403 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5175 (secure 0 0 0) 0x7f8a1005ee00 con 0x7f8a180d2900 2024-04-25T11:21:17.802 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:17.802 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2288158899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2288158899},{"type":"v1","addr":"172.21.15.7:6837","nonce":2288158899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3545919245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3545919245},{"type":"v1","addr":"172.21.15.87:6835","nonce":3545919245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8478,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/951407928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":951407928},{"type":"v1","addr":"172.21.15.7:6836","nonce":951407928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:42.908797+0000","modified":"2024-04-25T11:21:15.439726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.87:6833/1243096872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1243096872},{"type":"v1","addr":"172.21.15.87:6833","nonce":1243096872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8484,"qdb_cluster":[8484]},"id":9}]} 2024-04-25T11:21:17.802 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2024-04-25T11:21:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.807+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2293131403 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f40744e0 msgr2=0x7f89f40769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.807+0000 7f8a1ca00640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f40744e0 0x7f89f40769f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8a08034000 tx=0x7f8a0802e000 comp rx=0 tx=0).stop 2024-04-25T11:21:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.808+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2293131403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a180d2900 msgr2=0x7f8a18070c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.808+0000 7f8a1ca00640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a180d2900 0x7f8a18070c90 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000ba10 tx=0x7f8a1000bed0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.808+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2293131403 shutdown_connections 2024-04-25T11:21:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.808+0000 7f8a1ca00640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f89f40744e0 0x7f89f40769f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.808+0000 7f8a1ca00640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a180d2900 0x7f8a18070c90 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.808+0000 7f8a1ca00640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a180d24f0 0x7f8a18151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.808+0000 7f8a1ca00640 1 --2- 172.21.15.7:0/2293131403 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a1806dc70 0x7f8a181f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.808+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2293131403 >> 172.21.15.7:0/2293131403 conn(0x7f8a18073b60 msgr2=0x7f8a18159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.808+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2293131403 shutdown_connections 2024-04-25T11:21:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:17.808+0000 7f8a1ca00640 1 -- 172.21.15.7:0/2293131403 wait complete. 2024-04-25T11:21:17.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:21:18.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.113+0000 7f2ff5800640 1 Processor -- start 2024-04-25T11:21:18.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.113+0000 7f2ff5800640 1 -- start start 2024-04-25T11:21:18.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.115+0000 7f2ff5800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2ff00aed40 0x7f2ff015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.115+0000 7f2ff5800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ff00b5220 0x7f2ff00ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.115+0000 7f2ff5800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ff00b64b0 con 0x7f2ff00af420 2024-04-25T11:21:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.115+0000 7f2ff5800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2ff015e1f0 con 0x7f2ff00b5220 2024-04-25T11:21:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.115+0000 7f2ff5800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2ff006c610 con 0x7f2ff00aed40 2024-04-25T11:21:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.116+0000 7f2fef400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ff00b5220 0x7f2ff00ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.116+0000 7f2fefe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2ff00aed40 0x7f2ff015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.116+0000 7f2fef400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ff00b5220 0x7f2ff00ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37664/0 (socket says 172.21.15.7:37664) 2024-04-25T11:21:18.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.116+0000 7f2fef400640 1 -- 172.21.15.7:0/2920869066 learned_addr learned my addr 172.21.15.7:0/2920869066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:18.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.116+0000 7f2fef400640 1 -- 172.21.15.7:0/2920869066 >> v2:172.21.15.87:3300/0 conn(0x7f2ff00aed40 msgr2=0x7f2ff015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:18.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.116+0000 7f2fef400640 1 --2- 172.21.15.7:0/2920869066 >> v2:172.21.15.87:3300/0 conn(0x7f2ff00aed40 0x7f2ff015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.116+0000 7f2fef400640 1 -- 172.21.15.7:0/2920869066 >> v1:172.21.15.87:6789/0 conn(0x7f2ff00af420 legacy=0x7f2ff00af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.116+0000 7f2fef400640 1 -- 172.21.15.7:0/2920869066 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff006c8f0 con 0x7f2ff00b5220 2024-04-25T11:21:18.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.117+0000 7f2fef400640 1 --2- 172.21.15.7:0/2920869066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff00b5220 0x7f2ff00ce4a0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00725e0 tx=0x7f2fdc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cc0cb8cd81712f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.118+0000 7f2feea00640 1 -- 172.21.15.7:0/2920869066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc037070 con 0x7f2ff00b5220 2024-04-25T11:21:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.118+0000 7f2feea00640 1 -- 172.21.15.7:0/2920869066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fdc032440 con 0x7f2ff00b5220 2024-04-25T11:21:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.118+0000 7f2feea00640 1 -- 172.21.15.7:0/2920869066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc032810 con 0x7f2ff00b5220 2024-04-25T11:21:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.119+0000 7f2ff5800640 1 -- 172.21.15.7:0/2920869066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff00b5220 msgr2=0x7f2ff00ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.119+0000 7f2ff5800640 1 --2- 172.21.15.7:0/2920869066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff00b5220 0x7f2ff00ce4a0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00725e0 tx=0x7f2fdc02a270 comp rx=0 tx=0).stop 2024-04-25T11:21:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.119+0000 7f2ff5800640 1 -- 172.21.15.7:0/2920869066 shutdown_connections 2024-04-25T11:21:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.119+0000 7f2ff5800640 1 --2- 172.21.15.7:0/2920869066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff00b5220 0x7f2ff00ce4a0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.119+0000 7f2ff5800640 1 --2- 172.21.15.7:0/2920869066 >> v2:172.21.15.87:3300/0 conn(0x7f2ff00aed40 0x7f2ff015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.119+0000 7f2ff5800640 1 -- 172.21.15.7:0/2920869066 >> 172.21.15.7:0/2920869066 conn(0x7f2ff00b3820 msgr2=0x7f2ff00b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.119+0000 7f2ff5800640 1 -- 172.21.15.7:0/2920869066 shutdown_connections 2024-04-25T11:21:18.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.119+0000 7f2ff5800640 1 -- 172.21.15.7:0/2920869066 wait complete. 2024-04-25T11:21:18.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.121+0000 7f2ff5800640 1 Processor -- start 2024-04-25T11:21:18.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.121+0000 7f2ff5800640 1 -- start start 2024-04-25T11:21:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.123+0000 7f2ff5800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ff00aed40 0x7f2ff015b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.123+0000 7f2ff5800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ff00af420 0x7f2ff01f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.123+0000 7f2ff5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff00b5220 0x7f2ff015e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.123+0000 7f2ff5800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff00b6110 con 0x7f2ff00b5220 2024-04-25T11:21:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.123+0000 7f2ff5800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff006ceb0 con 0x7f2ff00aed40 2024-04-25T11:21:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.123+0000 7f2ff5800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff00b6290 con 0x7f2ff00af420 2024-04-25T11:21:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.123+0000 7f2ff4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff00b5220 0x7f2ff015e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.123+0000 7f2ff4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff00b5220 0x7f2ff015e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37666/0 (socket says 172.21.15.7:37666) 2024-04-25T11:21:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.123+0000 7f2ff4e00640 1 -- 172.21.15.7:0/1918609640 learned_addr learned my addr 172.21.15.7:0/1918609640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.124+0000 7f2fefe00640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ff00aed40 0x7f2ff015b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.124+0000 7f2fef400640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ff00af420 0x7f2ff01f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.124+0000 7f2ff4e00640 1 -- 172.21.15.7:0/1918609640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ff00af420 msgr2=0x7f2ff01f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.124+0000 7f2ff4e00640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ff00af420 0x7f2ff01f3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.124+0000 7f2ff4e00640 1 -- 172.21.15.7:0/1918609640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ff00aed40 msgr2=0x7f2ff015b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.124+0000 7f2ff4e00640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ff00aed40 0x7f2ff015b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.124+0000 7f2ff4e00640 1 -- 172.21.15.7:0/1918609640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff015c0d0 con 0x7f2ff00b5220 2024-04-25T11:21:18.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.124+0000 7f2ff4e00640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff00b5220 0x7f2ff015e830 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c6e0 tx=0x7f2fe800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.124+0000 7f2fe7e00640 1 -- 172.21.15.7:0/1918609640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8003b70 con 0x7f2ff00b5220 2024-04-25T11:21:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.125+0000 7f2fe7e00640 1 -- 172.21.15.7:0/1918609640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe8003d30 con 0x7f2ff00b5220 2024-04-25T11:21:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.125+0000 7f2fe7e00640 1 -- 172.21.15.7:0/1918609640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8005310 con 0x7f2ff00b5220 2024-04-25T11:21:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.125+0000 7f2ff5800640 1 -- 172.21.15.7:0/1918609640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff015c3b0 con 0x7f2ff00b5220 2024-04-25T11:21:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.125+0000 7f2ff5800640 1 -- 172.21.15.7:0/1918609640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff015c970 con 0x7f2ff00b5220 2024-04-25T11:21:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.126+0000 7f2ff5800640 1 -- 172.21.15.7:0/1918609640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff015be40 con 0x7f2ff00b5220 2024-04-25T11:21:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.127+0000 7f2fe7e00640 1 -- 172.21.15.7:0/1918609640 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2fe80054b0 con 0x7f2ff00b5220 2024-04-25T11:21:18.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.128+0000 7f2fe7e00640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fd00744b0 0x7f2fd00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.128+0000 7f2fe7e00640 1 -- 172.21.15.7:0/1918609640 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f2fe80952d0 con 0x7f2ff00b5220 2024-04-25T11:21:18.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.128+0000 7f2fefe00640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fd00744b0 0x7f2fd00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.129+0000 7f2fefe00640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fd00744b0 0x7f2fd00769c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0071200 tx=0x7f2fe0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:18.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.133+0000 7f2fe7e00640 1 -- 172.21.15.7:0/1918609640 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fe805f830 con 0x7f2ff00b5220 2024-04-25T11:21:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.337+0000 7f2ff5800640 1 -- 172.21.15.7:0/1918609640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ff0068f40 con 0x7f2ff00b5220 2024-04-25T11:21:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.339+0000 7f2fe7e00640 1 -- 172.21.15.7:0/1918609640 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16804 (secure 0 0 0) 0x7f2ff0068f40 con 0x7f2ff00b5220 2024-04-25T11:21:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":130,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:20:44.396251+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:41.047420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:41.908907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.342+0000 7f2ff5800640 1 -- 172.21.15.7:0/1918609640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fd00744b0 msgr2=0x7f2fd00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.342+0000 7f2ff5800640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fd00744b0 0x7f2fd00769c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0071200 tx=0x7f2fe0008040 comp rx=0 tx=0).stop 2024-04-25T11:21:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.342+0000 7f2ff5800640 1 -- 172.21.15.7:0/1918609640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff00b5220 msgr2=0x7f2ff015e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.342+0000 7f2ff5800640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff00b5220 0x7f2ff015e830 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800c6e0 tx=0x7f2fe800cba0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.342+0000 7f2ff5800640 1 -- 172.21.15.7:0/1918609640 shutdown_connections 2024-04-25T11:21:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.342+0000 7f2ff5800640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fd00744b0 0x7f2fd00769c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.342+0000 7f2ff5800640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ff00b5220 0x7f2ff015e830 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.342+0000 7f2ff5800640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ff00af420 0x7f2ff01f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.342+0000 7f2ff5800640 1 --2- 172.21.15.7:0/1918609640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ff00aed40 0x7f2ff015b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.342+0000 7f2ff5800640 1 -- 172.21.15.7:0/1918609640 >> 172.21.15.7:0/1918609640 conn(0x7f2ff00b3820 msgr2=0x7f2ff00af830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.343+0000 7f2ff5800640 1 -- 172.21.15.7:0/1918609640 shutdown_connections 2024-04-25T11:21:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.343+0000 7f2ff5800640 1 -- 172.21.15.7:0/1918609640 wait complete. 2024-04-25T11:21:18.368 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:21:18.368 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:21:18.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.660+0000 7f3e78e00640 1 Processor -- start 2024-04-25T11:21:18.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.660+0000 7f3e78e00640 1 -- start start 2024-04-25T11:21:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.662+0000 7f3e78e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e7406e510 0x7f3e740b7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.662+0000 7f3e78e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3e740d0e60 0x7f3e740b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.662+0000 7f3e78e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e740d4390 con 0x7f3e7406e920 2024-04-25T11:21:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.662+0000 7f3e78e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3e740b9ee0 con 0x7f3e7406e510 2024-04-25T11:21:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.663+0000 7f3e78e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3e740d4cc0 con 0x7f3e740d0e60 2024-04-25T11:21:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.663+0000 7f3e72a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3e7406e920 0x7f3e740d0750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39992/0 (socket says 172.21.15.7:39992) 2024-04-25T11:21:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.663+0000 7f3e72a00640 1 -- 172.21.15.7:0/870521380 learned_addr learned my addr 172.21.15.7:0/870521380 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.663+0000 7f3e73400640 1 --2- 172.21.15.7:0/870521380 >> v2:172.21.15.7:3300/0 conn(0x7f3e7406e510 0x7f3e740b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.663+0000 7f3e72000640 1 -- 172.21.15.7:0/870521380 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3934002057 0 0) 0x7f3e740d4390 con 0x7f3e7406e920 2024-04-25T11:21:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.663+0000 7f3e72000640 1 -- 172.21.15.7:0/870521380 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e58003520 con 0x7f3e7406e920 2024-04-25T11:21:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.664+0000 7f3e73400640 1 -- 172.21.15.7:0/870521380 >> v2:172.21.15.87:3300/0 conn(0x7f3e740d0e60 msgr2=0x7f3e740b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.663+0000 7f3e73e00640 1 --2- 172.21.15.7:0/870521380 >> v2:172.21.15.87:3300/0 conn(0x7f3e740d0e60 0x7f3e740b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.664+0000 7f3e73400640 1 --2- 172.21.15.7:0/870521380 >> v2:172.21.15.87:3300/0 conn(0x7f3e740d0e60 0x7f3e740b7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.664+0000 7f3e73400640 1 -- 172.21.15.7:0/870521380 >> v1:172.21.15.7:6789/0 conn(0x7f3e7406e920 legacy=0x7f3e740d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.664+0000 7f3e73400640 1 -- 172.21.15.7:0/870521380 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e740d4670 con 0x7f3e7406e510 2024-04-25T11:21:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.664+0000 7f3e73e00640 1 --2- 172.21.15.7:0/870521380 >> v2:172.21.15.87:3300/0 conn(0x7f3e740d0e60 0x7f3e740b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.665+0000 7f3e73400640 1 --2- 172.21.15.7:0/870521380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e7406e510 0x7f3e740b7750 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f3e740cfec0 tx=0x7f3e5c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d56d9744c84a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.665+0000 7f3e72000640 1 -- 172.21.15.7:0/870521380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c037070 con 0x7f3e7406e510 2024-04-25T11:21:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.665+0000 7f3e72000640 1 -- 172.21.15.7:0/870521380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e5c032440 con 0x7f3e7406e510 2024-04-25T11:21:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.665+0000 7f3e72000640 1 -- 172.21.15.7:0/870521380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c02ae60 con 0x7f3e7406e510 2024-04-25T11:21:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.666+0000 7f3e78e00640 1 -- 172.21.15.7:0/870521380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e7406e510 msgr2=0x7f3e740b7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.666+0000 7f3e78e00640 1 --2- 172.21.15.7:0/870521380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e7406e510 0x7f3e740b7750 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f3e740cfec0 tx=0x7f3e5c02a110 comp rx=0 tx=0).stop 2024-04-25T11:21:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.666+0000 7f3e78e00640 1 -- 172.21.15.7:0/870521380 shutdown_connections 2024-04-25T11:21:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.666+0000 7f3e78e00640 1 --2- 172.21.15.7:0/870521380 >> v2:172.21.15.87:3300/0 conn(0x7f3e740d0e60 0x7f3e740b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.666+0000 7f3e78e00640 1 --2- 172.21.15.7:0/870521380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e7406e510 0x7f3e740b7750 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.666+0000 7f3e78e00640 1 -- 172.21.15.7:0/870521380 >> 172.21.15.7:0/870521380 conn(0x7f3e740d27b0 msgr2=0x7f3e740d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.666+0000 7f3e78e00640 1 -- 172.21.15.7:0/870521380 shutdown_connections 2024-04-25T11:21:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.666+0000 7f3e78e00640 1 -- 172.21.15.7:0/870521380 wait complete. 2024-04-25T11:21:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.668+0000 7f3e78e00640 1 Processor -- start 2024-04-25T11:21:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.669+0000 7f3e78e00640 1 -- start start 2024-04-25T11:21:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.670+0000 7f3e78e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e7406e510 0x7f3e741f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.670+0000 7f3e78e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e7406e920 0x7f3e74156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.670+0000 7f3e78e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e740d0e60 0x7f3e74154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.670+0000 7f3e78e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e740d1520 con 0x7f3e740d0e60 2024-04-25T11:21:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.670+0000 7f3e78e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3e740d13a0 con 0x7f3e7406e510 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.670+0000 7f3e78e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3e740d5150 con 0x7f3e7406e920 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e740d0e60 0x7f3e74154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e740d0e60 0x7f3e74154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37676/0 (socket says 172.21.15.7:37676) 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e73e00640 1 -- 172.21.15.7:0/3371506845 learned_addr learned my addr 172.21.15.7:0/3371506845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e73400640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e7406e510 0x7f3e741f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e72a00640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e7406e920 0x7f3e74156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e73e00640 1 -- 172.21.15.7:0/3371506845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e7406e920 msgr2=0x7f3e74156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e73e00640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e7406e920 0x7f3e74156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e73e00640 1 -- 172.21.15.7:0/3371506845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e7406e510 msgr2=0x7f3e741f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e73e00640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e7406e510 0x7f3e741f6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e73e00640 1 -- 172.21.15.7:0/3371506845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e741559a0 con 0x7f3e740d0e60 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e73400640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e7406e510 0x7f3e741f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.671+0000 7f3e73e00640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e740d0e60 0x7f3e74154be0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c5c0 tx=0x7f3e6c00ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.672+0000 7f3e6b400640 1 -- 172.21.15.7:0/3371506845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c015070 con 0x7f3e740d0e60 2024-04-25T11:21:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.672+0000 7f3e6b400640 1 -- 172.21.15.7:0/3371506845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e6c011820 con 0x7f3e740d0e60 2024-04-25T11:21:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.672+0000 7f3e78e00640 1 -- 172.21.15.7:0/3371506845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e74155ea0 con 0x7f3e740d0e60 2024-04-25T11:21:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.672+0000 7f3e6b400640 1 -- 172.21.15.7:0/3371506845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c011c10 con 0x7f3e740d0e60 2024-04-25T11:21:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.672+0000 7f3e78e00640 1 -- 172.21.15.7:0/3371506845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e74156460 con 0x7f3e740d0e60 2024-04-25T11:21:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.673+0000 7f3e78e00640 1 -- 172.21.15.7:0/3371506845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e74050740 con 0x7f3e740d0e60 2024-04-25T11:21:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.674+0000 7f3e6b400640 1 -- 172.21.15.7:0/3371506845 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3e6c018070 con 0x7f3e740d0e60 2024-04-25T11:21:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.675+0000 7f3e6b400640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3e500744b0 0x7f3e500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.675+0000 7f3e6b400640 1 -- 172.21.15.7:0/3371506845 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f3e6c095320 con 0x7f3e740d0e60 2024-04-25T11:21:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.676+0000 7f3e73400640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3e500744b0 0x7f3e500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.676+0000 7f3e73400640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3e500744b0 0x7f3e500769c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c034000 tx=0x7f3e5c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.680+0000 7f3e6b400640 1 -- 172.21.15.7:0/3371506845 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3e6c05f890 con 0x7f3e740d0e60 2024-04-25T11:21:18.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.929+0000 7f3e78e00640 1 -- 172.21.15.7:0/3371506845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e74155c30 con 0x7f3e740d0e60 2024-04-25T11:21:18.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.931+0000 7f3e6b400640 1 -- 172.21.15.7:0/3371506845 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5175 (secure 0 0 0) 0x7f3e6c05f190 con 0x7f3e740d0e60 2024-04-25T11:21:18.928 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:18.928 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2288158899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2288158899},{"type":"v1","addr":"172.21.15.7:6837","nonce":2288158899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3545919245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3545919245},{"type":"v1","addr":"172.21.15.87:6835","nonce":3545919245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8478,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/951407928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":951407928},{"type":"v1","addr":"172.21.15.7:6836","nonce":951407928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:42.908797+0000","modified":"2024-04-25T11:21:15.439726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.87:6833/1243096872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1243096872},{"type":"v1","addr":"172.21.15.87:6833","nonce":1243096872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8484,"qdb_cluster":[8484]},"id":9}]} 2024-04-25T11:21:18.928 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2024-04-25T11:21:18.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.933+0000 7f3e78e00640 1 -- 172.21.15.7:0/3371506845 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3e500744b0 msgr2=0x7f3e500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.933+0000 7f3e78e00640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3e500744b0 0x7f3e500769c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c034000 tx=0x7f3e5c02e000 comp rx=0 tx=0).stop 2024-04-25T11:21:18.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.933+0000 7f3e78e00640 1 -- 172.21.15.7:0/3371506845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e740d0e60 msgr2=0x7f3e74154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:18.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.933+0000 7f3e78e00640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e740d0e60 0x7f3e74154be0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c5c0 tx=0x7f3e6c00ca80 comp rx=0 tx=0).stop 2024-04-25T11:21:18.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.934+0000 7f3e78e00640 1 -- 172.21.15.7:0/3371506845 shutdown_connections 2024-04-25T11:21:18.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.934+0000 7f3e78e00640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3e500744b0 0x7f3e500769c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.934+0000 7f3e78e00640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e740d0e60 0x7f3e74154be0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.934+0000 7f3e78e00640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e7406e920 0x7f3e74156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.934+0000 7f3e78e00640 1 --2- 172.21.15.7:0/3371506845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e7406e510 0x7f3e741f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:18.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.934+0000 7f3e78e00640 1 -- 172.21.15.7:0/3371506845 >> 172.21.15.7:0/3371506845 conn(0x7f3e740d27b0 msgr2=0x7f3e740b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:18.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.934+0000 7f3e78e00640 1 -- 172.21.15.7:0/3371506845 shutdown_connections 2024-04-25T11:21:18.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:18.934+0000 7f3e78e00640 1 -- 172.21.15.7:0/3371506845 wait complete. 2024-04-25T11:21:18.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:21:19.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.252+0000 7facd7000640 1 Processor -- start 2024-04-25T11:21:19.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.252+0000 7facd7000640 1 -- start start 2024-04-25T11:21:19.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.255+0000 7facd7000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7facd00b9140 0x7facd00b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.255+0000 7facd7000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7facd00b71e0 0x7facd00b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.255+0000 7facd7000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd00cf900 con 0x7facd00cf4f0 2024-04-25T11:21:19.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.255+0000 7facd7000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7facd009d2c0 con 0x7facd00b9140 2024-04-25T11:21:19.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.255+0000 7facd7000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7facd00d0240 con 0x7facd00b71e0 2024-04-25T11:21:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.255+0000 7faccfe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7facd00b9140 0x7facd00b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.255+0000 7faccfe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7facd00b9140 0x7facd00b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37688/0 (socket says 172.21.15.7:37688) 2024-04-25T11:21:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.255+0000 7faccfe00640 1 -- 172.21.15.7:0/168497424 learned_addr learned my addr 172.21.15.7:0/168497424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.256+0000 7facd5000640 1 --2- 172.21.15.7:0/168497424 >> v2:172.21.15.87:3300/0 conn(0x7facd00b71e0 0x7facd00b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.256+0000 7faccfe00640 1 -- 172.21.15.7:0/168497424 >> v2:172.21.15.87:3300/0 conn(0x7facd00b71e0 msgr2=0x7facd00b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.256+0000 7faccfe00640 1 --2- 172.21.15.7:0/168497424 >> v2:172.21.15.87:3300/0 conn(0x7facd00b71e0 0x7facd00b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.256+0000 7faccfe00640 1 -- 172.21.15.7:0/168497424 >> v1:172.21.15.87:6789/0 conn(0x7facd00cf4f0 legacy=0x7facd00ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:19.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.256+0000 7faccfe00640 1 -- 172.21.15.7:0/168497424 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd00d2700 con 0x7facd00b9140 2024-04-25T11:21:19.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.256+0000 7facd5000640 1 --2- 172.21.15.7:0/168497424 >> v2:172.21.15.87:3300/0 conn(0x7facd00b71e0 0x7facd00b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:21:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.257+0000 7faccfe00640 1 --2- 172.21.15.7:0/168497424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd00b9140 0x7facd00b8a80 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7facd0074cd0 tx=0x7facc802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74a0c903fd91101 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.258+0000 7faccea00640 1 -- 172.21.15.7:0/168497424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8036070 con 0x7facd00b9140 2024-04-25T11:21:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.258+0000 7faccea00640 1 -- 172.21.15.7:0/168497424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc8031600 con 0x7facd00b9140 2024-04-25T11:21:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.258+0000 7faccea00640 1 -- 172.21.15.7:0/168497424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc80319d0 con 0x7facd00b9140 2024-04-25T11:21:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.259+0000 7facd7000640 1 -- 172.21.15.7:0/168497424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd00b9140 msgr2=0x7facd00b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.259+0000 7facd7000640 1 --2- 172.21.15.7:0/168497424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd00b9140 0x7facd00b8a80 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7facd0074cd0 tx=0x7facc802a730 comp rx=0 tx=0).stop 2024-04-25T11:21:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.259+0000 7facd7000640 1 -- 172.21.15.7:0/168497424 shutdown_connections 2024-04-25T11:21:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.259+0000 7facd7000640 1 --2- 172.21.15.7:0/168497424 >> v2:172.21.15.87:3300/0 conn(0x7facd00b71e0 0x7facd00b75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.259+0000 7facd7000640 1 --2- 172.21.15.7:0/168497424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd00b9140 0x7facd00b8a80 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.259+0000 7facd7000640 1 -- 172.21.15.7:0/168497424 >> 172.21.15.7:0/168497424 conn(0x7facd00d0400 msgr2=0x7facd00d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:19.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.259+0000 7facd7000640 1 -- 172.21.15.7:0/168497424 shutdown_connections 2024-04-25T11:21:19.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.259+0000 7facd7000640 1 -- 172.21.15.7:0/168497424 wait complete. 2024-04-25T11:21:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.261+0000 7facd7000640 1 Processor -- start 2024-04-25T11:21:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.262+0000 7facd7000640 1 -- start start 2024-04-25T11:21:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.263+0000 7facd7000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd00b71e0 0x7facd00ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.263+0000 7facd7000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facd00b9140 0x7facd01dab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.263+0000 7facd7000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd00cf4f0 0x7facd01eac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.263+0000 7facd7000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7facd00cef40 con 0x7facd00cf4f0 2024-04-25T11:21:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.263+0000 7facd7000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7facd009cfe0 con 0x7facd00b71e0 2024-04-25T11:21:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.263+0000 7facd7000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7facd00b7b30 con 0x7facd00b9140 2024-04-25T11:21:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.264+0000 7facd5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd00cf4f0 0x7facd01eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.264+0000 7facd5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd00cf4f0 0x7facd01eac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37694/0 (socket says 172.21.15.7:37694) 2024-04-25T11:21:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.264+0000 7facd5000640 1 -- 172.21.15.7:0/3805509855 learned_addr learned my addr 172.21.15.7:0/3805509855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:19.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.264+0000 7faccfe00640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd00b71e0 0x7facd00ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.264+0000 7faccf400640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facd00b9140 0x7facd01dab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.264+0000 7facd5000640 1 -- 172.21.15.7:0/3805509855 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facd00b9140 msgr2=0x7facd01dab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:19.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.264+0000 7facd5000640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facd00b9140 0x7facd01dab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.264+0000 7facd5000640 1 -- 172.21.15.7:0/3805509855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd00b71e0 msgr2=0x7facd00ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:19.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.264+0000 7facd5000640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd00b71e0 0x7facd00ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.264+0000 7facd5000640 1 -- 172.21.15.7:0/3805509855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd00cb420 con 0x7facd00cf4f0 2024-04-25T11:21:19.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.264+0000 7faccf400640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facd00b9140 0x7facd01dab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:21:19.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.265+0000 7facd5000640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd00cf4f0 0x7facd01eac40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7facc000d800 tx=0x7facc000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.266+0000 7facc7e00640 1 -- 172.21.15.7:0/3805509855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0004030 con 0x7facd00cf4f0 2024-04-25T11:21:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.267+0000 7facc7e00640 1 -- 172.21.15.7:0/3805509855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc00041d0 con 0x7facd00cf4f0 2024-04-25T11:21:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.267+0000 7facc7e00640 1 -- 172.21.15.7:0/3805509855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc00096a0 con 0x7facd00cf4f0 2024-04-25T11:21:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.267+0000 7facd7000640 1 -- 172.21.15.7:0/3805509855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd00cb700 con 0x7facd00cf4f0 2024-04-25T11:21:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.267+0000 7facd7000640 1 -- 172.21.15.7:0/3805509855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd00cbcc0 con 0x7facd00cf4f0 2024-04-25T11:21:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.268+0000 7facd7000640 1 -- 172.21.15.7:0/3805509855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd006a100 con 0x7facd00cf4f0 2024-04-25T11:21:19.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.269+0000 7facc7e00640 1 -- 172.21.15.7:0/3805509855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7facc0009840 con 0x7facd00cf4f0 2024-04-25T11:21:19.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.271+0000 7facc7e00640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7facac0744b0 0x7facac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.271+0000 7facc7e00640 1 -- 172.21.15.7:0/3805509855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7facc0095730 con 0x7facd00cf4f0 2024-04-25T11:21:19.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.271+0000 7faccfe00640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7facac0744b0 0x7facac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.272+0000 7faccfe00640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7facac0744b0 0x7facac0769c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7facc8033000 tx=0x7facc8009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.276+0000 7facc7e00640 1 -- 172.21.15.7:0/3805509855 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7facc005fc90 con 0x7facd00cf4f0 2024-04-25T11:21:19.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.517+0000 7facd7000640 1 -- 172.21.15.7:0/3805509855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facd00cd240 con 0x7facd00cf4f0 2024-04-25T11:21:19.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.518+0000 7facc7e00640 1 -- 172.21.15.7:0/3805509855 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5175 (secure 0 0 0) 0x7facc005f590 con 0x7facd00cf4f0 2024-04-25T11:21:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:19.516 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2288158899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2288158899},{"type":"v1","addr":"172.21.15.7:6837","nonce":2288158899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3545919245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3545919245},{"type":"v1","addr":"172.21.15.87:6835","nonce":3545919245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":8478,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/951407928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":951407928},{"type":"v1","addr":"172.21.15.7:6836","nonce":951407928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:20:42.908797+0000","modified":"2024-04-25T11:21:15.439726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8484},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8484":{"gid":8484,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.87:6833/1243096872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1243096872},{"type":"v1","addr":"172.21.15.87:6833","nonce":1243096872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8484,"qdb_cluster":[8484]},"id":9}]} 2024-04-25T11:21:19.516 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2024-04-25T11:21:19.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 -- 172.21.15.7:0/3805509855 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7facac0744b0 msgr2=0x7facac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7facac0744b0 0x7facac0769c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7facc8033000 tx=0x7facc8009000 comp rx=0 tx=0).stop 2024-04-25T11:21:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 -- 172.21.15.7:0/3805509855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd00cf4f0 msgr2=0x7facd01eac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd00cf4f0 0x7facd01eac40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7facc000d800 tx=0x7facc000dcc0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 -- 172.21.15.7:0/3805509855 shutdown_connections 2024-04-25T11:21:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7facac0744b0 0x7facac0769c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facd00cf4f0 0x7facd01eac40 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facd00b9140 0x7facd01dab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 --2- 172.21.15.7:0/3805509855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facd00b71e0 0x7facd00ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 -- 172.21.15.7:0/3805509855 >> 172.21.15.7:0/3805509855 conn(0x7facd00d0400 msgr2=0x7facd006f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 -- 172.21.15.7:0/3805509855 shutdown_connections 2024-04-25T11:21:19.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.521+0000 7facd7000640 1 -- 172.21.15.7:0/3805509855 wait complete. 2024-04-25T11:21:19.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:21:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.831+0000 7ff650c00640 1 Processor -- start 2024-04-25T11:21:19.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.831+0000 7ff650c00640 1 -- start start 2024-04-25T11:21:19.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.833+0000 7ff650c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06dc90 0x7ff64c0afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.833+0000 7ff650c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff64c160a30 0x7ff64c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.833+0000 7ff650c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64c0b4980 con 0x7ff64c06f300 2024-04-25T11:21:19.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.833+0000 7ff650c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff64c0b4c60 con 0x7ff64c160a30 2024-04-25T11:21:19.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.833+0000 7ff650c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff64c0b4f40 con 0x7ff64c06dc90 2024-04-25T11:21:19.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.834+0000 7ff641400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff64c06f300 0x7ff64c0ae960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39994/0 (socket says 172.21.15.7:39994) 2024-04-25T11:21:19.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.834+0000 7ff641400640 1 -- 172.21.15.7:0/2159307436 learned_addr learned my addr 172.21.15.7:0/2159307436 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:19.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.834+0000 7ff64a800640 1 --2- 172.21.15.7:0/2159307436 >> v2:172.21.15.7:3300/0 conn(0x7ff64c160a30 0x7ff64c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.834+0000 7ff649e00640 1 --2- 172.21.15.7:0/2159307436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06dc90 0x7ff64c0afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.834+0000 7ff649400640 1 -- 172.21.15.7:0/2159307436 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2251463960 0 0) 0x7ff64c0b4980 con 0x7ff64c06f300 2024-04-25T11:21:19.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.834+0000 7ff649400640 1 -- 172.21.15.7:0/2159307436 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff630003520 con 0x7ff64c06f300 2024-04-25T11:21:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.834+0000 7ff64a800640 1 -- 172.21.15.7:0/2159307436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06dc90 msgr2=0x7ff64c0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.834+0000 7ff64a800640 1 --2- 172.21.15.7:0/2159307436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06dc90 0x7ff64c0afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.835+0000 7ff64a800640 1 -- 172.21.15.7:0/2159307436 >> v1:172.21.15.7:6789/0 conn(0x7ff64c06f300 legacy=0x7ff64c0ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.835+0000 7ff64a800640 1 -- 172.21.15.7:0/2159307436 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c0b5220 con 0x7ff64c160a30 2024-04-25T11:21:19.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.835+0000 7ff649e00640 1 --2- 172.21.15.7:0/2159307436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06dc90 0x7ff64c0afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:21:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.836+0000 7ff64a800640 1 --2- 172.21.15.7:0/2159307436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff64c160a30 0x7ff64c0b00c0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff644009d90 tx=0x7ff64402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f6aafd9ccde174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:19.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.837+0000 7ff649400640 1 -- 172.21.15.7:0/2159307436 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644007740 con 0x7ff64c160a30 2024-04-25T11:21:19.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.837+0000 7ff649400640 1 -- 172.21.15.7:0/2159307436 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6440078e0 con 0x7ff64c160a30 2024-04-25T11:21:19.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.837+0000 7ff649400640 1 -- 172.21.15.7:0/2159307436 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff644007c90 con 0x7ff64c160a30 2024-04-25T11:21:19.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.837+0000 7ff650c00640 1 -- 172.21.15.7:0/2159307436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff64c160a30 msgr2=0x7ff64c0b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:19.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.837+0000 7ff650c00640 1 --2- 172.21.15.7:0/2159307436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff64c160a30 0x7ff64c0b00c0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff644009d90 tx=0x7ff64402a660 comp rx=0 tx=0).stop 2024-04-25T11:21:19.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.837+0000 7ff650c00640 1 -- 172.21.15.7:0/2159307436 shutdown_connections 2024-04-25T11:21:19.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.837+0000 7ff650c00640 1 --2- 172.21.15.7:0/2159307436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff64c160a30 0x7ff64c0b00c0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.837+0000 7ff650c00640 1 --2- 172.21.15.7:0/2159307436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06dc90 0x7ff64c0afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.837+0000 7ff650c00640 1 -- 172.21.15.7:0/2159307436 >> 172.21.15.7:0/2159307436 conn(0x7ff64c074200 msgr2=0x7ff64c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:19.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.838+0000 7ff650c00640 1 -- 172.21.15.7:0/2159307436 shutdown_connections 2024-04-25T11:21:19.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.838+0000 7ff650c00640 1 -- 172.21.15.7:0/2159307436 wait complete. 2024-04-25T11:21:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.840+0000 7ff650c00640 1 Processor -- start 2024-04-25T11:21:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.840+0000 7ff650c00640 1 -- start start 2024-04-25T11:21:19.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.842+0000 7ff650c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c06f300 0x7ff64c1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.842+0000 7ff650c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff64c160a30 0x7ff64c1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.842+0000 7ff650c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c158f30 0x7ff64c155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.842+0000 7ff650c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c0b6a30 con 0x7ff64c160a30 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.842+0000 7ff650c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c15d220 con 0x7ff64c06f300 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.842+0000 7ff650c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff64c0ce920 con 0x7ff64c158f30 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.842+0000 7ff641400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff64c160a30 0x7ff64c1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.842+0000 7ff641400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff64c160a30 0x7ff64c1589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37708/0 (socket says 172.21.15.7:37708) 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.842+0000 7ff641400640 1 -- 172.21.15.7:0/1208670934 learned_addr learned my addr 172.21.15.7:0/1208670934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.842+0000 7ff649e00640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c06f300 0x7ff64c1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.842+0000 7ff64a800640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c158f30 0x7ff64c155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.843+0000 7ff641400640 1 -- 172.21.15.7:0/1208670934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c158f30 msgr2=0x7ff64c155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.843+0000 7ff641400640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c158f30 0x7ff64c155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.843+0000 7ff641400640 1 -- 172.21.15.7:0/1208670934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c06f300 msgr2=0x7ff64c1428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:19.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.843+0000 7ff641400640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c06f300 0x7ff64c1428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:19.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.843+0000 7ff641400640 1 -- 172.21.15.7:0/1208670934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c16a410 con 0x7ff64c160a30 2024-04-25T11:21:19.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.843+0000 7ff641400640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff64c160a30 0x7ff64c1589f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c3c0 tx=0x7ff634001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.843+0000 7ff642a00640 1 -- 172.21.15.7:0/1208670934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634017070 con 0x7ff64c160a30 2024-04-25T11:21:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.843+0000 7ff642a00640 1 -- 172.21.15.7:0/1208670934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff634005ce0 con 0x7ff64c160a30 2024-04-25T11:21:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.843+0000 7ff642a00640 1 -- 172.21.15.7:0/1208670934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634007310 con 0x7ff64c160a30 2024-04-25T11:21:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.843+0000 7ff650c00640 1 -- 172.21.15.7:0/1208670934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c169b70 con 0x7ff64c160a30 2024-04-25T11:21:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.844+0000 7ff650c00640 1 -- 172.21.15.7:0/1208670934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c16a130 con 0x7ff64c160a30 2024-04-25T11:21:19.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.845+0000 7ff650c00640 1 -- 172.21.15.7:0/1208670934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64c06a100 con 0x7ff64c160a30 2024-04-25T11:21:19.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.846+0000 7ff642a00640 1 -- 172.21.15.7:0/1208670934 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff63401a070 con 0x7ff64c160a30 2024-04-25T11:21:19.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.848+0000 7ff642a00640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6280744b0 0x7ff6280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:19.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.848+0000 7ff642a00640 1 -- 172.21.15.7:0/1208670934 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7ff6340947a0 con 0x7ff64c160a30 2024-04-25T11:21:19.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.848+0000 7ff649e00640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6280744b0 0x7ff6280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:19.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.849+0000 7ff649e00640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6280744b0 0x7ff6280769c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff63c006fd0 tx=0x7ff63c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:19.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:19.853+0000 7ff642a00640 1 -- 172.21.15.7:0/1208670934 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff63405ed00 con 0x7ff64c160a30 2024-04-25T11:21:20.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.050+0000 7ff650c00640 1 -- 172.21.15.7:0/1208670934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff64c070fb0 con 0x7ff64c160a30 2024-04-25T11:21:20.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.051+0000 7ff642a00640 1 -- 172.21.15.7:0/1208670934 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+16804 (secure 0 0 0) 0x7ff63405e600 con 0x7ff64c160a30 2024-04-25T11:21:20.049 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:20.049 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":130,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:20:44.396251+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:20:41.047420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T11:20:41.908907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"130","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:20.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.054+0000 7ff650c00640 1 -- 172.21.15.7:0/1208670934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6280744b0 msgr2=0x7ff6280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.054+0000 7ff650c00640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6280744b0 0x7ff6280769c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff63c006fd0 tx=0x7ff63c008040 comp rx=0 tx=0).stop 2024-04-25T11:21:20.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.054+0000 7ff650c00640 1 -- 172.21.15.7:0/1208670934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff64c160a30 msgr2=0x7ff64c1589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.054+0000 7ff650c00640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff64c160a30 0x7ff64c1589f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c3c0 tx=0x7ff634001e70 comp rx=0 tx=0).stop 2024-04-25T11:21:20.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.054+0000 7ff650c00640 1 -- 172.21.15.7:0/1208670934 shutdown_connections 2024-04-25T11:21:20.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.054+0000 7ff650c00640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff6280744b0 0x7ff6280769c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.054+0000 7ff650c00640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c158f30 0x7ff64c155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.054+0000 7ff650c00640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff64c160a30 0x7ff64c1589f0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.054+0000 7ff650c00640 1 --2- 172.21.15.7:0/1208670934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c06f300 0x7ff64c1428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.054+0000 7ff650c00640 1 -- 172.21.15.7:0/1208670934 >> 172.21.15.7:0/1208670934 conn(0x7ff64c074200 msgr2=0x7ff64c14d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:20.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.055+0000 7ff650c00640 1 -- 172.21.15.7:0/1208670934 shutdown_connections 2024-04-25T11:21:20.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.055+0000 7ff650c00640 1 -- 172.21.15.7:0/1208670934 wait complete. 2024-04-25T11:21:20.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:21:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.366+0000 7f3e0f200640 1 Processor -- start 2024-04-25T11:21:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.366+0000 7f3e0f200640 1 -- start start 2024-04-25T11:21:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.368+0000 7f3e0f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e0806d580 0x7f3e0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.368+0000 7f3e0f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3e0806f1e0 0x7f3e0806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.368+0000 7f3e0f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e0806ded0 con 0x7f3e0815f180 2024-04-25T11:21:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.368+0000 7f3e0f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3e0815f590 con 0x7f3e0806d580 2024-04-25T11:21:20.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.368+0000 7f3e0f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3e08158ae0 con 0x7f3e0806f1e0 2024-04-25T11:21:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.368+0000 7f3e07400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e0806d580 0x7f3e0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.368+0000 7f3e07e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3e0815f180 0x7f3e08161c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58458/0 (socket says 172.21.15.7:58458) 2024-04-25T11:21:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.368+0000 7f3e07e00640 1 -- 172.21.15.7:0/2607386508 learned_addr learned my addr 172.21.15.7:0/2607386508 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:20.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.369+0000 7f3e0d200640 1 --2- 172.21.15.7:0/2607386508 >> v2:172.21.15.87:3300/0 conn(0x7f3e0806f1e0 0x7f3e0806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.369+0000 7f3e07400640 1 -- 172.21.15.7:0/2607386508 >> v2:172.21.15.87:3300/0 conn(0x7f3e0806f1e0 msgr2=0x7f3e0806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.369+0000 7f3e07400640 1 --2- 172.21.15.7:0/2607386508 >> v2:172.21.15.87:3300/0 conn(0x7f3e0806f1e0 0x7f3e0806f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.369+0000 7f3e07400640 1 -- 172.21.15.7:0/2607386508 >> v1:172.21.15.7:6789/0 conn(0x7f3e0815f180 legacy=0x7f3e08161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.369+0000 7f3e07400640 1 -- 172.21.15.7:0/2607386508 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e08162700 con 0x7f3e0806d580 2024-04-25T11:21:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.369+0000 7f3e0d200640 1 --2- 172.21.15.7:0/2607386508 >> v2:172.21.15.87:3300/0 conn(0x7f3e0806f1e0 0x7f3e0806f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:20.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.370+0000 7f3e07400640 1 --2- 172.21.15.7:0/2607386508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0806d580 0x7f3e0806d990 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f3e0815f750 tx=0x7f3df402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afd7cc2c41e992d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.370+0000 7f3e06a00640 1 -- 172.21.15.7:0/2607386508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4037070 con 0x7f3e0806d580 2024-04-25T11:21:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.370+0000 7f3e06a00640 1 -- 172.21.15.7:0/2607386508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df402ae50 con 0x7f3e0806d580 2024-04-25T11:21:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.370+0000 7f3e06a00640 1 -- 172.21.15.7:0/2607386508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4032660 con 0x7f3e0806d580 2024-04-25T11:21:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.371+0000 7f3e0f200640 1 -- 172.21.15.7:0/2607386508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0806d580 msgr2=0x7f3e0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.371+0000 7f3e0f200640 1 --2- 172.21.15.7:0/2607386508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0806d580 0x7f3e0806d990 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f3e0815f750 tx=0x7f3df402a140 comp rx=0 tx=0).stop 2024-04-25T11:21:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.371+0000 7f3e0f200640 1 -- 172.21.15.7:0/2607386508 shutdown_connections 2024-04-25T11:21:20.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.371+0000 7f3e0f200640 1 --2- 172.21.15.7:0/2607386508 >> v2:172.21.15.87:3300/0 conn(0x7f3e0806f1e0 0x7f3e0806f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.371+0000 7f3e0f200640 1 --2- 172.21.15.7:0/2607386508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0806d580 0x7f3e0806d990 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.371+0000 7f3e0f200640 1 -- 172.21.15.7:0/2607386508 >> 172.21.15.7:0/2607386508 conn(0x7f3e080732b0 msgr2=0x7f3e080736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:20.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.371+0000 7f3e0f200640 1 -- 172.21.15.7:0/2607386508 shutdown_connections 2024-04-25T11:21:20.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.371+0000 7f3e0f200640 1 -- 172.21.15.7:0/2607386508 wait complete. 2024-04-25T11:21:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.373+0000 7f3e0f200640 1 Processor -- start 2024-04-25T11:21:20.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.374+0000 7f3e0f200640 1 -- start start 2024-04-25T11:21:20.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.375+0000 7f3e0f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e0806d580 0x7f3e08154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:20.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.375+0000 7f3e0f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0806f1e0 0x7f3e080b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:20.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.375+0000 7f3e0f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e0815f180 0x7f3e080b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:20.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.375+0000 7f3e0f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0815ec60 con 0x7f3e0806f1e0 2024-04-25T11:21:20.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.375+0000 7f3e0f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0815e980 con 0x7f3e0815f180 2024-04-25T11:21:20.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.375+0000 7f3e0f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3e0815e670 con 0x7f3e0806d580 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.375+0000 7f3e07400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0806f1e0 0x7f3e080b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.375+0000 7f3e07400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0806f1e0 0x7f3e080b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36134/0 (socket says 172.21.15.7:36134) 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.375+0000 7f3e07400640 1 -- 172.21.15.7:0/4049827900 learned_addr learned my addr 172.21.15.7:0/4049827900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.376+0000 7f3e07e00640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e0806d580 0x7f3e08154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.376+0000 7f3e0d200640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e0815f180 0x7f3e080b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.376+0000 7f3e07400640 1 -- 172.21.15.7:0/4049827900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e0806d580 msgr2=0x7f3e08154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.376+0000 7f3e07400640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e0806d580 0x7f3e08154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.376+0000 7f3e07400640 1 -- 172.21.15.7:0/4049827900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e0815f180 msgr2=0x7f3e080b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.376+0000 7f3e07400640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e0815f180 0x7f3e080b6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.376+0000 7f3e07400640 1 -- 172.21.15.7:0/4049827900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e081f6a50 con 0x7f3e0806f1e0 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.376+0000 7f3e07400640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0806f1e0 0x7f3e080b5c50 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f3df4035040 tx=0x7f3df40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.377+0000 7f3dffe00640 1 -- 172.21.15.7:0/4049827900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4041020 con 0x7f3e0806f1e0 2024-04-25T11:21:20.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.377+0000 7f3dffe00640 1 -- 172.21.15.7:0/4049827900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df403c670 con 0x7f3e0806f1e0 2024-04-25T11:21:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.377+0000 7f3dffe00640 1 -- 172.21.15.7:0/4049827900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4037040 con 0x7f3e0806f1e0 2024-04-25T11:21:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.377+0000 7f3e0f200640 1 -- 172.21.15.7:0/4049827900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e081f6d30 con 0x7f3e0806f1e0 2024-04-25T11:21:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.377+0000 7f3e0f200640 1 -- 172.21.15.7:0/4049827900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e081f72f0 con 0x7f3e0806f1e0 2024-04-25T11:21:20.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.378+0000 7f3e0f200640 1 -- 172.21.15.7:0/4049827900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e08050740 con 0x7f3e0806f1e0 2024-04-25T11:21:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.380+0000 7f3dffe00640 1 -- 172.21.15.7:0/4049827900 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3df4044020 con 0x7f3e0806f1e0 2024-04-25T11:21:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.381+0000 7f3dffe00640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3de80744b0 0x7f3de80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.381+0000 7f3dffe00640 1 -- 172.21.15.7:0/4049827900 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 6101+0+0 (secure 0 0 0) 0x7f3df40b5be0 con 0x7f3e0806f1e0 2024-04-25T11:21:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.381+0000 7f3e07e00640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3de80744b0 0x7f3de80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:20.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.382+0000 7f3e07e00640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3de80744b0 0x7f3de80769c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000f60 tx=0x7f3df8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:20.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.386+0000 7f3dffe00640 1 -- 172.21.15.7:0/4049827900 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3df404b030 con 0x7f3e0806f1e0 2024-04-25T11:21:20.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.632+0000 7f3e0f200640 1 -- 172.21.15.7:0/4049827900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3e08074250 con 0x7f3e0806f1e0 2024-04-25T11:21:20.630 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-04-25T11:21:20.633+0000 7fac34e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:21:20.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.637+0000 7f3dffe00640 1 -- 172.21.15.7:0/4049827900 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) ==== 161+0+0 (secure 0 0 0) 0x7f3df40801e0 con 0x7f3e0806f1e0 2024-04-25T11:21:20.635 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:21:20.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 -- 172.21.15.7:0/4049827900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3de80744b0 msgr2=0x7f3de80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3de80744b0 0x7f3de80769c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000f60 tx=0x7f3df8009040 comp rx=0 tx=0).stop 2024-04-25T11:21:20.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 -- 172.21.15.7:0/4049827900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0806f1e0 msgr2=0x7f3e080b5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0806f1e0 0x7f3e080b5c50 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f3df4035040 tx=0x7f3df40046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 -- 172.21.15.7:0/4049827900 shutdown_connections 2024-04-25T11:21:20.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3de80744b0 0x7f3de80769c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e0815f180 0x7f3e080b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e0806f1e0 0x7f3e080b5c50 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 --2- 172.21.15.7:0/4049827900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3e0806d580 0x7f3e08154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 -- 172.21.15.7:0/4049827900 >> 172.21.15.7:0/4049827900 conn(0x7f3e080732b0 msgr2=0x7f3e080b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:20.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 -- 172.21.15.7:0/4049827900 shutdown_connections 2024-04-25T11:21:20.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.641+0000 7f3e0f200640 1 -- 172.21.15.7:0/4049827900 wait complete. 2024-04-25T11:21:20.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:21:20.839 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:21:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.952+0000 7f494fe00640 1 Processor -- start 2024-04-25T11:21:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.952+0000 7f494fe00640 1 -- start start 2024-04-25T11:21:20.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.954+0000 7f494fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49500cff00 0x7f49500d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:20.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.954+0000 7f494fe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49500d09e0 con 0x7f4950167de0 2024-04-25T11:21:20.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.954+0000 7f494fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49500d0cc0 con 0x7f495006f330 2024-04-25T11:21:20.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.954+0000 7f494fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f495006eea0 con 0x7f49500cff00 2024-04-25T11:21:20.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.954+0000 7f4947e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4950167de0 0x7f49501681f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58464/0 (socket says 172.21.15.7:58464) 2024-04-25T11:21:20.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.954+0000 7f4947e00640 1 -- 172.21.15.7:0/1403673415 learned_addr learned my addr 172.21.15.7:0/1403673415 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:20.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.955+0000 7f494ea00640 1 --2- 172.21.15.7:0/1403673415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49500cff00 0x7f49500d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:20.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.955+0000 7f494e000640 1 -- 172.21.15.7:0/1403673415 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2209029844 0 0) 0x7f49500d09e0 con 0x7f4950167de0 2024-04-25T11:21:20.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.955+0000 7f494e000640 1 -- 172.21.15.7:0/1403673415 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4930003520 con 0x7f4950167de0 2024-04-25T11:21:20.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.955+0000 7f494e000640 1 -- 172.21.15.7:0/1403673415 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 459389145 0 0) 0x7f49500d0cc0 con 0x7f495006f330 2024-04-25T11:21:20.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.955+0000 7f494e000640 1 -- 172.21.15.7:0/1403673415 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49500d09e0 con 0x7f495006f330 2024-04-25T11:21:20.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.955+0000 7f494ea00640 1 -- 172.21.15.7:0/1403673415 >> v1:172.21.15.87:6789/0 conn(0x7f495006f330 legacy=0x7f495006f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.955+0000 7f494ea00640 1 -- 172.21.15.7:0/1403673415 >> v1:172.21.15.7:6789/0 conn(0x7f4950167de0 legacy=0x7f49501681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.955+0000 7f494ea00640 1 -- 172.21.15.7:0/1403673415 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49500d0fa0 con 0x7f49500cff00 2024-04-25T11:21:20.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.957+0000 7f494ea00640 1 --2- 172.21.15.7:0/1403673415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49500cff00 0x7f49500d0310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49400097d0 tx=0x7f494002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ee473d9955ccf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:20.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.957+0000 7f494e000640 1 -- 172.21.15.7:0/1403673415 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940037070 con 0x7f49500cff00 2024-04-25T11:21:20.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.957+0000 7f494e000640 1 -- 172.21.15.7:0/1403673415 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4940032440 con 0x7f49500cff00 2024-04-25T11:21:20.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.957+0000 7f494e000640 1 -- 172.21.15.7:0/1403673415 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940032810 con 0x7f49500cff00 2024-04-25T11:21:20.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.958+0000 7f494fe00640 1 -- 172.21.15.7:0/1403673415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49500cff00 msgr2=0x7f49500d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.958+0000 7f494fe00640 1 --2- 172.21.15.7:0/1403673415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49500cff00 0x7f49500d0310 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49400097d0 tx=0x7f494002a210 comp rx=0 tx=0).stop 2024-04-25T11:21:20.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.958+0000 7f494fe00640 1 -- 172.21.15.7:0/1403673415 shutdown_connections 2024-04-25T11:21:20.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.958+0000 7f494fe00640 1 --2- 172.21.15.7:0/1403673415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49500cff00 0x7f49500d0310 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.958+0000 7f494fe00640 1 -- 172.21.15.7:0/1403673415 >> 172.21.15.7:0/1403673415 conn(0x7f49500740f0 msgr2=0x7f495006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.958+0000 7f494fe00640 1 -- 172.21.15.7:0/1403673415 shutdown_connections 2024-04-25T11:21:20.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.958+0000 7f494fe00640 1 -- 172.21.15.7:0/1403673415 wait complete. 2024-04-25T11:21:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.961+0000 7f494fe00640 1 Processor -- start 2024-04-25T11:21:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.961+0000 7f494fe00640 1 -- start start 2024-04-25T11:21:20.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495006f330 0x7f49501fa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49500cff00 0x7f4950147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4950167de0 0x7f49500b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4950168bb0 con 0x7f495006f330 2024-04-25T11:21:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4950168a30 con 0x7f49500cff00 2024-04-25T11:21:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4950147340 con 0x7f4950167de0 2024-04-25T11:21:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495006f330 0x7f49501fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495006f330 0x7f49501fa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36140/0 (socket says 172.21.15.7:36140) 2024-04-25T11:21:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494ea00640 1 -- 172.21.15.7:0/1249108209 learned_addr learned my addr 172.21.15.7:0/1249108209 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:20.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f4947e00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49500cff00 0x7f4950147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:20.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494f400640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4950167de0 0x7f49500b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:20.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494ea00640 1 -- 172.21.15.7:0/1249108209 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4950167de0 msgr2=0x7f49500b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494ea00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4950167de0 0x7f49500b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494ea00640 1 -- 172.21.15.7:0/1249108209 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49500cff00 msgr2=0x7f4950147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:20.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494ea00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49500cff00 0x7f4950147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.963+0000 7f494ea00640 1 -- 172.21.15.7:0/1249108209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4950149590 con 0x7f495006f330 2024-04-25T11:21:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.964+0000 7f494ea00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495006f330 0x7f49501fa4d0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f49400098f0 tx=0x7f49400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:20.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.964+0000 7f4947400640 1 -- 172.21.15.7:0/1249108209 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940037050 con 0x7f495006f330 2024-04-25T11:21:20.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.964+0000 7f4947400640 1 -- 172.21.15.7:0/1249108209 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4940043070 con 0x7f495006f330 2024-04-25T11:21:20.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.964+0000 7f4947400640 1 -- 172.21.15.7:0/1249108209 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494003cb10 con 0x7f495006f330 2024-04-25T11:21:20.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.964+0000 7f494fe00640 1 -- 172.21.15.7:0/1249108209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49501484f0 con 0x7f495006f330 2024-04-25T11:21:20.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.964+0000 7f494fe00640 1 -- 172.21.15.7:0/1249108209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4950148ab0 con 0x7f495006f330 2024-04-25T11:21:20.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.966+0000 7f494fe00640 1 -- 172.21.15.7:0/1249108209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495006dec0 con 0x7f495006f330 2024-04-25T11:21:20.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.967+0000 7f4947400640 1 -- 172.21.15.7:0/1249108209 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f494003ccb0 con 0x7f495006f330 2024-04-25T11:21:20.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.968+0000 7f4947400640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f492c0744e0 0x7f492c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:20.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.968+0000 7f4947400640 1 -- 172.21.15.7:0/1249108209 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 6187+0+0 (secure 0 0 0) 0x7f49400b60d0 con 0x7f495006f330 2024-04-25T11:21:20.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.968+0000 7f4947e00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f492c0744e0 0x7f492c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:20.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.969+0000 7f4947e00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f492c0744e0 0x7f492c0769f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4950143620 tx=0x7f4938009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:20.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:20.973+0000 7f4947400640 1 -- 172.21.15.7:0/1249108209 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f495006dec0 con 0x7f495006f330 2024-04-25T11:21:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.220+0000 7f494fe00640 1 -- 172.21.15.7:0/1249108209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4950050740 con 0x7f495006f330 2024-04-25T11:21:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.642+0000 7f4947400640 1 -- 172.21.15.7:0/1249108209 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) ==== 104+0+0 (secure 0 0 0) 0x7f49400805f0 con 0x7f495006f330 2024-04-25T11:21:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.646+0000 7f494fe00640 1 -- 172.21.15.7:0/1249108209 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f492c0744e0 msgr2=0x7f492c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.646+0000 7f494fe00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f492c0744e0 0x7f492c0769f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4950143620 tx=0x7f4938009040 comp rx=0 tx=0).stop 2024-04-25T11:21:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.646+0000 7f494fe00640 1 -- 172.21.15.7:0/1249108209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495006f330 msgr2=0x7f49501fa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.646+0000 7f494fe00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495006f330 0x7f49501fa4d0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f49400098f0 tx=0x7f49400046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:21.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.646+0000 7f494fe00640 1 -- 172.21.15.7:0/1249108209 shutdown_connections 2024-04-25T11:21:21.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.646+0000 7f494fe00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f492c0744e0 0x7f492c0769f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:21.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.646+0000 7f494fe00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4950167de0 0x7f49500b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:21.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.646+0000 7f494fe00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49500cff00 0x7f4950147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.646+0000 7f494fe00640 1 --2- 172.21.15.7:0/1249108209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f495006f330 0x7f49501fa4d0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.646+0000 7f494fe00640 1 -- 172.21.15.7:0/1249108209 >> 172.21.15.7:0/1249108209 conn(0x7f49500740f0 msgr2=0x7f4950165990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.647+0000 7f494fe00640 1 -- 172.21.15.7:0/1249108209 shutdown_connections 2024-04-25T11:21:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.647+0000 7f494fe00640 1 -- 172.21.15.7:0/1249108209 wait complete. 2024-04-25T11:21:21.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:21:21.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.966+0000 7fd431c00640 1 Processor -- start 2024-04-25T11:21:21.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.966+0000 7fd431c00640 1 -- start start 2024-04-25T11:21:21.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.968+0000 7fd431c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd42c0b48a0 0x7fd42c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:21.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.968+0000 7fd431c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd42c0aec60 0x7fd42c0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:21.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.968+0000 7fd431c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd42c0b4da0 con 0x7fd42c0b0940 2024-04-25T11:21:21.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.968+0000 7fd431c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fd42c0b5080 con 0x7fd42c0aec60 2024-04-25T11:21:21.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.968+0000 7fd431c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd42c0b5360 con 0x7fd42c0b48a0 2024-04-25T11:21:21.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.968+0000 7fd42a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd42c0aec60 0x7fd42c0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:21.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.969+0000 7fd42a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd42c0aec60 0x7fd42c0b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36154/0 (socket says 172.21.15.7:36154) 2024-04-25T11:21:21.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.969+0000 7fd42a000640 1 -- 172.21.15.7:0/1773911464 learned_addr learned my addr 172.21.15.7:0/1773911464 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:21.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.969+0000 7fd42a000640 1 -- 172.21.15.7:0/1773911464 >> v2:172.21.15.87:3300/0 conn(0x7fd42c0b48a0 msgr2=0x7fd42c15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:21.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.969+0000 7fd42a000640 1 --2- 172.21.15.7:0/1773911464 >> v2:172.21.15.87:3300/0 conn(0x7fd42c0b48a0 0x7fd42c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:21.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.969+0000 7fd42a000640 1 -- 172.21.15.7:0/1773911464 >> v1:172.21.15.87:6789/0 conn(0x7fd42c0b0940 legacy=0x7fd42c0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:21.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.969+0000 7fd42a000640 1 -- 172.21.15.7:0/1773911464 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42c0b5640 con 0x7fd42c0aec60 2024-04-25T11:21:21.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.970+0000 7fd42a000640 1 --2- 172.21.15.7:0/1773911464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0aec60 0x7fd42c0b5e40 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fd42c071390 tx=0x7fd41402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398911bf45b4de2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.971+0000 7fd429600640 1 -- 172.21.15.7:0/1773911464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414037070 con 0x7fd42c0aec60 2024-04-25T11:21:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.971+0000 7fd429600640 1 -- 172.21.15.7:0/1773911464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd414032440 con 0x7fd42c0aec60 2024-04-25T11:21:21.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.971+0000 7fd429600640 1 -- 172.21.15.7:0/1773911464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414032810 con 0x7fd42c0aec60 2024-04-25T11:21:21.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.972+0000 7fd431c00640 1 -- 172.21.15.7:0/1773911464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0aec60 msgr2=0x7fd42c0b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:21.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.972+0000 7fd431c00640 1 --2- 172.21.15.7:0/1773911464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0aec60 0x7fd42c0b5e40 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fd42c071390 tx=0x7fd41402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:21:21.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.972+0000 7fd431c00640 1 -- 172.21.15.7:0/1773911464 shutdown_connections 2024-04-25T11:21:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.972+0000 7fd431c00640 1 --2- 172.21.15.7:0/1773911464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0aec60 0x7fd42c0b5e40 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.972+0000 7fd431c00640 1 --2- 172.21.15.7:0/1773911464 >> v2:172.21.15.87:3300/0 conn(0x7fd42c0b48a0 0x7fd42c15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.972+0000 7fd431c00640 1 -- 172.21.15.7:0/1773911464 >> 172.21.15.7:0/1773911464 conn(0x7fd42c073920 msgr2=0x7fd42c073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.972+0000 7fd431c00640 1 -- 172.21.15.7:0/1773911464 shutdown_connections 2024-04-25T11:21:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.972+0000 7fd431c00640 1 -- 172.21.15.7:0/1773911464 wait complete. 2024-04-25T11:21:21.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.974+0000 7fd431c00640 1 Processor -- start 2024-04-25T11:21:21.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.974+0000 7fd431c00640 1 -- start start 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.976+0000 7fd431c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0aec60 0x7fd42c154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.976+0000 7fd431c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd42c0b0940 0x7fd42c14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.976+0000 7fd431c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd42c0b48a0 0x7fd42c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.976+0000 7fd431c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c0b6380 con 0x7fd42c0aec60 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.976+0000 7fd431c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c15e1d0 con 0x7fd42c0b0940 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.976+0000 7fd431c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd42c0b03b0 con 0x7fd42c0b48a0 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.976+0000 7fd42aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0aec60 0x7fd42c154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.976+0000 7fd42aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0aec60 0x7fd42c154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36160/0 (socket says 172.21.15.7:36160) 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.976+0000 7fd42aa00640 1 -- 172.21.15.7:0/1552004848 learned_addr learned my addr 172.21.15.7:0/1552004848 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.977+0000 7fd42aa00640 1 -- 172.21.15.7:0/1552004848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd42c0b48a0 msgr2=0x7fd42c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.977+0000 7fd42aa00640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd42c0b48a0 0x7fd42c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.977+0000 7fd42aa00640 1 -- 172.21.15.7:0/1552004848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd42c0b0940 msgr2=0x7fd42c14c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:21.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.977+0000 7fd42aa00640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd42c0b0940 0x7fd42c14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.977+0000 7fd42aa00640 1 -- 172.21.15.7:0/1552004848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42c0b0d50 con 0x7fd42c0aec60 2024-04-25T11:21:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.977+0000 7fd42aa00640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0aec60 0x7fd42c154080 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00b9f0 tx=0x7fd41c00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.977+0000 7fd422a00640 1 -- 172.21.15.7:0/1552004848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c019070 con 0x7fd42c0aec60 2024-04-25T11:21:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.977+0000 7fd422a00640 1 -- 172.21.15.7:0/1552004848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41c00cb70 con 0x7fd42c0aec60 2024-04-25T11:21:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.977+0000 7fd422a00640 1 -- 172.21.15.7:0/1552004848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c014440 con 0x7fd42c0aec60 2024-04-25T11:21:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.977+0000 7fd431c00640 1 -- 172.21.15.7:0/1552004848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42c0b1030 con 0x7fd42c0aec60 2024-04-25T11:21:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.977+0000 7fd431c00640 1 -- 172.21.15.7:0/1552004848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42c0b15f0 con 0x7fd42c0aec60 2024-04-25T11:21:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.979+0000 7fd431c00640 1 -- 172.21.15.7:0/1552004848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd42c06a100 con 0x7fd42c0aec60 2024-04-25T11:21:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.980+0000 7fd422a00640 1 -- 172.21.15.7:0/1552004848 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd41c01c070 con 0x7fd42c0aec60 2024-04-25T11:21:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.981+0000 7fd422a00640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4080744e0 0x7fd4080769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.981+0000 7fd422a00640 1 -- 172.21.15.7:0/1552004848 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 6187+0+0 (secure 0 0 0) 0x7fd41c095170 con 0x7fd42c0aec60 2024-04-25T11:21:21.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.981+0000 7fd42a000640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4080744e0 0x7fd4080769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.982+0000 7fd42a000640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4080744e0 0x7fd4080769f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd414034000 tx=0x7fd41402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:21.986+0000 7fd422a00640 1 -- 172.21.15.7:0/1552004848 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd41c05f720 con 0x7fd42c0aec60 2024-04-25T11:21:22.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.189+0000 7fd431c00640 1 -- 172.21.15.7:0/1552004848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd42c0afd70 con 0x7fd42c0aec60 2024-04-25T11:21:22.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.657+0000 7fd422a00640 1 -- 172.21.15.7:0/1552004848 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v132) ==== 181+0+0 (secure 0 0 0) 0x7fd41c05f020 con 0x7fd42c0aec60 2024-04-25T11:21:22.654 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:21:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.661+0000 7fd431c00640 1 -- 172.21.15.7:0/1552004848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4080744e0 msgr2=0x7fd4080769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.661+0000 7fd431c00640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4080744e0 0x7fd4080769f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd414034000 tx=0x7fd41402c000 comp rx=0 tx=0).stop 2024-04-25T11:21:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.661+0000 7fd431c00640 1 -- 172.21.15.7:0/1552004848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0aec60 msgr2=0x7fd42c154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:22.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.661+0000 7fd431c00640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0aec60 0x7fd42c154080 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fd41c00b9f0 tx=0x7fd41c00beb0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.662+0000 7fd431c00640 1 -- 172.21.15.7:0/1552004848 shutdown_connections 2024-04-25T11:21:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.662+0000 7fd431c00640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4080744e0 0x7fd4080769f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.662+0000 7fd431c00640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd42c0b48a0 0x7fd42c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.662+0000 7fd431c00640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd42c0b0940 0x7fd42c14c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.662+0000 7fd431c00640 1 --2- 172.21.15.7:0/1552004848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c0aec60 0x7fd42c154080 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.662+0000 7fd431c00640 1 -- 172.21.15.7:0/1552004848 >> 172.21.15.7:0/1552004848 conn(0x7fd42c073920 msgr2=0x7fd42c156b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.662+0000 7fd431c00640 1 -- 172.21.15.7:0/1552004848 shutdown_connections 2024-04-25T11:21:22.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.662+0000 7fd431c00640 1 -- 172.21.15.7:0/1552004848 wait complete. 2024-04-25T11:21:22.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:21:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.987+0000 7f876f600640 1 Processor -- start 2024-04-25T11:21:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.987+0000 7f876f600640 1 -- start start 2024-04-25T11:21:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.989+0000 7f876f600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87680b48a0 0x7f876815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.989+0000 7f876f600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87680aec60 0x7f87680b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.989+0000 7f876f600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f87680b0940 0x7f87680b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.989+0000 7f876f600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f87680b1290 con 0x7f87680b48a0 2024-04-25T11:21:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.989+0000 7f876f600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f87680b4da0 con 0x7f87680b0940 2024-04-25T11:21:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.989+0000 7f876f600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87680b5080 con 0x7f87680aec60 2024-04-25T11:21:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.989+0000 7f876cc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87680b48a0 0x7f876815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.990+0000 7f876cc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87680b48a0 0x7f876815dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36176/0 (socket says 172.21.15.7:36176) 2024-04-25T11:21:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.990+0000 7f876cc00640 1 -- 172.21.15.7:0/2012352657 learned_addr learned my addr 172.21.15.7:0/2012352657 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.990+0000 7f875fe00640 1 --2- 172.21.15.7:0/2012352657 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87680aec60 0x7f87680b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.990+0000 7f876cc00640 1 -- 172.21.15.7:0/2012352657 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87680aec60 msgr2=0x7f87680b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.990+0000 7f876d600640 1 --2- 172.21.15.7:0/2012352657 >> v2:172.21.15.87:3300/0 conn(0x7f87680b0940 0x7f87680b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.990+0000 7f876cc00640 1 --2- 172.21.15.7:0/2012352657 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87680aec60 0x7f87680b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.990+0000 7f876cc00640 1 -- 172.21.15.7:0/2012352657 >> v2:172.21.15.87:3300/0 conn(0x7f87680b0940 msgr2=0x7f87680b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.990+0000 7f876cc00640 1 --2- 172.21.15.7:0/2012352657 >> v2:172.21.15.87:3300/0 conn(0x7f87680b0940 0x7f87680b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.990+0000 7f876cc00640 1 -- 172.21.15.7:0/2012352657 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87680b5360 con 0x7f87680b48a0 2024-04-25T11:21:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.991+0000 7f876cc00640 1 --2- 172.21.15.7:0/2012352657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87680b48a0 0x7f876815dc90 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f8768074740 tx=0x7f875802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa7e433d0505e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.992+0000 7f8767e00640 1 -- 172.21.15.7:0/2012352657 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758037070 con 0x7f87680b48a0 2024-04-25T11:21:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.992+0000 7f8767e00640 1 -- 172.21.15.7:0/2012352657 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8758032440 con 0x7f87680b48a0 2024-04-25T11:21:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.992+0000 7f8767e00640 1 -- 172.21.15.7:0/2012352657 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758032810 con 0x7f87680b48a0 2024-04-25T11:21:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.992+0000 7f876f600640 1 -- 172.21.15.7:0/2012352657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87680b48a0 msgr2=0x7f876815dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.992+0000 7f876f600640 1 --2- 172.21.15.7:0/2012352657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87680b48a0 0x7f876815dc90 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f8768074740 tx=0x7f875802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.993+0000 7f876f600640 1 -- 172.21.15.7:0/2012352657 shutdown_connections 2024-04-25T11:21:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.993+0000 7f876f600640 1 --2- 172.21.15.7:0/2012352657 >> v2:172.21.15.87:3300/0 conn(0x7f87680b0940 0x7f87680b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.993+0000 7f876f600640 1 --2- 172.21.15.7:0/2012352657 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87680aec60 0x7f87680b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.993+0000 7f876f600640 1 --2- 172.21.15.7:0/2012352657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87680b48a0 0x7f876815dc90 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.993+0000 7f876f600640 1 -- 172.21.15.7:0/2012352657 >> 172.21.15.7:0/2012352657 conn(0x7f8768073920 msgr2=0x7f8768073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.993+0000 7f876f600640 1 -- 172.21.15.7:0/2012352657 shutdown_connections 2024-04-25T11:21:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.993+0000 7f876f600640 1 -- 172.21.15.7:0/2012352657 wait complete. 2024-04-25T11:21:22.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.995+0000 7f876f600640 1 Processor -- start 2024-04-25T11:21:22.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.995+0000 7f876f600640 1 -- start start 2024-04-25T11:21:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.997+0000 7f876f600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87680aec60 0x7f8768160b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.997+0000 7f876f600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87680b0940 0x7f8768145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.997+0000 7f876f600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87680b48a0 0x7f876815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.997+0000 7f876f600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f876815e1d0 con 0x7f87680b48a0 2024-04-25T11:21:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.997+0000 7f876f600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f87680b5920 con 0x7f87680b0940 2024-04-25T11:21:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.997+0000 7f876f600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87680b6380 con 0x7f87680aec60 2024-04-25T11:21:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.997+0000 7f876d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87680b48a0 0x7f876815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.997+0000 7f876d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87680b48a0 0x7f876815ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36188/0 (socket says 172.21.15.7:36188) 2024-04-25T11:21:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.997+0000 7f876d600640 1 -- 172.21.15.7:0/3163842424 learned_addr learned my addr 172.21.15.7:0/3163842424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.997+0000 7f876cc00640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87680aec60 0x7f8768160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f875fe00640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87680b0940 0x7f8768145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f876d600640 1 -- 172.21.15.7:0/3163842424 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87680aec60 msgr2=0x7f8768160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f876d600640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87680aec60 0x7f8768160b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f876d600640 1 -- 172.21.15.7:0/3163842424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87680b0940 msgr2=0x7f8768145f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f876d600640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87680b0940 0x7f8768145f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f876d600640 1 -- 172.21.15.7:0/3163842424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87681610c0 con 0x7f87680b48a0 2024-04-25T11:21:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f876d600640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87680b48a0 0x7f876815ebd0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f876000cc30 tx=0x7f876000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:23.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f8765600640 1 -- 172.21.15.7:0/3163842424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876000ce40 con 0x7f87680b48a0 2024-04-25T11:21:23.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f8765600640 1 -- 172.21.15.7:0/3163842424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8760003990 con 0x7f87680b48a0 2024-04-25T11:21:23.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f876f600640 1 -- 172.21.15.7:0/3163842424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876806f520 con 0x7f87680b48a0 2024-04-25T11:21:23.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f8765600640 1 -- 172.21.15.7:0/3163842424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760003d40 con 0x7f87680b48a0 2024-04-25T11:21:23.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.998+0000 7f876f600640 1 -- 172.21.15.7:0/3163842424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876806fae0 con 0x7f87680b48a0 2024-04-25T11:21:23.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:22.999+0000 7f876f600640 1 -- 172.21.15.7:0/3163842424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876806a100 con 0x7f87680b48a0 2024-04-25T11:21:23.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.001+0000 7f8765600640 1 -- 172.21.15.7:0/3163842424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8760003ee0 con 0x7f87680b48a0 2024-04-25T11:21:23.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.002+0000 7f8765600640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87440744b0 0x7f87440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:23.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.002+0000 7f8765600640 1 -- 172.21.15.7:0/3163842424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5749+0+0 (secure 0 0 0) 0x7f87600956b0 con 0x7f87680b48a0 2024-04-25T11:21:23.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.002+0000 7f876cc00640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87440744b0 0x7f87440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:23.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.003+0000 7f876cc00640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87440744b0 0x7f87440769c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8758034000 tx=0x7f875802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:23.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.007+0000 7f8765600640 1 -- 172.21.15.7:0/3163842424 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8760014030 con 0x7f87680b48a0 2024-04-25T11:21:23.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.212+0000 7f876f600640 1 -- 172.21.15.7:0/3163842424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f87680b6b30 con 0x7f87680b48a0 2024-04-25T11:21:23.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.671+0000 7f8765600640 1 -- 172.21.15.7:0/3163842424 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v133) ==== 169+0+0 (secure 0 0 0) 0x7f876005fd70 con 0x7f87680b48a0 2024-04-25T11:21:23.668 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-25T11:21:23.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.674+0000 7f876f600640 1 -- 172.21.15.7:0/3163842424 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87440744b0 msgr2=0x7f87440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:23.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.674+0000 7f876f600640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87440744b0 0x7f87440769c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8758034000 tx=0x7f875802e000 comp rx=0 tx=0).stop 2024-04-25T11:21:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.674+0000 7f876f600640 1 -- 172.21.15.7:0/3163842424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87680b48a0 msgr2=0x7f876815ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.674+0000 7f876f600640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87680b48a0 0x7f876815ebd0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f876000cc30 tx=0x7f876000a5c0 comp rx=0 tx=0).stop 2024-04-25T11:21:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.675+0000 7f876f600640 1 -- 172.21.15.7:0/3163842424 shutdown_connections 2024-04-25T11:21:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.675+0000 7f876f600640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87440744b0 0x7f87440769c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.675+0000 7f876f600640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87680b48a0 0x7f876815ebd0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.675+0000 7f876f600640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87680b0940 0x7f8768145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.675+0000 7f876f600640 1 --2- 172.21.15.7:0/3163842424 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87680aec60 0x7f8768160b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.675+0000 7f876f600640 1 -- 172.21.15.7:0/3163842424 >> 172.21.15.7:0/3163842424 conn(0x7f8768073920 msgr2=0x7f87680d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.675+0000 7f876f600640 1 -- 172.21.15.7:0/3163842424 shutdown_connections 2024-04-25T11:21:23.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.675+0000 7f876f600640 1 -- 172.21.15.7:0/3163842424 wait complete. 2024-04-25T11:21:23.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-04-25T11:21:23.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.992+0000 7f50afa00640 1 Processor -- start 2024-04-25T11:21:23.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.992+0000 7f50afa00640 1 -- start start 2024-04-25T11:21:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.994+0000 7f50afa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50a80d0ee0 0x7f50a80affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.994+0000 7f50afa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f50a80b5020 0x7f50a8160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.994+0000 7f50afa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a806f5b0 con 0x7f50a815aaa0 2024-04-25T11:21:23.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.994+0000 7f50afa00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f50a80af290 con 0x7f50a80b5020 2024-04-25T11:21:23.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.994+0000 7f50afa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f50a80d14a0 con 0x7f50a80d0ee0 2024-04-25T11:21:23.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.995+0000 7f50ada00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f50a80b5020 0x7f50a8160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:23.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.995+0000 7f50ada00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f50a80b5020 0x7f50a8160990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36204/0 (socket says 172.21.15.7:36204) 2024-04-25T11:21:23.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.995+0000 7f50ada00640 1 -- 172.21.15.7:0/1618350878 learned_addr learned my addr 172.21.15.7:0/1618350878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:23.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.995+0000 7f50a7e00640 1 --2- 172.21.15.7:0/1618350878 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50a80d0ee0 0x7f50a80affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:23.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.995+0000 7f50ada00640 1 -- 172.21.15.7:0/1618350878 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50a80d0ee0 msgr2=0x7f50a80affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:23.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.995+0000 7f50ada00640 1 --2- 172.21.15.7:0/1618350878 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50a80d0ee0 0x7f50a80affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:23.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.995+0000 7f50ada00640 1 -- 172.21.15.7:0/1618350878 >> v1:172.21.15.87:6789/0 conn(0x7f50a815aaa0 legacy=0x7f50a80b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:23.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.995+0000 7f50ada00640 1 -- 172.21.15.7:0/1618350878 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a80af4e0 con 0x7f50a80b5020 2024-04-25T11:21:23.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.995+0000 7f50a7e00640 1 --2- 172.21.15.7:0/1618350878 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50a80d0ee0 0x7f50a80affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.996+0000 7f50ada00640 1 --2- 172.21.15.7:0/1618350878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a80b5020 0x7f50a8160990 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009e50 tx=0x7f50a002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51841c4a2ae4d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.997+0000 7f50a7400640 1 -- 172.21.15.7:0/1618350878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0036070 con 0x7f50a80b5020 2024-04-25T11:21:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.997+0000 7f50a7400640 1 -- 172.21.15.7:0/1618350878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a0031600 con 0x7f50a80b5020 2024-04-25T11:21:23.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.997+0000 7f50a7400640 1 -- 172.21.15.7:0/1618350878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a00319d0 con 0x7f50a80b5020 2024-04-25T11:21:23.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.998+0000 7f50afa00640 1 -- 172.21.15.7:0/1618350878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a80b5020 msgr2=0x7f50a8160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:23.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.998+0000 7f50afa00640 1 --2- 172.21.15.7:0/1618350878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a80b5020 0x7f50a8160990 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009e50 tx=0x7f50a002a710 comp rx=0 tx=0).stop 2024-04-25T11:21:23.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.998+0000 7f50afa00640 1 -- 172.21.15.7:0/1618350878 shutdown_connections 2024-04-25T11:21:23.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.998+0000 7f50afa00640 1 --2- 172.21.15.7:0/1618350878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a80b5020 0x7f50a8160990 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:23.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.998+0000 7f50afa00640 1 --2- 172.21.15.7:0/1618350878 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50a80d0ee0 0x7f50a80affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:23.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.998+0000 7f50afa00640 1 -- 172.21.15.7:0/1618350878 >> 172.21.15.7:0/1618350878 conn(0x7f50a80742c0 msgr2=0x7f50a806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:23.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.998+0000 7f50afa00640 1 -- 172.21.15.7:0/1618350878 shutdown_connections 2024-04-25T11:21:23.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:23.998+0000 7f50afa00640 1 -- 172.21.15.7:0/1618350878 wait complete. 2024-04-25T11:21:23.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.000+0000 7f50afa00640 1 Processor -- start 2024-04-25T11:21:23.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.001+0000 7f50afa00640 1 -- start start 2024-04-25T11:21:23.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.002+0000 7f50afa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a80b5020 0x7f50a81420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:23.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.002+0000 7f50afa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50a80d0ee0 0x7f50a80b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:23.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.002+0000 7f50afa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50a815aaa0 0x7f50a8158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:23.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.002+0000 7f50afa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50a80b5430 con 0x7f50a80b5020 2024-04-25T11:21:23.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.002+0000 7f50afa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f50a80afaa0 con 0x7f50a815aaa0 2024-04-25T11:21:23.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.002+0000 7f50afa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f50a80b55b0 con 0x7f50a80d0ee0 2024-04-25T11:21:23.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.003+0000 7f50ad000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a80b5020 0x7f50a81420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:23.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.003+0000 7f50ad000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a80b5020 0x7f50a81420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36210/0 (socket says 172.21.15.7:36210) 2024-04-25T11:21:23.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.003+0000 7f50ad000640 1 -- 172.21.15.7:0/2786952651 learned_addr learned my addr 172.21.15.7:0/2786952651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:24.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.003+0000 7f50ada00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50a815aaa0 0x7f50a8158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:24.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.003+0000 7f50ad000640 1 -- 172.21.15.7:0/2786952651 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50a80d0ee0 msgr2=0x7f50a80b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:24.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.003+0000 7f50a7e00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50a80d0ee0 0x7f50a80b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.003+0000 7f50ad000640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50a80d0ee0 0x7f50a80b2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.003+0000 7f50ad000640 1 -- 172.21.15.7:0/2786952651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50a815aaa0 msgr2=0x7f50a8158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.003+0000 7f50ad000640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50a815aaa0 0x7f50a8158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.003+0000 7f50ad000640 1 -- 172.21.15.7:0/2786952651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a8142a70 con 0x7f50a80b5020 2024-04-25T11:21:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.003+0000 7f50ada00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50a815aaa0 0x7f50a8158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.004+0000 7f50ad000640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a80b5020 0x7f50a81420b0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f509c00c3c0 tx=0x7f509c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.004+0000 7f50a4c00640 1 -- 172.21.15.7:0/2786952651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c0052d0 con 0x7f50a80b5020 2024-04-25T11:21:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.004+0000 7f50a4c00640 1 -- 172.21.15.7:0/2786952651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509c005490 con 0x7f50a80b5020 2024-04-25T11:21:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.004+0000 7f50a4c00640 1 -- 172.21.15.7:0/2786952651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509c007260 con 0x7f50a80b5020 2024-04-25T11:21:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.004+0000 7f50afa00640 1 -- 172.21.15.7:0/2786952651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a8142d50 con 0x7f50a80b5020 2024-04-25T11:21:24.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.004+0000 7f50afa00640 1 -- 172.21.15.7:0/2786952651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a8143310 con 0x7f50a80b5020 2024-04-25T11:21:24.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.005+0000 7f50afa00640 1 -- 172.21.15.7:0/2786952651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a8050740 con 0x7f50a80b5020 2024-04-25T11:21:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.007+0000 7f50a4c00640 1 -- 172.21.15.7:0/2786952651 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f509c007400 con 0x7f50a80b5020 2024-04-25T11:21:24.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.009+0000 7f50a4c00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50880744b0 0x7f50880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:24.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.009+0000 7f50a4c00640 1 -- 172.21.15.7:0/2786952651 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7f509c094630 con 0x7f50a80b5020 2024-04-25T11:21:24.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.009+0000 7f50a7e00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50880744b0 0x7f50880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:24.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.010+0000 7f50a7e00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50880744b0 0x7f50880769c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f50a8146f50 tx=0x7f5094008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:24.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.013+0000 7f50a4c00640 1 -- 172.21.15.7:0/2786952651 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f509c05ee90 con 0x7f50a80b5020 2024-04-25T11:21:24.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.217+0000 7f50afa00640 1 -- 172.21.15.7:0/2786952651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7f50a8070f20 con 0x7f50a80b5020 2024-04-25T11:21:24.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.226+0000 7f50a4c00640 1 -- 172.21.15.7:0/2786952651 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v51) ==== 102+0+0 (secure 0 0 0) 0x7f509c05e790 con 0x7f50a80b5020 2024-04-25T11:21:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 -- 172.21.15.7:0/2786952651 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50880744b0 msgr2=0x7f50880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50880744b0 0x7f50880769c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f50a8146f50 tx=0x7f5094008040 comp rx=0 tx=0).stop 2024-04-25T11:21:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 -- 172.21.15.7:0/2786952651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a80b5020 msgr2=0x7f50a81420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a80b5020 0x7f50a81420b0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f509c00c3c0 tx=0x7f509c00c880 comp rx=0 tx=0).stop 2024-04-25T11:21:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 -- 172.21.15.7:0/2786952651 shutdown_connections 2024-04-25T11:21:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50880744b0 0x7f50880769c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50a815aaa0 0x7f50a8158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50a80d0ee0 0x7f50a80b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 --2- 172.21.15.7:0/2786952651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50a80b5020 0x7f50a81420b0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 -- 172.21.15.7:0/2786952651 >> 172.21.15.7:0/2786952651 conn(0x7f50a80742c0 msgr2=0x7f50a806fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:24.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 -- 172.21.15.7:0/2786952651 shutdown_connections 2024-04-25T11:21:24.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.229+0000 7f50afa00640 1 -- 172.21.15.7:0/2786952651 wait complete. 2024-04-25T11:21:24.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-04-25T11:21:24.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.542+0000 7f472a600640 1 Processor -- start 2024-04-25T11:21:24.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.542+0000 7f472a600640 1 -- start start 2024-04-25T11:21:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.545+0000 7f472a600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f472415aaa0 0x7f47240affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.545+0000 7f472a600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f47240d0ee0 0x7f47240b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.545+0000 7f472a600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f472406f5b0 con 0x7f47240b4e50 2024-04-25T11:21:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.545+0000 7f472a600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f47240af290 con 0x7f47240d0ee0 2024-04-25T11:21:24.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.545+0000 7f472a600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f47240d14a0 con 0x7f472415aaa0 2024-04-25T11:21:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.545+0000 7f4723e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f47240b4e50 0x7f472415d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58470/0 (socket says 172.21.15.7:58470) 2024-04-25T11:21:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.545+0000 7f4723e00640 1 -- 172.21.15.7:0/1578284720 learned_addr learned my addr 172.21.15.7:0/1578284720 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.545+0000 7f4723400640 1 --2- 172.21.15.7:0/1578284720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f472415aaa0 0x7f47240affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.546+0000 7f4722a00640 1 --2- 172.21.15.7:0/1578284720 >> v2:172.21.15.87:3300/0 conn(0x7f47240d0ee0 0x7f47240b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.546+0000 7f4722000640 1 -- 172.21.15.7:0/1578284720 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3954524676 0 0) 0x7f472406f5b0 con 0x7f47240b4e50 2024-04-25T11:21:24.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.546+0000 7f4722000640 1 -- 172.21.15.7:0/1578284720 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4708003520 con 0x7f47240b4e50 2024-04-25T11:21:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.546+0000 7f4723400640 1 -- 172.21.15.7:0/1578284720 >> v2:172.21.15.87:3300/0 conn(0x7f47240d0ee0 msgr2=0x7f47240b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.546+0000 7f4723400640 1 --2- 172.21.15.7:0/1578284720 >> v2:172.21.15.87:3300/0 conn(0x7f47240d0ee0 0x7f47240b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.546+0000 7f4722000640 1 -- 172.21.15.7:0/1578284720 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2366282195 0 0) 0x7f4708003520 con 0x7f47240b4e50 2024-04-25T11:21:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.546+0000 7f4723400640 1 -- 172.21.15.7:0/1578284720 >> v1:172.21.15.7:6789/0 conn(0x7f47240b4e50 legacy=0x7f472415d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.546+0000 7f4723400640 1 -- 172.21.15.7:0/1578284720 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47240af4e0 con 0x7f472415aaa0 2024-04-25T11:21:24.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.548+0000 7f4723400640 1 --2- 172.21.15.7:0/1578284720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f472415aaa0 0x7f47240affc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47140097a0 tx=0x7f471402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74baf73b6a90a9ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.548+0000 7f4722000640 1 -- 172.21.15.7:0/1578284720 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714037070 con 0x7f472415aaa0 2024-04-25T11:21:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.548+0000 7f4722000640 1 -- 172.21.15.7:0/1578284720 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f471402ab10 con 0x7f472415aaa0 2024-04-25T11:21:24.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.548+0000 7f4722000640 1 -- 172.21.15.7:0/1578284720 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c0019f0 con 0x7f472415aaa0 2024-04-25T11:21:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.549+0000 7f472a600640 1 -- 172.21.15.7:0/1578284720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f472415aaa0 msgr2=0x7f47240affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.549+0000 7f472a600640 1 --2- 172.21.15.7:0/1578284720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f472415aaa0 0x7f47240affc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47140097a0 tx=0x7f471402a080 comp rx=0 tx=0).stop 2024-04-25T11:21:24.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.549+0000 7f472a600640 1 -- 172.21.15.7:0/1578284720 shutdown_connections 2024-04-25T11:21:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.549+0000 7f472a600640 1 --2- 172.21.15.7:0/1578284720 >> v2:172.21.15.87:3300/0 conn(0x7f47240d0ee0 0x7f47240b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.549+0000 7f472a600640 1 --2- 172.21.15.7:0/1578284720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f472415aaa0 0x7f47240affc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.549+0000 7f472a600640 1 -- 172.21.15.7:0/1578284720 >> 172.21.15.7:0/1578284720 conn(0x7f47240742c0 msgr2=0x7f472406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.549+0000 7f472a600640 1 -- 172.21.15.7:0/1578284720 shutdown_connections 2024-04-25T11:21:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.550+0000 7f472a600640 1 -- 172.21.15.7:0/1578284720 wait complete. 2024-04-25T11:21:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.552+0000 7f472a600640 1 Processor -- start 2024-04-25T11:21:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.552+0000 7f472a600640 1 -- start start 2024-04-25T11:21:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.554+0000 7f472a600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47240b4e50 0x7f47241f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.554+0000 7f472a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47240d0ee0 0x7f472414b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.554+0000 7f472a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f472415aaa0 0x7f47240b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.554+0000 7f472a600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47240b5260 con 0x7f47240d0ee0 2024-04-25T11:21:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.554+0000 7f472a600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f47240afaa0 con 0x7f472415aaa0 2024-04-25T11:21:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.554+0000 7f472a600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f47240b53e0 con 0x7f47240b4e50 2024-04-25T11:21:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.554+0000 7f4722a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47240d0ee0 0x7f472414b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.554+0000 7f4722a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47240d0ee0 0x7f472414b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36222/0 (socket says 172.21.15.7:36222) 2024-04-25T11:21:24.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.554+0000 7f4722a00640 1 -- 172.21.15.7:0/3897778567 learned_addr learned my addr 172.21.15.7:0/3897778567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.555+0000 7f4723400640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47240b4e50 0x7f47241f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.555+0000 7f4723e00640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f472415aaa0 0x7f47240b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.555+0000 7f4722a00640 1 -- 172.21.15.7:0/3897778567 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47240b4e50 msgr2=0x7f47241f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.555+0000 7f4722a00640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47240b4e50 0x7f47241f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.555+0000 7f4722a00640 1 -- 172.21.15.7:0/3897778567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f472415aaa0 msgr2=0x7f47240b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.555+0000 7f4722a00640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f472415aaa0 0x7f47240b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.555+0000 7f4722a00640 1 -- 172.21.15.7:0/3897778567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4724148ff0 con 0x7f47240d0ee0 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.555+0000 7f4722a00640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47240d0ee0 0x7f472414b810 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f471000e870 tx=0x7f471000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.556+0000 7f471b400640 1 -- 172.21.15.7:0/3897778567 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710017070 con 0x7f47240d0ee0 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.556+0000 7f471b400640 1 -- 172.21.15.7:0/3897778567 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f471000c940 con 0x7f47240d0ee0 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.556+0000 7f471b400640 1 -- 172.21.15.7:0/3897778567 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471000cd10 con 0x7f47240d0ee0 2024-04-25T11:21:24.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.556+0000 7f472a600640 1 -- 172.21.15.7:0/3897778567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47241492d0 con 0x7f47240d0ee0 2024-04-25T11:21:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.556+0000 7f472a600640 1 -- 172.21.15.7:0/3897778567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4724149890 con 0x7f47240d0ee0 2024-04-25T11:21:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.557+0000 7f472a600640 1 -- 172.21.15.7:0/3897778567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4724148ac0 con 0x7f47240d0ee0 2024-04-25T11:21:24.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.558+0000 7f471b400640 1 -- 172.21.15.7:0/3897778567 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f471001a070 con 0x7f47240d0ee0 2024-04-25T11:21:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.560+0000 7f471b400640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f47040744e0 0x7f47040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.560+0000 7f471b400640 1 -- 172.21.15.7:0/3897778567 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7f4710095040 con 0x7f47240d0ee0 2024-04-25T11:21:24.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.560+0000 7f4723400640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f47040744e0 0x7f47040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:24.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.561+0000 7f4723400640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f47040744e0 0x7f47040769f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4714034000 tx=0x7f471402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:24.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.565+0000 7f471b400640 1 -- 172.21.15.7:0/3897778567 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4724148ac0 con 0x7f47240d0ee0 2024-04-25T11:21:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.772+0000 7f472a600640 1 -- 172.21.15.7:0/3897778567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7f4724050740 con 0x7f47240d0ee0 2024-04-25T11:21:24.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.781+0000 7f471b400640 1 -- 172.21.15.7:0/3897778567 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v52) ==== 107+0+0 (secure 0 0 0) 0x7f471005f820 con 0x7f47240d0ee0 2024-04-25T11:21:24.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 -- 172.21.15.7:0/3897778567 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f47040744e0 msgr2=0x7f47040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:24.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f47040744e0 0x7f47040769f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4714034000 tx=0x7f471402e000 comp rx=0 tx=0).stop 2024-04-25T11:21:24.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 -- 172.21.15.7:0/3897778567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47240d0ee0 msgr2=0x7f472414b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:24.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47240d0ee0 0x7f472414b810 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f471000e870 tx=0x7f471000ed30 comp rx=0 tx=0).stop 2024-04-25T11:21:24.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 -- 172.21.15.7:0/3897778567 shutdown_connections 2024-04-25T11:21:24.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f47040744e0 0x7f47040769f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f472415aaa0 0x7f47240b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47240d0ee0 0x7f472414b810 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 --2- 172.21.15.7:0/3897778567 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47240b4e50 0x7f47241f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 -- 172.21.15.7:0/3897778567 >> 172.21.15.7:0/3897778567 conn(0x7f47240742c0 msgr2=0x7f4724155d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 -- 172.21.15.7:0/3897778567 shutdown_connections 2024-04-25T11:21:24.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:24.784+0000 7f472a600640 1 -- 172.21.15.7:0/3897778567 wait complete. 2024-04-25T11:21:24.814 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:21:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.081+0000 7fa461600640 1 Processor -- start 2024-04-25T11:21:25.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.082+0000 7fa461600640 1 -- start start 2024-04-25T11:21:25.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.084+0000 7fa461600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa45c15dfb0 0x7fa45c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.084+0000 7fa461600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa45c0d0b40 0x7fa45c06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.084+0000 7fa461600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa45c15e3c0 con 0x7fa45c0d0f50 2024-04-25T11:21:25.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.084+0000 7fa461600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa45c0b09b0 con 0x7fa45c0d0b40 2024-04-25T11:21:25.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.084+0000 7fa461600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa45c0cf800 con 0x7fa45c15dfb0 2024-04-25T11:21:25.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.084+0000 7fa453e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa45c0d0b40 0x7fa45c06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.085+0000 7fa453e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa45c0d0b40 0x7fa45c06f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36224/0 (socket says 172.21.15.7:36224) 2024-04-25T11:21:25.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.085+0000 7fa453e00640 1 -- 172.21.15.7:0/4147697293 learned_addr learned my addr 172.21.15.7:0/4147697293 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:25.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.085+0000 7fa45aa00640 1 --2- 172.21.15.7:0/4147697293 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa45c15dfb0 0x7fa45c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.085+0000 7fa453e00640 1 -- 172.21.15.7:0/4147697293 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa45c15dfb0 msgr2=0x7fa45c0b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:25.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.085+0000 7fa453e00640 1 --2- 172.21.15.7:0/4147697293 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa45c15dfb0 0x7fa45c0b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.085+0000 7fa453e00640 1 -- 172.21.15.7:0/4147697293 >> v1:172.21.15.87:6789/0 conn(0x7fa45c0d0f50 legacy=0x7fa45c161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:25.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.085+0000 7fa453e00640 1 -- 172.21.15.7:0/4147697293 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45c0cfaa0 con 0x7fa45c0d0b40 2024-04-25T11:21:25.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.087+0000 7fa453e00640 1 --2- 172.21.15.7:0/4147697293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa45c0d0b40 0x7fa45c06f3a0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fa444009870 tx=0x7fa44402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e0d230b48c3ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.087+0000 7fa45a000640 1 -- 172.21.15.7:0/4147697293 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444037070 con 0x7fa45c0d0b40 2024-04-25T11:21:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.087+0000 7fa45a000640 1 -- 172.21.15.7:0/4147697293 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa444032440 con 0x7fa45c0d0b40 2024-04-25T11:21:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.087+0000 7fa45a000640 1 -- 172.21.15.7:0/4147697293 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444032810 con 0x7fa45c0d0b40 2024-04-25T11:21:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.088+0000 7fa461600640 1 -- 172.21.15.7:0/4147697293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa45c0d0b40 msgr2=0x7fa45c06f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.088+0000 7fa461600640 1 --2- 172.21.15.7:0/4147697293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa45c0d0b40 0x7fa45c06f3a0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fa444009870 tx=0x7fa44402a270 comp rx=0 tx=0).stop 2024-04-25T11:21:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.088+0000 7fa461600640 1 -- 172.21.15.7:0/4147697293 shutdown_connections 2024-04-25T11:21:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.088+0000 7fa461600640 1 --2- 172.21.15.7:0/4147697293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa45c0d0b40 0x7fa45c06f3a0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.088+0000 7fa461600640 1 --2- 172.21.15.7:0/4147697293 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa45c15dfb0 0x7fa45c0b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.088+0000 7fa461600640 1 -- 172.21.15.7:0/4147697293 >> 172.21.15.7:0/4147697293 conn(0x7fa45c0742c0 msgr2=0x7fa45c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:25.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.088+0000 7fa461600640 1 -- 172.21.15.7:0/4147697293 shutdown_connections 2024-04-25T11:21:25.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.088+0000 7fa461600640 1 -- 172.21.15.7:0/4147697293 wait complete. 2024-04-25T11:21:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.091+0000 7fa461600640 1 Processor -- start 2024-04-25T11:21:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.091+0000 7fa461600640 1 -- start start 2024-04-25T11:21:25.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.093+0000 7fa461600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa45c0d0b40 0x7fa45c147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.093+0000 7fa461600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa45c0d0f50 0x7fa45c14a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.093+0000 7fa461600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa45c15dfb0 0x7fa45c14a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.093+0000 7fa461600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c0d0340 con 0x7fa45c0d0b40 2024-04-25T11:21:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.093+0000 7fa461600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c0d0060 con 0x7fa45c0d0f50 2024-04-25T11:21:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.093+0000 7fa461600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa45c0d0620 con 0x7fa45c15dfb0 2024-04-25T11:21:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa45aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa45c0d0b40 0x7fa45c147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa45aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa45c0d0b40 0x7fa45c147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36230/0 (socket says 172.21.15.7:36230) 2024-04-25T11:21:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa45aa00640 1 -- 172.21.15.7:0/810606947 learned_addr learned my addr 172.21.15.7:0/810606947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:25.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa453e00640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa45c0d0f50 0x7fa45c14a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa45b400640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa45c15dfb0 0x7fa45c14a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa45aa00640 1 -- 172.21.15.7:0/810606947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa45c15dfb0 msgr2=0x7fa45c14a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa45aa00640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa45c15dfb0 0x7fa45c14a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa45aa00640 1 -- 172.21.15.7:0/810606947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa45c0d0f50 msgr2=0x7fa45c14a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa45aa00640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa45c0d0f50 0x7fa45c14a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa45aa00640 1 -- 172.21.15.7:0/810606947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45c0cd9b0 con 0x7fa45c0d0b40 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa453e00640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa45c0d0f50 0x7fa45c14a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.094+0000 7fa45b400640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa45c15dfb0 0x7fa45c14a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.095+0000 7fa45aa00640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa45c0d0b40 0x7fa45c147cc0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00d5d0 tx=0x7fa44c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.095+0000 7fa452a00640 1 -- 172.21.15.7:0/810606947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c004240 con 0x7fa45c0d0b40 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.095+0000 7fa452a00640 1 -- 172.21.15.7:0/810606947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa44c0043e0 con 0x7fa45c0d0b40 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.095+0000 7fa461600640 1 -- 172.21.15.7:0/810606947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45c148480 con 0x7fa45c0d0b40 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.095+0000 7fa452a00640 1 -- 172.21.15.7:0/810606947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c003830 con 0x7fa45c0d0b40 2024-04-25T11:21:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.095+0000 7fa461600640 1 -- 172.21.15.7:0/810606947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45c148a40 con 0x7fa45c0d0b40 2024-04-25T11:21:25.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.096+0000 7fa461600640 1 -- 172.21.15.7:0/810606947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa45c050740 con 0x7fa45c0d0b40 2024-04-25T11:21:25.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.098+0000 7fa452a00640 1 -- 172.21.15.7:0/810606947 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa44c0039d0 con 0x7fa45c0d0b40 2024-04-25T11:21:25.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.099+0000 7fa452a00640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4380744b0 0x7fa4380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.099+0000 7fa452a00640 1 -- 172.21.15.7:0/810606947 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7fa44c095420 con 0x7fa45c0d0b40 2024-04-25T11:21:25.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.099+0000 7fa453e00640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4380744b0 0x7fa4380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.100+0000 7fa453e00640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4380744b0 0x7fa4380769c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa444034000 tx=0x7fa44402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:25.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.104+0000 7fa452a00640 1 -- 172.21.15.7:0/810606947 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa44c05fc80 con 0x7fa45c0d0b40 2024-04-25T11:21:25.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.314+0000 7fa461600640 1 -- 172.21.15.7:0/810606947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa45c070fb0 con 0x7fa45c0d0b40 2024-04-25T11:21:25.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.319+0000 7fa452a00640 1 -- 172.21.15.7:0/810606947 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v53) ==== 104+0+0 (secure 0 0 0) 0x7fa44c05f580 con 0x7fa45c0d0b40 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.322+0000 7fa461600640 1 -- 172.21.15.7:0/810606947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4380744b0 msgr2=0x7fa4380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.322+0000 7fa461600640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4380744b0 0x7fa4380769c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa444034000 tx=0x7fa44402e000 comp rx=0 tx=0).stop 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.323+0000 7fa461600640 1 -- 172.21.15.7:0/810606947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa45c0d0b40 msgr2=0x7fa45c147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.323+0000 7fa461600640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa45c0d0b40 0x7fa45c147cc0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00d5d0 tx=0x7fa44c00da90 comp rx=0 tx=0).stop 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.323+0000 7fa461600640 1 -- 172.21.15.7:0/810606947 shutdown_connections 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.323+0000 7fa461600640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4380744b0 0x7fa4380769c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.323+0000 7fa461600640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa45c15dfb0 0x7fa45c14a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.323+0000 7fa461600640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa45c0d0f50 0x7fa45c14a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.323+0000 7fa461600640 1 --2- 172.21.15.7:0/810606947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa45c0d0b40 0x7fa45c147cc0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.323+0000 7fa461600640 1 -- 172.21.15.7:0/810606947 >> 172.21.15.7:0/810606947 conn(0x7fa45c0742c0 msgr2=0x7fa45c158650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.323+0000 7fa461600640 1 -- 172.21.15.7:0/810606947 shutdown_connections 2024-04-25T11:21:25.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.323+0000 7fa461600640 1 -- 172.21.15.7:0/810606947 wait complete. 2024-04-25T11:21:25.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-04-25T11:21:25.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.638+0000 7f1912200640 1 Processor -- start 2024-04-25T11:21:25.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.638+0000 7f1912200640 1 -- start start 2024-04-25T11:21:25.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.640+0000 7f1912200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f190c0b4a10 0x7f190c0d53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.640+0000 7f1912200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f190c160090 0x7f190c0b9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.641+0000 7f1912200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f190c1604a0 con 0x7f190c0b7440 2024-04-25T11:21:25.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.641+0000 7f1912200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f190c0d1490 con 0x7f190c160090 2024-04-25T11:21:25.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.641+0000 7f1912200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f190c0b9550 con 0x7f190c0b4a10 2024-04-25T11:21:25.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.641+0000 7f1911800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f190c0b7440 0x7f190c0b7850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58484/0 (socket says 172.21.15.7:58484) 2024-04-25T11:21:25.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.641+0000 7f1911800640 1 -- 172.21.15.7:0/1485969531 learned_addr learned my addr 172.21.15.7:0/1485969531 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:25.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.641+0000 7f1903e00640 1 --2- 172.21.15.7:0/1485969531 >> v2:172.21.15.87:3300/0 conn(0x7f190c160090 0x7f190c0b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.641+0000 7f1910e00640 1 --2- 172.21.15.7:0/1485969531 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f190c0b4a10 0x7f190c0d53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.642+0000 7f190be00640 1 -- 172.21.15.7:0/1485969531 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 608320735 0 0) 0x7f190c1604a0 con 0x7f190c0b7440 2024-04-25T11:21:25.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.642+0000 7f190be00640 1 -- 172.21.15.7:0/1485969531 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18ec003520 con 0x7f190c0b7440 2024-04-25T11:21:25.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.642+0000 7f190be00640 1 -- 172.21.15.7:0/1485969531 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3073951473 0 0) 0x7f18ec003520 con 0x7f190c0b7440 2024-04-25T11:21:25.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.642+0000 7f1903e00640 1 -- 172.21.15.7:0/1485969531 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f190c0b4a10 msgr2=0x7f190c0d53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:25.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.642+0000 7f1903e00640 1 --2- 172.21.15.7:0/1485969531 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f190c0b4a10 0x7f190c0d53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.642+0000 7f1903e00640 1 -- 172.21.15.7:0/1485969531 >> v1:172.21.15.7:6789/0 conn(0x7f190c0b7440 legacy=0x7f190c0b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:25.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.642+0000 7f1903e00640 1 -- 172.21.15.7:0/1485969531 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c0d0880 con 0x7f190c160090 2024-04-25T11:21:25.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.643+0000 7f1903e00640 1 --2- 172.21.15.7:0/1485969531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f190c160090 0x7f190c0b9010 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0097a0 tx=0x7f18fc02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d8243b9d0517a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:25.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.644+0000 7f190be00640 1 -- 172.21.15.7:0/1485969531 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc037070 con 0x7f190c160090 2024-04-25T11:21:25.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.644+0000 7f190be00640 1 -- 172.21.15.7:0/1485969531 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18fc02ab10 con 0x7f190c160090 2024-04-25T11:21:25.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.644+0000 7f190be00640 1 -- 172.21.15.7:0/1485969531 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040019f0 con 0x7f190c160090 2024-04-25T11:21:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.645+0000 7f1912200640 1 -- 172.21.15.7:0/1485969531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f190c160090 msgr2=0x7f190c0b9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.645+0000 7f1912200640 1 --2- 172.21.15.7:0/1485969531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f190c160090 0x7f190c0b9010 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0097a0 tx=0x7f18fc02a080 comp rx=0 tx=0).stop 2024-04-25T11:21:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.645+0000 7f1912200640 1 -- 172.21.15.7:0/1485969531 shutdown_connections 2024-04-25T11:21:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.645+0000 7f1912200640 1 --2- 172.21.15.7:0/1485969531 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f190c160090 0x7f190c0b9010 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.645+0000 7f1912200640 1 --2- 172.21.15.7:0/1485969531 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f190c0b4a10 0x7f190c0d53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.645+0000 7f1912200640 1 -- 172.21.15.7:0/1485969531 >> 172.21.15.7:0/1485969531 conn(0x7f190c0d5de0 msgr2=0x7f190c0d61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.645+0000 7f1912200640 1 -- 172.21.15.7:0/1485969531 shutdown_connections 2024-04-25T11:21:25.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.645+0000 7f1912200640 1 -- 172.21.15.7:0/1485969531 wait complete. 2024-04-25T11:21:25.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.647+0000 7f1912200640 1 Processor -- start 2024-04-25T11:21:25.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.647+0000 7f1912200640 1 -- start start 2024-04-25T11:21:25.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.649+0000 7f1912200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f190c0b4a10 0x7f190c0b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.649+0000 7f1912200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190c0b7440 0x7f190c1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.649+0000 7f1912200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f190c160090 0x7f190c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.649+0000 7f1912200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f190c0d1120 con 0x7f190c0b7440 2024-04-25T11:21:25.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.649+0000 7f1912200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f190c0d0e40 con 0x7f190c0b4a10 2024-04-25T11:21:25.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.649+0000 7f1912200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f190c163d40 con 0x7f190c160090 2024-04-25T11:21:25.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.649+0000 7f1903e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190c0b7440 0x7f190c1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.649+0000 7f1903e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190c0b7440 0x7f190c1560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36238/0 (socket says 172.21.15.7:36238) 2024-04-25T11:21:25.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.649+0000 7f1903e00640 1 -- 172.21.15.7:0/1952443996 learned_addr learned my addr 172.21.15.7:0/1952443996 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:25.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.650+0000 7f1910e00640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f190c0b4a10 0x7f190c0b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.650+0000 7f1911800640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f190c160090 0x7f190c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.650+0000 7f1903e00640 1 -- 172.21.15.7:0/1952443996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f190c160090 msgr2=0x7f190c1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:25.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.650+0000 7f1903e00640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f190c160090 0x7f190c1565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.650+0000 7f1903e00640 1 -- 172.21.15.7:0/1952443996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f190c0b4a10 msgr2=0x7f190c0b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:25.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.650+0000 7f1903e00640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f190c0b4a10 0x7f190c0b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.650+0000 7f1903e00640 1 -- 172.21.15.7:0/1952443996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c153ee0 con 0x7f190c0b7440 2024-04-25T11:21:25.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.650+0000 7f1903e00640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190c0b7440 0x7f190c1560b0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000c00 tx=0x7f18fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:25.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.651+0000 7f1909600640 1 -- 172.21.15.7:0/1952443996 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc037040 con 0x7f190c0b7440 2024-04-25T11:21:25.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.651+0000 7f1909600640 1 -- 172.21.15.7:0/1952443996 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18fc032e50 con 0x7f190c0b7440 2024-04-25T11:21:25.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.651+0000 7f1909600640 1 -- 172.21.15.7:0/1952443996 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc004270 con 0x7f190c0b7440 2024-04-25T11:21:25.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.651+0000 7f1912200640 1 -- 172.21.15.7:0/1952443996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c1541c0 con 0x7f190c0b7440 2024-04-25T11:21:25.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.651+0000 7f1912200640 1 -- 172.21.15.7:0/1952443996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c0709d0 con 0x7f190c0b7440 2024-04-25T11:21:25.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.652+0000 7f1912200640 1 -- 172.21.15.7:0/1952443996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190c156be0 con 0x7f190c0b7440 2024-04-25T11:21:25.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.653+0000 7f1909600640 1 -- 172.21.15.7:0/1952443996 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f18fc003610 con 0x7f190c0b7440 2024-04-25T11:21:25.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.654+0000 7f1909600640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18e80744b0 0x7f18e80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:25.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.655+0000 7f1909600640 1 -- 172.21.15.7:0/1952443996 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7f18fc0b52e0 con 0x7f190c0b7440 2024-04-25T11:21:25.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.655+0000 7f1910e00640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18e80744b0 0x7f18e80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:25.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.656+0000 7f1910e00640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18e80744b0 0x7f18e80769c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f190c0d1cb0 tx=0x7f18f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:25.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.660+0000 7f1909600640 1 -- 172.21.15.7:0/1952443996 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18fc042020 con 0x7f190c0b7440 2024-04-25T11:21:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.868+0000 7f1912200640 1 -- 172.21.15.7:0/1952443996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7f190c050740 con 0x7f190c0b7440 2024-04-25T11:21:25.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.876+0000 7f1909600640 1 -- 172.21.15.7:0/1952443996 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v54) ==== 111+0+0 (secure 0 0 0) 0x7f18fc07fb40 con 0x7f190c0b7440 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.879+0000 7f1912200640 1 -- 172.21.15.7:0/1952443996 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18e80744b0 msgr2=0x7f18e80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.879+0000 7f1912200640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18e80744b0 0x7f18e80769c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f190c0d1cb0 tx=0x7f18f4008040 comp rx=0 tx=0).stop 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.879+0000 7f1912200640 1 -- 172.21.15.7:0/1952443996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190c0b7440 msgr2=0x7f190c1560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.879+0000 7f1912200640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190c0b7440 0x7f190c1560b0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000c00 tx=0x7f18fc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.880+0000 7f1912200640 1 -- 172.21.15.7:0/1952443996 shutdown_connections 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.880+0000 7f1912200640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f18e80744b0 0x7f18e80769c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.880+0000 7f1912200640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f190c160090 0x7f190c1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.880+0000 7f1912200640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f190c0b7440 0x7f190c1560b0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.880+0000 7f1912200640 1 --2- 172.21.15.7:0/1952443996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f190c0b4a10 0x7f190c0b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.880+0000 7f1912200640 1 -- 172.21.15.7:0/1952443996 >> 172.21.15.7:0/1952443996 conn(0x7f190c0d5de0 msgr2=0x7f190c0b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.880+0000 7f1912200640 1 -- 172.21.15.7:0/1952443996 shutdown_connections 2024-04-25T11:21:25.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:25.880+0000 7f1912200640 1 -- 172.21.15.7:0/1952443996 wait complete. 2024-04-25T11:21:25.903 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-04-25T11:21:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.193+0000 7f450a000640 1 Processor -- start 2024-04-25T11:21:26.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.193+0000 7f450a000640 1 -- start start 2024-04-25T11:21:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.196+0000 7f450a000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f450415aaa0 0x7f45040affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.196+0000 7f450a000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f45040b5020 0x7f4504160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.196+0000 7f450a000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f450406f5b0 con 0x7f45040d0ee0 2024-04-25T11:21:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.196+0000 7f450a000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f45040af290 con 0x7f45040b5020 2024-04-25T11:21:26.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.196+0000 7f450a000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f45040d14a0 con 0x7f450415aaa0 2024-04-25T11:21:26.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.196+0000 7f4503e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f45040b5020 0x7f4504160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.196+0000 7f4503e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f45040b5020 0x7f4504160990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36242/0 (socket says 172.21.15.7:36242) 2024-04-25T11:21:26.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.196+0000 7f4503e00640 1 -- 172.21.15.7:0/470938492 learned_addr learned my addr 172.21.15.7:0/470938492 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:26.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.196+0000 7f4502a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f45040d0ee0 0x7f45040b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58496/0 (socket says 172.21.15.7:58496) 2024-04-25T11:21:26.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.197+0000 7f4503400640 1 --2- 172.21.15.7:0/470938492 >> v2:172.21.15.87:3300/0 conn(0x7f450415aaa0 0x7f45040affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.197+0000 7f4503e00640 1 -- 172.21.15.7:0/470938492 >> v2:172.21.15.87:3300/0 conn(0x7f450415aaa0 msgr2=0x7f45040affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:26.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.197+0000 7f4503e00640 1 --2- 172.21.15.7:0/470938492 >> v2:172.21.15.87:3300/0 conn(0x7f450415aaa0 0x7f45040affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.197+0000 7f4503e00640 1 -- 172.21.15.7:0/470938492 >> v1:172.21.15.7:6789/0 conn(0x7f45040d0ee0 legacy=0x7f45040b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:26.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.197+0000 7f4503e00640 1 -- 172.21.15.7:0/470938492 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45040af4e0 con 0x7f45040b5020 2024-04-25T11:21:26.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.197+0000 7f4503400640 1 --2- 172.21.15.7:0/470938492 >> v2:172.21.15.87:3300/0 conn(0x7f450415aaa0 0x7f45040affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:21:26.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.197+0000 7f4502000640 1 -- 172.21.15.7:0/470938492 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2168479692 0 0) 0x7f450406f5b0 con 0x7f45040d0ee0 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.198+0000 7f4503e00640 1 --2- 172.21.15.7:0/470938492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45040b5020 0x7f4504160990 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f45040af410 tx=0x7f44fc02c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ad9ac15260c8c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.199+0000 7f4502000640 1 -- 172.21.15.7:0/470938492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc009600 con 0x7f45040b5020 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.199+0000 7f4502000640 1 -- 172.21.15.7:0/470938492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44fc037070 con 0x7f45040b5020 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.199+0000 7f4502000640 1 -- 172.21.15.7:0/470938492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0099c0 con 0x7f45040b5020 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.200+0000 7f450a000640 1 -- 172.21.15.7:0/470938492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45040b5020 msgr2=0x7f4504160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.200+0000 7f450a000640 1 --2- 172.21.15.7:0/470938492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45040b5020 0x7f4504160990 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f45040af410 tx=0x7f44fc02c930 comp rx=0 tx=0).stop 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.200+0000 7f450a000640 1 -- 172.21.15.7:0/470938492 shutdown_connections 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.200+0000 7f450a000640 1 --2- 172.21.15.7:0/470938492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45040b5020 0x7f4504160990 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.200+0000 7f450a000640 1 --2- 172.21.15.7:0/470938492 >> v2:172.21.15.87:3300/0 conn(0x7f450415aaa0 0x7f45040affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.200+0000 7f450a000640 1 -- 172.21.15.7:0/470938492 >> 172.21.15.7:0/470938492 conn(0x7f45040742c0 msgr2=0x7f450406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.200+0000 7f450a000640 1 -- 172.21.15.7:0/470938492 shutdown_connections 2024-04-25T11:21:26.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.200+0000 7f450a000640 1 -- 172.21.15.7:0/470938492 wait complete. 2024-04-25T11:21:26.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.202+0000 7f450a000640 1 Processor -- start 2024-04-25T11:21:26.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.202+0000 7f450a000640 1 -- start start 2024-04-25T11:21:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.204+0000 7f450a000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45040b5020 0x7f45041518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.204+0000 7f450a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45040d0ee0 0x7f450409da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.204+0000 7f450a000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450415aaa0 0x7f450409df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.204+0000 7f450a000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f45040b5430 con 0x7f45040d0ee0 2024-04-25T11:21:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.204+0000 7f450a000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f45040afaa0 con 0x7f45040b5020 2024-04-25T11:21:26.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.204+0000 7f450a000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f45040b55b0 con 0x7f450415aaa0 2024-04-25T11:21:26.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.204+0000 7f4502a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45040d0ee0 0x7f450409da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.204+0000 7f4502a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45040d0ee0 0x7f450409da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36248/0 (socket says 172.21.15.7:36248) 2024-04-25T11:21:26.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.204+0000 7f4502a00640 1 -- 172.21.15.7:0/1535208383 learned_addr learned my addr 172.21.15.7:0/1535208383 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:26.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.205+0000 7f4503e00640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450415aaa0 0x7f450409df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.205+0000 7f4503400640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45040b5020 0x7f45041518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.205+0000 7f4502a00640 1 -- 172.21.15.7:0/1535208383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450415aaa0 msgr2=0x7f450409df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.205+0000 7f4502a00640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450415aaa0 0x7f450409df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.205+0000 7f4502a00640 1 -- 172.21.15.7:0/1535208383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45040b5020 msgr2=0x7f45041518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.205+0000 7f4502a00640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45040b5020 0x7f45041518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.205+0000 7f4502a00640 1 -- 172.21.15.7:0/1535208383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45040cfd10 con 0x7f45040d0ee0 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.205+0000 7f4502a00640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45040d0ee0 0x7f450409da50 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00c430 tx=0x7f44ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.206+0000 7f44fb400640 1 -- 172.21.15.7:0/1535208383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0074e0 con 0x7f45040d0ee0 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.206+0000 7f44fb400640 1 -- 172.21.15.7:0/1535208383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ec0047e0 con 0x7f45040d0ee0 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.206+0000 7f44fb400640 1 -- 172.21.15.7:0/1535208383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec004bd0 con 0x7f45040d0ee0 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.206+0000 7f450a000640 1 -- 172.21.15.7:0/1535208383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45040cfff0 con 0x7f45040d0ee0 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.206+0000 7f450a000640 1 -- 172.21.15.7:0/1535208383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45040d05b0 con 0x7f45040d0ee0 2024-04-25T11:21:26.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.207+0000 7f450a000640 1 -- 172.21.15.7:0/1535208383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450409c3a0 con 0x7f45040d0ee0 2024-04-25T11:21:26.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.208+0000 7f44fb400640 1 -- 172.21.15.7:0/1535208383 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f44ec011070 con 0x7f45040d0ee0 2024-04-25T11:21:26.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.209+0000 7f44fb400640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f44e00744e0 0x7f44e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.209+0000 7f44fb400640 1 -- 172.21.15.7:0/1535208383 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7f44ec094ce0 con 0x7f45040d0ee0 2024-04-25T11:21:26.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.210+0000 7f4503400640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f44e00744e0 0x7f44e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.210+0000 7f4503400640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f44e00744e0 0x7f44e00769f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f44f40096d0 tx=0x7f44f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:26.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.214+0000 7f44fb400640 1 -- 172.21.15.7:0/1535208383 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f44ec05f650 con 0x7f45040d0ee0 2024-04-25T11:21:26.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.423+0000 7f450a000640 1 -- 172.21.15.7:0/1535208383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7f4504068cf0 con 0x7f45040d0ee0 2024-04-25T11:21:26.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.430+0000 7f44fb400640 1 -- 172.21.15.7:0/1535208383 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v55) ==== 105+0+0 (secure 0 0 0) 0x7f4504068cf0 con 0x7f45040d0ee0 2024-04-25T11:21:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 -- 172.21.15.7:0/1535208383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f44e00744e0 msgr2=0x7f44e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f44e00744e0 0x7f44e00769f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f44f40096d0 tx=0x7f44f4008040 comp rx=0 tx=0).stop 2024-04-25T11:21:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 -- 172.21.15.7:0/1535208383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45040d0ee0 msgr2=0x7f450409da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45040d0ee0 0x7f450409da50 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00c430 tx=0x7f44ec00c8f0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 -- 172.21.15.7:0/1535208383 shutdown_connections 2024-04-25T11:21:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f44e00744e0 0x7f44e00769f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450415aaa0 0x7f450409df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45040d0ee0 0x7f450409da50 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 --2- 172.21.15.7:0/1535208383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45040b5020 0x7f45041518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 -- 172.21.15.7:0/1535208383 >> 172.21.15.7:0/1535208383 conn(0x7f45040742c0 msgr2=0x7f45040685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 -- 172.21.15.7:0/1535208383 shutdown_connections 2024-04-25T11:21:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.433+0000 7f450a000640 1 -- 172.21.15.7:0/1535208383 wait complete. 2024-04-25T11:21:26.458 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps' 2024-04-25T11:21:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.740+0000 7fc0b3200640 1 Processor -- start 2024-04-25T11:21:26.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.740+0000 7fc0b3200640 1 -- start start 2024-04-25T11:21:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.742+0000 7fc0b3200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0b40d12a0 0x7fc0b40d4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.743+0000 7fc0b3200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0b40b8ef0 con 0x7fc0b406d1d0 2024-04-25T11:21:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.743+0000 7fc0b3200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0b40d4e60 con 0x7fc0b40d4850 2024-04-25T11:21:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.743+0000 7fc0b3200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc0b40d5420 con 0x7fc0b40d12a0 2024-04-25T11:21:26.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.743+0000 7fc0b2800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc0b406d1d0 0x7fc0b415f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58504/0 (socket says 172.21.15.7:58504) 2024-04-25T11:21:26.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.743+0000 7fc0b2800640 1 -- 172.21.15.7:0/1749591529 learned_addr learned my addr 172.21.15.7:0/1749591529 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:26.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.743+0000 7fc0b1e00640 1 --2- 172.21.15.7:0/1749591529 >> v2:172.21.15.87:3300/0 conn(0x7fc0b40d12a0 0x7fc0b40d4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3961213035 0 0) 0x7fc0b40b8ef0 con 0x7fc0b406d1d0 2024-04-25T11:21:26.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc098003520 con 0x7fc0b406d1d0 2024-04-25T11:21:26.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1539941921 0 0) 0x7fc098003520 con 0x7fc0b406d1d0 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 >> v2:172.21.15.87:3300/0 conn(0x7fc0b40d12a0 msgr2=0x7fc0b40d4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 --2- 172.21.15.7:0/1749591529 >> v2:172.21.15.87:3300/0 conn(0x7fc0b40d12a0 0x7fc0b40d4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 >> v1:172.21.15.87:6789/0 conn(0x7fc0b40d4850 legacy=0x7fc0b406cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b40d5140 con 0x7fc0b406d1d0 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fc0ac0019f0 con 0x7fc0b406d1d0 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 >> v1:172.21.15.7:6789/0 conn(0x7fc0b406d1d0 legacy=0x7fc0b415f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 0x7fc098003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc098004800 0x7fc098004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0980051c0 0x7fc09800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc09800c130 con 0x7fc0980039b0 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc09800c2e0 con 0x7fc0980051c0 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.744+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc09800c4e0 con 0x7fc098004800 2024-04-25T11:21:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.745+0000 7fc0b1e00640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 0x7fc098003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.745+0000 7fc0b1e00640 1 -- 172.21.15.7:0/1749591529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc098004800 msgr2=0x7fc098004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.745+0000 7fc0b1400640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc098004800 0x7fc098004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.745+0000 7fc0b1e00640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc098004800 0x7fc098004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.745+0000 7fc0b2800640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0980051c0 0x7fc09800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.745+0000 7fc0b1e00640 1 -- 172.21.15.7:0/1749591529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0980051c0 msgr2=0x7fc09800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.745+0000 7fc0b1e00640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0980051c0 0x7fc09800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.745+0000 7fc0b1e00640 1 -- 172.21.15.7:0/1749591529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc098003520 con 0x7fc0980039b0 2024-04-25T11:21:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.745+0000 7fc0b2800640 1 -- 172.21.15.7:0/1749591529 reap_dead start 2024-04-25T11:21:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.745+0000 7fc0b1400640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc098004800 0x7fc098004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.746+0000 7fc0b1e00640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 0x7fc098003e20 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009ac0 tx=0x7fc098005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc5e2247deecb7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.746+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4036070 con 0x7fc0980039b0 2024-04-25T11:21:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.747+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a4031400 con 0x7fc0980039b0 2024-04-25T11:21:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.747+0000 7fc0b0a00640 1 -- 172.21.15.7:0/1749591529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4009690 con 0x7fc0980039b0 2024-04-25T11:21:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.747+0000 7fc0b3200640 1 -- 172.21.15.7:0/1749591529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 msgr2=0x7fc098003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.747+0000 7fc0b3200640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 0x7fc098003e20 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009ac0 tx=0x7fc098005830 comp rx=0 tx=0).stop 2024-04-25T11:21:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.747+0000 7fc0b3200640 1 -- 172.21.15.7:0/1749591529 shutdown_connections 2024-04-25T11:21:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.747+0000 7fc0b3200640 1 --2- 172.21.15.7:0/1749591529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 0x7fc098003e20 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.747+0000 7fc0b3200640 1 -- 172.21.15.7:0/1749591529 >> 172.21.15.7:0/1749591529 conn(0x7fc0b40d2880 msgr2=0x7fc0b40d2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.747+0000 7fc0b3200640 1 -- 172.21.15.7:0/1749591529 shutdown_connections 2024-04-25T11:21:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.747+0000 7fc0b3200640 1 -- 172.21.15.7:0/1749591529 wait complete. 2024-04-25T11:21:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.750+0000 7fc0b3200640 1 Processor -- start 2024-04-25T11:21:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.750+0000 7fc0b3200640 1 -- start start 2024-04-25T11:21:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.751+0000 7fc0b3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 0x7fc0b40caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.751+0000 7fc0b3200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b415fd60 0x7fc0b40cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b3200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0b4160170 0x7fc0b40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b40d3e70 con 0x7fc0980039b0 2024-04-25T11:21:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b3200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b40b9470 con 0x7fc0b415fd60 2024-04-25T11:21:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b3200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b40d4310 con 0x7fc0b4160170 2024-04-25T11:21:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 0x7fc0b40caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 0x7fc0b40caee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36260/0 (socket says 172.21.15.7:36260) 2024-04-25T11:21:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b1e00640 1 -- 172.21.15.7:0/3534040808 learned_addr learned my addr 172.21.15.7:0/3534040808 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b1400640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b415fd60 0x7fc0b40cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b2800640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0b4160170 0x7fc0b40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b1e00640 1 -- 172.21.15.7:0/3534040808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0b4160170 msgr2=0x7fc0b40cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b1e00640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0b4160170 0x7fc0b40cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b1e00640 1 -- 172.21.15.7:0/3534040808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b415fd60 msgr2=0x7fc0b40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:26.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b1e00640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b415fd60 0x7fc0b40cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b1e00640 1 -- 172.21.15.7:0/3534040808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b40cb420 con 0x7fc0980039b0 2024-04-25T11:21:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.752+0000 7fc0b1400640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b415fd60 0x7fc0b40cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.753+0000 7fc0b1e00640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 0x7fc0b40caee0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4034040 tx=0x7fc0a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.753+0000 7fc0aa000640 1 -- 172.21.15.7:0/3534040808 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a403f020 con 0x7fc0980039b0 2024-04-25T11:21:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.753+0000 7fc0aa000640 1 -- 172.21.15.7:0/3534040808 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a4037040 con 0x7fc0980039b0 2024-04-25T11:21:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.753+0000 7fc0aa000640 1 -- 172.21.15.7:0/3534040808 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4036070 con 0x7fc0980039b0 2024-04-25T11:21:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.753+0000 7fc0b3200640 1 -- 172.21.15.7:0/3534040808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b40ac730 con 0x7fc0980039b0 2024-04-25T11:21:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.753+0000 7fc0b3200640 1 -- 172.21.15.7:0/3534040808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b40accf0 con 0x7fc0980039b0 2024-04-25T11:21:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.754+0000 7fc0b3200640 1 -- 172.21.15.7:0/3534040808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b40d3b90 con 0x7fc0980039b0 2024-04-25T11:21:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.756+0000 7fc0aa000640 1 -- 172.21.15.7:0/3534040808 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc0a4042020 con 0x7fc0980039b0 2024-04-25T11:21:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.757+0000 7fc0aa000640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0940744b0 0x7fc0940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.757+0000 7fc0aa000640 1 -- 172.21.15.7:0/3534040808 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7fc0a40b51e0 con 0x7fc0980039b0 2024-04-25T11:21:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.757+0000 7fc0b1400640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0940744b0 0x7fc0940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.758+0000 7fc0b1400640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0940744b0 0x7fc0940769c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc0b41630f0 tx=0x7fc0a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:26.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:26.763+0000 7fc0aa000640 1 -- 172.21.15.7:0/3534040808 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0b40d3b90 con 0x7fc0980039b0 2024-04-25T11:21:27.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.043+0000 7fc0b3200640 1 -- 172.21.15.7:0/3534040808 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]} v 0) -- 0x7fc0b4050740 con 0x7fc0980039b0 2024-04-25T11:21:27.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.692+0000 7fc0aa000640 1 -- 172.21.15.7:0/3534040808 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_mincaps"]}]=0 v574) ==== 151+0+0 (secure 0 0 0) 0x7fc0a4049030 con 0x7fc0980039b0 2024-04-25T11:21:27.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.695+0000 7fc0b3200640 1 -- 172.21.15.7:0/3534040808 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0940744b0 msgr2=0x7fc0940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:27.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.695+0000 7fc0b3200640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0940744b0 0x7fc0940769c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc0b41630f0 tx=0x7fc0a0009040 comp rx=0 tx=0).stop 2024-04-25T11:21:27.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.695+0000 7fc0b3200640 1 -- 172.21.15.7:0/3534040808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 msgr2=0x7fc0b40caee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:27.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.695+0000 7fc0b3200640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 0x7fc0b40caee0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4034040 tx=0x7fc0a4004730 comp rx=0 tx=0).stop 2024-04-25T11:21:27.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.695+0000 7fc0b3200640 1 -- 172.21.15.7:0/3534040808 shutdown_connections 2024-04-25T11:21:27.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.695+0000 7fc0b3200640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0940744b0 0x7fc0940769c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:27.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.695+0000 7fc0b3200640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0b4160170 0x7fc0b40cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:27.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.695+0000 7fc0b3200640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0b415fd60 0x7fc0b40cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:27.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.695+0000 7fc0b3200640 1 --2- 172.21.15.7:0/3534040808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0980039b0 0x7fc0b40caee0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:27.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.695+0000 7fc0b3200640 1 -- 172.21.15.7:0/3534040808 >> 172.21.15.7:0/3534040808 conn(0x7fc0b40d2880 msgr2=0x7fc0b40bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.696+0000 7fc0b3200640 1 -- 172.21.15.7:0/3534040808 shutdown_connections 2024-04-25T11:21:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:27.696+0000 7fc0b3200640 1 -- 172.21.15.7:0/3534040808 wait complete. 2024-04-25T11:21:27.721 INFO:tasks.cephfs_test_runner:test_client_pin_mincaps (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-04-25T11:21:27.721 INFO:tasks.cephfs_test_runner:Starting test: test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-25T11:21:27.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:21:28.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.014+0000 7fce6d400640 1 Processor -- start 2024-04-25T11:21:28.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.014+0000 7fce6d400640 1 -- start start 2024-04-25T11:21:28.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.017+0000 7fce6d400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6815ef90 0x7fce6809f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.017+0000 7fce6d400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce680af540 0x7fce680cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.017+0000 7fce6d400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce6815f4e0 con 0x7fce6815bc40 2024-04-25T11:21:28.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.017+0000 7fce6d400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fce68071190 con 0x7fce680af540 2024-04-25T11:21:28.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.017+0000 7fce6d400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce680b0560 con 0x7fce6815ef90 2024-04-25T11:21:28.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.018+0000 7fce67200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce680af540 0x7fce680cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:28.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.018+0000 7fce67200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce680af540 0x7fce680cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36274/0 (socket says 172.21.15.7:36274) 2024-04-25T11:21:28.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.018+0000 7fce67200640 1 -- 172.21.15.7:0/2036110100 learned_addr learned my addr 172.21.15.7:0/2036110100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:28.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.018+0000 7fce66800640 1 --2- 172.21.15.7:0/2036110100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6815ef90 0x7fce6809f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:28.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.018+0000 7fce67200640 1 -- 172.21.15.7:0/2036110100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6815ef90 msgr2=0x7fce6809f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.018+0000 7fce67200640 1 --2- 172.21.15.7:0/2036110100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6815ef90 0x7fce6809f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.018+0000 7fce67200640 1 -- 172.21.15.7:0/2036110100 >> v1:172.21.15.87:6789/0 conn(0x7fce6815bc40 legacy=0x7fce680aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.018+0000 7fce67200640 1 -- 172.21.15.7:0/2036110100 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce680b0840 con 0x7fce680af540 2024-04-25T11:21:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.019+0000 7fce67200640 1 --2- 172.21.15.7:0/2036110100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce680af540 0x7fce680cf980 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fce60009870 tx=0x7fce6002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f984b2ec07ee4e4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:28.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.020+0000 7fce65e00640 1 -- 172.21.15.7:0/2036110100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60037070 con 0x7fce680af540 2024-04-25T11:21:28.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.020+0000 7fce65e00640 1 -- 172.21.15.7:0/2036110100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce60032440 con 0x7fce680af540 2024-04-25T11:21:28.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.020+0000 7fce65e00640 1 -- 172.21.15.7:0/2036110100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60032810 con 0x7fce680af540 2024-04-25T11:21:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.020+0000 7fce6d400640 1 -- 172.21.15.7:0/2036110100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce680af540 msgr2=0x7fce680cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.020+0000 7fce6d400640 1 --2- 172.21.15.7:0/2036110100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce680af540 0x7fce680cf980 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fce60009870 tx=0x7fce6002a1c0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.021+0000 7fce6d400640 1 -- 172.21.15.7:0/2036110100 shutdown_connections 2024-04-25T11:21:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.021+0000 7fce6d400640 1 --2- 172.21.15.7:0/2036110100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce680af540 0x7fce680cf980 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.021+0000 7fce6d400640 1 --2- 172.21.15.7:0/2036110100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6815ef90 0x7fce6809f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.021+0000 7fce6d400640 1 -- 172.21.15.7:0/2036110100 >> 172.21.15.7:0/2036110100 conn(0x7fce68074200 msgr2=0x7fce6806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.021+0000 7fce6d400640 1 -- 172.21.15.7:0/2036110100 shutdown_connections 2024-04-25T11:21:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.021+0000 7fce6d400640 1 -- 172.21.15.7:0/2036110100 wait complete. 2024-04-25T11:21:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.023+0000 7fce6d400640 1 Processor -- start 2024-04-25T11:21:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.023+0000 7fce6d400640 1 -- start start 2024-04-25T11:21:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.025+0000 7fce6d400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6815bc40 0x7fce681fb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.025+0000 7fce6d400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce6815ef90 0x7fce68159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.025+0000 7fce6d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce680b5930 0x7fce680b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.025+0000 7fce6d400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce680b1320 con 0x7fce680b5930 2024-04-25T11:21:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.025+0000 7fce6d400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce680b0e00 con 0x7fce6815ef90 2024-04-25T11:21:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.025+0000 7fce6d400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce680afb80 con 0x7fce6815bc40 2024-04-25T11:21:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.025+0000 7fce67200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce680b5930 0x7fce680b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.025+0000 7fce67200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce680b5930 0x7fce680b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36288/0 (socket says 172.21.15.7:36288) 2024-04-25T11:21:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.025+0000 7fce67200640 1 -- 172.21.15.7:0/3915727662 learned_addr learned my addr 172.21.15.7:0/3915727662 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce5de00640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce6815ef90 0x7fce68159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce66800640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6815bc40 0x7fce681fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce67200640 1 -- 172.21.15.7:0/3915727662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6815bc40 msgr2=0x7fce681fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce67200640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6815bc40 0x7fce681fb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce67200640 1 -- 172.21.15.7:0/3915727662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce6815ef90 msgr2=0x7fce68159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce67200640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce6815ef90 0x7fce68159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce67200640 1 -- 172.21.15.7:0/3915727662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce680b6280 con 0x7fce680b5930 2024-04-25T11:21:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce67200640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce680b5930 0x7fce680b5d40 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fce60009870 tx=0x7fce600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce5f400640 1 -- 172.21.15.7:0/3915727662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60037040 con 0x7fce680b5930 2024-04-25T11:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce5f400640 1 -- 172.21.15.7:0/3915727662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce60042070 con 0x7fce680b5930 2024-04-25T11:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce5f400640 1 -- 172.21.15.7:0/3915727662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60030920 con 0x7fce680b5930 2024-04-25T11:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.026+0000 7fce6d400640 1 -- 172.21.15.7:0/3915727662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6815e050 con 0x7fce680b5930 2024-04-25T11:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.027+0000 7fce6d400640 1 -- 172.21.15.7:0/3915727662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6815c3c0 con 0x7fce680b5930 2024-04-25T11:21:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.028+0000 7fce6d400640 1 -- 172.21.15.7:0/3915727662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce6815ddc0 con 0x7fce680b5930 2024-04-25T11:21:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.029+0000 7fce5f400640 1 -- 172.21.15.7:0/3915727662 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fce6002ad10 con 0x7fce680b5930 2024-04-25T11:21:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.030+0000 7fce5f400640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744e0 0x7fce440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.030+0000 7fce5f400640 1 -- 172.21.15.7:0/3915727662 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7fce600b57c0 con 0x7fce680b5930 2024-04-25T11:21:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.030+0000 7fce66800640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744e0 0x7fce440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.031+0000 7fce66800640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744e0 0x7fce440769f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e90 tx=0x7fce5800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.035+0000 7fce5f400640 1 -- 172.21.15.7:0/3915727662 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fce680b6280 con 0x7fce680b5930 2024-04-25T11:21:28.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.238+0000 7fce6d400640 1 -- 172.21.15.7:0/3915727662 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce6806a100 con 0x7fce680b5930 2024-04-25T11:21:28.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.240+0000 7fce5f400640 1 -- 172.21.15.7:0/3915727662 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+13207 (secure 0 0 0) 0x7fce60080020 con 0x7fce680b5930 2024-04-25T11:21:28.236 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:28.236 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":133,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:21:23.658592+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6833/1243096872":"2024-04-26T11:21:20.635008+0000","172.21.15.87:6832/1243096872":"2024-04-26T11:21:20.635008+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:28.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.243+0000 7fce6d400640 1 -- 172.21.15.7:0/3915727662 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744e0 msgr2=0x7fce440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.243+0000 7fce6d400640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744e0 0x7fce440769f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fce58009e90 tx=0x7fce5800b040 comp rx=0 tx=0).stop 2024-04-25T11:21:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.243+0000 7fce6d400640 1 -- 172.21.15.7:0/3915727662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce680b5930 msgr2=0x7fce680b5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.243+0000 7fce6d400640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce680b5930 0x7fce680b5d40 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fce60009870 tx=0x7fce600046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.243+0000 7fce6d400640 1 -- 172.21.15.7:0/3915727662 shutdown_connections 2024-04-25T11:21:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.243+0000 7fce6d400640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fce440744e0 0x7fce440769f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.243+0000 7fce6d400640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce680b5930 0x7fce680b5d40 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.243+0000 7fce6d400640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce6815ef90 0x7fce68159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.243+0000 7fce6d400640 1 --2- 172.21.15.7:0/3915727662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce6815bc40 0x7fce681fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.243+0000 7fce6d400640 1 -- 172.21.15.7:0/3915727662 >> 172.21.15.7:0/3915727662 conn(0x7fce68074200 msgr2=0x7fce6806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.244+0000 7fce6d400640 1 -- 172.21.15.7:0/3915727662 shutdown_connections 2024-04-25T11:21:28.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.244+0000 7fce6d400640 1 -- 172.21.15.7:0/3915727662 wait complete. 2024-04-25T11:21:28.269 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:21:28.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:21:28.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.542+0000 7f09dee00640 1 Processor -- start 2024-04-25T11:21:28.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.542+0000 7f09dee00640 1 -- start start 2024-04-25T11:21:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.544+0000 7f09dee00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09d81583c0 0x7f09d80cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.544+0000 7f09dee00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09d806eea0 0x7f09d806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.544+0000 7f09dee00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09d80ce570 con 0x7f09d80cfb50 2024-04-25T11:21:28.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.544+0000 7f09dee00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f09d806f7f0 con 0x7f09d81583c0 2024-04-25T11:21:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.544+0000 7f09dee00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f09d815f190 con 0x7f09d806eea0 2024-04-25T11:21:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.545+0000 7f09dda00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09d81583c0 0x7f09d80cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.545+0000 7f09dd000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f09d80cfb50 0x7f09d80d0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58512/0 (socket says 172.21.15.7:58512) 2024-04-25T11:21:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.545+0000 7f09dd000640 1 -- 172.21.15.7:0/701943670 learned_addr learned my addr 172.21.15.7:0/701943670 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.545+0000 7f09de400640 1 --2- 172.21.15.7:0/701943670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09d806eea0 0x7f09d806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:28.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.545+0000 7f09dda00640 1 -- 172.21.15.7:0/701943670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09d806eea0 msgr2=0x7f09d806f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:28.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.545+0000 7f09dda00640 1 --2- 172.21.15.7:0/701943670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09d806eea0 0x7f09d806f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.545+0000 7f09dda00640 1 -- 172.21.15.7:0/701943670 >> v1:172.21.15.7:6789/0 conn(0x7f09d80cfb50 legacy=0x7f09d80d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.545+0000 7f09dda00640 1 -- 172.21.15.7:0/701943670 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d80ce850 con 0x7f09d81583c0 2024-04-25T11:21:28.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.547+0000 7f09dda00640 1 --2- 172.21.15.7:0/701943670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d81583c0 0x7f09d80cf610 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f09c80097a0 tx=0x7f09c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770a25ed9a594180 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:28.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.547+0000 7f09d7e00640 1 -- 172.21.15.7:0/701943670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8037070 con 0x7f09d81583c0 2024-04-25T11:21:28.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.547+0000 7f09d7e00640 1 -- 172.21.15.7:0/701943670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c8032440 con 0x7f09d81583c0 2024-04-25T11:21:28.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.547+0000 7f09d7e00640 1 -- 172.21.15.7:0/701943670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8032810 con 0x7f09d81583c0 2024-04-25T11:21:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.547+0000 7f09dee00640 1 -- 172.21.15.7:0/701943670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d81583c0 msgr2=0x7f09d80cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.547+0000 7f09dee00640 1 --2- 172.21.15.7:0/701943670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d81583c0 0x7f09d80cf610 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f09c80097a0 tx=0x7f09c802a180 comp rx=0 tx=0).stop 2024-04-25T11:21:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.548+0000 7f09dee00640 1 -- 172.21.15.7:0/701943670 shutdown_connections 2024-04-25T11:21:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.548+0000 7f09dee00640 1 --2- 172.21.15.7:0/701943670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09d806eea0 0x7f09d806f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.548+0000 7f09dee00640 1 --2- 172.21.15.7:0/701943670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d81583c0 0x7f09d80cf610 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.548+0000 7f09dee00640 1 -- 172.21.15.7:0/701943670 >> 172.21.15.7:0/701943670 conn(0x7f09d80740f0 msgr2=0x7f09d806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.548+0000 7f09dee00640 1 -- 172.21.15.7:0/701943670 shutdown_connections 2024-04-25T11:21:28.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.548+0000 7f09dee00640 1 -- 172.21.15.7:0/701943670 wait complete. 2024-04-25T11:21:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.550+0000 7f09dee00640 1 Processor -- start 2024-04-25T11:21:28.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.550+0000 7f09dee00640 1 -- start start 2024-04-25T11:21:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.552+0000 7f09dee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d806eea0 0x7f09d8141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.552+0000 7f09dee00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09d80cfb50 0x7f09d8151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.552+0000 7f09dee00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09d81583c0 0x7f09d813dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.552+0000 7f09dee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09d80cf0f0 con 0x7f09d806eea0 2024-04-25T11:21:28.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.552+0000 7f09dee00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f09d80cee10 con 0x7f09d80cfb50 2024-04-25T11:21:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.552+0000 7f09dee00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f09d80d0b90 con 0x7f09d81583c0 2024-04-25T11:21:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.552+0000 7f09dda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d806eea0 0x7f09d8141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.552+0000 7f09dda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d806eea0 0x7f09d8141a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36308/0 (socket says 172.21.15.7:36308) 2024-04-25T11:21:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.552+0000 7f09dda00640 1 -- 172.21.15.7:0/52441035 learned_addr learned my addr 172.21.15.7:0/52441035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.553+0000 7f09dda00640 1 -- 172.21.15.7:0/52441035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09d81583c0 msgr2=0x7f09d813dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.553+0000 7f09dda00640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09d81583c0 0x7f09d813dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.553+0000 7f09dda00640 1 -- 172.21.15.7:0/52441035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09d80cfb50 msgr2=0x7f09d8151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.553+0000 7f09dda00640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09d80cfb50 0x7f09d8151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.553+0000 7f09dda00640 1 -- 172.21.15.7:0/52441035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d813c020 con 0x7f09d806eea0 2024-04-25T11:21:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.553+0000 7f09dda00640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d806eea0 0x7f09d8141a40 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f09c80098c0 tx=0x7f09c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.553+0000 7f09d5600640 1 -- 172.21.15.7:0/52441035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8037040 con 0x7f09d806eea0 2024-04-25T11:21:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.553+0000 7f09d5600640 1 -- 172.21.15.7:0/52441035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c8030770 con 0x7f09d806eea0 2024-04-25T11:21:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.553+0000 7f09d5600640 1 -- 172.21.15.7:0/52441035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c8030b40 con 0x7f09d806eea0 2024-04-25T11:21:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.553+0000 7f09dee00640 1 -- 172.21.15.7:0/52441035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d813c300 con 0x7f09d806eea0 2024-04-25T11:21:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.553+0000 7f09dee00640 1 -- 172.21.15.7:0/52441035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d813c8c0 con 0x7f09d806eea0 2024-04-25T11:21:28.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.554+0000 7f09dee00640 1 -- 172.21.15.7:0/52441035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d8050740 con 0x7f09d806eea0 2024-04-25T11:21:28.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.556+0000 7f09d5600640 1 -- 172.21.15.7:0/52441035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f09c8044020 con 0x7f09d806eea0 2024-04-25T11:21:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.557+0000 7f09d5600640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f09b80744b0 0x7f09b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.557+0000 7f09d5600640 1 -- 172.21.15.7:0/52441035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7f09c80b5670 con 0x7f09d806eea0 2024-04-25T11:21:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.557+0000 7f09dd000640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f09b80744b0 0x7f09b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:28.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.558+0000 7f09dd000640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f09b80744b0 0x7f09b80769c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f09c4000f60 tx=0x7f09c4007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.562+0000 7f09d5600640 1 -- 172.21.15.7:0/52441035 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f09c8032e20 con 0x7f09d806eea0 2024-04-25T11:21:28.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.770+0000 7f09dee00640 1 -- 172.21.15.7:0/52441035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f09d80cf930 con 0x7f09d806eea0 2024-04-25T11:21:28.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.771+0000 7f09d5600640 1 -- 172.21.15.7:0/52441035 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) ==== 93+0+10 (secure 0 0 0) 0x7f09c807ff60 con 0x7f09d806eea0 2024-04-25T11:21:28.767 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:21:28.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.774+0000 7f09dee00640 1 -- 172.21.15.7:0/52441035 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f09b80744b0 msgr2=0x7f09b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.774+0000 7f09dee00640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f09b80744b0 0x7f09b80769c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f09c4000f60 tx=0x7f09c4007a80 comp rx=0 tx=0).stop 2024-04-25T11:21:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.774+0000 7f09dee00640 1 -- 172.21.15.7:0/52441035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d806eea0 msgr2=0x7f09d8141a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.774+0000 7f09dee00640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d806eea0 0x7f09d8141a40 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f09c80098c0 tx=0x7f09c80046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.774+0000 7f09dee00640 1 -- 172.21.15.7:0/52441035 shutdown_connections 2024-04-25T11:21:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.774+0000 7f09dee00640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f09b80744b0 0x7f09b80769c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.774+0000 7f09dee00640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09d81583c0 0x7f09d813dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.774+0000 7f09dee00640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09d80cfb50 0x7f09d8151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.774+0000 7f09dee00640 1 --2- 172.21.15.7:0/52441035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09d806eea0 0x7f09d8141a40 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.774+0000 7f09dee00640 1 -- 172.21.15.7:0/52441035 >> 172.21.15.7:0/52441035 conn(0x7f09d80740f0 msgr2=0x7f09d8153f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:28.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.774+0000 7f09dee00640 1 -- 172.21.15.7:0/52441035 shutdown_connections 2024-04-25T11:21:28.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:28.775+0000 7f09dee00640 1 -- 172.21.15.7:0/52441035 wait complete. 2024-04-25T11:21:28.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2024-04-25T11:21:29.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.087+0000 7f0630c00640 1 Processor -- start 2024-04-25T11:21:29.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.087+0000 7f0630c00640 1 -- start start 2024-04-25T11:21:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.090+0000 7f0630c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f062c15baa0 0x7f062c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.090+0000 7f0630c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c06d580 0x7f062c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.090+0000 7f0630c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f062c06ded0 con 0x7f062c070640 2024-04-25T11:21:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.090+0000 7f0630c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f062c15e980 con 0x7f062c15baa0 2024-04-25T11:21:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.090+0000 7f0630c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f062c15ec60 con 0x7f062c06d580 2024-04-25T11:21:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.090+0000 7f062aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f062c070640 0x7f062c161ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58524/0 (socket says 172.21.15.7:58524) 2024-04-25T11:21:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.090+0000 7f062aa00640 1 -- 172.21.15.7:0/310892429 learned_addr learned my addr 172.21.15.7:0/310892429 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:29.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062be00640 1 --2- 172.21.15.7:0/310892429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c06d580 0x7f062c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:29.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062b400640 1 --2- 172.21.15.7:0/310892429 >> v2:172.21.15.87:3300/0 conn(0x7f062c15baa0 0x7f062c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:29.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2752691125 0 0) 0x7f0618001110 con 0x7f062c070640 2024-04-25T11:21:29.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0610003520 con 0x7f062c070640 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1219784080 0 0) 0x7f0610003520 con 0x7f062c070640 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c06d580 msgr2=0x7f062c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 --2- 172.21.15.7:0/310892429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c06d580 0x7f062c06d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 >> v2:172.21.15.87:3300/0 conn(0x7f062c15baa0 msgr2=0x7f062c15e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 --2- 172.21.15.7:0/310892429 >> v2:172.21.15.87:3300/0 conn(0x7f062c15baa0 0x7f062c15e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062c15ef40 con 0x7f062c070640 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0618001860 con 0x7f062c070640 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 >> v1:172.21.15.7:6789/0 conn(0x7f062c070640 legacy=0x7f062c161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 --2- 172.21.15.7:0/310892429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c06d580 0x7f06100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 --2- 172.21.15.7:0/310892429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0610003f30 0x7f0610004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 --2- 172.21.15.7:0/310892429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0610004d40 0x7f0610005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f062c15e980 con 0x7f0610003f30 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.091+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f062c15ec60 con 0x7f0610004d40 2024-04-25T11:21:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.092+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f061000fe20 con 0x7f062c06d580 2024-04-25T11:21:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.092+0000 7f062aa00640 1 --2- 172.21.15.7:0/310892429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0610003f30 0x7f0610004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.092+0000 7f062b400640 1 --2- 172.21.15.7:0/310892429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c06d580 0x7f06100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.092+0000 7f062aa00640 1 -- 172.21.15.7:0/310892429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c06d580 msgr2=0x7f06100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.092+0000 7f062aa00640 1 --2- 172.21.15.7:0/310892429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c06d580 0x7f06100039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.092+0000 7f062aa00640 1 -- 172.21.15.7:0/310892429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0610004d40 msgr2=0x7f0610005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.092+0000 7f062aa00640 1 --2- 172.21.15.7:0/310892429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0610004d40 0x7f0610005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.092+0000 7f062aa00640 1 -- 172.21.15.7:0/310892429 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0610003520 con 0x7f0610003f30 2024-04-25T11:21:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.093+0000 7f062aa00640 1 --2- 172.21.15.7:0/310892429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0610003f30 0x7f0610004800 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f062c074950 tx=0x7f0610005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ddd8ac76032ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.094+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061800c0e0 con 0x7f0610003f30 2024-04-25T11:21:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.094+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0618034070 con 0x7f0610003f30 2024-04-25T11:21:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.094+0000 7f062a000640 1 -- 172.21.15.7:0/310892429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618005d10 con 0x7f0610003f30 2024-04-25T11:21:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.094+0000 7f0630c00640 1 -- 172.21.15.7:0/310892429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0610003f30 msgr2=0x7f0610004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.094+0000 7f0630c00640 1 --2- 172.21.15.7:0/310892429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0610003f30 0x7f0610004800 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f062c074950 tx=0x7f0610005830 comp rx=0 tx=0).stop 2024-04-25T11:21:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.095+0000 7f062be00640 1 -- 172.21.15.7:0/310892429 reap_dead start 2024-04-25T11:21:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.095+0000 7f0630c00640 1 -- 172.21.15.7:0/310892429 shutdown_connections 2024-04-25T11:21:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.095+0000 7f0630c00640 1 -- 172.21.15.7:0/310892429 >> 172.21.15.7:0/310892429 conn(0x7f062c073090 msgr2=0x7f062c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.095+0000 7f0630c00640 1 -- 172.21.15.7:0/310892429 shutdown_connections 2024-04-25T11:21:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.095+0000 7f0630c00640 1 -- 172.21.15.7:0/310892429 wait complete. 2024-04-25T11:21:29.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.097+0000 7f0630c00640 1 Processor -- start 2024-04-25T11:21:29.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.097+0000 7f0630c00640 1 -- start start 2024-04-25T11:21:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.099+0000 7f0630c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f062c143410 0x7f062c15a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.099+0000 7f0630c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f062c146060 0x7f062c146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.099+0000 7f0630c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c13f7c0 0x7f062c13fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.099+0000 7f0630c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f062c070bd0 con 0x7f062c146060 2024-04-25T11:21:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.099+0000 7f0630c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f062c070a50 con 0x7f062c143410 2024-04-25T11:21:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.099+0000 7f0630c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f062c1625f0 con 0x7f062c13f7c0 2024-04-25T11:21:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.099+0000 7f062aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f062c146060 0x7f062c146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.099+0000 7f062aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f062c146060 0x7f062c146470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36326/0 (socket says 172.21.15.7:36326) 2024-04-25T11:21:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.099+0000 7f062aa00640 1 -- 172.21.15.7:0/1861552171 learned_addr learned my addr 172.21.15.7:0/1861552171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.100+0000 7f062be00640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c13f7c0 0x7f062c13fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.100+0000 7f062b400640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f062c143410 0x7f062c15a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.100+0000 7f062aa00640 1 -- 172.21.15.7:0/1861552171 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c13f7c0 msgr2=0x7f062c13fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.100+0000 7f062aa00640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c13f7c0 0x7f062c13fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.100+0000 7f062aa00640 1 -- 172.21.15.7:0/1861552171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f062c143410 msgr2=0x7f062c15a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.100+0000 7f062aa00640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f062c143410 0x7f062c15a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.100+0000 7f062aa00640 1 -- 172.21.15.7:0/1861552171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062c15d680 con 0x7f062c146060 2024-04-25T11:21:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.100+0000 7f062aa00640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f062c146060 0x7f062c146470 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f061802af60 tx=0x7f0618001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.101+0000 7f0623400640 1 -- 172.21.15.7:0/1861552171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618007270 con 0x7f062c146060 2024-04-25T11:21:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.101+0000 7f0623400640 1 -- 172.21.15.7:0/1861552171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061803f070 con 0x7f062c146060 2024-04-25T11:21:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.101+0000 7f0623400640 1 -- 172.21.15.7:0/1861552171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618007650 con 0x7f062c146060 2024-04-25T11:21:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.101+0000 7f0630c00640 1 -- 172.21.15.7:0/1861552171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062c1469b0 con 0x7f062c146060 2024-04-25T11:21:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.101+0000 7f0630c00640 1 -- 172.21.15.7:0/1861552171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f062c15bff0 con 0x7f062c146060 2024-04-25T11:21:29.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.102+0000 7f0630c00640 1 -- 172.21.15.7:0/1861552171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f062c074020 con 0x7f062c146060 2024-04-25T11:21:29.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.103+0000 7f0623400640 1 -- 172.21.15.7:0/1861552171 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0618044470 con 0x7f062c146060 2024-04-25T11:21:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.104+0000 7f0623400640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f060c0744e0 0x7f060c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.104+0000 7f0623400640 1 -- 172.21.15.7:0/1861552171 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7f06180b5fe0 con 0x7f062c146060 2024-04-25T11:21:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.105+0000 7f062b400640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f060c0744e0 0x7f060c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:29.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.105+0000 7f062b400640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f060c0744e0 0x7f060c0769f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f062c1f5520 tx=0x7f061c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:29.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.110+0000 7f0623400640 1 -- 172.21.15.7:0/1861552171 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f062c06ded0 con 0x7f062c146060 2024-04-25T11:21:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.388+0000 7f0630c00640 1 -- 172.21.15.7:0/1861552171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) -- 0x7f062c147840 con 0x7f062c146060 2024-04-25T11:21:29.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.709+0000 7f0623400640 1 -- 172.21.15.7:0/1861552171 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v576) ==== 151+0+0 (secure 0 0 0) 0x7f06180808d0 con 0x7f062c146060 2024-04-25T11:21:29.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.712+0000 7f0630c00640 1 -- 172.21.15.7:0/1861552171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f060c0744e0 msgr2=0x7f060c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:29.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.712+0000 7f0630c00640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f060c0744e0 0x7f060c0769f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f062c1f5520 tx=0x7f061c00b040 comp rx=0 tx=0).stop 2024-04-25T11:21:29.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.712+0000 7f0630c00640 1 -- 172.21.15.7:0/1861552171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f062c146060 msgr2=0x7f062c146470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:29.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.712+0000 7f0630c00640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f062c146060 0x7f062c146470 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f061802af60 tx=0x7f0618001c30 comp rx=0 tx=0).stop 2024-04-25T11:21:29.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.713+0000 7f0630c00640 1 -- 172.21.15.7:0/1861552171 shutdown_connections 2024-04-25T11:21:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.713+0000 7f0630c00640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f060c0744e0 0x7f060c0769f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.713+0000 7f0630c00640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f062c13f7c0 0x7f062c13fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.713+0000 7f0630c00640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f062c146060 0x7f062c146470 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.713+0000 7f0630c00640 1 --2- 172.21.15.7:0/1861552171 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f062c143410 0x7f062c15a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.713+0000 7f0630c00640 1 -- 172.21.15.7:0/1861552171 >> 172.21.15.7:0/1861552171 conn(0x7f062c073090 msgr2=0x7f062c0b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.713+0000 7f0630c00640 1 -- 172.21.15.7:0/1861552171 shutdown_connections 2024-04-25T11:21:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:29.713+0000 7f0630c00640 1 -- 172.21.15.7:0/1861552171 wait complete. 2024-04-25T11:21:29.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:21:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.043+0000 7f782ee00640 1 Processor -- start 2024-04-25T11:21:30.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.044+0000 7f782ee00640 1 -- start start 2024-04-25T11:21:30.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.046+0000 7f782ee00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f782815dfb0 0x7f78280b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:30.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.046+0000 7f782ee00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f782815e3c0 con 0x7f78280d0b40 2024-04-25T11:21:30.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.046+0000 7f782ee00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78280cf800 con 0x7f782806fab0 2024-04-25T11:21:30.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.046+0000 7f782ee00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f78280b09b0 con 0x7f782815dfb0 2024-04-25T11:21:30.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.047+0000 7f7827e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f782815dfb0 0x7f78280b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:30.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.047+0000 7f7827e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f782815dfb0 0x7f78280b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50726/0 (socket says 172.21.15.7:50726) 2024-04-25T11:21:30.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.047+0000 7f7827e00640 1 -- 172.21.15.7:0/3561256739 learned_addr learned my addr 172.21.15.7:0/3561256739 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:30.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.047+0000 7f7827e00640 1 -- 172.21.15.7:0/3561256739 >> v1:172.21.15.87:6789/0 conn(0x7f782806fab0 legacy=0x7f78280aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:30.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.047+0000 7f7827e00640 1 -- 172.21.15.7:0/3561256739 >> v1:172.21.15.7:6789/0 conn(0x7f78280d0b40 legacy=0x7f782806f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:30.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.047+0000 7f7827e00640 1 -- 172.21.15.7:0/3561256739 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78280cfaa0 con 0x7f782815dfb0 2024-04-25T11:21:30.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.049+0000 7f7827e00640 1 --2- 172.21.15.7:0/3561256739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782815dfb0 0x7f78280b0470 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f78280ceb10 tx=0x7f781802a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf2eab8a2575275 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:30.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.049+0000 7f7827400640 1 -- 172.21.15.7:0/3561256739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818037070 con 0x7f782815dfb0 2024-04-25T11:21:30.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.049+0000 7f7827400640 1 -- 172.21.15.7:0/3561256739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7818032640 con 0x7f782815dfb0 2024-04-25T11:21:30.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.049+0000 7f7827400640 1 -- 172.21.15.7:0/3561256739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818032a10 con 0x7f782815dfb0 2024-04-25T11:21:30.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.050+0000 7f782ee00640 1 -- 172.21.15.7:0/3561256739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782815dfb0 msgr2=0x7f78280b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:30.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.050+0000 7f782ee00640 1 --2- 172.21.15.7:0/3561256739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782815dfb0 0x7f78280b0470 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f78280ceb10 tx=0x7f781802a2e0 comp rx=0 tx=0).stop 2024-04-25T11:21:30.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.050+0000 7f782ee00640 1 -- 172.21.15.7:0/3561256739 shutdown_connections 2024-04-25T11:21:30.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.050+0000 7f782ee00640 1 --2- 172.21.15.7:0/3561256739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782815dfb0 0x7f78280b0470 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:30.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.050+0000 7f782ee00640 1 -- 172.21.15.7:0/3561256739 >> 172.21.15.7:0/3561256739 conn(0x7f78280742c0 msgr2=0x7f782806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:30.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.050+0000 7f782ee00640 1 -- 172.21.15.7:0/3561256739 shutdown_connections 2024-04-25T11:21:30.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.050+0000 7f782ee00640 1 -- 172.21.15.7:0/3561256739 wait complete. 2024-04-25T11:21:30.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.053+0000 7f782ee00640 1 Processor -- start 2024-04-25T11:21:30.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.053+0000 7f782ee00640 1 -- start start 2024-04-25T11:21:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.054+0000 7f782ee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782806fab0 0x7f7828151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.054+0000 7f782ee00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78280d0b40 0x7f78281fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.054+0000 7f782ee00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f782815dfb0 0x7f782815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:30.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.054+0000 7f782ee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f78280d0340 con 0x7f782806fab0 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.054+0000 7f782ee00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f78280d0060 con 0x7f782815dfb0 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.054+0000 7f782ee00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f78280d0620 con 0x7f78280d0b40 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f7827e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782806fab0 0x7f7828151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f7827e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782806fab0 0x7f7828151e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50734/0 (socket says 172.21.15.7:50734) 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f7827e00640 1 -- 172.21.15.7:0/3010967719 learned_addr learned my addr 172.21.15.7:0/3010967719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f781fe00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78280d0b40 0x7f78281fa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f7827e00640 1 -- 172.21.15.7:0/3010967719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78280d0b40 msgr2=0x7f78281fa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f7827e00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78280d0b40 0x7f78281fa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f782ce00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f782815dfb0 0x7f782815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f7827e00640 1 -- 172.21.15.7:0/3010967719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f782815dfb0 msgr2=0x7f782815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f7827e00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f782815dfb0 0x7f782815cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f7827e00640 1 -- 172.21.15.7:0/3010967719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7828152bd0 con 0x7f782806fab0 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f781fe00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78280d0b40 0x7f78281fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.055+0000 7f782ce00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f782815dfb0 0x7f782815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:30.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.056+0000 7f7827e00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782806fab0 0x7f7828151e80 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7818000c00 tx=0x7f78180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:30.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.056+0000 7f7824c00640 1 -- 172.21.15.7:0/3010967719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818041020 con 0x7f782806fab0 2024-04-25T11:21:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.056+0000 7f7824c00640 1 -- 172.21.15.7:0/3010967719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7818030970 con 0x7f782806fab0 2024-04-25T11:21:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.056+0000 7f782ee00640 1 -- 172.21.15.7:0/3010967719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78280cad30 con 0x7f782806fab0 2024-04-25T11:21:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.056+0000 7f7824c00640 1 -- 172.21.15.7:0/3010967719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7818037040 con 0x7f782806fab0 2024-04-25T11:21:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.056+0000 7f782ee00640 1 -- 172.21.15.7:0/3010967719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78280cb2f0 con 0x7f782806fab0 2024-04-25T11:21:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.057+0000 7f782ee00640 1 -- 172.21.15.7:0/3010967719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7828050740 con 0x7f782806fab0 2024-04-25T11:21:30.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.059+0000 7f7824c00640 1 -- 172.21.15.7:0/3010967719 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7818044020 con 0x7f782806fab0 2024-04-25T11:21:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.060+0000 7f7824c00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78040744e0 0x7f78040769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.060+0000 7f7824c00640 1 -- 172.21.15.7:0/3010967719 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7f78180b5b50 con 0x7f782806fab0 2024-04-25T11:21:30.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.061+0000 7f781fe00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78040744e0 0x7f78040769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:30.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.061+0000 7f781fe00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78040744e0 0x7f78040769f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f782806c9c0 tx=0x7f7810009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:30.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.065+0000 7f7824c00640 1 -- 172.21.15.7:0/3010967719 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f781804b030 con 0x7f782806fab0 2024-04-25T11:21:30.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.268+0000 7f782ee00640 1 -- 172.21.15.7:0/3010967719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f78280cd010 con 0x7f782806fab0 2024-04-25T11:21:30.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.275+0000 7f7824c00640 1 -- 172.21.15.7:0/3010967719 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v56)=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7f78180803b0 con 0x7f782806fab0 2024-04-25T11:21:30.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 -- 172.21.15.7:0/3010967719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78040744e0 msgr2=0x7f78040769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:30.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78040744e0 0x7f78040769f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f782806c9c0 tx=0x7f7810009040 comp rx=0 tx=0).stop 2024-04-25T11:21:30.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 -- 172.21.15.7:0/3010967719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782806fab0 msgr2=0x7f7828151e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:30.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782806fab0 0x7f7828151e80 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7818000c00 tx=0x7f78180046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:30.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 -- 172.21.15.7:0/3010967719 shutdown_connections 2024-04-25T11:21:30.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f78040744e0 0x7f78040769f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:30.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f782815dfb0 0x7f782815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78280d0b40 0x7f78281fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 --2- 172.21.15.7:0/3010967719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782806fab0 0x7f7828151e80 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 -- 172.21.15.7:0/3010967719 >> 172.21.15.7:0/3010967719 conn(0x7f78280742c0 msgr2=0x7f782806d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 -- 172.21.15.7:0/3010967719 shutdown_connections 2024-04-25T11:21:30.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:30.278+0000 7f782ee00640 1 -- 172.21.15.7:0/3010967719 wait complete. 2024-04-25T11:21:30.302 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:21:30.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:21:30.414 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:21:30.519 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:21:30.519 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:21:30.519 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:30.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:21:30.576 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:21:30.577 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:30.630 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:21:30.631 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:21:30.631 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:30.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:21:30.691 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:21:30.691 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:30.747 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:21:30.747 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:21:30.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:21:31.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.044+0000 7f27f0400640 1 Processor -- start 2024-04-25T11:21:31.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.044+0000 7f27f0400640 1 -- start start 2024-04-25T11:21:31.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.047+0000 7f27f0400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f27e815ef90 0x7f27e809f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.047+0000 7f27f0400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815bc40 0x7f27e80aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.047+0000 7f27f0400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27e815f4e0 con 0x7f27e80af370 2024-04-25T11:21:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.047+0000 7f27f0400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f27e8071190 con 0x7f27e815ef90 2024-04-25T11:21:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.047+0000 7f27f0400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f27e80af780 con 0x7f27e815bc40 2024-04-25T11:21:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.047+0000 7f27ed000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815bc40 0x7f27e80aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.048+0000 7f27ed000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815bc40 0x7f27e80aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:52376/0 (socket says 172.21.15.7:52376) 2024-04-25T11:21:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.048+0000 7f27ed000640 1 -- 172.21.15.7:0/3024862325 learned_addr learned my addr 172.21.15.7:0/3024862325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.048+0000 7f27eda00640 1 --2- 172.21.15.7:0/3024862325 >> v2:172.21.15.87:3300/0 conn(0x7f27e815ef90 0x7f27e809f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.048+0000 7f27ed000640 1 -- 172.21.15.7:0/3024862325 >> v2:172.21.15.87:3300/0 conn(0x7f27e815ef90 msgr2=0x7f27e809f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.048+0000 7f27ed000640 1 --2- 172.21.15.7:0/3024862325 >> v2:172.21.15.87:3300/0 conn(0x7f27e815ef90 0x7f27e809f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.049+0000 7f27ed000640 1 -- 172.21.15.7:0/3024862325 >> v1:172.21.15.87:6789/0 conn(0x7f27e80af370 legacy=0x7f27e8161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.049+0000 7f27ed000640 1 -- 172.21.15.7:0/3024862325 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e80b0560 con 0x7f27e815bc40 2024-04-25T11:21:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.050+0000 7f27ed000640 1 --2- 172.21.15.7:0/3024862325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815bc40 0x7f27e80aee30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009870 tx=0x7f27d402a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa44bf4f05c88caf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.050+0000 7f27e7e00640 1 -- 172.21.15.7:0/3024862325 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4037070 con 0x7f27e815bc40 2024-04-25T11:21:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.050+0000 7f27e7e00640 1 -- 172.21.15.7:0/3024862325 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d4032440 con 0x7f27e815bc40 2024-04-25T11:21:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.050+0000 7f27e7e00640 1 -- 172.21.15.7:0/3024862325 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4032810 con 0x7f27e815bc40 2024-04-25T11:21:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.051+0000 7f27f0400640 1 -- 172.21.15.7:0/3024862325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815bc40 msgr2=0x7f27e80aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.051+0000 7f27f0400640 1 --2- 172.21.15.7:0/3024862325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815bc40 0x7f27e80aee30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009870 tx=0x7f27d402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.051+0000 7f27f0400640 1 -- 172.21.15.7:0/3024862325 shutdown_connections 2024-04-25T11:21:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.051+0000 7f27f0400640 1 --2- 172.21.15.7:0/3024862325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815bc40 0x7f27e80aee30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.051+0000 7f27f0400640 1 --2- 172.21.15.7:0/3024862325 >> v2:172.21.15.87:3300/0 conn(0x7f27e815ef90 0x7f27e809f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.051+0000 7f27f0400640 1 -- 172.21.15.7:0/3024862325 >> 172.21.15.7:0/3024862325 conn(0x7f27e8074200 msgr2=0x7f27e806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.051+0000 7f27f0400640 1 -- 172.21.15.7:0/3024862325 shutdown_connections 2024-04-25T11:21:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.051+0000 7f27f0400640 1 -- 172.21.15.7:0/3024862325 wait complete. 2024-04-25T11:21:31.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.054+0000 7f27f0400640 1 Processor -- start 2024-04-25T11:21:31.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.054+0000 7f27f0400640 1 -- start start 2024-04-25T11:21:31.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27f0400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e80af370 0x7f27e80cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27f0400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e815bc40 0x7f27e80b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27f0400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815ef90 0x7f27e80d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27f0400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f27e80b0e00 con 0x7f27e815bc40 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27f0400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27e80b0b20 con 0x7f27e80af370 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27f0400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f27e80b1320 con 0x7f27e815ef90 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27ed000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e815bc40 0x7f27e80b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27ed000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e815bc40 0x7f27e80b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50746/0 (socket says 172.21.15.7:50746) 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27ed000640 1 -- 172.21.15.7:0/2407735250 learned_addr learned my addr 172.21.15.7:0/2407735250 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27eda00640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e80af370 0x7f27e80cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27ee400640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815ef90 0x7f27e80d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.056+0000 7f27ed000640 1 -- 172.21.15.7:0/2407735250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815ef90 msgr2=0x7f27e80d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.057+0000 7f27ed000640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815ef90 0x7f27e80d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.057+0000 7f27ed000640 1 -- 172.21.15.7:0/2407735250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e80af370 msgr2=0x7f27e80cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.057+0000 7f27ed000640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e80af370 0x7f27e80cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.057+0000 7f27ed000640 1 -- 172.21.15.7:0/2407735250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e80b5db0 con 0x7f27e815bc40 2024-04-25T11:21:31.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.057+0000 7f27ed000640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e815bc40 0x7f27e80b5130 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f27d4000c00 tx=0x7f27d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:31.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.057+0000 7f27e5600640 1 -- 172.21.15.7:0/2407735250 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4037040 con 0x7f27e815bc40 2024-04-25T11:21:31.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.057+0000 7f27e5600640 1 -- 172.21.15.7:0/2407735250 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27d4042070 con 0x7f27e815bc40 2024-04-25T11:21:31.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.057+0000 7f27e5600640 1 -- 172.21.15.7:0/2407735250 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4030860 con 0x7f27e815bc40 2024-04-25T11:21:31.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.057+0000 7f27f0400640 1 -- 172.21.15.7:0/2407735250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e80b6090 con 0x7f27e815bc40 2024-04-25T11:21:31.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.057+0000 7f27f0400640 1 -- 172.21.15.7:0/2407735250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e80b6650 con 0x7f27e815bc40 2024-04-25T11:21:31.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.059+0000 7f27f0400640 1 -- 172.21.15.7:0/2407735250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e80b5b20 con 0x7f27e815bc40 2024-04-25T11:21:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.060+0000 7f27e5600640 1 -- 172.21.15.7:0/2407735250 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f27d4030a00 con 0x7f27e815bc40 2024-04-25T11:21:31.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.061+0000 7f27e5600640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27c80744b0 0x7f27c80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.061+0000 7f27e5600640 1 -- 172.21.15.7:0/2407735250 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7f27d40b5ac0 con 0x7f27e815bc40 2024-04-25T11:21:31.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.062+0000 7f27eda00640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27c80744b0 0x7f27c80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.062+0000 7f27eda00640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27c80744b0 0x7f27c80769c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009a30 tx=0x7f27dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:31.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.067+0000 7f27e5600640 1 -- 172.21.15.7:0/2407735250 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f27d4080320 con 0x7f27e815bc40 2024-04-25T11:21:31.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.307+0000 7f27f0400640 1 -- 172.21.15.7:0/2407735250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27e806a100 con 0x7f27e815bc40 2024-04-25T11:21:31.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.308+0000 7f27e5600640 1 -- 172.21.15.7:0/2407735250 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+3860 (secure 0 0 0) 0x7f27d407fc20 con 0x7f27e815bc40 2024-04-25T11:21:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:31.304 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":96,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2288158899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":2288158899},{"type":"v1","addr":"172.21.15.7:6837","nonce":2288158899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6242,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3545919245","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3545919245},{"type":"v1","addr":"172.21.15.87:6835","nonce":3545919245}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":7216,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/1399644840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1399644840},{"type":"v1","addr":"172.21.15.87:6833","nonce":1399644840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8478,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/951407928","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":951407928},{"type":"v1","addr":"172.21.15.7:6836","nonce":951407928}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84}],"filesystems":[]} 2024-04-25T11:21:31.305 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 96 2024-04-25T11:21:31.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.310+0000 7f27f0400640 1 -- 172.21.15.7:0/2407735250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27c80744b0 msgr2=0x7f27c80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.310+0000 7f27f0400640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27c80744b0 0x7f27c80769c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009a30 tx=0x7f27dc008040 comp rx=0 tx=0).stop 2024-04-25T11:21:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.310+0000 7f27f0400640 1 -- 172.21.15.7:0/2407735250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e815bc40 msgr2=0x7f27e80b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.310+0000 7f27f0400640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e815bc40 0x7f27e80b5130 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f27d4000c00 tx=0x7f27d40046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.310+0000 7f27f0400640 1 -- 172.21.15.7:0/2407735250 shutdown_connections 2024-04-25T11:21:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.310+0000 7f27f0400640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f27c80744b0 0x7f27c80769c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.310+0000 7f27f0400640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27e815ef90 0x7f27e80d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.310+0000 7f27f0400640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27e815bc40 0x7f27e80b5130 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.310+0000 7f27f0400640 1 --2- 172.21.15.7:0/2407735250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27e80af370 0x7f27e80cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.310+0000 7f27f0400640 1 -- 172.21.15.7:0/2407735250 >> 172.21.15.7:0/2407735250 conn(0x7f27e8074200 msgr2=0x7f27e8159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.311+0000 7f27f0400640 1 -- 172.21.15.7:0/2407735250 shutdown_connections 2024-04-25T11:21:31.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.311+0000 7f27f0400640 1 -- 172.21.15.7:0/2407735250 wait complete. 2024-04-25T11:21:31.335 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:21:31.335 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:21:31.335 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:21:31.335 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:31.335 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:21:31.335 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:21:31.336 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:21:31.336 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:31.336 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:21:31.336 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:21:31.336 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:21:31.337 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:31.337 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:21:31.337 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:21:31.337 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:21:31.337 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:21:31.359 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:21:31.359 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:21:31.374 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:21:31.375 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:21:31.377 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:21:31.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:21:31.379 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:21:31.379 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:21:31.421 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:21:31.422 DEBUG:teuthology.parallel:result is None 2024-04-25T11:21:31.426 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:21:31.427 DEBUG:teuthology.parallel:result is None 2024-04-25T11:21:31.428 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:21:31.428 DEBUG:teuthology.parallel:result is None 2024-04-25T11:21:31.433 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:21:31.433 DEBUG:teuthology.parallel:result is None 2024-04-25T11:21:31.433 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:21:31.633 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:21:31.640 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:21:31.660 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:21:31.661 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:21:31.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.683+0000 7fabaa800640 1 Processor -- start 2024-04-25T11:21:31.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.683+0000 7fabaa800640 1 -- start start 2024-04-25T11:21:31.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.685+0000 7fabaa800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faba40d04c0 0x7faba4163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.685+0000 7fabaa800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faba40bae70 0x7faba40b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.685+0000 7fabaa800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faba40b7430 0x7faba40b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.685+0000 7fabaa800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7faba40b7d80 con 0x7faba40bae70 2024-04-25T11:21:31.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.685+0000 7fabaa800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7faba4164980 con 0x7faba40b7430 2024-04-25T11:21:31.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.685+0000 7fabaa800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faba41640e0 con 0x7faba40d04c0 2024-04-25T11:21:31.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.685+0000 7faba8a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faba40bae70 0x7faba40b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.685+0000 7faba9e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faba40b7430 0x7faba40b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.685+0000 7faba9400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faba40d04c0 0x7faba4163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.685+0000 7faba8a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faba40bae70 0x7faba40b8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50786/0 (socket says 172.21.15.7:50786) 2024-04-25T11:21:31.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.686+0000 7faba8a00640 1 -- 172.21.15.7:0/1429111158 learned_addr learned my addr 172.21.15.7:0/1429111158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:31.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.686+0000 7faba8a00640 1 -- 172.21.15.7:0/1429111158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faba40d04c0 msgr2=0x7faba4163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:31.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.686+0000 7faba8a00640 1 --2- 172.21.15.7:0/1429111158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faba40d04c0 0x7faba4163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.686+0000 7faba8a00640 1 -- 172.21.15.7:0/1429111158 >> v2:172.21.15.87:3300/0 conn(0x7faba40b7430 msgr2=0x7faba40b7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:31.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.686+0000 7faba8a00640 1 --2- 172.21.15.7:0/1429111158 >> v2:172.21.15.87:3300/0 conn(0x7faba40b7430 0x7faba40b7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.686+0000 7faba8a00640 1 -- 172.21.15.7:0/1429111158 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba41643c0 con 0x7faba40bae70 2024-04-25T11:21:31.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.686+0000 7faba9400640 1 --2- 172.21.15.7:0/1429111158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faba40d04c0 0x7faba4163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.687+0000 7faba8a00640 1 --2- 172.21.15.7:0/1429111158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba40bae70 0x7faba40b8e10 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7faba406d8b0 tx=0x7fab9002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd37a0a8c9712f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.687+0000 7faba3e00640 1 -- 172.21.15.7:0/1429111158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90037070 con 0x7faba40bae70 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.687+0000 7faba3e00640 1 -- 172.21.15.7:0/1429111158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab90032440 con 0x7faba40bae70 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.687+0000 7faba3e00640 1 -- 172.21.15.7:0/1429111158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90032810 con 0x7faba40bae70 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.688+0000 7fabaa800640 1 -- 172.21.15.7:0/1429111158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba40bae70 msgr2=0x7faba40b8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.688+0000 7fabaa800640 1 --2- 172.21.15.7:0/1429111158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba40bae70 0x7faba40b8e10 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7faba406d8b0 tx=0x7fab9002a140 comp rx=0 tx=0).stop 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.688+0000 7fabaa800640 1 -- 172.21.15.7:0/1429111158 shutdown_connections 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.688+0000 7fabaa800640 1 --2- 172.21.15.7:0/1429111158 >> v2:172.21.15.87:3300/0 conn(0x7faba40b7430 0x7faba40b7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.688+0000 7fabaa800640 1 --2- 172.21.15.7:0/1429111158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba40bae70 0x7faba40b8e10 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.688+0000 7fabaa800640 1 --2- 172.21.15.7:0/1429111158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faba40d04c0 0x7faba4163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.688+0000 7fabaa800640 1 -- 172.21.15.7:0/1429111158 >> 172.21.15.7:0/1429111158 conn(0x7faba40d2610 msgr2=0x7faba40d2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.688+0000 7fabaa800640 1 -- 172.21.15.7:0/1429111158 shutdown_connections 2024-04-25T11:21:31.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.688+0000 7fabaa800640 1 -- 172.21.15.7:0/1429111158 wait complete. 2024-04-25T11:21:31.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.691+0000 7fabaa800640 1 Processor -- start 2024-04-25T11:21:31.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.691+0000 7fabaa800640 1 -- start start 2024-04-25T11:21:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.692+0000 7fabaa800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faba40b7430 0x7faba4155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.692+0000 7fabaa800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faba40bae70 0x7faba40cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.692+0000 7fabaa800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba40d04c0 0x7faba41eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.692+0000 7fabaa800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faba40b51c0 con 0x7faba40d04c0 2024-04-25T11:21:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.692+0000 7fabaa800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faba40d4210 con 0x7faba40b7430 2024-04-25T11:21:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.692+0000 7fabaa800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faba40b9350 con 0x7faba40bae70 2024-04-25T11:21:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.692+0000 7faba9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba40d04c0 0x7faba41eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.693+0000 7faba9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba40d04c0 0x7faba41eb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50790/0 (socket says 172.21.15.7:50790) 2024-04-25T11:21:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.693+0000 7faba9e00640 1 -- 172.21.15.7:0/3086049574 learned_addr learned my addr 172.21.15.7:0/3086049574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:31.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.693+0000 7faba9400640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faba40b7430 0x7faba4155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.693+0000 7faba8a00640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faba40bae70 0x7faba40cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.693+0000 7faba9e00640 1 -- 172.21.15.7:0/3086049574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faba40bae70 msgr2=0x7faba40cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:31.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.693+0000 7faba9e00640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faba40bae70 0x7faba40cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.693+0000 7faba9e00640 1 -- 172.21.15.7:0/3086049574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faba40b7430 msgr2=0x7faba4155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:31.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.693+0000 7faba9e00640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faba40b7430 0x7faba4155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:31.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.693+0000 7faba9e00640 1 -- 172.21.15.7:0/3086049574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba4156390 con 0x7faba40d04c0 2024-04-25T11:21:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.693+0000 7faba9e00640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba40d04c0 0x7faba41eb9e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009d30 tx=0x7fab9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.694+0000 7faba1600640 1 -- 172.21.15.7:0/3086049574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c018020 con 0x7faba40d04c0 2024-04-25T11:21:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.694+0000 7faba1600640 1 -- 172.21.15.7:0/3086049574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9c014480 con 0x7faba40d04c0 2024-04-25T11:21:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.694+0000 7fabaa800640 1 -- 172.21.15.7:0/3086049574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba40cd500 con 0x7faba40d04c0 2024-04-25T11:21:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.694+0000 7faba1600640 1 -- 172.21.15.7:0/3086049574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c014870 con 0x7faba40d04c0 2024-04-25T11:21:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.694+0000 7fabaa800640 1 -- 172.21.15.7:0/3086049574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba40cdfa0 con 0x7faba40d04c0 2024-04-25T11:21:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.695+0000 7fabaa800640 1 -- 172.21.15.7:0/3086049574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba40cd270 con 0x7faba40d04c0 2024-04-25T11:21:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.696+0000 7faba1600640 1 -- 172.21.15.7:0/3086049574 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fab9c005d10 con 0x7faba40d04c0 2024-04-25T11:21:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.697+0000 7faba1600640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab840744e0 0x7fab840769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.697+0000 7faba1600640 1 -- 172.21.15.7:0/3086049574 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5335+0+0 (secure 0 0 0) 0x7fab9c0959b0 con 0x7faba40d04c0 2024-04-25T11:21:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.698+0000 7faba9400640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab840744e0 0x7fab840769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.698+0000 7faba9400640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab840744e0 0x7fab840769f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fab98009560 tx=0x7fab98009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.702+0000 7faba1600640 1 -- 172.21.15.7:0/3086049574 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab9c022070 con 0x7faba40d04c0 2024-04-25T11:21:31.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:31.918+0000 7fabaa800640 1 -- 172.21.15.7:0/3086049574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7faba4050740 con 0x7faba40d04c0 2024-04-25T11:21:32.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.773+0000 7faba1600640 1 -- 172.21.15.7:0/3086049574 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v138) ==== 97+0+0 (secure 0 0 0) 0x7faba4156390 con 0x7faba40d04c0 2024-04-25T11:21:32.769 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-04-25T11:21:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.776+0000 7fabaa800640 1 -- 172.21.15.7:0/3086049574 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab840744e0 msgr2=0x7fab840769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.776+0000 7fabaa800640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab840744e0 0x7fab840769f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fab98009560 tx=0x7fab98009190 comp rx=0 tx=0).stop 2024-04-25T11:21:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.776+0000 7fabaa800640 1 -- 172.21.15.7:0/3086049574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba40d04c0 msgr2=0x7faba41eb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.776+0000 7fabaa800640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba40d04c0 0x7faba41eb9e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009d30 tx=0x7fab9c00c490 comp rx=0 tx=0).stop 2024-04-25T11:21:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.777+0000 7fabaa800640 1 -- 172.21.15.7:0/3086049574 shutdown_connections 2024-04-25T11:21:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.777+0000 7fabaa800640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab840744e0 0x7fab840769f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.777+0000 7fabaa800640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba40d04c0 0x7faba41eb9e0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.777+0000 7fabaa800640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faba40bae70 0x7faba40cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.777+0000 7fabaa800640 1 --2- 172.21.15.7:0/3086049574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faba40b7430 0x7faba4155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.777+0000 7fabaa800640 1 -- 172.21.15.7:0/3086049574 >> 172.21.15.7:0/3086049574 conn(0x7faba40d2610 msgr2=0x7faba40b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.777+0000 7fabaa800640 1 -- 172.21.15.7:0/3086049574 shutdown_connections 2024-04-25T11:21:32.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:32.777+0000 7fabaa800640 1 -- 172.21.15.7:0/3086049574 wait complete. 2024-04-25T11:21:32.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:21:33.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.073+0000 7fa6ffa00640 1 Processor -- start 2024-04-25T11:21:33.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.073+0000 7fa6ffa00640 1 -- start start 2024-04-25T11:21:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.076+0000 7fa6ffa00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa6f8162330 0x7fa6f81683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.076+0000 7fa6ffa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6f8168910 0x7fa6f80cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.076+0000 7fa6ffa00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f8162740 con 0x7fa6f80cff20 2024-04-25T11:21:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.076+0000 7fa6ffa00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f809b6e0 con 0x7fa6f8162330 2024-04-25T11:21:33.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.076+0000 7fa6ffa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f809b9c0 con 0x7fa6f8168910 2024-04-25T11:21:33.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.077+0000 7fa6fd000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa6f8162330 0x7fa6f81683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.077+0000 7fa6efe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6f8168910 0x7fa6f80cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.077+0000 7fa6fd000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa6f8162330 0x7fa6f81683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:47050/0 (socket says 172.21.15.7:47050) 2024-04-25T11:21:33.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.077+0000 7fa6fd000640 1 -- 172.21.15.7:0/965176235 learned_addr learned my addr 172.21.15.7:0/965176235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:33.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.078+0000 7fa6fd000640 1 -- 172.21.15.7:0/965176235 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6f8168910 msgr2=0x7fa6f80cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:33.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.078+0000 7fa6fd000640 1 --2- 172.21.15.7:0/965176235 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6f8168910 0x7fa6f80cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.078+0000 7fa6fd000640 1 -- 172.21.15.7:0/965176235 >> v1:172.21.15.87:6789/0 conn(0x7fa6f80cff20 legacy=0x7fa6f8165370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.078+0000 7fa6fd000640 1 -- 172.21.15.7:0/965176235 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f809bca0 con 0x7fa6f8162330 2024-04-25T11:21:33.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.078+0000 7fa6efe00640 1 --2- 172.21.15.7:0/965176235 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6f8168910 0x7fa6f80cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:21:33.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.079+0000 7fa6fd000640 1 --2- 172.21.15.7:0/965176235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6f8162330 0x7fa6f81683d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009870 tx=0x7fa6e802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=110f72ac5625be84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:33.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.080+0000 7fa6f7e00640 1 -- 172.21.15.7:0/965176235 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8037070 con 0x7fa6f8162330 2024-04-25T11:21:33.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.080+0000 7fa6f7e00640 1 -- 172.21.15.7:0/965176235 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e8032440 con 0x7fa6f8162330 2024-04-25T11:21:33.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.080+0000 7fa6f7e00640 1 -- 172.21.15.7:0/965176235 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8032810 con 0x7fa6f8162330 2024-04-25T11:21:33.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.081+0000 7fa6ffa00640 1 -- 172.21.15.7:0/965176235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6f8162330 msgr2=0x7fa6f81683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.081+0000 7fa6ffa00640 1 --2- 172.21.15.7:0/965176235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6f8162330 0x7fa6f81683d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009870 tx=0x7fa6e802a250 comp rx=0 tx=0).stop 2024-04-25T11:21:33.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.081+0000 7fa6ffa00640 1 -- 172.21.15.7:0/965176235 shutdown_connections 2024-04-25T11:21:33.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.081+0000 7fa6ffa00640 1 --2- 172.21.15.7:0/965176235 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6f8168910 0x7fa6f80cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.081+0000 7fa6ffa00640 1 --2- 172.21.15.7:0/965176235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6f8162330 0x7fa6f81683d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.081+0000 7fa6ffa00640 1 -- 172.21.15.7:0/965176235 >> 172.21.15.7:0/965176235 conn(0x7fa6f80b7620 msgr2=0x7fa6f80b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:33.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.081+0000 7fa6ffa00640 1 -- 172.21.15.7:0/965176235 shutdown_connections 2024-04-25T11:21:33.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.081+0000 7fa6ffa00640 1 -- 172.21.15.7:0/965176235 wait complete. 2024-04-25T11:21:33.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.083+0000 7fa6ffa00640 1 Processor -- start 2024-04-25T11:21:33.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.083+0000 7fa6ffa00640 1 -- start start 2024-04-25T11:21:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.085+0000 7fa6ffa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f80cff20 0x7fa6f81fea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.085+0000 7fa6ffa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6f8162330 0x7fa6f8168050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.085+0000 7fa6ffa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6f8168910 0x7fa6f8071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.085+0000 7fa6ffa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8165a80 con 0x7fa6f80cff20 2024-04-25T11:21:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.085+0000 7fa6ffa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f80d0330 con 0x7fa6f8168910 2024-04-25T11:21:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.085+0000 7fa6ffa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f8165c00 con 0x7fa6f8162330 2024-04-25T11:21:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.085+0000 7fa6fd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f80cff20 0x7fa6f81fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.085+0000 7fa6fd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f80cff20 0x7fa6f81fea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50802/0 (socket says 172.21.15.7:50802) 2024-04-25T11:21:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.085+0000 7fa6fd000640 1 -- 172.21.15.7:0/3625133381 learned_addr learned my addr 172.21.15.7:0/3625133381 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.086+0000 7fa6fd000640 1 -- 172.21.15.7:0/3625133381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6f8162330 msgr2=0x7fa6f8168050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.086+0000 7fa6fd000640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6f8162330 0x7fa6f8168050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.086+0000 7fa6fda00640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6f8168910 0x7fa6f8071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.086+0000 7fa6fd000640 1 -- 172.21.15.7:0/3625133381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6f8168910 msgr2=0x7fa6f8071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.086+0000 7fa6fd000640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6f8168910 0x7fa6f8071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.086+0000 7fa6fd000640 1 -- 172.21.15.7:0/3625133381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f8166090 con 0x7fa6f80cff20 2024-04-25T11:21:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.086+0000 7fa6fd000640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f80cff20 0x7fa6f81fea90 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009990 tx=0x7fa6e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.086+0000 7fa6f5600640 1 -- 172.21.15.7:0/3625133381 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8037040 con 0x7fa6f80cff20 2024-04-25T11:21:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.086+0000 7fa6f5600640 1 -- 172.21.15.7:0/3625133381 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e8042070 con 0x7fa6f80cff20 2024-04-25T11:21:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.086+0000 7fa6f5600640 1 -- 172.21.15.7:0/3625133381 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8030940 con 0x7fa6f80cff20 2024-04-25T11:21:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.086+0000 7fa6ffa00640 1 -- 172.21.15.7:0/3625133381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f8166370 con 0x7fa6f80cff20 2024-04-25T11:21:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.087+0000 7fa6ffa00640 1 -- 172.21.15.7:0/3625133381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f8166930 con 0x7fa6f80cff20 2024-04-25T11:21:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.088+0000 7fa6ffa00640 1 -- 172.21.15.7:0/3625133381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f8050740 con 0x7fa6f80cff20 2024-04-25T11:21:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.089+0000 7fa6f5600640 1 -- 172.21.15.7:0/3625133381 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa6e8030ae0 con 0x7fa6f80cff20 2024-04-25T11:21:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.090+0000 7fa6f5600640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6d40744b0 0x7fa6d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.090+0000 7fa6f5600640 1 -- 172.21.15.7:0/3625133381 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5249+0+0 (secure 0 0 0) 0x7fa6e80b5a30 con 0x7fa6f80cff20 2024-04-25T11:21:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.090+0000 7fa6efe00640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6d40744b0 0x7fa6d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.091+0000 7fa6efe00640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6d40744b0 0x7fa6d40769c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0006e80 tx=0x7fa6e0006dd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:33.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.095+0000 7fa6f5600640 1 -- 172.21.15.7:0/3625133381 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa6e80802e0 con 0x7fa6f80cff20 2024-04-25T11:21:33.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.370+0000 7fa6ffa00640 1 -- 172.21.15.7:0/3625133381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa6f80cf520 con 0x7fa6f80cff20 2024-04-25T11:21:33.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.371+0000 7fa6f5600640 1 -- 172.21.15.7:0/3625133381 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7fa6e8088360 con 0x7fa6f80cff20 2024-04-25T11:21:33.367 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:33.367 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:21:33.367 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-04-25T11:21:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-04-25T11:21:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w==", 2024-04-25T11:21:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:21:33.368 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw==", 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.369 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw==", 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A==", 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA==", 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:21:33.371 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg==", 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw==", 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ==", 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.373 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw==", 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng==", 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:21:33.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:21:33.375 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.375 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-04-25T11:21:33.375 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ==", 2024-04-25T11:21:33.375 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:21:33.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:21:33.375 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A==", 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg==", 2024-04-25T11:21:33.376 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA==", 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg==", 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ==", 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.378 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDZOSpmS8ObHxAAkXXXg1OI1YFC2/OZER3woA==", 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw==", 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w==", 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:21:33.380 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:21:33.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:33.381 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-25T11:21:33.381 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:21:33.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 -- 172.21.15.7:0/3625133381 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6d40744b0 msgr2=0x7fa6d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6d40744b0 0x7fa6d40769c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0006e80 tx=0x7fa6e0006dd0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 -- 172.21.15.7:0/3625133381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f80cff20 msgr2=0x7fa6f81fea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f80cff20 0x7fa6f81fea90 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009990 tx=0x7fa6e80046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 -- 172.21.15.7:0/3625133381 shutdown_connections 2024-04-25T11:21:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6d40744b0 0x7fa6d40769c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6f8168910 0x7fa6f8071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6f8162330 0x7fa6f8168050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 --2- 172.21.15.7:0/3625133381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f80cff20 0x7fa6f81fea90 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 -- 172.21.15.7:0/3625133381 >> 172.21.15.7:0/3625133381 conn(0x7fa6f80b7620 msgr2=0x7fa6f80703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 -- 172.21.15.7:0/3625133381 shutdown_connections 2024-04-25T11:21:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.374+0000 7fa6ffa00640 1 -- 172.21.15.7:0/3625133381 wait complete. 2024-04-25T11:21:33.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:21:33.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.674+0000 7fe3b6400640 1 Processor -- start 2024-04-25T11:21:33.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.675+0000 7fe3b6400640 1 -- start start 2024-04-25T11:21:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.677+0000 7fe3b6400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe3b0164690 0x7fe3b0070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.677+0000 7fe3b6400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3b0156cd0 0x7fe3b0163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.677+0000 7fe3b6400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b00b5080 con 0x7fe3b01570e0 2024-04-25T11:21:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.677+0000 7fe3b6400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b00b5360 con 0x7fe3b0164690 2024-04-25T11:21:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.677+0000 7fe3b6400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b00b5640 con 0x7fe3b0156cd0 2024-04-25T11:21:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.678+0000 7fe3b5000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe3b0164690 0x7fe3b0070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.678+0000 7fe3b5000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe3b0164690 0x7fe3b0070a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50808/0 (socket says 172.21.15.7:50808) 2024-04-25T11:21:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.678+0000 7fe3b5000640 1 -- 172.21.15.7:0/4083182126 learned_addr learned my addr 172.21.15.7:0/4083182126 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:33.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.678+0000 7fe3afe00640 1 --2- 172.21.15.7:0/4083182126 >> v2:172.21.15.87:3300/0 conn(0x7fe3b0156cd0 0x7fe3b0163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.678+0000 7fe3b5000640 1 -- 172.21.15.7:0/4083182126 >> v2:172.21.15.87:3300/0 conn(0x7fe3b0156cd0 msgr2=0x7fe3b0163ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.678+0000 7fe3b5000640 1 --2- 172.21.15.7:0/4083182126 >> v2:172.21.15.87:3300/0 conn(0x7fe3b0156cd0 0x7fe3b0163ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.678+0000 7fe3b5000640 1 -- 172.21.15.7:0/4083182126 >> v1:172.21.15.87:6789/0 conn(0x7fe3b01570e0 legacy=0x7fe3b00b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.678+0000 7fe3b5000640 1 -- 172.21.15.7:0/4083182126 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b00b5920 con 0x7fe3b0164690 2024-04-25T11:21:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.679+0000 7fe3b5000640 1 --2- 172.21.15.7:0/4083182126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b0164690 0x7fe3b0070a20 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00722d0 tx=0x7fe3a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40f128c631549 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:33.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.680+0000 7fe3af400640 1 -- 172.21.15.7:0/4083182126 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0037070 con 0x7fe3b0164690 2024-04-25T11:21:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.680+0000 7fe3af400640 1 -- 172.21.15.7:0/4083182126 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a0032440 con 0x7fe3b0164690 2024-04-25T11:21:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.680+0000 7fe3af400640 1 -- 172.21.15.7:0/4083182126 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0032810 con 0x7fe3b0164690 2024-04-25T11:21:33.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.680+0000 7fe3b6400640 1 -- 172.21.15.7:0/4083182126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b0164690 msgr2=0x7fe3b0070a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.680+0000 7fe3b6400640 1 --2- 172.21.15.7:0/4083182126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b0164690 0x7fe3b0070a20 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00722d0 tx=0x7fe3a002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.681+0000 7fe3b6400640 1 -- 172.21.15.7:0/4083182126 shutdown_connections 2024-04-25T11:21:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.681+0000 7fe3b6400640 1 --2- 172.21.15.7:0/4083182126 >> v2:172.21.15.87:3300/0 conn(0x7fe3b0156cd0 0x7fe3b0163ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.681+0000 7fe3b6400640 1 --2- 172.21.15.7:0/4083182126 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b0164690 0x7fe3b0070a20 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.681+0000 7fe3b6400640 1 -- 172.21.15.7:0/4083182126 >> 172.21.15.7:0/4083182126 conn(0x7fe3b00b3820 msgr2=0x7fe3b00b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.681+0000 7fe3b6400640 1 -- 172.21.15.7:0/4083182126 shutdown_connections 2024-04-25T11:21:33.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.681+0000 7fe3b6400640 1 -- 172.21.15.7:0/4083182126 wait complete. 2024-04-25T11:21:33.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.683+0000 7fe3b6400640 1 Processor -- start 2024-04-25T11:21:33.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.683+0000 7fe3b6400640 1 -- start start 2024-04-25T11:21:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.685+0000 7fe3b6400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b0156cd0 0x7fe3b015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.685+0000 7fe3b6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b01570e0 0x7fe3b0158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.685+0000 7fe3b6400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b0164690 0x7fe3b0159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.685+0000 7fe3b6400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0164010 con 0x7fe3b01570e0 2024-04-25T11:21:33.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.685+0000 7fe3b6400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b0070f60 con 0x7fe3b0156cd0 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.685+0000 7fe3b6400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b0164190 con 0x7fe3b0164690 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.685+0000 7fe3afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b01570e0 0x7fe3b0158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.685+0000 7fe3afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b01570e0 0x7fe3b0158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50820/0 (socket says 172.21.15.7:50820) 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.685+0000 7fe3afe00640 1 -- 172.21.15.7:0/3977973339 learned_addr learned my addr 172.21.15.7:0/3977973339 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.685+0000 7fe3b5000640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b0156cd0 0x7fe3b015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.685+0000 7fe3b5a00640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b0164690 0x7fe3b0159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.686+0000 7fe3afe00640 1 -- 172.21.15.7:0/3977973339 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b0164690 msgr2=0x7fe3b0159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.686+0000 7fe3afe00640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b0164690 0x7fe3b0159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.686+0000 7fe3afe00640 1 -- 172.21.15.7:0/3977973339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b0156cd0 msgr2=0x7fe3b015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.686+0000 7fe3afe00640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b0156cd0 0x7fe3b015ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.686+0000 7fe3afe00640 1 -- 172.21.15.7:0/3977973339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b015f7c0 con 0x7fe3b01570e0 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.686+0000 7fe3afe00640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b01570e0 0x7fe3b0158b40 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fe39c007e70 tx=0x7fe39c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.686+0000 7fe3acc00640 1 -- 172.21.15.7:0/3977973339 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c014620 con 0x7fe3b01570e0 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.686+0000 7fe3acc00640 1 -- 172.21.15.7:0/3977973339 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe39c019070 con 0x7fe3b01570e0 2024-04-25T11:21:33.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.686+0000 7fe3b6400640 1 -- 172.21.15.7:0/3977973339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b015faa0 con 0x7fe3b01570e0 2024-04-25T11:21:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.686+0000 7fe3acc00640 1 -- 172.21.15.7:0/3977973339 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c014a00 con 0x7fe3b01570e0 2024-04-25T11:21:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.687+0000 7fe3b6400640 1 -- 172.21.15.7:0/3977973339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b0159bc0 con 0x7fe3b01570e0 2024-04-25T11:21:33.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.688+0000 7fe3b6400640 1 -- 172.21.15.7:0/3977973339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b0050740 con 0x7fe3b01570e0 2024-04-25T11:21:33.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.689+0000 7fe3acc00640 1 -- 172.21.15.7:0/3977973339 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe39c014ba0 con 0x7fe3b01570e0 2024-04-25T11:21:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.691+0000 7fe3acc00640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3900744b0 0x7fe3900769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.691+0000 7fe3acc00640 1 -- 172.21.15.7:0/3977973339 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5249+0+0 (secure 0 0 0) 0x7fe39c095940 con 0x7fe3b01570e0 2024-04-25T11:21:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.691+0000 7fe3b5000640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3900744b0 0x7fe3900769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:33.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.692+0000 7fe3b5000640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3900744b0 0x7fe3900769c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0034000 tx=0x7fe3a002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:33.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.695+0000 7fe3acc00640 1 -- 172.21.15.7:0/3977973339 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe39c060280 con 0x7fe3b01570e0 2024-04-25T11:21:33.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.901+0000 7fe3b6400640 1 -- 172.21.15.7:0/3977973339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3b00b1140 con 0x7fe3b01570e0 2024-04-25T11:21:33.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.903+0000 7fe3acc00640 1 -- 172.21.15.7:0/3977973339 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+13078 (secure 0 0 0) 0x7fe39c05fb80 con 0x7fe3b01570e0 2024-04-25T11:21:33.899 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:33.899 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":138,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:21:32.762110+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.905+0000 7fe3b6400640 1 -- 172.21.15.7:0/3977973339 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3900744b0 msgr2=0x7fe3900769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.905+0000 7fe3b6400640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3900744b0 0x7fe3900769c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0034000 tx=0x7fe3a002e000 comp rx=0 tx=0).stop 2024-04-25T11:21:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.905+0000 7fe3b6400640 1 -- 172.21.15.7:0/3977973339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b01570e0 msgr2=0x7fe3b0158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.906+0000 7fe3b6400640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b01570e0 0x7fe3b0158b40 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fe39c007e70 tx=0x7fe39c00e4b0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.906+0000 7fe3b6400640 1 -- 172.21.15.7:0/3977973339 shutdown_connections 2024-04-25T11:21:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.906+0000 7fe3b6400640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe3900744b0 0x7fe3900769c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.906+0000 7fe3b6400640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3b0164690 0x7fe3b0159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.906+0000 7fe3b6400640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3b01570e0 0x7fe3b0158b40 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.906+0000 7fe3b6400640 1 --2- 172.21.15.7:0/3977973339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3b0156cd0 0x7fe3b015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.906+0000 7fe3b6400640 1 -- 172.21.15.7:0/3977973339 >> 172.21.15.7:0/3977973339 conn(0x7fe3b00b3820 msgr2=0x7fe3b0162c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:33.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.906+0000 7fe3b6400640 1 -- 172.21.15.7:0/3977973339 shutdown_connections 2024-04-25T11:21:33.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:33.906+0000 7fe3b6400640 1 -- 172.21.15.7:0/3977973339 wait complete. 2024-04-25T11:21:33.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:21:33.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:21:34.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.215+0000 7fedea000640 1 Processor -- start 2024-04-25T11:21:34.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.215+0000 7fedea000640 1 -- start start 2024-04-25T11:21:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.217+0000 7fedea000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fede4146180 0x7fede4070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.217+0000 7fedea000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fede40b08e0 0x7fede4149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.218+0000 7fedea000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede4070860 con 0x7fede41490f0 2024-04-25T11:21:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.218+0000 7fedea000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fede4146590 con 0x7fede4146180 2024-04-25T11:21:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.218+0000 7fedea000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fede406f280 con 0x7fede40b08e0 2024-04-25T11:21:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.218+0000 7fede8c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fede4146180 0x7fede4070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.218+0000 7fede8c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fede4146180 0x7fede4070320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50836/0 (socket says 172.21.15.7:50836) 2024-04-25T11:21:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.218+0000 7fede8c00640 1 -- 172.21.15.7:0/3272976821 learned_addr learned my addr 172.21.15.7:0/3272976821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.218+0000 7fede9600640 1 --2- 172.21.15.7:0/3272976821 >> v2:172.21.15.87:3300/0 conn(0x7fede40b08e0 0x7fede4149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.219+0000 7fede8c00640 1 -- 172.21.15.7:0/3272976821 >> v2:172.21.15.87:3300/0 conn(0x7fede40b08e0 msgr2=0x7fede4149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.219+0000 7fede8c00640 1 --2- 172.21.15.7:0/3272976821 >> v2:172.21.15.87:3300/0 conn(0x7fede40b08e0 0x7fede4149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.219+0000 7fede8c00640 1 -- 172.21.15.7:0/3272976821 >> v1:172.21.15.87:6789/0 conn(0x7fede41490f0 legacy=0x7fede40b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.219+0000 7fede8c00640 1 -- 172.21.15.7:0/3272976821 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede406f560 con 0x7fede4146180 2024-04-25T11:21:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.220+0000 7fede8c00640 1 --2- 172.21.15.7:0/3272976821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede4146180 0x7fede4070320 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fede4164a70 tx=0x7fedd802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe26b9aac8380343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.221+0000 7fede3400640 1 -- 172.21.15.7:0/3272976821 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8037070 con 0x7fede4146180 2024-04-25T11:21:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.221+0000 7fede3400640 1 -- 172.21.15.7:0/3272976821 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd8032440 con 0x7fede4146180 2024-04-25T11:21:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.221+0000 7fede3400640 1 -- 172.21.15.7:0/3272976821 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8032810 con 0x7fede4146180 2024-04-25T11:21:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.221+0000 7fedea000640 1 -- 172.21.15.7:0/3272976821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede4146180 msgr2=0x7fede4070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.221+0000 7fedea000640 1 --2- 172.21.15.7:0/3272976821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede4146180 0x7fede4070320 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fede4164a70 tx=0x7fedd802a230 comp rx=0 tx=0).stop 2024-04-25T11:21:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.222+0000 7fedea000640 1 -- 172.21.15.7:0/3272976821 shutdown_connections 2024-04-25T11:21:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.222+0000 7fedea000640 1 --2- 172.21.15.7:0/3272976821 >> v2:172.21.15.87:3300/0 conn(0x7fede40b08e0 0x7fede4149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.222+0000 7fedea000640 1 --2- 172.21.15.7:0/3272976821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede4146180 0x7fede4070320 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.222+0000 7fedea000640 1 -- 172.21.15.7:0/3272976821 >> 172.21.15.7:0/3272976821 conn(0x7fede40740f0 msgr2=0x7fede406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.222+0000 7fedea000640 1 -- 172.21.15.7:0/3272976821 shutdown_connections 2024-04-25T11:21:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.222+0000 7fedea000640 1 -- 172.21.15.7:0/3272976821 wait complete. 2024-04-25T11:21:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.224+0000 7fedea000640 1 Processor -- start 2024-04-25T11:21:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.224+0000 7fedea000640 1 -- start start 2024-04-25T11:21:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fedea000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fede40b08e0 0x7fede415f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fedea000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede4146180 0x7fede4160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fedea000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede41490f0 0x7fede41eb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fedea000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fede406fe00 con 0x7fede4146180 2024-04-25T11:21:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fedea000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fede406fb20 con 0x7fede41490f0 2024-04-25T11:21:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fedea000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fede406eea0 con 0x7fede40b08e0 2024-04-25T11:21:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fede3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede4146180 0x7fede4160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fede3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede4146180 0x7fede4160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50844/0 (socket says 172.21.15.7:50844) 2024-04-25T11:21:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fede3e00640 1 -- 172.21.15.7:0/1793038597 learned_addr learned my addr 172.21.15.7:0/1793038597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fede9600640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede41490f0 0x7fede41eb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fede8c00640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fede40b08e0 0x7fede415f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fede3e00640 1 -- 172.21.15.7:0/1793038597 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fede40b08e0 msgr2=0x7fede415f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fede3e00640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fede40b08e0 0x7fede415f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fede3e00640 1 -- 172.21.15.7:0/1793038597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede41490f0 msgr2=0x7fede41eb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.226+0000 7fede3e00640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede41490f0 0x7fede41eb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.227+0000 7fede3e00640 1 -- 172.21.15.7:0/1793038597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede41613b0 con 0x7fede4146180 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.227+0000 7fede8c00640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fede40b08e0 0x7fede415f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.227+0000 7fede3e00640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede4146180 0x7fede4160680 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fedd400b3f0 tx=0x7fedd400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.227+0000 7fede0c00640 1 -- 172.21.15.7:0/1793038597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4015020 con 0x7fede4146180 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.227+0000 7fede0c00640 1 -- 172.21.15.7:0/1793038597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd4011830 con 0x7fede4146180 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.227+0000 7fede0c00640 1 -- 172.21.15.7:0/1793038597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4011c20 con 0x7fede4146180 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.227+0000 7fedea000640 1 -- 172.21.15.7:0/1793038597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede4161690 con 0x7fede4146180 2024-04-25T11:21:34.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.227+0000 7fedea000640 1 -- 172.21.15.7:0/1793038597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede4161c50 con 0x7fede4146180 2024-04-25T11:21:34.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.228+0000 7fedea000640 1 -- 172.21.15.7:0/1793038597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede4050740 con 0x7fede4146180 2024-04-25T11:21:34.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.230+0000 7fede0c00640 1 -- 172.21.15.7:0/1793038597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fedd4018020 con 0x7fede4146180 2024-04-25T11:21:34.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.231+0000 7fede0c00640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fedc80744e0 0x7fedc80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.231+0000 7fede0c00640 1 -- 172.21.15.7:0/1793038597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5249+0+0 (secure 0 0 0) 0x7fedd4094d90 con 0x7fede4146180 2024-04-25T11:21:34.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.231+0000 7fede8c00640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fedc80744e0 0x7fedc80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.232+0000 7fede8c00640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fedc80744e0 0x7fedc80769f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fedd8034000 tx=0x7fedd802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:34.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.236+0000 7fede0c00640 1 -- 172.21.15.7:0/1793038597 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fedd401f030 con 0x7fede4146180 2024-04-25T11:21:34.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.441+0000 7fedea000640 1 -- 172.21.15.7:0/1793038597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fede40b52c0 con 0x7fede4146180 2024-04-25T11:21:34.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.442+0000 7fede0c00640 1 -- 172.21.15.7:0/1793038597 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) ==== 93+0+10 (secure 0 0 0) 0x7fedd405f640 con 0x7fede4146180 2024-04-25T11:21:34.438 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.444+0000 7fedea000640 1 -- 172.21.15.7:0/1793038597 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fedc80744e0 msgr2=0x7fedc80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.444+0000 7fedea000640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fedc80744e0 0x7fedc80769f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fedd8034000 tx=0x7fedd802e000 comp rx=0 tx=0).stop 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.445+0000 7fedea000640 1 -- 172.21.15.7:0/1793038597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede4146180 msgr2=0x7fede4160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.445+0000 7fedea000640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede4146180 0x7fede4160680 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fedd400b3f0 tx=0x7fedd400b8b0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.445+0000 7fedea000640 1 -- 172.21.15.7:0/1793038597 shutdown_connections 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.445+0000 7fedea000640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fedc80744e0 0x7fedc80769f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.445+0000 7fedea000640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fede41490f0 0x7fede41eb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.445+0000 7fedea000640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fede4146180 0x7fede4160680 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.445+0000 7fedea000640 1 --2- 172.21.15.7:0/1793038597 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fede40b08e0 0x7fede415f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.445+0000 7fedea000640 1 -- 172.21.15.7:0/1793038597 >> 172.21.15.7:0/1793038597 conn(0x7fede40740f0 msgr2=0x7fede40b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.445+0000 7fedea000640 1 -- 172.21.15.7:0/1793038597 shutdown_connections 2024-04-25T11:21:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.445+0000 7fedea000640 1 -- 172.21.15.7:0/1793038597 wait complete. 2024-04-25T11:21:34.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:21:34.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.755+0000 7ff404e00640 1 Processor -- start 2024-04-25T11:21:34.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.755+0000 7ff404e00640 1 -- start start 2024-04-25T11:21:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.757+0000 7ff404e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4000ce400 0x7ff4000d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.757+0000 7ff404e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4000d0c20 0x7ff4000b2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.757+0000 7ff404e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4000d28a0 con 0x7ff4000d35a0 2024-04-25T11:21:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.757+0000 7ff404e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff4000d1d20 con 0x7ff4000d0c20 2024-04-25T11:21:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.757+0000 7ff404e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff4000d2000 con 0x7ff4000ce400 2024-04-25T11:21:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.758+0000 7ff3ffe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff4000d35a0 0x7ff4000d39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53872/0 (socket says 172.21.15.7:53872) 2024-04-25T11:21:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.758+0000 7ff3ffe00640 1 -- 172.21.15.7:0/2067341869 learned_addr learned my addr 172.21.15.7:0/2067341869 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.758+0000 7ff3ff400640 1 --2- 172.21.15.7:0/2067341869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4000ce400 0x7ff4000d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.758+0000 7ff3fea00640 1 --2- 172.21.15.7:0/2067341869 >> v2:172.21.15.87:3300/0 conn(0x7ff4000d0c20 0x7ff4000b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.759+0000 7ff3fea00640 1 -- 172.21.15.7:0/2067341869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4000ce400 msgr2=0x7ff4000d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.759+0000 7ff3fea00640 1 --2- 172.21.15.7:0/2067341869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4000ce400 0x7ff4000d1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.759+0000 7ff3fea00640 1 -- 172.21.15.7:0/2067341869 >> v1:172.21.15.7:6789/0 conn(0x7ff4000d35a0 legacy=0x7ff4000d39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.759+0000 7ff3fea00640 1 -- 172.21.15.7:0/2067341869 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4000d22e0 con 0x7ff4000d0c20 2024-04-25T11:21:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.760+0000 7ff3fea00640 1 --2- 172.21.15.7:0/2067341869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4000d0c20 0x7ff4000b2980 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0097a0 tx=0x7ff3ec02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e0bfdea5267cc5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.765+0000 7ff3fe000640 1 -- 172.21.15.7:0/2067341869 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec037070 con 0x7ff4000d0c20 2024-04-25T11:21:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.765+0000 7ff3fe000640 1 -- 172.21.15.7:0/2067341869 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ec02ae50 con 0x7ff4000d0c20 2024-04-25T11:21:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.765+0000 7ff3fe000640 1 -- 172.21.15.7:0/2067341869 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec032660 con 0x7ff4000d0c20 2024-04-25T11:21:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.766+0000 7ff404e00640 1 -- 172.21.15.7:0/2067341869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4000d0c20 msgr2=0x7ff4000b2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.766+0000 7ff404e00640 1 --2- 172.21.15.7:0/2067341869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4000d0c20 0x7ff4000b2980 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0097a0 tx=0x7ff3ec02a100 comp rx=0 tx=0).stop 2024-04-25T11:21:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.766+0000 7ff404e00640 1 -- 172.21.15.7:0/2067341869 shutdown_connections 2024-04-25T11:21:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.766+0000 7ff404e00640 1 --2- 172.21.15.7:0/2067341869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4000d0c20 0x7ff4000b2980 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.766+0000 7ff404e00640 1 --2- 172.21.15.7:0/2067341869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4000ce400 0x7ff4000d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.766+0000 7ff404e00640 1 -- 172.21.15.7:0/2067341869 >> 172.21.15.7:0/2067341869 conn(0x7ff40008ce10 msgr2=0x7ff40008d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:34.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.766+0000 7ff404e00640 1 -- 172.21.15.7:0/2067341869 shutdown_connections 2024-04-25T11:21:34.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.766+0000 7ff404e00640 1 -- 172.21.15.7:0/2067341869 wait complete. 2024-04-25T11:21:34.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.769+0000 7ff404e00640 1 Processor -- start 2024-04-25T11:21:34.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.769+0000 7ff404e00640 1 -- start start 2024-04-25T11:21:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.770+0000 7ff404e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4000ce400 0x7ff4000b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.771+0000 7ff404e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4000d0c20 0x7ff4001e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.771+0000 7ff404e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000d35a0 0x7ff400157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.771+0000 7ff404e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4000d1030 con 0x7ff4000d35a0 2024-04-25T11:21:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.771+0000 7ff404e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff4000b2ec0 con 0x7ff4000ce400 2024-04-25T11:21:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.771+0000 7ff404e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff4000b41a0 con 0x7ff4000d0c20 2024-04-25T11:21:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.771+0000 7ff3ffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000d35a0 0x7ff400157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.771+0000 7ff3ffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000d35a0 0x7ff400157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50852/0 (socket says 172.21.15.7:50852) 2024-04-25T11:21:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.771+0000 7ff3ffe00640 1 -- 172.21.15.7:0/3128827745 learned_addr learned my addr 172.21.15.7:0/3128827745 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.771+0000 7ff3fea00640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4000d0c20 0x7ff4001e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.771+0000 7ff3ff400640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4000ce400 0x7ff4000b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.771+0000 7ff3ffe00640 1 -- 172.21.15.7:0/3128827745 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4000d0c20 msgr2=0x7ff4001e81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.772+0000 7ff3ffe00640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4000d0c20 0x7ff4001e81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.772+0000 7ff3ffe00640 1 -- 172.21.15.7:0/3128827745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4000ce400 msgr2=0x7ff4000b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.772+0000 7ff3ffe00640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4000ce400 0x7ff4000b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.772+0000 7ff3ffe00640 1 -- 172.21.15.7:0/3128827745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4001e6a10 con 0x7ff4000d35a0 2024-04-25T11:21:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.772+0000 7ff3ffe00640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000d35a0 0x7ff400157730 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800caa0 tx=0x7ff3f800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.772+0000 7ff3f7400640 1 -- 172.21.15.7:0/3128827745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8017070 con 0x7ff4000d35a0 2024-04-25T11:21:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.772+0000 7ff3f7400640 1 -- 172.21.15.7:0/3128827745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f800ac00 con 0x7ff4000d35a0 2024-04-25T11:21:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.772+0000 7ff404e00640 1 -- 172.21.15.7:0/3128827745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4001e6cf0 con 0x7ff4000d35a0 2024-04-25T11:21:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.772+0000 7ff3f7400640 1 -- 172.21.15.7:0/3128827745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8012400 con 0x7ff4000d35a0 2024-04-25T11:21:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.772+0000 7ff404e00640 1 -- 172.21.15.7:0/3128827745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4001e72b0 con 0x7ff4000d35a0 2024-04-25T11:21:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.773+0000 7ff404e00640 1 -- 172.21.15.7:0/3128827745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4001e6780 con 0x7ff4000d35a0 2024-04-25T11:21:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.775+0000 7ff3f7400640 1 -- 172.21.15.7:0/3128827745 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff3f801a070 con 0x7ff4000d35a0 2024-04-25T11:21:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.776+0000 7ff3f7400640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3e00744e0 0x7ff3e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.776+0000 7ff3f7400640 1 -- 172.21.15.7:0/3128827745 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5249+0+0 (secure 0 0 0) 0x7ff3f8094c10 con 0x7ff4000d35a0 2024-04-25T11:21:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.776+0000 7ff3ff400640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3e00744e0 0x7ff3e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:34.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.777+0000 7ff3ff400640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3e00744e0 0x7ff3e00769f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff4000d3330 tx=0x7ff3f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:34.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.781+0000 7ff3f7400640 1 -- 172.21.15.7:0/3128827745 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff3f805ef00 con 0x7ff4000d35a0 2024-04-25T11:21:34.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.989+0000 7ff404e00640 1 -- 172.21.15.7:0/3128827745 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff400050740 con 0x7ff4000d35a0 2024-04-25T11:21:34.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.991+0000 7ff3f7400640 1 -- 172.21.15.7:0/3128827745 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v138) ==== 84+0+65 (secure 0 0 0) 0x7ff3f8063200 con 0x7ff4000d35a0 2024-04-25T11:21:34.986 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:34.987 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-25T11:21:34.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:21:34.987 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-25T11:21:34.987 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-25T11:21:34.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:21:34.987 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-25T11:21:34.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.993+0000 7ff404e00640 1 -- 172.21.15.7:0/3128827745 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3e00744e0 msgr2=0x7ff3e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:34.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.993+0000 7ff404e00640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3e00744e0 0x7ff3e00769f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff4000d3330 tx=0x7ff3f0008040 comp rx=0 tx=0).stop 2024-04-25T11:21:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.993+0000 7ff404e00640 1 -- 172.21.15.7:0/3128827745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000d35a0 msgr2=0x7ff400157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.993+0000 7ff404e00640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000d35a0 0x7ff400157730 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800caa0 tx=0x7ff3f800cf60 comp rx=0 tx=0).stop 2024-04-25T11:21:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.993+0000 7ff404e00640 1 -- 172.21.15.7:0/3128827745 shutdown_connections 2024-04-25T11:21:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.993+0000 7ff404e00640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff3e00744e0 0x7ff3e00769f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.993+0000 7ff404e00640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4000d35a0 0x7ff400157730 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.993+0000 7ff404e00640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4000d0c20 0x7ff4001e81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.993+0000 7ff404e00640 1 --2- 172.21.15.7:0/3128827745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4000ce400 0x7ff4000b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.994+0000 7ff404e00640 1 -- 172.21.15.7:0/3128827745 >> 172.21.15.7:0/3128827745 conn(0x7ff40008ce10 msgr2=0x7ff400072b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.994+0000 7ff404e00640 1 -- 172.21.15.7:0/3128827745 shutdown_connections 2024-04-25T11:21:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:34.994+0000 7ff404e00640 1 -- 172.21.15.7:0/3128827745 wait complete. 2024-04-25T11:21:35.015 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:21:35.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:21:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.315+0000 7fc946a00640 1 Processor -- start 2024-04-25T11:21:35.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.315+0000 7fc946a00640 1 -- start start 2024-04-25T11:21:35.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.317+0000 7fc946a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc9400b2a50 0x7fc9400b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:35.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.317+0000 7fc946a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc94008cad0 0x7fc94006d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:35.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.317+0000 7fc946a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc94006dbc0 0x7fc94006dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:35.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.317+0000 7fc946a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc9400d0cf0 con 0x7fc94006dbc0 2024-04-25T11:21:35.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.317+0000 7fc946a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc94006ebc0 con 0x7fc9400b2a50 2024-04-25T11:21:35.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.317+0000 7fc946a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc9400d0f40 con 0x7fc94008cad0 2024-04-25T11:21:35.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.318+0000 7fc946000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc94006dbc0 0x7fc94006dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:35.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.318+0000 7fc946000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc94006dbc0 0x7fc94006dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50864/0 (socket says 172.21.15.7:50864) 2024-04-25T11:21:35.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.318+0000 7fc946000640 1 -- 172.21.15.7:0/2194892311 learned_addr learned my addr 172.21.15.7:0/2194892311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:35.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.318+0000 7fc945600640 1 --2- 172.21.15.7:0/2194892311 >> v2:172.21.15.87:3300/0 conn(0x7fc9400b2a50 0x7fc9400b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:35.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.318+0000 7fc944c00640 1 --2- 172.21.15.7:0/2194892311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc94008cad0 0x7fc94006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:35.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.318+0000 7fc946000640 1 -- 172.21.15.7:0/2194892311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc94008cad0 msgr2=0x7fc94006d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:35.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.318+0000 7fc946000640 1 --2- 172.21.15.7:0/2194892311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc94008cad0 0x7fc94006d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.318+0000 7fc946000640 1 -- 172.21.15.7:0/2194892311 >> v2:172.21.15.87:3300/0 conn(0x7fc9400b2a50 msgr2=0x7fc9400b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:35.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.318+0000 7fc946000640 1 --2- 172.21.15.7:0/2194892311 >> v2:172.21.15.87:3300/0 conn(0x7fc9400b2a50 0x7fc9400b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.318+0000 7fc946000640 1 -- 172.21.15.7:0/2194892311 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94006e8e0 con 0x7fc94006dbc0 2024-04-25T11:21:35.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.320+0000 7fc946000640 1 --2- 172.21.15.7:0/2194892311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94006dbc0 0x7fc94006dfd0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fc940051060 tx=0x7fc93802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa9fb5850c47247 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:35.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.320+0000 7fc93fe00640 1 -- 172.21.15.7:0/2194892311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc938036070 con 0x7fc94006dbc0 2024-04-25T11:21:35.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.320+0000 7fc93fe00640 1 -- 172.21.15.7:0/2194892311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc938031600 con 0x7fc94006dbc0 2024-04-25T11:21:35.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.320+0000 7fc93fe00640 1 -- 172.21.15.7:0/2194892311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9380319d0 con 0x7fc94006dbc0 2024-04-25T11:21:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.321+0000 7fc946a00640 1 -- 172.21.15.7:0/2194892311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94006dbc0 msgr2=0x7fc94006dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.321+0000 7fc946a00640 1 --2- 172.21.15.7:0/2194892311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94006dbc0 0x7fc94006dfd0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fc940051060 tx=0x7fc93802a6a0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.321+0000 7fc946a00640 1 -- 172.21.15.7:0/2194892311 shutdown_connections 2024-04-25T11:21:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.321+0000 7fc946a00640 1 --2- 172.21.15.7:0/2194892311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94006dbc0 0x7fc94006dfd0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.321+0000 7fc946a00640 1 --2- 172.21.15.7:0/2194892311 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc94008cad0 0x7fc94006d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.321+0000 7fc946a00640 1 --2- 172.21.15.7:0/2194892311 >> v2:172.21.15.87:3300/0 conn(0x7fc9400b2a50 0x7fc9400b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.321+0000 7fc946a00640 1 -- 172.21.15.7:0/2194892311 >> 172.21.15.7:0/2194892311 conn(0x7fc9400d01d0 msgr2=0x7fc9400d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.321+0000 7fc946a00640 1 -- 172.21.15.7:0/2194892311 shutdown_connections 2024-04-25T11:21:35.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.321+0000 7fc946a00640 1 -- 172.21.15.7:0/2194892311 wait complete. 2024-04-25T11:21:35.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.323+0000 7fc946a00640 1 Processor -- start 2024-04-25T11:21:35.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.324+0000 7fc946a00640 1 -- start start 2024-04-25T11:21:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.325+0000 7fc946a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc94006dbc0 0x7fc9401f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.325+0000 7fc946a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94008cad0 0x7fc9401df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.325+0000 7fc946a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9400b2a50 0x7fc94009ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.325+0000 7fc946a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc94006e690 con 0x7fc94008cad0 2024-04-25T11:21:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.325+0000 7fc946a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc94006e510 con 0x7fc94006dbc0 2024-04-25T11:21:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.325+0000 7fc946a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc94009d000 con 0x7fc9400b2a50 2024-04-25T11:21:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.326+0000 7fc944c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94008cad0 0x7fc9401df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.326+0000 7fc944c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94008cad0 0x7fc9401df9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50872/0 (socket says 172.21.15.7:50872) 2024-04-25T11:21:35.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.326+0000 7fc944c00640 1 -- 172.21.15.7:0/602375295 learned_addr learned my addr 172.21.15.7:0/602375295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.326+0000 7fc945600640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc94006dbc0 0x7fc9401f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.326+0000 7fc946000640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9400b2a50 0x7fc94009ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.326+0000 7fc944c00640 1 -- 172.21.15.7:0/602375295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9400b2a50 msgr2=0x7fc94009ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.326+0000 7fc944c00640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9400b2a50 0x7fc94009ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.326+0000 7fc944c00640 1 -- 172.21.15.7:0/602375295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc94006dbc0 msgr2=0x7fc9401f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.326+0000 7fc944c00640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc94006dbc0 0x7fc9401f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.326+0000 7fc944c00640 1 -- 172.21.15.7:0/602375295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9401f7980 con 0x7fc94008cad0 2024-04-25T11:21:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.327+0000 7fc944c00640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94008cad0 0x7fc9401df9e0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00d7d0 tx=0x7fc92c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.327+0000 7fc93d600640 1 -- 172.21.15.7:0/602375295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c018020 con 0x7fc94008cad0 2024-04-25T11:21:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.327+0000 7fc93d600640 1 -- 172.21.15.7:0/602375295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc92c0096a0 con 0x7fc94008cad0 2024-04-25T11:21:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.327+0000 7fc93d600640 1 -- 172.21.15.7:0/602375295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92c009a90 con 0x7fc94008cad0 2024-04-25T11:21:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.327+0000 7fc946a00640 1 -- 172.21.15.7:0/602375295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9400cea00 con 0x7fc94008cad0 2024-04-25T11:21:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.327+0000 7fc946a00640 1 -- 172.21.15.7:0/602375295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9400caee0 con 0x7fc94008cad0 2024-04-25T11:21:35.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.328+0000 7fc946a00640 1 -- 172.21.15.7:0/602375295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9400b98f0 con 0x7fc94008cad0 2024-04-25T11:21:35.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.329+0000 7fc93d600640 1 -- 172.21.15.7:0/602375295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc92c01b020 con 0x7fc94008cad0 2024-04-25T11:21:35.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.330+0000 7fc93d600640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9200744b0 0x7fc9200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:35.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.331+0000 7fc93d600640 1 -- 172.21.15.7:0/602375295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5249+0+0 (secure 0 0 0) 0x7fc92c095540 con 0x7fc94008cad0 2024-04-25T11:21:35.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.331+0000 7fc945600640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9200744b0 0x7fc9200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:35.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.332+0000 7fc945600640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9200744b0 0x7fc9200769c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc934006fd0 tx=0x7fc934008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:35.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.335+0000 7fc93d600640 1 -- 172.21.15.7:0/602375295 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc9400b98f0 con 0x7fc94008cad0 2024-04-25T11:21:35.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.540+0000 7fc946a00640 1 -- 172.21.15.7:0/602375295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc940050740 con 0x7fc94008cad0 2024-04-25T11:21:35.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.786+0000 7fc93d600640 1 -- 172.21.15.7:0/602375295 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v139) ==== 138+0+0 (secure 0 0 0) 0x7fc92c022020 con 0x7fc94008cad0 2024-04-25T11:21:35.782 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-25T11:21:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.789+0000 7fc946a00640 1 -- 172.21.15.7:0/602375295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9200744b0 msgr2=0x7fc9200769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.789+0000 7fc946a00640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9200744b0 0x7fc9200769c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc934006fd0 tx=0x7fc934008040 comp rx=0 tx=0).stop 2024-04-25T11:21:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.790+0000 7fc946a00640 1 -- 172.21.15.7:0/602375295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94008cad0 msgr2=0x7fc9401df9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.790+0000 7fc946a00640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94008cad0 0x7fc9401df9e0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fc92c00d7d0 tx=0x7fc92c00dc90 comp rx=0 tx=0).stop 2024-04-25T11:21:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.790+0000 7fc946a00640 1 -- 172.21.15.7:0/602375295 shutdown_connections 2024-04-25T11:21:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.790+0000 7fc946a00640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc9200744b0 0x7fc9200769c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.790+0000 7fc946a00640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9400b2a50 0x7fc94009ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.790+0000 7fc946a00640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc94008cad0 0x7fc9401df9e0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.790+0000 7fc946a00640 1 --2- 172.21.15.7:0/602375295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc94006dbc0 0x7fc9401f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.790+0000 7fc946a00640 1 -- 172.21.15.7:0/602375295 >> 172.21.15.7:0/602375295 conn(0x7fc9400d01d0 msgr2=0x7fc9400b82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.790+0000 7fc946a00640 1 -- 172.21.15.7:0/602375295 shutdown_connections 2024-04-25T11:21:35.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:35.790+0000 7fc946a00640 1 -- 172.21.15.7:0/602375295 wait complete. 2024-04-25T11:21:35.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:21:36.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.098+0000 7f8277200640 1 Processor -- start 2024-04-25T11:21:36.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.099+0000 7f8277200640 1 -- start start 2024-04-25T11:21:36.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.101+0000 7f8277200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82700d0480 0x7f82700d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:36.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.101+0000 7f8277200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82700ce570 con 0x7f82701583c0 2024-04-25T11:21:36.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.101+0000 7f8277200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82700ce850 con 0x7f827006eea0 2024-04-25T11:21:36.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.101+0000 7f8277200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f827015f190 con 0x7f82700d0480 2024-04-25T11:21:36.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.101+0000 7f8275400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82700d0480 0x7f82700d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:36.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.101+0000 7f8275e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f82701583c0 0x7f82700cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53888/0 (socket says 172.21.15.7:53888) 2024-04-25T11:21:36.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.102+0000 7f8275400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f82700d0480 0x7f82700d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50888/0 (socket says 172.21.15.7:50888) 2024-04-25T11:21:36.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.102+0000 7f8275e00640 1 -- 172.21.15.7:0/3482805670 learned_addr learned my addr 172.21.15.7:0/3482805670 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:36.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.102+0000 7f8275400640 1 -- 172.21.15.7:0/3482805670 >> v1:172.21.15.87:6789/0 conn(0x7f827006eea0 legacy=0x7f82700b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:36.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.102+0000 7f8275400640 1 -- 172.21.15.7:0/3482805670 >> v1:172.21.15.7:6789/0 conn(0x7f82701583c0 legacy=0x7f82700cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:36.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.102+0000 7f8275400640 1 -- 172.21.15.7:0/3482805670 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82700ceb30 con 0x7f82700d0480 2024-04-25T11:21:36.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.102+0000 7f8274a00640 1 -- 172.21.15.7:0/3482805670 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 654636953 0 0) 0x7f82700ce570 con 0x7f82701583c0 2024-04-25T11:21:36.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.103+0000 7f8275400640 1 --2- 172.21.15.7:0/3482805670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82700d0480 0x7f82700d0890 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f82600097a0 tx=0x7f826002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabf6cabcff46bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:36.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.103+0000 7f8274a00640 1 -- 172.21.15.7:0/3482805670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260037070 con 0x7f82700d0480 2024-04-25T11:21:36.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.103+0000 7f8274a00640 1 -- 172.21.15.7:0/3482805670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826002ae50 con 0x7f82700d0480 2024-04-25T11:21:36.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.103+0000 7f8274a00640 1 -- 172.21.15.7:0/3482805670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260032660 con 0x7f82700d0480 2024-04-25T11:21:36.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.104+0000 7f8277200640 1 -- 172.21.15.7:0/3482805670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82700d0480 msgr2=0x7f82700d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:36.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.104+0000 7f8277200640 1 --2- 172.21.15.7:0/3482805670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82700d0480 0x7f82700d0890 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f82600097a0 tx=0x7f826002a140 comp rx=0 tx=0).stop 2024-04-25T11:21:36.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.104+0000 7f8277200640 1 -- 172.21.15.7:0/3482805670 shutdown_connections 2024-04-25T11:21:36.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.104+0000 7f8277200640 1 --2- 172.21.15.7:0/3482805670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82700d0480 0x7f82700d0890 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:36.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.104+0000 7f8277200640 1 -- 172.21.15.7:0/3482805670 >> 172.21.15.7:0/3482805670 conn(0x7f82700740f0 msgr2=0x7f827006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:36.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.104+0000 7f8277200640 1 -- 172.21.15.7:0/3482805670 shutdown_connections 2024-04-25T11:21:36.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.104+0000 7f8277200640 1 -- 172.21.15.7:0/3482805670 wait complete. 2024-04-25T11:21:36.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.107+0000 7f8277200640 1 Processor -- start 2024-04-25T11:21:36.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.107+0000 7f8277200640 1 -- start start 2024-04-25T11:21:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.108+0000 7f8277200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f827006eea0 0x7f82700b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.108+0000 7f8277200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82700d0480 0x7f827014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.108+0000 7f8277200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82701583c0 0x7f82700b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.108+0000 7f8277200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82700cfd20 con 0x7f82700d0480 2024-04-25T11:21:36.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.108+0000 7f8277200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82700cf0f0 con 0x7f82701583c0 2024-04-25T11:21:36.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.108+0000 7f8277200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82700b0e70 con 0x7f827006eea0 2024-04-25T11:21:36.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.109+0000 7f8275400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82700d0480 0x7f827014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:36.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.109+0000 7f8275400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82700d0480 0x7f827014afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50902/0 (socket says 172.21.15.7:50902) 2024-04-25T11:21:36.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.109+0000 7f8275400640 1 -- 172.21.15.7:0/2386670432 learned_addr learned my addr 172.21.15.7:0/2386670432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:36.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.109+0000 7f8275400640 1 -- 172.21.15.7:0/2386670432 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f827006eea0 msgr2=0x7f82700b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:36.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.109+0000 7f8275400640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f827006eea0 0x7f82700b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:36.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.109+0000 7f8275400640 1 -- 172.21.15.7:0/2386670432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82701583c0 msgr2=0x7f82700b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:36.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.109+0000 7f8276800640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82701583c0 0x7f82700b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:36.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.109+0000 7f8275400640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82701583c0 0x7f82700b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.109+0000 7f8275400640 1 -- 172.21.15.7:0/2386670432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82701603c0 con 0x7f82700d0480 2024-04-25T11:21:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.109+0000 7f8276800640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82701583c0 0x7f82700b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.110+0000 7f8275400640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82700d0480 0x7f827014afd0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f826002a730 tx=0x7f82600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.110+0000 7f826a000640 1 -- 172.21.15.7:0/2386670432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260042070 con 0x7f82700d0480 2024-04-25T11:21:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.110+0000 7f826a000640 1 -- 172.21.15.7:0/2386670432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826003d850 con 0x7f82700d0480 2024-04-25T11:21:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.110+0000 7f826a000640 1 -- 172.21.15.7:0/2386670432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260037040 con 0x7f82700d0480 2024-04-25T11:21:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.110+0000 7f8277200640 1 -- 172.21.15.7:0/2386670432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827014c940 con 0x7f82700d0480 2024-04-25T11:21:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.110+0000 7f8277200640 1 -- 172.21.15.7:0/2386670432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827014f5f0 con 0x7f82700d0480 2024-04-25T11:21:36.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.111+0000 7f8277200640 1 -- 172.21.15.7:0/2386670432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8270050740 con 0x7f82700d0480 2024-04-25T11:21:36.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.114+0000 7f826a000640 1 -- 172.21.15.7:0/2386670432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8260045070 con 0x7f82700d0480 2024-04-25T11:21:36.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.115+0000 7f826a000640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82540744e0 0x7f82540769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:36.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.115+0000 7f826a000640 1 -- 172.21.15.7:0/2386670432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5619+0+0 (secure 0 0 0) 0x7f82600b5ce0 con 0x7f82700d0480 2024-04-25T11:21:36.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.115+0000 7f8275e00640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82540744e0 0x7f82540769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:36.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.116+0000 7f8275e00640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82540744e0 0x7f82540769f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8264000c00 tx=0x7f82640079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.119+0000 7f826a000640 1 -- 172.21.15.7:0/2386670432 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f826004c060 con 0x7f82700d0480 2024-04-25T11:21:36.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.321+0000 7f8277200640 1 -- 172.21.15.7:0/2386670432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8270070dd0 con 0x7f82700d0480 2024-04-25T11:21:36.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.793+0000 7f826a000640 1 -- 172.21.15.7:0/2386670432 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v140) ==== 170+0+0 (secure 0 0 0) 0x7f82600804a0 con 0x7f82700d0480 2024-04-25T11:21:36.789 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-25T11:21:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 -- 172.21.15.7:0/2386670432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82540744e0 msgr2=0x7f82540769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82540744e0 0x7f82540769f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8264000c00 tx=0x7f82640079c0 comp rx=0 tx=0).stop 2024-04-25T11:21:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 -- 172.21.15.7:0/2386670432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82700d0480 msgr2=0x7f827014afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82700d0480 0x7f827014afd0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f826002a730 tx=0x7f82600046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 -- 172.21.15.7:0/2386670432 shutdown_connections 2024-04-25T11:21:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f82540744e0 0x7f82540769f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82701583c0 0x7f82700b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82700d0480 0x7f827014afd0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 --2- 172.21.15.7:0/2386670432 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f827006eea0 0x7f82700b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:36.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 -- 172.21.15.7:0/2386670432 >> 172.21.15.7:0/2386670432 conn(0x7f82700740f0 msgr2=0x7f8270161980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 -- 172.21.15.7:0/2386670432 shutdown_connections 2024-04-25T11:21:36.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:36.797+0000 7f8277200640 1 -- 172.21.15.7:0/2386670432 wait complete. 2024-04-25T11:21:36.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:21:37.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.100+0000 7f3fe2600640 1 Processor -- start 2024-04-25T11:21:37.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.100+0000 7f3fe2600640 1 -- start start 2024-04-25T11:21:37.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.102+0000 7f3fe2600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc06e8c0 0x7f3fdc0d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:37.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.102+0000 7f3fe2600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3fdc0d2220 0x7f3fdc0d2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:37.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.102+0000 7f3fe2600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fdc09cb10 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.102+0000 7f3fe2600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3fdc0b92e0 con 0x7f3fdc0d2220 2024-04-25T11:21:37.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.102+0000 7f3fe2600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3fdc06d810 con 0x7f3fdc06e8c0 2024-04-25T11:21:37.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.103+0000 7f3fdbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc06e8c0 0x7f3fdc0d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:37.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.103+0000 7f3fe1c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3fdc0d2220 0x7f3fdc0d2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:37.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.103+0000 7f3fe1200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f3fdc06e4b0 0x7f3fdc0b3ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:57630/0 (socket says 172.21.15.7:57630) 2024-04-25T11:21:37.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.103+0000 7f3fe1200640 1 -- 172.21.15.7:0/422624294 learned_addr learned my addr 172.21.15.7:0/422624294 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.104+0000 7f3fdb400640 1 -- 172.21.15.7:0/422624294 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1588910604 0 0) 0x7f3fdc09cb10 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.104+0000 7f3fdb400640 1 -- 172.21.15.7:0/422624294 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fc4003520 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.104+0000 7f3fdbe00640 1 -- 172.21.15.7:0/422624294 >> v2:172.21.15.87:3300/0 conn(0x7f3fdc0d2220 msgr2=0x7f3fdc0d2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.104+0000 7f3fdbe00640 1 --2- 172.21.15.7:0/422624294 >> v2:172.21.15.87:3300/0 conn(0x7f3fdc0d2220 0x7f3fdc0d2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.104+0000 7f3fdbe00640 1 -- 172.21.15.7:0/422624294 >> v1:172.21.15.87:6789/0 conn(0x7f3fdc06e4b0 legacy=0x7f3fdc0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:37.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.104+0000 7f3fdbe00640 1 -- 172.21.15.7:0/422624294 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fdc09d010 con 0x7f3fdc06e8c0 2024-04-25T11:21:37.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.106+0000 7f3fdbe00640 1 --2- 172.21.15.7:0/422624294 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc06e8c0 0x7f3fdc0d1ce0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0097a0 tx=0x7f3fcc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f45655c2cef1af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:37.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.106+0000 7f3fdb400640 1 -- 172.21.15.7:0/422624294 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc037070 con 0x7f3fdc06e8c0 2024-04-25T11:21:37.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.106+0000 7f3fdb400640 1 -- 172.21.15.7:0/422624294 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fcc02ae50 con 0x7f3fdc06e8c0 2024-04-25T11:21:37.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.106+0000 7f3fdb400640 1 -- 172.21.15.7:0/422624294 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc032640 con 0x7f3fdc06e8c0 2024-04-25T11:21:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.107+0000 7f3fe2600640 1 -- 172.21.15.7:0/422624294 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc06e8c0 msgr2=0x7f3fdc0d1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.107+0000 7f3fe2600640 1 --2- 172.21.15.7:0/422624294 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc06e8c0 0x7f3fdc0d1ce0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0097a0 tx=0x7f3fcc02a100 comp rx=0 tx=0).stop 2024-04-25T11:21:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.107+0000 7f3fe2600640 1 -- 172.21.15.7:0/422624294 shutdown_connections 2024-04-25T11:21:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.107+0000 7f3fe2600640 1 --2- 172.21.15.7:0/422624294 >> v2:172.21.15.87:3300/0 conn(0x7f3fdc0d2220 0x7f3fdc0d2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.107+0000 7f3fe2600640 1 --2- 172.21.15.7:0/422624294 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc06e8c0 0x7f3fdc0d1ce0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.107+0000 7f3fe2600640 1 -- 172.21.15.7:0/422624294 >> 172.21.15.7:0/422624294 conn(0x7f3fdc0d01d0 msgr2=0x7f3fdc0d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.107+0000 7f3fe2600640 1 -- 172.21.15.7:0/422624294 shutdown_connections 2024-04-25T11:21:37.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.107+0000 7f3fe2600640 1 -- 172.21.15.7:0/422624294 wait complete. 2024-04-25T11:21:37.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.110+0000 7f3fe2600640 1 Processor -- start 2024-04-25T11:21:37.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.110+0000 7f3fe2600640 1 -- start start 2024-04-25T11:21:37.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.112+0000 7f3fe2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06e4b0 0x7f3fdc0b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:37.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.112+0000 7f3fe2600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fdc06e8c0 0x7f3fdc1f9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:37.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.112+0000 7f3fe2600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc0d2220 0x7f3fdc1f9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:37.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.112+0000 7f3fe2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc0b8fd0 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.112+0000 7f3fe2600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc06da60 con 0x7f3fdc06e8c0 2024-04-25T11:21:37.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.112+0000 7f3fe2600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3fdc0cda60 con 0x7f3fdc0d2220 2024-04-25T11:21:37.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.112+0000 7f3fe1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06e4b0 0x7f3fdc0b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:37.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.112+0000 7f3fe1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06e4b0 0x7f3fdc0b7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50906/0 (socket says 172.21.15.7:50906) 2024-04-25T11:21:37.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.112+0000 7f3fe1200640 1 -- 172.21.15.7:0/171182644 learned_addr learned my addr 172.21.15.7:0/171182644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.112+0000 7f3fe1c00640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc0d2220 0x7f3fdc1f9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.112+0000 7f3fdbe00640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fdc06e8c0 0x7f3fdc1f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.113+0000 7f3fe1200640 1 -- 172.21.15.7:0/171182644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc0d2220 msgr2=0x7f3fdc1f9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.113+0000 7f3fe1200640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc0d2220 0x7f3fdc1f9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.113+0000 7f3fe1200640 1 -- 172.21.15.7:0/171182644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fdc06e8c0 msgr2=0x7f3fdc1f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.113+0000 7f3fe1200640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fdc06e8c0 0x7f3fdc1f9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.113+0000 7f3fe1200640 1 -- 172.21.15.7:0/171182644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fdc0ca9c0 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.113+0000 7f3fe1c00640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc0d2220 0x7f3fdc1f9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.113+0000 7f3fdbe00640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fdc06e8c0 0x7f3fdc1f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.113+0000 7f3fe1200640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06e4b0 0x7f3fdc0b7c70 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000c390 tx=0x7f3fd000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.113+0000 7f3fd8c00640 1 -- 172.21.15.7:0/171182644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0017070 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.113+0000 7f3fd8c00640 1 -- 172.21.15.7:0/171182644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd0005d00 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.113+0000 7f3fd8c00640 1 -- 172.21.15.7:0/171182644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0007310 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.114+0000 7f3fe2600640 1 -- 172.21.15.7:0/171182644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fdc0caca0 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.114+0000 7f3fe2600640 1 -- 172.21.15.7:0/171182644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fdc160cb0 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.115+0000 7f3fe2600640 1 -- 172.21.15.7:0/171182644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fdc050740 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.117+0000 7f3fd8c00640 1 -- 172.21.15.7:0/171182644 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3fd001a070 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.117+0000 7f3fd8c00640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3fc00744b0 0x7f3fc00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:37.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.118+0000 7f3fd8c00640 1 -- 172.21.15.7:0/171182644 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 6001+0+0 (secure 0 0 0) 0x7f3fd0094b90 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.118+0000 7f3fdbe00640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3fc00744b0 0x7f3fc00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:37.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.119+0000 7f3fdbe00640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3fc00744b0 0x7f3fc00769c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc034000 tx=0x7f3fcc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:37.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.122+0000 7f3fd8c00640 1 -- 172.21.15.7:0/171182644 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3fd0021020 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.363+0000 7f3fe2600640 1 -- 172.21.15.7:0/171182644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3fdc0d0fa0 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.842+0000 7f3fd8c00640 1 -- 172.21.15.7:0/171182644 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v99) ==== 174+0+0 (secure 0 0 0) 0x7f3fd005f260 con 0x7f3fdc06e4b0 2024-04-25T11:21:37.838 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 21 and data pool 22 2024-04-25T11:21:37.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.851+0000 7f3fe2600640 1 -- 172.21.15.7:0/171182644 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3fc00744b0 msgr2=0x7f3fc00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:37.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.851+0000 7f3fe2600640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3fc00744b0 0x7f3fc00769c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc034000 tx=0x7f3fcc02e000 comp rx=0 tx=0).stop 2024-04-25T11:21:37.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.851+0000 7f3fe2600640 1 -- 172.21.15.7:0/171182644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06e4b0 msgr2=0x7f3fdc0b7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:37.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.851+0000 7f3fe2600640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06e4b0 0x7f3fdc0b7c70 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000c390 tx=0x7f3fd000c850 comp rx=0 tx=0).stop 2024-04-25T11:21:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.854+0000 7f3fe2600640 1 -- 172.21.15.7:0/171182644 shutdown_connections 2024-04-25T11:21:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.854+0000 7f3fe2600640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3fc00744b0 0x7f3fc00769c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.854+0000 7f3fe2600640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3fdc0d2220 0x7f3fdc1f9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.854+0000 7f3fe2600640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3fdc06e8c0 0x7f3fdc1f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.854+0000 7f3fe2600640 1 --2- 172.21.15.7:0/171182644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3fdc06e4b0 0x7f3fdc0b7c70 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.854+0000 7f3fe2600640 1 -- 172.21.15.7:0/171182644 >> 172.21.15.7:0/171182644 conn(0x7f3fdc0d01d0 msgr2=0x7f3fdc072510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:37.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.857+0000 7f3fe2600640 1 -- 172.21.15.7:0/171182644 shutdown_connections 2024-04-25T11:21:37.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:37.861+0000 7f3fe2600640 1 -- 172.21.15.7:0/171182644 wait complete. 2024-04-25T11:21:37.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:21:38.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.153+0000 7fcfb7200640 1 Processor -- start 2024-04-25T11:21:38.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.154+0000 7fcfb7200640 1 -- start start 2024-04-25T11:21:38.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.155+0000 7fcfb7200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfb80aed40 0x7fcfb815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.156+0000 7fcfb7200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcfb80af420 0x7fcfb80af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.156+0000 7fcfb7200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfb80cebb0 con 0x7fcfb80b5220 2024-04-25T11:21:38.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.156+0000 7fcfb7200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fcfb815e1f0 con 0x7fcfb80af420 2024-04-25T11:21:38.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.156+0000 7fcfb7200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcfb80afd70 con 0x7fcfb80aed40 2024-04-25T11:21:38.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.156+0000 7fcfafe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcfb80b5220 0x7fcfb80ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53904/0 (socket says 172.21.15.7:53904) 2024-04-25T11:21:38.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.156+0000 7fcfafe00640 1 -- 172.21.15.7:0/1121971448 learned_addr learned my addr 172.21.15.7:0/1121971448 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:38.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.156+0000 7fcfb6800640 1 --2- 172.21.15.7:0/1121971448 >> v2:172.21.15.7:3300/0 conn(0x7fcfb80af420 0x7fcfb80af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.156+0000 7fcfb5e00640 1 --2- 172.21.15.7:0/1121971448 >> v2:172.21.15.87:3300/0 conn(0x7fcfb80aed40 0x7fcfb815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.156+0000 7fcfb5400640 1 -- 172.21.15.7:0/1121971448 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 191893278 0 0) 0x7fcfa0001110 con 0x7fcfb80b5220 2024-04-25T11:21:38.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.156+0000 7fcfb5400640 1 -- 172.21.15.7:0/1121971448 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf9c003520 con 0x7fcfb80b5220 2024-04-25T11:21:38.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.157+0000 7fcfb6800640 1 -- 172.21.15.7:0/1121971448 >> v2:172.21.15.87:3300/0 conn(0x7fcfb80aed40 msgr2=0x7fcfb815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:38.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.157+0000 7fcfb6800640 1 --2- 172.21.15.7:0/1121971448 >> v2:172.21.15.87:3300/0 conn(0x7fcfb80aed40 0x7fcfb815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.157+0000 7fcfb6800640 1 -- 172.21.15.7:0/1121971448 >> v1:172.21.15.7:6789/0 conn(0x7fcfb80b5220 legacy=0x7fcfb80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.157+0000 7fcfb6800640 1 -- 172.21.15.7:0/1121971448 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb80b64b0 con 0x7fcfb80af420 2024-04-25T11:21:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.158+0000 7fcfb6800640 1 --2- 172.21.15.7:0/1121971448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80af420 0x7fcfb80af830 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80725e0 tx=0x7fcfb002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9f29ff4e80f4e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.158+0000 7fcfb5400640 1 -- 172.21.15.7:0/1121971448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0007740 con 0x7fcfb80af420 2024-04-25T11:21:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.158+0000 7fcfb5400640 1 -- 172.21.15.7:0/1121971448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb00078e0 con 0x7fcfb80af420 2024-04-25T11:21:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.158+0000 7fcfb5400640 1 -- 172.21.15.7:0/1121971448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0007cb0 con 0x7fcfb80af420 2024-04-25T11:21:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.159+0000 7fcfb7200640 1 -- 172.21.15.7:0/1121971448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80af420 msgr2=0x7fcfb80af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.159+0000 7fcfb7200640 1 --2- 172.21.15.7:0/1121971448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80af420 0x7fcfb80af830 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80725e0 tx=0x7fcfb002a680 comp rx=0 tx=0).stop 2024-04-25T11:21:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.159+0000 7fcfb7200640 1 -- 172.21.15.7:0/1121971448 shutdown_connections 2024-04-25T11:21:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.159+0000 7fcfb7200640 1 --2- 172.21.15.7:0/1121971448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80af420 0x7fcfb80af830 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.159+0000 7fcfb7200640 1 --2- 172.21.15.7:0/1121971448 >> v2:172.21.15.87:3300/0 conn(0x7fcfb80aed40 0x7fcfb815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.159+0000 7fcfb7200640 1 -- 172.21.15.7:0/1121971448 >> 172.21.15.7:0/1121971448 conn(0x7fcfb80b3820 msgr2=0x7fcfb80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:38.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.159+0000 7fcfb7200640 1 -- 172.21.15.7:0/1121971448 shutdown_connections 2024-04-25T11:21:38.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.159+0000 7fcfb7200640 1 -- 172.21.15.7:0/1121971448 wait complete. 2024-04-25T11:21:38.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.161+0000 7fcfb7200640 1 Processor -- start 2024-04-25T11:21:38.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.162+0000 7fcfb7200640 1 -- start start 2024-04-25T11:21:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.163+0000 7fcfb7200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb80aed40 0x7fcfb8147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.163+0000 7fcfb7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80af420 0x7fcfb81f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.163+0000 7fcfb7200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80b5220 0x7fcfb8147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.163+0000 7fcfb7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb806cbd0 con 0x7fcfb80af420 2024-04-25T11:21:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.163+0000 7fcfb7200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb806c8f0 con 0x7fcfb80aed40 2024-04-25T11:21:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.163+0000 7fcfb7200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb806ceb0 con 0x7fcfb80b5220 2024-04-25T11:21:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.163+0000 7fcfafe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80af420 0x7fcfb81f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.163+0000 7fcfafe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80af420 0x7fcfb81f4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50918/0 (socket says 172.21.15.7:50918) 2024-04-25T11:21:38.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.163+0000 7fcfafe00640 1 -- 172.21.15.7:0/4087981554 learned_addr learned my addr 172.21.15.7:0/4087981554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.164+0000 7fcfb5e00640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb80aed40 0x7fcfb8147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.164+0000 7fcfb6800640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80b5220 0x7fcfb8147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.164+0000 7fcfafe00640 1 -- 172.21.15.7:0/4087981554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80b5220 msgr2=0x7fcfb8147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.164+0000 7fcfafe00640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80b5220 0x7fcfb8147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.164+0000 7fcfafe00640 1 -- 172.21.15.7:0/4087981554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb80aed40 msgr2=0x7fcfb8147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.164+0000 7fcfafe00640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb80aed40 0x7fcfb8147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.164+0000 7fcfafe00640 1 -- 172.21.15.7:0/4087981554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb81f51e0 con 0x7fcfb80af420 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.164+0000 7fcfb6800640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80b5220 0x7fcfb8147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.164+0000 7fcfafe00640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80af420 0x7fcfb81f4ca0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c3d0 tx=0x7fcfa000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.164+0000 7fcfae000640 1 -- 172.21.15.7:0/4087981554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0017070 con 0x7fcfb80af420 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.165+0000 7fcfae000640 1 -- 172.21.15.7:0/4087981554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa0005ce0 con 0x7fcfb80af420 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.165+0000 7fcfae000640 1 -- 172.21.15.7:0/4087981554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0007310 con 0x7fcfb80af420 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.165+0000 7fcfb7200640 1 -- 172.21.15.7:0/4087981554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb8149d70 con 0x7fcfb80af420 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.165+0000 7fcfb7200640 1 -- 172.21.15.7:0/4087981554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb814a330 con 0x7fcfb80af420 2024-04-25T11:21:38.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.166+0000 7fcfb7200640 1 -- 172.21.15.7:0/4087981554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb8149ae0 con 0x7fcfb80af420 2024-04-25T11:21:38.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.167+0000 7fcfae000640 1 -- 172.21.15.7:0/4087981554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcfa001a070 con 0x7fcfb80af420 2024-04-25T11:21:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.168+0000 7fcfae000640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf940744b0 0x7fcf940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.169+0000 7fcfae000640 1 -- 172.21.15.7:0/4087981554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6101+0+0 (secure 0 0 0) 0x7fcfa0094b20 con 0x7fcfb80af420 2024-04-25T11:21:38.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.169+0000 7fcfb5e00640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf940744b0 0x7fcf940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.170+0000 7fcfb5e00640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf940744b0 0x7fcf940769c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8071200 tx=0x7fcfa8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:38.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.172+0000 7fcfae000640 1 -- 172.21.15.7:0/4087981554 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcfa0021020 con 0x7fcfb80af420 2024-04-25T11:21:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.376+0000 7fcfb7200640 1 -- 172.21.15.7:0/4087981554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfb8050740 con 0x7fcfb80af420 2024-04-25T11:21:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.379+0000 7fcfae000640 1 -- 172.21.15.7:0/4087981554 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16809 (secure 0 0 0) 0x7fcfb80cebb0 con 0x7fcfb80af420 2024-04-25T11:21:38.375 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:38.375 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":141,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:21:37.365497+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:21:35.562066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T11:21:36.342958+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.381+0000 7fcfb7200640 1 -- 172.21.15.7:0/4087981554 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf940744b0 msgr2=0x7fcf940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.381+0000 7fcfb7200640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf940744b0 0x7fcf940769c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8071200 tx=0x7fcfa8008040 comp rx=0 tx=0).stop 2024-04-25T11:21:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.381+0000 7fcfb7200640 1 -- 172.21.15.7:0/4087981554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80af420 msgr2=0x7fcfb81f4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.381+0000 7fcfb7200640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80af420 0x7fcfb81f4ca0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c3d0 tx=0x7fcfa000c890 comp rx=0 tx=0).stop 2024-04-25T11:21:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.382+0000 7fcfb7200640 1 -- 172.21.15.7:0/4087981554 shutdown_connections 2024-04-25T11:21:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.382+0000 7fcfb7200640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf940744b0 0x7fcf940769c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.382+0000 7fcfb7200640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfb80b5220 0x7fcfb8147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.382+0000 7fcfb7200640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfb80af420 0x7fcfb81f4ca0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.382+0000 7fcfb7200640 1 --2- 172.21.15.7:0/4087981554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfb80aed40 0x7fcfb8147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.382+0000 7fcfb7200640 1 -- 172.21.15.7:0/4087981554 >> 172.21.15.7:0/4087981554 conn(0x7fcfb80b3820 msgr2=0x7fcfb80ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:38.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.382+0000 7fcfb7200640 1 -- 172.21.15.7:0/4087981554 shutdown_connections 2024-04-25T11:21:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.382+0000 7fcfb7200640 1 -- 172.21.15.7:0/4087981554 wait complete. 2024-04-25T11:21:38.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:21:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.696+0000 7f24f9400640 1 Processor -- start 2024-04-25T11:21:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.696+0000 7f24f9400640 1 -- start start 2024-04-25T11:21:38.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.698+0000 7f24f9400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f4160350 0x7f24f40d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.698+0000 7f24f9400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24f40d1650 con 0x7f24f40b4730 2024-04-25T11:21:38.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.698+0000 7f24f9400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24f4163ce0 con 0x7f24f40b4c10 2024-04-25T11:21:38.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.698+0000 7f24f9400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f24f41645f0 con 0x7f24f4160350 2024-04-25T11:21:38.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.698+0000 7f24f1e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f24f40b4730 0x7f24f40d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53916/0 (socket says 172.21.15.7:53916) 2024-04-25T11:21:38.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.698+0000 7f24f1e00640 1 -- 172.21.15.7:0/426078552 learned_addr learned my addr 172.21.15.7:0/426078552 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:38.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f2800640 1 --2- 172.21.15.7:0/426078552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f4160350 0x7f24f40d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 421167622 0 0) 0x7f24f40d1650 con 0x7f24f40b4730 2024-04-25T11:21:38.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24d8003520 con 0x7f24f40b4730 2024-04-25T11:21:38.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1376537152 0 0) 0x7f24d8003520 con 0x7f24f40b4730 2024-04-25T11:21:38.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f4160350 msgr2=0x7f24f40d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:38.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 --2- 172.21.15.7:0/426078552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f4160350 0x7f24f40d23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 >> v1:172.21.15.87:6789/0 conn(0x7f24f40b4c10 legacy=0x7f24f40d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f40b83f0 con 0x7f24f40b4730 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f24dc001670 con 0x7f24f40b4730 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 >> v1:172.21.15.7:6789/0 conn(0x7f24f40b4730 legacy=0x7f24f40d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 --2- 172.21.15.7:0/426078552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 0x7f24d8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 --2- 172.21.15.7:0/426078552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d8004800 0x7f24d8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 --2- 172.21.15.7:0/426078552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f4160350 0x7f24d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f24f41645f0 con 0x7f24d80039e0 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8005590 con 0x7f24d8004800 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.699+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f24d800bf00 con 0x7f24f4160350 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.700+0000 7f24f2800640 1 --2- 172.21.15.7:0/426078552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 0x7f24d8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.700+0000 7f24f1e00640 1 --2- 172.21.15.7:0/426078552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d8004800 0x7f24d8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.700+0000 7f24f2800640 1 -- 172.21.15.7:0/426078552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f4160350 msgr2=0x7f24d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.700+0000 7f24f2800640 1 --2- 172.21.15.7:0/426078552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f4160350 0x7f24d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.700+0000 7f24f2800640 1 -- 172.21.15.7:0/426078552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d8004800 msgr2=0x7f24d8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.700+0000 7f24f2800640 1 --2- 172.21.15.7:0/426078552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24d8004800 0x7f24d8004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.700+0000 7f24f2800640 1 -- 172.21.15.7:0/426078552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d8003520 con 0x7f24d80039e0 2024-04-25T11:21:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.701+0000 7f24f2800640 1 --2- 172.21.15.7:0/426078552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 0x7f24d8003e50 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f24e400ba30 tx=0x7f24d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b31543d1a7b6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.701+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4030070 con 0x7f24d80039e0 2024-04-25T11:21:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.701+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e402ae10 con 0x7f24d80039e0 2024-04-25T11:21:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.701+0000 7f24f1400640 1 -- 172.21.15.7:0/426078552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e400b600 con 0x7f24d80039e0 2024-04-25T11:21:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.702+0000 7f24f9400640 1 -- 172.21.15.7:0/426078552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 msgr2=0x7f24d8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.702+0000 7f24f9400640 1 --2- 172.21.15.7:0/426078552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 0x7f24d8003e50 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f24e400ba30 tx=0x7f24d8005830 comp rx=0 tx=0).stop 2024-04-25T11:21:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.702+0000 7f24f3200640 1 -- 172.21.15.7:0/426078552 reap_dead start 2024-04-25T11:21:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.702+0000 7f24f9400640 1 -- 172.21.15.7:0/426078552 shutdown_connections 2024-04-25T11:21:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.702+0000 7f24f9400640 1 -- 172.21.15.7:0/426078552 >> 172.21.15.7:0/426078552 conn(0x7f24f406edb0 msgr2=0x7f24f406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:38.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.702+0000 7f24f9400640 1 -- 172.21.15.7:0/426078552 shutdown_connections 2024-04-25T11:21:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.702+0000 7f24f9400640 1 -- 172.21.15.7:0/426078552 wait complete. 2024-04-25T11:21:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.705+0000 7f24f9400640 1 Processor -- start 2024-04-25T11:21:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.705+0000 7f24f9400640 1 -- start start 2024-04-25T11:21:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 0x7f24f415dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f9400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f40cbb70 0x7f24f40cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f4147ef0 0x7f24f4148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f9400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f24f4164280 con 0x7f24d80039e0 2024-04-25T11:21:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f9400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f24f40d54e0 con 0x7f24f4147ef0 2024-04-25T11:21:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f9400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f24f40b8080 con 0x7f24f40cbb70 2024-04-25T11:21:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 0x7f24f415dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 0x7f24f415dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50930/0 (socket says 172.21.15.7:50930) 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f2800640 1 -- 172.21.15.7:0/638746131 learned_addr learned my addr 172.21.15.7:0/638746131 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f1e00640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f40cbb70 0x7f24f40cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f3200640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f4147ef0 0x7f24f4148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f2800640 1 -- 172.21.15.7:0/638746131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f40cbb70 msgr2=0x7f24f40cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f2800640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f40cbb70 0x7f24f40cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f2800640 1 -- 172.21.15.7:0/638746131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f4147ef0 msgr2=0x7f24f4148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f2800640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f4147ef0 0x7f24f4148300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.707+0000 7f24f2800640 1 -- 172.21.15.7:0/638746131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f415e1b0 con 0x7f24d80039e0 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.708+0000 7f24f2800640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 0x7f24f415dc70 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f24e402a900 tx=0x7f24e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.708+0000 7f24eaa00640 1 -- 172.21.15.7:0/638746131 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4042030 con 0x7f24d80039e0 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.708+0000 7f24eaa00640 1 -- 172.21.15.7:0/638746131 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e402ac10 con 0x7f24d80039e0 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.708+0000 7f24eaa00640 1 -- 172.21.15.7:0/638746131 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e4004240 con 0x7f24d80039e0 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.708+0000 7f24f9400640 1 -- 172.21.15.7:0/638746131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f415e6c0 con 0x7f24d80039e0 2024-04-25T11:21:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.708+0000 7f24f9400640 1 -- 172.21.15.7:0/638746131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f415ec80 con 0x7f24d80039e0 2024-04-25T11:21:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.710+0000 7f24f9400640 1 -- 172.21.15.7:0/638746131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f406a100 con 0x7f24d80039e0 2024-04-25T11:21:38.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.711+0000 7f24eaa00640 1 -- 172.21.15.7:0/638746131 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f24e4045070 con 0x7f24d80039e0 2024-04-25T11:21:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.712+0000 7f24eaa00640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24d00744e0 0x7f24d00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.712+0000 7f24eaa00640 1 -- 172.21.15.7:0/638746131 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 6101+0+0 (secure 0 0 0) 0x7f24e40b5570 con 0x7f24d80039e0 2024-04-25T11:21:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.712+0000 7f24f1e00640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24d00744e0 0x7f24d00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.713+0000 7f24f1e00640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24d00744e0 0x7f24d00769f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f24dc000f60 tx=0x7f24dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:38.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.717+0000 7f24eaa00640 1 -- 172.21.15.7:0/638746131 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f24e407fad0 con 0x7f24d80039e0 2024-04-25T11:21:38.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.925+0000 7f24f9400640 1 -- 172.21.15.7:0/638746131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24f40d3ff0 con 0x7f24d80039e0 2024-04-25T11:21:38.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.927+0000 7f24eaa00640 1 -- 172.21.15.7:0/638746131 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16796 (secure 0 0 0) 0x7f24e407f3d0 con 0x7f24d80039e0 2024-04-25T11:21:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:38.924 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":142,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:21:38.842722+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:21:35.562066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T11:21:36.342958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 -- 172.21.15.7:0/638746131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24d00744e0 msgr2=0x7f24d00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24d00744e0 0x7f24d00769f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f24dc000f60 tx=0x7f24dc009040 comp rx=0 tx=0).stop 2024-04-25T11:21:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 -- 172.21.15.7:0/638746131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 msgr2=0x7f24f415dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 0x7f24f415dc70 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f24e402a900 tx=0x7f24e4004730 comp rx=0 tx=0).stop 2024-04-25T11:21:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 -- 172.21.15.7:0/638746131 shutdown_connections 2024-04-25T11:21:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f24d00744e0 0x7f24d00769f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24f4147ef0 0x7f24f4148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24f40cbb70 0x7f24f40cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 --2- 172.21.15.7:0/638746131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24d80039e0 0x7f24f415dc70 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:38.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 -- 172.21.15.7:0/638746131 >> 172.21.15.7:0/638746131 conn(0x7f24f406edb0 msgr2=0x7f24f40ba0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 -- 172.21.15.7:0/638746131 shutdown_connections 2024-04-25T11:21:38.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:38.930+0000 7f24f9400640 1 -- 172.21.15.7:0/638746131 wait complete. 2024-04-25T11:21:38.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:21:39.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.236+0000 7f61fee00640 1 Processor -- start 2024-04-25T11:21:39.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.237+0000 7f61fee00640 1 -- start start 2024-04-25T11:21:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.239+0000 7f61fee00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f61f81684f0 0x7f61f80d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.239+0000 7f61fee00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61f806f330 0x7f61f806f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.239+0000 7f61fee00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f80d09e0 con 0x7f61f80cff00 2024-04-25T11:21:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.239+0000 7f61fee00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f61f806eea0 con 0x7f61f81684f0 2024-04-25T11:21:39.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.239+0000 7f61fee00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f61f80d0cc0 con 0x7f61f806f330 2024-04-25T11:21:39.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.240+0000 7f61fda00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f61f80cff00 0x7f61f8167de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53924/0 (socket says 172.21.15.7:53924) 2024-04-25T11:21:39.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.240+0000 7f61fda00640 1 -- 172.21.15.7:0/2336147276 learned_addr learned my addr 172.21.15.7:0/2336147276 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:39.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.240+0000 7f61fd000640 1 --2- 172.21.15.7:0/2336147276 >> v2:172.21.15.87:3300/0 conn(0x7f61f81684f0 0x7f61f80d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:39.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.240+0000 7f61fe400640 1 --2- 172.21.15.7:0/2336147276 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61f806f330 0x7f61f806f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:39.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.241+0000 7f61f7e00640 1 -- 172.21.15.7:0/2336147276 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613418635 0 0) 0x7f61f80d09e0 con 0x7f61f80cff00 2024-04-25T11:21:39.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.241+0000 7f61f7e00640 1 -- 172.21.15.7:0/2336147276 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61e0003520 con 0x7f61f80cff00 2024-04-25T11:21:39.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.241+0000 7f61fd000640 1 -- 172.21.15.7:0/2336147276 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61f806f330 msgr2=0x7f61f806f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:39.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.241+0000 7f61fd000640 1 --2- 172.21.15.7:0/2336147276 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61f806f330 0x7f61f806f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:39.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.241+0000 7f61f7e00640 1 -- 172.21.15.7:0/2336147276 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4189701615 0 0) 0x7f61e0003520 con 0x7f61f80cff00 2024-04-25T11:21:39.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.241+0000 7f61fd000640 1 -- 172.21.15.7:0/2336147276 >> v1:172.21.15.7:6789/0 conn(0x7f61f80cff00 legacy=0x7f61f8167de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:39.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.241+0000 7f61fd000640 1 -- 172.21.15.7:0/2336147276 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f80d0fa0 con 0x7f61f81684f0 2024-04-25T11:21:39.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.243+0000 7f61fd000640 1 --2- 172.21.15.7:0/2336147276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f81684f0 0x7f61f80d0310 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f61e40097a0 tx=0x7f61e402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a8a6e7682a0fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:39.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.243+0000 7f61f7e00640 1 -- 172.21.15.7:0/2336147276 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4037070 con 0x7f61f81684f0 2024-04-25T11:21:39.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.243+0000 7f61f7e00640 1 -- 172.21.15.7:0/2336147276 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e402aba0 con 0x7f61f81684f0 2024-04-25T11:21:39.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.243+0000 7f61f7e00640 1 -- 172.21.15.7:0/2336147276 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61ec001690 con 0x7f61f81684f0 2024-04-25T11:21:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.244+0000 7f61fee00640 1 -- 172.21.15.7:0/2336147276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f81684f0 msgr2=0x7f61f80d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.244+0000 7f61fee00640 1 --2- 172.21.15.7:0/2336147276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f81684f0 0x7f61f80d0310 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f61e40097a0 tx=0x7f61e402a110 comp rx=0 tx=0).stop 2024-04-25T11:21:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.244+0000 7f61fee00640 1 -- 172.21.15.7:0/2336147276 shutdown_connections 2024-04-25T11:21:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.244+0000 7f61fee00640 1 --2- 172.21.15.7:0/2336147276 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61f806f330 0x7f61f806f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.244+0000 7f61fee00640 1 --2- 172.21.15.7:0/2336147276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f81684f0 0x7f61f80d0310 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.244+0000 7f61fee00640 1 -- 172.21.15.7:0/2336147276 >> 172.21.15.7:0/2336147276 conn(0x7f61f80740f0 msgr2=0x7f61f806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.244+0000 7f61fee00640 1 -- 172.21.15.7:0/2336147276 shutdown_connections 2024-04-25T11:21:39.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.244+0000 7f61fee00640 1 -- 172.21.15.7:0/2336147276 wait complete. 2024-04-25T11:21:39.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.247+0000 7f61fee00640 1 Processor -- start 2024-04-25T11:21:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.247+0000 7f61fee00640 1 -- start start 2024-04-25T11:21:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fee00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f806f330 0x7f61f815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fee00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61f80cff00 0x7f61f806c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f81496c0 0x7f61f806cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61f8168bb0 con 0x7f61f81496c0 2024-04-25T11:21:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fee00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f61f8168a30 con 0x7f61f806f330 2024-04-25T11:21:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fee00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f61f8147340 con 0x7f61f80cff00 2024-04-25T11:21:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fe400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f81496c0 0x7f61f806cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fe400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f81496c0 0x7f61f806cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50946/0 (socket says 172.21.15.7:50946) 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fe400640 1 -- 172.21.15.7:0/3529168794 learned_addr learned my addr 172.21.15.7:0/3529168794 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fda00640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f806f330 0x7f61f815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fd000640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61f80cff00 0x7f61f806c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fe400640 1 -- 172.21.15.7:0/3529168794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61f80cff00 msgr2=0x7f61f806c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fe400640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61f80cff00 0x7f61f806c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fe400640 1 -- 172.21.15.7:0/3529168794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f806f330 msgr2=0x7f61f815a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.249+0000 7f61fe400640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f806f330 0x7f61f815a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.250+0000 7f61fe400640 1 -- 172.21.15.7:0/3529168794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f8147e20 con 0x7f61f81496c0 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.250+0000 7f61fe400640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f81496c0 0x7f61f806cab0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009d30 tx=0x7f61f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.250+0000 7f61f5600640 1 -- 172.21.15.7:0/3529168794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0018020 con 0x7f61f81496c0 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.250+0000 7f61f5600640 1 -- 172.21.15.7:0/3529168794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f0014480 con 0x7f61f81496c0 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.250+0000 7f61fee00640 1 -- 172.21.15.7:0/3529168794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f8148100 con 0x7f61f81496c0 2024-04-25T11:21:39.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.250+0000 7f61f5600640 1 -- 172.21.15.7:0/3529168794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0014870 con 0x7f61f81496c0 2024-04-25T11:21:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.250+0000 7f61fee00640 1 -- 172.21.15.7:0/3529168794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f81fb780 con 0x7f61f81496c0 2024-04-25T11:21:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.252+0000 7f61fee00640 1 -- 172.21.15.7:0/3529168794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f8050740 con 0x7f61f81496c0 2024-04-25T11:21:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.253+0000 7f61f5600640 1 -- 172.21.15.7:0/3529168794 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f61f0005d10 con 0x7f61f81496c0 2024-04-25T11:21:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.254+0000 7f61f5600640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f61d80744e0 0x7f61d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.255+0000 7f61f5600640 1 -- 172.21.15.7:0/3529168794 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f61f0095f80 con 0x7f61f81496c0 2024-04-25T11:21:39.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.255+0000 7f61fda00640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f61d80744e0 0x7f61d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:39.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.256+0000 7f61fda00640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f61d80744e0 0x7f61d80769f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f61ec005880 tx=0x7f61ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:39.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.259+0000 7f61f5600640 1 -- 172.21.15.7:0/3529168794 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f61f0022070 con 0x7f61f81496c0 2024-04-25T11:21:39.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.502+0000 7f61fee00640 1 -- 172.21.15.7:0/3529168794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f61f8147bb0 con 0x7f61f81496c0 2024-04-25T11:21:39.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.857+0000 7f61f5600640 1 -- 172.21.15.7:0/3529168794 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v102) ==== 118+0+0 (secure 0 0 0) 0x7f61f0010040 con 0x7f61f81496c0 2024-04-25T11:21:39.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.861+0000 7f61fee00640 1 -- 172.21.15.7:0/3529168794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f61d80744e0 msgr2=0x7f61d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:39.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.861+0000 7f61fee00640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f61d80744e0 0x7f61d80769f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f61ec005880 tx=0x7f61ec009040 comp rx=0 tx=0).stop 2024-04-25T11:21:39.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.861+0000 7f61fee00640 1 -- 172.21.15.7:0/3529168794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f81496c0 msgr2=0x7f61f806cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:39.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.861+0000 7f61fee00640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f81496c0 0x7f61f806cab0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009d30 tx=0x7f61f000c490 comp rx=0 tx=0).stop 2024-04-25T11:21:39.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.861+0000 7f61fee00640 1 -- 172.21.15.7:0/3529168794 shutdown_connections 2024-04-25T11:21:39.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.861+0000 7f61fee00640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f61d80744e0 0x7f61d80769f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:39.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.861+0000 7f61fee00640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f81496c0 0x7f61f806cab0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:39.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.861+0000 7f61fee00640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f61f80cff00 0x7f61f806c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:39.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.861+0000 7f61fee00640 1 --2- 172.21.15.7:0/3529168794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61f806f330 0x7f61f815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:39.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.861+0000 7f61fee00640 1 -- 172.21.15.7:0/3529168794 >> 172.21.15.7:0/3529168794 conn(0x7f61f80740f0 msgr2=0x7f61f80b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:39.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.861+0000 7f61fee00640 1 -- 172.21.15.7:0/3529168794 shutdown_connections 2024-04-25T11:21:39.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:39.862+0000 7f61fee00640 1 -- 172.21.15.7:0/3529168794 wait complete. 2024-04-25T11:21:39.897 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:21:39.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:21:40.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.187+0000 7f93c1e00640 1 Processor -- start 2024-04-25T11:21:40.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.187+0000 7f93c1e00640 1 -- start start 2024-04-25T11:21:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.189+0000 7f93c1e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f93bc15aaa0 0x7f93bc0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.189+0000 7f93c1e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93bc0d0ee0 0x7f93bc0b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.189+0000 7f93c1e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93bc06f5b0 con 0x7f93bc0b4e50 2024-04-25T11:21:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.189+0000 7f93c1e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f93bc0af290 con 0x7f93bc15aaa0 2024-04-25T11:21:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.189+0000 7f93c1e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f93bc0d14a0 con 0x7f93bc0d0ee0 2024-04-25T11:21:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.190+0000 7f93bb400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f93bc0b4e50 0x7f93bc15d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47598/0 (socket says 172.21.15.7:47598) 2024-04-25T11:21:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.190+0000 7f93bb400640 1 -- 172.21.15.7:0/988899614 learned_addr learned my addr 172.21.15.7:0/988899614 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.190+0000 7f93baa00640 1 --2- 172.21.15.7:0/988899614 >> v2:172.21.15.7:3300/0 conn(0x7f93bc15aaa0 0x7f93bc0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.190+0000 7f93ba000640 1 --2- 172.21.15.7:0/988899614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93bc0d0ee0 0x7f93bc0b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.190+0000 7f93b9600640 1 -- 172.21.15.7:0/988899614 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3952389088 0 0) 0x7f93bc06f5b0 con 0x7f93bc0b4e50 2024-04-25T11:21:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.190+0000 7f93b9600640 1 -- 172.21.15.7:0/988899614 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93a0003520 con 0x7f93bc0b4e50 2024-04-25T11:21:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.190+0000 7f93baa00640 1 -- 172.21.15.7:0/988899614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93bc0d0ee0 msgr2=0x7f93bc0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.190+0000 7f93baa00640 1 --2- 172.21.15.7:0/988899614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93bc0d0ee0 0x7f93bc0b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.190+0000 7f93baa00640 1 -- 172.21.15.7:0/988899614 >> v1:172.21.15.7:6789/0 conn(0x7f93bc0b4e50 legacy=0x7f93bc15d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.190+0000 7f93baa00640 1 -- 172.21.15.7:0/988899614 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93bc0af4e0 con 0x7f93bc15aaa0 2024-04-25T11:21:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.190+0000 7f93ba000640 1 --2- 172.21.15.7:0/988899614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93bc0d0ee0 0x7f93bc0b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.191+0000 7f93baa00640 1 --2- 172.21.15.7:0/988899614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93bc15aaa0 0x7f93bc0affc0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0097a0 tx=0x7f93ac02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfe498b2b25c05f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.192+0000 7f93b9600640 1 -- 172.21.15.7:0/988899614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac037070 con 0x7f93bc15aaa0 2024-04-25T11:21:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.192+0000 7f93b9600640 1 -- 172.21.15.7:0/988899614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac02ae50 con 0x7f93bc15aaa0 2024-04-25T11:21:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.192+0000 7f93b9600640 1 -- 172.21.15.7:0/988899614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac032660 con 0x7f93bc15aaa0 2024-04-25T11:21:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.192+0000 7f93c1e00640 1 -- 172.21.15.7:0/988899614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93bc15aaa0 msgr2=0x7f93bc0affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.192+0000 7f93c1e00640 1 --2- 172.21.15.7:0/988899614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93bc15aaa0 0x7f93bc0affc0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0097a0 tx=0x7f93ac02a100 comp rx=0 tx=0).stop 2024-04-25T11:21:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.192+0000 7f93c1e00640 1 -- 172.21.15.7:0/988899614 shutdown_connections 2024-04-25T11:21:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.192+0000 7f93c1e00640 1 --2- 172.21.15.7:0/988899614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93bc0d0ee0 0x7f93bc0b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.192+0000 7f93c1e00640 1 --2- 172.21.15.7:0/988899614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93bc15aaa0 0x7f93bc0affc0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.192+0000 7f93c1e00640 1 -- 172.21.15.7:0/988899614 >> 172.21.15.7:0/988899614 conn(0x7f93bc0742c0 msgr2=0x7f93bc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.192+0000 7f93c1e00640 1 -- 172.21.15.7:0/988899614 shutdown_connections 2024-04-25T11:21:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.192+0000 7f93c1e00640 1 -- 172.21.15.7:0/988899614 wait complete. 2024-04-25T11:21:40.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.195+0000 7f93c1e00640 1 Processor -- start 2024-04-25T11:21:40.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.195+0000 7f93c1e00640 1 -- start start 2024-04-25T11:21:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.196+0000 7f93c1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93bc0b4e50 0x7f93bc15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.196+0000 7f93c1e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93bc0d0ee0 0x7f93bc1490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.196+0000 7f93c1e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93bc15aaa0 0x7f93bc159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.196+0000 7f93c1e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc0b5260 con 0x7f93bc0b4e50 2024-04-25T11:21:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.196+0000 7f93c1e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc0afaa0 con 0x7f93bc15aaa0 2024-04-25T11:21:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.196+0000 7f93c1e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f93bc0b53e0 con 0x7f93bc0d0ee0 2024-04-25T11:21:40.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.196+0000 7f93baa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93bc0b4e50 0x7f93bc15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:40.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.197+0000 7f93baa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93bc0b4e50 0x7f93bc15bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46506/0 (socket says 172.21.15.7:46506) 2024-04-25T11:21:40.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.197+0000 7f93baa00640 1 -- 172.21.15.7:0/1453819343 learned_addr learned my addr 172.21.15.7:0/1453819343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.197+0000 7f93baa00640 1 -- 172.21.15.7:0/1453819343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93bc0d0ee0 msgr2=0x7f93bc1490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.197+0000 7f93baa00640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93bc0d0ee0 0x7f93bc1490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.197+0000 7f93baa00640 1 -- 172.21.15.7:0/1453819343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93bc15aaa0 msgr2=0x7f93bc159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.197+0000 7f93baa00640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93bc15aaa0 0x7f93bc159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.197+0000 7f93baa00640 1 -- 172.21.15.7:0/1453819343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93bc1495f0 con 0x7f93bc0b4e50 2024-04-25T11:21:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.197+0000 7f93baa00640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93bc0b4e50 0x7f93bc15bb50 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0098c0 tx=0x7f93ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.198+0000 7f93b3400640 1 -- 172.21.15.7:0/1453819343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac041030 con 0x7f93bc0b4e50 2024-04-25T11:21:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.198+0000 7f93b3400640 1 -- 172.21.15.7:0/1453819343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ac02ac50 con 0x7f93bc0b4e50 2024-04-25T11:21:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.198+0000 7f93b3400640 1 -- 172.21.15.7:0/1453819343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac037040 con 0x7f93bc0b4e50 2024-04-25T11:21:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.198+0000 7f93c1e00640 1 -- 172.21.15.7:0/1453819343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93bc161120 con 0x7f93bc0b4e50 2024-04-25T11:21:40.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.198+0000 7f93c1e00640 1 -- 172.21.15.7:0/1453819343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93bc0cf760 con 0x7f93bc0b4e50 2024-04-25T11:21:40.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.199+0000 7f93c1e00640 1 -- 172.21.15.7:0/1453819343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93bc160e80 con 0x7f93bc0b4e50 2024-04-25T11:21:40.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.200+0000 7f93b3400640 1 -- 172.21.15.7:0/1453819343 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f93ac044070 con 0x7f93bc0b4e50 2024-04-25T11:21:40.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.201+0000 7f93b3400640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f939c0744e0 0x7f939c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:40.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.201+0000 7f93b3400640 1 -- 172.21.15.7:0/1453819343 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f93ac0b5a60 con 0x7f93bc0b4e50 2024-04-25T11:21:40.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.202+0000 7f93ba000640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f939c0744e0 0x7f939c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:40.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.202+0000 7f93ba000640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f939c0744e0 0x7f939c0769f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f93a8006fd0 tx=0x7f93a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:40.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.206+0000 7f93b3400640 1 -- 172.21.15.7:0/1453819343 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f93bc160e80 con 0x7f93bc0b4e50 2024-04-25T11:21:40.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.442+0000 7f93c1e00640 1 -- 172.21.15.7:0/1453819343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f93bc070f20 con 0x7f93bc0b4e50 2024-04-25T11:21:40.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.860+0000 7f93b3400640 1 -- 172.21.15.7:0/1453819343 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v103) ==== 122+0+0 (secure 0 0 0) 0x7f93ac07ffc0 con 0x7f93bc0b4e50 2024-04-25T11:21:40.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.865+0000 7f93c1e00640 1 -- 172.21.15.7:0/1453819343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f939c0744e0 msgr2=0x7f939c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:40.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.865+0000 7f93c1e00640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f939c0744e0 0x7f939c0769f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f93a8006fd0 tx=0x7f93a8008040 comp rx=0 tx=0).stop 2024-04-25T11:21:40.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.865+0000 7f93c1e00640 1 -- 172.21.15.7:0/1453819343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93bc0b4e50 msgr2=0x7f93bc15bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:40.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.865+0000 7f93c1e00640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93bc0b4e50 0x7f93bc15bb50 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0098c0 tx=0x7f93ac0046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:40.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.865+0000 7f93c1e00640 1 -- 172.21.15.7:0/1453819343 shutdown_connections 2024-04-25T11:21:40.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.865+0000 7f93c1e00640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f939c0744e0 0x7f939c0769f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:40.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.865+0000 7f93c1e00640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93bc15aaa0 0x7f93bc159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.865+0000 7f93c1e00640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f93bc0d0ee0 0x7f93bc1490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.865+0000 7f93c1e00640 1 --2- 172.21.15.7:0/1453819343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93bc0b4e50 0x7f93bc15bb50 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.865+0000 7f93c1e00640 1 -- 172.21.15.7:0/1453819343 >> 172.21.15.7:0/1453819343 conn(0x7f93bc0742c0 msgr2=0x7f93bc156f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.866+0000 7f93c1e00640 1 -- 172.21.15.7:0/1453819343 shutdown_connections 2024-04-25T11:21:40.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:40.866+0000 7f93c1e00640 1 -- 172.21.15.7:0/1453819343 wait complete. 2024-04-25T11:21:40.898 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:21:40.898 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:21:41.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.187+0000 7f56aae00640 1 Processor -- start 2024-04-25T11:21:41.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.188+0000 7f56aae00640 1 -- start start 2024-04-25T11:21:41.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.190+0000 7f56aae00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f56a40aed40 0x7f56a415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.190+0000 7f56aae00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f56a40b5220 0x7f56a40ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.190+0000 7f56aae00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a40b64b0 con 0x7f56a40af420 2024-04-25T11:21:41.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.190+0000 7f56aae00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f56a415e1f0 con 0x7f56a40b5220 2024-04-25T11:21:41.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.190+0000 7f56aae00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f56a406c610 con 0x7f56a40aed40 2024-04-25T11:21:41.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.191+0000 7f56a9000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f56a40b5220 0x7f56a40ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.191+0000 7f56a9000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f56a40b5220 0x7f56a40ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46514/0 (socket says 172.21.15.7:46514) 2024-04-25T11:21:41.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.191+0000 7f56a9000640 1 -- 172.21.15.7:0/2952347066 learned_addr learned my addr 172.21.15.7:0/2952347066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:41.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.191+0000 7f56a9a00640 1 --2- 172.21.15.7:0/2952347066 >> v2:172.21.15.87:3300/0 conn(0x7f56a40aed40 0x7f56a415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.191+0000 7f56a9000640 1 -- 172.21.15.7:0/2952347066 >> v2:172.21.15.87:3300/0 conn(0x7f56a40aed40 msgr2=0x7f56a415dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:41.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.191+0000 7f56a9000640 1 --2- 172.21.15.7:0/2952347066 >> v2:172.21.15.87:3300/0 conn(0x7f56a40aed40 0x7f56a415dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.191+0000 7f56a9000640 1 -- 172.21.15.7:0/2952347066 >> v1:172.21.15.87:6789/0 conn(0x7f56a40af420 legacy=0x7f56a40af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:41.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.191+0000 7f56a9000640 1 -- 172.21.15.7:0/2952347066 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a406c8f0 con 0x7f56a40b5220 2024-04-25T11:21:41.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.191+0000 7f56a9a00640 1 --2- 172.21.15.7:0/2952347066 >> v2:172.21.15.87:3300/0 conn(0x7f56a40aed40 0x7f56a415dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:41.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.192+0000 7f56a9000640 1 --2- 172.21.15.7:0/2952347066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a40b5220 0x7f56a40ce4a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f56a40725e0 tx=0x7f569002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a247517967512b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:41.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.193+0000 7f56a3e00640 1 -- 172.21.15.7:0/2952347066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690037070 con 0x7f56a40b5220 2024-04-25T11:21:41.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.193+0000 7f56a3e00640 1 -- 172.21.15.7:0/2952347066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5690032440 con 0x7f56a40b5220 2024-04-25T11:21:41.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.193+0000 7f56a3e00640 1 -- 172.21.15.7:0/2952347066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690032810 con 0x7f56a40b5220 2024-04-25T11:21:41.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.193+0000 7f56aae00640 1 -- 172.21.15.7:0/2952347066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a40b5220 msgr2=0x7f56a40ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:41.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.193+0000 7f56aae00640 1 --2- 172.21.15.7:0/2952347066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a40b5220 0x7f56a40ce4a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f56a40725e0 tx=0x7f569002a230 comp rx=0 tx=0).stop 2024-04-25T11:21:41.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.193+0000 7f56aae00640 1 -- 172.21.15.7:0/2952347066 shutdown_connections 2024-04-25T11:21:41.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.194+0000 7f56aae00640 1 --2- 172.21.15.7:0/2952347066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a40b5220 0x7f56a40ce4a0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.194+0000 7f56aae00640 1 --2- 172.21.15.7:0/2952347066 >> v2:172.21.15.87:3300/0 conn(0x7f56a40aed40 0x7f56a415dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.194+0000 7f56aae00640 1 -- 172.21.15.7:0/2952347066 >> 172.21.15.7:0/2952347066 conn(0x7f56a40b3820 msgr2=0x7f56a40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:41.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.194+0000 7f56aae00640 1 -- 172.21.15.7:0/2952347066 shutdown_connections 2024-04-25T11:21:41.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.194+0000 7f56aae00640 1 -- 172.21.15.7:0/2952347066 wait complete. 2024-04-25T11:21:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.196+0000 7f56aae00640 1 Processor -- start 2024-04-25T11:21:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.196+0000 7f56aae00640 1 -- start start 2024-04-25T11:21:41.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.198+0000 7f56aae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a40aed40 0x7f56a40cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.198+0000 7f56aae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a40af420 0x7f56a4149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.198+0000 7f56aae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56a40b5220 0x7f56a414abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.198+0000 7f56aae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f56a40b6110 con 0x7f56a40aed40 2024-04-25T11:21:41.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.198+0000 7f56aae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f56a406ceb0 con 0x7f56a40af420 2024-04-25T11:21:41.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.198+0000 7f56aae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f56a40b6290 con 0x7f56a40b5220 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.198+0000 7f56a9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a40aed40 0x7f56a40cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.198+0000 7f56a9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a40aed40 0x7f56a40cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46524/0 (socket says 172.21.15.7:46524) 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.198+0000 7f56a9a00640 1 -- 172.21.15.7:0/3131066638 learned_addr learned my addr 172.21.15.7:0/3131066638 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.198+0000 7f56a9000640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a40af420 0x7f56a4149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.199+0000 7f56aa400640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56a40b5220 0x7f56a414abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.199+0000 7f56a9a00640 1 -- 172.21.15.7:0/3131066638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56a40b5220 msgr2=0x7f56a414abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.199+0000 7f56a9a00640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56a40b5220 0x7f56a414abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.199+0000 7f56a9a00640 1 -- 172.21.15.7:0/3131066638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a40af420 msgr2=0x7f56a4149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.199+0000 7f56a9a00640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a40af420 0x7f56a4149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.199+0000 7f56a9a00640 1 -- 172.21.15.7:0/3131066638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a40cd190 con 0x7f56a40aed40 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.199+0000 7f56aa400640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56a40b5220 0x7f56a414abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.199+0000 7f56a9a00640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a40aed40 0x7f56a40cb650 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f569800c7f0 tx=0x7f569800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.199+0000 7f56a1600640 1 -- 172.21.15.7:0/3131066638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698015070 con 0x7f56a40aed40 2024-04-25T11:21:41.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.199+0000 7f56a1600640 1 -- 172.21.15.7:0/3131066638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5698007820 con 0x7f56a40aed40 2024-04-25T11:21:41.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.200+0000 7f56aae00640 1 -- 172.21.15.7:0/3131066638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a40cc0f0 con 0x7f56a40aed40 2024-04-25T11:21:41.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.200+0000 7f56a1600640 1 -- 172.21.15.7:0/3131066638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698007c10 con 0x7f56a40aed40 2024-04-25T11:21:41.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.200+0000 7f56aae00640 1 -- 172.21.15.7:0/3131066638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a40cc6b0 con 0x7f56a40aed40 2024-04-25T11:21:41.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.201+0000 7f56aae00640 1 -- 172.21.15.7:0/3131066638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a4050740 con 0x7f56a40aed40 2024-04-25T11:21:41.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.202+0000 7f56a1600640 1 -- 172.21.15.7:0/3131066638 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5698018070 con 0x7f56a40aed40 2024-04-25T11:21:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.203+0000 7f56a1600640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5684074500 0x7f5684076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.204+0000 7f56a1600640 1 -- 172.21.15.7:0/3131066638 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f5698095240 con 0x7f56a40aed40 2024-04-25T11:21:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.204+0000 7f56a9000640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5684074500 0x7f5684076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.204+0000 7f56a9000640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5684074500 0x7f5684076a10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5690034000 tx=0x7f569002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.208+0000 7f56a1600640 1 -- 172.21.15.7:0/3131066638 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f569805f8b0 con 0x7f56a40aed40 2024-04-25T11:21:41.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.454+0000 7f56aae00640 1 -- 172.21.15.7:0/3131066638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56a40d13a0 con 0x7f56a40aed40 2024-04-25T11:21:41.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.456+0000 7f56a1600640 1 -- 172.21.15.7:0/3131066638 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5172 (secure 0 0 0) 0x7f569805f1b0 con 0x7f56a40aed40 2024-04-25T11:21:41.452 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:41.452 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/3503931395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3503931395},{"type":"v1","addr":"172.21.15.87:6834","nonce":3503931395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/63829697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":63829697},{"type":"v1","addr":"172.21.15.87:6835","nonce":63829697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/415877708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":415877708},{"type":"v1","addr":"172.21.15.7:6835","nonce":415877708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:21:37.365427+0000","modified":"2024-04-25T11:21:40.443636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8967},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8967":{"gid":8967,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2966579341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2966579341},{"type":"v1","addr":"172.21.15.7:6837","nonce":2966579341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8967,"qdb_cluster":[8967]},"id":10}]} 2024-04-25T11:21:41.453 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 103 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 -- 172.21.15.7:0/3131066638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5684074500 msgr2=0x7f5684076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5684074500 0x7f5684076a10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5690034000 tx=0x7f569002e000 comp rx=0 tx=0).stop 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 -- 172.21.15.7:0/3131066638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a40aed40 msgr2=0x7f56a40cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a40aed40 0x7f56a40cb650 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f569800c7f0 tx=0x7f569800ccb0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 -- 172.21.15.7:0/3131066638 shutdown_connections 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5684074500 0x7f5684076a10 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56a40b5220 0x7f56a414abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a40af420 0x7f56a4149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 --2- 172.21.15.7:0/3131066638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56a40aed40 0x7f56a40cb650 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 -- 172.21.15.7:0/3131066638 >> 172.21.15.7:0/3131066638 conn(0x7f56a40b3820 msgr2=0x7f56a40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 -- 172.21.15.7:0/3131066638 shutdown_connections 2024-04-25T11:21:41.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.459+0000 7f56aae00640 1 -- 172.21.15.7:0/3131066638 wait complete. 2024-04-25T11:21:41.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:21:41.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.780+0000 7fdf92400640 1 Processor -- start 2024-04-25T11:21:41.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.780+0000 7fdf92400640 1 -- start start 2024-04-25T11:21:41.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.783+0000 7fdf92400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdf8c0af540 0x7fdf8c09f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.783+0000 7fdf92400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf8c15bc40 0x7fdf8c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.783+0000 7fdf92400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8c15f4e0 con 0x7fdf8c15ef90 2024-04-25T11:21:41.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.783+0000 7fdf92400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fdf8c071190 con 0x7fdf8c15bc40 2024-04-25T11:21:41.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.783+0000 7fdf92400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fdf8c0b0560 con 0x7fdf8c0af540 2024-04-25T11:21:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.783+0000 7fdf8b400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdf8c15ef90 0x7fdf8c0aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47600/0 (socket says 172.21.15.7:47600) 2024-04-25T11:21:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.783+0000 7fdf8be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf8c15bc40 0x7fdf8c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.783+0000 7fdf8b400640 1 -- 172.21.15.7:0/349819415 learned_addr learned my addr 172.21.15.7:0/349819415 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.784+0000 7fdf82a00640 1 --2- 172.21.15.7:0/349819415 >> v2:172.21.15.87:3300/0 conn(0x7fdf8c0af540 0x7fdf8c09f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.784+0000 7fdf8be00640 1 -- 172.21.15.7:0/349819415 >> v2:172.21.15.87:3300/0 conn(0x7fdf8c0af540 msgr2=0x7fdf8c09f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.784+0000 7fdf8be00640 1 --2- 172.21.15.7:0/349819415 >> v2:172.21.15.87:3300/0 conn(0x7fdf8c0af540 0x7fdf8c09f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.784+0000 7fdf8be00640 1 -- 172.21.15.7:0/349819415 >> v1:172.21.15.7:6789/0 conn(0x7fdf8c15ef90 legacy=0x7fdf8c0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.784+0000 7fdf8be00640 1 -- 172.21.15.7:0/349819415 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c0b0840 con 0x7fdf8c15bc40 2024-04-25T11:21:41.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.785+0000 7fdf8be00640 1 --2- 172.21.15.7:0/349819415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8c15bc40 0x7fdf8c0cf980 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c06f2b0 tx=0x7fdf8402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735787e2a1d7ed87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.785+0000 7fdf8aa00640 1 -- 172.21.15.7:0/349819415 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84007740 con 0x7fdf8c15bc40 2024-04-25T11:21:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.786+0000 7fdf8aa00640 1 -- 172.21.15.7:0/349819415 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf840078e0 con 0x7fdf8c15bc40 2024-04-25T11:21:41.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.786+0000 7fdf8aa00640 1 -- 172.21.15.7:0/349819415 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84007cb0 con 0x7fdf8c15bc40 2024-04-25T11:21:41.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.786+0000 7fdf92400640 1 -- 172.21.15.7:0/349819415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8c15bc40 msgr2=0x7fdf8c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:41.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.786+0000 7fdf92400640 1 --2- 172.21.15.7:0/349819415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8c15bc40 0x7fdf8c0cf980 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c06f2b0 tx=0x7fdf8402a5f0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.786+0000 7fdf92400640 1 -- 172.21.15.7:0/349819415 shutdown_connections 2024-04-25T11:21:41.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.786+0000 7fdf92400640 1 --2- 172.21.15.7:0/349819415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8c15bc40 0x7fdf8c0cf980 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.786+0000 7fdf92400640 1 --2- 172.21.15.7:0/349819415 >> v2:172.21.15.87:3300/0 conn(0x7fdf8c0af540 0x7fdf8c09f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.787+0000 7fdf92400640 1 -- 172.21.15.7:0/349819415 >> 172.21.15.7:0/349819415 conn(0x7fdf8c074200 msgr2=0x7fdf8c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:41.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.787+0000 7fdf92400640 1 -- 172.21.15.7:0/349819415 shutdown_connections 2024-04-25T11:21:41.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.787+0000 7fdf92400640 1 -- 172.21.15.7:0/349819415 wait complete. 2024-04-25T11:21:41.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.789+0000 7fdf92400640 1 Processor -- start 2024-04-25T11:21:41.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.789+0000 7fdf92400640 1 -- start start 2024-04-25T11:21:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf92400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf8c15bc40 0x7fdf8c160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf92400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf8c15ef90 0x7fdf8c15f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf92400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8c15fbf0 0x7fdf8c160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf92400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c0b1320 con 0x7fdf8c15fbf0 2024-04-25T11:21:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf92400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c0b0e00 con 0x7fdf8c15ef90 2024-04-25T11:21:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf92400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8c0afb80 con 0x7fdf8c15bc40 2024-04-25T11:21:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf8be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8c15fbf0 0x7fdf8c160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf8be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8c15fbf0 0x7fdf8c160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46540/0 (socket says 172.21.15.7:46540) 2024-04-25T11:21:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf8be00640 1 -- 172.21.15.7:0/531547534 learned_addr learned my addr 172.21.15.7:0/531547534 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf82a00640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf8c15ef90 0x7fdf8c15f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf8b400640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf8c15bc40 0x7fdf8c160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf8be00640 1 -- 172.21.15.7:0/531547534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf8c15bc40 msgr2=0x7fdf8c160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf8be00640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf8c15bc40 0x7fdf8c160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf8be00640 1 -- 172.21.15.7:0/531547534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf8c15ef90 msgr2=0x7fdf8c15f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf8be00640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf8c15ef90 0x7fdf8c15f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.791+0000 7fdf8be00640 1 -- 172.21.15.7:0/531547534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c161010 con 0x7fdf8c15fbf0 2024-04-25T11:21:41.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.792+0000 7fdf8be00640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8c15fbf0 0x7fdf8c160000 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009e50 tx=0x7fdf84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:41.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.792+0000 7fdf83e00640 1 -- 172.21.15.7:0/531547534 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8403f070 con 0x7fdf8c15fbf0 2024-04-25T11:21:41.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.792+0000 7fdf83e00640 1 -- 172.21.15.7:0/531547534 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf84031c90 con 0x7fdf8c15fbf0 2024-04-25T11:21:41.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.792+0000 7fdf83e00640 1 -- 172.21.15.7:0/531547534 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf84043650 con 0x7fdf8c15fbf0 2024-04-25T11:21:41.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.792+0000 7fdf92400640 1 -- 172.21.15.7:0/531547534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8c0b5180 con 0x7fdf8c15fbf0 2024-04-25T11:21:41.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.792+0000 7fdf92400640 1 -- 172.21.15.7:0/531547534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf8c0b5700 con 0x7fdf8c15fbf0 2024-04-25T11:21:41.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.794+0000 7fdf92400640 1 -- 172.21.15.7:0/531547534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf8c14bb10 con 0x7fdf8c15fbf0 2024-04-25T11:21:41.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.795+0000 7fdf83e00640 1 -- 172.21.15.7:0/531547534 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdf8403b410 con 0x7fdf8c15fbf0 2024-04-25T11:21:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.796+0000 7fdf83e00640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf680744b0 0x7fdf680769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:41.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.796+0000 7fdf83e00640 1 -- 172.21.15.7:0/531547534 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7fdf840b6550 con 0x7fdf8c15fbf0 2024-04-25T11:21:41.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.796+0000 7fdf8b400640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf680744b0 0x7fdf680769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:41.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.797+0000 7fdf8b400640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf680744b0 0x7fdf680769c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c000f60 tx=0x7fdf7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:41.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:41.802+0000 7fdf83e00640 1 -- 172.21.15.7:0/531547534 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdf8c161010 con 0x7fdf8c15fbf0 2024-04-25T11:21:42.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.008+0000 7fdf92400640 1 -- 172.21.15.7:0/531547534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf8c06d6d0 con 0x7fdf8c15fbf0 2024-04-25T11:21:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.010+0000 7fdf83e00640 1 -- 172.21.15.7:0/531547534 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16796 (secure 0 0 0) 0x7fdf8c06d6d0 con 0x7fdf8c15fbf0 2024-04-25T11:21:42.007 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:42.007 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":142,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:21:38.842722+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:21:35.562066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T11:21:36.342958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:21:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.013+0000 7fdf92400640 1 -- 172.21.15.7:0/531547534 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf680744b0 msgr2=0x7fdf680769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.013+0000 7fdf92400640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf680744b0 0x7fdf680769c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c000f60 tx=0x7fdf7c009040 comp rx=0 tx=0).stop 2024-04-25T11:21:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.013+0000 7fdf92400640 1 -- 172.21.15.7:0/531547534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8c15fbf0 msgr2=0x7fdf8c160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.013+0000 7fdf92400640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8c15fbf0 0x7fdf8c160000 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009e50 tx=0x7fdf84004a40 comp rx=0 tx=0).stop 2024-04-25T11:21:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.014+0000 7fdf92400640 1 -- 172.21.15.7:0/531547534 shutdown_connections 2024-04-25T11:21:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.014+0000 7fdf92400640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdf680744b0 0x7fdf680769c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.014+0000 7fdf92400640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf8c15fbf0 0x7fdf8c160000 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.014+0000 7fdf92400640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf8c15ef90 0x7fdf8c15f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.014+0000 7fdf92400640 1 --2- 172.21.15.7:0/531547534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdf8c15bc40 0x7fdf8c160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.014+0000 7fdf92400640 1 -- 172.21.15.7:0/531547534 >> 172.21.15.7:0/531547534 conn(0x7fdf8c074200 msgr2=0x7fdf8c0686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.014+0000 7fdf92400640 1 -- 172.21.15.7:0/531547534 shutdown_connections 2024-04-25T11:21:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.014+0000 7fdf92400640 1 -- 172.21.15.7:0/531547534 wait complete. 2024-04-25T11:21:42.037 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:21:42.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:21:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.325+0000 7f2bdd200640 1 Processor -- start 2024-04-25T11:21:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.325+0000 7f2bdd200640 1 -- start start 2024-04-25T11:21:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.328+0000 7f2bdd200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd80b0290 0x7f2bd80b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.328+0000 7f2bdd200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bd80b4980 con 0x7f2bd806f300 2024-04-25T11:21:42.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.328+0000 7f2bdd200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bd80b4c60 con 0x7f2bd806dc90 2024-04-25T11:21:42.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.328+0000 7f2bdd200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd80b4f40 con 0x7f2bd80b0290 2024-04-25T11:21:42.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.328+0000 7f2bd6800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2bd806f300 0x7f2bd80b5f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47614/0 (socket says 172.21.15.7:47614) 2024-04-25T11:21:42.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.328+0000 7f2bd6800640 1 -- 172.21.15.7:0/1485971267 learned_addr learned my addr 172.21.15.7:0/1485971267 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:42.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.329+0000 7f2bcd400640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd80b0290 0x7f2bd80b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.329+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494316076 0 0) 0x7f2bd80b4980 con 0x7f2bd806f300 2024-04-25T11:21:42.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.329+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bbc003520 con 0x7f2bd806f300 2024-04-25T11:21:42.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.329+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3377037428 0 0) 0x7f2bbc003520 con 0x7f2bd806f300 2024-04-25T11:21:42.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.329+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd80b0290 msgr2=0x7f2bd80b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:42.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.329+0000 7f2bd5400640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd80b0290 0x7f2bd80b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.329+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 >> v1:172.21.15.87:6789/0 conn(0x7f2bd806dc90 legacy=0x7f2bd80afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.329+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd80b5220 con 0x7f2bd806f300 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.329+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2bd00019f0 con 0x7f2bd806f300 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.329+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 >> v1:172.21.15.7:6789/0 conn(0x7f2bd806f300 legacy=0x7f2bd80b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5400640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 0x7f2bbc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5400640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bbc004800 0x7f2bbc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5400640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bbc005150 0x7f2bbc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80b4f40 con 0x7f2bbc0039e0 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2bbc00e110 con 0x7f2bbc005150 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2bbc00e310 con 0x7f2bbc004800 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5e00640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 0x7f2bbc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bcd400640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bbc004800 0x7f2bbc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd6800640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bbc005150 0x7f2bbc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5e00640 1 -- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bbc004800 msgr2=0x7f2bbc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5e00640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bbc004800 0x7f2bbc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5e00640 1 -- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bbc005150 msgr2=0x7f2bbc00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5e00640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bbc005150 0x7f2bbc00dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bd5e00640 1 -- 172.21.15.7:0/1485971267 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bbc003520 con 0x7f2bbc0039e0 2024-04-25T11:21:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.330+0000 7f2bcd400640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bbc004800 0x7f2bbc004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.331+0000 7f2bd5e00640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 0x7f2bbc003e50 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8001720 tx=0x7f2bbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdf52c81feae6ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.332+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8001370 con 0x7f2bbc0039e0 2024-04-25T11:21:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.332+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc80052d0 con 0x7f2bbc0039e0 2024-04-25T11:21:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.332+0000 7f2bd5400640 1 -- 172.21.15.7:0/1485971267 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc802abc0 con 0x7f2bbc0039e0 2024-04-25T11:21:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.333+0000 7f2bdd200640 1 -- 172.21.15.7:0/1485971267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 msgr2=0x7f2bbc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.333+0000 7f2bdd200640 1 --2- 172.21.15.7:0/1485971267 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 0x7f2bbc003e50 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8001720 tx=0x7f2bbc005830 comp rx=0 tx=0).stop 2024-04-25T11:21:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.333+0000 7f2bd6800640 1 -- 172.21.15.7:0/1485971267 reap_dead start 2024-04-25T11:21:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.333+0000 7f2bdd200640 1 -- 172.21.15.7:0/1485971267 shutdown_connections 2024-04-25T11:21:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.333+0000 7f2bdd200640 1 -- 172.21.15.7:0/1485971267 >> 172.21.15.7:0/1485971267 conn(0x7f2bd8074200 msgr2=0x7f2bd806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.333+0000 7f2bdd200640 1 -- 172.21.15.7:0/1485971267 shutdown_connections 2024-04-25T11:21:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.333+0000 7f2bdd200640 1 -- 172.21.15.7:0/1485971267 wait complete. 2024-04-25T11:21:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.335+0000 7f2bdd200640 1 Processor -- start 2024-04-25T11:21:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.336+0000 7f2bdd200640 1 -- start start 2024-04-25T11:21:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.337+0000 7f2bdd200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 0x7f2bd815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.337+0000 7f2bdd200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bd8144530 0x7f2bd80b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.337+0000 7f2bdd200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd80b6040 0x7f2bd8157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.337+0000 7f2bdd200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd80ce920 con 0x7f2bbc0039e0 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.337+0000 7f2bdd200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd815d220 con 0x7f2bd8144530 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.337+0000 7f2bdd200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd80ceaa0 con 0x7f2bd80b6040 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.337+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 0x7f2bd815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.338+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 0x7f2bd815a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46552/0 (socket says 172.21.15.7:46552) 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.338+0000 7f2bd5e00640 1 -- 172.21.15.7:0/3570058867 learned_addr learned my addr 172.21.15.7:0/3570058867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.338+0000 7f2bcd400640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bd8144530 0x7f2bd80b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.338+0000 7f2bd6800640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd80b6040 0x7f2bd8157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.338+0000 7f2bd5e00640 1 -- 172.21.15.7:0/3570058867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd80b6040 msgr2=0x7f2bd8157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.338+0000 7f2bd5e00640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd80b6040 0x7f2bd8157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.338+0000 7f2bd5e00640 1 -- 172.21.15.7:0/3570058867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bd8144530 msgr2=0x7f2bd80b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.338+0000 7f2bd5e00640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bd8144530 0x7f2bd80b5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.338+0000 7f2bd5e00640 1 -- 172.21.15.7:0/3570058867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd815cdd0 con 0x7f2bbc0039e0 2024-04-25T11:21:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.338+0000 7f2bd5e00640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 0x7f2bd815a8a0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8001840 tx=0x7f2bc8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.339+0000 7f2bcea00640 1 -- 172.21.15.7:0/3570058867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8037040 con 0x7f2bbc0039e0 2024-04-25T11:21:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.339+0000 7f2bcea00640 1 -- 172.21.15.7:0/3570058867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bc8042070 con 0x7f2bbc0039e0 2024-04-25T11:21:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.339+0000 7f2bcea00640 1 -- 172.21.15.7:0/3570058867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8007a30 con 0x7f2bbc0039e0 2024-04-25T11:21:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.339+0000 7f2bdd200640 1 -- 172.21.15.7:0/3570058867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd815c810 con 0x7f2bbc0039e0 2024-04-25T11:21:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.339+0000 7f2bdd200640 1 -- 172.21.15.7:0/3570058867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81573c0 con 0x7f2bbc0039e0 2024-04-25T11:21:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.340+0000 7f2bdd200640 1 -- 172.21.15.7:0/3570058867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd806a100 con 0x7f2bbc0039e0 2024-04-25T11:21:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.341+0000 7f2bcea00640 1 -- 172.21.15.7:0/3570058867 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2bc80052d0 con 0x7f2bbc0039e0 2024-04-25T11:21:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.342+0000 7f2bcea00640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bb40744b0 0x7f2bb40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.343+0000 7f2bcea00640 1 -- 172.21.15.7:0/3570058867 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f2bc80b5900 con 0x7f2bbc0039e0 2024-04-25T11:21:42.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.343+0000 7f2bcd400640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bb40744b0 0x7f2bb40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.343+0000 7f2bcd400640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bb40744b0 0x7f2bb40769c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009160 tx=0x7f2bc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:42.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.347+0000 7f2bcea00640 1 -- 172.21.15.7:0/3570058867 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2bc807fe60 con 0x7f2bbc0039e0 2024-04-25T11:21:42.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.549+0000 7f2bdd200640 1 -- 172.21.15.7:0/3570058867 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bd8070fb0 con 0x7f2bb40744b0 2024-04-25T11:21:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.570+0000 7f2bcea00640 1 -- 172.21.15.7:0/3570058867 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252466 (secure 0 0 0) 0x7f2bd8070fb0 con 0x7f2bb40744b0 2024-04-25T11:21:42.569 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:42.571 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:21:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 -- 172.21.15.7:0/3570058867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bb40744b0 msgr2=0x7f2bb40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bb40744b0 0x7f2bb40769c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009160 tx=0x7f2bc0008040 comp rx=0 tx=0).stop 2024-04-25T11:21:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 -- 172.21.15.7:0/3570058867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 msgr2=0x7f2bd815a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 0x7f2bd815a8a0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8001840 tx=0x7f2bc8001120 comp rx=0 tx=0).stop 2024-04-25T11:21:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 -- 172.21.15.7:0/3570058867 shutdown_connections 2024-04-25T11:21:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bb40744b0 0x7f2bb40769c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2bd80b6040 0x7f2bd8157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2bd8144530 0x7f2bd80b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 --2- 172.21.15.7:0/3570058867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bbc0039e0 0x7f2bd815a8a0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 -- 172.21.15.7:0/3570058867 >> 172.21.15.7:0/3570058867 conn(0x7f2bd8074200 msgr2=0x7f2bd80af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 -- 172.21.15.7:0/3570058867 shutdown_connections 2024-04-25T11:21:42.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.576+0000 7f2bdd200640 1 -- 172.21.15.7:0/3570058867 wait complete. 2024-04-25T11:21:42.599 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-04-25T11:21:42.052623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281132,"kb_used_data":15024,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754693588,"statfs":{"total":773094113280,"available":772806234112,"internally_reserved":0,"allocated":15384576,"data_stored":7724092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002248"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880017+0000","last_change":"2024-04-25T11:21:37.862629+0000","last_active":"2024-04-25T11:21:38.880017+0000","last_peered":"2024-04-25T11:21:38.880017+0000","last_clean":"2024-04-25T11:21:38.880017+0000","last_became_active":"2024-04-25T11:21:37.862262+0000","last_became_peered":"2024-04-25T11:21:37.862262+0000","last_unstale":"2024-04-25T11:21:38.880017+0000","last_undegraded":"2024-04-25T11:21:38.880017+0000","last_fullsized":"2024-04-25T11:21:38.880017+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:33.889420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859566+0000","last_change":"2024-04-25T11:21:37.862282+0000","last_active":"2024-04-25T11:21:38.859566+0000","last_peered":"2024-04-25T11:21:38.859566+0000","last_clean":"2024-04-25T11:21:38.859566+0000","last_became_active":"2024-04-25T11:21:37.861921+0000","last_became_peered":"2024-04-25T11:21:37.861921+0000","last_unstale":"2024-04-25T11:21:38.859566+0000","last_undegraded":"2024-04-25T11:21:38.859566+0000","last_fullsized":"2024-04-25T11:21:38.859566+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:05.598346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859859+0000","last_change":"2024-04-25T11:21:37.859543+0000","last_active":"2024-04-25T11:21:38.859859+0000","last_peered":"2024-04-25T11:21:38.859859+0000","last_clean":"2024-04-25T11:21:38.859859+0000","last_became_active":"2024-04-25T11:21:37.859191+0000","last_became_peered":"2024-04-25T11:21:37.859191+0000","last_unstale":"2024-04-25T11:21:38.859859+0000","last_undegraded":"2024-04-25T11:21:38.859859+0000","last_fullsized":"2024-04-25T11:21:38.859859+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:59.739927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863049+0000","last_change":"2024-04-25T11:21:37.861991+0000","last_active":"2024-04-25T11:21:38.863049+0000","last_peered":"2024-04-25T11:21:38.863049+0000","last_clean":"2024-04-25T11:21:38.863049+0000","last_became_active":"2024-04-25T11:21:37.861655+0000","last_became_peered":"2024-04-25T11:21:37.861655+0000","last_unstale":"2024-04-25T11:21:38.863049+0000","last_undegraded":"2024-04-25T11:21:38.863049+0000","last_fullsized":"2024-04-25T11:21:38.863049+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:56.149851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511528+0000","last_change":"2024-04-25T11:21:37.863943+0000","last_active":"2024-04-25T11:21:39.511528+0000","last_peered":"2024-04-25T11:21:39.511528+0000","last_clean":"2024-04-25T11:21:39.511528+0000","last_became_active":"2024-04-25T11:21:37.853267+0000","last_became_peered":"2024-04-25T11:21:37.853267+0000","last_unstale":"2024-04-25T11:21:39.511528+0000","last_undegraded":"2024-04-25T11:21:39.511528+0000","last_fullsized":"2024-04-25T11:21:39.511528+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:22.520785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511976+0000","last_change":"2024-04-25T11:21:37.850523+0000","last_active":"2024-04-25T11:21:39.511976+0000","last_peered":"2024-04-25T11:21:39.511976+0000","last_clean":"2024-04-25T11:21:39.511976+0000","last_became_active":"2024-04-25T11:21:37.850211+0000","last_became_peered":"2024-04-25T11:21:37.850211+0000","last_unstale":"2024-04-25T11:21:39.511976+0000","last_undegraded":"2024-04-25T11:21:39.511976+0000","last_fullsized":"2024-04-25T11:21:39.511976+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:45.489827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879591+0000","last_change":"2024-04-25T11:21:37.849431+0000","last_active":"2024-04-25T11:21:38.879591+0000","last_peered":"2024-04-25T11:21:38.879591+0000","last_clean":"2024-04-25T11:21:38.879591+0000","last_became_active":"2024-04-25T11:21:37.849011+0000","last_became_peered":"2024-04-25T11:21:37.849011+0000","last_unstale":"2024-04-25T11:21:38.879591+0000","last_undegraded":"2024-04-25T11:21:38.879591+0000","last_fullsized":"2024-04-25T11:21:38.879591+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:03.475738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510461+0000","last_change":"2024-04-25T11:21:37.864988+0000","last_active":"2024-04-25T11:21:39.510461+0000","last_peered":"2024-04-25T11:21:39.510461+0000","last_clean":"2024-04-25T11:21:39.510461+0000","last_became_active":"2024-04-25T11:21:37.864317+0000","last_became_peered":"2024-04-25T11:21:37.864317+0000","last_unstale":"2024-04-25T11:21:39.510461+0000","last_undegraded":"2024-04-25T11:21:39.510461+0000","last_fullsized":"2024-04-25T11:21:39.510461+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:36.844379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511921+0000","last_change":"2024-04-25T11:21:37.872632+0000","last_active":"2024-04-25T11:21:39.511921+0000","last_peered":"2024-04-25T11:21:39.511921+0000","last_clean":"2024-04-25T11:21:39.511921+0000","last_became_active":"2024-04-25T11:21:37.872308+0000","last_became_peered":"2024-04-25T11:21:37.872308+0000","last_unstale":"2024-04-25T11:21:39.511921+0000","last_undegraded":"2024-04-25T11:21:39.511921+0000","last_fullsized":"2024-04-25T11:21:39.511921+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:23.638662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512050+0000","last_change":"2024-04-25T11:21:37.871717+0000","last_active":"2024-04-25T11:21:39.512050+0000","last_peered":"2024-04-25T11:21:39.512050+0000","last_clean":"2024-04-25T11:21:39.512050+0000","last_became_active":"2024-04-25T11:21:37.871268+0000","last_became_peered":"2024-04-25T11:21:37.871268+0000","last_unstale":"2024-04-25T11:21:39.512050+0000","last_undegraded":"2024-04-25T11:21:39.512050+0000","last_fullsized":"2024-04-25T11:21:39.512050+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:29.164577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.864299+0000","last_change":"2024-04-25T11:21:37.860096+0000","last_active":"2024-04-25T11:21:37.864299+0000","last_peered":"2024-04-25T11:21:37.864299+0000","last_clean":"2024-04-25T11:21:37.864299+0000","last_became_active":"2024-04-25T11:21:37.859399+0000","last_became_peered":"2024-04-25T11:21:37.859399+0000","last_unstale":"2024-04-25T11:21:37.864299+0000","last_undegraded":"2024-04-25T11:21:37.864299+0000","last_fullsized":"2024-04-25T11:21:37.864299+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:20.179524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863582+0000","last_change":"2024-04-25T11:21:37.875500+0000","last_active":"2024-04-25T11:21:38.863582+0000","last_peered":"2024-04-25T11:21:38.863582+0000","last_clean":"2024-04-25T11:21:38.863582+0000","last_became_active":"2024-04-25T11:21:37.875146+0000","last_became_peered":"2024-04-25T11:21:37.875146+0000","last_unstale":"2024-04-25T11:21:38.863582+0000","last_undegraded":"2024-04-25T11:21:38.863582+0000","last_fullsized":"2024-04-25T11:21:38.863582+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:56.956131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511332+0000","last_change":"2024-04-25T11:21:37.865603+0000","last_active":"2024-04-25T11:21:39.511332+0000","last_peered":"2024-04-25T11:21:39.511332+0000","last_clean":"2024-04-25T11:21:39.511332+0000","last_became_active":"2024-04-25T11:21:37.854582+0000","last_became_peered":"2024-04-25T11:21:37.854582+0000","last_unstale":"2024-04-25T11:21:39.511332+0000","last_undegraded":"2024-04-25T11:21:39.511332+0000","last_fullsized":"2024-04-25T11:21:39.511332+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:18.497127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863054+0000","last_change":"2024-04-25T11:21:37.876260+0000","last_active":"2024-04-25T11:21:38.863054+0000","last_peered":"2024-04-25T11:21:38.863054+0000","last_clean":"2024-04-25T11:21:38.863054+0000","last_became_active":"2024-04-25T11:21:37.875537+0000","last_became_peered":"2024-04-25T11:21:37.875537+0000","last_unstale":"2024-04-25T11:21:38.863054+0000","last_undegraded":"2024-04-25T11:21:38.863054+0000","last_fullsized":"2024-04-25T11:21:38.863054+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:33.202111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859901+0000","last_change":"2024-04-25T11:21:37.857192+0000","last_active":"2024-04-25T11:21:38.859901+0000","last_peered":"2024-04-25T11:21:38.859901+0000","last_clean":"2024-04-25T11:21:38.859901+0000","last_became_active":"2024-04-25T11:21:37.856254+0000","last_became_peered":"2024-04-25T11:21:37.856254+0000","last_unstale":"2024-04-25T11:21:38.859901+0000","last_undegraded":"2024-04-25T11:21:38.859901+0000","last_fullsized":"2024-04-25T11:21:38.859901+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:36.700564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511344+0000","last_change":"2024-04-25T11:21:37.873642+0000","last_active":"2024-04-25T11:21:39.511344+0000","last_peered":"2024-04-25T11:21:39.511344+0000","last_clean":"2024-04-25T11:21:39.511344+0000","last_became_active":"2024-04-25T11:21:37.873276+0000","last_became_peered":"2024-04-25T11:21:37.873276+0000","last_unstale":"2024-04-25T11:21:39.511344+0000","last_undegraded":"2024-04-25T11:21:39.511344+0000","last_fullsized":"2024-04-25T11:21:39.511344+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:25.417788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.864172+0000","last_change":"2024-04-25T11:21:37.860637+0000","last_active":"2024-04-25T11:21:38.864172+0000","last_peered":"2024-04-25T11:21:38.864172+0000","last_clean":"2024-04-25T11:21:38.864172+0000","last_became_active":"2024-04-25T11:21:37.860266+0000","last_became_peered":"2024-04-25T11:21:37.860266+0000","last_unstale":"2024-04-25T11:21:38.864172+0000","last_undegraded":"2024-04-25T11:21:38.864172+0000","last_fullsized":"2024-04-25T11:21:38.864172+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:06.290875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.861613+0000","last_change":"2024-04-25T11:21:37.853524+0000","last_active":"2024-04-25T11:21:37.861613+0000","last_peered":"2024-04-25T11:21:37.861613+0000","last_clean":"2024-04-25T11:21:37.861613+0000","last_became_active":"2024-04-25T11:21:37.853100+0000","last_became_peered":"2024-04-25T11:21:37.853100+0000","last_unstale":"2024-04-25T11:21:37.861613+0000","last_undegraded":"2024-04-25T11:21:37.861613+0000","last_fullsized":"2024-04-25T11:21:37.861613+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:22.202712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.867307+0000","last_change":"2024-04-25T11:21:37.868036+0000","last_active":"2024-04-25T11:21:38.867307+0000","last_peered":"2024-04-25T11:21:38.867307+0000","last_clean":"2024-04-25T11:21:38.867307+0000","last_became_active":"2024-04-25T11:21:37.867747+0000","last_became_peered":"2024-04-25T11:21:37.867747+0000","last_unstale":"2024-04-25T11:21:38.867307+0000","last_undegraded":"2024-04-25T11:21:38.867307+0000","last_fullsized":"2024-04-25T11:21:38.867307+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:15.858707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510070+0000","last_change":"2024-04-25T11:21:37.856361+0000","last_active":"2024-04-25T11:21:39.510070+0000","last_peered":"2024-04-25T11:21:39.510070+0000","last_clean":"2024-04-25T11:21:39.510070+0000","last_became_active":"2024-04-25T11:21:37.855956+0000","last_became_peered":"2024-04-25T11:21:37.855956+0000","last_unstale":"2024-04-25T11:21:39.510070+0000","last_undegraded":"2024-04-25T11:21:39.510070+0000","last_fullsized":"2024-04-25T11:21:39.510070+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:22.520785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511637+0000","last_change":"2024-04-25T11:21:37.868213+0000","last_active":"2024-04-25T11:21:39.511637+0000","last_peered":"2024-04-25T11:21:39.511637+0000","last_clean":"2024-04-25T11:21:39.511637+0000","last_became_active":"2024-04-25T11:21:37.867467+0000","last_became_peered":"2024-04-25T11:21:37.867467+0000","last_unstale":"2024-04-25T11:21:39.511637+0000","last_undegraded":"2024-04-25T11:21:39.511637+0000","last_fullsized":"2024-04-25T11:21:39.511637+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:16.677903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879329+0000","last_change":"2024-04-25T11:21:37.848122+0000","last_active":"2024-04-25T11:21:38.879329+0000","last_peered":"2024-04-25T11:21:38.879329+0000","last_clean":"2024-04-25T11:21:38.879329+0000","last_became_active":"2024-04-25T11:21:37.847741+0000","last_became_peered":"2024-04-25T11:21:37.847741+0000","last_unstale":"2024-04-25T11:21:38.879329+0000","last_undegraded":"2024-04-25T11:21:38.879329+0000","last_fullsized":"2024-04-25T11:21:38.879329+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:16.279883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510442+0000","last_change":"2024-04-25T11:21:37.866220+0000","last_active":"2024-04-25T11:21:39.510442+0000","last_peered":"2024-04-25T11:21:39.510442+0000","last_clean":"2024-04-25T11:21:39.510442+0000","last_became_active":"2024-04-25T11:21:37.865840+0000","last_became_peered":"2024-04-25T11:21:37.865840+0000","last_unstale":"2024-04-25T11:21:39.510442+0000","last_undegraded":"2024-04-25T11:21:39.510442+0000","last_fullsized":"2024-04-25T11:21:39.510442+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:39.233435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.870701+0000","last_change":"2024-04-25T11:21:37.868982+0000","last_active":"2024-04-25T11:21:37.870701+0000","last_peered":"2024-04-25T11:21:37.870701+0000","last_clean":"2024-04-25T11:21:37.870701+0000","last_became_active":"2024-04-25T11:21:37.868100+0000","last_became_peered":"2024-04-25T11:21:37.868100+0000","last_unstale":"2024-04-25T11:21:37.870701+0000","last_undegraded":"2024-04-25T11:21:37.870701+0000","last_fullsized":"2024-04-25T11:21:37.870701+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:49.628923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880233+0000","last_change":"2024-04-25T11:21:37.850688+0000","last_active":"2024-04-25T11:21:38.880233+0000","last_peered":"2024-04-25T11:21:38.880233+0000","last_clean":"2024-04-25T11:21:38.880233+0000","last_became_active":"2024-04-25T11:21:37.850350+0000","last_became_peered":"2024-04-25T11:21:37.850350+0000","last_unstale":"2024-04-25T11:21:38.880233+0000","last_undegraded":"2024-04-25T11:21:38.880233+0000","last_fullsized":"2024-04-25T11:21:38.880233+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:06.615128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880169+0000","last_change":"2024-04-25T11:21:37.853356+0000","last_active":"2024-04-25T11:21:38.880169+0000","last_peered":"2024-04-25T11:21:38.880169+0000","last_clean":"2024-04-25T11:21:38.880169+0000","last_became_active":"2024-04-25T11:21:37.852930+0000","last_became_peered":"2024-04-25T11:21:37.852930+0000","last_unstale":"2024-04-25T11:21:38.880169+0000","last_undegraded":"2024-04-25T11:21:38.880169+0000","last_fullsized":"2024-04-25T11:21:38.880169+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:34.967219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511451+0000","last_change":"2024-04-25T11:21:37.873935+0000","last_active":"2024-04-25T11:21:39.511451+0000","last_peered":"2024-04-25T11:21:39.511451+0000","last_clean":"2024-04-25T11:21:39.511451+0000","last_became_active":"2024-04-25T11:21:37.867492+0000","last_became_peered":"2024-04-25T11:21:37.867492+0000","last_unstale":"2024-04-25T11:21:39.511451+0000","last_undegraded":"2024-04-25T11:21:39.511451+0000","last_fullsized":"2024-04-25T11:21:39.511451+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:42.548022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.864241+0000","last_change":"2024-04-25T11:21:37.860373+0000","last_active":"2024-04-25T11:21:37.864241+0000","last_peered":"2024-04-25T11:21:37.864241+0000","last_clean":"2024-04-25T11:21:37.864241+0000","last_became_active":"2024-04-25T11:21:37.859465+0000","last_became_peered":"2024-04-25T11:21:37.859465+0000","last_unstale":"2024-04-25T11:21:37.864241+0000","last_undegraded":"2024-04-25T11:21:37.864241+0000","last_fullsized":"2024-04-25T11:21:37.864241+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:41.955144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511619+0000","last_change":"2024-04-25T11:21:37.875010+0000","last_active":"2024-04-25T11:21:39.511619+0000","last_peered":"2024-04-25T11:21:39.511619+0000","last_clean":"2024-04-25T11:21:39.511619+0000","last_became_active":"2024-04-25T11:21:37.874146+0000","last_became_peered":"2024-04-25T11:21:37.874146+0000","last_unstale":"2024-04-25T11:21:39.511619+0000","last_undegraded":"2024-04-25T11:21:39.511619+0000","last_fullsized":"2024-04-25T11:21:39.511619+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:56.149851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.861632+0000","last_change":"2024-04-25T11:21:37.856141+0000","last_active":"2024-04-25T11:21:37.861632+0000","last_peered":"2024-04-25T11:21:37.861632+0000","last_clean":"2024-04-25T11:21:37.861632+0000","last_became_active":"2024-04-25T11:21:37.855478+0000","last_became_peered":"2024-04-25T11:21:37.855478+0000","last_unstale":"2024-04-25T11:21:37.861632+0000","last_undegraded":"2024-04-25T11:21:37.861632+0000","last_fullsized":"2024-04-25T11:21:37.861632+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:15.784485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511224+0000","last_change":"2024-04-25T11:21:37.872419+0000","last_active":"2024-04-25T11:21:39.511224+0000","last_peered":"2024-04-25T11:21:39.511224+0000","last_clean":"2024-04-25T11:21:39.511224+0000","last_became_active":"2024-04-25T11:21:37.872066+0000","last_became_peered":"2024-04-25T11:21:37.872066+0000","last_unstale":"2024-04-25T11:21:39.511224+0000","last_undegraded":"2024-04-25T11:21:39.511224+0000","last_fullsized":"2024-04-25T11:21:39.511224+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:15.858707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510543+0000","last_change":"2024-04-25T11:21:37.864666+0000","last_active":"2024-04-25T11:21:39.510543+0000","last_peered":"2024-04-25T11:21:39.510543+0000","last_clean":"2024-04-25T11:21:39.510543+0000","last_became_active":"2024-04-25T11:21:37.864343+0000","last_became_peered":"2024-04-25T11:21:37.864343+0000","last_unstale":"2024-04-25T11:21:39.510543+0000","last_undegraded":"2024-04-25T11:21:39.510543+0000","last_fullsized":"2024-04-25T11:21:39.510543+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:29.425446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859938+0000","last_change":"2024-04-25T11:21:37.857857+0000","last_active":"2024-04-25T11:21:38.859938+0000","last_peered":"2024-04-25T11:21:38.859938+0000","last_clean":"2024-04-25T11:21:38.859938+0000","last_became_active":"2024-04-25T11:21:37.855522+0000","last_became_peered":"2024-04-25T11:21:37.855522+0000","last_unstale":"2024-04-25T11:21:38.859938+0000","last_undegraded":"2024-04-25T11:21:38.859938+0000","last_fullsized":"2024-04-25T11:21:38.859938+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:57.353175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"141'4","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512071+0000","last_change":"2024-04-25T11:21:36.836219+0000","last_active":"2024-04-25T11:21:39.512071+0000","last_peered":"2024-04-25T11:21:39.512071+0000","last_clean":"2024-04-25T11:21:39.512071+0000","last_became_active":"2024-04-25T11:21:36.835875+0000","last_became_peered":"2024-04-25T11:21:36.835875+0000","last_unstale":"2024-04-25T11:21:39.512071+0000","last_undegraded":"2024-04-25T11:21:39.512071+0000","last_fullsized":"2024-04-25T11:21:39.512071+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:58.378850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860063+0000","last_change":"2024-04-25T11:21:37.857381+0000","last_active":"2024-04-25T11:21:38.860063+0000","last_peered":"2024-04-25T11:21:38.860063+0000","last_clean":"2024-04-25T11:21:38.860063+0000","last_became_active":"2024-04-25T11:21:37.856799+0000","last_became_peered":"2024-04-25T11:21:37.856799+0000","last_unstale":"2024-04-25T11:21:38.860063+0000","last_undegraded":"2024-04-25T11:21:38.860063+0000","last_fullsized":"2024-04-25T11:21:38.860063+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:31.934675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866436+0000","last_change":"2024-04-25T11:21:36.831101+0000","last_active":"2024-04-25T11:21:38.866436+0000","last_peered":"2024-04-25T11:21:38.866436+0000","last_clean":"2024-04-25T11:21:38.866436+0000","last_became_active":"2024-04-25T11:21:36.830755+0000","last_became_peered":"2024-04-25T11:21:36.830755+0000","last_unstale":"2024-04-25T11:21:38.866436+0000","last_undegraded":"2024-04-25T11:21:38.866436+0000","last_fullsized":"2024-04-25T11:21:38.866436+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:58.378850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511720+0000","last_change":"2024-04-25T11:21:37.866862+0000","last_active":"2024-04-25T11:21:39.511720+0000","last_peered":"2024-04-25T11:21:39.511720+0000","last_clean":"2024-04-25T11:21:39.511720+0000","last_became_active":"2024-04-25T11:21:37.865283+0000","last_became_peered":"2024-04-25T11:21:37.865283+0000","last_unstale":"2024-04-25T11:21:39.511720+0000","last_undegraded":"2024-04-25T11:21:39.511720+0000","last_fullsized":"2024-04-25T11:21:39.511720+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:33.202111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.867405+0000","last_change":"2024-04-25T11:21:36.831197+0000","last_active":"2024-04-25T11:21:38.867405+0000","last_peered":"2024-04-25T11:21:38.867405+0000","last_clean":"2024-04-25T11:21:38.867405+0000","last_became_active":"2024-04-25T11:21:36.830442+0000","last_became_peered":"2024-04-25T11:21:36.830442+0000","last_unstale":"2024-04-25T11:21:38.867405+0000","last_undegraded":"2024-04-25T11:21:38.867405+0000","last_fullsized":"2024-04-25T11:21:38.867405+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:00.825842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866674+0000","last_change":"2024-04-25T11:21:37.860630+0000","last_active":"2024-04-25T11:21:38.866674+0000","last_peered":"2024-04-25T11:21:38.866674+0000","last_clean":"2024-04-25T11:21:38.866674+0000","last_became_active":"2024-04-25T11:21:37.859711+0000","last_became_peered":"2024-04-25T11:21:37.859711+0000","last_unstale":"2024-04-25T11:21:38.866674+0000","last_undegraded":"2024-04-25T11:21:38.866674+0000","last_fullsized":"2024-04-25T11:21:38.866674+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:20.253487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860063+0000","last_change":"2024-04-25T11:21:36.837424+0000","last_active":"2024-04-25T11:21:38.860063+0000","last_peered":"2024-04-25T11:21:38.860063+0000","last_clean":"2024-04-25T11:21:38.860063+0000","last_became_active":"2024-04-25T11:21:36.836920+0000","last_became_peered":"2024-04-25T11:21:36.836920+0000","last_unstale":"2024-04-25T11:21:38.860063+0000","last_undegraded":"2024-04-25T11:21:38.860063+0000","last_fullsized":"2024-04-25T11:21:38.860063+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.263510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510361+0000","last_change":"2024-04-25T11:21:37.865789+0000","last_active":"2024-04-25T11:21:39.510361+0000","last_peered":"2024-04-25T11:21:39.510361+0000","last_clean":"2024-04-25T11:21:39.510361+0000","last_became_active":"2024-04-25T11:21:37.865440+0000","last_became_peered":"2024-04-25T11:21:37.865440+0000","last_unstale":"2024-04-25T11:21:39.510361+0000","last_undegraded":"2024-04-25T11:21:39.510361+0000","last_fullsized":"2024-04-25T11:21:39.510361+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:16.677903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511531+0000","last_change":"2024-04-25T11:21:36.832331+0000","last_active":"2024-04-25T11:21:39.511531+0000","last_peered":"2024-04-25T11:21:39.511531+0000","last_clean":"2024-04-25T11:21:39.511531+0000","last_became_active":"2024-04-25T11:21:36.831955+0000","last_became_peered":"2024-04-25T11:21:36.831955+0000","last_unstale":"2024-04-25T11:21:39.511531+0000","last_undegraded":"2024-04-25T11:21:39.511531+0000","last_fullsized":"2024-04-25T11:21:39.511531+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:38.907256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.862803+0000","last_change":"2024-04-25T11:21:37.853591+0000","last_active":"2024-04-25T11:21:38.862803+0000","last_peered":"2024-04-25T11:21:38.862803+0000","last_clean":"2024-04-25T11:21:38.862803+0000","last_became_active":"2024-04-25T11:21:37.852133+0000","last_became_peered":"2024-04-25T11:21:37.852133+0000","last_unstale":"2024-04-25T11:21:38.862803+0000","last_undegraded":"2024-04-25T11:21:38.862803+0000","last_fullsized":"2024-04-25T11:21:38.862803+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:23.282068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512054+0000","last_change":"2024-04-25T11:21:36.832162+0000","last_active":"2024-04-25T11:21:39.512054+0000","last_peered":"2024-04-25T11:21:39.512054+0000","last_clean":"2024-04-25T11:21:39.512054+0000","last_became_active":"2024-04-25T11:21:36.831788+0000","last_became_peered":"2024-04-25T11:21:36.831788+0000","last_unstale":"2024-04-25T11:21:39.512054+0000","last_undegraded":"2024-04-25T11:21:39.512054+0000","last_fullsized":"2024-04-25T11:21:39.512054+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:50.617654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859709+0000","last_change":"2024-04-25T11:21:37.850475+0000","last_active":"2024-04-25T11:21:38.859709+0000","last_peered":"2024-04-25T11:21:38.859709+0000","last_clean":"2024-04-25T11:21:38.859709+0000","last_became_active":"2024-04-25T11:21:37.849891+0000","last_became_peered":"2024-04-25T11:21:37.849891+0000","last_unstale":"2024-04-25T11:21:38.859709+0000","last_undegraded":"2024-04-25T11:21:38.859709+0000","last_fullsized":"2024-04-25T11:21:38.859709+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:42.312337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879760+0000","last_change":"2024-04-25T11:21:36.831876+0000","last_active":"2024-04-25T11:21:38.879760+0000","last_peered":"2024-04-25T11:21:38.879760+0000","last_clean":"2024-04-25T11:21:38.879760+0000","last_became_active":"2024-04-25T11:21:36.831526+0000","last_became_peered":"2024-04-25T11:21:36.831526+0000","last_unstale":"2024-04-25T11:21:38.879760+0000","last_undegraded":"2024-04-25T11:21:38.879760+0000","last_fullsized":"2024-04-25T11:21:38.879760+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:37.812097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879343+0000","last_change":"2024-04-25T11:21:37.860454+0000","last_active":"2024-04-25T11:21:38.879343+0000","last_peered":"2024-04-25T11:21:38.879343+0000","last_clean":"2024-04-25T11:21:38.879343+0000","last_became_active":"2024-04-25T11:21:37.860104+0000","last_became_peered":"2024-04-25T11:21:37.860104+0000","last_unstale":"2024-04-25T11:21:38.879343+0000","last_undegraded":"2024-04-25T11:21:38.879343+0000","last_fullsized":"2024-04-25T11:21:38.879343+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:35.030082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511407+0000","last_change":"2024-04-25T11:21:36.833376+0000","last_active":"2024-04-25T11:21:39.511407+0000","last_peered":"2024-04-25T11:21:39.511407+0000","last_clean":"2024-04-25T11:21:39.511407+0000","last_became_active":"2024-04-25T11:21:36.833043+0000","last_became_peered":"2024-04-25T11:21:36.833043+0000","last_unstale":"2024-04-25T11:21:39.511407+0000","last_undegraded":"2024-04-25T11:21:39.511407+0000","last_fullsized":"2024-04-25T11:21:39.511407+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:56.334617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511292+0000","last_change":"2024-04-25T11:21:37.864561+0000","last_active":"2024-04-25T11:21:39.511292+0000","last_peered":"2024-04-25T11:21:39.511292+0000","last_clean":"2024-04-25T11:21:39.511292+0000","last_became_active":"2024-04-25T11:21:37.864168+0000","last_became_peered":"2024-04-25T11:21:37.864168+0000","last_unstale":"2024-04-25T11:21:39.511292+0000","last_undegraded":"2024-04-25T11:21:39.511292+0000","last_fullsized":"2024-04-25T11:21:39.511292+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:23.282068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866827+0000","last_change":"2024-04-25T11:21:36.833895+0000","last_active":"2024-04-25T11:21:38.866827+0000","last_peered":"2024-04-25T11:21:38.866827+0000","last_clean":"2024-04-25T11:21:38.866827+0000","last_became_active":"2024-04-25T11:21:36.832511+0000","last_became_peered":"2024-04-25T11:21:36.832511+0000","last_unstale":"2024-04-25T11:21:38.866827+0000","last_undegraded":"2024-04-25T11:21:38.866827+0000","last_fullsized":"2024-04-25T11:21:38.866827+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:46.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.864271+0000","last_change":"2024-04-25T11:21:37.859787+0000","last_active":"2024-04-25T11:21:37.864271+0000","last_peered":"2024-04-25T11:21:37.864271+0000","last_clean":"2024-04-25T11:21:37.864271+0000","last_became_active":"2024-04-25T11:21:37.858966+0000","last_became_peered":"2024-04-25T11:21:37.858966+0000","last_unstale":"2024-04-25T11:21:37.864271+0000","last_undegraded":"2024-04-25T11:21:37.864271+0000","last_fullsized":"2024-04-25T11:21:37.864271+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:29.069474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880489+0000","last_change":"2024-04-25T11:21:36.820101+0000","last_active":"2024-04-25T11:21:38.880489+0000","last_peered":"2024-04-25T11:21:38.880489+0000","last_clean":"2024-04-25T11:21:38.880489+0000","last_became_active":"2024-04-25T11:21:36.817759+0000","last_became_peered":"2024-04-25T11:21:36.817759+0000","last_unstale":"2024-04-25T11:21:38.880489+0000","last_undegraded":"2024-04-25T11:21:38.880489+0000","last_fullsized":"2024-04-25T11:21:38.880489+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:38.214877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.861018+0000","last_change":"2024-04-25T11:21:37.853434+0000","last_active":"2024-04-25T11:21:37.861018+0000","last_peered":"2024-04-25T11:21:37.861018+0000","last_clean":"2024-04-25T11:21:37.861018+0000","last_became_active":"2024-04-25T11:21:37.852678+0000","last_became_peered":"2024-04-25T11:21:37.852678+0000","last_unstale":"2024-04-25T11:21:37.861018+0000","last_undegraded":"2024-04-25T11:21:37.861018+0000","last_fullsized":"2024-04-25T11:21:37.861018+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:50.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"141'3","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.882999+0000","last_change":"2024-04-25T11:21:36.833059+0000","last_active":"2024-04-25T11:21:37.882999+0000","last_peered":"2024-04-25T11:21:37.882999+0000","last_clean":"2024-04-25T11:21:37.882999+0000","last_became_active":"2024-04-25T11:21:36.832460+0000","last_became_peered":"2024-04-25T11:21:36.832460+0000","last_unstale":"2024-04-25T11:21:37.882999+0000","last_undegraded":"2024-04-25T11:21:37.882999+0000","last_fullsized":"2024-04-25T11:21:37.882999+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:49.666663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510951+0000","last_change":"2024-04-25T11:21:37.865250+0000","last_active":"2024-04-25T11:21:39.510951+0000","last_peered":"2024-04-25T11:21:39.510951+0000","last_clean":"2024-04-25T11:21:39.510951+0000","last_became_active":"2024-04-25T11:21:37.863769+0000","last_became_peered":"2024-04-25T11:21:37.863769+0000","last_unstale":"2024-04-25T11:21:39.510951+0000","last_undegraded":"2024-04-25T11:21:39.510951+0000","last_fullsized":"2024-04-25T11:21:39.510951+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:05.096282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511730+0000","last_change":"2024-04-25T11:21:36.833841+0000","last_active":"2024-04-25T11:21:39.511730+0000","last_peered":"2024-04-25T11:21:39.511730+0000","last_clean":"2024-04-25T11:21:39.511730+0000","last_became_active":"2024-04-25T11:21:36.833478+0000","last_became_peered":"2024-04-25T11:21:36.833478+0000","last_unstale":"2024-04-25T11:21:39.511730+0000","last_undegraded":"2024-04-25T11:21:39.511730+0000","last_fullsized":"2024-04-25T11:21:39.511730+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:30.916117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860336+0000","last_change":"2024-04-25T11:21:37.855990+0000","last_active":"2024-04-25T11:21:38.860336+0000","last_peered":"2024-04-25T11:21:38.860336+0000","last_clean":"2024-04-25T11:21:38.860336+0000","last_became_active":"2024-04-25T11:21:37.854789+0000","last_became_peered":"2024-04-25T11:21:37.854789+0000","last_unstale":"2024-04-25T11:21:38.860336+0000","last_undegraded":"2024-04-25T11:21:38.860336+0000","last_fullsized":"2024-04-25T11:21:38.860336+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:51.636211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.862868+0000","last_change":"2024-04-25T11:21:36.834158+0000","last_active":"2024-04-25T11:21:38.862868+0000","last_peered":"2024-04-25T11:21:38.862868+0000","last_clean":"2024-04-25T11:21:38.862868+0000","last_became_active":"2024-04-25T11:21:36.833778+0000","last_became_peered":"2024-04-25T11:21:36.833778+0000","last_unstale":"2024-04-25T11:21:38.862868+0000","last_undegraded":"2024-04-25T11:21:38.862868+0000","last_fullsized":"2024-04-25T11:21:38.862868+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:39.711941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511875+0000","last_change":"2024-04-25T11:21:37.868534+0000","last_active":"2024-04-25T11:21:39.511875+0000","last_peered":"2024-04-25T11:21:39.511875+0000","last_clean":"2024-04-25T11:21:39.511875+0000","last_became_active":"2024-04-25T11:21:37.868139+0000","last_became_peered":"2024-04-25T11:21:37.868139+0000","last_unstale":"2024-04-25T11:21:39.511875+0000","last_undegraded":"2024-04-25T11:21:39.511875+0000","last_fullsized":"2024-04-25T11:21:39.511875+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:12.376728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.862958+0000","last_change":"2024-04-25T11:21:36.825262+0000","last_active":"2024-04-25T11:21:38.862958+0000","last_peered":"2024-04-25T11:21:38.862958+0000","last_clean":"2024-04-25T11:21:38.862958+0000","last_became_active":"2024-04-25T11:21:36.824311+0000","last_became_peered":"2024-04-25T11:21:36.824311+0000","last_unstale":"2024-04-25T11:21:38.862958+0000","last_undegraded":"2024-04-25T11:21:38.862958+0000","last_fullsized":"2024-04-25T11:21:38.862958+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:31.001996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859407+0000","last_change":"2024-04-25T11:21:37.875452+0000","last_active":"2024-04-25T11:21:38.859407+0000","last_peered":"2024-04-25T11:21:38.859407+0000","last_clean":"2024-04-25T11:21:38.859407+0000","last_became_active":"2024-04-25T11:21:37.874912+0000","last_became_peered":"2024-04-25T11:21:37.874912+0000","last_unstale":"2024-04-25T11:21:38.859407+0000","last_undegraded":"2024-04-25T11:21:38.859407+0000","last_fullsized":"2024-04-25T11:21:38.859407+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:37.904299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"141'8","reported_seq":22,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.878916+0000","last_change":"2024-04-25T11:21:36.821279+0000","last_active":"2024-04-25T11:21:38.878916+0000","last_peered":"2024-04-25T11:21:38.878916+0000","last_clean":"2024-04-25T11:21:38.878916+0000","last_became_active":"2024-04-25T11:21:36.820900+0000","last_became_peered":"2024-04-25T11:21:36.820900+0000","last_unstale":"2024-04-25T11:21:38.878916+0000","last_undegraded":"2024-04-25T11:21:38.878916+0000","last_fullsized":"2024-04-25T11:21:38.878916+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:39.711941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866447+0000","last_change":"2024-04-25T11:21:37.879517+0000","last_active":"2024-04-25T11:21:38.866447+0000","last_peered":"2024-04-25T11:21:38.866447+0000","last_clean":"2024-04-25T11:21:38.866447+0000","last_became_active":"2024-04-25T11:21:37.879190+0000","last_became_peered":"2024-04-25T11:21:37.879190+0000","last_unstale":"2024-04-25T11:21:38.866447+0000","last_undegraded":"2024-04-25T11:21:38.866447+0000","last_fullsized":"2024-04-25T11:21:38.866447+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:45.924348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"141'1","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.873876+0000","last_change":"2024-04-25T11:21:36.825400+0000","last_active":"2024-04-25T11:21:37.873876+0000","last_peered":"2024-04-25T11:21:37.873876+0000","last_clean":"2024-04-25T11:21:37.873876+0000","last_became_active":"2024-04-25T11:21:36.824827+0000","last_became_peered":"2024-04-25T11:21:36.824827+0000","last_unstale":"2024-04-25T11:21:37.873876+0000","last_undegraded":"2024-04-25T11:21:37.873876+0000","last_fullsized":"2024-04-25T11:21:37.873876+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:18.626686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.867309+0000","last_change":"2024-04-25T11:21:37.865043+0000","last_active":"2024-04-25T11:21:38.867309+0000","last_peered":"2024-04-25T11:21:38.867309+0000","last_clean":"2024-04-25T11:21:38.867309+0000","last_became_active":"2024-04-25T11:21:37.863907+0000","last_became_peered":"2024-04-25T11:21:37.863907+0000","last_unstale":"2024-04-25T11:21:38.867309+0000","last_undegraded":"2024-04-25T11:21:38.867309+0000","last_fullsized":"2024-04-25T11:21:38.867309+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:25.417788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866680+0000","last_change":"2024-04-25T11:21:36.833386+0000","last_active":"2024-04-25T11:21:38.866680+0000","last_peered":"2024-04-25T11:21:38.866680+0000","last_clean":"2024-04-25T11:21:38.866680+0000","last_became_active":"2024-04-25T11:21:36.833055+0000","last_became_peered":"2024-04-25T11:21:36.833055+0000","last_unstale":"2024-04-25T11:21:38.866680+0000","last_undegraded":"2024-04-25T11:21:38.866680+0000","last_fullsized":"2024-04-25T11:21:38.866680+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:46.031381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880393+0000","last_change":"2024-04-25T11:21:37.857002+0000","last_active":"2024-04-25T11:21:38.880393+0000","last_peered":"2024-04-25T11:21:38.880393+0000","last_clean":"2024-04-25T11:21:38.880393+0000","last_became_active":"2024-04-25T11:21:37.856155+0000","last_became_peered":"2024-04-25T11:21:37.856155+0000","last_unstale":"2024-04-25T11:21:38.880393+0000","last_undegraded":"2024-04-25T11:21:38.880393+0000","last_fullsized":"2024-04-25T11:21:38.880393+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:37.904299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860202+0000","last_change":"2024-04-25T11:21:36.829548+0000","last_active":"2024-04-25T11:21:38.860202+0000","last_peered":"2024-04-25T11:21:38.860202+0000","last_clean":"2024-04-25T11:21:38.860202+0000","last_became_active":"2024-04-25T11:21:36.829117+0000","last_became_peered":"2024-04-25T11:21:36.829117+0000","last_unstale":"2024-04-25T11:21:38.860202+0000","last_undegraded":"2024-04-25T11:21:38.860202+0000","last_fullsized":"2024-04-25T11:21:38.860202+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:19.372757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880460+0000","last_change":"2024-04-25T11:21:37.854334+0000","last_active":"2024-04-25T11:21:38.880460+0000","last_peered":"2024-04-25T11:21:38.880460+0000","last_clean":"2024-04-25T11:21:38.880460+0000","last_became_active":"2024-04-25T11:21:37.854007+0000","last_became_peered":"2024-04-25T11:21:37.854007+0000","last_unstale":"2024-04-25T11:21:38.880460+0000","last_undegraded":"2024-04-25T11:21:38.880460+0000","last_fullsized":"2024-04-25T11:21:38.880460+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:25.261420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863628+0000","last_change":"2024-04-25T11:21:36.835849+0000","last_active":"2024-04-25T11:21:38.863628+0000","last_peered":"2024-04-25T11:21:38.863628+0000","last_clean":"2024-04-25T11:21:38.863628+0000","last_became_active":"2024-04-25T11:21:36.835478+0000","last_became_peered":"2024-04-25T11:21:36.835478+0000","last_unstale":"2024-04-25T11:21:38.863628+0000","last_undegraded":"2024-04-25T11:21:38.863628+0000","last_fullsized":"2024-04-25T11:21:38.863628+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:33.381133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.865716+0000","last_change":"2024-04-25T11:21:37.865043+0000","last_active":"2024-04-25T11:21:38.865716+0000","last_peered":"2024-04-25T11:21:38.865716+0000","last_clean":"2024-04-25T11:21:38.865716+0000","last_became_active":"2024-04-25T11:21:37.864015+0000","last_became_peered":"2024-04-25T11:21:37.864015+0000","last_unstale":"2024-04-25T11:21:38.865716+0000","last_undegraded":"2024-04-25T11:21:38.865716+0000","last_fullsized":"2024-04-25T11:21:38.865716+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:27.727832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.509973+0000","last_change":"2024-04-25T11:21:36.839291+0000","last_active":"2024-04-25T11:21:39.509973+0000","last_peered":"2024-04-25T11:21:39.509973+0000","last_clean":"2024-04-25T11:21:39.509973+0000","last_became_active":"2024-04-25T11:21:36.838114+0000","last_became_peered":"2024-04-25T11:21:36.838114+0000","last_unstale":"2024-04-25T11:21:39.509973+0000","last_undegraded":"2024-04-25T11:21:39.509973+0000","last_fullsized":"2024-04-25T11:21:39.509973+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:14.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860458+0000","last_change":"2024-04-25T11:21:37.857105+0000","last_active":"2024-04-25T11:21:38.860458+0000","last_peered":"2024-04-25T11:21:38.860458+0000","last_clean":"2024-04-25T11:21:38.860458+0000","last_became_active":"2024-04-25T11:21:37.856711+0000","last_became_peered":"2024-04-25T11:21:37.856711+0000","last_unstale":"2024-04-25T11:21:38.860458+0000","last_undegraded":"2024-04-25T11:21:38.860458+0000","last_fullsized":"2024-04-25T11:21:38.860458+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:59.397407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510361+0000","last_change":"2024-04-25T11:21:36.837206+0000","last_active":"2024-04-25T11:21:39.510361+0000","last_peered":"2024-04-25T11:21:39.510361+0000","last_clean":"2024-04-25T11:21:39.510361+0000","last_became_active":"2024-04-25T11:21:36.836772+0000","last_became_peered":"2024-04-25T11:21:36.836772+0000","last_unstale":"2024-04-25T11:21:39.510361+0000","last_undegraded":"2024-04-25T11:21:39.510361+0000","last_fullsized":"2024-04-25T11:21:39.510361+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:06.399013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866348+0000","last_change":"2024-04-25T11:21:37.856653+0000","last_active":"2024-04-25T11:21:38.866348+0000","last_peered":"2024-04-25T11:21:38.866348+0000","last_clean":"2024-04-25T11:21:38.866348+0000","last_became_active":"2024-04-25T11:21:37.856343+0000","last_became_peered":"2024-04-25T11:21:37.856343+0000","last_unstale":"2024-04-25T11:21:38.866348+0000","last_undegraded":"2024-04-25T11:21:38.866348+0000","last_fullsized":"2024-04-25T11:21:38.866348+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:46.354275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.509610+0000","last_change":"2024-04-25T11:21:36.840531+0000","last_active":"2024-04-25T11:21:39.509610+0000","last_peered":"2024-04-25T11:21:39.509610+0000","last_clean":"2024-04-25T11:21:39.509610+0000","last_became_active":"2024-04-25T11:21:36.839923+0000","last_became_peered":"2024-04-25T11:21:36.839923+0000","last_unstale":"2024-04-25T11:21:39.509610+0000","last_undegraded":"2024-04-25T11:21:39.509610+0000","last_fullsized":"2024-04-25T11:21:39.509610+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:19.234930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859620+0000","last_change":"2024-04-25T11:21:37.865695+0000","last_active":"2024-04-25T11:21:38.859620+0000","last_peered":"2024-04-25T11:21:38.859620+0000","last_clean":"2024-04-25T11:21:38.859620+0000","last_became_active":"2024-04-25T11:21:37.864120+0000","last_became_peered":"2024-04-25T11:21:37.864120+0000","last_unstale":"2024-04-25T11:21:38.859620+0000","last_undegraded":"2024-04-25T11:21:38.859620+0000","last_fullsized":"2024-04-25T11:21:38.859620+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:25.261420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511144+0000","last_change":"2024-04-25T11:21:36.839357+0000","last_active":"2024-04-25T11:21:39.511144+0000","last_peered":"2024-04-25T11:21:39.511144+0000","last_clean":"2024-04-25T11:21:39.511144+0000","last_became_active":"2024-04-25T11:21:36.838813+0000","last_became_peered":"2024-04-25T11:21:36.838813+0000","last_unstale":"2024-04-25T11:21:39.511144+0000","last_undegraded":"2024-04-25T11:21:39.511144+0000","last_fullsized":"2024-04-25T11:21:39.511144+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:39.782765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512150+0000","last_change":"2024-04-25T11:21:37.857611+0000","last_active":"2024-04-25T11:21:39.512150+0000","last_peered":"2024-04-25T11:21:39.512150+0000","last_clean":"2024-04-25T11:21:39.512150+0000","last_became_active":"2024-04-25T11:21:37.857121+0000","last_became_peered":"2024-04-25T11:21:37.857121+0000","last_unstale":"2024-04-25T11:21:39.512150+0000","last_undegraded":"2024-04-25T11:21:39.512150+0000","last_fullsized":"2024-04-25T11:21:39.512150+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:08.443311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863854+0000","last_change":"2024-04-25T11:21:36.838832+0000","last_active":"2024-04-25T11:21:38.863854+0000","last_peered":"2024-04-25T11:21:38.863854+0000","last_clean":"2024-04-25T11:21:38.863854+0000","last_became_active":"2024-04-25T11:21:36.838251+0000","last_became_peered":"2024-04-25T11:21:36.838251+0000","last_unstale":"2024-04-25T11:21:38.863854+0000","last_undegraded":"2024-04-25T11:21:38.863854+0000","last_fullsized":"2024-04-25T11:21:38.863854+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:37.812097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.864131+0000","last_change":"2024-04-25T11:21:37.878247+0000","last_active":"2024-04-25T11:21:38.864131+0000","last_peered":"2024-04-25T11:21:38.864131+0000","last_clean":"2024-04-25T11:21:38.864131+0000","last_became_active":"2024-04-25T11:21:37.877612+0000","last_became_peered":"2024-04-25T11:21:37.877612+0000","last_unstale":"2024-04-25T11:21:38.864131+0000","last_undegraded":"2024-04-25T11:21:38.864131+0000","last_fullsized":"2024-04-25T11:21:38.864131+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:29.312058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"142'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880609+0000","last_change":"2024-04-25T11:21:36.827403+0000","last_active":"2024-04-25T11:21:38.880609+0000","last_peered":"2024-04-25T11:21:38.880609+0000","last_clean":"2024-04-25T11:21:38.880609+0000","last_became_active":"2024-04-25T11:21:36.826964+0000","last_became_peered":"2024-04-25T11:21:36.826964+0000","last_unstale":"2024-04-25T11:21:38.880609+0000","last_undegraded":"2024-04-25T11:21:38.880609+0000","last_fullsized":"2024-04-25T11:21:38.880609+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:35.825822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880443+0000","last_change":"2024-04-25T11:21:37.862311+0000","last_active":"2024-04-25T11:21:38.880443+0000","last_peered":"2024-04-25T11:21:38.880443+0000","last_clean":"2024-04-25T11:21:38.880443+0000","last_became_active":"2024-04-25T11:21:37.860211+0000","last_became_peered":"2024-04-25T11:21:37.860211+0000","last_unstale":"2024-04-25T11:21:38.880443+0000","last_undegraded":"2024-04-25T11:21:38.880443+0000","last_fullsized":"2024-04-25T11:21:38.880443+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:05.598346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860244+0000","last_change":"2024-04-25T11:21:36.839486+0000","last_active":"2024-04-25T11:21:38.860244+0000","last_peered":"2024-04-25T11:21:38.860244+0000","last_clean":"2024-04-25T11:21:38.860244+0000","last_became_active":"2024-04-25T11:21:36.839123+0000","last_became_peered":"2024-04-25T11:21:36.839123+0000","last_unstale":"2024-04-25T11:21:38.860244+0000","last_undegraded":"2024-04-25T11:21:38.860244+0000","last_fullsized":"2024-04-25T11:21:38.860244+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:55.131293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512032+0000","last_change":"2024-04-25T11:21:37.866908+0000","last_active":"2024-04-25T11:21:39.512032+0000","last_peered":"2024-04-25T11:21:39.512032+0000","last_clean":"2024-04-25T11:21:39.512032+0000","last_became_active":"2024-04-25T11:21:37.866577+0000","last_became_peered":"2024-04-25T11:21:37.866577+0000","last_unstale":"2024-04-25T11:21:39.512032+0000","last_undegraded":"2024-04-25T11:21:39.512032+0000","last_fullsized":"2024-04-25T11:21:39.512032+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:00.851705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512046+0000","last_change":"2024-04-25T11:21:36.839386+0000","last_active":"2024-04-25T11:21:39.512046+0000","last_peered":"2024-04-25T11:21:39.512046+0000","last_clean":"2024-04-25T11:21:39.512046+0000","last_became_active":"2024-04-25T11:21:36.839055+0000","last_became_peered":"2024-04-25T11:21:36.839055+0000","last_unstale":"2024-04-25T11:21:39.512046+0000","last_undegraded":"2024-04-25T11:21:39.512046+0000","last_fullsized":"2024-04-25T11:21:39.512046+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:46.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511753+0000","last_change":"2024-04-25T11:21:37.870052+0000","last_active":"2024-04-25T11:21:39.511753+0000","last_peered":"2024-04-25T11:21:39.511753+0000","last_clean":"2024-04-25T11:21:39.511753+0000","last_became_active":"2024-04-25T11:21:37.869748+0000","last_became_peered":"2024-04-25T11:21:37.869748+0000","last_unstale":"2024-04-25T11:21:39.511753+0000","last_undegraded":"2024-04-25T11:21:39.511753+0000","last_fullsized":"2024-04-25T11:21:39.511753+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:56.956131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879327+0000","last_change":"2024-04-25T11:21:36.823622+0000","last_active":"2024-04-25T11:21:38.879327+0000","last_peered":"2024-04-25T11:21:38.879327+0000","last_clean":"2024-04-25T11:21:38.879327+0000","last_became_active":"2024-04-25T11:21:36.823264+0000","last_became_peered":"2024-04-25T11:21:36.823264+0000","last_unstale":"2024-04-25T11:21:38.879327+0000","last_undegraded":"2024-04-25T11:21:38.879327+0000","last_fullsized":"2024-04-25T11:21:38.879327+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:51.482526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863657+0000","last_change":"2024-04-25T11:21:37.869787+0000","last_active":"2024-04-25T11:21:38.863657+0000","last_peered":"2024-04-25T11:21:38.863657+0000","last_clean":"2024-04-25T11:21:38.863657+0000","last_became_active":"2024-04-25T11:21:37.869359+0000","last_became_peered":"2024-04-25T11:21:37.869359+0000","last_unstale":"2024-04-25T11:21:38.863657+0000","last_undegraded":"2024-04-25T11:21:38.863657+0000","last_fullsized":"2024-04-25T11:21:38.863657+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:33.193064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.848712+0000","last_change":"2024-04-25T11:21:36.827079+0000","last_active":"2024-04-25T11:21:37.848712+0000","last_peered":"2024-04-25T11:21:37.848712+0000","last_clean":"2024-04-25T11:21:37.848712+0000","last_became_active":"2024-04-25T11:21:36.826731+0000","last_became_peered":"2024-04-25T11:21:36.826731+0000","last_unstale":"2024-04-25T11:21:37.848712+0000","last_undegraded":"2024-04-25T11:21:37.848712+0000","last_fullsized":"2024-04-25T11:21:37.848712+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:27.145048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859899+0000","last_change":"2024-04-25T11:21:37.865367+0000","last_active":"2024-04-25T11:21:38.859899+0000","last_peered":"2024-04-25T11:21:38.859899+0000","last_clean":"2024-04-25T11:21:38.859899+0000","last_became_active":"2024-04-25T11:21:37.865003+0000","last_became_peered":"2024-04-25T11:21:37.865003+0000","last_unstale":"2024-04-25T11:21:38.859899+0000","last_undegraded":"2024-04-25T11:21:38.859899+0000","last_fullsized":"2024-04-25T11:21:38.859899+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:47.692135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879993+0000","last_change":"2024-04-25T11:21:36.819347+0000","last_active":"2024-04-25T11:21:38.879993+0000","last_peered":"2024-04-25T11:21:38.879993+0000","last_clean":"2024-04-25T11:21:38.879993+0000","last_became_active":"2024-04-25T11:21:36.819000+0000","last_became_peered":"2024-04-25T11:21:36.819000+0000","last_unstale":"2024-04-25T11:21:38.879993+0000","last_undegraded":"2024-04-25T11:21:38.879993+0000","last_fullsized":"2024-04-25T11:21:38.879993+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:31.181592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511160+0000","last_change":"2024-04-25T11:21:37.874813+0000","last_active":"2024-04-25T11:21:39.511160+0000","last_peered":"2024-04-25T11:21:39.511160+0000","last_clean":"2024-04-25T11:21:39.511160+0000","last_became_active":"2024-04-25T11:21:37.874199+0000","last_became_peered":"2024-04-25T11:21:37.874199+0000","last_unstale":"2024-04-25T11:21:39.511160+0000","last_undegraded":"2024-04-25T11:21:39.511160+0000","last_fullsized":"2024-04-25T11:21:39.511160+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:20.391314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.865482+0000","last_change":"2024-04-25T11:21:36.830719+0000","last_active":"2024-04-25T11:21:38.865482+0000","last_peered":"2024-04-25T11:21:38.865482+0000","last_clean":"2024-04-25T11:21:38.865482+0000","last_became_active":"2024-04-25T11:21:36.829900+0000","last_became_peered":"2024-04-25T11:21:36.829900+0000","last_unstale":"2024-04-25T11:21:38.865482+0000","last_undegraded":"2024-04-25T11:21:38.865482+0000","last_fullsized":"2024-04-25T11:21:38.865482+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:50.617654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511919+0000","last_change":"2024-04-25T11:21:37.867478+0000","last_active":"2024-04-25T11:21:39.511919+0000","last_peered":"2024-04-25T11:21:39.511919+0000","last_clean":"2024-04-25T11:21:39.511919+0000","last_became_active":"2024-04-25T11:21:37.867180+0000","last_became_peered":"2024-04-25T11:21:37.867180+0000","last_unstale":"2024-04-25T11:21:39.511919+0000","last_undegraded":"2024-04-25T11:21:39.511919+0000","last_fullsized":"2024-04-25T11:21:39.511919+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:33.193064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"141'6","reported_seq":20,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510284+0000","last_change":"2024-04-25T11:21:36.838957+0000","last_active":"2024-04-25T11:21:39.510284+0000","last_peered":"2024-04-25T11:21:39.510284+0000","last_clean":"2024-04-25T11:21:39.510284+0000","last_became_active":"2024-04-25T11:21:36.838625+0000","last_became_peered":"2024-04-25T11:21:36.838625+0000","last_unstale":"2024-04-25T11:21:39.510284+0000","last_undegraded":"2024-04-25T11:21:39.510284+0000","last_fullsized":"2024-04-25T11:21:39.510284+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.399231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":353,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.865849+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:21:38.865849+0000","last_peered":"2024-04-25T11:21:38.865849+0000","last_clean":"2024-04-25T11:21:38.865849+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:21:38.865849+0000","last_undegraded":"2024-04-25T11:21:38.865849+0000","last_fullsized":"2024-04-25T11:21:38.865849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33496,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338344,"statfs":{"total":96636764160,"available":96602464256,"internally_reserved":0,"allocated":2367488,"data_stored":1409380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476877,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1773568,"data_stored":819060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1769472,"data_stored":818990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476877,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1781760,"data_stored":820602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37064,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334776,"statfs":{"total":96636764160,"available":96598810624,"internally_reserved":0,"allocated":1765376,"data_stored":812914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509581,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32916,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338924,"statfs":{"total":96636764160,"available":96603058176,"internally_reserved":0,"allocated":1773568,"data_stored":819060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2387968,"data_stored":1411676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542285,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1765376,"data_stored":812410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:21:42.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:21:42.764 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-04-25T11:21:42.769+0000 7fd1fb600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:21:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.937+0000 7f2db1600640 1 Processor -- start 2024-04-25T11:21:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.937+0000 7f2db1600640 1 -- start start 2024-04-25T11:21:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.939+0000 7f2db1600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac0b37a0 0x7f2dac0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.940+0000 7f2db1600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2dac163b60 0x7f2dac09b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.940+0000 7f2db1600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dac1644d0 con 0x7f2dac163f70 2024-04-25T11:21:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.940+0000 7f2db1600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2dac164a90 con 0x7f2dac163b60 2024-04-25T11:21:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.940+0000 7f2db1600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2dac0b2620 con 0x7f2dac0b37a0 2024-04-25T11:21:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.940+0000 7f2db0c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2dac163f70 0x7f2dac0af5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47620/0 (socket says 172.21.15.7:47620) 2024-04-25T11:21:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.940+0000 7f2db0c00640 1 -- 172.21.15.7:0/1859879658 learned_addr learned my addr 172.21.15.7:0/1859879658 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.940+0000 7f2dabe00640 1 --2- 172.21.15.7:0/1859879658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac0b37a0 0x7f2dac0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.940+0000 7f2dab400640 1 --2- 172.21.15.7:0/1859879658 >> v2:172.21.15.87:3300/0 conn(0x7f2dac163b60 0x7f2dac09b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.940+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 65005278 0 0) 0x7f2dac1644d0 con 0x7f2dac163f70 2024-04-25T11:21:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d90003520 con 0x7f2dac163f70 2024-04-25T11:21:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3970179453 0 0) 0x7f2d90003520 con 0x7f2dac163f70 2024-04-25T11:21:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac0b37a0 msgr2=0x7f2dac0b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 --2- 172.21.15.7:0/1859879658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac0b37a0 0x7f2dac0b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 >> v2:172.21.15.87:3300/0 conn(0x7f2dac163b60 msgr2=0x7f2dac09b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 --2- 172.21.15.7:0/1859879658 >> v2:172.21.15.87:3300/0 conn(0x7f2dac163b60 0x7f2dac09b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac1647b0 con 0x7f2dac163f70 2024-04-25T11:21:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2da4001670 con 0x7f2dac163f70 2024-04-25T11:21:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 >> v1:172.21.15.7:6789/0 conn(0x7f2dac163f70 legacy=0x7f2dac0af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 --2- 172.21.15.7:0/1859879658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d900039b0 0x7f2d90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 --2- 172.21.15.7:0/1859879658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d90004800 0x7f2d90004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 --2- 172.21.15.7:0/1859879658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac0b37a0 0x7f2d90005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac0b2620 con 0x7f2d900039b0 2024-04-25T11:21:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9000ba70 con 0x7f2d90004800 2024-04-25T11:21:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2d9000bc20 con 0x7f2dac0b37a0 2024-04-25T11:21:42.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.941+0000 7f2dabe00640 1 --2- 172.21.15.7:0/1859879658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d900039b0 0x7f2d90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.942+0000 7f2dab400640 1 --2- 172.21.15.7:0/1859879658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d90004800 0x7f2d90004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.942+0000 7f2dabe00640 1 -- 172.21.15.7:0/1859879658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac0b37a0 msgr2=0x7f2d90005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.942+0000 7f2dabe00640 1 --2- 172.21.15.7:0/1859879658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac0b37a0 0x7f2d90005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.942+0000 7f2dabe00640 1 -- 172.21.15.7:0/1859879658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d90004800 msgr2=0x7f2d90004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.942+0000 7f2dabe00640 1 --2- 172.21.15.7:0/1859879658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d90004800 0x7f2d90004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.942+0000 7f2dabe00640 1 -- 172.21.15.7:0/1859879658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d90003520 con 0x7f2d900039b0 2024-04-25T11:21:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.943+0000 7f2dabe00640 1 --2- 172.21.15.7:0/1859879658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d900039b0 0x7f2d90003e20 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009a90 tx=0x7f2d90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae23437b61bc7403 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.943+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c036070 con 0x7f2d900039b0 2024-04-25T11:21:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.943+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9c031400 con 0x7f2d900039b0 2024-04-25T11:21:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.943+0000 7f2daaa00640 1 -- 172.21.15.7:0/1859879658 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c009690 con 0x7f2d900039b0 2024-04-25T11:21:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.944+0000 7f2db1600640 1 -- 172.21.15.7:0/1859879658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d900039b0 msgr2=0x7f2d90003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.944+0000 7f2db1600640 1 --2- 172.21.15.7:0/1859879658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d900039b0 0x7f2d90003e20 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009a90 tx=0x7f2d90005830 comp rx=0 tx=0).stop 2024-04-25T11:21:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.944+0000 7f2db0c00640 1 -- 172.21.15.7:0/1859879658 reap_dead start 2024-04-25T11:21:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.944+0000 7f2db1600640 1 -- 172.21.15.7:0/1859879658 shutdown_connections 2024-04-25T11:21:42.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.944+0000 7f2db1600640 1 -- 172.21.15.7:0/1859879658 >> 172.21.15.7:0/1859879658 conn(0x7f2dac074410 msgr2=0x7f2dac0b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:42.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.944+0000 7f2db1600640 1 -- 172.21.15.7:0/1859879658 shutdown_connections 2024-04-25T11:21:42.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.944+0000 7f2db1600640 1 -- 172.21.15.7:0/1859879658 wait complete. 2024-04-25T11:21:42.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.947+0000 7f2db1600640 1 Processor -- start 2024-04-25T11:21:42.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.947+0000 7f2db1600640 1 -- start start 2024-04-25T11:21:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.948+0000 7f2db1600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d900039b0 0x7f2dac1430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.948+0000 7f2db1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac140270 0x7f2dac1f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.948+0000 7f2db1600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac1f9270 0x7f2dac1f9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.948+0000 7f2db1600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac09ba80 con 0x7f2dac140270 2024-04-25T11:21:42.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.948+0000 7f2db1600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac0b1e80 con 0x7f2d900039b0 2024-04-25T11:21:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.948+0000 7f2db1600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2dac09bc00 con 0x7f2dac1f9270 2024-04-25T11:21:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.949+0000 7f2dab400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac140270 0x7f2dac1f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.949+0000 7f2dab400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac140270 0x7f2dac1f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46570/0 (socket says 172.21.15.7:46570) 2024-04-25T11:21:42.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.949+0000 7f2dab400640 1 -- 172.21.15.7:0/1917699414 learned_addr learned my addr 172.21.15.7:0/1917699414 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.949+0000 7f2dabe00640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d900039b0 0x7f2dac1430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.949+0000 7f2db0c00640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac1f9270 0x7f2dac1f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.949+0000 7f2dab400640 1 -- 172.21.15.7:0/1917699414 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac1f9270 msgr2=0x7f2dac1f9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.949+0000 7f2dab400640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac1f9270 0x7f2dac1f9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.949+0000 7f2dab400640 1 -- 172.21.15.7:0/1917699414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d900039b0 msgr2=0x7f2dac1430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.949+0000 7f2dab400640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d900039b0 0x7f2dac1430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.949+0000 7f2dab400640 1 -- 172.21.15.7:0/1917699414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac1f3640 con 0x7f2dac140270 2024-04-25T11:21:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.949+0000 7f2dabe00640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d900039b0 0x7f2dac1430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.950+0000 7f2dab400640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac140270 0x7f2dac1f3100 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009f30 tx=0x7f2d980115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.950+0000 7f2da3e00640 1 -- 172.21.15.7:0/1917699414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98003990 con 0x7f2dac140270 2024-04-25T11:21:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.950+0000 7f2da3e00640 1 -- 172.21.15.7:0/1917699414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9801b070 con 0x7f2dac140270 2024-04-25T11:21:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.950+0000 7f2db1600640 1 -- 172.21.15.7:0/1917699414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dac1f9bc0 con 0x7f2dac140270 2024-04-25T11:21:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.950+0000 7f2da3e00640 1 -- 172.21.15.7:0/1917699414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98003b70 con 0x7f2dac140270 2024-04-25T11:21:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.950+0000 7f2db1600640 1 -- 172.21.15.7:0/1917699414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dac141a90 con 0x7f2dac140270 2024-04-25T11:21:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.952+0000 7f2db1600640 1 -- 172.21.15.7:0/1917699414 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dac050740 con 0x7f2dac140270 2024-04-25T11:21:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.953+0000 7f2da3e00640 1 -- 172.21.15.7:0/1917699414 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d98005d10 con 0x7f2dac140270 2024-04-25T11:21:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.954+0000 7f2da3e00640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d8c0744b0 0x7f2d8c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.955+0000 7f2da3e00640 1 -- 172.21.15.7:0/1917699414 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f2d98095ad0 con 0x7f2dac140270 2024-04-25T11:21:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.955+0000 7f2dabe00640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d8c0744b0 0x7f2d8c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:42.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.956+0000 7f2dabe00640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d8c0744b0 0x7f2d8c0769c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c033000 tx=0x7f2d9c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:42.960+0000 7f2da3e00640 1 -- 172.21.15.7:0/1917699414 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2dac1f9bc0 con 0x7f2dac140270 2024-04-25T11:21:43.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.162+0000 7f2db1600640 1 -- 172.21.15.7:0/1917699414 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2dac149a40 con 0x7f2d8c0744b0 2024-04-25T11:21:43.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.181+0000 7f2da3e00640 1 -- 172.21.15.7:0/1917699414 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252466 (secure 0 0 0) 0x7f2dac149a40 con 0x7f2d8c0744b0 2024-04-25T11:21:43.180 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:21:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.186+0000 7f2db1600640 1 -- 172.21.15.7:0/1917699414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d8c0744b0 msgr2=0x7f2d8c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.186+0000 7f2db1600640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d8c0744b0 0x7f2d8c0769c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c033000 tx=0x7f2d9c008030 comp rx=0 tx=0).stop 2024-04-25T11:21:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.186+0000 7f2db1600640 1 -- 172.21.15.7:0/1917699414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac140270 msgr2=0x7f2dac1f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.186+0000 7f2db1600640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac140270 0x7f2dac1f3100 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009f30 tx=0x7f2d980115a0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.187+0000 7f2db1600640 1 -- 172.21.15.7:0/1917699414 shutdown_connections 2024-04-25T11:21:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.187+0000 7f2db1600640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d8c0744b0 0x7f2d8c0769c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.187+0000 7f2db1600640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2dac1f9270 0x7f2dac1f9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.187+0000 7f2db1600640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac140270 0x7f2dac1f3100 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.187+0000 7f2db1600640 1 --2- 172.21.15.7:0/1917699414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d900039b0 0x7f2dac1430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.187+0000 7f2db1600640 1 -- 172.21.15.7:0/1917699414 >> 172.21.15.7:0/1917699414 conn(0x7f2dac074410 msgr2=0x7f2dac070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.187+0000 7f2db1600640 1 -- 172.21.15.7:0/1917699414 shutdown_connections 2024-04-25T11:21:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.187+0000 7f2db1600640 1 -- 172.21.15.7:0/1917699414 wait complete. 2024-04-25T11:21:43.211 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-04-25T11:21:42.052623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281132,"kb_used_data":15024,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754693588,"statfs":{"total":773094113280,"available":772806234112,"internally_reserved":0,"allocated":15384576,"data_stored":7724092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002248"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880017+0000","last_change":"2024-04-25T11:21:37.862629+0000","last_active":"2024-04-25T11:21:38.880017+0000","last_peered":"2024-04-25T11:21:38.880017+0000","last_clean":"2024-04-25T11:21:38.880017+0000","last_became_active":"2024-04-25T11:21:37.862262+0000","last_became_peered":"2024-04-25T11:21:37.862262+0000","last_unstale":"2024-04-25T11:21:38.880017+0000","last_undegraded":"2024-04-25T11:21:38.880017+0000","last_fullsized":"2024-04-25T11:21:38.880017+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:33.889420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859566+0000","last_change":"2024-04-25T11:21:37.862282+0000","last_active":"2024-04-25T11:21:38.859566+0000","last_peered":"2024-04-25T11:21:38.859566+0000","last_clean":"2024-04-25T11:21:38.859566+0000","last_became_active":"2024-04-25T11:21:37.861921+0000","last_became_peered":"2024-04-25T11:21:37.861921+0000","last_unstale":"2024-04-25T11:21:38.859566+0000","last_undegraded":"2024-04-25T11:21:38.859566+0000","last_fullsized":"2024-04-25T11:21:38.859566+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:05.598346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859859+0000","last_change":"2024-04-25T11:21:37.859543+0000","last_active":"2024-04-25T11:21:38.859859+0000","last_peered":"2024-04-25T11:21:38.859859+0000","last_clean":"2024-04-25T11:21:38.859859+0000","last_became_active":"2024-04-25T11:21:37.859191+0000","last_became_peered":"2024-04-25T11:21:37.859191+0000","last_unstale":"2024-04-25T11:21:38.859859+0000","last_undegraded":"2024-04-25T11:21:38.859859+0000","last_fullsized":"2024-04-25T11:21:38.859859+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:59.739927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863049+0000","last_change":"2024-04-25T11:21:37.861991+0000","last_active":"2024-04-25T11:21:38.863049+0000","last_peered":"2024-04-25T11:21:38.863049+0000","last_clean":"2024-04-25T11:21:38.863049+0000","last_became_active":"2024-04-25T11:21:37.861655+0000","last_became_peered":"2024-04-25T11:21:37.861655+0000","last_unstale":"2024-04-25T11:21:38.863049+0000","last_undegraded":"2024-04-25T11:21:38.863049+0000","last_fullsized":"2024-04-25T11:21:38.863049+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:56.149851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511528+0000","last_change":"2024-04-25T11:21:37.863943+0000","last_active":"2024-04-25T11:21:39.511528+0000","last_peered":"2024-04-25T11:21:39.511528+0000","last_clean":"2024-04-25T11:21:39.511528+0000","last_became_active":"2024-04-25T11:21:37.853267+0000","last_became_peered":"2024-04-25T11:21:37.853267+0000","last_unstale":"2024-04-25T11:21:39.511528+0000","last_undegraded":"2024-04-25T11:21:39.511528+0000","last_fullsized":"2024-04-25T11:21:39.511528+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:22.520785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511976+0000","last_change":"2024-04-25T11:21:37.850523+0000","last_active":"2024-04-25T11:21:39.511976+0000","last_peered":"2024-04-25T11:21:39.511976+0000","last_clean":"2024-04-25T11:21:39.511976+0000","last_became_active":"2024-04-25T11:21:37.850211+0000","last_became_peered":"2024-04-25T11:21:37.850211+0000","last_unstale":"2024-04-25T11:21:39.511976+0000","last_undegraded":"2024-04-25T11:21:39.511976+0000","last_fullsized":"2024-04-25T11:21:39.511976+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:45.489827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879591+0000","last_change":"2024-04-25T11:21:37.849431+0000","last_active":"2024-04-25T11:21:38.879591+0000","last_peered":"2024-04-25T11:21:38.879591+0000","last_clean":"2024-04-25T11:21:38.879591+0000","last_became_active":"2024-04-25T11:21:37.849011+0000","last_became_peered":"2024-04-25T11:21:37.849011+0000","last_unstale":"2024-04-25T11:21:38.879591+0000","last_undegraded":"2024-04-25T11:21:38.879591+0000","last_fullsized":"2024-04-25T11:21:38.879591+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:03.475738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510461+0000","last_change":"2024-04-25T11:21:37.864988+0000","last_active":"2024-04-25T11:21:39.510461+0000","last_peered":"2024-04-25T11:21:39.510461+0000","last_clean":"2024-04-25T11:21:39.510461+0000","last_became_active":"2024-04-25T11:21:37.864317+0000","last_became_peered":"2024-04-25T11:21:37.864317+0000","last_unstale":"2024-04-25T11:21:39.510461+0000","last_undegraded":"2024-04-25T11:21:39.510461+0000","last_fullsized":"2024-04-25T11:21:39.510461+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:36.844379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511921+0000","last_change":"2024-04-25T11:21:37.872632+0000","last_active":"2024-04-25T11:21:39.511921+0000","last_peered":"2024-04-25T11:21:39.511921+0000","last_clean":"2024-04-25T11:21:39.511921+0000","last_became_active":"2024-04-25T11:21:37.872308+0000","last_became_peered":"2024-04-25T11:21:37.872308+0000","last_unstale":"2024-04-25T11:21:39.511921+0000","last_undegraded":"2024-04-25T11:21:39.511921+0000","last_fullsized":"2024-04-25T11:21:39.511921+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:23.638662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512050+0000","last_change":"2024-04-25T11:21:37.871717+0000","last_active":"2024-04-25T11:21:39.512050+0000","last_peered":"2024-04-25T11:21:39.512050+0000","last_clean":"2024-04-25T11:21:39.512050+0000","last_became_active":"2024-04-25T11:21:37.871268+0000","last_became_peered":"2024-04-25T11:21:37.871268+0000","last_unstale":"2024-04-25T11:21:39.512050+0000","last_undegraded":"2024-04-25T11:21:39.512050+0000","last_fullsized":"2024-04-25T11:21:39.512050+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:29.164577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.864299+0000","last_change":"2024-04-25T11:21:37.860096+0000","last_active":"2024-04-25T11:21:37.864299+0000","last_peered":"2024-04-25T11:21:37.864299+0000","last_clean":"2024-04-25T11:21:37.864299+0000","last_became_active":"2024-04-25T11:21:37.859399+0000","last_became_peered":"2024-04-25T11:21:37.859399+0000","last_unstale":"2024-04-25T11:21:37.864299+0000","last_undegraded":"2024-04-25T11:21:37.864299+0000","last_fullsized":"2024-04-25T11:21:37.864299+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:20.179524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863582+0000","last_change":"2024-04-25T11:21:37.875500+0000","last_active":"2024-04-25T11:21:38.863582+0000","last_peered":"2024-04-25T11:21:38.863582+0000","last_clean":"2024-04-25T11:21:38.863582+0000","last_became_active":"2024-04-25T11:21:37.875146+0000","last_became_peered":"2024-04-25T11:21:37.875146+0000","last_unstale":"2024-04-25T11:21:38.863582+0000","last_undegraded":"2024-04-25T11:21:38.863582+0000","last_fullsized":"2024-04-25T11:21:38.863582+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:56.956131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511332+0000","last_change":"2024-04-25T11:21:37.865603+0000","last_active":"2024-04-25T11:21:39.511332+0000","last_peered":"2024-04-25T11:21:39.511332+0000","last_clean":"2024-04-25T11:21:39.511332+0000","last_became_active":"2024-04-25T11:21:37.854582+0000","last_became_peered":"2024-04-25T11:21:37.854582+0000","last_unstale":"2024-04-25T11:21:39.511332+0000","last_undegraded":"2024-04-25T11:21:39.511332+0000","last_fullsized":"2024-04-25T11:21:39.511332+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:18.497127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863054+0000","last_change":"2024-04-25T11:21:37.876260+0000","last_active":"2024-04-25T11:21:38.863054+0000","last_peered":"2024-04-25T11:21:38.863054+0000","last_clean":"2024-04-25T11:21:38.863054+0000","last_became_active":"2024-04-25T11:21:37.875537+0000","last_became_peered":"2024-04-25T11:21:37.875537+0000","last_unstale":"2024-04-25T11:21:38.863054+0000","last_undegraded":"2024-04-25T11:21:38.863054+0000","last_fullsized":"2024-04-25T11:21:38.863054+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:33.202111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859901+0000","last_change":"2024-04-25T11:21:37.857192+0000","last_active":"2024-04-25T11:21:38.859901+0000","last_peered":"2024-04-25T11:21:38.859901+0000","last_clean":"2024-04-25T11:21:38.859901+0000","last_became_active":"2024-04-25T11:21:37.856254+0000","last_became_peered":"2024-04-25T11:21:37.856254+0000","last_unstale":"2024-04-25T11:21:38.859901+0000","last_undegraded":"2024-04-25T11:21:38.859901+0000","last_fullsized":"2024-04-25T11:21:38.859901+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:36.700564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511344+0000","last_change":"2024-04-25T11:21:37.873642+0000","last_active":"2024-04-25T11:21:39.511344+0000","last_peered":"2024-04-25T11:21:39.511344+0000","last_clean":"2024-04-25T11:21:39.511344+0000","last_became_active":"2024-04-25T11:21:37.873276+0000","last_became_peered":"2024-04-25T11:21:37.873276+0000","last_unstale":"2024-04-25T11:21:39.511344+0000","last_undegraded":"2024-04-25T11:21:39.511344+0000","last_fullsized":"2024-04-25T11:21:39.511344+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:25.417788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.864172+0000","last_change":"2024-04-25T11:21:37.860637+0000","last_active":"2024-04-25T11:21:38.864172+0000","last_peered":"2024-04-25T11:21:38.864172+0000","last_clean":"2024-04-25T11:21:38.864172+0000","last_became_active":"2024-04-25T11:21:37.860266+0000","last_became_peered":"2024-04-25T11:21:37.860266+0000","last_unstale":"2024-04-25T11:21:38.864172+0000","last_undegraded":"2024-04-25T11:21:38.864172+0000","last_fullsized":"2024-04-25T11:21:38.864172+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:06.290875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.861613+0000","last_change":"2024-04-25T11:21:37.853524+0000","last_active":"2024-04-25T11:21:37.861613+0000","last_peered":"2024-04-25T11:21:37.861613+0000","last_clean":"2024-04-25T11:21:37.861613+0000","last_became_active":"2024-04-25T11:21:37.853100+0000","last_became_peered":"2024-04-25T11:21:37.853100+0000","last_unstale":"2024-04-25T11:21:37.861613+0000","last_undegraded":"2024-04-25T11:21:37.861613+0000","last_fullsized":"2024-04-25T11:21:37.861613+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:22.202712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.867307+0000","last_change":"2024-04-25T11:21:37.868036+0000","last_active":"2024-04-25T11:21:38.867307+0000","last_peered":"2024-04-25T11:21:38.867307+0000","last_clean":"2024-04-25T11:21:38.867307+0000","last_became_active":"2024-04-25T11:21:37.867747+0000","last_became_peered":"2024-04-25T11:21:37.867747+0000","last_unstale":"2024-04-25T11:21:38.867307+0000","last_undegraded":"2024-04-25T11:21:38.867307+0000","last_fullsized":"2024-04-25T11:21:38.867307+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:15.858707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510070+0000","last_change":"2024-04-25T11:21:37.856361+0000","last_active":"2024-04-25T11:21:39.510070+0000","last_peered":"2024-04-25T11:21:39.510070+0000","last_clean":"2024-04-25T11:21:39.510070+0000","last_became_active":"2024-04-25T11:21:37.855956+0000","last_became_peered":"2024-04-25T11:21:37.855956+0000","last_unstale":"2024-04-25T11:21:39.510070+0000","last_undegraded":"2024-04-25T11:21:39.510070+0000","last_fullsized":"2024-04-25T11:21:39.510070+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:22.520785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511637+0000","last_change":"2024-04-25T11:21:37.868213+0000","last_active":"2024-04-25T11:21:39.511637+0000","last_peered":"2024-04-25T11:21:39.511637+0000","last_clean":"2024-04-25T11:21:39.511637+0000","last_became_active":"2024-04-25T11:21:37.867467+0000","last_became_peered":"2024-04-25T11:21:37.867467+0000","last_unstale":"2024-04-25T11:21:39.511637+0000","last_undegraded":"2024-04-25T11:21:39.511637+0000","last_fullsized":"2024-04-25T11:21:39.511637+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:16.677903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879329+0000","last_change":"2024-04-25T11:21:37.848122+0000","last_active":"2024-04-25T11:21:38.879329+0000","last_peered":"2024-04-25T11:21:38.879329+0000","last_clean":"2024-04-25T11:21:38.879329+0000","last_became_active":"2024-04-25T11:21:37.847741+0000","last_became_peered":"2024-04-25T11:21:37.847741+0000","last_unstale":"2024-04-25T11:21:38.879329+0000","last_undegraded":"2024-04-25T11:21:38.879329+0000","last_fullsized":"2024-04-25T11:21:38.879329+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:16.279883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510442+0000","last_change":"2024-04-25T11:21:37.866220+0000","last_active":"2024-04-25T11:21:39.510442+0000","last_peered":"2024-04-25T11:21:39.510442+0000","last_clean":"2024-04-25T11:21:39.510442+0000","last_became_active":"2024-04-25T11:21:37.865840+0000","last_became_peered":"2024-04-25T11:21:37.865840+0000","last_unstale":"2024-04-25T11:21:39.510442+0000","last_undegraded":"2024-04-25T11:21:39.510442+0000","last_fullsized":"2024-04-25T11:21:39.510442+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:39.233435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.870701+0000","last_change":"2024-04-25T11:21:37.868982+0000","last_active":"2024-04-25T11:21:37.870701+0000","last_peered":"2024-04-25T11:21:37.870701+0000","last_clean":"2024-04-25T11:21:37.870701+0000","last_became_active":"2024-04-25T11:21:37.868100+0000","last_became_peered":"2024-04-25T11:21:37.868100+0000","last_unstale":"2024-04-25T11:21:37.870701+0000","last_undegraded":"2024-04-25T11:21:37.870701+0000","last_fullsized":"2024-04-25T11:21:37.870701+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:49.628923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880233+0000","last_change":"2024-04-25T11:21:37.850688+0000","last_active":"2024-04-25T11:21:38.880233+0000","last_peered":"2024-04-25T11:21:38.880233+0000","last_clean":"2024-04-25T11:21:38.880233+0000","last_became_active":"2024-04-25T11:21:37.850350+0000","last_became_peered":"2024-04-25T11:21:37.850350+0000","last_unstale":"2024-04-25T11:21:38.880233+0000","last_undegraded":"2024-04-25T11:21:38.880233+0000","last_fullsized":"2024-04-25T11:21:38.880233+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:06.615128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880169+0000","last_change":"2024-04-25T11:21:37.853356+0000","last_active":"2024-04-25T11:21:38.880169+0000","last_peered":"2024-04-25T11:21:38.880169+0000","last_clean":"2024-04-25T11:21:38.880169+0000","last_became_active":"2024-04-25T11:21:37.852930+0000","last_became_peered":"2024-04-25T11:21:37.852930+0000","last_unstale":"2024-04-25T11:21:38.880169+0000","last_undegraded":"2024-04-25T11:21:38.880169+0000","last_fullsized":"2024-04-25T11:21:38.880169+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:34.967219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511451+0000","last_change":"2024-04-25T11:21:37.873935+0000","last_active":"2024-04-25T11:21:39.511451+0000","last_peered":"2024-04-25T11:21:39.511451+0000","last_clean":"2024-04-25T11:21:39.511451+0000","last_became_active":"2024-04-25T11:21:37.867492+0000","last_became_peered":"2024-04-25T11:21:37.867492+0000","last_unstale":"2024-04-25T11:21:39.511451+0000","last_undegraded":"2024-04-25T11:21:39.511451+0000","last_fullsized":"2024-04-25T11:21:39.511451+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:42.548022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.864241+0000","last_change":"2024-04-25T11:21:37.860373+0000","last_active":"2024-04-25T11:21:37.864241+0000","last_peered":"2024-04-25T11:21:37.864241+0000","last_clean":"2024-04-25T11:21:37.864241+0000","last_became_active":"2024-04-25T11:21:37.859465+0000","last_became_peered":"2024-04-25T11:21:37.859465+0000","last_unstale":"2024-04-25T11:21:37.864241+0000","last_undegraded":"2024-04-25T11:21:37.864241+0000","last_fullsized":"2024-04-25T11:21:37.864241+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:41.955144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511619+0000","last_change":"2024-04-25T11:21:37.875010+0000","last_active":"2024-04-25T11:21:39.511619+0000","last_peered":"2024-04-25T11:21:39.511619+0000","last_clean":"2024-04-25T11:21:39.511619+0000","last_became_active":"2024-04-25T11:21:37.874146+0000","last_became_peered":"2024-04-25T11:21:37.874146+0000","last_unstale":"2024-04-25T11:21:39.511619+0000","last_undegraded":"2024-04-25T11:21:39.511619+0000","last_fullsized":"2024-04-25T11:21:39.511619+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:56.149851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.861632+0000","last_change":"2024-04-25T11:21:37.856141+0000","last_active":"2024-04-25T11:21:37.861632+0000","last_peered":"2024-04-25T11:21:37.861632+0000","last_clean":"2024-04-25T11:21:37.861632+0000","last_became_active":"2024-04-25T11:21:37.855478+0000","last_became_peered":"2024-04-25T11:21:37.855478+0000","last_unstale":"2024-04-25T11:21:37.861632+0000","last_undegraded":"2024-04-25T11:21:37.861632+0000","last_fullsized":"2024-04-25T11:21:37.861632+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:15.784485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511224+0000","last_change":"2024-04-25T11:21:37.872419+0000","last_active":"2024-04-25T11:21:39.511224+0000","last_peered":"2024-04-25T11:21:39.511224+0000","last_clean":"2024-04-25T11:21:39.511224+0000","last_became_active":"2024-04-25T11:21:37.872066+0000","last_became_peered":"2024-04-25T11:21:37.872066+0000","last_unstale":"2024-04-25T11:21:39.511224+0000","last_undegraded":"2024-04-25T11:21:39.511224+0000","last_fullsized":"2024-04-25T11:21:39.511224+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:15.858707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510543+0000","last_change":"2024-04-25T11:21:37.864666+0000","last_active":"2024-04-25T11:21:39.510543+0000","last_peered":"2024-04-25T11:21:39.510543+0000","last_clean":"2024-04-25T11:21:39.510543+0000","last_became_active":"2024-04-25T11:21:37.864343+0000","last_became_peered":"2024-04-25T11:21:37.864343+0000","last_unstale":"2024-04-25T11:21:39.510543+0000","last_undegraded":"2024-04-25T11:21:39.510543+0000","last_fullsized":"2024-04-25T11:21:39.510543+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:29.425446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859938+0000","last_change":"2024-04-25T11:21:37.857857+0000","last_active":"2024-04-25T11:21:38.859938+0000","last_peered":"2024-04-25T11:21:38.859938+0000","last_clean":"2024-04-25T11:21:38.859938+0000","last_became_active":"2024-04-25T11:21:37.855522+0000","last_became_peered":"2024-04-25T11:21:37.855522+0000","last_unstale":"2024-04-25T11:21:38.859938+0000","last_undegraded":"2024-04-25T11:21:38.859938+0000","last_fullsized":"2024-04-25T11:21:38.859938+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:57.353175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"141'4","reported_seq":18,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512071+0000","last_change":"2024-04-25T11:21:36.836219+0000","last_active":"2024-04-25T11:21:39.512071+0000","last_peered":"2024-04-25T11:21:39.512071+0000","last_clean":"2024-04-25T11:21:39.512071+0000","last_became_active":"2024-04-25T11:21:36.835875+0000","last_became_peered":"2024-04-25T11:21:36.835875+0000","last_unstale":"2024-04-25T11:21:39.512071+0000","last_undegraded":"2024-04-25T11:21:39.512071+0000","last_fullsized":"2024-04-25T11:21:39.512071+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:58.378850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860063+0000","last_change":"2024-04-25T11:21:37.857381+0000","last_active":"2024-04-25T11:21:38.860063+0000","last_peered":"2024-04-25T11:21:38.860063+0000","last_clean":"2024-04-25T11:21:38.860063+0000","last_became_active":"2024-04-25T11:21:37.856799+0000","last_became_peered":"2024-04-25T11:21:37.856799+0000","last_unstale":"2024-04-25T11:21:38.860063+0000","last_undegraded":"2024-04-25T11:21:38.860063+0000","last_fullsized":"2024-04-25T11:21:38.860063+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:31.934675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866436+0000","last_change":"2024-04-25T11:21:36.831101+0000","last_active":"2024-04-25T11:21:38.866436+0000","last_peered":"2024-04-25T11:21:38.866436+0000","last_clean":"2024-04-25T11:21:38.866436+0000","last_became_active":"2024-04-25T11:21:36.830755+0000","last_became_peered":"2024-04-25T11:21:36.830755+0000","last_unstale":"2024-04-25T11:21:38.866436+0000","last_undegraded":"2024-04-25T11:21:38.866436+0000","last_fullsized":"2024-04-25T11:21:38.866436+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:58.378850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511720+0000","last_change":"2024-04-25T11:21:37.866862+0000","last_active":"2024-04-25T11:21:39.511720+0000","last_peered":"2024-04-25T11:21:39.511720+0000","last_clean":"2024-04-25T11:21:39.511720+0000","last_became_active":"2024-04-25T11:21:37.865283+0000","last_became_peered":"2024-04-25T11:21:37.865283+0000","last_unstale":"2024-04-25T11:21:39.511720+0000","last_undegraded":"2024-04-25T11:21:39.511720+0000","last_fullsized":"2024-04-25T11:21:39.511720+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:33.202111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.867405+0000","last_change":"2024-04-25T11:21:36.831197+0000","last_active":"2024-04-25T11:21:38.867405+0000","last_peered":"2024-04-25T11:21:38.867405+0000","last_clean":"2024-04-25T11:21:38.867405+0000","last_became_active":"2024-04-25T11:21:36.830442+0000","last_became_peered":"2024-04-25T11:21:36.830442+0000","last_unstale":"2024-04-25T11:21:38.867405+0000","last_undegraded":"2024-04-25T11:21:38.867405+0000","last_fullsized":"2024-04-25T11:21:38.867405+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:00.825842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866674+0000","last_change":"2024-04-25T11:21:37.860630+0000","last_active":"2024-04-25T11:21:38.866674+0000","last_peered":"2024-04-25T11:21:38.866674+0000","last_clean":"2024-04-25T11:21:38.866674+0000","last_became_active":"2024-04-25T11:21:37.859711+0000","last_became_peered":"2024-04-25T11:21:37.859711+0000","last_unstale":"2024-04-25T11:21:38.866674+0000","last_undegraded":"2024-04-25T11:21:38.866674+0000","last_fullsized":"2024-04-25T11:21:38.866674+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:20.253487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860063+0000","last_change":"2024-04-25T11:21:36.837424+0000","last_active":"2024-04-25T11:21:38.860063+0000","last_peered":"2024-04-25T11:21:38.860063+0000","last_clean":"2024-04-25T11:21:38.860063+0000","last_became_active":"2024-04-25T11:21:36.836920+0000","last_became_peered":"2024-04-25T11:21:36.836920+0000","last_unstale":"2024-04-25T11:21:38.860063+0000","last_undegraded":"2024-04-25T11:21:38.860063+0000","last_fullsized":"2024-04-25T11:21:38.860063+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:22.263510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510361+0000","last_change":"2024-04-25T11:21:37.865789+0000","last_active":"2024-04-25T11:21:39.510361+0000","last_peered":"2024-04-25T11:21:39.510361+0000","last_clean":"2024-04-25T11:21:39.510361+0000","last_became_active":"2024-04-25T11:21:37.865440+0000","last_became_peered":"2024-04-25T11:21:37.865440+0000","last_unstale":"2024-04-25T11:21:39.510361+0000","last_undegraded":"2024-04-25T11:21:39.510361+0000","last_fullsized":"2024-04-25T11:21:39.510361+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:16.677903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511531+0000","last_change":"2024-04-25T11:21:36.832331+0000","last_active":"2024-04-25T11:21:39.511531+0000","last_peered":"2024-04-25T11:21:39.511531+0000","last_clean":"2024-04-25T11:21:39.511531+0000","last_became_active":"2024-04-25T11:21:36.831955+0000","last_became_peered":"2024-04-25T11:21:36.831955+0000","last_unstale":"2024-04-25T11:21:39.511531+0000","last_undegraded":"2024-04-25T11:21:39.511531+0000","last_fullsized":"2024-04-25T11:21:39.511531+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:38.907256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.862803+0000","last_change":"2024-04-25T11:21:37.853591+0000","last_active":"2024-04-25T11:21:38.862803+0000","last_peered":"2024-04-25T11:21:38.862803+0000","last_clean":"2024-04-25T11:21:38.862803+0000","last_became_active":"2024-04-25T11:21:37.852133+0000","last_became_peered":"2024-04-25T11:21:37.852133+0000","last_unstale":"2024-04-25T11:21:38.862803+0000","last_undegraded":"2024-04-25T11:21:38.862803+0000","last_fullsized":"2024-04-25T11:21:38.862803+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:23.282068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512054+0000","last_change":"2024-04-25T11:21:36.832162+0000","last_active":"2024-04-25T11:21:39.512054+0000","last_peered":"2024-04-25T11:21:39.512054+0000","last_clean":"2024-04-25T11:21:39.512054+0000","last_became_active":"2024-04-25T11:21:36.831788+0000","last_became_peered":"2024-04-25T11:21:36.831788+0000","last_unstale":"2024-04-25T11:21:39.512054+0000","last_undegraded":"2024-04-25T11:21:39.512054+0000","last_fullsized":"2024-04-25T11:21:39.512054+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:50.617654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859709+0000","last_change":"2024-04-25T11:21:37.850475+0000","last_active":"2024-04-25T11:21:38.859709+0000","last_peered":"2024-04-25T11:21:38.859709+0000","last_clean":"2024-04-25T11:21:38.859709+0000","last_became_active":"2024-04-25T11:21:37.849891+0000","last_became_peered":"2024-04-25T11:21:37.849891+0000","last_unstale":"2024-04-25T11:21:38.859709+0000","last_undegraded":"2024-04-25T11:21:38.859709+0000","last_fullsized":"2024-04-25T11:21:38.859709+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:42.312337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879760+0000","last_change":"2024-04-25T11:21:36.831876+0000","last_active":"2024-04-25T11:21:38.879760+0000","last_peered":"2024-04-25T11:21:38.879760+0000","last_clean":"2024-04-25T11:21:38.879760+0000","last_became_active":"2024-04-25T11:21:36.831526+0000","last_became_peered":"2024-04-25T11:21:36.831526+0000","last_unstale":"2024-04-25T11:21:38.879760+0000","last_undegraded":"2024-04-25T11:21:38.879760+0000","last_fullsized":"2024-04-25T11:21:38.879760+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:37.812097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879343+0000","last_change":"2024-04-25T11:21:37.860454+0000","last_active":"2024-04-25T11:21:38.879343+0000","last_peered":"2024-04-25T11:21:38.879343+0000","last_clean":"2024-04-25T11:21:38.879343+0000","last_became_active":"2024-04-25T11:21:37.860104+0000","last_became_peered":"2024-04-25T11:21:37.860104+0000","last_unstale":"2024-04-25T11:21:38.879343+0000","last_undegraded":"2024-04-25T11:21:38.879343+0000","last_fullsized":"2024-04-25T11:21:38.879343+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:35.030082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511407+0000","last_change":"2024-04-25T11:21:36.833376+0000","last_active":"2024-04-25T11:21:39.511407+0000","last_peered":"2024-04-25T11:21:39.511407+0000","last_clean":"2024-04-25T11:21:39.511407+0000","last_became_active":"2024-04-25T11:21:36.833043+0000","last_became_peered":"2024-04-25T11:21:36.833043+0000","last_unstale":"2024-04-25T11:21:39.511407+0000","last_undegraded":"2024-04-25T11:21:39.511407+0000","last_fullsized":"2024-04-25T11:21:39.511407+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:56.334617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511292+0000","last_change":"2024-04-25T11:21:37.864561+0000","last_active":"2024-04-25T11:21:39.511292+0000","last_peered":"2024-04-25T11:21:39.511292+0000","last_clean":"2024-04-25T11:21:39.511292+0000","last_became_active":"2024-04-25T11:21:37.864168+0000","last_became_peered":"2024-04-25T11:21:37.864168+0000","last_unstale":"2024-04-25T11:21:39.511292+0000","last_undegraded":"2024-04-25T11:21:39.511292+0000","last_fullsized":"2024-04-25T11:21:39.511292+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:23.282068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866827+0000","last_change":"2024-04-25T11:21:36.833895+0000","last_active":"2024-04-25T11:21:38.866827+0000","last_peered":"2024-04-25T11:21:38.866827+0000","last_clean":"2024-04-25T11:21:38.866827+0000","last_became_active":"2024-04-25T11:21:36.832511+0000","last_became_peered":"2024-04-25T11:21:36.832511+0000","last_unstale":"2024-04-25T11:21:38.866827+0000","last_undegraded":"2024-04-25T11:21:38.866827+0000","last_fullsized":"2024-04-25T11:21:38.866827+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:46.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.864271+0000","last_change":"2024-04-25T11:21:37.859787+0000","last_active":"2024-04-25T11:21:37.864271+0000","last_peered":"2024-04-25T11:21:37.864271+0000","last_clean":"2024-04-25T11:21:37.864271+0000","last_became_active":"2024-04-25T11:21:37.858966+0000","last_became_peered":"2024-04-25T11:21:37.858966+0000","last_unstale":"2024-04-25T11:21:37.864271+0000","last_undegraded":"2024-04-25T11:21:37.864271+0000","last_fullsized":"2024-04-25T11:21:37.864271+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:29.069474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880489+0000","last_change":"2024-04-25T11:21:36.820101+0000","last_active":"2024-04-25T11:21:38.880489+0000","last_peered":"2024-04-25T11:21:38.880489+0000","last_clean":"2024-04-25T11:21:38.880489+0000","last_became_active":"2024-04-25T11:21:36.817759+0000","last_became_peered":"2024-04-25T11:21:36.817759+0000","last_unstale":"2024-04-25T11:21:38.880489+0000","last_undegraded":"2024-04-25T11:21:38.880489+0000","last_fullsized":"2024-04-25T11:21:38.880489+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:38.214877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.861018+0000","last_change":"2024-04-25T11:21:37.853434+0000","last_active":"2024-04-25T11:21:37.861018+0000","last_peered":"2024-04-25T11:21:37.861018+0000","last_clean":"2024-04-25T11:21:37.861018+0000","last_became_active":"2024-04-25T11:21:37.852678+0000","last_became_peered":"2024-04-25T11:21:37.852678+0000","last_unstale":"2024-04-25T11:21:37.861018+0000","last_undegraded":"2024-04-25T11:21:37.861018+0000","last_fullsized":"2024-04-25T11:21:37.861018+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:50.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"141'3","reported_seq":15,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.882999+0000","last_change":"2024-04-25T11:21:36.833059+0000","last_active":"2024-04-25T11:21:37.882999+0000","last_peered":"2024-04-25T11:21:37.882999+0000","last_clean":"2024-04-25T11:21:37.882999+0000","last_became_active":"2024-04-25T11:21:36.832460+0000","last_became_peered":"2024-04-25T11:21:36.832460+0000","last_unstale":"2024-04-25T11:21:37.882999+0000","last_undegraded":"2024-04-25T11:21:37.882999+0000","last_fullsized":"2024-04-25T11:21:37.882999+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:49.666663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510951+0000","last_change":"2024-04-25T11:21:37.865250+0000","last_active":"2024-04-25T11:21:39.510951+0000","last_peered":"2024-04-25T11:21:39.510951+0000","last_clean":"2024-04-25T11:21:39.510951+0000","last_became_active":"2024-04-25T11:21:37.863769+0000","last_became_peered":"2024-04-25T11:21:37.863769+0000","last_unstale":"2024-04-25T11:21:39.510951+0000","last_undegraded":"2024-04-25T11:21:39.510951+0000","last_fullsized":"2024-04-25T11:21:39.510951+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:05.096282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511730+0000","last_change":"2024-04-25T11:21:36.833841+0000","last_active":"2024-04-25T11:21:39.511730+0000","last_peered":"2024-04-25T11:21:39.511730+0000","last_clean":"2024-04-25T11:21:39.511730+0000","last_became_active":"2024-04-25T11:21:36.833478+0000","last_became_peered":"2024-04-25T11:21:36.833478+0000","last_unstale":"2024-04-25T11:21:39.511730+0000","last_undegraded":"2024-04-25T11:21:39.511730+0000","last_fullsized":"2024-04-25T11:21:39.511730+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:30.916117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860336+0000","last_change":"2024-04-25T11:21:37.855990+0000","last_active":"2024-04-25T11:21:38.860336+0000","last_peered":"2024-04-25T11:21:38.860336+0000","last_clean":"2024-04-25T11:21:38.860336+0000","last_became_active":"2024-04-25T11:21:37.854789+0000","last_became_peered":"2024-04-25T11:21:37.854789+0000","last_unstale":"2024-04-25T11:21:38.860336+0000","last_undegraded":"2024-04-25T11:21:38.860336+0000","last_fullsized":"2024-04-25T11:21:38.860336+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:51.636211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.862868+0000","last_change":"2024-04-25T11:21:36.834158+0000","last_active":"2024-04-25T11:21:38.862868+0000","last_peered":"2024-04-25T11:21:38.862868+0000","last_clean":"2024-04-25T11:21:38.862868+0000","last_became_active":"2024-04-25T11:21:36.833778+0000","last_became_peered":"2024-04-25T11:21:36.833778+0000","last_unstale":"2024-04-25T11:21:38.862868+0000","last_undegraded":"2024-04-25T11:21:38.862868+0000","last_fullsized":"2024-04-25T11:21:38.862868+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:39.711941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511875+0000","last_change":"2024-04-25T11:21:37.868534+0000","last_active":"2024-04-25T11:21:39.511875+0000","last_peered":"2024-04-25T11:21:39.511875+0000","last_clean":"2024-04-25T11:21:39.511875+0000","last_became_active":"2024-04-25T11:21:37.868139+0000","last_became_peered":"2024-04-25T11:21:37.868139+0000","last_unstale":"2024-04-25T11:21:39.511875+0000","last_undegraded":"2024-04-25T11:21:39.511875+0000","last_fullsized":"2024-04-25T11:21:39.511875+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:12.376728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.862958+0000","last_change":"2024-04-25T11:21:36.825262+0000","last_active":"2024-04-25T11:21:38.862958+0000","last_peered":"2024-04-25T11:21:38.862958+0000","last_clean":"2024-04-25T11:21:38.862958+0000","last_became_active":"2024-04-25T11:21:36.824311+0000","last_became_peered":"2024-04-25T11:21:36.824311+0000","last_unstale":"2024-04-25T11:21:38.862958+0000","last_undegraded":"2024-04-25T11:21:38.862958+0000","last_fullsized":"2024-04-25T11:21:38.862958+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:45:31.001996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859407+0000","last_change":"2024-04-25T11:21:37.875452+0000","last_active":"2024-04-25T11:21:38.859407+0000","last_peered":"2024-04-25T11:21:38.859407+0000","last_clean":"2024-04-25T11:21:38.859407+0000","last_became_active":"2024-04-25T11:21:37.874912+0000","last_became_peered":"2024-04-25T11:21:37.874912+0000","last_unstale":"2024-04-25T11:21:38.859407+0000","last_undegraded":"2024-04-25T11:21:38.859407+0000","last_fullsized":"2024-04-25T11:21:38.859407+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:37.904299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"141'8","reported_seq":22,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.878916+0000","last_change":"2024-04-25T11:21:36.821279+0000","last_active":"2024-04-25T11:21:38.878916+0000","last_peered":"2024-04-25T11:21:38.878916+0000","last_clean":"2024-04-25T11:21:38.878916+0000","last_became_active":"2024-04-25T11:21:36.820900+0000","last_became_peered":"2024-04-25T11:21:36.820900+0000","last_unstale":"2024-04-25T11:21:38.878916+0000","last_undegraded":"2024-04-25T11:21:38.878916+0000","last_fullsized":"2024-04-25T11:21:38.878916+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:39.711941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866447+0000","last_change":"2024-04-25T11:21:37.879517+0000","last_active":"2024-04-25T11:21:38.866447+0000","last_peered":"2024-04-25T11:21:38.866447+0000","last_clean":"2024-04-25T11:21:38.866447+0000","last_became_active":"2024-04-25T11:21:37.879190+0000","last_became_peered":"2024-04-25T11:21:37.879190+0000","last_unstale":"2024-04-25T11:21:38.866447+0000","last_undegraded":"2024-04-25T11:21:38.866447+0000","last_fullsized":"2024-04-25T11:21:38.866447+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:45.924348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"141'1","reported_seq":13,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.873876+0000","last_change":"2024-04-25T11:21:36.825400+0000","last_active":"2024-04-25T11:21:37.873876+0000","last_peered":"2024-04-25T11:21:37.873876+0000","last_clean":"2024-04-25T11:21:37.873876+0000","last_became_active":"2024-04-25T11:21:36.824827+0000","last_became_peered":"2024-04-25T11:21:36.824827+0000","last_unstale":"2024-04-25T11:21:37.873876+0000","last_undegraded":"2024-04-25T11:21:37.873876+0000","last_fullsized":"2024-04-25T11:21:37.873876+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:18.626686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.867309+0000","last_change":"2024-04-25T11:21:37.865043+0000","last_active":"2024-04-25T11:21:38.867309+0000","last_peered":"2024-04-25T11:21:38.867309+0000","last_clean":"2024-04-25T11:21:38.867309+0000","last_became_active":"2024-04-25T11:21:37.863907+0000","last_became_peered":"2024-04-25T11:21:37.863907+0000","last_unstale":"2024-04-25T11:21:38.867309+0000","last_undegraded":"2024-04-25T11:21:38.867309+0000","last_fullsized":"2024-04-25T11:21:38.867309+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:25.417788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866680+0000","last_change":"2024-04-25T11:21:36.833386+0000","last_active":"2024-04-25T11:21:38.866680+0000","last_peered":"2024-04-25T11:21:38.866680+0000","last_clean":"2024-04-25T11:21:38.866680+0000","last_became_active":"2024-04-25T11:21:36.833055+0000","last_became_peered":"2024-04-25T11:21:36.833055+0000","last_unstale":"2024-04-25T11:21:38.866680+0000","last_undegraded":"2024-04-25T11:21:38.866680+0000","last_fullsized":"2024-04-25T11:21:38.866680+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:46.031381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880393+0000","last_change":"2024-04-25T11:21:37.857002+0000","last_active":"2024-04-25T11:21:38.880393+0000","last_peered":"2024-04-25T11:21:38.880393+0000","last_clean":"2024-04-25T11:21:38.880393+0000","last_became_active":"2024-04-25T11:21:37.856155+0000","last_became_peered":"2024-04-25T11:21:37.856155+0000","last_unstale":"2024-04-25T11:21:38.880393+0000","last_undegraded":"2024-04-25T11:21:38.880393+0000","last_fullsized":"2024-04-25T11:21:38.880393+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:37.904299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"141'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860202+0000","last_change":"2024-04-25T11:21:36.829548+0000","last_active":"2024-04-25T11:21:38.860202+0000","last_peered":"2024-04-25T11:21:38.860202+0000","last_clean":"2024-04-25T11:21:38.860202+0000","last_became_active":"2024-04-25T11:21:36.829117+0000","last_became_peered":"2024-04-25T11:21:36.829117+0000","last_unstale":"2024-04-25T11:21:38.860202+0000","last_undegraded":"2024-04-25T11:21:38.860202+0000","last_fullsized":"2024-04-25T11:21:38.860202+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:19.372757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880460+0000","last_change":"2024-04-25T11:21:37.854334+0000","last_active":"2024-04-25T11:21:38.880460+0000","last_peered":"2024-04-25T11:21:38.880460+0000","last_clean":"2024-04-25T11:21:38.880460+0000","last_became_active":"2024-04-25T11:21:37.854007+0000","last_became_peered":"2024-04-25T11:21:37.854007+0000","last_unstale":"2024-04-25T11:21:38.880460+0000","last_undegraded":"2024-04-25T11:21:38.880460+0000","last_fullsized":"2024-04-25T11:21:38.880460+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:25.261420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863628+0000","last_change":"2024-04-25T11:21:36.835849+0000","last_active":"2024-04-25T11:21:38.863628+0000","last_peered":"2024-04-25T11:21:38.863628+0000","last_clean":"2024-04-25T11:21:38.863628+0000","last_became_active":"2024-04-25T11:21:36.835478+0000","last_became_peered":"2024-04-25T11:21:36.835478+0000","last_unstale":"2024-04-25T11:21:38.863628+0000","last_undegraded":"2024-04-25T11:21:38.863628+0000","last_fullsized":"2024-04-25T11:21:38.863628+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:33.381133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.865716+0000","last_change":"2024-04-25T11:21:37.865043+0000","last_active":"2024-04-25T11:21:38.865716+0000","last_peered":"2024-04-25T11:21:38.865716+0000","last_clean":"2024-04-25T11:21:38.865716+0000","last_became_active":"2024-04-25T11:21:37.864015+0000","last_became_peered":"2024-04-25T11:21:37.864015+0000","last_unstale":"2024-04-25T11:21:38.865716+0000","last_undegraded":"2024-04-25T11:21:38.865716+0000","last_fullsized":"2024-04-25T11:21:38.865716+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:27.727832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.509973+0000","last_change":"2024-04-25T11:21:36.839291+0000","last_active":"2024-04-25T11:21:39.509973+0000","last_peered":"2024-04-25T11:21:39.509973+0000","last_clean":"2024-04-25T11:21:39.509973+0000","last_became_active":"2024-04-25T11:21:36.838114+0000","last_became_peered":"2024-04-25T11:21:36.838114+0000","last_unstale":"2024-04-25T11:21:39.509973+0000","last_undegraded":"2024-04-25T11:21:39.509973+0000","last_fullsized":"2024-04-25T11:21:39.509973+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:14.840149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860458+0000","last_change":"2024-04-25T11:21:37.857105+0000","last_active":"2024-04-25T11:21:38.860458+0000","last_peered":"2024-04-25T11:21:38.860458+0000","last_clean":"2024-04-25T11:21:38.860458+0000","last_became_active":"2024-04-25T11:21:37.856711+0000","last_became_peered":"2024-04-25T11:21:37.856711+0000","last_unstale":"2024-04-25T11:21:38.860458+0000","last_undegraded":"2024-04-25T11:21:38.860458+0000","last_fullsized":"2024-04-25T11:21:38.860458+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:59.397407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510361+0000","last_change":"2024-04-25T11:21:36.837206+0000","last_active":"2024-04-25T11:21:39.510361+0000","last_peered":"2024-04-25T11:21:39.510361+0000","last_clean":"2024-04-25T11:21:39.510361+0000","last_became_active":"2024-04-25T11:21:36.836772+0000","last_became_peered":"2024-04-25T11:21:36.836772+0000","last_unstale":"2024-04-25T11:21:39.510361+0000","last_undegraded":"2024-04-25T11:21:39.510361+0000","last_fullsized":"2024-04-25T11:21:39.510361+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:06.399013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.866348+0000","last_change":"2024-04-25T11:21:37.856653+0000","last_active":"2024-04-25T11:21:38.866348+0000","last_peered":"2024-04-25T11:21:38.866348+0000","last_clean":"2024-04-25T11:21:38.866348+0000","last_became_active":"2024-04-25T11:21:37.856343+0000","last_became_peered":"2024-04-25T11:21:37.856343+0000","last_unstale":"2024-04-25T11:21:38.866348+0000","last_undegraded":"2024-04-25T11:21:38.866348+0000","last_fullsized":"2024-04-25T11:21:38.866348+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:46.354275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.509610+0000","last_change":"2024-04-25T11:21:36.840531+0000","last_active":"2024-04-25T11:21:39.509610+0000","last_peered":"2024-04-25T11:21:39.509610+0000","last_clean":"2024-04-25T11:21:39.509610+0000","last_became_active":"2024-04-25T11:21:36.839923+0000","last_became_peered":"2024-04-25T11:21:36.839923+0000","last_unstale":"2024-04-25T11:21:39.509610+0000","last_undegraded":"2024-04-25T11:21:39.509610+0000","last_fullsized":"2024-04-25T11:21:39.509610+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:19.234930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859620+0000","last_change":"2024-04-25T11:21:37.865695+0000","last_active":"2024-04-25T11:21:38.859620+0000","last_peered":"2024-04-25T11:21:38.859620+0000","last_clean":"2024-04-25T11:21:38.859620+0000","last_became_active":"2024-04-25T11:21:37.864120+0000","last_became_peered":"2024-04-25T11:21:37.864120+0000","last_unstale":"2024-04-25T11:21:38.859620+0000","last_undegraded":"2024-04-25T11:21:38.859620+0000","last_fullsized":"2024-04-25T11:21:38.859620+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:25.261420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511144+0000","last_change":"2024-04-25T11:21:36.839357+0000","last_active":"2024-04-25T11:21:39.511144+0000","last_peered":"2024-04-25T11:21:39.511144+0000","last_clean":"2024-04-25T11:21:39.511144+0000","last_became_active":"2024-04-25T11:21:36.838813+0000","last_became_peered":"2024-04-25T11:21:36.838813+0000","last_unstale":"2024-04-25T11:21:39.511144+0000","last_undegraded":"2024-04-25T11:21:39.511144+0000","last_fullsized":"2024-04-25T11:21:39.511144+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:39.782765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512150+0000","last_change":"2024-04-25T11:21:37.857611+0000","last_active":"2024-04-25T11:21:39.512150+0000","last_peered":"2024-04-25T11:21:39.512150+0000","last_clean":"2024-04-25T11:21:39.512150+0000","last_became_active":"2024-04-25T11:21:37.857121+0000","last_became_peered":"2024-04-25T11:21:37.857121+0000","last_unstale":"2024-04-25T11:21:39.512150+0000","last_undegraded":"2024-04-25T11:21:39.512150+0000","last_fullsized":"2024-04-25T11:21:39.512150+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:08.443311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"141'2","reported_seq":16,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863854+0000","last_change":"2024-04-25T11:21:36.838832+0000","last_active":"2024-04-25T11:21:38.863854+0000","last_peered":"2024-04-25T11:21:38.863854+0000","last_clean":"2024-04-25T11:21:38.863854+0000","last_became_active":"2024-04-25T11:21:36.838251+0000","last_became_peered":"2024-04-25T11:21:36.838251+0000","last_unstale":"2024-04-25T11:21:38.863854+0000","last_undegraded":"2024-04-25T11:21:38.863854+0000","last_fullsized":"2024-04-25T11:21:38.863854+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:37.812097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.864131+0000","last_change":"2024-04-25T11:21:37.878247+0000","last_active":"2024-04-25T11:21:38.864131+0000","last_peered":"2024-04-25T11:21:38.864131+0000","last_clean":"2024-04-25T11:21:38.864131+0000","last_became_active":"2024-04-25T11:21:37.877612+0000","last_became_peered":"2024-04-25T11:21:37.877612+0000","last_unstale":"2024-04-25T11:21:38.864131+0000","last_undegraded":"2024-04-25T11:21:38.864131+0000","last_fullsized":"2024-04-25T11:21:38.864131+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:29.312058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"142'1","reported_seq":15,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880609+0000","last_change":"2024-04-25T11:21:36.827403+0000","last_active":"2024-04-25T11:21:38.880609+0000","last_peered":"2024-04-25T11:21:38.880609+0000","last_clean":"2024-04-25T11:21:38.880609+0000","last_became_active":"2024-04-25T11:21:36.826964+0000","last_became_peered":"2024-04-25T11:21:36.826964+0000","last_unstale":"2024-04-25T11:21:38.880609+0000","last_undegraded":"2024-04-25T11:21:38.880609+0000","last_fullsized":"2024-04-25T11:21:38.880609+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:35.825822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.880443+0000","last_change":"2024-04-25T11:21:37.862311+0000","last_active":"2024-04-25T11:21:38.880443+0000","last_peered":"2024-04-25T11:21:38.880443+0000","last_clean":"2024-04-25T11:21:38.880443+0000","last_became_active":"2024-04-25T11:21:37.860211+0000","last_became_peered":"2024-04-25T11:21:37.860211+0000","last_unstale":"2024-04-25T11:21:38.880443+0000","last_undegraded":"2024-04-25T11:21:38.880443+0000","last_fullsized":"2024-04-25T11:21:38.880443+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:05.598346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.860244+0000","last_change":"2024-04-25T11:21:36.839486+0000","last_active":"2024-04-25T11:21:38.860244+0000","last_peered":"2024-04-25T11:21:38.860244+0000","last_clean":"2024-04-25T11:21:38.860244+0000","last_became_active":"2024-04-25T11:21:36.839123+0000","last_became_peered":"2024-04-25T11:21:36.839123+0000","last_unstale":"2024-04-25T11:21:38.860244+0000","last_undegraded":"2024-04-25T11:21:38.860244+0000","last_fullsized":"2024-04-25T11:21:38.860244+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:55.131293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512032+0000","last_change":"2024-04-25T11:21:37.866908+0000","last_active":"2024-04-25T11:21:39.512032+0000","last_peered":"2024-04-25T11:21:39.512032+0000","last_clean":"2024-04-25T11:21:39.512032+0000","last_became_active":"2024-04-25T11:21:37.866577+0000","last_became_peered":"2024-04-25T11:21:37.866577+0000","last_unstale":"2024-04-25T11:21:39.512032+0000","last_undegraded":"2024-04-25T11:21:39.512032+0000","last_fullsized":"2024-04-25T11:21:39.512032+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:00.851705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.512046+0000","last_change":"2024-04-25T11:21:36.839386+0000","last_active":"2024-04-25T11:21:39.512046+0000","last_peered":"2024-04-25T11:21:39.512046+0000","last_clean":"2024-04-25T11:21:39.512046+0000","last_became_active":"2024-04-25T11:21:36.839055+0000","last_became_peered":"2024-04-25T11:21:36.839055+0000","last_unstale":"2024-04-25T11:21:39.512046+0000","last_undegraded":"2024-04-25T11:21:39.512046+0000","last_fullsized":"2024-04-25T11:21:39.512046+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:46.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511753+0000","last_change":"2024-04-25T11:21:37.870052+0000","last_active":"2024-04-25T11:21:39.511753+0000","last_peered":"2024-04-25T11:21:39.511753+0000","last_clean":"2024-04-25T11:21:39.511753+0000","last_became_active":"2024-04-25T11:21:37.869748+0000","last_became_peered":"2024-04-25T11:21:37.869748+0000","last_unstale":"2024-04-25T11:21:39.511753+0000","last_undegraded":"2024-04-25T11:21:39.511753+0000","last_fullsized":"2024-04-25T11:21:39.511753+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:32:56.956131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879327+0000","last_change":"2024-04-25T11:21:36.823622+0000","last_active":"2024-04-25T11:21:38.879327+0000","last_peered":"2024-04-25T11:21:38.879327+0000","last_clean":"2024-04-25T11:21:38.879327+0000","last_became_active":"2024-04-25T11:21:36.823264+0000","last_became_peered":"2024-04-25T11:21:36.823264+0000","last_unstale":"2024-04-25T11:21:38.879327+0000","last_undegraded":"2024-04-25T11:21:38.879327+0000","last_fullsized":"2024-04-25T11:21:38.879327+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:51.482526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.863657+0000","last_change":"2024-04-25T11:21:37.869787+0000","last_active":"2024-04-25T11:21:38.863657+0000","last_peered":"2024-04-25T11:21:38.863657+0000","last_clean":"2024-04-25T11:21:38.863657+0000","last_became_active":"2024-04-25T11:21:37.869359+0000","last_became_peered":"2024-04-25T11:21:37.869359+0000","last_unstale":"2024-04-25T11:21:38.863657+0000","last_undegraded":"2024-04-25T11:21:38.863657+0000","last_fullsized":"2024-04-25T11:21:38.863657+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:33.193064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T11:21:37.848712+0000","last_change":"2024-04-25T11:21:36.827079+0000","last_active":"2024-04-25T11:21:37.848712+0000","last_peered":"2024-04-25T11:21:37.848712+0000","last_clean":"2024-04-25T11:21:37.848712+0000","last_became_active":"2024-04-25T11:21:36.826731+0000","last_became_peered":"2024-04-25T11:21:36.826731+0000","last_unstale":"2024-04-25T11:21:37.848712+0000","last_undegraded":"2024-04-25T11:21:37.848712+0000","last_fullsized":"2024-04-25T11:21:37.848712+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:27.145048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.859899+0000","last_change":"2024-04-25T11:21:37.865367+0000","last_active":"2024-04-25T11:21:38.859899+0000","last_peered":"2024-04-25T11:21:38.859899+0000","last_clean":"2024-04-25T11:21:38.859899+0000","last_became_active":"2024-04-25T11:21:37.865003+0000","last_became_peered":"2024-04-25T11:21:37.865003+0000","last_unstale":"2024-04-25T11:21:38.859899+0000","last_undegraded":"2024-04-25T11:21:38.859899+0000","last_fullsized":"2024-04-25T11:21:38.859899+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:47.692135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.879993+0000","last_change":"2024-04-25T11:21:36.819347+0000","last_active":"2024-04-25T11:21:38.879993+0000","last_peered":"2024-04-25T11:21:38.879993+0000","last_clean":"2024-04-25T11:21:38.879993+0000","last_became_active":"2024-04-25T11:21:36.819000+0000","last_became_peered":"2024-04-25T11:21:36.819000+0000","last_unstale":"2024-04-25T11:21:38.879993+0000","last_undegraded":"2024-04-25T11:21:38.879993+0000","last_fullsized":"2024-04-25T11:21:38.879993+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:31.181592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511160+0000","last_change":"2024-04-25T11:21:37.874813+0000","last_active":"2024-04-25T11:21:39.511160+0000","last_peered":"2024-04-25T11:21:39.511160+0000","last_clean":"2024-04-25T11:21:39.511160+0000","last_became_active":"2024-04-25T11:21:37.874199+0000","last_became_peered":"2024-04-25T11:21:37.874199+0000","last_unstale":"2024-04-25T11:21:39.511160+0000","last_undegraded":"2024-04-25T11:21:39.511160+0000","last_fullsized":"2024-04-25T11:21:39.511160+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:20.391314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.865482+0000","last_change":"2024-04-25T11:21:36.830719+0000","last_active":"2024-04-25T11:21:38.865482+0000","last_peered":"2024-04-25T11:21:38.865482+0000","last_clean":"2024-04-25T11:21:38.865482+0000","last_became_active":"2024-04-25T11:21:36.829900+0000","last_became_peered":"2024-04-25T11:21:36.829900+0000","last_unstale":"2024-04-25T11:21:38.865482+0000","last_undegraded":"2024-04-25T11:21:38.865482+0000","last_fullsized":"2024-04-25T11:21:38.865482+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:50.617654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.511919+0000","last_change":"2024-04-25T11:21:37.867478+0000","last_active":"2024-04-25T11:21:39.511919+0000","last_peered":"2024-04-25T11:21:39.511919+0000","last_clean":"2024-04-25T11:21:39.511919+0000","last_became_active":"2024-04-25T11:21:37.867180+0000","last_became_peered":"2024-04-25T11:21:37.867180+0000","last_unstale":"2024-04-25T11:21:39.511919+0000","last_undegraded":"2024-04-25T11:21:39.511919+0000","last_fullsized":"2024-04-25T11:21:39.511919+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:36.786423+0000","last_clean_scrub_stamp":"2024-04-25T11:21:36.786423+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:33.193064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"141'6","reported_seq":20,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:39.510284+0000","last_change":"2024-04-25T11:21:36.838957+0000","last_active":"2024-04-25T11:21:39.510284+0000","last_peered":"2024-04-25T11:21:39.510284+0000","last_clean":"2024-04-25T11:21:39.510284+0000","last_became_active":"2024-04-25T11:21:36.838625+0000","last_became_peered":"2024-04-25T11:21:36.838625+0000","last_unstale":"2024-04-25T11:21:39.510284+0000","last_undegraded":"2024-04-25T11:21:39.510284+0000","last_fullsized":"2024-04-25T11:21:39.510284+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":139,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:21:35.767865+0000","last_clean_scrub_stamp":"2024-04-25T11:21:35.767865+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:24.399231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":353,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T11:21:38.865849+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:21:38.865849+0000","last_peered":"2024-04-25T11:21:38.865849+0000","last_clean":"2024-04-25T11:21:38.865849+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:21:38.865849+0000","last_undegraded":"2024-04-25T11:21:38.865849+0000","last_fullsized":"2024-04-25T11:21:38.865849+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33496,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338344,"statfs":{"total":96636764160,"available":96602464256,"internally_reserved":0,"allocated":2367488,"data_stored":1409380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476877,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1773568,"data_stored":819060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1769472,"data_stored":818990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476877,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37080,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334760,"statfs":{"total":96636764160,"available":96598794240,"internally_reserved":0,"allocated":1781760,"data_stored":820602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37064,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334776,"statfs":{"total":96636764160,"available":96598810624,"internally_reserved":0,"allocated":1765376,"data_stored":812914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509581,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32916,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338924,"statfs":{"total":96636764160,"available":96603058176,"internally_reserved":0,"allocated":1773568,"data_stored":819060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2387968,"data_stored":1411676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542285,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1765376,"data_stored":812410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:21:43.273 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:21:43.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:21:43.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.581+0000 7fb4a9200640 1 Processor -- start 2024-04-25T11:21:43.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.581+0000 7fb4a9200640 1 -- start start 2024-04-25T11:21:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.584+0000 7fb4a9200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a409f790 0x7fb4a415dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.584+0000 7fb4a9200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4a406dc90 0x7fb4a40ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.584+0000 7fb4a9200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a406f470 con 0x7fb4a4070d60 2024-04-25T11:21:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.584+0000 7fb4a9200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb4a415e2a0 con 0x7fb4a406dc90 2024-04-25T11:21:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.584+0000 7fb4a9200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a4074550 con 0x7fb4a409f790 2024-04-25T11:21:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.584+0000 7fb4a1400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb4a4070d60 0x7fb4a406d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47632/0 (socket says 172.21.15.7:47632) 2024-04-25T11:21:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.584+0000 7fb4a1400640 1 -- 172.21.15.7:0/3171058078 learned_addr learned my addr 172.21.15.7:0/3171058078 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:43.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.585+0000 7fb4a2800640 1 --2- 172.21.15.7:0/3171058078 >> v2:172.21.15.7:3300/0 conn(0x7fb4a406dc90 0x7fb4a40ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:43.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.585+0000 7fb4a1e00640 1 --2- 172.21.15.7:0/3171058078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a409f790 0x7fb4a415dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:43.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.585+0000 7fb4a0a00640 1 -- 172.21.15.7:0/3171058078 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 487646694 0 0) 0x7fb4a406f470 con 0x7fb4a4070d60 2024-04-25T11:21:43.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.585+0000 7fb4a0a00640 1 -- 172.21.15.7:0/3171058078 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb488003520 con 0x7fb4a4070d60 2024-04-25T11:21:43.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.585+0000 7fb4a2800640 1 -- 172.21.15.7:0/3171058078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a409f790 msgr2=0x7fb4a415dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:43.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.585+0000 7fb4a2800640 1 --2- 172.21.15.7:0/3171058078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a409f790 0x7fb4a415dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.585+0000 7fb4a2800640 1 -- 172.21.15.7:0/3171058078 >> v1:172.21.15.7:6789/0 conn(0x7fb4a4070d60 legacy=0x7fb4a406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:43.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.585+0000 7fb4a2800640 1 -- 172.21.15.7:0/3171058078 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a406f750 con 0x7fb4a406dc90 2024-04-25T11:21:43.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.585+0000 7fb4a1e00640 1 --2- 172.21.15.7:0/3171058078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a409f790 0x7fb4a415dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:21:43.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.586+0000 7fb4a2800640 1 --2- 172.21.15.7:0/3171058078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a406dc90 0x7fb4a40ce900 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009d90 tx=0x7fb49c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc54c99753aea142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:43.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.587+0000 7fb4a0a00640 1 -- 172.21.15.7:0/3171058078 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c007740 con 0x7fb4a406dc90 2024-04-25T11:21:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.587+0000 7fb4a0a00640 1 -- 172.21.15.7:0/3171058078 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb49c0078e0 con 0x7fb4a406dc90 2024-04-25T11:21:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.587+0000 7fb4a0a00640 1 -- 172.21.15.7:0/3171058078 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c007cb0 con 0x7fb4a406dc90 2024-04-25T11:21:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.587+0000 7fb4a9200640 1 -- 172.21.15.7:0/3171058078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a406dc90 msgr2=0x7fb4a40ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:43.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.587+0000 7fb4a9200640 1 --2- 172.21.15.7:0/3171058078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a406dc90 0x7fb4a40ce900 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009d90 tx=0x7fb49c02a660 comp rx=0 tx=0).stop 2024-04-25T11:21:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.588+0000 7fb4a9200640 1 -- 172.21.15.7:0/3171058078 shutdown_connections 2024-04-25T11:21:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.588+0000 7fb4a9200640 1 --2- 172.21.15.7:0/3171058078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a406dc90 0x7fb4a40ce900 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.588+0000 7fb4a9200640 1 --2- 172.21.15.7:0/3171058078 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a409f790 0x7fb4a415dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.588+0000 7fb4a9200640 1 -- 172.21.15.7:0/3171058078 >> 172.21.15.7:0/3171058078 conn(0x7fb4a40d1d40 msgr2=0x7fb4a4073220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.588+0000 7fb4a9200640 1 -- 172.21.15.7:0/3171058078 shutdown_connections 2024-04-25T11:21:43.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.588+0000 7fb4a9200640 1 -- 172.21.15.7:0/3171058078 wait complete. 2024-04-25T11:21:43.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.590+0000 7fb4a9200640 1 Processor -- start 2024-04-25T11:21:43.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.590+0000 7fb4a9200640 1 -- start start 2024-04-25T11:21:43.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.592+0000 7fb4a9200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4070d60 0x7fb4a40cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:43.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.592+0000 7fb4a9200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a409f790 0x7fb4a40cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.592+0000 7fb4a9200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a414eb30 0x7fb4a40cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.592+0000 7fb4a9200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a406fd10 con 0x7fb4a409f790 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.592+0000 7fb4a9200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a4070080 con 0x7fb4a4070d60 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.592+0000 7fb4a9200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a40cee40 con 0x7fb4a414eb30 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.592+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a409f790 0x7fb4a40cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.592+0000 7fb4a1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a409f790 0x7fb4a40cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46596/0 (socket says 172.21.15.7:46596) 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.592+0000 7fb4a1400640 1 -- 172.21.15.7:0/890549161 learned_addr learned my addr 172.21.15.7:0/890549161 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb4a1e00640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4070d60 0x7fb4a40cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb4a2800640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a414eb30 0x7fb4a40cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb4a1400640 1 -- 172.21.15.7:0/890549161 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a414eb30 msgr2=0x7fb4a40cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb4a1400640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a414eb30 0x7fb4a40cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb4a1400640 1 -- 172.21.15.7:0/890549161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4070d60 msgr2=0x7fb4a40cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb4a1400640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4070d60 0x7fb4a40cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb4a1400640 1 -- 172.21.15.7:0/890549161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a415c760 con 0x7fb4a409f790 2024-04-25T11:21:43.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb4a1e00640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4070d60 0x7fb4a40cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:43.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb4a1400640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a409f790 0x7fb4a40cb340 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fb490001eb0 tx=0x7fb49000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:43.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb49a000640 1 -- 172.21.15.7:0/890549161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490017070 con 0x7fb4a409f790 2024-04-25T11:21:43.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb49a000640 1 -- 172.21.15.7:0/890549161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb490005d00 con 0x7fb4a409f790 2024-04-25T11:21:43.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb4a9200640 1 -- 172.21.15.7:0/890549161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a415ca40 con 0x7fb4a409f790 2024-04-25T11:21:43.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.593+0000 7fb49a000640 1 -- 172.21.15.7:0/890549161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb490007310 con 0x7fb4a409f790 2024-04-25T11:21:43.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.594+0000 7fb4a9200640 1 -- 172.21.15.7:0/890549161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a415d000 con 0x7fb4a409f790 2024-04-25T11:21:43.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.595+0000 7fb4a9200640 1 -- 172.21.15.7:0/890549161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a4050740 con 0x7fb4a409f790 2024-04-25T11:21:43.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.596+0000 7fb49a000640 1 -- 172.21.15.7:0/890549161 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb49001a070 con 0x7fb4a409f790 2024-04-25T11:21:43.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.599+0000 7fb49a000640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4840744b0 0x7fb4840769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:43.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.599+0000 7fb49a000640 1 -- 172.21.15.7:0/890549161 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7fb490094820 con 0x7fb4a409f790 2024-04-25T11:21:43.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.599+0000 7fb4a1e00640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4840744b0 0x7fb4840769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:43.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.600+0000 7fb4a1e00640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4840744b0 0x7fb4840769c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40b0460 tx=0x7fb494008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:43.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.604+0000 7fb49a000640 1 -- 172.21.15.7:0/890549161 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb49005ed80 con 0x7fb4a409f790 2024-04-25T11:21:43.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:21:43.734 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:21:43.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.880+0000 7fb4a9200640 1 -- 172.21.15.7:0/890549161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb4a40712d0 con 0x7fb4a409f790 2024-04-25T11:21:43.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.885+0000 7fb49a000640 1 -- 172.21.15.7:0/890549161 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7fb49005e680 con 0x7fb4a409f790 2024-04-25T11:21:43.882 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-25T11:21:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:21:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-04-25T11:21:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-04-25T11:21:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:21:43.882 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-04-25T11:21:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 -- 172.21.15.7:0/890549161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4840744b0 msgr2=0x7fb4840769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4840744b0 0x7fb4840769c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40b0460 tx=0x7fb494008040 comp rx=0 tx=0).stop 2024-04-25T11:21:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 -- 172.21.15.7:0/890549161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a409f790 msgr2=0x7fb4a40cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a409f790 0x7fb4a40cb340 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fb490001eb0 tx=0x7fb49000c820 comp rx=0 tx=0).stop 2024-04-25T11:21:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 -- 172.21.15.7:0/890549161 shutdown_connections 2024-04-25T11:21:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb4840744b0 0x7fb4840769c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb4a414eb30 0x7fb4a40cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4a409f790 0x7fb4a40cb340 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 --2- 172.21.15.7:0/890549161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4a4070d60 0x7fb4a40cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 -- 172.21.15.7:0/890549161 >> 172.21.15.7:0/890549161 conn(0x7fb4a40d1d40 msgr2=0x7fb4a40b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 -- 172.21.15.7:0/890549161 shutdown_connections 2024-04-25T11:21:43.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:43.888+0000 7fb4a9200640 1 -- 172.21.15.7:0/890549161 wait complete. 2024-04-25T11:21:43.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:21:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.242+0000 7f5460a00640 1 Processor -- start 2024-04-25T11:21:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.242+0000 7f5460a00640 1 -- start start 2024-04-25T11:21:44.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.244+0000 7f5460a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f545c163ac0 0x7f545c0b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.244+0000 7f5460a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f545c06dcd0 0x7f545c06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.244+0000 7f5460a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f545c0d35a0 con 0x7f545c0d0570 2024-04-25T11:21:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.244+0000 7f5460a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f545c06e620 con 0x7f545c163ac0 2024-04-25T11:21:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.244+0000 7f5460a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f545c164310 con 0x7f545c06dcd0 2024-04-25T11:21:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.244+0000 7f545a800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f545c163ac0 0x7f545c0b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.244+0000 7f545a800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f545c163ac0 0x7f545c0b4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46604/0 (socket says 172.21.15.7:46604) 2024-04-25T11:21:44.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.244+0000 7f545a800640 1 -- 172.21.15.7:0/1974541685 learned_addr learned my addr 172.21.15.7:0/1974541685 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.244+0000 7f5451e00640 1 --2- 172.21.15.7:0/1974541685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f545c06dcd0 0x7f545c06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.245+0000 7f545a800640 1 -- 172.21.15.7:0/1974541685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f545c06dcd0 msgr2=0x7f545c06e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.245+0000 7f545a800640 1 --2- 172.21.15.7:0/1974541685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f545c06dcd0 0x7f545c06e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.245+0000 7f545a800640 1 -- 172.21.15.7:0/1974541685 >> v1:172.21.15.7:6789/0 conn(0x7f545c0d0570 legacy=0x7f545c0d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.245+0000 7f545a800640 1 -- 172.21.15.7:0/1974541685 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c1645f0 con 0x7f545c163ac0 2024-04-25T11:21:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.245+0000 7f5451e00640 1 --2- 172.21.15.7:0/1974541685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f545c06dcd0 0x7f545c06e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.246+0000 7f545a800640 1 --2- 172.21.15.7:0/1974541685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545c163ac0 0x7f545c0b4b90 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f544c009870 tx=0x7f544c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80938bd0ab346b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.246+0000 7f5459e00640 1 -- 172.21.15.7:0/1974541685 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c037070 con 0x7f545c163ac0 2024-04-25T11:21:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.246+0000 7f5459e00640 1 -- 172.21.15.7:0/1974541685 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f544c032440 con 0x7f545c163ac0 2024-04-25T11:21:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.246+0000 7f5459e00640 1 -- 172.21.15.7:0/1974541685 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c032810 con 0x7f545c163ac0 2024-04-25T11:21:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.247+0000 7f5460a00640 1 -- 172.21.15.7:0/1974541685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545c163ac0 msgr2=0x7f545c0b4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.247+0000 7f5460a00640 1 --2- 172.21.15.7:0/1974541685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545c163ac0 0x7f545c0b4b90 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f544c009870 tx=0x7f544c02a1d0 comp rx=0 tx=0).stop 2024-04-25T11:21:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.247+0000 7f5460a00640 1 -- 172.21.15.7:0/1974541685 shutdown_connections 2024-04-25T11:21:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.247+0000 7f5460a00640 1 --2- 172.21.15.7:0/1974541685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f545c06dcd0 0x7f545c06e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.247+0000 7f5460a00640 1 --2- 172.21.15.7:0/1974541685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545c163ac0 0x7f545c0b4b90 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.247+0000 7f5460a00640 1 -- 172.21.15.7:0/1974541685 >> 172.21.15.7:0/1974541685 conn(0x7f545c0d29e0 msgr2=0x7f545c0d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.247+0000 7f5460a00640 1 -- 172.21.15.7:0/1974541685 shutdown_connections 2024-04-25T11:21:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.247+0000 7f5460a00640 1 -- 172.21.15.7:0/1974541685 wait complete. 2024-04-25T11:21:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.249+0000 7f5460a00640 1 Processor -- start 2024-04-25T11:21:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.249+0000 7f5460a00640 1 -- start start 2024-04-25T11:21:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.251+0000 7f5460a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545c06dcd0 0x7f545c156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.251+0000 7f5460a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f545c0d0570 0x7f545c0cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.251+0000 7f5460a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f545c163ac0 0x7f545c158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.251+0000 7f5460a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f545c163ed0 con 0x7f545c06dcd0 2024-04-25T11:21:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.251+0000 7f5460a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f545c0b50d0 con 0x7f545c0d0570 2024-04-25T11:21:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.251+0000 7f5460a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f545c164050 con 0x7f545c163ac0 2024-04-25T11:21:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.251+0000 7f545a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545c06dcd0 0x7f545c156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.251+0000 7f545a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545c06dcd0 0x7f545c156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46610/0 (socket says 172.21.15.7:46610) 2024-04-25T11:21:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.251+0000 7f545a800640 1 -- 172.21.15.7:0/3324985182 learned_addr learned my addr 172.21.15.7:0/3324985182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.252+0000 7f545a800640 1 -- 172.21.15.7:0/3324985182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f545c163ac0 msgr2=0x7f545c158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.252+0000 7f545a800640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f545c163ac0 0x7f545c158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.252+0000 7f545a800640 1 -- 172.21.15.7:0/3324985182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f545c0d0570 msgr2=0x7f545c0cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.252+0000 7f545a800640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f545c0d0570 0x7f545c0cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.252+0000 7f545a800640 1 -- 172.21.15.7:0/3324985182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545c145c80 con 0x7f545c06dcd0 2024-04-25T11:21:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.252+0000 7f545a800640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545c06dcd0 0x7f545c156400 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f544c02a210 tx=0x7f544c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.252+0000 7f5453400640 1 -- 172.21.15.7:0/3324985182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c037040 con 0x7f545c06dcd0 2024-04-25T11:21:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.252+0000 7f5453400640 1 -- 172.21.15.7:0/3324985182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f544c042070 con 0x7f545c06dcd0 2024-04-25T11:21:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.252+0000 7f5460a00640 1 -- 172.21.15.7:0/3324985182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545c145f60 con 0x7f545c06dcd0 2024-04-25T11:21:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.252+0000 7f5453400640 1 -- 172.21.15.7:0/3324985182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544c030bd0 con 0x7f545c06dcd0 2024-04-25T11:21:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.252+0000 7f5460a00640 1 -- 172.21.15.7:0/3324985182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545c146520 con 0x7f545c06dcd0 2024-04-25T11:21:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.254+0000 7f5460a00640 1 -- 172.21.15.7:0/3324985182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545c050740 con 0x7f545c06dcd0 2024-04-25T11:21:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.255+0000 7f5453400640 1 -- 172.21.15.7:0/3324985182 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f544c047460 con 0x7f545c06dcd0 2024-04-25T11:21:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.256+0000 7f5453400640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54380744b0 0x7f54380769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.256+0000 7f5453400640 1 -- 172.21.15.7:0/3324985182 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f544c0b6030 con 0x7f545c06dcd0 2024-04-25T11:21:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.256+0000 7f5451e00640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54380744b0 0x7f54380769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.257+0000 7f5451e00640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54380744b0 0x7f54380769c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5444006fd0 tx=0x7f5444008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.261+0000 7f5453400640 1 -- 172.21.15.7:0/3324985182 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f544c07ffe0 con 0x7f545c06dcd0 2024-04-25T11:21:44.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.498+0000 7f5460a00640 1 -- 172.21.15.7:0/3324985182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f545c163580 con 0x7f545c06dcd0 2024-04-25T11:21:44.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.499+0000 7f5453400640 1 -- 172.21.15.7:0/3324985182 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5172 (secure 0 0 0) 0x7f544c0842e0 con 0x7f545c06dcd0 2024-04-25T11:21:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:44.495 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/3503931395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3503931395},{"type":"v1","addr":"172.21.15.87:6834","nonce":3503931395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/63829697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":63829697},{"type":"v1","addr":"172.21.15.87:6835","nonce":63829697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/415877708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":415877708},{"type":"v1","addr":"172.21.15.7:6835","nonce":415877708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:21:37.365427+0000","modified":"2024-04-25T11:21:40.443636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8967},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8967":{"gid":8967,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2966579341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2966579341},{"type":"v1","addr":"172.21.15.7:6837","nonce":2966579341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8967,"qdb_cluster":[8967]},"id":10}]} 2024-04-25T11:21:44.495 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 103 2024-04-25T11:21:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 -- 172.21.15.7:0/3324985182 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54380744b0 msgr2=0x7f54380769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54380744b0 0x7f54380769c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5444006fd0 tx=0x7f5444008040 comp rx=0 tx=0).stop 2024-04-25T11:21:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 -- 172.21.15.7:0/3324985182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545c06dcd0 msgr2=0x7f545c156400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545c06dcd0 0x7f545c156400 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f544c02a210 tx=0x7f544c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 -- 172.21.15.7:0/3324985182 shutdown_connections 2024-04-25T11:21:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54380744b0 0x7f54380769c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f545c163ac0 0x7f545c158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f545c0d0570 0x7f545c0cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:44.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 --2- 172.21.15.7:0/3324985182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545c06dcd0 0x7f545c156400 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:44.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 -- 172.21.15.7:0/3324985182 >> 172.21.15.7:0/3324985182 conn(0x7f545c0d29e0 msgr2=0x7f545c160650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:44.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 -- 172.21.15.7:0/3324985182 shutdown_connections 2024-04-25T11:21:44.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:44.501+0000 7f5460a00640 1 -- 172.21.15.7:0/3324985182 wait complete. 2024-04-25T11:21:44.924 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:21:37.365427+0000', 'modified': '2024-04-25T11:21:40.443636+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8967}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8967': {'gid': 8967, 'name': 'c', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6837/2966579341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6836', 'nonce': 2966579341}, {'type': 'v1', 'addr': '172.21.15.7:6837', 'nonce': 2966579341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8967, 'qdb_cluster': [8967]} 2024-04-25T11:21:44.925 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:21:44.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:21:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.221+0000 7f3767c00640 1 Processor -- start 2024-04-25T11:21:45.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.221+0000 7f3767c00640 1 -- start start 2024-04-25T11:21:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.224+0000 7f3767c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f376006f1e0 0x7f376006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.224+0000 7f3767c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3760162700 con 0x7f376006d580 2024-04-25T11:21:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.224+0000 7f3767c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37601629e0 con 0x7f376015f180 2024-04-25T11:21:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.224+0000 7f3767c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f376015f590 con 0x7f376006f1e0 2024-04-25T11:21:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.224+0000 7f3757e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f376006d580 0x7f376006d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47646/0 (socket says 172.21.15.7:47646) 2024-04-25T11:21:45.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.224+0000 7f3757e00640 1 -- 172.21.15.7:0/1817546417 learned_addr learned my addr 172.21.15.7:0/1817546417 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.225+0000 7f3765c00640 1 --2- 172.21.15.7:0/1817546417 >> v2:172.21.15.87:3300/0 conn(0x7f376006f1e0 0x7f376006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.225+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3016797776 0 0) 0x7f3760162700 con 0x7f376006d580 2024-04-25T11:21:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.225+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3744003520 con 0x7f376006d580 2024-04-25T11:21:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.225+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 11286329 0 0) 0x7f3744003520 con 0x7f376006d580 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.225+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 >> v2:172.21.15.87:3300/0 conn(0x7f376006f1e0 msgr2=0x7f376006f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.225+0000 7f375fe00640 1 --2- 172.21.15.7:0/1817546417 >> v2:172.21.15.87:3300/0 conn(0x7f376006f1e0 0x7f376006f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.225+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 >> v1:172.21.15.87:6789/0 conn(0x7f376015f180 legacy=0x7f3760161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.225+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376015e980 con 0x7f376006d580 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.225+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f3748001670 con 0x7f376006d580 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.225+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 >> v1:172.21.15.7:6789/0 conn(0x7f376006d580 legacy=0x7f376006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f375fe00640 1 --2- 172.21.15.7:0/1817546417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37440039b0 0x7f3744003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f375fe00640 1 --2- 172.21.15.7:0/1817546417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3744004800 0x7f3744004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f375fe00640 1 --2- 172.21.15.7:0/1817546417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3744005250 0x7f374400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f376015f590 con 0x7f37440039b0 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f374400c1e0 con 0x7f3744005250 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f374400c3e0 con 0x7f3744004800 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f3765200640 1 --2- 172.21.15.7:0/1817546417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37440039b0 0x7f3744003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f3757e00640 1 --2- 172.21.15.7:0/1817546417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3744004800 0x7f3744004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f3765c00640 1 --2- 172.21.15.7:0/1817546417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3744005250 0x7f374400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f3765200640 1 -- 172.21.15.7:0/1817546417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3744004800 msgr2=0x7f3744004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f3765200640 1 --2- 172.21.15.7:0/1817546417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3744004800 0x7f3744004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f3765200640 1 -- 172.21.15.7:0/1817546417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3744005250 msgr2=0x7f374400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.226+0000 7f3765200640 1 --2- 172.21.15.7:0/1817546417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3744005250 0x7f374400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.227+0000 7f3765200640 1 -- 172.21.15.7:0/1817546417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3744003520 con 0x7f37440039b0 2024-04-25T11:21:45.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.227+0000 7f3765c00640 1 -- 172.21.15.7:0/1817546417 reap_dead start 2024-04-25T11:21:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.228+0000 7f3765200640 1 --2- 172.21.15.7:0/1817546417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37440039b0 0x7f3744003e20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f3750001720 tx=0x7f3744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d98f614545d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.228+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750001370 con 0x7f37440039b0 2024-04-25T11:21:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.228+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37500052d0 con 0x7f37440039b0 2024-04-25T11:21:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.228+0000 7f375fe00640 1 -- 172.21.15.7:0/1817546417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375002ab90 con 0x7f37440039b0 2024-04-25T11:21:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.228+0000 7f3767c00640 1 -- 172.21.15.7:0/1817546417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37440039b0 msgr2=0x7f3744003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.228+0000 7f3767c00640 1 --2- 172.21.15.7:0/1817546417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37440039b0 0x7f3744003e20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f3750001720 tx=0x7f3744005830 comp rx=0 tx=0).stop 2024-04-25T11:21:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.229+0000 7f3767c00640 1 -- 172.21.15.7:0/1817546417 shutdown_connections 2024-04-25T11:21:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.229+0000 7f3767c00640 1 --2- 172.21.15.7:0/1817546417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37440039b0 0x7f3744003e20 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.229+0000 7f3767c00640 1 -- 172.21.15.7:0/1817546417 >> 172.21.15.7:0/1817546417 conn(0x7f37600732b0 msgr2=0x7f37600736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.229+0000 7f3767c00640 1 -- 172.21.15.7:0/1817546417 shutdown_connections 2024-04-25T11:21:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.229+0000 7f3767c00640 1 -- 172.21.15.7:0/1817546417 wait complete. 2024-04-25T11:21:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.231+0000 7f3767c00640 1 Processor -- start 2024-04-25T11:21:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.231+0000 7f3767c00640 1 -- start start 2024-04-25T11:21:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.233+0000 7f3767c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37440039b0 0x7f37601f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.233+0000 7f3767c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601513e0 0x7f3760152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.233+0000 7f3767c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37601570e0 0x7f37601574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.233+0000 7f3767c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f376015e670 con 0x7f37601513e0 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.233+0000 7f3767c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3760158ae0 con 0x7f37601570e0 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.233+0000 7f3767c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f37600d1310 con 0x7f37440039b0 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.233+0000 7f3757e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601513e0 0x7f3760152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.233+0000 7f3757e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601513e0 0x7f3760152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46624/0 (socket says 172.21.15.7:46624) 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.233+0000 7f3757e00640 1 -- 172.21.15.7:0/3147480882 learned_addr learned my addr 172.21.15.7:0/3147480882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.234+0000 7f3765c00640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37601570e0 0x7f37601574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.234+0000 7f3765200640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37440039b0 0x7f37601f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.234+0000 7f3757e00640 1 -- 172.21.15.7:0/3147480882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37440039b0 msgr2=0x7f37601f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.234+0000 7f3757e00640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37440039b0 0x7f37601f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.234+0000 7f3757e00640 1 -- 172.21.15.7:0/3147480882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37601570e0 msgr2=0x7f37601574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.234+0000 7f3757e00640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37601570e0 0x7f37601574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.234+0000 7f3757e00640 1 -- 172.21.15.7:0/3147480882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3760157a30 con 0x7f37601513e0 2024-04-25T11:21:45.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.234+0000 7f3765200640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37440039b0 0x7f37601f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.234+0000 7f3757e00640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601513e0 0x7f3760152a50 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f3748001050 tx=0x7f374800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.235+0000 7f375d600640 1 -- 172.21.15.7:0/3147480882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374800ee70 con 0x7f37601513e0 2024-04-25T11:21:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.235+0000 7f375d600640 1 -- 172.21.15.7:0/3147480882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3748018070 con 0x7f37601513e0 2024-04-25T11:21:45.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.235+0000 7f3767c00640 1 -- 172.21.15.7:0/3147480882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3760156040 con 0x7f37601513e0 2024-04-25T11:21:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.235+0000 7f375d600640 1 -- 172.21.15.7:0/3147480882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374800cad0 con 0x7f37601513e0 2024-04-25T11:21:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.235+0000 7f3767c00640 1 -- 172.21.15.7:0/3147480882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3760156600 con 0x7f37601513e0 2024-04-25T11:21:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.237+0000 7f375d600640 1 -- 172.21.15.7:0/3147480882 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f374800cc70 con 0x7f37601513e0 2024-04-25T11:21:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.239+0000 7f375d600640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f373c0744b0 0x7f373c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.239+0000 7f375d600640 1 -- 172.21.15.7:0/3147480882 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f3748095260 con 0x7f37601513e0 2024-04-25T11:21:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.239+0000 7f3765200640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f373c0744b0 0x7f373c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.240+0000 7f376eb97740 1 Processor -- start 2024-04-25T11:21:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.240+0000 7f376eb97740 1 -- start start 2024-04-25T11:21:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.240+0000 7f3765200640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f373c0744b0 0x7f373c0769c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3750032000 tx=0x7f375002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.241+0000 7f376eb97740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556377312270 0x5563776df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.241+0000 7f376eb97740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772e93b0 0x5563776f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.241+0000 7f376eb97740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.241+0000 7f376eb97740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556377618600 con 0x5563772f1590 2024-04-25T11:21:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.241+0000 7f376eb97740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5563775b5100 con 0x5563772e93b0 2024-04-25T11:21:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.241+0000 7f376eb97740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5563776145c0 con 0x556377312270 2024-04-25T11:21:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.242+0000 7f3765c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.242+0000 7f3765c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776fa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46628/0 (socket says 172.21.15.7:46628) 2024-04-25T11:21:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.242+0000 7f3765c00640 1 -- 172.21.15.7:0/1112942762 learned_addr learned my addr 172.21.15.7:0/1112942762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.242+0000 7f3765c00640 1 -- 172.21.15.7:0/1112942762 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556377312270 msgr2=0x5563776df040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.242+0000 7f3765c00640 1 --2- 172.21.15.7:0/1112942762 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556377312270 0x5563776df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.242+0000 7f3765c00640 1 -- 172.21.15.7:0/1112942762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772e93b0 msgr2=0x5563776f9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.242+0000 7f3757e00640 1 --2- 172.21.15.7:0/1112942762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772e93b0 0x5563776f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.242+0000 7f3765c00640 1 --2- 172.21.15.7:0/1112942762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772e93b0 0x5563776f9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.242+0000 7f3765c00640 1 -- 172.21.15.7:0/1112942762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637732a2d0 con 0x5563772f1590 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.243+0000 7f3765c00640 1 --2- 172.21.15.7:0/1112942762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776fa2b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f37600d16d0 tx=0x7f3758009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a537753b242fba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.243+0000 7f3755600640 1 -- 172.21.15.7:0/1112942762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758007e40 con 0x5563772f1590 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.243+0000 7f3755600640 1 -- 172.21.15.7:0/1112942762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375801d070 con 0x5563772f1590 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.243+0000 7f3755600640 1 -- 172.21.15.7:0/1112942762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758017880 con 0x5563772f1590 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.244+0000 7f376eb97740 1 -- 172.21.15.7:0/1112942762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 msgr2=0x5563776fa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.244+0000 7f376eb97740 1 --2- 172.21.15.7:0/1112942762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776fa2b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f37600d16d0 tx=0x7f3758009750 comp rx=0 tx=0).stop 2024-04-25T11:21:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.244+0000 7f376eb97740 1 -- 172.21.15.7:0/1112942762 shutdown_connections 2024-04-25T11:21:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.244+0000 7f376eb97740 1 --2- 172.21.15.7:0/1112942762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776fa2b0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.244+0000 7f376eb97740 1 --2- 172.21.15.7:0/1112942762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772e93b0 0x5563776f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.244+0000 7f376eb97740 1 --2- 172.21.15.7:0/1112942762 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556377312270 0x5563776df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.244+0000 7f376eb97740 1 -- 172.21.15.7:0/1112942762 >> 172.21.15.7:0/1112942762 conn(0x5563773063c0 msgr2=0x5563776b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.244+0000 7f376eb97740 1 -- 172.21.15.7:0/1112942762 shutdown_connections 2024-04-25T11:21:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.244+0000 7f376eb97740 1 -- 172.21.15.7:0/1112942762 wait complete. 2024-04-25T11:21:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.245+0000 7f376eb97740 1 -- start start 2024-04-25T11:21:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.245+0000 7f376eb97740 1 Processor -- start 2024-04-25T11:21:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.247+0000 7f376eb97740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5563772e93b0 0x5563776f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.247+0000 7f376eb97740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772f1590 0x5563776fdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.247+0000 7f376eb97740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556377312270 0x5563776fe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.247+0000 7f376eb97740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55637759ff70 con 0x556377312270 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.247+0000 7f376eb97740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55637770a270 con 0x5563772f1590 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.247+0000 7f376eb97740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55637770a690 con 0x5563772e93b0 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.247+0000 7f376eb97740 10 client.0 _ll_register_callbacks cb 0x55637760e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.247+0000 7f3765c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556377312270 0x5563776fe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.247+0000 7f3765c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556377312270 0x5563776fe4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46630/0 (socket says 172.21.15.7:46630) 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.247+0000 7f3765c00640 1 -- 172.21.15.7:0/3623160706 learned_addr learned my addr 172.21.15.7:0/3623160706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.248+0000 7f3765200640 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5563772e93b0 0x5563776f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.248+0000 7f3757e00640 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772f1590 0x5563776fdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.248+0000 7f3765c00640 1 -- 172.21.15.7:0/3623160706 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5563772e93b0 msgr2=0x5563776f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.248+0000 7f3765c00640 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5563772e93b0 0x5563776f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.248+0000 7f3765c00640 1 -- 172.21.15.7:0/3623160706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772f1590 msgr2=0x5563776fdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.248+0000 7f3765c00640 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772f1590 0x5563776fdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.248+0000 7f3765c00640 1 -- 172.21.15.7:0/3623160706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563773f46b0 con 0x556377312270 2024-04-25T11:21:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.248+0000 7f3765c00640 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556377312270 0x5563776fe4c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f3758009790 tx=0x7f3758004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11404556600fa1da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.248+0000 7f374e000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:21:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f374e000640 1 -- 172.21.15.7:0/3623160706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758003ba0 con 0x556377312270 2024-04-25T11:21:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f376eb97740 1 -- 172.21.15.7:0/3623160706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375801d070 con 0x556377312270 2024-04-25T11:21:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f374e000640 1 -- 172.21.15.7:0/3623160706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f3758003d40 con 0x556377312270 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f374e000640 1 -- 172.21.15.7:0/3623160706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f3758032050 con 0x556377312270 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f376eb97740 10 client.9108 fetch_fsmap learned FSMap version 103 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f376eb97740 1 -- 172.21.15.7:0/3623160706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) -- 0x55637770a410 con 0x556377312270 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f374e000640 1 -- 172.21.15.7:0/3623160706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375801d070 con 0x556377312270 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f374e000640 1 -- 172.21.15.7:0/3623160706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 103) ==== 3494+0+0 (secure 0 0 0) 0x7f3758035020 con 0x556377312270 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f376eb97740 10 client.9108 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f376eb97740 10 client.9108 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f376eb97740 20 client.9108 populate_metadata read hostname 'smithi007' 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f376eb97740 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.7:6836/2966579341,v1:172.21.15.7:6837/2966579341] conn(0x5563772d0e40 0x5563776e03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f376eb97740 4 client.9108 mds_command: new command op to 8967 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.249+0000 7f376eb97740 1 -- 172.21.15.7:0/3623160706 --> [v2:172.21.15.7:6836/2966579341,v1:172.21.15.7:6837/2966579341] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556377626210 con 0x5563772d0e40 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.250+0000 7f3765200640 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.7:6836/2966579341,v1:172.21.15.7:6837/2966579341] conn(0x5563772d0e40 0x5563776e03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.250+0000 7f3765200640 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.7:6836/2966579341,v1:172.21.15.7:6837/2966579341] conn(0x5563772d0e40 0x5563776e03b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e510727ba15ea50 server_cookie=a361f6185b421fb7 in_seq=0 out_seq=0 2024-04-25T11:21:45.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.250+0000 7f374e000640 10 client.9108 ms_handle_connect on v2:172.21.15.7:6836/2966579341 2024-04-25T11:21:45.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f374e000640 1 -- 172.21.15.7:0/3623160706 <== mds.0 v2:172.21.15.7:6836/2966579341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556377626210 con 0x5563772d0e40 2024-04-25T11:21:45.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f374e000640 10 client.9108 handle_command_reply: tid=0 2024-04-25T11:21:45.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 1 client.9108 shutdown 2024-04-25T11:21:45.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 1 -- 172.21.15.7:0/3623160706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556377312270 msgr2=0x5563776fe4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556377312270 0x5563776fe4c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f3758009790 tx=0x7f3758004a40 comp rx=0 tx=0).stop 2024-04-25T11:21:45.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 1 -- 172.21.15.7:0/3623160706 shutdown_connections 2024-04-25T11:21:45.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.7:6836/2966579341,v1:172.21.15.7:6837/2966579341] conn(0x5563772d0e40 0x5563776e03b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556377312270 0x5563776fe4c0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772f1590 0x5563776fdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f374e000640 0 client.9108 ms_handle_reset on v2:172.21.15.7:6836/2966579341 2024-04-25T11:21:45.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 1 --2- 172.21.15.7:0/3623160706 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5563772e93b0 0x5563776f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 1 -- 172.21.15.7:0/3623160706 >> 172.21.15.7:0/3623160706 conn(0x5563773063c0 msgr2=0x5563776b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:45.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 1 -- 172.21.15.7:0/3623160706 shutdown_connections 2024-04-25T11:21:45.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 1 -- 172.21.15.7:0/3623160706 wait complete. 2024-04-25T11:21:45.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.252+0000 7f376eb97740 20 client.9108 trim_cache size 0 max 16384 2024-04-25T11:21:45.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.273+0000 7f376eb97740 1 Processor -- start 2024-04-25T11:21:45.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.273+0000 7f376eb97740 1 -- start start 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.275+0000 7f376eb97740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5563772e93b0 0x55637770ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.275+0000 7f376eb97740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776ae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.275+0000 7f376eb97740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556377312270 0x55637770a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.275+0000 7f376eb97740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3760202870 con 0x5563772f1590 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.275+0000 7f376eb97740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3760201c00 con 0x556377312270 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.275+0000 7f376eb97740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3760202220 con 0x5563772e93b0 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.275+0000 7f3757e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.275+0000 7f3757e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776ae460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46646/0 (socket says 172.21.15.7:46646) 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.275+0000 7f3757e00640 1 -- 172.21.15.7:0/1142093880 learned_addr learned my addr 172.21.15.7:0/1142093880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.276+0000 7f3757e00640 1 -- 172.21.15.7:0/1142093880 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5563772e93b0 msgr2=0x55637770ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.276+0000 7f3757e00640 1 --2- 172.21.15.7:0/1142093880 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5563772e93b0 0x55637770ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.276+0000 7f3757e00640 1 -- 172.21.15.7:0/1142093880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556377312270 msgr2=0x55637770a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.276+0000 7f3765c00640 1 --2- 172.21.15.7:0/1142093880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556377312270 0x55637770a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.276+0000 7f3757e00640 1 --2- 172.21.15.7:0/1142093880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556377312270 0x55637770a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.276+0000 7f3757e00640 1 -- 172.21.15.7:0/1142093880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563776dfc50 con 0x5563772f1590 2024-04-25T11:21:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.276+0000 7f3757e00640 1 --2- 172.21.15.7:0/1142093880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776ae460 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f374805f050 tx=0x7f374805ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a170b98c598af941 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.276+0000 7f3754c00640 1 -- 172.21.15.7:0/1142093880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374805f7c0 con 0x5563772f1590 2024-04-25T11:21:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.276+0000 7f3754c00640 1 -- 172.21.15.7:0/1142093880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f374805f0c0 con 0x5563772f1590 2024-04-25T11:21:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.276+0000 7f3754c00640 1 -- 172.21.15.7:0/1142093880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3748063470 con 0x5563772f1590 2024-04-25T11:21:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.277+0000 7f376eb97740 1 -- 172.21.15.7:0/1142093880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 msgr2=0x5563776ae460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.277+0000 7f376eb97740 1 --2- 172.21.15.7:0/1142093880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776ae460 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f374805f050 tx=0x7f374805ead0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.277+0000 7f376eb97740 1 -- 172.21.15.7:0/1142093880 shutdown_connections 2024-04-25T11:21:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.277+0000 7f376eb97740 1 --2- 172.21.15.7:0/1142093880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556377312270 0x55637770a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.277+0000 7f376eb97740 1 --2- 172.21.15.7:0/1142093880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776ae460 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.277+0000 7f376eb97740 1 --2- 172.21.15.7:0/1142093880 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5563772e93b0 0x55637770ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.277+0000 7f376eb97740 1 -- 172.21.15.7:0/1142093880 >> 172.21.15.7:0/1142093880 conn(0x5563773063c0 msgr2=0x5563776b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.277+0000 7f376eb97740 1 -- 172.21.15.7:0/1142093880 shutdown_connections 2024-04-25T11:21:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.277+0000 7f376eb97740 1 -- 172.21.15.7:0/1142093880 wait complete. 2024-04-25T11:21:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.278+0000 7f376eb97740 1 -- start start 2024-04-25T11:21:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.278+0000 7f376eb97740 1 Processor -- start 2024-04-25T11:21:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.280+0000 7f376eb97740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772e93b0 0x5563776d8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.280+0000 7f376eb97740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776d92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.280+0000 7f376eb97740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556377312270 0x55637775d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.280+0000 7f376eb97740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556377709390 con 0x5563772f1590 2024-04-25T11:21:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.280+0000 7f376eb97740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5563776f6c90 con 0x5563772e93b0 2024-04-25T11:21:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.280+0000 7f376eb97740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x556377709920 con 0x556377312270 2024-04-25T11:21:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.280+0000 7f376eb97740 10 client.0 _ll_register_callbacks cb 0x7f3758009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:21:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.280+0000 7f3757e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776d92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.280+0000 7f3757e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776d92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46660/0 (socket says 172.21.15.7:46660) 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.280+0000 7f3757e00640 1 -- 172.21.15.7:0/3854877486 learned_addr learned my addr 172.21.15.7:0/3854877486 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f3765c00640 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556377312270 0x55637775d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f3765200640 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772e93b0 0x5563776d8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f3757e00640 1 -- 172.21.15.7:0/3854877486 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556377312270 msgr2=0x55637775d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f3757e00640 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556377312270 0x55637775d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f3757e00640 1 -- 172.21.15.7:0/3854877486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772e93b0 msgr2=0x5563776d8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f3757e00640 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772e93b0 0x5563776d8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f3757e00640 1 -- 172.21.15.7:0/3854877486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563776dbe00 con 0x5563772f1590 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f3765200640 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772e93b0 0x5563776d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f3757e00640 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776d92c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f374800f010 tx=0x7f3748001910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc6c9d288bdc2a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f374e000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f374e000640 1 -- 172.21.15.7:0/3854877486 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374805e520 con 0x5563772f1590 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f374e000640 1 -- 172.21.15.7:0/3854877486 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f374805aa30 con 0x5563772f1590 2024-04-25T11:21:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.281+0000 7f376eb97740 1 -- 172.21.15.7:0/3854877486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f374805f0c0 con 0x5563772f1590 2024-04-25T11:21:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f374e000640 1 -- 172.21.15.7:0/3854877486 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f374805f7c0 con 0x5563772f1590 2024-04-25T11:21:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f376eb97740 10 client.9114 fetch_fsmap learned FSMap version 103 2024-04-25T11:21:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f376eb97740 1 -- 172.21.15.7:0/3854877486 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) -- 0x55637775e310 con 0x5563772f1590 2024-04-25T11:21:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f374e000640 1 -- 172.21.15.7:0/3854877486 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f374805f0c0 con 0x5563772f1590 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f374e000640 1 -- 172.21.15.7:0/3854877486 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 103) ==== 3494+0+0 (secure 0 0 0) 0x7f374805ac30 con 0x5563772f1590 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f376eb97740 10 client.9114 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f376eb97740 10 client.9114 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f376eb97740 20 client.9114 populate_metadata read hostname 'smithi007' 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f376eb97740 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.7:6836/2966579341,v1:172.21.15.7:6837/2966579341] conn(0x55637775dc50 0x5563776b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f376eb97740 4 client.9114 mds_command: new command op to 8967 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f376eb97740 1 -- 172.21.15.7:0/3854877486 --> [v2:172.21.15.7:6836/2966579341,v1:172.21.15.7:6837/2966579341] -- command(tid 0: {"prefix": "status"}) -- 0x556377620080 con 0x55637775dc50 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.282+0000 7f3765200640 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.7:6836/2966579341,v1:172.21.15.7:6837/2966579341] conn(0x55637775dc50 0x5563776b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.283+0000 7f3765200640 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.7:6836/2966579341,v1:172.21.15.7:6837/2966579341] conn(0x55637775dc50 0x5563776b4970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c85a0d74feee20d2 server_cookie=78dc58fcb13cf45a in_seq=0 out_seq=0 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.283+0000 7f374e000640 10 client.9114 ms_handle_connect on v2:172.21.15.7:6836/2966579341 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.283+0000 7f374e000640 1 -- 172.21.15.7:0/3854877486 <== mds.0 v2:172.21.15.7:6836/2966579341 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556377620080 con 0x55637775dc50 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.283+0000 7f374e000640 10 client.9114 handle_command_reply: tid=0 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.283+0000 7f376eb97740 1 client.9114 shutdown 2024-04-25T11:21:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.283+0000 7f376eb97740 1 -- 172.21.15.7:0/3854877486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 msgr2=0x5563776d92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.283+0000 7f376eb97740 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776d92c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f374800f010 tx=0x7f3748001910 comp rx=0 tx=0).stop 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f376eb97740 1 -- 172.21.15.7:0/3854877486 shutdown_connections 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f376eb97740 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556377312270 0x55637775d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f376eb97740 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.7:6836/2966579341,v1:172.21.15.7:6837/2966579341] conn(0x55637775dc50 0x5563776b4970 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f376eb97740 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5563772f1590 0x5563776d92c0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f376eb97740 1 --2- 172.21.15.7:0/3854877486 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563772e93b0 0x5563776d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f374e000640 0 client.9114 ms_handle_reset on v2:172.21.15.7:6836/2966579341 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f376eb97740 1 -- 172.21.15.7:0/3854877486 >> 172.21.15.7:0/3854877486 conn(0x5563773063c0 msgr2=0x5563776e18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f376eb97740 1 -- 172.21.15.7:0/3854877486 shutdown_connections 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f376eb97740 1 -- 172.21.15.7:0/3854877486 wait complete. 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f376eb97740 20 client.9114 trim_cache size 0 max 16384 2024-04-25T11:21:45.282 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 8967, 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.4387575029999997, 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 103, 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 142, 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 142, 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.63414356 2024-04-25T11:21:45.283 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:21:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f3767c00640 1 -- 172.21.15.7:0/3147480882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f373c0744b0 msgr2=0x7f373c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f3767c00640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f373c0744b0 0x7f373c0769c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3750032000 tx=0x7f375002e000 comp rx=0 tx=0).stop 2024-04-25T11:21:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f3767c00640 1 -- 172.21.15.7:0/3147480882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601513e0 msgr2=0x7f3760152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.284+0000 7f3767c00640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601513e0 0x7f3760152a50 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f3748001050 tx=0x7f374800ec80 comp rx=0 tx=0).stop 2024-04-25T11:21:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.285+0000 7f3767c00640 1 -- 172.21.15.7:0/3147480882 shutdown_connections 2024-04-25T11:21:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.285+0000 7f3767c00640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f373c0744b0 0x7f373c0769c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.285+0000 7f3767c00640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37601570e0 0x7f37601574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.285+0000 7f3767c00640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37601513e0 0x7f3760152a50 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.285+0000 7f3767c00640 1 --2- 172.21.15.7:0/3147480882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37440039b0 0x7f37601f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.285+0000 7f3767c00640 1 -- 172.21.15.7:0/3147480882 >> 172.21.15.7:0/3147480882 conn(0x7f37600732b0 msgr2=0x7f37600b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.285+0000 7f3767c00640 1 -- 172.21.15.7:0/3147480882 shutdown_connections 2024-04-25T11:21:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.285+0000 7f3767c00640 1 -- 172.21.15.7:0/3147480882 wait complete. 2024-04-25T11:21:45.306 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:21:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.594+0000 7faabbe00640 1 Processor -- start 2024-04-25T11:21:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.594+0000 7faabbe00640 1 -- start start 2024-04-25T11:21:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.596+0000 7faabbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faab4163c80 0x7faab40d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.596+0000 7faabbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faab40d2550 0x7faab40d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.596+0000 7faabbe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faab4164090 con 0x7faab4160180 2024-04-25T11:21:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.596+0000 7faabbe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7faab40d0c00 con 0x7faab40d2550 2024-04-25T11:21:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.597+0000 7faabbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faab40d4510 con 0x7faab4163c80 2024-04-25T11:21:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.597+0000 7faab9e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faab40d2550 0x7faab40d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.597+0000 7faab9e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faab40d2550 0x7faab40d2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46666/0 (socket says 172.21.15.7:46666) 2024-04-25T11:21:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.597+0000 7faab9e00640 1 -- 172.21.15.7:0/1925533468 learned_addr learned my addr 172.21.15.7:0/1925533468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.597+0000 7faab8a00640 1 --2- 172.21.15.7:0/1925533468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faab4163c80 0x7faab40d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.598+0000 7faab9e00640 1 -- 172.21.15.7:0/1925533468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faab4163c80 msgr2=0x7faab40d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.598+0000 7faab9e00640 1 --2- 172.21.15.7:0/1925533468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faab4163c80 0x7faab40d06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.598+0000 7faab9e00640 1 -- 172.21.15.7:0/1925533468 >> v1:172.21.15.87:6789/0 conn(0x7faab4160180 legacy=0x7faab40b9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.598+0000 7faab9e00640 1 -- 172.21.15.7:0/1925533468 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab4164bb0 con 0x7faab40d2550 2024-04-25T11:21:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.599+0000 7faab9e00640 1 --2- 172.21.15.7:0/1925533468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faab40d2550 0x7faab40d2960 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7faab40d4690 tx=0x7faaac02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358c419d401f4a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.599+0000 7faab3e00640 1 -- 172.21.15.7:0/1925533468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac037070 con 0x7faab40d2550 2024-04-25T11:21:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.599+0000 7faab3e00640 1 -- 172.21.15.7:0/1925533468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaac032440 con 0x7faab40d2550 2024-04-25T11:21:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.599+0000 7faab3e00640 1 -- 172.21.15.7:0/1925533468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac032810 con 0x7faab40d2550 2024-04-25T11:21:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.600+0000 7faabbe00640 1 -- 172.21.15.7:0/1925533468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faab40d2550 msgr2=0x7faab40d2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.600+0000 7faabbe00640 1 --2- 172.21.15.7:0/1925533468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faab40d2550 0x7faab40d2960 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7faab40d4690 tx=0x7faaac02a250 comp rx=0 tx=0).stop 2024-04-25T11:21:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.600+0000 7faabbe00640 1 -- 172.21.15.7:0/1925533468 shutdown_connections 2024-04-25T11:21:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.600+0000 7faabbe00640 1 --2- 172.21.15.7:0/1925533468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faab40d2550 0x7faab40d2960 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.600+0000 7faabbe00640 1 --2- 172.21.15.7:0/1925533468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faab4163c80 0x7faab40d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.600+0000 7faabbe00640 1 -- 172.21.15.7:0/1925533468 >> 172.21.15.7:0/1925533468 conn(0x7faab406c750 msgr2=0x7faab406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.600+0000 7faabbe00640 1 -- 172.21.15.7:0/1925533468 shutdown_connections 2024-04-25T11:21:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.600+0000 7faabbe00640 1 -- 172.21.15.7:0/1925533468 wait complete. 2024-04-25T11:21:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.602+0000 7faabbe00640 1 Processor -- start 2024-04-25T11:21:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.603+0000 7faabbe00640 1 -- start start 2024-04-25T11:21:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.604+0000 7faabbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faab40d2550 0x7faab40cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.604+0000 7faabbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faab4160180 0x7faab41f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.604+0000 7faabbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faab4163c80 0x7faab40d58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.604+0000 7faabbe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faab41645f0 con 0x7faab4163c80 2024-04-25T11:21:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.604+0000 7faabbe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faab4164310 con 0x7faab4160180 2024-04-25T11:21:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.604+0000 7faabbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faab41648d0 con 0x7faab40d2550 2024-04-25T11:21:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.604+0000 7faab9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faab4163c80 0x7faab40d58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.605+0000 7faab9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faab4163c80 0x7faab40d58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46672/0 (socket says 172.21.15.7:46672) 2024-04-25T11:21:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.605+0000 7faab9e00640 1 -- 172.21.15.7:0/3618535307 learned_addr learned my addr 172.21.15.7:0/3618535307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.605+0000 7faab9400640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faab40d2550 0x7faab40cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.605+0000 7faab8a00640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faab4160180 0x7faab41f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.605+0000 7faab9e00640 1 -- 172.21.15.7:0/3618535307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faab40d2550 msgr2=0x7faab40cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.605+0000 7faab9e00640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faab40d2550 0x7faab40cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.605+0000 7faab9e00640 1 -- 172.21.15.7:0/3618535307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faab4160180 msgr2=0x7faab41f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.605+0000 7faab9e00640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faab4160180 0x7faab41f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.605+0000 7faab9e00640 1 -- 172.21.15.7:0/3618535307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab40cfee0 con 0x7faab4163c80 2024-04-25T11:21:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.605+0000 7faab9e00640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faab4163c80 0x7faab40d58f0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7faaac02a290 tx=0x7faaac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.606+0000 7faab1600640 1 -- 172.21.15.7:0/3618535307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac037040 con 0x7faab4163c80 2024-04-25T11:21:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.606+0000 7faab1600640 1 -- 172.21.15.7:0/3618535307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaac042070 con 0x7faab4163c80 2024-04-25T11:21:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.606+0000 7faab1600640 1 -- 172.21.15.7:0/3618535307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac030920 con 0x7faab4163c80 2024-04-25T11:21:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.606+0000 7faabbe00640 1 -- 172.21.15.7:0/3618535307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab40d01c0 con 0x7faab4163c80 2024-04-25T11:21:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.606+0000 7faabbe00640 1 -- 172.21.15.7:0/3618535307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab414bed0 con 0x7faab4163c80 2024-04-25T11:21:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.607+0000 7faabbe00640 1 -- 172.21.15.7:0/3618535307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab40cfc50 con 0x7faab4163c80 2024-04-25T11:21:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.608+0000 7faab1600640 1 -- 172.21.15.7:0/3618535307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faaac030ac0 con 0x7faab4163c80 2024-04-25T11:21:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.609+0000 7faab1600640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa940744e0 0x7faa940769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.610+0000 7faab1600640 1 -- 172.21.15.7:0/3618535307 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7faaac0b5dc0 con 0x7faab4163c80 2024-04-25T11:21:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.610+0000 7faab9400640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa940744e0 0x7faa940769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.611+0000 7faab9400640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa940744e0 0x7faa940769f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7faaa4001500 tx=0x7faaa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:45.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.614+0000 7faab1600640 1 -- 172.21.15.7:0/3618535307 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faaac07fcd0 con 0x7faab4163c80 2024-04-25T11:21:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.860+0000 7faabbe00640 1 -- 172.21.15.7:0/3618535307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faab40c9190 con 0x7faab4163c80 2024-04-25T11:21:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.861+0000 7faab1600640 1 -- 172.21.15.7:0/3618535307 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) ==== 53+0+73 (secure 0 0 0) 0x7faaac083fd0 con 0x7faab4163c80 2024-04-25T11:21:45.857 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:21:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 -- 172.21.15.7:0/3618535307 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa940744e0 msgr2=0x7faa940769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa940744e0 0x7faa940769f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7faaa4001500 tx=0x7faaa4009040 comp rx=0 tx=0).stop 2024-04-25T11:21:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 -- 172.21.15.7:0/3618535307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faab4163c80 msgr2=0x7faab40d58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faab4163c80 0x7faab40d58f0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7faaac02a290 tx=0x7faaac0046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 -- 172.21.15.7:0/3618535307 shutdown_connections 2024-04-25T11:21:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faa940744e0 0x7faa940769f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faab4163c80 0x7faab40d58f0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faab4160180 0x7faab41f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 --2- 172.21.15.7:0/3618535307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faab40d2550 0x7faab40cf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:45.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 -- 172.21.15.7:0/3618535307 >> 172.21.15.7:0/3618535307 conn(0x7faab406c750 msgr2=0x7faab40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:45.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 -- 172.21.15.7:0/3618535307 shutdown_connections 2024-04-25T11:21:45.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:45.864+0000 7faabbe00640 1 -- 172.21.15.7:0/3618535307 wait complete. 2024-04-25T11:21:45.888 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:21:45.888 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:21:45.888 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:21:45.888 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:21:45.888 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:21:45.888 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:21:45.888 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:21:45.889 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:21:45.889 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:21:45.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:45.889 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:21:45.939 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:21:45.940 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41869sec preferred_lft 41869sec 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6098:c7ff:fe8a:4a30/64 scope link 2024-04-25T11:21:45.941 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:45.942 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:21:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:21:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::646a:35ff:fe7b:ad0/64 scope link 2024-04-25T11:21:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:45.942 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:21:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:21:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4465:c4ff:fe62:f75f/64 scope link 2024-04-25T11:21:45.942 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:45.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:45.943 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:21:46.019 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:21:46.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:46.023 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:21:46.075 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:21:46.075 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:21:46.075 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:21:46.075 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:21:46.075 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:21:46.076 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:21:46.076 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:21:46.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:21:46.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:21:46.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:21:46.077 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:21:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:21:46.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:21:46.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:46.296 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:21:46.346 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:21:46.346 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:21:46.347 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:21:46.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:46.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:21:46.431 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:21:46.431 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:21:46.487 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:21:46.488 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:21:46.488 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:21:46.540 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:21:46.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:46.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:21:46.756 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:21:46.757 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:21:46.803 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:21:46.803 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:21:46.803 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:21:46.803 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:21:46.803 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:21:46.807 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:21:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:21:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:21:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:21:46.807 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:21:46.811 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:21:46.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:46.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:21:46.894 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:21:47.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:21:47.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:21:47.029 INFO:teuthology.orchestra.run.smithi007.stdout:client7381 2024-04-25T11:21:47.030 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:21:47.030 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:47.083 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:21:47.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.374+0000 7f146ea00640 1 Processor -- start 2024-04-25T11:21:47.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.374+0000 7f146ea00640 1 -- start start 2024-04-25T11:21:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.376+0000 7f146ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1468070810 0x7f146806d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.376+0000 7f146ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f146806dac0 0x7f146815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.376+0000 7f146ea00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14680cffa0 con 0x7f1468070c20 2024-04-25T11:21:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.376+0000 7f146ea00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f146806ded0 con 0x7f1468070810 2024-04-25T11:21:47.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.376+0000 7f146ea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1468160d90 con 0x7f146806dac0 2024-04-25T11:21:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.377+0000 7f146cc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1468070c20 0x7f146815d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47656/0 (socket says 172.21.15.7:47656) 2024-04-25T11:21:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.377+0000 7f146cc00640 1 -- 172.21.15.7:0/3842032903 learned_addr learned my addr 172.21.15.7:0/3842032903 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.377+0000 7f146d600640 1 --2- 172.21.15.7:0/3842032903 >> v2:172.21.15.7:3300/0 conn(0x7f1468070810 0x7f146806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.377+0000 7f146e000640 1 --2- 172.21.15.7:0/3842032903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f146806dac0 0x7f146815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.377+0000 7f146d600640 1 -- 172.21.15.7:0/3842032903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f146806dac0 msgr2=0x7f146815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.377+0000 7f146d600640 1 --2- 172.21.15.7:0/3842032903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f146806dac0 0x7f146815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.377+0000 7f146d600640 1 -- 172.21.15.7:0/3842032903 >> v1:172.21.15.7:6789/0 conn(0x7f1468070c20 legacy=0x7f146815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:47.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.377+0000 7f146d600640 1 -- 172.21.15.7:0/3842032903 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1468161070 con 0x7f1468070810 2024-04-25T11:21:47.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.379+0000 7f146d600640 1 --2- 172.21.15.7:0/3842032903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1468070810 0x7f146806d580 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f14580097a0 tx=0x7f145802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33a979a0a8709ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:47.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.379+0000 7f1467e00640 1 -- 172.21.15.7:0/3842032903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458037070 con 0x7f1468070810 2024-04-25T11:21:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.379+0000 7f1467e00640 1 -- 172.21.15.7:0/3842032903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1458032440 con 0x7f1468070810 2024-04-25T11:21:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.379+0000 7f1467e00640 1 -- 172.21.15.7:0/3842032903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458032810 con 0x7f1468070810 2024-04-25T11:21:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.380+0000 7f146ea00640 1 -- 172.21.15.7:0/3842032903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1468070810 msgr2=0x7f146806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:47.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.380+0000 7f146ea00640 1 --2- 172.21.15.7:0/3842032903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1468070810 0x7f146806d580 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f14580097a0 tx=0x7f145802a180 comp rx=0 tx=0).stop 2024-04-25T11:21:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.380+0000 7f146ea00640 1 -- 172.21.15.7:0/3842032903 shutdown_connections 2024-04-25T11:21:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.380+0000 7f146ea00640 1 --2- 172.21.15.7:0/3842032903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f146806dac0 0x7f146815d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.380+0000 7f146ea00640 1 --2- 172.21.15.7:0/3842032903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1468070810 0x7f146806d580 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.380+0000 7f146ea00640 1 -- 172.21.15.7:0/3842032903 >> 172.21.15.7:0/3842032903 conn(0x7f1468073090 msgr2=0x7f14680734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.380+0000 7f146ea00640 1 -- 172.21.15.7:0/3842032903 shutdown_connections 2024-04-25T11:21:47.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.380+0000 7f146ea00640 1 -- 172.21.15.7:0/3842032903 wait complete. 2024-04-25T11:21:47.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.383+0000 7f146ea00640 1 Processor -- start 2024-04-25T11:21:47.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.383+0000 7f146ea00640 1 -- start start 2024-04-25T11:21:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.384+0000 7f146ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f146806dac0 0x7f1468144c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.384+0000 7f146ea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1468070810 0x7f1468141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1468070c20 0x7f1468142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:47.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f14681433b0 con 0x7f146806dac0 2024-04-25T11:21:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146ea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1468161630 con 0x7f1468070810 2024-04-25T11:21:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146ea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1468143530 con 0x7f1468070c20 2024-04-25T11:21:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f146806dac0 0x7f1468144c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f146806dac0 0x7f1468144c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46692/0 (socket says 172.21.15.7:46692) 2024-04-25T11:21:47.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146d600640 1 -- 172.21.15.7:0/2766856324 learned_addr learned my addr 172.21.15.7:0/2766856324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146d600640 1 -- 172.21.15.7:0/2766856324 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1468070c20 msgr2=0x7f1468142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146d600640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1468070c20 0x7f1468142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146d600640 1 -- 172.21.15.7:0/2766856324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1468070810 msgr2=0x7f1468141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:47.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146d600640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1468070810 0x7f1468141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.385+0000 7f146d600640 1 -- 172.21.15.7:0/2766856324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1468140c80 con 0x7f146806dac0 2024-04-25T11:21:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.386+0000 7f146d600640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f146806dac0 0x7f1468144c60 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f1458035040 tx=0x7f14580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.386+0000 7f1465600640 1 -- 172.21.15.7:0/2766856324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458037040 con 0x7f146806dac0 2024-04-25T11:21:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.386+0000 7f1465600640 1 -- 172.21.15.7:0/2766856324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1458030770 con 0x7f146806dac0 2024-04-25T11:21:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.386+0000 7f1465600640 1 -- 172.21.15.7:0/2766856324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458030b40 con 0x7f146806dac0 2024-04-25T11:21:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.386+0000 7f146ea00640 1 -- 172.21.15.7:0/2766856324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1468140f60 con 0x7f146806dac0 2024-04-25T11:21:47.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.386+0000 7f146ea00640 1 -- 172.21.15.7:0/2766856324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1468141520 con 0x7f146806dac0 2024-04-25T11:21:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.387+0000 7f146ea00640 1 -- 172.21.15.7:0/2766856324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14680698e0 con 0x7f146806dac0 2024-04-25T11:21:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.389+0000 7f1465600640 1 -- 172.21.15.7:0/2766856324 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1458044020 con 0x7f146806dac0 2024-04-25T11:21:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.390+0000 7f1465600640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f144c074500 0x7f144c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.390+0000 7f1465600640 1 -- 172.21.15.7:0/2766856324 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f14580b5900 con 0x7f146806dac0 2024-04-25T11:21:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.391+0000 7f146cc00640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f144c074500 0x7f144c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:47.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.391+0000 7f146cc00640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f144c074500 0x7f144c076a10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f145c000f60 tx=0x7f145c007a80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:47.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.396+0000 7f1465600640 1 -- 172.21.15.7:0/2766856324 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f14680698e0 con 0x7f146806dac0 2024-04-25T11:21:47.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.640+0000 7f146ea00640 1 -- 172.21.15.7:0/2766856324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1468050740 con 0x7f146806dac0 2024-04-25T11:21:47.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.641+0000 7f1465600640 1 -- 172.21.15.7:0/2766856324 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) ==== 53+0+73 (secure 0 0 0) 0x7f1458032e20 con 0x7f146806dac0 2024-04-25T11:21:47.638 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:21:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 -- 172.21.15.7:0/2766856324 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f144c074500 msgr2=0x7f144c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f144c074500 0x7f144c076a10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f145c000f60 tx=0x7f145c007a80 comp rx=0 tx=0).stop 2024-04-25T11:21:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 -- 172.21.15.7:0/2766856324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f146806dac0 msgr2=0x7f1468144c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f146806dac0 0x7f1468144c60 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f1458035040 tx=0x7f14580046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 -- 172.21.15.7:0/2766856324 shutdown_connections 2024-04-25T11:21:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f144c074500 0x7f144c076a10 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1468070c20 0x7f1468142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:47.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1468070810 0x7f1468141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 --2- 172.21.15.7:0/2766856324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f146806dac0 0x7f1468144c60 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 -- 172.21.15.7:0/2766856324 >> 172.21.15.7:0/2766856324 conn(0x7f1468073090 msgr2=0x7f146806cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 -- 172.21.15.7:0/2766856324 shutdown_connections 2024-04-25T11:21:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:47.644+0000 7f146ea00640 1 -- 172.21.15.7:0/2766856324 wait complete. 2024-04-25T11:21:47.667 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:21:47.667 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:21:47.667 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:21:47.667 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:21:47.667 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:21:47.667 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:21:47.667 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:21:47.667 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:21:47.667 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:21:47.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:47.667 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:21:47.729 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:21:47.730 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:21:47.730 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:21:47.730 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:47.730 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:21:47.731 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:47.731 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:21:47.731 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:21:47.731 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:21:47.731 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:21:47.731 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:21:47.732 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:21:47.732 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:21:47.732 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:21:47.732 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:21:47.732 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:21:47.732 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:21:47.733 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41868sec preferred_lft 41868sec 2024-04-25T11:21:47.733 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:21:47.733 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:47.733 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:21:47.733 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:21:47.733 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:21:47.734 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:47.734 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::58f2:6bff:fe8b:8b4c/64 scope link 2024-04-25T11:21:47.734 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:47.734 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:21:47.734 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:21:47.734 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::98fe:1bff:fe9f:d71c/64 scope link 2024-04-25T11:21:47.735 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:47.735 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:21:47.735 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:21:47.735 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::3c38:a8ff:fec7:ea00/64 scope link 2024-04-25T11:21:47.735 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:21:47.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:47.736 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:21:47.817 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:21:47.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:47.822 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:21:47.875 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:21:47.875 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:21:47.876 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:21:47.876 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:21:47.876 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:21:47.877 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:21:47.877 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:21:47.877 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:21:47.877 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:21:47.877 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:21:47.877 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:21:47.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:21:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:21:48.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:21:48.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:48.092 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:21:48.144 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:21:48.144 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:21:48.145 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:21:48.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:48.145 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T11:21:48.224 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:21:48.225 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:21:48.276 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:21:48.276 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:21:48.276 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:21:48.326 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:21:48.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:48.327 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:21:48.533 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:21:48.533 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:21:48.579 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:21:48.579 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:21:48.579 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:21:48.579 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 1@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:21:48.579 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:21:48.581 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:21:48.581 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:21:48.581 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:21:48.581 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:21:48.581 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:21:48.585 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:21:48.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:21:48.586 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:21:48.662 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:21:48.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:21:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:21:48.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:21:48.783 INFO:teuthology.orchestra.run.smithi087.stdout:client9132 2024-04-25T11:21:48.784 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:21:48.784 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:48.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_cache_memory_limit 1K 2024-04-25T11:21:49.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.147+0000 7f6384c00640 1 Processor -- start 2024-04-25T11:21:49.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.147+0000 7f6384c00640 1 -- start start 2024-04-25T11:21:49.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.149+0000 7f6384c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63800b1660 0x7f638006c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.149+0000 7f6384c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63800b7440 0x7f63800d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.149+0000 7f6384c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63801600e0 0x7f63800d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.149+0000 7f6384c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f63800c6cc0 con 0x7f63800b7440 2024-04-25T11:21:49.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.149+0000 7f6384c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f63800c7280 con 0x7f63801600e0 2024-04-25T11:21:49.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.149+0000 7f6384c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63800c8700 con 0x7f63800b1660 2024-04-25T11:21:49.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.149+0000 7f637ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63800b7440 0x7f63800d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:49.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.149+0000 7f637ea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63800b7440 0x7f63800d1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46708/0 (socket says 172.21.15.7:46708) 2024-04-25T11:21:49.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.149+0000 7f637ea00640 1 -- 172.21.15.7:0/3261483359 learned_addr learned my addr 172.21.15.7:0/3261483359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:49.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.150+0000 7f637fe00640 1 --2- 172.21.15.7:0/3261483359 >> v2:172.21.15.87:3300/0 conn(0x7f63801600e0 0x7f63800d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:49.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.150+0000 7f637f400640 1 --2- 172.21.15.7:0/3261483359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63800b1660 0x7f638006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:49.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.150+0000 7f637ea00640 1 -- 172.21.15.7:0/3261483359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63800b1660 msgr2=0x7f638006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:49.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.150+0000 7f637ea00640 1 --2- 172.21.15.7:0/3261483359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63800b1660 0x7f638006c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.150+0000 7f637ea00640 1 -- 172.21.15.7:0/3261483359 >> v2:172.21.15.87:3300/0 conn(0x7f63801600e0 msgr2=0x7f63800d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:49.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.150+0000 7f637ea00640 1 --2- 172.21.15.7:0/3261483359 >> v2:172.21.15.87:3300/0 conn(0x7f63801600e0 0x7f63800d21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.150+0000 7f637ea00640 1 -- 172.21.15.7:0/3261483359 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63800c89a0 con 0x7f63800b7440 2024-04-25T11:21:49.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.150+0000 7f637fe00640 1 --2- 172.21.15.7:0/3261483359 >> v2:172.21.15.87:3300/0 conn(0x7f63801600e0 0x7f63800d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:49.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.150+0000 7f637f400640 1 --2- 172.21.15.7:0/3261483359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63800b1660 0x7f638006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.151+0000 7f637ea00640 1 --2- 172.21.15.7:0/3261483359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63800b7440 0x7f63800d1c90 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f6380074510 tx=0x7f636802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6eaf4687b390e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.151+0000 7f637e000640 1 -- 172.21.15.7:0/3261483359 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368037070 con 0x7f63800b7440 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.151+0000 7f637e000640 1 -- 172.21.15.7:0/3261483359 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6368032440 con 0x7f63800b7440 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.151+0000 7f637e000640 1 -- 172.21.15.7:0/3261483359 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368032810 con 0x7f63800b7440 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.152+0000 7f6384c00640 1 -- 172.21.15.7:0/3261483359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63800b7440 msgr2=0x7f63800d1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.152+0000 7f6384c00640 1 --2- 172.21.15.7:0/3261483359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63800b7440 0x7f63800d1c90 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f6380074510 tx=0x7f636802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.152+0000 7f6384c00640 1 -- 172.21.15.7:0/3261483359 shutdown_connections 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.152+0000 7f6384c00640 1 --2- 172.21.15.7:0/3261483359 >> v2:172.21.15.87:3300/0 conn(0x7f63801600e0 0x7f63800d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.152+0000 7f6384c00640 1 --2- 172.21.15.7:0/3261483359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63800b7440 0x7f63800d1c90 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.152+0000 7f6384c00640 1 --2- 172.21.15.7:0/3261483359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63800b1660 0x7f638006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.152+0000 7f6384c00640 1 -- 172.21.15.7:0/3261483359 >> 172.21.15.7:0/3261483359 conn(0x7f638009b540 msgr2=0x7f63800b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:49.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.152+0000 7f6384c00640 1 -- 172.21.15.7:0/3261483359 shutdown_connections 2024-04-25T11:21:49.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.152+0000 7f6384c00640 1 -- 172.21.15.7:0/3261483359 wait complete. 2024-04-25T11:21:49.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.155+0000 7f6384c00640 1 Processor -- start 2024-04-25T11:21:49.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.155+0000 7f6384c00640 1 -- start start 2024-04-25T11:21:49.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.156+0000 7f6384c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63800b1660 0x7f638009d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.156+0000 7f6384c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63800b7440 0x7f638015d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.156+0000 7f6384c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63801600e0 0x7f638015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.156+0000 7f6384c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63800b3680 con 0x7f63800b7440 2024-04-25T11:21:49.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.156+0000 7f6384c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f638006cab0 con 0x7f63800b1660 2024-04-25T11:21:49.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.156+0000 7f6384c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63800b3800 con 0x7f63801600e0 2024-04-25T11:21:49.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.157+0000 7f637ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63800b7440 0x7f638015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:49.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.157+0000 7f637ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63800b7440 0x7f638015d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46710/0 (socket says 172.21.15.7:46710) 2024-04-25T11:21:49.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.157+0000 7f637ea00640 1 -- 172.21.15.7:0/2767060458 learned_addr learned my addr 172.21.15.7:0/2767060458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.157+0000 7f637fe00640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63801600e0 0x7f638015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.157+0000 7f637ea00640 1 -- 172.21.15.7:0/2767060458 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63801600e0 msgr2=0x7f638015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.157+0000 7f637ea00640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63801600e0 0x7f638015dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.157+0000 7f637ea00640 1 -- 172.21.15.7:0/2767060458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63800b1660 msgr2=0x7f638009d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.157+0000 7f637ea00640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63800b1660 0x7f638009d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.157+0000 7f637ea00640 1 -- 172.21.15.7:0/2767060458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638009c280 con 0x7f63800b7440 2024-04-25T11:21:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.158+0000 7f637ea00640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63800b7440 0x7f638015d660 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f6368000c00 tx=0x7f63680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.158+0000 7f6377400640 1 -- 172.21.15.7:0/2767060458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368041070 con 0x7f63800b7440 2024-04-25T11:21:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.158+0000 7f6377400640 1 -- 172.21.15.7:0/2767060458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f636803c870 con 0x7f63800b7440 2024-04-25T11:21:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.158+0000 7f6377400640 1 -- 172.21.15.7:0/2767060458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6368037040 con 0x7f63800b7440 2024-04-25T11:21:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.158+0000 7f6384c00640 1 -- 172.21.15.7:0/2767060458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638009c560 con 0x7f63800b7440 2024-04-25T11:21:49.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.158+0000 7f6384c00640 1 -- 172.21.15.7:0/2767060458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638009cb20 con 0x7f63800b7440 2024-04-25T11:21:49.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.159+0000 7f6384c00640 1 -- 172.21.15.7:0/2767060458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6380050740 con 0x7f63800b7440 2024-04-25T11:21:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.161+0000 7f6377400640 1 -- 172.21.15.7:0/2767060458 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6368044070 con 0x7f63800b7440 2024-04-25T11:21:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.162+0000 7f6377400640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f635c074530 0x7f635c076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.163+0000 7f6377400640 1 -- 172.21.15.7:0/2767060458 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f63680b5f60 con 0x7f63800b7440 2024-04-25T11:21:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.163+0000 7f637f400640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f635c074530 0x7f635c076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.164+0000 7f637f400640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f635c074530 0x7f635c076a40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6370006fd0 tx=0x7f6370008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:49.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.168+0000 7f6377400640 1 -- 172.21.15.7:0/2767060458 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63680805d0 con 0x7f63800b7440 2024-04-25T11:21:49.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.374+0000 7f6384c00640 1 -- 172.21.15.7:0/2767060458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_cache_memory_limit}] v 0) -- 0x7f63800d2930 con 0x7f63800b7440 2024-04-25T11:21:49.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.381+0000 7f6377400640 1 -- 172.21.15.7:0/2767060458 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_cache_memory_limit}]=0 v57)=0 v57) ==== 121+0+0 (secure 0 0 0) 0x7f636807fed0 con 0x7f63800b7440 2024-04-25T11:21:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.384+0000 7f6384c00640 1 -- 172.21.15.7:0/2767060458 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f635c074530 msgr2=0x7f635c076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.384+0000 7f6384c00640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f635c074530 0x7f635c076a40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6370006fd0 tx=0x7f6370008040 comp rx=0 tx=0).stop 2024-04-25T11:21:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.384+0000 7f6384c00640 1 -- 172.21.15.7:0/2767060458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63800b7440 msgr2=0x7f638015d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.384+0000 7f6384c00640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63800b7440 0x7f638015d660 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f6368000c00 tx=0x7f63680046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.384+0000 7f6384c00640 1 -- 172.21.15.7:0/2767060458 shutdown_connections 2024-04-25T11:21:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.384+0000 7f6384c00640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f635c074530 0x7f635c076a40 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.384+0000 7f6384c00640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63801600e0 0x7f638015dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.384+0000 7f6384c00640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63800b7440 0x7f638015d660 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.384+0000 7f6384c00640 1 --2- 172.21.15.7:0/2767060458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63800b1660 0x7f638009d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.384+0000 7f6384c00640 1 -- 172.21.15.7:0/2767060458 >> 172.21.15.7:0/2767060458 conn(0x7f638009b540 msgr2=0x7f638015af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.385+0000 7f6384c00640 1 -- 172.21.15.7:0/2767060458 shutdown_connections 2024-04-25T11:21:49.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.385+0000 7f6384c00640 1 -- 172.21.15.7:0/2767060458 wait complete. 2024-04-25T11:21:49.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_max_caps 200 2024-04-25T11:21:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.700+0000 7f66f2600640 1 Processor -- start 2024-04-25T11:21:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.700+0000 7f66f2600640 1 -- start start 2024-04-25T11:21:49.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.702+0000 7f66f2600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66ec145e40 0x7f66ec146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.702+0000 7f66f2600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66ec0b4870 0x7f66ec0b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.702+0000 7f66f2600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66ec0b66b0 con 0x7f66ec0b51c0 2024-04-25T11:21:49.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.702+0000 7f66f2600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f66ec145b70 con 0x7f66ec0b4870 2024-04-25T11:21:49.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.702+0000 7f66f2600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f66ec0b5b30 con 0x7f66ec145e40 2024-04-25T11:21:49.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.703+0000 7f66ebe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66ec0b4870 0x7f66ec0b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:49.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.703+0000 7f66ebe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66ec0b4870 0x7f66ec0b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46712/0 (socket says 172.21.15.7:46712) 2024-04-25T11:21:49.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.703+0000 7f66ebe00640 1 -- 172.21.15.7:0/2158230034 learned_addr learned my addr 172.21.15.7:0/2158230034 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:49.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.703+0000 7f66f1200640 1 --2- 172.21.15.7:0/2158230034 >> v2:172.21.15.87:3300/0 conn(0x7f66ec145e40 0x7f66ec146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:49.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.704+0000 7f66ebe00640 1 -- 172.21.15.7:0/2158230034 >> v2:172.21.15.87:3300/0 conn(0x7f66ec145e40 msgr2=0x7f66ec146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:49.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.704+0000 7f66ebe00640 1 --2- 172.21.15.7:0/2158230034 >> v2:172.21.15.87:3300/0 conn(0x7f66ec145e40 0x7f66ec146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.704+0000 7f66ebe00640 1 -- 172.21.15.7:0/2158230034 >> v1:172.21.15.87:6789/0 conn(0x7f66ec0b51c0 legacy=0x7f66ec143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:49.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.704+0000 7f66ebe00640 1 -- 172.21.15.7:0/2158230034 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec0b5e10 con 0x7f66ec0b4870 2024-04-25T11:21:49.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.704+0000 7f66f1200640 1 --2- 172.21.15.7:0/2158230034 >> v2:172.21.15.87:3300/0 conn(0x7f66ec145e40 0x7f66ec146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:49.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.705+0000 7f66ebe00640 1 --2- 172.21.15.7:0/2158230034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0b4870 0x7f66ec0b4c80 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0730b0 tx=0x7f66d802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29d680b0dc16ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:49.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.705+0000 7f66eb400640 1 -- 172.21.15.7:0/2158230034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8037070 con 0x7f66ec0b4870 2024-04-25T11:21:49.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.705+0000 7f66eb400640 1 -- 172.21.15.7:0/2158230034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d8032440 con 0x7f66ec0b4870 2024-04-25T11:21:49.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.705+0000 7f66eb400640 1 -- 172.21.15.7:0/2158230034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8032810 con 0x7f66ec0b4870 2024-04-25T11:21:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.706+0000 7f66f2600640 1 -- 172.21.15.7:0/2158230034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0b4870 msgr2=0x7f66ec0b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.706+0000 7f66f2600640 1 --2- 172.21.15.7:0/2158230034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0b4870 0x7f66ec0b4c80 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0730b0 tx=0x7f66d802a160 comp rx=0 tx=0).stop 2024-04-25T11:21:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.706+0000 7f66f2600640 1 -- 172.21.15.7:0/2158230034 shutdown_connections 2024-04-25T11:21:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.706+0000 7f66f2600640 1 --2- 172.21.15.7:0/2158230034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0b4870 0x7f66ec0b4c80 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.706+0000 7f66f2600640 1 --2- 172.21.15.7:0/2158230034 >> v2:172.21.15.87:3300/0 conn(0x7f66ec145e40 0x7f66ec146250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.706+0000 7f66f2600640 1 -- 172.21.15.7:0/2158230034 >> 172.21.15.7:0/2158230034 conn(0x7f66ec0740f0 msgr2=0x7f66ec06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.706+0000 7f66f2600640 1 -- 172.21.15.7:0/2158230034 shutdown_connections 2024-04-25T11:21:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.706+0000 7f66f2600640 1 -- 172.21.15.7:0/2158230034 wait complete. 2024-04-25T11:21:49.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.709+0000 7f66f2600640 1 Processor -- start 2024-04-25T11:21:49.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.709+0000 7f66f2600640 1 -- start start 2024-04-25T11:21:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.710+0000 7f66f2600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66ec0b4870 0x7f66ec13f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.710+0000 7f66f2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0b51c0 0x7f66ec15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.710+0000 7f66f2600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66ec145e40 0x7f66ec15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.710+0000 7f66f2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec0b55d0 con 0x7f66ec0b51c0 2024-04-25T11:21:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.710+0000 7f66f2600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec0b63d0 con 0x7f66ec145e40 2024-04-25T11:21:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.710+0000 7f66f2600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66ec0b5750 con 0x7f66ec0b4870 2024-04-25T11:21:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.711+0000 7f66ebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0b51c0 0x7f66ec15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.711+0000 7f66ebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0b51c0 0x7f66ec15ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46722/0 (socket says 172.21.15.7:46722) 2024-04-25T11:21:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.711+0000 7f66ebe00640 1 -- 172.21.15.7:0/1359556887 learned_addr learned my addr 172.21.15.7:0/1359556887 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.711+0000 7f66ebe00640 1 -- 172.21.15.7:0/1359556887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66ec0b4870 msgr2=0x7f66ec13f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.711+0000 7f66ebe00640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66ec0b4870 0x7f66ec13f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.711+0000 7f66ebe00640 1 -- 172.21.15.7:0/1359556887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66ec145e40 msgr2=0x7f66ec15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.711+0000 7f66ebe00640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66ec145e40 0x7f66ec15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.711+0000 7f66ebe00640 1 -- 172.21.15.7:0/1359556887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec15be00 con 0x7f66ec0b51c0 2024-04-25T11:21:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.712+0000 7f66ebe00640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0b51c0 0x7f66ec15ad30 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f66d8000c00 tx=0x7f66d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.712+0000 7f66e8c00640 1 -- 172.21.15.7:0/1359556887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8041020 con 0x7f66ec0b51c0 2024-04-25T11:21:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.712+0000 7f66e8c00640 1 -- 172.21.15.7:0/1359556887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d8030970 con 0x7f66ec0b51c0 2024-04-25T11:21:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.712+0000 7f66e8c00640 1 -- 172.21.15.7:0/1359556887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8037040 con 0x7f66ec0b51c0 2024-04-25T11:21:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.712+0000 7f66f2600640 1 -- 172.21.15.7:0/1359556887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ec15c0e0 con 0x7f66ec0b51c0 2024-04-25T11:21:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.712+0000 7f66f2600640 1 -- 172.21.15.7:0/1359556887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ec168bc0 con 0x7f66ec0b51c0 2024-04-25T11:21:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.713+0000 7f66f2600640 1 -- 172.21.15.7:0/1359556887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66ec050740 con 0x7f66ec0b51c0 2024-04-25T11:21:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.715+0000 7f66e8c00640 1 -- 172.21.15.7:0/1359556887 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f66d8044020 con 0x7f66ec0b51c0 2024-04-25T11:21:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.716+0000 7f66e8c00640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66cc0744b0 0x7f66cc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.716+0000 7f66e8c00640 1 -- 172.21.15.7:0/1359556887 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f66d80b5ed0 con 0x7f66ec0b51c0 2024-04-25T11:21:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.716+0000 7f66f1200640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66cc0744b0 0x7f66cc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.717+0000 7f66f1200640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66cc0744b0 0x7f66cc0769c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f66e0006fd0 tx=0x7f66e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:49.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.720+0000 7f66e8c00640 1 -- 172.21.15.7:0/1359556887 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66d804b030 con 0x7f66ec0b51c0 2024-04-25T11:21:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.971+0000 7f66f2600640 1 -- 172.21.15.7:0/1359556887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_recall_max_caps}] v 0) -- 0x7f66ec140110 con 0x7f66ec0b51c0 2024-04-25T11:21:49.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.979+0000 7f66e8c00640 1 -- 172.21.15.7:0/1359556887 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_max_caps}]=0 v58)=0 v58) ==== 119+0+0 (secure 0 0 0) 0x7f66d8080430 con 0x7f66ec0b51c0 2024-04-25T11:21:49.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.982+0000 7f66f2600640 1 -- 172.21.15.7:0/1359556887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66cc0744b0 msgr2=0x7f66cc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:49.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.982+0000 7f66f2600640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66cc0744b0 0x7f66cc0769c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f66e0006fd0 tx=0x7f66e0008040 comp rx=0 tx=0).stop 2024-04-25T11:21:49.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.982+0000 7f66f2600640 1 -- 172.21.15.7:0/1359556887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0b51c0 msgr2=0x7f66ec15ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.982+0000 7f66f2600640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0b51c0 0x7f66ec15ad30 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f66d8000c00 tx=0x7f66d80046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.983+0000 7f66f2600640 1 -- 172.21.15.7:0/1359556887 shutdown_connections 2024-04-25T11:21:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.983+0000 7f66f2600640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66cc0744b0 0x7f66cc0769c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.983+0000 7f66f2600640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66ec145e40 0x7f66ec15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.983+0000 7f66f2600640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66ec0b51c0 0x7f66ec15ad30 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.983+0000 7f66f2600640 1 --2- 172.21.15.7:0/1359556887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66ec0b4870 0x7f66ec13f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.983+0000 7f66f2600640 1 -- 172.21.15.7:0/1359556887 >> 172.21.15.7:0/1359556887 conn(0x7f66ec0740f0 msgr2=0x7f66ec06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.983+0000 7f66f2600640 1 -- 172.21.15.7:0/1359556887 shutdown_connections 2024-04-25T11:21:49.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:49.983+0000 7f66f2600640 1 -- 172.21.15.7:0/1359556887 wait complete. 2024-04-25T11:21:50.005 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_recall_warning_threshold 400 2024-04-25T11:21:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.296+0000 7ff755200640 1 Processor -- start 2024-04-25T11:21:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.296+0000 7ff755200640 1 -- start start 2024-04-25T11:21:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.298+0000 7ff755200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff7500b2550 0x7ff7500b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.298+0000 7ff755200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff7500b2a80 0x7ff7500d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.298+0000 7ff755200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff750160770 con 0x7ff7500d2df0 2024-04-25T11:21:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.298+0000 7ff755200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff75006e960 con 0x7ff7500b2550 2024-04-25T11:21:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.298+0000 7ff755200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff7500d1dd0 con 0x7ff7500b2a80 2024-04-25T11:21:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.299+0000 7ff74f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff7500b2550 0x7ff7500b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.299+0000 7ff74f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff7500b2550 0x7ff7500b8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58586/0 (socket says 172.21.15.7:58586) 2024-04-25T11:21:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.299+0000 7ff74f400640 1 -- 172.21.15.7:0/1809358148 learned_addr learned my addr 172.21.15.7:0/1809358148 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.299+0000 7ff74f400640 1 -- 172.21.15.7:0/1809358148 >> v2:172.21.15.87:3300/0 conn(0x7ff7500b2a80 msgr2=0x7ff7500d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.299+0000 7ff74f400640 1 --2- 172.21.15.7:0/1809358148 >> v2:172.21.15.87:3300/0 conn(0x7ff7500b2a80 0x7ff7500d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.299+0000 7ff74f400640 1 -- 172.21.15.7:0/1809358148 >> v1:172.21.15.87:6789/0 conn(0x7ff7500d2df0 legacy=0x7ff7500b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.299+0000 7ff74f400640 1 -- 172.21.15.7:0/1809358148 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7500d20b0 con 0x7ff7500b2550 2024-04-25T11:21:50.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.300+0000 7ff74f400640 1 --2- 172.21.15.7:0/1809358148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7500b2550 0x7ff7500b8eb0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7ff7500d3700 tx=0x7ff74002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5111c17505222493 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:50.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.301+0000 7ff74ea00640 1 -- 172.21.15.7:0/1809358148 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740037070 con 0x7ff7500b2550 2024-04-25T11:21:50.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.301+0000 7ff74ea00640 1 -- 172.21.15.7:0/1809358148 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff74002ae20 con 0x7ff7500b2550 2024-04-25T11:21:50.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.301+0000 7ff74ea00640 1 -- 172.21.15.7:0/1809358148 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740032660 con 0x7ff7500b2550 2024-04-25T11:21:50.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.301+0000 7ff755200640 1 -- 172.21.15.7:0/1809358148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7500b2550 msgr2=0x7ff7500b8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:50.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.301+0000 7ff755200640 1 --2- 172.21.15.7:0/1809358148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7500b2550 0x7ff7500b8eb0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7ff7500d3700 tx=0x7ff74002a110 comp rx=0 tx=0).stop 2024-04-25T11:21:50.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.302+0000 7ff755200640 1 -- 172.21.15.7:0/1809358148 shutdown_connections 2024-04-25T11:21:50.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.302+0000 7ff755200640 1 --2- 172.21.15.7:0/1809358148 >> v2:172.21.15.87:3300/0 conn(0x7ff7500b2a80 0x7ff7500d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.302+0000 7ff755200640 1 --2- 172.21.15.7:0/1809358148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7500b2550 0x7ff7500b8eb0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.302+0000 7ff755200640 1 -- 172.21.15.7:0/1809358148 >> 172.21.15.7:0/1809358148 conn(0x7ff7500b5240 msgr2=0x7ff7500b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.302+0000 7ff755200640 1 -- 172.21.15.7:0/1809358148 shutdown_connections 2024-04-25T11:21:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.302+0000 7ff755200640 1 -- 172.21.15.7:0/1809358148 wait complete. 2024-04-25T11:21:50.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.304+0000 7ff755200640 1 Processor -- start 2024-04-25T11:21:50.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.304+0000 7ff755200640 1 -- start start 2024-04-25T11:21:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.306+0000 7ff755200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7500b2550 0x7ff7501f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.306+0000 7ff755200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7500b2a80 0x7ff7501f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.306+0000 7ff755200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7500d2df0 0x7ff7500723e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.306+0000 7ff755200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff75006e5f0 con 0x7ff7500b2550 2024-04-25T11:21:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.306+0000 7ff755200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff7500d1ac0 con 0x7ff7500d2df0 2024-04-25T11:21:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.306+0000 7ff755200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff7500b3700 con 0x7ff7500b2a80 2024-04-25T11:21:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.306+0000 7ff74f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7500b2550 0x7ff7501f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.306+0000 7ff74f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7500b2550 0x7ff7501f3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58602/0 (socket says 172.21.15.7:58602) 2024-04-25T11:21:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.306+0000 7ff74f400640 1 -- 172.21.15.7:0/2306590877 learned_addr learned my addr 172.21.15.7:0/2306590877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff74fe00640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7500d2df0 0x7ff7500723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff74f400640 1 -- 172.21.15.7:0/2306590877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7500b2a80 msgr2=0x7ff7501f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff747e00640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7500b2a80 0x7ff7501f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff74f400640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7500b2a80 0x7ff7501f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff74f400640 1 -- 172.21.15.7:0/2306590877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7500d2df0 msgr2=0x7ff7500723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff74f400640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7500d2df0 0x7ff7500723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff74f400640 1 -- 172.21.15.7:0/2306590877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff750162a10 con 0x7ff7500b2550 2024-04-25T11:21:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff74fe00640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7500d2df0 0x7ff7500723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff747e00640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7500b2a80 0x7ff7501f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff74f400640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7500b2550 0x7ff7501f3ff0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7ff74002a150 tx=0x7ff7400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff747400640 1 -- 172.21.15.7:0/2306590877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740042070 con 0x7ff7500b2550 2024-04-25T11:21:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.307+0000 7ff747400640 1 -- 172.21.15.7:0/2306590877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff74003e670 con 0x7ff7500b2550 2024-04-25T11:21:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.308+0000 7ff747400640 1 -- 172.21.15.7:0/2306590877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740037040 con 0x7ff7500b2550 2024-04-25T11:21:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.308+0000 7ff755200640 1 -- 172.21.15.7:0/2306590877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7501f7660 con 0x7ff7500b2550 2024-04-25T11:21:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.308+0000 7ff755200640 1 -- 172.21.15.7:0/2306590877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7501609e0 con 0x7ff7500b2550 2024-04-25T11:21:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.309+0000 7ff755200640 1 -- 172.21.15.7:0/2306590877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff750050740 con 0x7ff7500b2550 2024-04-25T11:21:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.310+0000 7ff747400640 1 -- 172.21.15.7:0/2306590877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff740045070 con 0x7ff7500b2550 2024-04-25T11:21:50.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.311+0000 7ff747400640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff72c0744b0 0x7ff72c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.311+0000 7ff747400640 1 -- 172.21.15.7:0/2306590877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7ff7400b5b80 con 0x7ff7500b2550 2024-04-25T11:21:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.311+0000 7ff747e00640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff72c0744b0 0x7ff72c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:50.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.312+0000 7ff747e00640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff72c0744b0 0x7ff72c0769c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff7500b8690 tx=0x7ff738008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:50.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.316+0000 7ff747400640 1 -- 172.21.15.7:0/2306590877 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff74002ae20 con 0x7ff7500b2550 2024-04-25T11:21:50.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.523+0000 7ff755200640 1 -- 172.21.15.7:0/2306590877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_recall_warning_threshold}] v 0) -- 0x7ff75009c710 con 0x7ff7500b2550 2024-04-25T11:21:50.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.530+0000 7ff747400640 1 -- 172.21.15.7:0/2306590877 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_recall_warning_threshold}]=0 v59)=0 v59) ==== 128+0+0 (secure 0 0 0) 0x7ff750162a10 con 0x7ff7500b2550 2024-04-25T11:21:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.533+0000 7ff755200640 1 -- 172.21.15.7:0/2306590877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff72c0744b0 msgr2=0x7ff72c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.533+0000 7ff755200640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff72c0744b0 0x7ff72c0769c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff7500b8690 tx=0x7ff738008040 comp rx=0 tx=0).stop 2024-04-25T11:21:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.533+0000 7ff755200640 1 -- 172.21.15.7:0/2306590877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7500b2550 msgr2=0x7ff7501f3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.533+0000 7ff755200640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7500b2550 0x7ff7501f3ff0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7ff74002a150 tx=0x7ff7400046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.534+0000 7ff755200640 1 -- 172.21.15.7:0/2306590877 shutdown_connections 2024-04-25T11:21:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.534+0000 7ff755200640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff72c0744b0 0x7ff72c0769c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.534+0000 7ff755200640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7500d2df0 0x7ff7500723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.534+0000 7ff755200640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7500b2a80 0x7ff7501f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.534+0000 7ff755200640 1 --2- 172.21.15.7:0/2306590877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7500b2550 0x7ff7501f3ff0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.534+0000 7ff755200640 1 -- 172.21.15.7:0/2306590877 >> 172.21.15.7:0/2306590877 conn(0x7ff7500b5240 msgr2=0x7ff75015fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.534+0000 7ff755200640 1 -- 172.21.15.7:0/2306590877 shutdown_connections 2024-04-25T11:21:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.534+0000 7ff755200640 1 -- 172.21.15.7:0/2306590877 wait complete. 2024-04-25T11:21:50.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_min_caps_per_client 2024-04-25T11:21:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.851+0000 7f546a400640 1 Processor -- start 2024-04-25T11:21:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.851+0000 7f546a400640 1 -- start start 2024-04-25T11:21:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.854+0000 7f546a400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f54640d4d80 0x7f54640d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.854+0000 7f546a400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5464161df0 0x7f54640d2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.854+0000 7f546a400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5464161030 con 0x7f54640d5190 2024-04-25T11:21:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.854+0000 7f546a400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5464161310 con 0x7f54640d4d80 2024-04-25T11:21:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.854+0000 7f546a400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f54640d4580 con 0x7f5464161df0 2024-04-25T11:21:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.854+0000 7f5469000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f54640d4d80 0x7f54640d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.854+0000 7f5469000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f54640d4d80 0x7f54640d2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58614/0 (socket says 172.21.15.7:58614) 2024-04-25T11:21:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.854+0000 7f5469000640 1 -- 172.21.15.7:0/2043926260 learned_addr learned my addr 172.21.15.7:0/2043926260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.855+0000 7f5469000640 1 -- 172.21.15.7:0/2043926260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5464161df0 msgr2=0x7f54640d2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.855+0000 7f5469000640 1 --2- 172.21.15.7:0/2043926260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5464161df0 0x7f54640d2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.855+0000 7f5469000640 1 -- 172.21.15.7:0/2043926260 >> v1:172.21.15.87:6789/0 conn(0x7f54640d5190 legacy=0x7f54641616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.855+0000 7f5469000640 1 -- 172.21.15.7:0/2043926260 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54640d4860 con 0x7f54640d4d80 2024-04-25T11:21:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.856+0000 7f5469000640 1 --2- 172.21.15.7:0/2043926260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54640d4d80 0x7f54640d2780 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f5454009e50 tx=0x7f545402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932eae1bee990a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.856+0000 7f5463400640 1 -- 172.21.15.7:0/2043926260 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454036070 con 0x7f54640d4d80 2024-04-25T11:21:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.856+0000 7f5463400640 1 -- 172.21.15.7:0/2043926260 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5454031600 con 0x7f54640d4d80 2024-04-25T11:21:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.856+0000 7f5463400640 1 -- 172.21.15.7:0/2043926260 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54540319d0 con 0x7f54640d4d80 2024-04-25T11:21:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.857+0000 7f546a400640 1 -- 172.21.15.7:0/2043926260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54640d4d80 msgr2=0x7f54640d2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.857+0000 7f546a400640 1 --2- 172.21.15.7:0/2043926260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54640d4d80 0x7f54640d2780 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f5454009e50 tx=0x7f545402a6c0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.857+0000 7f546a400640 1 -- 172.21.15.7:0/2043926260 shutdown_connections 2024-04-25T11:21:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.857+0000 7f546a400640 1 --2- 172.21.15.7:0/2043926260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5464161df0 0x7f54640d2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.857+0000 7f546a400640 1 --2- 172.21.15.7:0/2043926260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54640d4d80 0x7f54640d2780 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.857+0000 7f546a400640 1 -- 172.21.15.7:0/2043926260 >> 172.21.15.7:0/2043926260 conn(0x7f546406c750 msgr2=0x7f546406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.857+0000 7f546a400640 1 -- 172.21.15.7:0/2043926260 shutdown_connections 2024-04-25T11:21:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.857+0000 7f546a400640 1 -- 172.21.15.7:0/2043926260 wait complete. 2024-04-25T11:21:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.860+0000 7f546a400640 1 Processor -- start 2024-04-25T11:21:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.860+0000 7f546a400640 1 -- start start 2024-04-25T11:21:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f546a400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54640d4d80 0x7f54640b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f546a400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54640d5190 0x7f54640cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f546a400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5464161df0 0x7f54640cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f546a400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5464160320 con 0x7f54640d4d80 2024-04-25T11:21:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f546a400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54641601a0 con 0x7f54640d5190 2024-04-25T11:21:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f546a400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f54641604a0 con 0x7f5464161df0 2024-04-25T11:21:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f5469000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54640d4d80 0x7f54640b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f5469000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54640d4d80 0x7f54640b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58618/0 (socket says 172.21.15.7:58618) 2024-04-25T11:21:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f5469000640 1 -- 172.21.15.7:0/4071863980 learned_addr learned my addr 172.21.15.7:0/4071863980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:50.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f5469a00640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5464161df0 0x7f54640cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:50.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f5469000640 1 -- 172.21.15.7:0/4071863980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5464161df0 msgr2=0x7f54640cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:50.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f5469000640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5464161df0 0x7f54640cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f5469000640 1 -- 172.21.15.7:0/4071863980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54640d5190 msgr2=0x7f54640cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:50.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f5469000640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54640d5190 0x7f54640cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:50.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f5469000640 1 -- 172.21.15.7:0/4071863980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54640b5a50 con 0x7f54640d4d80 2024-04-25T11:21:50.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.862+0000 7f5469a00640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5464161df0 0x7f54640cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:50.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.863+0000 7f5469000640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54640d4d80 0x7f54640b5100 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f5454034040 tx=0x7f5454004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:50.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.863+0000 7f5460c00640 1 -- 172.21.15.7:0/4071863980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454036050 con 0x7f54640d4d80 2024-04-25T11:21:50.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.863+0000 7f5460c00640 1 -- 172.21.15.7:0/4071863980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5454042070 con 0x7f54640d4d80 2024-04-25T11:21:50.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.863+0000 7f5460c00640 1 -- 172.21.15.7:0/4071863980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454009510 con 0x7f54640d4d80 2024-04-25T11:21:50.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.863+0000 7f546a400640 1 -- 172.21.15.7:0/4071863980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54640b5d30 con 0x7f54640d4d80 2024-04-25T11:21:50.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.863+0000 7f546a400640 1 -- 172.21.15.7:0/4071863980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54640b62f0 con 0x7f54640d4d80 2024-04-25T11:21:50.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.865+0000 7f546a400640 1 -- 172.21.15.7:0/4071863980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5464050740 con 0x7f54640d4d80 2024-04-25T11:21:50.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.866+0000 7f5460c00640 1 -- 172.21.15.7:0/4071863980 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5454007c50 con 0x7f54640d4d80 2024-04-25T11:21:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.867+0000 7f5460c00640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54400744b0 0x7f54400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.867+0000 7f5460c00640 1 -- 172.21.15.7:0/4071863980 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f54540b65c0 con 0x7f54640d4d80 2024-04-25T11:21:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.867+0000 7f5463e00640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54400744b0 0x7f54400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:50.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.868+0000 7f5463e00640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54400744b0 0x7f54400769c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f544c006fd0 tx=0x7f544c006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:50.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:50.872+0000 7f5460c00640 1 -- 172.21.15.7:0/4071863980 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5454080bb0 con 0x7f54640d4d80 2024-04-25T11:21:51.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.076+0000 7f546a400640 1 -- 172.21.15.7:0/4071863980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"} v 0) -- 0x7f546406d030 con 0x7f54640d4d80 2024-04-25T11:21:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.077+0000 7f5460c00640 1 -- 172.21.15.7:0/4071863980 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_min_caps_per_client"}]=0 v59) ==== 106+0+4 (secure 0 0 0) 0x7f54540804b0 con 0x7f54640d4d80 2024-04-25T11:21:51.073 INFO:teuthology.orchestra.run.smithi007.stdout:100 2024-04-25T11:21:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 -- 172.21.15.7:0/4071863980 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54400744b0 msgr2=0x7f54400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54400744b0 0x7f54400769c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f544c006fd0 tx=0x7f544c006bc0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 -- 172.21.15.7:0/4071863980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54640d4d80 msgr2=0x7f54640b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54640d4d80 0x7f54640b5100 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f5454034040 tx=0x7f5454004a40 comp rx=0 tx=0).stop 2024-04-25T11:21:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 -- 172.21.15.7:0/4071863980 shutdown_connections 2024-04-25T11:21:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f54400744b0 0x7f54400769c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5464161df0 0x7f54640cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54640d5190 0x7f54640cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 --2- 172.21.15.7:0/4071863980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54640d4d80 0x7f54640b5100 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 -- 172.21.15.7:0/4071863980 >> 172.21.15.7:0/4071863980 conn(0x7f546406c750 msgr2=0x7f546415cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 -- 172.21.15.7:0/4071863980 shutdown_connections 2024-04-25T11:21:51.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.080+0000 7f546a400640 1 -- 172.21.15.7:0/4071863980 wait complete. 2024-04-25T11:21:51.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_min_caps_working_set 100 2024-04-25T11:21:51.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.400+0000 7f646ba00640 1 Processor -- start 2024-04-25T11:21:51.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.400+0000 7f646ba00640 1 -- start start 2024-04-25T11:21:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.402+0000 7f646ba00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64640b0470 0x7f646406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.402+0000 7f646ba00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f646406f8e0 0x7f64640aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.402+0000 7f646ba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f646415e3c0 con 0x7f646415dfb0 2024-04-25T11:21:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.402+0000 7f646ba00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f64640cf800 con 0x7f64640b0470 2024-04-25T11:21:51.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.402+0000 7f646ba00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f64640d1250 con 0x7f646406f8e0 2024-04-25T11:21:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.403+0000 7f6463e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64640b0470 0x7f646406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.403+0000 7f6463e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64640b0470 0x7f646406f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58622/0 (socket says 172.21.15.7:58622) 2024-04-25T11:21:51.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.403+0000 7f6463e00640 1 -- 172.21.15.7:0/4215216563 learned_addr learned my addr 172.21.15.7:0/4215216563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.403+0000 7f6469a00640 1 --2- 172.21.15.7:0/4215216563 >> v2:172.21.15.87:3300/0 conn(0x7f646406f8e0 0x7f64640aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.404+0000 7f6463e00640 1 -- 172.21.15.7:0/4215216563 >> v2:172.21.15.87:3300/0 conn(0x7f646406f8e0 msgr2=0x7f64640aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.404+0000 7f6463e00640 1 --2- 172.21.15.7:0/4215216563 >> v2:172.21.15.87:3300/0 conn(0x7f646406f8e0 0x7f64640aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.404+0000 7f6463e00640 1 -- 172.21.15.7:0/4215216563 >> v1:172.21.15.87:6789/0 conn(0x7f646415dfb0 legacy=0x7f64640d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.404+0000 7f6463e00640 1 -- 172.21.15.7:0/4215216563 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64640cfaa0 con 0x7f64640b0470 2024-04-25T11:21:51.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.406+0000 7f6463e00640 1 --2- 172.21.15.7:0/4215216563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64640b0470 0x7f646406f3a0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f646409fb50 tx=0x7f644c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13be55e81afae2f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:51.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.406+0000 7f6463400640 1 -- 172.21.15.7:0/4215216563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c037070 con 0x7f64640b0470 2024-04-25T11:21:51.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.406+0000 7f6463400640 1 -- 172.21.15.7:0/4215216563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644c032440 con 0x7f64640b0470 2024-04-25T11:21:51.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.406+0000 7f6463400640 1 -- 172.21.15.7:0/4215216563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c032810 con 0x7f64640b0470 2024-04-25T11:21:51.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.407+0000 7f646ba00640 1 -- 172.21.15.7:0/4215216563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64640b0470 msgr2=0x7f646406f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:51.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.407+0000 7f646ba00640 1 --2- 172.21.15.7:0/4215216563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64640b0470 0x7f646406f3a0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f646409fb50 tx=0x7f644c02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.407+0000 7f646ba00640 1 -- 172.21.15.7:0/4215216563 shutdown_connections 2024-04-25T11:21:51.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.407+0000 7f646ba00640 1 --2- 172.21.15.7:0/4215216563 >> v2:172.21.15.87:3300/0 conn(0x7f646406f8e0 0x7f64640aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.407+0000 7f646ba00640 1 --2- 172.21.15.7:0/4215216563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64640b0470 0x7f646406f3a0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.407+0000 7f646ba00640 1 -- 172.21.15.7:0/4215216563 >> 172.21.15.7:0/4215216563 conn(0x7f64640742c0 msgr2=0x7f646406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:51.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.407+0000 7f646ba00640 1 -- 172.21.15.7:0/4215216563 shutdown_connections 2024-04-25T11:21:51.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.407+0000 7f646ba00640 1 -- 172.21.15.7:0/4215216563 wait complete. 2024-04-25T11:21:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.410+0000 7f646ba00640 1 Processor -- start 2024-04-25T11:21:51.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.410+0000 7f646ba00640 1 -- start start 2024-04-25T11:21:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.411+0000 7f646ba00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646406f8e0 0x7f64641441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.411+0000 7f646ba00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64640b0470 0x7f64641551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.411+0000 7f646ba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f646415dfb0 0x7f6464155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.411+0000 7f646ba00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f64640d0340 con 0x7f646406f8e0 2024-04-25T11:21:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.411+0000 7f646ba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f64640d0060 con 0x7f646415dfb0 2024-04-25T11:21:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.411+0000 7f646ba00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f64640d0620 con 0x7f64640b0470 2024-04-25T11:21:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.412+0000 7f6469000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646406f8e0 0x7f64641441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.412+0000 7f6469000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646406f8e0 0x7f64641441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58630/0 (socket says 172.21.15.7:58630) 2024-04-25T11:21:51.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.412+0000 7f6469000640 1 -- 172.21.15.7:0/4167491828 learned_addr learned my addr 172.21.15.7:0/4167491828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.412+0000 7f6463e00640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64640b0470 0x7f64641551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.412+0000 7f6469a00640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f646415dfb0 0x7f6464155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.412+0000 7f6469000640 1 -- 172.21.15.7:0/4167491828 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64640b0470 msgr2=0x7f64641551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.412+0000 7f6469000640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64640b0470 0x7f64641551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.412+0000 7f6469000640 1 -- 172.21.15.7:0/4167491828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f646415dfb0 msgr2=0x7f6464155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.412+0000 7f6469000640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f646415dfb0 0x7f6464155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.412+0000 7f6469000640 1 -- 172.21.15.7:0/4167491828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6464160ec0 con 0x7f646406f8e0 2024-04-25T11:21:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.413+0000 7f6469000640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646406f8e0 0x7f64641441f0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f645400c6b0 tx=0x7f645400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.413+0000 7f6460c00640 1 -- 172.21.15.7:0/4167491828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454016020 con 0x7f646406f8e0 2024-04-25T11:21:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.413+0000 7f6460c00640 1 -- 172.21.15.7:0/4167491828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6454006d50 con 0x7f646406f8e0 2024-04-25T11:21:51.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.413+0000 7f6460c00640 1 -- 172.21.15.7:0/4167491828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454003bb0 con 0x7f646406f8e0 2024-04-25T11:21:51.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.413+0000 7f646ba00640 1 -- 172.21.15.7:0/4167491828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64641611a0 con 0x7f646406f8e0 2024-04-25T11:21:51.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.413+0000 7f646ba00640 1 -- 172.21.15.7:0/4167491828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64641e7bf0 con 0x7f646406f8e0 2024-04-25T11:21:51.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.414+0000 7f646ba00640 1 -- 172.21.15.7:0/4167491828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6464050740 con 0x7f646406f8e0 2024-04-25T11:21:51.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.416+0000 7f6460c00640 1 -- 172.21.15.7:0/4167491828 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f64540050d0 con 0x7f646406f8e0 2024-04-25T11:21:51.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.417+0000 7f6460c00640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64400744e0 0x7f64400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.417+0000 7f6460c00640 1 -- 172.21.15.7:0/4167491828 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f6454094fe0 con 0x7f646406f8e0 2024-04-25T11:21:51.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.417+0000 7f6463e00640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64400744e0 0x7f64400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.418+0000 7f6463e00640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64400744e0 0x7f64400769f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f644c034000 tx=0x7f644c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:51.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.422+0000 7f6460c00640 1 -- 172.21.15.7:0/4167491828 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6454020020 con 0x7f646406f8e0 2024-04-25T11:21:51.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.627+0000 7f646ba00640 1 -- 172.21.15.7:0/4167491828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_min_caps_working_set}] v 0) -- 0x7f6464070fb0 con 0x7f646406f8e0 2024-04-25T11:21:51.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.633+0000 7f6460c00640 1 -- 172.21.15.7:0/4167491828 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_min_caps_working_set}]=0 v60)=0 v60) ==== 124+0+0 (secure 0 0 0) 0x7f645405f5f0 con 0x7f646406f8e0 2024-04-25T11:21:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.636+0000 7f646ba00640 1 -- 172.21.15.7:0/4167491828 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64400744e0 msgr2=0x7f64400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:51.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.636+0000 7f646ba00640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64400744e0 0x7f64400769f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f644c034000 tx=0x7f644c02e000 comp rx=0 tx=0).stop 2024-04-25T11:21:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.636+0000 7f646ba00640 1 -- 172.21.15.7:0/4167491828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646406f8e0 msgr2=0x7f64641441f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.636+0000 7f646ba00640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646406f8e0 0x7f64641441f0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f645400c6b0 tx=0x7f645400cb70 comp rx=0 tx=0).stop 2024-04-25T11:21:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.637+0000 7f646ba00640 1 -- 172.21.15.7:0/4167491828 shutdown_connections 2024-04-25T11:21:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.637+0000 7f646ba00640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f64400744e0 0x7f64400769f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.637+0000 7f646ba00640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f646415dfb0 0x7f6464155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.637+0000 7f646ba00640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64640b0470 0x7f64641551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.637+0000 7f646ba00640 1 --2- 172.21.15.7:0/4167491828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f646406f8e0 0x7f64641441f0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.637+0000 7f646ba00640 1 -- 172.21.15.7:0/4167491828 >> 172.21.15.7:0/4167491828 conn(0x7f64640742c0 msgr2=0x7f646406cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.637+0000 7f646ba00640 1 -- 172.21.15.7:0/4167491828 shutdown_connections 2024-04-25T11:21:51.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.637+0000 7f646ba00640 1 -- 172.21.15.7:0/4167491828 wait complete. 2024-04-25T11:21:51.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_max_caps_per_client 2024-04-25T11:21:51.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.949+0000 7f7702c00640 1 Processor -- start 2024-04-25T11:21:51.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.949+0000 7f7702c00640 1 -- start start 2024-04-25T11:21:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.951+0000 7f7702c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76fc0b2550 0x7f76fc09bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.951+0000 7f7702c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76fc09c080 0x7f76fc0d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.951+0000 7f7702c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76fc0b2b40 con 0x7f76fc0d1d00 2024-04-25T11:21:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.951+0000 7f7702c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f76fc0b4180 con 0x7f76fc09c080 2024-04-25T11:21:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.951+0000 7f7702c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f76fc0b2e20 con 0x7f76fc0b2550 2024-04-25T11:21:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.952+0000 7f7702200640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f76fc0d1d00 0x7f76fc162600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:38016/0 (socket says 172.21.15.7:38016) 2024-04-25T11:21:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.952+0000 7f7702200640 1 -- 172.21.15.7:0/2976601842 learned_addr learned my addr 172.21.15.7:0/2976601842 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.952+0000 7f7701800640 1 --2- 172.21.15.7:0/2976601842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76fc0b2550 0x7f76fc09bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.952+0000 7f7700e00640 1 --2- 172.21.15.7:0/2976601842 >> v2:172.21.15.87:3300/0 conn(0x7f76fc09c080 0x7f76fc0d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.953+0000 7f76fbe00640 1 -- 172.21.15.7:0/2976601842 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671954934 0 0) 0x7f76fc0b2b40 con 0x7f76fc0d1d00 2024-04-25T11:21:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.953+0000 7f76fbe00640 1 -- 172.21.15.7:0/2976601842 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76e4003520 con 0x7f76fc0d1d00 2024-04-25T11:21:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.953+0000 7f7701800640 1 -- 172.21.15.7:0/2976601842 >> v2:172.21.15.87:3300/0 conn(0x7f76fc09c080 msgr2=0x7f76fc0d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.953+0000 7f7701800640 1 --2- 172.21.15.7:0/2976601842 >> v2:172.21.15.87:3300/0 conn(0x7f76fc09c080 0x7f76fc0d17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.953+0000 7f7701800640 1 -- 172.21.15.7:0/2976601842 >> v1:172.21.15.87:6789/0 conn(0x7f76fc0d1d00 legacy=0x7f76fc162600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.953+0000 7f7701800640 1 -- 172.21.15.7:0/2976601842 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc0b3b30 con 0x7f76fc0b2550 2024-04-25T11:21:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.953+0000 7f7700e00640 1 --2- 172.21.15.7:0/2976601842 >> v2:172.21.15.87:3300/0 conn(0x7f76fc09c080 0x7f76fc0d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:21:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.955+0000 7f7701800640 1 --2- 172.21.15.7:0/2976601842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76fc0b2550 0x7f76fc09bb40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f76f00097a0 tx=0x7f76f002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aab0770c23f049be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.955+0000 7f76fbe00640 1 -- 172.21.15.7:0/2976601842 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0037070 con 0x7f76fc0b2550 2024-04-25T11:21:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.956+0000 7f76fbe00640 1 -- 172.21.15.7:0/2976601842 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f002ae50 con 0x7f76fc0b2550 2024-04-25T11:21:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.956+0000 7f76fbe00640 1 -- 172.21.15.7:0/2976601842 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f0032660 con 0x7f76fc0b2550 2024-04-25T11:21:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.956+0000 7f7702c00640 1 -- 172.21.15.7:0/2976601842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76fc0b2550 msgr2=0x7f76fc09bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.956+0000 7f7702c00640 1 --2- 172.21.15.7:0/2976601842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76fc0b2550 0x7f76fc09bb40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f76f00097a0 tx=0x7f76f002a100 comp rx=0 tx=0).stop 2024-04-25T11:21:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.956+0000 7f7702c00640 1 -- 172.21.15.7:0/2976601842 shutdown_connections 2024-04-25T11:21:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.956+0000 7f7702c00640 1 --2- 172.21.15.7:0/2976601842 >> v2:172.21.15.87:3300/0 conn(0x7f76fc09c080 0x7f76fc0d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.956+0000 7f7702c00640 1 --2- 172.21.15.7:0/2976601842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76fc0b2550 0x7f76fc09bb40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.956+0000 7f7702c00640 1 -- 172.21.15.7:0/2976601842 >> 172.21.15.7:0/2976601842 conn(0x7f76fc0b5240 msgr2=0x7f76fc0b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.957+0000 7f7702c00640 1 -- 172.21.15.7:0/2976601842 shutdown_connections 2024-04-25T11:21:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.957+0000 7f7702c00640 1 -- 172.21.15.7:0/2976601842 wait complete. 2024-04-25T11:21:51.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.959+0000 7f7702c00640 1 Processor -- start 2024-04-25T11:21:51.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.959+0000 7f7702c00640 1 -- start start 2024-04-25T11:21:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.961+0000 7f7702c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76fc09c080 0x7f76fc0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.961+0000 7f7702c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76fc0b2550 0x7f76fc0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.961+0000 7f7702c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76fc0d1d00 0x7f76fc0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.961+0000 7f7702c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc0b4f60 con 0x7f76fc0d1d00 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.961+0000 7f7702c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f76fc0b4de0 con 0x7f76fc09c080 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.961+0000 7f7702c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f76fc0b4680 con 0x7f76fc0b2550 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.961+0000 7f7702200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76fc0d1d00 0x7f76fc0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.962+0000 7f7702200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76fc0d1d00 0x7f76fc0cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58638/0 (socket says 172.21.15.7:58638) 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.962+0000 7f7702200640 1 -- 172.21.15.7:0/3299951280 learned_addr learned my addr 172.21.15.7:0/3299951280 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.962+0000 7f7701800640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76fc09c080 0x7f76fc0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.962+0000 7f7700e00640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76fc0b2550 0x7f76fc0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.962+0000 7f7702200640 1 -- 172.21.15.7:0/3299951280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76fc0b2550 msgr2=0x7f76fc0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.962+0000 7f7702200640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76fc0b2550 0x7f76fc0cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.962+0000 7f7702200640 1 -- 172.21.15.7:0/3299951280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76fc09c080 msgr2=0x7f76fc0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.962+0000 7f7702200640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76fc09c080 0x7f76fc0ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.962+0000 7f7702200640 1 -- 172.21.15.7:0/3299951280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76fc09ee40 con 0x7f76fc0d1d00 2024-04-25T11:21:51.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.962+0000 7f7702200640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76fc0d1d00 0x7f76fc0cb690 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f76f400c390 tx=0x7f76f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:51.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.963+0000 7f76f9600640 1 -- 172.21.15.7:0/3299951280 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4017070 con 0x7f76fc0d1d00 2024-04-25T11:21:51.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.963+0000 7f76f9600640 1 -- 172.21.15.7:0/3299951280 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f4005ce0 con 0x7f76fc0d1d00 2024-04-25T11:21:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.963+0000 7f76f9600640 1 -- 172.21.15.7:0/3299951280 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4007310 con 0x7f76fc0d1d00 2024-04-25T11:21:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.963+0000 7f7702c00640 1 -- 172.21.15.7:0/3299951280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76fc1d9110 con 0x7f76fc0d1d00 2024-04-25T11:21:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.963+0000 7f7702c00640 1 -- 172.21.15.7:0/3299951280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76fc06da90 con 0x7f76fc0d1d00 2024-04-25T11:21:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.964+0000 7f7702c00640 1 -- 172.21.15.7:0/3299951280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76fc050740 con 0x7f76fc0d1d00 2024-04-25T11:21:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.965+0000 7f76f9600640 1 -- 172.21.15.7:0/3299951280 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f76f401a070 con 0x7f76fc0d1d00 2024-04-25T11:21:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.967+0000 7f76f9600640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76e00744e0 0x7f76e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.967+0000 7f76f9600640 1 -- 172.21.15.7:0/3299951280 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f76f4094620 con 0x7f76fc0d1d00 2024-04-25T11:21:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.967+0000 7f7701800640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76e00744e0 0x7f76e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:51.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.968+0000 7f7701800640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76e00744e0 0x7f76e00769f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f76f0034000 tx=0x7f76f002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:51.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:51.972+0000 7f76f9600640 1 -- 172.21.15.7:0/3299951280 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f76f405e650 con 0x7f76fc0d1d00 2024-04-25T11:21:52.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.178+0000 7f7702c00640 1 -- 172.21.15.7:0/3299951280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"} v 0) -- 0x7f76fc09f010 con 0x7f76fc0d1d00 2024-04-25T11:21:52.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.179+0000 7f76f9600640 1 -- 172.21.15.7:0/3299951280 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_max_caps_per_client"}]=0 v60) ==== 106+0+8 (secure 0 0 0) 0x7f76f4062950 con 0x7f76fc0d1d00 2024-04-25T11:21:52.175 INFO:teuthology.orchestra.run.smithi007.stdout:1048576 2024-04-25T11:21:52.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.182+0000 7f7702c00640 1 -- 172.21.15.7:0/3299951280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76e00744e0 msgr2=0x7f76e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:52.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.182+0000 7f7702c00640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76e00744e0 0x7f76e00769f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f76f0034000 tx=0x7f76f002e000 comp rx=0 tx=0).stop 2024-04-25T11:21:52.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.182+0000 7f7702c00640 1 -- 172.21.15.7:0/3299951280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76fc0d1d00 msgr2=0x7f76fc0cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.182+0000 7f7702c00640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76fc0d1d00 0x7f76fc0cb690 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f76f400c390 tx=0x7f76f400c850 comp rx=0 tx=0).stop 2024-04-25T11:21:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.182+0000 7f7702c00640 1 -- 172.21.15.7:0/3299951280 shutdown_connections 2024-04-25T11:21:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.182+0000 7f7702c00640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f76e00744e0 0x7f76e00769f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.182+0000 7f7702c00640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76fc0d1d00 0x7f76fc0cb690 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.182+0000 7f7702c00640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76fc0b2550 0x7f76fc0cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.182+0000 7f7702c00640 1 --2- 172.21.15.7:0/3299951280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76fc09c080 0x7f76fc0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.182+0000 7f7702c00640 1 -- 172.21.15.7:0/3299951280 >> 172.21.15.7:0/3299951280 conn(0x7f76fc0b5240 msgr2=0x7f76fc0b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.183+0000 7f7702c00640 1 -- 172.21.15.7:0/3299951280 shutdown_connections 2024-04-25T11:21:52.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.183+0000 7f7702c00640 1 -- 172.21.15.7:0/3299951280 wait complete. 2024-04-25T11:21:52.206 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mds mds_recall_warning_decay_rate 2024-04-25T11:21:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.485+0000 7fab75e00640 1 Processor -- start 2024-04-25T11:21:52.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.485+0000 7fab75e00640 1 -- start start 2024-04-25T11:21:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.487+0000 7fab75e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab7015a7a0 0x7fab70161160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.487+0000 7fab75e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab7006c570 0x7fab7006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.487+0000 7fab75e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fab700b7440 0x7fab700b7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.487+0000 7fab75e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fab70160cd0 con 0x7fab7015a7a0 2024-04-25T11:21:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.487+0000 7fab75e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fab700b7d90 con 0x7fab700b7440 2024-04-25T11:21:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.487+0000 7fab75e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fab7006cfc0 con 0x7fab7006c570 2024-04-25T11:21:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.488+0000 7fab74a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab7015a7a0 0x7fab70161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.488+0000 7fab74a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab7015a7a0 0x7fab70161160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58640/0 (socket says 172.21.15.7:58640) 2024-04-25T11:21:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.488+0000 7fab74a00640 1 -- 172.21.15.7:0/48111587 learned_addr learned my addr 172.21.15.7:0/48111587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.488+0000 7fab75400640 1 --2- 172.21.15.7:0/48111587 >> v2:172.21.15.87:3300/0 conn(0x7fab700b7440 0x7fab700b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.488+0000 7fab6fe00640 1 --2- 172.21.15.7:0/48111587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab7006c570 0x7fab7006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.488+0000 7fab74a00640 1 -- 172.21.15.7:0/48111587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab7006c570 msgr2=0x7fab7006c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.488+0000 7fab74a00640 1 --2- 172.21.15.7:0/48111587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab7006c570 0x7fab7006c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.488+0000 7fab74a00640 1 -- 172.21.15.7:0/48111587 >> v2:172.21.15.87:3300/0 conn(0x7fab700b7440 msgr2=0x7fab700b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.488+0000 7fab74a00640 1 --2- 172.21.15.7:0/48111587 >> v2:172.21.15.87:3300/0 conn(0x7fab700b7440 0x7fab700b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.488+0000 7fab74a00640 1 -- 172.21.15.7:0/48111587 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7006d2a0 con 0x7fab7015a7a0 2024-04-25T11:21:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.488+0000 7fab75400640 1 --2- 172.21.15.7:0/48111587 >> v2:172.21.15.87:3300/0 conn(0x7fab700b7440 0x7fab700b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:21:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.490+0000 7fab74a00640 1 --2- 172.21.15.7:0/48111587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7015a7a0 0x7fab70161160 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fab700b8d50 tx=0x7fab5802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac856fe226d6c22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.490+0000 7fab6f400640 1 -- 172.21.15.7:0/48111587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58037070 con 0x7fab7015a7a0 2024-04-25T11:21:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.490+0000 7fab6f400640 1 -- 172.21.15.7:0/48111587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab58032440 con 0x7fab7015a7a0 2024-04-25T11:21:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.490+0000 7fab6f400640 1 -- 172.21.15.7:0/48111587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58032810 con 0x7fab7015a7a0 2024-04-25T11:21:52.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.491+0000 7fab75e00640 1 -- 172.21.15.7:0/48111587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7015a7a0 msgr2=0x7fab70161160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.491+0000 7fab75e00640 1 --2- 172.21.15.7:0/48111587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7015a7a0 0x7fab70161160 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fab700b8d50 tx=0x7fab5802a140 comp rx=0 tx=0).stop 2024-04-25T11:21:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.491+0000 7fab75e00640 1 -- 172.21.15.7:0/48111587 shutdown_connections 2024-04-25T11:21:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.491+0000 7fab75e00640 1 --2- 172.21.15.7:0/48111587 >> v2:172.21.15.87:3300/0 conn(0x7fab700b7440 0x7fab700b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.491+0000 7fab75e00640 1 --2- 172.21.15.7:0/48111587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab7006c570 0x7fab7006c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.491+0000 7fab75e00640 1 --2- 172.21.15.7:0/48111587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7015a7a0 0x7fab70161160 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.491+0000 7fab75e00640 1 -- 172.21.15.7:0/48111587 >> 172.21.15.7:0/48111587 conn(0x7fab700d59c0 msgr2=0x7fab700d5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.491+0000 7fab75e00640 1 -- 172.21.15.7:0/48111587 shutdown_connections 2024-04-25T11:21:52.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.491+0000 7fab75e00640 1 -- 172.21.15.7:0/48111587 wait complete. 2024-04-25T11:21:52.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.493+0000 7fab75e00640 1 Processor -- start 2024-04-25T11:21:52.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.493+0000 7fab75e00640 1 -- start start 2024-04-25T11:21:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.495+0000 7fab75e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7006c570 0x7fab700c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:52.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.495+0000 7fab75e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab700b7440 0x7fab700cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.495+0000 7fab75e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab7015a7a0 0x7fab700cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.495+0000 7fab75e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab700d0870 con 0x7fab7006c570 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.495+0000 7fab75e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fab70160610 con 0x7fab7015a7a0 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.495+0000 7fab75e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fab700d09f0 con 0x7fab700b7440 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.495+0000 7fab74a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7006c570 0x7fab700c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.495+0000 7fab74a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7006c570 0x7fab700c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58644/0 (socket says 172.21.15.7:58644) 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.495+0000 7fab74a00640 1 -- 172.21.15.7:0/3522723910 learned_addr learned my addr 172.21.15.7:0/3522723910 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.496+0000 7fab6fe00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab700b7440 0x7fab700cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.496+0000 7fab75400640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab7015a7a0 0x7fab700cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.496+0000 7fab74a00640 1 -- 172.21.15.7:0/3522723910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab700b7440 msgr2=0x7fab700cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.496+0000 7fab74a00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab700b7440 0x7fab700cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.496+0000 7fab74a00640 1 -- 172.21.15.7:0/3522723910 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab7015a7a0 msgr2=0x7fab700cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.496+0000 7fab74a00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab7015a7a0 0x7fab700cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.496+0000 7fab74a00640 1 -- 172.21.15.7:0/3522723910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab700c8020 con 0x7fab7006c570 2024-04-25T11:21:52.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.496+0000 7fab6fe00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab700b7440 0x7fab700cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:52.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.496+0000 7fab74a00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7006c570 0x7fab700c7ae0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fab58000c00 tx=0x7fab580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:52.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.496+0000 7fab6cc00640 1 -- 172.21.15.7:0/3522723910 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58041020 con 0x7fab7006c570 2024-04-25T11:21:52.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.497+0000 7fab6cc00640 1 -- 172.21.15.7:0/3522723910 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab580306c0 con 0x7fab7006c570 2024-04-25T11:21:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.497+0000 7fab75e00640 1 -- 172.21.15.7:0/3522723910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab700cdf90 con 0x7fab7006c570 2024-04-25T11:21:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.497+0000 7fab6cc00640 1 -- 172.21.15.7:0/3522723910 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58037040 con 0x7fab7006c570 2024-04-25T11:21:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.497+0000 7fab75e00640 1 -- 172.21.15.7:0/3522723910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab700ce260 con 0x7fab7006c570 2024-04-25T11:21:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.498+0000 7fab75e00640 1 -- 172.21.15.7:0/3522723910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab701e9d90 con 0x7fab7006c570 2024-04-25T11:21:52.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.499+0000 7fab6cc00640 1 -- 172.21.15.7:0/3522723910 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fab58044020 con 0x7fab7006c570 2024-04-25T11:21:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.500+0000 7fab6cc00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab4c0744e0 0x7fab4c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.500+0000 7fab6cc00640 1 -- 172.21.15.7:0/3522723910 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7fab580b5c30 con 0x7fab7006c570 2024-04-25T11:21:52.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.500+0000 7fab6fe00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab4c0744e0 0x7fab4c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:52.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.501+0000 7fab6fe00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab4c0744e0 0x7fab4c0769f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fab7015a590 tx=0x7fab6000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:52.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.505+0000 7fab6cc00640 1 -- 172.21.15.7:0/3522723910 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fab701e9d90 con 0x7fab7006c570 2024-04-25T11:21:52.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.712+0000 7fab75e00640 1 -- 172.21.15.7:0/3522723910 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"} v 0) -- 0x7fab700b6190 con 0x7fab7006c570 2024-04-25T11:21:52.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.713+0000 7fab6cc00640 1 -- 172.21.15.7:0/3522723910 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mds", "key": "mds_recall_warning_decay_rate"}]=0 v60) ==== 112+0+10 (secure 0 0 0) 0x7fab5804b030 con 0x7fab7006c570 2024-04-25T11:21:52.708 INFO:teuthology.orchestra.run.smithi007.stdout:60.000000 2024-04-25T11:21:52.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.715+0000 7fab75e00640 1 -- 172.21.15.7:0/3522723910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab4c0744e0 msgr2=0x7fab4c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:52.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.715+0000 7fab75e00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab4c0744e0 0x7fab4c0769f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fab7015a590 tx=0x7fab6000b040 comp rx=0 tx=0).stop 2024-04-25T11:21:52.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.715+0000 7fab75e00640 1 -- 172.21.15.7:0/3522723910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7006c570 msgr2=0x7fab700c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:52.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.715+0000 7fab75e00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7006c570 0x7fab700c7ae0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fab58000c00 tx=0x7fab580046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.716+0000 7fab75e00640 1 -- 172.21.15.7:0/3522723910 shutdown_connections 2024-04-25T11:21:52.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.716+0000 7fab75e00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fab4c0744e0 0x7fab4c0769f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.716+0000 7fab75e00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab7015a7a0 0x7fab700cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.716+0000 7fab75e00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab700b7440 0x7fab700cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.716+0000 7fab75e00640 1 --2- 172.21.15.7:0/3522723910 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7006c570 0x7fab700c7ae0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:52.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.716+0000 7fab75e00640 1 -- 172.21.15.7:0/3522723910 >> 172.21.15.7:0/3522723910 conn(0x7fab700d59c0 msgr2=0x7fab700b5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:52.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.716+0000 7fab75e00640 1 -- 172.21.15.7:0/3522723910 shutdown_connections 2024-04-25T11:21:52.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52.716+0000 7fab75e00640 1 -- 172.21.15.7:0/3522723910 wait complete. 2024-04-25T11:21:52.738 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:21:52.738 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:52.792 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:21:52.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:21:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:21:52.923 INFO:teuthology.orchestra.run.smithi007.stdout:client7381 2024-04-25T11:21:52.924 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:21:52.924 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:21:52.980 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-04-25T11:21:52.980 DEBUG:teuthology.orchestra.run.smithi007:> import fcntl 2024-04-25T11:21:52.980 DEBUG:teuthology.orchestra.run.smithi007:> import sys 2024-04-25T11:21:52.980 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> n = 400 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> abs_path = "/home/ubuntu/cephtest/mnt.0/." 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> if not os.path.exists(abs_path): 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> os.makedirs(abs_path) 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> handles = [] 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> for i in range(0, n): 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> fname = "file_"+str(i) 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> path = os.path.join(abs_path, fname) 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> handles.append(open(path, '"'"'w'"'"')) 2024-04-25T11:21:52.981 DEBUG:teuthology.orchestra.run.smithi007:> 2024-04-25T11:21:52.982 DEBUG:teuthology.orchestra.run.smithi007:> print("waiting with handles open", file=sys.stderr) 2024-04-25T11:21:52.982 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-04-25T11:21:52.982 DEBUG:teuthology.orchestra.run.smithi007:> print("open_n_background: keeping files open", file=sys.stderr) 2024-04-25T11:21:52.982 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-04-25T11:21:52.982 DEBUG:teuthology.orchestra.run.smithi007:> if os.read(0, 4096) == b"": 2024-04-25T11:21:52.982 DEBUG:teuthology.orchestra.run.smithi007:> break 2024-04-25T11:21:52.982 DEBUG:teuthology.orchestra.run.smithi007:> except BlockingIOError: 2024-04-25T11:21:52.982 DEBUG:teuthology.orchestra.run.smithi007:> pass 2024-04-25T11:21:52.982 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(2) 2024-04-25T11:21:52.982 DEBUG:teuthology.orchestra.run.smithi007:> print("stdin closed, goodbye!", file=sys.stderr) 2024-04-25T11:21:52.982 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-04-25T11:21:52.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:21:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.273+0000 7f66e4000640 1 Processor -- start 2024-04-25T11:21:53.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.273+0000 7f66e4000640 1 -- start start 2024-04-25T11:21:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.275+0000 7f66e4000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66dc06dc90 0x7f66dc0afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.276+0000 7f66e4000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 0x7f66dc0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.276+0000 7f66e4000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66dc0b4980 con 0x7f66dc06f300 2024-04-25T11:21:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.276+0000 7f66e4000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f66dc0b4c60 con 0x7f66dc06dc90 2024-04-25T11:21:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.276+0000 7f66e4000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66dc0b4f40 con 0x7f66dc160a30 2024-04-25T11:21:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.276+0000 7f66e2000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 0x7f66dc0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.277+0000 7f66e1600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66dc06dc90 0x7f66dc0afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.277+0000 7f66e0c00640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f66dc06f300 0x7f66dc0ae960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:38024/0 (socket says 172.21.15.7:38024) 2024-04-25T11:21:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.277+0000 7f66e2000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 0x7f66dc0b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:35096/0 (socket says 172.21.15.7:35096) 2024-04-25T11:21:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.277+0000 7f66e0c00640 1 -- 172.21.15.7:0/1124770530 learned_addr learned my addr 172.21.15.7:0/1124770530 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.277+0000 7f66e2000640 1 -- 172.21.15.7:0/1124770530 >> v2:172.21.15.87:3300/0 conn(0x7f66dc06dc90 msgr2=0x7f66dc0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.277+0000 7f66e2000640 1 --2- 172.21.15.7:0/1124770530 >> v2:172.21.15.87:3300/0 conn(0x7f66dc06dc90 0x7f66dc0afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.277+0000 7f66e2000640 1 -- 172.21.15.7:0/1124770530 >> v1:172.21.15.87:6789/0 conn(0x7f66dc06f300 legacy=0x7f66dc0ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.278+0000 7f66e2000640 1 -- 172.21.15.7:0/1124770530 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc0b5220 con 0x7f66dc160a30 2024-04-25T11:21:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.279+0000 7f66e2000640 1 --2- 172.21.15.7:0/1124770530 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 0x7f66dc0b00c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f66d40097a0 tx=0x7f66d402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15ac683f7d65564 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.280+0000 7f66dbe00640 1 -- 172.21.15.7:0/1124770530 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4037070 con 0x7f66dc160a30 2024-04-25T11:21:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.280+0000 7f66dbe00640 1 -- 172.21.15.7:0/1124770530 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d4032440 con 0x7f66dc160a30 2024-04-25T11:21:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.280+0000 7f66dbe00640 1 -- 172.21.15.7:0/1124770530 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4032810 con 0x7f66dc160a30 2024-04-25T11:21:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.281+0000 7f66e4000640 1 -- 172.21.15.7:0/1124770530 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 msgr2=0x7f66dc0b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.281+0000 7f66e4000640 1 --2- 172.21.15.7:0/1124770530 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 0x7f66dc0b00c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f66d40097a0 tx=0x7f66d402a180 comp rx=0 tx=0).stop 2024-04-25T11:21:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.281+0000 7f66e4000640 1 -- 172.21.15.7:0/1124770530 shutdown_connections 2024-04-25T11:21:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.281+0000 7f66e4000640 1 --2- 172.21.15.7:0/1124770530 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 0x7f66dc0b00c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.281+0000 7f66e4000640 1 --2- 172.21.15.7:0/1124770530 >> v2:172.21.15.87:3300/0 conn(0x7f66dc06dc90 0x7f66dc0afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.281+0000 7f66e4000640 1 -- 172.21.15.7:0/1124770530 >> 172.21.15.7:0/1124770530 conn(0x7f66dc074200 msgr2=0x7f66dc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.281+0000 7f66e4000640 1 -- 172.21.15.7:0/1124770530 shutdown_connections 2024-04-25T11:21:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.281+0000 7f66e4000640 1 -- 172.21.15.7:0/1124770530 wait complete. 2024-04-25T11:21:53.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.284+0000 7f66e4000640 1 Processor -- start 2024-04-25T11:21:53.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.284+0000 7f66e4000640 1 -- start start 2024-04-25T11:21:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.285+0000 7f66e4000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66dc06f300 0x7f66dc15c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.285+0000 7f66e4000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 0x7f66dc145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:53.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.285+0000 7f66e4000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66dc1463b0 0x7f66dc1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.285+0000 7f66e4000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc0b6a30 con 0x7f66dc1463b0 2024-04-25T11:21:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.285+0000 7f66e4000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc15d220 con 0x7f66dc06f300 2024-04-25T11:21:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.285+0000 7f66e4000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66dc0ce920 con 0x7f66dc160a30 2024-04-25T11:21:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.286+0000 7f66e2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66dc1463b0 0x7f66dc1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.286+0000 7f66e2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66dc1463b0 0x7f66dc1557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58660/0 (socket says 172.21.15.7:58660) 2024-04-25T11:21:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.286+0000 7f66e2000640 1 -- 172.21.15.7:0/295609242 learned_addr learned my addr 172.21.15.7:0/295609242 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.286+0000 7f66e0c00640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 0x7f66dc145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:53.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.286+0000 7f66e1600640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66dc06f300 0x7f66dc15c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.286+0000 7f66e2000640 1 -- 172.21.15.7:0/295609242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 msgr2=0x7f66dc145d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.286+0000 7f66e2000640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 0x7f66dc145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.286+0000 7f66e2000640 1 -- 172.21.15.7:0/295609242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66dc06f300 msgr2=0x7f66dc15c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.286+0000 7f66e2000640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66dc06f300 0x7f66dc15c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:53.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.286+0000 7f66e2000640 1 -- 172.21.15.7:0/295609242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc15e740 con 0x7f66dc1463b0 2024-04-25T11:21:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.287+0000 7f66e2000640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66dc1463b0 0x7f66dc1557e0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f66d40097a0 tx=0x7f66d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.287+0000 7f66d9600640 1 -- 172.21.15.7:0/295609242 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4037040 con 0x7f66dc1463b0 2024-04-25T11:21:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.287+0000 7f66d9600640 1 -- 172.21.15.7:0/295609242 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d4042070 con 0x7f66dc1463b0 2024-04-25T11:21:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.287+0000 7f66e4000640 1 -- 172.21.15.7:0/295609242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66dc15ea20 con 0x7f66dc1463b0 2024-04-25T11:21:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.287+0000 7f66d9600640 1 -- 172.21.15.7:0/295609242 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4030940 con 0x7f66dc1463b0 2024-04-25T11:21:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.287+0000 7f66e4000640 1 -- 172.21.15.7:0/295609242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66dc15efe0 con 0x7f66dc1463b0 2024-04-25T11:21:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.289+0000 7f66e4000640 1 -- 172.21.15.7:0/295609242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66dc06a100 con 0x7f66dc1463b0 2024-04-25T11:21:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.291+0000 7f66d9600640 1 -- 172.21.15.7:0/295609242 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f66d4030ae0 con 0x7f66dc1463b0 2024-04-25T11:21:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.293+0000 7f66d9600640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66bc0744e0 0x7f66bc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.293+0000 7f66d9600640 1 -- 172.21.15.7:0/295609242 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f66d40b5be0 con 0x7f66dc1463b0 2024-04-25T11:21:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.293+0000 7f66e1600640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66bc0744e0 0x7f66bc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.294+0000 7f66e1600640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66bc0744e0 0x7f66bc0769f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0730d0 tx=0x7f66d000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.299+0000 7f66d9600640 1 -- 172.21.15.7:0/295609242 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66d404c020 con 0x7f66dc1463b0 2024-04-25T11:21:53.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.558+0000 7f66e4000640 1 -- 172.21.15.7:0/295609242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66dc0cc550 con 0x7f66dc1463b0 2024-04-25T11:21:53.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.560+0000 7f66d9600640 1 -- 172.21.15.7:0/295609242 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+5172 (secure 0 0 0) 0x7f66d40801e0 con 0x7f66dc1463b0 2024-04-25T11:21:53.556 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:53.556 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":103,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/3503931395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3503931395},{"type":"v1","addr":"172.21.15.87:6834","nonce":3503931395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/63829697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":63829697},{"type":"v1","addr":"172.21.15.87:6835","nonce":63829697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/415877708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":415877708},{"type":"v1","addr":"172.21.15.7:6835","nonce":415877708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:21:37.365427+0000","modified":"2024-04-25T11:21:40.443636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8967},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8967":{"gid":8967,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/2966579341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2966579341},{"type":"v1","addr":"172.21.15.7:6837","nonce":2966579341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8967,"qdb_cluster":[8967]},"id":10}]} 2024-04-25T11:21:53.556 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 103 2024-04-25T11:21:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.562+0000 7f66e4000640 1 -- 172.21.15.7:0/295609242 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66bc0744e0 msgr2=0x7f66bc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:53.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.562+0000 7f66e4000640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66bc0744e0 0x7f66bc0769f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0730d0 tx=0x7f66d000b040 comp rx=0 tx=0).stop 2024-04-25T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.563+0000 7f66e4000640 1 -- 172.21.15.7:0/295609242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66dc1463b0 msgr2=0x7f66dc1557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.563+0000 7f66e4000640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66dc1463b0 0x7f66dc1557e0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f66d40097a0 tx=0x7f66d40046c0 comp rx=0 tx=0).stop 2024-04-25T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.563+0000 7f66e4000640 1 -- 172.21.15.7:0/295609242 shutdown_connections 2024-04-25T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.563+0000 7f66e4000640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66bc0744e0 0x7f66bc0769f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.563+0000 7f66e4000640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66dc1463b0 0x7f66dc1557e0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.563+0000 7f66e4000640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66dc160a30 0x7f66dc145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.563+0000 7f66e4000640 1 --2- 172.21.15.7:0/295609242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66dc06f300 0x7f66dc15c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.563+0000 7f66e4000640 1 -- 172.21.15.7:0/295609242 >> 172.21.15.7:0/295609242 conn(0x7f66dc074200 msgr2=0x7f66dc06fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.563+0000 7f66e4000640 1 -- 172.21.15.7:0/295609242 shutdown_connections 2024-04-25T11:21:53.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:53.563+0000 7f66e4000640 1 -- 172.21.15.7:0/295609242 wait complete. 2024-04-25T11:21:53.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T11:21:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:waiting with handles open 2024-04-25T11:21:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:21:53.883 INFO:teuthology.orchestra.run.smithi007.stdout:[{"id":9132,"entity":{"name":{"type":"client","num":9132},"addr":{"type":"any","addr":"172.21.15.87:0","nonce":359952538}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":5.2929154369999996,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9843847243399311,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.9132 172.21.15.87:0/359952538","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi087","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":7381,"entity":{"name":{"type":"client","num":7381},"addr":{"type":"any","addr":"172.21.15.7:0","nonce":3319717725}},"state":"open","num_leases":400,"num_caps":401,"request_load_avg":422,"uptime":7.0718870149999997,"requests_in_flight":399,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1200.6758278558762,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":29,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7381 172.21.15.7:0/3319717725","completed_requests":[],"prealloc_inos":[{"start":"0x10000000001","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi007","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T11:21:53.884 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi087", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.87:0", "nonce": 359952538, "type": "any" }, "name": { "num": 9132, "type": "client" } }, "id": 9132, "inst": "client.9132 172.21.15.87:0/359952538", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.984384724339931 }, "state": "open", "uptime": 5.292915437 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi007", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.7:0", "nonce": 3319717725, "type": "any" }, "name": { "num": 7381, "type": "client" } }, "id": 7381, "inst": "client.7381 172.21.15.7:0/3319717725", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 29, "num_caps": 401, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 400, "prealloc_inos": [ { "length": 1000, "start": "0x10000000001" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 422, "requests_in_flight": 399, "session_cache_liveness": { "halflife": 300, "value": 1200.6758278558762 }, "state": "open", "uptime": 7.071887015 } ] 2024-04-25T11:21:53.884 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CLIENT_RECALL 2024-04-25T11:21:53.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:21:54.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.212+0000 7fb59da00640 1 Processor -- start 2024-04-25T11:21:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.212+0000 7fb59da00640 1 -- start start 2024-04-25T11:21:54.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.214+0000 7fb59da00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb598162330 0x7fb59806d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:54.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.215+0000 7fb59da00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb59806ded0 con 0x7fb598070aa0 2024-04-25T11:21:54.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.215+0000 7fb59da00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb598165b20 con 0x7fb59806dac0 2024-04-25T11:21:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.215+0000 7fb59da00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fb598070eb0 con 0x7fb598162330 2024-04-25T11:21:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.215+0000 7fb596a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb598162330 0x7fb59806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.215+0000 7fb597400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb598070aa0 0x7fb5980b5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47602/0 (socket says 172.21.15.7:47602) 2024-04-25T11:21:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.215+0000 7fb597400640 1 -- 172.21.15.7:0/622658047 learned_addr learned my addr 172.21.15.7:0/622658047 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.216+0000 7fb596a00640 1 -- 172.21.15.7:0/622658047 >> v1:172.21.15.87:6789/0 conn(0x7fb59806dac0 legacy=0x7fb598070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.216+0000 7fb596a00640 1 -- 172.21.15.7:0/622658047 >> v1:172.21.15.7:6789/0 conn(0x7fb598070aa0 legacy=0x7fb5980b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.216+0000 7fb596a00640 1 -- 172.21.15.7:0/622658047 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59806f2f0 con 0x7fb598162330 2024-04-25T11:21:54.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.216+0000 7fb595600640 1 -- 172.21.15.7:0/622658047 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3413416097 0 0) 0x7fb590001110 con 0x7fb598070aa0 2024-04-25T11:21:54.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.217+0000 7fb596a00640 1 --2- 172.21.15.7:0/622658047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb598162330 0x7fb59806d580 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fb5800097a0 tx=0x7fb58002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9182c77ffbe8dab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.218+0000 7fb595600640 1 -- 172.21.15.7:0/622658047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580037070 con 0x7fb598162330 2024-04-25T11:21:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.218+0000 7fb595600640 1 -- 172.21.15.7:0/622658047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb580032440 con 0x7fb598162330 2024-04-25T11:21:54.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.218+0000 7fb595600640 1 -- 172.21.15.7:0/622658047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580032810 con 0x7fb598162330 2024-04-25T11:21:54.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.219+0000 7fb59da00640 1 -- 172.21.15.7:0/622658047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb598162330 msgr2=0x7fb59806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.219+0000 7fb59da00640 1 --2- 172.21.15.7:0/622658047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb598162330 0x7fb59806d580 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fb5800097a0 tx=0x7fb58002a230 comp rx=0 tx=0).stop 2024-04-25T11:21:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.219+0000 7fb59da00640 1 -- 172.21.15.7:0/622658047 shutdown_connections 2024-04-25T11:21:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.219+0000 7fb59da00640 1 --2- 172.21.15.7:0/622658047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb598162330 0x7fb59806d580 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.219+0000 7fb59da00640 1 -- 172.21.15.7:0/622658047 >> 172.21.15.7:0/622658047 conn(0x7fb5980732b0 msgr2=0x7fb5980736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.219+0000 7fb59da00640 1 -- 172.21.15.7:0/622658047 shutdown_connections 2024-04-25T11:21:54.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.219+0000 7fb59da00640 1 -- 172.21.15.7:0/622658047 wait complete. 2024-04-25T11:21:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.221+0000 7fb59da00640 1 Processor -- start 2024-04-25T11:21:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.221+0000 7fb59da00640 1 -- start start 2024-04-25T11:21:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.223+0000 7fb59da00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb59806dac0 0x7fb5981fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.223+0000 7fb59da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb598070aa0 0x7fb598144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.223+0000 7fb59da00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb598162330 0x7fb598166790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.223+0000 7fb59da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb59806fb90 con 0x7fb598070aa0 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.223+0000 7fb59da00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb59806f8b0 con 0x7fb598162330 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.223+0000 7fb59da00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb5980b6050 con 0x7fb59806dac0 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.223+0000 7fb596000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb598070aa0 0x7fb598144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.223+0000 7fb596000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb598070aa0 0x7fb598144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58674/0 (socket says 172.21.15.7:58674) 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.223+0000 7fb596000640 1 -- 172.21.15.7:0/1046065383 learned_addr learned my addr 172.21.15.7:0/1046065383 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.224+0000 7fb596a00640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb59806dac0 0x7fb5981fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.224+0000 7fb597400640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb598162330 0x7fb598166790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.224+0000 7fb596000640 1 -- 172.21.15.7:0/1046065383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb59806dac0 msgr2=0x7fb5981fdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.224+0000 7fb596000640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb59806dac0 0x7fb5981fdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.224+0000 7fb596000640 1 -- 172.21.15.7:0/1046065383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb598162330 msgr2=0x7fb598166790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.224+0000 7fb596000640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb598162330 0x7fb598166790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:54.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.224+0000 7fb596000640 1 -- 172.21.15.7:0/1046065383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5981fe3b0 con 0x7fb598070aa0 2024-04-25T11:21:54.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.224+0000 7fb596a00640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb59806dac0 0x7fb5981fdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:54.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.224+0000 7fb597400640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb598162330 0x7fb598166790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:21:54.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.224+0000 7fb596000640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb598070aa0 0x7fb598144860 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fb58800c730 tx=0x7fb58800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.225+0000 7fb58ea00640 1 -- 172.21.15.7:0/1046065383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588003b70 con 0x7fb598070aa0 2024-04-25T11:21:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.225+0000 7fb58ea00640 1 -- 172.21.15.7:0/1046065383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb588003d30 con 0x7fb598070aa0 2024-04-25T11:21:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.225+0000 7fb59da00640 1 -- 172.21.15.7:0/1046065383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5981fe690 con 0x7fb598070aa0 2024-04-25T11:21:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.225+0000 7fb58ea00640 1 -- 172.21.15.7:0/1046065383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58800aa90 con 0x7fb598070aa0 2024-04-25T11:21:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.225+0000 7fb59da00640 1 -- 172.21.15.7:0/1046065383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59814d3b0 con 0x7fb598070aa0 2024-04-25T11:21:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.226+0000 7fb59da00640 1 -- 172.21.15.7:0/1046065383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb598050740 con 0x7fb598070aa0 2024-04-25T11:21:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.227+0000 7fb58ea00640 1 -- 172.21.15.7:0/1046065383 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb58800ac30 con 0x7fb598070aa0 2024-04-25T11:21:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.229+0000 7fb58ea00640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5740744b0 0x7fb5740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.229+0000 7fb58ea00640 1 -- 172.21.15.7:0/1046065383 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7fb5880952f0 con 0x7fb598070aa0 2024-04-25T11:21:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.229+0000 7fb596a00640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5740744b0 0x7fb5740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.230+0000 7fb596a00640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5740744b0 0x7fb5740769c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb580034000 tx=0x7fb58002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.234+0000 7fb58ea00640 1 -- 172.21.15.7:0/1046065383 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb58805f8e0 con 0x7fb598070aa0 2024-04-25T11:21:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.478+0000 7fb59da00640 1 -- 172.21.15.7:0/1046065383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb5981fe140 con 0x7fb598070aa0 2024-04-25T11:21:54.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.479+0000 7fb58ea00640 1 -- 172.21.15.7:0/1046065383 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb58805f1e0 con 0x7fb598070aa0 2024-04-25T11:21:54.475 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:21:54.475 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T11:21:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.482+0000 7fb59da00640 1 -- 172.21.15.7:0/1046065383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5740744b0 msgr2=0x7fb5740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.482+0000 7fb59da00640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5740744b0 0x7fb5740769c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb580034000 tx=0x7fb58002e000 comp rx=0 tx=0).stop 2024-04-25T11:21:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.482+0000 7fb59da00640 1 -- 172.21.15.7:0/1046065383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb598070aa0 msgr2=0x7fb598144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.482+0000 7fb59da00640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb598070aa0 0x7fb598144860 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fb58800c730 tx=0x7fb58800cbf0 comp rx=0 tx=0).stop 2024-04-25T11:21:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.482+0000 7fb59da00640 1 -- 172.21.15.7:0/1046065383 shutdown_connections 2024-04-25T11:21:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.482+0000 7fb59da00640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb5740744b0 0x7fb5740769c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:54.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.482+0000 7fb59da00640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb598162330 0x7fb598166790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.482+0000 7fb59da00640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb598070aa0 0x7fb598144860 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.482+0000 7fb59da00640 1 --2- 172.21.15.7:0/1046065383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb59806dac0 0x7fb5981fdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.482+0000 7fb59da00640 1 -- 172.21.15.7:0/1046065383 >> 172.21.15.7:0/1046065383 conn(0x7fb5980732b0 msgr2=0x7fb5980b64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.483+0000 7fb59da00640 1 -- 172.21.15.7:0/1046065383 shutdown_connections 2024-04-25T11:21:54.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:54.483+0000 7fb59da00640 1 -- 172.21.15.7:0/1046065383 wait complete. 2024-04-25T11:21:54.505 DEBUG:tasks.ceph_test_case:Not expected number of summary strings ([]) 2024-04-25T11:21:54.505 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120.0 retry_count=0)... 2024-04-25T11:21:55.610 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:21:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:21:59.506 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:21:59.614 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:21:59.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.805+0000 7f9ad7e00640 1 Processor -- start 2024-04-25T11:21:59.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.805+0000 7f9ad7e00640 1 -- start start 2024-04-25T11:21:59.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.808+0000 7f9ad7e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9ad006dc90 0x7f9ad0070390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:59.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.808+0000 7f9ad7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ad00708d0 0x7f9ad00b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.808+0000 7f9ad7e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ad0165b20 con 0x7f9ad0162330 2024-04-25T11:21:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.808+0000 7f9ad7e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad006f2f0 con 0x7f9ad006dc90 2024-04-25T11:21:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.808+0000 7f9ad7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad006f5d0 con 0x7f9ad00708d0 2024-04-25T11:21:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.809+0000 7f9ad5e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ad00708d0 0x7f9ad00b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.809+0000 7f9ad5400640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9ad0162330 0x7f9ad006d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:38044/0 (socket says 172.21.15.7:38044) 2024-04-25T11:21:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.809+0000 7f9ad5400640 1 -- 172.21.15.7:0/1494652001 learned_addr learned my addr 172.21.15.7:0/1494652001 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:21:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.809+0000 7f9ad4a00640 1 --2- 172.21.15.7:0/1494652001 >> v2:172.21.15.87:3300/0 conn(0x7f9ad006dc90 0x7f9ad0070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.810+0000 7f9acfe00640 1 -- 172.21.15.7:0/1494652001 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3165599932 0 0) 0x7f9ad0165b20 con 0x7f9ad0162330 2024-04-25T11:21:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.810+0000 7f9acfe00640 1 -- 172.21.15.7:0/1494652001 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ab8003520 con 0x7f9ad0162330 2024-04-25T11:21:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.810+0000 7f9ad5e00640 1 -- 172.21.15.7:0/1494652001 >> v2:172.21.15.87:3300/0 conn(0x7f9ad006dc90 msgr2=0x7f9ad0070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:21:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.810+0000 7f9ad5e00640 1 --2- 172.21.15.7:0/1494652001 >> v2:172.21.15.87:3300/0 conn(0x7f9ad006dc90 0x7f9ad0070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.810+0000 7f9ad5e00640 1 -- 172.21.15.7:0/1494652001 >> v1:172.21.15.87:6789/0 conn(0x7f9ad0162330 legacy=0x7f9ad006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.810+0000 7f9ad5e00640 1 -- 172.21.15.7:0/1494652001 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad006f8b0 con 0x7f9ad00708d0 2024-04-25T11:21:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.811+0000 7f9ad5e00640 1 --2- 172.21.15.7:0/1494652001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ad00708d0 0x7f9ad00b5940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009d90 tx=0x7f9ac802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95471c1e8a28715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.812+0000 7f9acfe00640 1 -- 172.21.15.7:0/1494652001 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8007740 con 0x7f9ad00708d0 2024-04-25T11:21:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.812+0000 7f9acfe00640 1 -- 172.21.15.7:0/1494652001 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac80078e0 con 0x7f9ad00708d0 2024-04-25T11:21:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.812+0000 7f9acfe00640 1 -- 172.21.15.7:0/1494652001 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8007cb0 con 0x7f9ad00708d0 2024-04-25T11:21:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.813+0000 7f9ad7e00640 1 -- 172.21.15.7:0/1494652001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ad00708d0 msgr2=0x7f9ad00b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.813+0000 7f9ad7e00640 1 --2- 172.21.15.7:0/1494652001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ad00708d0 0x7f9ad00b5940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009d90 tx=0x7f9ac802a660 comp rx=0 tx=0).stop 2024-04-25T11:21:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.813+0000 7f9ad7e00640 1 -- 172.21.15.7:0/1494652001 shutdown_connections 2024-04-25T11:21:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.813+0000 7f9ad7e00640 1 --2- 172.21.15.7:0/1494652001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ad00708d0 0x7f9ad00b5940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.813+0000 7f9ad7e00640 1 --2- 172.21.15.7:0/1494652001 >> v2:172.21.15.87:3300/0 conn(0x7f9ad006dc90 0x7f9ad0070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.813+0000 7f9ad7e00640 1 -- 172.21.15.7:0/1494652001 >> 172.21.15.7:0/1494652001 conn(0x7f9ad00732b0 msgr2=0x7f9ad00736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:21:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.813+0000 7f9ad7e00640 1 -- 172.21.15.7:0/1494652001 shutdown_connections 2024-04-25T11:21:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.813+0000 7f9ad7e00640 1 -- 172.21.15.7:0/1494652001 wait complete. 2024-04-25T11:21:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.815+0000 7f9ad7e00640 1 Processor -- start 2024-04-25T11:21:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.815+0000 7f9ad7e00640 1 -- start start 2024-04-25T11:21:59.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.817+0000 7f9ad7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ad00708d0 0x7f9ad0160e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.817+0000 7f9ad7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ad0162330 0x7f9ad009e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.817+0000 7f9ad7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad009e6c0 0x7f9ad0167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.817+0000 7f9ad7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0070e60 con 0x7f9ad009e6c0 2024-04-25T11:21:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.817+0000 7f9ad7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0070ce0 con 0x7f9ad00708d0 2024-04-25T11:21:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.817+0000 7f9ad7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad00b5e80 con 0x7f9ad0162330 2024-04-25T11:21:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.817+0000 7f9ad5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad009e6c0 0x7f9ad0167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.817+0000 7f9ad5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad009e6c0 0x7f9ad0167fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58682/0 (socket says 172.21.15.7:58682) 2024-04-25T11:21:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.817+0000 7f9ad5e00640 1 -- 172.21.15.7:0/3006061180 learned_addr learned my addr 172.21.15.7:0/3006061180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:21:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.817+0000 7f9ad5400640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ad00708d0 0x7f9ad0160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.817+0000 7f9ad4a00640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ad0162330 0x7f9ad009e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:59.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.818+0000 7f9ad5e00640 1 -- 172.21.15.7:0/3006061180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ad0162330 msgr2=0x7f9ad009e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:59.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.818+0000 7f9ad5e00640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ad0162330 0x7f9ad009e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.818+0000 7f9ad5e00640 1 -- 172.21.15.7:0/3006061180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ad00708d0 msgr2=0x7f9ad0160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:21:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.818+0000 7f9ad5e00640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ad00708d0 0x7f9ad0160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:21:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.818+0000 7f9ad5e00640 1 -- 172.21.15.7:0/3006061180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad009f390 con 0x7f9ad009e6c0 2024-04-25T11:21:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.818+0000 7f9ad5e00640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad009e6c0 0x7f9ad0167fb0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9ac802ac50 tx=0x7f9ac8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.818+0000 7f9acd600640 1 -- 172.21.15.7:0/3006061180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac803f070 con 0x7f9ad009e6c0 2024-04-25T11:21:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.819+0000 7f9acd600640 1 -- 172.21.15.7:0/3006061180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac8031bc0 con 0x7f9ad009e6c0 2024-04-25T11:21:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.819+0000 7f9acd600640 1 -- 172.21.15.7:0/3006061180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8009450 con 0x7f9ad009e6c0 2024-04-25T11:21:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.819+0000 7f9ad7e00640 1 -- 172.21.15.7:0/3006061180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad00aec30 con 0x7f9ad009e6c0 2024-04-25T11:21:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.819+0000 7f9ad7e00640 1 -- 172.21.15.7:0/3006061180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad009f0b0 con 0x7f9ad009e6c0 2024-04-25T11:21:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.820+0000 7f9ad7e00640 1 -- 172.21.15.7:0/3006061180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad0050740 con 0x7f9ad009e6c0 2024-04-25T11:21:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.821+0000 7f9acd600640 1 -- 172.21.15.7:0/3006061180 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ac803b410 con 0x7f9ad009e6c0 2024-04-25T11:21:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.822+0000 7f9acd600640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ab00744e0 0x7f9ab00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:21:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.822+0000 7f9acd600640 1 -- 172.21.15.7:0/3006061180 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f9ac80b6b60 con 0x7f9ad009e6c0 2024-04-25T11:21:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.822+0000 7f9ad5400640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ab00744e0 0x7f9ab00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:21:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.823+0000 7f9ad5400640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ab00744e0 0x7f9ab00769f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4005880 tx=0x7f9ac4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:21:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:21:59.827+0000 7f9acd600640 1 -- 172.21.15.7:0/3006061180 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ac8080b00 con 0x7f9ad009e6c0 2024-04-25T11:22:00.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.073+0000 7f9ad7e00640 1 -- 172.21.15.7:0/3006061180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9ad009eb60 con 0x7f9ad009e6c0 2024-04-25T11:22:00.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.074+0000 7f9acd600640 1 -- 172.21.15.7:0/3006061180 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7f9ac8084e00 con 0x7f9ad009e6c0 2024-04-25T11:22:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:00.070 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:22:00.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.077+0000 7f9ad7e00640 1 -- 172.21.15.7:0/3006061180 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ab00744e0 msgr2=0x7f9ab00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:00.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.077+0000 7f9ad7e00640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ab00744e0 0x7f9ab00769f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4005880 tx=0x7f9ac4009040 comp rx=0 tx=0).stop 2024-04-25T11:22:00.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.077+0000 7f9ad7e00640 1 -- 172.21.15.7:0/3006061180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad009e6c0 msgr2=0x7f9ad0167fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:00.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.077+0000 7f9ad7e00640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad009e6c0 0x7f9ad0167fb0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9ac802ac50 tx=0x7f9ac8004a40 comp rx=0 tx=0).stop 2024-04-25T11:22:00.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.078+0000 7f9ad7e00640 1 -- 172.21.15.7:0/3006061180 shutdown_connections 2024-04-25T11:22:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.078+0000 7f9ad7e00640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ab00744e0 0x7f9ab00769f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.078+0000 7f9ad7e00640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ad009e6c0 0x7f9ad0167fb0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.078+0000 7f9ad7e00640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ad0162330 0x7f9ad009e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.078+0000 7f9ad7e00640 1 --2- 172.21.15.7:0/3006061180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ad00708d0 0x7f9ad0160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.078+0000 7f9ad7e00640 1 -- 172.21.15.7:0/3006061180 >> 172.21.15.7:0/3006061180 conn(0x7f9ad00732b0 msgr2=0x7f9ad014c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.078+0000 7f9ad7e00640 1 -- 172.21.15.7:0/3006061180 shutdown_connections 2024-04-25T11:22:00.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.078+0000 7f9ad7e00640 1 -- 172.21.15.7:0/3006061180 wait complete. 2024-04-25T11:22:00.101 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T11:22:00.101 INFO:tasks.ceph_test_case:waiting 120.0s for health warning matching MDS_CACHE_OVERSIZED 2024-04-25T11:22:00.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T11:22:00.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.389+0000 7faf38e00640 1 Processor -- start 2024-04-25T11:22:00.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.389+0000 7faf38e00640 1 -- start start 2024-04-25T11:22:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.392+0000 7faf38e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faf3415baa0 0x7faf3415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.392+0000 7faf38e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf34070640 0x7faf34161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.392+0000 7faf38e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf34162420 0x7faf3406d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.392+0000 7faf38e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7faf34162830 con 0x7faf34070640 2024-04-25T11:22:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.392+0000 7faf38e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7faf3415e980 con 0x7faf3415baa0 2024-04-25T11:22:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.392+0000 7faf38e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faf3415ec60 con 0x7faf34162420 2024-04-25T11:22:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.392+0000 7faf32a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf34070640 0x7faf34161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.392+0000 7faf32a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf34070640 0x7faf34161ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55998/0 (socket says 172.21.15.7:55998) 2024-04-25T11:22:00.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.392+0000 7faf32a00640 1 -- 172.21.15.7:0/1973545564 learned_addr learned my addr 172.21.15.7:0/1973545564 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:00.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.393+0000 7faf33400640 1 --2- 172.21.15.7:0/1973545564 >> v2:172.21.15.87:3300/0 conn(0x7faf3415baa0 0x7faf3415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:00.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.393+0000 7faf33e00640 1 --2- 172.21.15.7:0/1973545564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf34162420 0x7faf3406d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:00.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.393+0000 7faf32a00640 1 -- 172.21.15.7:0/1973545564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf34162420 msgr2=0x7faf3406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:00.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.393+0000 7faf32a00640 1 --2- 172.21.15.7:0/1973545564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf34162420 0x7faf3406d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.393+0000 7faf32a00640 1 -- 172.21.15.7:0/1973545564 >> v2:172.21.15.87:3300/0 conn(0x7faf3415baa0 msgr2=0x7faf3415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:00.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.393+0000 7faf32a00640 1 --2- 172.21.15.7:0/1973545564 >> v2:172.21.15.87:3300/0 conn(0x7faf3415baa0 0x7faf3415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.393+0000 7faf32a00640 1 -- 172.21.15.7:0/1973545564 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3415ef40 con 0x7faf34070640 2024-04-25T11:22:00.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.394+0000 7faf32a00640 1 --2- 172.21.15.7:0/1973545564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf34070640 0x7faf34161ee0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7faf340b6d80 tx=0x7faf2002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a519a6105966e72a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:00.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.394+0000 7faf32000640 1 -- 172.21.15.7:0/1973545564 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20037070 con 0x7faf34070640 2024-04-25T11:22:00.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.394+0000 7faf32000640 1 -- 172.21.15.7:0/1973545564 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf20032440 con 0x7faf34070640 2024-04-25T11:22:00.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.394+0000 7faf32000640 1 -- 172.21.15.7:0/1973545564 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20032810 con 0x7faf34070640 2024-04-25T11:22:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.395+0000 7faf38e00640 1 -- 172.21.15.7:0/1973545564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf34070640 msgr2=0x7faf34161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.395+0000 7faf38e00640 1 --2- 172.21.15.7:0/1973545564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf34070640 0x7faf34161ee0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7faf340b6d80 tx=0x7faf2002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.395+0000 7faf38e00640 1 -- 172.21.15.7:0/1973545564 shutdown_connections 2024-04-25T11:22:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.395+0000 7faf38e00640 1 --2- 172.21.15.7:0/1973545564 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf34162420 0x7faf3406d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.395+0000 7faf38e00640 1 --2- 172.21.15.7:0/1973545564 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf34070640 0x7faf34161ee0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.395+0000 7faf38e00640 1 --2- 172.21.15.7:0/1973545564 >> v2:172.21.15.87:3300/0 conn(0x7faf3415baa0 0x7faf3415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.395+0000 7faf38e00640 1 -- 172.21.15.7:0/1973545564 >> 172.21.15.7:0/1973545564 conn(0x7faf34073090 msgr2=0x7faf340734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.395+0000 7faf38e00640 1 -- 172.21.15.7:0/1973545564 shutdown_connections 2024-04-25T11:22:00.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.396+0000 7faf38e00640 1 -- 172.21.15.7:0/1973545564 wait complete. 2024-04-25T11:22:00.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.398+0000 7faf38e00640 1 Processor -- start 2024-04-25T11:22:00.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.398+0000 7faf38e00640 1 -- start start 2024-04-25T11:22:00.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.400+0000 7faf38e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf34070640 0x7faf3415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:00.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.400+0000 7faf38e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf34162420 0x7faf340aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:00.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.400+0000 7faf38e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf34131b20 0x7faf34131f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:00.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.400+0000 7faf38e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf34070bd0 con 0x7faf34162420 2024-04-25T11:22:00.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.400+0000 7faf38e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faf34070a50 con 0x7faf34131b20 2024-04-25T11:22:00.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.400+0000 7faf38e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faf3406dac0 con 0x7faf34070640 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.400+0000 7faf32a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf34162420 0x7faf340aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.400+0000 7faf32a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf34162420 0x7faf340aa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56002/0 (socket says 172.21.15.7:56002) 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.400+0000 7faf32a00640 1 -- 172.21.15.7:0/2618140428 learned_addr learned my addr 172.21.15.7:0/2618140428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.400+0000 7faf33400640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf34070640 0x7faf3415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.401+0000 7faf33e00640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf34131b20 0x7faf34131f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.401+0000 7faf32a00640 1 -- 172.21.15.7:0/2618140428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf34070640 msgr2=0x7faf3415d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.401+0000 7faf32a00640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf34070640 0x7faf3415d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.401+0000 7faf32a00640 1 -- 172.21.15.7:0/2618140428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf34131b20 msgr2=0x7faf34131f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.401+0000 7faf32a00640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf34131b20 0x7faf34131f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.401+0000 7faf32a00640 1 -- 172.21.15.7:0/2618140428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3415c920 con 0x7faf34162420 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.401+0000 7faf33e00640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf34131b20 0x7faf34131f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.401+0000 7faf32a00640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf34162420 0x7faf340aa9b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7faf2002a220 tx=0x7faf200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.401+0000 7faf2b400640 1 -- 172.21.15.7:0/2618140428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20041020 con 0x7faf34162420 2024-04-25T11:22:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.401+0000 7faf2b400640 1 -- 172.21.15.7:0/2618140428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf200306c0 con 0x7faf34162420 2024-04-25T11:22:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.402+0000 7faf38e00640 1 -- 172.21.15.7:0/2618140428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3415cc00 con 0x7faf34162420 2024-04-25T11:22:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.402+0000 7faf2b400640 1 -- 172.21.15.7:0/2618140428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf20037040 con 0x7faf34162420 2024-04-25T11:22:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.402+0000 7faf38e00640 1 -- 172.21.15.7:0/2618140428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3415d1c0 con 0x7faf34162420 2024-04-25T11:22:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.403+0000 7faf38e00640 1 -- 172.21.15.7:0/2618140428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3415c690 con 0x7faf34162420 2024-04-25T11:22:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.404+0000 7faf2b400640 1 -- 172.21.15.7:0/2618140428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7faf20044020 con 0x7faf34162420 2024-04-25T11:22:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.405+0000 7faf2b400640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf140744e0 0x7faf140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:00.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.405+0000 7faf2b400640 1 -- 172.21.15.7:0/2618140428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7faf200b5c30 con 0x7faf34162420 2024-04-25T11:22:00.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.406+0000 7faf33400640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf140744e0 0x7faf140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:00.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.407+0000 7faf33400640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf140744e0 0x7faf140769f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7faf24006fd0 tx=0x7faf24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.411+0000 7faf2b400640 1 -- 172.21.15.7:0/2618140428 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7faf2004b030 con 0x7faf34162420 2024-04-25T11:22:00.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.655+0000 7faf38e00640 1 -- 172.21.15.7:0/2618140428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7faf34074020 con 0x7faf34162420 2024-04-25T11:22:00.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.656+0000 7faf2b400640 1 -- 172.21.15.7:0/2618140428 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+316 (secure 0 0 0) 0x7faf20080220 con 0x7faf34162420 2024-04-25T11:22:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:00.652 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_WARN","checks":{"MDS_CACHE_OVERSIZED":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs report oversized cache","count":1},"muted":false},"MDS_CLIENT_RECALL":{"severity":"HEALTH_WARN","summary":{"message":"1 clients failing to respond to cache pressure","count":1},"muted":false}},"mutes":[]} 2024-04-25T11:22:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 -- 172.21.15.7:0/2618140428 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf140744e0 msgr2=0x7faf140769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf140744e0 0x7faf140769f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7faf24006fd0 tx=0x7faf24008040 comp rx=0 tx=0).stop 2024-04-25T11:22:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 -- 172.21.15.7:0/2618140428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf34162420 msgr2=0x7faf340aa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf34162420 0x7faf340aa9b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7faf2002a220 tx=0x7faf200046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 -- 172.21.15.7:0/2618140428 shutdown_connections 2024-04-25T11:22:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf140744e0 0x7faf140769f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf34131b20 0x7faf34131f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf34162420 0x7faf340aa9b0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 --2- 172.21.15.7:0/2618140428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faf34070640 0x7faf3415d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 -- 172.21.15.7:0/2618140428 >> 172.21.15.7:0/2618140428 conn(0x7faf34073090 msgr2=0x7faf340684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:00.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 -- 172.21.15.7:0/2618140428 shutdown_connections 2024-04-25T11:22:00.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00.659+0000 7faf38e00640 1 -- 172.21.15.7:0/2618140428 wait complete. 2024-04-25T11:22:00.681 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:22:00.681 INFO:tasks.cephfs.test_client_limits:Terminating process holding files open 2024-04-25T11:22:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:00 stdin-killer INFO: expiration expected; waiting 5 seconds for command to complete 2024-04-25T11:22:01.616 INFO:teuthology.orchestra.run.smithi007.stderr:open_n_background: keeping files open 2024-04-25T11:22:01.616 INFO:teuthology.orchestra.run.smithi007.stderr:stdin closed, goodbye! 2024-04-25T11:22:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:22:01.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.011+0000 7f2c13e00640 1 Processor -- start 2024-04-25T11:22:02.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.011+0000 7f2c13e00640 1 -- start start 2024-04-25T11:22:02.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.013+0000 7f2c13e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2c14070a70 0x7f2c140ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.013+0000 7f2c13e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c1415e190 con 0x7f2c140aedc0 2024-04-25T11:22:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.013+0000 7f2c13e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c1415d610 con 0x7f2c140b0be0 2024-04-25T11:22:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.013+0000 7f2c13e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2c1415d8f0 con 0x7f2c14070a70 2024-04-25T11:22:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.013+0000 7f2c12a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2c14070a70 0x7f2c140ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.013+0000 7f2c0a000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2c140aedc0 0x7f2c140b04d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37490/0 (socket says 172.21.15.7:37490) 2024-04-25T11:22:02.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.013+0000 7f2c0a000640 1 -- 172.21.15.7:0/1174719333 learned_addr learned my addr 172.21.15.7:0/1174719333 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.014+0000 7f2c12a00640 1 -- 172.21.15.7:0/1174719333 >> v1:172.21.15.87:6789/0 conn(0x7f2c140b0be0 legacy=0x7f2c140b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.014+0000 7f2c12a00640 1 -- 172.21.15.7:0/1174719333 >> v1:172.21.15.7:6789/0 conn(0x7f2c140aedc0 legacy=0x7f2c140b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.014+0000 7f2c12000640 1 -- 172.21.15.7:0/1174719333 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3729861310 0 0) 0x7f2c1415e190 con 0x7f2c140aedc0 2024-04-25T11:22:02.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.014+0000 7f2c12a00640 1 -- 172.21.15.7:0/1174719333 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1415dbd0 con 0x7f2c14070a70 2024-04-25T11:22:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.015+0000 7f2c12a00640 1 --2- 172.21.15.7:0/1174719333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c14070a70 0x7f2c140ae880 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2c040097a0 tx=0x7f2c0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9e8bd4db63c1d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.015+0000 7f2c12000640 1 -- 172.21.15.7:0/1174719333 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04037070 con 0x7f2c14070a70 2024-04-25T11:22:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.015+0000 7f2c12000640 1 -- 172.21.15.7:0/1174719333 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c04032440 con 0x7f2c14070a70 2024-04-25T11:22:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.015+0000 7f2c12000640 1 -- 172.21.15.7:0/1174719333 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04032810 con 0x7f2c14070a70 2024-04-25T11:22:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.016+0000 7f2c13e00640 1 -- 172.21.15.7:0/1174719333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c14070a70 msgr2=0x7f2c140ae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.016+0000 7f2c13e00640 1 --2- 172.21.15.7:0/1174719333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c14070a70 0x7f2c140ae880 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2c040097a0 tx=0x7f2c0402a1e0 comp rx=0 tx=0).stop 2024-04-25T11:22:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.016+0000 7f2c13e00640 1 -- 172.21.15.7:0/1174719333 shutdown_connections 2024-04-25T11:22:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.016+0000 7f2c13e00640 1 --2- 172.21.15.7:0/1174719333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c14070a70 0x7f2c140ae880 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.016+0000 7f2c13e00640 1 -- 172.21.15.7:0/1174719333 >> 172.21.15.7:0/1174719333 conn(0x7f2c140b3820 msgr2=0x7f2c140b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.016+0000 7f2c13e00640 1 -- 172.21.15.7:0/1174719333 shutdown_connections 2024-04-25T11:22:02.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.016+0000 7f2c13e00640 1 -- 172.21.15.7:0/1174719333 wait complete. 2024-04-25T11:22:02.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.018+0000 7f2c13e00640 1 Processor -- start 2024-04-25T11:22:02.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.019+0000 7f2c13e00640 1 -- start start 2024-04-25T11:22:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.020+0000 7f2c13e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c14070a70 0x7f2c1415f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.020+0000 7f2c13e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c140aedc0 0x7f2c141547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.020+0000 7f2c13e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2c140b0be0 0x7f2c140b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.020+0000 7f2c13e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1415d2f0 con 0x7f2c14070a70 2024-04-25T11:22:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.020+0000 7f2c13e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1415d170 con 0x7f2c140aedc0 2024-04-25T11:22:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.020+0000 7f2c13e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2c140b5190 con 0x7f2c140b0be0 2024-04-25T11:22:02.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.020+0000 7f2c12a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c14070a70 0x7f2c1415f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.020+0000 7f2c12a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c14070a70 0x7f2c1415f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56022/0 (socket says 172.21.15.7:56022) 2024-04-25T11:22:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.020+0000 7f2c12a00640 1 -- 172.21.15.7:0/3267879603 learned_addr learned my addr 172.21.15.7:0/3267879603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.021+0000 7f2c0a000640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c140aedc0 0x7f2c141547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.021+0000 7f2c12a00640 1 -- 172.21.15.7:0/3267879603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2c140b0be0 msgr2=0x7f2c140b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.021+0000 7f2c12a00640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2c140b0be0 0x7f2c140b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.021+0000 7f2c12a00640 1 -- 172.21.15.7:0/3267879603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c140aedc0 msgr2=0x7f2c141547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.021+0000 7f2c12a00640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c140aedc0 0x7f2c141547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:02.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.021+0000 7f2c12a00640 1 -- 172.21.15.7:0/3267879603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c14160260 con 0x7f2c14070a70 2024-04-25T11:22:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.021+0000 7f2c12a00640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c14070a70 0x7f2c1415f980 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2c0402a7d0 tx=0x7f2c040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.022+0000 7f2c0b400640 1 -- 172.21.15.7:0/3267879603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04040020 con 0x7f2c14070a70 2024-04-25T11:22:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.022+0000 7f2c0b400640 1 -- 172.21.15.7:0/3267879603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c04038040 con 0x7f2c14070a70 2024-04-25T11:22:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.022+0000 7f2c0b400640 1 -- 172.21.15.7:0/3267879603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04037070 con 0x7f2c14070a70 2024-04-25T11:22:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.022+0000 7f2c13e00640 1 -- 172.21.15.7:0/3267879603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c140b15d0 con 0x7f2c14070a70 2024-04-25T11:22:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.022+0000 7f2c13e00640 1 -- 172.21.15.7:0/3267879603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c140b1b90 con 0x7f2c14070a70 2024-04-25T11:22:02.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.023+0000 7f2c13e00640 1 -- 172.21.15.7:0/3267879603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c14050740 con 0x7f2c14070a70 2024-04-25T11:22:02.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.024+0000 7f2c0b400640 1 -- 172.21.15.7:0/3267879603 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2c04043020 con 0x7f2c14070a70 2024-04-25T11:22:02.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.025+0000 7f2c0b400640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bf00744e0 0x7f2bf00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:02.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.025+0000 7f2c0b400640 1 -- 172.21.15.7:0/3267879603 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f2c040b5850 con 0x7f2c14070a70 2024-04-25T11:22:02.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.026+0000 7f2c0a000640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bf00744e0 0x7f2bf00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:02.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.026+0000 7f2c0a000640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bf00744e0 0x7f2bf00769f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2c140b4260 tx=0x7f2bfc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:02.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.030+0000 7f2c0b400640 1 -- 172.21.15.7:0/3267879603 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c0404a070 con 0x7f2c14070a70 2024-04-25T11:22:02.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.267+0000 7f2c13e00640 1 -- 172.21.15.7:0/3267879603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c140b3cd0 con 0x7f2c14070a70 2024-04-25T11:22:02.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.268+0000 7f2c0b400640 1 -- 172.21.15.7:0/3267879603 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5172 (secure 0 0 0) 0x7f2c04032e20 con 0x7f2c14070a70 2024-04-25T11:22:02.264 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:02.264 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":105,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/3503931395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3503931395},{"type":"v1","addr":"172.21.15.87:6834","nonce":3503931395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/63829697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":63829697},{"type":"v1","addr":"172.21.15.87:6835","nonce":63829697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/415877708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":415877708},{"type":"v1","addr":"172.21.15.7:6835","nonce":415877708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:21:37.365427+0000","modified":"2024-04-25T11:21:57.891068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8967},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8967":{"gid":8967,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":8,"addr":"172.21.15.7:6837/2966579341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2966579341},{"type":"v1","addr":"172.21.15.7:6837","nonce":2966579341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8967,"qdb_cluster":[8967]},"id":10}]} 2024-04-25T11:22:02.264 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 105 2024-04-25T11:22:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 -- 172.21.15.7:0/3267879603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bf00744e0 msgr2=0x7f2bf00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bf00744e0 0x7f2bf00769f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2c140b4260 tx=0x7f2bfc009040 comp rx=0 tx=0).stop 2024-04-25T11:22:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 -- 172.21.15.7:0/3267879603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c14070a70 msgr2=0x7f2c1415f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c14070a70 0x7f2c1415f980 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2c0402a7d0 tx=0x7f2c040046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 -- 172.21.15.7:0/3267879603 shutdown_connections 2024-04-25T11:22:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2bf00744e0 0x7f2bf00769f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2c140b0be0 0x7f2c140b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c140aedc0 0x7f2c141547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 --2- 172.21.15.7:0/3267879603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c14070a70 0x7f2c1415f980 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 -- 172.21.15.7:0/3267879603 >> 172.21.15.7:0/3267879603 conn(0x7f2c140b3820 msgr2=0x7f2c140d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 -- 172.21.15.7:0/3267879603 shutdown_connections 2024-04-25T11:22:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:02.271+0000 7f2c13e00640 1 -- 172.21.15.7:0/3267879603 wait complete. 2024-04-25T11:22:02.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-04-25T11:22:02.604 INFO:teuthology.orchestra.run.smithi007.stdout:[{"id":9132,"entity":{"name":{"type":"client","num":9132},"addr":{"type":"any","addr":"172.21.15.87:0","nonce":359952538}},"state":"open","num_leases":0,"num_caps":1,"request_load_avg":3,"uptime":14.01577606,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":2.9248392315727192,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":4,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.9132 172.21.15.87:0/359952538","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"1","hostname":"smithi087","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}},{"id":7381,"entity":{"name":{"type":"client","num":7381},"addr":{"type":"any","addr":"172.21.15.7:0","nonce":3319717725}},"state":"open","num_leases":400,"num_caps":201,"request_load_avg":382,"uptime":15.794747638,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":1507.318645612768,"halflife":60},"release_caps":{"value":198.59638058492826,"halflife":60},"recall_caps_throttle":{"value":388.28684261764022,"halflife":1.5},"recall_caps_throttle2o":{"value":103.94784865337215,"halflife":0.5},"session_cache_liveness":{"value":1176.719521807174,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":0,"last_trim_completed_flushes_tid":0,"delegated_inos":[{"start":"0x10000000002","length":500}],"inst":"client.7381 172.21.15.7:0/3319717725","completed_requests":[],"prealloc_inos":[{"start":"0x10000000002","length":500},{"start":"0x10000000384","length":603}],"client_metadata":{"client_features":{"feature_bits":"0x000000000037fbff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000080ffff"}},"entity_id":"0","hostname":"smithi007","kernel_version":"6.9.0-rc3-g806ec36b2112","root":"/"}}] 2024-04-25T11:22:02.605 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "1", "hostname": "smithi087", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.87:0", "nonce": 359952538, "type": "any" }, "name": { "num": 9132, "type": "client" } }, "id": 9132, "inst": "client.9132 172.21.15.87:0/359952538", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 4, "num_caps": 1, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 3, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 2.924839231572719 }, "state": "open", "uptime": 14.01577606 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "client_features": { "feature_bits": "0x000000000037fbff" }, "entity_id": "0", "hostname": "smithi007", "kernel_version": "6.9.0-rc3-g806ec36b2112", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000080ffff" } }, "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000002" } ], "entity": { "addr": { "addr": "172.21.15.7:0", "nonce": 3319717725, "type": "any" }, "name": { "num": 7381, "type": "client" } }, "id": 7381, "inst": "client.7381 172.21.15.7:0/3319717725", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 0, "num_caps": 201, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 400, "prealloc_inos": [ { "length": 500, "start": "0x10000000002" }, { "length": 603, "start": "0x10000000384" } ], "recall_caps": { "halflife": 60, "value": 1507.318645612768 }, "recall_caps_throttle": { "halflife": 1.5, "value": 388.2868426176402 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 103.94784865337215 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 198.59638058492826 }, "request_load_avg": 382, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1176.719521807174 }, "state": "open", "uptime": 15.794747638 } ] 2024-04-25T11:22:02.605 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T11:22:02.605 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:22:02.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:22:02.712 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:22:02.839 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:22:02.839 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:22:02.840 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport sys\nimport time\nimport os\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nn = 400\nabs_path = "/home/ubuntu/cephtest/mnt.0/."\n\nif not os.path.exists(abs_path):\n os.makedirs(abs_path)\n\nhandles = []\nfor i in range(0, n):\n fname = "file_"+str(i)\n path = os.path.join(abs_path, fname)\n handles.append(open(path, \'w\'))\n\nprint("waiting with handles open", file=sys.stderr)\nwhile True:\n print("open_n_background: keeping files open", file=sys.stderr)\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nprint("stdin closed, goodbye!", file=sys.stderr)\n'], check_status=True, hostname='smithi007') 2024-04-25T11:22:02.840 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:22:02.840 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:02.902 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:22:02.902 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:02.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.240+0000 7f2d23e00640 1 Processor -- start 2024-04-25T11:22:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.240+0000 7f2d23e00640 1 -- start start 2024-04-25T11:22:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.242+0000 7f2d23e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2415de40 0x7f2d24162350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.242+0000 7f2d23e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2d240b7be0 0x7f2d2406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.242+0000 7f2d23e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d240ce850 con 0x7f2d240ce060 2024-04-25T11:22:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.242+0000 7f2d23e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2415e250 con 0x7f2d240b7be0 2024-04-25T11:22:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.242+0000 7f2d23e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2d240cf0f0 con 0x7f2d2415de40 2024-04-25T11:22:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.243+0000 7f2d1be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2d240ce060 0x7f2d240b74d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37498/0 (socket says 172.21.15.7:37498) 2024-04-25T11:22:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.243+0000 7f2d1be00640 1 -- 172.21.15.7:0/780012897 learned_addr learned my addr 172.21.15.7:0/780012897 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.243+0000 7f2d22a00640 1 --2- 172.21.15.7:0/780012897 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2415de40 0x7f2d24162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.243+0000 7f2d23400640 1 --2- 172.21.15.7:0/780012897 >> v2:172.21.15.87:3300/0 conn(0x7f2d240b7be0 0x7f2d2406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.243+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 326599259 0 0) 0x7f2d240ce850 con 0x7f2d240ce060 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.243+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d08003520 con 0x7f2d240ce060 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3062793564 0 0) 0x7f2d08003520 con 0x7f2d240ce060 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2415de40 msgr2=0x7f2d24162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 --2- 172.21.15.7:0/780012897 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2415de40 0x7f2d24162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 >> v2:172.21.15.87:3300/0 conn(0x7f2d240b7be0 msgr2=0x7f2d2406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 --2- 172.21.15.7:0/780012897 >> v2:172.21.15.87:3300/0 conn(0x7f2d240b7be0 0x7f2d2406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d240ceb30 con 0x7f2d240ce060 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2d0c001670 con 0x7f2d240ce060 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 >> v1:172.21.15.7:6789/0 conn(0x7f2d240ce060 legacy=0x7f2d240b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 --2- 172.21.15.7:0/780012897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d080039b0 0x7f2d08003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 --2- 172.21.15.7:0/780012897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d08004800 0x7f2d08004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 --2- 172.21.15.7:0/780012897 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2415de40 0x7f2d08005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2415e250 con 0x7f2d080039b0 2024-04-25T11:22:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2d240cf0f0 con 0x7f2d08004800 2024-04-25T11:22:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0800b9c0 con 0x7f2d2415de40 2024-04-25T11:22:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.244+0000 7f2d22a00640 1 --2- 172.21.15.7:0/780012897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d080039b0 0x7f2d08003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.245+0000 7f2d22a00640 1 -- 172.21.15.7:0/780012897 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2415de40 msgr2=0x7f2d08005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.245+0000 7f2d22a00640 1 --2- 172.21.15.7:0/780012897 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d2415de40 0x7f2d08005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.245+0000 7f2d22a00640 1 -- 172.21.15.7:0/780012897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d08004800 msgr2=0x7f2d08004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.245+0000 7f2d22a00640 1 --2- 172.21.15.7:0/780012897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d08004800 0x7f2d08004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.245+0000 7f2d22a00640 1 -- 172.21.15.7:0/780012897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d08003520 con 0x7f2d080039b0 2024-04-25T11:22:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.246+0000 7f2d22a00640 1 --2- 172.21.15.7:0/780012897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d080039b0 0x7f2d08003e20 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009a90 tx=0x7f2d08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21eb66fd43f2256 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.246+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14036070 con 0x7f2d080039b0 2024-04-25T11:22:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.246+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d14031400 con 0x7f2d080039b0 2024-04-25T11:22:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.246+0000 7f2d22000640 1 -- 172.21.15.7:0/780012897 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14009690 con 0x7f2d080039b0 2024-04-25T11:22:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.247+0000 7f2d23e00640 1 -- 172.21.15.7:0/780012897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d080039b0 msgr2=0x7f2d08003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.247+0000 7f2d23e00640 1 --2- 172.21.15.7:0/780012897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d080039b0 0x7f2d08003e20 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009a90 tx=0x7f2d08005830 comp rx=0 tx=0).stop 2024-04-25T11:22:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.247+0000 7f2d23400640 1 -- 172.21.15.7:0/780012897 reap_dead start 2024-04-25T11:22:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.247+0000 7f2d23e00640 1 -- 172.21.15.7:0/780012897 shutdown_connections 2024-04-25T11:22:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.247+0000 7f2d23e00640 1 -- 172.21.15.7:0/780012897 >> 172.21.15.7:0/780012897 conn(0x7f2d2409b540 msgr2=0x7f2d2409b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.247+0000 7f2d23e00640 1 -- 172.21.15.7:0/780012897 shutdown_connections 2024-04-25T11:22:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.247+0000 7f2d23e00640 1 -- 172.21.15.7:0/780012897 wait complete. 2024-04-25T11:22:03.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.249+0000 7f2d23e00640 1 Processor -- start 2024-04-25T11:22:03.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.250+0000 7f2d23e00640 1 -- start start 2024-04-25T11:22:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.251+0000 7f2d23e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d080039b0 0x7f2d2409ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.251+0000 7f2d23e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d240ca1b0 0x7f2d240ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.251+0000 7f2d23e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2415a200 0x7f2d2415a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.251+0000 7f2d23e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2d240ce5f0 con 0x7f2d240ca1b0 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.251+0000 7f2d23e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2d240ce470 con 0x7f2d2415a200 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.251+0000 7f2d23e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2d24162350 con 0x7f2d080039b0 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.252+0000 7f2d1be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d240ca1b0 0x7f2d240ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.252+0000 7f2d1be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d240ca1b0 0x7f2d240ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56030/0 (socket says 172.21.15.7:56030) 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.252+0000 7f2d1be00640 1 -- 172.21.15.7:0/1284970354 learned_addr learned my addr 172.21.15.7:0/1284970354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.252+0000 7f2d22a00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d080039b0 0x7f2d2409ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.252+0000 7f2d1be00640 1 -- 172.21.15.7:0/1284970354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d080039b0 msgr2=0x7f2d2409ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.252+0000 7f2d1be00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d080039b0 0x7f2d2409ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.252+0000 7f2d1be00640 1 -- 172.21.15.7:0/1284970354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2415a200 msgr2=0x7f2d2415a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.252+0000 7f2d1be00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2415a200 0x7f2d2415a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.252+0000 7f2d1be00640 1 -- 172.21.15.7:0/1284970354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2415ab50 con 0x7f2d240ca1b0 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.252+0000 7f2d22a00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d080039b0 0x7f2d2409ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:03.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.252+0000 7f2d1be00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d240ca1b0 0x7f2d240ca5c0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00c7a0 tx=0x7f2d0c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.253+0000 7f2d1aa00640 1 -- 172.21.15.7:0/1284970354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c018070 con 0x7f2d240ca1b0 2024-04-25T11:22:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.253+0000 7f2d1aa00640 1 -- 172.21.15.7:0/1284970354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c0074e0 con 0x7f2d240ca1b0 2024-04-25T11:22:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.253+0000 7f2d23e00640 1 -- 172.21.15.7:0/1284970354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d240cae50 con 0x7f2d240ca1b0 2024-04-25T11:22:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.253+0000 7f2d1aa00640 1 -- 172.21.15.7:0/1284970354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c004a20 con 0x7f2d240ca1b0 2024-04-25T11:22:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.253+0000 7f2d23e00640 1 -- 172.21.15.7:0/1284970354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d240cb410 con 0x7f2d240ca1b0 2024-04-25T11:22:03.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.254+0000 7f2d23e00640 1 -- 172.21.15.7:0/1284970354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d24162760 con 0x7f2d240ca1b0 2024-04-25T11:22:03.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.256+0000 7f2d1aa00640 1 -- 172.21.15.7:0/1284970354 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2d0c01b070 con 0x7f2d240ca1b0 2024-04-25T11:22:03.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.257+0000 7f2d1aa00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d000744b0 0x7f2d000769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.257+0000 7f2d1aa00640 1 -- 172.21.15.7:0/1284970354 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 6101+0+0 (secure 0 0 0) 0x7f2d0c094960 con 0x7f2d240ca1b0 2024-04-25T11:22:03.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.258+0000 7f2d22a00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d000744b0 0x7f2d000769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.258+0000 7f2d22a00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d000744b0 0x7f2d000769c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2d14033000 tx=0x7f2d14008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:03.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.262+0000 7f2d1aa00640 1 -- 172.21.15.7:0/1284970354 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2d24162760 con 0x7f2d240ca1b0 2024-04-25T11:22:03.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.466+0000 7f2d23e00640 1 -- 172.21.15.7:0/1284970354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d24050740 con 0x7f2d240ca1b0 2024-04-25T11:22:03.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.469+0000 7f2d1aa00640 1 -- 172.21.15.7:0/1284970354 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16796 (secure 0 0 0) 0x7f2d2415ab50 con 0x7f2d240ca1b0 2024-04-25T11:22:03.466 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:03.466 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":142,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:21:38.842722+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:21:35.562066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T11:21:36.342958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:03.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 -- 172.21.15.7:0/1284970354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d000744b0 msgr2=0x7f2d000769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:03.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d000744b0 0x7f2d000769c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2d14033000 tx=0x7f2d14008030 comp rx=0 tx=0).stop 2024-04-25T11:22:03.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 -- 172.21.15.7:0/1284970354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d240ca1b0 msgr2=0x7f2d240ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:03.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d240ca1b0 0x7f2d240ca5c0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00c7a0 tx=0x7f2d0c00cc60 comp rx=0 tx=0).stop 2024-04-25T11:22:03.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 -- 172.21.15.7:0/1284970354 shutdown_connections 2024-04-25T11:22:03.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2d000744b0 0x7f2d000769c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d2415a200 0x7f2d2415a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d240ca1b0 0x7f2d240ca5c0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 --2- 172.21.15.7:0/1284970354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d080039b0 0x7f2d2409ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 -- 172.21.15.7:0/1284970354 >> 172.21.15.7:0/1284970354 conn(0x7f2d2409b540 msgr2=0x7f2d2415d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:03.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 -- 172.21.15.7:0/1284970354 shutdown_connections 2024-04-25T11:22:03.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.472+0000 7f2d23e00640 1 -- 172.21.15.7:0/1284970354 wait complete. 2024-04-25T11:22:03.498 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:21:35.562066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:21:36.342958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:22:03.498 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:22:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.778+0000 7fd4f0a00640 1 Processor -- start 2024-04-25T11:22:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.778+0000 7fd4f0a00640 1 -- start start 2024-04-25T11:22:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.780+0000 7fd4f0a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4ec0d0480 0x7fd4ec0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.780+0000 7fd4f0a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4ec0ce570 con 0x7fd4ec1583c0 2024-04-25T11:22:03.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.780+0000 7fd4f0a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4ec0ce850 con 0x7fd4ec06eea0 2024-04-25T11:22:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.780+0000 7fd4f0a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd4ec15f190 con 0x7fd4ec0d0480 2024-04-25T11:22:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.780+0000 7fd4eb400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd4ec1583c0 0x7fd4ec0cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37508/0 (socket says 172.21.15.7:37508) 2024-04-25T11:22:03.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.780+0000 7fd4eb400640 1 -- 172.21.15.7:0/4294953758 learned_addr learned my addr 172.21.15.7:0/4294953758 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.781+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 678896017 0 0) 0x7fd4ec0ce570 con 0x7fd4ec1583c0 2024-04-25T11:22:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.781+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4d0003520 con 0x7fd4ec1583c0 2024-04-25T11:22:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.781+0000 7fd4eaa00640 1 --2- 172.21.15.7:0/4294953758 >> v2:172.21.15.87:3300/0 conn(0x7fd4ec0d0480 0x7fd4ec0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.781+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2911441184 0 0) 0x7fd4d0003520 con 0x7fd4ec1583c0 2024-04-25T11:22:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.781+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 >> v2:172.21.15.87:3300/0 conn(0x7fd4ec0d0480 msgr2=0x7fd4ec0d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.781+0000 7fd4ea000640 1 --2- 172.21.15.7:0/4294953758 >> v2:172.21.15.87:3300/0 conn(0x7fd4ec0d0480 0x7fd4ec0d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.781+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 >> v1:172.21.15.87:6789/0 conn(0x7fd4ec06eea0 legacy=0x7fd4ec0b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.781+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec0ceb30 con 0x7fd4ec1583c0 2024-04-25T11:22:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.781+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fd4dc001670 con 0x7fd4ec1583c0 2024-04-25T11:22:03.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.781+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 >> v1:172.21.15.7:6789/0 conn(0x7fd4ec1583c0 legacy=0x7fd4ec0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.781+0000 7fd4ea000640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0003a00 0x7fd4d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4ea000640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d0004800 0x7fd4d0004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4ea000640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4d0005350 0x7fd4d000ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec15f190 con 0x7fd4d0003a00 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d000c2e0 con 0x7fd4d0004800 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d000c4e0 con 0x7fd4d0005350 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4eb400640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0003a00 0x7fd4d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4ebe00640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4d0005350 0x7fd4d000ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4eaa00640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d0004800 0x7fd4d0004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4eb400640 1 -- 172.21.15.7:0/4294953758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4d0005350 msgr2=0x7fd4d000ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4eb400640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4d0005350 0x7fd4d000ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4eb400640 1 -- 172.21.15.7:0/4294953758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d0004800 msgr2=0x7fd4d0004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4eb400640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d0004800 0x7fd4d0004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4eb400640 1 -- 172.21.15.7:0/4294953758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d0003520 con 0x7fd4d0003a00 2024-04-25T11:22:03.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4ebe00640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4d0005350 0x7fd4d000ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:03.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.782+0000 7fd4ebe00640 1 -- 172.21.15.7:0/4294953758 reap_dead start 2024-04-25T11:22:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.783+0000 7fd4eb400640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0003a00 0x7fd4d0003e70 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0b43c0 tx=0x7fd4d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc323031952f5948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.784+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc007c40 con 0x7fd4d0003a00 2024-04-25T11:22:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.784+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4dc005af0 con 0x7fd4d0003a00 2024-04-25T11:22:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.784+0000 7fd4ea000640 1 -- 172.21.15.7:0/4294953758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc007070 con 0x7fd4d0003a00 2024-04-25T11:22:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.785+0000 7fd4f0a00640 1 -- 172.21.15.7:0/4294953758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0003a00 msgr2=0x7fd4d0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.785+0000 7fd4f0a00640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0003a00 0x7fd4d0003e70 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0b43c0 tx=0x7fd4d0005830 comp rx=0 tx=0).stop 2024-04-25T11:22:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.785+0000 7fd4f0a00640 1 -- 172.21.15.7:0/4294953758 shutdown_connections 2024-04-25T11:22:03.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.785+0000 7fd4f0a00640 1 --2- 172.21.15.7:0/4294953758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d0003a00 0x7fd4d0003e70 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.785+0000 7fd4f0a00640 1 -- 172.21.15.7:0/4294953758 >> 172.21.15.7:0/4294953758 conn(0x7fd4ec0740f0 msgr2=0x7fd4ec06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.785+0000 7fd4f0a00640 1 -- 172.21.15.7:0/4294953758 shutdown_connections 2024-04-25T11:22:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.785+0000 7fd4f0a00640 1 -- 172.21.15.7:0/4294953758 wait complete. 2024-04-25T11:22:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.787+0000 7fd4f0a00640 1 Processor -- start 2024-04-25T11:22:03.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.787+0000 7fd4f0a00640 1 -- start start 2024-04-25T11:22:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.789+0000 7fd4f0a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d0003a00 0x7fd4ec144060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.789+0000 7fd4f0a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4ec159630 0x7fd4ec14b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.789+0000 7fd4f0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ec06f2b0 0x7fd4ec06f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.789+0000 7fd4f0a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec0d0890 con 0x7fd4ec06f2b0 2024-04-25T11:22:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.789+0000 7fd4f0a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec0cf0f0 con 0x7fd4d0003a00 2024-04-25T11:22:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.789+0000 7fd4f0a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ec0d0a10 con 0x7fd4ec159630 2024-04-25T11:22:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.789+0000 7fd4ebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ec06f2b0 0x7fd4ec06f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.789+0000 7fd4ebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ec06f2b0 0x7fd4ec06f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56052/0 (socket says 172.21.15.7:56052) 2024-04-25T11:22:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.789+0000 7fd4ebe00640 1 -- 172.21.15.7:0/223204038 learned_addr learned my addr 172.21.15.7:0/223204038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4eb400640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d0003a00 0x7fd4ec144060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4eaa00640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4ec159630 0x7fd4ec14b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4ebe00640 1 -- 172.21.15.7:0/223204038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4ec159630 msgr2=0x7fd4ec14b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:03.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4ebe00640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4ec159630 0x7fd4ec14b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4ebe00640 1 -- 172.21.15.7:0/223204038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d0003a00 msgr2=0x7fd4ec144060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4ebe00640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d0003a00 0x7fd4ec144060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4ebe00640 1 -- 172.21.15.7:0/223204038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec06fc00 con 0x7fd4ec06f2b0 2024-04-25T11:22:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4eaa00640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4ec159630 0x7fd4ec14b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4ebe00640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ec06f2b0 0x7fd4ec06f6c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0d0dd0 tx=0x7fd4e400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4e3400640 1 -- 172.21.15.7:0/223204038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4018070 con 0x7fd4ec06f2b0 2024-04-25T11:22:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4e3400640 1 -- 172.21.15.7:0/223204038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e4003b70 con 0x7fd4ec06f2b0 2024-04-25T11:22:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.790+0000 7fd4e3400640 1 -- 172.21.15.7:0/223204038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4005070 con 0x7fd4ec06f2b0 2024-04-25T11:22:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.791+0000 7fd4f0a00640 1 -- 172.21.15.7:0/223204038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec14be40 con 0x7fd4ec06f2b0 2024-04-25T11:22:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.791+0000 7fd4f0a00640 1 -- 172.21.15.7:0/223204038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec14c400 con 0x7fd4ec06f2b0 2024-04-25T11:22:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.792+0000 7fd4f0a00640 1 -- 172.21.15.7:0/223204038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ec14bbb0 con 0x7fd4ec06f2b0 2024-04-25T11:22:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.793+0000 7fd4e3400640 1 -- 172.21.15.7:0/223204038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd4e401b070 con 0x7fd4ec06f2b0 2024-04-25T11:22:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.795+0000 7fd4e3400640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4cc0744e0 0x7fd4cc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.795+0000 7fd4e3400640 1 -- 172.21.15.7:0/223204038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6159+0+0 (secure 0 0 0) 0x7fd4e4094ab0 con 0x7fd4ec06f2b0 2024-04-25T11:22:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.796+0000 7fd4eb400640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4cc0744e0 0x7fd4cc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.796+0000 7fd4eb400640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4cc0744e0 0x7fd4cc0769f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc032000 tx=0x7fd4dc02d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:03.800+0000 7fd4e3400640 1 -- 172.21.15.7:0/223204038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd4ec06fc00 con 0x7fd4ec06f2b0 2024-04-25T11:22:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.005+0000 7fd4f0a00640 1 -- 172.21.15.7:0/223204038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd4ec050740 con 0x7fd4ec06f2b0 2024-04-25T11:22:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.006+0000 7fd4e3400640 1 -- 172.21.15.7:0/223204038 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) ==== 93+0+10 (secure 0 0 0) 0x7fd4e405f100 con 0x7fd4ec06f2b0 2024-04-25T11:22:04.003 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:22:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 -- 172.21.15.7:0/223204038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4cc0744e0 msgr2=0x7fd4cc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4cc0744e0 0x7fd4cc0769f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc032000 tx=0x7fd4dc02d000 comp rx=0 tx=0).stop 2024-04-25T11:22:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 -- 172.21.15.7:0/223204038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ec06f2b0 msgr2=0x7fd4ec06f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ec06f2b0 0x7fd4ec06f6c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0d0dd0 tx=0x7fd4e400dc00 comp rx=0 tx=0).stop 2024-04-25T11:22:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 -- 172.21.15.7:0/223204038 shutdown_connections 2024-04-25T11:22:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4cc0744e0 0x7fd4cc0769f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4ec06f2b0 0x7fd4ec06f6c0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4ec159630 0x7fd4ec14b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 --2- 172.21.15.7:0/223204038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d0003a00 0x7fd4ec144060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 -- 172.21.15.7:0/223204038 >> 172.21.15.7:0/223204038 conn(0x7fd4ec0740f0 msgr2=0x7fd4ec0b4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 -- 172.21.15.7:0/223204038 shutdown_connections 2024-04-25T11:22:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.009+0000 7fd4f0a00640 1 -- 172.21.15.7:0/223204038 wait complete. 2024-04-25T11:22:04.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:22:04.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.331+0000 7fedb8600640 1 Processor -- start 2024-04-25T11:22:04.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.331+0000 7fedb8600640 1 -- start start 2024-04-25T11:22:04.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.333+0000 7fedb8600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fedb015dfb0 0x7fedb00b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.333+0000 7fedb8600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedb006fab0 0x7fedb00aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.333+0000 7fedb8600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedb015e3c0 con 0x7fedb00d0b40 2024-04-25T11:22:04.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.333+0000 7fedb8600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fedb00b09b0 con 0x7fedb015dfb0 2024-04-25T11:22:04.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.333+0000 7fedb8600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fedb00cf800 con 0x7fedb006fab0 2024-04-25T11:22:04.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.334+0000 7fedb6600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedb006fab0 0x7fedb00aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.334+0000 7fedb6600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedb006fab0 0x7fedb00aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:41324/0 (socket says 172.21.15.7:41324) 2024-04-25T11:22:04.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.334+0000 7fedb6600640 1 -- 172.21.15.7:0/1454060648 learned_addr learned my addr 172.21.15.7:0/1454060648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:04.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.334+0000 7fedb5c00640 1 --2- 172.21.15.7:0/1454060648 >> v2:172.21.15.87:3300/0 conn(0x7fedb015dfb0 0x7fedb00b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.335+0000 7fedb5c00640 1 -- 172.21.15.7:0/1454060648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedb006fab0 msgr2=0x7fedb00aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:04.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.335+0000 7fedb5c00640 1 --2- 172.21.15.7:0/1454060648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedb006fab0 0x7fedb00aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.335+0000 7fedb5c00640 1 -- 172.21.15.7:0/1454060648 >> v1:172.21.15.87:6789/0 conn(0x7fedb00d0b40 legacy=0x7fedb006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:04.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.335+0000 7fedb5c00640 1 -- 172.21.15.7:0/1454060648 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb00cfaa0 con 0x7fedb015dfb0 2024-04-25T11:22:04.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.336+0000 7fedb5c00640 1 --2- 172.21.15.7:0/1454060648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb015dfb0 0x7fedb00b0470 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7feda40097a0 tx=0x7feda402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9db44f66daede96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:04.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.336+0000 7fedafe00640 1 -- 172.21.15.7:0/1454060648 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4037070 con 0x7fedb015dfb0 2024-04-25T11:22:04.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.336+0000 7fedafe00640 1 -- 172.21.15.7:0/1454060648 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda4032440 con 0x7fedb015dfb0 2024-04-25T11:22:04.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.336+0000 7fedafe00640 1 -- 172.21.15.7:0/1454060648 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4032810 con 0x7fedb015dfb0 2024-04-25T11:22:04.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.337+0000 7fedb8600640 1 -- 172.21.15.7:0/1454060648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb015dfb0 msgr2=0x7fedb00b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:04.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.337+0000 7fedb8600640 1 --2- 172.21.15.7:0/1454060648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb015dfb0 0x7fedb00b0470 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7feda40097a0 tx=0x7feda402a180 comp rx=0 tx=0).stop 2024-04-25T11:22:04.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.338+0000 7fedb8600640 1 -- 172.21.15.7:0/1454060648 shutdown_connections 2024-04-25T11:22:04.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.338+0000 7fedb8600640 1 --2- 172.21.15.7:0/1454060648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedb006fab0 0x7fedb00aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.338+0000 7fedb8600640 1 --2- 172.21.15.7:0/1454060648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb015dfb0 0x7fedb00b0470 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.338+0000 7fedb8600640 1 -- 172.21.15.7:0/1454060648 >> 172.21.15.7:0/1454060648 conn(0x7fedb00742c0 msgr2=0x7fedb006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:04.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.338+0000 7fedb8600640 1 -- 172.21.15.7:0/1454060648 shutdown_connections 2024-04-25T11:22:04.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.338+0000 7fedb8600640 1 -- 172.21.15.7:0/1454060648 wait complete. 2024-04-25T11:22:04.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.341+0000 7fedb8600640 1 Processor -- start 2024-04-25T11:22:04.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.341+0000 7fedb8600640 1 -- start start 2024-04-25T11:22:04.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.344+0000 7fedb8600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb006fab0 0x7fedb015e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.344+0000 7fedb8600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb00d0b40 0x7fedb0142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.344+0000 7fedb8600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedb015dfb0 0x7fedb0155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.344+0000 7fedb8600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fedb00d0340 con 0x7fedb00d0b40 2024-04-25T11:22:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.344+0000 7fedb8600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fedb00d0060 con 0x7fedb006fab0 2024-04-25T11:22:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.344+0000 7fedb8600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fedb00d0620 con 0x7fedb015dfb0 2024-04-25T11:22:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.344+0000 7fedb5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb00d0b40 0x7fedb0142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.344+0000 7fedb5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb00d0b40 0x7fedb0142120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56066/0 (socket says 172.21.15.7:56066) 2024-04-25T11:22:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.344+0000 7fedb5200640 1 -- 172.21.15.7:0/1958393751 learned_addr learned my addr 172.21.15.7:0/1958393751 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.344+0000 7fedb6600640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedb015dfb0 0x7fedb0155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.344+0000 7fedb5c00640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb006fab0 0x7fedb015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.345+0000 7fedb5200640 1 -- 172.21.15.7:0/1958393751 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedb015dfb0 msgr2=0x7fedb0155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.345+0000 7fedb5200640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedb015dfb0 0x7fedb0155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.345+0000 7fedb5200640 1 -- 172.21.15.7:0/1958393751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb006fab0 msgr2=0x7fedb015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.345+0000 7fedb5200640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb006fab0 0x7fedb015e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.345+0000 7fedb5200640 1 -- 172.21.15.7:0/1958393751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb0142660 con 0x7fedb00d0b40 2024-04-25T11:22:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.345+0000 7fedb5200640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb00d0b40 0x7fedb0142120 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00c3c0 tx=0x7fed9c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.346+0000 7fedad600640 1 -- 172.21.15.7:0/1958393751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0052d0 con 0x7fedb00d0b40 2024-04-25T11:22:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.346+0000 7fedad600640 1 -- 172.21.15.7:0/1958393751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9c005490 con 0x7fedb00d0b40 2024-04-25T11:22:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.346+0000 7fedad600640 1 -- 172.21.15.7:0/1958393751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c007260 con 0x7fedb00d0b40 2024-04-25T11:22:04.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.346+0000 7fedb8600640 1 -- 172.21.15.7:0/1958393751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb01542b0 con 0x7fedb00d0b40 2024-04-25T11:22:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.346+0000 7fedb8600640 1 -- 172.21.15.7:0/1958393751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb0154870 con 0x7fedb00d0b40 2024-04-25T11:22:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.347+0000 7fedb8600640 1 -- 172.21.15.7:0/1958393751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb0050740 con 0x7fedb00d0b40 2024-04-25T11:22:04.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.348+0000 7fedad600640 1 -- 172.21.15.7:0/1958393751 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fed9c007400 con 0x7fedb00d0b40 2024-04-25T11:22:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.349+0000 7fedad600640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed900744e0 0x7fed900769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.349+0000 7fedad600640 1 -- 172.21.15.7:0/1958393751 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 6159+0+0 (secure 0 0 0) 0x7fed9c0946e0 con 0x7fedb00d0b40 2024-04-25T11:22:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.350+0000 7fedb5c00640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed900744e0 0x7fed900769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.351+0000 7fedb5c00640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed900744e0 0x7fed900769f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7feda4034000 tx=0x7feda402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.355+0000 7fedad600640 1 -- 172.21.15.7:0/1958393751 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fed9c05e650 con 0x7fedb00d0b40 2024-04-25T11:22:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.575+0000 7fedb8600640 1 -- 172.21.15.7:0/1958393751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fedb0070fb0 con 0x7fedb00d0b40 2024-04-25T11:22:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.577+0000 7fedad600640 1 -- 172.21.15.7:0/1958393751 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) ==== 104+0+11 (secure 0 0 0) 0x7fed9c062950 con 0x7fedb00d0b40 2024-04-25T11:22:04.573 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:22:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.579+0000 7fedb8600640 1 -- 172.21.15.7:0/1958393751 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed900744e0 msgr2=0x7fed900769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.579+0000 7fedb8600640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed900744e0 0x7fed900769f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7feda4034000 tx=0x7feda402e000 comp rx=0 tx=0).stop 2024-04-25T11:22:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.579+0000 7fedb8600640 1 -- 172.21.15.7:0/1958393751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb00d0b40 msgr2=0x7fedb0142120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.579+0000 7fedb8600640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb00d0b40 0x7fedb0142120 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00c3c0 tx=0x7fed9c00c7c0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.579+0000 7fedb8600640 1 -- 172.21.15.7:0/1958393751 shutdown_connections 2024-04-25T11:22:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.579+0000 7fedb8600640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fed900744e0 0x7fed900769f0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.579+0000 7fedb8600640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fedb015dfb0 0x7fedb0155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.580+0000 7fedb8600640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedb00d0b40 0x7fedb0142120 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.580+0000 7fedb8600640 1 --2- 172.21.15.7:0/1958393751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedb006fab0 0x7fedb015e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.580+0000 7fedb8600640 1 -- 172.21.15.7:0/1958393751 >> 172.21.15.7:0/1958393751 conn(0x7fedb00742c0 msgr2=0x7fedb006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.580+0000 7fedb8600640 1 -- 172.21.15.7:0/1958393751 shutdown_connections 2024-04-25T11:22:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.580+0000 7fedb8600640 1 -- 172.21.15.7:0/1958393751 wait complete. 2024-04-25T11:22:04.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:22:04.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.885+0000 7effe8e00640 1 Processor -- start 2024-04-25T11:22:04.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.885+0000 7effe8e00640 1 -- start start 2024-04-25T11:22:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.887+0000 7effe8e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7effe4160180 0x7effe4163c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.887+0000 7effe8e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effe40d06c0 0x7effe40b9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.887+0000 7effe8e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe40d2550 0x7effe40d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.887+0000 7effe8e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7effe40d4510 con 0x7effe40d06c0 2024-04-25T11:22:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.887+0000 7effe8e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7effe4164bb0 con 0x7effe4160180 2024-04-25T11:22:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.887+0000 7effe8e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7effe40d2ea0 con 0x7effe40d2550 2024-04-25T11:22:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.888+0000 7effe1400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effe40d06c0 0x7effe40b9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.888+0000 7effe1400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effe40d06c0 0x7effe40b9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56080/0 (socket says 172.21.15.7:56080) 2024-04-25T11:22:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.888+0000 7effe1400640 1 -- 172.21.15.7:0/921640920 learned_addr learned my addr 172.21.15.7:0/921640920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.888+0000 7effe1e00640 1 --2- 172.21.15.7:0/921640920 >> v2:172.21.15.87:3300/0 conn(0x7effe4160180 0x7effe4163c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.888+0000 7effe2800640 1 --2- 172.21.15.7:0/921640920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe40d2550 0x7effe40d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.888+0000 7effe1400640 1 -- 172.21.15.7:0/921640920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe40d2550 msgr2=0x7effe40d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.888+0000 7effe1400640 1 --2- 172.21.15.7:0/921640920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe40d2550 0x7effe40d2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.888+0000 7effe1400640 1 -- 172.21.15.7:0/921640920 >> v2:172.21.15.87:3300/0 conn(0x7effe4160180 msgr2=0x7effe4163c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.888+0000 7effe1400640 1 --2- 172.21.15.7:0/921640920 >> v2:172.21.15.87:3300/0 conn(0x7effe4160180 0x7effe4163c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.888+0000 7effe1400640 1 -- 172.21.15.7:0/921640920 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe4164310 con 0x7effe40d06c0 2024-04-25T11:22:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.889+0000 7effe1400640 1 --2- 172.21.15.7:0/921640920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe40d06c0 0x7effe40b9090 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7effe408ee50 tx=0x7effd002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0cf0c790f45fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.890+0000 7effe0a00640 1 -- 172.21.15.7:0/921640920 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0037070 con 0x7effe40d06c0 2024-04-25T11:22:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.890+0000 7effe0a00640 1 -- 172.21.15.7:0/921640920 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd0032440 con 0x7effe40d06c0 2024-04-25T11:22:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.890+0000 7effe0a00640 1 -- 172.21.15.7:0/921640920 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0032810 con 0x7effe40d06c0 2024-04-25T11:22:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.890+0000 7effe8e00640 1 -- 172.21.15.7:0/921640920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe40d06c0 msgr2=0x7effe40b9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.890+0000 7effe8e00640 1 --2- 172.21.15.7:0/921640920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe40d06c0 0x7effe40b9090 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7effe408ee50 tx=0x7effd002a1e0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.891+0000 7effe8e00640 1 -- 172.21.15.7:0/921640920 shutdown_connections 2024-04-25T11:22:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.891+0000 7effe8e00640 1 --2- 172.21.15.7:0/921640920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe40d2550 0x7effe40d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.891+0000 7effe8e00640 1 --2- 172.21.15.7:0/921640920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe40d06c0 0x7effe40b9090 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.891+0000 7effe8e00640 1 --2- 172.21.15.7:0/921640920 >> v2:172.21.15.87:3300/0 conn(0x7effe4160180 0x7effe4163c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.891+0000 7effe8e00640 1 -- 172.21.15.7:0/921640920 >> 172.21.15.7:0/921640920 conn(0x7effe406c750 msgr2=0x7effe406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.891+0000 7effe8e00640 1 -- 172.21.15.7:0/921640920 shutdown_connections 2024-04-25T11:22:04.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.891+0000 7effe8e00640 1 -- 172.21.15.7:0/921640920 wait complete. 2024-04-25T11:22:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.893+0000 7effe8e00640 1 Processor -- start 2024-04-25T11:22:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.893+0000 7effe8e00640 1 -- start start 2024-04-25T11:22:04.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.895+0000 7effe8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe40d06c0 0x7effe40cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.895+0000 7effe8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe40d2550 0x7effe40b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.895+0000 7effe8e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe4160180 0x7effe4162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.895+0000 7effe8e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7effe40d0ad0 con 0x7effe40d06c0 2024-04-25T11:22:04.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.895+0000 7effe8e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7effe41648d0 con 0x7effe40d2550 2024-04-25T11:22:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.895+0000 7effe8e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7effe40b95d0 con 0x7effe4160180 2024-04-25T11:22:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.895+0000 7effe1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe40d06c0 0x7effe40cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.895+0000 7effe1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe40d06c0 0x7effe40cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56094/0 (socket says 172.21.15.7:56094) 2024-04-25T11:22:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.895+0000 7effe1e00640 1 -- 172.21.15.7:0/3278563793 learned_addr learned my addr 172.21.15.7:0/3278563793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.896+0000 7effe1400640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe40d2550 0x7effe40b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.896+0000 7effe2800640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe4160180 0x7effe4162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.896+0000 7effe1e00640 1 -- 172.21.15.7:0/3278563793 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe4160180 msgr2=0x7effe4162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.896+0000 7effe1e00640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe4160180 0x7effe4162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.896+0000 7effe1e00640 1 -- 172.21.15.7:0/3278563793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe40d2550 msgr2=0x7effe40b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.896+0000 7effe1e00640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe40d2550 0x7effe40b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.896+0000 7effe1e00640 1 -- 172.21.15.7:0/3278563793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe40cf8e0 con 0x7effe40d06c0 2024-04-25T11:22:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.896+0000 7effe2800640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe4160180 0x7effe4162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.896+0000 7effe1e00640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe40d06c0 0x7effe40cf3a0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7effd400d5f0 tx=0x7effd400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.896+0000 7effda000640 1 -- 172.21.15.7:0/3278563793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd40125f0 con 0x7effe40d06c0 2024-04-25T11:22:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.897+0000 7effda000640 1 -- 172.21.15.7:0/3278563793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd4019070 con 0x7effe40d06c0 2024-04-25T11:22:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.897+0000 7effda000640 1 -- 172.21.15.7:0/3278563793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd40129d0 con 0x7effe40d06c0 2024-04-25T11:22:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.897+0000 7effe8e00640 1 -- 172.21.15.7:0/3278563793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe40b41a0 con 0x7effe40d06c0 2024-04-25T11:22:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.897+0000 7effe8e00640 1 -- 172.21.15.7:0/3278563793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe40b4760 con 0x7effe40d06c0 2024-04-25T11:22:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.898+0000 7effe8e00640 1 -- 172.21.15.7:0/3278563793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe40d52d0 con 0x7effe40d06c0 2024-04-25T11:22:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.899+0000 7effda000640 1 -- 172.21.15.7:0/3278563793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7effd4012b70 con 0x7effe40d06c0 2024-04-25T11:22:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.901+0000 7effda000640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 0x7effc40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.901+0000 7effda000640 1 -- 172.21.15.7:0/3278563793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7effd40957d0 con 0x7effe40d06c0 2024-04-25T11:22:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.901+0000 7effe1400640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 0x7effc40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.902+0000 7effe1400640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 0x7effc40769c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7effd0034000 tx=0x7effd002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:04.906+0000 7effda000640 1 -- 172.21.15.7:0/3278563793 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7effe40d52d0 con 0x7effe40d06c0 2024-04-25T11:22:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.113+0000 7effe8e00640 1 -- 172.21.15.7:0/3278563793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7effe4050740 con 0x7effe40d06c0 2024-04-25T11:22:05.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.113+0000 7effda000640 1 -- 172.21.15.7:0/3278563793 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) ==== 100+0+11 (secure 0 0 0) 0x7effe40cf8e0 con 0x7effe40d06c0 2024-04-25T11:22:05.110 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 -- 172.21.15.7:0/3278563793 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 msgr2=0x7effc40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 0x7effc40769c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7effd0034000 tx=0x7effd002e000 comp rx=0 tx=0).stop 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 -- 172.21.15.7:0/3278563793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe40d06c0 msgr2=0x7effe40cf3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe40d06c0 0x7effe40cf3a0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7effd400d5f0 tx=0x7effd400dab0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 -- 172.21.15.7:0/3278563793 shutdown_connections 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7effc40744b0 0x7effc40769c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effe4160180 0x7effe4162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effe40d2550 0x7effe40b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 --2- 172.21.15.7:0/3278563793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effe40d06c0 0x7effe40cf3a0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 -- 172.21.15.7:0/3278563793 >> 172.21.15.7:0/3278563793 conn(0x7effe406c750 msgr2=0x7effe40b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 -- 172.21.15.7:0/3278563793 shutdown_connections 2024-04-25T11:22:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.116+0000 7effe8e00640 1 -- 172.21.15.7:0/3278563793 wait complete. 2024-04-25T11:22:05.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:22:05.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.423+0000 7f6462400640 1 Processor -- start 2024-04-25T11:22:05.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.424+0000 7f6462400640 1 -- start start 2024-04-25T11:22:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.426+0000 7f6462400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c15eed0 0x7f645c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.426+0000 7f6462400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f645c141290 0x7f645c1416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.426+0000 7f6462400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645c06c570 con 0x7f645c06d610 2024-04-25T11:22:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.426+0000 7f6462400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f645c06c850 con 0x7f645c141290 2024-04-25T11:22:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.426+0000 7f6462400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f645c06cb30 con 0x7f645c15eed0 2024-04-25T11:22:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.426+0000 7f645be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f645c141290 0x7f645c1416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.426+0000 7f645be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f645c141290 0x7f645c1416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56100/0 (socket says 172.21.15.7:56100) 2024-04-25T11:22:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.426+0000 7f645be00640 1 -- 172.21.15.7:0/2675743592 learned_addr learned my addr 172.21.15.7:0/2675743592 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.426+0000 7f6461a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f645c06d610 0x7f645c0b4590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37522/0 (socket says 172.21.15.7:37522) 2024-04-25T11:22:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.427+0000 7f6461000640 1 --2- 172.21.15.7:0/2675743592 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c15eed0 0x7f645c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:05.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.427+0000 7f645be00640 1 -- 172.21.15.7:0/2675743592 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c15eed0 msgr2=0x7f645c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.427+0000 7f645be00640 1 --2- 172.21.15.7:0/2675743592 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c15eed0 0x7f645c0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.427+0000 7f645be00640 1 -- 172.21.15.7:0/2675743592 >> v1:172.21.15.7:6789/0 conn(0x7f645c06d610 legacy=0x7f645c0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.427+0000 7f645be00640 1 -- 172.21.15.7:0/2675743592 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c06ce10 con 0x7f645c141290 2024-04-25T11:22:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.427+0000 7f645b400640 1 -- 172.21.15.7:0/2675743592 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2343398563 0 0) 0x7f645c06c570 con 0x7f645c06d610 2024-04-25T11:22:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.428+0000 7f645be00640 1 --2- 172.21.15.7:0/2675743592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f645c141290 0x7f645c1416a0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f6448009870 tx=0x7f644802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305e8c8c5d413b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.429+0000 7f645b400640 1 -- 172.21.15.7:0/2675743592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448037070 con 0x7f645c141290 2024-04-25T11:22:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.429+0000 7f645b400640 1 -- 172.21.15.7:0/2675743592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6448032440 con 0x7f645c141290 2024-04-25T11:22:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.429+0000 7f645b400640 1 -- 172.21.15.7:0/2675743592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448032810 con 0x7f645c141290 2024-04-25T11:22:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.429+0000 7f6462400640 1 -- 172.21.15.7:0/2675743592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f645c141290 msgr2=0x7f645c1416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.429+0000 7f6462400640 1 --2- 172.21.15.7:0/2675743592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f645c141290 0x7f645c1416a0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f6448009870 tx=0x7f644802a250 comp rx=0 tx=0).stop 2024-04-25T11:22:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.429+0000 7f6462400640 1 -- 172.21.15.7:0/2675743592 shutdown_connections 2024-04-25T11:22:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.429+0000 7f6462400640 1 --2- 172.21.15.7:0/2675743592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f645c141290 0x7f645c1416a0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.429+0000 7f6462400640 1 --2- 172.21.15.7:0/2675743592 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c15eed0 0x7f645c0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.430+0000 7f6462400640 1 -- 172.21.15.7:0/2675743592 >> 172.21.15.7:0/2675743592 conn(0x7f645c0d1540 msgr2=0x7f645c0d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.430+0000 7f6462400640 1 -- 172.21.15.7:0/2675743592 shutdown_connections 2024-04-25T11:22:05.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.430+0000 7f6462400640 1 -- 172.21.15.7:0/2675743592 wait complete. 2024-04-25T11:22:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.432+0000 7f6462400640 1 Processor -- start 2024-04-25T11:22:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.432+0000 7f6462400640 1 -- start start 2024-04-25T11:22:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.433+0000 7f6462400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c06d610 0x7f645c1ebf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f6462400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f645c141290 0x7f645c1e9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f6462400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f645c15eed0 0x7f645c1eb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f6462400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f645c140da0 con 0x7f645c141290 2024-04-25T11:22:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f6462400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f645c0cdcc0 con 0x7f645c15eed0 2024-04-25T11:22:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f6462400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f645c140f20 con 0x7f645c06d610 2024-04-25T11:22:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f645be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f645c141290 0x7f645c1e9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f645be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f645c141290 0x7f645c1e9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56106/0 (socket says 172.21.15.7:56106) 2024-04-25T11:22:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f645be00640 1 -- 172.21.15.7:0/1455219900 learned_addr learned my addr 172.21.15.7:0/1455219900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f6461000640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c06d610 0x7f645c1ebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f645be00640 1 -- 172.21.15.7:0/1455219900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c06d610 msgr2=0x7f645c1ebf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f6461a00640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f645c15eed0 0x7f645c1eb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f645be00640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c06d610 0x7f645c1ebf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f645be00640 1 -- 172.21.15.7:0/1455219900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f645c15eed0 msgr2=0x7f645c1eb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f645be00640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f645c15eed0 0x7f645c1eb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.434+0000 7f645be00640 1 -- 172.21.15.7:0/1455219900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c1e9f70 con 0x7f645c141290 2024-04-25T11:22:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.435+0000 7f645be00640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f645c141290 0x7f645c1e9a30 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f6448009870 tx=0x7f64480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.435+0000 7f6458c00640 1 -- 172.21.15.7:0/1455219900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448037040 con 0x7f645c141290 2024-04-25T11:22:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.435+0000 7f6458c00640 1 -- 172.21.15.7:0/1455219900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6448004230 con 0x7f645c141290 2024-04-25T11:22:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.435+0000 7f6458c00640 1 -- 172.21.15.7:0/1455219900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448003610 con 0x7f645c141290 2024-04-25T11:22:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.435+0000 7f6462400640 1 -- 172.21.15.7:0/1455219900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c1ea200 con 0x7f645c141290 2024-04-25T11:22:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.435+0000 7f6462400640 1 -- 172.21.15.7:0/1455219900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c1ea7c0 con 0x7f645c141290 2024-04-25T11:22:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.436+0000 7f6462400640 1 -- 172.21.15.7:0/1455219900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c050740 con 0x7f645c141290 2024-04-25T11:22:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.437+0000 7f6458c00640 1 -- 172.21.15.7:0/1455219900 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f64480037b0 con 0x7f645c141290 2024-04-25T11:22:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.439+0000 7f6458c00640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f643c0744b0 0x7f643c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.439+0000 7f6458c00640 1 -- 172.21.15.7:0/1455219900 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7f64480b6290 con 0x7f645c141290 2024-04-25T11:22:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.439+0000 7f6461000640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f643c0744b0 0x7f643c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:05.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.440+0000 7f6461000640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f643c0744b0 0x7f643c0769c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f645c071fe0 tx=0x7f644c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:05.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.444+0000 7f6458c00640 1 -- 172.21.15.7:0/1455219900 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f64480807c0 con 0x7f645c141290 2024-04-25T11:22:05.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.661+0000 7f6462400640 1 -- 172.21.15.7:0/1455219900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f645c0b5c70 con 0x7f645c141290 2024-04-25T11:22:05.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.662+0000 7f6458c00640 1 -- 172.21.15.7:0/1455219900 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v144) ==== 80+0+0 (secure 0 0 0) 0x7f64480800c0 con 0x7f645c141290 2024-04-25T11:22:05.658 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:22:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 -- 172.21.15.7:0/1455219900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f643c0744b0 msgr2=0x7f643c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f643c0744b0 0x7f643c0769c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f645c071fe0 tx=0x7f644c00b040 comp rx=0 tx=0).stop 2024-04-25T11:22:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 -- 172.21.15.7:0/1455219900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f645c141290 msgr2=0x7f645c1e9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f645c141290 0x7f645c1e9a30 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f6448009870 tx=0x7f64480046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 -- 172.21.15.7:0/1455219900 shutdown_connections 2024-04-25T11:22:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f643c0744b0 0x7f643c0769c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f645c15eed0 0x7f645c1eb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f645c141290 0x7f645c1e9a30 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 --2- 172.21.15.7:0/1455219900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c06d610 0x7f645c1ebf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 -- 172.21.15.7:0/1455219900 >> 172.21.15.7:0/1455219900 conn(0x7f645c0d1540 msgr2=0x7f645c0b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 -- 172.21.15.7:0/1455219900 shutdown_connections 2024-04-25T11:22:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:05.665+0000 7f6462400640 1 -- 172.21.15.7:0/1455219900 wait complete. 2024-04-25T11:22:05.687 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:22:05.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:05.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:22:05.898 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:22:05.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:05.898 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:22:05.938 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:22:05.938 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:22:05.939 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:05.979 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:22:05.979 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:06.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:06.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.356+0000 7f9249400640 1 Processor -- start 2024-04-25T11:22:06.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.356+0000 7f9249400640 1 -- start start 2024-04-25T11:22:06.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.358+0000 7f9249400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9244163bb0 0x7f92440d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.358+0000 7f9249400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9244164120 con 0x7f924406c570 2024-04-25T11:22:06.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.358+0000 7f9249400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9244164400 con 0x7f92440bae90 2024-04-25T11:22:06.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.358+0000 7f9249400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f92440d08e0 con 0x7f9244163bb0 2024-04-25T11:22:06.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.359+0000 7f9248a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f924406c570 0x7f924406c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37530/0 (socket says 172.21.15.7:37530) 2024-04-25T11:22:06.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.359+0000 7f9248a00640 1 -- 172.21.15.7:0/3780926719 learned_addr learned my addr 172.21.15.7:0/3780926719 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:06.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.359+0000 7f9243e00640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9244163bb0 0x7f92440d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.359+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 47197283 0 0) 0x7f9244164120 con 0x7f924406c570 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.359+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9228003520 con 0x7f924406c570 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1704087089 0 0) 0x7f9228003520 con 0x7f924406c570 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9244163bb0 msgr2=0x7f92440d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9244163bb0 0x7f92440d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 >> v1:172.21.15.87:6789/0 conn(0x7f92440bae90 legacy=0x7f92440b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92441646e0 con 0x7f924406c570 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f923c001a10 con 0x7f924406c570 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 >> v1:172.21.15.7:6789/0 conn(0x7f924406c570 legacy=0x7f924406c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9244163bb0 0x7f92280039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 0x7f9228004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9228004d40 0x7f9228005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92440d08e0 con 0x7f9228003f60 2024-04-25T11:22:06.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f922800bb50 con 0x7f9228004d40 2024-04-25T11:22:06.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f922800bd50 con 0x7f9244163bb0 2024-04-25T11:22:06.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.360+0000 7f9243400640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 0x7f9228004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.361+0000 7f9243e00640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9244163bb0 0x7f92280039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.361+0000 7f9248a00640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9228004d40 0x7f9228005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.361+0000 7f9243400640 1 -- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9244163bb0 msgr2=0x7f92280039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.361+0000 7f9243400640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9244163bb0 0x7f92280039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.361+0000 7f9243400640 1 -- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9228004d40 msgr2=0x7f9228005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.361+0000 7f9243400640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9228004d40 0x7f9228005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.361+0000 7f9243400640 1 -- 172.21.15.7:0/3780926719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9228003520 con 0x7f9228003f60 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.361+0000 7f9248a00640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9228004d40 0x7f9228005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.362+0000 7f9243400640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 0x7f9228004800 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f92440d68a0 tx=0x7f9228005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34b660d09077f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.362+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230001370 con 0x7f9228003f60 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.362+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92300052d0 con 0x7f9228003f60 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.362+0000 7f9242a00640 1 -- 172.21.15.7:0/3780926719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923002ab90 con 0x7f9228003f60 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.363+0000 7f9249400640 1 -- 172.21.15.7:0/3780926719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 msgr2=0x7f9228004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.363+0000 7f9249400640 1 --2- 172.21.15.7:0/3780926719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 0x7f9228004800 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f92440d68a0 tx=0x7f9228005830 comp rx=0 tx=0).stop 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.363+0000 7f9248a00640 1 -- 172.21.15.7:0/3780926719 reap_dead start 2024-04-25T11:22:06.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.363+0000 7f9249400640 1 -- 172.21.15.7:0/3780926719 shutdown_connections 2024-04-25T11:22:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.363+0000 7f9249400640 1 -- 172.21.15.7:0/3780926719 >> 172.21.15.7:0/3780926719 conn(0x7f924406eb00 msgr2=0x7f92440d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.363+0000 7f9249400640 1 -- 172.21.15.7:0/3780926719 shutdown_connections 2024-04-25T11:22:06.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.363+0000 7f9249400640 1 -- 172.21.15.7:0/3780926719 wait complete. 2024-04-25T11:22:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.366+0000 7f9249400640 1 Processor -- start 2024-04-25T11:22:06.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.366+0000 7f9249400640 1 -- start start 2024-04-25T11:22:06.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.367+0000 7f9249400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 0x7f924415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.367+0000 7f9249400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f924415b6a0 0x7f9244150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.367+0000 7f9249400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92440d4590 0x7f92440d49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.367+0000 7f9249400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92440d4170 con 0x7f9228003f60 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.367+0000 7f9249400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f92440b51c0 con 0x7f924415b6a0 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.367+0000 7f9249400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f92440bb2a0 con 0x7f92440d4590 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.368+0000 7f9243e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 0x7f924415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.368+0000 7f9243e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 0x7f924415ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56124/0 (socket says 172.21.15.7:56124) 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.368+0000 7f9243e00640 1 -- 172.21.15.7:0/894572128 learned_addr learned my addr 172.21.15.7:0/894572128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.368+0000 7f9243400640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f924415b6a0 0x7f9244150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.368+0000 7f9248a00640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92440d4590 0x7f92440d49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.368+0000 7f9243e00640 1 -- 172.21.15.7:0/894572128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92440d4590 msgr2=0x7f92440d49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.368+0000 7f9243e00640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92440d4590 0x7f92440d49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.368+0000 7f9243e00640 1 -- 172.21.15.7:0/894572128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f924415b6a0 msgr2=0x7f9244150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.368+0000 7f9243e00640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f924415b6a0 0x7f9244150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.368+0000 7f9243e00640 1 -- 172.21.15.7:0/894572128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924414e030 con 0x7f9228003f60 2024-04-25T11:22:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.369+0000 7f9243e00640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 0x7f924415ec00 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f9234011600 tx=0x7f9234011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.369+0000 7f923be00640 1 -- 172.21.15.7:0/894572128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923400f920 con 0x7f9228003f60 2024-04-25T11:22:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.369+0000 7f923be00640 1 -- 172.21.15.7:0/894572128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f923400fac0 con 0x7f9228003f60 2024-04-25T11:22:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.369+0000 7f9249400640 1 -- 172.21.15.7:0/894572128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92440d4ee0 con 0x7f9228003f60 2024-04-25T11:22:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.369+0000 7f923be00640 1 -- 172.21.15.7:0/894572128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923400fe70 con 0x7f9228003f60 2024-04-25T11:22:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.369+0000 7f9249400640 1 -- 172.21.15.7:0/894572128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f924414b700 con 0x7f9228003f60 2024-04-25T11:22:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.370+0000 7f9249400640 1 -- 172.21.15.7:0/894572128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9244050740 con 0x7f9228003f60 2024-04-25T11:22:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.372+0000 7f923be00640 1 -- 172.21.15.7:0/894572128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9234015460 con 0x7f9228003f60 2024-04-25T11:22:06.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.373+0000 7f923be00640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92240744b0 0x7f92240769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.373+0000 7f923be00640 1 -- 172.21.15.7:0/894572128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7f9234095d60 con 0x7f9228003f60 2024-04-25T11:22:06.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.373+0000 7f9243400640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92240744b0 0x7f92240769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.374+0000 7f9243400640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92240744b0 0x7f92240769c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f92440d68a0 tx=0x7f923002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:06.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.378+0000 7f923be00640 1 -- 172.21.15.7:0/894572128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f92440d4ee0 con 0x7f9228003f60 2024-04-25T11:22:06.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.584+0000 7f9249400640 1 -- 172.21.15.7:0/894572128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92440d3d20 con 0x7f9228003f60 2024-04-25T11:22:06.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.586+0000 7f923be00640 1 -- 172.21.15.7:0/894572128 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16954 (secure 0 0 0) 0x7f9234019030 con 0x7f9228003f60 2024-04-25T11:22:06.583 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:06.583 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":144,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:04.499877+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:21:35.562066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T11:21:36.342958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.34","mappings":[{"from":3,"to":0}]},{"pgid":"22.37","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.589+0000 7f9249400640 1 -- 172.21.15.7:0/894572128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92240744b0 msgr2=0x7f92240769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.589+0000 7f9249400640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92240744b0 0x7f92240769c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f92440d68a0 tx=0x7f923002e000 comp rx=0 tx=0).stop 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.589+0000 7f9249400640 1 -- 172.21.15.7:0/894572128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 msgr2=0x7f924415ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.589+0000 7f9249400640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 0x7f924415ec00 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f9234011600 tx=0x7f9234011ac0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.589+0000 7f9249400640 1 -- 172.21.15.7:0/894572128 shutdown_connections 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.589+0000 7f9249400640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92240744b0 0x7f92240769c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.589+0000 7f9249400640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92440d4590 0x7f92440d49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.589+0000 7f9249400640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f924415b6a0 0x7f9244150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.589+0000 7f9249400640 1 --2- 172.21.15.7:0/894572128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9228003f60 0x7f924415ec00 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.589+0000 7f9249400640 1 -- 172.21.15.7:0/894572128 >> 172.21.15.7:0/894572128 conn(0x7f924406eb00 msgr2=0x7f92440b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.590+0000 7f9249400640 1 -- 172.21.15.7:0/894572128 shutdown_connections 2024-04-25T11:22:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.590+0000 7f9249400640 1 -- 172.21.15.7:0/894572128 wait complete. 2024-04-25T11:22:06.615 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:21:35.562066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:21:36.342958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:22:06.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:22:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.908+0000 7ff88cc00640 1 Processor -- start 2024-04-25T11:22:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.908+0000 7ff88cc00640 1 -- start start 2024-04-25T11:22:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.910+0000 7ff88cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8881583c0 0x7ff8880cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.911+0000 7ff88cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff8880cfb50 0x7ff8880d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.911+0000 7ff88cc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8880ce570 con 0x7ff8880d09c0 2024-04-25T11:22:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.911+0000 7ff88cc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff88815f190 con 0x7ff8880cfb50 2024-04-25T11:22:06.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.911+0000 7ff88cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff8880ce850 con 0x7ff8881583c0 2024-04-25T11:22:06.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.911+0000 7ff887400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8881583c0 0x7ff8880cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.911+0000 7ff886a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff8880cfb50 0x7ff8880d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.912+0000 7ff887400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8881583c0 0x7ff8880cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:41406/0 (socket says 172.21.15.7:41406) 2024-04-25T11:22:06.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.912+0000 7ff887400640 1 -- 172.21.15.7:0/891200421 learned_addr learned my addr 172.21.15.7:0/891200421 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:06.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.912+0000 7ff886a00640 1 -- 172.21.15.7:0/891200421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8881583c0 msgr2=0x7ff8880cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:06.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.912+0000 7ff886a00640 1 --2- 172.21.15.7:0/891200421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8881583c0 0x7ff8880cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.912+0000 7ff886a00640 1 -- 172.21.15.7:0/891200421 >> v1:172.21.15.87:6789/0 conn(0x7ff8880d09c0 legacy=0x7ff88806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:06.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.912+0000 7ff886a00640 1 -- 172.21.15.7:0/891200421 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8880ceb30 con 0x7ff8880cfb50 2024-04-25T11:22:06.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.914+0000 7ff886a00640 1 --2- 172.21.15.7:0/891200421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8880cfb50 0x7ff8880d0480 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff8700097a0 tx=0x7ff87002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19c42881d36e97a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.915+0000 7ff886000640 1 -- 172.21.15.7:0/891200421 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870037070 con 0x7ff8880cfb50 2024-04-25T11:22:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.915+0000 7ff886000640 1 -- 172.21.15.7:0/891200421 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff870032440 con 0x7ff8880cfb50 2024-04-25T11:22:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.915+0000 7ff886000640 1 -- 172.21.15.7:0/891200421 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870032810 con 0x7ff8880cfb50 2024-04-25T11:22:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.915+0000 7ff88cc00640 1 -- 172.21.15.7:0/891200421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8880cfb50 msgr2=0x7ff8880d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.915+0000 7ff88cc00640 1 --2- 172.21.15.7:0/891200421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8880cfb50 0x7ff8880d0480 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff8700097a0 tx=0x7ff87002a180 comp rx=0 tx=0).stop 2024-04-25T11:22:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.916+0000 7ff88cc00640 1 -- 172.21.15.7:0/891200421 shutdown_connections 2024-04-25T11:22:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.916+0000 7ff88cc00640 1 --2- 172.21.15.7:0/891200421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8880cfb50 0x7ff8880d0480 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.916+0000 7ff88cc00640 1 --2- 172.21.15.7:0/891200421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8881583c0 0x7ff8880cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.916+0000 7ff88cc00640 1 -- 172.21.15.7:0/891200421 >> 172.21.15.7:0/891200421 conn(0x7ff8880740f0 msgr2=0x7ff88806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.916+0000 7ff88cc00640 1 -- 172.21.15.7:0/891200421 shutdown_connections 2024-04-25T11:22:06.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.916+0000 7ff88cc00640 1 -- 172.21.15.7:0/891200421 wait complete. 2024-04-25T11:22:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.918+0000 7ff88cc00640 1 Processor -- start 2024-04-25T11:22:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.918+0000 7ff88cc00640 1 -- start start 2024-04-25T11:22:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.920+0000 7ff88cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8880cfb50 0x7ff88814f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.920+0000 7ff88cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880d09c0 0x7ff88814ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.920+0000 7ff88cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8881583c0 0x7ff8881f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.920+0000 7ff88cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff88806f5b0 con 0x7ff8880d09c0 2024-04-25T11:22:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.920+0000 7ff88cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff8880cf0f0 con 0x7ff8881583c0 2024-04-25T11:22:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.920+0000 7ff88cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff88806f730 con 0x7ff8880cfb50 2024-04-25T11:22:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.920+0000 7ff886a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880d09c0 0x7ff88814ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.920+0000 7ff886a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880d09c0 0x7ff88814ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56134/0 (socket says 172.21.15.7:56134) 2024-04-25T11:22:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.921+0000 7ff886a00640 1 -- 172.21.15.7:0/2803942138 learned_addr learned my addr 172.21.15.7:0/2803942138 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.921+0000 7ff887400640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8880cfb50 0x7ff88814f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.921+0000 7ff887e00640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8881583c0 0x7ff8881f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.921+0000 7ff886a00640 1 -- 172.21.15.7:0/2803942138 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8880cfb50 msgr2=0x7ff88814f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:06.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.921+0000 7ff886a00640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8880cfb50 0x7ff88814f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.921+0000 7ff886a00640 1 -- 172.21.15.7:0/2803942138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8881583c0 msgr2=0x7ff8881f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:06.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.921+0000 7ff886a00640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8881583c0 0x7ff8881f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:06.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.921+0000 7ff886a00640 1 -- 172.21.15.7:0/2803942138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8881f7100 con 0x7ff8880d09c0 2024-04-25T11:22:06.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.922+0000 7ff886a00640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880d09c0 0x7ff88814ec50 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7ff87002a770 tx=0x7ff8700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:06.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.922+0000 7ff87f400640 1 -- 172.21.15.7:0/2803942138 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870037050 con 0x7ff8880d09c0 2024-04-25T11:22:06.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.922+0000 7ff87f400640 1 -- 172.21.15.7:0/2803942138 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff870043070 con 0x7ff8880d09c0 2024-04-25T11:22:06.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.922+0000 7ff87f400640 1 -- 172.21.15.7:0/2803942138 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87003cb30 con 0x7ff8880d09c0 2024-04-25T11:22:06.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.922+0000 7ff88cc00640 1 -- 172.21.15.7:0/2803942138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8881f73e0 con 0x7ff8880d09c0 2024-04-25T11:22:06.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.922+0000 7ff88cc00640 1 -- 172.21.15.7:0/2803942138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8881e8b10 con 0x7ff8880d09c0 2024-04-25T11:22:06.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.923+0000 7ff88cc00640 1 -- 172.21.15.7:0/2803942138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff888050740 con 0x7ff8880d09c0 2024-04-25T11:22:06.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.925+0000 7ff87f400640 1 -- 172.21.15.7:0/2803942138 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff87003ccd0 con 0x7ff8880d09c0 2024-04-25T11:22:06.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.926+0000 7ff87f400640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744b0 0x7ff8640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:06.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.926+0000 7ff87f400640 1 -- 172.21.15.7:0/2803942138 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7ff8700b6290 con 0x7ff8880d09c0 2024-04-25T11:22:06.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.926+0000 7ff887400640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744b0 0x7ff8640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:06.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.927+0000 7ff887400640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744b0 0x7ff8640769c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff8880aa990 tx=0x7ff878008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:06.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:06.931+0000 7ff87f400640 1 -- 172.21.15.7:0/2803942138 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff8700807e0 con 0x7ff8880d09c0 2024-04-25T11:22:07.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.137+0000 7ff88cc00640 1 -- 172.21.15.7:0/2803942138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8881f6e90 con 0x7ff8880d09c0 2024-04-25T11:22:07.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.138+0000 7ff87f400640 1 -- 172.21.15.7:0/2803942138 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) ==== 93+0+10 (secure 0 0 0) 0x7ff8700800e0 con 0x7ff8880d09c0 2024-04-25T11:22:07.134 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:22:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.140+0000 7ff88cc00640 1 -- 172.21.15.7:0/2803942138 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744b0 msgr2=0x7ff8640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.140+0000 7ff88cc00640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744b0 0x7ff8640769c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff8880aa990 tx=0x7ff878008040 comp rx=0 tx=0).stop 2024-04-25T11:22:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.140+0000 7ff88cc00640 1 -- 172.21.15.7:0/2803942138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880d09c0 msgr2=0x7ff88814ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.140+0000 7ff88cc00640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880d09c0 0x7ff88814ec50 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7ff87002a770 tx=0x7ff8700046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.140+0000 7ff88cc00640 1 -- 172.21.15.7:0/2803942138 shutdown_connections 2024-04-25T11:22:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.140+0000 7ff88cc00640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff8640744b0 0x7ff8640769c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.140+0000 7ff88cc00640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8881583c0 0x7ff8881f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.140+0000 7ff88cc00640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8880d09c0 0x7ff88814ec50 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.140+0000 7ff88cc00640 1 --2- 172.21.15.7:0/2803942138 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff8880cfb50 0x7ff88814f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.140+0000 7ff88cc00640 1 -- 172.21.15.7:0/2803942138 >> 172.21.15.7:0/2803942138 conn(0x7ff8880740f0 msgr2=0x7ff888152dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.141+0000 7ff88cc00640 1 -- 172.21.15.7:0/2803942138 shutdown_connections 2024-04-25T11:22:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.141+0000 7ff88cc00640 1 -- 172.21.15.7:0/2803942138 wait complete. 2024-04-25T11:22:07.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:22:07.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.463+0000 7fcdb5c00640 1 Processor -- start 2024-04-25T11:22:07.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.463+0000 7fcdb5c00640 1 -- start start 2024-04-25T11:22:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.466+0000 7fcdb5c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcdb015aaa0 0x7fcdb00affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.466+0000 7fcdb5c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcdb00d0ee0 0x7fcdb00b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.466+0000 7fcdb5c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb006f5b0 con 0x7fcdb00b4e50 2024-04-25T11:22:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.466+0000 7fcdb5c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb00af290 con 0x7fcdb015aaa0 2024-04-25T11:22:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.466+0000 7fcdb5c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb00d14a0 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.466+0000 7fcdaea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcdb015aaa0 0x7fcdb00affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.466+0000 7fcdaea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcdb015aaa0 0x7fcdb00affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56144/0 (socket says 172.21.15.7:56144) 2024-04-25T11:22:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.466+0000 7fcdaea00640 1 -- 172.21.15.7:0/3116188838 learned_addr learned my addr 172.21.15.7:0/3116188838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.467+0000 7fcda6000640 1 --2- 172.21.15.7:0/3116188838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcdb00d0ee0 0x7fcdb00b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.467+0000 7fcdaea00640 1 -- 172.21.15.7:0/3116188838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcdb00d0ee0 msgr2=0x7fcdb00b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.467+0000 7fcdaea00640 1 --2- 172.21.15.7:0/3116188838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcdb00d0ee0 0x7fcdb00b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.467+0000 7fcdaea00640 1 -- 172.21.15.7:0/3116188838 >> v1:172.21.15.7:6789/0 conn(0x7fcdb00b4e50 legacy=0x7fcdb015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.467+0000 7fcdaea00640 1 -- 172.21.15.7:0/3116188838 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb00af4e0 con 0x7fcdb015aaa0 2024-04-25T11:22:07.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.468+0000 7fcdaea00640 1 --2- 172.21.15.7:0/3116188838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb015aaa0 0x7fcdb00affc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fcda000bb10 tx=0x7fcda002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3e9f7541b3a573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:07.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.468+0000 7fcdae000640 1 -- 172.21.15.7:0/3116188838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0032790 con 0x7fcdb015aaa0 2024-04-25T11:22:07.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.468+0000 7fcdae000640 1 -- 172.21.15.7:0/3116188838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda0038070 con 0x7fcdb015aaa0 2024-04-25T11:22:07.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.468+0000 7fcdae000640 1 -- 172.21.15.7:0/3116188838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0032b50 con 0x7fcdb015aaa0 2024-04-25T11:22:07.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.469+0000 7fcdb5c00640 1 -- 172.21.15.7:0/3116188838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb015aaa0 msgr2=0x7fcdb00affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:07.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.469+0000 7fcdb5c00640 1 --2- 172.21.15.7:0/3116188838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb015aaa0 0x7fcdb00affc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fcda000bb10 tx=0x7fcda002c890 comp rx=0 tx=0).stop 2024-04-25T11:22:07.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.469+0000 7fcdb5c00640 1 -- 172.21.15.7:0/3116188838 shutdown_connections 2024-04-25T11:22:07.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.469+0000 7fcdb5c00640 1 --2- 172.21.15.7:0/3116188838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcdb00d0ee0 0x7fcdb00b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.469+0000 7fcdb5c00640 1 --2- 172.21.15.7:0/3116188838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb015aaa0 0x7fcdb00affc0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.469+0000 7fcdb5c00640 1 -- 172.21.15.7:0/3116188838 >> 172.21.15.7:0/3116188838 conn(0x7fcdb00742c0 msgr2=0x7fcdb006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:07.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.469+0000 7fcdb5c00640 1 -- 172.21.15.7:0/3116188838 shutdown_connections 2024-04-25T11:22:07.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.469+0000 7fcdb5c00640 1 -- 172.21.15.7:0/3116188838 wait complete. 2024-04-25T11:22:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.472+0000 7fcdb5c00640 1 Processor -- start 2024-04-25T11:22:07.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.472+0000 7fcdb5c00640 1 -- start start 2024-04-25T11:22:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.473+0000 7fcdb5c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdb00b4e50 0x7fcdb014e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.473+0000 7fcdb5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb00d0ee0 0x7fcdb0159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.473+0000 7fcdb5c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcdb015aaa0 0x7fcdb00b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.473+0000 7fcdb5c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb00b5260 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.473+0000 7fcdb5c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb00afaa0 con 0x7fcdb00b4e50 2024-04-25T11:22:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.473+0000 7fcdb5c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb00b53e0 con 0x7fcdb015aaa0 2024-04-25T11:22:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.474+0000 7fcda6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb00d0ee0 0x7fcdb0159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.474+0000 7fcda6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb00d0ee0 0x7fcdb0159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56160/0 (socket says 172.21.15.7:56160) 2024-04-25T11:22:07.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.474+0000 7fcda6000640 1 -- 172.21.15.7:0/910370439 learned_addr learned my addr 172.21.15.7:0/910370439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.474+0000 7fcda6000640 1 -- 172.21.15.7:0/910370439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcdb015aaa0 msgr2=0x7fcdb00b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.474+0000 7fcdaf400640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcdb015aaa0 0x7fcdb00b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.474+0000 7fcdaea00640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdb00b4e50 0x7fcdb014e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.474+0000 7fcda6000640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcdb015aaa0 0x7fcdb00b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.474+0000 7fcda6000640 1 -- 172.21.15.7:0/910370439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdb00b4e50 msgr2=0x7fcdb014e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.474+0000 7fcda6000640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdb00b4e50 0x7fcdb014e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.474+0000 7fcda6000640 1 -- 172.21.15.7:0/910370439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb0152130 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.474+0000 7fcdaf400640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcdb015aaa0 0x7fcdb00b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.475+0000 7fcda6000640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb00d0ee0 0x7fcdb0159d20 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800c820 tx=0x7fcd9800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.475+0000 7fcda7400640 1 -- 172.21.15.7:0/910370439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98015070 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.475+0000 7fcda7400640 1 -- 172.21.15.7:0/910370439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd98004020 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.475+0000 7fcdb5c00640 1 -- 172.21.15.7:0/910370439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb01f9bb0 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.475+0000 7fcda7400640 1 -- 172.21.15.7:0/910370439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd980043f0 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.475+0000 7fcdb5c00640 1 -- 172.21.15.7:0/910370439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb01fa300 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.476+0000 7fcdb5c00640 1 -- 172.21.15.7:0/910370439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb0050740 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.478+0000 7fcda7400640 1 -- 172.21.15.7:0/910370439 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcd98007830 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.479+0000 7fcda7400640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd8c0744e0 0x7fcd8c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:07.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.479+0000 7fcda7400640 1 -- 172.21.15.7:0/910370439 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7fcd980951e0 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.479+0000 7fcdaea00640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd8c0744e0 0x7fcd8c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:07.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.480+0000 7fcdaea00640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd8c0744e0 0x7fcd8c0769f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcda0037000 tx=0x7fcda0032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:07.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.484+0000 7fcda7400640 1 -- 172.21.15.7:0/910370439 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcd9805f7a0 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.689+0000 7fcdb5c00640 1 -- 172.21.15.7:0/910370439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcdb01f9d80 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.689+0000 7fcda7400640 1 -- 172.21.15.7:0/910370439 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) ==== 104+0+11 (secure 0 0 0) 0x7fcd9805f0a0 con 0x7fcdb00d0ee0 2024-04-25T11:22:07.686 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:22:07.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 -- 172.21.15.7:0/910370439 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd8c0744e0 msgr2=0x7fcd8c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:07.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd8c0744e0 0x7fcd8c0769f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcda0037000 tx=0x7fcda0032000 comp rx=0 tx=0).stop 2024-04-25T11:22:07.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 -- 172.21.15.7:0/910370439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb00d0ee0 msgr2=0x7fcdb0159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:07.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb00d0ee0 0x7fcdb0159d20 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800c820 tx=0x7fcd9800cce0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 -- 172.21.15.7:0/910370439 shutdown_connections 2024-04-25T11:22:07.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcd8c0744e0 0x7fcd8c0769f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcdb015aaa0 0x7fcdb00b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdb00d0ee0 0x7fcdb0159d20 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 --2- 172.21.15.7:0/910370439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdb00b4e50 0x7fcdb014e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:07.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 -- 172.21.15.7:0/910370439 >> 172.21.15.7:0/910370439 conn(0x7fcdb00742c0 msgr2=0x7fcdb006fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:07.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 -- 172.21.15.7:0/910370439 shutdown_connections 2024-04-25T11:22:07.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:07.693+0000 7fcdb5c00640 1 -- 172.21.15.7:0/910370439 wait complete. 2024-04-25T11:22:07.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:22:08.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.009+0000 7f1c5b400640 1 Processor -- start 2024-04-25T11:22:08.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.009+0000 7f1c5b400640 1 -- start start 2024-04-25T11:22:08.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.012+0000 7f1c5b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1c540d2c60 0x7f1c540d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.012+0000 7f1c5b400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c54164090 con 0x7f1c54160180 2024-04-25T11:22:08.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.012+0000 7f1c5b400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c54164bb0 con 0x7f1c54163c80 2024-04-25T11:22:08.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.012+0000 7f1c5b400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1c540d0c00 con 0x7f1c540d2c60 2024-04-25T11:22:08.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.012+0000 7f1c58a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1c54160180 0x7f1c540b9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37538/0 (socket says 172.21.15.7:37538) 2024-04-25T11:22:08.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.012+0000 7f1c58a00640 1 -- 172.21.15.7:0/4113711860 learned_addr learned my addr 172.21.15.7:0/4113711860 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:08.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.012+0000 7f1c59400640 1 --2- 172.21.15.7:0/4113711860 >> v2:172.21.15.87:3300/0 conn(0x7f1c540d2c60 0x7f1c540d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3565402074 0 0) 0x7f1c44001110 con 0x7f1c54160180 2024-04-25T11:22:08.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c38003520 con 0x7f1c54160180 2024-04-25T11:22:08.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 545777080 0 0) 0x7f1c38003520 con 0x7f1c54160180 2024-04-25T11:22:08.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 >> v2:172.21.15.87:3300/0 conn(0x7f1c540d2c60 msgr2=0x7f1c540d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 --2- 172.21.15.7:0/4113711860 >> v2:172.21.15.87:3300/0 conn(0x7f1c540d2c60 0x7f1c540d06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 >> v1:172.21.15.87:6789/0 conn(0x7f1c54163c80 legacy=0x7f1c540d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c54164310 con 0x7f1c54160180 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1c44001840 con 0x7f1c54160180 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 >> v1:172.21.15.7:6789/0 conn(0x7f1c54160180 legacy=0x7f1c540b9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 --2- 172.21.15.7:0/4113711860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c380039b0 0x7f1c38003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 --2- 172.21.15.7:0/4113711860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c38004800 0x7f1c38004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 --2- 172.21.15.7:0/4113711860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c380051c0 0x7f1c3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c540d0c00 con 0x7f1c38004800 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3800c160 con 0x7f1c380039b0 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1c3800c360 con 0x7f1c380051c0 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.013+0000 7f1c4be00640 1 --2- 172.21.15.7:0/4113711860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c38004800 0x7f1c38004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.014+0000 7f1c58a00640 1 --2- 172.21.15.7:0/4113711860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c380039b0 0x7f1c38003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.014+0000 7f1c4be00640 1 -- 172.21.15.7:0/4113711860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c380051c0 msgr2=0x7f1c3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.014+0000 7f1c4be00640 1 --2- 172.21.15.7:0/4113711860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c380051c0 0x7f1c3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.014+0000 7f1c4be00640 1 -- 172.21.15.7:0/4113711860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c380039b0 msgr2=0x7f1c38003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:08.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.014+0000 7f1c4be00640 1 --2- 172.21.15.7:0/4113711860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c380039b0 0x7f1c38003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.014+0000 7f1c4be00640 1 -- 172.21.15.7:0/4113711860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c38003520 con 0x7f1c38004800 2024-04-25T11:22:08.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.014+0000 7f1c59400640 1 -- 172.21.15.7:0/4113711860 reap_dead start 2024-04-25T11:22:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.015+0000 7f1c4be00640 1 --2- 172.21.15.7:0/4113711860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c38004800 0x7f1c38004c10 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c001750 tx=0x7f1c38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b4033bcfb30665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.015+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c001370 con 0x7f1c38004800 2024-04-25T11:22:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.015+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c3c0052d0 con 0x7f1c38004800 2024-04-25T11:22:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.015+0000 7f1c53e00640 1 -- 172.21.15.7:0/4113711860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c02abc0 con 0x7f1c38004800 2024-04-25T11:22:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.016+0000 7f1c5b400640 1 -- 172.21.15.7:0/4113711860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c38004800 msgr2=0x7f1c38004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.016+0000 7f1c5b400640 1 --2- 172.21.15.7:0/4113711860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c38004800 0x7f1c38004c10 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c001750 tx=0x7f1c38005830 comp rx=0 tx=0).stop 2024-04-25T11:22:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.016+0000 7f1c5b400640 1 -- 172.21.15.7:0/4113711860 shutdown_connections 2024-04-25T11:22:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.016+0000 7f1c5b400640 1 --2- 172.21.15.7:0/4113711860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c38004800 0x7f1c38004c10 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.016+0000 7f1c5b400640 1 -- 172.21.15.7:0/4113711860 >> 172.21.15.7:0/4113711860 conn(0x7f1c5406c750 msgr2=0x7f1c5406cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.016+0000 7f1c5b400640 1 -- 172.21.15.7:0/4113711860 shutdown_connections 2024-04-25T11:22:08.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.016+0000 7f1c5b400640 1 -- 172.21.15.7:0/4113711860 wait complete. 2024-04-25T11:22:08.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.019+0000 7f1c5b400640 1 Processor -- start 2024-04-25T11:22:08.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.019+0000 7f1c5b400640 1 -- start start 2024-04-25T11:22:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.020+0000 7f1c5b400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c38004800 0x7f1c54150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.020+0000 7f1c5b400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c540ad900 0x7f1c5414e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.020+0000 7f1c5b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c54162ce0 0x7f1c5415df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.020+0000 7f1c5b400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c541648d0 con 0x7f1c54162ce0 2024-04-25T11:22:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.020+0000 7f1c5b400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1c540d4510 con 0x7f1c38004800 2024-04-25T11:22:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.020+0000 7f1c5b400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1c540d06c0 con 0x7f1c540ad900 2024-04-25T11:22:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.021+0000 7f1c59400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c54162ce0 0x7f1c5415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.021+0000 7f1c59400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c54162ce0 0x7f1c5415df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56176/0 (socket says 172.21.15.7:56176) 2024-04-25T11:22:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.021+0000 7f1c59400640 1 -- 172.21.15.7:0/795445406 learned_addr learned my addr 172.21.15.7:0/795445406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.021+0000 7f1c4be00640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c540ad900 0x7f1c5414e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.021+0000 7f1c58a00640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c38004800 0x7f1c54150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.021+0000 7f1c59400640 1 -- 172.21.15.7:0/795445406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c540ad900 msgr2=0x7f1c5414e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.021+0000 7f1c59400640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c540ad900 0x7f1c5414e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.021+0000 7f1c59400640 1 -- 172.21.15.7:0/795445406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c38004800 msgr2=0x7f1c54150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.021+0000 7f1c59400640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c38004800 0x7f1c54150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.021+0000 7f1c59400640 1 -- 172.21.15.7:0/795445406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c540ac860 con 0x7f1c54162ce0 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.021+0000 7f1c59400640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c54162ce0 0x7f1c5415df60 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f1c540d38a0 tx=0x7f1c4c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.022+0000 7f1c51600640 1 -- 172.21.15.7:0/795445406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c00bcb0 con 0x7f1c54162ce0 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.022+0000 7f1c51600640 1 -- 172.21.15.7:0/795445406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c4c01a070 con 0x7f1c54162ce0 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.022+0000 7f1c51600640 1 -- 172.21.15.7:0/795445406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c014640 con 0x7f1c54162ce0 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.022+0000 7f1c5b400640 1 -- 172.21.15.7:0/795445406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c540acb40 con 0x7f1c54162ce0 2024-04-25T11:22:08.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.022+0000 7f1c5b400640 1 -- 172.21.15.7:0/795445406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c540ad100 con 0x7f1c54162ce0 2024-04-25T11:22:08.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.023+0000 7f1c5b400640 1 -- 172.21.15.7:0/795445406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c54050740 con 0x7f1c54162ce0 2024-04-25T11:22:08.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.024+0000 7f1c51600640 1 -- 172.21.15.7:0/795445406 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1c4c013070 con 0x7f1c54162ce0 2024-04-25T11:22:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.025+0000 7f1c51600640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c300744e0 0x7f1c300769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.025+0000 7f1c51600640 1 -- 172.21.15.7:0/795445406 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7f1c4c0956f0 con 0x7f1c54162ce0 2024-04-25T11:22:08.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.026+0000 7f1c58a00640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c300744e0 0x7f1c300769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.026+0000 7f1c58a00640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c300744e0 0x7f1c300769f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1c440012f0 tx=0x7f1c44000f30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:08.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.031+0000 7f1c51600640 1 -- 172.21.15.7:0/795445406 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1c4c05fcc0 con 0x7f1c54162ce0 2024-04-25T11:22:08.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.241+0000 7f1c5b400640 1 -- 172.21.15.7:0/795445406 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c540d3f90 con 0x7f1c54162ce0 2024-04-25T11:22:08.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.242+0000 7f1c51600640 1 -- 172.21.15.7:0/795445406 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) ==== 100+0+11 (secure 0 0 0) 0x7f1c4c05f5c0 con 0x7f1c54162ce0 2024-04-25T11:22:08.238 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:22:08.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 -- 172.21.15.7:0/795445406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c300744e0 msgr2=0x7f1c300769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c300744e0 0x7f1c300769f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1c440012f0 tx=0x7f1c44000f30 comp rx=0 tx=0).stop 2024-04-25T11:22:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 -- 172.21.15.7:0/795445406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c54162ce0 msgr2=0x7f1c5415df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c54162ce0 0x7f1c5415df60 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f1c540d38a0 tx=0x7f1c4c00b480 comp rx=0 tx=0).stop 2024-04-25T11:22:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 -- 172.21.15.7:0/795445406 shutdown_connections 2024-04-25T11:22:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1c300744e0 0x7f1c300769f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c54162ce0 0x7f1c5415df60 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c540ad900 0x7f1c5414e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 --2- 172.21.15.7:0/795445406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c38004800 0x7f1c54150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 -- 172.21.15.7:0/795445406 >> 172.21.15.7:0/795445406 conn(0x7f1c5406c750 msgr2=0x7f1c540b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 -- 172.21.15.7:0/795445406 shutdown_connections 2024-04-25T11:22:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.245+0000 7f1c5b400640 1 -- 172.21.15.7:0/795445406 wait complete. 2024-04-25T11:22:08.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:22:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.563+0000 7f1872200640 1 Processor -- start 2024-04-25T11:22:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.563+0000 7f1872200640 1 -- start start 2024-04-25T11:22:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.565+0000 7f1872200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f186c0b5190 0x7f186c0d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.565+0000 7f1872200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f186c06f390 0x7f186c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.565+0000 7f1872200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f186c0b55a0 con 0x7f186c06d580 2024-04-25T11:22:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.565+0000 7f1872200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f186c15d560 con 0x7f186c0b5190 2024-04-25T11:22:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.565+0000 7f1872200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f186c15d820 con 0x7f186c06f390 2024-04-25T11:22:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.566+0000 7f186aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f186c06d580 0x7f186c06d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37548/0 (socket says 172.21.15.7:37548) 2024-04-25T11:22:08.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.566+0000 7f186aa00640 1 -- 172.21.15.7:0/2956406423 learned_addr learned my addr 172.21.15.7:0/2956406423 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.566+0000 7f186be00640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f186c06f390 0x7f186c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.566+0000 7f186b400640 1 --2- 172.21.15.7:0/2956406423 >> v2:172.21.15.87:3300/0 conn(0x7f186c0b5190 0x7f186c0d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.566+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3887950566 0 0) 0x7f186c0b55a0 con 0x7f186c06d580 2024-04-25T11:22:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.566+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1850003520 con 0x7f186c06d580 2024-04-25T11:22:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.566+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1204977250 0 0) 0x7f1850003520 con 0x7f186c06d580 2024-04-25T11:22:08.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f186c06f390 msgr2=0x7f186c06f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f186c06f390 0x7f186c06f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 >> v2:172.21.15.87:3300/0 conn(0x7f186c0b5190 msgr2=0x7f186c0d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 --2- 172.21.15.7:0/2956406423 >> v2:172.21.15.87:3300/0 conn(0x7f186c0b5190 0x7f186c0d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c15db00 con 0x7f186c06d580 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1858001670 con 0x7f186c06d580 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 >> v1:172.21.15.7:6789/0 conn(0x7f186c06d580 legacy=0x7f186c06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18500039b0 0x7f1850003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f186c06f390 0x7f1850004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1850004d40 0x7f1850005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f186c15d560 con 0x7f1850004d40 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f186c15d820 con 0x7f18500039b0 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f185000ba70 con 0x7f186c06f390 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.567+0000 7f186be00640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1850004d40 0x7f1850005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.568+0000 7f186b400640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18500039b0 0x7f1850003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.568+0000 7f186aa00640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f186c06f390 0x7f1850004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.568+0000 7f186be00640 1 -- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f186c06f390 msgr2=0x7f1850004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.568+0000 7f186be00640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f186c06f390 0x7f1850004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.568+0000 7f186be00640 1 -- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18500039b0 msgr2=0x7f1850003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.568+0000 7f186be00640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18500039b0 0x7f1850003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.568+0000 7f186be00640 1 -- 172.21.15.7:0/2956406423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1850003520 con 0x7f1850004d40 2024-04-25T11:22:08.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.568+0000 7f186b400640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18500039b0 0x7f1850003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:08.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.569+0000 7f186be00640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1850004d40 0x7f1850005150 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f1864009ac0 tx=0x7f1850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b824255bb999d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:08.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.569+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864036070 con 0x7f1850004d40 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.569+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1864031400 con 0x7f1850004d40 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.569+0000 7f186a000640 1 -- 172.21.15.7:0/2956406423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864009690 con 0x7f1850004d40 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.570+0000 7f1872200640 1 -- 172.21.15.7:0/2956406423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1850004d40 msgr2=0x7f1850005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.570+0000 7f1872200640 1 --2- 172.21.15.7:0/2956406423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1850004d40 0x7f1850005150 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f1864009ac0 tx=0x7f1850005830 comp rx=0 tx=0).stop 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.570+0000 7f186be00640 1 -- 172.21.15.7:0/2956406423 reap_dead start 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.570+0000 7f1872200640 1 -- 172.21.15.7:0/2956406423 shutdown_connections 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.570+0000 7f1872200640 1 -- 172.21.15.7:0/2956406423 >> 172.21.15.7:0/2956406423 conn(0x7f186c0732b0 msgr2=0x7f186c0736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.570+0000 7f1872200640 1 -- 172.21.15.7:0/2956406423 shutdown_connections 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.570+0000 7f1872200640 1 -- 172.21.15.7:0/2956406423 wait complete. 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.573+0000 7f1872200640 1 Processor -- start 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.573+0000 7f1872200640 1 -- start start 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.574+0000 7f1872200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1850004d40 0x7f186c14c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.574+0000 7f1872200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c09e1c0 0x7f186c09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.574+0000 7f1872200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186c06f7a0 0x7f186c06fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.574+0000 7f1872200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f186c0d1530 con 0x7f186c06f7a0 2024-04-25T11:22:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.574+0000 7f1872200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f186c15e0c0 con 0x7f186c09e1c0 2024-04-25T11:22:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.574+0000 7f1872200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f186c09f8e0 con 0x7f1850004d40 2024-04-25T11:22:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.575+0000 7f186be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186c06f7a0 0x7f186c06fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.575+0000 7f186be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186c06f7a0 0x7f186c06fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56196/0 (socket says 172.21.15.7:56196) 2024-04-25T11:22:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.575+0000 7f186be00640 1 -- 172.21.15.7:0/1124346695 learned_addr learned my addr 172.21.15.7:0/1124346695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.575+0000 7f186b400640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1850004d40 0x7f186c14c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.575+0000 7f186aa00640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c09e1c0 0x7f186c09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.575+0000 7f186be00640 1 -- 172.21.15.7:0/1124346695 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1850004d40 msgr2=0x7f186c14c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.575+0000 7f186be00640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1850004d40 0x7f186c14c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.575+0000 7f186be00640 1 -- 172.21.15.7:0/1124346695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c09e1c0 msgr2=0x7f186c09e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.575+0000 7f186be00640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c09e1c0 0x7f186c09e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.575+0000 7f186be00640 1 -- 172.21.15.7:0/1124346695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c0700f0 con 0x7f186c06f7a0 2024-04-25T11:22:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.576+0000 7f186be00640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186c06f7a0 0x7f186c06fbb0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f1864009be0 tx=0x7f1864004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.576+0000 7f1863400640 1 -- 172.21.15.7:0/1124346695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186403f020 con 0x7f186c06f7a0 2024-04-25T11:22:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.576+0000 7f1863400640 1 -- 172.21.15.7:0/1124346695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1864004030 con 0x7f186c06f7a0 2024-04-25T11:22:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.576+0000 7f1872200640 1 -- 172.21.15.7:0/1124346695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c0aead0 con 0x7f186c06f7a0 2024-04-25T11:22:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.576+0000 7f1863400640 1 -- 172.21.15.7:0/1124346695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864036040 con 0x7f186c06f7a0 2024-04-25T11:22:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.576+0000 7f1872200640 1 -- 172.21.15.7:0/1124346695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c0af090 con 0x7f186c06f7a0 2024-04-25T11:22:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.577+0000 7f1872200640 1 -- 172.21.15.7:0/1124346695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c068cf0 con 0x7f186c06f7a0 2024-04-25T11:22:08.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.579+0000 7f1863400640 1 -- 172.21.15.7:0/1124346695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1864042020 con 0x7f186c06f7a0 2024-04-25T11:22:08.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.580+0000 7f1863400640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f184c0744b0 0x7f184c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:08.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.580+0000 7f1863400640 1 -- 172.21.15.7:0/1124346695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7f18640b4f40 con 0x7f186c06f7a0 2024-04-25T11:22:08.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.580+0000 7f186b400640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f184c0744b0 0x7f184c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:08.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.581+0000 7f186b400640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f184c0744b0 0x7f184c0769c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f186c074160 tx=0x7f185c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:08.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.584+0000 7f1863400640 1 -- 172.21.15.7:0/1124346695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1864049070 con 0x7f186c06f7a0 2024-04-25T11:22:08.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.805+0000 7f1872200640 1 -- 172.21.15.7:0/1124346695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f186c074430 con 0x7f186c06f7a0 2024-04-25T11:22:08.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.806+0000 7f1863400640 1 -- 172.21.15.7:0/1124346695 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v144) ==== 80+0+0 (secure 0 0 0) 0x7f186c0700f0 con 0x7f186c06f7a0 2024-04-25T11:22:08.803 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:22:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.809+0000 7f1872200640 1 -- 172.21.15.7:0/1124346695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f184c0744b0 msgr2=0x7f184c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.809+0000 7f1872200640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f184c0744b0 0x7f184c0769c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f186c074160 tx=0x7f185c00b040 comp rx=0 tx=0).stop 2024-04-25T11:22:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.809+0000 7f1872200640 1 -- 172.21.15.7:0/1124346695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186c06f7a0 msgr2=0x7f186c06fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:08.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.809+0000 7f1872200640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186c06f7a0 0x7f186c06fbb0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f1864009be0 tx=0x7f1864004730 comp rx=0 tx=0).stop 2024-04-25T11:22:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.809+0000 7f1872200640 1 -- 172.21.15.7:0/1124346695 shutdown_connections 2024-04-25T11:22:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.809+0000 7f1872200640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f184c0744b0 0x7f184c0769c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.809+0000 7f1872200640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f186c06f7a0 0x7f186c06fbb0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.809+0000 7f1872200640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f186c09e1c0 0x7f186c09e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.809+0000 7f1872200640 1 --2- 172.21.15.7:0/1124346695 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1850004d40 0x7f186c14c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:08.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.809+0000 7f1872200640 1 -- 172.21.15.7:0/1124346695 >> 172.21.15.7:0/1124346695 conn(0x7f186c0732b0 msgr2=0x7f186c0b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.810+0000 7f1872200640 1 -- 172.21.15.7:0/1124346695 shutdown_connections 2024-04-25T11:22:08.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:08.810+0000 7f1872200640 1 -- 172.21.15.7:0/1124346695 wait complete. 2024-04-25T11:22:08.835 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:22:08.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:08.835 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T11:22:08.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:22:08.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:08.936 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:22:08.988 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:22:08.988 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:22:08.988 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:09.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:22:09.050 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:22:09.050 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:22:09.050 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:09.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:22:09.109 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.405+0000 7fb292400640 1 Processor -- start 2024-04-25T11:22:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.405+0000 7fb292400640 1 -- start start 2024-04-25T11:22:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.407+0000 7fb292400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c070a70 0x7fb28c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.407+0000 7fb292400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb28c0aedc0 0x7fb28c0b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.407+0000 7fb292400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb28c15e190 con 0x7fb28c0b0a10 2024-04-25T11:22:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.407+0000 7fb292400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb28c15d610 con 0x7fb28c0aedc0 2024-04-25T11:22:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.407+0000 7fb292400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb28c15d8f0 con 0x7fb28c070a70 2024-04-25T11:22:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.408+0000 7fb291a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb28c0b0a10 0x7fb28c1607b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37562/0 (socket says 172.21.15.7:37562) 2024-04-25T11:22:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.408+0000 7fb291a00640 1 -- 172.21.15.7:0/3663897361 learned_addr learned my addr 172.21.15.7:0/3663897361 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.408+0000 7fb291000640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c070a70 0x7fb28c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.408+0000 7fb283e00640 1 --2- 172.21.15.7:0/3663897361 >> v2:172.21.15.87:3300/0 conn(0x7fb28c0aedc0 0x7fb28c0b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.408+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1031592992 0 0) 0x7fb28c15e190 con 0x7fb28c0b0a10 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.408+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb270005730 con 0x7fb28c0b0a10 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3878540462 0 0) 0x7fb270005730 con 0x7fb28c0b0a10 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c070a70 msgr2=0x7fb28c0ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c070a70 0x7fb28c0ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 >> v2:172.21.15.87:3300/0 conn(0x7fb28c0aedc0 msgr2=0x7fb28c0b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 --2- 172.21.15.7:0/3663897361 >> v2:172.21.15.87:3300/0 conn(0x7fb28c0aedc0 0x7fb28c0b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28c15dbd0 con 0x7fb28c0b0a10 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fb2840019f0 con 0x7fb28c0b0a10 2024-04-25T11:22:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 >> v1:172.21.15.7:6789/0 conn(0x7fb28c0b0a10 legacy=0x7fb28c1607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb270005bc0 0x7fb270006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 0x7fb270006e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c070a70 0x7fb270007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c15d610 con 0x7fb270006a10 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c15d8f0 con 0x7fb270005bc0 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb27000dc00 con 0x7fb28c070a70 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.409+0000 7fb283e00640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 0x7fb270006e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.410+0000 7fb291000640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb270005bc0 0x7fb270006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.410+0000 7fb291a00640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c070a70 0x7fb270007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.410+0000 7fb283e00640 1 -- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c070a70 msgr2=0x7fb270007360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.410+0000 7fb283e00640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c070a70 0x7fb270007360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.410+0000 7fb283e00640 1 -- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb270005bc0 msgr2=0x7fb270006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:09.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.410+0000 7fb283e00640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb270005bc0 0x7fb270006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:09.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.410+0000 7fb283e00640 1 -- 172.21.15.7:0/3663897361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb270005730 con 0x7fb270006a10 2024-04-25T11:22:09.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.410+0000 7fb291a00640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c070a70 0x7fb270007360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.411+0000 7fb283e00640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 0x7fb270006e20 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fb28c071180 tx=0x7fb270007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0bae5073960374 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.411+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb274036070 con 0x7fb270006a10 2024-04-25T11:22:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.411+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb274031400 con 0x7fb270006a10 2024-04-25T11:22:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.411+0000 7fb28be00640 1 -- 172.21.15.7:0/3663897361 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb274009690 con 0x7fb270006a10 2024-04-25T11:22:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.412+0000 7fb292400640 1 -- 172.21.15.7:0/3663897361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 msgr2=0x7fb270006e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.412+0000 7fb292400640 1 --2- 172.21.15.7:0/3663897361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 0x7fb270006e20 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fb28c071180 tx=0x7fb270007a40 comp rx=0 tx=0).stop 2024-04-25T11:22:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.412+0000 7fb291a00640 1 -- 172.21.15.7:0/3663897361 reap_dead start 2024-04-25T11:22:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.412+0000 7fb292400640 1 -- 172.21.15.7:0/3663897361 shutdown_connections 2024-04-25T11:22:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.412+0000 7fb292400640 1 -- 172.21.15.7:0/3663897361 >> 172.21.15.7:0/3663897361 conn(0x7fb28c0b3820 msgr2=0x7fb28c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.412+0000 7fb292400640 1 -- 172.21.15.7:0/3663897361 shutdown_connections 2024-04-25T11:22:09.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.412+0000 7fb292400640 1 -- 172.21.15.7:0/3663897361 wait complete. 2024-04-25T11:22:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.415+0000 7fb292400640 1 Processor -- start 2024-04-25T11:22:09.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.415+0000 7fb292400640 1 -- start start 2024-04-25T11:22:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.416+0000 7fb292400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 0x7fb28c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.416+0000 7fb292400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28c0b5100 0x7fb28c146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.416+0000 7fb292400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c0ceab0 0x7fb28c147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.416+0000 7fb292400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c15d170 con 0x7fb270006a10 2024-04-25T11:22:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.416+0000 7fb292400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c0b0e20 con 0x7fb28c0b5100 2024-04-25T11:22:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.416+0000 7fb292400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb28c15d2f0 con 0x7fb28c0ceab0 2024-04-25T11:22:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.417+0000 7fb291000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 0x7fb28c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.417+0000 7fb291000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 0x7fb28c0b4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56218/0 (socket says 172.21.15.7:56218) 2024-04-25T11:22:09.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.417+0000 7fb291000640 1 -- 172.21.15.7:0/3610179864 learned_addr learned my addr 172.21.15.7:0/3610179864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:09.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.417+0000 7fb283e00640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28c0b5100 0x7fb28c146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:09.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.417+0000 7fb291a00640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c0ceab0 0x7fb28c147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:09.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.417+0000 7fb291000640 1 -- 172.21.15.7:0/3610179864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c0ceab0 msgr2=0x7fb28c147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:09.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.417+0000 7fb291000640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c0ceab0 0x7fb28c147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:09.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.417+0000 7fb291000640 1 -- 172.21.15.7:0/3610179864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28c0b5100 msgr2=0x7fb28c146c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:09.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.417+0000 7fb291000640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28c0b5100 0x7fb28c146c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:09.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.417+0000 7fb291000640 1 -- 172.21.15.7:0/3610179864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28c0d04e0 con 0x7fb270006a10 2024-04-25T11:22:09.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.418+0000 7fb291000640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 0x7fb28c0b4bc0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fb27c011630 tx=0x7fb27c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.418+0000 7fb289600640 1 -- 172.21.15.7:0/3610179864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00f8c0 con 0x7fb270006a10 2024-04-25T11:22:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.418+0000 7fb289600640 1 -- 172.21.15.7:0/3610179864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb27c00fa60 con 0x7fb270006a10 2024-04-25T11:22:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.418+0000 7fb292400640 1 -- 172.21.15.7:0/3610179864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28c0d0780 con 0x7fb270006a10 2024-04-25T11:22:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.418+0000 7fb289600640 1 -- 172.21.15.7:0/3610179864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00fe10 con 0x7fb270006a10 2024-04-25T11:22:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.418+0000 7fb292400640 1 -- 172.21.15.7:0/3610179864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28c156f40 con 0x7fb270006a10 2024-04-25T11:22:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.419+0000 7fb292400640 1 -- 172.21.15.7:0/3610179864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28c050740 con 0x7fb270006a10 2024-04-25T11:22:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.421+0000 7fb289600640 1 -- 172.21.15.7:0/3610179864 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fb27c015460 con 0x7fb270006a10 2024-04-25T11:22:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.422+0000 7fb289600640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2680744e0 0x7fb2680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.422+0000 7fb289600640 1 -- 172.21.15.7:0/3610179864 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7fb27c095a10 con 0x7fb270006a10 2024-04-25T11:22:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.422+0000 7fb283e00640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2680744e0 0x7fb2680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:09.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.423+0000 7fb283e00640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2680744e0 0x7fb2680769f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb274033000 tx=0x7fb274008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.426+0000 7fb289600640 1 -- 172.21.15.7:0/3610179864 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fb27c019030 con 0x7fb270006a10 2024-04-25T11:22:09.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.679+0000 7fb292400640 1 -- 172.21.15.7:0/3610179864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb28c06f640 con 0x7fb270006a10 2024-04-25T11:22:09.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.680+0000 7fb289600640 1 -- 172.21.15.7:0/3610179864 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5172 (secure 0 0 0) 0x7fb27c05ff40 con 0x7fb270006a10 2024-04-25T11:22:09.677 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:09.677 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":105,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/3503931395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3503931395},{"type":"v1","addr":"172.21.15.87:6834","nonce":3503931395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/63829697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":63829697},{"type":"v1","addr":"172.21.15.87:6835","nonce":63829697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/415877708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":415877708},{"type":"v1","addr":"172.21.15.7:6835","nonce":415877708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:21:37.365427+0000","modified":"2024-04-25T11:21:57.891068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8967},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8967":{"gid":8967,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":8,"addr":"172.21.15.7:6837/2966579341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2966579341},{"type":"v1","addr":"172.21.15.7:6837","nonce":2966579341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8967,"qdb_cluster":[8967]},"id":10}]} 2024-04-25T11:22:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 105 2024-04-25T11:22:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.683+0000 7fb292400640 1 -- 172.21.15.7:0/3610179864 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2680744e0 msgr2=0x7fb2680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.683+0000 7fb292400640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2680744e0 0x7fb2680769f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb274033000 tx=0x7fb274008030 comp rx=0 tx=0).stop 2024-04-25T11:22:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.683+0000 7fb292400640 1 -- 172.21.15.7:0/3610179864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 msgr2=0x7fb28c0b4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.683+0000 7fb292400640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 0x7fb28c0b4bc0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fb27c011630 tx=0x7fb27c011af0 comp rx=0 tx=0).stop 2024-04-25T11:22:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.683+0000 7fb292400640 1 -- 172.21.15.7:0/3610179864 shutdown_connections 2024-04-25T11:22:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.683+0000 7fb292400640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fb2680744e0 0x7fb2680769f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:09.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.683+0000 7fb292400640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb28c0ceab0 0x7fb28c147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.683+0000 7fb292400640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28c0b5100 0x7fb28c146c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.683+0000 7fb292400640 1 --2- 172.21.15.7:0/3610179864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb270006a10 0x7fb28c0b4bc0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.683+0000 7fb292400640 1 -- 172.21.15.7:0/3610179864 >> 172.21.15.7:0/3610179864 conn(0x7fb28c0b3820 msgr2=0x7fb28c0b6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.684+0000 7fb292400640 1 -- 172.21.15.7:0/3610179864 shutdown_connections 2024-04-25T11:22:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:09.684+0000 7fb292400640 1 -- 172.21.15.7:0/3610179864 wait complete. 2024-04-25T11:22:09.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.005+0000 7fc5fea00640 1 Processor -- start 2024-04-25T11:22:10.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.005+0000 7fc5fea00640 1 -- start start 2024-04-25T11:22:10.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.007+0000 7fc5fea00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5f806dc90 0x7fc5f80afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.007+0000 7fc5fea00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc5f806f300 0x7fc5f80b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.007+0000 7fc5fea00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5f80b4980 con 0x7fc5f8160a30 2024-04-25T11:22:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.007+0000 7fc5fea00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc5f80b4c60 con 0x7fc5f806dc90 2024-04-25T11:22:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.007+0000 7fc5fea00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc5f80b4f40 con 0x7fc5f806f300 2024-04-25T11:22:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.008+0000 7fc5f7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5f806dc90 0x7fc5f80afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:10.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.008+0000 7fc5fca00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc5f8160a30 0x7fc5f80ae960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46056/0 (socket says 172.21.15.7:46056) 2024-04-25T11:22:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.008+0000 7fc5fca00640 1 -- 172.21.15.7:0/3345699109 learned_addr learned my addr 172.21.15.7:0/3345699109 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.008+0000 7fc5efe00640 1 --2- 172.21.15.7:0/3345699109 >> v2:172.21.15.87:3300/0 conn(0x7fc5f806f300 0x7fc5f80b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.008+0000 7fc5f7e00640 1 -- 172.21.15.7:0/3345699109 >> v2:172.21.15.87:3300/0 conn(0x7fc5f806f300 msgr2=0x7fc5f80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.008+0000 7fc5f7e00640 1 --2- 172.21.15.7:0/3345699109 >> v2:172.21.15.87:3300/0 conn(0x7fc5f806f300 0x7fc5f80b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.008+0000 7fc5f7e00640 1 -- 172.21.15.7:0/3345699109 >> v1:172.21.15.7:6789/0 conn(0x7fc5f8160a30 legacy=0x7fc5f80ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.008+0000 7fc5f7e00640 1 -- 172.21.15.7:0/3345699109 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f80b5220 con 0x7fc5f806dc90 2024-04-25T11:22:10.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.009+0000 7fc5f7e00640 1 --2- 172.21.15.7:0/3345699109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f806dc90 0x7fc5f80afb80 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8071e80 tx=0x7fc5e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b096bf5030fba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:10.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.010+0000 7fc5f7400640 1 -- 172.21.15.7:0/3345699109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8037070 con 0x7fc5f806dc90 2024-04-25T11:22:10.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.010+0000 7fc5f7400640 1 -- 172.21.15.7:0/3345699109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e8032440 con 0x7fc5f806dc90 2024-04-25T11:22:10.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.010+0000 7fc5f7400640 1 -- 172.21.15.7:0/3345699109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8032810 con 0x7fc5f806dc90 2024-04-25T11:22:10.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.011+0000 7fc5fea00640 1 -- 172.21.15.7:0/3345699109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f806dc90 msgr2=0x7fc5f80afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.011+0000 7fc5fea00640 1 --2- 172.21.15.7:0/3345699109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f806dc90 0x7fc5f80afb80 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8071e80 tx=0x7fc5e802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.011+0000 7fc5fea00640 1 -- 172.21.15.7:0/3345699109 shutdown_connections 2024-04-25T11:22:10.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.011+0000 7fc5fea00640 1 --2- 172.21.15.7:0/3345699109 >> v2:172.21.15.87:3300/0 conn(0x7fc5f806f300 0x7fc5f80b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.011+0000 7fc5fea00640 1 --2- 172.21.15.7:0/3345699109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f806dc90 0x7fc5f80afb80 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.011+0000 7fc5fea00640 1 -- 172.21.15.7:0/3345699109 >> 172.21.15.7:0/3345699109 conn(0x7fc5f8074200 msgr2=0x7fc5f806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:10.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.011+0000 7fc5fea00640 1 -- 172.21.15.7:0/3345699109 shutdown_connections 2024-04-25T11:22:10.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.011+0000 7fc5fea00640 1 -- 172.21.15.7:0/3345699109 wait complete. 2024-04-25T11:22:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.013+0000 7fc5fea00640 1 Processor -- start 2024-04-25T11:22:10.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.013+0000 7fc5fea00640 1 -- start start 2024-04-25T11:22:10.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.015+0000 7fc5fea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5f806f300 0x7fc5f8154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.015+0000 7fc5fea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8160a30 0x7fc5f8157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.015+0000 7fc5fea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5f815b5b0 0x7fc5f8157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.015+0000 7fc5fea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f80b6a30 con 0x7fc5f8160a30 2024-04-25T11:22:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.015+0000 7fc5fea00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f815d220 con 0x7fc5f806f300 2024-04-25T11:22:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.015+0000 7fc5fea00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc5f80ce920 con 0x7fc5f815b5b0 2024-04-25T11:22:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.015+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8160a30 0x7fc5f8157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.015+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8160a30 0x7fc5f8157530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53218/0 (socket says 172.21.15.7:53218) 2024-04-25T11:22:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.015+0000 7fc5efe00640 1 -- 172.21.15.7:0/822952416 learned_addr learned my addr 172.21.15.7:0/822952416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.016+0000 7fc5f7e00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5f806f300 0x7fc5f8154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:10.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.016+0000 7fc5fca00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5f815b5b0 0x7fc5f8157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.016+0000 7fc5efe00640 1 -- 172.21.15.7:0/822952416 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5f815b5b0 msgr2=0x7fc5f8157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.016+0000 7fc5efe00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5f815b5b0 0x7fc5f8157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.016+0000 7fc5efe00640 1 -- 172.21.15.7:0/822952416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5f806f300 msgr2=0x7fc5f8154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.016+0000 7fc5efe00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5f806f300 0x7fc5f8154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.016+0000 7fc5efe00640 1 -- 172.21.15.7:0/822952416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f8154d70 con 0x7fc5f8160a30 2024-04-25T11:22:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.016+0000 7fc5efe00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8160a30 0x7fc5f8157530 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000d620 tx=0x7fc5e000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.017+0000 7fc5f4c00640 1 -- 172.21.15.7:0/822952416 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e000b620 con 0x7fc5f8160a30 2024-04-25T11:22:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.017+0000 7fc5f4c00640 1 -- 172.21.15.7:0/822952416 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5e0019070 con 0x7fc5f8160a30 2024-04-25T11:22:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.017+0000 7fc5f4c00640 1 -- 172.21.15.7:0/822952416 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e000ba00 con 0x7fc5f8160a30 2024-04-25T11:22:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.017+0000 7fc5fea00640 1 -- 172.21.15.7:0/822952416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f815bd20 con 0x7fc5f8160a30 2024-04-25T11:22:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.017+0000 7fc5fea00640 1 -- 172.21.15.7:0/822952416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f815c2e0 con 0x7fc5f8160a30 2024-04-25T11:22:10.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.018+0000 7fc5fea00640 1 -- 172.21.15.7:0/822952416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f815ba90 con 0x7fc5f8160a30 2024-04-25T11:22:10.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.019+0000 7fc5f4c00640 1 -- 172.21.15.7:0/822952416 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc5e000bba0 con 0x7fc5f8160a30 2024-04-25T11:22:10.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.020+0000 7fc5f4c00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc5d40744b0 0x7fc5d40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:10.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.020+0000 7fc5f4c00640 1 -- 172.21.15.7:0/822952416 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7fc5e00957a0 con 0x7fc5f8160a30 2024-04-25T11:22:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.021+0000 7fc5f7e00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc5d40744b0 0x7fc5d40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:10.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.021+0000 7fc5f7e00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc5d40744b0 0x7fc5d40769c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8034000 tx=0x7fc5e802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:10.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.027+0000 7fc5f4c00640 1 -- 172.21.15.7:0/822952416 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc5e005fd60 con 0x7fc5f8160a30 2024-04-25T11:22:10.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.232+0000 7fc5fea00640 1 -- 172.21.15.7:0/822952416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5f806a100 con 0x7fc5f8160a30 2024-04-25T11:22:10.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.236+0000 7fc5f4c00640 1 -- 172.21.15.7:0/822952416 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16954 (secure 0 0 0) 0x7fc5e005f660 con 0x7fc5f8160a30 2024-04-25T11:22:10.233 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:10.233 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":144,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:04.499877+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:21:35.562066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T11:21:36.342958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.34","mappings":[{"from":3,"to":0}]},{"pgid":"22.37","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 -- 172.21.15.7:0/822952416 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc5d40744b0 msgr2=0x7fc5d40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc5d40744b0 0x7fc5d40769c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8034000 tx=0x7fc5e802e000 comp rx=0 tx=0).stop 2024-04-25T11:22:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 -- 172.21.15.7:0/822952416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8160a30 msgr2=0x7fc5f8157530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8160a30 0x7fc5f8157530 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000d620 tx=0x7fc5e000dae0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 -- 172.21.15.7:0/822952416 shutdown_connections 2024-04-25T11:22:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc5d40744b0 0x7fc5d40769c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5f815b5b0 0x7fc5f8157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5f8160a30 0x7fc5f8157530 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 --2- 172.21.15.7:0/822952416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5f806f300 0x7fc5f8154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 -- 172.21.15.7:0/822952416 >> 172.21.15.7:0/822952416 conn(0x7fc5f8074200 msgr2=0x7fc5f8156ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 -- 172.21.15.7:0/822952416 shutdown_connections 2024-04-25T11:22:10.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.239+0000 7fc5fea00640 1 -- 172.21.15.7:0/822952416 wait complete. 2024-04-25T11:22:10.262 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:21:35.562066+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:21:36.342958+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:22:10.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:22:10.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.547+0000 7f43d9400640 1 Processor -- start 2024-04-25T11:22:10.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.548+0000 7f43d9400640 1 -- start start 2024-04-25T11:22:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.550+0000 7f43d9400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f43d40d0480 0x7f43d40d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.550+0000 7f43d9400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d40ce570 con 0x7f43d406eea0 2024-04-25T11:22:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.550+0000 7f43d9400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d40ce850 con 0x7f43d41583c0 2024-04-25T11:22:10.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.550+0000 7f43d9400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f43d415f190 con 0x7f43d40d0480 2024-04-25T11:22:10.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.551+0000 7f43d8a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f43d406eea0 0x7f43d40b0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46068/0 (socket says 172.21.15.7:46068) 2024-04-25T11:22:10.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.551+0000 7f43d8a00640 1 -- 172.21.15.7:0/1226891059 learned_addr learned my addr 172.21.15.7:0/1226891059 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:10.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.551+0000 7f43d3400640 1 --2- 172.21.15.7:0/1226891059 >> v2:172.21.15.87:3300/0 conn(0x7f43d40d0480 0x7f43d40d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.552+0000 7f43d3400640 1 -- 172.21.15.7:0/1226891059 >> v1:172.21.15.87:6789/0 conn(0x7f43d41583c0 legacy=0x7f43d40cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.552+0000 7f43d3400640 1 -- 172.21.15.7:0/1226891059 >> v1:172.21.15.7:6789/0 conn(0x7f43d406eea0 legacy=0x7f43d40b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.552+0000 7f43d3400640 1 -- 172.21.15.7:0/1226891059 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d40ceb30 con 0x7f43d40d0480 2024-04-25T11:22:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.554+0000 7f43d3400640 1 --2- 172.21.15.7:0/1226891059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43d40d0480 0x7f43d40d0890 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009d90 tx=0x7f43c402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8935a3b39c4884ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.557+0000 7f43d2a00640 1 -- 172.21.15.7:0/1226891059 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4007740 con 0x7f43d40d0480 2024-04-25T11:22:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.557+0000 7f43d2a00640 1 -- 172.21.15.7:0/1226891059 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c40078e0 con 0x7f43d40d0480 2024-04-25T11:22:10.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.557+0000 7f43d2a00640 1 -- 172.21.15.7:0/1226891059 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4007cb0 con 0x7f43d40d0480 2024-04-25T11:22:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.559+0000 7f43d9400640 1 -- 172.21.15.7:0/1226891059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43d40d0480 msgr2=0x7f43d40d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.559+0000 7f43d9400640 1 --2- 172.21.15.7:0/1226891059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43d40d0480 0x7f43d40d0890 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f43c4009d90 tx=0x7f43c402a660 comp rx=0 tx=0).stop 2024-04-25T11:22:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.559+0000 7f43d9400640 1 -- 172.21.15.7:0/1226891059 shutdown_connections 2024-04-25T11:22:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.559+0000 7f43d9400640 1 --2- 172.21.15.7:0/1226891059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43d40d0480 0x7f43d40d0890 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.559+0000 7f43d9400640 1 -- 172.21.15.7:0/1226891059 >> 172.21.15.7:0/1226891059 conn(0x7f43d40740f0 msgr2=0x7f43d406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.559+0000 7f43d9400640 1 -- 172.21.15.7:0/1226891059 shutdown_connections 2024-04-25T11:22:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.559+0000 7f43d9400640 1 -- 172.21.15.7:0/1226891059 wait complete. 2024-04-25T11:22:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.563+0000 7f43d9400640 1 Processor -- start 2024-04-25T11:22:10.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.563+0000 7f43d9400640 1 -- start start 2024-04-25T11:22:10.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.565+0000 7f43d9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d406eea0 0x7f43d40b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.565+0000 7f43d9400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43d40d0480 0x7f43d414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.565+0000 7f43d9400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d41583c0 0x7f43d415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.565+0000 7f43d9400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f43d40cfd20 con 0x7f43d406eea0 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.565+0000 7f43d9400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f43d40cf0f0 con 0x7f43d40d0480 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.565+0000 7f43d9400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f43d40b0e70 con 0x7f43d41583c0 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.566+0000 7f43d3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d406eea0 0x7f43d40b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.566+0000 7f43d3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d406eea0 0x7f43d40b6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53224/0 (socket says 172.21.15.7:53224) 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.566+0000 7f43d3e00640 1 -- 172.21.15.7:0/2910755194 learned_addr learned my addr 172.21.15.7:0/2910755194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.566+0000 7f43d3400640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43d40d0480 0x7f43d414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.566+0000 7f43d8a00640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d41583c0 0x7f43d415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.566+0000 7f43d3e00640 1 -- 172.21.15.7:0/2910755194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d41583c0 msgr2=0x7f43d415ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.566+0000 7f43d3e00640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d41583c0 0x7f43d415ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.566+0000 7f43d3e00640 1 -- 172.21.15.7:0/2910755194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43d40d0480 msgr2=0x7f43d414c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.566+0000 7f43d3e00640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43d40d0480 0x7f43d414c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.566+0000 7f43d3e00640 1 -- 172.21.15.7:0/2910755194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d41607f0 con 0x7f43d406eea0 2024-04-25T11:22:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.566+0000 7f43d8a00640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d41583c0 0x7f43d415ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.567+0000 7f43d3e00640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d406eea0 0x7f43d40b6350 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00c3f0 tx=0x7f43bc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.567+0000 7f43cbe00640 1 -- 172.21.15.7:0/2910755194 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc017070 con 0x7f43d406eea0 2024-04-25T11:22:10.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.567+0000 7f43cbe00640 1 -- 172.21.15.7:0/2910755194 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43bc0052d0 con 0x7f43d406eea0 2024-04-25T11:22:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.567+0000 7f43d9400640 1 -- 172.21.15.7:0/2910755194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d4160ad0 con 0x7f43d406eea0 2024-04-25T11:22:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.567+0000 7f43cbe00640 1 -- 172.21.15.7:0/2910755194 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc003830 con 0x7f43d406eea0 2024-04-25T11:22:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.567+0000 7f43d9400640 1 -- 172.21.15.7:0/2910755194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d4161090 con 0x7f43d406eea0 2024-04-25T11:22:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.569+0000 7f43d9400640 1 -- 172.21.15.7:0/2910755194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d4050740 con 0x7f43d406eea0 2024-04-25T11:22:10.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.569+0000 7f43cbe00640 1 -- 172.21.15.7:0/2910755194 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f43bc01a070 con 0x7f43d406eea0 2024-04-25T11:22:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.571+0000 7f43cbe00640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f43b00744b0 0x7f43b00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.571+0000 7f43cbe00640 1 -- 172.21.15.7:0/2910755194 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7f43bc094770 con 0x7f43d406eea0 2024-04-25T11:22:10.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.571+0000 7f43d3400640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f43b00744b0 0x7f43b00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:10.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.572+0000 7f43d3400640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f43b00744b0 0x7f43b00769c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f43c4034000 tx=0x7f43c4033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.576+0000 7f43cbe00640 1 -- 172.21.15.7:0/2910755194 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f43bc05ecb0 con 0x7f43d406eea0 2024-04-25T11:22:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.781+0000 7f43d9400640 1 -- 172.21.15.7:0/2910755194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f43d4070dd0 con 0x7f43d406eea0 2024-04-25T11:22:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.782+0000 7f43cbe00640 1 -- 172.21.15.7:0/2910755194 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) ==== 93+0+10 (secure 0 0 0) 0x7f43bc05e5b0 con 0x7f43d406eea0 2024-04-25T11:22:10.778 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.785+0000 7f43d9400640 1 -- 172.21.15.7:0/2910755194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f43b00744b0 msgr2=0x7f43b00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.785+0000 7f43d9400640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f43b00744b0 0x7f43b00769c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f43c4034000 tx=0x7f43c4033040 comp rx=0 tx=0).stop 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.785+0000 7f43d9400640 1 -- 172.21.15.7:0/2910755194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d406eea0 msgr2=0x7f43d40b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.785+0000 7f43d9400640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d406eea0 0x7f43d40b6350 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f43bc00c3f0 tx=0x7f43bc00c8b0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.785+0000 7f43d9400640 1 -- 172.21.15.7:0/2910755194 shutdown_connections 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.785+0000 7f43d9400640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f43b00744b0 0x7f43b00769c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.785+0000 7f43d9400640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d41583c0 0x7f43d415ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.785+0000 7f43d9400640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43d40d0480 0x7f43d414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.785+0000 7f43d9400640 1 --2- 172.21.15.7:0/2910755194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43d406eea0 0x7f43d40b6350 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.785+0000 7f43d9400640 1 -- 172.21.15.7:0/2910755194 >> 172.21.15.7:0/2910755194 conn(0x7f43d40740f0 msgr2=0x7f43d40b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.786+0000 7f43d9400640 1 -- 172.21.15.7:0/2910755194 shutdown_connections 2024-04-25T11:22:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:10.786+0000 7f43d9400640 1 -- 172.21.15.7:0/2910755194 wait complete. 2024-04-25T11:22:10.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:22:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.102+0000 7f20df200640 1 Processor -- start 2024-04-25T11:22:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.102+0000 7f20df200640 1 -- start start 2024-04-25T11:22:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.104+0000 7f20df200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20d815dfb0 0x7f20d80b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.104+0000 7f20df200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f20d80d0b40 0x7f20d806f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.104+0000 7f20df200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d815e3c0 con 0x7f20d80d0f50 2024-04-25T11:22:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.104+0000 7f20df200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f20d80b09b0 con 0x7f20d80d0b40 2024-04-25T11:22:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.104+0000 7f20df200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f20d80cf800 con 0x7f20d815dfb0 2024-04-25T11:22:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.105+0000 7f20dd200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f20d80d0f50 0x7f20d8161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46080/0 (socket says 172.21.15.7:46080) 2024-04-25T11:22:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.105+0000 7f20dd200640 1 -- 172.21.15.7:0/2576263055 learned_addr learned my addr 172.21.15.7:0/2576263055 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.105+0000 7f20d7e00640 1 --2- 172.21.15.7:0/2576263055 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20d815dfb0 0x7f20d80b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.105+0000 7f20d7400640 1 --2- 172.21.15.7:0/2576263055 >> v2:172.21.15.87:3300/0 conn(0x7f20d80d0b40 0x7f20d806f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.105+0000 7f20d6a00640 1 -- 172.21.15.7:0/2576263055 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2482940175 0 0) 0x7f20d815e3c0 con 0x7f20d80d0f50 2024-04-25T11:22:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.106+0000 7f20d6a00640 1 -- 172.21.15.7:0/2576263055 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20bc003520 con 0x7f20d80d0f50 2024-04-25T11:22:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.106+0000 7f20d7400640 1 -- 172.21.15.7:0/2576263055 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20d815dfb0 msgr2=0x7f20d80b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.106+0000 7f20d7400640 1 --2- 172.21.15.7:0/2576263055 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20d815dfb0 0x7f20d80b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.106+0000 7f20d7400640 1 -- 172.21.15.7:0/2576263055 >> v1:172.21.15.7:6789/0 conn(0x7f20d80d0f50 legacy=0x7f20d8161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.106+0000 7f20d6a00640 1 -- 172.21.15.7:0/2576263055 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2315893919 0 0) 0x7f20bc003520 con 0x7f20d80d0f50 2024-04-25T11:22:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.106+0000 7f20d7400640 1 -- 172.21.15.7:0/2576263055 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d80cfaa0 con 0x7f20d80d0b40 2024-04-25T11:22:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.106+0000 7f20d7e00640 1 --2- 172.21.15.7:0/2576263055 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20d815dfb0 0x7f20d80b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:22:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.107+0000 7f20d7400640 1 --2- 172.21.15.7:0/2576263055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d80d0b40 0x7f20d806f3a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009d90 tx=0x7f20c402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=535cc6bc92854418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.108+0000 7f20d6a00640 1 -- 172.21.15.7:0/2576263055 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0001670 con 0x7f20d80d0b40 2024-04-25T11:22:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.108+0000 7f20d6a00640 1 -- 172.21.15.7:0/2576263055 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c4007540 con 0x7f20d80d0b40 2024-04-25T11:22:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.108+0000 7f20d6a00640 1 -- 172.21.15.7:0/2576263055 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4007910 con 0x7f20d80d0b40 2024-04-25T11:22:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.109+0000 7f20df200640 1 -- 172.21.15.7:0/2576263055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d80d0b40 msgr2=0x7f20d806f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.109+0000 7f20df200640 1 --2- 172.21.15.7:0/2576263055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d80d0b40 0x7f20d806f3a0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009d90 tx=0x7f20c402a5f0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.109+0000 7f20df200640 1 -- 172.21.15.7:0/2576263055 shutdown_connections 2024-04-25T11:22:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.109+0000 7f20df200640 1 --2- 172.21.15.7:0/2576263055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d80d0b40 0x7f20d806f3a0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.109+0000 7f20df200640 1 --2- 172.21.15.7:0/2576263055 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20d815dfb0 0x7f20d80b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.109+0000 7f20df200640 1 -- 172.21.15.7:0/2576263055 >> 172.21.15.7:0/2576263055 conn(0x7f20d80742c0 msgr2=0x7f20d806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.110+0000 7f20df200640 1 -- 172.21.15.7:0/2576263055 shutdown_connections 2024-04-25T11:22:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.110+0000 7f20df200640 1 -- 172.21.15.7:0/2576263055 wait complete. 2024-04-25T11:22:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.112+0000 7f20df200640 1 Processor -- start 2024-04-25T11:22:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.112+0000 7f20df200640 1 -- start start 2024-04-25T11:22:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.114+0000 7f20df200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20d80d0b40 0x7f20d8160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.114+0000 7f20df200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d80d0f50 0x7f20d8144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.114+0000 7f20df200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d815dfb0 0x7f20d81f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.114+0000 7f20df200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f20d80d0340 con 0x7f20d80d0f50 2024-04-25T11:22:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.114+0000 7f20df200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f20d80d0060 con 0x7f20d815dfb0 2024-04-25T11:22:11.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.114+0000 7f20df200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f20d80d0620 con 0x7f20d80d0b40 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.114+0000 7f20d7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d80d0f50 0x7f20d8144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.114+0000 7f20d7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d80d0f50 0x7f20d8144260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53238/0 (socket says 172.21.15.7:53238) 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.114+0000 7f20d7400640 1 -- 172.21.15.7:0/3422768744 learned_addr learned my addr 172.21.15.7:0/3422768744 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.114+0000 7f20d7e00640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20d80d0b40 0x7f20d8160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20dd200640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d815dfb0 0x7f20d81f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20d7400640 1 -- 172.21.15.7:0/3422768744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20d80d0b40 msgr2=0x7f20d8160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20d7400640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20d80d0b40 0x7f20d8160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20d7400640 1 -- 172.21.15.7:0/3422768744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d815dfb0 msgr2=0x7f20d81f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20d7400640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d815dfb0 0x7f20d81f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20d7400640 1 -- 172.21.15.7:0/3422768744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d81613c0 con 0x7f20d80d0f50 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20dd200640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d815dfb0 0x7f20d81f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20d7400640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d80d0f50 0x7f20d8144260 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009eb0 tx=0x7f20c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20cfe00640 1 -- 172.21.15.7:0/3422768744 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c403f070 con 0x7f20d80d0f50 2024-04-25T11:22:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20cfe00640 1 -- 172.21.15.7:0/3422768744 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c4031bc0 con 0x7f20d80d0f50 2024-04-25T11:22:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20cfe00640 1 -- 172.21.15.7:0/3422768744 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4007e70 con 0x7f20d80d0f50 2024-04-25T11:22:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.115+0000 7f20df200640 1 -- 172.21.15.7:0/3422768744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d80b1340 con 0x7f20d80d0f50 2024-04-25T11:22:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.116+0000 7f20df200640 1 -- 172.21.15.7:0/3422768744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d80b1900 con 0x7f20d80d0f50 2024-04-25T11:22:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.117+0000 7f20df200640 1 -- 172.21.15.7:0/3422768744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d8161130 con 0x7f20d80d0f50 2024-04-25T11:22:11.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.118+0000 7f20cfe00640 1 -- 172.21.15.7:0/3422768744 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f20c403b410 con 0x7f20d80d0f50 2024-04-25T11:22:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.119+0000 7f20cfe00640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20b80744b0 0x7f20b80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.119+0000 7f20cfe00640 1 -- 172.21.15.7:0/3422768744 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7f20c40b6ae0 con 0x7f20d80d0f50 2024-04-25T11:22:11.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.119+0000 7f20d7e00640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20b80744b0 0x7f20b80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:11.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.120+0000 7f20d7e00640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20b80744b0 0x7f20b80769c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f20d8072df0 tx=0x7f20c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:11.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.124+0000 7f20cfe00640 1 -- 172.21.15.7:0/3422768744 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f20c4080a50 con 0x7f20d80d0f50 2024-04-25T11:22:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.327+0000 7f20df200640 1 -- 172.21.15.7:0/3422768744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f20d80ce460 con 0x7f20d80d0f50 2024-04-25T11:22:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.328+0000 7f20cfe00640 1 -- 172.21.15.7:0/3422768744 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) ==== 104+0+11 (secure 0 0 0) 0x7f20d80ce460 con 0x7f20d80d0f50 2024-04-25T11:22:11.324 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:22:11.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 -- 172.21.15.7:0/3422768744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20b80744b0 msgr2=0x7f20b80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20b80744b0 0x7f20b80769c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f20d8072df0 tx=0x7f20c8008040 comp rx=0 tx=0).stop 2024-04-25T11:22:11.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 -- 172.21.15.7:0/3422768744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d80d0f50 msgr2=0x7f20d8144260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d80d0f50 0x7f20d8144260 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009eb0 tx=0x7f20c4004a40 comp rx=0 tx=0).stop 2024-04-25T11:22:11.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 -- 172.21.15.7:0/3422768744 shutdown_connections 2024-04-25T11:22:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f20b80744b0 0x7f20b80769c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d815dfb0 0x7f20d81f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d80d0f50 0x7f20d8144260 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 --2- 172.21.15.7:0/3422768744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20d80d0b40 0x7f20d8160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 -- 172.21.15.7:0/3422768744 >> 172.21.15.7:0/3422768744 conn(0x7f20d80742c0 msgr2=0x7f20d8070740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 -- 172.21.15.7:0/3422768744 shutdown_connections 2024-04-25T11:22:11.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.331+0000 7f20df200640 1 -- 172.21.15.7:0/3422768744 wait complete. 2024-04-25T11:22:11.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:22:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.647+0000 7f3611800640 1 Processor -- start 2024-04-25T11:22:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.648+0000 7f3611800640 1 -- start start 2024-04-25T11:22:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.650+0000 7f3611800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f360c0b7b40 0x7f360c16ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.650+0000 7f3611800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f360c163cd0 con 0x7f360c0b4ab0 2024-04-25T11:22:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.650+0000 7f3611800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f360c163fb0 con 0x7f360c160080 2024-04-25T11:22:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.650+0000 7f3611800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f360c163a80 con 0x7f360c0b7b40 2024-04-25T11:22:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.650+0000 7f360be00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f360c0b4ab0 0x7f360c0d0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46088/0 (socket says 172.21.15.7:46088) 2024-04-25T11:22:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.650+0000 7f360be00640 1 -- 172.21.15.7:0/4289072715 learned_addr learned my addr 172.21.15.7:0/4289072715 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.650+0000 7f3610e00640 1 --2- 172.21.15.7:0/4289072715 >> v2:172.21.15.7:3300/0 conn(0x7f360c0b7b40 0x7f360c16ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.651+0000 7f360aa00640 1 -- 172.21.15.7:0/4289072715 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3635478285 0 0) 0x7f360c163cd0 con 0x7f360c0b4ab0 2024-04-25T11:22:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.651+0000 7f3610e00640 1 -- 172.21.15.7:0/4289072715 >> v1:172.21.15.87:6789/0 conn(0x7f360c160080 legacy=0x7f360c0b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.651+0000 7f3610e00640 1 -- 172.21.15.7:0/4289072715 >> v1:172.21.15.7:6789/0 conn(0x7f360c0b4ab0 legacy=0x7f360c0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.651+0000 7f3610e00640 1 -- 172.21.15.7:0/4289072715 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360c164290 con 0x7f360c0b7b40 2024-04-25T11:22:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.652+0000 7f3610e00640 1 --2- 172.21.15.7:0/4289072715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f360c0b7b40 0x7f360c16ae10 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f3604009d90 tx=0x7f360402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1755e1a5b391e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.652+0000 7f360aa00640 1 -- 172.21.15.7:0/4289072715 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604007740 con 0x7f360c0b7b40 2024-04-25T11:22:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.652+0000 7f360aa00640 1 -- 172.21.15.7:0/4289072715 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36040078e0 con 0x7f360c0b7b40 2024-04-25T11:22:11.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.652+0000 7f360aa00640 1 -- 172.21.15.7:0/4289072715 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604007cb0 con 0x7f360c0b7b40 2024-04-25T11:22:11.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.653+0000 7f3611800640 1 -- 172.21.15.7:0/4289072715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f360c0b7b40 msgr2=0x7f360c16ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.653+0000 7f3611800640 1 --2- 172.21.15.7:0/4289072715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f360c0b7b40 0x7f360c16ae10 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f3604009d90 tx=0x7f360402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.653+0000 7f3611800640 1 -- 172.21.15.7:0/4289072715 shutdown_connections 2024-04-25T11:22:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.653+0000 7f3611800640 1 --2- 172.21.15.7:0/4289072715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f360c0b7b40 0x7f360c16ae10 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.653+0000 7f3611800640 1 -- 172.21.15.7:0/4289072715 >> 172.21.15.7:0/4289072715 conn(0x7f360c0d2610 msgr2=0x7f360c0d2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.653+0000 7f3611800640 1 -- 172.21.15.7:0/4289072715 shutdown_connections 2024-04-25T11:22:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.653+0000 7f3611800640 1 -- 172.21.15.7:0/4289072715 wait complete. 2024-04-25T11:22:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.656+0000 7f3611800640 1 Processor -- start 2024-04-25T11:22:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.656+0000 7f3611800640 1 -- start start 2024-04-25T11:22:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.657+0000 7f3611800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f360c0b4ab0 0x7f360c151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.657+0000 7f3611800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c160080 0x7f360c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.657+0000 7f3611800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f360c1613f0 0x7f360c0b56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.657+0000 7f3611800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f360c0b9000 con 0x7f360c0b4ab0 2024-04-25T11:22:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.657+0000 7f3611800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f360c0b8e80 con 0x7f360c160080 2024-04-25T11:22:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.657+0000 7f3611800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f360c0b9180 con 0x7f360c1613f0 2024-04-25T11:22:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.658+0000 7f360be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f360c0b4ab0 0x7f360c151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.658+0000 7f360be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f360c0b4ab0 0x7f360c151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53264/0 (socket says 172.21.15.7:53264) 2024-04-25T11:22:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.658+0000 7f360be00640 1 -- 172.21.15.7:0/3057678239 learned_addr learned my addr 172.21.15.7:0/3057678239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.658+0000 7f360b400640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c160080 0x7f360c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.658+0000 7f3610e00640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f360c1613f0 0x7f360c0b56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.658+0000 7f360be00640 1 -- 172.21.15.7:0/3057678239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f360c1613f0 msgr2=0x7f360c0b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.658+0000 7f360be00640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f360c1613f0 0x7f360c0b56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.658+0000 7f360be00640 1 -- 172.21.15.7:0/3057678239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c160080 msgr2=0x7f360c1597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.658+0000 7f360be00640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c160080 0x7f360c1597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.658+0000 7f360be00640 1 -- 172.21.15.7:0/3057678239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360c160630 con 0x7f360c0b4ab0 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.658+0000 7f360b400640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c160080 0x7f360c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.659+0000 7f360be00640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f360c0b4ab0 0x7f360c151d80 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00c460 tx=0x7f35fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.659+0000 7f3603e00640 1 -- 172.21.15.7:0/3057678239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc018070 con 0x7f360c0b4ab0 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.659+0000 7f3603e00640 1 -- 172.21.15.7:0/3057678239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35fc007500 con 0x7f360c0b4ab0 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.659+0000 7f3603e00640 1 -- 172.21.15.7:0/3057678239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc004a20 con 0x7f360c0b4ab0 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.659+0000 7f3611800640 1 -- 172.21.15.7:0/3057678239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360c160910 con 0x7f360c0b4ab0 2024-04-25T11:22:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.659+0000 7f3611800640 1 -- 172.21.15.7:0/3057678239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f360c159d30 con 0x7f360c0b4ab0 2024-04-25T11:22:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.660+0000 7f3611800640 1 -- 172.21.15.7:0/3057678239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f360c0bb700 con 0x7f360c0b4ab0 2024-04-25T11:22:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.662+0000 7f3603e00640 1 -- 172.21.15.7:0/3057678239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f35fc004bc0 con 0x7f360c0b4ab0 2024-04-25T11:22:11.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.662+0000 7f3603e00640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35ec0744b0 0x7f35ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.663+0000 7f3603e00640 1 -- 172.21.15.7:0/3057678239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7f35fc094800 con 0x7f360c0b4ab0 2024-04-25T11:22:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.663+0000 7f360b400640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35ec0744b0 0x7f35ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.664+0000 7f360b400640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35ec0744b0 0x7f35ec0769c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f360c0ba8a0 tx=0x7f35f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.668+0000 7f3603e00640 1 -- 172.21.15.7:0/3057678239 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f360c0bb700 con 0x7f360c0b4ab0 2024-04-25T11:22:11.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.876+0000 7f3611800640 1 -- 172.21.15.7:0/3057678239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f360c050740 con 0x7f360c0b4ab0 2024-04-25T11:22:11.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.877+0000 7f3603e00640 1 -- 172.21.15.7:0/3057678239 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v144) ==== 100+0+11 (secure 0 0 0) 0x7f35fc05ed30 con 0x7f360c0b4ab0 2024-04-25T11:22:11.874 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.880+0000 7f3611800640 1 -- 172.21.15.7:0/3057678239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35ec0744b0 msgr2=0x7f35ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.880+0000 7f3611800640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35ec0744b0 0x7f35ec0769c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f360c0ba8a0 tx=0x7f35f8008040 comp rx=0 tx=0).stop 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.880+0000 7f3611800640 1 -- 172.21.15.7:0/3057678239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f360c0b4ab0 msgr2=0x7f360c151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.880+0000 7f3611800640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f360c0b4ab0 0x7f360c151d80 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00c460 tx=0x7f35fc00c920 comp rx=0 tx=0).stop 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.880+0000 7f3611800640 1 -- 172.21.15.7:0/3057678239 shutdown_connections 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.880+0000 7f3611800640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f35ec0744b0 0x7f35ec0769c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.880+0000 7f3611800640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f360c1613f0 0x7f360c0b56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.880+0000 7f3611800640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f360c160080 0x7f360c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.880+0000 7f3611800640 1 --2- 172.21.15.7:0/3057678239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f360c0b4ab0 0x7f360c151d80 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.880+0000 7f3611800640 1 -- 172.21.15.7:0/3057678239 >> 172.21.15.7:0/3057678239 conn(0x7f360c0d2610 msgr2=0x7f360c0baaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.880+0000 7f3611800640 1 -- 172.21.15.7:0/3057678239 shutdown_connections 2024-04-25T11:22:11.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:11.881+0000 7f3611800640 1 -- 172.21.15.7:0/3057678239 wait complete. 2024-04-25T11:22:11.905 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.194+0000 7f9666600640 1 Processor -- start 2024-04-25T11:22:12.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.194+0000 7f9666600640 1 -- start start 2024-04-25T11:22:12.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.197+0000 7f9666600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f966009ceb0 0x7f96600b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.197+0000 7f9666600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f966009bd30 0x7f966009c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.197+0000 7f9666600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96600b3c10 con 0x7f966006d9b0 2024-04-25T11:22:12.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.197+0000 7f9666600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f96600b4540 con 0x7f966009bd30 2024-04-25T11:22:12.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.197+0000 7f9666600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f96600b3ef0 con 0x7f966009ceb0 2024-04-25T11:22:12.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.197+0000 7f965fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f966006d9b0 0x7f966006ddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46090/0 (socket says 172.21.15.7:46090) 2024-04-25T11:22:12.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.197+0000 7f9656a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f966009bd30 0x7f966009c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.197+0000 7f965fe00640 1 -- 172.21.15.7:0/2453335983 learned_addr learned my addr 172.21.15.7:0/2453335983 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:12.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.198+0000 7f965f400640 1 --2- 172.21.15.7:0/2453335983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f966009ceb0 0x7f96600b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.198+0000 7f9656a00640 1 -- 172.21.15.7:0/2453335983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f966009ceb0 msgr2=0x7f96600b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:12.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.198+0000 7f9656a00640 1 --2- 172.21.15.7:0/2453335983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f966009ceb0 0x7f96600b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.198+0000 7f9656a00640 1 -- 172.21.15.7:0/2453335983 >> v1:172.21.15.7:6789/0 conn(0x7f966006d9b0 legacy=0x7f966006ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.198+0000 7f9656a00640 1 -- 172.21.15.7:0/2453335983 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96600b41d0 con 0x7f966009bd30 2024-04-25T11:22:12.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.198+0000 7f965ea00640 1 -- 172.21.15.7:0/2453335983 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199183548 0 0) 0x7f96600b3c10 con 0x7f966006d9b0 2024-04-25T11:22:12.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.199+0000 7f9656a00640 1 --2- 172.21.15.7:0/2453335983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966009bd30 0x7f966009c140 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f96480097a0 tx=0x7f964802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f62fd34da30043a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:12.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.200+0000 7f965ea00640 1 -- 172.21.15.7:0/2453335983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648037070 con 0x7f966009bd30 2024-04-25T11:22:12.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.200+0000 7f965ea00640 1 -- 172.21.15.7:0/2453335983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9648032440 con 0x7f966009bd30 2024-04-25T11:22:12.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.200+0000 7f965ea00640 1 -- 172.21.15.7:0/2453335983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648032810 con 0x7f966009bd30 2024-04-25T11:22:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.200+0000 7f9666600640 1 -- 172.21.15.7:0/2453335983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966009bd30 msgr2=0x7f966009c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.200+0000 7f9666600640 1 --2- 172.21.15.7:0/2453335983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966009bd30 0x7f966009c140 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f96480097a0 tx=0x7f964802a180 comp rx=0 tx=0).stop 2024-04-25T11:22:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.200+0000 7f9666600640 1 -- 172.21.15.7:0/2453335983 shutdown_connections 2024-04-25T11:22:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.200+0000 7f9666600640 1 --2- 172.21.15.7:0/2453335983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966009bd30 0x7f966009c140 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.200+0000 7f9666600640 1 --2- 172.21.15.7:0/2453335983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f966009ceb0 0x7f96600b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.200+0000 7f9666600640 1 -- 172.21.15.7:0/2453335983 >> 172.21.15.7:0/2453335983 conn(0x7f96600b5470 msgr2=0x7f96600b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.201+0000 7f9666600640 1 -- 172.21.15.7:0/2453335983 shutdown_connections 2024-04-25T11:22:12.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.201+0000 7f9666600640 1 -- 172.21.15.7:0/2453335983 wait complete. 2024-04-25T11:22:12.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.203+0000 7f9666600640 1 Processor -- start 2024-04-25T11:22:12.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.203+0000 7f9666600640 1 -- start start 2024-04-25T11:22:12.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.205+0000 7f9666600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f966006d9b0 0x7f96601f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.205+0000 7f9666600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f966009bd30 0x7f966009eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.205+0000 7f9666600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966009ceb0 0x7f96601f7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.205+0000 7f9666600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f966006e650 con 0x7f966009ceb0 2024-04-25T11:22:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.205+0000 7f9666600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f966006e4d0 con 0x7f966009bd30 2024-04-25T11:22:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.205+0000 7f9666600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f966006e7d0 con 0x7f966006d9b0 2024-04-25T11:22:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.205+0000 7f965fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966009ceb0 0x7f96601f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.205+0000 7f965fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966009ceb0 0x7f96601f7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53288/0 (socket says 172.21.15.7:53288) 2024-04-25T11:22:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.205+0000 7f965fe00640 1 -- 172.21.15.7:0/2955308838 learned_addr learned my addr 172.21.15.7:0/2955308838 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.206+0000 7f965f400640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f966006d9b0 0x7f96601f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.206+0000 7f9656a00640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f966009bd30 0x7f966009eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.206+0000 7f965fe00640 1 -- 172.21.15.7:0/2955308838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f966006d9b0 msgr2=0x7f96601f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.206+0000 7f965fe00640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f966006d9b0 0x7f96601f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.206+0000 7f965fe00640 1 -- 172.21.15.7:0/2955308838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f966009bd30 msgr2=0x7f966009eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.206+0000 7f965fe00640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f966009bd30 0x7f966009eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.206+0000 7f965fe00640 1 -- 172.21.15.7:0/2955308838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96601f7100 con 0x7f966009ceb0 2024-04-25T11:22:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.206+0000 7f965fe00640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966009ceb0 0x7f96601f7320 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f965800c780 tx=0x7f965800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.207+0000 7f9657e00640 1 -- 172.21.15.7:0/2955308838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658018070 con 0x7f966009ceb0 2024-04-25T11:22:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.207+0000 7f9657e00640 1 -- 172.21.15.7:0/2955308838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965800ce30 con 0x7f966009ceb0 2024-04-25T11:22:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.207+0000 7f9666600640 1 -- 172.21.15.7:0/2955308838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966009de20 con 0x7f966009ceb0 2024-04-25T11:22:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.207+0000 7f9657e00640 1 -- 172.21.15.7:0/2955308838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965800a860 con 0x7f966009ceb0 2024-04-25T11:22:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.207+0000 7f9666600640 1 -- 172.21.15.7:0/2955308838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f966009e3e0 con 0x7f966009ceb0 2024-04-25T11:22:12.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.208+0000 7f9666600640 1 -- 172.21.15.7:0/2955308838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96600b5d90 con 0x7f966009ceb0 2024-04-25T11:22:12.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.209+0000 7f9657e00640 1 -- 172.21.15.7:0/2955308838 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f965801b070 con 0x7f966009ceb0 2024-04-25T11:22:12.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.211+0000 7f9657e00640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f963c0744b0 0x7f963c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.211+0000 7f9657e00640 1 -- 172.21.15.7:0/2955308838 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7f9658094940 con 0x7f966009ceb0 2024-04-25T11:22:12.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.211+0000 7f965f400640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f963c0744b0 0x7f963c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.212+0000 7f965f400640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f963c0744b0 0x7f963c0769c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9650009a30 tx=0x7f9650008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:12.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.216+0000 7f9657e00640 1 -- 172.21.15.7:0/2955308838 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f96600b5d90 con 0x7f966009ceb0 2024-04-25T11:22:12.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.459+0000 7f9666600640 1 -- 172.21.15.7:0/2955308838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f966009c780 con 0x7f966009ceb0 2024-04-25T11:22:12.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.461+0000 7f9657e00640 1 -- 172.21.15.7:0/2955308838 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5173 (secure 0 0 0) 0x7f965805ef00 con 0x7f966009ceb0 2024-04-25T11:22:12.457 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:12.458 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/3503931395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3503931395},{"type":"v1","addr":"172.21.15.87:6834","nonce":3503931395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/63829697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":63829697},{"type":"v1","addr":"172.21.15.87:6835","nonce":63829697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/415877708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":415877708},{"type":"v1","addr":"172.21.15.7:6835","nonce":415877708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:21:37.365427+0000","modified":"2024-04-25T11:22:09.892126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8967},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8967":{"gid":8967,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.7:6837/2966579341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2966579341},{"type":"v1","addr":"172.21.15.7:6837","nonce":2966579341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8967,"qdb_cluster":[8967]},"id":10}]} 2024-04-25T11:22:12.458 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 106 2024-04-25T11:22:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 -- 172.21.15.7:0/2955308838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f963c0744b0 msgr2=0x7f963c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f963c0744b0 0x7f963c0769c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9650009a30 tx=0x7f9650008040 comp rx=0 tx=0).stop 2024-04-25T11:22:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 -- 172.21.15.7:0/2955308838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966009ceb0 msgr2=0x7f96601f7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966009ceb0 0x7f96601f7320 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f965800c780 tx=0x7f965800cc40 comp rx=0 tx=0).stop 2024-04-25T11:22:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 -- 172.21.15.7:0/2955308838 shutdown_connections 2024-04-25T11:22:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f963c0744b0 0x7f963c0769c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f966009ceb0 0x7f96601f7320 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f966009bd30 0x7f966009eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 --2- 172.21.15.7:0/2955308838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f966006d9b0 0x7f96601f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 -- 172.21.15.7:0/2955308838 >> 172.21.15.7:0/2955308838 conn(0x7f96600b5470 msgr2=0x7f966006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 -- 172.21.15.7:0/2955308838 shutdown_connections 2024-04-25T11:22:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.464+0000 7f9666600640 1 -- 172.21.15.7:0/2955308838 wait complete. 2024-04-25T11:22:12.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.763+0000 7f16bb200640 1 Processor -- start 2024-04-25T11:22:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.763+0000 7f16bb200640 1 -- start start 2024-04-25T11:22:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.765+0000 7f16bb200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16bc0b5220 0x7f16bc15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.765+0000 7f16bb200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16bc0af420 0x7f16bc0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.765+0000 7f16bb200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16bc0cebb0 con 0x7f16bc0aed40 2024-04-25T11:22:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.765+0000 7f16bb200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f16bc15e1f0 con 0x7f16bc0b5220 2024-04-25T11:22:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.765+0000 7f16bb200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f16bc0afd70 con 0x7f16bc0af420 2024-04-25T11:22:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.766+0000 7f16b9e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f16bc0aed40 0x7f16bc0ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46094/0 (socket says 172.21.15.7:46094) 2024-04-25T11:22:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.766+0000 7f16b9e00640 1 -- 172.21.15.7:0/4034597978 learned_addr learned my addr 172.21.15.7:0/4034597978 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.766+0000 7f16b9400640 1 --2- 172.21.15.7:0/4034597978 >> v2:172.21.15.7:3300/0 conn(0x7f16bc0b5220 0x7f16bc15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.766+0000 7f16ba800640 1 --2- 172.21.15.7:0/4034597978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16bc0af420 0x7f16bc0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.766+0000 7f16b8a00640 1 -- 172.21.15.7:0/4034597978 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2517941057 0 0) 0x7f16ac001110 con 0x7f16bc0aed40 2024-04-25T11:22:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.766+0000 7f16b8a00640 1 -- 172.21.15.7:0/4034597978 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16a0003520 con 0x7f16bc0aed40 2024-04-25T11:22:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.766+0000 7f16b9400640 1 -- 172.21.15.7:0/4034597978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16bc0af420 msgr2=0x7f16bc0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.766+0000 7f16b9400640 1 --2- 172.21.15.7:0/4034597978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16bc0af420 0x7f16bc0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.766+0000 7f16b9400640 1 -- 172.21.15.7:0/4034597978 >> v1:172.21.15.7:6789/0 conn(0x7f16bc0aed40 legacy=0x7f16bc0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.766+0000 7f16b9400640 1 -- 172.21.15.7:0/4034597978 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc0b64b0 con 0x7f16bc0b5220 2024-04-25T11:22:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.767+0000 7f16b9400640 1 --2- 172.21.15.7:0/4034597978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc0b5220 0x7f16bc15dcb0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f16a80097a0 tx=0x7f16a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457049d3883ddf76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.768+0000 7f16b8a00640 1 -- 172.21.15.7:0/4034597978 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8037070 con 0x7f16bc0b5220 2024-04-25T11:22:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.768+0000 7f16b8a00640 1 -- 172.21.15.7:0/4034597978 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a8032440 con 0x7f16bc0b5220 2024-04-25T11:22:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.768+0000 7f16b8a00640 1 -- 172.21.15.7:0/4034597978 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80327f0 con 0x7f16bc0b5220 2024-04-25T11:22:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.768+0000 7f16bb200640 1 -- 172.21.15.7:0/4034597978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc0b5220 msgr2=0x7f16bc15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.768+0000 7f16bb200640 1 --2- 172.21.15.7:0/4034597978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc0b5220 0x7f16bc15dcb0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f16a80097a0 tx=0x7f16a802a180 comp rx=0 tx=0).stop 2024-04-25T11:22:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.769+0000 7f16bb200640 1 -- 172.21.15.7:0/4034597978 shutdown_connections 2024-04-25T11:22:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.769+0000 7f16bb200640 1 --2- 172.21.15.7:0/4034597978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16bc0af420 0x7f16bc0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.769+0000 7f16bb200640 1 --2- 172.21.15.7:0/4034597978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc0b5220 0x7f16bc15dcb0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.769+0000 7f16bb200640 1 -- 172.21.15.7:0/4034597978 >> 172.21.15.7:0/4034597978 conn(0x7f16bc0b3820 msgr2=0x7f16bc0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.769+0000 7f16bb200640 1 -- 172.21.15.7:0/4034597978 shutdown_connections 2024-04-25T11:22:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.769+0000 7f16bb200640 1 -- 172.21.15.7:0/4034597978 wait complete. 2024-04-25T11:22:12.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.771+0000 7f16bb200640 1 Processor -- start 2024-04-25T11:22:12.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.771+0000 7f16bb200640 1 -- start start 2024-04-25T11:22:12.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.773+0000 7f16bb200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16bc0aed40 0x7f16bc157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.773+0000 7f16bb200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc0af420 0x7f16bc15a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.773+0000 7f16bb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc0b5220 0x7f16bc1f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.773+0000 7f16bb200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc06cbd0 con 0x7f16bc0b5220 2024-04-25T11:22:12.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.773+0000 7f16bb200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc06c8f0 con 0x7f16bc0af420 2024-04-25T11:22:12.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.773+0000 7f16bb200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f16bc06ceb0 con 0x7f16bc0aed40 2024-04-25T11:22:12.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.773+0000 7f16ba800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc0b5220 0x7f16bc1f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.773+0000 7f16ba800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc0b5220 0x7f16bc1f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53308/0 (socket says 172.21.15.7:53308) 2024-04-25T11:22:12.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.773+0000 7f16ba800640 1 -- 172.21.15.7:0/3875955292 learned_addr learned my addr 172.21.15.7:0/3875955292 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.773+0000 7f16b9e00640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16bc0aed40 0x7f16bc157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16b9400640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc0af420 0x7f16bc15a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16ba800640 1 -- 172.21.15.7:0/3875955292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16bc0aed40 msgr2=0x7f16bc157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16ba800640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16bc0aed40 0x7f16bc157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16ba800640 1 -- 172.21.15.7:0/3875955292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc0af420 msgr2=0x7f16bc15a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16ba800640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc0af420 0x7f16bc15a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16ba800640 1 -- 172.21.15.7:0/3875955292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc1587c0 con 0x7f16bc0b5220 2024-04-25T11:22:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16b9e00640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16bc0aed40 0x7f16bc157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16ba800640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc0b5220 0x7f16bc1f57b0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f16b4007e60 tx=0x7f16b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16b2000640 1 -- 172.21.15.7:0/3875955292 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4014420 con 0x7f16bc0b5220 2024-04-25T11:22:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16b2000640 1 -- 172.21.15.7:0/3875955292 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b4019070 con 0x7f16bc0b5220 2024-04-25T11:22:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16bb200640 1 -- 172.21.15.7:0/3875955292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc158c30 con 0x7f16bc0b5220 2024-04-25T11:22:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16b2000640 1 -- 172.21.15.7:0/3875955292 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4014820 con 0x7f16bc0b5220 2024-04-25T11:22:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.774+0000 7f16bb200640 1 -- 172.21.15.7:0/3875955292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc15f4a0 con 0x7f16bc0b5220 2024-04-25T11:22:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.775+0000 7f16bb200640 1 -- 172.21.15.7:0/3875955292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16bc050740 con 0x7f16bc0b5220 2024-04-25T11:22:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.777+0000 7f16b2000640 1 -- 172.21.15.7:0/3875955292 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f16b40149c0 con 0x7f16bc0b5220 2024-04-25T11:22:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.778+0000 7f16b2000640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f169c0744b0 0x7f169c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.778+0000 7f16b2000640 1 -- 172.21.15.7:0/3875955292 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7f16b4095d80 con 0x7f16bc0b5220 2024-04-25T11:22:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.778+0000 7f16b9e00640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f169c0744b0 0x7f169c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.779+0000 7f16b9e00640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f169c0744b0 0x7f169c0769c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0012b0 tx=0x7f16ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.783+0000 7f16b2000640 1 -- 172.21.15.7:0/3875955292 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f16b4060230 con 0x7f16bc0b5220 2024-04-25T11:22:12.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.979+0000 7f16bb200640 1 -- 172.21.15.7:0/3875955292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16bc1589c0 con 0x7f16bc0b5220 2024-04-25T11:22:12.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.981+0000 7f16b2000640 1 -- 172.21.15.7:0/3875955292 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16954 (secure 0 0 0) 0x7f16b405fb30 con 0x7f16bc0b5220 2024-04-25T11:22:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:12.978 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":144,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:04.499877+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:21:35.562066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T11:21:36.342958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.34","mappings":[{"from":3,"to":0}]},{"pgid":"22.37","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:12.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.984+0000 7f16bb200640 1 -- 172.21.15.7:0/3875955292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f169c0744b0 msgr2=0x7f169c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.984+0000 7f16bb200640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f169c0744b0 0x7f169c0769c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0012b0 tx=0x7f16ac009040 comp rx=0 tx=0).stop 2024-04-25T11:22:12.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.984+0000 7f16bb200640 1 -- 172.21.15.7:0/3875955292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc0b5220 msgr2=0x7f16bc1f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:12.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.984+0000 7f16bb200640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc0b5220 0x7f16bc1f57b0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f16b4007e60 tx=0x7f16b400e4b0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.984+0000 7f16bb200640 1 -- 172.21.15.7:0/3875955292 shutdown_connections 2024-04-25T11:22:12.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.984+0000 7f16bb200640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f169c0744b0 0x7f169c0769c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.984+0000 7f16bb200640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16bc0b5220 0x7f16bc1f57b0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.984+0000 7f16bb200640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16bc0af420 0x7f16bc15a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.984+0000 7f16bb200640 1 --2- 172.21.15.7:0/3875955292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16bc0aed40 0x7f16bc157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:12.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.984+0000 7f16bb200640 1 -- 172.21.15.7:0/3875955292 >> 172.21.15.7:0/3875955292 conn(0x7f16bc0b3820 msgr2=0x7f16bc0d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:12.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.985+0000 7f16bb200640 1 -- 172.21.15.7:0/3875955292 shutdown_connections 2024-04-25T11:22:12.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:12.985+0000 7f16bb200640 1 -- 172.21.15.7:0/3875955292 wait complete. 2024-04-25T11:22:13.008 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:22:13.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.291+0000 7fefde200640 1 Processor -- start 2024-04-25T11:22:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.291+0000 7fefde200640 1 -- start start 2024-04-25T11:22:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.293+0000 7fefde200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fefd80aed40 0x7fefd815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.294+0000 7fefde200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd80cebb0 con 0x7fefd80af420 2024-04-25T11:22:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.294+0000 7fefde200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefd80b64b0 con 0x7fefd80b5220 2024-04-25T11:22:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.294+0000 7fefde200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fefd815e1f0 con 0x7fefd80aed40 2024-04-25T11:22:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.294+0000 7fefdd800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fefd80af420 0x7fefd80af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46102/0 (socket says 172.21.15.7:46102) 2024-04-25T11:22:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.294+0000 7fefdd800640 1 -- 172.21.15.7:0/1960338161 learned_addr learned my addr 172.21.15.7:0/1960338161 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.294+0000 7fefdce00640 1 --2- 172.21.15.7:0/1960338161 >> v2:172.21.15.7:3300/0 conn(0x7fefd80aed40 0x7fefd815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:13.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.295+0000 7fefd7400640 1 -- 172.21.15.7:0/1960338161 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3516636517 0 0) 0x7fefd0001490 con 0x7fefd80af420 2024-04-25T11:22:13.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.295+0000 7fefd7400640 1 -- 172.21.15.7:0/1960338161 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefc0003520 con 0x7fefd80af420 2024-04-25T11:22:13.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.295+0000 7fefdce00640 1 -- 172.21.15.7:0/1960338161 >> v1:172.21.15.87:6789/0 conn(0x7fefd80b5220 legacy=0x7fefd80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:13.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.295+0000 7fefdce00640 1 -- 172.21.15.7:0/1960338161 >> v1:172.21.15.7:6789/0 conn(0x7fefd80af420 legacy=0x7fefd80af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:13.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.295+0000 7fefdce00640 1 -- 172.21.15.7:0/1960338161 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd806c610 con 0x7fefd80aed40 2024-04-25T11:22:13.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.296+0000 7fefdce00640 1 --2- 172.21.15.7:0/1960338161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefd80aed40 0x7fefd815dcb0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0097a0 tx=0x7fefcc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca01563b138b9ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:13.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.296+0000 7fefd7400640 1 -- 172.21.15.7:0/1960338161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc037070 con 0x7fefd80aed40 2024-04-25T11:22:13.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.296+0000 7fefd7400640 1 -- 172.21.15.7:0/1960338161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefcc032440 con 0x7fefd80aed40 2024-04-25T11:22:13.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.296+0000 7fefd7400640 1 -- 172.21.15.7:0/1960338161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefcc0327f0 con 0x7fefd80aed40 2024-04-25T11:22:13.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.297+0000 7fefde200640 1 -- 172.21.15.7:0/1960338161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefd80aed40 msgr2=0x7fefd815dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.297+0000 7fefde200640 1 --2- 172.21.15.7:0/1960338161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefd80aed40 0x7fefd815dcb0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0097a0 tx=0x7fefcc02a230 comp rx=0 tx=0).stop 2024-04-25T11:22:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.297+0000 7fefde200640 1 -- 172.21.15.7:0/1960338161 shutdown_connections 2024-04-25T11:22:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.297+0000 7fefde200640 1 --2- 172.21.15.7:0/1960338161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefd80aed40 0x7fefd815dcb0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.297+0000 7fefde200640 1 -- 172.21.15.7:0/1960338161 >> 172.21.15.7:0/1960338161 conn(0x7fefd80b3820 msgr2=0x7fefd80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.297+0000 7fefde200640 1 -- 172.21.15.7:0/1960338161 shutdown_connections 2024-04-25T11:22:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.297+0000 7fefde200640 1 -- 172.21.15.7:0/1960338161 wait complete. 2024-04-25T11:22:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.300+0000 7fefde200640 1 Processor -- start 2024-04-25T11:22:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.300+0000 7fefde200640 1 -- start start 2024-04-25T11:22:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.302+0000 7fefde200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fefd80aed40 0x7fefd814d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.302+0000 7fefde200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fefd80af420 0x7fefd80caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.302+0000 7fefde200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefd80b5220 0x7fefd80caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.302+0000 7fefde200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fefd806ceb0 con 0x7fefd80b5220 2024-04-25T11:22:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.302+0000 7fefde200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fefd806cbd0 con 0x7fefd80af420 2024-04-25T11:22:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.302+0000 7fefde200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fefd80b6110 con 0x7fefd80aed40 2024-04-25T11:22:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.303+0000 7fefdd800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefd80b5220 0x7fefd80caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.303+0000 7fefdd800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefd80b5220 0x7fefd80caff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53324/0 (socket says 172.21.15.7:53324) 2024-04-25T11:22:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.303+0000 7fefdd800640 1 -- 172.21.15.7:0/4242562917 learned_addr learned my addr 172.21.15.7:0/4242562917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.303+0000 7fefd7e00640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fefd80af420 0x7fefd80caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.303+0000 7fefdce00640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fefd80aed40 0x7fefd814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.303+0000 7fefdd800640 1 -- 172.21.15.7:0/4242562917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fefd80aed40 msgr2=0x7fefd814d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.303+0000 7fefdd800640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fefd80aed40 0x7fefd814d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.303+0000 7fefdd800640 1 -- 172.21.15.7:0/4242562917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fefd80af420 msgr2=0x7fefd80caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.303+0000 7fefdd800640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fefd80af420 0x7fefd80caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.303+0000 7fefdd800640 1 -- 172.21.15.7:0/4242562917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd80cdb40 con 0x7fefd80b5220 2024-04-25T11:22:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.304+0000 7fefdd800640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefd80b5220 0x7fefd80caff0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fefd000cb10 tx=0x7fefd000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.304+0000 7fefd4c00640 1 -- 172.21.15.7:0/4242562917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0017070 con 0x7fefd80b5220 2024-04-25T11:22:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.304+0000 7fefd4c00640 1 -- 172.21.15.7:0/4242562917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd000abb0 con 0x7fefd80b5220 2024-04-25T11:22:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.304+0000 7fefd4c00640 1 -- 172.21.15.7:0/4242562917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0012400 con 0x7fefd80b5220 2024-04-25T11:22:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.304+0000 7fefde200640 1 -- 172.21.15.7:0/4242562917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd80cde20 con 0x7fefd80b5220 2024-04-25T11:22:13.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.304+0000 7fefde200640 1 -- 172.21.15.7:0/4242562917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd8159210 con 0x7fefd80b5220 2024-04-25T11:22:13.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.305+0000 7fefde200640 1 -- 172.21.15.7:0/4242562917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd8050740 con 0x7fefd80b5220 2024-04-25T11:22:13.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.307+0000 7fefd4c00640 1 -- 172.21.15.7:0/4242562917 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fefd001a070 con 0x7fefd80b5220 2024-04-25T11:22:13.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.308+0000 7fefd4c00640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fefb80744b0 0x7fefb80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:13.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.308+0000 7fefd4c00640 1 -- 172.21.15.7:0/4242562917 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7fefd0095070 con 0x7fefd80b5220 2024-04-25T11:22:13.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.308+0000 7fefdce00640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fefb80744b0 0x7fefb80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:13.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.309+0000 7fefdce00640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fefb80744b0 0x7fefb80769c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fefcc034000 tx=0x7fefcc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:13.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.313+0000 7fefd4c00640 1 -- 172.21.15.7:0/4242562917 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fefd80cebb0 con 0x7fefd80b5220 2024-04-25T11:22:13.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.556+0000 7fefde200640 1 -- 172.21.15.7:0/4242562917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefd80ce6b0 con 0x7fefd80b5220 2024-04-25T11:22:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.557+0000 7fefd4c00640 1 -- 172.21.15.7:0/4242562917 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5173 (secure 0 0 0) 0x7fefd005f700 con 0x7fefd80b5220 2024-04-25T11:22:13.554 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:13.554 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/3503931395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3503931395},{"type":"v1","addr":"172.21.15.87:6834","nonce":3503931395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/63829697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":63829697},{"type":"v1","addr":"172.21.15.87:6835","nonce":63829697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/415877708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":415877708},{"type":"v1","addr":"172.21.15.7:6835","nonce":415877708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:21:37.365427+0000","modified":"2024-04-25T11:22:09.892126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8967},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8967":{"gid":8967,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.7:6837/2966579341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2966579341},{"type":"v1","addr":"172.21.15.7:6837","nonce":2966579341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8967,"qdb_cluster":[8967]},"id":10}]} 2024-04-25T11:22:13.555 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 106 2024-04-25T11:22:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.560+0000 7fefde200640 1 -- 172.21.15.7:0/4242562917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fefb80744b0 msgr2=0x7fefb80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.560+0000 7fefde200640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fefb80744b0 0x7fefb80769c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fefcc034000 tx=0x7fefcc02e000 comp rx=0 tx=0).stop 2024-04-25T11:22:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.560+0000 7fefde200640 1 -- 172.21.15.7:0/4242562917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefd80b5220 msgr2=0x7fefd80caff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.560+0000 7fefde200640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefd80b5220 0x7fefd80caff0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fefd000cb10 tx=0x7fefd000a060 comp rx=0 tx=0).stop 2024-04-25T11:22:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.561+0000 7fefde200640 1 -- 172.21.15.7:0/4242562917 shutdown_connections 2024-04-25T11:22:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.561+0000 7fefde200640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fefb80744b0 0x7fefb80769c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.561+0000 7fefde200640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefd80b5220 0x7fefd80caff0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.561+0000 7fefde200640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fefd80af420 0x7fefd80caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.561+0000 7fefde200640 1 --2- 172.21.15.7:0/4242562917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fefd80aed40 0x7fefd814d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.561+0000 7fefde200640 1 -- 172.21.15.7:0/4242562917 >> 172.21.15.7:0/4242562917 conn(0x7fefd80b3820 msgr2=0x7fefd80d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:13.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.561+0000 7fefde200640 1 -- 172.21.15.7:0/4242562917 shutdown_connections 2024-04-25T11:22:13.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.561+0000 7fefde200640 1 -- 172.21.15.7:0/4242562917 wait complete. 2024-04-25T11:22:13.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:13.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:22:13.863 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:22:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.875+0000 7f6700c00640 1 Processor -- start 2024-04-25T11:22:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.875+0000 7f6700c00640 1 -- start start 2024-04-25T11:22:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.878+0000 7f6700c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f66fc0aed40 0x7f66fc15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.878+0000 7f6700c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66fc0af420 0x7f66fc0af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.878+0000 7f6700c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66fc0cebb0 con 0x7f66fc0b5220 2024-04-25T11:22:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.878+0000 7f6700c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f66fc15e1f0 con 0x7f66fc0af420 2024-04-25T11:22:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.878+0000 7f6700c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f66fc0afd70 con 0x7f66fc0aed40 2024-04-25T11:22:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.878+0000 7f66faa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f66fc0b5220 0x7f66fc0ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46112/0 (socket says 172.21.15.7:46112) 2024-04-25T11:22:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.878+0000 7f66faa00640 1 -- 172.21.15.7:0/2375809867 learned_addr learned my addr 172.21.15.7:0/2375809867 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.878+0000 7f66fbe00640 1 --2- 172.21.15.7:0/2375809867 >> v2:172.21.15.7:3300/0 conn(0x7f66fc0af420 0x7f66fc0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.878+0000 7f66fb400640 1 --2- 172.21.15.7:0/2375809867 >> v2:172.21.15.87:3300/0 conn(0x7f66fc0aed40 0x7f66fc15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.879+0000 7f66fa000640 1 -- 172.21.15.7:0/2375809867 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3311339066 0 0) 0x7f66e8001110 con 0x7f66fc0b5220 2024-04-25T11:22:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.879+0000 7f66fa000640 1 -- 172.21.15.7:0/2375809867 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66e0003520 con 0x7f66fc0b5220 2024-04-25T11:22:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.879+0000 7f66fbe00640 1 -- 172.21.15.7:0/2375809867 >> v2:172.21.15.87:3300/0 conn(0x7f66fc0aed40 msgr2=0x7f66fc15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.879+0000 7f66fbe00640 1 --2- 172.21.15.7:0/2375809867 >> v2:172.21.15.87:3300/0 conn(0x7f66fc0aed40 0x7f66fc15dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.879+0000 7f66fbe00640 1 -- 172.21.15.7:0/2375809867 >> v1:172.21.15.7:6789/0 conn(0x7f66fc0b5220 legacy=0x7f66fc0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.879+0000 7f66fbe00640 1 -- 172.21.15.7:0/2375809867 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66fc0b64b0 con 0x7f66fc0af420 2024-04-25T11:22:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.880+0000 7f66fbe00640 1 --2- 172.21.15.7:0/2375809867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0af420 0x7f66fc0af830 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0725e0 tx=0x7f66f402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdfda33a35ec5ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.880+0000 7f66fa000640 1 -- 172.21.15.7:0/2375809867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4007740 con 0x7f66fc0af420 2024-04-25T11:22:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.880+0000 7f66fa000640 1 -- 172.21.15.7:0/2375809867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f40078e0 con 0x7f66fc0af420 2024-04-25T11:22:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.880+0000 7f66fa000640 1 -- 172.21.15.7:0/2375809867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4007cb0 con 0x7f66fc0af420 2024-04-25T11:22:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.881+0000 7f6700c00640 1 -- 172.21.15.7:0/2375809867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0af420 msgr2=0x7f66fc0af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.881+0000 7f6700c00640 1 --2- 172.21.15.7:0/2375809867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0af420 0x7f66fc0af830 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0725e0 tx=0x7f66f402a680 comp rx=0 tx=0).stop 2024-04-25T11:22:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.881+0000 7f6700c00640 1 -- 172.21.15.7:0/2375809867 shutdown_connections 2024-04-25T11:22:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.881+0000 7f6700c00640 1 --2- 172.21.15.7:0/2375809867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0af420 0x7f66fc0af830 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.881+0000 7f6700c00640 1 --2- 172.21.15.7:0/2375809867 >> v2:172.21.15.87:3300/0 conn(0x7f66fc0aed40 0x7f66fc15dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.881+0000 7f6700c00640 1 -- 172.21.15.7:0/2375809867 >> 172.21.15.7:0/2375809867 conn(0x7f66fc0b3820 msgr2=0x7f66fc0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.881+0000 7f6700c00640 1 -- 172.21.15.7:0/2375809867 shutdown_connections 2024-04-25T11:22:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.881+0000 7f6700c00640 1 -- 172.21.15.7:0/2375809867 wait complete. 2024-04-25T11:22:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.884+0000 7f6700c00640 1 Processor -- start 2024-04-25T11:22:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.884+0000 7f6700c00640 1 -- start start 2024-04-25T11:22:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f6700c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66fc0aed40 0x7f66fc1480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f6700c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66fc0af420 0x7f66fc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f6700c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0b5220 0x7f66fc160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f6700c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc06cbd0 con 0x7f66fc0b5220 2024-04-25T11:22:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f6700c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc06c8f0 con 0x7f66fc0af420 2024-04-25T11:22:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f6700c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66fc06ceb0 con 0x7f66fc0aed40 2024-04-25T11:22:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f66fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0b5220 0x7f66fc160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f66fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0b5220 0x7f66fc160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53336/0 (socket says 172.21.15.7:53336) 2024-04-25T11:22:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f66fbe00640 1 -- 172.21.15.7:0/1587928404 learned_addr learned my addr 172.21.15.7:0/1587928404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f66fb400640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66fc0aed40 0x7f66fc1480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f66faa00640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66fc0af420 0x7f66fc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f66fbe00640 1 -- 172.21.15.7:0/1587928404 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66fc0aed40 msgr2=0x7f66fc1480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.886+0000 7f66fbe00640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66fc0aed40 0x7f66fc1480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.887+0000 7f66fbe00640 1 -- 172.21.15.7:0/1587928404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66fc0af420 msgr2=0x7f66fc06ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.887+0000 7f66fbe00640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66fc0af420 0x7f66fc06ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.887+0000 7f66fbe00640 1 -- 172.21.15.7:0/1587928404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66fc06fe40 con 0x7f66fc0b5220 2024-04-25T11:22:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.887+0000 7f66fbe00640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0b5220 0x7f66fc160380 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009e50 tx=0x7f66f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.887+0000 7f66f3400640 1 -- 172.21.15.7:0/1587928404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f403f070 con 0x7f66fc0b5220 2024-04-25T11:22:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.887+0000 7f66f3400640 1 -- 172.21.15.7:0/1587928404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f4031bf0 con 0x7f66fc0b5220 2024-04-25T11:22:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.887+0000 7f66f3400640 1 -- 172.21.15.7:0/1587928404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4009450 con 0x7f66fc0b5220 2024-04-25T11:22:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.887+0000 7f6700c00640 1 -- 172.21.15.7:0/1587928404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66fc148600 con 0x7f66fc0b5220 2024-04-25T11:22:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.888+0000 7f6700c00640 1 -- 172.21.15.7:0/1587928404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66fc1489f0 con 0x7f66fc0b5220 2024-04-25T11:22:13.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.889+0000 7f6700c00640 1 -- 172.21.15.7:0/1587928404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66fc06f8d0 con 0x7f66fc0b5220 2024-04-25T11:22:13.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.890+0000 7f66f3400640 1 -- 172.21.15.7:0/1587928404 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f66f403b410 con 0x7f66fc0b5220 2024-04-25T11:22:13.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.891+0000 7f66f3400640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66dc0744e0 0x7f66dc0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:13.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.891+0000 7f66f3400640 1 -- 172.21.15.7:0/1587928404 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7f66f40b6c10 con 0x7f66fc0b5220 2024-04-25T11:22:13.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.892+0000 7f66fb400640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66dc0744e0 0x7f66dc0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:13.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.893+0000 7f66fb400640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66dc0744e0 0x7f66dc0769f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f66fc071200 tx=0x7f66ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:13.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:13.896+0000 7f66f3400640 1 -- 172.21.15.7:0/1587928404 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f66fc148600 con 0x7f66fc0b5220 2024-04-25T11:22:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.149+0000 7f6700c00640 1 -- 172.21.15.7:0/1587928404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66fc050740 con 0x7f66fc0b5220 2024-04-25T11:22:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.150+0000 7f66f3400640 1 -- 172.21.15.7:0/1587928404 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5173 (secure 0 0 0) 0x7f66f4080b80 con 0x7f66fc0b5220 2024-04-25T11:22:14.146 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:14.146 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/3503931395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3503931395},{"type":"v1","addr":"172.21.15.87:6834","nonce":3503931395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/63829697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":63829697},{"type":"v1","addr":"172.21.15.87:6835","nonce":63829697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/415877708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":415877708},{"type":"v1","addr":"172.21.15.7:6835","nonce":415877708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:21:37.365427+0000","modified":"2024-04-25T11:22:09.892126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8967},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8967":{"gid":8967,"name":"c","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.7:6837/2966579341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2966579341},{"type":"v1","addr":"172.21.15.7:6837","nonce":2966579341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8967,"qdb_cluster":[8967]},"id":10}]} 2024-04-25T11:22:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 106 2024-04-25T11:22:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 -- 172.21.15.7:0/1587928404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66dc0744e0 msgr2=0x7f66dc0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66dc0744e0 0x7f66dc0769f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f66fc071200 tx=0x7f66ec008040 comp rx=0 tx=0).stop 2024-04-25T11:22:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 -- 172.21.15.7:0/1587928404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0b5220 msgr2=0x7f66fc160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0b5220 0x7f66fc160380 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009e50 tx=0x7f66f4004a40 comp rx=0 tx=0).stop 2024-04-25T11:22:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 -- 172.21.15.7:0/1587928404 shutdown_connections 2024-04-25T11:22:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f66dc0744e0 0x7f66dc0769f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0b5220 0x7f66fc160380 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66fc0af420 0x7f66fc06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 --2- 172.21.15.7:0/1587928404 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66fc0aed40 0x7f66fc1480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 -- 172.21.15.7:0/1587928404 >> 172.21.15.7:0/1587928404 conn(0x7f66fc0b3820 msgr2=0x7f66fc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 -- 172.21.15.7:0/1587928404 shutdown_connections 2024-04-25T11:22:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.153+0000 7f6700c00640 1 -- 172.21.15.7:0/1587928404 wait complete. 2024-04-25T11:22:14.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:14.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.454+0000 7f0e6be00640 1 Processor -- start 2024-04-25T11:22:14.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.454+0000 7f0e6be00640 1 -- start start 2024-04-25T11:22:14.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.456+0000 7f0e6be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e6c15c390 0x7f0e6c0b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:14.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.456+0000 7f0e6be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0e6c06ce00 0x7f0e6c06d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:14.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.456+0000 7f0e6be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e6c06ef10 con 0x7f0e6c070890 2024-04-25T11:22:14.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.456+0000 7f0e6be00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0e6c06f210 con 0x7f0e6c15c390 2024-04-25T11:22:14.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.456+0000 7f0e6be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0e6c06d750 con 0x7f0e6c06ce00 2024-04-25T11:22:14.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.457+0000 7f0e62000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e6c15c390 0x7f0e6c0b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:14.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.457+0000 7f0e62000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e6c15c390 0x7f0e6c0b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53348/0 (socket says 172.21.15.7:53348) 2024-04-25T11:22:14.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.457+0000 7f0e62000640 1 -- 172.21.15.7:0/1960544089 learned_addr learned my addr 172.21.15.7:0/1960544089 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:14.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.457+0000 7f0e6b400640 1 --2- 172.21.15.7:0/1960544089 >> v2:172.21.15.87:3300/0 conn(0x7f0e6c06ce00 0x7f0e6c06d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:14.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.457+0000 7f0e62000640 1 -- 172.21.15.7:0/1960544089 >> v2:172.21.15.87:3300/0 conn(0x7f0e6c06ce00 msgr2=0x7f0e6c06d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:14.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.457+0000 7f0e62000640 1 --2- 172.21.15.7:0/1960544089 >> v2:172.21.15.87:3300/0 conn(0x7f0e6c06ce00 0x7f0e6c06d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.457+0000 7f0e62000640 1 -- 172.21.15.7:0/1960544089 >> v1:172.21.15.87:6789/0 conn(0x7f0e6c070890 legacy=0x7f0e6c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:14.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.457+0000 7f0e62000640 1 -- 172.21.15.7:0/1960544089 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6c0b6390 con 0x7f0e6c15c390 2024-04-25T11:22:14.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.459+0000 7f0e62000640 1 --2- 172.21.15.7:0/1960544089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c15c390 0x7f0e6c0b5140 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0722d0 tx=0x7f0e5402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73118dbf3ec7c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:14.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.459+0000 7f0e6a000640 1 -- 172.21.15.7:0/1960544089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54032710 con 0x7f0e6c15c390 2024-04-25T11:22:14.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.459+0000 7f0e6a000640 1 -- 172.21.15.7:0/1960544089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e540328b0 con 0x7f0e6c15c390 2024-04-25T11:22:14.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.459+0000 7f0e6a000640 1 -- 172.21.15.7:0/1960544089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54032ca0 con 0x7f0e6c15c390 2024-04-25T11:22:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.460+0000 7f0e6be00640 1 -- 172.21.15.7:0/1960544089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c15c390 msgr2=0x7f0e6c0b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:14.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.460+0000 7f0e6be00640 1 --2- 172.21.15.7:0/1960544089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c15c390 0x7f0e6c0b5140 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0722d0 tx=0x7f0e5402c850 comp rx=0 tx=0).stop 2024-04-25T11:22:14.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.460+0000 7f0e6be00640 1 -- 172.21.15.7:0/1960544089 shutdown_connections 2024-04-25T11:22:14.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.460+0000 7f0e6be00640 1 --2- 172.21.15.7:0/1960544089 >> v2:172.21.15.87:3300/0 conn(0x7f0e6c06ce00 0x7f0e6c06d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.460+0000 7f0e6be00640 1 --2- 172.21.15.7:0/1960544089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c15c390 0x7f0e6c0b5140 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.460+0000 7f0e6be00640 1 -- 172.21.15.7:0/1960544089 >> 172.21.15.7:0/1960544089 conn(0x7f0e6c0b3820 msgr2=0x7f0e6c0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:14.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.460+0000 7f0e6be00640 1 -- 172.21.15.7:0/1960544089 shutdown_connections 2024-04-25T11:22:14.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.460+0000 7f0e6be00640 1 -- 172.21.15.7:0/1960544089 wait complete. 2024-04-25T11:22:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.463+0000 7f0e6be00640 1 Processor -- start 2024-04-25T11:22:14.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.463+0000 7f0e6be00640 1 -- start start 2024-04-25T11:22:14.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.464+0000 7f0e6be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e6c06ce00 0x7f0e6c14f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.464+0000 7f0e6be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c070890 0x7f0e6c154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.464+0000 7f0e6be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e6c15c390 0x7f0e6c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.464+0000 7f0e6be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c15c7a0 con 0x7f0e6c070890 2024-04-25T11:22:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.464+0000 7f0e6be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6c06fdb0 con 0x7f0e6c15c390 2024-04-25T11:22:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.464+0000 7f0e6be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0e6c0b6950 con 0x7f0e6c06ce00 2024-04-25T11:22:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.465+0000 7f0e62000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c070890 0x7f0e6c154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.465+0000 7f0e62000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c070890 0x7f0e6c154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53356/0 (socket says 172.21.15.7:53356) 2024-04-25T11:22:14.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.465+0000 7f0e62000640 1 -- 172.21.15.7:0/3200523041 learned_addr learned my addr 172.21.15.7:0/3200523041 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.465+0000 7f0e6aa00640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e6c06ce00 0x7f0e6c14f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.465+0000 7f0e62000640 1 -- 172.21.15.7:0/3200523041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e6c06ce00 msgr2=0x7f0e6c14f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.465+0000 7f0e62000640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e6c06ce00 0x7f0e6c14f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.465+0000 7f0e62000640 1 -- 172.21.15.7:0/3200523041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e6c15c390 msgr2=0x7f0e6c156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.465+0000 7f0e62000640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e6c15c390 0x7f0e6c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.465+0000 7f0e62000640 1 -- 172.21.15.7:0/3200523041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6c1580a0 con 0x7f0e6c070890 2024-04-25T11:22:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.465+0000 7f0e62000640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c070890 0x7f0e6c154270 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f0e5402c890 tx=0x7f0e540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.466+0000 7f0e63e00640 1 -- 172.21.15.7:0/3200523041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54042070 con 0x7f0e6c070890 2024-04-25T11:22:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.466+0000 7f0e63e00640 1 -- 172.21.15.7:0/3200523041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e54003830 con 0x7f0e6c070890 2024-04-25T11:22:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.466+0000 7f0e63e00640 1 -- 172.21.15.7:0/3200523041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54003c20 con 0x7f0e6c070890 2024-04-25T11:22:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.466+0000 7f0e6be00640 1 -- 172.21.15.7:0/3200523041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6c158380 con 0x7f0e6c070890 2024-04-25T11:22:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.466+0000 7f0e6be00640 1 -- 172.21.15.7:0/3200523041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e6c15f4f0 con 0x7f0e6c070890 2024-04-25T11:22:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.467+0000 7f0e6be00640 1 -- 172.21.15.7:0/3200523041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e6c157e10 con 0x7f0e6c070890 2024-04-25T11:22:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.468+0000 7f0e63e00640 1 -- 172.21.15.7:0/3200523041 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0e54045070 con 0x7f0e6c070890 2024-04-25T11:22:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.470+0000 7f0e63e00640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e480744b0 0x7f0e480769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.470+0000 7f0e63e00640 1 -- 172.21.15.7:0/3200523041 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7f0e540b8140 con 0x7f0e6c070890 2024-04-25T11:22:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.470+0000 7f0e6aa00640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e480744b0 0x7f0e480769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.471+0000 7f0e6aa00640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e480744b0 0x7f0e480769c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c09fbd0 tx=0x7f0e5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.475+0000 7f0e63e00640 1 -- 172.21.15.7:0/3200523041 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0e540827a0 con 0x7f0e6c070890 2024-04-25T11:22:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.677+0000 7f0e6be00640 1 -- 172.21.15.7:0/3200523041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e6c050740 con 0x7f0e6c070890 2024-04-25T11:22:14.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.679+0000 7f0e63e00640 1 -- 172.21.15.7:0/3200523041 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16954 (secure 0 0 0) 0x7f0e540820a0 con 0x7f0e6c070890 2024-04-25T11:22:14.676 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:14.676 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":144,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:04.499877+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:21:35.562066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T11:21:36.342958+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.34","mappings":[{"from":3,"to":0}]},{"pgid":"22.37","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:14.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 -- 172.21.15.7:0/3200523041 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e480744b0 msgr2=0x7f0e480769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:14.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e480744b0 0x7f0e480769c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c09fbd0 tx=0x7f0e5c008040 comp rx=0 tx=0).stop 2024-04-25T11:22:14.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 -- 172.21.15.7:0/3200523041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c070890 msgr2=0x7f0e6c154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:14.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c070890 0x7f0e6c154270 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f0e5402c890 tx=0x7f0e540046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 -- 172.21.15.7:0/3200523041 shutdown_connections 2024-04-25T11:22:14.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0e480744b0 0x7f0e480769c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e6c15c390 0x7f0e6c156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e6c070890 0x7f0e6c154270 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 --2- 172.21.15.7:0/3200523041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e6c06ce00 0x7f0e6c14f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 -- 172.21.15.7:0/3200523041 >> 172.21.15.7:0/3200523041 conn(0x7f0e6c0b3820 msgr2=0x7f0e6c0684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:14.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 -- 172.21.15.7:0/3200523041 shutdown_connections 2024-04-25T11:22:14.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.682+0000 7f0e6be00640 1 -- 172.21.15.7:0/3200523041 wait complete. 2024-04-25T11:22:14.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:22:14.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.987+0000 7fa695400640 1 Processor -- start 2024-04-25T11:22:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.987+0000 7fa695400640 1 -- start start 2024-04-25T11:22:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.990+0000 7fa695400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa69006f1e0 0x7fa69006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.990+0000 7fa695400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa690162700 con 0x7fa69006d580 2024-04-25T11:22:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.990+0000 7fa695400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6901629e0 con 0x7fa69015f180 2024-04-25T11:22:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.990+0000 7fa695400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa69015f590 con 0x7fa69006f1e0 2024-04-25T11:22:14.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.990+0000 7fa68f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa69006f1e0 0x7fa69006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:14.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.990+0000 7fa68de00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa69006d580 0x7fa69006d990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46124/0 (socket says 172.21.15.7:46124) 2024-04-25T11:22:14.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.990+0000 7fa68de00640 1 -- 172.21.15.7:0/2329667041 learned_addr learned my addr 172.21.15.7:0/2329667041 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.991+0000 7fa68f200640 1 -- 172.21.15.7:0/2329667041 >> v1:172.21.15.87:6789/0 conn(0x7fa69015f180 legacy=0x7fa690161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.991+0000 7fa68f200640 1 -- 172.21.15.7:0/2329667041 >> v1:172.21.15.7:6789/0 conn(0x7fa69006d580 legacy=0x7fa69006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.991+0000 7fa68d400640 1 -- 172.21.15.7:0/2329667041 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2526333171 0 0) 0x7fa690162700 con 0x7fa69006d580 2024-04-25T11:22:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.991+0000 7fa68f200640 1 -- 172.21.15.7:0/2329667041 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69015e980 con 0x7fa69006f1e0 2024-04-25T11:22:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.992+0000 7fa68f200640 1 --2- 172.21.15.7:0/2329667041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa69006f1e0 0x7fa69006f5f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fa688009d90 tx=0x7fa68802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f890d857789720e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.992+0000 7fa68d400640 1 -- 172.21.15.7:0/2329667041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688031600 con 0x7fa69006f1e0 2024-04-25T11:22:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.992+0000 7fa68d400640 1 -- 172.21.15.7:0/2329667041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6880317a0 con 0x7fa69006f1e0 2024-04-25T11:22:14.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.992+0000 7fa68d400640 1 -- 172.21.15.7:0/2329667041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688031b70 con 0x7fa69006f1e0 2024-04-25T11:22:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.993+0000 7fa695400640 1 -- 172.21.15.7:0/2329667041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa69006f1e0 msgr2=0x7fa69006f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.993+0000 7fa695400640 1 --2- 172.21.15.7:0/2329667041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa69006f1e0 0x7fa69006f5f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fa688009d90 tx=0x7fa68802a610 comp rx=0 tx=0).stop 2024-04-25T11:22:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.993+0000 7fa695400640 1 -- 172.21.15.7:0/2329667041 shutdown_connections 2024-04-25T11:22:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.993+0000 7fa695400640 1 --2- 172.21.15.7:0/2329667041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa69006f1e0 0x7fa69006f5f0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.993+0000 7fa695400640 1 -- 172.21.15.7:0/2329667041 >> 172.21.15.7:0/2329667041 conn(0x7fa6900732b0 msgr2=0x7fa6900736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.993+0000 7fa695400640 1 -- 172.21.15.7:0/2329667041 shutdown_connections 2024-04-25T11:22:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.993+0000 7fa695400640 1 -- 172.21.15.7:0/2329667041 wait complete. 2024-04-25T11:22:14.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.995+0000 7fa695400640 1 Processor -- start 2024-04-25T11:22:14.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.995+0000 7fa695400640 1 -- start start 2024-04-25T11:22:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.997+0000 7fa695400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa69006d580 0x7fa69006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.997+0000 7fa695400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa69006f1e0 0x7fa690156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.997+0000 7fa695400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa69015f180 0x7fa690156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.997+0000 7fa695400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa69015e670 con 0x7fa69006d580 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.997+0000 7fa695400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa690158ae0 con 0x7fa69006f1e0 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.997+0000 7fa695400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa6900d1310 con 0x7fa69015f180 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.997+0000 7fa68e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa69006d580 0x7fa69006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.997+0000 7fa68e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa69006d580 0x7fa69006f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53370/0 (socket says 172.21.15.7:53370) 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.997+0000 7fa68e800640 1 -- 172.21.15.7:0/2212186889 learned_addr learned my addr 172.21.15.7:0/2212186889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa68de00640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa69006f1e0 0x7fa690156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa68f200640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa69015f180 0x7fa690156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa68e800640 1 -- 172.21.15.7:0/2212186889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa69015f180 msgr2=0x7fa690156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa68e800640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa69015f180 0x7fa690156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa68e800640 1 -- 172.21.15.7:0/2212186889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa69006f1e0 msgr2=0x7fa690156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa68e800640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa69006f1e0 0x7fa690156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa68e800640 1 -- 172.21.15.7:0/2212186889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa690156a90 con 0x7fa69006d580 2024-04-25T11:22:14.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa68de00640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa69006f1e0 0x7fa690156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:14.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa68e800640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa69006d580 0x7fa69006f7d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b3c0 tx=0x7fa68000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:14.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa686a00640 1 -- 172.21.15.7:0/2212186889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680011830 con 0x7fa69006d580 2024-04-25T11:22:14.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa686a00640 1 -- 172.21.15.7:0/2212186889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa680016070 con 0x7fa69006d580 2024-04-25T11:22:14.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa686a00640 1 -- 172.21.15.7:0/2212186889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680011c50 con 0x7fa69006d580 2024-04-25T11:22:14.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.998+0000 7fa695400640 1 -- 172.21.15.7:0/2212186889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69009eb90 con 0x7fa69006d580 2024-04-25T11:22:14.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:14.999+0000 7fa695400640 1 -- 172.21.15.7:0/2212186889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa69009ddd0 con 0x7fa69006d580 2024-04-25T11:22:14.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.000+0000 7fa695400640 1 -- 172.21.15.7:0/2212186889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa690050740 con 0x7fa69006d580 2024-04-25T11:22:14.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.001+0000 7fa686a00640 1 -- 172.21.15.7:0/2212186889 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa68001b460 con 0x7fa69006d580 2024-04-25T11:22:14.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.002+0000 7fa686a00640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6700744b0 0x7fa6700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:14.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.002+0000 7fa686a00640 1 -- 172.21.15.7:0/2212186889 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 6159+0+0 (secure 0 0 0) 0x7fa680094e50 con 0x7fa69006d580 2024-04-25T11:22:14.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.002+0000 7fa68de00640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6700744b0 0x7fa6700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:14.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.003+0000 7fa68de00640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6700744b0 0x7fa6700769c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa6900b1480 tx=0x7fa67c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:15.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.006+0000 7fa686a00640 1 -- 172.21.15.7:0/2212186889 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa680020020 con 0x7fa69006d580 2024-04-25T11:22:15.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.255+0000 7fa695400640 1 -- 172.21.15.7:0/2212186889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa690074250 con 0x7fa69006d580 2024-04-25T11:22:15.252 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-04-25T11:22:15.256+0000 7fac34e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:22:15.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.260+0000 7fa686a00640 1 -- 172.21.15.7:0/2212186889 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) ==== 161+0+0 (secure 0 0 0) 0x7fa68005f4b0 con 0x7fa69006d580 2024-04-25T11:22:15.256 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:22:15.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 -- 172.21.15.7:0/2212186889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6700744b0 msgr2=0x7fa6700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:15.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6700744b0 0x7fa6700769c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa6900b1480 tx=0x7fa67c009040 comp rx=0 tx=0).stop 2024-04-25T11:22:15.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 -- 172.21.15.7:0/2212186889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa69006d580 msgr2=0x7fa69006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:15.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa69006d580 0x7fa69006f7d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fa68000b3c0 tx=0x7fa68000b880 comp rx=0 tx=0).stop 2024-04-25T11:22:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 -- 172.21.15.7:0/2212186889 shutdown_connections 2024-04-25T11:22:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6700744b0 0x7fa6700769c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa69015f180 0x7fa690156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa69006f1e0 0x7fa690156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 --2- 172.21.15.7:0/2212186889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa69006d580 0x7fa69006f7d0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 -- 172.21.15.7:0/2212186889 >> 172.21.15.7:0/2212186889 conn(0x7fa6900732b0 msgr2=0x7fa6900b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 -- 172.21.15.7:0/2212186889 shutdown_connections 2024-04-25T11:22:15.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.263+0000 7fa695400640 1 -- 172.21.15.7:0/2212186889 wait complete. 2024-04-25T11:22:15.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:22:15.445 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:22:15.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.571+0000 7f1ce2400640 1 Processor -- start 2024-04-25T11:22:15.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.571+0000 7f1ce2400640 1 -- start start 2024-04-25T11:22:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.573+0000 7f1ce2400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1cdc0d0ee0 0x7f1cdc0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.573+0000 7f1ce2400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cdc0b5020 0x7f1cdc160990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.573+0000 7f1ce2400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cdc06f5b0 con 0x7f1cdc15aaa0 2024-04-25T11:22:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.573+0000 7f1ce2400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1cdc0af290 con 0x7f1cdc0b5020 2024-04-25T11:22:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.573+0000 7f1ce2400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1cdc0d14a0 con 0x7f1cdc0d0ee0 2024-04-25T11:22:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.574+0000 7f1cdbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cdc0b5020 0x7f1cdc160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.574+0000 7f1cdb400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1cdc15aaa0 0x7f1cdc0b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46148/0 (socket says 172.21.15.7:46148) 2024-04-25T11:22:15.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.574+0000 7f1cdb400640 1 -- 172.21.15.7:0/1028359957 learned_addr learned my addr 172.21.15.7:0/1028359957 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.574+0000 7f1cdaa00640 1 --2- 172.21.15.7:0/1028359957 >> v2:172.21.15.87:3300/0 conn(0x7f1cdc0d0ee0 0x7f1cdc0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.574+0000 7f1cdbe00640 1 -- 172.21.15.7:0/1028359957 >> v2:172.21.15.87:3300/0 conn(0x7f1cdc0d0ee0 msgr2=0x7f1cdc0affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.574+0000 7f1cdbe00640 1 --2- 172.21.15.7:0/1028359957 >> v2:172.21.15.87:3300/0 conn(0x7f1cdc0d0ee0 0x7f1cdc0affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:15.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.574+0000 7f1cdbe00640 1 -- 172.21.15.7:0/1028359957 >> v1:172.21.15.7:6789/0 conn(0x7f1cdc15aaa0 legacy=0x7f1cdc0b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:15.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.574+0000 7f1cdbe00640 1 -- 172.21.15.7:0/1028359957 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc0af4e0 con 0x7f1cdc0b5020 2024-04-25T11:22:15.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.574+0000 7f1cda000640 1 -- 172.21.15.7:0/1028359957 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3511073982 0 0) 0x7f1cdc06f5b0 con 0x7f1cdc15aaa0 2024-04-25T11:22:15.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.575+0000 7f1cdbe00640 1 --2- 172.21.15.7:0/1028359957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc0b5020 0x7f1cdc160990 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc06eba0 tx=0x7f1cd402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea2f9d0e0f4e9bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.576+0000 7f1cda000640 1 -- 172.21.15.7:0/1028359957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4033640 con 0x7f1cdc0b5020 2024-04-25T11:22:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.576+0000 7f1cda000640 1 -- 172.21.15.7:0/1028359957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd4038070 con 0x7f1cdc0b5020 2024-04-25T11:22:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.576+0000 7f1cda000640 1 -- 172.21.15.7:0/1028359957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4033a00 con 0x7f1cdc0b5020 2024-04-25T11:22:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.576+0000 7f1ce2400640 1 -- 172.21.15.7:0/1028359957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc0b5020 msgr2=0x7f1cdc160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.576+0000 7f1ce2400640 1 --2- 172.21.15.7:0/1028359957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc0b5020 0x7f1cdc160990 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc06eba0 tx=0x7f1cd402c8b0 comp rx=0 tx=0).stop 2024-04-25T11:22:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.577+0000 7f1ce2400640 1 -- 172.21.15.7:0/1028359957 shutdown_connections 2024-04-25T11:22:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.577+0000 7f1ce2400640 1 --2- 172.21.15.7:0/1028359957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc0b5020 0x7f1cdc160990 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:15.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.577+0000 7f1ce2400640 1 --2- 172.21.15.7:0/1028359957 >> v2:172.21.15.87:3300/0 conn(0x7f1cdc0d0ee0 0x7f1cdc0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.577+0000 7f1ce2400640 1 -- 172.21.15.7:0/1028359957 >> 172.21.15.7:0/1028359957 conn(0x7f1cdc0742c0 msgr2=0x7f1cdc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.577+0000 7f1ce2400640 1 -- 172.21.15.7:0/1028359957 shutdown_connections 2024-04-25T11:22:15.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.577+0000 7f1ce2400640 1 -- 172.21.15.7:0/1028359957 wait complete. 2024-04-25T11:22:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.579+0000 7f1ce2400640 1 Processor -- start 2024-04-25T11:22:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.579+0000 7f1ce2400640 1 -- start start 2024-04-25T11:22:15.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1ce2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc0b5020 0x7f1cdc14c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:15.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1ce2400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cdc0d0ee0 0x7f1cdc09b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1ce2400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cdc15aaa0 0x7f1cdc0cf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1ce2400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc0b5430 con 0x7f1cdc0b5020 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1ce2400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc0afaa0 con 0x7f1cdc0d0ee0 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1ce2400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1cdc0b55b0 con 0x7f1cdc15aaa0 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1cdb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc0b5020 0x7f1cdc14c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1cdb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc0b5020 0x7f1cdc14c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53402/0 (socket says 172.21.15.7:53402) 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1cdb400640 1 -- 172.21.15.7:0/1768280763 learned_addr learned my addr 172.21.15.7:0/1768280763 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1cdaa00640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cdc0d0ee0 0x7f1cdc09b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1cdb400640 1 -- 172.21.15.7:0/1768280763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cdc15aaa0 msgr2=0x7f1cdc0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.581+0000 7f1cdbe00640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cdc15aaa0 0x7f1cdc0cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.582+0000 7f1cdb400640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cdc15aaa0 0x7f1cdc0cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.582+0000 7f1cdb400640 1 -- 172.21.15.7:0/1768280763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cdc0d0ee0 msgr2=0x7f1cdc09b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:15.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.582+0000 7f1cdb400640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cdc0d0ee0 0x7f1cdc09b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:15.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.582+0000 7f1cdb400640 1 -- 172.21.15.7:0/1768280763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc1f5730 con 0x7f1cdc0b5020 2024-04-25T11:22:15.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.582+0000 7f1cdaa00640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cdc0d0ee0 0x7f1cdc09b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:15.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.582+0000 7f1cdb400640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc0b5020 0x7f1cdc14c480 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00c420 tx=0x7f1ccc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:15.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.582+0000 7f1cd3400640 1 -- 172.21.15.7:0/1768280763 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0074e0 con 0x7f1cdc0b5020 2024-04-25T11:22:15.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.582+0000 7f1cd3400640 1 -- 172.21.15.7:0/1768280763 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ccc0047e0 con 0x7f1cdc0b5020 2024-04-25T11:22:15.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.582+0000 7f1cd3400640 1 -- 172.21.15.7:0/1768280763 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc004c10 con 0x7f1cdc0b5020 2024-04-25T11:22:15.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.582+0000 7f1ce2400640 1 -- 172.21.15.7:0/1768280763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cdc1f5a10 con 0x7f1cdc0b5020 2024-04-25T11:22:15.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.582+0000 7f1ce2400640 1 -- 172.21.15.7:0/1768280763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cdc1f5fd0 con 0x7f1cdc0b5020 2024-04-25T11:22:15.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.584+0000 7f1ce2400640 1 -- 172.21.15.7:0/1768280763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cdc050740 con 0x7f1cdc0b5020 2024-04-25T11:22:15.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.585+0000 7f1cd3400640 1 -- 172.21.15.7:0/1768280763 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1ccc011070 con 0x7f1cdc0b5020 2024-04-25T11:22:15.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.586+0000 7f1cd3400640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb80744b0 0x7f1cb80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:15.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.586+0000 7f1cd3400640 1 -- 172.21.15.7:0/1768280763 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6245+0+0 (secure 0 0 0) 0x7f1ccc095000 con 0x7f1cdc0b5020 2024-04-25T11:22:15.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.586+0000 7f1cdaa00640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb80744b0 0x7f1cb80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:15.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.587+0000 7f1cdaa00640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb80744b0 0x7f1cb80769c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4006fd0 tx=0x7f1cc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:15.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.592+0000 7f1cd3400640 1 -- 172.21.15.7:0/1768280763 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1ccc05f600 con 0x7f1cdc0b5020 2024-04-25T11:22:15.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:15.834+0000 7f1ce2400640 1 -- 172.21.15.7:0/1768280763 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1cdc09e1a0 con 0x7f1cdc0b5020 2024-04-25T11:22:16.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.264+0000 7f1cd3400640 1 -- 172.21.15.7:0/1768280763 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v109) ==== 104+0+0 (secure 0 0 0) 0x7f1ccc05ef00 con 0x7f1cdc0b5020 2024-04-25T11:22:16.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 -- 172.21.15.7:0/1768280763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb80744b0 msgr2=0x7f1cb80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:16.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb80744b0 0x7f1cb80769c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4006fd0 tx=0x7f1cc4008040 comp rx=0 tx=0).stop 2024-04-25T11:22:16.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 -- 172.21.15.7:0/1768280763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc0b5020 msgr2=0x7f1cdc14c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:16.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc0b5020 0x7f1cdc14c480 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00c420 tx=0x7f1ccc00c8e0 comp rx=0 tx=0).stop 2024-04-25T11:22:16.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 -- 172.21.15.7:0/1768280763 shutdown_connections 2024-04-25T11:22:16.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1cb80744b0 0x7f1cb80769c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:16.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cdc15aaa0 0x7f1cdc0cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:16.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cdc0d0ee0 0x7f1cdc09b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:16.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 --2- 172.21.15.7:0/1768280763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cdc0b5020 0x7f1cdc14c480 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:16.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 -- 172.21.15.7:0/1768280763 >> 172.21.15.7:0/1768280763 conn(0x7f1cdc0742c0 msgr2=0x7f1cdc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:16.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 -- 172.21.15.7:0/1768280763 shutdown_connections 2024-04-25T11:22:16.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.268+0000 7f1ce2400640 1 -- 172.21.15.7:0/1768280763 wait complete. 2024-04-25T11:22:16.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:22:16.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.602+0000 7fbc2ba00640 1 Processor -- start 2024-04-25T11:22:16.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.603+0000 7fbc2ba00640 1 -- start start 2024-04-25T11:22:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.605+0000 7fbc2ba00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc240b47d0 0x7fbc241601f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.605+0000 7fbc2ba00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbc2406e5e0 0x7fbc240b9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.605+0000 7fbc2ba00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc240d12e0 con 0x7fbc240d5690 2024-04-25T11:22:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.605+0000 7fbc2ba00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbc240d15c0 con 0x7fbc240b47d0 2024-04-25T11:22:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.605+0000 7fbc2ba00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbc240d4bb0 con 0x7fbc2406e5e0 2024-04-25T11:22:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.606+0000 7fbc29000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc240b47d0 0x7fbc241601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.606+0000 7fbc29000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc240b47d0 0x7fbc241601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53416/0 (socket says 172.21.15.7:53416) 2024-04-25T11:22:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.606+0000 7fbc29000640 1 -- 172.21.15.7:0/1603025780 learned_addr learned my addr 172.21.15.7:0/1603025780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.606+0000 7fbc29a00640 1 --2- 172.21.15.7:0/1603025780 >> v2:172.21.15.87:3300/0 conn(0x7fbc2406e5e0 0x7fbc240b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.606+0000 7fbc29000640 1 -- 172.21.15.7:0/1603025780 >> v2:172.21.15.87:3300/0 conn(0x7fbc2406e5e0 msgr2=0x7fbc240b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.606+0000 7fbc29000640 1 --2- 172.21.15.7:0/1603025780 >> v2:172.21.15.87:3300/0 conn(0x7fbc2406e5e0 0x7fbc240b9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.606+0000 7fbc29000640 1 -- 172.21.15.7:0/1603025780 >> v1:172.21.15.87:6789/0 conn(0x7fbc240d5690 legacy=0x7fbc2406ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.606+0000 7fbc29000640 1 -- 172.21.15.7:0/1603025780 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc240d4e90 con 0x7fbc240b47d0 2024-04-25T11:22:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.606+0000 7fbc29a00640 1 --2- 172.21.15.7:0/1603025780 >> v2:172.21.15.87:3300/0 conn(0x7fbc2406e5e0 0x7fbc240b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:16.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.607+0000 7fbc29000640 1 --2- 172.21.15.7:0/1603025780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc240b47d0 0x7fbc241601f0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fbc240d36b0 tx=0x7fbc1402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901aa124482aa169 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:16.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.608+0000 7fbc23400640 1 -- 172.21.15.7:0/1603025780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14037070 con 0x7fbc240b47d0 2024-04-25T11:22:16.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.608+0000 7fbc23400640 1 -- 172.21.15.7:0/1603025780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc14032440 con 0x7fbc240b47d0 2024-04-25T11:22:16.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.608+0000 7fbc23400640 1 -- 172.21.15.7:0/1603025780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14032810 con 0x7fbc240b47d0 2024-04-25T11:22:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.608+0000 7fbc2ba00640 1 -- 172.21.15.7:0/1603025780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc240b47d0 msgr2=0x7fbc241601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.608+0000 7fbc2ba00640 1 --2- 172.21.15.7:0/1603025780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc240b47d0 0x7fbc241601f0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fbc240d36b0 tx=0x7fbc1402a210 comp rx=0 tx=0).stop 2024-04-25T11:22:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.608+0000 7fbc2ba00640 1 -- 172.21.15.7:0/1603025780 shutdown_connections 2024-04-25T11:22:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.608+0000 7fbc2ba00640 1 --2- 172.21.15.7:0/1603025780 >> v2:172.21.15.87:3300/0 conn(0x7fbc2406e5e0 0x7fbc240b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.609+0000 7fbc2ba00640 1 --2- 172.21.15.7:0/1603025780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc240b47d0 0x7fbc241601f0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.609+0000 7fbc2ba00640 1 -- 172.21.15.7:0/1603025780 >> 172.21.15.7:0/1603025780 conn(0x7fbc240d2c60 msgr2=0x7fbc240d5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.609+0000 7fbc2ba00640 1 -- 172.21.15.7:0/1603025780 shutdown_connections 2024-04-25T11:22:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.609+0000 7fbc2ba00640 1 -- 172.21.15.7:0/1603025780 wait complete. 2024-04-25T11:22:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.611+0000 7fbc2ba00640 1 Processor -- start 2024-04-25T11:22:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.611+0000 7fbc2ba00640 1 -- start start 2024-04-25T11:22:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc2ba00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc240b47d0 0x7fbc2414ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc2ba00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc240d5690 0x7fbc24159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc2ba00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc24160940 0x7fbc24159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc2ba00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbc240b4d00 con 0x7fbc240b47d0 2024-04-25T11:22:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc2ba00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbc24160730 con 0x7fbc240d5690 2024-04-25T11:22:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc2ba00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbc240b4e80 con 0x7fbc24160940 2024-04-25T11:22:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc29000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc240b47d0 0x7fbc2414ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc29000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc240b47d0 0x7fbc2414ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53420/0 (socket says 172.21.15.7:53420) 2024-04-25T11:22:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc29000640 1 -- 172.21.15.7:0/3639141257 learned_addr learned my addr 172.21.15.7:0/3639141257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc29000640 1 -- 172.21.15.7:0/3639141257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc24160940 msgr2=0x7fbc24159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc29000640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc24160940 0x7fbc24159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc29000640 1 -- 172.21.15.7:0/3639141257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc240d5690 msgr2=0x7fbc24159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc29000640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc240d5690 0x7fbc24159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.613+0000 7fbc29000640 1 -- 172.21.15.7:0/3639141257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc24163760 con 0x7fbc240b47d0 2024-04-25T11:22:16.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.614+0000 7fbc29000640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc240b47d0 0x7fbc2414ded0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fbc14000c00 tx=0x7fbc140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:16.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.614+0000 7fbc20c00640 1 -- 172.21.15.7:0/3639141257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14041070 con 0x7fbc240b47d0 2024-04-25T11:22:16.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.614+0000 7fbc20c00640 1 -- 172.21.15.7:0/3639141257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1403c890 con 0x7fbc240b47d0 2024-04-25T11:22:16.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.614+0000 7fbc2ba00640 1 -- 172.21.15.7:0/3639141257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc241f9e00 con 0x7fbc240b47d0 2024-04-25T11:22:16.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.614+0000 7fbc20c00640 1 -- 172.21.15.7:0/3639141257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc14037040 con 0x7fbc240b47d0 2024-04-25T11:22:16.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.614+0000 7fbc2ba00640 1 -- 172.21.15.7:0/3639141257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc241fa3c0 con 0x7fbc240b47d0 2024-04-25T11:22:16.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.616+0000 7fbc2ba00640 1 -- 172.21.15.7:0/3639141257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2406a100 con 0x7fbc240b47d0 2024-04-25T11:22:16.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.617+0000 7fbc20c00640 1 -- 172.21.15.7:0/3639141257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbc14044070 con 0x7fbc240b47d0 2024-04-25T11:22:16.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.618+0000 7fbc20c00640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc000744e0 0x7fbc000769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:16.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.618+0000 7fbc20c00640 1 -- 172.21.15.7:0/3639141257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 6245+0+0 (secure 0 0 0) 0x7fbc140b60f0 con 0x7fbc240b47d0 2024-04-25T11:22:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.619+0000 7fbc23e00640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc000744e0 0x7fbc000769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.619+0000 7fbc23e00640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc000744e0 0x7fbc000769f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbc24163c50 tx=0x7fbc0c009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:16.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.624+0000 7fbc20c00640 1 -- 172.21.15.7:0/3639141257 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbc140805c0 con 0x7fbc240b47d0 2024-04-25T11:22:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:16.831+0000 7fbc2ba00640 1 -- 172.21.15.7:0/3639141257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbc240b45b0 con 0x7fbc240b47d0 2024-04-25T11:22:17.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.610+0000 7fbc20c00640 1 -- 172.21.15.7:0/3639141257 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v146) ==== 181+0+0 (secure 0 0 0) 0x7fbc1407fec0 con 0x7fbc240b47d0 2024-04-25T11:22:17.607 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:22:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 -- 172.21.15.7:0/3639141257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc000744e0 msgr2=0x7fbc000769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc000744e0 0x7fbc000769f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbc24163c50 tx=0x7fbc0c009250 comp rx=0 tx=0).stop 2024-04-25T11:22:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 -- 172.21.15.7:0/3639141257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc240b47d0 msgr2=0x7fbc2414ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:17.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc240b47d0 0x7fbc2414ded0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fbc14000c00 tx=0x7fbc140046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 -- 172.21.15.7:0/3639141257 shutdown_connections 2024-04-25T11:22:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbc000744e0 0x7fbc000769f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc24160940 0x7fbc24159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc240d5690 0x7fbc24159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 --2- 172.21.15.7:0/3639141257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc240b47d0 0x7fbc2414ded0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 -- 172.21.15.7:0/3639141257 >> 172.21.15.7:0/3639141257 conn(0x7fbc240d2c60 msgr2=0x7fbc2415f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 -- 172.21.15.7:0/3639141257 shutdown_connections 2024-04-25T11:22:17.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.614+0000 7fbc2ba00640 1 -- 172.21.15.7:0/3639141257 wait complete. 2024-04-25T11:22:17.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:22:17.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.950+0000 7fd0e5000640 1 Processor -- start 2024-04-25T11:22:17.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.950+0000 7fd0e5000640 1 -- start start 2024-04-25T11:22:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.952+0000 7fd0e5000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd0e006d580 0x7fd0e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.952+0000 7fd0e5000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0e006ded0 con 0x7fd0e00b4c60 2024-04-25T11:22:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.952+0000 7fd0e5000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0e0070e40 con 0x7fd0e0143220 2024-04-25T11:22:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.952+0000 7fd0e5000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e00d02f0 con 0x7fd0e006d580 2024-04-25T11:22:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.953+0000 7fd0dfe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd0e00b4c60 0x7fd0e00b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46158/0 (socket says 172.21.15.7:46158) 2024-04-25T11:22:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.953+0000 7fd0dfe00640 1 -- 172.21.15.7:0/1570703492 learned_addr learned my addr 172.21.15.7:0/1570703492 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.953+0000 7fd0dea00640 1 --2- 172.21.15.7:0/1570703492 >> v2:172.21.15.87:3300/0 conn(0x7fd0e006d580 0x7fd0e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.953+0000 7fd0de000640 1 -- 172.21.15.7:0/1570703492 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3242045331 0 0) 0x7fd0d8001110 con 0x7fd0e00b4c60 2024-04-25T11:22:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.953+0000 7fd0de000640 1 -- 172.21.15.7:0/1570703492 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0c4003520 con 0x7fd0e00b4c60 2024-04-25T11:22:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.954+0000 7fd0dea00640 1 -- 172.21.15.7:0/1570703492 >> v1:172.21.15.87:6789/0 conn(0x7fd0e0143220 legacy=0x7fd0e00d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.954+0000 7fd0dea00640 1 -- 172.21.15.7:0/1570703492 >> v1:172.21.15.7:6789/0 conn(0x7fd0e00b4c60 legacy=0x7fd0e00b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:17.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.954+0000 7fd0dea00640 1 -- 172.21.15.7:0/1570703492 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e00d05d0 con 0x7fd0e006d580 2024-04-25T11:22:17.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.955+0000 7fd0dea00640 1 --2- 172.21.15.7:0/1570703492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0e006d580 0x7fd0e006d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80097a0 tx=0x7fd0c802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4de40e223728d252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:17.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.956+0000 7fd0de000640 1 -- 172.21.15.7:0/1570703492 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8037070 con 0x7fd0e006d580 2024-04-25T11:22:17.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.956+0000 7fd0de000640 1 -- 172.21.15.7:0/1570703492 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c8032440 con 0x7fd0e006d580 2024-04-25T11:22:17.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.956+0000 7fd0de000640 1 -- 172.21.15.7:0/1570703492 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8032810 con 0x7fd0e006d580 2024-04-25T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.957+0000 7fd0e5000640 1 -- 172.21.15.7:0/1570703492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0e006d580 msgr2=0x7fd0e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.957+0000 7fd0e5000640 1 --2- 172.21.15.7:0/1570703492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0e006d580 0x7fd0e006d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd0c80097a0 tx=0x7fd0c802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.957+0000 7fd0e5000640 1 -- 172.21.15.7:0/1570703492 shutdown_connections 2024-04-25T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.957+0000 7fd0e5000640 1 --2- 172.21.15.7:0/1570703492 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0e006d580 0x7fd0e006d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.957+0000 7fd0e5000640 1 -- 172.21.15.7:0/1570703492 >> 172.21.15.7:0/1570703492 conn(0x7fd0e0073160 msgr2=0x7fd0e0073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.957+0000 7fd0e5000640 1 -- 172.21.15.7:0/1570703492 shutdown_connections 2024-04-25T11:22:17.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.957+0000 7fd0e5000640 1 -- 172.21.15.7:0/1570703492 wait complete. 2024-04-25T11:22:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.959+0000 7fd0e5000640 1 Processor -- start 2024-04-25T11:22:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.960+0000 7fd0e5000640 1 -- start start 2024-04-25T11:22:17.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.961+0000 7fd0e5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0e006d580 0x7fd0e016ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:17.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.961+0000 7fd0e5000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0e00b4c60 0x7fd0e0138fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.961+0000 7fd0e5000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0e0143220 0x7fd0e006f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.961+0000 7fd0e5000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e00d00b0 con 0x7fd0e006d580 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.961+0000 7fd0e5000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e00cff30 con 0x7fd0e00b4c60 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.961+0000 7fd0e5000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e00700d0 con 0x7fd0e0143220 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.961+0000 7fd0df400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0e006d580 0x7fd0e016ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.961+0000 7fd0df400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0e006d580 0x7fd0e016ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53430/0 (socket says 172.21.15.7:53430) 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.961+0000 7fd0df400640 1 -- 172.21.15.7:0/1067549720 learned_addr learned my addr 172.21.15.7:0/1067549720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0dea00640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0e00b4c60 0x7fd0e0138fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0df400640 1 -- 172.21.15.7:0/1067549720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0e0143220 msgr2=0x7fd0e006f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0df400640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0e0143220 0x7fd0e006f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0df400640 1 -- 172.21.15.7:0/1067549720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0e00b4c60 msgr2=0x7fd0e0138fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0df400640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0e00b4c60 0x7fd0e0138fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0df400640 1 -- 172.21.15.7:0/1067549720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e016b470 con 0x7fd0e006d580 2024-04-25T11:22:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0dea00640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0e00b4c60 0x7fd0e0138fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0df400640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0e006d580 0x7fd0e016ab70 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000c760 tx=0x7fd0d000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0d7400640 1 -- 172.21.15.7:0/1067549720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0016020 con 0x7fd0e006d580 2024-04-25T11:22:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0d7400640 1 -- 172.21.15.7:0/1067549720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d0006d50 con 0x7fd0e006d580 2024-04-25T11:22:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0d7400640 1 -- 172.21.15.7:0/1067549720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0003bb0 con 0x7fd0e006d580 2024-04-25T11:22:17.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.962+0000 7fd0e5000640 1 -- 172.21.15.7:0/1067549720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e0141c70 con 0x7fd0e006d580 2024-04-25T11:22:17.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.963+0000 7fd0e5000640 1 -- 172.21.15.7:0/1067549720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e0142230 con 0x7fd0e006d580 2024-04-25T11:22:17.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.964+0000 7fd0e5000640 1 -- 172.21.15.7:0/1067549720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e0050740 con 0x7fd0e006d580 2024-04-25T11:22:17.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.965+0000 7fd0d7400640 1 -- 172.21.15.7:0/1067549720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd0d000a850 con 0x7fd0e006d580 2024-04-25T11:22:17.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.967+0000 7fd0d7400640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0bc0744b0 0x7fd0bc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:17.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.967+0000 7fd0d7400640 1 -- 172.21.15.7:0/1067549720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5807+0+0 (secure 0 0 0) 0x7fd0d0095d60 con 0x7fd0e006d580 2024-04-25T11:22:17.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.967+0000 7fd0dea00640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0bc0744b0 0x7fd0bc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:17.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.968+0000 7fd0dea00640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0bc0744b0 0x7fd0bc0769c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8034000 tx=0x7fd0c802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:17.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:17.970+0000 7fd0d7400640 1 -- 172.21.15.7:0/1067549720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd0d0020020 con 0x7fd0e006d580 2024-04-25T11:22:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.179+0000 7fd0e5000640 1 -- 172.21.15.7:0/1067549720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd0e0074320 con 0x7fd0e006d580 2024-04-25T11:22:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.622+0000 7fd0d7400640 1 -- 172.21.15.7:0/1067549720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v147) ==== 169+0+0 (secure 0 0 0) 0x7fd0d0060500 con 0x7fd0e006d580 2024-04-25T11:22:18.619 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-25T11:22:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.626+0000 7fd0e5000640 1 -- 172.21.15.7:0/1067549720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0bc0744b0 msgr2=0x7fd0bc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.626+0000 7fd0e5000640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0bc0744b0 0x7fd0bc0769c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8034000 tx=0x7fd0c802e000 comp rx=0 tx=0).stop 2024-04-25T11:22:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.626+0000 7fd0e5000640 1 -- 172.21.15.7:0/1067549720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0e006d580 msgr2=0x7fd0e016ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:18.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.626+0000 7fd0e5000640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0e006d580 0x7fd0e016ab70 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000c760 tx=0x7fd0d000cc20 comp rx=0 tx=0).stop 2024-04-25T11:22:18.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.627+0000 7fd0e5000640 1 -- 172.21.15.7:0/1067549720 shutdown_connections 2024-04-25T11:22:18.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.627+0000 7fd0e5000640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd0bc0744b0 0x7fd0bc0769c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:18.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.627+0000 7fd0e5000640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0e0143220 0x7fd0e006f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:18.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.627+0000 7fd0e5000640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0e00b4c60 0x7fd0e0138fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:18.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.627+0000 7fd0e5000640 1 --2- 172.21.15.7:0/1067549720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0e006d580 0x7fd0e016ab70 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:18.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.627+0000 7fd0e5000640 1 -- 172.21.15.7:0/1067549720 >> 172.21.15.7:0/1067549720 conn(0x7fd0e0073160 msgr2=0x7fd0e00b5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.628+0000 7fd0e5000640 1 -- 172.21.15.7:0/1067549720 shutdown_connections 2024-04-25T11:22:18.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.629+0000 7fd0e5000640 1 -- 172.21.15.7:0/1067549720 wait complete. 2024-04-25T11:22:18.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_max_caps 2024-04-25T11:22:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.955+0000 7f5a2ec00640 1 Processor -- start 2024-04-25T11:22:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.955+0000 7f5a2ec00640 1 -- start start 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.957+0000 7f5a2ec00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5a280b2570 0x7f5a280b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.957+0000 7f5a2ec00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a280b44a0 0x7f5a280b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.957+0000 7f5a2ec00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a280b4df0 con 0x7f5a2806d680 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.957+0000 7f5a2ec00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f5a280cf620 con 0x7f5a280b44a0 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.957+0000 7f5a2ec00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5a280d3520 con 0x7f5a280b2570 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.957+0000 7f5a27400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a280b44a0 0x7f5a280b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.957+0000 7f5a27400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5a280b44a0 0x7f5a280b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53440/0 (socket says 172.21.15.7:53440) 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.957+0000 7f5a27400640 1 -- 172.21.15.7:0/2759453824 learned_addr learned my addr 172.21.15.7:0/2759453824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.957+0000 7f5a27e00640 1 --2- 172.21.15.7:0/2759453824 >> v2:172.21.15.87:3300/0 conn(0x7f5a280b2570 0x7f5a280b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.958+0000 7f5a27400640 1 -- 172.21.15.7:0/2759453824 >> v2:172.21.15.87:3300/0 conn(0x7f5a280b2570 msgr2=0x7f5a280b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.958+0000 7f5a27400640 1 --2- 172.21.15.7:0/2759453824 >> v2:172.21.15.87:3300/0 conn(0x7f5a280b2570 0x7f5a280b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.958+0000 7f5a27400640 1 -- 172.21.15.7:0/2759453824 >> v1:172.21.15.87:6789/0 conn(0x7f5a2806d680 legacy=0x7f5a2806da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.958+0000 7f5a27400640 1 -- 172.21.15.7:0/2759453824 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a280cf870 con 0x7f5a280b44a0 2024-04-25T11:22:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.959+0000 7f5a27400640 1 --2- 172.21.15.7:0/2759453824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a280b44a0 0x7f5a280b48b0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f5a28073650 tx=0x7f5a1402c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673e90aadc204ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.959+0000 7f5a26a00640 1 -- 172.21.15.7:0/2759453824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14031710 con 0x7f5a280b44a0 2024-04-25T11:22:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.959+0000 7f5a26a00640 1 -- 172.21.15.7:0/2759453824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a140318b0 con 0x7f5a280b44a0 2024-04-25T11:22:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.959+0000 7f5a26a00640 1 -- 172.21.15.7:0/2759453824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14031c80 con 0x7f5a280b44a0 2024-04-25T11:22:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.960+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2759453824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a280b44a0 msgr2=0x7f5a280b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.960+0000 7f5a2ec00640 1 --2- 172.21.15.7:0/2759453824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a280b44a0 0x7f5a280b48b0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f5a28073650 tx=0x7f5a1402c7d0 comp rx=0 tx=0).stop 2024-04-25T11:22:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.960+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2759453824 shutdown_connections 2024-04-25T11:22:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.960+0000 7f5a2ec00640 1 --2- 172.21.15.7:0/2759453824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a280b44a0 0x7f5a280b48b0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.960+0000 7f5a2ec00640 1 --2- 172.21.15.7:0/2759453824 >> v2:172.21.15.87:3300/0 conn(0x7f5a280b2570 0x7f5a280b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.960+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2759453824 >> 172.21.15.7:0/2759453824 conn(0x7f5a280b7670 msgr2=0x7f5a280b7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.960+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2759453824 shutdown_connections 2024-04-25T11:22:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.960+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2759453824 wait complete. 2024-04-25T11:22:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.963+0000 7f5a2ec00640 1 Processor -- start 2024-04-25T11:22:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.963+0000 7f5a2ec00640 1 -- start start 2024-04-25T11:22:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.964+0000 7f5a2ec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a2806d680 0x7f5a281f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.964+0000 7f5a2ec00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a280b2570 0x7f5a28150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.964+0000 7f5a2ec00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a280b44a0 0x7f5a280b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.964+0000 7f5a2ec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280b1f20 con 0x7f5a2806d680 2024-04-25T11:22:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.964+0000 7f5a2ec00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280b40b0 con 0x7f5a280b44a0 2024-04-25T11:22:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.964+0000 7f5a2ec00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5a280b20a0 con 0x7f5a280b2570 2024-04-25T11:22:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.964+0000 7f5a27e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a2806d680 0x7f5a281f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.965+0000 7f5a27e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a2806d680 0x7f5a281f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53456/0 (socket says 172.21.15.7:53456) 2024-04-25T11:22:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.965+0000 7f5a27e00640 1 -- 172.21.15.7:0/2646139186 learned_addr learned my addr 172.21.15.7:0/2646139186 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.965+0000 7f5a27400640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a280b2570 0x7f5a28150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.965+0000 7f5a2cc00640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a280b44a0 0x7f5a280b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.965+0000 7f5a27e00640 1 -- 172.21.15.7:0/2646139186 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a280b2570 msgr2=0x7f5a28150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.965+0000 7f5a27e00640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a280b2570 0x7f5a28150360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.965+0000 7f5a27e00640 1 -- 172.21.15.7:0/2646139186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a280b44a0 msgr2=0x7f5a280b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.965+0000 7f5a27e00640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a280b44a0 0x7f5a280b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.965+0000 7f5a27e00640 1 -- 172.21.15.7:0/2646139186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2814fed0 con 0x7f5a2806d680 2024-04-25T11:22:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.965+0000 7f5a27e00640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a2806d680 0x7f5a281f1fb0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800d5a0 tx=0x7f5a1800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.966+0000 7f5a1fe00640 1 -- 172.21.15.7:0/2646139186 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18019070 con 0x7f5a2806d680 2024-04-25T11:22:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.966+0000 7f5a1fe00640 1 -- 172.21.15.7:0/2646139186 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a18004240 con 0x7f5a2806d680 2024-04-25T11:22:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.966+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2646139186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2809f8e0 con 0x7f5a2806d680 2024-04-25T11:22:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.966+0000 7f5a1fe00640 1 -- 172.21.15.7:0/2646139186 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18003610 con 0x7f5a2806d680 2024-04-25T11:22:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.966+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2646139186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2809fea0 con 0x7f5a2806d680 2024-04-25T11:22:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.967+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2646139186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a28050740 con 0x7f5a2806d680 2024-04-25T11:22:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.968+0000 7f5a1fe00640 1 -- 172.21.15.7:0/2646139186 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f5a1801c070 con 0x7f5a2806d680 2024-04-25T11:22:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.969+0000 7f5a1fe00640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a080744b0 0x7f5a080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.970+0000 7f5a1fe00640 1 -- 172.21.15.7:0/2646139186 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5335+0+0 (secure 0 0 0) 0x7f5a180949c0 con 0x7f5a2806d680 2024-04-25T11:22:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.970+0000 7f5a27400640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a080744b0 0x7f5a080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.971+0000 7f5a27400640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a080744b0 0x7f5a080769c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5a14036000 tx=0x7f5a14031000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:18.974+0000 7f5a1fe00640 1 -- 172.21.15.7:0/2646139186 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f5a1805ebd0 con 0x7f5a2806d680 2024-04-25T11:22:19.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.203+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2646139186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"} v 0) -- 0x7f5a280cecd0 con 0x7f5a2806d680 2024-04-25T11:22:19.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.211+0000 7f5a1fe00640 1 -- 172.21.15.7:0/2646139186 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_max_caps"}]=0 v61) ==== 102+0+0 (secure 0 0 0) 0x7f5a18062ed0 con 0x7f5a2806d680 2024-04-25T11:22:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.214+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2646139186 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a080744b0 msgr2=0x7f5a080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.214+0000 7f5a2ec00640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a080744b0 0x7f5a080769c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5a14036000 tx=0x7f5a14031000 comp rx=0 tx=0).stop 2024-04-25T11:22:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.214+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2646139186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a2806d680 msgr2=0x7f5a281f1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:19.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.214+0000 7f5a2ec00640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a2806d680 0x7f5a281f1fb0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800d5a0 tx=0x7f5a1800da60 comp rx=0 tx=0).stop 2024-04-25T11:22:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.214+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2646139186 shutdown_connections 2024-04-25T11:22:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.214+0000 7f5a2ec00640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f5a080744b0 0x7f5a080769c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.214+0000 7f5a2ec00640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a280b44a0 0x7f5a280b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.214+0000 7f5a2ec00640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a280b2570 0x7f5a28150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.214+0000 7f5a2ec00640 1 --2- 172.21.15.7:0/2646139186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a2806d680 0x7f5a281f1fb0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.214+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2646139186 >> 172.21.15.7:0/2646139186 conn(0x7f5a280b7670 msgr2=0x7f5a2815c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.215+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2646139186 shutdown_connections 2024-04-25T11:22:19.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.215+0000 7f5a2ec00640 1 -- 172.21.15.7:0/2646139186 wait complete. 2024-04-25T11:22:19.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_min_caps_working_set 2024-04-25T11:22:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.528+0000 7fafc5a00640 1 Processor -- start 2024-04-25T11:22:19.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.528+0000 7fafc5a00640 1 -- start start 2024-04-25T11:22:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.530+0000 7fafc5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fafc015dfb0 0x7fafc00b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.530+0000 7fafc5a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc00d0b40 0x7fafc006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.530+0000 7fafc5a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc015e3c0 con 0x7fafc00d0f50 2024-04-25T11:22:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.530+0000 7fafc5a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fafc00b09b0 con 0x7fafc015dfb0 2024-04-25T11:22:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.530+0000 7fafc5a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fafc00cf800 con 0x7fafc00d0b40 2024-04-25T11:22:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.531+0000 7fafbf400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fafc00d0f50 0x7fafc0161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46172/0 (socket says 172.21.15.7:46172) 2024-04-25T11:22:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.531+0000 7fafbf400640 1 -- 172.21.15.7:0/2326634690 learned_addr learned my addr 172.21.15.7:0/2326634690 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.531+0000 7fafbea00640 1 --2- 172.21.15.7:0/2326634690 >> v2:172.21.15.7:3300/0 conn(0x7fafc015dfb0 0x7fafc00b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:19.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.531+0000 7fafb6000640 1 --2- 172.21.15.7:0/2326634690 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc00d0b40 0x7fafc006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.531+0000 7fafbe000640 1 -- 172.21.15.7:0/2326634690 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2545345834 0 0) 0x7fafc015e3c0 con 0x7fafc00d0f50 2024-04-25T11:22:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.531+0000 7fafbe000640 1 -- 172.21.15.7:0/2326634690 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafa4003520 con 0x7fafc00d0f50 2024-04-25T11:22:19.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.531+0000 7fafbea00640 1 -- 172.21.15.7:0/2326634690 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc00d0b40 msgr2=0x7fafc006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.531+0000 7fafbea00640 1 --2- 172.21.15.7:0/2326634690 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc00d0b40 0x7fafc006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.531+0000 7fafbea00640 1 -- 172.21.15.7:0/2326634690 >> v1:172.21.15.7:6789/0 conn(0x7fafc00d0f50 legacy=0x7fafc0161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.531+0000 7fafbea00640 1 -- 172.21.15.7:0/2326634690 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc00cfaa0 con 0x7fafc015dfb0 2024-04-25T11:22:19.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.533+0000 7fafbea00640 1 --2- 172.21.15.7:0/2326634690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafc015dfb0 0x7fafc00b0470 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009d90 tx=0x7fafb002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2325e98a90d50d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.533+0000 7fafbe000640 1 -- 172.21.15.7:0/2326634690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0007740 con 0x7fafc015dfb0 2024-04-25T11:22:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.533+0000 7fafbe000640 1 -- 172.21.15.7:0/2326634690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb00078e0 con 0x7fafc015dfb0 2024-04-25T11:22:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.533+0000 7fafbe000640 1 -- 172.21.15.7:0/2326634690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0007cb0 con 0x7fafc015dfb0 2024-04-25T11:22:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.534+0000 7fafc5a00640 1 -- 172.21.15.7:0/2326634690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafc015dfb0 msgr2=0x7fafc00b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.534+0000 7fafc5a00640 1 --2- 172.21.15.7:0/2326634690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafc015dfb0 0x7fafc00b0470 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009d90 tx=0x7fafb002a660 comp rx=0 tx=0).stop 2024-04-25T11:22:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.534+0000 7fafc5a00640 1 -- 172.21.15.7:0/2326634690 shutdown_connections 2024-04-25T11:22:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.534+0000 7fafc5a00640 1 --2- 172.21.15.7:0/2326634690 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc00d0b40 0x7fafc006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.534+0000 7fafc5a00640 1 --2- 172.21.15.7:0/2326634690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafc015dfb0 0x7fafc00b0470 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.534+0000 7fafc5a00640 1 -- 172.21.15.7:0/2326634690 >> 172.21.15.7:0/2326634690 conn(0x7fafc00742c0 msgr2=0x7fafc006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.534+0000 7fafc5a00640 1 -- 172.21.15.7:0/2326634690 shutdown_connections 2024-04-25T11:22:19.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.534+0000 7fafc5a00640 1 -- 172.21.15.7:0/2326634690 wait complete. 2024-04-25T11:22:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.536+0000 7fafc5a00640 1 Processor -- start 2024-04-25T11:22:19.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.537+0000 7fafc5a00640 1 -- start start 2024-04-25T11:22:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.538+0000 7fafc5a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafc00d0b40 0x7fafc014d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.538+0000 7fafc5a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc00d0f50 0x7fafc00cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.538+0000 7fafc5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafc015dfb0 0x7fafc00b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.538+0000 7fafc5a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fafc00d0340 con 0x7fafc015dfb0 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.538+0000 7fafc5a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fafc00d0060 con 0x7fafc00d0b40 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.538+0000 7fafc5a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fafc00d0620 con 0x7fafc00d0f50 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.538+0000 7fafbf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafc015dfb0 0x7fafc00b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafbf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafc015dfb0 0x7fafc00b5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53478/0 (socket says 172.21.15.7:53478) 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafbf400640 1 -- 172.21.15.7:0/2220410744 learned_addr learned my addr 172.21.15.7:0/2220410744 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafb6000640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc00d0f50 0x7fafc00cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafbea00640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafc00d0b40 0x7fafc014d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafbf400640 1 -- 172.21.15.7:0/2220410744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc00d0f50 msgr2=0x7fafc00cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafbf400640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc00d0f50 0x7fafc00cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafbf400640 1 -- 172.21.15.7:0/2220410744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafc00d0b40 msgr2=0x7fafc014d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafbf400640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafc00d0b40 0x7fafc014d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafbf400640 1 -- 172.21.15.7:0/2220410744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc00b2020 con 0x7fafc015dfb0 2024-04-25T11:22:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafbea00640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafc00d0b40 0x7fafc014d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafb6000640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc00d0f50 0x7fafc00cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.539+0000 7fafbf400640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafc015dfb0 0x7fafc00b5710 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fafb8001eb0 tx=0x7fafb800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:19.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.540+0000 7fafb7400640 1 -- 172.21.15.7:0/2220410744 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8017070 con 0x7fafc015dfb0 2024-04-25T11:22:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.540+0000 7fafb7400640 1 -- 172.21.15.7:0/2220410744 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb8005ce0 con 0x7fafc015dfb0 2024-04-25T11:22:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.540+0000 7fafb7400640 1 -- 172.21.15.7:0/2220410744 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80072d0 con 0x7fafc015dfb0 2024-04-25T11:22:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.540+0000 7fafc5a00640 1 -- 172.21.15.7:0/2220410744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc00b2300 con 0x7fafc015dfb0 2024-04-25T11:22:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.540+0000 7fafc5a00640 1 -- 172.21.15.7:0/2220410744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc015cb30 con 0x7fafc015dfb0 2024-04-25T11:22:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.541+0000 7fafc5a00640 1 -- 172.21.15.7:0/2220410744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc00b1d90 con 0x7fafc015dfb0 2024-04-25T11:22:19.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.542+0000 7fafb7400640 1 -- 172.21.15.7:0/2220410744 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fafb801a070 con 0x7fafc015dfb0 2024-04-25T11:22:19.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.543+0000 7fafb7400640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf9c0744b0 0x7faf9c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:19.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.544+0000 7fafb7400640 1 -- 172.21.15.7:0/2220410744 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5335+0+0 (secure 0 0 0) 0x7fafb80943f0 con 0x7fafc015dfb0 2024-04-25T11:22:19.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.544+0000 7fafbea00640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf9c0744b0 0x7faf9c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:19.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.544+0000 7fafbea00640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf9c0744b0 0x7faf9c0769c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fafb0034000 tx=0x7fafb0033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:19.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.549+0000 7fafb7400640 1 -- 172.21.15.7:0/2220410744 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fafb805e600 con 0x7fafc015dfb0 2024-04-25T11:22:19.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.754+0000 7fafc5a00640 1 -- 172.21.15.7:0/2220410744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"} v 0) -- 0x7fafc0050740 con 0x7fafc015dfb0 2024-04-25T11:22:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.760+0000 7fafb7400640 1 -- 172.21.15.7:0/2220410744 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_min_caps_working_set"}]=0 v62) ==== 107+0+0 (secure 0 0 0) 0x7fafb8062900 con 0x7fafc015dfb0 2024-04-25T11:22:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.763+0000 7fafc5a00640 1 -- 172.21.15.7:0/2220410744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf9c0744b0 msgr2=0x7faf9c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.763+0000 7fafc5a00640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf9c0744b0 0x7faf9c0769c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fafb0034000 tx=0x7fafb0033040 comp rx=0 tx=0).stop 2024-04-25T11:22:19.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.763+0000 7fafc5a00640 1 -- 172.21.15.7:0/2220410744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafc015dfb0 msgr2=0x7fafc00b5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:19.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.763+0000 7fafc5a00640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafc015dfb0 0x7fafc00b5710 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fafb8001eb0 tx=0x7fafb800c820 comp rx=0 tx=0).stop 2024-04-25T11:22:19.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.764+0000 7fafc5a00640 1 -- 172.21.15.7:0/2220410744 shutdown_connections 2024-04-25T11:22:19.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.764+0000 7fafc5a00640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7faf9c0744b0 0x7faf9c0769c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.764+0000 7fafc5a00640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fafc015dfb0 0x7fafc00b5710 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.764+0000 7fafc5a00640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc00d0f50 0x7fafc00cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.764+0000 7fafc5a00640 1 --2- 172.21.15.7:0/2220410744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafc00d0b40 0x7fafc014d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:19.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.764+0000 7fafc5a00640 1 -- 172.21.15.7:0/2220410744 >> 172.21.15.7:0/2220410744 conn(0x7fafc00742c0 msgr2=0x7fafc00ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:19.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.764+0000 7fafc5a00640 1 -- 172.21.15.7:0/2220410744 shutdown_connections 2024-04-25T11:22:19.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:19.764+0000 7fafc5a00640 1 -- 172.21.15.7:0/2220410744 wait complete. 2024-04-25T11:22:19.788 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:22:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.074+0000 7fad76c00640 1 Processor -- start 2024-04-25T11:22:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.075+0000 7fad76c00640 1 -- start start 2024-04-25T11:22:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.078+0000 7fad76c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fad700cfa80 0x7fad700d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.078+0000 7fad76c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad700d0800 0x7fad7006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.078+0000 7fad76c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad700d0c10 con 0x7fad70070630 2024-04-25T11:22:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.078+0000 7fad76c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fad700b0be0 con 0x7fad700cfa80 2024-04-25T11:22:20.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.078+0000 7fad76c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad700cfe90 con 0x7fad700d0800 2024-04-25T11:22:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.078+0000 7fad74e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fad700cfa80 0x7fad700d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.078+0000 7fad75800640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fad70070630 0x7fad7015ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.7:55702/0 (socket says 172.21.15.7:55702) 2024-04-25T11:22:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.079+0000 7fad75800640 1 -- 172.21.15.7:0/1461745010 learned_addr learned my addr 172.21.15.7:0/1461745010 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.079+0000 7fad76200640 1 --2- 172.21.15.7:0/1461745010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad700d0800 0x7fad7006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.080+0000 7fad76200640 1 -- 172.21.15.7:0/1461745010 >> v2:172.21.15.87:3300/0 conn(0x7fad700cfa80 msgr2=0x7fad700d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.080+0000 7fad76200640 1 --2- 172.21.15.7:0/1461745010 >> v2:172.21.15.87:3300/0 conn(0x7fad700cfa80 0x7fad700d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.080+0000 7fad76200640 1 -- 172.21.15.7:0/1461745010 >> v1:172.21.15.87:6789/0 conn(0x7fad70070630 legacy=0x7fad7015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.080+0000 7fad76200640 1 -- 172.21.15.7:0/1461745010 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad700ce9e0 con 0x7fad700d0800 2024-04-25T11:22:20.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.080+0000 7fad74e00640 1 --2- 172.21.15.7:0/1461745010 >> v2:172.21.15.87:3300/0 conn(0x7fad700cfa80 0x7fad700d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:22:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.081+0000 7fad76200640 1 --2- 172.21.15.7:0/1461745010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad700d0800 0x7fad7006eea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fad680097a0 tx=0x7fad6802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c5d5386f4c4a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.081+0000 7fad6fe00640 1 -- 172.21.15.7:0/1461745010 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68037070 con 0x7fad700d0800 2024-04-25T11:22:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.081+0000 7fad6fe00640 1 -- 172.21.15.7:0/1461745010 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad68032440 con 0x7fad700d0800 2024-04-25T11:22:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.081+0000 7fad6fe00640 1 -- 172.21.15.7:0/1461745010 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68032810 con 0x7fad700d0800 2024-04-25T11:22:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.082+0000 7fad76c00640 1 -- 172.21.15.7:0/1461745010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad700d0800 msgr2=0x7fad7006eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.082+0000 7fad76c00640 1 --2- 172.21.15.7:0/1461745010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad700d0800 0x7fad7006eea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fad680097a0 tx=0x7fad6802a180 comp rx=0 tx=0).stop 2024-04-25T11:22:20.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.082+0000 7fad76c00640 1 -- 172.21.15.7:0/1461745010 shutdown_connections 2024-04-25T11:22:20.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.082+0000 7fad76c00640 1 --2- 172.21.15.7:0/1461745010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad700d0800 0x7fad7006eea0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.082+0000 7fad76c00640 1 --2- 172.21.15.7:0/1461745010 >> v2:172.21.15.87:3300/0 conn(0x7fad700cfa80 0x7fad700d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.082+0000 7fad76c00640 1 -- 172.21.15.7:0/1461745010 >> 172.21.15.7:0/1461745010 conn(0x7fad70074140 msgr2=0x7fad7006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:20.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.083+0000 7fad76c00640 1 -- 172.21.15.7:0/1461745010 shutdown_connections 2024-04-25T11:22:20.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.083+0000 7fad76c00640 1 -- 172.21.15.7:0/1461745010 wait complete. 2024-04-25T11:22:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.085+0000 7fad76c00640 1 Processor -- start 2024-04-25T11:22:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.085+0000 7fad76c00640 1 -- start start 2024-04-25T11:22:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.087+0000 7fad76c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad70070630 0x7fad7014af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.087+0000 7fad76c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad700cfa80 0x7fad700aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.087+0000 7fad76c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad700d0800 0x7fad700ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.087+0000 7fad76c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad700cf280 con 0x7fad70070630 2024-04-25T11:22:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.087+0000 7fad76c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fad700cefa0 con 0x7fad700d0800 2024-04-25T11:22:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.087+0000 7fad76c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad700cf560 con 0x7fad700cfa80 2024-04-25T11:22:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.087+0000 7fad75800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad70070630 0x7fad7014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.087+0000 7fad75800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad70070630 0x7fad7014af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47866/0 (socket says 172.21.15.7:47866) 2024-04-25T11:22:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.087+0000 7fad75800640 1 -- 172.21.15.7:0/4158330477 learned_addr learned my addr 172.21.15.7:0/4158330477 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.088+0000 7fad76200640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad700d0800 0x7fad700ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.088+0000 7fad75800640 1 -- 172.21.15.7:0/4158330477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad700cfa80 msgr2=0x7fad700aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.088+0000 7fad75800640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad700cfa80 0x7fad700aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.088+0000 7fad75800640 1 -- 172.21.15.7:0/4158330477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad700d0800 msgr2=0x7fad700ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.088+0000 7fad75800640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad700d0800 0x7fad700ab360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.088+0000 7fad75800640 1 -- 172.21.15.7:0/4158330477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7014b480 con 0x7fad70070630 2024-04-25T11:22:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.088+0000 7fad75800640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad70070630 0x7fad7014af40 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fad6400caa0 tx=0x7fad6400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.088+0000 7fad6d600640 1 -- 172.21.15.7:0/4158330477 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64017070 con 0x7fad70070630 2024-04-25T11:22:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.089+0000 7fad6d600640 1 -- 172.21.15.7:0/4158330477 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6400ac00 con 0x7fad70070630 2024-04-25T11:22:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.089+0000 7fad76c00640 1 -- 172.21.15.7:0/4158330477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad701f3ff0 con 0x7fad70070630 2024-04-25T11:22:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.089+0000 7fad6d600640 1 -- 172.21.15.7:0/4158330477 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad64012400 con 0x7fad70070630 2024-04-25T11:22:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.089+0000 7fad76c00640 1 -- 172.21.15.7:0/4158330477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad700aff50 con 0x7fad70070630 2024-04-25T11:22:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.090+0000 7fad76c00640 1 -- 172.21.15.7:0/4158330477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad70050740 con 0x7fad70070630 2024-04-25T11:22:20.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.091+0000 7fad6d600640 1 -- 172.21.15.7:0/4158330477 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fad6401a070 con 0x7fad70070630 2024-04-25T11:22:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.093+0000 7fad6d600640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad500744b0 0x7fad500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.093+0000 7fad6d600640 1 -- 172.21.15.7:0/4158330477 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5335+0+0 (secure 0 0 0) 0x7fad64094e20 con 0x7fad70070630 2024-04-25T11:22:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.093+0000 7fad74e00640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad500744b0 0x7fad500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.094+0000 7fad74e00640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad500744b0 0x7fad500769c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fad5c006e80 tx=0x7fad5c006dd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.098+0000 7fad6d600640 1 -- 172.21.15.7:0/4158330477 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fad700d0c10 con 0x7fad70070630 2024-04-25T11:22:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.304+0000 7fad76c00640 1 -- 172.21.15.7:0/4158330477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fad70070dd0 con 0x7fad70070630 2024-04-25T11:22:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.310+0000 7fad6d600640 1 -- 172.21.15.7:0/4158330477 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v63) ==== 104+0+0 (secure 0 0 0) 0x7fad6405f720 con 0x7fad70070630 2024-04-25T11:22:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 -- 172.21.15.7:0/4158330477 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad500744b0 msgr2=0x7fad500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad500744b0 0x7fad500769c0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fad5c006e80 tx=0x7fad5c006dd0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 -- 172.21.15.7:0/4158330477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad70070630 msgr2=0x7fad7014af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad70070630 0x7fad7014af40 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fad6400caa0 tx=0x7fad6400cf60 comp rx=0 tx=0).stop 2024-04-25T11:22:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 -- 172.21.15.7:0/4158330477 shutdown_connections 2024-04-25T11:22:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fad500744b0 0x7fad500769c0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad700d0800 0x7fad700ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad700cfa80 0x7fad700aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 --2- 172.21.15.7:0/4158330477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad70070630 0x7fad7014af40 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 -- 172.21.15.7:0/4158330477 >> 172.21.15.7:0/4158330477 conn(0x7fad70074140 msgr2=0x7fad700b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 -- 172.21.15.7:0/4158330477 shutdown_connections 2024-04-25T11:22:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.313+0000 7fad76c00640 1 -- 172.21.15.7:0/4158330477 wait complete. 2024-04-25T11:22:20.337 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_recall_warning_threshold 2024-04-25T11:22:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.615+0000 7f196f200640 1 Processor -- start 2024-04-25T11:22:20.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.615+0000 7f196f200640 1 -- start start 2024-04-25T11:22:20.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.617+0000 7f196f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19700d0480 0x7f19700d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.617+0000 7f196f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19700ce570 con 0x7f19701583c0 2024-04-25T11:22:20.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.617+0000 7f196f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19700ce850 con 0x7f197006eea0 2024-04-25T11:22:20.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.617+0000 7f196f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f197015f190 con 0x7f19700d0480 2024-04-25T11:22:20.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.617+0000 7f196de00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f19701583c0 0x7f19700cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41118/0 (socket says 172.21.15.7:41118) 2024-04-25T11:22:20.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.617+0000 7f196de00640 1 -- 172.21.15.7:0/2870135896 learned_addr learned my addr 172.21.15.7:0/2870135896 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.618+0000 7f196d400640 1 --2- 172.21.15.7:0/2870135896 >> v2:172.21.15.87:3300/0 conn(0x7f19700d0480 0x7f19700d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.618+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131601032 0 0) 0x7f19700ce570 con 0x7f19701583c0 2024-04-25T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.618+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1954003520 con 0x7f19701583c0 2024-04-25T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.618+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2928993553 0 0) 0x7f1954003520 con 0x7f19701583c0 2024-04-25T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.618+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 >> v2:172.21.15.87:3300/0 conn(0x7f19700d0480 msgr2=0x7f19700d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.618+0000 7f196ca00640 1 --2- 172.21.15.7:0/2870135896 >> v2:172.21.15.87:3300/0 conn(0x7f19700d0480 0x7f19700d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.618+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 >> v1:172.21.15.87:6789/0 conn(0x7f197006eea0 legacy=0x7f19700b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.618+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19700ceb30 con 0x7f19701583c0 2024-04-25T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.618+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f1960001670 con 0x7f19701583c0 2024-04-25T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.618+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 >> v1:172.21.15.7:6789/0 conn(0x7f19701583c0 legacy=0x7f19700cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196ca00640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 0x7f1954003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196ca00640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1954004800 0x7f1954004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196ca00640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19540052a0 0x7f195400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f197015f190 con 0x7f1954003a00 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f195400c230 con 0x7f19540052a0 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f195400c430 con 0x7f1954004800 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196de00640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 0x7f1954003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196d400640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1954004800 0x7f1954004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196de00640 1 -- 172.21.15.7:0/2870135896 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1954004800 msgr2=0x7f1954004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196de00640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1954004800 0x7f1954004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196e800640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19540052a0 0x7f195400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196de00640 1 -- 172.21.15.7:0/2870135896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19540052a0 msgr2=0x7f195400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.619+0000 7f196de00640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19540052a0 0x7f195400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.620+0000 7f196de00640 1 -- 172.21.15.7:0/2870135896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1954003520 con 0x7f1954003a00 2024-04-25T11:22:20.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.620+0000 7f196e800640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19540052a0 0x7f195400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:20.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.620+0000 7f196e800640 1 -- 172.21.15.7:0/2870135896 reap_dead start 2024-04-25T11:22:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.621+0000 7f196de00640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 0x7f1954003e70 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f19700b43c0 tx=0x7f1954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22033820faa7c2ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.621+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960007c40 con 0x7f1954003a00 2024-04-25T11:22:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.621+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1960005af0 con 0x7f1954003a00 2024-04-25T11:22:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.621+0000 7f196ca00640 1 -- 172.21.15.7:0/2870135896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960007070 con 0x7f1954003a00 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.621+0000 7f196f200640 1 -- 172.21.15.7:0/2870135896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 msgr2=0x7f1954003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.621+0000 7f196f200640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 0x7f1954003e70 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f19700b43c0 tx=0x7f1954005830 comp rx=0 tx=0).stop 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.622+0000 7f196f200640 1 -- 172.21.15.7:0/2870135896 shutdown_connections 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.622+0000 7f196f200640 1 --2- 172.21.15.7:0/2870135896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 0x7f1954003e70 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.622+0000 7f196f200640 1 -- 172.21.15.7:0/2870135896 >> 172.21.15.7:0/2870135896 conn(0x7f19700740f0 msgr2=0x7f197006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.622+0000 7f196f200640 1 -- 172.21.15.7:0/2870135896 shutdown_connections 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.622+0000 7f196f200640 1 -- 172.21.15.7:0/2870135896 wait complete. 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.624+0000 7f196f200640 1 Processor -- start 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.624+0000 7f196f200640 1 -- start start 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.626+0000 7f196f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 0x7f1970146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.626+0000 7f196f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1970157200 0x7f1970157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.626+0000 7f196f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1970148d50 0x7f1970149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.626+0000 7f196f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f19700d0890 con 0x7f1954003a00 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.626+0000 7f196f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f19700cf0f0 con 0x7f1970157200 2024-04-25T11:22:20.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.626+0000 7f196f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f19700d0a10 con 0x7f1970148d50 2024-04-25T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.626+0000 7f196de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 0x7f1970146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.626+0000 7f196de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 0x7f1970146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47880/0 (socket says 172.21.15.7:47880) 2024-04-25T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.626+0000 7f196de00640 1 -- 172.21.15.7:0/101539754 learned_addr learned my addr 172.21.15.7:0/101539754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.627+0000 7f196d400640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1970157200 0x7f1970157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.627+0000 7f196e800640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1970148d50 0x7f1970149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.627+0000 7f196de00640 1 -- 172.21.15.7:0/101539754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1970148d50 msgr2=0x7f1970149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.627+0000 7f196de00640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1970148d50 0x7f1970149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.627+0000 7f196de00640 1 -- 172.21.15.7:0/101539754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1970157200 msgr2=0x7f1970157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.627+0000 7f196de00640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1970157200 0x7f1970157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.627+0000 7f196de00640 1 -- 172.21.15.7:0/101539754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19701496a0 con 0x7f1954003a00 2024-04-25T11:22:20.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.627+0000 7f196de00640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 0x7f1970146e20 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f196002acf0 tx=0x7f19700ce740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:20.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.630+0000 7f1966000640 1 -- 172.21.15.7:0/101539754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960037040 con 0x7f1954003a00 2024-04-25T11:22:20.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.630+0000 7f1966000640 1 -- 172.21.15.7:0/101539754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1960041070 con 0x7f1954003a00 2024-04-25T11:22:20.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.630+0000 7f1966000640 1 -- 172.21.15.7:0/101539754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960001670 con 0x7f1954003a00 2024-04-25T11:22:20.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.630+0000 7f196f200640 1 -- 172.21.15.7:0/101539754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1970155ae0 con 0x7f1954003a00 2024-04-25T11:22:20.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.630+0000 7f196f200640 1 -- 172.21.15.7:0/101539754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19701560a0 con 0x7f1954003a00 2024-04-25T11:22:20.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.631+0000 7f196f200640 1 -- 172.21.15.7:0/101539754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1970050740 con 0x7f1954003a00 2024-04-25T11:22:20.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.633+0000 7f1966000640 1 -- 172.21.15.7:0/101539754 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f19600049e0 con 0x7f1954003a00 2024-04-25T11:22:20.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.634+0000 7f1966000640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19500744b0 0x7f19500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:20.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.634+0000 7f1966000640 1 -- 172.21.15.7:0/101539754 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5335+0+0 (secure 0 0 0) 0x7f19600b5160 con 0x7f1954003a00 2024-04-25T11:22:20.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.634+0000 7f196d400640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19500744b0 0x7f19500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:20.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.635+0000 7f196d400640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19500744b0 0x7f19500769c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f195c009670 tx=0x7f195c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:20.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.639+0000 7f1966000640 1 -- 172.21.15.7:0/101539754 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f19701496a0 con 0x7f1954003a00 2024-04-25T11:22:20.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.835+0000 7f196f200640 1 -- 172.21.15.7:0/101539754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"} v 0) -- 0x7f1970070dd0 con 0x7f1954003a00 2024-04-25T11:22:20.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.843+0000 7f1966000640 1 -- 172.21.15.7:0/101539754 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_recall_warning_threshold"}]=0 v64) ==== 111+0+0 (secure 0 0 0) 0x7f196007f9c0 con 0x7f1954003a00 2024-04-25T11:22:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.846+0000 7f196f200640 1 -- 172.21.15.7:0/101539754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19500744b0 msgr2=0x7f19500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.846+0000 7f196f200640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19500744b0 0x7f19500769c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f195c009670 tx=0x7f195c00b040 comp rx=0 tx=0).stop 2024-04-25T11:22:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.846+0000 7f196f200640 1 -- 172.21.15.7:0/101539754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 msgr2=0x7f1970146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.846+0000 7f196f200640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 0x7f1970146e20 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f196002acf0 tx=0x7f19700ce740 comp rx=0 tx=0).stop 2024-04-25T11:22:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.846+0000 7f196f200640 1 -- 172.21.15.7:0/101539754 shutdown_connections 2024-04-25T11:22:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.846+0000 7f196f200640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f19500744b0 0x7f19500769c0 secure :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f195c009670 tx=0x7f195c00b040 comp rx=0 tx=0).stop 2024-04-25T11:22:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.846+0000 7f196f200640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1970148d50 0x7f1970149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.846+0000 7f196f200640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1970157200 0x7f1970157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.846+0000 7f196f200640 1 --2- 172.21.15.7:0/101539754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1954003a00 0x7f1970146e20 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.846+0000 7f196f200640 1 -- 172.21.15.7:0/101539754 >> 172.21.15.7:0/101539754 conn(0x7f19700740f0 msgr2=0x7f197015e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.847+0000 7f196f200640 1 -- 172.21.15.7:0/101539754 shutdown_connections 2024-04-25T11:22:20.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:20.847+0000 7f196f200640 1 -- 172.21.15.7:0/101539754 wait complete. 2024-04-25T11:22:20.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_cache_memory_limit 2024-04-25T11:22:21.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.153+0000 7f187b800640 1 Processor -- start 2024-04-25T11:22:21.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.153+0000 7f187b800640 1 -- start start 2024-04-25T11:22:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.155+0000 7f187b800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18740b0470 0x7f187406f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.155+0000 7f187b800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f187406f8e0 0x7f18740aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.155+0000 7f187b800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f187415e3c0 con 0x7f187415dfb0 2024-04-25T11:22:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.155+0000 7f187b800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f18740cf800 con 0x7f18740b0470 2024-04-25T11:22:21.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.155+0000 7f187b800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f18740d1250 con 0x7f187406f8e0 2024-04-25T11:22:21.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.155+0000 7f1873e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18740b0470 0x7f187406f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.155+0000 7f1873e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18740b0470 0x7f187406f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47886/0 (socket says 172.21.15.7:47886) 2024-04-25T11:22:21.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.155+0000 7f1873e00640 1 -- 172.21.15.7:0/1734231441 learned_addr learned my addr 172.21.15.7:0/1734231441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:21.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.156+0000 7f1879800640 1 --2- 172.21.15.7:0/1734231441 >> v2:172.21.15.87:3300/0 conn(0x7f187406f8e0 0x7f18740aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.156+0000 7f1873e00640 1 -- 172.21.15.7:0/1734231441 >> v2:172.21.15.87:3300/0 conn(0x7f187406f8e0 msgr2=0x7f18740aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:21.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.156+0000 7f1873e00640 1 --2- 172.21.15.7:0/1734231441 >> v2:172.21.15.87:3300/0 conn(0x7f187406f8e0 0x7f18740aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.156+0000 7f1873e00640 1 -- 172.21.15.7:0/1734231441 >> v1:172.21.15.87:6789/0 conn(0x7f187415dfb0 legacy=0x7f18740d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:21.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.156+0000 7f1873e00640 1 -- 172.21.15.7:0/1734231441 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18740cfaa0 con 0x7f18740b0470 2024-04-25T11:22:21.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.157+0000 7f1873e00640 1 --2- 172.21.15.7:0/1734231441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18740b0470 0x7f187406f3a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f187409fb50 tx=0x7f185c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79093738b3ede5e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:21.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.157+0000 7f1873400640 1 -- 172.21.15.7:0/1734231441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c037070 con 0x7f18740b0470 2024-04-25T11:22:21.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.157+0000 7f1873400640 1 -- 172.21.15.7:0/1734231441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f185c032440 con 0x7f18740b0470 2024-04-25T11:22:21.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.157+0000 7f1873400640 1 -- 172.21.15.7:0/1734231441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c032810 con 0x7f18740b0470 2024-04-25T11:22:21.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.158+0000 7f187b800640 1 -- 172.21.15.7:0/1734231441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18740b0470 msgr2=0x7f187406f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:21.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.158+0000 7f187b800640 1 --2- 172.21.15.7:0/1734231441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18740b0470 0x7f187406f3a0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f187409fb50 tx=0x7f185c02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.158+0000 7f187b800640 1 -- 172.21.15.7:0/1734231441 shutdown_connections 2024-04-25T11:22:21.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.158+0000 7f187b800640 1 --2- 172.21.15.7:0/1734231441 >> v2:172.21.15.87:3300/0 conn(0x7f187406f8e0 0x7f18740aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.158+0000 7f187b800640 1 --2- 172.21.15.7:0/1734231441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18740b0470 0x7f187406f3a0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.158+0000 7f187b800640 1 -- 172.21.15.7:0/1734231441 >> 172.21.15.7:0/1734231441 conn(0x7f18740742c0 msgr2=0x7f187406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:21.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.159+0000 7f187b800640 1 -- 172.21.15.7:0/1734231441 shutdown_connections 2024-04-25T11:22:21.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.159+0000 7f187b800640 1 -- 172.21.15.7:0/1734231441 wait complete. 2024-04-25T11:22:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.161+0000 7f187b800640 1 Processor -- start 2024-04-25T11:22:21.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.161+0000 7f187b800640 1 -- start start 2024-04-25T11:22:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.162+0000 7f187b800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f187406f8e0 0x7f1874150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f187b800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18740b0470 0x7f18740b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f187b800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f187415dfb0 0x7f1874160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f187b800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18740d0340 con 0x7f18740b0470 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f187b800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f18740d0060 con 0x7f187415dfb0 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f187b800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18740d0620 con 0x7f187406f8e0 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f1873e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18740b0470 0x7f18740b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f1873e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18740b0470 0x7f18740b45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47892/0 (socket says 172.21.15.7:47892) 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f1873e00640 1 -- 172.21.15.7:0/914620199 learned_addr learned my addr 172.21.15.7:0/914620199 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f1873e00640 1 -- 172.21.15.7:0/914620199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f187406f8e0 msgr2=0x7f1874150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f1879800640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f187415dfb0 0x7f1874160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f1878e00640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f187406f8e0 0x7f1874150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f1873e00640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f187406f8e0 0x7f1874150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f1873e00640 1 -- 172.21.15.7:0/914620199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f187415dfb0 msgr2=0x7f1874160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f1873e00640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f187415dfb0 0x7f1874160d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.163+0000 7f1873e00640 1 -- 172.21.15.7:0/914620199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1874160880 con 0x7f18740b0470 2024-04-25T11:22:21.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.164+0000 7f1873e00640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18740b0470 0x7f18740b45e0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f185c000c00 tx=0x7f185c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:21.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.164+0000 7f1870c00640 1 -- 172.21.15.7:0/914620199 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c041020 con 0x7f18740b0470 2024-04-25T11:22:21.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.164+0000 7f1870c00640 1 -- 172.21.15.7:0/914620199 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f185c0307a0 con 0x7f18740b0470 2024-04-25T11:22:21.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.164+0000 7f1870c00640 1 -- 172.21.15.7:0/914620199 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c037040 con 0x7f18740b0470 2024-04-25T11:22:21.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.164+0000 7f187b800640 1 -- 172.21.15.7:0/914620199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18740cd4a0 con 0x7f18740b0470 2024-04-25T11:22:21.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.164+0000 7f187b800640 1 -- 172.21.15.7:0/914620199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18740cda60 con 0x7f18740b0470 2024-04-25T11:22:21.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.165+0000 7f187b800640 1 -- 172.21.15.7:0/914620199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18740b0f90 con 0x7f18740b0470 2024-04-25T11:22:21.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.167+0000 7f1870c00640 1 -- 172.21.15.7:0/914620199 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f185c044020 con 0x7f18740b0470 2024-04-25T11:22:21.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.168+0000 7f1870c00640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1850074500 0x7f1850076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.169+0000 7f1870c00640 1 -- 172.21.15.7:0/914620199 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5335+0+0 (secure 0 0 0) 0x7f185c0b5990 con 0x7f18740b0470 2024-04-25T11:22:21.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.169+0000 7f1878e00640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1850074500 0x7f1850076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.169+0000 7f1878e00640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1850074500 0x7f1850076a10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1874072df0 tx=0x7f1864007240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:21.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.174+0000 7f1870c00640 1 -- 172.21.15.7:0/914620199 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18740b0f90 con 0x7f18740b0470 2024-04-25T11:22:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.378+0000 7f187b800640 1 -- 172.21.15.7:0/914620199 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"} v 0) -- 0x7f1874050740 con 0x7f18740b0470 2024-04-25T11:22:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.385+0000 7f1870c00640 1 -- 172.21.15.7:0/914620199 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_cache_memory_limit"}]=0 v65) ==== 105+0+0 (secure 0 0 0) 0x7f185c04b030 con 0x7f18740b0470 2024-04-25T11:22:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 -- 172.21.15.7:0/914620199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1850074500 msgr2=0x7f1850076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1850074500 0x7f1850076a10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1874072df0 tx=0x7f1864007240 comp rx=0 tx=0).stop 2024-04-25T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 -- 172.21.15.7:0/914620199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18740b0470 msgr2=0x7f18740b45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18740b0470 0x7f18740b45e0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f185c000c00 tx=0x7f185c0046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 -- 172.21.15.7:0/914620199 shutdown_connections 2024-04-25T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1850074500 0x7f1850076a10 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f187415dfb0 0x7f1874160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18740b0470 0x7f18740b45e0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 --2- 172.21.15.7:0/914620199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f187406f8e0 0x7f1874150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 -- 172.21.15.7:0/914620199 >> 172.21.15.7:0/914620199 conn(0x7f18740742c0 msgr2=0x7f18740b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 -- 172.21.15.7:0/914620199 shutdown_connections 2024-04-25T11:22:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.388+0000 7f187b800640 1 -- 172.21.15.7:0/914620199 wait complete. 2024-04-25T11:22:21.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root' 2024-04-25T11:22:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.688+0000 7f915c000640 1 Processor -- start 2024-04-25T11:22:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.688+0000 7f915c000640 1 -- start start 2024-04-25T11:22:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.690+0000 7f915c000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f915406d580 0x7f915406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.690+0000 7f915c000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f915406f390 0x7f915406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.690+0000 7f915c000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91540b55a0 con 0x7f91540b5190 2024-04-25T11:22:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.690+0000 7f915c000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f915406ded0 con 0x7f915406f390 2024-04-25T11:22:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.691+0000 7f915c000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f915415d560 con 0x7f915406d580 2024-04-25T11:22:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.691+0000 7f9159600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91540b5190 0x7f91540d0ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41128/0 (socket says 172.21.15.7:41128) 2024-04-25T11:22:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.691+0000 7f9159600640 1 -- 172.21.15.7:0/3078106630 learned_addr learned my addr 172.21.15.7:0/3078106630 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.691+0000 7f915a000640 1 --2- 172.21.15.7:0/3078106630 >> v2:172.21.15.7:3300/0 conn(0x7f915406f390 0x7f915406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.691+0000 7f9158c00640 1 --2- 172.21.15.7:0/3078106630 >> v2:172.21.15.87:3300/0 conn(0x7f915406d580 0x7f915406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.691+0000 7f9153e00640 1 -- 172.21.15.7:0/3078106630 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3824140806 0 0) 0x7f91540b55a0 con 0x7f91540b5190 2024-04-25T11:22:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.691+0000 7f9153e00640 1 -- 172.21.15.7:0/3078106630 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f913c003520 con 0x7f91540b5190 2024-04-25T11:22:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.692+0000 7f915a000640 1 -- 172.21.15.7:0/3078106630 >> v2:172.21.15.87:3300/0 conn(0x7f915406d580 msgr2=0x7f915406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.692+0000 7f915a000640 1 --2- 172.21.15.7:0/3078106630 >> v2:172.21.15.87:3300/0 conn(0x7f915406d580 0x7f915406d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.692+0000 7f915a000640 1 -- 172.21.15.7:0/3078106630 >> v1:172.21.15.7:6789/0 conn(0x7f91540b5190 legacy=0x7f91540d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:21.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.692+0000 7f915a000640 1 -- 172.21.15.7:0/3078106630 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915415d820 con 0x7f915406f390 2024-04-25T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.693+0000 7f915a000640 1 --2- 172.21.15.7:0/3078106630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f915406f390 0x7f915406f7a0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f91540cf5c0 tx=0x7f914c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442b61c46792db2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.693+0000 7f9153e00640 1 -- 172.21.15.7:0/3078106630 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c037070 con 0x7f915406f390 2024-04-25T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.693+0000 7f9153e00640 1 -- 172.21.15.7:0/3078106630 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f914c032440 con 0x7f915406f390 2024-04-25T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.693+0000 7f9153e00640 1 -- 172.21.15.7:0/3078106630 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914c02ae60 con 0x7f915406f390 2024-04-25T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.694+0000 7f915c000640 1 -- 172.21.15.7:0/3078106630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f915406f390 msgr2=0x7f915406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.694+0000 7f915c000640 1 --2- 172.21.15.7:0/3078106630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f915406f390 0x7f915406f7a0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f91540cf5c0 tx=0x7f914c02a110 comp rx=0 tx=0).stop 2024-04-25T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.694+0000 7f915c000640 1 -- 172.21.15.7:0/3078106630 shutdown_connections 2024-04-25T11:22:21.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.694+0000 7f915c000640 1 --2- 172.21.15.7:0/3078106630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f915406f390 0x7f915406f7a0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.694+0000 7f915c000640 1 --2- 172.21.15.7:0/3078106630 >> v2:172.21.15.87:3300/0 conn(0x7f915406d580 0x7f915406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.694+0000 7f915c000640 1 -- 172.21.15.7:0/3078106630 >> 172.21.15.7:0/3078106630 conn(0x7f91540732b0 msgr2=0x7f91540736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.694+0000 7f915c000640 1 -- 172.21.15.7:0/3078106630 shutdown_connections 2024-04-25T11:22:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.694+0000 7f915c000640 1 -- 172.21.15.7:0/3078106630 wait complete. 2024-04-25T11:22:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.696+0000 7f915c000640 1 Processor -- start 2024-04-25T11:22:21.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.696+0000 7f915c000640 1 -- start start 2024-04-25T11:22:21.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.698+0000 7f915c000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f915406d580 0x7f915414ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.698+0000 7f915c000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f915406f390 0x7f9154153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.698+0000 7f915c000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91540b5190 0x7f9154152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.698+0000 7f915c000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f915415e0c0 con 0x7f915406f390 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.698+0000 7f915c000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f915415dde0 con 0x7f91540b5190 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.698+0000 7f915c000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f915409f8e0 con 0x7f915406d580 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.698+0000 7f9158c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f915406f390 0x7f9154153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.698+0000 7f9158c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f915406f390 0x7f9154153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47914/0 (socket says 172.21.15.7:47914) 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.698+0000 7f9158c00640 1 -- 172.21.15.7:0/1288832473 learned_addr learned my addr 172.21.15.7:0/1288832473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f915a000640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91540b5190 0x7f9154152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f9159600640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f915406d580 0x7f915414ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f9158c00640 1 -- 172.21.15.7:0/1288832473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f915406d580 msgr2=0x7f915414ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f9158c00640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f915406d580 0x7f915414ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f9158c00640 1 -- 172.21.15.7:0/1288832473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91540b5190 msgr2=0x7f9154152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f9158c00640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91540b5190 0x7f9154152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f9158c00640 1 -- 172.21.15.7:0/1288832473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915406fe80 con 0x7f915406f390 2024-04-25T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f915a000640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91540b5190 0x7f9154152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f9158c00640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f915406f390 0x7f9154153550 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f914000e8a0 tx=0x7f914000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f9151600640 1 -- 172.21.15.7:0/1288832473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9140017070 con 0x7f915406f390 2024-04-25T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f9151600640 1 -- 172.21.15.7:0/1288832473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f914000c940 con 0x7f915406f390 2024-04-25T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f9151600640 1 -- 172.21.15.7:0/1288832473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f914000cd30 con 0x7f915406f390 2024-04-25T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.699+0000 7f915c000640 1 -- 172.21.15.7:0/1288832473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9154070160 con 0x7f915406f390 2024-04-25T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.700+0000 7f915c000640 1 -- 172.21.15.7:0/1288832473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9154151c10 con 0x7f915406f390 2024-04-25T11:22:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.701+0000 7f915c000640 1 -- 172.21.15.7:0/1288832473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f915414e9f0 con 0x7f915406f390 2024-04-25T11:22:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.702+0000 7f9151600640 1 -- 172.21.15.7:0/1288832473 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f914001a070 con 0x7f915406f390 2024-04-25T11:22:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.704+0000 7f9151600640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f91340744b0 0x7f91340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.704+0000 7f9151600640 1 -- 172.21.15.7:0/1288832473 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5335+0+0 (secure 0 0 0) 0x7f9140095360 con 0x7f915406f390 2024-04-25T11:22:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.704+0000 7f9159600640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f91340744b0 0x7f91340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.705+0000 7f9159600640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f91340744b0 0x7f91340769c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f91480012e0 tx=0x7f9148009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.709+0000 7f9151600640 1 -- 172.21.15.7:0/1288832473 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f914005fc50 con 0x7f915406f390 2024-04-25T11:22:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:21.984+0000 7f915c000640 1 -- 172.21.15.7:0/1288832473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]} v 0) -- 0x7f9154050740 con 0x7f915406f390 2024-04-25T11:22:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.646+0000 7f9151600640 1 -- 172.21.15.7:0/1288832473 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_pin_root"]}]=0 v626) ==== 148+0+0 (secure 0 0 0) 0x7f9140095120 con 0x7f915406f390 2024-04-25T11:22:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.650+0000 7f915c000640 1 -- 172.21.15.7:0/1288832473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f91340744b0 msgr2=0x7f91340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.650+0000 7f915c000640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f91340744b0 0x7f91340769c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f91480012e0 tx=0x7f9148009040 comp rx=0 tx=0).stop 2024-04-25T11:22:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.650+0000 7f915c000640 1 -- 172.21.15.7:0/1288832473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f915406f390 msgr2=0x7f9154153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.650+0000 7f915c000640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f915406f390 0x7f9154153550 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f914000e8a0 tx=0x7f914000ed60 comp rx=0 tx=0).stop 2024-04-25T11:22:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.651+0000 7f915c000640 1 -- 172.21.15.7:0/1288832473 shutdown_connections 2024-04-25T11:22:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.651+0000 7f915c000640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f91340744b0 0x7f91340769c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.651+0000 7f915c000640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91540b5190 0x7f9154152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.651+0000 7f915c000640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f915406f390 0x7f9154153550 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.651+0000 7f915c000640 1 --2- 172.21.15.7:0/1288832473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f915406d580 0x7f915414ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:22.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.651+0000 7f915c000640 1 -- 172.21.15.7:0/1288832473 >> 172.21.15.7:0/1288832473 conn(0x7f91540732b0 msgr2=0x7f91540b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:22.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.651+0000 7f915c000640 1 -- 172.21.15.7:0/1288832473 shutdown_connections 2024-04-25T11:22:22.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.651+0000 7f915c000640 1 -- 172.21.15.7:0/1288832473 wait complete. 2024-04-25T11:22:22.686 INFO:tasks.cephfs_test_runner:test_client_pin_root (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2024-04-25T11:22:22.686 INFO:tasks.cephfs_test_runner:Starting test: test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) When a client has a bug (which we will simulate) preventing it from releasing caps, 2024-04-25T11:22:22.686 INFO:tasks.cephfs_test_runner:test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) 2024-04-25T11:22:22.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:22.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.973+0000 7fe64be00640 1 Processor -- start 2024-04-25T11:22:22.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.973+0000 7fe64be00640 1 -- start start 2024-04-25T11:22:22.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.976+0000 7fe64be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c15baa0 0x7fe64c15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:22.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.976+0000 7fe64be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe64c070640 0x7fe64c161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:22.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.976+0000 7fe64be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe64c15e980 con 0x7fe64c162420 2024-04-25T11:22:22.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.976+0000 7fe64be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe64c162830 con 0x7fe64c070640 2024-04-25T11:22:22.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.976+0000 7fe64be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe64c15ec60 con 0x7fe64c15baa0 2024-04-25T11:22:22.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.976+0000 7fe642000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe64c070640 0x7fe64c161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:22.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.976+0000 7fe64aa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c15baa0 0x7fe64c15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:22.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.976+0000 7fe642000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe64c070640 0x7fe64c161ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:43828/0 (socket says 172.21.15.7:43828) 2024-04-25T11:22:22.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.976+0000 7fe642000640 1 -- 172.21.15.7:0/1481882736 learned_addr learned my addr 172.21.15.7:0/1481882736 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:22.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.977+0000 7fe64aa00640 1 -- 172.21.15.7:0/1481882736 >> v2:172.21.15.87:3300/0 conn(0x7fe64c070640 msgr2=0x7fe64c161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:22.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.977+0000 7fe64aa00640 1 --2- 172.21.15.7:0/1481882736 >> v2:172.21.15.87:3300/0 conn(0x7fe64c070640 0x7fe64c161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:22.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.977+0000 7fe64aa00640 1 -- 172.21.15.7:0/1481882736 >> v1:172.21.15.87:6789/0 conn(0x7fe64c162420 legacy=0x7fe64c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:22.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.977+0000 7fe64aa00640 1 -- 172.21.15.7:0/1481882736 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c15ef40 con 0x7fe64c15baa0 2024-04-25T11:22:22.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.977+0000 7fe642000640 1 --2- 172.21.15.7:0/1481882736 >> v2:172.21.15.87:3300/0 conn(0x7fe64c070640 0x7fe64c161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:22:22.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.978+0000 7fe64aa00640 1 --2- 172.21.15.7:0/1481882736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c15baa0 0x7fe64c15e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0097a0 tx=0x7fe63c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a56ce3db0a15a52e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:22.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.979+0000 7fe64a000640 1 -- 172.21.15.7:0/1481882736 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c037070 con 0x7fe64c15baa0 2024-04-25T11:22:22.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.979+0000 7fe64a000640 1 -- 172.21.15.7:0/1481882736 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe63c032440 con 0x7fe64c15baa0 2024-04-25T11:22:22.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.979+0000 7fe64a000640 1 -- 172.21.15.7:0/1481882736 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c032810 con 0x7fe64c15baa0 2024-04-25T11:22:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.979+0000 7fe64be00640 1 -- 172.21.15.7:0/1481882736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c15baa0 msgr2=0x7fe64c15e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.979+0000 7fe64be00640 1 --2- 172.21.15.7:0/1481882736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c15baa0 0x7fe64c15e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0097a0 tx=0x7fe63c02a180 comp rx=0 tx=0).stop 2024-04-25T11:22:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.980+0000 7fe64be00640 1 -- 172.21.15.7:0/1481882736 shutdown_connections 2024-04-25T11:22:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.980+0000 7fe64be00640 1 --2- 172.21.15.7:0/1481882736 >> v2:172.21.15.87:3300/0 conn(0x7fe64c070640 0x7fe64c161ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.980+0000 7fe64be00640 1 --2- 172.21.15.7:0/1481882736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c15baa0 0x7fe64c15e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.980+0000 7fe64be00640 1 -- 172.21.15.7:0/1481882736 >> 172.21.15.7:0/1481882736 conn(0x7fe64c073090 msgr2=0x7fe64c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.980+0000 7fe64be00640 1 -- 172.21.15.7:0/1481882736 shutdown_connections 2024-04-25T11:22:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.980+0000 7fe64be00640 1 -- 172.21.15.7:0/1481882736 wait complete. 2024-04-25T11:22:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.983+0000 7fe64be00640 1 Processor -- start 2024-04-25T11:22:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.983+0000 7fe64be00640 1 -- start start 2024-04-25T11:22:22.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.983+0000 7fe64be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c070640 0x7fe64c137ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:22.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.983+0000 7fe64be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe64c162420 0x7fe64c137330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:22.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.984+0000 7fe64be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c137870 0x7fe64c134650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:22.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.984+0000 7fe64be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c070bd0 con 0x7fe64c137870 2024-04-25T11:22:22.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.984+0000 7fe64be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c070a50 con 0x7fe64c162420 2024-04-25T11:22:22.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.984+0000 7fe64be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe64c06dc90 con 0x7fe64c070640 2024-04-25T11:22:22.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.984+0000 7fe64b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c137870 0x7fe64c134650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.984+0000 7fe64b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c137870 0x7fe64c134650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47918/0 (socket says 172.21.15.7:47918) 2024-04-25T11:22:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.984+0000 7fe64b400640 1 -- 172.21.15.7:0/3928188197 learned_addr learned my addr 172.21.15.7:0/3928188197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.984+0000 7fe642000640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe64c162420 0x7fe64c137330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.984+0000 7fe64aa00640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c070640 0x7fe64c137ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe64b400640 1 -- 172.21.15.7:0/3928188197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c070640 msgr2=0x7fe64c137ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe64b400640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c070640 0x7fe64c137ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe64b400640 1 -- 172.21.15.7:0/3928188197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe64c162420 msgr2=0x7fe64c137330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:22.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe64b400640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe64c162420 0x7fe64c137330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe64b400640 1 -- 172.21.15.7:0/3928188197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c137c80 con 0x7fe64c137870 2024-04-25T11:22:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe64aa00640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c070640 0x7fe64c137ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:22:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe642000640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe64c162420 0x7fe64c137330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:22:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe64b400640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c137870 0x7fe64c134650 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe64400c730 tx=0x7fe64400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe643400640 1 -- 172.21.15.7:0/3928188197 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644003b70 con 0x7fe64c137870 2024-04-25T11:22:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe643400640 1 -- 172.21.15.7:0/3928188197 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe644003d10 con 0x7fe64c137870 2024-04-25T11:22:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe64be00640 1 -- 172.21.15.7:0/3928188197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c0b0bd0 con 0x7fe64c137870 2024-04-25T11:22:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe643400640 1 -- 172.21.15.7:0/3928188197 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644005310 con 0x7fe64c137870 2024-04-25T11:22:22.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.985+0000 7fe64be00640 1 -- 172.21.15.7:0/3928188197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c1354f0 con 0x7fe64c137870 2024-04-25T11:22:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.987+0000 7fe64be00640 1 -- 172.21.15.7:0/3928188197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64c0ad8f0 con 0x7fe64c137870 2024-04-25T11:22:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.988+0000 7fe643400640 1 -- 172.21.15.7:0/3928188197 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe6440056c0 con 0x7fe64c137870 2024-04-25T11:22:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.989+0000 7fe643400640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6280744e0 0x7fe6280769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.989+0000 7fe643400640 1 -- 172.21.15.7:0/3928188197 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5335+0+0 (secure 0 0 0) 0x7fe6440951d0 con 0x7fe64c137870 2024-04-25T11:22:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.989+0000 7fe64aa00640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6280744e0 0x7fe6280769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:22.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.990+0000 7fe64aa00640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6280744e0 0x7fe6280769f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe63c034000 tx=0x7fe63c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:22.994+0000 7fe643400640 1 -- 172.21.15.7:0/3928188197 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe64c0ad8f0 con 0x7fe64c137870 2024-04-25T11:22:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.190+0000 7fe64be00640 1 -- 172.21.15.7:0/3928188197 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe64c0ac660 con 0x7fe64c137870 2024-04-25T11:22:23.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.192+0000 7fe643400640 1 -- 172.21.15.7:0/3928188197 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) ==== 74+0+13205 (secure 0 0 0) 0x7fe64c137c80 con 0x7fe64c137870 2024-04-25T11:22:23.188 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:23.188 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":147,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:18.612282+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6837/2966579341":"2024-04-26T11:22:15.257431+0000","172.21.15.7:6836/2966579341":"2024-04-26T11:22:15.257431+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.194+0000 7fe64be00640 1 -- 172.21.15.7:0/3928188197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6280744e0 msgr2=0x7fe6280769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.194+0000 7fe64be00640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6280744e0 0x7fe6280769f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe63c034000 tx=0x7fe63c02e000 comp rx=0 tx=0).stop 2024-04-25T11:22:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.194+0000 7fe64be00640 1 -- 172.21.15.7:0/3928188197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c137870 msgr2=0x7fe64c134650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.194+0000 7fe64be00640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c137870 0x7fe64c134650 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe64400c730 tx=0x7fe64400cbf0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.195+0000 7fe64be00640 1 -- 172.21.15.7:0/3928188197 shutdown_connections 2024-04-25T11:22:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.195+0000 7fe64be00640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe6280744e0 0x7fe6280769f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.195+0000 7fe64be00640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe64c137870 0x7fe64c134650 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.195+0000 7fe64be00640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe64c162420 0x7fe64c137330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.195+0000 7fe64be00640 1 --2- 172.21.15.7:0/3928188197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe64c070640 0x7fe64c137ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.195+0000 7fe64be00640 1 -- 172.21.15.7:0/3928188197 >> 172.21.15.7:0/3928188197 conn(0x7fe64c073090 msgr2=0x7fe64c0b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.195+0000 7fe64be00640 1 -- 172.21.15.7:0/3928188197 shutdown_connections 2024-04-25T11:22:23.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.195+0000 7fe64be00640 1 -- 172.21.15.7:0/3928188197 wait complete. 2024-04-25T11:22:23.217 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:22:23.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:22:23.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.503+0000 7fa296000640 1 Processor -- start 2024-04-25T11:22:23.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.503+0000 7fa296000640 1 -- start start 2024-04-25T11:22:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.506+0000 7fa296000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa290146180 0x7fa290070320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.506+0000 7fa296000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa2900b08e0 0x7fa290149500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.506+0000 7fa296000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa290070860 con 0x7fa2901490f0 2024-04-25T11:22:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.506+0000 7fa296000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa290146590 con 0x7fa290146180 2024-04-25T11:22:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.506+0000 7fa296000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa29006f280 con 0x7fa2900b08e0 2024-04-25T11:22:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.506+0000 7fa294c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa290146180 0x7fa290070320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.506+0000 7fa294c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa290146180 0x7fa290070320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47930/0 (socket says 172.21.15.7:47930) 2024-04-25T11:22:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.506+0000 7fa294c00640 1 -- 172.21.15.7:0/4046284365 learned_addr learned my addr 172.21.15.7:0/4046284365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.506+0000 7fa295600640 1 --2- 172.21.15.7:0/4046284365 >> v2:172.21.15.87:3300/0 conn(0x7fa2900b08e0 0x7fa290149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.507+0000 7fa294c00640 1 -- 172.21.15.7:0/4046284365 >> v2:172.21.15.87:3300/0 conn(0x7fa2900b08e0 msgr2=0x7fa290149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.507+0000 7fa294c00640 1 --2- 172.21.15.7:0/4046284365 >> v2:172.21.15.87:3300/0 conn(0x7fa2900b08e0 0x7fa290149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.507+0000 7fa294c00640 1 -- 172.21.15.7:0/4046284365 >> v1:172.21.15.87:6789/0 conn(0x7fa2901490f0 legacy=0x7fa2900b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.507+0000 7fa294c00640 1 -- 172.21.15.7:0/4046284365 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29006f560 con 0x7fa290146180 2024-04-25T11:22:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.508+0000 7fa294c00640 1 --2- 172.21.15.7:0/4046284365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa290146180 0x7fa290070320 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fa290164a70 tx=0x7fa28002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248b8c51f9252e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.508+0000 7fa28f400640 1 -- 172.21.15.7:0/4046284365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280036070 con 0x7fa290146180 2024-04-25T11:22:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.508+0000 7fa28f400640 1 -- 172.21.15.7:0/4046284365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa280031600 con 0x7fa290146180 2024-04-25T11:22:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.508+0000 7fa28f400640 1 -- 172.21.15.7:0/4046284365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2800319d0 con 0x7fa290146180 2024-04-25T11:22:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.509+0000 7fa296000640 1 -- 172.21.15.7:0/4046284365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa290146180 msgr2=0x7fa290070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.509+0000 7fa296000640 1 --2- 172.21.15.7:0/4046284365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa290146180 0x7fa290070320 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fa290164a70 tx=0x7fa28002a730 comp rx=0 tx=0).stop 2024-04-25T11:22:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.509+0000 7fa296000640 1 -- 172.21.15.7:0/4046284365 shutdown_connections 2024-04-25T11:22:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.509+0000 7fa296000640 1 --2- 172.21.15.7:0/4046284365 >> v2:172.21.15.87:3300/0 conn(0x7fa2900b08e0 0x7fa290149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.509+0000 7fa296000640 1 --2- 172.21.15.7:0/4046284365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa290146180 0x7fa290070320 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.509+0000 7fa296000640 1 -- 172.21.15.7:0/4046284365 >> 172.21.15.7:0/4046284365 conn(0x7fa2900740f0 msgr2=0x7fa29006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.509+0000 7fa296000640 1 -- 172.21.15.7:0/4046284365 shutdown_connections 2024-04-25T11:22:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.509+0000 7fa296000640 1 -- 172.21.15.7:0/4046284365 wait complete. 2024-04-25T11:22:23.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.514+0000 7fa296000640 1 Processor -- start 2024-04-25T11:22:23.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.514+0000 7fa296000640 1 -- start start 2024-04-25T11:22:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa296000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900b08e0 0x7fa290144210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa296000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa290146180 0x7fa290161c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa296000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2901490f0 0x7fa290162190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa296000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa29006fe00 con 0x7fa2900b08e0 2024-04-25T11:22:23.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa296000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa29006fb20 con 0x7fa290146180 2024-04-25T11:22:23.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa296000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa29006eea0 con 0x7fa2901490f0 2024-04-25T11:22:23.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa294c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900b08e0 0x7fa290144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:23.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa294c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900b08e0 0x7fa290144210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47932/0 (socket says 172.21.15.7:47932) 2024-04-25T11:22:23.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa294c00640 1 -- 172.21.15.7:0/4288292680 learned_addr learned my addr 172.21.15.7:0/4288292680 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:23.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa294c00640 1 -- 172.21.15.7:0/4288292680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2901490f0 msgr2=0x7fa290162190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:23.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa294c00640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2901490f0 0x7fa290162190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa294c00640 1 -- 172.21.15.7:0/4288292680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa290146180 msgr2=0x7fa290161c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:23.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa294c00640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa290146180 0x7fa290161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa294c00640 1 -- 172.21.15.7:0/4288292680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa290144af0 con 0x7fa2900b08e0 2024-04-25T11:22:23.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.516+0000 7fa294c00640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900b08e0 0x7fa290144210 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fa280034040 tx=0x7fa280004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:23.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.517+0000 7fa28cc00640 1 -- 172.21.15.7:0/4288292680 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280036050 con 0x7fa2900b08e0 2024-04-25T11:22:23.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.517+0000 7fa28cc00640 1 -- 172.21.15.7:0/4288292680 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa280042070 con 0x7fa2900b08e0 2024-04-25T11:22:23.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.517+0000 7fa28cc00640 1 -- 172.21.15.7:0/4288292680 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa280009510 con 0x7fa2900b08e0 2024-04-25T11:22:23.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.517+0000 7fa296000640 1 -- 172.21.15.7:0/4288292680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa290144dd0 con 0x7fa2900b08e0 2024-04-25T11:22:23.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.517+0000 7fa296000640 1 -- 172.21.15.7:0/4288292680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2901500d0 con 0x7fa2900b08e0 2024-04-25T11:22:23.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.518+0000 7fa296000640 1 -- 172.21.15.7:0/4288292680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa290050740 con 0x7fa2900b08e0 2024-04-25T11:22:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.519+0000 7fa28cc00640 1 -- 172.21.15.7:0/4288292680 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa280007c70 con 0x7fa2900b08e0 2024-04-25T11:22:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.520+0000 7fa28cc00640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2700744b0 0x7fa2700769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.521+0000 7fa28cc00640 1 -- 172.21.15.7:0/4288292680 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5335+0+0 (secure 0 0 0) 0x7fa2800b5e50 con 0x7fa2900b08e0 2024-04-25T11:22:23.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.521+0000 7fa28fe00640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2700744b0 0x7fa2700769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:23.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.522+0000 7fa28fe00640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2700744b0 0x7fa2700769c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa27c000f60 tx=0x7fa27c0079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:23.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.525+0000 7fa28cc00640 1 -- 172.21.15.7:0/4288292680 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa280080870 con 0x7fa2900b08e0 2024-04-25T11:22:23.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.740+0000 7fa296000640 1 -- 172.21.15.7:0/4288292680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa2900b0330 con 0x7fa2900b08e0 2024-04-25T11:22:23.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.742+0000 7fa28cc00640 1 -- 172.21.15.7:0/4288292680 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) ==== 93+0+10 (secure 0 0 0) 0x7fa280080170 con 0x7fa2900b08e0 2024-04-25T11:22:23.737 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:22:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.743+0000 7fa296000640 1 -- 172.21.15.7:0/4288292680 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2700744b0 msgr2=0x7fa2700769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.743+0000 7fa296000640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2700744b0 0x7fa2700769c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa27c000f60 tx=0x7fa27c0079d0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.743+0000 7fa296000640 1 -- 172.21.15.7:0/4288292680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900b08e0 msgr2=0x7fa290144210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.743+0000 7fa296000640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900b08e0 0x7fa290144210 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fa280034040 tx=0x7fa280004a40 comp rx=0 tx=0).stop 2024-04-25T11:22:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.744+0000 7fa296000640 1 -- 172.21.15.7:0/4288292680 shutdown_connections 2024-04-25T11:22:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.744+0000 7fa296000640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa2700744b0 0x7fa2700769c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.744+0000 7fa296000640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2901490f0 0x7fa290162190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.744+0000 7fa296000640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa290146180 0x7fa290161c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.744+0000 7fa296000640 1 --2- 172.21.15.7:0/4288292680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2900b08e0 0x7fa290144210 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.744+0000 7fa296000640 1 -- 172.21.15.7:0/4288292680 >> 172.21.15.7:0/4288292680 conn(0x7fa2900740f0 msgr2=0x7fa2900b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.744+0000 7fa296000640 1 -- 172.21.15.7:0/4288292680 shutdown_connections 2024-04-25T11:22:23.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:23.744+0000 7fa296000640 1 -- 172.21.15.7:0/4288292680 wait complete. 2024-04-25T11:22:23.767 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2024-04-25T11:22:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.059+0000 7f48dcc00640 1 Processor -- start 2024-04-25T11:22:24.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.059+0000 7f48dcc00640 1 -- start start 2024-04-25T11:22:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.061+0000 7f48dcc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f48d80d0fb0 0x7f48d815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.061+0000 7f48dcc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d80b8e50 0x7f48d80b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.061+0000 7f48dcc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f48d80b7920 0x7f48d80d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.061+0000 7f48dcc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f48d80b7d30 con 0x7f48d80b7920 2024-04-25T11:22:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.061+0000 7f48dcc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f48d80d13c0 con 0x7f48d80d0fb0 2024-04-25T11:22:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.061+0000 7f48dcc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48d8163f10 con 0x7f48d80b8e50 2024-04-25T11:22:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.062+0000 7f48d7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f48d80b7920 0x7f48d80d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.062+0000 7f48d7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f48d80b7920 0x7f48d80d45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47944/0 (socket says 172.21.15.7:47944) 2024-04-25T11:22:24.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.062+0000 7f48d7e00640 1 -- 172.21.15.7:0/1772780624 learned_addr learned my addr 172.21.15.7:0/1772780624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.062+0000 7f48d6a00640 1 --2- 172.21.15.7:0/1772780624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d80b8e50 0x7f48d80b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.062+0000 7f48d7400640 1 --2- 172.21.15.7:0/1772780624 >> v2:172.21.15.87:3300/0 conn(0x7f48d80d0fb0 0x7f48d815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.062+0000 7f48d7e00640 1 -- 172.21.15.7:0/1772780624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d80b8e50 msgr2=0x7f48d80b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.062+0000 7f48d7e00640 1 --2- 172.21.15.7:0/1772780624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d80b8e50 0x7f48d80b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.062+0000 7f48d7e00640 1 -- 172.21.15.7:0/1772780624 >> v2:172.21.15.87:3300/0 conn(0x7f48d80d0fb0 msgr2=0x7f48d815fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.062+0000 7f48d7e00640 1 --2- 172.21.15.7:0/1772780624 >> v2:172.21.15.87:3300/0 conn(0x7f48d80d0fb0 0x7f48d815fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.062+0000 7f48d7e00640 1 -- 172.21.15.7:0/1772780624 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d81641f0 con 0x7f48d80b7920 2024-04-25T11:22:24.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.064+0000 7f48d7e00640 1 --2- 172.21.15.7:0/1772780624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d80b7920 0x7f48d80d45e0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f48d80d33d0 tx=0x7f48d002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785a458d7bb14719 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:24.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.064+0000 7f48d6000640 1 -- 172.21.15.7:0/1772780624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0037070 con 0x7f48d80b7920 2024-04-25T11:22:24.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.064+0000 7f48d6000640 1 -- 172.21.15.7:0/1772780624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d0032440 con 0x7f48d80b7920 2024-04-25T11:22:24.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.064+0000 7f48d6000640 1 -- 172.21.15.7:0/1772780624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0032810 con 0x7f48d80b7920 2024-04-25T11:22:24.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.065+0000 7f48dcc00640 1 -- 172.21.15.7:0/1772780624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d80b7920 msgr2=0x7f48d80d45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:24.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.065+0000 7f48dcc00640 1 --2- 172.21.15.7:0/1772780624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d80b7920 0x7f48d80d45e0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f48d80d33d0 tx=0x7f48d002a140 comp rx=0 tx=0).stop 2024-04-25T11:22:24.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.065+0000 7f48dcc00640 1 -- 172.21.15.7:0/1772780624 shutdown_connections 2024-04-25T11:22:24.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.065+0000 7f48dcc00640 1 --2- 172.21.15.7:0/1772780624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d80b7920 0x7f48d80d45e0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.065+0000 7f48dcc00640 1 --2- 172.21.15.7:0/1772780624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d80b8e50 0x7f48d80b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.065+0000 7f48dcc00640 1 --2- 172.21.15.7:0/1772780624 >> v2:172.21.15.87:3300/0 conn(0x7f48d80d0fb0 0x7f48d815fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.065+0000 7f48dcc00640 1 -- 172.21.15.7:0/1772780624 >> 172.21.15.7:0/1772780624 conn(0x7f48d80d23e0 msgr2=0x7f48d80d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:24.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.065+0000 7f48dcc00640 1 -- 172.21.15.7:0/1772780624 shutdown_connections 2024-04-25T11:22:24.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.065+0000 7f48dcc00640 1 -- 172.21.15.7:0/1772780624 wait complete. 2024-04-25T11:22:24.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.068+0000 7f48dcc00640 1 Processor -- start 2024-04-25T11:22:24.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.068+0000 7f48dcc00640 1 -- start start 2024-04-25T11:22:24.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.069+0000 7f48dcc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d80b7920 0x7f48d8155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:24.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.069+0000 7f48dcc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d80b8e50 0x7f48d80ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:24.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.069+0000 7f48dcc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48d80d0fb0 0x7f48d80cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:24.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.069+0000 7f48dcc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48d80b4bc0 con 0x7f48d80b8e50 2024-04-25T11:22:24.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.069+0000 7f48dcc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48d81647b0 con 0x7f48d80d0fb0 2024-04-25T11:22:24.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.069+0000 7f48dcc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48d80b9260 con 0x7f48d80b7920 2024-04-25T11:22:24.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.070+0000 7f48d6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d80b8e50 0x7f48d80ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:24.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.070+0000 7f48d6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d80b8e50 0x7f48d80ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47954/0 (socket says 172.21.15.7:47954) 2024-04-25T11:22:24.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.070+0000 7f48d6a00640 1 -- 172.21.15.7:0/3373839525 learned_addr learned my addr 172.21.15.7:0/3373839525 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:24.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.070+0000 7f48d6a00640 1 -- 172.21.15.7:0/3373839525 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d80b7920 msgr2=0x7f48d8155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.070+0000 7f48d6a00640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d80b7920 0x7f48d8155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.070+0000 7f48d6a00640 1 -- 172.21.15.7:0/3373839525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48d80d0fb0 msgr2=0x7f48d80cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.070+0000 7f48d7e00640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48d80d0fb0 0x7f48d80cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.070+0000 7f48d6a00640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48d80d0fb0 0x7f48d80cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.070+0000 7f48d6a00640 1 -- 172.21.15.7:0/3373839525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d80cc340 con 0x7f48d80b8e50 2024-04-25T11:22:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.070+0000 7f48d7e00640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48d80d0fb0 0x7f48d80cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:24.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.071+0000 7f48d6a00640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d80b8e50 0x7f48d80ca460 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f48c8007e90 tx=0x7f48c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:24.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.071+0000 7f48cf400640 1 -- 172.21.15.7:0/3373839525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8014420 con 0x7f48d80b8e50 2024-04-25T11:22:24.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.071+0000 7f48cf400640 1 -- 172.21.15.7:0/3373839525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c8019070 con 0x7f48d80b8e50 2024-04-25T11:22:24.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.071+0000 7f48cf400640 1 -- 172.21.15.7:0/3373839525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c80147e0 con 0x7f48d80b8e50 2024-04-25T11:22:24.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.071+0000 7f48dcc00640 1 -- 172.21.15.7:0/3373839525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d80cad60 con 0x7f48d80b8e50 2024-04-25T11:22:24.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.071+0000 7f48dcc00640 1 -- 172.21.15.7:0/3373839525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d80cb320 con 0x7f48d80b8e50 2024-04-25T11:22:24.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.072+0000 7f48dcc00640 1 -- 172.21.15.7:0/3373839525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d80caad0 con 0x7f48d80b8e50 2024-04-25T11:22:24.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.074+0000 7f48cf400640 1 -- 172.21.15.7:0/3373839525 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f48c8014980 con 0x7f48d80b8e50 2024-04-25T11:22:24.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.075+0000 7f48cf400640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48b40744b0 0x7f48b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:24.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.075+0000 7f48cf400640 1 -- 172.21.15.7:0/3373839525 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5335+0+0 (secure 0 0 0) 0x7f48c8095960 con 0x7f48d80b8e50 2024-04-25T11:22:24.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.075+0000 7f48d7400640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48b40744b0 0x7f48b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:24.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.076+0000 7f48d7400640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48b40744b0 0x7f48b40769c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f48c0006fd0 tx=0x7f48c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:24.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.080+0000 7f48cf400640 1 -- 172.21.15.7:0/3373839525 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f48c80601c0 con 0x7f48d80b8e50 2024-04-25T11:22:24.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.357+0000 7f48dcc00640 1 -- 172.21.15.7:0/3373839525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) -- 0x7f48d8162d80 con 0x7f48d80b8e50 2024-04-25T11:22:24.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.665+0000 7f48cf400640 1 -- 172.21.15.7:0/3373839525 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v628) ==== 154+0+0 (secure 0 0 0) 0x7f48d8162d80 con 0x7f48d80b8e50 2024-04-25T11:22:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.668+0000 7f48dcc00640 1 -- 172.21.15.7:0/3373839525 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48b40744b0 msgr2=0x7f48b40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.668+0000 7f48dcc00640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48b40744b0 0x7f48b40769c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f48c0006fd0 tx=0x7f48c0008040 comp rx=0 tx=0).stop 2024-04-25T11:22:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.668+0000 7f48dcc00640 1 -- 172.21.15.7:0/3373839525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d80b8e50 msgr2=0x7f48d80ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:24.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.668+0000 7f48dcc00640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d80b8e50 0x7f48d80ca460 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f48c8007e90 tx=0x7f48c800e4b0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.668+0000 7f48dcc00640 1 -- 172.21.15.7:0/3373839525 shutdown_connections 2024-04-25T11:22:24.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.668+0000 7f48dcc00640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f48b40744b0 0x7f48b40769c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.668+0000 7f48dcc00640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48d80d0fb0 0x7f48d80cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.668+0000 7f48dcc00640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48d80b8e50 0x7f48d80ca460 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.668+0000 7f48dcc00640 1 --2- 172.21.15.7:0/3373839525 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48d80b7920 0x7f48d8155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.668+0000 7f48dcc00640 1 -- 172.21.15.7:0/3373839525 >> 172.21.15.7:0/3373839525 conn(0x7f48d80d23e0 msgr2=0x7f48d80d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:24.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.669+0000 7f48dcc00640 1 -- 172.21.15.7:0/3373839525 shutdown_connections 2024-04-25T11:22:24.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.669+0000 7f48dcc00640 1 -- 172.21.15.7:0/3373839525 wait complete. 2024-04-25T11:22:24.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T11:22:24.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.997+0000 7fdc32e00640 1 Processor -- start 2024-04-25T11:22:24.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.997+0000 7fdc32e00640 1 -- start start 2024-04-25T11:22:24.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.999+0000 7fdc32e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdc2c0b7be0 0x7fdc2c0d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:24.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.999+0000 7fdc32e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc2c0d1cd0 0x7fdc2c0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:24.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.999+0000 7fdc32e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc2c1603d0 con 0x7fdc2c06de10 2024-04-25T11:22:24.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.999+0000 7fdc32e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fdc2c06d090 con 0x7fdc2c0d1cd0 2024-04-25T11:22:24.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:24.999+0000 7fdc32e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fdc2c0d3490 con 0x7fdc2c0b7be0 2024-04-25T11:22:24.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.000+0000 7fdc31a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdc2c06de10 0x7fdc2c0b74d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41132/0 (socket says 172.21.15.7:41132) 2024-04-25T11:22:24.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.000+0000 7fdc31a00640 1 -- 172.21.15.7:0/2806559093 learned_addr learned my addr 172.21.15.7:0/2806559093 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:24.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.000+0000 7fdc31000640 1 --2- 172.21.15.7:0/2806559093 >> v2:172.21.15.87:3300/0 conn(0x7fdc2c0b7be0 0x7fdc2c0d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:24.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.000+0000 7fdc32400640 1 --2- 172.21.15.7:0/2806559093 >> v2:172.21.15.7:3300/0 conn(0x7fdc2c0d1cd0 0x7fdc2c0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:24.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.000+0000 7fdc2be00640 1 -- 172.21.15.7:0/2806559093 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3496705996 0 0) 0x7fdc2c1603d0 con 0x7fdc2c06de10 2024-04-25T11:22:24.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.000+0000 7fdc2be00640 1 -- 172.21.15.7:0/2806559093 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc14003520 con 0x7fdc2c06de10 2024-04-25T11:22:24.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.001+0000 7fdc32400640 1 -- 172.21.15.7:0/2806559093 >> v2:172.21.15.87:3300/0 conn(0x7fdc2c0b7be0 msgr2=0x7fdc2c0d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:24.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.001+0000 7fdc32400640 1 --2- 172.21.15.7:0/2806559093 >> v2:172.21.15.87:3300/0 conn(0x7fdc2c0b7be0 0x7fdc2c0d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.001+0000 7fdc32400640 1 -- 172.21.15.7:0/2806559093 >> v1:172.21.15.7:6789/0 conn(0x7fdc2c06de10 legacy=0x7fdc2c0b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:24.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.001+0000 7fdc2be00640 1 -- 172.21.15.7:0/2806559093 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3618321039 0 0) 0x7fdc14003520 con 0x7fdc2c06de10 2024-04-25T11:22:24.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.001+0000 7fdc32400640 1 -- 172.21.15.7:0/2806559093 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c06d330 con 0x7fdc2c0d1cd0 2024-04-25T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.002+0000 7fdc32400640 1 --2- 172.21.15.7:0/2806559093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc2c0d1cd0 0x7fdc2c0b2150 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0a08a0 tx=0x7fdc2402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecd5393991f5671 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.002+0000 7fdc2be00640 1 -- 172.21.15.7:0/2806559093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20001670 con 0x7fdc2c0d1cd0 2024-04-25T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.002+0000 7fdc2be00640 1 -- 172.21.15.7:0/2806559093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc24007540 con 0x7fdc2c0d1cd0 2024-04-25T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.002+0000 7fdc2be00640 1 -- 172.21.15.7:0/2806559093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24007910 con 0x7fdc2c0d1cd0 2024-04-25T11:22:24.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.003+0000 7fdc32e00640 1 -- 172.21.15.7:0/2806559093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc2c0d1cd0 msgr2=0x7fdc2c0b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:24.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.003+0000 7fdc32e00640 1 --2- 172.21.15.7:0/2806559093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc2c0d1cd0 0x7fdc2c0b2150 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0a08a0 tx=0x7fdc2402a610 comp rx=0 tx=0).stop 2024-04-25T11:22:24.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.003+0000 7fdc32e00640 1 -- 172.21.15.7:0/2806559093 shutdown_connections 2024-04-25T11:22:24.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.003+0000 7fdc32e00640 1 --2- 172.21.15.7:0/2806559093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc2c0d1cd0 0x7fdc2c0b2150 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.003+0000 7fdc32e00640 1 --2- 172.21.15.7:0/2806559093 >> v2:172.21.15.87:3300/0 conn(0x7fdc2c0b7be0 0x7fdc2c0d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:24.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.003+0000 7fdc32e00640 1 -- 172.21.15.7:0/2806559093 >> 172.21.15.7:0/2806559093 conn(0x7fdc2c09b540 msgr2=0x7fdc2c09b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:24.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.003+0000 7fdc32e00640 1 -- 172.21.15.7:0/2806559093 shutdown_connections 2024-04-25T11:22:24.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.003+0000 7fdc32e00640 1 -- 172.21.15.7:0/2806559093 wait complete. 2024-04-25T11:22:25.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.006+0000 7fdc32e00640 1 Processor -- start 2024-04-25T11:22:25.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.006+0000 7fdc32e00640 1 -- start start 2024-04-25T11:22:25.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.007+0000 7fdc32e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc2c06de10 0x7fdc2c0cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:25.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.007+0000 7fdc32e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc2c0d1cd0 0x7fdc2c1f9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.007+0000 7fdc32e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c14dfb0 0x7fdc2c1f9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.007+0000 7fdc32e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c0d20e0 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.007+0000 7fdc32e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c06d8f0 con 0x7fdc2c14dfb0 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc32e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdc2c0d2260 con 0x7fdc2c06de10 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc31000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc2c0d1cd0 0x7fdc2c1f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc31000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc2c0d1cd0 0x7fdc2c1f9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47980/0 (socket says 172.21.15.7:47980) 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc31000640 1 -- 172.21.15.7:0/2559872000 learned_addr learned my addr 172.21.15.7:0/2559872000 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc32400640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c14dfb0 0x7fdc2c1f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc31000640 1 -- 172.21.15.7:0/2559872000 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc2c06de10 msgr2=0x7fdc2c0cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc31a00640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc2c06de10 0x7fdc2c0cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc31000640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc2c06de10 0x7fdc2c0cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:25.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc31000640 1 -- 172.21.15.7:0/2559872000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c14dfb0 msgr2=0x7fdc2c1f9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:25.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc31000640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c14dfb0 0x7fdc2c1f9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:25.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc31000640 1 -- 172.21.15.7:0/2559872000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c0ccd20 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.008+0000 7fdc32400640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c14dfb0 0x7fdc2c1f9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:25.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.009+0000 7fdc31000640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc2c0d1cd0 0x7fdc2c1f9210 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800e8a0 tx=0x7fdc1800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:25.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.009+0000 7fdc29600640 1 -- 172.21.15.7:0/2559872000 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18017070 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.009+0000 7fdc29600640 1 -- 172.21.15.7:0/2559872000 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc1800c8e0 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.009+0000 7fdc32e00640 1 -- 172.21.15.7:0/2559872000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c0cd000 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.009+0000 7fdc29600640 1 -- 172.21.15.7:0/2559872000 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800ccd0 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.009+0000 7fdc32e00640 1 -- 172.21.15.7:0/2559872000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c0cbf30 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.010+0000 7fdc32e00640 1 -- 172.21.15.7:0/2559872000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2c050740 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.012+0000 7fdc29600640 1 -- 172.21.15.7:0/2559872000 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fdc1801a070 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.013+0000 7fdc29600640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc0c0744e0 0x7fdc0c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:25.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.013+0000 7fdc29600640 1 -- 172.21.15.7:0/2559872000 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5335+0+0 (secure 0 0 0) 0x7fdc18095200 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.013+0000 7fdc31a00640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc0c0744e0 0x7fdc0c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:25.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.014+0000 7fdc31a00640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc0c0744e0 0x7fdc0c0769f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdc200011e0 tx=0x7fdc20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:25.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.018+0000 7fdc29600640 1 -- 172.21.15.7:0/2559872000 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fdc1805fb80 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.220+0000 7fdc32e00640 1 -- 172.21.15.7:0/2559872000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdc2c0ca7f0 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.226+0000 7fdc29600640 1 -- 172.21.15.7:0/2559872000 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v66)=0 v66) ==== 122+0+0 (secure 0 0 0) 0x7fdc1805f480 con 0x7fdc2c0d1cd0 2024-04-25T11:22:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.229+0000 7fdc32e00640 1 -- 172.21.15.7:0/2559872000 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc0c0744e0 msgr2=0x7fdc0c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.229+0000 7fdc32e00640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc0c0744e0 0x7fdc0c0769f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdc200011e0 tx=0x7fdc20009040 comp rx=0 tx=0).stop 2024-04-25T11:22:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.229+0000 7fdc32e00640 1 -- 172.21.15.7:0/2559872000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc2c0d1cd0 msgr2=0x7fdc2c1f9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.229+0000 7fdc32e00640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc2c0d1cd0 0x7fdc2c1f9210 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800e8a0 tx=0x7fdc1800ed60 comp rx=0 tx=0).stop 2024-04-25T11:22:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.230+0000 7fdc32e00640 1 -- 172.21.15.7:0/2559872000 shutdown_connections 2024-04-25T11:22:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.230+0000 7fdc32e00640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fdc0c0744e0 0x7fdc0c0769f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.230+0000 7fdc32e00640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc2c14dfb0 0x7fdc2c1f9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.230+0000 7fdc32e00640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc2c0d1cd0 0x7fdc2c1f9210 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.230+0000 7fdc32e00640 1 --2- 172.21.15.7:0/2559872000 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc2c06de10 0x7fdc2c0cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.230+0000 7fdc32e00640 1 -- 172.21.15.7:0/2559872000 >> 172.21.15.7:0/2559872000 conn(0x7fdc2c09b540 msgr2=0x7fdc2c071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.230+0000 7fdc32e00640 1 -- 172.21.15.7:0/2559872000 shutdown_connections 2024-04-25T11:22:25.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:25.230+0000 7fdc32e00640 1 -- 172.21.15.7:0/2559872000 wait complete. 2024-04-25T11:22:25.253 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:22:25.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:22:25.358 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:22:25.471 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:22:25.472 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:22:25.472 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:25.536 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:22:25.536 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:22:25.536 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:25.592 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:22:25.592 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:22:25.592 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:25.650 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:22:25.650 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:22:25.650 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:25.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:22:25.707 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T11:22:25.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:26.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.006+0000 7fbda5a00640 1 Processor -- start 2024-04-25T11:22:26.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.006+0000 7fbda5a00640 1 -- start start 2024-04-25T11:22:26.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.008+0000 7fbda5a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbda015ef90 0x7fbda009f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:26.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.008+0000 7fbda5a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbda00af540 0x7fbda00cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:26.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.008+0000 7fbda5a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbda015f4e0 con 0x7fbda015bc40 2024-04-25T11:22:26.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.008+0000 7fbda5a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbda0071190 con 0x7fbda00af540 2024-04-25T11:22:26.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.008+0000 7fbda5a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbda00b0560 con 0x7fbda015ef90 2024-04-25T11:22:26.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.009+0000 7fbd9f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbda00af540 0x7fbda00cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:26.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.009+0000 7fbd9f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbda00af540 0x7fbda00cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47986/0 (socket says 172.21.15.7:47986) 2024-04-25T11:22:26.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.009+0000 7fbd9f400640 1 -- 172.21.15.7:0/1227977936 learned_addr learned my addr 172.21.15.7:0/1227977936 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:26.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.009+0000 7fbd9ea00640 1 --2- 172.21.15.7:0/1227977936 >> v2:172.21.15.87:3300/0 conn(0x7fbda015ef90 0x7fbda009f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:26.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.009+0000 7fbd9f400640 1 -- 172.21.15.7:0/1227977936 >> v2:172.21.15.87:3300/0 conn(0x7fbda015ef90 msgr2=0x7fbda009f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:26.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.009+0000 7fbd9f400640 1 --2- 172.21.15.7:0/1227977936 >> v2:172.21.15.87:3300/0 conn(0x7fbda015ef90 0x7fbda009f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.009+0000 7fbd9f400640 1 -- 172.21.15.7:0/1227977936 >> v1:172.21.15.87:6789/0 conn(0x7fbda015bc40 legacy=0x7fbda00aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:26.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.009+0000 7fbd9f400640 1 -- 172.21.15.7:0/1227977936 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda00b0840 con 0x7fbda00af540 2024-04-25T11:22:26.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.010+0000 7fbd9f400640 1 --2- 172.21.15.7:0/1227977936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda00af540 0x7fbda00cf980 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fbda00b4740 tx=0x7fbd9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68e19c0dc44b1c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:26.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.011+0000 7fbd9e000640 1 -- 172.21.15.7:0/1227977936 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98037070 con 0x7fbda00af540 2024-04-25T11:22:26.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.011+0000 7fbd9e000640 1 -- 172.21.15.7:0/1227977936 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd98032440 con 0x7fbda00af540 2024-04-25T11:22:26.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.011+0000 7fbd9e000640 1 -- 172.21.15.7:0/1227977936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98032810 con 0x7fbda00af540 2024-04-25T11:22:26.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.011+0000 7fbda5a00640 1 -- 172.21.15.7:0/1227977936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda00af540 msgr2=0x7fbda00cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:26.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.011+0000 7fbda5a00640 1 --2- 172.21.15.7:0/1227977936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda00af540 0x7fbda00cf980 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fbda00b4740 tx=0x7fbd9802a1a0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.012+0000 7fbda5a00640 1 -- 172.21.15.7:0/1227977936 shutdown_connections 2024-04-25T11:22:26.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.012+0000 7fbda5a00640 1 --2- 172.21.15.7:0/1227977936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda00af540 0x7fbda00cf980 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.012+0000 7fbda5a00640 1 --2- 172.21.15.7:0/1227977936 >> v2:172.21.15.87:3300/0 conn(0x7fbda015ef90 0x7fbda009f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.012+0000 7fbda5a00640 1 -- 172.21.15.7:0/1227977936 >> 172.21.15.7:0/1227977936 conn(0x7fbda0074200 msgr2=0x7fbda006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:26.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.012+0000 7fbda5a00640 1 -- 172.21.15.7:0/1227977936 shutdown_connections 2024-04-25T11:22:26.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.012+0000 7fbda5a00640 1 -- 172.21.15.7:0/1227977936 wait complete. 2024-04-25T11:22:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.014+0000 7fbda5a00640 1 Processor -- start 2024-04-25T11:22:26.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.014+0000 7fbda5a00640 1 -- start start 2024-04-25T11:22:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.016+0000 7fbda5a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbda015bc40 0x7fbda00b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.016+0000 7fbda5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda015ef90 0x7fbda01fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.016+0000 7fbda5a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbda00b5a60 0x7fbda015cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.016+0000 7fbda5a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbda00b1320 con 0x7fbda015ef90 2024-04-25T11:22:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.016+0000 7fbda5a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbda00b0e00 con 0x7fbda00b5a60 2024-04-25T11:22:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.016+0000 7fbda5a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbda00afb80 con 0x7fbda015bc40 2024-04-25T11:22:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.016+0000 7fbd96000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda015ef90 0x7fbda01fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:26.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.016+0000 7fbd96000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda015ef90 0x7fbda01fa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48002/0 (socket says 172.21.15.7:48002) 2024-04-25T11:22:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.016+0000 7fbd96000640 1 -- 172.21.15.7:0/3221833353 learned_addr learned my addr 172.21.15.7:0/3221833353 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbd96000640 1 -- 172.21.15.7:0/3221833353 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbda015bc40 msgr2=0x7fbda00b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbd9ea00640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbda015bc40 0x7fbda00b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbd96000640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbda015bc40 0x7fbda00b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbd9f400640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbda00b5a60 0x7fbda015cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbd96000640 1 -- 172.21.15.7:0/3221833353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbda00b5a60 msgr2=0x7fbda015cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbd96000640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbda00b5a60 0x7fbda015cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbd96000640 1 -- 172.21.15.7:0/3221833353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda00b6450 con 0x7fbda015ef90 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbd96000640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda015ef90 0x7fbda01fa0a0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800b420 tx=0x7fbd8800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbd97400640 1 -- 172.21.15.7:0/3221833353 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88011830 con 0x7fbda015ef90 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbd97400640 1 -- 172.21.15.7:0/3221833353 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd88016070 con 0x7fbda015ef90 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbd97400640 1 -- 172.21.15.7:0/3221833353 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88011c10 con 0x7fbda015ef90 2024-04-25T11:22:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.017+0000 7fbda5a00640 1 -- 172.21.15.7:0/3221833353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda00b6170 con 0x7fbda015ef90 2024-04-25T11:22:26.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.018+0000 7fbda5a00640 1 -- 172.21.15.7:0/3221833353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda01e7cc0 con 0x7fbda015ef90 2024-04-25T11:22:26.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.019+0000 7fbda5a00640 1 -- 172.21.15.7:0/3221833353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda00b5ee0 con 0x7fbda015ef90 2024-04-25T11:22:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.020+0000 7fbd97400640 1 -- 172.21.15.7:0/3221833353 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbd8801b460 con 0x7fbda015ef90 2024-04-25T11:22:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.021+0000 7fbd97400640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd7c074500 0x7fbd7c076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.021+0000 7fbd97400640 1 -- 172.21.15.7:0/3221833353 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5335+0+0 (secure 0 0 0) 0x7fbd880949c0 con 0x7fbda015ef90 2024-04-25T11:22:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.021+0000 7fbd9ea00640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd7c074500 0x7fbd7c076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:26.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.022+0000 7fbd9ea00640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd7c074500 0x7fbd7c076a10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009e90 tx=0x7fbd9000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:26.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.026+0000 7fbd97400640 1 -- 172.21.15.7:0/3221833353 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fbd88020020 con 0x7fbda015ef90 2024-04-25T11:22:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.275+0000 7fbda5a00640 1 -- 172.21.15.7:0/3221833353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbda006a100 con 0x7fbda015ef90 2024-04-25T11:22:26.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.276+0000 7fbd97400640 1 -- 172.21.15.7:0/3221833353 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+3856 (secure 0 0 0) 0x7fbd8805f4c0 con 0x7fbda015ef90 2024-04-25T11:22:26.271 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:26.271 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":109,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/3503931395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3503931395},{"type":"v1","addr":"172.21.15.87:6834","nonce":3503931395}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/63829697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":63829697},{"type":"v1","addr":"172.21.15.87:6835","nonce":63829697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/415877708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":415877708},{"type":"v1","addr":"172.21.15.7:6835","nonce":415877708}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":9351,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2208326018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":2208326018},{"type":"v1","addr":"172.21.15.7:6837","nonce":2208326018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":109}],"filesystems":[]} 2024-04-25T11:22:26.271 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 109 2024-04-25T11:22:26.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.278+0000 7fbda5a00640 1 -- 172.21.15.7:0/3221833353 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd7c074500 msgr2=0x7fbd7c076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:26.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.278+0000 7fbda5a00640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd7c074500 0x7fbd7c076a10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fbd90009e90 tx=0x7fbd9000b040 comp rx=0 tx=0).stop 2024-04-25T11:22:26.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.278+0000 7fbda5a00640 1 -- 172.21.15.7:0/3221833353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda015ef90 msgr2=0x7fbda01fa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:26.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.278+0000 7fbda5a00640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda015ef90 0x7fbda01fa0a0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800b420 tx=0x7fbd8800b8e0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.279+0000 7fbda5a00640 1 -- 172.21.15.7:0/3221833353 shutdown_connections 2024-04-25T11:22:26.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.279+0000 7fbda5a00640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbd7c074500 0x7fbd7c076a10 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.279+0000 7fbda5a00640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbda00b5a60 0x7fbda015cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.279+0000 7fbda5a00640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbda015ef90 0x7fbda01fa0a0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.279+0000 7fbda5a00640 1 --2- 172.21.15.7:0/3221833353 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbda015bc40 0x7fbda00b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.279+0000 7fbda5a00640 1 -- 172.21.15.7:0/3221833353 >> 172.21.15.7:0/3221833353 conn(0x7fbda0074200 msgr2=0x7fbda015b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:26.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.279+0000 7fbda5a00640 1 -- 172.21.15.7:0/3221833353 shutdown_connections 2024-04-25T11:22:26.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.279+0000 7fbda5a00640 1 -- 172.21.15.7:0/3221833353 wait complete. 2024-04-25T11:22:26.303 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T11:22:26.303 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T11:22:26.303 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:22:26.303 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:22:26.303 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T11:22:26.303 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T11:22:26.304 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:22:26.304 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:22:26.304 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T11:22:26.304 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T11:22:26.304 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:22:26.304 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:22:26.304 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T11:22:26.305 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T11:22:26.305 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:22:26.305 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:22:26.339 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:22:26.340 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T11:22:26.342 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:22:26.343 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T11:22:26.344 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:22:26.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T11:22:26.350 INFO:tasks.ceph.mds.a:Started 2024-04-25T11:22:26.350 DEBUG:teuthology.parallel:result is None 2024-04-25T11:22:26.351 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:22:26.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T11:22:26.390 INFO:tasks.ceph.mds.b:Started 2024-04-25T11:22:26.391 DEBUG:teuthology.parallel:result is None 2024-04-25T11:22:26.395 INFO:tasks.ceph.mds.c:Started 2024-04-25T11:22:26.395 DEBUG:teuthology.parallel:result is None 2024-04-25T11:22:26.396 INFO:tasks.ceph.mds.d:Started 2024-04-25T11:22:26.397 DEBUG:teuthology.parallel:result is None 2024-04-25T11:22:26.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T11:22:26.578 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-04-25T11:22:26.614 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T11:22:26.614 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T11:22:26.619 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:22:26.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.658+0000 7f15aa400640 1 Processor -- start 2024-04-25T11:22:26.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.658+0000 7f15aa400640 1 -- start start 2024-04-25T11:22:26.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.660+0000 7f15aa400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15a406f1e0 0x7f15a406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:26.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.660+0000 7f15aa400640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a4162700 con 0x7f15a415f180 2024-04-25T11:22:26.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.660+0000 7f15aa400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a41629e0 con 0x7f15a406d580 2024-04-25T11:22:26.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.660+0000 7f15aa400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f15a415f590 con 0x7f15a406f1e0 2024-04-25T11:22:26.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.660+0000 7f15a3400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f15a415f180 0x7f15a4161c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41144/0 (socket says 172.21.15.7:41144) 2024-04-25T11:22:26.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.660+0000 7f15a3400640 1 -- 172.21.15.7:0/2779131222 learned_addr learned my addr 172.21.15.7:0/2779131222 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:26.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.660+0000 7f15a3e00640 1 --2- 172.21.15.7:0/2779131222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15a406f1e0 0x7f15a406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:26.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.661+0000 7f15a2a00640 1 -- 172.21.15.7:0/2779131222 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2580721481 0 0) 0x7f15a4162700 con 0x7f15a415f180 2024-04-25T11:22:26.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.661+0000 7f15a2a00640 1 -- 172.21.15.7:0/2779131222 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1588003520 con 0x7f15a415f180 2024-04-25T11:22:26.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.661+0000 7f15a2a00640 1 -- 172.21.15.7:0/2779131222 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699475953 0 0) 0x7f15a41629e0 con 0x7f15a406d580 2024-04-25T11:22:26.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.661+0000 7f15a2a00640 1 -- 172.21.15.7:0/2779131222 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15a4162700 con 0x7f15a406d580 2024-04-25T11:22:26.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.661+0000 7f15a3e00640 1 -- 172.21.15.7:0/2779131222 >> v1:172.21.15.87:6789/0 conn(0x7f15a406d580 legacy=0x7f15a406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:26.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.661+0000 7f15a3e00640 1 -- 172.21.15.7:0/2779131222 >> v1:172.21.15.7:6789/0 conn(0x7f15a415f180 legacy=0x7f15a4161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:26.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.661+0000 7f15a3e00640 1 -- 172.21.15.7:0/2779131222 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a415e980 con 0x7f15a406f1e0 2024-04-25T11:22:26.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.662+0000 7f15a3e00640 1 --2- 172.21.15.7:0/2779131222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15a406f1e0 0x7f15a406f5f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f159c009dc0 tx=0x7f159c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=921fe1640d8fb456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:26.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.663+0000 7f15a2a00640 1 -- 172.21.15.7:0/2779131222 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c007740 con 0x7f15a406f1e0 2024-04-25T11:22:26.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.663+0000 7f15a2a00640 1 -- 172.21.15.7:0/2779131222 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159c0078e0 con 0x7f15a406f1e0 2024-04-25T11:22:26.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.663+0000 7f15a2a00640 1 -- 172.21.15.7:0/2779131222 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c007cb0 con 0x7f15a406f1e0 2024-04-25T11:22:26.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.663+0000 7f15aa400640 1 -- 172.21.15.7:0/2779131222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15a406f1e0 msgr2=0x7f15a406f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:26.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.663+0000 7f15aa400640 1 --2- 172.21.15.7:0/2779131222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15a406f1e0 0x7f15a406f5f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f159c009dc0 tx=0x7f159c02a6d0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.663+0000 7f15aa400640 1 -- 172.21.15.7:0/2779131222 shutdown_connections 2024-04-25T11:22:26.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.663+0000 7f15aa400640 1 --2- 172.21.15.7:0/2779131222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15a406f1e0 0x7f15a406f5f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.663+0000 7f15aa400640 1 -- 172.21.15.7:0/2779131222 >> 172.21.15.7:0/2779131222 conn(0x7f15a40732b0 msgr2=0x7f15a40736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:26.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.664+0000 7f15aa400640 1 -- 172.21.15.7:0/2779131222 shutdown_connections 2024-04-25T11:22:26.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.664+0000 7f15aa400640 1 -- 172.21.15.7:0/2779131222 wait complete. 2024-04-25T11:22:26.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.666+0000 7f15aa400640 1 Processor -- start 2024-04-25T11:22:26.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.666+0000 7f15aa400640 1 -- start start 2024-04-25T11:22:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.667+0000 7f15aa400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15a406d580 0x7f15a409c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.667+0000 7f15aa400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15a406f1e0 0x7f15a415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.667+0000 7f15aa400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a415f180 0x7f15a4148a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15aa400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f15a415e670 con 0x7f15a415f180 2024-04-25T11:22:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15aa400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4158ae0 con 0x7f15a406d580 2024-04-25T11:22:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15aa400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f15a40d1310 con 0x7f15a406f1e0 2024-04-25T11:22:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15a3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a415f180 0x7f15a4148a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15a3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a415f180 0x7f15a4148a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48036/0 (socket says 172.21.15.7:48036) 2024-04-25T11:22:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15a3e00640 1 -- 172.21.15.7:0/1906867743 learned_addr learned my addr 172.21.15.7:0/1906867743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15a3400640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15a406d580 0x7f15a409c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f159be00640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15a406f1e0 0x7f15a415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15a3e00640 1 -- 172.21.15.7:0/1906867743 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15a406f1e0 msgr2=0x7f15a415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15a3e00640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15a406f1e0 0x7f15a415cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15a3e00640 1 -- 172.21.15.7:0/1906867743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15a406d580 msgr2=0x7f15a409c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15a3e00640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15a406d580 0x7f15a409c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.668+0000 7f15a3e00640 1 -- 172.21.15.7:0/1906867743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a409c580 con 0x7f15a415f180 2024-04-25T11:22:26.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.669+0000 7f15a3e00640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a415f180 0x7f15a4148a60 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f159c02acc0 tx=0x7f159c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:26.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.673+0000 7f159b400640 1 -- 172.21.15.7:0/1906867743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c03f070 con 0x7f15a415f180 2024-04-25T11:22:26.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.673+0000 7f15aa400640 1 -- 172.21.15.7:0/1906867743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a41f8230 con 0x7f15a415f180 2024-04-25T11:22:26.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.673+0000 7f159b400640 1 -- 172.21.15.7:0/1906867743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159c031bc0 con 0x7f15a415f180 2024-04-25T11:22:26.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.673+0000 7f159b400640 1 -- 172.21.15.7:0/1906867743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c009450 con 0x7f15a415f180 2024-04-25T11:22:26.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.673+0000 7f15aa400640 1 -- 172.21.15.7:0/1906867743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a41f87f0 con 0x7f15a415f180 2024-04-25T11:22:26.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.675+0000 7f15aa400640 1 -- 172.21.15.7:0/1906867743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a4050740 con 0x7f15a415f180 2024-04-25T11:22:26.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.676+0000 7f159b400640 1 -- 172.21.15.7:0/1906867743 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f159c03b410 con 0x7f15a415f180 2024-04-25T11:22:26.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.678+0000 7f159b400640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15800744b0 0x7f15800769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:26.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.678+0000 7f159b400640 1 -- 172.21.15.7:0/1906867743 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5335+0+0 (secure 0 0 0) 0x7f159c0b67b0 con 0x7f15a415f180 2024-04-25T11:22:26.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.679+0000 7f15a3400640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15800744b0 0x7f15800769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:26.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.679+0000 7f15a3400640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15800744b0 0x7f15800769c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1594000f60 tx=0x7f1594007d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:26.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.687+0000 7f159b400640 1 -- 172.21.15.7:0/1906867743 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f159c0809d0 con 0x7f15a415f180 2024-04-25T11:22:26.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:26.904+0000 7f15aa400640 1 -- 172.21.15.7:0/1906867743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f15a40b0d90 con 0x7f15a415f180 2024-04-25T11:22:27.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.708+0000 7f159b400640 1 -- 172.21.15.7:0/1906867743 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v152) ==== 97+0+0 (secure 0 0 0) 0x7f159c084cd0 con 0x7f15a415f180 2024-04-25T11:22:27.703 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-04-25T11:22:27.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.712+0000 7f15aa400640 1 -- 172.21.15.7:0/1906867743 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15800744b0 msgr2=0x7f15800769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:27.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.712+0000 7f15aa400640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15800744b0 0x7f15800769c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1594000f60 tx=0x7f1594007d70 comp rx=0 tx=0).stop 2024-04-25T11:22:27.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.712+0000 7f15aa400640 1 -- 172.21.15.7:0/1906867743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a415f180 msgr2=0x7f15a4148a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:27.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.712+0000 7f15aa400640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a415f180 0x7f15a4148a60 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f159c02acc0 tx=0x7f159c004a40 comp rx=0 tx=0).stop 2024-04-25T11:22:27.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.713+0000 7f15aa400640 1 -- 172.21.15.7:0/1906867743 shutdown_connections 2024-04-25T11:22:27.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.713+0000 7f15aa400640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f15800744b0 0x7f15800769c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:27.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.713+0000 7f15aa400640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15a415f180 0x7f15a4148a60 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:27.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.713+0000 7f15aa400640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15a406f1e0 0x7f15a415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:27.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.713+0000 7f15aa400640 1 --2- 172.21.15.7:0/1906867743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15a406d580 0x7f15a409c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:27.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.713+0000 7f15aa400640 1 -- 172.21.15.7:0/1906867743 >> 172.21.15.7:0/1906867743 conn(0x7f15a40732b0 msgr2=0x7f15a40b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:27.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.713+0000 7f15aa400640 1 -- 172.21.15.7:0/1906867743 shutdown_connections 2024-04-25T11:22:27.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:27.713+0000 7f15aa400640 1 -- 172.21.15.7:0/1906867743 wait complete. 2024-04-25T11:22:27.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T11:22:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.009+0000 7f6259600640 1 Processor -- start 2024-04-25T11:22:28.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.010+0000 7f6259600640 1 -- start start 2024-04-25T11:22:28.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.012+0000 7f6259600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f625406d580 0x7f625406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.012+0000 7f6259600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f625406f1e0 0x7f625406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.012+0000 7f6259600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f625406ded0 con 0x7f625415f180 2024-04-25T11:22:28.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.012+0000 7f6259600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f625415f590 con 0x7f625406f1e0 2024-04-25T11:22:28.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.012+0000 7f6259600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6254158ae0 con 0x7f625406d580 2024-04-25T11:22:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.013+0000 7f6253400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f625406f1e0 0x7f625406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.013+0000 7f624be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f625406d580 0x7f625406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.013+0000 7f624be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f625406d580 0x7f625406d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:36008/0 (socket says 172.21.15.7:36008) 2024-04-25T11:22:28.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.013+0000 7f624be00640 1 -- 172.21.15.7:0/136301182 learned_addr learned my addr 172.21.15.7:0/136301182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.014+0000 7f6253400640 1 -- 172.21.15.7:0/136301182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f625406d580 msgr2=0x7f625406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.014+0000 7f6253400640 1 --2- 172.21.15.7:0/136301182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f625406d580 0x7f625406d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.014+0000 7f6253400640 1 -- 172.21.15.7:0/136301182 >> v1:172.21.15.87:6789/0 conn(0x7f625415f180 legacy=0x7f6254161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.014+0000 7f6253400640 1 -- 172.21.15.7:0/136301182 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6254162700 con 0x7f625406f1e0 2024-04-25T11:22:28.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.015+0000 7f6253400640 1 --2- 172.21.15.7:0/136301182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f625406f1e0 0x7f625406f5f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f624c009dc0 tx=0x7f624c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e30e690e01f8b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:28.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.016+0000 7f6252000640 1 -- 172.21.15.7:0/136301182 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c02ae60 con 0x7f625406f1e0 2024-04-25T11:22:28.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.016+0000 7f6252000640 1 -- 172.21.15.7:0/136301182 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624c007740 con 0x7f625406f1e0 2024-04-25T11:22:28.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.016+0000 7f6252000640 1 -- 172.21.15.7:0/136301182 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c007b10 con 0x7f625406f1e0 2024-04-25T11:22:28.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.017+0000 7f6259600640 1 -- 172.21.15.7:0/136301182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f625406f1e0 msgr2=0x7f625406f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.017+0000 7f6259600640 1 --2- 172.21.15.7:0/136301182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f625406f1e0 0x7f625406f5f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f624c009dc0 tx=0x7f624c02a680 comp rx=0 tx=0).stop 2024-04-25T11:22:28.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.017+0000 7f6259600640 1 -- 172.21.15.7:0/136301182 shutdown_connections 2024-04-25T11:22:28.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.017+0000 7f6259600640 1 --2- 172.21.15.7:0/136301182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f625406f1e0 0x7f625406f5f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.017+0000 7f6259600640 1 --2- 172.21.15.7:0/136301182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f625406d580 0x7f625406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.017+0000 7f6259600640 1 -- 172.21.15.7:0/136301182 >> 172.21.15.7:0/136301182 conn(0x7f62540732b0 msgr2=0x7f62540736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:28.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.017+0000 7f6259600640 1 -- 172.21.15.7:0/136301182 shutdown_connections 2024-04-25T11:22:28.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.017+0000 7f6259600640 1 -- 172.21.15.7:0/136301182 wait complete. 2024-04-25T11:22:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.020+0000 7f6259600640 1 Processor -- start 2024-04-25T11:22:28.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.020+0000 7f6259600640 1 -- start start 2024-04-25T11:22:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.022+0000 7f6259600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f625406d580 0x7f6254155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.022+0000 7f6259600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f625406f1e0 0x7f625409c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.023+0000 7f6259600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625415f180 0x7f625409cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.023+0000 7f6259600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f625415ec60 con 0x7f625415f180 2024-04-25T11:22:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.023+0000 7f6259600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f625415e980 con 0x7f625406d580 2024-04-25T11:22:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.023+0000 7f6259600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f625415e670 con 0x7f625406f1e0 2024-04-25T11:22:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.023+0000 7f6253400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625415f180 0x7f625409cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.023+0000 7f6253400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625415f180 0x7f625409cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48040/0 (socket says 172.21.15.7:48040) 2024-04-25T11:22:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.023+0000 7f6253400640 1 -- 172.21.15.7:0/1589623155 learned_addr learned my addr 172.21.15.7:0/1589623155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.023+0000 7f6252a00640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f625406d580 0x7f6254155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.023+0000 7f624be00640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f625406f1e0 0x7f625409c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.024+0000 7f6253400640 1 -- 172.21.15.7:0/1589623155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f625406f1e0 msgr2=0x7f625409c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.024+0000 7f6253400640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f625406f1e0 0x7f625409c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.024+0000 7f6253400640 1 -- 172.21.15.7:0/1589623155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f625406d580 msgr2=0x7f6254155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.024+0000 7f6253400640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f625406d580 0x7f6254155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.024+0000 7f6253400640 1 -- 172.21.15.7:0/1589623155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62541564e0 con 0x7f625415f180 2024-04-25T11:22:28.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.024+0000 7f6252a00640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f625406d580 0x7f6254155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:22:28.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.024+0000 7f6253400640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625415f180 0x7f625409cd40 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f624c009ee0 tx=0x7f624c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.024+0000 7f624aa00640 1 -- 172.21.15.7:0/1589623155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c03f070 con 0x7f625415f180 2024-04-25T11:22:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.024+0000 7f624aa00640 1 -- 172.21.15.7:0/1589623155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624c02ae60 con 0x7f625415f180 2024-04-25T11:22:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.024+0000 7f624aa00640 1 -- 172.21.15.7:0/1589623155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c031be0 con 0x7f625415f180 2024-04-25T11:22:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.024+0000 7f6259600640 1 -- 172.21.15.7:0/1589623155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625409c8f0 con 0x7f625415f180 2024-04-25T11:22:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.025+0000 7f6259600640 1 -- 172.21.15.7:0/1589623155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625409ed40 con 0x7f625415f180 2024-04-25T11:22:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.026+0000 7f6259600640 1 -- 172.21.15.7:0/1589623155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6254050740 con 0x7f625415f180 2024-04-25T11:22:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.027+0000 7f624aa00640 1 -- 172.21.15.7:0/1589623155 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f624c03b410 con 0x7f625415f180 2024-04-25T11:22:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.028+0000 7f624aa00640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62300744b0 0x7f62300769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.028+0000 7f624aa00640 1 -- 172.21.15.7:0/1589623155 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5249+0+0 (secure 0 0 0) 0x7f624c0b6650 con 0x7f625415f180 2024-04-25T11:22:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.028+0000 7f6252a00640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62300744b0 0x7f62300769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.029+0000 7f6252a00640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62300744b0 0x7f62300769c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6254070310 tx=0x7f6244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.032+0000 7f624aa00640 1 -- 172.21.15.7:0/1589623155 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f624c04b020 con 0x7f625415f180 2024-04-25T11:22:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.317+0000 7f6259600640 1 -- 172.21.15.7:0/1589623155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f625409e4b0 con 0x7f625415f180 2024-04-25T11:22:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.318+0000 7f624aa00640 1 -- 172.21.15.7:0/1589623155 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7f62541564e0 con 0x7f625415f180 2024-04-25T11:22:28.313 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:28.313 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:22:28.313 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-04-25T11:22:28.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmLBe/MhAAQB7HOXK48piotFioQuqm3w==", 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-04-25T11:22:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmyayrHhAAKDfh+Avj6yk8cTrqOGuLaw==", 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmm+VcCxAAJSg2tNPc9myM+GtIMqsAXw==", 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:22:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDbOSpmeoo2MRAAmFXEi1d0C18NEAP08FGO5A==", 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.316 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDeOSpmFkKeKhAA29wdR7lJe0vVB3ic5kaVNA==", 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDiOSpmxNBLAhAAbusQ6p28fw7lRmEzYhx/cg==", 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDlOSpm5Pu3IRAAAmyBP5hl1PeImY+8eRypYw==", 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-04-25T11:22:28.318 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDpOSpmizCjBhAA354ZjXRIJFZCkx4pnfZ9IQ==", 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvOSpmY85EKhAA9vkorAvS6eFZrUvnhCpTlw==", 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDzOSpmUsi0DRAARhK1/QNunLI0CXKBGTAvng==", 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD2OSpm0oPGHBAAg8Lr7lDkSQZGf/5IVKAUAQ==", 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6OSpmBKg6ABAAxPFpsk/JLsKwHtzokg9M4A==", 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg==", 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-04-25T11:22:28.321 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm5IXGJRAAWor4bXf79OCrYW2gEO53MA==", 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.322 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpm78e+FhAA+93lzLJHV/0n2CBr1E+Qgg==", 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDcOSpmdeHtMxAAXArI/ezJxV5zgQrQyOarsQ==", 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-04-25T11:22:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-04-25T11:22:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDZOSpmS8ObHxAAkXXXg1OI1YFC2/OZER3woA==", 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpm30b2IhAAo/IgpazwQyod9rSnbJ88Tw==", 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:22:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:22:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-04-25T11:22:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-04-25T11:22:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDaOSpmd8YhEhAAiQJ87+dnzesecWQrTFXi8w==", 2024-04-25T11:22:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-04-25T11:22:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-04-25T11:22:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-04-25T11:22:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-04-25T11:22:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-04-25T11:22:28.327 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:22:28.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.321+0000 7f6259600640 1 -- 172.21.15.7:0/1589623155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62300744b0 msgr2=0x7f62300769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.321+0000 7f6259600640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62300744b0 0x7f62300769c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f6254070310 tx=0x7f6244009040 comp rx=0 tx=0).stop 2024-04-25T11:22:28.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.321+0000 7f6259600640 1 -- 172.21.15.7:0/1589623155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625415f180 msgr2=0x7f625409cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.321+0000 7f6259600640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625415f180 0x7f625409cd40 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f624c009ee0 tx=0x7f624c004a40 comp rx=0 tx=0).stop 2024-04-25T11:22:28.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.321+0000 7f6259600640 1 -- 172.21.15.7:0/1589623155 shutdown_connections 2024-04-25T11:22:28.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.321+0000 7f6259600640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62300744b0 0x7f62300769c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.321+0000 7f6259600640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f625415f180 0x7f625409cd40 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.321+0000 7f6259600640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f625406f1e0 0x7f625409c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.321+0000 7f6259600640 1 --2- 172.21.15.7:0/1589623155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f625406d580 0x7f6254155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.321+0000 7f6259600640 1 -- 172.21.15.7:0/1589623155 >> 172.21.15.7:0/1589623155 conn(0x7f62540732b0 msgr2=0x7f62540af670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:28.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.322+0000 7f6259600640 1 -- 172.21.15.7:0/1589623155 shutdown_connections 2024-04-25T11:22:28.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.322+0000 7f6259600640 1 -- 172.21.15.7:0/1589623155 wait complete. 2024-04-25T11:22:28.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.635+0000 7f3cc7e00640 1 Processor -- start 2024-04-25T11:22:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.635+0000 7f3cc7e00640 1 -- start start 2024-04-25T11:22:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.637+0000 7f3cc7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3cc80aed40 0x7f3cc815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.637+0000 7f3cc7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3cc80b5220 0x7f3cc80ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.637+0000 7f3cc7e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cc80b64b0 con 0x7f3cc80af420 2024-04-25T11:22:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.637+0000 7f3cc7e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f3cc815e1f0 con 0x7f3cc80aed40 2024-04-25T11:22:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.637+0000 7f3cc7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc806c610 con 0x7f3cc80b5220 2024-04-25T11:22:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.638+0000 7f3cc6a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3cc80aed40 0x7f3cc815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.638+0000 7f3cc6a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3cc80aed40 0x7f3cc815dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48044/0 (socket says 172.21.15.7:48044) 2024-04-25T11:22:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.638+0000 7f3cc6a00640 1 -- 172.21.15.7:0/2416209209 learned_addr learned my addr 172.21.15.7:0/2416209209 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.638+0000 7f3cc6000640 1 --2- 172.21.15.7:0/2416209209 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3cc80b5220 0x7f3cc80ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.639+0000 7f3cc6a00640 1 -- 172.21.15.7:0/2416209209 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3cc80b5220 msgr2=0x7f3cc80ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.639+0000 7f3cc6a00640 1 --2- 172.21.15.7:0/2416209209 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3cc80b5220 0x7f3cc80ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.639+0000 7f3cc6a00640 1 -- 172.21.15.7:0/2416209209 >> v1:172.21.15.87:6789/0 conn(0x7f3cc80af420 legacy=0x7f3cc80af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.639+0000 7f3cc6a00640 1 -- 172.21.15.7:0/2416209209 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc806c8f0 con 0x7f3cc80aed40 2024-04-25T11:22:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.640+0000 7f3cc6a00640 1 --2- 172.21.15.7:0/2416209209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc80aed40 0x7f3cc815dcb0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f3cc815e370 tx=0x7f3cb802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392c0afa905a19da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.641+0000 7f3cc5600640 1 -- 172.21.15.7:0/2416209209 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8037070 con 0x7f3cc80aed40 2024-04-25T11:22:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.641+0000 7f3cc5600640 1 -- 172.21.15.7:0/2416209209 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb8032440 con 0x7f3cc80aed40 2024-04-25T11:22:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.641+0000 7f3cc5600640 1 -- 172.21.15.7:0/2416209209 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8032810 con 0x7f3cc80aed40 2024-04-25T11:22:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.641+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2416209209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc80aed40 msgr2=0x7f3cc815dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.641+0000 7f3cc7e00640 1 --2- 172.21.15.7:0/2416209209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc80aed40 0x7f3cc815dcb0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f3cc815e370 tx=0x7f3cb802a250 comp rx=0 tx=0).stop 2024-04-25T11:22:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.642+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2416209209 shutdown_connections 2024-04-25T11:22:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.642+0000 7f3cc7e00640 1 --2- 172.21.15.7:0/2416209209 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3cc80b5220 0x7f3cc80ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.642+0000 7f3cc7e00640 1 --2- 172.21.15.7:0/2416209209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc80aed40 0x7f3cc815dcb0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.642+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2416209209 >> 172.21.15.7:0/2416209209 conn(0x7f3cc80b3820 msgr2=0x7f3cc80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.642+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2416209209 shutdown_connections 2024-04-25T11:22:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.642+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2416209209 wait complete. 2024-04-25T11:22:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.644+0000 7f3cc7e00640 1 Processor -- start 2024-04-25T11:22:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.644+0000 7f3cc7e00640 1 -- start start 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.646+0000 7f3cc7e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc80aed40 0x7f3cc81e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.646+0000 7f3cc7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc80af420 0x7f3cc815a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.646+0000 7f3cc7e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3cc80b5220 0x7f3cc8146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.646+0000 7f3cc7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc80b6110 con 0x7f3cc80af420 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.646+0000 7f3cc7e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc806ceb0 con 0x7f3cc80aed40 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.646+0000 7f3cc7e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc80b6290 con 0x7f3cc80b5220 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.646+0000 7f3cc6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc80af420 0x7f3cc815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.646+0000 7f3cc6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc80af420 0x7f3cc815a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48060/0 (socket says 172.21.15.7:48060) 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.646+0000 7f3cc6000640 1 -- 172.21.15.7:0/2793872271 learned_addr learned my addr 172.21.15.7:0/2793872271 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.647+0000 7f3cc6a00640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc80aed40 0x7f3cc81e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.647+0000 7f3cc7400640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3cc80b5220 0x7f3cc8146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.647+0000 7f3cc6000640 1 -- 172.21.15.7:0/2793872271 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3cc80b5220 msgr2=0x7f3cc8146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.647+0000 7f3cc6000640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3cc80b5220 0x7f3cc8146bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.647+0000 7f3cc6000640 1 -- 172.21.15.7:0/2793872271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc80aed40 msgr2=0x7f3cc81e7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.647+0000 7f3cc6000640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc80aed40 0x7f3cc81e7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.647+0000 7f3cc6000640 1 -- 172.21.15.7:0/2793872271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc815aba0 con 0x7f3cc80af420 2024-04-25T11:22:28.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.647+0000 7f3cc6a00640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc80aed40 0x7f3cc81e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.647+0000 7f3cc6000640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc80af420 0x7f3cc815a660 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4007e60 tx=0x7f3cb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.647+0000 7f3cbea00640 1 -- 172.21.15.7:0/2793872271 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4014620 con 0x7f3cc80af420 2024-04-25T11:22:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.648+0000 7f3cbea00640 1 -- 172.21.15.7:0/2793872271 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb4019070 con 0x7f3cc80af420 2024-04-25T11:22:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.648+0000 7f3cbea00640 1 -- 172.21.15.7:0/2793872271 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb4014a20 con 0x7f3cc80af420 2024-04-25T11:22:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.648+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2793872271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc80cd7c0 con 0x7f3cc80af420 2024-04-25T11:22:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.648+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2793872271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc81e75b0 con 0x7f3cc80af420 2024-04-25T11:22:28.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.649+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2793872271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc80cd530 con 0x7f3cc80af420 2024-04-25T11:22:28.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.650+0000 7f3cbea00640 1 -- 172.21.15.7:0/2793872271 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3cb4014bc0 con 0x7f3cc80af420 2024-04-25T11:22:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.651+0000 7f3cbea00640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ca80744b0 0x7f3ca80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.651+0000 7f3cbea00640 1 -- 172.21.15.7:0/2793872271 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5249+0+0 (secure 0 0 0) 0x7f3cb4095a70 con 0x7f3cc80af420 2024-04-25T11:22:28.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.651+0000 7f3cc6a00640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ca80744b0 0x7f3ca80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:28.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.652+0000 7f3cc6a00640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ca80744b0 0x7f3ca80769c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8034000 tx=0x7f3cb802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:28.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.656+0000 7f3cbea00640 1 -- 172.21.15.7:0/2793872271 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3cb4060320 con 0x7f3cc80af420 2024-04-25T11:22:28.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.865+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2793872271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cc81e70c0 con 0x7f3cc80af420 2024-04-25T11:22:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.867+0000 7f3cbea00640 1 -- 172.21.15.7:0/2793872271 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+13078 (secure 0 0 0) 0x7f3cb405fc20 con 0x7f3cc80af420 2024-04-25T11:22:28.862 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:28.863 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":152,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:27.704652+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:28.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2793872271 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ca80744b0 msgr2=0x7f3ca80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ca80744b0 0x7f3ca80769c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8034000 tx=0x7f3cb802e000 comp rx=0 tx=0).stop 2024-04-25T11:22:28.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2793872271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc80af420 msgr2=0x7f3cc815a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:28.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc80af420 0x7f3cc815a660 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4007e60 tx=0x7f3cb400e4b0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2793872271 shutdown_connections 2024-04-25T11:22:28.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3ca80744b0 0x7f3ca80769c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3cc80b5220 0x7f3cc8146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc80af420 0x7f3cc815a660 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 --2- 172.21.15.7:0/2793872271 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3cc80aed40 0x7f3cc81e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:28.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2793872271 >> 172.21.15.7:0/2793872271 conn(0x7f3cc80b3820 msgr2=0x7f3cc80b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:28.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2793872271 shutdown_connections 2024-04-25T11:22:28.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:28.870+0000 7f3cc7e00640 1 -- 172.21.15.7:0/2793872271 wait complete. 2024-04-25T11:22:28.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:22:28.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:22:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.173+0000 7f928f600640 1 Processor -- start 2024-04-25T11:22:29.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.173+0000 7f928f600640 1 -- start start 2024-04-25T11:22:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.175+0000 7f928f600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f92880affc0 0x7f9288167d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.175+0000 7f928f600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92880b5730 con 0x7f928815aaa0 2024-04-25T11:22:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.175+0000 7f928f600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f928806f5b0 con 0x7f92880d0ee0 2024-04-25T11:22:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.175+0000 7f928f600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f92880af290 con 0x7f92880affc0 2024-04-25T11:22:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.176+0000 7f928cc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f928815aaa0 0x7f92880b4910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41148/0 (socket says 172.21.15.7:41148) 2024-04-25T11:22:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.176+0000 7f928cc00640 1 -- 172.21.15.7:0/3798163122 learned_addr learned my addr 172.21.15.7:0/3798163122 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.176+0000 7f928d600640 1 --2- 172.21.15.7:0/3798163122 >> v2:172.21.15.87:3300/0 conn(0x7f92880affc0 0x7f9288167d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.176+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1983151912 0 0) 0x7f9278001110 con 0x7f928815aaa0 2024-04-25T11:22:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.176+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f926c003520 con 0x7f928815aaa0 2024-04-25T11:22:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 981581458 0 0) 0x7f926c003520 con 0x7f928815aaa0 2024-04-25T11:22:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 >> v2:172.21.15.87:3300/0 conn(0x7f92880affc0 msgr2=0x7f9288167d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 --2- 172.21.15.7:0/3798163122 >> v2:172.21.15.87:3300/0 conn(0x7f92880affc0 0x7f9288167d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 >> v1:172.21.15.87:6789/0 conn(0x7f92880d0ee0 legacy=0x7f92880b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92880d14a0 con 0x7f928815aaa0 2024-04-25T11:22:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f9278001840 con 0x7f928815aaa0 2024-04-25T11:22:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 >> v1:172.21.15.7:6789/0 conn(0x7f928815aaa0 legacy=0x7f92880b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 --2- 172.21.15.7:0/3798163122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f926c003a00 0x7f926c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 --2- 172.21.15.7:0/3798163122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f926c004800 0x7f926c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 --2- 172.21.15.7:0/3798163122 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f926c0052a0 0x7f926c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92880af290 con 0x7f926c003a00 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f926c00c230 con 0x7f926c004800 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f926c00c430 con 0x7f926c0052a0 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.177+0000 7f928cc00640 1 --2- 172.21.15.7:0/3798163122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f926c003a00 0x7f926c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.178+0000 7f9287e00640 1 --2- 172.21.15.7:0/3798163122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f926c004800 0x7f926c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.178+0000 7f928cc00640 1 -- 172.21.15.7:0/3798163122 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f926c0052a0 msgr2=0x7f926c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.178+0000 7f928cc00640 1 --2- 172.21.15.7:0/3798163122 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f926c0052a0 0x7f926c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.178+0000 7f928cc00640 1 -- 172.21.15.7:0/3798163122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f926c004800 msgr2=0x7f926c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.178+0000 7f928cc00640 1 --2- 172.21.15.7:0/3798163122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f926c004800 0x7f926c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.178+0000 7f928cc00640 1 -- 172.21.15.7:0/3798163122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c003520 con 0x7f926c003a00 2024-04-25T11:22:29.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.178+0000 7f928d600640 1 -- 172.21.15.7:0/3798163122 reap_dead start 2024-04-25T11:22:29.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.178+0000 7f9287e00640 1 --2- 172.21.15.7:0/3798163122 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f926c004800 0x7f926c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.179+0000 7f928cc00640 1 --2- 172.21.15.7:0/3798163122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f926c003a00 0x7f926c003e70 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f927800c320 tx=0x7f926c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93288a4648b7ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.179+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278036070 con 0x7f926c003a00 2024-04-25T11:22:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.179+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f927802d070 con 0x7f926c003a00 2024-04-25T11:22:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.179+0000 7f9287400640 1 -- 172.21.15.7:0/3798163122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278007e10 con 0x7f926c003a00 2024-04-25T11:22:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.180+0000 7f928f600640 1 -- 172.21.15.7:0/3798163122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f926c003a00 msgr2=0x7f926c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.180+0000 7f928f600640 1 --2- 172.21.15.7:0/3798163122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f926c003a00 0x7f926c003e70 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f927800c320 tx=0x7f926c005830 comp rx=0 tx=0).stop 2024-04-25T11:22:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.180+0000 7f928f600640 1 -- 172.21.15.7:0/3798163122 shutdown_connections 2024-04-25T11:22:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.180+0000 7f928f600640 1 --2- 172.21.15.7:0/3798163122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f926c003a00 0x7f926c003e70 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.180+0000 7f928f600640 1 -- 172.21.15.7:0/3798163122 >> 172.21.15.7:0/3798163122 conn(0x7f92880742c0 msgr2=0x7f928806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.180+0000 7f928f600640 1 -- 172.21.15.7:0/3798163122 shutdown_connections 2024-04-25T11:22:29.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.180+0000 7f928f600640 1 -- 172.21.15.7:0/3798163122 wait complete. 2024-04-25T11:22:29.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.182+0000 7f928f600640 1 Processor -- start 2024-04-25T11:22:29.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.182+0000 7f928f600640 1 -- start start 2024-04-25T11:22:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.184+0000 7f928f600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f926c003a00 0x7f9288154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.184+0000 7f928f600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f928814d640 0x7f928815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.184+0000 7f928f600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92881597f0 0x7f9288159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.184+0000 7f928f600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92880afaa0 con 0x7f928814d640 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.184+0000 7f928f600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f92880af7c0 con 0x7f92881597f0 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.184+0000 7f928f600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f92880b4910 con 0x7f926c003a00 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.184+0000 7f9287e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f928814d640 0x7f928815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.184+0000 7f9287e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f928814d640 0x7f928815eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48076/0 (socket says 172.21.15.7:48076) 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.184+0000 7f9287e00640 1 -- 172.21.15.7:0/1128893522 learned_addr learned my addr 172.21.15.7:0/1128893522 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.184+0000 7f928cc00640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f926c003a00 0x7f9288154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f9287e00640 1 -- 172.21.15.7:0/1128893522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f926c003a00 msgr2=0x7f9288154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f9287e00640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f926c003a00 0x7f9288154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f9287e00640 1 -- 172.21.15.7:0/1128893522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92881597f0 msgr2=0x7f9288159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f928d600640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92881597f0 0x7f9288159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f9287e00640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92881597f0 0x7f9288159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f9287e00640 1 -- 172.21.15.7:0/1128893522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928814da50 con 0x7f928814d640 2024-04-25T11:22:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f9287e00640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f928814d640 0x7f928815eee0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f927400d480 tx=0x7f927400d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f9284c00640 1 -- 172.21.15.7:0/1128893522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274005480 con 0x7f928814d640 2024-04-25T11:22:29.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f9284c00640 1 -- 172.21.15.7:0/1128893522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9274017070 con 0x7f928814d640 2024-04-25T11:22:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f9284c00640 1 -- 172.21.15.7:0/1128893522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274003850 con 0x7f928814d640 2024-04-25T11:22:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f928f600640 1 -- 172.21.15.7:0/1128893522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9288160fb0 con 0x7f928814d640 2024-04-25T11:22:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.185+0000 7f928f600640 1 -- 172.21.15.7:0/1128893522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9288158ff0 con 0x7f928814d640 2024-04-25T11:22:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.186+0000 7f928f600640 1 -- 172.21.15.7:0/1128893522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9288050740 con 0x7f928814d640 2024-04-25T11:22:29.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.188+0000 7f9284c00640 1 -- 172.21.15.7:0/1128893522 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f92740039f0 con 0x7f928814d640 2024-04-25T11:22:29.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.189+0000 7f9284c00640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9268074530 0x7f9268076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.189+0000 7f9284c00640 1 -- 172.21.15.7:0/1128893522 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5249+0+0 (secure 0 0 0) 0x7f9274094d70 con 0x7f928814d640 2024-04-25T11:22:29.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.189+0000 7f928cc00640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9268074530 0x7f9268076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.190+0000 7f928cc00640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9268074530 0x7f9268076a40 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9278033000 tx=0x7f927802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:29.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.193+0000 7f9284c00640 1 -- 172.21.15.7:0/1128893522 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f927405f7f0 con 0x7f928814d640 2024-04-25T11:22:29.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.398+0000 7f928f600640 1 -- 172.21.15.7:0/1128893522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9288070f20 con 0x7f928814d640 2024-04-25T11:22:29.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.399+0000 7f9284c00640 1 -- 172.21.15.7:0/1128893522 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v152) ==== 93+0+10 (secure 0 0 0) 0x7f927405f0f0 con 0x7f928814d640 2024-04-25T11:22:29.394 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:22:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.401+0000 7f928f600640 1 -- 172.21.15.7:0/1128893522 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9268074530 msgr2=0x7f9268076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.401+0000 7f928f600640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9268074530 0x7f9268076a40 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9278033000 tx=0x7f927802e000 comp rx=0 tx=0).stop 2024-04-25T11:22:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.402+0000 7f928f600640 1 -- 172.21.15.7:0/1128893522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f928814d640 msgr2=0x7f928815eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.402+0000 7f928f600640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f928814d640 0x7f928815eee0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f927400d480 tx=0x7f927400d940 comp rx=0 tx=0).stop 2024-04-25T11:22:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.402+0000 7f928f600640 1 -- 172.21.15.7:0/1128893522 shutdown_connections 2024-04-25T11:22:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.402+0000 7f928f600640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9268074530 0x7f9268076a40 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.402+0000 7f928f600640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92881597f0 0x7f9288159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.402+0000 7f928f600640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f928814d640 0x7f928815eee0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.402+0000 7f928f600640 1 --2- 172.21.15.7:0/1128893522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f926c003a00 0x7f9288154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.402+0000 7f928f600640 1 -- 172.21.15.7:0/1128893522 >> 172.21.15.7:0/1128893522 conn(0x7f92880742c0 msgr2=0x7f928806d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:29.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.402+0000 7f928f600640 1 -- 172.21.15.7:0/1128893522 shutdown_connections 2024-04-25T11:22:29.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.402+0000 7f928f600640 1 -- 172.21.15.7:0/1128893522 wait complete. 2024-04-25T11:22:29.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T11:22:29.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.712+0000 7f9afd600640 1 Processor -- start 2024-04-25T11:22:29.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.712+0000 7f9afd600640 1 -- start start 2024-04-25T11:22:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.715+0000 7f9afd600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9af8070810 0x7f9af806d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.715+0000 7f9afd600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9af80cffa0 con 0x7f9af8070c20 2024-04-25T11:22:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.715+0000 7f9afd600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9af806ded0 con 0x7f9af806dac0 2024-04-25T11:22:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.715+0000 7f9afd600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9af8160d90 con 0x7f9af8070810 2024-04-25T11:22:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.715+0000 7f9af7400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9af8070c20 0x7f9af815d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41164/0 (socket says 172.21.15.7:41164) 2024-04-25T11:22:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.715+0000 7f9af7400640 1 -- 172.21.15.7:0/3341071273 learned_addr learned my addr 172.21.15.7:0/3341071273 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:29.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.715+0000 7f9af7e00640 1 --2- 172.21.15.7:0/3341071273 >> v2:172.21.15.87:3300/0 conn(0x7f9af8070810 0x7f9af806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.716+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 718630962 0 0) 0x7f9ae0001110 con 0x7f9af8070c20 2024-04-25T11:22:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.716+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9adc003520 con 0x7f9af8070c20 2024-04-25T11:22:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.716+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 335066349 0 0) 0x7f9adc003520 con 0x7f9af8070c20 2024-04-25T11:22:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.716+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 >> v2:172.21.15.87:3300/0 conn(0x7f9af8070810 msgr2=0x7f9af806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.716+0000 7f9af6a00640 1 --2- 172.21.15.7:0/3341071273 >> v2:172.21.15.87:3300/0 conn(0x7f9af8070810 0x7f9af806d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.716+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 >> v1:172.21.15.87:6789/0 conn(0x7f9af806dac0 legacy=0x7f9af815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.716+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af8161070 con 0x7f9af8070c20 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.716+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f9ae0001840 con 0x7f9af8070c20 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.716+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 >> v1:172.21.15.7:6789/0 conn(0x7f9af8070c20 legacy=0x7f9af815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.716+0000 7f9af6a00640 1 --2- 172.21.15.7:0/3341071273 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9adc003a00 0x7f9adc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9af6a00640 1 --2- 172.21.15.7:0/3341071273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9adc004800 0x7f9adc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9af6a00640 1 --2- 172.21.15.7:0/3341071273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc005270 0x7f9adc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8160d90 con 0x7f9adc005270 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc00e3a0 con 0x7f9adc004800 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9adc00e5a0 con 0x7f9adc003a00 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9afcc00640 1 --2- 172.21.15.7:0/3341071273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc005270 0x7f9adc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9af7400640 1 --2- 172.21.15.7:0/3341071273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9adc004800 0x7f9adc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9af7e00640 1 --2- 172.21.15.7:0/3341071273 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9adc003a00 0x7f9adc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9afcc00640 1 -- 172.21.15.7:0/3341071273 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9adc003a00 msgr2=0x7f9adc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9afcc00640 1 --2- 172.21.15.7:0/3341071273 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9adc003a00 0x7f9adc003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9afcc00640 1 -- 172.21.15.7:0/3341071273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9adc004800 msgr2=0x7f9adc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9afcc00640 1 --2- 172.21.15.7:0/3341071273 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9adc004800 0x7f9adc004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.717+0000 7f9afcc00640 1 -- 172.21.15.7:0/3341071273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9adc003520 con 0x7f9adc005270 2024-04-25T11:22:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.718+0000 7f9afcc00640 1 -- 172.21.15.7:0/3341071273 reap_dead start 2024-04-25T11:22:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.718+0000 7f9afcc00640 1 --2- 172.21.15.7:0/3341071273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc005270 0x7f9adc00dbd0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f9af0001720 tx=0x7f9adc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a34fc25307fac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.719+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0001370 con 0x7f9adc005270 2024-04-25T11:22:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.719+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af00052d0 con 0x7f9adc005270 2024-04-25T11:22:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.719+0000 7f9af6a00640 1 -- 172.21.15.7:0/3341071273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af002abc0 con 0x7f9adc005270 2024-04-25T11:22:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.719+0000 7f9afd600640 1 -- 172.21.15.7:0/3341071273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc005270 msgr2=0x7f9adc00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.719+0000 7f9afd600640 1 --2- 172.21.15.7:0/3341071273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc005270 0x7f9adc00dbd0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f9af0001720 tx=0x7f9adc005830 comp rx=0 tx=0).stop 2024-04-25T11:22:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.720+0000 7f9afd600640 1 -- 172.21.15.7:0/3341071273 shutdown_connections 2024-04-25T11:22:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.720+0000 7f9afd600640 1 --2- 172.21.15.7:0/3341071273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc005270 0x7f9adc00dbd0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.720+0000 7f9afd600640 1 -- 172.21.15.7:0/3341071273 >> 172.21.15.7:0/3341071273 conn(0x7f9af8073090 msgr2=0x7f9af80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.720+0000 7f9afd600640 1 -- 172.21.15.7:0/3341071273 shutdown_connections 2024-04-25T11:22:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.720+0000 7f9afd600640 1 -- 172.21.15.7:0/3341071273 wait complete. 2024-04-25T11:22:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.722+0000 7f9afd600640 1 Processor -- start 2024-04-25T11:22:29.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.722+0000 7f9afd600640 1 -- start start 2024-04-25T11:22:29.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.724+0000 7f9afd600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9adc005270 0x7f9af815d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.724+0000 7f9afd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af8142560 0x7f9af815ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.724+0000 7f9afd600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9af815b5d0 0x7f9af815b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.724+0000 7f9afd600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81433b0 con 0x7f9af8142560 2024-04-25T11:22:29.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.724+0000 7f9afd600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8161630 con 0x7f9af815b5d0 2024-04-25T11:22:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.724+0000 7f9afd600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9af8143530 con 0x7f9adc005270 2024-04-25T11:22:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.724+0000 7f9af7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af8142560 0x7f9af815ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.724+0000 7f9af7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af8142560 0x7f9af815ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48098/0 (socket says 172.21.15.7:48098) 2024-04-25T11:22:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.724+0000 7f9af7400640 1 -- 172.21.15.7:0/3054790263 learned_addr learned my addr 172.21.15.7:0/3054790263 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.724+0000 7f9af7e00640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9adc005270 0x7f9af815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.725+0000 7f9afcc00640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9af815b5d0 0x7f9af815b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.725+0000 7f9af7400640 1 -- 172.21.15.7:0/3054790263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9adc005270 msgr2=0x7f9af815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.725+0000 7f9af7400640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9adc005270 0x7f9af815d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.725+0000 7f9af7400640 1 -- 172.21.15.7:0/3054790263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9af815b5d0 msgr2=0x7f9af815b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.725+0000 7f9af7400640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9af815b5d0 0x7f9af815b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.725+0000 7f9af7400640 1 -- 172.21.15.7:0/3054790263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af815d880 con 0x7f9af8142560 2024-04-25T11:22:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.725+0000 7f9af7400640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af8142560 0x7f9af815ee70 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000e780 tx=0x7f9ae000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.726+0000 7f9aefe00640 1 -- 172.21.15.7:0/3054790263 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000c660 con 0x7f9af8142560 2024-04-25T11:22:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.726+0000 7f9aefe00640 1 -- 172.21.15.7:0/3054790263 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae0018070 con 0x7f9af8142560 2024-04-25T11:22:29.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.726+0000 7f9aefe00640 1 -- 172.21.15.7:0/3054790263 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000ca60 con 0x7f9af8142560 2024-04-25T11:22:29.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.726+0000 7f9afd600640 1 -- 172.21.15.7:0/3054790263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af8160050 con 0x7f9af8142560 2024-04-25T11:22:29.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.726+0000 7f9afd600640 1 -- 172.21.15.7:0/3054790263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af815e670 con 0x7f9af8142560 2024-04-25T11:22:29.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.727+0000 7f9afd600640 1 -- 172.21.15.7:0/3054790263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af815fdc0 con 0x7f9af8142560 2024-04-25T11:22:29.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.728+0000 7f9aefe00640 1 -- 172.21.15.7:0/3054790263 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ae000cc00 con 0x7f9af8142560 2024-04-25T11:22:29.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.729+0000 7f9aefe00640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ad40744b0 0x7f9ad40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:29.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.729+0000 7f9aefe00640 1 -- 172.21.15.7:0/3054790263 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5249+0+0 (secure 0 0 0) 0x7f9ae0094e70 con 0x7f9af8142560 2024-04-25T11:22:29.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.730+0000 7f9af7e00640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ad40744b0 0x7f9ad40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:29.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.730+0000 7f9af7e00640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ad40744b0 0x7f9ad40769c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80074a0 tx=0x7f9ae800d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:29.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.734+0000 7f9aefe00640 1 -- 172.21.15.7:0/3054790263 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9af80cffa0 con 0x7f9af8142560 2024-04-25T11:22:29.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.932+0000 7f9afd600640 1 -- 172.21.15.7:0/3054790263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9af8160570 con 0x7f9af8142560 2024-04-25T11:22:29.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.933+0000 7f9aefe00640 1 -- 172.21.15.7:0/3054790263 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v152) ==== 84+0+65 (secure 0 0 0) 0x7f9ae005f720 con 0x7f9af8142560 2024-04-25T11:22:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:29.928 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-04-25T11:22:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-04-25T11:22:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-04-25T11:22:29.928 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-04-25T11:22:29.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-04-25T11:22:29.929 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-04-25T11:22:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 -- 172.21.15.7:0/3054790263 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ad40744b0 msgr2=0x7f9ad40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ad40744b0 0x7f9ad40769c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80074a0 tx=0x7f9ae800d040 comp rx=0 tx=0).stop 2024-04-25T11:22:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 -- 172.21.15.7:0/3054790263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af8142560 msgr2=0x7f9af815ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af8142560 0x7f9af815ee70 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000e780 tx=0x7f9ae000ec40 comp rx=0 tx=0).stop 2024-04-25T11:22:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 -- 172.21.15.7:0/3054790263 shutdown_connections 2024-04-25T11:22:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9ad40744b0 0x7f9ad40769c0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9af815b5d0 0x7f9af815b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9af8142560 0x7f9af815ee70 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 --2- 172.21.15.7:0/3054790263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9adc005270 0x7f9af815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 -- 172.21.15.7:0/3054790263 >> 172.21.15.7:0/3054790263 conn(0x7f9af8073090 msgr2=0x7f9af806cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 -- 172.21.15.7:0/3054790263 shutdown_connections 2024-04-25T11:22:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:29.936+0000 7f9afd600640 1 -- 172.21.15.7:0/3054790263 wait complete. 2024-04-25T11:22:29.957 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T11:22:29.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T11:22:30.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.237+0000 7f3f51000640 1 Processor -- start 2024-04-25T11:22:30.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.237+0000 7f3f51000640 1 -- start start 2024-04-25T11:22:30.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.239+0000 7f3f51000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3f4c09c680 0x7f3f4c06d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:30.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.239+0000 7f3f51000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4c06ddf0 con 0x7f3f4c0b91c0 2024-04-25T11:22:30.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.239+0000 7f3f51000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f4c0ce070 con 0x7f3f4c0c7060 2024-04-25T11:22:30.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.239+0000 7f3f51000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3f4c06dbc0 con 0x7f3f4c09c680 2024-04-25T11:22:30.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.240+0000 7f3f49e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3f4c0b91c0 0x7f3f4c0c6950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39914/0 (socket says 172.21.15.7:39914) 2024-04-25T11:22:30.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.240+0000 7f3f49e00640 1 -- 172.21.15.7:0/1769064457 learned_addr learned my addr 172.21.15.7:0/1769064457 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:30.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.240+0000 7f3f4a800640 1 --2- 172.21.15.7:0/1769064457 >> v2:172.21.15.87:3300/0 conn(0x7f3f4c09c680 0x7f3f4c06d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:30.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.240+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4017654393 0 0) 0x7f3f4c06ddf0 con 0x7f3f4c0b91c0 2024-04-25T11:22:30.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.240+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f30003520 con 0x7f3f4c0b91c0 2024-04-25T11:22:30.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 851213115 0 0) 0x7f3f30003520 con 0x7f3f4c0b91c0 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 >> v2:172.21.15.87:3300/0 conn(0x7f3f4c09c680 msgr2=0x7f3f4c06d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 --2- 172.21.15.7:0/1769064457 >> v2:172.21.15.87:3300/0 conn(0x7f3f4c09c680 0x7f3f4c06d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 >> v1:172.21.15.87:6789/0 conn(0x7f3f4c0c7060 legacy=0x7f3f4c0b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c0ce350 con 0x7f3f4c0b91c0 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f3f3c0019f0 con 0x7f3f4c0b91c0 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 >> v1:172.21.15.7:6789/0 conn(0x7f3f4c0b91c0 legacy=0x7f3f4c0c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 --2- 172.21.15.7:0/1769064457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f30003a00 0x7f3f30003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 --2- 172.21.15.7:0/1769064457 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f30004800 0x7f3f30004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 --2- 172.21.15.7:0/1769064457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f30005270 0x7f3f3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3000c160 con 0x7f3f30003a00 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3000c310 con 0x7f3f30005270 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3000c510 con 0x7f3f30004800 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.241+0000 7f3f49e00640 1 --2- 172.21.15.7:0/1769064457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f30003a00 0x7f3f30003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:30.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.242+0000 7f3f49e00640 1 -- 172.21.15.7:0/1769064457 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f30004800 msgr2=0x7f3f30004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:30.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.242+0000 7f3f49e00640 1 --2- 172.21.15.7:0/1769064457 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f30004800 0x7f3f30004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:30.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.242+0000 7f3f49e00640 1 -- 172.21.15.7:0/1769064457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f30005270 msgr2=0x7f3f3000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:30.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.242+0000 7f3f49e00640 1 --2- 172.21.15.7:0/1769064457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f30005270 0x7f3f3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:30.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.242+0000 7f3f4a800640 1 -- 172.21.15.7:0/1769064457 reap_dead start 2024-04-25T11:22:30.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.242+0000 7f3f49e00640 1 -- 172.21.15.7:0/1769064457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f30003520 con 0x7f3f30003a00 2024-04-25T11:22:30.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.243+0000 7f3f49e00640 1 --2- 172.21.15.7:0/1769064457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f30003a00 0x7f3f30003e70 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00a760 tx=0x7f3f30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83152747b739432b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:30.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.243+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00a3b0 con 0x7f3f30003a00 2024-04-25T11:22:30.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.243+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3c02f070 con 0x7f3f30003a00 2024-04-25T11:22:30.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.243+0000 7f3f48a00640 1 -- 172.21.15.7:0/1769064457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c02bc20 con 0x7f3f30003a00 2024-04-25T11:22:30.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.244+0000 7f3f51000640 1 -- 172.21.15.7:0/1769064457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f30003a00 msgr2=0x7f3f30003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:30.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.244+0000 7f3f51000640 1 --2- 172.21.15.7:0/1769064457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f30003a00 0x7f3f30003e70 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00a760 tx=0x7f3f30005830 comp rx=0 tx=0).stop 2024-04-25T11:22:30.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.244+0000 7f3f51000640 1 -- 172.21.15.7:0/1769064457 shutdown_connections 2024-04-25T11:22:30.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.244+0000 7f3f51000640 1 --2- 172.21.15.7:0/1769064457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f30003a00 0x7f3f30003e70 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:30.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.244+0000 7f3f51000640 1 -- 172.21.15.7:0/1769064457 >> 172.21.15.7:0/1769064457 conn(0x7f3f4c0b5470 msgr2=0x7f3f4c0b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:30.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.244+0000 7f3f51000640 1 -- 172.21.15.7:0/1769064457 shutdown_connections 2024-04-25T11:22:30.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.244+0000 7f3f51000640 1 -- 172.21.15.7:0/1769064457 wait complete. 2024-04-25T11:22:30.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.246+0000 7f3f51000640 1 Processor -- start 2024-04-25T11:22:30.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.246+0000 7f3f51000640 1 -- start start 2024-04-25T11:22:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.248+0000 7f3f51000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f30003a00 0x7f3f4c159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.248+0000 7f3f51000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c162270 0x7f3f4c0b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.248+0000 7f3f51000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f4c162680 0x7f3f4c0b7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.248+0000 7f3f51000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c09ca90 con 0x7f3f4c162270 2024-04-25T11:22:30.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.248+0000 7f3f51000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c0ce9a0 con 0x7f3f30003a00 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.248+0000 7f3f51000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4c0b3270 con 0x7f3f4c162680 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.248+0000 7f3f49400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c162270 0x7f3f4c0b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.248+0000 7f3f49400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c162270 0x7f3f4c0b7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34410/0 (socket says 172.21.15.7:34410) 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.248+0000 7f3f49400640 1 -- 172.21.15.7:0/1034302371 learned_addr learned my addr 172.21.15.7:0/1034302371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f49e00640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f30003a00 0x7f3f4c159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f4a800640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f4c162680 0x7f3f4c0b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f49400640 1 -- 172.21.15.7:0/1034302371 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f4c162680 msgr2=0x7f3f4c0b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f49400640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f4c162680 0x7f3f4c0b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f49400640 1 -- 172.21.15.7:0/1034302371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f30003a00 msgr2=0x7f3f4c159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f49400640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f30003a00 0x7f3f4c159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f49400640 1 -- 172.21.15.7:0/1034302371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c15a1f0 con 0x7f3f4c162270 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f49400640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c162270 0x7f3f4c0b7420 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c430 tx=0x7f3f3400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f42000640 1 -- 172.21.15.7:0/1034302371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34017070 con 0x7f3f4c162270 2024-04-25T11:22:30.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f42000640 1 -- 172.21.15.7:0/1034302371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f340052f0 con 0x7f3f4c162270 2024-04-25T11:22:30.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f42000640 1 -- 172.21.15.7:0/1034302371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34003830 con 0x7f3f4c162270 2024-04-25T11:22:30.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.249+0000 7f3f51000640 1 -- 172.21.15.7:0/1034302371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4c159c90 con 0x7f3f4c162270 2024-04-25T11:22:30.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.250+0000 7f3f51000640 1 -- 172.21.15.7:0/1034302371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4c163130 con 0x7f3f4c162270 2024-04-25T11:22:30.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.251+0000 7f3f51000640 1 -- 172.21.15.7:0/1034302371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f4c06a100 con 0x7f3f4c162270 2024-04-25T11:22:30.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.252+0000 7f3f42000640 1 -- 172.21.15.7:0/1034302371 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3f3401a070 con 0x7f3f4c162270 2024-04-25T11:22:30.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.253+0000 7f3f42000640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3f280744b0 0x7f3f280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:30.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.253+0000 7f3f42000640 1 -- 172.21.15.7:0/1034302371 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5249+0+0 (secure 0 0 0) 0x7f3f340944e0 con 0x7f3f4c162270 2024-04-25T11:22:30.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.253+0000 7f3f49e00640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3f280744b0 0x7f3f280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:30.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.254+0000 7f3f49e00640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3f280744b0 0x7f3f280769c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c033000 tx=0x7f3f3c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:30.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.258+0000 7f3f42000640 1 -- 172.21.15.7:0/1034302371 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f3f4c15a1f0 con 0x7f3f4c162270 2024-04-25T11:22:30.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.452+0000 7f3f51000640 1 -- 172.21.15.7:0/1034302371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3f4c162be0 con 0x7f3f4c162270 2024-04-25T11:22:30.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.732+0000 7f3f42000640 1 -- 172.21.15.7:0/1034302371 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v153) ==== 138+0+0 (secure 0 0 0) 0x7f3f3405ee20 con 0x7f3f4c162270 2024-04-25T11:22:30.727 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-04-25T11:22:30.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.736+0000 7f3f51000640 1 -- 172.21.15.7:0/1034302371 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3f280744b0 msgr2=0x7f3f280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:30.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.736+0000 7f3f51000640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3f280744b0 0x7f3f280769c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c033000 tx=0x7f3f3c009030 comp rx=0 tx=0).stop 2024-04-25T11:22:30.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.736+0000 7f3f51000640 1 -- 172.21.15.7:0/1034302371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c162270 msgr2=0x7f3f4c0b7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:30.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.736+0000 7f3f51000640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c162270 0x7f3f4c0b7420 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c430 tx=0x7f3f3400c8f0 comp rx=0 tx=0).stop 2024-04-25T11:22:30.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.737+0000 7f3f51000640 1 -- 172.21.15.7:0/1034302371 shutdown_connections 2024-04-25T11:22:30.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.737+0000 7f3f51000640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f3f280744b0 0x7f3f280769c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:30.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.737+0000 7f3f51000640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f4c162680 0x7f3f4c0b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:30.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.737+0000 7f3f51000640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c162270 0x7f3f4c0b7420 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:30.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.737+0000 7f3f51000640 1 --2- 172.21.15.7:0/1034302371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f30003a00 0x7f3f4c159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:30.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.737+0000 7f3f51000640 1 -- 172.21.15.7:0/1034302371 >> 172.21.15.7:0/1034302371 conn(0x7f3f4c0b5470 msgr2=0x7f3f4c072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:30.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.737+0000 7f3f51000640 1 -- 172.21.15.7:0/1034302371 shutdown_connections 2024-04-25T11:22:30.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:30.737+0000 7f3f51000640 1 -- 172.21.15.7:0/1034302371 wait complete. 2024-04-25T11:22:30.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T11:22:31.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.045+0000 7f31a5c00640 1 Processor -- start 2024-04-25T11:22:31.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.045+0000 7f31a5c00640 1 -- start start 2024-04-25T11:22:31.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.048+0000 7f31a5c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31a00d5140 0x7f31a00b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.048+0000 7f31a5c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f31a01647c0 0x7f31a00d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.048+0000 7f31a5c00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a00d1100 con 0x7f31a00b7580 2024-04-25T11:22:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.048+0000 7f31a5c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f31a0164330 con 0x7f31a01647c0 2024-04-25T11:22:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.048+0000 7f31a5c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f31a00d13e0 con 0x7f31a00d5140 2024-04-25T11:22:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.048+0000 7f3196000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f31a01647c0 0x7f31a00d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.048+0000 7f319ea00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31a00d5140 0x7f31a00b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.048+0000 7f3196000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f31a01647c0 0x7f31a00d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:41958/0 (socket says 172.21.15.7:41958) 2024-04-25T11:22:31.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.048+0000 7f3196000640 1 -- 172.21.15.7:0/1923023188 learned_addr learned my addr 172.21.15.7:0/1923023188 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.049+0000 7f3196000640 1 -- 172.21.15.7:0/1923023188 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31a00d5140 msgr2=0x7f31a00b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.049+0000 7f3196000640 1 --2- 172.21.15.7:0/1923023188 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31a00d5140 0x7f31a00b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.049+0000 7f3196000640 1 -- 172.21.15.7:0/1923023188 >> v1:172.21.15.87:6789/0 conn(0x7f31a00b7580 legacy=0x7f31a00b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:31.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.049+0000 7f3196000640 1 -- 172.21.15.7:0/1923023188 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a0163fc0 con 0x7f31a01647c0 2024-04-25T11:22:31.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.051+0000 7f3196000640 1 --2- 172.21.15.7:0/1923023188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31a01647c0 0x7f31a00d08b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f31880098a0 tx=0x7f318802a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=628cb1307b58b6da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.051+0000 7f319e000640 1 -- 172.21.15.7:0/1923023188 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188037070 con 0x7f31a01647c0 2024-04-25T11:22:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.051+0000 7f319e000640 1 -- 172.21.15.7:0/1923023188 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3188032440 con 0x7f31a01647c0 2024-04-25T11:22:31.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.051+0000 7f319e000640 1 -- 172.21.15.7:0/1923023188 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188032810 con 0x7f31a01647c0 2024-04-25T11:22:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.052+0000 7f31a5c00640 1 -- 172.21.15.7:0/1923023188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31a01647c0 msgr2=0x7f31a00d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.052+0000 7f31a5c00640 1 --2- 172.21.15.7:0/1923023188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31a01647c0 0x7f31a00d08b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f31880098a0 tx=0x7f318802a1f0 comp rx=0 tx=0).stop 2024-04-25T11:22:31.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.053+0000 7f31a5c00640 1 -- 172.21.15.7:0/1923023188 shutdown_connections 2024-04-25T11:22:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.053+0000 7f31a5c00640 1 --2- 172.21.15.7:0/1923023188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31a01647c0 0x7f31a00d08b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.053+0000 7f31a5c00640 1 --2- 172.21.15.7:0/1923023188 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31a00d5140 0x7f31a00b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.053+0000 7f31a5c00640 1 -- 172.21.15.7:0/1923023188 >> 172.21.15.7:0/1923023188 conn(0x7f31a00d2760 msgr2=0x7f31a00d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.053+0000 7f31a5c00640 1 -- 172.21.15.7:0/1923023188 shutdown_connections 2024-04-25T11:22:31.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.053+0000 7f31a5c00640 1 -- 172.21.15.7:0/1923023188 wait complete. 2024-04-25T11:22:31.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.056+0000 7f31a5c00640 1 Processor -- start 2024-04-25T11:22:31.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.056+0000 7f31a5c00640 1 -- start start 2024-04-25T11:22:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.058+0000 7f31a5c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31a00b7580 0x7f31a01590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.058+0000 7f31a5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a00d5140 0x7f31a0148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:31.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.058+0000 7f31a5c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31a01647c0 0x7f31a01f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:31.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.058+0000 7f31a5c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31a00b8fd0 con 0x7f31a00d5140 2024-04-25T11:22:31.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.058+0000 7f31a5c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f31a0164bd0 con 0x7f31a00b7580 2024-04-25T11:22:31.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.058+0000 7f31a5c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f31a00b9150 con 0x7f31a01647c0 2024-04-25T11:22:31.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.059+0000 7f3196000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a00d5140 0x7f31a0148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:31.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.059+0000 7f3196000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a00d5140 0x7f31a0148320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34412/0 (socket says 172.21.15.7:34412) 2024-04-25T11:22:31.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.059+0000 7f3196000640 1 -- 172.21.15.7:0/623723728 learned_addr learned my addr 172.21.15.7:0/623723728 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:31.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.059+0000 7f319ea00640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31a00b7580 0x7f31a01590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:31.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.059+0000 7f319f400640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31a01647c0 0x7f31a01f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:31.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.059+0000 7f3196000640 1 -- 172.21.15.7:0/623723728 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31a01647c0 msgr2=0x7f31a01f9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.059+0000 7f3196000640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31a01647c0 0x7f31a01f9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.059+0000 7f3196000640 1 -- 172.21.15.7:0/623723728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31a00b7580 msgr2=0x7f31a01590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.059+0000 7f3196000640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31a00b7580 0x7f31a01590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.059+0000 7f3196000640 1 -- 172.21.15.7:0/623723728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a0158910 con 0x7f31a00d5140 2024-04-25T11:22:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.060+0000 7f3196000640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a00d5140 0x7f31a0148320 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f318802a7e0 tx=0x7f31880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.060+0000 7f3197400640 1 -- 172.21.15.7:0/623723728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188043070 con 0x7f31a00d5140 2024-04-25T11:22:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.060+0000 7f3197400640 1 -- 172.21.15.7:0/623723728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3188038040 con 0x7f31a00d5140 2024-04-25T11:22:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.060+0000 7f3197400640 1 -- 172.21.15.7:0/623723728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3188037050 con 0x7f31a00d5140 2024-04-25T11:22:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.060+0000 7f31a5c00640 1 -- 172.21.15.7:0/623723728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a0157870 con 0x7f31a00d5140 2024-04-25T11:22:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.060+0000 7f31a5c00640 1 -- 172.21.15.7:0/623723728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a0157e30 con 0x7f31a00d5140 2024-04-25T11:22:31.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.062+0000 7f318fe00640 1 -- 172.21.15.7:0/623723728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a0050740 con 0x7f31a00d5140 2024-04-25T11:22:31.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.065+0000 7f3197400640 1 -- 172.21.15.7:0/623723728 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f3188046070 con 0x7f31a00d5140 2024-04-25T11:22:31.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.066+0000 7f3197400640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f317c0744e0 0x7f317c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:31.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.066+0000 7f3197400640 1 -- 172.21.15.7:0/623723728 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5619+0+0 (secure 0 0 0) 0x7f31880b58b0 con 0x7f31a00d5140 2024-04-25T11:22:31.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.066+0000 7f319ea00640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f317c0744e0 0x7f317c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:31.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.067+0000 7f319ea00640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f317c0744e0 0x7f317c0769f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f31a00b5530 tx=0x7f3190009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:31.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.070+0000 7f3197400640 1 -- 172.21.15.7:0/623723728 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f318807fbb0 con 0x7f31a00d5140 2024-04-25T11:22:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.276+0000 7f318fe00640 1 -- 172.21.15.7:0/623723728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f31a0163750 con 0x7f31a00d5140 2024-04-25T11:22:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.738+0000 7f3197400640 1 -- 172.21.15.7:0/623723728 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v154) ==== 170+0+0 (secure 0 0 0) 0x7f3188083eb0 con 0x7f31a00d5140 2024-04-25T11:22:31.734 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-04-25T11:22:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.742+0000 7f318fe00640 1 -- 172.21.15.7:0/623723728 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f317c0744e0 msgr2=0x7f317c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.742+0000 7f318fe00640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f317c0744e0 0x7f317c0769f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f31a00b5530 tx=0x7f3190009190 comp rx=0 tx=0).stop 2024-04-25T11:22:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.742+0000 7f318fe00640 1 -- 172.21.15.7:0/623723728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a00d5140 msgr2=0x7f31a0148320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.742+0000 7f318fe00640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a00d5140 0x7f31a0148320 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f318802a7e0 tx=0x7f31880046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.742+0000 7f318fe00640 1 -- 172.21.15.7:0/623723728 shutdown_connections 2024-04-25T11:22:31.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.742+0000 7f318fe00640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f317c0744e0 0x7f317c0769f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:31.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.742+0000 7f318fe00640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31a01647c0 0x7f31a01f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:31.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.742+0000 7f318fe00640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31a00d5140 0x7f31a0148320 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:31.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.742+0000 7f318fe00640 1 --2- 172.21.15.7:0/623723728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31a00b7580 0x7f31a01590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:31.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.742+0000 7f318fe00640 1 -- 172.21.15.7:0/623723728 >> 172.21.15.7:0/623723728 conn(0x7f31a00d2760 msgr2=0x7f31a00b9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:31.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.743+0000 7f318fe00640 1 -- 172.21.15.7:0/623723728 shutdown_connections 2024-04-25T11:22:31.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:31.743+0000 7f318fe00640 1 -- 172.21.15.7:0/623723728 wait complete. 2024-04-25T11:22:31.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T11:22:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.045+0000 7f63d2000640 1 Processor -- start 2024-04-25T11:22:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.045+0000 7f63d2000640 1 -- start start 2024-04-25T11:22:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.048+0000 7f63d2000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63cc1583c0 0x7f63cc0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.048+0000 7f63d2000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63cc0cfb50 0x7f63cc0d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.048+0000 7f63d2000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63cc0d09c0 0x7f63cc06eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.048+0000 7f63d2000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f63cc15f190 con 0x7f63cc0d09c0 2024-04-25T11:22:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.048+0000 7f63d2000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f63cc0ce570 con 0x7f63cc0cfb50 2024-04-25T11:22:32.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.048+0000 7f63d2000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63cc0ce850 con 0x7f63cc1583c0 2024-04-25T11:22:32.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.048+0000 7f63d1600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63cc0d09c0 0x7f63cc06eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:32.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.048+0000 7f63d1600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63cc0d09c0 0x7f63cc06eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34416/0 (socket says 172.21.15.7:34416) 2024-04-25T11:22:32.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.048+0000 7f63d1600640 1 -- 172.21.15.7:0/3805862672 learned_addr learned my addr 172.21.15.7:0/3805862672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.049+0000 7f63cbe00640 1 --2- 172.21.15.7:0/3805862672 >> v2:172.21.15.87:3300/0 conn(0x7f63cc0cfb50 0x7f63cc0d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.049+0000 7f63d0c00640 1 --2- 172.21.15.7:0/3805862672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63cc1583c0 0x7f63cc0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.049+0000 7f63d1600640 1 -- 172.21.15.7:0/3805862672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63cc1583c0 msgr2=0x7f63cc0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.049+0000 7f63d1600640 1 --2- 172.21.15.7:0/3805862672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63cc1583c0 0x7f63cc0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.049+0000 7f63d1600640 1 -- 172.21.15.7:0/3805862672 >> v2:172.21.15.87:3300/0 conn(0x7f63cc0cfb50 msgr2=0x7f63cc0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.049+0000 7f63d1600640 1 --2- 172.21.15.7:0/3805862672 >> v2:172.21.15.87:3300/0 conn(0x7f63cc0cfb50 0x7f63cc0d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.049+0000 7f63d1600640 1 -- 172.21.15.7:0/3805862672 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63cc0ceb30 con 0x7f63cc0d09c0 2024-04-25T11:22:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.049+0000 7f63cbe00640 1 --2- 172.21.15.7:0/3805862672 >> v2:172.21.15.87:3300/0 conn(0x7f63cc0cfb50 0x7f63cc0d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:22:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.050+0000 7f63d1600640 1 --2- 172.21.15.7:0/3805862672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63cc0d09c0 0x7f63cc06eea0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f63cc06e0f0 tx=0x7f63c402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657c44a26d0caed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:32.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.051+0000 7f63cb400640 1 -- 172.21.15.7:0/3805862672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4036070 con 0x7f63cc0d09c0 2024-04-25T11:22:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.051+0000 7f63cb400640 1 -- 172.21.15.7:0/3805862672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63c4031600 con 0x7f63cc0d09c0 2024-04-25T11:22:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.051+0000 7f63cb400640 1 -- 172.21.15.7:0/3805862672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c40319d0 con 0x7f63cc0d09c0 2024-04-25T11:22:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.052+0000 7f63d2000640 1 -- 172.21.15.7:0/3805862672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63cc0d09c0 msgr2=0x7f63cc06eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.052+0000 7f63d2000640 1 --2- 172.21.15.7:0/3805862672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63cc0d09c0 0x7f63cc06eea0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f63cc06e0f0 tx=0x7f63c402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.052+0000 7f63d2000640 1 -- 172.21.15.7:0/3805862672 shutdown_connections 2024-04-25T11:22:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.052+0000 7f63d2000640 1 --2- 172.21.15.7:0/3805862672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63cc0d09c0 0x7f63cc06eea0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.052+0000 7f63d2000640 1 --2- 172.21.15.7:0/3805862672 >> v2:172.21.15.87:3300/0 conn(0x7f63cc0cfb50 0x7f63cc0d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.052+0000 7f63d2000640 1 --2- 172.21.15.7:0/3805862672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63cc1583c0 0x7f63cc0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.052+0000 7f63d2000640 1 -- 172.21.15.7:0/3805862672 >> 172.21.15.7:0/3805862672 conn(0x7f63cc0740f0 msgr2=0x7f63cc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.052+0000 7f63d2000640 1 -- 172.21.15.7:0/3805862672 shutdown_connections 2024-04-25T11:22:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.052+0000 7f63d2000640 1 -- 172.21.15.7:0/3805862672 wait complete. 2024-04-25T11:22:32.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.054+0000 7f63d2000640 1 Processor -- start 2024-04-25T11:22:32.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.054+0000 7f63d2000640 1 -- start start 2024-04-25T11:22:32.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.056+0000 7f63d2000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63cc0cfb50 0x7f63cc149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:32.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.056+0000 7f63d2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63cc0d09c0 0x7f63cc141c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.056+0000 7f63d2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63cc1583c0 0x7f63cc15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.056+0000 7f63d2000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc06f3e0 con 0x7f63cc0d09c0 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.056+0000 7f63d2000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc0cf0f0 con 0x7f63cc1583c0 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.056+0000 7f63d2000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63cc06f560 con 0x7f63cc0cfb50 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.056+0000 7f63cbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63cc0d09c0 0x7f63cc141c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.056+0000 7f63cbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63cc0d09c0 0x7f63cc141c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34432/0 (socket says 172.21.15.7:34432) 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.056+0000 7f63cbe00640 1 -- 172.21.15.7:0/3139121778 learned_addr learned my addr 172.21.15.7:0/3139121778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63d1600640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63cc1583c0 0x7f63cc15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63d0c00640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63cc0cfb50 0x7f63cc149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63cbe00640 1 -- 172.21.15.7:0/3139121778 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63cc0cfb50 msgr2=0x7f63cc149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63cbe00640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63cc0cfb50 0x7f63cc149430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63cbe00640 1 -- 172.21.15.7:0/3139121778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63cc1583c0 msgr2=0x7f63cc15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63cbe00640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63cc1583c0 0x7f63cc15cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63cbe00640 1 -- 172.21.15.7:0/3139121778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63cc0b4df0 con 0x7f63cc0d09c0 2024-04-25T11:22:32.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63d1600640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63cc1583c0 0x7f63cc15cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:32.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63cbe00640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63cc0d09c0 0x7f63cc141c60 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f63b800d800 tx=0x7f63b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:32.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63c8c00640 1 -- 172.21.15.7:0/3139121778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8018070 con 0x7f63cc0d09c0 2024-04-25T11:22:32.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63c8c00640 1 -- 172.21.15.7:0/3139121778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63b8004030 con 0x7f63cc0d09c0 2024-04-25T11:22:32.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63c8c00640 1 -- 172.21.15.7:0/3139121778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8004400 con 0x7f63cc0d09c0 2024-04-25T11:22:32.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.057+0000 7f63d2000640 1 -- 172.21.15.7:0/3139121778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63cc1571d0 con 0x7f63cc0d09c0 2024-04-25T11:22:32.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.058+0000 7f63d2000640 1 -- 172.21.15.7:0/3139121778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63cc0b5090 con 0x7f63cc0d09c0 2024-04-25T11:22:32.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.059+0000 7f63d2000640 1 -- 172.21.15.7:0/3139121778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63cc050740 con 0x7f63cc0d09c0 2024-04-25T11:22:32.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.061+0000 7f63c8c00640 1 -- 172.21.15.7:0/3139121778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f63b80096a0 con 0x7f63cc0d09c0 2024-04-25T11:22:32.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.062+0000 7f63c8c00640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63ac0744e0 0x7f63ac0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:32.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.062+0000 7f63c8c00640 1 -- 172.21.15.7:0/3139121778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 6001+0+0 (secure 0 0 0) 0x7f63b8096140 con 0x7f63cc0d09c0 2024-04-25T11:22:32.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.062+0000 7f63d0c00640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63ac0744e0 0x7f63ac0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:32.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.063+0000 7f63d0c00640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63ac0744e0 0x7f63ac0769f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f63c0006fd0 tx=0x7f63c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:32.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.067+0000 7f63c8c00640 1 -- 172.21.15.7:0/3139121778 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f63b8060810 con 0x7f63cc0d09c0 2024-04-25T11:22:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.313+0000 7f63d2000640 1 -- 172.21.15.7:0/3139121778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f63cc070dd0 con 0x7f63cc0d09c0 2024-04-25T11:22:32.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.769+0000 7f63c8c00640 1 -- 172.21.15.7:0/3139121778 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v112) ==== 174+0+0 (secure 0 0 0) 0x7f63b8060110 con 0x7f63cc0d09c0 2024-04-25T11:22:32.766 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 23 and data pool 24 2024-04-25T11:22:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.775+0000 7f63d2000640 1 -- 172.21.15.7:0/3139121778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63ac0744e0 msgr2=0x7f63ac0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.775+0000 7f63d2000640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63ac0744e0 0x7f63ac0769f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f63c0006fd0 tx=0x7f63c0008040 comp rx=0 tx=0).stop 2024-04-25T11:22:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.775+0000 7f63d2000640 1 -- 172.21.15.7:0/3139121778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63cc0d09c0 msgr2=0x7f63cc141c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.775+0000 7f63d2000640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63cc0d09c0 0x7f63cc141c60 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f63b800d800 tx=0x7f63b800dcc0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.775+0000 7f63d2000640 1 -- 172.21.15.7:0/3139121778 shutdown_connections 2024-04-25T11:22:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.775+0000 7f63d2000640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f63ac0744e0 0x7f63ac0769f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.775+0000 7f63d2000640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63cc1583c0 0x7f63cc15cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.775+0000 7f63d2000640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63cc0d09c0 0x7f63cc141c60 secure :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f63b800d800 tx=0x7f63b800dcc0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.775+0000 7f63d2000640 1 --2- 172.21.15.7:0/3139121778 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63cc0cfb50 0x7f63cc149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.775+0000 7f63d2000640 1 -- 172.21.15.7:0/3139121778 >> 172.21.15.7:0/3139121778 conn(0x7f63cc0740f0 msgr2=0x7f63cc1614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:32.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.776+0000 7f63d2000640 1 -- 172.21.15.7:0/3139121778 shutdown_connections 2024-04-25T11:22:32.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:32.776+0000 7f63d2000640 1 -- 172.21.15.7:0/3139121778 wait complete. 2024-04-25T11:22:32.827 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:33.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.082+0000 7fd1d4a00640 1 Processor -- start 2024-04-25T11:22:33.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.082+0000 7fd1d4a00640 1 -- start start 2024-04-25T11:22:33.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.084+0000 7fd1d4a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd1d009c000 0x7fd1d0161340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.084+0000 7fd1d4a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d0160740 con 0x7fd1d00b9010 2024-04-25T11:22:33.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.084+0000 7fd1d4a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d006dc20 con 0x7fd1d00b2ae0 2024-04-25T11:22:33.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.084+0000 7fd1d4a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d0161880 con 0x7fd1d009c000 2024-04-25T11:22:33.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.085+0000 7fd1ce800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd1d00b9010 0x7fd1d009b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39928/0 (socket says 172.21.15.7:39928) 2024-04-25T11:22:33.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.085+0000 7fd1ce800640 1 -- 172.21.15.7:0/344754184 learned_addr learned my addr 172.21.15.7:0/344754184 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:33.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.085+0000 7fd1cde00640 1 --2- 172.21.15.7:0/344754184 >> v2:172.21.15.87:3300/0 conn(0x7fd1d009c000 0x7fd1d0161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.085+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1664910642 0 0) 0x7fd1d0160740 con 0x7fd1d00b9010 2024-04-25T11:22:33.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.085+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1b4003520 con 0x7fd1d00b9010 2024-04-25T11:22:33.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 515777077 0 0) 0x7fd1b4003520 con 0x7fd1d00b9010 2024-04-25T11:22:33.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 >> v2:172.21.15.87:3300/0 conn(0x7fd1d009c000 msgr2=0x7fd1d0161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:33.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 --2- 172.21.15.7:0/344754184 >> v2:172.21.15.87:3300/0 conn(0x7fd1d009c000 0x7fd1d0161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 >> v1:172.21.15.87:6789/0 conn(0x7fd1d00b2ae0 legacy=0x7fd1d00d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d006df00 con 0x7fd1d00b9010 2024-04-25T11:22:33.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fd1c0001670 con 0x7fd1d00b9010 2024-04-25T11:22:33.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 >> v1:172.21.15.7:6789/0 conn(0x7fd1d00b9010 legacy=0x7fd1d009b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 --2- 172.21.15.7:0/344754184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b40039b0 0x7fd1b4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 --2- 172.21.15.7:0/344754184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4004800 0x7fd1b4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 --2- 172.21.15.7:0/344754184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1b40051c0 0x7fd1b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d0161880 con 0x7fd1b40039b0 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b400c160 con 0x7fd1b4004800 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b400c360 con 0x7fd1b40051c0 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.086+0000 7fd1ce800640 1 --2- 172.21.15.7:0/344754184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b40039b0 0x7fd1b4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.087+0000 7fd1cde00640 1 --2- 172.21.15.7:0/344754184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4004800 0x7fd1b4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.087+0000 7fd1ce800640 1 -- 172.21.15.7:0/344754184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1b40051c0 msgr2=0x7fd1b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.087+0000 7fd1ce800640 1 --2- 172.21.15.7:0/344754184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1b40051c0 0x7fd1b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.087+0000 7fd1ce800640 1 -- 172.21.15.7:0/344754184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4004800 msgr2=0x7fd1b4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.087+0000 7fd1ce800640 1 --2- 172.21.15.7:0/344754184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4004800 0x7fd1b4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.087+0000 7fd1ce800640 1 -- 172.21.15.7:0/344754184 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b4003520 con 0x7fd1b40039b0 2024-04-25T11:22:33.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.087+0000 7fd1cf200640 1 -- 172.21.15.7:0/344754184 reap_dead start 2024-04-25T11:22:33.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.088+0000 7fd1ce800640 1 --2- 172.21.15.7:0/344754184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b40039b0 0x7fd1b4003e20 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00b6310 tx=0x7fd1b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feefafabd831a35b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.088+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0007d30 con 0x7fd1b40039b0 2024-04-25T11:22:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.088+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c0005af0 con 0x7fd1b40039b0 2024-04-25T11:22:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.088+0000 7fd1cd400640 1 -- 172.21.15.7:0/344754184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0007070 con 0x7fd1b40039b0 2024-04-25T11:22:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.089+0000 7fd1d4a00640 1 -- 172.21.15.7:0/344754184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b40039b0 msgr2=0x7fd1b4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.089+0000 7fd1d4a00640 1 --2- 172.21.15.7:0/344754184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b40039b0 0x7fd1b4003e20 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00b6310 tx=0x7fd1b4005830 comp rx=0 tx=0).stop 2024-04-25T11:22:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.089+0000 7fd1d4a00640 1 -- 172.21.15.7:0/344754184 shutdown_connections 2024-04-25T11:22:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.089+0000 7fd1d4a00640 1 --2- 172.21.15.7:0/344754184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1b40039b0 0x7fd1b4003e20 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.089+0000 7fd1d4a00640 1 -- 172.21.15.7:0/344754184 >> 172.21.15.7:0/344754184 conn(0x7fd1d00b5210 msgr2=0x7fd1d00b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.089+0000 7fd1d4a00640 1 -- 172.21.15.7:0/344754184 shutdown_connections 2024-04-25T11:22:33.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.089+0000 7fd1d4a00640 1 -- 172.21.15.7:0/344754184 wait complete. 2024-04-25T11:22:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.092+0000 7fd1d4a00640 1 Processor -- start 2024-04-25T11:22:33.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.092+0000 7fd1d4a00640 1 -- start start 2024-04-25T11:22:33.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.093+0000 7fd1d4a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1b40039b0 0x7fd1d00cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.093+0000 7fd1d4a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1d00b1980 0x7fd1d01f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.093+0000 7fd1d4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d01f73b0 0x7fd1d00cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.093+0000 7fd1d4a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d006e7a0 con 0x7fd1d01f73b0 2024-04-25T11:22:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.093+0000 7fd1d4a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d006e4c0 con 0x7fd1d00b1980 2024-04-25T11:22:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.093+0000 7fd1d4a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d006ea80 con 0x7fd1b40039b0 2024-04-25T11:22:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.094+0000 7fd1cf200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d01f73b0 0x7fd1d00cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.094+0000 7fd1cf200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d01f73b0 0x7fd1d00cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34442/0 (socket says 172.21.15.7:34442) 2024-04-25T11:22:33.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.094+0000 7fd1cf200640 1 -- 172.21.15.7:0/1424209949 learned_addr learned my addr 172.21.15.7:0/1424209949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:33.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.094+0000 7fd1cde00640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1d00b1980 0x7fd1d01f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.094+0000 7fd1cf200640 1 -- 172.21.15.7:0/1424209949 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1b40039b0 msgr2=0x7fd1d00cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.094+0000 7fd1ce800640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1b40039b0 0x7fd1d00cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.094+0000 7fd1cf200640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1b40039b0 0x7fd1d00cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.094+0000 7fd1cf200640 1 -- 172.21.15.7:0/1424209949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1d00b1980 msgr2=0x7fd1d01f6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.094+0000 7fd1cf200640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1d00b1980 0x7fd1d01f6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.094+0000 7fd1cf200640 1 -- 172.21.15.7:0/1424209949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d01f77c0 con 0x7fd1d01f73b0 2024-04-25T11:22:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.094+0000 7fd1cde00640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1d00b1980 0x7fd1d01f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.095+0000 7fd1cf200640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d01f73b0 0x7fd1d00cdd50 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fd1d006f030 tx=0x7fd1c800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.095+0000 7fd1c6a00640 1 -- 172.21.15.7:0/1424209949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8017070 con 0x7fd1d01f73b0 2024-04-25T11:22:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.095+0000 7fd1c6a00640 1 -- 172.21.15.7:0/1424209949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c8005d40 con 0x7fd1d01f73b0 2024-04-25T11:22:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.095+0000 7fd1c6a00640 1 -- 172.21.15.7:0/1424209949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c800a850 con 0x7fd1d01f73b0 2024-04-25T11:22:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.095+0000 7fd1d4a00640 1 -- 172.21.15.7:0/1424209949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d00b08e0 con 0x7fd1d01f73b0 2024-04-25T11:22:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.095+0000 7fd1d4a00640 1 -- 172.21.15.7:0/1424209949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d00b0ea0 con 0x7fd1d01f73b0 2024-04-25T11:22:33.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.096+0000 7fd1d4a00640 1 -- 172.21.15.7:0/1424209949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d00b0650 con 0x7fd1d01f73b0 2024-04-25T11:22:33.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.097+0000 7fd1c6a00640 1 -- 172.21.15.7:0/1424209949 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd1c801a070 con 0x7fd1d01f73b0 2024-04-25T11:22:33.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.098+0000 7fd1c6a00640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1b00744e0 0x7fd1b00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.099+0000 7fd1c6a00640 1 -- 172.21.15.7:0/1424209949 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7fd1c80946b0 con 0x7fd1d01f73b0 2024-04-25T11:22:33.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.099+0000 7fd1ce800640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1b00744e0 0x7fd1b00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.100+0000 7fd1ce800640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1b00744e0 0x7fd1b00769f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0032000 tx=0x7fd1c002d000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:33.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.103+0000 7fd1c6a00640 1 -- 172.21.15.7:0/1424209949 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd1c805ecd0 con 0x7fd1d01f73b0 2024-04-25T11:22:33.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.307+0000 7fd1d4a00640 1 -- 172.21.15.7:0/1424209949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1d009c730 con 0x7fd1d01f73b0 2024-04-25T11:22:33.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.310+0000 7fd1c6a00640 1 -- 172.21.15.7:0/1424209949 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16801 (secure 0 0 0) 0x7fd1c805e5d0 con 0x7fd1d01f73b0 2024-04-25T11:22:33.305 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:33.305 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":155,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:32.315134+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:22:30.472409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T11:22:31.299177+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.312+0000 7fd1d4a00640 1 -- 172.21.15.7:0/1424209949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1b00744e0 msgr2=0x7fd1b00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.312+0000 7fd1d4a00640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1b00744e0 0x7fd1b00769f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0032000 tx=0x7fd1c002d000 comp rx=0 tx=0).stop 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.312+0000 7fd1d4a00640 1 -- 172.21.15.7:0/1424209949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d01f73b0 msgr2=0x7fd1d00cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.312+0000 7fd1d4a00640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d01f73b0 0x7fd1d00cdd50 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fd1d006f030 tx=0x7fd1c800cb90 comp rx=0 tx=0).stop 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.313+0000 7fd1d4a00640 1 -- 172.21.15.7:0/1424209949 shutdown_connections 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.313+0000 7fd1d4a00640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd1b00744e0 0x7fd1b00769f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.313+0000 7fd1d4a00640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d01f73b0 0x7fd1d00cdd50 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.313+0000 7fd1d4a00640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1d00b1980 0x7fd1d01f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.313+0000 7fd1d4a00640 1 --2- 172.21.15.7:0/1424209949 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1b40039b0 0x7fd1d00cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.313+0000 7fd1d4a00640 1 -- 172.21.15.7:0/1424209949 >> 172.21.15.7:0/1424209949 conn(0x7fd1d00b5210 msgr2=0x7fd1d00b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.313+0000 7fd1d4a00640 1 -- 172.21.15.7:0/1424209949 shutdown_connections 2024-04-25T11:22:33.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.313+0000 7fd1d4a00640 1 -- 172.21.15.7:0/1424209949 wait complete. 2024-04-25T11:22:33.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:33.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.615+0000 7f7246600640 1 Processor -- start 2024-04-25T11:22:33.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.615+0000 7f7246600640 1 -- start start 2024-04-25T11:22:33.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.617+0000 7f7246600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72400b0290 0x7f72400b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.617+0000 7f7246600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f724006f300 0x7f72400b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.617+0000 7f7246600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72400b4980 con 0x7f724006dc90 2024-04-25T11:22:33.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.617+0000 7f7246600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f72400b4c60 con 0x7f724006f300 2024-04-25T11:22:33.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.617+0000 7f7246600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f72400b4f40 con 0x7f72400b0290 2024-04-25T11:22:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.618+0000 7f723fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f724006f300 0x7f72400b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.618+0000 7f723f400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f724006dc90 0x7f72400afb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39930/0 (socket says 172.21.15.7:39930) 2024-04-25T11:22:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.618+0000 7f723f400640 1 -- 172.21.15.7:0/2968354146 learned_addr learned my addr 172.21.15.7:0/2968354146 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.618+0000 7f723ea00640 1 --2- 172.21.15.7:0/2968354146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72400b0290 0x7f72400b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.618+0000 7f723fe00640 1 -- 172.21.15.7:0/2968354146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72400b0290 msgr2=0x7f72400b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.618+0000 7f723fe00640 1 --2- 172.21.15.7:0/2968354146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72400b0290 0x7f72400b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.618+0000 7f723fe00640 1 -- 172.21.15.7:0/2968354146 >> v1:172.21.15.7:6789/0 conn(0x7f724006dc90 legacy=0x7f72400afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.618+0000 7f723fe00640 1 -- 172.21.15.7:0/2968354146 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72400b5220 con 0x7f724006f300 2024-04-25T11:22:33.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.618+0000 7f723e000640 1 -- 172.21.15.7:0/2968354146 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1416291701 0 0) 0x7f72400b4980 con 0x7f724006dc90 2024-04-25T11:22:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.620+0000 7f723fe00640 1 --2- 172.21.15.7:0/2968354146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724006f300 0x7f72400b5f60 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f72380097a0 tx=0x7f723802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141fd8969d09f54c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.620+0000 7f723e000640 1 -- 172.21.15.7:0/2968354146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238037070 con 0x7f724006f300 2024-04-25T11:22:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.620+0000 7f723e000640 1 -- 172.21.15.7:0/2968354146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7238032440 con 0x7f724006f300 2024-04-25T11:22:33.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.620+0000 7f723e000640 1 -- 172.21.15.7:0/2968354146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238032810 con 0x7f724006f300 2024-04-25T11:22:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.621+0000 7f7246600640 1 -- 172.21.15.7:0/2968354146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724006f300 msgr2=0x7f72400b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.621+0000 7f7246600640 1 --2- 172.21.15.7:0/2968354146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724006f300 0x7f72400b5f60 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f72380097a0 tx=0x7f723802a180 comp rx=0 tx=0).stop 2024-04-25T11:22:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.621+0000 7f7246600640 1 -- 172.21.15.7:0/2968354146 shutdown_connections 2024-04-25T11:22:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.621+0000 7f7246600640 1 --2- 172.21.15.7:0/2968354146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724006f300 0x7f72400b5f60 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.621+0000 7f7246600640 1 --2- 172.21.15.7:0/2968354146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72400b0290 0x7f72400b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.621+0000 7f7246600640 1 -- 172.21.15.7:0/2968354146 >> 172.21.15.7:0/2968354146 conn(0x7f7240074200 msgr2=0x7f724006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.621+0000 7f7246600640 1 -- 172.21.15.7:0/2968354146 shutdown_connections 2024-04-25T11:22:33.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.621+0000 7f7246600640 1 -- 172.21.15.7:0/2968354146 wait complete. 2024-04-25T11:22:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.623+0000 7f7246600640 1 Processor -- start 2024-04-25T11:22:33.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.623+0000 7f7246600640 1 -- start start 2024-04-25T11:22:33.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.625+0000 7f7246600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724006f300 0x7f724014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.625+0000 7f7246600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f724015a8a0 0x7f7240153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.625+0000 7f7246600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72401518c0 0x7f7240151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.625+0000 7f7246600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72400ce920 con 0x7f724006f300 2024-04-25T11:22:33.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.625+0000 7f7246600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f724015d220 con 0x7f724015a8a0 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.625+0000 7f7246600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f72400ceaa0 con 0x7f72401518c0 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.625+0000 7f723f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724006f300 0x7f724014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.625+0000 7f723f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724006f300 0x7f724014f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34462/0 (socket says 172.21.15.7:34462) 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.625+0000 7f723f400640 1 -- 172.21.15.7:0/738005884 learned_addr learned my addr 172.21.15.7:0/738005884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.626+0000 7f723fe00640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72401518c0 0x7f7240151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.626+0000 7f723ea00640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f724015a8a0 0x7f7240153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.626+0000 7f723f400640 1 -- 172.21.15.7:0/738005884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72401518c0 msgr2=0x7f7240151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.626+0000 7f723f400640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72401518c0 0x7f7240151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.626+0000 7f723f400640 1 -- 172.21.15.7:0/738005884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f724015a8a0 msgr2=0x7f7240153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.626+0000 7f723f400640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f724015a8a0 0x7f7240153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.626+0000 7f723f400640 1 -- 172.21.15.7:0/738005884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724014fc10 con 0x7f724006f300 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.626+0000 7f723fe00640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72401518c0 0x7f7240151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.626+0000 7f723f400640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724006f300 0x7f724014f6d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f723000c440 tx=0x7f723000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:33.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.626+0000 7f7237400640 1 -- 172.21.15.7:0/738005884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7230018070 con 0x7f724006f300 2024-04-25T11:22:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.626+0000 7f7237400640 1 -- 172.21.15.7:0/738005884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72300074e0 con 0x7f724006f300 2024-04-25T11:22:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.627+0000 7f7237400640 1 -- 172.21.15.7:0/738005884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7230004a20 con 0x7f724006f300 2024-04-25T11:22:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.627+0000 7f7246600640 1 -- 172.21.15.7:0/738005884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7240150820 con 0x7f724006f300 2024-04-25T11:22:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.627+0000 7f7246600640 1 -- 172.21.15.7:0/738005884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7240150de0 con 0x7f724006f300 2024-04-25T11:22:33.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.628+0000 7f7246600640 1 -- 172.21.15.7:0/738005884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72400b1010 con 0x7f724006f300 2024-04-25T11:22:33.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.629+0000 7f7237400640 1 -- 172.21.15.7:0/738005884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7230004bc0 con 0x7f724006f300 2024-04-25T11:22:33.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.631+0000 7f7237400640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f721c0744e0 0x7f721c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:33.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.631+0000 7f7237400640 1 -- 172.21.15.7:0/738005884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 6101+0+0 (secure 0 0 0) 0x7f7230094680 con 0x7f724006f300 2024-04-25T11:22:33.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.631+0000 7f723ea00640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f721c0744e0 0x7f721c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:33.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.632+0000 7f723ea00640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f721c0744e0 0x7f721c0769f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f724006df60 tx=0x7f722800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:33.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.636+0000 7f7237400640 1 -- 172.21.15.7:0/738005884 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f72400b1010 con 0x7f724006f300 2024-04-25T11:22:33.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.846+0000 7f7246600640 1 -- 172.21.15.7:0/738005884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f724006a100 con 0x7f724006f300 2024-04-25T11:22:33.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.848+0000 7f7237400640 1 -- 172.21.15.7:0/738005884 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16788 (secure 0 0 0) 0x7f724014fc10 con 0x7f724006f300 2024-04-25T11:22:33.843 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:33.843 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:33.767424+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:22:30.472409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T11:22:31.299177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:33.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 -- 172.21.15.7:0/738005884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f721c0744e0 msgr2=0x7f721c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f721c0744e0 0x7f721c0769f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f724006df60 tx=0x7f722800b040 comp rx=0 tx=0).stop 2024-04-25T11:22:33.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 -- 172.21.15.7:0/738005884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724006f300 msgr2=0x7f724014f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:33.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724006f300 0x7f724014f6d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f723000c440 tx=0x7f723000c900 comp rx=0 tx=0).stop 2024-04-25T11:22:33.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 -- 172.21.15.7:0/738005884 shutdown_connections 2024-04-25T11:22:33.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f721c0744e0 0x7f721c0769f0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72401518c0 0x7f7240151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f724015a8a0 0x7f7240153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 --2- 172.21.15.7:0/738005884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f724006f300 0x7f724014f6d0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 -- 172.21.15.7:0/738005884 >> 172.21.15.7:0/738005884 conn(0x7f7240074200 msgr2=0x7f72400b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 -- 172.21.15.7:0/738005884 shutdown_connections 2024-04-25T11:22:33.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:33.851+0000 7f7246600640 1 -- 172.21.15.7:0/738005884 wait complete. 2024-04-25T11:22:33.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T11:22:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.135+0000 7fe923000640 1 Processor -- start 2024-04-25T11:22:34.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.135+0000 7fe923000640 1 -- start start 2024-04-25T11:22:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.137+0000 7fe923000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe91c164990 0x7fe91c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.138+0000 7fe923000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe91c161210 con 0x7fe91c0b7660 2024-04-25T11:22:34.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.138+0000 7fe923000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe91c1614a0 con 0x7fe91c1606f0 2024-04-25T11:22:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.138+0000 7fe923000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe91c163eb0 con 0x7fe91c164990 2024-04-25T11:22:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.138+0000 7fe91b400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe91c0b7660 0x7fe91c0b7a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39940/0 (socket says 172.21.15.7:39940) 2024-04-25T11:22:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.138+0000 7fe91b400640 1 -- 172.21.15.7:0/510417579 learned_addr learned my addr 172.21.15.7:0/510417579 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:34.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.138+0000 7fe91be00640 1 --2- 172.21.15.7:0/510417579 >> v2:172.21.15.87:3300/0 conn(0x7fe91c164990 0x7fe91c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1857899646 0 0) 0x7fe91c161210 con 0x7fe91c0b7660 2024-04-25T11:22:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8fc003520 con 0x7fe91c0b7660 2024-04-25T11:22:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3698843134 0 0) 0x7fe8fc003520 con 0x7fe91c0b7660 2024-04-25T11:22:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 >> v2:172.21.15.87:3300/0 conn(0x7fe91c164990 msgr2=0x7fe91c15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 --2- 172.21.15.7:0/510417579 >> v2:172.21.15.87:3300/0 conn(0x7fe91c164990 0x7fe91c15dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 >> v1:172.21.15.87:6789/0 conn(0x7fe91c1606f0 legacy=0x7fe91c160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c164190 con 0x7fe91c0b7660 2024-04-25T11:22:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fe9040019f0 con 0x7fe91c0b7660 2024-04-25T11:22:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 >> v1:172.21.15.7:6789/0 conn(0x7fe91c0b7660 legacy=0x7fe91c0b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 --2- 172.21.15.7:0/510417579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 0x7fe8fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 --2- 172.21.15.7:0/510417579 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8fc004800 0x7fe8fc004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 --2- 172.21.15.7:0/510417579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc0051c0 0x7fe8fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c163eb0 con 0x7fe8fc0039b0 2024-04-25T11:22:34.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc00c160 con 0x7fe8fc0051c0 2024-04-25T11:22:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.139+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe8fc00c360 con 0x7fe8fc004800 2024-04-25T11:22:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.140+0000 7fe91be00640 1 --2- 172.21.15.7:0/510417579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 0x7fe8fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.140+0000 7fe91be00640 1 -- 172.21.15.7:0/510417579 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8fc004800 msgr2=0x7fe8fc004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.140+0000 7fe91be00640 1 --2- 172.21.15.7:0/510417579 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8fc004800 0x7fe8fc004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.140+0000 7fe91be00640 1 -- 172.21.15.7:0/510417579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc0051c0 msgr2=0x7fe8fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.140+0000 7fe921000640 1 --2- 172.21.15.7:0/510417579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc0051c0 0x7fe8fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.140+0000 7fe91be00640 1 --2- 172.21.15.7:0/510417579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc0051c0 0x7fe8fc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.140+0000 7fe91be00640 1 -- 172.21.15.7:0/510417579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc003520 con 0x7fe8fc0039b0 2024-04-25T11:22:34.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.140+0000 7fe921000640 1 -- 172.21.15.7:0/510417579 reap_dead start 2024-04-25T11:22:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.141+0000 7fe91be00640 1 --2- 172.21.15.7:0/510417579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 0x7fe8fc003e20 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009a90 tx=0x7fe8fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ff41f2088b4ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:34.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.141+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c036070 con 0x7fe8fc0039b0 2024-04-25T11:22:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.141+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90c031400 con 0x7fe8fc0039b0 2024-04-25T11:22:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.141+0000 7fe91aa00640 1 -- 172.21.15.7:0/510417579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c009690 con 0x7fe8fc0039b0 2024-04-25T11:22:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.142+0000 7fe923000640 1 -- 172.21.15.7:0/510417579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 msgr2=0x7fe8fc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.142+0000 7fe923000640 1 --2- 172.21.15.7:0/510417579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 0x7fe8fc003e20 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009a90 tx=0x7fe8fc005830 comp rx=0 tx=0).stop 2024-04-25T11:22:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.142+0000 7fe923000640 1 -- 172.21.15.7:0/510417579 shutdown_connections 2024-04-25T11:22:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.142+0000 7fe923000640 1 --2- 172.21.15.7:0/510417579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 0x7fe8fc003e20 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.142+0000 7fe923000640 1 -- 172.21.15.7:0/510417579 >> 172.21.15.7:0/510417579 conn(0x7fe91c0d2840 msgr2=0x7fe91c0d2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.142+0000 7fe923000640 1 -- 172.21.15.7:0/510417579 shutdown_connections 2024-04-25T11:22:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.142+0000 7fe923000640 1 -- 172.21.15.7:0/510417579 wait complete. 2024-04-25T11:22:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.144+0000 7fe923000640 1 Processor -- start 2024-04-25T11:22:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.145+0000 7fe923000640 1 -- start start 2024-04-25T11:22:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.146+0000 7fe923000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 0x7fe91c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.146+0000 7fe923000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe91c0cf280 0x7fe91c0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.146+0000 7fe923000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe91c0b3db0 0x7fe91c0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.146+0000 7fe923000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c15de30 con 0x7fe8fc0039b0 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.146+0000 7fe923000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe91c15dcb0 con 0x7fe91c0b3db0 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.146+0000 7fe923000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe91c15dfb0 con 0x7fe91c0cf280 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.146+0000 7fe91be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 0x7fe91c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.147+0000 7fe91be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 0x7fe91c15d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34480/0 (socket says 172.21.15.7:34480) 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.147+0000 7fe91be00640 1 -- 172.21.15.7:0/1456532091 learned_addr learned my addr 172.21.15.7:0/1456532091 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.147+0000 7fe91b400640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe91c0cf280 0x7fe91c0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.147+0000 7fe921000640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe91c0b3db0 0x7fe91c0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.147+0000 7fe91be00640 1 -- 172.21.15.7:0/1456532091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe91c0cf280 msgr2=0x7fe91c0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.147+0000 7fe91be00640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe91c0cf280 0x7fe91c0cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.147+0000 7fe91be00640 1 -- 172.21.15.7:0/1456532091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe91c0b3db0 msgr2=0x7fe91c0b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.147+0000 7fe91be00640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe91c0b3db0 0x7fe91c0b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.147+0000 7fe91be00640 1 -- 172.21.15.7:0/1456532091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91c0b4700 con 0x7fe8fc0039b0 2024-04-25T11:22:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.147+0000 7fe921000640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe91c0b3db0 0x7fe91c0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.147+0000 7fe91be00640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 0x7fe91c15d480 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009a90 tx=0x7fe90c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.148+0000 7fe913e00640 1 -- 172.21.15.7:0/1456532091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c040020 con 0x7fe8fc0039b0 2024-04-25T11:22:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.148+0000 7fe913e00640 1 -- 172.21.15.7:0/1456532091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90c03b680 con 0x7fe8fc0039b0 2024-04-25T11:22:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.148+0000 7fe913e00640 1 -- 172.21.15.7:0/1456532091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c036040 con 0x7fe8fc0039b0 2024-04-25T11:22:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.148+0000 7fe923000640 1 -- 172.21.15.7:0/1456532091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91c0ce080 con 0x7fe8fc0039b0 2024-04-25T11:22:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.148+0000 7fe923000640 1 -- 172.21.15.7:0/1456532091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91c0ce640 con 0x7fe8fc0039b0 2024-04-25T11:22:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.149+0000 7fe923000640 1 -- 172.21.15.7:0/1456532091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91c050740 con 0x7fe8fc0039b0 2024-04-25T11:22:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.150+0000 7fe913e00640 1 -- 172.21.15.7:0/1456532091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe90c043020 con 0x7fe8fc0039b0 2024-04-25T11:22:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.151+0000 7fe913e00640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8f80744e0 0x7fe8f80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.152+0000 7fe913e00640 1 -- 172.21.15.7:0/1456532091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7fe90c0b59d0 con 0x7fe8fc0039b0 2024-04-25T11:22:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.152+0000 7fe91b400640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8f80744e0 0x7fe8f80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.152+0000 7fe91b400640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8f80744e0 0x7fe8f80769f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe904005fd0 tx=0x7fe904001ab0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.156+0000 7fe913e00640 1 -- 172.21.15.7:0/1456532091 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe90c04a070 con 0x7fe8fc0039b0 2024-04-25T11:22:34.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.397+0000 7fe923000640 1 -- 172.21.15.7:0/1456532091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe91c0d4000 con 0x7fe8fc0039b0 2024-04-25T11:22:34.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.783+0000 7fe913e00640 1 -- 172.21.15.7:0/1456532091 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v115) ==== 118+0+0 (secure 0 0 0) 0x7fe91c0b4700 con 0x7fe8fc0039b0 2024-04-25T11:22:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 -- 172.21.15.7:0/1456532091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8f80744e0 msgr2=0x7fe8f80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8f80744e0 0x7fe8f80769f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe904005fd0 tx=0x7fe904001ab0 comp rx=0 tx=0).stop 2024-04-25T11:22:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 -- 172.21.15.7:0/1456532091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 msgr2=0x7fe91c15d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 0x7fe91c15d480 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009a90 tx=0x7fe90c004730 comp rx=0 tx=0).stop 2024-04-25T11:22:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 -- 172.21.15.7:0/1456532091 shutdown_connections 2024-04-25T11:22:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe8f80744e0 0x7fe8f80769f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe91c0b3db0 0x7fe91c0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe91c0cf280 0x7fe91c0cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:34.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 --2- 172.21.15.7:0/1456532091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc0039b0 0x7fe91c15d480 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 -- 172.21.15.7:0/1456532091 >> 172.21.15.7:0/1456532091 conn(0x7fe91c0d2840 msgr2=0x7fe91c161c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 -- 172.21.15.7:0/1456532091 shutdown_connections 2024-04-25T11:22:34.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:34.787+0000 7fe923000640 1 -- 172.21.15.7:0/1456532091 wait complete. 2024-04-25T11:22:34.811 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T11:22:34.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T11:22:35.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.101+0000 7fd6c5000640 1 Processor -- start 2024-04-25T11:22:35.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.101+0000 7fd6c5000640 1 -- start start 2024-04-25T11:22:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.104+0000 7fd6c5000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd6c015dfb0 0x7fd6c00b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.104+0000 7fd6c5000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c00d0b40 0x7fd6c006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.104+0000 7fd6c5000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c015e3c0 con 0x7fd6c00d0f50 2024-04-25T11:22:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.104+0000 7fd6c5000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd6c00b09b0 con 0x7fd6c015dfb0 2024-04-25T11:22:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.104+0000 7fd6c5000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c00cf800 con 0x7fd6c00d0b40 2024-04-25T11:22:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.104+0000 7fd6be800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd6c00d0f50 0x7fd6c0161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39956/0 (socket says 172.21.15.7:39956) 2024-04-25T11:22:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.104+0000 7fd6be800640 1 -- 172.21.15.7:0/388867640 learned_addr learned my addr 172.21.15.7:0/388867640 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.105+0000 7fd6b5400640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c00d0b40 0x7fd6c006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.105+0000 7fd6bde00640 1 --2- 172.21.15.7:0/388867640 >> v2:172.21.15.87:3300/0 conn(0x7fd6c015dfb0 0x7fd6c00b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.105+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1228893698 0 0) 0x7fd6c015e3c0 con 0x7fd6c00d0f50 2024-04-25T11:22:35.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.105+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6a4003520 con 0x7fd6c00d0f50 2024-04-25T11:22:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.105+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 744693906 0 0) 0x7fd6a4003520 con 0x7fd6c00d0f50 2024-04-25T11:22:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c00d0b40 msgr2=0x7fd6c006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c00d0b40 0x7fd6c006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 >> v2:172.21.15.87:3300/0 conn(0x7fd6c015dfb0 msgr2=0x7fd6c00b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 --2- 172.21.15.7:0/388867640 >> v2:172.21.15.87:3300/0 conn(0x7fd6c015dfb0 0x7fd6c00b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c00cfaa0 con 0x7fd6c00d0f50 2024-04-25T11:22:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fd6b8001670 con 0x7fd6c00d0f50 2024-04-25T11:22:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 >> v1:172.21.15.7:6789/0 conn(0x7fd6c00d0f50 legacy=0x7fd6c0161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 0x7fd6a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c00d0b40 0x7fd6a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:35.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6a4004d40 0x7fd6a4005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c00b09b0 con 0x7fd6a40039b0 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c00cf800 con 0x7fd6a4004d40 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a400fe00 con 0x7fd6c00d0b40 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6bde00640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 0x7fd6a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6b5400640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c00d0b40 0x7fd6a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.106+0000 7fd6be800640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6a4004d40 0x7fd6a4005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.107+0000 7fd6bde00640 1 -- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c00d0b40 msgr2=0x7fd6a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.107+0000 7fd6bde00640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c00d0b40 0x7fd6a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.107+0000 7fd6bde00640 1 -- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6a4004d40 msgr2=0x7fd6a4005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.107+0000 7fd6bde00640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6a4004d40 0x7fd6a4005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.107+0000 7fd6bde00640 1 -- 172.21.15.7:0/388867640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a4003520 con 0x7fd6a40039b0 2024-04-25T11:22:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.107+0000 7fd6be800640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6a4004d40 0x7fd6a4005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.108+0000 7fd6bde00640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 0x7fd6a4003e20 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0072df0 tx=0x7fd6a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b65e3b7eefc4f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.108+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0009c10 con 0x7fd6a40039b0 2024-04-25T11:22:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.108+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0036070 con 0x7fd6a40039b0 2024-04-25T11:22:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.108+0000 7fd6bd400640 1 -- 172.21.15.7:0/388867640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0031630 con 0x7fd6a40039b0 2024-04-25T11:22:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.109+0000 7fd6c5000640 1 -- 172.21.15.7:0/388867640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 msgr2=0x7fd6a4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.109+0000 7fd6c5000640 1 --2- 172.21.15.7:0/388867640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 0x7fd6a4003e20 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0072df0 tx=0x7fd6a4005830 comp rx=0 tx=0).stop 2024-04-25T11:22:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.109+0000 7fd6be800640 1 -- 172.21.15.7:0/388867640 reap_dead start 2024-04-25T11:22:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.109+0000 7fd6c5000640 1 -- 172.21.15.7:0/388867640 shutdown_connections 2024-04-25T11:22:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.109+0000 7fd6c5000640 1 -- 172.21.15.7:0/388867640 >> 172.21.15.7:0/388867640 conn(0x7fd6c00742c0 msgr2=0x7fd6c006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.109+0000 7fd6c5000640 1 -- 172.21.15.7:0/388867640 shutdown_connections 2024-04-25T11:22:35.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.109+0000 7fd6c5000640 1 -- 172.21.15.7:0/388867640 wait complete. 2024-04-25T11:22:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.111+0000 7fd6c5000640 1 Processor -- start 2024-04-25T11:22:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.112+0000 7fd6c5000640 1 -- start start 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.113+0000 7fd6c5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 0x7fd6c015edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.113+0000 7fd6c5000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c015c0e0 0x7fd6c015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.113+0000 7fd6c5000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6c015ae10 0x7fd6c015b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.113+0000 7fd6c5000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c00d0340 con 0x7fd6a40039b0 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.113+0000 7fd6c5000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c00d0060 con 0x7fd6c015ae10 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.113+0000 7fd6c5000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c00d0620 con 0x7fd6c015c0e0 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.113+0000 7fd6bde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 0x7fd6c015edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.113+0000 7fd6bde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 0x7fd6c015edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34486/0 (socket says 172.21.15.7:34486) 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.113+0000 7fd6bde00640 1 -- 172.21.15.7:0/3988669502 learned_addr learned my addr 172.21.15.7:0/3988669502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.114+0000 7fd6b5400640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c015c0e0 0x7fd6c015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.114+0000 7fd6be800640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6c015ae10 0x7fd6c015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.114+0000 7fd6bde00640 1 -- 172.21.15.7:0/3988669502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c015c0e0 msgr2=0x7fd6c015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.114+0000 7fd6bde00640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c015c0e0 0x7fd6c015c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.114+0000 7fd6bde00640 1 -- 172.21.15.7:0/3988669502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6c015ae10 msgr2=0x7fd6c015b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.114+0000 7fd6bde00640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6c015ae10 0x7fd6c015b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.114+0000 7fd6bde00640 1 -- 172.21.15.7:0/3988669502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c015ca30 con 0x7fd6a40039b0 2024-04-25T11:22:35.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.114+0000 7fd6be800640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6c015ae10 0x7fd6c015b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.114+0000 7fd6bde00640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 0x7fd6c015edd0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0033040 tx=0x7fd6b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.115+0000 7fd6b6a00640 1 -- 172.21.15.7:0/3988669502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0009c10 con 0x7fd6a40039b0 2024-04-25T11:22:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.115+0000 7fd6b6a00640 1 -- 172.21.15.7:0/3988669502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0045650 con 0x7fd6a40039b0 2024-04-25T11:22:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.115+0000 7fd6b6a00640 1 -- 172.21.15.7:0/3988669502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0045a40 con 0x7fd6a40039b0 2024-04-25T11:22:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.115+0000 7fd6c5000640 1 -- 172.21.15.7:0/3988669502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c015b760 con 0x7fd6a40039b0 2024-04-25T11:22:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.115+0000 7fd6c5000640 1 -- 172.21.15.7:0/3988669502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c015a050 con 0x7fd6a40039b0 2024-04-25T11:22:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.116+0000 7fd6c5000640 1 -- 172.21.15.7:0/3988669502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c0050740 con 0x7fd6a40039b0 2024-04-25T11:22:35.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.117+0000 7fd6b6a00640 1 -- 172.21.15.7:0/3988669502 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd6b0005d10 con 0x7fd6a40039b0 2024-04-25T11:22:35.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.118+0000 7fd6b6a00640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd69c0744b0 0x7fd69c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:35.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.118+0000 7fd6b6a00640 1 -- 172.21.15.7:0/3988669502 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7fd6b002e030 con 0x7fd6a40039b0 2024-04-25T11:22:35.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.119+0000 7fd6b5400640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd69c0744b0 0x7fd69c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:35.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.119+0000 7fd6b5400640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd69c0744b0 0x7fd69c0769c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0161620 tx=0x7fd6a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:35.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.124+0000 7fd6b6a00640 1 -- 172.21.15.7:0/3988669502 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd6b00b8e20 con 0x7fd6a40039b0 2024-04-25T11:22:35.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.371+0000 7fd6c5000640 1 -- 172.21.15.7:0/3988669502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd6c0159920 con 0x7fd6a40039b0 2024-04-25T11:22:35.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.796+0000 7fd6b6a00640 1 -- 172.21.15.7:0/3988669502 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v116) ==== 122+0+0 (secure 0 0 0) 0x7fd6c015ca30 con 0x7fd6a40039b0 2024-04-25T11:22:35.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.799+0000 7fd6c5000640 1 -- 172.21.15.7:0/3988669502 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd69c0744b0 msgr2=0x7fd69c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:35.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.799+0000 7fd6c5000640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd69c0744b0 0x7fd69c0769c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0161620 tx=0x7fd6a8008040 comp rx=0 tx=0).stop 2024-04-25T11:22:35.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.799+0000 7fd6c5000640 1 -- 172.21.15.7:0/3988669502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 msgr2=0x7fd6c015edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:35.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.799+0000 7fd6c5000640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 0x7fd6c015edd0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0033040 tx=0x7fd6b0009750 comp rx=0 tx=0).stop 2024-04-25T11:22:35.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.800+0000 7fd6c5000640 1 -- 172.21.15.7:0/3988669502 shutdown_connections 2024-04-25T11:22:35.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.800+0000 7fd6c5000640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd69c0744b0 0x7fd69c0769c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:35.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.800+0000 7fd6c5000640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6c015ae10 0x7fd6c015b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:35.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.800+0000 7fd6c5000640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6c015c0e0 0x7fd6c015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:35.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.800+0000 7fd6c5000640 1 --2- 172.21.15.7:0/3988669502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6a40039b0 0x7fd6c015edd0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:35.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.800+0000 7fd6c5000640 1 -- 172.21.15.7:0/3988669502 >> 172.21.15.7:0/3988669502 conn(0x7fd6c00742c0 msgr2=0x7fd6c00705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:35.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.800+0000 7fd6c5000640 1 -- 172.21.15.7:0/3988669502 shutdown_connections 2024-04-25T11:22:35.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:35.800+0000 7fd6c5000640 1 -- 172.21.15.7:0/3988669502 wait complete. 2024-04-25T11:22:35.833 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T11:22:35.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:36.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.114+0000 7f8809a00640 1 Processor -- start 2024-04-25T11:22:36.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.114+0000 7f8809a00640 1 -- start start 2024-04-25T11:22:36.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.117+0000 7f8809a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88040b0290 0x7f88040b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.117+0000 7f8809a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f880406f300 0x7f88040b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.117+0000 7f8809a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88040b4980 con 0x7f880406dc90 2024-04-25T11:22:36.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.117+0000 7f8809a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f88040b4c60 con 0x7f880406f300 2024-04-25T11:22:36.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.117+0000 7f8809a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f88040b4f40 con 0x7f88040b0290 2024-04-25T11:22:36.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.117+0000 7f8802a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f880406dc90 0x7f88040afb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39970/0 (socket says 172.21.15.7:39970) 2024-04-25T11:22:36.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.117+0000 7f8802a00640 1 -- 172.21.15.7:0/2153461517 learned_addr learned my addr 172.21.15.7:0/2153461517 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:36.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8802000640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88040b0290 0x7f88040b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1494141921 0 0) 0x7f88040b4980 con 0x7f880406dc90 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8803400640 1 --2- 172.21.15.7:0/2153461517 >> v2:172.21.15.87:3300/0 conn(0x7f880406f300 0x7f88040b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87e8003520 con 0x7f880406dc90 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 252883700 0 0) 0x7f87e8003520 con 0x7f880406dc90 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88040b0290 msgr2=0x7f88040b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8801600640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88040b0290 0x7f88040b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 >> v2:172.21.15.87:3300/0 conn(0x7f880406f300 msgr2=0x7f88040b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8801600640 1 --2- 172.21.15.7:0/2153461517 >> v2:172.21.15.87:3300/0 conn(0x7f880406f300 0x7f88040b5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88040b5220 con 0x7f880406dc90 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f87f4001670 con 0x7f880406dc90 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 >> v1:172.21.15.7:6789/0 conn(0x7f880406dc90 legacy=0x7f88040afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.118+0000 7f8801600640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e80039b0 0x7f87e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8801600640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e8004800 0x7f87e8004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8801600640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e8005150 0x7f87e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88040b4c60 con 0x7f87e80039b0 2024-04-25T11:22:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f88040b4f40 con 0x7f87e8005150 2024-04-25T11:22:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87e800bf00 con 0x7f87e8004800 2024-04-25T11:22:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8802a00640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e80039b0 0x7f87e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8802000640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e8004800 0x7f87e8004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8802a00640 1 -- 172.21.15.7:0/2153461517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e8004800 msgr2=0x7f87e8004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8802a00640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e8004800 0x7f87e8004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8802a00640 1 -- 172.21.15.7:0/2153461517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e8005150 msgr2=0x7f87e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8803400640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e8005150 0x7f87e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8802a00640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e8005150 0x7f87e800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.119+0000 7f8802a00640 1 -- 172.21.15.7:0/2153461517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e8003520 con 0x7f87e80039b0 2024-04-25T11:22:36.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.120+0000 7f8802a00640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e80039b0 0x7f87e8003e20 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f88040730d0 tx=0x7f87e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75add51cf84f3256 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:36.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.121+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4007c80 con 0x7f87e80039b0 2024-04-25T11:22:36.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.121+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f4005af0 con 0x7f87e80039b0 2024-04-25T11:22:36.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.121+0000 7f8801600640 1 -- 172.21.15.7:0/2153461517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4007070 con 0x7f87e80039b0 2024-04-25T11:22:36.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.121+0000 7f8809a00640 1 -- 172.21.15.7:0/2153461517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e80039b0 msgr2=0x7f87e8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.121+0000 7f8809a00640 1 --2- 172.21.15.7:0/2153461517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87e80039b0 0x7f87e8003e20 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f88040730d0 tx=0x7f87e8005830 comp rx=0 tx=0).stop 2024-04-25T11:22:36.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.122+0000 7f8803400640 1 -- 172.21.15.7:0/2153461517 reap_dead start 2024-04-25T11:22:36.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.122+0000 7f8809a00640 1 -- 172.21.15.7:0/2153461517 shutdown_connections 2024-04-25T11:22:36.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.122+0000 7f8809a00640 1 -- 172.21.15.7:0/2153461517 >> 172.21.15.7:0/2153461517 conn(0x7f8804074200 msgr2=0x7f880406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:36.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.122+0000 7f8809a00640 1 -- 172.21.15.7:0/2153461517 shutdown_connections 2024-04-25T11:22:36.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.122+0000 7f8809a00640 1 -- 172.21.15.7:0/2153461517 wait complete. 2024-04-25T11:22:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.124+0000 7f8809a00640 1 Processor -- start 2024-04-25T11:22:36.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.124+0000 7f8809a00640 1 -- start start 2024-04-25T11:22:36.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.126+0000 7f8809a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e80039b0 0x7f88041f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.126+0000 7f8809a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f880415f8b0 0x7f88041524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.126+0000 7f8809a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040b1a30 0x7f88040b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.126+0000 7f8809a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88040ce920 con 0x7f88040b1a30 2024-04-25T11:22:36.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.126+0000 7f8809a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f880415d220 con 0x7f87e80039b0 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.126+0000 7f8809a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f88040ceaa0 con 0x7f880415f8b0 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.126+0000 7f8803400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040b1a30 0x7f88040b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.126+0000 7f8803400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040b1a30 0x7f88040b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34502/0 (socket says 172.21.15.7:34502) 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.126+0000 7f8803400640 1 -- 172.21.15.7:0/2021438104 learned_addr learned my addr 172.21.15.7:0/2021438104 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f8802000640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f880415f8b0 0x7f88041524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f8802a00640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e80039b0 0x7f88041f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f8803400640 1 -- 172.21.15.7:0/2021438104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f880415f8b0 msgr2=0x7f88041524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f8803400640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f880415f8b0 0x7f88041524d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f8803400640 1 -- 172.21.15.7:0/2021438104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e80039b0 msgr2=0x7f88041f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f8803400640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e80039b0 0x7f88041f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f8803400640 1 -- 172.21.15.7:0/2021438104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8804152a10 con 0x7f88040b1a30 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f8802a00640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e80039b0 0x7f88041f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f8803400640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040b1a30 0x7f88040b1e40 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f88040d18a0 tx=0x7f87fc00e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f87faa00640 1 -- 172.21.15.7:0/2021438104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc007e20 con 0x7f88040b1a30 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f87faa00640 1 -- 172.21.15.7:0/2021438104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87fc01a070 con 0x7f88040b1a30 2024-04-25T11:22:36.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f87faa00640 1 -- 172.21.15.7:0/2021438104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc015630 con 0x7f88040b1a30 2024-04-25T11:22:36.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f8809a00640 1 -- 172.21.15.7:0/2021438104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880415e810 con 0x7f88040b1a30 2024-04-25T11:22:36.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.127+0000 7f8809a00640 1 -- 172.21.15.7:0/2021438104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880415edd0 con 0x7f88040b1a30 2024-04-25T11:22:36.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.129+0000 7f8809a00640 1 -- 172.21.15.7:0/2021438104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88040d12a0 con 0x7f88040b1a30 2024-04-25T11:22:36.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.130+0000 7f87faa00640 1 -- 172.21.15.7:0/2021438104 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f87fc012030 con 0x7f88040b1a30 2024-04-25T11:22:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.131+0000 7f87faa00640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87e00744e0 0x7f87e00769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.131+0000 7f87faa00640 1 -- 172.21.15.7:0/2021438104 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f87fc095b80 con 0x7f88040b1a30 2024-04-25T11:22:36.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.131+0000 7f8802a00640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87e00744e0 0x7f87e00769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.132+0000 7f8802a00640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87e00744e0 0x7f87e00769f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f87f4032000 tx=0x7f87f4009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:36.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.136+0000 7f87faa00640 1 -- 172.21.15.7:0/2021438104 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f88040d12a0 con 0x7f88040b1a30 2024-04-25T11:22:36.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.379+0000 7f8809a00640 1 -- 172.21.15.7:0/2021438104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f880406a100 con 0x7f88040b1a30 2024-04-25T11:22:36.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.381+0000 7f87faa00640 1 -- 172.21.15.7:0/2021438104 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5179 (secure 0 0 0) 0x7f87fc010040 con 0x7f88040b1a30 2024-04-25T11:22:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:36.376 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7588,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2938174159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2938174159},{"type":"v1","addr":"172.21.15.87:6834","nonce":2938174159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/793358721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":793358721},{"type":"v1","addr":"172.21.15.87:6835","nonce":793358721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":9447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2654444897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2654444897},{"type":"v1","addr":"172.21.15.7:6835","nonce":2654444897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:22:32.315016+0000","modified":"2024-04-25T11:22:35.372928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/339418645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":339418645},{"type":"v1","addr":"172.21.15.7:6837","nonce":339418645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":11}]} 2024-04-25T11:22:36.376 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 116 2024-04-25T11:22:36.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.383+0000 7f8809a00640 1 -- 172.21.15.7:0/2021438104 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87e00744e0 msgr2=0x7f87e00769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.383+0000 7f8809a00640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87e00744e0 0x7f87e00769f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f87f4032000 tx=0x7f87f4009030 comp rx=0 tx=0).stop 2024-04-25T11:22:36.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.383+0000 7f8809a00640 1 -- 172.21.15.7:0/2021438104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040b1a30 msgr2=0x7f88040b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.383+0000 7f8809a00640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040b1a30 0x7f88040b1e40 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f88040d18a0 tx=0x7f87fc00e510 comp rx=0 tx=0).stop 2024-04-25T11:22:36.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.384+0000 7f8809a00640 1 -- 172.21.15.7:0/2021438104 shutdown_connections 2024-04-25T11:22:36.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.384+0000 7f8809a00640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f87e00744e0 0x7f87e00769f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.384+0000 7f8809a00640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040b1a30 0x7f88040b1e40 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.384+0000 7f8809a00640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f880415f8b0 0x7f88041524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.384+0000 7f8809a00640 1 --2- 172.21.15.7:0/2021438104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e80039b0 0x7f88041f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.384+0000 7f8809a00640 1 -- 172.21.15.7:0/2021438104 >> 172.21.15.7:0/2021438104 conn(0x7f8804074200 msgr2=0x7f88041569f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:36.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.384+0000 7f8809a00640 1 -- 172.21.15.7:0/2021438104 shutdown_connections 2024-04-25T11:22:36.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.384+0000 7f8809a00640 1 -- 172.21.15.7:0/2021438104 wait complete. 2024-04-25T11:22:36.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.672+0000 7f498ca00640 1 Processor -- start 2024-04-25T11:22:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.672+0000 7f498ca00640 1 -- start start 2024-04-25T11:22:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.674+0000 7f498ca00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4988160590 0x7f49880d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.674+0000 7f498ca00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49880d35a0 con 0x7f498806dcd0 2024-04-25T11:22:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.675+0000 7f498ca00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4988164310 con 0x7f4988163ac0 2024-04-25T11:22:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.675+0000 7f498ca00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f49881645f0 con 0x7f4988160590 2024-04-25T11:22:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.675+0000 7f4985e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f498806dcd0 0x7f498806e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39982/0 (socket says 172.21.15.7:39982) 2024-04-25T11:22:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.675+0000 7f4985e00640 1 -- 172.21.15.7:0/1653343665 learned_addr learned my addr 172.21.15.7:0/1653343665 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.675+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 533744734 0 0) 0x7f49880d35a0 con 0x7f498806dcd0 2024-04-25T11:22:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.675+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f496c003520 con 0x7f498806dcd0 2024-04-25T11:22:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1475862105 0 0) 0x7f496c003520 con 0x7f498806dcd0 2024-04-25T11:22:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4987200640 1 --2- 172.21.15.7:0/1653343665 >> v2:172.21.15.87:3300/0 conn(0x7f4988160590 0x7f49880d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 >> v2:172.21.15.87:3300/0 conn(0x7f4988160590 msgr2=0x7f49880d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 --2- 172.21.15.7:0/1653343665 >> v2:172.21.15.87:3300/0 conn(0x7f4988160590 0x7f49880d3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 >> v1:172.21.15.87:6789/0 conn(0x7f4988163ac0 legacy=0x7f49880b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49881648d0 con 0x7f498806dcd0 2024-04-25T11:22:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f4970001670 con 0x7f498806dcd0 2024-04-25T11:22:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4987200640 1 --2- 172.21.15.7:0/1653343665 >> v2:172.21.15.87:3300/0 conn(0x7f4988160590 0x7f49880d3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 >> v1:172.21.15.7:6789/0 conn(0x7f498806dcd0 legacy=0x7f498806e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f496c003a00 0x7f496c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f496c004800 0x7f496c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 0x7f496c00bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49881645f0 con 0x7f496c0099b0 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f496c00c3f0 con 0x7f496c004800 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f496c00c5f0 con 0x7f496c003a00 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.676+0000 7f4987200640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 0x7f496c00bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.677+0000 7f4986800640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f496c003a00 0x7f496c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.677+0000 7f4985e00640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f496c004800 0x7f496c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.677+0000 7f4987200640 1 -- 172.21.15.7:0/1653343665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f496c003a00 msgr2=0x7f496c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.677+0000 7f4987200640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f496c003a00 0x7f496c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.677+0000 7f4987200640 1 -- 172.21.15.7:0/1653343665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f496c004800 msgr2=0x7f496c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.677+0000 7f4987200640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f496c004800 0x7f496c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.677+0000 7f4987200640 1 -- 172.21.15.7:0/1653343665 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496c003520 con 0x7f496c0099b0 2024-04-25T11:22:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.677+0000 7f4985e00640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f496c004800 0x7f496c004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.678+0000 7f4987200640 1 -- 172.21.15.7:0/1653343665 reap_dead start 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.678+0000 7f4987200640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 0x7f496c00bdd0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f498000aa80 tx=0x7f496c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350caf9953870896 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.678+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980037070 con 0x7f496c0099b0 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.678+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f498002ac70 con 0x7f496c0099b0 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.678+0000 7f4985400640 1 -- 172.21.15.7:0/1653343665 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498000a650 con 0x7f496c0099b0 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.679+0000 7f498ca00640 1 -- 172.21.15.7:0/1653343665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 msgr2=0x7f496c00bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.679+0000 7f498ca00640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 0x7f496c00bdd0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f498000aa80 tx=0x7f496c005830 comp rx=0 tx=0).stop 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.679+0000 7f498ca00640 1 -- 172.21.15.7:0/1653343665 shutdown_connections 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.679+0000 7f498ca00640 1 --2- 172.21.15.7:0/1653343665 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 0x7f496c00bdd0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.679+0000 7f498ca00640 1 -- 172.21.15.7:0/1653343665 >> 172.21.15.7:0/1653343665 conn(0x7f49880d29e0 msgr2=0x7f49880d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.679+0000 7f498ca00640 1 -- 172.21.15.7:0/1653343665 shutdown_connections 2024-04-25T11:22:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.679+0000 7f498ca00640 1 -- 172.21.15.7:0/1653343665 wait complete. 2024-04-25T11:22:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.681+0000 7f498ca00640 1 Processor -- start 2024-04-25T11:22:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.681+0000 7f498ca00640 1 -- start start 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.683+0000 7f498ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 0x7f498815d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.683+0000 7f498ca00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f498815f220 0x7f49881ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.683+0000 7f498ca00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49881ebff0 0x7f49880b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.683+0000 7f498ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f498806e0e0 con 0x7f496c0099b0 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.683+0000 7f498ca00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f498806e260 con 0x7f49881ebff0 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.683+0000 7f498ca00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f498806e3e0 con 0x7f498815f220 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.683+0000 7f4986800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 0x7f498815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.683+0000 7f4986800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 0x7f498815d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34516/0 (socket says 172.21.15.7:34516) 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.683+0000 7f4986800640 1 -- 172.21.15.7:0/2438229022 learned_addr learned my addr 172.21.15.7:0/2438229022 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f4985e00640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f498815f220 0x7f49881ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f4987200640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49881ebff0 0x7f49880b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f4986800640 1 -- 172.21.15.7:0/2438229022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f498815f220 msgr2=0x7f49881ebab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f4986800640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f498815f220 0x7f49881ebab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f4986800640 1 -- 172.21.15.7:0/2438229022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49881ebff0 msgr2=0x7f49880b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f4986800640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49881ebff0 0x7f49880b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f4986800640 1 -- 172.21.15.7:0/2438229022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49881ec400 con 0x7f496c0099b0 2024-04-25T11:22:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f4986800640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 0x7f498815d720 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f497800e920 tx=0x7f497800ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f497ea00640 1 -- 172.21.15.7:0/2438229022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4978003b80 con 0x7f496c0099b0 2024-04-25T11:22:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f497ea00640 1 -- 172.21.15.7:0/2438229022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4978018070 con 0x7f496c0099b0 2024-04-25T11:22:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f497ea00640 1 -- 172.21.15.7:0/2438229022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49780050d0 con 0x7f496c0099b0 2024-04-25T11:22:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.684+0000 7f498ca00640 1 -- 172.21.15.7:0/2438229022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49880adcc0 con 0x7f496c0099b0 2024-04-25T11:22:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.685+0000 7f498ca00640 1 -- 172.21.15.7:0/2438229022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49880ae280 con 0x7f496c0099b0 2024-04-25T11:22:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.686+0000 7f498ca00640 1 -- 172.21.15.7:0/2438229022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4988050740 con 0x7f496c0099b0 2024-04-25T11:22:36.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.687+0000 7f497ea00640 1 -- 172.21.15.7:0/2438229022 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4978005270 con 0x7f496c0099b0 2024-04-25T11:22:36.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.688+0000 7f497ea00640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49640744b0 0x7f49640769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:36.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.688+0000 7f497ea00640 1 -- 172.21.15.7:0/2438229022 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f49780954f0 con 0x7f496c0099b0 2024-04-25T11:22:36.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.688+0000 7f4985e00640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49640744b0 0x7f49640769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:36.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.689+0000 7f4985e00640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49640744b0 0x7f49640769c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4988160140 tx=0x7f4970009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:36.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.693+0000 7f497ea00640 1 -- 172.21.15.7:0/2438229022 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f49881ec400 con 0x7f496c0099b0 2024-04-25T11:22:36.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.895+0000 7f498ca00640 1 -- 172.21.15.7:0/2438229022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49880d4f80 con 0x7f496c0099b0 2024-04-25T11:22:36.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.898+0000 7f497ea00640 1 -- 172.21.15.7:0/2438229022 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16788 (secure 0 0 0) 0x7f497805fa50 con 0x7f496c0099b0 2024-04-25T11:22:36.894 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:36.894 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:33.767424+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:22:30.472409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T11:22:31.299177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:36.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.901+0000 7f498ca00640 1 -- 172.21.15.7:0/2438229022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49640744b0 msgr2=0x7f49640769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.901+0000 7f498ca00640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49640744b0 0x7f49640769c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4988160140 tx=0x7f4970009040 comp rx=0 tx=0).stop 2024-04-25T11:22:36.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.901+0000 7f498ca00640 1 -- 172.21.15.7:0/2438229022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 msgr2=0x7f498815d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.901+0000 7f498ca00640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 0x7f498815d720 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f497800e920 tx=0x7f497800ede0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.901+0000 7f498ca00640 1 -- 172.21.15.7:0/2438229022 shutdown_connections 2024-04-25T11:22:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.901+0000 7f498ca00640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f49640744b0 0x7f49640769c0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.901+0000 7f498ca00640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49881ebff0 0x7f49880b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.901+0000 7f498ca00640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f498815f220 0x7f49881ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.901+0000 7f498ca00640 1 --2- 172.21.15.7:0/2438229022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c0099b0 0x7f498815d720 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.901+0000 7f498ca00640 1 -- 172.21.15.7:0/2438229022 >> 172.21.15.7:0/2438229022 conn(0x7f49880d29e0 msgr2=0x7f4988162b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.901+0000 7f498ca00640 1 -- 172.21.15.7:0/2438229022 shutdown_connections 2024-04-25T11:22:36.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:36.902+0000 7f498ca00640 1 -- 172.21.15.7:0/2438229022 wait complete. 2024-04-25T11:22:36.923 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T11:22:36.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:22:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.185+0000 7f454be00640 1 Processor -- start 2024-04-25T11:22:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.185+0000 7f454be00640 1 -- start start 2024-04-25T11:22:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.187+0000 7f454be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c0d03a0 0x7f454c0bae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.187+0000 7f454be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f454c06c570 0x7f454c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.187+0000 7f454be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f454c06cec0 con 0x7f454c163bb0 2024-04-25T11:22:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.187+0000 7f454be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f454c0b51c0 con 0x7f454c06c570 2024-04-25T11:22:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.187+0000 7f454be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f454c0bb3d0 con 0x7f454c0d03a0 2024-04-25T11:22:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.188+0000 7f454a000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c0d03a0 0x7f454c0bae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.188+0000 7f454b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f454c06c570 0x7f454c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.188+0000 7f454a000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c0d03a0 0x7f454c0bae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:48160/0 (socket says 172.21.15.7:48160) 2024-04-25T11:22:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.188+0000 7f454a000640 1 -- 172.21.15.7:0/964530007 learned_addr learned my addr 172.21.15.7:0/964530007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.189+0000 7f454b400640 1 -- 172.21.15.7:0/964530007 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c0d03a0 msgr2=0x7f454c0bae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.189+0000 7f454b400640 1 --2- 172.21.15.7:0/964530007 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c0d03a0 0x7f454c0bae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.189+0000 7f454b400640 1 -- 172.21.15.7:0/964530007 >> v1:172.21.15.87:6789/0 conn(0x7f454c163bb0 legacy=0x7f454c0b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.189+0000 7f454b400640 1 -- 172.21.15.7:0/964530007 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454c164120 con 0x7f454c06c570 2024-04-25T11:22:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.189+0000 7f454a000640 1 --2- 172.21.15.7:0/964530007 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c0d03a0 0x7f454c0bae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:22:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.190+0000 7f454b400640 1 --2- 172.21.15.7:0/964530007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f454c06c570 0x7f454c06c980 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f45440097a0 tx=0x7f454402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da09fba022b1c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.191+0000 7f4549600640 1 -- 172.21.15.7:0/964530007 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544037070 con 0x7f454c06c570 2024-04-25T11:22:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.191+0000 7f4549600640 1 -- 172.21.15.7:0/964530007 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4544032440 con 0x7f454c06c570 2024-04-25T11:22:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.191+0000 7f4549600640 1 -- 172.21.15.7:0/964530007 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544032810 con 0x7f454c06c570 2024-04-25T11:22:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.191+0000 7f454be00640 1 -- 172.21.15.7:0/964530007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f454c06c570 msgr2=0x7f454c06c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.191+0000 7f454be00640 1 --2- 172.21.15.7:0/964530007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f454c06c570 0x7f454c06c980 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f45440097a0 tx=0x7f454402a180 comp rx=0 tx=0).stop 2024-04-25T11:22:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.192+0000 7f454be00640 1 -- 172.21.15.7:0/964530007 shutdown_connections 2024-04-25T11:22:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.192+0000 7f454be00640 1 --2- 172.21.15.7:0/964530007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f454c06c570 0x7f454c06c980 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.192+0000 7f454be00640 1 --2- 172.21.15.7:0/964530007 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c0d03a0 0x7f454c0bae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.192+0000 7f454be00640 1 -- 172.21.15.7:0/964530007 >> 172.21.15.7:0/964530007 conn(0x7f454c06eb00 msgr2=0x7f454c0d59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.192+0000 7f454be00640 1 -- 172.21.15.7:0/964530007 shutdown_connections 2024-04-25T11:22:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.192+0000 7f454be00640 1 -- 172.21.15.7:0/964530007 wait complete. 2024-04-25T11:22:37.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.194+0000 7f454be00640 1 Processor -- start 2024-04-25T11:22:37.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.194+0000 7f454be00640 1 -- start start 2024-04-25T11:22:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c06c570 0x7f454c0ac560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f454c0d03a0 0x7f454c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f454c163bb0 0x7f454c1f9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f454c1649c0 con 0x7f454c0d03a0 2024-04-25T11:22:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f454c1646e0 con 0x7f454c163bb0 2024-04-25T11:22:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f454c0d4170 con 0x7f454c06c570 2024-04-25T11:22:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f454c0d03a0 0x7f454c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f454c0d03a0 0x7f454c0cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34518/0 (socket says 172.21.15.7:34518) 2024-04-25T11:22:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454a000640 1 -- 172.21.15.7:0/2501809357 learned_addr learned my addr 172.21.15.7:0/2501809357 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454aa00640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c06c570 0x7f454c0ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454b400640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f454c163bb0 0x7f454c1f9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454a000640 1 -- 172.21.15.7:0/2501809357 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c06c570 msgr2=0x7f454c0ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454a000640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c06c570 0x7f454c0ac560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454a000640 1 -- 172.21.15.7:0/2501809357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f454c163bb0 msgr2=0x7f454c1f9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454a000640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f454c163bb0 0x7f454c1f9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.196+0000 7f454a000640 1 -- 172.21.15.7:0/2501809357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454c0cea10 con 0x7f454c0d03a0 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.197+0000 7f454a000640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f454c0d03a0 0x7f454c0cdce0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f453400e9b0 tx=0x7f453400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.197+0000 7f4542a00640 1 -- 172.21.15.7:0/2501809357 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534017070 con 0x7f454c0d03a0 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.197+0000 7f4542a00640 1 -- 172.21.15.7:0/2501809357 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f453400c9a0 con 0x7f454c0d03a0 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.197+0000 7f454be00640 1 -- 172.21.15.7:0/2501809357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454c0cecf0 con 0x7f454c0d03a0 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.197+0000 7f4542a00640 1 -- 172.21.15.7:0/2501809357 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453400cd50 con 0x7f454c0d03a0 2024-04-25T11:22:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.197+0000 7f454be00640 1 -- 172.21.15.7:0/2501809357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f454c0cf2b0 con 0x7f454c0d03a0 2024-04-25T11:22:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.198+0000 7f454be00640 1 -- 172.21.15.7:0/2501809357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f454c0ce780 con 0x7f454c0d03a0 2024-04-25T11:22:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.200+0000 7f4542a00640 1 -- 172.21.15.7:0/2501809357 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f453401a070 con 0x7f454c0d03a0 2024-04-25T11:22:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.201+0000 7f4542a00640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45280744b0 0x7f45280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.201+0000 7f4542a00640 1 -- 172.21.15.7:0/2501809357 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f4534095380 con 0x7f454c0d03a0 2024-04-25T11:22:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.201+0000 7f454aa00640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45280744b0 0x7f45280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.202+0000 7f454aa00640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45280744b0 0x7f45280769c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f453c001ca0 tx=0x7f453c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.206+0000 7f4542a00640 1 -- 172.21.15.7:0/2501809357 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f453405f8e0 con 0x7f454c0d03a0 2024-04-25T11:22:37.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.401+0000 7f454be00640 1 -- 172.21.15.7:0/2501809357 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f454c050740 con 0x7f45280744b0 2024-04-25T11:22:37.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.421+0000 7f4542a00640 1 -- 172.21.15.7:0/2501809357 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252459 (secure 0 0 0) 0x7f454c050740 con 0x7f45280744b0 2024-04-25T11:22:37.417 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:37.421 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:22:37.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 -- 172.21.15.7:0/2501809357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45280744b0 msgr2=0x7f45280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:37.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45280744b0 0x7f45280769c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f453c001ca0 tx=0x7f453c009040 comp rx=0 tx=0).stop 2024-04-25T11:22:37.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 -- 172.21.15.7:0/2501809357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f454c0d03a0 msgr2=0x7f454c0cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f454c0d03a0 0x7f454c0cdce0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f453400e9b0 tx=0x7f453400ee70 comp rx=0 tx=0).stop 2024-04-25T11:22:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 -- 172.21.15.7:0/2501809357 shutdown_connections 2024-04-25T11:22:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f45280744b0 0x7f45280769c0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f454c163bb0 0x7f454c1f9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f454c0d03a0 0x7f454c0cdce0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 --2- 172.21.15.7:0/2501809357 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f454c06c570 0x7f454c0ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 -- 172.21.15.7:0/2501809357 >> 172.21.15.7:0/2501809357 conn(0x7f454c06eb00 msgr2=0x7f454c0d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 -- 172.21.15.7:0/2501809357 shutdown_connections 2024-04-25T11:22:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.427+0000 7f454be00640 1 -- 172.21.15.7:0/2501809357 wait complete. 2024-04-25T11:22:37.448 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-04-25T11:22:36.073422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":286728,"kb_used_data":16444,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687992,"statfs":{"total":773094113280,"available":772800503808,"internally_reserved":0,"allocated":16838656,"data_stored":8420008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.339282"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792688+0000","last_change":"2024-04-25T11:22:32.780335+0000","last_active":"2024-04-25T11:22:33.792688+0000","last_peered":"2024-04-25T11:22:33.792688+0000","last_clean":"2024-04-25T11:22:33.792688+0000","last_became_active":"2024-04-25T11:22:32.779139+0000","last_became_peered":"2024-04-25T11:22:32.779139+0000","last_unstale":"2024-04-25T11:22:33.792688+0000","last_undegraded":"2024-04-25T11:22:33.792688+0000","last_fullsized":"2024-04-25T11:22:33.792688+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:40.870281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792327+0000","last_change":"2024-04-25T11:22:32.773514+0000","last_active":"2024-04-25T11:22:33.792327+0000","last_peered":"2024-04-25T11:22:33.792327+0000","last_clean":"2024-04-25T11:22:33.792327+0000","last_became_active":"2024-04-25T11:22:32.771229+0000","last_became_peered":"2024-04-25T11:22:32.771229+0000","last_unstale":"2024-04-25T11:22:33.792327+0000","last_undegraded":"2024-04-25T11:22:33.792327+0000","last_fullsized":"2024-04-25T11:22:33.792327+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:48.079910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793786+0000","last_change":"2024-04-25T11:22:32.772354+0000","last_active":"2024-04-25T11:22:33.793786+0000","last_peered":"2024-04-25T11:22:33.793786+0000","last_clean":"2024-04-25T11:22:33.793786+0000","last_became_active":"2024-04-25T11:22:32.771896+0000","last_became_peered":"2024-04-25T11:22:32.771896+0000","last_unstale":"2024-04-25T11:22:33.793786+0000","last_undegraded":"2024-04-25T11:22:33.793786+0000","last_fullsized":"2024-04-25T11:22:33.793786+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:13.443059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.794108+0000","last_change":"2024-04-25T11:22:32.786253+0000","last_active":"2024-04-25T11:22:33.794108+0000","last_peered":"2024-04-25T11:22:33.794108+0000","last_clean":"2024-04-25T11:22:33.794108+0000","last_became_active":"2024-04-25T11:22:32.785932+0000","last_became_peered":"2024-04-25T11:22:32.785932+0000","last_unstale":"2024-04-25T11:22:33.794108+0000","last_undegraded":"2024-04-25T11:22:33.794108+0000","last_fullsized":"2024-04-25T11:22:33.794108+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:37.493955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187703+0000","last_change":"2024-04-25T11:22:32.781582+0000","last_active":"2024-04-25T11:22:34.187703+0000","last_peered":"2024-04-25T11:22:34.187703+0000","last_clean":"2024-04-25T11:22:34.187703+0000","last_became_active":"2024-04-25T11:22:32.781269+0000","last_became_peered":"2024-04-25T11:22:32.781269+0000","last_unstale":"2024-04-25T11:22:34.187703+0000","last_undegraded":"2024-04-25T11:22:34.187703+0000","last_fullsized":"2024-04-25T11:22:34.187703+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:35.764125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.790984+0000","last_change":"2024-04-25T11:22:32.770279+0000","last_active":"2024-04-25T11:22:33.790984+0000","last_peered":"2024-04-25T11:22:33.790984+0000","last_clean":"2024-04-25T11:22:33.790984+0000","last_became_active":"2024-04-25T11:22:32.768696+0000","last_became_peered":"2024-04-25T11:22:32.768696+0000","last_unstale":"2024-04-25T11:22:33.790984+0000","last_undegraded":"2024-04-25T11:22:33.790984+0000","last_fullsized":"2024-04-25T11:22:33.790984+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:23.288712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.967989+0000","last_change":"2024-04-25T11:22:32.806012+0000","last_active":"2024-04-25T11:22:33.967989+0000","last_peered":"2024-04-25T11:22:33.967989+0000","last_clean":"2024-04-25T11:22:33.967989+0000","last_became_active":"2024-04-25T11:22:32.805607+0000","last_became_peered":"2024-04-25T11:22:32.805607+0000","last_unstale":"2024-04-25T11:22:33.967989+0000","last_undegraded":"2024-04-25T11:22:33.967989+0000","last_fullsized":"2024-04-25T11:22:33.967989+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:52.430209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.967965+0000","last_change":"2024-04-25T11:22:32.775451+0000","last_active":"2024-04-25T11:22:33.967965+0000","last_peered":"2024-04-25T11:22:33.967965+0000","last_clean":"2024-04-25T11:22:33.967965+0000","last_became_active":"2024-04-25T11:22:32.775143+0000","last_became_peered":"2024-04-25T11:22:32.775143+0000","last_unstale":"2024-04-25T11:22:33.967965+0000","last_undegraded":"2024-04-25T11:22:33.967965+0000","last_fullsized":"2024-04-25T11:22:33.967965+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:03.032065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.969000+0000","last_change":"2024-04-25T11:22:32.783643+0000","last_active":"2024-04-25T11:22:33.969000+0000","last_peered":"2024-04-25T11:22:33.969000+0000","last_clean":"2024-04-25T11:22:33.969000+0000","last_became_active":"2024-04-25T11:22:32.783302+0000","last_became_peered":"2024-04-25T11:22:32.783302+0000","last_unstale":"2024-04-25T11:22:33.969000+0000","last_undegraded":"2024-04-25T11:22:33.969000+0000","last_fullsized":"2024-04-25T11:22:33.969000+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:26.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.969142+0000","last_change":"2024-04-25T11:22:32.784537+0000","last_active":"2024-04-25T11:22:33.969142+0000","last_peered":"2024-04-25T11:22:33.969142+0000","last_clean":"2024-04-25T11:22:33.969142+0000","last_became_active":"2024-04-25T11:22:32.784188+0000","last_became_peered":"2024-04-25T11:22:32.784188+0000","last_unstale":"2024-04-25T11:22:33.969142+0000","last_undegraded":"2024-04-25T11:22:33.969142+0000","last_fullsized":"2024-04-25T11:22:33.969142+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:41.577988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.789000+0000","last_change":"2024-04-25T11:22:32.793029+0000","last_active":"2024-04-25T11:22:33.789000+0000","last_peered":"2024-04-25T11:22:33.789000+0000","last_clean":"2024-04-25T11:22:33.789000+0000","last_became_active":"2024-04-25T11:22:32.792632+0000","last_became_peered":"2024-04-25T11:22:32.792632+0000","last_unstale":"2024-04-25T11:22:33.789000+0000","last_undegraded":"2024-04-25T11:22:33.789000+0000","last_fullsized":"2024-04-25T11:22:33.789000+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:50.860031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793777+0000","last_change":"2024-04-25T11:22:32.771841+0000","last_active":"2024-04-25T11:22:33.793777+0000","last_peered":"2024-04-25T11:22:33.793777+0000","last_clean":"2024-04-25T11:22:33.793777+0000","last_became_active":"2024-04-25T11:22:32.771457+0000","last_became_peered":"2024-04-25T11:22:32.771457+0000","last_unstale":"2024-04-25T11:22:33.793777+0000","last_undegraded":"2024-04-25T11:22:33.793777+0000","last_fullsized":"2024-04-25T11:22:33.793777+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:40.458064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190690+0000","last_change":"2024-04-25T11:22:32.780633+0000","last_active":"2024-04-25T11:22:34.190690+0000","last_peered":"2024-04-25T11:22:34.190690+0000","last_clean":"2024-04-25T11:22:34.190690+0000","last_became_active":"2024-04-25T11:22:32.780324+0000","last_became_peered":"2024-04-25T11:22:32.780324+0000","last_unstale":"2024-04-25T11:22:34.190690+0000","last_undegraded":"2024-04-25T11:22:34.190690+0000","last_fullsized":"2024-04-25T11:22:34.190690+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:49.743490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187155+0000","last_change":"2024-04-25T11:22:32.806816+0000","last_active":"2024-04-25T11:22:34.187155+0000","last_peered":"2024-04-25T11:22:34.187155+0000","last_clean":"2024-04-25T11:22:34.187155+0000","last_became_active":"2024-04-25T11:22:32.804381+0000","last_became_peered":"2024-04-25T11:22:32.804381+0000","last_unstale":"2024-04-25T11:22:34.187155+0000","last_undegraded":"2024-04-25T11:22:34.187155+0000","last_fullsized":"2024-04-25T11:22:34.187155+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:10.466936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.787436+0000","last_change":"2024-04-25T11:22:32.791553+0000","last_active":"2024-04-25T11:22:33.787436+0000","last_peered":"2024-04-25T11:22:33.787436+0000","last_clean":"2024-04-25T11:22:33.787436+0000","last_became_active":"2024-04-25T11:22:32.791045+0000","last_became_peered":"2024-04-25T11:22:32.791045+0000","last_unstale":"2024-04-25T11:22:33.787436+0000","last_undegraded":"2024-04-25T11:22:33.787436+0000","last_fullsized":"2024-04-25T11:22:33.787436+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:03.389243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793008+0000","last_change":"2024-04-25T11:22:32.792648+0000","last_active":"2024-04-25T11:22:33.793008+0000","last_peered":"2024-04-25T11:22:33.793008+0000","last_clean":"2024-04-25T11:22:33.793008+0000","last_became_active":"2024-04-25T11:22:32.792031+0000","last_became_peered":"2024-04-25T11:22:32.792031+0000","last_unstale":"2024-04-25T11:22:33.793008+0000","last_undegraded":"2024-04-25T11:22:33.793008+0000","last_fullsized":"2024-04-25T11:22:33.793008+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:15.199420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.969256+0000","last_change":"2024-04-25T11:22:32.782759+0000","last_active":"2024-04-25T11:22:33.969256+0000","last_peered":"2024-04-25T11:22:33.969256+0000","last_clean":"2024-04-25T11:22:33.969256+0000","last_became_active":"2024-04-25T11:22:32.782415+0000","last_became_peered":"2024-04-25T11:22:32.782415+0000","last_unstale":"2024-04-25T11:22:33.969256+0000","last_undegraded":"2024-04-25T11:22:33.969256+0000","last_fullsized":"2024-04-25T11:22:33.969256+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:41.070365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188366+0000","last_change":"2024-04-25T11:22:32.789726+0000","last_active":"2024-04-25T11:22:34.188366+0000","last_peered":"2024-04-25T11:22:34.188366+0000","last_clean":"2024-04-25T11:22:34.188366+0000","last_became_active":"2024-04-25T11:22:32.789261+0000","last_became_peered":"2024-04-25T11:22:32.789261+0000","last_unstale":"2024-04-25T11:22:34.188366+0000","last_undegraded":"2024-04-25T11:22:34.188366+0000","last_fullsized":"2024-04-25T11:22:34.188366+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:41.300207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.789684+0000","last_change":"2024-04-25T11:22:32.791455+0000","last_active":"2024-04-25T11:22:33.789684+0000","last_peered":"2024-04-25T11:22:33.789684+0000","last_clean":"2024-04-25T11:22:33.789684+0000","last_became_active":"2024-04-25T11:22:32.790983+0000","last_became_peered":"2024-04-25T11:22:32.790983+0000","last_unstale":"2024-04-25T11:22:33.789684+0000","last_undegraded":"2024-04-25T11:22:33.789684+0000","last_fullsized":"2024-04-25T11:22:33.789684+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:49.866011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.795951+0000","last_change":"2024-04-25T11:22:32.790667+0000","last_active":"2024-04-25T11:22:32.795951+0000","last_peered":"2024-04-25T11:22:32.795951+0000","last_clean":"2024-04-25T11:22:32.795951+0000","last_became_active":"2024-04-25T11:22:32.790330+0000","last_became_peered":"2024-04-25T11:22:32.790330+0000","last_unstale":"2024-04-25T11:22:32.795951+0000","last_undegraded":"2024-04-25T11:22:32.795951+0000","last_fullsized":"2024-04-25T11:22:32.795951+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:34.936868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.191045+0000","last_change":"2024-04-25T11:22:32.788504+0000","last_active":"2024-04-25T11:22:34.191045+0000","last_peered":"2024-04-25T11:22:34.191045+0000","last_clean":"2024-04-25T11:22:34.191045+0000","last_became_active":"2024-04-25T11:22:32.786465+0000","last_became_peered":"2024-04-25T11:22:32.786465+0000","last_unstale":"2024-04-25T11:22:34.191045+0000","last_undegraded":"2024-04-25T11:22:34.191045+0000","last_fullsized":"2024-04-25T11:22:34.191045+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:42.312405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.791487+0000","last_change":"2024-04-25T11:22:32.794560+0000","last_active":"2024-04-25T11:22:33.791487+0000","last_peered":"2024-04-25T11:22:33.791487+0000","last_clean":"2024-04-25T11:22:33.791487+0000","last_became_active":"2024-04-25T11:22:32.793560+0000","last_became_peered":"2024-04-25T11:22:32.793560+0000","last_unstale":"2024-04-25T11:22:33.791487+0000","last_undegraded":"2024-04-25T11:22:33.791487+0000","last_fullsized":"2024-04-25T11:22:33.791487+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:03.180193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.186796+0000","last_change":"2024-04-25T11:22:32.795553+0000","last_active":"2024-04-25T11:22:34.186796+0000","last_peered":"2024-04-25T11:22:34.186796+0000","last_clean":"2024-04-25T11:22:34.186796+0000","last_became_active":"2024-04-25T11:22:32.789732+0000","last_became_peered":"2024-04-25T11:22:32.789732+0000","last_unstale":"2024-04-25T11:22:34.186796+0000","last_undegraded":"2024-04-25T11:22:34.186796+0000","last_fullsized":"2024-04-25T11:22:34.186796+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:43.540485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793408+0000","last_change":"2024-04-25T11:22:32.790395+0000","last_active":"2024-04-25T11:22:33.793408+0000","last_peered":"2024-04-25T11:22:33.793408+0000","last_clean":"2024-04-25T11:22:33.793408+0000","last_became_active":"2024-04-25T11:22:32.790039+0000","last_became_peered":"2024-04-25T11:22:32.790039+0000","last_unstale":"2024-04-25T11:22:33.793408+0000","last_undegraded":"2024-04-25T11:22:33.793408+0000","last_fullsized":"2024-04-25T11:22:33.793408+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:20.363721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.191373+0000","last_change":"2024-04-25T11:22:32.787404+0000","last_active":"2024-04-25T11:22:34.191373+0000","last_peered":"2024-04-25T11:22:34.191373+0000","last_clean":"2024-04-25T11:22:34.191373+0000","last_became_active":"2024-04-25T11:22:32.786969+0000","last_became_peered":"2024-04-25T11:22:32.786969+0000","last_unstale":"2024-04-25T11:22:34.191373+0000","last_undegraded":"2024-04-25T11:22:34.191373+0000","last_fullsized":"2024-04-25T11:22:34.191373+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:52.535988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.792512+0000","last_change":"2024-04-25T11:22:32.772739+0000","last_active":"2024-04-25T11:22:32.792512+0000","last_peered":"2024-04-25T11:22:32.792512+0000","last_clean":"2024-04-25T11:22:32.792512+0000","last_became_active":"2024-04-25T11:22:32.771715+0000","last_became_peered":"2024-04-25T11:22:32.771715+0000","last_unstale":"2024-04-25T11:22:32.792512+0000","last_undegraded":"2024-04-25T11:22:32.792512+0000","last_fullsized":"2024-04-25T11:22:32.792512+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:34.670303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.794271+0000","last_change":"2024-04-25T11:22:32.793033+0000","last_active":"2024-04-25T11:22:33.794271+0000","last_peered":"2024-04-25T11:22:33.794271+0000","last_clean":"2024-04-25T11:22:33.794271+0000","last_became_active":"2024-04-25T11:22:32.792648+0000","last_became_peered":"2024-04-25T11:22:32.792648+0000","last_unstale":"2024-04-25T11:22:33.794271+0000","last_undegraded":"2024-04-25T11:22:33.794271+0000","last_fullsized":"2024-04-25T11:22:33.794271+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:43.540485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.189990+0000","last_change":"2024-04-25T11:22:32.788492+0000","last_active":"2024-04-25T11:22:34.189990+0000","last_peered":"2024-04-25T11:22:34.189990+0000","last_clean":"2024-04-25T11:22:34.189990+0000","last_became_active":"2024-04-25T11:22:32.787525+0000","last_became_peered":"2024-04-25T11:22:32.787525+0000","last_unstale":"2024-04-25T11:22:34.189990+0000","last_undegraded":"2024-04-25T11:22:34.189990+0000","last_fullsized":"2024-04-25T11:22:34.189990+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:03.751789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792472+0000","last_change":"2024-04-25T11:22:32.785439+0000","last_active":"2024-04-25T11:22:33.792472+0000","last_peered":"2024-04-25T11:22:33.792472+0000","last_clean":"2024-04-25T11:22:33.792472+0000","last_became_active":"2024-04-25T11:22:32.785101+0000","last_became_peered":"2024-04-25T11:22:32.785101+0000","last_unstale":"2024-04-25T11:22:33.792472+0000","last_undegraded":"2024-04-25T11:22:33.792472+0000","last_fullsized":"2024-04-25T11:22:33.792472+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:11.883617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187945+0000","last_change":"2024-04-25T11:22:32.790412+0000","last_active":"2024-04-25T11:22:34.187945+0000","last_peered":"2024-04-25T11:22:34.187945+0000","last_clean":"2024-04-25T11:22:34.187945+0000","last_became_active":"2024-04-25T11:22:32.787812+0000","last_became_peered":"2024-04-25T11:22:32.787812+0000","last_unstale":"2024-04-25T11:22:34.187945+0000","last_undegraded":"2024-04-25T11:22:34.187945+0000","last_fullsized":"2024-04-25T11:22:34.187945+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:21.130519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.787555+0000","last_change":"2024-04-25T11:22:32.784418+0000","last_active":"2024-04-25T11:22:33.787555+0000","last_peered":"2024-04-25T11:22:33.787555+0000","last_clean":"2024-04-25T11:22:33.787555+0000","last_became_active":"2024-04-25T11:22:32.784035+0000","last_became_peered":"2024-04-25T11:22:32.784035+0000","last_unstale":"2024-04-25T11:22:33.787555+0000","last_undegraded":"2024-04-25T11:22:33.787555+0000","last_fullsized":"2024-04-25T11:22:33.787555+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:55.711779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188218+0000","last_change":"2024-04-25T11:22:32.780733+0000","last_active":"2024-04-25T11:22:34.188218+0000","last_peered":"2024-04-25T11:22:34.188218+0000","last_clean":"2024-04-25T11:22:34.188218+0000","last_became_active":"2024-04-25T11:22:32.778331+0000","last_became_peered":"2024-04-25T11:22:32.778331+0000","last_unstale":"2024-04-25T11:22:34.188218+0000","last_undegraded":"2024-04-25T11:22:34.188218+0000","last_fullsized":"2024-04-25T11:22:34.188218+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:22.673764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.791262+0000","last_change":"2024-04-25T11:22:32.794525+0000","last_active":"2024-04-25T11:22:33.791262+0000","last_peered":"2024-04-25T11:22:33.791262+0000","last_clean":"2024-04-25T11:22:33.791262+0000","last_became_active":"2024-04-25T11:22:32.794175+0000","last_became_peered":"2024-04-25T11:22:32.794175+0000","last_unstale":"2024-04-25T11:22:33.791262+0000","last_undegraded":"2024-04-25T11:22:33.791262+0000","last_fullsized":"2024-04-25T11:22:33.791262+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:46.831944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"155'4","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793546+0000","last_change":"2024-04-25T11:22:31.770859+0000","last_active":"2024-04-25T11:22:33.793546+0000","last_peered":"2024-04-25T11:22:33.793546+0000","last_clean":"2024-04-25T11:22:33.793546+0000","last_became_active":"2024-04-25T11:22:31.770038+0000","last_became_peered":"2024-04-25T11:22:31.770038+0000","last_unstale":"2024-04-25T11:22:33.793546+0000","last_undegraded":"2024-04-25T11:22:33.793546+0000","last_fullsized":"2024-04-25T11:22:33.793546+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:55.772590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.189079+0000","last_change":"2024-04-25T11:22:32.784604+0000","last_active":"2024-04-25T11:22:34.189079+0000","last_peered":"2024-04-25T11:22:34.189079+0000","last_clean":"2024-04-25T11:22:34.189079+0000","last_became_active":"2024-04-25T11:22:32.784272+0000","last_became_peered":"2024-04-25T11:22:32.784272+0000","last_unstale":"2024-04-25T11:22:34.189079+0000","last_undegraded":"2024-04-25T11:22:34.189079+0000","last_fullsized":"2024-04-25T11:22:34.189079+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:40.496638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188056+0000","last_change":"2024-04-25T11:22:31.780719+0000","last_active":"2024-04-25T11:22:34.188056+0000","last_peered":"2024-04-25T11:22:34.188056+0000","last_clean":"2024-04-25T11:22:34.188056+0000","last_became_active":"2024-04-25T11:22:31.780404+0000","last_became_peered":"2024-04-25T11:22:31.780404+0000","last_unstale":"2024-04-25T11:22:34.188056+0000","last_undegraded":"2024-04-25T11:22:34.188056+0000","last_fullsized":"2024-04-25T11:22:34.188056+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:22.270970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187445+0000","last_change":"2024-04-25T11:22:32.800323+0000","last_active":"2024-04-25T11:22:34.187445+0000","last_peered":"2024-04-25T11:22:34.187445+0000","last_clean":"2024-04-25T11:22:34.187445+0000","last_became_active":"2024-04-25T11:22:32.800034+0000","last_became_peered":"2024-04-25T11:22:32.800034+0000","last_unstale":"2024-04-25T11:22:34.187445+0000","last_undegraded":"2024-04-25T11:22:34.187445+0000","last_fullsized":"2024-04-25T11:22:34.187445+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:43.741911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.789223+0000","last_change":"2024-04-25T11:22:31.768099+0000","last_active":"2024-04-25T11:22:33.789223+0000","last_peered":"2024-04-25T11:22:33.789223+0000","last_clean":"2024-04-25T11:22:33.789223+0000","last_became_active":"2024-04-25T11:22:31.765513+0000","last_became_peered":"2024-04-25T11:22:31.765513+0000","last_unstale":"2024-04-25T11:22:33.789223+0000","last_undegraded":"2024-04-25T11:22:33.789223+0000","last_fullsized":"2024-04-25T11:22:33.789223+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:12.425318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792887+0000","last_change":"2024-04-25T11:22:32.784199+0000","last_active":"2024-04-25T11:22:33.792887+0000","last_peered":"2024-04-25T11:22:33.792887+0000","last_clean":"2024-04-25T11:22:33.792887+0000","last_became_active":"2024-04-25T11:22:32.783881+0000","last_became_peered":"2024-04-25T11:22:32.783881+0000","last_unstale":"2024-04-25T11:22:33.792887+0000","last_undegraded":"2024-04-25T11:22:33.792887+0000","last_fullsized":"2024-04-25T11:22:33.792887+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.487512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.807351+0000","last_change":"2024-04-25T11:22:31.775341+0000","last_active":"2024-04-25T11:22:32.807351+0000","last_peered":"2024-04-25T11:22:32.807351+0000","last_clean":"2024-04-25T11:22:32.807351+0000","last_became_active":"2024-04-25T11:22:31.774938+0000","last_became_peered":"2024-04-25T11:22:31.774938+0000","last_unstale":"2024-04-25T11:22:32.807351+0000","last_undegraded":"2024-04-25T11:22:32.807351+0000","last_fullsized":"2024-04-25T11:22:32.807351+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:39.138099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968464+0000","last_change":"2024-04-25T11:22:32.788620+0000","last_active":"2024-04-25T11:22:33.968464+0000","last_peered":"2024-04-25T11:22:33.968464+0000","last_clean":"2024-04-25T11:22:33.968464+0000","last_became_active":"2024-04-25T11:22:32.784393+0000","last_became_peered":"2024-04-25T11:22:32.784393+0000","last_unstale":"2024-04-25T11:22:33.968464+0000","last_undegraded":"2024-04-25T11:22:33.968464+0000","last_fullsized":"2024-04-25T11:22:33.968464+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:58.110943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968218+0000","last_change":"2024-04-25T11:22:31.782517+0000","last_active":"2024-04-25T11:22:33.968218+0000","last_peered":"2024-04-25T11:22:33.968218+0000","last_clean":"2024-04-25T11:22:33.968218+0000","last_became_active":"2024-04-25T11:22:31.782078+0000","last_became_peered":"2024-04-25T11:22:31.782078+0000","last_unstale":"2024-04-25T11:22:33.968218+0000","last_undegraded":"2024-04-25T11:22:33.968218+0000","last_fullsized":"2024-04-25T11:22:33.968218+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:17.566853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.791536+0000","last_change":"2024-04-25T11:22:32.782529+0000","last_active":"2024-04-25T11:22:32.791536+0000","last_peered":"2024-04-25T11:22:32.791536+0000","last_clean":"2024-04-25T11:22:32.791536+0000","last_became_active":"2024-04-25T11:22:32.782147+0000","last_became_peered":"2024-04-25T11:22:32.782147+0000","last_unstale":"2024-04-25T11:22:32.791536+0000","last_undegraded":"2024-04-25T11:22:32.791536+0000","last_fullsized":"2024-04-25T11:22:32.791536+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:27.247528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187285+0000","last_change":"2024-04-25T11:22:31.779535+0000","last_active":"2024-04-25T11:22:34.187285+0000","last_peered":"2024-04-25T11:22:34.187285+0000","last_clean":"2024-04-25T11:22:34.187285+0000","last_became_active":"2024-04-25T11:22:31.779178+0000","last_became_peered":"2024-04-25T11:22:31.779178+0000","last_unstale":"2024-04-25T11:22:34.187285+0000","last_undegraded":"2024-04-25T11:22:34.187285+0000","last_fullsized":"2024-04-25T11:22:34.187285+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:25.844240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.796021+0000","last_change":"2024-04-25T11:22:32.792347+0000","last_active":"2024-04-25T11:22:32.796021+0000","last_peered":"2024-04-25T11:22:32.796021+0000","last_clean":"2024-04-25T11:22:32.796021+0000","last_became_active":"2024-04-25T11:22:32.792006+0000","last_became_peered":"2024-04-25T11:22:32.792006+0000","last_unstale":"2024-04-25T11:22:32.796021+0000","last_undegraded":"2024-04-25T11:22:32.796021+0000","last_fullsized":"2024-04-25T11:22:32.796021+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:12.171647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.749395+0000","last_change":"2024-04-25T11:22:31.776727+0000","last_active":"2024-04-25T11:22:32.749395+0000","last_peered":"2024-04-25T11:22:32.749395+0000","last_clean":"2024-04-25T11:22:32.749395+0000","last_became_active":"2024-04-25T11:22:31.774282+0000","last_became_peered":"2024-04-25T11:22:31.774282+0000","last_unstale":"2024-04-25T11:22:32.749395+0000","last_undegraded":"2024-04-25T11:22:32.749395+0000","last_fullsized":"2024-04-25T11:22:32.749395+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:06.029184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793174+0000","last_change":"2024-04-25T11:22:32.774508+0000","last_active":"2024-04-25T11:22:33.793174+0000","last_peered":"2024-04-25T11:22:33.793174+0000","last_clean":"2024-04-25T11:22:33.793174+0000","last_became_active":"2024-04-25T11:22:32.774180+0000","last_became_peered":"2024-04-25T11:22:32.774180+0000","last_unstale":"2024-04-25T11:22:33.793174+0000","last_undegraded":"2024-04-25T11:22:33.793174+0000","last_fullsized":"2024-04-25T11:22:33.793174+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:10.804639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968109+0000","last_change":"2024-04-25T11:22:31.782618+0000","last_active":"2024-04-25T11:22:33.968109+0000","last_peered":"2024-04-25T11:22:33.968109+0000","last_clean":"2024-04-25T11:22:33.968109+0000","last_became_active":"2024-04-25T11:22:31.782203+0000","last_became_peered":"2024-04-25T11:22:31.782203+0000","last_unstale":"2024-04-25T11:22:33.968109+0000","last_undegraded":"2024-04-25T11:22:33.968109+0000","last_fullsized":"2024-04-25T11:22:33.968109+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:06.304919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.790798+0000","last_change":"2024-04-25T11:22:32.794152+0000","last_active":"2024-04-25T11:22:33.790798+0000","last_peered":"2024-04-25T11:22:33.790798+0000","last_clean":"2024-04-25T11:22:33.790798+0000","last_became_active":"2024-04-25T11:22:32.793682+0000","last_became_peered":"2024-04-25T11:22:32.793682+0000","last_unstale":"2024-04-25T11:22:33.790798+0000","last_undegraded":"2024-04-25T11:22:33.790798+0000","last_fullsized":"2024-04-25T11:22:33.790798+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:15.493348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.191285+0000","last_change":"2024-04-25T11:22:31.777691+0000","last_active":"2024-04-25T11:22:34.191285+0000","last_peered":"2024-04-25T11:22:34.191285+0000","last_clean":"2024-04-25T11:22:34.191285+0000","last_became_active":"2024-04-25T11:22:31.776845+0000","last_became_peered":"2024-04-25T11:22:31.776845+0000","last_unstale":"2024-04-25T11:22:34.191285+0000","last_undegraded":"2024-04-25T11:22:34.191285+0000","last_fullsized":"2024-04-25T11:22:34.191285+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:00.857805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.189933+0000","last_change":"2024-04-25T11:22:32.797266+0000","last_active":"2024-04-25T11:22:34.189933+0000","last_peered":"2024-04-25T11:22:34.189933+0000","last_clean":"2024-04-25T11:22:34.189933+0000","last_became_active":"2024-04-25T11:22:32.796876+0000","last_became_peered":"2024-04-25T11:22:32.796876+0000","last_unstale":"2024-04-25T11:22:34.189933+0000","last_undegraded":"2024-04-25T11:22:34.189933+0000","last_fullsized":"2024-04-25T11:22:34.189933+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:01.572428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"155'3","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793949+0000","last_change":"2024-04-25T11:22:31.773621+0000","last_active":"2024-04-25T11:22:33.793949+0000","last_peered":"2024-04-25T11:22:33.793949+0000","last_clean":"2024-04-25T11:22:33.793949+0000","last_became_active":"2024-04-25T11:22:31.773309+0000","last_became_peered":"2024-04-25T11:22:31.773309+0000","last_unstale":"2024-04-25T11:22:33.793949+0000","last_undegraded":"2024-04-25T11:22:33.793949+0000","last_fullsized":"2024-04-25T11:22:33.793949+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:40.978129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.803391+0000","last_change":"2024-04-25T11:22:32.799006+0000","last_active":"2024-04-25T11:22:32.803391+0000","last_peered":"2024-04-25T11:22:32.803391+0000","last_clean":"2024-04-25T11:22:32.803391+0000","last_became_active":"2024-04-25T11:22:32.798620+0000","last_became_peered":"2024-04-25T11:22:32.798620+0000","last_unstale":"2024-04-25T11:22:32.803391+0000","last_undegraded":"2024-04-25T11:22:32.803391+0000","last_fullsized":"2024-04-25T11:22:32.803391+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:01.295347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"155'6","reported_seq":20,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190434+0000","last_change":"2024-04-25T11:22:31.769198+0000","last_active":"2024-04-25T11:22:34.190434+0000","last_peered":"2024-04-25T11:22:34.190434+0000","last_clean":"2024-04-25T11:22:34.190434+0000","last_became_active":"2024-04-25T11:22:31.768891+0000","last_became_peered":"2024-04-25T11:22:31.768891+0000","last_unstale":"2024-04-25T11:22:34.190434+0000","last_undegraded":"2024-04-25T11:22:34.190434+0000","last_fullsized":"2024-04-25T11:22:34.190434+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:42.724169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188201+0000","last_change":"2024-04-25T11:22:32.780170+0000","last_active":"2024-04-25T11:22:34.188201+0000","last_peered":"2024-04-25T11:22:34.188201+0000","last_clean":"2024-04-25T11:22:34.188201+0000","last_became_active":"2024-04-25T11:22:32.778828+0000","last_became_peered":"2024-04-25T11:22:32.778828+0000","last_unstale":"2024-04-25T11:22:34.188201+0000","last_undegraded":"2024-04-25T11:22:34.188201+0000","last_fullsized":"2024-04-25T11:22:34.188201+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:44.463619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187941+0000","last_change":"2024-04-25T11:22:31.777227+0000","last_active":"2024-04-25T11:22:34.187941+0000","last_peered":"2024-04-25T11:22:34.187941+0000","last_clean":"2024-04-25T11:22:34.187941+0000","last_became_active":"2024-04-25T11:22:31.776897+0000","last_became_peered":"2024-04-25T11:22:31.776897+0000","last_unstale":"2024-04-25T11:22:34.187941+0000","last_undegraded":"2024-04-25T11:22:34.187941+0000","last_fullsized":"2024-04-25T11:22:34.187941+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:40.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968780+0000","last_change":"2024-04-25T11:22:32.790974+0000","last_active":"2024-04-25T11:22:33.968780+0000","last_peered":"2024-04-25T11:22:33.968780+0000","last_clean":"2024-04-25T11:22:33.968780+0000","last_became_active":"2024-04-25T11:22:32.789355+0000","last_became_peered":"2024-04-25T11:22:32.789355+0000","last_unstale":"2024-04-25T11:22:33.968780+0000","last_undegraded":"2024-04-25T11:22:33.968780+0000","last_fullsized":"2024-04-25T11:22:33.968780+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:01.998081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190051+0000","last_change":"2024-04-25T11:22:31.779317+0000","last_active":"2024-04-25T11:22:34.190051+0000","last_peered":"2024-04-25T11:22:34.190051+0000","last_clean":"2024-04-25T11:22:34.190051+0000","last_became_active":"2024-04-25T11:22:31.778538+0000","last_became_peered":"2024-04-25T11:22:31.778538+0000","last_unstale":"2024-04-25T11:22:34.190051+0000","last_undegraded":"2024-04-25T11:22:34.190051+0000","last_fullsized":"2024-04-25T11:22:34.190051+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.440322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.789779+0000","last_change":"2024-04-25T11:22:32.771868+0000","last_active":"2024-04-25T11:22:33.789779+0000","last_peered":"2024-04-25T11:22:33.789779+0000","last_clean":"2024-04-25T11:22:33.789779+0000","last_became_active":"2024-04-25T11:22:32.771555+0000","last_became_peered":"2024-04-25T11:22:32.771555+0000","last_unstale":"2024-04-25T11:22:33.789779+0000","last_undegraded":"2024-04-25T11:22:33.789779+0000","last_fullsized":"2024-04-25T11:22:33.789779+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:46.566980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.787605+0000","last_change":"2024-04-25T11:22:31.777309+0000","last_active":"2024-04-25T11:22:33.787605+0000","last_peered":"2024-04-25T11:22:33.787605+0000","last_clean":"2024-04-25T11:22:33.787605+0000","last_became_active":"2024-04-25T11:22:31.776783+0000","last_became_peered":"2024-04-25T11:22:31.776783+0000","last_unstale":"2024-04-25T11:22:33.787605+0000","last_undegraded":"2024-04-25T11:22:33.787605+0000","last_fullsized":"2024-04-25T11:22:33.787605+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:14.181678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.797262+0000","last_change":"2024-04-25T11:22:32.772467+0000","last_active":"2024-04-25T11:22:32.797262+0000","last_peered":"2024-04-25T11:22:32.797262+0000","last_clean":"2024-04-25T11:22:32.797262+0000","last_became_active":"2024-04-25T11:22:32.771450+0000","last_became_peered":"2024-04-25T11:22:32.771450+0000","last_unstale":"2024-04-25T11:22:32.797262+0000","last_undegraded":"2024-04-25T11:22:32.797262+0000","last_fullsized":"2024-04-25T11:22:32.797262+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:59.893072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"155'8","reported_seq":22,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968283+0000","last_change":"2024-04-25T11:22:31.767264+0000","last_active":"2024-04-25T11:22:33.968283+0000","last_peered":"2024-04-25T11:22:33.968283+0000","last_clean":"2024-04-25T11:22:33.968283+0000","last_became_active":"2024-04-25T11:22:31.766916+0000","last_became_peered":"2024-04-25T11:22:31.766916+0000","last_unstale":"2024-04-25T11:22:33.968283+0000","last_undegraded":"2024-04-25T11:22:33.968283+0000","last_fullsized":"2024-04-25T11:22:33.968283+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:54.779895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.967988+0000","last_change":"2024-04-25T11:22:32.797205+0000","last_active":"2024-04-25T11:22:33.967988+0000","last_peered":"2024-04-25T11:22:33.967988+0000","last_clean":"2024-04-25T11:22:33.967988+0000","last_became_active":"2024-04-25T11:22:32.796865+0000","last_became_peered":"2024-04-25T11:22:32.796865+0000","last_unstale":"2024-04-25T11:22:33.967988+0000","last_undegraded":"2024-04-25T11:22:33.967988+0000","last_fullsized":"2024-04-25T11:22:33.967988+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:07.169534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.191485+0000","last_change":"2024-04-25T11:22:31.772512+0000","last_active":"2024-04-25T11:22:34.191485+0000","last_peered":"2024-04-25T11:22:34.191485+0000","last_clean":"2024-04-25T11:22:34.191485+0000","last_became_active":"2024-04-25T11:22:31.770109+0000","last_became_peered":"2024-04-25T11:22:31.770109+0000","last_unstale":"2024-04-25T11:22:34.191485+0000","last_undegraded":"2024-04-25T11:22:34.191485+0000","last_fullsized":"2024-04-25T11:22:34.191485+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:58.567458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188502+0000","last_change":"2024-04-25T11:22:32.781632+0000","last_active":"2024-04-25T11:22:34.188502+0000","last_peered":"2024-04-25T11:22:34.188502+0000","last_clean":"2024-04-25T11:22:34.188502+0000","last_became_active":"2024-04-25T11:22:32.777757+0000","last_became_peered":"2024-04-25T11:22:32.777757+0000","last_unstale":"2024-04-25T11:22:34.188502+0000","last_undegraded":"2024-04-25T11:22:34.188502+0000","last_fullsized":"2024-04-25T11:22:34.188502+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:40.870281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.969235+0000","last_change":"2024-04-25T11:22:31.774644+0000","last_active":"2024-04-25T11:22:33.969235+0000","last_peered":"2024-04-25T11:22:33.969235+0000","last_clean":"2024-04-25T11:22:33.969235+0000","last_became_active":"2024-04-25T11:22:31.774236+0000","last_became_peered":"2024-04-25T11:22:31.774236+0000","last_unstale":"2024-04-25T11:22:33.969235+0000","last_undegraded":"2024-04-25T11:22:33.969235+0000","last_fullsized":"2024-04-25T11:22:33.969235+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:23.092767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.788032+0000","last_change":"2024-04-25T11:22:32.770227+0000","last_active":"2024-04-25T11:22:33.788032+0000","last_peered":"2024-04-25T11:22:33.788032+0000","last_clean":"2024-04-25T11:22:33.788032+0000","last_became_active":"2024-04-25T11:22:32.769856+0000","last_became_peered":"2024-04-25T11:22:32.769856+0000","last_unstale":"2024-04-25T11:22:33.788032+0000","last_undegraded":"2024-04-25T11:22:33.788032+0000","last_fullsized":"2024-04-25T11:22:33.788032+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:35.016222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188090+0000","last_change":"2024-04-25T11:22:31.781779+0000","last_active":"2024-04-25T11:22:34.188090+0000","last_peered":"2024-04-25T11:22:34.188090+0000","last_clean":"2024-04-25T11:22:34.188090+0000","last_became_active":"2024-04-25T11:22:31.778701+0000","last_became_peered":"2024-04-25T11:22:31.778701+0000","last_unstale":"2024-04-25T11:22:34.188090+0000","last_undegraded":"2024-04-25T11:22:34.188090+0000","last_fullsized":"2024-04-25T11:22:34.188090+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:14.475606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.788393+0000","last_change":"2024-04-25T11:22:32.793004+0000","last_active":"2024-04-25T11:22:33.788393+0000","last_peered":"2024-04-25T11:22:33.788393+0000","last_clean":"2024-04-25T11:22:33.788393+0000","last_became_active":"2024-04-25T11:22:32.792666+0000","last_became_peered":"2024-04-25T11:22:32.792666+0000","last_unstale":"2024-04-25T11:22:33.788393+0000","last_undegraded":"2024-04-25T11:22:33.788393+0000","last_fullsized":"2024-04-25T11:22:33.788393+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:23.955995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968621+0000","last_change":"2024-04-25T11:22:31.780674+0000","last_active":"2024-04-25T11:22:33.968621+0000","last_peered":"2024-04-25T11:22:33.968621+0000","last_clean":"2024-04-25T11:22:33.968621+0000","last_became_active":"2024-04-25T11:22:31.780281+0000","last_became_peered":"2024-04-25T11:22:31.780281+0000","last_unstale":"2024-04-25T11:22:33.968621+0000","last_undegraded":"2024-04-25T11:22:33.968621+0000","last_fullsized":"2024-04-25T11:22:33.968621+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:39.418017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190317+0000","last_change":"2024-04-25T11:22:32.788188+0000","last_active":"2024-04-25T11:22:34.190317+0000","last_peered":"2024-04-25T11:22:34.190317+0000","last_clean":"2024-04-25T11:22:34.190317+0000","last_became_active":"2024-04-25T11:22:32.787025+0000","last_became_peered":"2024-04-25T11:22:32.787025+0000","last_unstale":"2024-04-25T11:22:34.190317+0000","last_undegraded":"2024-04-25T11:22:34.190317+0000","last_fullsized":"2024-04-25T11:22:34.190317+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:22.270282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187983+0000","last_change":"2024-04-25T11:22:31.781956+0000","last_active":"2024-04-25T11:22:34.187983+0000","last_peered":"2024-04-25T11:22:34.187983+0000","last_clean":"2024-04-25T11:22:34.187983+0000","last_became_active":"2024-04-25T11:22:31.781570+0000","last_became_peered":"2024-04-25T11:22:31.781570+0000","last_unstale":"2024-04-25T11:22:34.187983+0000","last_undegraded":"2024-04-25T11:22:34.187983+0000","last_fullsized":"2024-04-25T11:22:34.187983+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:40.052623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187668+0000","last_change":"2024-04-25T11:22:32.796427+0000","last_active":"2024-04-25T11:22:34.187668+0000","last_peered":"2024-04-25T11:22:34.187668+0000","last_clean":"2024-04-25T11:22:34.187668+0000","last_became_active":"2024-04-25T11:22:32.788384+0000","last_became_peered":"2024-04-25T11:22:32.788384+0000","last_unstale":"2024-04-25T11:22:34.187668+0000","last_undegraded":"2024-04-25T11:22:34.187668+0000","last_fullsized":"2024-04-25T11:22:34.187668+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:29.594612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.788244+0000","last_change":"2024-04-25T11:22:31.761532+0000","last_active":"2024-04-25T11:22:33.788244+0000","last_peered":"2024-04-25T11:22:33.788244+0000","last_clean":"2024-04-25T11:22:33.788244+0000","last_became_active":"2024-04-25T11:22:31.760905+0000","last_became_peered":"2024-04-25T11:22:31.760905+0000","last_unstale":"2024-04-25T11:22:33.788244+0000","last_undegraded":"2024-04-25T11:22:33.788244+0000","last_fullsized":"2024-04-25T11:22:33.788244+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:39.852539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188129+0000","last_change":"2024-04-25T11:22:32.795804+0000","last_active":"2024-04-25T11:22:34.188129+0000","last_peered":"2024-04-25T11:22:34.188129+0000","last_clean":"2024-04-25T11:22:34.188129+0000","last_became_active":"2024-04-25T11:22:32.795140+0000","last_became_peered":"2024-04-25T11:22:32.795140+0000","last_unstale":"2024-04-25T11:22:34.188129+0000","last_undegraded":"2024-04-25T11:22:34.188129+0000","last_fullsized":"2024-04-25T11:22:34.188129+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:01.875547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.749015+0000","last_change":"2024-04-25T11:22:31.775481+0000","last_active":"2024-04-25T11:22:32.749015+0000","last_peered":"2024-04-25T11:22:32.749015+0000","last_clean":"2024-04-25T11:22:32.749015+0000","last_became_active":"2024-04-25T11:22:31.775083+0000","last_became_peered":"2024-04-25T11:22:31.775083+0000","last_unstale":"2024-04-25T11:22:32.749015+0000","last_undegraded":"2024-04-25T11:22:32.749015+0000","last_fullsized":"2024-04-25T11:22:32.749015+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:14.950481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.791438+0000","last_change":"2024-04-25T11:22:32.771438+0000","last_active":"2024-04-25T11:22:33.791438+0000","last_peered":"2024-04-25T11:22:33.791438+0000","last_clean":"2024-04-25T11:22:33.791438+0000","last_became_active":"2024-04-25T11:22:32.771038+0000","last_became_peered":"2024-04-25T11:22:32.771038+0000","last_unstale":"2024-04-25T11:22:33.791438+0000","last_undegraded":"2024-04-25T11:22:33.791438+0000","last_fullsized":"2024-04-25T11:22:33.791438+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:01.227691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.791414+0000","last_change":"2024-04-25T11:22:31.776920+0000","last_active":"2024-04-25T11:22:33.791414+0000","last_peered":"2024-04-25T11:22:33.791414+0000","last_clean":"2024-04-25T11:22:33.791414+0000","last_became_active":"2024-04-25T11:22:31.776204+0000","last_became_peered":"2024-04-25T11:22:31.776204+0000","last_unstale":"2024-04-25T11:22:33.791414+0000","last_undegraded":"2024-04-25T11:22:33.791414+0000","last_fullsized":"2024-04-25T11:22:33.791414+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:59.111785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.787352+0000","last_change":"2024-04-25T11:22:32.784599+0000","last_active":"2024-04-25T11:22:33.787352+0000","last_peered":"2024-04-25T11:22:33.787352+0000","last_clean":"2024-04-25T11:22:33.787352+0000","last_became_active":"2024-04-25T11:22:32.783299+0000","last_became_peered":"2024-04-25T11:22:32.783299+0000","last_unstale":"2024-04-25T11:22:33.787352+0000","last_undegraded":"2024-04-25T11:22:33.787352+0000","last_fullsized":"2024-04-25T11:22:33.787352+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:24.110509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187910+0000","last_change":"2024-04-25T11:22:31.776243+0000","last_active":"2024-04-25T11:22:34.187910+0000","last_peered":"2024-04-25T11:22:34.187910+0000","last_clean":"2024-04-25T11:22:34.187910+0000","last_became_active":"2024-04-25T11:22:31.775891+0000","last_became_peered":"2024-04-25T11:22:31.775891+0000","last_unstale":"2024-04-25T11:22:34.187910+0000","last_undegraded":"2024-04-25T11:22:34.187910+0000","last_fullsized":"2024-04-25T11:22:34.187910+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:45.814203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.967964+0000","last_change":"2024-04-25T11:22:32.785339+0000","last_active":"2024-04-25T11:22:33.967964+0000","last_peered":"2024-04-25T11:22:33.967964+0000","last_clean":"2024-04-25T11:22:33.967964+0000","last_became_active":"2024-04-25T11:22:32.785023+0000","last_became_peered":"2024-04-25T11:22:32.785023+0000","last_unstale":"2024-04-25T11:22:33.967964+0000","last_undegraded":"2024-04-25T11:22:33.967964+0000","last_fullsized":"2024-04-25T11:22:33.967964+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:04.992792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.191030+0000","last_change":"2024-04-25T11:22:31.777766+0000","last_active":"2024-04-25T11:22:34.191030+0000","last_peered":"2024-04-25T11:22:34.191030+0000","last_clean":"2024-04-25T11:22:34.191030+0000","last_became_active":"2024-04-25T11:22:31.777342+0000","last_became_peered":"2024-04-25T11:22:31.777342+0000","last_unstale":"2024-04-25T11:22:34.191030+0000","last_undegraded":"2024-04-25T11:22:34.191030+0000","last_fullsized":"2024-04-25T11:22:34.191030+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:45.469770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.796539+0000","last_change":"2024-04-25T11:22:32.786492+0000","last_active":"2024-04-25T11:22:32.796539+0000","last_peered":"2024-04-25T11:22:32.796539+0000","last_clean":"2024-04-25T11:22:32.796539+0000","last_became_active":"2024-04-25T11:22:32.786147+0000","last_became_peered":"2024-04-25T11:22:32.786147+0000","last_unstale":"2024-04-25T11:22:32.796539+0000","last_undegraded":"2024-04-25T11:22:32.796539+0000","last_fullsized":"2024-04-25T11:22:32.796539+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:11.981516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190469+0000","last_change":"2024-04-25T11:22:31.775179+0000","last_active":"2024-04-25T11:22:34.190469+0000","last_peered":"2024-04-25T11:22:34.190469+0000","last_clean":"2024-04-25T11:22:34.190469+0000","last_became_active":"2024-04-25T11:22:31.774862+0000","last_became_peered":"2024-04-25T11:22:31.774862+0000","last_unstale":"2024-04-25T11:22:34.190469+0000","last_undegraded":"2024-04-25T11:22:34.190469+0000","last_fullsized":"2024-04-25T11:22:34.190469+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:10.865875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968479+0000","last_change":"2024-04-25T11:22:32.807314+0000","last_active":"2024-04-25T11:22:33.968479+0000","last_peered":"2024-04-25T11:22:33.968479+0000","last_clean":"2024-04-25T11:22:33.968479+0000","last_became_active":"2024-04-25T11:22:32.806983+0000","last_became_peered":"2024-04-25T11:22:32.806983+0000","last_unstale":"2024-04-25T11:22:33.968479+0000","last_undegraded":"2024-04-25T11:22:33.968479+0000","last_fullsized":"2024-04-25T11:22:33.968479+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:17.724076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.751452+0000","last_change":"2024-04-25T11:22:31.773240+0000","last_active":"2024-04-25T11:22:32.751452+0000","last_peered":"2024-04-25T11:22:32.751452+0000","last_clean":"2024-04-25T11:22:32.751452+0000","last_became_active":"2024-04-25T11:22:31.772119+0000","last_became_peered":"2024-04-25T11:22:31.772119+0000","last_unstale":"2024-04-25T11:22:32.751452+0000","last_undegraded":"2024-04-25T11:22:32.751452+0000","last_fullsized":"2024-04-25T11:22:32.751452+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:23.838529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.794238+0000","last_change":"2024-04-25T11:22:32.789486+0000","last_active":"2024-04-25T11:22:33.794238+0000","last_peered":"2024-04-25T11:22:33.794238+0000","last_clean":"2024-04-25T11:22:33.794238+0000","last_became_active":"2024-04-25T11:22:32.788796+0000","last_became_peered":"2024-04-25T11:22:32.788796+0000","last_unstale":"2024-04-25T11:22:33.794238+0000","last_undegraded":"2024-04-25T11:22:33.794238+0000","last_fullsized":"2024-04-25T11:22:33.794238+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:17.466718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792332+0000","last_change":"2024-04-25T11:22:31.784168+0000","last_active":"2024-04-25T11:22:33.792332+0000","last_peered":"2024-04-25T11:22:33.792332+0000","last_clean":"2024-04-25T11:22:33.792332+0000","last_became_active":"2024-04-25T11:22:31.783637+0000","last_became_peered":"2024-04-25T11:22:31.783637+0000","last_unstale":"2024-04-25T11:22:33.792332+0000","last_undegraded":"2024-04-25T11:22:33.792332+0000","last_fullsized":"2024-04-25T11:22:33.792332+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:41.635463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.794508+0000","last_change":"2024-04-25T11:22:32.785913+0000","last_active":"2024-04-25T11:22:33.794508+0000","last_peered":"2024-04-25T11:22:33.794508+0000","last_clean":"2024-04-25T11:22:33.794508+0000","last_became_active":"2024-04-25T11:22:32.785570+0000","last_became_peered":"2024-04-25T11:22:32.785570+0000","last_unstale":"2024-04-25T11:22:33.794508+0000","last_undegraded":"2024-04-25T11:22:33.794508+0000","last_fullsized":"2024-04-25T11:22:33.794508+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:10.466936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190168+0000","last_change":"2024-04-25T11:22:31.766620+0000","last_active":"2024-04-25T11:22:34.190168+0000","last_peered":"2024-04-25T11:22:34.190168+0000","last_clean":"2024-04-25T11:22:34.190168+0000","last_became_active":"2024-04-25T11:22:31.765078+0000","last_became_peered":"2024-04-25T11:22:31.765078+0000","last_unstale":"2024-04-25T11:22:34.190168+0000","last_undegraded":"2024-04-25T11:22:34.190168+0000","last_fullsized":"2024-04-25T11:22:34.190168+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:21.404508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190278+0000","last_change":"2024-04-25T11:22:32.791067+0000","last_active":"2024-04-25T11:22:34.190278+0000","last_peered":"2024-04-25T11:22:34.190278+0000","last_clean":"2024-04-25T11:22:34.190278+0000","last_became_active":"2024-04-25T11:22:32.790510+0000","last_became_peered":"2024-04-25T11:22:32.790510+0000","last_unstale":"2024-04-25T11:22:34.190278+0000","last_undegraded":"2024-04-25T11:22:34.190278+0000","last_fullsized":"2024-04-25T11:22:34.190278+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:15.933797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.787800+0000","last_change":"2024-04-25T11:22:31.763119+0000","last_active":"2024-04-25T11:22:33.787800+0000","last_peered":"2024-04-25T11:22:33.787800+0000","last_clean":"2024-04-25T11:22:33.787800+0000","last_became_active":"2024-04-25T11:22:31.762786+0000","last_became_peered":"2024-04-25T11:22:31.762786+0000","last_unstale":"2024-04-25T11:22:33.787800+0000","last_undegraded":"2024-04-25T11:22:33.787800+0000","last_fullsized":"2024-04-25T11:22:33.787800+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:09.786897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.797287+0000","last_change":"2024-04-25T11:22:32.783447+0000","last_active":"2024-04-25T11:22:32.797287+0000","last_peered":"2024-04-25T11:22:32.797287+0000","last_clean":"2024-04-25T11:22:32.797287+0000","last_became_active":"2024-04-25T11:22:32.783068+0000","last_became_peered":"2024-04-25T11:22:32.783068+0000","last_unstale":"2024-04-25T11:22:32.797287+0000","last_undegraded":"2024-04-25T11:22:32.797287+0000","last_fullsized":"2024-04-25T11:22:32.797287+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:09.632714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.186870+0000","last_change":"2024-04-25T11:22:31.781887+0000","last_active":"2024-04-25T11:22:34.186870+0000","last_peered":"2024-04-25T11:22:34.186870+0000","last_clean":"2024-04-25T11:22:34.186870+0000","last_became_active":"2024-04-25T11:22:31.781532+0000","last_became_peered":"2024-04-25T11:22:31.781532+0000","last_unstale":"2024-04-25T11:22:34.186870+0000","last_undegraded":"2024-04-25T11:22:34.186870+0000","last_fullsized":"2024-04-25T11:22:34.186870+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:00.980339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792299+0000","last_change":"2024-04-25T11:22:32.783228+0000","last_active":"2024-04-25T11:22:33.792299+0000","last_peered":"2024-04-25T11:22:33.792299+0000","last_clean":"2024-04-25T11:22:33.792299+0000","last_became_active":"2024-04-25T11:22:32.782799+0000","last_became_peered":"2024-04-25T11:22:32.782799+0000","last_unstale":"2024-04-25T11:22:33.792299+0000","last_undegraded":"2024-04-25T11:22:33.792299+0000","last_fullsized":"2024-04-25T11:22:33.792299+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:22.422250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190518+0000","last_change":"2024-04-25T11:22:31.776972+0000","last_active":"2024-04-25T11:22:34.190518+0000","last_peered":"2024-04-25T11:22:34.190518+0000","last_clean":"2024-04-25T11:22:34.190518+0000","last_became_active":"2024-04-25T11:22:31.775760+0000","last_became_peered":"2024-04-25T11:22:31.775760+0000","last_unstale":"2024-04-25T11:22:34.190518+0000","last_undegraded":"2024-04-25T11:22:34.190518+0000","last_fullsized":"2024-04-25T11:22:34.190518+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:42.522743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":382,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.791633+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:22:33.791633+0000","last_peered":"2024-04-25T11:22:33.791633+0000","last_clean":"2024-04-25T11:22:33.791633+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:22:33.791633+0000","last_undegraded":"2024-04-25T11:22:33.791633+0000","last_fullsized":"2024-04-25T11:22:33.791633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476887,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33668,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338172,"statfs":{"total":96636764160,"available":96602288128,"internally_reserved":0,"allocated":2543616,"data_stored":1495511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1949696,"data_stored":905143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":905267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1966080,"data_stored":905907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509591,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1970176,"data_stored":907359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509592,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1961984,"data_stored":906755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509591,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37832,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334008,"statfs":{"total":96636764160,"available":96598024192,"internally_reserved":0,"allocated":2551808,"data_stored":1495615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542296,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":898451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:22:37.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:22:37.706 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-04-25T11:22:37.711+0000 7ff677c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:22:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.778+0000 7f9cbec00640 1 Processor -- start 2024-04-25T11:22:37.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.778+0000 7f9cbec00640 1 -- start start 2024-04-25T11:22:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.780+0000 7f9cbec00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb80aed40 0x7f9cb815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.780+0000 7f9cbec00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9cb80b5220 0x7f9cb80ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.780+0000 7f9cbec00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb80b64b0 con 0x7f9cb80af420 2024-04-25T11:22:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.780+0000 7f9cbec00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb815e1f0 con 0x7f9cb80b5220 2024-04-25T11:22:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.780+0000 7f9cbec00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb806c610 con 0x7f9cb80aed40 2024-04-25T11:22:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.781+0000 7f9cbe200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9cb80af420 0x7f9cb80af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39992/0 (socket says 172.21.15.7:39992) 2024-04-25T11:22:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.781+0000 7f9cbe200640 1 -- 172.21.15.7:0/3522008146 learned_addr learned my addr 172.21.15.7:0/3522008146 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:37.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.781+0000 7f9cbce00640 1 --2- 172.21.15.7:0/3522008146 >> v2:172.21.15.7:3300/0 conn(0x7f9cb80b5220 0x7f9cb80ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.781+0000 7f9cbd800640 1 --2- 172.21.15.7:0/3522008146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb80aed40 0x7f9cb815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.781+0000 7f9cb7e00640 1 -- 172.21.15.7:0/3522008146 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3264204432 0 0) 0x7f9cb80b64b0 con 0x7f9cb80af420 2024-04-25T11:22:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.781+0000 7f9cb7e00640 1 -- 172.21.15.7:0/3522008146 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c9c003520 con 0x7f9cb80af420 2024-04-25T11:22:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.782+0000 7f9cbce00640 1 -- 172.21.15.7:0/3522008146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb80aed40 msgr2=0x7f9cb815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.782+0000 7f9cbce00640 1 --2- 172.21.15.7:0/3522008146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb80aed40 0x7f9cb815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.782+0000 7f9cbce00640 1 -- 172.21.15.7:0/3522008146 >> v1:172.21.15.7:6789/0 conn(0x7f9cb80af420 legacy=0x7f9cb80af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.782+0000 7f9cbce00640 1 -- 172.21.15.7:0/3522008146 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb806c8f0 con 0x7f9cb80b5220 2024-04-25T11:22:37.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.782+0000 7f9cbd800640 1 --2- 172.21.15.7:0/3522008146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb80aed40 0x7f9cb815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:22:37.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.783+0000 7f9cbce00640 1 --2- 172.21.15.7:0/3522008146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80b5220 0x7f9cb80ce4a0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009d90 tx=0x7f9ca402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e059cdfece855f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:37.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.783+0000 7f9cb7e00640 1 -- 172.21.15.7:0/3522008146 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4007740 con 0x7f9cb80b5220 2024-04-25T11:22:37.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.783+0000 7f9cb7e00640 1 -- 172.21.15.7:0/3522008146 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca40078e0 con 0x7f9cb80b5220 2024-04-25T11:22:37.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.783+0000 7f9cb7e00640 1 -- 172.21.15.7:0/3522008146 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4007cb0 con 0x7f9cb80b5220 2024-04-25T11:22:37.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.784+0000 7f9cbec00640 1 -- 172.21.15.7:0/3522008146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80b5220 msgr2=0x7f9cb80ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:37.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.784+0000 7f9cbec00640 1 --2- 172.21.15.7:0/3522008146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80b5220 0x7f9cb80ce4a0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009d90 tx=0x7f9ca402a660 comp rx=0 tx=0).stop 2024-04-25T11:22:37.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.784+0000 7f9cbec00640 1 -- 172.21.15.7:0/3522008146 shutdown_connections 2024-04-25T11:22:37.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.784+0000 7f9cbec00640 1 --2- 172.21.15.7:0/3522008146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80b5220 0x7f9cb80ce4a0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.784+0000 7f9cbec00640 1 --2- 172.21.15.7:0/3522008146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb80aed40 0x7f9cb815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.784+0000 7f9cbec00640 1 -- 172.21.15.7:0/3522008146 >> 172.21.15.7:0/3522008146 conn(0x7f9cb80b3820 msgr2=0x7f9cb80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:37.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.784+0000 7f9cbec00640 1 -- 172.21.15.7:0/3522008146 shutdown_connections 2024-04-25T11:22:37.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.784+0000 7f9cbec00640 1 -- 172.21.15.7:0/3522008146 wait complete. 2024-04-25T11:22:37.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.786+0000 7f9cbec00640 1 Processor -- start 2024-04-25T11:22:37.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.787+0000 7f9cbec00640 1 -- start start 2024-04-25T11:22:37.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.788+0000 7f9cbec00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb80aed40 0x7f9cb814eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.788+0000 7f9cbec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80af420 0x7f9cb81505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.788+0000 7f9cbec00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb80b5220 0x7f9cb8150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.788+0000 7f9cbec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb80b6110 con 0x7f9cb80af420 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.788+0000 7f9cbec00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb806ceb0 con 0x7f9cb80b5220 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.788+0000 7f9cbec00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb80b6290 con 0x7f9cb80aed40 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.788+0000 7f9cbce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80af420 0x7f9cb81505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.788+0000 7f9cbce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80af420 0x7f9cb81505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34546/0 (socket says 172.21.15.7:34546) 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.788+0000 7f9cbce00640 1 -- 172.21.15.7:0/1380025342 learned_addr learned my addr 172.21.15.7:0/1380025342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.789+0000 7f9cbd800640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb80aed40 0x7f9cb814eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.789+0000 7f9cbe200640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb80b5220 0x7f9cb8150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.789+0000 7f9cbce00640 1 -- 172.21.15.7:0/1380025342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb80aed40 msgr2=0x7f9cb814eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.789+0000 7f9cbce00640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb80aed40 0x7f9cb814eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.789+0000 7f9cbce00640 1 -- 172.21.15.7:0/1380025342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb80b5220 msgr2=0x7f9cb8150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:37.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.789+0000 7f9cbce00640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb80b5220 0x7f9cb8150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.789+0000 7f9cbce00640 1 -- 172.21.15.7:0/1380025342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb8155e30 con 0x7f9cb80af420 2024-04-25T11:22:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.789+0000 7f9cbce00640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80af420 0x7f9cb81505e0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009d90 tx=0x7f9ca4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.789+0000 7f9cb5600640 1 -- 172.21.15.7:0/1380025342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca403f070 con 0x7f9cb80af420 2024-04-25T11:22:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.789+0000 7f9cb5600640 1 -- 172.21.15.7:0/1380025342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca4031c40 con 0x7f9cb80af420 2024-04-25T11:22:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.789+0000 7f9cbec00640 1 -- 172.21.15.7:0/1380025342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb8156110 con 0x7f9cb80af420 2024-04-25T11:22:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.790+0000 7f9cb5600640 1 -- 172.21.15.7:0/1380025342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4009450 con 0x7f9cb80af420 2024-04-25T11:22:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.790+0000 7f9cbec00640 1 -- 172.21.15.7:0/1380025342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb81566d0 con 0x7f9cb80af420 2024-04-25T11:22:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.791+0000 7f9cbec00640 1 -- 172.21.15.7:0/1380025342 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb8050740 con 0x7f9cb80af420 2024-04-25T11:22:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.792+0000 7f9cb5600640 1 -- 172.21.15.7:0/1380025342 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9ca4044460 con 0x7f9cb80af420 2024-04-25T11:22:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.793+0000 7f9cb5600640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c980744b0 0x7f9c980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.793+0000 7f9cb5600640 1 -- 172.21.15.7:0/1380025342 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f9ca40b6cf0 con 0x7f9cb80af420 2024-04-25T11:22:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.793+0000 7f9cbd800640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c980744b0 0x7f9c980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:37.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.794+0000 7f9cbd800640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c980744b0 0x7f9c980769c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8006fd0 tx=0x7f9ca8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.798+0000 7f9cb5600640 1 -- 172.21.15.7:0/1380025342 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9ca4080c00 con 0x7f9cb80af420 2024-04-25T11:22:37.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:37.990+0000 7f9cbec00640 1 -- 172.21.15.7:0/1380025342 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9cb8153b90 con 0x7f9c980744b0 2024-04-25T11:22:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.008+0000 7f9cb5600640 1 -- 172.21.15.7:0/1380025342 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252459 (secure 0 0 0) 0x7f9ca800e4c0 con 0x7f9c980744b0 2024-04-25T11:22:38.007 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:38.008 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:22:38.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.013+0000 7f9cbec00640 1 -- 172.21.15.7:0/1380025342 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c980744b0 msgr2=0x7f9c980769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.013+0000 7f9cbec00640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c980744b0 0x7f9c980769c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8006fd0 tx=0x7f9ca8008040 comp rx=0 tx=0).stop 2024-04-25T11:22:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.014+0000 7f9cbec00640 1 -- 172.21.15.7:0/1380025342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80af420 msgr2=0x7f9cb81505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.014+0000 7f9cbec00640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80af420 0x7f9cb81505e0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009d90 tx=0x7f9ca4004a40 comp rx=0 tx=0).stop 2024-04-25T11:22:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.014+0000 7f9cbec00640 1 -- 172.21.15.7:0/1380025342 shutdown_connections 2024-04-25T11:22:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.014+0000 7f9cbec00640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9c980744b0 0x7f9c980769c0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.014+0000 7f9cbec00640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb80b5220 0x7f9cb8150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.014+0000 7f9cbec00640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cb80af420 0x7f9cb81505e0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.014+0000 7f9cbec00640 1 --2- 172.21.15.7:0/1380025342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb80aed40 0x7f9cb814eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.014+0000 7f9cbec00640 1 -- 172.21.15.7:0/1380025342 >> 172.21.15.7:0/1380025342 conn(0x7f9cb80b3820 msgr2=0x7f9cb80d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.014+0000 7f9cbec00640 1 -- 172.21.15.7:0/1380025342 shutdown_connections 2024-04-25T11:22:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.014+0000 7f9cbec00640 1 -- 172.21.15.7:0/1380025342 wait complete. 2024-04-25T11:22:38.035 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-04-25T11:22:36.073422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":286728,"kb_used_data":16444,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687992,"statfs":{"total":773094113280,"available":772800503808,"internally_reserved":0,"allocated":16838656,"data_stored":8420008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.339282"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792688+0000","last_change":"2024-04-25T11:22:32.780335+0000","last_active":"2024-04-25T11:22:33.792688+0000","last_peered":"2024-04-25T11:22:33.792688+0000","last_clean":"2024-04-25T11:22:33.792688+0000","last_became_active":"2024-04-25T11:22:32.779139+0000","last_became_peered":"2024-04-25T11:22:32.779139+0000","last_unstale":"2024-04-25T11:22:33.792688+0000","last_undegraded":"2024-04-25T11:22:33.792688+0000","last_fullsized":"2024-04-25T11:22:33.792688+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:40.870281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792327+0000","last_change":"2024-04-25T11:22:32.773514+0000","last_active":"2024-04-25T11:22:33.792327+0000","last_peered":"2024-04-25T11:22:33.792327+0000","last_clean":"2024-04-25T11:22:33.792327+0000","last_became_active":"2024-04-25T11:22:32.771229+0000","last_became_peered":"2024-04-25T11:22:32.771229+0000","last_unstale":"2024-04-25T11:22:33.792327+0000","last_undegraded":"2024-04-25T11:22:33.792327+0000","last_fullsized":"2024-04-25T11:22:33.792327+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:48.079910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793786+0000","last_change":"2024-04-25T11:22:32.772354+0000","last_active":"2024-04-25T11:22:33.793786+0000","last_peered":"2024-04-25T11:22:33.793786+0000","last_clean":"2024-04-25T11:22:33.793786+0000","last_became_active":"2024-04-25T11:22:32.771896+0000","last_became_peered":"2024-04-25T11:22:32.771896+0000","last_unstale":"2024-04-25T11:22:33.793786+0000","last_undegraded":"2024-04-25T11:22:33.793786+0000","last_fullsized":"2024-04-25T11:22:33.793786+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:13.443059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.794108+0000","last_change":"2024-04-25T11:22:32.786253+0000","last_active":"2024-04-25T11:22:33.794108+0000","last_peered":"2024-04-25T11:22:33.794108+0000","last_clean":"2024-04-25T11:22:33.794108+0000","last_became_active":"2024-04-25T11:22:32.785932+0000","last_became_peered":"2024-04-25T11:22:32.785932+0000","last_unstale":"2024-04-25T11:22:33.794108+0000","last_undegraded":"2024-04-25T11:22:33.794108+0000","last_fullsized":"2024-04-25T11:22:33.794108+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:37.493955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187703+0000","last_change":"2024-04-25T11:22:32.781582+0000","last_active":"2024-04-25T11:22:34.187703+0000","last_peered":"2024-04-25T11:22:34.187703+0000","last_clean":"2024-04-25T11:22:34.187703+0000","last_became_active":"2024-04-25T11:22:32.781269+0000","last_became_peered":"2024-04-25T11:22:32.781269+0000","last_unstale":"2024-04-25T11:22:34.187703+0000","last_undegraded":"2024-04-25T11:22:34.187703+0000","last_fullsized":"2024-04-25T11:22:34.187703+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:35.764125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.790984+0000","last_change":"2024-04-25T11:22:32.770279+0000","last_active":"2024-04-25T11:22:33.790984+0000","last_peered":"2024-04-25T11:22:33.790984+0000","last_clean":"2024-04-25T11:22:33.790984+0000","last_became_active":"2024-04-25T11:22:32.768696+0000","last_became_peered":"2024-04-25T11:22:32.768696+0000","last_unstale":"2024-04-25T11:22:33.790984+0000","last_undegraded":"2024-04-25T11:22:33.790984+0000","last_fullsized":"2024-04-25T11:22:33.790984+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:23.288712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.967989+0000","last_change":"2024-04-25T11:22:32.806012+0000","last_active":"2024-04-25T11:22:33.967989+0000","last_peered":"2024-04-25T11:22:33.967989+0000","last_clean":"2024-04-25T11:22:33.967989+0000","last_became_active":"2024-04-25T11:22:32.805607+0000","last_became_peered":"2024-04-25T11:22:32.805607+0000","last_unstale":"2024-04-25T11:22:33.967989+0000","last_undegraded":"2024-04-25T11:22:33.967989+0000","last_fullsized":"2024-04-25T11:22:33.967989+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:52.430209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.967965+0000","last_change":"2024-04-25T11:22:32.775451+0000","last_active":"2024-04-25T11:22:33.967965+0000","last_peered":"2024-04-25T11:22:33.967965+0000","last_clean":"2024-04-25T11:22:33.967965+0000","last_became_active":"2024-04-25T11:22:32.775143+0000","last_became_peered":"2024-04-25T11:22:32.775143+0000","last_unstale":"2024-04-25T11:22:33.967965+0000","last_undegraded":"2024-04-25T11:22:33.967965+0000","last_fullsized":"2024-04-25T11:22:33.967965+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:03.032065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.969000+0000","last_change":"2024-04-25T11:22:32.783643+0000","last_active":"2024-04-25T11:22:33.969000+0000","last_peered":"2024-04-25T11:22:33.969000+0000","last_clean":"2024-04-25T11:22:33.969000+0000","last_became_active":"2024-04-25T11:22:32.783302+0000","last_became_peered":"2024-04-25T11:22:32.783302+0000","last_unstale":"2024-04-25T11:22:33.969000+0000","last_undegraded":"2024-04-25T11:22:33.969000+0000","last_fullsized":"2024-04-25T11:22:33.969000+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:26.861982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.969142+0000","last_change":"2024-04-25T11:22:32.784537+0000","last_active":"2024-04-25T11:22:33.969142+0000","last_peered":"2024-04-25T11:22:33.969142+0000","last_clean":"2024-04-25T11:22:33.969142+0000","last_became_active":"2024-04-25T11:22:32.784188+0000","last_became_peered":"2024-04-25T11:22:32.784188+0000","last_unstale":"2024-04-25T11:22:33.969142+0000","last_undegraded":"2024-04-25T11:22:33.969142+0000","last_fullsized":"2024-04-25T11:22:33.969142+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:41.577988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.789000+0000","last_change":"2024-04-25T11:22:32.793029+0000","last_active":"2024-04-25T11:22:33.789000+0000","last_peered":"2024-04-25T11:22:33.789000+0000","last_clean":"2024-04-25T11:22:33.789000+0000","last_became_active":"2024-04-25T11:22:32.792632+0000","last_became_peered":"2024-04-25T11:22:32.792632+0000","last_unstale":"2024-04-25T11:22:33.789000+0000","last_undegraded":"2024-04-25T11:22:33.789000+0000","last_fullsized":"2024-04-25T11:22:33.789000+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:50.860031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793777+0000","last_change":"2024-04-25T11:22:32.771841+0000","last_active":"2024-04-25T11:22:33.793777+0000","last_peered":"2024-04-25T11:22:33.793777+0000","last_clean":"2024-04-25T11:22:33.793777+0000","last_became_active":"2024-04-25T11:22:32.771457+0000","last_became_peered":"2024-04-25T11:22:32.771457+0000","last_unstale":"2024-04-25T11:22:33.793777+0000","last_undegraded":"2024-04-25T11:22:33.793777+0000","last_fullsized":"2024-04-25T11:22:33.793777+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:40.458064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190690+0000","last_change":"2024-04-25T11:22:32.780633+0000","last_active":"2024-04-25T11:22:34.190690+0000","last_peered":"2024-04-25T11:22:34.190690+0000","last_clean":"2024-04-25T11:22:34.190690+0000","last_became_active":"2024-04-25T11:22:32.780324+0000","last_became_peered":"2024-04-25T11:22:32.780324+0000","last_unstale":"2024-04-25T11:22:34.190690+0000","last_undegraded":"2024-04-25T11:22:34.190690+0000","last_fullsized":"2024-04-25T11:22:34.190690+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:49.743490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187155+0000","last_change":"2024-04-25T11:22:32.806816+0000","last_active":"2024-04-25T11:22:34.187155+0000","last_peered":"2024-04-25T11:22:34.187155+0000","last_clean":"2024-04-25T11:22:34.187155+0000","last_became_active":"2024-04-25T11:22:32.804381+0000","last_became_peered":"2024-04-25T11:22:32.804381+0000","last_unstale":"2024-04-25T11:22:34.187155+0000","last_undegraded":"2024-04-25T11:22:34.187155+0000","last_fullsized":"2024-04-25T11:22:34.187155+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:10.466936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.787436+0000","last_change":"2024-04-25T11:22:32.791553+0000","last_active":"2024-04-25T11:22:33.787436+0000","last_peered":"2024-04-25T11:22:33.787436+0000","last_clean":"2024-04-25T11:22:33.787436+0000","last_became_active":"2024-04-25T11:22:32.791045+0000","last_became_peered":"2024-04-25T11:22:32.791045+0000","last_unstale":"2024-04-25T11:22:33.787436+0000","last_undegraded":"2024-04-25T11:22:33.787436+0000","last_fullsized":"2024-04-25T11:22:33.787436+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:03.389243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793008+0000","last_change":"2024-04-25T11:22:32.792648+0000","last_active":"2024-04-25T11:22:33.793008+0000","last_peered":"2024-04-25T11:22:33.793008+0000","last_clean":"2024-04-25T11:22:33.793008+0000","last_became_active":"2024-04-25T11:22:32.792031+0000","last_became_peered":"2024-04-25T11:22:32.792031+0000","last_unstale":"2024-04-25T11:22:33.793008+0000","last_undegraded":"2024-04-25T11:22:33.793008+0000","last_fullsized":"2024-04-25T11:22:33.793008+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:15.199420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.969256+0000","last_change":"2024-04-25T11:22:32.782759+0000","last_active":"2024-04-25T11:22:33.969256+0000","last_peered":"2024-04-25T11:22:33.969256+0000","last_clean":"2024-04-25T11:22:33.969256+0000","last_became_active":"2024-04-25T11:22:32.782415+0000","last_became_peered":"2024-04-25T11:22:32.782415+0000","last_unstale":"2024-04-25T11:22:33.969256+0000","last_undegraded":"2024-04-25T11:22:33.969256+0000","last_fullsized":"2024-04-25T11:22:33.969256+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:41.070365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188366+0000","last_change":"2024-04-25T11:22:32.789726+0000","last_active":"2024-04-25T11:22:34.188366+0000","last_peered":"2024-04-25T11:22:34.188366+0000","last_clean":"2024-04-25T11:22:34.188366+0000","last_became_active":"2024-04-25T11:22:32.789261+0000","last_became_peered":"2024-04-25T11:22:32.789261+0000","last_unstale":"2024-04-25T11:22:34.188366+0000","last_undegraded":"2024-04-25T11:22:34.188366+0000","last_fullsized":"2024-04-25T11:22:34.188366+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:41.300207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.789684+0000","last_change":"2024-04-25T11:22:32.791455+0000","last_active":"2024-04-25T11:22:33.789684+0000","last_peered":"2024-04-25T11:22:33.789684+0000","last_clean":"2024-04-25T11:22:33.789684+0000","last_became_active":"2024-04-25T11:22:32.790983+0000","last_became_peered":"2024-04-25T11:22:32.790983+0000","last_unstale":"2024-04-25T11:22:33.789684+0000","last_undegraded":"2024-04-25T11:22:33.789684+0000","last_fullsized":"2024-04-25T11:22:33.789684+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:49.866011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.795951+0000","last_change":"2024-04-25T11:22:32.790667+0000","last_active":"2024-04-25T11:22:32.795951+0000","last_peered":"2024-04-25T11:22:32.795951+0000","last_clean":"2024-04-25T11:22:32.795951+0000","last_became_active":"2024-04-25T11:22:32.790330+0000","last_became_peered":"2024-04-25T11:22:32.790330+0000","last_unstale":"2024-04-25T11:22:32.795951+0000","last_undegraded":"2024-04-25T11:22:32.795951+0000","last_fullsized":"2024-04-25T11:22:32.795951+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:34.936868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.191045+0000","last_change":"2024-04-25T11:22:32.788504+0000","last_active":"2024-04-25T11:22:34.191045+0000","last_peered":"2024-04-25T11:22:34.191045+0000","last_clean":"2024-04-25T11:22:34.191045+0000","last_became_active":"2024-04-25T11:22:32.786465+0000","last_became_peered":"2024-04-25T11:22:32.786465+0000","last_unstale":"2024-04-25T11:22:34.191045+0000","last_undegraded":"2024-04-25T11:22:34.191045+0000","last_fullsized":"2024-04-25T11:22:34.191045+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:42.312405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.791487+0000","last_change":"2024-04-25T11:22:32.794560+0000","last_active":"2024-04-25T11:22:33.791487+0000","last_peered":"2024-04-25T11:22:33.791487+0000","last_clean":"2024-04-25T11:22:33.791487+0000","last_became_active":"2024-04-25T11:22:32.793560+0000","last_became_peered":"2024-04-25T11:22:32.793560+0000","last_unstale":"2024-04-25T11:22:33.791487+0000","last_undegraded":"2024-04-25T11:22:33.791487+0000","last_fullsized":"2024-04-25T11:22:33.791487+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:03.180193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.186796+0000","last_change":"2024-04-25T11:22:32.795553+0000","last_active":"2024-04-25T11:22:34.186796+0000","last_peered":"2024-04-25T11:22:34.186796+0000","last_clean":"2024-04-25T11:22:34.186796+0000","last_became_active":"2024-04-25T11:22:32.789732+0000","last_became_peered":"2024-04-25T11:22:32.789732+0000","last_unstale":"2024-04-25T11:22:34.186796+0000","last_undegraded":"2024-04-25T11:22:34.186796+0000","last_fullsized":"2024-04-25T11:22:34.186796+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:43.540485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793408+0000","last_change":"2024-04-25T11:22:32.790395+0000","last_active":"2024-04-25T11:22:33.793408+0000","last_peered":"2024-04-25T11:22:33.793408+0000","last_clean":"2024-04-25T11:22:33.793408+0000","last_became_active":"2024-04-25T11:22:32.790039+0000","last_became_peered":"2024-04-25T11:22:32.790039+0000","last_unstale":"2024-04-25T11:22:33.793408+0000","last_undegraded":"2024-04-25T11:22:33.793408+0000","last_fullsized":"2024-04-25T11:22:33.793408+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:20.363721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.191373+0000","last_change":"2024-04-25T11:22:32.787404+0000","last_active":"2024-04-25T11:22:34.191373+0000","last_peered":"2024-04-25T11:22:34.191373+0000","last_clean":"2024-04-25T11:22:34.191373+0000","last_became_active":"2024-04-25T11:22:32.786969+0000","last_became_peered":"2024-04-25T11:22:32.786969+0000","last_unstale":"2024-04-25T11:22:34.191373+0000","last_undegraded":"2024-04-25T11:22:34.191373+0000","last_fullsized":"2024-04-25T11:22:34.191373+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:52.535988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.792512+0000","last_change":"2024-04-25T11:22:32.772739+0000","last_active":"2024-04-25T11:22:32.792512+0000","last_peered":"2024-04-25T11:22:32.792512+0000","last_clean":"2024-04-25T11:22:32.792512+0000","last_became_active":"2024-04-25T11:22:32.771715+0000","last_became_peered":"2024-04-25T11:22:32.771715+0000","last_unstale":"2024-04-25T11:22:32.792512+0000","last_undegraded":"2024-04-25T11:22:32.792512+0000","last_fullsized":"2024-04-25T11:22:32.792512+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:34.670303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.794271+0000","last_change":"2024-04-25T11:22:32.793033+0000","last_active":"2024-04-25T11:22:33.794271+0000","last_peered":"2024-04-25T11:22:33.794271+0000","last_clean":"2024-04-25T11:22:33.794271+0000","last_became_active":"2024-04-25T11:22:32.792648+0000","last_became_peered":"2024-04-25T11:22:32.792648+0000","last_unstale":"2024-04-25T11:22:33.794271+0000","last_undegraded":"2024-04-25T11:22:33.794271+0000","last_fullsized":"2024-04-25T11:22:33.794271+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:43.540485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.189990+0000","last_change":"2024-04-25T11:22:32.788492+0000","last_active":"2024-04-25T11:22:34.189990+0000","last_peered":"2024-04-25T11:22:34.189990+0000","last_clean":"2024-04-25T11:22:34.189990+0000","last_became_active":"2024-04-25T11:22:32.787525+0000","last_became_peered":"2024-04-25T11:22:32.787525+0000","last_unstale":"2024-04-25T11:22:34.189990+0000","last_undegraded":"2024-04-25T11:22:34.189990+0000","last_fullsized":"2024-04-25T11:22:34.189990+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:03.751789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792472+0000","last_change":"2024-04-25T11:22:32.785439+0000","last_active":"2024-04-25T11:22:33.792472+0000","last_peered":"2024-04-25T11:22:33.792472+0000","last_clean":"2024-04-25T11:22:33.792472+0000","last_became_active":"2024-04-25T11:22:32.785101+0000","last_became_peered":"2024-04-25T11:22:32.785101+0000","last_unstale":"2024-04-25T11:22:33.792472+0000","last_undegraded":"2024-04-25T11:22:33.792472+0000","last_fullsized":"2024-04-25T11:22:33.792472+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:11.883617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187945+0000","last_change":"2024-04-25T11:22:32.790412+0000","last_active":"2024-04-25T11:22:34.187945+0000","last_peered":"2024-04-25T11:22:34.187945+0000","last_clean":"2024-04-25T11:22:34.187945+0000","last_became_active":"2024-04-25T11:22:32.787812+0000","last_became_peered":"2024-04-25T11:22:32.787812+0000","last_unstale":"2024-04-25T11:22:34.187945+0000","last_undegraded":"2024-04-25T11:22:34.187945+0000","last_fullsized":"2024-04-25T11:22:34.187945+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:21.130519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.787555+0000","last_change":"2024-04-25T11:22:32.784418+0000","last_active":"2024-04-25T11:22:33.787555+0000","last_peered":"2024-04-25T11:22:33.787555+0000","last_clean":"2024-04-25T11:22:33.787555+0000","last_became_active":"2024-04-25T11:22:32.784035+0000","last_became_peered":"2024-04-25T11:22:32.784035+0000","last_unstale":"2024-04-25T11:22:33.787555+0000","last_undegraded":"2024-04-25T11:22:33.787555+0000","last_fullsized":"2024-04-25T11:22:33.787555+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:55.711779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188218+0000","last_change":"2024-04-25T11:22:32.780733+0000","last_active":"2024-04-25T11:22:34.188218+0000","last_peered":"2024-04-25T11:22:34.188218+0000","last_clean":"2024-04-25T11:22:34.188218+0000","last_became_active":"2024-04-25T11:22:32.778331+0000","last_became_peered":"2024-04-25T11:22:32.778331+0000","last_unstale":"2024-04-25T11:22:34.188218+0000","last_undegraded":"2024-04-25T11:22:34.188218+0000","last_fullsized":"2024-04-25T11:22:34.188218+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:22.673764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.791262+0000","last_change":"2024-04-25T11:22:32.794525+0000","last_active":"2024-04-25T11:22:33.791262+0000","last_peered":"2024-04-25T11:22:33.791262+0000","last_clean":"2024-04-25T11:22:33.791262+0000","last_became_active":"2024-04-25T11:22:32.794175+0000","last_became_peered":"2024-04-25T11:22:32.794175+0000","last_unstale":"2024-04-25T11:22:33.791262+0000","last_undegraded":"2024-04-25T11:22:33.791262+0000","last_fullsized":"2024-04-25T11:22:33.791262+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:46.831944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"155'4","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793546+0000","last_change":"2024-04-25T11:22:31.770859+0000","last_active":"2024-04-25T11:22:33.793546+0000","last_peered":"2024-04-25T11:22:33.793546+0000","last_clean":"2024-04-25T11:22:33.793546+0000","last_became_active":"2024-04-25T11:22:31.770038+0000","last_became_peered":"2024-04-25T11:22:31.770038+0000","last_unstale":"2024-04-25T11:22:33.793546+0000","last_undegraded":"2024-04-25T11:22:33.793546+0000","last_fullsized":"2024-04-25T11:22:33.793546+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:55.772590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.189079+0000","last_change":"2024-04-25T11:22:32.784604+0000","last_active":"2024-04-25T11:22:34.189079+0000","last_peered":"2024-04-25T11:22:34.189079+0000","last_clean":"2024-04-25T11:22:34.189079+0000","last_became_active":"2024-04-25T11:22:32.784272+0000","last_became_peered":"2024-04-25T11:22:32.784272+0000","last_unstale":"2024-04-25T11:22:34.189079+0000","last_undegraded":"2024-04-25T11:22:34.189079+0000","last_fullsized":"2024-04-25T11:22:34.189079+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:40.496638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188056+0000","last_change":"2024-04-25T11:22:31.780719+0000","last_active":"2024-04-25T11:22:34.188056+0000","last_peered":"2024-04-25T11:22:34.188056+0000","last_clean":"2024-04-25T11:22:34.188056+0000","last_became_active":"2024-04-25T11:22:31.780404+0000","last_became_peered":"2024-04-25T11:22:31.780404+0000","last_unstale":"2024-04-25T11:22:34.188056+0000","last_undegraded":"2024-04-25T11:22:34.188056+0000","last_fullsized":"2024-04-25T11:22:34.188056+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:22.270970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187445+0000","last_change":"2024-04-25T11:22:32.800323+0000","last_active":"2024-04-25T11:22:34.187445+0000","last_peered":"2024-04-25T11:22:34.187445+0000","last_clean":"2024-04-25T11:22:34.187445+0000","last_became_active":"2024-04-25T11:22:32.800034+0000","last_became_peered":"2024-04-25T11:22:32.800034+0000","last_unstale":"2024-04-25T11:22:34.187445+0000","last_undegraded":"2024-04-25T11:22:34.187445+0000","last_fullsized":"2024-04-25T11:22:34.187445+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:43.741911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.789223+0000","last_change":"2024-04-25T11:22:31.768099+0000","last_active":"2024-04-25T11:22:33.789223+0000","last_peered":"2024-04-25T11:22:33.789223+0000","last_clean":"2024-04-25T11:22:33.789223+0000","last_became_active":"2024-04-25T11:22:31.765513+0000","last_became_peered":"2024-04-25T11:22:31.765513+0000","last_unstale":"2024-04-25T11:22:33.789223+0000","last_undegraded":"2024-04-25T11:22:33.789223+0000","last_fullsized":"2024-04-25T11:22:33.789223+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:12.425318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792887+0000","last_change":"2024-04-25T11:22:32.784199+0000","last_active":"2024-04-25T11:22:33.792887+0000","last_peered":"2024-04-25T11:22:33.792887+0000","last_clean":"2024-04-25T11:22:33.792887+0000","last_became_active":"2024-04-25T11:22:32.783881+0000","last_became_peered":"2024-04-25T11:22:32.783881+0000","last_unstale":"2024-04-25T11:22:33.792887+0000","last_undegraded":"2024-04-25T11:22:33.792887+0000","last_fullsized":"2024-04-25T11:22:33.792887+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.487512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"155'2","reported_seq":14,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.807351+0000","last_change":"2024-04-25T11:22:31.775341+0000","last_active":"2024-04-25T11:22:32.807351+0000","last_peered":"2024-04-25T11:22:32.807351+0000","last_clean":"2024-04-25T11:22:32.807351+0000","last_became_active":"2024-04-25T11:22:31.774938+0000","last_became_peered":"2024-04-25T11:22:31.774938+0000","last_unstale":"2024-04-25T11:22:32.807351+0000","last_undegraded":"2024-04-25T11:22:32.807351+0000","last_fullsized":"2024-04-25T11:22:32.807351+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:39.138099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968464+0000","last_change":"2024-04-25T11:22:32.788620+0000","last_active":"2024-04-25T11:22:33.968464+0000","last_peered":"2024-04-25T11:22:33.968464+0000","last_clean":"2024-04-25T11:22:33.968464+0000","last_became_active":"2024-04-25T11:22:32.784393+0000","last_became_peered":"2024-04-25T11:22:32.784393+0000","last_unstale":"2024-04-25T11:22:33.968464+0000","last_undegraded":"2024-04-25T11:22:33.968464+0000","last_fullsized":"2024-04-25T11:22:33.968464+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:58.110943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968218+0000","last_change":"2024-04-25T11:22:31.782517+0000","last_active":"2024-04-25T11:22:33.968218+0000","last_peered":"2024-04-25T11:22:33.968218+0000","last_clean":"2024-04-25T11:22:33.968218+0000","last_became_active":"2024-04-25T11:22:31.782078+0000","last_became_peered":"2024-04-25T11:22:31.782078+0000","last_unstale":"2024-04-25T11:22:33.968218+0000","last_undegraded":"2024-04-25T11:22:33.968218+0000","last_fullsized":"2024-04-25T11:22:33.968218+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:17.566853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.791536+0000","last_change":"2024-04-25T11:22:32.782529+0000","last_active":"2024-04-25T11:22:32.791536+0000","last_peered":"2024-04-25T11:22:32.791536+0000","last_clean":"2024-04-25T11:22:32.791536+0000","last_became_active":"2024-04-25T11:22:32.782147+0000","last_became_peered":"2024-04-25T11:22:32.782147+0000","last_unstale":"2024-04-25T11:22:32.791536+0000","last_undegraded":"2024-04-25T11:22:32.791536+0000","last_fullsized":"2024-04-25T11:22:32.791536+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:27.247528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187285+0000","last_change":"2024-04-25T11:22:31.779535+0000","last_active":"2024-04-25T11:22:34.187285+0000","last_peered":"2024-04-25T11:22:34.187285+0000","last_clean":"2024-04-25T11:22:34.187285+0000","last_became_active":"2024-04-25T11:22:31.779178+0000","last_became_peered":"2024-04-25T11:22:31.779178+0000","last_unstale":"2024-04-25T11:22:34.187285+0000","last_undegraded":"2024-04-25T11:22:34.187285+0000","last_fullsized":"2024-04-25T11:22:34.187285+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:25.844240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.796021+0000","last_change":"2024-04-25T11:22:32.792347+0000","last_active":"2024-04-25T11:22:32.796021+0000","last_peered":"2024-04-25T11:22:32.796021+0000","last_clean":"2024-04-25T11:22:32.796021+0000","last_became_active":"2024-04-25T11:22:32.792006+0000","last_became_peered":"2024-04-25T11:22:32.792006+0000","last_unstale":"2024-04-25T11:22:32.796021+0000","last_undegraded":"2024-04-25T11:22:32.796021+0000","last_fullsized":"2024-04-25T11:22:32.796021+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:12.171647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.749395+0000","last_change":"2024-04-25T11:22:31.776727+0000","last_active":"2024-04-25T11:22:32.749395+0000","last_peered":"2024-04-25T11:22:32.749395+0000","last_clean":"2024-04-25T11:22:32.749395+0000","last_became_active":"2024-04-25T11:22:31.774282+0000","last_became_peered":"2024-04-25T11:22:31.774282+0000","last_unstale":"2024-04-25T11:22:32.749395+0000","last_undegraded":"2024-04-25T11:22:32.749395+0000","last_fullsized":"2024-04-25T11:22:32.749395+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:06.029184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793174+0000","last_change":"2024-04-25T11:22:32.774508+0000","last_active":"2024-04-25T11:22:33.793174+0000","last_peered":"2024-04-25T11:22:33.793174+0000","last_clean":"2024-04-25T11:22:33.793174+0000","last_became_active":"2024-04-25T11:22:32.774180+0000","last_became_peered":"2024-04-25T11:22:32.774180+0000","last_unstale":"2024-04-25T11:22:33.793174+0000","last_undegraded":"2024-04-25T11:22:33.793174+0000","last_fullsized":"2024-04-25T11:22:33.793174+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:10.804639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968109+0000","last_change":"2024-04-25T11:22:31.782618+0000","last_active":"2024-04-25T11:22:33.968109+0000","last_peered":"2024-04-25T11:22:33.968109+0000","last_clean":"2024-04-25T11:22:33.968109+0000","last_became_active":"2024-04-25T11:22:31.782203+0000","last_became_peered":"2024-04-25T11:22:31.782203+0000","last_unstale":"2024-04-25T11:22:33.968109+0000","last_undegraded":"2024-04-25T11:22:33.968109+0000","last_fullsized":"2024-04-25T11:22:33.968109+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:06.304919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.790798+0000","last_change":"2024-04-25T11:22:32.794152+0000","last_active":"2024-04-25T11:22:33.790798+0000","last_peered":"2024-04-25T11:22:33.790798+0000","last_clean":"2024-04-25T11:22:33.790798+0000","last_became_active":"2024-04-25T11:22:32.793682+0000","last_became_peered":"2024-04-25T11:22:32.793682+0000","last_unstale":"2024-04-25T11:22:33.790798+0000","last_undegraded":"2024-04-25T11:22:33.790798+0000","last_fullsized":"2024-04-25T11:22:33.790798+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:15.493348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.191285+0000","last_change":"2024-04-25T11:22:31.777691+0000","last_active":"2024-04-25T11:22:34.191285+0000","last_peered":"2024-04-25T11:22:34.191285+0000","last_clean":"2024-04-25T11:22:34.191285+0000","last_became_active":"2024-04-25T11:22:31.776845+0000","last_became_peered":"2024-04-25T11:22:31.776845+0000","last_unstale":"2024-04-25T11:22:34.191285+0000","last_undegraded":"2024-04-25T11:22:34.191285+0000","last_fullsized":"2024-04-25T11:22:34.191285+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:00.857805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.189933+0000","last_change":"2024-04-25T11:22:32.797266+0000","last_active":"2024-04-25T11:22:34.189933+0000","last_peered":"2024-04-25T11:22:34.189933+0000","last_clean":"2024-04-25T11:22:34.189933+0000","last_became_active":"2024-04-25T11:22:32.796876+0000","last_became_peered":"2024-04-25T11:22:32.796876+0000","last_unstale":"2024-04-25T11:22:34.189933+0000","last_undegraded":"2024-04-25T11:22:34.189933+0000","last_fullsized":"2024-04-25T11:22:34.189933+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:01.572428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"155'3","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.793949+0000","last_change":"2024-04-25T11:22:31.773621+0000","last_active":"2024-04-25T11:22:33.793949+0000","last_peered":"2024-04-25T11:22:33.793949+0000","last_clean":"2024-04-25T11:22:33.793949+0000","last_became_active":"2024-04-25T11:22:31.773309+0000","last_became_peered":"2024-04-25T11:22:31.773309+0000","last_unstale":"2024-04-25T11:22:33.793949+0000","last_undegraded":"2024-04-25T11:22:33.793949+0000","last_fullsized":"2024-04-25T11:22:33.793949+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:40.978129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.803391+0000","last_change":"2024-04-25T11:22:32.799006+0000","last_active":"2024-04-25T11:22:32.803391+0000","last_peered":"2024-04-25T11:22:32.803391+0000","last_clean":"2024-04-25T11:22:32.803391+0000","last_became_active":"2024-04-25T11:22:32.798620+0000","last_became_peered":"2024-04-25T11:22:32.798620+0000","last_unstale":"2024-04-25T11:22:32.803391+0000","last_undegraded":"2024-04-25T11:22:32.803391+0000","last_fullsized":"2024-04-25T11:22:32.803391+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:01.295347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"155'6","reported_seq":20,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190434+0000","last_change":"2024-04-25T11:22:31.769198+0000","last_active":"2024-04-25T11:22:34.190434+0000","last_peered":"2024-04-25T11:22:34.190434+0000","last_clean":"2024-04-25T11:22:34.190434+0000","last_became_active":"2024-04-25T11:22:31.768891+0000","last_became_peered":"2024-04-25T11:22:31.768891+0000","last_unstale":"2024-04-25T11:22:34.190434+0000","last_undegraded":"2024-04-25T11:22:34.190434+0000","last_fullsized":"2024-04-25T11:22:34.190434+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:42.724169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188201+0000","last_change":"2024-04-25T11:22:32.780170+0000","last_active":"2024-04-25T11:22:34.188201+0000","last_peered":"2024-04-25T11:22:34.188201+0000","last_clean":"2024-04-25T11:22:34.188201+0000","last_became_active":"2024-04-25T11:22:32.778828+0000","last_became_peered":"2024-04-25T11:22:32.778828+0000","last_unstale":"2024-04-25T11:22:34.188201+0000","last_undegraded":"2024-04-25T11:22:34.188201+0000","last_fullsized":"2024-04-25T11:22:34.188201+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:44.463619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187941+0000","last_change":"2024-04-25T11:22:31.777227+0000","last_active":"2024-04-25T11:22:34.187941+0000","last_peered":"2024-04-25T11:22:34.187941+0000","last_clean":"2024-04-25T11:22:34.187941+0000","last_became_active":"2024-04-25T11:22:31.776897+0000","last_became_peered":"2024-04-25T11:22:31.776897+0000","last_unstale":"2024-04-25T11:22:34.187941+0000","last_undegraded":"2024-04-25T11:22:34.187941+0000","last_fullsized":"2024-04-25T11:22:34.187941+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:40.560246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968780+0000","last_change":"2024-04-25T11:22:32.790974+0000","last_active":"2024-04-25T11:22:33.968780+0000","last_peered":"2024-04-25T11:22:33.968780+0000","last_clean":"2024-04-25T11:22:33.968780+0000","last_became_active":"2024-04-25T11:22:32.789355+0000","last_became_peered":"2024-04-25T11:22:32.789355+0000","last_unstale":"2024-04-25T11:22:33.968780+0000","last_undegraded":"2024-04-25T11:22:33.968780+0000","last_fullsized":"2024-04-25T11:22:33.968780+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:01.998081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190051+0000","last_change":"2024-04-25T11:22:31.779317+0000","last_active":"2024-04-25T11:22:34.190051+0000","last_peered":"2024-04-25T11:22:34.190051+0000","last_clean":"2024-04-25T11:22:34.190051+0000","last_became_active":"2024-04-25T11:22:31.778538+0000","last_became_peered":"2024-04-25T11:22:31.778538+0000","last_unstale":"2024-04-25T11:22:34.190051+0000","last_undegraded":"2024-04-25T11:22:34.190051+0000","last_fullsized":"2024-04-25T11:22:34.190051+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:39.440322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.789779+0000","last_change":"2024-04-25T11:22:32.771868+0000","last_active":"2024-04-25T11:22:33.789779+0000","last_peered":"2024-04-25T11:22:33.789779+0000","last_clean":"2024-04-25T11:22:33.789779+0000","last_became_active":"2024-04-25T11:22:32.771555+0000","last_became_peered":"2024-04-25T11:22:32.771555+0000","last_unstale":"2024-04-25T11:22:33.789779+0000","last_undegraded":"2024-04-25T11:22:33.789779+0000","last_fullsized":"2024-04-25T11:22:33.789779+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:46.566980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.787605+0000","last_change":"2024-04-25T11:22:31.777309+0000","last_active":"2024-04-25T11:22:33.787605+0000","last_peered":"2024-04-25T11:22:33.787605+0000","last_clean":"2024-04-25T11:22:33.787605+0000","last_became_active":"2024-04-25T11:22:31.776783+0000","last_became_peered":"2024-04-25T11:22:31.776783+0000","last_unstale":"2024-04-25T11:22:33.787605+0000","last_undegraded":"2024-04-25T11:22:33.787605+0000","last_fullsized":"2024-04-25T11:22:33.787605+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:14.181678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.797262+0000","last_change":"2024-04-25T11:22:32.772467+0000","last_active":"2024-04-25T11:22:32.797262+0000","last_peered":"2024-04-25T11:22:32.797262+0000","last_clean":"2024-04-25T11:22:32.797262+0000","last_became_active":"2024-04-25T11:22:32.771450+0000","last_became_peered":"2024-04-25T11:22:32.771450+0000","last_unstale":"2024-04-25T11:22:32.797262+0000","last_undegraded":"2024-04-25T11:22:32.797262+0000","last_fullsized":"2024-04-25T11:22:32.797262+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:59.893072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"155'8","reported_seq":22,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968283+0000","last_change":"2024-04-25T11:22:31.767264+0000","last_active":"2024-04-25T11:22:33.968283+0000","last_peered":"2024-04-25T11:22:33.968283+0000","last_clean":"2024-04-25T11:22:33.968283+0000","last_became_active":"2024-04-25T11:22:31.766916+0000","last_became_peered":"2024-04-25T11:22:31.766916+0000","last_unstale":"2024-04-25T11:22:33.968283+0000","last_undegraded":"2024-04-25T11:22:33.968283+0000","last_fullsized":"2024-04-25T11:22:33.968283+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:54.779895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.967988+0000","last_change":"2024-04-25T11:22:32.797205+0000","last_active":"2024-04-25T11:22:33.967988+0000","last_peered":"2024-04-25T11:22:33.967988+0000","last_clean":"2024-04-25T11:22:33.967988+0000","last_became_active":"2024-04-25T11:22:32.796865+0000","last_became_peered":"2024-04-25T11:22:32.796865+0000","last_unstale":"2024-04-25T11:22:33.967988+0000","last_undegraded":"2024-04-25T11:22:33.967988+0000","last_fullsized":"2024-04-25T11:22:33.967988+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:07.169534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.191485+0000","last_change":"2024-04-25T11:22:31.772512+0000","last_active":"2024-04-25T11:22:34.191485+0000","last_peered":"2024-04-25T11:22:34.191485+0000","last_clean":"2024-04-25T11:22:34.191485+0000","last_became_active":"2024-04-25T11:22:31.770109+0000","last_became_peered":"2024-04-25T11:22:31.770109+0000","last_unstale":"2024-04-25T11:22:34.191485+0000","last_undegraded":"2024-04-25T11:22:34.191485+0000","last_fullsized":"2024-04-25T11:22:34.191485+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:58.567458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188502+0000","last_change":"2024-04-25T11:22:32.781632+0000","last_active":"2024-04-25T11:22:34.188502+0000","last_peered":"2024-04-25T11:22:34.188502+0000","last_clean":"2024-04-25T11:22:34.188502+0000","last_became_active":"2024-04-25T11:22:32.777757+0000","last_became_peered":"2024-04-25T11:22:32.777757+0000","last_unstale":"2024-04-25T11:22:34.188502+0000","last_undegraded":"2024-04-25T11:22:34.188502+0000","last_fullsized":"2024-04-25T11:22:34.188502+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:40.870281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.969235+0000","last_change":"2024-04-25T11:22:31.774644+0000","last_active":"2024-04-25T11:22:33.969235+0000","last_peered":"2024-04-25T11:22:33.969235+0000","last_clean":"2024-04-25T11:22:33.969235+0000","last_became_active":"2024-04-25T11:22:31.774236+0000","last_became_peered":"2024-04-25T11:22:31.774236+0000","last_unstale":"2024-04-25T11:22:33.969235+0000","last_undegraded":"2024-04-25T11:22:33.969235+0000","last_fullsized":"2024-04-25T11:22:33.969235+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:23.092767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.788032+0000","last_change":"2024-04-25T11:22:32.770227+0000","last_active":"2024-04-25T11:22:33.788032+0000","last_peered":"2024-04-25T11:22:33.788032+0000","last_clean":"2024-04-25T11:22:33.788032+0000","last_became_active":"2024-04-25T11:22:32.769856+0000","last_became_peered":"2024-04-25T11:22:32.769856+0000","last_unstale":"2024-04-25T11:22:33.788032+0000","last_undegraded":"2024-04-25T11:22:33.788032+0000","last_fullsized":"2024-04-25T11:22:33.788032+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:35.016222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188090+0000","last_change":"2024-04-25T11:22:31.781779+0000","last_active":"2024-04-25T11:22:34.188090+0000","last_peered":"2024-04-25T11:22:34.188090+0000","last_clean":"2024-04-25T11:22:34.188090+0000","last_became_active":"2024-04-25T11:22:31.778701+0000","last_became_peered":"2024-04-25T11:22:31.778701+0000","last_unstale":"2024-04-25T11:22:34.188090+0000","last_undegraded":"2024-04-25T11:22:34.188090+0000","last_fullsized":"2024-04-25T11:22:34.188090+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:14.475606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.788393+0000","last_change":"2024-04-25T11:22:32.793004+0000","last_active":"2024-04-25T11:22:33.788393+0000","last_peered":"2024-04-25T11:22:33.788393+0000","last_clean":"2024-04-25T11:22:33.788393+0000","last_became_active":"2024-04-25T11:22:32.792666+0000","last_became_peered":"2024-04-25T11:22:32.792666+0000","last_unstale":"2024-04-25T11:22:33.788393+0000","last_undegraded":"2024-04-25T11:22:33.788393+0000","last_fullsized":"2024-04-25T11:22:33.788393+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:23.955995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968621+0000","last_change":"2024-04-25T11:22:31.780674+0000","last_active":"2024-04-25T11:22:33.968621+0000","last_peered":"2024-04-25T11:22:33.968621+0000","last_clean":"2024-04-25T11:22:33.968621+0000","last_became_active":"2024-04-25T11:22:31.780281+0000","last_became_peered":"2024-04-25T11:22:31.780281+0000","last_unstale":"2024-04-25T11:22:33.968621+0000","last_undegraded":"2024-04-25T11:22:33.968621+0000","last_fullsized":"2024-04-25T11:22:33.968621+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:39.418017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190317+0000","last_change":"2024-04-25T11:22:32.788188+0000","last_active":"2024-04-25T11:22:34.190317+0000","last_peered":"2024-04-25T11:22:34.190317+0000","last_clean":"2024-04-25T11:22:34.190317+0000","last_became_active":"2024-04-25T11:22:32.787025+0000","last_became_peered":"2024-04-25T11:22:32.787025+0000","last_unstale":"2024-04-25T11:22:34.190317+0000","last_undegraded":"2024-04-25T11:22:34.190317+0000","last_fullsized":"2024-04-25T11:22:34.190317+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:22.270282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187983+0000","last_change":"2024-04-25T11:22:31.781956+0000","last_active":"2024-04-25T11:22:34.187983+0000","last_peered":"2024-04-25T11:22:34.187983+0000","last_clean":"2024-04-25T11:22:34.187983+0000","last_became_active":"2024-04-25T11:22:31.781570+0000","last_became_peered":"2024-04-25T11:22:31.781570+0000","last_unstale":"2024-04-25T11:22:34.187983+0000","last_undegraded":"2024-04-25T11:22:34.187983+0000","last_fullsized":"2024-04-25T11:22:34.187983+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:40.052623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187668+0000","last_change":"2024-04-25T11:22:32.796427+0000","last_active":"2024-04-25T11:22:34.187668+0000","last_peered":"2024-04-25T11:22:34.187668+0000","last_clean":"2024-04-25T11:22:34.187668+0000","last_became_active":"2024-04-25T11:22:32.788384+0000","last_became_peered":"2024-04-25T11:22:32.788384+0000","last_unstale":"2024-04-25T11:22:34.187668+0000","last_undegraded":"2024-04-25T11:22:34.187668+0000","last_fullsized":"2024-04-25T11:22:34.187668+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:29.594612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.788244+0000","last_change":"2024-04-25T11:22:31.761532+0000","last_active":"2024-04-25T11:22:33.788244+0000","last_peered":"2024-04-25T11:22:33.788244+0000","last_clean":"2024-04-25T11:22:33.788244+0000","last_became_active":"2024-04-25T11:22:31.760905+0000","last_became_peered":"2024-04-25T11:22:31.760905+0000","last_unstale":"2024-04-25T11:22:33.788244+0000","last_undegraded":"2024-04-25T11:22:33.788244+0000","last_fullsized":"2024-04-25T11:22:33.788244+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:39.852539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.188129+0000","last_change":"2024-04-25T11:22:32.795804+0000","last_active":"2024-04-25T11:22:34.188129+0000","last_peered":"2024-04-25T11:22:34.188129+0000","last_clean":"2024-04-25T11:22:34.188129+0000","last_became_active":"2024-04-25T11:22:32.795140+0000","last_became_peered":"2024-04-25T11:22:32.795140+0000","last_unstale":"2024-04-25T11:22:34.188129+0000","last_undegraded":"2024-04-25T11:22:34.188129+0000","last_fullsized":"2024-04-25T11:22:34.188129+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:01.875547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.749015+0000","last_change":"2024-04-25T11:22:31.775481+0000","last_active":"2024-04-25T11:22:32.749015+0000","last_peered":"2024-04-25T11:22:32.749015+0000","last_clean":"2024-04-25T11:22:32.749015+0000","last_became_active":"2024-04-25T11:22:31.775083+0000","last_became_peered":"2024-04-25T11:22:31.775083+0000","last_unstale":"2024-04-25T11:22:32.749015+0000","last_undegraded":"2024-04-25T11:22:32.749015+0000","last_fullsized":"2024-04-25T11:22:32.749015+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:14.950481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.791438+0000","last_change":"2024-04-25T11:22:32.771438+0000","last_active":"2024-04-25T11:22:33.791438+0000","last_peered":"2024-04-25T11:22:33.791438+0000","last_clean":"2024-04-25T11:22:33.791438+0000","last_became_active":"2024-04-25T11:22:32.771038+0000","last_became_peered":"2024-04-25T11:22:32.771038+0000","last_unstale":"2024-04-25T11:22:33.791438+0000","last_undegraded":"2024-04-25T11:22:33.791438+0000","last_fullsized":"2024-04-25T11:22:33.791438+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:01.227691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.791414+0000","last_change":"2024-04-25T11:22:31.776920+0000","last_active":"2024-04-25T11:22:33.791414+0000","last_peered":"2024-04-25T11:22:33.791414+0000","last_clean":"2024-04-25T11:22:33.791414+0000","last_became_active":"2024-04-25T11:22:31.776204+0000","last_became_peered":"2024-04-25T11:22:31.776204+0000","last_unstale":"2024-04-25T11:22:33.791414+0000","last_undegraded":"2024-04-25T11:22:33.791414+0000","last_fullsized":"2024-04-25T11:22:33.791414+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:59.111785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.787352+0000","last_change":"2024-04-25T11:22:32.784599+0000","last_active":"2024-04-25T11:22:33.787352+0000","last_peered":"2024-04-25T11:22:33.787352+0000","last_clean":"2024-04-25T11:22:33.787352+0000","last_became_active":"2024-04-25T11:22:32.783299+0000","last_became_peered":"2024-04-25T11:22:32.783299+0000","last_unstale":"2024-04-25T11:22:33.787352+0000","last_undegraded":"2024-04-25T11:22:33.787352+0000","last_fullsized":"2024-04-25T11:22:33.787352+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:24.110509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"155'2","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.187910+0000","last_change":"2024-04-25T11:22:31.776243+0000","last_active":"2024-04-25T11:22:34.187910+0000","last_peered":"2024-04-25T11:22:34.187910+0000","last_clean":"2024-04-25T11:22:34.187910+0000","last_became_active":"2024-04-25T11:22:31.775891+0000","last_became_peered":"2024-04-25T11:22:31.775891+0000","last_unstale":"2024-04-25T11:22:34.187910+0000","last_undegraded":"2024-04-25T11:22:34.187910+0000","last_fullsized":"2024-04-25T11:22:34.187910+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:45.814203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.967964+0000","last_change":"2024-04-25T11:22:32.785339+0000","last_active":"2024-04-25T11:22:33.967964+0000","last_peered":"2024-04-25T11:22:33.967964+0000","last_clean":"2024-04-25T11:22:33.967964+0000","last_became_active":"2024-04-25T11:22:32.785023+0000","last_became_peered":"2024-04-25T11:22:32.785023+0000","last_unstale":"2024-04-25T11:22:33.967964+0000","last_undegraded":"2024-04-25T11:22:33.967964+0000","last_fullsized":"2024-04-25T11:22:33.967964+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:04.992792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"155'1","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.191030+0000","last_change":"2024-04-25T11:22:31.777766+0000","last_active":"2024-04-25T11:22:34.191030+0000","last_peered":"2024-04-25T11:22:34.191030+0000","last_clean":"2024-04-25T11:22:34.191030+0000","last_became_active":"2024-04-25T11:22:31.777342+0000","last_became_peered":"2024-04-25T11:22:31.777342+0000","last_unstale":"2024-04-25T11:22:34.191030+0000","last_undegraded":"2024-04-25T11:22:34.191030+0000","last_fullsized":"2024-04-25T11:22:34.191030+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:45.469770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.796539+0000","last_change":"2024-04-25T11:22:32.786492+0000","last_active":"2024-04-25T11:22:32.796539+0000","last_peered":"2024-04-25T11:22:32.796539+0000","last_clean":"2024-04-25T11:22:32.796539+0000","last_became_active":"2024-04-25T11:22:32.786147+0000","last_became_peered":"2024-04-25T11:22:32.786147+0000","last_unstale":"2024-04-25T11:22:32.796539+0000","last_undegraded":"2024-04-25T11:22:32.796539+0000","last_fullsized":"2024-04-25T11:22:32.796539+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:11.981516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190469+0000","last_change":"2024-04-25T11:22:31.775179+0000","last_active":"2024-04-25T11:22:34.190469+0000","last_peered":"2024-04-25T11:22:34.190469+0000","last_clean":"2024-04-25T11:22:34.190469+0000","last_became_active":"2024-04-25T11:22:31.774862+0000","last_became_peered":"2024-04-25T11:22:31.774862+0000","last_unstale":"2024-04-25T11:22:34.190469+0000","last_undegraded":"2024-04-25T11:22:34.190469+0000","last_fullsized":"2024-04-25T11:22:34.190469+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:10.865875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.968479+0000","last_change":"2024-04-25T11:22:32.807314+0000","last_active":"2024-04-25T11:22:33.968479+0000","last_peered":"2024-04-25T11:22:33.968479+0000","last_clean":"2024-04-25T11:22:33.968479+0000","last_became_active":"2024-04-25T11:22:32.806983+0000","last_became_peered":"2024-04-25T11:22:32.806983+0000","last_unstale":"2024-04-25T11:22:33.968479+0000","last_undegraded":"2024-04-25T11:22:33.968479+0000","last_fullsized":"2024-04-25T11:22:33.968479+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:17.724076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.751452+0000","last_change":"2024-04-25T11:22:31.773240+0000","last_active":"2024-04-25T11:22:32.751452+0000","last_peered":"2024-04-25T11:22:32.751452+0000","last_clean":"2024-04-25T11:22:32.751452+0000","last_became_active":"2024-04-25T11:22:31.772119+0000","last_became_peered":"2024-04-25T11:22:31.772119+0000","last_unstale":"2024-04-25T11:22:32.751452+0000","last_undegraded":"2024-04-25T11:22:32.751452+0000","last_fullsized":"2024-04-25T11:22:32.751452+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:23.838529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.794238+0000","last_change":"2024-04-25T11:22:32.789486+0000","last_active":"2024-04-25T11:22:33.794238+0000","last_peered":"2024-04-25T11:22:33.794238+0000","last_clean":"2024-04-25T11:22:33.794238+0000","last_became_active":"2024-04-25T11:22:32.788796+0000","last_became_peered":"2024-04-25T11:22:32.788796+0000","last_unstale":"2024-04-25T11:22:33.794238+0000","last_undegraded":"2024-04-25T11:22:33.794238+0000","last_fullsized":"2024-04-25T11:22:33.794238+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:17.466718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792332+0000","last_change":"2024-04-25T11:22:31.784168+0000","last_active":"2024-04-25T11:22:33.792332+0000","last_peered":"2024-04-25T11:22:33.792332+0000","last_clean":"2024-04-25T11:22:33.792332+0000","last_became_active":"2024-04-25T11:22:31.783637+0000","last_became_peered":"2024-04-25T11:22:31.783637+0000","last_unstale":"2024-04-25T11:22:33.792332+0000","last_undegraded":"2024-04-25T11:22:33.792332+0000","last_fullsized":"2024-04-25T11:22:33.792332+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:41.635463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.794508+0000","last_change":"2024-04-25T11:22:32.785913+0000","last_active":"2024-04-25T11:22:33.794508+0000","last_peered":"2024-04-25T11:22:33.794508+0000","last_clean":"2024-04-25T11:22:33.794508+0000","last_became_active":"2024-04-25T11:22:32.785570+0000","last_became_peered":"2024-04-25T11:22:32.785570+0000","last_unstale":"2024-04-25T11:22:33.794508+0000","last_undegraded":"2024-04-25T11:22:33.794508+0000","last_fullsized":"2024-04-25T11:22:33.794508+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:10.466936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190168+0000","last_change":"2024-04-25T11:22:31.766620+0000","last_active":"2024-04-25T11:22:34.190168+0000","last_peered":"2024-04-25T11:22:34.190168+0000","last_clean":"2024-04-25T11:22:34.190168+0000","last_became_active":"2024-04-25T11:22:31.765078+0000","last_became_peered":"2024-04-25T11:22:31.765078+0000","last_unstale":"2024-04-25T11:22:34.190168+0000","last_undegraded":"2024-04-25T11:22:34.190168+0000","last_fullsized":"2024-04-25T11:22:34.190168+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:21.404508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190278+0000","last_change":"2024-04-25T11:22:32.791067+0000","last_active":"2024-04-25T11:22:34.190278+0000","last_peered":"2024-04-25T11:22:34.190278+0000","last_clean":"2024-04-25T11:22:34.190278+0000","last_became_active":"2024-04-25T11:22:32.790510+0000","last_became_peered":"2024-04-25T11:22:32.790510+0000","last_unstale":"2024-04-25T11:22:34.190278+0000","last_undegraded":"2024-04-25T11:22:34.190278+0000","last_fullsized":"2024-04-25T11:22:34.190278+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:15.933797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.787800+0000","last_change":"2024-04-25T11:22:31.763119+0000","last_active":"2024-04-25T11:22:33.787800+0000","last_peered":"2024-04-25T11:22:33.787800+0000","last_clean":"2024-04-25T11:22:33.787800+0000","last_became_active":"2024-04-25T11:22:31.762786+0000","last_became_peered":"2024-04-25T11:22:31.762786+0000","last_unstale":"2024-04-25T11:22:33.787800+0000","last_undegraded":"2024-04-25T11:22:33.787800+0000","last_fullsized":"2024-04-25T11:22:33.787800+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:09.786897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-04-25T11:22:32.797287+0000","last_change":"2024-04-25T11:22:32.783447+0000","last_active":"2024-04-25T11:22:32.797287+0000","last_peered":"2024-04-25T11:22:32.797287+0000","last_clean":"2024-04-25T11:22:32.797287+0000","last_became_active":"2024-04-25T11:22:32.783068+0000","last_became_peered":"2024-04-25T11:22:32.783068+0000","last_unstale":"2024-04-25T11:22:32.797287+0000","last_undegraded":"2024-04-25T11:22:32.797287+0000","last_fullsized":"2024-04-25T11:22:32.797287+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:09.632714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.186870+0000","last_change":"2024-04-25T11:22:31.781887+0000","last_active":"2024-04-25T11:22:34.186870+0000","last_peered":"2024-04-25T11:22:34.186870+0000","last_clean":"2024-04-25T11:22:34.186870+0000","last_became_active":"2024-04-25T11:22:31.781532+0000","last_became_peered":"2024-04-25T11:22:31.781532+0000","last_unstale":"2024-04-25T11:22:34.186870+0000","last_undegraded":"2024-04-25T11:22:34.186870+0000","last_fullsized":"2024-04-25T11:22:34.186870+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:00.980339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.792299+0000","last_change":"2024-04-25T11:22:32.783228+0000","last_active":"2024-04-25T11:22:33.792299+0000","last_peered":"2024-04-25T11:22:33.792299+0000","last_clean":"2024-04-25T11:22:33.792299+0000","last_became_active":"2024-04-25T11:22:32.782799+0000","last_became_peered":"2024-04-25T11:22:32.782799+0000","last_unstale":"2024-04-25T11:22:33.792299+0000","last_undegraded":"2024-04-25T11:22:33.792299+0000","last_fullsized":"2024-04-25T11:22:33.792299+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:31.732355+0000","last_clean_scrub_stamp":"2024-04-25T11:22:31.732355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:22.422250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:34.190518+0000","last_change":"2024-04-25T11:22:31.776972+0000","last_active":"2024-04-25T11:22:34.190518+0000","last_peered":"2024-04-25T11:22:34.190518+0000","last_clean":"2024-04-25T11:22:34.190518+0000","last_became_active":"2024-04-25T11:22:31.775760+0000","last_became_peered":"2024-04-25T11:22:31.775760+0000","last_unstale":"2024-04-25T11:22:34.190518+0000","last_undegraded":"2024-04-25T11:22:34.190518+0000","last_fullsized":"2024-04-25T11:22:34.190518+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:22:30.714613+0000","last_clean_scrub_stamp":"2024-04-25T11:22:30.714613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:42.522743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":382,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-04-25T11:22:33.791633+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:22:33.791633+0000","last_peered":"2024-04-25T11:22:33.791633+0000","last_clean":"2024-04-25T11:22:33.791633+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:22:33.791633+0000","last_undegraded":"2024-04-25T11:22:33.791633+0000","last_fullsized":"2024-04-25T11:22:33.791633+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476887,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33668,"kb_used_data":2484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338172,"statfs":{"total":96636764160,"available":96602288128,"internally_reserved":0,"allocated":2543616,"data_stored":1495511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1949696,"data_stored":905143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476887,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":905267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1966080,"data_stored":905907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509591,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37272,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1970176,"data_stored":907359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509592,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1961984,"data_stored":906755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509591,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37832,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334008,"statfs":{"total":96636764160,"available":96598024192,"internally_reserved":0,"allocated":2551808,"data_stored":1495615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542296,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":898451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:22:38.038 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T11:22:38.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T11:22:38.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.372+0000 7f7ccbe00640 1 Processor -- start 2024-04-25T11:22:38.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.372+0000 7f7ccbe00640 1 -- start start 2024-04-25T11:22:38.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.375+0000 7f7ccbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ccc1583c0 0x7f7ccc0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:38.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.375+0000 7f7ccbe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7ccc0cfb50 0x7f7ccc0d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:38.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.375+0000 7f7ccbe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ccc0ce570 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.375+0000 7f7ccbe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7ccc15f190 con 0x7f7ccc0cfb50 2024-04-25T11:22:38.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.375+0000 7f7ccbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7ccc0ce850 con 0x7f7ccc1583c0 2024-04-25T11:22:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.376+0000 7f7cc3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7ccc0cfb50 0x7f7ccc0d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.376+0000 7f7cc3e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7ccc0cfb50 0x7f7ccc0d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:42140/0 (socket says 172.21.15.7:42140) 2024-04-25T11:22:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.376+0000 7f7cc3e00640 1 -- 172.21.15.7:0/3997786618 learned_addr learned my addr 172.21.15.7:0/3997786618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.376+0000 7f7ccaa00640 1 --2- 172.21.15.7:0/3997786618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ccc1583c0 0x7f7ccc0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.377+0000 7f7cc3e00640 1 -- 172.21.15.7:0/3997786618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ccc1583c0 msgr2=0x7f7ccc0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:38.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.377+0000 7f7cc3e00640 1 --2- 172.21.15.7:0/3997786618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ccc1583c0 0x7f7ccc0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.377+0000 7f7cc3e00640 1 -- 172.21.15.7:0/3997786618 >> v1:172.21.15.87:6789/0 conn(0x7f7ccc0d09c0 legacy=0x7f7ccc06eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.377+0000 7f7cc3e00640 1 -- 172.21.15.7:0/3997786618 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ccc0ceb30 con 0x7f7ccc0cfb50 2024-04-25T11:22:38.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.378+0000 7f7cc3e00640 1 --2- 172.21.15.7:0/3997786618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ccc0cfb50 0x7f7ccc0d0480 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009870 tx=0x7f7cb402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e724ac9414ed5e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.378+0000 7f7cca000640 1 -- 172.21.15.7:0/3997786618 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4037070 con 0x7f7ccc0cfb50 2024-04-25T11:22:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.379+0000 7f7cca000640 1 -- 172.21.15.7:0/3997786618 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb4032440 con 0x7f7ccc0cfb50 2024-04-25T11:22:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.379+0000 7f7cca000640 1 -- 172.21.15.7:0/3997786618 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4032810 con 0x7f7ccc0cfb50 2024-04-25T11:22:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.379+0000 7f7ccbe00640 1 -- 172.21.15.7:0/3997786618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ccc0cfb50 msgr2=0x7f7ccc0d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.379+0000 7f7ccbe00640 1 --2- 172.21.15.7:0/3997786618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ccc0cfb50 0x7f7ccc0d0480 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4009870 tx=0x7f7cb402a270 comp rx=0 tx=0).stop 2024-04-25T11:22:38.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.380+0000 7f7ccbe00640 1 -- 172.21.15.7:0/3997786618 shutdown_connections 2024-04-25T11:22:38.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.380+0000 7f7ccbe00640 1 --2- 172.21.15.7:0/3997786618 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ccc0cfb50 0x7f7ccc0d0480 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.380+0000 7f7ccbe00640 1 --2- 172.21.15.7:0/3997786618 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ccc1583c0 0x7f7ccc0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.380+0000 7f7ccbe00640 1 -- 172.21.15.7:0/3997786618 >> 172.21.15.7:0/3997786618 conn(0x7f7ccc0740f0 msgr2=0x7f7ccc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:38.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.380+0000 7f7ccbe00640 1 -- 172.21.15.7:0/3997786618 shutdown_connections 2024-04-25T11:22:38.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.380+0000 7f7ccbe00640 1 -- 172.21.15.7:0/3997786618 wait complete. 2024-04-25T11:22:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.382+0000 7f7ccbe00640 1 Processor -- start 2024-04-25T11:22:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.382+0000 7f7ccbe00640 1 -- start start 2024-04-25T11:22:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.383+0000 7f7ccbe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ccc0cfb50 0x7f7ccc157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:38.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7ccbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ccc0d09c0 0x7f7ccc1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7ccbe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ccc1583c0 0x7f7ccc159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7ccbe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc06f5b0 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7ccbe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc0cf0f0 con 0x7f7ccc0cfb50 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7ccbe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7ccc06f730 con 0x7f7ccc1583c0 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7cc3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ccc0d09c0 0x7f7ccc1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7cc3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ccc0d09c0 0x7f7ccc1557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34548/0 (socket says 172.21.15.7:34548) 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7cc3e00640 1 -- 172.21.15.7:0/2634538083 learned_addr learned my addr 172.21.15.7:0/2634538083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7ccaa00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ccc0cfb50 0x7f7ccc157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7ccb400640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ccc1583c0 0x7f7ccc159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7cc3e00640 1 -- 172.21.15.7:0/2634538083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ccc1583c0 msgr2=0x7f7ccc159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7cc3e00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ccc1583c0 0x7f7ccc159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7cc3e00640 1 -- 172.21.15.7:0/2634538083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ccc0cfb50 msgr2=0x7f7ccc157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7cc3e00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ccc0cfb50 0x7f7ccc157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.384+0000 7f7cc3e00640 1 -- 172.21.15.7:0/2634538083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ccc1563c0 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.385+0000 7f7cc3e00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ccc0d09c0 0x7f7ccc1557f0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4000c00 tx=0x7f7cb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:38.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.385+0000 7f7cc2a00640 1 -- 172.21.15.7:0/2634538083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4037040 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.385+0000 7f7cc2a00640 1 -- 172.21.15.7:0/2634538083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb4042070 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.385+0000 7f7cc2a00640 1 -- 172.21.15.7:0/2634538083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4030880 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.385+0000 7f7ccbe00640 1 -- 172.21.15.7:0/2634538083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ccc1566a0 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.385+0000 7f7ccbe00640 1 -- 172.21.15.7:0/2634538083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ccc156c60 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.386+0000 7f7ccbe00640 1 -- 172.21.15.7:0/2634538083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ccc050740 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.388+0000 7f7cc2a00640 1 -- 172.21.15.7:0/2634538083 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7cb4030a20 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.389+0000 7f7cc2a00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ca80744b0 0x7f7ca80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:38.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.389+0000 7f7cc2a00640 1 -- 172.21.15.7:0/2634538083 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f7cb40b5d00 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.389+0000 7f7ccaa00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ca80744b0 0x7f7ca80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:38.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.390+0000 7f7ccaa00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ca80744b0 0x7f7ca80769c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0aa990 tx=0x7f7cbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:38.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.394+0000 7f7cc2a00640 1 -- 172.21.15.7:0/2634538083 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7cb40802f0 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.668+0000 7f7ccbe00640 1 -- 172.21.15.7:0/2634538083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7ccc070dd0 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.675+0000 7f7cc2a00640 1 -- 172.21.15.7:0/2634538083 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f7cb407fbf0 con 0x7f7ccc0d09c0 2024-04-25T11:22:38.670 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-04-25T11:22:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDcOSpmpeoqBxAAyHpGKq34Rc69atn0Zk9fNg== 2024-04-25T11:22:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-04-25T11:22:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-04-25T11:22:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T11:22:38.671 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-04-25T11:22:38.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.677+0000 7f7ccbe00640 1 -- 172.21.15.7:0/2634538083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ca80744b0 msgr2=0x7f7ca80769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.677+0000 7f7ccbe00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ca80744b0 0x7f7ca80769c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0aa990 tx=0x7f7cbc008040 comp rx=0 tx=0).stop 2024-04-25T11:22:38.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.677+0000 7f7ccbe00640 1 -- 172.21.15.7:0/2634538083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ccc0d09c0 msgr2=0x7f7ccc1557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.677+0000 7f7ccbe00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ccc0d09c0 0x7f7ccc1557f0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4000c00 tx=0x7f7cb40046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.677+0000 7f7ccbe00640 1 -- 172.21.15.7:0/2634538083 shutdown_connections 2024-04-25T11:22:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.677+0000 7f7ccbe00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ca80744b0 0x7f7ca80769c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.677+0000 7f7ccbe00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ccc1583c0 0x7f7ccc159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.677+0000 7f7ccbe00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ccc0d09c0 0x7f7ccc1557f0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.677+0000 7f7ccbe00640 1 --2- 172.21.15.7:0/2634538083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ccc0cfb50 0x7f7ccc157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.677+0000 7f7ccbe00640 1 -- 172.21.15.7:0/2634538083 >> 172.21.15.7:0/2634538083 conn(0x7f7ccc0740f0 msgr2=0x7f7ccc15fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.678+0000 7f7ccbe00640 1 -- 172.21.15.7:0/2634538083 shutdown_connections 2024-04-25T11:22:38.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.678+0000 7f7ccbe00640 1 -- 172.21.15.7:0/2634538083 wait complete. 2024-04-25T11:22:38.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.989+0000 7f6237200640 1 Processor -- start 2024-04-25T11:22:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.989+0000 7f6237200640 1 -- start start 2024-04-25T11:22:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.991+0000 7f6237200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62380b3960 0x7f62380b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.991+0000 7f6237200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62380d2990 con 0x7f62380b3d70 2024-04-25T11:22:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.991+0000 7f6237200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62380d2c50 con 0x7f62380b08b0 2024-04-25T11:22:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.991+0000 7f6237200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f62380d2f30 con 0x7f62380b3960 2024-04-25T11:22:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.992+0000 7f6235400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f62380b3d70 0x7f62380d1790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40004/0 (socket says 172.21.15.7:40004) 2024-04-25T11:22:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.992+0000 7f6235e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62380b3960 0x7f62380b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.992+0000 7f6235400640 1 -- 172.21.15.7:0/502610768 learned_addr learned my addr 172.21.15.7:0/502610768 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.992+0000 7f6235e00640 1 -- 172.21.15.7:0/502610768 >> v1:172.21.15.87:6789/0 conn(0x7f62380b08b0 legacy=0x7f62380d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.992+0000 7f6235e00640 1 -- 172.21.15.7:0/502610768 >> v1:172.21.15.7:6789/0 conn(0x7f62380b3d70 legacy=0x7f62380d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.992+0000 7f6235e00640 1 -- 172.21.15.7:0/502610768 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62380d3210 con 0x7f62380b3960 2024-04-25T11:22:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.992+0000 7f6234a00640 1 -- 172.21.15.7:0/502610768 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3442060221 0 0) 0x7f62380d2990 con 0x7f62380b3d70 2024-04-25T11:22:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.993+0000 7f6235e00640 1 --2- 172.21.15.7:0/502610768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62380b3960 0x7f62380b73d0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f62280097a0 tx=0x7f622802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a53e9966cdb03c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.994+0000 7f6234a00640 1 -- 172.21.15.7:0/502610768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228037070 con 0x7f62380b3960 2024-04-25T11:22:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.994+0000 7f6234a00640 1 -- 172.21.15.7:0/502610768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6228032440 con 0x7f62380b3960 2024-04-25T11:22:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.994+0000 7f6234a00640 1 -- 172.21.15.7:0/502610768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228032810 con 0x7f62380b3960 2024-04-25T11:22:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.994+0000 7f6237200640 1 -- 172.21.15.7:0/502610768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62380b3960 msgr2=0x7f62380b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.994+0000 7f6237200640 1 --2- 172.21.15.7:0/502610768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62380b3960 0x7f62380b73d0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f62280097a0 tx=0x7f622802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.995+0000 7f6237200640 1 -- 172.21.15.7:0/502610768 shutdown_connections 2024-04-25T11:22:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.995+0000 7f6237200640 1 --2- 172.21.15.7:0/502610768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62380b3960 0x7f62380b73d0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.995+0000 7f6237200640 1 -- 172.21.15.7:0/502610768 >> 172.21.15.7:0/502610768 conn(0x7f623809b540 msgr2=0x7f623809b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.995+0000 7f6237200640 1 -- 172.21.15.7:0/502610768 shutdown_connections 2024-04-25T11:22:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.995+0000 7f6237200640 1 -- 172.21.15.7:0/502610768 wait complete. 2024-04-25T11:22:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.997+0000 7f6237200640 1 Processor -- start 2024-04-25T11:22:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.997+0000 7f6237200640 1 -- start start 2024-04-25T11:22:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.999+0000 7f6237200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62380b08b0 0x7f6238154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.999+0000 7f6237200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62380b3960 0x7f623815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.999+0000 7f6237200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62380b3d70 0x7f623815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.999+0000 7f6237200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62380d2730 con 0x7f62380b3960 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.999+0000 7f6237200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f62380d25b0 con 0x7f62380b3d70 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.999+0000 7f6237200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f623806e990 con 0x7f62380b08b0 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.999+0000 7f6235400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62380b3960 0x7f623815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.999+0000 7f6235400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62380b3960 0x7f623815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34556/0 (socket says 172.21.15.7:34556) 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.999+0000 7f6235400640 1 -- 172.21.15.7:0/1535147136 learned_addr learned my addr 172.21.15.7:0/1535147136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.999+0000 7f6235e00640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62380b08b0 0x7f6238154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:38.999+0000 7f6236800640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62380b3d70 0x7f623815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f6235400640 1 -- 172.21.15.7:0/1535147136 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62380b08b0 msgr2=0x7f6238154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f6235400640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62380b08b0 0x7f6238154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f6235400640 1 -- 172.21.15.7:0/1535147136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62380b3d70 msgr2=0x7f623815d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f6235400640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62380b3d70 0x7f623815d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:38.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f6235400640 1 -- 172.21.15.7:0/1535147136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623815bbf0 con 0x7f62380b3960 2024-04-25T11:22:38.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f6236800640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62380b3d70 0x7f623815d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:38.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f6235400640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62380b3960 0x7f623815b6b0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f622400c7c0 tx=0x7f622400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:38.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f622e000640 1 -- 172.21.15.7:0/1535147136 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224018070 con 0x7f62380b3960 2024-04-25T11:22:38.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f622e000640 1 -- 172.21.15.7:0/1535147136 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f622400a5e0 con 0x7f62380b3960 2024-04-25T11:22:38.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f622e000640 1 -- 172.21.15.7:0/1535147136 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622400a9d0 con 0x7f62380b3960 2024-04-25T11:22:38.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f6237200640 1 -- 172.21.15.7:0/1535147136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623806d560 con 0x7f62380b3960 2024-04-25T11:22:38.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.000+0000 7f6237200640 1 -- 172.21.15.7:0/1535147136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623806db20 con 0x7f62380b3960 2024-04-25T11:22:38.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.002+0000 7f6237200640 1 -- 172.21.15.7:0/1535147136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6238050740 con 0x7f62380b3960 2024-04-25T11:22:38.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.003+0000 7f622e000640 1 -- 172.21.15.7:0/1535147136 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f622400ab70 con 0x7f62380b3960 2024-04-25T11:22:38.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.004+0000 7f622e000640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62180744e0 0x7f62180769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:38.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.004+0000 7f622e000640 1 -- 172.21.15.7:0/1535147136 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f6224094900 con 0x7f62380b3960 2024-04-25T11:22:38.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.005+0000 7f6235e00640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62180744e0 0x7f62180769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:39.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.005+0000 7f6235e00640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62180744e0 0x7f62180769f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6228034000 tx=0x7f622802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.009+0000 7f622e000640 1 -- 172.21.15.7:0/1535147136 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f622405eef0 con 0x7f62380b3960 2024-04-25T11:22:39.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.251+0000 7f6237200640 1 -- 172.21.15.7:0/1535147136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f623815f3b0 con 0x7f62380b3960 2024-04-25T11:22:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.253+0000 7f622e000640 1 -- 172.21.15.7:0/1535147136 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5179 (secure 0 0 0) 0x7f622405e7f0 con 0x7f62380b3960 2024-04-25T11:22:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:39.248 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7588,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2938174159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2938174159},{"type":"v1","addr":"172.21.15.87:6834","nonce":2938174159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/793358721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":793358721},{"type":"v1","addr":"172.21.15.87:6835","nonce":793358721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":9447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2654444897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2654444897},{"type":"v1","addr":"172.21.15.7:6835","nonce":2654444897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:22:32.315016+0000","modified":"2024-04-25T11:22:35.372928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/339418645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":339418645},{"type":"v1","addr":"172.21.15.7:6837","nonce":339418645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":11}]} 2024-04-25T11:22:39.248 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 116 2024-04-25T11:22:39.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.256+0000 7f6237200640 1 -- 172.21.15.7:0/1535147136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62180744e0 msgr2=0x7f62180769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:39.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.256+0000 7f6237200640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62180744e0 0x7f62180769f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6228034000 tx=0x7f622802e000 comp rx=0 tx=0).stop 2024-04-25T11:22:39.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.256+0000 7f6237200640 1 -- 172.21.15.7:0/1535147136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62380b3960 msgr2=0x7f623815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:39.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.256+0000 7f6237200640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62380b3960 0x7f623815b6b0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f622400c7c0 tx=0x7f622400cc80 comp rx=0 tx=0).stop 2024-04-25T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.256+0000 7f6237200640 1 -- 172.21.15.7:0/1535147136 shutdown_connections 2024-04-25T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.256+0000 7f6237200640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f62180744e0 0x7f62180769f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.256+0000 7f6237200640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62380b3d70 0x7f623815d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.256+0000 7f6237200640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62380b3960 0x7f623815b6b0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.256+0000 7f6237200640 1 --2- 172.21.15.7:0/1535147136 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62380b08b0 0x7f6238154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.256+0000 7f6237200640 1 -- 172.21.15.7:0/1535147136 >> 172.21.15.7:0/1535147136 conn(0x7f623809b540 msgr2=0x7f62380b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.257+0000 7f6237200640 1 -- 172.21.15.7:0/1535147136 shutdown_connections 2024-04-25T11:22:39.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.257+0000 7f6237200640 1 -- 172.21.15.7:0/1535147136 wait complete. 2024-04-25T11:22:39.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 116, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T11:22:32.315016+0000', 'modified': '2024-04-25T11:22:35.372928+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9474}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9474': {'gid': 9474, 'name': 'c', 'rank': 0, 'incarnation': 112, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6837/339418645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6836', 'nonce': 339418645}, {'type': 'v1', 'addr': '172.21.15.7:6837', 'nonce': 339418645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9474, 'qdb_cluster': [9474]} 2024-04-25T11:22:39.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T11:22:39.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T11:22:39.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.974+0000 7fbeb0600640 1 Processor -- start 2024-04-25T11:22:39.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.974+0000 7fbeb0600640 1 -- start start 2024-04-25T11:22:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.977+0000 7fbeb0600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea806d580 0x7fbea806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.977+0000 7fbeb0600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbea806f1e0 0x7fbea806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.977+0000 7fbeb0600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbea806ded0 con 0x7fbea815f180 2024-04-25T11:22:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.977+0000 7fbeb0600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbea815f590 con 0x7fbea806f1e0 2024-04-25T11:22:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.977+0000 7fbeb0600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbea8158ae0 con 0x7fbea806d580 2024-04-25T11:22:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.977+0000 7fbeae600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbea806f1e0 0x7fbea806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.977+0000 7fbeae600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbea806f1e0 0x7fbea806f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46752/0 (socket says 172.21.15.7:46752) 2024-04-25T11:22:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.977+0000 7fbeae600640 1 -- 172.21.15.7:0/1679831689 learned_addr learned my addr 172.21.15.7:0/1679831689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.978+0000 7fbead200640 1 --2- 172.21.15.7:0/1679831689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea806d580 0x7fbea806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.978+0000 7fbeae600640 1 -- 172.21.15.7:0/1679831689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea806d580 msgr2=0x7fbea806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.978+0000 7fbeae600640 1 --2- 172.21.15.7:0/1679831689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea806d580 0x7fbea806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.978+0000 7fbeae600640 1 -- 172.21.15.7:0/1679831689 >> v1:172.21.15.87:6789/0 conn(0x7fbea815f180 legacy=0x7fbea8161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.978+0000 7fbeae600640 1 -- 172.21.15.7:0/1679831689 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea8162700 con 0x7fbea806f1e0 2024-04-25T11:22:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.978+0000 7fbead200640 1 --2- 172.21.15.7:0/1679831689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea806d580 0x7fbea806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.979+0000 7fbeae600640 1 --2- 172.21.15.7:0/1679831689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea806f1e0 0x7fbea806f5f0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009e80 tx=0x7fbea002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be651822c4a17f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.979+0000 7fbea7e00640 1 -- 172.21.15.7:0/1679831689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0036070 con 0x7fbea806f1e0 2024-04-25T11:22:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.979+0000 7fbea7e00640 1 -- 172.21.15.7:0/1679831689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea0031600 con 0x7fbea806f1e0 2024-04-25T11:22:39.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.979+0000 7fbea7e00640 1 -- 172.21.15.7:0/1679831689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00319d0 con 0x7fbea806f1e0 2024-04-25T11:22:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.980+0000 7fbeb0600640 1 -- 172.21.15.7:0/1679831689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea806f1e0 msgr2=0x7fbea806f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.980+0000 7fbeb0600640 1 --2- 172.21.15.7:0/1679831689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea806f1e0 0x7fbea806f5f0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009e80 tx=0x7fbea002a6f0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.980+0000 7fbeb0600640 1 -- 172.21.15.7:0/1679831689 shutdown_connections 2024-04-25T11:22:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.980+0000 7fbeb0600640 1 --2- 172.21.15.7:0/1679831689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea806f1e0 0x7fbea806f5f0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.980+0000 7fbeb0600640 1 --2- 172.21.15.7:0/1679831689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea806d580 0x7fbea806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.980+0000 7fbeb0600640 1 -- 172.21.15.7:0/1679831689 >> 172.21.15.7:0/1679831689 conn(0x7fbea80732b0 msgr2=0x7fbea80736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:39.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.980+0000 7fbeb0600640 1 -- 172.21.15.7:0/1679831689 shutdown_connections 2024-04-25T11:22:39.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.980+0000 7fbeb0600640 1 -- 172.21.15.7:0/1679831689 wait complete. 2024-04-25T11:22:39.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.983+0000 7fbeb0600640 1 Processor -- start 2024-04-25T11:22:39.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.984+0000 7fbeb0600640 1 -- start start 2024-04-25T11:22:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.985+0000 7fbeb0600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea806d580 0x7fbea81f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.985+0000 7fbeb0600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea806f1e0 0x7fbea81463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbeb0600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea815f180 0x7fbea80b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbeb0600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbea815ec60 con 0x7fbea806f1e0 2024-04-25T11:22:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbeb0600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbea815e980 con 0x7fbea806d580 2024-04-25T11:22:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbeb0600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbea815e670 con 0x7fbea815f180 2024-04-25T11:22:39.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbead200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea806f1e0 0x7fbea81463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbead200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea806f1e0 0x7fbea81463a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46768/0 (socket says 172.21.15.7:46768) 2024-04-25T11:22:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbead200640 1 -- 172.21.15.7:0/716091066 learned_addr learned my addr 172.21.15.7:0/716091066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbeae600640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea815f180 0x7fbea80b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbeadc00640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea806d580 0x7fbea81f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbead200640 1 -- 172.21.15.7:0/716091066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea815f180 msgr2=0x7fbea80b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbead200640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea815f180 0x7fbea80b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbead200640 1 -- 172.21.15.7:0/716091066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea806d580 msgr2=0x7fbea81f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbead200640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea806d580 0x7fbea81f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.986+0000 7fbead200640 1 -- 172.21.15.7:0/716091066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea8147110 con 0x7fbea806f1e0 2024-04-25T11:22:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.987+0000 7fbead200640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea806f1e0 0x7fbea81463a0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400c7f0 tx=0x7fbe9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.987+0000 7fbea5600640 1 -- 172.21.15.7:0/716091066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94015070 con 0x7fbea806f1e0 2024-04-25T11:22:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.987+0000 7fbea5600640 1 -- 172.21.15.7:0/716091066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe940077c0 con 0x7fbea806f1e0 2024-04-25T11:22:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.987+0000 7fbea5600640 1 -- 172.21.15.7:0/716091066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94007b70 con 0x7fbea806f1e0 2024-04-25T11:22:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.987+0000 7fbeb0600640 1 -- 172.21.15.7:0/716091066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea81473f0 con 0x7fbea806f1e0 2024-04-25T11:22:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.987+0000 7fbeb0600640 1 -- 172.21.15.7:0/716091066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea81479b0 con 0x7fbea806f1e0 2024-04-25T11:22:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.990+0000 7fbea5600640 1 -- 172.21.15.7:0/716091066 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbe94018070 con 0x7fbea806f1e0 2024-04-25T11:22:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.990+0000 7fbea5600640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 0x7fbe880769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.991+0000 7fbea5600640 1 -- 172.21.15.7:0/716091066 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7fbe94095460 con 0x7fbea806f1e0 2024-04-25T11:22:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.991+0000 7fbeadc00640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 0x7fbe880769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.991+0000 7fbeadc00640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 0x7fbe880769c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c006fd0 tx=0x7fbe9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.992+0000 7fbeb75a8740 1 Processor -- start 2024-04-25T11:22:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.992+0000 7fbeb75a8740 1 -- start start 2024-04-25T11:22:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.993+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206ad270 0x55ad20a7a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.993+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206843b0 0x55ad20a94d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.993+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad2068c590 0x55ad20a952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.993+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ad209b3600 con 0x55ad206843b0 2024-04-25T11:22:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.993+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ad20950100 con 0x55ad2068c590 2024-04-25T11:22:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.993+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55ad209af5c0 con 0x55ad206ad270 2024-04-25T11:22:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.993+0000 7fbead200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206843b0 0x55ad20a94d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.993+0000 7fbead200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206843b0 0x55ad20a94d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46770/0 (socket says 172.21.15.7:46770) 2024-04-25T11:22:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.993+0000 7fbead200640 1 -- 172.21.15.7:0/1904688166 learned_addr learned my addr 172.21.15.7:0/1904688166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.994+0000 7fbead200640 1 -- 172.21.15.7:0/1904688166 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206ad270 msgr2=0x55ad20a7a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.994+0000 7fbead200640 1 --2- 172.21.15.7:0/1904688166 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206ad270 0x55ad20a7a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.994+0000 7fbead200640 1 -- 172.21.15.7:0/1904688166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad2068c590 msgr2=0x55ad20a952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.994+0000 7fbead200640 1 --2- 172.21.15.7:0/1904688166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad2068c590 0x55ad20a952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.994+0000 7fbead200640 1 -- 172.21.15.7:0/1904688166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad206c52d0 con 0x55ad206843b0 2024-04-25T11:22:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.994+0000 7fbead200640 1 --2- 172.21.15.7:0/1904688166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206843b0 0x55ad20a94d70 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fbe94007fd0 tx=0x7fbe9405efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e544069095099493 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.994+0000 7fbe9a000640 1 -- 172.21.15.7:0/1904688166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9401f040 con 0x55ad206843b0 2024-04-25T11:22:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.994+0000 7fbe9a000640 1 -- 172.21.15.7:0/1904688166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe940637a0 con 0x55ad206843b0 2024-04-25T11:22:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.994+0000 7fbe9a000640 1 -- 172.21.15.7:0/1904688166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94067f30 con 0x55ad206843b0 2024-04-25T11:22:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.995+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1904688166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206843b0 msgr2=0x55ad20a94d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.995+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/1904688166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206843b0 0x55ad20a94d70 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fbe94007fd0 tx=0x7fbe9405efc0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.995+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1904688166 shutdown_connections 2024-04-25T11:22:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.995+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/1904688166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad2068c590 0x55ad20a952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.995+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/1904688166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206843b0 0x55ad20a94d70 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.995+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/1904688166 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206ad270 0x55ad20a7a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.995+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1904688166 >> 172.21.15.7:0/1904688166 conn(0x55ad206a13c0 msgr2=0x55ad20a4f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.995+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1904688166 shutdown_connections 2024-04-25T11:22:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.995+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1904688166 wait complete. 2024-04-25T11:22:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.996+0000 7fbeb75a8740 1 -- start start 2024-04-25T11:22:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.996+0000 7fbeb75a8740 1 Processor -- start 2024-04-25T11:22:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.998+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206843b0 0x55ad20a749c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.998+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20a8f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.998+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206ad270 0x55ad20aa55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.998+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ad2093af70 con 0x55ad2068c590 2024-04-25T11:22:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.998+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ad20aa6040 con 0x55ad206843b0 2024-04-25T11:22:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.998+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55ad20a49790 con 0x55ad206ad270 2024-04-25T11:22:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.998+0000 7fbeb75a8740 10 client.0 _ll_register_callbacks cb 0x55ad209a9480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:22:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.998+0000 7fbead200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20a8f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.998+0000 7fbead200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20a8f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46776/0 (socket says 172.21.15.7:46776) 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.998+0000 7fbead200640 1 -- 172.21.15.7:0/2233035939 learned_addr learned my addr 172.21.15.7:0/2233035939 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbead200640 1 -- 172.21.15.7:0/2233035939 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206ad270 msgr2=0x55ad20aa55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbead200640 1 --2- 172.21.15.7:0/2233035939 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206ad270 0x55ad20aa55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbead200640 1 -- 172.21.15.7:0/2233035939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206843b0 msgr2=0x55ad20a749c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbead200640 1 --2- 172.21.15.7:0/2233035939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206843b0 0x55ad20a749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbead200640 1 -- 172.21.15.7:0/2233035939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad2078f6b0 con 0x55ad2068c590 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbead200640 1 --2- 172.21.15.7:0/2233035939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20a8f490 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fbe9405f0f0 tx=0x7fbe94007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818bc0d776161392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbe8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbe8ea00640 1 -- 172.21.15.7:0/2233035939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9401f040 con 0x55ad2068c590 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbe8ea00640 1 -- 172.21.15.7:0/2233035939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7fbe94078b50 con 0x55ad2068c590 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbeb75a8740 1 -- 172.21.15.7:0/2233035939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe940637a0 con 0x55ad2068c590 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbe8ea00640 1 -- 172.21.15.7:0/2233035939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fbe94080050 con 0x55ad2068c590 2024-04-25T11:22:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbeb75a8740 10 client.9591 fetch_fsmap learned FSMap version 116 2024-04-25T11:22:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:39.999+0000 7fbeb75a8740 1 -- 172.21.15.7:0/2233035939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x55ad20aa5d80 con 0x55ad2068c590 2024-04-25T11:22:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.000+0000 7fbe8ea00640 1 -- 172.21.15.7:0/2233035939 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe940637a0 con 0x55ad2068c590 2024-04-25T11:22:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.000+0000 7fbe8ea00640 1 -- 172.21.15.7:0/2233035939 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 116) ==== 3494+0+0 (secure 0 0 0) 0x7fbe94093220 con 0x55ad2068c590 2024-04-25T11:22:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.000+0000 7fbeb75a8740 10 client.9591 fetch_fsmap finished waiting for FSMap version 116 2024-04-25T11:22:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.000+0000 7fbeb75a8740 10 client.9591 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:22:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.000+0000 7fbeb75a8740 20 client.9591 populate_metadata read hostname 'smithi007' 2024-04-25T11:22:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.000+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/2233035939 >> [v2:172.21.15.7:6836/339418645,v1:172.21.15.7:6837/339418645] conn(0x55ad2066be40 0x55ad20aa7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.000+0000 7fbeb75a8740 4 client.9591 mds_command: new command op to 9474 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T11:22:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.000+0000 7fbeb75a8740 1 -- 172.21.15.7:0/2233035939 --> [v2:172.21.15.7:6836/339418645,v1:172.21.15.7:6837/339418645] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad209c1210 con 0x55ad2066be40 2024-04-25T11:22:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.000+0000 7fbeadc00640 1 --2- 172.21.15.7:0/2233035939 >> [v2:172.21.15.7:6836/339418645,v1:172.21.15.7:6837/339418645] conn(0x55ad2066be40 0x55ad20aa7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.001+0000 7fbeadc00640 1 --2- 172.21.15.7:0/2233035939 >> [v2:172.21.15.7:6836/339418645,v1:172.21.15.7:6837/339418645] conn(0x55ad2066be40 0x55ad20aa7c70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bc79c896cce5cdd server_cookie=62057cdc247c5862 in_seq=0 out_seq=0 2024-04-25T11:22:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.001+0000 7fbe8ea00640 10 client.9591 ms_handle_connect on v2:172.21.15.7:6836/339418645 2024-04-25T11:22:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbe8ea00640 1 -- 172.21.15.7:0/2233035939 <== mds.0 v2:172.21.15.7:6836/339418645 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ad209c1210 con 0x55ad2066be40 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbe8ea00640 10 client.9591 handle_command_reply: tid=0 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 1 client.9591 shutdown 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 1 -- 172.21.15.7:0/2233035939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 msgr2=0x55ad20a8f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/2233035939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20a8f490 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fbe9405f0f0 tx=0x7fbe94007f20 comp rx=0 tx=0).stop 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 1 -- 172.21.15.7:0/2233035939 shutdown_connections 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/2233035939 >> [v2:172.21.15.7:6836/339418645,v1:172.21.15.7:6837/339418645] conn(0x55ad2066be40 0x55ad20aa7c70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/2233035939 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206ad270 0x55ad20aa55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/2233035939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20a8f490 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/2233035939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206843b0 0x55ad20a749c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbe8ea00640 0 client.9591 ms_handle_reset on v2:172.21.15.7:6836/339418645 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 1 -- 172.21.15.7:0/2233035939 >> 172.21.15.7:0/2233035939 conn(0x55ad206a13c0 msgr2=0x55ad20a7a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 1 -- 172.21.15.7:0/2233035939 shutdown_connections 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 1 -- 172.21.15.7:0/2233035939 wait complete. 2024-04-25T11:22:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.002+0000 7fbeb75a8740 20 client.9591 trim_cache size 0 max 16384 2024-04-25T11:22:40.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.022+0000 7fbeb75a8740 1 Processor -- start 2024-04-25T11:22:40.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.022+0000 7fbeb75a8740 1 -- start start 2024-04-25T11:22:40.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206843b0 0x55ad20aadcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad2068c590 0x55ad20a8e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206ad270 0x55ad20a8e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8202330 con 0x55ad206ad270 2024-04-25T11:22:40.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8148bf0 con 0x55ad206843b0 2024-04-25T11:22:40.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbea8201f00 con 0x55ad2068c590 2024-04-25T11:22:40.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeae600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206ad270 0x55ad20a8e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:40.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeae600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206ad270 0x55ad20a8e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46792/0 (socket says 172.21.15.7:46792) 2024-04-25T11:22:40.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeae600640 1 -- 172.21.15.7:0/3495140522 learned_addr learned my addr 172.21.15.7:0/3495140522 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:40.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeae600640 1 -- 172.21.15.7:0/3495140522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad2068c590 msgr2=0x55ad20a8e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:40.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeae600640 1 --2- 172.21.15.7:0/3495140522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad2068c590 0x55ad20a8e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeae600640 1 -- 172.21.15.7:0/3495140522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206843b0 msgr2=0x55ad20aadcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:40.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeae600640 1 --2- 172.21.15.7:0/3495140522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206843b0 0x55ad20aadcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.024+0000 7fbeae600640 1 -- 172.21.15.7:0/3495140522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad20a51a20 con 0x55ad206ad270 2024-04-25T11:22:40.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.025+0000 7fbeae600640 1 --2- 172.21.15.7:0/3495140522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206ad270 0x55ad20a8e740 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fbea002ace0 tx=0x7fbea0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee47736a70baa2d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.025+0000 7fbe99600640 1 -- 172.21.15.7:0/3495140522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0040020 con 0x55ad206ad270 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.025+0000 7fbe99600640 1 -- 172.21.15.7:0/3495140522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea0037040 con 0x55ad206ad270 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.025+0000 7fbe99600640 1 -- 172.21.15.7:0/3495140522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0036030 con 0x55ad206ad270 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.026+0000 7fbeb75a8740 1 -- 172.21.15.7:0/3495140522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206ad270 msgr2=0x55ad20a8e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.026+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/3495140522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206ad270 0x55ad20a8e740 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fbea002ace0 tx=0x7fbea0004ab0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.026+0000 7fbeb75a8740 1 -- 172.21.15.7:0/3495140522 shutdown_connections 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.026+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/3495140522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad206ad270 0x55ad20a8e740 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.026+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/3495140522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad2068c590 0x55ad20a8e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.026+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/3495140522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206843b0 0x55ad20aadcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.026+0000 7fbeb75a8740 1 -- 172.21.15.7:0/3495140522 >> 172.21.15.7:0/3495140522 conn(0x55ad206a13c0 msgr2=0x55ad20a747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.026+0000 7fbeb75a8740 1 -- 172.21.15.7:0/3495140522 shutdown_connections 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.026+0000 7fbeb75a8740 1 -- 172.21.15.7:0/3495140522 wait complete. 2024-04-25T11:22:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.027+0000 7fbeb75a8740 1 -- start start 2024-04-25T11:22:40.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.027+0000 7fbeb75a8740 1 Processor -- start 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.028+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206843b0 0x55ad20b12850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20aa2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbeb75a8740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206ad270 0x55ad20a95270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ad20a49270 con 0x55ad2068c590 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55ad20aa51a0 con 0x55ad206ad270 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbeb75a8740 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55ad20a91b50 con 0x55ad206843b0 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbeb75a8740 10 client.0 _ll_register_callbacks cb 0x7fbe9405ebf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbead200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20aa2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbead200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20aa2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46796/0 (socket says 172.21.15.7:46796) 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbead200640 1 -- 172.21.15.7:0/1999453473 learned_addr learned my addr 172.21.15.7:0/1999453473 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbead200640 1 -- 172.21.15.7:0/1999453473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206843b0 msgr2=0x55ad20b12850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbead200640 1 --2- 172.21.15.7:0/1999453473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206843b0 0x55ad20b12850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbead200640 1 -- 172.21.15.7:0/1999453473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206ad270 msgr2=0x55ad20a95270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbead200640 1 --2- 172.21.15.7:0/1999453473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206ad270 0x55ad20a95270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.029+0000 7fbead200640 1 -- 172.21.15.7:0/1999453473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad20aa2ca0 con 0x55ad2068c590 2024-04-25T11:22:40.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbead200640 1 --2- 172.21.15.7:0/1999453473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20aa2760 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fbe94000c00 tx=0x7fbe940936d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee53dd5fcec3974 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:40.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbe8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbe8ea00640 1 -- 172.21.15.7:0/1999453473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9401f050 con 0x55ad2068c590 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbe8ea00640 1 -- 172.21.15.7:0/1999453473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7fbe94080e00 con 0x55ad2068c590 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1999453473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea0037040 con 0x55ad2068c590 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbe8ea00640 1 -- 172.21.15.7:0/1999453473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fbe940637a0 con 0x55ad2068c590 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbeb75a8740 10 client.9597 fetch_fsmap learned FSMap version 116 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1999453473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=116,monmap=2+}) -- 0x55ad20a8e9d0 con 0x55ad2068c590 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbe8ea00640 1 -- 172.21.15.7:0/1999453473 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea0037040 con 0x55ad2068c590 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbe8ea00640 1 -- 172.21.15.7:0/1999453473 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 116) ==== 3494+0+0 (secure 0 0 0) 0x7fbe9405ad60 con 0x55ad2068c590 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbeb75a8740 10 client.9597 fetch_fsmap finished waiting for FSMap version 116 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbeb75a8740 10 client.9597 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbeb75a8740 20 client.9597 populate_metadata read hostname 'smithi007' 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.030+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/1999453473 >> [v2:172.21.15.7:6836/339418645,v1:172.21.15.7:6837/339418645] conn(0x55ad20ab73f0 0x55ad20a8c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.031+0000 7fbeb75a8740 4 client.9597 mds_command: new command op to 9474 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.031+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1999453473 --> [v2:172.21.15.7:6836/339418645,v1:172.21.15.7:6837/339418645] -- command(tid 0: {"prefix": "status"}) -- 0x55ad209bb080 con 0x55ad20ab73f0 2024-04-25T11:22:40.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.031+0000 7fbeadc00640 1 --2- 172.21.15.7:0/1999453473 >> [v2:172.21.15.7:6836/339418645,v1:172.21.15.7:6837/339418645] conn(0x55ad20ab73f0 0x55ad20a8c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.031+0000 7fbeadc00640 1 --2- 172.21.15.7:0/1999453473 >> [v2:172.21.15.7:6836/339418645,v1:172.21.15.7:6837/339418645] conn(0x55ad20ab73f0 0x55ad20a8c7f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8dbce972a4678d server_cookie=83aa9c73dd62a67c in_seq=0 out_seq=0 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.031+0000 7fbe8ea00640 10 client.9597 ms_handle_connect on v2:172.21.15.7:6836/339418645 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbe8ea00640 1 -- 172.21.15.7:0/1999453473 <== mds.0 v2:172.21.15.7:6836/339418645 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55ad209bb080 con 0x55ad20ab73f0 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbe8ea00640 10 client.9597 handle_command_reply: tid=0 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 1 client.9597 shutdown 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1999453473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 msgr2=0x55ad20aa2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/1999453473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20aa2760 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fbe94000c00 tx=0x7fbe940936d0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1999453473 shutdown_connections 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/1999453473 >> [v2:172.21.15.7:6836/339418645,v1:172.21.15.7:6837/339418645] conn(0x55ad20ab73f0 0x55ad20a8c7f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/1999453473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55ad206ad270 0x55ad20a95270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/1999453473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ad2068c590 0x55ad20aa2760 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 1 --2- 172.21.15.7:0/1999453473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55ad206843b0 0x55ad20b12850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbe8ea00640 0 client.9597 ms_handle_reset on v2:172.21.15.7:6836/339418645 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1999453473 >> 172.21.15.7:0/1999453473 conn(0x55ad206a13c0 msgr2=0x55ad20a76200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1999453473 shutdown_connections 2024-04-25T11:22:40.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 1 -- 172.21.15.7:0/1999453473 wait complete. 2024-04-25T11:22:40.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.032+0000 7fbeb75a8740 20 client.9597 trim_cache size 0 max 16384 2024-04-25T11:22:40.030 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:22:40.030 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "4ee91e9b-5e2d-4052-ab33-0a56271c8c7f", 2024-04-25T11:22:40.030 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-04-25T11:22:40.030 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 9474, 2024-04-25T11:22:40.030 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-04-25T11:22:40.030 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-04-25T11:22:40.030 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-04-25T11:22:40.030 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 7.2608823109999996, 2024-04-25T11:22:40.031 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 116, 2024-04-25T11:22:40.031 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 156, 2024-04-25T11:22:40.031 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 156, 2024-04-25T11:22:40.031 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.402836655 2024-04-25T11:22:40.031 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 -- 172.21.15.7:0/716091066 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 msgr2=0x7fbe880769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 0x7fbe880769c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c006fd0 tx=0x7fbe9c008040 comp rx=0 tx=0).stop 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 -- 172.21.15.7:0/716091066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea806f1e0 msgr2=0x7fbea81463a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea806f1e0 0x7fbea81463a0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400c7f0 tx=0x7fbe9400ccb0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 -- 172.21.15.7:0/716091066 shutdown_connections 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe880744b0 0x7fbe880769c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbea815f180 0x7fbea80b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbea806f1e0 0x7fbea81463a0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 --2- 172.21.15.7:0/716091066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbea806d580 0x7fbea81f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 -- 172.21.15.7:0/716091066 >> 172.21.15.7:0/716091066 conn(0x7fbea80732b0 msgr2=0x7fbea80af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 -- 172.21.15.7:0/716091066 shutdown_connections 2024-04-25T11:22:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.033+0000 7fbeb0600640 1 -- 172.21.15.7:0/716091066 wait complete. 2024-04-25T11:22:40.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:22:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.319+0000 7ff43ee00640 1 Processor -- start 2024-04-25T11:22:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.319+0000 7ff43ee00640 1 -- start start 2024-04-25T11:22:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.321+0000 7ff43ee00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff43815e2f0 0x7ff438161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.321+0000 7ff43ee00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff438162420 0x7ff43806d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.321+0000 7ff43ee00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff43815e700 con 0x7ff43815baa0 2024-04-25T11:22:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.321+0000 7ff43ee00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7ff438162830 con 0x7ff43815e2f0 2024-04-25T11:22:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.321+0000 7ff43ee00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff43815e980 con 0x7ff438162420 2024-04-25T11:22:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.321+0000 7ff43da00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff43815baa0 0x7ff438070640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33780/0 (socket says 172.21.15.7:33780) 2024-04-25T11:22:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.321+0000 7ff43da00640 1 -- 172.21.15.7:0/1403008867 learned_addr learned my addr 172.21.15.7:0/1403008867 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.321+0000 7ff43d000640 1 --2- 172.21.15.7:0/1403008867 >> v2:172.21.15.7:3300/0 conn(0x7ff43815e2f0 0x7ff438161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.321+0000 7ff43e400640 1 --2- 172.21.15.7:0/1403008867 >> v2:172.21.15.87:3300/0 conn(0x7ff438162420 0x7ff43806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.321+0000 7ff437e00640 1 -- 172.21.15.7:0/1403008867 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3476620017 0 0) 0x7ff42c001110 con 0x7ff43815baa0 2024-04-25T11:22:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.321+0000 7ff437e00640 1 -- 172.21.15.7:0/1403008867 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff420003520 con 0x7ff43815baa0 2024-04-25T11:22:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.322+0000 7ff43d000640 1 -- 172.21.15.7:0/1403008867 >> v2:172.21.15.87:3300/0 conn(0x7ff438162420 msgr2=0x7ff43806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.322+0000 7ff43d000640 1 --2- 172.21.15.7:0/1403008867 >> v2:172.21.15.87:3300/0 conn(0x7ff438162420 0x7ff43806d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.322+0000 7ff43d000640 1 -- 172.21.15.7:0/1403008867 >> v1:172.21.15.7:6789/0 conn(0x7ff43815baa0 legacy=0x7ff438070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.322+0000 7ff43d000640 1 -- 172.21.15.7:0/1403008867 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43815ec60 con 0x7ff43815e2f0 2024-04-25T11:22:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.323+0000 7ff43d000640 1 --2- 172.21.15.7:0/1403008867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff43815e2f0 0x7ff438161ee0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff438073d30 tx=0x7ff42402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762b7601e5f4e409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.323+0000 7ff437e00640 1 -- 172.21.15.7:0/1403008867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424037070 con 0x7ff43815e2f0 2024-04-25T11:22:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.323+0000 7ff437e00640 1 -- 172.21.15.7:0/1403008867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff424032440 con 0x7ff43815e2f0 2024-04-25T11:22:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.323+0000 7ff437e00640 1 -- 172.21.15.7:0/1403008867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424032810 con 0x7ff43815e2f0 2024-04-25T11:22:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.324+0000 7ff43ee00640 1 -- 172.21.15.7:0/1403008867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff43815e2f0 msgr2=0x7ff438161ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.324+0000 7ff43ee00640 1 --2- 172.21.15.7:0/1403008867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff43815e2f0 0x7ff438161ee0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff438073d30 tx=0x7ff42402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.324+0000 7ff43ee00640 1 -- 172.21.15.7:0/1403008867 shutdown_connections 2024-04-25T11:22:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.324+0000 7ff43ee00640 1 --2- 172.21.15.7:0/1403008867 >> v2:172.21.15.87:3300/0 conn(0x7ff438162420 0x7ff43806d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.324+0000 7ff43ee00640 1 --2- 172.21.15.7:0/1403008867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff43815e2f0 0x7ff438161ee0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.324+0000 7ff43ee00640 1 -- 172.21.15.7:0/1403008867 >> 172.21.15.7:0/1403008867 conn(0x7ff438073090 msgr2=0x7ff4380734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.324+0000 7ff43ee00640 1 -- 172.21.15.7:0/1403008867 shutdown_connections 2024-04-25T11:22:40.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.324+0000 7ff43ee00640 1 -- 172.21.15.7:0/1403008867 wait complete. 2024-04-25T11:22:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.326+0000 7ff43ee00640 1 Processor -- start 2024-04-25T11:22:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.326+0000 7ff43ee00640 1 -- start start 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.328+0000 7ff43ee00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff43815e2f0 0x7ff4380cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.328+0000 7ff43ee00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff438162420 0x7ff438137d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.328+0000 7ff43ee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4380afe60 0x7ff4381382b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.328+0000 7ff43ee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff43806dac0 con 0x7ff4380afe60 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.328+0000 7ff43ee00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff4381619c0 con 0x7ff438162420 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.328+0000 7ff43ee00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff43806dc40 con 0x7ff43815e2f0 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.328+0000 7ff43e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4380afe60 0x7ff4381382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.328+0000 7ff43e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4380afe60 0x7ff4381382b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46814/0 (socket says 172.21.15.7:46814) 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.328+0000 7ff43e400640 1 -- 172.21.15.7:0/1412923325 learned_addr learned my addr 172.21.15.7:0/1412923325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.329+0000 7ff43da00640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff43815e2f0 0x7ff4380cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.329+0000 7ff43d000640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff438162420 0x7ff438137d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.329+0000 7ff43e400640 1 -- 172.21.15.7:0/1412923325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff43815e2f0 msgr2=0x7ff4380cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.329+0000 7ff43e400640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff43815e2f0 0x7ff4380cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.329+0000 7ff43e400640 1 -- 172.21.15.7:0/1412923325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff438162420 msgr2=0x7ff438137d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.329+0000 7ff43e400640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff438162420 0x7ff438137d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.329+0000 7ff43e400640 1 -- 172.21.15.7:0/1412923325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4380aedc0 con 0x7ff4380afe60 2024-04-25T11:22:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.329+0000 7ff43e400640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4380afe60 0x7ff4381382b0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff430009fd0 tx=0x7ff43000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.330+0000 7ff435600640 1 -- 172.21.15.7:0/1412923325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430014420 con 0x7ff4380afe60 2024-04-25T11:22:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.330+0000 7ff435600640 1 -- 172.21.15.7:0/1412923325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff430019070 con 0x7ff4380afe60 2024-04-25T11:22:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.330+0000 7ff435600640 1 -- 172.21.15.7:0/1412923325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430014800 con 0x7ff4380afe60 2024-04-25T11:22:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.330+0000 7ff43ee00640 1 -- 172.21.15.7:0/1412923325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4380af0a0 con 0x7ff4380afe60 2024-04-25T11:22:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.330+0000 7ff43ee00640 1 -- 172.21.15.7:0/1412923325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4380af660 con 0x7ff4380afe60 2024-04-25T11:22:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.331+0000 7ff43ee00640 1 -- 172.21.15.7:0/1412923325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43815b690 con 0x7ff4380afe60 2024-04-25T11:22:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.332+0000 7ff435600640 1 -- 172.21.15.7:0/1412923325 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff4300149a0 con 0x7ff4380afe60 2024-04-25T11:22:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.333+0000 7ff435600640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4180744b0 0x7ff4180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.333+0000 7ff435600640 1 -- 172.21.15.7:0/1412923325 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7ff430095c20 con 0x7ff4380afe60 2024-04-25T11:22:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.334+0000 7ff43da00640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4180744b0 0x7ff4180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.334+0000 7ff43da00640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4180744b0 0x7ff4180769c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff42c007fd0 tx=0x7ff42c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.338+0000 7ff435600640 1 -- 172.21.15.7:0/1412923325 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff43815b690 con 0x7ff4380afe60 2024-04-25T11:22:40.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.586+0000 7ff43ee00640 1 -- 172.21.15.7:0/1412923325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff4380aeb10 con 0x7ff4380afe60 2024-04-25T11:22:40.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.587+0000 7ff435600640 1 -- 172.21.15.7:0/1412923325 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v116) ==== 53+0+73 (secure 0 0 0) 0x7ff430068030 con 0x7ff4380afe60 2024-04-25T11:22:40.582 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:22:40.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 -- 172.21.15.7:0/1412923325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4180744b0 msgr2=0x7ff4180769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:40.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4180744b0 0x7ff4180769c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff42c007fd0 tx=0x7ff42c009040 comp rx=0 tx=0).stop 2024-04-25T11:22:40.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 -- 172.21.15.7:0/1412923325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4380afe60 msgr2=0x7ff4381382b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:40.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4380afe60 0x7ff4381382b0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff430009fd0 tx=0x7ff43000e4b0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 -- 172.21.15.7:0/1412923325 shutdown_connections 2024-04-25T11:22:40.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff4180744b0 0x7ff4180769c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4380afe60 0x7ff4381382b0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff438162420 0x7ff438137d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 --2- 172.21.15.7:0/1412923325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff43815e2f0 0x7ff4380cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:40.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 -- 172.21.15.7:0/1412923325 >> 172.21.15.7:0/1412923325 conn(0x7ff438073090 msgr2=0x7ff4380684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:40.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 -- 172.21.15.7:0/1412923325 shutdown_connections 2024-04-25T11:22:40.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40.590+0000 7ff43ee00640 1 -- 172.21.15.7:0/1412923325 wait complete. 2024-04-25T11:22:40.611 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:22:40.611 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:22:40.611 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-04-25T11:22:40.611 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T11:22:40.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T11:22:40.611 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:22:40.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:22:40.611 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:22:40.611 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:22:40.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:40.612 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-04-25T11:22:40.667 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:22:40.667 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:22:40.667 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:22:40.667 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 41814sec preferred_lft 41814sec 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link noprefixroute 2024-04-25T11:22:40.668 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::6098:c7ff:fe8a:4a30/64 scope link 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 66:6a:35:7b:0a:d0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::646a:35ff:fe7b:ad0/64 scope link 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 46:65:c4:62:f7:5f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:22:40.669 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::4465:c4ff:fe62:f75f/64 scope link 2024-04-25T11:22:40.670 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:40.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:40.671 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:22:40.745 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-04-25T11:22:40.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:40.749 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-04-25T11:22:40.807 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-04-25T11:22:40.807 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:22:40.807 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T11:22:40.807 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T11:22:40.807 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:22:40.808 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:22:40.808 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-04-25T11:22:40.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T11:22:40.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T11:22:40.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T11:22:40.808 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-04-25T11:22:40.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:22:41.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:22:41.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:41.010 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:22:41.055 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:22:41.055 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:22:41.055 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:22:41.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:41.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-04-25T11:22:41.187 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:22:41.187 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T11:22:41.244 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T11:22:41.244 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:22:41.244 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:22:41.301 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-04-25T11:22:41.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:41.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:22:41.516 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:22:41.516 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:22:41.560 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:22:41.560 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:22:41.561 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:22:41.561 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:22:41.561 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:22:41.563 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T11:22:41.563 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:22:41.563 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:22:41.563 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:22:41.563 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:22:41.567 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:22:41.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:41.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T11:22:41.651 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:22:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:22:41.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:22:41.798 INFO:teuthology.orchestra.run.smithi007.stdout:client7696 2024-04-25T11:22:41.799 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:22:41.799 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:41.846 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T11:22:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.188+0000 7f0f46000640 1 Processor -- start 2024-04-25T11:22:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.188+0000 7f0f46000640 1 -- start start 2024-04-25T11:22:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.191+0000 7f0f46000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4015f180 0x7f0f40161c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.191+0000 7f0f46000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f40162700 con 0x7f0f4006dc90 2024-04-25T11:22:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.191+0000 7f0f46000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f401629e0 con 0x7f0f401621a0 2024-04-25T11:22:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.191+0000 7f0f46000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4015f590 con 0x7f0f4015f180 2024-04-25T11:22:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.191+0000 7f0f3fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0f4006dc90 0x7f0f4006f1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33784/0 (socket says 172.21.15.7:33784) 2024-04-25T11:22:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.191+0000 7f0f3fe00640 1 -- 172.21.15.7:0/3346597596 learned_addr learned my addr 172.21.15.7:0/3346597596 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:42.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.191+0000 7f0f3f400640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4015f180 0x7f0f40161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:42.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1000969830 0 0) 0x7f0f40162700 con 0x7f0f4006dc90 2024-04-25T11:22:42.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f24003520 con 0x7f0f4006dc90 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1421356986 0 0) 0x7f0f24003520 con 0x7f0f4006dc90 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4015f180 msgr2=0x7f0f40161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4015f180 0x7f0f40161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 >> v1:172.21.15.87:6789/0 conn(0x7f0f401621a0 legacy=0x7f0f4006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4015e980 con 0x7f0f4006dc90 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f0f380019f0 con 0x7f0f4006dc90 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 >> v1:172.21.15.7:6789/0 conn(0x7f0f4006dc90 legacy=0x7f0f4006f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 0x7f0f24003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f24004800 0x7f0f24004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4015f180 0x7f0f2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4015f590 con 0x7f0f240039e0 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0f240054e0 con 0x7f0f24004800 2024-04-25T11:22:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.192+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2400bf00 con 0x7f0f4015f180 2024-04-25T11:22:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.193+0000 7f0f3f400640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 0x7f0f24003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.193+0000 7f0f36a00640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f24004800 0x7f0f24004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.193+0000 7f0f3fe00640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4015f180 0x7f0f2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.193+0000 7f0f3f400640 1 -- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4015f180 msgr2=0x7f0f2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.193+0000 7f0f3f400640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4015f180 0x7f0f2400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.193+0000 7f0f3f400640 1 -- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f24004800 msgr2=0x7f0f24004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.193+0000 7f0f3f400640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f24004800 0x7f0f24004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.193+0000 7f0f3f400640 1 -- 172.21.15.7:0/3346597596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f24003520 con 0x7f0f240039e0 2024-04-25T11:22:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.193+0000 7f0f3fe00640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4015f180 0x7f0f2400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.193+0000 7f0f36a00640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f24004800 0x7f0f24004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:42.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.194+0000 7f0f3f400640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 0x7f0f24003e50 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009a90 tx=0x7f0f24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b9163d51e4afec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:42.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.195+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30036070 con 0x7f0f240039e0 2024-04-25T11:22:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.195+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f30031400 con 0x7f0f240039e0 2024-04-25T11:22:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.195+0000 7f0f3ea00640 1 -- 172.21.15.7:0/3346597596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30009690 con 0x7f0f240039e0 2024-04-25T11:22:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.195+0000 7f0f46000640 1 -- 172.21.15.7:0/3346597596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 msgr2=0x7f0f24003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.195+0000 7f0f46000640 1 --2- 172.21.15.7:0/3346597596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 0x7f0f24003e50 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009a90 tx=0x7f0f24005830 comp rx=0 tx=0).stop 2024-04-25T11:22:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.195+0000 7f0f3fe00640 1 -- 172.21.15.7:0/3346597596 reap_dead start 2024-04-25T11:22:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.195+0000 7f0f46000640 1 -- 172.21.15.7:0/3346597596 shutdown_connections 2024-04-25T11:22:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.195+0000 7f0f46000640 1 -- 172.21.15.7:0/3346597596 >> 172.21.15.7:0/3346597596 conn(0x7f0f400732b0 msgr2=0x7f0f400736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.196+0000 7f0f46000640 1 -- 172.21.15.7:0/3346597596 shutdown_connections 2024-04-25T11:22:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.196+0000 7f0f46000640 1 -- 172.21.15.7:0/3346597596 wait complete. 2024-04-25T11:22:42.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.198+0000 7f0f46000640 1 Processor -- start 2024-04-25T11:22:42.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.198+0000 7f0f46000640 1 -- start start 2024-04-25T11:22:42.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.200+0000 7f0f46000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 0x7f0f401619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:42.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.200+0000 7f0f46000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4009c870 0x7f0f40154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:42.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.200+0000 7f0f46000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f4009d5f0 0x7f0f4009da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:42.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.200+0000 7f0f46000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4015e670 con 0x7f0f240039e0 2024-04-25T11:22:42.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.200+0000 7f0f46000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0f40158ae0 con 0x7f0f4009d5f0 2024-04-25T11:22:42.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.200+0000 7f0f46000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0f400d1310 con 0x7f0f4009c870 2024-04-25T11:22:42.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.200+0000 7f0f3f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 0x7f0f401619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:42.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.200+0000 7f0f3f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 0x7f0f401619b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46832/0 (socket says 172.21.15.7:46832) 2024-04-25T11:22:42.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.200+0000 7f0f3f400640 1 -- 172.21.15.7:0/1488245872 learned_addr learned my addr 172.21.15.7:0/1488245872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:42.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.200+0000 7f0f3fe00640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f4009d5f0 0x7f0f4009da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:42.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.200+0000 7f0f36a00640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4009c870 0x7f0f40154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:42.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.201+0000 7f0f3f400640 1 -- 172.21.15.7:0/1488245872 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4009c870 msgr2=0x7f0f40154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:42.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.201+0000 7f0f3f400640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4009c870 0x7f0f40154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:42.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.201+0000 7f0f3f400640 1 -- 172.21.15.7:0/1488245872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f4009d5f0 msgr2=0x7f0f4009da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:42.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.201+0000 7f0f3f400640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f4009d5f0 0x7f0f4009da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:42.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.201+0000 7f0f3f400640 1 -- 172.21.15.7:0/1488245872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4009df40 con 0x7f0f240039e0 2024-04-25T11:22:42.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.201+0000 7f0f3fe00640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f4009d5f0 0x7f0f4009da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:42.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.201+0000 7f0f3f400640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 0x7f0f401619b0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f0f30000c00 tx=0x7f0f30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:42.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.201+0000 7f0f37e00640 1 -- 172.21.15.7:0/1488245872 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30040020 con 0x7f0f240039e0 2024-04-25T11:22:42.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.201+0000 7f0f37e00640 1 -- 172.21.15.7:0/1488245872 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f3003b680 con 0x7f0f240039e0 2024-04-25T11:22:42.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.201+0000 7f0f46000640 1 -- 172.21.15.7:0/1488245872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f401551b0 con 0x7f0f240039e0 2024-04-25T11:22:42.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.201+0000 7f0f37e00640 1 -- 172.21.15.7:0/1488245872 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30036040 con 0x7f0f240039e0 2024-04-25T11:22:42.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.202+0000 7f0f46000640 1 -- 172.21.15.7:0/1488245872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f40160610 con 0x7f0f240039e0 2024-04-25T11:22:42.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.203+0000 7f0f46000640 1 -- 172.21.15.7:0/1488245872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f40050740 con 0x7f0f240039e0 2024-04-25T11:22:42.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.204+0000 7f0f37e00640 1 -- 172.21.15.7:0/1488245872 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0f30043020 con 0x7f0f240039e0 2024-04-25T11:22:42.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.205+0000 7f0f37e00640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f1c0744b0 0x7f0f1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:42.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.205+0000 7f0f37e00640 1 -- 172.21.15.7:0/1488245872 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f0f300b5920 con 0x7f0f240039e0 2024-04-25T11:22:42.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.205+0000 7f0f36a00640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f1c0744b0 0x7f0f1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:42.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.206+0000 7f0f36a00640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f1c0744b0 0x7f0f1c0769c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0f400b0c60 tx=0x7f0f28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:42.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.210+0000 7f0f37e00640 1 -- 172.21.15.7:0/1488245872 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0f3004a070 con 0x7f0f240039e0 2024-04-25T11:22:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.464+0000 7f0f46000640 1 -- 172.21.15.7:0/1488245872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0f40074250 con 0x7f0f240039e0 2024-04-25T11:22:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.464+0000 7f0f37e00640 1 -- 172.21.15.7:0/1488245872 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v116) ==== 53+0+73 (secure 0 0 0) 0x7f0f4009df40 con 0x7f0f240039e0 2024-04-25T11:22:42.459 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T11:22:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 -- 172.21.15.7:0/1488245872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f1c0744b0 msgr2=0x7f0f1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f1c0744b0 0x7f0f1c0769c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0f400b0c60 tx=0x7f0f28009040 comp rx=0 tx=0).stop 2024-04-25T11:22:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 -- 172.21.15.7:0/1488245872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 msgr2=0x7f0f401619b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 0x7f0f401619b0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f0f30000c00 tx=0x7f0f30004730 comp rx=0 tx=0).stop 2024-04-25T11:22:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 -- 172.21.15.7:0/1488245872 shutdown_connections 2024-04-25T11:22:42.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0f1c0744b0 0x7f0f1c0769c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f4009d5f0 0x7f0f4009da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f4009c870 0x7f0f40154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 --2- 172.21.15.7:0/1488245872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f240039e0 0x7f0f401619b0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 -- 172.21.15.7:0/1488245872 >> 172.21.15.7:0/1488245872 conn(0x7f0f400732b0 msgr2=0x7f0f400b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 -- 172.21.15.7:0/1488245872 shutdown_connections 2024-04-25T11:22:42.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:42.467+0000 7f0f46000640 1 -- 172.21.15.7:0/1488245872 wait complete. 2024-04-25T11:22:42.489 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T11:22:42.489 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T11:22:42.489 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T11:22:42.490 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T11:22:42.490 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T11:22:42.490 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T11:22:42.490 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T11:22:42.490 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T11:22:42.490 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:22:42.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:42.490 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T11:22:42.546 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T11:22:42.546 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T11:22:42.546 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T11:22:42.546 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:42.546 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T11:22:42.547 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41813sec preferred_lft 41813sec 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::58f2:6bff:fe8b:8b4c/64 scope link 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 9a:fe:1b:9f:d7:1c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::98fe:1bff:fe9f:d71c/64 scope link 2024-04-25T11:22:42.548 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:42.549 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T11:22:42.549 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 3e:38:a8:c7:ea:00 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:22:42.549 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::3c38:a8ff:fec7:ea00/64 scope link 2024-04-25T11:22:42.549 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T11:22:42.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:42.550 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T11:22:42.625 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T11:22:42.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:42.629 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T11:22:42.681 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T11:22:42.682 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T11:22:42.682 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T11:22:42.682 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T11:22:42.682 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T11:22:42.682 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T11:22:42.682 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T11:22:42.682 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T11:22:42.682 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T11:22:42.682 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T11:22:42.682 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T11:22:42.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:22:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T11:22:42.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:22:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:22:42.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:42.893 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:22:42.938 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:22:42.939 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:22:42.939 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:22:42.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:42.940 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T11:22:43.025 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:22:43.025 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T11:22:43.077 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T11:22:43.077 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T11:22:43.077 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:22:43.127 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-04-25T11:22:43.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:43.127 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:22:43.325 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:22:43.325 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:22:43.369 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-04-25T11:22:43.370 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-04-25T11:22:43.370 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.87:3300,172.21.15.87:3301" 2024-04-25T11:22:43.370 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 1@4ee91e9b-5e2d-4052-ab33-0a56271c8c7f.cephfs=/ 2024-04-25T11:22:43.370 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.87:3300/172.21.15.87:3301" will pass to kernel 2024-04-25T11:22:43.371 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T11:22:43.371 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-04-25T11:22:43.371 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T11:22:43.372 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T11:22:43.372 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T11:22:43.375 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T11:22:43.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:43.375 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T11:22:43.452 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T11:22:43.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:22:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T11:22:43.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T11:22:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T11:22:43.565 INFO:teuthology.orchestra.run.smithi087.stdout:client7702 2024-04-25T11:22:43.565 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:22:43.565 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:43.615 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T11:22:43.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:22:43.739 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T11:22:43.845 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T11:22:43.845 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:22:43.846 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:22:43.846 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:43.906 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:22:43.907 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:43.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:43.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:22:44.015 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:22:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.250+0000 7f2608c00640 1 Processor -- start 2024-04-25T11:22:44.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.250+0000 7f2608c00640 1 -- start start 2024-04-25T11:22:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.252+0000 7f2608c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f260415ef90 0x7f260409f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.252+0000 7f2608c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f260415bc40 0x7f26040aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.253+0000 7f2608c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f260415f4e0 con 0x7f26040af370 2024-04-25T11:22:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.253+0000 7f2608c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2604071190 con 0x7f260415ef90 2024-04-25T11:22:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.253+0000 7f2608c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f26040af780 con 0x7f260415bc40 2024-04-25T11:22:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.253+0000 7f2602800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f26040af370 0x7f2604161b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33794/0 (socket says 172.21.15.7:33794) 2024-04-25T11:22:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.253+0000 7f2602800640 1 -- 172.21.15.7:0/7554061 learned_addr learned my addr 172.21.15.7:0/7554061 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.253+0000 7f2601400640 1 --2- 172.21.15.7:0/7554061 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f260415bc40 0x7f26040aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.253+0000 7f2601e00640 1 --2- 172.21.15.7:0/7554061 >> v2:172.21.15.87:3300/0 conn(0x7f260415ef90 0x7f260409f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1983224429 0 0) 0x7f260415f4e0 con 0x7f26040af370 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25e8003520 con 0x7f26040af370 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2106492547 0 0) 0x7f25e8003520 con 0x7f26040af370 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f260415bc40 msgr2=0x7f26040aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 --2- 172.21.15.7:0/7554061 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f260415bc40 0x7f26040aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 >> v2:172.21.15.87:3300/0 conn(0x7f260415ef90 msgr2=0x7f260409f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 --2- 172.21.15.7:0/7554061 >> v2:172.21.15.87:3300/0 conn(0x7f260415ef90 0x7f260409f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26040b0560 con 0x7f26040af370 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f25fc001670 con 0x7f26040af370 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 >> v1:172.21.15.7:6789/0 conn(0x7f26040af370 legacy=0x7f2604161b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 --2- 172.21.15.7:0/7554061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80039b0 0x7f25e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 --2- 172.21.15.7:0/7554061 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f260415bc40 0x7f25e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 --2- 172.21.15.7:0/7554061 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25e8004d40 0x7f25e8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2604071190 con 0x7f25e80039b0 2024-04-25T11:22:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f26040af780 con 0x7f25e8004d40 2024-04-25T11:22:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.254+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f25e800ba40 con 0x7f260415bc40 2024-04-25T11:22:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.255+0000 7f2601e00640 1 --2- 172.21.15.7:0/7554061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80039b0 0x7f25e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.255+0000 7f2601e00640 1 -- 172.21.15.7:0/7554061 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f260415bc40 msgr2=0x7f25e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.255+0000 7f2601400640 1 --2- 172.21.15.7:0/7554061 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f260415bc40 0x7f25e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.255+0000 7f2601e00640 1 --2- 172.21.15.7:0/7554061 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f260415bc40 0x7f25e8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.255+0000 7f2601e00640 1 -- 172.21.15.7:0/7554061 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25e8004d40 msgr2=0x7f25e8005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.255+0000 7f2601e00640 1 --2- 172.21.15.7:0/7554061 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25e8004d40 0x7f25e8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.255+0000 7f2601e00640 1 -- 172.21.15.7:0/7554061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e8003520 con 0x7f25e80039b0 2024-04-25T11:22:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.256+0000 7f2601e00640 1 --2- 172.21.15.7:0/7554061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80039b0 0x7f25e8003e20 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009ac0 tx=0x7f25e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c899cf76b1a2fa02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.256+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4036070 con 0x7f25e80039b0 2024-04-25T11:22:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.256+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f4031400 con 0x7f25e80039b0 2024-04-25T11:22:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.256+0000 7f2600a00640 1 -- 172.21.15.7:0/7554061 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4009690 con 0x7f25e80039b0 2024-04-25T11:22:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.257+0000 7f2608c00640 1 -- 172.21.15.7:0/7554061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80039b0 msgr2=0x7f25e8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.257+0000 7f2608c00640 1 --2- 172.21.15.7:0/7554061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25e80039b0 0x7f25e8003e20 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009ac0 tx=0x7f25e8005830 comp rx=0 tx=0).stop 2024-04-25T11:22:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.257+0000 7f2602800640 1 -- 172.21.15.7:0/7554061 reap_dead start 2024-04-25T11:22:44.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.257+0000 7f2608c00640 1 -- 172.21.15.7:0/7554061 shutdown_connections 2024-04-25T11:22:44.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.257+0000 7f2608c00640 1 -- 172.21.15.7:0/7554061 >> 172.21.15.7:0/7554061 conn(0x7f2604074200 msgr2=0x7f260406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:44.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.257+0000 7f2608c00640 1 -- 172.21.15.7:0/7554061 shutdown_connections 2024-04-25T11:22:44.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.257+0000 7f2608c00640 1 -- 172.21.15.7:0/7554061 wait complete. 2024-04-25T11:22:44.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.260+0000 7f2608c00640 1 Processor -- start 2024-04-25T11:22:44.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.260+0000 7f2608c00640 1 -- start start 2024-04-25T11:22:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.261+0000 7f2608c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25e80039b0 0x7f26040cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.261+0000 7f2608c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26040b53a0 0x7f26041f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2608c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26041f9940 0x7f26041f9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2608c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f26040b0e00 con 0x7f26040b53a0 2024-04-25T11:22:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2608c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f26040b0b20 con 0x7f26041f9940 2024-04-25T11:22:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2608c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f26040b1320 con 0x7f25e80039b0 2024-04-25T11:22:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2601400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26040b53a0 0x7f26041f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2601400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26040b53a0 0x7f26041f9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46852/0 (socket says 172.21.15.7:46852) 2024-04-25T11:22:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2601400640 1 -- 172.21.15.7:0/891866874 learned_addr learned my addr 172.21.15.7:0/891866874 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2601e00640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25e80039b0 0x7f26040cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2602800640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26041f9940 0x7f26041f9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2601400640 1 -- 172.21.15.7:0/891866874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25e80039b0 msgr2=0x7f26040cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2601400640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25e80039b0 0x7f26040cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2601400640 1 -- 172.21.15.7:0/891866874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26041f9940 msgr2=0x7f26041f9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2601400640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26041f9940 0x7f26041f9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2601400640 1 -- 172.21.15.7:0/891866874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26041fa600 con 0x7f26040b53a0 2024-04-25T11:22:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.262+0000 7f2602800640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26041f9940 0x7f26041f9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.263+0000 7f2601400640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26040b53a0 0x7f26041f9080 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00f350 tx=0x7f25ec00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.263+0000 7f25fa000640 1 -- 172.21.15.7:0/891866874 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec01c070 con 0x7f26040b53a0 2024-04-25T11:22:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.263+0000 7f25fa000640 1 -- 172.21.15.7:0/891866874 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ec017620 con 0x7f26040b53a0 2024-04-25T11:22:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.263+0000 7f25fa000640 1 -- 172.21.15.7:0/891866874 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec017a10 con 0x7f26040b53a0 2024-04-25T11:22:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.263+0000 7f2608c00640 1 -- 172.21.15.7:0/891866874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26040b48c0 con 0x7f26040b53a0 2024-04-25T11:22:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.263+0000 7f2608c00640 1 -- 172.21.15.7:0/891866874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26040cb530 con 0x7f26040b53a0 2024-04-25T11:22:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.265+0000 7f2608c00640 1 -- 172.21.15.7:0/891866874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26041fa370 con 0x7f26040b53a0 2024-04-25T11:22:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.266+0000 7f25fa000640 1 -- 172.21.15.7:0/891866874 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f25ec005d10 con 0x7f26040b53a0 2024-04-25T11:22:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.267+0000 7f25fa000640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25e00744b0 0x7f25e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.267+0000 7f25fa000640 1 -- 172.21.15.7:0/891866874 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f25ec0959a0 con 0x7f26040b53a0 2024-04-25T11:22:44.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.267+0000 7f2601e00640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25e00744b0 0x7f25e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.268+0000 7f2601e00640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25e00744b0 0x7f25e00769c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f25f4033000 tx=0x7f25f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:44.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.272+0000 7f25fa000640 1 -- 172.21.15.7:0/891866874 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f25ec012070 con 0x7f26040b53a0 2024-04-25T11:22:44.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.475+0000 7f2608c00640 1 -- 172.21.15.7:0/891866874 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26040709a0 con 0x7f26040b53a0 2024-04-25T11:22:44.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.478+0000 7f25fa000640 1 -- 172.21.15.7:0/891866874 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16788 (secure 0 0 0) 0x7f26040709a0 con 0x7f26040b53a0 2024-04-25T11:22:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:44.473 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:33.767424+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:22:30.472409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T11:22:31.299177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.481+0000 7f2608c00640 1 -- 172.21.15.7:0/891866874 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25e00744b0 msgr2=0x7f25e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.481+0000 7f2608c00640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25e00744b0 0x7f25e00769c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f25f4033000 tx=0x7f25f4008030 comp rx=0 tx=0).stop 2024-04-25T11:22:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.481+0000 7f2608c00640 1 -- 172.21.15.7:0/891866874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26040b53a0 msgr2=0x7f26041f9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.481+0000 7f2608c00640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26040b53a0 0x7f26041f9080 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00f350 tx=0x7f25ec00f810 comp rx=0 tx=0).stop 2024-04-25T11:22:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.481+0000 7f2608c00640 1 -- 172.21.15.7:0/891866874 shutdown_connections 2024-04-25T11:22:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.481+0000 7f2608c00640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f25e00744b0 0x7f25e00769c0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.481+0000 7f2608c00640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26041f9940 0x7f26041f9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.481+0000 7f2608c00640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26040b53a0 0x7f26041f9080 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.481+0000 7f2608c00640 1 --2- 172.21.15.7:0/891866874 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25e80039b0 0x7f26040cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.481+0000 7f2608c00640 1 -- 172.21.15.7:0/891866874 >> 172.21.15.7:0/891866874 conn(0x7f2604074200 msgr2=0x7f26040d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.482+0000 7f2608c00640 1 -- 172.21.15.7:0/891866874 shutdown_connections 2024-04-25T11:22:44.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.482+0000 7f2608c00640 1 -- 172.21.15.7:0/891866874 wait complete. 2024-04-25T11:22:44.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:22:30.472409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:22:31.299177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:22:44.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:22:44.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.804+0000 7f080fe00640 1 Processor -- start 2024-04-25T11:22:44.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.804+0000 7f080fe00640 1 -- start start 2024-04-25T11:22:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.806+0000 7f080fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08100cea20 0x7f0810161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.806+0000 7f080fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0810071fb0 0x7f08100c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.807+0000 7f080fe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0810161580 con 0x7f08100b19d0 2024-04-25T11:22:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.807+0000 7f080fe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0810160bb0 con 0x7f0810071fb0 2024-04-25T11:22:44.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.807+0000 7f080fe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f08100d1e60 con 0x7f08100cea20 2024-04-25T11:22:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.807+0000 7f0807e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f08100b19d0 0x7f08100718a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33808/0 (socket says 172.21.15.7:33808) 2024-04-25T11:22:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.807+0000 7f0807e00640 1 -- 172.21.15.7:0/4015831719 learned_addr learned my addr 172.21.15.7:0/4015831719 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.807+0000 7f080f400640 1 --2- 172.21.15.7:0/4015831719 >> v2:172.21.15.7:3300/0 conn(0x7f0810071fb0 0x7f08100c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.807+0000 7f080ea00640 1 --2- 172.21.15.7:0/4015831719 >> v2:172.21.15.87:3300/0 conn(0x7f08100cea20 0x7f0810161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.808+0000 7f080e000640 1 -- 172.21.15.7:0/4015831719 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2644631634 0 0) 0x7f0810161580 con 0x7f08100b19d0 2024-04-25T11:22:44.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.808+0000 7f080e000640 1 -- 172.21.15.7:0/4015831719 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07f4003520 con 0x7f08100b19d0 2024-04-25T11:22:44.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.808+0000 7f080f400640 1 -- 172.21.15.7:0/4015831719 >> v2:172.21.15.87:3300/0 conn(0x7f08100cea20 msgr2=0x7f0810161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:44.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.808+0000 7f080f400640 1 --2- 172.21.15.7:0/4015831719 >> v2:172.21.15.87:3300/0 conn(0x7f08100cea20 0x7f0810161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.808+0000 7f080f400640 1 -- 172.21.15.7:0/4015831719 >> v1:172.21.15.7:6789/0 conn(0x7f08100b19d0 legacy=0x7f08100718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:44.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.808+0000 7f080f400640 1 -- 172.21.15.7:0/4015831719 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08100d2140 con 0x7f0810071fb0 2024-04-25T11:22:44.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.809+0000 7f080f400640 1 --2- 172.21.15.7:0/4015831719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810071fb0 0x7f08100c7500 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f08100b52e0 tx=0x7f080802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05fa3c7cc771fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.809+0000 7f080e000640 1 -- 172.21.15.7:0/4015831719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808007740 con 0x7f0810071fb0 2024-04-25T11:22:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.809+0000 7f080e000640 1 -- 172.21.15.7:0/4015831719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08080078e0 con 0x7f0810071fb0 2024-04-25T11:22:44.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.809+0000 7f080e000640 1 -- 172.21.15.7:0/4015831719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808007cb0 con 0x7f0810071fb0 2024-04-25T11:22:44.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.810+0000 7f080fe00640 1 -- 172.21.15.7:0/4015831719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810071fb0 msgr2=0x7f08100c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:44.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.810+0000 7f080fe00640 1 --2- 172.21.15.7:0/4015831719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810071fb0 0x7f08100c7500 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f08100b52e0 tx=0x7f080802a680 comp rx=0 tx=0).stop 2024-04-25T11:22:44.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.810+0000 7f080fe00640 1 -- 172.21.15.7:0/4015831719 shutdown_connections 2024-04-25T11:22:44.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.810+0000 7f080fe00640 1 --2- 172.21.15.7:0/4015831719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0810071fb0 0x7f08100c7500 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.810+0000 7f080fe00640 1 --2- 172.21.15.7:0/4015831719 >> v2:172.21.15.87:3300/0 conn(0x7f08100cea20 0x7f0810161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.810+0000 7f080fe00640 1 -- 172.21.15.7:0/4015831719 >> 172.21.15.7:0/4015831719 conn(0x7f08100d2660 msgr2=0x7f08100d2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:44.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.810+0000 7f080fe00640 1 -- 172.21.15.7:0/4015831719 shutdown_connections 2024-04-25T11:22:44.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.810+0000 7f080fe00640 1 -- 172.21.15.7:0/4015831719 wait complete. 2024-04-25T11:22:44.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.813+0000 7f080fe00640 1 Processor -- start 2024-04-25T11:22:44.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.813+0000 7f080fe00640 1 -- start start 2024-04-25T11:22:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.814+0000 7f080fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08100b19d0 0x7f08101f9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.814+0000 7f080fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08100cea20 0x7f0810157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f080fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08101576d0 0x7f08100b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f080fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08100c8e80 con 0x7f08100cea20 2024-04-25T11:22:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f080fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f08100cee30 con 0x7f08101576d0 2024-04-25T11:22:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f080fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f08100c9000 con 0x7f08100b19d0 2024-04-25T11:22:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f0807e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08100cea20 0x7f0810157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f0807e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08100cea20 0x7f0810157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46866/0 (socket says 172.21.15.7:46866) 2024-04-25T11:22:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f0807e00640 1 -- 172.21.15.7:0/1249726619 learned_addr learned my addr 172.21.15.7:0/1249726619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f080ea00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08100b19d0 0x7f08101f9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f080f400640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08101576d0 0x7f08100b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f0807e00640 1 -- 172.21.15.7:0/1249726619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08100b19d0 msgr2=0x7f08101f9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f0807e00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08100b19d0 0x7f08101f9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f0807e00640 1 -- 172.21.15.7:0/1249726619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08101576d0 msgr2=0x7f08100b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f0807e00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08101576d0 0x7f08100b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f0807e00640 1 -- 172.21.15.7:0/1249726619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08101f6190 con 0x7f08100cea20 2024-04-25T11:22:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.815+0000 7f080ea00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08100b19d0 0x7f08101f9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.816+0000 7f0807e00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08100cea20 0x7f0810157190 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f07f800c390 tx=0x7f07f800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.816+0000 7f0806a00640 1 -- 172.21.15.7:0/1249726619 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8017070 con 0x7f08100cea20 2024-04-25T11:22:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.816+0000 7f0806a00640 1 -- 172.21.15.7:0/1249726619 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f8005d00 con 0x7f08100cea20 2024-04-25T11:22:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.816+0000 7f080fe00640 1 -- 172.21.15.7:0/1249726619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08101f6450 con 0x7f08100cea20 2024-04-25T11:22:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.816+0000 7f0806a00640 1 -- 172.21.15.7:0/1249726619 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8007310 con 0x7f08100cea20 2024-04-25T11:22:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.816+0000 7f080fe00640 1 -- 172.21.15.7:0/1249726619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08101f6a10 con 0x7f08100cea20 2024-04-25T11:22:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.818+0000 7f080fe00640 1 -- 172.21.15.7:0/1249726619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0810050740 con 0x7f08100cea20 2024-04-25T11:22:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.819+0000 7f0806a00640 1 -- 172.21.15.7:0/1249726619 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f07f801a070 con 0x7f08100cea20 2024-04-25T11:22:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.820+0000 7f0806a00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07ec0744b0 0x7f07ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.820+0000 7f0806a00640 1 -- 172.21.15.7:0/1249726619 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f07f80947b0 con 0x7f08100cea20 2024-04-25T11:22:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.820+0000 7f080ea00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07ec0744b0 0x7f07ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.821+0000 7f080ea00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07ec0744b0 0x7f07ec0769c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f08100d18b0 tx=0x7f0800008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:44.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:44.825+0000 7f0806a00640 1 -- 172.21.15.7:0/1249726619 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f07f805edb0 con 0x7f08100cea20 2024-04-25T11:22:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.037+0000 7f080fe00640 1 -- 172.21.15.7:0/1249726619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08100c8b10 con 0x7f08100cea20 2024-04-25T11:22:45.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.038+0000 7f0806a00640 1 -- 172.21.15.7:0/1249726619 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7f07f805e6b0 con 0x7f08100cea20 2024-04-25T11:22:45.033 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.041+0000 7f080fe00640 1 -- 172.21.15.7:0/1249726619 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07ec0744b0 msgr2=0x7f07ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.041+0000 7f080fe00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07ec0744b0 0x7f07ec0769c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f08100d18b0 tx=0x7f0800008040 comp rx=0 tx=0).stop 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.041+0000 7f080fe00640 1 -- 172.21.15.7:0/1249726619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08100cea20 msgr2=0x7f0810157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.041+0000 7f080fe00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08100cea20 0x7f0810157190 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f07f800c390 tx=0x7f07f800c850 comp rx=0 tx=0).stop 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.042+0000 7f080fe00640 1 -- 172.21.15.7:0/1249726619 shutdown_connections 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.042+0000 7f080fe00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f07ec0744b0 0x7f07ec0769c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.042+0000 7f080fe00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08101576d0 0x7f08100b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.042+0000 7f080fe00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08100cea20 0x7f0810157190 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.042+0000 7f080fe00640 1 --2- 172.21.15.7:0/1249726619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08100b19d0 0x7f08101f9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.042+0000 7f080fe00640 1 -- 172.21.15.7:0/1249726619 >> 172.21.15.7:0/1249726619 conn(0x7f08100d2660 msgr2=0x7f0810162aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.042+0000 7f080fe00640 1 -- 172.21.15.7:0/1249726619 shutdown_connections 2024-04-25T11:22:45.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.042+0000 7f080fe00640 1 -- 172.21.15.7:0/1249726619 wait complete. 2024-04-25T11:22:45.064 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:22:45.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.364+0000 7f7b0b800640 1 Processor -- start 2024-04-25T11:22:45.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.365+0000 7f7b0b800640 1 -- start start 2024-04-25T11:22:45.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.366+0000 7f7b0b800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b0409cd90 0x7f7b0406e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.366+0000 7f7b0b800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7b0406e840 0x7f7b040b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.366+0000 7f7b0b800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b040b3810 0x7f7b040b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.366+0000 7f7b0b800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7b0406df50 con 0x7f7b040b3810 2024-04-25T11:22:45.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.367+0000 7f7b0b800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7b0409d1a0 con 0x7f7b0406e840 2024-04-25T11:22:45.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.367+0000 7f7b0b800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7b040d4870 con 0x7f7b0409cd90 2024-04-25T11:22:45.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.367+0000 7f7b09800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b040b3810 0x7f7b040b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.367+0000 7f7b09800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b040b3810 0x7f7b040b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46868/0 (socket says 172.21.15.7:46868) 2024-04-25T11:22:45.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.367+0000 7f7b09800640 1 -- 172.21.15.7:0/1653609889 learned_addr learned my addr 172.21.15.7:0/1653609889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:45.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.367+0000 7f7b08e00640 1 --2- 172.21.15.7:0/1653609889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b0409cd90 0x7f7b0406e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.367+0000 7f7b03e00640 1 --2- 172.21.15.7:0/1653609889 >> v2:172.21.15.87:3300/0 conn(0x7f7b0406e840 0x7f7b040b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.368+0000 7f7b09800640 1 -- 172.21.15.7:0/1653609889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b0409cd90 msgr2=0x7f7b0406e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:45.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.368+0000 7f7b09800640 1 --2- 172.21.15.7:0/1653609889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b0409cd90 0x7f7b0406e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.368+0000 7f7b09800640 1 -- 172.21.15.7:0/1653609889 >> v2:172.21.15.87:3300/0 conn(0x7f7b0406e840 msgr2=0x7f7b040b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:45.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.368+0000 7f7b09800640 1 --2- 172.21.15.7:0/1653609889 >> v2:172.21.15.87:3300/0 conn(0x7f7b0406e840 0x7f7b040b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.368+0000 7f7b09800640 1 -- 172.21.15.7:0/1653609889 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0406d960 con 0x7f7b040b3810 2024-04-25T11:22:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.370+0000 7f7b09800640 1 --2- 172.21.15.7:0/1653609889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040b3810 0x7f7b040b3c20 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f7b040708e0 tx=0x7f7af402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d211d337d858392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.370+0000 7f7b03400640 1 -- 172.21.15.7:0/1653609889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4037070 con 0x7f7b040b3810 2024-04-25T11:22:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.370+0000 7f7b03400640 1 -- 172.21.15.7:0/1653609889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af4032440 con 0x7f7b040b3810 2024-04-25T11:22:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.370+0000 7f7b03400640 1 -- 172.21.15.7:0/1653609889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4032810 con 0x7f7b040b3810 2024-04-25T11:22:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.371+0000 7f7b0b800640 1 -- 172.21.15.7:0/1653609889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040b3810 msgr2=0x7f7b040b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.371+0000 7f7b0b800640 1 --2- 172.21.15.7:0/1653609889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040b3810 0x7f7b040b3c20 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f7b040708e0 tx=0x7f7af402a140 comp rx=0 tx=0).stop 2024-04-25T11:22:45.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.371+0000 7f7b0b800640 1 -- 172.21.15.7:0/1653609889 shutdown_connections 2024-04-25T11:22:45.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.371+0000 7f7b0b800640 1 --2- 172.21.15.7:0/1653609889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040b3810 0x7f7b040b3c20 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.371+0000 7f7b0b800640 1 --2- 172.21.15.7:0/1653609889 >> v2:172.21.15.87:3300/0 conn(0x7f7b0406e840 0x7f7b040b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.371+0000 7f7b0b800640 1 --2- 172.21.15.7:0/1653609889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b0409cd90 0x7f7b0406e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.371+0000 7f7b0b800640 1 -- 172.21.15.7:0/1653609889 >> 172.21.15.7:0/1653609889 conn(0x7f7b040d03f0 msgr2=0x7f7b040d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:45.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.371+0000 7f7b0b800640 1 -- 172.21.15.7:0/1653609889 shutdown_connections 2024-04-25T11:22:45.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.371+0000 7f7b0b800640 1 -- 172.21.15.7:0/1653609889 wait complete. 2024-04-25T11:22:45.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.374+0000 7f7b0b800640 1 Processor -- start 2024-04-25T11:22:45.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.374+0000 7f7b0b800640 1 -- start start 2024-04-25T11:22:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.375+0000 7f7b0b800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b0406e840 0x7f7b040cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.375+0000 7f7b0b800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b0409cd90 0x7f7b041dfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.375+0000 7f7b0b800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040b3810 0x7f7b040cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.375+0000 7f7b0b800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b040b9180 con 0x7f7b040b3810 2024-04-25T11:22:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.375+0000 7f7b0b800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0406dc70 con 0x7f7b0406e840 2024-04-25T11:22:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.375+0000 7f7b0b800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7b040b9460 con 0x7f7b0409cd90 2024-04-25T11:22:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.376+0000 7f7b09800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040b3810 0x7f7b040cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.376+0000 7f7b09800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040b3810 0x7f7b040cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46870/0 (socket says 172.21.15.7:46870) 2024-04-25T11:22:45.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.376+0000 7f7b09800640 1 -- 172.21.15.7:0/873349193 learned_addr learned my addr 172.21.15.7:0/873349193 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:45.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.376+0000 7f7b03e00640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b0409cd90 0x7f7b041dfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.376+0000 7f7b08e00640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b0406e840 0x7f7b040cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.376+0000 7f7b09800640 1 -- 172.21.15.7:0/873349193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b0409cd90 msgr2=0x7f7b041dfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:45.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.376+0000 7f7b09800640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b0409cd90 0x7f7b041dfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.376+0000 7f7b09800640 1 -- 172.21.15.7:0/873349193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b0406e840 msgr2=0x7f7b040cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:45.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.376+0000 7f7b09800640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b0406e840 0x7f7b040cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.376+0000 7f7b09800640 1 -- 172.21.15.7:0/873349193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b040cece0 con 0x7f7b040b3810 2024-04-25T11:22:45.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.376+0000 7f7b08e00640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b0406e840 0x7f7b040cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:45.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.377+0000 7f7b09800640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040b3810 0x7f7b040cad00 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f7af40090b0 tx=0x7f7af40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:45.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.377+0000 7f7b00c00640 1 -- 172.21.15.7:0/873349193 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4041020 con 0x7f7b040b3810 2024-04-25T11:22:45.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.377+0000 7f7b00c00640 1 -- 172.21.15.7:0/873349193 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7af403d640 con 0x7f7b040b3810 2024-04-25T11:22:45.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.377+0000 7f7b00c00640 1 -- 172.21.15.7:0/873349193 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4037040 con 0x7f7b040b3810 2024-04-25T11:22:45.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.377+0000 7f7b0b800640 1 -- 172.21.15.7:0/873349193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b040cefc0 con 0x7f7b040b3810 2024-04-25T11:22:45.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.377+0000 7f7b0b800640 1 -- 172.21.15.7:0/873349193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b040cf580 con 0x7f7b040b3810 2024-04-25T11:22:45.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.379+0000 7f7b0b800640 1 -- 172.21.15.7:0/873349193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b04050740 con 0x7f7b040b3810 2024-04-25T11:22:45.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.380+0000 7f7b00c00640 1 -- 172.21.15.7:0/873349193 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7af4044020 con 0x7f7b040b3810 2024-04-25T11:22:45.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.381+0000 7f7b00c00640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ae00744b0 0x7f7ae00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.381+0000 7f7b00c00640 1 -- 172.21.15.7:0/873349193 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f7af40b5ae0 con 0x7f7b040b3810 2024-04-25T11:22:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.381+0000 7f7b08e00640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ae00744b0 0x7f7ae00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.382+0000 7f7b08e00640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ae00744b0 0x7f7ae00769c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7afc007440 tx=0x7f7afc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.386+0000 7f7b00c00640 1 -- 172.21.15.7:0/873349193 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7af404b030 con 0x7f7b040b3810 2024-04-25T11:22:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.594+0000 7f7b0b800640 1 -- 172.21.15.7:0/873349193 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7b040cea70 con 0x7f7b040b3810 2024-04-25T11:22:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.595+0000 7f7b00c00640 1 -- 172.21.15.7:0/873349193 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) ==== 104+0+11 (secure 0 0 0) 0x7f7af407f9f0 con 0x7f7b040b3810 2024-04-25T11:22:45.590 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:22:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.598+0000 7f7b0b800640 1 -- 172.21.15.7:0/873349193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ae00744b0 msgr2=0x7f7ae00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.598+0000 7f7b0b800640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ae00744b0 0x7f7ae00769c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7afc007440 tx=0x7f7afc00b040 comp rx=0 tx=0).stop 2024-04-25T11:22:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.598+0000 7f7b0b800640 1 -- 172.21.15.7:0/873349193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040b3810 msgr2=0x7f7b040cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.598+0000 7f7b0b800640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040b3810 0x7f7b040cad00 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f7af40090b0 tx=0x7f7af40046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.598+0000 7f7b0b800640 1 -- 172.21.15.7:0/873349193 shutdown_connections 2024-04-25T11:22:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.598+0000 7f7b0b800640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7ae00744b0 0x7f7ae00769c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.598+0000 7f7b0b800640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b040b3810 0x7f7b040cad00 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.599+0000 7f7b0b800640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b0409cd90 0x7f7b041dfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.599+0000 7f7b0b800640 1 --2- 172.21.15.7:0/873349193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b0406e840 0x7f7b040cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.599+0000 7f7b0b800640 1 -- 172.21.15.7:0/873349193 >> 172.21.15.7:0/873349193 conn(0x7f7b040d03f0 msgr2=0x7f7b040d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.599+0000 7f7b0b800640 1 -- 172.21.15.7:0/873349193 shutdown_connections 2024-04-25T11:22:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.599+0000 7f7b0b800640 1 -- 172.21.15.7:0/873349193 wait complete. 2024-04-25T11:22:45.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:22:45.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.916+0000 7f506b000640 1 Processor -- start 2024-04-25T11:22:45.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.916+0000 7f506b000640 1 -- start start 2024-04-25T11:22:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.919+0000 7f506b000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50640cfa80 0x7f50640d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.919+0000 7f506b000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f50640d0800 0x7f506406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.919+0000 7f506b000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50640d0c10 con 0x7f5064070630 2024-04-25T11:22:45.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.919+0000 7f506b000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f50640b0be0 con 0x7f50640d0800 2024-04-25T11:22:45.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.919+0000 7f506b000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f50640cfe90 con 0x7f50640cfa80 2024-04-25T11:22:45.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.919+0000 7f5069c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5064070630 0x7f506415ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33812/0 (socket says 172.21.15.7:33812) 2024-04-25T11:22:45.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.919+0000 7f5069c00640 1 -- 172.21.15.7:0/671275534 learned_addr learned my addr 172.21.15.7:0/671275534 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:45.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.919+0000 7f5069200640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50640cfa80 0x7f50640d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.919+0000 7f506a600640 1 --2- 172.21.15.7:0/671275534 >> v2:172.21.15.87:3300/0 conn(0x7f50640d0800 0x7f506406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.919+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 300020024 0 0) 0x7f5058001110 con 0x7f5064070630 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f504c003520 con 0x7f5064070630 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1559441775 0 0) 0x7f504c003520 con 0x7f5064070630 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50640cfa80 msgr2=0x7f50640d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50640cfa80 0x7f50640d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 >> v2:172.21.15.87:3300/0 conn(0x7f50640d0800 msgr2=0x7f506406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 --2- 172.21.15.7:0/671275534 >> v2:172.21.15.87:3300/0 conn(0x7f50640d0800 0x7f506406eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50640ce9e0 con 0x7f5064070630 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f5058001840 con 0x7f5064070630 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 >> v1:172.21.15.7:6789/0 conn(0x7f5064070630 legacy=0x7f506415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c0039b0 0x7f504c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50640cfa80 0x7f504c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f504c004d40 0x7f504c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50640b0be0 con 0x7f504c0039b0 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f50640cfe90 con 0x7f504c004d40 2024-04-25T11:22:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f504c00b9c0 con 0x7f50640cfa80 2024-04-25T11:22:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.920+0000 7f5069c00640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c0039b0 0x7f504c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.921+0000 7f5069200640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50640cfa80 0x7f504c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.921+0000 7f506a600640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f504c004d40 0x7f504c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.921+0000 7f5069c00640 1 -- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50640cfa80 msgr2=0x7f504c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.921+0000 7f5069c00640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50640cfa80 0x7f504c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.921+0000 7f5069c00640 1 -- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f504c004d40 msgr2=0x7f504c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.921+0000 7f5069c00640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f504c004d40 0x7f504c005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.921+0000 7f5069c00640 1 -- 172.21.15.7:0/671275534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c003520 con 0x7f504c0039b0 2024-04-25T11:22:45.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.921+0000 7f5069200640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50640cfa80 0x7f504c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:45.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.922+0000 7f5069c00640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c0039b0 0x7f504c003e20 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f505800c4c0 tx=0x7f504c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb4614457eb83a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:45.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.922+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505800c0e0 con 0x7f504c0039b0 2024-04-25T11:22:45.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.922+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5058034070 con 0x7f504c0039b0 2024-04-25T11:22:45.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.922+0000 7f5063e00640 1 -- 172.21.15.7:0/671275534 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058005d30 con 0x7f504c0039b0 2024-04-25T11:22:45.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.923+0000 7f506b000640 1 -- 172.21.15.7:0/671275534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c0039b0 msgr2=0x7f504c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:45.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.923+0000 7f506b000640 1 --2- 172.21.15.7:0/671275534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f504c0039b0 0x7f504c003e20 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f505800c4c0 tx=0x7f504c005830 comp rx=0 tx=0).stop 2024-04-25T11:22:45.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.923+0000 7f506a600640 1 -- 172.21.15.7:0/671275534 reap_dead start 2024-04-25T11:22:45.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.923+0000 7f506b000640 1 -- 172.21.15.7:0/671275534 shutdown_connections 2024-04-25T11:22:45.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.923+0000 7f506b000640 1 -- 172.21.15.7:0/671275534 >> 172.21.15.7:0/671275534 conn(0x7f5064074140 msgr2=0x7f506406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:45.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.923+0000 7f506b000640 1 -- 172.21.15.7:0/671275534 shutdown_connections 2024-04-25T11:22:45.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.923+0000 7f506b000640 1 -- 172.21.15.7:0/671275534 wait complete. 2024-04-25T11:22:45.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.927+0000 7f506b000640 1 Processor -- start 2024-04-25T11:22:45.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.927+0000 7f506b000640 1 -- start start 2024-04-25T11:22:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.929+0000 7f506b000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f504c0039b0 0x7f506415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.929+0000 7f506b000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5064147e10 0x7f50641ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.929+0000 7f506b000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50641eaa10 0x7f50641f9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.929+0000 7f506b000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50640cf280 con 0x7f5064147e10 2024-04-25T11:22:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.929+0000 7f506b000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f50640cefa0 con 0x7f50641eaa10 2024-04-25T11:22:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.929+0000 7f506b000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f50640cf560 con 0x7f504c0039b0 2024-04-25T11:22:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.929+0000 7f5069200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5064147e10 0x7f50641ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.929+0000 7f5069200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5064147e10 0x7f50641ea4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46890/0 (socket says 172.21.15.7:46890) 2024-04-25T11:22:45.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.929+0000 7f5069200640 1 -- 172.21.15.7:0/2945674060 learned_addr learned my addr 172.21.15.7:0/2945674060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.930+0000 7f5069c00640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f504c0039b0 0x7f506415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.930+0000 7f506a600640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50641eaa10 0x7f50641f9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.930+0000 7f5069200640 1 -- 172.21.15.7:0/2945674060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f504c0039b0 msgr2=0x7f506415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.930+0000 7f5069200640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f504c0039b0 0x7f506415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.930+0000 7f5069200640 1 -- 172.21.15.7:0/2945674060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50641eaa10 msgr2=0x7f50641f9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.930+0000 7f5069200640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50641eaa10 0x7f50641f9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.930+0000 7f5069200640 1 -- 172.21.15.7:0/2945674060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50641eae20 con 0x7f5064147e10 2024-04-25T11:22:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.930+0000 7f5069200640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5064147e10 0x7f50641ea4d0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f50500108b0 tx=0x7f5050010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.931+0000 7f5061600640 1 -- 172.21.15.7:0/2945674060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505000f040 con 0x7f5064147e10 2024-04-25T11:22:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.931+0000 7f5061600640 1 -- 172.21.15.7:0/2945674060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505000eb60 con 0x7f5064147e10 2024-04-25T11:22:45.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.931+0000 7f5061600640 1 -- 172.21.15.7:0/2945674060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505001c400 con 0x7f5064147e10 2024-04-25T11:22:45.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.931+0000 7f506b000640 1 -- 172.21.15.7:0/2945674060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506415fea0 con 0x7f5064147e10 2024-04-25T11:22:45.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.931+0000 7f506b000640 1 -- 172.21.15.7:0/2945674060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50641e7da0 con 0x7f5064147e10 2024-04-25T11:22:45.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.932+0000 7f506b000640 1 -- 172.21.15.7:0/2945674060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f506415fc10 con 0x7f5064147e10 2024-04-25T11:22:45.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.934+0000 7f5061600640 1 -- 172.21.15.7:0/2945674060 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f505001c5a0 con 0x7f5064147e10 2024-04-25T11:22:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.934+0000 7f5061600640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50440744b0 0x7f50440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.935+0000 7f5061600640 1 -- 172.21.15.7:0/2945674060 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f5050095ad0 con 0x7f5064147e10 2024-04-25T11:22:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.935+0000 7f5069c00640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50440744b0 0x7f50440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:45.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.936+0000 7f5069c00640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50440744b0 0x7f50440769c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f504c004780 tx=0x7f505802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:45.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:45.940+0000 7f5061600640 1 -- 172.21.15.7:0/2945674060 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f50641eae20 con 0x7f5064147e10 2024-04-25T11:22:46.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.146+0000 7f506b000640 1 -- 172.21.15.7:0/2945674060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5064050740 con 0x7f5064147e10 2024-04-25T11:22:46.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.150+0000 7f5061600640 1 -- 172.21.15.7:0/2945674060 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) ==== 100+0+11 (secure 0 0 0) 0x7f5050018030 con 0x7f5064147e10 2024-04-25T11:22:46.145 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:22:46.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 -- 172.21.15.7:0/2945674060 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50440744b0 msgr2=0x7f50440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:46.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50440744b0 0x7f50440769c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f504c004780 tx=0x7f505802e000 comp rx=0 tx=0).stop 2024-04-25T11:22:46.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 -- 172.21.15.7:0/2945674060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5064147e10 msgr2=0x7f50641ea4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:46.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5064147e10 0x7f50641ea4d0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f50500108b0 tx=0x7f5050010d70 comp rx=0 tx=0).stop 2024-04-25T11:22:46.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 -- 172.21.15.7:0/2945674060 shutdown_connections 2024-04-25T11:22:46.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f50440744b0 0x7f50440769c0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50641eaa10 0x7f50641f9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5064147e10 0x7f50641ea4d0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 --2- 172.21.15.7:0/2945674060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f504c0039b0 0x7f506415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 -- 172.21.15.7:0/2945674060 >> 172.21.15.7:0/2945674060 conn(0x7f5064074140 msgr2=0x7f50641574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:46.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 -- 172.21.15.7:0/2945674060 shutdown_connections 2024-04-25T11:22:46.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.153+0000 7f506b000640 1 -- 172.21.15.7:0/2945674060 wait complete. 2024-04-25T11:22:46.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:22:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.441+0000 7fcbf8e00640 1 Processor -- start 2024-04-25T11:22:46.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.441+0000 7fcbf8e00640 1 -- start start 2024-04-25T11:22:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.444+0000 7fcbf8e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcbf4156d90 0x7fcbf4160460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.444+0000 7fcbf8e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf41609a0 0x7fcbf4160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.444+0000 7fcbf8e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbf41612f0 con 0x7fcbf4070520 2024-04-25T11:22:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.444+0000 7fcbf8e00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf406f760 con 0x7fcbf4156d90 2024-04-25T11:22:46.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.444+0000 7fcbf8e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf406fa40 con 0x7fcbf41609a0 2024-04-25T11:22:46.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.444+0000 7fcbf3400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcbf4070520 0x7fcbf40b0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33824/0 (socket says 172.21.15.7:33824) 2024-04-25T11:22:46.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.444+0000 7fcbf3400640 1 -- 172.21.15.7:0/1458708768 learned_addr learned my addr 172.21.15.7:0/1458708768 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.445+0000 7fcbf3e00640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf41609a0 0x7fcbf4160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.445+0000 7fcbf2a00640 1 --2- 172.21.15.7:0/1458708768 >> v2:172.21.15.87:3300/0 conn(0x7fcbf4156d90 0x7fcbf4160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.445+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3263549423 0 0) 0x7fcbf41612f0 con 0x7fcbf4070520 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.445+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbd8003520 con 0x7fcbf4070520 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2391098543 0 0) 0x7fcbd8003520 con 0x7fcbf4070520 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf41609a0 msgr2=0x7fcbf4160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf41609a0 0x7fcbf4160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 >> v2:172.21.15.87:3300/0 conn(0x7fcbf4156d90 msgr2=0x7fcbf4160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 --2- 172.21.15.7:0/1458708768 >> v2:172.21.15.87:3300/0 conn(0x7fcbf4156d90 0x7fcbf4160460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf406fd20 con 0x7fcbf4070520 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fcbe4001670 con 0x7fcbf4070520 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 >> v1:172.21.15.7:6789/0 conn(0x7fcbf4070520 legacy=0x7fcbf40b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd80039b0 0x7fcbd8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf41609a0 0x7fcbd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:46.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd8004d40 0x7fcbd8005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf406f760 con 0x7fcbd8004d40 2024-04-25T11:22:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf406fa40 con 0x7fcbd80039b0 2024-04-25T11:22:46.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd800baa0 con 0x7fcbf41609a0 2024-04-25T11:22:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.446+0000 7fcbf3e00640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd8004d40 0x7fcbd8005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.447+0000 7fcbf3400640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd80039b0 0x7fcbd8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.447+0000 7fcbf2a00640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf41609a0 0x7fcbd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.447+0000 7fcbf3e00640 1 -- 172.21.15.7:0/1458708768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf41609a0 msgr2=0x7fcbd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.447+0000 7fcbf3e00640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf41609a0 0x7fcbd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.447+0000 7fcbf3e00640 1 -- 172.21.15.7:0/1458708768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd80039b0 msgr2=0x7fcbd8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.447+0000 7fcbf3e00640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd80039b0 0x7fcbd8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.447+0000 7fcbf3e00640 1 -- 172.21.15.7:0/1458708768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd8003520 con 0x7fcbd8004d40 2024-04-25T11:22:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.448+0000 7fcbf3e00640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd8004d40 0x7fcbd8005150 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009f90 tx=0x7fcbd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36882c1972239c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.448+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec009c10 con 0x7fcbd8004d40 2024-04-25T11:22:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.448+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbec036070 con 0x7fcbd8004d40 2024-04-25T11:22:46.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.448+0000 7fcbf2000640 1 -- 172.21.15.7:0/1458708768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec031850 con 0x7fcbd8004d40 2024-04-25T11:22:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.449+0000 7fcbf8e00640 1 -- 172.21.15.7:0/1458708768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd8004d40 msgr2=0x7fcbd8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.449+0000 7fcbf8e00640 1 --2- 172.21.15.7:0/1458708768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd8004d40 0x7fcbd8005150 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009f90 tx=0x7fcbd8005830 comp rx=0 tx=0).stop 2024-04-25T11:22:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.449+0000 7fcbf3e00640 1 -- 172.21.15.7:0/1458708768 reap_dead start 2024-04-25T11:22:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.449+0000 7fcbf8e00640 1 -- 172.21.15.7:0/1458708768 shutdown_connections 2024-04-25T11:22:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.449+0000 7fcbf8e00640 1 -- 172.21.15.7:0/1458708768 >> 172.21.15.7:0/1458708768 conn(0x7fcbf40d1540 msgr2=0x7fcbf40d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.449+0000 7fcbf8e00640 1 -- 172.21.15.7:0/1458708768 shutdown_connections 2024-04-25T11:22:46.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.449+0000 7fcbf8e00640 1 -- 172.21.15.7:0/1458708768 wait complete. 2024-04-25T11:22:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.452+0000 7fcbf8e00640 1 Processor -- start 2024-04-25T11:22:46.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.452+0000 7fcbf8e00640 1 -- start start 2024-04-25T11:22:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.453+0000 7fcbf8e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd8004d40 0x7fcbf4148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.453+0000 7fcbf8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbf4162f10 0x7fcbf41e9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.453+0000 7fcbf8e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf4163320 0x7fcbf41ea350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.453+0000 7fcbf8e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf4070ab0 con 0x7fcbf4162f10 2024-04-25T11:22:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.453+0000 7fcbf8e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf4070930 con 0x7fcbd8004d40 2024-04-25T11:22:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.453+0000 7fcbf8e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf41571a0 con 0x7fcbf4163320 2024-04-25T11:22:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbf4162f10 0x7fcbf41e9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:46.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbf4162f10 0x7fcbf41e9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46908/0 (socket says 172.21.15.7:46908) 2024-04-25T11:22:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf2a00640 1 -- 172.21.15.7:0/2388476991 learned_addr learned my addr 172.21.15.7:0/2388476991 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf3e00640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf4163320 0x7fcbf41ea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf3400640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd8004d40 0x7fcbf4148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf2a00640 1 -- 172.21.15.7:0/2388476991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf4163320 msgr2=0x7fcbf41ea350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf2a00640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf4163320 0x7fcbf41ea350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf2a00640 1 -- 172.21.15.7:0/2388476991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd8004d40 msgr2=0x7fcbf4148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf2a00640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd8004d40 0x7fcbf4148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf2a00640 1 -- 172.21.15.7:0/2388476991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf4149500 con 0x7fcbf4162f10 2024-04-25T11:22:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf3e00640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf4163320 0x7fcbf41ea350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.454+0000 7fcbf3400640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd8004d40 0x7fcbf4148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:46.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.455+0000 7fcbf2a00640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbf4162f10 0x7fcbf41e9e10 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00116d0 tx=0x7fcbe0011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.455+0000 7fcbeb400640 1 -- 172.21.15.7:0/2388476991 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe000f9c0 con 0x7fcbf4162f10 2024-04-25T11:22:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.455+0000 7fcbeb400640 1 -- 172.21.15.7:0/2388476991 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe000fb60 con 0x7fcbf4162f10 2024-04-25T11:22:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.455+0000 7fcbeb400640 1 -- 172.21.15.7:0/2388476991 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0015460 con 0x7fcbf4162f10 2024-04-25T11:22:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.455+0000 7fcbf8e00640 1 -- 172.21.15.7:0/2388476991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf4163ab0 con 0x7fcbf4162f10 2024-04-25T11:22:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.455+0000 7fcbf8e00640 1 -- 172.21.15.7:0/2388476991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf4158350 con 0x7fcbf4162f10 2024-04-25T11:22:46.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.456+0000 7fcbf8e00640 1 -- 172.21.15.7:0/2388476991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf4163820 con 0x7fcbf4162f10 2024-04-25T11:22:46.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.458+0000 7fcbeb400640 1 -- 172.21.15.7:0/2388476991 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcbe0015600 con 0x7fcbf4162f10 2024-04-25T11:22:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.459+0000 7fcbeb400640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbd40744b0 0x7fcbd40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.459+0000 7fcbeb400640 1 -- 172.21.15.7:0/2388476991 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7fcbe0095e10 con 0x7fcbf4162f10 2024-04-25T11:22:46.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.459+0000 7fcbf3400640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbd40744b0 0x7fcbd40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:46.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.460+0000 7fcbf3400640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbd40744b0 0x7fcbd40769c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4001eb0 tx=0x7fcbe4000f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:46.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.464+0000 7fcbeb400640 1 -- 172.21.15.7:0/2388476991 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcbe0099030 con 0x7fcbf4162f10 2024-04-25T11:22:46.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.681+0000 7fcbf8e00640 1 -- 172.21.15.7:0/2388476991 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcbf4050740 con 0x7fcbf4162f10 2024-04-25T11:22:46.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.682+0000 7fcbeb400640 1 -- 172.21.15.7:0/2388476991 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v156) ==== 80+0+0 (secure 0 0 0) 0x7fcbe0019030 con 0x7fcbf4162f10 2024-04-25T11:22:46.676 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:22:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.684+0000 7fcbf8e00640 1 -- 172.21.15.7:0/2388476991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbd40744b0 msgr2=0x7fcbd40769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.684+0000 7fcbf8e00640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbd40744b0 0x7fcbd40769c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4001eb0 tx=0x7fcbe4000f90 comp rx=0 tx=0).stop 2024-04-25T11:22:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.684+0000 7fcbf8e00640 1 -- 172.21.15.7:0/2388476991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbf4162f10 msgr2=0x7fcbf41e9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:46.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.684+0000 7fcbf8e00640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbf4162f10 0x7fcbf41e9e10 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00116d0 tx=0x7fcbe0011b90 comp rx=0 tx=0).stop 2024-04-25T11:22:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.684+0000 7fcbf8e00640 1 -- 172.21.15.7:0/2388476991 shutdown_connections 2024-04-25T11:22:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.684+0000 7fcbf8e00640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcbd40744b0 0x7fcbd40769c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.684+0000 7fcbf8e00640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbf4163320 0x7fcbf41ea350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.684+0000 7fcbf8e00640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbf4162f10 0x7fcbf41e9e10 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.684+0000 7fcbf8e00640 1 --2- 172.21.15.7:0/2388476991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbd8004d40 0x7fcbf4148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.684+0000 7fcbf8e00640 1 -- 172.21.15.7:0/2388476991 >> 172.21.15.7:0/2388476991 conn(0x7fcbf40d1540 msgr2=0x7fcbf40b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.685+0000 7fcbf8e00640 1 -- 172.21.15.7:0/2388476991 shutdown_connections 2024-04-25T11:22:46.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:46.685+0000 7fcbf8e00640 1 -- 172.21.15.7:0/2388476991 wait complete. 2024-04-25T11:22:46.708 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T11:22:46.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:46.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T11:22:46.804 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:22:46.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:46.805 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T11:22:46.857 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:22:46.858 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:22:46.858 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:46.916 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:22:46.916 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:46.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:47.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.267+0000 7f748e000640 1 Processor -- start 2024-04-25T11:22:47.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.267+0000 7f748e000640 1 -- start start 2024-04-25T11:22:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.270+0000 7f748e000640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f748809c110 0x7f74880cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.270+0000 7f748e000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74880d31a0 0x7f74880ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.270+0000 7f748e000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74880d1cf0 con 0x7f74880d2d90 2024-04-25T11:22:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.270+0000 7f748e000640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f74880d1fd0 con 0x7f748809c110 2024-04-25T11:22:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.270+0000 7f748e000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f74880d22b0 con 0x7f74880d31a0 2024-04-25T11:22:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.270+0000 7f748cc00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f74880d2d90 0x7f748809ba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33838/0 (socket says 172.21.15.7:33838) 2024-04-25T11:22:47.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.270+0000 7f748cc00640 1 -- 172.21.15.7:0/2538369023 learned_addr learned my addr 172.21.15.7:0/2538369023 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487e00640 1 --2- 172.21.15.7:0/2538369023 >> v2:172.21.15.87:3300/0 conn(0x7f748809c110 0x7f74880cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f748d600640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74880d31a0 0x7f74880ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 129016900 0 0) 0x7f74880d1cf0 con 0x7f74880d2d90 2024-04-25T11:22:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7470003520 con 0x7f74880d2d90 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4009531884 0 0) 0x7f7470003520 con 0x7f74880d2d90 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74880d31a0 msgr2=0x7f74880ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74880d31a0 0x7f74880ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 >> v2:172.21.15.87:3300/0 conn(0x7f748809c110 msgr2=0x7f74880cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 --2- 172.21.15.7:0/2538369023 >> v2:172.21.15.87:3300/0 conn(0x7f748809c110 0x7f74880cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74880d2590 con 0x7f74880d2d90 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f747c0017f0 con 0x7f74880d2d90 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f748d600640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74880d31a0 0x7f74880ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 >> v1:172.21.15.7:6789/0 conn(0x7f74880d2d90 legacy=0x7f748809ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74700039b0 0x7f7470003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7470004800 0x7f7470004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74880d31a0 0x7f7470005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74880d1fd0 con 0x7f7470004800 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f74880d22b0 con 0x7f74700039b0 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.271+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f747000ba70 con 0x7f74880d31a0 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.272+0000 7f7487e00640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7470004800 0x7f7470004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.272+0000 7f748cc00640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74700039b0 0x7f7470003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.272+0000 7f748d600640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74880d31a0 0x7f7470005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.272+0000 7f7487e00640 1 -- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74880d31a0 msgr2=0x7f7470005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.272+0000 7f7487e00640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74880d31a0 0x7f7470005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.272+0000 7f7487e00640 1 -- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74700039b0 msgr2=0x7f7470003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.272+0000 7f7487e00640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74700039b0 0x7f7470003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.272+0000 7f7487e00640 1 -- 172.21.15.7:0/2538369023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7470003520 con 0x7f7470004800 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.272+0000 7f748cc00640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74700039b0 0x7f7470003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.272+0000 7f748d600640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74880d31a0 0x7f7470005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.273+0000 7f7487e00640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7470004800 0x7f7470004c10 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f7478009af0 tx=0x7f7470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8d2f2ffdf7f523 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.274+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478036070 con 0x7f7470004800 2024-04-25T11:22:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.274+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7478031400 con 0x7f7470004800 2024-04-25T11:22:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.274+0000 7f7487400640 1 -- 172.21.15.7:0/2538369023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478009690 con 0x7f7470004800 2024-04-25T11:22:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.274+0000 7f748e000640 1 -- 172.21.15.7:0/2538369023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7470004800 msgr2=0x7f7470004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.274+0000 7f748e000640 1 --2- 172.21.15.7:0/2538369023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7470004800 0x7f7470004c10 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f7478009af0 tx=0x7f7470005830 comp rx=0 tx=0).stop 2024-04-25T11:22:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.274+0000 7f748d600640 1 -- 172.21.15.7:0/2538369023 reap_dead start 2024-04-25T11:22:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.274+0000 7f748e000640 1 -- 172.21.15.7:0/2538369023 shutdown_connections 2024-04-25T11:22:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.274+0000 7f748e000640 1 -- 172.21.15.7:0/2538369023 >> 172.21.15.7:0/2538369023 conn(0x7f74880b5250 msgr2=0x7f74880b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.274+0000 7f748e000640 1 -- 172.21.15.7:0/2538369023 shutdown_connections 2024-04-25T11:22:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.274+0000 7f748e000640 1 -- 172.21.15.7:0/2538369023 wait complete. 2024-04-25T11:22:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.277+0000 7f748e000640 1 Processor -- start 2024-04-25T11:22:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.277+0000 7f748e000640 1 -- start start 2024-04-25T11:22:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.278+0000 7f748e000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7470004800 0x7f748815e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.278+0000 7f748e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74880cd0e0 0x7f74880c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.278+0000 7f748e000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f748815ceb0 0x7f74880ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.278+0000 7f748e000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74880b91e0 con 0x7f74880cd0e0 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.278+0000 7f748e000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f74880d19a0 con 0x7f7470004800 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f748e000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f74880ce2f0 con 0x7f748815ceb0 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f7487e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74880cd0e0 0x7f74880c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f7487e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74880cd0e0 0x7f74880c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46920/0 (socket says 172.21.15.7:46920) 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f7487e00640 1 -- 172.21.15.7:0/2019009966 learned_addr learned my addr 172.21.15.7:0/2019009966 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f748d600640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f748815ceb0 0x7f74880ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f748cc00640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7470004800 0x7f748815e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f7487e00640 1 -- 172.21.15.7:0/2019009966 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f748815ceb0 msgr2=0x7f74880ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f7487e00640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f748815ceb0 0x7f74880ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f7487e00640 1 -- 172.21.15.7:0/2019009966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7470004800 msgr2=0x7f748815e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f7487e00640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7470004800 0x7f748815e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f7487e00640 1 -- 172.21.15.7:0/2019009966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74880c7c70 con 0x7f74880cd0e0 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.279+0000 7f748cc00640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7470004800 0x7f748815e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:47.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.280+0000 7f7487e00640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74880cd0e0 0x7f74880c8d10 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7478009af0 tx=0x7f7478004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:47.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.280+0000 7f7484c00640 1 -- 172.21.15.7:0/2019009966 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478041020 con 0x7f74880cd0e0 2024-04-25T11:22:47.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.280+0000 7f7484c00640 1 -- 172.21.15.7:0/2019009966 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747802ade0 con 0x7f74880cd0e0 2024-04-25T11:22:47.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.280+0000 7f7484c00640 1 -- 172.21.15.7:0/2019009966 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478036040 con 0x7f74880cd0e0 2024-04-25T11:22:47.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.280+0000 7f748e000640 1 -- 172.21.15.7:0/2019009966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74880c7f50 con 0x7f74880cd0e0 2024-04-25T11:22:47.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.280+0000 7f748e000640 1 -- 172.21.15.7:0/2019009966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74880c8510 con 0x7f74880cd0e0 2024-04-25T11:22:47.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.281+0000 7f748e000640 1 -- 172.21.15.7:0/2019009966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7488050740 con 0x7f74880cd0e0 2024-04-25T11:22:47.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.283+0000 7f7484c00640 1 -- 172.21.15.7:0/2019009966 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7478044020 con 0x7f74880cd0e0 2024-04-25T11:22:47.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.284+0000 7f7484c00640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f746c0744b0 0x7f746c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.284+0000 7f7484c00640 1 -- 172.21.15.7:0/2019009966 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f74780b5aa0 con 0x7f74880cd0e0 2024-04-25T11:22:47.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.284+0000 7f748cc00640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f746c0744b0 0x7f746c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.285+0000 7f748cc00640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f746c0744b0 0x7f746c0769c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f747c0019e0 tx=0x7f747c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:47.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.289+0000 7f7484c00640 1 -- 172.21.15.7:0/2019009966 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f747804b030 con 0x7f74880cd0e0 2024-04-25T11:22:47.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.495+0000 7f748e000640 1 -- 172.21.15.7:0/2019009966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f748815ef50 con 0x7f74880cd0e0 2024-04-25T11:22:47.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.497+0000 7f7484c00640 1 -- 172.21.15.7:0/2019009966 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16788 (secure 0 0 0) 0x7f7478080090 con 0x7f74880cd0e0 2024-04-25T11:22:47.492 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:47.492 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:33.767424+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:22:30.472409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T11:22:31.299177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 -- 172.21.15.7:0/2019009966 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f746c0744b0 msgr2=0x7f746c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f746c0744b0 0x7f746c0769c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f747c0019e0 tx=0x7f747c009040 comp rx=0 tx=0).stop 2024-04-25T11:22:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 -- 172.21.15.7:0/2019009966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74880cd0e0 msgr2=0x7f74880c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74880cd0e0 0x7f74880c8d10 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7478009af0 tx=0x7f7478004730 comp rx=0 tx=0).stop 2024-04-25T11:22:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 -- 172.21.15.7:0/2019009966 shutdown_connections 2024-04-25T11:22:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f746c0744b0 0x7f746c0769c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f748815ceb0 0x7f74880ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74880cd0e0 0x7f74880c8d10 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 --2- 172.21.15.7:0/2019009966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7470004800 0x7f748815e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 -- 172.21.15.7:0/2019009966 >> 172.21.15.7:0/2019009966 conn(0x7f74880b5250 msgr2=0x7f7488072c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:47.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 -- 172.21.15.7:0/2019009966 shutdown_connections 2024-04-25T11:22:47.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.500+0000 7f748e000640 1 -- 172.21.15.7:0/2019009966 wait complete. 2024-04-25T11:22:47.522 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:22:30.472409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:22:31.299177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:22:47.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:22:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.817+0000 7f28d2c00640 1 Processor -- start 2024-04-25T11:22:47.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.817+0000 7f28d2c00640 1 -- start start 2024-04-25T11:22:47.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.820+0000 7f28d2c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc145e40 0x7f28cc146250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.820+0000 7f28d2c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f28cc144a80 0x7f28cc0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.820+0000 7f28d2c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28cc0b66b0 con 0x7f28cc0b4870 2024-04-25T11:22:47.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.820+0000 7f28d2c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f28cc145b70 con 0x7f28cc144a80 2024-04-25T11:22:47.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.820+0000 7f28d2c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc0b58d0 con 0x7f28cc145e40 2024-04-25T11:22:47.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.820+0000 7f28d0e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f28cc0b4870 0x7f28cc0b4c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33852/0 (socket says 172.21.15.7:33852) 2024-04-25T11:22:47.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.820+0000 7f28d0e00640 1 -- 172.21.15.7:0/2831137277 learned_addr learned my addr 172.21.15.7:0/2831137277 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:47.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28d2200640 1 --2- 172.21.15.7:0/2831137277 >> v2:172.21.15.87:3300/0 conn(0x7f28cc144a80 0x7f28cc0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28d1800640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc145e40 0x7f28cc146250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4238401495 0 0) 0x7f28cc0b66b0 con 0x7f28cc0b4870 2024-04-25T11:22:47.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28b4005730 con 0x7f28cc0b4870 2024-04-25T11:22:47.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1178463742 0 0) 0x7f28b4005730 con 0x7f28cc0b4870 2024-04-25T11:22:47.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc145e40 msgr2=0x7f28cc146250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:47.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc145e40 0x7f28cc146250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 >> v2:172.21.15.87:3300/0 conn(0x7f28cc144a80 msgr2=0x7f28cc0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:47.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 --2- 172.21.15.7:0/2831137277 >> v2:172.21.15.87:3300/0 conn(0x7f28cc144a80 0x7f28cc0b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc0b5b30 con 0x7f28cc0b4870 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f28b8001670 con 0x7f28cc0b4870 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 >> v1:172.21.15.7:6789/0 conn(0x7f28cc0b4870 legacy=0x7f28cc0b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc145e40 0x7f28b4005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28b4006140 0x7f28b4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b4006f50 0x7f28b4007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc0b58d0 con 0x7f28b4006f50 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f28b400dc80 con 0x7f28b4006140 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.821+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f28b400de50 con 0x7f28cc145e40 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.822+0000 7f28d2200640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b4006f50 0x7f28b4007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.822+0000 7f28d1800640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc145e40 0x7f28b4005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.822+0000 7f28d2200640 1 -- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc145e40 msgr2=0x7f28b4005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.822+0000 7f28d2200640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc145e40 0x7f28b4005bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.822+0000 7f28d2200640 1 -- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28b4006140 msgr2=0x7f28b4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:47.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.822+0000 7f28d0e00640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28b4006140 0x7f28b4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.822+0000 7f28d2200640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28b4006140 0x7f28b4006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.822+0000 7f28d2200640 1 -- 172.21.15.7:0/2831137277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b4005730 con 0x7f28b4006f50 2024-04-25T11:22:47.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.822+0000 7f28d0e00640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28b4006140 0x7f28b4006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:47.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.823+0000 7f28d2200640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b4006f50 0x7f28b4007360 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06e0f0 tx=0x7f28b4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18e5c148c48d4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:47.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.824+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4009c10 con 0x7f28b4006f50 2024-04-25T11:22:47.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.824+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c4036070 con 0x7f28b4006f50 2024-04-25T11:22:47.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.824+0000 7f28cbe00640 1 -- 172.21.15.7:0/2831137277 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4031810 con 0x7f28b4006f50 2024-04-25T11:22:47.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.824+0000 7f28d2c00640 1 -- 172.21.15.7:0/2831137277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b4006f50 msgr2=0x7f28b4007360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.824+0000 7f28d2c00640 1 --2- 172.21.15.7:0/2831137277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28b4006f50 0x7f28b4007360 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06e0f0 tx=0x7f28b4007a40 comp rx=0 tx=0).stop 2024-04-25T11:22:47.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.824+0000 7f28d2200640 1 -- 172.21.15.7:0/2831137277 reap_dead start 2024-04-25T11:22:47.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.825+0000 7f28d2c00640 1 -- 172.21.15.7:0/2831137277 shutdown_connections 2024-04-25T11:22:47.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.825+0000 7f28d2c00640 1 -- 172.21.15.7:0/2831137277 >> 172.21.15.7:0/2831137277 conn(0x7f28cc0740f0 msgr2=0x7f28cc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:47.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.825+0000 7f28d2c00640 1 -- 172.21.15.7:0/2831137277 shutdown_connections 2024-04-25T11:22:47.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.825+0000 7f28d2c00640 1 -- 172.21.15.7:0/2831137277 wait complete. 2024-04-25T11:22:47.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.827+0000 7f28d2c00640 1 Processor -- start 2024-04-25T11:22:47.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.827+0000 7f28d2c00640 1 -- start start 2024-04-25T11:22:47.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d2c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28b4006f50 0x7f28cc0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d2c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28cc140510 0x7f28cc13fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d2c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc1f19c0 0x7f28cc1f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d2c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc0b63d0 con 0x7f28cc140510 2024-04-25T11:22:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d2c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc0b60f0 con 0x7f28b4006f50 2024-04-25T11:22:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d2c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc146250 con 0x7f28cc1f19c0 2024-04-25T11:22:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28cc140510 0x7f28cc13fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d0e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28cc140510 0x7f28cc13fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46946/0 (socket says 172.21.15.7:46946) 2024-04-25T11:22:47.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d0e00640 1 -- 172.21.15.7:0/1740305142 learned_addr learned my addr 172.21.15.7:0/1740305142 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:47.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d1800640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28b4006f50 0x7f28cc0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d0e00640 1 -- 172.21.15.7:0/1740305142 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc1f19c0 msgr2=0x7f28cc1f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d0e00640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc1f19c0 0x7f28cc1f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d0e00640 1 -- 172.21.15.7:0/1740305142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28b4006f50 msgr2=0x7f28cc0cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:47.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d0e00640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28b4006f50 0x7f28cc0cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.829+0000 7f28d0e00640 1 -- 172.21.15.7:0/1740305142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc0cca50 con 0x7f28cc140510 2024-04-25T11:22:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.830+0000 7f28d0e00640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28cc140510 0x7f28cc13fd90 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f28b800d8b0 tx=0x7f28b800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.830+0000 7f28c9600640 1 -- 172.21.15.7:0/1740305142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8019070 con 0x7f28cc140510 2024-04-25T11:22:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.830+0000 7f28c9600640 1 -- 172.21.15.7:0/1740305142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b80074e0 con 0x7f28cc140510 2024-04-25T11:22:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.830+0000 7f28c9600640 1 -- 172.21.15.7:0/1740305142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b80049e0 con 0x7f28cc140510 2024-04-25T11:22:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.830+0000 7f28d2c00640 1 -- 172.21.15.7:0/1740305142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc1402d0 con 0x7f28cc140510 2024-04-25T11:22:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.830+0000 7f28d2c00640 1 -- 172.21.15.7:0/1740305142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc0cb750 con 0x7f28cc140510 2024-04-25T11:22:47.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.832+0000 7f28d2c00640 1 -- 172.21.15.7:0/1740305142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc0cb1e0 con 0x7f28cc140510 2024-04-25T11:22:47.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.833+0000 7f28c9600640 1 -- 172.21.15.7:0/1740305142 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f28b801c070 con 0x7f28cc140510 2024-04-25T11:22:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.834+0000 7f28c9600640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28ac0744b0 0x7f28ac0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.834+0000 7f28c9600640 1 -- 172.21.15.7:0/1740305142 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f28b8094ef0 con 0x7f28cc140510 2024-04-25T11:22:47.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.834+0000 7f28d1800640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28ac0744b0 0x7f28ac0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:47.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.835+0000 7f28d1800640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28ac0744b0 0x7f28ac0769c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009910 tx=0x7f28c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:47.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:47.838+0000 7f28c9600640 1 -- 172.21.15.7:0/1740305142 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f28cc0cca50 con 0x7f28cc140510 2024-04-25T11:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.036+0000 7f28d2c00640 1 -- 172.21.15.7:0/1740305142 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f28cc050740 con 0x7f28cc140510 2024-04-25T11:22:48.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.037+0000 7f28c9600640 1 -- 172.21.15.7:0/1740305142 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7f28b805f4f0 con 0x7f28cc140510 2024-04-25T11:22:48.031 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:22:48.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.039+0000 7f28d2c00640 1 -- 172.21.15.7:0/1740305142 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28ac0744b0 msgr2=0x7f28ac0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:48.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.039+0000 7f28d2c00640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28ac0744b0 0x7f28ac0769c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009910 tx=0x7f28c0008040 comp rx=0 tx=0).stop 2024-04-25T11:22:48.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.039+0000 7f28d2c00640 1 -- 172.21.15.7:0/1740305142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28cc140510 msgr2=0x7f28cc13fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:48.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.039+0000 7f28d2c00640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28cc140510 0x7f28cc13fd90 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f28b800d8b0 tx=0x7f28b800dd70 comp rx=0 tx=0).stop 2024-04-25T11:22:48.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.039+0000 7f28d2c00640 1 -- 172.21.15.7:0/1740305142 shutdown_connections 2024-04-25T11:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.039+0000 7f28d2c00640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f28ac0744b0 0x7f28ac0769c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.039+0000 7f28d2c00640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28cc1f19c0 0x7f28cc1f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.039+0000 7f28d2c00640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28cc140510 0x7f28cc13fd90 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.039+0000 7f28d2c00640 1 --2- 172.21.15.7:0/1740305142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28b4006f50 0x7f28cc0cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.039+0000 7f28d2c00640 1 -- 172.21.15.7:0/1740305142 >> 172.21.15.7:0/1740305142 conn(0x7f28cc0740f0 msgr2=0x7f28cc144e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.040+0000 7f28d2c00640 1 -- 172.21.15.7:0/1740305142 shutdown_connections 2024-04-25T11:22:48.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.040+0000 7f28d2c00640 1 -- 172.21.15.7:0/1740305142 wait complete. 2024-04-25T11:22:48.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:22:48.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.354+0000 7fef9ac00640 1 Processor -- start 2024-04-25T11:22:48.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.354+0000 7fef9ac00640 1 -- start start 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.357+0000 7fef9ac00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef941604c0 0x7fef940af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.357+0000 7fef9ac00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fef940cf840 0x7fef940b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.357+0000 7fef9ac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef940b78f0 0x7fef940d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.357+0000 7fef9ac00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fef940cffe0 con 0x7fef941604c0 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.357+0000 7fef9ac00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fef940d2310 con 0x7fef940cf840 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.357+0000 7fef9ac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fef940b7d00 con 0x7fef940b78f0 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.357+0000 7fef99800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef941604c0 0x7fef940af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.358+0000 7fef99800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef941604c0 0x7fef940af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46956/0 (socket says 172.21.15.7:46956) 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.358+0000 7fef99800640 1 -- 172.21.15.7:0/1686313806 learned_addr learned my addr 172.21.15.7:0/1686313806 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.358+0000 7fef98e00640 1 --2- 172.21.15.7:0/1686313806 >> v2:172.21.15.87:3300/0 conn(0x7fef940cf840 0x7fef940b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.358+0000 7fef9a200640 1 --2- 172.21.15.7:0/1686313806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef940b78f0 0x7fef940d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.358+0000 7fef99800640 1 -- 172.21.15.7:0/1686313806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef940b78f0 msgr2=0x7fef940d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:48.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.358+0000 7fef99800640 1 --2- 172.21.15.7:0/1686313806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef940b78f0 0x7fef940d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.358+0000 7fef99800640 1 -- 172.21.15.7:0/1686313806 >> v2:172.21.15.87:3300/0 conn(0x7fef940cf840 msgr2=0x7fef940b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:48.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.358+0000 7fef99800640 1 --2- 172.21.15.7:0/1686313806 >> v2:172.21.15.87:3300/0 conn(0x7fef940cf840 0x7fef940b73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.358+0000 7fef99800640 1 -- 172.21.15.7:0/1686313806 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef940d1740 con 0x7fef941604c0 2024-04-25T11:22:48.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.358+0000 7fef9a200640 1 --2- 172.21.15.7:0/1686313806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef940b78f0 0x7fef940d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.359+0000 7fef99800640 1 --2- 172.21.15.7:0/1686313806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef941604c0 0x7fef940af3e0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fef940b13d0 tx=0x7fef8802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09509968f002f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.360+0000 7fef93e00640 1 -- 172.21.15.7:0/1686313806 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88037070 con 0x7fef941604c0 2024-04-25T11:22:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.360+0000 7fef93e00640 1 -- 172.21.15.7:0/1686313806 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef88032440 con 0x7fef941604c0 2024-04-25T11:22:48.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.360+0000 7fef93e00640 1 -- 172.21.15.7:0/1686313806 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88032810 con 0x7fef941604c0 2024-04-25T11:22:48.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.361+0000 7fef9ac00640 1 -- 172.21.15.7:0/1686313806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef941604c0 msgr2=0x7fef940af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:48.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.361+0000 7fef9ac00640 1 --2- 172.21.15.7:0/1686313806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef941604c0 0x7fef940af3e0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fef940b13d0 tx=0x7fef8802a1e0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.361+0000 7fef9ac00640 1 -- 172.21.15.7:0/1686313806 shutdown_connections 2024-04-25T11:22:48.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.361+0000 7fef9ac00640 1 --2- 172.21.15.7:0/1686313806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef940b78f0 0x7fef940d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.361+0000 7fef9ac00640 1 --2- 172.21.15.7:0/1686313806 >> v2:172.21.15.87:3300/0 conn(0x7fef940cf840 0x7fef940b73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.361+0000 7fef9ac00640 1 --2- 172.21.15.7:0/1686313806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef941604c0 0x7fef940af3e0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.361+0000 7fef9ac00640 1 -- 172.21.15.7:0/1686313806 >> 172.21.15.7:0/1686313806 conn(0x7fef9409b540 msgr2=0x7fef9409b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:48.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.361+0000 7fef9ac00640 1 -- 172.21.15.7:0/1686313806 shutdown_connections 2024-04-25T11:22:48.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.361+0000 7fef9ac00640 1 -- 172.21.15.7:0/1686313806 wait complete. 2024-04-25T11:22:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.363+0000 7fef9ac00640 1 Processor -- start 2024-04-25T11:22:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.363+0000 7fef9ac00640 1 -- start start 2024-04-25T11:22:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.365+0000 7fef9ac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef940b78f0 0x7fef940b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.365+0000 7fef9ac00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef940cf840 0x7fef9414d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.365+0000 7fef9ac00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941604c0 0x7fef941f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.365+0000 7fef9ac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fef940d1fa0 con 0x7fef940b78f0 2024-04-25T11:22:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.365+0000 7fef9ac00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fef940d1cc0 con 0x7fef941604c0 2024-04-25T11:22:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.365+0000 7fef9ac00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fef940cfc50 con 0x7fef940cf840 2024-04-25T11:22:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.365+0000 7fef99800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef940b78f0 0x7fef940b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.365+0000 7fef99800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef940b78f0 0x7fef940b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46970/0 (socket says 172.21.15.7:46970) 2024-04-25T11:22:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.365+0000 7fef99800640 1 -- 172.21.15.7:0/2466405449 learned_addr learned my addr 172.21.15.7:0/2466405449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.366+0000 7fef98e00640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef940cf840 0x7fef9414d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.366+0000 7fef99800640 1 -- 172.21.15.7:0/2466405449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef940cf840 msgr2=0x7fef9414d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.366+0000 7fef9a200640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941604c0 0x7fef941f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.366+0000 7fef99800640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef940cf840 0x7fef9414d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.366+0000 7fef99800640 1 -- 172.21.15.7:0/2466405449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941604c0 msgr2=0x7fef941f99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.366+0000 7fef99800640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941604c0 0x7fef941f99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.366+0000 7fef99800640 1 -- 172.21.15.7:0/2466405449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9414dc00 con 0x7fef940b78f0 2024-04-25T11:22:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.366+0000 7fef99800640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef940b78f0 0x7fef940b0a30 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fef8802a220 tx=0x7fef880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.366+0000 7fef91600640 1 -- 172.21.15.7:0/2466405449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88041020 con 0x7fef940b78f0 2024-04-25T11:22:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.367+0000 7fef91600640 1 -- 172.21.15.7:0/2466405449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef880306c0 con 0x7fef940b78f0 2024-04-25T11:22:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.367+0000 7fef9ac00640 1 -- 172.21.15.7:0/2466405449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef94162b60 con 0x7fef940b78f0 2024-04-25T11:22:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.367+0000 7fef91600640 1 -- 172.21.15.7:0/2466405449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88037040 con 0x7fef940b78f0 2024-04-25T11:22:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.367+0000 7fef9ac00640 1 -- 172.21.15.7:0/2466405449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef94163120 con 0x7fef940b78f0 2024-04-25T11:22:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.368+0000 7fef9ac00640 1 -- 172.21.15.7:0/2466405449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef94050740 con 0x7fef940b78f0 2024-04-25T11:22:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.369+0000 7fef91600640 1 -- 172.21.15.7:0/2466405449 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fef88044020 con 0x7fef940b78f0 2024-04-25T11:22:48.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.371+0000 7fef91600640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef740744b0 0x7fef740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.371+0000 7fef91600640 1 -- 172.21.15.7:0/2466405449 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7fef880b5c70 con 0x7fef940b78f0 2024-04-25T11:22:48.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.371+0000 7fef98e00640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef740744b0 0x7fef740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.372+0000 7fef98e00640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef740744b0 0x7fef740769c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fef80006fd0 tx=0x7fef80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:48.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.376+0000 7fef91600640 1 -- 172.21.15.7:0/2466405449 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fef8804b030 con 0x7fef940b78f0 2024-04-25T11:22:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.580+0000 7fef9ac00640 1 -- 172.21.15.7:0/2466405449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fef940d3440 con 0x7fef940b78f0 2024-04-25T11:22:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.581+0000 7fef91600640 1 -- 172.21.15.7:0/2466405449 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) ==== 104+0+11 (secure 0 0 0) 0x7fef88080260 con 0x7fef940b78f0 2024-04-25T11:22:48.576 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:22:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.583+0000 7fef9ac00640 1 -- 172.21.15.7:0/2466405449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef740744b0 msgr2=0x7fef740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.583+0000 7fef9ac00640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef740744b0 0x7fef740769c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fef80006fd0 tx=0x7fef80008040 comp rx=0 tx=0).stop 2024-04-25T11:22:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.583+0000 7fef9ac00640 1 -- 172.21.15.7:0/2466405449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef940b78f0 msgr2=0x7fef940b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.583+0000 7fef9ac00640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef940b78f0 0x7fef940b0a30 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fef8802a220 tx=0x7fef880046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.583+0000 7fef9ac00640 1 -- 172.21.15.7:0/2466405449 shutdown_connections 2024-04-25T11:22:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.583+0000 7fef9ac00640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fef740744b0 0x7fef740769c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.583+0000 7fef9ac00640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef941604c0 0x7fef941f99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.583+0000 7fef9ac00640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef940cf840 0x7fef9414d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.583+0000 7fef9ac00640 1 --2- 172.21.15.7:0/2466405449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef940b78f0 0x7fef940b0a30 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.583+0000 7fef9ac00640 1 -- 172.21.15.7:0/2466405449 >> 172.21.15.7:0/2466405449 conn(0x7fef9409b540 msgr2=0x7fef9409b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.584+0000 7fef9ac00640 1 -- 172.21.15.7:0/2466405449 shutdown_connections 2024-04-25T11:22:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.584+0000 7fef9ac00640 1 -- 172.21.15.7:0/2466405449 wait complete. 2024-04-25T11:22:48.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:22:48.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.901+0000 7f18c2400640 1 Processor -- start 2024-04-25T11:22:48.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.901+0000 7f18c2400640 1 -- start start 2024-04-25T11:22:48.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.903+0000 7f18c2400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f18bc0cfa80 0x7f18bc0d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.904+0000 7f18c2400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18bc0d0800 0x7f18bc06eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.904+0000 7f18c2400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18bc0d0c10 con 0x7f18bc070630 2024-04-25T11:22:48.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.904+0000 7f18c2400640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f18bc0b0be0 con 0x7f18bc0d0800 2024-04-25T11:22:48.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.904+0000 7f18c2400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f18bc0cfe90 con 0x7f18bc0cfa80 2024-04-25T11:22:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.904+0000 7f18c1a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18bc0d0800 0x7f18bc06eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.904+0000 7f18c1a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18bc0d0800 0x7f18bc06eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46978/0 (socket says 172.21.15.7:46978) 2024-04-25T11:22:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.904+0000 7f18c1a00640 1 -- 172.21.15.7:0/3423688472 learned_addr learned my addr 172.21.15.7:0/3423688472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.904+0000 7f18bbe00640 1 --2- 172.21.15.7:0/3423688472 >> v2:172.21.15.87:3300/0 conn(0x7f18bc0cfa80 0x7f18bc0d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.904+0000 7f18c1a00640 1 -- 172.21.15.7:0/3423688472 >> v2:172.21.15.87:3300/0 conn(0x7f18bc0cfa80 msgr2=0x7f18bc0d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.905+0000 7f18c1a00640 1 --2- 172.21.15.7:0/3423688472 >> v2:172.21.15.87:3300/0 conn(0x7f18bc0cfa80 0x7f18bc0d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.905+0000 7f18c1a00640 1 -- 172.21.15.7:0/3423688472 >> v1:172.21.15.87:6789/0 conn(0x7f18bc070630 legacy=0x7f18bc15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:48.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.905+0000 7f18c1a00640 1 -- 172.21.15.7:0/3423688472 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc0ce9e0 con 0x7f18bc0d0800 2024-04-25T11:22:48.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.905+0000 7f18bbe00640 1 --2- 172.21.15.7:0/3423688472 >> v2:172.21.15.87:3300/0 conn(0x7f18bc0cfa80 0x7f18bc0d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:48.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.906+0000 7f18c1a00640 1 --2- 172.21.15.7:0/3423688472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18bc0d0800 0x7f18bc06eea0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0b4740 tx=0x7f18b402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfed3f2ca1ebc031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.906+0000 7f18bb400640 1 -- 172.21.15.7:0/3423688472 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4037070 con 0x7f18bc0d0800 2024-04-25T11:22:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.906+0000 7f18bb400640 1 -- 172.21.15.7:0/3423688472 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b4032440 con 0x7f18bc0d0800 2024-04-25T11:22:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.906+0000 7f18bb400640 1 -- 172.21.15.7:0/3423688472 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4032810 con 0x7f18bc0d0800 2024-04-25T11:22:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.907+0000 7f18c2400640 1 -- 172.21.15.7:0/3423688472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18bc0d0800 msgr2=0x7f18bc06eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.907+0000 7f18c2400640 1 --2- 172.21.15.7:0/3423688472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18bc0d0800 0x7f18bc06eea0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0b4740 tx=0x7f18b402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.907+0000 7f18c2400640 1 -- 172.21.15.7:0/3423688472 shutdown_connections 2024-04-25T11:22:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.907+0000 7f18c2400640 1 --2- 172.21.15.7:0/3423688472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18bc0d0800 0x7f18bc06eea0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.907+0000 7f18c2400640 1 --2- 172.21.15.7:0/3423688472 >> v2:172.21.15.87:3300/0 conn(0x7f18bc0cfa80 0x7f18bc0d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.907+0000 7f18c2400640 1 -- 172.21.15.7:0/3423688472 >> 172.21.15.7:0/3423688472 conn(0x7f18bc074140 msgr2=0x7f18bc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.907+0000 7f18c2400640 1 -- 172.21.15.7:0/3423688472 shutdown_connections 2024-04-25T11:22:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.908+0000 7f18c2400640 1 -- 172.21.15.7:0/3423688472 wait complete. 2024-04-25T11:22:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.910+0000 7f18c2400640 1 Processor -- start 2024-04-25T11:22:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.910+0000 7f18c2400640 1 -- start start 2024-04-25T11:22:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c2400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18bc070630 0x7f18bc155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c2400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18bc0cfa80 0x7f18bc1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18bc0d0800 0x7f18bc151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c2400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc0cf280 con 0x7f18bc0d0800 2024-04-25T11:22:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c2400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc0cefa0 con 0x7f18bc0cfa80 2024-04-25T11:22:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c2400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc0cf560 con 0x7f18bc070630 2024-04-25T11:22:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18bc0d0800 0x7f18bc151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18bc0d0800 0x7f18bc151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46990/0 (socket says 172.21.15.7:46990) 2024-04-25T11:22:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c1a00640 1 -- 172.21.15.7:0/4061526893 learned_addr learned my addr 172.21.15.7:0/4061526893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18bbe00640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18bc0cfa80 0x7f18bc1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c1000640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18bc070630 0x7f18bc155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c1a00640 1 -- 172.21.15.7:0/4061526893 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18bc070630 msgr2=0x7f18bc155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c1a00640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18bc070630 0x7f18bc155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c1a00640 1 -- 172.21.15.7:0/4061526893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18bc0cfa80 msgr2=0x7f18bc1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c1a00640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18bc0cfa80 0x7f18bc1592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.912+0000 7f18c1a00640 1 -- 172.21.15.7:0/4061526893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc159830 con 0x7f18bc0d0800 2024-04-25T11:22:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.913+0000 7f18c1a00640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18bc0d0800 0x7f18bc151410 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f18b40090b0 tx=0x7f18b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.913+0000 7f18b8c00640 1 -- 172.21.15.7:0/4061526893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4041020 con 0x7f18bc0d0800 2024-04-25T11:22:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.913+0000 7f18b8c00640 1 -- 172.21.15.7:0/4061526893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b4030990 con 0x7f18bc0d0800 2024-04-25T11:22:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.913+0000 7f18b8c00640 1 -- 172.21.15.7:0/4061526893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4037040 con 0x7f18bc0d0800 2024-04-25T11:22:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.913+0000 7f18c2400640 1 -- 172.21.15.7:0/4061526893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc154bb0 con 0x7f18bc0d0800 2024-04-25T11:22:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.913+0000 7f18c2400640 1 -- 172.21.15.7:0/4061526893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc155170 con 0x7f18bc0d0800 2024-04-25T11:22:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.915+0000 7f18c2400640 1 -- 172.21.15.7:0/4061526893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18bc159a00 con 0x7f18bc0d0800 2024-04-25T11:22:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.916+0000 7f18b8c00640 1 -- 172.21.15.7:0/4061526893 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f18b4044020 con 0x7f18bc0d0800 2024-04-25T11:22:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.917+0000 7f18b8c00640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f189c0744b0 0x7f189c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.918+0000 7f18b8c00640 1 -- 172.21.15.7:0/4061526893 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f18b40b5e10 con 0x7f18bc0d0800 2024-04-25T11:22:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.918+0000 7f18c1000640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f189c0744b0 0x7f189c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.919+0000 7f18c1000640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f189c0744b0 0x7f189c0769c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f18ac007dc0 tx=0x7f18ac0072f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:48.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:48.923+0000 7f18b8c00640 1 -- 172.21.15.7:0/4061526893 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f18b40b9030 con 0x7f18bc0d0800 2024-04-25T11:22:49.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.128+0000 7f18c2400640 1 -- 172.21.15.7:0/4061526893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f18bc070dd0 con 0x7f18bc0d0800 2024-04-25T11:22:49.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.129+0000 7f18b8c00640 1 -- 172.21.15.7:0/4061526893 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) ==== 100+0+11 (secure 0 0 0) 0x7f18b404b030 con 0x7f18bc0d0800 2024-04-25T11:22:49.124 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:22:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.132+0000 7f18c2400640 1 -- 172.21.15.7:0/4061526893 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f189c0744b0 msgr2=0x7f189c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.132+0000 7f18c2400640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f189c0744b0 0x7f189c0769c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f18ac007dc0 tx=0x7f18ac0072f0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.132+0000 7f18c2400640 1 -- 172.21.15.7:0/4061526893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18bc0d0800 msgr2=0x7f18bc151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.132+0000 7f18c2400640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18bc0d0800 0x7f18bc151410 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f18b40090b0 tx=0x7f18b40046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.132+0000 7f18c2400640 1 -- 172.21.15.7:0/4061526893 shutdown_connections 2024-04-25T11:22:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.132+0000 7f18c2400640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f189c0744b0 0x7f189c0769c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.132+0000 7f18c2400640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18bc0d0800 0x7f18bc151410 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.132+0000 7f18c2400640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18bc0cfa80 0x7f18bc1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.132+0000 7f18c2400640 1 --2- 172.21.15.7:0/4061526893 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18bc070630 0x7f18bc155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.132+0000 7f18c2400640 1 -- 172.21.15.7:0/4061526893 >> 172.21.15.7:0/4061526893 conn(0x7f18bc074140 msgr2=0x7f18bc15f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.133+0000 7f18c2400640 1 -- 172.21.15.7:0/4061526893 shutdown_connections 2024-04-25T11:22:49.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.133+0000 7f18c2400640 1 -- 172.21.15.7:0/4061526893 wait complete. 2024-04-25T11:22:49.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T11:22:49.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.448+0000 7f2fc8a00640 1 Processor -- start 2024-04-25T11:22:49.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.449+0000 7f2fc8a00640 1 -- start start 2024-04-25T11:22:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.451+0000 7f2fc8a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc41416a0 0x7f2fc40b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.451+0000 7f2fc8a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc406c570 con 0x7f2fc415eed0 2024-04-25T11:22:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.451+0000 7f2fc8a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc406c850 con 0x7f2fc4141290 2024-04-25T11:22:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.451+0000 7f2fc8a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc406cb30 con 0x7f2fc41416a0 2024-04-25T11:22:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.452+0000 7f2fc2800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2fc415eed0 0x7f2fc40cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33854/0 (socket says 172.21.15.7:33854) 2024-04-25T11:22:49.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.452+0000 7f2fc2800640 1 -- 172.21.15.7:0/2558231474 learned_addr learned my addr 172.21.15.7:0/2558231474 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.452+0000 7f2fc3200640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc41416a0 0x7f2fc40b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.452+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1717517185 0 0) 0x7f2fc406c570 con 0x7f2fc415eed0 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.452+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fa8003520 con 0x7f2fc415eed0 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3065038306 0 0) 0x7f2fa8003520 con 0x7f2fc415eed0 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc41416a0 msgr2=0x7f2fc40b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc41416a0 0x7f2fc40b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 >> v1:172.21.15.87:6789/0 conn(0x7f2fc4141290 legacy=0x7f2fc406d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc406ce10 con 0x7f2fc415eed0 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f2fb4001a10 con 0x7f2fc415eed0 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 >> v1:172.21.15.7:6789/0 conn(0x7f2fc415eed0 legacy=0x7f2fc40cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 0x7f2fa8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:49.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc41416a0 0x7f2fa8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8004d40 0x7f2fa80051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc406cb30 con 0x7f2fa80039e0 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa800bb80 con 0x7f2fa8004d40 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa800bd80 con 0x7f2fc41416a0 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.453+0000 7f2fc2800640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 0x7f2fa8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.454+0000 7f2fb9e00640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc41416a0 0x7f2fa8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.454+0000 7f2fc3200640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8004d40 0x7f2fa80051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.454+0000 7f2fc2800640 1 -- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc41416a0 msgr2=0x7f2fa8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.454+0000 7f2fc2800640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc41416a0 0x7f2fa8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.454+0000 7f2fc2800640 1 -- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8004d40 msgr2=0x7f2fa80051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.454+0000 7f2fc2800640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8004d40 0x7f2fa80051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.454+0000 7f2fc2800640 1 -- 172.21.15.7:0/2558231474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa8003520 con 0x7f2fa80039e0 2024-04-25T11:22:49.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.454+0000 7f2fc3200640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fa8004d40 0x7f2fa80051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:49.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.455+0000 7f2fc2800640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 0x7f2fa8003e50 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4074c70 tx=0x7f2fa8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264062e281372782 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:49.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.455+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb400a3d0 con 0x7f2fa80039e0 2024-04-25T11:22:49.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.455+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb402f070 con 0x7f2fa80039e0 2024-04-25T11:22:49.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.455+0000 7f2fc1e00640 1 -- 172.21.15.7:0/2558231474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb402bb60 con 0x7f2fa80039e0 2024-04-25T11:22:49.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.456+0000 7f2fc8a00640 1 -- 172.21.15.7:0/2558231474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 msgr2=0x7f2fa8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:49.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.456+0000 7f2fc8a00640 1 --2- 172.21.15.7:0/2558231474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 0x7f2fa8003e50 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4074c70 tx=0x7f2fa8005830 comp rx=0 tx=0).stop 2024-04-25T11:22:49.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.456+0000 7f2fc3200640 1 -- 172.21.15.7:0/2558231474 reap_dead start 2024-04-25T11:22:49.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.456+0000 7f2fc8a00640 1 -- 172.21.15.7:0/2558231474 shutdown_connections 2024-04-25T11:22:49.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.456+0000 7f2fc8a00640 1 -- 172.21.15.7:0/2558231474 >> 172.21.15.7:0/2558231474 conn(0x7f2fc40d1540 msgr2=0x7f2fc40d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:49.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.456+0000 7f2fc8a00640 1 -- 172.21.15.7:0/2558231474 shutdown_connections 2024-04-25T11:22:49.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.456+0000 7f2fc8a00640 1 -- 172.21.15.7:0/2558231474 wait complete. 2024-04-25T11:22:49.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.459+0000 7f2fc8a00640 1 Processor -- start 2024-04-25T11:22:49.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.459+0000 7f2fc8a00640 1 -- start start 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.460+0000 7f2fc8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 0x7f2fc4151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.460+0000 7f2fc8a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4152d90 0x7f2fc4156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.460+0000 7f2fc8a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc41f5a80 0x7f2fc41f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.460+0000 7f2fc8a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc406dd20 con 0x7f2fa80039e0 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.460+0000 7f2fc8a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc4141ab0 con 0x7f2fc41f5a80 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.460+0000 7f2fc8a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc406dea0 con 0x7f2fc4152d90 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.461+0000 7f2fc2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 0x7f2fc4151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.461+0000 7f2fc2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 0x7f2fc4151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47010/0 (socket says 172.21.15.7:47010) 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.461+0000 7f2fc2800640 1 -- 172.21.15.7:0/1347788314 learned_addr learned my addr 172.21.15.7:0/1347788314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.461+0000 7f2fc3200640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc41f5a80 0x7f2fc41f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.461+0000 7f2fb9e00640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4152d90 0x7f2fc4156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.461+0000 7f2fc2800640 1 -- 172.21.15.7:0/1347788314 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4152d90 msgr2=0x7f2fc4156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:49.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.461+0000 7f2fc2800640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4152d90 0x7f2fc4156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.461+0000 7f2fc2800640 1 -- 172.21.15.7:0/1347788314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc41f5a80 msgr2=0x7f2fc41f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:49.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.461+0000 7f2fc2800640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc41f5a80 0x7f2fc41f5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.461+0000 7f2fc2800640 1 -- 172.21.15.7:0/1347788314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc41f9f00 con 0x7f2fa80039e0 2024-04-25T11:22:49.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.462+0000 7f2fc2800640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 0x7f2fc4151ce0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f2fb402b410 tx=0x7f2fc406ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:49.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.462+0000 7f2fbb400640 1 -- 172.21.15.7:0/1347788314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb402b720 con 0x7f2fa80039e0 2024-04-25T11:22:49.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.462+0000 7f2fbb400640 1 -- 172.21.15.7:0/1347788314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb402b8c0 con 0x7f2fa80039e0 2024-04-25T11:22:49.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.462+0000 7f2fc8a00640 1 -- 172.21.15.7:0/1347788314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc41fa1e0 con 0x7f2fa80039e0 2024-04-25T11:22:49.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.462+0000 7f2fbb400640 1 -- 172.21.15.7:0/1347788314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb402bcb0 con 0x7f2fa80039e0 2024-04-25T11:22:49.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.462+0000 7f2fc8a00640 1 -- 172.21.15.7:0/1347788314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc41fa7a0 con 0x7f2fa80039e0 2024-04-25T11:22:49.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.463+0000 7f2fc8a00640 1 -- 172.21.15.7:0/1347788314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc4157de0 con 0x7f2fa80039e0 2024-04-25T11:22:49.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.465+0000 7f2fbb400640 1 -- 172.21.15.7:0/1347788314 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2fb403e070 con 0x7f2fa80039e0 2024-04-25T11:22:49.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.466+0000 7f2fbb400640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa00744b0 0x7f2fa00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:49.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.466+0000 7f2fbb400640 1 -- 172.21.15.7:0/1347788314 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f2fb40b6250 con 0x7f2fa80039e0 2024-04-25T11:22:49.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.467+0000 7f2fb9e00640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa00744b0 0x7f2fa00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:49.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.467+0000 7f2fb9e00640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa00744b0 0x7f2fa00769c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2fac007fd0 tx=0x7f2fac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:49.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.470+0000 7f2fbb400640 1 -- 172.21.15.7:0/1347788314 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2fc4157de0 con 0x7f2fa80039e0 2024-04-25T11:22:49.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.686+0000 7f2fc8a00640 1 -- 172.21.15.7:0/1347788314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2fc4050740 con 0x7f2fa80039e0 2024-04-25T11:22:49.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.687+0000 7f2fbb400640 1 -- 172.21.15.7:0/1347788314 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v156) ==== 80+0+0 (secure 0 0 0) 0x7f2fb404f020 con 0x7f2fa80039e0 2024-04-25T11:22:49.683 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-04-25T11:22:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.690+0000 7f2fc8a00640 1 -- 172.21.15.7:0/1347788314 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa00744b0 msgr2=0x7f2fa00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.690+0000 7f2fc8a00640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa00744b0 0x7f2fa00769c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2fac007fd0 tx=0x7f2fac009040 comp rx=0 tx=0).stop 2024-04-25T11:22:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.690+0000 7f2fc8a00640 1 -- 172.21.15.7:0/1347788314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 msgr2=0x7f2fc4151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.690+0000 7f2fc8a00640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 0x7f2fc4151ce0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f2fb402b410 tx=0x7f2fc406ca20 comp rx=0 tx=0).stop 2024-04-25T11:22:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.690+0000 7f2fc8a00640 1 -- 172.21.15.7:0/1347788314 shutdown_connections 2024-04-25T11:22:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.690+0000 7f2fc8a00640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2fa00744b0 0x7f2fa00769c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.690+0000 7f2fc8a00640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2fc41f5a80 0x7f2fc41f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.690+0000 7f2fc8a00640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2fc4152d90 0x7f2fc4156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.690+0000 7f2fc8a00640 1 --2- 172.21.15.7:0/1347788314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2fa80039e0 0x7f2fc4151ce0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.690+0000 7f2fc8a00640 1 -- 172.21.15.7:0/1347788314 >> 172.21.15.7:0/1347788314 conn(0x7f2fc40d1540 msgr2=0x7f2fc415f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.690+0000 7f2fc8a00640 1 -- 172.21.15.7:0/1347788314 shutdown_connections 2024-04-25T11:22:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:49.691+0000 7f2fc8a00640 1 -- 172.21.15.7:0/1347788314 wait complete. 2024-04-25T11:22:49.712 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T11:22:49.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:49.712 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-25T11:22:49.819 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:22:49.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:22:49.819 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T11:22:49.875 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:22:49.875 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:22:49.876 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:49.935 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:22:49.936 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T11:22:49.936 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:22:49.936 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:22:49.993 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:22:49.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:50.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.293+0000 7fe469000640 1 Processor -- start 2024-04-25T11:22:50.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.293+0000 7fe469000640 1 -- start start 2024-04-25T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.296+0000 7fe469000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4640b5220 0x7fe46415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.296+0000 7fe469000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4640af420 0x7fe4640af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.296+0000 7fe469000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4640cebb0 con 0x7fe4640aed40 2024-04-25T11:22:50.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.296+0000 7fe469000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe46415e1f0 con 0x7fe4640b5220 2024-04-25T11:22:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.296+0000 7fe469000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe4640afd70 con 0x7fe4640af420 2024-04-25T11:22:50.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.296+0000 7fe463400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe4640aed40 0x7fe4640ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38946/0 (socket says 172.21.15.7:38946) 2024-04-25T11:22:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.296+0000 7fe463400640 1 -- 172.21.15.7:0/3887310651 learned_addr learned my addr 172.21.15.7:0/3887310651 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.296+0000 7fe462a00640 1 --2- 172.21.15.7:0/3887310651 >> v2:172.21.15.7:3300/0 conn(0x7fe4640b5220 0x7fe46415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.296+0000 7fe463e00640 1 --2- 172.21.15.7:0/3887310651 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4640af420 0x7fe4640af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.297+0000 7fe462a00640 1 -- 172.21.15.7:0/3887310651 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4640af420 msgr2=0x7fe4640af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.297+0000 7fe462a00640 1 --2- 172.21.15.7:0/3887310651 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4640af420 0x7fe4640af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.297+0000 7fe462a00640 1 -- 172.21.15.7:0/3887310651 >> v1:172.21.15.7:6789/0 conn(0x7fe4640aed40 legacy=0x7fe4640ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.297+0000 7fe462a00640 1 -- 172.21.15.7:0/3887310651 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4640b64b0 con 0x7fe4640b5220 2024-04-25T11:22:50.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.297+0000 7fe462000640 1 -- 172.21.15.7:0/3887310651 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4068547028 0 0) 0x7fe454001110 con 0x7fe4640aed40 2024-04-25T11:22:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.298+0000 7fe462a00640 1 --2- 172.21.15.7:0/3887310651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4640b5220 0x7fe46415dcb0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fe4500097a0 tx=0x7fe45002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b260be009133f1ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.298+0000 7fe462000640 1 -- 172.21.15.7:0/3887310651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450037070 con 0x7fe4640b5220 2024-04-25T11:22:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.298+0000 7fe462000640 1 -- 172.21.15.7:0/3887310651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe450032440 con 0x7fe4640b5220 2024-04-25T11:22:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.298+0000 7fe462000640 1 -- 172.21.15.7:0/3887310651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450032810 con 0x7fe4640b5220 2024-04-25T11:22:50.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.299+0000 7fe469000640 1 -- 172.21.15.7:0/3887310651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4640b5220 msgr2=0x7fe46415dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.299+0000 7fe469000640 1 --2- 172.21.15.7:0/3887310651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4640b5220 0x7fe46415dcb0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fe4500097a0 tx=0x7fe45002a180 comp rx=0 tx=0).stop 2024-04-25T11:22:50.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.299+0000 7fe469000640 1 -- 172.21.15.7:0/3887310651 shutdown_connections 2024-04-25T11:22:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.299+0000 7fe469000640 1 --2- 172.21.15.7:0/3887310651 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4640af420 0x7fe4640af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.299+0000 7fe469000640 1 --2- 172.21.15.7:0/3887310651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4640b5220 0x7fe46415dcb0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.299+0000 7fe469000640 1 -- 172.21.15.7:0/3887310651 >> 172.21.15.7:0/3887310651 conn(0x7fe4640b3820 msgr2=0x7fe4640b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.299+0000 7fe469000640 1 -- 172.21.15.7:0/3887310651 shutdown_connections 2024-04-25T11:22:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.299+0000 7fe469000640 1 -- 172.21.15.7:0/3887310651 wait complete. 2024-04-25T11:22:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.302+0000 7fe469000640 1 Processor -- start 2024-04-25T11:22:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.302+0000 7fe469000640 1 -- start start 2024-04-25T11:22:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.303+0000 7fe469000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4640aed40 0x7fe464070670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.303+0000 7fe469000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4640af420 0x7fe4640cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.303+0000 7fe469000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4640b5220 0x7fe464160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.303+0000 7fe469000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe46406cbd0 con 0x7fe4640af420 2024-04-25T11:22:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.303+0000 7fe469000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe46406c8f0 con 0x7fe4640aed40 2024-04-25T11:22:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.303+0000 7fe469000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe46406ceb0 con 0x7fe4640b5220 2024-04-25T11:22:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.304+0000 7fe462a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4640af420 0x7fe4640cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.304+0000 7fe462a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4640af420 0x7fe4640cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49840/0 (socket says 172.21.15.7:49840) 2024-04-25T11:22:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.304+0000 7fe462a00640 1 -- 172.21.15.7:0/3456118994 learned_addr learned my addr 172.21.15.7:0/3456118994 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.304+0000 7fe462a00640 1 -- 172.21.15.7:0/3456118994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4640b5220 msgr2=0x7fe464160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.304+0000 7fe462a00640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4640b5220 0x7fe464160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.304+0000 7fe462a00640 1 -- 172.21.15.7:0/3456118994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4640aed40 msgr2=0x7fe464070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.304+0000 7fe463400640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4640aed40 0x7fe464070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.304+0000 7fe462a00640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4640aed40 0x7fe464070670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.304+0000 7fe462a00640 1 -- 172.21.15.7:0/3456118994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe464157460 con 0x7fe4640af420 2024-04-25T11:22:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.304+0000 7fe462a00640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4640af420 0x7fe4640cbcf0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe4500098c0 tx=0x7fe4500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.305+0000 7fe45b400640 1 -- 172.21.15.7:0/3456118994 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450037040 con 0x7fe4640af420 2024-04-25T11:22:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.305+0000 7fe45b400640 1 -- 172.21.15.7:0/3456118994 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe450042070 con 0x7fe4640af420 2024-04-25T11:22:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.305+0000 7fe469000640 1 -- 172.21.15.7:0/3456118994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4640cc840 con 0x7fe4640af420 2024-04-25T11:22:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.305+0000 7fe45b400640 1 -- 172.21.15.7:0/3456118994 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450030bb0 con 0x7fe4640af420 2024-04-25T11:22:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.305+0000 7fe469000640 1 -- 172.21.15.7:0/3456118994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4640cce00 con 0x7fe4640af420 2024-04-25T11:22:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.306+0000 7fe469000640 1 -- 172.21.15.7:0/3456118994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4640afef0 con 0x7fe4640af420 2024-04-25T11:22:50.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.307+0000 7fe45b400640 1 -- 172.21.15.7:0/3456118994 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe450047460 con 0x7fe4640af420 2024-04-25T11:22:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.308+0000 7fe45b400640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4440744e0 0x7fe4440769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.309+0000 7fe45b400640 1 -- 172.21.15.7:0/3456118994 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7fe4500b5e00 con 0x7fe4640af420 2024-04-25T11:22:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.309+0000 7fe463400640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4440744e0 0x7fe4440769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.309+0000 7fe463400640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4440744e0 0x7fe4440769f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe454000f60 tx=0x7fe454009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.313+0000 7fe45b400640 1 -- 172.21.15.7:0/3456118994 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe45004c020 con 0x7fe4640af420 2024-04-25T11:22:50.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.555+0000 7fe469000640 1 -- 172.21.15.7:0/3456118994 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe464050740 con 0x7fe4640af420 2024-04-25T11:22:50.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.557+0000 7fe45b400640 1 -- 172.21.15.7:0/3456118994 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5179 (secure 0 0 0) 0x7fe450032e20 con 0x7fe4640af420 2024-04-25T11:22:50.552 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:50.552 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7588,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2938174159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2938174159},{"type":"v1","addr":"172.21.15.87:6834","nonce":2938174159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/793358721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":793358721},{"type":"v1","addr":"172.21.15.87:6835","nonce":793358721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":9447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2654444897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2654444897},{"type":"v1","addr":"172.21.15.7:6835","nonce":2654444897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:22:32.315016+0000","modified":"2024-04-25T11:22:35.372928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/339418645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":339418645},{"type":"v1","addr":"172.21.15.7:6837","nonce":339418645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":11}]} 2024-04-25T11:22:50.552 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 116 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 -- 172.21.15.7:0/3456118994 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4440744e0 msgr2=0x7fe4440769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4440744e0 0x7fe4440769f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe454000f60 tx=0x7fe454009040 comp rx=0 tx=0).stop 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 -- 172.21.15.7:0/3456118994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4640af420 msgr2=0x7fe4640cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4640af420 0x7fe4640cbcf0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe4500098c0 tx=0x7fe4500046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 -- 172.21.15.7:0/3456118994 shutdown_connections 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe4440744e0 0x7fe4440769f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4640b5220 0x7fe464160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4640af420 0x7fe4640cbcf0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 --2- 172.21.15.7:0/3456118994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4640aed40 0x7fe464070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 -- 172.21.15.7:0/3456118994 >> 172.21.15.7:0/3456118994 conn(0x7fe4640b3820 msgr2=0x7fe4640ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 -- 172.21.15.7:0/3456118994 shutdown_connections 2024-04-25T11:22:50.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.560+0000 7fe469000640 1 -- 172.21.15.7:0/3456118994 wait complete. 2024-04-25T11:22:50.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.846+0000 7f1ef1800640 1 Processor -- start 2024-04-25T11:22:50.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.846+0000 7f1ef1800640 1 -- start start 2024-04-25T11:22:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.848+0000 7f1ef1800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1eec0aed40 0x7f1eec15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.848+0000 7f1ef1800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eec0cebb0 con 0x7f1eec0af420 2024-04-25T11:22:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.848+0000 7f1ef1800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eec0b64b0 con 0x7f1eec0b5220 2024-04-25T11:22:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.848+0000 7f1ef1800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1eec15e1f0 con 0x7f1eec0aed40 2024-04-25T11:22:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.848+0000 7f1ef0e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1eec0af420 0x7f1eec0af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38954/0 (socket says 172.21.15.7:38954) 2024-04-25T11:22:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.848+0000 7f1ef0e00640 1 -- 172.21.15.7:0/4200045162 learned_addr learned my addr 172.21.15.7:0/4200045162 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:50.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.848+0000 7f1eebe00640 1 --2- 172.21.15.7:0/4200045162 >> v2:172.21.15.7:3300/0 conn(0x7f1eec0aed40 0x7f1eec15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.849+0000 7f1eebe00640 1 -- 172.21.15.7:0/4200045162 >> v1:172.21.15.87:6789/0 conn(0x7f1eec0b5220 legacy=0x7f1eec0ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.849+0000 7f1eebe00640 1 -- 172.21.15.7:0/4200045162 >> v1:172.21.15.7:6789/0 conn(0x7f1eec0af420 legacy=0x7f1eec0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.849+0000 7f1eebe00640 1 -- 172.21.15.7:0/4200045162 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec06c610 con 0x7f1eec0aed40 2024-04-25T11:22:50.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.849+0000 7f1eeaa00640 1 -- 172.21.15.7:0/4200045162 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3988861943 0 0) 0x7f1ee4001490 con 0x7f1eec0af420 2024-04-25T11:22:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.850+0000 7f1eebe00640 1 --2- 172.21.15.7:0/4200045162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eec0aed40 0x7f1eec15dcb0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0097a0 tx=0x7f1edc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb6910736ff4273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.850+0000 7f1eeaa00640 1 -- 172.21.15.7:0/4200045162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc037070 con 0x7f1eec0aed40 2024-04-25T11:22:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.850+0000 7f1eeaa00640 1 -- 172.21.15.7:0/4200045162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1edc032440 con 0x7f1eec0aed40 2024-04-25T11:22:50.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.850+0000 7f1eeaa00640 1 -- 172.21.15.7:0/4200045162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc032810 con 0x7f1eec0aed40 2024-04-25T11:22:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.851+0000 7f1ef1800640 1 -- 172.21.15.7:0/4200045162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eec0aed40 msgr2=0x7f1eec15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.851+0000 7f1ef1800640 1 --2- 172.21.15.7:0/4200045162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eec0aed40 0x7f1eec15dcb0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0097a0 tx=0x7f1edc02a1e0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.851+0000 7f1ef1800640 1 -- 172.21.15.7:0/4200045162 shutdown_connections 2024-04-25T11:22:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.851+0000 7f1ef1800640 1 --2- 172.21.15.7:0/4200045162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eec0aed40 0x7f1eec15dcb0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.851+0000 7f1ef1800640 1 -- 172.21.15.7:0/4200045162 >> 172.21.15.7:0/4200045162 conn(0x7f1eec0b3820 msgr2=0x7f1eec0b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.851+0000 7f1ef1800640 1 -- 172.21.15.7:0/4200045162 shutdown_connections 2024-04-25T11:22:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.851+0000 7f1ef1800640 1 -- 172.21.15.7:0/4200045162 wait complete. 2024-04-25T11:22:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.853+0000 7f1ef1800640 1 Processor -- start 2024-04-25T11:22:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.854+0000 7f1ef1800640 1 -- start start 2024-04-25T11:22:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.855+0000 7f1ef1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eec0aed40 0x7f1eec0d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.855+0000 7f1ef1800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1eec0af420 0x7f1eec159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.855+0000 7f1ef1800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1eec0b5220 0x7f1eec14abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.855+0000 7f1ef1800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec06ceb0 con 0x7f1eec0aed40 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.855+0000 7f1ef1800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1eec06cbd0 con 0x7f1eec0af420 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.855+0000 7f1ef1800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1eec0b6110 con 0x7f1eec0b5220 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.855+0000 7f1eebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eec0aed40 0x7f1eec0d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.855+0000 7f1eebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eec0aed40 0x7f1eec0d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49850/0 (socket says 172.21.15.7:49850) 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.855+0000 7f1eebe00640 1 -- 172.21.15.7:0/3864260804 learned_addr learned my addr 172.21.15.7:0/3864260804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.856+0000 7f1eeb400640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1eec0af420 0x7f1eec159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.856+0000 7f1ef0e00640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1eec0b5220 0x7f1eec14abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.856+0000 7f1eebe00640 1 -- 172.21.15.7:0/3864260804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1eec0b5220 msgr2=0x7f1eec14abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.856+0000 7f1eebe00640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1eec0b5220 0x7f1eec14abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.856+0000 7f1eebe00640 1 -- 172.21.15.7:0/3864260804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1eec0af420 msgr2=0x7f1eec159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.856+0000 7f1eebe00640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1eec0af420 0x7f1eec159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:50.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.856+0000 7f1eebe00640 1 -- 172.21.15.7:0/3864260804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eec15e440 con 0x7f1eec0aed40 2024-04-25T11:22:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.856+0000 7f1eebe00640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eec0aed40 0x7f1eec0d0a70 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0097a0 tx=0x7f1edc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.857+0000 7f1ee3e00640 1 -- 172.21.15.7:0/3864260804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc037040 con 0x7f1eec0aed40 2024-04-25T11:22:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.857+0000 7f1ee3e00640 1 -- 172.21.15.7:0/3864260804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1edc004250 con 0x7f1eec0aed40 2024-04-25T11:22:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.857+0000 7f1ef1800640 1 -- 172.21.15.7:0/3864260804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eec15f1c0 con 0x7f1eec0aed40 2024-04-25T11:22:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.857+0000 7f1ee3e00640 1 -- 172.21.15.7:0/3864260804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc032e20 con 0x7f1eec0aed40 2024-04-25T11:22:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.857+0000 7f1ef1800640 1 -- 172.21.15.7:0/3864260804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eec15e6e0 con 0x7f1eec0aed40 2024-04-25T11:22:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.858+0000 7f1ef1800640 1 -- 172.21.15.7:0/3864260804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eec050740 con 0x7f1eec0aed40 2024-04-25T11:22:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.859+0000 7f1ee3e00640 1 -- 172.21.15.7:0/3864260804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f1edc030770 con 0x7f1eec0aed40 2024-04-25T11:22:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.860+0000 7f1ee3e00640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ecc0744b0 0x7f1ecc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.860+0000 7f1ee3e00640 1 -- 172.21.15.7:0/3864260804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f1edc0b64c0 con 0x7f1eec0aed40 2024-04-25T11:22:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.860+0000 7f1eeb400640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ecc0744b0 0x7f1ecc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:50.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.861+0000 7f1eeb400640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ecc0744b0 0x7f1ecc0769c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8006fd0 tx=0x7f1ed8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:50.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:50.865+0000 7f1ee3e00640 1 -- 172.21.15.7:0/3864260804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f1edc04c070 con 0x7f1eec0aed40 2024-04-25T11:22:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.061+0000 7f1ef1800640 1 -- 172.21.15.7:0/3864260804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1eec1f39b0 con 0x7f1eec0aed40 2024-04-25T11:22:51.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.063+0000 7f1ee3e00640 1 -- 172.21.15.7:0/3864260804 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16788 (secure 0 0 0) 0x7f1edc080ab0 con 0x7f1eec0aed40 2024-04-25T11:22:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:51.059 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:33.767424+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:22:30.472409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T11:22:31.299177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:51.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 -- 172.21.15.7:0/3864260804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ecc0744b0 msgr2=0x7f1ecc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ecc0744b0 0x7f1ecc0769c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8006fd0 tx=0x7f1ed8008040 comp rx=0 tx=0).stop 2024-04-25T11:22:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 -- 172.21.15.7:0/3864260804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eec0aed40 msgr2=0x7f1eec0d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eec0aed40 0x7f1eec0d0a70 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0097a0 tx=0x7f1edc0046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 -- 172.21.15.7:0/3864260804 shutdown_connections 2024-04-25T11:22:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f1ecc0744b0 0x7f1ecc0769c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1eec0b5220 0x7f1eec14abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1eec0af420 0x7f1eec159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 --2- 172.21.15.7:0/3864260804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eec0aed40 0x7f1eec0d0a70 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 -- 172.21.15.7:0/3864260804 >> 172.21.15.7:0/3864260804 conn(0x7f1eec0b3820 msgr2=0x7f1eec0ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 -- 172.21.15.7:0/3864260804 shutdown_connections 2024-04-25T11:22:51.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.066+0000 7f1ef1800640 1 -- 172.21.15.7:0/3864260804 wait complete. 2024-04-25T11:22:51.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T11:10:09.817225+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T11:22:30.472409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T11:22:31.299177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T11:22:51.087 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T11:22:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.370+0000 7f6d3f200640 1 Processor -- start 2024-04-25T11:22:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.370+0000 7f6d3f200640 1 -- start start 2024-04-25T11:22:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.372+0000 7f6d3f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d400d0480 0x7f6d400d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.372+0000 7f6d3f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d400ce570 con 0x7f6d401583c0 2024-04-25T11:22:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.373+0000 7f6d3f200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d400ce850 con 0x7f6d4006eea0 2024-04-25T11:22:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.373+0000 7f6d3f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6d4015f190 con 0x7f6d400d0480 2024-04-25T11:22:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.373+0000 7f6d3de00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6d401583c0 0x7f6d400cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38968/0 (socket says 172.21.15.7:38968) 2024-04-25T11:22:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.373+0000 7f6d3de00640 1 -- 172.21.15.7:0/4130376292 learned_addr learned my addr 172.21.15.7:0/4130376292 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.373+0000 7f6d37e00640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d400d0480 0x7f6d400d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.373+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1197743655 0 0) 0x7f6d400ce570 con 0x7f6d401583c0 2024-04-25T11:22:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.373+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d24003520 con 0x7f6d401583c0 2024-04-25T11:22:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4111496399 0 0) 0x7f6d24003520 con 0x7f6d401583c0 2024-04-25T11:22:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d400d0480 msgr2=0x7f6d400d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d400d0480 0x7f6d400d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 >> v1:172.21.15.87:6789/0 conn(0x7f6d4006eea0 legacy=0x7f6d400b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d400ceb30 con 0x7f6d401583c0 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f6d30001670 con 0x7f6d401583c0 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d37e00640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d400d0480 0x7f6d400d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 >> v1:172.21.15.7:6789/0 conn(0x7f6d401583c0 legacy=0x7f6d400cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d400d0480 0x7f6d240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d24004040 0x7f6d24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24004d90 0x7f6d24005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d4015f190 con 0x7f6d24004d90 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2400bc20 con 0x7f6d24004040 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2400be20 con 0x7f6d400d0480 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3e800640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24004d90 0x7f6d24005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d3de00640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d400d0480 0x7f6d240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.374+0000 7f6d37e00640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d24004040 0x7f6d24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.375+0000 7f6d3e800640 1 -- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d400d0480 msgr2=0x7f6d240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:51.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.375+0000 7f6d3e800640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d400d0480 0x7f6d240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.375+0000 7f6d3e800640 1 -- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d24004040 msgr2=0x7f6d24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:51.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.375+0000 7f6d3e800640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d24004040 0x7f6d24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.375+0000 7f6d3e800640 1 -- 172.21.15.7:0/4130376292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d24003520 con 0x7f6d24004d90 2024-04-25T11:22:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.376+0000 7f6d3e800640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24004d90 0x7f6d24005260 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f6d38001e50 tx=0x7f6d24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91159a6e7e478fce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.376+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38001a70 con 0x7f6d24004d90 2024-04-25T11:22:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.376+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d38003970 con 0x7f6d24004d90 2024-04-25T11:22:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.376+0000 7f6d3d400640 1 -- 172.21.15.7:0/4130376292 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38003d60 con 0x7f6d24004d90 2024-04-25T11:22:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.376+0000 7f6d3f200640 1 -- 172.21.15.7:0/4130376292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24004d90 msgr2=0x7f6d24005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.376+0000 7f6d3f200640 1 --2- 172.21.15.7:0/4130376292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d24004d90 0x7f6d24005260 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f6d38001e50 tx=0x7f6d24005830 comp rx=0 tx=0).stop 2024-04-25T11:22:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.377+0000 7f6d3e800640 1 -- 172.21.15.7:0/4130376292 reap_dead start 2024-04-25T11:22:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.377+0000 7f6d3f200640 1 -- 172.21.15.7:0/4130376292 shutdown_connections 2024-04-25T11:22:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.377+0000 7f6d3f200640 1 -- 172.21.15.7:0/4130376292 >> 172.21.15.7:0/4130376292 conn(0x7f6d400740f0 msgr2=0x7f6d4006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.377+0000 7f6d3f200640 1 -- 172.21.15.7:0/4130376292 shutdown_connections 2024-04-25T11:22:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.377+0000 7f6d3f200640 1 -- 172.21.15.7:0/4130376292 wait complete. 2024-04-25T11:22:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.379+0000 7f6d3f200640 1 Processor -- start 2024-04-25T11:22:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.379+0000 7f6d3f200640 1 -- start start 2024-04-25T11:22:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d3f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d24004d90 0x7f6d4014fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d3f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d4014ccc0 0x7f6d40150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d3f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d401511d0 0x7f6d40160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d3f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d400cfd20 con 0x7f6d4014ccc0 2024-04-25T11:22:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d3f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d400cf0f0 con 0x7f6d24004d90 2024-04-25T11:22:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d3f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6d400b0e70 con 0x7f6d401511d0 2024-04-25T11:22:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d37e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d4014ccc0 0x7f6d40150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d37e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d4014ccc0 0x7f6d40150c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49874/0 (socket says 172.21.15.7:49874) 2024-04-25T11:22:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d37e00640 1 -- 172.21.15.7:0/1795559989 learned_addr learned my addr 172.21.15.7:0/1795559989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d3de00640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d24004d90 0x7f6d4014fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d37e00640 1 -- 172.21.15.7:0/1795559989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d401511d0 msgr2=0x7f6d40160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d37e00640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d401511d0 0x7f6d40160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d37e00640 1 -- 172.21.15.7:0/1795559989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d24004d90 msgr2=0x7f6d4014fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d37e00640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d24004d90 0x7f6d4014fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.381+0000 7f6d37e00640 1 -- 172.21.15.7:0/1795559989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d401515e0 con 0x7f6d4014ccc0 2024-04-25T11:22:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.382+0000 7f6d37e00640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d4014ccc0 0x7f6d40150c90 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f6d28010810 tx=0x7f6d28010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.382+0000 7f6d36000640 1 -- 172.21.15.7:0/1795559989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2801a070 con 0x7f6d4014ccc0 2024-04-25T11:22:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.382+0000 7f6d36000640 1 -- 172.21.15.7:0/1795559989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d2800c930 con 0x7f6d4014ccc0 2024-04-25T11:22:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.382+0000 7f6d36000640 1 -- 172.21.15.7:0/1795559989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800f040 con 0x7f6d4014ccc0 2024-04-25T11:22:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.382+0000 7f6d3f200640 1 -- 172.21.15.7:0/1795559989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4006cdc0 con 0x7f6d4014ccc0 2024-04-25T11:22:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.382+0000 7f6d3f200640 1 -- 172.21.15.7:0/1795559989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d4015d3e0 con 0x7f6d4014ccc0 2024-04-25T11:22:51.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.383+0000 7f6d3f200640 1 -- 172.21.15.7:0/1795559989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d40070dd0 con 0x7f6d4014ccc0 2024-04-25T11:22:51.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.385+0000 7f6d36000640 1 -- 172.21.15.7:0/1795559989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6d2801d070 con 0x7f6d4014ccc0 2024-04-25T11:22:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.386+0000 7f6d36000640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6d1c0744b0 0x7f6d1c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.386+0000 7f6d36000640 1 -- 172.21.15.7:0/1795559989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f6d280957d0 con 0x7f6d4014ccc0 2024-04-25T11:22:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.386+0000 7f6d3de00640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6d1c0744b0 0x7f6d1c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.387+0000 7f6d3de00640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6d1c0744b0 0x7f6d1c0769c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6d30001bf0 tx=0x7f6d30000f50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:51.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.390+0000 7f6d36000640 1 -- 172.21.15.7:0/1795559989 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6d401515e0 con 0x7f6d4014ccc0 2024-04-25T11:22:51.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.593+0000 7f6d3f200640 1 -- 172.21.15.7:0/1795559989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6d401f2c60 con 0x7f6d4014ccc0 2024-04-25T11:22:51.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.594+0000 7f6d36000640 1 -- 172.21.15.7:0/1795559989 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) ==== 93+0+10 (secure 0 0 0) 0x7f6d2805fe50 con 0x7f6d4014ccc0 2024-04-25T11:22:51.589 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-04-25T11:22:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.596+0000 7f6d3f200640 1 -- 172.21.15.7:0/1795559989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6d1c0744b0 msgr2=0x7f6d1c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.596+0000 7f6d3f200640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6d1c0744b0 0x7f6d1c0769c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f6d30001bf0 tx=0x7f6d30000f50 comp rx=0 tx=0).stop 2024-04-25T11:22:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.596+0000 7f6d3f200640 1 -- 172.21.15.7:0/1795559989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d4014ccc0 msgr2=0x7f6d40150c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.596+0000 7f6d3f200640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d4014ccc0 0x7f6d40150c90 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f6d28010810 tx=0x7f6d28010cd0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.597+0000 7f6d3f200640 1 -- 172.21.15.7:0/1795559989 shutdown_connections 2024-04-25T11:22:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.597+0000 7f6d3f200640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6d1c0744b0 0x7f6d1c0769c0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.597+0000 7f6d3f200640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d401511d0 0x7f6d40160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.597+0000 7f6d3f200640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d4014ccc0 0x7f6d40150c90 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.597+0000 7f6d3f200640 1 --2- 172.21.15.7:0/1795559989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d24004d90 0x7f6d4014fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.597+0000 7f6d3f200640 1 -- 172.21.15.7:0/1795559989 >> 172.21.15.7:0/1795559989 conn(0x7f6d400740f0 msgr2=0x7f6d400b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.597+0000 7f6d3f200640 1 -- 172.21.15.7:0/1795559989 shutdown_connections 2024-04-25T11:22:51.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.597+0000 7f6d3f200640 1 -- 172.21.15.7:0/1795559989 wait complete. 2024-04-25T11:22:51.617 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T11:22:51.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.897+0000 7ff462600640 1 Processor -- start 2024-04-25T11:22:51.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.897+0000 7ff462600640 1 -- start start 2024-04-25T11:22:51.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.900+0000 7ff462600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff45c0d0890 0x7ff45c0b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.900+0000 7ff462600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff45c0d0ca0 con 0x7ff45c1583c0 2024-04-25T11:22:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.900+0000 7ff462600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff45c0ce570 con 0x7ff45c0d0480 2024-04-25T11:22:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.900+0000 7ff462600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff45c15f190 con 0x7ff45c0d0890 2024-04-25T11:22:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.900+0000 7ff461200640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff45c1583c0 0x7ff45c0cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38980/0 (socket says 172.21.15.7:38980) 2024-04-25T11:22:51.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.900+0000 7ff461200640 1 -- 172.21.15.7:0/1497233794 learned_addr learned my addr 172.21.15.7:0/1497233794 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:51.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1322644463 0 0) 0x7ff450001110 con 0x7ff45c1583c0 2024-04-25T11:22:51.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff444003520 con 0x7ff45c1583c0 2024-04-25T11:22:51.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff461c00640 1 --2- 172.21.15.7:0/1497233794 >> v2:172.21.15.87:3300/0 conn(0x7ff45c0d0890 0x7ff45c0b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3711969108 0 0) 0x7ff444003520 con 0x7ff45c1583c0 2024-04-25T11:22:51.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 >> v2:172.21.15.87:3300/0 conn(0x7ff45c0d0890 msgr2=0x7ff45c0b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:51.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 --2- 172.21.15.7:0/1497233794 >> v2:172.21.15.87:3300/0 conn(0x7ff45c0d0890 0x7ff45c0b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 >> v1:172.21.15.87:6789/0 conn(0x7ff45c0d0480 legacy=0x7ff45c06eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:51.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c0ce850 con 0x7ff45c1583c0 2024-04-25T11:22:51.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7ff450001840 con 0x7ff45c1583c0 2024-04-25T11:22:51.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 >> v1:172.21.15.7:6789/0 conn(0x7ff45c1583c0 legacy=0x7ff45c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4440039b0 0x7ff444003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff444004800 0x7ff444004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff444005350 0x7ff44400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c15f190 con 0x7ff444005350 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff44400c280 con 0x7ff4440039b0 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff44400c480 con 0x7ff444004800 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.901+0000 7ff461c00640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff444005350 0x7ff44400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.902+0000 7ff461200640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4440039b0 0x7ff444003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.902+0000 7ff45be00640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff444004800 0x7ff444004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.902+0000 7ff461c00640 1 -- 172.21.15.7:0/1497233794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff444004800 msgr2=0x7ff444004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.902+0000 7ff461c00640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff444004800 0x7ff444004c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.902+0000 7ff461c00640 1 -- 172.21.15.7:0/1497233794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4440039b0 msgr2=0x7ff444003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.902+0000 7ff461c00640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4440039b0 0x7ff444003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.902+0000 7ff461c00640 1 -- 172.21.15.7:0/1497233794 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff444003520 con 0x7ff444005350 2024-04-25T11:22:51.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.902+0000 7ff461200640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4440039b0 0x7ff444003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.903+0000 7ff461c00640 1 -- 172.21.15.7:0/1497233794 reap_dead start 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.903+0000 7ff461c00640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff444005350 0x7ff44400b9c0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff454007330 tx=0x7ff444005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9c6fc7726f2b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.903+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454009be0 con 0x7ff444005350 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.903+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff454003b90 con 0x7ff444005350 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.903+0000 7ff45b400640 1 -- 172.21.15.7:0/1497233794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454004020 con 0x7ff444005350 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.904+0000 7ff462600640 1 -- 172.21.15.7:0/1497233794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff444005350 msgr2=0x7ff44400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.904+0000 7ff462600640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff444005350 0x7ff44400b9c0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff454007330 tx=0x7ff444005830 comp rx=0 tx=0).stop 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.904+0000 7ff462600640 1 -- 172.21.15.7:0/1497233794 shutdown_connections 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.904+0000 7ff462600640 1 --2- 172.21.15.7:0/1497233794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff444005350 0x7ff44400b9c0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.904+0000 7ff462600640 1 -- 172.21.15.7:0/1497233794 >> 172.21.15.7:0/1497233794 conn(0x7ff45c0740f0 msgr2=0x7ff45c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.904+0000 7ff462600640 1 -- 172.21.15.7:0/1497233794 shutdown_connections 2024-04-25T11:22:51.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.904+0000 7ff462600640 1 -- 172.21.15.7:0/1497233794 wait complete. 2024-04-25T11:22:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.906+0000 7ff462600640 1 Processor -- start 2024-04-25T11:22:51.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.907+0000 7ff462600640 1 -- start start 2024-04-25T11:22:51.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.908+0000 7ff462600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff444005350 0x7ff45c14d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.908+0000 7ff462600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c155a40 0x7ff45c15d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.908+0000 7ff462600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff45c14fc30 0x7ff45c150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.908+0000 7ff462600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c0cf0f0 con 0x7ff45c155a40 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.908+0000 7ff462600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c0cee10 con 0x7ff45c14fc30 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.908+0000 7ff462600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff45c0cf610 con 0x7ff444005350 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.908+0000 7ff45be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c155a40 0x7ff45c15d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.908+0000 7ff45be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c155a40 0x7ff45c15d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49890/0 (socket says 172.21.15.7:49890) 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.908+0000 7ff45be00640 1 -- 172.21.15.7:0/2110079161 learned_addr learned my addr 172.21.15.7:0/2110079161 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.909+0000 7ff461200640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff444005350 0x7ff45c14d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.909+0000 7ff45be00640 1 -- 172.21.15.7:0/2110079161 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff444005350 msgr2=0x7ff45c14d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.909+0000 7ff45be00640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff444005350 0x7ff45c14d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.909+0000 7ff45be00640 1 -- 172.21.15.7:0/2110079161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff45c14fc30 msgr2=0x7ff45c150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.909+0000 7ff461c00640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff45c14fc30 0x7ff45c150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.909+0000 7ff45be00640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff45c14fc30 0x7ff45c150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:51.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.909+0000 7ff45be00640 1 -- 172.21.15.7:0/2110079161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c150580 con 0x7ff45c155a40 2024-04-25T11:22:51.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.909+0000 7ff461c00640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff45c14fc30 0x7ff45c150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:51.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.909+0000 7ff45be00640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c155a40 0x7ff45c15d460 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff44800d4b0 tx=0x7ff44800d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:51.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.910+0000 7ff458c00640 1 -- 172.21.15.7:0/2110079161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448007230 con 0x7ff45c155a40 2024-04-25T11:22:51.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.910+0000 7ff458c00640 1 -- 172.21.15.7:0/2110079161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4480073d0 con 0x7ff45c155a40 2024-04-25T11:22:51.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.910+0000 7ff462600640 1 -- 172.21.15.7:0/2110079161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c15cb50 con 0x7ff45c155a40 2024-04-25T11:22:51.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.910+0000 7ff458c00640 1 -- 172.21.15.7:0/2110079161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4480067d0 con 0x7ff45c155a40 2024-04-25T11:22:51.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.910+0000 7ff462600640 1 -- 172.21.15.7:0/2110079161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c14e040 con 0x7ff45c155a40 2024-04-25T11:22:51.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.911+0000 7ff462600640 1 -- 172.21.15.7:0/2110079161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c15d070 con 0x7ff45c155a40 2024-04-25T11:22:51.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.913+0000 7ff458c00640 1 -- 172.21.15.7:0/2110079161 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7ff448006970 con 0x7ff45c155a40 2024-04-25T11:22:51.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.914+0000 7ff458c00640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff43c074480 0x7ff43c076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.914+0000 7ff458c00640 1 -- 172.21.15.7:0/2110079161 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7ff448095270 con 0x7ff45c155a40 2024-04-25T11:22:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.914+0000 7ff461200640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff43c074480 0x7ff43c076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:51.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.915+0000 7ff461200640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff43c074480 0x7ff43c076990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff450001eb0 tx=0x7ff450000f50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:51.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:51.919+0000 7ff458c00640 1 -- 172.21.15.7:0/2110079161 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7ff45c15d070 con 0x7ff45c155a40 2024-04-25T11:22:52.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.116+0000 7ff462600640 1 -- 172.21.15.7:0/2110079161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff45c050740 con 0x7ff45c155a40 2024-04-25T11:22:52.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.117+0000 7ff458c00640 1 -- 172.21.15.7:0/2110079161 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v156) ==== 104+0+11 (secure 0 0 0) 0x7ff45c150580 con 0x7ff45c155a40 2024-04-25T11:22:52.112 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-04-25T11:22:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 -- 172.21.15.7:0/2110079161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff43c074480 msgr2=0x7ff43c076990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff43c074480 0x7ff43c076990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff450001eb0 tx=0x7ff450000f50 comp rx=0 tx=0).stop 2024-04-25T11:22:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 -- 172.21.15.7:0/2110079161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c155a40 msgr2=0x7ff45c15d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c155a40 0x7ff45c15d460 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff44800d4b0 tx=0x7ff44800d970 comp rx=0 tx=0).stop 2024-04-25T11:22:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 -- 172.21.15.7:0/2110079161 shutdown_connections 2024-04-25T11:22:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7ff43c074480 0x7ff43c076990 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff45c14fc30 0x7ff45c150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff45c155a40 0x7ff45c15d460 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 --2- 172.21.15.7:0/2110079161 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff444005350 0x7ff45c14d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 -- 172.21.15.7:0/2110079161 >> 172.21.15.7:0/2110079161 conn(0x7ff45c0740f0 msgr2=0x7ff45c154110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:52.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 -- 172.21.15.7:0/2110079161 shutdown_connections 2024-04-25T11:22:52.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.120+0000 7ff462600640 1 -- 172.21.15.7:0/2110079161 wait complete. 2024-04-25T11:22:52.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T11:22:52.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.420+0000 7f8937a00640 1 Processor -- start 2024-04-25T11:22:52.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.420+0000 7f8937a00640 1 -- start start 2024-04-25T11:22:52.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.423+0000 7f8937a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f89300b0470 0x7f893006f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.423+0000 7f8937a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893006f8e0 0x7f89300aea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.423+0000 7f8937a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f893015e3c0 con 0x7f893015dfb0 2024-04-25T11:22:52.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.423+0000 7f8937a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f89300cf800 con 0x7f89300b0470 2024-04-25T11:22:52.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.423+0000 7f8937a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89300d1250 con 0x7f893006f8e0 2024-04-25T11:22:52.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.423+0000 7f8935000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f893015dfb0 0x7f89300d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38982/0 (socket says 172.21.15.7:38982) 2024-04-25T11:22:52.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.423+0000 7f8935000640 1 -- 172.21.15.7:0/784844163 learned_addr learned my addr 172.21.15.7:0/784844163 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.424+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 642356013 0 0) 0x7f893015e3c0 con 0x7f893015dfb0 2024-04-25T11:22:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.424+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8914003520 con 0x7f893015dfb0 2024-04-25T11:22:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.424+0000 7f8935a00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893006f8e0 0x7f89300aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.424+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3252062407 0 0) 0x7f8914003520 con 0x7f893015dfb0 2024-04-25T11:22:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.424+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893006f8e0 msgr2=0x7f89300aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.424+0000 7f892fe00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893006f8e0 0x7f89300aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.424+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 >> v2:172.21.15.87:3300/0 conn(0x7f89300b0470 msgr2=0x7f893006f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.424+0000 7f8927e00640 1 --2- 172.21.15.7:0/784844163 >> v2:172.21.15.87:3300/0 conn(0x7f89300b0470 0x7f893006f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f892fe00640 1 --2- 172.21.15.7:0/784844163 >> v2:172.21.15.87:3300/0 conn(0x7f89300b0470 0x7f893006f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89300cfaa0 con 0x7f893015dfb0 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f89200019f0 con 0x7f893015dfb0 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f8927e00640 1 --2- 172.21.15.7:0/784844163 >> v2:172.21.15.87:3300/0 conn(0x7f89300b0470 0x7f893006f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 >> v1:172.21.15.7:6789/0 conn(0x7f893015dfb0 legacy=0x7f89300d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f892fe00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893006f8e0 0x7f89140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f892fe00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8914004030 0x7f8914004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f892fe00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8914004d90 0x7f8914005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89300cf800 con 0x7f8914004d90 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89300d1250 con 0x7f8914004030 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f891400bca0 con 0x7f893006f8e0 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f8935a00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8914004d90 0x7f8914005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.425+0000 7f8927e00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8914004030 0x7f8914004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.426+0000 7f8935000640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893006f8e0 0x7f89140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.426+0000 7f8935a00640 1 -- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893006f8e0 msgr2=0x7f89140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.426+0000 7f8935a00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893006f8e0 0x7f89140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.426+0000 7f8935a00640 1 -- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8914004030 msgr2=0x7f8914004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.426+0000 7f8935a00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8914004030 0x7f8914004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.426+0000 7f8935a00640 1 -- 172.21.15.7:0/784844163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8914003520 con 0x7f8914004d90 2024-04-25T11:22:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.426+0000 7f8927e00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8914004030 0x7f8914004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:52.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.426+0000 7f8935000640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893006f8e0 0x7f89140039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:52.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.427+0000 7f8935a00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8914004d90 0x7f8914005220 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f892800ba30 tx=0x7f8914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6b82039adbcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:52.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.427+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928030070 con 0x7f8914004d90 2024-04-25T11:22:52.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.427+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892802ade0 con 0x7f8914004d90 2024-04-25T11:22:52.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.427+0000 7f892fe00640 1 -- 172.21.15.7:0/784844163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892800b600 con 0x7f8914004d90 2024-04-25T11:22:52.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.428+0000 7f8937a00640 1 -- 172.21.15.7:0/784844163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8914004d90 msgr2=0x7f8914005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.428+0000 7f8937a00640 1 --2- 172.21.15.7:0/784844163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8914004d90 0x7f8914005220 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f892800ba30 tx=0x7f8914005830 comp rx=0 tx=0).stop 2024-04-25T11:22:52.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.428+0000 7f8935a00640 1 -- 172.21.15.7:0/784844163 reap_dead start 2024-04-25T11:22:52.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.428+0000 7f8937a00640 1 -- 172.21.15.7:0/784844163 shutdown_connections 2024-04-25T11:22:52.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.428+0000 7f8937a00640 1 -- 172.21.15.7:0/784844163 >> 172.21.15.7:0/784844163 conn(0x7f89300742c0 msgr2=0x7f893006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:52.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.428+0000 7f8937a00640 1 -- 172.21.15.7:0/784844163 shutdown_connections 2024-04-25T11:22:52.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.428+0000 7f8937a00640 1 -- 172.21.15.7:0/784844163 wait complete. 2024-04-25T11:22:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.430+0000 7f8937a00640 1 Processor -- start 2024-04-25T11:22:52.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.431+0000 7f8937a00640 1 -- start start 2024-04-25T11:22:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.432+0000 7f8937a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8914004d90 0x7f89301537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.432+0000 7f8937a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89300b2e90 0x7f89300cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.432+0000 7f8937a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89300cb6a0 0x7f89300cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.432+0000 7f8937a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f89300d0340 con 0x7f89300cb6a0 2024-04-25T11:22:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.432+0000 7f8937a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89300d0060 con 0x7f8914004d90 2024-04-25T11:22:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.432+0000 7f8937a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89300d0620 con 0x7f89300b2e90 2024-04-25T11:22:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.432+0000 7f8935a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89300cb6a0 0x7f89300cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.433+0000 7f8935a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89300cb6a0 0x7f89300cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49904/0 (socket says 172.21.15.7:49904) 2024-04-25T11:22:52.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.433+0000 7f8935a00640 1 -- 172.21.15.7:0/1838371158 learned_addr learned my addr 172.21.15.7:0/1838371158 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:52.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.433+0000 7f8935000640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8914004d90 0x7f89301537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.433+0000 7f8935a00640 1 -- 172.21.15.7:0/1838371158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89300b2e90 msgr2=0x7f89300cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.433+0000 7f8935a00640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89300b2e90 0x7f89300cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.433+0000 7f8935a00640 1 -- 172.21.15.7:0/1838371158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8914004d90 msgr2=0x7f89301537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.433+0000 7f8935a00640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8914004d90 0x7f89301537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.433+0000 7f8935a00640 1 -- 172.21.15.7:0/1838371158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89300cbab0 con 0x7f89300cb6a0 2024-04-25T11:22:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.433+0000 7f8935a00640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89300cb6a0 0x7f89300cc810 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f892800bb50 tx=0x7f89280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.434+0000 7f892d600640 1 -- 172.21.15.7:0/1838371158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928043030 con 0x7f89300cb6a0 2024-04-25T11:22:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.434+0000 7f892d600640 1 -- 172.21.15.7:0/1838371158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892802abe0 con 0x7f89300cb6a0 2024-04-25T11:22:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.434+0000 7f892d600640 1 -- 172.21.15.7:0/1838371158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892800b600 con 0x7f89300cb6a0 2024-04-25T11:22:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.434+0000 7f8937a00640 1 -- 172.21.15.7:0/1838371158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89301606a0 con 0x7f89300cb6a0 2024-04-25T11:22:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.434+0000 7f8937a00640 1 -- 172.21.15.7:0/1838371158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8930160c60 con 0x7f89300cb6a0 2024-04-25T11:22:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.435+0000 7f8937a00640 1 -- 172.21.15.7:0/1838371158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8930160410 con 0x7f89300cb6a0 2024-04-25T11:22:52.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.437+0000 7f892d600640 1 -- 172.21.15.7:0/1838371158 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8928046070 con 0x7f89300cb6a0 2024-04-25T11:22:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.438+0000 7f892d600640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f890c0744b0 0x7f890c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.438+0000 7f892d600640 1 -- 172.21.15.7:0/1838371158 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f89280b5610 con 0x7f89300cb6a0 2024-04-25T11:22:52.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.438+0000 7f8935000640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f890c0744b0 0x7f890c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.439+0000 7f8935000640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f890c0744b0 0x7f890c0769c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8920007c40 tx=0x7f8920009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:52.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.443+0000 7f892d600640 1 -- 172.21.15.7:0/1838371158 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f89300cbab0 con 0x7f89300cb6a0 2024-04-25T11:22:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.639+0000 7f8937a00640 1 -- 172.21.15.7:0/1838371158 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8930050740 con 0x7f89300cb6a0 2024-04-25T11:22:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.640+0000 7f892d600640 1 -- 172.21.15.7:0/1838371158 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v156) ==== 100+0+11 (secure 0 0 0) 0x7f892807fc90 con 0x7f89300cb6a0 2024-04-25T11:22:52.635 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-04-25T11:22:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.642+0000 7f8937a00640 1 -- 172.21.15.7:0/1838371158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f890c0744b0 msgr2=0x7f890c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.642+0000 7f8937a00640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f890c0744b0 0x7f890c0769c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8920007c40 tx=0x7f8920009040 comp rx=0 tx=0).stop 2024-04-25T11:22:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.642+0000 7f8937a00640 1 -- 172.21.15.7:0/1838371158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89300cb6a0 msgr2=0x7f89300cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.642+0000 7f8937a00640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89300cb6a0 0x7f89300cc810 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f892800bb50 tx=0x7f89280046c0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.643+0000 7f8937a00640 1 -- 172.21.15.7:0/1838371158 shutdown_connections 2024-04-25T11:22:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.643+0000 7f8937a00640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f890c0744b0 0x7f890c0769c0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.643+0000 7f8937a00640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89300cb6a0 0x7f89300cc810 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.643+0000 7f8937a00640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89300b2e90 0x7f89300cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.643+0000 7f8937a00640 1 --2- 172.21.15.7:0/1838371158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8914004d90 0x7f89301537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.643+0000 7f8937a00640 1 -- 172.21.15.7:0/1838371158 >> 172.21.15.7:0/1838371158 conn(0x7f89300742c0 msgr2=0x7f89300b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.643+0000 7f8937a00640 1 -- 172.21.15.7:0/1838371158 shutdown_connections 2024-04-25T11:22:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.643+0000 7f8937a00640 1 -- 172.21.15.7:0/1838371158 wait complete. 2024-04-25T11:22:52.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:52.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.952+0000 7f0fc8c00640 1 Processor -- start 2024-04-25T11:22:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.952+0000 7f0fc8c00640 1 -- start start 2024-04-25T11:22:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.954+0000 7f0fc8c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0fc40aed40 0x7f0fc415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.954+0000 7f0fc8c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0fc40af420 0x7f0fc40af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.954+0000 7f0fc8c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc40cebb0 con 0x7f0fc40b5220 2024-04-25T11:22:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.954+0000 7f0fc8c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc415e1f0 con 0x7f0fc40aed40 2024-04-25T11:22:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.954+0000 7f0fc8c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc40afd70 con 0x7f0fc40af420 2024-04-25T11:22:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.955+0000 7f0fc3400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0fc40aed40 0x7f0fc415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.955+0000 7f0fbaa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0fc40b5220 0x7f0fc40ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38988/0 (socket says 172.21.15.7:38988) 2024-04-25T11:22:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.955+0000 7f0fbaa00640 1 -- 172.21.15.7:0/1397188928 learned_addr learned my addr 172.21.15.7:0/1397188928 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.955+0000 7f0fc3e00640 1 --2- 172.21.15.7:0/1397188928 >> v2:172.21.15.87:3300/0 conn(0x7f0fc40af420 0x7f0fc40af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.956+0000 7f0fc3400640 1 -- 172.21.15.7:0/1397188928 >> v2:172.21.15.87:3300/0 conn(0x7f0fc40af420 msgr2=0x7f0fc40af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.956+0000 7f0fc3400640 1 --2- 172.21.15.7:0/1397188928 >> v2:172.21.15.87:3300/0 conn(0x7f0fc40af420 0x7f0fc40af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.956+0000 7f0fc3400640 1 -- 172.21.15.7:0/1397188928 >> v1:172.21.15.7:6789/0 conn(0x7f0fc40b5220 legacy=0x7f0fc40ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.956+0000 7f0fc3400640 1 -- 172.21.15.7:0/1397188928 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc40b64b0 con 0x7f0fc40aed40 2024-04-25T11:22:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.956+0000 7f0fc2a00640 1 -- 172.21.15.7:0/1397188928 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031292168 0 0) 0x7f0fac001110 con 0x7f0fc40b5220 2024-04-25T11:22:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.956+0000 7f0fc3e00640 1 --2- 172.21.15.7:0/1397188928 >> v2:172.21.15.87:3300/0 conn(0x7f0fc40af420 0x7f0fc40af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:22:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.958+0000 7f0fc3400640 1 --2- 172.21.15.7:0/1397188928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fc40aed40 0x7f0fc415dcb0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4072ec0 tx=0x7f0fb402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2a17588220dfed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.958+0000 7f0fc2a00640 1 -- 172.21.15.7:0/1397188928 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4037070 con 0x7f0fc40aed40 2024-04-25T11:22:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.958+0000 7f0fc2a00640 1 -- 172.21.15.7:0/1397188928 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb4032440 con 0x7f0fc40aed40 2024-04-25T11:22:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.958+0000 7f0fc2a00640 1 -- 172.21.15.7:0/1397188928 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4032810 con 0x7f0fc40aed40 2024-04-25T11:22:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.959+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1397188928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fc40aed40 msgr2=0x7f0fc415dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.959+0000 7f0fc8c00640 1 --2- 172.21.15.7:0/1397188928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fc40aed40 0x7f0fc415dcb0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4072ec0 tx=0x7f0fb402a1a0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.959+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1397188928 shutdown_connections 2024-04-25T11:22:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.959+0000 7f0fc8c00640 1 --2- 172.21.15.7:0/1397188928 >> v2:172.21.15.87:3300/0 conn(0x7f0fc40af420 0x7f0fc40af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.959+0000 7f0fc8c00640 1 --2- 172.21.15.7:0/1397188928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fc40aed40 0x7f0fc415dcb0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.959+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1397188928 >> 172.21.15.7:0/1397188928 conn(0x7f0fc40b3820 msgr2=0x7f0fc40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.959+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1397188928 shutdown_connections 2024-04-25T11:22:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.959+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1397188928 wait complete. 2024-04-25T11:22:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.961+0000 7f0fc8c00640 1 Processor -- start 2024-04-25T11:22:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.962+0000 7f0fc8c00640 1 -- start start 2024-04-25T11:22:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.963+0000 7f0fc8c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc40aed40 0x7f0fc40d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.963+0000 7f0fc8c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fc40af420 0x7f0fc4149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.963+0000 7f0fc8c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fc40b5220 0x7f0fc415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.963+0000 7f0fc8c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc406cbd0 con 0x7f0fc40af420 2024-04-25T11:22:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.963+0000 7f0fc8c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc406c8f0 con 0x7f0fc40aed40 2024-04-25T11:22:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.963+0000 7f0fc8c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc406ceb0 con 0x7f0fc40b5220 2024-04-25T11:22:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.963+0000 7f0fbaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fc40af420 0x7f0fc4149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.963+0000 7f0fbaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fc40af420 0x7f0fc4149fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49922/0 (socket says 172.21.15.7:49922) 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.963+0000 7f0fbaa00640 1 -- 172.21.15.7:0/1663641085 learned_addr learned my addr 172.21.15.7:0/1663641085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.964+0000 7f0fc3e00640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fc40b5220 0x7f0fc415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.964+0000 7f0fbaa00640 1 -- 172.21.15.7:0/1663641085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fc40b5220 msgr2=0x7f0fc415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.964+0000 7f0fbaa00640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fc40b5220 0x7f0fc415e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.964+0000 7f0fbaa00640 1 -- 172.21.15.7:0/1663641085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc40aed40 msgr2=0x7f0fc40d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.964+0000 7f0fbaa00640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc40aed40 0x7f0fc40d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.964+0000 7f0fbaa00640 1 -- 172.21.15.7:0/1663641085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc4158fa0 con 0x7f0fc40af420 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.964+0000 7f0fbaa00640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fc40af420 0x7f0fc4149fc0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00c460 tx=0x7f0fac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.964+0000 7f0fbbe00640 1 -- 172.21.15.7:0/1663641085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0074e0 con 0x7f0fc40af420 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.965+0000 7f0fbbe00640 1 -- 172.21.15.7:0/1663641085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fac0047e0 con 0x7f0fc40af420 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.965+0000 7f0fbbe00640 1 -- 172.21.15.7:0/1663641085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac004bd0 con 0x7f0fc40af420 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.965+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1663641085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc4159280 con 0x7f0fc40af420 2024-04-25T11:22:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.965+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1663641085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc4159840 con 0x7f0fc40af420 2024-04-25T11:22:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.966+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1663641085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc4158d10 con 0x7f0fc40af420 2024-04-25T11:22:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.967+0000 7f0fbbe00640 1 -- 172.21.15.7:0/1663641085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0fac011070 con 0x7f0fc40af420 2024-04-25T11:22:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.968+0000 7f0fbbe00640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0fa00788c0 0x7f0fa007add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.968+0000 7f0fbbe00640 1 -- 172.21.15.7:0/1663641085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f0fac095020 con 0x7f0fc40af420 2024-04-25T11:22:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.969+0000 7f0fc3400640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0fa00788c0 0x7f0fa007add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.969+0000 7f0fc3400640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0fa00788c0 0x7f0fa007add0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4034000 tx=0x7f0fb402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:52.973+0000 7f0fbbe00640 1 -- 172.21.15.7:0/1663641085 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0fc40cebb0 con 0x7f0fc40af420 2024-04-25T11:22:53.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.223+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1663641085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fc4050740 con 0x7f0fc40af420 2024-04-25T11:22:53.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.224+0000 7f0fbbe00640 1 -- 172.21.15.7:0/1663641085 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5179 (secure 0 0 0) 0x7f0fac05f620 con 0x7f0fc40af420 2024-04-25T11:22:53.219 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:53.219 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7588,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2938174159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2938174159},{"type":"v1","addr":"172.21.15.87:6834","nonce":2938174159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/793358721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":793358721},{"type":"v1","addr":"172.21.15.87:6835","nonce":793358721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":9447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2654444897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2654444897},{"type":"v1","addr":"172.21.15.7:6835","nonce":2654444897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:22:32.315016+0000","modified":"2024-04-25T11:22:35.372928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/339418645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":339418645},{"type":"v1","addr":"172.21.15.7:6837","nonce":339418645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":11}]} 2024-04-25T11:22:53.220 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 116 2024-04-25T11:22:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1663641085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0fa00788c0 msgr2=0x7f0fa007add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0fa00788c0 0x7f0fa007add0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4034000 tx=0x7f0fb402c000 comp rx=0 tx=0).stop 2024-04-25T11:22:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1663641085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fc40af420 msgr2=0x7f0fc4149fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fc40af420 0x7f0fc4149fc0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00c460 tx=0x7f0fac00c920 comp rx=0 tx=0).stop 2024-04-25T11:22:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1663641085 shutdown_connections 2024-04-25T11:22:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0fa00788c0 0x7f0fa007add0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fc40b5220 0x7f0fc415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fc40af420 0x7f0fc4149fc0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 --2- 172.21.15.7:0/1663641085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc40aed40 0x7f0fc40d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1663641085 >> 172.21.15.7:0/1663641085 conn(0x7f0fc40b3820 msgr2=0x7f0fc40b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1663641085 shutdown_connections 2024-04-25T11:22:53.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.227+0000 7f0fc8c00640 1 -- 172.21.15.7:0/1663641085 wait complete. 2024-04-25T11:22:53.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.542+0000 7f06be800640 1 Processor -- start 2024-04-25T11:22:53.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.543+0000 7f06be800640 1 -- start start 2024-04-25T11:22:53.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.545+0000 7f06be800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06b80af540 0x7f06b809f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.545+0000 7f06be800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06b815bc40 0x7f06b80cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.545+0000 7f06be800640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b815f4e0 con 0x7f06b815ef90 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.545+0000 7f06be800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f06b8071190 con 0x7f06b815bc40 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.545+0000 7f06be800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f06b80b0560 con 0x7f06b80af540 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.546+0000 7f06b7e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06b815bc40 0x7f06b80cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.546+0000 7f06b7400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f06b815ef90 0x7f06b80aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38992/0 (socket says 172.21.15.7:38992) 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.546+0000 7f06b7400640 1 -- 172.21.15.7:0/4265386336 learned_addr learned my addr 172.21.15.7:0/4265386336 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.546+0000 7f06afe00640 1 --2- 172.21.15.7:0/4265386336 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06b80af540 0x7f06b809f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.546+0000 7f06b7e00640 1 -- 172.21.15.7:0/4265386336 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06b80af540 msgr2=0x7f06b809f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.546+0000 7f06b7e00640 1 --2- 172.21.15.7:0/4265386336 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06b80af540 0x7f06b809f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.546+0000 7f06b7e00640 1 -- 172.21.15.7:0/4265386336 >> v1:172.21.15.7:6789/0 conn(0x7f06b815ef90 legacy=0x7f06b80aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.546+0000 7f06b7e00640 1 -- 172.21.15.7:0/4265386336 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b80b0840 con 0x7f06b815bc40 2024-04-25T11:22:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.547+0000 7f06b7e00640 1 --2- 172.21.15.7:0/4265386336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06b815bc40 0x7f06b80cf980 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009d90 tx=0x7f06b002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae1d2e174341cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.548+0000 7f06b6a00640 1 -- 172.21.15.7:0/4265386336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0031600 con 0x7f06b815bc40 2024-04-25T11:22:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.548+0000 7f06b6a00640 1 -- 172.21.15.7:0/4265386336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b00317a0 con 0x7f06b815bc40 2024-04-25T11:22:53.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.548+0000 7f06b6a00640 1 -- 172.21.15.7:0/4265386336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0031b70 con 0x7f06b815bc40 2024-04-25T11:22:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.548+0000 7f06be800640 1 -- 172.21.15.7:0/4265386336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06b815bc40 msgr2=0x7f06b80cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.548+0000 7f06be800640 1 --2- 172.21.15.7:0/4265386336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06b815bc40 0x7f06b80cf980 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009d90 tx=0x7f06b002a5d0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.549+0000 7f06be800640 1 -- 172.21.15.7:0/4265386336 shutdown_connections 2024-04-25T11:22:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.549+0000 7f06be800640 1 --2- 172.21.15.7:0/4265386336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06b815bc40 0x7f06b80cf980 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.549+0000 7f06be800640 1 --2- 172.21.15.7:0/4265386336 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06b80af540 0x7f06b809f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.549+0000 7f06be800640 1 -- 172.21.15.7:0/4265386336 >> 172.21.15.7:0/4265386336 conn(0x7f06b8074200 msgr2=0x7f06b806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:53.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.549+0000 7f06be800640 1 -- 172.21.15.7:0/4265386336 shutdown_connections 2024-04-25T11:22:53.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.549+0000 7f06be800640 1 -- 172.21.15.7:0/4265386336 wait complete. 2024-04-25T11:22:53.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.551+0000 7f06be800640 1 Processor -- start 2024-04-25T11:22:53.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.551+0000 7f06be800640 1 -- start start 2024-04-25T11:22:53.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.553+0000 7f06be800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06b815bc40 0x7f06b814c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:53.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.553+0000 7f06be800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b815ef90 0x7f06b815df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:53.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.553+0000 7f06be800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06b8158d20 0x7f06b8159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.553+0000 7f06be800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06b80b1320 con 0x7f06b8158d20 2024-04-25T11:22:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.553+0000 7f06be800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f06b80b0e00 con 0x7f06b815ef90 2024-04-25T11:22:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.553+0000 7f06be800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f06b80afb80 con 0x7f06b815bc40 2024-04-25T11:22:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.553+0000 7f06b7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06b8158d20 0x7f06b8159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.553+0000 7f06b7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06b8158d20 0x7f06b8159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49932/0 (socket says 172.21.15.7:49932) 2024-04-25T11:22:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.553+0000 7f06b7e00640 1 -- 172.21.15.7:0/1357871934 learned_addr learned my addr 172.21.15.7:0/1357871934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.554+0000 7f06b7400640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06b815bc40 0x7f06b814c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.554+0000 7f06afe00640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b815ef90 0x7f06b815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.554+0000 7f06b7e00640 1 -- 172.21.15.7:0/1357871934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06b815bc40 msgr2=0x7f06b814c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.554+0000 7f06b7e00640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06b815bc40 0x7f06b814c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.554+0000 7f06b7e00640 1 -- 172.21.15.7:0/1357871934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b815ef90 msgr2=0x7f06b815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.554+0000 7f06b7e00640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b815ef90 0x7f06b815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.554+0000 7f06b7e00640 1 -- 172.21.15.7:0/1357871934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b8159670 con 0x7f06b8158d20 2024-04-25T11:22:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.554+0000 7f06b7e00640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06b8158d20 0x7f06b8159130 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009eb0 tx=0x7f06b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.554+0000 7f06af400640 1 -- 172.21.15.7:0/1357871934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b003f070 con 0x7f06b8158d20 2024-04-25T11:22:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.555+0000 7f06af400640 1 -- 172.21.15.7:0/1357871934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b0030b60 con 0x7f06b8158d20 2024-04-25T11:22:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.555+0000 7f06af400640 1 -- 172.21.15.7:0/1357871934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0009450 con 0x7f06b8158d20 2024-04-25T11:22:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.555+0000 7f06be800640 1 -- 172.21.15.7:0/1357871934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b80cd9c0 con 0x7f06b8158d20 2024-04-25T11:22:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.555+0000 7f06be800640 1 -- 172.21.15.7:0/1357871934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b80ccc00 con 0x7f06b8158d20 2024-04-25T11:22:53.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.556+0000 7f06be800640 1 -- 172.21.15.7:0/1357871934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b80b7ca0 con 0x7f06b8158d20 2024-04-25T11:22:53.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.557+0000 7f06af400640 1 -- 172.21.15.7:0/1357871934 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f06b003b410 con 0x7f06b8158d20 2024-04-25T11:22:53.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.558+0000 7f06af400640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06940744b0 0x7f06940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:53.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.558+0000 7f06af400640 1 -- 172.21.15.7:0/1357871934 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f06b00b6b60 con 0x7f06b8158d20 2024-04-25T11:22:53.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.559+0000 7f06b7400640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06940744b0 0x7f06940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:53.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.559+0000 7f06b7400640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06940744b0 0x7f06940769c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f06a8000f60 tx=0x7f06a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.563+0000 7f06af400640 1 -- 172.21.15.7:0/1357871934 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f06b80b7ca0 con 0x7f06b8158d20 2024-04-25T11:22:53.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.770+0000 7f06be800640 1 -- 172.21.15.7:0/1357871934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f06b806a100 con 0x7f06b8158d20 2024-04-25T11:22:53.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.773+0000 7f06af400640 1 -- 172.21.15.7:0/1357871934 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16788 (secure 0 0 0) 0x7f06b8159670 con 0x7f06b8158d20 2024-04-25T11:22:53.768 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:53.768 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:33.767424+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:22:30.472409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T11:22:31.299177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:53.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.775+0000 7f06be800640 1 -- 172.21.15.7:0/1357871934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06940744b0 msgr2=0x7f06940769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.775+0000 7f06be800640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06940744b0 0x7f06940769c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f06a8000f60 tx=0x7f06a8009040 comp rx=0 tx=0).stop 2024-04-25T11:22:53.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.775+0000 7f06be800640 1 -- 172.21.15.7:0/1357871934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06b8158d20 msgr2=0x7f06b8159130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:53.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.775+0000 7f06be800640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06b8158d20 0x7f06b8159130 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f06b0009eb0 tx=0x7f06b0004a40 comp rx=0 tx=0).stop 2024-04-25T11:22:53.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.776+0000 7f06be800640 1 -- 172.21.15.7:0/1357871934 shutdown_connections 2024-04-25T11:22:53.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.776+0000 7f06be800640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f06940744b0 0x7f06940769c0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.776+0000 7f06be800640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06b8158d20 0x7f06b8159130 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.776+0000 7f06be800640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b815ef90 0x7f06b815df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.776+0000 7f06be800640 1 --2- 172.21.15.7:0/1357871934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06b815bc40 0x7f06b814c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:53.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.776+0000 7f06be800640 1 -- 172.21.15.7:0/1357871934 >> 172.21.15.7:0/1357871934 conn(0x7f06b8074200 msgr2=0x7f06b80aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:53.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.776+0000 7f06be800640 1 -- 172.21.15.7:0/1357871934 shutdown_connections 2024-04-25T11:22:53.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:53.776+0000 7f06be800640 1 -- 172.21.15.7:0/1357871934 wait complete. 2024-04-25T11:22:53.798 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:22:53.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:54.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.090+0000 7fde27200640 1 Processor -- start 2024-04-25T11:22:54.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.090+0000 7fde27200640 1 -- start start 2024-04-25T11:22:54.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.093+0000 7fde27200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2806dc70 0x7fde280d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.093+0000 7fde27200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fde280d24f0 0x7fde280aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.093+0000 7fde27200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde280b2820 con 0x7fde280d2900 2024-04-25T11:22:54.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.093+0000 7fde27200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fde280b2260 con 0x7fde280d24f0 2024-04-25T11:22:54.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.093+0000 7fde27200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fde280b2540 con 0x7fde2806dc70 2024-04-25T11:22:54.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.093+0000 7fde26800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fde280d2900 0x7fde280af4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39006/0 (socket says 172.21.15.7:39006) 2024-04-25T11:22:54.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.093+0000 7fde26800640 1 -- 172.21.15.7:0/1311517402 learned_addr learned my addr 172.21.15.7:0/1311517402 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25e00640 1 --2- 172.21.15.7:0/1311517402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2806dc70 0x7fde280d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde1fe00640 1 --2- 172.21.15.7:0/1311517402 >> v2:172.21.15.87:3300/0 conn(0x7fde280d24f0 0x7fde280aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3059560901 0 0) 0x7fde280b2820 con 0x7fde280d2900 2024-04-25T11:22:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde0c003520 con 0x7fde280d2900 2024-04-25T11:22:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3647024100 0 0) 0x7fde0c003520 con 0x7fde280d2900 2024-04-25T11:22:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2806dc70 msgr2=0x7fde280d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 --2- 172.21.15.7:0/1311517402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2806dc70 0x7fde280d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 >> v2:172.21.15.87:3300/0 conn(0x7fde280d24f0 msgr2=0x7fde280aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 --2- 172.21.15.7:0/1311517402 >> v2:172.21.15.87:3300/0 conn(0x7fde280d24f0 0x7fde280aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2806d120 con 0x7fde280d2900 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fde200019f0 con 0x7fde280d2900 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 >> v1:172.21.15.7:6789/0 conn(0x7fde280d2900 legacy=0x7fde280af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 --2- 172.21.15.7:0/1311517402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0c0039b0 0x7fde0c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.094+0000 7fde25400640 1 --2- 172.21.15.7:0/1311517402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2806dc70 0x7fde0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.095+0000 7fde25400640 1 --2- 172.21.15.7:0/1311517402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde0c004d40 0x7fde0c005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.095+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde280b2260 con 0x7fde0c0039b0 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.095+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fde280b2540 con 0x7fde0c004d40 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.095+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fde0c00b9c0 con 0x7fde2806dc70 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.095+0000 7fde25e00640 1 --2- 172.21.15.7:0/1311517402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0c0039b0 0x7fde0c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.095+0000 7fde25e00640 1 -- 172.21.15.7:0/1311517402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2806dc70 msgr2=0x7fde0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.095+0000 7fde25e00640 1 --2- 172.21.15.7:0/1311517402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2806dc70 0x7fde0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.095+0000 7fde25e00640 1 -- 172.21.15.7:0/1311517402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde0c004d40 msgr2=0x7fde0c005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:54.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.095+0000 7fde25e00640 1 --2- 172.21.15.7:0/1311517402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde0c004d40 0x7fde0c005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.095+0000 7fde25e00640 1 -- 172.21.15.7:0/1311517402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c003520 con 0x7fde0c0039b0 2024-04-25T11:22:54.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.096+0000 7fde25e00640 1 --2- 172.21.15.7:0/1311517402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0c0039b0 0x7fde0c003e20 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fde18009ac0 tx=0x7fde0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b72e0a31862587 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:54.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.097+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18036070 con 0x7fde0c0039b0 2024-04-25T11:22:54.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.097+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde18031400 con 0x7fde0c0039b0 2024-04-25T11:22:54.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.097+0000 7fde25400640 1 -- 172.21.15.7:0/1311517402 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18009690 con 0x7fde0c0039b0 2024-04-25T11:22:54.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.097+0000 7fde27200640 1 -- 172.21.15.7:0/1311517402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0c0039b0 msgr2=0x7fde0c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.097+0000 7fde27200640 1 --2- 172.21.15.7:0/1311517402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde0c0039b0 0x7fde0c003e20 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fde18009ac0 tx=0x7fde0c005830 comp rx=0 tx=0).stop 2024-04-25T11:22:54.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.097+0000 7fde26800640 1 -- 172.21.15.7:0/1311517402 reap_dead start 2024-04-25T11:22:54.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.098+0000 7fde27200640 1 -- 172.21.15.7:0/1311517402 shutdown_connections 2024-04-25T11:22:54.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.098+0000 7fde27200640 1 -- 172.21.15.7:0/1311517402 >> 172.21.15.7:0/1311517402 conn(0x7fde28073b60 msgr2=0x7fde280b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:54.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.098+0000 7fde27200640 1 -- 172.21.15.7:0/1311517402 shutdown_connections 2024-04-25T11:22:54.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.098+0000 7fde27200640 1 -- 172.21.15.7:0/1311517402 wait complete. 2024-04-25T11:22:54.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.100+0000 7fde27200640 1 Processor -- start 2024-04-25T11:22:54.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.100+0000 7fde27200640 1 -- start start 2024-04-25T11:22:54.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.102+0000 7fde27200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde0c0039b0 0x7fde281455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.102+0000 7fde27200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2809fa60 0x7fde28146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.102+0000 7fde27200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde28147530 0x7fde28147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.102+0000 7fde27200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fde280b1ea0 con 0x7fde28147530 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.102+0000 7fde27200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fde2809b990 con 0x7fde2809fa60 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.102+0000 7fde27200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fde2809b540 con 0x7fde0c0039b0 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.102+0000 7fde26800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde28147530 0x7fde28147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.102+0000 7fde26800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde28147530 0x7fde28147940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49946/0 (socket says 172.21.15.7:49946) 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.102+0000 7fde26800640 1 -- 172.21.15.7:0/3273040836 learned_addr learned my addr 172.21.15.7:0/3273040836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.103+0000 7fde25e00640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde0c0039b0 0x7fde281455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.103+0000 7fde1fe00640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2809fa60 0x7fde28146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.103+0000 7fde26800640 1 -- 172.21.15.7:0/3273040836 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde0c0039b0 msgr2=0x7fde281455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.103+0000 7fde26800640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde0c0039b0 0x7fde281455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.103+0000 7fde26800640 1 -- 172.21.15.7:0/3273040836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2809fa60 msgr2=0x7fde28146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.103+0000 7fde26800640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2809fa60 0x7fde28146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.103+0000 7fde26800640 1 -- 172.21.15.7:0/3273040836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde28145f50 con 0x7fde28147530 2024-04-25T11:22:54.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.103+0000 7fde26800640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde28147530 0x7fde28147940 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fde2806d7f0 tx=0x7fde2000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.104+0000 7fde1e000640 1 -- 172.21.15.7:0/3273040836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2000a610 con 0x7fde28147530 2024-04-25T11:22:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.104+0000 7fde1e000640 1 -- 172.21.15.7:0/3273040836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde2000a7b0 con 0x7fde28147530 2024-04-25T11:22:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.104+0000 7fde1e000640 1 -- 172.21.15.7:0/3273040836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2000ab80 con 0x7fde28147530 2024-04-25T11:22:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.104+0000 7fde27200640 1 -- 172.21.15.7:0/3273040836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde28146230 con 0x7fde28147530 2024-04-25T11:22:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.104+0000 7fde27200640 1 -- 172.21.15.7:0/3273040836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde281467f0 con 0x7fde28147530 2024-04-25T11:22:54.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.105+0000 7fde27200640 1 -- 172.21.15.7:0/3273040836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde28050740 con 0x7fde28147530 2024-04-25T11:22:54.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.106+0000 7fde1e000640 1 -- 172.21.15.7:0/3273040836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fde20011070 con 0x7fde28147530 2024-04-25T11:22:54.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.107+0000 7fde1e000640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fde040744b0 0x7fde040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.107+0000 7fde1e000640 1 -- 172.21.15.7:0/3273040836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7fde200949c0 con 0x7fde28147530 2024-04-25T11:22:54.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.108+0000 7fde25e00640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fde040744b0 0x7fde040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.109+0000 7fde25e00640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fde040744b0 0x7fde040769c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fde18033000 tx=0x7fde18008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:54.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.113+0000 7fde1e000640 1 -- 172.21.15.7:0/3273040836 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fde2005efb0 con 0x7fde28147530 2024-04-25T11:22:54.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.358+0000 7fde27200640 1 -- 172.21.15.7:0/3273040836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde280afbc0 con 0x7fde28147530 2024-04-25T11:22:54.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.359+0000 7fde1e000640 1 -- 172.21.15.7:0/3273040836 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5179 (secure 0 0 0) 0x7fde2005e8b0 con 0x7fde28147530 2024-04-25T11:22:54.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:54.354 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7588,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2938174159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2938174159},{"type":"v1","addr":"172.21.15.87:6834","nonce":2938174159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/793358721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":793358721},{"type":"v1","addr":"172.21.15.87:6835","nonce":793358721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":9447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2654444897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2654444897},{"type":"v1","addr":"172.21.15.7:6835","nonce":2654444897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:22:32.315016+0000","modified":"2024-04-25T11:22:35.372928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/339418645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":339418645},{"type":"v1","addr":"172.21.15.7:6837","nonce":339418645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":11}]} 2024-04-25T11:22:54.354 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 116 2024-04-25T11:22:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.361+0000 7fde27200640 1 -- 172.21.15.7:0/3273040836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fde040744b0 msgr2=0x7fde040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.361+0000 7fde27200640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fde040744b0 0x7fde040769c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fde18033000 tx=0x7fde18008030 comp rx=0 tx=0).stop 2024-04-25T11:22:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.361+0000 7fde27200640 1 -- 172.21.15.7:0/3273040836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde28147530 msgr2=0x7fde28147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.361+0000 7fde27200640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde28147530 0x7fde28147940 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fde2806d7f0 tx=0x7fde2000cc20 comp rx=0 tx=0).stop 2024-04-25T11:22:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.362+0000 7fde27200640 1 -- 172.21.15.7:0/3273040836 shutdown_connections 2024-04-25T11:22:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.362+0000 7fde27200640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fde040744b0 0x7fde040769c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.362+0000 7fde27200640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde28147530 0x7fde28147940 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.362+0000 7fde27200640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2809fa60 0x7fde28146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.362+0000 7fde27200640 1 --2- 172.21.15.7:0/3273040836 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde0c0039b0 0x7fde281455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.362+0000 7fde27200640 1 -- 172.21.15.7:0/3273040836 >> 172.21.15.7:0/3273040836 conn(0x7fde28073b60 msgr2=0x7fde280d1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.362+0000 7fde27200640 1 -- 172.21.15.7:0/3273040836 shutdown_connections 2024-04-25T11:22:54.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.362+0000 7fde27200640 1 -- 172.21.15.7:0/3273040836 wait complete. 2024-04-25T11:22:54.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:22:54.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.670+0000 7f7e5fa00640 1 Processor -- start 2024-04-25T11:22:54.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.670+0000 7f7e5fa00640 1 -- start start 2024-04-25T11:22:54.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e5fa00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e580d2400 0x7f7e580d2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e5fa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e580b9370 0x7f7e580b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e5fa00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e580d5300 con 0x7f7e580d4780 2024-04-25T11:22:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e5fa00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f7e580b8ee0 con 0x7f7e580d2400 2024-04-25T11:22:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e5fa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7e580d4d40 con 0x7f7e580b9370 2024-04-25T11:22:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e57e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e580d2400 0x7f7e580d2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e5d000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7e580d4780 0x7f7e580d0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39012/0 (socket says 172.21.15.7:39012) 2024-04-25T11:22:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e5d000640 1 -- 172.21.15.7:0/913933141 learned_addr learned my addr 172.21.15.7:0/913933141 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:54.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e5da00640 1 --2- 172.21.15.7:0/913933141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e580b9370 0x7f7e580b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e57e00640 1 -- 172.21.15.7:0/913933141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e580b9370 msgr2=0x7f7e580b9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:54.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e57e00640 1 --2- 172.21.15.7:0/913933141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e580b9370 0x7f7e580b9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e57e00640 1 -- 172.21.15.7:0/913933141 >> v1:172.21.15.7:6789/0 conn(0x7f7e580d4780 legacy=0x7f7e580d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.674+0000 7f7e57e00640 1 -- 172.21.15.7:0/913933141 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e580d5020 con 0x7f7e580d2400 2024-04-25T11:22:54.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.673+0000 7f7e57400640 1 -- 172.21.15.7:0/913933141 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2989665198 0 0) 0x7f7e580d5300 con 0x7f7e580d4780 2024-04-25T11:22:54.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.674+0000 7f7e5da00640 1 --2- 172.21.15.7:0/913933141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e580b9370 0x7f7e580b9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:54.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.675+0000 7f7e57e00640 1 --2- 172.21.15.7:0/913933141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e580d2400 0x7f7e580d2810 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f7e400097a0 tx=0x7f7e4002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea26081ba5282cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:54.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.675+0000 7f7e57400640 1 -- 172.21.15.7:0/913933141 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40037070 con 0x7f7e580d2400 2024-04-25T11:22:54.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.675+0000 7f7e57400640 1 -- 172.21.15.7:0/913933141 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4002adb0 con 0x7f7e580d2400 2024-04-25T11:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.675+0000 7f7e57400640 1 -- 172.21.15.7:0/913933141 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40032660 con 0x7f7e580d2400 2024-04-25T11:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.676+0000 7f7e5fa00640 1 -- 172.21.15.7:0/913933141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e580d2400 msgr2=0x7f7e580d2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.676+0000 7f7e5fa00640 1 --2- 172.21.15.7:0/913933141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e580d2400 0x7f7e580d2810 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f7e400097a0 tx=0x7f7e4002a060 comp rx=0 tx=0).stop 2024-04-25T11:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.676+0000 7f7e5fa00640 1 -- 172.21.15.7:0/913933141 shutdown_connections 2024-04-25T11:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.676+0000 7f7e5fa00640 1 --2- 172.21.15.7:0/913933141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e580b9370 0x7f7e580b9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.676+0000 7f7e5fa00640 1 --2- 172.21.15.7:0/913933141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e580d2400 0x7f7e580d2810 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.676+0000 7f7e5fa00640 1 -- 172.21.15.7:0/913933141 >> 172.21.15.7:0/913933141 conn(0x7f7e5806edb0 msgr2=0x7f7e5806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.676+0000 7f7e5fa00640 1 -- 172.21.15.7:0/913933141 shutdown_connections 2024-04-25T11:22:54.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.676+0000 7f7e5fa00640 1 -- 172.21.15.7:0/913933141 wait complete. 2024-04-25T11:22:54.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.678+0000 7f7e5fa00640 1 Processor -- start 2024-04-25T11:22:54.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.678+0000 7f7e5fa00640 1 -- start start 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.680+0000 7f7e5fa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e580b9370 0x7f7e5815f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.680+0000 7f7e5fa00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e580d2400 0x7f7e58153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.680+0000 7f7e5fa00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e580d4780 0x7f7e58071850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.680+0000 7f7e5fa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e58164850 con 0x7f7e580b9370 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.680+0000 7f7e5fa00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7e580d2d50 con 0x7f7e580d4780 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.680+0000 7f7e5fa00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7e581649d0 con 0x7f7e580d2400 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.680+0000 7f7e5d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e580b9370 0x7f7e5815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.680+0000 7f7e5d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e580b9370 0x7f7e5815f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49962/0 (socket says 172.21.15.7:49962) 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.680+0000 7f7e5d000640 1 -- 172.21.15.7:0/1383971730 learned_addr learned my addr 172.21.15.7:0/1383971730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.680+0000 7f7e57e00640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e580d2400 0x7f7e58153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e5d000640 1 -- 172.21.15.7:0/1383971730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e580d2400 msgr2=0x7f7e58153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e5d000640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e580d2400 0x7f7e58153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e5da00640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e580d4780 0x7f7e58071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e5d000640 1 -- 172.21.15.7:0/1383971730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e580d4780 msgr2=0x7f7e58071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e5d000640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e580d4780 0x7f7e58071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e5d000640 1 -- 172.21.15.7:0/1383971730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e580702a0 con 0x7f7e580b9370 2024-04-25T11:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e5d000640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e580b9370 0x7f7e5815f720 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800c440 tx=0x7f7e4800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e54c00640 1 -- 172.21.15.7:0/1383971730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480074e0 con 0x7f7e580b9370 2024-04-25T11:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e54c00640 1 -- 172.21.15.7:0/1383971730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e480047e0 con 0x7f7e580b9370 2024-04-25T11:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e5fa00640 1 -- 172.21.15.7:0/1383971730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e581536a0 con 0x7f7e580b9370 2024-04-25T11:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e54c00640 1 -- 172.21.15.7:0/1383971730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48004bd0 con 0x7f7e580b9370 2024-04-25T11:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.681+0000 7f7e5fa00640 1 -- 172.21.15.7:0/1383971730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e580707a0 con 0x7f7e580b9370 2024-04-25T11:22:54.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.682+0000 7f7e5fa00640 1 -- 172.21.15.7:0/1383971730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5806a100 con 0x7f7e580b9370 2024-04-25T11:22:54.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.684+0000 7f7e54c00640 1 -- 172.21.15.7:0/1383971730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f7e48011070 con 0x7f7e580b9370 2024-04-25T11:22:54.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.685+0000 7f7e54c00640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e340744b0 0x7f7e340769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:54.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.685+0000 7f7e54c00640 1 -- 172.21.15.7:0/1383971730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f7e48095070 con 0x7f7e580b9370 2024-04-25T11:22:54.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.685+0000 7f7e57e00640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e340744b0 0x7f7e340769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:54.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.686+0000 7f7e57e00640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e340744b0 0x7f7e340769c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7e40034000 tx=0x7f7e4002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:54.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.689+0000 7f7e54c00640 1 -- 172.21.15.7:0/1383971730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f7e4805f5d0 con 0x7f7e580b9370 2024-04-25T11:22:54.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.936+0000 7f7e5fa00640 1 -- 172.21.15.7:0/1383971730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e581653a0 con 0x7f7e580b9370 2024-04-25T11:22:54.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.938+0000 7f7e54c00640 1 -- 172.21.15.7:0/1383971730 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+5179 (secure 0 0 0) 0x7f7e4805eed0 con 0x7f7e580b9370 2024-04-25T11:22:54.933 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:54.933 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7588,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2938174159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2938174159},{"type":"v1","addr":"172.21.15.87:6834","nonce":2938174159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/793358721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":793358721},{"type":"v1","addr":"172.21.15.87:6835","nonce":793358721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":9447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2654444897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2654444897},{"type":"v1","addr":"172.21.15.7:6835","nonce":2654444897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T11:22:32.315016+0000","modified":"2024-04-25T11:22:35.372928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"c","rank":0,"incarnation":112,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6837/339418645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":339418645},{"type":"v1","addr":"172.21.15.7:6837","nonce":339418645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":11}]} 2024-04-25T11:22:54.933 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 116 2024-04-25T11:22:54.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.940+0000 7f7e5fa00640 1 -- 172.21.15.7:0/1383971730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e340744b0 msgr2=0x7f7e340769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.940+0000 7f7e5fa00640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e340744b0 0x7f7e340769c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7e40034000 tx=0x7f7e4002e000 comp rx=0 tx=0).stop 2024-04-25T11:22:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.941+0000 7f7e5fa00640 1 -- 172.21.15.7:0/1383971730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e580b9370 msgr2=0x7f7e5815f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.941+0000 7f7e5fa00640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e580b9370 0x7f7e5815f720 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800c440 tx=0x7f7e4800c900 comp rx=0 tx=0).stop 2024-04-25T11:22:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.941+0000 7f7e5fa00640 1 -- 172.21.15.7:0/1383971730 shutdown_connections 2024-04-25T11:22:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.941+0000 7f7e5fa00640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f7e340744b0 0x7f7e340769c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.941+0000 7f7e5fa00640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e580d4780 0x7f7e58071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.941+0000 7f7e5fa00640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e580d2400 0x7f7e58153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.941+0000 7f7e5fa00640 1 --2- 172.21.15.7:0/1383971730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e580b9370 0x7f7e5815f720 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.941+0000 7f7e5fa00640 1 -- 172.21.15.7:0/1383971730 >> 172.21.15.7:0/1383971730 conn(0x7f7e5806edb0 msgr2=0x7f7e58166180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.941+0000 7f7e5fa00640 1 -- 172.21.15.7:0/1383971730 shutdown_connections 2024-04-25T11:22:54.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:54.941+0000 7f7e5fa00640 1 -- 172.21.15.7:0/1383971730 wait complete. 2024-04-25T11:22:54.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:22:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.257+0000 7f08a4c00640 1 Processor -- start 2024-04-25T11:22:55.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.257+0000 7f08a4c00640 1 -- start start 2024-04-25T11:22:55.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.259+0000 7f08a4c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08a00aed40 0x7f08a015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.259+0000 7f08a4c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a00b5220 0x7f08a00ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.259+0000 7f08a4c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a00b64b0 con 0x7f08a00af420 2024-04-25T11:22:55.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.259+0000 7f08a4c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f08a015e1f0 con 0x7f08a00aed40 2024-04-25T11:22:55.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.259+0000 7f08a4c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f08a006c610 con 0x7f08a00b5220 2024-04-25T11:22:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.259+0000 7f089fe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f08a00af420 0x7f08a00af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39020/0 (socket says 172.21.15.7:39020) 2024-04-25T11:22:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.259+0000 7f089fe00640 1 -- 172.21.15.7:0/95463723 learned_addr learned my addr 172.21.15.7:0/95463723 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.259+0000 7f089ea00640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a00b5220 0x7f08a00ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089f400640 1 --2- 172.21.15.7:0/95463723 >> v2:172.21.15.87:3300/0 conn(0x7f08a00aed40 0x7f08a015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 869994914 0 0) 0x7f08a00b64b0 con 0x7f08a00af420 2024-04-25T11:22:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0884003520 con 0x7f08a00af420 2024-04-25T11:22:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1510660115 0 0) 0x7f0884003520 con 0x7f08a00af420 2024-04-25T11:22:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a00b5220 msgr2=0x7f08a00ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:55.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a00b5220 0x7f08a00ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 >> v2:172.21.15.87:3300/0 conn(0x7f08a00aed40 msgr2=0x7f08a015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 --2- 172.21.15.7:0/95463723 >> v2:172.21.15.87:3300/0 conn(0x7f08a00aed40 0x7f08a015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a006c8f0 con 0x7f08a00af420 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f08980019f0 con 0x7f08a00af420 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 >> v1:172.21.15.7:6789/0 conn(0x7f08a00af420 legacy=0x7f08a00af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08840039b0 0x7f0884003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 0x7f0884004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a00b5220 0x7f0884005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08a015e1f0 con 0x7f0884004800 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f08a006c610 con 0x7f08840039b0 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.260+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f088400b9c0 con 0x7f08a00b5220 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.261+0000 7f089ea00640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 0x7f0884004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.261+0000 7f089f400640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08840039b0 0x7f0884003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.261+0000 7f089ea00640 1 -- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a00b5220 msgr2=0x7f0884005150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.261+0000 7f089fe00640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a00b5220 0x7f0884005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.261+0000 7f089ea00640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a00b5220 0x7f0884005150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.261+0000 7f089ea00640 1 -- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08840039b0 msgr2=0x7f0884003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.261+0000 7f089ea00640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08840039b0 0x7f0884003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.261+0000 7f089ea00640 1 -- 172.21.15.7:0/95463723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0884003520 con 0x7f0884004800 2024-04-25T11:22:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.261+0000 7f089f400640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08840039b0 0x7f0884003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:55.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.261+0000 7f089fe00640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a00b5220 0x7f0884005150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:55.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.262+0000 7f089ea00640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 0x7f0884004c10 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f0888009ac0 tx=0x7f0884005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66617cf17b1f6392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:55.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.263+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888036070 con 0x7f0884004800 2024-04-25T11:22:55.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.263+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0888031400 con 0x7f0884004800 2024-04-25T11:22:55.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.263+0000 7f089e000640 1 -- 172.21.15.7:0/95463723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888009690 con 0x7f0884004800 2024-04-25T11:22:55.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.263+0000 7f08a4c00640 1 -- 172.21.15.7:0/95463723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 msgr2=0x7f0884004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:55.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.263+0000 7f08a4c00640 1 --2- 172.21.15.7:0/95463723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 0x7f0884004c10 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f0888009ac0 tx=0x7f0884005830 comp rx=0 tx=0).stop 2024-04-25T11:22:55.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.263+0000 7f089fe00640 1 -- 172.21.15.7:0/95463723 reap_dead start 2024-04-25T11:22:55.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.264+0000 7f08a4c00640 1 -- 172.21.15.7:0/95463723 shutdown_connections 2024-04-25T11:22:55.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.264+0000 7f08a4c00640 1 -- 172.21.15.7:0/95463723 >> 172.21.15.7:0/95463723 conn(0x7f08a00b3820 msgr2=0x7f08a00b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:55.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.264+0000 7f08a4c00640 1 -- 172.21.15.7:0/95463723 shutdown_connections 2024-04-25T11:22:55.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.264+0000 7f08a4c00640 1 -- 172.21.15.7:0/95463723 wait complete. 2024-04-25T11:22:55.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.266+0000 7f08a4c00640 1 Processor -- start 2024-04-25T11:22:55.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.266+0000 7f08a4c00640 1 -- start start 2024-04-25T11:22:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.268+0000 7f08a4c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 0x7f08a0160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.268+0000 7f08a4c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a0146500 0x7f08a0160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.268+0000 7f08a4c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a015e370 0x7f08a0145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.268+0000 7f08a4c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08a00b6110 con 0x7f0884004800 2024-04-25T11:22:55.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.268+0000 7f08a4c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f08a006ceb0 con 0x7f08a0146500 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.268+0000 7f08a4c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f08a00b6290 con 0x7f08a015e370 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.268+0000 7f089f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 0x7f08a0160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.268+0000 7f089f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 0x7f08a0160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49972/0 (socket says 172.21.15.7:49972) 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.268+0000 7f089f400640 1 -- 172.21.15.7:0/1954760071 learned_addr learned my addr 172.21.15.7:0/1954760071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.268+0000 7f089ea00640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a0146500 0x7f08a0160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f089fe00640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a015e370 0x7f08a0145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f089f400640 1 -- 172.21.15.7:0/1954760071 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a015e370 msgr2=0x7f08a0145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f089f400640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a015e370 0x7f08a0145820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f089f400640 1 -- 172.21.15.7:0/1954760071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a0146500 msgr2=0x7f08a0160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f089f400640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a0146500 0x7f08a0160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f089f400640 1 -- 172.21.15.7:0/1954760071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a00ce700 con 0x7f0884004800 2024-04-25T11:22:55.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f089f400640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 0x7f08a0160550 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f08900109a0 tx=0x7f0890010e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:55.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f0897400640 1 -- 172.21.15.7:0/1954760071 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089000f040 con 0x7f0884004800 2024-04-25T11:22:55.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f0897400640 1 -- 172.21.15.7:0/1954760071 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f089000ebc0 con 0x7f0884004800 2024-04-25T11:22:55.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f08a4c00640 1 -- 172.21.15.7:0/1954760071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a00ce9e0 con 0x7f0884004800 2024-04-25T11:22:55.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f0897400640 1 -- 172.21.15.7:0/1954760071 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089001c400 con 0x7f0884004800 2024-04-25T11:22:55.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.269+0000 7f08a4c00640 1 -- 172.21.15.7:0/1954760071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a00cefa0 con 0x7f0884004800 2024-04-25T11:22:55.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.271+0000 7f08a4c00640 1 -- 172.21.15.7:0/1954760071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a0050740 con 0x7f0884004800 2024-04-25T11:22:55.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.272+0000 7f0897400640 1 -- 172.21.15.7:0/1954760071 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f089001c5a0 con 0x7f0884004800 2024-04-25T11:22:55.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.273+0000 7f0897400640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f087c0744e0 0x7f087c0769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.273+0000 7f0897400640 1 -- 172.21.15.7:0/1954760071 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f0890095ae0 con 0x7f0884004800 2024-04-25T11:22:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.273+0000 7f089ea00640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f087c0744e0 0x7f087c0769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.274+0000 7f089ea00640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f087c0744e0 0x7f087c0769f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0888033000 tx=0x7f0888008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:55.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.278+0000 7f0897400640 1 -- 172.21.15.7:0/1954760071 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f0890018030 con 0x7f0884004800 2024-04-25T11:22:55.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.481+0000 7f08a4c00640 1 -- 172.21.15.7:0/1954760071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08a0070a20 con 0x7f0884004800 2024-04-25T11:22:55.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.482+0000 7f0897400640 1 -- 172.21.15.7:0/1954760071 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16788 (secure 0 0 0) 0x7f08900600e0 con 0x7f0884004800 2024-04-25T11:22:55.478 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:22:55.478 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:33.767424+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T11:22:30.472409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T11:22:31.299177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:22:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.485+0000 7f08a4c00640 1 -- 172.21.15.7:0/1954760071 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f087c0744e0 msgr2=0x7f087c0769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.485+0000 7f08a4c00640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f087c0744e0 0x7f087c0769f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0888033000 tx=0x7f0888008030 comp rx=0 tx=0).stop 2024-04-25T11:22:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.485+0000 7f08a4c00640 1 -- 172.21.15.7:0/1954760071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 msgr2=0x7f08a0160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.485+0000 7f08a4c00640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 0x7f08a0160550 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f08900109a0 tx=0x7f0890010e60 comp rx=0 tx=0).stop 2024-04-25T11:22:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.485+0000 7f08a4c00640 1 -- 172.21.15.7:0/1954760071 shutdown_connections 2024-04-25T11:22:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.485+0000 7f08a4c00640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f087c0744e0 0x7f087c0769f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.485+0000 7f08a4c00640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08a015e370 0x7f08a0145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.485+0000 7f08a4c00640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a0146500 0x7f08a0160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.485+0000 7f08a4c00640 1 --2- 172.21.15.7:0/1954760071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0884004800 0x7f08a0160550 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.485+0000 7f08a4c00640 1 -- 172.21.15.7:0/1954760071 >> 172.21.15.7:0/1954760071 conn(0x7f08a00b3820 msgr2=0x7f08a006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.486+0000 7f08a4c00640 1 -- 172.21.15.7:0/1954760071 shutdown_connections 2024-04-25T11:22:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.486+0000 7f08a4c00640 1 -- 172.21.15.7:0/1954760071 wait complete. 2024-04-25T11:22:55.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T11:22:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.802+0000 7f539be00640 1 Processor -- start 2024-04-25T11:22:55.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.802+0000 7f539be00640 1 -- start start 2024-04-25T11:22:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.804+0000 7f539be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f539c15eed0 0x7f539c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.804+0000 7f539be00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f539c141290 0x7f539c1416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.804+0000 7f539be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f539c06d610 0x7f539c06da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.804+0000 7f539be00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f539c06c570 con 0x7f539c06d610 2024-04-25T11:22:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.804+0000 7f539be00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f539c06c850 con 0x7f539c141290 2024-04-25T11:22:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.804+0000 7f539be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f539c06cb30 con 0x7f539c15eed0 2024-04-25T11:22:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.805+0000 7f539b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f539c06d610 0x7f539c06da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.805+0000 7f539b400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f539c06d610 0x7f539c06da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49984/0 (socket says 172.21.15.7:49984) 2024-04-25T11:22:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.805+0000 7f539b400640 1 -- 172.21.15.7:0/1163546718 learned_addr learned my addr 172.21.15.7:0/1163546718 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.805+0000 7f539aa00640 1 --2- 172.21.15.7:0/1163546718 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f539c15eed0 0x7f539c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.805+0000 7f5393e00640 1 --2- 172.21.15.7:0/1163546718 >> v2:172.21.15.87:3300/0 conn(0x7f539c141290 0x7f539c1416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.805+0000 7f539b400640 1 -- 172.21.15.7:0/1163546718 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f539c15eed0 msgr2=0x7f539c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.805+0000 7f539b400640 1 --2- 172.21.15.7:0/1163546718 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f539c15eed0 0x7f539c0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.805+0000 7f539b400640 1 -- 172.21.15.7:0/1163546718 >> v2:172.21.15.87:3300/0 conn(0x7f539c141290 msgr2=0x7f539c1416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.805+0000 7f539b400640 1 --2- 172.21.15.7:0/1163546718 >> v2:172.21.15.87:3300/0 conn(0x7f539c141290 0x7f539c1416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.805+0000 7f539b400640 1 -- 172.21.15.7:0/1163546718 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c06ce10 con 0x7f539c06d610 2024-04-25T11:22:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.807+0000 7f539b400640 1 --2- 172.21.15.7:0/1163546718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c06d610 0x7f539c06da20 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f539c0cefc0 tx=0x7f539402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbabc897a75a0ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.807+0000 7f539a000640 1 -- 172.21.15.7:0/1163546718 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394036070 con 0x7f539c06d610 2024-04-25T11:22:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.807+0000 7f539a000640 1 -- 172.21.15.7:0/1163546718 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5394031600 con 0x7f539c06d610 2024-04-25T11:22:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.807+0000 7f539a000640 1 -- 172.21.15.7:0/1163546718 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940319d0 con 0x7f539c06d610 2024-04-25T11:22:55.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.808+0000 7f539be00640 1 -- 172.21.15.7:0/1163546718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c06d610 msgr2=0x7f539c06da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.808+0000 7f539be00640 1 --2- 172.21.15.7:0/1163546718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c06d610 0x7f539c06da20 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f539c0cefc0 tx=0x7f539402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.808+0000 7f539be00640 1 -- 172.21.15.7:0/1163546718 shutdown_connections 2024-04-25T11:22:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.808+0000 7f539be00640 1 --2- 172.21.15.7:0/1163546718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c06d610 0x7f539c06da20 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.808+0000 7f539be00640 1 --2- 172.21.15.7:0/1163546718 >> v2:172.21.15.87:3300/0 conn(0x7f539c141290 0x7f539c1416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.808+0000 7f539be00640 1 --2- 172.21.15.7:0/1163546718 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f539c15eed0 0x7f539c0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.808+0000 7f539be00640 1 -- 172.21.15.7:0/1163546718 >> 172.21.15.7:0/1163546718 conn(0x7f539c0d1540 msgr2=0x7f539c0d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.808+0000 7f539be00640 1 -- 172.21.15.7:0/1163546718 shutdown_connections 2024-04-25T11:22:55.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.808+0000 7f539be00640 1 -- 172.21.15.7:0/1163546718 wait complete. 2024-04-25T11:22:55.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.811+0000 7f539be00640 1 Processor -- start 2024-04-25T11:22:55.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.811+0000 7f539be00640 1 -- start start 2024-04-25T11:22:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.812+0000 7f539be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f539c06d610 0x7f539c154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.812+0000 7f539be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c141290 0x7f539c150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.812+0000 7f539be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f539c15eed0 0x7f539c156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.812+0000 7f539be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f539c140da0 con 0x7f539c141290 2024-04-25T11:22:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.812+0000 7f539be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0cdcc0 con 0x7f539c15eed0 2024-04-25T11:22:55.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.812+0000 7f539be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f539c140f20 con 0x7f539c06d610 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.813+0000 7f5393e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c141290 0x7f539c150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.813+0000 7f5393e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c141290 0x7f539c150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49996/0 (socket says 172.21.15.7:49996) 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.813+0000 7f5393e00640 1 -- 172.21.15.7:0/975228627 learned_addr learned my addr 172.21.15.7:0/975228627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.813+0000 7f539aa00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f539c06d610 0x7f539c154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.813+0000 7f539b400640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f539c15eed0 0x7f539c156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.813+0000 7f5393e00640 1 -- 172.21.15.7:0/975228627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f539c06d610 msgr2=0x7f539c154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.813+0000 7f5393e00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f539c06d610 0x7f539c154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.813+0000 7f5393e00640 1 -- 172.21.15.7:0/975228627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f539c15eed0 msgr2=0x7f539c156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.813+0000 7f5393e00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f539c15eed0 0x7f539c156890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.813+0000 7f5393e00640 1 -- 172.21.15.7:0/975228627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c154d80 con 0x7f539c141290 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.814+0000 7f5393e00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c141290 0x7f539c150530 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f538400d620 tx=0x7f538400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.814+0000 7f5393400640 1 -- 172.21.15.7:0/975228627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538400b620 con 0x7f539c141290 2024-04-25T11:22:55.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.814+0000 7f5393400640 1 -- 172.21.15.7:0/975228627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5384019070 con 0x7f539c141290 2024-04-25T11:22:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.814+0000 7f5393400640 1 -- 172.21.15.7:0/975228627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538400ba00 con 0x7f539c141290 2024-04-25T11:22:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.814+0000 7f539be00640 1 -- 172.21.15.7:0/975228627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c155060 con 0x7f539c141290 2024-04-25T11:22:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.814+0000 7f539be00640 1 -- 172.21.15.7:0/975228627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c155620 con 0x7f539c141290 2024-04-25T11:22:55.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.815+0000 7f539be00640 1 -- 172.21.15.7:0/975228627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c154af0 con 0x7f539c141290 2024-04-25T11:22:55.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.817+0000 7f5393400640 1 -- 172.21.15.7:0/975228627 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f538400bba0 con 0x7f539c141290 2024-04-25T11:22:55.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.818+0000 7f5393400640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53780744b0 0x7f53780769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:55.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.818+0000 7f5393400640 1 -- 172.21.15.7:0/975228627 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 6101+0+0 (secure 0 0 0) 0x7f53840957b0 con 0x7f539c141290 2024-04-25T11:22:55.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.818+0000 7f539aa00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53780744b0 0x7f53780769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:55.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.819+0000 7f539aa00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53780744b0 0x7f53780769c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f538c006fd0 tx=0x7f538c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:55.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:55.822+0000 7f5393400640 1 -- 172.21.15.7:0/975228627 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f538405fe40 con 0x7f539c141290 2024-04-25T11:22:56.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.070+0000 7f539be00640 1 -- 172.21.15.7:0/975228627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f539c050740 con 0x7f539c141290 2024-04-25T11:22:56.066 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-04-25T11:22:56.072+0000 7fac34e00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T11:22:56.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.075+0000 7f5393400640 1 -- 172.21.15.7:0/975228627 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v117) ==== 161+0+0 (secure 0 0 0) 0x7f538405f740 con 0x7f539c141290 2024-04-25T11:22:56.070 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T11:22:56.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.079+0000 7f539be00640 1 -- 172.21.15.7:0/975228627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53780744b0 msgr2=0x7f53780769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:56.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.079+0000 7f539be00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53780744b0 0x7f53780769c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f538c006fd0 tx=0x7f538c008040 comp rx=0 tx=0).stop 2024-04-25T11:22:56.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.079+0000 7f539be00640 1 -- 172.21.15.7:0/975228627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c141290 msgr2=0x7f539c150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:56.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.079+0000 7f539be00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c141290 0x7f539c150530 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f538400d620 tx=0x7f538400dae0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.079+0000 7f539be00640 1 -- 172.21.15.7:0/975228627 shutdown_connections 2024-04-25T11:22:56.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.079+0000 7f539be00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f53780744b0 0x7f53780769c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.079+0000 7f539be00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f539c15eed0 0x7f539c156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.079+0000 7f539be00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c141290 0x7f539c150530 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.079+0000 7f539be00640 1 --2- 172.21.15.7:0/975228627 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f539c06d610 0x7f539c154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.079+0000 7f539be00640 1 -- 172.21.15.7:0/975228627 >> 172.21.15.7:0/975228627 conn(0x7f539c0d1540 msgr2=0x7f539c0cf180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:56.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.080+0000 7f539be00640 1 -- 172.21.15.7:0/975228627 shutdown_connections 2024-04-25T11:22:56.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.080+0000 7f539be00640 1 -- 172.21.15.7:0/975228627 wait complete. 2024-04-25T11:22:56.085 INFO:tasks.ceph.mds.c.smithi007.stderr: -2404> 2024-04-25T11:22:37.711+0000 7ff677c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T11:22:56.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T11:22:56.259 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-04-25T11:22:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.373+0000 7f6b61e00640 1 Processor -- start 2024-04-25T11:22:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.374+0000 7f6b61e00640 1 -- start start 2024-04-25T11:22:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.376+0000 7f6b61e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6b5c146250 0x7f6b5c161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.376+0000 7f6b61e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b5c0b66b0 con 0x7f6b5c145e40 2024-04-25T11:22:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.376+0000 7f6b61e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b5c0b5b30 con 0x7f6b5c144a80 2024-04-25T11:22:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.376+0000 7f6b61e00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6b5c145b70 con 0x7f6b5c146250 2024-04-25T11:22:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.376+0000 7f6b61400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6b5c146250 0x7f6b5c161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.376+0000 7f6b60a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6b5c145e40 0x7f6b5c0b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39040/0 (socket says 172.21.15.7:39040) 2024-04-25T11:22:56.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.376+0000 7f6b60a00640 1 -- 172.21.15.7:0/3604067951 learned_addr learned my addr 172.21.15.7:0/3604067951 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.377+0000 7f6b61400640 1 -- 172.21.15.7:0/3604067951 >> v1:172.21.15.87:6789/0 conn(0x7f6b5c144a80 legacy=0x7f6b5c0b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.377+0000 7f6b61400640 1 -- 172.21.15.7:0/3604067951 >> v1:172.21.15.7:6789/0 conn(0x7f6b5c145e40 legacy=0x7f6b5c0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.377+0000 7f6b61400640 1 -- 172.21.15.7:0/3604067951 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c0b5e10 con 0x7f6b5c146250 2024-04-25T11:22:56.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.377+0000 7f6b5b400640 1 -- 172.21.15.7:0/3604067951 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3622926790 0 0) 0x7f6b5c0b66b0 con 0x7f6b5c145e40 2024-04-25T11:22:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.378+0000 7f6b61400640 1 --2- 172.21.15.7:0/3604067951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b5c146250 0x7f6b5c161700 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f6b540097a0 tx=0x7f6b5402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b264805cfecf5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.378+0000 7f6b5b400640 1 -- 172.21.15.7:0/3604067951 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54037070 con 0x7f6b5c146250 2024-04-25T11:22:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.378+0000 7f6b5b400640 1 -- 172.21.15.7:0/3604067951 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b5402ae50 con 0x7f6b5c146250 2024-04-25T11:22:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.378+0000 7f6b5b400640 1 -- 172.21.15.7:0/3604067951 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54032660 con 0x7f6b5c146250 2024-04-25T11:22:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.379+0000 7f6b61e00640 1 -- 172.21.15.7:0/3604067951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b5c146250 msgr2=0x7f6b5c161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:56.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.379+0000 7f6b61e00640 1 --2- 172.21.15.7:0/3604067951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b5c146250 0x7f6b5c161700 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f6b540097a0 tx=0x7f6b5402a140 comp rx=0 tx=0).stop 2024-04-25T11:22:56.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.379+0000 7f6b61e00640 1 -- 172.21.15.7:0/3604067951 shutdown_connections 2024-04-25T11:22:56.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.379+0000 7f6b61e00640 1 --2- 172.21.15.7:0/3604067951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b5c146250 0x7f6b5c161700 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.379+0000 7f6b61e00640 1 -- 172.21.15.7:0/3604067951 >> 172.21.15.7:0/3604067951 conn(0x7f6b5c0740f0 msgr2=0x7f6b5c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:56.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.379+0000 7f6b61e00640 1 -- 172.21.15.7:0/3604067951 shutdown_connections 2024-04-25T11:22:56.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.379+0000 7f6b61e00640 1 -- 172.21.15.7:0/3604067951 wait complete. 2024-04-25T11:22:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.382+0000 7f6b61e00640 1 Processor -- start 2024-04-25T11:22:56.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.382+0000 7f6b61e00640 1 -- start start 2024-04-25T11:22:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.384+0000 7f6b61e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b5c144a80 0x7f6b5c139bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.384+0000 7f6b61e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b5c145e40 0x7f6b5c1fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.384+0000 7f6b61e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5c146250 0x7f6b5c15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.384+0000 7f6b61e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c0b5690 con 0x7f6b5c145e40 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.384+0000 7f6b61e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c0b63d0 con 0x7f6b5c144a80 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.384+0000 7f6b61e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6b5c0b5810 con 0x7f6b5c146250 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.384+0000 7f6b5be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b5c145e40 0x7f6b5c1fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.384+0000 7f6b5be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b5c145e40 0x7f6b5c1fa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50022/0 (socket says 172.21.15.7:50022) 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.384+0000 7f6b5be00640 1 -- 172.21.15.7:0/2182894978 learned_addr learned my addr 172.21.15.7:0/2182894978 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.384+0000 7f6b60a00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b5c144a80 0x7f6b5c139bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b61400640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5c146250 0x7f6b5c15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b5be00640 1 -- 172.21.15.7:0/2182894978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5c146250 msgr2=0x7f6b5c15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b5be00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5c146250 0x7f6b5c15c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b5be00640 1 -- 172.21.15.7:0/2182894978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b5c144a80 msgr2=0x7f6b5c139bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b5be00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b5c144a80 0x7f6b5c139bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b5be00640 1 -- 172.21.15.7:0/2182894978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c1fa830 con 0x7f6b5c145e40 2024-04-25T11:22:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b5be00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b5c145e40 0x7f6b5c1fa2f0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400b3f0 tx=0x7f6b4400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b58c00640 1 -- 172.21.15.7:0/2182894978 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44015020 con 0x7f6b5c145e40 2024-04-25T11:22:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b58c00640 1 -- 172.21.15.7:0/2182894978 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b44011830 con 0x7f6b5c145e40 2024-04-25T11:22:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b58c00640 1 -- 172.21.15.7:0/2182894978 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44011c20 con 0x7f6b5c145e40 2024-04-25T11:22:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b61e00640 1 -- 172.21.15.7:0/2182894978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5c0aa040 con 0x7f6b5c145e40 2024-04-25T11:22:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.385+0000 7f6b61e00640 1 -- 172.21.15.7:0/2182894978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5c1f9af0 con 0x7f6b5c145e40 2024-04-25T11:22:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.387+0000 7f6b61e00640 1 -- 172.21.15.7:0/2182894978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b5c1f9910 con 0x7f6b5c145e40 2024-04-25T11:22:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.388+0000 7f6b58c00640 1 -- 172.21.15.7:0/2182894978 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6b44018020 con 0x7f6b5c145e40 2024-04-25T11:22:56.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.389+0000 7f6b58c00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 0x7f6b380769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.389+0000 7f6b58c00640 1 -- 172.21.15.7:0/2182894978 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6187+0+0 (secure 0 0 0) 0x7f6b44094ff0 con 0x7f6b5c145e40 2024-04-25T11:22:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.389+0000 7f6b60a00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 0x7f6b380769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.390+0000 7f6b60a00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 0x7f6b380769f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0012e0 tx=0x7f6b4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:56.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.394+0000 7f6b58c00640 1 -- 172.21.15.7:0/2182894978 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b5c1f9910 con 0x7f6b5c145e40 2024-04-25T11:22:56.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:56.639+0000 7f6b61e00640 1 -- 172.21.15.7:0/2182894978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6b5c050740 con 0x7f6b5c145e40 2024-04-25T11:22:57.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.212+0000 7f6b58c00640 1 -- 172.21.15.7:0/2182894978 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v118) ==== 104+0+0 (secure 0 0 0) 0x7f6b4401f030 con 0x7f6b5c145e40 2024-04-25T11:22:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.216+0000 7f6b61e00640 1 -- 172.21.15.7:0/2182894978 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 msgr2=0x7f6b380769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.216+0000 7f6b61e00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 0x7f6b380769f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0012e0 tx=0x7f6b4c009040 comp rx=0 tx=0).stop 2024-04-25T11:22:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.216+0000 7f6b61e00640 1 -- 172.21.15.7:0/2182894978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b5c145e40 msgr2=0x7f6b5c1fa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.216+0000 7f6b61e00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b5c145e40 0x7f6b5c1fa2f0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400b3f0 tx=0x7f6b4400b8b0 comp rx=0 tx=0).stop 2024-04-25T11:22:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.217+0000 7f6b61e00640 1 -- 172.21.15.7:0/2182894978 shutdown_connections 2024-04-25T11:22:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.217+0000 7f6b61e00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6b380744e0 0x7f6b380769f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.217+0000 7f6b61e00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b5c146250 0x7f6b5c15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.217+0000 7f6b61e00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b5c145e40 0x7f6b5c1fa2f0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.217+0000 7f6b61e00640 1 --2- 172.21.15.7:0/2182894978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b5c144a80 0x7f6b5c139bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:57.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.217+0000 7f6b61e00640 1 -- 172.21.15.7:0/2182894978 >> 172.21.15.7:0/2182894978 conn(0x7f6b5c0740f0 msgr2=0x7f6b5c0b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.217+0000 7f6b61e00640 1 -- 172.21.15.7:0/2182894978 shutdown_connections 2024-04-25T11:22:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.217+0000 7f6b61e00640 1 -- 172.21.15.7:0/2182894978 wait complete. 2024-04-25T11:22:57.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T11:22:57.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.530+0000 7f1793200640 1 Processor -- start 2024-04-25T11:22:57.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.530+0000 7f1793200640 1 -- start start 2024-04-25T11:22:57.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.532+0000 7f1793200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17940d0da0 0x7f179415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:57.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.532+0000 7f1793200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17941606c0 con 0x7f179406dc90 2024-04-25T11:22:57.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.533+0000 7f1793200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17941609a0 con 0x7f179413fae0 2024-04-25T11:22:57.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.533+0000 7f1793200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f1794160c80 con 0x7f17940d0da0 2024-04-25T11:22:57.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.533+0000 7f1791400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f179406dc90 0x7f179415d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39056/0 (socket says 172.21.15.7:39056) 2024-04-25T11:22:57.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.533+0000 7f1791400640 1 -- 172.21.15.7:0/4238214256 learned_addr learned my addr 172.21.15.7:0/4238214256 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:57.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.533+0000 7f1792800640 1 --2- 172.21.15.7:0/4238214256 >> v2:172.21.15.7:3300/0 conn(0x7f17940d0da0 0x7f179415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:57.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.533+0000 7f1790a00640 1 -- 172.21.15.7:0/4238214256 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2997531116 0 0) 0x7f17941606c0 con 0x7f179406dc90 2024-04-25T11:22:57.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.533+0000 7f1790a00640 1 -- 172.21.15.7:0/4238214256 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1778003520 con 0x7f179406dc90 2024-04-25T11:22:57.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.534+0000 7f1792800640 1 -- 172.21.15.7:0/4238214256 >> v1:172.21.15.87:6789/0 conn(0x7f179413fae0 legacy=0x7f179406d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:57.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.534+0000 7f1792800640 1 -- 172.21.15.7:0/4238214256 >> v1:172.21.15.7:6789/0 conn(0x7f179406dc90 legacy=0x7f179415d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:57.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.534+0000 7f1792800640 1 -- 172.21.15.7:0/4238214256 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1794160f60 con 0x7f17940d0da0 2024-04-25T11:22:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.535+0000 7f1792800640 1 --2- 172.21.15.7:0/4238214256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17940d0da0 0x7f179415db70 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f178c009d90 tx=0x7f178c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732907ff48f5ccba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.535+0000 7f1790a00640 1 -- 172.21.15.7:0/4238214256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c031600 con 0x7f17940d0da0 2024-04-25T11:22:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.535+0000 7f1790a00640 1 -- 172.21.15.7:0/4238214256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178c0317a0 con 0x7f17940d0da0 2024-04-25T11:22:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.535+0000 7f1790a00640 1 -- 172.21.15.7:0/4238214256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c031b70 con 0x7f17940d0da0 2024-04-25T11:22:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.536+0000 7f1793200640 1 -- 172.21.15.7:0/4238214256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17940d0da0 msgr2=0x7f179415db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.536+0000 7f1793200640 1 --2- 172.21.15.7:0/4238214256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17940d0da0 0x7f179415db70 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f178c009d90 tx=0x7f178c02a610 comp rx=0 tx=0).stop 2024-04-25T11:22:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.536+0000 7f1793200640 1 -- 172.21.15.7:0/4238214256 shutdown_connections 2024-04-25T11:22:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.536+0000 7f1793200640 1 --2- 172.21.15.7:0/4238214256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17940d0da0 0x7f179415db70 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.536+0000 7f1793200640 1 -- 172.21.15.7:0/4238214256 >> 172.21.15.7:0/4238214256 conn(0x7f1794073160 msgr2=0x7f1794073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.536+0000 7f1793200640 1 -- 172.21.15.7:0/4238214256 shutdown_connections 2024-04-25T11:22:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.536+0000 7f1793200640 1 -- 172.21.15.7:0/4238214256 wait complete. 2024-04-25T11:22:57.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.538+0000 7f1793200640 1 Processor -- start 2024-04-25T11:22:57.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.538+0000 7f1793200640 1 -- start start 2024-04-25T11:22:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.540+0000 7f1793200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f179406dc90 0x7f1794167490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.540+0000 7f1793200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17940d0da0 0x7f179413bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.540+0000 7f1793200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f179413fae0 0x7f179415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.540+0000 7f1793200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1794167bd0 con 0x7f17940d0da0 2024-04-25T11:22:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.540+0000 7f1793200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1794167a50 con 0x7f179413fae0 2024-04-25T11:22:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.540+0000 7f1793200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1794167d50 con 0x7f179406dc90 2024-04-25T11:22:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.540+0000 7f1791400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17940d0da0 0x7f179413bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.540+0000 7f1791400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17940d0da0 0x7f179413bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50036/0 (socket says 172.21.15.7:50036) 2024-04-25T11:22:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.540+0000 7f1791400640 1 -- 172.21.15.7:0/364314317 learned_addr learned my addr 172.21.15.7:0/364314317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.541+0000 7f1792800640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f179413fae0 0x7f179415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:57.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.541+0000 7f1791e00640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f179406dc90 0x7f1794167490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.541+0000 7f1791400640 1 -- 172.21.15.7:0/364314317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f179406dc90 msgr2=0x7f1794167490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.541+0000 7f1791400640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f179406dc90 0x7f1794167490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.541+0000 7f1791400640 1 -- 172.21.15.7:0/364314317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f179413fae0 msgr2=0x7f179415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.541+0000 7f1791400640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f179413fae0 0x7f179415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.541+0000 7f1791400640 1 -- 172.21.15.7:0/364314317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17941663f0 con 0x7f17940d0da0 2024-04-25T11:22:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.541+0000 7f1791400640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17940d0da0 0x7f179413bef0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f178000c3f0 tx=0x7f178000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.542+0000 7f178a000640 1 -- 172.21.15.7:0/364314317 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780017070 con 0x7f17940d0da0 2024-04-25T11:22:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.542+0000 7f178a000640 1 -- 172.21.15.7:0/364314317 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1780005d00 con 0x7f17940d0da0 2024-04-25T11:22:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.542+0000 7f1793200640 1 -- 172.21.15.7:0/364314317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17941666d0 con 0x7f17940d0da0 2024-04-25T11:22:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.542+0000 7f178a000640 1 -- 172.21.15.7:0/364314317 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780007310 con 0x7f17940d0da0 2024-04-25T11:22:57.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.542+0000 7f1793200640 1 -- 172.21.15.7:0/364314317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1794166c90 con 0x7f17940d0da0 2024-04-25T11:22:57.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.543+0000 7f1793200640 1 -- 172.21.15.7:0/364314317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179415c050 con 0x7f17940d0da0 2024-04-25T11:22:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.545+0000 7f178a000640 1 -- 172.21.15.7:0/364314317 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f178001a070 con 0x7f17940d0da0 2024-04-25T11:22:57.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.546+0000 7f178a000640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17740744b0 0x7f17740769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:57.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.546+0000 7f178a000640 1 -- 172.21.15.7:0/364314317 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 6187+0+0 (secure 0 0 0) 0x7f17800948e0 con 0x7f17940d0da0 2024-04-25T11:22:57.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.547+0000 7f1791e00640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17740744b0 0x7f17740769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:57.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.547+0000 7f1791e00640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17740744b0 0x7f17740769c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1784000f60 tx=0x7f1784009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:57.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.552+0000 7f178a000640 1 -- 172.21.15.7:0/364314317 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f178005eeb0 con 0x7f17940d0da0 2024-04-25T11:22:57.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:57.757+0000 7f1793200640 1 -- 172.21.15.7:0/364314317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f17940742d0 con 0x7f17940d0da0 2024-04-25T11:22:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.236+0000 7f178a000640 1 -- 172.21.15.7:0/364314317 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v158) ==== 181+0+0 (secure 0 0 0) 0x7f17940742d0 con 0x7f17940d0da0 2024-04-25T11:22:58.231 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-04-25T11:22:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.240+0000 7f1793200640 1 -- 172.21.15.7:0/364314317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17740744b0 msgr2=0x7f17740769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.240+0000 7f1793200640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17740744b0 0x7f17740769c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1784000f60 tx=0x7f1784009040 comp rx=0 tx=0).stop 2024-04-25T11:22:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.240+0000 7f1793200640 1 -- 172.21.15.7:0/364314317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17940d0da0 msgr2=0x7f179413bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.240+0000 7f1793200640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17940d0da0 0x7f179413bef0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f178000c3f0 tx=0x7f178000c8b0 comp rx=0 tx=0).stop 2024-04-25T11:22:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.240+0000 7f1793200640 1 -- 172.21.15.7:0/364314317 shutdown_connections 2024-04-25T11:22:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.240+0000 7f1793200640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f17740744b0 0x7f17740769c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.240+0000 7f1793200640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f179413fae0 0x7f179415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.240+0000 7f1793200640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17940d0da0 0x7f179413bef0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.240+0000 7f1793200640 1 --2- 172.21.15.7:0/364314317 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f179406dc90 0x7f1794167490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:58.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.241+0000 7f1793200640 1 -- 172.21.15.7:0/364314317 >> 172.21.15.7:0/364314317 conn(0x7f1794073160 msgr2=0x7f17940b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:58.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.241+0000 7f1793200640 1 -- 172.21.15.7:0/364314317 shutdown_connections 2024-04-25T11:22:58.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.241+0000 7f1793200640 1 -- 172.21.15.7:0/364314317 wait complete. 2024-04-25T11:22:58.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T11:22:58.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.548+0000 7efc2a200640 1 Processor -- start 2024-04-25T11:22:58.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.548+0000 7efc2a200640 1 -- start start 2024-04-25T11:22:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc2a200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efc2406d580 0x7efc2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc2a200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc240b4c60 0x7efc240b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc2a200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc2406ded0 con 0x7efc24143220 2024-04-25T11:22:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc2a200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7efc24070e40 con 0x7efc240b4c60 2024-04-25T11:22:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc2a200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efc240d02f0 con 0x7efc2406d580 2024-04-25T11:22:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc29800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc240b4c60 0x7efc240b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc29800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc240b4c60 0x7efc240b5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50052/0 (socket says 172.21.15.7:50052) 2024-04-25T11:22:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc29800640 1 -- 172.21.15.7:0/3883764262 learned_addr learned my addr 172.21.15.7:0/3883764262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc23e00640 1 --2- 172.21.15.7:0/3883764262 >> v2:172.21.15.87:3300/0 conn(0x7efc2406d580 0x7efc2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc29800640 1 -- 172.21.15.7:0/3883764262 >> v2:172.21.15.87:3300/0 conn(0x7efc2406d580 msgr2=0x7efc2406d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc29800640 1 --2- 172.21.15.7:0/3883764262 >> v2:172.21.15.87:3300/0 conn(0x7efc2406d580 0x7efc2406d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc29800640 1 -- 172.21.15.7:0/3883764262 >> v1:172.21.15.87:6789/0 conn(0x7efc24143220 legacy=0x7efc240d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:58.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.551+0000 7efc29800640 1 -- 172.21.15.7:0/3883764262 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc240d05d0 con 0x7efc240b4c60 2024-04-25T11:22:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.553+0000 7efc29800640 1 --2- 172.21.15.7:0/3883764262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc240b4c60 0x7efc240b5070 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7efc24075b30 tx=0x7efc1c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3451b9baf1ab1a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.553+0000 7efc23400640 1 -- 172.21.15.7:0/3883764262 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c036070 con 0x7efc240b4c60 2024-04-25T11:22:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.553+0000 7efc23400640 1 -- 172.21.15.7:0/3883764262 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1c031600 con 0x7efc240b4c60 2024-04-25T11:22:58.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.553+0000 7efc23400640 1 -- 172.21.15.7:0/3883764262 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0319d0 con 0x7efc240b4c60 2024-04-25T11:22:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.554+0000 7efc2a200640 1 -- 172.21.15.7:0/3883764262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc240b4c60 msgr2=0x7efc240b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.554+0000 7efc2a200640 1 --2- 172.21.15.7:0/3883764262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc240b4c60 0x7efc240b5070 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7efc24075b30 tx=0x7efc1c02a660 comp rx=0 tx=0).stop 2024-04-25T11:22:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.554+0000 7efc2a200640 1 -- 172.21.15.7:0/3883764262 shutdown_connections 2024-04-25T11:22:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.554+0000 7efc2a200640 1 --2- 172.21.15.7:0/3883764262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc240b4c60 0x7efc240b5070 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.554+0000 7efc2a200640 1 --2- 172.21.15.7:0/3883764262 >> v2:172.21.15.87:3300/0 conn(0x7efc2406d580 0x7efc2406d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.554+0000 7efc2a200640 1 -- 172.21.15.7:0/3883764262 >> 172.21.15.7:0/3883764262 conn(0x7efc24073160 msgr2=0x7efc24073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.554+0000 7efc2a200640 1 -- 172.21.15.7:0/3883764262 shutdown_connections 2024-04-25T11:22:58.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.554+0000 7efc2a200640 1 -- 172.21.15.7:0/3883764262 wait complete. 2024-04-25T11:22:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.557+0000 7efc2a200640 1 Processor -- start 2024-04-25T11:22:58.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.557+0000 7efc2a200640 1 -- start start 2024-04-25T11:22:58.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.558+0000 7efc2a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2406d580 0x7efc241573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.558+0000 7efc2a200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc240b4c60 0x7efc241f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.558+0000 7efc2a200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc24143220 0x7efc241e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.558+0000 7efc2a200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc240d00b0 con 0x7efc2406d580 2024-04-25T11:22:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.558+0000 7efc2a200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efc240cff30 con 0x7efc240b4c60 2024-04-25T11:22:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.558+0000 7efc2a200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efc240b55b0 con 0x7efc24143220 2024-04-25T11:22:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.559+0000 7efc28e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2406d580 0x7efc241573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.559+0000 7efc28e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2406d580 0x7efc241573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50066/0 (socket says 172.21.15.7:50066) 2024-04-25T11:22:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.559+0000 7efc28e00640 1 -- 172.21.15.7:0/3210022552 learned_addr learned my addr 172.21.15.7:0/3210022552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.559+0000 7efc23e00640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc240b4c60 0x7efc241f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:58.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.559+0000 7efc28e00640 1 -- 172.21.15.7:0/3210022552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc24143220 msgr2=0x7efc241e63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.559+0000 7efc28e00640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc24143220 0x7efc241e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.559+0000 7efc28e00640 1 -- 172.21.15.7:0/3210022552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc240b4c60 msgr2=0x7efc241f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.559+0000 7efc28e00640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc240b4c60 0x7efc241f35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:58.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.559+0000 7efc28e00640 1 -- 172.21.15.7:0/3210022552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc241f33b0 con 0x7efc2406d580 2024-04-25T11:22:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.560+0000 7efc28e00640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2406d580 0x7efc241573e0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7efc1800b3e0 tx=0x7efc1800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.560+0000 7efc20c00640 1 -- 172.21.15.7:0/3210022552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18015020 con 0x7efc2406d580 2024-04-25T11:22:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.560+0000 7efc20c00640 1 -- 172.21.15.7:0/3210022552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc18011850 con 0x7efc2406d580 2024-04-25T11:22:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.560+0000 7efc20c00640 1 -- 172.21.15.7:0/3210022552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18011c40 con 0x7efc2406d580 2024-04-25T11:22:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.560+0000 7efc2a200640 1 -- 172.21.15.7:0/3210022552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc241e5090 con 0x7efc2406d580 2024-04-25T11:22:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.560+0000 7efc2a200640 1 -- 172.21.15.7:0/3210022552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc241e5630 con 0x7efc2406d580 2024-04-25T11:22:58.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.561+0000 7efc2a200640 1 -- 172.21.15.7:0/3210022552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc24050740 con 0x7efc2406d580 2024-04-25T11:22:58.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.563+0000 7efc20c00640 1 -- 172.21.15.7:0/3210022552 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7efc18018020 con 0x7efc2406d580 2024-04-25T11:22:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.563+0000 7efc20c00640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc040744b0 0x7efc040769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.564+0000 7efc20c00640 1 -- 172.21.15.7:0/3210022552 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5749+0+0 (secure 0 0 0) 0x7efc18095290 con 0x7efc2406d580 2024-04-25T11:22:58.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.564+0000 7efc23e00640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc040744b0 0x7efc040769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:58.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.565+0000 7efc23e00640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc040744b0 0x7efc040769c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7efc2415b0d0 tx=0x7efc10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:58.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.569+0000 7efc20c00640 1 -- 172.21.15.7:0/3210022552 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7efc1801f020 con 0x7efc2406d580 2024-04-25T11:22:58.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:58.787+0000 7efc2a200640 1 -- 172.21.15.7:0/3210022552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efc24074320 con 0x7efc2406d580 2024-04-25T11:22:59.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.240+0000 7efc20c00640 1 -- 172.21.15.7:0/3210022552 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v159) ==== 169+0+0 (secure 0 0 0) 0x7efc1805f950 con 0x7efc2406d580 2024-04-25T11:22:59.235 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-04-25T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.244+0000 7efc2a200640 1 -- 172.21.15.7:0/3210022552 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc040744b0 msgr2=0x7efc040769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.244+0000 7efc2a200640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc040744b0 0x7efc040769c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7efc2415b0d0 tx=0x7efc10008040 comp rx=0 tx=0).stop 2024-04-25T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.244+0000 7efc2a200640 1 -- 172.21.15.7:0/3210022552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2406d580 msgr2=0x7efc241573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.244+0000 7efc2a200640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2406d580 0x7efc241573e0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7efc1800b3e0 tx=0x7efc1800b8a0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.247+0000 7efc2a200640 1 -- 172.21.15.7:0/3210022552 shutdown_connections 2024-04-25T11:22:59.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.247+0000 7efc2a200640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7efc040744b0 0x7efc040769c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.247+0000 7efc2a200640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc24143220 0x7efc241e63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.247+0000 7efc2a200640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc240b4c60 0x7efc241f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.247+0000 7efc2a200640 1 --2- 172.21.15.7:0/3210022552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2406d580 0x7efc241573e0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.247+0000 7efc2a200640 1 -- 172.21.15.7:0/3210022552 >> 172.21.15.7:0/3210022552 conn(0x7efc24073160 msgr2=0x7efc240b6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:59.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.250+0000 7efc2a200640 1 -- 172.21.15.7:0/3210022552 shutdown_connections 2024-04-25T11:22:59.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.251+0000 7efc2a200640 1 -- 172.21.15.7:0/3210022552 wait complete. 2024-04-25T11:22:59.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T11:22:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.553+0000 7fe21fe00640 1 Processor -- start 2024-04-25T11:22:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.553+0000 7fe21fe00640 1 -- start start 2024-04-25T11:22:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.555+0000 7fe21fe00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe2200b4690 0x7fe2200b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.555+0000 7fe21fe00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2200b5c40 con 0x7fe220143b10 2024-04-25T11:22:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.555+0000 7fe21fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2200b5f20 con 0x7fe220144630 2024-04-25T11:22:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.555+0000 7fe21fe00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe2200b6200 con 0x7fe2200b4690 2024-04-25T11:22:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.556+0000 7fe21e000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe220143b10 0x7fe220143f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39062/0 (socket says 172.21.15.7:39062) 2024-04-25T11:22:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.556+0000 7fe21e000640 1 -- 172.21.15.7:0/1977190919 learned_addr learned my addr 172.21.15.7:0/1977190919 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:22:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.556+0000 7fe21ea00640 1 --2- 172.21.15.7:0/1977190919 >> v2:172.21.15.87:3300/0 conn(0x7fe2200b4690 0x7fe2200b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.556+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425071820 0 0) 0x7fe2200b5c40 con 0x7fe220143b10 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.556+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe204003520 con 0x7fe220143b10 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 123134391 0 0) 0x7fe204003520 con 0x7fe220143b10 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 >> v2:172.21.15.87:3300/0 conn(0x7fe2200b4690 msgr2=0x7fe2200b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 --2- 172.21.15.7:0/1977190919 >> v2:172.21.15.87:3300/0 conn(0x7fe2200b4690 0x7fe2200b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 >> v1:172.21.15.87:6789/0 conn(0x7fe220144630 legacy=0x7fe2200cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2200b64e0 con 0x7fe220143b10 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fe208001670 con 0x7fe220143b10 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 >> v1:172.21.15.7:6789/0 conn(0x7fe220143b10 legacy=0x7fe220143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe204003a00 0x7fe204003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe204004800 0x7fe204004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe204005270 0x7fe20400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2200b6200 con 0x7fe204003a00 2024-04-25T11:22:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe20400c180 con 0x7fe204005270 2024-04-25T11:22:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe20400c380 con 0x7fe204004800 2024-04-25T11:22:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.557+0000 7fe21ea00640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe204003a00 0x7fe204003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.558+0000 7fe21f400640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe204005270 0x7fe20400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.558+0000 7fe21e000640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe204004800 0x7fe204004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.558+0000 7fe21ea00640 1 -- 172.21.15.7:0/1977190919 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe204004800 msgr2=0x7fe204004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.558+0000 7fe21ea00640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe204004800 0x7fe204004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.558+0000 7fe21ea00640 1 -- 172.21.15.7:0/1977190919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe204005270 msgr2=0x7fe20400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:22:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.558+0000 7fe21ea00640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe204005270 0x7fe20400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.558+0000 7fe21ea00640 1 -- 172.21.15.7:0/1977190919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe204003520 con 0x7fe204003a00 2024-04-25T11:22:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.558+0000 7fe21f400640 1 -- 172.21.15.7:0/1977190919 reap_dead start 2024-04-25T11:22:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.558+0000 7fe21e000640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe204004800 0x7fe204004c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:59.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.559+0000 7fe21ea00640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe204003a00 0x7fe204003e70 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fe210009fc0 tx=0x7fe204005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08ca55da149fb86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:59.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.559+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210009c10 con 0x7fe204003a00 2024-04-25T11:22:59.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.559+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe210036070 con 0x7fe204003a00 2024-04-25T11:22:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.559+0000 7fe21d600640 1 -- 172.21.15.7:0/1977190919 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210031880 con 0x7fe204003a00 2024-04-25T11:22:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.560+0000 7fe21fe00640 1 -- 172.21.15.7:0/1977190919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe204003a00 msgr2=0x7fe204003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.560+0000 7fe21fe00640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe204003a00 0x7fe204003e70 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fe210009fc0 tx=0x7fe204005830 comp rx=0 tx=0).stop 2024-04-25T11:22:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.560+0000 7fe21fe00640 1 -- 172.21.15.7:0/1977190919 shutdown_connections 2024-04-25T11:22:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.560+0000 7fe21fe00640 1 --2- 172.21.15.7:0/1977190919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe204003a00 0x7fe204003e70 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.560+0000 7fe21fe00640 1 -- 172.21.15.7:0/1977190919 >> 172.21.15.7:0/1977190919 conn(0x7fe220074140 msgr2=0x7fe22006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.560+0000 7fe21fe00640 1 -- 172.21.15.7:0/1977190919 shutdown_connections 2024-04-25T11:22:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.560+0000 7fe21fe00640 1 -- 172.21.15.7:0/1977190919 wait complete. 2024-04-25T11:22:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.563+0000 7fe21fe00640 1 Processor -- start 2024-04-25T11:22:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.563+0000 7fe21fe00640 1 -- start start 2024-04-25T11:22:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.564+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe204003a00 0x7fe2201378a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.564+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe22013c3d0 0x7fe22015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.564+0000 7fe21fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22013edb0 0x7fe22013f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe220143200 con 0x7fe22013edb0 2024-04-25T11:22:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe220070fa0 con 0x7fe204003a00 2024-04-25T11:22:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe220143380 con 0x7fe22013c3d0 2024-04-25T11:22:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22013edb0 0x7fe22013f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22013edb0 0x7fe22013f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50088/0 (socket says 172.21.15.7:50088) 2024-04-25T11:22:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21f400640 1 -- 172.21.15.7:0/2272057064 learned_addr learned my addr 172.21.15.7:0/2272057064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:22:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21ea00640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe204003a00 0x7fe2201378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21e000640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe22013c3d0 0x7fe22015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21f400640 1 -- 172.21.15.7:0/2272057064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe22013c3d0 msgr2=0x7fe22015c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21f400640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe22013c3d0 0x7fe22015c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21f400640 1 -- 172.21.15.7:0/2272057064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe204003a00 msgr2=0x7fe2201378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21f400640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe204003a00 0x7fe2201378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21f400640 1 -- 172.21.15.7:0/2272057064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22013f700 con 0x7fe22013edb0 2024-04-25T11:22:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.565+0000 7fe21ea00640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe204003a00 0x7fe2201378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:22:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.566+0000 7fe21f400640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22013edb0 0x7fe22013f1c0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fe22006ef20 tx=0x7fe21800e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.566+0000 7fe216a00640 1 -- 172.21.15.7:0/2272057064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218019070 con 0x7fe22013edb0 2024-04-25T11:22:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.566+0000 7fe216a00640 1 -- 172.21.15.7:0/2272057064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe218005280 con 0x7fe22013edb0 2024-04-25T11:22:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.566+0000 7fe21fe00640 1 -- 172.21.15.7:0/2272057064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2201380f0 con 0x7fe22013edb0 2024-04-25T11:22:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.566+0000 7fe216a00640 1 -- 172.21.15.7:0/2272057064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218007030 con 0x7fe22013edb0 2024-04-25T11:22:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.566+0000 7fe21fe00640 1 -- 172.21.15.7:0/2272057064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2201386b0 con 0x7fe22013edb0 2024-04-25T11:22:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.567+0000 7fe21fe00640 1 -- 172.21.15.7:0/2272057064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe220137e60 con 0x7fe22013edb0 2024-04-25T11:22:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.569+0000 7fe216a00640 1 -- 172.21.15.7:0/2272057064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe21801c070 con 0x7fe22013edb0 2024-04-25T11:22:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.570+0000 7fe216a00640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1fc0744b0 0x7fe1fc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:22:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.570+0000 7fe216a00640 1 -- 172.21.15.7:0/2272057064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7fe218094220 con 0x7fe22013edb0 2024-04-25T11:22:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.570+0000 7fe21ea00640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1fc0744b0 0x7fe1fc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:22:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.571+0000 7fe21ea00640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1fc0744b0 0x7fe1fc0769c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe210035000 tx=0x7fe21002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:22:59.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.575+0000 7fe216a00640 1 -- 172.21.15.7:0/2272057064 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe22013f700 con 0x7fe22013edb0 2024-04-25T11:22:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.787+0000 7fe21fe00640 1 -- 172.21.15.7:0/2272057064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe220138dd0 con 0x7fe22013edb0 2024-04-25T11:22:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.795+0000 7fe216a00640 1 -- 172.21.15.7:0/2272057064 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v67) ==== 104+0+0 (secure 0 0 0) 0x7fe21805eb10 con 0x7fe22013edb0 2024-04-25T11:22:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.798+0000 7fe21fe00640 1 -- 172.21.15.7:0/2272057064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1fc0744b0 msgr2=0x7fe1fc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.798+0000 7fe21fe00640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1fc0744b0 0x7fe1fc0769c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe210035000 tx=0x7fe21002f000 comp rx=0 tx=0).stop 2024-04-25T11:22:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.798+0000 7fe21fe00640 1 -- 172.21.15.7:0/2272057064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22013edb0 msgr2=0x7fe22013f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:22:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.798+0000 7fe21fe00640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22013edb0 0x7fe22013f1c0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fe22006ef20 tx=0x7fe21800e8a0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.798+0000 7fe21fe00640 1 -- 172.21.15.7:0/2272057064 shutdown_connections 2024-04-25T11:22:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.798+0000 7fe21fe00640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1fc0744b0 0x7fe1fc0769c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.798+0000 7fe21fe00640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe22013edb0 0x7fe22013f1c0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.798+0000 7fe21fe00640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe22013c3d0 0x7fe22015c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.798+0000 7fe21fe00640 1 --2- 172.21.15.7:0/2272057064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe204003a00 0x7fe2201378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:22:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.798+0000 7fe21fe00640 1 -- 172.21.15.7:0/2272057064 >> 172.21.15.7:0/2272057064 conn(0x7fe220074140 msgr2=0x7fe22015f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:22:59.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.798+0000 7fe21fe00640 1 -- 172.21.15.7:0/2272057064 shutdown_connections 2024-04-25T11:22:59.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:22:59.799+0000 7fe21fe00640 1 -- 172.21.15.7:0/2272057064 wait complete. 2024-04-25T11:22:59.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug' 2024-04-25T11:23:00.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.115+0000 7f8e87a00640 1 Processor -- start 2024-04-25T11:23:00.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.116+0000 7f8e87a00640 1 -- start start 2024-04-25T11:23:00.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.118+0000 7f8e87a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8e8006f1e0 0x7f8e8006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:00.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.118+0000 7f8e87a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e80162700 con 0x7f8e8015f180 2024-04-25T11:23:00.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.118+0000 7f8e87a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e801629e0 con 0x7f8e8006d580 2024-04-25T11:23:00.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.118+0000 7f8e87a00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8e8015f590 con 0x7f8e8006f1e0 2024-04-25T11:23:00.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.118+0000 7f8e85000640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8e8015f180 0x7f8e80161c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34490/0 (socket says 172.21.15.7:34490) 2024-04-25T11:23:00.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.118+0000 7f8e85000640 1 -- 172.21.15.7:0/3089236045 learned_addr learned my addr 172.21.15.7:0/3089236045 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e85a00640 1 --2- 172.21.15.7:0/3089236045 >> v2:172.21.15.87:3300/0 conn(0x7f8e8006f1e0 0x7f8e8006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3052164928 0 0) 0x7f8e80162700 con 0x7f8e8015f180 2024-04-25T11:23:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e64003520 con 0x7f8e8015f180 2024-04-25T11:23:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2416283889 0 0) 0x7f8e64003520 con 0x7f8e8015f180 2024-04-25T11:23:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 >> v2:172.21.15.87:3300/0 conn(0x7f8e8006f1e0 msgr2=0x7f8e8006f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 --2- 172.21.15.7:0/3089236045 >> v2:172.21.15.87:3300/0 conn(0x7f8e8006f1e0 0x7f8e8006f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 >> v1:172.21.15.87:6789/0 conn(0x7f8e8006d580 legacy=0x7f8e8006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8015e980 con 0x7f8e8015f180 2024-04-25T11:23:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f8e700019f0 con 0x7f8e8015f180 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 >> v1:172.21.15.7:6789/0 conn(0x7f8e8015f180 legacy=0x7f8e80161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 --2- 172.21.15.7:0/3089236045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e640039b0 0x7f8e64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 --2- 172.21.15.7:0/3089236045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e64004800 0x7f8e64004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 --2- 172.21.15.7:0/3089236045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e64005350 0x7f8e6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8015f590 con 0x7f8e64004800 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6400c280 con 0x7f8e640039b0 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.119+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8e6400c480 con 0x7f8e64005350 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.120+0000 7f8e7fe00640 1 --2- 172.21.15.7:0/3089236045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e64004800 0x7f8e64004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.120+0000 7f8e85000640 1 --2- 172.21.15.7:0/3089236045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e640039b0 0x7f8e64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.120+0000 7f8e85a00640 1 --2- 172.21.15.7:0/3089236045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e64005350 0x7f8e6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.120+0000 7f8e7fe00640 1 -- 172.21.15.7:0/3089236045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e64005350 msgr2=0x7f8e6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.120+0000 7f8e7fe00640 1 --2- 172.21.15.7:0/3089236045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e64005350 0x7f8e6400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.120+0000 7f8e7fe00640 1 -- 172.21.15.7:0/3089236045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e640039b0 msgr2=0x7f8e64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.120+0000 7f8e7fe00640 1 --2- 172.21.15.7:0/3089236045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e640039b0 0x7f8e64003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.120+0000 7f8e7fe00640 1 -- 172.21.15.7:0/3089236045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e64003520 con 0x7f8e64004800 2024-04-25T11:23:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.120+0000 7f8e85a00640 1 -- 172.21.15.7:0/3089236045 reap_dead start 2024-04-25T11:23:00.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.121+0000 7f8e7fe00640 1 --2- 172.21.15.7:0/3089236045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e64004800 0x7f8e64004c10 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f8e68003db0 tx=0x7f8e64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7376dadb331b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:00.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.122+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68037070 con 0x7f8e64004800 2024-04-25T11:23:00.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.122+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e6802a9f0 con 0x7f8e64004800 2024-04-25T11:23:00.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.122+0000 7f8e7f400640 1 -- 172.21.15.7:0/3089236045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68003980 con 0x7f8e64004800 2024-04-25T11:23:00.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.122+0000 7f8e87a00640 1 -- 172.21.15.7:0/3089236045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e64004800 msgr2=0x7f8e64004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:00.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.122+0000 7f8e87a00640 1 --2- 172.21.15.7:0/3089236045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e64004800 0x7f8e64004c10 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f8e68003db0 tx=0x7f8e64005830 comp rx=0 tx=0).stop 2024-04-25T11:23:00.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.122+0000 7f8e87a00640 1 -- 172.21.15.7:0/3089236045 shutdown_connections 2024-04-25T11:23:00.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.122+0000 7f8e87a00640 1 --2- 172.21.15.7:0/3089236045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e64004800 0x7f8e64004c10 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:00.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.122+0000 7f8e87a00640 1 -- 172.21.15.7:0/3089236045 >> 172.21.15.7:0/3089236045 conn(0x7f8e800732b0 msgr2=0x7f8e800736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:00.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.123+0000 7f8e87a00640 1 -- 172.21.15.7:0/3089236045 shutdown_connections 2024-04-25T11:23:00.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.123+0000 7f8e87a00640 1 -- 172.21.15.7:0/3089236045 wait complete. 2024-04-25T11:23:00.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.125+0000 7f8e87a00640 1 Processor -- start 2024-04-25T11:23:00.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.125+0000 7f8e87a00640 1 -- start start 2024-04-25T11:23:00.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e87a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e64004800 0x7f8e8014e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:00.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e87a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e8009d710 0x7f8e800aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:00.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e87a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8009db20 0x7f8e800b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:00.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e87a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8015e670 con 0x7f8e8009db20 2024-04-25T11:23:00.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e87a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8e80158ae0 con 0x7f8e64004800 2024-04-25T11:23:00.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e87a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8e800d1310 con 0x7f8e8009d710 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8009db20 0x7f8e800b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e85a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8009db20 0x7f8e800b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47856/0 (socket says 172.21.15.7:47856) 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e85a00640 1 -- 172.21.15.7:0/2354316136 learned_addr learned my addr 172.21.15.7:0/2354316136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e85000640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e64004800 0x7f8e8014e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e7fe00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e8009d710 0x7f8e800aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e85a00640 1 -- 172.21.15.7:0/2354316136 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e8009d710 msgr2=0x7f8e800aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e85a00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e8009d710 0x7f8e800aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e85a00640 1 -- 172.21.15.7:0/2354316136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e64004800 msgr2=0x7f8e8014e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e85a00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e64004800 0x7f8e8014e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e85a00640 1 -- 172.21.15.7:0/2354316136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8009e0a0 con 0x7f8e8009db20 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.127+0000 7f8e7fe00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e8009d710 0x7f8e800aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.128+0000 7f8e85000640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e64004800 0x7f8e8014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:00.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.128+0000 7f8e85a00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8009db20 0x7f8e800b04c0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f8e800d16d0 tx=0x7f8e7800e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.128+0000 7f8e7cc00640 1 -- 172.21.15.7:0/2354316136 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7800f040 con 0x7f8e8009db20 2024-04-25T11:23:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.128+0000 7f8e7cc00640 1 -- 172.21.15.7:0/2354316136 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e78019070 con 0x7f8e8009db20 2024-04-25T11:23:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.128+0000 7f8e7cc00640 1 -- 172.21.15.7:0/2354316136 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78013850 con 0x7f8e8009db20 2024-04-25T11:23:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.128+0000 7f8e87a00640 1 -- 172.21.15.7:0/2354316136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e800b6980 con 0x7f8e8009db20 2024-04-25T11:23:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.128+0000 7f8e87a00640 1 -- 172.21.15.7:0/2354316136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e800b5ea0 con 0x7f8e8009db20 2024-04-25T11:23:00.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.129+0000 7f8e87a00640 1 -- 172.21.15.7:0/2354316136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e80050740 con 0x7f8e8009db20 2024-04-25T11:23:00.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.131+0000 7f8e7cc00640 1 -- 172.21.15.7:0/2354316136 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f8e780139f0 con 0x7f8e8009db20 2024-04-25T11:23:00.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.132+0000 7f8e7cc00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8e5c0744b0 0x7f8e5c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:00.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.132+0000 7f8e7cc00640 1 -- 172.21.15.7:0/2354316136 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f8e78094ae0 con 0x7f8e8009db20 2024-04-25T11:23:00.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.132+0000 7f8e85000640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8e5c0744b0 0x7f8e5c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:00.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.133+0000 7f8e85000640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8e5c0744b0 0x7f8e5c0769c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8e70001580 tx=0x7f8e700014b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:00.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.136+0000 7f8e7cc00640 1 -- 172.21.15.7:0/2354316136 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f8e7805f340 con 0x7f8e8009db20 2024-04-25T11:23:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:00.419+0000 7f8e87a00640 1 -- 172.21.15.7:0/2354316136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]} v 0) -- 0x7f8e80074250 con 0x7f8e8009db20 2024-04-25T11:23:01.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.261+0000 7f8e7cc00640 1 -- 172.21.15.7:0/2354316136 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_client_limits.TestClientLimits.test_client_release_bug"]}]=0 v662) ==== 151+0+0 (secure 0 0 0) 0x7f8e7805ec40 con 0x7f8e8009db20 2024-04-25T11:23:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.265+0000 7f8e87a00640 1 -- 172.21.15.7:0/2354316136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8e5c0744b0 msgr2=0x7f8e5c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.265+0000 7f8e87a00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8e5c0744b0 0x7f8e5c0769c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8e70001580 tx=0x7f8e700014b0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.265+0000 7f8e87a00640 1 -- 172.21.15.7:0/2354316136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8009db20 msgr2=0x7f8e800b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.265+0000 7f8e87a00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8009db20 0x7f8e800b04c0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f8e800d16d0 tx=0x7f8e7800e830 comp rx=0 tx=0).stop 2024-04-25T11:23:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.265+0000 7f8e87a00640 1 -- 172.21.15.7:0/2354316136 shutdown_connections 2024-04-25T11:23:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.265+0000 7f8e87a00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f8e5c0744b0 0x7f8e5c0769c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.265+0000 7f8e87a00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e8009db20 0x7f8e800b04c0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.265+0000 7f8e87a00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e8009d710 0x7f8e800aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.265+0000 7f8e87a00640 1 --2- 172.21.15.7:0/2354316136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e64004800 0x7f8e8014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.265+0000 7f8e87a00640 1 -- 172.21.15.7:0/2354316136 >> 172.21.15.7:0/2354316136 conn(0x7f8e800732b0 msgr2=0x7f8e80161c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.266+0000 7f8e87a00640 1 -- 172.21.15.7:0/2354316136 shutdown_connections 2024-04-25T11:23:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:01.266+0000 7f8e87a00640 1 -- 172.21.15.7:0/2354316136 wait complete. 2024-04-25T11:23:01.290 INFO:tasks.cephfs_test_runner:When a client has a bug (which we will simulate) preventing it from releasing caps, ... skipped 'Require FUSE client to inject client release failure' 2024-04-25T11:23:01.291 INFO:tasks.cephfs_test_runner: 2024-04-25T11:23:01.291 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T11:23:01.291 INFO:tasks.cephfs_test_runner:Ran 10 tests in 683.617s 2024-04-25T11:23:01.291 INFO:tasks.cephfs_test_runner: 2024-04-25T11:23:01.291 INFO:tasks.cephfs_test_runner:OK (skipped=4) 2024-04-25T11:23:01.291 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T11:23:01.302 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T11:23:01.314 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T11:23:01.315 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:23:01.315 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:23:01.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:23:01.372 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-04-25T11:23:01.372 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:23:01.427 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T11:23:01.427 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:23:01.427 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:23:01.484 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:23:01.484 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T11:23:01.484 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T11:23:01.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052716,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T11:23:01.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:01.536 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:23:01.597 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T11:23:01.597 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T11:23:01.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:01.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T11:23:01.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:01.689 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T11:23:01.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:01.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-04-25T11:23:01.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:01.897 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-04-25T11:23:01.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:01.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-04-25T11:23:02.069 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-04-25T11:23:02.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:23:02.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:02.073 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:23:02.128 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T11:23:02.128 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T11:23:02.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:02.129 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T11:23:02.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:02.207 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T11:23:02.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:02.283 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-04-25T11:23:02.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:02.410 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T11:23:02.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T11:23:02.452 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-04-25T11:23:02.570 INFO:teuthology.orchestra.run.smithi087.stderr:Cannot find device "ceph-brx" 2024-04-25T11:23:02.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:23:02.574 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T11:23:02.584 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T11:23:02.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:23:02.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.885+0000 7fd43a600640 1 Processor -- start 2024-04-25T11:23:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.885+0000 7fd43a600640 1 -- start start 2024-04-25T11:23:02.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.887+0000 7fd43a600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4340aed40 0x7fd43415dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:02.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.888+0000 7fd43a600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4340cebb0 con 0x7fd4340af420 2024-04-25T11:23:02.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.888+0000 7fd43a600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4340b64b0 con 0x7fd4340b5220 2024-04-25T11:23:02.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.888+0000 7fd43a600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd43415e1f0 con 0x7fd4340aed40 2024-04-25T11:23:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.888+0000 7fd439c00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd4340af420 0x7fd4340af830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34504/0 (socket says 172.21.15.7:34504) 2024-04-25T11:23:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.888+0000 7fd439c00640 1 -- 172.21.15.7:0/820246082 learned_addr learned my addr 172.21.15.7:0/820246082 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.888+0000 7fd439200640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4340aed40 0x7fd43415dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.888+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2667169229 0 0) 0x7fd42c001490 con 0x7fd4340af420 2024-04-25T11:23:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd418003520 con 0x7fd4340af420 2024-04-25T11:23:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 584591515 0 0) 0x7fd418003520 con 0x7fd4340af420 2024-04-25T11:23:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4340aed40 msgr2=0x7fd43415dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4340aed40 0x7fd43415dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 >> v1:172.21.15.87:6789/0 conn(0x7fd4340b5220 legacy=0x7fd4340ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43406c610 con 0x7fd4340af420 2024-04-25T11:23:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fd42c001bc0 con 0x7fd4340af420 2024-04-25T11:23:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 >> v1:172.21.15.7:6789/0 conn(0x7fd4340af420 legacy=0x7fd4340af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4180039e0 0x7fd418003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4340aed40 0x7fd418004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd418004d40 0x7fd418005150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:02.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd43415e1f0 con 0x7fd418004d40 2024-04-25T11:23:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd41800baf0 con 0x7fd4180039e0 2024-04-25T11:23:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.889+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd41800bcf0 con 0x7fd4340aed40 2024-04-25T11:23:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.890+0000 7fd439c00640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd418004d40 0x7fd418005150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.890+0000 7fd439200640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4180039e0 0x7fd418003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.890+0000 7fd433e00640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4340aed40 0x7fd418004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.890+0000 7fd439c00640 1 -- 172.21.15.7:0/820246082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4340aed40 msgr2=0x7fd418004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.890+0000 7fd439c00640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4340aed40 0x7fd418004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.890+0000 7fd439c00640 1 -- 172.21.15.7:0/820246082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4180039e0 msgr2=0x7fd418003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.890+0000 7fd439c00640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4180039e0 0x7fd418003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.890+0000 7fd439c00640 1 -- 172.21.15.7:0/820246082 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd418003520 con 0x7fd418004d40 2024-04-25T11:23:02.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.891+0000 7fd439c00640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd418004d40 0x7fd418005150 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00aa30 tx=0x7fd418005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8343670bef38fe02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:02.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.892+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c00a650 con 0x7fd418004d40 2024-04-25T11:23:02.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.892+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42c02f070 con 0x7fd418004d40 2024-04-25T11:23:02.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.892+0000 7fd433400640 1 -- 172.21.15.7:0/820246082 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c034470 con 0x7fd418004d40 2024-04-25T11:23:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.892+0000 7fd43a600640 1 -- 172.21.15.7:0/820246082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd418004d40 msgr2=0x7fd418005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.892+0000 7fd43a600640 1 --2- 172.21.15.7:0/820246082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd418004d40 0x7fd418005150 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00aa30 tx=0x7fd418005830 comp rx=0 tx=0).stop 2024-04-25T11:23:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.892+0000 7fd439c00640 1 -- 172.21.15.7:0/820246082 reap_dead start 2024-04-25T11:23:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.893+0000 7fd43a600640 1 -- 172.21.15.7:0/820246082 shutdown_connections 2024-04-25T11:23:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.893+0000 7fd43a600640 1 -- 172.21.15.7:0/820246082 >> 172.21.15.7:0/820246082 conn(0x7fd4340b3820 msgr2=0x7fd4340b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.893+0000 7fd43a600640 1 -- 172.21.15.7:0/820246082 shutdown_connections 2024-04-25T11:23:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.893+0000 7fd43a600640 1 -- 172.21.15.7:0/820246082 wait complete. 2024-04-25T11:23:02.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.895+0000 7fd43a600640 1 Processor -- start 2024-04-25T11:23:02.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.895+0000 7fd43a600640 1 -- start start 2024-04-25T11:23:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.897+0000 7fd43a600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd43414d4e0 0x7fd434161700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.897+0000 7fd43a600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd434161c40 0x7fd434158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.897+0000 7fd43a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4341587c0 0x7fd43415e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.897+0000 7fd43a600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd43406ceb0 con 0x7fd4341587c0 2024-04-25T11:23:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.897+0000 7fd43a600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd43406cbd0 con 0x7fd43414d4e0 2024-04-25T11:23:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.897+0000 7fd43a600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4340b6110 con 0x7fd434161c40 2024-04-25T11:23:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.897+0000 7fd439c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4341587c0 0x7fd43415e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.897+0000 7fd439c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4341587c0 0x7fd43415e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47874/0 (socket says 172.21.15.7:47874) 2024-04-25T11:23:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.898+0000 7fd439c00640 1 -- 172.21.15.7:0/1053016756 learned_addr learned my addr 172.21.15.7:0/1053016756 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:02.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.898+0000 7fd439200640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd43414d4e0 0x7fd434161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:02.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.898+0000 7fd433e00640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd434161c40 0x7fd434158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.898+0000 7fd439c00640 1 -- 172.21.15.7:0/1053016756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd434161c40 msgr2=0x7fd434158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.898+0000 7fd439c00640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd434161c40 0x7fd434158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.898+0000 7fd439c00640 1 -- 172.21.15.7:0/1053016756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd43414d4e0 msgr2=0x7fd434161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.898+0000 7fd439c00640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd43414d4e0 0x7fd434161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.898+0000 7fd439c00640 1 -- 172.21.15.7:0/1053016756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd434162050 con 0x7fd4341587c0 2024-04-25T11:23:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.898+0000 7fd439200640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd43414d4e0 0x7fd434161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.898+0000 7fd439c00640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4341587c0 0x7fd43415e3a0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00a0b0 tx=0x7fd43406c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.899+0000 7fd430c00640 1 -- 172.21.15.7:0/1053016756 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c00a650 con 0x7fd4341587c0 2024-04-25T11:23:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.899+0000 7fd430c00640 1 -- 172.21.15.7:0/1053016756 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42c001bb0 con 0x7fd4341587c0 2024-04-25T11:23:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.899+0000 7fd430c00640 1 -- 172.21.15.7:0/1053016756 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0348b0 con 0x7fd4341587c0 2024-04-25T11:23:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.899+0000 7fd43a600640 1 -- 172.21.15.7:0/1053016756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd434158eb0 con 0x7fd4341587c0 2024-04-25T11:23:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.899+0000 7fd43a600640 1 -- 172.21.15.7:0/1053016756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd434155de0 con 0x7fd4341587c0 2024-04-25T11:23:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.900+0000 7fd43a600640 1 -- 172.21.15.7:0/1053016756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd434050740 con 0x7fd4341587c0 2024-04-25T11:23:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.902+0000 7fd430c00640 1 -- 172.21.15.7:0/1053016756 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fd42c034a50 con 0x7fd4341587c0 2024-04-25T11:23:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.903+0000 7fd430c00640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4140744b0 0x7fd4140769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.903+0000 7fd430c00640 1 -- 172.21.15.7:0/1053016756 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7fd42c0b4b70 con 0x7fd4341587c0 2024-04-25T11:23:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.903+0000 7fd439200640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4140744b0 0x7fd4140769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.904+0000 7fd439200640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4140744b0 0x7fd4140769c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd43414dd50 tx=0x7fd424008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:02.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:02.908+0000 7fd430c00640 1 -- 172.21.15.7:0/1053016756 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fd434162050 con 0x7fd4341587c0 2024-04-25T11:23:03.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.104+0000 7fd43a600640 1 -- 172.21.15.7:0/1053016756 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd434158bd0 con 0x7fd4140744b0 2024-04-25T11:23:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.106+0000 7fd430c00640 1 -- 172.21.15.7:0/1053016756 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13494 (secure 0 0 0) 0x7fd434158bd0 con 0x7fd4140744b0 2024-04-25T11:23:03.101 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:23:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 -- 172.21.15.7:0/1053016756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4140744b0 msgr2=0x7fd4140769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4140744b0 0x7fd4140769c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd43414dd50 tx=0x7fd424008040 comp rx=0 tx=0).stop 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 -- 172.21.15.7:0/1053016756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4341587c0 msgr2=0x7fd43415e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4341587c0 0x7fd43415e3a0 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00a0b0 tx=0x7fd43406c7e0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 -- 172.21.15.7:0/1053016756 shutdown_connections 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fd4140744b0 0x7fd4140769c0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4341587c0 0x7fd43415e3a0 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd434161c40 0x7fd434158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 --2- 172.21.15.7:0/1053016756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd43414d4e0 0x7fd434161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 -- 172.21.15.7:0/1053016756 >> 172.21.15.7:0/1053016756 conn(0x7fd4340b3820 msgr2=0x7fd434070840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 -- 172.21.15.7:0/1053016756 shutdown_connections 2024-04-25T11:23:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.109+0000 7fd43a600640 1 -- 172.21.15.7:0/1053016756 wait complete. 2024-04-25T11:23:03.131 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":539,"stamp":"2024-04-25T11:23:02.083511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":286900,"kb_used_data":16648,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687820,"statfs":{"total":773094113280,"available":772800327680,"internally_reserved":0,"allocated":17047552,"data_stored":8536300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4195,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.006061"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":388,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T11:22:59.253029+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:22:59.253029+0000","last_peered":"2024-04-25T11:22:59.253029+0000","last_clean":"2024-04-25T11:22:59.253029+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:22:59.253029+0000","last_undegraded":"2024-04-25T11:22:59.253029+0000","last_fullsized":"2024-04-25T11:22:59.253029+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33708,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338132,"statfs":{"total":96636764160,"available":96602247168,"internally_reserved":0,"allocated":2588672,"data_stored":1514336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476892,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33116,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1978368,"data_stored":918011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476893,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37272,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1978368,"data_stored":918011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509596,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509597,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37272,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1978368,"data_stored":918011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37852,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333988,"statfs":{"total":96636764160,"available":96598003712,"internally_reserved":0,"allocated":2572288,"data_stored":1508379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542301,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1961984,"data_stored":911616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:23:03.132 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:23:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.422+0000 7f9408a00640 1 Processor -- start 2024-04-25T11:23:03.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.422+0000 7f9408a00640 1 -- start start 2024-04-25T11:23:03.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.425+0000 7f9408a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9404164710 0x7f94040b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.425+0000 7f9408a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94040b0da0 0x7f94041680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.425+0000 7f9408a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94040af070 con 0x7f94040b4aa0 2024-04-25T11:23:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.425+0000 7f9408a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f94041655b0 con 0x7f9404164710 2024-04-25T11:23:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.425+0000 7f9408a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9404165a40 con 0x7f94040b0da0 2024-04-25T11:23:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.425+0000 7f9401400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9404164710 0x7f94040b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.425+0000 7f9401400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9404164710 0x7f94040b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47878/0 (socket says 172.21.15.7:47878) 2024-04-25T11:23:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.425+0000 7f9401400640 1 -- 172.21.15.7:0/3163266354 learned_addr learned my addr 172.21.15.7:0/3163266354 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:03.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.425+0000 7f9402800640 1 --2- 172.21.15.7:0/3163266354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94040b0da0 0x7f94041680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.426+0000 7f9401400640 1 -- 172.21.15.7:0/3163266354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94040b0da0 msgr2=0x7f94041680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.426+0000 7f9401400640 1 --2- 172.21.15.7:0/3163266354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94040b0da0 0x7f94041680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.426+0000 7f9401400640 1 -- 172.21.15.7:0/3163266354 >> v1:172.21.15.87:6789/0 conn(0x7f94040b4aa0 legacy=0x7f94040ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.426+0000 7f9401400640 1 -- 172.21.15.7:0/3163266354 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9404164c80 con 0x7f9404164710 2024-04-25T11:23:03.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.427+0000 7f9401400640 1 --2- 172.21.15.7:0/3163266354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9404164710 0x7f94040b0860 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009870 tx=0x7f93ec02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c9f17caef43217 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.427+0000 7f9400a00640 1 -- 172.21.15.7:0/3163266354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec037070 con 0x7f9404164710 2024-04-25T11:23:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.427+0000 7f9400a00640 1 -- 172.21.15.7:0/3163266354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ec032440 con 0x7f9404164710 2024-04-25T11:23:03.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.428+0000 7f9400a00640 1 -- 172.21.15.7:0/3163266354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec032810 con 0x7f9404164710 2024-04-25T11:23:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.428+0000 7f9408a00640 1 -- 172.21.15.7:0/3163266354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9404164710 msgr2=0x7f94040b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.428+0000 7f9408a00640 1 --2- 172.21.15.7:0/3163266354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9404164710 0x7f94040b0860 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009870 tx=0x7f93ec02a250 comp rx=0 tx=0).stop 2024-04-25T11:23:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.428+0000 7f9408a00640 1 -- 172.21.15.7:0/3163266354 shutdown_connections 2024-04-25T11:23:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.428+0000 7f9408a00640 1 --2- 172.21.15.7:0/3163266354 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94040b0da0 0x7f94041680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.428+0000 7f9408a00640 1 --2- 172.21.15.7:0/3163266354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9404164710 0x7f94040b0860 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.428+0000 7f9408a00640 1 -- 172.21.15.7:0/3163266354 >> 172.21.15.7:0/3163266354 conn(0x7f9404074200 msgr2=0x7f940406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.428+0000 7f9408a00640 1 -- 172.21.15.7:0/3163266354 shutdown_connections 2024-04-25T11:23:03.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.428+0000 7f9408a00640 1 -- 172.21.15.7:0/3163266354 wait complete. 2024-04-25T11:23:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.431+0000 7f9408a00640 1 Processor -- start 2024-04-25T11:23:03.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.431+0000 7f9408a00640 1 -- start start 2024-04-25T11:23:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.432+0000 7f9408a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94040b0da0 0x7f9404156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.432+0000 7f9408a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94040b4aa0 0x7f94041fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.432+0000 7f9408a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9404164710 0x7f940415e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.432+0000 7f9408a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94040af340 con 0x7f94040b4aa0 2024-04-25T11:23:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.432+0000 7f9408a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9404165240 con 0x7f94040b0da0 2024-04-25T11:23:03.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.432+0000 7f9408a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f940406f300 con 0x7f9404164710 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.433+0000 7f9401400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94040b4aa0 0x7f94041fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.433+0000 7f9401400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94040b4aa0 0x7f94041fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47888/0 (socket says 172.21.15.7:47888) 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.433+0000 7f9401400640 1 -- 172.21.15.7:0/2821022092 learned_addr learned my addr 172.21.15.7:0/2821022092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.433+0000 7f9402800640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9404164710 0x7f940415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.433+0000 7f9401e00640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94040b0da0 0x7f9404156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.433+0000 7f9401400640 1 -- 172.21.15.7:0/2821022092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9404164710 msgr2=0x7f940415e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.433+0000 7f9401400640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9404164710 0x7f940415e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.433+0000 7f9401400640 1 -- 172.21.15.7:0/2821022092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94040b0da0 msgr2=0x7f9404156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.433+0000 7f9401400640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94040b0da0 0x7f9404156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.433+0000 7f9401400640 1 -- 172.21.15.7:0/2821022092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94041fec00 con 0x7f94040b4aa0 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.433+0000 7f9401e00640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94040b0da0 0x7f9404156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.434+0000 7f9401400640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94040b4aa0 0x7f94041fe140 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f93ec000c00 tx=0x7f93ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.434+0000 7f93fa000640 1 -- 172.21.15.7:0/2821022092 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec041020 con 0x7f94040b4aa0 2024-04-25T11:23:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.434+0000 7f93fa000640 1 -- 172.21.15.7:0/2821022092 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93ec0306c0 con 0x7f94040b4aa0 2024-04-25T11:23:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.434+0000 7f93fa000640 1 -- 172.21.15.7:0/2821022092 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec037040 con 0x7f94040b4aa0 2024-04-25T11:23:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.434+0000 7f9408a00640 1 -- 172.21.15.7:0/2821022092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94041feee0 con 0x7f94040b4aa0 2024-04-25T11:23:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.434+0000 7f9408a00640 1 -- 172.21.15.7:0/2821022092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94041fe680 con 0x7f94040b4aa0 2024-04-25T11:23:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.435+0000 7f9408a00640 1 -- 172.21.15.7:0/2821022092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94041fe970 con 0x7f94040b4aa0 2024-04-25T11:23:03.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.437+0000 7f93fa000640 1 -- 172.21.15.7:0/2821022092 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f93ec044020 con 0x7f94040b4aa0 2024-04-25T11:23:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.438+0000 7f93fa000640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93e00744b0 0x7f93e00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.438+0000 7f93fa000640 1 -- 172.21.15.7:0/2821022092 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f93ec0b5870 con 0x7f94040b4aa0 2024-04-25T11:23:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.438+0000 7f9401e00640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93e00744b0 0x7f93e00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.439+0000 7f9401e00640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93e00744b0 0x7f93e00769c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f93f40012e0 tx=0x7f93f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:03.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.443+0000 7f93fa000640 1 -- 172.21.15.7:0/2821022092 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f93ec04b030 con 0x7f94040b4aa0 2024-04-25T11:23:03.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.651+0000 7f9408a00640 1 -- 172.21.15.7:0/2821022092 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f940406a100 con 0x7f93e00744b0 2024-04-25T11:23:03.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.652+0000 7f93fa000640 1 -- 172.21.15.7:0/2821022092 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13494 (secure 0 0 0) 0x7f940406a100 con 0x7f93e00744b0 2024-04-25T11:23:03.647 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:23:03.647 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:23:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.655+0000 7f9408a00640 1 -- 172.21.15.7:0/2821022092 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93e00744b0 msgr2=0x7f93e00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.655+0000 7f9408a00640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93e00744b0 0x7f93e00769c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f93f40012e0 tx=0x7f93f4009040 comp rx=0 tx=0).stop 2024-04-25T11:23:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.655+0000 7f9408a00640 1 -- 172.21.15.7:0/2821022092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94040b4aa0 msgr2=0x7f94041fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.655+0000 7f9408a00640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94040b4aa0 0x7f94041fe140 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f93ec000c00 tx=0x7f93ec0046c0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.655+0000 7f9408a00640 1 -- 172.21.15.7:0/2821022092 shutdown_connections 2024-04-25T11:23:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.655+0000 7f9408a00640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f93e00744b0 0x7f93e00769c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.655+0000 7f9408a00640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9404164710 0x7f940415e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.655+0000 7f9408a00640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94040b4aa0 0x7f94041fe140 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.655+0000 7f9408a00640 1 --2- 172.21.15.7:0/2821022092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94040b0da0 0x7f9404156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.655+0000 7f9408a00640 1 -- 172.21.15.7:0/2821022092 >> 172.21.15.7:0/2821022092 conn(0x7f9404074200 msgr2=0x7f94041625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.655+0000 7f9408a00640 1 -- 172.21.15.7:0/2821022092 shutdown_connections 2024-04-25T11:23:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.656+0000 7f9408a00640 1 -- 172.21.15.7:0/2821022092 wait complete. 2024-04-25T11:23:03.678 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":539,"stamp":"2024-04-25T11:23:02.083511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":286900,"kb_used_data":16648,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687820,"statfs":{"total":773094113280,"available":772800327680,"internally_reserved":0,"allocated":17047552,"data_stored":8536300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4195,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.006061"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":388,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T11:22:59.253029+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:22:59.253029+0000","last_peered":"2024-04-25T11:22:59.253029+0000","last_clean":"2024-04-25T11:22:59.253029+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:22:59.253029+0000","last_undegraded":"2024-04-25T11:22:59.253029+0000","last_fullsized":"2024-04-25T11:22:59.253029+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33708,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338132,"statfs":{"total":96636764160,"available":96602247168,"internally_reserved":0,"allocated":2588672,"data_stored":1514336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476892,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33116,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1978368,"data_stored":918011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476893,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37272,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1978368,"data_stored":918011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509596,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509597,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37272,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1978368,"data_stored":918011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37852,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333988,"statfs":{"total":96636764160,"available":96598003712,"internally_reserved":0,"allocated":2572288,"data_stored":1508379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542301,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1961984,"data_stored":911616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:23:03.678 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T11:23:03.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:23:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.967+0000 7f405fe00640 1 Processor -- start 2024-04-25T11:23:03.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.967+0000 7f405fe00640 1 -- start start 2024-04-25T11:23:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.969+0000 7f405fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40600b5220 0x7f406015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.969+0000 7f405fe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40600af420 0x7f40600af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.969+0000 7f405fe00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40600cebb0 con 0x7f40600aed40 2024-04-25T11:23:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.969+0000 7f405fe00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f406015e1f0 con 0x7f40600af420 2024-04-25T11:23:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.970+0000 7f405fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f40600afd70 con 0x7f40600b5220 2024-04-25T11:23:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.970+0000 7f405f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40600af420 0x7f40600af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.970+0000 7f405f400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40600af420 0x7f40600af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47900/0 (socket says 172.21.15.7:47900) 2024-04-25T11:23:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.970+0000 7f405f400640 1 -- 172.21.15.7:0/428516398 learned_addr learned my addr 172.21.15.7:0/428516398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.970+0000 7f405e000640 1 --2- 172.21.15.7:0/428516398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40600b5220 0x7f406015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.970+0000 7f405f400640 1 -- 172.21.15.7:0/428516398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40600b5220 msgr2=0x7f406015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.970+0000 7f405f400640 1 --2- 172.21.15.7:0/428516398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40600b5220 0x7f406015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.970+0000 7f405f400640 1 -- 172.21.15.7:0/428516398 >> v1:172.21.15.87:6789/0 conn(0x7f40600aed40 legacy=0x7f40600ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.970+0000 7f405f400640 1 -- 172.21.15.7:0/428516398 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40600b64b0 con 0x7f40600af420 2024-04-25T11:23:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.970+0000 7f405e000640 1 --2- 172.21.15.7:0/428516398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40600b5220 0x7f406015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.972+0000 7f405f400640 1 --2- 172.21.15.7:0/428516398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40600af420 0x7f40600af830 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f4058009e50 tx=0x7f405802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bac801a856b57ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:03.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.972+0000 7f405d600640 1 -- 172.21.15.7:0/428516398 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058036070 con 0x7f40600af420 2024-04-25T11:23:03.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.972+0000 7f405d600640 1 -- 172.21.15.7:0/428516398 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4058031600 con 0x7f40600af420 2024-04-25T11:23:03.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.972+0000 7f405d600640 1 -- 172.21.15.7:0/428516398 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580319d0 con 0x7f40600af420 2024-04-25T11:23:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.973+0000 7f405fe00640 1 -- 172.21.15.7:0/428516398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40600af420 msgr2=0x7f40600af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.973+0000 7f405fe00640 1 --2- 172.21.15.7:0/428516398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40600af420 0x7f40600af830 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f4058009e50 tx=0x7f405802a6c0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.973+0000 7f405fe00640 1 -- 172.21.15.7:0/428516398 shutdown_connections 2024-04-25T11:23:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.973+0000 7f405fe00640 1 --2- 172.21.15.7:0/428516398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40600af420 0x7f40600af830 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.973+0000 7f405fe00640 1 --2- 172.21.15.7:0/428516398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40600b5220 0x7f406015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.973+0000 7f405fe00640 1 -- 172.21.15.7:0/428516398 >> 172.21.15.7:0/428516398 conn(0x7f40600b3820 msgr2=0x7f40600b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.973+0000 7f405fe00640 1 -- 172.21.15.7:0/428516398 shutdown_connections 2024-04-25T11:23:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.973+0000 7f405fe00640 1 -- 172.21.15.7:0/428516398 wait complete. 2024-04-25T11:23:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.976+0000 7f405fe00640 1 Processor -- start 2024-04-25T11:23:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.976+0000 7f405fe00640 1 -- start start 2024-04-25T11:23:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.978+0000 7f405fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40600aed40 0x7f40601e6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.978+0000 7f405fe00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40600af420 0x7f40601f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.978+0000 7f405fe00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40600b5220 0x7f40601f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.978+0000 7f405fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f406006cbd0 con 0x7f40600aed40 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.978+0000 7f405fe00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f406006c8f0 con 0x7f40600b5220 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.978+0000 7f405fe00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f406006ceb0 con 0x7f40600af420 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.978+0000 7f405ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40600aed40 0x7f40601e6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.978+0000 7f405ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40600aed40 0x7f40601e6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47902/0 (socket says 172.21.15.7:47902) 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.978+0000 7f405ea00640 1 -- 172.21.15.7:0/426529954 learned_addr learned my addr 172.21.15.7:0/426529954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.979+0000 7f405e000640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40600af420 0x7f40601f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.979+0000 7f405f400640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40600b5220 0x7f40601f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.979+0000 7f405ea00640 1 -- 172.21.15.7:0/426529954 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40600af420 msgr2=0x7f40601f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.979+0000 7f405ea00640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40600af420 0x7f40601f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.979+0000 7f405ea00640 1 -- 172.21.15.7:0/426529954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40600b5220 msgr2=0x7f40601f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.979+0000 7f405ea00640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40600b5220 0x7f40601f9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.979+0000 7f405ea00640 1 -- 172.21.15.7:0/426529954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40601f4140 con 0x7f40600aed40 2024-04-25T11:23:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.979+0000 7f405ea00640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40600aed40 0x7f40601e6de0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f405000b3c0 tx=0x7f405000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.979+0000 7f4056a00640 1 -- 172.21.15.7:0/426529954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050011830 con 0x7f40600aed40 2024-04-25T11:23:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.980+0000 7f4056a00640 1 -- 172.21.15.7:0/426529954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4050016070 con 0x7f40600aed40 2024-04-25T11:23:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.980+0000 7f405fe00640 1 -- 172.21.15.7:0/426529954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406015f3a0 con 0x7f40600aed40 2024-04-25T11:23:03.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.980+0000 7f4056a00640 1 -- 172.21.15.7:0/426529954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4050011c50 con 0x7f40600aed40 2024-04-25T11:23:03.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.980+0000 7f405fe00640 1 -- 172.21.15.7:0/426529954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4060160760 con 0x7f40600aed40 2024-04-25T11:23:03.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.981+0000 7f405fe00640 1 -- 172.21.15.7:0/426529954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4060050740 con 0x7f40600aed40 2024-04-25T11:23:03.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.982+0000 7f4056a00640 1 -- 172.21.15.7:0/426529954 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f405001b460 con 0x7f40600aed40 2024-04-25T11:23:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.984+0000 7f4056a00640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40400744e0 0x7f40400769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.984+0000 7f4056a00640 1 -- 172.21.15.7:0/426529954 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f4050094be0 con 0x7f40600aed40 2024-04-25T11:23:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.984+0000 7f405e000640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40400744e0 0x7f40400769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.985+0000 7f405e000640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40400744e0 0x7f40400769f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f404c00b840 tx=0x7f404c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:03.990+0000 7f4056a00640 1 -- 172.21.15.7:0/426529954 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4050020020 con 0x7f40600aed40 2024-04-25T11:23:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.190+0000 7f405fe00640 1 -- 172.21.15.7:0/426529954 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4060153900 con 0x7f40400744e0 2024-04-25T11:23:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.191+0000 7f4056a00640 1 -- 172.21.15.7:0/426529954 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13493 (secure 0 0 0) 0x7f4060153900 con 0x7f40400744e0 2024-04-25T11:23:04.186 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:23:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:23:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 -- 172.21.15.7:0/426529954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40400744e0 msgr2=0x7f40400769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40400744e0 0x7f40400769f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f404c00b840 tx=0x7f404c008040 comp rx=0 tx=0).stop 2024-04-25T11:23:04.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 -- 172.21.15.7:0/426529954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40600aed40 msgr2=0x7f40601e6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40600aed40 0x7f40601e6de0 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f405000b3c0 tx=0x7f405000b880 comp rx=0 tx=0).stop 2024-04-25T11:23:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 -- 172.21.15.7:0/426529954 shutdown_connections 2024-04-25T11:23:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f40400744e0 0x7f40400769f0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40600b5220 0x7f40601f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40600af420 0x7f40601f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 --2- 172.21.15.7:0/426529954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40600aed40 0x7f40601e6de0 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 -- 172.21.15.7:0/426529954 >> 172.21.15.7:0/426529954 conn(0x7f40600b3820 msgr2=0x7f40600b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 -- 172.21.15.7:0/426529954 shutdown_connections 2024-04-25T11:23:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.194+0000 7f405fe00640 1 -- 172.21.15.7:0/426529954 wait complete. 2024-04-25T11:23:04.215 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":540,"stamp":"2024-04-25T11:23:04.083956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":286936,"kb_used_data":16680,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687784,"statfs":{"total":773094113280,"available":772800290816,"internally_reserved":0,"allocated":17080320,"data_stored":8548652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4195,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002537"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":388,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T11:22:59.253029+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:22:59.253029+0000","last_peered":"2024-04-25T11:22:59.253029+0000","last_clean":"2024-04-25T11:22:59.253029+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:22:59.253029+0000","last_undegraded":"2024-04-25T11:22:59.253029+0000","last_fullsized":"2024-04-25T11:22:59.253029+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33708,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338132,"statfs":{"total":96636764160,"available":96602247168,"internally_reserved":0,"allocated":2588672,"data_stored":1514336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476892,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33116,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1978368,"data_stored":918011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476893,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37272,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1978368,"data_stored":918011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509596,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509597,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37272,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334568,"statfs":{"total":96636764160,"available":96598597632,"internally_reserved":0,"allocated":1978368,"data_stored":918011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37852,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333988,"statfs":{"total":96636764160,"available":96598003712,"internally_reserved":0,"allocated":2572288,"data_stored":1508379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542302,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33132,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338708,"statfs":{"total":96636764160,"available":96602836992,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:23:04.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T11:23:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.505+0000 7fcf6ae00640 1 Processor -- start 2024-04-25T11:23:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.505+0000 7fcf6ae00640 1 -- start start 2024-04-25T11:23:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.508+0000 7fcf6ae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf640af540 0x7fcf6409f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.508+0000 7fcf6ae00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcf6415bc40 0x7fcf640cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.508+0000 7fcf6ae00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf6415f4e0 con 0x7fcf6415ef90 2024-04-25T11:23:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.508+0000 7fcf6ae00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fcf64071190 con 0x7fcf6415bc40 2024-04-25T11:23:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.508+0000 7fcf6ae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf640b0560 con 0x7fcf640af540 2024-04-25T11:23:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.508+0000 7fcf63e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcf6415ef90 0x7fcf640aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34512/0 (socket says 172.21.15.7:34512) 2024-04-25T11:23:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.508+0000 7fcf63e00640 1 -- 172.21.15.7:0/3912170524 learned_addr learned my addr 172.21.15.7:0/3912170524 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.508+0000 7fcf68e00640 1 --2- 172.21.15.7:0/3912170524 >> v2:172.21.15.7:3300/0 conn(0x7fcf6415bc40 0x7fcf640cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:04.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.509+0000 7fcf63400640 1 --2- 172.21.15.7:0/3912170524 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf640af540 0x7fcf6409f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:04.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.509+0000 7fcf62a00640 1 -- 172.21.15.7:0/3912170524 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3097683940 0 0) 0x7fcf6415f4e0 con 0x7fcf6415ef90 2024-04-25T11:23:04.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.509+0000 7fcf62a00640 1 -- 172.21.15.7:0/3912170524 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf48003520 con 0x7fcf6415ef90 2024-04-25T11:23:04.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.509+0000 7fcf68e00640 1 -- 172.21.15.7:0/3912170524 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf640af540 msgr2=0x7fcf6409f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:04.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.509+0000 7fcf68e00640 1 --2- 172.21.15.7:0/3912170524 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf640af540 0x7fcf6409f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.509+0000 7fcf68e00640 1 -- 172.21.15.7:0/3912170524 >> v1:172.21.15.7:6789/0 conn(0x7fcf6415ef90 legacy=0x7fcf640aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.509+0000 7fcf62a00640 1 -- 172.21.15.7:0/3912170524 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2015281873 0 0) 0x7fcf48003520 con 0x7fcf6415ef90 2024-04-25T11:23:04.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.509+0000 7fcf68e00640 1 -- 172.21.15.7:0/3912170524 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf640b0840 con 0x7fcf6415bc40 2024-04-25T11:23:04.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.511+0000 7fcf68e00640 1 --2- 172.21.15.7:0/3912170524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf6415bc40 0x7fcf640cf980 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009d90 tx=0x7fcf5c02a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ee97fddac4a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:04.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.511+0000 7fcf62a00640 1 -- 172.21.15.7:0/3912170524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54001670 con 0x7fcf6415bc40 2024-04-25T11:23:04.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.511+0000 7fcf62a00640 1 -- 172.21.15.7:0/3912170524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5c007540 con 0x7fcf6415bc40 2024-04-25T11:23:04.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.511+0000 7fcf62a00640 1 -- 172.21.15.7:0/3912170524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c007910 con 0x7fcf6415bc40 2024-04-25T11:23:04.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.512+0000 7fcf6ae00640 1 -- 172.21.15.7:0/3912170524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf6415bc40 msgr2=0x7fcf640cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.512+0000 7fcf6ae00640 1 --2- 172.21.15.7:0/3912170524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf6415bc40 0x7fcf640cf980 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009d90 tx=0x7fcf5c02a550 comp rx=0 tx=0).stop 2024-04-25T11:23:04.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.512+0000 7fcf6ae00640 1 -- 172.21.15.7:0/3912170524 shutdown_connections 2024-04-25T11:23:04.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.512+0000 7fcf6ae00640 1 --2- 172.21.15.7:0/3912170524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf6415bc40 0x7fcf640cf980 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.512+0000 7fcf6ae00640 1 --2- 172.21.15.7:0/3912170524 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf640af540 0x7fcf6409f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.512+0000 7fcf6ae00640 1 -- 172.21.15.7:0/3912170524 >> 172.21.15.7:0/3912170524 conn(0x7fcf64074200 msgr2=0x7fcf6406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:04.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.512+0000 7fcf6ae00640 1 -- 172.21.15.7:0/3912170524 shutdown_connections 2024-04-25T11:23:04.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.512+0000 7fcf6ae00640 1 -- 172.21.15.7:0/3912170524 wait complete. 2024-04-25T11:23:04.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.514+0000 7fcf6ae00640 1 Processor -- start 2024-04-25T11:23:04.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.515+0000 7fcf6ae00640 1 -- start start 2024-04-25T11:23:04.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.516+0000 7fcf6ae00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf6415bc40 0x7fcf641611c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:04.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.516+0000 7fcf6ae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf6415ef90 0x7fcf641520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:04.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.516+0000 7fcf6ae00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf640cad00 0x7fcf6415d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:04.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.516+0000 7fcf6ae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcf640b1320 con 0x7fcf6415ef90 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.516+0000 7fcf6ae00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf640b0e00 con 0x7fcf640cad00 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.516+0000 7fcf6ae00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf640afb80 con 0x7fcf6415bc40 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.516+0000 7fcf63400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf6415ef90 0x7fcf641520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.517+0000 7fcf63400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf6415ef90 0x7fcf641520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47924/0 (socket says 172.21.15.7:47924) 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.517+0000 7fcf63400640 1 -- 172.21.15.7:0/2152027746 learned_addr learned my addr 172.21.15.7:0/2152027746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.517+0000 7fcf63e00640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf6415bc40 0x7fcf641611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.517+0000 7fcf68e00640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf640cad00 0x7fcf6415d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.517+0000 7fcf63400640 1 -- 172.21.15.7:0/2152027746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf6415bc40 msgr2=0x7fcf641611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.517+0000 7fcf63400640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf6415bc40 0x7fcf641611c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.517+0000 7fcf63400640 1 -- 172.21.15.7:0/2152027746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf640cad00 msgr2=0x7fcf6415d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.517+0000 7fcf63400640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf640cad00 0x7fcf6415d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.517+0000 7fcf63400640 1 -- 172.21.15.7:0/2152027746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf640cb470 con 0x7fcf6415ef90 2024-04-25T11:23:04.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.517+0000 7fcf63400640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf6415ef90 0x7fcf641520f0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000d5f0 tx=0x7fcf5000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:04.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.518+0000 7fcf5be00640 1 -- 172.21.15.7:0/2152027746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500125f0 con 0x7fcf6415ef90 2024-04-25T11:23:04.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.518+0000 7fcf5be00640 1 -- 172.21.15.7:0/2152027746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf50019070 con 0x7fcf6415ef90 2024-04-25T11:23:04.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.518+0000 7fcf5be00640 1 -- 172.21.15.7:0/2152027746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500129d0 con 0x7fcf6415ef90 2024-04-25T11:23:04.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.518+0000 7fcf6ae00640 1 -- 172.21.15.7:0/2152027746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf640cb750 con 0x7fcf6415ef90 2024-04-25T11:23:04.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.518+0000 7fcf6ae00640 1 -- 172.21.15.7:0/2152027746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf640cbd10 con 0x7fcf6415ef90 2024-04-25T11:23:04.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.519+0000 7fcf6ae00640 1 -- 172.21.15.7:0/2152027746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf6406a100 con 0x7fcf6415ef90 2024-04-25T11:23:04.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.520+0000 7fcf5be00640 1 -- 172.21.15.7:0/2152027746 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fcf50012b70 con 0x7fcf6415ef90 2024-04-25T11:23:04.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.521+0000 7fcf5be00640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf440744b0 0x7fcf440769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:04.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.522+0000 7fcf5be00640 1 -- 172.21.15.7:0/2152027746 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7fcf500955a0 con 0x7fcf6415ef90 2024-04-25T11:23:04.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.522+0000 7fcf63e00640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf440744b0 0x7fcf440769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:04.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.523+0000 7fcf63e00640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf440744b0 0x7fcf440769c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcf54001eb0 tx=0x7fcf54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.526+0000 7fcf5be00640 1 -- 172.21.15.7:0/2152027746 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fcf5005fe00 con 0x7fcf6415ef90 2024-04-25T11:23:04.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.734+0000 7fcf6ae00640 1 -- 172.21.15.7:0/2152027746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf640cb200 con 0x7fcf6415ef90 2024-04-25T11:23:04.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.736+0000 7fcf5be00640 1 -- 172.21.15.7:0/2152027746 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) ==== 74+0+13203 (secure 0 0 0) 0x7fcf5005f700 con 0x7fcf6415ef90 2024-04-25T11:23:04.731 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:23:04.731 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":159,"fsid":"4ee91e9b-5e2d-4052-ab33-0a56271c8c7f","created":"2024-04-25T11:09:56.494058+0000","modified":"2024-04-25T11:22:59.236840+0000","last_up_change":"2024-04-25T11:10:10.320639+0000","last_in_change":"2024-04-25T11:10:00.454539+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T11:10:09.817225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"da1b353f-d9c1-458d-a27f-e53c50446727","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6801","nonce":3278797987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6803","nonce":3278797987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6807","nonce":3278797987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":3278797987},{"type":"v1","addr":"172.21.15.7:6805","nonce":3278797987}]},"public_addr":"172.21.15.7:6801/3278797987","cluster_addr":"172.21.15.7:6803/3278797987","heartbeat_back_addr":"172.21.15.7:6807/3278797987","heartbeat_front_addr":"172.21.15.7:6805/3278797987","state":["exists","up"]},{"osd":1,"uuid":"0eca90e3-34f2-4488-96b0-249fe063693a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6824","nonce":1767080083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6827","nonce":1767080083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6831","nonce":1767080083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1767080083},{"type":"v1","addr":"172.21.15.7:6829","nonce":1767080083}]},"public_addr":"172.21.15.7:6824/1767080083","cluster_addr":"172.21.15.7:6827/1767080083","heartbeat_back_addr":"172.21.15.7:6831/1767080083","heartbeat_front_addr":"172.21.15.7:6829/1767080083","state":["exists","up"]},{"osd":2,"uuid":"1d3b267f-385c-4080-a237-0caf41de8a05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6817","nonce":3878005205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6819","nonce":3878005205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6823","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6825","nonce":3878005205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3878005205},{"type":"v1","addr":"172.21.15.7:6821","nonce":3878005205}]},"public_addr":"172.21.15.7:6817/3878005205","cluster_addr":"172.21.15.7:6819/3878005205","heartbeat_back_addr":"172.21.15.7:6825/3878005205","heartbeat_front_addr":"172.21.15.7:6821/3878005205","state":["exists","up"]},{"osd":3,"uuid":"fd328260-3895-4610-9bfb-171d15b180fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6809","nonce":1382334542}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6811","nonce":1382334542}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6815","nonce":1382334542}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1382334542},{"type":"v1","addr":"172.21.15.7:6813","nonce":1382334542}]},"public_addr":"172.21.15.7:6809/1382334542","cluster_addr":"172.21.15.7:6811/1382334542","heartbeat_back_addr":"172.21.15.7:6815/1382334542","heartbeat_front_addr":"172.21.15.7:6813/1382334542","state":["exists","up"]},{"osd":4,"uuid":"99bfd73d-2071-467e-b193-41909aa6b352","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6809","nonce":1933779372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6811","nonce":1933779372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6815","nonce":1933779372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1933779372},{"type":"v1","addr":"172.21.15.87:6813","nonce":1933779372}]},"public_addr":"172.21.15.87:6809/1933779372","cluster_addr":"172.21.15.87:6811/1933779372","heartbeat_back_addr":"172.21.15.87:6815/1933779372","heartbeat_front_addr":"172.21.15.87:6813/1933779372","state":["exists","up"]},{"osd":5,"uuid":"9e92fd38-1801-468a-9bd6-e0bb0e885c84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6817","nonce":2772334579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6819","nonce":2772334579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6823","nonce":2772334579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2772334579},{"type":"v1","addr":"172.21.15.87:6821","nonce":2772334579}]},"public_addr":"172.21.15.87:6817/2772334579","cluster_addr":"172.21.15.87:6819/2772334579","heartbeat_back_addr":"172.21.15.87:6823/2772334579","heartbeat_front_addr":"172.21.15.87:6821/2772334579","state":["exists","up"]},{"osd":6,"uuid":"9b0485e2-f51d-462b-81ad-e00f2549b9df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6825","nonce":960154076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6827","nonce":960154076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6831","nonce":960154076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":960154076},{"type":"v1","addr":"172.21.15.87:6829","nonce":960154076}]},"public_addr":"172.21.15.87:6825/960154076","cluster_addr":"172.21.15.87:6827/960154076","heartbeat_back_addr":"172.21.15.87:6831/960154076","heartbeat_front_addr":"172.21.15.87:6829/960154076","state":["exists","up"]},{"osd":7,"uuid":"2a8ffed7-232e-4325-8580-5c86d07ecf97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6801","nonce":3856235626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6803","nonce":3856235626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6807","nonce":3856235626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3856235626},{"type":"v1","addr":"172.21.15.87:6805","nonce":3856235626}]},"public_addr":"172.21.15.87:6801/3856235626","cluster_addr":"172.21.15.87:6803/3856235626","heartbeat_back_addr":"172.21.15.87:6807/3856235626","heartbeat_front_addr":"172.21.15.87:6805/3856235626","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.200561+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.978282+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.749325+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.004639+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.985215+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:05.965503+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.181385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T11:10:06.157278+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6837/339418645":"2024-04-26T11:22:56.071953+0000","172.21.15.7:6836/339418645":"2024-04-26T11:22:56.071953+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T11:23:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.739+0000 7fcf6ae00640 1 -- 172.21.15.7:0/2152027746 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf440744b0 msgr2=0x7fcf440769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.739+0000 7fcf6ae00640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf440744b0 0x7fcf440769c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcf54001eb0 tx=0x7fcf54009040 comp rx=0 tx=0).stop 2024-04-25T11:23:04.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.739+0000 7fcf6ae00640 1 -- 172.21.15.7:0/2152027746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf6415ef90 msgr2=0x7fcf641520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:04.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.739+0000 7fcf6ae00640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf6415ef90 0x7fcf641520f0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000d5f0 tx=0x7fcf5000dab0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.739+0000 7fcf6ae00640 1 -- 172.21.15.7:0/2152027746 shutdown_connections 2024-04-25T11:23:04.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.739+0000 7fcf6ae00640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fcf440744b0 0x7fcf440769c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.739+0000 7fcf6ae00640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf640cad00 0x7fcf6415d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.739+0000 7fcf6ae00640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf6415ef90 0x7fcf641520f0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.739+0000 7fcf6ae00640 1 --2- 172.21.15.7:0/2152027746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf6415bc40 0x7fcf641611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:04.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.739+0000 7fcf6ae00640 1 -- 172.21.15.7:0/2152027746 >> 172.21.15.7:0/2152027746 conn(0x7fcf64074200 msgr2=0x7fcf6415e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:04.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.740+0000 7fcf6ae00640 1 -- 172.21.15.7:0/2152027746 shutdown_connections 2024-04-25T11:23:04.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:04.740+0000 7fcf6ae00640 1 -- 172.21.15.7:0/2152027746 wait complete. 2024-04-25T11:23:05.761 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T11:23:05.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:23:06.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.070+0000 7f0b2be00640 1 Processor -- start 2024-04-25T11:23:06.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.071+0000 7f0b2be00640 1 -- start start 2024-04-25T11:23:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.073+0000 7f0b2be00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b2c074090 0x7f0b2c15d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.073+0000 7f0b2be00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b2c15d7d0 con 0x7f0b2c073c80 2024-04-25T11:23:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.073+0000 7f0b2be00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b2c15ff00 con 0x7f0b2c06dc90 2024-04-25T11:23:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.073+0000 7f0b2be00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0b2c1601e0 con 0x7f0b2c074090 2024-04-25T11:23:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.073+0000 7f0b2aa00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0b2c073c80 0x7f0b2c06d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34526/0 (socket says 172.21.15.7:34526) 2024-04-25T11:23:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.073+0000 7f0b2aa00640 1 -- 172.21.15.7:0/1753150212 learned_addr learned my addr 172.21.15.7:0/1753150212 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:06.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.073+0000 7f0b2b400640 1 --2- 172.21.15.7:0/1753150212 >> v2:172.21.15.7:3300/0 conn(0x7f0b2c074090 0x7f0b2c15d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.074+0000 7f0b2a000640 1 -- 172.21.15.7:0/1753150212 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2947993156 0 0) 0x7f0b2c15d7d0 con 0x7f0b2c073c80 2024-04-25T11:23:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.074+0000 7f0b2a000640 1 -- 172.21.15.7:0/1753150212 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b10003520 con 0x7f0b2c073c80 2024-04-25T11:23:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.074+0000 7f0b2b400640 1 -- 172.21.15.7:0/1753150212 >> v1:172.21.15.87:6789/0 conn(0x7f0b2c06dc90 legacy=0x7f0b2c15ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.074+0000 7f0b2b400640 1 -- 172.21.15.7:0/1753150212 >> v1:172.21.15.7:6789/0 conn(0x7f0b2c073c80 legacy=0x7f0b2c06d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.074+0000 7f0b2b400640 1 -- 172.21.15.7:0/1753150212 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c1604c0 con 0x7f0b2c074090 2024-04-25T11:23:06.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.075+0000 7f0b2b400640 1 --2- 172.21.15.7:0/1753150212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c074090 0x7f0b2c15d170 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009d90 tx=0x7f0b2402c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47c365b1bbafd37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.076+0000 7f0b2a000640 1 -- 172.21.15.7:0/1753150212 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24032770 con 0x7f0b2c074090 2024-04-25T11:23:06.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.076+0000 7f0b2a000640 1 -- 172.21.15.7:0/1753150212 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b24038070 con 0x7f0b2c074090 2024-04-25T11:23:06.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.076+0000 7f0b2a000640 1 -- 172.21.15.7:0/1753150212 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24032b30 con 0x7f0b2c074090 2024-04-25T11:23:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.076+0000 7f0b2be00640 1 -- 172.21.15.7:0/1753150212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c074090 msgr2=0x7f0b2c15d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.076+0000 7f0b2be00640 1 --2- 172.21.15.7:0/1753150212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c074090 0x7f0b2c15d170 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009d90 tx=0x7f0b2402c870 comp rx=0 tx=0).stop 2024-04-25T11:23:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.076+0000 7f0b2be00640 1 -- 172.21.15.7:0/1753150212 shutdown_connections 2024-04-25T11:23:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.076+0000 7f0b2be00640 1 --2- 172.21.15.7:0/1753150212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c074090 0x7f0b2c15d170 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.077+0000 7f0b2be00640 1 -- 172.21.15.7:0/1753150212 >> 172.21.15.7:0/1753150212 conn(0x7f0b2c073160 msgr2=0x7f0b2c073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.077+0000 7f0b2be00640 1 -- 172.21.15.7:0/1753150212 shutdown_connections 2024-04-25T11:23:06.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.077+0000 7f0b2be00640 1 -- 172.21.15.7:0/1753150212 wait complete. 2024-04-25T11:23:06.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.079+0000 7f0b2be00640 1 Processor -- start 2024-04-25T11:23:06.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.079+0000 7f0b2be00640 1 -- start start 2024-04-25T11:23:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.081+0000 7f0b2be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c073c80 0x7f0b2c0af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.081+0000 7f0b2be00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b2c074090 0x7f0b2c15e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.081+0000 7f0b2be00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b2c1f9940 0x7f0b2c1f9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.081+0000 7f0b2be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c160cc0 con 0x7f0b2c073c80 2024-04-25T11:23:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.081+0000 7f0b2be00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c0d11e0 con 0x7f0b2c074090 2024-04-25T11:23:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.081+0000 7f0b2be00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0b2c0b5310 con 0x7f0b2c1f9940 2024-04-25T11:23:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.081+0000 7f0b2aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c073c80 0x7f0b2c0af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.081+0000 7f0b2aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c073c80 0x7f0b2c0af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47940/0 (socket says 172.21.15.7:47940) 2024-04-25T11:23:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.081+0000 7f0b2aa00640 1 -- 172.21.15.7:0/1133301951 learned_addr learned my addr 172.21.15.7:0/1133301951 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.081+0000 7f0b23e00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b2c074090 0x7f0b2c15e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.082+0000 7f0b2aa00640 1 -- 172.21.15.7:0/1133301951 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b2c1f9940 msgr2=0x7f0b2c1f9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.082+0000 7f0b2aa00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b2c1f9940 0x7f0b2c1f9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.082+0000 7f0b2aa00640 1 -- 172.21.15.7:0/1133301951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b2c074090 msgr2=0x7f0b2c15e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.082+0000 7f0b2aa00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b2c074090 0x7f0b2c15e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.082+0000 7f0b2aa00640 1 -- 172.21.15.7:0/1133301951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c1fa290 con 0x7f0b2c073c80 2024-04-25T11:23:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.082+0000 7f0b23e00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b2c074090 0x7f0b2c15e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.082+0000 7f0b2aa00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c073c80 0x7f0b2c0af7c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00c460 tx=0x7f0b1c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.082+0000 7f0b22a00640 1 -- 172.21.15.7:0/1133301951 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c017070 con 0x7f0b2c073c80 2024-04-25T11:23:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.082+0000 7f0b22a00640 1 -- 172.21.15.7:0/1133301951 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b1c005ce0 con 0x7f0b2c073c80 2024-04-25T11:23:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.082+0000 7f0b22a00640 1 -- 172.21.15.7:0/1133301951 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c0072f0 con 0x7f0b2c073c80 2024-04-25T11:23:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.082+0000 7f0b2be00640 1 -- 172.21.15.7:0/1133301951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c0ae720 con 0x7f0b2c073c80 2024-04-25T11:23:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.083+0000 7f0b2be00640 1 -- 172.21.15.7:0/1133301951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c0aece0 con 0x7f0b2c073c80 2024-04-25T11:23:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.084+0000 7f0b2be00640 1 -- 172.21.15.7:0/1133301951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b2c0b66b0 con 0x7f0b2c073c80 2024-04-25T11:23:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.085+0000 7f0b22a00640 1 -- 172.21.15.7:0/1133301951 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0b1c01a070 con 0x7f0b2c073c80 2024-04-25T11:23:06.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.086+0000 7f0b22a00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0b080744b0 0x7f0b080769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.086+0000 7f0b22a00640 1 -- 172.21.15.7:0/1133301951 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f0b1c0944c0 con 0x7f0b2c073c80 2024-04-25T11:23:06.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.086+0000 7f0b22a00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] conn(0x7f0b0807a9a0 0x7f0b0807ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.086+0000 7f0b22a00640 1 -- 172.21.15.7:0/1133301951 --> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b1c01ee50 con 0x7f0b0807a9a0 2024-04-25T11:23:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.086+0000 7f0b22a00640 1 -- 172.21.15.7:0/1133301951 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=159) ==== 24+0+0 (secure 0 0 0) 0x7f0b1c09b050 con 0x7f0b2c073c80 2024-04-25T11:23:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.086+0000 7f0b23e00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0b080744b0 0x7f0b080769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.086+0000 7f0b2b400640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] conn(0x7f0b0807a9a0 0x7f0b0807ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.087+0000 7f0b23e00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0b080744b0 0x7f0b080769c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c15e360 tx=0x7f0b14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.087+0000 7f0b2b400640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] conn(0x7f0b0807a9a0 0x7f0b0807ce30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.088+0000 7f0b22a00640 1 -- 172.21.15.7:0/1133301951 <== osd.0 v2:172.21.15.7:6800/3278797987 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b1c01ee50 con 0x7f0b0807a9a0 2024-04-25T11:23:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.103+0000 7f0b2be00640 1 -- 172.21.15.7:0/1133301951 --> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0b2c050740 con 0x7f0b0807a9a0 2024-04-25T11:23:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.103+0000 7f0b22a00640 1 -- 172.21.15.7:0/1133301951 <== osd.0 v2:172.21.15.7:6800/3278797987 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0b2c050740 con 0x7f0b0807a9a0 2024-04-25T11:23:06.099 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:23:06.099 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:23:06.099 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:23:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.104+0000 7f0b2be00640 1 -- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] conn(0x7f0b0807a9a0 msgr2=0x7f0b0807ce30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.104+0000 7f0b2be00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:6800/3278797987,v1:172.21.15.7:6801/3278797987] conn(0x7f0b0807a9a0 0x7f0b0807ce30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.104+0000 7f0b2be00640 1 -- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0b080744b0 msgr2=0x7f0b080769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.104+0000 7f0b2be00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0b080744b0 0x7f0b080769c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c15e360 tx=0x7f0b14009040 comp rx=0 tx=0).stop 2024-04-25T11:23:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.104+0000 7f0b2be00640 1 -- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c073c80 msgr2=0x7f0b2c0af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.104+0000 7f0b2be00640 1 --2- 172.21.15.7:0/1133301951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c073c80 0x7f0b2c0af7c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00c460 tx=0x7f0b1c00c920 comp rx=0 tx=0).stop 2024-04-25T11:23:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.104+0000 7f0b2b400640 1 -- 172.21.15.7:0/1133301951 reap_dead start 2024-04-25T11:23:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.104+0000 7f0b2be00640 1 -- 172.21.15.7:0/1133301951 shutdown_connections 2024-04-25T11:23:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.104+0000 7f0b2be00640 1 -- 172.21.15.7:0/1133301951 >> 172.21.15.7:0/1133301951 conn(0x7f0b2c073160 msgr2=0x7f0b2c073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.104+0000 7f0b2be00640 1 -- 172.21.15.7:0/1133301951 shutdown_connections 2024-04-25T11:23:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.104+0000 7f0b2be00640 1 -- 172.21.15.7:0/1133301951 wait complete. 2024-04-25T11:23:06.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T11:23:06.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.407+0000 7fc0d1600640 1 Processor -- start 2024-04-25T11:23:06.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.407+0000 7fc0d1600640 1 -- start start 2024-04-25T11:23:06.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.410+0000 7fc0d1600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc0cc15eed0 0x7fc0cc0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.410+0000 7fc0d1600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc0cc1416a0 0x7fc0cc0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.410+0000 7fc0d1600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0cc06c570 con 0x7fc0cc141290 2024-04-25T11:23:06.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.410+0000 7fc0d1600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fc0cc06c850 con 0x7fc0cc15eed0 2024-04-25T11:23:06.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.410+0000 7fc0d1600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc0cc06cb30 con 0x7fc0cc1416a0 2024-04-25T11:23:06.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.410+0000 7fc0cbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc0cc15eed0 0x7fc0cc0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.410+0000 7fc0cbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc0cc15eed0 0x7fc0cc0cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47956/0 (socket says 172.21.15.7:47956) 2024-04-25T11:23:06.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.410+0000 7fc0cbe00640 1 -- 172.21.15.7:0/2534564747 learned_addr learned my addr 172.21.15.7:0/2534564747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:06.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.411+0000 7fc0d0c00640 1 --2- 172.21.15.7:0/2534564747 >> v2:172.21.15.87:3300/0 conn(0x7fc0cc1416a0 0x7fc0cc0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.411+0000 7fc0cbe00640 1 -- 172.21.15.7:0/2534564747 >> v2:172.21.15.87:3300/0 conn(0x7fc0cc1416a0 msgr2=0x7fc0cc0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:06.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.411+0000 7fc0cbe00640 1 --2- 172.21.15.7:0/2534564747 >> v2:172.21.15.87:3300/0 conn(0x7fc0cc1416a0 0x7fc0cc0b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.411+0000 7fc0cbe00640 1 -- 172.21.15.7:0/2534564747 >> v1:172.21.15.87:6789/0 conn(0x7fc0cc141290 legacy=0x7fc0cc06d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:06.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.411+0000 7fc0cbe00640 1 -- 172.21.15.7:0/2534564747 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc06ce10 con 0x7fc0cc15eed0 2024-04-25T11:23:06.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.412+0000 7fc0cbe00640 1 --2- 172.21.15.7:0/2534564747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc15eed0 0x7fc0cc0cd780 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc072e80 tx=0x7fc0bc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd0f0e1ddad619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.412+0000 7fc0cb400640 1 -- 172.21.15.7:0/2534564747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc037070 con 0x7fc0cc15eed0 2024-04-25T11:23:06.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.413+0000 7fc0cb400640 1 -- 172.21.15.7:0/2534564747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0bc032440 con 0x7fc0cc15eed0 2024-04-25T11:23:06.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.413+0000 7fc0cb400640 1 -- 172.21.15.7:0/2534564747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc032810 con 0x7fc0cc15eed0 2024-04-25T11:23:06.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.413+0000 7fc0d1600640 1 -- 172.21.15.7:0/2534564747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc15eed0 msgr2=0x7fc0cc0cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.413+0000 7fc0d1600640 1 --2- 172.21.15.7:0/2534564747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc15eed0 0x7fc0cc0cd780 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc072e80 tx=0x7fc0bc02a230 comp rx=0 tx=0).stop 2024-04-25T11:23:06.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.414+0000 7fc0d1600640 1 -- 172.21.15.7:0/2534564747 shutdown_connections 2024-04-25T11:23:06.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.414+0000 7fc0d1600640 1 --2- 172.21.15.7:0/2534564747 >> v2:172.21.15.87:3300/0 conn(0x7fc0cc1416a0 0x7fc0cc0b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.414+0000 7fc0d1600640 1 --2- 172.21.15.7:0/2534564747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc15eed0 0x7fc0cc0cd780 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.414+0000 7fc0d1600640 1 -- 172.21.15.7:0/2534564747 >> 172.21.15.7:0/2534564747 conn(0x7fc0cc0d1540 msgr2=0x7fc0cc0d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.414+0000 7fc0d1600640 1 -- 172.21.15.7:0/2534564747 shutdown_connections 2024-04-25T11:23:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.414+0000 7fc0d1600640 1 -- 172.21.15.7:0/2534564747 wait complete. 2024-04-25T11:23:06.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.416+0000 7fc0d1600640 1 Processor -- start 2024-04-25T11:23:06.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.416+0000 7fc0d1600640 1 -- start start 2024-04-25T11:23:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.418+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0cc141290 0x7fc0cc0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.418+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc1416a0 0x7fc0cc1e9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.418+0000 7fc0d1600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0cc15eed0 0x7fc0cc15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.418+0000 7fc0d1600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc141ab0 con 0x7fc0cc1416a0 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.418+0000 7fc0d1600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc0cc0cdcc0 con 0x7fc0cc141290 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.418+0000 7fc0d1600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc0cc06dd20 con 0x7fc0cc15eed0 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.418+0000 7fc0c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc1416a0 0x7fc0cc1e9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.418+0000 7fc0c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc1416a0 0x7fc0cc1e9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47970/0 (socket says 172.21.15.7:47970) 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.418+0000 7fc0c3e00640 1 -- 172.21.15.7:0/3376043920 learned_addr learned my addr 172.21.15.7:0/3376043920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.419+0000 7fc0cbe00640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0cc141290 0x7fc0cc0d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.419+0000 7fc0d0c00640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0cc15eed0 0x7fc0cc15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.419+0000 7fc0c3e00640 1 -- 172.21.15.7:0/3376043920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0cc15eed0 msgr2=0x7fc0cc15ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.419+0000 7fc0c3e00640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0cc15eed0 0x7fc0cc15ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.419+0000 7fc0c3e00640 1 -- 172.21.15.7:0/3376043920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0cc141290 msgr2=0x7fc0cc0d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.419+0000 7fc0c3e00640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0cc141290 0x7fc0cc0d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.419+0000 7fc0c3e00640 1 -- 172.21.15.7:0/3376043920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0cc1e7b90 con 0x7fc0cc1416a0 2024-04-25T11:23:06.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.419+0000 7fc0c3e00640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc1416a0 0x7fc0cc1e9150 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400b420 tx=0x7fc0b400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.419+0000 7fc0c8c00640 1 -- 172.21.15.7:0/3376043920 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4011830 con 0x7fc0cc1416a0 2024-04-25T11:23:06.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.420+0000 7fc0c8c00640 1 -- 172.21.15.7:0/3376043920 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b4016070 con 0x7fc0cc1416a0 2024-04-25T11:23:06.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.420+0000 7fc0c8c00640 1 -- 172.21.15.7:0/3376043920 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4011c10 con 0x7fc0cc1416a0 2024-04-25T11:23:06.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.420+0000 7fc0d1600640 1 -- 172.21.15.7:0/3376043920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0cc1e7e70 con 0x7fc0cc1416a0 2024-04-25T11:23:06.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.420+0000 7fc0d1600640 1 -- 172.21.15.7:0/3376043920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0cc1e8430 con 0x7fc0cc1416a0 2024-04-25T11:23:06.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.421+0000 7fc0d1600640 1 -- 172.21.15.7:0/3376043920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0cc1e7900 con 0x7fc0cc1416a0 2024-04-25T11:23:06.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.423+0000 7fc0c8c00640 1 -- 172.21.15.7:0/3376043920 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fc0b401b460 con 0x7fc0cc1416a0 2024-04-25T11:23:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.425+0000 7fc0c8c00640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0a80744e0 0x7fc0a80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.425+0000 7fc0c8c00640 1 -- 172.21.15.7:0/3376043920 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7fc0b4094b20 con 0x7fc0cc1416a0 2024-04-25T11:23:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.425+0000 7fc0cbe00640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0a80744e0 0x7fc0a80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.426+0000 7fc0cbe00640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0a80744e0 0x7fc0a80769f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc034000 tx=0x7fc0bc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.430+0000 7fc0c8c00640 1 -- 172.21.15.7:0/3376043920 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fc0b4020020 con 0x7fc0cc1416a0 2024-04-25T11:23:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.642+0000 7fc0d1600640 1 -- 172.21.15.7:0/3376043920 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fc0cc050740 con 0x7fc0a80744e0 2024-04-25T11:23:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.643+0000 7fc0c8c00640 1 -- 172.21.15.7:0/3376043920 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc0cc050740 con 0x7fc0a80744e0 2024-04-25T11:23:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T11:23:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.645+0000 7fc0d1600640 1 -- 172.21.15.7:0/3376043920 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0a80744e0 msgr2=0x7fc0a80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.645+0000 7fc0d1600640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0a80744e0 0x7fc0a80769f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc034000 tx=0x7fc0bc02e000 comp rx=0 tx=0).stop 2024-04-25T11:23:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.645+0000 7fc0d1600640 1 -- 172.21.15.7:0/3376043920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc1416a0 msgr2=0x7fc0cc1e9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.645+0000 7fc0d1600640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc1416a0 0x7fc0cc1e9150 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400b420 tx=0x7fc0b400b8e0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.646+0000 7fc0d1600640 1 -- 172.21.15.7:0/3376043920 shutdown_connections 2024-04-25T11:23:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.646+0000 7fc0d1600640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fc0a80744e0 0x7fc0a80769f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.646+0000 7fc0d1600640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0cc15eed0 0x7fc0cc15ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.646+0000 7fc0d1600640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0cc1416a0 0x7fc0cc1e9150 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.646+0000 7fc0d1600640 1 --2- 172.21.15.7:0/3376043920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0cc141290 0x7fc0cc0d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.646+0000 7fc0d1600640 1 -- 172.21.15.7:0/3376043920 >> 172.21.15.7:0/3376043920 conn(0x7fc0cc0d1540 msgr2=0x7fc0cc06d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.646+0000 7fc0d1600640 1 -- 172.21.15.7:0/3376043920 shutdown_connections 2024-04-25T11:23:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.646+0000 7fc0d1600640 1 -- 172.21.15.7:0/3376043920 wait complete. 2024-04-25T11:23:06.668 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T11:23:06.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:23:06.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.966+0000 7f0d02800640 1 Processor -- start 2024-04-25T11:23:06.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.966+0000 7f0d02800640 1 -- start start 2024-04-25T11:23:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.968+0000 7f0d02800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0cfc0b4db0 0x7f0cfc1644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.968+0000 7f0d02800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cfc0d2720 0x7f0cfc0d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.968+0000 7f0d02800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cfc0d4140 0x7f0cfc0d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.968+0000 7f0d02800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f0cfc0d3070 con 0x7f0cfc0d4140 2024-04-25T11:23:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.968+0000 7f0d02800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0cfc0b51c0 con 0x7f0cfc0b4db0 2024-04-25T11:23:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.968+0000 7f0d02800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0cfc163a00 con 0x7f0cfc0d2720 2024-04-25T11:23:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.969+0000 7f0d01e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cfc0d4140 0x7f0cfc0d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.969+0000 7f0d01e00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cfc0d4140 0x7f0cfc0d4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47984/0 (socket says 172.21.15.7:47984) 2024-04-25T11:23:06.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.969+0000 7f0d01e00640 1 -- 172.21.15.7:0/523583155 learned_addr learned my addr 172.21.15.7:0/523583155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:06.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.969+0000 7f0d01400640 1 --2- 172.21.15.7:0/523583155 >> v2:172.21.15.87:3300/0 conn(0x7f0cfc0b4db0 0x7f0cfc1644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.969+0000 7f0d00a00640 1 --2- 172.21.15.7:0/523583155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cfc0d2720 0x7f0cfc0d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.969+0000 7f0d01e00640 1 -- 172.21.15.7:0/523583155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cfc0d2720 msgr2=0x7f0cfc0d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:06.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.969+0000 7f0d01e00640 1 --2- 172.21.15.7:0/523583155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cfc0d2720 0x7f0cfc0d2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.969+0000 7f0d01e00640 1 -- 172.21.15.7:0/523583155 >> v2:172.21.15.87:3300/0 conn(0x7f0cfc0b4db0 msgr2=0x7f0cfc1644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:06.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.969+0000 7f0d01e00640 1 --2- 172.21.15.7:0/523583155 >> v2:172.21.15.87:3300/0 conn(0x7f0cfc0b4db0 0x7f0cfc1644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.969+0000 7f0d01e00640 1 -- 172.21.15.7:0/523583155 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc0d0780 con 0x7f0cfc0d4140 2024-04-25T11:23:06.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.971+0000 7f0d01e00640 1 --2- 172.21.15.7:0/523583155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc0d4140 0x7f0cfc0d4550 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc08e290 tx=0x7f0cf402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55de0f0af3e458db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.971+0000 7f0cfbe00640 1 -- 172.21.15.7:0/523583155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4036070 con 0x7f0cfc0d4140 2024-04-25T11:23:06.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.971+0000 7f0cfbe00640 1 -- 172.21.15.7:0/523583155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf4031600 con 0x7f0cfc0d4140 2024-04-25T11:23:06.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.971+0000 7f0cfbe00640 1 -- 172.21.15.7:0/523583155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf40319d0 con 0x7f0cfc0d4140 2024-04-25T11:23:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.972+0000 7f0d02800640 1 -- 172.21.15.7:0/523583155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc0d4140 msgr2=0x7f0cfc0d4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.972+0000 7f0d02800640 1 --2- 172.21.15.7:0/523583155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc0d4140 0x7f0cfc0d4550 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc08e290 tx=0x7f0cf402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.972+0000 7f0d02800640 1 -- 172.21.15.7:0/523583155 shutdown_connections 2024-04-25T11:23:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.972+0000 7f0d02800640 1 --2- 172.21.15.7:0/523583155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc0d4140 0x7f0cfc0d4550 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.972+0000 7f0d02800640 1 --2- 172.21.15.7:0/523583155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cfc0d2720 0x7f0cfc0d2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.972+0000 7f0d02800640 1 --2- 172.21.15.7:0/523583155 >> v2:172.21.15.87:3300/0 conn(0x7f0cfc0b4db0 0x7f0cfc1644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.972+0000 7f0d02800640 1 -- 172.21.15.7:0/523583155 >> 172.21.15.7:0/523583155 conn(0x7f0cfc08cec0 msgr2=0x7f0cfc08d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.972+0000 7f0d02800640 1 -- 172.21.15.7:0/523583155 shutdown_connections 2024-04-25T11:23:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.973+0000 7f0d02800640 1 -- 172.21.15.7:0/523583155 wait complete. 2024-04-25T11:23:06.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.975+0000 7f0d02800640 1 Processor -- start 2024-04-25T11:23:06.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.975+0000 7f0d02800640 1 -- start start 2024-04-25T11:23:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d02800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cfc0b4db0 0x7f0cfc151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d02800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cfc0d2720 0x7f0cfc157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d02800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc0d4140 0x7f0cfc15a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d02800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc163cc0 con 0x7f0cfc0d4140 2024-04-25T11:23:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d02800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1612b0 con 0x7f0cfc0b4db0 2024-04-25T11:23:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d02800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0cfc163fa0 con 0x7f0cfc0d2720 2024-04-25T11:23:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d01e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc0d4140 0x7f0cfc15a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d01e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc0d4140 0x7f0cfc15a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47992/0 (socket says 172.21.15.7:47992) 2024-04-25T11:23:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d01e00640 1 -- 172.21.15.7:0/2090819717 learned_addr learned my addr 172.21.15.7:0/2090819717 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:06.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d01400640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cfc0b4db0 0x7f0cfc151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d00a00640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cfc0d2720 0x7f0cfc157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d01e00640 1 -- 172.21.15.7:0/2090819717 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cfc0d2720 msgr2=0x7f0cfc157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d01e00640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cfc0d2720 0x7f0cfc157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d01e00640 1 -- 172.21.15.7:0/2090819717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cfc0b4db0 msgr2=0x7f0cfc151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d01e00640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cfc0b4db0 0x7f0cfc151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.977+0000 7f0d01e00640 1 -- 172.21.15.7:0/2090819717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc0b5450 con 0x7f0cfc0d4140 2024-04-25T11:23:06.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.978+0000 7f0d01400640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cfc0b4db0 0x7f0cfc151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:06.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.978+0000 7f0d01e00640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc0d4140 0x7f0cfc15a340 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009f40 tx=0x7f0cf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.978+0000 7f0cf9600640 1 -- 172.21.15.7:0/2090819717 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf403f020 con 0x7f0cfc0d4140 2024-04-25T11:23:06.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.978+0000 7f0cf9600640 1 -- 172.21.15.7:0/2090819717 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf4037040 con 0x7f0cfc0d4140 2024-04-25T11:23:06.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.978+0000 7f0cf9600640 1 -- 172.21.15.7:0/2090819717 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4036070 con 0x7f0cfc0d4140 2024-04-25T11:23:06.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.978+0000 7f0d02800640 1 -- 172.21.15.7:0/2090819717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc0b5730 con 0x7f0cfc0d4140 2024-04-25T11:23:06.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.978+0000 7f0d02800640 1 -- 172.21.15.7:0/2090819717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc0b5cf0 con 0x7f0cfc0d4140 2024-04-25T11:23:06.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.979+0000 7f0d02800640 1 -- 172.21.15.7:0/2090819717 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0cfc050740 con 0x7f0cfc0d4140 2024-04-25T11:23:06.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.981+0000 7f0cf9600640 1 -- 172.21.15.7:0/2090819717 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f0cf4042020 con 0x7f0cfc0d4140 2024-04-25T11:23:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.982+0000 7f0cf9600640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0cdc074530 0x7f0cdc076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.982+0000 7f0cf9600640 1 -- 172.21.15.7:0/2090819717 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f0cf40b5ba0 con 0x7f0cfc0d4140 2024-04-25T11:23:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.982+0000 7f0d01400640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0cdc074530 0x7f0cdc076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.982+0000 7f0cf9600640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] conn(0x7f0cdc07aa20 0x7f0cdc07ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.982+0000 7f0cf9600640 1 -- 172.21.15.7:0/2090819717 --> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0cf4031600 con 0x7f0cdc07aa20 2024-04-25T11:23:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.982+0000 7f0cf9600640 1 -- 172.21.15.7:0/2090819717 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=159) ==== 24+0+0 (secure 0 0 0) 0x7f0cf40b5f50 con 0x7f0cfc0d4140 2024-04-25T11:23:06.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.983+0000 7f0d00a00640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] conn(0x7f0cdc07aa20 0x7f0cdc07ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.983+0000 7f0d01400640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0cdc074530 0x7f0cdc076a40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0cec006fd0 tx=0x7f0cec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.983+0000 7f0d00a00640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] conn(0x7f0cdc07aa20 0x7f0cdc07ceb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.984+0000 7f0cf9600640 1 -- 172.21.15.7:0/2090819717 <== osd.1 v2:172.21.15.7:6822/1767080083 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0cf4031600 con 0x7f0cdc07aa20 2024-04-25T11:23:06.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:06.999+0000 7f0d02800640 1 -- 172.21.15.7:0/2090819717 --> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0cfc0d3cf0 con 0x7f0cdc07aa20 2024-04-25T11:23:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.000+0000 7f0cf9600640 1 -- 172.21.15.7:0/2090819717 <== osd.1 v2:172.21.15.7:6822/1767080083 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0cfc0d3cf0 con 0x7f0cdc07aa20 2024-04-25T11:23:06.995 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:23:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:23:06.995 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:23:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.000+0000 7f0d02800640 1 -- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] conn(0x7f0cdc07aa20 msgr2=0x7f0cdc07ceb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.000+0000 7f0d02800640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:6822/1767080083,v1:172.21.15.7:6824/1767080083] conn(0x7f0cdc07aa20 0x7f0cdc07ceb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.000+0000 7f0d02800640 1 -- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0cdc074530 msgr2=0x7f0cdc076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.000+0000 7f0d02800640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f0cdc074530 0x7f0cdc076a40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0cec006fd0 tx=0x7f0cec008040 comp rx=0 tx=0).stop 2024-04-25T11:23:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.000+0000 7f0d02800640 1 -- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc0d4140 msgr2=0x7f0cfc15a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.000+0000 7f0d02800640 1 --2- 172.21.15.7:0/2090819717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cfc0d4140 0x7f0cfc15a340 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009f40 tx=0x7f0cf4004a40 comp rx=0 tx=0).stop 2024-04-25T11:23:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.000+0000 7f0d01e00640 1 -- 172.21.15.7:0/2090819717 reap_dead start 2024-04-25T11:23:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.000+0000 7f0d02800640 1 -- 172.21.15.7:0/2090819717 shutdown_connections 2024-04-25T11:23:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.001+0000 7f0d02800640 1 -- 172.21.15.7:0/2090819717 >> 172.21.15.7:0/2090819717 conn(0x7f0cfc08cec0 msgr2=0x7f0cfc15cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.001+0000 7f0d02800640 1 -- 172.21.15.7:0/2090819717 shutdown_connections 2024-04-25T11:23:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.001+0000 7f0d02800640 1 -- 172.21.15.7:0/2090819717 wait complete. 2024-04-25T11:23:07.020 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T11:23:07.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.309+0000 7fe18cc00640 1 Processor -- start 2024-04-25T11:23:07.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.309+0000 7fe18cc00640 1 -- start start 2024-04-25T11:23:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.312+0000 7fe18cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe1880b4e30 0x7fe1880bb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.312+0000 7fe18cc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1880bbb90 con 0x7fe1880b7d60 2024-04-25T11:23:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.312+0000 7fe18cc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe188164d60 con 0x7fe1880b9470 2024-04-25T11:23:07.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.312+0000 7fe18cc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe1880d0b50 con 0x7fe1880b4e30 2024-04-25T11:23:07.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.312+0000 7fe186800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe1880b7d60 0x7fe1880d0fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34536/0 (socket says 172.21.15.7:34536) 2024-04-25T11:23:07.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.312+0000 7fe186800640 1 -- 172.21.15.7:0/2273089044 learned_addr learned my addr 172.21.15.7:0/2273089044 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:07.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.312+0000 7fe185e00640 1 --2- 172.21.15.7:0/2273089044 >> v2:172.21.15.87:3300/0 conn(0x7fe1880b4e30 0x7fe1880bb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956023235 0 0) 0x7fe180001490 con 0x7fe1880b7d60 2024-04-25T11:23:07.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe16c003520 con 0x7fe1880b7d60 2024-04-25T11:23:07.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3998760819 0 0) 0x7fe16c003520 con 0x7fe1880b7d60 2024-04-25T11:23:07.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 >> v2:172.21.15.87:3300/0 conn(0x7fe1880b4e30 msgr2=0x7fe1880bb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:07.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 --2- 172.21.15.7:0/2273089044 >> v2:172.21.15.87:3300/0 conn(0x7fe1880b4e30 0x7fe1880bb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 >> v1:172.21.15.87:6789/0 conn(0x7fe1880b9470 legacy=0x7fe1880b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1880d0220 con 0x7fe1880b7d60 2024-04-25T11:23:07.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7fe180001bc0 con 0x7fe1880b7d60 2024-04-25T11:23:07.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 >> v1:172.21.15.7:6789/0 conn(0x7fe1880b7d60 legacy=0x7fe1880d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 --2- 172.21.15.7:0/2273089044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c0039b0 0x7fe16c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 --2- 172.21.15.7:0/2273089044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe16c004800 0x7fe16c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 --2- 172.21.15.7:0/2273089044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c005250 0x7fe16c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1880d0b50 con 0x7fe16c0039b0 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c00c1e0 con 0x7fe16c005250 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.313+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe16c00c3e0 con 0x7fe16c004800 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.314+0000 7fe185e00640 1 --2- 172.21.15.7:0/2273089044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c0039b0 0x7fe16c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.314+0000 7fe185400640 1 --2- 172.21.15.7:0/2273089044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe16c004800 0x7fe16c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.314+0000 7fe185e00640 1 -- 172.21.15.7:0/2273089044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe16c004800 msgr2=0x7fe16c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.314+0000 7fe185e00640 1 --2- 172.21.15.7:0/2273089044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe16c004800 0x7fe16c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.314+0000 7fe185e00640 1 -- 172.21.15.7:0/2273089044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c005250 msgr2=0x7fe16c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.314+0000 7fe185e00640 1 --2- 172.21.15.7:0/2273089044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c005250 0x7fe16c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.314+0000 7fe185e00640 1 -- 172.21.15.7:0/2273089044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16c003520 con 0x7fe16c0039b0 2024-04-25T11:23:07.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.314+0000 7fe186800640 1 -- 172.21.15.7:0/2273089044 reap_dead start 2024-04-25T11:23:07.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.315+0000 7fe185e00640 1 --2- 172.21.15.7:0/2273089044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c0039b0 0x7fe16c003e20 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fe178009a90 tx=0x7fe16c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f861af7fcaba05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:07.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.316+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178036070 con 0x7fe16c0039b0 2024-04-25T11:23:07.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.316+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe178031400 con 0x7fe16c0039b0 2024-04-25T11:23:07.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.316+0000 7fe184a00640 1 -- 172.21.15.7:0/2273089044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178009690 con 0x7fe16c0039b0 2024-04-25T11:23:07.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.316+0000 7fe18cc00640 1 -- 172.21.15.7:0/2273089044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c0039b0 msgr2=0x7fe16c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.316+0000 7fe18cc00640 1 --2- 172.21.15.7:0/2273089044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c0039b0 0x7fe16c003e20 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fe178009a90 tx=0x7fe16c005830 comp rx=0 tx=0).stop 2024-04-25T11:23:07.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.316+0000 7fe18cc00640 1 -- 172.21.15.7:0/2273089044 shutdown_connections 2024-04-25T11:23:07.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.316+0000 7fe18cc00640 1 --2- 172.21.15.7:0/2273089044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16c0039b0 0x7fe16c003e20 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.316+0000 7fe18cc00640 1 -- 172.21.15.7:0/2273089044 >> 172.21.15.7:0/2273089044 conn(0x7fe1880d2650 msgr2=0x7fe1880d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:07.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.317+0000 7fe18cc00640 1 -- 172.21.15.7:0/2273089044 shutdown_connections 2024-04-25T11:23:07.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.317+0000 7fe18cc00640 1 -- 172.21.15.7:0/2273089044 wait complete. 2024-04-25T11:23:07.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.319+0000 7fe18cc00640 1 Processor -- start 2024-04-25T11:23:07.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.319+0000 7fe18cc00640 1 -- start start 2024-04-25T11:23:07.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.321+0000 7fe18cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c0039b0 0x7fe1881624a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.321+0000 7fe18cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1880cda40 0x7fe1880d5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.321+0000 7fe18cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1880d6280 0x7fe18815ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.321+0000 7fe18cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1880d07e0 con 0x7fe1880d6280 2024-04-25T11:23:07.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.321+0000 7fe18cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe18808e3a0 con 0x7fe16c0039b0 2024-04-25T11:23:07.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.321+0000 7fe18cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe1881649c0 con 0x7fe1880cda40 2024-04-25T11:23:07.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.321+0000 7fe186800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1880d6280 0x7fe18815ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.321+0000 7fe186800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1880d6280 0x7fe18815ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48008/0 (socket says 172.21.15.7:48008) 2024-04-25T11:23:07.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.321+0000 7fe186800640 1 -- 172.21.15.7:0/2534852780 learned_addr learned my addr 172.21.15.7:0/2534852780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:07.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.321+0000 7fe185e00640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c0039b0 0x7fe1881624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.321+0000 7fe185400640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1880cda40 0x7fe1880d5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.322+0000 7fe186800640 1 -- 172.21.15.7:0/2534852780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1880cda40 msgr2=0x7fe1880d5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.322+0000 7fe186800640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1880cda40 0x7fe1880d5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.322+0000 7fe186800640 1 -- 172.21.15.7:0/2534852780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c0039b0 msgr2=0x7fe1881624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.322+0000 7fe186800640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c0039b0 0x7fe1881624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.322+0000 7fe186800640 1 -- 172.21.15.7:0/2534852780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1880cc190 con 0x7fe1880d6280 2024-04-25T11:23:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.322+0000 7fe186800640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1880d6280 0x7fe18815ade0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fe1880b8f60 tx=0x7fe18000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.322+0000 7fe17e000640 1 -- 172.21.15.7:0/2534852780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18000a610 con 0x7fe1880d6280 2024-04-25T11:23:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.322+0000 7fe17e000640 1 -- 172.21.15.7:0/2534852780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe18000a7b0 con 0x7fe1880d6280 2024-04-25T11:23:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.322+0000 7fe17e000640 1 -- 172.21.15.7:0/2534852780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18000aba0 con 0x7fe1880d6280 2024-04-25T11:23:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.322+0000 7fe18cc00640 1 -- 172.21.15.7:0/2534852780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1880c9810 con 0x7fe1880d6280 2024-04-25T11:23:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.323+0000 7fe18cc00640 1 -- 172.21.15.7:0/2534852780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1880c9dd0 con 0x7fe1880d6280 2024-04-25T11:23:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.324+0000 7fe18cc00640 1 -- 172.21.15.7:0/2534852780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1880cbf00 con 0x7fe1880d6280 2024-04-25T11:23:07.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.325+0000 7fe17e000640 1 -- 172.21.15.7:0/2534852780 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe18000ad40 con 0x7fe1880d6280 2024-04-25T11:23:07.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.326+0000 7fe17e000640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1680744e0 0x7fe1680769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.326+0000 7fe17e000640 1 -- 172.21.15.7:0/2534852780 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7fe180094680 con 0x7fe1880d6280 2024-04-25T11:23:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.327+0000 7fe185e00640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1680744e0 0x7fe1680769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.328+0000 7fe185e00640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1680744e0 0x7fe1680769f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe178033000 tx=0x7fe178008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.332+0000 7fe17e000640 1 -- 172.21.15.7:0/2534852780 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe18001a030 con 0x7fe1880d6280 2024-04-25T11:23:07.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.539+0000 7fe18cc00640 1 -- 172.21.15.7:0/2534852780 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fe188050740 con 0x7fe1680744e0 2024-04-25T11:23:07.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.540+0000 7fe17e000640 1 -- 172.21.15.7:0/2534852780 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe188050740 con 0x7fe1680744e0 2024-04-25T11:23:07.536 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T11:23:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.543+0000 7fe18cc00640 1 -- 172.21.15.7:0/2534852780 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1680744e0 msgr2=0x7fe1680769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.543+0000 7fe18cc00640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1680744e0 0x7fe1680769f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe178033000 tx=0x7fe178008030 comp rx=0 tx=0).stop 2024-04-25T11:23:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.543+0000 7fe18cc00640 1 -- 172.21.15.7:0/2534852780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1880d6280 msgr2=0x7fe18815ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.543+0000 7fe18cc00640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1880d6280 0x7fe18815ade0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fe1880b8f60 tx=0x7fe18000ccd0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.543+0000 7fe18cc00640 1 -- 172.21.15.7:0/2534852780 shutdown_connections 2024-04-25T11:23:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.543+0000 7fe18cc00640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe1680744e0 0x7fe1680769f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.543+0000 7fe18cc00640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1880d6280 0x7fe18815ade0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.543+0000 7fe18cc00640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1880cda40 0x7fe1880d5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.543+0000 7fe18cc00640 1 --2- 172.21.15.7:0/2534852780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe16c0039b0 0x7fe1881624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.543+0000 7fe18cc00640 1 -- 172.21.15.7:0/2534852780 >> 172.21.15.7:0/2534852780 conn(0x7fe1880d2650 msgr2=0x7fe1880cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:07.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.543+0000 7fe18cc00640 1 -- 172.21.15.7:0/2534852780 shutdown_connections 2024-04-25T11:23:07.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.544+0000 7fe18cc00640 1 -- 172.21.15.7:0/2534852780 wait complete. 2024-04-25T11:23:07.568 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T11:23:07.569 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:23:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.862+0000 7f0600a00640 1 Processor -- start 2024-04-25T11:23:07.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.863+0000 7f0600a00640 1 -- start start 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.865+0000 7f0600a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05fc0cf370 0x7f05fc0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.865+0000 7f0600a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05fc06d580 0x7f05fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.865+0000 7f0600a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05fc06ded0 con 0x7f05fc1440a0 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.865+0000 7f0600a00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f05fc15e620 con 0x7f05fc0cf370 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.865+0000 7f0600a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f05fc15e900 con 0x7f05fc06d580 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.865+0000 7f05fa800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f05fc1440a0 0x7f05fc15f230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34548/0 (socket says 172.21.15.7:34548) 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.865+0000 7f05fa800640 1 -- 172.21.15.7:0/3805356771 learned_addr learned my addr 172.21.15.7:0/3805356771 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.866+0000 7f05f9e00640 1 --2- 172.21.15.7:0/3805356771 >> v2:172.21.15.7:3300/0 conn(0x7f05fc0cf370 0x7f05fc0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.866+0000 7f05fb200640 1 --2- 172.21.15.7:0/3805356771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05fc06d580 0x7f05fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.866+0000 7f05f9400640 1 -- 172.21.15.7:0/3805356771 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3016010654 0 0) 0x7f05ec001110 con 0x7f05fc1440a0 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.866+0000 7f05f9400640 1 -- 172.21.15.7:0/3805356771 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05e0003520 con 0x7f05fc1440a0 2024-04-25T11:23:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.866+0000 7f05f9e00640 1 -- 172.21.15.7:0/3805356771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05fc06d580 msgr2=0x7f05fc06d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.866+0000 7f05f9e00640 1 --2- 172.21.15.7:0/3805356771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05fc06d580 0x7f05fc06d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.866+0000 7f05f9e00640 1 -- 172.21.15.7:0/3805356771 >> v1:172.21.15.7:6789/0 conn(0x7f05fc1440a0 legacy=0x7f05fc15f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.866+0000 7f05f9e00640 1 -- 172.21.15.7:0/3805356771 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc15ebe0 con 0x7f05fc0cf370 2024-04-25T11:23:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.867+0000 7f05f9e00640 1 --2- 172.21.15.7:0/3805356771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc0cf370 0x7f05fc0cf780 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f05e80097a0 tx=0x7f05e802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f2acd5ee993d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.868+0000 7f05f9400640 1 -- 172.21.15.7:0/3805356771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8037070 con 0x7f05fc0cf370 2024-04-25T11:23:07.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.868+0000 7f05f9400640 1 -- 172.21.15.7:0/3805356771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e802ae50 con 0x7f05fc0cf370 2024-04-25T11:23:07.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.868+0000 7f05f9400640 1 -- 172.21.15.7:0/3805356771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8032660 con 0x7f05fc0cf370 2024-04-25T11:23:07.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.869+0000 7f0600a00640 1 -- 172.21.15.7:0/3805356771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc0cf370 msgr2=0x7f05fc0cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.869+0000 7f0600a00640 1 --2- 172.21.15.7:0/3805356771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc0cf370 0x7f05fc0cf780 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f05e80097a0 tx=0x7f05e802a100 comp rx=0 tx=0).stop 2024-04-25T11:23:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.869+0000 7f0600a00640 1 -- 172.21.15.7:0/3805356771 shutdown_connections 2024-04-25T11:23:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.869+0000 7f0600a00640 1 --2- 172.21.15.7:0/3805356771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05fc06d580 0x7f05fc06d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.869+0000 7f0600a00640 1 --2- 172.21.15.7:0/3805356771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc0cf370 0x7f05fc0cf780 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.869+0000 7f0600a00640 1 -- 172.21.15.7:0/3805356771 >> 172.21.15.7:0/3805356771 conn(0x7f05fc073160 msgr2=0x7f05fc073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.869+0000 7f0600a00640 1 -- 172.21.15.7:0/3805356771 shutdown_connections 2024-04-25T11:23:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.869+0000 7f0600a00640 1 -- 172.21.15.7:0/3805356771 wait complete. 2024-04-25T11:23:07.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.871+0000 7f0600a00640 1 Processor -- start 2024-04-25T11:23:07.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.872+0000 7f0600a00640 1 -- start start 2024-04-25T11:23:07.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.873+0000 7f0600a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05fc06d580 0x7f05fc0af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.873+0000 7f0600a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc0cf370 0x7f05fc137d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.873+0000 7f0600a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05fc1440a0 0x7f05fc1382d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.873+0000 7f0600a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc15e200 con 0x7f05fc0cf370 2024-04-25T11:23:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.873+0000 7f0600a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc1444b0 con 0x7f05fc1440a0 2024-04-25T11:23:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.873+0000 7f0600a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f05fc15e380 con 0x7f05fc06d580 2024-04-25T11:23:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.874+0000 7f05f9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc0cf370 0x7f05fc137d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.874+0000 7f05f9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc0cf370 0x7f05fc137d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48022/0 (socket says 172.21.15.7:48022) 2024-04-25T11:23:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.874+0000 7f05f9e00640 1 -- 172.21.15.7:0/1566093300 learned_addr learned my addr 172.21.15.7:0/1566093300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.874+0000 7f05fa800640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05fc06d580 0x7f05fc0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.874+0000 7f05fb200640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05fc1440a0 0x7f05fc1382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.874+0000 7f05f9e00640 1 -- 172.21.15.7:0/1566093300 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05fc06d580 msgr2=0x7f05fc0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.874+0000 7f05f9e00640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05fc06d580 0x7f05fc0af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.874+0000 7f05f9e00640 1 -- 172.21.15.7:0/1566093300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05fc1440a0 msgr2=0x7f05fc1382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.874+0000 7f05f9e00640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05fc1440a0 0x7f05fc1382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.874+0000 7f05f9e00640 1 -- 172.21.15.7:0/1566093300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc0aff50 con 0x7f05fc0cf370 2024-04-25T11:23:07.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.874+0000 7f05fb200640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05fc1440a0 0x7f05fc1382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:07.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.875+0000 7f05f9e00640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc0cf370 0x7f05fc137d90 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f05e8035040 tx=0x7f05e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:07.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.875+0000 7f05f2a00640 1 -- 172.21.15.7:0/1566093300 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8041070 con 0x7f05fc0cf370 2024-04-25T11:23:07.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.875+0000 7f05f2a00640 1 -- 172.21.15.7:0/1566093300 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e803c460 con 0x7f05fc0cf370 2024-04-25T11:23:07.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.875+0000 7f05f2a00640 1 -- 172.21.15.7:0/1566093300 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8037040 con 0x7f05fc0cf370 2024-04-25T11:23:07.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.875+0000 7f0600a00640 1 -- 172.21.15.7:0/1566093300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc0b0230 con 0x7f05fc0cf370 2024-04-25T11:23:07.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.875+0000 7f0600a00640 1 -- 172.21.15.7:0/1566093300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc0b07f0 con 0x7f05fc0cf370 2024-04-25T11:23:07.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.876+0000 7f0600a00640 1 -- 172.21.15.7:0/1566093300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f05fc050740 con 0x7f05fc0cf370 2024-04-25T11:23:07.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.877+0000 7f05f2a00640 1 -- 172.21.15.7:0/1566093300 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f05e803a070 con 0x7f05fc0cf370 2024-04-25T11:23:07.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.878+0000 7f05f2a00640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f05dc0744b0 0x7f05dc0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.878+0000 7f05f2a00640 1 -- 172.21.15.7:0/1566093300 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f05e80b4f10 con 0x7f05fc0cf370 2024-04-25T11:23:07.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.879+0000 7f05fa800640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f05dc0744b0 0x7f05dc0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.879+0000 7f05f2a00640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] conn(0x7f05dc07a9a0 0x7f05dc07ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:07.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.879+0000 7f05f2a00640 1 -- 172.21.15.7:0/1566093300 --> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05dc07d500 con 0x7f05dc07a9a0 2024-04-25T11:23:07.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.879+0000 7f05f2a00640 1 -- 172.21.15.7:0/1566093300 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=159) ==== 24+0+0 (secure 0 0 0) 0x7f05e80bc050 con 0x7f05fc0cf370 2024-04-25T11:23:07.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.879+0000 7f05fb200640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] conn(0x7f05dc07a9a0 0x7f05dc07ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:07.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.879+0000 7f05fa800640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f05dc0744b0 0x7f05dc0769c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f05ec007fd0 tx=0x7f05ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:07.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.880+0000 7f05fb200640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] conn(0x7f05dc07a9a0 0x7f05dc07ce30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:07.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.881+0000 7f05f2a00640 1 -- 172.21.15.7:0/1566093300 <== osd.2 v2:172.21.15.7:6816/3878005205 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f05dc07d500 con 0x7f05dc07a9a0 2024-04-25T11:23:07.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.896+0000 7f0600a00640 1 -- 172.21.15.7:0/1566093300 --> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f05fc0afce0 con 0x7f05dc07a9a0 2024-04-25T11:23:07.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.896+0000 7f05f2a00640 1 -- 172.21.15.7:0/1566093300 <== osd.2 v2:172.21.15.7:6816/3878005205 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f05fc0afce0 con 0x7f05dc07a9a0 2024-04-25T11:23:07.892 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:23:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:23:07.892 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:23:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.897+0000 7f0600a00640 1 -- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] conn(0x7f05dc07a9a0 msgr2=0x7f05dc07ce30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.897+0000 7f0600a00640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:6816/3878005205,v1:172.21.15.7:6817/3878005205] conn(0x7f05dc07a9a0 0x7f05dc07ce30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.897+0000 7f0600a00640 1 -- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f05dc0744b0 msgr2=0x7f05dc0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.897+0000 7f0600a00640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f05dc0744b0 0x7f05dc0769c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f05ec007fd0 tx=0x7f05ec009040 comp rx=0 tx=0).stop 2024-04-25T11:23:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.897+0000 7f0600a00640 1 -- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc0cf370 msgr2=0x7f05fc137d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.897+0000 7f0600a00640 1 --2- 172.21.15.7:0/1566093300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05fc0cf370 0x7f05fc137d90 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f05e8035040 tx=0x7f05e80046c0 comp rx=0 tx=0).stop 2024-04-25T11:23:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.897+0000 7f05fb200640 1 -- 172.21.15.7:0/1566093300 reap_dead start 2024-04-25T11:23:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.898+0000 7f0600a00640 1 -- 172.21.15.7:0/1566093300 shutdown_connections 2024-04-25T11:23:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.898+0000 7f0600a00640 1 -- 172.21.15.7:0/1566093300 >> 172.21.15.7:0/1566093300 conn(0x7f05fc073160 msgr2=0x7f05fc1605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.898+0000 7f0600a00640 1 -- 172.21.15.7:0/1566093300 shutdown_connections 2024-04-25T11:23:07.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:07.898+0000 7f0600a00640 1 -- 172.21.15.7:0/1566093300 wait complete. 2024-04-25T11:23:07.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T11:23:08.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.212+0000 7f92fdc00640 1 Processor -- start 2024-04-25T11:23:08.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.213+0000 7f92fdc00640 1 -- start start 2024-04-25T11:23:08.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.215+0000 7f92fdc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f92f80d1350 0x7f92f81648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.215+0000 7f92fdc00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92f806d580 0x7f92f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.215+0000 7f92fdc00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f80b5660 con 0x7f92f80b6960 2024-04-25T11:23:08.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.215+0000 7f92fdc00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f92f80b64d0 con 0x7f92f806d580 2024-04-25T11:23:08.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.215+0000 7f92fdc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f92f806ded0 con 0x7f92f80d1350 2024-04-25T11:23:08.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.215+0000 7f92f7400640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92f806d580 0x7f92f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.215+0000 7f92f6a00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f92f80b6960 0x7f92f8160220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34552/0 (socket says 172.21.15.7:34552) 2024-04-25T11:23:08.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.215+0000 7f92f6a00640 1 -- 172.21.15.7:0/948707241 learned_addr learned my addr 172.21.15.7:0/948707241 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:08.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.216+0000 7f92f6000640 1 --2- 172.21.15.7:0/948707241 >> v2:172.21.15.87:3300/0 conn(0x7f92f80d1350 0x7f92f81648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.216+0000 7f92f7400640 1 -- 172.21.15.7:0/948707241 >> v2:172.21.15.87:3300/0 conn(0x7f92f80d1350 msgr2=0x7f92f81648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:08.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.216+0000 7f92f7400640 1 --2- 172.21.15.7:0/948707241 >> v2:172.21.15.87:3300/0 conn(0x7f92f80d1350 0x7f92f81648b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.216+0000 7f92f7400640 1 -- 172.21.15.7:0/948707241 >> v1:172.21.15.7:6789/0 conn(0x7f92f80b6960 legacy=0x7f92f8160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.216+0000 7f92f7400640 1 -- 172.21.15.7:0/948707241 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f80b58e0 con 0x7f92f806d580 2024-04-25T11:23:08.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.216+0000 7f92f5600640 1 -- 172.21.15.7:0/948707241 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1060745700 0 0) 0x7f92e8001490 con 0x7f92f80b6960 2024-04-25T11:23:08.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.217+0000 7f92f7400640 1 --2- 172.21.15.7:0/948707241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f806d580 0x7f92f806d990 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f92f80cf5c0 tx=0x7f92f002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2f707722471a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:08.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.217+0000 7f92f5600640 1 -- 172.21.15.7:0/948707241 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0037070 con 0x7f92f806d580 2024-04-25T11:23:08.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.218+0000 7f92f5600640 1 -- 172.21.15.7:0/948707241 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f0032440 con 0x7f92f806d580 2024-04-25T11:23:08.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.218+0000 7f92f5600640 1 -- 172.21.15.7:0/948707241 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f002ae60 con 0x7f92f806d580 2024-04-25T11:23:08.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.218+0000 7f92fdc00640 1 -- 172.21.15.7:0/948707241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f806d580 msgr2=0x7f92f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.218+0000 7f92fdc00640 1 --2- 172.21.15.7:0/948707241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f806d580 0x7f92f806d990 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f92f80cf5c0 tx=0x7f92f002a110 comp rx=0 tx=0).stop 2024-04-25T11:23:08.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.218+0000 7f92fdc00640 1 -- 172.21.15.7:0/948707241 shutdown_connections 2024-04-25T11:23:08.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.218+0000 7f92fdc00640 1 --2- 172.21.15.7:0/948707241 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f806d580 0x7f92f806d990 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.218+0000 7f92fdc00640 1 --2- 172.21.15.7:0/948707241 >> v2:172.21.15.87:3300/0 conn(0x7f92f80d1350 0x7f92f81648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.219+0000 7f92fdc00640 1 -- 172.21.15.7:0/948707241 >> 172.21.15.7:0/948707241 conn(0x7f92f80732b0 msgr2=0x7f92f80736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:08.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.219+0000 7f92fdc00640 1 -- 172.21.15.7:0/948707241 shutdown_connections 2024-04-25T11:23:08.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.219+0000 7f92fdc00640 1 -- 172.21.15.7:0/948707241 wait complete. 2024-04-25T11:23:08.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.221+0000 7f92fdc00640 1 Processor -- start 2024-04-25T11:23:08.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.221+0000 7f92fdc00640 1 -- start start 2024-04-25T11:23:08.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92fdc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92f806d580 0x7f92f815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92fdc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92f80b6960 0x7f92f80cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92fdc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f80d1350 0x7f92f8149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92fdc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f92f80b5e80 con 0x7f92f80d1350 2024-04-25T11:23:08.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92fdc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f92f80b5ba0 con 0x7f92f806d580 2024-04-25T11:23:08.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92fdc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f92f80b6160 con 0x7f92f80b6960 2024-04-25T11:23:08.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92f7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f80d1350 0x7f92f8149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92f7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f80d1350 0x7f92f8149d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48046/0 (socket says 172.21.15.7:48046) 2024-04-25T11:23:08.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92f7400640 1 -- 172.21.15.7:0/2133214475 learned_addr learned my addr 172.21.15.7:0/2133214475 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:08.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92f6a00640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92f806d580 0x7f92f815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92f6000640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92f80b6960 0x7f92f80cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.223+0000 7f92f7400640 1 -- 172.21.15.7:0/2133214475 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92f80b6960 msgr2=0x7f92f80cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.224+0000 7f92f7400640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92f80b6960 0x7f92f80cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.224+0000 7f92f7400640 1 -- 172.21.15.7:0/2133214475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92f806d580 msgr2=0x7f92f815aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.224+0000 7f92f7400640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92f806d580 0x7f92f815aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.224+0000 7f92f7400640 1 -- 172.21.15.7:0/2133214475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f81582c0 con 0x7f92f80d1350 2024-04-25T11:23:08.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.224+0000 7f92f6a00640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92f806d580 0x7f92f815aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:08.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.224+0000 7f92f7400640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f80d1350 0x7f92f8149d40 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f92f0035040 tx=0x7f92f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:08.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.224+0000 7f92eea00640 1 -- 172.21.15.7:0/2133214475 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0037050 con 0x7f92f80d1350 2024-04-25T11:23:08.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.224+0000 7f92eea00640 1 -- 172.21.15.7:0/2133214475 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f0038040 con 0x7f92f80d1350 2024-04-25T11:23:08.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.224+0000 7f92fdc00640 1 -- 172.21.15.7:0/2133214475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f8161f40 con 0x7f92f80d1350 2024-04-25T11:23:08.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.224+0000 7f92eea00640 1 -- 172.21.15.7:0/2133214475 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0006e10 con 0x7f92f80d1350 2024-04-25T11:23:08.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.225+0000 7f92fdc00640 1 -- 172.21.15.7:0/2133214475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f8162500 con 0x7f92f80d1350 2024-04-25T11:23:08.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.226+0000 7f92fdc00640 1 -- 172.21.15.7:0/2133214475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f8158030 con 0x7f92f80d1350 2024-04-25T11:23:08.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.227+0000 7f92eea00640 1 -- 172.21.15.7:0/2133214475 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f92f0030980 con 0x7f92f80d1350 2024-04-25T11:23:08.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.228+0000 7f92eea00640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92d80744e0 0x7f92d80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.228+0000 7f92eea00640 1 -- 172.21.15.7:0/2133214475 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f92f00b59a0 con 0x7f92f80d1350 2024-04-25T11:23:08.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.228+0000 7f92f6a00640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92d80744e0 0x7f92d80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.229+0000 7f92f6a00640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92d80744e0 0x7f92d80769f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f92e8007fd0 tx=0x7f92e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:08.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.233+0000 7f92eea00640 1 -- 172.21.15.7:0/2133214475 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f92f004c020 con 0x7f92f80d1350 2024-04-25T11:23:08.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.440+0000 7f92fdc00640 1 -- 172.21.15.7:0/2133214475 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f92f8161d30 con 0x7f92d80744e0 2024-04-25T11:23:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.441+0000 7f92eea00640 1 -- 172.21.15.7:0/2133214475 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f92f8161d30 con 0x7f92d80744e0 2024-04-25T11:23:08.436 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T11:23:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 -- 172.21.15.7:0/2133214475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92d80744e0 msgr2=0x7f92d80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92d80744e0 0x7f92d80769f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f92e8007fd0 tx=0x7f92e8009040 comp rx=0 tx=0).stop 2024-04-25T11:23:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 -- 172.21.15.7:0/2133214475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f80d1350 msgr2=0x7f92f8149d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f80d1350 0x7f92f8149d40 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f92f0035040 tx=0x7f92f00046c0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 -- 172.21.15.7:0/2133214475 shutdown_connections 2024-04-25T11:23:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f92d80744e0 0x7f92d80769f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92f80d1350 0x7f92f8149d40 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92f80b6960 0x7f92f80cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 --2- 172.21.15.7:0/2133214475 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92f806d580 0x7f92f815aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 -- 172.21.15.7:0/2133214475 >> 172.21.15.7:0/2133214475 conn(0x7f92f80732b0 msgr2=0x7f92f8160220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 -- 172.21.15.7:0/2133214475 shutdown_connections 2024-04-25T11:23:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.444+0000 7f92fdc00640 1 -- 172.21.15.7:0/2133214475 wait complete. 2024-04-25T11:23:08.466 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T11:23:08.466 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:23:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.754+0000 7f9956000640 1 Processor -- start 2024-04-25T11:23:08.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.755+0000 7f9956000640 1 -- start start 2024-04-25T11:23:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.757+0000 7f9956000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9950143220 0x7f99500d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.757+0000 7f9956000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9950070e40 con 0x7f99500b4c60 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.757+0000 7f9956000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99500d02f0 con 0x7f995006d580 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.757+0000 7f9956000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f99500d05d0 con 0x7f9950143220 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.757+0000 7f9954c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9950143220 0x7f99500d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.757+0000 7f9955600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f99500b4c60 0x7f99500b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34558/0 (socket says 172.21.15.7:34558) 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.757+0000 7f9955600640 1 -- 172.21.15.7:0/3401458859 learned_addr learned my addr 172.21.15.7:0/3401458859 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:08.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.758+0000 7f9954c00640 1 -- 172.21.15.7:0/3401458859 >> v1:172.21.15.87:6789/0 conn(0x7f995006d580 legacy=0x7f995006d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.758+0000 7f9954c00640 1 -- 172.21.15.7:0/3401458859 >> v1:172.21.15.7:6789/0 conn(0x7f99500b4c60 legacy=0x7f99500b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.758+0000 7f9954c00640 1 -- 172.21.15.7:0/3401458859 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99500d08b0 con 0x7f9950143220 2024-04-25T11:23:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.758+0000 7f994fe00640 1 -- 172.21.15.7:0/3401458859 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3322963529 0 0) 0x7f9950070e40 con 0x7f99500b4c60 2024-04-25T11:23:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.759+0000 7f9954c00640 1 --2- 172.21.15.7:0/3401458859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950143220 0x7f99500d0dd0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f99400097a0 tx=0x7f994002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29111be5a0974c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.759+0000 7f994fe00640 1 -- 172.21.15.7:0/3401458859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940037070 con 0x7f9950143220 2024-04-25T11:23:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.759+0000 7f994fe00640 1 -- 172.21.15.7:0/3401458859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994002ae50 con 0x7f9950143220 2024-04-25T11:23:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.760+0000 7f994fe00640 1 -- 172.21.15.7:0/3401458859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940032660 con 0x7f9950143220 2024-04-25T11:23:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.760+0000 7f9956000640 1 -- 172.21.15.7:0/3401458859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950143220 msgr2=0x7f99500d0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.760+0000 7f9956000640 1 --2- 172.21.15.7:0/3401458859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950143220 0x7f99500d0dd0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f99400097a0 tx=0x7f994002a140 comp rx=0 tx=0).stop 2024-04-25T11:23:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.760+0000 7f9956000640 1 -- 172.21.15.7:0/3401458859 shutdown_connections 2024-04-25T11:23:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.760+0000 7f9956000640 1 --2- 172.21.15.7:0/3401458859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9950143220 0x7f99500d0dd0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.760+0000 7f9956000640 1 -- 172.21.15.7:0/3401458859 >> 172.21.15.7:0/3401458859 conn(0x7f9950073160 msgr2=0x7f9950073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.760+0000 7f9956000640 1 -- 172.21.15.7:0/3401458859 shutdown_connections 2024-04-25T11:23:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.760+0000 7f9956000640 1 -- 172.21.15.7:0/3401458859 wait complete. 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.763+0000 7f9956000640 1 Processor -- start 2024-04-25T11:23:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.763+0000 7f9956000640 1 -- start start 2024-04-25T11:23:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.764+0000 7f9956000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f995006d580 0x7f995013fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.764+0000 7f9956000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99500b4c60 0x7f9950158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.764+0000 7f9956000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9950143220 0x7f9950153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.764+0000 7f9956000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99500d00b0 con 0x7f995006d580 2024-04-25T11:23:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.764+0000 7f9956000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f99500cff30 con 0x7f9950143220 2024-04-25T11:23:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.764+0000 7f9956000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f99500700d0 con 0x7f99500b4c60 2024-04-25T11:23:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.765+0000 7f9954c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f995006d580 0x7f995013fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.765+0000 7f9954c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f995006d580 0x7f995013fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48062/0 (socket says 172.21.15.7:48062) 2024-04-25T11:23:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.765+0000 7f9954c00640 1 -- 172.21.15.7:0/3789726567 learned_addr learned my addr 172.21.15.7:0/3789726567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.765+0000 7f9955600640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9950143220 0x7f9950153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.765+0000 7f9954c00640 1 -- 172.21.15.7:0/3789726567 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99500b4c60 msgr2=0x7f9950158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.765+0000 7f9954c00640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99500b4c60 0x7f9950158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.765+0000 7f9954c00640 1 -- 172.21.15.7:0/3789726567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9950143220 msgr2=0x7f9950153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.765+0000 7f9954c00640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9950143220 0x7f9950153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.765+0000 7f9954c00640 1 -- 172.21.15.7:0/3789726567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9950159780 con 0x7f995006d580 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.766+0000 7f9954c00640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f995006d580 0x7f995013fc30 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f9940035040 tx=0x7f99400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.766+0000 7f994d600640 1 -- 172.21.15.7:0/3789726567 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940037040 con 0x7f995006d580 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.766+0000 7f994d600640 1 -- 172.21.15.7:0/3789726567 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9940004030 con 0x7f995006d580 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.766+0000 7f994d600640 1 -- 172.21.15.7:0/3789726567 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940006e10 con 0x7f995006d580 2024-04-25T11:23:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.766+0000 7f9956000640 1 -- 172.21.15.7:0/3789726567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9950150a90 con 0x7f995006d580 2024-04-25T11:23:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.766+0000 7f9956000640 1 -- 172.21.15.7:0/3789726567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99501544b0 con 0x7f995006d580 2024-04-25T11:23:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.767+0000 7f9956000640 1 -- 172.21.15.7:0/3789726567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9950050740 con 0x7f995006d580 2024-04-25T11:23:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.769+0000 7f994d600640 1 -- 172.21.15.7:0/3789726567 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f994003d650 con 0x7f995006d580 2024-04-25T11:23:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.770+0000 7f994d600640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f992c0744b0 0x7f992c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.770+0000 7f994d600640 1 -- 172.21.15.7:0/3789726567 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f99400b5ff0 con 0x7f995006d580 2024-04-25T11:23:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.770+0000 7f9947e00640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f992c0744b0 0x7f992c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.770+0000 7f994d600640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] conn(0x7f992c07a9a0 0x7f992c07ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.770+0000 7f994d600640 1 -- 172.21.15.7:0/3789726567 --> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f992c07d500 con 0x7f992c07a9a0 2024-04-25T11:23:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.770+0000 7f994d600640 1 -- 172.21.15.7:0/3789726567 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=159) ==== 24+0+0 (secure 0 0 0) 0x7f99400b63e0 con 0x7f995006d580 2024-04-25T11:23:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.770+0000 7f9955600640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] conn(0x7f992c07a9a0 0x7f992c07ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.771+0000 7f9947e00640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f992c0744b0 0x7f992c0769c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9938006fd0 tx=0x7f9938006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.771+0000 7f9955600640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] conn(0x7f992c07a9a0 0x7f992c07ce30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.772+0000 7f994d600640 1 -- 172.21.15.7:0/3789726567 <== osd.3 v2:172.21.15.7:6808/1382334542 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f992c07d500 con 0x7f992c07a9a0 2024-04-25T11:23:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.787+0000 7f9956000640 1 -- 172.21.15.7:0/3789726567 --> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9950071010 con 0x7f992c07a9a0 2024-04-25T11:23:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.787+0000 7f994d600640 1 -- 172.21.15.7:0/3789726567 <== osd.3 v2:172.21.15.7:6808/1382334542 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9950071010 con 0x7f992c07a9a0 2024-04-25T11:23:08.783 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:23:08.783 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:23:08.783 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:23:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.788+0000 7f9956000640 1 -- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] conn(0x7f992c07a9a0 msgr2=0x7f992c07ce30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.788+0000 7f9956000640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:6808/1382334542,v1:172.21.15.7:6809/1382334542] conn(0x7f992c07a9a0 0x7f992c07ce30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.788+0000 7f9956000640 1 -- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f992c0744b0 msgr2=0x7f992c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.788+0000 7f9956000640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f992c0744b0 0x7f992c0769c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9938006fd0 tx=0x7f9938006bc0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.788+0000 7f9956000640 1 -- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f995006d580 msgr2=0x7f995013fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.788+0000 7f9956000640 1 --2- 172.21.15.7:0/3789726567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f995006d580 0x7f995013fc30 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f9940035040 tx=0x7f99400046c0 comp rx=0 tx=0).stop 2024-04-25T11:23:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.788+0000 7f9955600640 1 -- 172.21.15.7:0/3789726567 reap_dead start 2024-04-25T11:23:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.788+0000 7f9956000640 1 -- 172.21.15.7:0/3789726567 shutdown_connections 2024-04-25T11:23:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.788+0000 7f9956000640 1 -- 172.21.15.7:0/3789726567 >> 172.21.15.7:0/3789726567 conn(0x7f9950073160 msgr2=0x7f99500b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.788+0000 7f9956000640 1 -- 172.21.15.7:0/3789726567 shutdown_connections 2024-04-25T11:23:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:08.788+0000 7f9956000640 1 -- 172.21.15.7:0/3789726567 wait complete. 2024-04-25T11:23:08.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T11:23:09.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.090+0000 7f4cd6600640 1 Processor -- start 2024-04-25T11:23:09.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.090+0000 7f4cd6600640 1 -- start start 2024-04-25T11:23:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.093+0000 7f4cd6600640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4cd0070520 0x7f4cd00b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.093+0000 7f4cd6600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cd0164160 0x7f4cd006c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.093+0000 7f4cd6600640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cd00b0e40 con 0x7f4cd0156d90 2024-04-25T11:23:09.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.093+0000 7f4cd6600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f4cd006f760 con 0x7f4cd0164160 2024-04-25T11:23:09.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.093+0000 7f4cd6600640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4cd006fa40 con 0x7f4cd0070520 2024-04-25T11:23:09.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.093+0000 7f4cd5c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cd0164160 0x7f4cd006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.093+0000 7f4ccfe00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4cd0156d90 0x7f4cd0163a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34564/0 (socket says 172.21.15.7:34564) 2024-04-25T11:23:09.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.093+0000 7f4ccfe00640 1 -- 172.21.15.7:0/776969718 learned_addr learned my addr 172.21.15.7:0/776969718 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:09.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.094+0000 7f4cd5200640 1 --2- 172.21.15.7:0/776969718 >> v2:172.21.15.87:3300/0 conn(0x7f4cd0070520 0x7f4cd00b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.094+0000 7f4cd5c00640 1 -- 172.21.15.7:0/776969718 >> v2:172.21.15.87:3300/0 conn(0x7f4cd0070520 msgr2=0x7f4cd00b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:09.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.094+0000 7f4cd5c00640 1 --2- 172.21.15.7:0/776969718 >> v2:172.21.15.87:3300/0 conn(0x7f4cd0070520 0x7f4cd00b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.094+0000 7f4cd5c00640 1 -- 172.21.15.7:0/776969718 >> v1:172.21.15.7:6789/0 conn(0x7f4cd0156d90 legacy=0x7f4cd0163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.094+0000 7f4cd5c00640 1 -- 172.21.15.7:0/776969718 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd006fd20 con 0x7f4cd0164160 2024-04-25T11:23:09.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.094+0000 7f4ccf400640 1 -- 172.21.15.7:0/776969718 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2426407351 0 0) 0x7f4cd00b0e40 con 0x7f4cd0156d90 2024-04-25T11:23:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.095+0000 7f4cd5c00640 1 --2- 172.21.15.7:0/776969718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd0164160 0x7f4cd006c570 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00725a0 tx=0x7f4cc802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e82942623bbc81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.095+0000 7f4ccf400640 1 -- 172.21.15.7:0/776969718 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8007740 con 0x7f4cd0164160 2024-04-25T11:23:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.095+0000 7f4ccf400640 1 -- 172.21.15.7:0/776969718 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc80078e0 con 0x7f4cd0164160 2024-04-25T11:23:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.095+0000 7f4ccf400640 1 -- 172.21.15.7:0/776969718 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8007cb0 con 0x7f4cd0164160 2024-04-25T11:23:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.096+0000 7f4cd6600640 1 -- 172.21.15.7:0/776969718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd0164160 msgr2=0x7f4cd006c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.096+0000 7f4cd6600640 1 --2- 172.21.15.7:0/776969718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd0164160 0x7f4cd006c570 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00725a0 tx=0x7f4cc802a5f0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.096+0000 7f4cd6600640 1 -- 172.21.15.7:0/776969718 shutdown_connections 2024-04-25T11:23:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.096+0000 7f4cd6600640 1 --2- 172.21.15.7:0/776969718 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd0164160 0x7f4cd006c570 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.096+0000 7f4cd6600640 1 --2- 172.21.15.7:0/776969718 >> v2:172.21.15.87:3300/0 conn(0x7f4cd0070520 0x7f4cd00b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.096+0000 7f4cd6600640 1 -- 172.21.15.7:0/776969718 >> 172.21.15.7:0/776969718 conn(0x7f4cd00d1540 msgr2=0x7f4cd00d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.096+0000 7f4cd6600640 1 -- 172.21.15.7:0/776969718 shutdown_connections 2024-04-25T11:23:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.096+0000 7f4cd6600640 1 -- 172.21.15.7:0/776969718 wait complete. 2024-04-25T11:23:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.099+0000 7f4cd6600640 1 Processor -- start 2024-04-25T11:23:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.099+0000 7f4cd6600640 1 -- start start 2024-04-25T11:23:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.100+0000 7f4cd6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd0070520 0x7f4cd01fa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.100+0000 7f4cd6600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cd0156d90 0x7f4cd014b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.100+0000 7f4cd6600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cd0164160 0x7f4cd00b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.100+0000 7f4cd6600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0070ab0 con 0x7f4cd0070520 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.100+0000 7f4cd6600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd0070930 con 0x7f4cd0164160 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.100+0000 7f4cd6600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd01571a0 con 0x7f4cd0156d90 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.101+0000 7f4cd5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd0070520 0x7f4cd01fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.101+0000 7f4cd5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd0070520 0x7f4cd01fa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48086/0 (socket says 172.21.15.7:48086) 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.101+0000 7f4cd5200640 1 -- 172.21.15.7:0/32574632 learned_addr learned my addr 172.21.15.7:0/32574632 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.101+0000 7f4cd5200640 1 -- 172.21.15.7:0/32574632 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cd0156d90 msgr2=0x7f4cd014b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.101+0000 7f4cd5200640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cd0156d90 0x7f4cd014b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.101+0000 7f4cd5200640 1 -- 172.21.15.7:0/32574632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cd0164160 msgr2=0x7f4cd00b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.101+0000 7f4cd5200640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cd0164160 0x7f4cd00b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.101+0000 7f4cd5200640 1 -- 172.21.15.7:0/32574632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd015af20 con 0x7f4cd0070520 2024-04-25T11:23:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.101+0000 7f4cd5200640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd0070520 0x7f4cd01fa210 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400d5f0 tx=0x7f4cc400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.102+0000 7f4cccc00640 1 -- 172.21.15.7:0/32574632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40125f0 con 0x7f4cd0070520 2024-04-25T11:23:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.102+0000 7f4cccc00640 1 -- 172.21.15.7:0/32574632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc4019070 con 0x7f4cd0070520 2024-04-25T11:23:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.102+0000 7f4cccc00640 1 -- 172.21.15.7:0/32574632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40129d0 con 0x7f4cd0070520 2024-04-25T11:23:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.102+0000 7f4cd6600640 1 -- 172.21.15.7:0/32574632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd015b200 con 0x7f4cd0070520 2024-04-25T11:23:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.102+0000 7f4cd6600640 1 -- 172.21.15.7:0/32574632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd015b7c0 con 0x7f4cd0070520 2024-04-25T11:23:09.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.103+0000 7f4cd6600640 1 -- 172.21.15.7:0/32574632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd0050740 con 0x7f4cd0070520 2024-04-25T11:23:09.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.104+0000 7f4cccc00640 1 -- 172.21.15.7:0/32574632 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4cc4012b70 con 0x7f4cd0070520 2024-04-25T11:23:09.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.105+0000 7f4cccc00640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cb00744b0 0x7f4cb00769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.105+0000 7f4cccc00640 1 -- 172.21.15.7:0/32574632 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f4cc4095500 con 0x7f4cd0070520 2024-04-25T11:23:09.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.106+0000 7f4ccfe00640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cb00744b0 0x7f4cb00769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.106+0000 7f4ccfe00640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cb00744b0 0x7f4cb00769c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc007fd0 tx=0x7f4cbc007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:09.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.110+0000 7f4cccc00640 1 -- 172.21.15.7:0/32574632 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4cc405fe00 con 0x7f4cd0070520 2024-04-25T11:23:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.307+0000 7f4cd6600640 1 -- 172.21.15.7:0/32574632 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f4cd015ab00 con 0x7f4cb00744b0 2024-04-25T11:23:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.308+0000 7f4cccc00640 1 -- 172.21.15.7:0/32574632 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4cd015ab00 con 0x7f4cb00744b0 2024-04-25T11:23:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T11:23:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.310+0000 7f4cd6600640 1 -- 172.21.15.7:0/32574632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cb00744b0 msgr2=0x7f4cb00769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.310+0000 7f4cd6600640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cb00744b0 0x7f4cb00769c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc007fd0 tx=0x7f4cbc007960 comp rx=0 tx=0).stop 2024-04-25T11:23:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.310+0000 7f4cd6600640 1 -- 172.21.15.7:0/32574632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd0070520 msgr2=0x7f4cd01fa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.310+0000 7f4cd6600640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd0070520 0x7f4cd01fa210 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400d5f0 tx=0x7f4cc400dab0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.310+0000 7f4cd6600640 1 -- 172.21.15.7:0/32574632 shutdown_connections 2024-04-25T11:23:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.310+0000 7f4cd6600640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4cb00744b0 0x7f4cb00769c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.310+0000 7f4cd6600640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cd0164160 0x7f4cd00b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.310+0000 7f4cd6600640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cd0156d90 0x7f4cd014b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.310+0000 7f4cd6600640 1 --2- 172.21.15.7:0/32574632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cd0070520 0x7f4cd01fa210 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.310+0000 7f4cd6600640 1 -- 172.21.15.7:0/32574632 >> 172.21.15.7:0/32574632 conn(0x7f4cd00d1540 msgr2=0x7f4cd0073bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:09.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.311+0000 7f4cd6600640 1 -- 172.21.15.7:0/32574632 shutdown_connections 2024-04-25T11:23:09.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.311+0000 7f4cd6600640 1 -- 172.21.15.7:0/32574632 wait complete. 2024-04-25T11:23:09.335 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T11:23:09.335 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:23:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.632+0000 7fa62f200640 1 Processor -- start 2024-04-25T11:23:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.632+0000 7fa62f200640 1 -- start start 2024-04-25T11:23:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.634+0000 7fa62f200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa6300b2930 0x7fa6300b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.634+0000 7fa62f200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa63006dc70 0x7fa630164c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.634+0000 7fa62f200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6300b8ba0 con 0x7fa63006d860 2024-04-25T11:23:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.634+0000 7fa62f200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa6300d2ae0 con 0x7fa63006dc70 2024-04-25T11:23:09.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.634+0000 7fa62f200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa63009cb40 con 0x7fa6300b2930 2024-04-25T11:23:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.635+0000 7fa62d400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa63006d860 0x7fa630164570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34580/0 (socket says 172.21.15.7:34580) 2024-04-25T11:23:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.635+0000 7fa62d400640 1 -- 172.21.15.7:0/1452606990 learned_addr learned my addr 172.21.15.7:0/1452606990 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.635+0000 7fa62e800640 1 --2- 172.21.15.7:0/1452606990 >> v2:172.21.15.7:3300/0 conn(0x7fa63006dc70 0x7fa630164c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.635+0000 7fa62de00640 1 --2- 172.21.15.7:0/1452606990 >> v2:172.21.15.87:3300/0 conn(0x7fa6300b2930 0x7fa6300b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.635+0000 7fa62ca00640 1 -- 172.21.15.7:0/1452606990 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3598897618 0 0) 0x7fa61c001110 con 0x7fa63006d860 2024-04-25T11:23:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.635+0000 7fa62ca00640 1 -- 172.21.15.7:0/1452606990 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa614003520 con 0x7fa63006d860 2024-04-25T11:23:09.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.635+0000 7fa62e800640 1 -- 172.21.15.7:0/1452606990 >> v2:172.21.15.87:3300/0 conn(0x7fa6300b2930 msgr2=0x7fa6300b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.635+0000 7fa62e800640 1 --2- 172.21.15.7:0/1452606990 >> v2:172.21.15.87:3300/0 conn(0x7fa6300b2930 0x7fa6300b8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.635+0000 7fa62e800640 1 -- 172.21.15.7:0/1452606990 >> v1:172.21.15.7:6789/0 conn(0x7fa63006d860 legacy=0x7fa630164570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.636+0000 7fa62e800640 1 -- 172.21.15.7:0/1452606990 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6300d1110 con 0x7fa63006dc70 2024-04-25T11:23:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.637+0000 7fa62e800640 1 --2- 172.21.15.7:0/1452606990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63006dc70 0x7fa630164c80 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fa63008cd00 tx=0x7fa62802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6b02e4996a7469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.637+0000 7fa62ca00640 1 -- 172.21.15.7:0/1452606990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628007740 con 0x7fa63006dc70 2024-04-25T11:23:09.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.637+0000 7fa62ca00640 1 -- 172.21.15.7:0/1452606990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6280078e0 con 0x7fa63006dc70 2024-04-25T11:23:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.637+0000 7fa62ca00640 1 -- 172.21.15.7:0/1452606990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628007c90 con 0x7fa63006dc70 2024-04-25T11:23:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.638+0000 7fa62f200640 1 -- 172.21.15.7:0/1452606990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63006dc70 msgr2=0x7fa630164c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.638+0000 7fa62f200640 1 --2- 172.21.15.7:0/1452606990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63006dc70 0x7fa630164c80 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fa63008cd00 tx=0x7fa62802a5f0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.638+0000 7fa62f200640 1 -- 172.21.15.7:0/1452606990 shutdown_connections 2024-04-25T11:23:09.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.638+0000 7fa62f200640 1 --2- 172.21.15.7:0/1452606990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63006dc70 0x7fa630164c80 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.638+0000 7fa62f200640 1 --2- 172.21.15.7:0/1452606990 >> v2:172.21.15.87:3300/0 conn(0x7fa6300b2930 0x7fa6300b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.638+0000 7fa62f200640 1 -- 172.21.15.7:0/1452606990 >> 172.21.15.7:0/1452606990 conn(0x7fa6300d01e0 msgr2=0x7fa6300d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:09.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.638+0000 7fa62f200640 1 -- 172.21.15.7:0/1452606990 shutdown_connections 2024-04-25T11:23:09.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.638+0000 7fa62f200640 1 -- 172.21.15.7:0/1452606990 wait complete. 2024-04-25T11:23:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.640+0000 7fa62f200640 1 Processor -- start 2024-04-25T11:23:09.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.640+0000 7fa62f200640 1 -- start start 2024-04-25T11:23:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.642+0000 7fa62f200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa63006d860 0x7fa6301fa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.642+0000 7fa62f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63006dc70 0x7fa6301594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.642+0000 7fa62f200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6300b2930 0x7fa63014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.642+0000 7fa62f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6300d2ff0 con 0x7fa63006dc70 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.642+0000 7fa62f200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa6300d2d10 con 0x7fa63006d860 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.642+0000 7fa62f200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa63009cfd0 con 0x7fa6300b2930 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.642+0000 7fa62d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63006dc70 0x7fa6301594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.642+0000 7fa62d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63006dc70 0x7fa6301594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48092/0 (socket says 172.21.15.7:48092) 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.642+0000 7fa62d400640 1 -- 172.21.15.7:0/360939349 learned_addr learned my addr 172.21.15.7:0/360939349 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.643+0000 7fa62de00640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa63006d860 0x7fa6301fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.643+0000 7fa62e800640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6300b2930 0x7fa63014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.643+0000 7fa62d400640 1 -- 172.21.15.7:0/360939349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6300b2930 msgr2=0x7fa63014c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.643+0000 7fa62d400640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6300b2930 0x7fa63014c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.643+0000 7fa62d400640 1 -- 172.21.15.7:0/360939349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa63006d860 msgr2=0x7fa6301fa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.643+0000 7fa62d400640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa63006d860 0x7fa6301fa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.643+0000 7fa62d400640 1 -- 172.21.15.7:0/360939349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6300b12b0 con 0x7fa63006dc70 2024-04-25T11:23:09.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.643+0000 7fa62d400640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63006dc70 0x7fa6301594d0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00c3d0 tx=0x7fa61c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:09.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.643+0000 7fa626000640 1 -- 172.21.15.7:0/360939349 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c017070 con 0x7fa63006dc70 2024-04-25T11:23:09.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.644+0000 7fa626000640 1 -- 172.21.15.7:0/360939349 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c005ce0 con 0x7fa63006dc70 2024-04-25T11:23:09.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.644+0000 7fa62f200640 1 -- 172.21.15.7:0/360939349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6300b1590 con 0x7fa63006dc70 2024-04-25T11:23:09.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.644+0000 7fa626000640 1 -- 172.21.15.7:0/360939349 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c007310 con 0x7fa63006dc70 2024-04-25T11:23:09.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.644+0000 7fa62f200640 1 -- 172.21.15.7:0/360939349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6300b1b50 con 0x7fa63006dc70 2024-04-25T11:23:09.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.645+0000 7fa62f200640 1 -- 172.21.15.7:0/360939349 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa630050740 con 0x7fa63006dc70 2024-04-25T11:23:09.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.646+0000 7fa626000640 1 -- 172.21.15.7:0/360939349 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa61c01a070 con 0x7fa63006dc70 2024-04-25T11:23:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.648+0000 7fa626000640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6100744b0 0x7fa6100769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.648+0000 7fa626000640 1 -- 172.21.15.7:0/360939349 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7fa61c094400 con 0x7fa63006dc70 2024-04-25T11:23:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.648+0000 7fa62de00640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6100744b0 0x7fa6100769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.648+0000 7fa626000640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] conn(0x7fa61007a9a0 0x7fa61007ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.648+0000 7fa626000640 1 -- 172.21.15.7:0/360939349 --> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa61007d500 con 0x7fa61007a9a0 2024-04-25T11:23:09.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.648+0000 7fa626000640 1 -- 172.21.15.7:0/360939349 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=159) ==== 24+0+0 (secure 0 0 0) 0x7fa61c09b050 con 0x7fa63006dc70 2024-04-25T11:23:09.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.649+0000 7fa62de00640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6100744b0 0x7fa6100769c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa62000b9f0 tx=0x7fa620008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:09.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.649+0000 7fa62e800640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] conn(0x7fa61007a9a0 0x7fa61007ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.650+0000 7fa62e800640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] conn(0x7fa61007a9a0 0x7fa61007ce30 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:09.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.652+0000 7fa626000640 1 -- 172.21.15.7:0/360939349 <== osd.4 v2:172.21.15.87:6808/1933779372 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa61007d500 con 0x7fa61007a9a0 2024-04-25T11:23:09.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.667+0000 7fa62f200640 1 -- 172.21.15.7:0/360939349 --> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa6300b2710 con 0x7fa61007a9a0 2024-04-25T11:23:09.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.668+0000 7fa626000640 1 -- 172.21.15.7:0/360939349 <== osd.4 v2:172.21.15.87:6808/1933779372 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa6300b2710 con 0x7fa61007a9a0 2024-04-25T11:23:09.664 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:23:09.664 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:23:09.664 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:23:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.669+0000 7fa62f200640 1 -- 172.21.15.7:0/360939349 >> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] conn(0x7fa61007a9a0 msgr2=0x7fa61007ce30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.669+0000 7fa62f200640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.87:6808/1933779372,v1:172.21.15.87:6809/1933779372] conn(0x7fa61007a9a0 0x7fa61007ce30 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.669+0000 7fa62f200640 1 -- 172.21.15.7:0/360939349 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6100744b0 msgr2=0x7fa6100769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.669+0000 7fa62f200640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa6100744b0 0x7fa6100769c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa62000b9f0 tx=0x7fa620008040 comp rx=0 tx=0).stop 2024-04-25T11:23:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.669+0000 7fa62f200640 1 -- 172.21.15.7:0/360939349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63006dc70 msgr2=0x7fa6301594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.669+0000 7fa62f200640 1 --2- 172.21.15.7:0/360939349 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa63006dc70 0x7fa6301594d0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00c3d0 tx=0x7fa61c00c890 comp rx=0 tx=0).stop 2024-04-25T11:23:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.669+0000 7fa62e800640 1 -- 172.21.15.7:0/360939349 reap_dead start 2024-04-25T11:23:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.669+0000 7fa62f200640 1 -- 172.21.15.7:0/360939349 shutdown_connections 2024-04-25T11:23:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.669+0000 7fa62f200640 1 -- 172.21.15.7:0/360939349 >> 172.21.15.7:0/360939349 conn(0x7fa6300d01e0 msgr2=0x7fa6300720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.669+0000 7fa62f200640 1 -- 172.21.15.7:0/360939349 shutdown_connections 2024-04-25T11:23:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.669+0000 7fa62f200640 1 -- 172.21.15.7:0/360939349 wait complete. 2024-04-25T11:23:09.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T11:23:09.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.983+0000 7f9066800640 1 Processor -- start 2024-04-25T11:23:09.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.983+0000 7f9066800640 1 -- start start 2024-04-25T11:23:09.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.985+0000 7f9066800640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f906015cdb0 0x7f90600b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.985+0000 7f9066800640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f90600b7960 0x7f90600ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.985+0000 7f9066800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90600b7d70 con 0x7f90600cf7d0 2024-04-25T11:23:09.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.985+0000 7f9066800640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f90600ceed0 con 0x7f906015cdb0 2024-04-25T11:23:09.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.985+0000 7f9066800640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f90600cf490 con 0x7f90600b7960 2024-04-25T11:23:09.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.986+0000 7f9064a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f906015cdb0 0x7f90600b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.986+0000 7f9064a00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f906015cdb0 0x7f90600b7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40740/0 (socket says 172.21.15.7:40740) 2024-04-25T11:23:09.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.986+0000 7f9064a00640 1 -- 172.21.15.7:0/2744309690 learned_addr learned my addr 172.21.15.7:0/2744309690 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:09.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.986+0000 7f9065e00640 1 --2- 172.21.15.7:0/2744309690 >> v2:172.21.15.87:3300/0 conn(0x7f90600b7960 0x7f90600ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.986+0000 7f9064a00640 1 -- 172.21.15.7:0/2744309690 >> v2:172.21.15.87:3300/0 conn(0x7f90600b7960 msgr2=0x7f90600ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:09.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.986+0000 7f9064a00640 1 --2- 172.21.15.7:0/2744309690 >> v2:172.21.15.87:3300/0 conn(0x7f90600b7960 0x7f90600ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.986+0000 7f9064a00640 1 -- 172.21.15.7:0/2744309690 >> v1:172.21.15.87:6789/0 conn(0x7f90600cf7d0 legacy=0x7f906006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:09.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.986+0000 7f9064a00640 1 -- 172.21.15.7:0/2744309690 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906006e210 con 0x7f906015cdb0 2024-04-25T11:23:09.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.986+0000 7f9065e00640 1 --2- 172.21.15.7:0/2744309690 >> v2:172.21.15.87:3300/0 conn(0x7f90600b7960 0x7f90600ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T11:23:09.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.988+0000 7f9064a00640 1 --2- 172.21.15.7:0/2744309690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f906015cdb0 0x7f90600b7420 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f90600d2240 tx=0x7f904c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918bfeac35cc028e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:09.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.988+0000 7f905fe00640 1 -- 172.21.15.7:0/2744309690 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c037070 con 0x7f906015cdb0 2024-04-25T11:23:09.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.988+0000 7f905fe00640 1 -- 172.21.15.7:0/2744309690 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f904c032440 con 0x7f906015cdb0 2024-04-25T11:23:09.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.988+0000 7f905fe00640 1 -- 172.21.15.7:0/2744309690 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904c032810 con 0x7f906015cdb0 2024-04-25T11:23:09.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.989+0000 7f9066800640 1 -- 172.21.15.7:0/2744309690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f906015cdb0 msgr2=0x7f90600b7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.989+0000 7f9066800640 1 --2- 172.21.15.7:0/2744309690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f906015cdb0 0x7f90600b7420 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f90600d2240 tx=0x7f904c02a270 comp rx=0 tx=0).stop 2024-04-25T11:23:09.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.989+0000 7f9066800640 1 -- 172.21.15.7:0/2744309690 shutdown_connections 2024-04-25T11:23:09.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.989+0000 7f9066800640 1 --2- 172.21.15.7:0/2744309690 >> v2:172.21.15.87:3300/0 conn(0x7f90600b7960 0x7f90600ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.989+0000 7f9066800640 1 --2- 172.21.15.7:0/2744309690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f906015cdb0 0x7f90600b7420 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.989+0000 7f9066800640 1 -- 172.21.15.7:0/2744309690 >> 172.21.15.7:0/2744309690 conn(0x7f906009b540 msgr2=0x7f906009b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:09.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.989+0000 7f9066800640 1 -- 172.21.15.7:0/2744309690 shutdown_connections 2024-04-25T11:23:09.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.989+0000 7f9066800640 1 -- 172.21.15.7:0/2744309690 wait complete. 2024-04-25T11:23:09.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.991+0000 7f9066800640 1 Processor -- start 2024-04-25T11:23:09.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.992+0000 7f9066800640 1 -- start start 2024-04-25T11:23:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.993+0000 7f9066800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90600b7960 0x7f906015b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.993+0000 7f9066800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90600cf7d0 0x7f906014d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.993+0000 7f9066800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f906015cdb0 0x7f906014fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.993+0000 7f9066800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f906015d1c0 con 0x7f90600b7960 2024-04-25T11:23:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.993+0000 7f9066800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f90600cf120 con 0x7f90600cf7d0 2024-04-25T11:23:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.993+0000 7f9066800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f90600cd450 con 0x7f906015cdb0 2024-04-25T11:23:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.993+0000 7f9065400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90600b7960 0x7f906015b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.994+0000 7f9065400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90600b7960 0x7f906015b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40748/0 (socket says 172.21.15.7:40748) 2024-04-25T11:23:09.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.994+0000 7f9065400640 1 -- 172.21.15.7:0/4255818300 learned_addr learned my addr 172.21.15.7:0/4255818300 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.994+0000 7f9064a00640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90600cf7d0 0x7f906014d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.994+0000 7f9065e00640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f906015cdb0 0x7f906014fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.994+0000 7f9065400640 1 -- 172.21.15.7:0/4255818300 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f906015cdb0 msgr2=0x7f906014fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.994+0000 7f9065400640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f906015cdb0 0x7f906014fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.994+0000 7f9065400640 1 -- 172.21.15.7:0/4255818300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90600cf7d0 msgr2=0x7f906014d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.994+0000 7f9065400640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90600cf7d0 0x7f906014d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.994+0000 7f9065400640 1 -- 172.21.15.7:0/4255818300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906015bd20 con 0x7f90600b7960 2024-04-25T11:23:09.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.995+0000 7f9065400640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90600b7960 0x7f906015b7e0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f905400c3c0 tx=0x7f905400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:09.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.995+0000 7f905d600640 1 -- 172.21.15.7:0/4255818300 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90540052d0 con 0x7f90600b7960 2024-04-25T11:23:09.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.995+0000 7f905d600640 1 -- 172.21.15.7:0/4255818300 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9054005490 con 0x7f90600b7960 2024-04-25T11:23:09.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.995+0000 7f9066800640 1 -- 172.21.15.7:0/4255818300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90600d0330 con 0x7f90600b7960 2024-04-25T11:23:09.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.995+0000 7f905d600640 1 -- 172.21.15.7:0/4255818300 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054007260 con 0x7f90600b7960 2024-04-25T11:23:09.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.995+0000 7f9066800640 1 -- 172.21.15.7:0/4255818300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f906015afe0 con 0x7f90600b7960 2024-04-25T11:23:09.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.997+0000 7f9066800640 1 -- 172.21.15.7:0/4255818300 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f906009c0a0 con 0x7f90600b7960 2024-04-25T11:23:09.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:09.998+0000 7f905d600640 1 -- 172.21.15.7:0/4255818300 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9054007400 con 0x7f90600b7960 2024-04-25T11:23:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.000+0000 7f905d600640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90400744b0 0x7f90400769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:09.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.000+0000 7f905d600640 1 -- 172.21.15.7:0/4255818300 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f9054094620 con 0x7f90600b7960 2024-04-25T11:23:09.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.000+0000 7f9064a00640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90400744b0 0x7f90400769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:09.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.001+0000 7f9064a00640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90400744b0 0x7f90400769c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f904c034000 tx=0x7f904c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:10.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.005+0000 7f905d600640 1 -- 172.21.15.7:0/4255818300 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f906009c0a0 con 0x7f90600b7960 2024-04-25T11:23:10.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.210+0000 7f9066800640 1 -- 172.21.15.7:0/4255818300 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f9060050740 con 0x7f90400744b0 2024-04-25T11:23:10.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.211+0000 7f905d600640 1 -- 172.21.15.7:0/4255818300 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9060050740 con 0x7f90400744b0 2024-04-25T11:23:10.206 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T11:23:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 -- 172.21.15.7:0/4255818300 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90400744b0 msgr2=0x7f90400769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90400744b0 0x7f90400769c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f904c034000 tx=0x7f904c02e000 comp rx=0 tx=0).stop 2024-04-25T11:23:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 -- 172.21.15.7:0/4255818300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90600b7960 msgr2=0x7f906015b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90600b7960 0x7f906015b7e0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f905400c3c0 tx=0x7f905400c880 comp rx=0 tx=0).stop 2024-04-25T11:23:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 -- 172.21.15.7:0/4255818300 shutdown_connections 2024-04-25T11:23:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f90400744b0 0x7f90400769c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f906015cdb0 0x7f906014fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90600cf7d0 0x7f906014d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 --2- 172.21.15.7:0/4255818300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90600b7960 0x7f906015b7e0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 -- 172.21.15.7:0/4255818300 >> 172.21.15.7:0/4255818300 conn(0x7f906009b540 msgr2=0x7f90600ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:10.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 -- 172.21.15.7:0/4255818300 shutdown_connections 2024-04-25T11:23:10.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.214+0000 7f9066800640 1 -- 172.21.15.7:0/4255818300 wait complete. 2024-04-25T11:23:10.238 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T11:23:10.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:23:10.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.536+0000 7f4214a00640 1 Processor -- start 2024-04-25T11:23:10.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.536+0000 7f4214a00640 1 -- start start 2024-04-25T11:23:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.539+0000 7f4214a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f421015de20 0x7f42100b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.539+0000 7f4214a00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42100b6400 con 0x7f42100b5ff0 2024-04-25T11:23:10.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.539+0000 7f4214a00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42100afcb0 con 0x7f42100ae880 2024-04-25T11:23:10.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.539+0000 7f4214a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f42100aed60 con 0x7f421015de20 2024-04-25T11:23:10.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.539+0000 7f420e800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f42100b5ff0 0x7f421006f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58582/0 (socket says 172.21.15.7:58582) 2024-04-25T11:23:10.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.539+0000 7f420e800640 1 -- 172.21.15.7:0/271544363 learned_addr learned my addr 172.21.15.7:0/271544363 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:10.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.540+0000 7f420d400640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f421015de20 0x7f42100b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.540+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2973573499 0 0) 0x7f4208001110 con 0x7f42100b5ff0 2024-04-25T11:23:10.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.540+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41f4003520 con 0x7f42100b5ff0 2024-04-25T11:23:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.540+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3156331697 0 0) 0x7f41f4003520 con 0x7f42100b5ff0 2024-04-25T11:23:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.540+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f421015de20 msgr2=0x7f42100b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.540+0000 7f420ca00640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f421015de20 0x7f42100b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.540+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 >> v1:172.21.15.87:6789/0 conn(0x7f42100ae880 legacy=0x7f42100cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.540+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42100af040 con 0x7f42100b5ff0 2024-04-25T11:23:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.540+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f4208001840 con 0x7f42100b5ff0 2024-04-25T11:23:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.540+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 >> v1:172.21.15.7:6789/0 conn(0x7f42100b5ff0 legacy=0x7f421006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.540+0000 7f420ca00640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f40039e0 0x7f41f4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420ca00640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41f4004800 0x7f41f4004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420ca00640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f421015de20 0x7f41f4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42100aed60 con 0x7f41f40039e0 2024-04-25T11:23:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f41f400baf0 con 0x7f41f4004800 2024-04-25T11:23:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f41f400bcf0 con 0x7f421015de20 2024-04-25T11:23:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420de00640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f40039e0 0x7f41f4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420e800640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f421015de20 0x7f41f4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420d400640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41f4004800 0x7f41f4004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420de00640 1 -- 172.21.15.7:0/271544363 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f421015de20 msgr2=0x7f41f4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420de00640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f421015de20 0x7f41f4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420de00640 1 -- 172.21.15.7:0/271544363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41f4004800 msgr2=0x7f41f4004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420de00640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41f4004800 0x7f41f4004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.541+0000 7f420de00640 1 -- 172.21.15.7:0/271544363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f4003520 con 0x7f41f40039e0 2024-04-25T11:23:10.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.543+0000 7f420de00640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f40039e0 0x7f41f4003e50 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f4200001e20 tx=0x7f41f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec87649a27a1a745 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:10.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.543+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200001a70 con 0x7f41f40039e0 2024-04-25T11:23:10.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.543+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420002ae40 con 0x7f41f40039e0 2024-04-25T11:23:10.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.543+0000 7f420ca00640 1 -- 172.21.15.7:0/271544363 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200003bb0 con 0x7f41f40039e0 2024-04-25T11:23:10.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.544+0000 7f4214a00640 1 -- 172.21.15.7:0/271544363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f40039e0 msgr2=0x7f41f4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.544+0000 7f4214a00640 1 --2- 172.21.15.7:0/271544363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41f40039e0 0x7f41f4003e50 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f4200001e20 tx=0x7f41f4005830 comp rx=0 tx=0).stop 2024-04-25T11:23:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.544+0000 7f420e800640 1 -- 172.21.15.7:0/271544363 reap_dead start 2024-04-25T11:23:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.544+0000 7f4214a00640 1 -- 172.21.15.7:0/271544363 shutdown_connections 2024-04-25T11:23:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.544+0000 7f4214a00640 1 -- 172.21.15.7:0/271544363 >> 172.21.15.7:0/271544363 conn(0x7f4210073920 msgr2=0x7f4210073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.544+0000 7f4214a00640 1 -- 172.21.15.7:0/271544363 shutdown_connections 2024-04-25T11:23:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.544+0000 7f4214a00640 1 -- 172.21.15.7:0/271544363 wait complete. 2024-04-25T11:23:10.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.546+0000 7f4214a00640 1 Processor -- start 2024-04-25T11:23:10.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.546+0000 7f4214a00640 1 -- start start 2024-04-25T11:23:10.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.548+0000 7f4214a00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41f40039e0 0x7f421014ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.548+0000 7f4214a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42100b24e0 0x7f42100b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.548+0000 7f4214a00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f421015a0b0 0x7f421015a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.548+0000 7f4214a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f421015e230 con 0x7f42100b24e0 2024-04-25T11:23:10.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.548+0000 7f4214a00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f42100af600 con 0x7f421015a0b0 2024-04-25T11:23:10.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.548+0000 7f4214a00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f42100714b0 con 0x7f41f40039e0 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.548+0000 7f420d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42100b24e0 0x7f42100b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.548+0000 7f420d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42100b24e0 0x7f42100b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40760/0 (socket says 172.21.15.7:40760) 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.548+0000 7f420d400640 1 -- 172.21.15.7:0/96256198 learned_addr learned my addr 172.21.15.7:0/96256198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f420de00640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41f40039e0 0x7f421014ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f420e800640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f421015a0b0 0x7f421015a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f420d400640 1 -- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41f40039e0 msgr2=0x7f421014ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f420d400640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41f40039e0 0x7f421014ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f420d400640 1 -- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f421015a0b0 msgr2=0x7f421015a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f420d400640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f421015a0b0 0x7f421015a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f420d400640 1 -- 172.21.15.7:0/96256198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42100b1440 con 0x7f42100b24e0 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f420e800640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f421015a0b0 0x7f421015a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:10.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f420d400640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42100b24e0 0x7f42100b28f0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f41f8011600 tx=0x7f41f8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:10.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f4206000640 1 -- 172.21.15.7:0/96256198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f800f920 con 0x7f42100b24e0 2024-04-25T11:23:10.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f4206000640 1 -- 172.21.15.7:0/96256198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f800fac0 con 0x7f42100b24e0 2024-04-25T11:23:10.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.549+0000 7f4214a00640 1 -- 172.21.15.7:0/96256198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42100b1720 con 0x7f42100b24e0 2024-04-25T11:23:10.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.550+0000 7f4206000640 1 -- 172.21.15.7:0/96256198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f800fe70 con 0x7f42100b24e0 2024-04-25T11:23:10.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.550+0000 7f4214a00640 1 -- 172.21.15.7:0/96256198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42100b1ce0 con 0x7f42100b24e0 2024-04-25T11:23:10.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.551+0000 7f4214a00640 1 -- 172.21.15.7:0/96256198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f421006a100 con 0x7f42100b24e0 2024-04-25T11:23:10.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.552+0000 7f4206000640 1 -- 172.21.15.7:0/96256198 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f41f8015460 con 0x7f42100b24e0 2024-04-25T11:23:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.553+0000 7f4206000640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f41ec0744b0 0x7f41ec0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.554+0000 7f4206000640 1 -- 172.21.15.7:0/96256198 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f41f8095ae0 con 0x7f42100b24e0 2024-04-25T11:23:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.554+0000 7f420de00640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f41ec0744b0 0x7f41ec0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.554+0000 7f4206000640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] conn(0x7f41ec07a9a0 0x7f41ec07ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.554+0000 7f4206000640 1 -- 172.21.15.7:0/96256198 --> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41f8012e90 con 0x7f41ec07a9a0 2024-04-25T11:23:10.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.554+0000 7f4206000640 1 -- 172.21.15.7:0/96256198 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=159) ==== 24+0+0 (secure 0 0 0) 0x7f41f8095e90 con 0x7f42100b24e0 2024-04-25T11:23:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.555+0000 7f420e800640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] conn(0x7f41ec07a9a0 0x7f41ec07ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.555+0000 7f420de00640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f41ec0744b0 0x7f41ec0769c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4200032000 tx=0x7f420002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:10.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.556+0000 7f420e800640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] conn(0x7f41ec07a9a0 0x7f41ec07ce30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:10.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.558+0000 7f4206000640 1 -- 172.21.15.7:0/96256198 <== osd.5 v2:172.21.15.87:6816/2772334579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41f8012e90 con 0x7f41ec07a9a0 2024-04-25T11:23:10.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.573+0000 7f4214a00640 1 -- 172.21.15.7:0/96256198 --> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f421015ac20 con 0x7f41ec07a9a0 2024-04-25T11:23:10.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.574+0000 7f4206000640 1 -- 172.21.15.7:0/96256198 <== osd.5 v2:172.21.15.87:6816/2772334579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f421015ac20 con 0x7f41ec07a9a0 2024-04-25T11:23:10.570 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:23:10.570 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:23:10.570 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.575+0000 7f4214a00640 1 -- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] conn(0x7f41ec07a9a0 msgr2=0x7f41ec07ce30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.575+0000 7f4214a00640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.87:6816/2772334579,v1:172.21.15.87:6817/2772334579] conn(0x7f41ec07a9a0 0x7f41ec07ce30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.575+0000 7f4214a00640 1 -- 172.21.15.7:0/96256198 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f41ec0744b0 msgr2=0x7f41ec0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.575+0000 7f4214a00640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f41ec0744b0 0x7f41ec0769c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4200032000 tx=0x7f420002e000 comp rx=0 tx=0).stop 2024-04-25T11:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.576+0000 7f4214a00640 1 -- 172.21.15.7:0/96256198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42100b24e0 msgr2=0x7f42100b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.576+0000 7f4214a00640 1 --2- 172.21.15.7:0/96256198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42100b24e0 0x7f42100b28f0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f41f8011600 tx=0x7f41f8011ac0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.576+0000 7f420e800640 1 -- 172.21.15.7:0/96256198 reap_dead start 2024-04-25T11:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.576+0000 7f4214a00640 1 -- 172.21.15.7:0/96256198 shutdown_connections 2024-04-25T11:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.576+0000 7f4214a00640 1 -- 172.21.15.7:0/96256198 >> 172.21.15.7:0/96256198 conn(0x7f4210073920 msgr2=0x7f421015fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.576+0000 7f4214a00640 1 -- 172.21.15.7:0/96256198 shutdown_connections 2024-04-25T11:23:10.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.576+0000 7f4214a00640 1 -- 172.21.15.7:0/96256198 wait complete. 2024-04-25T11:23:10.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T11:23:10.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.893+0000 7fe1c2000640 1 Processor -- start 2024-04-25T11:23:10.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.893+0000 7fe1c2000640 1 -- start start 2024-04-25T11:23:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.896+0000 7fe1c2000640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe1bc156be0 0x7fe1bc06ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.896+0000 7fe1c2000640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1bc0cf1a0 con 0x7fe1bc06d340 2024-04-25T11:23:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.896+0000 7fe1c2000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1bc06d750 con 0x7fe1bc0cfb30 2024-04-25T11:23:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.896+0000 7fe1c2000640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fe1bc070fc0 con 0x7fe1bc156be0 2024-04-25T11:23:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.896+0000 7fe1bbe00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe1bc156be0 0x7fe1bc06ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.896+0000 7fe1c1600640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe1bc06d340 0x7fe1bc0b5670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58594/0 (socket says 172.21.15.7:58594) 2024-04-25T11:23:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.896+0000 7fe1c1600640 1 -- 172.21.15.7:0/3544763585 learned_addr learned my addr 172.21.15.7:0/3544763585 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.896+0000 7fe1bbe00640 1 --2- 172.21.15.7:0/3544763585 >> v2:172.21.15.7:3300/0 conn(0x7fe1bc156be0 0x7fe1bc06ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40772/0 (socket says 172.21.15.7:40772) 2024-04-25T11:23:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.897+0000 7fe1bbe00640 1 -- 172.21.15.7:0/3544763585 >> v1:172.21.15.87:6789/0 conn(0x7fe1bc0cfb30 legacy=0x7fe1bc15d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.897+0000 7fe1bbe00640 1 -- 172.21.15.7:0/3544763585 >> v1:172.21.15.7:6789/0 conn(0x7fe1bc06d340 legacy=0x7fe1bc0b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.897+0000 7fe1bbe00640 1 -- 172.21.15.7:0/3544763585 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc0b4c10 con 0x7fe1bc156be0 2024-04-25T11:23:10.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.897+0000 7fe1bb400640 1 -- 172.21.15.7:0/3544763585 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2373937004 0 0) 0x7fe1b4001490 con 0x7fe1bc06d340 2024-04-25T11:23:10.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.898+0000 7fe1bbe00640 1 --2- 172.21.15.7:0/3544763585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc156be0 0x7fe1bc06ce00 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80097a0 tx=0x7fe1a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a7543e707dbc15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:10.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.898+0000 7fe1bb400640 1 -- 172.21.15.7:0/3544763585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8037070 con 0x7fe1bc156be0 2024-04-25T11:23:10.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.898+0000 7fe1bb400640 1 -- 172.21.15.7:0/3544763585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a8032440 con 0x7fe1bc156be0 2024-04-25T11:23:10.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.898+0000 7fe1bb400640 1 -- 172.21.15.7:0/3544763585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8032810 con 0x7fe1bc156be0 2024-04-25T11:23:10.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.899+0000 7fe1c2000640 1 -- 172.21.15.7:0/3544763585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc156be0 msgr2=0x7fe1bc06ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.899+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3544763585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc156be0 0x7fe1bc06ce00 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80097a0 tx=0x7fe1a802a230 comp rx=0 tx=0).stop 2024-04-25T11:23:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.899+0000 7fe1c2000640 1 -- 172.21.15.7:0/3544763585 shutdown_connections 2024-04-25T11:23:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.899+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3544763585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc156be0 0x7fe1bc06ce00 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.899+0000 7fe1c2000640 1 -- 172.21.15.7:0/3544763585 >> 172.21.15.7:0/3544763585 conn(0x7fe1bc0d1540 msgr2=0x7fe1bc0d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.899+0000 7fe1c2000640 1 -- 172.21.15.7:0/3544763585 shutdown_connections 2024-04-25T11:23:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.899+0000 7fe1c2000640 1 -- 172.21.15.7:0/3544763585 wait complete. 2024-04-25T11:23:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.902+0000 7fe1c2000640 1 Processor -- start 2024-04-25T11:23:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.902+0000 7fe1c2000640 1 -- start start 2024-04-25T11:23:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.903+0000 7fe1c2000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1bc06d340 0x7fe1bc14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.903+0000 7fe1c2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc156be0 0x7fe1bc152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.903+0000 7fe1c2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1bc151060 0x7fe1bc152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.903+0000 7fe1c2000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc0cfff0 con 0x7fe1bc156be0 2024-04-25T11:23:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.903+0000 7fe1c2000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc0b4eb0 con 0x7fe1bc151060 2024-04-25T11:23:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.903+0000 7fe1c2000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe1bc0d02d0 con 0x7fe1bc06d340 2024-04-25T11:23:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc156be0 0x7fe1bc152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc156be0 0x7fe1bc152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40784/0 (socket says 172.21.15.7:40784) 2024-04-25T11:23:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1bbe00640 1 -- 172.21.15.7:0/3135130270 learned_addr learned my addr 172.21.15.7:0/3135130270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1c0c00640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1bc06d340 0x7fe1bc14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1c1600640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1bc151060 0x7fe1bc152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1bbe00640 1 -- 172.21.15.7:0/3135130270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1bc06d340 msgr2=0x7fe1bc14c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1bbe00640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1bc06d340 0x7fe1bc14c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1bbe00640 1 -- 172.21.15.7:0/3135130270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1bc151060 msgr2=0x7fe1bc152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1bbe00640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1bc151060 0x7fe1bc152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1bbe00640 1 -- 172.21.15.7:0/3135130270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc1517d0 con 0x7fe1bc156be0 2024-04-25T11:23:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1c0c00640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1bc06d340 0x7fe1bc14c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.904+0000 7fe1c1600640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1bc151060 0x7fe1bc152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.905+0000 7fe1bbe00640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc156be0 0x7fe1bc152980 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80097a0 tx=0x7fe1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.905+0000 7fe1b8c00640 1 -- 172.21.15.7:0/3135130270 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8041070 con 0x7fe1bc156be0 2024-04-25T11:23:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.905+0000 7fe1b8c00640 1 -- 172.21.15.7:0/3135130270 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a803c870 con 0x7fe1bc156be0 2024-04-25T11:23:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.905+0000 7fe1b8c00640 1 -- 172.21.15.7:0/3135130270 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8037040 con 0x7fe1bc156be0 2024-04-25T11:23:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.905+0000 7fe1c2000640 1 -- 172.21.15.7:0/3135130270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc151ab0 con 0x7fe1bc156be0 2024-04-25T11:23:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.905+0000 7fe1c2000640 1 -- 172.21.15.7:0/3135130270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc14e7a0 con 0x7fe1bc156be0 2024-04-25T11:23:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.907+0000 7fe1c2000640 1 -- 172.21.15.7:0/3135130270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1bc151540 con 0x7fe1bc156be0 2024-04-25T11:23:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.908+0000 7fe1b8c00640 1 -- 172.21.15.7:0/3135130270 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fe1a8044070 con 0x7fe1bc156be0 2024-04-25T11:23:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.909+0000 7fe1b8c00640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe19c0744b0 0x7fe19c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.909+0000 7fe1b8c00640 1 -- 172.21.15.7:0/3135130270 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7fe1a80b5a70 con 0x7fe1bc156be0 2024-04-25T11:23:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.909+0000 7fe1c0c00640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe19c0744b0 0x7fe19c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.910+0000 7fe1c0c00640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe19c0744b0 0x7fe19c0769c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0000f60 tx=0x7fe1b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:10.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:10.914+0000 7fe1b8c00640 1 -- 172.21.15.7:0/3135130270 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fe1a8080360 con 0x7fe1bc156be0 2024-04-25T11:23:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.123+0000 7fe1c2000640 1 -- 172.21.15.7:0/3135130270 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fe1bc050740 con 0x7fe19c0744b0 2024-04-25T11:23:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.124+0000 7fe1b8c00640 1 -- 172.21.15.7:0/3135130270 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe1bc050740 con 0x7fe19c0744b0 2024-04-25T11:23:11.119 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.127+0000 7fe1c2000640 1 -- 172.21.15.7:0/3135130270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe19c0744b0 msgr2=0x7fe19c0769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.127+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe19c0744b0 0x7fe19c0769c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0000f60 tx=0x7fe1b0009040 comp rx=0 tx=0).stop 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.127+0000 7fe1c2000640 1 -- 172.21.15.7:0/3135130270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc156be0 msgr2=0x7fe1bc152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.127+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc156be0 0x7fe1bc152980 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80097a0 tx=0x7fe1a80046c0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.127+0000 7fe1c2000640 1 -- 172.21.15.7:0/3135130270 shutdown_connections 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.127+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fe19c0744b0 0x7fe19c0769c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.127+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1bc151060 0x7fe1bc152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.127+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1bc156be0 0x7fe1bc152980 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.127+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3135130270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1bc06d340 0x7fe1bc14c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.127+0000 7fe1c2000640 1 -- 172.21.15.7:0/3135130270 >> 172.21.15.7:0/3135130270 conn(0x7fe1bc0d1540 msgr2=0x7fe1bc06f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.128+0000 7fe1c2000640 1 -- 172.21.15.7:0/3135130270 shutdown_connections 2024-04-25T11:23:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.128+0000 7fe1c2000640 1 -- 172.21.15.7:0/3135130270 wait complete. 2024-04-25T11:23:11.151 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T11:23:11.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:23:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.439+0000 7f4e4cc00640 1 Processor -- start 2024-04-25T11:23:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.439+0000 7f4e4cc00640 1 -- start start 2024-04-25T11:23:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.441+0000 7f4e4cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e481440a0 0x7f4e4815f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.441+0000 7f4e4cc00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4e4806dc90 0x7f4e480cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.442+0000 7f4e4cc00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e4815e620 con 0x7f4e480cf370 2024-04-25T11:23:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.442+0000 7f4e4cc00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4e4815e900 con 0x7f4e4806dc90 2024-04-25T11:23:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.442+0000 7f4e4cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4815ebe0 con 0x7f4e481440a0 2024-04-25T11:23:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.442+0000 7f4e47e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4e4806dc90 0x7f4e480cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.442+0000 7f4e47400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e481440a0 0x7f4e4815f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.443+0000 7f4e47e00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4e4806dc90 0x7f4e480cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:43154/0 (socket says 172.21.15.7:43154) 2024-04-25T11:23:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.443+0000 7f4e47e00640 1 -- 172.21.15.7:0/2870246071 learned_addr learned my addr 172.21.15.7:0/2870246071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.443+0000 7f4e47400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e481440a0 0x7f4e4815f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:33684/0 (socket says 172.21.15.7:33684) 2024-04-25T11:23:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.443+0000 7f4e47e00640 1 -- 172.21.15.7:0/2870246071 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e481440a0 msgr2=0x7f4e4815f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.443+0000 7f4e47e00640 1 --2- 172.21.15.7:0/2870246071 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e481440a0 0x7f4e4815f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.443+0000 7f4e47e00640 1 -- 172.21.15.7:0/2870246071 >> v1:172.21.15.87:6789/0 conn(0x7f4e480cf370 legacy=0x7f4e4806d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.443+0000 7f4e47e00640 1 -- 172.21.15.7:0/2870246071 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4815eec0 con 0x7f4e4806dc90 2024-04-25T11:23:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.445+0000 7f4e47e00640 1 --2- 172.21.15.7:0/2870246071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e4806dc90 0x7f4e480cf780 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009e50 tx=0x7f4e4002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c2d8f60558c2eba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.445+0000 7f4e46000640 1 -- 172.21.15.7:0/2870246071 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40036070 con 0x7f4e4806dc90 2024-04-25T11:23:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.445+0000 7f4e46000640 1 -- 172.21.15.7:0/2870246071 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e40031600 con 0x7f4e4806dc90 2024-04-25T11:23:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.445+0000 7f4e46000640 1 -- 172.21.15.7:0/2870246071 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e400319d0 con 0x7f4e4806dc90 2024-04-25T11:23:11.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.446+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2870246071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e4806dc90 msgr2=0x7f4e480cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.446+0000 7f4e4cc00640 1 --2- 172.21.15.7:0/2870246071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e4806dc90 0x7f4e480cf780 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009e50 tx=0x7f4e4002a680 comp rx=0 tx=0).stop 2024-04-25T11:23:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.446+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2870246071 shutdown_connections 2024-04-25T11:23:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.446+0000 7f4e4cc00640 1 --2- 172.21.15.7:0/2870246071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e4806dc90 0x7f4e480cf780 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.446+0000 7f4e4cc00640 1 --2- 172.21.15.7:0/2870246071 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e481440a0 0x7f4e4815f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.446+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2870246071 >> 172.21.15.7:0/2870246071 conn(0x7f4e48073160 msgr2=0x7f4e48073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.446+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2870246071 shutdown_connections 2024-04-25T11:23:11.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.446+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2870246071 wait complete. 2024-04-25T11:23:11.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.449+0000 7f4e4cc00640 1 Processor -- start 2024-04-25T11:23:11.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.449+0000 7f4e4cc00640 1 -- start start 2024-04-25T11:23:11.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.450+0000 7f4e4cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e480cf370 0x7f4e48140880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.450+0000 7f4e4cc00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e481440a0 0x7f4e481e78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.450+0000 7f4e4cc00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e481e73e0 0x7f4e481e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.450+0000 7f4e4cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4815f770 con 0x7f4e480cf370 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.450+0000 7f4e4cc00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4815e200 con 0x7f4e481e73e0 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.450+0000 7f4e4cc00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e481f5b60 con 0x7f4e481440a0 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.451+0000 7f4e47400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e480cf370 0x7f4e48140880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.451+0000 7f4e47400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e480cf370 0x7f4e48140880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40800/0 (socket says 172.21.15.7:40800) 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.451+0000 7f4e47400640 1 -- 172.21.15.7:0/170431401 learned_addr learned my addr 172.21.15.7:0/170431401 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.451+0000 7f4e47e00640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e481e73e0 0x7f4e481e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.451+0000 7f4e46a00640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e481440a0 0x7f4e481e78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.451+0000 7f4e47400640 1 -- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e481440a0 msgr2=0x7f4e481e78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.451+0000 7f4e47400640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e481440a0 0x7f4e481e78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.451+0000 7f4e47400640 1 -- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e481e73e0 msgr2=0x7f4e481e7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.451+0000 7f4e47400640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e481e73e0 0x7f4e481e7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.451+0000 7f4e47400640 1 -- 172.21.15.7:0/170431401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e48141160 con 0x7f4e480cf370 2024-04-25T11:23:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.451+0000 7f4e47e00640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e481e73e0 0x7f4e481e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.452+0000 7f4e47400640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e480cf370 0x7f4e48140880 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800e870 tx=0x7f4e3800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.452+0000 7f4e3f400640 1 -- 172.21.15.7:0/170431401 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38017070 con 0x7f4e480cf370 2024-04-25T11:23:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.452+0000 7f4e3f400640 1 -- 172.21.15.7:0/170431401 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3800c940 con 0x7f4e480cf370 2024-04-25T11:23:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.452+0000 7f4e4cc00640 1 -- 172.21.15.7:0/170431401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e48141440 con 0x7f4e480cf370 2024-04-25T11:23:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.452+0000 7f4e3f400640 1 -- 172.21.15.7:0/170431401 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3800cd10 con 0x7f4e480cf370 2024-04-25T11:23:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.452+0000 7f4e4cc00640 1 -- 172.21.15.7:0/170431401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e481f4300 con 0x7f4e480cf370 2024-04-25T11:23:11.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.453+0000 7f4e4cc00640 1 -- 172.21.15.7:0/170431401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e48050740 con 0x7f4e480cf370 2024-04-25T11:23:11.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.455+0000 7f4e3f400640 1 -- 172.21.15.7:0/170431401 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4e3801a070 con 0x7f4e480cf370 2024-04-25T11:23:11.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.456+0000 7f4e3f400640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e280744b0 0x7f4e280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.456+0000 7f4e3f400640 1 -- 172.21.15.7:0/170431401 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f4e38095740 con 0x7f4e480cf370 2024-04-25T11:23:11.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.456+0000 7f4e46a00640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e280744b0 0x7f4e280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.456+0000 7f4e3f400640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] conn(0x7f4e2807a9a0 0x7f4e2807ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.456+0000 7f4e3f400640 1 -- 172.21.15.7:0/170431401 --> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e2807d500 con 0x7f4e2807a9a0 2024-04-25T11:23:11.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.456+0000 7f4e3f400640 1 -- 172.21.15.7:0/170431401 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=159) ==== 24+0+0 (secure 0 0 0) 0x7f4e38095af0 con 0x7f4e480cf370 2024-04-25T11:23:11.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.457+0000 7f4e46a00640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e280744b0 0x7f4e280769c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4e4806f810 tx=0x7f4e34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:11.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.457+0000 7f4e47e00640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] conn(0x7f4e2807a9a0 0x7f4e2807ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.458+0000 7f4e47e00640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] conn(0x7f4e2807a9a0 0x7f4e2807ce30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:11.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.460+0000 7f4e3f400640 1 -- 172.21.15.7:0/170431401 <== osd.6 v2:172.21.15.87:6824/960154076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e2807d500 con 0x7f4e2807a9a0 2024-04-25T11:23:11.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.478+0000 7f4e4cc00640 1 -- 172.21.15.7:0/170431401 --> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4e48141960 con 0x7f4e2807a9a0 2024-04-25T11:23:11.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e3f400640 1 -- 172.21.15.7:0/170431401 <== osd.6 v2:172.21.15.87:6824/960154076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4e48141960 con 0x7f4e2807a9a0 2024-04-25T11:23:11.474 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:23:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:23:11.475 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:23:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e4cc00640 1 -- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] conn(0x7f4e2807a9a0 msgr2=0x7f4e2807ce30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e4cc00640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.87:6824/960154076,v1:172.21.15.87:6825/960154076] conn(0x7f4e2807a9a0 0x7f4e2807ce30 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e4cc00640 1 -- 172.21.15.7:0/170431401 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e280744b0 msgr2=0x7f4e280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e4cc00640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e280744b0 0x7f4e280769c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4e4806f810 tx=0x7f4e34009040 comp rx=0 tx=0).stop 2024-04-25T11:23:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e4cc00640 1 -- 172.21.15.7:0/170431401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e480cf370 msgr2=0x7f4e48140880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e4cc00640 1 --2- 172.21.15.7:0/170431401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e480cf370 0x7f4e48140880 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800e870 tx=0x7f4e3800ed30 comp rx=0 tx=0).stop 2024-04-25T11:23:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e47e00640 1 -- 172.21.15.7:0/170431401 reap_dead start 2024-04-25T11:23:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e4cc00640 1 -- 172.21.15.7:0/170431401 shutdown_connections 2024-04-25T11:23:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e4cc00640 1 -- 172.21.15.7:0/170431401 >> 172.21.15.7:0/170431401 conn(0x7f4e48073160 msgr2=0x7f4e480b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:11.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e4cc00640 1 -- 172.21.15.7:0/170431401 shutdown_connections 2024-04-25T11:23:11.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.480+0000 7f4e4cc00640 1 -- 172.21.15.7:0/170431401 wait complete. 2024-04-25T11:23:11.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T11:23:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.801+0000 7fa4c0600640 1 Processor -- start 2024-04-25T11:23:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.801+0000 7fa4c0600640 1 -- start start 2024-04-25T11:23:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.804+0000 7fa4c0600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4b806d580 0x7fa4b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.804+0000 7fa4c0600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa4b806f1e0 0x7fa4b806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.804+0000 7fa4c0600640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b806ded0 con 0x7fa4b815f180 2024-04-25T11:23:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.804+0000 7fa4c0600640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b815f590 con 0x7fa4b806f1e0 2024-04-25T11:23:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.804+0000 7fa4c0600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b8158ae0 con 0x7fa4b806d580 2024-04-25T11:23:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.804+0000 7fa4be600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa4b806f1e0 0x7fa4b806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.804+0000 7fa4be600640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa4b806f1e0 0x7fa4b806f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40806/0 (socket says 172.21.15.7:40806) 2024-04-25T11:23:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.804+0000 7fa4be600640 1 -- 172.21.15.7:0/3509371060 learned_addr learned my addr 172.21.15.7:0/3509371060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.804+0000 7fa4bd200640 1 --2- 172.21.15.7:0/3509371060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4b806d580 0x7fa4b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.805+0000 7fa4be600640 1 -- 172.21.15.7:0/3509371060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4b806d580 msgr2=0x7fa4b806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.805+0000 7fa4be600640 1 --2- 172.21.15.7:0/3509371060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4b806d580 0x7fa4b806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.805+0000 7fa4be600640 1 -- 172.21.15.7:0/3509371060 >> v1:172.21.15.87:6789/0 conn(0x7fa4b815f180 legacy=0x7fa4b8161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.805+0000 7fa4be600640 1 -- 172.21.15.7:0/3509371060 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b8162700 con 0x7fa4b806f1e0 2024-04-25T11:23:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.806+0000 7fa4be600640 1 --2- 172.21.15.7:0/3509371060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4b806f1e0 0x7fa4b806f5f0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00098a0 tx=0x7fa4b002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c52a39af9c47f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.806+0000 7fa4b7e00640 1 -- 172.21.15.7:0/3509371060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0037070 con 0x7fa4b806f1e0 2024-04-25T11:23:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.806+0000 7fa4b7e00640 1 -- 172.21.15.7:0/3509371060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b0032440 con 0x7fa4b806f1e0 2024-04-25T11:23:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.806+0000 7fa4b7e00640 1 -- 172.21.15.7:0/3509371060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0032810 con 0x7fa4b806f1e0 2024-04-25T11:23:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.807+0000 7fa4c0600640 1 -- 172.21.15.7:0/3509371060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4b806f1e0 msgr2=0x7fa4b806f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.807+0000 7fa4c0600640 1 --2- 172.21.15.7:0/3509371060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4b806f1e0 0x7fa4b806f5f0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00098a0 tx=0x7fa4b002a2a0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.807+0000 7fa4c0600640 1 -- 172.21.15.7:0/3509371060 shutdown_connections 2024-04-25T11:23:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.807+0000 7fa4c0600640 1 --2- 172.21.15.7:0/3509371060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4b806f1e0 0x7fa4b806f5f0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.807+0000 7fa4c0600640 1 --2- 172.21.15.7:0/3509371060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4b806d580 0x7fa4b806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.807+0000 7fa4c0600640 1 -- 172.21.15.7:0/3509371060 >> 172.21.15.7:0/3509371060 conn(0x7fa4b80732b0 msgr2=0x7fa4b80736c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.807+0000 7fa4c0600640 1 -- 172.21.15.7:0/3509371060 shutdown_connections 2024-04-25T11:23:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.807+0000 7fa4c0600640 1 -- 172.21.15.7:0/3509371060 wait complete. 2024-04-25T11:23:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.810+0000 7fa4c0600640 1 Processor -- start 2024-04-25T11:23:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.810+0000 7fa4c0600640 1 -- start start 2024-04-25T11:23:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.811+0000 7fa4c0600640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4b806d580 0x7fa4b814d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.811+0000 7fa4c0600640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4b806f1e0 0x7fa4b814f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.811+0000 7fa4c0600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4b815f180 0x7fa4b8161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.811+0000 7fa4c0600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b815ec60 con 0x7fa4b815f180 2024-04-25T11:23:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.811+0000 7fa4c0600640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b815e980 con 0x7fa4b806d580 2024-04-25T11:23:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.811+0000 7fa4c0600640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b815e670 con 0x7fa4b806f1e0 2024-04-25T11:23:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.812+0000 7fa4be600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4b815f180 0x7fa4b8161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.812+0000 7fa4be600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4b815f180 0x7fa4b8161300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40818/0 (socket says 172.21.15.7:40818) 2024-04-25T11:23:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.812+0000 7fa4be600640 1 -- 172.21.15.7:0/438060968 learned_addr learned my addr 172.21.15.7:0/438060968 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.812+0000 7fa4be600640 1 -- 172.21.15.7:0/438060968 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4b806f1e0 msgr2=0x7fa4b814f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.812+0000 7fa4bdc00640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4b806d580 0x7fa4b814d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.812+0000 7fa4be600640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4b806f1e0 0x7fa4b814f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.812+0000 7fa4be600640 1 -- 172.21.15.7:0/438060968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4b806d580 msgr2=0x7fa4b814d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.812+0000 7fa4be600640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4b806d580 0x7fa4b814d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.812+0000 7fa4be600640 1 -- 172.21.15.7:0/438060968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b814dbf0 con 0x7fa4b815f180 2024-04-25T11:23:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.812+0000 7fa4be600640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4b815f180 0x7fa4b8161300 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fa4b002a2e0 tx=0x7fa4b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.813+0000 7fa4b5600640 1 -- 172.21.15.7:0/438060968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0037040 con 0x7fa4b815f180 2024-04-25T11:23:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.813+0000 7fa4b5600640 1 -- 172.21.15.7:0/438060968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b0042070 con 0x7fa4b815f180 2024-04-25T11:23:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.813+0000 7fa4b5600640 1 -- 172.21.15.7:0/438060968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0030920 con 0x7fa4b815f180 2024-04-25T11:23:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.813+0000 7fa4c0600640 1 -- 172.21.15.7:0/438060968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b8150300 con 0x7fa4b815f180 2024-04-25T11:23:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.813+0000 7fa4c0600640 1 -- 172.21.15.7:0/438060968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b81508c0 con 0x7fa4b815f180 2024-04-25T11:23:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.814+0000 7fa4c0600640 1 -- 172.21.15.7:0/438060968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b8050740 con 0x7fa4b815f180 2024-04-25T11:23:11.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.816+0000 7fa4b5600640 1 -- 172.21.15.7:0/438060968 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fa4b0030ac0 con 0x7fa4b815f180 2024-04-25T11:23:11.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.817+0000 7fa4b5600640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4980744e0 0x7fa4980769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:11.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.817+0000 7fa4b5600640 1 -- 172.21.15.7:0/438060968 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7fa4b00b5a90 con 0x7fa4b815f180 2024-04-25T11:23:11.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.817+0000 7fa4bdc00640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4980744e0 0x7fa4980769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:11.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.818+0000 7fa4bdc00640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4980744e0 0x7fa4980769f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac007fd0 tx=0x7fa4ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:11.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:11.822+0000 7fa4b5600640 1 -- 172.21.15.7:0/438060968 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7fa4b00802f0 con 0x7fa4b815f180 2024-04-25T11:23:12.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.032+0000 7fa4c0600640 1 -- 172.21.15.7:0/438060968 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fa4b8074250 con 0x7fa4980744e0 2024-04-25T11:23:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.033+0000 7fa4b5600640 1 -- 172.21.15.7:0/438060968 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa4b8074250 con 0x7fa4980744e0 2024-04-25T11:23:12.028 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T11:23:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.035+0000 7fa4c0600640 1 -- 172.21.15.7:0/438060968 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4980744e0 msgr2=0x7fa4980769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.035+0000 7fa4c0600640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4980744e0 0x7fa4980769f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac007fd0 tx=0x7fa4ac009040 comp rx=0 tx=0).stop 2024-04-25T11:23:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.035+0000 7fa4c0600640 1 -- 172.21.15.7:0/438060968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4b815f180 msgr2=0x7fa4b8161300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.035+0000 7fa4c0600640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4b815f180 0x7fa4b8161300 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fa4b002a2e0 tx=0x7fa4b00046c0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.036+0000 7fa4c0600640 1 -- 172.21.15.7:0/438060968 shutdown_connections 2024-04-25T11:23:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.036+0000 7fa4c0600640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fa4980744e0 0x7fa4980769f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.036+0000 7fa4c0600640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4b815f180 0x7fa4b8161300 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.036+0000 7fa4c0600640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4b806f1e0 0x7fa4b814f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.036+0000 7fa4c0600640 1 --2- 172.21.15.7:0/438060968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4b806d580 0x7fa4b814d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.036+0000 7fa4c0600640 1 -- 172.21.15.7:0/438060968 >> 172.21.15.7:0/438060968 conn(0x7fa4b80732b0 msgr2=0x7fa4b8156ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.036+0000 7fa4c0600640 1 -- 172.21.15.7:0/438060968 shutdown_connections 2024-04-25T11:23:12.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.036+0000 7fa4c0600640 1 -- 172.21.15.7:0/438060968 wait complete. 2024-04-25T11:23:12.058 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T11:23:12.058 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T11:23:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.348+0000 7fbe98200640 1 Processor -- start 2024-04-25T11:23:12.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.348+0000 7fbe98200640 1 -- start start 2024-04-25T11:23:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.350+0000 7fbe98200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe9015de20 0x7fbe900b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.350+0000 7fbe98200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbe900b5ff0 0x7fbe9006f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.350+0000 7fbe98200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe900b6400 con 0x7fbe900ae880 2024-04-25T11:23:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.350+0000 7fbe98200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7fbe900afcb0 con 0x7fbe9015de20 2024-04-25T11:23:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.350+0000 7fbe98200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbe900aed60 con 0x7fbe900b5ff0 2024-04-25T11:23:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.350+0000 7fbe95800640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbe900ae880 0x7fbe900cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58598/0 (socket says 172.21.15.7:58598) 2024-04-25T11:23:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.350+0000 7fbe95800640 1 -- 172.21.15.7:0/875318250 learned_addr learned my addr 172.21.15.7:0/875318250 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.351+0000 7fbe94e00640 1 --2- 172.21.15.7:0/875318250 >> v2:172.21.15.7:3300/0 conn(0x7fbe9015de20 0x7fbe900b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.351+0000 7fbe8fe00640 1 -- 172.21.15.7:0/875318250 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 387831796 0 0) 0x7fbe84001110 con 0x7fbe900ae880 2024-04-25T11:23:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.351+0000 7fbe8fe00640 1 -- 172.21.15.7:0/875318250 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe78003520 con 0x7fbe900ae880 2024-04-25T11:23:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.351+0000 7fbe94e00640 1 -- 172.21.15.7:0/875318250 >> v2:172.21.15.87:3300/0 conn(0x7fbe900b5ff0 msgr2=0x7fbe9006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.351+0000 7fbe96200640 1 --2- 172.21.15.7:0/875318250 >> v2:172.21.15.87:3300/0 conn(0x7fbe900b5ff0 0x7fbe9006f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.351+0000 7fbe94e00640 1 --2- 172.21.15.7:0/875318250 >> v2:172.21.15.87:3300/0 conn(0x7fbe900b5ff0 0x7fbe9006f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.351+0000 7fbe94e00640 1 -- 172.21.15.7:0/875318250 >> v1:172.21.15.7:6789/0 conn(0x7fbe900ae880 legacy=0x7fbe900cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.351+0000 7fbe94e00640 1 -- 172.21.15.7:0/875318250 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe900af040 con 0x7fbe9015de20 2024-04-25T11:23:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.352+0000 7fbe94e00640 1 --2- 172.21.15.7:0/875318250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe9015de20 0x7fbe900b5ab0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fbe9006da40 tx=0x7fbe7c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef5e24405df465f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.353+0000 7fbe8fe00640 1 -- 172.21.15.7:0/875318250 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c037070 con 0x7fbe9015de20 2024-04-25T11:23:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.353+0000 7fbe8fe00640 1 -- 172.21.15.7:0/875318250 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7c032440 con 0x7fbe9015de20 2024-04-25T11:23:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.353+0000 7fbe8fe00640 1 -- 172.21.15.7:0/875318250 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c032810 con 0x7fbe9015de20 2024-04-25T11:23:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.353+0000 7fbe98200640 1 -- 172.21.15.7:0/875318250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe9015de20 msgr2=0x7fbe900b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.353+0000 7fbe98200640 1 --2- 172.21.15.7:0/875318250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe9015de20 0x7fbe900b5ab0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fbe9006da40 tx=0x7fbe7c02a1a0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.354+0000 7fbe98200640 1 -- 172.21.15.7:0/875318250 shutdown_connections 2024-04-25T11:23:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.354+0000 7fbe98200640 1 --2- 172.21.15.7:0/875318250 >> v2:172.21.15.87:3300/0 conn(0x7fbe900b5ff0 0x7fbe9006f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.354+0000 7fbe98200640 1 --2- 172.21.15.7:0/875318250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe9015de20 0x7fbe900b5ab0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.354+0000 7fbe98200640 1 -- 172.21.15.7:0/875318250 >> 172.21.15.7:0/875318250 conn(0x7fbe90073920 msgr2=0x7fbe90073d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.354+0000 7fbe98200640 1 -- 172.21.15.7:0/875318250 shutdown_connections 2024-04-25T11:23:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.354+0000 7fbe98200640 1 -- 172.21.15.7:0/875318250 wait complete. 2024-04-25T11:23:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.356+0000 7fbe98200640 1 Processor -- start 2024-04-25T11:23:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.356+0000 7fbe98200640 1 -- start start 2024-04-25T11:23:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.358+0000 7fbe98200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe900ae880 0x7fbe9013b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.358+0000 7fbe98200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe900b5ff0 0x7fbe901e6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.358+0000 7fbe98200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe901e6ed0 0x7fbe90154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.358+0000 7fbe98200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9015e230 con 0x7fbe900ae880 2024-04-25T11:23:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.358+0000 7fbe98200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbe900af600 con 0x7fbe901e6ed0 2024-04-25T11:23:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.358+0000 7fbe98200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9006fa00 con 0x7fbe900b5ff0 2024-04-25T11:23:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.358+0000 7fbe95800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe900ae880 0x7fbe9013b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.358+0000 7fbe95800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe900ae880 0x7fbe9013b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40848/0 (socket says 172.21.15.7:40848) 2024-04-25T11:23:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.358+0000 7fbe95800640 1 -- 172.21.15.7:0/1835063281 learned_addr learned my addr 172.21.15.7:0/1835063281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.359+0000 7fbe94e00640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe900b5ff0 0x7fbe901e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.359+0000 7fbe95800640 1 -- 172.21.15.7:0/1835063281 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe900b5ff0 msgr2=0x7fbe901e6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.359+0000 7fbe95800640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe900b5ff0 0x7fbe901e6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.359+0000 7fbe95800640 1 -- 172.21.15.7:0/1835063281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe901e6ed0 msgr2=0x7fbe90154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.359+0000 7fbe95800640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe901e6ed0 0x7fbe90154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.359+0000 7fbe95800640 1 -- 172.21.15.7:0/1835063281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe901e76c0 con 0x7fbe900ae880 2024-04-25T11:23:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.359+0000 7fbe95800640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe900ae880 0x7fbe9013b040 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400c3d0 tx=0x7fbe8400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.359+0000 7fbe8d600640 1 -- 172.21.15.7:0/1835063281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84017070 con 0x7fbe900ae880 2024-04-25T11:23:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.359+0000 7fbe8d600640 1 -- 172.21.15.7:0/1835063281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe84005ce0 con 0x7fbe900ae880 2024-04-25T11:23:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.359+0000 7fbe8d600640 1 -- 172.21.15.7:0/1835063281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84007310 con 0x7fbe900ae880 2024-04-25T11:23:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.359+0000 7fbe98200640 1 -- 172.21.15.7:0/1835063281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe901e79a0 con 0x7fbe900ae880 2024-04-25T11:23:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.360+0000 7fbe98200640 1 -- 172.21.15.7:0/1835063281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe901e7f60 con 0x7fbe900ae880 2024-04-25T11:23:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.361+0000 7fbe98200640 1 -- 172.21.15.7:0/1835063281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe9006a100 con 0x7fbe900ae880 2024-04-25T11:23:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.362+0000 7fbe8d600640 1 -- 172.21.15.7:0/1835063281 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7fbe8401a070 con 0x7fbe900ae880 2024-04-25T11:23:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.363+0000 7fbe8d600640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe700744e0 0x7fbe700769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.363+0000 7fbe8d600640 1 -- 172.21.15.7:0/1835063281 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7fbe840944e0 con 0x7fbe900ae880 2024-04-25T11:23:12.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.363+0000 7fbe8d600640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] conn(0x7fbe7007a9d0 0x7fbe7007ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.363+0000 7fbe8d600640 1 -- 172.21.15.7:0/1835063281 --> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe7007d530 con 0x7fbe7007a9d0 2024-04-25T11:23:12.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.363+0000 7fbe8d600640 1 -- 172.21.15.7:0/1835063281 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=159) ==== 24+0+0 (secure 0 0 0) 0x7fbe8409b050 con 0x7fbe900ae880 2024-04-25T11:23:12.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.363+0000 7fbe94e00640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe700744e0 0x7fbe700769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.364+0000 7fbe94e00640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe700744e0 0x7fbe700769f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c034000 tx=0x7fbe7c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:12.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.364+0000 7fbe96200640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] conn(0x7fbe7007a9d0 0x7fbe7007ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.365+0000 7fbe96200640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] conn(0x7fbe7007a9d0 0x7fbe7007ce60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:12.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.368+0000 7fbe8d600640 1 -- 172.21.15.7:0/1835063281 <== osd.7 v2:172.21.15.87:6800/3856235626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe7007d530 con 0x7fbe7007a9d0 2024-04-25T11:23:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.382+0000 7fbe98200640 1 -- 172.21.15.7:0/1835063281 --> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbe90154bc0 con 0x7fbe7007a9d0 2024-04-25T11:23:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.383+0000 7fbe8d600640 1 -- 172.21.15.7:0/1835063281 <== osd.7 v2:172.21.15.87:6800/3856235626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbe90154bc0 con 0x7fbe7007a9d0 2024-04-25T11:23:12.379 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-04-25T11:23:12.379 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T11:23:12.379 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-04-25T11:23:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.384+0000 7fbe98200640 1 -- 172.21.15.7:0/1835063281 >> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] conn(0x7fbe7007a9d0 msgr2=0x7fbe7007ce60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.384+0000 7fbe98200640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.87:6800/3856235626,v1:172.21.15.87:6801/3856235626] conn(0x7fbe7007a9d0 0x7fbe7007ce60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.384+0000 7fbe98200640 1 -- 172.21.15.7:0/1835063281 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe700744e0 msgr2=0x7fbe700769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.384+0000 7fbe98200640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7fbe700744e0 0x7fbe700769f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c034000 tx=0x7fbe7c02e000 comp rx=0 tx=0).stop 2024-04-25T11:23:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.384+0000 7fbe98200640 1 -- 172.21.15.7:0/1835063281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe900ae880 msgr2=0x7fbe9013b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.384+0000 7fbe98200640 1 --2- 172.21.15.7:0/1835063281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe900ae880 0x7fbe9013b040 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400c3d0 tx=0x7fbe8400c890 comp rx=0 tx=0).stop 2024-04-25T11:23:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.384+0000 7fbe96200640 1 -- 172.21.15.7:0/1835063281 reap_dead start 2024-04-25T11:23:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.385+0000 7fbe98200640 1 -- 172.21.15.7:0/1835063281 shutdown_connections 2024-04-25T11:23:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.385+0000 7fbe98200640 1 -- 172.21.15.7:0/1835063281 >> 172.21.15.7:0/1835063281 conn(0x7fbe90073920 msgr2=0x7fbe90070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.385+0000 7fbe98200640 1 -- 172.21.15.7:0/1835063281 shutdown_connections 2024-04-25T11:23:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.385+0000 7fbe98200640 1 -- 172.21.15.7:0/1835063281 wait complete. 2024-04-25T11:23:12.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T11:23:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.694+0000 7f2c2e200640 1 Processor -- start 2024-04-25T11:23:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.694+0000 7f2c2e200640 1 -- start start 2024-04-25T11:23:12.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.696+0000 7f2c2e200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2c2809cb20 0x7f2c280b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.696+0000 7f2c2e200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2c28163850 0x7f2c280ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.696+0000 7f2c2e200640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c280d1d30 con 0x7f2c280ce550 2024-04-25T11:23:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.696+0000 7f2c2e200640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f2c280d2620 con 0x7f2c28163850 2024-04-25T11:23:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.696+0000 7f2c2e200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2c2806ebb0 con 0x7f2c2809cb20 2024-04-25T11:23:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.697+0000 7f2c27e00640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2c280ce550 0x7f2c280d2ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58604/0 (socket says 172.21.15.7:58604) 2024-04-25T11:23:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.697+0000 7f2c27e00640 1 -- 172.21.15.7:0/1572650337 learned_addr learned my addr 172.21.15.7:0/1572650337 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.697+0000 7f2c1ea00640 1 --2- 172.21.15.7:0/1572650337 >> v2:172.21.15.7:3300/0 conn(0x7f2c28163850 0x7f2c280ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.697+0000 7f2c27400640 1 --2- 172.21.15.7:0/1572650337 >> v2:172.21.15.87:3300/0 conn(0x7f2c2809cb20 0x7f2c280b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.697+0000 7f2c1ea00640 1 -- 172.21.15.7:0/1572650337 >> v2:172.21.15.87:3300/0 conn(0x7f2c2809cb20 msgr2=0x7f2c280b91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.697+0000 7f2c1ea00640 1 --2- 172.21.15.7:0/1572650337 >> v2:172.21.15.87:3300/0 conn(0x7f2c2809cb20 0x7f2c280b91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.697+0000 7f2c1ea00640 1 -- 172.21.15.7:0/1572650337 >> v1:172.21.15.7:6789/0 conn(0x7f2c280ce550 legacy=0x7f2c280d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.697+0000 7f2c1ea00640 1 -- 172.21.15.7:0/1572650337 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2806e5f0 con 0x7f2c28163850 2024-04-25T11:23:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.699+0000 7f2c1ea00640 1 --2- 172.21.15.7:0/1572650337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c28163850 0x7f2c280ce010 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f2c2808cde0 tx=0x7f2c1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a684aa6605a1a8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.699+0000 7f2c26a00640 1 -- 172.21.15.7:0/1572650337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10037070 con 0x7f2c28163850 2024-04-25T11:23:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.699+0000 7f2c26a00640 1 -- 172.21.15.7:0/1572650337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c10032440 con 0x7f2c28163850 2024-04-25T11:23:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.699+0000 7f2c26a00640 1 -- 172.21.15.7:0/1572650337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10032810 con 0x7f2c28163850 2024-04-25T11:23:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.700+0000 7f2c2e200640 1 -- 172.21.15.7:0/1572650337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c28163850 msgr2=0x7f2c280ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.700+0000 7f2c2e200640 1 --2- 172.21.15.7:0/1572650337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c28163850 0x7f2c280ce010 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f2c2808cde0 tx=0x7f2c1002a1a0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.700+0000 7f2c2e200640 1 -- 172.21.15.7:0/1572650337 shutdown_connections 2024-04-25T11:23:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.700+0000 7f2c2e200640 1 --2- 172.21.15.7:0/1572650337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c28163850 0x7f2c280ce010 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.700+0000 7f2c2e200640 1 --2- 172.21.15.7:0/1572650337 >> v2:172.21.15.87:3300/0 conn(0x7f2c2809cb20 0x7f2c280b91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.700+0000 7f2c2e200640 1 -- 172.21.15.7:0/1572650337 >> 172.21.15.7:0/1572650337 conn(0x7f2c280b54b0 msgr2=0x7f2c280b58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.700+0000 7f2c2e200640 1 -- 172.21.15.7:0/1572650337 shutdown_connections 2024-04-25T11:23:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.700+0000 7f2c2e200640 1 -- 172.21.15.7:0/1572650337 wait complete. 2024-04-25T11:23:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.703+0000 7f2c2e200640 1 Processor -- start 2024-04-25T11:23:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.703+0000 7f2c2e200640 1 -- start start 2024-04-25T11:23:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.704+0000 7f2c2e200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2c2809cb20 0x7f2c2815f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.704+0000 7f2c2e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c280ce550 0x7f2c280b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.704+0000 7f2c2e200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c28163850 0x7f2c280b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.704+0000 7f2c2e200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c280d22b0 con 0x7f2c280ce550 2024-04-25T11:23:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.704+0000 7f2c2e200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2c280d1fd0 con 0x7f2c28163850 2024-04-25T11:23:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.704+0000 7f2c2e200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2c28163c60 con 0x7f2c2809cb20 2024-04-25T11:23:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.705+0000 7f2c1ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c280ce550 0x7f2c280b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.705+0000 7f2c1ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c280ce550 0x7f2c280b79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40858/0 (socket says 172.21.15.7:40858) 2024-04-25T11:23:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.705+0000 7f2c1ea00640 1 -- 172.21.15.7:0/3713936113 learned_addr learned my addr 172.21.15.7:0/3713936113 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.705+0000 7f2c27e00640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c28163850 0x7f2c280b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.705+0000 7f2c27400640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2c2809cb20 0x7f2c2815f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.705+0000 7f2c1ea00640 1 -- 172.21.15.7:0/3713936113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2c2809cb20 msgr2=0x7f2c2815f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.705+0000 7f2c1ea00640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2c2809cb20 0x7f2c2815f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.705+0000 7f2c1ea00640 1 -- 172.21.15.7:0/3713936113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c28163850 msgr2=0x7f2c280b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.705+0000 7f2c1ea00640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c28163850 0x7f2c280b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.705+0000 7f2c1ea00640 1 -- 172.21.15.7:0/3713936113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2815fd20 con 0x7f2c280ce550 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.706+0000 7f2c1ea00640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c280ce550 0x7f2c280b79b0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009890 tx=0x7f2c100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.706+0000 7f2c1fe00640 1 -- 172.21.15.7:0/3713936113 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10037040 con 0x7f2c280ce550 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.706+0000 7f2c1fe00640 1 -- 172.21.15.7:0/3713936113 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c10042070 con 0x7f2c280ce550 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.706+0000 7f2c1fe00640 1 -- 172.21.15.7:0/3713936113 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10030bd0 con 0x7f2c280ce550 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.706+0000 7f2c2e200640 1 -- 172.21.15.7:0/3713936113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c280b7ef0 con 0x7f2c280ce550 2024-04-25T11:23:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.706+0000 7f2c2e200640 1 -- 172.21.15.7:0/3713936113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2815de00 con 0x7f2c280ce550 2024-04-25T11:23:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.708+0000 7f2c2e200640 1 -- 172.21.15.7:0/3713936113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c281f6ca0 con 0x7f2c280ce550 2024-04-25T11:23:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.709+0000 7f2c1fe00640 1 -- 172.21.15.7:0/3713936113 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f2c10047460 con 0x7f2c280ce550 2024-04-25T11:23:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.710+0000 7f2c1fe00640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2c04074500 0x7f2c04076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.711+0000 7f2c1fe00640 1 -- 172.21.15.7:0/3713936113 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f2c100b6890 con 0x7f2c280ce550 2024-04-25T11:23:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.711+0000 7f2c27400640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2c04074500 0x7f2c04076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:12.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.712+0000 7f2c27400640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2c04074500 0x7f2c04076a10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2c280d17c0 tx=0x7f2c18003990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:12.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.716+0000 7f2c1fe00640 1 -- 172.21.15.7:0/3713936113 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f2c1004c020 con 0x7f2c280ce550 2024-04-25T11:23:12.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.927+0000 7f2c2e200640 1 -- 172.21.15.7:0/3713936113 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f2c281f6f30 con 0x7f2c04074500 2024-04-25T11:23:12.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.928+0000 7f2c1fe00640 1 -- 172.21.15.7:0/3713936113 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2c281f6f30 con 0x7f2c04074500 2024-04-25T11:23:12.923 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T11:23:12.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.930+0000 7f2c2e200640 1 -- 172.21.15.7:0/3713936113 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2c04074500 msgr2=0x7f2c04076a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.930+0000 7f2c2e200640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2c04074500 0x7f2c04076a10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2c280d17c0 tx=0x7f2c18003990 comp rx=0 tx=0).stop 2024-04-25T11:23:12.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.930+0000 7f2c2e200640 1 -- 172.21.15.7:0/3713936113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c280ce550 msgr2=0x7f2c280b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:12.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.930+0000 7f2c2e200640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c280ce550 0x7f2c280b79b0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009890 tx=0x7f2c100046c0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.931+0000 7f2c2e200640 1 -- 172.21.15.7:0/3713936113 shutdown_connections 2024-04-25T11:23:12.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.931+0000 7f2c2e200640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f2c04074500 0x7f2c04076a10 secure :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2c280d17c0 tx=0x7f2c18003990 comp rx=0 tx=0).stop 2024-04-25T11:23:12.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.931+0000 7f2c2e200640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c28163850 0x7f2c280b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.931+0000 7f2c2e200640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c280ce550 0x7f2c280b79b0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.931+0000 7f2c2e200640 1 --2- 172.21.15.7:0/3713936113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2c2809cb20 0x7f2c2815f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:12.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.931+0000 7f2c2e200640 1 -- 172.21.15.7:0/3713936113 >> 172.21.15.7:0/3713936113 conn(0x7f2c280b54b0 msgr2=0x7f2c28072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:12.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.931+0000 7f2c2e200640 1 -- 172.21.15.7:0/3713936113 shutdown_connections 2024-04-25T11:23:12.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:12.931+0000 7f2c2e200640 1 -- 172.21.15.7:0/3713936113 wait complete. 2024-04-25T11:23:12.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:23:13.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.249+0000 7feb81200640 1 Processor -- start 2024-04-25T11:23:13.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.249+0000 7feb81200640 1 -- start start 2024-04-25T11:23:13.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.251+0000 7feb81200640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feb7c09b540 0x7feb7c0b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:13.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.251+0000 7feb81200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c075b10 0x7feb7c06c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:13.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.251+0000 7feb81200640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb7c0b1a60 con 0x7feb7c06cab0 2024-04-25T11:23:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.251+0000 7feb81200640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7feb7c0b2670 con 0x7feb7c09b540 2024-04-25T11:23:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.251+0000 7feb81200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feb7c0b1d40 con 0x7feb7c075b10 2024-04-25T11:23:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.252+0000 7feb73e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c075b10 0x7feb7c06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.252+0000 7feb7b400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feb7c09b540 0x7feb7c0b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.252+0000 7feb73e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c075b10 0x7feb7c06c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.7:33748/0 (socket says 172.21.15.7:33748) 2024-04-25T11:23:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.252+0000 7feb73e00640 1 -- 172.21.15.7:0/2176811351 learned_addr learned my addr 172.21.15.7:0/2176811351 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.253+0000 7feb73e00640 1 -- 172.21.15.7:0/2176811351 >> v2:172.21.15.87:3300/0 conn(0x7feb7c09b540 msgr2=0x7feb7c0b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.253+0000 7feb73e00640 1 --2- 172.21.15.7:0/2176811351 >> v2:172.21.15.87:3300/0 conn(0x7feb7c09b540 0x7feb7c0b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.253+0000 7feb73e00640 1 -- 172.21.15.7:0/2176811351 >> v1:172.21.15.87:6789/0 conn(0x7feb7c06cab0 legacy=0x7feb7c0b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.253+0000 7feb73e00640 1 -- 172.21.15.7:0/2176811351 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c0b2020 con 0x7feb7c075b10 2024-04-25T11:23:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.253+0000 7feb7b400640 1 --2- 172.21.15.7:0/2176811351 >> v2:172.21.15.87:3300/0 conn(0x7feb7c09b540 0x7feb7c0b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T11:23:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.254+0000 7feb73e00640 1 --2- 172.21.15.7:0/2176811351 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c075b10 0x7feb7c06c570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feb64009870 tx=0x7feb6402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccf4dca48bb292e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.255+0000 7feb7aa00640 1 -- 172.21.15.7:0/2176811351 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64037070 con 0x7feb7c075b10 2024-04-25T11:23:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.255+0000 7feb7aa00640 1 -- 172.21.15.7:0/2176811351 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb64032440 con 0x7feb7c075b10 2024-04-25T11:23:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.255+0000 7feb7aa00640 1 -- 172.21.15.7:0/2176811351 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64032810 con 0x7feb7c075b10 2024-04-25T11:23:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.256+0000 7feb81200640 1 -- 172.21.15.7:0/2176811351 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c075b10 msgr2=0x7feb7c06c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.256+0000 7feb81200640 1 --2- 172.21.15.7:0/2176811351 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c075b10 0x7feb7c06c570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feb64009870 tx=0x7feb6402a1c0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.256+0000 7feb81200640 1 -- 172.21.15.7:0/2176811351 shutdown_connections 2024-04-25T11:23:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.256+0000 7feb81200640 1 --2- 172.21.15.7:0/2176811351 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c075b10 0x7feb7c06c570 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.256+0000 7feb81200640 1 --2- 172.21.15.7:0/2176811351 >> v2:172.21.15.87:3300/0 conn(0x7feb7c09b540 0x7feb7c0b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.256+0000 7feb81200640 1 -- 172.21.15.7:0/2176811351 >> 172.21.15.7:0/2176811351 conn(0x7feb7c073ba0 msgr2=0x7feb7c0b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.256+0000 7feb81200640 1 -- 172.21.15.7:0/2176811351 shutdown_connections 2024-04-25T11:23:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.256+0000 7feb81200640 1 -- 172.21.15.7:0/2176811351 wait complete. 2024-04-25T11:23:13.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.258+0000 7feb81200640 1 Processor -- start 2024-04-25T11:23:13.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.259+0000 7feb81200640 1 -- start start 2024-04-25T11:23:13.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.260+0000 7feb81200640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb7c06cab0 0x7feb7c1f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:13.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.260+0000 7feb81200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c075b10 0x7feb7c14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:13.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.260+0000 7feb81200640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c09b540 0x7feb7c154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.260+0000 7feb81200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c0760a0 con 0x7feb7c075b10 2024-04-25T11:23:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.260+0000 7feb81200640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c075f20 con 0x7feb7c06cab0 2024-04-25T11:23:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.260+0000 7feb81200640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feb7c06cec0 con 0x7feb7c09b540 2024-04-25T11:23:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c075b10 0x7feb7c14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c075b10 0x7feb7c14a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40870/0 (socket says 172.21.15.7:40870) 2024-04-25T11:23:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb73e00640 1 -- 172.21.15.7:0/1924318723 learned_addr learned my addr 172.21.15.7:0/1924318723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb7b400640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb7c06cab0 0x7feb7c1f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb7be00640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c09b540 0x7feb7c154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb73e00640 1 -- 172.21.15.7:0/1924318723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c09b540 msgr2=0x7feb7c154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb73e00640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c09b540 0x7feb7c154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb73e00640 1 -- 172.21.15.7:0/1924318723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb7c06cab0 msgr2=0x7feb7c1f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb73e00640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb7c06cab0 0x7feb7c1f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb73e00640 1 -- 172.21.15.7:0/1924318723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c1f5700 con 0x7feb7c075b10 2024-04-25T11:23:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb7b400640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb7c06cab0 0x7feb7c1f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.261+0000 7feb73e00640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c075b10 0x7feb7c14a900 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7feb64000c00 tx=0x7feb640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.262+0000 7feb73400640 1 -- 172.21.15.7:0/1924318723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64037040 con 0x7feb7c075b10 2024-04-25T11:23:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.262+0000 7feb73400640 1 -- 172.21.15.7:0/1924318723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb64042070 con 0x7feb7c075b10 2024-04-25T11:23:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.262+0000 7feb73400640 1 -- 172.21.15.7:0/1924318723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64030860 con 0x7feb7c075b10 2024-04-25T11:23:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.262+0000 7feb81200640 1 -- 172.21.15.7:0/1924318723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7c1f5b80 con 0x7feb7c075b10 2024-04-25T11:23:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.262+0000 7feb81200640 1 -- 172.21.15.7:0/1924318723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7c09ece0 con 0x7feb7c075b10 2024-04-25T11:23:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.263+0000 7feb81200640 1 -- 172.21.15.7:0/1924318723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7c1f60a0 con 0x7feb7c075b10 2024-04-25T11:23:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.265+0000 7feb73400640 1 -- 172.21.15.7:0/1924318723 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7feb6402ad10 con 0x7feb7c075b10 2024-04-25T11:23:13.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.266+0000 7feb73400640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feb580744b0 0x7feb580769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:13.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.266+0000 7feb73400640 1 -- 172.21.15.7:0/1924318723 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7feb640b57a0 con 0x7feb7c075b10 2024-04-25T11:23:13.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.266+0000 7feb7b400640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feb580744b0 0x7feb580769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.267+0000 7feb7b400640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feb580744b0 0x7feb580769c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7feb7c09bb00 tx=0x7feb6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.271+0000 7feb73400640 1 -- 172.21.15.7:0/1924318723 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7feb7c1f60a0 con 0x7feb7c075b10 2024-04-25T11:23:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.473+0000 7feb81200640 1 -- 172.21.15.7:0/1924318723 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb7c050740 con 0x7feb580744b0 2024-04-25T11:23:13.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.475+0000 7feb73400640 1 -- 172.21.15.7:0/1924318723 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13357 (secure 0 0 0) 0x7feb7c050740 con 0x7feb580744b0 2024-04-25T11:23:13.470 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:23:13.471 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:23:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 -- 172.21.15.7:0/1924318723 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feb580744b0 msgr2=0x7feb580769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feb580744b0 0x7feb580769c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7feb7c09bb00 tx=0x7feb6c008040 comp rx=0 tx=0).stop 2024-04-25T11:23:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 -- 172.21.15.7:0/1924318723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c075b10 msgr2=0x7feb7c14a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c075b10 0x7feb7c14a900 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7feb64000c00 tx=0x7feb640046c0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 -- 172.21.15.7:0/1924318723 shutdown_connections 2024-04-25T11:23:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7feb580744b0 0x7feb580769c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb7c09b540 0x7feb7c154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c075b10 0x7feb7c14a900 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 --2- 172.21.15.7:0/1924318723 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb7c06cab0 0x7feb7c1f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 -- 172.21.15.7:0/1924318723 >> 172.21.15.7:0/1924318723 conn(0x7feb7c073ba0 msgr2=0x7feb7c158000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 -- 172.21.15.7:0/1924318723 shutdown_connections 2024-04-25T11:23:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:13.478+0000 7feb81200640 1 -- 172.21.15.7:0/1924318723 wait complete. 2024-04-25T11:23:13.501 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":544,"stamp":"2024-04-25T11:23:12.086138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":287004,"kb_used_data":16744,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687716,"statfs":{"total":773094113280,"available":772800221184,"internally_reserved":0,"allocated":17145856,"data_stored":8572480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003223"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":388,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T11:22:59.253029+0000","last_change":"2024-04-25T11:10:11.379293+0000","last_active":"2024-04-25T11:22:59.253029+0000","last_peered":"2024-04-25T11:22:59.253029+0000","last_clean":"2024-04-25T11:22:59.253029+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:22:59.253029+0000","last_undegraded":"2024-04-25T11:22:59.253029+0000","last_fullsized":"2024-04-25T11:22:59.253029+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T11:10:10.320639+0000","last_clean_scrub_stamp":"2024-04-25T11:10:10.320639+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:42.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33708,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338132,"statfs":{"total":96636764160,"available":96602247168,"internally_reserved":0,"allocated":2588672,"data_stored":1514336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33128,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37288,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334552,"statfs":{"total":96636764160,"available":96598581248,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509598,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509599,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509598,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37872,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333968,"statfs":{"total":96636764160,"available":96597983232,"internally_reserved":0,"allocated":2588672,"data_stored":1514336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542303,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33132,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338708,"statfs":{"total":96636764160,"available":96602836992,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:23:13.504 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T11:10:10.320639+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=10, tm_sec=10, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=11, tm_min=23, tm_sec=4, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T11:23:13.504 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T11:23:14.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:23:14.146 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:23:33.506 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T11:23:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.812+0000 7f4e78400640 1 Processor -- start 2024-04-25T11:23:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.812+0000 7f4e78400640 1 -- start start 2024-04-25T11:23:33.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.815+0000 7f4e78400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4e7015ef90 0x7f4e7009f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:33.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.815+0000 7f4e78400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e700af540 0x7f4e700cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:33.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.815+0000 7f4e78400640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e7015f4e0 con 0x7f4e7015bc40 2024-04-25T11:23:33.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.815+0000 7f4e78400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4e70071190 con 0x7f4e7015ef90 2024-04-25T11:23:33.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.815+0000 7f4e78400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e700b0560 con 0x7f4e700af540 2024-04-25T11:23:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.815+0000 7f4e76400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e700af540 0x7f4e700cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.815+0000 7f4e75a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4e7015ef90 0x7f4e7009f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.816+0000 7f4e75a00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4e7015ef90 0x7f4e7009f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.7:56712/0 (socket says 172.21.15.7:56712) 2024-04-25T11:23:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.816+0000 7f4e75a00640 1 -- 172.21.15.7:0/2254932898 learned_addr learned my addr 172.21.15.7:0/2254932898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:33.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.816+0000 7f4e76400640 1 -- 172.21.15.7:0/2254932898 >> v2:172.21.15.87:3300/0 conn(0x7f4e7015ef90 msgr2=0x7f4e7009f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.816+0000 7f4e76400640 1 --2- 172.21.15.7:0/2254932898 >> v2:172.21.15.87:3300/0 conn(0x7f4e7015ef90 0x7f4e7009f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.817+0000 7f4e76400640 1 -- 172.21.15.7:0/2254932898 >> v1:172.21.15.87:6789/0 conn(0x7f4e7015bc40 legacy=0x7f4e700aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:33.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.817+0000 7f4e76400640 1 -- 172.21.15.7:0/2254932898 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e700b0840 con 0x7f4e700af540 2024-04-25T11:23:33.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.818+0000 7f4e76400640 1 --2- 172.21.15.7:0/2254932898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e700af540 0x7f4e700cf980 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009d90 tx=0x7f4e6802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f8dac81de4f28b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:33.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.818+0000 7f4e6fe00640 1 -- 172.21.15.7:0/2254932898 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68031600 con 0x7f4e700af540 2024-04-25T11:23:33.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.818+0000 7f4e6fe00640 1 -- 172.21.15.7:0/2254932898 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e680317a0 con 0x7f4e700af540 2024-04-25T11:23:33.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.818+0000 7f4e6fe00640 1 -- 172.21.15.7:0/2254932898 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68031b70 con 0x7f4e700af540 2024-04-25T11:23:33.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.819+0000 7f4e78400640 1 -- 172.21.15.7:0/2254932898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e700af540 msgr2=0x7f4e700cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:33.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.819+0000 7f4e78400640 1 --2- 172.21.15.7:0/2254932898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e700af540 0x7f4e700cf980 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009d90 tx=0x7f4e6802a5d0 comp rx=0 tx=0).stop 2024-04-25T11:23:33.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.819+0000 7f4e78400640 1 -- 172.21.15.7:0/2254932898 shutdown_connections 2024-04-25T11:23:33.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.819+0000 7f4e78400640 1 --2- 172.21.15.7:0/2254932898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e700af540 0x7f4e700cf980 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:33.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.819+0000 7f4e78400640 1 --2- 172.21.15.7:0/2254932898 >> v2:172.21.15.87:3300/0 conn(0x7f4e7015ef90 0x7f4e7009f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:33.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.819+0000 7f4e78400640 1 -- 172.21.15.7:0/2254932898 >> 172.21.15.7:0/2254932898 conn(0x7f4e70074200 msgr2=0x7f4e7006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:33.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.819+0000 7f4e78400640 1 -- 172.21.15.7:0/2254932898 shutdown_connections 2024-04-25T11:23:33.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.820+0000 7f4e78400640 1 -- 172.21.15.7:0/2254932898 wait complete. 2024-04-25T11:23:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.822+0000 7f4e78400640 1 Processor -- start 2024-04-25T11:23:33.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.822+0000 7f4e78400640 1 -- start start 2024-04-25T11:23:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e78400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7015bc40 0x7f4e700cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e78400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7015ef90 0x7f4e70151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e78400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e700b28e0 0x7f4e700b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e78400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e700b1320 con 0x7f4e7015bc40 2024-04-25T11:23:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e78400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e700b0e00 con 0x7f4e7015ef90 2024-04-25T11:23:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e78400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e700afb80 con 0x7f4e700b28e0 2024-04-25T11:23:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e75a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7015bc40 0x7f4e700cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e75a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7015bc40 0x7f4e700cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55280/0 (socket says 172.21.15.7:55280) 2024-04-25T11:23:33.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e75a00640 1 -- 172.21.15.7:0/2980359559 learned_addr learned my addr 172.21.15.7:0/2980359559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e75a00640 1 -- 172.21.15.7:0/2980359559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e700b28e0 msgr2=0x7f4e700b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e76400640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e700b28e0 0x7f4e700b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e75a00640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e700b28e0 0x7f4e700b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e75000640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7015ef90 0x7f4e70151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e75a00640 1 -- 172.21.15.7:0/2980359559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7015ef90 msgr2=0x7f4e70151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.824+0000 7f4e75a00640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7015ef90 0x7f4e70151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.825+0000 7f4e75a00640 1 -- 172.21.15.7:0/2980359559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e70158680 con 0x7f4e7015bc40 2024-04-25T11:23:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.825+0000 7f4e75a00640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7015bc40 0x7f4e700cb3f0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400e840 tx=0x7f4e6400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:33.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.825+0000 7f4e6d600640 1 -- 172.21.15.7:0/2980359559 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64017070 con 0x7f4e7015bc40 2024-04-25T11:23:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.825+0000 7f4e6d600640 1 -- 172.21.15.7:0/2980359559 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e6400c8e0 con 0x7f4e7015bc40 2024-04-25T11:23:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.825+0000 7f4e78400640 1 -- 172.21.15.7:0/2980359559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e700cb930 con 0x7f4e7015bc40 2024-04-25T11:23:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.825+0000 7f4e6d600640 1 -- 172.21.15.7:0/2980359559 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6400ccd0 con 0x7f4e7015bc40 2024-04-25T11:23:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.825+0000 7f4e78400640 1 -- 172.21.15.7:0/2980359559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e701f6e50 con 0x7f4e7015bc40 2024-04-25T11:23:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.827+0000 7f4e78400640 1 -- 172.21.15.7:0/2980359559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7006a100 con 0x7f4e7015bc40 2024-04-25T11:23:33.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.828+0000 7f4e6d600640 1 -- 172.21.15.7:0/2980359559 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f4e6401a070 con 0x7f4e7015bc40 2024-04-25T11:23:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.829+0000 7f4e6d600640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e500744b0 0x7f4e500769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.829+0000 7f4e6d600640 1 -- 172.21.15.7:0/2980359559 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f4e640950d0 con 0x7f4e7015bc40 2024-04-25T11:23:33.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.829+0000 7f4e75000640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e500744b0 0x7f4e500769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:33.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.830+0000 7f4e75000640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e500744b0 0x7f4e500769c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4e700b3510 tx=0x7f4e5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:33.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:33.835+0000 7f4e6d600640 1 -- 172.21.15.7:0/2980359559 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f4e6405f930 con 0x7f4e7015bc40 2024-04-25T11:23:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.034+0000 7f4e78400640 1 -- 172.21.15.7:0/2980359559 --> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e700ce500 con 0x7f4e500744b0 2024-04-25T11:23:34.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.036+0000 7f4e6d600640 1 -- 172.21.15.7:0/2980359559 <== mgr.4101 v2:172.21.15.7:6832/2350428036 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13365 (secure 0 0 0) 0x7f4e700ce500 con 0x7f4e500744b0 2024-04-25T11:23:34.031 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:23:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-04-25T11:23:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 -- 172.21.15.7:0/2980359559 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e500744b0 msgr2=0x7f4e500769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e500744b0 0x7f4e500769c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4e700b3510 tx=0x7f4e5c009040 comp rx=0 tx=0).stop 2024-04-25T11:23:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 -- 172.21.15.7:0/2980359559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7015bc40 msgr2=0x7f4e700cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7015bc40 0x7f4e700cb3f0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400e840 tx=0x7f4e6400ed00 comp rx=0 tx=0).stop 2024-04-25T11:23:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 -- 172.21.15.7:0/2980359559 shutdown_connections 2024-04-25T11:23:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f4e500744b0 0x7f4e500769c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e700b28e0 0x7f4e700b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e7015ef90 0x7f4e70151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 --2- 172.21.15.7:0/2980359559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e7015bc40 0x7f4e700cb3f0 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 -- 172.21.15.7:0/2980359559 >> 172.21.15.7:0/2980359559 conn(0x7f4e70074200 msgr2=0x7f4e7015e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 -- 172.21.15.7:0/2980359559 shutdown_connections 2024-04-25T11:23:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.039+0000 7f4e78400640 1 -- 172.21.15.7:0/2980359559 wait complete. 2024-04-25T11:23:34.062 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":554,"stamp":"2024-04-25T11:23:32.092353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":286992,"kb_used_data":16744,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687728,"statfs":{"total":773094113280,"available":772800233472,"internally_reserved":0,"allocated":17145856,"data_stored":8572480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003952"},"pg_stats":[{"pgid":"1.0","version":"159'74","reported_seq":399,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T11:23:13.142697+0000","last_change":"2024-04-25T11:23:13.142851+0000","last_active":"2024-04-25T11:23:13.142697+0000","last_peered":"2024-04-25T11:23:13.142697+0000","last_clean":"2024-04-25T11:23:13.142697+0000","last_became_active":"2024-04-25T11:10:11.371822+0000","last_became_peered":"2024-04-25T11:10:11.371822+0000","last_unstale":"2024-04-25T11:23:13.142697+0000","last_undegraded":"2024-04-25T11:23:13.142697+0000","last_fullsized":"2024-04-25T11:23:13.142697+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'74","last_scrub_stamp":"2024-04-25T11:23:13.142459+0000","last_deep_scrub":"159'74","last_deep_scrub_stamp":"2024-04-25T11:23:13.142459+0000","last_clean_scrub_stamp":"2024-04-25T11:23:13.142459+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:33.568142+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476898,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33704,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338136,"statfs":{"total":96636764160,"available":96602251264,"internally_reserved":0,"allocated":2588672,"data_stored":1514336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33128,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37288,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334552,"statfs":{"total":96636764160,"available":96598581248,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509602,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509603,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509602,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37864,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333976,"statfs":{"total":96636764160,"available":96597991424,"internally_reserved":0,"allocated":2588672,"data_stored":1514336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542307,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33132,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338708,"statfs":{"total":96636764160,"available":96602836992,"internally_reserved":0,"allocated":1994752,"data_stored":923968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T11:23:34.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T11:23:34.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.339+0000 7f9a47e00640 1 Processor -- start 2024-04-25T11:23:34.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.339+0000 7f9a47e00640 1 -- start start 2024-04-25T11:23:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.342+0000 7f9a47e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a480d0480 0x7f9a480d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.342+0000 7f9a47e00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a480ce570 con 0x7f9a4806eea0 2024-04-25T11:23:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.342+0000 7f9a47e00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a480ce850 con 0x7f9a481583c0 2024-04-25T11:23:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.342+0000 7f9a47e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4815f190 con 0x7f9a480d0480 2024-04-25T11:23:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.342+0000 7f9a47400640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9a4806eea0 0x7f9a480b0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36686/0 (socket says 172.21.15.7:36686) 2024-04-25T11:23:34.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.342+0000 7f9a47400640 1 -- 172.21.15.7:0/608917753 learned_addr learned my addr 172.21.15.7:0/608917753 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.343+0000 7f9a46000640 1 --2- 172.21.15.7:0/608917753 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a480d0480 0x7f9a480d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.343+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2882653847 0 0) 0x7f9a480ce570 con 0x7f9a4806eea0 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.343+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a2c003520 con 0x7f9a4806eea0 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.343+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3998088073 0 0) 0x7f9a2c003520 con 0x7f9a4806eea0 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.343+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a480d0480 msgr2=0x7f9a480d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.343+0000 7f9a45600640 1 --2- 172.21.15.7:0/608917753 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a480d0480 0x7f9a480d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.343+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 >> v1:172.21.15.87:6789/0 conn(0x7f9a481583c0 legacy=0x7f9a480cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.343+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a480ceb30 con 0x7f9a4806eea0 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.343+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1195050550 0 0) 0x7f9a400019f0 con 0x7f9a4806eea0 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.343+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 >> v1:172.21.15.7:6789/0 conn(0x7f9a4806eea0 legacy=0x7f9a480b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a45600640 1 --2- 172.21.15.7:0/608917753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a2c0039e0 0x7f9a2c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a45600640 1 --2- 172.21.15.7:0/608917753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c004800 0x7f9a2c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a45600640 1 --2- 172.21.15.7:0/608917753 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a480d0480 0x7f9a2c0052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4815f190 con 0x7f9a2c004800 2024-04-25T11:23:34.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c00bb80 con 0x7f9a2c0039e0 2024-04-25T11:23:34.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2c00bd80 con 0x7f9a480d0480 2024-04-25T11:23:34.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a46000640 1 --2- 172.21.15.7:0/608917753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c004800 0x7f9a2c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a46a00640 1 --2- 172.21.15.7:0/608917753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a2c0039e0 0x7f9a2c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a46000640 1 -- 172.21.15.7:0/608917753 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a480d0480 msgr2=0x7f9a2c0052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:34.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a46000640 1 --2- 172.21.15.7:0/608917753 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a480d0480 0x7f9a2c0052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a46000640 1 -- 172.21.15.7:0/608917753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a2c0039e0 msgr2=0x7f9a2c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:34.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a46000640 1 --2- 172.21.15.7:0/608917753 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a2c0039e0 0x7f9a2c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.344+0000 7f9a46000640 1 -- 172.21.15.7:0/608917753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c003520 con 0x7f9a2c004800 2024-04-25T11:23:34.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.346+0000 7f9a46000640 1 --2- 172.21.15.7:0/608917753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c004800 0x7f9a2c004c10 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009a90 tx=0x7f9a2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f3ef4193f26c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:34.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.346+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34036070 con 0x7f9a2c004800 2024-04-25T11:23:34.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.346+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a34031400 con 0x7f9a2c004800 2024-04-25T11:23:34.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.346+0000 7f9a45600640 1 -- 172.21.15.7:0/608917753 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34009690 con 0x7f9a2c004800 2024-04-25T11:23:34.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.347+0000 7f9a47e00640 1 -- 172.21.15.7:0/608917753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c004800 msgr2=0x7f9a2c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.347+0000 7f9a47e00640 1 --2- 172.21.15.7:0/608917753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c004800 0x7f9a2c004c10 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f9a34009a90 tx=0x7f9a2c005830 comp rx=0 tx=0).stop 2024-04-25T11:23:34.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.347+0000 7f9a47400640 1 -- 172.21.15.7:0/608917753 reap_dead start 2024-04-25T11:23:34.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.347+0000 7f9a47e00640 1 -- 172.21.15.7:0/608917753 shutdown_connections 2024-04-25T11:23:34.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.347+0000 7f9a47e00640 1 -- 172.21.15.7:0/608917753 >> 172.21.15.7:0/608917753 conn(0x7f9a480740f0 msgr2=0x7f9a4806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:34.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.347+0000 7f9a47e00640 1 -- 172.21.15.7:0/608917753 shutdown_connections 2024-04-25T11:23:34.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.347+0000 7f9a47e00640 1 -- 172.21.15.7:0/608917753 wait complete. 2024-04-25T11:23:34.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.349+0000 7f9a47e00640 1 Processor -- start 2024-04-25T11:23:34.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.350+0000 7f9a47e00640 1 -- start start 2024-04-25T11:23:34.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.351+0000 7f9a47e00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a2c004800 0x7f9a480b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.351+0000 7f9a47e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a4815cee0 0x7f9a48148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.351+0000 7f9a47e00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4815d2f0 0x7f9a48149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.351+0000 7f9a47e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a480cfd20 con 0x7f9a4815cee0 2024-04-25T11:23:34.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.351+0000 7f9a47e00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9a480cf0f0 con 0x7f9a2c004800 2024-04-25T11:23:34.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.351+0000 7f9a47e00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9a480b0e70 con 0x7f9a4815d2f0 2024-04-25T11:23:34.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.351+0000 7f9a46000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a4815cee0 0x7f9a48148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.352+0000 7f9a46000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a4815cee0 0x7f9a48148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55292/0 (socket says 172.21.15.7:55292) 2024-04-25T11:23:34.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.352+0000 7f9a46000640 1 -- 172.21.15.7:0/2556010940 learned_addr learned my addr 172.21.15.7:0/2556010940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:34.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.352+0000 7f9a47400640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4815d2f0 0x7f9a48149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.352+0000 7f9a46a00640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a2c004800 0x7f9a480b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.352+0000 7f9a46000640 1 -- 172.21.15.7:0/2556010940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4815d2f0 msgr2=0x7f9a48149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.352+0000 7f9a46000640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4815d2f0 0x7f9a48149530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.352+0000 7f9a46000640 1 -- 172.21.15.7:0/2556010940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a2c004800 msgr2=0x7f9a480b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.352+0000 7f9a46000640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a2c004800 0x7f9a480b62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.352+0000 7f9a46000640 1 -- 172.21.15.7:0/2556010940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a480b4d50 con 0x7f9a4815cee0 2024-04-25T11:23:34.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.353+0000 7f9a46000640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a4815cee0 0x7f9a48148ff0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f9a34034040 tx=0x7f9a34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:34.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.353+0000 7f9a3ea00640 1 -- 172.21.15.7:0/2556010940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34041020 con 0x7f9a4815cee0 2024-04-25T11:23:34.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.353+0000 7f9a3ea00640 1 -- 172.21.15.7:0/2556010940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a3403c470 con 0x7f9a4815cee0 2024-04-25T11:23:34.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.353+0000 7f9a47e00640 1 -- 172.21.15.7:0/2556010940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a480b5030 con 0x7f9a4815cee0 2024-04-25T11:23:34.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.353+0000 7f9a3ea00640 1 -- 172.21.15.7:0/2556010940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a34036040 con 0x7f9a4815cee0 2024-04-25T11:23:34.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.353+0000 7f9a47e00640 1 -- 172.21.15.7:0/2556010940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a480b55f0 con 0x7f9a4815cee0 2024-04-25T11:23:34.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.354+0000 7f9a47e00640 1 -- 172.21.15.7:0/2556010940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a48050740 con 0x7f9a4815cee0 2024-04-25T11:23:34.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.355+0000 7f9a3ea00640 1 -- 172.21.15.7:0/2556010940 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f9a34044020 con 0x7f9a4815cee0 2024-04-25T11:23:34.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.356+0000 7f9a3ea00640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 0x7f9a280769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.357+0000 7f9a3ea00640 1 -- 172.21.15.7:0/2556010940 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f9a340b5a00 con 0x7f9a4815cee0 2024-04-25T11:23:34.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.357+0000 7f9a46a00640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 0x7f9a280769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.358+0000 7f9a46a00640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 0x7f9a280769c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9a38007fd0 tx=0x7f9a38009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:34.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.363+0000 7f9a3ea00640 1 -- 172.21.15.7:0/2556010940 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f9a3404b030 con 0x7f9a4815cee0 2024-04-25T11:23:34.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.571+0000 7f9a47e00640 1 -- 172.21.15.7:0/2556010940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f9a48070dd0 con 0x7f9a4815cee0 2024-04-25T11:23:34.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.579+0000 7f9a3ea00640 1 -- 172.21.15.7:0/2556010940 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v68)=0 v68) ==== 123+0+0 (secure 0 0 0) 0x7f9a340802f0 con 0x7f9a4815cee0 2024-04-25T11:23:34.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.580+0000 7f9a3ea00640 1 -- 172.21.15.7:0/2556010940 <== mon.0 v2:172.21.15.7:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9a340bb080 con 0x7f9a4815cee0 2024-04-25T11:23:34.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.583+0000 7f9a47e00640 1 -- 172.21.15.7:0/2556010940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 msgr2=0x7f9a280769c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.583+0000 7f9a47e00640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 0x7f9a280769c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9a38007fd0 tx=0x7f9a38009040 comp rx=0 tx=0).stop 2024-04-25T11:23:34.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.583+0000 7f9a47e00640 1 -- 172.21.15.7:0/2556010940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a4815cee0 msgr2=0x7f9a48148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.583+0000 7f9a47e00640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a4815cee0 0x7f9a48148ff0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f9a34034040 tx=0x7f9a34004730 comp rx=0 tx=0).stop 2024-04-25T11:23:34.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.583+0000 7f9a47e00640 1 -- 172.21.15.7:0/2556010940 shutdown_connections 2024-04-25T11:23:34.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.583+0000 7f9a47e00640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f9a280744b0 0x7f9a280769c0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.583+0000 7f9a47e00640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4815d2f0 0x7f9a48149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.583+0000 7f9a47e00640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a4815cee0 0x7f9a48148ff0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.583+0000 7f9a47e00640 1 --2- 172.21.15.7:0/2556010940 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a2c004800 0x7f9a480b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.583+0000 7f9a47e00640 1 -- 172.21.15.7:0/2556010940 >> 172.21.15.7:0/2556010940 conn(0x7f9a480740f0 msgr2=0x7f9a480b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:34.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.584+0000 7f9a47e00640 1 -- 172.21.15.7:0/2556010940 shutdown_connections 2024-04-25T11:23:34.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.584+0000 7f9a47e00640 1 -- 172.21.15.7:0/2556010940 wait complete. 2024-04-25T11:23:34.608 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T11:23:34.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T11:23:34.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.896+0000 7f6b21c00640 1 Processor -- start 2024-04-25T11:23:34.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.897+0000 7f6b21c00640 1 -- start start 2024-04-25T11:23:34.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.899+0000 7f6b21c00640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6b1c0b9010 0x7f6b1c161340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.899+0000 7f6b21c00640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6b1c09c000 0x7f6b1c0d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.899+0000 7f6b21c00640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b1c160740 con 0x7f6b1c0b2ae0 2024-04-25T11:23:34.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.899+0000 7f6b21c00640 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1c161880 con 0x7f6b1c09c000 2024-04-25T11:23:34.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.899+0000 7f6b21c00640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1c06dc20 con 0x7f6b1c0b9010 2024-04-25T11:23:34.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.899+0000 7f6b21200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6b1c09c000 0x7f6b1c0d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.900+0000 7f6b21200640 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6b1c09c000 0x7f6b1c0d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55308/0 (socket says 172.21.15.7:55308) 2024-04-25T11:23:34.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.900+0000 7f6b21200640 1 -- 172.21.15.7:0/1548847954 learned_addr learned my addr 172.21.15.7:0/1548847954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:34.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.900+0000 7f6b1be00640 1 --2- 172.21.15.7:0/1548847954 >> v2:172.21.15.87:3300/0 conn(0x7f6b1c0b9010 0x7f6b1c161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.900+0000 7f6b21200640 1 -- 172.21.15.7:0/1548847954 >> v2:172.21.15.87:3300/0 conn(0x7f6b1c0b9010 msgr2=0x7f6b1c161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T11:23:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.900+0000 7f6b21200640 1 --2- 172.21.15.7:0/1548847954 >> v2:172.21.15.87:3300/0 conn(0x7f6b1c0b9010 0x7f6b1c161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.900+0000 7f6b21200640 1 -- 172.21.15.7:0/1548847954 >> v1:172.21.15.7:6789/0 conn(0x7f6b1c0b2ae0 legacy=0x7f6b1c09b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.900+0000 7f6b21200640 1 -- 172.21.15.7:0/1548847954 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c06df00 con 0x7f6b1c09c000 2024-04-25T11:23:34.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.901+0000 7f6b21200640 1 --2- 172.21.15.7:0/1548847954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b1c09c000 0x7f6b1c0d1e20 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c074650 tx=0x7f6b1402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508d1b497dc1d8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:34.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.901+0000 7f6b1b400640 1 -- 172.21.15.7:0/1548847954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14036070 con 0x7f6b1c09c000 2024-04-25T11:23:34.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.902+0000 7f6b1b400640 1 -- 172.21.15.7:0/1548847954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6b14031600 con 0x7f6b1c09c000 2024-04-25T11:23:34.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.902+0000 7f6b1b400640 1 -- 172.21.15.7:0/1548847954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b140319f0 con 0x7f6b1c09c000 2024-04-25T11:23:34.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.902+0000 7f6b21c00640 1 -- 172.21.15.7:0/1548847954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b1c09c000 msgr2=0x7f6b1c0d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.902+0000 7f6b21c00640 1 --2- 172.21.15.7:0/1548847954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b1c09c000 0x7f6b1c0d1e20 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c074650 tx=0x7f6b1402a6a0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.902+0000 7f6b21c00640 1 -- 172.21.15.7:0/1548847954 shutdown_connections 2024-04-25T11:23:34.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.902+0000 7f6b21c00640 1 --2- 172.21.15.7:0/1548847954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b1c09c000 0x7f6b1c0d1e20 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.902+0000 7f6b21c00640 1 --2- 172.21.15.7:0/1548847954 >> v2:172.21.15.87:3300/0 conn(0x7f6b1c0b9010 0x7f6b1c161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.902+0000 7f6b21c00640 1 -- 172.21.15.7:0/1548847954 >> 172.21.15.7:0/1548847954 conn(0x7f6b1c0b5210 msgr2=0x7f6b1c0b5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:34.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.902+0000 7f6b21c00640 1 -- 172.21.15.7:0/1548847954 shutdown_connections 2024-04-25T11:23:34.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.902+0000 7f6b21c00640 1 -- 172.21.15.7:0/1548847954 wait complete. 2024-04-25T11:23:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.905+0000 7f6b21c00640 1 Processor -- start 2024-04-25T11:23:34.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.905+0000 7f6b21c00640 1 -- start start 2024-04-25T11:23:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.906+0000 7f6b21c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b1c0b2ae0 0x7f6b1c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.906+0000 7f6b21c00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b1c0b9010 0x7f6b1c1f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.906+0000 7f6b21c00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b1c14f080 0x7f6b1c14e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.906+0000 7f6b21c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c06e7a0 con 0x7f6b1c0b2ae0 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.906+0000 7f6b21c00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c06e4c0 con 0x7f6b1c14f080 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.906+0000 7f6b21c00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c06ea80 con 0x7f6b1c0b9010 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.907+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b1c0b2ae0 0x7f6b1c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.907+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b1c0b2ae0 0x7f6b1c0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55322/0 (socket says 172.21.15.7:55322) 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.907+0000 7f6b1be00640 1 -- 172.21.15.7:0/1626149310 learned_addr learned my addr 172.21.15.7:0/1626149310 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.907+0000 7f6b1be00640 1 -- 172.21.15.7:0/1626149310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b1c0b9010 msgr2=0x7f6b1c1f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.907+0000 7f6b13e00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b1c0b9010 0x7f6b1c1f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.907+0000 7f6b21200640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b1c14f080 0x7f6b1c14e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.907+0000 7f6b1be00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b1c0b9010 0x7f6b1c1f5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.907+0000 7f6b1be00640 1 -- 172.21.15.7:0/1626149310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b1c14f080 msgr2=0x7f6b1c14e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.907+0000 7f6b1be00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b1c14f080 0x7f6b1c14e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:34.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.907+0000 7f6b1be00640 1 -- 172.21.15.7:0/1626149310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c0d0000 con 0x7f6b1c0b2ae0 2024-04-25T11:23:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.907+0000 7f6b21200640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b1c14f080 0x7f6b1c14e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T11:23:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.908+0000 7f6b1be00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b1c0b2ae0 0x7f6b1c0cf7b0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00d600 tx=0x7f6b0c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.908+0000 7f6b18c00640 1 -- 172.21.15.7:0/1626149310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c004240 con 0x7f6b1c0b2ae0 2024-04-25T11:23:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.908+0000 7f6b18c00640 1 -- 172.21.15.7:0/1626149310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6b0c0043e0 con 0x7f6b1c0b2ae0 2024-04-25T11:23:34.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.908+0000 7f6b18c00640 1 -- 172.21.15.7:0/1626149310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c003850 con 0x7f6b1c0b2ae0 2024-04-25T11:23:34.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.908+0000 7f6b21c00640 1 -- 172.21.15.7:0/1626149310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c0d02e0 con 0x7f6b1c0b2ae0 2024-04-25T11:23:34.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.908+0000 7f6b21c00640 1 -- 172.21.15.7:0/1626149310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c14db90 con 0x7f6b1c0b2ae0 2024-04-25T11:23:34.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.909+0000 7f6b21c00640 1 -- 172.21.15.7:0/1626149310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c050740 con 0x7f6b1c0b2ae0 2024-04-25T11:23:34.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.911+0000 7f6b18c00640 1 -- 172.21.15.7:0/1626149310 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 96782+0+0 (secure 0 0 0) 0x7f6b0c0039f0 con 0x7f6b1c0b2ae0 2024-04-25T11:23:34.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.912+0000 7f6b18c00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6af80744e0 0x7f6af80769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T11:23:34.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.912+0000 7f6b18c00640 1 -- 172.21.15.7:0/1626149310 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5335+0+0 (secure 0 0 0) 0x7f6b0c095400 con 0x7f6b1c0b2ae0 2024-04-25T11:23:34.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.913+0000 7f6b13e00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6af80744e0 0x7f6af80769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T11:23:34.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.913+0000 7f6b13e00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6af80744e0 0x7f6af80769f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c152af0 tx=0x7f6b04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T11:23:34.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:34.917+0000 7f6b18c00640 1 -- 172.21.15.7:0/1626149310 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187246 (secure 0 0 0) 0x7f6b0c05fc60 con 0x7f6b1c0b2ae0 2024-04-25T11:23:35.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.162+0000 7f6b21c00640 1 -- 172.21.15.7:0/1626149310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b1c0b8a90 con 0x7f6b1c0b2ae0 2024-04-25T11:23:35.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.164+0000 7f6b18c00640 1 -- 172.21.15.7:0/1626149310 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+3865 (secure 0 0 0) 0x7f6b0c05f560 con 0x7f6b1c0b2ae0 2024-04-25T11:23:35.158 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-04-25T11:23:35.158 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7588,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/2938174159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":2938174159},{"type":"v1","addr":"172.21.15.87:6834","nonce":2938174159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":7591,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/793358721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":793358721},{"type":"v1","addr":"172.21.15.87:6835","nonce":793358721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":9447,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/2654444897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2654444897},{"type":"v1","addr":"172.21.15.7:6835","nonce":2654444897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":110},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/1756191485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6836","nonce":1756191485},{"type":"v1","addr":"172.21.15.7:6837","nonce":1756191485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":118}],"filesystems":[]} 2024-04-25T11:23:35.158 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 118 2024-04-25T11:23:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.166+0000 7f6b21c00640 1 -- 172.21.15.7:0/1626149310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6af80744e0 msgr2=0x7f6af80769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.166+0000 7f6b21c00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6af80744e0 0x7f6af80769f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c152af0 tx=0x7f6b04009040 comp rx=0 tx=0).stop 2024-04-25T11:23:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.166+0000 7f6b21c00640 1 -- 172.21.15.7:0/1626149310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b1c0b2ae0 msgr2=0x7f6b1c0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T11:23:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.166+0000 7f6b21c00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b1c0b2ae0 0x7f6b1c0cf7b0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00d600 tx=0x7f6b0c00dac0 comp rx=0 tx=0).stop 2024-04-25T11:23:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.167+0000 7f6b21c00640 1 -- 172.21.15.7:0/1626149310 shutdown_connections 2024-04-25T11:23:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.167+0000 7f6b21c00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.7:6832/2350428036,v1:172.21.15.7:6833/2350428036] conn(0x7f6af80744e0 0x7f6af80769f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.167+0000 7f6b21c00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b1c14f080 0x7f6b1c14e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.167+0000 7f6b21c00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b1c0b9010 0x7f6b1c1f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.167+0000 7f6b21c00640 1 --2- 172.21.15.7:0/1626149310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b1c0b2ae0 0x7f6b1c0cf7b0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T11:23:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.167+0000 7f6b21c00640 1 -- 172.21.15.7:0/1626149310 >> 172.21.15.7:0/1626149310 conn(0x7f6b1c0b5210 msgr2=0x7f6b1c072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T11:23:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.167+0000 7f6b21c00640 1 -- 172.21.15.7:0/1626149310 shutdown_connections 2024-04-25T11:23:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-04-25T11:23:35.167+0000 7f6b21c00640 1 -- 172.21.15.7:0/1626149310 wait complete. 2024-04-25T11:23:35.188 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T11:23:35.188 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T11:23:35.189 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T11:23:35.189 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:35.218 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T11:23:35.218 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T11:23:35.218 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:35.309 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T11:23:35.309 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T11:23:35.309 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:35.347 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T11:23:35.347 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T11:23:35.347 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:35.437 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T11:23:35.437 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T11:23:35.437 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T11:23:35.437 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:35.522 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T11:23:35.522 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T11:23:35.522 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:41.624 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T11:23:41.624 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T11:23:41.625 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:42.092 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-25T11:23:44.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:23:44.287 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:23:47.726 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T11:23:47.726 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T11:23:47.727 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:47.802 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T11:23:47.803 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T11:23:47.803 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:47.901 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T11:23:47.901 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T11:23:47.901 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:48.099 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-25T11:23:48.100 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-25T11:23:53.805 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-25T11:23:53.805 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-25T11:23:54.003 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T11:23:54.004 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T11:23:54.004 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:54.095 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T11:23:54.096 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T11:23:54.096 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:54.163 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T11:23:54.163 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T11:23:54.163 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T11:23:54.163 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:54.257 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T11:23:54.258 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T11:23:54.258 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:23:59.109 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-04-25T11:23:59.109 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-25T11:24:00.359 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T11:24:00.360 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T11:24:00.360 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T11:24:00.360 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:24:00.432 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T11:24:00.433 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T11:24:00.433 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:24:00.499 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T11:24:00.499 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T11:24:00.499 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T11:24:00.538 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T11:24:00.539 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T11:24:00.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'responding to mclientcaps\(revoke\)' | egrep -v 'not advance its oldest_client_tid' | egrep -v 'failing to advance its oldest client/flush tid' | egrep -v 'Too many inodes in cache' | egrep -v 'failing to respond to cache pressure' | egrep -v 'slow requests are blocked' | egrep -v 'failing to respond to capability release' | egrep -v 'MDS cache is too large' | egrep -v '\(MDS_CLIENT_OLDEST_TID\)' | egrep -v '\(MDS_CACHE_OVERSIZED\)' | head -n 1 2024-04-25T11:24:00.657 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:24:00.657 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T11:24:00.854 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:24:00.854 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T11:24:01.000 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:24:01.001 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T11:24:01.133 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:24:01.133 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T11:24:01.297 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:24:01.297 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T11:24:01.476 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:24:01.476 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T11:24:01.609 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:24:01.609 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T11:24:01.742 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:24:01.742 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T11:24:01.920 INFO:tasks.ceph:Archiving mon data... 2024-04-25T11:24:01.922 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113/data/mon.a.tgz 2024-04-25T11:24:01.922 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-04-25T11:24:01.982 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.42oa5dpx5J 2024-04-25T11:24:01.982 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.42oa5dpx5J 2024-04-25T11:24:02.203 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.42oa5dpx5J 2024-04-25T11:24:02.350 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.42oa5dpx5J is 482KB 2024-04-25T11:24:02.462 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.42oa5dpx5J 2024-04-25T11:24:02.521 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113/data/mon.b.tgz 2024-04-25T11:24:02.521 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-04-25T11:24:02.579 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.2cGqPBKYBj 2024-04-25T11:24:02.579 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.2cGqPBKYBj 2024-04-25T11:24:02.784 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.2cGqPBKYBj 2024-04-25T11:24:02.925 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.2cGqPBKYBj is 495KB 2024-04-25T11:24:03.033 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.2cGqPBKYBj 2024-04-25T11:24:03.091 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113/data/mon.c.tgz 2024-04-25T11:24:03.092 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-04-25T11:24:03.144 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.fw0jQXdXdP 2024-04-25T11:24:03.145 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.fw0jQXdXdP 2024-04-25T11:24:03.338 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.fw0jQXdXdP 2024-04-25T11:24:03.480 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.fw0jQXdXdP is 495KB 2024-04-25T11:24:03.593 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.fw0jQXdXdP 2024-04-25T11:24:03.649 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T11:24:03.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T11:24:03.657 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T11:24:03.801 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T11:24:03.858 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T11:24:03.858 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T11:24:03.860 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113/remote/smithi007/crash 2024-04-25T11:24:03.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T11:24:03.952 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113/remote/smithi087/crash 2024-04-25T11:24:03.953 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T11:24:04.038 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T11:24:04.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:24:04.045 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T11:24:04.110 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-04-25T11:24:04.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T11:24:04.263 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T11:24:04.355 INFO:tasks.ceph:Compressing logs... 2024-04-25T11:24:04.355 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:24:04.363 DEBUG:teuthology.orchestra.run.smithi087:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T11:24:04.445 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49030.log 2024-04-25T11:24:04.446 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45161.log 2024-04-25T11:24:04.446 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72319.log 2024-04-25T11:24:04.447 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44679.log 2024-04-25T11:24:04.448 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49030.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49030.log.gz 2024-04-25T11:24:04.448 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43797.log 2024-04-25T11:24:04.449 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.45161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45161.log.gz 2024-04-25T11:24:04.449 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T11:24:04.450 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.44679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43040.log 2024-04-25T11:24:04.450 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44679.log.gz 2024-04-25T11:24:04.450 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76148.log 2024-04-25T11:24:04.451 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72319.log.gz 2024-04-25T11:24:04.451 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43797.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43797.log.gz 2024-04-25T11:24:04.451 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40450.log 2024-04-25T11:24:04.452 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77616.log 2024-04-25T11:24:04.452 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76148.log.gz 2024-04-25T11:24:04.452 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54267.log 2024-04-25T11:24:04.452 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40450.log.gz 2024-04-25T11:24:04.453 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41456.log 2024-04-25T11:24:04.453 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.43040.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43040.log.gz 2024-04-25T11:24:04.453 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T11:24:04.454 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77616.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.77616.log.gz 2024-04-25T11:24:04.454 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46410.log 2024-04-25T11:24:04.455 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.41456.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.33290.log -- replaced with /var/log/ceph/ceph-client.1.41456.log.gz 2024-04-25T11:24:04.455 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-04-25T11:24:04.455 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43533.log 2024-04-25T11:24:04.455 INFO:teuthology.orchestra.run.smithi087.stderr: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T11:24:04.455 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33290.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T11:24:04.456 INFO:teuthology.orchestra.run.smithi087.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33290.log.gz 2024-04-25T11:24:04.456 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.43533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43533.log.gz 2024-04-25T11:24:04.457 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T11:24:04.458 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71137.log 2024-04-25T11:24:04.458 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54267.log.gz 2024-04-25T11:24:04.458 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72840.log 2024-04-25T11:24:04.458 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46410.log.gz 2024-04-25T11:24:04.459 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71810.log 2024-04-25T11:24:04.459 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71137.log.gz 2024-04-25T11:24:04.459 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40096.log 2024-04-25T11:24:04.459 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72840.log.gz 2024-04-25T11:24:04.459 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45306.log 2024-04-25T11:24:04.459 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71810.log.gz 2024-04-25T11:24:04.461 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43587.log 2024-04-25T11:24:04.461 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40096.log.gz 2024-04-25T11:24:04.461 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62020.log 2024-04-25T11:24:04.461 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45306.log.gz 2024-04-25T11:24:04.462 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T11:24:04.463 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33421.log 2024-04-25T11:24:04.463 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33541.log 2024-04-25T11:24:04.463 INFO:teuthology.orchestra.run.smithi087.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T11:24:04.463 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33421.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T11:24:04.463 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33421.log.gz 2024-04-25T11:24:04.463 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39359.log 2024-04-25T11:24:04.464 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.4.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33541.log.gz 2024-04-25T11:24:04.464 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33501.log 2024-04-25T11:24:04.466 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71347.log 2024-04-25T11:24:04.466 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43587.log.gz 2024-04-25T11:24:04.466 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62020.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62020.log.gz 2024-04-25T11:24:04.466 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67689.log 2024-04-25T11:24:04.466 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41445.log 2024-04-25T11:24:04.466 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71347.log.gz 2024-04-25T11:24:04.466 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52408.log 2024-04-25T11:24:04.466 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67689.log.gz 2024-04-25T11:24:04.466 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41109.log 2024-04-25T11:24:04.467 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41445.log.gz 2024-04-25T11:24:04.470 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.39359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39359.log.gz 2024-04-25T11:24:04.470 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33381.log 2024-04-25T11:24:04.470 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44123.log 2024-04-25T11:24:04.470 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33501.log.gz 2024-04-25T11:24:04.470 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33381.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T11:24:04.470 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33381.log.gz 2024-04-25T11:24:04.470 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.44123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44123.log.gz 2024-04-25T11:24:04.470 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72157.log 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52408.log.gz 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72626.log 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41109.log.gz 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43209.log 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72157.log.gz 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57809.log 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72626.log.gz 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75331.log 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43209.log.gz 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57809.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57809.log.gz 2024-04-25T11:24:04.472 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46708.log 2024-04-25T11:24:04.474 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40401.log 2024-04-25T11:24:04.474 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T11:24:04.474 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.40401.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40401.log.gz 2024-04-25T11:24:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62697.log 2024-04-25T11:24:04.476 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75331.log.gz 2024-04-25T11:24:04.476 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63404.log 2024-04-25T11:24:04.476 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46708.log.gz 2024-04-25T11:24:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77868.log 2024-04-25T11:24:04.477 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62697.log.gz 2024-04-25T11:24:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49709.log 2024-04-25T11:24:04.477 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63404.log.gz 2024-04-25T11:24:04.478 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40975.log 2024-04-25T11:24:04.479 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74192.log 2024-04-25T11:24:04.479 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77868.log.gz 2024-04-25T11:24:04.479 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75205.log 2024-04-25T11:24:04.479 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49709.log.gz 2024-04-25T11:24:04.479 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69860.log 2024-04-25T11:24:04.480 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74192.log.gz 2024-04-25T11:24:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33334.log 2024-04-25T11:24:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.40975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40975.log.gz 2024-04-25T11:24:04.480 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41966.log 2024-04-25T11:24:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45642.log 2024-04-25T11:24:04.481 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75205.log.gz 2024-04-25T11:24:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.69860.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69860.log.gz 2024-04-25T11:24:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73322.log 2024-04-25T11:24:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45642.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.45642.log.gz 2024-04-25T11:24:04.483 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62823.log 2024-04-25T11:24:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.73322.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.73322.log.gz 2024-04-25T11:24:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60013.log 2024-04-25T11:24:04.486 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33334.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33334.log.gz 2024-04-25T11:24:04.486 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.41966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33461.log 2024-04-25T11:24:04.486 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41966.log.gz 2024-04-25T11:24:04.486 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33250.log 2024-04-25T11:24:04.486 INFO:teuthology.orchestra.run.smithi087.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T11:24:04.486 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.33461.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33461.log.gz 2024-04-25T11:24:04.486 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39774.log/var/log/ceph/ceph-client.admin.33250.log: 2024-04-25T11:24:04.486 INFO:teuthology.orchestra.run.smithi087.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33250.log.gz 2024-04-25T11:24:04.493 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42464.log 2024-04-25T11:24:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.3.39774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.39774.log.gz 2024-04-25T11:24:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T11:24:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.42464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42464.log.gz 2024-04-25T11:24:04.494 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T11:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57144.log 2024-04-25T11:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62823.log.gz 2024-04-25T11:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52366.log 2024-04-25T11:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60013.log.gz 2024-04-25T11:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57144.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57144.log.gz 2024-04-25T11:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67062.log 2024-04-25T11:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52366.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52366.log.gz 2024-04-25T11:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77784.log 2024-04-25T11:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47419.log 2024-04-25T11:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67062.log.gz 2024-04-25T11:24:04.496 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40717.log 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77784.log.gz 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47419.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47419.log.gz 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65564.log 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75060.log 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40717.log.gz 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38043.log 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65564.log.gz 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75060.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75060.log.gz 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70191.log 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38211.log 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38043.log.gz 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67421.log 2024-04-25T11:24:04.497 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70191.log.gz 2024-04-25T11:24:04.499 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71431.log 2024-04-25T11:24:04.499 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38211.log.gz 2024-04-25T11:24:04.499 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67421.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67421.log.gz 2024-04-25T11:24:04.499 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48012.log 2024-04-25T11:24:04.499 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70605.log 2024-04-25T11:24:04.499 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71431.log.gz 2024-04-25T11:24:04.500 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58840.log 2024-04-25T11:24:04.500 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48012.log.gz 2024-04-25T11:24:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70605.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70605.log.gz 2024-04-25T11:24:04.501 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59699.log 2024-04-25T11:24:04.502 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60556.log 2024-04-25T11:24:04.502 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58840.log.gz 2024-04-25T11:24:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57244.log 2024-04-25T11:24:04.504 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59699.log.gz 2024-04-25T11:24:04.504 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60556.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60556.log.gz 2024-04-25T11:24:04.505 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43983.log 2024-04-25T11:24:04.505 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68235.log 2024-04-25T11:24:04.506 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57244.log.gz 2024-04-25T11:24:04.506 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57767.log 2024-04-25T11:24:04.507 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43983.log.gz 2024-04-25T11:24:04.507 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65778.log 2024-04-25T11:24:04.508 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68235.log.gz 2024-04-25T11:24:04.508 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57767.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57767.log.gzgzip 2024-04-25T11:24:04.509 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51795.log 2024-04-25T11:24:04.510 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57496.log 2024-04-25T11:24:04.511 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65778.log.gz 2024-04-25T11:24:04.511 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74580.log 2024-04-25T11:24:04.511 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51795.log.gz 2024-04-25T11:24:04.513 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57496.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57496.log.gz 2024-04-25T11:24:04.513 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60963.log 2024-04-25T11:24:04.513 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48174.log 2024-04-25T11:24:04.513 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2%/var/log/ceph/ceph-client.admin.60963.log: -- replaced with /var/log/ceph/ceph-client.admin.74580.log.gz 2024-04-25T11:24:04.514 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72361.log 2024-04-25T11:24:04.514 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60963.log.gz 2024-04-25T11:24:04.515 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48174.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48174.log.gz 2024-04-25T11:24:04.515 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46242.log 2024-04-25T11:24:04.516 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72361.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72361.log.gz 2024-04-25T11:24:04.516 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53855.log 2024-04-25T11:24:04.517 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52887.log 2024-04-25T11:24:04.517 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46242.log.gz 2024-04-25T11:24:04.518 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78541.log 2024-04-25T11:24:04.518 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53855.log.gz 2024-04-25T11:24:04.519 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52887.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52887.log.gz 2024-04-25T11:24:04.519 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74916.log 2024-04-25T11:24:04.520 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50758.log 2024-04-25T11:24:04.521 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78541.log.gz 2024-04-25T11:24:04.521 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62188.log 2024-04-25T11:24:04.521 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74916.log.gz 2024-04-25T11:24:04.522 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50758.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50758.log.gz -5 2024-04-25T11:24:04.522 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62739.log 2024-04-25T11:24:04.523 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49335.log 2024-04-25T11:24:04.523 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62188.log.gz 2024-04-25T11:24:04.524 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62398.log 2024-04-25T11:24:04.524 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62739.log.gz 2024-04-25T11:24:04.525 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43503.log 2024-04-25T11:24:04.525 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49335.log.gz 2024-04-25T11:24:04.526 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78055.log 2024-04-25T11:24:04.527 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62398.log.gz 2024-04-25T11:24:04.527 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67547.log 2024-04-25T11:24:04.528 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43503.log.gz 2024-04-25T11:24:04.528 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78055.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78055.log.gz 2024-04-25T11:24:04.529 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72531.log 2024-04-25T11:24:04.530 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48988.log 2024-04-25T11:24:04.530 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67547.log.gz 2024-04-25T11:24:04.531 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65862.log 2024-04-25T11:24:04.531 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72531.log.gz 2024-04-25T11:24:04.532 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48988.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48988.log.gz 2024-04-25T11:24:04.532 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70893.log 2024-04-25T11:24:04.533 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56958.log 2024-04-25T11:24:04.533 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65862.log.gz 2024-04-25T11:24:04.534 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78625.log 2024-04-25T11:24:04.534 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70893.log.gz 2024-04-25T11:24:04.535 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56958.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56958.log.gz 2024-04-25T11:24:04.535 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75415.log 2024-04-25T11:24:04.536 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75925.log 2024-04-25T11:24:04.537 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78625.log.gz 2024-04-25T11:24:04.537 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75415.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.69396.log -- replaced with /var/log/ceph/ceph-client.admin.75415.log.gz 2024-04-25T11:24:04.537 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-04-25T11:24:04.538 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38939.log 2024-04-25T11:24:04.538 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75925.log.gz 2024-04-25T11:24:04.539 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72668.log 2024-04-25T11:24:04.539 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69396.log.gz 2024-04-25T11:24:04.540 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73026.log 2024-04-25T11:24:04.541 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38939.log.gz 2024-04-25T11:24:04.541 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51272.log 2024-04-25T11:24:04.541 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72668.log.gz 2024-04-25T11:24:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73026.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73026.log.gz 2024-04-25T11:24:04.542 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43083.log 2024-04-25T11:24:04.543 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78097.log 2024-04-25T11:24:04.544 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51272.log.gz 2024-04-25T11:24:04.545 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77016.log 2024-04-25T11:24:04.545 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43083.log.gz 2024-04-25T11:24:04.545 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66936.log 2024-04-25T11:24:04.545 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78097.log.gz 2024-04-25T11:24:04.547 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74234.log 2024-04-25T11:24:04.547 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77016.log.gz 2024-04-25T11:24:04.548 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39639.log 2024-04-25T11:24:04.548 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66936.log.gz 2024-04-25T11:24:04.549 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74234.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74234.log.gz 2024-04-25T11:24:04.549 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50120.log 2024-04-25T11:24:04.550 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58277.log 2024-04-25T11:24:04.550 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39639.log.gz 2024-04-25T11:24:04.551 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46452.log 2024-04-25T11:24:04.552 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50120.log.gz 2024-04-25T11:24:04.552 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58277.log: gzip -5 89.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.58277.log.gz 2024-04-25T11:24:04.552 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.41655.log 2024-04-25T11:24:04.553 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78667.log 2024-04-25T11:24:04.554 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46452.log.gz 2024-04-25T11:24:04.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54141.log 2024-04-25T11:24:04.555 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41655.log.gz 2024-04-25T11:24:04.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78667.log: gzip -5 --verbose 87.8% -- -- replaced with /var/log/ceph/ceph-client.admin.78667.log.gz 2024-04-25T11:24:04.556 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.61538.log 2024-04-25T11:24:04.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40115.log 2024-04-25T11:24:04.557 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54141.log.gz 2024-04-25T11:24:04.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46013.log 88.6% 2024-04-25T11:24:04.558 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61538.log.gz 2024-04-25T11:24:04.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55215.log 2024-04-25T11:24:04.559 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40115.log.gz 2024-04-25T11:24:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39891.log 2024-04-25T11:24:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55215.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46013.log.gz 2024-04-25T11:24:04.561 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55215.log.gz 2024-04-25T11:24:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70977.log 2024-04-25T11:24:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65988.log 2024-04-25T11:24:04.562 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39891.log.gz 2024-04-25T11:24:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45600.log 2024-04-25T11:24:04.564 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70977.log.gz 2024-04-25T11:24:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65988.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65988.log.gz 2024-04-25T11:24:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40719.log 2024-04-25T11:24:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65946.log 2024-04-25T11:24:04.566 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45600.log.gz 2024-04-25T11:24:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40719.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T11:24:04.567 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40719.log.gz 2024-04-25T11:24:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65946.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.52929.log -- replaced with /var/log/ceph/ceph-client.admin.65946.log.gz 2024-04-25T11:24:04.567 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-04-25T11:24:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66156.log 2024-04-25T11:24:04.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40107.log 2024-04-25T11:24:04.570 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52929.log.gz 2024-04-25T11:24:04.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77060.log 2024-04-25T11:24:04.571 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.66156.log.gz 2024-04-25T11:24:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62953.log 2024-04-25T11:24:04.572 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40107.log.gz 2024-04-25T11:24:04.572 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T11:24:04.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77390.log 2024-04-25T11:24:04.573 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77060.log.gz 2024-04-25T11:24:04.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62953.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62953.log.gz 2024-04-25T11:24:04.574 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71095.log 2024-04-25T11:24:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40380.log 2024-04-25T11:24:04.575 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77390.log.gz 2024-04-25T11:24:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78961.log 2024-04-25T11:24:04.577 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71095.log.gz 2024-04-25T11:24:04.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51146.log 2024-04-25T11:24:04.577 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40380.log.gz 2024-04-25T11:24:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47725.log 87.6% 2024-04-25T11:24:04.578 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78961.log.gz 2024-04-25T11:24:04.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46624.log 2024-04-25T11:24:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47725.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51146.log.gz 2024-04-25T11:24:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40783.log 2024-04-25T11:24:04.580 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47725.log.gz 2024-04-25T11:24:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58798.log 2024-04-25T11:24:04.581 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46624.log.gz 2024-04-25T11:24:04.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40783.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40783.log.gz 2024-04-25T11:24:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65480.log 2024-04-25T11:24:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55948.log 2024-04-25T11:24:04.584 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58798.log.gz 2024-04-25T11:24:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65480.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65480.log.gz 2024-04-25T11:24:04.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68319.log 2024-04-25T11:24:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55948.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55948.log.gz 2024-04-25T11:24:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48904.log 2024-04-25T11:24:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43671.log 2024-04-25T11:24:04.587 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68319.log.gz 2024-04-25T11:24:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37659.log 2024-04-25T11:24:04.588 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48904.log.gz 2024-04-25T11:24:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43671.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43671.log.gz 2024-04-25T11:24:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71179.log 2024-04-25T11:24:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61454.log 2024-04-25T11:24:04.591 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37659.log.gz 2024-04-25T11:24:04.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67589.log 2024-04-25T11:24:04.592 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71179.log.gz 2024-04-25T11:24:04.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41361.log 2024-04-25T11:24:04.592 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61454.log.gz 2024-04-25T11:24:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50579.log 2024-04-25T11:24:04.594 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67589.log.gz 2024-04-25T11:24:04.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39393.log 2024-04-25T11:24:04.595 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41361.log.gz 2024-04-25T11:24:04.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50579.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50579.log.gz 2024-04-25T11:24:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67343.log 2024-04-25T11:24:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72756.log 2024-04-25T11:24:04.597 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39393.log.gz 2024-04-25T11:24:04.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59360.log 2024-04-25T11:24:04.598 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67343.log.gz 2024-04-25T11:24:04.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72756.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.48132.log -- replaced with /var/log/ceph/ceph-client.admin.72756.log.gz 2024-04-25T11:24:04.598 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-04-25T11:24:04.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48400.log 2024-04-25T11:24:04.600 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59360.log.gz 2024-04-25T11:24:04.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48132.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T11:24:04.601 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48132.log.gz 2024-04-25T11:24:04.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48400.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48400.log.gz 2024-04-25T11:24:04.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63700.log 2024-04-25T11:24:04.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51398.log 2024-04-25T11:24:04.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.63700.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63700.log.gz 2024-04-25T11:24:04.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71768.log 2024-04-25T11:24:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51398.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51398.log.gz 2024-04-25T11:24:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56916.log 2024-04-25T11:24:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63046.log 2024-04-25T11:24:04.606 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71768.log.gz 2024-04-25T11:24:04.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56916.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56916.log.gz 2024-04-25T11:24:04.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52076.log 2024-04-25T11:24:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65694.log 2024-04-25T11:24:04.608 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63046.log.gz 2024-04-25T11:24:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56299.log 2024-04-25T11:24:04.609 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52076.log.gz 2024-04-25T11:24:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51004.log 2024-04-25T11:24:04.610 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65694.log.gz 2024-04-25T11:24:04.611 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64120.log 2024-04-25T11:24:04.612 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56299.log.gz 2024-04-25T11:24:04.612 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59971.log 2024-04-25T11:24:04.612 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51004.log.gz 2024-04-25T11:24:04.613 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49240.log 2024-04-25T11:24:04.614 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64120.log.gz 2024-04-25T11:24:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59971.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59971.log.gz 2024-04-25T11:24:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44808.log 2024-04-25T11:24:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46182.log 2024-04-25T11:24:04.616 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49240.log.gz 2024-04-25T11:24:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75584.log 2024-04-25T11:24:04.617 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44808.log.gz 2024-04-25T11:24:04.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60514.log 2024-04-25T11:24:04.618 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46182.log.gz 2024-04-25T11:24:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65650.log 2024-04-25T11:24:04.619 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75584.log.gz 2024-04-25T11:24:04.620 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71263.log 2024-04-25T11:24:04.620 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60514.log.gz 2024-04-25T11:24:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50078.log 2024-04-25T11:24:04.621 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65650.log.gz 2024-04-25T11:24:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39849.log 2024-04-25T11:24:04.623 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71263.log.gz 2024-04-25T11:24:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45768.log 2024-04-25T11:24:04.623 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50078.log.gz 2024-04-25T11:24:04.623 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57412.log 2024-04-25T11:24:04.624 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39849.log.gz 2024-04-25T11:24:04.624 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45768.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45768.log.gz 2024-04-25T11:24:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45852.log 2024-04-25T11:24:04.625 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50920.log 2024-04-25T11:24:04.626 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57412.log.gz 2024-04-25T11:24:04.626 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69354.log 2024-04-25T11:24:04.626 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45852.log.gz 2024-04-25T11:24:04.627 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74622.log 2024-04-25T11:24:04.628 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50920.log.gz 2024-04-25T11:24:04.628 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77264.log 2024-04-25T11:24:04.628 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69354.log.gz 2024-04-25T11:24:04.629 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72924.log 2024-04-25T11:24:04.629 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74622.log.gz 2024-04-25T11:24:04.630 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42915.log 2024-04-25T11:24:04.630 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77264.log.gz 2024-04-25T11:24:04.631 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64724.log 2024-04-25T11:24:04.632 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72924.log.gz 2024-04-25T11:24:04.632 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66072.log 2024-04-25T11:24:04.632 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42915.log.gz 2024-04-25T11:24:04.633 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44424.log 2024-04-25T11:24:04.633 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64724.log.gz 2024-04-25T11:24:04.634 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43293.log 2024-04-25T11:24:04.634 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66072.log.gz 2024-04-25T11:24:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38169.log 2024-04-25T11:24:04.641 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44424.log.gz 2024-04-25T11:24:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43293.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43293.log.gz 2024-04-25T11:24:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64808.log 2024-04-25T11:24:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48820.log 2024-04-25T11:24:04.641 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38169.log.gz 2024-04-25T11:24:04.641 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45432.log 2024-04-25T11:24:04.642 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64808.log.gz 2024-04-25T11:24:04.642 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50800.log 2024-04-25T11:24:04.642 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48820.log.gz 2024-04-25T11:24:04.642 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45432.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45432.log.gz 2024-04-25T11:24:04.642 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48358.log 2024-04-25T11:24:04.643 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53813.log 2024-04-25T11:24:04.643 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50800.log.gz 2024-04-25T11:24:04.643 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59402.log 2024-04-25T11:24:04.643 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48358.log.gz 2024-04-25T11:24:04.644 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53813.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T11:24:04.644 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53813.log.gz 2024-04-25T11:24:04.645 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77490.log 2024-04-25T11:24:04.646 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59402.log.gz 2024-04-25T11:24:04.646 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74438.log 2024-04-25T11:24:04.647 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79283.log 2024-04-25T11:24:04.647 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77490.log.gz 2024-04-25T11:24:04.648 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64162.log 2024-04-25T11:24:04.648 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74438.log.gz 2024-04-25T11:24:04.649 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.79283.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T11:24:04.650 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79283.log.gz 2024-04-25T11:24:04.650 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49771.log 2024-04-25T11:24:04.651 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64162.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64162.log.gz 2024-04-25T11:24:04.651 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78499.log 2024-04-25T11:24:04.652 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73068.log 2024-04-25T11:24:04.652 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49771.log.gz 2024-04-25T11:24:04.653 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78499.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78499.log.gz 2024-04-25T11:24:04.653 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60742.log 2024-04-25T11:24:04.654 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75967.log 2024-04-25T11:24:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63278.log 2024-04-25T11:24:04.655 INFO:teuthology.orchestra.run.smithi007.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.73068.log.gz 2024-04-25T11:24:04.655 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60742.log.gz 2024-04-25T11:24:04.656 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68445.log 2024-04-25T11:24:04.656 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75967.log.gz 2024-04-25T11:24:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63278.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63278.log.gz 2024-04-25T11:24:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51188.log 2024-04-25T11:24:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63874.log 2024-04-25T11:24:04.659 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68445.log: /var/log/ceph/ceph-client.admin.51188.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68445.log.gz 2024-04-25T11:24:04.659 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76905.log 2024-04-25T11:24:04.659 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51188.log.gz 2024-04-25T11:24:04.660 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77348.log 2024-04-25T11:24:04.660 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63874.log.gz 2024-04-25T11:24:04.661 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78751.log 2024-04-25T11:24:04.661 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.76905.log.gz 2024-04-25T11:24:04.662 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76236.log 2024-04-25T11:24:04.662 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77348.log.gz 2024-04-25T11:24:04.663 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38981.log 2024-04-25T11:24:04.663 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78751.log.gz 2024-04-25T11:24:04.664 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61353.log 2024-04-25T11:24:04.664 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.76236.log.gz 2024-04-25T11:24:04.665 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59636.log 2024-04-25T11:24:04.665 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38981.log.gz 2024-04-25T11:24:04.666 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40451.log 2024-04-25T11:24:04.666 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61353.log.gz 2024-04-25T11:24:04.667 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71978.log 2024-04-25T11:24:04.667 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59636.log.gz 2024-04-25T11:24:04.668 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57454.log 2024-04-25T11:24:04.668 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40451.log.gz 2024-04-25T11:24:04.669 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71978.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71978.log.gz 2024-04-25T11:24:04.669 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39309.log 2024-04-25T11:24:04.670 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57683.log 2024-04-25T11:24:04.671 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57454.log.gz 2024-04-25T11:24:04.671 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68487.log 2024-04-25T11:24:04.671 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39309.log.gz 2024-04-25T11:24:04.672 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49072.log 2024-04-25T11:24:04.672 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57683.log.gz 2024-04-25T11:24:04.673 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68487.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68487.log.gz 2024-04-25T11:24:04.673 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33453.log 2024-04-25T11:24:04.674 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61580.log 2024-04-25T11:24:04.675 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49072.log.gz 2024-04-25T11:24:04.675 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74396.log 2024-04-25T11:24:04.675 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33453.log.gz 2024-04-25T11:24:04.676 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58588.log 2024-04-25T11:24:04.676 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61580.log.gz 2024-04-25T11:24:04.677 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74396.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74396.log.gz 2024-04-25T11:24:04.677 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75760.log 2024-04-25T11:24:04.678 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61047.log 2024-04-25T11:24:04.679 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58588.log.gz 2024-04-25T11:24:04.679 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46326.log 2024-04-25T11:24:04.679 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75760.log.gz 2024-04-25T11:24:04.681 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61089.log 2024-04-25T11:24:04.681 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61047.log.gz 2024-04-25T11:24:04.681 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78583.log 2024-04-25T11:24:04.682 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46326.log.gz 2024-04-25T11:24:04.682 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79129.log 2024-04-25T11:24:04.682 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61089.log.gz 2024-04-25T11:24:04.683 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61622.log 2024-04-25T11:24:04.683 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78583.log.gz 2024-04-25T11:24:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.79129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62440.log 2024-04-25T11:24:04.685 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79129.log.gz 2024-04-25T11:24:04.685 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45684.log 2024-04-25T11:24:04.685 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61622.log.gz 2024-04-25T11:24:04.686 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64953.log 2024-04-25T11:24:04.686 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62440.log.gz 2024-04-25T11:24:04.687 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45684.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45684.log.gz 2024-04-25T11:24:04.687 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64430.log 2024-04-25T11:24:04.688 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71473.log 2024-04-25T11:24:04.689 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64953.log.gz 2024-04-25T11:24:04.689 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59318.log 2024-04-25T11:24:04.689 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64430.log.gz 2024-04-25T11:24:04.691 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71305.log 2024-04-25T11:24:04.691 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71473.log.gz 2024-04-25T11:24:04.691 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65438.log 2024-04-25T11:24:04.691 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59318.log.gz 2024-04-25T11:24:04.693 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39023.log 2024-04-25T11:24:04.693 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71305.log.gz 2024-04-25T11:24:04.693 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65438.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65438.log.gz 2024-04-25T11:24:04.693 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61978.log 2024-04-25T11:24:04.695 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56409.log 2024-04-25T11:24:04.695 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39023.log.gz 2024-04-25T11:24:04.696 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41487.log 2024-04-25T11:24:04.696 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61978.log.gz 2024-04-25T11:24:04.696 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56409.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56409.log.gz 2024-04-25T11:24:04.697 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54057.log 2024-04-25T11:24:04.697 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54790.log 2024-04-25T11:24:04.698 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41487.log.gz 2024-04-25T11:24:04.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67020.log 2024-04-25T11:24:04.699 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54057.log.gz 2024-04-25T11:24:04.700 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41025.log 2024-04-25T11:24:04.700 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54790.log.gz 2024-04-25T11:24:04.700 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77532.log 2024-04-25T11:24:04.701 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67020.log.gz 2024-04-25T11:24:04.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59910.log 2024-04-25T11:24:04.702 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41025.log.gz 2024-04-25T11:24:04.703 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75457.log 2024-04-25T11:24:04.703 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77532.log.gz 2024-04-25T11:24:04.704 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53393.log 2024-04-25T11:24:04.704 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59910.log.gz 2024-04-25T11:24:04.705 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44766.log 2024-04-25T11:24:04.705 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75457.log.gz 2024-04-25T11:24:04.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.53393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53393.log.gz 2024-04-25T11:24:04.706 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73528.log 2024-04-25T11:24:04.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70935.log 2024-04-25T11:24:04.708 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44766.log.gz 2024-04-25T11:24:04.708 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54099.log 2024-04-25T11:24:04.708 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73528.log.gz 2024-04-25T11:24:04.709 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33186.log 2024-04-25T11:24:04.709 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70935.log.gz 2024-04-25T11:24:04.709 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60598.log 2024-04-25T11:24:04.710 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54099.log.gz 2024-04-25T11:24:04.710 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78331.log 2024-04-25T11:24:04.711 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33186.log.gz 2024-04-25T11:24:04.712 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60598.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60598.log.gz 2024-04-25T11:24:04.712 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41879.log 2024-04-25T11:24:04.713 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78331.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T11:24:04.714 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41879.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78331.log.gz 2024-04-25T11:24:04.715 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41879.log.gz 2024-04-25T11:24:04.715 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48484.log 2024-04-25T11:24:04.716 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40983.log 2024-04-25T11:24:04.717 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53651.log 2024-04-25T11:24:04.717 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48484.log.gz 2024-04-25T11:24:04.719 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73134.log 2024-04-25T11:24:04.719 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53651.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40983.log.gz 2024-04-25T11:24:04.719 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53651.log.gz 2024-04-25T11:24:04.719 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76826.log 2024-04-25T11:24:04.720 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73134.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73134.log.gz 2024-04-25T11:24:04.720 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46666.log 2024-04-25T11:24:04.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.76826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52324.log 2024-04-25T11:24:04.721 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76826.log.gz 2024-04-25T11:24:04.722 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46666.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46666.log.gz 2024-04-25T11:24:04.722 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77826.log 2024-04-25T11:24:04.724 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46960.log 2024-04-25T11:24:04.724 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77826.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52324.log.gz 2024-04-25T11:24:04.725 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77826.log.gz 2024-04-25T11:24:04.725 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52452.log 2024-04-25T11:24:04.726 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46960.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46960.log.gz 2024-04-25T11:24:04.726 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79045.log 2024-04-25T11:24:04.727 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66659.log 2024-04-25T11:24:04.729 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.79045.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52452.log.gz 2024-04-25T11:24:04.729 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79045.log.gz 2024-04-25T11:24:04.729 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62535.log 2024-04-25T11:24:04.730 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72487.log 2024-04-25T11:24:04.732 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62535.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66659.log.gz 2024-04-25T11:24:04.732 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62535.log.gz 2024-04-25T11:24:04.732 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70731.log 2024-04-25T11:24:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72487.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72487.log.gz 2024-04-25T11:24:04.733 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42957.log 2024-04-25T11:24:04.734 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47068.log 2024-04-25T11:24:04.735 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42957.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70731.log.gz 2024-04-25T11:24:04.735 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42957.log.gz 2024-04-25T11:24:04.735 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61664.log 2024-04-25T11:24:04.737 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48736.log 2024-04-25T11:24:04.737 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61664.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47068.log.gz 2024-04-25T11:24:04.738 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61664.log.gz 2024-04-25T11:24:04.738 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69480.log 2024-04-25T11:24:04.739 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48736.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48736.log.gz 2024-04-25T11:24:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27172.log 2024-04-25T11:24:04.740 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57935.log 2024-04-25T11:24:04.741 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.tmp-client.admin.27172.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69480.log.gz 2024-04-25T11:24:04.741 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27172.log.gz 2024-04-25T11:24:04.742 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66030.log 2024-04-25T11:24:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57935.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57935.log.gz 2024-04-25T11:24:04.744 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67217.log 2024-04-25T11:24:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51608.log 2024-04-25T11:24:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67217.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66030.log.gz 2024-04-25T11:24:04.746 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67217.log.gz 2024-04-25T11:24:04.746 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48652.log 2024-04-25T11:24:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77102.log 2024-04-25T11:24:04.748 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48652.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51608.log.gz 2024-04-25T11:24:04.750 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48652.log.gz 2024-04-25T11:24:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47256.log 2024-04-25T11:24:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77102.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77102.log.gz 2024-04-25T11:24:04.750 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64556.log 2024-04-25T11:24:04.752 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.47256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54748.log 2024-04-25T11:24:04.752 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47256.log.gz 2024-04-25T11:24:04.752 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64556.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64556.log.gz 2024-04-25T11:24:04.752 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66573.log 2024-04-25T11:24:04.753 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65037.log 2024-04-25T11:24:04.754 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66573.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54748.log.gz 2024-04-25T11:24:04.755 INFO:teuthology.orchestra.run.smithi007.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.66573.log.gz 2024-04-25T11:24:04.755 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58990.log 2024-04-25T11:24:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67815.log 2024-04-25T11:24:04.757 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58990.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65037.log.gz 2024-04-25T11:24:04.757 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58990.log.gz 2024-04-25T11:24:04.757 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71642.log 2024-04-25T11:24:04.758 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50674.log 2024-04-25T11:24:04.759 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67815.log.gz 2024-04-25T11:24:04.759 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71642.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.71642.log.gz 2024-04-25T11:24:04.760 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72062.log 2024-04-25T11:24:04.761 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33555.log 2024-04-25T11:24:04.761 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50674.log.gz 2024-04-25T11:24:04.762 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72062.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72062.log.gz 2024-04-25T11:24:04.762 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65606.log 2024-04-25T11:24:04.763 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2024-04-25T11:24:04.763 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33555.log.gz 2024-04-25T11:24:04.764 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47970.log 2024-04-25T11:24:04.764 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65606.log.gz 2024-04-25T11:24:04.765 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77306.log 2024-04-25T11:24:04.765 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2024-04-25T11:24:04.766 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47970.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47970.log.gz 2024-04-25T11:24:04.766 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64288.log 2024-04-25T11:24:04.767 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77306.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T11:24:04.768 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77306.log.gz 2024-04-25T11:24:04.769 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64288.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64288.log.gz 2024-04-25T11:24:04.769 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76097.log 2024-04-25T11:24:04.769 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62314.log 2024-04-25T11:24:04.771 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76097.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76097.log.gz 2024-04-25T11:24:04.771 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43377.log 2024-04-25T11:24:04.772 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68571.log 2024-04-25T11:24:04.773 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43377.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62314.log.gz 2024-04-25T11:24:04.773 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43377.log.gz 2024-04-25T11:24:04.774 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55654.log 2024-04-25T11:24:04.774 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55568.log 2024-04-25T11:24:04.775 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68571.log.gz 2024-04-25T11:24:04.776 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55654.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55654.log.gz 2024-04-25T11:24:04.776 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70341.log 2024-04-25T11:24:04.777 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41193.log 2024-04-25T11:24:04.778 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55568.log: /var/log/ceph/ceph-client.admin.70341.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55568.log.gz 2024-04-25T11:24:04.779 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.49665.log 2024-04-25T11:24:04.779 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70341.log.gz 2024-04-25T11:24:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41193.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41193.log.gz 2024-04-25T11:24:04.780 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37701.log 2024-04-25T11:24:04.781 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63236.log 2024-04-25T11:24:04.781 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49665.log.gz 2024-04-25T11:24:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37701.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37701.log.gz 2024-04-25T11:24:04.782 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62230.log 2024-04-25T11:24:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57000.log 2024-04-25T11:24:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63236.log: /var/log/ceph/ceph-client.admin.62230.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63236.log.gz 2024-04-25T11:24:04.785 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67301.log 2024-04-25T11:24:04.785 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62230.log.gz 2024-04-25T11:24:04.786 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57000.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57000.log.gz 2024-04-25T11:24:04.786 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68954.log 2024-04-25T11:24:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66978.log 2024-04-25T11:24:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67301.log: /var/log/ceph/ceph-client.admin.68954.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67301.log.gz 2024-04-25T11:24:04.789 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68954.log.gz 2024-04-25T11:24:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46876.log 2024-04-25T11:24:04.790 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73867.log 2024-04-25T11:24:04.790 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46876.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66978.log.gz 2024-04-25T11:24:04.791 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46876.log.gz 2024-04-25T11:24:04.791 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54664.log 2024-04-25T11:24:04.792 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53097.log 2024-04-25T11:24:04.792 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 94.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T11:24:04.793 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73867.log: /var/log/ceph/ceph-client.admin.54664.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73867.log.gz 2024-04-25T11:24:04.793 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54664.log.gz 2024-04-25T11:24:04.793 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58500.log 2024-04-25T11:24:04.794 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44304.log 2024-04-25T11:24:04.795 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53097.log.gz 2024-04-25T11:24:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58500.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58500.log.gz 2024-04-25T11:24:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64682.log 2024-04-25T11:24:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T11:24:04.797 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44304.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44304.log.gz/var/log/ceph/ceph-client.admin.64682.log: 2024-04-25T11:24:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53013.log 2024-04-25T11:24:04.798 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64682.log.gz 2024-04-25T11:24:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76053.log 2024-04-25T11:24:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60200.log 2024-04-25T11:24:04.801 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53013.log.gz 2024-04-25T11:24:04.802 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76053.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T11:24:04.802 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76053.log.gz 2024-04-25T11:24:04.803 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48442.log 2024-04-25T11:24:04.803 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60200.log.gz 2024-04-25T11:24:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74790.log 2024-04-25T11:24:04.806 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38127.log 2024-04-25T11:24:04.806 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48442.log.gz 2024-04-25T11:24:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70023.log 2024-04-25T11:24:04.808 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74790.log.gz 2024-04-25T11:24:04.809 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69176.log 2024-04-25T11:24:04.809 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38127.log.gz 2024-04-25T11:24:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57202.log 2024-04-25T11:24:04.811 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70023.log.gz 2024-04-25T11:24:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69176.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51837.log 2024-04-25T11:24:04.811 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69176.log.gz 2024-04-25T11:24:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71726.log 2024-04-25T11:24:04.813 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57202.log.gz 2024-04-25T11:24:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69270.log 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51837.log.gz 2024-04-25T11:24:04.815 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-04-25T11:24:04.815 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71726.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71726.log.gz 2024-04-25T11:24:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52281.log 2024-04-25T11:24:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45222.log 2024-04-25T11:24:04.817 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69270.log.gz 2024-04-25T11:24:04.819 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43167.log 2024-04-25T11:24:04.819 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52281.log.gz 2024-04-25T11:24:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45222.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45222.log.gz 2024-04-25T11:24:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68613.log 2024-04-25T11:24:04.822 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51981.log 2024-04-25T11:24:04.822 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43167.log.gz 2024-04-25T11:24:04.823 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44382.log 2024-04-25T11:24:04.823 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68613.log.gz 2024-04-25T11:24:04.825 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60868.log 2024-04-25T11:24:04.825 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51981.log.gz 2024-04-25T11:24:04.826 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58882.log 2024-04-25T11:24:04.826 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44382.log.gz 2024-04-25T11:24:04.827 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71515.log 2024-04-25T11:24:04.827 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60868.log.gz 2024-04-25T11:24:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67731.log 2024-04-25T11:24:04.829 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58882.log.gz 2024-04-25T11:24:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41277.log 2024-04-25T11:24:04.830 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71515.log.gz 2024-04-25T11:24:04.831 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59826.log 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67731.log.gz 2024-04-25T11:24:04.832 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-04-25T11:24:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41277.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41277.log.gz 2024-04-25T11:24:04.833 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T11:24:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59826.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63779.log 2024-04-25T11:24:04.835 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59826.log.gz 2024-04-25T11:24:04.837 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.63779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39681.log 2024-04-25T11:24:04.837 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63779.log.gz 2024-04-25T11:24:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57893.log 2024-04-25T11:24:04.843 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39681.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39681.log.gz 2024-04-25T11:24:04.843 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50036.log 2024-04-25T11:24:04.843 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57893.log.gz 2024-04-25T11:24:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50036.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50036.log.gz 2024-04-25T11:24:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63362.log 2024-04-25T11:24:04.845 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53693.log 2024-04-25T11:24:04.847 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63362.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63362.log.gz 2024-04-25T11:24:04.847 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77186.log 2024-04-25T11:24:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53693.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53693.log.gz 2024-04-25T11:24:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38001.log 2024-04-25T11:24:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77186.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77186.log.gz 2024-04-25T11:24:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73612.log 2024-04-25T11:24:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38001.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38001.log.gz 2024-04-25T11:24:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45012.log 2024-04-25T11:24:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73612.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73612.log.gz 2024-04-25T11:24:04.855 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74354.log 2024-04-25T11:24:04.856 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45012.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45012.log.gz 2024-04-25T11:24:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75289.log 2024-04-25T11:24:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74354.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74354.log.gz 2024-04-25T11:24:04.859 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.75289.log: /var/log/ceph/ceph-client.admin.54225.log 2024-04-25T11:24:04.859 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75289.log.gz 2024-04-25T11:24:04.860 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68361.log 2024-04-25T11:24:04.861 INFO:teuthology.orchestra.run.smithi087.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T11:24:04.862 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54225.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54225.log.gz 2024-04-25T11:24:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.68361.log: /var/log/ceph/ceph-client.admin.56754.log 2024-04-25T11:24:04.863 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68361.log.gz 2024-04-25T11:24:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72020.log 2024-04-25T11:24:04.866 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56754.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56754.log.gz 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57286.log 2024-04-25T11:24:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72020.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72020.log.gz 2024-04-25T11:24:04.868 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53567.log 2024-04-25T11:24:04.869 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57286.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57286.log.gz 2024-04-25T11:24:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.53567.log: -- /var/log/ceph/ceph-client.0.66410.log 2024-04-25T11:24:04.871 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53567.log.gz 2024-04-25T11:24:04.872 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57328.log 2024-04-25T11:24:04.873 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.66410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66410.log.gz 2024-04-25T11:24:04.874 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54622.log 2024-04-25T11:24:04.875 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57328.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57328.log.gz 2024-04-25T11:24:04.876 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68109.log 2024-04-25T11:24:04.877 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54622.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54622.log.gz 2024-04-25T11:24:04.878 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52677.log 2024-04-25T11:24:04.879 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68109.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68109.log.gz 2024-04-25T11:24:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38277.log 2024-04-25T11:24:04.881 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52677.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52677.log.gz 2024-04-25T11:24:04.882 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39149.log 2024-04-25T11:24:04.883 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38277.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38277.log.gz 2024-04-25T11:24:04.884 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45390.log 2024-04-25T11:24:04.885 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39149.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39149.log.gz 2024-04-25T11:24:04.886 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45390.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45390.log.gz 2024-04-25T11:24:04.887 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40787.log 2024-04-25T11:24:04.889 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48946.log 2024-04-25T11:24:04.890 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40787.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40787.log.gz 2024-04-25T11:24:04.890 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68067.log 2024-04-25T11:24:04.891 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48946.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48946.log.gz 2024-04-25T11:24:04.892 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61917.log 2024-04-25T11:24:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68067.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68067.log.gz 2024-04-25T11:24:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74150.log 2024-04-25T11:24:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61917.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61917.log.gz 2024-04-25T11:24:04.896 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56136.log 2024-04-25T11:24:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74150.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74150.log.gz 2024-04-25T11:24:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41067.log 2024-04-25T11:24:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.56136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56136.log.gz 2024-04-25T11:24:04.900 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47377.log 2024-04-25T11:24:04.901 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41067.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41067.log.gz 2024-04-25T11:24:04.902 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.47377.log: -- /var/log/ceph/ceph-client.admin.73993.log 2024-04-25T11:24:04.903 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47377.log.gz 2024-04-25T11:24:04.904 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77910.log 2024-04-25T11:24:04.905 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73993.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73993.log.gz 2024-04-25T11:24:04.906 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54580.log 2024-04-25T11:24:04.907 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77910.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77910.log.gz 2024-04-25T11:24:04.908 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48526.log 2024-04-25T11:24:04.909 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54580.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54580.log.gz 2024-04-25T11:24:04.910 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45138.log 2024-04-25T11:24:04.911 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48526.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48526.log.gz 2024-04-25T11:24:04.911 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77574.log 2024-04-25T11:24:04.913 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45138.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45138.log.gz 2024-04-25T11:24:04.914 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77574.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77574.log.gz 2024-04-25T11:24:04.915 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49994.log 2024-04-25T11:24:04.916 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64911.log 2024-04-25T11:24:04.918 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49994.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49994.log.gz 2024-04-25T11:24:04.918 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60097.log 2024-04-25T11:24:04.919 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64911.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64911.log.gz 2024-04-25T11:24:04.920 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51356.log 2024-04-25T11:24:04.921 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60097.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60097.log.gz 2024-04-25T11:24:04.922 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52761.log 2024-04-25T11:24:04.924 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51356.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51356.log.gz 2024-04-25T11:24:04.924 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67983.log 2024-04-25T11:24:04.925 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52761.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52761.log.gz 2024-04-25T11:24:04.926 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79325.log 2024-04-25T11:24:04.927 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67983.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67983.log.gz 2024-04-25T11:24:04.928 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49910.log 2024-04-25T11:24:04.929 INFO:teuthology.orchestra.run.smithi087.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T11:24:04.930 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.79325.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79325.log.gz 2024-04-25T11:24:04.930 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42999.log 2024-04-25T11:24:04.931 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49910.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49910.log.gz 2024-04-25T11:24:04.931 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52719.log 2024-04-25T11:24:04.932 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42999.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42999.log.gz 2024-04-25T11:24:04.933 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61833.log 2024-04-25T11:24:04.934 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52719.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52719.log.gz 2024-04-25T11:24:04.935 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59742.log 2024-04-25T11:24:04.936 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61833.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61833.log.gz 2024-04-25T11:24:04.937 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69606.log 2024-04-25T11:24:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59742.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59742.log.gz 2024-04-25T11:24:04.939 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62272.log 2024-04-25T11:24:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69606.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69606.log.gz 2024-04-25T11:24:04.941 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.62272.log: --verbose -- /var/log/ceph/ceph-client.admin.78877.log 2024-04-25T11:24:04.942 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62272.log.gz 2024-04-25T11:24:04.943 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79243.log 2024-04-25T11:24:04.944 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78877.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78877.log.gz 2024-04-25T11:24:04.945 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56628.log 2024-04-25T11:24:04.946 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.79243.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79243.log.gz 2024-04-25T11:24:04.947 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49539.log 2024-04-25T11:24:04.949 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56628.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56628.log.gz 2024-04-25T11:24:04.949 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.49539.log: --verbose -- /var/log/ceph/ceph-client.admin.55822.log 2024-04-25T11:24:04.950 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49539.log.gz 2024-04-25T11:24:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63916.log 2024-04-25T11:24:04.952 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55822.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55822.log.gz 2024-04-25T11:24:04.953 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.63916.log: -- /var/log/ceph/ceph-client.admin.58193.log 2024-04-25T11:24:04.954 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63916.log.gz 2024-04-25T11:24:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67773.log 2024-04-25T11:24:04.955 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58193.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58193.log.gz 2024-04-25T11:24:04.956 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50312.log 2024-04-25T11:24:04.957 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67773.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67773.log.gz 2024-04-25T11:24:04.958 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62615.log 2024-04-25T11:24:04.959 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50312.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50312.log.gz 2024-04-25T11:24:04.960 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55387.log 2024-04-25T11:24:04.961 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62615.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62615.log.gz 2024-04-25T11:24:04.962 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40382.log 2024-04-25T11:24:04.963 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55387.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55387.log.gz 2024-04-25T11:24:04.964 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56712.log 2024-04-25T11:24:04.965 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40382.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40382.log.gz 2024-04-25T11:24:04.965 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68912.log 2024-04-25T11:24:04.967 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56712.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56712.log.gz 2024-04-25T11:24:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68912.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68912.log.gz 2024-04-25T11:24:04.968 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37575.log 2024-04-25T11:24:04.970 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40694.log 2024-04-25T11:24:04.971 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37575.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37575.log.gz 2024-04-25T11:24:04.972 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51897.log 2024-04-25T11:24:04.973 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40694.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40694.log.gz 2024-04-25T11:24:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47335.log 2024-04-25T11:24:04.974 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51897.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51897.log.gz 2024-04-25T11:24:04.975 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39351.log 2024-04-25T11:24:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47335.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47335.log.gz 2024-04-25T11:24:04.977 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55049.log 2024-04-25T11:24:04.978 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39351.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39351.log.gz 2024-04-25T11:24:04.980 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55049.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55049.log.gz 2024-04-25T11:24:04.981 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51314.log 2024-04-25T11:24:04.982 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44850.log 2024-04-25T11:24:04.983 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51314.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51314.log.gz 2024-04-25T11:24:04.984 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74522.log 2024-04-25T11:24:04.985 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44850.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44850.log.gz 2024-04-25T11:24:04.986 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55131.log 2024-04-25T11:24:04.987 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74522.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74522.log.gz 2024-04-25T11:24:04.988 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55131.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55131.log.gz 2024-04-25T11:24:04.989 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66114.log 2024-04-25T11:24:04.990 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76278.log 2024-04-25T11:24:04.991 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66114.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66114.log.gz 2024-04-25T11:24:04.992 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.76278.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46834.log 2024-04-25T11:24:04.993 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76278.log.gz 2024-04-25T11:24:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69438.log 2024-04-25T11:24:04.995 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46834.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46834.log.gz 2024-04-25T11:24:04.996 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58672.log 2024-04-25T11:24:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69438.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69438.log.gz 2024-04-25T11:24:04.998 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53735.log/var/log/ceph/ceph-client.admin.58672.log: 2024-04-25T11:24:04.999 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58672.log.gz 2024-04-25T11:24:04.999 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43755.log 2024-04-25T11:24:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53735.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53735.log.gz 2024-04-25T11:24:05.002 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43755.log: /var/log/ceph/ceph-client.admin.43713.log 2024-04-25T11:24:05.002 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43755.log.gz 2024-04-25T11:24:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66616.log 2024-04-25T11:24:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43713.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43713.log.gz 2024-04-25T11:24:05.005 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74706.log 2024-04-25T11:24:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66616.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66616.log.gz 2024-04-25T11:24:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73402.log 2024-04-25T11:24:05.008 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74706.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74706.log.gz 2024-04-25T11:24:05.009 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49198.log 2024-04-25T11:24:05.010 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73402.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73402.log.gz 2024-04-25T11:24:05.012 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49198.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49198.log.gz 2024-04-25T11:24:05.012 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59784.log 2024-04-25T11:24:05.014 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69082.log 2024-04-25T11:24:05.015 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59784.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59784.log.gz 2024-04-25T11:24:05.015 INFO:teuthology.orchestra.run.smithi007.stderr: 91.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44724.log 2024-04-25T11:24:05.016 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T11:24:05.016 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69082.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69082.log.gz 2024-04-25T11:24:05.017 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53139.log 2024-04-25T11:24:05.018 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70149.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44724.log.gz 2024-04-25T11:24:05.018 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-04-25T11:24:05.020 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53139.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T11:24:05.020 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53139.log.gz 2024-04-25T11:24:05.021 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65904.log 2024-04-25T11:24:05.021 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70149.log.gz 2024-04-25T11:24:05.024 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.65904.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65904.log.gz 2024-04-25T11:24:05.024 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46496.log 2024-04-25T11:24:05.026 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39435.log 2024-04-25T11:24:05.027 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46496.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46496.log.gz 2024-04-25T11:24:05.028 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43041.log 2024-04-25T11:24:05.028 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39435.log.gz 2024-04-25T11:24:05.030 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43041.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43041.log.gz 2024-04-25T11:24:05.031 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63092.log 2024-04-25T11:24:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71599.log 2024-04-25T11:24:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63092.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63092.log.gz 2024-04-25T11:24:05.034 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61311.log 2024-04-25T11:24:05.035 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71599.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71599.log.gz 2024-04-25T11:24:05.035 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70510.log 2024-04-25T11:24:05.037 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61311.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61311.log.gz 2024-04-25T11:24:05.038 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.70510.log: --verbose -- /var/log/ceph/ceph-client.admin.43545.log 2024-04-25T11:24:05.038 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70510.log.gz 2024-04-25T11:24:05.039 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55480.log 2024-04-25T11:24:05.041 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43545.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43545.log.gz 2024-04-25T11:24:05.042 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.55480.log: /var/log/ceph/ceph-client.admin.52845.log 2024-04-25T11:24:05.042 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55480.log.gz 2024-04-25T11:24:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76320.log 2024-04-25T11:24:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52845.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52845.log.gz 2024-04-25T11:24:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.76320.log: --verbose -- /var/log/ceph/ceph-client.admin.65338.log 2024-04-25T11:24:05.048 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76320.log.gz 2024-04-25T11:24:05.048 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60285.log 2024-04-25T11:24:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65338.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65338.log.gz 2024-04-25T11:24:05.050 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64640.log/var/log/ceph/ceph-client.admin.60285.log: 2024-04-25T11:24:05.050 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60285.log.gz 2024-04-25T11:24:05.051 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68708.log 2024-04-25T11:24:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64640.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64640.log.gz 2024-04-25T11:24:05.053 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.68708.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.78415.log 2024-04-25T11:24:05.053 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68708.log.gz 2024-04-25T11:24:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68151.log 2024-04-25T11:24:05.056 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78415.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78415.log.gz 2024-04-25T11:24:05.057 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.68151.log: /var/log/ceph/ceph-client.admin.52803.log 2024-04-25T11:24:05.057 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68151.log.gz 2024-04-25T11:24:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50500.log 2024-04-25T11:24:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52803.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52803.log.gz 2024-04-25T11:24:05.061 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.0.50500.log: -- /var/log/ceph/ceph-client.admin.74976.log 2024-04-25T11:24:05.061 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50500.log.gz 2024-04-25T11:24:05.062 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61706.log 2024-04-25T11:24:05.065 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74976.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74976.log.gz 2024-04-25T11:24:05.065 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.61706.log: /var/log/ceph/ceph-client.admin.72445.log 2024-04-25T11:24:05.065 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61706.log.gz 2024-04-25T11:24:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40731.log 2024-04-25T11:24:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72445.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72445.log.gz 2024-04-25T11:24:05.068 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46792.log 2024-04-25T11:24:05.069 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40731.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40731.log.gz 2024-04-25T11:24:05.070 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39555.log 2024-04-25T11:24:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46792.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46792.log.gz 2024-04-25T11:24:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72714.log 2024-04-25T11:24:05.072 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39555.log.gz 2024-04-25T11:24:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40445.log 2024-04-25T11:24:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72714.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72714.log.gz 2024-04-25T11:24:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40445.log: -- /var/log/ceph/ceph-client.admin.60242.log 2024-04-25T11:24:05.076 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40445.log.gz 2024-04-25T11:24:05.078 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43335.log 2024-04-25T11:24:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60242.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60242.log.gz 2024-04-25T11:24:05.080 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43335.log: -- /var/log/ceph/ceph-client.admin.58449.log 2024-04-25T11:24:05.081 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43335.log.gz 2024-04-25T11:24:05.081 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55780.log 2024-04-25T11:24:05.083 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58449.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58449.log.gz 2024-04-25T11:24:05.084 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.55780.log: --verbose -- /var/log/ceph/ceph-client.admin.61790.log 2024-04-25T11:24:05.084 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55780.log.gz 2024-04-25T11:24:05.085 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64346.log 2024-04-25T11:24:05.087 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61790.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61790.log.gz 2024-04-25T11:24:05.087 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.64346.log: /var/log/ceph/ceph-client.admin.71037.log 2024-04-25T11:24:05.088 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64346.log.gz 2024-04-25T11:24:05.089 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66893.log 2024-04-25T11:24:05.091 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71037.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71037.log.gz 2024-04-25T11:24:05.092 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48694.log 2024-04-25T11:24:05.092 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66893.log.gz 2024-04-25T11:24:05.093 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42831.log 2024-04-25T11:24:05.094 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48694.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48694.log.gz 2024-04-25T11:24:05.095 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42831.log: /var/log/ceph/ceph-client.admin.52156.log 2024-04-25T11:24:05.096 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42831.log.gz 2024-04-25T11:24:05.097 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74874.log 2024-04-25T11:24:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52156.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52156.log.gz 2024-04-25T11:24:05.100 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.74874.log: --verbose -- /var/log/ceph/ceph-client.admin.40782.log 2024-04-25T11:24:05.100 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74874.log.gz 2024-04-25T11:24:05.101 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45932.log 2024-04-25T11:24:05.102 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40782.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40782.log.gz 2024-04-25T11:24:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40113.log 2024-04-25T11:24:05.103 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45932.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45932.log.gz 2024-04-25T11:24:05.105 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40113.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40113.log.gz 2024-04-25T11:24:05.106 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54351.log 2024-04-25T11:24:05.107 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42789.log 2024-04-25T11:24:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54351.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54351.log.gz 2024-04-25T11:24:05.109 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67647.log 2024-04-25T11:24:05.110 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42789.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42789.log.gz 2024-04-25T11:24:05.111 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46918.log 2024-04-25T11:24:05.112 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67647.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67647.log.gz 2024-04-25T11:24:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71389.log 2024-04-25T11:24:05.113 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46918.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46918.log.gz 2024-04-25T11:24:05.114 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58111.log 2024-04-25T11:24:05.116 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71389.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71389.log.gz 2024-04-25T11:24:05.117 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.58111.log: --verbose -- /var/log/ceph/ceph-client.admin.65258.log 2024-04-25T11:24:05.117 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58111.log.gz 2024-04-25T11:24:05.118 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47002.log 2024-04-25T11:24:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65258.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65258.log.gz 2024-04-25T11:24:05.121 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59486.log/var/log/ceph/ceph-client.admin.47002.log: 2024-04-25T11:24:05.122 INFO:teuthology.orchestra.run.smithi007.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.47002.log.gz 2024-04-25T11:24:05.122 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49952.log 2024-04-25T11:24:05.123 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59486.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59486.log.gz 2024-04-25T11:24:05.125 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49952.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49952.log.gz 2024-04-25T11:24:05.125 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73444.log 2024-04-25T11:24:05.127 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48778.log 2024-04-25T11:24:05.129 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73444.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73444.log.gz 2024-04-25T11:24:05.129 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51753.log 2024-04-25T11:24:05.130 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48778.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48778.log.gz 2024-04-25T11:24:05.131 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77432.log 2024-04-25T11:24:05.132 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51753.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51753.log.gz 2024-04-25T11:24:05.133 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77432.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77432.log.gz 2024-04-25T11:24:05.134 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77658.log 2024-04-25T11:24:05.135 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78793.log 2024-04-25T11:24:05.137 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77658.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77658.log.gz 2024-04-25T11:24:05.137 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37959.log 2024-04-25T11:24:05.138 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78793.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78793.log.gz 2024-04-25T11:24:05.139 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66222.log 2024-04-25T11:24:05.140 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37959.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37959.log.gz 2024-04-25T11:24:05.141 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.66222.log: --verbose -- /var/log/ceph/ceph-client.admin.52547.log 2024-04-25T11:24:05.141 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66222.log.gz 2024-04-25T11:24:05.142 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61748.log 2024-04-25T11:24:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52547.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52547.log.gz 2024-04-25T11:24:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.61748.log: -- /var/log/ceph/ceph-client.admin.50204.log 2024-04-25T11:24:05.145 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61748.log.gz 2024-04-25T11:24:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62104.log 2024-04-25T11:24:05.147 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50204.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50204.log.gz 2024-04-25T11:24:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.62104.log: /var/log/ceph/ceph-client.admin.52589.log 2024-04-25T11:24:05.149 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62104.log.gz 2024-04-25T11:24:05.150 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57851.log 2024-04-25T11:24:05.151 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52589.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52589.log.gz 2024-04-25T11:24:05.152 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57851.log: -- /var/log/ceph/ceph-client.admin.57100.log 2024-04-25T11:24:05.153 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57851.log.gz 2024-04-25T11:24:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60055.log 2024-04-25T11:24:05.155 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57100.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57100.log.gz 2024-04-25T11:24:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.60055.log: --verbose -- /var/log/ceph/ceph-client.admin.55526.log 2024-04-25T11:24:05.157 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60055.log.gz 2024-04-25T11:24:05.158 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52635.log 2024-04-25T11:24:05.159 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55526.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55526.log.gz 2024-04-25T11:24:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.52635.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.75163.log 2024-04-25T11:24:05.160 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52635.log.gz 2024-04-25T11:24:05.161 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49415.log 2024-04-25T11:24:05.163 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75163.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.75163.log.gz 2024-04-25T11:24:05.164 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49415.log: -- /var/log/ceph/ceph-client.admin.37409.log 2024-04-25T11:24:05.164 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49415.log.gz 2024-04-25T11:24:05.165 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59444.log 2024-04-25T11:24:05.166 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37409.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37409.log.gz 2024-04-25T11:24:05.167 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.59444.log: --verbose -- /var/log/ceph/ceph-client.admin.65522.log 2024-04-25T11:24:05.168 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59444.log.gz 2024-04-25T11:24:05.169 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61496.log 2024-04-25T11:24:05.170 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65522.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65522.log.gz 2024-04-25T11:24:05.171 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.61496.log: /var/log/ceph/ceph-client.admin.75121.log 2024-04-25T11:24:05.171 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61496.log.gz 2024-04-25T11:24:05.172 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66701.log 2024-04-25T11:24:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75121.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75121.log.gz 2024-04-25T11:24:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.66701.log: -- /var/log/ceph/ceph-client.admin.69672.log 2024-04-25T11:24:05.175 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66701.log.gz 2024-04-25T11:24:05.176 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79003.log 2024-04-25T11:24:05.179 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69672.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69672.log.gz 2024-04-25T11:24:05.179 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.79003.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.73486.log 2024-04-25T11:24:05.179 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79003.log.gz 2024-04-25T11:24:05.180 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50716.log 2024-04-25T11:24:05.181 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73486.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73486.log.gz 2024-04-25T11:24:05.182 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.50716.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51482.log 2024-04-25T11:24:05.183 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50716.log.gz 2024-04-25T11:24:05.184 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43839.log 2024-04-25T11:24:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51482.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51482.log.gz 2024-04-25T11:24:05.186 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66743.log 2024-04-25T11:24:05.188 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43839.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43839.log.gz 2024-04-25T11:24:05.188 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65121.log 2024-04-25T11:24:05.189 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66743.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66743.log.gz 2024-04-25T11:24:05.190 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60784.log 2024-04-25T11:24:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65121.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65121.log.gz 2024-04-25T11:24:05.191 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37617.log 2024-04-25T11:24:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60784.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60784.log.gz 2024-04-25T11:24:05.194 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37617.log: -- /var/log/ceph/ceph-client.admin.39723.log 2024-04-25T11:24:05.194 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37617.log.gz 2024-04-25T11:24:05.195 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65163.log 2024-04-25T11:24:05.198 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39723.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39723.log.gz 2024-04-25T11:24:05.198 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.65163.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43419.log 2024-04-25T11:24:05.198 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65163.log.gz 2024-04-25T11:24:05.199 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78373.log 2024-04-25T11:24:05.200 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43419.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43419.log.gz 2024-04-25T11:24:05.201 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74480.log/var/log/ceph/ceph-client.admin.78373.log: 2024-04-25T11:24:05.202 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78373.log.gz 2024-04-25T11:24:05.202 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57599.log 2024-04-25T11:24:05.204 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74480.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74480.log.gz 2024-04-25T11:24:05.205 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57599.log: -- /var/log/ceph/ceph-client.admin.75841.log 2024-04-25T11:24:05.205 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57599.log.gz 2024-04-25T11:24:05.207 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68996.log 2024-04-25T11:24:05.208 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75841.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75841.log.gz 2024-04-25T11:24:05.209 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61395.log/var/log/ceph/ceph-client.admin.68996.log: 2024-04-25T11:24:05.209 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68996.log.gz 2024-04-25T11:24:05.210 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39065.log 2024-04-25T11:24:05.211 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61395.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61395.log.gz 2024-04-25T11:24:05.212 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57370.log/var/log/ceph/ceph-client.admin.39065.log: 2024-04-25T11:24:05.214 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39065.log.gz 2024-04-25T11:24:05.214 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56471.log 2024-04-25T11:24:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57370.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57370.log.gz 2024-04-25T11:24:05.216 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52496.log 2024-04-25T11:24:05.217 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56471.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56471.log.gz 2024-04-25T11:24:05.218 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63512.log 2024-04-25T11:24:05.219 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52496.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52496.log.gz 2024-04-25T11:24:05.220 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.63512.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48568.log 2024-04-25T11:24:05.221 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63512.log.gz 2024-04-25T11:24:05.221 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48258.log 2024-04-25T11:24:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48568.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48568.log.gz 2024-04-25T11:24:05.224 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48258.log: /var/log/ceph/ceph-client.admin.60700.log 2024-04-25T11:24:05.224 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48258.log.gz 2024-04-25T11:24:05.225 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77742.log 2024-04-25T11:24:05.227 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60700.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60700.log.gz 2024-04-25T11:24:05.228 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.77742.log: /var/log/ceph/ceph-client.admin.54519.log 2024-04-25T11:24:05.228 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77742.log.gz 2024-04-25T11:24:05.229 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47928.log 2024-04-25T11:24:05.232 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54519.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54519.log.gz 2024-04-25T11:24:05.232 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.47928.log: -- /var/log/ceph/ceph-client.admin.55299.log 2024-04-25T11:24:05.232 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47928.log.gz 2024-04-25T11:24:05.233 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69038.log 2024-04-25T11:24:05.235 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55299.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55299.log.gz 2024-04-25T11:24:05.235 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55438.log 2024-04-25T11:24:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69038.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69038.log.gz 2024-04-25T11:24:05.236 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67157.log 2024-04-25T11:24:05.239 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55438.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55438.log.gz 2024-04-25T11:24:05.239 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59178.log 2024-04-25T11:24:05.240 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67157.log.gz 2024-04-25T11:24:05.240 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72798.log 2024-04-25T11:24:05.242 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.59178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59178.log.gz 2024-04-25T11:24:05.243 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67463.log 2024-04-25T11:24:05.244 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72798.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72798.log.gz 2024-04-25T11:24:05.244 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40391.log 2024-04-25T11:24:05.247 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67463.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67463.log.gz 2024-04-25T11:24:05.249 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40391.log: /var/log/ceph/ceph-client.admin.73951.log 2024-04-25T11:24:05.249 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40391.log.gz 2024-04-25T11:24:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49623.log 2024-04-25T11:24:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73951.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73951.log.gz 2024-04-25T11:24:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49623.log: /var/log/ceph/ceph-client.admin.76572.log 2024-04-25T11:24:05.253 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49623.log.gz 2024-04-25T11:24:05.253 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66785.log 2024-04-25T11:24:05.254 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76572.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.76572.log.gz 2024-04-25T11:24:05.254 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56832.log 2024-04-25T11:24:05.254 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66785.log.gz 2024-04-25T11:24:05.256 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37831.log 2024-04-25T11:24:05.257 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56832.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56832.log.gz 2024-04-25T11:24:05.258 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37831.log: -- /var/log/ceph/ceph-client.admin.39597.log 2024-04-25T11:24:05.258 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37831.log.gz 2024-04-25T11:24:05.259 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69228.log 2024-04-25T11:24:05.262 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39597.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39597.log.gz 2024-04-25T11:24:05.262 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69939.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69228.log.gz 2024-04-25T11:24:05.262 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-04-25T11:24:05.263 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50162.log 2024-04-25T11:24:05.266 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69939.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69939.log.gz 2024-04-25T11:24:05.266 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41235.log 2024-04-25T11:24:05.267 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50162.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50162.log.gz 2024-04-25T11:24:05.268 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41235.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41235.log.gz 2024-04-25T11:24:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76488.log 2024-04-25T11:24:05.270 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51230.log 2024-04-25T11:24:05.271 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76488.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76488.log.gz 2024-04-25T11:24:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41529.log 2024-04-25T11:24:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51230.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51230.log.gz 2024-04-25T11:24:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47886.log 2024-04-25T11:24:05.275 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41529.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41529.log.gz 2024-04-25T11:24:05.276 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64246.log 2024-04-25T11:24:05.277 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47886.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47886.log.gz 2024-04-25T11:24:05.278 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78139.log 2024-04-25T11:24:05.279 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64246.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64246.log.gz 2024-04-25T11:24:05.280 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78139.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78139.log.gz 2024-04-25T11:24:05.281 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40114.log 2024-04-25T11:24:05.282 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58924.log 2024-04-25T11:24:05.284 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40114.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40114.log.gz 2024-04-25T11:24:05.284 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74108.log 2024-04-25T11:24:05.286 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58924.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58924.log.gz 2024-04-25T11:24:05.286 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68193.log 2024-04-25T11:24:05.287 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74108.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74108.log.gz 2024-04-25T11:24:05.288 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49114.log 2024-04-25T11:24:05.289 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68193.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68193.log.gz 2024-04-25T11:24:05.290 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37203.log 2024-04-25T11:24:05.291 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49114.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49114.log.gz 2024-04-25T11:24:05.292 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37203.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37203.log.gz 2024-04-25T11:24:05.293 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45516.log 2024-04-25T11:24:05.294 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73570.log 2024-04-25T11:24:05.296 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45516.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45516.log.gz 2024-04-25T11:24:05.296 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46098.log 2024-04-25T11:24:05.297 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73570.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73570.log.gz 2024-04-25T11:24:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54706.log 2024-04-25T11:24:05.299 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46098.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46098.log.gz 2024-04-25T11:24:05.300 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.54706.log: /var/log/ceph/ceph-client.admin.51088.log 2024-04-25T11:24:05.301 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54706.log.gz 2024-04-25T11:24:05.301 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54874.log 2024-04-25T11:24:05.303 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51088.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51088.log.gz 2024-04-25T11:24:05.304 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44592.log/var/log/ceph/ceph-client.admin.54874.log: 2024-04-25T11:24:05.304 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54874.log.gz 2024-04-25T11:24:05.305 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76404.log 2024-04-25T11:24:05.307 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44592.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44592.log.gz 2024-04-25T11:24:05.308 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.76404.log: -- /var/log/ceph/ceph-client.admin.69312.log 2024-04-25T11:24:05.308 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76404.log.gz 2024-04-25T11:24:05.309 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42351.log 2024-04-25T11:24:05.311 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69312.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69312.log.gz 2024-04-25T11:24:05.312 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44508.log 2024-04-25T11:24:05.312 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42351.log.gz 2024-04-25T11:24:05.313 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72575.log 2024-04-25T11:24:05.314 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44508.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44508.log.gz 2024-04-25T11:24:05.315 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.72575.log: /var/log/ceph/ceph-client.admin.70468.log 2024-04-25T11:24:05.316 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72575.log.gz 2024-04-25T11:24:05.317 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57641.log 2024-04-25T11:24:05.319 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70468.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70468.log.gz 2024-04-25T11:24:05.319 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76362.log 2024-04-25T11:24:05.320 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57641.log.gz 2024-04-25T11:24:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67505.log 2024-04-25T11:24:05.323 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76362.log: 88.9%gzip/var/log/ceph/ceph-client.admin.67505.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56586.log 2024-04-25T11:24:05.323 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67505.log.gz 2024-04-25T11:24:05.323 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76362.log.gz 2024-04-25T11:24:05.325 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56586.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56586.log.gz 2024-04-25T11:24:05.326 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64000.log 2024-04-25T11:24:05.327 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37873.log 2024-04-25T11:24:05.328 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64000.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64000.log.gz 2024-04-25T11:24:05.329 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40393.log 2024-04-25T11:24:05.330 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37873.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37873.log.gz 2024-04-25T11:24:05.331 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37490.log 2024-04-25T11:24:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40393.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40393.log.gz 2024-04-25T11:24:05.333 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45474.log 2024-04-25T11:24:05.334 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37490.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37490.log.gz 2024-04-25T11:24:05.335 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64514.log 2024-04-25T11:24:05.336 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45474.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45474.log.gz 2024-04-25T11:24:05.337 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64514.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64514.log.gz 2024-04-25T11:24:05.338 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76638.log 2024-04-25T11:24:05.339 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41613.log 2024-04-25T11:24:05.340 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76638.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76638.log.gz 2024-04-25T11:24:05.341 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61227.log 2024-04-25T11:24:05.343 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41613.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41613.log.gz 2024-04-25T11:24:05.343 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75018.log 2024-04-25T11:24:05.344 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61227.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61227.log.gz 2024-04-25T11:24:05.345 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56257.log 2024-04-25T11:24:05.346 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75018.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75018.log.gz 2024-04-25T11:24:05.347 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42198.log 2024-04-25T11:24:05.348 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56257.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56257.log.gz 2024-04-25T11:24:05.349 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.42198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42198.log.gz 2024-04-25T11:24:05.349 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54832.log 2024-04-25T11:24:05.351 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49497.log 2024-04-25T11:24:05.352 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54832.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54832.log.gz 2024-04-25T11:24:05.353 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76009.log 2024-04-25T11:24:05.354 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49497.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49497.log.gz 2024-04-25T11:24:05.355 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63176.log 2024-04-25T11:24:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76009.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76009.log.gz 2024-04-25T11:24:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41403.log 2024-04-25T11:24:05.357 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63176.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63176.log.gz 2024-04-25T11:24:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49581.log 2024-04-25T11:24:05.359 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41403.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41403.log.gz 2024-04-25T11:24:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43251.log 2024-04-25T11:24:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49581.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49581.log.gz 2024-04-25T11:24:05.362 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45054.log 2024-04-25T11:24:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43251.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43251.log.gz 2024-04-25T11:24:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39807.log 2024-04-25T11:24:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45054.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45054.log.gz 2024-04-25T11:24:05.366 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67259.log 2024-04-25T11:24:05.366 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39807.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39807.log.gz 2024-04-25T11:24:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67259.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67259.log.gz 2024-04-25T11:24:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70107.log 2024-04-25T11:24:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75883.log 2024-04-25T11:24:05.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70107.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70107.log.gz 2024-04-25T11:24:05.372 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78919.log 2024-04-25T11:24:05.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75883.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75883.log.gz 2024-04-25T11:24:05.374 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70647.log 2024-04-25T11:24:05.375 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78919.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78919.log.gz 2024-04-25T11:24:05.376 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71894.log 2024-04-25T11:24:05.377 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70647.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70647.log.gz 2024-04-25T11:24:05.377 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68788.log 2024-04-25T11:24:05.379 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71894.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71894.log.gz 2024-04-25T11:24:05.380 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68788.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68788.log.gz 2024-04-25T11:24:05.381 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39513.log 2024-04-25T11:24:05.382 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59868.log 2024-04-25T11:24:05.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39513.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39513.log.gz 2024-04-25T11:24:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59868.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59868.log.gz 2024-04-25T11:24:05.385 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51440.log 2024-04-25T11:24:05.387 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46140.log 2024-04-25T11:24:05.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51440.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51440.log.gz 2024-04-25T11:24:05.389 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59528.log 2024-04-25T11:24:05.390 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46140.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46140.log.gz 2024-04-25T11:24:05.391 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51711.log 2024-04-25T11:24:05.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59528.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59528.log.gz 2024-04-25T11:24:05.393 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78709.log 2024-04-25T11:24:05.393 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51711.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51711.log.gz 2024-04-25T11:24:05.394 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48216.log 2024-04-25T11:24:05.395 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78709.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78709.log.gz 2024-04-25T11:24:05.396 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75247.log 2024-04-25T11:24:05.397 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48216.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48216.log.gz 2024-04-25T11:24:05.398 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37745.log 2024-04-25T11:24:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75247.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75247.log.gz 2024-04-25T11:24:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55257.log 2024-04-25T11:24:05.401 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37745.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37745.log.gz 2024-04-25T11:24:05.401 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64204.log 2024-04-25T11:24:05.403 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55257.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55257.log.gz 2024-04-25T11:24:05.404 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70851.log 2024-04-25T11:24:05.404 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64204.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64204.log.gz 2024-04-25T11:24:05.405 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71221.log 2024-04-25T11:24:05.406 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70851.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70851.log.gz 2024-04-25T11:24:05.407 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.71221.log: /var/log/ceph/ceph-client.admin.78457.log 2024-04-25T11:24:05.408 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71221.log.gz 2024-04-25T11:24:05.409 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45810.log 2024-04-25T11:24:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78457.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78457.log.gz 2024-04-25T11:24:05.411 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45810.log: /var/log/ceph/ceph-client.admin.54309.log 2024-04-25T11:24:05.412 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45810.log.gz 2024-04-25T11:24:05.413 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38714.log 2024-04-25T11:24:05.414 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54309.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54309.log.gz 2024-04-25T11:24:05.415 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.38714.log: /var/log/ceph/ceph-client.admin.37789.log 2024-04-25T11:24:05.417 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37789.log: gzip -5 --verbose 92.2% -- replaced with /var/log/ceph/ceph-client.admin.38714.log.gz 2024-04-25T11:24:05.418 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.46750.log 2024-04-25T11:24:05.419 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37789.log.gz 2024-04-25T11:24:05.419 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58756.log 2024-04-25T11:24:05.420 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46750.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46750.log.gz 2024-04-25T11:24:05.421 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39107.log 2024-04-25T11:24:05.422 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58756.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58756.log.gz 2024-04-25T11:24:05.423 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70233.log 2024-04-25T11:24:05.424 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39107.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39107.log.gz 2024-04-25T11:24:05.425 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.70233.log: /var/log/ceph/ceph-client.admin.52238.log 2024-04-25T11:24:05.426 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70233.log.gz 2024-04-25T11:24:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48862.log 2024-04-25T11:24:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52238.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52238.log.gz 2024-04-25T11:24:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40786.log 2024-04-25T11:24:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48862.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48862.log.gz 2024-04-25T11:24:05.430 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68529.log 2024-04-25T11:24:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40786.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40786.log.gz 2024-04-25T11:24:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39225.log 2024-04-25T11:24:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68529.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68529.log.gz 2024-04-25T11:24:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56874.log 2024-04-25T11:24:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39225.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39225.log.gz 2024-04-25T11:24:05.436 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75499.log 2024-04-25T11:24:05.438 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56874.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56874.log.gz 2024-04-25T11:24:05.438 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58235.log 2024-04-25T11:24:05.439 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75499.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75499.log.gz 2024-04-25T11:24:05.440 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44550.log 2024-04-25T11:24:05.441 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58235.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58235.log.gz 2024-04-25T11:24:05.441 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37532.log 2024-04-25T11:24:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44550.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44550.log.gz 2024-04-25T11:24:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73696.log 2024-04-25T11:24:05.444 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37532.log.gz 2024-04-25T11:24:05.446 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53609.log 2024-04-25T11:24:05.447 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73696.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73696.log.gz 2024-04-25T11:24:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63958.log 2024-04-25T11:24:05.448 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53609.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53609.log.gz 2024-04-25T11:24:05.449 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44178.log 2024-04-25T11:24:05.450 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63958.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63958.log.gz 2024-04-25T11:24:05.451 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54393.log 2024-04-25T11:24:05.452 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44178.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44178.log.gz 2024-04-25T11:24:05.453 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70065.log 2024-04-25T11:24:05.454 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54393.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54393.log.gz 2024-04-25T11:24:05.455 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66531.log/var/log/ceph/ceph-client.admin.70065.log: 2024-04-25T11:24:05.456 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70065.log.gz 2024-04-25T11:24:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62356.log 2024-04-25T11:24:05.458 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66531.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66531.log.gz 2024-04-25T11:24:05.459 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64869.log 2024-04-25T11:24:05.459 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62356.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62356.log.gz 2024-04-25T11:24:05.461 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64869.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64869.log.gz 2024-04-25T11:24:05.462 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71557.log 2024-04-25T11:24:05.463 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41151.log 2024-04-25T11:24:05.464 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71557.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71557.log.gz 2024-04-25T11:24:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44892.log 2024-04-25T11:24:05.466 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41151.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41151.log.gz 2024-04-25T11:24:05.467 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62781.log 2024-04-25T11:24:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44892.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44892.log.gz 2024-04-25T11:24:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48610.log 2024-04-25T11:24:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62781.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62781.log.gz 2024-04-25T11:24:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48610.log: -- /var/log/ceph/ceph-client.admin.53999.log 2024-04-25T11:24:05.471 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48610.log.gz 2024-04-25T11:24:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50246.log 2024-04-25T11:24:05.473 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53999.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53999.log.gz 2024-04-25T11:24:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55343.log/var/log/ceph/ceph-client.admin.50246.log: 2024-04-25T11:24:05.476 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50246.log.gz 2024-04-25T11:24:05.477 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.55343.log: /var/log/ceph/ceph-client.admin.46540.log 2024-04-25T11:24:05.477 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55343.log.gz 2024-04-25T11:24:05.478 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59258.log 2024-04-25T11:24:05.480 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46540.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46540.log.gz 2024-04-25T11:24:05.480 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76190.log 2024-04-25T11:24:05.481 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59258.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59258.log.gz 2024-04-25T11:24:05.482 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46582.log 2024-04-25T11:24:05.483 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76190.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76190.log.gz 2024-04-25T11:24:05.484 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64995.log 2024-04-25T11:24:05.485 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46582.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46582.log.gz 2024-04-25T11:24:05.485 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56670.log 2024-04-25T11:24:05.486 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64995.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64995.log.gz 2024-04-25T11:24:05.488 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56670.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56670.log.gz 2024-04-25T11:24:05.488 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40110.log 2024-04-25T11:24:05.490 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41715.log 2024-04-25T11:24:05.491 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40110.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40110.log.gz 2024-04-25T11:24:05.492 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68025.log 2024-04-25T11:24:05.494 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41715.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41715.log.gz 2024-04-25T11:24:05.494 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41319.log 2024-04-25T11:24:05.494 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68025.log.gz 2024-04-25T11:24:05.495 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70426.log 2024-04-25T11:24:05.497 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41319.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41319.log.gz 2024-04-25T11:24:05.497 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78013.log/var/log/ceph/ceph-client.admin.70426.log: 2024-04-25T11:24:05.498 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70426.log.gz 2024-04-25T11:24:05.499 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77952.log 2024-04-25T11:24:05.500 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78013.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78013.log.gz 2024-04-25T11:24:05.501 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70773.log/var/log/ceph/ceph-client.admin.77952.log: 2024-04-25T11:24:05.502 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77952.log.gz 2024-04-25T11:24:05.503 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62146.log 2024-04-25T11:24:05.504 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62909.log 2024-04-25T11:24:05.504 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70773.log.gz 2024-04-25T11:24:05.507 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62146.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49156.log/var/log/ceph/ceph-client.admin.62909.log: 2024-04-25T11:24:05.507 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62146.log.gz 2024-04-25T11:24:05.507 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62909.log.gz 2024-04-25T11:24:05.509 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45264.log 2024-04-25T11:24:05.510 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62062.log 2024-04-25T11:24:05.511 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49156.log.gz 2024-04-25T11:24:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45264.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45264.log.gz 2024-04-25T11:24:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69981.log 2024-04-25T11:24:05.513 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62062.log.gz 2024-04-25T11:24:05.515 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58714.log 2024-04-25T11:24:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69981.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69981.log.gz 2024-04-25T11:24:05.517 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53957.log 2024-04-25T11:24:05.518 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58714.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58714.log.gz 2024-04-25T11:24:05.519 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58542.log 2024-04-25T11:24:05.520 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53957.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53957.log.gz 2024-04-25T11:24:05.521 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51046.log 2024-04-25T11:24:05.522 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58542.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58542.log.gz 2024-04-25T11:24:05.523 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55738.log 2024-04-25T11:24:05.524 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51046.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51046.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57725.log 2024-04-25T11:24:05.524 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-04-25T11:24:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55738.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55738.log.gz 2024-04-25T11:24:05.526 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57725.log: -- /var/log/ceph/ceph-client.admin.39933.log 2024-04-25T11:24:05.527 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57725.log.gz 2024-04-25T11:24:05.528 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70384.log 2024-04-25T11:24:05.529 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39933.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39933.log.gz 2024-04-25T11:24:05.530 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79171.log 2024-04-25T11:24:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70384.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70384.log.gz 2024-04-25T11:24:05.532 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.79171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72237.log 2024-04-25T11:24:05.532 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79171.log.gz 2024-04-25T11:24:05.534 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44682.log 2024-04-25T11:24:05.535 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72237.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72237.log.gz 2024-04-25T11:24:05.536 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40105.log 2024-04-25T11:24:05.537 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44682.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44682.log.gz 2024-04-25T11:24:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40105.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40105.log.gz 2024-04-25T11:24:05.539 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45180.log 2024-04-25T11:24:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60139.log 2024-04-25T11:24:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45180.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45180.log.gz 2024-04-25T11:24:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70689.log 2024-04-25T11:24:05.543 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60139.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60139.log.gz 2024-04-25T11:24:05.544 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44220.log/var/log/ceph/ceph-client.admin.70689.log: 2024-04-25T11:24:05.545 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70689.log.gz 2024-04-25T11:24:05.546 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43629.log 2024-04-25T11:24:05.547 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44220.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44220.log.gz 2024-04-25T11:24:05.548 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51669.log/var/log/ceph/ceph-client.admin.43629.log: 2024-04-25T11:24:05.549 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43629.log.gz 2024-04-25T11:24:05.549 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38085.log 2024-04-25T11:24:05.551 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51669.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51669.log.gz 2024-04-25T11:24:05.551 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60412.log 2024-04-25T11:24:05.553 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38085.log: 88.1%gzip -5 --verbose --/var/log/ceph/ceph-client.admin.60412.log: /var/log/ceph/ceph-client.admin.37313.log 2024-04-25T11:24:05.554 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38085.log.gz 2024-04-25T11:24:05.554 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60412.log.gz 2024-04-25T11:24:05.555 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55696.log 2024-04-25T11:24:05.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37313.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37313.log.gz 2024-04-25T11:24:05.557 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40112.log 2024-04-25T11:24:05.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55696.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55696.log.gz 2024-04-25T11:24:05.559 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60369.log 2024-04-25T11:24:05.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40112.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40112.log.gz 2024-04-25T11:24:05.561 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53055.log 2024-04-25T11:24:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60369.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60369.log.gz 2024-04-25T11:24:05.563 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52971.log 2024-04-25T11:24:05.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53055.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53055.log.gz 2024-04-25T11:24:05.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52971.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52971.log.gz 2024-04-25T11:24:05.566 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47767.log 2024-04-25T11:24:05.568 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68277.log 2024-04-25T11:24:05.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47767.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47767.log.gz 2024-04-25T11:24:05.570 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T11:24:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68277.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68277.log.gz 2024-04-25T11:24:05.571 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61269.log 2024-04-25T11:24:05.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.61269.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61269.log.gz 2024-04-25T11:24:05.575 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73804.log 2024-04-25T11:24:05.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54477.log 2024-04-25T11:24:05.578 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73804.log.gz 2024-04-25T11:24:05.580 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61875.log 2024-04-25T11:24:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55173.log 2024-04-25T11:24:05.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61875.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61875.log.gz 2024-04-25T11:24:05.583 INFO:teuthology.orchestra.run.smithi007.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54477.log.gz 2024-04-25T11:24:05.584 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39267.log 2024-04-25T11:24:05.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55173.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55173.log.gz 2024-04-25T11:24:05.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63004.log 2024-04-25T11:24:05.590 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39267.log.gz 2024-04-25T11:24:05.591 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74276.log 2024-04-25T11:24:05.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63004.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54183.log 2024-04-25T11:24:05.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74276.log: -- replaced with /var/log/ceph/ceph-client.admin.63004.log.gz 2024-04-25T11:24:05.595 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74276.log.gz 2024-04-25T11:24:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61005.log 2024-04-25T11:24:05.599 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54183.log.gz 2024-04-25T11:24:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44466.log 2024-04-25T11:24:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61005.log: /var/log/ceph/ceph-client.admin.44466.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T11:24:05.604 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61005.log.gz 2024-04-25T11:24:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75373.log 2024-04-25T11:24:05.605 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44466.log.gz 2024-04-25T11:24:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63134.log 2024-04-25T11:24:05.608 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75373.log.gz 2024-04-25T11:24:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48054.log 2024-04-25T11:24:05.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63134.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63134.log.gz 2024-04-25T11:24:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.48054.log: --verbose -- /var/log/ceph/ceph-client.admin.54969.log 2024-04-25T11:24:05.612 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48054.log.gz 2024-04-25T11:24:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50962.log 2024-04-25T11:24:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54969.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54969.log.gz 2024-04-25T11:24:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58630.log 2024-04-25T11:24:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50962.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50962.log.gz 2024-04-25T11:24:05.616 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40449.log 2024-04-25T11:24:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58630.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58630.log.gz 2024-04-25T11:24:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61131.log 2024-04-25T11:24:05.619 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40449.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40449.log.gz 2024-04-25T11:24:05.620 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42873.log 2024-04-25T11:24:05.621 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61131.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61131.log.gz 2024-04-25T11:24:05.622 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60327.log 2024-04-25T11:24:05.623 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42873.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42873.log.gz 2024-04-25T11:24:05.624 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77144.log 2024-04-25T11:24:05.625 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60327.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60327.log.gz 2024-04-25T11:24:05.626 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77144.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77144.log.gz 2024-04-25T11:24:05.627 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64388.log 2024-04-25T11:24:05.628 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65820.log 2024-04-25T11:24:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64388.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64388.log.gz 2024-04-25T11:24:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54435.log 2024-04-25T11:24:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65820.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65820.log.gz 2024-04-25T11:24:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39765.log 2024-04-25T11:24:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54435.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54435.log.gz 2024-04-25T11:24:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74748.log 2024-04-25T11:24:05.635 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39765.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39765.log.gz 2024-04-25T11:24:05.635 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73909.log 2024-04-25T11:24:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74748.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74748.log.gz 2024-04-25T11:24:05.638 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73909.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73909.log.gz 2024-04-25T11:24:05.639 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65736.log 2024-04-25T11:24:05.640 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79087.log 2024-04-25T11:24:05.641 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65736.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65736.log.gz 2024-04-25T11:24:05.642 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43881.log 2024-04-25T11:24:05.643 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.79087.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79087.log.gz 2024-04-25T11:24:05.645 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67857.log 2024-04-25T11:24:05.645 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43881.log.gz 2024-04-25T11:24:05.646 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66489.log 2024-04-25T11:24:05.647 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67857.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67857.log.gz 2024-04-25T11:24:05.648 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46284.log 2024-04-25T11:24:05.649 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66489.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66489.log.gz 2024-04-25T11:24:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69564.log 2024-04-25T11:24:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46284.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46284.log.gz 2024-04-25T11:24:05.652 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45348.log 2024-04-25T11:24:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69564.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69564.log.gz 2024-04-25T11:24:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55612.log 2024-04-25T11:24:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45348.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45348.log.gz 2024-04-25T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.55612.log: --verbose -- /var/log/ceph/ceph-client.admin.78835.log 2024-04-25T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55612.log.gz 2024-04-25T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.78835.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78835.log.gz 2024-04-25T11:24:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58405.log 2024-04-25T11:24:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72966.log 2024-04-25T11:24:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58405.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58405.log.gz 2024-04-25T11:24:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58031.log 2024-04-25T11:24:05.663 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72966.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72966.log.gz 2024-04-25T11:24:05.664 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67899.log 2024-04-25T11:24:05.665 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58031.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58031.log.gz 2024-04-25T11:24:05.666 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55882.log 2024-04-25T11:24:05.667 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67899.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67899.log.gz 2024-04-25T11:24:05.668 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64598.log 2024-04-25T11:24:05.669 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55882.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55882.log.gz 2024-04-25T11:24:05.670 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64598.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64598.log.gz 2024-04-25T11:24:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48300.log 2024-04-25T11:24:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49864.log 2024-04-25T11:24:05.673 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48300.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48300.log.gz 2024-04-25T11:24:05.674 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71852.log 2024-04-25T11:24:05.675 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49864.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49864.log.gz 2024-04-25T11:24:05.676 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44970.log 2024-04-25T11:24:05.677 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71852.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71852.log.gz 2024-04-25T11:24:05.678 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43923.log 2024-04-25T11:24:05.679 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44970.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44970.log.gz 2024-04-25T11:24:05.680 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43923.log: -- /var/log/ceph/ceph-client.admin.51524.log 2024-04-25T11:24:05.680 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43923.log.gz 2024-04-25T11:24:05.681 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76446.log 2024-04-25T11:24:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51524.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51524.log.gz 2024-04-25T11:24:05.683 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75679.log/var/log/ceph/ceph-client.admin.76446.log: 2024-04-25T11:24:05.684 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76446.log.gz 2024-04-25T11:24:05.685 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69522.log 2024-04-25T11:24:05.686 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75679.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75679.log.gz 2024-04-25T11:24:05.687 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53205.log 2024-04-25T11:24:05.688 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69522.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69522.log.gz 2024-04-25T11:24:05.689 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74832.log 2024-04-25T11:24:05.690 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53205.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53205.log.gz 2024-04-25T11:24:05.691 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.74832.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68870.log 2024-04-25T11:24:05.691 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74832.log.gz 2024-04-25T11:24:05.692 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45726.log 2024-04-25T11:24:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68870.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68870.log.gz 2024-04-25T11:24:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58319.log 2024-04-25T11:24:05.695 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45726.log.gz 2024-04-25T11:24:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53915.log 2024-04-25T11:24:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58319.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58319.log.gz 2024-04-25T11:24:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72403.log 2024-04-25T11:24:05.699 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53915.log.gz 2024-04-25T11:24:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67941.log 2024-04-25T11:24:05.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72403.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72403.log.gz 2024-04-25T11:24:05.703 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57058.log 2024-04-25T11:24:05.704 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67941.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67941.log.gz 2024-04-25T11:24:05.704 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68403.log 2024-04-25T11:24:05.706 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57058.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57058.log.gz 2024-04-25T11:24:05.706 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77700.log 2024-04-25T11:24:05.707 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68403.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68403.log.gz 2024-04-25T11:24:05.708 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46055.log 2024-04-25T11:24:05.709 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.77700.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77700.log.gz 2024-04-25T11:24:05.710 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50842.log 2024-04-25T11:24:05.711 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46055.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46055.log.gz 2024-04-25T11:24:05.712 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50842.log: /var/log/ceph/ceph-client.admin.43125.log 2024-04-25T11:24:05.712 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50842.log.gz 2024-04-25T11:24:05.713 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69126.log 2024-04-25T11:24:05.715 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43125.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43125.log.gz 2024-04-25T11:24:05.715 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.69126.log: /var/log/ceph/ceph-client.admin.47683.log 2024-04-25T11:24:05.716 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69126.log.gz 2024-04-25T11:24:05.718 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47683.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47683.log.gz 2024-04-25T11:24:05.719 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56215.log 2024-04-25T11:24:05.720 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73654.log 2024-04-25T11:24:05.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56215.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56215.log.gz 2024-04-25T11:24:05.722 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60472.log 2024-04-25T11:24:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.73654.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73654.log.gz 2024-04-25T11:24:05.724 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45096.log 2024-04-25T11:24:05.724 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60472.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60472.log.gz 2024-04-25T11:24:05.725 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41571.log 2024-04-25T11:24:05.727 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45096.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45096.log.gz 2024-04-25T11:24:05.728 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.41571.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.72882.log 2024-04-25T11:24:05.728 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41571.log.gz 2024-04-25T11:24:05.729 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62865.log 2024-04-25T11:24:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72882.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72882.log.gz 2024-04-25T11:24:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.62865.log: -5 --verbose -- /var/log/ceph/ceph-client.2.42655.log 2024-04-25T11:24:05.732 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62865.log.gz 2024-04-25T11:24:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45558.log 2024-04-25T11:24:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.2.42655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.42655.log.gz 2024-04-25T11:24:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64766.log 2024-04-25T11:24:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45558.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45558.log.gz 2024-04-25T11:24:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71684.log 2024-04-25T11:24:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64766.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64766.log.gz 2024-04-25T11:24:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63320.log 2024-04-25T11:24:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71684.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71684.log.gz 2024-04-25T11:24:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63320.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63320.log.gz 2024-04-25T11:24:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75542.log 2024-04-25T11:24:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63446.log 2024-04-25T11:24:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.75542.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75542.log.gz 2024-04-25T11:24:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76530.log 2024-04-25T11:24:05.747 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63446.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63446.log.gz 2024-04-25T11:24:05.748 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.76530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44262.log 2024-04-25T11:24:05.748 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76530.log.gz 2024-04-25T11:24:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64472.log 2024-04-25T11:24:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44262.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44262.log.gz 2024-04-25T11:24:05.751 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.64472.log: -- /var/log/ceph/ceph-client.admin.46368.log 2024-04-25T11:24:05.752 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64472.log.gz 2024-04-25T11:24:05.753 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43461.log 2024-04-25T11:24:05.755 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46368.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46368.log.gz 2024-04-25T11:24:05.755 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71936.log/var/log/ceph/ceph-client.admin.43461.log: 2024-04-25T11:24:05.756 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43461.log.gz 2024-04-25T11:24:05.757 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64042.log 2024-04-25T11:24:05.758 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71936.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71936.log.gz 2024-04-25T11:24:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37917.log 2024-04-25T11:24:05.759 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64042.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64042.log.gz 2024-04-25T11:24:05.760 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44094.log 2024-04-25T11:24:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37917.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37917.log.gz 2024-04-25T11:24:05.762 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44094.log: -- /var/log/ceph/ceph-client.admin.53472.log 2024-04-25T11:24:05.763 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44094.log.gz 2024-04-25T11:24:05.764 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60826.log 2024-04-25T11:24:05.765 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53472.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53472.log.gz 2024-04-25T11:24:05.766 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.60826.log: --verbose -- /var/log/ceph/ceph-client.admin.44136.log 2024-04-25T11:24:05.767 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60826.log.gz 2024-04-25T11:24:05.767 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65079.log 2024-04-25T11:24:05.769 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44136.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44136.log.gz 2024-04-25T11:24:05.770 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.65079.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60640.log 2024-04-25T11:24:05.770 INFO:teuthology.orchestra.run.smithi007.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65079.log.gz 2024-04-25T11:24:05.771 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51566.log 2024-04-25T11:24:05.773 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60640.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60640.log.gz 2024-04-25T11:24:05.774 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58361.log 2024-04-25T11:24:05.774 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51566.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51566.log.gz 2024-04-25T11:24:05.775 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57538.log 2024-04-25T11:24:05.776 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58361.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58361.log.gz 2024-04-25T11:24:05.778 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.57538.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.74664.log 2024-04-25T11:24:05.778 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57538.log.gz 2024-04-25T11:24:05.779 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51939.log 2024-04-25T11:24:05.780 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.74664.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74664.log.gz 2024-04-25T11:24:05.781 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51939.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51939.log.gz 2024-04-25T11:24:05.948 INFO:teuthology.orchestra.run.smithi087.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T11:24:06.112 INFO:teuthology.orchestra.run.smithi087.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T11:24:06.218 INFO:teuthology.orchestra.run.smithi087.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T11:24:06.296 INFO:teuthology.orchestra.run.smithi007.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T11:24:06.307 INFO:teuthology.orchestra.run.smithi087.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T11:24:06.357 INFO:teuthology.orchestra.run.smithi087.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T11:24:06.360 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-04-25T11:24:06.360 INFO:teuthology.orchestra.run.smithi087.stderr:real 0m1.944s 2024-04-25T11:24:06.360 INFO:teuthology.orchestra.run.smithi087.stderr:user 0m9.087s 2024-04-25T11:24:06.360 INFO:teuthology.orchestra.run.smithi087.stderr:sys 0m0.832s 2024-04-25T11:24:06.600 INFO:teuthology.orchestra.run.smithi007.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T11:24:06.643 INFO:teuthology.orchestra.run.smithi007.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T11:24:06.685 INFO:teuthology.orchestra.run.smithi007.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T11:24:06.802 INFO:teuthology.orchestra.run.smithi007.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T11:24:07.035 INFO:teuthology.orchestra.run.smithi007.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T11:24:07.039 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-04-25T11:24:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m2.627s 2024-04-25T11:24:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m11.042s 2024-04-25T11:24:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m3.058s 2024-04-25T11:24:07.040 INFO:tasks.ceph:Archiving logs... 2024-04-25T11:24:07.040 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113/remote/smithi007/log 2024-04-25T11:24:07.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:24:09.075 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113/remote/smithi087/log 2024-04-25T11:24:09.076 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T11:24:10.325 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T11:24:10.337 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T11:24:10.337 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:24:10.346 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T11:24:10.487 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T11:24:10.497 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T11:24:10.497 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:24:10.505 DEBUG:teuthology.orchestra.run.smithi087:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T11:24:10.555 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:24:10.561 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-04-25T11:24:10.566 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:24:10.566 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-04-25T11:24:10.566 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv01.front.sepia.ceph.com 3 7 377 5 +903us[ +929us] +/- 56ms 2024-04-25T11:24:10.566 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 4 -455us[ -455us] +/- 65ms 2024-04-25T11:24:10.566 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -1233us[-1207us] +/- 90ms 2024-04-25T11:24:10.566 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:24:10.570 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T11:24:10.570 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-04-25T11:24:10.570 INFO:teuthology.orchestra.run.smithi087.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +999us[+1028us] +/- 56ms 2024-04-25T11:24:10.570 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -334us[ -334us] +/- 65ms 2024-04-25T11:24:10.570 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -1140us[-1140us] +/- 90ms 2024-04-25T11:24:10.570 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T11:24:10.572 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T11:24:10.584 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T11:24:10.584 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T11:24:10.598 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:24:10.836 DEBUG:teuthology.orchestra.run.smithi087:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T11:24:11.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:24:11.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:24:11.135 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T11:24:11.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:24:11.232 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T11:24:11.243 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T11:24:11.252 INFO:teuthology.task.internal:Duration was 1429.667462 seconds 2024-04-25T11:24:11.252 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T11:24:11.261 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T11:24:11.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:24:11.270 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T11:24:11.401 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:24:11.404 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T11:24:11.833 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T11:24:11.833 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-04-25T11:24:11.833 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:24:11.898 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T11:24:11.899 DEBUG:teuthology.orchestra.run.smithi087:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T11:24:11.970 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T11:24:11.970 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:24:11.978 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T11:24:12.065 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T11:24:12.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:24:12.071 DEBUG:teuthology.orchestra.run.smithi087:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T11:24:12.277 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T11:24:12.287 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T11:24:12.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:24:12.379 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T11:24:12.464 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T11:24:12.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:24:12.484 DEBUG:teuthology.orchestra.run.smithi087:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:24:12.583 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-04-25T11:24:12.584 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = core 2024-04-25T11:24:12.629 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:24:12.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:24:12.677 DEBUG:teuthology.orchestra.run.smithi087:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T11:24:12.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T11:24:12.732 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T11:24:12.742 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T11:24:12.743 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113/remote/smithi007 2024-04-25T11:24:12.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:24:12.846 DEBUG:teuthology.misc:Transferring archived files from smithi087:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673113/remote/smithi087 2024-04-25T11:24:12.847 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T11:24:12.945 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T11:24:12.945 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:24:12.953 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T11:24:13.006 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T11:24:13.017 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T11:24:13.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T11:24:13.028 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T11:24:13.028 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:24:13.036 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T11:24:13.088 INFO:teuthology.orchestra.run.smithi007.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:24 /home/ubuntu/cephtest 2024-04-25T11:24:13.092 INFO:teuthology.orchestra.run.smithi087.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 11:24 /home/ubuntu/cephtest 2024-04-25T11:24:13.095 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T11:24:13.106 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T11:24:13.132 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/client-limits} duration: 1429.667462348938 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T11:24:13.132 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T11:24:13.230 INFO:teuthology.run:pass